[
  {
    "path": ".dockerignore",
    "content": "// Ignore everything\n*\n\n// Allow what is needed\n!crates\n!tests\n!resources\n\n!Cargo.lock\n!Cargo.toml\n"
  },
  {
    "path": ".editorconfig",
    "content": "# https://EditorConfig.org\nroot = true\n\n[*]\ncharset = utf-8\nindent_size = 4\nindent_style = space\nend_of_line = lf\ninsert_final_newline = true\ntrim_trailing_whitespace = true\nmax_line_length = 100\n"
  },
  {
    "path": ".github/DISCUSSION_TEMPLATE/issue-triage.yml",
    "content": "labels: [\"triage\"]\nbody:\n  - type: markdown\n    attributes:\n      value: |\n        > [!IMPORTANT]\n        > Please review the [documentation](https://stalw.art/docs/), check the [FAQ](https://stalw.art/docs/faq), and search existing [Discussions](https://github.com/stalwartlabs/stalwart/discussions) and [Issues](https://github.com/stalwartlabs/stalwart/issues?q=sort%3Areactions-desc) before opening a new Discussion.\n        >\n        > Most reported issues turn out to be configuration problems rather than actual bugs. Starting here helps us triage effectively—if this is confirmed as a bug, we'll create an Issue for tracking.\n  - type: markdown\n    attributes:\n      value: \"# Issue Details\"\n  - type: textarea\n    attributes:\n      label: Issue Description\n      description: |\n        Provide a detailed description of the issue. Include relevant context such as your configuration, environment, and any recent changes that might have led to the issue.\n      placeholder: |\n        When trying to send an email via SMTP, the connection is accepted but the message is rejected with error 550.\n    validations:\n      required: true\n  - type: textarea\n    attributes:\n      label: Expected Behavior\n      description: |\n        Describe how you expect Stalwart to behave in this situation. Include any relevant documentation links.\n      placeholder: |\n        The email should be accepted and delivered to the recipient's mailbox.\n    validations:\n      required: false\n  - type: textarea\n    attributes:\n      label: Actual Behavior\n      description: |\n        Describe how Stalwart actually behaves in this situation. If it is not immediately obvious how the actual behavior differs from the expected behavior described above, please mention the deviation specifically.\n      placeholder: |\n        The email is rejected with error: \"550 5.7.1 Relay access denied\"\n    validations:\n      required: false\n  - type: textarea\n    attributes:\n      label: Reproduction Steps\n      description: |\n        Provide a detailed set of step-by-step instructions for reproducing this issue.\n      placeholder: |\n        1. Configure Stalwart with the attached configuration.\n        2. Attempt to send an email from user@example.com to external@domain.com.\n        3. Observe the 550 error in the SMTP session.\n    validations:\n      required: false\n  - type: textarea\n    attributes:\n      label: Relevant Log Output\n      description: |\n        Please copy and paste any relevant log output. Set logging level to `trace` if you can't find any relevant errors in the log.\n      render: shell\n  - type: dropdown\n    attributes:\n      label: Stalwart Version\n      description: What version of Stalwart are you running?\n      options:\n        - v0.15.x\n        - v0.14.x\n        - v0.13.x\n        - v0.12.x or lower\n    validations:\n      required: true\n  - type: dropdown\n    attributes:\n      label: Installation Method\n      description: How did you install Stalwart?\n      options:\n        - Docker\n        - Binary (Linux)\n        - Binary (macOS)\n        - Binary (FreeBSD)\n        - Binary (Windows)\n        - NixOS\n        - Built from source\n    validations:\n      required: true\n  - type: dropdown\n    attributes:\n      label: Database Backend\n      description: What database are you using for the data store?\n      options:\n        - RocksDB\n        - FoundationDB\n        - PostgreSQL\n        - MySQL\n        - SQLite\n    validations:\n      required: true\n  - type: dropdown\n    attributes:\n      label: Blob Storage\n      description: What blob storage are you using?\n      options:\n        - RocksDB\n        - FoundationDB\n        - PostgreSQL\n        - MySQL\n        - SQLite\n        - Filesystem\n        - S3-compatible\n        - Azure\n    validations:\n      required: true\n  - type: dropdown\n    attributes:\n      label: Search Engine\n      description: What search engine are you using?\n      options:\n        - Internal\n        - Meilisearch\n        - Elasticsearch\n        - PostgreSQL\n        - MySQL\n    validations:\n      required: true\n  - type: dropdown\n    attributes:\n      label: Directory Backend\n      description: Where is your directory/user database located?\n      options:\n        - Internal\n        - SQL\n        - LDAP\n        - OIDC\n    validations:\n      required: true\n  - type: textarea\n    attributes:\n      label: Additional Context\n      description: |\n        Add any other context about the problem here. This could include:\n        - Client software and versions (Thunderbird, Apple Mail, K-9, etc.)\n        - Proxy or reverse proxy configuration (nginx, Traefik, etc.)\n        - Network setup (NAT, firewall rules, etc.)\n    validations:\n      required: false\n  - type: markdown\n    attributes:\n      value: |\n        # Acknowledgements\n        > [!TIP]\n        > Use these links to review existing [Discussions](https://github.com/stalwartlabs/stalwart/discussions) and [Issues](https://github.com/stalwartlabs/stalwart/issues?q=sort%3Areactions-desc).\n  - type: checkboxes\n    attributes:\n      label: \"I acknowledge that:\"\n      options:\n        - label: I have reviewed the documentation and FAQ and confirm that my issue is NOT addressed there.\n          required: true\n        - label: I have searched the Stalwart repository (both open and closed Discussions and Issues) and confirm this is not a duplicate.\n          required: true\n        - label: I have set the logging level to `trace` and included relevant log output if applicable.\n          required: false\n        - label: I agree to follow the project's [Code of Conduct](https://github.com/stalwartlabs/.github/blob/main/CODE_OF_CONDUCT.md).\n          required: true\n"
  },
  {
    "path": ".github/ISSUE_TEMPLATE/config.yml",
    "content": "blank_issues_enabled: false\ncontact_links:\n  - name: Report an Issue\n    url: https://github.com/stalwartlabs/stalwart/discussions/new?category=issue-triage\n    about: Report a potential bug. Confirmed bugs will be converted to Issues.\n  - name: Questions & Support\n    url: https://github.com/stalwartlabs/stalwart/discussions/new?category=q-a\n    about: Get help with configuration, troubleshooting, or general questions.\n  - name: Feature Requests\n    url: https://github.com/stalwartlabs/stalwart/discussions/new?category=feature-requests-and-ideas\n    about: Suggest new features or improvements.\n  - name: Join Stalwart's Reddit\n    url: https://www.reddit.com/r/stalwartlabs\n    about: Join our subreddit for help, discussions and release announcements.\n  - name: Join Stalwart's Discord\n    url: https://discord.com/servers/stalwart-923615863037390889\n    about: Join our Discord server for help, discussions and release announcements.\n"
  },
  {
    "path": ".github/ISSUE_TEMPLATE/confirmed_issue.yml",
    "content": "name: Confirmed Bug Report\ndescription: Only for issues that have been discussed and confirmed as bugs in GitHub Discussions.\nlabels: [\"bug\"]\ntitle: \"🪲: \"\nbody:\n  - type: markdown\n    attributes:\n      value: |\n        > [!IMPORTANT]\n        > This template is **only** for issues that have already been discussed and confirmed as bugs in the [Discussions](https://github.com/stalwartlabs/stalwart/discussions) section.\n        >\n        > If you haven't had your issue confirmed yet, please [start a Discussion](https://github.com/stalwartlabs/stalwart/discussions/new?category=issue-triage) first.\n  - type: input\n    attributes:\n      label: Discussion Link\n      description: Provide the link to the Discussion where this issue was confirmed as a bug.\n      placeholder: https://github.com/stalwartlabs/stalwart/discussions/1234\n    validations:\n      required: true\n  - type: textarea\n    attributes:\n      label: Bug Summary\n      description: Brief summary of the confirmed bug.\n    validations:\n      required: true\n  - type: textarea\n    attributes:\n      label: Additional Information\n      description: |\n        Include any additional information not covered in the original Discussion, such as new findings, workarounds discovered, or updated reproduction steps.\n    validations:\n      required: false\n  - type: checkboxes\n    attributes:\n      label: Confirmation\n      options:\n        - label: This issue was discussed and confirmed as a bug in the linked Discussion.\n          required: true\n        - label: I have included the link to the Discussion above.\n          required: true\n"
  },
  {
    "path": ".github/dependabot.yml",
    "content": "# To get started with Dependabot version updates, you'll need to specify which\n# package ecosystems to update and where the package manifests are located.\n# Please see the documentation for all configuration options:\n# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file\n\nversion: 2\nupdates:\n  - package-ecosystem: \"cargo\" # See documentation for possible values\n    directory: \"/\" # Location of package manifests\n    schedule:\n      interval: \"weekly\"\n\n  # Enable version updates for GitHub Actions\n  - package-ecosystem: \"github-actions\"\n    # Workflow files stored in the default location of `.github/workflows`\n    # You don't need to specify `/.github/workflows` for `directory`. You can use `directory: \"/\"`.\n    directory: \"/\"\n    schedule:\n      interval: \"weekly\"\n"
  },
  {
    "path": ".github/workflows/ci.yml",
    "content": "name: \"CI\"\n\non:\n  workflow_dispatch:\n    inputs:\n      Docker:\n        required: false\n        default: false\n        type: boolean\n      Release:\n        required: false\n        default: false\n        type: boolean\n  push:\n    tags: [\"v*.*.*\"]\n\nenv:\n  SCCACHE_GHA_ENABLED: true\n  RUSTC_WRAPPER: sccache\n  CARGO_TERM_COLOR: always\n\nconcurrency:\n  group: ${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: true\n\njobs:\n  multiarch:\n    strategy:\n      fail-fast: false\n      matrix:\n        include:\n          - variant: gnu\n          - variant: musl\n    name: Merge image / ${{matrix.variant}}\n    runs-on: ubuntu-latest\n    permissions:\n      id-token: write\n      contents: read\n      attestations: write\n      packages: write\n    needs: [linux]\n    if: github.event_name == 'push' || inputs.Docker\n    steps:\n      - name: Install Cosign\n        uses: sigstore/cosign-installer@v3\n      - name: Log In to GitHub Container Registry\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{github.repository_owner}}\n          password: ${{github.token}}\n\n      - name: Log In to DockerHub\n        uses: docker/login-action@v3\n        with:\n          username: ${{secrets.DOCKERHUB_USERNAME}}\n          password: ${{secrets.DOCKERHUB_TOKEN}}\n\n      - name: Download ${{matrix.variant}} meta bake definition\n        uses: actions/download-artifact@v7\n        with:\n          name: bake-meta-${{matrix.variant}}\n          path: ${{ runner.temp }}/${{matrix.variant}}\n\n      - name: Download ${{matrix.variant}} digests\n        uses: actions/download-artifact@v7\n        with:\n          path: ${{ runner.temp }}/${{matrix.variant}}/digests\n          pattern: digests-${{matrix.variant}}-*\n          merge-multiple: true\n\n      - name: Create ${{matrix.variant}} manifest list and push\n        working-directory: ${{ runner.temp }}/${{matrix.variant}}/digests\n        run: |\n          docker buildx imagetools create $(jq -cr '.target.\"docker-metadata-action\".tags | map(select(startswith(\"ghcr.io/${{github.repository}}\")) | \"-t \" + .) | join(\" \")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) \\\n            $(printf 'ghcr.io/${{github.repository}}@sha256:%s ' *)\n          docker buildx imagetools create $(jq -cr '.target.\"docker-metadata-action\".tags | map(select(startswith(\"index.docker.io/${{github.repository}}\")) | \"-t \" + .) | join(\" \")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) \\\n            $(printf 'index.docker.io/${{github.repository}}@sha256:%s ' *)\n\n      - name: Inspect ${{matrix.variant}} image\n        id: manifest-digest\n        run: |\n          docker buildx imagetools inspect --format '{{json .Manifest}}' ghcr.io/${{github.repository}}:$(jq -r '.target.\"docker-metadata-action\".args.DOCKER_META_VERSION' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) | jq -r '.digest' > GHCR_DIGEST_SHA\n          echo \"GHCR_DIGEST_SHA=$(cat GHCR_DIGEST_SHA)\" | tee -a \"${GITHUB_ENV}\"\n          docker buildx imagetools inspect --format '{{json .Manifest}}' index.docker.io/${{github.repository}}:$(jq -r '.target.\"docker-metadata-action\".args.DOCKER_META_VERSION' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json) | jq -r '.digest' > DOCKERHUB_DIGEST_SHA\n          echo \"DOCKERHUB_DIGEST_SHA=$(cat DOCKERHUB_DIGEST_SHA)\" | tee -a \"${GITHUB_ENV}\"\n          cosign sign --yes $(jq --arg GHCR_DIGEST_SHA \"$(cat GHCR_DIGEST_SHA)\" -cr '.target.\"docker-metadata-action\".tags | map(select(startswith(\"ghcr.io/${{github.repository}}\")) | . + \"@\" + $GHCR_DIGEST_SHA) | join(\" \")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json)\n          cosign sign --yes $(jq --arg DOCKERHUB_DIGEST_SHA \"$(cat DOCKERHUB_DIGEST_SHA)\" -cr '.target.\"docker-metadata-action\".tags | map(select(startswith(\"index.docker.io/${{github.repository}}\")) | . + \"@\" + $DOCKERHUB_DIGEST_SHA) | join(\" \")' ${{ runner.temp }}/${{matrix.variant}}/bake-meta.json)\n\n      - name: Attest GHCR\n        uses: actions/attest-build-provenance@v3\n        with:\n          subject-name: ghcr.io/${{github.repository}}\n          subject-digest: ${{ env.GHCR_DIGEST_SHA }}\n          push-to-registry: true\n\n      - name: Attest Dockerhub\n        uses: actions/attest-build-provenance@v3\n        with:\n          subject-name: index.docker.io/${{github.repository}}\n          subject-digest: ${{ env.DOCKERHUB_DIGEST_SHA }}\n          push-to-registry: true\n\n  linux:\n    permissions:\n      id-token: write\n      contents: write\n      attestations: write\n      packages: write\n    strategy:\n      fail-fast: false\n      matrix:\n        include:\n          - target: x86_64-unknown-linux-gnu\n            platform: linux/amd64\n            suffix: \"\"\n            build_env: \"\"\n          - target: x86_64-unknown-linux-musl\n            platform: linux/amd64\n            suffix: \"-alpine\"\n            build_env: \"\"\n          - target: aarch64-unknown-linux-gnu\n            platform: linux/arm64\n            suffix: \"\"\n            build_env: \"JEMALLOC_SYS_WITH_LG_PAGE=16 \"\n          - target: aarch64-unknown-linux-musl\n            platform: linux/arm64\n            suffix: \"-alpine\"\n            build_env: \"JEMALLOC_SYS_WITH_LG_PAGE=16 \"\n          - target: armv7-unknown-linux-gnueabihf\n            platform: linux/arm/v7\n            suffix: \"\"\n            build_env: \"JEMALLOC_SYS_WITH_LG_PAGE=16 \"\n          - target: armv7-unknown-linux-musleabihf\n            platform: linux/arm/v7\n            suffix: \"-alpine\"\n            build_env: \"JEMALLOC_SYS_WITH_LG_PAGE=16 \"\n          - target: arm-unknown-linux-gnueabihf\n            platform: linux/arm/v6\n            suffix: \"\"\n            build_env: \"\"\n          - target: arm-unknown-linux-musleabihf\n            platform: linux/arm/v6\n            suffix: \"-alpine\"\n            build_env: \"\"\n    name: Build / ${{matrix.target}}\n    runs-on: ubuntu-latest\n    steps:\n      - name: Checkout\n        uses: actions/checkout@v6.0.1\n\n      - name: Set up QEMU\n        uses: docker/setup-qemu-action@v3\n        with:\n          platforms: \"arm64,arm\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n        with:\n          buildkitd-config-inline: |\n            [registry.\"docker.io\"]\n              mirrors = [\"https://mirror.gcr.io\"]\n          driver-opts: |\n            network=host\n\n      - name: Log In to GitHub Container Registry\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{github.repository_owner}}\n          password: ${{github.token}}\n\n      - name: Log In to DockerHub\n        uses: docker/login-action@v3\n        with:\n          username: ${{secrets.DOCKERHUB_USERNAME}}\n          password: ${{secrets.DOCKERHUB_TOKEN}}\n\n      - name: Calculate shasum of external deps\n        id: cal-dep-shasum\n        run: |\n          echo \"checksum=$(yq -p toml -oy '.package[] | select((.source | contains(\"\")) or (.checksum | contains(\"\")))' Cargo.lock | sha256sum | awk '{print $1}')\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Cache apt\n        uses: actions/cache@v5\n        id: apt-cache\n        with:\n          path: |\n            var-cache-apt\n            var-lib-apt\n          key: apt-cache-${{ hashFiles('Dockerfile.build') }}\n\n      - name: Cache Cargo\n        uses: actions/cache@v5\n        id: cargo-cache\n        with:\n          path: |\n            usr-local-cargo-registry\n            usr-local-cargo-git\n          key: cargo-cache-${{ steps.cal-dep-shasum.outputs.checksum }}\n\n      - name: Inject cache into docker\n        uses: reproducible-containers/buildkit-cache-dance@v3.3.0\n        with:\n          cache-map: |\n            {\n              \"var-cache-apt\": \"/var/cache/apt\",\n              \"var-lib-apt\": \"/var/lib/apt\",\n              \"usr-local-cargo-registry\": \"/usr/local/cargo/registry\",\n              \"usr-local-cargo-git\": \"/usr/local/cargo/git\"\n            }\n          skip-extraction: ${{ steps.cargo-cache.outputs.cache-hit }} && ${{ steps.apt-cache.outputs.cache-hit }}\n\n      - name: Extract Metadata for Docker\n        uses: docker/metadata-action@v5\n        id: meta\n        with:\n          images: |\n            index.docker.io/${{github.repository}}\n            ghcr.io/${{github.repository}}\n          flavor: |\n            suffix=${{matrix.suffix}},onlatest=true\n          tags: |\n            type=ref,event=tag\n            type=ref,event=branch,prefix=branch-\n            type=edge,branch=main\n            type=semver,pattern=v{{major}}.{{minor}}\n\n      - name: Build Artifact\n        id: bake\n        uses: docker/bake-action@v6\n        env:\n          DOCKER_BUILD_RECORD_UPLOAD: false\n          TARGET: ${{matrix.target}}\n          GHCR_REPO: ghcr.io/${{github.repository}}\n          BUILD_ENV: ${{matrix.build_env}}\n          DOCKER_PLATFORM: ${{matrix.platform}}\n          SUFFIX: ${{matrix.suffix}}\n        with:\n          source: .\n          set: |\n            *.tags=\n            image.output=type=image,\"name=ghcr.io/${{github.repository}},index.docker.io/${{github.repository}}\",push-by-digest=true,name-canonical=true,push=true,compression=zstd,compression-level=9,force-compression=true,oci-mediatypes=true\n          files: |\n            docker-bake.hcl\n            ${{ steps.meta.outputs.bake-file }}\n          targets: ${{(github.event_name == 'push' || inputs.Docker) && 'build,image' || 'build'}}\n\n      - name: Upload Artifacts\n        uses: actions/upload-artifact@v6\n        with:\n          name: artifact-${{matrix.target}}\n          path: |\n            artifact\n            !artifact/*.json\n\n      - name: Export digest & Rename meta bake definition file\n        if: github.event_name == 'push' || inputs.Docker\n        run: |\n          mv \"${{ steps.meta.outputs.bake-file }}\" \"${{ runner.temp }}/bake-meta.json\"\n          mkdir -p ${{ runner.temp }}/digests\n          digest=\"${{ fromJSON(steps.bake.outputs.metadata).image['containerimage.digest'] }}\"\n          touch \"${{ runner.temp }}/digests/${digest#sha256:}\"\n\n      - name: Upload digest\n        if: github.event_name == 'push' || inputs.Docker\n        uses: actions/upload-artifact@v6\n        with:\n          name: digests-${{matrix.suffix == '' && 'gnu' || 'musl'}}-${{ matrix.target }}\n          path: ${{ runner.temp }}/digests/*\n          if-no-files-found: error\n          retention-days: 1\n\n      - name: Upload GNU meta bake definition\n        uses: actions/upload-artifact@v6\n        if: (github.event_name == 'push' || inputs.Docker) && endsWith(matrix.target,'gnu') && startsWith(matrix.target,'x86')\n        with:\n          name: bake-meta-gnu\n          path: ${{ runner.temp }}/bake-meta.json\n          if-no-files-found: error\n          retention-days: 1\n\n      - name: Upload musl meta bake definition\n        uses: actions/upload-artifact@v6\n        if: (github.event_name == 'push' || inputs.Docker) && endsWith(matrix.target,'musl') && startsWith(matrix.target,'x86')\n        with:\n          name: bake-meta-musl\n          path: ${{ runner.temp }}/bake-meta.json\n          if-no-files-found: error\n          retention-days: 1\n\n  windows:\n    name: Build / ${{matrix.target}}\n    runs-on: windows-latest\n    strategy:\n      fail-fast: false\n      matrix:\n        include:\n          #   - target: aarch64-pc-windows-msvc\n          - target: x86_64-pc-windows-msvc\n\n    steps:\n      - name: Checkout\n        uses: actions/checkout@v6.0.1\n\n      - name: Run sccache-cache\n        uses: mozilla-actions/sccache-action@v0.0.9\n        with:\n          disable_annotations: true\n\n      - name: Build\n        run: |\n          rustup target add ${{matrix.target}}\n          cargo build --release --target ${{matrix.target}} -p stalwart --no-default-features --features \"sqlite postgres mysql rocks s3 redis azure nats enterprise\"\n          cargo build --release --target ${{matrix.target}} -p stalwart-cli\n          mkdir -p artifacts\n          mv ./target/${{matrix.target}}/release/stalwart.exe ./artifacts/stalwart.exe\n          mv ./target/${{matrix.target}}/release/stalwart-cli.exe ./artifacts/stalwart-cli.exe\n\n      - name: Upload Artifacts\n        uses: actions/upload-artifact@v6\n        with:\n          name: artifact-${{matrix.target}}\n          path: artifacts\n\n  macos:\n    name: Build / ${{matrix.target}}\n    runs-on: macos-latest\n    strategy:\n      fail-fast: false\n      matrix:\n        include:\n          - target: aarch64-apple-darwin\n          - target: x86_64-apple-darwin\n    steps:\n      - name: Checkout\n        uses: actions/checkout@v6.0.1\n\n      - name: Run sccache-cache\n        uses: mozilla-actions/sccache-action@v0.0.9\n        with:\n          disable_annotations: true\n\n      #- name: Build FoundationDB Edition\n      #  env:\n      #    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n      #  run: |\n      #    rustup target add ${{matrix.target}}\n      #    # Get latest FoundationDB installer\n      #    curl --retry 5 -Lso foundationdb.pkg \"$(gh api -X GET /repos/apple/foundationdb/releases --jq '.[] | select(.prerelease == false) | .assets[] | select(.name | test(\"${{startsWith(matrix.target, 'x86') && 'x86_64' || 'arm64'}}\" + \".pkg$\")) | .browser_download_url' | head -n1)\"\n      #    echo \"=== Package contents ===\"\n      #    pkgutil --payload-files foundationdb.pkg || true\n      #    sudo installer -allowUntrusted -verbose -dumplog -pkg foundationdb.pkg -target /\n      #    cargo build --release --target ${{matrix.target}} -p stalwart --no-default-features --features \"foundationdb s3 redis nats enterprise\"\n      #    mkdir -p artifacts\n      #    mv ./target/${{matrix.target}}/release/stalwart ./artifacts/stalwart-foundationdb\n\n      - name: Build\n        run: |\n          rustup target add ${{matrix.target}}\n          cargo build --release --target ${{matrix.target}} -p stalwart --no-default-features --features \"sqlite postgres mysql rocks s3 redis azure nats enterprise\"\n          cargo build --release --target ${{matrix.target}} -p stalwart-cli\n          mkdir -p artifacts\n          mv ./target/${{matrix.target}}/release/stalwart ./artifacts/stalwart\n          mv ./target/${{matrix.target}}/release/stalwart-cli ./artifacts/stalwart-cli\n\n      - name: Upload Artifacts\n        uses: actions/upload-artifact@v6\n        with:\n          name: artifact-${{matrix.target}}\n          path: artifacts\n\n  release:\n    name: Release\n    permissions:\n      id-token: write\n      contents: write\n      attestations: write\n    if: github.event_name == 'push' || inputs.Release\n    needs: [linux, windows, macos]\n    runs-on: ubuntu-latest\n    steps:\n      - name: Download Artifacts\n        uses: actions/download-artifact@v7\n        with:\n          path: archive\n          pattern: artifact-*\n\n      - name: Compress\n        run: |\n          set -eux\n          BASE_DIR=\"$(pwd)/archive\"\n          compress_files() {\n              local dir=\"$1\"\n              local archive_dir_name=\"${dir#artifact-}\"\n              cd \"$dir\"\n              # Process each file in the directory\n              for file in `ls`; do\n                  filename=\"${file%.*}\"\n                  extension=\"${file##*.}\"\n                  if [ \"$extension\" = \"exe\" ]; then\n                      7z a -tzip \"${filename}-${archive_dir_name}.zip\" \"$file\" > /dev/null\n                  else\n                      tar -czf \"${filename}-${archive_dir_name}.tar.gz\" \"$file\"\n                  fi\n              done\n              cd $BASE_DIR\n          }\n          cd $BASE_DIR\n          for arch_dir in `ls`; do\n              dir_name=$(basename \"$arch_dir\")\n              compress_files \"$dir_name\"\n          done\n\n      - name: Attest binary\n        id: attest\n        uses: actions/attest-build-provenance@v3\n        with:\n          subject-path: |\n            archive/**/*.tar.gz\n            archive/**/*.zip\n\n      - name: Use cosign to sign existing artifacts\n        uses: sigstore/gh-action-sigstore-python@v3.2.0\n        with:\n          inputs: |\n            archive/**/*.tar.gz\n            archive/**/*.zip\n\n      - name: Release\n        uses: softprops/action-gh-release@v2\n        with:\n          files: |\n            archive/**/*.tar.gz\n            archive/**/*.zip\n            archive/**/*.sigstore.json\n          prerelease: ${{!startsWith(github.ref, 'refs/tags/') || null}}\n          tag_name: ${{!startsWith(github.ref, 'refs/tags/') && 'nightly' || null}}\n          # TODO add instructions about using cosign to verify binary artifact\n          append_body: true\n          body: |\n            <hr />\n\n            ### Check binary attestation at [here](${{ steps.attest.outputs.attestation-url }})\n"
  },
  {
    "path": ".github/workflows/scorecard.yml",
    "content": "# This workflow uses actions that are not certified by GitHub. They are provided\n# by a third-party and are governed by separate terms of service, privacy\n# policy, and support documentation.\n\nname: Scorecard supply-chain security\non:\n  # For Branch-Protection check. Only the default branch is supported. See\n  # https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection\n  branch_protection_rule:\n  # To guarantee Maintained check is occasionally updated. See\n  # https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained\n  schedule:\n    - cron: '31 6 * * 0'\n  push:\n    branches: [ \"main\" ]\n\n# Declare default permissions as read only.\npermissions: read-all\n\njobs:\n  analysis:\n    name: Scorecard analysis\n    runs-on: ubuntu-latest\n    # `publish_results: true` only works when run from the default branch. conditional can be removed if disabled.\n    if: github.event.repository.default_branch == github.ref_name || github.event_name == 'pull_request'\n    permissions:\n      # Needed to upload the results to code-scanning dashboard.\n      security-events: write\n      # Needed to publish results and get a badge (see publish_results below).\n      id-token: write\n      # Uncomment the permissions below if installing in a private repository.\n      # contents: read\n      # actions: read\n\n    steps:\n      - name: \"Checkout code\"\n        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v4.2.2\n        with:\n          persist-credentials: false\n\n      - name: \"Run analysis\"\n        uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a # v2.4.3\n        with:\n          results_file: results.sarif\n          results_format: sarif\n          # (Optional) \"write\" PAT token. Uncomment the `repo_token` line below if:\n          # - you want to enable the Branch-Protection check on a *public* repository, or\n          # - you are installing Scorecard on a *private* repository\n          # To create the PAT, follow the steps in https://github.com/ossf/scorecard-action?tab=readme-ov-file#authentication-with-fine-grained-pat-optional.\n          # repo_token: ${{ secrets.SCORECARD_TOKEN }}\n\n          # Public repositories:\n          #   - Publish results to OpenSSF REST API for easy access by consumers\n          #   - Allows the repository to include the Scorecard badge.\n          #   - See https://github.com/ossf/scorecard-action#publishing-results.\n          # For private repositories:\n          #   - `publish_results` will always be set to `false`, regardless\n          #     of the value entered here.\n          publish_results: true\n\n          # (Optional) Uncomment file_mode if you have a .gitattributes with files marked export-ignore\n          # file_mode: git\n\n      # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF\n      # format to the repository Actions tab.\n      - name: \"Upload artifact\"\n        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0\n        with:\n          name: SARIF file\n          path: results.sarif\n          retention-days: 5\n\n      # Upload the results to GitHub's code scanning dashboard (optional).\n      # Commenting out will disable upload of results to your repo's Code Scanning dashboard\n      - name: \"Upload to code-scanning\"\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: results.sarif\n"
  },
  {
    "path": ".github/workflows/test.yml",
    "content": "name: Test\n\non:\n  workflow_dispatch:\n\njobs:\n  style:\n    name: Check Style\n    runs-on: ubuntu-latest\n    steps:\n      - name: Checkout\n        uses: actions/checkout@v6.0.1\n\n      - name: Check Style\n        run: cargo fmt --all --check\n\n  test:\n    name: Test\n    needs: style\n    runs-on: ubuntu-latest\n\n    steps:\n      - name: Checkout\n        uses: actions/checkout@v6.0.1\n\n      - name: Install dependencies\n        run: |\n          sudo apt-get update -y\n          curl -LO https://github.com/glauth/glauth/releases/download/v2.2.0/glauth-linux-arm64\n          chmod a+rx glauth-linux-arm64\n          nohup ./glauth-linux-arm64 -c tests/resources/ldap.cfg &\n          curl -Lo minio.deb https://dl.min.io/server/minio/release/linux-amd64/archive/minio_20230629051228.0.0_amd64.deb\n          sudo dpkg -i minio.deb\n          mkdir ~/minio\n          nohup minio server ~/minio --console-address :9090 &\n          curl -LO https://dl.min.io/client/mc/release/linux-amd64/mc\n          chmod a+rx mc\n          ./mc alias set myminio http://localhost:9000 minioadmin minioadmin\n          ./mc mb tmp\n\n      - name: Rust Cache\n        uses: Swatinem/rust-cache@v2\n\n      - name: JMAP Protocol Tests\n        run: cargo test -p jmap_proto -- --nocapture\n\n      - name: IMAP Protocol Tests\n        run: cargo test -p imap_proto -- --nocapture\n\n      - name: Full-text search Tests\n        run: cargo test -p store -- --nocapture\n\n      - name: Directory Tests\n        run: cargo test -p tests directory -- --nocapture\n\n      - name: SMTP Tests\n        run: cargo test -p tests smtp -- --nocapture\n\n      - name: IMAP Tests\n        run: cargo test -p tests imap -- --nocapture\n\n      - name: JMAP Tests\n        run: cargo test -p tests jmap -- --nocapture\n"
  },
  {
    "path": ".github/workflows/trivy.yml",
    "content": "# trivy ci workflow\nname: trivy\n\non:\n  workflow_dispatch:\n  push:\n    branches: [ \"main\" ]\n  pull_request:\n    # The branches below must be a subset of the branches above\n    branches: [ \"main\" ]\n  schedule:\n    - cron: '00 12 * * *'\n\npermissions:\n  contents: read\n\njobs:\n  build:\n    permissions:\n      contents: read # for actions/checkout to fetch code\n      security-events: write # for github/codeql-action/upload-sarif to upload SARIF results\n      actions: read # only required for a private repository by github/codeql-action/upload-sarif to get the Action run status\n    name: Check\n    runs-on: ubuntu-latest\n    steps:\n      - name: Checkout code\n        uses: actions/checkout@v6.0.1\n        \n      - name: Run Trivy vulnerability scanner\n        uses: aquasecurity/trivy-action@master\n        with:\n          scan-type: 'fs'\n          ignore-unfixed: true\n          format: 'sarif'\n          output: 'trivy-results.sarif'\n          severity: 'CRITICAL,HIGH'\n\n      - name: Upload Trivy scan results to GitHub Security tab\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: 'trivy-results.sarif'\n"
  },
  {
    "path": ".gitignore",
    "content": "/target\n.vscode\n.idea\n*.failed\n*_failed\nrun.sh\n_ignore\n.DS_Store\n"
  },
  {
    "path": "CHANGELOG.md",
    "content": "# Change Log\n\nAll notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/).\n\n## [0.15.5] - 2026-02-14\n\nIf you are upgrading from v0.14.x and below, this version includes **multiple breaking changes**. Please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_15.md) for more information on how to upgrade from previous versions.\nIf you are upgrading from v0.15.x, replace the binary and update the webadmin.\n\n## Added\n\n## Changed\n\n## Fixed\n- IMAP/JMAP: OOM when `mail-parser` returns cyclical MIME structures [CVE-2026-26312](https://github.com/stalwartlabs/stalwart/security/advisories/GHSA-jm95-876q-c9gw).\n- Tracing: Fix tracing indexing when using separate stores.\n- JMAP: Fix `upToId` computation in `*/queryChanges`.\n- JMAP: Include createdIds when the property is present.\n- JMAP: Respect query arguments in `Email/queryChanges`.\n- JMAP: Return the correct container/item change id when there are no changes.\n\n## [0.15.4] - 2026-01-19\n\nIf you are upgrading from v0.14.x and below, this version includes **multiple breaking changes**. Please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_15.md) for more information on how to upgrade from previous versions.\nIf you are upgrading from v0.15.x, replace the binary and update the webadmin.\n\n## Added\n- IMAP: Map `HEADER SUBJECT/FROM/TO` searches to `SUBJECT/FROM/TO` queries.\n- Sieve: Update spam status on user scripts.\n\n## Changed\n\n## Fixed\n- Search: Return all document ids when no filters are provided.\n- Search: Filters not applied when a single message is in the account.\n- IMAP: Return `ALREADYEXISTS` code when creating existing mailboxes.\n- IMAP: Do not return quota resources if no quota is set.\n- JMAP/changes: Update `newState` with last changeId if an invalid fromChangeId is provided.\n- JMAP/CalendarIdentity: Do not update invalid calendar identities.\n- AI API: Include request error details if available.\n\n## [0.15.3] - 2025-12-29\n\nIf you are upgrading from v0.14.x and below, this version includes **multiple breaking changes**. Please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_15.md) for more information on how to upgrade from previous versions.\nIf you are upgrading from v0.15.x, replace the binary and update the webadmin.\n\n## Added\n- Polish locale support (contributed by @mrxkp) (#2480)\n\n## Changed\n\n## Fixed\n- Meilisearch: Return correct error messages when failing to create indexes (#2574)\n- PostgreSQL search: Truncate emails to 650kb for full-text search indexing.\n- FoundationDB search: Batch large transactions (#2567).\n- Spam filter: Fix training sample size checks\n- IMAP: Fix UTF7 encoding with Emojis (contributed by @dojiong) (#2564).\n\n## [0.15.2] - 2025-12-22\n\nIf you are upgrading from v0.14.x and below, this version includes **multiple breaking changes**. Please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_15.md) for more information on how to upgrade from previous versions.\nIf you are upgrading from v0.15.x, replace the binary and update the webadmin.\n\n## Added\n- OAuth: Add device authorization endpoint (#2225).\n\n## Changed\n- Antispam: Only auto-learn spam from traps or multiple RBL hits.\n\n## Fixed\n- mySQL search: Use `MEDIUMTEXT` field type for email body and attachments (#2544).\n- PostgreSQL search: Truncate large text fields.\n- ElasticSearch: Implement pagination (#2551).\n- Antispam: Fix `NO_SPACE_IN_FROM` spam tag detection logic (#2372).\n- IMAP: Fix shared folder double nesting (test suite credits to @ochnygosch) (#2358).\n- JMAP: Use latest `Received` header in JMAP `Email/import` (credits to @apexskier) (#2374).\n- JMAP: Return unsorted search results when the index is not ready (#2544).\n- LDAP: Lowercase attribute comparison (credits to @pdf) (#2363).\n- CLI: Fix same-host JMAP redirection on non-standard ports (#2271).\n\n## [0.15.1] - 2025-12-17\n\nThis version includes **multiple breaking changes**. If you are upgrading from v0.14.x and below, please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_15.md) for more information on how to upgrade from previous versions.\n\n## Added\n\n## Changed\n\n## Fixed\n- PostgreSQL: Sanitize search index values (#2533)\n- Elasticsearch: Ignore `resource_already_exists_exception` errors when creating indexes (#2535)\n- Migrate 0.13.x data (#2534)\n\n## [0.15.0] - 2025-12-16\n\nThis version includes **multiple breaking changes**. Please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_15.md) for more information on how to upgrade from previous versions.\n\n## Added\n- Linear spam classifier using FTRL-Proximal and feature/cuckoo hashing.\n- Meilisearch store backend implementation (#1482).\n- PostgreSQL and mySQL native full-text search support.\n- Multiple performance improvements and database access optimizations.\n- Encryption-at-rest: Spam training privacy setting. \n- Enterprise: Undelete e-mail feature now includes From/Subject/Received information.\n- IMAP: Implemented new keywords and mailbox attributes described in [draft-ietf-mailmaint-messageflag-mailboxattribute-13](https://datatracker.ietf.org/doc/html/draft-ietf-mailmaint-messageflag-mailboxattribute-13)\n\n## Changed\n- IMAP: Always return special use flags in responses.\n\n## Fixed\n- JMAP: `FileNode/set` fails to delete files (#2485).\n- JMAP: Return error when using `blobId` in JSContact and JSCalendar (#2431).\n- Directory: Deletion of list or domain issues (#2415).\n- MTA: Headers and body stripped from mail delivery subsystem failure notifications (#2344).\n- MTA: Hooks only run if sieve script, milter or rewrite is configured (#2317).\n- Autodiscover: Endpoint should be case insensitive (#2440).\n- Housekeeper: Panic during DST transition (#2366).\n- Import/Export: Fix import/export utility (#1882).\n- Enterprise: Remove tenant admin permissions when license is invalid.\n\n## [0.14.1] - 2025-10-28\n\nIf you are upgrading from v0.13.4 and below, this version includes **breaking changes** to the internal directory, calendar and contacts. Please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_14.md) for more information on how to upgrade from previous versions.\n\n## Added\n- Autoconfig for CalDAV, CardDAV and WebDAV (#1937)\n\n## Changed\n- HTTP: Remove HTTP STS `preload` directive.\n\n## Fixed\n- Directory: Keep OTP Auth and AppPasswords unless the remote directory provides new ones (#2319)\n- JMAP: Fix `ContactCard/set` and `CalendarEvent/set` destroy methods (#2308).\n\n## [0.14.0] - 2025-10-22\n\nIf you are upgrading from v0.13.4 and below, this version includes **breaking changes** to the internal directory, calendar and contacts. Please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_14.md) for more information on how to upgrade from previous versions.\n\n## Added\n- JMAP for Calendars ([draft-ietf-jmap-calendars](https://datatracker.ietf.org/doc/draft-ietf-jmap-calendars/)).\n- JMAP for Contacts  ([RFC 9610](https://datatracker.ietf.org/doc/rfc9610/)).\n- JMAP for File Storage ([draft-ietf-jmap-filenode](https://datatracker.ietf.org/doc/draft-ietf-jmap-filenode/)).\n- JMAP Sharing ([RFC 9670](https://datatracker.ietf.org/doc/rfc9670/))\n- CalDAV: support for `supported-calendar-component-set` (#1893)\n- i18n: Greek language support (contributed by @infl00p)\n- i18n: Swedish language support (contributed by @purung)\n\n## Changed\n- **Breaking Database Changes** (migrated automatically on first start):\n  - Internal directory schema changed.\n  - Calendar and Contacts storage schema changed.\n  - Sieve scripts storage schema changed.\n  - Push Subscriptions storage schema changed.\n- Replaced `sieve.untrusted.limits.max-scripts` and `jmap.push.max-total` with `object-quota.*` settings. \n- Cluster node roles now allow sharding.\n\n\n## Fixed\n- Push Subscription: Clean-up of expired subscriptions and cluster notification of changes (#1248)\n- CalDAV: Per-user CalDAV properties (#2058)\n\n## [0.13.4] - 2025-09-30\n\nIf you are upgrading from v0.11.x or v0.12.x, this version includes **breaking changes** to the message queue and MTA configuration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n\n## Changed\n- JMAP: Protocol layer rewrite for zero-copy deserialization and architectural improvements.\n\n## Fixed\n- IMAP: Unbounded memory allocation in request parser ([CVE-2025-61600 ](https://github.com/stalwartlabs/stalwart/security/advisories/GHSA-8jqj-qj5p-v5rr)).\n- IMAP: Wrong permission checked for GETACL.\n- JMAP: References to previous method fail when there are no results (#1507).\n- JMAP: Enforce quota checks on `Blob/copy`.\n- JMAP: `Mailbox/get` fails without `accountId` argument (#1936).\n- JMAP: Do not return `invalidProperties` when email update doesn't contain changes (#1139)\n- iTIP: Include date properties in `REPLY` (#2102).\n- OIDC: Do not set `username` field if it is the same as the `email` field.\n- Telemetry: Fix `calculateMetrics` housekeeper task (#2155).\n- Directory: Always use `rsplit` to extract the domain part from email addresses.\n\n## [0.13.3] - 2025-09-10\n\nIf you are upgrading from v0.11.x or v0.12.x, this version includes **breaking changes** to the message queue and MTA configuration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- CLI: Health checks (contributed by @Codekloeppler)\n\n## Changed\n- WebDAV: Assisted discovery v2\n\n## Fixed\n- iTIP: Do not send a REPLY when deleting an event that was not accepted.\n- iTIP: Include event details in REPLY messages (#2102).\n- iTIP: Add organizer to iMIP replies if missing to deal with MS Exchange 2010 bug.\n- OIDC: Do not overwrite locally defined aliases (#2065).\n- HTTP: Scan ban should only be triggered by HTTP parse errors.\n- HTTP: Skip scanner fail2ban checks when the proxy client IP can't be parsed (#2121).\n- JMAP: Do not allow roles to be removed from system mailboxes (#1977).\n- JMAP WS: Fix panic when using invalid server url.\n- SMTP: Do no send `EHLO` twice when `STARTTLS` is unavailable (#2050).\n- IMAP: Allow `ENABLE UTF8` in IMAPrev1.\n- IMAP: Include `administer` permission in ACL responses.\n- IMAP: Add owner rights to ACL get responses.\n- IMAP: Do not auto-train Bayes when moving messages from Junk to Trash.\n- IMAP/ManageSieve: Increase maximum quoted argument size (#2039).\n- CalDAV: Limit recurrence expansions in calendar reports ([CVE-2025-59045](https://github.com/stalwartlabs/stalwart/security/advisories/GHSA-xv4r-q6gr-6pfg)).\n- WebDAV: Do not fix percent encoding on WebDAV FS (#2036).\n\n## [0.13.2] - 2025-07-28\n\nIf you are upgrading from v0.11.x or v0.12.x, this version includes **breaking changes** to the message queue and MTA configuration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- ACME: DeSEC cloud DNS provider support (contributed by @Tyr3al).\n- ACME: OVH cloud DNS provider support (contributed by @srachner).\n- CalDAV Scheduling: Catalan language support (contributed by @jolupa) (#1873).\n- MTA: Allow to send e-mails as group, while member of that group (#485).\n- OIDC: Allow local access tokens to be used with third-party OIDC backends (#1311 stalwartlabs/webadmin#52).\n\n## Changed\n- IMAP: Return `OK` when moving/copying non-existent messages (#670).\n- IMAP: Copy flags when copying/moving messages between accounts.\n\n## Fixed\n- MTA: Do not convert e-mail local parts to lowercase (#1916).\n- Sieve: `fileinto` should override spam filter (#1917).\n- JMAP: Incorrect `accountId` used in email set and import methods (#1777).\n- WebDAV: Always return `MULTISTATUS` when calendar-query yields no results.\n- LDAP: Only set account name if not returned in LDAP query (#1471).\n- Enterprise: Invalidate logo cache when changes are made (#1856).\n- Enterprise: Fix tenant quota update API.\n\n## [0.13.1] - 2025-07-16\n\nIf you are upgrading from v0.11.x or v0.12.x, this version includes **breaking changes** to the message queue and MTA configuration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- ACME: DigitalOcean cloud DNS provider support (#1667).\n\n## Changed\n\n## Fixed\n- Migration: Old queue events not deleted causing high CPU usage in some deployments (#1833).\n- MTA: `mta-sts` setting parsing issue (#1830).\n- JMAP: `sortOrder` should not be null (#1831).\n- Allow invalid TOML when parsing database settings (#1822).\n\n## [0.13.0] - 2025-07-15\n\nIf you are upgrading from v0.11.x or v0.12.x, this version includes **breaking changes** to the message queue and MTA configuration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- MTA queue enhancements (#1246 #1035 #457).\n- Danish locale support (contributed by @Fadil2k) (#1772).\n- DKIM support for `stalwart-cli` (contributed by @rmsc) (#1804).\n\n## Changed\n- Invalidate access token caches in a cluster using pub/sub (#1741).\n- Allow updating secrets for all directory types.\n\n## Fixed\n- WebDAV: Return all shared resources in `calendar-home-set` and `addressbook-home-set` (#1796).\n- WebDAV ACL: Fix write permission and `multiget` reports (#1768).\n- CalDAV Scheduling: Include `DTSTART`/`DTEND` properties in iMIP `CANCEL` messages (#1775).\n- HTTP: Do not include `WWW-Authenticate` headers in API responses (#1795).\n- API: Allow API keys to be used with external directories (#1815).\n- IMAP: Fix issue creating subfolders under INBOX for group shared folder (#1817).\n- IMAP: Custom Name for Shared Folders ignored (#1620).\n- LDAP: `local` placeholder should return username when its not an email address (#1784).\n\n## [0.12.5] - 2025-06-25\n\nIf you are upgrading from v0.11.x, this version includes **breaking changes** to the database layout and requires a migration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- Calendar Scheduling Extensions to CalDAV - RFC6368 (#1514)\n- Calendar E-Mail Notifications (#1514)\n- Limited i18n support for calendaring events.\n- Assisted CalDAV/CardDAV shared resource discovery (#1691).\n\n## Changed\n- JMAP: Allow unauthenticated access to JMAP session object.\n\n## Fixed\n- WebDAV: Return NOTFOUND error instead of MULTISTATUS on empty PROPFIND responses (#1657).\n- WebDAV: Update account name when refreshing DAV caches (#1694).\n- JMAP: Do not include email address in identity names (#1688).\n- IMAP: Normalize `INBOX` name when creating/renaming folders (#1636).\n- LDAP: Request `secret-changed` attribute in LDAP queries (#1409).\n- Branding: Unable to change logos (#1652).\n- Antispam: Skip `card-is-ham` override when sender does not pass DMARC (#1648).\n- FoundationDB: Renew old/expired FDB read transactions after the `1007` error code is received rather than estimating expiration time.\n\n## [0.12.4] - 2025-06-03\n\nIf you are upgrading from v0.11.x, this version includes **breaking changes** to the database layout and requires a migration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- LDAP authentication enhancements (#1269 #1471 #795 #1496).\n- MTA: Return Queue IDs during message acceptance (#927).\n\n## Changed\n- LDAP: `bind.auth.enable` is now `bind.auth.method`, read the updated [LDAP documentation](https://stalw.art/docs/auth/backend/ldap) for more information.\n\n## Fixed\n- DNS: `hickory-resolver` bug hitting 100% CPU usage when resolving DNSSEC records.\n- IMAP: Return the message UID in the destination mailbox if the message already exists (#1201).\n- MTA: TLS reports being issued for sent TLS reports (infinite loop) (#1301).\n- WebDAV: Return `CTag` on `/dav/cal/account` resources to force iOS synchronize.\n- CardDAV: Strict vCard parsing (#1607).\n- WebDAV: Dead property updates (#1611).\n- WebDAV: Use last change id in `CTag`.\n\n## [0.12.3] - 2025-05-30\n\nIf you are upgrading from v0.11.x, this version includes **breaking changes** to the database layout and requires a migration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- Store vanished IMAP UIDs and WebDAV paths in the changelog.\n\n## Changed\n\n## Fixed\n- XML `CDATA` injection (credits to @andreymal for the report).\n- Macro references are replaced with their content when writing config file (#1595).\n- Double nested CalDAV and CardDAV property tags (#1591).\n- Allow empty properties in PROPPATCH requests (#1580).\n\n## [0.12.2] - 2025-05-27\n\nIf you are upgrading from v0.11.x, this version includes **breaking changes** to the database layout and requires a migration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n- CardDAV: Legacy vCard 2.1 and 3.0 serialization support.\n- WebDAV: Add SRV Records to help DAV autodiscovery (closes #1565).\n\n## Changed\n\n## Fixed\n- Report list attempts to deserialize empty values (#1562)\n- Refresh expired FoundationDB transactions while retrieving large blobs (#1555).\n\n## [0.12.1] - 2025-05-26\n\nIf you are upgrading from v0.11.x, this version includes **breaking changes** to the database layout and requires a migration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n## Added\n\n## Changed\n\n## Fixed\n- Migration tool to generate the correct next id (#1561).\n- Failed to parse setting dav.lock.max-timeout (closes #1559).\n- Failed to build OpenTelemetry span exporter: no http client specified (#1571).\n\n## [0.12.0] - 2025-05-26\n\nThis version includes **breaking changes** to the database layout and requires a migration. Please read the [UPGRADING.md](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n### Added\n- [Collaboration](https://stalw.art/docs/collaboration/overview) features including [Calendars over CalDAV](https://stalw.art/docs/http/calendar/), [Contacts over CardDAV](https://stalw.art/docs/http/contact/) and [File Storage over WebDAV](https://stalw.art/docs/http/file-storage/).\n- Peer-to-peer [cluster coordination](https://stalw.art/docs/cluster/coordination/overview) or with Apache Kafka, Redpanda, NATS or Redis.\n- Incremental caching of emails, calendars, contacts and file metadata.\n- Zero-copy deserialization.\n- Train spam messages as ham when the sender is in the user's address book.  \n- `XOAUTH2` SASL mechanism support (#1194 #1369).\n- Support for RFC9698, the `JMAPACCESS` Extension for IMAP.\n- Search index for accounts and other principals (#1368).\n- Add `description` property to OIDC ID token (#1234).\n\n### Changed\n- Deprecated gossip protocol in favor of the new [coordinator](https://stalw.art/docs/cluster/coordination/overview) options.\n- Renamed Git repository from `stalwartlabs/mail-server` to `stalwartlabs/stalwart` and the Docker image from `stalwartlabs/mail-server` to `stalwartlabs/stalwart`.\n- Renamed multiple settings:\n  - `server.http.*` to `http.*`.\n  - `jmap.folders.*` to `email.folders.*`.\n  - `jmap.account.purge.frequency` to `account.purge.frequency`.\n  - `jmap.email.auto-expunge` to `email.auto-expunge`.\n  - `jmap.protocol.changes.max-history` to `changes.max-history`.\n  - `storage.encryption.*` to `email.encryption.*`.\n- Deprecated `lookup.default.*` settings in favor of `server.hostname` and `report.domain`. v0.11 and before supported both, v0.12 will only support the new settings.\n\n### Fixed\n- Allow undiscovered UIDs to be used in IMAP `COPY`/`MOVE` operations (#1201).\n- Refuse loopback SMTP delivery (#1377).\n- Hide the current server version (#1435).\n- Use the newest `X-Spam-Status` Header (#1308).\n- MySQL Driver error: Transactions couldn't be nested (#1271).\n- Spawn a delivery thread for `EmailSubmission/set` requests (#1540).\n- ACME: Don't restrict challenge types (#1522).\n- Autoconfig: return `%EMAILADDRESS%` if no e-mail address is provided (#1537).\n\n## [0.11.8] - 2025-04-30\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n\n### Changed\n\n### Fixed\n- Allow undiscovered UIDs to be used in `COPY`/`MOVE` operations (#1201).\n\n## [0.11.7] - 2025-03-23\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- LDAP attribute to indicate password change (#1156).\n\n### Changed\n- Lazy DKIM key parsing (#1211).\n- Enable `edns0` for system resolver by default (#1282).\n- Bump FoundationDB to `7.3`.\n\n### Fixed\n- Fix incorrect `UIDNEXT` when mailbox is empty (#1201).\n- Sender variable not set when evaluating `must-match-sender` (#1294).\n- Do not panic when mailboxId is not found (#1293).\n- Prioritize local over span keys when serializing webhook payloads (#1250).\n- Allow TLS name mismatch as per RFC7671 Section 5.1.\n- Try with implicit MX when no MX records are found.\n- SQL `secrets` directory query.\n\n## [0.11.5] - 2025-02-01\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n\n### Changed\n- Open source third party OIDC support.\n\n### Fixed\n- Case insensitive flag parsing (#1138).\n- BCC not removed from JMAP EmailSubmissions (#618).\n- Group pipelined IMAP FETCH and STATUS operations (#1096).\n\n## [0.11.4] - 2025-01-29\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- RFC 9208 - IMAP QUOTA Extension (#484).\n\n### Changed\n- `session.throttle.*` is now `queue.limiter.inbound.*`.\n- `queue.throttle.*` is now `queue.limiter.outbound.*`.\n- Changed DNSBL error level to debug (#1107).\n\n### Fixed\n- Creating a mailbox in a shared folder results in wrong hierarchy (#1128).\n- IMAP LIST-STATUS (RFC 5819) returns items in wrong order (#1129).\n- Avoid non-RFC SMTP status codes (#1109).\n- Do not DNSBL check invalid domains (#1107).\n- Sieve message flag parser (#1059).\n- Sieve script import case insensitivity (#962).\n- `mailto:` parsing in HTMLs.\n\n## [0.11.2] - 2025-01-17\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Automatic revoking of access tokens when secrets, permissions, ACLs or group memberships change (#649).\n- Increased concurrency for local message delivery (configurable via `queue.threads.local`).\n- Cluster node roles.\n- `config_get` expression function.\n\n### Changed\n- `queue.outbound.concurrency` is now `queue.threads.remote`.\n- `lookup.default.hostname` is now `server.hostname`.\n- `lookup.default.domain` is now `report.domain`.\n\n### Fixed\n- Distributed locking issues in non-Redis stores (#1066).\n- S3 incorrect backoff wait time after failures.\n- Panic parsing broken HTMLs.\n- Update CLI response serializer to v0.11.x (#1082).\n- Histogram bucket counts (#1079).\n- Do not rate limit trusted IPs (#1078).\n- Avoid double encrypting PGP parts encoded as plain text (#1083).\n- Return empty SASL challenge rather than \"\" (#1064).\n\n## [0.11.0] - 2025-01-06\n\nThis version includes breaking changes to the configuration file, please read [UPGRADING.md](UPGRADING.md) for details.\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Spam filter rewritten in Rust for a significant performance improvement.\n- Multiple spam filter improvements (#947) such as training spam/ham when moving between inbox and spam folders (#819).\n- Improved distributed locking and handling of large distributed SMTP queues.\n- ASN and GeoIP lookups.\n- Bulk operations REST endpoints (#925).\n- Faster S3-FIFO caching.\n- Support adding the `Delivered-To` header (#916).\n- Semver compatibility checks when upgrading (#844).\n- Sharded In-Memory Store.\n\n### Changed\n- Removed authentication rate limit (no longer necessary since there is fail2ban).\n- Pipes have been deprecated in favor of MTA hooks.\n\n### Fixed\n- OpenPGP EOF error (#1024).\n- Convert emails obtained from external directories to lowercase (#1004).\n- LDAP: Support both name and email fields to be mapped to the same attribute.\n- Admin role can't be assigned if an account with the same name exists.\n- Fix macro detection in DNS record generation (#978).\n- Use host FQDN in install script (#1003).\n\n## [0.10.7] - 2024-12-04\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Delivery and DMARC Troubleshooting (#420).\n- Support for external email addresses on mailing lists (#152).\n- Azure blob storage support.\n\n### Changed\n\n### Fixed\n- Some mails can't be moved out of the junk folder (#670).\n- Out of bound index error on Sieve script (#941).\n- Missing `User-Agent` header for ACME (#937).\n- UTF8 support in IMAP4rev1 (#948).\n- Account alias owner leak on autodiscover.\n- Include all events in OTEL traces + Include spanId in webhooks.\n- Implement `todo!()` causing panic on concurrency and rate limits.\n- Mark SQL store as active if used as a telemetry store.\n- Discard empty form submissions.\n\n## [0.10.6] - 2024-11-07\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Enterprise license automatic renewals before expiration (disabled by default).\n- Allow to LDAP search using bind dn instead of auth bind connection when bind auth is enabled (#873)\n\n### Changed\n\n### Fixed\n- Include `preferred_username` and `email` in OIDC `id_token`.\n- Verify roles and permissions when creating or modifying accounts (#874)\n\n## [0.10.5] - 2024-10-15\n\nTo upgrade replace the `stalwart-mail` binary. \n\n### Added\n- Data store CLI.\n\n### Changed\n\n### Fixed\n- Tokenizer performance issue (#863)\n- Incorrect AI model endpoint setting.\n\n## [0.10.4] - 2024-10-08\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin. \n\n### Added\n- Detect and ban port scanners as well as other forms of abuse (#820).\n- ACME External Account Binding support (#379).\n\n### Changed\n- The settings `server.fail2ban.*` have been moved to `server.auto-ban.*`.\n- The event `security.brute-force-ban` is now `security.abuse-ban`.\n\n### Fixed\n- Do not send SPF failures reports to local domains.\n- Allow `nonce` in OAuth code requests.\n- Warn when there are errors migrating domains rather than aborting migration.\n\n## [0.10.3] - 2024-10-07\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin. Enterprise users wishing to use the new LLM-powered spam filter should also upgrade the spam filter rules.\n\n### Added\n- AI-powered Spam filtering and Sieve scripting (Enterprise feature).\n\n### Changed\n- The untrusted Sieve interpreter now has the `vnd.stalwart.expressions` extension enabled by default. This allows Sieve users to use the `eval` function to evaluate expressions in their scripts. If you would like to disable this extension, you can do so by adding `vnd.stalwart.expressions` to `sieve.untrusted.disabled-capabilities`.\n\n### Fixed\n- S3-compatible backends: Retry on `5xx` errors.\n- OIDC: Include `nonce` parameter in `id_token` response.\n\n## [0.10.2] - 2024-10-02\n\nTo upgrade first upgrade the webadmin and then replace the `stalwart-mail` binary. If you read these instructions too late, you can upgrade to the latest web-admin using `curl -k -u admin:yourpass https://yourserver/api/update/webadmin`.\n\n### Added\n- OpenID Connect server (#298).\n- OpenID Connect backend support (Enterprise feature).\n- OpenID Connect Dynamic Client Registration (#4)\n- OAuth 2.0 Dynamic Client Registration Protocol ([RFC7591](https://datatracker.ietf.org/doc/html/rfc7591)) (#136)\n- OAuth 2.0 Token Introspection ([RFC7662](https://datatracker.ietf.org/doc/html/rfc7662)).\n- Contact form submission handling.\n- `webadmin.path` setting to override unpack directory (#792).\n\n### Changed\n\n### Fixed\n- Missing `LIST-STATUS` from RFC5819 in IMAP capability responses (#816).\n- Do not allow tenant domains to be deleted if they have members (#812).\n- Tenant principal limits (#810).\n\n## [0.10.1] - 2024-09-26\n\nTo upgrade replace the `stalwart-mail` binary.\n\n### Added\n- `OAUTHBEARER` SASL support in all services (#627).\n\n### Changed\n\n### Fixed\n- Fixed `migrate_directory` range scan (#784).\n\n## [0.10.0] - 2024-09-21\n\nThis version includes breaking changes to how accounts are stored. Please read [UPGRADING.md](UPGRADING.md) for details.\n\n### Added\n- Multi-tenancy (Enterprise feature).\n- Branding (Enterprise feature).\n- Roles and permissions.\n- Full-text search re-indexing.\n- Partial database backups (#497).\n\n### Changed\n\n### Fixed\n- IMAP `IDLE` support for command pipelining, aka the Apple Mail iOS 18 bug (#765).\n- Case insensitive INBOX `fileinto` (#763).\n- Properly decode undelete account name (#761).\n\n## [0.9.4] - 2024-09-09\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Support for global Sieve scripts that can be used by users to filter their incoming mail.\n- Allow localhost to override HTTP access controls to prevent lockouts.\n\n### Changed\n- Sieve runtime error default log level is now `debug`.\n\n### Fixed\n- Ignore INBOX case on Sieve's `fileinto` (#725)\n- Local keys parsing and retrieval issues.\n- Lookup reload does not include database settings.\n- Account count is incorrect.\n\n## [0.9.3] - 2024-08-29\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Dashboard (Enterprise feature)\n- Alerts (Enterprise feature)\n- SYN Flood (session \"loitering\") attack protection (#482)\n- Mailbox brute force protection (#688)\n- Mail from is allowed (`session.mail.is-allowed`) expression (#609)\n\n### Changed\n- `authentication.fail2ban` setting renamed to `server.fail2ban.authentication`.\n- Added elapsed times to message filtering events.\n\n### Fixed\n- Include queueId in MTA Hooks (#708)\n- Do not insert empty keywords in FTS index.\n\n## [0.9.2] - 2024-08-21\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Message delivery history (Enterprise feature)\n- Live tracing and logging (Enterprise feature)\n- SQL Read Replicas (Enterprise feature)\n- Distributed S3 Blob Store (Enterprise feature)\n\n### Changed\n\n### Fixed\n- Autodiscover request parser issues.\n- Do not create tables when using SQL as an external directory (fixes #291)\n- Do not hardcode logger id (fixes #348)\n- Include `Forwarded-For IP` address in `http.request-url` event (fixes #682)\n\n## [0.9.1] - 2024-08-08\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Metrics support (closes #478)\n  - OpenTelemetry Push Exporter\n  - Prometheus Pull Exporter (closes #275)\n- HTTP endpoint access controls (closes #266 #329 #542)\n- Add `options` setting to PostgreSQL driver (closes #662)\n- Add `isActive` property to defaults on Sieve/get JMAP method (closes #624)\n\n### Changed\n- Perform `must-match-sender` checks after sender rewriting (closes #394)\n- Only perform email ingest duplicate check on the target mailbox (closes #632)\n\n### Fixed\n- Properly parse `Forwarded` and `X-Forwarded-For` headers (fixes #669)\n- Resolve DKIM macros when generating DNS records (fixes #666)\n- Fixed `is_local_domain` Sieve function (fixes #622)\n\n## [0.9.0] - 2024-08-01\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin. This version includes breaking changes to the Webhooks configuration and produces a slightly different log output, read [UPGRADING.md](UPGRADING.md) for details.\n\n### Added\n- Improved and faster tracing and logging.\n- Customizable event logging levels.\n\n### Changed\n\n### Fixed\n- ManageSieve: Return capabilities after successful `STARTTLS`\n- Do not provide `{auth_authen}` Milter macro unless the user is authenticated\n\n## [0.8.5] - 2024-07-07\n\nTo upgrade replace the `stalwart-mail` binary.\n\n### Added\n- Restore deleted e-mails (Enterprise Edition only)\n- Kubernetes (K8S) livenessProbe and readinessProbe endpoints.\n\n### Changed\n- Avoid sending reports for DMARC/delivery reports (#173)\n\n### Fixed\n- Refresh old FoundationDB read transactions (#520)\n- Subscribing shared mailboxes doesn't work (#251)\n\n## [0.8.4] - 2024-07-03\n\nTo upgrade replace the `stalwart-mail` binary.\n\n### Added\n\n### Changed\n\n### Fixed\n- Fix TOTP validation order.\n- Increase Jemalloc page size on armv7 builds.\n\n## [0.8.3] - 2024-07-01\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin.\n\n### Added\n- Two-factor authentication with Time-based One-Time Passwords (#436)\n- Application passwords (#479).\n- Option to disable user accounts.\n\n### Changed\n- DANE success on EndEntity match regardless of TrustAnchor validation.\n\n### Fixed\n- Fix ManageSieve GETSCRIPT response: Add missing CRLF (#563)\n- Do not return CAPABILITIES after ManageSieve AUTH=PLAIN SASL exchange (#548)\n- POP3 QUIT must write a response (#568)\n\n## [0.8.2] - 2024-06-22\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin and spam filter versions.\n\n### Added\n- Webhooks support (#480)\n- MTA Hooks (like milter but over HTTP)\n- Manually train and test spam classifier (#473 #264 #257 #471)\n- Allow configuring default mailbox names, roles and subscriptions (#125 #290 #458 #498)\n- Include `robots.txt` (#542)\n\n### Changed\n- Milter support on all SMTP stages (#183)\n- Do not announce `STARTTLS` if the listener does not support it.\n\n### Fixed\n- Incoming reports stored in the wrong subspace (#543)\n- Return `OK` after a successful ManageSieve SASL authentication flow (#187)\n- Case-insensitive search in settings API (#487)\n- Fix `session.rcpt.script` default variable name (#502)\n\n## [0.8.1] - 2024-05-23\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin and spam filter versions.\n\n### Added\n- POP3 support.\n- DKIM signature length exploit protection.\n- Faster email deletion.\n- Junk/Trash folder auto-expunge and changelog auto-expiry (#403)\n- IP allowlists.\n- HTTP Strict Transport Security option.\n- Add TLS Reporting DNS entry (#464).\n\n### Changed\n- Use separate account for master user.\n- Include server hostname in SMTP greetings (#448).\n\n### Fixed\n- IP addresses trigger `R_SUSPICIOUS_URL` false positive (#461 #419).\n- JMAP identities should not return null signatures.\n- Include authentication headers and check queue quotas on Sieve message forwards.\n- ARC seal using just one signature.\n- Remove technical subdomains from MTA-STS policies and TLS records (#429).\n\n## [0.8.0] - 2024-05-13\n\nThis version uses a different database layout which is incompatible with previous versions. Please read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n### Added\n- Clustering support with node auto-discovery and partition-tolerant failure detection.\n- Autoconfig and MS Autodiscover support (#336)\n- New variables `retry_num`, `notify_num`, `last_error` add `last_status` available in queue expressions.\n- Performance improvements, in particular for FoundationDB.\n- Improved full-text indexing with lower disk space usage.\n- MTA-STS policy management.\n- TLSA Records generation for DANE (#397)\n- Queued message visualization from the web-admin.\n- Master user support.\n\n### Changed\n- Make `certificate.*` local keys by default.\n- Removed `server.run-as.*` settings.\n- Add Microsoft Office Macro types to bad mime types (#391)\n\n### Fixed\n- mySQL TLS support (#415)\n- Resolve file macros after dropping root privileges.\n- Updated order of SPF Records (#395).\n- Avoid duplicate accountIds when using case insensitive external directories (#399)\n- `authenticated_as` variable not usable for must-match-sender (#372)\n- Remove `StandardOutput`, `StandardError` in service (#390)\n- SMTP `AUTH=LOGIN` compatibility issues with Microsoft Outlook (#400)\n\n## [0.7.3] - 2024-05-01\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin version.\n\n### Added\n- Full database export and import functionality\n- Add --help and --version command line arguments (#365)\n- Allow catch-all addresses when validating must match sender\n\n### Changed\n- Add `groupOfUniqueNames` to the list of LDAP object classes\n\n### Fixed\n- Trim spaces in DNS-01 ACME secrets (#382)\n- Allow only one journald tracer (#375)\n- `authenticated_as` variable not usable for must-match-sender (#372)\n- Fixed `BOGUS_ENCRYPTED_AND_TEXT` spam filter rule\n- Fixed parsing of IPv6 DNS server addresses\n\n## [0.7.2] - 2024-04-17\n\nTo upgrade replace the `stalwart-mail` binary and then upgrade to the latest web-admin version.\n\n### Added\n- Support for `DNS-01` and `HTTP-01` ACME challenges (#226)\n- Configurable external resources (#355)\n\n### Changed\n\n### Fixed\n- Startup failure when Elasticsearch is down/starting up (#334)\n- URL decode path elements in REST API.\n\n## [0.7.1] - 2024-04-12\n\nTo upgrade replace the `stalwart-mail` binary.\n\n### Added\n- Make initial admin password configurable via env (#311)\n\n### Changed\n- WebAdmin download URL.\n\n### Fixed\n- Remove ASN.1 DER structure from DKIM ED25519 public keys.\n- Filter out invalid timestamps on log entries.\n\n## [0.7.0] - 2024-04-09\n\nThis version uses a different database layout and introduces multiple breaking changes in the configuration files. Please read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n### Added\n- Web-based administration interface.\n- REST API for management and configuration.\n- Automatic RSA and ED25519 DKIM key generation.\n- Support for compressing binaries in the blob store (#227).\n- Improved performance accessing IMAP mailboxes with a large number of messages.\n- Support for custom DNS resolvers.\n- Support for multiple loggers with different levels and outputs.\n\n### Changed\n\n### Fixed\n- Store quotas as `u64` rather than `u32`.\n- Second IDLE connections disconnects the first one (#280).\n- Use relaxed DNS parsing, allowing underscores in DNS labels (#172).\n- Escape regexes within `matches()` expressions (#155).\n- ManageSieve LOGOUT should reply with `OK` instead of `BYE`.\n\n## [0.6.0] - 2024-02-14\n\nThis version introduces breaking changes in the configuration file. Please read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n### Added\n- Distributed and fault-tolerant SMTP message queues.\n- Distributed rate-limiting and fail2ban.\n- Expressions in configuration files.\n\n### Changed\n\n### Fixed\n- Do not include `STATUS` in IMAP `NOOP` responses (#234).\n- Allow multiple SMTP `HELO` commands.\n- Redirect OAuth using a `301` instead of a `307` code.\n\n## [0.5.3] - 2024-01-14\n\nPlease read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n### Added\n- Built-in [fail2ban](https://stalw.art/docs/server/fail2ban) and IP address/mask blocking (#164).\n- CLI: Read URL and credentials from environment variables (#88).\n- mySQL driver: Add `max-allowed-packet` setting (#201).\n\n### Changed\n- Unified storage settings for all services (read the [UPGRADING.md](UPGRADING.md) for details)\n\n### Fixed\n- IMAP retrieval of auto-encrypted emails (#203).\n- mySQL driver: Parse `timeout.wait` property as duration (#202).\n- `X-Forwarded-For` header on JMAP Rate-Limit does not work (#208).\n- Use timeouts in install script (#138).\n\n## [0.5.2] - 2024-01-07\n\nPlease read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.\n\n### Added\n- [ACME](https://stalw.art/docs/server/tls/acme) support for automatic TLS certificate generation and renewal (#160).\n- TLS certificate [hot-reloading](https://stalw.art/docs/management/database/maintenance#tls-certificate-reloading).\n- [HAProxy protocol](https://stalw.art/docs/server/proxy) support (#36).\n\n### Changed\n\n### Fixed\n- IMAP command `SEARCH <seqnum>` is using UIDs rather than sequence numbers.\n- IMAP responses to `APPEND` and `EXPUNGE` should include `HIGHESTMODSEQ` when `CONDSTORE` is enabled.\n\n## [0.5.1] - 2024-01-02\n\n### Added\n- SMTP smuggling protection: Sanitization of outgoing messages that do not use `CRLF` as line endings.\n- SMTP sender validation for authenticated users: Added the `session.auth.must-match-sender` configuration option to enforce that the sender address used in the `MAIL FROM` command matches the authenticated user or any of their associated e-mail addresses.\n\n### Changed\n\n### Fixed\n- Invalid DKIM signatures for empty message bodies.\n- IMAP command `SEARCH BEFORE` is not properly parsed.\n- IMAP command `FETCH` fails to parse single arguments without parentheses.\n- IMAP command `ENABLE QRESYNC` should also enable `CONDSTORE` extension.\n- IMAP response to `ENABLE` command does not include enabled capabilities list.\n- IMAP response to `FETCH ENVELOPE` should not return `NIL` when the `From` header is missing.\n\n## [0.5.0] - 2023-12-27\n\nThis version requires a database migration and introduces breaking changes in the configuration file. Please read the [UPGRADING.md](UPGRADING.md) file for more information.\n\n### Added\n- Performance enhancements:\n  - Messages are parsed only once and their offsets stored in the database, which avoids having to parse them on every `FETCH` request.\n  - Background full-text indexing.\n  - Optimization of database access functions.\n- Storage layer improvements:\n  - In addition to `FoundationDB` and `SQLite`, now it is also possible to use `RocksDB`, `PostgreSQL` and `mySQL` as a storage backend.\n  - Blobs can now be stored in any of the supported data stores, it is no longer limited to the file system or S3/MinIO. \n  - Full-text searching con now be done internally or delegated to `ElasticSearch`.\n  - Spam databases can now be stored in any of the supported data stores or `Redis`. It is no longer necessary to have an SQL server to use the spam filter.\n- Internal directory: \n  - User account, groups and mailing lists can now be managed directly from Stalwart without the need of an external LDAP or SQL directory.\n  - HTTP API to manage users, groups, domains and mailing lists.\n- IMAP4rev1 `Recent` flag support, which improves compatibility with old IMAP clients.\n- LDAP bind authentication, to support some LDAP servers such as `lldap` which do not expose the userPassword attribute.\n- Messages marked a spam by the spam filter can now be automatically moved to the account's `Junk Mail` folder.\n- Automatic creation of JMAP identities.\n\n### Changed\n\n### Fixed\n- Spamhaus DNSBL return codes.\n- CLI tool reports authentication errors rather than a parsing error.\n\n## [0.4.2] - 2023-11-01\n\n### Added\n- JMAP for Quotas support ([RFC9425](https://www.rfc-editor.org/rfc/rfc9425.html))\n- JMAP Blob Management Extension support ([RFC9404](https://www.rfc-editor.org/rfc/rfc9404.html))\n- Spam Filter - Empty header rules.\n\n### Changed\n\n### Fixed\n- Daylight savings time support for crontabs.\n- JMAP `oldState` doesn’t reflect in `*/changes` (#56)\n\n## [0.4.1] - 2023-10-26\n\n### Added\n\n### Changed\n\n### Fixed\n- Dockerfile entrypoint script.\n- `bayes_is_balanced` function.\n\n## [0.4.0] - 2023-10-25\n\nThis version introduces some breaking changes in the configuration file. Please read the [UPGRADING.md](UPGRADING.md) file for more information.\n\n### Added\n- Built-in Spam and Phishing filter.\n- Scheduled queries on some directory types.\n- In-memory maps and lists containing glob or regex patterns.\n- Remote retrieval of in-memory list/maps with fallback mechanisms.\n- Macros and support for including files from TOML config files.\n\n### Changed\n- `config.toml` is now split in multiple TOML files for better organization.\n- **BREAKING:** Configuration key prefix `jmap.sieve` (JMAP Sieve Interpreter) has been renamed to `sieve.untrusted`.\n- **BREAKING:** Configuration key prefix `sieve` (SMTP Sieve Interpreter) has been renamed to `sieve.trusted`.\n\n### Fixed\n\n## [0.3.10] - 2023-10-17\n\n### Added\n- Option to allow invalid certificates on outbound SMTP connections.\n- Option to disable ansi colors on `stdout`.\n\n### Changed\n- SMTP reject messages are now logged as `info` rather than `debug`.\n\n### Fixed\n\n## [0.3.9] - 2023-10-07\n\n### Added\n- Support for reading environment variables from the configuration file using the `!ENV_VAR_NAME` special keyword.\n- Option to disable ANSI color codes in logs.\n\n### Changed\n- Querying directories from a Sieve script is now done using the `query()` method from `eval`. Your scripts will need to be updated, please refer to the [new syntax](https://stalw.art/docs/smtp/filter/sieve#directory-queries).\n\n### Fixed\n- IPrev lookups of IPv4 mapped to IPv6 addresses.\n\n## [0.3.8] - 2023-09-19\n\n### Added\n- Journal logging support\n- IMAP support for UTF8 APPEND\n\n### Changed\n- Replaced `rpgp` with `sequoia-pgp` due to rpgp bug.\n\n### Fixed\n- Fix: IMAP folders that contain a & can't be used (#90) \n- Fix: Ignore empty lines in IMAP requests\n\n## [0.3.7] - 2023-09-05\n\n### Added\n- Option to disable IMAP All Messages folder (#68).\n- Option to allow unencrypted SMTP AUTH (#72)\n- Support for `rcpt-domain` key in `rcpt.relay` SMTP rule evaluation.\n\n### Changed\n \n### Fixed\n- SMTP strategy `Ipv6thenIpv4` returns only IPv6 addresses (#70)\n- Invalid IMAP `FETCH` responses for non-UTF-8 messages (#70)\n- Allow `STATUS` and `ACL` IMAP operations on virtual mailboxes.\n- IMAP `SELECT QRESYNC` without specifying a UID causes panic (#67)\n- Milter `DATA` command is sent after headers which causes ClamAV to hang.\n- Sieve `redirect` of unmodified messages does not work.\n\n## [0.3.6] - 2023-08-29\n\n### Added\n- Arithmetic and logical expression evaluation in Sieve scripts.\n- Support for storing query results in Sieve variables.\n- Results of SPF, DKIM, ARC, DMARC and IPREV checks available as environment variables in Sieve scripts.\n- Configurable protocol flags for Milter filters.\n- Fall-back to plain text when `STARTTLS` fails and `starttls` is set to `optional`.\n\n### Changed\n \n### Fixed\n- Do not panic when `hash = 0` in reports. (#60)\n- JMAP Session resource returns `EmailSubmission` capabilities using arrays rather than objects.\n- ManageSieve `PUTSCRIPT` should replace existing scripts.\n\n## [0.3.5] - 2023-08-18\n\n### Added\n- TCP listener option `nodelay`.\n \n### Changed\n \n### Fixed\n- SMTP: Allow disabling `STARTTLS`.\n- JMAP: Support for `OPTIONS` HTTP method.\n\n## [0.3.4] - 2023-08-09\n\n### Added\n- JMAP: Support for setting custom HTTP response headers (#52)\n \n### Changed\n \n### Fixed\n- SMTP: Missing envelope keys in rewrite rules (#25) \n- SMTP: Remove CRLF from Milter headers\n- JMAP/IMAP: Successful authentication requests should not count when rate limiting\n- IMAP: Case insensitive Inbox selection\n- IMAP: Automatically create Inbox for group accounts\n\n## [0.3.3] - 2023-08-02\n\n### Added\n- Encryption at rest with **S/MIME** or **OpenPGP**.\n- Support for referencing context variables from dynamic values.\n \n### Changed\n \n### Fixed\n- Support for PKCS8v1 ED25519 keys (#20).\n- Automatic retry for import/export blob downloads (#14)\n\n## [0.3.2] - 2023-07-28\n\n### Added\n- Sender and recipient address rewriting using regular expressions and sieve scripts.\n- Subaddressing and catch-all addresses using regular expressions (#10).\n- Dynamic variables in SMTP rules.\n \n### Changed\n- Added CLI to Docker container (#19).\n \n### Fixed\n- Workaround for a bug in `sqlx` that caused SQL time-outs (#15).\n- Support for ED25519 certificates in PEM files (#20). \n- Better handling of concurrent IMAP UID map modifications (#17).\n- LDAP domain lookups from SMTP rules.\n\n## [0.3.1] - 2023-07-22\n\n### Added\n- Milter filter support.\n- Match IP address type using /0 mask (#16).\n \n### Changed\n \n### Fixed\n- Support for OpenLDAP password hashing schemes between curly brackets (#8). \n- Add CA certificates to Docker runtime (#5).\n\n## [0.3.0] - 2023-07-16\n\n### Added\n- **LDAP** and **SQL** authentication.\n- **subaddressing** and **catch-all** addresses.\n- **S3-compatible** storage.\n\n### Changed\n- Merged the `stalwart-jmap`, `stalwart-imap` and `stalwart-smtp` repositories into\n  `stalwart-mail`.\n- Removed clustering module and replaced it with a **FoundationDB** backend option.\n- Integrated Stalwart SMTP into Stalwart JMAP.\n- Rewritten JMAP protocol parser.\n- Rewritten store backend.\n- Rewritten IMAP server to have direct access to the message store (no more IMAP proxy).\n- Replaced `actix` with `hyper`.\n \n### Fixed\n\n"
  },
  {
    "path": "CNAME",
    "content": "get.stalw.art\n"
  },
  {
    "path": "CONTRIBUTING.md",
    "content": "# Contributing\n\n## Contributions are Temporarily Limited\n\nThank you for your interest in contributing to Stalwart. We appreciate the support and enthusiasm of the open-source community. However, at this stage of the project, we are **limiting the scope of external contributions**.\n\nStalwart is currently **not accepting external contributions**, except for bug fixes and small, well-scoped changes. The project is approaching version 1.0, and as we move toward this milestone, development is progressing rapidly. The architecture of Stalwart is still evolving, and many internal components are subject to change.\n\nDue to these ongoing changes and the fast pace of development, we do not have the time or resources to thoroughly review and integrate most pull requests. Accepting broad contributions at this time could lead to confusion and unnecessary rework for both contributors and maintainers.\n\nWhile we are not accepting most code contributions, you can still support the project in meaningful ways. Reporting bugs, providing feedback, and helping test the software are all valuable forms of participation. If you encounter an issue, please open a detailed report that includes steps to reproduce the problem and any relevant logs or context. We also welcome thoughtful suggestions and questions through our issue tracker or discussion channels.\n\nWe plan to open the project to broader contributions once we reach a stable 1.0 release. At that point, with a more mature architecture and clearer development roadmap, we will be better positioned to collaborate with the community. We will update this policy accordingly when the time comes.\n\nThank you for your understanding and continued support. We’re excited about the future of Stalwart and look forward to working with the community in the near future.\n\n## Code of Conduct\n\nPlease note we have a code of conduct, please follow it in all your interactions with the project.\n\n## Licensing\n\nThis project is licensed under the Affero General Public License (AGPL) version 3.0. By contributing to this project, you agree that your contributions will be licensed under the AGPL-3.0 license.\n\n## Fiduciary Contributor License Agreement\n\nBefore making any contributions, all contributors are required to sign the Fiduciary Contributor License Agreement (FLA). The FLA is a legal agreement that assigns the copyright of contributions to a designated fiduciary, who manages these rights on behalf of the project. This arrangement ensures that the software remains free and open, even as contributors come and go.\n\nKey points of the FLA:\n\n- Ensures the software remains free and open source\n- Protects the project from potential copyright issues\n- Includes a reversion clause: if the fiduciary violates Free Software principles, rights revert to the original contributors\n\nFor more details about FLA, please refer to the [FLA FAQ](https://fsfe.org/activities/fla/fla.en.html).\n\n## Pull Request Process\n\n1. Ensure any install or build dependencies are removed before the end of the layer when doing a \n   build.\n2. Update the README.md with details of changes to the interface, this includes new environment \n   variables, exposed ports, useful file locations and container parameters.\n3. Increase the version numbers in any examples files and the README.md to the new version that this\n   Pull Request would represent. The versioning scheme we use is [SemVer](http://semver.org/).\n4. You may merge the Pull Request in once you have the sign-off of two other developers, or if you \n   do not have permission to do that, you may request the second reviewer to merge it for you.\n\n## Code of Conduct\n\nWe as members, contributors, and leaders pledge to make participation in our community a harassment-free \nexperience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex \ncharacteristics, gender identity and expression, level of experience, education, socio-economic status, \nnationality, personal appearance, race, religion, or sexual identity and orientation.\nWe pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, \nand healthy community.\n\nYou can read the full Code of Conduct [here](https://github.com/stalwartlabs/.github/blob/main/CODE_OF_CONDUCT.md).\n"
  },
  {
    "path": "Cargo.toml",
    "content": "[workspace]\nresolver = \"2\"\nmembers = [\n    \"crates/main\",\n    \"crates/types\",\n    \"crates/http\",\n    \"crates/http-proto\",\n    \"crates/jmap\",\n    \"crates/jmap-proto\",\n    \"crates/email\",\n    \"crates/imap\",\n    \"crates/imap-proto\",\n    \"crates/smtp\",\n    \"crates/managesieve\",\n    \"crates/pop3\",\n    \"crates/dav-proto\",\n    \"crates/dav\",\n    \"crates/groupware\",\n    \"crates/spam-filter\",\n    \"crates/nlp\",\n    \"crates/store\",\n    \"crates/directory\",\n    \"crates/services\",\n    \"crates/utils\",\n    \"crates/common\",\n    \"crates/trc\",\n    \"crates/migration\",\n    \"crates/cli\",\n    \"tests\",\n]\n\n[profile.dev]\nopt-level = 0\ndebug = 1\n#codegen-units = 4\nlto = false\nincremental = true\npanic = 'unwind'\ndebug-assertions = true\noverflow-checks = false\nrpath = false\n\n[profile.release]\nopt-level = 3\ndebug = false\ncodegen-units = 1\nlto = true\nincremental = false\npanic = 'unwind'\ndebug-assertions = false\noverflow-checks = false\nrpath = false\nstrip = true\n\n[profile.test]\nopt-level = 0\ndebug = 1\n#codegen-units = 16\nlto = false\nincremental = true\ndebug-assertions = true\noverflow-checks = true\nrpath = false\n\n[profile.bench]\nopt-level = 3\ndebug = false\ncodegen-units = 1\nlto = true\nincremental = false\ndebug-assertions = false\noverflow-checks = false\nrpath = false\n"
  },
  {
    "path": "Dockerfile",
    "content": "# Stalwart Dockerfile\n# Credits: https://github.com/33KK \n\nFROM --platform=$BUILDPLATFORM docker.io/lukemathwalker/cargo-chef:latest-rust-slim-trixie AS chef\nWORKDIR /build\n\nFROM --platform=$BUILDPLATFORM chef AS planner\nCOPY . .\nRUN cargo chef prepare --recipe-path /recipe.json\n\nFROM --platform=$BUILDPLATFORM chef AS builder\nARG TARGETPLATFORM\nRUN case \"${TARGETPLATFORM}\" in \\\n    \"linux/arm64\") echo \"aarch64-unknown-linux-gnu\" > /target.txt && echo \"-C linker=aarch64-linux-gnu-gcc\" > /flags.txt ;; \\\n    \"linux/amd64\") echo \"x86_64-unknown-linux-gnu\" > /target.txt && echo \"-C linker=x86_64-linux-gnu-gcc\" > /flags.txt ;; \\\n    *) exit 1 ;; \\\n    esac\nRUN export DEBIAN_FRONTEND=noninteractive && \\\n    apt-get update && \\\n    apt-get install -yq --no-install-recommends build-essential libclang-19-dev \\\n    g++-aarch64-linux-gnu binutils-aarch64-linux-gnu \\\n    g++-x86-64-linux-gnu binutils-x86-64-linux-gnu\nRUN rustup target add \"$(cat /target.txt)\"\nCOPY --from=planner /recipe.json /recipe.json\nRUN RUSTFLAGS=\"$(cat /flags.txt)\" cargo chef cook --target \"$(cat /target.txt)\" --release --no-default-features --features \"sqlite postgres mysql rocks s3 redis azure nats enterprise\" --recipe-path /recipe.json\nCOPY . .\nRUN RUSTFLAGS=\"$(cat /flags.txt)\" cargo build --target \"$(cat /target.txt)\" --release -p stalwart --no-default-features --features \"sqlite postgres mysql rocks s3 redis azure nats enterprise\"\nRUN RUSTFLAGS=\"$(cat /flags.txt)\" cargo build --target \"$(cat /target.txt)\" --release -p stalwart-cli\nRUN mv \"/build/target/$(cat /target.txt)/release\" \"/output\"\n\nFROM docker.io/debian:trixie-slim\nWORKDIR /opt/stalwart\nRUN export DEBIAN_FRONTEND=noninteractive && \\\n    apt-get update && \\\n    apt-get install -yq --no-install-recommends ca-certificates\nCOPY --from=builder /output/stalwart /usr/local/bin\nCOPY --from=builder /output/stalwart-cli /usr/local/bin\nCOPY ./resources/docker/entrypoint.sh /usr/local/bin/entrypoint.sh\nRUN chmod -R 755 /usr/local/bin\nCMD [\"/usr/local/bin/stalwart\"]\nVOLUME [ \"/opt/stalwart\" ]\nEXPOSE\t443 25 110 587 465 143 993 995 4190 8080\nENTRYPOINT [\"/bin/sh\", \"/usr/local/bin/entrypoint.sh\"]\n"
  },
  {
    "path": "Dockerfile.build",
    "content": "# syntax=docker/dockerfile:1\n# check=skip=FromPlatformFlagConstDisallowed,RedundantTargetPlatform\n\n# *****************\n# Base image for planner & builder\n# *****************\nFROM --platform=$BUILDPLATFORM rust:slim-trixie AS base\n\nENV DEBIAN_FRONTEND=\"noninteractive\" \\\n    BINSTALL_DISABLE_TELEMETRY=true \\\n    CARGO_TERM_COLOR=always \\\n    LANG=C.UTF-8 \\\n    TZ=UTC \\\n    TERM=xterm-256color\n# With zig, we only need libclang and make\nRUN \\\n    --mount=type=cache,target=/var/cache/apt,sharing=locked \\\n    --mount=type=cache,target=/var/lib/apt,sharing=locked \\\n    rm -f /etc/apt/apt.conf.d/docker-clean && \\\n    echo 'Binary::apt::APT::Keep-Downloaded-Packages \"true\";' >/etc/apt/apt.conf.d/keep-cache && \\\n    apt-get update && \\\n    apt-get install -yq --no-install-recommends curl jq xz-utils make libclang-19-dev\n# Install zig\nRUN \\\n    ZIG_VERSION=0.13.0 && \\\n    [ ! -z \"$ZIG_VERSION\" ] && \\\n    curl --retry 5 -Ls \"https://ziglang.org/download/${ZIG_VERSION}/zig-linux-$(uname -m)-${ZIG_VERSION}.tar.xz\" | tar -J -x -C /usr/local && \\\n    ln -s \"/usr/local/zig-linux-$(uname -m)-${ZIG_VERSION}/zig\" /usr/local/bin/zig\n# Install cargo-binstall\nRUN curl --retry 5 -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash\n# Install cargo-chef & sccache & cargo-zigbuild\nRUN cargo binstall --no-confirm cargo-chef sccache cargo-zigbuild\n\n# *****************\n# Planner\n# *****************\nFROM base AS planner\nWORKDIR /app\nCOPY . .\n# Generate recipe file\nRUN cargo chef prepare --recipe-path recipe.json\n\n# *****************\n# Builder\n# *****************\nFROM base AS builder\nWORKDIR /app\nCOPY --from=planner /app/recipe.json recipe.json\nARG TARGET\nARG BUILD_ENV\nSHELL [\"/bin/bash\", \"-o\", \"pipefail\", \"-c\"]\n# Install toolchain and specify some env variables\nRUN \\\n    rustup set profile minimal && \\\n    rustup target add ${TARGET} && \\\n    mkdir -p artifact && \\\n    touch /env-cargo && \\\n    if [ ! -z \"${BUILD_ENV}\" ]; then \\\n        echo \"export ${BUILD_ENV}\" >> /env-cargo; \\\n        echo \"Setting up ${BUILD_ENV}\"; \\\n    fi && \\\n    if [[ \"${TARGET}\" == *gnu ]]; then \\\n        base_arch=\"${TARGET%%-*}\"; \\\n        case \"$base_arch\" in \\\n            x86_64) \\\n                echo \"export FDB_ARCH=amd64\" >> /env-cargo; \\\n                ;; \\\n            aarch64) \\\n                echo \"export FDB_ARCH=aarch64\" >> /env-cargo; \\\n                ;; \\\n            *) \\\n                exit 1; \\\n                ;; \\\n        esac; \\\n    fi\n# Install FoundationDB\nRUN \\\n    source /env-cargo && \\\n    if [ ! -z \"${FDB_ARCH}\" ]; then \\\n        curl --retry 5 -Lso fdb-client.deb \"$(curl --retry 5 -Ls 'https://api.github.com/repos/apple/foundationdb/releases' | jq --arg FDB_ARCH \"$FDB_ARCH\" -r '.[] | select(.prerelease == false) | .assets[] | select(.name | test(\"foundationdb-clients.*\" + $FDB_ARCH + \".deb$\")) | .browser_download_url' | head -n1)\" && \\\n        mkdir -p /fdb && \\\n        dpkg -x fdb-client.deb /fdb && \\\n        mv /fdb/usr/include/foundationdb /usr/include && \\\n        mv /fdb/usr/lib/libfdb_c.so /usr/lib && \\\n        rm -rf fdb-client.deb /fdb; \\\n    fi\n# Cargo-chef Cache layer\nRUN \\\n    --mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \\\n    --mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \\\n    --mount=type=cache,target=/usr/local/cargo/registry \\\n    --mount=type=cache,target=/usr/local/cargo/git \\\n    source /env-cargo && \\\n    if [ ! -z \"${FDB_ARCH}\" ]; then \\\n        RUSTFLAGS=\"-L /usr/lib\" cargo chef cook --recipe-path recipe.json --zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features \"foundationdb s3 redis nats enterprise\"; \\\n    fi\nRUN \\\n    --mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \\\n    --mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \\\n    --mount=type=cache,target=/usr/local/cargo/registry \\\n    --mount=type=cache,target=/usr/local/cargo/git \\\n    source /env-cargo && \\\n    cargo chef cook --recipe-path recipe.json --zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features \"sqlite postgres mysql rocks s3 redis azure nats enterprise\" && \\\n    cargo chef cook --recipe-path recipe.json --zigbuild --release --target ${TARGET} -p stalwart-cli\n# Copy the source code\nCOPY . .\nENV RUSTC_WRAPPER=\"sccache\" \\\n    SCCACHE_GHA_ENABLED=true\n# Build FoundationDB version\nRUN \\\n    --mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \\\n    --mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \\\n    --mount=type=cache,target=/usr/local/cargo/registry \\\n    --mount=type=cache,target=/usr/local/cargo/git \\\n    source /env-cargo && \\\n    if [ ! -z \"${FDB_ARCH}\" ]; then \\\n        RUSTFLAGS=\"-L /usr/lib\" cargo zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features \"foundationdb s3 redis nats enterprise\" && \\\n        mv /app/target/${TARGET}/release/stalwart /app/artifact/stalwart-foundationdb; \\\n    fi\n# Build generic version\nRUN \\\n    --mount=type=secret,id=ACTIONS_RESULTS_URL,env=ACTIONS_RESULTS_URL \\\n    --mount=type=secret,id=ACTIONS_RUNTIME_TOKEN,env=ACTIONS_RUNTIME_TOKEN \\\n    --mount=type=cache,target=/usr/local/cargo/registry \\\n    --mount=type=cache,target=/usr/local/cargo/git \\\n    source /env-cargo && \\\n    cargo zigbuild --release --target ${TARGET} -p stalwart --no-default-features --features \"sqlite postgres mysql rocks s3 redis azure nats enterprise\" && \\\n    cargo zigbuild --release --target ${TARGET} -p stalwart-cli && \\\n    mv /app/target/${TARGET}/release/stalwart /app/artifact/stalwart && \\\n    mv /app/target/${TARGET}/release/stalwart-cli /app/artifact/stalwart-cli\n\n# *****************\n# Binary stage\n# *****************\nFROM scratch AS binaries\nCOPY --from=builder /app/artifact /\n\n# *****************\n# Runtime image for GNU targets\n# *****************\nFROM --platform=$TARGETPLATFORM docker.io/library/debian:trixie-slim AS gnu\nWORKDIR /opt/stalwart\nRUN export DEBIAN_FRONTEND=noninteractive && \\\n    apt-get update && \\\n    apt-get install -yq --no-install-recommends ca-certificates tzdata\nCOPY --from=builder /app/artifact/stalwart /usr/local/bin\nCOPY --from=builder /app/artifact/stalwart-cli /usr/local/bin\nCOPY ./resources/docker/entrypoint.sh /usr/local/bin/entrypoint.sh\nRUN chmod -R 755 /usr/local/bin\nCMD [\"/usr/local/bin/stalwart\"]\nVOLUME [ \"/opt/stalwart\" ]\nEXPOSE\t443 25 110 587 465 143 993 995 4190 8080\nENTRYPOINT [\"/bin/sh\", \"/usr/local/bin/entrypoint.sh\"]\n\n# *****************\n# Runtime image for musl targets\n# *****************\nFROM --platform=$TARGETPLATFORM alpine AS musl\nWORKDIR /opt/stalwart\nRUN apk add --update --no-cache ca-certificates tzdata && rm -rf /var/cache/apk/*\nCOPY --from=builder /app/artifact/stalwart /usr/local/bin\nCOPY --from=builder /app/artifact/stalwart-cli /usr/local/bin\nCOPY ./resources/docker/entrypoint.sh /usr/local/bin/entrypoint.sh\nRUN chmod -R 755 /usr/local/bin\nCMD [\"/usr/local/bin/stalwart\"]\nVOLUME [ \"/opt/stalwart\" ]\nEXPOSE\t443 25 110 587 465 143 993 995 4190 8080\nENTRYPOINT [\"/bin/sh\", \"/usr/local/bin/entrypoint.sh\"]\n"
  },
  {
    "path": "LICENSES/AGPL-3.0-only.txt",
    "content": "GNU AFFERO GENERAL PUBLIC LICENSE\nVersion 3, 19 November 2007\n\nCopyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>\n\nEveryone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.\n\n                            Preamble\n\nThe GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software.\n\nThe licenses for most software and other practical works are designed to take away your freedom to share and change the works.  By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users.\n\nWhen we speak of free software, we are referring to freedom, not price.  Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.\n\nDevelopers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software.\n\nA secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate.  Many developers of free software are heartened and encouraged by the resulting cooperation.  However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public.\n\nThe GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community.  It requires the operator of a network server to provide the source code of the modified version running there to the users of that server.  Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version.\n\nAn older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals.  This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license.\n\nThe precise terms and conditions for copying, distribution and modification follow.\n\n                       TERMS AND CONDITIONS\n\n0. Definitions.\n\n\"This License\" refers to version 3 of the GNU Affero General Public License.\n\n\"Copyright\" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks.\n\n\"The Program\" refers to any copyrightable work licensed under this License.  Each licensee is addressed as \"you\".  \"Licensees\" and \"recipients\" may be individuals or organizations.\n\nTo \"modify\" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy.  The resulting work is called a \"modified version\" of the earlier work or a work \"based on\" the earlier work.\n\nA \"covered work\" means either the unmodified Program or a work based on the Program.\n\nTo \"propagate\" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy.  Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well.\n\nTo \"convey\" a work means any kind of propagation that enables other parties to make or receive copies.  Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying.\n\nAn interactive user interface displays \"Appropriate Legal Notices\" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License.  If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion.\n\n1. Source Code.\nThe \"source code\" for a work means the preferred form of the work for making modifications to it.  \"Object code\" means any non-source form of a work.\n\nA \"Standard Interface\" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language.\n\nThe \"System Libraries\" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form.  A \"Major Component\", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it.\n\nThe \"Corresponding Source\" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities.  However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work.  For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those\nsubprograms and other parts of the work.\n\nThe Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source.\n\nThe Corresponding Source for a work in source code form is that same work.\n\n2. Basic Permissions.\nAll rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met.  This License explicitly affirms your unlimited permission to run the unmodified Program.  The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work.  This License acknowledges your rights of fair use or other equivalent, as provided by copyright law.\n\nYou may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force.  You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright.  Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you.\n\nConveying under any other circumstances is permitted solely under the conditions stated below.  Sublicensing is not allowed; section 10 makes it unnecessary.\n\n3. Protecting Users' Legal Rights From Anti-Circumvention Law.\nNo covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures.\n\nWhen you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures.\n\n4. Conveying Verbatim Copies.\nYou may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program.\n\nYou may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee.\n\n5. Conveying Modified Source Versions.\nYou may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions:\n\n    a) The work must carry prominent notices stating that you modified it, and giving a relevant date.\n\n    b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7.  This requirement modifies the requirement in section 4 to \"keep intact all notices\".\n\n    c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy.  This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged.  This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it.\n\n    d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so.\n\nA compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an \"aggregate\" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit.  Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate.\n\n6. Conveying Non-Source Forms.\nYou may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:\n\n    a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange.\n\n    b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge.\n\n    c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source.  This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.\n\n    d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge.  You need not require recipients to copy the Corresponding Source along with the object code.  If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source.  Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.\n\n    e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d.\n\nA separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work.\n\nA \"User Product\" is either (1) a \"consumer product\", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling.  In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage.  For a particular product received by a particular user, \"normally used\" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product.  A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product.\n\n\"Installation Information\" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source.  The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.\n\nIf you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information.  But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM).\n\nThe requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed.  Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network.\n\nCorresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying.\n\n7. Additional Terms.\n\"Additional permissions\" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law.  If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions.\n\nWhen you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it.  (Additional permissions may be written to require their own removal in certain cases when you modify the work.)  You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission.\n\nNotwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms:\n\n    a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or\n\n    b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or\n\n    c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or\n\n    d) Limiting the use for publicity purposes of names of licensors or authors of the material; or\n\n    e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or\n\n    f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors.\n\nAll other non-permissive additional terms are considered \"further restrictions\" within the meaning of section 10.  If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term.  If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying.\n\nIf you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms.\n\nAdditional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way.\n\n8. Termination.\n\nYou may not propagate or modify a covered work except as expressly provided under this License.  Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11).\n\nHowever, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.\n\nMoreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.\n\nTermination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License.  If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10.\n\n9. Acceptance Not Required for Having Copies.\n\nYou are not required to accept this License in order to receive or run a copy of the Program.  Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance.  However, nothing other than this License grants you permission to propagate or modify any covered work.  These actions infringe copyright if you do not accept this License.  Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so.\n\n10. Automatic Licensing of Downstream Recipients.\n\nEach time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License.  You are not responsible for enforcing compliance by third parties with this License.\n\nAn \"entity transaction\" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations.  If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts.\n\nYou may not impose any further restrictions on the exercise of the rights granted or affirmed under this License.  For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it.\n\n11. Patents.\n\nA \"contributor\" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based.  The work thus licensed is called the contributor's \"contributor version\".\n\nA contributor's \"essential patent claims\" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version.  For purposes of this definition, \"control\" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License.\n\nEach contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version.\n\nIn the following three paragraphs, a \"patent license\" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement).  To \"grant\" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party.\n\nIf you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent\nlicense to downstream recipients.  \"Knowingly relying\" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid.\n\nIf, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it.\n\nA patent license is \"discriminatory\" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License.  You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007.\n\nNothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law.\n\n12. No Surrender of Others' Freedom.\n\nIf conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License.  If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may\nnot convey it at all.  For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program.\n\n13. Remote Network Interaction; Use with the GNU General Public License.\n\nNotwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software.  This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph.\n\nNotwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work.  The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License.\n\n14. Revised Versions of this License.\n\nThe Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time.  Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.\n\nEach version is given a distinguishing version number.  If the Program specifies that a certain numbered version of the GNU Affero General Public License \"or any later version\" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation.  If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation.\n\nIf the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program.\n\nLater license versions may give you additional or different permissions.  However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version.\n\n15. Disclaimer of Warranty.\n\nTHERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM \"AS IS\" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.\n\n16. Limitation of Liability.\n\nIN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.\n\n17. Interpretation of Sections 15 and 16.\n\nIf the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee.\n\nEND OF TERMS AND CONDITIONS\n\n            How to Apply These Terms to Your New Programs\n\nIf you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.\n\nTo do so, attach the following notices to the program.  It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the \"copyright\" line and a pointer to where the full notice is found.\n\n     <one line to give the program's name and a brief idea of what it does.>\n     Copyright (C) <year>  <name of author>\n\n     This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.\n\n     This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more details.\n\n     You should have received a copy of the GNU Affero General Public License along with this program.  If not, see <http://www.gnu.org/licenses/>.\n\nAlso add information on how to contact you by electronic and paper mail.\n\nIf your software can interact with users remotely through a computer network, you should also make sure that it provides a way for users to get its source.  For example, if your program is a web application, its interface could display a \"Source\" link that leads users to an archive of the code.  There are many ways you could offer source, and different solutions will be better for different programs; see section 13 for the specific requirements.\n\nYou should also get your employer (if you work as a programmer) or school, if any, to sign a \"copyright disclaimer\" for the program, if necessary. For more information on this, and how to apply and follow the GNU AGPL, see <http://www.gnu.org/licenses/>.\n"
  },
  {
    "path": "LICENSES/LicenseRef-SEL.txt",
    "content": "Stalwart Enterprise License 1.0 (SELv1) Agreement\n=================================================\n\nLast Update: April 29, 2025\n\nPLEASE CAREFULLY READ THIS STALWART ENTERPRISE LICENSE AGREEMENT (\"AGREEMENT\"). THIS AGREEMENT CONSTITUTES A LEGALLY BINDING AGREEMENT BETWEEN YOU AND Stalwart Labs LLC AND GOVERNS YOUR USE OF THE SOFTWARE (DEFINED BELOW). IF YOU DO NOT AGREE WITH THIS AGREEMENT, YOU MAY NOT USE THE SOFTWARE. IF YOU ARE USING THE SOFTWARE ON BEHALF OF A LEGAL ENTITY, YOU REPRESENT AND WARRANT THAT YOU HAVE AUTHORITY TO AGREE TO THIS AGREEMENT ON BEHALF OF SUCH ENTITY. IF YOU DO NOT HAVE SUCH AUTHORITY, DO NOT USE THE SOFTWARE IN ANY MANNER.\n\nThis Agreement is entered into by and between Stalwart Labs LLC and you, or the legal entity on behalf of whom you are acting.\n\n1. DEFINITIONS\n\n1.1. \"Software\" refers to the Stalwart Server Enterprise Edition software, including all its versions, updates, modifications, accompanying documentation, and related materials.\n1.2. \"Subscription\" refers to the paid access to the Software provided by Licensor to Licensee.\n1.3. \"Licensor\" refers to Stalwart Labs LLC, the entity providing the Software.\n1.4. \"Licensee\" refers to the individual or entity installing, accessing, or using the Software with a valid Subscription.\n1.5. \"License Key\" refers to the unique code provided by Licensor upon purchasing a Subscription which activates the full features of the Software.\n1.6. \"Source Code\" refers to the human-readable version of the Software's code, as opposed to the compiled machine-readable version.\n\n2. GRANT OF LICENSE\n\n2.1. Licensor grants Licensee a revocable, non-exclusive, non-transferable, non-sublicensable, limited license to download, install, and use the Software.\n2.2. The use of the Software is conditioned upon Licensee maintaining an active and valid paid subscription with Licensor. The paid subscription covers all versions of the Software and all updates and modifications.\n2.3. This license grants Licensee the right to use the Software for both personal and commercial purposes. However, Licensee is expressly prohibited from reselling, leasing, sublicensing, or otherwise redistributing the Software itself.\n2.4. This license is further governed by the terms and conditions set forth in any licensing agreements separately executed between Licensor and Licensee. In the event of any conflict between the terms of this Agreement and the terms of a signed licensing agreement, the terms of the signed licensing agreement shall control.\n2.5. You are not granted any other rights beyond what is expressly stated herein.\n\n3. LICENSE KEYS\n\n3.1. The Software shall not be used without a valid License Key issued by Licensor.\n3.2. Licensee is required to use valid License Keys issued by Licensor to run the Software, including any modified versions. Any attempts to bypass the License Key requirement is a violation of this Agreement.\n3.3. Distribution or sharing of License Keys to third parties, not associated with Licensee, is strictly prohibited.\n3.4. License Keys are bound to the subscription period. Should your subscription expire, all License Keys will become invalid after 15 days from the subscription expiration date.\n3.5. Any instance of the Software using such an expired key will revert to the Community Edition functionality after the aforementioned 15-day period.\n\n4. SOURCE CODE USAGE\n\n4.1. Licensee is permitted to view, copy, and modify the Software's Source Code, as made available by Licensor, solely for Licensee's internal business use and in compliance with this Agreement's terms.\n4.2. Any modifications to the Source Code do not grant Licensee any ownership rights to the original Software or any modifications. All rights, title, and interest to the Software and its Source Code remain exclusively with Licensor.\n4.3. Licensee is strictly prohibited from altering, removing, or in any way tampering with the License Key validation system within the Software. Any such unauthorized modifications will be considered a material breach of this Agreement and may result in legal action.\n4.3. Notwithstanding the availability of the Software's Source Code for review and limited modification, the Software and its Source Code are not open source and remain proprietary to Licensor. The provision of access to the Source Code does not confer any rights typically associated with open source software, including but not limited to the right to freely sublicense, or create derivative works for public distribution. All rights not expressly granted herein are reserved by Licensor.\n4.4. Notwithstanding the foregoing, you may copy the Source Code for development and testing purposes, without requiring a Subscription.\n\n5. INTELLECTUAL PROPERTY RIGHTS\n\n5.1. The Licensor retains all rights, title, and interest in and to the Software, including all intellectual property rights therein. This Agreement does not transfer any ownership rights to the Licensee.\n5.2. The Licensee must not remove, alter, or obscure any proprietary notices (including copyright and trademark notices) on the Software.\n\n6. TERMINATION\n\n6.1. Licensor reserves the right to terminate this Agreement immediately if the Licensee fails to comply with any terms and conditions of this Agreement.\n6.2. In the event of a termination, you will be provided with a written notice, sent to the email address used during your subscription to the Software, outlining the reasons for the termination.\n6.3. Upon termination, all rights granted to you under this Agreement will cease, and you must promptly cease all use of the Software.\n\n7. LIMITATION OF LIABILITY\n\n7.1. In no event will the Licensor be liable for any indirect, incidental, special, consequential, or punitive damages, or any loss of profits or revenues, whether incurred directly or indirectly, or any loss of data, use, goodwill, or other intangible losses, resulting from (i) your use or inability to use the Software; (ii) any unauthorized access to or use of our servers and/or any personal information stored therein.\n7.2. Except for liability arising from death or personal injury caused by negligence, fraud, or willful misconduct, Licensor's total aggregate liability for any and all claims under this Agreement shall be limited to the total Subscription fees paid by Licensee to Licensor in the twelve (12) months immediately preceding the event giving rise to the claim.\n\n8. GOVERNING LAW & JURISDICTION\n\nThis Agreement shall be governed by and construed under the laws of the United Kingdom. Any disputes arising from or related to this Agreement shall be resolved in the jurisdiction of London, UK.\n\n9. DATA PROTECTION & PRIVACY\n\nBy using the Software, you consent to the collection, processing, and use of any personal data as required for the functionality of the Software. The specifics of data handling and storage will be outlined in the company's Privacy Policy, which can be accessed on the company's website.\n\n10. ACCEPTANCE\n\nBy downloading, installing, or using the Software software, even without explicitly clicking on an \"I Agree\" button or a similar mechanism, you acknowledge that you have read, understood, and agreed to be bound by the terms and conditions of this Agreement.\n\n11. ASSIGNMENT\n\nThis Agreement and the rights granted hereunder may not be transferred or assigned by you but may be assigned by Licensor without restriction.\n\n12. SEVERABILITY\n\nIf any provision of this Agreement is held to be unenforceable or invalid for any reason, that provision shall be reformed to the extent necessary to make it enforceable and consistent with the intent of the parties, and the remaining provisions shall remain in full force and effect.\n\n13. ENTIRE AGREEMENT\n\nThis Agreement constitutes the entire agreement between the Licensor and the Licensee with respect to the subject matter hereof and supersedes all prior or contemporaneous understandings regarding such subject matter. No amendment to or modification of this Agreement will be binding unless in writing and signed by the Licensor.\n\n14. DISCLAIMERS AND WARRANTIES\n\nThe Software is provided \"AS IS\" and \"AS AVAILABLE\", without warranty of any kind, either express or implied, including, without limitation, warranties of merchantability, fitness for a particular purpose, and non-infringement. Licensor does not warrant that the Software will be error-free, that access thereto will be uninterrupted, or that defects will be corrected.\n\n15. INDEMNIFICATION\n\nLicensee agrees to indemnify, defend, and hold harmless Licensor, its officers, directors, employees, agents, licensors, suppliers, and any third-party information providers from and against all claims, losses, expenses, damages, and costs, including reasonable attorneys' fees, resulting from any violation of this Agreement or any activity related to your use or misuse of the Software (including negligent or wrongful conduct).\n\n16. FORCE MAJEURE\n\nNeither party shall be in default or otherwise liable for any delay in or failure of its performance under this Agreement if such delay or failure arises by any reason of any event beyond the reasonable control of a party, including acts of God, the elements, earthquakes, floods, fires, epidemics, riots, failures or delays in transportation or communications, or any act or failure to act by the other party or such other party’s officers, employees, agents, or contractors. The parties will promptly inform and consult with each other as to any of the above causes which, in their judgment, may or could be the cause of a delay in the performance of this Agreement.\n\n17. CONTACT INFORMATION\n\nIf you have any questions about this Agreement, please contact Stalwart Labs LLC at:\n\nStalwart Labs LLC\n1309 Coffeen Avenue STE 1200\nSheridan, Wyoming 82801\nUSA\nhello@stalw.art\n"
  },
  {
    "path": "README.md",
    "content": "<p align=\"center\">\n    <a href=\"https://stalw.art\">\n    <img src=\"./img/logo-red.svg\" height=\"150\">\n    </a>\n</p>\n\n<h3 align=\"center\">\n  Secure, scalable mail & collaboration server with comprehensive protocol support 🛡️ <br/>(IMAP, JMAP, SMTP, CalDAV, CardDAV, WebDAV)\n</h3>\n\n<br>\n\n<p align=\"center\">\n  <a href=\"https://github.com/stalwartlabs/stalwart/actions/workflows/ci.yml\"><img src=\"https://img.shields.io/github/actions/workflow/status/stalwartlabs/stalwart/ci.yml?style=flat-square\" alt=\"continuous integration\"></a>\n  &nbsp;\n  <a href=\"https://www.gnu.org/licenses/agpl-3.0\"><img src=\"https://img.shields.io/badge/License-AGPL_v3-blue.svg?label=license&style=flat-square\" alt=\"License: AGPL v3\"></a>\n  &nbsp;\n  <a href=\"https://stalw.art/docs/install/get-started\"><img src=\"https://img.shields.io/badge/read_the-docs-red?style=flat-square\" alt=\"Documentation\"></a>\n</p>\n<p align=\"center\">\n  <a href=\"https://mastodon.social/@stalwartlabs\"><img src=\"https://img.shields.io/mastodon/follow/109929667531941122?style=flat-square&logo=mastodon&color=%236364ff&label=Follow%20on%20Mastodon\" alt=\"Mastodon\"></a>\n  &nbsp;\n  <a href=\"https://twitter.com/stalwartlabs\"><img src=\"https://img.shields.io/twitter/follow/stalwartlabs?style=flat-square&logo=x&label=Follow%20on%20Twitter\" alt=\"Twitter\"></a>\n</p>\n<p align=\"center\">\n  <a href=\"https://discord.com/servers/stalwart-923615863037390889\"><img src=\"https://img.shields.io/discord/923615863037390889?label=Join%20Discord&logo=discord&style=flat-square\" alt=\"Discord\"></a>\n  &nbsp;\n  <a href=\"https://www.reddit.com/r/stalwartlabs/\"><img src=\"https://img.shields.io/reddit/subreddit-subscribers/stalwartlabs?label=Join%20%2Fr%2Fstalwartlabs&logo=reddit&style=flat-square\" alt=\"Reddit\"></a>\n</p>\n\n## Features\n\n**Stalwart** is an open-source mail & collaboration server with JMAP, IMAP4, POP3, SMTP, CalDAV, CardDAV and WebDAV support and a wide range of modern features. It is written in Rust and designed to be secure, fast, robust and scalable.\n\nKey features:\n\n- **Email** server with complete protocol support:\n  - JMAP: \n    * [JMAP for Mail](https://datatracker.ietf.org/doc/html/rfc8621) server.\n    * [JMAP for Sieve Scripts](https://www.ietf.org/archive/id/draft-ietf-jmap-sieve-22.html).\n    * [WebSocket](https://datatracker.ietf.org/doc/html/rfc8887), [Blob Management](https://www.rfc-editor.org/rfc/rfc9404.html) and [Quotas](https://www.rfc-editor.org/rfc/rfc9425.html) extensions.\n  - IMAP:\n    * [IMAP4rev2](https://datatracker.ietf.org/doc/html/rfc9051) and [IMAP4rev1](https://datatracker.ietf.org/doc/html/rfc3501) server.\n    * [ManageSieve](https://datatracker.ietf.org/doc/html/rfc5804) server.\n    * Numerous [extensions](https://stalw.art/docs/development/rfcs#imap4-and-extensions) supported.\n  - POP3:\n    - [POP3](https://datatracker.ietf.org/doc/html/rfc1939) server.\n    - [STLS](https://datatracker.ietf.org/doc/html/rfc2595) and [SASL](https://datatracker.ietf.org/doc/html/rfc5034) support as well as other [extensions](https://datatracker.ietf.org/doc/html/rfc2449).\n  - SMTP:\n    * SMTP server with built-in [DMARC](https://datatracker.ietf.org/doc/html/rfc7489), [DKIM](https://datatracker.ietf.org/doc/html/rfc6376), [SPF](https://datatracker.ietf.org/doc/html/rfc7208) and [ARC](https://datatracker.ietf.org/doc/html/rfc8617) support for message authentication.\n    * Strong transport security through [DANE](https://datatracker.ietf.org/doc/html/rfc6698), [MTA-STS](https://datatracker.ietf.org/doc/html/rfc8461) and [SMTP TLS](https://datatracker.ietf.org/doc/html/rfc8460) reporting.\n    * Inbound throttling and filtering with granular configuration rules, sieve scripting, MTA hooks and milter integration.\n    * Distributed virtual queues with delayed delivery, priority delivery, quotas, routing rules and throttling support.\n    * Envelope rewriting and message modification.\n- **Collaboration** server:\n  - Calendaring and scheduling:\n    - [CalDAV](https://datatracker.ietf.org/doc/html/rfc4791) and [CalDAV Scheduling](https://datatracker.ietf.org/doc/html/rfc6638) support.\n    - [JMAP for Calendars](https://datatracker.ietf.org/doc/html/draft-ietf-jmap-calendars-24) support.\n  - Contact management:\n    - [CardDAV](https://datatracker.ietf.org/doc/html/rfc6352) support.\n    - [JMAP for Contacts](https://datatracker.ietf.org/doc/html/rfc9610) support.\n  - File storage:\n    - [WebDAV](https://datatracker.ietf.org/doc/html/rfc4918) support.\n    - [JMAP for File Storage](https://datatracker.ietf.org/doc/html/draft-ietf-jmap-filenode-03) support.\n  - Sharing with fine-grained access controls:\n    - [WebDAV ACL](https://datatracker.ietf.org/doc/html/rfc3744) support.\n    - [JMAP Sharing](https://datatracker.ietf.org/doc/html/rfc9670) support.\n- **Spam** and **Phishing** built-in filter:\n  - Comprehensive set of filtering **rules** on par with popular solutions.\n  - LLM-driven spam filtering and message analysis.\n  - Statistical **spam classifier** with collaborative filtering, automatic training capabilities and address book integration.\n  - DNS Blocklists (**DNSBLs**) checking of IP addresses, domains, and hashes.\n  - Collaborative digest-based spam filtering with **Pyzor**.\n  - **Phishing** protection against homographic URL attacks, sender spoofing and other techniques.\n  - Trusted **reply** tracking to recognize and prioritize genuine e-mail replies.\n  - Sender **reputation** monitoring by IP address, ASN, domain and email address.\n  - **Greylisting** to temporarily defer unknown senders.\n  - **Spam traps** to set up decoy email addresses that catch and analyze spam.\n- **Flexible**:\n  - Pluggable storage backends with **RocksDB**, **FoundationDB**, **PostgreSQL**, **mySQL**, **SQLite**, **S3-Compatible**, **Azure** and **Redis** support.\n  - Full-text search available in 17 languages using the built-in search engine or via **Meilisearch**, **ElasticSearch**, **OpenSearch**, **PostgreSQL** or **mySQL** backends.\n  - Sieve scripting language with support for all [registered extensions](https://www.iana.org/assignments/sieve-extensions/sieve-extensions.xhtml).\n  - Email aliases, mailing lists, subaddressing and catch-all addresses support.\n  - Automatic account configuration and discovery with [autoconfig](https://www.ietf.org/id/draft-bucksch-autoconfig-02.html) and [autodiscover](https://learn.microsoft.com/en-us/exchange/architecture/client-access/autodiscover?view=exchserver-2019). \n  - Multi-tenancy support with domain and tenant isolation.\n  - Disk quotas per user and tenant.\n- **Secure and robust**:\n  - Encryption at rest with **S/MIME** or **OpenPGP**.\n  - Automatic TLS certificate provisioning with [ACME](https://datatracker.ietf.org/doc/html/rfc8555) using `TLS-ALPN-01`, `DNS-01` or `HTTP-01` challenges.\n  - Automated blocking of IP addresses that attack, abuse or scan the server for exploits.\n  - Rate limiting.\n  - Security audited (read the [report](https://stalw.art/blog/security-audit)).\n  - Memory safe (thanks to Rust).\n- **Scalable and fault-tolerant**:\n  - Designed to handle growth seamlessly, from small setups to large-scale deployments of thousands of nodes.\n  - Built with **fault tolerance** and **high availability** in mind, recovers from hardware or software failures with minimal operational impact. \n  - Peer-to-peer cluster coordination or with **Kafka**, **Redpanda**, **NATS** or **Redis**.\n  - **Kubernetes**, **Apache Mesos** and **Docker Swarm** support for automated scaling and container orchestration.\n  - Read replicas, sharded blob storage and in-memory data stores for high performance and low latency.\n- **Authentication and Authorization**:\n  - **OpenID Connect** authentication.\n  - OAuth 2.0 authorization with [authorization code](https://www.rfc-editor.org/rfc/rfc8628) and [device authorization](https://www.rfc-editor.org/rfc/rfc8628) flows.\n  - **LDAP**, **OIDC**, **SQL** or built-in authentication backend support.\n  - Two-factor authentication with Time-based One-Time Passwords (`2FA-TOTP`) \n  - Application passwords (App Passwords).\n  - Roles and permissions.\n  - Access Control Lists (ACLs).\n- **Observability**:\n  - Logging and tracing with **OpenTelemetry**, journald, log files and console support.\n  - Metrics with **OpenTelemetry** and **Prometheus** integration.\n  - Webhooks for event-driven automation.\n  - Alerts with email and webhook notifications.\n  - Live tracing and metrics.\n- **Web-based administration**:\n  - Dashboard with real-time statistics and monitoring.\n  - Account, domain, group and mailing list management.\n  - SMTP queue management for messages and outbound DMARC and TLS reports.\n  - Report visualization interface for received DMARC, TLS-RPT and Failure (ARF) reports.\n  - Configuration of every aspect of the mail server.\n  - Log viewer with search and filtering capabilities.\n  - Self-service portal for password reset and encryption-at-rest key management.\n\n## Screenshots\n\n<img src=\"./img/screencast-setup.gif\">\n\n## Presentation\n\n**Want a deeper dive?** Need to explain to your boss why Stalwart is the perfect fit? Whether you're evaluating options, making a case to your team, or simply curious about how it all works under the hood, these slides walk you through the key features, architecture, and benefits of Stalwart. Browse the [slides](https://stalw.art/slides) to see what makes it stand out.\n\n## Get Started\n\nInstall Stalwart on your server by following the instructions for your platform:\n\n- [Linux / MacOS](https://stalw.art/docs/install/platform/linux)\n- [Windows](https://stalw.art/docs/install/platform/windows)\n- [Docker](https://stalw.art/docs/install/platform/docker)\n\nAll documentation is available at [stalw.art/docs](https://stalw.art/docs/install/get-started).\n\n## Support\n\nIf you are having problems running Stalwart, you found a bug or just have a question, do not hesitate to reach us on [GitHub Discussions](https://github.com/stalwartlabs/stalwart/discussions), [Reddit](https://www.reddit.com/r/stalwartlabs) or [Discord](https://discord.com/servers/stalwart-923615863037390889).\nAdditionally you may purchase an [Enterprise License](https://stalw.art/enterprise) to obtain priority support from Stalwart Labs LLC.\n\n## Roadmap\n\nStalwart has reached an exciting point in its journey, it’s now **feature complete**. All the core functionality and open standard email and collaboration protocols that we set out to support are in place. In other words, Stalwart already does everything you’d expect from a modern, standards-compliant mail and collaboration platform.\n\nThe next major milestone is all about refinement: finalizing the database schema and focusing on performance optimizations to ensure everything runs as efficiently and reliably as possible. Once that’s done, we’ll be ready to roll out version **1.0**.\n\nOf course, development doesn’t stop there. The community has contributed hundreds of great ideas for improvements and new features, everything from subtle usability tweaks to entirely new integrations. You can see the full list of proposals over on our [GitHub issues](https://github.com/stalwartlabs/stalwart/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc+label%3Aenhancement). If there’s something you’d like to see prioritized, just give it a thumbs up as we plan to implement enhancements based on the community’s votes.\n\n## Sponsorship\n\nYour support is crucial in helping us continue to improve the project, add new features, and maintain the highest level of quality. By [becoming a sponsor](https://opencollective.com/stalwart), you help fund the development and future of Stalwart. As a thank-you, sponsors who contribute $5 per month or more will automatically receive a [Enterprise edition](https://stalw.art/enterprise/) license. And, sponsors who contribute $30 per month or more, also have access to [Premium Support](https://stalw.art/support) from Stalwart Labs.\n\n## Funding\n\nPart of the development of this project was funded through:\n\n- [NGI0 Entrust Fund](https://nlnet.nl/entrust), a fund established by [NLnet](https://nlnet.nl/) with financial support from the European Commission's [Next Generation Internet](https://ngi.eu/) programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101069594.\n- [NGI Zero Core](https://nlnet.nl/NGI0/), a fund established by [NLnet](https://nlnet.nl/) with financial support from the European Commission's programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101092990.\n\nIf you find the project useful you can help by [becoming a sponsor](https://opencollective.com/stalwart). Thank you!\n\n## License\n\nThis project is dual-licensed under the **GNU Affero General Public License v3.0** (AGPL-3.0; as published by the Free Software Foundation) and the **Stalwart Enterprise License v1 (SELv1)**:\n\n- The [GNU Affero General Public License v3.0](./LICENSES/AGPL-3.0-only.txt) is a free software license that ensures your freedom to use, modify, and distribute the software, with the condition that any modified versions of the software must also be distributed under the same license. \n- The [Stalwart Enterprise License v1 (SELv1)](./LICENSES/LicenseRef-SEL.txt) is a proprietary license designed for commercial use. It offers additional features and greater flexibility for businesses that do not wish to comply with the AGPL-3.0 license requirements. \n\nEach file in this project contains a license notice at the top, indicating the applicable license(s). The license notice follows the [REUSE guidelines](https://reuse.software/) to ensure clarity and consistency. The full text of each license is available in the [LICENSES](./LICENSES/) directory.\n\n## Copyright\n\nCopyright (C) 2020, Stalwart Labs LLC\n"
  },
  {
    "path": "SECURITY.md",
    "content": "# Security Policy for Stalwart\n\n## Supported Versions\n\nWe provide security updates for the following versions of Stalwart:\n\n| Version | Supported          | End of Support |\n| ------- | ------------------ | -------------- |\n| 0.15.x  | :white_check_mark: | TBD            |\n| 0.14.x  | :white_check_mark: | 2026-06-08     |\n| 0.13.x  | :white_check_mark: | 2026-03-31     |\n| < 0.13  | :x:                | Ended          |\n\n**Note**: We typically support the current major version and one previous major version. Users are strongly encouraged to upgrade to the latest version for the best security posture.\n\n## Reporting a Vulnerability\n\nWe take the security of Stalwart very seriously. If you believe you've found a security vulnerability, we encourage you to inform us responsibly through coordinated disclosure.\n\n### How to Report\n\n**Do not report security vulnerabilities through public GitHub issues, discussions, or social media.**\n\nInstead, please use one of these secure channels:\n\n1. **Email** (preferred): Send details to `security@stalw.art`\n2. **GitHub Security Advisories**: Use the \"Report a vulnerability\" button in the Security tab\n3. **Backup contact**: If no response within 48 hours, email `hello@stalw.art`\n\n### What to Include\n\nTo help us understand and address the issue quickly, please include:\n\n**Required Information:**\n- Brief description of the vulnerability type\n- Affected version(s) and components\n- Steps to reproduce the issue\n- Impact assessment (what could an attacker achieve?)\n\n**Helpful Additional Details:**\n- Full paths of affected source files\n- Specific commit/branch where the issue exists\n- Required configuration to reproduce\n- Proof-of-concept code (if available)\n- Suggested mitigation or fix (if you have ideas)\n\n### Our Response Process\n\n**Timeline Commitments:**\n- **Initial acknowledgment**: Within 24 hours\n- **Detailed response**: Within 72 hours\n- **Status updates**: Every 7 days until resolved\n- **Resolution target**: 90 days for most issues\n\n**What We'll Do:**\n1. Acknowledge your report and assign a tracking ID\n2. Assess the vulnerability and determine severity\n3. Develop and test a fix\n4. Coordinate disclosure timeline with you\n5. Release security update and publish advisory\n6. Credit you in our security advisory (if desired)\n\n## Disclosure Policy\n\nWe follow responsible disclosure principles:\n\n- **Coordinated disclosure**: We'll work with you to determine appropriate disclosure timing\n- **Typical timeline**: 90 days from report to public disclosure\n- **Early disclosure**: May occur if issue is being actively exploited\n- **Delayed disclosure**: May be necessary for complex issues requiring significant changes\n\n## Scope\n\nThis security policy applies to:\n\n**In Scope:**\n- Stalwart (all supported versions)\n- Official Docker images\n- Documentation that could lead to insecure configurations\n- Dependencies with security implications\n\n**Out of Scope:**\n- Third-party integrations or plugins\n- Issues requiring physical access to the server\n- Social engineering attacks\n- Attacks requiring compromised credentials (unless the vulnerability enables credential compromise)\n- Theoretical vulnerabilities without practical exploitation\n\n## Security Measures\n\n**Our Commitments:**\n- Regular security audits of dependencies using `cargo audit`\n- Automated security scanning in CI/CD pipeline\n- Following Rust security best practices\n- Prompt security updates for critical dependencies\n- Security-focused code review process\n\n**User Responsibilities:**\n- Keep Stalwart updated to supported versions\n- Follow security configuration guidelines\n- Implement proper network security (firewalls, TLS, etc.)\n- Regular security monitoring and logging\n- Secure credential management\n\n## Legal Safe Harbor\n\nWe support security research conducted in good faith. If you follow these guidelines:\n\n**We will NOT:**\n- Initiate legal action against you\n- Contact law enforcement about your research\n- Suspend or terminate your access to Stalwart services\n\n**You must:**\n- Only test against your own Stalwart installations\n- Not access, modify, or delete user data\n- Not perform testing that could degrade service availability\n- Not publicly disclose the issue before coordinated disclosure\n- Act in good faith and not for malicious purposes\n\n## Recognition\n\nWe believe in recognizing security researchers who help keep Stalwart secure:\n\n- **Security Advisory Credits**: We'll credit you in our GitHub Security Advisories (unless you prefer to remain anonymous)\n- **Hall of Fame**: Significant contributors may be listed in our security acknowledgments\n- **Swag**: We may send Stalwart merchandise for notable contributions\n\n## Security Updates\n\n**Stay Informed:**\n- Subscribe to our [GitHub releases](https://github.com/stalwartlabs/stalwart/releases) for security updates\n- Join our community channels for security announcements\n- Enable GitHub notifications for security advisories\n\n**Update Process:**\n- Security updates are published as patch releases (e.g., 0.12.1 → 0.12.2)\n- Critical vulnerabilities may receive out-of-band releases\n- Docker images are updated simultaneously with releases\n- Security advisories are published through GitHub Security Advisories\n\n## Contact Information\n\n- **Security reports**: security@stalw.art\n- **General inquiries**: hello@stalw.art\n- **PGP Key**: Available upon request for sensitive communications\n\n## Additional Resources\n\n- [Stalwart Security Incident Response Process](SECURITY_PROCESS.md)\n- [Security Configuration Guide](https://stalw.art/docs/install/security)\n- [Rust Security Advisory Database](https://rustsec.org/)\n\n*This security policy is effective as of June 20, 2025 and may be updated periodically. Check back regularly for updates.*\n\n"
  },
  {
    "path": "SECURITY_PROCESS.md",
    "content": "# Stalwart Security Incident Response Checklist\n\n## Phase 1 : Initial Assessment & Validation\n\n### Updates\n\n<< Use this section to detail the report received, initial assessment, and validation results >>\n\nExample:\n\nI've reviewed the security report and confirmed this vulnerability exists in Stalwart version X.Y.Z.\n\nAssessment of exploitability:\n\n- Attack complexity: [High/Medium/Low]\n- Prerequisites: [Authentication required/Network access/Specific configuration/etc.]\n- User interaction required: [Yes/No]\n\nPotential impact:\n- Email data confidentiality: [At risk/Not affected]\n- Server integrity: [At risk/Not affected] \n- Service availability: [At risk/Not affected]\n- Estimated affected installations: [Number/Percentage]\n\n### Resources\n\n- [Stalwart Security Policy](https://github.com/stalwartlabs/stalwart/blob/main/SECURITY.md)\n- [CVE Scoring Calculator](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator)\n- [Rust Security Advisory Database](https://rustsec.org/)\n\n### Tasks\n\n- [ ] Reproduce the vulnerability in test environment\n- [ ] Assess CVSS score and severity level\n- [ ] Check if vulnerability affects current stable version\n- [ ] Check if vulnerability affects LTS versions (if applicable)\n- [ ] Determine if this requires immediate action or can wait for next release cycle\n- [ ] Document technical details and root cause\n\n### Assessment Summary\n\n- **Severity Level**: `Critical|High|Medium|Low`\n- **CVSS Score**: `X.X`\n- **Affects versions**: `X.Y.Z to X.Y.Z`\n- **Root cause**: Brief technical explanation\n- **Introduced in commit/version**: `commit-hash` or `vX.Y.Z`\n- **Attack vector**: `Network|Local|Physical`\n- **Estimated timeline for fix**: `X days/weeks`\n\n## Phase 2: Immediate Response & Mitigation\n\n### Updates\n\n<< Document immediate actions taken and mitigation strategies >>\n\nExample:\n\nWorking on hotfix for version X.Y.Z. Temporary workaround available by disabling [feature] in configuration.\n\n### Tasks\n\n- [ ] Implement immediate workaround if possible\n- [ ] Update security advisory draft\n- [ ] Prepare patch/hotfix\n- [ ] Test fix thoroughly in development environment\n- [ ] Prepare updated Docker images and binaries\n- [ ] Draft security advisory for GitHub Security Advisories\n- [ ] Consider if coordinated disclosure timeline needs adjustment\n\n### Mitigation Details\n\n- **Workaround available**: `Yes|No` - If yes, describe briefly\n- **Fix implemented on**: `YYYY-MM-DD`\n- **Patch/hotfix version**: `vX.Y.Z`\n- **GitHub Security Advisory ID**: `GHSA-XXXX-XXXX-XXXX`\n\n## Phase 3: Impact Assessment & User Analysis\n\n### Updates\n\n<< Analysis of potential impact on the Stalwart deployments >>\n\nBased on telemetry data and version statistics, approximately X installations may be affected.\n\n### Tasks\n\n- [ ] Analyze version adoption from update checks (if available)\n- [ ] Estimate number of vulnerable installations\n- [ ] Assess if default configurations are vulnerable\n- [ ] Review if vulnerability has been exploited (check logs, reports)\n- [ ] Determine if any user data may have been compromised\n- [ ] Check for indicators of active exploitation in the wild\n\n### Analysis Notes\n\n_Document your impact assessment process and findings_\n\n### Impact Summary\n\n- **Estimated vulnerable installations**: `~X out of Y`\n- **Default configuration vulnerable**: `Yes|No`\n- **Evidence of exploitation**: `Found|Not found|Unknown`\n- **User data potentially at risk**: `Email content|Credentials|Configuration|None`\n- **Confidence in assessment**: `High|Medium|Low`\n\n## Phase 4: Communication & Release\n\n### Updates\n\n<< Communication strategy and release timeline >>\n\nSecurity release vX.Y.Z will be published on YYYY-MM-DD with coordinated disclosure.\n\n### Tasks\n\n**Pre-release preparation:**\n\n- [ ] Finalize security patch\n- [ ] Prepare release notes with security details\n- [ ] Update documentation if needed\n- [ ] Test automated update mechanisms\n- [ ] Prepare GitHub Security Advisory\n\n**Communication channels:**\n\n- [ ] Draft announcement for Stalwart community forum/Discord\n- [ ] Prepare release announcement for GitHub\n- [ ] Draft security advisory content\n- [ ] Consider notification to major distributors/packagers\n\n**Release execution:**\n\n- [ ] Publish patched version to GitHub releases\n- [ ] Update Docker images on Docker Hub\n- [ ] Publish GitHub Security Advisory\n- [ ] Post to community channels (Discord/forum)\n- [ ] Update project website/documentation\n- [ ] Submit CVE request if warranted (CVSS ≥ 4.0)\n\n**Post-release:**\n\n- [ ] Monitor community channels for questions\n- [ ] Track adoption of security update\n- [ ] Follow up on any additional reports\n- [ ] Document lessons learned\n\n### Communication Record\n\n- **Security release published**: `YYYY-MM-DD HH:MM UTC`\n- **GitHub Security Advisory**: `GHSA-XXXX-XXXX-XXXX`\n- **CVE ID** (if applicable): `CVE-YYYY-XXXXX`\n- **Community announcement**: [Link to forum/Discord post]\n- **Estimated time to 50% adoption**: `X days/weeks`\n\n## Post-Incident Review\n\n### What went well?\n- \n\n### What could be improved?\n- \n\n### Action items for future incidents:\n- [ ] \n- [ ] \n- [ ] \n\n### Process improvements:\n- [ ] \n- [ ] \n\n## Emergency Contacts\n- **Primary maintainer**: hello@stalw.art\n"
  },
  {
    "path": "SECURITY_TEMPLATE.md",
    "content": "# Stalwart Security Advisory\n\n**CVE ID:** CVE-YYYY-NNNNN\n**Publication Date:** YYYY-MM-DD  \n**Last Updated:** YYYY-MM-DD  \n\n## Summary\n\n[Provide a brief, non-technical summary of the vulnerability in 1-2 sentences]\n\n## Affected Products and Versions\n\n**Product:** Stalwart Mail and Collaboration Server\n\n**Affected Versions:**\n- Version X.X.X through Y.Y.Y\n- [List specific affected version ranges]\n\n**Fixed Versions:**\n- Version Z.Z.Z and later\n- [List all versions that include the fix]\n\n## Vulnerability Details\n\n### Description\n\n[Detailed technical description of the vulnerability, including how it can be exploited]\n\n### Impact\n\n[Describe the potential impact if this vulnerability is exploited]\n\n### CVSS Score\n\n**CVSS v3.1 Base Score:** X.X ([SEVERITY])  \n**Vector String:** CVSS:3.1/AV:X/AC:X/PR:X/UI:X/S:X/C:X/I:X/A:X\n\n**Severity Breakdown:**\n- **Attack Vector:** [Network/Adjacent/Local/Physical]\n- **Attack Complexity:** [Low/High]\n- **Privileges Required:** [None/Low/High]\n- **User Interaction:** [None/Required]\n- **Scope:** [Unchanged/Changed]\n- **Confidentiality Impact:** [None/Low/High]\n- **Integrity Impact:** [None/Low/High]\n- **Availability Impact:** [None/Low/High]\n\n### CWE Classification\n\n**CWE-XXX:** [Weakness Name]\n\n## Technical Details\n\n### Root Cause\n\n[Explain the underlying cause of the vulnerability]\n\n### Attack Scenario\n\n[Describe a realistic attack scenario or proof of concept, without providing exploit code]\n\n### Prerequisites\n\n[List any conditions that must be met for successful exploitation]\n\n## Remediation\n\n### Recommended Actions\n\n1. **Immediate:** Upgrade to version Z.Z.Z or later\n2. **Short-term:** [Any temporary mitigation measures]\n3. **Long-term:** [Any additional security hardening recommendations]\n\n### Upgrade Instructions\n\n```bash\n# Example upgrade commands\n[Provide specific upgrade instructions for Stalwart]\n```\n\n### Workarounds\n\n[If applicable, describe any temporary workarounds for systems that cannot be immediately upgraded]\n\n**Note:** Workarounds are temporary measures and do not fully resolve the vulnerability. Upgrading is strongly recommended.\n\n## Detection\n\n### Indicators of Compromise\n\n[List any logs, patterns, or indicators that may suggest exploitation attempts]\n\n### Log Entries\n\n```\n[Example log entries that administrators should look for]\n```\n\n## Timeline\n\n- **YYYY-MM-DD:** Vulnerability discovered [by researcher/team name]\n- **YYYY-MM-DD:** Vendor notified\n- **YYYY-MM-DD:** Vendor acknowledged issue\n- **YYYY-MM-DD:** Fix developed and tested\n- **YYYY-MM-DD:** Fixed version released\n- **YYYY-MM-DD:** Public disclosure\n\n## Credits\n\nThis vulnerability was discovered by [Researcher Name / Organization].\n\n## References\n\n- Stalwart Mail Server: https://stalw.art/\n- CVE Entry: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-NNNNN\n- GitHub Advisory: [Link to GitHub Security Advisory if applicable]\n- Release Notes: [Link to release notes with fix]\n\n## Contact Information\n\nFor questions or concerns regarding this advisory, please contact:\n\n**Security Team:** hello@stalw.art  \n**Website:** https://stalw.art\n\nTo report security vulnerabilities in Stalwart, please follow our [responsible disclosure policy](https://github.com/stalwartlabs/stalwart/security/policy).\n\n## Disclaimer\n\nThis advisory is provided \"as is\" without warranty of any kind. The information contained in this advisory is subject to change without notice.\n\n---\n\n**Document Version:** 1.0  \n**Classification:** Public\n"
  },
  {
    "path": "UPGRADING/v0_04.md",
    "content": "# Upgrading from `v0.4.0` to `v0.4.x`\n\n- Replace the binary with the new version.\n- Restart the service.\n\n# Upgrading from `v0.3.x` to `v0.4.0`\n\n## What's changed\n\n- **Configuration File Split:** While the `config.toml` configuration file format hasn't changed much, the new version has divided it into multiple sub-files. These sub-files are now included from the new `config.toml`. This division was implemented because the config file had grown significantly, and splitting it improves organization.\n\n- **Changes in the Sieve Interpreter Attribute Names:** \n  - The configuration key prefix `jmap.sieve` (JMAP Sieve Interpreter) has been renamed to `sieve.untrusted`.\n  - The configuration key prefix `sieve` (SMTP Sieve Interpreter) has been renamed to `sieve.trusted`.\n\n## What's been added\n\n- **SPAM Filter Module:** The most notable addition in this version is the SPAM filter module. It comprises:\n  - A TOML configuration file located at `etc/smtp/spamfilter.toml`.\n  - A set of Sieve scripts in `etc/spamfilter/scripts`.\n  - Lookup maps in `etc/spamfilter/maps`.\n\n- **New Configuration Key:** A new key `resolver.public-suffix` has been added. This specifies the URL of the list of public suffixes.\n\n## Migration Steps\n\n1. **Backup:** Ensure you have a backup of your current `config.toml` file.\n2. **Download Configuration Bundle:** Fetch the new configuration bundle from [this link](https://get.stalw.art/resources/config.zip). Unpack it under `BASE_DIR/etc` (for example `/opt/stalwart-mail/etc`).\n3. **Update Configuration Files:** Modify the following files with your domain name, host name, certificate paths, DKIM signatures, and so on:\n   - `etc/config.toml`\n   - `etc/jmap/store.toml`\n   - `etc/jmap/oauth.toml`\n   - `etc/smtp/signature.toml`\n   - `etc/common/tls.toml`\n4. **Adjust included files:** If you are using an LDAP directory for authentication, edit `etc/config.toml` and replace the `etc/directory/sql.toml` include with `etc/directory/ldap.toml`.\n5. **Configure the SPAM Filter Database:** Set up and configure the SPAM filter database. More details can be found [here](https://stalw.art/docs/spamfilter/settings/database).\n6. **Review All TOML Files:** Navigate to every TOML file under the `etc/` directory and make necessary changes.\n7. **Update Binary:** Download and substitute the v0.4.0 binary suitable for your platform from [here](https://github.com/stalwartlabs/mail-server/releases/tag/v0.4.0).\n8. **Restart Service:** Conclude by restarting the Stalwart service.\n\n### Alternative Method:\n\n1. **Separate Installation:** Install v0.4.0 in a distinct directory. This will auto-update all configuration files and establish the spam filter database in SQLite format.\n2. **Move Configuration Files:** Transfer the configuration files from `etc/` and the SQLite spam filter database from `data/` to your current installation's directory.\n3. **Replace Binary:** Move the binary from the `bin/` directory to your current installation's `data/` directory.\n4. **Restart Service:** Finally, restart the Stalwart service.\n\n\nWe apologize for the lack of an automated migration tool for this upgrade. However, we are planning on introducing an automated migration tool in the near future. Thank you for your understanding and patience.\n"
  },
  {
    "path": "UPGRADING/v0_05.md",
    "content": "# Upgrading from `v0.5.2` to `v0.5.3`\n\n- The following configuration attributes have been renamed, see [store.toml](https://github.com/stalwartlabs/mail-server/blob/main/resources/config/common/store.toml) for an example:\n  - `jmap.store.data` -> `storage.data`\n  - `jmap.store.fts` -> `storage.fts`\n  - `jmap.store.blob` -> `storage.blob`\n  - `jmap.encryption.*` -> `storage.encryption.*`\n  - `jmap.spam.header` -> `storage.spam.header`\n  - `jmap.fts.default-language` -> `storage.fts.default-language`\n  - `jmap.cluster.node-id` -> `storage.cluster.node-id`\n  - `management.directory` and `sieve.trusted.default.directory` -> `storage.directory`\n  - `sieve.trusted.default.store` -> `storage.lookup`\n- Proxy networks are now configured under `server.proxy.trusted-networks` rather than `server.proxy-trusted-networks`. IP addresses/masks have to be defined within a set (`{}`) rather than a list (`[]`), see [server.toml](https://github.com/stalwartlabs/mail-server/blob/main/resources/config/common/server.toml) for an example.\n\n\n# Upgrading from `v0.5.1` to `v0.5.2`\n\n- Make sure that implicit TLS is enabled for the JMAP [listener](https://stalw.art/docs/server/listener) configured under `ets/jmap/listener.toml`:\n  ```toml\n  [server.listener.\"jmap\".tls]\n  implicit = true\n  ```\n- Optional: Enable automatic TLS with [ACME](https://stalw.art/docs/server/tls/acme).\n- Replace the binary with the new version.\n- Restart the service.\n\n# Upgrading from `v0.5.0` to `v0.5.1`\n\n- Replace the binary with the new version.\n- Restart the service.\n\n# Upgrading from `v0.4.x` to `v0.5.0`\n\n## What's changed\n\n- **Database Layout**: Version 0.5.0 utilizes a different database layout which is more efficient and allows multiple backends to be supported. For this reason, the database must be migrated to the new layout.\n- **Configuration file changes**: The configuration file has been updated to support multiple stores, most configuration attributes starting with `store.*` and `directory.*` need to be reviewed.\n- **SPAM filter**: Sieve scripts that interact with databases need to be updated. The functions `lookup` and `lookup_map` has been renamed to `key_exists` and `key_get`. It is recommended to replace all scripts with the new versions rather than updating them manually. Additionally, the SPAM database no longer requires an SQL server, it can now be stored in Redis or any of the supported databases.\n- **Directory superusers**: Due to problems and confusion with the `superuser-group` attribute, the concept of a superuser group has been removed. Instead, a new attribute `type` has been added to external directories. The value of this attribute can be `individual`, `group` or `admin`. The `admin` type is equivalent to the old superuser group. The `type` attribute is required for all principals in the directory, it defaults to `individual` if not specified.\n- **Purge schedules**: The attributes `jmap.purge.schedule.db` and `jmap.purge.schedule.blobs` have been removed. Instead, the purge frequency is now specified per store in `store.<name>.purge.frequency`. The attribute `jmap.purge.schedule.sessions` has been renamed to `jmap.purge.sessions.frequency`.\n\n## What's been added\n\n- **Multiple stores**: The server now supports multiple stores to be defined in the configuration file under `store.<name>`. Which store to use is defined in the `jmap.store.data`, `jmap.store.fts` and `jmap.store.blob` settings.\n- **More backend options**: It is now possible to use `RocksDB`, `PostgreSQL` and `MySQL` as data stores. It is also now possible to store blobs in any of the supported databases instead of being limited to the filesystem or an S3-compatible storage. Full-text indexing can now be done using `Elasticsearch` and the Spam database stored in `Redis`.\n- **Internal Directory**: The server now has an internal directory that can be used to store user accounts, passwords and group membership. This directory can be used instead of an external directory such as LDAP or SQL.\n- **New settings**: When running Stalwart in a cluster, `jmap.cluster.node-id` allows to specify a unique identifier for each node. Messages containing the SPAM headers defined in `jmap.spam.header` are moved automatically to the user's Junk Mail folder.\n- **Default Sieve stores**: For Sieve scripts such as the Spam filter that require access to a directory and a lookup store, it is now possible to configure the default lookup store and directory using the `sieve.trusted.default.directory` and `sieve.trusted.default.store` settings.\n\n## Migration Steps\n\nRather than manually updating the configuration file, it is recommended to start with a fresh configuration file and update it with the necessary settings:\n\n- Install `v0.5.0` in a distinct directory. You now have the option to use an [internal directory](https://stalw.art/docs/directory/types/internal), which will allow you to manage users and groups directly from Stalwart server. Alternatively, you can continue to use an external directory such as LDAP or SQL.\n- Update the configuration files with your previous settings. All configuration attributes are backward compatible, except those starting with `store.*`, `directory.*` and `jmap.purge.*`.\n- Export each account following the procedure described in the [migration guide](https://stalw.art/docs/management/database/migrate).\n- Stop the old `v0.4.x` server.\n- If there are messages pending to be delivered in the SMTP queue, move the `queue` directory to the new installation.\n- Start the new `v0.5.0` server.\n- Import each account following the procedure described in the [migration guide](https://stalw.art/docs/management/database/migrate).\n\n\nOnce again, we apologize for the lack of an automated migration tool for this upgrade. However, we are planning on introducing an automated migration tool once the web-admin is released in Q1 2024. Thank you for your understanding and patience.\n"
  },
  {
    "path": "UPGRADING/v0_06.md",
    "content": "# Upgrading from `v0.5.3` to `v0.6.0`\n\n- In order to support [expressions](https://stalw.art/docs/configuration/expressions/overview), version `0.6.0` introduces multiple breaking changes in the SMTP server configuration file. It is recommended to download the new SMTP configuration files from the [repository](https://github.com/stalwartlabs/mail-server/tree/main/resources/config/smtp), make any necessary changes and replace the old files under `INSTALL_DIR/etc/smtp` with the new ones.\n- If you are using custom subaddressing of catch-all rules, you'll need to replace these rules with expressions. Check out the updated [syntax](https://stalw.art/docs/directory/addresses).\n- Message queues are now distributed and stored in the backend specified by the `storage.data` and `storage.blob` settings. Make sure to flush your SMTP message queue before upgrading to `0.6.0` to avoid losing any outgoing messages pending delivery.\n- Replace the binary with the new version.\n- Restart the service.\n"
  },
  {
    "path": "UPGRADING/v0_07.md",
    "content": "# Upgrading from `v0.6.0` to `v0.7.0`\n\nVersion `0.7.0` of Stalwart introduces significant improvements and features that enhance performance and functionality. However, it also comes with multiple breaking changes in the configuration files and a revamped database layout optimized for accessing large mailboxes. Additionally, Stalwart now supports compression for binaries stored in the blob store, further increasing efficiency.\nDue to these extensive changes, the recommended approach for upgrading is to perform a clean reinstallation of Stalwart and manually migrate your accounts to the new version.\n\n## Pre-Upgrade Steps\n- Download the `v0.7.0` mail-server and CLI binaries for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/).\n- Initialize the setup on a distinct directory using the command `sudo ./stalwart-mail --init /path/to/new-install`. This command will print the administrator password required to access the web-admin.\n- Create the `bin` directory using `mkdir /path/to/new-install/bin`.\n- Move the downloaded binaries to the `bin` directory using the command `mv stalwart-mail stalwart-cli /path/to/new-install/bin`.\n- Open `/path/to/new-install/etc/config.toml` in a text editor and comment out all listeners except the HTTP listener for port `8080`.\n- Start the new installation from the terminal using the command `sudo /path/to/new-install/bin/stalwart-mail --config /path/to/new-install/etc/config.toml`.\n- Point your browser to the web-admin at `http://yourserver.org:8080` and login using the auto-generated administrator password. \n- Configure the new installation with your domain, hostname, certificates, and other settings following the instructions at [stalw.art/docs/get-started](https://stalw.art/docs/get-started). Ignore the part about using the installation script, we are performing a manual installation.\n- Add your user accounts.\n- Configure Stalwart to run as the `stalwart-mail` user and `stalwart-mail` group from `Settings` > `Server` > `System`. This is not necessary if you are using Docker.\n- Stop the new installation by pressing `Ctrl+C` in the terminal.\n\n## Upgrade Steps\n- On your `v0.6.0` installation, open in a text editor the `smtp/listener.toml`, `imap/listener.toml` files and comment out all listeners except the JMAP/HTTP listener (we are going to need it to export the user accounts) and then restart the service.\n- If you are using an external store, backup the database using the appropriate method for your database system.\n- Create the `~/exports` directory, here we will store the exported accounts.\n- Using the existing CLI tool (not the one you just downloaded as it is not compatible), export each user account using the command `./stalwart-cli -u https://your-old-server.org -c <ADMIN_PASSWORD> export account <ACCOUNT_NAME> ~/exports`.\n- Stop the `v0.6.0` installation using the command `sudo systemctl stop stalwart-mail`.\n- Move the old `v0.6.0` installation to a backup directory, for example `mv /opt/stalwart-mail /opt/stalwart-mail-backup`.\n- Move the new `v0.7.0` installation to the old installation directory, for example `mv /path/to/new-install /opt/stalwart-mail`.\n- Set the right permissions for the new installation using the command `sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail`.\n- Start the new installation using the command `sudo systemctl start stalwart-mail`.\n- Import the accounts using the new CLI tool with the command `./stalwart-cli -u http://yourserver.org:8080 -c <ADMIN_PASSWORD> import account <ACCOUNT> ~/exports/<ACCOUNT>`.\n- Using the admin tool, reactivate all the necessary listener (SMTP, IMAP, etc.)\n- Restart the service using the command `sudo systemctl restart stalwart-mail`.\n\nWe apologize for the complexity of the upgrade process associated with this version of Stalwart. We understand the challenges and inconveniences that the requirement for a clean reinstallation and manual account migration poses. Moving forward, an automated migration tool will be included in any future releases that necessitate changes to the database layout, aiming to streamline the upgrade process for you. Furthermore, as we approach the milestone of version 1.0.0, we anticipate that such foundational changes will become increasingly infrequent, leading to more straightforward updates. We appreciate your patience and commitment to Stalwart during this upgrade.\n"
  },
  {
    "path": "UPGRADING/v0_08.md",
    "content": "# Upgrading from `v0.7.3` to `v0.8.0`\n\nVersion `0.8.0` includes both performance and security enhancements that require your data to be migrated to a new database layout. Luckily version `0.7.3` includes a migration tool which should make this process much easier than previous upgrades. In addition to the new layout, you will have to change the systemd service file to use the `CAP_NET_BIND_SERVICE` capability.\n\n## Preparation\n- Upgrade to version `0.7.3` if you haven't already. If you are on a version previous to `0.7.0`, you will have to do a manual migration of your data using the Command-line Interface.\n- Create a directory where your data will be exported to, for example `/opt/stalwart-mail/export`.\n\n## Systemd service upgrade (Linux only)\n- Stop the `v0.7.3` installation:\n  ```bash\n  $ sudo systemctl stop stalwart-mail\n  ```\n- Update your systemd file to include the `CAP_NET_BIND_SERVICE` capability. Open the file `/etc/systemd/system/stalwart-mail.service` in a text editor and add the following lines under the `[Service]` section:\n  ```\n  User=stalwart-mail\n  Group=stalwart-mail\n  AmbientCapabilities=CAP_NET_BIND_SERVICE\n  ```\n- Reload the daemon:\n  ```bash\n  $ systemctl daemon-reload\n  ```\n- Do not start the service yet.\n\n## Data migration\n- Stop Stalwart and export your data:\n\n  ```bash\n  $ sudo systemctl stop stalwart-mail\n  $ sudo /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export\n  $ sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail/export\n  ```\n\n  or, if you are using the Docker image:\n\n  ```bash\n  $ docker stop stalwart-mail\n  $ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export\n  ```\n- Backup your `v0.7.3` installation:\n  - If you are using RocksDB or SQLite, simply rename the `data` directory to `data-backup`, for example:\n    ```bash\n    $ mv /opt/stalwart-mail/data /opt/stalwart-mail/data-backup\n    $ mkdir /opt/stalwart-mail/data\n    $ chown stalwart-mail:stalwart-mail /opt/stalwart-mail/data\n    ```\n  - If you are using PostgreSQL, rename the database and create a blank database with the same name, for example:\n    ```sql\n    ALTER DATABASE stalwart RENAME TO stalwart_old; \n    CREATE database stalwart;\n    ```\n  - If you are using MySQL, rename the database and create a blank database with the same name, for example:\n    ```sql\n    CREATE DATABASE stalwart_old;\n    RENAME TABLE stalwart.b TO stalwart_old.b;\n    RENAME TABLE stalwart.v TO stalwart_old.v;\n    RENAME TABLE stalwart.l TO stalwart_old.l;\n    RENAME TABLE stalwart.i TO stalwart_old.i;\n    RENAME TABLE stalwart.t TO stalwart_old.t;\n    RENAME TABLE stalwart.c TO stalwart_old.c;\n    DROP DATABASE stalwart;\n    CREATE database stalwart;\n    ```\n  - If you are using FoundationDB, backup your database and clean the entire key range.\n- Download the `v0.8.0` mail-server for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/) and replace the binary in `/opt/stalwart-mail/bin`. If you are using the Docker image, pull the latest image.\n- Import your data:\n\n  ```bash\n  $ sudo -u stalwart-mail /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --import /opt/stalwart-mail/export\n  ```\n\n  or, if you are using the Docker image:\n  \n  ```bash\n  $ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --import /opt/stalwart-mail/export\n  ```\n- Start the service:\n  ```bash\n  $ sudo systemctl start stalwart-mail\n  ```\n\n  Or, if you are using the Docker image:\n  ```bash\n  $ docker start stalwart-mail\n  ```"
  },
  {
    "path": "UPGRADING/v0_09.md",
    "content": "# Upgrading from `v0.8.x` to `v0.9.0`\n\nVersion `0.9.0` introduces significant internal improvements while maintaining compatibility with existing database layouts and configuration file formats from version `0.8.0`. As a result, no data or configuration migration is necessary. This release focuses on enhancing performance and functionality, particularly in logging and tracing capabilities.\n\nTo upgrade to Stalwart version `0.9.0` from `0.8.x`, begin by downloading the latest version of the `stalwart-mail` binary. Once downloaded, replace the existing binary with the new version. Additionally, it's important to update the WebAdmin interface to the latest version to ensure compatibility and to access new features introduced in this release.\n\nIn terms of breaking changes, this release brings significant updates to webhooks. All webhook event names have been modified, requiring a thorough review and adjustment of existing webhook configurations. Furthermore, the update introduces hundreds of new event types, enhancing the granularity and specificity of event handling capabilities. Users should familiarize themselves with these changes to effectively integrate them into their systems.\n\nThe reason for this release being classified as a major version, despite the absence of changes to the database or configuration formats, is the complete rewrite of the logging and tracing layer. This overhaul substantially improves the efficiency and speed of generating detailed tracing and logging events, making the system more robust and facilitating easier debugging and monitoring.\n"
  },
  {
    "path": "UPGRADING/v0_10.md",
    "content": "\n# Upgrading from `v0.9.x` to `v0.10.0`\n\n## Important Notes\n\n- In version `0.10.0` accounts are associated with roles and permissions, which define what resources they can access. The concept of administrator or super user accounts no longer exists, now there is a single account type (the `individual` principal) which can be assigned the `admin` role or custom permissions to have administrator access.\n- Due to the changes in the database layout in order to support roles and permissions, the database must be migrated to the new layout. The migration is automatic and should not require any manual intervention.\n- While the database migration is automatic, it's recommended to **back up your data** before upgrading.\n- The webadmin must be upgraded **before** the mail server to maintain access post-upgrade. This is true even if you run Stalwart in Docker.\n\n## Step-by-Step Upgrade Process\n\n- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.\n- Stop Stalwart and backup your data:\n\n  ```bash\n  $ sudo systemctl stop stalwart-mail\n  $ sudo /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export\n  $ sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail/export\n  ```\n\n  or, if you are using the Docker image:\n\n  ```bash\n  $ docker stop stalwart-mail\n  $ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /usr/local/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export\n  ```\n- Download the `v0.10.0` mail-server for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/) and replace the binary in `/opt/stalwart-mail/bin`. If you are using the Docker image, pull the latest image.\n- Start the service:\n  ```bash\n  $ sudo systemctl start stalwart-mail\n  ```\n\n  Or, if you are using the Docker image:\n  ```bash\n  $ docker start stalwart-mail\n  ```\n"
  },
  {
    "path": "UPGRADING/v0_11.md",
    "content": "\n# Upgrading from `v0.10.x` to `v0.11.0`\n\nVersion `0.11.0` introduces breaking changes to the spam filter configuration. Although no data migration is required, if changes were made to the previous spam filter, the configuration of the new spam filter should be reviewed. In particular:\n\n- `lookup.spam-*` settings are no longer used, these have been replaced by `spam-filter.*` settings. Review the [updated documentation](http://stalw.art/docs/spamfilter/overview).\n- Previous `spam-filter` and `track-replies` Sieve scripts cannot be used with the new version. They have been replaced by a built-in spam filter written in Rust.\n- Cache settings have changed, see the [documentation](https://stalw.art/docs/server/cache) for details.\n- Support for Pipes was removed in favor of MTA hooks and Milter.\n- `config.resource.spam-filter` is now `spam-filter.resource`.\n- `config.resource.webadmin` is now `webadmin.resource`.\n- `authentication.rate-limit` was removed as security is handled by fail2ban.\n\n"
  },
  {
    "path": "UPGRADING/v0_12.md",
    "content": "\n# Upgrading from `v0.11.x` to `v0.12.x`\n\n## Important Notes\n\nVersion `0.12.x` introduces significant improvements such as zero-copy deserialization which make the new database layout incompatible with the previous version. As a result, the database must be migrated to the new layout. The migration is done automatically on startup and should not require any manual intervention. However, it is highly recommended to **back up your data** before upgrading since it is not possible to downgrade the database once it has been migrated. You may also want to run a mock migration before upgrading to ensure that everything works as expected.\n\nIn addition to the database layout changes, multiple settings were renamed:\n\n- `server.http.*` to `http.*`.\n- `jmap.folders.*` to `email.folders.*`.\n- `jmap.account.purge.frequency` to `account.purge.frequency`.\n- `jmap.email.auto-expunge` to `email.auto-expunge`.\n- `jmap.protocol.changes.max-history` to `changes.max-history`.\n- `storage.encryption.*` to `email.encryption.*`.\n\n## Step-by-Step Upgrade Process\n\n- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:\n\n  ```bash\n  $ sudo systemctl stop stalwart-mail\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the [built-in migration utility](https://stalw.art/docs/management/migration) to export your data to a file. For example:\n\n  ```bash\n  $ sudo /opt/stalwart-mail/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export\n  $ sudo chown -R stalwart-mail:stalwart-mail /opt/stalwart-mail/export\n  ```\n\n- Download the `v0.12.x` binary for your platform (which is now called `stalwart` rather than `mail-server`) from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart-mail/bin`. If you rename the binary from `stalwart` to `stalwart-mail`, you can keep the same systemd service file, otherwise you will need to update the service file to point to the new binary name. \n\n- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once. \n  ```bash\n  $ sudo systemctl start stalwart-mail\n  ```\n\n- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.\n\n## Step-by-Step Upgrade Process (Docker)\n\n- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:\n\n  ```bash\n  $ docker stop stalwart-mail\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the `--export` command to export your data to a file. For example:\n\n  ```bash\n  $ docker run --rm -v <STALWART_DIR>:/opt/stalwart-mail -it stalwart-mail /usr/local/bin/stalwart-mail --config /opt/stalwart-mail/etc/config.toml --export /opt/stalwart-mail/export\n  ```\n\n- The Docker image location has now changed to `stalwartlabs/stalwart` instead of `stalwartlabs/mail-server`. Pull the latest image and configure it to use your existing data directory:\n\n  ```bash\n  $ docker run -d -ti -p 443:443 -p 8080:8080 \\\n             -p 25:25 -p 587:587 -p 465:465 \\\n             -p 143:143 -p 993:993 -p 4190:4190 \\\n             -p 110:110 -p 995:995 \\\n             -v <STALWART_DIR>:/opt/stalwart \\\n             --name stalwart stalwartlabs/stalwart:latest\n  ```\n  \n- Since the mount point has changed from `/opt/stalwart-mail` to `/opt/stalwart`, you will need to update your Stalwart's configuration file to reflect this change. Open the file `/opt/stalwart/etc/config.toml` and update the paths accordingly.\n\n- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.\n"
  },
  {
    "path": "UPGRADING/v0_13.md",
    "content": "# Upgrading from `v0.12.x` (and `v0.11.x`) to `v0.13.x`\n\n## Important Notes\n\nVersion `0.13.x` introduces a significant redesign of the MTA’s delivery and queueing subsystem. This includes a transition to a new message queue serialization format and a move to a strategy-based configuration model for routing, scheduling, and delivery control. Upon first launch of version `0.13.0`, any messages currently in the outbound queue will be automatically migrated to the new format. This migration is handled internally and does not require manual intervention.\n\nHowever, if your deployment includes custom routing rules or queueing logic, it is important to manually reconfigure those settings using the new strategy framework. The previous configuration format for routing is no longer compatible and will need to be updated. For systems that rely solely on the default configuration, no changes are required and the upgrade should proceed without issue.\n\nEven if your system uses the default settings, it is strongly recommended to read the accompanying [blog announcement](https://stalw.art/blog/virtual-queues) and consult the [updated documentation](https://stalw.art/docs/mta/outbound/overview). These resources provide a full overview of the new delivery architecture and can help you determine whether any adjustments are needed for your environment.\n\nBefore applying the upgrade to a production system, take time to familiarize yourself with the new configuration structure and validate that your delivery behavior aligns with the new model.\n\n## Step-by-Step Upgrade Process\n\n- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:\n\n  ```bash\n  $ sudo systemctl stop stalwart\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the [built-in migration utility](https://stalw.art/docs/management/migration) to export your data to a file. For example:\n\n  ```bash\n  $ sudo /opt/stalwart/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export\n  $ sudo chown -R stalwart:stalwart /opt/stalwart/export\n  ```\n\n- Download the `v0.13.x` binary for your platform from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart/bin`. \n\n- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once. \n  ```bash\n  $ sudo systemctl start stalwart\n  ```\n\n- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.\n\n## Step-by-Step Upgrade Process (Docker)\n\n- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:\n\n  ```bash\n  $ docker stop stalwart\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the `--export` command to export your data to a file. For example:\n\n  ```bash\n  $ docker run --rm -v <STALWART_DIR>:/opt/stalwart -it stalwart /usr/local/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export\n  ```\n\n- Pull the latest image and restart the container:\n\n  ```bash\n  $ docker pull stalwartlabs/stalwart:latest\n  $ docker start stalwart\n  ```\n  \n- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.\n\n"
  },
  {
    "path": "UPGRADING/v0_14.md",
    "content": "# Upgrading from `v0.13.x` to `v0.14.x`\n\n## Binary installation\n\n- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:\n\n  ```bash\n  $ sudo systemctl stop stalwart\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the [built-in migration utility](https://stalw.art/docs/management/migration) to export your data to a file. For example:\n\n  ```bash\n  $ sudo /opt/stalwart/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export\n  $ sudo chown -R stalwart:stalwart /opt/stalwart/export\n  ```\n\n- Download the latest binary for your platform from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart/bin`. \n\n- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once. \n  ```bash\n  $ sudo systemctl start stalwart\n  ```\n\n- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.\n\n## Containerized\n\n- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:\n\n  ```bash\n  $ docker stop stalwart\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database. If your database does not support backups, you can use the `--export` command to export your data to a file. For example:\n\n  ```bash\n  $ docker run --rm -v <STALWART_DIR>:/opt/stalwart -it stalwart /usr/local/bin/stalwart --config /opt/stalwart/etc/config.toml --export /opt/stalwart/export\n  ```\n\n- Pull the latest image and restart the container:\n\n  ```bash\n  $ docker pull stalwartlabs/stalwart:latest\n  $ docker start stalwart\n  ```\n  \n- Upgrade the webadmin by clicking on `Manage` > `Maintenance` > `Update Webadmin`.\n"
  },
  {
    "path": "UPGRADING/v0_15.md",
    "content": "# Upgrading from `v0.14.x` to `v0.15.x`\n\nStalwart `v0.15.x` introduces **breaking changes** to both the **database schema** and some **configuration options**.  \nUpgrading to this version **requires a schema migration**, which is performed **automatically when Stalwart starts** for the first time on `v0.15.x`.\n\nBecause this migration modifies how data is stored and indexed, it is important to understand what will change, what will be migrated, and how the upgrade may impact your deployment—especially for larger installations.\n\n## What's changed\n\nVersion `0.15.x` introduces significant internal improvements focused on performance, storage efficiency, and accuracy:\n\n- **Optimized database schema**: The database schema has been redesigned to use less storage space and significantly reduce the number of read and write operations required for common tasks.\n- **Rewritten search layer**: The search subsystem has been completely rewritten to use a more efficient and scalable indexing strategy.\n- **Native full-text search for SQL backends**: When using **PostgreSQL** or **MySQL** as the backend, Stalwart now leverages the database’s **native full-text search capabilities**, replacing the previous custom full-text search implementation.\n- **New spam classifier engine** : The spam classifier has been rewritten to use the **FTRL-Proximal** algorithm instead of the previous **Naive Bayes** implementation. This change improves classification accuracy, reduces memory usage, and reduces storage requirements for training data.\n\n\n## What will be migrated\n\nThe migration process runs automatically at startup and will migrate the following data:\n\n- **E-mail metadata**, including flags, folders, and parsed message representations. *(The raw e-mail content stored in the blob store is not migrated.)*\n- **Encryption-at-rest settings**, which now also include a **spam training privacy option**\n- **MTA message queue metadata** *(The actual message contents are not migrated.)*\n- **Maintenance tasks**\n- **Blob links** *(The underlying blobs themselves are not migrated.)*\n- **Search indexes**, which will be **rebuilt** using the new indexing strategy\n\n## Important considerations\n\n- For deployments with **1,000 or more mailboxes**, the migration may take a **considerable amount of time**, depending on the volume of stored data.\n- During migration, **Stalwart runs in read-only mode**:\n  - No new e-mail can be received\n  - No outbound e-mail can be sent\n- It is **strongly recommended** to perform this upgrade during a **maintenance window**.\n- By default, the migration process is **multithreaded** and uses two threads for each available CPUs. You can control the number of threads by setting the following environment variable ``NUM_THREADS=<number>``\n\n> **Note:** If you do **not** require any of the features introduced in `v0.15.x`, consider **waiting for the next major release**, which will introduce a proxy-based architecture allowing **zero-downtime upgrades**.\n\n## Upgrading steps\n\n### Binary installation\n\n- Stop Stalwart in **every single node of your cluster**. If you are using the systemd service, you can do this with the following command:\n\n  ```bash\n  $ sudo systemctl stop stalwart\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database.\n\n- Download the latest binary for your platform from the [releases page](https://github.com/stalwartlabs/stalwart/releases/latest/) and replace the binary in `/opt/stalwart/bin`. \n\n- Start the service. In a cluster, you can speed up the migration process by starting all nodes at once. \n  ```bash\n  $ sudo systemctl start stalwart\n  ```\n\n### Containerized\n\n- Stop the Stalwart container in **every single node of your cluster**. If you are using Docker, you can do this with the following command:\n\n  ```bash\n  $ docker stop stalwart\n  ```\n\n- Backup your data following your database system's instructions. For example, if you are using RocksDB or SQLite, you can simply copy the `data` directory to a backup location. If you are using PostgreSQL or MySQL, you can use the `pg_dump` or `mysqldump` commands to create a backup of your database.\n\n- Pull the latest image and restart the container:\n\n  ```bash\n  $ docker pull stalwartlabs/stalwart:latest\n  $ docker start stalwart\n  ```\n\n## Post-upgrade steps\n\nAfter the upgrade and migration complete, several follow-up steps are required or recommended:\n\n- **Upgrade the webadmin**: Upgrade the webadmin interface by navigating to ``Manage → Maintenance → Update Webadmin``\n\n- **Update the spam rules**: Download and apply the latest spam rules from the webadmin ``Manage → Maintenance → Update Spam rules``\n\n- **Update search settings**: Review the updated documentation for search settings, as some configuration options have changed.  In particular, the Elasticsearch backend now uses **different authentication settings** than previous versions.\n\n- **Rebuild search indexes**: All search indexes must be rebuilt to take advantage of the new indexing strategy. This can be done from the webadmin interface ``Manage → Maintenance``.\n\n- **Recalculate disk quotas for all accounts**: This step is **not required immediately**, but it is recommended to perform it at some point after the upgrade. The new version includes additional metadata in quota calculations, so recalculating ensures accurate disk usage reporting.\n    ```bash\n    $ curl -X DELETE https://myserver.org/api/store/quota/<account_name> -u <admin_user>:<admin_pass> -k\n    ```\n\n- **Delete deprecated spam classifier keys**: Remove deprecated spam classifier keys from the memory store. These are the keys starting with the integer prefixes `12` to `16` and `17` to `18`:\n  - If you are using Redis:\n    \n    ```bash\n    $ for code in {12..18}; do\n        char=$(printf \"\\\\x$(printf '%02x' $code)\")\n        redis-cli --scan --pattern \"${char}*\" | xargs -r redis-cli DEL\n    done\n    ```\n  - If you are using your database as the in-memory store:\n    \n    ```bash\n    $ /opt/stalwart/bin/stalwart --config /opt/stalwart/etc/config.toml --console\n    Stalwart Server v0.15.2 Data Store CLI\n\n    > delete y\\x0c\\x00 y\\x12\\xff\n    > delete m\\x0c\\x00 m\\x12\\xff\n    > exit\n    ```\n  - If you are using your database as the in-memory store with Docker:\n\n    ```bash\n    $ docker stop stalwart\n    $ docker run -it --rm \\\n        -v <STALWART_DIR>:/opt/stalwart \\\n        --entrypoint /usr/local/bin/stalwart \\\n        stalwartlabs/stalwart:latest \\\n        --config /opt/stalwart/etc/config.toml --console\n    Stalwart Server v0.15.2 Data Store CLI\n\n    > delete y\\x0c\\x00 y\\x12\\xff\n    > delete m\\x0c\\x00 m\\x12\\xff\n    > exit\n\n    $ docker start stalwart\n    ```\n\n\n## Troubleshooting\n\n### Interrupted or stopped migration\n\nIf the migration process is interrupted or stopped, it can be **resumed automatically** by simply restarting Stalwart.\n\n### `Data corruption detected` error\n\nIf you see an error message similar to: ``Data corruption detected``. This indicates that **another node wrote data using the old format while the migration was in progress**. This usually happens when the cluster was **not fully stopped** before starting the upgrade.\n\nIn order to resolve this issue, follow these steps:\n\n1. Stop **all** Stalwart nodes.\n2. Ensure **all nodes are upgraded** to `v0.15.x`.\n3. Start the nodes again.\n\n### Forcing a migration\n\nIf the migration does not resume because the node responsible for it already marked it as completed, you can force migration using environment variables:\n\n- **Force re-migration of MTA queue metadata**: ``FORCE_MIGRATE_QUEUE=4``\n- **Force re-migration of blob links**: ``FORCE_MIGRATE_BLOBS=4``\n- **Force re-migration of a specific account**: ``FORCE_MIGRATE_ACCOUNT=<account-id>``\n- **Force re-migration of all data**: ``FORCE_MIGRATE=4``\n\nUse these options with care and only when necessary.\n"
  },
  {
    "path": "api/v1/openapi.yml",
    "content": "# SPDX-FileCopyrightText: 2025 Stalwart Labs LLC <hello@stalw.art>\n#\n# SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n\nopenapi: 3.0.0\ninfo:\n  title: Stalwart API\n  version: 1.0.0\nservers:\n  - url: https://mail.example.org/api\n    description: Sample server\npaths:\n  /oauth:\n    post:\n      summary: Obtain OAuth token\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      code:\n                        type: string\n                      permissions:\n                        type: array\n                        items:\n                          type: string\n                      version:\n                        type: string\n                      isEnterprise:\n                        type: boolean\n              example:\n                data:\n                  code: 4YmRFLu9Df1t4JO7Iffnuney4B8tVLAxjimdRxEg\n                  permissions:\n                    - webadmin-update\n                    - spam-filter-update\n                    - dkim-signature-get\n                    - dkim-signature-create\n                    - undelete\n                    - fts-reindex\n                    - purge-account\n                    - purge-in-memory-store\n                    - purge-data-store\n                    - purge-blob-store\n                  version: 0.11.0\n                  isEnterprise: true\n        \"401\":\n          description: Unauthorized\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  type:\n                    type: string\n                  status:\n                    type: number\n                  title:\n                    type: string\n                  detail:\n                    type: string\n              example:\n                type: about:blank\n                status: 401\n                title: Unauthorized\n                detail: You have to authenticate first.\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                type:\n                  type: string\n                client_id:\n                  type: string\n                redirect_uri:\n                  type: string\n                nonce:\n                  type: string\n            example:\n              type: code\n              client_id: webadmin\n              redirect_uri: stalwart://auth\n              nonce: ttsaXca3qx\n  /telemetry/metrics:\n    get:\n      summary: Fetch Telemetry Metrics\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  error:\n                    type: string\n                  details:\n                    type: string\n                  reason:\n                    type: string\n              example:\n                error: other\n                details: No metrics store has been defined\n                reason:\n                  You need to configure a metrics store in order to use this\n                  feature.\n      parameters:\n        - name: after\n          in: query\n          required: false\n          schema:\n            type: string\n  /telemetry/live/metrics-token:\n    get:\n      summary: Obtain Metrics Telemetry token\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: string\n              example:\n                data: 2GO4RahIkSAms6S00R9BRsroo97ZdYTz4QVxFCOwGrGkr7zguP0AVyTMA/iha3Vz/////w8DhZi1+ALBmLX4AndlYg==\n  /telemetry/metrics/live:\n    get:\n      summary: Live Metrics\n      responses:\n        \"200\":\n          description: OK\n          content: {}\n      parameters:\n        - name: metrics\n          in: query\n          required: false\n          schema:\n            type: string\n        - name: interval\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: token\n          in: query\n          required: false\n          schema:\n            type: string\n  /principal:\n    get:\n      summary: List Principals\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items: {}\n                      total:\n                        type: number\n              example:\n                data:\n                  items: []\n                  total: 0\n      parameters:\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: types\n          in: query\n          required: false\n          schema:\n            type: string\n    post:\n      summary: Create Principal\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: number\n              example:\n                data: 50\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                type:\n                  type: string\n                quota:\n                  type: number\n                name:\n                  type: string\n                description:\n                  type: string\n                secrets:\n                  type: array\n                  items: {}\n                emails:\n                  type: array\n                  items: {}\n                urls:\n                  type: array\n                  items: {}\n                memberOf:\n                  type: array\n                  items: {}\n                roles:\n                  type: array\n                  items: {}\n                lists:\n                  type: array\n                  items: {}\n                members:\n                  type: array\n                  items: {}\n                enabledPermissions:\n                  type: array\n                  items: {}\n                disabledPermissions:\n                  type: array\n                  items: {}\n                externalMembers:\n                  type: array\n                  items: {}\n            example:\n              type: domain\n              quota: 0\n              name: example.org\n              description: Example domain\n              secrets: []\n              emails: []\n              urls: []\n              memberOf: []\n              roles: []\n              lists: []\n              members: []\n              enabledPermissions: []\n              disabledPermissions: []\n              externalMembers: []\n  /dkim:\n    post:\n      summary: Create DKIM Signature\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                id:\n                  type: object\n                  nullable: true\n                algorithm:\n                  type: string\n                domain:\n                  type: string\n                selector:\n                  type: object\n                  nullable: true\n            example:\n              id:\n              algorithm: Ed25519\n              domain: example.org\n              selector:\n  /principal/{principal_id}:\n    get:\n      summary: Fetch Principal\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      id:\n                        type: number\n                      type:\n                        type: string\n                      secrets:\n                        type: string\n                      name:\n                        type: string\n                      quota:\n                        type: number\n                      description:\n                        type: string\n                      emails:\n                        type: string\n                      roles:\n                        type: array\n                        items:\n                          type: string\n                      lists:\n                        type: array\n                        items:\n                          type: string\n              example:\n                data:\n                  id: 90\n                  type: individual\n                  secrets: $6$ONjGT6nQtmPNaxw0$NNF5DXtPfOay2mfVnPJ0uQ77C.L3LNxXO/QMyphP/DzpODqbDBBGd4/gCnckYPQj3st6pqwY8/KeBsCJ.oe1Y1\n                  name: jane\n                  quota: 0\n                  description: Jane Doe\n                  emails: jane@example.org\n                  roles:\n                    - user\n                  lists:\n                    - all\n      parameters:\n        - name: principal_id\n          in: path\n          required: true\n          schema:\n            type: string\n    patch:\n      summary: Update Principal\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      parameters:\n        - name: principal_id\n          in: path\n          required: true\n          schema:\n            type: string\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: array\n              items:\n                type: object\n                properties:\n                  action:\n                    type: string\n                  field:\n                    type: string\n                  value:\n                    type: string\n            example:\n              - action: set\n                field: name\n                value: jane.doe\n              - action: set\n                field: description\n                value: Jane Mary Doe\n              - action: addItem\n                field: emails\n                value: jane-doe@example.org\n              - action: removeItem\n                field: emails\n                value: jane@example.org\n    delete:\n      summary: Delete Principal\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      parameters:\n        - name: principal_id\n          in: path\n          required: true\n          schema:\n            type: string\n  /queue/messages:\n    get:\n      summary: List Queued Messages\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items: {}\n                      total:\n                        type: number\n                      status:\n                        type: boolean\n              example:\n                data:\n                  items: []\n                  total: 0\n                  status: true\n      parameters:\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: max-total\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: values\n          in: query\n          required: false\n          schema:\n            type: number\n    patch:\n      summary: Reschedule Queued Messages\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: boolean\n              example:\n                data: true\n      parameters:\n        - name: filter\n          in: query\n          required: false\n          schema:\n            type: string\n    delete:\n      summary: Delete Queued Messages\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: boolean\n              example:\n                data: true\n      parameters:\n        - name: text\n          in: query\n          required: false\n          schema:\n            type: string\n  /queue/reports:\n    get:\n      summary: List Queued Reports\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items: {}\n                      total:\n                        type: number\n              example:\n                data:\n                  items: []\n                  total: 0\n      parameters:\n        - name: max-total\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n  /reports/dmarc:\n    get:\n      summary: List Incoming DMARC Reports\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items: {}\n                      total:\n                        type: number\n              example:\n                data:\n                  items: []\n                  total: 0\n      parameters:\n        - name: max-total\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n  /reports/tls:\n    get:\n      summary: List Incoming TLS Reports\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items: {}\n                      total:\n                        type: number\n              example:\n                data:\n                  items: []\n                  total: 0\n      parameters:\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: max-total\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n  /reports/arf:\n    get:\n      summary: List Incoming ARF Reports\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items: {}\n                      total:\n                        type: number\n              example:\n                data:\n                  items: []\n                  total: 0\n      parameters:\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: max-total\n          in: query\n          required: false\n          schema:\n            type: number\n  /telemetry/traces:\n    get:\n      summary: List Stored Traces\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  error:\n                    type: string\n                  details:\n                    type: string\n              example:\n                error: unsupported\n                details: No tracing store has been configured\n      parameters:\n        - name: type\n          in: query\n          required: false\n          schema:\n            type: string\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: values\n          in: query\n          required: false\n          schema:\n            type: number\n  /logs:\n    get:\n      summary: Quere Log Files\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items:\n                          type: object\n                          properties:\n                            timestamp:\n                              type: string\n                            level:\n                              type: string\n                            event:\n                              type: string\n                            event_id:\n                              type: string\n                            details:\n                              type: string\n                      total:\n                        type: number\n              example:\n                data:\n                  items:\n                    - timestamp: \"2025-01-05T14:06:29Z\"\n                      level: TRACE\n                      event: HTTP request body\n                      event_id: http.request-body\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, contents = \"\", size = 0\n                    - timestamp: \"2025-01-05T14:06:29Z\"\n                      level: TRACE\n                      event: Write batch operation\n                      event_id: store.data-write\n                      details: elapsed = 0ms, total = 2\n                    - timestamp: \"2025-01-05T14:06:29Z\"\n                      level: TRACE\n                      event: Expression evaluation result\n                      event_id: eval.result\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, id = \"server.http.allowed-endpoint\", result\n                        = \"Integer(200)\"\n                    - timestamp: \"2025-01-05T14:06:29Z\"\n                      level: DEBUG\n                      event: HTTP request URL\n                      event_id: http.request-url\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, url = \"/api/logs?page=1&limit=50&\"\n                    - timestamp: \"2025-01-05T14:06:23Z\"\n                      level: TRACE\n                      event: HTTP response body\n                      event_id: http.response-body\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, contents = \"{\"error\":\"unsupported\",\"details\":\"No\n                        tracing store has been configured\"}\", code = 200, size = 72\n                    - timestamp: \"2025-01-05T14:06:23Z\"\n                      level: DEBUG\n                      event: Management operation not supported\n                      event_id: manage.not-supported\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, details = No tracing store has been configured\n                    - timestamp: \"2025-01-05T14:06:23Z\"\n                      level: TRACE\n                      event: HTTP request body\n                      event_id: http.request-body\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, contents = \"\", size = 0\n                    - timestamp: \"2025-01-05T14:06:23Z\"\n                      level: TRACE\n                      event: Write batch operation\n                      event_id: store.data-write\n                      details: elapsed = 0ms, total = 2\n                    - timestamp: \"2025-01-05T14:06:23Z\"\n                      level: TRACE\n                      event: Expression evaluation result\n                      event_id: eval.result\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, id = \"server.http.allowed-endpoint\", result\n                        = \"Integer(200)\"\n                    - timestamp: \"2025-01-05T14:06:23Z\"\n                      level: DEBUG\n                      event: HTTP request URL\n                      event_id: http.request-url\n                      details:\n                        listenerId = \"http\", localPort = 1443, remoteIp = ::1,\n                        remotePort = 57223, url = \"/api/telemetry/traces?page=1&type=delivery.attempt-start&limit=10&values=1&\"\n                  total: 100\n      parameters:\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n  /spam-filter/train/spam:\n    post:\n      summary: Train Spam Filter as Spam\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      requestBody:\n        content:\n          application/x-www-form-urlencoded:\n            schema:\n              type: object\n              properties:\n                ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing,\n                  please ignore\\nContent-Type: text/plain; charset\"\n                : type: string\n            example:\n              ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing, please\n                ignore\\nContent-Type: text/plain; charset\"\n              : \"\\\"utf-8\\\"\\nContent-Transfer-Encoding: 8bit\\n\\nTesting 1, 2, 3\\n\"\n  /spam-filter/train/ham:\n    post:\n      summary: Train Spam Filter as Ham\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      requestBody:\n        content:\n          application/x-www-form-urlencoded:\n            schema:\n              type: object\n              properties:\n                ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing,\n                  please ignore\\nContent-Type: text/plain; charset\"\n                : type: string\n            example:\n              ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing, please\n                ignore\\nContent-Type: text/plain; charset\"\n              : \"\\\"utf-8\\\"\\nContent-Transfer-Encoding: 8bit\\n\\nTesting 1, 2, 3\\n\"\n  /spam-filter/train/spam/{account_id}:\n    post:\n      summary: Train Account's Spam Filter as Spam\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      parameters:\n        - name: account_id\n          in: path\n          required: true\n          schema:\n            type: string\n      requestBody:\n        content:\n          application/x-www-form-urlencoded:\n            schema:\n              type: object\n              properties:\n                ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing,\n                  please ignore\\nContent-Type: text/plain; charset\"\n                : type: string\n            example:\n              ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing, please\n                ignore\\nContent-Type: text/plain; charset\"\n              : \"\\\"utf-8\\\"\\nContent-Transfer-Encoding: 8bit\\n\\nTesting 1, 2, 3\\n\"\n  /spam-filter/train/ham/{account_id}:\n    post:\n      summary: Train Account's Spam Filter as Ham\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      parameters:\n        - name: account_id\n          in: path\n          required: true\n          schema:\n            type: string\n      requestBody:\n        content:\n          application/x-www-form-urlencoded:\n            schema:\n              type: object\n              properties:\n                ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing,\n                  please ignore\\nContent-Type: text/plain; charset\"\n                : type: string\n            example:\n              ? \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing, please\n                ignore\\nContent-Type: text/plain; charset\"\n              : \"\\\"utf-8\\\"\\nContent-Transfer-Encoding: 8bit\\n\\nTesting 1, 2, 3\\n\"\n  /spam-filter/classify:\n    post:\n      summary: Test Spam Filter Classification\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      score:\n                        type: number\n                      tags:\n                        type: object\n                        properties:\n                          FROM_NO_DN:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          SOURCE_ASN_15169:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          SOURCE_COUNTRY_US:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          MISSING_DATE:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          FROMHOST_NORES_A_OR_MX:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          MISSING_MIME_VERSION:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          FORGED_SENDER:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          SPF_NA:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          X_HDR_TO:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          HELO_IPREV_MISMATCH:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          X_HDR_CONTENT_TYPE:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          AUTH_NA:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          FORGED_RECIPIENTS:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          RBL_SENDERSCORE_REPUT_BLOCKED:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          RCVD_COUNT_ZERO:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          X_HDR_SUBJECT:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          X_HDR_FROM:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          RCPT_COUNT_ONE:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          MISSING_MID:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          TO_DOM_EQ_FROM_DOM:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          ARC_NA:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          RCVD_TLS_LAST:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          X_HDR_CONTENT_TRANSFER_ENCODING:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          HELO_NORES_A_OR_MX:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          TO_DN_NONE:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          FROM_NEQ_ENV_FROM:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          DMARC_NA:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          SINGLE_SHORT_PART:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                          DKIM_NA:\n                            type: object\n                            properties:\n                              action:\n                                type: string\n                              value:\n                                type: number\n                      disposition:\n                        type: object\n                        properties:\n                          action:\n                            type: string\n                          value:\n                            type: string\n              example:\n                data:\n                  score: 12.7\n                  tags:\n                    FROM_NO_DN:\n                      action: allow\n                      value: 0.0\n                    SOURCE_ASN_15169:\n                      action: allow\n                      value: 0.0\n                    SOURCE_COUNTRY_US:\n                      action: allow\n                      value: 0.0\n                    MISSING_DATE:\n                      action: allow\n                      value: 1.0\n                    FROMHOST_NORES_A_OR_MX:\n                      action: allow\n                      value: 1.5\n                    MISSING_MIME_VERSION:\n                      action: allow\n                      value: 2.0\n                    FORGED_SENDER:\n                      action: allow\n                      value: 0.3\n                    SPF_NA:\n                      action: allow\n                      value: 0.0\n                    X_HDR_TO:\n                      action: allow\n                      value: 0.0\n                    HELO_IPREV_MISMATCH:\n                      action: allow\n                      value: 1.0\n                    X_HDR_CONTENT_TYPE:\n                      action: allow\n                      value: 0.0\n                    AUTH_NA:\n                      action: allow\n                      value: 1.0\n                    FORGED_RECIPIENTS:\n                      action: allow\n                      value: 2.0\n                    RBL_SENDERSCORE_REPUT_BLOCKED:\n                      action: allow\n                      value: 0.0\n                    RCVD_COUNT_ZERO:\n                      action: allow\n                      value: 0.1\n                    X_HDR_SUBJECT:\n                      action: allow\n                      value: 0.0\n                    X_HDR_FROM:\n                      action: allow\n                      value: 0.0\n                    RCPT_COUNT_ONE:\n                      action: allow\n                      value: 0.0\n                    MISSING_MID:\n                      action: allow\n                      value: 2.5\n                    TO_DOM_EQ_FROM_DOM:\n                      action: allow\n                      value: 0.0\n                    ARC_NA:\n                      action: allow\n                      value: 0.0\n                    RCVD_TLS_LAST:\n                      action: allow\n                      value: 0.0\n                    X_HDR_CONTENT_TRANSFER_ENCODING:\n                      action: allow\n                      value: 0.0\n                    HELO_NORES_A_OR_MX:\n                      action: allow\n                      value: 0.3\n                    TO_DN_NONE:\n                      action: allow\n                      value: 0.0\n                    FROM_NEQ_ENV_FROM:\n                      action: allow\n                      value: 0.0\n                    DMARC_NA:\n                      action: allow\n                      value: 1.0\n                    SINGLE_SHORT_PART:\n                      action: allow\n                      value: 0.0\n                    DKIM_NA:\n                      action: allow\n                      value: 0.0\n                  disposition:\n                    action: allow\n                    value:\n                      \"X-Spam-Result: ARC_NA (0.00),\\r\\n\\tDKIM_NA (0.00),\\r\\n\n                      \\tFROM_NEQ_ENV_FROM (0.00),\\r\\n\\tFROM_NO_DN (0.00),\\r\\n\\tRBL_SENDERSCORE_REPUT_BLOCKED\n                      (0.00),\\r\\n\\tRCPT_COUNT_ONE (0.00),\\r\\n\\tRCVD_TLS_LAST (0.00),\\r\n                      \\n\\tSINGLE_SHORT_PART (0.00),\\r\\n\\tSPF_NA (0.00),\\r\\n\\tTO_DN_NONE\n                      (0.00),\\r\\n\\tTO_DOM_EQ_FROM_DOM (0.00),\\r\\n\\tRCVD_COUNT_ZERO\n                      (0.10),\\r\\n\\tFORGED_SENDER (0.30),\\r\\n\\tHELO_NORES_A_OR_MX (0.30),\\r\n                      \\n\\tAUTH_NA (1.00),\\r\\n\\tDMARC_NA (1.00),\\r\\n\\tHELO_IPREV_MISMATCH\n                      (1.00),\\r\\n\\tMISSING_DATE (1.00),\\r\\n\\tFROMHOST_NORES_A_OR_MX\n                      (1.50),\\r\\n\\tFORGED_RECIPIENTS (2.00),\\r\\n\\tMISSING_MIME_VERSION\n                      (2.00),\\r\\n\\tMISSING_MID (2.50)\\r\\nX-Spam-Status: Yes, score=12.70\\r\\\n                      \\n\"\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                message:\n                  type: string\n                remoteIp:\n                  type: string\n                ehloDomain:\n                  type: string\n                authenticatedAs:\n                  type: object\n                  nullable: true\n                isTls:\n                  type: boolean\n                envFrom:\n                  type: string\n                envFromFlags:\n                  type: number\n                envRcptTo:\n                  type: array\n                  items:\n                    type: string\n            example:\n              message:\n                \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing,\n                please ignore\\nContent-Type: text/plain; charset=\\\"utf-8\\\"\\nContent-Transfer-Encoding:\n                8bit\\n\\nTesting 1, 2, 3\\n\"\n              remoteIp: 8.8.8.8\n              ehloDomain: foo.org\n              authenticatedAs:\n              isTls: true\n              envFrom: bill@foo.org\n              envFromFlags: 0\n              envRcptTo:\n                - john@example.org\n  /troubleshoot/token:\n    get:\n      summary: Obtain a Troubleshooting Token\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: string\n              example:\n                data: +bS1rCUcrjoEtl9f7Vz1P6daqVs4nywxa56bHltPIASijRFrj1JrwvHxJCWphPKs/////w8E8p21+AKunrX4AndlYg==\n  /troubleshoot/delivery/{recipient}:\n    get:\n      summary: Run Delivery Troubleshooting\n      responses:\n        \"200\":\n          description: OK\n          content: {}\n      parameters:\n        - name: recipient\n          in: path\n          required: true\n          schema:\n            type: string\n        - name: token\n          in: query\n          required: false\n          schema:\n            type: string\n  /troubleshoot/dmarc:\n    post:\n      summary: Run DMARC Troubleshooting\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      spfEhloDomain:\n                        type: string\n                      spfEhloResult:\n                        type: object\n                        properties:\n                          type:\n                            type: string\n                      spfMailFromDomain:\n                        type: string\n                      spfMailFromResult:\n                        type: object\n                        properties:\n                          type:\n                            type: string\n                          details:\n                            type: object\n                            nullable: true\n                      ipRevResult:\n                        type: object\n                        properties:\n                          type:\n                            type: string\n                      ipRevPtr:\n                        type: array\n                        items:\n                          type: string\n                      dkimResults:\n                        type: array\n                        items: {}\n                      dkimPass:\n                        type: boolean\n                      arcResult:\n                        type: object\n                        properties:\n                          type:\n                            type: string\n                      dmarcResult:\n                        type: object\n                        properties:\n                          type:\n                            type: string\n                      dmarcPass:\n                        type: boolean\n                      dmarcPolicy:\n                        type: string\n                      elapsed:\n                        type: number\n              example:\n                data:\n                  spfEhloDomain: mx.google.com\n                  spfEhloResult:\n                    type: none\n                  spfMailFromDomain: google.com\n                  spfMailFromResult:\n                    type: softFail\n                    details:\n                  ipRevResult:\n                    type: pass\n                  ipRevPtr:\n                    - dns.google.\n                  dkimResults: []\n                  dkimPass: false\n                  arcResult:\n                    type: none\n                  dmarcResult:\n                    type: none\n                  dmarcPass: false\n                  dmarcPolicy: reject\n                  elapsed: 200\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                remoteIp:\n                  type: string\n                ehloDomain:\n                  type: string\n                mailFrom:\n                  type: string\n                body:\n                  type: string\n            example:\n              remoteIp: 8.8.8.8\n              ehloDomain: mx.google.com\n              mailFrom: john@google.com\n              body:\n                \"From: john@example.org\\nTo: list@example.org\\nSubject: Testing,\n                please ignore\\nContent-Type: text/plain; charset=\\\"utf-8\\\"\\nContent-Transfer-Encoding:\n                8bit\\n\\nTesting 1, 2, 3\\n\"\n  /reload:\n    get:\n      summary: Reload Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      warnings:\n                        type: object\n                        properties: {}\n                      errors:\n                        type: object\n                        properties: {}\n              example:\n                data:\n                  warnings: {}\n                  errors: {}\n      parameters:\n        - name: dry-run\n          in: query\n          required: false\n          schema:\n            type: string\n  /update/spam-filter:\n    get:\n      summary: Update Spam Filter\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /update/webadmin:\n    get:\n      summary: Update WebAdmin\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /store/reindex:\n    get:\n      summary: Request FTS Reindex\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /store/purge/in-memory/default/bayes-global:\n    get:\n      summary: Delete Global Bayes Model\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /settings/keys:\n    get:\n      summary: List Settings by Key\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      lookup.default.hostname:\n                        type: string\n              example:\n                data:\n                  lookup.default.hostname: mx.fr.email\n      parameters:\n        - name: prefixes\n          in: query\n          required: false\n          schema:\n            type: string\n        - name: keys\n          in: query\n          required: false\n          schema:\n            type: string\n  /settings/group:\n    get:\n      summary: List Settings by Group\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      total:\n                        type: number\n                      items:\n                        type: array\n                        items:\n                          type: object\n                          properties:\n                            _id:\n                              type: string\n                            bind:\n                              type: string\n                            protocol:\n                              type: string\n              example:\n                data:\n                  total: 11\n                  items:\n                    - _id: http\n                      bind: \"[::]:1443\"\n                      protocol: http\n                    - bind: \"[::]:443\"\n                      _id: https\n                      protocol: http\n                      tls.implicit: \"true\"\n                    - protocol: imap\n                      bind: \"[::]:143\"\n                      _id: imap\n                    - bind: \"[::]:1143\"\n                      tls.implicit: \"false\"\n                      _id: imapnotls\n                      protocol: imap\n                      proxy.override: \"false\"\n                      tls.override: \"false\"\n                      tls.enable: \"false\"\n                      socket.override: \"false\"\n                    - bind: \"[::]:993\"\n                      tls.implicit: \"true\"\n                      protocol: imap\n                      _id: imaptls\n                    - bind: \"[::]:110\"\n                      protocol: pop3\n                      _id: pop3\n                    - tls.implicit: \"true\"\n                      _id: pop3s\n                      protocol: pop3\n                      bind: \"[::]:995\"\n                    - protocol: managesieve\n                      _id: sieve\n                      bind: \"[::]:4190\"\n                    - bind: \"[::]:25\"\n                      _id: smtp\n                      protocol: smtp\n                    - _id: submission\n                      bind: \"[::]:587\"\n                      protocol: smtp\n      parameters:\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: suffix\n          in: query\n          required: false\n          schema:\n            type: string\n        - name: prefix\n          in: query\n          required: false\n          schema:\n            type: string\n  /settings/list:\n    get:\n      summary: List Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      total:\n                        type: number\n                      items:\n                        type: object\n                        properties:\n                          enable:\n                            type: string\n                          format:\n                            type: string\n                          limits.entries:\n                            type: string\n                          limits.entry-size:\n                            type: string\n                          limits.size:\n                            type: string\n                          refresh:\n                            type: string\n                          retry:\n                            type: string\n                          timeout:\n                            type: string\n                          url:\n                            type: string\n              example:\n                data:\n                  total: 9\n                  items:\n                    enable: \"true\"\n                    format: list\n                    limits.entries: \"100000\"\n                    limits.entry-size: \"512\"\n                    limits.size: \"104857600\"\n                    refresh: 12h\n                    retry: 1h\n                    timeout: 30s\n                    url: https://openphish.com/feed.txt\n      parameters:\n        - name: prefix\n          in: query\n          required: false\n          schema:\n            type: string\n  /settings:\n    post:\n      summary: Update Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: array\n              items:\n                type: object\n                properties:\n                  type:\n                    type: string\n                  prefix:\n                    type: string\n            example:\n              - type: clear\n                prefix: spam-filter.rule.stwt_arc_signed.\n  /account/crypto:\n    get:\n      summary: Obtain Encryption-at-Rest Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      type:\n                        type: string\n              example:\n                data:\n                  type: disabled\n    post:\n      summary: Update Encryption-at-Rest Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: number\n              example:\n                data: 1\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                type:\n                  type: string\n                algo:\n                  type: string\n                certs:\n                  type: string\n            example:\n              type: pGP\n              algo: Aes256\n              certs:\n                \"-----BEGIN PGP PUBLIC KEY BLOCK-----\\n\\nxsFNBGTGHwkBEADRB5EEtfsnUwgF2ZRg6h1fp2E8LNhv4lb9AWersI8KNFoWM6qx\\n\n                Bk/MfEpgILSPdW3g7PWHOxPV/hxjtStFHfbU/Ye5VvfbkU49faIPiw1V3MQJJ171\\n\n                cN6kgMnABfdixNiutDkHP4f34ABrEqexX2myOP+btxL24gI/N9UpOD5PiKTyKR7i\\n\n                GwNpi+O022rs/KvjlWR7iSJ4vk7bGFfTNHvWI6dZworey1tZoTIZ0CgvgMeB/F1q\\n\n                OOa0FvrJdNYR227RpHmICqFqTptNZ2EfdkJ6QUXW7bZ9dWgL36ds9QPJOGcG3c5i\\n\n                JebeX5YdJnniBefiWjfZElcqh/N6SqVuEwoTLyMCnMZ6gjNMn6tddwPH24kavZhT\\n\n                p6+vhTHmyq8XBqK/XEt9r+clSfg2hi5s7GO7hQV+W26xRjX7sQJY41PfzkgYJ0BM\\n\n                6+w09X1ZO/iMjEp44t2rd3xSudwGYhlbazXbdB+OJaa3RtyjOAeFgY8OyNlODx3V\\n\n                xXLtF+104HGSL7nkpBsu6LLighSgEEF2Vok43grr0omyb1NPhWoAZhM8sT5iv5gW\\n\n                fKvB1O13c+hDc/iGTAvcrtdLLnF2Cs+6HD7r7zPPM4L6DrD1+oQt510H/oOEE5NZ\\n\n                wIS9CmBf0txqwk7n1U5V95lonaCK9nfoKeQ1fKl/tu01dCeERRbMXG2nCQARAQAB\\n\n                zRtKb2huIERvZSA8am9obkBleGFtcGxlLm9yZz7CwYcEEwEIADEWIQQWwx1eM+Aa\\n\n                o8okGzL45grMTSggxQUCZMYfCQIbAwQLCQgHBRUICQoLBRYCAwEAAAoJEPjmCsxN\\n\n                KCDFWP4QAI3eS5nPxmU0AC9/h8jeKNgjgpENroNQZKeWZQ8x4PfncDRkcbsJfT7Y\\n\n                IVZl4zw6gFKY5EoB1s1KkYJxPgYsqicmKNiR7Tnzabb3mzomU48FKaIyVCBzFUnJ\\n\n                YMroL/rm7QhoW2WWLvT+CPCPway/tA3By8Be/YOjhavJ8mf1W3rPzt87/4Vo6erf\\n\n                yzL0lN+FQmmhKfT4j42jF4SMSyyC2yzvfC7PT49u+KUKQm/LpQsfKHpwXZ/VI6+X\\n\n                GtZjTqsc+uglJYRo69oosImLzieA/ST1ltjmUutZQOSvlQFpDUEFrMej8XZ0qsrf\\n\n                0gP2iwxyl0vkhV8c6wO6CacDHPivvQEHed9H1PNGn3DBfKb7Mq/jado2DapRtJg3\\n\n                2OH0F0HTvQ0uNKl30xMUcwGQB0cKOlaFtksZT1LsosQPhtPLpFy1TuWaXOInpQLq\\n\n                JmNVcTbydOsCKq0mb6bgGcvhElC1q39tclKP3rOEDOnJ8hE6wYNaMGrt6WSKr3Tt\\n\n                h52M6KwTXOuMAecMvpDBSS3UFEVQ+T5puzInDTkjINxmj23ip+swA1x3HH2IgNrO\\n\n                VJ7O20oEf0+qC47R5rTRUxrvh/U0U3DRE5xt2J2T3xetFDT2mnQv0jcyMg/UlXXv\\n\n                GpGVfwNkvN0Cxmb1tFiBNLKCcPVizxq4MLrwx+MVfQBaRCwjJrUszsFNBGTGHwoB\\n\n                EACr5lA+j5pH0Er6Q76btbS4q9JgNjDNrjKJwX9brdBY1oXIUeBqCW9ekoqDTFpn\\n\n                xA5EFGJvPO++/0ZCa+zXE4IAcXS9+I9HVBouenPYBLETnXK0Phws+OCLoe0cAIvG\\n\n                e9Xo9VrHcGXCs9tJruVSAW3NF04YejHmnHNfEuD8mbaUdxVn5zc23w/2gLaY/ABL\\n\n                ZfNV8XZw0jBVBm3YXS3Ob3uIO+RvsNqBgnhGYN/C51QI9hdxXWUDlD1vdRacXmcI\\n\n                LDCYC3w6u8caxL0ktXTS4zwN+hEu7jHxBNiKcovCeIF5VZ5NcPpp6+6Y+vNdmmXw\\n\n                +lWNwAzj3ah6iu+y25LKSsz+7IkCh5liOwwYohO+YI7SjtTD+gL9HiHYAIO+PtBh\\n\n                7GudmUwFoARu/q54hE4ThpzkeOzJzPqGkM/CzmwdKKM3u81ze+72ptJOqVKbFEsQ\\n\n                3+RURrIAfyYyeJj4VVCfHNzrRRVpARZc9hJm1AXefxPnDN9dxbikjQgbg5UxrKaJ\\n\n                cjVU+go5CH5lg2D1LRGfKqTJtfiWFPjtztNgMp/SeslkhhFXsyJ0RJDcU8VfRBrO\\n\n                DBnZvPnZi4nLaWCL1LdHA8Y9EJgSwVOsfdRqL/Xk9qxqgl5R8m8lsNKZN2EYkfMN\\n\n                4Vd+/8UBbmibHYoGIQi7UlNSPthc0XQcRzFen+3H4sg5kQARAQABwsF2BBgBCAAg\\n\n                FiEEFsMdXjPgGqPKJBsy+OYKzE0oIMUFAmTGHwsCGwwACgkQ+OYKzE0oIMXn4hAA\\n\n                lUWeF7tDdyENsOYyhsbtLIuLipYe6orHFY5m68NNOoLWwqEeTvutJgFeDT4WxYi0\\n\n                PJaNQYFPyGVyg7N0hCx5cGwajdnwGpb5zpSNyvG2Yes9I1O/u7+FFrbSwOuo61t1\\n\n                scGa8YlgTKoyGc9cwxl5U8krrlEwXTWQ/qF1Gq2wHG23wm1D2d2PXFDRvw3gPxJn\\n\n                yWkrx5k26ru1kguM7XFVyRi7B+uG4vdvMlxMBXM3jpH1CJRr82VvzYPv7f05Z5To\\n\n                C7XDqHpWKx3+AQvh/ZsSBpBhzK8qaixysMwnawe05rOPydWvsLlnMCGManKVnq9Y\\n\n                Wek1P2dwYT9zuroBR5nmrECY+xVWk7vhsDasKsYlQ/LdDyzSL7qh0Vq3DjcoHxLI\\n\n                uL7qQ3O0YRcKGfmQibpKdDzvIqA+48Nfh2nDnTxvfuwOxb41zdLTZQftaSXc0Xwd\\n\n                HgquBAFbRDr5TyWlUUc8iACowKkk01pEPc8coxPCp6F/hz6kgmebRevzs7sxwrS7\\n\n                aUWycSls783JC7WO267DRD30FNx+9S7SY4ECzhDGjLdne6wIoib1L9SFkk1AAKb3\\n\n                m2+6BB/HxCXtMqi95pFeCjV99bp+PBqoifx9SlFYZq9qcGDr/jyrdG8V2Wf/HF4n\\n\n                K8RIPxB+daAPMLTpj4WBhNquSE6mRQvABEf0GPi2eLA=\\n=0TDv\\n-----END PGP\n                PUBLIC KEY BLOCK-----\\n\\n\\n\"\n  /account/auth:\n    get:\n      summary: Obtain Account Authentication Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      otpEnabled:\n                        type: boolean\n                      appPasswords:\n                        type: array\n                        items: {}\n              example:\n                data:\n                  otpEnabled: false\n                  appPasswords: []\n    post:\n      summary: Update Account Authentication Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  error:\n                    type: string\n                  details:\n                    type: string\n                  reason:\n                    type: object\n                    nullable: true\n              example:\n                error: other\n                details: Fallback administrator accounts do not support 2FA or AppPasswords\n                reason:\n        \"401\":\n          description: Unauthorized\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  type:\n                    type: string\n                  status:\n                    type: number\n                  title:\n                    type: string\n                  detail:\n                    type: string\n              example:\n                type: about:blank\n                status: 401\n                title: Unauthorized\n                detail: You have to authenticate first.\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: array\n              items:\n                type: object\n                properties:\n                  type:\n                    type: string\n                  name:\n                    type: string\n                  password:\n                    type: string\n            example:\n              - type: addAppPassword\n                name: dGVzdCQyMDI1LTAxLTA1VDE0OjEyOjUxLjg0NyswMDowMA==\n                password: $6$4M/5LmG7b13r0cdE$6zb.i6wJ3pAQHA2MRHkKg0t8bgSYb2IeqiIU115t.NugwW6VXifE0VKI5n2BQUNwdeDMUzaX82TmhuVVgC0Gx1\n  /reload/:\n    get:\n      summary: Reload Settings\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      warnings:\n                        type: object\n                        properties: {}\n                      errors:\n                        type: object\n                        properties: {}\n              example:\n                data:\n                  warnings: {}\n                  errors: {}\n  /queue/status/stop:\n    patch:\n      summary: Stop Queue Processing\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: boolean\n              example:\n                data: true\n  /queue/status/start:\n    patch:\n      summary: Resume Queue Processing\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: boolean\n              example:\n                data: false\n  /queue/messages/{message_id}:\n    get:\n      summary: Obtain Queued Message Details\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      id:\n                        type: number\n                      return_path:\n                        type: string\n                      domains:\n                        type: array\n                        items:\n                          type: object\n                          properties:\n                            name:\n                              type: string\n                            status:\n                              type: string\n                            recipients:\n                              type: array\n                              items:\n                                type: object\n                                properties:\n                                  address:\n                                    type: string\n                                  status:\n                                    type: string\n                            retry_num:\n                              type: number\n                            next_retry:\n                              type: string\n                            next_notify:\n                              type: string\n                            expires:\n                              type: string\n                      created:\n                        type: string\n                      size:\n                        type: number\n                      blob_hash:\n                        type: string\n              example:\n                data:\n                  id: 217700302698266624\n                  return_path: pepe@pepe.com\n                  domains:\n                    - name: example.org\n                      status: scheduled\n                      recipients:\n                        - address: john@example.org\n                          status: scheduled\n                      retry_num: 0\n                      next_retry: \"2025-01-05T14:33:15Z\"\n                      next_notify: \"2025-01-06T14:33:15Z\"\n                      expires: \"2025-01-10T14:33:15Z\"\n                  created: \"2025-01-05T14:33:15Z\"\n                  size: 1451\n                  blob_hash: ykrZ_KghvdG2AdjH4AZajkSvZvcsxP_oI2HEZvw-tS0\n        \"404\":\n          description: Not Found\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  type:\n                    type: string\n                  status:\n                    type: number\n                  title:\n                    type: string\n                  detail:\n                    type: string\n              example:\n                type: about:blank\n                status: 404\n                title: Not Found\n                detail: The requested resource does not exist on this server.\n      parameters:\n        - name: message_id\n          in: path\n          required: true\n          schema:\n            type: string\n    patch:\n      summary: Reschedule Delivery of Queued Message\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: boolean\n              example:\n                data: true\n      parameters:\n        - name: message_id\n          in: path\n          required: true\n          schema:\n            type: string\n    delete:\n      summary: Cancel Delivery of Queued Message\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: boolean\n              example:\n                data: true\n      parameters:\n        - name: message_id\n          in: path\n          required: true\n          schema:\n            type: string\n  /store/blobs/{blob_id}:\n    get:\n      summary: Fetch Blob by ID\n      responses:\n        \"200\":\n          description: OK\n          content: {}\n      parameters:\n        - name: blob_id\n          in: path\n          required: true\n          schema:\n            type: string\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n  /telemetry/trace/{trace_id}:\n    get:\n      summary: Obtain Trace Details\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: array\n                    items:\n                      type: object\n                      properties:\n                        text:\n                          type: string\n                        details:\n                          type: string\n                        createdAt:\n                          type: string\n                        type:\n                          type: string\n                        data:\n                          type: object\n                          properties:\n                            listenerId:\n                              type: string\n                            localPort:\n                              type: number\n                            remoteIp:\n                              type: string\n                            remotePort:\n                              type: number\n              example:\n                data:\n                  - text: SMTP connection started\n                    details: A new SMTP connection was started\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.connection-start\n                    data:\n                      listenerId: smtp\n                      localPort: 25\n                      remoteIp: ::1\n                      remotePort: 57513\n                  - text: SMTP EHLO command\n                    details: The remote server sent an EHLO command\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.ehlo\n                    data:\n                      domain: test.eml\n                  - text: SPF EHLO check failed\n                    details: EHLO identity failed SPF check\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.spf-ehlo-fail\n                    data:\n                      domain: test.eml\n                      result:\n                        type: spf.none\n                        text: No SPF record\n                        details: No SPF record was found\n                        data: {}\n                      elapsed: 24\n                  - text: IPREV check passed\n                    details: Reverse IP check passed\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.iprev-pass\n                    data:\n                      domain: test.eml\n                      result:\n                        type: iprev.pass\n                        text: IPREV check passed\n                        details: The IPREV check has passed\n                        data:\n                          details:\n                            - localhost.\n                      elapsed: 0\n                  - text: SPF From check failed\n                    details: MAIL FROM identity failed SPF check\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.spf-from-fail\n                    data:\n                      domain: test.eml\n                      from: pepe@pepe.com\n                      result:\n                        type: spf.none\n                        text: No SPF record\n                        details: No SPF record was found\n                        data: {}\n                      elapsed: 18\n                  - text: SMTP MAIL FROM command\n                    details: The remote client sent a MAIL FROM command\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.mail-from\n                    data:\n                      from: pepe@pepe.com\n                  - text: SMTP RCPT TO command\n                    details: The remote client sent an RCPT TO command\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.rcpt-to\n                    data:\n                      to: john@example.org\n                  - text: DKIM verification failed\n                    details: Failed to verify DKIM signature\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.dkim-fail\n                    data:\n                      strict: false\n                      result: []\n                      elapsed: 0\n                  - text: ARC verification passed\n                    details: Successful ARC verification\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.arc-pass\n                    data:\n                      strict: false\n                      result:\n                        type: dkim.none\n                        text: No DKIM signature\n                        details: No DKIM signature was found\n                        data: {}\n                      elapsed: 0\n                  - text: DMARC check failed\n                    details: Failed to verify DMARC policy\n                    createdAt: \"2025-01-05T14:34:50Z\"\n                    type: smtp.dmarc-fail\n                    data:\n                      strict: false\n                      domain: example.org\n                      policy: reject\n                      result:\n                        type: dmarc.none\n                        text: No DMARC record\n                        details: No DMARC record was found\n                        data: {}\n                      elapsed: 0\n      parameters:\n        - name: trace_id\n          in: path\n          required: true\n          schema:\n            type: string\n  /telemetry/live/tracing-token:\n    get:\n      summary: Request a Tracing Token\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: string\n              example:\n                data: VLxkixOwgDF8Frj0wi8kPhx3SpzKqtsDvbo25wgKw2tBIz/O8La0dwioQw9pN11c/////w8Ctau1+ALxq7X4AndlYg==\n  /telemetry/traces/live:\n    get:\n      summary: Start Live Tracing\n      responses:\n        \"200\":\n          description: OK\n          content: {}\n      parameters:\n        - name: filter\n          in: query\n          required: false\n          schema:\n            type: string\n        - name: token\n          in: query\n          required: false\n          schema:\n            type: string\n  /dns/records/{domain}:\n    get:\n      summary: Obtain DNS Records for Domain\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: array\n                    items:\n                      type: object\n                      properties:\n                        type:\n                          type: string\n                        name:\n                          type: string\n                        content:\n                          type: string\n              example:\n                data:\n                  - type: MX\n                    name: example.org.\n                    content: 10 mx.fr.email.\n                  - type: CNAME\n                    name: mail.example.org.\n                    content: mx.fr.email.\n                  - type: TXT\n                    name: 202501e._domainkey.example.org.\n                    content: v=DKIM1; k=ed25519; h=sha256; p=82LqzMGRHEBI2HGDogjojWGz+Crrv0TAi8pcaOBd1vw=\n                  - type: TXT\n                    name: 202501r._domainkey.example.org.\n                    content: v=DKIM1; k=rsa; h=sha256;\n                      p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu1qtCbIlrZffIqm7gHqpihPUlxOq1zD6K3j1RO/enhkZRp5dEdCqcLbyFk5d+rqRsVIWwUZiU4HXHWqMTN1hlKojUlzmU1JYtlHRMwtM5vN4mzG4x1KA0i8ZHxkahE8ITsP+kPByDF9x0vAySHXpyErNXq3BeFyu/VW+6X+fmUW6x39PfWq7kQQTcwU0Ogo447oJfmAX9H4Z+/cD5WJVNiLgvLY6faVgoXm0mJJjRU5xoEStXoUcKwrwbl7G3K7JfxtmWsgEn97auV6v4he2LRRfTxbY9smkqUtcJs61E9iyyYroJv0iRda2pv71qg8e4wTb2sqBloZv/F2FZQhM+wIDAQAB\n                  - type: TXT\n                    name: example.org.\n                    content: v=spf1 mx ra=postmaster -all\n                  - type: SRV\n                    name: _jmap._tcp.example.org.\n                    content: 0 1 443 mx.fr.email.\n                  - type: SRV\n                    name: _imaps._tcp.example.org.\n                    content: 0 1 993 mx.fr.email.\n                  - type: SRV\n                    name: _imap._tcp.example.org.\n                    content: 0 1 143 mx.fr.email.\n                  - type: SRV\n                    name: _imap._tcp.example.org.\n                    content: 0 1 1143 mx.fr.email.\n                  - type: SRV\n                    name: _pop3s._tcp.example.org.\n                    content: 0 1 995 mx.fr.email.\n      parameters:\n        - name: domain\n          in: path\n          required: true\n          schema:\n            type: string\n  /store/purge/account/{account_id}:\n    get:\n      summary: Purge Account\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      parameters:\n        - name: account_id\n          in: path\n          required: true\n          schema:\n            type: string\n  /store/purge/in-memory/default/bayes-account/{account_id}:\n    get:\n      summary: Delete Bayes Model for Account\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n      parameters:\n        - name: account_id\n          in: path\n          required: true\n          schema:\n            type: string\n  /store/undelete/{account_id}:\n    get:\n      summary: List Deleted Messages\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    properties:\n                      items:\n                        type: array\n                        items: {}\n                      total:\n                        type: number\n              example:\n                data:\n                  items: []\n                  total: 0\n      parameters:\n        - name: account_id\n          in: path\n          required: true\n          schema:\n            type: string\n        - name: limit\n          in: query\n          required: false\n          schema:\n            type: number\n        - name: page\n          in: query\n          required: false\n          schema:\n            type: number\n    post:\n      summary: Undelete Messages\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: array\n                    items:\n                      type: object\n                      properties:\n                        type:\n                          type: string\n              example:\n                data:\n                  - type: success\n      parameters:\n        - name: account_id\n          in: path\n          required: true\n          schema:\n            type: string\n      requestBody:\n        content:\n          application/json:\n            schema:\n              type: array\n              items:\n                type: object\n                properties:\n                  hash:\n                    type: string\n                  collection:\n                    type: string\n                  restoreTime:\n                    type: string\n                  cancelDeletion:\n                    type: string\n            example:\n              - hash: 9pDYGrkDlLYuBNl062qhi0wStnDYyq4ZWalnj2vXbLY\n                collection: email\n                restoreTime: \"2025-01-05T14:50:13Z\"\n                cancelDeletion: \"2025-02-04T14:50:13Z\"\n  /queue/status:\n    get:\n      summary: Obtain Queue Status\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: boolean\n              example:\n                data: true\n  /store/purge/blob:\n    get:\n      summary: Purge Blob Store\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /store/purge/data:\n    get:\n      summary: Purge Data Store\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /store/purge/in-memory:\n    get:\n      summary: Purge In-Memory Store\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /store/purge/account:\n    get:\n      summary: Purge All Accounts\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: object\n                    nullable: true\n              example:\n                data:\n  /store/uids/{account_id}:\n    delete:\n      summary: Reset IMAP UIDs for Account\n      responses:\n        \"200\":\n          description: OK\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  data:\n                    type: array\n                    items:\n                      type: number\n              example:\n                data:\n                  - 0\n                  - 0\n      parameters:\n        - name: account_id\n          in: path\n          required: true\n          schema:\n            type: string\n"
  },
  {
    "path": "crates/cli/Cargo.toml",
    "content": "[package]\nname = \"stalwart-cli\"\ndescription = \"Stalwart Server CLI\"\nauthors = [\"Stalwart Labs LLC <hello@stalw.art>\"]\nlicense = \"AGPL-3.0-only OR LicenseRef-SEL\"\nrepository = \"https://github.com/stalwartlabs/cli\"\nhomepage = \"https://github.com/stalwartlabs/cli\"\nversion = \"0.15.5\"\nedition = \"2024\"\nreadme = \"README.md\"\n\n[dependencies]\njmap-client = { version = \"0.3\", features = [\"async\"] } \nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"serde\"] } \nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\"]}\ntokio = { version = \"1.47\", features = [\"full\"] }\nnum_cpus = \"1.13.1\"\nclap = { version = \"4.1.6\", features = [\"derive\"] }\nprettytable-rs = \"0.10.0\"\nrpassword = \"7.0\"\nindicatif = \"0.17.0\"\nconsole = { version = \"0.15\", default-features = false, features = [\"ansi-parsing\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\ncsv = \"1.1\"\nform_urlencoded = \"1.1.0\"\nhuman-size = \"0.4.2\"\nfutures = \"0.3.28\"\npwhash = \"1.0.0\"\nrand = \"0.9.0\"\nmail-auth = { version = \"0.7.1\" }\n"
  },
  {
    "path": "crates/cli/src/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    collections::HashMap,\n    fmt::Display,\n    io::{BufRead, Write},\n    time::Duration,\n};\n\nuse clap::Parser;\nuse console::style;\nuse jmap_client::client::Credentials;\nuse modules::{\n    UnwrapResult,\n    cli::{Cli, Client, Commands},\n    host, is_localhost,\n};\nuse reqwest::{Method, StatusCode, header::AUTHORIZATION};\nuse serde::{Deserialize, Serialize, de::DeserializeOwned};\n\nuse crate::modules::OAuthResponse;\n\npub mod modules;\n\n#[tokio::main]\nasync fn main() -> std::io::Result<()> {\n    let args = Cli::parse();\n    let url = args\n        .url\n        .or_else(|| std::env::var(\"URL\").ok())\n        .map(|url| url.trim_end_matches('/').to_string())\n        .unwrap_or_else(|| {\n            eprintln!(\"No URL specified. Use --url or set the URL environment variable.\");\n            std::process::exit(1);\n        });\n    let client = Client {\n        credentials: if let Some(credentials) = args.credentials {\n            parse_credentials(&credentials)\n        } else if let Ok(credentials) = std::env::var(\"CREDENTIALS\") {\n            parse_credentials(&credentials)\n        } else if args.anonymous {\n            let credentials = \"anonymous:\".to_string();\n            parse_credentials(&credentials)\n        } else {\n            let credentials = rpassword::prompt_password(\n                \"\\nEnter administrator credentials or press [ENTER] to use OAuth: \",\n            )\n            .unwrap();\n\n            if !credentials.is_empty() {\n                parse_credentials(&credentials)\n            } else {\n                oauth(&url).await\n            }\n        },\n        timeout: args.timeout,\n        url,\n    };\n\n    match args.command {\n        Commands::Import(command) => {\n            command.exec(client).await;\n        }\n        Commands::Export(command) => {\n            command.exec(client).await;\n        }\n        Commands::Server(command) => command.exec(client).await,\n        /*Commands::Account(command) => command.exec(client).await,\n        Commands::Domain(command) => command.exec(client).await,\n        Commands::List(command) => command.exec(client).await,\n        Commands::Group(command) => command.exec(client).await,*/\n        Commands::Dkim(command) => command.exec(client).await,\n        Commands::Queue(command) => command.exec(client).await,\n        Commands::Report(command) => command.exec(client).await,\n    }\n\n    Ok(())\n}\n\nfn parse_credentials(credentials: &str) -> Credentials {\n    if let Some((account, secret)) = credentials.split_once(':') {\n        Credentials::basic(account, secret)\n    } else {\n        Credentials::basic(\"admin\", credentials)\n    }\n}\n\nasync fn oauth(url: &str) -> Credentials {\n    let metadata: HashMap<String, serde_json::Value> = serde_json::from_slice(\n        &reqwest::Client::builder()\n            .danger_accept_invalid_certs(is_localhost(url))\n            .build()\n            .unwrap_or_default()\n            .get(format!(\"{}/.well-known/oauth-authorization-server\", url))\n            .send()\n            .await\n            .unwrap_result(\"send OAuth GET request\")\n            .bytes()\n            .await\n            .unwrap_result(\"fetch bytes\"),\n    )\n    .unwrap_result(\"deserialize OAuth GET response\");\n\n    let token_endpoint = metadata.property(\"token_endpoint\");\n    let mut params: HashMap<String, String> =\n        HashMap::from_iter([(\"client_id\".to_string(), \"Stalwart_CLI\".to_string())]);\n    let response: HashMap<String, serde_json::Value> = serde_json::from_slice(\n        &reqwest::Client::builder()\n            .danger_accept_invalid_certs(is_localhost(url))\n            .build()\n            .unwrap_or_default()\n            .post(metadata.property(\"device_authorization_endpoint\"))\n            .form(&params)\n            .send()\n            .await\n            .unwrap_result(\"send OAuth POST request\")\n            .bytes()\n            .await\n            .unwrap_result(\"fetch bytes\"),\n    )\n    .unwrap_result(\"deserialize OAuth POST response\");\n\n    params.insert(\n        \"grant_type\".to_string(),\n        \"urn:ietf:params:oauth:grant-type:device_code\".to_string(),\n    );\n    params.insert(\n        \"device_code\".to_string(),\n        response.property(\"device_code\").to_string(),\n    );\n\n    print!(\n        \"\\nAuthenticate this request using code {} at {}. Please ENTER when done.\",\n        style(response.property(\"user_code\")).bold(),\n        style(response.property(\"verification_uri\")).bold().dim()\n    );\n\n    std::io::stdout().flush().unwrap();\n    std::io::stdin().lock().lines().next();\n\n    let mut response: HashMap<String, serde_json::Value> = serde_json::from_slice(\n        &reqwest::Client::builder()\n            .danger_accept_invalid_certs(is_localhost(url))\n            .build()\n            .unwrap_or_default()\n            .post(token_endpoint)\n            .form(&params)\n            .send()\n            .await\n            .unwrap_result(\"send OAuth POST request\")\n            .bytes()\n            .await\n            .unwrap_result(\"fetch bytes\"),\n    )\n    .unwrap_result(\"deserialize OAuth POST response\");\n\n    if let Some(serde_json::Value::String(access_token)) = response.remove(\"access_token\") {\n        Credentials::Bearer(access_token)\n    } else {\n        eprintln!(\n            \"OAuth failed with code {}.\",\n            response\n                .get(\"error\")\n                .and_then(|s| s.as_str())\n                .unwrap_or(\"<unknown>\")\n        );\n        std::process::exit(1);\n    }\n}\n\n#[derive(Deserialize)]\n#[serde(untagged)]\npub enum Response<T> {\n    Error(ManagementApiError),\n    Data { data: T },\n}\n\n#[derive(Deserialize)]\n#[serde(tag = \"error\")]\n#[serde(rename_all = \"camelCase\")]\npub enum ManagementApiError {\n    FieldAlreadyExists { field: String, value: String },\n    FieldMissing { field: String },\n    NotFound { item: String },\n    Unsupported { details: String },\n    AssertFailed,\n    Other { details: String },\n}\n\nimpl Client {\n    pub async fn into_jmap_client(self) -> jmap_client::client::Client {\n        jmap_client::client::Client::new()\n            .credentials(self.credentials)\n            .accept_invalid_certs(is_localhost(&self.url))\n            .follow_redirects([host(&self.url).expect(\"Invalid host\").to_owned()])\n            .timeout(Duration::from_secs(self.timeout.unwrap_or(60)))\n            .connect(&self.url)\n            .await\n            .unwrap_or_else(|err| {\n                eprintln!(\"Failed to connect to JMAP server {}: {}.\", &self.url, err);\n                std::process::exit(1);\n            })\n    }\n\n    pub async fn http_request<R: DeserializeOwned, B: Serialize>(\n        &self,\n        method: Method,\n        url: &str,\n        body: Option<B>,\n    ) -> R {\n        self.try_http_request(method, url, body)\n            .await\n            .unwrap_or_else(|| {\n                eprintln!(\"Request failed: No data returned.\");\n                std::process::exit(1);\n            })\n    }\n\n    pub async fn try_http_request<R: DeserializeOwned, B: Serialize>(\n        &self,\n        method: Method,\n        url: &str,\n        body: Option<B>,\n    ) -> Option<R> {\n        let url = format!(\n            \"{}{}{}\",\n            self.url,\n            if !self.url.ends_with('/') && !url.starts_with('/') {\n                \"/\"\n            } else {\n                \"\"\n            },\n            url\n        );\n        let mut request = reqwest::Client::builder()\n            .danger_accept_invalid_certs(is_localhost(&url))\n            .timeout(Duration::from_secs(self.timeout.unwrap_or(60)))\n            .build()\n            .unwrap_or_default()\n            .request(method, url)\n            .header(\n                AUTHORIZATION,\n                match &self.credentials {\n                    Credentials::Basic(s) => format!(\"Basic {s}\"),\n                    Credentials::Bearer(s) => format!(\"Bearer {s}\"),\n                },\n            );\n\n        if let Some(body) = body {\n            request = request.body(serde_json::to_string(&body).unwrap_result(\"serialize body\"));\n        }\n\n        let response = request.send().await.unwrap_result(\"send HTTP request\");\n\n        match response.status() {\n            StatusCode::OK => (),\n            StatusCode::NOT_FOUND => {\n                return None;\n            }\n            StatusCode::UNAUTHORIZED => {\n                eprintln!(\n                    \"Authentication failed. Make sure the credentials are correct and that the account has administrator rights.\"\n                );\n                std::process::exit(1);\n            }\n            _ => {\n                eprintln!(\n                    \"Request failed: {}\",\n                    response.text().await.unwrap_result(\"fetch text\")\n                );\n                std::process::exit(1);\n            }\n        }\n\n        let bytes = response.bytes().await.unwrap_result(\"fetch bytes\");\n        match serde_json::from_slice::<Response<R>>(&bytes).unwrap_result(&format!(\n            \"deserialize response {}\",\n            String::from_utf8_lossy(bytes.as_ref())\n        )) {\n            Response::Data { data } => Some(data),\n            Response::Error(error) => {\n                eprintln!(\"Request failed: {error})\");\n                std::process::exit(1);\n            }\n        }\n    }\n}\n\nimpl Display for ManagementApiError {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            ManagementApiError::FieldAlreadyExists { field, value } => {\n                write!(f, \"Field {} already exists with value {}.\", field, value)\n            }\n            ManagementApiError::FieldMissing { field } => {\n                write!(f, \"Field {} is missing.\", field)\n            }\n            ManagementApiError::NotFound { item } => {\n                write!(f, \"{} not found.\", item)\n            }\n            ManagementApiError::Unsupported { details } => {\n                write!(f, \"Unsupported: {}\", details)\n            }\n            ManagementApiError::AssertFailed => {\n                write!(f, \"Assertion failed.\")\n            }\n            ManagementApiError::Other { details } => {\n                write!(f, \"{}\", details)\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/account.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse prettytable::{Attr, Cell, Row, Table};\nuse pwhash::sha512_crypt;\nuse reqwest::Method;\nuse serde_json::Value;\n\nuse super::{\n    Principal, PrincipalField, PrincipalUpdate, PrincipalValue, Type,\n    cli::{AccountCommands, Client},\n};\n\nimpl AccountCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            AccountCommands::Create {\n                name,\n                password,\n                description,\n                quota,\n                is_admin,\n                addresses,\n                member_of,\n            } => {\n                let principal = Principal {\n                    typ: if is_admin.unwrap_or_default() {\n                        Type::Superuser\n                    } else {\n                        Type::Individual\n                    }\n                    .into(),\n                    quota,\n                    name: name.clone().into(),\n                    secrets: vec![sha512_crypt::hash(password).unwrap()],\n                    emails: addresses.unwrap_or_default(),\n                    member_of: member_of.unwrap_or_default(),\n                    description,\n                    ..Default::default()\n                };\n                let account_id = client\n                    .http_request::<u32, _>(Method::POST, \"/api/principal\", Some(principal))\n                    .await;\n                eprintln!(\"Successfully created account {name:?} with id {account_id}.\");\n            }\n            AccountCommands::Update {\n                name,\n                new_name,\n                password,\n                description,\n                quota,\n                is_admin,\n                addresses,\n                member_of,\n            } => {\n                let mut changes = Vec::new();\n                if let Some(new_name) = new_name {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Name,\n                        PrincipalValue::String(new_name),\n                    ));\n                }\n                if let Some(password) = password {\n                    changes.push(PrincipalUpdate::add_item(\n                        PrincipalField::Secrets,\n                        PrincipalValue::String(sha512_crypt::hash(password).unwrap()),\n                    ));\n                }\n                if let Some(description) = description {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Description,\n                        PrincipalValue::String(description),\n                    ));\n                }\n                if let Some(quota) = quota {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Quota,\n                        PrincipalValue::Integer(quota),\n                    ));\n                }\n                if let Some(is_admin) = is_admin {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Type,\n                        PrincipalValue::String(\n                            if is_admin {\n                                Type::Superuser\n                            } else {\n                                Type::Individual\n                            }\n                            .to_string()\n                            .to_ascii_lowercase(),\n                        ),\n                    ));\n                }\n                if let Some(addresses) = addresses {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Emails,\n                        PrincipalValue::StringList(addresses),\n                    ));\n                }\n                if let Some(member_of) = member_of {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::MemberOf,\n                        PrincipalValue::StringList(member_of),\n                    ));\n                }\n\n                if !changes.is_empty() {\n                    client\n                        .http_request::<Value, _>(\n                            Method::PATCH,\n                            &format!(\"/api/principal/{name}\"),\n                            Some(changes),\n                        )\n                        .await;\n                    eprintln!(\"Successfully updated account {name:?}.\");\n                } else {\n                    eprintln!(\"No changes to apply.\");\n                }\n            }\n            AccountCommands::AddEmail { name, addresses } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            addresses\n                                .into_iter()\n                                .map(|address| {\n                                    PrincipalUpdate::add_item(\n                                        PrincipalField::Emails,\n                                        PrincipalValue::String(address),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated account {name:?}.\");\n            }\n            AccountCommands::RemoveEmail { name, addresses } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            addresses\n                                .into_iter()\n                                .map(|address| {\n                                    PrincipalUpdate::remove_item(\n                                        PrincipalField::Emails,\n                                        PrincipalValue::String(address),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated account {name:?}.\");\n            }\n            AccountCommands::AddToGroup { name, member_of } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            member_of\n                                .into_iter()\n                                .map(|group| {\n                                    PrincipalUpdate::add_item(\n                                        PrincipalField::MemberOf,\n                                        PrincipalValue::String(group),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated account {name:?}.\");\n            }\n            AccountCommands::RemoveFromGroup { name, member_of } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            member_of\n                                .into_iter()\n                                .map(|group| {\n                                    PrincipalUpdate::remove_item(\n                                        PrincipalField::MemberOf,\n                                        PrincipalValue::String(group),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated account {name:?}.\");\n            }\n            AccountCommands::Delete { name } => {\n                client\n                    .http_request::<Value, String>(\n                        Method::DELETE,\n                        &format!(\"/api/principal/{name}\"),\n                        None,\n                    )\n                    .await;\n                eprintln!(\"Successfully deleted account {name:?}.\");\n            }\n            AccountCommands::Display { name } => {\n                client.display_principal(&name).await;\n            }\n            AccountCommands::List {\n                filter,\n                limit,\n                page,\n            } => {\n                client\n                    .list_principals(\"individual\", \"Account\", filter, page, limit)\n                    .await;\n            }\n        }\n    }\n}\n\nimpl Client {\n    pub async fn display_principal(&self, name: &str) {\n        let principal = self\n            .http_request::<Principal, String>(Method::GET, &format!(\"/api/principal/{name}\"), None)\n            .await;\n        let mut table = Table::new();\n        if let Some(name) = principal.name {\n            table.add_row(Row::new(vec![\n                Cell::new(\"Name\").with_style(Attr::Bold),\n                Cell::new(&name),\n            ]));\n        }\n        if let Some(typ) = principal.typ {\n            table.add_row(Row::new(vec![\n                Cell::new(\"Type\").with_style(Attr::Bold),\n                Cell::new(&typ.to_string()),\n            ]));\n        }\n        if let Some(description) = principal.description {\n            table.add_row(Row::new(vec![\n                Cell::new(\"Description\").with_style(Attr::Bold),\n                Cell::new(&description),\n            ]));\n        }\n        if matches!(\n            principal.typ,\n            Some(Type::Individual | Type::Superuser | Type::Group)\n        ) {\n            if let Some(quota) = principal.quota {\n                table.add_row(Row::new(vec![\n                    Cell::new(\"Quota\").with_style(Attr::Bold),\n                    if quota != 0 {\n                        Cell::new(&quota.to_string())\n                    } else {\n                        Cell::new(\"Unlimited\")\n                    },\n                ]));\n            }\n            if let Some(used_quota) = principal.used_quota {\n                table.add_row(Row::new(vec![\n                    Cell::new(\"Used Quota\").with_style(Attr::Bold),\n                    Cell::new(&used_quota.to_string()),\n                ]));\n            }\n        }\n        if !principal.members.is_empty() {\n            table.add_row(Row::new(vec![\n                Cell::new(\"Members\").with_style(Attr::Bold),\n                Cell::new(&principal.members.join(\", \")),\n            ]));\n        }\n        if !principal.member_of.is_empty() {\n            table.add_row(Row::new(vec![\n                Cell::new(\"Member of\").with_style(Attr::Bold),\n                Cell::new(&principal.member_of.join(\", \")),\n            ]));\n        }\n        if !principal.emails.is_empty() {\n            table.add_row(Row::new(vec![\n                Cell::new(\"E-mail address(es)\").with_style(Attr::Bold),\n                Cell::new(&principal.emails.join(\", \")),\n            ]));\n        }\n        eprintln!();\n        table.printstd();\n        eprintln!();\n    }\n\n    pub async fn list_principals(\n        &self,\n        record_type: &str,\n        record_name: &str,\n        filter: Option<String>,\n        page: Option<usize>,\n        limit: Option<usize>,\n    ) {\n        let mut query = form_urlencoded::Serializer::new(\"/api/principal?\".to_string());\n\n        query.append_pair(\"type\", record_type);\n\n        if let Some(filter) = &filter {\n            query.append_pair(\"filter\", filter);\n        }\n        if let Some(limit) = limit {\n            query.append_pair(\"limit\", &limit.to_string());\n        }\n        if let Some(page) = page {\n            query.append_pair(\"page\", &page.to_string());\n        }\n\n        let results = self\n            .http_request::<ListResponse, String>(Method::GET, &query.finish(), None)\n            .await;\n        if !results.items.is_empty() {\n            let mut table = Table::new();\n            table.add_row(Row::new(vec![\n                Cell::new(&format!(\"{record_name} Name\")).with_style(Attr::Bold),\n            ]));\n\n            for item in &results.items {\n                table.add_row(Row::new(vec![Cell::new(item)]));\n            }\n\n            eprintln!();\n            table.printstd();\n            eprintln!();\n        }\n\n        eprintln!(\n            \"\\n\\n{} {}{} found.\\n\",\n            results.total,\n            record_name.to_ascii_lowercase(),\n            if results.total == 1 { \"\" } else { \"s\" }\n        );\n    }\n}\n\n#[derive(Debug, serde::Deserialize)]\nstruct ListResponse {\n    pub total: usize,\n    pub items: Vec<String>,\n}\n\nimpl Display for Type {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Type::Superuser => write!(f, \"Superuser\"),\n            Type::Individual => write!(f, \"Individual\"),\n            Type::Group => write!(f, \"Group\"),\n            Type::List => write!(f, \"List\"),\n            Type::Resource => write!(f, \"Resource\"),\n            Type::Location => write!(f, \"Location\"),\n            Type::Other => write!(f, \"Other\"),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/cli.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::dkim::Algorithm;\nuse clap::{Parser, Subcommand, ValueEnum};\nuse jmap_client::client::Credentials;\nuse mail_parser::DateTime;\nuse serde::Deserialize;\n\n#[derive(Parser)]\n#[clap(version, about, long_about = None)]\n#[clap(name = \"stalwart-cli\")]\npub struct Cli {\n    #[clap(subcommand)]\n    pub command: Commands,\n    /// Server base URL\n    #[clap(short, long)]\n    pub url: Option<String>,\n    /// Authentication credentials\n    #[clap(short, long)]\n    pub credentials: Option<String>,\n    /// Connection timeout in seconds\n    #[clap(short, long)]\n    pub timeout: Option<u64>,\n    /// Do not ask for credentials\n    #[clap(short, long)]\n    pub anonymous: bool,\n}\n\n#[derive(Subcommand)]\npub enum Commands {\n    /// Manage user accounts\n    /* #[clap(subcommand)]\n        Account(AccountCommands),\n\n        /// Manage domains\n        #[clap(subcommand)]\n        Domain(DomainCommands),\n\n        /// Manage mailing lists\n        #[clap(subcommand)]\n        List(ListCommands),\n\n        /// Manage groups\n        #[clap(subcommand)]\n        Group(GroupCommands),\n    */\n\n    /// Manage DKIM signatures\n    #[clap(subcommand)]\n    Dkim(DkimCommands),\n\n    /// Import JMAP accounts and Maildir/mbox mailboxes\n    #[clap(subcommand)]\n    Import(ImportCommands),\n\n    /// Export JMAP accounts\n    #[clap(subcommand)]\n    Export(ExportCommands),\n\n    /// Manage JMAP database\n    #[clap(subcommand)]\n    Server(ServerCommands),\n\n    /// Manage SMTP message queue\n    #[clap(subcommand)]\n    Queue(QueueCommands),\n\n    /// Manage SMTP DMARC/TLS report queue\n    #[clap(subcommand)]\n    Report(ReportCommands),\n}\n\npub struct Client {\n    pub url: String,\n    pub credentials: Credentials,\n    pub timeout: Option<u64>,\n}\n\n#[derive(Subcommand)]\npub enum AccountCommands {\n    /// Create a new user account\n    Create {\n        /// Login Name\n        name: String,\n        /// Password\n        password: String,\n        /// Account description\n        #[clap(short, long)]\n        description: Option<String>,\n        /// Quota in bytes\n        #[clap(short, long)]\n        quota: Option<u32>,\n        /// Whether the account is an administrator\n        #[clap(short, long)]\n        is_admin: Option<bool>,\n        /// E-mail addresses\n        #[clap(short, long)]\n        addresses: Option<Vec<String>>,\n        /// Groups this account is a member of\n        #[clap(short, long)]\n        member_of: Option<Vec<String>>,\n    },\n\n    /// Update an existing user account\n    Update {\n        /// Account login\n        name: String,\n        /// Rename account login\n        #[clap(short, long)]\n        new_name: Option<String>,\n        /// Update password\n        #[clap(short, long)]\n        password: Option<String>,\n        /// Update account description\n        #[clap(short, long)]\n        description: Option<String>,\n        /// Update quota in bytes\n        #[clap(short, long)]\n        quota: Option<u64>,\n        /// Whether the account is an administrator\n        #[clap(short, long)]\n        is_admin: Option<bool>,\n        /// Update e-mail addresses\n        #[clap(short, long)]\n        addresses: Option<Vec<String>>,\n        /// Update groups this account is a member of\n        #[clap(short, long)]\n        member_of: Option<Vec<String>>,\n    },\n\n    /// Add e-mail aliases to a user account\n    AddEmail {\n        /// Account login\n        name: String,\n        /// E-mail aliases to add\n        #[clap(required = true)]\n        addresses: Vec<String>,\n    },\n\n    /// Remove e-mail aliases to a user account\n    RemoveEmail {\n        /// Account login\n        name: String,\n        /// E-mail aliases to remove\n        #[clap(required = true)]\n        addresses: Vec<String>,\n    },\n\n    /// Add a user account to groups\n    AddToGroup {\n        /// Account login\n        name: String,\n        /// Groups to add\n        #[clap(required = true)]\n        member_of: Vec<String>,\n    },\n\n    /// Remove a user account from groups\n    RemoveFromGroup {\n        /// Account login\n        name: String,\n        /// Groups to remove\n        #[clap(required = true)]\n        member_of: Vec<String>,\n    },\n\n    /// Delete an existing user account\n    Delete {\n        /// Account name to delete\n        name: String,\n    },\n\n    /// Display an existing user account\n    Display {\n        /// Account name to display\n        name: String,\n    },\n\n    /// List all user accounts\n    List {\n        /// Filter accounts by keywords\n        filter: Option<String>,\n        /// Maximum number of accounts to list\n        limit: Option<usize>,\n        /// Page number\n        page: Option<usize>,\n    },\n}\n\n#[derive(Subcommand)]\npub enum ListCommands {\n    /// Create a new mailing list\n    Create {\n        /// List Name\n        name: String,\n        /// List email address\n        email: String,\n        /// Description\n        #[clap(short, long)]\n        description: Option<String>,\n        /// Mailing list members\n        #[clap(short, long)]\n        members: Option<Vec<String>>,\n    },\n\n    /// Update an existing mailing list\n    Update {\n        /// List Name\n        name: String,\n        /// Rename list\n        new_name: Option<String>,\n        /// List email address\n        email: Option<String>,\n        /// Description\n        #[clap(short, long)]\n        description: Option<String>,\n        /// Mailing list members\n        #[clap(short, long)]\n        members: Option<Vec<String>>,\n    },\n\n    /// Add members to a mailing list\n    AddMembers {\n        /// List Name\n        name: String,\n        /// Members to add\n        #[clap(required = true)]\n        members: Vec<String>,\n    },\n\n    /// Remove members from a mailing list\n    RemoveMembers {\n        /// List Name\n        name: String,\n        /// Members to remove\n        #[clap(required = true)]\n        members: Vec<String>,\n    },\n\n    /// Display an existing mailing list\n    Display {\n        /// Mailing list to display\n        name: String,\n    },\n\n    /// List all mailing lists\n    List {\n        /// Filter mailing lists by keywords\n        filter: Option<String>,\n        /// Maximum number of mailing lists to list\n        limit: Option<usize>,\n        /// Page number\n        page: Option<usize>,\n    },\n}\n\n#[derive(Subcommand)]\npub enum GroupCommands {\n    /// Create a group\n    Create {\n        /// Group Name\n        name: String,\n        /// Group email address\n        email: Option<String>,\n        /// Description\n        #[clap(short, long)]\n        description: Option<String>,\n        /// Group members\n        #[clap(short, long)]\n        members: Option<Vec<String>>,\n    },\n\n    /// Update an existing group\n    Update {\n        /// Group Name\n        name: String,\n        /// Rename group\n        new_name: Option<String>,\n        /// Group email address\n        email: Option<String>,\n        /// Description\n        #[clap(short, long)]\n        description: Option<String>,\n        /// Update groups that this group is a member of\n        #[clap(short, long)]\n        members: Option<Vec<String>>,\n    },\n\n    /// Add members to a group\n    AddMembers {\n        /// Group name\n        name: String,\n        /// Groups to add\n        #[clap(required = true)]\n        members: Vec<String>,\n    },\n\n    /// Remove members from a group\n    RemoveMembers {\n        /// Group name\n        name: String,\n        /// Groups to remove\n        #[clap(required = true)]\n        members: Vec<String>,\n    },\n\n    /// Display an existing group\n    Display {\n        /// Group name to display\n        name: String,\n    },\n\n    /// List all groups\n    List {\n        /// Filter groups by keywords\n        filter: Option<String>,\n        /// Maximum number of groups to list\n        limit: Option<usize>,\n        /// Page number\n        page: Option<usize>,\n    },\n}\n\n#[derive(Subcommand)]\npub enum DomainCommands {\n    /// Create a new domain\n    Create {\n        /// Domain name to create\n        name: String,\n    },\n\n    /// Delete an existing domain\n    Delete {\n        /// Domain name to delete\n        name: String,\n    },\n\n    /// List DNS records for domain\n    DNSRecords {\n        /// Domain name to list DNS records for\n        name: String,\n    },\n\n    /// List all domains\n    List {\n        /// Starting point for listing domains\n        from: Option<String>,\n        /// Maximum number of domains to list\n        limit: Option<usize>,\n    },\n}\n\n#[derive(Subcommand)]\npub enum DkimCommands {\n    /// Create DKIM signature\n    Create {\n        /// Algorithm to use\n        algorithm: Algorithm,\n        /// Domain name for which to create\n        domain: String,\n        /// Id\n        signature_id: Option<String>,\n        /// Selector\n        selector: Option<String>,\n    },\n\n    /// Get DKIM public key\n    GetPublicKey {\n        /// Signature id\n        signature_id: String,\n    },\n}\n\n#[derive(Subcommand)]\npub enum ImportCommands {\n    /// Import messages and folders\n    Messages {\n        #[clap(value_enum)]\n        #[clap(short, long)]\n        format: MailboxFormat,\n\n        /// Number of messages to import concurrently, defaults to the number of CPUs.\n        #[clap(short, long)]\n        num_concurrent: Option<usize>,\n\n        /// Account name or email to import messages into\n        account: String,\n\n        /// Path to the mailbox to import, or '-' for stdin (stdin only supported for mbox)\n        path: String,\n    },\n    /// Import a JMAP account\n    Account {\n        /// Number of concurrent requests, defaults to the number of CPUs.\n        #[clap(short, long)]\n        num_concurrent: Option<usize>,\n\n        /// Account name or email to import messages into\n        account: String,\n\n        /// Path to the exported account directory\n        path: String,\n    },\n}\n\n#[derive(Subcommand)]\npub enum ExportCommands {\n    /// Export a JMAP account\n    Account {\n        /// Number of concurrent blob downloads to perform, defaults to the number of CPUs.\n        #[clap(short, long)]\n        num_concurrent: Option<usize>,\n\n        /// Account name or email to import messages into\n        account: String,\n\n        /// Path to export the account to\n        path: String,\n    },\n}\n\n#[derive(Subcommand)]\npub enum ServerCommands {\n    /// Perform database maintenance\n    DatabaseMaintenance {},\n\n    /// Reload TLS certificates\n    ReloadCertificates {},\n\n    /// Reload configuration\n    ReloadConfig {},\n\n    /// Create a new configuration key\n    AddConfig {\n        /// Key to add\n        key: String,\n        /// Value to set\n        value: Option<String>,\n    },\n\n    /// Delete a configuration key or prefix\n    DeleteConfig {\n        /// Configuration key or prefix to delete\n        key: String,\n    },\n\n    /// List all configuration entries\n    ListConfig {\n        /// Prefix to filter configuration entries by\n        prefix: Option<String>,\n    },\n\n    /// Perform Healthcheck\n    Healthcheck {\n        /// Status `ready` (default) or `live` to check for\n        check: Option<String>\n    },\n}\n\n#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, ValueEnum)]\npub enum MailboxFormat {\n    /// Mbox format\n    Mbox,\n    /// Maildir and Maildir++ formats\n    Maildir,\n    /// Maildir with hierarchical folders (i.e. Dovecot)\n    MaildirNested,\n}\n\n#[derive(Subcommand)]\npub enum QueueCommands {\n    /// Shows messages queued for delivery\n    List {\n        /// Filter by sender address\n        #[clap(short, long)]\n        sender: Option<String>,\n        /// Filter by recipient\n        #[clap(short, long)]\n        rcpt: Option<String>,\n        /// Filter messages due for delivery before a certain datetime\n        #[clap(short, long)]\n        #[arg(value_parser = parse_datetime)]\n        before: Option<DateTime>,\n        /// Filter messages due for delivery after a certain datetime\n        #[clap(short, long)]\n        #[arg(value_parser = parse_datetime)]\n        after: Option<DateTime>,\n        /// Number of items to show per page\n        #[clap(short, long)]\n        page_size: Option<usize>,\n    },\n\n    /// Displays details about a queued message\n    Status {\n        #[clap(required = true)]\n        ids: Vec<String>,\n    },\n\n    /// Reschedule delivery\n    Retry {\n        /// Apply to messages matching a sender address\n        #[clap(short, long)]\n        sender: Option<String>,\n        /// Apply to a specific domain\n        #[clap(short, long)]\n        domain: Option<String>,\n        /// Apply to messages due before a certain datetime\n        #[clap(short, long)]\n        #[arg(value_parser = parse_datetime)]\n        before: Option<DateTime>,\n        /// Apply to messages due after a certain datetime\n        #[clap(short, long)]\n        #[arg(value_parser = parse_datetime)]\n        after: Option<DateTime>,\n        /// Schedule delivery at a specific time\n        #[clap(short, long)]\n        #[arg(value_parser = parse_datetime)]\n        time: Option<DateTime>,\n        // Reschedule one or multiple message ids\n        ids: Vec<String>,\n    },\n\n    /// Cancel delivery\n    Cancel {\n        /// Apply to messages matching a sender address\n        #[clap(short, long)]\n        sender: Option<String>,\n        /// Apply to specific recipients or domains\n        #[clap(short, long)]\n        rcpt: Option<String>,\n        /// Apply to messages due before a certain datetime\n        #[clap(short, long)]\n        #[arg(value_parser = parse_datetime)]\n        before: Option<DateTime>,\n        /// Apply to messages due after a certain datetime\n        #[clap(short, long)]\n        #[arg(value_parser = parse_datetime)]\n        after: Option<DateTime>,\n        // Cancel one or multiple message ids\n        ids: Vec<String>,\n    },\n}\n\n#[derive(Subcommand)]\npub enum ReportCommands {\n    /// Shows reports queued for delivery\n    List {\n        /// Filter by report domain\n        #[clap(short, long)]\n        domain: Option<String>,\n        /// Filter by report type\n        #[clap(short, long)]\n        #[clap(value_enum)]\n        format: Option<ReportFormat>,\n        /// Number of items to show per page\n        #[clap(short, long)]\n        page_size: Option<usize>,\n    },\n\n    /// Displays details about a queued report\n    Status {\n        #[clap(required = true)]\n        ids: Vec<String>,\n    },\n\n    /// Cancel report delivery\n    Cancel {\n        #[clap(required = true)]\n        ids: Vec<String>,\n    },\n}\n\n#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, ValueEnum, Deserialize)]\npub enum ReportFormat {\n    /// DMARC report\n    #[serde(rename = \"dmarc\")]\n    Dmarc,\n    /// TLS report\n    #[serde(rename = \"tls\")]\n    Tls,\n}\n\nfn parse_datetime(arg: &str) -> Result<DateTime, &'static str> {\n    if arg.contains('T') {\n        DateTime::parse_rfc3339(arg).ok_or(\"Failed to parse RFC3339 datetime\")\n    } else {\n        DateTime::parse_rfc3339(&format!(\"{arg}T00:00:00Z\"))\n            .ok_or(\"Failed to parse RFC3339 datetime\")\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/database.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::collections::HashMap;\nuse prettytable::{Attr, Cell, Row, Table};\nuse reqwest::{Method, StatusCode};\nuse serde_json::Value;\n\nuse crate::modules::{Response, UnwrapResult};\n\nuse super::cli::{Client, ServerCommands};\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum UpdateSettings {\n    Delete {\n        keys: Vec<String>,\n    },\n    Clear {\n        prefix: String,\n        #[serde(default)]\n        filter: Option<String>,\n    },\n    Insert {\n        prefix: Option<String>,\n        values: Vec<(String, String)>,\n        assert_empty: bool,\n    },\n}\n\nimpl ServerCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            ServerCommands::DatabaseMaintenance {} => {\n                client\n                    .http_request::<Value, String>(Method::GET, \"/api/store/maintenance\", None)\n                    .await;\n                eprintln!(\"Success.\");\n            }\n            ServerCommands::ReloadCertificates {} => {\n                client\n                    .http_request::<Value, String>(Method::GET, \"/api/reload/certificate\", None)\n                    .await;\n                eprintln!(\"Success.\");\n            }\n            ServerCommands::ReloadConfig {} => {\n                client\n                    .http_request::<Value, String>(Method::GET, \"/api/reload\", None)\n                    .await;\n                eprintln!(\"Success.\");\n            }\n            ServerCommands::AddConfig { key, value } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::POST,\n                        \"/api/settings\",\n                        Some(vec![UpdateSettings::Insert {\n                            prefix: None,\n                            values: vec![(key.clone(), value.unwrap_or_default())],\n                            assert_empty: false,\n                        }]),\n                    )\n                    .await;\n                eprintln!(\"Successfully added key {key}.\");\n            }\n            ServerCommands::DeleteConfig { key } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::POST,\n                        \"/api/settings\",\n                        Some(vec![UpdateSettings::Delete {\n                            keys: vec![key.clone()],\n                        }]),\n                    )\n                    .await;\n                eprintln!(\"Successfully deleted key {key}.\");\n            }\n            ServerCommands::ListConfig { prefix } => {\n                let results = client\n                    .http_request::<Response<HashMap<String, String>>, String>(\n                        Method::GET,\n                        &format!(\"/api/settings/list?prefix={}\", prefix.unwrap_or_default()),\n                        None,\n                    )\n                    .await\n                    .items;\n\n                if !results.is_empty() {\n                    let mut table = Table::new();\n                    table.add_row(Row::new(vec![\n                        Cell::new(\"Key\").with_style(Attr::Bold),\n                        Cell::new(\"Value\").with_style(Attr::Bold),\n                    ]));\n\n                    for (key, value) in &results {\n                        table.add_row(Row::new(vec![Cell::new(key), Cell::new(value)]));\n                    }\n\n                    eprintln!();\n                    table.printstd();\n                    eprintln!();\n                }\n\n                eprintln!(\n                    \"\\n\\n{} key{} found.\\n\",\n                    results.len(),\n                    if results.len() == 1 { \"\" } else { \"s\" }\n                );\n            }\n            ServerCommands::Healthcheck { check } => {\n                let response = reqwest::get(\n                    format!(\"{}/healthz/{}\",\n                            client.url,\n                            check.unwrap_or(\"ready\".to_string()))\n                ).await;\n                match response {\n                    Ok(resp) => {\n                        match resp.status() {\n                            StatusCode::OK => {\n                                eprintln!(\"Success\")\n                            },\n                            _ => {\n                                eprintln!(\n                                    \"Request failed: {}\",\n                                    resp.text().await.unwrap_result(\"fetch text\")\n                                );\n                                std::process::exit(1);\n                            }\n                        }\n                    }\n                    Err(err) => {\n                        eprintln!(\"Request failed: {}\", err);\n                        std::process::exit(1);                        \n                    }\n                }               \n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/dkim.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::cli::{Client, DkimCommands};\nuse clap::ValueEnum;\nuse reqwest::Method;\nuse serde::{Deserialize, Serialize};\nuse serde_json::Value;\n\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ValueEnum)]\npub enum Algorithm {\n    /// RSA\n    #[default]\n    Rsa,\n    /// ED25519\n    Ed25519,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize)]\nstruct DkimSignature {\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub id: Option<String>,\n\n    pub algorithm: Algorithm,\n\n    pub domain: String,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub selector: Option<String>,\n}\n\nimpl DkimCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            DkimCommands::Create {\n                signature_id,\n                algorithm,\n                domain,\n                selector,\n            } => {\n                let signature_req = DkimSignature {\n                    id: signature_id,\n                    algorithm,\n                    domain: domain.clone(),\n                    selector,\n                };\n                client\n                    .http_request::<Value, _>(Method::POST, \"/api/dkim\", Some(signature_req))\n                    .await;\n                eprintln!(\"Successfully created {algorithm:?} signature for domain {domain:?}\");\n            }\n            DkimCommands::GetPublicKey { signature_id } => {\n                let response = client\n                    .http_request::<Value, String>(\n                        Method::GET,\n                        &format!(\"/api/dkim/{signature_id}\"),\n                        None,\n                    )\n                    .await;\n\n                eprintln!();\n                eprintln!(\"Public DKIM key for signature {signature_id}: {response}\");\n                eprintln!();\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/domain.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse prettytable::{Attr, Cell, Row, Table, format};\nuse reqwest::Method;\nuse serde_json::Value;\n\nuse crate::modules::List;\n\nuse super::cli::{Client, DomainCommands};\n\nuse serde::{Deserialize, Serialize};\n#[derive(Debug, Serialize, Deserialize, Clone)]\nstruct DnsRecord {\n    #[serde(rename = \"type\")]\n    typ: String,\n    name: String,\n    content: String,\n}\n\nimpl DomainCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            DomainCommands::Create { name } => {\n                client\n                    .http_request::<Value, String>(\n                        Method::POST,\n                        &format!(\"/api/domain/{name}\"),\n                        None,\n                    )\n                    .await;\n                eprintln!(\"Successfully created domain {name:?}\");\n            }\n            DomainCommands::Delete { name } => {\n                client\n                    .http_request::<Value, String>(\n                        Method::DELETE,\n                        &format!(\"/api/domain/{name}\"),\n                        None,\n                    )\n                    .await;\n                eprintln!(\"Successfully deleted domain {name:?}\");\n            }\n            DomainCommands::DNSRecords { name } => {\n                let records = client\n                    .http_request::<Vec<DnsRecord>, String>(\n                        Method::GET,\n                        &format!(\"/api/domain/{name}\"),\n                        None,\n                    )\n                    .await;\n\n                if !records.is_empty() {\n                    let mut table = Table::new();\n                    // no borderline separator separator, as long values will mess it up\n                    table.set_format(*format::consts::FORMAT_NO_BORDER_LINE_SEPARATOR);\n\n                    table.add_row(Row::new(vec![\n                        Cell::new(\"Type\").with_style(Attr::Bold),\n                        Cell::new(\"Name\").with_style(Attr::Bold),\n                        Cell::new(\"Contents\").with_style(Attr::Bold),\n                    ]));\n\n                    for record in &records {\n                        table.add_row(Row::new(vec![\n                            Cell::new(&record.typ),\n                            Cell::new(&record.name),\n                            Cell::new(&record.content),\n                        ]));\n                    }\n\n                    eprintln!();\n                    table.printstd();\n                    eprintln!();\n                }\n            }\n            DomainCommands::List { from, limit } => {\n                let query = if from.is_none() && limit.is_none() {\n                    Cow::Borrowed(\"/api/domain\")\n                } else {\n                    let mut query = \"/api/domain?\".to_string();\n                    if let Some(from) = &from {\n                        query.push_str(&format!(\"from={from}\"));\n                    }\n                    if let Some(limit) = limit {\n                        query.push_str(&format!(\n                            \"{}limit={limit}\",\n                            if from.is_some() { \"&\" } else { \"\" }\n                        ));\n                    }\n                    Cow::Owned(query)\n                };\n\n                let domains = client\n                    .http_request::<List<String>, String>(Method::GET, query.as_ref(), None)\n                    .await;\n                if !domains.items.is_empty() {\n                    let mut table = Table::new();\n                    table.add_row(Row::new(vec![\n                        Cell::new(\"Domain Name\").with_style(Attr::Bold),\n                    ]));\n\n                    for domain in &domains.items {\n                        table.add_row(Row::new(vec![Cell::new(domain)]));\n                    }\n\n                    eprintln!();\n                    table.printstd();\n                    eprintln!();\n                }\n\n                eprintln!(\n                    \"\\n\\n{} domain{} found.\\n\",\n                    domains.total,\n                    if domains.total == 1 { \"\" } else { \"s\" }\n                );\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/export.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    path::{Path, PathBuf},\n    sync::Arc,\n};\n\nuse futures::{StreamExt, stream::FuturesUnordered};\nuse jmap_client::{\n    email::{self, Email},\n    identity::{self, Identity},\n    mailbox::{self, Mailbox},\n    sieve::{self, SieveScript},\n    vacation_response::{self, VacationResponse},\n};\nuse serde::Serialize;\nuse tokio::io::AsyncWriteExt;\n\nuse crate::modules::RETRY_ATTEMPTS;\n\nuse super::{\n    UnwrapResult,\n    cli::{Client, ExportCommands},\n    name_to_id,\n};\n\nimpl ExportCommands {\n    pub async fn exec(self, client: Client) {\n        let mut client = client.into_jmap_client().await;\n        match self {\n            ExportCommands::Account {\n                num_concurrent,\n                account,\n                path,\n            } => {\n                client.set_default_account_id(name_to_id(&client, &account).await);\n                let max_objects_in_get = client\n                    .session()\n                    .core_capabilities()\n                    .map(|c| c.max_objects_in_get())\n                    .unwrap_or(500);\n\n                // Create directory\n                let mut path = PathBuf::from(path);\n                if !path.is_dir() {\n                    eprintln!(\"Directory {} does not exist.\", path.display());\n                    std::process::exit(1);\n                }\n                path.push(&account);\n                if !path.is_dir() {\n                    std::fs::create_dir(&path).unwrap_or_else(|_| {\n                        eprintln!(\"Failed to create directory: {}\", path.display());\n                        std::process::exit(1);\n                    });\n                }\n\n                // Export metadata\n                let mut blobs = Vec::new();\n                export_mailboxes(&client, max_objects_in_get, &path).await;\n                export_emails(&client, max_objects_in_get, &mut blobs, &path).await;\n                export_sieve_scripts(&client, max_objects_in_get, &mut blobs, &path).await;\n                export_identities(&client, &path).await;\n                export_vacation_responses(&client, &path).await;\n\n                // Export blobs\n                path.push(\"blobs\");\n                if !path.exists() {\n                    std::fs::create_dir(&path).unwrap_or_else(|_| {\n                        eprintln!(\"Failed to create directory: {}\", path.display());\n                        std::process::exit(1);\n                    });\n                }\n                let client = Arc::new(client);\n                let num_concurrent = num_concurrent.unwrap_or_else(num_cpus::get);\n                let mut futures = FuturesUnordered::new();\n                eprintln!(\"Exporting {} blobs...\", blobs.len());\n                for blob_id in blobs {\n                    let client = client.clone();\n                    let mut blob_path = path.clone();\n                    blob_path.push(&blob_id);\n\n                    if tokio::fs::metadata(&blob_path).await.is_err() {\n                        futures.push(async move {\n                            let mut retry_count = 0;\n\n                            let bytes = loop {\n                                match client.download(&blob_id).await {\n                                    Ok(bytes) => break bytes,\n                                    Err(_) if retry_count < RETRY_ATTEMPTS => {\n                                        tokio::time::sleep(std::time::Duration::from_secs(1)).await;\n                                        retry_count += 1;\n                                    }\n                                    result => {\n                                        result.unwrap_result(\"download blob\");\n                                        return;\n                                    }\n                                }\n                            };\n\n                            tokio::fs::OpenOptions::new()\n                                .create(true)\n                                .write(true)\n                                .truncate(true)\n                                .open(&blob_path)\n                                .await\n                                .unwrap_result(&format!(\"open {}\", blob_path.display()))\n                                .write_all(&bytes)\n                                .await\n                                .unwrap_result(&format!(\"write {}\", blob_path.display()));\n                        });\n                    }\n\n                    if futures.len() == num_concurrent {\n                        futures.next().await.unwrap();\n                    }\n                }\n\n                // Wait for remaining futures\n                while futures.next().await.is_some() {}\n            }\n        }\n    }\n}\n\npub async fn fetch_mailboxes(\n    client: &jmap_client::client::Client,\n    max_objects_in_get: usize,\n) -> Vec<Mailbox> {\n    let mut position = 0;\n    let mut results = Vec::new();\n    loop {\n        let mut request = client.build();\n        let query_result = request\n            .query_mailbox()\n            .calculate_total(true)\n            .position(position)\n            .limit(max_objects_in_get)\n            .result_reference();\n        request.get_mailbox().ids_ref(query_result).properties([\n            mailbox::Property::Id,\n            mailbox::Property::Name,\n            mailbox::Property::IsSubscribed,\n            mailbox::Property::ParentId,\n            mailbox::Property::Role,\n            mailbox::Property::SortOrder,\n            mailbox::Property::ShareWith,\n        ]);\n\n        let mut response = request\n            .send()\n            .await\n            .unwrap_result(\"send JMAP request\")\n            .unwrap_method_responses();\n        if response.len() != 2 {\n            eprintln!(\"Invalid response while fetching mailboxes\");\n            std::process::exit(1);\n        }\n        let mut get_response = response\n            .pop()\n            .unwrap()\n            .unwrap_get_mailbox()\n            .unwrap_result(\"fetch mailboxes\");\n        let mailboxes_part = get_response.take_list();\n        let total_mailboxes = response\n            .pop()\n            .unwrap()\n            .unwrap_query_mailbox()\n            .unwrap_result(\"query mailboxes\")\n            .total()\n            .unwrap_or(0);\n\n        let mailboxes_part_len = mailboxes_part.len();\n        if mailboxes_part_len > 0 {\n            results.extend(mailboxes_part);\n            if results.len() < total_mailboxes {\n                position += mailboxes_part_len as i32;\n                continue;\n            }\n        }\n        break;\n    }\n    results\n}\n\nasync fn export_mailboxes(\n    client: &jmap_client::client::Client,\n    max_objects_in_get: usize,\n    path: &Path,\n) {\n    eprintln!(\n        \"Exported {} mailboxes.\",\n        write_file(\n            path,\n            \"mailboxes.json\",\n            fetch_mailboxes(client, max_objects_in_get).await,\n        )\n        .await\n    );\n}\n\npub async fn fetch_emails(\n    client: &jmap_client::client::Client,\n    max_objects_in_get: usize,\n) -> Vec<Email> {\n    let mut position = 0;\n    let mut results = Vec::new();\n\n    loop {\n        let mut request = client.build();\n        let query_result = request\n            .query_email()\n            .calculate_total(true)\n            .position(position)\n            .limit(max_objects_in_get)\n            .result_reference();\n        request.get_email().ids_ref(query_result).properties([\n            email::Property::Id,\n            email::Property::MailboxIds,\n            email::Property::Keywords,\n            email::Property::ReceivedAt,\n            email::Property::BlobId,\n            email::Property::MessageId,\n        ]);\n\n        let mut response = request\n            .send()\n            .await\n            .unwrap_result(\"send JMAP request\")\n            .unwrap_method_responses();\n        if response.len() != 2 {\n            eprintln!(\"Invalid response while fetching emails\");\n            std::process::exit(1);\n        }\n        let mut get_response = response\n            .pop()\n            .unwrap()\n            .unwrap_get_email()\n            .unwrap_result(\"fetch emails\");\n        let emails_part = get_response.take_list();\n        let total_emails = response\n            .pop()\n            .unwrap()\n            .unwrap_query_email()\n            .unwrap_result(\"query emails\")\n            .total()\n            .unwrap_or(0);\n\n        let emails_part_len = emails_part.len();\n        if emails_part_len > 0 {\n            results.extend(emails_part);\n            if results.len() < total_emails {\n                position += emails_part_len as i32;\n                continue;\n            }\n        }\n        break;\n    }\n\n    results\n}\n\nasync fn export_emails(\n    client: &jmap_client::client::Client,\n    max_objects_in_get: usize,\n    blobs: &mut Vec<String>,\n    path: &Path,\n) {\n    let emails = fetch_emails(client, max_objects_in_get).await;\n\n    for email in &emails {\n        if let Some(blob_id) = email.blob_id() {\n            blobs.push(blob_id.to_string());\n        } else {\n            eprintln!(\n                \"Warning: email {:?} has no blobId\",\n                email.id().unwrap_or_default()\n            );\n        }\n    }\n\n    eprintln!(\n        \"Exported {} emails.\",\n        write_file(path, \"emails.json\", emails,).await\n    );\n}\n\npub async fn fetch_sieve_scripts(\n    client: &jmap_client::client::Client,\n    max_objects_in_get: usize,\n) -> Vec<SieveScript> {\n    let mut position = 0;\n    let mut results = Vec::new();\n\n    loop {\n        let mut request = client.build();\n        let query_result = request\n            .query_sieve_script()\n            .calculate_total(true)\n            .position(position)\n            .limit(max_objects_in_get)\n            .result_reference();\n        request\n            .get_sieve_script()\n            .ids_ref(query_result)\n            .properties([\n                sieve::Property::Id,\n                sieve::Property::Name,\n                sieve::Property::BlobId,\n                sieve::Property::IsActive,\n            ]);\n\n        let mut response = request\n            .send()\n            .await\n            .unwrap_result(\"send JMAP request\")\n            .unwrap_method_responses();\n        if response.len() != 2 {\n            eprintln!(\"Invalid response while fetching sieve_scripts\");\n            std::process::exit(1);\n        }\n        let mut get_response = response\n            .pop()\n            .unwrap()\n            .unwrap_get_sieve_script()\n            .unwrap_result(\"fetch sieve_scripts\");\n        let sieve_scripts_part = get_response.take_list();\n        let total_sieve_scripts = response\n            .pop()\n            .unwrap()\n            .unwrap_query_sieve_script()\n            .unwrap_result(\"query sieve_scripts\")\n            .total()\n            .unwrap_or(0);\n\n        let sieve_scripts_part_len = sieve_scripts_part.len();\n        if sieve_scripts_part_len > 0 {\n            results.extend(sieve_scripts_part);\n\n            if results.len() < total_sieve_scripts {\n                position += sieve_scripts_part_len as i32;\n                continue;\n            }\n        }\n        break;\n    }\n    results\n}\n\nasync fn export_sieve_scripts(\n    client: &jmap_client::client::Client,\n    max_objects_in_get: usize,\n    blobs: &mut Vec<String>,\n    path: &Path,\n) {\n    let sieves = fetch_sieve_scripts(client, max_objects_in_get).await;\n    for sieve in &sieves {\n        if let Some(blob_id) = sieve.blob_id() {\n            blobs.push(blob_id.to_string());\n        } else {\n            eprintln!(\n                \"Warning: sieve script {:?} has no blobId\",\n                sieve.id().unwrap_or_default()\n            );\n        }\n    }\n\n    eprintln!(\n        \"Exported {} sieve scripts.\",\n        write_file(path, \"sieve.json\", sieves,).await\n    );\n}\n\npub async fn fetch_identities(client: &jmap_client::client::Client) -> Vec<Identity> {\n    let mut request = client.build();\n    request.get_identity().properties([\n        identity::Property::Id,\n        identity::Property::Name,\n        identity::Property::Email,\n        identity::Property::ReplyTo,\n        identity::Property::Bcc,\n        identity::Property::TextSignature,\n        identity::Property::HtmlSignature,\n    ]);\n    request\n        .send_get_identity()\n        .await\n        .unwrap_result(\"send JMAP request\")\n        .take_list()\n}\n\nasync fn export_identities(client: &jmap_client::client::Client, path: &Path) {\n    eprintln!(\n        \"Exported {} identities.\",\n        write_file(path, \"identities.json\", fetch_identities(client).await).await\n    );\n}\n\npub async fn fetch_vacation_responses(\n    client: &jmap_client::client::Client,\n) -> Vec<VacationResponse> {\n    let mut request = client.build();\n    request.get_vacation_response().properties([\n        vacation_response::Property::Id,\n        vacation_response::Property::FromDate,\n        vacation_response::Property::ToDate,\n        vacation_response::Property::Subject,\n        vacation_response::Property::TextBody,\n        vacation_response::Property::HtmlBody,\n        vacation_response::Property::IsEnabled,\n    ]);\n    request\n        .send_get_vacation_response()\n        .await\n        .unwrap_result(\"send JMAP request\")\n        .take_list()\n}\n\nasync fn export_vacation_responses(client: &jmap_client::client::Client, path: &Path) {\n    eprintln!(\n        \"Exported {} vacation responses.\",\n        write_file(\n            path,\n            \"vacation.json\",\n            fetch_vacation_responses(client).await\n        )\n        .await\n    );\n}\n\nasync fn write_file<T: Serialize>(path: &Path, name: &str, contents: Vec<T>) -> usize {\n    let mut path = PathBuf::from(path);\n    path.push(name);\n    let len = contents.len();\n    tokio::fs::OpenOptions::new()\n        .create(true)\n        .write(true)\n        .truncate(true)\n        .open(&path)\n        .await\n        .unwrap_result(&format!(\"open {}\", path.display()))\n        .write_all(serde_json::to_string(&contents).unwrap().as_bytes())\n        .await\n        .unwrap_result(&format!(\"write to {}\", path.display()));\n    len\n}\n"
  },
  {
    "path": "crates/cli/src/modules/group.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::vec;\n\nuse reqwest::Method;\nuse serde_json::Value;\n\nuse crate::modules::{Principal, Type};\n\nuse super::{\n    PrincipalField, PrincipalUpdate, PrincipalValue,\n    cli::{Client, GroupCommands},\n};\n\nimpl GroupCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            GroupCommands::Create {\n                name,\n                email,\n                description,\n                members,\n            } => {\n                let principal = Principal {\n                    typ: Some(Type::Group),\n                    name: name.clone().into(),\n                    emails: email.map(|e| vec![e]).unwrap_or_default(),\n                    description,\n                    ..Default::default()\n                };\n                let account_id = client\n                    .http_request::<u32, _>(Method::POST, \"/api/principal\", Some(principal))\n                    .await;\n                if let Some(members) = members {\n                    client\n                        .http_request::<Value, _>(\n                            Method::PATCH,\n                            &format!(\"/api/principal/{name}\"),\n                            Some(vec![PrincipalUpdate::set(\n                                PrincipalField::Members,\n                                PrincipalValue::StringList(members),\n                            )]),\n                        )\n                        .await;\n                }\n                eprintln!(\"Successfully created group {name:?} with id {account_id}.\");\n            }\n            GroupCommands::Update {\n                name,\n                new_name,\n                email,\n                description,\n                members,\n            } => {\n                let mut changes = Vec::new();\n                if let Some(new_name) = new_name {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Name,\n                        PrincipalValue::String(new_name),\n                    ));\n                }\n                if let Some(email) = email {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Emails,\n                        PrincipalValue::StringList(vec![email]),\n                    ));\n                }\n                if let Some(members) = members {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Members,\n                        PrincipalValue::StringList(members),\n                    ));\n                }\n                if let Some(description) = description {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Description,\n                        PrincipalValue::String(description),\n                    ));\n                }\n\n                if !changes.is_empty() {\n                    client\n                        .http_request::<Value, _>(\n                            Method::PATCH,\n                            &format!(\"/api/principal/{name}\"),\n                            Some(changes),\n                        )\n                        .await;\n                    eprintln!(\"Successfully updated group {name:?}.\");\n                } else {\n                    eprintln!(\"No changes to apply.\");\n                }\n            }\n            GroupCommands::AddMembers { name, members } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            members\n                                .into_iter()\n                                .map(|group| {\n                                    PrincipalUpdate::add_item(\n                                        PrincipalField::Members,\n                                        PrincipalValue::String(group),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated group {name:?}.\");\n            }\n            GroupCommands::RemoveMembers { name, members } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            members\n                                .into_iter()\n                                .map(|group| {\n                                    PrincipalUpdate::remove_item(\n                                        PrincipalField::Members,\n                                        PrincipalValue::String(group),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated group {name:?}.\");\n            }\n            GroupCommands::Display { name } => {\n                client.display_principal(&name).await;\n            }\n            GroupCommands::List {\n                filter,\n                limit,\n                page,\n            } => {\n                client\n                    .list_principals(\"group\", \"Group\", filter, page, limit)\n                    .await;\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/import.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    collections::{HashMap, HashSet},\n    io::{self, Cursor},\n    path::{Path, PathBuf},\n    sync::{\n        Arc, Mutex,\n        atomic::{AtomicUsize, Ordering},\n    },\n    time::Duration,\n};\n\nuse console::style;\nuse futures::{StreamExt, stream::FuturesUnordered};\nuse indicatif::{MultiProgress, ProgressBar, ProgressStyle};\nuse jmap_client::{\n    core::set::SetObject,\n    mailbox::{self, Role},\n};\nuse mail_parser::mailbox::{\n    maildir,\n    mbox::{self, MessageIterator},\n};\nuse rand::Rng;\nuse serde::de::DeserializeOwned;\nuse tokio::{fs::File, io::AsyncReadExt};\n\nuse crate::modules::{RETRY_ATTEMPTS, UnwrapResult, name_to_id};\n\nuse super::{\n    cli::{Client, ImportCommands, MailboxFormat},\n    export::{\n        fetch_emails, fetch_identities, fetch_mailboxes, fetch_sieve_scripts,\n        fetch_vacation_responses,\n    },\n    read_file,\n};\n\nenum Mailbox {\n    Mbox(mbox::MessageIterator<Cursor<Vec<u8>>>),\n    Maildir(maildir::MessageIterator),\n    None,\n}\n\n#[derive(Debug)]\nenum MailboxId<'x> {\n    ExistingId(&'x str),\n    CreateId(String),\n    None,\n}\n\n#[derive(Debug)]\nstruct Message {\n    identifier: String,\n    flags: Vec<maildir::Flag>,\n    internal_date: u64,\n    contents: Vec<u8>,\n}\nimpl ImportCommands {\n    pub async fn exec(self, client: Client) {\n        let mut client = client.into_jmap_client().await;\n\n        match self {\n            ImportCommands::Messages {\n                num_concurrent,\n                format,\n                account,\n                path,\n            } => {\n                client.set_default_account_id(name_to_id(&client, &account).await);\n                let mut create_mailboxes = Vec::new();\n                let mut create_mailbox_names = Vec::new();\n                let mut create_mailbox_ids = Vec::new();\n\n                eprintln!(\"{} Parsing mailbox...\", style(\"[1/4]\").bold().dim(),);\n\n                match format {\n                    MailboxFormat::Mbox => {\n                        create_mailbox_names.push(Vec::new());\n                        create_mailboxes.push(Mailbox::Mbox(MessageIterator::new(Cursor::new(\n                            read_file(&path),\n                        ))));\n                    }\n                    MailboxFormat::Maildir | MailboxFormat::MaildirNested => {\n                        let (folder_sep, folder_split) = if format == MailboxFormat::Maildir {\n                            (Some(\".\"), \".\")\n                        } else {\n                            (None, \"/\")\n                        };\n\n                        for folder in maildir::FolderIterator::new(path, folder_sep)\n                            .unwrap_result(\"read Maildir folder\")\n                        {\n                            let folder = folder.unwrap_result(\"read Maildir folder\");\n                            if let Some(folder_name) = folder.name() {\n                                let mut folder_parts = Vec::new();\n                                for folder_name in folder_name.split(folder_split) {\n                                    let mut folder_name = folder_name.trim();\n                                    if folder_name.is_empty() {\n                                        folder_name = \".\";\n                                    }\n                                    folder_parts.push(folder_name.to_string());\n                                    if !create_mailbox_names.contains(&folder_parts) {\n                                        create_mailboxes.push(Mailbox::None);\n                                        create_mailbox_names.push(folder_parts.clone());\n                                    }\n                                }\n\n                                *create_mailboxes.last_mut().unwrap() = Mailbox::Maildir(folder);\n                            } else {\n                                create_mailboxes.push(Mailbox::Maildir(folder));\n                                create_mailbox_names.push(Vec::new());\n                            };\n                        }\n                    }\n                }\n\n                // Fetch all mailboxes for the account\n                eprintln!(\n                    \"{} Fetching existing mailboxes for account...\",\n                    style(\"[2/4]\").bold().dim(),\n                );\n\n                let mut inbox_id = None;\n                let mut mailbox_ids = HashMap::new();\n                let mut children: HashMap<Option<&str>, Vec<&str>> =\n                    HashMap::from_iter([(None, Vec::new())]);\n                let mut request = client.build();\n                request.get_mailbox().properties([\n                    mailbox::Property::Name,\n                    mailbox::Property::ParentId,\n                    mailbox::Property::Role,\n                    mailbox::Property::Id,\n                ]);\n                let response = request\n                    .send_get_mailbox()\n                    .await\n                    .unwrap_result(\"fetch mailboxes\");\n                for mailbox in response.list() {\n                    let mailbox_id = mailbox.id().unwrap();\n                    if mailbox.role() == Role::Inbox {\n                        inbox_id = mailbox_id.into();\n                    }\n                    children\n                        .entry(mailbox.parent_id())\n                        .or_default()\n                        .push(mailbox_id);\n                    mailbox_ids.insert(mailbox_id, mailbox.name().unwrap_or(\"Untitled\"));\n                }\n                let inbox_id = inbox_id\n                    .unwrap_result(\"locate Inbox on account, please check the server logs.\");\n                let mut it = children.get(&None).unwrap().iter();\n                let mut it_stack = Vec::new();\n                let mut name_stack = Vec::new();\n                let mut mailbox_names = HashMap::with_capacity(mailbox_ids.len());\n\n                // Build mailbox hierarchy on the server\n                eprintln!(\n                    \"{} Creating missing mailboxes...\",\n                    style(\"[3/4]\").bold().dim(),\n                );\n\n                loop {\n                    while let Some(mailbox_id) = it.next() {\n                        let name = mailbox_ids[mailbox_id];\n                        let mut mailbox_name = name_stack.clone();\n                        mailbox_name.push(name.to_string());\n\n                        mailbox_names.insert(mailbox_name, mailbox_id);\n                        if let Some(next_it) = children.get(&Some(mailbox_id)).map(|c| c.iter()) {\n                            name_stack.push(name.to_string());\n                            it_stack.push(it);\n                            it = next_it;\n                        }\n                    }\n\n                    if let Some(prev_it) = it_stack.pop() {\n                        name_stack.pop();\n                        it = prev_it;\n                    } else {\n                        break;\n                    }\n                }\n\n                // Check whether the mailboxes to be created already exist\n                let mut has_missing_mailboxes = false;\n                for mailbox_name in &create_mailbox_names {\n                    create_mailbox_ids.push(if !mailbox_name.is_empty() {\n                        if let Some(mailbox_id) = mailbox_names.get(mailbox_name) {\n                            MailboxId::ExistingId(mailbox_id)\n                        } else {\n                            has_missing_mailboxes = true;\n                            MailboxId::None\n                        }\n                    } else {\n                        MailboxId::ExistingId(inbox_id)\n                    });\n                }\n\n                // Create any missing mailboxes\n                if has_missing_mailboxes {\n                    let mut request = client.build();\n                    let set_request = request.set_mailbox();\n\n                    for pos in 0..create_mailbox_ids.len() {\n                        if let MailboxId::None = create_mailbox_ids[pos] {\n                            let mailbox_name = &create_mailbox_names[pos];\n                            let create_request =\n                                set_request.create().name(mailbox_name.last().unwrap());\n\n                            if mailbox_name.len() > 1 {\n                                let parent_mailbox_name = &mailbox_name[..mailbox_name.len() - 1];\n                                let parent_mailbox_pos = create_mailbox_names\n                                    .iter()\n                                    .position(|n| n == parent_mailbox_name)\n                                    .unwrap();\n                                match &create_mailbox_ids[parent_mailbox_pos] {\n                                    MailboxId::ExistingId(id) => {\n                                        create_request.parent_id((*id).into());\n                                    }\n                                    MailboxId::CreateId(id_ref) => {\n                                        create_request.parent_id_ref(id_ref);\n                                    }\n                                    MailboxId::None => unreachable!(),\n                                }\n                            } else {\n                                create_request.parent_id(None::<String>);\n                            }\n                            create_mailbox_ids[pos] =\n                                MailboxId::CreateId(create_request.create_id().unwrap());\n                        }\n                    }\n\n                    // Create mailboxes\n                    let mut response = request\n                        .send_set_mailbox()\n                        .await\n                        .unwrap_result(\"create mailboxes\");\n                    for create_mailbox_id in create_mailbox_ids.iter_mut() {\n                        if let MailboxId::CreateId(id) = create_mailbox_id {\n                            *id = response\n                                .created(id)\n                                .unwrap_result(\"create mailbox\")\n                                .take_id();\n                        }\n                    }\n                }\n\n                // Import messages\n                eprintln!(\"{} Importing messages...\", style(\"[4/4]\").bold().dim(),);\n\n                let client = Arc::new(client);\n                let total_imported = Arc::new(AtomicUsize::from(0));\n                let m = MultiProgress::new();\n                let num_concurrent = num_concurrent.unwrap_or_else(num_cpus::get);\n                let spinner_style =\n                    ProgressStyle::with_template(\"{prefix:.bold.dim} {spinner} {wide_msg}\")\n                        .unwrap()\n                        .tick_chars(\"⠁⠂⠄⡀⢀⠠⠐⠈ \");\n                let pbs = Arc::new(Mutex::new((\n                    (0..num_concurrent)\n                        .map(|n| {\n                            let pb = m.add(ProgressBar::new(40));\n                            pb.set_style(spinner_style.clone());\n                            pb.set_prefix(format!(\"[{}/?]\", n + 1));\n                            pb\n                        })\n                        .collect::<Vec<_>>(),\n                    0usize,\n                )));\n                let failures = Arc::new(Mutex::new(Vec::new()));\n                let mut message_num = 0;\n\n                for ((mut mailbox, mailbox_id), mailbox_name) in create_mailboxes\n                    .into_iter()\n                    .zip(create_mailbox_ids)\n                    .zip(create_mailbox_names)\n                {\n                    let mut futures = FuturesUnordered::new();\n                    let mailbox_id = Arc::new(match mailbox_id {\n                        MailboxId::ExistingId(id) => id.to_string(),\n                        MailboxId::CreateId(id) => id,\n                        MailboxId::None => unreachable!(),\n                    });\n                    let mailbox_name = Arc::new(if !mailbox_name.is_empty() {\n                        mailbox_name.join(\"/\")\n                    } else {\n                        \"Inbox\".to_string()\n                    });\n\n                    for result in mailbox.by_ref() {\n                        match result {\n                            Ok(message) => {\n                                message_num += 1;\n                                let client = client.clone();\n                                let mailbox_id = mailbox_id.clone();\n                                let mailbox_name = mailbox_name.clone();\n                                let total_imported = total_imported.clone();\n                                let pbs = pbs.clone();\n                                let failures = failures.clone();\n\n                                futures.push(async move {\n                                    // Update progress bar\n                                    {\n                                        let mut pbs = pbs.lock().unwrap();\n                                        let pb = &pbs.0[pbs.1 % pbs.0.len()];\n                                        pb.set_message(format!(\n                                            \"Importing {}: {}/{}\",\n                                            message_num, mailbox_name, message.identifier\n                                        ));\n                                        pb.inc(1);\n                                        pbs.1 += 1;\n                                    }\n\n                                    let mut retry_count = 0;\n                                    loop {\n                                        // Sanitize message\n                                        let mut contents =\n                                            Vec::with_capacity(message.contents.len());\n                                        let mut last_ch = 0;\n                                        for &ch in message.contents.iter() {\n                                            if ch == b'\\n' && last_ch != b'\\r' {\n                                                contents.push(b'\\r');\n                                            }\n                                            contents.push(ch);\n                                            last_ch = ch;\n                                        }\n\n                                        match client\n                                            .email_import(\n                                                contents,\n                                                [mailbox_id.as_ref()],\n                                                if !message.flags.is_empty() {\n                                                    message\n                                                        .flags\n                                                        .iter()\n                                                        .map(|f| match f {\n                                                            maildir::Flag::Passed => \"$passed\",\n                                                            maildir::Flag::Replied => \"$answered\",\n                                                            maildir::Flag::Seen => \"$seen\",\n                                                            maildir::Flag::Trashed => \"$deleted\",\n                                                            maildir::Flag::Draft => \"$draft\",\n                                                            maildir::Flag::Flagged => \"$flagged\",\n                                                        })\n                                                        .into()\n                                                } else {\n                                                    None\n                                                },\n                                                if message.internal_date > 0 {\n                                                    (message.internal_date as i64).into()\n                                                } else {\n                                                    None\n                                                },\n                                            )\n                                            .await\n                                        {\n                                            Ok(_) => {\n                                                total_imported.fetch_add(1, Ordering::Relaxed);\n                                            }\n                                            Err(_) if retry_count < RETRY_ATTEMPTS => {\n                                                let backoff = rand::rng().random_range(50..=300);\n                                                tokio::time::sleep(Duration::from_millis(backoff))\n                                                    .await;\n                                                retry_count += 1;\n                                                continue;\n                                            }\n                                            Err(err) => {\n                                                failures.lock().unwrap().push(format!(\n                                                    concat!(\n                                                        \"Failed to import message {} \",\n                                                        \"with identifier '{}': {}\"\n                                                    ),\n                                                    message_num, message.identifier, err\n                                                ));\n                                            }\n                                        }\n                                        break;\n                                    }\n                                });\n\n                                if futures.len() == num_concurrent {\n                                    futures.next().await.unwrap();\n                                }\n                            }\n                            Err(e) => {\n                                failures\n                                    .lock()\n                                    .unwrap()\n                                    .push(format!(\"I/O error reading message: {}\", e));\n                            }\n                        }\n                    }\n\n                    // Wait for remaining futures\n                    while futures.next().await.is_some() {}\n                }\n\n                // Done\n                for pb in pbs.lock().unwrap().0.iter() {\n                    pb.finish_with_message(\"Done\");\n                }\n                let failures = failures.lock().unwrap();\n                eprintln!(\n                    \"\\n\\nSuccessfully imported {} messages.\\n\",\n                    total_imported.load(Ordering::Relaxed)\n                );\n\n                if !failures.is_empty() {\n                    eprintln!(\"There were {} failures:\\n\", failures.len());\n                    for failure in failures.iter() {\n                        eprintln!(\"{}\", failure);\n                    }\n                }\n            }\n\n            ImportCommands::Account {\n                num_concurrent,\n                account,\n                path,\n            } => {\n                client.set_default_account_id(name_to_id(&client, &account).await);\n                let path = PathBuf::from(path);\n                if !path.exists() {\n                    eprintln!(\"Path '{}' does not exist.\", path.display());\n                    return;\n                }\n                let num_concurrent = num_concurrent.unwrap_or_else(num_cpus::get);\n\n                // Import objects\n                import_emails(\n                    &client,\n                    &path,\n                    import_mailboxes(&client, &path).await.into(),\n                    num_concurrent,\n                )\n                .await;\n                import_sieve_scripts(&client, &path, num_concurrent).await;\n                import_identities(&client, &path).await;\n                import_vacation_responses(&client, &path).await;\n            }\n        }\n    }\n}\n\nasync fn import_mailboxes(\n    client: &jmap_client::client::Client,\n    path: &Path,\n) -> HashMap<String, String> {\n    // Deserialize mailboxes\n    let mailboxes = read_json::<jmap_client::mailbox::Mailbox>(path, \"mailboxes.json\").await;\n    if mailboxes.is_empty() {\n        return HashMap::new();\n    }\n\n    // Obtain current mailboxes\n    let existing_mailboxes = fetch_mailboxes(\n        client,\n        client\n            .session()\n            .core_capabilities()\n            .map(|c| c.max_objects_in_get())\n            .unwrap_or(500),\n    )\n    .await;\n    let nested_existing_mailboxes = build_mailbox_tree(&existing_mailboxes);\n    let mut id_mappings: HashMap<String, String> = HashMap::new();\n    let mut id_missing = Vec::new();\n    for (path, mailbox) in build_mailbox_tree(&mailboxes) {\n        let id = mailbox.id().unwrap_result(\"obtain mailbox id\");\n        // Find existing mailbox based on role\n        if !matches!(mailbox.role(), Role::None)\n            && let Some(existing_mailbox) = existing_mailboxes\n                .iter()\n                .find(|m| m.role() == mailbox.role())\n        {\n            id_mappings.insert(\n                id.to_string(),\n                existing_mailbox\n                    .id()\n                    .unwrap_result(\"obtain mailbox id\")\n                    .to_string(),\n            );\n            continue;\n        }\n\n        // Find existing mailbox by name\n        if let Some(mailbox) = nested_existing_mailboxes.get(&path) {\n            id_mappings.insert(\n                id.to_string(),\n                mailbox.id().unwrap_result(\"obtain mailbox id\").to_string(),\n            );\n        } else {\n            id_missing.push(id);\n        }\n    }\n    let mut total_imported = 0;\n    let mut total_existing = 0;\n    if !id_missing.is_empty() {\n        let mut request = client.build();\n        let set_request = request.set_mailbox();\n\n        for mailbox in &mailboxes {\n            // Skip if mailbox already exists\n            let id = mailbox.id().unwrap_result(\"obtain mailbox id\").to_string();\n            if id_mappings.contains_key(&id) {\n                total_existing += 1;\n                continue;\n            }\n            let create_request = set_request\n                .create_with_id(&id)\n                .name(mailbox.name().unwrap())\n                .role(mailbox.role());\n            if let Some(parent_id) = mailbox.parent_id() {\n                if let Some(existing_id) = id_mappings.get(parent_id) {\n                    create_request.parent_id(Some(existing_id.to_string()));\n                } else {\n                    create_request.parent_id_ref(parent_id);\n                }\n            } else {\n                create_request.parent_id(None::<String>);\n            }\n            if mailbox.sort_order() > 0 {\n                create_request.sort_order(mailbox.sort_order());\n            }\n            /*if let Some(acls) = mailbox.acl() {\n                create_request.acls(acls.clone().into_iter());\n            }*/\n            if mailbox.is_subscribed() {\n                create_request.is_subscribed(true);\n            }\n        }\n\n        // Create mailboxes\n        let mut response = request\n            .send_set_mailbox()\n            .await\n            .unwrap_result(\"create mailboxes\");\n        for missing_id in id_missing {\n            id_mappings.insert(\n                missing_id.to_string(),\n                response\n                    .created(missing_id)\n                    .unwrap_result(\"create mailbox\")\n                    .take_id(),\n            );\n            total_imported += 1;\n        }\n    } else {\n        total_existing = mailboxes.len();\n    }\n\n    eprintln!(\n        \"Successfully processed {} mailboxes ({} imported, {} already exist).\",\n        total_existing + total_imported,\n        total_imported,\n        total_existing\n    );\n\n    id_mappings\n}\n\nasync fn import_emails(\n    client: &jmap_client::client::Client,\n    path: &Path,\n    mailbox_ids: Arc<HashMap<String, String>>,\n    num_concurrent: usize,\n) {\n    // Deserialize emails\n    let emails = read_json::<jmap_client::email::Email>(path, \"emails.json\").await;\n    if emails.is_empty() {\n        return;\n    }\n\n    // Obtain existing emails\n    let existing_emails = fetch_emails(\n        client,\n        client\n            .session()\n            .core_capabilities()\n            .map(|c| c.max_objects_in_get())\n            .unwrap_or(500),\n    )\n    .await;\n    let existing_ids = existing_emails\n        .iter()\n        .map(|email| (email.message_id(), email.received_at()))\n        .collect::<HashSet<_>>();\n    let mut futures = FuturesUnordered::new();\n    let total_imported = Arc::new(AtomicUsize::from(0));\n    let mut total_existing = 0;\n    let mut path = PathBuf::from(path);\n    path.push(\"blobs\");\n\n    for email in emails {\n        // Skip messages that already exist in the server\n        if existing_ids.contains(&(email.message_id(), email.received_at())) {\n            total_existing += 1;\n            continue;\n        }\n\n        // Spawn import tasks\n        let mailbox_ids = mailbox_ids.clone();\n        let mut path = path.clone();\n        let total_imported = total_imported.clone();\n\n        futures.push(async move {\n            // Obtain mailbox ids\n            let id = if let Some(id) = email.id() {\n                id\n            } else {\n                eprintln!(\"Skipping email with no id\");\n                return;\n            };\n            if email.mailbox_ids().is_empty() {\n                eprintln!(\"Skipping emailId {id} with no mailboxIds\");\n                return;\n            }\n            let mut mailboxes = Vec::with_capacity(email.mailbox_ids().len());\n            for mailbox_id in email.mailbox_ids() {\n                if let Some(mailbox_id) = mailbox_ids.get(mailbox_id) {\n                    mailboxes.push(mailbox_id.to_string());\n                } else {\n                    eprintln!(\"Skipping emailId {id} with unknown mailboxId {mailbox_id}\");\n                    return;\n                }\n            }\n            let keywords = email.keywords();\n\n            // Read blob\n            if let Some(blob_id) = email.blob_id() {\n                path.push(blob_id);\n            } else {\n                eprintln!(\"Skipping emailId {id} with no blobId\");\n                return;\n            }\n            let mut contents = vec![];\n            match File::open(&path).await {\n                Ok(mut file) => match file.read_to_end(&mut contents).await {\n                    Ok(_) => {}\n                    Err(err) => {\n                        eprintln!(\n                            \"Failed to read blob file for emailId {id} at {path:?}: {err}\",\n                            id = id,\n                            path = path,\n                            err = err\n                        );\n                        return;\n                    }\n                },\n                Err(err) => {\n                    eprintln!(\n                        \"Failed to open blob file for emailId {id} at {path:?}: {err}\",\n                        id = id,\n                        path = path,\n                        err = err\n                    );\n                    return;\n                }\n            }\n\n            let mut retry_count = 0;\n            loop {\n                match client\n                    .email_import(\n                        contents.clone(),\n                        mailboxes.clone(),\n                        if !keywords.is_empty() {\n                            Some(keywords.clone())\n                        } else {\n                            None\n                        },\n                        email.received_at(),\n                    )\n                    .await\n                {\n                    Ok(_) => {\n                        total_imported.fetch_add(1, Ordering::Relaxed);\n                    }\n                    Err(_) if retry_count < RETRY_ATTEMPTS => {\n                        retry_count += 1;\n                        continue;\n                    }\n                    Err(err) => {\n                        eprintln!(\"Failed to import emailId {id}: {err}\");\n                    }\n                }\n                break;\n            }\n        });\n\n        if futures.len() == num_concurrent {\n            futures.next().await.unwrap();\n        }\n    }\n\n    // Wait for remaining futures\n    while futures.next().await.is_some() {}\n\n    // Done\n    eprintln!(\n        \"Successfully processed {} emails ({} imported, {} already exist).\",\n        total_imported.load(Ordering::Relaxed) + total_existing,\n        total_imported.load(Ordering::Relaxed),\n        total_existing\n    );\n}\n\nasync fn import_sieve_scripts(\n    client: &jmap_client::client::Client,\n    path: &Path,\n    num_concurrent: usize,\n) {\n    // Deserialize scripts\n    let scripts = read_json::<jmap_client::sieve::SieveScript>(path, \"sieve.json\").await;\n    if scripts.is_empty() {\n        return;\n    }\n    let existing_scripts = fetch_sieve_scripts(\n        client,\n        client\n            .session()\n            .core_capabilities()\n            .map(|c| c.max_objects_in_get())\n            .unwrap_or(500),\n    )\n    .await;\n    let mut path = PathBuf::from(path);\n    path.push(\"blobs\");\n\n    // Spawn tasks\n    let mut futures = FuturesUnordered::new();\n    let total_imported = Arc::new(AtomicUsize::from(0));\n    let mut total_existing = 0;\n\n    'outer: for script in scripts {\n        // Skip scripts that already exist\n        for existing_script in &existing_scripts {\n            if existing_script.name() == script.name() {\n                total_existing += 1;\n                continue 'outer;\n            }\n        }\n        let mut path = path.clone();\n        let total_imported = total_imported.clone();\n\n        futures.push(async move {\n            let id = if let Some(id) = script.id() {\n                id\n            } else {\n                eprintln!(\"Skipping script with no id.\");\n                return;\n            };\n\n            // Read blob\n            let name = if let (Some(blob_id), Some(name)) = (script.blob_id(), script.name()) {\n                path.push(blob_id);\n                name\n            } else {\n                eprintln!(\"Skipping script {id} with no blobId and/or name\");\n                return;\n            };\n            let mut contents = vec![];\n            match File::open(&path).await {\n                Ok(mut file) => match file.read_to_end(&mut contents).await {\n                    Ok(_) => {}\n                    Err(err) => {\n                        eprintln!(\n                            \"Failed to read blob file for script {id} at {path:?}: {err}\",\n                            id = id,\n                            path = path,\n                            err = err\n                        );\n                        return;\n                    }\n                },\n                Err(err) => {\n                    eprintln!(\n                        \"Failed to open blob file for script {id} at {path:?}: {err}\",\n                        id = id,\n                        path = path,\n                        err = err\n                    );\n                    return;\n                }\n            }\n\n            // Upload blob\n            match client\n                .sieve_script_create(name, contents, script.is_active())\n                .await\n            {\n                Ok(_) => {\n                    total_imported.fetch_add(1, Ordering::Relaxed);\n                }\n                Err(err) => {\n                    eprintln!(\"Failed to import script {id}: {err}\");\n                }\n            }\n        });\n\n        if futures.len() == num_concurrent {\n            futures.next().await.unwrap();\n        }\n    }\n\n    // Wait for remaining futures\n    while futures.next().await.is_some() {}\n\n    // Done\n    eprintln!(\n        \"Successfully processed {} sieve scripts ({} imported, {} already exist).\",\n        total_imported.load(Ordering::Relaxed) + total_existing,\n        total_imported.load(Ordering::Relaxed),\n        total_existing\n    );\n}\n\nasync fn import_identities(client: &jmap_client::client::Client, path: &Path) {\n    // Deserialize mailboxes\n    let identities = read_json::<jmap_client::identity::Identity>(path, \"identities.json\").await;\n    if identities.is_empty() {\n        return;\n    }\n    let existing_identities = fetch_identities(client).await;\n    let mut request = client.build();\n    let set_request = request.set_identity();\n    let mut create_ids = Vec::new();\n    let mut total_existing = 0;\n\n    'outer: for identity in &identities {\n        for existing_identity in &existing_identities {\n            if identity.name() == existing_identity.name()\n                && identity.email() == existing_identity.email()\n            {\n                total_existing += 1;\n                continue 'outer;\n            }\n        }\n\n        if let (Some(id), Some(name), Some(email)) =\n            (identity.id(), identity.name(), identity.email())\n        {\n            if name != \"vacation\" {\n                create_ids.push(id);\n                let create_request = set_request.create_with_id(id).name(name).email(email);\n                if let Some(reply_to) = identity.reply_to() {\n                    create_request.reply_to(reply_to.iter().cloned().into());\n                }\n                if let Some(bcc) = identity.bcc() {\n                    create_request.bcc(bcc.iter().cloned().into());\n                }\n                if let Some(html_signature) = identity.html_signature() {\n                    create_request.html_signature(html_signature);\n                }\n                if let Some(text_signature) = identity.text_signature() {\n                    create_request.text_signature(text_signature);\n                }\n            }\n        } else {\n            eprintln!(\"Skipping identity with no id, name, and/or email.\");\n            continue;\n        }\n    }\n\n    let mut total_imported = 0;\n    if !create_ids.is_empty() {\n        match request.send_set_identity().await {\n            Ok(mut response) => {\n                for id in create_ids {\n                    if let Err(err) = response.created(id) {\n                        eprintln!(\"Failed to import identity {id}: {err}\");\n                    } else {\n                        total_imported += 1;\n                    }\n                }\n            }\n            Err(err) => {\n                eprintln!(\"Failed to import identities: {err}\");\n                return;\n            }\n        }\n    }\n\n    eprintln!(\n        \"Successfully processed {} identities ({} imported, {} already exist).\",\n        total_imported + total_existing,\n        total_imported,\n        total_existing\n    );\n}\n\nasync fn import_vacation_responses(client: &jmap_client::client::Client, path: &Path) {\n    // Deserialize mailboxes\n    let vacation_responses =\n        read_json::<jmap_client::vacation_response::VacationResponse>(path, \"vacation.json\").await;\n    if vacation_responses.is_empty() {\n        return;\n    }\n    let existing_vacation_responses = fetch_vacation_responses(client).await;\n    if !existing_vacation_responses.is_empty() {\n        eprintln!(\"Successfully processed 1 vacation response (0 imported, 1 already exist).\",);\n        return;\n    }\n\n    let vacation_response = vacation_responses.into_iter().next().unwrap();\n    let mut request = client.build();\n    let set_request = request.set_vacation_response().create();\n\n    if vacation_response.is_enabled() {\n        set_request.is_enabled(true);\n    }\n    if let Some(from_date) = vacation_response.from_date() {\n        set_request.from_date(from_date.into());\n    }\n    if let Some(to_date) = vacation_response.to_date() {\n        set_request.to_date(to_date.into());\n    }\n    if let Some(subject) = vacation_response.subject() {\n        set_request.subject(subject.into());\n    }\n    if let Some(text_body) = vacation_response.text_body() {\n        set_request.text_body(text_body.into());\n    }\n    if let Some(html_body) = vacation_response.html_body() {\n        set_request.html_body(html_body.into());\n    }\n    let create_id = set_request.create_id().unwrap();\n\n    match request.send_set_vacation_response().await {\n        Ok(mut response) => {\n            if let Err(err) = response.created(&create_id) {\n                eprintln!(\"Failed to import vacation response: {err}\");\n            } else {\n                eprintln!(\n                    \"Successfully processed 1 vacation response (1 imported, 0 already exist).\",\n                );\n            }\n        }\n        Err(err) => {\n            eprintln!(\"Failed to import vacation response: {err}\");\n        }\n    }\n}\n\nfn build_mailbox_tree(\n    mailboxes: &[jmap_client::mailbox::Mailbox],\n) -> HashMap<Vec<&str>, &jmap_client::mailbox::Mailbox> {\n    let mut path = Vec::new();\n    let mut parent_id = None;\n    let mut mailboxes_iter = mailboxes.iter();\n    let mut stack = Vec::new();\n    let mut results = HashMap::with_capacity(mailboxes.len());\n    let parents = mailboxes\n        .iter()\n        .map(|m| m.parent_id())\n        .collect::<HashSet<_>>();\n\n    'outer: loop {\n        while let Some(mailbox) = mailboxes_iter.next() {\n            if parent_id == mailbox.parent_id() {\n                let name = mailbox.name().unwrap_result(\"obtain mailbox name\");\n                if parents.contains(&mailbox.id()) {\n                    stack.push((path.clone(), parent_id, mailboxes_iter));\n                    parent_id = mailbox.id();\n                    path.push(name);\n                    results.insert(path.clone(), mailbox);\n                    mailboxes_iter = mailboxes.iter();\n                    continue 'outer;\n                } else {\n                    let mut path = path.clone();\n                    path.push(name);\n                    results.insert(path, mailbox);\n                }\n            }\n        }\n        if let Some((prev_path, prev_parent_id, prev_iter)) = stack.pop() {\n            parent_id = prev_parent_id;\n            path = prev_path;\n            mailboxes_iter = prev_iter;\n        } else {\n            break;\n        }\n    }\n    debug_assert_eq!(results.len(), mailboxes.len());\n\n    results\n}\n\nasync fn read_json<T: DeserializeOwned>(path: &Path, filename: &str) -> Vec<T> {\n    let mut path = PathBuf::from(path);\n    path.push(filename);\n    if path.exists() {\n        let mut file = File::open(path).await.unwrap_result(\"open file\");\n        let mut contents = String::new();\n        file.read_to_string(&mut contents)\n            .await\n            .unwrap_result(\"read file\");\n        serde_json::from_str(&contents).unwrap_result(\"parse JSON\")\n    } else {\n        Vec::new()\n    }\n}\n\nimpl Iterator for Mailbox {\n    type Item = io::Result<Message>;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        match self {\n            Mailbox::Mbox(it) => it.next().map(|r| {\n                r.map(|m| Message {\n                    identifier: m.from().to_string(),\n                    flags: Vec::new(),\n                    internal_date: m.internal_date(),\n                    contents: m.unwrap_contents(),\n                })\n                .map_err(|_| std::io::Error::other(\"Failed to parse from mbox file.\"))\n            }),\n            Mailbox::Maildir(it) => it.next().map(|r| {\n                r.map(|m| Message {\n                    identifier: m\n                        .path()\n                        .file_name()\n                        .and_then(|f| f.to_str())\n                        .unwrap_or(\"unknown\")\n                        .to_string(),\n                    flags: m.flags().to_vec(),\n                    internal_date: m.internal_date(),\n                    contents: m.unwrap_contents(),\n                })\n            }),\n            Mailbox::None => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/list.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::vec;\n\nuse reqwest::Method;\nuse serde_json::Value;\n\nuse crate::modules::{Principal, Type};\n\nuse super::{\n    PrincipalField, PrincipalUpdate, PrincipalValue,\n    cli::{Client, ListCommands},\n};\n\nimpl ListCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            ListCommands::Create {\n                name,\n                email,\n                description,\n                members,\n            } => {\n                let principal = Principal {\n                    typ: Some(Type::List),\n                    name: name.clone().into(),\n                    emails: vec![email],\n                    description,\n                    ..Default::default()\n                };\n                let account_id = client\n                    .http_request::<u32, _>(Method::POST, \"/api/principal\", Some(principal))\n                    .await;\n                if let Some(members) = members {\n                    client\n                        .http_request::<Value, _>(\n                            Method::PATCH,\n                            &format!(\"/api/principal/{name}\"),\n                            Some(vec![PrincipalUpdate::set(\n                                PrincipalField::Members,\n                                PrincipalValue::StringList(members),\n                            )]),\n                        )\n                        .await;\n                }\n                eprintln!(\"Successfully created mailing list {name:?} with id {account_id}.\");\n            }\n            ListCommands::Update {\n                name,\n                new_name,\n                email,\n                description,\n                members,\n            } => {\n                let mut changes = Vec::new();\n                if let Some(new_name) = new_name {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Name,\n                        PrincipalValue::String(new_name),\n                    ));\n                }\n                if let Some(email) = email {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Emails,\n                        PrincipalValue::StringList(vec![email]),\n                    ));\n                }\n                if let Some(members) = members {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Members,\n                        PrincipalValue::StringList(members),\n                    ));\n                }\n                if let Some(description) = description {\n                    changes.push(PrincipalUpdate::set(\n                        PrincipalField::Description,\n                        PrincipalValue::String(description),\n                    ));\n                }\n\n                if !changes.is_empty() {\n                    client\n                        .http_request::<Value, _>(\n                            Method::PATCH,\n                            &format!(\"/api/principal/{name}\"),\n                            Some(changes),\n                        )\n                        .await;\n                    eprintln!(\"Successfully updated mailing list {name:?}.\");\n                } else {\n                    eprintln!(\"No changes to apply.\");\n                }\n            }\n            ListCommands::AddMembers { name, members } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            members\n                                .into_iter()\n                                .map(|group| {\n                                    PrincipalUpdate::add_item(\n                                        PrincipalField::Members,\n                                        PrincipalValue::String(group),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated mailing list {name:?}.\");\n            }\n            ListCommands::RemoveMembers { name, members } => {\n                client\n                    .http_request::<Value, _>(\n                        Method::PATCH,\n                        &format!(\"/api/principal/{name}\"),\n                        Some(\n                            members\n                                .into_iter()\n                                .map(|group| {\n                                    PrincipalUpdate::remove_item(\n                                        PrincipalField::Members,\n                                        PrincipalValue::String(group),\n                                    )\n                                })\n                                .collect::<Vec<_>>(),\n                        ),\n                    )\n                    .await;\n                eprintln!(\"Successfully updated mailing list {name:?}.\");\n            }\n            ListCommands::Display { name } => {\n                client.display_principal(&name).await;\n            }\n            ListCommands::List {\n                filter,\n                limit,\n                page,\n            } => {\n                client\n                    .list_principals(\"list\", \"Mailing List\", filter, page, limit)\n                    .await;\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{collections::HashMap, fmt::Display, io::Read};\n\nuse jmap_client::{\n    client::Client,\n    principal::query::{self},\n};\nuse serde::{Deserialize, Serialize};\n\npub mod account;\npub mod cli;\npub mod database;\npub mod dkim;\npub mod domain;\npub mod export;\npub mod group;\npub mod import;\npub mod list;\npub mod queue;\npub mod report;\n\nconst RETRY_ATTEMPTS: usize = 5;\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)]\npub struct Principal {\n    #[serde(default, skip_serializing_if = \"Option::is_none\")]\n    pub id: Option<u32>,\n\n    #[serde(rename = \"type\")]\n    pub typ: Option<Type>,\n\n    #[serde(default, skip_serializing_if = \"Option::is_none\")]\n    pub quota: Option<u32>,\n\n    #[serde(rename = \"usedQuota\")]\n    #[serde(default, skip_serializing_if = \"Option::is_none\")]\n    pub used_quota: Option<u64>,\n\n    #[serde(default, skip_serializing_if = \"Option::is_none\")]\n    pub name: Option<String>,\n\n    #[serde(default, skip_serializing_if = \"Vec::is_empty\")]\n    pub secrets: Vec<String>,\n\n    #[serde(default, skip_serializing_if = \"Vec::is_empty\")]\n    pub emails: Vec<String>,\n\n    #[serde(default, skip_serializing_if = \"Vec::is_empty\")]\n    #[serde(rename = \"memberOf\")]\n    pub member_of: Vec<String>,\n\n    #[serde(default, skip_serializing_if = \"Vec::is_empty\")]\n    #[serde(rename = \"members\")]\n    pub members: Vec<String>,\n\n    #[serde(default, skip_serializing_if = \"Option::is_none\")]\n    pub description: Option<String>,\n}\n\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]\npub enum Type {\n    #[serde(rename = \"individual\")]\n    #[default]\n    Individual = 0,\n    #[serde(rename = \"group\")]\n    Group = 1,\n    #[serde(rename = \"resource\")]\n    Resource = 2,\n    #[serde(rename = \"location\")]\n    Location = 3,\n    #[serde(rename = \"superuser\")]\n    Superuser = 4,\n    #[serde(rename = \"list\")]\n    List = 5,\n    #[serde(rename = \"other\")]\n    Other = 6,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]\npub enum PrincipalField {\n    #[serde(rename = \"name\")]\n    Name,\n    #[serde(rename = \"type\")]\n    Type,\n    #[serde(rename = \"quota\")]\n    Quota,\n    #[serde(rename = \"description\")]\n    Description,\n    #[serde(rename = \"secrets\")]\n    Secrets,\n    #[serde(rename = \"emails\")]\n    Emails,\n    #[serde(rename = \"memberOf\")]\n    MemberOf,\n    #[serde(rename = \"members\")]\n    Members,\n}\n\n#[derive(Clone, serde::Serialize, serde::Deserialize, Default)]\npub struct List<T> {\n    pub items: Vec<T>,\n    pub total: u64,\n}\n\n#[derive(Clone, serde::Serialize, serde::Deserialize, Default)]\npub struct Response<T> {\n    pub items: T,\n    pub total: u64,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]\npub struct PrincipalUpdate {\n    action: PrincipalAction,\n    field: PrincipalField,\n    value: PrincipalValue,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]\npub enum PrincipalAction {\n    #[serde(rename = \"set\")]\n    Set,\n    #[serde(rename = \"addItem\")]\n    AddItem,\n    #[serde(rename = \"removeItem\")]\n    RemoveItem,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]\n#[serde(untagged)]\npub enum PrincipalValue {\n    String(String),\n    StringList(Vec<String>),\n    Integer(u64),\n}\n\nimpl PrincipalUpdate {\n    pub fn set(field: PrincipalField, value: PrincipalValue) -> PrincipalUpdate {\n        PrincipalUpdate {\n            action: PrincipalAction::Set,\n            field,\n            value,\n        }\n    }\n\n    pub fn add_item(field: PrincipalField, value: PrincipalValue) -> PrincipalUpdate {\n        PrincipalUpdate {\n            action: PrincipalAction::AddItem,\n            field,\n            value,\n        }\n    }\n\n    pub fn remove_item(field: PrincipalField, value: PrincipalValue) -> PrincipalUpdate {\n        PrincipalUpdate {\n            action: PrincipalAction::RemoveItem,\n            field,\n            value,\n        }\n    }\n}\n\npub trait UnwrapResult<T> {\n    fn unwrap_result(self, action: &str) -> T;\n}\n\nimpl<T> UnwrapResult<T> for Option<T> {\n    fn unwrap_result(self, message: &str) -> T {\n        match self {\n            Some(result) => result,\n            None => {\n                eprintln!(\"Failed to {}\", message);\n                std::process::exit(1);\n            }\n        }\n    }\n}\n\nimpl<T, E: Display> UnwrapResult<T> for Result<T, E> {\n    fn unwrap_result(self, message: &str) -> T {\n        match self {\n            Ok(result) => result,\n            Err(err) => {\n                eprintln!(\"Failed to {}: {}\", message, err);\n                std::process::exit(1);\n            }\n        }\n    }\n}\n\npub fn read_file(path: &str) -> Vec<u8> {\n    if path == \"-\" {\n        let mut stdin = std::io::stdin().lock();\n        let mut raw_message = Vec::with_capacity(1024);\n        let mut buf = [0; 1024];\n        loop {\n            let n = stdin.read(&mut buf).unwrap();\n            if n == 0 {\n                break;\n            }\n            raw_message.extend_from_slice(&buf[..n]);\n        }\n        raw_message\n    } else {\n        std::fs::read(path).unwrap_or_else(|_| {\n            eprintln!(\"Failed to read file: {}\", path);\n            std::process::exit(1);\n        })\n    }\n}\n\npub async fn name_to_id(client: &Client, name: &str) -> String {\n    let filter = if name.contains('@') {\n        query::Filter::email(name)\n    } else {\n        query::Filter::name(name)\n    };\n    let mut response = client\n        .principal_query(filter.into(), None::<Vec<_>>)\n        .await\n        .unwrap_result(\"query principals\");\n    match response.ids().len() {\n        1 => response.take_ids().pop().unwrap(),\n        0 => {\n            eprintln!(\"Error: No principal found with name '{}'.\", name);\n            std::process::exit(1);\n        }\n        _ => {\n            eprintln!(\"Error: Multiple principals found with name '{}'.\", name);\n            std::process::exit(1);\n        }\n    }\n}\n\npub fn host(url: &str) -> Option<&str> {\n    url.split_once(\"://\")\n        .map(|(_, url)| url.split_once('/').map_or(url, |(host, _)| host))\n        .map(|host| host.rsplit_once(':').map_or(host, |(host, _)| host))\n}\n\npub fn is_localhost(url: &str) -> bool {\n    host(url).is_some_and(|host| host == \"localhost\" || host == \"127.0.0.1\" || host == \"[::1]\")\n}\n\npub trait OAuthResponse {\n    fn property(&self, name: &str) -> &str;\n}\n\nimpl OAuthResponse for HashMap<String, serde_json::Value> {\n    fn property(&self, name: &str) -> &str {\n        self.get(name)\n            .unwrap_result(&format!(\"find '{}' in OAuth response\", name))\n            .as_str()\n            .unwrap_result(&format!(\"invalid '{}' value\", name))\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/queue.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    List,\n    cli::{Client, QueueCommands},\n};\nuse console::Term;\nuse human_size::{Byte, SpecificSize};\nuse mail_parser::DateTime;\nuse prettytable::{Attr, Cell, Row, Table, format::Alignment};\nuse reqwest::Method;\nuse serde::{Deserialize, Deserializer};\n\n#[derive(Debug, Deserialize, PartialEq, Eq)]\npub struct Message {\n    pub id: u64,\n    pub return_path: String,\n    pub recipients: Vec<Recipient>,\n    #[serde(deserialize_with = \"deserialize_datetime\")]\n    pub created: DateTime,\n    pub size: usize,\n    #[serde(default)]\n    pub priority: i16,\n    #[serde(default)]\n    pub env_id: Option<String>,\n    pub blob_hash: String,\n}\n\n#[derive(Debug, Deserialize, PartialEq, Eq)]\npub struct Recipient {\n    pub address: String,\n    pub status: Status,\n    pub queue: String,\n    pub retry_num: u32,\n    #[serde(deserialize_with = \"deserialize_maybe_datetime\", default)]\n    pub next_retry: Option<DateTime>,\n    #[serde(deserialize_with = \"deserialize_maybe_datetime\", default)]\n    pub next_notify: Option<DateTime>,\n    #[serde(deserialize_with = \"deserialize_maybe_datetime\", default)]\n    pub expires: Option<DateTime>,\n    #[serde(default)]\n    pub orcpt: Option<String>,\n}\n\n#[derive(Debug, PartialEq, Eq, Deserialize)]\npub enum Status {\n    #[serde(rename = \"scheduled\")]\n    Scheduled,\n    #[serde(rename = \"completed\")]\n    Completed(String),\n    #[serde(rename = \"temp_fail\")]\n    TemporaryFailure(String),\n    #[serde(rename = \"perm_fail\")]\n    PermanentFailure(String),\n}\n\nimpl QueueCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            QueueCommands::List {\n                sender,\n                rcpt,\n                before,\n                after,\n                page_size,\n            } => {\n                let stdout = Term::buffered_stdout();\n                let ids = client.query_messages(&sender, &rcpt, &before, &after).await;\n                let ids_len = ids.len();\n                let page_size = page_size.map(|p| std::cmp::max(p, 1)).unwrap_or(20);\n                let pages_total = (ids_len as f64 / page_size as f64).ceil() as usize;\n                for (page_num, chunk) in ids.chunks(page_size).enumerate() {\n                    // Build table\n                    let mut table = Table::new();\n                    table.add_row(Row::new(\n                        [\"ID\", \"Delivery Due\", \"Sender\", \"Recipients\", \"Size\"]\n                            .iter()\n                            .map(|p| Cell::new(p).with_style(Attr::Bold))\n                            .collect(),\n                    ));\n                    for id in chunk {\n                        let message = client\n                            .http_request::<Message, String>(\n                                Method::GET,\n                                &format!(\"/api/queue/messages/{id}\"),\n                                None,\n                            )\n                            .await;\n\n                        let mut rcpts = String::new();\n                        let mut deliver_at = i64::MAX;\n                        let mut deliver_pos = 0;\n\n                        for (pos, rcpt) in message.recipients.iter().enumerate() {\n                            if let Some(next_retry) = &rcpt.next_retry {\n                                let ts = next_retry.to_timestamp();\n                                if ts < deliver_at {\n                                    deliver_at = ts;\n                                    deliver_pos = pos;\n                                }\n                            }\n                            if !rcpts.is_empty() {\n                                rcpts.push('\\n');\n                            }\n                            rcpts.push_str(&rcpt.address);\n                            rcpts.push_str(\" (\");\n                            rcpts.push_str(rcpt.status.status_short());\n                            rcpts.push(')');\n                        }\n\n                        let mut cells = Vec::new();\n                        cells.push(Cell::new(&format!(\"{id:X}\")));\n                        cells.push(if deliver_at != i64::MAX {\n                            Cell::new(\n                                &message.recipients[deliver_pos]\n                                    .next_retry\n                                    .as_ref()\n                                    .unwrap()\n                                    .to_rfc822(),\n                            )\n                        } else {\n                            Cell::new(\"None\")\n                        });\n                        cells.push(Cell::new(if !message.return_path.is_empty() {\n                            &message.return_path\n                        } else {\n                            \"<>\"\n                        }));\n                        cells.push(Cell::new(&rcpts));\n                        cells.push(Cell::new(\n                            &SpecificSize::new(message.size as u32, Byte)\n                                .unwrap()\n                                .to_string(),\n                        ));\n                        table.add_row(Row::new(cells));\n                    }\n\n                    eprintln!();\n                    table.printstd();\n                    eprintln!();\n                    if page_num + 1 != pages_total {\n                        eprintln!(\"\\n--- Press any key to continue or 'q' to exit ---\");\n                        if let Ok('q' | 'Q') = stdout.read_char() {\n                            break;\n                        }\n                    }\n                }\n                eprintln!(\"\\n{ids_len} queued message(s) found.\")\n            }\n            QueueCommands::Status { ids } => {\n                for (uid, id) in parse_ids(&ids).into_iter().zip(ids) {\n                    let message = client\n                        .try_http_request::<Message, String>(\n                            Method::GET,\n                            &format!(\"/api/queue/messages/{uid}\"),\n                            None,\n                        )\n                        .await;\n                    let mut table = Table::new();\n                    table.add_row(Row::new(vec![\n                        Cell::new(\"ID\").with_style(Attr::Bold),\n                        Cell::new(&id),\n                    ]));\n\n                    if let Some(message) = message {\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"Sender\").with_style(Attr::Bold),\n                            Cell::new(if !message.return_path.is_empty() {\n                                &message.return_path\n                            } else {\n                                \"<>\"\n                            }),\n                        ]));\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"Created\").with_style(Attr::Bold),\n                            Cell::new(&message.created.to_rfc822()),\n                        ]));\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"Size\").with_style(Attr::Bold),\n                            Cell::new(\n                                &SpecificSize::new(message.size as u32, Byte)\n                                    .unwrap()\n                                    .to_string(),\n                            ),\n                        ]));\n                        if let Some(env_id) = &message.env_id {\n                            table.add_row(Row::new(vec![\n                                Cell::new(\"Env-Id\").with_style(Attr::Bold),\n                                Cell::new(env_id),\n                            ]));\n                        }\n                        if message.priority != 0 {\n                            table.add_row(Row::new(vec![\n                                Cell::new(\"Priority\").with_style(Attr::Bold),\n                                Cell::new(&message.priority.to_string()),\n                            ]));\n                        }\n                        for rcpt in &message.recipients {\n                            table.add_row(Row::new(vec![\n                                Cell::new_align(&rcpt.address, Alignment::RIGHT)\n                                    .with_style(Attr::Bold)\n                                    .with_style(Attr::Italic(true))\n                                    .with_hspan(2),\n                            ]));\n                            table.add_row(Row::new(vec![\n                                Cell::new(\"Status\").with_style(Attr::Bold),\n                                Cell::new(rcpt.status.status()),\n                            ]));\n                            table.add_row(Row::new(vec![\n                                Cell::new(\"Details\").with_style(Attr::Bold),\n                                Cell::new(rcpt.status.details()),\n                            ]));\n                            table.add_row(Row::new(vec![\n                                Cell::new(\"Retry #\").with_style(Attr::Bold),\n                                Cell::new(&rcpt.retry_num.to_string()),\n                            ]));\n                            if let Some(dt) = &rcpt.next_retry {\n                                table.add_row(Row::new(vec![\n                                    Cell::new(\"Delivery Due\").with_style(Attr::Bold),\n                                    Cell::new(&dt.to_rfc822()),\n                                ]));\n                            }\n                            if let Some(dt) = &rcpt.next_notify {\n                                table.add_row(Row::new(vec![\n                                    Cell::new(\"Notify at\").with_style(Attr::Bold),\n                                    Cell::new(&dt.to_rfc822()),\n                                ]));\n                            }\n                            table.add_row(Row::new(vec![\n                                Cell::new(\"Expires\").with_style(Attr::Bold),\n                                if let Some(dt) = &rcpt.expires {\n                                    Cell::new(&dt.to_rfc822())\n                                } else {\n                                    Cell::new(\"N/A\")\n                                },\n                            ]));\n                        }\n                    } else {\n                        table.add_row(Row::new(vec![\n                            Cell::new_align(\"-- Not found --\", Alignment::CENTER).with_hspan(2),\n                        ]));\n                    }\n\n                    eprintln!();\n                    table.printstd();\n                    eprintln!();\n                }\n            }\n            QueueCommands::Retry {\n                sender,\n                domain,\n                before,\n                after,\n                time,\n                ids,\n            } => {\n                let (parsed_ids, ids) = if ids.is_empty() {\n                    if sender.is_some() || domain.is_some() || before.is_some() || after.is_some() {\n                        let parsed_ids = client\n                            .query_messages(&sender, &domain, &before, &after)\n                            .await;\n                        let ids = parsed_ids.iter().map(|id| format!(\"{id:X}\")).collect();\n                        (parsed_ids, ids)\n                    } else {\n                        (vec![], vec![])\n                    }\n                } else {\n                    (parse_ids(&ids), ids)\n                };\n\n                if ids.is_empty() {\n                    eprintln!(\"No messages were found.\");\n                    std::process::exit(1);\n                }\n\n                let mut success_count = 0;\n                let mut failed_list = vec![];\n\n                for id in parsed_ids {\n                    let mut query =\n                        form_urlencoded::Serializer::new(format!(\"/api/queue/messages/{id}\"));\n\n                    if let Some(filter) = &domain {\n                        query.append_pair(\"filter\", filter);\n                    }\n                    if let Some(at) = time {\n                        query.append_pair(\"at\", &at.to_rfc3339());\n                    }\n\n                    if client\n                        .try_http_request::<bool, String>(Method::PATCH, &query.finish(), None)\n                        .await\n                        .unwrap_or(false)\n                    {\n                        success_count += 1;\n                    } else {\n                        failed_list.push(id.to_string());\n                    }\n                }\n\n                eprint!(\"\\nSuccessfully rescheduled {success_count} message(s).\");\n                if !failed_list.is_empty() {\n                    eprint!(\" Unable to reschedule id(s): {}.\", failed_list.join(\", \"));\n                }\n                eprintln!();\n            }\n            QueueCommands::Cancel {\n                sender,\n                rcpt,\n                before,\n                after,\n                ids,\n            } => {\n                let (parsed_ids, ids) = if ids.is_empty() {\n                    if sender.is_some() || rcpt.is_some() || before.is_some() || after.is_some() {\n                        let parsed_ids =\n                            client.query_messages(&sender, &rcpt, &before, &after).await;\n                        let ids = parsed_ids.iter().map(|id| format!(\"{id:X}\")).collect();\n                        (parsed_ids, ids)\n                    } else {\n                        (vec![], vec![])\n                    }\n                } else {\n                    (parse_ids(&ids), ids)\n                };\n\n                if ids.is_empty() {\n                    eprintln!(\"No messages were found.\");\n                    std::process::exit(1);\n                }\n\n                let mut success_count = 0;\n                let mut failed_list = vec![];\n\n                for id in parsed_ids {\n                    let mut query =\n                        form_urlencoded::Serializer::new(format!(\"/api/queue/messages/{id}\"));\n\n                    if let Some(filter) = &rcpt {\n                        query.append_pair(\"filter\", filter);\n                    }\n\n                    if client\n                        .try_http_request::<bool, String>(Method::DELETE, &query.finish(), None)\n                        .await\n                        .unwrap_or(false)\n                    {\n                        success_count += 1;\n                    } else {\n                        failed_list.push(id.to_string());\n                    }\n                }\n\n                eprint!(\"\\nCancelled delivery of {success_count} message(s).\");\n                if !failed_list.is_empty() {\n                    eprint!(\n                        \" Unable to cancel delivery for id(s): {}.\",\n                        failed_list.join(\", \")\n                    );\n                }\n                eprintln!();\n            }\n        }\n    }\n}\n\nimpl Client {\n    async fn query_messages(\n        &self,\n        from: &Option<String>,\n        rcpt: &Option<String>,\n        before: &Option<DateTime>,\n        after: &Option<DateTime>,\n    ) -> Vec<u64> {\n        let mut query = form_urlencoded::Serializer::new(\"/api/queue/messages\".to_string());\n\n        if let Some(sender) = from {\n            query.append_pair(\"from\", sender);\n        }\n        if let Some(rcpt) = rcpt {\n            query.append_pair(\"to\", rcpt);\n        }\n        if let Some(before) = before {\n            query.append_pair(\"before\", &before.to_rfc3339());\n        }\n        if let Some(after) = after {\n            query.append_pair(\"after\", &after.to_rfc3339());\n        }\n\n        self.http_request::<List<u64>, String>(Method::GET, &query.finish(), None)\n            .await\n            .items\n    }\n}\n\nfn deserialize_maybe_datetime<'de, D>(deserializer: D) -> Result<Option<DateTime>, D::Error>\nwhere\n    D: Deserializer<'de>,\n{\n    if let Some(value) = Option::<&str>::deserialize(deserializer)? {\n        if let Some(value) = DateTime::parse_rfc3339(value) {\n            Ok(Some(value))\n        } else {\n            Err(serde::de::Error::custom(\n                \"Failed to parse RFC3339 timestamp\",\n            ))\n        }\n    } else {\n        Ok(None)\n    }\n}\n\npub fn deserialize_datetime<'de, D>(deserializer: D) -> Result<DateTime, D::Error>\nwhere\n    D: Deserializer<'de>,\n{\n    if let Some(value) = DateTime::parse_rfc3339(<&str>::deserialize(deserializer)?) {\n        Ok(value)\n    } else {\n        Err(serde::de::Error::custom(\n            \"Failed to parse RFC3339 timestamp\",\n        ))\n    }\n}\n\nfn parse_ids(ids: &[String]) -> Vec<u64> {\n    let mut result = Vec::with_capacity(ids.len());\n    for id in ids {\n        match u64::from_str_radix(id, 16) {\n            Ok(id) => {\n                result.push(id);\n            }\n            Err(_) => {\n                eprintln!(\"Failed to parse id {id:?}.\");\n                std::process::exit(1);\n            }\n        }\n    }\n    result\n}\n\nimpl Status {\n    fn status_short(&self) -> &str {\n        match self {\n            Status::Scheduled => \"scheduled\",\n            Status::Completed(_) => \"delivered\",\n            Status::TemporaryFailure(_) => \"tempfail\",\n            Status::PermanentFailure(_) => \"permfail\",\n        }\n    }\n\n    fn status(&self) -> &str {\n        match self {\n            Status::Scheduled => \"Scheduled\",\n            Status::Completed(_) => \"Delivered\",\n            Status::TemporaryFailure(_) => \"Temporary Failure\",\n            Status::PermanentFailure(_) => \"Permanent Failure\",\n        }\n    }\n\n    fn details(&self) -> &str {\n        match self {\n            Status::Scheduled => \"\",\n            Status::Completed(status) => status,\n            Status::TemporaryFailure(status) => status,\n            Status::PermanentFailure(status) => status,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/cli/src/modules/report.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::cli::{Client, ReportCommands, ReportFormat};\nuse crate::modules::{List, queue::deserialize_datetime};\nuse console::Term;\nuse human_size::{Byte, SpecificSize};\nuse mail_auth::{\n    dmarc::URI,\n    mta_sts::ReportUri,\n    report::{self, tlsrpt::TlsReport},\n};\nuse mail_parser::DateTime;\nuse prettytable::{Attr, Cell, Row, Table, format};\nuse reqwest::Method;\nuse serde::{Deserialize, Serialize};\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum Report {\n    Tls {\n        id: String,\n        domain: String,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        range_from: DateTime,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        range_to: DateTime,\n        report: TlsReport,\n        rua: Vec<ReportUri>,\n    },\n    Dmarc {\n        id: String,\n        domain: String,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        range_from: DateTime,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        range_to: DateTime,\n        report: report::Report,\n        rua: Vec<URI>,\n    },\n}\n\nimpl Report {\n    pub fn domain(&self) -> &str {\n        match self {\n            Report::Tls { domain, .. } => domain,\n            Report::Dmarc { domain, .. } => domain,\n        }\n    }\n\n    pub fn type_(&self) -> &str {\n        match self {\n            Report::Tls { .. } => \"TLS\",\n            Report::Dmarc { .. } => \"DMARC\",\n        }\n    }\n\n    pub fn range_from(&self) -> &DateTime {\n        match self {\n            Report::Tls { range_from, .. } => range_from,\n            Report::Dmarc { range_from, .. } => range_from,\n        }\n    }\n\n    pub fn range_to(&self) -> &DateTime {\n        match self {\n            Report::Tls { range_to, .. } => range_to,\n            Report::Dmarc { range_to, .. } => range_to,\n        }\n    }\n\n    pub fn num_records(&self) -> usize {\n        match self {\n            Report::Tls { report, .. } => report\n                .policies\n                .iter()\n                .map(|p| p.failure_details.len())\n                .sum(),\n            Report::Dmarc { report, .. } => report.records().len(),\n        }\n    }\n}\n\nimpl ReportCommands {\n    pub async fn exec(self, client: Client) {\n        match self {\n            ReportCommands::List {\n                domain,\n                format,\n                page_size,\n            } => {\n                let stdout = Term::buffered_stdout();\n                let mut query = form_urlencoded::Serializer::new(\"/api/queue/reports\".to_string());\n\n                if let Some(domain) = &domain {\n                    query.append_pair(\"domain\", domain);\n                }\n                if let Some(format) = &format {\n                    query.append_pair(\"type\", format.id());\n                }\n\n                let ids = client\n                    .http_request::<List<String>, String>(Method::GET, &query.finish(), None)\n                    .await\n                    .items;\n                let ids_len = ids.len();\n                let page_size = page_size.map(|p| std::cmp::max(p, 1)).unwrap_or(20);\n                let pages_total = (ids_len as f64 / page_size as f64).ceil() as usize;\n                for (page_num, chunk) in ids.chunks(page_size).enumerate() {\n                    // Build table\n                    let mut table = Table::new();\n                    table.add_row(Row::new(\n                        [\"ID\", \"Domain\", \"Type\", \"From Date\", \"To Date\", \"Records\"]\n                            .iter()\n                            .map(|p| Cell::new(p).with_style(Attr::Bold))\n                            .collect(),\n                    ));\n                    for id in chunk {\n                        let report = client\n                            .try_http_request::<Report, String>(\n                                Method::GET,\n                                &format!(\"/api/queue/reports/{id}\"),\n                                None,\n                            )\n                            .await;\n\n                        if let Some(report) = report {\n                            table.add_row(Row::new(vec![\n                                Cell::new(id),\n                                Cell::new(report.domain()),\n                                Cell::new(report.type_()),\n                                Cell::new(&report.range_from().to_rfc822()),\n                                Cell::new(&report.range_to().to_rfc822()),\n                                Cell::new(\n                                    &SpecificSize::new(report.num_records() as u32, Byte)\n                                        .unwrap()\n                                        .to_string(),\n                                ),\n                            ]));\n                        }\n                    }\n\n                    eprintln!();\n                    table.printstd();\n                    eprintln!();\n                    if page_num + 1 != pages_total {\n                        eprintln!(\"\\n--- Press any key to continue or 'q' to exit ---\");\n                        if let Ok('q' | 'Q') = stdout.read_char() {\n                            break;\n                        }\n                    }\n                }\n                eprintln!(\"\\n{ids_len} queued message(s) found.\")\n            }\n            ReportCommands::Status { ids } => {\n                for id in ids {\n                    let report = client\n                        .try_http_request::<Report, String>(\n                            Method::GET,\n                            &format!(\"/api/queue/reports/{id}\"),\n                            None,\n                        )\n                        .await;\n\n                    let mut table = Table::new();\n                    table.add_row(Row::new(vec![\n                        Cell::new(\"ID\").with_style(Attr::Bold),\n                        Cell::new(&id),\n                    ]));\n                    if let Some(report) = report {\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"Domain Name\").with_style(Attr::Bold),\n                            Cell::new(report.domain()),\n                        ]));\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"Type\").with_style(Attr::Bold),\n                            Cell::new(report.type_()),\n                        ]));\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"From Date\").with_style(Attr::Bold),\n                            Cell::new(&report.range_from().to_rfc822()),\n                        ]));\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"To Date\").with_style(Attr::Bold),\n                            Cell::new(&report.range_to().to_rfc822()),\n                        ]));\n                        table.add_row(Row::new(vec![\n                            Cell::new(\"Records\").with_style(Attr::Bold),\n                            Cell::new(\n                                &SpecificSize::new(report.num_records() as u32, Byte)\n                                    .unwrap()\n                                    .to_string(),\n                            ),\n                        ]));\n                    } else {\n                        table.add_row(Row::new(vec![\n                            Cell::new_align(\"-- Not found --\", format::Alignment::CENTER)\n                                .with_hspan(2),\n                        ]));\n                    }\n\n                    eprintln!();\n                    table.printstd();\n                    eprintln!();\n                }\n            }\n            ReportCommands::Cancel { ids } => {\n                let mut success_count = 0;\n                let mut failed_list = vec![];\n                for id in ids {\n                    let success = client\n                        .try_http_request::<bool, String>(\n                            Method::DELETE,\n                            &format!(\"/api/queue/reports/{id}\"),\n                            None,\n                        )\n                        .await;\n\n                    if success.unwrap_or_default() {\n                        success_count += 1;\n                    } else {\n                        failed_list.push(id);\n                    }\n                }\n                eprint!(\"\\nRemoved {success_count} report(s).\");\n                if !failed_list.is_empty() {\n                    eprint!(\n                        \" Unable to remove report id(s): {}.\",\n                        failed_list.join(\", \")\n                    );\n                }\n                eprintln!();\n            }\n        }\n    }\n}\n\nimpl ReportFormat {\n    fn id(&self) -> &'static str {\n        match self {\n            ReportFormat::Dmarc => \"dmarc\",\n            ReportFormat::Tls => \"tls\",\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/Cargo.toml",
    "content": "[package]\nname = \"common\"\nversion = \"0.15.5\"\nedition = \"2024\"\nbuild = \"build.rs\"\n\n[dependencies]\nutils = { path = \"../utils\" }\nnlp = { path = \"../nlp\" }\nstore = { path = \"../store\" }\ntrc = { path = \"../trc\" }\ndirectory = { path = \"../directory\" }\ntypes = { path = \"../types\" }\njmap_proto = { path = \"../jmap-proto\" }\nimap_proto = { path = \"../imap-proto\" }\nsieve-rs = { version = \"0.7\", features = [\"rkyv\", \"serde\"] }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-builder = { version = \"0.4\" }\nmail-auth = { version = \"0.7.1\" }\nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nsmtp-proto = { version = \"0.2\", features = [\"rkyv\"] }\ndns-update = { version = \"0.1.5\" }\ncalcard = { version = \"0.3\", features = [\"rkyv\"] }\nahash = { version = \"0.8.2\", features = [\"serde\"] }\nparking_lot = \"0.12.1\"\nregex = \"1.7.0\"\nproxy-header = { version = \"0.1.0\", features = [\"tokio\"] }\narc-swap = \"1.6.0\"\nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pemfile = \"2.0\"\nrustls-pki-types = { version = \"1\" }\nring = { version = \"0.17\" }\ntokio = { version = \"1.47\", features = [\"net\", \"macros\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\nfutures = \"0.3\"\nrcgen = \"0.12\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\", \"stream\"]}\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\nbase64 = \"0.22\"\nx509-parser = \"0.18\"\npem = \"3.0\"\nchrono = { version = \"0.4\", features = [\"serde\"] }\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\nopentelemetry = { version = \"0.29\" }\nopentelemetry_sdk = { version = \"0.29\" }\nopentelemetry-otlp = { version = \"0.29\", default-features = false, features = [\"reqwest-client\", \"http-proto\", \"trace\", \"metrics\", \"logs\", \"internal-logs\", \"grpc-tonic\", \"tls-webpki-roots\", \"reqwest-rustls-webpki-roots\"] }\nopentelemetry-semantic-conventions = { version = \"0.29.0\" }\nprometheus = { version = \"0.14\", default-features = false }\nimagesize = \"0.14\"\nsha1 = \"0.10\"\nsha2 = \"0.10.6\"\nmd5 = \"0.8.0\"\nwhatlang = \"0.18\"\nidna = \"1.0\"\ndecancer = \"3.0.1\"\nunicode-security = \"0.1.0\"\ninfer = \"0.19\"\nbincode = { version = \"2.0\", features = [\"serde\"] }\nhostname = \"0.4.0\"\nzip = \"6.0\"\npwhash = \"1.0.0\"\nxxhash-rust = { version = \"0.8.5\", features = [\"xxh3\"] }\npsl = \"2\"\naes-gcm-siv = \"0.11.1\"\nbiscuit = \"0.7.0\"\nrsa = \"0.9.2\"\np256 = { version = \"0.13\", features = [\"ecdh\"] }\np384 = { version = \"0.13\", features = [\"ecdh\"] }\nnum_cpus = \"1.13.1\"\nhashify = \"0.2\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\nindexmap = \"2.7.1\"\ntinyvec = \"1.9.0\"\ncompact_str = { version = \"0.9.0\", features = [\"rkyv\", \"serde\"] }\nlz4_flex = { version = \"0.12\", features = [\"frame\"], default-features = false }\n\n[target.'cfg(unix)'.dependencies]\nprivdrop = \"0.5.3\"\nlibc = \"0.2.126\"\n\n[features]\ntest_mode = []\nenterprise = []\nfoundation = []\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/common/build.rs",
    "content": "use std::collections::HashMap;\nuse std::env;\nuse std::fs;\nuse std::path::Path;\n\nfn main() {\n    let out_dir = env::var(\"OUT_DIR\").unwrap();\n    let dest_path = Path::new(&out_dir).join(\"locales.rs\");\n\n    // Read the YAML file\n    let manifest_dir = env::var(\"CARGO_MANIFEST_DIR\").unwrap();\n    let repo_root = Path::new(&manifest_dir).parent().unwrap().parent().unwrap();\n    let yaml_path = repo_root.join(\"resources/locales/i18n.yml\");\n    let yaml_content =\n        fs::read_to_string(&yaml_path).unwrap_or_else(|_| panic!(\"Failed to read {yaml_path:?}\"));\n\n    let locales = parse_yaml(&yaml_content);\n\n    let generated_code = generate_locale_code(&locales);\n\n    fs::write(&dest_path, generated_code).expect(\"Failed to write generated locales.\");\n\n    println!(\"cargo:rerun-if-changed={}\", yaml_path.display());\n}\n\nfn parse_yaml(content: &str) -> HashMap<String, HashMap<String, String>> {\n    let mut result: HashMap<String, HashMap<String, String>> = HashMap::new();\n    let mut current_key = None;\n\n    for line in content.lines() {\n        if let Some((key, value)) = line.split_once(':') {\n            let is_translation = key\n                .as_bytes()\n                .first()\n                .is_some_and(|&b| b.is_ascii_whitespace());\n            let key = key.trim();\n            if !key.starts_with('#') && !key.is_empty() {\n                if !is_translation {\n                    current_key = result.entry(key.replace('.', \"_\")).or_default().into();\n                } else {\n                    current_key\n                        .as_mut()\n                        .unwrap()\n                        .insert(key.to_string(), value.trim().trim_matches('\"').to_string());\n                }\n            }\n        }\n    }\n\n    result\n}\n\nfn generate_locale_code(locales: &HashMap<String, HashMap<String, String>>) -> String {\n    let mut code = String::new();\n\n    code.push_str(\"#[derive(Debug, Clone)]\\n\");\n    code.push_str(\"pub struct Locale {\\n\");\n\n    for key in locales.keys() {\n        code.push_str(&format!(\"    pub {}: &'static str,\\n\", key));\n    }\n\n    code.push_str(\"}\\n\\n\");\n\n    let mut languages = std::collections::HashSet::new();\n    for translations in locales.values() {\n        for lang in translations.keys() {\n            languages.insert(lang.clone());\n        }\n    }\n\n    for lang in &languages {\n        code.push_str(&format!(\n            \"pub static {}_LOCALES: Locale = Locale {{\\n\",\n            lang.to_uppercase()\n        ));\n\n        for (key, translations) in locales {\n            let value = translations\n                .get(lang)\n                .unwrap_or_else(|| panic!(\"Missing: {}\", key));\n            code.push_str(&format!(\"    {key}: {value:?},\\n\"));\n        }\n\n        code.push_str(\"};\\n\\n\");\n    }\n\n    code.push_str(\"pub fn locale(name: &str) -> Option<&'static Locale> {\\n\");\n    code.push_str(\"    hashify::tiny_map!(name.as_bytes(),\\n\");\n    for lang in &languages {\n        code.push_str(&format!(\n            \"        \\\"{}\\\" => &{}_LOCALES,\\n\",\n            lang,\n            lang.to_uppercase()\n        ));\n    }\n    code.push_str(\"    )\\n\");\n    code.push_str(\"}\\n\");\n    code\n}\n"
  },
  {
    "path": "crates/common/src/addresses.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse directory::{Directory, backend::RcptType};\nuse std::borrow::Cow;\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::{\n    Server,\n    config::smtp::session::AddressMapping,\n    expr::{\n        V_RECIPIENT, Variable, functions::ResolveVariable, if_block::IfBlock, tokenizer::TokenMap,\n    },\n};\n\nimpl Server {\n    pub async fn email_to_id(\n        &self,\n        directory: &Directory,\n        email: &str,\n        session_id: u64,\n    ) -> trc::Result<Option<u32>> {\n        let mut address = self\n            .core\n            .smtp\n            .session\n            .rcpt\n            .subaddressing\n            .to_subaddress(self, email, session_id)\n            .await;\n\n        for _ in 0..2 {\n            let result = directory.email_to_id(address.as_ref()).await?;\n\n            if result.is_some() {\n                return Ok(result);\n            } else if let Some(catch_all) = self\n                .core\n                .smtp\n                .session\n                .rcpt\n                .catch_all\n                .to_catch_all(self, email, session_id)\n                .await\n            {\n                address = catch_all;\n            } else {\n                break;\n            }\n        }\n\n        Ok(None)\n    }\n\n    pub async fn rcpt(\n        &self,\n        directory: &Directory,\n        email: &str,\n        session_id: u64,\n    ) -> trc::Result<RcptType> {\n        // Expand subaddress\n        let mut address = self\n            .core\n            .smtp\n            .session\n            .rcpt\n            .subaddressing\n            .to_subaddress(self, email, session_id)\n            .await;\n\n        for _ in 0..2 {\n            let rcpt_type = directory.rcpt(address.as_ref()).await?;\n            if rcpt_type != RcptType::Invalid {\n                return Ok(rcpt_type);\n            } else if let Some(catch_all) = self\n                .core\n                .smtp\n                .session\n                .rcpt\n                .catch_all\n                .to_catch_all(self, email, session_id)\n                .await\n            {\n                address = catch_all;\n            } else {\n                break;\n            }\n        }\n\n        Ok(RcptType::Invalid)\n    }\n\n    pub async fn vrfy(\n        &self,\n        directory: &Directory,\n        address: &str,\n        session_id: u64,\n    ) -> trc::Result<Vec<String>> {\n        directory\n            .vrfy(\n                self.core\n                    .smtp\n                    .session\n                    .rcpt\n                    .subaddressing\n                    .to_subaddress(self, address, session_id)\n                    .await\n                    .as_ref(),\n            )\n            .await\n    }\n\n    pub async fn expn(\n        &self,\n        directory: &Directory,\n        address: &str,\n        session_id: u64,\n    ) -> trc::Result<Vec<String>> {\n        directory\n            .expn(\n                self.core\n                    .smtp\n                    .session\n                    .rcpt\n                    .subaddressing\n                    .to_subaddress(self, address, session_id)\n                    .await\n                    .as_ref(),\n            )\n            .await\n    }\n}\n\nimpl AddressMapping {\n    pub fn parse(config: &mut Config, key: impl AsKey) -> Self {\n        let key = key.as_key();\n        if let Some(value) = config.value(key.as_str()) {\n            match value {\n                \"true\" => AddressMapping::Enable,\n                \"false\" => AddressMapping::Disable,\n                _ => {\n                    config.new_parse_error(\n                        key,\n                        format!(\"Invalid value for address mapping {value:?}\",),\n                    );\n                    AddressMapping::Disable\n                }\n            }\n        } else if let Some(if_block) = IfBlock::try_parse(\n            config,\n            key,\n            &TokenMap::default().with_variables_map([\n                (\"address\", V_RECIPIENT),\n                (\"email\", V_RECIPIENT),\n                (\"rcpt\", V_RECIPIENT),\n            ]),\n        ) {\n            AddressMapping::Custom(if_block)\n        } else {\n            AddressMapping::Enable\n        }\n    }\n}\n\nstruct Address<'x>(&'x str);\n\nimpl ResolveVariable for Address<'_> {\n    fn resolve_variable(&'_ self, _: u32) -> crate::expr::Variable<'_> {\n        Variable::from(self.0)\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl AddressMapping {\n    pub async fn to_subaddress<'x, 'y: 'x>(\n        &'x self,\n        core: &Server,\n        address: &'y str,\n        session_id: u64,\n    ) -> Cow<'x, str> {\n        match self {\n            AddressMapping::Enable => {\n                if let Some((local_part, domain_part)) = address.rsplit_once('@')\n                    && let Some((local_part, _)) = local_part.split_once('+')\n                {\n                    return format!(\"{}@{}\", local_part, domain_part).into();\n                }\n            }\n            AddressMapping::Custom(if_block) => {\n                if let Some(result) = core\n                    .eval_if::<String, _>(if_block, &Address(address), session_id)\n                    .await\n                {\n                    return result.into();\n                }\n            }\n            AddressMapping::Disable => (),\n        }\n\n        address.into()\n    }\n\n    pub async fn to_catch_all<'x, 'y: 'x>(\n        &'x self,\n        core: &Server,\n        address: &'y str,\n        session_id: u64,\n    ) -> Option<Cow<'x, str>> {\n        match self {\n            AddressMapping::Enable => address\n                .rsplit_once('@')\n                .map(|(_, domain_part)| format!(\"@{}\", domain_part))\n                .map(Cow::Owned),\n            AddressMapping::Custom(if_block) => core\n                .eval_if::<String, _>(if_block, &Address(address), session_id)\n                .await\n                .map(Cow::Owned),\n            AddressMapping::Disable => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/access_token.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{AccessToken, ResourceToken, TenantInfo, roles::RolePermissions};\nuse crate::{\n    Server,\n    ipc::BroadcastEvent,\n    listener::limiter::{ConcurrencyLimiter, LimiterResult},\n};\nuse ahash::AHashSet;\nuse directory::{\n    Permission, Principal, PrincipalData, QueryParams, Type,\n    backend::internal::{\n        lookup::DirectoryStore,\n        manage::{ChangedPrincipals, ManageDirectory},\n    },\n};\nuse std::{\n    hash::{DefaultHasher, Hash, Hasher},\n    sync::Arc,\n};\nuse store::{query::acl::AclQuery, rand};\nuse trc::AddContext;\nuse types::{acl::Acl, collection::Collection};\nuse utils::map::{\n    bitmap::{Bitmap, BitmapItem},\n    vec_map::VecMap,\n};\n\npub enum PrincipalOrId {\n    Principal(Principal),\n    Id(u32),\n}\n\nimpl Server {\n    async fn build_access_token_from_principal(\n        &self,\n        principal: Principal,\n        revision: u64,\n    ) -> trc::Result<AccessToken> {\n        let mut role_permissions = RolePermissions::default();\n\n        // Extract data\n        let mut object_quota = self.core.jmap.max_objects;\n        let mut description = None;\n        let mut tenant_id = None;\n        let mut quota = None;\n        let mut locale = None;\n        let mut member_of = Vec::new();\n        let mut emails = Vec::new();\n        for data in principal.data {\n            match data {\n                PrincipalData::Tenant(v) => tenant_id = Some(v),\n                PrincipalData::MemberOf(v) => member_of.push(v),\n                PrincipalData::Role(v) => {\n                    role_permissions.union(self.get_role_permissions(v).await?.as_ref());\n                }\n                PrincipalData::Permission {\n                    permission_id,\n                    grant,\n                } => {\n                    if grant {\n                        role_permissions.enabled.set(permission_id as usize);\n                    } else {\n                        role_permissions.disabled.set(permission_id as usize);\n                    }\n                }\n                PrincipalData::DiskQuota(v) => quota = Some(v),\n                PrincipalData::ObjectQuota { quota, typ } => {\n                    object_quota[typ as usize] = quota;\n                }\n                PrincipalData::Description(v) => description = Some(v),\n                PrincipalData::PrimaryEmail(v) => {\n                    if emails.is_empty() {\n                        emails.push(v);\n                    } else {\n                        emails.insert(0, v);\n                    }\n                }\n                PrincipalData::EmailAlias(v) => {\n                    emails.push(v);\n                }\n                PrincipalData::Locale(v) => locale = Some(v),\n                _ => (),\n            }\n        }\n\n        // Apply principal permissions\n        let mut permissions = role_permissions.finalize();\n        let mut tenant = None;\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        #[cfg(feature = \"enterprise\")]\n        {\n            use directory::{QueryParams, ROLE_USER};\n\n            if let Some(tenant_id) = tenant_id {\n                if self.is_enterprise_edition() {\n                    // Limit tenant permissions\n                    permissions.intersection(&self.get_role_permissions(tenant_id).await?.enabled);\n\n                    // Obtain tenant quota\n                    tenant = Some(TenantInfo {\n                        id: tenant_id,\n                        quota: self\n                            .store()\n                            .query(QueryParams::id(tenant_id).with_return_member_of(false))\n                            .await\n                            .caused_by(trc::location!())?\n                            .ok_or_else(|| {\n                                trc::SecurityEvent::Unauthorized\n                                    .into_err()\n                                    .details(\"Tenant not found\")\n                                    .id(tenant_id)\n                                    .caused_by(trc::location!())\n                            })?\n                            .quota()\n                            .unwrap_or_default(),\n                    });\n                } else {\n                    // Enterprise edition downgrade, remove any tenant administrator permissions\n                    permissions.intersection(&self.get_role_permissions(ROLE_USER).await?.enabled);\n                }\n            }\n        }\n\n        // SPDX-SnippetEnd\n\n        // Build member of and e-mail addresses\n        for &group_id in &member_of {\n            if let Some(group) = self\n                .store()\n                .query(QueryParams::id(group_id).with_return_member_of(false))\n                .await\n                .caused_by(trc::location!())?\n                && group.typ == Type::Group\n            {\n                emails.extend(group.into_email_addresses());\n            }\n        }\n\n        // Build access token\n        let mut access_token = AccessToken {\n            primary_id: principal.id,\n            member_of,\n            access_to: VecMap::new(),\n            tenant,\n            name: principal.name,\n            description,\n            emails,\n            quota: quota.unwrap_or_default(),\n            locale,\n            permissions,\n            object_quota,\n            concurrent_imap_requests: self.core.imap.rate_concurrent.map(ConcurrencyLimiter::new),\n            concurrent_http_requests: self\n                .core\n                .jmap\n                .request_max_concurrent\n                .map(ConcurrencyLimiter::new),\n            concurrent_uploads: self\n                .core\n                .jmap\n                .upload_max_concurrent\n                .map(ConcurrencyLimiter::new),\n            obj_size: 0,\n            revision,\n        };\n\n        for grant_account_id in [access_token.primary_id]\n            .into_iter()\n            .chain(access_token.member_of.iter().copied())\n        {\n            for acl_item in self\n                .store()\n                .acl_query(AclQuery::HasAccess { grant_account_id })\n                .await\n                .caused_by(trc::location!())?\n            {\n                if !access_token.is_member(acl_item.to_account_id) {\n                    let acl = Bitmap::<Acl>::from(acl_item.permissions);\n                    let collection = acl_item.to_collection;\n                    if !collection.is_valid() {\n                        return Err(trc::StoreEvent::DataCorruption\n                            .ctx(trc::Key::Reason, \"Corrupted collection found in ACL key.\")\n                            .details(format!(\"{acl_item:?}\"))\n                            .account_id(grant_account_id)\n                            .caused_by(trc::location!()));\n                    }\n\n                    let mut collections: Bitmap<Collection> = Bitmap::new();\n                    if acl.contains(Acl::Read) {\n                        collections.insert(collection);\n                    }\n                    if acl.contains(Acl::ReadItems)\n                        && let Some(child_col) = collection.child_collection()\n                    {\n                        collections.insert(child_col);\n                    }\n\n                    if !collections.is_empty() {\n                        access_token\n                            .access_to\n                            .get_mut_or_insert_with(acl_item.to_account_id, Bitmap::new)\n                            .union(&collections);\n                    }\n                }\n            }\n        }\n\n        Ok(access_token.update_size())\n    }\n\n    async fn build_access_token(&self, account_id: u32, revision: u64) -> trc::Result<AccessToken> {\n        let err = match self\n            .directory()\n            .query(QueryParams::id(account_id).with_return_member_of(true))\n            .await\n        {\n            Ok(Some(principal)) => {\n                return self\n                    .build_access_token_from_principal(principal, revision)\n                    .await;\n            }\n            Ok(None) => Err(trc::AuthEvent::Error\n                .into_err()\n                .details(\"Account not found.\")\n                .caused_by(trc::location!())),\n            Err(err) => Err(err),\n        };\n\n        match &self.core.jmap.fallback_admin {\n            Some((_, secret)) if account_id == u32::MAX => {\n                self.build_access_token_from_principal(Principal::fallback_admin(secret), revision)\n                    .await\n            }\n            _ => err,\n        }\n    }\n\n    pub async fn get_access_token(\n        &self,\n        principal: impl Into<PrincipalOrId>,\n    ) -> trc::Result<Arc<AccessToken>> {\n        let principal = principal.into();\n\n        // Obtain current revision\n        let principal_id = principal.id();\n\n        match self\n            .inner\n            .cache\n            .access_tokens\n            .get_value_or_guard_async(&principal_id)\n            .await\n        {\n            Ok(token) => Ok(token),\n            Err(guard) => {\n                let revision = rand::random::<u64>();\n                let token: Arc<AccessToken> = match principal {\n                    PrincipalOrId::Principal(principal) => {\n                        self.build_access_token_from_principal(principal, revision)\n                            .await?\n                    }\n                    PrincipalOrId::Id(account_id) => {\n                        self.build_access_token(account_id, revision).await?\n                    }\n                }\n                .into();\n                let _ = guard.insert(token.clone());\n                Ok(token)\n            }\n        }\n    }\n\n    pub async fn invalidate_principal_caches(&self, changed_principals: ChangedPrincipals) {\n        let mut nested_principals = Vec::new();\n        let mut changed_ids = AHashSet::new();\n        let mut changed_names = Vec::new();\n\n        for (id, changed_principal) in changed_principals.iter() {\n            changed_ids.insert(*id);\n\n            if changed_principal.name_change {\n                self.inner.cache.files.remove(id);\n                self.inner.cache.contacts.remove(id);\n                self.inner.cache.events.remove(id);\n                self.inner.cache.scheduling.remove(id);\n                changed_names.push(*id);\n            }\n\n            if changed_principal.member_change {\n                if changed_principal.typ == Type::Tenant {\n                    match self\n                        .store()\n                        .list_principals(\n                            None,\n                            (*id).into(),\n                            &[Type::Individual, Type::Group, Type::Role, Type::ApiKey],\n                            false,\n                            0,\n                            0,\n                        )\n                        .await\n                    {\n                        Ok(principals) => {\n                            for principal in principals.items {\n                                changed_ids.insert(principal.id());\n                            }\n                        }\n                        Err(err) => {\n                            trc::error!(\n                                err.details(\"Failed to list principals\")\n                                    .caused_by(trc::location!())\n                                    .account_id(*id)\n                            );\n                        }\n                    }\n                } else {\n                    nested_principals.push(*id);\n                }\n            }\n        }\n\n        if !nested_principals.is_empty() {\n            let mut ids = nested_principals.into_iter();\n            let mut ids_stack = vec![];\n\n            loop {\n                if let Some(id) = ids.next() {\n                    // Skip if already fetched\n                    if !changed_ids.insert(id) {\n                        continue;\n                    }\n\n                    // Obtain principal\n                    match self.store().get_members(id).await {\n                        Ok(members) => {\n                            ids_stack.push(ids);\n                            ids = members.into_iter();\n                        }\n                        Err(err) => {\n                            trc::error!(\n                                err.details(\"Failed to obtain principal\")\n                                    .caused_by(trc::location!())\n                                    .account_id(id)\n                            );\n                        }\n                    }\n                } else if let Some(prev_ids) = ids_stack.pop() {\n                    ids = prev_ids;\n                } else {\n                    break;\n                }\n            }\n        }\n\n        // Invalidate access tokens in cluster\n        if !changed_ids.is_empty() {\n            let mut ids = Vec::with_capacity(changed_ids.len());\n            for id in changed_ids {\n                self.inner.cache.permissions.remove(&id);\n                self.inner.cache.access_tokens.remove(&id);\n                ids.push(id);\n            }\n            self.cluster_broadcast(BroadcastEvent::InvalidateAccessTokens(ids))\n                .await;\n        }\n\n        // Invalidate DAV caches\n        if !changed_names.is_empty() {\n            self.cluster_broadcast(BroadcastEvent::InvalidateGroupwareCache(changed_names))\n                .await;\n        }\n    }\n}\n\nimpl From<u32> for PrincipalOrId {\n    fn from(id: u32) -> Self {\n        Self::Id(id)\n    }\n}\n\nimpl From<Principal> for PrincipalOrId {\n    fn from(principal: Principal) -> Self {\n        Self::Principal(principal)\n    }\n}\n\nimpl PrincipalOrId {\n    pub fn id(&self) -> u32 {\n        match self {\n            Self::Principal(principal) => principal.id(),\n            Self::Id(id) => *id,\n        }\n    }\n}\n\nimpl AccessToken {\n    pub fn from_id(primary_id: u32) -> Self {\n        Self {\n            primary_id,\n            ..Default::default()\n        }\n    }\n\n    pub fn with_access_to(self, access_to: VecMap<u32, Bitmap<Collection>>) -> Self {\n        Self { access_to, ..self }\n    }\n\n    pub fn with_permission(mut self, permission: Permission) -> Self {\n        self.permissions.set(permission.id() as usize);\n        self\n    }\n\n    pub fn with_tenant_id(mut self, tenant_id: Option<u32>) -> Self {\n        self.tenant = tenant_id.map(|id| TenantInfo { id, quota: 0 });\n        self\n    }\n\n    pub fn state(&self) -> u32 {\n        // Hash state\n        let mut s = DefaultHasher::new();\n        self.member_of.hash(&mut s);\n        self.access_to.hash(&mut s);\n        s.finish() as u32\n    }\n\n    #[inline(always)]\n    pub fn primary_id(&self) -> u32 {\n        self.primary_id\n    }\n\n    #[inline(always)]\n    pub fn tenant_id(&self) -> Option<u32> {\n        self.tenant.as_ref().map(|t| t.id)\n    }\n\n    pub fn secondary_ids(&self) -> impl Iterator<Item = &u32> {\n        self.member_of\n            .iter()\n            .chain(self.access_to.iter().map(|(id, _)| id))\n    }\n\n    pub fn member_ids(&self) -> impl Iterator<Item = u32> {\n        [self.primary_id]\n            .into_iter()\n            .chain(self.member_of.iter().copied())\n    }\n\n    pub fn all_ids(&self) -> impl Iterator<Item = u32> {\n        [self.primary_id]\n            .into_iter()\n            .chain(self.member_of.iter().copied())\n            .chain(self.access_to.iter().map(|(id, _)| *id))\n    }\n\n    pub fn all_ids_by_collection(&self, collection: Collection) -> impl Iterator<Item = u32> {\n        [self.primary_id]\n            .into_iter()\n            .chain(self.member_of.iter().copied())\n            .chain(self.access_to.iter().filter_map(move |(id, cols)| {\n                if cols.contains(collection) {\n                    Some(*id)\n                } else {\n                    None\n                }\n            }))\n    }\n\n    pub fn is_member(&self, account_id: u32) -> bool {\n        self.primary_id == account_id\n            || self.member_of.contains(&account_id)\n            || self.has_permission(Permission::Impersonate)\n    }\n\n    pub fn is_primary_id(&self, account_id: u32) -> bool {\n        self.primary_id == account_id\n    }\n\n    #[inline(always)]\n    pub fn has_permission(&self, permission: Permission) -> bool {\n        self.permissions.get(permission.id() as usize)\n    }\n\n    pub fn assert_has_permission(&self, permission: Permission) -> trc::Result<bool> {\n        if self.has_permission(permission) {\n            Ok(true)\n        } else {\n            Err(trc::SecurityEvent::Unauthorized\n                .into_err()\n                .details(permission.name()))\n        }\n    }\n\n    pub fn permissions(&self) -> Vec<Permission> {\n        const USIZE_BITS: usize = std::mem::size_of::<usize>() * 8;\n        const USIZE_MASK: u32 = USIZE_BITS as u32 - 1;\n        let mut permissions = Vec::new();\n\n        for (block_num, bytes) in self.permissions.inner().iter().enumerate() {\n            let mut bytes = *bytes;\n\n            while bytes != 0 {\n                let item = USIZE_MASK - bytes.leading_zeros();\n                bytes ^= 1 << item;\n                if let Some(permission) =\n                    Permission::from_id(((block_num * USIZE_BITS) + item as usize) as u32)\n                {\n                    permissions.push(permission);\n                }\n            }\n        }\n        permissions\n    }\n\n    #[inline(always)]\n    pub fn object_quota(&self, collection: Collection) -> u32 {\n        self.object_quota[collection as usize]\n    }\n\n    pub fn is_shared(&self, account_id: u32) -> bool {\n        !self.is_member(account_id) && self.access_to.iter().any(|(id, _)| *id == account_id)\n    }\n\n    pub fn shared_accounts(&self, collection: Collection) -> impl Iterator<Item = &u32> {\n        self.member_of\n            .iter()\n            .chain(self.access_to.iter().filter_map(move |(id, cols)| {\n                if cols.contains(collection) {\n                    id.into()\n                } else {\n                    None\n                }\n            }))\n    }\n\n    pub fn has_access(&self, to_account_id: u32, to_collection: impl Into<Collection>) -> bool {\n        let to_collection = to_collection.into();\n        self.is_member(to_account_id)\n            || self.access_to.iter().any(|(id, collections)| {\n                *id == to_account_id && collections.contains(to_collection)\n            })\n    }\n\n    pub fn has_account_access(&self, to_account_id: u32) -> bool {\n        self.is_member(to_account_id) || self.access_to.iter().any(|(id, _)| *id == to_account_id)\n    }\n\n    pub fn as_resource_token(&self) -> ResourceToken {\n        ResourceToken {\n            account_id: self.primary_id,\n            quota: self.quota,\n            tenant: self.tenant,\n        }\n    }\n\n    pub fn is_http_request_allowed(&self) -> LimiterResult {\n        self.concurrent_http_requests\n            .as_ref()\n            .map_or(LimiterResult::Disabled, |limiter| limiter.is_allowed())\n    }\n\n    pub fn is_imap_request_allowed(&self) -> LimiterResult {\n        self.concurrent_imap_requests\n            .as_ref()\n            .map_or(LimiterResult::Disabled, |limiter| limiter.is_allowed())\n    }\n\n    pub fn is_upload_allowed(&self) -> LimiterResult {\n        self.concurrent_uploads\n            .as_ref()\n            .map_or(LimiterResult::Disabled, |limiter| limiter.is_allowed())\n    }\n\n    pub fn update_size(mut self) -> Self {\n        self.obj_size = (std::mem::size_of::<AccessToken>()\n            + (self.member_of.len() * std::mem::size_of::<u32>())\n            + (self.access_to.len() * (std::mem::size_of::<u32>() + std::mem::size_of::<u64>()))\n            + self.name.len()\n            + self.description.as_ref().map_or(0, |v| v.len())\n            + self.locale.as_ref().map_or(0, |v| v.len())\n            + self.emails.iter().map(|v| v.len()).sum::<usize>()) as u64;\n        self\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{Server, listener::limiter::ConcurrencyLimiter};\nuse directory::{\n    Directory, FALLBACK_ADMIN_ID, Permission, Permissions, Principal, QueryParams, Type,\n    backend::internal::lookup::DirectoryStore, core::secret::verify_secret_hash,\n};\nuse mail_send::Credentials;\nuse oauth::GrantType;\nuse std::{net::IpAddr, sync::Arc};\nuse types::collection::Collection;\nuse utils::{\n    cache::CacheItemWeight,\n    map::{bitmap::Bitmap, vec_map::VecMap},\n};\n\npub mod access_token;\npub mod oauth;\npub mod rate_limit;\npub mod roles;\npub mod sasl;\n\n#[derive(Debug, Default)]\npub struct AccessToken {\n    pub primary_id: u32,\n    pub member_of: Vec<u32>,\n    pub access_to: VecMap<u32, Bitmap<Collection>>,\n    pub name: String,\n    pub description: Option<String>,\n    pub locale: Option<String>,\n    pub emails: Vec<String>,\n    pub quota: u64,\n    pub object_quota: [u32; Collection::MAX],\n    pub permissions: Permissions,\n    pub tenant: Option<TenantInfo>,\n    pub concurrent_http_requests: Option<ConcurrencyLimiter>,\n    pub concurrent_imap_requests: Option<ConcurrencyLimiter>,\n    pub concurrent_uploads: Option<ConcurrencyLimiter>,\n    pub revision: u64,\n    pub obj_size: u64,\n}\n\n#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]\npub struct TenantInfo {\n    pub id: u32,\n    pub quota: u64,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct ResourceToken {\n    pub account_id: u32,\n    pub quota: u64,\n    pub tenant: Option<TenantInfo>,\n}\n\npub struct AuthRequest<'x> {\n    credentials: Credentials<String>,\n    session_id: u64,\n    remote_ip: IpAddr,\n    return_member_of: bool,\n    allow_api_access: bool,\n    directory: Option<&'x Directory>,\n}\n\nimpl Server {\n    pub async fn authenticate(&self, req: &AuthRequest<'_>) -> trc::Result<Arc<AccessToken>> {\n        // Resolve directory\n        let directory = req.directory.unwrap_or(&self.core.storage.directory);\n\n        // Validate credentials\n        match &req.credentials {\n            Credentials::OAuthBearer { token } if !directory.has_bearer_token_support() => {\n                match self\n                    .validate_access_token(GrantType::AccessToken.into(), token)\n                    .await\n                {\n                    Ok(token_into) => self.get_access_token(token_into.account_id).await,\n                    Err(err) => Err(err),\n                }\n            }\n            _ => match self.authenticate_credentials(req, directory).await {\n                Ok(principal) => self.get_access_token(principal).await,\n                Err(err) => Err(err),\n            },\n        }\n        .and_then(|token| {\n            token\n                .assert_has_permission(Permission::Authenticate)\n                .map(|_| token)\n        })\n    }\n\n    async fn authenticate_credentials(\n        &self,\n        req: &AuthRequest<'_>,\n        directory: &Directory,\n    ) -> trc::Result<Principal> {\n        // First try to authenticate the user against the default directory\n        let result = match directory\n            .query(\n                QueryParams::credentials(&req.credentials)\n                    .with_return_member_of(req.return_member_of),\n            )\n            .await\n        {\n            Ok(Some(principal)) => {\n                trc::event!(\n                    Auth(trc::AuthEvent::Success),\n                    AccountName = principal.name().to_string(),\n                    AccountId = principal.id(),\n                    SpanId = req.session_id,\n                );\n\n                return Ok(principal);\n            }\n            Ok(None) => Ok(()),\n            Err(err) => {\n                if err.matches(trc::EventType::Auth(trc::AuthEvent::MissingTotp)) {\n                    return Err(err);\n                } else {\n                    Err(err)\n                }\n            }\n        };\n\n        match &req.credentials {\n            Credentials::Plain { username, secret } => {\n                // Then check if the credentials match the fallback admin or master user\n                match (&self.core.jmap.fallback_admin, &self.core.jmap.master_user) {\n                    (Some((fallback_admin, fallback_pass)), _) if username == fallback_admin => {\n                        if verify_secret_hash(fallback_pass, secret).await? {\n                            trc::event!(\n                                Auth(trc::AuthEvent::Success),\n                                AccountName = username.clone(),\n                                SpanId = req.session_id,\n                            );\n\n                            return Ok(Principal::fallback_admin(fallback_pass));\n                        }\n                    }\n                    (_, Some((master_user, master_pass))) if username.ends_with(master_user) => {\n                        if verify_secret_hash(master_pass, secret).await? {\n                            let username = username.strip_suffix(master_user).unwrap();\n                            let username = username.strip_suffix('%').unwrap_or(username);\n\n                            if let Some(principal) = directory\n                                .query(\n                                    QueryParams::name(username)\n                                        .with_return_member_of(req.return_member_of),\n                                )\n                                .await?\n                            {\n                                trc::event!(\n                                    Auth(trc::AuthEvent::Success),\n                                    AccountName = username.to_string(),\n                                    SpanId = req.session_id,\n                                    AccountId = principal.id(),\n                                    Type = principal.typ().description(),\n                                );\n\n                                return Ok(principal);\n                            }\n                        }\n                    }\n                    _ => {\n                        // Validate API credentials\n                        if req.allow_api_access\n                            && let Ok(Some(principal)) = self\n                                .store()\n                                .query(\n                                    QueryParams::credentials(&req.credentials)\n                                        .with_return_member_of(req.return_member_of),\n                                )\n                                .await\n                            && principal.typ == Type::ApiKey\n                        {\n                            trc::event!(\n                                Auth(trc::AuthEvent::Success),\n                                AccountName = principal.name().to_string(),\n                                AccountId = principal.id(),\n                                SpanId = req.session_id,\n                            );\n\n                            return Ok(principal);\n                        }\n                    }\n                }\n            }\n            Credentials::OAuthBearer { token } if directory.has_bearer_token_support() => {\n                // Check for bearer tokens issued locally\n                if let Ok(token_info) = self\n                    .validate_access_token(GrantType::AccessToken.into(), token)\n                    .await\n                {\n                    let principal = if token_info.account_id != FALLBACK_ADMIN_ID {\n                        directory\n                            .query(\n                                QueryParams::id(token_info.account_id)\n                                    .with_return_member_of(req.return_member_of),\n                            )\n                            .await\n                            .unwrap_or_default()\n                    } else if let Some((_, fallback_pass)) = &self.core.jmap.fallback_admin {\n                        Principal::fallback_admin(fallback_pass).into()\n                    } else {\n                        None\n                    };\n                    if let Some(principal) = principal {\n                        trc::event!(\n                            Auth(trc::AuthEvent::Success),\n                            AccountName = principal.name().to_string(),\n                            AccountId = principal.id(),\n                            SpanId = req.session_id,\n                        );\n\n                        return Ok(principal);\n                    }\n                }\n            }\n            _ => (),\n        };\n\n        if let Err(err) = result {\n            Err(err)\n        } else if self.has_auth_fail2ban() {\n            let login = req.credentials.login();\n            if self.is_auth_fail2banned(req.remote_ip, login).await? {\n                Err(trc::SecurityEvent::AuthenticationBan\n                    .into_err()\n                    .ctx(trc::Key::RemoteIp, req.remote_ip)\n                    .ctx_opt(trc::Key::AccountName, login.map(|s| s.to_string())))\n            } else {\n                Err(trc::AuthEvent::Failed\n                    .ctx(trc::Key::RemoteIp, req.remote_ip)\n                    .ctx_opt(trc::Key::AccountName, login.map(|s| s.to_string())))\n            }\n        } else {\n            Err(trc::AuthEvent::Failed\n                .ctx(trc::Key::RemoteIp, req.remote_ip)\n                .ctx_opt(\n                    trc::Key::AccountName,\n                    req.credentials.login().map(|s| s.to_string()),\n                ))\n        }\n    }\n}\n\nimpl<'x> AuthRequest<'x> {\n    pub fn from_credentials(\n        credentials: Credentials<String>,\n        session_id: u64,\n        remote_ip: IpAddr,\n    ) -> Self {\n        Self {\n            credentials,\n            session_id,\n            remote_ip,\n            return_member_of: true,\n            directory: None,\n            allow_api_access: false,\n        }\n    }\n\n    pub fn from_plain(\n        user: impl Into<String>,\n        pass: impl Into<String>,\n        session_id: u64,\n        remote_ip: IpAddr,\n    ) -> Self {\n        Self::from_credentials(\n            Credentials::Plain {\n                username: user.into(),\n                secret: pass.into(),\n            },\n            session_id,\n            remote_ip,\n        )\n    }\n\n    pub fn without_members(mut self) -> Self {\n        self.return_member_of = false;\n        self\n    }\n\n    pub fn with_directory(mut self, directory: &'x Directory) -> Self {\n        self.directory = Some(directory);\n        self\n    }\n\n    pub fn with_api_access(mut self, allow_api_access: bool) -> Self {\n        self.allow_api_access = allow_api_access;\n        self\n    }\n}\n\nimpl CacheItemWeight for AccessToken {\n    fn weight(&self) -> u64 {\n        self.obj_size\n    }\n}\n\npub(crate) trait CredentialsUsername {\n    fn login(&self) -> Option<&str>;\n}\n\nimpl CredentialsUsername for Credentials<String> {\n    fn login(&self) -> Option<&str> {\n        match self {\n            Credentials::Plain { username, .. } | Credentials::XOauth2 { username, .. } => {\n                username.as_str().into()\n            }\n            Credentials::OAuthBearer { .. } => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/oauth/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse biscuit::{\n    jwa::{Algorithm, SignatureAlgorithm},\n    jwk::{\n        AlgorithmParameters, CommonParameters, EllipticCurve, EllipticCurveKeyParameters,\n        EllipticCurveKeyType, JWK, JWKSet, OctetKeyParameters, OctetKeyType, PublicKeyUse,\n        RSAKeyParameters, RSAKeyType,\n    },\n    jws::Secret,\n};\nuse ring::signature::{self, KeyPair};\nuse rsa::{RsaPublicKey, pkcs1::DecodeRsaPublicKey, traits::PublicKeyParts};\nuse store::rand::{Rng, distr::Alphanumeric, rng};\nuse utils::config::Config;\nuse x509_parser::num_bigint::BigUint;\n\nuse crate::{\n    config::{build_ecdsa_pem, build_rsa_keypair},\n    manager::webadmin::Resource,\n};\n\n#[derive(Clone)]\npub struct OAuthConfig {\n    pub oauth_key: String,\n    pub oauth_expiry_user_code: u64,\n    pub oauth_expiry_auth_code: u64,\n    pub oauth_expiry_token: u64,\n    pub oauth_expiry_refresh_token: u64,\n    pub oauth_expiry_refresh_token_renew: u64,\n    pub oauth_max_auth_attempts: u32,\n\n    pub allow_anonymous_client_registration: bool,\n    pub require_client_authentication: bool,\n\n    pub oidc_expiry_id_token: u64,\n    pub oidc_signing_secret: Secret,\n    pub oidc_signature_algorithm: SignatureAlgorithm,\n    pub oidc_jwks: Resource<Vec<u8>>,\n}\n\nimpl OAuthConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        let oidc_signature_algorithm = match config.value(\"oauth.oidc.signature-algorithm\") {\n            Some(alg) => match alg.to_uppercase().as_str() {\n                \"HS256\" => SignatureAlgorithm::HS256,\n                \"HS384\" => SignatureAlgorithm::HS384,\n                \"HS512\" => SignatureAlgorithm::HS512,\n\n                \"RS256\" => SignatureAlgorithm::RS256,\n                \"RS384\" => SignatureAlgorithm::RS384,\n                \"RS512\" => SignatureAlgorithm::RS512,\n\n                \"ES256\" => SignatureAlgorithm::ES256,\n                \"ES384\" => SignatureAlgorithm::ES384,\n\n                \"PS256\" => SignatureAlgorithm::PS256,\n                \"PS384\" => SignatureAlgorithm::PS384,\n                \"PS512\" => SignatureAlgorithm::PS512,\n                _ => {\n                    config.new_parse_error(\n                        \"oauth.oidc.signature-algorithm\",\n                        format!(\"Invalid OIDC signature algorithm: {}\", alg),\n                    );\n                    SignatureAlgorithm::HS256\n                }\n            },\n            None => SignatureAlgorithm::HS256,\n        };\n\n        let rand_key = rng()\n            .sample_iter(Alphanumeric)\n            .take(64)\n            .map(char::from)\n            .collect::<String>()\n            .into_bytes();\n\n        let (oidc_signing_secret, algorithm) = match oidc_signature_algorithm {\n            SignatureAlgorithm::None\n            | SignatureAlgorithm::HS256\n            | SignatureAlgorithm::HS384\n            | SignatureAlgorithm::HS512 => {\n                let key = config\n                    .value(\"oauth.oidc.signature-key\")\n                    .map(|s| s.to_string().into_bytes())\n                    .unwrap_or(rand_key);\n\n                (\n                    Secret::Bytes(key.clone()),\n                    AlgorithmParameters::OctetKey(OctetKeyParameters {\n                        key_type: OctetKeyType::Octet,\n                        value: key,\n                    }),\n                )\n            }\n            SignatureAlgorithm::RS256\n            | SignatureAlgorithm::RS384\n            | SignatureAlgorithm::RS512\n            | SignatureAlgorithm::PS256\n            | SignatureAlgorithm::PS384\n            | SignatureAlgorithm::PS512 => parse_rsa_key(config).unwrap_or_else(|| {\n                (\n                    Secret::Bytes(rand_key.clone()),\n                    AlgorithmParameters::OctetKey(OctetKeyParameters {\n                        key_type: OctetKeyType::Octet,\n                        value: rand_key,\n                    }),\n                )\n            }),\n            SignatureAlgorithm::ES256 | SignatureAlgorithm::ES384 | SignatureAlgorithm::ES512 => {\n                parse_ecdsa_key(config, oidc_signature_algorithm).unwrap_or_else(|| {\n                    (\n                        Secret::Bytes(rand_key.clone()),\n                        AlgorithmParameters::OctetKey(OctetKeyParameters {\n                            key_type: OctetKeyType::Octet,\n                            value: rand_key,\n                        }),\n                    )\n                })\n            }\n        };\n\n        let oidc_jwks = Resource {\n            content_type: \"application/json\".into(),\n            contents: serde_json::to_string(&JWKSet {\n                keys: vec![JWK {\n                    common: CommonParameters {\n                        public_key_use: PublicKeyUse::Signature.into(),\n                        algorithm: Algorithm::Signature(oidc_signature_algorithm).into(),\n                        key_id: \"default\".to_string().into(),\n                        ..Default::default()\n                    },\n                    algorithm,\n                    additional: (),\n                }],\n            })\n            .unwrap_or_default()\n            .into_bytes(),\n        };\n\n        OAuthConfig {\n            oauth_key: config\n                .value(\"oauth.key\")\n                .map(|s| s.to_string())\n                .unwrap_or_else(|| {\n                    rng()\n                        .sample_iter(Alphanumeric)\n                        .take(64)\n                        .map(char::from)\n                        .collect::<String>()\n                }),\n            oauth_expiry_user_code: config\n                .property_or_default::<Duration>(\"oauth.expiry.user-code\", \"30m\")\n                .unwrap_or_else(|| Duration::from_secs(30 * 60))\n                .as_secs(),\n            oauth_expiry_auth_code: config\n                .property_or_default::<Duration>(\"oauth.expiry.auth-code\", \"10m\")\n                .unwrap_or_else(|| Duration::from_secs(10 * 60))\n                .as_secs(),\n            oauth_expiry_token: config\n                .property_or_default::<Duration>(\"oauth.expiry.token\", \"1h\")\n                .unwrap_or_else(|| Duration::from_secs(60 * 60))\n                .as_secs(),\n            oauth_expiry_refresh_token: config\n                .property_or_default::<Duration>(\"oauth.expiry.refresh-token\", \"30d\")\n                .unwrap_or_else(|| Duration::from_secs(30 * 24 * 60 * 60))\n                .as_secs(),\n            oauth_expiry_refresh_token_renew: config\n                .property_or_default::<Duration>(\"oauth.expiry.refresh-token-renew\", \"4d\")\n                .unwrap_or_else(|| Duration::from_secs(4 * 24 * 60 * 60))\n                .as_secs(),\n            oauth_max_auth_attempts: config\n                .property_or_default(\"oauth.auth.max-attempts\", \"3\")\n                .unwrap_or(10),\n            oidc_expiry_id_token: config\n                .property_or_default::<Duration>(\"oauth.oidc.expiry.id-token\", \"15m\")\n                .unwrap_or_else(|| Duration::from_secs(15 * 60))\n                .as_secs(),\n            allow_anonymous_client_registration: config\n                .property_or_default(\"oauth.client-registration.anonymous\", \"false\")\n                .unwrap_or(false),\n            require_client_authentication: config\n                .property_or_default(\"oauth.client-registration.require\", \"false\")\n                .unwrap_or(true),\n            oidc_signing_secret,\n            oidc_signature_algorithm,\n            oidc_jwks,\n        }\n    }\n}\n\nimpl Default for OAuthConfig {\n    fn default() -> Self {\n        Self {\n            oauth_key: Default::default(),\n            oauth_expiry_user_code: Default::default(),\n            oauth_expiry_auth_code: Default::default(),\n            oauth_expiry_token: Default::default(),\n            oauth_expiry_refresh_token: Default::default(),\n            oauth_expiry_refresh_token_renew: Default::default(),\n            oauth_max_auth_attempts: Default::default(),\n            oidc_expiry_id_token: Default::default(),\n            allow_anonymous_client_registration: Default::default(),\n            require_client_authentication: Default::default(),\n            oidc_signing_secret: Secret::Bytes(\"secret\".to_string().into_bytes()),\n            oidc_signature_algorithm: SignatureAlgorithm::HS256,\n            oidc_jwks: Resource {\n                content_type: \"application/json\".into(),\n                contents: serde_json::to_string(&JWKSet::<()> { keys: vec![] })\n                    .unwrap_or_default()\n                    .into_bytes(),\n            },\n        }\n    }\n}\n\nfn parse_rsa_key(config: &mut Config) -> Option<(Secret, AlgorithmParameters)> {\n    let rsa_key_pair = match build_rsa_keypair(config.value_require(\"oauth.oidc.signature-key\")?) {\n        Ok(key) => key,\n        Err(err) => {\n            config.new_build_error(\n                \"oauth.oidc.signature-key\",\n                format!(\"Failed to build RSA key: {}\", err),\n            );\n            return None;\n        }\n    };\n\n    let rsa_public_key = match RsaPublicKey::from_pkcs1_der(rsa_key_pair.public_key().as_ref()) {\n        Ok(key) => key,\n        Err(err) => {\n            config.new_build_error(\n                \"oauth.oidc.signature-key\",\n                format!(\"Failed to obtain RSA public key: {}\", err),\n            );\n            return None;\n        }\n    };\n\n    let rsa_key_params = RSAKeyParameters {\n        key_type: RSAKeyType::RSA,\n        n: BigUint::from_bytes_be(&rsa_public_key.n().to_bytes_be()),\n        e: BigUint::from_bytes_be(&rsa_public_key.e().to_bytes_be()),\n        ..Default::default()\n    };\n\n    (\n        Secret::RsaKeyPair(rsa_key_pair.into()),\n        AlgorithmParameters::RSA(rsa_key_params),\n    )\n        .into()\n}\n\nfn parse_ecdsa_key(\n    config: &mut Config,\n    oidc_signature_algorithm: SignatureAlgorithm,\n) -> Option<(Secret, AlgorithmParameters)> {\n    let (alg, curve) = match oidc_signature_algorithm {\n        SignatureAlgorithm::ES256 => (\n            &signature::ECDSA_P256_SHA256_FIXED_SIGNING,\n            EllipticCurve::P256,\n        ),\n        SignatureAlgorithm::ES384 => (\n            &signature::ECDSA_P384_SHA384_FIXED_SIGNING,\n            EllipticCurve::P384,\n        ),\n        _ => unreachable!(),\n    };\n\n    let ecdsa_key_pair =\n        match build_ecdsa_pem(alg, config.value_require(\"oauth.oidc.signature-key\")?) {\n            Ok(key) => key,\n            Err(err) => {\n                config.new_build_error(\n                    \"oauth.oidc.signature-key\",\n                    format!(\"Failed to build ECDSA key: {}\", err),\n                );\n                return None;\n            }\n        };\n\n    let ecdsa_public_key = ecdsa_key_pair.public_key().as_ref();\n\n    let (x, y) = match oidc_signature_algorithm {\n        SignatureAlgorithm::ES256 => {\n            let points = match p256::EncodedPoint::from_bytes(ecdsa_public_key) {\n                Ok(points) => points,\n                Err(err) => {\n                    config.new_build_error(\n                        \"oauth.oidc.signature-key\",\n                        format!(\"Failed to parse ECDSA key: {}\", err),\n                    );\n                    return None;\n                }\n            };\n\n            (\n                points.x().map(|x| x.to_vec()).unwrap_or_default(),\n                points.y().map(|y| y.to_vec()).unwrap_or_default(),\n            )\n        }\n        SignatureAlgorithm::ES384 => {\n            let points = match p384::EncodedPoint::from_bytes(ecdsa_public_key) {\n                Ok(points) => points,\n                Err(err) => {\n                    config.new_build_error(\n                        \"oauth.oidc.signature-key\",\n                        format!(\"Failed to parse ECDSA key: {}\", err),\n                    );\n                    return None;\n                }\n            };\n\n            (\n                points.x().map(|x| x.to_vec()).unwrap_or_default(),\n                points.y().map(|y| y.to_vec()).unwrap_or_default(),\n            )\n        }\n        _ => unreachable!(),\n    };\n\n    let ecdsa_key_params = EllipticCurveKeyParameters {\n        key_type: EllipticCurveKeyType::EC,\n        curve,\n        x,\n        y,\n        d: None,\n    };\n\n    (\n        Secret::EcdsaKeyPair(ecdsa_key_pair.into()),\n        AlgorithmParameters::EllipticCurve(ecdsa_key_params),\n    )\n        .into()\n}\n"
  },
  {
    "path": "crates/common/src/auth/oauth/crypto.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse aes_gcm_siv::{AeadInPlace, Aes256GcmSiv, KeyInit, Nonce, aead::Aead};\nuse store::blake3;\n\npub struct SymmetricEncrypt {\n    aes: Aes256GcmSiv,\n}\n\n//TODO: Remove allow deprecated when aes-gcm is updated\n#[allow(deprecated)]\nimpl SymmetricEncrypt {\n    pub const ENCRYPT_TAG_LEN: usize = 16;\n    pub const NONCE_LEN: usize = 12;\n\n    pub fn new(key: &[u8], context: &str) -> Self {\n        SymmetricEncrypt {\n            aes: Aes256GcmSiv::new(\n                &sha1::digest::generic_array::GenericArray::clone_from_slice(\n                    &blake3::derive_key(context, key)[..],\n                ),\n            ),\n        }\n    }\n\n    #[allow(clippy::ptr_arg)]\n    pub fn encrypt_in_place(&self, bytes: &mut Vec<u8>, nonce: &[u8]) -> Result<(), String> {\n        self.aes\n            .encrypt_in_place(Nonce::from_slice(nonce), b\"\", bytes)\n            .map_err(|e| e.to_string())\n    }\n\n    pub fn encrypt(&self, bytes: &[u8], nonce: &[u8]) -> Result<Vec<u8>, String> {\n        self.aes\n            .encrypt(Nonce::from_slice(nonce), bytes)\n            .map_err(|e| e.to_string())\n    }\n\n    pub fn decrypt(&self, bytes: &[u8], nonce: &[u8]) -> Result<Vec<u8>, String> {\n        self.aes\n            .decrypt(Nonce::from_slice(nonce), bytes)\n            .map_err(|e| e.to_string())\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/oauth/introspect.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse serde::{Deserialize, Serialize};\nuse trc::{AddContext, AuthEvent, EventType};\n\nuse crate::{Server, auth::AccessToken};\n\n#[derive(Debug, Default, Clone, Eq, PartialEq, Deserialize, Serialize)]\npub struct OAuthIntrospect {\n    #[serde(default)]\n    pub active: bool,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub scope: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub client_id: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub username: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub token_type: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub exp: Option<i64>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub iat: Option<i64>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub nbf: Option<i64>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub sub: Option<String>,\n}\n\nimpl Server {\n    pub async fn introspect_access_token(\n        &self,\n        token: &str,\n        access_token: &AccessToken,\n    ) -> trc::Result<OAuthIntrospect> {\n        match self.validate_access_token(None, token).await {\n            Ok(token_info) => Ok(OAuthIntrospect {\n                active: true,\n                client_id: Some(token_info.client_id),\n                username: if access_token.primary_id() == token_info.account_id {\n                    access_token.name.clone()\n                } else {\n                    self.get_access_token(token_info.account_id)\n                        .await\n                        .caused_by(trc::location!())?\n                        .name\n                        .clone()\n                }\n                .into(),\n                token_type: Some(\"bearer\".into()),\n                exp: Some(token_info.expiry as i64),\n                iat: Some(token_info.issued_at as i64),\n                ..Default::default()\n            }),\n            Err(err)\n                if matches!(\n                    err.event_type(),\n                    EventType::Auth(AuthEvent::Error) | EventType::Auth(AuthEvent::TokenExpired)\n                ) =>\n            {\n                Ok(OAuthIntrospect::default())\n            }\n            Err(err) => Err(err),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/oauth/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod config;\npub mod crypto;\npub mod introspect;\npub mod oidc;\npub mod registration;\npub mod token;\n\npub const DEVICE_CODE_LEN: usize = 40;\npub const USER_CODE_LEN: usize = 8;\npub const RANDOM_CODE_LEN: usize = 32;\npub const CLIENT_ID_MAX_LEN: usize = 20;\n\npub const USER_CODE_ALPHABET: &[u8] = b\"ABCDEFGHJKLMNPQRSTUVWXYZ23456789\"; // No 0, O, I, 1\n\n#[derive(Debug, Clone, Copy, Eq, PartialEq)]\npub enum GrantType {\n    AccessToken,\n    RefreshToken,\n    LiveTracing,\n    LiveMetrics,\n    Troubleshoot,\n    Rsvp,\n}\n\nimpl GrantType {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            GrantType::AccessToken => \"access_token\",\n            GrantType::RefreshToken => \"refresh_token\",\n            GrantType::LiveTracing => \"live_tracing\",\n            GrantType::LiveMetrics => \"live_metrics\",\n            GrantType::Troubleshoot => \"troubleshoot\",\n            GrantType::Rsvp => \"rsvp\",\n        }\n    }\n\n    pub fn id(&self) -> u8 {\n        match self {\n            GrantType::AccessToken => 0,\n            GrantType::RefreshToken => 1,\n            GrantType::LiveTracing => 2,\n            GrantType::LiveMetrics => 3,\n            GrantType::Troubleshoot => 4,\n            GrantType::Rsvp => 5,\n        }\n    }\n\n    pub fn from_id(id: u8) -> Option<Self> {\n        match id {\n            0 => Some(GrantType::AccessToken),\n            1 => Some(GrantType::RefreshToken),\n            2 => Some(GrantType::LiveTracing),\n            3 => Some(GrantType::LiveMetrics),\n            4 => Some(GrantType::Troubleshoot),\n            5 => Some(GrantType::Rsvp),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/oauth/oidc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt;\n\nuse biscuit::{ClaimsSet, JWT, RegisteredClaims, SingleOrMultiple, jws::RegisteredHeader};\n\nuse serde::{\n    Deserialize, Deserializer, Serialize,\n    de::{self, Visitor},\n};\nuse store::write::now;\n\nuse crate::Server;\n\n#[derive(Debug, Default, Clone, Eq, PartialEq, Deserialize, Serialize)]\npub struct Userinfo {\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub sub: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub name: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub given_name: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub family_name: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub middle_name: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub nickname: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub preferred_username: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub profile: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub picture: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub website: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub email: Option<String>,\n\n    #[serde(default, deserialize_with = \"any_bool\")]\n    #[serde(skip_serializing_if = \"std::ops::Not::not\")]\n    pub email_verified: bool,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub zoneinfo: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub locale: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub updated_at: Option<i64>,\n}\n\n#[derive(Debug, Default, Eq, PartialEq, Deserialize, Serialize)]\npub struct StandardClaims {\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(default)]\n    pub nonce: Option<String>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(default)]\n    pub preferred_username: Option<String>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(default)]\n    pub email: Option<String>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(default)]\n    pub description: Option<String>,\n}\n\nimpl Server {\n    pub fn issue_id_token(\n        &self,\n        subject: impl Into<String>,\n        issuer: impl Into<String>,\n        audience: impl Into<String>,\n        claims: StandardClaims,\n    ) -> trc::Result<String> {\n        let now = now() as i64;\n\n        JWT::new_decoded(\n            From::from(RegisteredHeader {\n                algorithm: self.core.oauth.oidc_signature_algorithm,\n                key_id: Some(\"default\".into()),\n                ..Default::default()\n            }),\n            ClaimsSet::<StandardClaims> {\n                registered: RegisteredClaims {\n                    issuer: Some(issuer.into()),\n                    subject: Some(subject.into()),\n                    audience: Some(SingleOrMultiple::Single(audience.into())),\n                    not_before: Some(now.into()),\n                    issued_at: Some(now.into()),\n                    expiry: Some((now + self.core.oauth.oidc_expiry_id_token as i64).into()),\n                    ..Default::default()\n                },\n                private: claims,\n            },\n        )\n        .into_encoded(&self.core.oauth.oidc_signing_secret)\n        .map(|token| token.unwrap_encoded().to_string())\n        .map_err(|err| {\n            trc::AuthEvent::Error\n                .into_err()\n                .reason(err)\n                .details(\"Failed to encode ID token\")\n        })\n    }\n}\n\nfn any_bool<'de, D>(deserializer: D) -> Result<bool, D::Error>\nwhere\n    D: Deserializer<'de>,\n{\n    struct AnyBoolVisitor;\n\n    impl Visitor<'_> for AnyBoolVisitor {\n        type Value = bool;\n\n        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n            formatter.write_str(\"a boolean value\")\n        }\n\n        fn visit_str<E>(self, value: &str) -> Result<bool, E>\n        where\n            E: de::Error,\n        {\n            match value {\n                \"true\" => Ok(true),\n                \"false\" => Ok(false),\n                _ => Err(E::custom(format!(\"Unknown boolean: {value}\"))),\n            }\n        }\n\n        fn visit_bool<E>(self, value: bool) -> Result<bool, E>\n        where\n            E: de::Error,\n        {\n            Ok(value)\n        }\n    }\n\n    deserializer.deserialize_any(AnyBoolVisitor)\n}\n"
  },
  {
    "path": "crates/common/src/auth/oauth/registration.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse serde::{Deserialize, Serialize};\nuse std::collections::HashMap;\n\n#[derive(Serialize, Deserialize, Debug, Default)]\n#[serde(rename_all = \"snake_case\")]\npub struct ClientRegistrationRequest {\n    pub redirect_uris: Vec<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub response_types: Vec<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub grant_types: Vec<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub application_type: Option<ApplicationType>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub contacts: Vec<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub client_name: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub logo_uri: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub client_uri: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub policy_uri: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub tos_uri: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub jwks_uri: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub jwks: Option<serde_json::Value>, // Using serde_json::Value for flexibility\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub sector_identifier_uri: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub subject_type: Option<SubjectType>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub id_token_signed_response_alg: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub id_token_encrypted_response_alg: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub id_token_encrypted_response_enc: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub userinfo_signed_response_alg: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub userinfo_encrypted_response_alg: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub userinfo_encrypted_response_enc: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub request_object_signing_alg: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub request_object_encryption_alg: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub request_object_encryption_enc: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub token_endpoint_auth_method: Option<TokenEndpointAuthMethod>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub token_endpoint_auth_signing_alg: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub default_max_age: Option<u64>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub require_auth_time: Option<bool>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub default_acr_values: Vec<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub initiate_login_uri: Option<String>,\n\n    #[serde(default)]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub request_uris: Vec<String>,\n\n    #[serde(flatten)]\n    #[serde(skip_serializing_if = \"HashMap::is_empty\")]\n    pub additional_fields: HashMap<String, serde_json::Value>,\n}\n\n#[derive(Serialize, Deserialize, Debug, Default)]\n#[serde(rename_all = \"snake_case\")]\npub struct ClientRegistrationResponse {\n    // Required fields\n    pub client_id: String,\n\n    // Optional fields specific to the response\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub client_secret: Option<String>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub registration_access_token: Option<String>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub registration_client_uri: Option<String>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub client_id_issued_at: Option<u64>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub client_secret_expires_at: Option<u64>,\n\n    // Echo back the request\n    #[serde(flatten)]\n    pub request: ClientRegistrationRequest,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\n#[serde(rename_all = \"lowercase\")]\npub enum ApplicationType {\n    Web,\n    Native,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\n#[serde(rename_all = \"lowercase\")]\npub enum SubjectType {\n    Pairwise,\n    Public,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\n#[serde(rename_all = \"snake_case\")]\npub enum TokenEndpointAuthMethod {\n    ClientSecretPost,\n    ClientSecretBasic,\n    ClientSecretJwt,\n    PrivateKeyJwt,\n    None,\n}\n"
  },
  {
    "path": "crates/common/src/auth/oauth/token.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{CLIENT_ID_MAX_LEN, GrantType, RANDOM_CODE_LEN, crypto::SymmetricEncrypt};\nuse crate::Server;\nuse directory::{PrincipalData, QueryParams};\nuse mail_builder::encoders::base64::base64_encode;\nuse mail_parser::decoders::base64::base64_decode;\nuse std::time::SystemTime;\nuse store::{\n    blake3,\n    rand::{Rng, rng},\n};\nuse trc::AddContext;\nuse utils::codec::leb128::{Leb128Iterator, Leb128Vec};\n\npub struct TokenInfo {\n    pub grant_type: GrantType,\n    pub account_id: u32,\n    pub client_id: String,\n    pub expiry: u64,\n    pub issued_at: u64,\n    pub expires_in: u64,\n}\n\nconst OAUTH_EPOCH: u64 = 946684800; // Jan 1, 2000\n\nimpl Server {\n    pub async fn encode_access_token(\n        &self,\n        grant_type: GrantType,\n        account_id: u32,\n        client_id: &str,\n        expiry_in: u64,\n    ) -> trc::Result<String> {\n        // Build context\n        let mut password_hash = String::new();\n\n        if !matches!(grant_type, GrantType::Rsvp) {\n            if client_id.len() > CLIENT_ID_MAX_LEN {\n                return Err(trc::AuthEvent::Error\n                    .into_err()\n                    .details(\"Client id too long\"));\n            }\n\n            // Include password hash if expiration is over 1 hour\n            if expiry_in > 3600 {\n                password_hash = self\n                    .password_hash(account_id)\n                    .await\n                    .caused_by(trc::location!())?\n            }\n        }\n\n        let key = &self.core.oauth.oauth_key;\n        let context = format!(\n            \"{} {} {} {}\",\n            grant_type.as_str(),\n            client_id,\n            account_id,\n            password_hash\n        );\n\n        // Set expiration time\n        let issued_at = SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .map_or(0, |d| d.as_secs())\n            .saturating_sub(OAUTH_EPOCH); // Jan 1, 2000\n        let expiry = issued_at + expiry_in;\n\n        // Calculate nonce\n        let mut hasher = blake3::Hasher::new();\n        if !password_hash.is_empty() {\n            hasher.update(password_hash.as_bytes());\n        }\n        hasher.update(grant_type.as_str().as_bytes());\n        hasher.update(issued_at.to_be_bytes().as_slice());\n        hasher.update(expiry.to_be_bytes().as_slice());\n        let nonce = hasher\n            .finalize()\n            .as_bytes()\n            .iter()\n            .take(SymmetricEncrypt::NONCE_LEN)\n            .copied()\n            .collect::<Vec<_>>();\n\n        // Encrypt random bytes\n        let mut token = SymmetricEncrypt::new(key.as_bytes(), &context)\n            .encrypt(&rng().random::<[u8; RANDOM_CODE_LEN]>(), &nonce)\n            .map_err(|_| {\n                trc::AuthEvent::Error\n                    .into_err()\n                    .ctx(trc::Key::Reason, \"Failed to encrypt token\")\n                    .caused_by(trc::location!())\n            })?;\n        token.push_leb128(account_id);\n        token.push(grant_type.id());\n        token.push_leb128(issued_at);\n        token.push_leb128(expiry);\n        token.extend_from_slice(client_id.as_bytes());\n\n        Ok(String::from_utf8(base64_encode(&token).unwrap_or_default()).unwrap())\n    }\n\n    pub async fn validate_access_token(\n        &self,\n        expected_grant_type: Option<GrantType>,\n        token_: &str,\n    ) -> trc::Result<TokenInfo> {\n        // Base64 decode token\n        let token = base64_decode(token_.as_bytes()).ok_or_else(|| {\n            trc::AuthEvent::Error\n                .into_err()\n                .ctx(trc::Key::Reason, \"Failed to decode token\")\n                .caused_by(trc::location!())\n                .details(token_.to_string())\n        })?;\n        let (account_id, grant_type, issued_at, expiry, client_id) = token\n            .get((RANDOM_CODE_LEN + SymmetricEncrypt::ENCRYPT_TAG_LEN)..)\n            .and_then(|bytes| {\n                let mut bytes = bytes.iter();\n                (\n                    bytes.next_leb128()?,\n                    GrantType::from_id(bytes.next().copied()?)?,\n                    bytes.next_leb128::<u64>()?,\n                    bytes.next_leb128::<u64>()?,\n                    bytes.copied().map(char::from).collect::<String>(),\n                )\n                    .into()\n            })\n            .ok_or_else(|| {\n                trc::AuthEvent::Error\n                    .into_err()\n                    .ctx(trc::Key::Reason, \"Failed to decode token\")\n                    .caused_by(trc::location!())\n                    .details(token_.to_string())\n            })?;\n\n        // Validate expiration\n        let now = SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .map_or(0, |d| d.as_secs())\n            .saturating_sub(OAUTH_EPOCH); // Jan 1, 2000\n        if expiry <= now || issued_at > now {\n            return Err(trc::AuthEvent::TokenExpired.into_err());\n        }\n\n        // Validate grant type\n        if expected_grant_type.is_some_and(|g| g != grant_type) {\n            return Err(trc::AuthEvent::Error\n                .into_err()\n                .details(\"Invalid grant type\"));\n        }\n\n        // Obtain password hash\n        let password_hash = if !matches!(grant_type, GrantType::Rsvp) && expiry - issued_at > 3600 {\n            self.password_hash(account_id)\n                .await\n                .map_err(|err| trc::AuthEvent::Error.into_err().ctx(trc::Key::Details, err))?\n        } else {\n            \"\".into()\n        };\n\n        // Build context\n        let key = self.core.oauth.oauth_key.clone();\n        let context = format!(\n            \"{} {} {} {}\",\n            grant_type.as_str(),\n            client_id,\n            account_id,\n            password_hash\n        );\n\n        // Calculate nonce\n        let mut hasher = blake3::Hasher::new();\n        if !password_hash.is_empty() {\n            hasher.update(password_hash.as_bytes());\n        }\n        hasher.update(grant_type.as_str().as_bytes());\n        hasher.update(issued_at.to_be_bytes().as_slice());\n        hasher.update(expiry.to_be_bytes().as_slice());\n        let nonce = hasher\n            .finalize()\n            .as_bytes()\n            .iter()\n            .take(SymmetricEncrypt::NONCE_LEN)\n            .copied()\n            .collect::<Vec<_>>();\n\n        // Decrypt\n        SymmetricEncrypt::new(key.as_bytes(), &context)\n            .decrypt(\n                &token[..RANDOM_CODE_LEN + SymmetricEncrypt::ENCRYPT_TAG_LEN],\n                &nonce,\n            )\n            .map_err(|err| {\n                trc::AuthEvent::Error\n                    .into_err()\n                    .ctx(trc::Key::Details, \"Failed to decode token\")\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })?;\n\n        // Success\n        Ok(TokenInfo {\n            grant_type,\n            account_id,\n            client_id,\n            expiry: expiry + OAUTH_EPOCH,\n            issued_at: issued_at + OAUTH_EPOCH,\n            expires_in: expiry - now,\n        })\n    }\n\n    pub async fn password_hash(&self, account_id: u32) -> trc::Result<String> {\n        if account_id != u32::MAX {\n            self.core\n                .storage\n                .directory\n                .query(QueryParams::id(account_id).with_return_member_of(false))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| {\n                    trc::AuthEvent::Error\n                        .into_err()\n                        .details(\"Account no longer exists\")\n                })?\n                .data\n                .into_iter()\n                .filter_map(|v| {\n                    if let PrincipalData::Password(secret) = v {\n                        Some(secret)\n                    } else {\n                        None\n                    }\n                })\n                .next()\n                .ok_or(\n                    trc::AuthEvent::Error\n                        .into_err()\n                        .details(\"Account does not contain secrets\")\n                        .caused_by(trc::location!()),\n                )\n        } else if let Some((_, secret)) = &self.core.jmap.fallback_admin {\n            Ok(secret.into())\n        } else {\n            Err(trc::AuthEvent::Error\n                .into_err()\n                .details(\"Invalid account ID\")\n                .caused_by(trc::location!()))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/rate_limit.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse crate::{\n    KV_RATE_LIMIT_HTTP_ANONYMOUS, KV_RATE_LIMIT_HTTP_AUTHENTICATED, Server, ip_to_bytes,\n    listener::limiter::{InFlight, LimiterResult},\n};\nuse directory::Permission;\nuse trc::AddContext;\n\nuse crate::auth::AccessToken;\n\nimpl Server {\n    pub async fn is_http_authenticated_request_allowed(\n        &self,\n        access_token: &AccessToken,\n    ) -> trc::Result<Option<InFlight>> {\n        let is_rate_allowed = if let Some(rate) = &self.core.jmap.rate_authenticated {\n            self.core\n                .storage\n                .lookup\n                .is_rate_allowed(\n                    KV_RATE_LIMIT_HTTP_AUTHENTICATED,\n                    &access_token.primary_id.to_be_bytes(),\n                    rate,\n                    false,\n                )\n                .await\n                .caused_by(trc::location!())?\n                .is_none()\n        } else {\n            true\n        };\n\n        if is_rate_allowed {\n            match access_token.is_http_request_allowed() {\n                LimiterResult::Allowed(in_flight) => Ok(Some(in_flight)),\n                LimiterResult::Forbidden => {\n                    if access_token.has_permission(Permission::UnlimitedRequests) {\n                        Ok(None)\n                    } else {\n                        Err(trc::LimitEvent::ConcurrentRequest.into_err())\n                    }\n                }\n                LimiterResult::Disabled => Ok(None),\n            }\n        } else if access_token.has_permission(Permission::UnlimitedRequests) {\n            Ok(None)\n        } else {\n            Err(trc::LimitEvent::TooManyRequests.into_err())\n        }\n    }\n\n    pub async fn is_http_anonymous_request_allowed(&self, addr: &IpAddr) -> trc::Result<()> {\n        if let Some(rate) = &self.core.jmap.rate_anonymous\n            && !self.is_ip_allowed(addr)\n            && self\n                .core\n                .storage\n                .lookup\n                .is_rate_allowed(\n                    KV_RATE_LIMIT_HTTP_ANONYMOUS,\n                    &ip_to_bytes(addr),\n                    rate,\n                    false,\n                )\n                .await\n                .caused_by(trc::location!())?\n                .is_some()\n        {\n            return Err(trc::LimitEvent::TooManyRequests.into_err());\n        }\n        Ok(())\n    }\n\n    pub fn is_upload_allowed(&self, access_token: &AccessToken) -> trc::Result<Option<InFlight>> {\n        match access_token.is_upload_allowed() {\n            LimiterResult::Allowed(in_flight) => Ok(Some(in_flight)),\n            LimiterResult::Forbidden => {\n                if access_token.has_permission(Permission::UnlimitedRequests) {\n                    Ok(None)\n                } else {\n                    Err(trc::LimitEvent::ConcurrentUpload.into_err())\n                }\n            }\n            LimiterResult::Disabled => Ok(None),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/roles.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::Server;\nuse ahash::AHashSet;\nuse directory::{\n    Permission, Permissions, QueryParams, ROLE_ADMIN, ROLE_TENANT_ADMIN, ROLE_USER,\n    backend::internal::lookup::DirectoryStore,\n};\nuse std::sync::{Arc, LazyLock};\nuse trc::AddContext;\nuse utils::cache::CacheItemWeight;\n\n#[derive(Debug, Clone, Default)]\npub struct RolePermissions {\n    pub enabled: Permissions,\n    pub disabled: Permissions,\n}\n\nstatic USER_PERMISSIONS: LazyLock<Arc<RolePermissions>> = LazyLock::new(user_permissions);\nstatic ADMIN_PERMISSIONS: LazyLock<Arc<RolePermissions>> = LazyLock::new(admin_permissions);\nstatic TENANT_ADMIN_PERMISSIONS: LazyLock<Arc<RolePermissions>> =\n    LazyLock::new(tenant_admin_permissions);\n\nimpl Server {\n    pub async fn get_role_permissions(&self, role_id: u32) -> trc::Result<Arc<RolePermissions>> {\n        match role_id {\n            ROLE_USER => Ok(USER_PERMISSIONS.clone()),\n            ROLE_ADMIN => Ok(ADMIN_PERMISSIONS.clone()),\n            ROLE_TENANT_ADMIN => Ok(TENANT_ADMIN_PERMISSIONS.clone()),\n            role_id => {\n                match self\n                    .inner\n                    .cache\n                    .permissions\n                    .get_value_or_guard_async(&role_id)\n                    .await\n                {\n                    Ok(permissions) => Ok(permissions),\n                    Err(guard) => {\n                        let permissions = self.build_role_permissions(role_id).await?;\n                        let _ = guard.insert(permissions.clone());\n                        Ok(permissions)\n                    }\n                }\n            }\n        }\n    }\n\n    async fn build_role_permissions(&self, role_id: u32) -> trc::Result<Arc<RolePermissions>> {\n        let mut role_ids = vec![role_id].into_iter();\n        let mut role_ids_stack = vec![];\n        let mut fetched_role_ids = AHashSet::new();\n        let mut return_permissions = RolePermissions::default();\n\n        'outer: loop {\n            if let Some(role_id) = role_ids.next() {\n                // Skip if already fetched\n                if !fetched_role_ids.insert(role_id) {\n                    continue;\n                }\n\n                match role_id {\n                    ROLE_USER => {\n                        return_permissions.enabled.union(&USER_PERMISSIONS.enabled);\n                        return_permissions\n                            .disabled\n                            .union(&USER_PERMISSIONS.disabled);\n                    }\n                    ROLE_ADMIN => {\n                        return_permissions.enabled.union(&ADMIN_PERMISSIONS.enabled);\n                        return_permissions\n                            .disabled\n                            .union(&ADMIN_PERMISSIONS.disabled);\n                        break 'outer;\n                    }\n                    ROLE_TENANT_ADMIN => {\n                        return_permissions\n                            .enabled\n                            .union(&TENANT_ADMIN_PERMISSIONS.enabled);\n                        return_permissions\n                            .disabled\n                            .union(&TENANT_ADMIN_PERMISSIONS.disabled);\n                    }\n                    role_id => {\n                        // Try with the cache\n                        if let Some(role_permissions) = self.inner.cache.permissions.get(&role_id) {\n                            return_permissions.union(role_permissions.as_ref());\n                        } else {\n                            let mut role_permissions = RolePermissions::default();\n\n                            // Obtain principal\n                            let principal = self\n                                .store()\n                                .query(QueryParams::id(role_id).with_return_member_of(true))\n                                .await\n                                .caused_by(trc::location!())?\n                                .ok_or_else(|| {\n                                    trc::SecurityEvent::Unauthorized\n                                        .into_err()\n                                        .details(\n                                            \"Principal not found while building role permissions\",\n                                        )\n                                        .ctx(trc::Key::Id, role_id)\n                                })?;\n\n                            // Add permissions\n                            for permission in principal.permissions() {\n                                if permission.grant {\n                                    role_permissions.enabled.set(permission.permission as usize);\n                                } else {\n                                    role_permissions\n                                        .disabled\n                                        .set(permission.permission as usize);\n                                }\n                            }\n\n                            // Add permissions\n                            return_permissions.union(&role_permissions);\n\n                            // Add parent roles\n                            let mut principal_roles = principal.roles().peekable();\n                            if principal_roles.peek().is_some() {\n                                role_ids_stack.push(role_ids);\n                                role_ids = principal_roles.collect::<Vec<_>>().into_iter();\n                            } else {\n                                // Cache role\n                                self.inner\n                                    .cache\n                                    .permissions\n                                    .insert(role_id, Arc::new(role_permissions));\n                            }\n                        }\n                    }\n                }\n            } else if let Some(prev_role_ids) = role_ids_stack.pop() {\n                role_ids = prev_role_ids;\n            } else {\n                break;\n            }\n        }\n\n        Ok(Arc::new(return_permissions))\n    }\n}\n\nimpl RolePermissions {\n    pub fn union(&mut self, other: &RolePermissions) {\n        self.enabled.union(&other.enabled);\n        self.disabled.union(&other.disabled);\n    }\n\n    pub fn finalize(mut self) -> Permissions {\n        self.enabled.difference(&self.disabled);\n        self.enabled\n    }\n\n    pub fn finalize_as_ref(&self) -> Permissions {\n        let mut enabled = self.enabled.clone();\n        enabled.difference(&self.disabled);\n        enabled\n    }\n}\n\nfn tenant_admin_permissions() -> Arc<RolePermissions> {\n    let mut permissions = RolePermissions::default();\n\n    for permission_id in 0..Permission::COUNT {\n        let permission = Permission::from_id(permission_id as u32).unwrap();\n        if permission.is_tenant_admin_permission() {\n            permissions.enabled.set(permission_id);\n        }\n    }\n\n    Arc::new(permissions)\n}\n\nfn user_permissions() -> Arc<RolePermissions> {\n    let mut permissions = RolePermissions::default();\n\n    for permission_id in 0..Permission::COUNT {\n        let permission = Permission::from_id(permission_id as u32).unwrap();\n        if permission.is_user_permission() {\n            permissions.enabled.set(permission_id);\n        }\n    }\n\n    Arc::new(permissions)\n}\n\nfn admin_permissions() -> Arc<RolePermissions> {\n    Arc::new(RolePermissions {\n        enabled: Permissions::all(),\n        disabled: Permissions::new(),\n    })\n}\n\nimpl CacheItemWeight for RolePermissions {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<RolePermissions>() as u64\n    }\n}\n"
  },
  {
    "path": "crates/common/src/auth/sasl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mail_send::Credentials;\n\npub fn sasl_decode_challenge_plain(challenge: &[u8]) -> Option<Credentials<String>> {\n    let mut username = Vec::new();\n    let mut secret = Vec::new();\n    let mut arg_num = 0;\n    for &ch in challenge {\n        if ch != 0 {\n            if arg_num == 1 {\n                username.push(ch);\n            } else if arg_num == 2 {\n                secret.push(ch);\n            }\n        } else {\n            arg_num += 1;\n        }\n    }\n\n    match (String::from_utf8(username), String::from_utf8(secret)) {\n        (Ok(username), Ok(secret)) if !username.is_empty() && !secret.is_empty() => {\n            Some((username, secret).into())\n        }\n        _ => None,\n    }\n}\n\npub fn sasl_decode_challenge_oauth(challenge: &[u8]) -> Option<Credentials<String>> {\n    extract_oauth_bearer(challenge).map(|s| Credentials::OAuthBearer { token: s.into() })\n}\n\nfn extract_oauth_bearer(bytes: &[u8]) -> Option<&str> {\n    let mut start_pos = 0;\n    let eof = bytes.len().saturating_sub(1);\n\n    for (pos, ch) in bytes.iter().enumerate() {\n        let is_separator = *ch == 1;\n        if is_separator || pos == eof {\n            if bytes\n                .get(start_pos..start_pos + 12)\n                .is_some_and(|s| s.eq_ignore_ascii_case(b\"auth=Bearer \"))\n            {\n                return bytes\n                    .get(start_pos + 12..if is_separator { pos } else { bytes.len() })\n                    .and_then(|s| std::str::from_utf8(s).ok());\n            }\n\n            start_pos = pos + 1;\n        }\n    }\n\n    None\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn test_extract_oauth_bearer() {\n        let input = b\"auth=Bearer validtoken\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, Some(\"validtoken\"));\n\n        let input = b\"auth=Invalid validtoken\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, None);\n\n        let input = b\"auth=Bearer\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, None);\n\n        let input = b\"\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, None);\n\n        let input = b\"auth=Bearer token1\\x01auth=Bearer token2\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, Some(\"token1\"));\n\n        let input = b\"auth=Bearer VALIDTOKEN\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, Some(\"VALIDTOKEN\"));\n\n        let input = b\"auth=Bearer token with spaces\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, Some(\"token with spaces\"));\n\n        let input = b\"auth=Bearer token_with_special_chars!@#\";\n        let result = extract_oauth_bearer(input);\n        assert_eq!(result, Some(\"token_with_special_chars!@#\"));\n\n        let input = \"n,a=user@example.com,\\x01host=server.example.com\\x01port=143\\x01auth=Bearer vF9dft4qmTc2Nvb3RlckBhbHRhdmlzdGEuY29tCg==\\x01\\x01\";\n        let result = extract_oauth_bearer(input.as_bytes());\n        assert_eq!(result, Some(\"vF9dft4qmTc2Nvb3RlckBhbHRhdmlzdGEuY29tCg==\"));\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/groupware.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{str::FromStr, time::Duration};\n\nuse utils::{config::Config, template::Template};\n\n#[derive(Debug, Clone, Default)]\npub struct GroupwareConfig {\n    // DAV settings\n    pub max_request_size: usize,\n    pub dead_property_size: Option<usize>,\n    pub live_property_size: usize,\n    pub max_lock_timeout: u64,\n    pub max_locks_per_user: usize,\n    pub max_results: usize,\n    pub assisted_discovery: bool,\n\n    // Calendar settings\n    pub max_ical_size: usize,\n    pub max_ical_instances: usize,\n    pub max_ical_attendees_per_instance: usize,\n    pub default_calendar_name: Option<String>,\n    pub default_calendar_display_name: Option<String>,\n    pub alarms_enabled: bool,\n    pub alarms_minimum_interval: i64,\n    pub alarms_allow_external_recipients: bool,\n    pub alarms_from_name: String,\n    pub alarms_from_email: Option<String>,\n    pub alarms_template: Template<CalendarTemplateVariable>,\n    pub itip_enabled: bool,\n    pub itip_auto_add: bool,\n    pub itip_inbound_max_ical_size: usize,\n    pub itip_outbound_max_recipients: usize,\n    pub itip_http_rsvp_url: Option<String>,\n    pub itip_http_rsvp_expiration: u64,\n    pub itip_inbox_auto_expunge: Option<u64>,\n    pub itip_template: Template<CalendarTemplateVariable>,\n\n    // Addressbook settings\n    pub max_vcard_size: usize,\n    pub default_addressbook_name: Option<String>,\n    pub default_addressbook_display_name: Option<String>,\n\n    // File storage settings\n    pub max_file_size: usize,\n\n    // Sharing settings\n    pub max_shares_per_item: usize,\n    pub allow_directory_query: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Default, Hash)]\npub enum CalendarTemplateVariable {\n    #[default]\n    PageTitle,\n    Header,\n    Footer,\n    EventTitle,\n    EventDescription,\n    EventDetails,\n    Actions,\n    ActionUrl,\n    ActionName,\n    AttendeesTitle,\n    Attendees,\n    Key,\n    Color,\n    Changed,\n    Value,\n    LogoCid,\n    OldValue,\n    Rsvp,\n}\n\nimpl GroupwareConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        GroupwareConfig {\n            max_request_size: config\n                .property(\"dav.request.max-size\")\n                .unwrap_or(25 * 1024 * 1024),\n            dead_property_size: config\n                .property_or_default::<Option<usize>>(\"dav.property.max-size.dead\", \"1024\")\n                .unwrap_or(Some(1024)),\n            live_property_size: config.property(\"dav.property.max-size.live\").unwrap_or(250),\n            assisted_discovery: config\n                .property(\"dav.collection.assisted-discovery\")\n                .unwrap_or(true),\n            max_lock_timeout: config\n                .property::<Duration>(\"dav.lock.max-timeout\")\n                .map(|d| d.as_secs())\n                .unwrap_or(3600),\n            max_locks_per_user: config.property(\"dav.locks.max-per-user\").unwrap_or(10),\n            max_results: config.property(\"dav.response.max-results\").unwrap_or(2000),\n            default_calendar_name: config\n                .property_or_default::<Option<String>>(\"calendar.default.href-name\", \"default\")\n                .unwrap_or_default(),\n            default_calendar_display_name: config\n                .property_or_default::<Option<String>>(\n                    \"calendar.default.display-name\",\n                    \"Stalwart Calendar\",\n                )\n                .unwrap_or_default(),\n            default_addressbook_name: config\n                .property_or_default::<Option<String>>(\"contacts.default.href-name\", \"default\")\n                .unwrap_or_default(),\n            default_addressbook_display_name: config\n                .property_or_default::<Option<String>>(\n                    \"contacts.default.display-name\",\n                    \"Stalwart Address Book\",\n                )\n                .unwrap_or_default(),\n            max_ical_size: config.property(\"calendar.max-size\").unwrap_or(512 * 1024),\n            max_ical_instances: config\n                .property(\"calendar.max-recurrence-expansions\")\n                .unwrap_or(3000),\n            max_ical_attendees_per_instance: config\n                .property(\"calendar.max-attendees-per-instance\")\n                .unwrap_or(20),\n            max_vcard_size: config.property(\"contacts.max-size\").unwrap_or(512 * 1024),\n            max_file_size: config\n                .property(\"file-storage.max-size\")\n                .unwrap_or(25 * 1024 * 1024),\n            alarms_enabled: config.property(\"calendar.alarms.enabled\").unwrap_or(true),\n            alarms_minimum_interval: config\n                .property_or_default::<Duration>(\"calendar.alarms.minimum-interval\", \"1h\")\n                .unwrap_or(Duration::from_secs(60 * 60))\n                .as_secs() as i64,\n            alarms_allow_external_recipients: config\n                .property(\"calendar.alarms.allow-external-recipients\")\n                .unwrap_or(false),\n            alarms_from_name: config\n                .value(\"calendar.alarms.from.name\")\n                .unwrap_or(\"Stalwart Calendar\")\n                .to_string(),\n            alarms_from_email: config\n                .value(\"calendar.alarms.from.email\")\n                .map(|s| s.to_string()),\n            alarms_template: Template::parse(include_str!(concat!(\n                env!(\"CARGO_MANIFEST_DIR\"),\n                \"/../../resources/html-templates/calendar-alarm.html.min\"\n            )))\n            .expect(\"Failed to parse calendar template\"),\n            itip_enabled: config\n                .property(\"calendar.scheduling.enable\")\n                .unwrap_or(true),\n            itip_auto_add: config\n                .property(\"calendar.scheduling.inbound.auto-add\")\n                .unwrap_or(false),\n            itip_inbound_max_ical_size: config\n                .property(\"calendar.scheduling.inbound.max-size\")\n                .unwrap_or(512 * 1024),\n            itip_outbound_max_recipients: config\n                .property(\"calendar.scheduling.outbound.max-recipients\")\n                .unwrap_or(100),\n            itip_inbox_auto_expunge: config\n                .property_or_default::<Option<Duration>>(\n                    \"calendar.scheduling.inbox.auto-expunge\",\n                    \"30d\",\n                )\n                .map(|d| d.map(|d| d.as_secs()))\n                .unwrap_or(Some(30 * 24 * 60 * 60)),\n            itip_http_rsvp_url: if config\n                .property(\"calendar.scheduling.http-rsvp.enable\")\n                .unwrap_or(true)\n            {\n                if let Some(url) = config\n                    .value(\"calendar.scheduling.http-rsvp.url\")\n                    .map(|v| v.trim().trim_end_matches('/'))\n                    .filter(|v| !v.is_empty())\n                {\n                    Some(url.to_string())\n                } else {\n                    Some(format!(\n                        \"https://{}/calendar/rsvp\",\n                        config.value(\"server.hostname\").unwrap_or(\"localhost\")\n                    ))\n                }\n            } else {\n                None\n            },\n            max_shares_per_item: config.property(\"sharing.max-shares-per-item\").unwrap_or(10),\n            allow_directory_query: config\n                .property(\"sharing.allow-directory-query\")\n                .unwrap_or(false),\n            itip_http_rsvp_expiration: config\n                .property_or_default::<Duration>(\"calendar.scheduling.http-rsvp.expiration\", \"90d\")\n                .map(|d| d.as_secs())\n                .unwrap_or(90 * 24 * 60 * 60),\n            itip_template: Template::parse(include_str!(concat!(\n                env!(\"CARGO_MANIFEST_DIR\"),\n                \"/../../resources/html-templates/calendar-invite.html.min\"\n            )))\n            .expect(\"Failed to parse calendar template\"),\n        }\n    }\n}\n\nimpl FromStr for CalendarTemplateVariable {\n    type Err = String;\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        match s {\n            \"page_title\" => Ok(CalendarTemplateVariable::PageTitle),\n            \"header\" => Ok(CalendarTemplateVariable::Header),\n            \"footer\" => Ok(CalendarTemplateVariable::Footer),\n            \"event_title\" => Ok(CalendarTemplateVariable::EventTitle),\n            \"event_description\" => Ok(CalendarTemplateVariable::EventDescription),\n            \"event_details\" => Ok(CalendarTemplateVariable::EventDetails),\n            \"action_url\" => Ok(CalendarTemplateVariable::ActionUrl),\n            \"action_name\" => Ok(CalendarTemplateVariable::ActionName),\n            \"attendees\" => Ok(CalendarTemplateVariable::Attendees),\n            \"attendees_title\" => Ok(CalendarTemplateVariable::AttendeesTitle),\n            \"key\" => Ok(CalendarTemplateVariable::Key),\n            \"value\" => Ok(CalendarTemplateVariable::Value),\n            \"logo_cid\" => Ok(CalendarTemplateVariable::LogoCid),\n            \"actions\" => Ok(CalendarTemplateVariable::Actions),\n            \"changed\" => Ok(CalendarTemplateVariable::Changed),\n            \"old_value\" => Ok(CalendarTemplateVariable::OldValue),\n            \"rsvp\" => Ok(CalendarTemplateVariable::Rsvp),\n            \"color\" => Ok(CalendarTemplateVariable::Color),\n            _ => Err(format!(\"Unknown calendar template variable: {}\", s)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/imap.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse utils::config::{Config, Rate};\n\n#[derive(Default, Clone)]\npub struct ImapConfig {\n    pub max_request_size: usize,\n    pub max_auth_failures: u32,\n    pub allow_plain_auth: bool,\n\n    pub timeout_auth: Duration,\n    pub timeout_unauth: Duration,\n    pub timeout_idle: Duration,\n\n    pub rate_requests: Option<Rate>,\n    pub rate_concurrent: Option<u64>,\n}\n\nimpl ImapConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        ImapConfig {\n            max_request_size: config\n                .property_or_default(\"imap.request.max-size\", \"52428800\")\n                .unwrap_or(52428800),\n            max_auth_failures: config\n                .property_or_default(\"imap.auth.max-failures\", \"3\")\n                .unwrap_or(3),\n            timeout_auth: config\n                .property_or_default(\"imap.timeout.authenticated\", \"30m\")\n                .unwrap_or_else(|| Duration::from_secs(1800)),\n            timeout_unauth: config\n                .property_or_default(\"imap.timeout.anonymous\", \"1m\")\n                .unwrap_or_else(|| Duration::from_secs(60)),\n            timeout_idle: config\n                .property_or_default(\"imap.timeout.idle\", \"30m\")\n                .unwrap_or_else(|| Duration::from_secs(1800)),\n            rate_requests: config\n                .property_or_default::<Option<Rate>>(\"imap.rate-limit.requests\", \"2000/1m\")\n                .unwrap_or_default(),\n            rate_concurrent: config\n                .property::<Option<u64>>(\"imap.rate-limit.concurrent\")\n                .unwrap_or_default(),\n            allow_plain_auth: config\n                .property_or_default(\"imap.auth.allow-plain-text\", \"false\")\n                .unwrap_or(false),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/inner.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::server::tls::{build_self_signed_cert, parse_certificates};\nuse crate::{\n    CacheSwap, Caches, Data, DavResource, DavResources, MailboxCache, MessageStoreCache,\n    MessageUidCache, TlsConnectors,\n    auth::{AccessToken, roles::RolePermissions},\n    config::{\n        smtp::resolver::{Policy, Tlsa},\n        spamfilter::SpamClassifier,\n    },\n    listener::blocked::BlockedIps,\n    manager::webadmin::WebAdminManager,\n};\nuse ahash::{AHashMap, AHashSet};\nuse arc_swap::ArcSwap;\nuse mail_auth::{MX, Parameters, Txt};\nuse mail_send::smtp::tls::build_tls_connector;\nuse parking_lot::RwLock;\nuse std::{\n    net::{IpAddr, Ipv4Addr, Ipv6Addr},\n    sync::Arc,\n};\nuse utils::{\n    cache::{Cache, CacheWithTtl},\n    config::Config,\n    snowflake::SnowflakeIdGenerator,\n};\n\nimpl Data {\n    pub fn parse(config: &mut Config) -> Self {\n        // Parse certificates\n        let mut certificates = AHashMap::new();\n        let mut subject_names = AHashSet::new();\n        parse_certificates(config, &mut certificates, &mut subject_names);\n        if subject_names.is_empty() {\n            subject_names.insert(\"localhost\".to_string());\n        }\n\n        // Build and test snowflake id generator\n        let node_id = config\n            .property::<u64>(\"cluster.node-id\")\n            .unwrap_or_else(store::rand::random);\n        let id_generator = SnowflakeIdGenerator::with_node_id(node_id);\n        if !id_generator.is_valid() {\n            panic!(\"Invalid system time, panicking to avoid data corruption\");\n        }\n\n        Data {\n            spam_classifier: ArcSwap::from_pointee(SpamClassifier::default()),\n            tls_certificates: ArcSwap::from_pointee(certificates),\n            tls_self_signed_cert: build_self_signed_cert(\n                subject_names.into_iter().collect::<Vec<_>>(),\n            )\n            .or_else(|err| {\n                config.new_build_error(\"certificate.self-signed\", err);\n                build_self_signed_cert(vec![\"localhost\".to_string()])\n            })\n            .ok()\n            .map(Arc::new),\n            blocked_ips: RwLock::new(BlockedIps::parse(config).blocked_ip_addresses),\n            jmap_id_gen: id_generator.clone(),\n            queue_id_gen: id_generator.clone(),\n            span_id_gen: id_generator,\n            queue_status: true.into(),\n            webadmin: config\n                .value(\"webadmin.path\")\n                .map(|path| WebAdminManager::new(path.into()))\n                .unwrap_or_default(),\n            logos: Default::default(),\n            smtp_connectors: TlsConnectors::default(),\n            asn_geo_data: Default::default(),\n        }\n    }\n}\n\nimpl Caches {\n    pub fn parse(config: &mut Config) -> Self {\n        const MB_50: u64 = 50 * 1024 * 1024;\n        const MB_10: u64 = 10 * 1024 * 1024;\n        const MB_5: u64 = 5 * 1024 * 1024;\n        const MB_1: u64 = 1024 * 1024;\n\n        Caches {\n            access_tokens: Cache::from_config(\n                config,\n                \"access-token\",\n                MB_10,\n                (std::mem::size_of::<AccessToken>() + 255) as u64,\n            ),\n            http_auth: Cache::from_config(\n                config,\n                \"http-auth\",\n                MB_1,\n                (50 + std::mem::size_of::<u32>()) as u64,\n            ),\n            permissions: Cache::from_config(\n                config,\n                \"permission\",\n                MB_5,\n                std::mem::size_of::<RolePermissions>() as u64,\n            ),\n            messages: Cache::from_config(\n                config,\n                \"message\",\n                MB_50,\n                (std::mem::size_of::<u32>()\n                    + std::mem::size_of::<CacheSwap<MessageStoreCache>>()\n                    + (1024 * std::mem::size_of::<MessageUidCache>())\n                    + (15 * (std::mem::size_of::<MailboxCache>() + 60))) as u64,\n            ),\n            files: Cache::from_config(\n                config,\n                \"files\",\n                MB_10,\n                (std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))\n                    as u64,\n            ),\n            events: Cache::from_config(\n                config,\n                \"events\",\n                MB_10,\n                (std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))\n                    as u64,\n            ),\n            contacts: Cache::from_config(\n                config,\n                \"contacts\",\n                MB_10,\n                (std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))\n                    as u64,\n            ),\n            scheduling: Cache::from_config(\n                config,\n                \"scheduling\",\n                MB_1,\n                (std::mem::size_of::<DavResources>() + (500 * std::mem::size_of::<DavResource>()))\n                    as u64,\n            ),\n            dns_txt: CacheWithTtl::from_config(\n                config,\n                \"dns.txt\",\n                MB_5,\n                (std::mem::size_of::<Txt>() + 255) as u64,\n            ),\n            dns_mx: CacheWithTtl::from_config(\n                config,\n                \"dns.mx\",\n                MB_5,\n                ((std::mem::size_of::<MX>() + 255) * 2) as u64,\n            ),\n            dns_ptr: CacheWithTtl::from_config(\n                config,\n                \"dns.ptr\",\n                MB_1,\n                (std::mem::size_of::<IpAddr>() + 255) as u64,\n            ),\n            dns_ipv4: CacheWithTtl::from_config(\n                config,\n                \"dns.ipv4\",\n                MB_5,\n                ((std::mem::size_of::<Ipv4Addr>() + 255) * 2) as u64,\n            ),\n            dns_ipv6: CacheWithTtl::from_config(\n                config,\n                \"dns.ipv6\",\n                MB_5,\n                ((std::mem::size_of::<Ipv6Addr>() + 255) * 2) as u64,\n            ),\n            dns_tlsa: CacheWithTtl::from_config(\n                config,\n                \"dns.tlsa\",\n                MB_1,\n                (std::mem::size_of::<Tlsa>() + 255) as u64,\n            ),\n            dbs_mta_sts: CacheWithTtl::from_config(\n                config,\n                \"dns.mta-sts\",\n                MB_1,\n                (std::mem::size_of::<Policy>() + 255) as u64,\n            ),\n            dns_rbl: CacheWithTtl::from_config(\n                config,\n                \"dns.rbl\",\n                MB_5,\n                ((std::mem::size_of::<Ipv4Addr>() + 255) * 2) as u64,\n            ),\n        }\n    }\n\n    #[allow(clippy::type_complexity)]\n    #[inline(always)]\n    pub fn build_auth_parameters<T>(\n        &self,\n        params: T,\n    ) -> Parameters<\n        '_,\n        T,\n        CacheWithTtl<String, Txt>,\n        CacheWithTtl<String, Arc<Vec<MX>>>,\n        CacheWithTtl<String, Arc<Vec<Ipv4Addr>>>,\n        CacheWithTtl<String, Arc<Vec<Ipv6Addr>>>,\n        CacheWithTtl<IpAddr, Arc<Vec<String>>>,\n    > {\n        Parameters {\n            params,\n            cache_txt: Some(&self.dns_txt),\n            cache_mx: Some(&self.dns_mx),\n            cache_ptr: Some(&self.dns_ptr),\n            cache_ipv4: Some(&self.dns_ipv4),\n            cache_ipv6: Some(&self.dns_ipv6),\n        }\n    }\n}\n\nimpl Default for Data {\n    fn default() -> Self {\n        Self {\n            spam_classifier: Default::default(),\n            tls_certificates: Default::default(),\n            tls_self_signed_cert: Default::default(),\n            blocked_ips: Default::default(),\n            jmap_id_gen: Default::default(),\n            queue_id_gen: Default::default(),\n            span_id_gen: Default::default(),\n            queue_status: true.into(),\n            webadmin: Default::default(),\n            logos: Default::default(),\n            smtp_connectors: Default::default(),\n            asn_geo_data: Default::default(),\n        }\n    }\n}\n\nimpl Default for TlsConnectors {\n    fn default() -> Self {\n        TlsConnectors {\n            pki_verify: build_tls_connector(false),\n            dummy_verify: build_tls_connector(true),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/jmap/capabilities.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::settings::JmapConfig;\nuse crate::config::groupware::GroupwareConfig;\nuse ahash::AHashSet;\nuse calcard::icalendar::ICalendarDuration;\nuse chrono::{DateTime, Utc};\nuse jmap_proto::{\n    object::email::EmailComparator,\n    request::capability::{\n        BlobCapabilities, CalendarCapabilities, Capabilities, Capability, ContactsCapabilities,\n        CoreCapabilities, EmptyCapabilities, FileNodeCapabilities, MailCapabilities,\n        PrincipalAvailabilityCapabilities, PrincipalCapabilities, SieveAccountCapabilities,\n        SieveSessionCapabilities, SubmissionCapabilities,\n    },\n    types::date::UTCDate,\n};\nuse types::{collection::Collection, type_state::DataType};\nuse utils::{config::Config, map::vec_map::VecMap};\n\nimpl JmapConfig {\n    pub fn add_capabilities(&mut self, config: &mut Config, groupware_config: &GroupwareConfig) {\n        // Add core capabilities\n        self.capabilities.session.append(\n            Capability::Core,\n            Capabilities::Core(CoreCapabilities {\n                max_size_upload: self.upload_max_size,\n                max_concurrent_upload: self.upload_max_concurrent.unwrap_or(u32::MAX as u64)\n                    as usize,\n                max_size_request: self.request_max_size,\n                max_concurrent_requests: self.request_max_concurrent.unwrap_or(u32::MAX as u64)\n                    as usize,\n                max_calls_in_request: self.request_max_calls,\n                max_objects_in_get: self.get_max_objects,\n                max_objects_in_set: self.set_max_objects,\n                collation_algorithms: vec![\n                    \"i;ascii-numeric\".to_string(),\n                    \"i;ascii-casemap\".to_string(),\n                    \"i;unicode-casemap\".to_string(),\n                ],\n            }),\n        );\n\n        // Add email capabilities\n        self.capabilities.session.append(\n            Capability::Mail,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Mail,\n            Capabilities::Mail(MailCapabilities {\n                max_mailboxes_per_email: None,\n                max_mailbox_depth: self.mailbox_max_depth,\n                max_size_mailbox_name: self.mailbox_name_max_len,\n                max_size_attachments_per_email: self.mail_attachments_max_size,\n                email_query_sort_options: vec![\n                    EmailComparator::ReceivedAt,\n                    EmailComparator::Size,\n                    EmailComparator::From,\n                    EmailComparator::To,\n                    EmailComparator::Subject,\n                    EmailComparator::SentAt,\n                    EmailComparator::HasKeyword(Default::default()),\n                    EmailComparator::AllInThreadHaveKeyword(Default::default()),\n                    EmailComparator::SomeInThreadHaveKeyword(Default::default()),\n                ],\n                may_create_top_level_mailbox: true,\n            }),\n        );\n\n        // Add calendar capabilities\n        self.capabilities.session.append(\n            Capability::Calendars,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Calendars,\n            Capabilities::Calendar(CalendarCapabilities {\n                max_calendars_per_event: None,\n                min_date_time: UTCDate::from_timestamp(DateTime::<Utc>::MIN_UTC.timestamp()),\n                max_date_time: UTCDate::from_timestamp(DateTime::<Utc>::MAX_UTC.timestamp()),\n                max_expanded_query_duration: ICalendarDuration::from_seconds(86400 * 365)\n                    .to_string(),\n                max_participants_per_event: groupware_config.max_ical_attendees_per_instance.into(),\n                may_create_calendar: true,\n            }),\n        );\n\n        self.capabilities.session.append(\n            Capability::CalendarsParse,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::CalendarsParse,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n\n        // Add contacts capabilities\n        self.capabilities.session.append(\n            Capability::Contacts,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Contacts,\n            Capabilities::Contacts(ContactsCapabilities {\n                max_address_books_per_card: None,\n                may_create_address_book: true,\n            }),\n        );\n        self.capabilities.session.append(\n            Capability::ContactsParse,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::ContactsParse,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n\n        // Add file node capabilities\n        self.capabilities.session.append(\n            Capability::FileNode,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::FileNode,\n            Capabilities::FileNode(FileNodeCapabilities {\n                max_file_node_depth: None,\n                max_size_file_node_name: 255,\n                file_node_query_sort_options: vec![],\n                may_create_top_level_file_node: true,\n            }),\n        );\n\n        // Add principal capabilities\n        self.capabilities.session.append(\n            Capability::Principals,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Principals,\n            Capabilities::Principals(PrincipalCapabilities {\n                current_user_principal_id: None,\n            }),\n        );\n        self.capabilities.session.append(\n            Capability::PrincipalsAvailability,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::PrincipalsAvailability,\n            Capabilities::PrincipalsAvailability(PrincipalAvailabilityCapabilities {\n                max_availability_duration: ICalendarDuration::from_seconds(86400 * 365).to_string(),\n            }),\n        );\n\n        // Add submission capabilities\n        self.capabilities.session.append(\n            Capability::Submission,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Submission,\n            Capabilities::Submission(SubmissionCapabilities {\n                max_delayed_send: 86400 * 30,\n                submission_extensions: VecMap::from_iter([\n                    (\"FUTURERELEASE\".to_string(), Vec::new()),\n                    (\"SIZE\".to_string(), Vec::new()),\n                    (\"DSN\".to_string(), Vec::new()),\n                    (\"DELIVERYBY\".to_string(), Vec::new()),\n                    (\"MT-PRIORITY\".to_string(), vec![\"MIXER\".to_string()]),\n                    (\"REQUIRETLS\".to_string(), vec![]),\n                ]),\n            }),\n        );\n\n        // Add vacation response capabilities\n        self.capabilities.session.append(\n            Capability::VacationResponse,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::VacationResponse,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n\n        // Add Sieve capabilities\n        let mut notification_methods = Vec::new();\n\n        for (_, uri) in config.values(\"sieve.untrusted.notification-uris\") {\n            notification_methods.push(uri.to_string());\n        }\n        if notification_methods.is_empty() {\n            notification_methods.push(\"mailto\".to_string());\n        }\n\n        let mut capabilities: AHashSet<sieve::compiler::grammar::Capability> =\n            AHashSet::from_iter(sieve::compiler::grammar::Capability::all().iter().cloned());\n\n        for (_, capability) in config.values(\"sieve.untrusted.disabled-capabilities\") {\n            capabilities.remove(&sieve::compiler::grammar::Capability::parse(capability));\n        }\n\n        let mut extensions = capabilities\n            .into_iter()\n            .map(|c| c.to_string())\n            .collect::<Vec<String>>();\n        extensions.sort_unstable();\n\n        self.capabilities.session.append(\n            Capability::Sieve,\n            Capabilities::SieveSession(SieveSessionCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Sieve,\n            Capabilities::SieveAccount(SieveAccountCapabilities {\n                max_script_name: self.sieve_max_script_name,\n                max_script_size: config\n                    .property(\"sieve.untrusted.max-script-size\")\n                    .unwrap_or(1024 * 1024),\n                max_scripts: self.max_objects[Collection::SieveScript as usize] as usize,\n                max_redirects: config\n                    .property(\"sieve.untrusted.max-redirects\")\n                    .unwrap_or(1),\n                extensions,\n                notification_methods: if !notification_methods.is_empty() {\n                    notification_methods.into()\n                } else {\n                    None\n                },\n                ext_lists: None,\n            }),\n        );\n\n        // Add Blob capabilities\n        self.capabilities.session.append(\n            Capability::Blob,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Blob,\n            Capabilities::Blob(BlobCapabilities {\n                max_size_blob_set: (self.request_max_size * 3 / 4) - 512,\n                max_data_sources: self.request_max_calls,\n                supported_type_names: vec![\n                    DataType::Email,\n                    DataType::Thread,\n                    DataType::SieveScript,\n                ],\n                supported_digest_algorithms: vec![\"sha\", \"sha-256\", \"sha-512\"],\n            }),\n        );\n\n        // Add Quota capabilities\n        self.capabilities.session.append(\n            Capability::Quota,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n        self.capabilities.account.insert(\n            Capability::Quota,\n            Capabilities::Empty(EmptyCapabilities::default()),\n        );\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/jmap/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod capabilities;\npub mod settings;\n"
  },
  {
    "path": "crates/common/src/config/jmap/settings.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::config::groupware::GroupwareConfig;\nuse ahash::{AHashMap, AHashSet};\nuse jmap_proto::request::capability::BaseCapabilities;\nuse nlp::language::Language;\nuse std::{str::FromStr, time::Duration};\nuse store::{search::SearchField, write::SearchIndex};\nuse types::{collection::Collection, special_use::SpecialUse};\nuse utils::{\n    config::{Config, Rate, cron::SimpleCron, utils::ParseValue},\n    map::bitmap::Bitmap,\n};\n\n#[derive(Default, Clone)]\npub struct JmapConfig {\n    pub default_language: Language,\n    pub query_max_results: usize,\n    pub snippet_max_results: usize,\n\n    pub changes_max_results: Option<usize>,\n    pub changes_max_history: Option<usize>,\n    pub share_notification_max_history: Option<Duration>,\n\n    pub request_max_size: usize,\n    pub request_max_calls: usize,\n    pub request_max_concurrent: Option<u64>,\n\n    pub get_max_objects: usize,\n    pub set_max_objects: usize,\n\n    pub upload_max_size: usize,\n    pub upload_max_concurrent: Option<u64>,\n\n    pub upload_tmp_quota_size: usize,\n    pub upload_tmp_quota_amount: usize,\n    pub upload_tmp_ttl: u64,\n\n    pub mailbox_max_depth: usize,\n    pub mailbox_name_max_len: usize,\n    pub mail_attachments_max_size: usize,\n    pub mail_parse_max_items: usize,\n    pub mail_max_size: usize,\n    pub mail_autoexpunge_after: Option<u64>,\n    pub email_submission_autoexpunge_after: Option<u64>,\n\n    pub contact_parse_max_items: usize,\n    pub calendar_parse_max_items: usize,\n\n    pub sieve_max_script_name: usize,\n    pub max_objects: [u32; Collection::MAX],\n\n    pub rate_authenticated: Option<Rate>,\n    pub rate_anonymous: Option<Rate>,\n\n    pub event_source_throttle: Duration,\n    pub push_attempt_interval: Duration,\n    pub push_attempts_max: u32,\n    pub push_retry_interval: Duration,\n    pub push_timeout: Duration,\n    pub push_verify_timeout: Duration,\n    pub push_throttle: Duration,\n\n    pub web_socket_throttle: Duration,\n    pub web_socket_timeout: Duration,\n    pub web_socket_heartbeat: Duration,\n\n    pub fallback_admin: Option<(String, String)>,\n    pub master_user: Option<(String, String)>,\n\n    pub default_folders: Vec<DefaultFolder>,\n    pub shared_folder: String,\n\n    pub http_headers: Vec<(hyper::header::HeaderName, hyper::header::HeaderValue)>,\n    pub http_use_forwarded: bool,\n\n    pub encrypt: bool,\n    pub encrypt_append: bool,\n\n    pub index_batch_size: usize,\n    pub index_fields: AHashMap<SearchIndex, AHashSet<SearchField>>,\n\n    pub capabilities: BaseCapabilities,\n    pub account_purge_frequency: SimpleCron,\n}\n\n#[derive(Clone, Debug)]\npub struct DefaultFolder {\n    pub name: String,\n    pub aliases: Vec<String>,\n    pub special_use: SpecialUse,\n    pub subscribe: bool,\n    pub create: bool,\n}\n\nimpl JmapConfig {\n    pub fn parse(config: &mut Config, groupware_config: &GroupwareConfig) -> Self {\n        // Parse HTTP headers\n        let mut http_headers = config\n            .values(\"http.headers\")\n            .map(|(_, v)| {\n                if let Some((k, v)) = v.split_once(':') {\n                    Ok((\n                        hyper::header::HeaderName::from_str(k.trim()).map_err(|err| {\n                            format!(\"Invalid header found in property \\\"http.headers\\\": {}\", err)\n                        })?,\n                        hyper::header::HeaderValue::from_str(v.trim()).map_err(|err| {\n                            format!(\"Invalid header found in property \\\"http.headers\\\": {}\", err)\n                        })?,\n                    ))\n                } else {\n                    Err(format!(\n                        \"Invalid header found in property \\\"http.headers\\\": {}\",\n                        v\n                    ))\n                }\n            })\n            .collect::<Result<Vec<_>, String>>()\n            .map_err(|e| config.new_parse_error(\"http.headers\", e))\n            .unwrap_or_default();\n\n        // Parse default folders\n        let mut default_folders = Vec::new();\n        let mut shared_folder = \"Shared Folders\".to_string();\n        for key in config.sub_keys(\"email.folders\", \".name\") {\n            match SpecialUse::parse_value(&key) {\n                Ok(SpecialUse::Shared) => {\n                    if let Some(value) = config.value((\"email.folders\", key.as_str(), \"name\")) {\n                        shared_folder = value.to_string();\n                    }\n                }\n                Ok(special_use) => {\n                    let subscribe = config\n                        .property_or_default((\"email.folders\", key.as_str(), \"subscribe\"), \"true\")\n                        .unwrap_or(true);\n                    let create = config\n                        .property_or_default((\"email.folders\", key.as_str(), \"create\"), \"true\")\n                        .unwrap_or(true)\n                        | [SpecialUse::Inbox, SpecialUse::Trash, SpecialUse::Junk]\n                            .contains(&special_use);\n                    if let Some(name) = config\n                        .value((\"email.folders\", key.as_str(), \"name\"))\n                        .map(|name| name.trim())\n                        .filter(|name| !name.is_empty())\n                    {\n                        default_folders.push(DefaultFolder {\n                            name: name.to_string(),\n                            aliases: config\n                                .value((\"email.folders\", key.as_str(), \"aliases\"))\n                                .unwrap_or_default()\n                                .split(',')\n                                .map(|s| s.trim().to_string())\n                                .filter(|s| !s.is_empty())\n                                .collect(),\n                            special_use,\n                            subscribe,\n                            create,\n                        });\n                    }\n                }\n                Err(err) => {\n                    config.new_parse_error(key, err);\n                }\n            }\n        }\n        for (special_use, name) in [\n            (SpecialUse::Inbox, \"Inbox\"),\n            (SpecialUse::Trash, \"Deleted Items\"),\n            (SpecialUse::Junk, \"Junk Mail\"),\n            (SpecialUse::Drafts, \"Drafts\"),\n            (SpecialUse::Sent, \"Sent Items\"),\n        ] {\n            if !default_folders.iter().any(|f| f.special_use == special_use) {\n                default_folders.push(DefaultFolder {\n                    name: name.to_string(),\n                    aliases: Vec::new(),\n                    special_use,\n                    subscribe: true,\n                    create: true,\n                });\n            }\n        }\n\n        // Add permissive CORS headers\n        if config\n            .property::<bool>(\"http.permissive-cors\")\n            .unwrap_or(false)\n        {\n            http_headers.push((\n                hyper::header::ACCESS_CONTROL_ALLOW_ORIGIN,\n                hyper::header::HeaderValue::from_static(\"*\"),\n            ));\n            http_headers.push((\n                hyper::header::ACCESS_CONTROL_ALLOW_HEADERS,\n                hyper::header::HeaderValue::from_static(\n                    \"Authorization, Content-Type, Accept, X-Requested-With\",\n                ),\n            ));\n            http_headers.push((\n                hyper::header::ACCESS_CONTROL_ALLOW_METHODS,\n                hyper::header::HeaderValue::from_static(\n                    \"POST, GET, PATCH, PUT, DELETE, HEAD, OPTIONS\",\n                ),\n            ));\n        }\n\n        // Add HTTP Strict Transport Security\n        if config.property::<bool>(\"http.hsts\").unwrap_or(false) {\n            http_headers.push((\n                hyper::header::STRICT_TRANSPORT_SECURITY,\n                hyper::header::HeaderValue::from_static(\"max-age=31536000; includeSubDomains\"),\n            ));\n        }\n\n        let mut jmap = JmapConfig {\n            default_language: Language::from_iso_639(\n                config\n                    .value(\"storage.search-index.default-language\")\n                    .unwrap_or(\"en\"),\n            )\n            .unwrap_or(Language::English),\n            query_max_results: config\n                .property(\"jmap.protocol.query.max-results\")\n                .unwrap_or(5000),\n            changes_max_results: config\n                .property_or_default::<Option<usize>>(\"jmap.protocol.changes.max-results\", \"5000\")\n                .unwrap_or_default(),\n            changes_max_history: config\n                .property_or_default::<Option<usize>>(\"changes.max-history\", \"10000\")\n                .unwrap_or_default(),\n            share_notification_max_history: config\n                .property_or_default::<Option<Duration>>(\"sharing.max-history\", \"30d\")\n                .unwrap_or_default(),\n            snippet_max_results: config\n                .property(\"jmap.protocol.search-snippet.max-results\")\n                .unwrap_or(100),\n            request_max_size: config\n                .property(\"jmap.protocol.request.max-size\")\n                .unwrap_or(10000000),\n            request_max_calls: config\n                .property(\"jmap.protocol.request.max-calls\")\n                .unwrap_or(16),\n            request_max_concurrent: config\n                .property_or_default::<Option<u64>>(\"jmap.protocol.request.max-concurrent\", \"4\")\n                .unwrap_or(Some(4)),\n            get_max_objects: config\n                .property(\"jmap.protocol.get.max-objects\")\n                .unwrap_or(500),\n            set_max_objects: config\n                .property(\"jmap.protocol.set.max-objects\")\n                .unwrap_or(500),\n            upload_max_size: config\n                .property(\"jmap.protocol.upload.max-size\")\n                .unwrap_or(50000000),\n            upload_max_concurrent: config\n                .property_or_default::<Option<u64>>(\"jmap.protocol.upload.max-concurrent\", \"4\")\n                .unwrap_or(Some(4)),\n            upload_tmp_quota_size: config\n                .property(\"jmap.protocol.upload.quota.size\")\n                .unwrap_or(50000000),\n            upload_tmp_quota_amount: config\n                .property(\"jmap.protocol.upload.quota.files\")\n                .unwrap_or(1000),\n            upload_tmp_ttl: config\n                .property_or_default::<Duration>(\"jmap.protocol.upload.ttl\", \"1h\")\n                .unwrap_or_else(|| Duration::from_secs(3600))\n                .as_secs(),\n            mailbox_max_depth: config.property(\"jmap.mailbox.max-depth\").unwrap_or(10),\n            mailbox_name_max_len: config\n                .property(\"jmap.mailbox.max-name-length\")\n                .unwrap_or(255),\n            mail_attachments_max_size: config\n                .property(\"jmap.email.max-attachment-size\")\n                .unwrap_or(50000000),\n            mail_max_size: config.property(\"jmap.email.max-size\").unwrap_or(75000000),\n            mail_parse_max_items: config.property(\"jmap.email.parse.max-items\").unwrap_or(10),\n            mail_autoexpunge_after: config\n                .property_or_default::<Option<Duration>>(\"email.auto-expunge\", \"30d\")\n                .map(|d| d.map(|d| d.as_secs()))\n                .unwrap_or_default(),\n            email_submission_autoexpunge_after: config\n                .property_or_default::<Option<Duration>>(\"email-submission.auto-expunge\", \"3d\")\n                .map(|d| d.map(|d| d.as_secs()))\n                .unwrap_or_default(),\n            sieve_max_script_name: config\n                .property(\"sieve.untrusted.limits.name-length\")\n                .unwrap_or(512),\n            max_objects: [u32::MAX; Collection::MAX],\n            capabilities: BaseCapabilities::default(),\n            rate_authenticated: config\n                .property_or_default::<Option<Rate>>(\"http.rate-limit.account\", \"1000/1m\")\n                .unwrap_or_default(),\n            rate_anonymous: config\n                .property_or_default::<Option<Rate>>(\"http.rate-limit.anonymous\", \"100/1m\")\n                .unwrap_or_default(),\n            event_source_throttle: config\n                .property_or_default(\"jmap.event-source.throttle\", \"1s\")\n                .unwrap_or_else(|| Duration::from_secs(1)),\n            web_socket_throttle: config\n                .property_or_default(\"jmap.web-socket.throttle\", \"1s\")\n                .unwrap_or_else(|| Duration::from_secs(1)),\n            web_socket_timeout: config\n                .property_or_default(\"jmap.web-socket.timeout\", \"10m\")\n                .unwrap_or_else(|| Duration::from_secs(10 * 60)),\n            web_socket_heartbeat: config\n                .property_or_default(\"jmap.web-socket.heartbeat\", \"1m\")\n                .unwrap_or_else(|| Duration::from_secs(60)),\n            encrypt: config\n                .property_or_default(\"email.encryption.enable\", \"true\")\n                .unwrap_or(true),\n            encrypt_append: config\n                .property_or_default(\"email.encryption.append\", \"false\")\n                .unwrap_or(false),\n            http_use_forwarded: config.property(\"http.use-x-forwarded\").unwrap_or(false),\n            http_headers,\n            push_attempt_interval: config\n                .property_or_default(\"jmap.push.attempts.interval\", \"1m\")\n                .unwrap_or_else(|| Duration::from_secs(60)),\n            push_attempts_max: config\n                .property_or_default(\"jmap.push.attempts.max\", \"3\")\n                .unwrap_or(3),\n            push_retry_interval: config\n                .property_or_default(\"jmap.push.retry.interval\", \"1s\")\n                .unwrap_or_else(|| Duration::from_secs(1)),\n            push_timeout: config\n                .property_or_default(\"jmap.push.timeout.request\", \"10s\")\n                .unwrap_or_else(|| Duration::from_secs(10)),\n            push_verify_timeout: config\n                .property_or_default(\"jmap.push.timeout.verify\", \"1m\")\n                .unwrap_or_else(|| Duration::from_secs(60)),\n            push_throttle: config\n                .property_or_default(\"jmap.push.throttle\", \"1s\")\n                .unwrap_or_else(|| Duration::from_secs(1)),\n            account_purge_frequency: config\n                .property_or_default::<SimpleCron>(\"account.purge.frequency\", \"0 0 *\")\n                .unwrap_or_else(|| SimpleCron::parse_value(\"0 0 *\").unwrap()),\n            fallback_admin: config\n                .value(\"authentication.fallback-admin.user\")\n                .and_then(|u| {\n                    config\n                        .value(\"authentication.fallback-admin.secret\")\n                        .map(|p| (u.to_string(), p.to_string()))\n                }),\n            master_user: config.value(\"authentication.master.user\").and_then(|u| {\n                config\n                    .value(\"authentication.master.secret\")\n                    .map(|p| (u.to_string(), p.to_string()))\n            }),\n            contact_parse_max_items: config\n                .property(\"jmap.contact.parse.max-items\")\n                .unwrap_or(10),\n            calendar_parse_max_items: config\n                .property(\"jmap.calendar.parse.max-items\")\n                .unwrap_or(10),\n            index_batch_size: config\n                .property(\"storage.search-index.batch-size\")\n                .unwrap_or(100),\n            index_fields: AHashMap::new(),\n            default_folders,\n            shared_folder,\n        };\n\n        // Parse index fields\n        for index in [\n            SearchIndex::Email,\n            SearchIndex::Contacts,\n            SearchIndex::Calendar,\n            SearchIndex::Tracing,\n        ] {\n            let mut fields = AHashSet::new();\n            let index_name = match index {\n                SearchIndex::Email => \"email\",\n                SearchIndex::Contacts => \"contacts\",\n                SearchIndex::Calendar => \"calendar\",\n                SearchIndex::Tracing => \"tracing\",\n                _ => unreachable!(),\n            };\n\n            if !config\n                .property_or_default::<bool>(\n                    &format!(\"storage.search-index.{index_name}.enabled\"),\n                    \"true\",\n                )\n                .unwrap_or(true)\n            {\n                continue;\n            }\n\n            for (_, field) in config\n                .properties::<SearchField>(&format!(\"storage.search-index.{index_name}.fields\"))\n            {\n                fields.insert(field);\n            }\n            jmap.index_fields.insert(index, fields);\n        }\n\n        for collection in Bitmap::<Collection>::all() {\n            let key = format!(\"object-quota.{}\", collection.as_config_case());\n            jmap.max_objects[collection as usize] =\n                if let Some(value) = config.property::<u32>(&key) {\n                    value\n                } else {\n                    match collection {\n                        Collection::Mailbox => 250,\n                        Collection::SieveScript => 100,\n                        Collection::Identity => 20,\n                        Collection::EmailSubmission => 500,\n                        Collection::PushSubscription => 15,\n                        Collection::Calendar => 250,\n                        Collection::AddressBook => 250,\n                        Collection::Principal\n                        | Collection::None\n                        | Collection::CalendarEventNotification\n                        | Collection::CalendarEvent\n                        | Collection::ContactCard\n                        | Collection::FileNode\n                        | Collection::Email\n                        | Collection::Thread => u32::MAX,\n                    }\n                };\n        }\n\n        // Add capabilities\n        jmap.add_capabilities(config, groupware_config);\n        jmap\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse self::{\n    imap::ImapConfig, jmap::settings::JmapConfig, scripts::Scripting, smtp::SmtpConfig,\n    storage::Storage,\n};\nuse crate::{\n    Core, Network, Security, auth::oauth::config::OAuthConfig, expr::*,\n    listener::tls::AcmeProviders, manager::config::ConfigManager,\n};\nuse arc_swap::ArcSwap;\nuse directory::{Directories, Directory};\nuse groupware::GroupwareConfig;\nuse hyper::HeaderMap;\nuse ring::signature::{EcdsaKeyPair, RsaKeyPair};\nuse spamfilter::SpamFilterConfig;\nuse std::sync::Arc;\nuse store::{BlobBackend, BlobStore, InMemoryStore, SearchStore, Store, Stores};\nuse telemetry::Metrics;\nuse utils::config::{Config, utils::AsKey};\n\npub mod groupware;\npub mod imap;\npub mod inner;\npub mod jmap;\npub mod network;\npub mod scripts;\npub mod server;\npub mod smtp;\npub mod spamfilter;\npub mod storage;\npub mod telemetry;\n\npub(crate) const CONNECTION_VARS: &[u32; 9] = &[\n    V_LISTENER,\n    V_REMOTE_IP,\n    V_REMOTE_PORT,\n    V_LOCAL_IP,\n    V_LOCAL_PORT,\n    V_PROTOCOL,\n    V_TLS,\n    V_ASN,\n    V_COUNTRY,\n];\n\nimpl Core {\n    pub async fn parse(\n        config: &mut Config,\n        mut stores: Stores,\n        config_manager: ConfigManager,\n    ) -> Self {\n        let mut data = config\n            .value_require(\"storage.data\")\n            .map(|id| id.to_string())\n            .and_then(|id| {\n                if let Some(store) = stores.stores.get(&id) {\n                    store.clone().into()\n                } else {\n                    config.new_parse_error(\"storage.data\", format!(\"Data store {id:?} not found\"));\n                    None\n                }\n            })\n            .unwrap_or_default();\n\n        #[cfg(not(feature = \"enterprise\"))]\n        let is_enterprise = false;\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n        #[cfg(feature = \"enterprise\")]\n        let enterprise =\n            crate::enterprise::Enterprise::parse(config, &config_manager, &stores, &data).await;\n\n        #[cfg(feature = \"enterprise\")]\n        let is_enterprise = enterprise.is_some();\n\n        #[cfg(feature = \"enterprise\")]\n        if !is_enterprise {\n            if data.is_enterprise_store() {\n                config\n                    .new_build_error(\"storage.data\", \"SQL read replicas is an Enterprise feature\");\n                data = Store::None;\n            }\n            stores.disable_enterprise_only();\n        }\n        // SPDX-SnippetEnd\n\n        let mut blob = config\n            .value_require(\"storage.blob\")\n            .map(|id| id.to_string())\n            .and_then(|id| {\n                if let Some(store) = stores.blob_stores.get(&id) {\n                    store.clone().into()\n                } else {\n                    config.new_parse_error(\"storage.blob\", format!(\"Blob store {id:?} not found\"));\n                    None\n                }\n            })\n            .unwrap_or_default();\n        let mut lookup = config\n            .value_require(\"storage.lookup\")\n            .map(|id| id.to_string())\n            .and_then(|id| {\n                if let Some(store) = stores.in_memory_stores.get(&id) {\n                    store.clone().into()\n                } else {\n                    config.new_parse_error(\n                        \"storage.lookup\",\n                        format!(\"In-memory store {id:?} not found\"),\n                    );\n                    None\n                }\n            })\n            .unwrap_or_default();\n        let mut fts = config\n            .value_require(\"storage.fts\")\n            .map(|id| id.to_string())\n            .and_then(|id| {\n                if let Some(store) = stores.search_stores.get(&id) {\n                    store.clone().into()\n                } else {\n                    config.new_parse_error(\n                        \"storage.fts\",\n                        format!(\"Full-text store {id:?} not found\"),\n                    );\n                    None\n                }\n            })\n            .unwrap_or_default();\n        let pubsub = config\n            .value(\"cluster.coordinator\")\n            .map(|id| id.to_string())\n            .and_then(|id| {\n                if let Some(store) = stores.pubsub_stores.get(&id) {\n                    store.clone().into()\n                } else {\n                    config.new_parse_error(\n                        \"cluster.coordinator\",\n                        format!(\"Coordinator backend {id:?} not found\"),\n                    );\n                    None\n                }\n            })\n            .unwrap_or_default();\n        let mut directories =\n            Directories::parse(config, &stores, data.clone(), is_enterprise).await;\n        let directory = config\n            .value_require(\"storage.directory\")\n            .map(|id| id.to_string())\n            .and_then(|id| {\n                if let Some(directory) = directories.directories.get(&id) {\n                    directory.clone().into()\n                } else {\n                    config.new_parse_error(\n                        \"storage.directory\",\n                        format!(\"Directory {id:?} not found\"),\n                    );\n                    None\n                }\n            })\n            .unwrap_or_else(|| Arc::new(Directory::default()));\n        directories\n            .directories\n            .insert(\"*\".to_string(), directory.clone());\n\n        // If any of the stores are missing, disable all stores to avoid data loss\n        if matches!(data, Store::None)\n            || matches!(&blob.backend, BlobBackend::Store(Store::None))\n            || matches!(lookup, InMemoryStore::Store(Store::None))\n            || matches!(fts, SearchStore::Store(Store::None))\n        {\n            data = Store::default();\n            blob = BlobStore::default();\n            lookup = InMemoryStore::default();\n            fts = SearchStore::default();\n            config.new_build_error(\n                \"storage.*\",\n                \"One or more stores are missing, disabling all stores\",\n            )\n        }\n\n        let groupware = GroupwareConfig::parse(config);\n        Self {\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            enterprise,\n            // SPDX-SnippetEnd\n            sieve: Scripting::parse(config, &stores).await,\n            network: Network::parse(config),\n            smtp: SmtpConfig::parse(config).await,\n            jmap: JmapConfig::parse(config, &groupware),\n            imap: ImapConfig::parse(config),\n            oauth: OAuthConfig::parse(config),\n            acme: AcmeProviders::parse(config),\n            metrics: Metrics::parse(config),\n            spam: SpamFilterConfig::parse(config).await,\n            groupware,\n            storage: Storage {\n                data,\n                blob,\n                fts,\n                lookup,\n                pubsub,\n                directory,\n                directories: directories.directories,\n                purge_schedules: stores.purge_schedules,\n                config: config_manager,\n                stores: stores.stores,\n                lookups: stores.in_memory_stores,\n                blobs: stores.blob_stores,\n                ftss: stores.search_stores,\n            },\n        }\n    }\n\n    pub fn into_shared(self) -> ArcSwap<Self> {\n        ArcSwap::from_pointee(self)\n    }\n}\n\npub fn build_rsa_keypair(pem: &str) -> Result<RsaKeyPair, String> {\n    match rustls_pemfile::read_one(&mut pem.as_bytes()) {\n        Ok(Some(rustls_pemfile::Item::Pkcs1Key(key))) => {\n            RsaKeyPair::from_der(key.secret_pkcs1_der())\n                .map_err(|err| format!(\"Failed to parse PKCS1 RSA key: {err}\"))\n        }\n        Ok(Some(rustls_pemfile::Item::Pkcs8Key(key))) => {\n            RsaKeyPair::from_pkcs8(key.secret_pkcs8_der())\n                .map_err(|err| format!(\"Failed to parse PKCS8 RSA key: {err}\"))\n        }\n        Err(err) => Err(format!(\"Failed to read PEM: {err}\")),\n        Ok(Some(key)) => Err(format!(\"Unsupported key type: {key:?}\")),\n        Ok(None) => Err(\"No RSA key found in PEM\".to_string()),\n    }\n}\n\npub fn build_ecdsa_pem(\n    alg: &'static ring::signature::EcdsaSigningAlgorithm,\n    pem: &str,\n) -> Result<EcdsaKeyPair, String> {\n    match rustls_pemfile::read_one(&mut pem.as_bytes()) {\n        Ok(Some(rustls_pemfile::Item::Pkcs8Key(key))) => EcdsaKeyPair::from_pkcs8(\n            alg,\n            key.secret_pkcs8_der(),\n            &ring::rand::SystemRandom::new(),\n        )\n        .map_err(|err| format!(\"Failed to parse PKCS8 ECDSA key: {err}\")),\n        Err(err) => Err(format!(\"Failed to read PEM: {err}\")),\n        Ok(Some(key)) => Err(format!(\"Unsupported key type: {key:?}\")),\n        Ok(None) => Err(\"No ECDSA key found in PEM\".to_string()),\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/network.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::*;\nuse crate::expr::{if_block::IfBlock, tokenizer::TokenMap};\nuse ahash::AHashSet;\nuse std::{hash::Hasher, time::Duration};\nuse utils::config::{Config, Rate, http::parse_http_headers, utils::ParseValue};\nuse xxhash_rust::xxh3::Xxh3Builder;\n\n#[derive(Clone)]\npub struct Network {\n    pub node_id: u64,\n    pub roles: ClusterRoles,\n    pub server_name: String,\n    pub report_domain: String,\n    pub security: Security,\n    pub contact_form: Option<ContactForm>,\n    pub http_response_url: IfBlock,\n    pub http_allowed_endpoint: IfBlock,\n    pub asn_geo_lookup: AsnGeoLookupConfig,\n}\n\n#[derive(Clone)]\npub struct ContactForm {\n    pub rcpt_to: Vec<String>,\n    pub max_size: usize,\n    pub rate: Option<Rate>,\n    pub validate_domain: bool,\n    pub from_email: FieldOrDefault,\n    pub from_subject: FieldOrDefault,\n    pub from_name: FieldOrDefault,\n    pub field_honey_pot: Option<String>,\n}\n\n#[derive(Clone, Default)]\npub struct ClusterRoles {\n    pub purge_stores: ClusterRole,\n    pub purge_accounts: ClusterRole,\n    pub push_notifications: ClusterRole,\n    pub fts_indexing: ClusterRole,\n    pub spam_training: ClusterRole,\n    pub imip_processing: ClusterRole,\n    pub merge_threads: ClusterRole,\n    pub calendar_alerts: ClusterRole,\n    pub renew_acme: ClusterRole,\n    pub calculate_metrics: ClusterRole,\n    pub push_metrics: ClusterRole,\n}\n\n#[derive(Clone, Copy, Default)]\npub enum ClusterRole {\n    #[default]\n    Enabled,\n    Disabled,\n    Sharded {\n        shard_id: u32,\n        total_shards: u32,\n    },\n}\n\n#[derive(Clone, Default)]\npub enum AsnGeoLookupConfig {\n    Resource {\n        expires: Duration,\n        timeout: Duration,\n        max_size: usize,\n        headers: HeaderMap,\n        asn_resources: Vec<String>,\n        geo_resources: Vec<String>,\n    },\n    Dns {\n        zone_ipv4: String,\n        zone_ipv6: String,\n        separator: String,\n        index_asn: usize,\n        index_asn_name: Option<usize>,\n        index_country: Option<usize>,\n    },\n    #[default]\n    Disabled,\n}\n\n#[derive(Clone)]\npub struct FieldOrDefault {\n    pub field: Option<String>,\n    pub default: String,\n}\n\npub(crate) const HTTP_VARS: &[u32; 11] = &[\n    V_LISTENER,\n    V_REMOTE_IP,\n    V_REMOTE_PORT,\n    V_LOCAL_IP,\n    V_LOCAL_PORT,\n    V_PROTOCOL,\n    V_TLS,\n    V_URL,\n    V_URL_PATH,\n    V_HEADERS,\n    V_METHOD,\n];\n\nimpl Default for Network {\n    fn default() -> Self {\n        Self {\n            security: Default::default(),\n            contact_form: None,\n            node_id: 1,\n            http_response_url: IfBlock::new::<()>(\n                \"http.url\",\n                [],\n                \"protocol + '://' + config_get('server.hostname') + ':' + local_port\",\n            ),\n            http_allowed_endpoint: IfBlock::new::<()>(\"http.allowed-endpoint\", [], \"200\"),\n            asn_geo_lookup: AsnGeoLookupConfig::Disabled,\n            server_name: Default::default(),\n            report_domain: Default::default(),\n            roles: ClusterRoles::default(),\n        }\n    }\n}\n\nimpl ContactForm {\n    pub fn parse(config: &mut Config) -> Option<Self> {\n        if !config\n            .property_or_default::<bool>(\"form.enable\", \"false\")\n            .unwrap_or_default()\n        {\n            return None;\n        }\n\n        let form = ContactForm {\n            rcpt_to: config\n                .values(\"form.deliver-to\")\n                .filter_map(|(_, addr)| {\n                    if addr.contains('@') && addr.contains('.') {\n                        Some(addr.trim().to_lowercase())\n                    } else {\n                        None\n                    }\n                })\n                .collect(),\n            max_size: config.property(\"form.max-size\").unwrap_or(100 * 1024),\n            validate_domain: config\n                .property_or_default::<bool>(\"form.validate-domain\", \"true\")\n                .unwrap_or(true),\n            from_email: FieldOrDefault::parse(config, \"form.email\", \"postmaster@localhost\"),\n            from_subject: FieldOrDefault::parse(config, \"form.subject\", \"Contact form submission\"),\n            from_name: FieldOrDefault::parse(config, \"form.name\", \"Anonymous\"),\n            field_honey_pot: config.value(\"form.honey-pot.field\").map(|v| v.into()),\n            rate: config\n                .property_or_default::<Option<Rate>>(\"form.rate-limit\", \"5/1h\")\n                .unwrap_or_default(),\n        };\n\n        if !form.rcpt_to.is_empty() {\n            Some(form)\n        } else {\n            config.new_build_error(\"form.deliver-to\", \"No valid email addresses found\");\n            None\n        }\n    }\n}\n\nimpl FieldOrDefault {\n    pub fn parse(config: &mut Config, key: &str, default: &str) -> Self {\n        FieldOrDefault {\n            field: config.value((key, \"field\")).map(|s| s.to_string()),\n            default: config\n                .value((key, \"default\"))\n                .unwrap_or(default)\n                .to_string(),\n        }\n    }\n}\n\nimpl Network {\n    pub fn parse(config: &mut Config) -> Self {\n        let server_name = config\n            .value(\"server.hostname\")\n            .map(|v| v.to_string())\n            .or_else(|| {\n                config\n                    .value(\"lookup.default.hostname\")\n                    .map(|v| v.to_lowercase())\n            })\n            .unwrap_or_else(|| {\n                hostname::get()\n                    .map(|v| v.to_string_lossy().to_lowercase())\n                    .unwrap_or_else(|_| \"localhost\".to_string())\n            });\n        let report_domain = config\n            .value(\"report.domain\")\n            .map(|v| v.to_lowercase())\n            .or_else(|| {\n                config\n                    .value(\"lookup.default.domain\")\n                    .map(|v| v.to_lowercase())\n            })\n            .unwrap_or_else(|| {\n                psl::domain_str(&server_name)\n                    .unwrap_or(server_name.as_str())\n                    .to_string()\n            });\n\n        let mut network = Network {\n            node_id: config.property(\"cluster.node-id\").unwrap_or(1),\n            report_domain,\n            server_name,\n            security: Security::parse(config),\n            contact_form: ContactForm::parse(config),\n            asn_geo_lookup: AsnGeoLookupConfig::parse(config).unwrap_or_default(),\n            ..Default::default()\n        };\n        let token_map = &TokenMap::default().with_variables(HTTP_VARS);\n\n        // Node roles\n        for (value, key) in [\n            (\n                &mut network.roles.purge_stores,\n                \"cluster.roles.purge.stores\",\n            ),\n            (\n                &mut network.roles.purge_accounts,\n                \"cluster.roles.purge.accounts\",\n            ),\n            (&mut network.roles.renew_acme, \"cluster.roles.acme.renew\"),\n            (\n                &mut network.roles.calculate_metrics,\n                \"cluster.roles.metrics.calculate\",\n            ),\n            (\n                &mut network.roles.push_metrics,\n                \"cluster.roles.metrics.push\",\n            ),\n            (\n                &mut network.roles.push_notifications,\n                \"cluster.roles.push-notifications\",\n            ),\n            (\n                &mut network.roles.fts_indexing,\n                \"cluster.roles.fts-indexing\",\n            ),\n            (\n                &mut network.roles.spam_training,\n                \"cluster.roles.spam-training\",\n            ),\n            (\n                &mut network.roles.imip_processing,\n                \"cluster.roles.imip-processing\",\n            ),\n            (\n                &mut network.roles.calendar_alerts,\n                \"cluster.roles.calendar-alerts\",\n            ),\n            (\n                &mut network.roles.merge_threads,\n                \"cluster.roles.merge-threads\",\n            ),\n        ] {\n            let shards = config\n                .properties::<NodeList>(key)\n                .into_iter()\n                .map(|(_, v)| v)\n                .collect::<Vec<_>>();\n            let shard_size = shards.len() as u32;\n            let mut found_node = false;\n            for (shard_id, shard) in shards.iter().enumerate() {\n                if shard.0.contains(&network.node_id) {\n                    if shard_size > 1 {\n                        *value = ClusterRole::Sharded {\n                            shard_id: shard_id as u32,\n                            total_shards: shard_size,\n                        };\n                    }\n                    found_node = true;\n                    break;\n                }\n            }\n\n            if !shards.is_empty() && !found_node {\n                *value = ClusterRole::Disabled;\n            }\n        }\n\n        for (value, key) in [\n            (&mut network.http_response_url, \"http.url\"),\n            (&mut network.http_allowed_endpoint, \"http.allowed-endpoint\"),\n        ] {\n            if let Some(if_block) = IfBlock::try_parse(config, key, token_map) {\n                *value = if_block;\n            }\n        }\n\n        network\n    }\n}\n\nstruct NodeList(AHashSet<u64>);\n\nimpl ParseValue for NodeList {\n    fn parse_value(value: &str) -> utils::config::Result<Self> {\n        value\n            .split(',')\n            .map(|s| s.trim().parse::<u64>().map_err(|e| e.to_string()))\n            .collect::<Result<AHashSet<u64>, String>>()\n            .map(NodeList)\n    }\n}\n\nimpl AsnGeoLookupConfig {\n    pub fn parse(config: &mut Config) -> Option<Self> {\n        match config.value(\"asn.type\")? {\n            \"dns\" => AsnGeoLookupConfig::Dns {\n                zone_ipv4: config.value_require_non_empty(\"asn.zone.ipv4\")?.to_string(),\n                zone_ipv6: config.value_require_non_empty(\"asn.zone.ipv6\")?.to_string(),\n                separator: config.value_require_non_empty(\"asn.separator\")?.to_string(),\n                index_asn: config.property_require(\"asn.index.asn\")?,\n                index_asn_name: config.property(\"asn.index.asn-name\"),\n                index_country: config.property(\"asn.index.country\"),\n            }\n            .into(),\n            \"resource\" => {\n                let asn_resources = config\n                    .values(\"asn.urls.asn\")\n                    .map(|(_, v)| v.to_string())\n                    .collect::<Vec<_>>();\n                let geo_resources = config\n                    .values(\"asn.urls.geo\")\n                    .map(|(_, v)| v.to_string())\n                    .collect::<Vec<_>>();\n\n                if asn_resources.is_empty() && geo_resources.is_empty() {\n                    config.new_build_error(\"asn.urls\", \"No resources found\");\n                    return None;\n                }\n\n                AsnGeoLookupConfig::Resource {\n                    headers: parse_http_headers(config, \"asn\"),\n                    expires: config.property_or_default::<Duration>(\"asn.expires\", \"1d\")?,\n                    timeout: config.property_or_default::<Duration>(\"asn.timeout\", \"5m\")?,\n                    max_size: config.property(\"asn.max-size\").unwrap_or(100 * 1024 * 1024),\n                    asn_resources,\n                    geo_resources,\n                }\n                .into()\n            }\n            \"disable\" | \"disabled\" | \"none\" | \"false\" => AsnGeoLookupConfig::Disabled.into(),\n            _ => {\n                config.new_build_error(\"asn.type\", \"Invalid value\");\n                None\n            }\n        }\n    }\n}\n\nimpl ClusterRole {\n    pub fn is_enabled_or_sharded(&self) -> bool {\n        matches!(self, ClusterRole::Enabled | ClusterRole::Sharded { .. })\n    }\n\n    pub fn is_enabled_for_integer(&self, value: u32) -> bool {\n        match self {\n            ClusterRole::Enabled => true,\n            ClusterRole::Disabled => false,\n            ClusterRole::Sharded {\n                shard_id,\n                total_shards,\n            } => (value % total_shards) == *shard_id,\n        }\n    }\n\n    pub fn is_enabled_for_hash(&self, item: &impl std::hash::Hash) -> bool {\n        match self {\n            ClusterRole::Enabled => true,\n            ClusterRole::Disabled => false,\n            ClusterRole::Sharded {\n                shard_id,\n                total_shards,\n            } => {\n                let mut hasher = Xxh3Builder::new().with_seed(191179).build();\n                item.hash(&mut hasher);\n                hasher.finish() % (*total_shards as u64) == *shard_id as u64\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/scripts.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{sync::Arc, time::Duration};\n\nuse ahash::AHashMap;\nuse sieve::{Compiler, Runtime, Sieve, compiler::grammar::Capability};\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::{\n    VERSION_PUBLIC,\n    scripts::{\n        functions::{register_functions_trusted, register_functions_untrusted},\n        plugins::RegisterSievePlugins,\n    },\n};\n\nuse super::{if_block::IfBlock, smtp::SMTP_RCPT_TO_VARS, tokenizer::TokenMap};\n\npub struct Scripting {\n    pub untrusted_compiler: Compiler,\n    pub untrusted_runtime: Runtime,\n    pub trusted_runtime: Runtime,\n    pub from_addr: IfBlock,\n    pub from_name: IfBlock,\n    pub return_path: IfBlock,\n    pub sign: IfBlock,\n    pub trusted_scripts: AHashMap<String, Arc<Sieve>>,\n    pub untrusted_scripts: AHashMap<String, Arc<Sieve>>,\n}\n\nimpl Scripting {\n    pub async fn parse(config: &mut Config, stores: &Stores) -> Self {\n        // Parse untrusted compiler\n        let mut fnc_map_untrusted = register_functions_untrusted().register_plugins_untrusted();\n        let untrusted_compiler = Compiler::new()\n            .with_max_script_size(\n                config\n                    .property(\"sieve.untrusted.limits.script-size\")\n                    .unwrap_or(1024 * 1024),\n            )\n            .with_max_string_size(\n                config\n                    .property(\"sieve.untrusted.limits.string-length\")\n                    .unwrap_or(4096),\n            )\n            .with_max_variable_name_size(\n                config\n                    .property(\"sieve.untrusted.limits.variable-name-length\")\n                    .unwrap_or(32),\n            )\n            .with_max_nested_blocks(\n                config\n                    .property(\"sieve.untrusted.limits.nested-blocks\")\n                    .unwrap_or(15),\n            )\n            .with_max_nested_tests(\n                config\n                    .property(\"sieve.untrusted.limits.nested-tests\")\n                    .unwrap_or(15),\n            )\n            .with_max_nested_foreverypart(\n                config\n                    .property(\"sieve.untrusted.limits.nested-foreverypart\")\n                    .unwrap_or(3),\n            )\n            .with_max_match_variables(\n                config\n                    .property(\"sieve.untrusted.limits.match-variables\")\n                    .unwrap_or(30),\n            )\n            .with_max_local_variables(\n                config\n                    .property(\"sieve.untrusted.limits.local-variables\")\n                    .unwrap_or(128),\n            )\n            .with_max_header_size(\n                config\n                    .property(\"sieve.untrusted.limits.header-size\")\n                    .unwrap_or(1024),\n            )\n            .with_max_includes(\n                config\n                    .property(\"sieve.untrusted.limits.includes\")\n                    .unwrap_or(3),\n            )\n            .register_functions(&mut fnc_map_untrusted);\n\n        // Parse untrusted runtime\n        let untrusted_runtime = Runtime::new()\n            .with_functions(&mut fnc_map_untrusted)\n            .with_max_nested_includes(\n                config\n                    .property(\"sieve.untrusted.limits.nested-includes\")\n                    .unwrap_or(3),\n            )\n            .with_cpu_limit(\n                config\n                    .property(\"sieve.untrusted.limits.cpu\")\n                    .unwrap_or(5000),\n            )\n            .with_max_variable_size(\n                config\n                    .property(\"sieve.untrusted.limits.variable-size\")\n                    .unwrap_or(4096),\n            )\n            .with_max_redirects(\n                config\n                    .property(\"sieve.untrusted.limits.redirects\")\n                    .unwrap_or(1),\n            )\n            .with_max_received_headers(\n                config\n                    .property(\"sieve.untrusted.limits.received-headers\")\n                    .unwrap_or(10),\n            )\n            .with_max_header_size(\n                config\n                    .property(\"sieve.untrusted.limits.header-size\")\n                    .unwrap_or(1024),\n            )\n            .with_max_out_messages(\n                config\n                    .property(\"sieve.untrusted.limits.outgoing-messages\")\n                    .unwrap_or(3),\n            )\n            .with_default_vacation_expiry(\n                config\n                    .property::<Duration>(\"sieve.untrusted.default-expiry.vacation\")\n                    .unwrap_or(Duration::from_secs(30 * 86400))\n                    .as_secs(),\n            )\n            .with_default_duplicate_expiry(\n                config\n                    .property::<Duration>(\"sieve.untrusted.default-expiry.duplicate\")\n                    .unwrap_or(Duration::from_secs(7 * 86400))\n                    .as_secs(),\n            )\n            .with_capability(Capability::Expressions)\n            .without_capabilities(\n                config\n                    .values(\"sieve.untrusted.disable-capabilities\")\n                    .map(|(_, v)| v),\n            )\n            .with_valid_notification_uris({\n                let values = config\n                    .values(\"sieve.untrusted.notification-uris\")\n                    .map(|(_, v)| v.to_string())\n                    .collect::<Vec<_>>();\n                if !values.is_empty() {\n                    values\n                } else {\n                    vec![\"mailto\".to_string()]\n                }\n            })\n            .with_protected_headers({\n                let values = config\n                    .values(\"sieve.untrusted.protected-headers\")\n                    .map(|(_, v)| v.to_string())\n                    .collect::<Vec<_>>();\n                if !values.is_empty() {\n                    values\n                } else {\n                    vec![\n                        \"Original-Subject\".to_string(),\n                        \"Original-From\".to_string(),\n                        \"Received\".to_string(),\n                        \"Auto-Submitted\".to_string(),\n                    ]\n                }\n            })\n            .with_vacation_default_subject(\n                config\n                    .value(\"sieve.untrusted.vacation.default-subject\")\n                    .unwrap_or(\"Automated reply\")\n                    .to_string(),\n            )\n            .with_vacation_subject_prefix(\n                config\n                    .value(\"sieve.untrusted.vacation.subject-prefix\")\n                    .unwrap_or(\"Auto: \")\n                    .to_string(),\n            )\n            .with_env_variable(\"name\", \"Stalwart Server\")\n            .with_env_variable(\"version\", VERSION_PUBLIC)\n            .with_env_variable(\"location\", \"MS\")\n            .with_env_variable(\"phase\", \"during\");\n\n        // Parse trusted compiler and runtime\n        let mut fnc_map_trusted = register_functions_trusted().register_plugins_trusted();\n\n        // Allocate compiler and runtime\n        let trusted_compiler = Compiler::new()\n            .with_max_string_size(52428800)\n            .with_max_variable_name_size(100)\n            .with_max_nested_blocks(50)\n            .with_max_nested_tests(50)\n            .with_max_nested_foreverypart(10)\n            .with_max_local_variables(8192)\n            .with_max_header_size(10240)\n            .with_max_includes(10)\n            .with_no_capability_check(\n                config\n                    .property_or_default(\"sieve.trusted.no-capability-check\", \"true\")\n                    .unwrap_or(true),\n            )\n            .register_functions(&mut fnc_map_trusted);\n\n        let mut trusted_runtime = Runtime::new()\n            .without_capabilities([\n                Capability::FileInto,\n                Capability::Vacation,\n                Capability::VacationSeconds,\n                Capability::Fcc,\n                Capability::Mailbox,\n                Capability::MailboxId,\n                Capability::MboxMetadata,\n                Capability::ServerMetadata,\n                Capability::ImapSieve,\n                Capability::Duplicate,\n            ])\n            .with_capability(Capability::Expressions)\n            .with_capability(Capability::While)\n            .with_max_variable_size(\n                config\n                    .property_or_default(\"sieve.trusted.limits.variable-size\", \"52428800\")\n                    .unwrap_or(52428800),\n            )\n            .with_max_header_size(10240)\n            .with_valid_notification_uri(\"mailto\")\n            .with_valid_ext_lists(stores.in_memory_stores.keys().map(|k| k.to_string()))\n            .with_functions(&mut fnc_map_trusted)\n            .with_max_redirects(\n                config\n                    .property_or_default(\"sieve.trusted.limits.redirects\", \"3\")\n                    .unwrap_or(3),\n            )\n            .with_max_out_messages(\n                config\n                    .property_or_default(\"sieve.trusted.limits.out-messages\", \"5\")\n                    .unwrap_or(5),\n            )\n            .with_cpu_limit(\n                config\n                    .property_or_default(\"sieve.trusted.limits.cpu\", \"1048576\")\n                    .unwrap_or(1048576),\n            )\n            .with_max_nested_includes(\n                config\n                    .property_or_default(\"sieve.trusted.limits.nested-includes\", \"5\")\n                    .unwrap_or(5),\n            )\n            .with_max_received_headers(\n                config\n                    .property_or_default(\"sieve.trusted.limits.received-headers\", \"50\")\n                    .unwrap_or(50),\n            )\n            .with_default_duplicate_expiry(\n                config\n                    .property_or_default::<Duration>(\"sieve.trusted.limits.duplicate-expiry\", \"7d\")\n                    .unwrap_or_else(|| Duration::from_secs(604800))\n                    .as_secs(),\n            );\n\n        let hostname = config\n            .value(\"sieve.trusted.hostname\")\n            .or_else(|| config.value(\"server.hostname\"))\n            .unwrap_or(\"localhost\")\n            .to_string();\n        trusted_runtime.set_local_hostname(hostname.clone());\n\n        // Parse trusted scripts\n        let mut trusted_scripts = AHashMap::new();\n        for id in config.sub_keys(\"sieve.trusted.scripts\", \".contents\") {\n            match trusted_compiler.compile(\n                config\n                    .value((\"sieve.trusted.scripts\", id.as_str(), \"contents\"))\n                    .unwrap()\n                    .as_bytes(),\n            ) {\n                Ok(compiled) => {\n                    trusted_scripts.insert(id, compiled.into());\n                }\n                Err(err) => config.new_build_error(\n                    (\"sieve.trusted.scripts\", id.as_str(), \"contents\"),\n                    format!(\"Failed to compile trusted Sieve script: {err}\"),\n                ),\n            }\n        }\n\n        // Parse untrusted scripts\n        let mut untrusted_scripts = AHashMap::new();\n        for id in config.sub_keys(\"sieve.untrusted.scripts\", \".contents\") {\n            match untrusted_compiler.compile(\n                config\n                    .value((\"sieve.untrusted.scripts\", id.as_str(), \"contents\"))\n                    .unwrap()\n                    .as_bytes(),\n            ) {\n                Ok(compiled) => {\n                    untrusted_scripts.insert(id, compiled.into());\n                }\n                Err(err) => config.new_build_error(\n                    (\"sieve.untrusted.scripts\", id.as_str(), \"contents\"),\n                    format!(\"Failed to compile untrusted Sieve script: {err}\"),\n                ),\n            }\n        }\n\n        let token_map = TokenMap::default().with_variables(SMTP_RCPT_TO_VARS);\n\n        Scripting {\n            untrusted_compiler,\n            untrusted_runtime,\n            trusted_runtime,\n            from_addr: IfBlock::try_parse(config, \"sieve.trusted.from-addr\", &token_map)\n                .unwrap_or_else(|| {\n                    IfBlock::new::<()>(\n                        \"sieve.trusted.from-addr\",\n                        [],\n                        \"'MAILER-DAEMON@' + config_get('report.domain')\",\n                    )\n                }),\n            from_name: IfBlock::try_parse(config, \"sieve.trusted.from-name\", &token_map)\n                .unwrap_or_else(|| {\n                    IfBlock::new::<()>(\"sieve.trusted.from-name\", [], \"'Automated Message'\")\n                }),\n            return_path: IfBlock::try_parse(config, \"sieve.trusted.return-path\", &token_map)\n                .unwrap_or_else(|| IfBlock::empty(\"sieve.trusted.return-path\")),\n            sign: IfBlock::try_parse(config, \"sieve.trusted.sign\", &token_map).unwrap_or_else(\n                || {\n                    IfBlock::new::<()>(\n                        \"sieve.trusted.sign\",\n                        [],\n                        concat!(\n                            \"['rsa-' + config_get('report.domain'), \",\n                            \"'ed25519-' + config_get('report.domain')]\"\n                        ),\n                    )\n                },\n            ),\n            untrusted_scripts,\n            trusted_scripts,\n        }\n    }\n}\n\nimpl Default for Scripting {\n    fn default() -> Self {\n        Scripting {\n            untrusted_compiler: Compiler::new(),\n            untrusted_runtime: Runtime::new(),\n            trusted_runtime: Runtime::new(),\n            from_addr: IfBlock::new::<()>(\n                \"sieve.trusted.from-addr\",\n                [],\n                \"'MAILER-DAEMON@' + config_get('report.domain')\",\n            ),\n            from_name: IfBlock::new::<()>(\"sieve.trusted.from-name\", [], \"'Mailer Daemon'\"),\n            return_path: IfBlock::empty(\"sieve.trusted.return-path\"),\n            sign: IfBlock::new::<()>(\n                \"sieve.trusted.sign\",\n                [],\n                concat!(\n                    \"['rsa-' + config_get('report.domain'), \",\n                    \"'ed25519-' + config_get('report.domain')]\"\n                ),\n            ),\n            untrusted_scripts: AHashMap::new(),\n            trusted_scripts: AHashMap::new(),\n        }\n    }\n}\n\nimpl Clone for Scripting {\n    fn clone(&self) -> Self {\n        Self {\n            untrusted_compiler: self.untrusted_compiler.clone(),\n            untrusted_runtime: self.untrusted_runtime.clone(),\n            trusted_runtime: self.trusted_runtime.clone(),\n            from_addr: self.from_addr.clone(),\n            from_name: self.from_name.clone(),\n            return_path: self.return_path.clone(),\n            sign: self.sign.clone(),\n            trusted_scripts: self.trusted_scripts.clone(),\n            untrusted_scripts: self.untrusted_scripts.clone(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/server/listener.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{net::SocketAddr, sync::Arc, time::Duration};\n\nuse rustls::{\n    ALL_VERSIONS, ServerConfig, SupportedCipherSuite,\n    crypto::ring::{ALL_CIPHER_SUITES, default_provider},\n};\n\nuse tokio::net::TcpSocket;\nuse tokio_rustls::TlsAcceptor;\nuse utils::{\n    config::{\n        Config,\n        utils::{AsKey, ParseValue},\n    },\n    snowflake::SnowflakeIdGenerator,\n};\n\nuse crate::{\n    Inner,\n    listener::{TcpAcceptor, tls::CertificateResolver},\n};\n\nuse super::{\n    Listener, Listeners, ServerProtocol, TcpListener,\n    tls::{TLS12_VERSION, TLS13_VERSION},\n};\n\nimpl Listeners {\n    pub fn parse(config: &mut Config) -> Self {\n        // Parse ACME managers\n        let mut servers = Listeners {\n            span_id_gen: Arc::new(\n                config\n                    .property::<u64>(\"cluster.node-id\")\n                    .map(SnowflakeIdGenerator::with_node_id)\n                    .unwrap_or_default(),\n            ),\n            ..Default::default()\n        };\n\n        // Parse servers\n        for id in config.sub_keys(\"server.listener\", \".protocol\") {\n            servers.parse_server(config, id);\n        }\n        servers\n    }\n\n    fn parse_server(&mut self, config: &mut Config, id_: String) {\n        // Parse protocol\n        let id = id_.as_str();\n        let protocol =\n            if let Some(protocol) = config.property_require((\"server.listener\", id, \"protocol\")) {\n                protocol\n            } else {\n                return;\n            };\n\n        // Build listeners\n        let mut listeners = Vec::new();\n        for (_, addr) in config.properties::<SocketAddr>((\"server.listener\", id, \"bind\")) {\n            // Parse bind address and build socket\n            let socket = match if addr.is_ipv4() {\n                TcpSocket::new_v4()\n            } else {\n                TcpSocket::new_v6()\n            } {\n                Ok(socket) => socket,\n                Err(err) => {\n                    config.new_build_error(\n                        (\"server.listener\", id, \"bind\"),\n                        format!(\"Failed to create socket: {err}\"),\n                    );\n                    return;\n                }\n            };\n\n            // Set socket options\n            for option in [\n                \"reuse-addr\",\n                \"reuse-port\",\n                \"send-buffer-size\",\n                \"recv-buffer-size\",\n                \"tos\",\n            ] {\n                if let Some(value) = config.value_or_else(\n                    (\"server.listener\", id, \"socket\", option),\n                    (\"server.socket\", option),\n                ) {\n                    let value = value.to_string();\n                    let key = (\"server.listener\", id, \"socket\", option);\n                    let result = match option {\n                        \"reuse-addr\" => socket\n                            .set_reuseaddr(config.try_parse_value(key, &value).unwrap_or(true)),\n                        #[cfg(not(target_env = \"msvc\"))]\n                        \"reuse-port\" => socket\n                            .set_reuseport(config.try_parse_value(key, &value).unwrap_or(false)),\n                        \"send-buffer-size\" => {\n                            if let Some(value) = config.try_parse_value(key, &value) {\n                                socket.set_send_buffer_size(value)\n                            } else {\n                                continue;\n                            }\n                        }\n                        \"recv-buffer-size\" => {\n                            if let Some(value) = config.try_parse_value(key, &value) {\n                                socket.set_recv_buffer_size(value)\n                            } else {\n                                continue;\n                            }\n                        }\n                        \"tos\" => {\n                            if let Some(value) = config.try_parse_value(key, &value) {\n                                socket.set_tos(value)\n                            } else {\n                                continue;\n                            }\n                        }\n                        _ => continue,\n                    };\n\n                    if let Err(err) = result {\n                        config.new_build_error(key, format!(\"Failed to set socket option: {err}\"));\n                    }\n                }\n            }\n\n            // Set default options\n            if !config.contains_key((\"server.listener\", id, \"socket.reuse-addr\")) {\n                let _ = socket.set_reuseaddr(true);\n            }\n\n            listeners.push(TcpListener {\n                socket,\n                addr,\n                ttl: config\n                    .property_or_else::<Option<u32>>(\n                        (\"server.listener\", id, \"socket.ttl\"),\n                        \"server.socket.ttl\",\n                        \"false\",\n                    )\n                    .unwrap_or_default(),\n                backlog: config\n                    .property_or_else::<Option<u32>>(\n                        (\"server.listener\", id, \"socket.backlog\"),\n                        \"server.socket.backlog\",\n                        \"1024\",\n                    )\n                    .unwrap_or_default(),\n                linger: config\n                    .property_or_else::<Option<Duration>>(\n                        (\"server.listener\", id, \"socket.linger\"),\n                        \"server.socket.linger\",\n                        \"false\",\n                    )\n                    .unwrap_or_default(),\n                nodelay: config\n                    .property_or_else(\n                        (\"server.listener\", id, \"socket.nodelay\"),\n                        \"server.socket.nodelay\",\n                        \"true\",\n                    )\n                    .unwrap_or(true),\n            });\n        }\n\n        if listeners.is_empty() {\n            config.new_build_error(\n                (\"server.listener\", id),\n                \"No 'bind' directive found for listener\",\n            );\n            return;\n        }\n\n        // Parse proxy networks\n        let mut proxy_networks = Vec::new();\n        let proxy_keys = if config\n            .value((\"server.listener\", id, \"proxy.trusted-networks\"))\n            .is_some()\n            || config.has_prefix((\"server.listener\", id, \"proxy.trusted-networks\"))\n        {\n            (\"server.listener\", id, \"proxy.trusted-networks\").as_key()\n        } else {\n            \"server.proxy.trusted-networks\".as_key()\n        };\n        for (_, network) in config.properties(proxy_keys) {\n            proxy_networks.push(network);\n        }\n\n        let span_id_gen = self.span_id_gen.clone();\n        self.servers.push(Listener {\n            max_connections: config\n                .property_or_else(\n                    (\"server.listener\", id, \"max-connections\"),\n                    \"server.max-connections\",\n                    \"8192\",\n                )\n                .unwrap_or(8192),\n            id: id_,\n            protocol,\n            listeners,\n            proxy_networks,\n            span_id_gen,\n        });\n    }\n\n    pub fn parse_tcp_acceptors(&mut self, config: &mut Config, inner: Arc<Inner>) {\n        let resolver = Arc::new(CertificateResolver::new(inner.clone()));\n\n        for id_ in config.sub_keys(\"server.listener\", \".protocol\") {\n            let id = id_.as_str();\n            // Build TLS config\n            let acceptor = if config\n                .property_or_default((\"server.listener\", id, \"tls.enable\"), \"true\")\n                .unwrap_or(true)\n            {\n                // Parse protocol versions\n                let mut tls_v2 = true;\n                let mut tls_v3 = true;\n                let mut proto_err = None;\n                for (_, protocol) in config.values_or_else(\n                    (\"server.listener\", id, \"tls.disable-protocols\"),\n                    \"server.tls.disable-protocols\",\n                ) {\n                    match protocol {\n                        \"TLSv1.2\" | \"0x0303\" => tls_v2 = false,\n                        \"TLSv1.3\" | \"0x0304\" => tls_v3 = false,\n                        protocol => {\n                            proto_err = format!(\"Unsupported TLS protocol {protocol:?}\").into();\n                        }\n                    }\n                }\n\n                if let Some(proto_err) = proto_err {\n                    config.new_parse_error(\n                        (\"server.listener\", id, \"tls.disable-protocols\"),\n                        proto_err,\n                    );\n                }\n\n                // Parse cipher suites\n                let mut disabled_ciphers: Vec<SupportedCipherSuite> = Vec::new();\n                let cipher_keys =\n                    if config.has_prefix((\"server.listener\", id, \"tls.disable-ciphers\")) {\n                        (\"server.listener\", id, \"tls.disable-ciphers\").as_key()\n                    } else {\n                        \"server.tls.disable-ciphers\".as_key()\n                    };\n                for (_, protocol) in config.properties::<SupportedCipherSuite>(cipher_keys) {\n                    disabled_ciphers.push(protocol);\n                }\n\n                // Build cert provider\n                let mut provider = default_provider();\n                if !disabled_ciphers.is_empty() {\n                    provider.cipher_suites = ALL_CIPHER_SUITES\n                        .iter()\n                        .filter(|suite| !disabled_ciphers.contains(suite))\n                        .copied()\n                        .collect();\n                }\n\n                // Build server config\n                let mut server_config = match ServerConfig::builder_with_provider(provider.into())\n                    .with_protocol_versions(if tls_v3 == tls_v2 {\n                        ALL_VERSIONS\n                    } else if tls_v3 {\n                        TLS13_VERSION\n                    } else {\n                        TLS12_VERSION\n                    }) {\n                    Ok(server_config) => server_config\n                        .with_no_client_auth()\n                        .with_cert_resolver(resolver.clone()),\n                    Err(err) => {\n                        config.new_build_error(\n                            (\"server.listener\", id, \"tls\"),\n                            format!(\"Failed to build TLS server config: {err}\"),\n                        );\n                        return;\n                    }\n                };\n\n                server_config.ignore_client_order = config\n                    .property_or_else(\n                        (\"server.listener\", id, \"tls.ignore-client-order\"),\n                        \"server.tls.ignore-client-order\",\n                        \"true\",\n                    )\n                    .unwrap_or(true);\n\n                // Build acceptor\n                let default_config = Arc::new(server_config);\n                TcpAcceptor::Tls {\n                    acceptor: TlsAcceptor::from(default_config.clone()),\n                    config: default_config,\n                    implicit: config\n                        .property_or_default((\"server.listener\", id, \"tls.implicit\"), \"false\")\n                        .unwrap_or(false),\n                }\n            } else {\n                TcpAcceptor::Plain\n            };\n\n            self.tcp_acceptors.insert(id_, acceptor);\n        }\n    }\n}\n\nimpl ParseValue for ServerProtocol {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        if value.eq_ignore_ascii_case(\"smtp\") {\n            Ok(Self::Smtp)\n        } else if value.eq_ignore_ascii_case(\"lmtp\") {\n            Ok(Self::Lmtp)\n        } else if value.eq_ignore_ascii_case(\"imap\") {\n            Ok(Self::Imap)\n        } else if value.eq_ignore_ascii_case(\"http\") | value.eq_ignore_ascii_case(\"https\") {\n            Ok(Self::Http)\n        } else if value.eq_ignore_ascii_case(\"managesieve\") {\n            Ok(Self::ManageSieve)\n        } else if value.eq_ignore_ascii_case(\"pop3\") {\n            Ok(Self::Pop3)\n        } else {\n            Err(format!(\"Invalid server protocol type {:?}.\", value,))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/server/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fmt::Display, net::SocketAddr, sync::Arc, time::Duration};\n\nuse ahash::AHashMap;\nuse serde::{Deserialize, Serialize};\nuse tokio::net::TcpSocket;\nuse utils::{config::ipmask::IpAddrMask, snowflake::SnowflakeIdGenerator};\n\nuse crate::listener::TcpAcceptor;\n\npub mod listener;\npub mod tls;\n\n#[derive(Default)]\npub struct Listeners {\n    pub servers: Vec<Listener>,\n    pub tcp_acceptors: AHashMap<String, TcpAcceptor>,\n    pub span_id_gen: Arc<SnowflakeIdGenerator>,\n}\n\n#[derive(Debug, Default)]\npub struct Listener {\n    pub id: String,\n    pub protocol: ServerProtocol,\n    pub listeners: Vec<TcpListener>,\n    pub proxy_networks: Vec<IpAddrMask>,\n    pub max_connections: u64,\n    pub span_id_gen: Arc<SnowflakeIdGenerator>,\n}\n\n#[derive(Debug)]\npub struct TcpListener {\n    pub socket: TcpSocket,\n    pub addr: SocketAddr,\n    pub backlog: Option<u32>,\n\n    // TCP options\n    pub ttl: Option<u32>,\n    pub linger: Option<Duration>,\n    pub nodelay: bool,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Clone, Copy, Default, Serialize, Deserialize)]\npub enum ServerProtocol {\n    #[default]\n    Smtp,\n    Lmtp,\n    Imap,\n    Pop3,\n    Http,\n    ManageSieve,\n}\n\nimpl ServerProtocol {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            ServerProtocol::Smtp => \"smtp\",\n            ServerProtocol::Lmtp => \"lmtp\",\n            ServerProtocol::Imap => \"imap\",\n            ServerProtocol::Http => \"http\",\n            ServerProtocol::Pop3 => \"pop3\",\n            ServerProtocol::ManageSieve => \"managesieve\",\n        }\n    }\n}\n\nimpl Display for ServerProtocol {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(self.as_str())\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/server/tls.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    io::Cursor,\n    net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr},\n    sync::Arc,\n    time::Duration,\n};\n\nuse ahash::{AHashMap, AHashSet};\nuse base64::{\n    Engine,\n    engine::general_purpose::{self, STANDARD},\n};\nuse dns_update::{DnsUpdater, TsigAlgorithm, providers::rfc2136::DnsAddress};\nuse rcgen::generate_simple_self_signed;\nuse rustls::{\n    SupportedProtocolVersion,\n    crypto::ring::sign::any_supported_type,\n    sign::CertifiedKey,\n    version::{TLS12, TLS13},\n};\nuse rustls_pemfile::{Item, certs, read_one};\nuse rustls_pki_types::PrivateKeyDer;\nuse utils::config::Config;\nuse x509_parser::{\n    certificate::X509Certificate,\n    der_parser::asn1_rs::FromDer,\n    extensions::{GeneralName, ParsedExtension},\n};\n\nuse crate::listener::{\n    acme::{\n        AcmeProvider, ChallengeSettings, EabSettings, directory::LETS_ENCRYPT_PRODUCTION_DIRECTORY,\n    },\n    tls::AcmeProviders,\n};\n\npub static TLS13_VERSION: &[&SupportedProtocolVersion] = &[&TLS13];\npub static TLS12_VERSION: &[&SupportedProtocolVersion] = &[&TLS12];\n\nimpl AcmeProviders {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut providers = AHashMap::new();\n\n        // Parse ACME providers\n        'outer: for acme_id in config.sub_keys(\"acme\", \".directory\") {\n            let acme_id = acme_id.as_str();\n            let directory = config\n                .value((\"acme\", acme_id, \"directory\"))\n                .unwrap_or(LETS_ENCRYPT_PRODUCTION_DIRECTORY)\n                .trim()\n                .to_string();\n            let contact = config\n                .values((\"acme\", acme_id, \"contact\"))\n                .filter_map(|(_, v)| {\n                    let v = v.trim().to_string();\n                    if !v.is_empty() { Some(v) } else { None }\n                })\n                .collect::<Vec<_>>();\n            let renew_before: Duration = config\n                .property_or_default((\"acme\", acme_id, \"renew-before\"), \"30d\")\n                .unwrap_or_else(|| Duration::from_secs(30 * 24 * 60 * 60));\n\n            if directory.is_empty() {\n                config.new_parse_error(format!(\"acme.{acme_id}.directory\"), \"Missing property\");\n                continue;\n            }\n\n            if contact.is_empty() {\n                config.new_parse_error(format!(\"acme.{acme_id}.contact\"), \"Missing property\");\n                continue;\n            }\n\n            // Parse challenge type\n            let challenge = match config\n                .value((\"acme\", acme_id, \"challenge\"))\n                .unwrap_or(\"tls-alpn-01\")\n            {\n                \"tls-alpn-01\" => ChallengeSettings::TlsAlpn01,\n                \"http-01\" => ChallengeSettings::Http01,\n                \"dns-01\" => match build_dns_updater(config, acme_id) {\n                    Some(updater) => ChallengeSettings::Dns01 {\n                        updater,\n                        origin: config\n                            .value((\"acme\", acme_id, \"origin\"))\n                            .map(|s| s.to_string()),\n                        polling_interval: config\n                            .property_or_default((\"acme\", acme_id, \"polling-interval\"), \"15s\")\n                            .unwrap_or_else(|| Duration::from_secs(15)),\n                        propagation_timeout: config\n                            .property_or_default((\"acme\", acme_id, \"propagation-timeout\"), \"1m\")\n                            .unwrap_or_else(|| Duration::from_secs(60)),\n                        ttl: config\n                            .property_or_default((\"acme\", acme_id, \"ttl\"), \"5m\")\n                            .unwrap_or_else(|| Duration::from_secs(5 * 60))\n                            .as_secs() as u32,\n                    },\n                    None => {\n                        continue;\n                    }\n                },\n                _ => {\n                    config\n                        .new_parse_error((\"acme\", acme_id, \"challenge\"), \"Invalid challenge type\");\n                    continue;\n                }\n            };\n\n            // Domains covered by this ACME manager\n            let domains = config\n                .values((\"acme\", acme_id, \"domains\"))\n                .map(|(_, s)| s.trim().to_string())\n                .collect::<Vec<_>>();\n            if !matches!(challenge, ChallengeSettings::Dns01 { .. })\n                && domains.iter().any(|d| d.starts_with(\"*.\"))\n            {\n                config.new_parse_error(\n                    (\"acme\", acme_id, \"domains\"),\n                    \"Wildcard domains are only supported with DNS-01 challenge\",\n                );\n                continue 'outer;\n            }\n\n            // Obtain EAB settings\n            let eab = if let (Some(eab_kid), Some(eab_hmac_key)) = (\n                config\n                    .value((\"acme\", acme_id, \"eab.kid\"))\n                    .filter(|s| !s.is_empty()),\n                config\n                    .value((\"acme\", acme_id, \"eab.hmac-key\"))\n                    .filter(|s| !s.is_empty()),\n            ) {\n                if let Ok(hmac_key) =\n                    general_purpose::URL_SAFE_NO_PAD.decode(eab_hmac_key.trim().as_bytes())\n                {\n                    EabSettings {\n                        kid: eab_kid.to_string(),\n                        hmac_key,\n                    }\n                    .into()\n                } else {\n                    config.new_build_error(\n                        format!(\"acme.{acme_id}.eab.hmac-key\"),\n                        \"Failed to base64 decode HMAC key\",\n                    );\n                    None\n                }\n            } else {\n                None\n            };\n\n            // This ACME manager is the default when SNI is not available\n            let default = config\n                .property::<bool>((\"acme\", acme_id, \"default\"))\n                .unwrap_or_default();\n\n            if !domains.is_empty() {\n                match AcmeProvider::new(\n                    acme_id.to_string(),\n                    directory,\n                    domains,\n                    contact,\n                    challenge,\n                    eab,\n                    renew_before,\n                    default,\n                ) {\n                    Ok(acme_provider) => {\n                        providers.insert(acme_id.to_string(), acme_provider);\n                    }\n                    Err(err) => {\n                        config.new_build_error(format!(\"acme.{acme_id}\"), err.to_string());\n                    }\n                }\n            }\n        }\n\n        AcmeProviders { providers }\n    }\n}\n\n#[allow(clippy::unnecessary_to_owned)]\nfn build_dns_updater(config: &mut Config, acme_id: &str) -> Option<DnsUpdater> {\n    let timeout = config\n        .property_or_default((\"acme\", acme_id, \"timeout\"), \"30s\")\n        .unwrap_or_else(|| Duration::from_secs(30));\n\n    match config.value_require((\"acme\", acme_id, \"provider\"))? {\n        \"rfc2136-tsig\" => {\n            let algorithm: TsigAlgorithm = config\n                .value_require((\"acme\", acme_id, \"tsig-algorithm\"))?\n                .parse()\n                .map_err(|_| {\n                    config.new_parse_error((\"acme\", acme_id, \"tsig-algorithm\"), \"Invalid algorithm\")\n                })\n                .ok()?;\n            let key = STANDARD\n                .decode(config.value_require((\"acme\", acme_id, \"secret\"))?.trim())\n                .map_err(|_| {\n                    config.new_parse_error(\n                        (\"acme\", acme_id, \"secret\"),\n                        \"Failed to base64 decode secret\",\n                    )\n                })\n                .ok()?;\n            let host = config.property_require::<IpAddr>((\"acme\", acme_id, \"host\"))?;\n            let port = config\n                .property_or_default::<u16>((\"acme\", acme_id, \"port\"), \"53\")\n                .unwrap_or(53);\n            let addr = if config.value((\"acme\", acme_id, \"protocol\")) == Some(\"tcp\") {\n                DnsAddress::Tcp(SocketAddr::new(host, port))\n            } else {\n                DnsAddress::Udp(SocketAddr::new(host, port))\n            };\n\n            DnsUpdater::new_rfc2136_tsig(\n                addr,\n                config\n                    .value_require((\"acme\", acme_id, \"key\"))?\n                    .trim()\n                    .to_string(),\n                key,\n                algorithm,\n            )\n            .map_err(|err| {\n                config.new_build_error(\n                    (\"acme\", acme_id, \"provider\"),\n                    format!(\"Failed to create RFC2136-TSIG DNS updater: {err}\"),\n                )\n            })\n            .ok()\n        }\n        \"cloudflare\" => DnsUpdater::new_cloudflare(\n            config\n                .value_require((\"acme\", acme_id, \"secret\"))?\n                .trim()\n                .to_string(),\n            config.value((\"acme\", acme_id, \"user\")).map(|s| s.trim()),\n            timeout.into(),\n        )\n        .map_err(|err| {\n            config.new_build_error(\n                (\"acme\", acme_id, \"provider\"),\n                format!(\"Failed to create Cloudflare DNS updater: {err}\"),\n            )\n        })\n        .ok(),\n        \"digitalocean\" => DnsUpdater::new_digitalocean(\n            config\n                .value_require((\"acme\", acme_id, \"secret\"))?\n                .trim()\n                .to_string(),\n            timeout.into(),\n        )\n        .map_err(|err| {\n            config.new_build_error(\n                (\"acme\", acme_id, \"provider\"),\n                format!(\"Failed to create DigitalOcean DNS updater: {err}\"),\n            )\n        })\n        .ok(),\n        \"desec\" => DnsUpdater::new_desec(\n            config\n                .value_require((\"acme\", acme_id, \"secret\"))?\n                .trim()\n                .to_string(),\n            timeout.into(),\n        )\n        .map_err(|err| {\n            config.new_build_error(\n                (\"acme\", acme_id, \"provider\"),\n                format!(\"Failed to create Desec DNS updater: {err}\"),\n            )\n        })\n        .ok(),\n        \"ovh\" => DnsUpdater::new_ovh(\n            config\n                .value_require((\"acme\", acme_id, \"key\"))\n                .map(|s| s.trim())?\n                .to_string(),\n            config\n                .value_require((\"acme\", acme_id, \"secret\"))?\n                .trim()\n                .to_string(),\n            config\n                .value_require((\"acme\", acme_id, \"consumer-key\"))?\n                .trim()\n                .to_string(),\n            config\n                .value_require((\"acme\", acme_id, \"ovh-endpoint\"))?\n                .parse()\n                .map_err(|_| {\n                    config\n                        .new_parse_error((\"acme\", acme_id, \"ovh-endpoint\"), \"Invalid OVH endpoint\")\n                })\n                .ok()?,\n            timeout.into(),\n        )\n        .map_err(|err| {\n            config.new_build_error(\n                (\"acme\", acme_id, \"provider\"),\n                format!(\"Failed to create OVH DNS updater: {err}\"),\n            )\n        })\n        .ok(),\n        _ => {\n            config.new_parse_error((\"acme\", acme_id, \"provider\"), \"Unsupported provider\");\n            None\n        }\n    }\n}\n\npub(crate) fn parse_certificates(\n    config: &mut Config,\n    certificates: &mut AHashMap<String, Arc<CertifiedKey>>,\n    subject_names: &mut AHashSet<String>,\n) {\n    // Parse certificates\n    for cert_id in config.sub_keys(\"certificate\", \".cert\") {\n        let cert_id = cert_id.as_str();\n        let key_cert = (\"certificate\", cert_id, \"cert\");\n        let key_pk = (\"certificate\", cert_id, \"private-key\");\n\n        let cert = config\n            .value_require(key_cert)\n            .map(|s| s.as_bytes().to_vec());\n        let pk = config.value_require(key_pk).map(|s| s.as_bytes().to_vec());\n\n        if let (Some(cert), Some(pk)) = (cert, pk) {\n            match build_certified_key(cert, pk) {\n                Ok(cert) => {\n                    match cert\n                        .end_entity_cert()\n                        .map_err(|err| format!(\"Failed to obtain end entity cert: {err}\"))\n                        .and_then(|cert| {\n                            X509Certificate::from_der(cert.as_ref())\n                                .map_err(|err| format!(\"Failed to parse end entity cert: {err}\"))\n                        }) {\n                        Ok((_, parsed)) => {\n                            // Add CNs and SANs to the list of names\n                            let mut names = AHashSet::new();\n                            for name in parsed.subject().iter_common_name() {\n                                if let Ok(name) = name.as_str() {\n                                    names.insert(name.to_string());\n                                }\n                            }\n                            for ext in parsed.extensions() {\n                                if let ParsedExtension::SubjectAlternativeName(san) =\n                                    ext.parsed_extension()\n                                {\n                                    for name in &san.general_names {\n                                        let name = match name {\n                                            GeneralName::DNSName(name) => name.to_string(),\n                                            GeneralName::IPAddress(ip) => match ip.len() {\n                                                4 => Ipv4Addr::from(\n                                                    <[u8; 4]>::try_from(*ip).unwrap(),\n                                                )\n                                                .to_string(),\n                                                16 => Ipv6Addr::from(\n                                                    <[u8; 16]>::try_from(*ip).unwrap(),\n                                                )\n                                                .to_string(),\n                                                _ => continue,\n                                            },\n                                            _ => {\n                                                continue;\n                                            }\n                                        };\n                                        names.insert(name);\n                                    }\n                                }\n                            }\n\n                            // Add custom SNIs\n                            names.extend(\n                                config\n                                    .values((\"certificate\", cert_id, \"subjects\"))\n                                    .map(|(_, v)| v.trim().to_string()),\n                            );\n\n                            // Add domain names\n                            subject_names.extend(names.iter().cloned());\n\n                            // Add certificates\n                            let cert = Arc::new(cert);\n                            for name in names {\n                                certificates.insert(\n                                    name.strip_prefix(\"*.\")\n                                        .map(|name| name.to_string())\n                                        .unwrap_or(name),\n                                    cert.clone(),\n                                );\n                            }\n\n                            // Add default certificate\n                            if config\n                                .property::<bool>((\"certificate\", cert_id, \"default\"))\n                                .unwrap_or_default()\n                            {\n                                certificates.insert(\"*\".to_string(), cert.clone());\n                            }\n                        }\n                        Err(err) => config.new_build_error(format!(\"certificate.{cert_id}\"), err),\n                    }\n                }\n                Err(err) => config.new_build_error(format!(\"certificate.{cert_id}\"), err),\n            }\n        }\n    }\n}\n\npub(crate) fn build_certified_key(cert: Vec<u8>, pk: Vec<u8>) -> Result<CertifiedKey, String> {\n    let cert = certs(&mut Cursor::new(cert))\n        .collect::<Result<Vec<_>, _>>()\n        .map_err(|err| format!(\"Failed to read certificates: {err}\"))?;\n    if cert.is_empty() {\n        return Err(\"No certificates found.\".to_string());\n    }\n    let pk = match read_one(&mut Cursor::new(pk))\n        .map_err(|err| format!(\"Failed to read private keys.: {err}\",))?\n        .into_iter()\n        .next()\n    {\n        Some(Item::Pkcs8Key(key)) => PrivateKeyDer::Pkcs8(key),\n        Some(Item::Pkcs1Key(key)) => PrivateKeyDer::Pkcs1(key),\n        Some(Item::Sec1Key(key)) => PrivateKeyDer::Sec1(key),\n        Some(_) => return Err(\"Unsupported private keys found.\".to_string()),\n        None => return Err(\"No private keys found.\".to_string()),\n    };\n\n    Ok(CertifiedKey {\n        cert,\n        key: any_supported_type(&pk)\n            .map_err(|err| format!(\"Failed to sign certificate: {err}\",))?,\n        ocsp: None,\n    })\n}\n\npub(crate) fn build_self_signed_cert(\n    domains: impl Into<Vec<String>>,\n) -> Result<CertifiedKey, String> {\n    let cert = generate_simple_self_signed(domains)\n        .map_err(|err| format!(\"Failed to generate self-signed certificate: {err}\",))?;\n    build_certified_key(\n        cert.serialize_pem().unwrap().into_bytes(),\n        cert.serialize_private_key_pem().into_bytes(),\n    )\n}\n"
  },
  {
    "path": "crates/common/src/config/smtp/auth.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{sync::Arc, time::Duration};\n\nuse ahash::AHashMap;\nuse mail_auth::{\n    common::crypto::{Algorithm, Ed25519Key, HashAlgorithm, RsaKey, Sha256, SigningKey},\n    dkim::{Canonicalization, Done},\n};\nuse mail_parser::decoders::base64::base64_decode;\nuse utils::config::{\n    Config,\n    utils::{AsKey, ParseValue},\n};\n\nuse crate::{\n    config::CONNECTION_VARS,\n    expr::{self, Constant, ConstantValue, if_block::IfBlock, tokenizer::TokenMap},\n};\n\nuse super::*;\n\n#[derive(Clone)]\npub struct MailAuthConfig {\n    pub dkim: DkimAuthConfig,\n    pub arc: ArcAuthConfig,\n    pub spf: SpfAuthConfig,\n    pub dmarc: DmarcAuthConfig,\n    pub iprev: IpRevAuthConfig,\n    pub signatures: AHashMap<String, Arc<ArcSwap<LazySignature>>>,\n}\n\n#[allow(clippy::large_enum_variant)]\npub enum LazySignature {\n    Resolved(ResolvedSignature),\n    Pending(Config),\n    Failed,\n}\n\n#[derive(Clone)]\npub struct ResolvedSignature {\n    pub signer: Arc<DkimSigner>,\n    pub sealer: Arc<ArcSealer>,\n}\n\n#[derive(Clone)]\npub struct DkimAuthConfig {\n    pub verify: IfBlock,\n    pub sign: IfBlock,\n    pub strict: bool,\n}\n\n#[derive(Clone)]\npub struct ArcAuthConfig {\n    pub verify: IfBlock,\n    pub seal: IfBlock,\n}\n\n#[derive(Clone)]\npub struct SpfAuthConfig {\n    pub verify_ehlo: IfBlock,\n    pub verify_mail_from: IfBlock,\n}\n\n#[derive(Clone)]\npub struct DmarcAuthConfig {\n    pub verify: IfBlock,\n}\n\n#[derive(Clone)]\npub struct IpRevAuthConfig {\n    pub verify: IfBlock,\n}\n\n#[derive(Debug, Clone, Copy, Default)]\npub enum VerifyStrategy {\n    #[default]\n    Relaxed,\n    Strict,\n    Disable,\n}\n\n#[derive(Debug, Clone)]\npub struct DkimCanonicalization {\n    pub headers: Canonicalization,\n    pub body: Canonicalization,\n}\n\npub enum DkimSigner {\n    RsaSha256(mail_auth::dkim::DkimSigner<RsaKey<Sha256>, Done>),\n    Ed25519Sha256(mail_auth::dkim::DkimSigner<Ed25519Key, Done>),\n}\n\npub enum ArcSealer {\n    RsaSha256(mail_auth::arc::ArcSealer<RsaKey<Sha256>, Done>),\n    Ed25519Sha256(mail_auth::arc::ArcSealer<Ed25519Key, Done>),\n}\n\nimpl Default for MailAuthConfig {\n    fn default() -> Self {\n        Self {\n            dkim: DkimAuthConfig {\n                verify: IfBlock::new::<VerifyStrategy>(\"auth.dkim.verify\", [], \"relaxed\"),\n                sign: IfBlock::new::<()>(\n                    \"auth.dkim.sign\",\n                    [(\n                        \"is_local_domain('*', sender_domain)\",\n                        \"['rsa-' + sender_domain, 'ed25519-' + sender_domain]\",\n                    )],\n                    \"false\",\n                ),\n                strict: true,\n            },\n            arc: ArcAuthConfig {\n                verify: IfBlock::new::<VerifyStrategy>(\"auth.arc.verify\", [], \"relaxed\"),\n                seal: IfBlock::new::<()>(\n                    \"auth.arc.seal\",\n                    [],\n                    \"'rsa-' + config_get('report.domain')\",\n                ),\n            },\n            spf: SpfAuthConfig {\n                verify_ehlo: IfBlock::new::<VerifyStrategy>(\n                    \"auth.spf.verify.ehlo\",\n                    [(\"local_port == 25\", \"relaxed\")],\n                    #[cfg(not(feature = \"test_mode\"))]\n                    \"disable\",\n                    #[cfg(feature = \"test_mode\")]\n                    \"relaxed\",\n                ),\n                verify_mail_from: IfBlock::new::<VerifyStrategy>(\n                    \"auth.spf.verify.mail-from\",\n                    [(\"local_port == 25\", \"relaxed\")],\n                    #[cfg(not(feature = \"test_mode\"))]\n                    \"disable\",\n                    #[cfg(feature = \"test_mode\")]\n                    \"relaxed\",\n                ),\n            },\n            dmarc: DmarcAuthConfig {\n                verify: IfBlock::new::<VerifyStrategy>(\n                    \"auth.dmarc.verify\",\n                    [(\"local_port == 25\", \"relaxed\")],\n                    #[cfg(not(feature = \"test_mode\"))]\n                    \"disable\",\n                    #[cfg(feature = \"test_mode\")]\n                    \"relaxed\",\n                ),\n            },\n            iprev: IpRevAuthConfig {\n                verify: IfBlock::new::<VerifyStrategy>(\n                    \"auth.iprev.verify\",\n                    [(\"local_port == 25\", \"relaxed\")],\n                    #[cfg(not(feature = \"test_mode\"))]\n                    \"disable\",\n                    #[cfg(feature = \"test_mode\")]\n                    \"relaxed\",\n                ),\n            },\n            signatures: Default::default(),\n        }\n    }\n}\n\nimpl MailAuthConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        let rcpt_vars = TokenMap::default()\n            .with_variables(SMTP_RCPT_TO_VARS)\n            .with_constants::<VerifyStrategy>();\n        let conn_vars = TokenMap::default()\n            .with_variables(CONNECTION_VARS)\n            .with_constants::<VerifyStrategy>();\n        let mut mail_auth = Self::default();\n\n        for (value, key, token_map) in [\n            (&mut mail_auth.dkim.verify, \"auth.dkim.verify\", &rcpt_vars),\n            (&mut mail_auth.dkim.sign, \"auth.dkim.sign\", &rcpt_vars),\n            (&mut mail_auth.arc.verify, \"auth.arc.verify\", &rcpt_vars),\n            (&mut mail_auth.arc.seal, \"auth.arc.seal\", &rcpt_vars),\n            (\n                &mut mail_auth.spf.verify_ehlo,\n                \"auth.spf.verify.ehlo\",\n                &conn_vars,\n            ),\n            (\n                &mut mail_auth.spf.verify_mail_from,\n                \"auth.spf.verify.mail-from\",\n                &conn_vars,\n            ),\n            (&mut mail_auth.dmarc.verify, \"auth.dmarc.verify\", &rcpt_vars),\n            (&mut mail_auth.iprev.verify, \"auth.iprev.verify\", &conn_vars),\n        ] {\n            if let Some(if_block) = IfBlock::try_parse(config, key, token_map) {\n                *value = if_block;\n            }\n        }\n        mail_auth.dkim.strict = config\n            .property_or_default(\"auth.dkim.strict\", \"true\")\n            .unwrap_or(true);\n\n        // Parse signatures\n        let mut signatures: AHashMap<&str, Config> = AHashMap::new();\n        let mut current_id = None;\n        for (k, v) in config.keys.iter() {\n            if let Some(prefix) = k.strip_prefix(\"signature.\") {\n                if let Some(id) = prefix.strip_suffix(\".algorithm\") {\n                    current_id = Some(id);\n                }\n                #[allow(clippy::unwrap_or_default)]\n                if let Some(current_id) = current_id {\n                    signatures\n                        .entry(current_id)\n                        .or_insert_with(Config::default)\n                        .keys\n                        .insert(k.to_string(), v.to_string());\n                }\n            } else if !signatures.is_empty() {\n                break;\n            }\n        }\n        mail_auth.signatures = signatures\n            .into_iter()\n            .map(|(id, config)| {\n                (\n                    id.to_string(),\n                    Arc::new(ArcSwap::from_pointee(LazySignature::Pending(config))),\n                )\n            })\n            .collect();\n\n        mail_auth\n    }\n}\n\npub fn build_signature(config: &mut Config, id: &str) -> Option<(DkimSigner, ArcSealer)> {\n    match config.property_require::<Algorithm>((\"signature\", id, \"algorithm\"))? {\n        Algorithm::RsaSha256 => {\n            let pk = config\n                .value_require((\"signature\", id, \"private-key\"))?\n                .trim()\n                .to_string();\n            let key = RsaKey::<Sha256>::from_rsa_pem(&pk)\n                .or_else(|_| RsaKey::<Sha256>::from_pkcs8_pem(&pk))\n                .map_err(|err| {\n                    config.new_build_error(\n                        (\"signature\", id, \"private-key\"),\n                        format!(\"Failed to build RSA key: {err}\",),\n                    )\n                })\n                .ok()?;\n            let key_clone = RsaKey::<Sha256>::from_rsa_pem(&pk)\n                .or_else(|_| RsaKey::<Sha256>::from_pkcs8_pem(&pk))\n                .map_err(|err| {\n                    config.new_build_error(\n                        (\"signature\", id, \"private-key\"),\n                        format!(\"Failed to build RSA key: {err}\",),\n                    )\n                })\n                .ok()?;\n            let (signer, sealer) = parse_signature(config, id, key_clone, key)?;\n            (DkimSigner::RsaSha256(signer), ArcSealer::RsaSha256(sealer)).into()\n        }\n        Algorithm::Ed25519Sha256 => {\n            let private_key = parse_pem(config, (\"signature\", id, \"private-key\"))?;\n            let key = Ed25519Key::from_pkcs8_maybe_unchecked_der(&private_key)\n                .map_err(|err| {\n                    config.new_build_error(\n                        (\"signature\", id),\n                        format!(\"Failed to build ED25519 key for signature {id:?}: {err}\"),\n                    )\n                })\n                .ok()?;\n            let key_clone = Ed25519Key::from_pkcs8_maybe_unchecked_der(&private_key)\n                .map_err(|err| {\n                    config.new_build_error(\n                        (\"signature\", id),\n                        format!(\"Failed to build ED25519 key for signature {id:?}: {err}\"),\n                    )\n                })\n                .ok()?;\n\n            let (signer, sealer) = parse_signature(config, id, key_clone, key)?;\n            (\n                DkimSigner::Ed25519Sha256(signer),\n                ArcSealer::Ed25519Sha256(sealer),\n            )\n                .into()\n        }\n        Algorithm::RsaSha1 => {\n            config.new_build_error(\n                (\"signature\", id),\n                format!(\"Could not build signature {id:?}: SHA1 signatures are deprecated.\",),\n            );\n            None\n        }\n    }\n}\n\nfn parse_pem(config: &mut Config, key: impl AsKey) -> Option<Vec<u8>> {\n    if let Some(der) = simple_pem_parse(config.value_require(key.clone())?) {\n        Some(der)\n    } else {\n        config.new_build_error(key, \"Failed to base64 decode key.\");\n        None\n    }\n}\n\npub fn simple_pem_parse(contents: &str) -> Option<Vec<u8>> {\n    let mut contents = contents.as_bytes().iter().copied();\n    let mut base64 = vec![];\n\n    'outer: while let Some(ch) = contents.next() {\n        if !ch.is_ascii_whitespace() {\n            if ch == b'-' {\n                for ch in contents.by_ref() {\n                    if ch == b'\\n' {\n                        break;\n                    }\n                }\n            } else {\n                base64.push(ch);\n            }\n\n            for ch in contents.by_ref() {\n                if ch == b'-' {\n                    break 'outer;\n                } else if !ch.is_ascii_whitespace() {\n                    base64.push(ch);\n                }\n            }\n        }\n    }\n\n    base64_decode(&base64)\n}\n\nfn parse_signature<T: SigningKey, U: SigningKey<Hasher = Sha256>>(\n    config: &mut Config,\n    id: &str,\n    key_dkim: T,\n    key_arc: U,\n) -> Option<(\n    mail_auth::dkim::DkimSigner<T, Done>,\n    mail_auth::arc::ArcSealer<U, Done>,\n)> {\n    let domain = config\n        .value_require((\"signature\", id, \"domain\"))?\n        .to_string();\n    let selector = config\n        .value_require((\"signature\", id, \"selector\"))?\n        .to_string();\n    let mut headers = config\n        .values((\"signature\", id, \"headers\"))\n        .filter_map(|(_, v)| {\n            if !v.is_empty() {\n                v.to_string().into()\n            } else {\n                None\n            }\n        })\n        .collect::<Vec<_>>();\n    if headers.is_empty() {\n        headers = vec![\n            \"From\".to_string(),\n            \"To\".to_string(),\n            \"Date\".to_string(),\n            \"Subject\".to_string(),\n            \"Message-ID\".to_string(),\n        ];\n    }\n\n    let mut signer = mail_auth::dkim::DkimSigner::from_key(key_dkim)\n        .domain(&domain)\n        .selector(&selector)\n        .headers(headers.clone());\n    if !headers\n        .iter()\n        .any(|h| h.eq_ignore_ascii_case(\"DKIM-Signature\"))\n    {\n        headers.push(\"DKIM-Signature\".to_string());\n    }\n    let mut sealer = mail_auth::arc::ArcSealer::from_key(key_arc)\n        .domain(domain)\n        .selector(selector)\n        .headers(headers);\n\n    if let Some(c) = config.property::<DkimCanonicalization>((\"signature\", id, \"canonicalization\"))\n    {\n        signer = signer\n            .body_canonicalization(c.body)\n            .header_canonicalization(c.headers);\n        sealer = sealer\n            .body_canonicalization(c.body)\n            .header_canonicalization(c.headers);\n    }\n\n    if let Some(c) = config.property::<Duration>((\"signature\", id, \"expire\")) {\n        signer = signer.expiration(c.as_secs());\n        sealer = sealer.expiration(c.as_secs());\n    }\n\n    if let Some(true) = config.property::<bool>((\"signature\", id, \"report\")) {\n        signer = signer.reporting(true);\n    }\n\n    if let Some(auid) = config.property::<String>((\"signature\", id, \"auid\")) {\n        signer = signer.agent_user_identifier(auid);\n    }\n\n    if let Some(atps) = config.property::<String>((\"signature\", id, \"third-party\")) {\n        signer = signer.atps(atps);\n    }\n\n    if let Some(atpsh) = config.property::<HashAlgorithm>((\"signature\", id, \"third-party-algo\")) {\n        signer = signer.atpsh(atpsh);\n    }\n\n    Some((signer, sealer))\n}\n\nimpl<'x> TryFrom<expr::Variable<'x>> for VerifyStrategy {\n    type Error = ();\n\n    fn try_from(value: expr::Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            expr::Variable::Integer(c) => match c {\n                2 => Ok(VerifyStrategy::Relaxed),\n                3 => Ok(VerifyStrategy::Strict),\n                4 => Ok(VerifyStrategy::Disable),\n                _ => Err(()),\n            },\n            _ => Err(()),\n        }\n    }\n}\n\nimpl From<VerifyStrategy> for Constant {\n    fn from(value: VerifyStrategy) -> Self {\n        Constant::Integer(match value {\n            VerifyStrategy::Relaxed => 2,\n            VerifyStrategy::Strict => 3,\n            VerifyStrategy::Disable => 4,\n        })\n    }\n}\n\nimpl VerifyStrategy {\n    #[inline(always)]\n    pub fn verify(&self) -> bool {\n        matches!(self, VerifyStrategy::Strict | VerifyStrategy::Relaxed)\n    }\n\n    #[inline(always)]\n    pub fn is_strict(&self) -> bool {\n        matches!(self, VerifyStrategy::Strict)\n    }\n}\n\nimpl ParseValue for VerifyStrategy {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        match value {\n            \"relaxed\" => Ok(VerifyStrategy::Relaxed),\n            \"strict\" => Ok(VerifyStrategy::Strict),\n            \"disable\" | \"disabled\" | \"never\" | \"none\" => Ok(VerifyStrategy::Disable),\n            _ => Err(format!(\"Invalid value {:?}.\", value)),\n        }\n    }\n}\n\nimpl ConstantValue for VerifyStrategy {\n    fn add_constants(token_map: &mut TokenMap) {\n        token_map\n            .add_constant(\"relaxed\", VerifyStrategy::Relaxed)\n            .add_constant(\"strict\", VerifyStrategy::Strict)\n            .add_constant(\"disable\", VerifyStrategy::Disable)\n            .add_constant(\"disabled\", VerifyStrategy::Disable)\n            .add_constant(\"never\", VerifyStrategy::Disable)\n            .add_constant(\"none\", VerifyStrategy::Disable);\n    }\n}\n\nimpl ParseValue for DkimCanonicalization {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        if let Some((headers, body)) = value.split_once('/') {\n            Ok(DkimCanonicalization {\n                headers: Canonicalization::parse_value(headers.trim())?,\n                body: Canonicalization::parse_value(body.trim())?,\n            })\n        } else {\n            let c = Canonicalization::parse_value(value)?;\n            Ok(DkimCanonicalization {\n                headers: c,\n                body: c,\n            })\n        }\n    }\n}\n\nimpl Default for DkimCanonicalization {\n    fn default() -> Self {\n        Self {\n            headers: Canonicalization::Relaxed,\n            body: Canonicalization::Relaxed,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/smtp/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse utils::config::{Config, Rate};\n\npub mod auth;\npub mod queue;\npub mod report;\npub mod resolver;\npub mod session;\npub mod throttle;\n\nuse crate::expr::{Expression, tokenizer::TokenMap};\n\nuse self::{\n    auth::MailAuthConfig, queue::QueueConfig, report::ReportConfig, resolver::Resolvers,\n    session::SessionConfig,\n};\n\nuse super::*;\n\n#[derive(Default, Clone)]\npub struct SmtpConfig {\n    pub session: SessionConfig,\n    pub queue: QueueConfig,\n    pub resolvers: Resolvers,\n    pub mail_auth: MailAuthConfig,\n    pub report: ReportConfig,\n}\n\n#[derive(Debug, Default, Clone)]\n#[cfg_attr(feature = \"test_mode\", derive(PartialEq, Eq))]\npub struct QueueRateLimiter {\n    pub id: String,\n    pub expr: Expression,\n    pub keys: u16,\n    pub rate: Rate,\n}\n\npub const THROTTLE_RCPT: u16 = 1 << 0;\npub const THROTTLE_RCPT_DOMAIN: u16 = 1 << 1;\npub const THROTTLE_SENDER: u16 = 1 << 2;\npub const THROTTLE_SENDER_DOMAIN: u16 = 1 << 3;\npub const THROTTLE_AUTH_AS: u16 = 1 << 4;\npub const THROTTLE_LISTENER: u16 = 1 << 5;\npub const THROTTLE_MX: u16 = 1 << 6;\npub const THROTTLE_REMOTE_IP: u16 = 1 << 7;\npub const THROTTLE_LOCAL_IP: u16 = 1 << 8;\npub const THROTTLE_HELO_DOMAIN: u16 = 1 << 9;\n\npub(crate) const RCPT_DOMAIN_VARS: &[u32; 1] = &[V_RECIPIENT_DOMAIN];\n\npub(crate) const SMTP_EHLO_VARS: &[u32; 10] = &[\n    V_LISTENER,\n    V_REMOTE_IP,\n    V_REMOTE_PORT,\n    V_LOCAL_IP,\n    V_LOCAL_PORT,\n    V_PROTOCOL,\n    V_TLS,\n    V_HELO_DOMAIN,\n    V_ASN,\n    V_COUNTRY,\n];\npub(crate) const SMTP_MAIL_FROM_VARS: &[u32; 12] = &[\n    V_LISTENER,\n    V_REMOTE_IP,\n    V_REMOTE_PORT,\n    V_LOCAL_IP,\n    V_LOCAL_PORT,\n    V_PROTOCOL,\n    V_TLS,\n    V_SENDER,\n    V_SENDER_DOMAIN,\n    V_AUTHENTICATED_AS,\n    V_ASN,\n    V_COUNTRY,\n];\npub(crate) const SMTP_RCPT_TO_VARS: &[u32; 17] = &[\n    V_SENDER,\n    V_SENDER_DOMAIN,\n    V_RECIPIENTS,\n    V_RECIPIENT,\n    V_RECIPIENT_DOMAIN,\n    V_AUTHENTICATED_AS,\n    V_LISTENER,\n    V_REMOTE_IP,\n    V_REMOTE_PORT,\n    V_LOCAL_IP,\n    V_LOCAL_PORT,\n    V_PROTOCOL,\n    V_TLS,\n    V_PRIORITY,\n    V_HELO_DOMAIN,\n    V_ASN,\n    V_COUNTRY,\n];\npub(crate) const SMTP_QUEUE_HOST_VARS: &[u32; 20] = &[\n    V_SENDER,\n    V_SENDER_DOMAIN,\n    V_RECIPIENT_DOMAIN,\n    V_RECIPIENT,\n    V_RECIPIENTS,\n    V_MX,\n    V_PRIORITY,\n    V_REMOTE_IP,\n    V_LOCAL_IP,\n    V_QUEUE_RETRY_NUM,\n    V_QUEUE_NOTIFY_NUM,\n    V_QUEUE_EXPIRES_IN,\n    V_QUEUE_LAST_STATUS,\n    V_QUEUE_LAST_ERROR,\n    V_QUEUE_NAME,\n    V_QUEUE_AGE,\n    V_RECEIVED_FROM_IP,\n    V_RECEIVED_VIA_PORT,\n    V_SOURCE,\n    V_SIZE,\n];\npub(crate) const SMTP_QUEUE_RCPT_VARS: &[u32; 17] = &[\n    V_RECIPIENT,\n    V_RECIPIENT_DOMAIN,\n    V_RECIPIENTS,\n    V_SENDER,\n    V_SENDER_DOMAIN,\n    V_PRIORITY,\n    V_QUEUE_RETRY_NUM,\n    V_QUEUE_NOTIFY_NUM,\n    V_QUEUE_EXPIRES_IN,\n    V_QUEUE_LAST_STATUS,\n    V_QUEUE_LAST_ERROR,\n    V_QUEUE_NAME,\n    V_QUEUE_AGE,\n    V_RECEIVED_FROM_IP,\n    V_RECEIVED_VIA_PORT,\n    V_SOURCE,\n    V_SIZE,\n];\npub(crate) const SMTP_QUEUE_SENDER_VARS: &[u32; 8] = &[\n    V_SENDER,\n    V_SENDER_DOMAIN,\n    V_PRIORITY,\n    V_QUEUE_RETRY_NUM,\n    V_QUEUE_NOTIFY_NUM,\n    V_QUEUE_EXPIRES_IN,\n    V_QUEUE_LAST_STATUS,\n    V_QUEUE_LAST_ERROR,\n];\n\nimpl SmtpConfig {\n    pub async fn parse(config: &mut Config) -> Self {\n        Self {\n            session: SessionConfig::parse(config),\n            queue: QueueConfig::parse(config),\n            resolvers: Resolvers::parse(config).await,\n            mail_auth: MailAuthConfig::parse(config),\n            report: ReportConfig::parse(config),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/smtp/queue.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse self::throttle::parse_queue_rate_limiter;\nuse super::*;\nuse crate::{\n    config::server::ServerProtocol,\n    expr::{if_block::IfBlock, *},\n};\nuse ahash::AHashMap;\nuse mail_auth::IpLookupStrategy;\nuse mail_send::Credentials;\nuse std::{\n    fmt::Display,\n    hash::{Hash, Hasher},\n    net::IpAddr,\n    time::Duration,\n};\nuse throttle::parse_queue_rate_limiter_key;\nuse utils::config::{Config, utils::ParseValue};\n\n#[derive(\n    Debug,\n    Clone,\n    Copy,\n    PartialEq,\n    Eq,\n    Hash,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\n#[rkyv(derive(Debug, Clone, Copy, PartialEq), compare(PartialEq))]\n#[repr(transparent)]\npub struct QueueName([u8; 8]);\n\npub const DEFAULT_QUEUE_NAME: QueueName = QueueName([b'd', b'e', b'f', b'a', b'u', b'l', b't', 0]);\n\n#[derive(Clone)]\npub struct QueueConfig {\n    // Strategy resolver\n    pub route: IfBlock,\n    pub queue: IfBlock,\n    pub connection: IfBlock,\n    pub tls: IfBlock,\n\n    // DSN\n    pub dsn: Dsn,\n\n    // Rate limits\n    pub inbound_limiters: QueueRateLimiters,\n    pub outbound_limiters: QueueRateLimiters,\n    pub quota: QueueQuotas,\n\n    // Strategies\n    pub queue_strategy: AHashMap<String, QueueStrategy>,\n    pub connection_strategy: AHashMap<String, ConnectionStrategy>,\n    pub routing_strategy: AHashMap<String, RoutingStrategy>,\n    pub tls_strategy: AHashMap<String, TlsStrategy>,\n    pub virtual_queues: AHashMap<QueueName, VirtualQueue>,\n}\n\n#[derive(Clone, Hash, PartialEq, Eq, Debug)]\npub enum RoutingStrategy {\n    Local,\n    Mx(MxConfig),\n    Relay(RelayConfig),\n}\n\n#[derive(Clone, Debug)]\npub struct MxConfig {\n    pub max_mx: usize,\n    pub max_multi_homed: usize,\n    pub ip_lookup_strategy: IpLookupStrategy,\n}\n\n#[derive(Clone)]\npub struct Dsn {\n    pub name: IfBlock,\n    pub address: IfBlock,\n    pub sign: IfBlock,\n}\n\n#[derive(Clone, Debug)]\npub struct VirtualQueue {\n    pub threads: usize,\n}\n\n#[derive(Clone, Debug)]\npub struct QueueStrategy {\n    pub retry: Vec<u64>,\n    pub notify: Vec<u64>,\n    pub expiry: QueueExpiry,\n    pub virtual_queue: QueueName,\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    Copy,\n    PartialEq,\n    Eq,\n    serde::Deserialize,\n)]\npub enum QueueExpiry {\n    Ttl(u64),\n    Attempts(u32),\n}\n\n#[derive(Clone, Debug)]\npub struct TlsStrategy {\n    pub dane: RequireOptional,\n    pub mta_sts: RequireOptional,\n    pub tls: RequireOptional,\n    pub allow_invalid_certs: bool,\n\n    pub timeout_tls: Duration,\n    pub timeout_mta_sts: Duration,\n}\n\n#[derive(Clone, Debug)]\npub struct ConnectionStrategy {\n    pub source_ipv4: Vec<IpAndHost>,\n    pub source_ipv6: Vec<IpAndHost>,\n    pub ehlo_hostname: Option<String>,\n\n    pub timeout_connect: Duration,\n    pub timeout_greeting: Duration,\n    pub timeout_ehlo: Duration,\n    pub timeout_mail: Duration,\n    pub timeout_rcpt: Duration,\n    pub timeout_data: Duration,\n}\n\n#[derive(Clone, Debug)]\npub struct IpAndHost {\n    pub ip: IpAddr,\n    pub host: Option<String>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct QueueRateLimiters {\n    pub sender: Vec<QueueRateLimiter>,\n    pub rcpt: Vec<QueueRateLimiter>,\n    pub remote: Vec<QueueRateLimiter>,\n}\n\n#[derive(Clone, Default)]\npub struct QueueQuotas {\n    pub sender: Vec<QueueQuota>,\n    pub rcpt: Vec<QueueQuota>,\n    pub rcpt_domain: Vec<QueueQuota>,\n}\n\n#[derive(Clone)]\npub struct QueueQuota {\n    pub id: String,\n    pub expr: Expression,\n    pub keys: u16,\n    pub size: Option<u64>,\n    pub messages: Option<u64>,\n}\n\n#[derive(Clone, Hash, PartialEq, Eq)]\npub struct RelayConfig {\n    pub address: String,\n    pub port: u16,\n    pub protocol: ServerProtocol,\n    pub auth: Option<Credentials<String>>,\n    pub tls_implicit: bool,\n    pub tls_allow_invalid_certs: bool,\n}\n\n#[derive(Debug, Clone, Copy, Default)]\npub enum RequireOptional {\n    #[default]\n    Optional,\n    Require,\n    Disable,\n}\n\nimpl Default for QueueConfig {\n    fn default() -> Self {\n        Self {\n            route: IfBlock::new::<()>(\n                \"queue.strategy.route\",\n                #[cfg(not(feature = \"test_mode\"))]\n                [(\"is_local_domain('*', rcpt_domain)\", \"'local'\")],\n                #[cfg(feature = \"test_mode\")]\n                [],\n                \"'mx'\",\n            ),\n            queue: IfBlock::new::<()>(\n                \"queue.strategy.schedule\",\n                #[cfg(not(feature = \"test_mode\"))]\n                [\n                    (\"is_local_domain('*', rcpt_domain)\", \"'local'\"),\n                    (\"source == 'dsn'\", \"'dsn'\"),\n                    (\"source == 'report'\", \"'report'\"),\n                ],\n                #[cfg(feature = \"test_mode\")]\n                [],\n                #[cfg(not(feature = \"test_mode\"))]\n                \"'remote'\",\n                #[cfg(feature = \"test_mode\")]\n                \"'default'\",\n            ),\n            connection: IfBlock::new::<()>(\"queue.strategy.connection\", [], \"'default'\"),\n            tls: IfBlock::new::<()>(\n                \"queue.strategy.tls\",\n                #[cfg(not(feature = \"test_mode\"))]\n                [(\"retry_num > 0 && last_error == 'tls'\", \"'invalid-tls'\")],\n                #[cfg(feature = \"test_mode\")]\n                [],\n                \"'default'\",\n            ),\n            dsn: Dsn {\n                name: IfBlock::new::<()>(\"report.dsn.from-name\", [], \"'Mail Delivery Subsystem'\"),\n                address: IfBlock::new::<()>(\n                    \"report.dsn.from-address\",\n                    [],\n                    \"'MAILER-DAEMON@' + config_get('report.domain')\",\n                ),\n                sign: IfBlock::new::<()>(\n                    \"report.dsn.sign\",\n                    [],\n                    \"['rsa-' + config_get('report.domain'), 'ed25519-' + config_get('report.domain')]\",\n                ),\n            },\n            inbound_limiters: QueueRateLimiters::default(),\n            outbound_limiters: QueueRateLimiters::default(),\n            quota: QueueQuotas::default(),\n            queue_strategy: Default::default(),\n            virtual_queues: Default::default(),\n            connection_strategy: Default::default(),\n            routing_strategy: Default::default(),\n            tls_strategy: Default::default(),\n        }\n    }\n}\n\nimpl QueueConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut queue = QueueConfig::default();\n        let rcpt_vars = TokenMap::default().with_variables(SMTP_QUEUE_RCPT_VARS);\n        let sender_vars = TokenMap::default().with_variables(SMTP_QUEUE_SENDER_VARS);\n        let host_vars = TokenMap::default().with_variables(SMTP_QUEUE_HOST_VARS);\n\n        for (value, key, token_map) in [\n            (&mut queue.route, \"queue.strategy.route\", &rcpt_vars),\n            (&mut queue.queue, \"queue.strategy.schedule\", &rcpt_vars),\n            (\n                &mut queue.connection,\n                \"queue.strategy.connection\",\n                &host_vars,\n            ),\n            (&mut queue.tls, \"queue.strategy.tls\", &host_vars),\n            (&mut queue.dsn.name, \"report.dsn.from-name\", &sender_vars),\n            (\n                &mut queue.dsn.address,\n                \"report.dsn.from-address\",\n                &sender_vars,\n            ),\n            (&mut queue.dsn.sign, \"report.dsn.sign\", &sender_vars),\n        ] {\n            if let Some(if_block) = IfBlock::try_parse(config, key, token_map) {\n                *value = if_block;\n            }\n        }\n\n        // Parse strategies\n        queue.virtual_queues = parse_virtual_queues(config);\n        queue.queue_strategy = parse_queue_strategies(config, &queue.virtual_queues);\n        queue.connection_strategy = parse_connection_strategies(config);\n        queue.routing_strategy = parse_routing_strategies(config);\n        queue.tls_strategy = parse_tls_strategies(config);\n\n        // Parse rate limiters\n        queue.inbound_limiters = parse_inbound_rate_limiters(config);\n        queue.outbound_limiters = parse_outbound_rate_limiters(config);\n        queue.quota = parse_queue_quota(config);\n        queue\n    }\n}\n\nfn parse_queue_strategies(\n    config: &mut Config,\n    queues: &AHashMap<QueueName, VirtualQueue>,\n) -> AHashMap<String, QueueStrategy> {\n    let mut entries = AHashMap::new();\n    for key in config.sub_keys_with_suffixes(\n        \"queue.schedule\",\n        &[\n            \".queue-name\",\n            \".retry\",\n            \".notify\",\n            \".expire\",\n            \".max-attempts\",\n        ],\n    ) {\n        if let Some(strategy) = parse_queue_strategy(config, &key, queues) {\n            entries.insert(key, strategy);\n        }\n    }\n    entries\n}\n\nfn parse_queue_strategy(\n    config: &mut Config,\n    id: &str,\n    queues: &AHashMap<QueueName, VirtualQueue>,\n) -> Option<QueueStrategy> {\n    let virtual_queue = config\n        .property_require::<QueueName>((\"queue.schedule\", id, \"queue-name\"))\n        .unwrap_or_default();\n    if virtual_queue != DEFAULT_QUEUE_NAME && !queues.contains_key(&virtual_queue) {\n        config.new_parse_error(\n            (\"queue.schedule\", id, \"queue-name\"),\n            format!(\"Virtual queue '{virtual_queue}' does not exist.\"),\n        );\n        return None;\n    }\n    let mut retry: Vec<u64> = config\n        .properties::<Duration>((\"queue.schedule\", id, \"retry\"))\n        .into_iter()\n        .map(|(_, d)| d.as_secs())\n        .collect();\n    let mut notify: Vec<u64> = config\n        .properties::<Duration>((\"queue.schedule\", id, \"notify\"))\n        .into_iter()\n        .map(|(_, d)| d.as_secs())\n        .collect();\n    if retry.is_empty() {\n        config.new_parse_error(\n            (\"queue.schedule\", id, \"retry\"),\n            \"At least one 'retry' duration must be specified.\".to_string(),\n        );\n        retry.push(60 * 60); // Default to 1 minute\n    }\n    if notify.is_empty() {\n        notify.push(10000 * 86400); // Disable notifications by default\n    }\n\n    Some(QueueStrategy {\n        retry,\n        notify,\n        expiry: match (\n            config.property::<Duration>((\"queue.schedule\", id, \"expire\")),\n            config.property::<u32>((\"queue.schedule\", id, \"max-attempts\")),\n        ) {\n            (Some(duration), None) => QueueExpiry::Ttl(duration.as_secs()),\n            (None, Some(count)) => QueueExpiry::Attempts(count),\n            (Some(_), Some(_)) => {\n                config.new_parse_error(\n                    (\"queue.schedule\", id, \"expire\"),\n                    \"Cannot specify both 'expire' and 'max-attempts'.\".to_string(),\n                );\n                return None;\n            }\n            (None, None) => QueueExpiry::Ttl(60 * 60 * 24 * 3), // Default to 3 days\n        },\n        virtual_queue,\n    })\n}\n\nfn parse_virtual_queues(config: &mut Config) -> AHashMap<QueueName, VirtualQueue> {\n    let mut entries = AHashMap::new();\n    for key in config.sub_keys(\"queue.virtual\", \".threads-per-node\") {\n        if let Some(queue_name) = QueueName::new(&key) {\n            if let Some(queue) = parse_virtual_queue(config, &key) {\n                entries.insert(queue_name, queue);\n            }\n        } else {\n            config.new_parse_error(\n                (\"queue.virtual\", &key, \"threads-per-node\"),\n                format!(\"Invalid virtual queue name: {key:?}. Must be 1-8 bytes long.\"),\n            );\n        }\n    }\n    entries\n}\n\nfn parse_virtual_queue(config: &mut Config, id: &str) -> Option<VirtualQueue> {\n    Some(VirtualQueue {\n        threads: config\n            .property_require::<usize>((\"queue.virtual\", id, \"threads-per-node\"))\n            .unwrap_or(1),\n    })\n}\n\nfn parse_routing_strategies(config: &mut Config) -> AHashMap<String, RoutingStrategy> {\n    let mut entries = AHashMap::new();\n    for key in config.sub_keys(\"queue.route\", \".type\") {\n        if let Some(strategy) = parse_route(config, &key) {\n            entries.insert(key, strategy);\n        }\n    }\n    entries\n}\n\nfn parse_route(config: &mut Config, id: &str) -> Option<RoutingStrategy> {\n    match config.value_require_non_empty((\"queue.route\", id, \"type\"))? {\n        \"relay\" => RoutingStrategy::Relay(RelayConfig {\n            address: config.property_require((\"queue.route\", id, \"address\"))?,\n            port: config\n                .property_require((\"queue.route\", id, \"port\"))\n                .unwrap_or(25),\n            protocol: config\n                .property_require((\"queue.route\", id, \"protocol\"))\n                .unwrap_or(ServerProtocol::Smtp),\n            auth: if let (Some(username), Some(secret)) = (\n                config.value((\"queue.route\", id, \"auth.username\")),\n                config.value((\"queue.route\", id, \"auth.secret\")),\n            ) {\n                Credentials::new(username.to_string(), secret.to_string()).into()\n            } else {\n                None\n            },\n            tls_implicit: config\n                .property((\"queue.route\", id, \"tls.implicit\"))\n                .unwrap_or(true),\n            tls_allow_invalid_certs: config\n                .property((\"queue.route\", id, \"tls.allow-invalid-certs\"))\n                .unwrap_or(false),\n        })\n        .into(),\n        \"local\" => RoutingStrategy::Local.into(),\n        \"mx\" => RoutingStrategy::Mx(MxConfig {\n            max_mx: config\n                .property((\"queue.route\", id, \"limits.mx\"))\n                .unwrap_or(5),\n            max_multi_homed: config\n                .property((\"queue.route\", id, \"limits.multihomed\"))\n                .unwrap_or(2),\n            ip_lookup_strategy: config\n                .property((\"queue.route\", id, \"ip-lookup\"))\n                .unwrap_or(IpLookupStrategy::Ipv4thenIpv6),\n        })\n        .into(),\n        invalid => {\n            let details =\n                format!(\"Invalid route type: {invalid:?}. Expected 'relay', 'local', or 'mx'.\");\n            config.new_parse_error((\"queue.route\", id, \"type\"), details);\n            None\n        }\n    }\n}\n\nfn parse_tls_strategies(config: &mut Config) -> AHashMap<String, TlsStrategy> {\n    let mut entries = AHashMap::new();\n    for key in config.sub_keys_with_suffixes(\n        \"queue.tls\",\n        &[\n            \".allow-invalid-certs\",\n            \".dane\",\n            \".starttls\",\n            \".timeout.tls\",\n            \".timeout.mta-sts\",\n        ],\n    ) {\n        if let Some(strategy) = parse_tls(config, &key) {\n            entries.insert(key, strategy);\n        }\n    }\n    entries\n}\n\nfn parse_tls(config: &mut Config, id: &str) -> Option<TlsStrategy> {\n    Some(TlsStrategy {\n        dane: config\n            .property::<RequireOptional>((\"queue.tls\", id, \"dane\"))\n            .unwrap_or(RequireOptional::Optional),\n        mta_sts: config\n            .property::<RequireOptional>((\"queue.tls\", id, \"mta-sts\"))\n            .unwrap_or(RequireOptional::Optional),\n        tls: config\n            .property::<RequireOptional>((\"queue.tls\", id, \"starttls\"))\n            .unwrap_or(RequireOptional::Optional),\n        allow_invalid_certs: config\n            .property::<bool>((\"queue.tls\", id, \"allow-invalid-certs\"))\n            .unwrap_or(false),\n        timeout_tls: config\n            .property::<Duration>((\"queue.tls\", id, \"timeout.tls\"))\n            .unwrap_or(Duration::from_secs(3 * 60)),\n        timeout_mta_sts: config\n            .property::<Duration>((\"queue.tls\", id, \"timeout.mta-sts\"))\n            .unwrap_or(Duration::from_secs(5 * 60)),\n    })\n}\n\nfn parse_connection_strategies(config: &mut Config) -> AHashMap<String, ConnectionStrategy> {\n    let mut entries = AHashMap::new();\n    for key in config.sub_keys_with_suffixes(\n        \"queue.connection\",\n        &[\n            \".timeout.connect\",\n            \".timeout.greeting\",\n            \".timeout.ehlo\",\n            \".timeout.mail-from\",\n            \".timeout.rcpt-to\",\n            \".timeout.data\",\n            \".ehlo-hostname\",\n        ],\n    ) {\n        if let Some(strategy) = parse_connection(config, &key) {\n            entries.insert(key, strategy);\n        }\n    }\n    entries\n}\n\nfn parse_connection(config: &mut Config, id: &str) -> Option<ConnectionStrategy> {\n    let mut source_ipv4 = Vec::new();\n    let mut source_ipv6 = Vec::new();\n\n    for (_, ip) in config.properties::<IpAddr>((\"queue.connection\", id, \"source-ips\")) {\n        let ip_and_host = IpAndHost {\n            ip,\n            host: config.property::<String>((\"queue.source-ip\", ip.to_string(), \"ehlo-hostname\")),\n        };\n\n        if ip.is_ipv4() {\n            source_ipv4.push(ip_and_host);\n        } else {\n            source_ipv6.push(ip_and_host);\n        }\n    }\n\n    Some(ConnectionStrategy {\n        source_ipv4,\n        source_ipv6,\n        ehlo_hostname: config.property::<String>((\"queue.connection\", id, \"ehlo-hostname\")),\n        timeout_connect: config\n            .property::<Duration>((\"queue.connection\", id, \"timeout.connect\"))\n            .unwrap_or(Duration::from_secs(5 * 60)),\n        timeout_greeting: config\n            .property::<Duration>((\"queue.connection\", id, \"timeout.greeting\"))\n            .unwrap_or(Duration::from_secs(5 * 60)),\n        timeout_ehlo: config\n            .property::<Duration>((\"queue.connection\", id, \"timeout.ehlo\"))\n            .unwrap_or(Duration::from_secs(5 * 60)),\n        timeout_mail: config\n            .property::<Duration>((\"queue.connection\", id, \"timeout.mail-from\"))\n            .unwrap_or(Duration::from_secs(5 * 60)),\n        timeout_rcpt: config\n            .property::<Duration>((\"queue.connection\", id, \"timeout.rcpt-to\"))\n            .unwrap_or(Duration::from_secs(5 * 60)),\n        timeout_data: config\n            .property::<Duration>((\"queue.connection\", id, \"timeout.data\"))\n            .unwrap_or(Duration::from_secs(10 * 60)),\n    })\n}\n\nfn parse_inbound_rate_limiters(config: &mut Config) -> QueueRateLimiters {\n    let mut throttle = QueueRateLimiters::default();\n    let all_throttles = parse_queue_rate_limiter(\n        config,\n        \"queue.limiter.inbound\",\n        &TokenMap::default().with_variables(SMTP_RCPT_TO_VARS),\n        THROTTLE_LISTENER\n            | THROTTLE_REMOTE_IP\n            | THROTTLE_LOCAL_IP\n            | THROTTLE_AUTH_AS\n            | THROTTLE_HELO_DOMAIN\n            | THROTTLE_RCPT\n            | THROTTLE_RCPT_DOMAIN\n            | THROTTLE_SENDER\n            | THROTTLE_SENDER_DOMAIN,\n    );\n    for t in all_throttles {\n        if (t.keys & (THROTTLE_RCPT | THROTTLE_RCPT_DOMAIN)) != 0\n            || t.expr.items().iter().any(|c| {\n                matches!(\n                    c,\n                    ExpressionItem::Variable(V_RECIPIENT | V_RECIPIENT_DOMAIN)\n                )\n            })\n        {\n            throttle.rcpt.push(t);\n        } else if (t.keys\n            & (THROTTLE_SENDER | THROTTLE_SENDER_DOMAIN | THROTTLE_HELO_DOMAIN | THROTTLE_AUTH_AS))\n            != 0\n            || t.expr.items().iter().any(|c| {\n                matches!(\n                    c,\n                    ExpressionItem::Variable(\n                        V_SENDER | V_SENDER_DOMAIN | V_HELO_DOMAIN | V_AUTHENTICATED_AS\n                    )\n                )\n            })\n        {\n            throttle.sender.push(t);\n        } else {\n            throttle.remote.push(t);\n        }\n    }\n\n    throttle\n}\n\nfn parse_outbound_rate_limiters(config: &mut Config) -> QueueRateLimiters {\n    // Parse throttle\n    let mut throttle = QueueRateLimiters::default();\n\n    let all_throttles = parse_queue_rate_limiter(\n        config,\n        \"queue.limiter.outbound\",\n        &TokenMap::default().with_variables(SMTP_QUEUE_HOST_VARS),\n        THROTTLE_RCPT_DOMAIN\n            | THROTTLE_SENDER\n            | THROTTLE_SENDER_DOMAIN\n            | THROTTLE_MX\n            | THROTTLE_REMOTE_IP\n            | THROTTLE_LOCAL_IP,\n    );\n    for t in all_throttles {\n        if (t.keys & (THROTTLE_MX | THROTTLE_REMOTE_IP | THROTTLE_LOCAL_IP)) != 0\n            || t.expr\n                .items()\n                .iter()\n                .any(|c| matches!(c, ExpressionItem::Variable(V_MX | V_REMOTE_IP | V_LOCAL_IP)))\n        {\n            throttle.remote.push(t);\n        } else if (t.keys & (THROTTLE_RCPT_DOMAIN)) != 0\n            || t.expr\n                .items()\n                .iter()\n                .any(|c| matches!(c, ExpressionItem::Variable(V_RECIPIENT_DOMAIN)))\n        {\n            throttle.rcpt.push(t);\n        } else {\n            throttle.sender.push(t);\n        }\n    }\n\n    throttle\n}\n\nfn parse_queue_quota(config: &mut Config) -> QueueQuotas {\n    let mut capacities = QueueQuotas {\n        sender: Vec::new(),\n        rcpt: Vec::new(),\n        rcpt_domain: Vec::new(),\n    };\n\n    for quota_id in config.sub_keys(\"queue.quota\", \"\") {\n        if let Some(quota) = parse_queue_quota_item(config, (\"queue.quota\", &quota_id), &quota_id) {\n            if (quota.keys & THROTTLE_RCPT) != 0\n                || quota\n                    .expr\n                    .items()\n                    .iter()\n                    .any(|c| matches!(c, ExpressionItem::Variable(V_RECIPIENT)))\n            {\n                capacities.rcpt.push(quota);\n            } else if (quota.keys & THROTTLE_RCPT_DOMAIN) != 0\n                || quota\n                    .expr\n                    .items()\n                    .iter()\n                    .any(|c| matches!(c, ExpressionItem::Variable(V_RECIPIENT_DOMAIN)))\n            {\n                capacities.rcpt_domain.push(quota);\n            } else {\n                capacities.sender.push(quota);\n            }\n        }\n    }\n\n    capacities\n}\n\nfn parse_queue_quota_item(config: &mut Config, prefix: impl AsKey, id: &str) -> Option<QueueQuota> {\n    let prefix = prefix.as_key();\n\n    // Skip disabled throttles\n    if !config\n        .property::<bool>((prefix.as_str(), \"enable\"))\n        .unwrap_or(true)\n    {\n        return None;\n    }\n\n    let mut keys = 0;\n    for (key_, value) in config\n        .values((&prefix, \"key\"))\n        .map(|(k, v)| (k.to_string(), v.to_string()))\n        .collect::<Vec<_>>()\n    {\n        match parse_queue_rate_limiter_key(&value) {\n            Ok(key) => {\n                if (key\n                    & (THROTTLE_RCPT_DOMAIN\n                        | THROTTLE_RCPT\n                        | THROTTLE_SENDER\n                        | THROTTLE_SENDER_DOMAIN))\n                    != 0\n                {\n                    keys |= key;\n                } else {\n                    let err = format!(\"Quota key {value:?} is not available in this context\");\n                    config.new_build_error(key_, err);\n                }\n            }\n            Err(err) => {\n                config.new_parse_error(key_, err);\n            }\n        }\n    }\n\n    let quota = QueueQuota {\n        id: id.to_string(),\n        expr: Expression::try_parse(\n            config,\n            (prefix.as_str(), \"match\"),\n            &TokenMap::default().with_variables(SMTP_QUEUE_HOST_VARS),\n        )\n        .unwrap_or_default(),\n        keys,\n        size: config\n            .property::<Option<u64>>((prefix.as_str(), \"size\"))\n            .filter(|&v| v.as_ref().is_some_and(|v| *v > 0))\n            .unwrap_or_default(),\n        messages: config\n            .property::<Option<u64>>((prefix.as_str(), \"messages\"))\n            .filter(|&v| v.as_ref().is_some_and(|v| *v > 0))\n            .unwrap_or_default(),\n    };\n\n    // Validate\n    if quota.size.is_none() && quota.messages.is_none() {\n        config.new_parse_error(\n            prefix.as_str(),\n            concat!(\n                \"Queue quota needs to define a \",\n                \"valid 'size' and/or 'messages' property.\"\n            )\n            .to_string(),\n        );\n        None\n    } else {\n        Some(quota)\n    }\n}\n\nimpl ParseValue for RequireOptional {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        match value {\n            \"optional\" => Ok(RequireOptional::Optional),\n            \"require\" | \"required\" => Ok(RequireOptional::Require),\n            \"disable\" | \"disabled\" | \"none\" | \"false\" => Ok(RequireOptional::Disable),\n            _ => Err(format!(\"Invalid TLS option value {:?}.\", value,)),\n        }\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for RequireOptional {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::Integer(2) => Ok(RequireOptional::Optional),\n            Variable::Integer(1) => Ok(RequireOptional::Require),\n            Variable::Integer(0) => Ok(RequireOptional::Disable),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl From<RequireOptional> for Constant {\n    fn from(value: RequireOptional) -> Self {\n        Constant::Integer(match value {\n            RequireOptional::Optional => 2,\n            RequireOptional::Require => 1,\n            RequireOptional::Disable => 0,\n        })\n    }\n}\n\nimpl ConstantValue for RequireOptional {\n    fn add_constants(token_map: &mut crate::expr::tokenizer::TokenMap) {\n        token_map\n            .add_constant(\"optional\", RequireOptional::Optional)\n            .add_constant(\"require\", RequireOptional::Require)\n            .add_constant(\"required\", RequireOptional::Require)\n            .add_constant(\"disable\", RequireOptional::Disable)\n            .add_constant(\"disabled\", RequireOptional::Disable)\n            .add_constant(\"none\", RequireOptional::Disable)\n            .add_constant(\"false\", RequireOptional::Disable);\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for IpLookupStrategy {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::Integer(value) => match value {\n                2 => Ok(IpLookupStrategy::Ipv4Only),\n                3 => Ok(IpLookupStrategy::Ipv6Only),\n                4 => Ok(IpLookupStrategy::Ipv6thenIpv4),\n                5 => Ok(IpLookupStrategy::Ipv4thenIpv6),\n                _ => Err(()),\n            },\n            Variable::String(value) => {\n                IpLookupStrategy::parse_value(value.as_str()).map_err(|_| ())\n            }\n            _ => Err(()),\n        }\n    }\n}\n\nimpl From<IpLookupStrategy> for Constant {\n    fn from(value: IpLookupStrategy) -> Self {\n        Constant::Integer(match value {\n            IpLookupStrategy::Ipv4Only => 2,\n            IpLookupStrategy::Ipv6Only => 3,\n            IpLookupStrategy::Ipv6thenIpv4 => 4,\n            IpLookupStrategy::Ipv4thenIpv6 => 5,\n        })\n    }\n}\n\nimpl ConstantValue for IpLookupStrategy {\n    fn add_constants(token_map: &mut crate::expr::tokenizer::TokenMap) {\n        token_map\n            .add_constant(\"ipv4_only\", IpLookupStrategy::Ipv4Only)\n            .add_constant(\"ipv6_only\", IpLookupStrategy::Ipv6Only)\n            .add_constant(\"ipv6_then_ipv4\", IpLookupStrategy::Ipv6thenIpv4)\n            .add_constant(\"ipv4_then_ipv6\", IpLookupStrategy::Ipv4thenIpv6);\n    }\n}\n\nimpl std::fmt::Debug for RelayConfig {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.debug_struct(\"RelayConfig\")\n            .field(\"address\", &self.address)\n            .field(\"port\", &self.port)\n            .field(\"protocol\", &self.protocol)\n            .field(\"tls_implicit\", &self.tls_implicit)\n            .field(\"tls_allow_invalid_certs\", &self.tls_allow_invalid_certs)\n            .finish()\n    }\n}\n\nimpl TlsStrategy {\n    #[inline(always)]\n    pub fn try_dane(&self) -> bool {\n        matches!(\n            self.dane,\n            RequireOptional::Require | RequireOptional::Optional\n        )\n    }\n\n    #[inline(always)]\n    pub fn try_start_tls(&self) -> bool {\n        matches!(\n            self.tls,\n            RequireOptional::Require | RequireOptional::Optional\n        )\n    }\n\n    #[inline(always)]\n    pub fn is_dane_required(&self) -> bool {\n        matches!(self.dane, RequireOptional::Require)\n    }\n\n    #[inline(always)]\n    pub fn try_mta_sts(&self) -> bool {\n        matches!(\n            self.mta_sts,\n            RequireOptional::Require | RequireOptional::Optional\n        )\n    }\n\n    #[inline(always)]\n    pub fn is_mta_sts_required(&self) -> bool {\n        matches!(self.mta_sts, RequireOptional::Require)\n    }\n\n    #[inline(always)]\n    pub fn is_tls_required(&self) -> bool {\n        matches!(self.tls, RequireOptional::Require)\n            || self.is_dane_required()\n            || self.is_mta_sts_required()\n    }\n}\n\nimpl Hash for MxConfig {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.max_mx.hash(state);\n        self.max_multi_homed.hash(state);\n    }\n}\n\nimpl PartialEq for MxConfig {\n    fn eq(&self, other: &Self) -> bool {\n        self.max_mx == other.max_mx && self.max_multi_homed == other.max_multi_homed\n    }\n}\n\nimpl Eq for MxConfig {}\n\nimpl QueueName {\n    pub fn new(name: impl AsRef<[u8]>) -> Option<Self> {\n        let name_bytes = name.as_ref();\n        if (1..=8).contains(&name_bytes.len()) {\n            let mut bytes = [0; 8];\n            bytes[..name_bytes.len()].copy_from_slice(name_bytes);\n            QueueName(bytes).into()\n        } else {\n            None\n        }\n    }\n\n    pub fn from_bytes(name: &[u8]) -> Option<Self> {\n        name.try_into().ok().map(|bytes: [u8; 8]| QueueName(bytes))\n    }\n\n    pub fn as_str(&self) -> &str {\n        std::str::from_utf8(&self.0)\n            .unwrap_or_default()\n            .trim_end_matches('\\0')\n    }\n\n    pub fn into_inner(self) -> [u8; 8] {\n        self.0\n    }\n\n    pub fn as_slice(&self) -> &[u8] {\n        &self.0\n    }\n}\n\nimpl ArchivedQueueName {\n    pub fn as_str(&self) -> &str {\n        std::str::from_utf8(self.0.as_ref())\n            .unwrap_or_default()\n            .trim_end_matches('\\0')\n    }\n\n    pub fn as_slice(&self) -> &[u8] {\n        self.0.as_ref()\n    }\n}\n\nimpl Default for QueueName {\n    fn default() -> Self {\n        DEFAULT_QUEUE_NAME\n    }\n}\n\nimpl ParseValue for QueueName {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        if let Some(name) = QueueName::new(value.trim().as_bytes()) {\n            Ok(name)\n        } else {\n            Err(format!(\n                \"Queue name '{value}' is too long. Maximum length is 8 bytes.\"\n            ))\n        }\n    }\n}\n\nimpl Display for QueueName {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        self.as_str().fmt(f)\n    }\n}\n\nimpl Display for ArchivedQueueName {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        self.as_str().fmt(f)\n    }\n}\n\nimpl AsRef<[u8]> for QueueName {\n    fn as_ref(&self) -> &[u8] {\n        &self.0\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/smtp/report.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse utils::config::{Config, utils::ParseValue};\n\nuse crate::expr::{Constant, ConstantValue, Variable, if_block::IfBlock, tokenizer::TokenMap};\n\nuse super::*;\n\n#[derive(Clone)]\npub struct ReportConfig {\n    pub submitter: IfBlock,\n    pub analysis: ReportAnalysis,\n\n    pub dkim: Report,\n    pub spf: Report,\n    pub dmarc: Report,\n    pub dmarc_aggregate: AggregateReport,\n    pub tls: AggregateReport,\n}\n\n#[derive(Clone)]\npub struct ReportAnalysis {\n    pub addresses: Vec<AddressMatch>,\n    pub forward: bool,\n    pub store: Option<Duration>,\n}\n\n#[derive(Clone)]\npub enum AddressMatch {\n    StartsWith(String),\n    EndsWith(String),\n    Equals(String),\n}\n\n#[derive(Clone)]\npub struct AggregateReport {\n    pub name: IfBlock,\n    pub address: IfBlock,\n    pub org_name: IfBlock,\n    pub contact_info: IfBlock,\n    pub send: IfBlock,\n    pub sign: IfBlock,\n    pub max_size: IfBlock,\n}\n\n#[derive(Clone)]\npub struct Report {\n    pub name: IfBlock,\n    pub address: IfBlock,\n    pub subject: IfBlock,\n    pub sign: IfBlock,\n    pub send: IfBlock,\n}\n\n#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]\npub enum AggregateFrequency {\n    Hourly,\n    Daily,\n    Weekly,\n    #[default]\n    Never,\n}\n\nimpl ReportConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        let sender_vars = TokenMap::default().with_variables(SMTP_MAIL_FROM_VARS);\n        let rcpt_vars = TokenMap::default().with_variables(SMTP_RCPT_TO_VARS);\n\n        Self {\n            submitter: IfBlock::try_parse(\n                config,\n                \"report.submitter\",\n                &TokenMap::default().with_variables(RCPT_DOMAIN_VARS),\n            )\n            .unwrap_or_else(|| {\n                IfBlock::new::<()>(\"report.submitter\", [], \"config_get('server.hostname')\")\n            }),\n            analysis: ReportAnalysis {\n                addresses: config\n                    .properties::<AddressMatch>(\"report.analysis.addresses\")\n                    .into_iter()\n                    .map(|(_, m)| m)\n                    .collect(),\n                forward: config.property(\"report.analysis.forward\").unwrap_or(true),\n                store: config\n                    .property_or_default::<Option<Duration>>(\"report.analysis.store\", \"30d\")\n                    .unwrap_or_default(),\n            },\n            dkim: Report::parse(config, \"dkim\", &rcpt_vars),\n            spf: Report::parse(config, \"spf\", &sender_vars),\n            dmarc: Report::parse(config, \"dmarc\", &rcpt_vars),\n            dmarc_aggregate: AggregateReport::parse(\n                config,\n                \"dmarc\",\n                &rcpt_vars.with_constants::<AggregateFrequency>(),\n            ),\n            tls: AggregateReport::parse(\n                config,\n                \"tls\",\n                &TokenMap::default()\n                    .with_variables(SMTP_QUEUE_HOST_VARS)\n                    .with_constants::<AggregateFrequency>(),\n            ),\n        }\n    }\n}\n\nimpl Report {\n    pub fn parse(config: &mut Config, id: &str, token_map: &TokenMap) -> Self {\n        let mut report = Self {\n            name: IfBlock::new::<()>(format!(\"report.{id}.from-name\"), [], \"'Report Subsystem'\"),\n            address: IfBlock::new::<()>(\n                format!(\"report.{id}.from-address\"),\n                [],\n                format!(\"'noreply-{id}@' + config_get('report.domain')\"),\n            ),\n            subject: IfBlock::new::<()>(\n                format!(\"report.{id}.subject\"),\n                [],\n                format!(\n                    \"'{} Authentication Failure Report'\",\n                    id.to_ascii_uppercase()\n                ),\n            ),\n            sign: IfBlock::new::<()>(\n                format!(\"report.{id}.sign\"),\n                [],\n                \"['rsa-' + config_get('report.domain'), 'ed25519-' + config_get('report.domain')]\",\n            ),\n            send: IfBlock::new::<()>(format!(\"report.{id}.send\"), [], \"[1, 1d]\"),\n        };\n        for (value, key) in [\n            (&mut report.name, \"from-name\"),\n            (&mut report.address, \"from-address\"),\n            (&mut report.subject, \"subject\"),\n            (&mut report.sign, \"sign\"),\n            (&mut report.send, \"send\"),\n        ] {\n            if let Some(if_block) = IfBlock::try_parse(config, (\"report\", id, key), token_map) {\n                *value = if_block;\n            }\n        }\n\n        report\n    }\n}\n\nimpl AggregateReport {\n    pub fn parse(config: &mut Config, id: &str, token_map: &TokenMap) -> Self {\n        let rcpt_vars = TokenMap::default().with_variables(RCPT_DOMAIN_VARS);\n\n        let mut report = Self {\n            name: IfBlock::new::<()>(\n                format!(\"report.{id}.aggregate.from-name\"),\n                [],\n                format!(\"'{} Aggregate Report'\", id.to_ascii_uppercase()),\n            ),\n            address: IfBlock::new::<()>(\n                format!(\"report.{id}.aggregate.from-address\"),\n                [],\n                format!(\"'noreply-{id}@' + config_get('report.domain')\"),\n            ),\n            org_name: IfBlock::new::<()>(\n                format!(\"report.{id}.aggregate.org-name\"),\n                [],\n                \"config_get('report.domain')\",\n            ),\n            contact_info: IfBlock::empty(format!(\"report.{id}.aggregate.contact-info\")),\n            send: IfBlock::new::<AggregateFrequency>(\n                format!(\"report.{id}.aggregate.send\"),\n                [],\n                \"daily\",\n            ),\n            sign: IfBlock::new::<()>(\n                format!(\"report.{id}.aggregate.sign\"),\n                [],\n                \"['rsa-' + config_get('report.domain'), 'ed25519-' + config_get('report.domain')]\",\n            ),\n            max_size: IfBlock::new::<()>(format!(\"report.{id}.aggregate.max-size\"), [], \"26214400\"),\n        };\n\n        for (value, key, token_map) in [\n            (&mut report.name, \"aggregate.from-name\", &rcpt_vars),\n            (&mut report.address, \"aggregate.from-address\", &rcpt_vars),\n            (&mut report.org_name, \"aggregate.org-name\", &rcpt_vars),\n            (\n                &mut report.contact_info,\n                \"aggregate.contact-info\",\n                &rcpt_vars,\n            ),\n            (&mut report.send, \"aggregate.send\", token_map),\n            (&mut report.sign, \"aggregate.sign\", &rcpt_vars),\n            (&mut report.max_size, \"aggregate.max-size\", &rcpt_vars),\n        ] {\n            if let Some(if_block) = IfBlock::try_parse(config, (\"report\", id, key), token_map) {\n                *value = if_block;\n            }\n        }\n\n        report\n    }\n}\n\nimpl Default for ReportConfig {\n    fn default() -> Self {\n        Self::parse(&mut Config::default())\n    }\n}\n\nimpl ParseValue for AggregateFrequency {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        match value {\n            \"daily\" | \"day\" => Ok(AggregateFrequency::Daily),\n            \"hourly\" | \"hour\" => Ok(AggregateFrequency::Hourly),\n            \"weekly\" | \"week\" => Ok(AggregateFrequency::Weekly),\n            \"never\" | \"disable\" | \"false\" => Ok(AggregateFrequency::Never),\n            _ => Err(format!(\"Invalid aggregate frequency value {:?}.\", value,)),\n        }\n    }\n}\n\nimpl From<AggregateFrequency> for Constant {\n    fn from(value: AggregateFrequency) -> Self {\n        match value {\n            AggregateFrequency::Never => 0.into(),\n            AggregateFrequency::Hourly => 2.into(),\n            AggregateFrequency::Daily => 3.into(),\n            AggregateFrequency::Weekly => 4.into(),\n        }\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for AggregateFrequency {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::Integer(0) => Ok(AggregateFrequency::Never),\n            Variable::Integer(2) => Ok(AggregateFrequency::Hourly),\n            Variable::Integer(3) => Ok(AggregateFrequency::Daily),\n            Variable::Integer(4) => Ok(AggregateFrequency::Weekly),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl ConstantValue for AggregateFrequency {\n    fn add_constants(token_map: &mut crate::expr::tokenizer::TokenMap) {\n        token_map\n            .add_constant(\"never\", AggregateFrequency::Never)\n            .add_constant(\"hourly\", AggregateFrequency::Hourly)\n            .add_constant(\"hour\", AggregateFrequency::Hourly)\n            .add_constant(\"daily\", AggregateFrequency::Daily)\n            .add_constant(\"day\", AggregateFrequency::Daily)\n            .add_constant(\"weekly\", AggregateFrequency::Weekly)\n            .add_constant(\"week\", AggregateFrequency::Weekly)\n            .add_constant(\"never\", AggregateFrequency::Never)\n            .add_constant(\"disable\", AggregateFrequency::Never)\n            .add_constant(\"false\", AggregateFrequency::Never);\n    }\n}\n\nimpl ParseValue for AddressMatch {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        if let Some(value) = value.strip_prefix('*').map(|v| v.trim()) {\n            if !value.is_empty() {\n                return Ok(AddressMatch::EndsWith(value.to_lowercase()));\n            }\n        } else if let Some(value) = value.strip_suffix('*').map(|v| v.trim()) {\n            if !value.is_empty() {\n                return Ok(AddressMatch::StartsWith(value.to_lowercase()));\n            }\n        } else if value.contains('@') {\n            return Ok(AddressMatch::Equals(value.trim().to_lowercase()));\n        }\n        Err(format!(\"Invalid address match value {:?}.\", value,))\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/smtp/resolver.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    fmt::Display,\n    hash::{DefaultHasher, Hash, Hasher},\n    net::{IpAddr, Ipv4Addr, SocketAddr},\n    time::Duration,\n};\n\nuse mail_auth::{\n    MessageAuthenticator,\n    hickory_resolver::{\n        TokioResolver,\n        config::{NameServerConfig, ProtocolConfig, ResolverConfig, ResolverOpts},\n        name_server::TokioConnectionProvider,\n        system_conf::read_system_conf,\n    },\n};\nuse serde::{Deserialize, Serialize};\nuse utils::{\n    cache::CacheItemWeight,\n    config::{Config, utils::ParseValue},\n};\n\nuse crate::Server;\n\npub struct Resolvers {\n    pub dns: MessageAuthenticator,\n    pub dnssec: DnssecResolver,\n}\n\n#[derive(Clone)]\npub struct DnssecResolver {\n    pub resolver: TokioResolver,\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]\npub struct TlsaEntry {\n    pub is_end_entity: bool,\n    pub is_sha256: bool,\n    pub is_spki: bool,\n    pub data: Vec<u8>,\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]\npub struct Tlsa {\n    pub entries: Vec<TlsaEntry>,\n    pub has_end_entities: bool,\n    pub has_intermediates: bool,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Default, Clone, Copy, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub enum Mode {\n    Enforce,\n    Testing,\n    #[default]\n    None,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Clone, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub enum MxPattern {\n    Equals(String),\n    StartsWith(String),\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Clone, Serialize, Deserialize)]\npub struct Policy {\n    pub id: String,\n    pub mode: Mode,\n    pub mx: Vec<MxPattern>,\n    pub max_age: u64,\n}\n\nimpl CacheItemWeight for Tlsa {\n    fn weight(&self) -> u64 {\n        self.entries\n            .iter()\n            .map(|entry| (entry.data.len() + std::mem::size_of::<TlsaEntry>()) as u64)\n            .sum::<u64>()\n            + std::mem::size_of::<Tlsa>() as u64\n    }\n}\n\nimpl CacheItemWeight for Policy {\n    fn weight(&self) -> u64 {\n        (std::mem::size_of::<Policy>()\n            + self\n                .mx\n                .iter()\n                .map(|mx| match mx {\n                    MxPattern::Equals(t) => t.len(),\n                    MxPattern::StartsWith(t) => t.len(),\n                })\n                .sum::<usize>()) as u64\n    }\n}\n\nimpl Resolvers {\n    pub async fn parse(config: &mut Config) -> Self {\n        let (resolver_config, mut opts) = match config.value(\"resolver.type\").unwrap_or(\"system\") {\n            \"cloudflare\" => (ResolverConfig::cloudflare(), ResolverOpts::default()),\n            \"cloudflare-tls\" => (ResolverConfig::cloudflare_tls(), ResolverOpts::default()),\n            \"quad9\" => (ResolverConfig::quad9(), ResolverOpts::default()),\n            \"quad9-tls\" => (ResolverConfig::quad9_tls(), ResolverOpts::default()),\n            \"google\" => (ResolverConfig::google(), ResolverOpts::default()),\n            \"system\" => read_system_conf()\n                .map_err(|err| {\n                    config.new_build_error(\n                        \"resolver.type\",\n                        format!(\"Failed to read system DNS config: {err}\"),\n                    )\n                })\n                .unwrap_or_else(|_| (ResolverConfig::cloudflare(), ResolverOpts::default())),\n            \"custom\" => {\n                let mut resolver_config = ResolverConfig::default();\n                for url in config\n                    .values(\"resolver.custom\")\n                    .map(|(_, v)| v.to_string())\n                    .collect::<Vec<_>>()\n                {\n                    let (proto, host) = if let Some((proto, host)) = url\n                        .split_once(\"://\")\n                        .map(|(a, b)| (a.to_string(), b.to_string()))\n                    {\n                        (\n                            match proto.as_str() {\n                                \"udp\" => ProtocolConfig::Udp,\n                                \"tcp\" => ProtocolConfig::Tcp,\n                                \"tls\" => ProtocolConfig::Tls {\n                                    server_name: host.clone().into(),\n                                },\n                                _ => {\n                                    config.new_parse_error(\n                                        \"resolver.custom\",\n                                        format!(\"Invalid custom resolver protocol {url:?}\"),\n                                    );\n                                    ProtocolConfig::Udp\n                                }\n                            },\n                            host.to_string(),\n                        )\n                    } else {\n                        (ProtocolConfig::Udp, url)\n                    };\n\n                    let (host, port) = if let Some(host) = host.strip_prefix('[') {\n                        let (host, maybe_port) = host.rsplit_once(']').unwrap_or_default();\n\n                        (\n                            host,\n                            maybe_port\n                                .rsplit_once(':')\n                                .map(|(_, port)| port)\n                                .unwrap_or(\"53\"),\n                        )\n                    } else if let Some((host, port)) = host.split_once(':') {\n                        (host, port)\n                    } else {\n                        (host.as_str(), \"53\")\n                    };\n\n                    let port = port\n                        .parse::<u16>()\n                        .map_err(|err| {\n                            config.new_parse_error(\n                                \"resolver.custom\",\n                                format!(\"Invalid custom resolver port {port:?}: {err}\"),\n                            );\n                        })\n                        .unwrap_or(53);\n\n                    let host = host\n                        .parse::<IpAddr>()\n                        .map_err(|err| {\n                            config.new_parse_error(\n                                \"resolver.custom\",\n                                format!(\"Invalid custom resolver IP {host:?}: {err}\"),\n                            )\n                        })\n                        .unwrap_or(IpAddr::V4(Ipv4Addr::new(8, 8, 8, 8)));\n                    resolver_config\n                        .add_name_server(NameServerConfig::new(SocketAddr::new(host, port), proto));\n                }\n                if !resolver_config.name_servers().is_empty() {\n                    (resolver_config, ResolverOpts::default())\n                } else {\n                    config.new_parse_error(\n                        \"resolver.custom\",\n                        \"At least one custom resolver must be specified.\",\n                    );\n                    (ResolverConfig::cloudflare(), ResolverOpts::default())\n                }\n            }\n            other => {\n                let err = format!(\"Unknown resolver type {other:?}.\");\n                config.new_parse_error(\"resolver.custom\", err);\n                (ResolverConfig::cloudflare(), ResolverOpts::default())\n            }\n        };\n        if let Some(concurrency) = config.property(\"resolver.concurrency\") {\n            opts.num_concurrent_reqs = concurrency;\n        }\n        if let Some(timeout) = config.property(\"resolver.timeout\") {\n            opts.timeout = timeout;\n        }\n        if let Some(preserve) = config.property(\"resolver.preserve-intermediates\") {\n            opts.preserve_intermediates = preserve;\n        }\n        if let Some(try_tcp_on_error) = config.property(\"resolver.try-tcp-on-error\") {\n            opts.try_tcp_on_error = try_tcp_on_error;\n        }\n        if let Some(attempts) = config.property(\"resolver.attempts\") {\n            opts.attempts = attempts;\n        }\n        opts.edns0 = config\n            .property_or_default(\"resolver.edns\", \"true\")\n            .unwrap_or(true);\n\n        // We already have a cache, so disable the built-in cache\n        opts.cache_size = 0;\n\n        // Prepare DNSSEC resolver options\n        let config_dnssec = resolver_config.clone();\n        let mut opts_dnssec = opts.clone();\n        opts_dnssec.validate = true;\n\n        Resolvers {\n            dns: MessageAuthenticator::new(resolver_config, opts).unwrap(),\n            dnssec: DnssecResolver {\n                resolver: TokioResolver::builder_with_config(\n                    config_dnssec,\n                    TokioConnectionProvider::default(),\n                )\n                .with_options(opts_dnssec)\n                .build(),\n            },\n        }\n    }\n}\n\nimpl Policy {\n    pub fn try_parse(config: &mut Config) -> Option<Self> {\n        let mode = config\n            .property_or_default::<Option<Mode>>(\"session.mta-sts.mode\", \"testing\")\n            .unwrap_or_default()?;\n        let max_age = config\n            .property_or_default::<Duration>(\"session.mta-sts.max-age\", \"7d\")\n            .unwrap_or_else(|| Duration::from_secs(604800))\n            .as_secs();\n        let mut mx = Vec::new();\n\n        for (_, item) in config.values(\"session.mta-sts.mx\") {\n            if let Some(item) = item.strip_prefix(\"*.\") {\n                mx.push(MxPattern::StartsWith(item.to_string()));\n            } else {\n                mx.push(MxPattern::Equals(item.to_string()));\n            }\n        }\n\n        let mut policy = Self {\n            id: Default::default(),\n            mode,\n            mx,\n            max_age,\n        };\n\n        if !policy.mx.is_empty() {\n            policy.mx.sort_unstable();\n            policy.id = policy.hash().to_string();\n        }\n\n        policy.into()\n    }\n\n    pub fn try_build<I, T>(mut self, names: I) -> Option<Self>\n    where\n        I: IntoIterator<Item = T>,\n        T: AsRef<str>,\n    {\n        if self.mx.is_empty() {\n            for name in names {\n                let name = name.as_ref();\n                if let Some(domain) = name.strip_prefix('.') {\n                    self.mx.push(MxPattern::StartsWith(domain.to_string()));\n                } else if name != \"*\" && !name.is_empty() {\n                    self.mx.push(MxPattern::Equals(name.to_string()));\n                }\n            }\n\n            if !self.mx.is_empty() {\n                self.mx.sort_unstable();\n                self.id = self.hash().to_string();\n                Some(self)\n            } else {\n                None\n            }\n        } else {\n            Some(self)\n        }\n    }\n\n    fn hash(&self) -> u64 {\n        let mut s = DefaultHasher::new();\n        self.mode.hash(&mut s);\n        self.max_age.hash(&mut s);\n        self.mx.hash(&mut s);\n        s.finish()\n    }\n}\n\nimpl Server {\n    pub fn build_mta_sts_policy(&self) -> Option<Policy> {\n        self.core\n            .smtp\n            .session\n            .mta_sts_policy\n            .clone()\n            .and_then(|policy| {\n                policy.try_build(\n                    self.inner\n                        .data\n                        .tls_certificates\n                        .load()\n                        .keys()\n                        .filter(|key| {\n                            !key.starts_with(\"mta-sts.\")\n                                && !key.starts_with(\"autoconfig.\")\n                                && !key.starts_with(\"autodiscover.\")\n                        }),\n                )\n            })\n    }\n}\n\nimpl ParseValue for Mode {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        match value {\n            \"enforce\" => Ok(Self::Enforce),\n            \"testing\" | \"test\" => Ok(Self::Testing),\n            \"none\" => Ok(Self::None),\n            _ => Err(format!(\"Invalid mode value {value:?}\")),\n        }\n    }\n}\n\nimpl Default for Resolvers {\n    fn default() -> Self {\n        let (config, opts) = match read_system_conf() {\n            Ok(conf) => conf,\n            Err(_) => (ResolverConfig::cloudflare(), ResolverOpts::default()),\n        };\n\n        let config_dnssec = config.clone();\n        let mut opts_dnssec = opts.clone();\n        opts_dnssec.validate = true;\n\n        Self {\n            dns: MessageAuthenticator::new(config, opts).expect(\"Failed to build DNS resolver\"),\n            dnssec: DnssecResolver {\n                resolver: TokioResolver::builder_with_config(\n                    config_dnssec,\n                    TokioConnectionProvider::default(),\n                )\n                .with_options(opts_dnssec)\n                .build(),\n            },\n        }\n    }\n}\n\nimpl Display for Policy {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(\"version: STSv1\\r\\n\")?;\n        f.write_str(\"mode: \")?;\n        match self.mode {\n            Mode::Enforce => f.write_str(\"enforce\")?,\n            Mode::Testing => f.write_str(\"testing\")?,\n            Mode::None => unreachable!(),\n        }\n        f.write_str(\"\\r\\nmax_age: \")?;\n        self.max_age.fmt(f)?;\n        f.write_str(\"\\r\\n\")?;\n\n        for mx in &self.mx {\n            f.write_str(\"mx: \")?;\n            mx.fmt(f)?;\n            f.write_str(\"\\r\\n\")?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Display for MxPattern {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            MxPattern::Equals(mx) => f.write_str(mx),\n            MxPattern::StartsWith(mx) => {\n                f.write_str(\"*.\")?;\n                f.write_str(mx)\n            }\n        }\n    }\n}\n\nimpl Clone for Resolvers {\n    fn clone(&self) -> Self {\n        Self {\n            dns: self.dns.clone(),\n            dnssec: self.dnssec.clone(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/smtp/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    net::{SocketAddr, ToSocketAddrs},\n    str::FromStr,\n    time::Duration,\n};\n\nuse ahash::AHashSet;\nuse base64::{Engine, engine::general_purpose::STANDARD};\n\nuse hyper::{\n    HeaderMap,\n    header::{AUTHORIZATION, CONTENT_TYPE, HeaderName, HeaderValue},\n};\nuse smtp_proto::*;\nuse utils::config::{Config, utils::ParseValue};\n\nuse crate::{\n    config::CONNECTION_VARS,\n    expr::{if_block::IfBlock, tokenizer::TokenMap, *},\n};\n\nuse self::resolver::Policy;\n\nuse super::*;\n\n#[derive(Clone)]\npub struct SessionConfig {\n    pub timeout: IfBlock,\n    pub duration: IfBlock,\n    pub transfer_limit: IfBlock,\n\n    pub connect: Connect,\n    pub ehlo: Ehlo,\n    pub auth: Auth,\n    pub mail: Mail,\n    pub rcpt: Rcpt,\n    pub data: Data,\n    pub extensions: Extensions,\n    pub mta_sts_policy: Option<Policy>,\n\n    pub milters: Vec<Milter>,\n    pub hooks: Vec<MTAHook>,\n}\n\n#[derive(Clone)]\npub struct Connect {\n    pub hostname: IfBlock,\n    pub script: IfBlock,\n    pub greeting: IfBlock,\n}\n\n#[derive(Clone)]\npub struct Ehlo {\n    pub script: IfBlock,\n    pub require: IfBlock,\n    pub reject_non_fqdn: IfBlock,\n}\n\n#[derive(Clone)]\npub struct Extensions {\n    pub pipelining: IfBlock,\n    pub chunking: IfBlock,\n    pub requiretls: IfBlock,\n    pub dsn: IfBlock,\n    pub vrfy: IfBlock,\n    pub expn: IfBlock,\n    pub no_soliciting: IfBlock,\n    pub future_release: IfBlock,\n    pub deliver_by: IfBlock,\n    pub mt_priority: IfBlock,\n}\n\n#[derive(Clone)]\npub struct Auth {\n    pub directory: IfBlock,\n    pub mechanisms: IfBlock,\n    pub require: IfBlock,\n    pub must_match_sender: IfBlock,\n    pub errors_max: IfBlock,\n    pub errors_wait: IfBlock,\n}\n\n#[derive(Clone)]\npub struct Mail {\n    pub script: IfBlock,\n    pub rewrite: IfBlock,\n    pub is_allowed: IfBlock,\n}\n\n#[derive(Clone)]\npub struct Rcpt {\n    pub script: IfBlock,\n    pub relay: IfBlock,\n    pub directory: IfBlock,\n    pub rewrite: IfBlock,\n\n    // Errors\n    pub errors_max: IfBlock,\n    pub errors_wait: IfBlock,\n\n    // Limits\n    pub max_recipients: IfBlock,\n\n    // Catch-all and sub-addressing\n    pub catch_all: AddressMapping,\n    pub subaddressing: AddressMapping,\n}\n\n#[derive(Debug, Default, Clone)]\npub enum AddressMapping {\n    Enable,\n    Custom(IfBlock),\n    #[default]\n    Disable,\n}\n\n#[derive(Clone)]\npub struct Data {\n    pub script: IfBlock,\n    pub spam_filter: IfBlock,\n\n    // Limits\n    pub max_messages: IfBlock,\n    pub max_message_size: IfBlock,\n    pub max_received_headers: IfBlock,\n\n    // Headers\n    pub add_received: IfBlock,\n    pub add_received_spf: IfBlock,\n    pub add_return_path: IfBlock,\n    pub add_auth_results: IfBlock,\n    pub add_message_id: IfBlock,\n    pub add_date: IfBlock,\n    pub add_delivered_to: bool,\n}\n\n#[derive(Clone)]\npub struct Milter {\n    pub enable: IfBlock,\n    pub id: Arc<String>,\n    pub addrs: Vec<SocketAddr>,\n    pub hostname: String,\n    pub port: u16,\n    pub timeout_connect: Duration,\n    pub timeout_command: Duration,\n    pub timeout_data: Duration,\n    pub tls: bool,\n    pub tls_allow_invalid_certs: bool,\n    pub tempfail_on_error: bool,\n    pub max_frame_len: usize,\n    pub protocol_version: MilterVersion,\n    pub flags_actions: Option<u32>,\n    pub flags_protocol: Option<u32>,\n    pub run_on_stage: AHashSet<Stage>,\n}\n\n#[derive(Clone, Copy)]\npub enum MilterVersion {\n    V2,\n    V6,\n}\n\n#[derive(Clone)]\npub struct MTAHook {\n    pub enable: IfBlock,\n    pub id: String,\n    pub url: String,\n    pub timeout: Duration,\n    pub headers: HeaderMap,\n    pub tls_allow_invalid_certs: bool,\n    pub tempfail_on_error: bool,\n    pub run_on_stage: AHashSet<Stage>,\n    pub max_response_size: usize,\n}\n\n#[derive(Clone, Copy, PartialEq, Eq, Hash)]\npub enum Stage {\n    Connect,\n    Ehlo,\n    Auth,\n    Mail,\n    Rcpt,\n    Data,\n}\n\nimpl SessionConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        let has_conn_vars = TokenMap::default().with_variables(CONNECTION_VARS);\n        let has_ehlo_hars = TokenMap::default().with_variables(SMTP_EHLO_VARS);\n        let has_sender_vars = TokenMap::default().with_variables(SMTP_MAIL_FROM_VARS);\n        let has_rcpt_vars = TokenMap::default().with_variables(SMTP_RCPT_TO_VARS);\n        let mt_priority_vars = has_sender_vars.clone().with_constants::<MtPriority>();\n        let mechanisms_vars = has_ehlo_hars.clone().with_constants::<Mechanism>();\n\n        let mut session = SessionConfig::default();\n        session.rcpt.catch_all = AddressMapping::parse(config, \"session.rcpt.catch-all\");\n        session.rcpt.subaddressing = AddressMapping::parse(config, \"session.rcpt.sub-addressing\");\n        session.milters = config\n            .sub_keys(\"session.milter\", \".hostname\")\n            .into_iter()\n            .filter_map(|id| parse_milter(config, &id, &has_rcpt_vars))\n            .collect();\n        session.hooks = config\n            .sub_keys(\"session.hook\", \".url\")\n            .into_iter()\n            .filter_map(|id| parse_hooks(config, &id, &has_rcpt_vars))\n            .collect();\n        session.mta_sts_policy = Policy::try_parse(config);\n\n        for (value, key, token_map) in [\n            (&mut session.duration, \"session.duration\", &has_conn_vars),\n            (\n                &mut session.transfer_limit,\n                \"session.transfer-limit\",\n                &has_conn_vars,\n            ),\n            (&mut session.timeout, \"session.timeout\", &has_conn_vars),\n            (\n                &mut session.connect.script,\n                \"session.connect.script\",\n                &has_conn_vars,\n            ),\n            (\n                &mut session.connect.hostname,\n                \"session.connect.hostname\",\n                &has_conn_vars,\n            ),\n            (\n                &mut session.connect.greeting,\n                \"session.connect.greeting\",\n                &has_conn_vars,\n            ),\n            (\n                &mut session.extensions.pipelining,\n                \"session.extensions.pipelining\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.dsn,\n                \"session.extensions.dsn\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.vrfy,\n                \"session.extensions.vrfy\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.expn,\n                \"session.extensions.expn\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.chunking,\n                \"session.extensions.chunking\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.requiretls,\n                \"session.extensions.requiretls\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.no_soliciting,\n                \"session.extensions.no-soliciting\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.future_release,\n                \"session.extensions.future-release\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.deliver_by,\n                \"session.extensions.deliver-by\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.extensions.mt_priority,\n                \"session.extensions.mt-priority\",\n                &mt_priority_vars,\n            ),\n            (\n                &mut session.ehlo.script,\n                \"session.ehlo.script\",\n                &has_conn_vars,\n            ),\n            (\n                &mut session.ehlo.require,\n                \"session.ehlo.require\",\n                &has_conn_vars,\n            ),\n            (\n                &mut session.ehlo.reject_non_fqdn,\n                \"session.ehlo.reject-non-fqdn\",\n                &has_conn_vars,\n            ),\n            (\n                &mut session.auth.directory,\n                \"session.auth.directory\",\n                &has_ehlo_hars,\n            ),\n            (\n                &mut session.auth.mechanisms,\n                \"session.auth.mechanisms\",\n                &mechanisms_vars,\n            ),\n            (\n                &mut session.auth.require,\n                \"session.auth.require\",\n                &has_ehlo_hars,\n            ),\n            (\n                &mut session.auth.errors_max,\n                \"session.auth.errors.total\",\n                &has_ehlo_hars,\n            ),\n            (\n                &mut session.auth.errors_wait,\n                \"session.auth.errors.wait\",\n                &has_ehlo_hars,\n            ),\n            (\n                &mut session.auth.must_match_sender,\n                \"session.auth.must-match-sender\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.mail.script,\n                \"session.mail.script\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.mail.rewrite,\n                \"session.mail.rewrite\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.mail.is_allowed,\n                \"session.mail.is-allowed\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.rcpt.script,\n                \"session.rcpt.script\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.rcpt.relay,\n                \"session.rcpt.relay\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.rcpt.directory,\n                \"session.rcpt.directory\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.rcpt.errors_max,\n                \"session.rcpt.errors.total\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.rcpt.errors_wait,\n                \"session.rcpt.errors.wait\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.rcpt.max_recipients,\n                \"session.rcpt.max-recipients\",\n                &has_sender_vars,\n            ),\n            (\n                &mut session.rcpt.rewrite,\n                \"session.rcpt.rewrite\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.script,\n                \"session.data.script\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.max_messages,\n                \"session.data.limits.messages\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.max_message_size,\n                \"session.data.limits.size\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.max_received_headers,\n                \"session.data.limits.received-headers\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.spam_filter,\n                \"session.data.spam-filter\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.add_received,\n                \"session.data.add-headers.received\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.add_received_spf,\n                \"session.data.add-headers.received-spf\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.add_return_path,\n                \"session.data.add-headers.return-path\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.add_auth_results,\n                \"session.data.add-headers.auth-results\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.add_message_id,\n                \"session.data.add-headers.message-id\",\n                &has_rcpt_vars,\n            ),\n            (\n                &mut session.data.add_date,\n                \"session.data.add-headers.date\",\n                &has_rcpt_vars,\n            ),\n        ] {\n            if let Some(if_block) = IfBlock::try_parse(config, key, token_map) {\n                *value = if_block;\n            }\n        }\n        session.data.add_delivered_to = config\n            .property_or_default(\"session.data.add-headers.delivered-to\", \"true\")\n            .unwrap_or(true);\n        session\n    }\n}\n\nfn parse_milter(config: &mut Config, id: &str, token_map: &TokenMap) -> Option<Milter> {\n    let hostname = config\n        .value_require((\"session.milter\", id, \"hostname\"))?\n        .to_string();\n    let port = config.property_require((\"session.milter\", id, \"port\"))?;\n    Some(Milter {\n        enable: IfBlock::try_parse(config, (\"session.milter\", id, \"enable\"), token_map)\n            .unwrap_or_else(|| {\n                IfBlock::new::<()>(format!(\"session.milter.{id}.enable\"), [], \"false\")\n            }),\n        id: Arc::new(id.into()),\n        addrs: format!(\"{}:{}\", hostname, port)\n            .to_socket_addrs()\n            .map_err(|err| {\n                config.new_build_error(\n                    (\"session.milter\", id, \"hostname\"),\n                    format!(\"Unable to resolve milter hostname {hostname}: {err}\"),\n                )\n            })\n            .ok()?\n            .collect(),\n        hostname,\n        port,\n        timeout_connect: config\n            .property_or_default((\"session.milter\", id, \"timeout.connect\"), \"30s\")\n            .unwrap_or_else(|| Duration::from_secs(30)),\n        timeout_command: config\n            .property_or_default((\"session.milter\", id, \"timeout.command\"), \"30s\")\n            .unwrap_or_else(|| Duration::from_secs(30)),\n        timeout_data: config\n            .property_or_default((\"session.milter\", id, \"timeout.data\"), \"60s\")\n            .unwrap_or_else(|| Duration::from_secs(60)),\n        tls: config\n            .property_or_default((\"session.milter\", id, \"tls\"), \"false\")\n            .unwrap_or_default(),\n        tls_allow_invalid_certs: config\n            .property_or_default((\"session.milter\", id, \"allow-invalid-certs\"), \"false\")\n            .unwrap_or_default(),\n        tempfail_on_error: config\n            .property_or_default((\"session.milter\", id, \"options.tempfail-on-error\"), \"true\")\n            .unwrap_or(true),\n        max_frame_len: config\n            .property_or_default(\n                (\"session.milter\", id, \"options.max-response-size\"),\n                \"52428800\",\n            )\n            .unwrap_or(52428800),\n        protocol_version: match config\n            .property_or_default::<u32>((\"session.milter\", id, \"options.version\"), \"6\")\n            .unwrap_or(6)\n        {\n            6 => MilterVersion::V6,\n            2 => MilterVersion::V2,\n            v => {\n                config.new_parse_error(\n                    (\"session.milter\", id, \"options.version\"),\n                    format!(\"Unsupported milter protocol version {v}\"),\n                );\n                MilterVersion::V6\n            }\n        },\n        flags_actions: config.property((\"session.milter\", id, \"options.flags.actions\")),\n        flags_protocol: config.property((\"session.milter\", id, \"options.flags.protocol\")),\n        run_on_stage: parse_stages(config, \"session.milter\", id),\n    })\n}\n\nfn parse_hooks(config: &mut Config, id: &str, token_map: &TokenMap) -> Option<MTAHook> {\n    let mut headers = HeaderMap::new();\n\n    for (header, value) in config\n        .values((\"session.hook\", id, \"headers\"))\n        .map(|(_, v)| {\n            if let Some((k, v)) = v.split_once(':') {\n                Ok((\n                    HeaderName::from_str(k.trim()).map_err(|err| {\n                        format!(\n                            \"Invalid header found in property \\\"session.hook.{id}.headers\\\": {err}\",\n                        )\n                    })?,\n                    HeaderValue::from_str(v.trim()).map_err(|err| {\n                        format!(\n                            \"Invalid header found in property \\\"session.hook.{id}.headers\\\": {err}\",\n                        )\n                    })?,\n                ))\n            } else {\n                Err(format!(\n                    \"Invalid header found in property \\\"session.hook.{id}.headers\\\": {v}\",\n                ))\n            }\n        })\n        .collect::<Result<Vec<(HeaderName, HeaderValue)>, String>>()\n        .map_err(|e| config.new_parse_error((\"session.hook\", id, \"headers\"), e))\n        .unwrap_or_default()\n    {\n        headers.insert(header, value);\n    }\n\n    headers.insert(CONTENT_TYPE, \"application/json\".parse().unwrap());\n    if let (Some(name), Some(secret)) = (\n        config.value((\"session.hook\", id, \"auth.username\")),\n        config.value((\"session.hook\", id, \"auth.secret\")),\n    ) {\n        headers.insert(\n            AUTHORIZATION,\n            format!(\"Basic {}\", STANDARD.encode(format!(\"{}:{}\", name, secret)))\n                .parse()\n                .unwrap(),\n        );\n    }\n\n    Some(MTAHook {\n        enable: IfBlock::try_parse(config, (\"session.hook\", id, \"enable\"), token_map)\n            .unwrap_or_else(|| {\n                IfBlock::new::<()>(format!(\"session.hook.{id}.enable\"), [], \"false\")\n            }),\n        id: id.to_string(),\n        url: config\n            .value_require((\"session.hook\", id, \"url\"))?\n            .to_string(),\n        timeout: config\n            .property_or_default((\"session.hook\", id, \"timeout\"), \"30s\")\n            .unwrap_or_else(|| Duration::from_secs(30)),\n        tls_allow_invalid_certs: config\n            .property_or_default((\"session.hook\", id, \"allow-invalid-certs\"), \"false\")\n            .unwrap_or_default(),\n        tempfail_on_error: config\n            .property_or_default((\"session.hook\", id, \"options.tempfail-on-error\"), \"true\")\n            .unwrap_or(true),\n        run_on_stage: parse_stages(config, \"session.hook\", id),\n        max_response_size: config\n            .property_or_default(\n                (\"session.hook\", id, \"options.max-response-size\"),\n                \"52428800\",\n            )\n            .unwrap_or(52428800),\n        headers,\n    })\n}\n\nfn parse_stages(config: &mut Config, prefix: &str, id: &str) -> AHashSet<Stage> {\n    let mut stages = AHashSet::default();\n    let mut invalid = Vec::new();\n    for (_, value) in config.values((prefix, id, \"stages\")) {\n        let value = value.to_ascii_lowercase();\n        let state = match value.as_str() {\n            \"connect\" => Stage::Connect,\n            \"ehlo\" => Stage::Ehlo,\n            \"auth\" => Stage::Auth,\n            \"mail\" => Stage::Mail,\n            \"rcpt\" => Stage::Rcpt,\n            \"data\" => Stage::Data,\n            _ => {\n                invalid.push(value);\n                continue;\n            }\n        };\n        stages.insert(state);\n    }\n\n    if !invalid.is_empty() {\n        config.new_parse_error(\n            (prefix, id, \"stages\"),\n            format!(\"Invalid stages: {}\", invalid.join(\", \")),\n        );\n    }\n\n    if stages.is_empty() {\n        stages.insert(Stage::Data);\n    }\n\n    stages\n}\n\nimpl Default for SessionConfig {\n    fn default() -> Self {\n        Self {\n            timeout: IfBlock::new::<()>(\"session.timeout\", [], \"5m\"),\n            duration: IfBlock::new::<()>(\"session.duration\", [], \"10m\"),\n            transfer_limit: IfBlock::new::<()>(\"session.transfer-limit\", [], \"262144000\"),\n            connect: Connect {\n                hostname: IfBlock::new::<()>(\n                    \"server.connect.hostname\",\n                    [],\n                    \"config_get('server.hostname')\",\n                ),\n                script: IfBlock::empty(\"session.connect.script\"),\n                greeting: IfBlock::new::<()>(\n                    \"session.connect.greeting\",\n                    [],\n                    \"config_get('server.hostname') + ' Stalwart ESMTP at your service'\",\n                ),\n            },\n            ehlo: Ehlo {\n                script: IfBlock::empty(\"session.ehlo.script\"),\n                require: IfBlock::new::<()>(\"session.ehlo.require\", [], \"true\"),\n                reject_non_fqdn: IfBlock::new::<()>(\n                    \"session.ehlo.reject-non-fqdn\",\n                    [(\"local_port == 25\", \"true\")],\n                    \"false\",\n                ),\n            },\n            auth: Auth {\n                directory: IfBlock::new::<()>(\n                    \"session.auth.directory\",\n                    #[cfg(feature = \"test_mode\")]\n                    [],\n                    #[cfg(not(feature = \"test_mode\"))]\n                    [(\"local_port != 25\", \"'*'\")],\n                    \"false\",\n                ),\n                mechanisms: IfBlock::new::<Mechanism>(\n                    \"session.auth.mechanisms\",\n                    [\n                        (\n                            \"local_port != 25 && is_tls\",\n                            \"[plain, login, oauthbearer, xoauth2]\",\n                        ),\n                        (\"local_port != 25\", \"[oauthbearer, xoauth2]\"),\n                    ],\n                    \"false\",\n                ),\n                require: IfBlock::new::<()>(\n                    \"session.auth.require\",\n                    #[cfg(feature = \"test_mode\")]\n                    [],\n                    #[cfg(not(feature = \"test_mode\"))]\n                    [(\"local_port != 25\", \"true\")],\n                    \"false\",\n                ),\n                must_match_sender: IfBlock::new::<()>(\"session.auth.must-match-sender\", [], \"true\"),\n                errors_max: IfBlock::new::<()>(\"session.auth.errors.total\", [], \"3\"),\n                errors_wait: IfBlock::new::<()>(\"session.auth.errors.wait\", [], \"5s\"),\n            },\n            mail: Mail {\n                script: IfBlock::empty(\"session.mail.script\"),\n                rewrite: IfBlock::empty(\"session.mail.rewrite\"),\n                is_allowed: IfBlock::new::<()>(\n                    \"session.mail.is-allowed\",\n                    [],\n                    \"!is_empty(authenticated_as) || !key_exists('blocked-domains', sender_domain)\",\n                ),\n            },\n            rcpt: Rcpt {\n                script: IfBlock::empty(\"session.rcpt.script\"),\n                relay: IfBlock::new::<()>(\n                    \"session.rcpt.relay\",\n                    [(\"!is_empty(authenticated_as)\", \"true\")],\n                    \"false\",\n                ),\n                directory: IfBlock::new::<()>(\n                    \"session.rcpt.directory\",\n                    [],\n                    #[cfg(feature = \"test_mode\")]\n                    \"false\",\n                    #[cfg(not(feature = \"test_mode\"))]\n                    \"'*'\",\n                ),\n                rewrite: IfBlock::empty(\"session.rcpt.rewrite\"),\n                errors_max: IfBlock::new::<()>(\"session.rcpt.errors.total\", [], \"5\"),\n                errors_wait: IfBlock::new::<()>(\"session.rcpt.errors.wait\", [], \"5s\"),\n                max_recipients: IfBlock::new::<()>(\"session.rcpt.max-recipients\", [], \"100\"),\n                catch_all: AddressMapping::Enable,\n                subaddressing: AddressMapping::Enable,\n            },\n            data: Data {\n                script: IfBlock::empty(\"session.data.script\"),\n                spam_filter: IfBlock::new::<()>(\"session.data.spam-filter\", [], \"true\"),\n                max_messages: IfBlock::new::<()>(\"session.data.limits.messages\", [], \"10\"),\n                max_message_size: IfBlock::new::<()>(\"session.data.limits.size\", [], \"104857600\"),\n                max_received_headers: IfBlock::new::<()>(\n                    \"session.data.limits.received-headers\",\n                    [],\n                    \"50\",\n                ),\n                add_received: IfBlock::new::<()>(\n                    \"session.data.add-headers.received\",\n                    [(\"local_port == 25\", \"true\")],\n                    \"false\",\n                ),\n                add_received_spf: IfBlock::new::<()>(\n                    \"session.data.add-headers.received-spf\",\n                    [(\"local_port == 25\", \"true\")],\n                    \"false\",\n                ),\n                add_return_path: IfBlock::new::<()>(\n                    \"session.data.add-headers.return-path\",\n                    [(\"local_port == 25\", \"true\")],\n                    \"false\",\n                ),\n                add_auth_results: IfBlock::new::<()>(\n                    \"session.data.add-headers.auth-results\",\n                    [(\"local_port == 25\", \"true\")],\n                    \"false\",\n                ),\n                add_message_id: IfBlock::new::<()>(\n                    \"session.data.add-headers.message-id\",\n                    [(\"local_port == 25\", \"true\")],\n                    \"false\",\n                ),\n                add_date: IfBlock::new::<()>(\n                    \"session.data.add-headers.date\",\n                    [(\"local_port == 25\", \"true\")],\n                    \"false\",\n                ),\n                add_delivered_to: false,\n            },\n            extensions: Extensions {\n                pipelining: IfBlock::new::<()>(\"session.extensions.pipelining\", [], \"true\"),\n                chunking: IfBlock::new::<()>(\"session.extensions.chunking\", [], \"true\"),\n                requiretls: IfBlock::new::<()>(\"session.extensions.requiretls\", [], \"true\"),\n                dsn: IfBlock::new::<()>(\n                    \"session.extensions.dsn\",\n                    [(\"!is_empty(authenticated_as)\", \"true\")],\n                    \"false\",\n                ),\n                vrfy: IfBlock::new::<()>(\n                    \"session.extensions.vrfy\",\n                    [(\"!is_empty(authenticated_as)\", \"true\")],\n                    \"false\",\n                ),\n                expn: IfBlock::new::<()>(\n                    \"session.extensions.expn\",\n                    [(\"!is_empty(authenticated_as)\", \"true\")],\n                    \"false\",\n                ),\n                no_soliciting: IfBlock::new::<()>(\"session.extensions.no-soliciting\", [], \"''\"),\n                future_release: IfBlock::new::<()>(\n                    \"session.extensions.future-release\",\n                    [(\"!is_empty(authenticated_as)\", \"7d\")],\n                    \"false\",\n                ),\n                deliver_by: IfBlock::new::<()>(\n                    \"session.extensions.deliver-by\",\n                    [(\"!is_empty(authenticated_as)\", \"15d\")],\n                    \"false\",\n                ),\n                mt_priority: IfBlock::new::<MtPriority>(\n                    \"session.extensions.mt-priority\",\n                    [(\"!is_empty(authenticated_as)\", \"mixer\")],\n                    \"false\",\n                ),\n            },\n            mta_sts_policy: None,\n            milters: Default::default(),\n            hooks: Default::default(),\n        }\n    }\n}\n\n#[derive(Default)]\npub struct Mechanism(u64);\n\nimpl ParseValue for Mechanism {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        Ok(Mechanism(match value.to_ascii_uppercase().as_str() {\n            \"LOGIN\" => AUTH_LOGIN,\n            \"PLAIN\" => AUTH_PLAIN,\n            \"XOAUTH2\" => AUTH_XOAUTH2,\n            \"OAUTHBEARER\" => AUTH_OAUTHBEARER,\n            /*\"SCRAM-SHA-256-PLUS\" => AUTH_SCRAM_SHA_256_PLUS,\n            \"SCRAM-SHA-256\" => AUTH_SCRAM_SHA_256,\n            \"SCRAM-SHA-1-PLUS\" => AUTH_SCRAM_SHA_1_PLUS,\n            \"SCRAM-SHA-1\" => AUTH_SCRAM_SHA_1,\n            \"XOAUTH\" => AUTH_XOAUTH,\n            \"9798-M-DSA-SHA1\" => AUTH_9798_M_DSA_SHA1,\n            \"9798-M-ECDSA-SHA1\" => AUTH_9798_M_ECDSA_SHA1,\n            \"9798-M-RSA-SHA1-ENC\" => AUTH_9798_M_RSA_SHA1_ENC,\n            \"9798-U-DSA-SHA1\" => AUTH_9798_U_DSA_SHA1,\n            \"9798-U-ECDSA-SHA1\" => AUTH_9798_U_ECDSA_SHA1,\n            \"9798-U-RSA-SHA1-ENC\" => AUTH_9798_U_RSA_SHA1_ENC,\n            \"EAP-AES128\" => AUTH_EAP_AES128,\n            \"EAP-AES128-PLUS\" => AUTH_EAP_AES128_PLUS,\n            \"ECDH-X25519-CHALLENGE\" => AUTH_ECDH_X25519_CHALLENGE,\n            \"ECDSA-NIST256P-CHALLENGE\" => AUTH_ECDSA_NIST256P_CHALLENGE,\n            \"EXTERNAL\" => AUTH_EXTERNAL,\n            \"GS2-KRB5\" => AUTH_GS2_KRB5,\n            \"GS2-KRB5-PLUS\" => AUTH_GS2_KRB5_PLUS,\n            \"GSS-SPNEGO\" => AUTH_GSS_SPNEGO,\n            \"GSSAPI\" => AUTH_GSSAPI,\n            \"KERBEROS_V4\" => AUTH_KERBEROS_V4,\n            \"KERBEROS_V5\" => AUTH_KERBEROS_V5,\n            \"NMAS-SAMBA-AUTH\" => AUTH_NMAS_SAMBA_AUTH,\n            \"NMAS_AUTHEN\" => AUTH_NMAS_AUTHEN,\n            \"NMAS_LOGIN\" => AUTH_NMAS_LOGIN,\n            \"NTLM\" => AUTH_NTLM,\n            \"OAUTH10A\" => AUTH_OAUTH10A,\n            \"OPENID20\" => AUTH_OPENID20,\n            \"OTP\" => AUTH_OTP,\n            \"SAML20\" => AUTH_SAML20,\n            \"SECURID\" => AUTH_SECURID,\n            \"SKEY\" => AUTH_SKEY,\n            \"SPNEGO\" => AUTH_SPNEGO,\n            \"SPNEGO-PLUS\" => AUTH_SPNEGO_PLUS,\n            \"SXOVER-PLUS\" => AUTH_SXOVER_PLUS,\n            \"CRAM-MD5\" => AUTH_CRAM_MD5,\n            \"DIGEST-MD5\" => AUTH_DIGEST_MD5,\n            \"ANONYMOUS\" => AUTH_ANONYMOUS,*/\n            _ => return Err(format!(\"Unsupported mechanism {:?}.\", value)),\n        }))\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for Mechanism {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::Integer(value) => Ok(Mechanism(value as u64)),\n            Variable::Array(items) => {\n                let mut mechanism = 0;\n\n                for item in items {\n                    match item {\n                        Variable::Integer(value) => mechanism |= value as u64,\n                        _ => return Err(()),\n                    }\n                }\n\n                Ok(Mechanism(mechanism))\n            }\n            _ => Err(()),\n        }\n    }\n}\n\nimpl From<Mechanism> for Constant {\n    fn from(value: Mechanism) -> Self {\n        Constant::Integer(value.0 as i64)\n    }\n}\n\nimpl ConstantValue for Mechanism {\n    fn add_constants(token_map: &mut crate::expr::tokenizer::TokenMap) {\n        token_map\n            .add_constant(\"login\", Mechanism(AUTH_LOGIN))\n            .add_constant(\"plain\", Mechanism(AUTH_PLAIN))\n            .add_constant(\"xoauth2\", Mechanism(AUTH_XOAUTH2))\n            .add_constant(\"oauthbearer\", Mechanism(AUTH_OAUTHBEARER));\n    }\n}\n\nimpl From<Mechanism> for u64 {\n    fn from(value: Mechanism) -> Self {\n        value.0\n    }\n}\n\nimpl From<u64> for Mechanism {\n    fn from(value: u64) -> Self {\n        Mechanism(value)\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for MtPriority {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::Integer(value) => match value {\n                2 => Ok(MtPriority::Mixer),\n                3 => Ok(MtPriority::Stanag4406),\n                4 => Ok(MtPriority::Nsep),\n                _ => Err(()),\n            },\n            Variable::String(value) => MtPriority::parse_value(value.as_str()).map_err(|_| ()),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl From<MtPriority> for Constant {\n    fn from(value: MtPriority) -> Self {\n        Constant::Integer(match value {\n            MtPriority::Mixer => 2,\n            MtPriority::Stanag4406 => 3,\n            MtPriority::Nsep => 4,\n        })\n    }\n}\n\nimpl ConstantValue for MtPriority {\n    fn add_constants(token_map: &mut TokenMap) {\n        token_map\n            .add_constant(\"mixer\", MtPriority::Mixer)\n            .add_constant(\"stanag4406\", MtPriority::Stanag4406)\n            .add_constant(\"nsep\", MtPriority::Nsep);\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/smtp/throttle.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse utils::config::{Config, Rate, utils::AsKey};\n\nuse crate::expr::{Expression, tokenizer::TokenMap};\n\nuse super::*;\n\npub fn parse_queue_rate_limiter(\n    config: &mut Config,\n    prefix: impl AsKey,\n    token_map: &TokenMap,\n    available_rate_limiter_keys: u16,\n) -> Vec<QueueRateLimiter> {\n    let prefix_ = prefix.as_key();\n    let mut rate_limiters = Vec::new();\n    for rate_limiter_id in config.sub_keys(prefix, \"\") {\n        let rate_limiter_id = rate_limiter_id.as_str();\n        if let Some(rate_limiter) = parse_queue_rate_limiter_item(\n            config,\n            (&prefix_, rate_limiter_id),\n            rate_limiter_id,\n            token_map,\n            available_rate_limiter_keys,\n        ) {\n            rate_limiters.push(rate_limiter);\n        }\n    }\n\n    rate_limiters\n}\n\nfn parse_queue_rate_limiter_item(\n    config: &mut Config,\n    prefix: impl AsKey,\n    rate_limiter_id: &str,\n    token_map: &TokenMap,\n    available_rate_limiter_keys: u16,\n) -> Option<QueueRateLimiter> {\n    let prefix = prefix.as_key();\n\n    // Skip disabled rate_limiters\n    if !config\n        .property::<bool>((prefix.as_str(), \"enable\"))\n        .unwrap_or(true)\n    {\n        return None;\n    }\n\n    let mut keys = 0;\n    for (key_, value) in config\n        .values((&prefix, \"key\"))\n        .map(|(k, v)| (k.to_string(), v.to_string()))\n        .collect::<Vec<_>>()\n    {\n        match parse_queue_rate_limiter_key(&value) {\n            Ok(key) => {\n                if (key & available_rate_limiter_keys) != 0 {\n                    keys |= key;\n                } else {\n                    let err =\n                        format!(\"Rate limiter key {value:?} is not available in this context\");\n                    config.new_build_error(key_, err);\n                }\n            }\n            Err(err) => {\n                config.new_parse_error(key_, err);\n            }\n        }\n    }\n\n    Some(QueueRateLimiter {\n        id: rate_limiter_id.to_string(),\n        expr: Expression::try_parse(config, (prefix.as_str(), \"match\"), token_map)\n            .unwrap_or_default(),\n        keys,\n        rate: config\n            .property_require::<Rate>((prefix.as_str(), \"rate\"))\n            .filter(|r| r.requests > 0)?,\n    })\n}\n\npub(crate) fn parse_queue_rate_limiter_key(value: &str) -> Result<u16, String> {\n    match value {\n        \"rcpt\" => Ok(THROTTLE_RCPT),\n        \"rcpt_domain\" => Ok(THROTTLE_RCPT_DOMAIN),\n        \"sender\" => Ok(THROTTLE_SENDER),\n        \"sender_domain\" => Ok(THROTTLE_SENDER_DOMAIN),\n        \"authenticated_as\" => Ok(THROTTLE_AUTH_AS),\n        \"listener\" => Ok(THROTTLE_LISTENER),\n        \"mx\" => Ok(THROTTLE_MX),\n        \"remote_ip\" => Ok(THROTTLE_REMOTE_IP),\n        \"local_ip\" => Ok(THROTTLE_LOCAL_IP),\n        \"helo_domain\" => Ok(THROTTLE_HELO_DOMAIN),\n        _ => Err(format!(\"Invalid THROTTLE key {value:?}\")),\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/spamfilter.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Variable, functions::ResolveVariable, if_block::IfBlock, tokenizer::TokenMap};\nuse ahash::AHashSet;\nuse mail_auth::common::resolver::ToReverseName;\nuse nlp::classifier::model::{CcfhClassifier, FhClassifier};\nuse std::{\n    net::{IpAddr, SocketAddr},\n    time::Duration,\n};\nuse tokio::net::lookup_host;\nuse utils::{\n    cache::CacheItemWeight,\n    config::{Config, utils::ParseValue},\n    glob::GlobMap,\n};\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]\npub enum SpamClassifier {\n    FhClassifier {\n        classifier: FhClassifier,\n        last_trained_at: u64,\n    },\n    CcfhClassifier {\n        classifier: CcfhClassifier,\n        last_trained_at: u64,\n    },\n    #[default]\n    Disabled,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct SpamFilterConfig {\n    pub enabled: bool,\n    pub card_is_ham: bool,\n    pub trusted_reply: bool,\n    pub grey_list_expiry: Option<u64>,\n\n    pub dnsbl: DnsBlConfig,\n    pub rules: SpamFilterRules,\n    pub lists: SpamFilterLists,\n    pub pyzor: Option<PyzorConfig>,\n    pub classifier: Option<ClassifierConfig>,\n    pub scores: SpamFilterScoreConfig,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct SpamFilterScoreConfig {\n    pub reject_threshold: f32,\n    pub discard_threshold: f32,\n    pub spam_threshold: f32,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct DnsBlConfig {\n    pub max_ip_checks: usize,\n    pub max_domain_checks: usize,\n    pub max_email_checks: usize,\n    pub max_url_checks: usize,\n    pub servers: Vec<DnsBlServer>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct SpamFilterLists {\n    pub file_extensions: GlobMap<FileExtension>,\n    pub scores: GlobMap<SpamFilterAction<f32>>,\n}\n\n#[derive(Debug, Clone)]\npub enum SpamFilterAction<T> {\n    Allow(T),\n    Discard,\n    Reject,\n    Disabled,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct ClassifierConfig {\n    pub w_params: FtrlParameters,\n    pub i_params: Option<FtrlParameters>,\n    pub reservoir_capacity: usize,\n    pub min_ham_samples: u64,\n    pub min_spam_samples: u64,\n    pub auto_learn_reply_ham: bool,\n    pub auto_learn_card_is_ham: bool,\n    pub auto_learn_spam_trap: bool,\n    pub auto_learn_spam_rbl_count: u32,\n    pub hold_samples_for: u64,\n    pub train_frequency: Option<u64>,\n    pub log_scale: bool,\n    pub l2_normalize: bool,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct FtrlParameters {\n    pub feature_hash_size: usize,\n    pub alpha: f64,\n    pub beta: f64,\n    pub l1_ratio: f64,\n    pub l2_ratio: f64,\n}\n\n#[derive(Debug, Clone)]\npub struct PyzorConfig {\n    pub address: SocketAddr,\n    pub timeout: Duration,\n    pub min_count: u64,\n    pub min_wl_count: u64,\n    pub ratio: f64,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\npub struct SpamFilterRules {\n    pub url: Vec<IfBlock>,\n    pub domain: Vec<IfBlock>,\n    pub email: Vec<IfBlock>,\n    pub ip: Vec<IfBlock>,\n    pub header: Vec<IfBlock>,\n    pub body: Vec<IfBlock>,\n    pub any: Vec<IfBlock>,\n}\n\n#[derive(Debug, Clone, Default, PartialEq, Eq)]\npub struct FileExtension {\n    pub known_types: AHashSet<String>,\n    pub is_bad: bool,\n    pub is_archive: bool,\n    pub is_nz: bool,\n}\n\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash)]\npub enum Element {\n    Url,\n    Domain,\n    Email,\n    Ip,\n    Header,\n    Body,\n    #[default]\n    Any,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum Location {\n    EnvelopeFrom,\n    EnvelopeTo,\n    HeaderDkimPass,\n    HeaderReceived,\n    HeaderFrom,\n    HeaderReplyTo,\n    HeaderSubject,\n    HeaderTo,\n    HeaderCc,\n    HeaderBcc,\n    HeaderMid,\n    HeaderDnt,\n    Ehlo,\n    BodyText,\n    BodyHtml,\n    Attachment,\n    Tcp,\n}\n\n#[derive(Debug, Clone)]\npub struct DnsBlServer {\n    pub id: String,\n    pub zone: IfBlock,\n    pub scope: Element,\n    pub tags: IfBlock,\n}\n\nimpl SpamFilterConfig {\n    pub async fn parse(config: &mut Config) -> Self {\n        SpamFilterConfig {\n            enabled: config\n                .property_or_default(\"spam-filter.enable\", \"true\")\n                .unwrap_or(true),\n            card_is_ham: config\n                .property_or_default(\"spam-filter.card-is-ham.enable\", \"true\")\n                .unwrap_or(true),\n            trusted_reply: config\n                .property_or_default(\"spam-filter.trusted-reply.enable\", \"true\")\n                .unwrap_or(true),\n            dnsbl: DnsBlConfig::parse(config),\n            rules: SpamFilterRules::parse(config),\n            lists: SpamFilterLists::parse(config),\n            pyzor: PyzorConfig::parse(config).await,\n            classifier: ClassifierConfig::parse(config),\n            scores: SpamFilterScoreConfig::parse(config),\n            grey_list_expiry: config\n                .property::<Option<Duration>>(\"spam-filter.grey-list.duration\")\n                .unwrap_or_default()\n                .map(|d| d.as_secs()),\n        }\n    }\n}\n\nimpl SpamFilterRules {\n    pub fn parse(config: &mut Config) -> SpamFilterRules {\n        let mut rules = vec![];\n        for id in config.sub_keys(\"spam-filter.rule\", \".scope\") {\n            if let Some(rule) = SpamFilterRule::parse(config, id) {\n                rules.push(rule);\n            }\n        }\n        rules.sort_by(|a, b| a.priority.cmp(&b.priority));\n\n        let mut result = SpamFilterRules::default();\n\n        for rule in rules {\n            match rule.scope {\n                Element::Url => result.url.push(rule.rule),\n                Element::Domain => result.domain.push(rule.rule),\n                Element::Email => result.email.push(rule.rule),\n                Element::Ip => result.ip.push(rule.rule),\n                Element::Header => result.header.push(rule.rule),\n                Element::Body => result.body.push(rule.rule),\n                Element::Any => result.any.push(rule.rule),\n            }\n        }\n\n        result\n    }\n}\n\nstruct SpamFilterRule {\n    rule: IfBlock,\n    priority: i32,\n    scope: Element,\n}\n\nimpl SpamFilterRule {\n    pub fn parse(config: &mut Config, id: String) -> Option<Self> {\n        let id = id.as_str();\n        if !config\n            .property_or_default((\"spam-filter.rule\", id, \"enable\"), \"true\")\n            .unwrap_or(true)\n        {\n            return None;\n        }\n        let priority = config\n            .property_or_default((\"spam-filter.rule\", id, \"priority\"), \"0\")\n            .unwrap_or(0);\n        let scope = config\n            .property_or_default::<Element>((\"spam-filter.rule\", id, \"scope\"), \"any\")\n            .unwrap_or_default();\n\n        SpamFilterRule {\n            rule: IfBlock::try_parse(\n                config,\n                (\"spam-filter.rule\", id, \"condition\"),\n                &scope.token_map(),\n            )?,\n            scope,\n            priority,\n        }\n        .into()\n    }\n}\n\nimpl DnsBlConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut servers = vec![];\n        for id in config.sub_keys(\"spam-filter.dnsbl.server\", \".scope\") {\n            if let Some(server) = DnsBlServer::parse(config, id) {\n                servers.push(server);\n            }\n        }\n\n        DnsBlConfig {\n            max_ip_checks: config\n                .property_or_default(\"spam-filter.dnsbl.max-check.ip\", \"50\")\n                .unwrap_or(20),\n            max_domain_checks: config\n                .property_or_default(\"spam-filter.dnsbl.max-check.domain\", \"50\")\n                .unwrap_or(20),\n            max_email_checks: config\n                .property_or_default(\"spam-filter.dnsbl.max-check.email\", \"50\")\n                .unwrap_or(20),\n            max_url_checks: config\n                .property_or_default(\"spam-filter.dnsbl.max-check.url\", \"50\")\n                .unwrap_or(20),\n            servers,\n        }\n    }\n}\n\nimpl DnsBlServer {\n    pub fn parse(config: &mut Config, id: String) -> Option<Self> {\n        let id_ = id.as_str();\n\n        if !config\n            .property_or_default((\"spam-filter.dnsbl.server\", id_, \"enable\"), \"true\")\n            .unwrap_or(true)\n        {\n            return None;\n        }\n\n        let scope =\n            config.property_require::<Element>((\"spam-filter.dnsbl.server\", id_, \"scope\"))?;\n\n        DnsBlServer {\n            zone: IfBlock::try_parse(\n                config,\n                (\"spam-filter.dnsbl.server\", id_, \"zone\"),\n                &scope.token_map(),\n            )?,\n            scope,\n            tags: IfBlock::try_parse(\n                config,\n                (\"spam-filter.dnsbl.server\", id_, \"tag\"),\n                &Element::Ip.token_map(),\n            )?,\n            id,\n        }\n        .into()\n    }\n}\n\nimpl SpamFilterLists {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut lists = SpamFilterLists {\n            file_extensions: GlobMap::default(),\n            scores: GlobMap::default(),\n        };\n\n        // Parse local lists\n        let mut errors = vec![];\n        for (key, value) in config.iterate_prefix(\"spam-filter.list\") {\n            if let Some((id, key)) = key\n                .split_once('.')\n                .filter(|(id, key)| !id.is_empty() && !key.is_empty())\n            {\n                match id {\n                    \"scores\" => {\n                        let action = match value.to_lowercase().as_str() {\n                            \"reject\" => SpamFilterAction::Reject,\n                            \"discard\" => SpamFilterAction::Discard,\n                            score => match score.parse() {\n                                Ok(score) => SpamFilterAction::Allow(score),\n                                Err(err) => {\n                                    errors.push((\n                                        format!(\"spam-filter.list.{id}.{key}\"),\n                                        format!(\"Invalid score: {}\", err),\n                                    ));\n                                    continue;\n                                }\n                            },\n                        };\n                        lists.scores.insert(key, action);\n                    }\n                    \"file-extensions\" => {\n                        let mut ext = FileExtension::default();\n\n                        for part in value.split('|') {\n                            let part = part.trim();\n                            match part {\n                                \"AR\" => {\n                                    ext.is_archive = true;\n                                }\n                                \"NZ\" => {\n                                    ext.is_nz = true;\n                                }\n                                \"BAD\" => {\n                                    ext.is_bad = true;\n                                }\n                                other => {\n                                    if other.contains('/') {\n                                        ext.known_types.insert(other.to_string());\n                                    } else if !other.is_empty() {\n                                        errors.push((\n                                            format!(\"spam-filter.list.{id}.{key}\"),\n                                            format!(\"Invalid file extension: {}\", other),\n                                        ));\n                                    }\n                                }\n                            }\n                        }\n\n                        lists.file_extensions.insert(key, ext);\n                    }\n                    _ => (),\n                }\n            }\n        }\n\n        for (key, error) in errors {\n            config.new_parse_error(key, error);\n        }\n\n        lists\n    }\n}\n\nimpl PyzorConfig {\n    pub async fn parse(config: &mut Config) -> Option<Self> {\n        if !config\n            .property_or_default(\"spam-filter.pyzor.enable\", \"true\")\n            .unwrap_or(true)\n        {\n            return None;\n        }\n\n        let port = config\n            .property_or_default::<u16>(\"spam-filter.pyzor.port\", \"24441\")\n            .unwrap_or(24441);\n        let host = config\n            .value(\"spam-filter.pyzor.host\")\n            .unwrap_or(\"public.pyzor.org\");\n        let address = match lookup_host(format!(\"{host}:{port}\"))\n            .await\n            .map(|mut a| a.next())\n        {\n            Ok(Some(address)) => address,\n            Ok(None) => {\n                config.new_build_error(\n                    \"spam-filter.pyzor.host\",\n                    \"Invalid address: No addresses found.\",\n                );\n                return None;\n            }\n            Err(err) => {\n                config.new_build_error(\n                    \"spam-filter.pyzor.host\",\n                    format!(\"Invalid address: {}\", err),\n                );\n                return None;\n            }\n        };\n\n        PyzorConfig {\n            address,\n            timeout: config\n                .property_or_default::<Duration>(\"spam-filter.pyzor.timeout\", \"5s\")\n                .unwrap_or(Duration::from_secs(5)),\n            min_count: config\n                .property_or_default(\"spam-filter.pyzor.count\", \"5\")\n                .unwrap_or(5),\n            min_wl_count: config\n                .property_or_default(\"spam-filter.pyzor.wl-count\", \"10\")\n                .unwrap_or(10),\n            ratio: config\n                .property_or_default(\"spam-filter.pyzor.ratio\", \"0.2\")\n                .unwrap_or(0.2),\n        }\n        .into()\n    }\n}\n\nimpl ClassifierConfig {\n    pub fn parse(config: &mut Config) -> Option<Self> {\n        let ccfh = match config.value(\"spam-filter.classifier.model\") {\n            Some(\"ftrl-fh\") | None => false,\n            Some(\"ftrl-ccfh\") => true,\n            Some(\"disabled\" | \"disable\") => return None,\n            Some(other) => {\n                config.new_build_error(\n                    \"spam-filter.classifier.model\",\n                    format!(\"Invalid model type: {}\", other),\n                );\n                return None;\n            }\n        };\n\n        let w_params = FtrlParameters::parse(config, \"spam-filter.classifier.parameters\", 20);\n        let i_params = if ccfh {\n            Some(FtrlParameters::parse(\n                config,\n                \"spam-filter.classifier.parameters.ccfh\",\n                w_params.feature_hash_size - 2,\n            ))\n        } else {\n            None\n        };\n\n        ClassifierConfig {\n            w_params,\n            i_params,\n            reservoir_capacity: config\n                .property_or_default(\"spam-filter.classifier.samples.reservoir-capacity\", \"1024\")\n                .unwrap_or(1024),\n            auto_learn_card_is_ham: config\n                .property_or_default(\"spam-filter.card-is-ham.learn\", \"true\")\n                .unwrap_or(true),\n            auto_learn_reply_ham: config\n                .property_or_default(\"spam-filter.trusted-reply.learn\", \"true\")\n                .unwrap_or(true),\n            auto_learn_spam_trap: config\n                .property_or_default(\"spam-filter.classifier.auto-learn.spam-trap\", \"true\")\n                .unwrap_or(true),\n            auto_learn_spam_rbl_count: config\n                .property_or_default(\"spam-filter.classifier.auto-learn.spam-rbl-count\", \"2\")\n                .unwrap_or(2),\n            hold_samples_for: config\n                .property_or_default::<Duration>(\"spam-filter.classifier.samples.hold-for\", \"180d\")\n                .unwrap_or(Duration::from_secs(180 * 24 * 60 * 60))\n                .as_secs(),\n            min_ham_samples: config\n                .property_or_default(\"spam-filter.classifier.samples.min-ham\", \"100\")\n                .unwrap_or(100),\n            min_spam_samples: config\n                .property_or_default(\"spam-filter.classifier.samples.min-spam\", \"100\")\n                .unwrap_or(100),\n            train_frequency: config\n                .property_or_default::<Option<Duration>>(\n                    \"spam-filter.classifier.training.frequency\",\n                    \"12h\",\n                )\n                .unwrap_or(Some(Duration::from_secs(12 * 60 * 60)))\n                .map(|d| d.as_secs()),\n            log_scale: config\n                .property_or_default(\"spam-filter.classifier.features.log-scale\", \"true\")\n                .unwrap_or(true),\n            l2_normalize: config\n                .property_or_default(\"spam-filter.classifier.features.l2-normalize\", \"true\")\n                .unwrap_or(true),\n        }\n        .into()\n    }\n}\n\nimpl FtrlParameters {\n    pub fn parse(config: &mut Config, prefix: &str, default_features: usize) -> Self {\n        let feature_hash_size: usize = config\n            .property((prefix, \"features\"))\n            .unwrap_or(default_features);\n\n        if !(16..=28).contains(&feature_hash_size) {\n            config.new_build_error(\n                (prefix, \"features\"),\n                \"Feature size must be between 2^16 and 2^28.\",\n            );\n        }\n\n        FtrlParameters {\n            feature_hash_size: 1 << feature_hash_size,\n            alpha: config\n                .property_or_default((prefix, \"alpha\"), \"2.0\")\n                .unwrap_or(2.0),\n            beta: config\n                .property_or_default((prefix, \"beta\"), \"1.0\")\n                .unwrap_or(1.0),\n            l1_ratio: config\n                .property_or_default((prefix, \"l1\"), \"0.001\")\n                .unwrap_or(0.001),\n            l2_ratio: config\n                .property_or_default((prefix, \"l2\"), \"0.0001\")\n                .unwrap_or(0.0001),\n        }\n    }\n}\n\nimpl SpamClassifier {\n    pub fn is_active(&self) -> bool {\n        !matches!(self, SpamClassifier::Disabled)\n    }\n}\n\nimpl SpamFilterScoreConfig {\n    pub fn parse(config: &mut Config) -> Self {\n        SpamFilterScoreConfig {\n            reject_threshold: config\n                .property(\"spam-filter.score.reject\")\n                .unwrap_or_default(),\n            discard_threshold: config\n                .property(\"spam-filter.score.discard\")\n                .unwrap_or_default(),\n            spam_threshold: config\n                .property_or_default(\"spam-filter.score.spam\", \"5.0\")\n                .unwrap_or(5.0),\n        }\n    }\n}\n\nimpl ParseValue for Element {\n    fn parse_value(value: &str) -> utils::config::Result<Self> {\n        match value {\n            \"url\" => Ok(Element::Url),\n            \"domain\" => Ok(Element::Domain),\n            \"email\" => Ok(Element::Email),\n            \"ip\" => Ok(Element::Ip),\n            \"header\" => Ok(Element::Header),\n            \"body\" => Ok(Element::Body),\n            \"any\" | \"message\" => Ok(Element::Any),\n            other => Err(format!(\"Invalid type {other:?}.\",)),\n        }\n    }\n}\n\nimpl Location {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Location::EnvelopeFrom => \"env_from\",\n            Location::EnvelopeTo => \"env_to\",\n            Location::HeaderDkimPass => \"dkim_pass\",\n            Location::HeaderReceived => \"received\",\n            Location::HeaderFrom => \"from\",\n            Location::HeaderReplyTo => \"reply_to\",\n            Location::HeaderSubject => \"subject\",\n            Location::HeaderTo => \"to\",\n            Location::HeaderCc => \"cc\",\n            Location::HeaderBcc => \"bcc\",\n            Location::HeaderMid => \"message_id\",\n            Location::HeaderDnt => \"dnt\",\n            Location::Ehlo => \"ehlo\",\n            Location::BodyText => \"body_text\",\n            Location::BodyHtml => \"body_html\",\n            Location::Attachment => \"attachment\",\n            Location::Tcp => \"tcp\",\n        }\n    }\n}\n\npub const V_SPAM_REMOTE_IP: u32 = 100;\npub const V_SPAM_REMOTE_IP_PTR: u32 = 101;\npub const V_SPAM_EHLO_DOMAIN: u32 = 102;\npub const V_SPAM_AUTH_AS: u32 = 103;\npub const V_SPAM_ASN: u32 = 104;\npub const V_SPAM_COUNTRY: u32 = 105;\npub const V_SPAM_IS_TLS: u32 = 106;\npub const V_SPAM_ENV_FROM: u32 = 108;\npub const V_SPAM_ENV_FROM_LOCAL: u32 = 109;\npub const V_SPAM_ENV_FROM_DOMAIN: u32 = 110;\npub const V_SPAM_ENV_TO: u32 = 111;\npub const V_SPAM_FROM: u32 = 112;\npub const V_SPAM_FROM_NAME: u32 = 113;\npub const V_SPAM_FROM_LOCAL: u32 = 114;\npub const V_SPAM_FROM_DOMAIN: u32 = 115;\npub const V_SPAM_REPLY_TO: u32 = 116;\npub const V_SPAM_REPLY_TO_NAME: u32 = 117;\npub const V_SPAM_REPLY_TO_LOCAL: u32 = 118;\npub const V_SPAM_REPLY_TO_DOMAIN: u32 = 119;\npub const V_SPAM_TO: u32 = 120;\npub const V_SPAM_TO_NAME: u32 = 121;\npub const V_SPAM_TO_LOCAL: u32 = 122;\npub const V_SPAM_TO_DOMAIN: u32 = 123;\npub const V_SPAM_CC: u32 = 124;\npub const V_SPAM_CC_NAME: u32 = 125;\npub const V_SPAM_CC_LOCAL: u32 = 126;\npub const V_SPAM_CC_DOMAIN: u32 = 127;\npub const V_SPAM_BCC: u32 = 128;\npub const V_SPAM_BCC_NAME: u32 = 129;\npub const V_SPAM_BCC_LOCAL: u32 = 130;\npub const V_SPAM_BCC_DOMAIN: u32 = 131;\npub const V_SPAM_BODY_TEXT: u32 = 132;\npub const V_SPAM_BODY_HTML: u32 = 133;\npub const V_SPAM_BODY_RAW: u32 = 134;\npub const V_SPAM_SUBJECT: u32 = 135;\npub const V_SPAM_SUBJECT_THREAD: u32 = 136;\npub const V_SPAM_LOCATION: u32 = 137;\npub const V_WORDS_SUBJECT: u32 = 138;\npub const V_WORDS_BODY: u32 = 139;\n\npub const V_RCPT_EMAIL: u32 = 0;\npub const V_RCPT_NAME: u32 = 1;\npub const V_RCPT_LOCAL: u32 = 2;\npub const V_RCPT_DOMAIN: u32 = 3;\npub const V_RCPT_DOMAIN_SLD: u32 = 4;\n\npub const V_URL_FULL: u32 = 0;\npub const V_URL_PATH_QUERY: u32 = 1;\npub const V_URL_PATH: u32 = 2;\npub const V_URL_QUERY: u32 = 3;\npub const V_URL_SCHEME: u32 = 4;\npub const V_URL_AUTHORITY: u32 = 5;\npub const V_URL_HOST: u32 = 6;\npub const V_URL_HOST_SLD: u32 = 7;\npub const V_URL_PORT: u32 = 8;\n\npub const V_HEADER_NAME: u32 = 0;\npub const V_HEADER_NAME_LOWER: u32 = 1;\npub const V_HEADER_VALUE: u32 = 2;\npub const V_HEADER_VALUE_LOWER: u32 = 3;\npub const V_HEADER_PROPERTY: u32 = 4;\npub const V_HEADER_RAW: u32 = 5;\npub const V_HEADER_RAW_LOWER: u32 = 6;\n\npub const V_IP: u32 = 0;\npub const V_IP_REVERSE: u32 = 1;\npub const V_IP_OCTETS: u32 = 2;\npub const V_IP_IS_V4: u32 = 3;\npub const V_IP_IS_V6: u32 = 4;\n\nimpl Element {\n    pub fn token_map(&self) -> TokenMap {\n        let map = TokenMap::default().with_variables_map([\n            (\"remote_ip\", V_SPAM_REMOTE_IP),\n            (\"remote_ip.ptr\", V_SPAM_REMOTE_IP_PTR),\n            (\"ehlo_domain\", V_SPAM_EHLO_DOMAIN),\n            (\"auth_as\", V_SPAM_AUTH_AS),\n            (\"asn\", V_SPAM_ASN),\n            (\"country\", V_SPAM_COUNTRY),\n            (\"is_tls\", V_SPAM_IS_TLS),\n            (\"env_from\", V_SPAM_ENV_FROM),\n            (\"env_from.local\", V_SPAM_ENV_FROM_LOCAL),\n            (\"env_from.domain\", V_SPAM_ENV_FROM_DOMAIN),\n            (\"env_to\", V_SPAM_ENV_TO),\n            (\"from\", V_SPAM_FROM),\n            (\"from.name\", V_SPAM_FROM_NAME),\n            (\"from.local\", V_SPAM_FROM_LOCAL),\n            (\"from.domain\", V_SPAM_FROM_DOMAIN),\n            (\"reply_to\", V_SPAM_REPLY_TO),\n            (\"reply_to.name\", V_SPAM_REPLY_TO_NAME),\n            (\"reply_to.local\", V_SPAM_REPLY_TO_LOCAL),\n            (\"reply_to.domain\", V_SPAM_REPLY_TO_DOMAIN),\n            (\"to\", V_SPAM_TO),\n            (\"to.name\", V_SPAM_TO_NAME),\n            (\"to.local\", V_SPAM_TO_LOCAL),\n            (\"to.domain\", V_SPAM_TO_DOMAIN),\n            (\"cc\", V_SPAM_CC),\n            (\"cc.name\", V_SPAM_CC_NAME),\n            (\"cc.local\", V_SPAM_CC_LOCAL),\n            (\"cc.domain\", V_SPAM_CC_DOMAIN),\n            (\"bcc\", V_SPAM_BCC),\n            (\"bcc.name\", V_SPAM_BCC_NAME),\n            (\"bcc.local\", V_SPAM_BCC_LOCAL),\n            (\"bcc.domain\", V_SPAM_BCC_DOMAIN),\n            (\"body\", V_SPAM_BODY_TEXT),\n            (\"body.text\", V_SPAM_BODY_TEXT),\n            (\"body.html\", V_SPAM_BODY_HTML),\n            (\"body.words\", V_WORDS_BODY),\n            (\"body.raw\", V_SPAM_BODY_RAW),\n            (\"subject\", V_SPAM_SUBJECT),\n            (\"subject.thread\", V_SPAM_SUBJECT_THREAD),\n            (\"subject.words\", V_WORDS_SUBJECT),\n            (\"location\", V_SPAM_LOCATION),\n        ]);\n\n        match self {\n            Element::Url => map.with_variables_map([\n                (\"url\", V_URL_FULL),\n                (\"value\", V_URL_FULL),\n                (\"path_query\", V_URL_PATH_QUERY),\n                (\"path\", V_URL_PATH),\n                (\"query\", V_URL_QUERY),\n                (\"scheme\", V_URL_SCHEME),\n                (\"authority\", V_URL_AUTHORITY),\n                (\"host\", V_URL_HOST),\n                (\"sld\", V_URL_HOST_SLD),\n                (\"port\", V_URL_PORT),\n            ]),\n            Element::Email => map.with_variables_map([\n                (\"email\", V_RCPT_EMAIL),\n                (\"value\", V_RCPT_EMAIL),\n                (\"name\", V_RCPT_NAME),\n                (\"local\", V_RCPT_LOCAL),\n                (\"domain\", V_RCPT_DOMAIN),\n                (\"sld\", V_RCPT_DOMAIN_SLD),\n            ]),\n            Element::Ip => map.with_variables_map([\n                (\"ip\", V_IP),\n                (\"value\", V_IP),\n                (\"input\", V_IP),\n                (\"reverse_ip\", V_IP_REVERSE),\n                (\"ip_reverse\", V_IP_REVERSE),\n                (\"octets\", V_IP_OCTETS),\n                (\"is_v4\", V_IP_IS_V4),\n                (\"is_v6\", V_IP_IS_V6),\n            ]),\n            Element::Header => map.with_variables_map([\n                (\"name\", V_HEADER_NAME),\n                (\"name_lower\", V_HEADER_NAME_LOWER),\n                (\"value\", V_HEADER_VALUE),\n                (\"value_lower\", V_HEADER_VALUE_LOWER),\n                (\"email\", V_HEADER_VALUE),\n                (\"email_lower\", V_HEADER_VALUE_LOWER),\n                (\"attributes\", V_HEADER_PROPERTY),\n                (\"raw\", V_HEADER_RAW),\n                (\"raw_lower\", V_HEADER_RAW_LOWER),\n            ]),\n            Element::Body | Element::Domain => {\n                map.with_variables_map([(\"input\", 0), (\"value\", 0), (\"result\", 0)])\n            }\n            Element::Any => map,\n        }\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Element::Url => \"url\",\n            Element::Domain => \"domain\",\n            Element::Email => \"email\",\n            Element::Ip => \"ip\",\n            Element::Header => \"header\",\n            Element::Body => \"body\",\n            Element::Any => \"any\",\n        }\n    }\n}\n\npub struct IpResolver {\n    ip: IpAddr,\n    ip_string: String,\n    reverse: String,\n    octets: Variable<'static>,\n}\n\nimpl ResolveVariable for IpResolver {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        match variable {\n            V_IP => self.ip_string.as_str().into(),\n            V_IP_REVERSE => self.reverse.as_str().into(),\n            V_IP_OCTETS => self.octets.clone(),\n            V_IP_IS_V4 => Variable::Integer(self.ip.is_ipv4() as _),\n            V_IP_IS_V6 => Variable::Integer(self.ip.is_ipv6() as _),\n            _ => Variable::Integer(0),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl IpResolver {\n    pub fn new(ip: IpAddr) -> Self {\n        Self {\n            ip_string: ip.to_string(),\n            reverse: ip.to_reverse_name(),\n            octets: Variable::Array(match ip {\n                IpAddr::V4(ipv4_addr) => ipv4_addr\n                    .octets()\n                    .iter()\n                    .map(|o| Variable::Integer(*o as _))\n                    .collect(),\n                IpAddr::V6(ipv6_addr) => ipv6_addr\n                    .octets()\n                    .iter()\n                    .map(|o| Variable::Integer(*o as _))\n                    .collect(),\n            }),\n            ip,\n        }\n    }\n}\n\nimpl CacheItemWeight for IpResolver {\n    fn weight(&self) -> u64 {\n        (std::mem::size_of::<IpResolver>() + self.ip_string.len() + self.reverse.len()) as u64\n    }\n}\n\nimpl<T> SpamFilterAction<T> {\n    pub fn as_score(&self) -> Option<&T> {\n        match self {\n            SpamFilterAction::Allow(value) => Some(value),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/config/storage.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse ahash::AHashMap;\nuse directory::Directory;\nuse store::{BlobStore, SearchStore, InMemoryStore, PubSubStore, PurgeSchedule, Store};\n\nuse crate::manager::config::ConfigManager;\n\n#[derive(Default, Clone)]\npub struct Storage {\n    pub data: Store,\n    pub blob: BlobStore,\n    pub fts: SearchStore,\n    pub lookup: InMemoryStore,\n    pub pubsub: PubSubStore,\n    pub directory: Arc<Directory>,\n    pub directories: AHashMap<String, Arc<Directory>>,\n    pub purge_schedules: Vec<PurgeSchedule>,\n    pub config: ConfigManager,\n\n    pub stores: AHashMap<String, Store>,\n    pub blobs: AHashMap<String, BlobStore>,\n    pub lookups: AHashMap<String, InMemoryStore>,\n    pub ftss: AHashMap<String, SearchStore>,\n}\n"
  },
  {
    "path": "crates/common/src/config/telemetry.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::{AHashMap, AHashSet};\nuse base64::{Engine, engine::general_purpose::STANDARD};\nuse hyper::{HeaderMap, header::CONTENT_TYPE};\nuse opentelemetry::{InstrumentationScope, KeyValue, logs::LoggerProvider};\nuse opentelemetry_otlp::{\n    LogExporter, MetricExporter, SpanExporter, WithExportConfig, WithHttpConfig,\n};\nuse opentelemetry_sdk::{\n    Resource,\n    logs::{SdkLogger, SdkLoggerProvider},\n    metrics::Temporality,\n};\nuse opentelemetry_semantic_conventions::resource::SERVICE_VERSION;\nuse std::{collections::HashMap, str::FromStr, sync::Arc, time::Duration};\nuse store::Stores;\nuse trc::{EventType, Level, TelemetryEvent, ipc::subscriber::Interests};\nuse utils::config::{Config, http::parse_http_headers, utils::ParseValue};\n\n#[derive(Debug)]\npub struct TelemetrySubscriber {\n    pub id: String,\n    pub interests: Interests,\n    pub typ: TelemetrySubscriberType,\n    pub lossy: bool,\n}\n\n#[allow(clippy::large_enum_variant)]\n#[derive(Debug)]\npub enum TelemetrySubscriberType {\n    ConsoleTracer(ConsoleTracer),\n    LogTracer(LogTracer),\n    OtelTracer(OtelTracer),\n    Webhook(WebhookTracer),\n    #[cfg(unix)]\n    JournalTracer(crate::telemetry::tracers::journald::Subscriber),\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    StoreTracer(StoreTracer),\n    // SPDX-SnippetEnd\n}\n\n#[derive(Debug)]\npub struct OtelTracer {\n    pub span_exporter: SpanExporter,\n    pub span_exporter_enable: bool,\n    pub log_exporter: LogExporter,\n    pub log_provider: SdkLogger,\n    pub log_exporter_enable: bool,\n    pub throttle: Duration,\n}\n\npub struct OtelMetrics {\n    pub resource: Resource,\n    pub instrumentation: InstrumentationScope,\n    pub exporter: MetricExporter,\n    pub interval: Duration,\n}\n\n#[derive(Debug)]\npub struct ConsoleTracer {\n    pub ansi: bool,\n    pub multiline: bool,\n    pub buffered: bool,\n}\n\n#[derive(Debug)]\npub struct LogTracer {\n    pub path: String,\n    pub prefix: String,\n    pub rotate: RotationStrategy,\n    pub ansi: bool,\n    pub multiline: bool,\n}\n\n#[derive(Debug)]\npub struct WebhookTracer {\n    pub url: String,\n    pub key: String,\n    pub timeout: Duration,\n    pub throttle: Duration,\n    pub discard_after: Duration,\n    pub tls_allow_invalid_certs: bool,\n    pub headers: HeaderMap,\n}\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[derive(Debug)]\n#[cfg(feature = \"enterprise\")]\npub struct StoreTracer {\n    pub store: store::Store,\n}\n// SPDX-SnippetEnd\n\n#[derive(Debug)]\npub enum RotationStrategy {\n    Daily,\n    Hourly,\n    Minutely,\n    Never,\n}\n\n#[derive(Debug)]\npub struct Telemetry {\n    pub tracers: Tracers,\n    pub metrics: Interests,\n}\n\n#[derive(Debug)]\npub struct Tracers {\n    pub interests: Interests,\n    pub levels: AHashMap<EventType, Level>,\n    pub subscribers: Vec<TelemetrySubscriber>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct Metrics {\n    pub prometheus: Option<PrometheusMetrics>,\n    pub otel: Option<Arc<OtelMetrics>>,\n    pub log_path: Option<String>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct PrometheusMetrics {\n    pub auth: Option<String>,\n}\n\nimpl Telemetry {\n    pub fn parse(config: &mut Config, stores: &Stores) -> Self {\n        let mut telemetry = Telemetry {\n            tracers: Tracers::parse(config, stores),\n            metrics: Interests::default(),\n        };\n\n        // Parse metrics\n        apply_events(\n            config\n                .properties::<EventOrMany>(\"metrics.disabled-events\")\n                .into_iter()\n                .map(|(_, e)| e),\n            false,\n            |event_type| {\n                if event_type.is_metric() {\n                    telemetry.metrics.set(event_type);\n                }\n            },\n        );\n\n        telemetry\n    }\n}\n\nimpl Tracers {\n    pub fn parse(config: &mut Config, stores: &Stores) -> Self {\n        // Parse custom logging levels\n        let mut custom_levels = AHashMap::new();\n        for event_name in config\n            .prefix(\"tracing.level\")\n            .map(|s| s.to_string())\n            .collect::<Vec<_>>()\n        {\n            if let Some(event_type) =\n                config.try_parse_value::<EventType>((\"tracing.level\", &event_name), &event_name)\n                && let Some(level) =\n                    config.property_require::<Level>((\"tracing.level\", &event_name))\n            {\n                custom_levels.insert(event_type, level);\n            }\n        }\n\n        // Parse tracers\n        let mut tracers: Vec<TelemetrySubscriber> = Vec::new();\n        let mut global_interests = Interests::default();\n        for tracer_id in config.sub_keys(\"tracer\", \".type\") {\n            let id = tracer_id.as_str();\n\n            // Skip disabled tracers\n            if !config\n                .property::<bool>((\"tracer\", id, \"enable\"))\n                .unwrap_or(true)\n            {\n                continue;\n            }\n\n            // Parse tracer\n            let typ = match config\n                .value((\"tracer\", id, \"type\"))\n                .unwrap_or_default()\n                .to_string()\n                .as_str()\n            {\n                \"log\" => {\n                    if let Some(path) = config\n                        .value_require((\"tracer\", id, \"path\"))\n                        .map(|s| s.to_string())\n                    {\n                        TelemetrySubscriberType::LogTracer(LogTracer {\n                            path,\n                            prefix: config\n                                .value((\"tracer\", id, \"prefix\"))\n                                .unwrap_or(\"stalwart\")\n                                .to_string(),\n                            rotate: match config.value((\"tracer\", id, \"rotate\")).unwrap_or(\"daily\")\n                            {\n                                \"daily\" => RotationStrategy::Daily,\n                                \"hourly\" => RotationStrategy::Hourly,\n                                \"minutely\" => RotationStrategy::Minutely,\n                                \"never\" => RotationStrategy::Never,\n                                rotate => {\n                                    let err = format!(\"Invalid rotation strategy: {rotate}\");\n                                    config.new_parse_error((\"tracer\", id, \"rotate\"), err);\n                                    RotationStrategy::Daily\n                                }\n                            },\n                            ansi: config\n                                .property_or_default((\"tracer\", id, \"ansi\"), \"false\")\n                                .unwrap_or(false),\n                            multiline: config\n                                .property_or_default((\"tracer\", id, \"multiline\"), \"false\")\n                                .unwrap_or(false),\n                        })\n                    } else {\n                        continue;\n                    }\n                }\n                \"console\" | \"stdout\" | \"stderr\" => {\n                    if !tracers\n                        .iter()\n                        .any(|t| matches!(t.typ, TelemetrySubscriberType::ConsoleTracer(_)))\n                    {\n                        TelemetrySubscriberType::ConsoleTracer(ConsoleTracer {\n                            ansi: config\n                                .property_or_default((\"tracer\", id, \"ansi\"), \"true\")\n                                .unwrap_or(true),\n                            multiline: config\n                                .property_or_default((\"tracer\", id, \"multiline\"), \"false\")\n                                .unwrap_or(false),\n                            buffered: config\n                                .property_or_default((\"tracer\", id, \"buffered\"), \"true\")\n                                .unwrap_or(true),\n                        })\n                    } else {\n                        config.new_build_error(\n                            (\"tracer\", id, \"type\"),\n                            \"Only one console tracer is allowed\".to_string(),\n                        );\n                        continue;\n                    }\n                }\n                \"otel\" | \"open-telemetry\" => {\n                    let timeout = config\n                        .property::<Duration>((\"tracer\", id, \"timeout\"))\n                        .unwrap_or(opentelemetry_otlp::OTEL_EXPORTER_OTLP_TIMEOUT_DEFAULT);\n                    let throttle = config\n                        .property_or_default((\"tracer\", id, \"throttle\"), \"1s\")\n                        .unwrap_or_else(|| Duration::from_secs(1));\n                    let log_exporter_enable = config\n                        .property_or_default((\"tracer\", id, \"enable.log-exporter\"), \"true\")\n                        .unwrap_or(true);\n                    let span_exporter_enable = config\n                        .property_or_default((\"tracer\", id, \"enable.span-exporter\"), \"true\")\n                        .unwrap_or(true);\n\n                    match config\n                        .value_require((\"tracer\", id, \"transport\"))\n                        .unwrap_or_default()\n                    {\n                        \"grpc\" => {\n                            let mut span_exporter = SpanExporter::builder()\n                                .with_tonic()\n                                .with_protocol(opentelemetry_otlp::Protocol::Grpc)\n                                .with_timeout(timeout);\n                            let mut log_exporter = LogExporter::builder()\n                                .with_tonic()\n                                .with_protocol(opentelemetry_otlp::Protocol::Grpc)\n                                .with_timeout(timeout);\n                            if let Some(endpoint) = config.value((\"tracer\", id, \"endpoint\")) {\n                                span_exporter = span_exporter.with_endpoint(endpoint);\n                                log_exporter = log_exporter.with_endpoint(endpoint);\n                            }\n\n                            match (span_exporter.build(), log_exporter.build()) {\n                                (Ok(span_exporter), Ok(log_exporter)) => {\n                                    TelemetrySubscriberType::OtelTracer(OtelTracer {\n                                        span_exporter,\n                                        log_exporter,\n                                        throttle,\n                                        span_exporter_enable,\n                                        log_exporter_enable,\n                                        log_provider: SdkLoggerProvider::builder()\n                                            .build()\n                                            .logger(\"stalwart\"),\n                                    })\n                                }\n                                (Err(err), _) => {\n                                    config.new_build_error(\n                                        (\"tracer\", id),\n                                        format!(\n                                            \"Failed to build OpenTelemetry span exporter: {err}\"\n                                        ),\n                                    );\n                                    continue;\n                                }\n                                (_, Err(err)) => {\n                                    config.new_build_error(\n                                        (\"tracer\", id),\n                                        format!(\n                                            \"Failed to build OpenTelemetry log exporter: {err}\"\n                                        ),\n                                    );\n                                    continue;\n                                }\n                            }\n                        }\n                        \"http\" => {\n                            if let Some(endpoint) = config\n                                .value_require((\"tracer\", id, \"endpoint\"))\n                                .map(|s| s.to_string())\n                            {\n                                let mut headers = HashMap::new();\n                                let mut err = None;\n                                for (_, value) in config.values((\"tracer\", id, \"headers\")) {\n                                    if let Some((key, value)) = value.split_once(':') {\n                                        headers.insert(\n                                            key.trim().to_string(),\n                                            value.trim().to_string(),\n                                        );\n                                    } else {\n                                        err = format!(\"Invalid open-telemetry header {value:?}\")\n                                            .into();\n                                        break;\n                                    }\n                                }\n                                if let Some(err) = err {\n                                    config.new_parse_error((\"tracer\", id, \"headers\"), err);\n                                }\n\n                                let mut span_exporter = SpanExporter::builder()\n                                    .with_http()\n                                    .with_endpoint(&endpoint)\n                                    .with_timeout(timeout);\n                                let mut log_exporter = LogExporter::builder()\n                                    .with_http()\n                                    .with_endpoint(&endpoint)\n                                    .with_timeout(timeout);\n                                if !headers.is_empty() {\n                                    span_exporter = span_exporter.with_headers(headers.clone());\n                                    log_exporter = log_exporter.with_headers(headers);\n                                }\n\n                                match (span_exporter.build(), log_exporter.build()) {\n                                    (Ok(span_exporter), Ok(log_exporter)) => {\n                                        TelemetrySubscriberType::OtelTracer(OtelTracer {\n                                            span_exporter,\n                                            log_exporter,\n                                            throttle,\n                                            span_exporter_enable,\n                                            log_exporter_enable,\n                                            log_provider: SdkLoggerProvider::builder()\n                                                .build()\n                                                .logger(\"stalwart\"),\n                                        })\n                                    }\n                                    (Err(err), _) => {\n                                        config.new_build_error(\n                                            (\"tracer\", id),\n                                            format!(\n                                                \"Failed to build OpenTelemetry span exporter: {err}\"\n                                            ),\n                                        );\n                                        continue;\n                                    }\n                                    (_, Err(err)) => {\n                                        config.new_build_error(\n                                            (\"tracer\", id),\n                                            format!(\n                                                \"Failed to build OpenTelemetry log exporter: {err}\"\n                                            ),\n                                        );\n                                        continue;\n                                    }\n                                }\n                            } else {\n                                continue;\n                            }\n                        }\n                        transport => {\n                            let err = format!(\"Invalid transport: {transport}\");\n                            config.new_parse_error((\"tracer\", id, \"transport\"), err);\n                            continue;\n                        }\n                    }\n                }\n                \"journal\" => {\n                    #[cfg(unix)]\n                    {\n                        if !tracers\n                            .iter()\n                            .any(|t| matches!(t.typ, TelemetrySubscriberType::JournalTracer(_)))\n                        {\n                            match crate::telemetry::tracers::journald::Subscriber::new() {\n                                Ok(subscriber) => {\n                                    TelemetrySubscriberType::JournalTracer(subscriber)\n                                }\n                                Err(e) => {\n                                    config.new_build_error(\n                                        (\"tracer\", id, \"type\"),\n                                        format!(\"Failed to create journald subscriber: {e}\"),\n                                    );\n                                    continue;\n                                }\n                            }\n                        } else {\n                            config.new_build_error(\n                                (\"tracer\", id, \"type\"),\n                                \"Only one journal tracer is allowed\".to_string(),\n                            );\n                            continue;\n                        }\n                    }\n\n                    #[cfg(not(unix))]\n                    {\n                        config.new_build_error(\n                            (\"tracer\", id, \"type\"),\n                            \"Journald is only available on Unix systems.\",\n                        );\n                        continue;\n                    }\n                }\n                unknown => {\n                    config.new_parse_error(\n                        (\"tracer\", id, \"type\"),\n                        format!(\"Unknown tracer type: {unknown}\"),\n                    );\n                    continue;\n                }\n            };\n\n            // Create tracer\n            let mut tracer = TelemetrySubscriber {\n                id: format!(\"t_{id}\"),\n                interests: Default::default(),\n                lossy: config\n                    .property_or_default((\"tracer\", id, \"lossy\"), \"false\")\n                    .unwrap_or(false),\n                typ,\n            };\n\n            // Parse level\n            let level = Level::from_str(config.value((\"tracer\", id, \"level\")).unwrap_or(\"info\"))\n                .map_err(|err| {\n                    config.new_parse_error(\n                        (\"tracer\", id, \"level\"),\n                        format!(\"Invalid log level: {err}\"),\n                    )\n                })\n                .unwrap_or(Level::Info);\n\n            // Parse disabled events\n            let exclude_event = match &tracer.typ {\n                TelemetrySubscriberType::ConsoleTracer(_) => None,\n                TelemetrySubscriberType::LogTracer(_) => {\n                    EventType::Telemetry(TelemetryEvent::LogError).into()\n                }\n                TelemetrySubscriberType::OtelTracer(_) => {\n                    EventType::Telemetry(TelemetryEvent::OtelExporterError).into()\n                }\n                TelemetrySubscriberType::Webhook(_) => {\n                    EventType::Telemetry(TelemetryEvent::WebhookError).into()\n                }\n                #[cfg(unix)]\n                TelemetrySubscriberType::JournalTracer(_) => {\n                    EventType::Telemetry(TelemetryEvent::JournalError).into()\n                }\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                TelemetrySubscriberType::StoreTracer(_) => None,\n                // SPDX-SnippetEnd\n            };\n\n            // Parse disabled events\n            apply_events(\n                config\n                    .properties::<EventOrMany>((\"tracer\", id, \"disabled-events\"))\n                    .into_iter()\n                    .map(|(_, e)| e),\n                false,\n                |event_type| {\n                    if exclude_event != Some(event_type) {\n                        let event_level = custom_levels\n                            .get(&event_type)\n                            .copied()\n                            .unwrap_or(event_type.level());\n                        if level.is_contained(event_level) {\n                            tracer.interests.set(event_type);\n                            global_interests.set(event_type);\n                        }\n                    }\n                },\n            );\n\n            if !tracer.interests.is_empty() {\n                tracers.push(tracer);\n            } else {\n                config.new_build_warning((\"tracer\", \"id\"), \"No events enabled for tracer\");\n            }\n        }\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Parse tracing history\n        #[cfg(feature = \"enterprise\")]\n        {\n            if config\n                .property_or_default(\"tracing.history.enable\", \"false\")\n                .unwrap_or(false)\n                && let Some(store_id) = config.value_require(\"tracing.history.store\")\n            {\n                if let Some(store) = stores.stores.get(store_id) {\n                    let mut tracer = TelemetrySubscriber {\n                        id: \"history\".to_string(),\n                        interests: Default::default(),\n                        lossy: false,\n                        typ: TelemetrySubscriberType::StoreTracer(StoreTracer {\n                            store: store.clone(),\n                        }),\n                    };\n\n                    for event_type in StoreTracer::default_events() {\n                        tracer.interests.set(event_type);\n                        global_interests.set(event_type);\n                    }\n\n                    tracers.push(tracer);\n                } else {\n                    let err = format!(\"Store {store_id} not found\");\n                    config.new_build_error(\"tracing.history.store\", err);\n                }\n            }\n        }\n        // SPDX-SnippetEnd\n\n        // Parse webhooks\n        for id in config.sub_keys(\"webhook\", \".url\") {\n            if let Some(webhook) = parse_webhook(config, &id, &mut global_interests) {\n                tracers.push(webhook);\n            }\n        }\n\n        // Add default tracer if none were found\n        #[cfg(not(feature = \"test_mode\"))]\n        if tracers.is_empty() {\n            for event_type in EventType::variants() {\n                let event_level = custom_levels\n                    .get(&event_type)\n                    .copied()\n                    .unwrap_or(event_type.level());\n                if Level::Info.is_contained(event_level) {\n                    global_interests.set(event_type);\n                }\n            }\n\n            tracers.push(TelemetrySubscriber {\n                id: \"default\".to_string(),\n                interests: global_interests.clone(),\n                typ: TelemetrySubscriberType::ConsoleTracer(ConsoleTracer {\n                    ansi: true,\n                    multiline: false,\n                    buffered: true,\n                }),\n                lossy: false,\n            });\n        }\n\n        Tracers {\n            subscribers: tracers,\n            interests: global_interests,\n            levels: custom_levels,\n        }\n    }\n}\n\nimpl Metrics {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut metrics = Metrics {\n            prometheus: None,\n            otel: None,\n            log_path: None,\n        };\n\n        // Obtain log path\n        for tracer_id in config.sub_keys(\"tracer\", \".type\") {\n            let tracer_id = tracer_id.as_str();\n            if config\n                .value((\"tracer\", tracer_id, \"enable\"))\n                .unwrap_or(\"true\")\n                == \"true\"\n                && config\n                    .value((\"tracer\", tracer_id, \"type\"))\n                    .unwrap_or_default()\n                    == \"log\"\n                && let Some(path) = config\n                    .value((\"tracer\", tracer_id, \"path\"))\n                    .map(|s| s.to_string())\n            {\n                metrics.log_path = Some(path);\n                break;\n            }\n        }\n\n        if config\n            .property_or_default(\"metrics.prometheus.enable\", \"false\")\n            .unwrap_or(false)\n        {\n            metrics.prometheus = Some(PrometheusMetrics {\n                auth: config\n                    .value(\"metrics.prometheus.auth.username\")\n                    .and_then(|user| {\n                        config\n                            .value(\"metrics.prometheus.auth.secret\")\n                            .map(|secret| STANDARD.encode(format!(\"{user}:{secret}\")))\n                    }),\n            });\n        }\n\n        let otel_enabled = match config\n            .value(\"metrics.open-telemetry.transport\")\n            .unwrap_or(\"disable\")\n        {\n            \"grpc\" => true.into(),\n            \"http\" | \"https\" => false.into(),\n            \"disable\" | \"disabled\" => None,\n            transport => {\n                let err = format!(\"Invalid transport: {transport}\");\n                config.new_parse_error(\"metrics.open-telemetry.transport\", err);\n                None\n            }\n        };\n\n        if let Some(is_grpc) = otel_enabled {\n            let timeout = config\n                .property::<Duration>(\"metrics.open-telemetry.timeout\")\n                .unwrap_or(opentelemetry_otlp::OTEL_EXPORTER_OTLP_TIMEOUT_DEFAULT);\n            let interval = config\n                .property_or_default(\"metrics.open-telemetry.interval\", \"1m\")\n                .unwrap_or_else(|| Duration::from_secs(60));\n            let resource = Resource::builder()\n                .with_service_name(\"stalwart\")\n                .with_attribute(KeyValue::new(SERVICE_VERSION, env!(\"CARGO_PKG_VERSION\")))\n                .build();\n            let instrumentation = InstrumentationScope::builder(\"stalwart\")\n                .with_version(env!(\"CARGO_PKG_VERSION\"))\n                .build();\n\n            if is_grpc {\n                let mut exporter = MetricExporter::builder()\n                    .with_temporality(Temporality::Delta)\n                    .with_tonic()\n                    .with_protocol(opentelemetry_otlp::Protocol::Grpc)\n                    .with_timeout(timeout);\n                if let Some(endpoint) = config.value(\"metrics.open-telemetry.endpoint\") {\n                    exporter = exporter.with_endpoint(endpoint);\n                }\n\n                match exporter.build() {\n                    Ok(exporter) => {\n                        metrics.otel = Some(Arc::new(OtelMetrics {\n                            exporter,\n                            interval,\n                            resource,\n                            instrumentation,\n                        }));\n                    }\n                    Err(err) => {\n                        config.new_build_error(\n                            \"metrics.open-telemetry\",\n                            format!(\"Failed to build OpenTelemetry metrics exporter: {err}\"),\n                        );\n                    }\n                }\n            } else if let Some(endpoint) = config\n                .value_require(\"metrics.open-telemetry.endpoint\")\n                .map(|s| s.to_string())\n            {\n                let mut headers = HashMap::new();\n                let mut err = None;\n                for (_, value) in config.values(\"metrics.open-telemetry.headers\") {\n                    if let Some((key, value)) = value.split_once(':') {\n                        headers.insert(key.trim().to_string(), value.trim().to_string());\n                    } else {\n                        err = format!(\"Invalid open-telemetry header {value:?}\").into();\n                        break;\n                    }\n                }\n                if let Some(err) = err {\n                    config.new_parse_error(\"metrics.open-telemetry.headers\", err);\n                }\n\n                let mut exporter = MetricExporter::builder()\n                    .with_temporality(Temporality::Delta)\n                    .with_http()\n                    .with_endpoint(&endpoint)\n                    .with_timeout(timeout);\n                if !headers.is_empty() {\n                    exporter = exporter.with_headers(headers);\n                }\n\n                match exporter.build() {\n                    Ok(exporter) => {\n                        metrics.otel = Some(Arc::new(OtelMetrics {\n                            exporter,\n                            interval,\n                            resource,\n                            instrumentation,\n                        }));\n                    }\n                    Err(err) => {\n                        config.new_build_error(\n                            \"metrics.open-telemetry\",\n                            format!(\"Failed to build OpenTelemetry metrics exporter: {err}\"),\n                        );\n                    }\n                }\n            }\n        }\n\n        metrics\n    }\n}\n\nfn parse_webhook(\n    config: &mut Config,\n    id: &str,\n    global_interests: &mut Interests,\n) -> Option<TelemetrySubscriber> {\n    let mut headers = parse_http_headers(config, (\"webhook\", id));\n    headers.insert(CONTENT_TYPE, \"application/json\".parse().unwrap());\n\n    // Build tracer\n    let mut tracer = TelemetrySubscriber {\n        id: format!(\"w_{id}\"),\n        interests: Default::default(),\n        lossy: config\n            .property_or_default((\"webhook\", id, \"lossy\"), \"false\")\n            .unwrap_or(false),\n        typ: TelemetrySubscriberType::Webhook(WebhookTracer {\n            url: config.value_require((\"webhook\", id, \"url\"))?.to_string(),\n            timeout: config\n                .property_or_default((\"webhook\", id, \"timeout\"), \"30s\")\n                .unwrap_or_else(|| Duration::from_secs(30)),\n            tls_allow_invalid_certs: config\n                .property_or_default((\"webhook\", id, \"allow-invalid-certs\"), \"false\")\n                .unwrap_or_default(),\n            headers,\n            key: config\n                .value((\"webhook\", id, \"signature-key\"))\n                .unwrap_or_default()\n                .to_string(),\n            throttle: config\n                .property_or_default((\"webhook\", id, \"throttle\"), \"1s\")\n                .unwrap_or_else(|| Duration::from_secs(1)),\n            discard_after: config\n                .property_or_default((\"webhook\", id, \"discard-after\"), \"5m\")\n                .unwrap_or_else(|| Duration::from_secs(300)),\n        }),\n    };\n\n    // Parse webhook events\n    apply_events(\n        config\n            .properties::<EventOrMany>((\"webhook\", id, \"events\"))\n            .into_iter()\n            .map(|(_, e)| e),\n        true,\n        |event_type| {\n            if event_type != EventType::Telemetry(TelemetryEvent::WebhookError) {\n                tracer.interests.set(event_type);\n                global_interests.set(event_type);\n            }\n        },\n    );\n\n    if !tracer.interests.is_empty() {\n        Some(tracer)\n    } else {\n        config.new_build_warning((\"webhook\", id), \"No events enabled for webhook\");\n        None\n    }\n}\n\nenum EventOrMany {\n    Event(EventType),\n    StartsWith(String),\n    EndsWith(String),\n    All,\n}\n\nfn apply_events(\n    event_types: impl IntoIterator<Item = EventOrMany>,\n    inclusive: bool,\n    mut apply_fn: impl FnMut(EventType),\n) {\n    let event_names = EventType::variants()\n        .into_iter()\n        .map(|e| (e, e.name()))\n        .collect::<Vec<_>>();\n    let mut exclude_events = AHashSet::new();\n\n    for event_or_many in event_types {\n        match event_or_many {\n            EventOrMany::Event(event_type) => {\n                if inclusive {\n                    apply_fn(event_type);\n                } else {\n                    exclude_events.insert(event_type);\n                }\n            }\n            EventOrMany::StartsWith(value) => {\n                for (event_type, name) in event_names.iter() {\n                    if name.starts_with(&value) {\n                        if inclusive {\n                            apply_fn(*event_type);\n                        } else {\n                            exclude_events.insert(*event_type);\n                        }\n                    }\n                }\n            }\n            EventOrMany::EndsWith(value) => {\n                for (event_type, name) in event_names.iter() {\n                    if name.ends_with(&value) {\n                        if inclusive {\n                            apply_fn(*event_type);\n                        } else {\n                            exclude_events.insert(*event_type);\n                        }\n                    }\n                }\n            }\n            EventOrMany::All => {\n                for (event_type, _) in event_names.iter() {\n                    if inclusive {\n                        apply_fn(*event_type);\n                    } else {\n                        exclude_events.insert(*event_type);\n                    }\n                }\n                break;\n            }\n        }\n    }\n\n    if !inclusive {\n        for (event_type, _) in event_names.iter() {\n            if !exclude_events.contains(event_type) {\n                apply_fn(*event_type);\n            }\n        }\n    }\n}\n\nimpl ParseValue for EventOrMany {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        let value = value.trim();\n        if value == \"*\" {\n            Ok(EventOrMany::All)\n        } else if let Some(suffix) = value.strip_prefix(\"*\") {\n            Ok(EventOrMany::EndsWith(suffix.to_string()))\n        } else if let Some(prefix) = value.strip_suffix(\"*\") {\n            Ok(EventOrMany::StartsWith(prefix.to_string()))\n        } else {\n            EventType::parse_value(value).map(EventOrMany::Event)\n        }\n    }\n}\n\nimpl std::fmt::Debug for OtelMetrics {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.debug_struct(\"OtelMetrics\")\n            .field(\"interval\", &self.interval)\n            .finish()\n    }\n}\n"
  },
  {
    "path": "crates/common/src/core.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    Inner, Server,\n    auth::{AccessToken, ResourceToken, TenantInfo},\n    config::{\n        smtp::{\n            auth::{ArcSealer, DkimSigner, LazySignature, ResolvedSignature, build_signature},\n            queue::{\n                ConnectionStrategy, DEFAULT_QUEUE_NAME, MxConfig, QueueExpiry, QueueName,\n                QueueStrategy, RequireOptional, RoutingStrategy, TlsStrategy, VirtualQueue,\n            },\n        },\n        spamfilter::SpamClassifier,\n    },\n    ipc::{BroadcastEvent, PushEvent, PushNotification},\n    manager::SPAM_CLASSIFIER_KEY,\n};\nuse directory::{Directory, QueryParams, Type, backend::internal::manage::ManageDirectory};\nuse mail_auth::IpLookupStrategy;\nuse sieve::Sieve;\nuse std::{\n    sync::{Arc, LazyLock},\n    time::Duration,\n};\nuse store::{\n    BlobStore, Deserialize, InMemoryStore, IndexKey, IndexKeyPrefix, IterateParams, Key, LogKey,\n    SUBSPACE_LOGS, SearchStore, SerializeInfallible, Store, U32_LEN, U64_LEN, ValueKey,\n    dispatch::DocumentSet,\n    roaring::RoaringBitmap,\n    write::{\n        AlignedBytes, AnyClass, Archive, AssignedIds, BatchBuilder, BlobLink, BlobOp,\n        DirectoryClass, QueueClass, ValueClass, key::DeserializeBigEndian, now,\n    },\n};\nuse trc::{AddContext, SpamEvent};\nuse types::{\n    blob::{BlobClass, BlobId},\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection},\n    field::Field,\n    type_state::{DataType, StateChange},\n};\nuse utils::{map::bitmap::Bitmap, snowflake::SnowflakeIdGenerator};\n\nimpl Server {\n    #[inline(always)]\n    pub fn store(&self) -> &Store {\n        &self.core.storage.data\n    }\n\n    #[inline(always)]\n    pub fn blob_store(&self) -> &BlobStore {\n        &self.core.storage.blob\n    }\n\n    #[inline(always)]\n    pub fn search_store(&self) -> &SearchStore {\n        &self.core.storage.fts\n    }\n\n    #[inline(always)]\n    pub fn in_memory_store(&self) -> &InMemoryStore {\n        &self.core.storage.lookup\n    }\n\n    #[inline(always)]\n    pub fn directory(&self) -> &Directory {\n        &self.core.storage.directory\n    }\n\n    pub fn get_directory(&self, name: &str) -> Option<&Arc<Directory>> {\n        self.core.storage.directories.get(name)\n    }\n\n    pub fn get_directory_or_default(&self, name: &str, session_id: u64) -> &Arc<Directory> {\n        self.core.storage.directories.get(name).unwrap_or_else(|| {\n            if !name.is_empty() {\n                trc::event!(\n                    Eval(trc::EvalEvent::DirectoryNotFound),\n                    Id = name.to_string(),\n                    SpanId = session_id,\n                );\n            }\n\n            &self.core.storage.directory\n        })\n    }\n\n    pub fn get_in_memory_store(&self, name: &str) -> Option<&InMemoryStore> {\n        self.core.storage.lookups.get(name)\n    }\n\n    pub fn get_in_memory_store_or_default(&self, name: &str, session_id: u64) -> &InMemoryStore {\n        self.core.storage.lookups.get(name).unwrap_or_else(|| {\n            if !name.is_empty() {\n                trc::event!(\n                    Eval(trc::EvalEvent::StoreNotFound),\n                    Id = name.to_string(),\n                    SpanId = session_id,\n                );\n            }\n\n            &self.core.storage.lookup\n        })\n    }\n\n    pub fn get_data_store(&self, name: &str, session_id: u64) -> &Store {\n        self.core.storage.stores.get(name).unwrap_or_else(|| {\n            if !name.is_empty() {\n                trc::event!(\n                    Eval(trc::EvalEvent::StoreNotFound),\n                    Id = name.to_string(),\n                    SpanId = session_id,\n                );\n            }\n\n            &self.core.storage.data\n        })\n    }\n\n    pub fn get_arc_sealer(&self, name: &str, session_id: u64) -> Option<Arc<ArcSealer>> {\n        self.resolve_signature(name).map(|s| s.sealer).or_else(|| {\n            trc::event!(\n                Arc(trc::ArcEvent::SealerNotFound),\n                Id = name.to_string(),\n                SpanId = session_id,\n            );\n\n            None\n        })\n    }\n\n    pub fn get_dkim_signer(&self, name: &str, session_id: u64) -> Option<Arc<DkimSigner>> {\n        self.resolve_signature(name).map(|s| s.signer).or_else(|| {\n            trc::event!(\n                Dkim(trc::DkimEvent::SignerNotFound),\n                Id = name.to_string(),\n                SpanId = session_id,\n            );\n\n            None\n        })\n    }\n\n    fn resolve_signature(&self, name: &str) -> Option<ResolvedSignature> {\n        let lazy_resolver_ = self.core.smtp.mail_auth.signatures.get(name)?;\n        match lazy_resolver_.load().as_ref() {\n            LazySignature::Resolved(resolved_signature) => Some(resolved_signature.clone()),\n            LazySignature::Pending(config) => {\n                let mut config = config.clone();\n                if let Some((signer, sealer)) = build_signature(&mut config, name) {\n                    let resolved = ResolvedSignature {\n                        signer: Arc::new(signer),\n                        sealer: Arc::new(sealer),\n                    };\n                    lazy_resolver_.store(Arc::new(LazySignature::Resolved(resolved.clone())));\n                    Some(resolved)\n                } else {\n                    config.log_errors();\n                    lazy_resolver_.store(Arc::new(LazySignature::Failed));\n                    None\n                }\n            }\n            LazySignature::Failed => None,\n        }\n    }\n\n    pub fn get_trusted_sieve_script(&self, name: &str, session_id: u64) -> Option<&Arc<Sieve>> {\n        self.core.sieve.trusted_scripts.get(name).or_else(|| {\n            trc::event!(\n                Sieve(trc::SieveEvent::ScriptNotFound),\n                Id = name.to_string(),\n                SpanId = session_id,\n            );\n\n            None\n        })\n    }\n\n    pub fn get_untrusted_sieve_script(&self, name: &str, session_id: u64) -> Option<&Arc<Sieve>> {\n        self.core.sieve.untrusted_scripts.get(name).or_else(|| {\n            trc::event!(\n                Sieve(trc::SieveEvent::ScriptNotFound),\n                Id = name.to_string(),\n                SpanId = session_id,\n            );\n\n            None\n        })\n    }\n\n    pub fn get_route_or_default(&self, name: &str, session_id: u64) -> &RoutingStrategy {\n        static LOCAL_GATEWAY: RoutingStrategy = RoutingStrategy::Local;\n        static MX_GATEWAY: RoutingStrategy = RoutingStrategy::Mx(MxConfig {\n            max_mx: 5,\n            max_multi_homed: 2,\n            ip_lookup_strategy: IpLookupStrategy::Ipv4thenIpv6,\n        });\n        self.core\n            .smtp\n            .queue\n            .routing_strategy\n            .get(name)\n            .unwrap_or_else(|| match name {\n                \"local\" => &LOCAL_GATEWAY,\n                \"mx\" => &MX_GATEWAY,\n                _ => {\n                    trc::event!(\n                        Smtp(trc::SmtpEvent::IdNotFound),\n                        Id = name.to_string(),\n                        Details = \"Gateway not found\",\n                        SpanId = session_id,\n                    );\n                    &MX_GATEWAY\n                }\n            })\n    }\n\n    pub fn get_virtual_queue_or_default(&self, name: &QueueName) -> &VirtualQueue {\n        static DEFAULT_QUEUE: VirtualQueue = VirtualQueue { threads: 25 };\n        self.core\n            .smtp\n            .queue\n            .virtual_queues\n            .get(name)\n            .unwrap_or_else(|| {\n                if name != &DEFAULT_QUEUE_NAME {\n                    trc::event!(\n                        Smtp(trc::SmtpEvent::IdNotFound),\n                        Id = name.to_string(),\n                        Details = \"Virtual queue not found\",\n                    );\n                }\n\n                &DEFAULT_QUEUE\n            })\n    }\n\n    pub fn get_queue_or_default(&self, name: &str, session_id: u64) -> &QueueStrategy {\n        static DEFAULT_SCHEDULE: LazyLock<QueueStrategy> = LazyLock::new(|| QueueStrategy {\n            retry: vec![\n                120,  // 2 minutes\n                300,  // 5 minutes\n                600,  // 10 minutes\n                900,  // 15 minutes\n                1800, // 30 minutes\n                3600, // 1 hour\n                7200, // 2 hours\n            ],\n            notify: vec![\n                86400,  // 1 day\n                259200, // 3 days\n            ],\n            expiry: QueueExpiry::Ttl(432000), // 5 days\n            virtual_queue: QueueName::default(),\n        });\n        self.core\n            .smtp\n            .queue\n            .queue_strategy\n            .get(name)\n            .unwrap_or_else(|| {\n                if name != \"default\" {\n                    trc::event!(\n                        Smtp(trc::SmtpEvent::IdNotFound),\n                        Id = name.to_string(),\n                        Details = \"Queue strategy not found\",\n                        SpanId = session_id,\n                    );\n                }\n\n                &DEFAULT_SCHEDULE\n            })\n    }\n\n    pub fn get_tls_or_default(&self, name: &str, session_id: u64) -> &TlsStrategy {\n        static DEFAULT_TLS: TlsStrategy = TlsStrategy {\n            dane: RequireOptional::Optional,\n            mta_sts: RequireOptional::Optional,\n            tls: RequireOptional::Optional,\n            allow_invalid_certs: false,\n            timeout_tls: Duration::from_secs(3 * 60),\n            timeout_mta_sts: Duration::from_secs(5 * 60),\n        };\n        self.core\n            .smtp\n            .queue\n            .tls_strategy\n            .get(name)\n            .unwrap_or_else(|| {\n                if name != \"default\" {\n                    trc::event!(\n                        Smtp(trc::SmtpEvent::IdNotFound),\n                        Id = name.to_string(),\n                        Details = \"TLS strategy not found\",\n                        SpanId = session_id,\n                    );\n                }\n\n                &DEFAULT_TLS\n            })\n    }\n\n    pub fn get_connection_or_default(&self, name: &str, session_id: u64) -> &ConnectionStrategy {\n        static DEFAULT_CONNECTION: ConnectionStrategy = ConnectionStrategy {\n            source_ipv4: Vec::new(),\n            source_ipv6: Vec::new(),\n            ehlo_hostname: None,\n            timeout_connect: Duration::from_secs(5 * 60),\n            timeout_greeting: Duration::from_secs(5 * 60),\n            timeout_ehlo: Duration::from_secs(5 * 60),\n            timeout_mail: Duration::from_secs(5 * 60),\n            timeout_rcpt: Duration::from_secs(5 * 60),\n            timeout_data: Duration::from_secs(10 * 60),\n        };\n\n        self.core\n            .smtp\n            .queue\n            .connection_strategy\n            .get(name)\n            .unwrap_or_else(|| {\n                if name != \"default\" {\n                    trc::event!(\n                        Smtp(trc::SmtpEvent::IdNotFound),\n                        Id = name.to_string(),\n                        Details = \"Connection strategy not found\",\n                        SpanId = session_id,\n                    );\n                }\n\n                &DEFAULT_CONNECTION\n            })\n    }\n\n    pub async fn get_used_quota(&self, account_id: u32) -> trc::Result<i64> {\n        self.core\n            .storage\n            .data\n            .get_counter(DirectoryClass::UsedQuota(account_id))\n            .await\n            .add_context(|err| err.caused_by(trc::location!()).account_id(account_id))\n    }\n\n    pub async fn has_available_quota(\n        &self,\n        quotas: &ResourceToken,\n        item_size: u64,\n    ) -> trc::Result<()> {\n        if quotas.quota != 0 {\n            let used_quota = self.get_used_quota(quotas.account_id).await? as u64;\n\n            if used_quota + item_size > quotas.quota {\n                return Err(trc::LimitEvent::Quota\n                    .into_err()\n                    .ctx(trc::Key::Limit, quotas.quota)\n                    .ctx(trc::Key::Size, used_quota));\n            }\n        }\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        #[cfg(feature = \"enterprise\")]\n        if self.core.is_enterprise_edition()\n            && let Some(tenant) = quotas.tenant.filter(|tenant| tenant.quota != 0)\n        {\n            let used_quota = self.get_used_quota(tenant.id).await? as u64;\n\n            if used_quota + item_size > tenant.quota {\n                return Err(trc::LimitEvent::TenantQuota\n                    .into_err()\n                    .ctx(trc::Key::Limit, tenant.quota)\n                    .ctx(trc::Key::Size, used_quota));\n            }\n        }\n\n        // SPDX-SnippetEnd\n\n        Ok(())\n    }\n\n    pub async fn get_resource_token(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> trc::Result<ResourceToken> {\n        Ok(if access_token.primary_id == account_id {\n            ResourceToken {\n                account_id,\n                quota: access_token.quota,\n                tenant: access_token.tenant,\n            }\n        } else {\n            let mut quotas = ResourceToken {\n                account_id,\n                ..Default::default()\n            };\n\n            if let Some(principal) = self\n                .core\n                .storage\n                .directory\n                .query(QueryParams::id(account_id).with_return_member_of(false))\n                .await\n                .add_context(|err| err.caused_by(trc::location!()).account_id(account_id))?\n            {\n                quotas.quota = principal.quota().unwrap_or_default();\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n\n                #[cfg(feature = \"enterprise\")]\n                if self.core.is_enterprise_edition()\n                    && let Some(tenant_id) = principal.tenant()\n                {\n                    quotas.tenant = TenantInfo {\n                        id: tenant_id,\n                        quota: self\n                            .core\n                            .storage\n                            .directory\n                            .query(QueryParams::id(tenant_id).with_return_member_of(false))\n                            .await\n                            .add_context(|err| {\n                                err.caused_by(trc::location!()).account_id(tenant_id)\n                            })?\n                            .and_then(|tenant| tenant.quota())\n                            .unwrap_or_default(),\n                    }\n                    .into();\n                }\n\n                // SPDX-SnippetEnd\n            }\n\n            quotas\n        })\n    }\n\n    pub async fn archives<I, CB>(\n        &self,\n        account_id: u32,\n        collection: Collection,\n        documents: &I,\n        mut cb: CB,\n    ) -> trc::Result<()>\n    where\n        I: DocumentSet + Send + Sync,\n        CB: FnMut(u32, Archive<AlignedBytes>) -> trc::Result<bool> + Send + Sync,\n    {\n        let collection: u8 = collection.into();\n\n        self.core\n            .storage\n            .data\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection,\n                        document_id: documents.min(),\n                        class: ValueClass::Property(Field::ARCHIVE.into()),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection,\n                        document_id: documents.max(),\n                        class: ValueClass::Property(Field::ARCHIVE.into()),\n                    },\n                ),\n                |key, value| {\n                    let document_id = key.deserialize_be_u32(key.len() - U32_LEN)?;\n                    if documents.contains(document_id) {\n                        <Archive<AlignedBytes> as Deserialize>::deserialize(value)\n                            .and_then(|archive| cb(document_id, archive))\n                    } else {\n                        Ok(true)\n                    }\n                },\n            )\n            .await\n            .add_context(|err| {\n                err.caused_by(trc::location!())\n                    .account_id(account_id)\n                    .collection(collection)\n            })\n    }\n\n    pub async fn all_archives<CB>(\n        &self,\n        account_id: u32,\n        collection: Collection,\n        field: u8,\n        mut cb: CB,\n    ) -> trc::Result<()>\n    where\n        CB: FnMut(u32, Archive<AlignedBytes>) -> trc::Result<()> + Send + Sync,\n    {\n        let collection: u8 = collection.into();\n\n        self.core\n            .storage\n            .data\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection,\n                        document_id: 0,\n                        class: ValueClass::Property(field),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection,\n                        document_id: u32::MAX,\n                        class: ValueClass::Property(field),\n                    },\n                ),\n                |key, value| {\n                    let document_id = key.deserialize_be_u32(key.len() - U32_LEN)?;\n                    let archive = <Archive<AlignedBytes> as Deserialize>::deserialize(value)?;\n                    cb(document_id, archive)?;\n\n                    Ok(true)\n                },\n            )\n            .await\n            .add_context(|err| {\n                err.caused_by(trc::location!())\n                    .account_id(account_id)\n                    .collection(collection)\n            })\n    }\n\n    pub async fn document_ids(\n        &self,\n        account_id: u32,\n        collection: Collection,\n        field: impl Into<u8>,\n    ) -> trc::Result<RoaringBitmap> {\n        let field = field.into();\n        let mut results = RoaringBitmap::new();\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    IndexKeyPrefix {\n                        account_id,\n                        collection: collection.into(),\n                        field,\n                    },\n                    IndexKeyPrefix {\n                        account_id,\n                        collection: collection.into(),\n                        field: field + 1,\n                    },\n                )\n                .no_values(),\n                |key, _| {\n                    results.insert(key.deserialize_be_u32(key.len() - U32_LEN)?);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())\n            .map(|_| results)\n    }\n\n    pub async fn document_exists(\n        &self,\n        account_id: u32,\n        collection: Collection,\n        field: impl Into<u8>,\n        filter: impl AsRef<[u8]>,\n    ) -> trc::Result<bool> {\n        let field = field.into();\n        let mut exists = false;\n        let filter = filter.as_ref();\n        let key_len = IndexKeyPrefix::len() + filter.len() + U32_LEN;\n\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    IndexKey {\n                        account_id,\n                        collection: collection.into(),\n                        document_id: 0,\n                        field,\n                        key: filter,\n                    },\n                    IndexKey {\n                        account_id,\n                        collection: collection.into(),\n                        document_id: u32::MAX,\n                        field,\n                        key: filter,\n                    },\n                )\n                .no_values(),\n                |key, _| {\n                    exists = key.len() == key_len;\n\n                    Ok(!exists)\n                },\n            )\n            .await\n            .caused_by(trc::location!())\n            .map(|_| exists)\n    }\n\n    pub async fn document_ids_matching(\n        &self,\n        account_id: u32,\n        collection: Collection,\n        field: impl Into<u8>,\n        filter: impl AsRef<[u8]>,\n    ) -> trc::Result<RoaringBitmap> {\n        let field = field.into();\n        let filter = filter.as_ref();\n        let key_len = IndexKeyPrefix::len() + filter.len() + U32_LEN;\n        let mut results = RoaringBitmap::new();\n\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    IndexKey {\n                        account_id,\n                        collection: collection.into(),\n                        document_id: 0,\n                        field,\n                        key: filter,\n                    },\n                    IndexKey {\n                        account_id,\n                        collection: collection.into(),\n                        document_id: u32::MAX,\n                        field,\n                        key: filter,\n                    },\n                )\n                .no_values(),\n                |key, _| {\n                    if key.len() == key_len {\n                        results.insert(key.deserialize_be_u32(key.len() - U32_LEN)?);\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())\n            .map(|_| results)\n    }\n\n    #[inline(always)]\n    pub fn notify_task_queue(&self) {\n        self.inner.ipc.task_tx.notify_one();\n    }\n\n    pub async fn total_queued_messages(&self) -> trc::Result<u64> {\n        let mut total = 0;\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey::from(ValueClass::Queue(QueueClass::Message(0))),\n                    ValueKey::from(ValueClass::Queue(QueueClass::Message(u64::MAX))),\n                )\n                .no_values(),\n                |_, _| {\n                    total += 1;\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())\n            .map(|_| total)\n    }\n\n    #[inline(always)]\n    pub fn generate_snowflake_id(&self) -> u64 {\n        self.inner.data.jmap_id_gen.generate()\n    }\n\n    pub async fn commit_batch(&self, mut builder: BatchBuilder) -> trc::Result<AssignedIds> {\n        let mut assigned_ids = AssignedIds::default();\n        let mut commit_points = builder.commit_points();\n\n        for commit_point in commit_points.iter() {\n            let batch = builder.build_one(commit_point);\n            assigned_ids\n                .ids\n                .extend(self.store().write(batch).await?.ids);\n        }\n\n        if let Some(changes) = builder.changes() {\n            for (account_id, changed_collections) in changes {\n                let mut state_change = StateChange::new(account_id);\n                for changed_collection in changed_collections.changed_containers {\n                    if let Some(data_type) = DataType::try_from_sync(changed_collection, true) {\n                        state_change.set_change(data_type);\n                    }\n                }\n                for changed_collection in changed_collections.changed_items {\n                    if let Some(data_type) = DataType::try_from_sync(changed_collection, false) {\n                        state_change.set_change(data_type);\n                    }\n                }\n                if state_change.has_changes() {\n                    self.broadcast_push_notification(PushNotification::StateChange(\n                        state_change.with_change_id(assigned_ids.last_change_id(account_id)?),\n                    ))\n                    .await;\n                }\n                if let Some(change_id) = changed_collections.share_notification_id {\n                    self.broadcast_push_notification(PushNotification::StateChange(StateChange {\n                        account_id,\n                        change_id,\n                        types: Bitmap::from_iter([DataType::ShareNotification]),\n                    }))\n                    .await;\n                }\n            }\n        }\n\n        Ok(assigned_ids)\n    }\n\n    pub async fn delete_changes(\n        &self,\n        account_id: u32,\n        max_entries: Option<usize>,\n        max_duration: Option<Duration>,\n    ) -> trc::Result<()> {\n        if let Some(max_entries) = max_entries {\n            for sync_collection in [\n                SyncCollection::Email,\n                SyncCollection::Thread,\n                SyncCollection::Identity,\n                SyncCollection::EmailSubmission,\n                SyncCollection::SieveScript,\n                SyncCollection::FileNode,\n                SyncCollection::AddressBook,\n                SyncCollection::Calendar,\n                SyncCollection::CalendarEventNotification,\n            ] {\n                let collection = sync_collection.into();\n                let from_key = LogKey {\n                    account_id,\n                    collection,\n                    change_id: 0,\n                };\n                let to_key = LogKey {\n                    account_id,\n                    collection,\n                    change_id: u64::MAX,\n                };\n\n                let mut first_change_id = 0;\n                let mut num_changes = 0;\n\n                self.store()\n                    .iterate(\n                        IterateParams::new(from_key, to_key)\n                            .descending()\n                            .no_values(),\n                        |key, _| {\n                            first_change_id = key.deserialize_be_u64(key.len() - U64_LEN)?;\n                            num_changes += 1;\n\n                            Ok(num_changes <= max_entries)\n                        },\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n\n                if num_changes > max_entries {\n                    self.store()\n                        .delete_range(\n                            LogKey {\n                                account_id,\n                                collection,\n                                change_id: 0,\n                            },\n                            LogKey {\n                                account_id,\n                                collection,\n                                change_id: first_change_id,\n                            },\n                        )\n                        .await\n                        .caused_by(trc::location!())?;\n\n                    // Delete vanished items\n                    if let Some(vanished_collection) =\n                        sync_collection.vanished_collection().map(u8::from)\n                    {\n                        self.store()\n                            .delete_range(\n                                LogKey {\n                                    account_id,\n                                    collection: vanished_collection,\n                                    change_id: 0,\n                                },\n                                LogKey {\n                                    account_id,\n                                    collection: vanished_collection,\n                                    change_id: first_change_id,\n                                },\n                            )\n                            .await\n                            .caused_by(trc::location!())?;\n                    }\n\n                    // Write truncation entry for cache\n                    let mut batch = BatchBuilder::new();\n                    batch.with_account_id(account_id).set(\n                        ValueClass::Any(AnyClass {\n                            subspace: SUBSPACE_LOGS,\n                            key: LogKey {\n                                account_id,\n                                collection,\n                                change_id: first_change_id,\n                            }\n                            .serialize(0),\n                        }),\n                        Vec::new(),\n                    );\n                    self.store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n            }\n        }\n\n        if let Some(max_duration) = max_duration {\n            self.store()\n                .delete_range(\n                    LogKey {\n                        account_id,\n                        collection: SyncCollection::ShareNotification.into(),\n                        change_id: 0,\n                    },\n                    LogKey {\n                        account_id,\n                        collection: SyncCollection::ShareNotification.into(),\n                        change_id: SnowflakeIdGenerator::from_duration(max_duration)\n                            .unwrap_or_default(),\n                    },\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n        Ok(())\n    }\n\n    pub async fn broadcast_push_notification(&self, notification: PushNotification) -> bool {\n        match self\n            .inner\n            .ipc\n            .push_tx\n            .clone()\n            .send(PushEvent::Publish {\n                notification,\n                broadcast: true,\n            })\n            .await\n        {\n            Ok(_) => true,\n            Err(_) => {\n                trc::event!(\n                    Server(trc::ServerEvent::ThreadError),\n                    Details = \"Error sending state change.\",\n                    CausedBy = trc::location!()\n                );\n\n                false\n            }\n        }\n    }\n\n    pub async fn cluster_broadcast(&self, event: BroadcastEvent) {\n        if let Some(broadcast_tx) = &self.inner.ipc.broadcast_tx.clone()\n            && broadcast_tx.send(event).await.is_err()\n        {\n            trc::event!(\n                Server(trc::ServerEvent::ThreadError),\n                Details = \"Error sending broadcast event.\",\n                CausedBy = trc::location!()\n            );\n        }\n    }\n\n    #[allow(clippy::blocks_in_conditions)]\n    pub async fn put_jmap_blob(&self, account_id: u32, data: &[u8]) -> trc::Result<BlobId> {\n        // First reserve the hash\n        let hash = BlobHash::generate(data);\n        let mut batch = BatchBuilder::new();\n        let until = now() + self.core.jmap.upload_tmp_ttl;\n\n        batch\n            .with_account_id(account_id)\n            .set(\n                BlobOp::Link {\n                    hash: hash.clone(),\n                    to: BlobLink::Temporary { until },\n                },\n                vec![BlobLink::QUOTA_LINK],\n            )\n            .set(\n                BlobOp::Quota {\n                    hash: hash.clone(),\n                    until,\n                },\n                (data.len() as u32).serialize(),\n            );\n\n        self.core\n            .storage\n            .data\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n\n        if !self\n            .core\n            .storage\n            .data\n            .blob_exists(&hash)\n            .await\n            .caused_by(trc::location!())?\n        {\n            // Upload blob to store\n            self.core\n                .storage\n                .blob\n                .put_blob(hash.as_ref(), data)\n                .await\n                .caused_by(trc::location!())?;\n\n            // Commit blob\n            let mut batch = BatchBuilder::new();\n            batch.set(BlobOp::Commit { hash: hash.clone() }, Vec::new());\n            self.core\n                .storage\n                .data\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(BlobId {\n            hash,\n            class: BlobClass::Reserved {\n                account_id,\n                expires: until,\n            },\n            section: None,\n        })\n    }\n\n    pub async fn put_temporary_blob(\n        &self,\n        account_id: u32,\n        data: &[u8],\n        hold_for: u64,\n    ) -> trc::Result<(BlobHash, BlobOp)> {\n        // First reserve the hash\n        let hash = BlobHash::generate(data);\n        let mut batch = BatchBuilder::new();\n        let until = now() + hold_for;\n\n        batch.with_account_id(account_id).set(\n            BlobOp::Link {\n                hash: hash.clone(),\n                to: BlobLink::Temporary { until },\n            },\n            vec![],\n        );\n\n        self.core\n            .storage\n            .data\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n\n        if !self\n            .core\n            .storage\n            .data\n            .blob_exists(&hash)\n            .await\n            .caused_by(trc::location!())?\n        {\n            // Upload blob to store\n            self.core\n                .storage\n                .blob\n                .put_blob(hash.as_ref(), data)\n                .await\n                .caused_by(trc::location!())?;\n\n            // Commit blob\n            let mut batch = BatchBuilder::new();\n            batch.set(BlobOp::Commit { hash: hash.clone() }, Vec::new());\n            self.core\n                .storage\n                .data\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok((\n            hash.clone(),\n            BlobOp::Link {\n                hash,\n                to: BlobLink::Temporary { until },\n            },\n        ))\n    }\n\n    pub async fn total_accounts(&self) -> trc::Result<u64> {\n        self.store()\n            .count_principals(None, Type::Individual.into(), None)\n            .await\n            .caused_by(trc::location!())\n    }\n\n    pub async fn total_domains(&self) -> trc::Result<u64> {\n        self.store()\n            .count_principals(None, Type::Domain.into(), None)\n            .await\n            .caused_by(trc::location!())\n    }\n\n    pub async fn spam_model_reload(&self) -> trc::Result<()> {\n        if self.core.spam.classifier.is_some() {\n            if let Some(model) = self\n                .blob_store()\n                .get_blob(SPAM_CLASSIFIER_KEY, 0..usize::MAX)\n                .await\n                .and_then(|archive| match archive {\n                    Some(archive) => <Archive<AlignedBytes> as Deserialize>::deserialize(&archive)\n                        .and_then(|archive| archive.deserialize_untrusted::<SpamClassifier>())\n                        .map(Some),\n                    None => Ok(None),\n                })\n                .caused_by(trc::location!())?\n            {\n                self.inner.data.spam_classifier.store(Arc::new(model));\n            } else {\n                trc::event!(Spam(SpamEvent::ModelNotFound));\n            }\n        }\n\n        Ok(())\n    }\n\n    #[cfg(not(feature = \"enterprise\"))]\n    pub async fn logo_resource(\n        &self,\n        _: &str,\n    ) -> trc::Result<Option<crate::manager::webadmin::Resource<Vec<u8>>>> {\n        Ok(None)\n    }\n}\n\npub trait BuildServer {\n    fn build_server(&self) -> Server;\n}\n\nimpl BuildServer for Arc<Inner> {\n    fn build_server(&self) -> Server {\n        Server {\n            inner: self.clone(),\n            core: self.shared_core.load_full(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/dns.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse mail_auth::{Error, IpLookupStrategy};\n\nuse crate::Server;\n\nimpl Server {\n    pub async fn dns_exists_mx(&self, entry: &str) -> trc::Result<bool> {\n        match self\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .mx_lookup(entry, Some(&self.inner.cache.dns_mx))\n            .await\n        {\n            Ok(result) => Ok(result.iter().any(|mx| !mx.exchanges.is_empty())),\n            Err(Error::DnsRecordNotFound(_)) => Ok(false),\n            Err(err) => Err(err.into()),\n        }\n    }\n\n    pub async fn dns_exists_ip(&self, entry: &str) -> trc::Result<bool> {\n        match self\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .ip_lookup(\n                entry,\n                IpLookupStrategy::Ipv4thenIpv6,\n                10,\n                Some(&self.inner.cache.dns_ipv4),\n                Some(&self.inner.cache.dns_ipv6),\n            )\n            .await\n        {\n            Ok(result) => Ok(!result.is_empty()),\n            Err(Error::DnsRecordNotFound(_)) => Ok(false),\n            Err(err) => Err(err.into()),\n        }\n    }\n\n    pub async fn dns_exists_ptr(&self, entry: &str) -> trc::Result<bool> {\n        if let Ok(addr) = entry.parse::<IpAddr>() {\n            match self\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .ptr_lookup(addr, Some(&self.inner.cache.dns_ptr))\n                .await\n            {\n                Ok(result) => Ok(!result.is_empty()),\n                Err(Error::DnsRecordNotFound(_)) => Ok(false),\n                Err(err) => Err(err.into()),\n            }\n        } else {\n            Err(trc::EventType::Resource(trc::ResourceEvent::BadParameters).into_err())\n        }\n    }\n\n    pub async fn dns_exists_ipv4(&self, entry: &str) -> trc::Result<bool> {\n        match self\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .ipv4_lookup(entry, Some(&self.inner.cache.dns_ipv4))\n            .await\n        {\n            Ok(result) => Ok(!result.is_empty()),\n            Err(Error::DnsRecordNotFound(_)) => Ok(false),\n            Err(err) => Err(err.into()),\n        }\n    }\n\n    pub async fn dns_exists_ipv6(&self, entry: &str) -> trc::Result<bool> {\n        match self\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .ipv6_lookup(entry, Some(&self.inner.cache.dns_ipv6))\n            .await\n        {\n            Ok(result) => Ok(!result.is_empty()),\n            Err(Error::DnsRecordNotFound(_)) => Ok(false),\n            Err(err) => Err(err.into()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/enterprise/alerts.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse mail_builder::{\n    MessageBuilder,\n    headers::{\n        HeaderType,\n        address::{Address, EmailAddress},\n    },\n};\nuse trc::{Collector, MetricType, TOTAL_EVENT_COUNT, TelemetryEvent};\n\nuse super::{AlertContent, AlertContentToken, AlertMethod};\nuse crate::{\n    Server,\n    expr::{Variable, functions::ResolveVariable},\n};\nuse std::fmt::Write;\n\n#[derive(Debug, PartialEq, Eq)]\npub struct AlertMessage {\n    pub from: String,\n    pub to: Vec<String>,\n    pub body: Vec<u8>,\n}\n\nstruct CollectorResolver;\n\nimpl Server {\n    pub async fn process_alerts(&self) -> Option<Vec<AlertMessage>> {\n        let alerts = &self.core.enterprise.as_ref()?.metrics_alerts;\n        if alerts.is_empty() {\n            return None;\n        }\n        let mut messages = Vec::new();\n\n        for alert in alerts {\n            if !self\n                .eval_expr(&alert.condition, &CollectorResolver, &alert.id, 0)\n                .await\n                .unwrap_or(false)\n            {\n                continue;\n            }\n            for method in &alert.method {\n                match method {\n                    AlertMethod::Email {\n                        from_name,\n                        from_addr,\n                        to,\n                        subject,\n                        body,\n                    } => {\n                        messages.push(AlertMessage {\n                            from: from_addr.clone(),\n                            to: to.clone(),\n                            body: MessageBuilder::new()\n                                .from(Address::Address(EmailAddress {\n                                    name: from_name.as_ref().map(|s| s.into()),\n                                    email: from_addr.as_str().into(),\n                                }))\n                                .header(\n                                    \"To\",\n                                    HeaderType::Address(Address::List(\n                                        to.iter()\n                                            .map(|to| {\n                                                Address::Address(EmailAddress {\n                                                    name: None,\n                                                    email: to.as_str().into(),\n                                                })\n                                            })\n                                            .collect(),\n                                    )),\n                                )\n                                .header(\"Auto-Submitted\", HeaderType::Text(\"auto-generated\".into()))\n                                .subject(subject.build())\n                                .text_body(body.build())\n                                .write_to_vec()\n                                .unwrap_or_default(),\n                        });\n                    }\n                    AlertMethod::Event { message } => {\n                        trc::event!(\n                            Telemetry(TelemetryEvent::Alert),\n                            Id = alert.id.to_string(),\n                            Details = message.as_ref().map(|m| m.build())\n                        );\n\n                        #[cfg(feature = \"test_mode\")]\n                        Collector::update_event_counter(\n                            trc::EventType::Telemetry(TelemetryEvent::Alert),\n                            1,\n                        );\n                    }\n                }\n            }\n        }\n\n        (!messages.is_empty()).then_some(messages)\n    }\n}\n\nimpl ResolveVariable for CollectorResolver {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        if (variable as usize) < TOTAL_EVENT_COUNT {\n            Variable::Integer(Collector::read_event_metric(variable as usize) as i64)\n        } else if let Some(metric_type) =\n            MetricType::from_code(variable as u64 - TOTAL_EVENT_COUNT as u64)\n        {\n            Variable::Float(Collector::read_metric(metric_type))\n        } else {\n            Variable::Integer(0)\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl AlertContent {\n    pub fn build(&self) -> String {\n        let mut buf = String::with_capacity(self.len());\n        for token in &self.0 {\n            token.write(&mut buf);\n        }\n        buf\n    }\n\n    #[allow(clippy::len_without_is_empty)]\n    pub fn len(&self) -> usize {\n        self.0.iter().map(|t| t.len()).sum()\n    }\n}\n\nimpl AlertContentToken {\n    fn write(&self, buf: &mut String) {\n        match self {\n            AlertContentToken::Text(text) => buf.push_str(text),\n            AlertContentToken::Metric(metric_type) => {\n                let _ = write!(buf, \"{}\", Collector::read_metric(*metric_type));\n            }\n            AlertContentToken::Event(event_type) => {\n                let _ = write!(buf, \"{}\", Collector::read_event_metric(event_type.id()));\n            }\n        }\n    }\n\n    fn len(&self) -> usize {\n        match self {\n            AlertContentToken::Text(s) => s.len(),\n            AlertContentToken::Metric(_) | AlertContentToken::Event(_) => 10,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/enterprise/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse super::{\n    AlertContent, AlertContentToken, AlertMethod, Enterprise, MetricAlert, MetricStore,\n    SpamFilterLlmConfig, TraceStore, Undelete, license::LicenseKey, llm::AiApiConfig,\n};\nuse crate::{\n    expr::{Expression, tokenizer::TokenMap},\n    manager::config::ConfigManager,\n};\nuse ahash::AHashMap;\nuse directory::{Type, backend::internal::manage::ManageDirectory};\nuse std::{sync::Arc, time::Duration};\nuse store::{Store, Stores};\nuse trc::{EventType, MetricType, TOTAL_EVENT_COUNT};\nuse utils::{\n    config::{\n        Config, ConfigKey,\n        cron::SimpleCron,\n        utils::{AsKey, ParseValue},\n    },\n    template::Template,\n};\n\nimpl Enterprise {\n    pub async fn parse(\n        config: &mut Config,\n        config_manager: &ConfigManager,\n        stores: &Stores,\n        data: &Store,\n    ) -> Option<Self> {\n        let server_hostname = config\n            .value(\"server.hostname\")\n            .or_else(|| config.value(\"lookup.default.hostname\"))?;\n        let mut update_license = None;\n\n        let license_result = match (\n            config.value(\"enterprise.license-key\"),\n            config.value(\"enterprise.api-key\"),\n        ) {\n            (Some(license_key), maybe_api_key) => {\n                match (LicenseKey::new(license_key, server_hostname), maybe_api_key) {\n                    (Ok(license), Some(api_key)) if license.is_near_expiration() => Ok(license\n                        .try_renew(api_key)\n                        .await\n                        .map(|result| {\n                            update_license = Some(result.encoded_key);\n                            result.key\n                        })\n                        .unwrap_or(license)),\n                    (Ok(license), None) => Ok(license),\n                    (Err(_), Some(api_key)) => LicenseKey::invalid(server_hostname)\n                        .try_renew(api_key)\n                        .await\n                        .map(|result| {\n                            update_license = Some(result.encoded_key);\n                            result.key\n                        }),\n                    (maybe_license, _) => maybe_license,\n                }\n            }\n            (None, Some(api_key)) => LicenseKey::invalid(server_hostname)\n                .try_renew(api_key)\n                .await\n                .map(|result| {\n                    update_license = Some(result.encoded_key);\n                    result.key\n                }),\n            (None, None) => {\n                return None;\n            }\n        };\n\n        // Report error\n        let license = match license_result {\n            Ok(license) => license,\n            Err(err) => {\n                config.new_build_warning(\"enterprise.license-key\", err.to_string());\n                return None;\n            }\n        };\n\n        // Update the license if a new one was obtained\n        if let Some(license) = update_license {\n            config\n                .keys\n                .insert(\"enterprise.license-key\".to_string(), license.clone());\n            if let Err(err) = config_manager\n                .set(\n                    [ConfigKey {\n                        key: \"enterprise.license-key\".to_string(),\n                        value: license.to_string(),\n                    }],\n                    true,\n                )\n                .await\n            {\n                trc::error!(\n                    err.caused_by(trc::location!())\n                        .details(\"Failed to update license key\")\n                );\n            }\n        }\n\n        match data\n            .count_principals(None, Type::Individual.into(), None)\n            .await\n        {\n            Ok(total) if total > license.accounts as u64 => {\n                config.new_build_warning(\n                    \"enterprise.license-key\",\n                    format!(\n                        \"License key is valid but only allows {} accounts, found {}.\",\n                        license.accounts, total\n                    ),\n                );\n                return None;\n            }\n            Err(e) => {\n                if !matches!(data, Store::None) {\n                    config.new_build_error(\"enterprise.license-key\", e.to_string());\n                }\n                return None;\n            }\n            _ => (),\n        }\n\n        let trace_store = if config\n            .property_or_default(\"tracing.history.enable\", \"false\")\n            .unwrap_or(false)\n        {\n            if let Some(store) = config\n                .value(\"tracing.history.store\")\n                .and_then(|name| stores.stores.get(name))\n                .cloned()\n            {\n                TraceStore {\n                    retention: config\n                        .property_or_default::<Option<Duration>>(\"tracing.history.retention\", \"30d\")\n                        .unwrap_or(Some(Duration::from_secs(30 * 24 * 60 * 60))),\n                    store,\n                }\n                .into()\n            } else {\n                None\n            }\n        } else {\n            None\n        };\n        let metrics_store = if config\n            .property_or_default(\"metrics.history.enable\", \"false\")\n            .unwrap_or(false)\n        {\n            if let Some(store) = config\n                .value(\"metrics.history.store\")\n                .and_then(|name| stores.stores.get(name))\n                .cloned()\n            {\n                MetricStore {\n                    retention: config\n                        .property_or_default::<Option<Duration>>(\"metrics.history.retention\", \"90d\")\n                        .unwrap_or(Some(Duration::from_secs(90 * 24 * 60 * 60))),\n                    store,\n                    interval: config\n                        .property_or_default::<SimpleCron>(\"metrics.history.interval\", \"0 * *\")\n                        .unwrap_or_else(|| SimpleCron::parse_value(\"0 * *\").unwrap()),\n                }\n                .into()\n            } else {\n                None\n            }\n        } else {\n            None\n        };\n\n        // Parse AI APIs\n        let mut ai_apis = AHashMap::new();\n        for id in config.sub_keys(\"enterprise.ai\", \".url\") {\n            if let Some(api) = AiApiConfig::parse(config, &id) {\n                ai_apis.insert(id, api.into());\n            }\n        }\n\n        // Build the enterprise configuration\n        let mut enterprise = Enterprise {\n            license,\n            undelete: config\n                .property_or_default::<Option<Duration>>(\"storage.undelete.retention\", \"false\")\n                .unwrap_or_default()\n                .map(|retention| Undelete { retention }),\n            logo_url: config.value(\"enterprise.logo-url\").map(|s| s.to_string()),\n            trace_store,\n            metrics_store,\n            metrics_alerts: parse_metric_alerts(config),\n            spam_filter_llm: SpamFilterLlmConfig::parse(config, &ai_apis),\n            ai_apis,\n            template_calendar_alarm: None,\n            template_scheduling_email: None,\n            template_scheduling_web: None,\n        };\n\n        // Parse templates\n        for (key, value) in [\n            (\n                \"calendar.alarms.template\",\n                &mut enterprise.template_calendar_alarm,\n            ),\n            (\n                \"calendar.scheduling.template.email\",\n                &mut enterprise.template_scheduling_email,\n            ),\n            (\n                \"calendar.scheduling.template.web\",\n                &mut enterprise.template_scheduling_web,\n            ),\n        ] {\n            if let Some(template) = config.value(key) {\n                match Template::parse(template) {\n                    Ok(template) => *value = Some(template),\n                    Err(err) => {\n                        config.new_build_error(key, format!(\"Invalid template: {err}\"));\n                    }\n                }\n            }\n        }\n\n        Some(enterprise)\n    }\n}\n\nimpl SpamFilterLlmConfig {\n    pub fn parse(config: &mut Config, models: &AHashMap<String, Arc<AiApiConfig>>) -> Option<Self> {\n        if !config\n            .property_or_default::<bool>(\"spam-filter.llm.enable\", \"false\")\n            .unwrap_or_default()\n        {\n            return None;\n        }\n        let model = config.value_require_non_empty(\"spam-filter.llm.model\")?;\n        let model = if let Some(model) = models.get(model) {\n            model.clone()\n        } else {\n            let message = format!(\"Model {model:?} not found in AI API configuration\");\n            config.new_build_error(\"spam-filter.llm.model\", message);\n            return None;\n        };\n\n        let llm = SpamFilterLlmConfig {\n            model,\n            temperature: config\n                .property_or_default(\"spam-filter.llm.temperature\", \"0.5\")\n                .unwrap_or(0.5),\n            prompt: config\n                .value_require_non_empty(\"spam-filter.llm.prompt\")?\n                .to_string(),\n            separator: config\n                .value_require_non_empty(\"spam-filter.llm.separator\")\n                .unwrap_or_default()\n                .chars()\n                .next()\n                .unwrap_or(','),\n            index_category: config\n                .property(\"spam-filter.llm.index.category\")\n                .unwrap_or_default(),\n            index_confidence: config.property(\"spam-filter.llm.index.confidence\"),\n            index_explanation: config.property(\"spam-filter.llm.index.explanation\"),\n            categories: config\n                .values(\"spam-filter.llm.categories\")\n                .map(|(_, v)| v.trim().to_uppercase())\n                .collect(),\n            confidence: config\n                .values(\"spam-filter.llm.confidence\")\n                .map(|(_, v)| v.trim().to_uppercase())\n                .collect(),\n        };\n\n        if llm.categories.is_empty() {\n            config.new_build_error(\"spam-filter.llm.categories\", \"No categories defined\");\n            return None;\n        }\n        if llm.index_confidence.is_some() && llm.confidence.is_empty() {\n            config.new_build_error(\n                \"spam-filter.llm.confidence\",\n                \"Confidence index is defined but no confidence values are provided\",\n            );\n            return None;\n        }\n\n        llm.into()\n    }\n}\n\npub fn parse_metric_alerts(config: &mut Config) -> Vec<MetricAlert> {\n    let mut alerts = Vec::new();\n\n    for metric_id in config.sub_keys(\"metrics.alerts\", \".enable\") {\n        if let Some(alert) = parse_metric_alert(config, metric_id) {\n            alerts.push(alert);\n        }\n    }\n\n    alerts\n}\n\nfn parse_metric_alert(config: &mut Config, id: String) -> Option<MetricAlert> {\n    if !config.property_or_default::<bool>((\"metrics.alerts\", id.as_str(), \"enable\"), \"false\")? {\n        return None;\n    }\n\n    let mut alert = MetricAlert {\n        condition: Expression::try_parse(\n            config,\n            (\"metrics.alerts\", id.as_str(), \"condition\"),\n            &TokenMap::default().with_variables_map(\n                EventType::variants()\n                    .into_iter()\n                    .map(|e| (sanitize_metric_name(e.name()), e.id() as u32))\n                    .chain(MetricType::variants().iter().map(|m| {\n                        (\n                            sanitize_metric_name(m.name()),\n                            m.code() as u32 + TOTAL_EVENT_COUNT as u32,\n                        )\n                    })),\n            ),\n        )?,\n        method: Vec::new(),\n        id,\n    };\n    let id_str = alert.id.as_str();\n\n    if config\n        .property_or_default::<bool>((\"metrics.alerts\", id_str, \"notify.event.enable\"), \"false\")\n        .unwrap_or_default()\n    {\n        alert.method.push(AlertMethod::Event {\n            message: parse_alert_content(\n                (\"metrics.alerts\", id_str, \"notify.event.message\"),\n                config,\n            ),\n        });\n    }\n\n    if config\n        .property_or_default::<bool>((\"metrics.alerts\", id_str, \"notify.email.enable\"), \"false\")\n        .unwrap_or_default()\n    {\n        let from_addr = config\n            .value_require((\"metrics.alerts\", id_str, \"notify.email.from-addr\"))?\n            .trim()\n            .to_string();\n        let from_name = config\n            .value((\"metrics.alerts\", id_str, \"notify.email.from-name\"))\n            .map(|s| s.to_string());\n        let to = config\n            .values((\"metrics.alerts\", id_str, \"notify.email.to\"))\n            .filter_map(|(_, s)| {\n                if s.contains('@') {\n                    s.trim().to_string().into()\n                } else {\n                    None\n                }\n            })\n            .collect::<Vec<_>>();\n        let subject =\n            parse_alert_content((\"metrics.alerts\", id_str, \"notify.email.subject\"), config)?;\n        let body = parse_alert_content((\"metrics.alerts\", id_str, \"notify.email.body\"), config)?;\n\n        if !from_addr.contains('@') {\n            config.new_build_error(\n                (\"metrics.alerts\", id_str, \"notify.email.from-addr\"),\n                \"Invalid from email address\",\n            );\n        }\n        if to.is_empty() {\n            config.new_build_error(\n                (\"metrics.alerts\", id_str, \"notify.email.to\"),\n                \"Missing recipient address(es)\",\n            );\n        }\n        if subject.0.is_empty() {\n            config.new_build_error(\n                (\"metrics.alerts\", id_str, \"notify.email.subject\"),\n                \"Missing email subject\",\n            );\n        }\n        if body.0.is_empty() {\n            config.new_build_error(\n                (\"metrics.alerts\", id_str, \"notify.email.body\"),\n                \"Missing email body\",\n            );\n        }\n\n        alert.method.push(AlertMethod::Email {\n            from_name,\n            from_addr,\n            to,\n            subject,\n            body,\n        });\n    }\n\n    if alert.method.is_empty() {\n        config.new_build_error(\n            (\"metrics.alerts\", id_str),\n            \"No notification method enabled for alert\",\n        );\n    }\n\n    alert.into()\n}\n\nfn parse_alert_content(key: impl AsKey, config: &mut Config) -> Option<AlertContent> {\n    let mut tokens = Vec::new();\n    let mut value = config.value(key)?.chars().peekable();\n    let mut buf = String::new();\n\n    while let Some(ch) = value.next() {\n        if ch == '%' && value.peek() == Some(&'{') {\n            value.next();\n\n            let mut var_name = String::new();\n            let mut found_curly = false;\n\n            for ch in value.by_ref() {\n                if ch == '}' {\n                    found_curly = true;\n                    break;\n                }\n                var_name.push(ch);\n            }\n\n            if found_curly && value.peek() == Some(&'%') {\n                value.next();\n                if let Some(event_type) = EventType::try_parse(&var_name)\n                    .map(AlertContentToken::Event)\n                    .or_else(|| MetricType::try_parse(&var_name).map(AlertContentToken::Metric))\n                {\n                    if !buf.is_empty() {\n                        tokens.push(AlertContentToken::Text(std::mem::take(&mut buf)));\n                    }\n                    tokens.push(event_type);\n                } else {\n                    buf.push('%');\n                    buf.push('{');\n                    buf.push_str(&var_name);\n                    buf.push('}');\n                    buf.push('%');\n                }\n            } else {\n                buf.push('%');\n                buf.push('{');\n                buf.push_str(&var_name);\n            }\n        } else {\n            buf.push(ch);\n        }\n    }\n\n    if !buf.is_empty() {\n        tokens.push(AlertContentToken::Text(buf));\n    }\n\n    AlertContent(tokens).into()\n}\n\nfn sanitize_metric_name(name: &str) -> String {\n    let mut result = String::with_capacity(name.len());\n    for ch in name.chars() {\n        if ch.is_ascii_alphanumeric() {\n            result.push(ch);\n        } else {\n            result.push('_');\n        }\n    }\n\n    result\n}\n"
  },
  {
    "path": "crates/common/src/enterprise/license.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\n/*\n * WARNING: TAMPERING WITH THIS CODE IS STRICTLY PROHIBITED\n * Any attempt to modify, bypass, or disable the license validation mechanism\n * constitutes a severe violation of the Stalwart Enterprise License Agreement.\n * Such actions may result in immediate termination of your license, legal action,\n * and substantial financial penalties. Stalwart Labs LLC actively monitors for\n * unauthorized modifications and will pursue all available legal remedies against\n * violators to the fullest extent of the law, including but not limited to claims\n * for copyright infringement, breach of contract, and fraud.\n */\n\nuse crate::manager::fetch_resource;\nuse base64::{Engine, engine::general_purpose::STANDARD};\nuse hyper::{HeaderMap, header::AUTHORIZATION};\nuse ring::signature::{ED25519, UnparsedPublicKey};\nuse std::{\n    fmt::{Display, Formatter},\n    time::Duration,\n};\nuse store::write::now;\nuse trc::ServerEvent;\n\n//const LICENSING_API: &str = \"https://localhost:444/api/license/\";\nconst LICENSING_API: &str = \"https://license.stalw.art/api/license/\";\nconst RENEW_THRESHOLD: u64 = 60 * 60 * 24 * 4; // 4 days\n\npub struct LicenseValidator {\n    public_key: UnparsedPublicKey<Vec<u8>>,\n}\n\n#[derive(Debug, Clone)]\npub struct LicenseKey {\n    pub valid_to: u64,\n    pub valid_from: u64,\n    pub domain: String,\n    pub accounts: u32,\n}\n\n#[derive(Debug)]\npub enum LicenseError {\n    Expired,\n    InvalidDomain { domain: String },\n    DomainMismatch { issued_to: String, current: String },\n    Parse,\n    Validation,\n    Decode,\n    InvalidParameters,\n    RenewalFailed { reason: String },\n}\n\npub struct RenewedLicense {\n    pub key: LicenseKey,\n    pub encoded_key: String,\n}\n\nconst U64_LEN: usize = std::mem::size_of::<u64>();\nconst U32_LEN: usize = std::mem::size_of::<u32>();\n\nimpl LicenseValidator {\n    #[allow(clippy::new_without_default)]\n    pub fn new() -> Self {\n        LicenseValidator {\n            public_key: UnparsedPublicKey::new(\n                &ED25519,\n                vec![\n                    118, 10, 182, 35, 89, 111, 11, 60, 154, 47, 205, 127, 107, 229, 55, 104, 72,\n                    54, 141, 14, 97, 219, 2, 4, 119, 143, 156, 10, 152, 216, 32, 194,\n                ],\n            ),\n        }\n    }\n\n    pub fn try_parse(&self, key: impl AsRef<str>) -> Result<LicenseKey, LicenseError> {\n        let key = STANDARD\n            .decode(key.as_ref())\n            .map_err(|_| LicenseError::Decode)?;\n        let valid_from = u64::from_le_bytes(\n            key.get(..U64_LEN)\n                .ok_or(LicenseError::Parse)?\n                .try_into()\n                .unwrap(),\n        );\n        let valid_to = u64::from_le_bytes(\n            key.get(U64_LEN..(U64_LEN * 2))\n                .ok_or(LicenseError::Parse)?\n                .try_into()\n                .unwrap(),\n        );\n        let accounts = u32::from_le_bytes(\n            key.get((U64_LEN * 2)..(U64_LEN * 2) + U32_LEN)\n                .ok_or(LicenseError::Parse)?\n                .try_into()\n                .unwrap(),\n        );\n        let domain_len = u32::from_le_bytes(\n            key.get((U64_LEN * 2) + U32_LEN..(U64_LEN * 2) + (U32_LEN * 2))\n                .ok_or(LicenseError::Parse)?\n                .try_into()\n                .unwrap(),\n        ) as usize;\n        let domain = String::from_utf8(\n            key.get((U64_LEN * 2) + (U32_LEN * 2)..(U64_LEN * 2) + (U32_LEN * 2) + domain_len)\n                .ok_or(LicenseError::Parse)?\n                .to_vec(),\n        )\n        .map_err(|_| LicenseError::Parse)?;\n        let signature = key\n            .get((U64_LEN * 2) + (U32_LEN * 2) + domain_len..)\n            .ok_or(LicenseError::Parse)?;\n\n        if valid_from == 0\n            || valid_to == 0\n            || valid_from >= valid_to\n            || accounts == 0\n            || domain.is_empty()\n        {\n            return Err(LicenseError::InvalidParameters);\n        }\n\n        // Validate signature\n        self.public_key\n            .verify(\n                &key[..(U64_LEN * 2) + (U32_LEN * 2) + domain_len],\n                signature,\n            )\n            .map_err(|_| LicenseError::Validation)?;\n\n        let key = LicenseKey {\n            valid_from,\n            valid_to,\n            domain,\n            accounts,\n        };\n\n        if !key.is_expired() {\n            Ok(key)\n        } else {\n            Err(LicenseError::Expired)\n        }\n    }\n}\n\nimpl LicenseKey {\n    pub fn new(\n        license_key: impl AsRef<str>,\n        hostname: impl AsRef<str>,\n    ) -> Result<Self, LicenseError> {\n        LicenseValidator::new()\n            .try_parse(license_key)\n            .and_then(|key| {\n                let local_domain = Self::base_domain(hostname)?;\n                let license_domain = Self::base_domain(&key.domain)?;\n                if local_domain == license_domain {\n                    Ok(key)\n                } else {\n                    Err(LicenseError::DomainMismatch {\n                        issued_to: license_domain,\n                        current: local_domain,\n                    })\n                }\n            })\n    }\n\n    pub fn invalid(domain: impl AsRef<str>) -> Self {\n        LicenseKey {\n            valid_from: 0,\n            valid_to: 0,\n            domain: Self::base_domain(domain).unwrap_or_default(),\n            accounts: 0,\n        }\n    }\n\n    pub async fn try_renew(&self, api_key: &str) -> Result<RenewedLicense, LicenseError> {\n        let mut headers = HeaderMap::new();\n        headers.insert(\n            AUTHORIZATION,\n            format!(\"Bearer {api_key}\")\n                .parse()\n                .map_err(|_| LicenseError::Validation)?,\n        );\n\n        trc::event!(\n            Server(ServerEvent::Licensing),\n            Details = \"Attempting to renew Enterprise license from license.stalw.art\",\n        );\n\n        match fetch_resource(\n            &format!(\"{}{}\", LICENSING_API, self.domain),\n            headers.into(),\n            Duration::from_secs(60),\n            1024,\n        )\n        .await\n        .and_then(|bytes| {\n            String::from_utf8(bytes)\n                .map_err(|_| String::from(\"Failed to UTF-8 decode server response\"))\n        }) {\n            Ok(encoded_key) => match LicenseKey::new(&encoded_key, &self.domain) {\n                Ok(key) => Ok(RenewedLicense { key, encoded_key }),\n                Err(err) => {\n                    trc::event!(\n                        Server(ServerEvent::Licensing),\n                        Details = \"Failed to decode license renewal\",\n                        Reason = err.to_string(),\n                    );\n                    Err(err)\n                }\n            },\n            Err(err) => {\n                trc::event!(\n                    Server(ServerEvent::Licensing),\n                    Details = \"Failed to renew Enterprise license\",\n                    Reason = err.clone(),\n                );\n                Err(LicenseError::RenewalFailed { reason: err })\n            }\n        }\n    }\n\n    pub fn is_near_expiration(&self) -> bool {\n        let now = now();\n        self.valid_to.saturating_sub(now) <= RENEW_THRESHOLD\n    }\n\n    pub fn expires_in(&self) -> Duration {\n        Duration::from_secs(self.valid_to.saturating_sub(now()))\n    }\n\n    pub fn renew_in(&self) -> Duration {\n        Duration::from_secs(self.valid_to.saturating_sub(now() + RENEW_THRESHOLD))\n    }\n\n    pub fn is_expired(&self) -> bool {\n        let now = now();\n        now >= self.valid_to || now < self.valid_from\n    }\n\n    pub fn base_domain(domain: impl AsRef<str>) -> Result<String, LicenseError> {\n        let domain = domain.as_ref();\n        psl::domain_str(domain)\n            .map(|d| d.to_string())\n            .ok_or(LicenseError::InvalidDomain {\n                domain: domain.to_string(),\n            })\n    }\n}\n\nimpl Display for LicenseError {\n    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {\n        match self {\n            LicenseError::Expired => write!(f, \"License is expired\"),\n            LicenseError::Parse => write!(f, \"Failed to parse license key\"),\n            LicenseError::Validation => write!(f, \"Failed to validate license key\"),\n            LicenseError::Decode => write!(f, \"Failed to decode license key\"),\n            LicenseError::InvalidParameters => write!(f, \"Invalid license key parameters\"),\n            LicenseError::DomainMismatch { issued_to, current } => {\n                write!(\n                    f,\n                    \"License issued to domain {issued_to:?} does not match {current:?}\",\n                )\n            }\n            LicenseError::InvalidDomain { domain } => {\n                write!(f, \"Invalid domain {domain:?}\")\n            }\n            LicenseError::RenewalFailed { reason } => {\n                write!(f, \"Failed to renew license: {reason}\")\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/enterprise/llm.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse hyper::{HeaderMap, header::CONTENT_TYPE};\nuse serde::{Deserialize, Serialize};\nuse std::time::Duration;\nuse utils::config::{Config, http::parse_http_headers};\n\n#[derive(Clone, Debug)]\npub struct AiApiConfig {\n    pub id: String,\n    pub api_type: ApiType,\n    pub url: String,\n    pub model: String,\n    pub timeout: Duration,\n    pub headers: HeaderMap,\n    pub tls_allow_invalid_certs: bool,\n    pub default_temperature: f64,\n}\n\n#[derive(Clone, Copy, Debug)]\npub enum ApiType {\n    ChatCompletion,\n    TextCompletion,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\npub struct ChatCompletionRequest {\n    pub model: String,\n    pub messages: Vec<Message>,\n    pub temperature: f64,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\npub struct Message {\n    pub role: String,\n    pub content: String,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\npub struct ChatCompletionResponse {\n    pub created: i64,\n    pub object: String,\n    pub id: String,\n    pub model: String,\n    pub choices: Vec<ChatCompletionChoice>,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\npub struct ChatCompletionChoice {\n    pub index: i32,\n    pub finish_reason: String,\n    pub message: Message,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\npub struct TextCompletionRequest {\n    pub model: String,\n    pub prompt: String,\n    pub temperature: f64,\n}\n\n#[derive(Deserialize, Debug)]\npub struct TextCompletionResponse {\n    pub created: i64,\n    pub object: String,\n    pub id: String,\n    pub model: String,\n    pub choices: Vec<TextCompletionChoice>,\n}\n\n#[derive(Deserialize, Debug)]\npub struct TextCompletionChoice {\n    pub index: i32,\n    pub finish_reason: String,\n    pub text: String,\n}\n\nimpl AiApiConfig {\n    pub async fn send_request(\n        &self,\n        prompt: impl Into<String>,\n        temperature: Option<f64>,\n    ) -> trc::Result<String> {\n        self.post_api(prompt, temperature).await.map_err(|err| {\n            trc::Error::new(trc::EventType::Ai(trc::AiEvent::ApiError))\n                .id(self.id.clone())\n                .details(\"OpenAPI request failed\")\n                .reason(err)\n        })\n    }\n\n    async fn post_api(\n        &self,\n        prompt: impl Into<String>,\n        temperature: Option<f64>,\n    ) -> Result<String, String> {\n        // Serialize body\n        let body = match self.api_type {\n            ApiType::ChatCompletion => serde_json::to_string(&ChatCompletionRequest {\n                model: self.model.to_string(),\n                messages: vec![Message {\n                    role: \"user\".to_string(),\n                    content: prompt.into(),\n                }],\n                temperature: temperature.unwrap_or(self.default_temperature),\n            })\n            .map_err(|err| format!(\"Failed to serialize request: {}\", err))?,\n            ApiType::TextCompletion => serde_json::to_string(&TextCompletionRequest {\n                model: self.model.to_string(),\n                prompt: prompt.into(),\n                temperature: temperature.unwrap_or(self.default_temperature),\n            })\n            .map_err(|err| format!(\"Failed to serialize request: {}\", err))?,\n        };\n\n        // Send request\n        let response = reqwest::Client::builder()\n            .timeout(self.timeout)\n            .danger_accept_invalid_certs(self.tls_allow_invalid_certs)\n            .build()\n            .map_err(|err| format!(\"Failed to create HTTP client: {}\", err))?\n            .post(&self.url)\n            .headers(self.headers.clone())\n            .body(body)\n            .send()\n            .await\n            .map_err(|err| format!(\"API request to {} failed: {err}\", self.url))?;\n\n        if response.status().is_success() {\n            let bytes = response.bytes().await.map_err(|err| {\n                format!(\"Failed to read response body from {}: {}\", self.url, err)\n            })?;\n\n            match self.api_type {\n                ApiType::ChatCompletion => {\n                    let response = serde_json::from_slice::<ChatCompletionResponse>(&bytes)\n                        .map_err(|err| {\n                            format!(\n                                \"Failed to chat completion parse response from {}: {}\",\n                                self.url, err\n                            )\n                        })?;\n                    response\n                        .choices\n                        .into_iter()\n                        .next()\n                        .map(|choice| choice.message.content)\n                        .filter(|text| !text.is_empty())\n                        .ok_or_else(|| {\n                            format!(\n                                \"Chat completion response from {} did not contain any choices: {}\",\n                                self.url,\n                                std::str::from_utf8(&bytes).unwrap_or_default()\n                            )\n                        })\n                }\n                ApiType::TextCompletion => {\n                    let response = serde_json::from_slice::<TextCompletionResponse>(&bytes)\n                        .map_err(|err| {\n                            format!(\n                                \"Failed to parse text completion response from {}: {}\",\n                                self.url, err\n                            )\n                        })?;\n                    response\n                        .choices\n                        .into_iter()\n                        .next()\n                        .map(|choice| choice.text)\n                        .filter(|text| !text.is_empty())\n                        .ok_or_else(|| {\n                            format!(\n                                \"Text completion response from {} did not contain any choices: {}\",\n                                self.url,\n                                std::str::from_utf8(&bytes).unwrap_or_default()\n                            )\n                        })\n                }\n            }\n        } else {\n            let status = response.status();\n            let bytes = response.bytes().await.unwrap_or_default();\n\n            Err(format!(\n                \"OpenAPI request to {} failed with code {} ({}): {}\",\n                self.url,\n                status.as_u16(),\n                status.canonical_reason().unwrap_or(\"Unknown\"),\n                std::str::from_utf8(&bytes).unwrap_or_default()\n            ))\n        }\n    }\n\n    pub fn parse(config: &mut Config, id: &str) -> Option<Self> {\n        let url = config.value((\"enterprise.ai\", id, \"url\"))?.to_string();\n        let api_type = match config.value((\"enterprise.ai\", id, \"type\"))? {\n            \"chat\" => ApiType::ChatCompletion,\n            \"text\" => ApiType::TextCompletion,\n            _ => {\n                config.new_build_error((\"enterprise.ai\", id, \"type\"), \"Invalid API type\");\n                return None;\n            }\n        };\n\n        let mut headers = parse_http_headers(config, (\"enterprise.ai\", id));\n        headers.insert(CONTENT_TYPE, \"application/json\".parse().unwrap());\n\n        Some(AiApiConfig {\n            id: id.to_string(),\n            api_type,\n            url,\n            headers,\n            model: config\n                .value_require((\"enterprise.ai\", id, \"model\"))?\n                .to_string(),\n            timeout: config\n                .property_or_default((\"enterprise.ai\", id, \"timeout\"), \"2m\")\n                .unwrap_or_else(|| Duration::from_secs(120)),\n            tls_allow_invalid_certs: config\n                .property_or_default((\"enterprise.ai\", id, \"allow-invalid-certs\"), \"false\")\n                .unwrap_or_default(),\n            default_temperature: config\n                .property_or_default((\"enterprise.ai\", id, \"default-temperature\"), \"0.7\")\n                .unwrap_or(0.7),\n        })\n    }\n}\n"
  },
  {
    "path": "crates/common/src/enterprise/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\npub mod alerts;\npub mod config;\npub mod license;\npub mod llm;\npub mod undelete;\n\nuse ahash::{AHashMap, AHashSet};\nuse directory::{\n    QueryParams, Type,\n    backend::internal::{lookup::DirectoryStore, manage::ManageDirectory},\n};\nuse license::LicenseKey;\nuse llm::AiApiConfig;\nuse mail_parser::DateTime;\nuse std::{sync::Arc, time::Duration};\nuse store::Store;\nuse trc::{AddContext, EventType, MetricType};\nuse utils::{HttpLimitResponse, config::cron::SimpleCron, template::Template};\n\nuse crate::{\n    Core, Server, config::groupware::CalendarTemplateVariable, expr::Expression,\n    manager::webadmin::Resource,\n};\n\n#[derive(Clone)]\npub struct Enterprise {\n    pub license: LicenseKey,\n    pub logo_url: Option<String>,\n    pub undelete: Option<Undelete>,\n    pub trace_store: Option<TraceStore>,\n    pub metrics_store: Option<MetricStore>,\n    pub metrics_alerts: Vec<MetricAlert>,\n    pub ai_apis: AHashMap<String, Arc<AiApiConfig>>,\n    pub spam_filter_llm: Option<SpamFilterLlmConfig>,\n    pub template_calendar_alarm: Option<Template<CalendarTemplateVariable>>,\n    pub template_scheduling_email: Option<Template<CalendarTemplateVariable>>,\n    pub template_scheduling_web: Option<Template<CalendarTemplateVariable>>,\n}\n\n#[derive(Debug, Clone)]\npub struct SpamFilterLlmConfig {\n    pub model: Arc<AiApiConfig>,\n    pub temperature: f64,\n    pub prompt: String,\n    pub separator: char,\n    pub index_category: usize,\n    pub index_confidence: Option<usize>,\n    pub index_explanation: Option<usize>,\n    pub categories: AHashSet<String>,\n    pub confidence: AHashSet<String>,\n}\n\n#[derive(Clone)]\npub struct Undelete {\n    pub retention: Duration,\n}\n\n#[derive(Clone)]\npub struct TraceStore {\n    pub retention: Option<Duration>,\n    pub store: Store,\n}\n\n#[derive(Clone)]\npub struct MetricStore {\n    pub retention: Option<Duration>,\n    pub store: Store,\n    pub interval: SimpleCron,\n}\n\n#[derive(Clone, Debug)]\npub struct MetricAlert {\n    pub id: String,\n    pub condition: Expression,\n    pub method: Vec<AlertMethod>,\n}\n\n#[derive(Clone, Debug)]\npub enum AlertMethod {\n    Email {\n        from_name: Option<String>,\n        from_addr: String,\n        to: Vec<String>,\n        subject: AlertContent,\n        body: AlertContent,\n    },\n    Event {\n        message: Option<AlertContent>,\n    },\n}\n\n#[derive(Clone, Debug)]\npub struct AlertContent(pub Vec<AlertContentToken>);\n\n#[derive(Clone, Debug)]\npub enum AlertContentToken {\n    Text(String),\n    Metric(MetricType),\n    Event(EventType),\n}\n\nimpl Core {\n    pub fn is_enterprise_edition(&self) -> bool {\n        self.enterprise\n            .as_ref()\n            .is_some_and(|e| !e.license.is_expired())\n    }\n}\n\nimpl Server {\n    // WARNING: TAMPERING WITH THIS FUNCTION IS STRICTLY PROHIBITED\n    // Any attempt to modify, bypass, or disable this license validation mechanism\n    // constitutes a severe violation of the Stalwart Enterprise License Agreement.\n    // Such actions may result in immediate termination of your license, legal action,\n    // and substantial financial penalties. Stalwart Labs LLC actively monitors for\n    // unauthorized modifications and will pursue all available legal remedies against\n    // violators to the fullest extent of the law, including but not limited to claims\n    // for copyright infringement, breach of contract, and fraud.\n\n    #[inline]\n    pub fn is_enterprise_edition(&self) -> bool {\n        self.core.is_enterprise_edition()\n    }\n\n    pub fn licensed_accounts(&self) -> u32 {\n        self.core\n            .enterprise\n            .as_ref()\n            .map_or(0, |e| e.license.accounts)\n    }\n\n    pub fn log_license_details(&self) {\n        if let Some(enterprise) = &self.core.enterprise {\n            trc::event!(\n                Server(trc::ServerEvent::Licensing),\n                Details = \"Stalwart Enterprise Edition license key is valid\",\n                Domain = enterprise.license.domain.clone(),\n                Total = enterprise.license.accounts,\n                ValidFrom =\n                    DateTime::from_timestamp(enterprise.license.valid_from as i64).to_rfc3339(),\n                ValidTo = DateTime::from_timestamp(enterprise.license.valid_to as i64).to_rfc3339(),\n            );\n        }\n    }\n\n    pub async fn can_create_account(&self) -> trc::Result<bool> {\n        if let Some(enterprise) = &self.core.enterprise {\n            let total_accounts = self\n                .store()\n                .count_principals(None, Type::Individual.into(), None)\n                .await\n                .caused_by(trc::location!())?;\n\n            if total_accounts + 1 > enterprise.license.accounts as u64 {\n                trc::event!(\n                    Server(trc::ServerEvent::Licensing),\n                    Details = \"Account creation not possible: license key account limit reached\",\n                    Domain = enterprise.license.domain.clone(),\n                    Total = total_accounts,\n                    Limit = enterprise.license.accounts,\n                );\n\n                return Ok(false);\n            }\n        }\n\n        Ok(true)\n    }\n\n    pub async fn logo_resource(&self, domain: &str) -> trc::Result<Option<Resource<Vec<u8>>>> {\n        const MAX_IMAGE_SIZE: usize = 1024 * 1024;\n\n        if self.is_enterprise_edition() {\n            let domain = psl::domain_str(domain).unwrap_or(domain);\n            let logo = { self.inner.data.logos.lock().get(domain).cloned() };\n\n            if let Some(logo) = logo {\n                Ok(logo)\n            } else {\n                // Try fetching the logo for the domain\n                let logo_url = if let Some(mut principal) = self\n                    .store()\n                    .query(QueryParams::name(domain).with_return_member_of(false))\n                    .await\n                    .caused_by(trc::location!())?\n                    .filter(|p| p.typ() == Type::Domain)\n                {\n                    if let Some(logo) = principal.picture_mut().filter(|l| l.starts_with(\"http\")) {\n                        std::mem::take(logo).into()\n                    } else if let Some(tenant_id) = principal.tenant() {\n                        if let Some(logo) = self\n                            .store()\n                            .query(QueryParams::id(tenant_id).with_return_member_of(false))\n                            .await\n                            .caused_by(trc::location!())?\n                            .and_then(|mut p| p.picture_mut().map(std::mem::take))\n                            .filter(|l| l.starts_with(\"http\"))\n                        {\n                            logo.clone().into()\n                        } else {\n                            self.default_logo_url()\n                        }\n                    } else {\n                        self.default_logo_url()\n                    }\n                } else {\n                    self.default_logo_url()\n                };\n\n                let mut logo = None;\n                if let Some(logo_url) = logo_url {\n                    let response = reqwest::get(logo_url.as_str()).await.map_err(|err| {\n                        trc::ResourceEvent::DownloadExternal\n                            .into_err()\n                            .details(\"Failed to download logo\")\n                            .reason(err)\n                    })?;\n\n                    let content_type = response\n                        .headers()\n                        .get(reqwest::header::CONTENT_TYPE)\n                        .and_then(|ct| ct.to_str().ok())\n                        .unwrap_or(\"image/svg+xml\")\n                        .to_string();\n\n                    let contents = response\n                        .bytes_with_limit(MAX_IMAGE_SIZE)\n                        .await\n                        .map_err(|err| {\n                            trc::ResourceEvent::DownloadExternal\n                                .into_err()\n                                .details(\"Failed to download logo\")\n                                .reason(err)\n                        })?\n                        .ok_or_else(|| {\n                            trc::ResourceEvent::DownloadExternal\n                                .into_err()\n                                .details(\"Download exceeded maximum size\")\n                        })?;\n\n                    logo = Resource::new(content_type, contents).into();\n                }\n\n                self.inner\n                    .data\n                    .logos\n                    .lock()\n                    .insert(domain.to_string(), logo.clone());\n\n                Ok(logo)\n            }\n        } else {\n            Ok(None)\n        }\n    }\n\n    fn default_logo_url(&self) -> Option<String> {\n        self.core\n            .enterprise\n            .as_ref()\n            .and_then(|e| e.logo_url.as_ref().map(|l| l.into()))\n    }\n}\n"
  },
  {
    "path": "crates/common/src/enterprise/undelete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse crate::Core;\nuse store::{\n    Deserialize, IterateParams, U32_LEN, U64_LEN, ValueKey,\n    write::{AlignedBytes, Archive, BlobOp, ValueClass, key::DeserializeBigEndian, now},\n};\nuse trc::AddContext;\nuse types::blob_hash::{BLOB_HASH_LEN, BlobHash};\n\npub struct DeletedBlob {\n    pub hash: BlobHash,\n    pub expires_at: u64,\n    pub item: DeletedItem,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct DeletedItem {\n    pub typ: DeletedItemType,\n    pub size: u32,\n    pub deleted_at: u64,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub enum DeletedItemType {\n    Email {\n        from: Box<str>,\n        subject: Box<str>,\n        received_at: u64,\n    },\n    FileNode {\n        name: Box<str>,\n    },\n    CalendarEvent {\n        title: Box<str>,\n        start_time: u64,\n    },\n    ContactCard {\n        name: Box<str>,\n    },\n    SieveScript {\n        name: Box<str>,\n    },\n}\n\nimpl Core {\n    pub async fn list_deleted(&self, account_id: u32) -> trc::Result<Vec<DeletedBlob>> {\n        let from_key = ValueKey {\n            account_id,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Blob(BlobOp::Undelete {\n                hash: BlobHash::default(),\n                until: 0,\n            }),\n        };\n        let to_key = ValueKey {\n            account_id,\n            collection: 0,\n            document_id: u32::MAX,\n            class: ValueClass::Blob(BlobOp::Undelete {\n                hash: BlobHash::new_max(),\n                until: u64::MAX,\n            }),\n        };\n\n        let now = now();\n        let mut results = Vec::new();\n\n        self.storage\n            .data\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending(),\n                |key, value| {\n                    let expires_at = key.deserialize_be_u64(key.len() - U64_LEN)?;\n                    if expires_at > now {\n                        let item = <Archive<AlignedBytes> as Deserialize>::deserialize(value)\n                            .and_then(|bytes| bytes.deserialize::<DeletedItem>())\n                            .add_context(|ctx| ctx.ctx(trc::Key::Key, key))?;\n\n                        results.push(DeletedBlob {\n                            hash: BlobHash::try_from_hash_slice(\n                                key.get(U32_LEN + 1..U32_LEN + 1 + BLOB_HASH_LEN)\n                                    .ok_or_else(|| {\n                                        trc::Error::corrupted_key(\n                                            key,\n                                            value.into(),\n                                            trc::location!(),\n                                        )\n                                    })?,\n                            )\n                            .unwrap(),\n                            expires_at,\n                            item,\n                        });\n                    }\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(results)\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/eval.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{cmp::Ordering, fmt::Display};\n\nuse compact_str::{CompactString, ToCompactString, format_compact};\nuse hyper::StatusCode;\nuse trc::EvalEvent;\n\nuse crate::Server;\n\nuse super::{\n    BinaryOperator, Constant, Expression, ExpressionItem, Setting, StringCow, UnaryOperator,\n    Variable,\n    functions::{FUNCTIONS, ResolveVariable},\n    if_block::IfBlock,\n};\n\nimpl Server {\n    pub async fn eval_if<'x, R: TryFrom<Variable<'x>>, V: ResolveVariable>(\n        &'x self,\n        if_block: &'x IfBlock,\n        resolver: &'x V,\n        session_id: u64,\n    ) -> Option<R> {\n        if if_block.is_empty() {\n            trc::event!(\n                Eval(EvalEvent::Result),\n                SpanId = session_id,\n                Id = if_block.key.clone(),\n                Result = \"\"\n            );\n\n            return None;\n        }\n\n        match (EvalContext {\n            resolver,\n            core: self,\n            expr: if_block,\n            captures: Vec::new(),\n            session_id,\n        })\n        .eval()\n        .await\n        {\n            Ok(result) => {\n                trc::event!(\n                    Eval(EvalEvent::Result),\n                    SpanId = session_id,\n                    Id = if_block.key.clone(),\n                    Result = format!(\"{result:?}\"),\n                );\n\n                match result.try_into() {\n                    Ok(value) => Some(value),\n                    Err(_) => {\n                        trc::event!(\n                            Eval(EvalEvent::Result),\n                            SpanId = session_id,\n                            Id = if_block.key.clone(),\n                            Result = \"\",\n                        );\n\n                        None\n                    }\n                }\n            }\n            Err(err) => {\n                trc::event!(\n                    Eval(EvalEvent::Error),\n                    SpanId = session_id,\n                    Id = if_block.key.clone(),\n                    CausedBy = err,\n                );\n\n                None\n            }\n        }\n    }\n\n    pub async fn eval_expr<'x, R: TryFrom<Variable<'x>>, V: ResolveVariable>(\n        &'x self,\n        expr: &'x Expression,\n        resolver: &'x V,\n        expr_id: &str,\n        session_id: u64,\n    ) -> Option<R> {\n        if expr.is_empty() {\n            return None;\n        }\n\n        match (EvalContext {\n            resolver,\n            core: self,\n            expr,\n            captures: &mut Vec::new(),\n            session_id,\n        })\n        .eval()\n        .await\n        {\n            Ok(result) => {\n                trc::event!(\n                    Eval(EvalEvent::Result),\n                    SpanId = session_id,\n                    Id = expr_id.to_compact_string(),\n                    Result = format!(\"{result:?}\"),\n                );\n\n                match result.try_into() {\n                    Ok(value) => Some(value),\n                    Err(_) => {\n                        trc::event!(\n                            Eval(EvalEvent::Error),\n                            SpanId = session_id,\n                            Id = expr_id.to_compact_string(),\n                            Details = \"Failed to convert result\",\n                        );\n\n                        None\n                    }\n                }\n            }\n            Err(err) => {\n                trc::event!(\n                    Eval(EvalEvent::Error),\n                    SpanId = session_id,\n                    Id = expr_id.to_compact_string(),\n                    CausedBy = err,\n                );\n\n                None\n            }\n        }\n    }\n}\n\nstruct EvalContext<'x, V: ResolveVariable, T, C> {\n    resolver: &'x V,\n    core: &'x Server,\n    expr: &'x T,\n    captures: C,\n    session_id: u64,\n}\n\nimpl<'x, V: ResolveVariable> EvalContext<'x, V, IfBlock, Vec<CompactString>> {\n    async fn eval(&mut self) -> trc::Result<Variable<'x>> {\n        for if_then in &self.expr.if_then {\n            if (EvalContext {\n                resolver: self.resolver,\n                core: self.core,\n                expr: &if_then.expr,\n                captures: &mut self.captures,\n                session_id: self.session_id,\n            })\n            .eval()\n            .await?\n            .to_bool()\n            {\n                return (EvalContext {\n                    resolver: self.resolver,\n                    core: self.core,\n                    expr: &if_then.then,\n                    captures: &mut self.captures,\n                    session_id: self.session_id,\n                })\n                .eval()\n                .await;\n            }\n        }\n\n        (EvalContext {\n            resolver: self.resolver,\n            core: self.core,\n            expr: &self.expr.default,\n            captures: &mut self.captures,\n            session_id: self.session_id,\n        })\n        .eval()\n        .await\n    }\n}\n\nimpl<'x, V: ResolveVariable> EvalContext<'x, V, Expression, &mut Vec<CompactString>> {\n    async fn eval(&mut self) -> trc::Result<Variable<'x>> {\n        let mut stack = Vec::new();\n        let mut exprs = self.expr.items.iter();\n\n        while let Some(expr) = exprs.next() {\n            match expr {\n                ExpressionItem::Variable(v) => {\n                    stack.push(self.resolver.resolve_variable(*v));\n                }\n                ExpressionItem::Global(v) => {\n                    stack.push(self.resolver.resolve_global(v));\n                }\n                ExpressionItem::Constant(val) => {\n                    stack.push(Variable::from(val));\n                }\n                ExpressionItem::Capture(v) => {\n                    stack.push(Variable::String(StringCow::Owned(\n                        self.captures\n                            .get(*v as usize)\n                            .map(|v| v.as_str())\n                            .unwrap_or_default()\n                            .to_compact_string(),\n                    )));\n                }\n                ExpressionItem::Setting(setting) => match setting {\n                    Setting::Hostname => {\n                        stack.push(self.core.core.network.server_name.as_str().into())\n                    }\n                    Setting::ReportDomain => {\n                        stack.push(self.core.core.network.report_domain.as_str().into())\n                    }\n                    Setting::NodeId => stack.push(self.core.core.network.node_id.into()),\n                    Setting::Other(key) => stack.push(\n                        self.core\n                            .core\n                            .storage\n                            .config\n                            .get(key)\n                            .await?\n                            .unwrap_or_default()\n                            .to_compact_string()\n                            .into(),\n                    ),\n                },\n                ExpressionItem::UnaryOperator(op) => {\n                    let value = stack.pop().unwrap_or_default();\n                    stack.push(match op {\n                        UnaryOperator::Not => value.op_not(),\n                        UnaryOperator::Minus => value.op_minus(),\n                    });\n                }\n                ExpressionItem::BinaryOperator(op) => {\n                    let right = stack.pop().unwrap_or_default();\n                    let left = stack.pop().unwrap_or_default();\n                    stack.push(match op {\n                        BinaryOperator::Add => left.op_add(right),\n                        BinaryOperator::Subtract => left.op_subtract(right),\n                        BinaryOperator::Multiply => left.op_multiply(right),\n                        BinaryOperator::Divide => left.op_divide(right),\n                        BinaryOperator::And => left.op_and(right),\n                        BinaryOperator::Or => left.op_or(right),\n                        BinaryOperator::Xor => left.op_xor(right),\n                        BinaryOperator::Eq => left.op_eq(right),\n                        BinaryOperator::Ne => left.op_ne(right),\n                        BinaryOperator::Lt => left.op_lt(right),\n                        BinaryOperator::Le => left.op_le(right),\n                        BinaryOperator::Gt => left.op_gt(right),\n                        BinaryOperator::Ge => left.op_ge(right),\n                    });\n                }\n                ExpressionItem::Function { id, num_args } => {\n                    let num_args = *num_args as usize;\n\n                    let mut arguments = Variable::array(num_args);\n                    for arg_num in 0..num_args {\n                        arguments[num_args - arg_num - 1] = stack.pop().unwrap_or_default();\n                    }\n\n                    let result = if let Some((_, fnc, _)) = FUNCTIONS.get(*id as usize) {\n                        (fnc)(arguments)\n                    } else {\n                        Box::pin(self.core.eval_fnc(\n                            *id - FUNCTIONS.len() as u32,\n                            arguments,\n                            self.session_id,\n                        ))\n                        .await?\n                    };\n\n                    stack.push(result);\n                }\n                ExpressionItem::JmpIf { val, pos } => {\n                    if stack.last().is_some_and(|v| v.to_bool()) == *val {\n                        for _ in 0..*pos {\n                            exprs.next();\n                        }\n                    }\n                }\n                ExpressionItem::ArrayAccess => {\n                    let index = stack\n                        .pop()\n                        .unwrap_or_default()\n                        .to_usize()\n                        .unwrap_or_default();\n                    let array = stack.pop().unwrap_or_default().into_array();\n                    stack.push(array.into_iter().nth(index).unwrap_or_default());\n                }\n                ExpressionItem::ArrayBuild(num_items) => {\n                    let num_items = *num_items as usize;\n                    let mut items = Variable::array(num_items);\n                    for arg_num in 0..num_items {\n                        items[num_items - arg_num - 1] = stack.pop().unwrap_or_default();\n                    }\n                    stack.push(Variable::Array(items));\n                }\n                ExpressionItem::Regex(regex) => {\n                    self.captures.clear();\n                    let value = stack.pop().unwrap_or_default().into_string();\n\n                    if let Some(captures_) = regex.captures(value.as_ref()) {\n                        for capture in captures_.iter() {\n                            self.captures\n                                .push(capture.map_or(\"\", |m| m.as_str()).to_compact_string());\n                        }\n                    }\n\n                    stack.push(Variable::Integer(!self.captures.is_empty() as i64));\n                }\n            }\n        }\n\n        Ok(stack.pop().unwrap_or_default())\n    }\n}\n\nimpl Expression {\n    pub fn is_empty(&self) -> bool {\n        self.items.is_empty()\n    }\n\n    pub fn items(&self) -> &[ExpressionItem] {\n        &self.items\n    }\n}\n\nimpl<'x> Variable<'x> {\n    pub fn op_add(self, other: Variable<'x>) -> Variable<'x> {\n        match (self, other) {\n            (Variable::Integer(a), Variable::Integer(b)) => Variable::Integer(a.saturating_add(b)),\n            (Variable::Float(a), Variable::Float(b)) => Variable::Float(a + b),\n            (Variable::Integer(i), Variable::Float(f))\n            | (Variable::Float(f), Variable::Integer(i)) => Variable::Float(i as f64 + f),\n            (Variable::Array(a), Variable::Array(b)) => {\n                Variable::Array(a.into_iter().chain(b).collect::<Vec<_>>())\n            }\n            (Variable::Array(a), b) => {\n                Variable::Array(a.into_iter().chain([b]).collect::<Vec<_>>())\n            }\n            (a, Variable::Array(b)) => {\n                Variable::Array([a].into_iter().chain(b).collect::<Vec<_>>())\n            }\n            (Variable::String(a), b) => {\n                if !a.is_empty() {\n                    Variable::String(StringCow::Owned(format_compact!(\"{}{}\", a, b)))\n                } else {\n                    b\n                }\n            }\n            (a, Variable::String(b)) => {\n                if !b.is_empty() {\n                    Variable::String(StringCow::Owned(format_compact!(\"{}{}\", a, b)))\n                } else {\n                    a\n                }\n            }\n        }\n    }\n\n    pub fn op_subtract(self, other: Variable<'x>) -> Variable<'x> {\n        match (self, other) {\n            (Variable::Integer(a), Variable::Integer(b)) => Variable::Integer(a.saturating_sub(b)),\n            (Variable::Float(a), Variable::Float(b)) => Variable::Float(a - b),\n            (Variable::Integer(a), Variable::Float(b)) => Variable::Float(a as f64 - b),\n            (Variable::Float(a), Variable::Integer(b)) => Variable::Float(a - b as f64),\n            (Variable::Array(a), b) | (b, Variable::Array(a)) => {\n                Variable::Array(a.into_iter().filter(|v| v != &b).collect::<Vec<_>>())\n            }\n            (a, b) => a.parse_number().op_subtract(b.parse_number()),\n        }\n    }\n\n    pub fn op_multiply(self, other: Variable<'x>) -> Variable<'x> {\n        match (self, other) {\n            (Variable::Integer(a), Variable::Integer(b)) => Variable::Integer(a.saturating_mul(b)),\n            (Variable::Float(a), Variable::Float(b)) => Variable::Float(a * b),\n            (Variable::Integer(i), Variable::Float(f))\n            | (Variable::Float(f), Variable::Integer(i)) => Variable::Float(i as f64 * f),\n            (a, b) => a.parse_number().op_multiply(b.parse_number()),\n        }\n    }\n\n    pub fn op_divide(self, other: Variable<'x>) -> Variable<'x> {\n        match (self, other) {\n            (Variable::Integer(a), Variable::Integer(b)) => {\n                Variable::Float(if b != 0 { a as f64 / b as f64 } else { 0.0 })\n            }\n            (Variable::Float(a), Variable::Float(b)) => {\n                Variable::Float(if b != 0.0 { a / b } else { 0.0 })\n            }\n            (Variable::Integer(a), Variable::Float(b)) => {\n                Variable::Float(if b != 0.0 { a as f64 / b } else { 0.0 })\n            }\n            (Variable::Float(a), Variable::Integer(b)) => {\n                Variable::Float(if b != 0 { a / b as f64 } else { 0.0 })\n            }\n            (a, b) => a.parse_number().op_divide(b.parse_number()),\n        }\n    }\n\n    pub fn op_and(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self.to_bool() & other.to_bool()))\n    }\n\n    pub fn op_or(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self.to_bool() | other.to_bool()))\n    }\n\n    pub fn op_xor(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self.to_bool() ^ other.to_bool()))\n    }\n\n    pub fn op_eq(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self == other))\n    }\n\n    pub fn op_ne(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self != other))\n    }\n\n    pub fn op_lt(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self < other))\n    }\n\n    pub fn op_le(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self <= other))\n    }\n\n    pub fn op_gt(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self > other))\n    }\n\n    pub fn op_ge(self, other: Variable) -> Variable {\n        Variable::Integer(i64::from(self >= other))\n    }\n\n    pub fn op_not(self) -> Variable<'static> {\n        Variable::Integer(i64::from(!self.to_bool()))\n    }\n\n    pub fn op_minus(self) -> Variable<'static> {\n        match self {\n            Variable::Integer(n) => Variable::Integer(-n),\n            Variable::Float(n) => Variable::Float(-n),\n            _ => self.parse_number().op_minus(),\n        }\n    }\n\n    pub fn parse_number(&self) -> Variable<'static> {\n        match self {\n            Variable::String(s) if !s.is_empty() => {\n                if let Ok(n) = s.as_str().parse::<i64>() {\n                    Variable::Integer(n)\n                } else if let Ok(n) = s.as_str().parse::<f64>() {\n                    Variable::Float(n)\n                } else {\n                    Variable::Integer(0)\n                }\n            }\n            Variable::Integer(n) => Variable::Integer(*n),\n            Variable::Float(n) => Variable::Float(*n),\n            Variable::Array(l) => Variable::Integer(l.is_empty() as i64),\n            _ => Variable::Integer(0),\n        }\n    }\n\n    #[inline(always)]\n    fn array(num_items: usize) -> Vec<Variable<'static>> {\n        let mut items = Vec::with_capacity(num_items);\n        for _ in 0..num_items {\n            items.push(Variable::Integer(0));\n        }\n        items\n    }\n\n    pub fn to_ref<'y: 'x>(&'y self) -> Variable<'x> {\n        match self {\n            Variable::String(s) => Variable::String(StringCow::Borrowed(s.as_str())),\n            Variable::Integer(n) => Variable::Integer(*n),\n            Variable::Float(n) => Variable::Float(*n),\n            Variable::Array(l) => Variable::Array(l.iter().map(|v| v.to_ref()).collect::<Vec<_>>()),\n        }\n    }\n\n    pub fn to_bool(&self) -> bool {\n        match self {\n            Variable::Float(f) => *f != 0.0,\n            Variable::Integer(n) => *n != 0,\n            Variable::String(s) => !s.is_empty(),\n            Variable::Array(a) => !a.is_empty(),\n        }\n    }\n\n    pub fn to_string(&'_ self) -> StringCow<'_> {\n        match self {\n            Variable::String(s) => StringCow::Borrowed(s.as_str()),\n            Variable::Integer(n) => StringCow::Owned(n.to_compact_string()),\n            Variable::Float(n) => StringCow::Owned(n.to_compact_string()),\n            Variable::Array(l) => {\n                let mut result = CompactString::with_capacity(self.len() * 10);\n                for item in l {\n                    if !result.is_empty() {\n                        result.push_str(\"\\r\\n\");\n                    }\n                    match item {\n                        Variable::String(v) => result.push_str(v.as_str()),\n                        Variable::Integer(v) => result.push_str(&v.to_compact_string()),\n                        Variable::Float(v) => result.push_str(&v.to_compact_string()),\n                        Variable::Array(_) => {}\n                    }\n                }\n                StringCow::Owned(result)\n            }\n        }\n    }\n\n    pub fn into_string(self) -> StringCow<'x> {\n        match self {\n            Variable::String(s) => s,\n            Variable::Integer(n) => StringCow::Owned(n.to_compact_string()),\n            Variable::Float(n) => StringCow::Owned(n.to_compact_string()),\n            Variable::Array(l) => {\n                let mut result = CompactString::with_capacity(l.len() * 10);\n                for item in l {\n                    if !result.is_empty() {\n                        result.push_str(\"\\r\\n\");\n                    }\n                    match item {\n                        Variable::String(v) => result.push_str(v.as_ref()),\n                        Variable::Integer(v) => result.push_str(&v.to_compact_string()),\n                        Variable::Float(v) => result.push_str(&v.to_compact_string()),\n                        Variable::Array(_) => {}\n                    }\n                }\n                StringCow::Owned(result)\n            }\n        }\n    }\n\n    pub fn to_integer(&self) -> Option<i64> {\n        match self {\n            Variable::Integer(n) => Some(*n),\n            Variable::Float(n) => Some(*n as i64),\n            Variable::String(s) if !s.is_empty() => s.as_str().parse::<i64>().ok(),\n            _ => None,\n        }\n    }\n\n    pub fn to_usize(&self) -> Option<usize> {\n        match self {\n            Variable::Integer(n) => Some(*n as usize),\n            Variable::Float(n) => Some(*n as usize),\n            Variable::String(s) if !s.is_empty() => s.as_str().parse::<usize>().ok(),\n            _ => None,\n        }\n    }\n\n    pub fn len(&self) -> usize {\n        match self {\n            Variable::String(s) => s.len(),\n            Variable::Integer(_) | Variable::Float(_) => 2,\n            Variable::Array(l) => l.iter().map(|v| v.len() + 2).sum(),\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        match self {\n            Variable::String(s) => s.is_empty(),\n            _ => false,\n        }\n    }\n\n    pub fn as_array(&'_ self) -> Option<&'_ [Variable<'_>]> {\n        match self {\n            Variable::Array(l) => Some(l),\n            _ => None,\n        }\n    }\n\n    pub fn into_array(self) -> Vec<Variable<'x>> {\n        match self {\n            Variable::Array(l) => l,\n            v if !v.is_empty() => vec![v],\n            _ => vec![],\n        }\n    }\n\n    pub fn to_array(&self) -> Vec<Variable<'_>> {\n        match self {\n            Variable::Array(l) => l.iter().map(|v| v.to_ref()).collect::<Vec<_>>(),\n            v if !v.is_empty() => vec![v.to_ref()],\n            _ => vec![],\n        }\n    }\n\n    pub fn into_owned(self) -> Variable<'static> {\n        match self {\n            Variable::String(s) => Variable::String(StringCow::Owned(s.into_owned())),\n            Variable::Integer(n) => Variable::Integer(n),\n            Variable::Float(n) => Variable::Float(n),\n            Variable::Array(l) => Variable::Array(l.into_iter().map(|v| v.into_owned()).collect()),\n        }\n    }\n}\n\nimpl PartialEq for Variable<'_> {\n    fn eq(&self, other: &Self) -> bool {\n        match (self, other) {\n            (Self::Integer(a), Self::Integer(b)) => a == b,\n            (Self::Float(a), Self::Float(b)) => a == b,\n            (Self::Integer(a), Self::Float(b)) | (Self::Float(b), Self::Integer(a)) => {\n                *a as f64 == *b\n            }\n            (Self::String(a), Self::String(b)) => a.as_str() == b.as_str(),\n            (Self::String(_), Self::Integer(_) | Self::Float(_)) => &self.parse_number() == other,\n            (Self::Integer(_) | Self::Float(_), Self::String(_)) => self == &other.parse_number(),\n            (Self::Array(a), Self::Array(b)) => a == b,\n            _ => false,\n        }\n    }\n}\n\nimpl Eq for Variable<'_> {}\n\n#[allow(clippy::non_canonical_partial_ord_impl)]\nimpl PartialOrd for Variable<'_> {\n    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {\n        match (self, other) {\n            (Self::Integer(a), Self::Integer(b)) => a.partial_cmp(b),\n            (Self::Float(a), Self::Float(b)) => a.partial_cmp(b),\n            (Self::Integer(a), Self::Float(b)) => (*a as f64).partial_cmp(b),\n            (Self::Float(a), Self::Integer(b)) => a.partial_cmp(&(*b as f64)),\n            (Self::String(a), Self::String(b)) => a.as_str().partial_cmp(b.as_str()),\n            (Self::String(_), Self::Integer(_) | Self::Float(_)) => {\n                self.parse_number().partial_cmp(other)\n            }\n            (Self::Integer(_) | Self::Float(_), Self::String(_)) => {\n                self.partial_cmp(&other.parse_number())\n            }\n            (Self::Array(a), Self::Array(b)) => a.partial_cmp(b),\n            (Self::Array(_) | Self::String(_), _) => Ordering::Greater.into(),\n            (_, Self::Array(_)) => Ordering::Less.into(),\n        }\n    }\n}\n\nimpl Ord for Variable<'_> {\n    fn cmp(&self, other: &Self) -> std::cmp::Ordering {\n        self.partial_cmp(other).unwrap_or(Ordering::Greater)\n    }\n}\n\nimpl Display for Variable<'_> {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Variable::String(v) => v.fmt(f),\n            Variable::Integer(v) => v.fmt(f),\n            Variable::Float(v) => v.fmt(f),\n            Variable::Array(v) => {\n                for (i, v) in v.iter().enumerate() {\n                    if i > 0 {\n                        f.write_str(\"\\n\")?;\n                    }\n                    v.fmt(f)?;\n                }\n                Ok(())\n            }\n        }\n    }\n}\n\nimpl<'x> From<&'x Constant> for Variable<'x> {\n    fn from(value: &'x Constant) -> Self {\n        match value {\n            Constant::Integer(i) => Variable::Integer(*i),\n            Constant::Float(f) => Variable::Float(*f),\n            Constant::String(s) => Variable::String(StringCow::Borrowed(s.as_str())),\n        }\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for CompactString {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        if let Variable::String(s) = value {\n            Ok(match s {\n                StringCow::Borrowed(v) => v.into(),\n                StringCow::Owned(v) => v,\n            })\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for String {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        if let Variable::String(s) = value {\n            Ok(match s {\n                StringCow::Borrowed(v) => v.to_string(),\n                StringCow::Owned(v) => v.into_string(),\n            })\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl<'x> From<Variable<'x>> for bool {\n    fn from(val: Variable<'x>) -> Self {\n        val.to_bool()\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for i64 {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        value.to_integer().ok_or(())\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for u64 {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        value.to_integer().map(|v| v as u64).ok_or(())\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for usize {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        value.to_usize().ok_or(())\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for StatusCode {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value.to_integer() {\n            Some(v) => match StatusCode::from_u16(v as u16) {\n                Ok(status) => Ok(status),\n                Err(_) => Err(()),\n            },\n            None => Err(()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/functions/array.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::expr::Variable;\n\npub(crate) fn fn_count(v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::Array(a) => a.len(),\n        v => {\n            if !v.is_empty() {\n                1\n            } else {\n                0\n            }\n        }\n    }\n    .into()\n}\n\npub(crate) fn fn_sort(mut v: Vec<Variable>) -> Variable {\n    let is_asc = v[1].to_bool();\n    let mut arr = v.remove(0).into_array();\n    if is_asc {\n        arr.sort_unstable_by(|a, b| b.cmp(a));\n    } else {\n        arr.sort_unstable();\n    }\n    arr.into()\n}\n\npub(crate) fn fn_dedup(mut v: Vec<Variable>) -> Variable {\n    let arr = v.remove(0).into_array();\n    let mut result = Vec::with_capacity(arr.len());\n\n    for item in arr {\n        if !result.contains(&item) {\n            result.push(item);\n        }\n    }\n\n    result.into()\n}\n\npub(crate) fn fn_is_intersect(v: Vec<Variable>) -> Variable {\n    match (&v[0], &v[1]) {\n        (Variable::Array(a), Variable::Array(b)) => a.iter().any(|x| b.contains(x)),\n        (Variable::Array(a), item) | (item, Variable::Array(a)) => a.contains(item),\n        _ => false,\n    }\n    .into()\n}\n\npub(crate) fn fn_winnow(mut v: Vec<Variable>) -> Variable {\n    match v.remove(0) {\n        Variable::Array(a) => a\n            .into_iter()\n            .filter(|i| !i.is_empty())\n            .collect::<Vec<_>>()\n            .into(),\n        v => v,\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/functions/asynch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{cmp::Ordering, net::IpAddr, vec::IntoIter};\n\nuse compact_str::{CompactString, ToCompactString};\nuse directory::backend::RcptType;\nuse mail_auth::IpLookupStrategy;\nuse store::{Deserialize, Rows, Value, dispatch::lookup::KeyValue};\nuse trc::AddContext;\n\nuse crate::{Server, expr::StringCow};\n\nuse super::*;\n\nimpl Server {\n    pub(crate) async fn eval_fnc<'x>(\n        &self,\n        fnc_id: u32,\n        params: Vec<Variable<'x>>,\n        session_id: u64,\n    ) -> trc::Result<Variable<'x>> {\n        let mut params = FncParams::new(params);\n\n        match fnc_id {\n            F_IS_LOCAL_DOMAIN => {\n                let directory = params.next_as_string();\n                let domain = params.next_as_string();\n\n                self.get_directory_or_default(directory.as_ref(), session_id)\n                    .is_local_domain(domain.as_ref())\n                    .await\n                    .caused_by(trc::location!())\n                    .map(|v| v.into())\n            }\n            F_IS_LOCAL_ADDRESS => {\n                let directory = params.next_as_string();\n                let address = params.next_as_string();\n\n                self.get_directory_or_default(directory.as_ref(), session_id)\n                    .rcpt(address.as_ref())\n                    .await\n                    .caused_by(trc::location!())\n                    .map(|v| (v != RcptType::Invalid).into())\n            }\n            F_KEY_GET => {\n                let store = params.next_as_string();\n                let key = params.next_as_string();\n\n                self.get_in_memory_store_or_default(store.as_str(), session_id)\n                    .key_get::<VariableWrapper>(key.as_str())\n                    .await\n                    .map(|value| value.map(|v| v.into_inner()).unwrap_or_default())\n                    .caused_by(trc::location!())\n            }\n            F_KEY_EXISTS => {\n                let store = params.next_as_string();\n                let key = params.next_as_string();\n\n                self.get_in_memory_store_or_default(store.as_str(), session_id)\n                    .key_exists(key.as_str())\n                    .await\n                    .caused_by(trc::location!())\n                    .map(|v| v.into())\n            }\n            F_KEY_SET => {\n                let store = params.next_as_string();\n                let key = params.next_as_string();\n                let value = params.next_as_string();\n\n                self.get_in_memory_store_or_default(store.as_ref(), session_id)\n                    .key_set(KeyValue::new(\n                        key.as_bytes().to_vec(),\n                        value.as_bytes().to_vec(),\n                    ))\n                    .await\n                    .map(|_| true)\n                    .caused_by(trc::location!())\n                    .map(|v| v.into())\n            }\n            F_COUNTER_INCR => {\n                let store = params.next_as_string();\n                let key = params.next_as_string();\n                let value = params.next_as_integer();\n\n                self.get_in_memory_store_or_default(store.as_ref(), session_id)\n                    .counter_incr(KeyValue::new(key.into_owned(), value), true)\n                    .await\n                    .map(Variable::Integer)\n                    .caused_by(trc::location!())\n            }\n            F_COUNTER_GET => {\n                let store = params.next_as_string();\n                let key = params.next_as_string();\n\n                self.get_in_memory_store_or_default(store.as_ref(), session_id)\n                    .counter_get(key.as_bytes().to_vec())\n                    .await\n                    .map(Variable::Integer)\n                    .caused_by(trc::location!())\n            }\n            F_DNS_QUERY => self.dns_query(params).await,\n            F_SQL_QUERY => self.sql_query(params, session_id).await,\n            _ => Ok(Variable::default()),\n        }\n    }\n\n    async fn sql_query<'x>(\n        &self,\n        mut arguments: FncParams<'x>,\n        session_id: u64,\n    ) -> trc::Result<Variable<'x>> {\n        let store = self.get_data_store(arguments.next_as_string().as_ref(), session_id);\n        let query = arguments.next_as_string();\n\n        if query.is_empty() {\n            return Err(trc::EventType::Eval(trc::EvalEvent::Error)\n                .into_err()\n                .details(\"Empty query string\"));\n        }\n\n        // Obtain arguments\n        let arguments = match arguments.next() {\n            Variable::Array(l) => l.into_iter().map(to_store_value).collect(),\n            v => vec![to_store_value(v)],\n        };\n\n        // Run query\n        if query\n            .as_bytes()\n            .get(..6)\n            .is_some_and(|q| q.eq_ignore_ascii_case(b\"SELECT\"))\n        {\n            let mut rows = store\n                .sql_query::<Rows>(query.as_str(), arguments)\n                .await\n                .caused_by(trc::location!())?;\n            Ok(match rows.rows.len().cmp(&1) {\n                Ordering::Equal => {\n                    let mut row = rows.rows.pop().unwrap().values;\n                    match row.len().cmp(&1) {\n                        Ordering::Equal if !matches!(row.first(), Some(Value::Null)) => {\n                            row.pop().map(into_variable).unwrap()\n                        }\n                        Ordering::Less => Variable::default(),\n                        _ => {\n                            Variable::Array(row.into_iter().map(into_variable).collect::<Vec<_>>())\n                        }\n                    }\n                }\n                Ordering::Less => Variable::default(),\n                Ordering::Greater => rows\n                    .rows\n                    .into_iter()\n                    .map(|r| {\n                        Variable::Array(r.values.into_iter().map(into_variable).collect::<Vec<_>>())\n                    })\n                    .collect::<Vec<_>>()\n                    .into(),\n            })\n        } else {\n            store\n                .sql_query::<usize>(query.as_str(), arguments)\n                .await\n                .caused_by(trc::location!())\n                .map(|v| v.into())\n        }\n    }\n\n    async fn dns_query<'x>(&self, mut arguments: FncParams<'x>) -> trc::Result<Variable<'x>> {\n        let entry = arguments.next_as_string();\n        let record_type = arguments.next_as_string();\n\n        if record_type.as_str().eq_ignore_ascii_case(\"ip\") {\n            self.core\n                .smtp\n                .resolvers\n                .dns\n                .ip_lookup(\n                    entry.as_ref(),\n                    IpLookupStrategy::Ipv4thenIpv6,\n                    10,\n                    Some(&self.inner.cache.dns_ipv4),\n                    Some(&self.inner.cache.dns_ipv6),\n                )\n                .await\n                .map_err(|err| trc::Error::from(err).caused_by(trc::location!()))\n                .map(|result| {\n                    result\n                        .iter()\n                        .map(|ip| Variable::from(ip.to_compact_string()))\n                        .collect::<Vec<_>>()\n                        .into()\n                })\n        } else if record_type.as_str().eq_ignore_ascii_case(\"mx\") {\n            self.core\n                .smtp\n                .resolvers\n                .dns\n                .mx_lookup(entry.as_str(), Some(&self.inner.cache.dns_mx))\n                .await\n                .map_err(|err| trc::Error::from(err).caused_by(trc::location!()))\n                .map(|result| {\n                    result\n                        .iter()\n                        .flat_map(|mx| {\n                            mx.exchanges.iter().map(|host| {\n                                Variable::String(StringCow::Owned(\n                                    host.strip_suffix('.')\n                                        .unwrap_or(host.as_str())\n                                        .to_compact_string(),\n                                ))\n                            })\n                        })\n                        .collect::<Vec<_>>()\n                        .into()\n                })\n        } else if record_type.as_str().eq_ignore_ascii_case(\"txt\") {\n            self.core\n                .smtp\n                .resolvers\n                .dns\n                .txt_raw_lookup(entry.as_str())\n                .await\n                .map_err(|err| trc::Error::from(err).caused_by(trc::location!()))\n                .map(|result| Variable::from(CompactString::from_utf8(result).unwrap_or_default()))\n        } else if record_type.as_str().eq_ignore_ascii_case(\"ptr\") {\n            self.core\n                .smtp\n                .resolvers\n                .dns\n                .ptr_lookup(\n                    entry.as_str().parse::<IpAddr>().map_err(|err| {\n                        trc::EventType::Eval(trc::EvalEvent::Error)\n                            .into_err()\n                            .details(\"Failed to parse IP address\")\n                            .reason(err)\n                    })?,\n                    Some(&self.inner.cache.dns_ptr),\n                )\n                .await\n                .map_err(|err| trc::Error::from(err).caused_by(trc::location!()))\n                .map(|result| {\n                    result\n                        .iter()\n                        .map(|host| Variable::from(host.to_compact_string()))\n                        .collect::<Vec<_>>()\n                        .into()\n                })\n        } else if record_type.as_str().eq_ignore_ascii_case(\"ipv4\") {\n            self.core\n                .smtp\n                .resolvers\n                .dns\n                .ipv4_lookup(entry.as_str(), Some(&self.inner.cache.dns_ipv4))\n                .await\n                .map_err(|err| trc::Error::from(err).caused_by(trc::location!()))\n                .map(|result| {\n                    result\n                        .iter()\n                        .map(|ip| Variable::from(ip.to_compact_string()))\n                        .collect::<Vec<_>>()\n                        .into()\n                })\n        } else if record_type.as_str().eq_ignore_ascii_case(\"ipv6\") {\n            self.core\n                .smtp\n                .resolvers\n                .dns\n                .ipv6_lookup(entry.as_str(), Some(&self.inner.cache.dns_ipv6))\n                .await\n                .map_err(|err| trc::Error::from(err).caused_by(trc::location!()))\n                .map(|result| {\n                    result\n                        .iter()\n                        .map(|ip| Variable::from(ip.to_compact_string()))\n                        .collect::<Vec<_>>()\n                        .into()\n                })\n        } else {\n            Ok(Variable::default())\n        }\n    }\n}\n\nstruct FncParams<'x> {\n    params: IntoIter<Variable<'x>>,\n}\n\nimpl<'x> FncParams<'x> {\n    pub fn new(params: Vec<Variable<'x>>) -> Self {\n        Self {\n            params: params.into_iter(),\n        }\n    }\n\n    pub fn next_as_string(&mut self) -> StringCow<'x> {\n        self.params.next().unwrap().into_string()\n    }\n\n    pub fn next_as_integer(&mut self) -> i64 {\n        self.params.next().unwrap().to_integer().unwrap_or_default()\n    }\n\n    pub fn next(&mut self) -> Variable<'x> {\n        self.params.next().unwrap()\n    }\n}\n\n#[derive(Debug)]\nstruct VariableWrapper(Variable<'static>);\n\nimpl From<i64> for VariableWrapper {\n    fn from(value: i64) -> Self {\n        VariableWrapper(Variable::Integer(value))\n    }\n}\n\nimpl Deserialize for VariableWrapper {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(VariableWrapper(Variable::String(StringCow::Owned(\n            CompactString::from_utf8_lossy(bytes),\n        ))))\n    }\n}\n\nimpl From<store::Value<'static>> for VariableWrapper {\n    fn from(value: store::Value<'static>) -> Self {\n        VariableWrapper(match value {\n            Value::Integer(v) => Variable::Integer(v),\n            Value::Bool(v) => Variable::Integer(v as i64),\n            Value::Float(v) => Variable::Float(v),\n            Value::Text(v) => Variable::String(StringCow::Owned(v.into())),\n            Value::Blob(v) => Variable::String(StringCow::Owned(match v {\n                std::borrow::Cow::Borrowed(v) => CompactString::from_utf8_lossy(v),\n                std::borrow::Cow::Owned(v) => CompactString::from_utf8_lossy(&v),\n            })),\n            Value::Null => Variable::String(StringCow::Borrowed(\"\")),\n        })\n    }\n}\n\nimpl VariableWrapper {\n    pub fn into_inner(self) -> Variable<'static> {\n        self.0\n    }\n}\n\nfn to_store_value(value: Variable) -> Value {\n    match value {\n        Variable::String(v) => Value::Text(v.to_string().into()),\n        Variable::Integer(v) => Value::Integer(v),\n        Variable::Float(v) => Value::Float(v),\n        v => Value::Text(v.to_string().into_owned().into()),\n    }\n}\n\nfn into_variable(value: Value) -> Variable {\n    match value {\n        Value::Integer(v) => Variable::Integer(v),\n        Value::Bool(v) => Variable::Integer(i64::from(v)),\n        Value::Float(v) => Variable::Float(v),\n        Value::Text(v) => Variable::String(v.into()),\n        Value::Blob(v) => Variable::String(StringCow::Owned(CompactString::from_utf8_lossy(&v))),\n        Value::Null => Variable::default(),\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/functions/email.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::CompactString;\n\nuse crate::expr::{StringCow, Variable};\n\npub(crate) fn fn_is_email(v: Vec<Variable>) -> Variable {\n    let mut last_ch = 0;\n    let mut in_quote = false;\n    let mut at_count = 0;\n    let mut dot_count = 0;\n    let mut lp_len = 0;\n    let mut value = 0;\n\n    for &ch in v[0].to_string().as_bytes() {\n        match ch {\n            b'0'..=b'9'\n            | b'a'..=b'z'\n            | b'A'..=b'Z'\n            | b'!'\n            | b'#'\n            | b'$'\n            | b'%'\n            | b'&'\n            | b'\\''\n            | b'*'\n            | b'+'\n            | b'-'\n            | b'/'\n            | b'='\n            | b'?'\n            | b'^'\n            | b'_'\n            | b'`'\n            | b'{'\n            | b'|'\n            | b'}'\n            | b'~'\n            | 0x7f..=u8::MAX => {\n                value += 1;\n            }\n            b'.' if !in_quote => {\n                if last_ch != b'.' && last_ch != b'@' && value != 0 {\n                    value += 1;\n                    if at_count == 1 {\n                        dot_count += 1;\n                    }\n                } else {\n                    return false.into();\n                }\n            }\n            b'@' if !in_quote => {\n                at_count += 1;\n                lp_len = value;\n                value = 0;\n            }\n            b'>' | b':' | b',' | b' ' if in_quote => {\n                value += 1;\n            }\n            b'\\\"' if !in_quote || last_ch != b'\\\\' => {\n                in_quote = !in_quote;\n            }\n            b'\\\\' if in_quote && last_ch != b'\\\\' => (),\n            _ => {\n                if !in_quote {\n                    return false.into();\n                }\n            }\n        }\n\n        last_ch = ch;\n    }\n\n    (at_count == 1 && dot_count > 0 && lp_len > 0 && value > 0).into()\n}\n\npub(crate) fn fn_email_part(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap();\n    let part = v.next().unwrap().into_string();\n\n    value.transform(|s| match s {\n        StringCow::Borrowed(s) => s\n            .rsplit_once('@')\n            .map(|(u, d)| match part.as_str() {\n                \"local\" => Variable::from(u.trim()),\n                \"domain\" => Variable::from(d.trim()),\n                _ => Variable::default(),\n            })\n            .unwrap_or_default(),\n        StringCow::Owned(s) => s\n            .rsplit_once('@')\n            .map(|(u, d)| match part.as_str() {\n                \"local\" => Variable::from(CompactString::new(u.trim())),\n                \"domain\" => Variable::from(CompactString::new(d.trim())),\n                _ => Variable::default(),\n            })\n            .unwrap_or_default(),\n    })\n}\n"
  },
  {
    "path": "crates/common/src/expr/functions/misc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse compact_str::CompactString;\nuse mail_auth::common::resolver::ToReverseName;\n\nuse crate::expr::Variable;\n\npub(crate) fn fn_is_empty(v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.is_empty(),\n        Variable::Integer(_) | Variable::Float(_) => false,\n        Variable::Array(a) => a.is_empty(),\n    }\n    .into()\n}\n\npub(crate) fn fn_is_number(v: Vec<Variable>) -> Variable {\n    matches!(&v[0], Variable::Integer(_) | Variable::Float(_)).into()\n}\n\npub(crate) fn fn_is_ip_addr(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .parse::<std::net::IpAddr>()\n        .is_ok()\n        .into()\n}\n\npub(crate) fn fn_is_ipv4_addr(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .parse::<std::net::IpAddr>()\n        .is_ok_and(|ip| matches!(ip, IpAddr::V4(_)))\n        .into()\n}\n\npub(crate) fn fn_is_ipv6_addr(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .parse::<std::net::IpAddr>()\n        .is_ok_and(|ip| matches!(ip, IpAddr::V6(_)))\n        .into()\n}\n\npub(crate) fn fn_ip_reverse_name(v: Vec<Variable>) -> Variable {\n    CompactString::new(\n        v[0].to_string()\n            .as_str()\n            .parse::<std::net::IpAddr>()\n            .map(|ip| ip.to_reverse_name())\n            .unwrap_or_default(),\n    )\n    .into()\n}\n\npub(crate) fn fn_if_then(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let condition = v.next().unwrap();\n    let iff = v.next().unwrap();\n    let then = v.next().unwrap();\n\n    if condition.to_bool() { iff } else { then }\n}\n"
  },
  {
    "path": "crates/common/src/expr/functions/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{StringCow, Variable};\n\npub mod array;\npub mod asynch;\npub mod email;\npub mod misc;\npub mod text;\n\npub trait ResolveVariable: Sync + Send {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_>;\n    fn resolve_global(&self, variable: &str) -> Variable<'_>;\n}\n\nimpl<'x> Variable<'x> {\n    fn transform(self, f: impl Fn(StringCow<'x>) -> Variable<'x>) -> Variable<'x> {\n        match self {\n            Variable::String(s) => f(s),\n            Variable::Array(list) => Variable::Array(\n                list.into_iter()\n                    .map(|v| match v {\n                        Variable::String(s) => f(s),\n                        v => f(v.into_string()),\n                    })\n                    .collect::<Vec<_>>(),\n            ),\n            v => f(v.into_string()),\n        }\n    }\n}\n\n#[allow(clippy::type_complexity)]\npub(crate) const FUNCTIONS: &[(&str, fn(Vec<Variable>) -> Variable, u32)] = &[\n    (\"count\", array::fn_count, 1),\n    (\"sort\", array::fn_sort, 2),\n    (\"dedup\", array::fn_dedup, 1),\n    (\"winnow\", array::fn_winnow, 1),\n    (\"is_intersect\", array::fn_is_intersect, 2),\n    (\"is_email\", email::fn_is_email, 1),\n    (\"email_part\", email::fn_email_part, 2),\n    (\"is_empty\", misc::fn_is_empty, 1),\n    (\"is_number\", misc::fn_is_number, 1),\n    (\"is_ip_addr\", misc::fn_is_ip_addr, 1),\n    (\"is_ipv4_addr\", misc::fn_is_ipv4_addr, 1),\n    (\"is_ipv6_addr\", misc::fn_is_ipv6_addr, 1),\n    (\"ip_reverse_name\", misc::fn_ip_reverse_name, 1),\n    (\"trim\", text::fn_trim, 1),\n    (\"trim_end\", text::fn_trim_end, 1),\n    (\"trim_start\", text::fn_trim_start, 1),\n    (\"len\", text::fn_len, 1),\n    (\"to_lowercase\", text::fn_to_lowercase, 1),\n    (\"to_uppercase\", text::fn_to_uppercase, 1),\n    (\"is_uppercase\", text::fn_is_uppercase, 1),\n    (\"is_lowercase\", text::fn_is_lowercase, 1),\n    (\"has_digits\", text::fn_has_digits, 1),\n    (\"count_spaces\", text::fn_count_spaces, 1),\n    (\"count_uppercase\", text::fn_count_uppercase, 1),\n    (\"count_lowercase\", text::fn_count_lowercase, 1),\n    (\"count_chars\", text::fn_count_chars, 1),\n    (\"contains\", text::fn_contains, 2),\n    (\"contains_ignore_case\", text::fn_contains_ignore_case, 2),\n    (\"eq_ignore_case\", text::fn_eq_ignore_case, 2),\n    (\"starts_with\", text::fn_starts_with, 2),\n    (\"ends_with\", text::fn_ends_with, 2),\n    (\"lines\", text::fn_lines, 1),\n    (\"substring\", text::fn_substring, 3),\n    (\"strip_prefix\", text::fn_strip_prefix, 2),\n    (\"strip_suffix\", text::fn_strip_suffix, 2),\n    (\"split\", text::fn_split, 2),\n    (\"rsplit\", text::fn_rsplit, 2),\n    (\"split_once\", text::fn_split_once, 2),\n    (\"rsplit_once\", text::fn_rsplit_once, 2),\n    (\"split_n\", text::fn_split_n, 3),\n    (\"split_words\", text::fn_split_words, 1),\n    (\"hash\", text::fn_hash, 2),\n    (\"if_then\", misc::fn_if_then, 3),\n];\n\npub const F_IS_LOCAL_DOMAIN: u32 = 0;\npub const F_IS_LOCAL_ADDRESS: u32 = 1;\npub const F_KEY_GET: u32 = 2;\npub const F_KEY_EXISTS: u32 = 3;\npub const F_KEY_SET: u32 = 4;\npub const F_COUNTER_INCR: u32 = 5;\npub const F_COUNTER_GET: u32 = 6;\npub const F_SQL_QUERY: u32 = 7;\npub const F_DNS_QUERY: u32 = 8;\n\npub const ASYNC_FUNCTIONS: &[(&str, u32, u32)] = &[\n    (\"is_local_domain\", F_IS_LOCAL_DOMAIN, 2),\n    (\"is_local_address\", F_IS_LOCAL_ADDRESS, 2),\n    (\"key_get\", F_KEY_GET, 2),\n    (\"key_exists\", F_KEY_EXISTS, 2),\n    (\"key_set\", F_KEY_SET, 3),\n    (\"counter_incr\", F_COUNTER_INCR, 3),\n    (\"counter_get\", F_COUNTER_GET, 2),\n    (\"dns_query\", F_DNS_QUERY, 2),\n    (\"sql_query\", F_SQL_QUERY, 3),\n];\n"
  },
  {
    "path": "crates/common/src/expr/functions/text.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::{CompactString, ToCompactString, format_compact};\nuse sha1::Sha1;\nuse sha2::{Sha256, Sha512};\n\nuse crate::expr::{StringCow, Variable};\n\npub(crate) fn fn_trim(mut v: Vec<Variable>) -> Variable {\n    v.remove(0).transform(|s| match s {\n        StringCow::Borrowed(s) => Variable::from(s.trim()),\n        StringCow::Owned(s) => Variable::from(s.trim().to_compact_string()),\n    })\n}\n\npub(crate) fn fn_trim_end(mut v: Vec<Variable>) -> Variable {\n    v.remove(0).transform(|s| match s {\n        StringCow::Borrowed(s) => Variable::from(s.trim_end()),\n        StringCow::Owned(s) => Variable::from(s.trim_end().to_compact_string()),\n    })\n}\n\npub(crate) fn fn_trim_start(mut v: Vec<Variable>) -> Variable {\n    v.remove(0).transform(|s| match s {\n        StringCow::Borrowed(s) => Variable::from(s.trim_start()),\n        StringCow::Owned(s) => Variable::from(s.trim_start().to_compact_string()),\n    })\n}\n\npub(crate) fn fn_len(v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.len(),\n        Variable::Array(a) => a.len(),\n        v => v.to_string().len(),\n    }\n    .into()\n}\n\npub(crate) fn fn_to_lowercase(mut v: Vec<Variable>) -> Variable {\n    v.remove(0)\n        .transform(|s| Variable::from(CompactString::from_str_to_lowercase(s.as_str())))\n}\n\npub(crate) fn fn_to_uppercase(mut v: Vec<Variable>) -> Variable {\n    v.remove(0)\n        .transform(|s| Variable::from(CompactString::from_str_to_uppercase(s.as_str())))\n}\n\npub(crate) fn fn_is_uppercase(mut v: Vec<Variable>) -> Variable {\n    v.remove(0).transform(|s| {\n        s.as_str()\n            .chars()\n            .filter(|c| c.is_alphabetic())\n            .all(|c| c.is_uppercase())\n            .into()\n    })\n}\n\npub(crate) fn fn_is_lowercase(mut v: Vec<Variable>) -> Variable {\n    v.remove(0).transform(|s| {\n        s.as_str()\n            .chars()\n            .filter(|c| c.is_alphabetic())\n            .all(|c| c.is_lowercase())\n            .into()\n    })\n}\n\npub(crate) fn fn_has_digits(mut v: Vec<Variable>) -> Variable {\n    v.remove(0)\n        .transform(|s| s.as_str().chars().any(|c| c.is_ascii_digit()).into())\n}\n\npub(crate) fn fn_split_words(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .split_whitespace()\n        .filter(|word| word.chars().all(|c| c.is_alphanumeric()))\n        .map(|word| Variable::from(CompactString::new(word)))\n        .collect::<Vec<_>>()\n        .into()\n}\n\npub(crate) fn fn_count_spaces(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .chars()\n        .filter(|c| c.is_whitespace())\n        .count()\n        .into()\n}\n\npub(crate) fn fn_count_uppercase(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .chars()\n        .filter(|c| c.is_alphabetic() && c.is_uppercase())\n        .count()\n        .into()\n}\n\npub(crate) fn fn_count_lowercase(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .chars()\n        .filter(|c| c.is_alphabetic() && c.is_lowercase())\n        .count()\n        .into()\n}\n\npub(crate) fn fn_count_chars(v: Vec<Variable>) -> Variable {\n    v[0].to_string().as_str().chars().count().into()\n}\n\npub(crate) fn fn_eq_ignore_case(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .eq_ignore_ascii_case(v[1].to_string().as_str())\n        .into()\n}\n\npub(crate) fn fn_contains(v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.as_str().contains(v[1].to_string().as_str()),\n        Variable::Array(arr) => arr.contains(&v[1]),\n        val => val.to_string().as_str().contains(v[1].to_string().as_str()),\n    }\n    .into()\n}\n\npub(crate) fn fn_contains_ignore_case(v: Vec<Variable>) -> Variable {\n    let needle = v[1].to_string();\n    match &v[0] {\n        Variable::String(s) => s\n            .as_str()\n            .to_lowercase()\n            .contains(&needle.as_str().to_lowercase()),\n        Variable::Array(arr) => arr.iter().any(|v| match v {\n            Variable::String(s) => s.as_str().eq_ignore_ascii_case(needle.as_str()),\n            _ => false,\n        }),\n        val => val.to_string().as_str().contains(needle.as_str()),\n    }\n    .into()\n}\n\npub(crate) fn fn_starts_with(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .starts_with(v[1].to_string().as_str())\n        .into()\n}\n\npub(crate) fn fn_ends_with(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .ends_with(v[1].to_string().as_str())\n        .into()\n}\n\npub(crate) fn fn_lines(mut v: Vec<Variable>) -> Variable {\n    match v.remove(0) {\n        Variable::String(s) => s\n            .as_str()\n            .lines()\n            .map(|s| Variable::from(CompactString::new(s)))\n            .collect::<Vec<_>>()\n            .into(),\n        val => val,\n    }\n}\n\npub(crate) fn fn_substring(v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_str()\n        .chars()\n        .skip(v[1].to_usize().unwrap_or_default())\n        .take(v[2].to_usize().unwrap_or_default())\n        .collect::<CompactString>()\n        .into()\n}\n\npub(crate) fn fn_strip_prefix(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap();\n    let prefix = v.next().unwrap().into_string();\n\n    value.transform(|s| match s {\n        StringCow::Borrowed(s) => s\n            .strip_prefix(prefix.as_str())\n            .map(Variable::from)\n            .unwrap_or_default(),\n        StringCow::Owned(s) => s\n            .strip_prefix(prefix.as_str())\n            .map(|s| Variable::from(CompactString::new(s)))\n            .unwrap_or_default(),\n    })\n}\n\npub(crate) fn fn_strip_suffix(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap();\n    let suffix = v.next().unwrap().into_string();\n\n    value.transform(|s| match s {\n        StringCow::Borrowed(s) => s\n            .strip_suffix(suffix.as_str())\n            .map(Variable::from)\n            .unwrap_or_default(),\n        StringCow::Owned(s) => s\n            .strip_suffix(suffix.as_str())\n            .map(|s| Variable::from(CompactString::new(s)))\n            .unwrap_or_default(),\n    })\n}\n\npub(crate) fn fn_split(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap().into_string();\n    let arg = v.next().unwrap().into_string();\n\n    match value {\n        StringCow::Borrowed(s) => s\n            .split(arg.as_str())\n            .map(Variable::from)\n            .collect::<Vec<_>>()\n            .into(),\n        StringCow::Owned(s) => s\n            .split(arg.as_str())\n            .map(|s| Variable::from(CompactString::new(s)))\n            .collect::<Vec<_>>()\n            .into(),\n    }\n}\n\npub(crate) fn fn_rsplit(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap().into_string();\n    let arg = v.next().unwrap().into_string();\n\n    match value {\n        StringCow::Borrowed(s) => s\n            .rsplit(arg.as_str())\n            .map(Variable::from)\n            .collect::<Vec<_>>()\n            .into(),\n        StringCow::Owned(s) => s\n            .rsplit(arg.as_str())\n            .map(|s| Variable::from(CompactString::new(s)))\n            .collect::<Vec<_>>()\n            .into(),\n    }\n}\n\npub(crate) fn fn_split_n(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap().into_string();\n    let arg = v.next().unwrap().into_string();\n    let num = v.next().unwrap().to_integer().unwrap_or_default() as usize;\n\n    fn split_n<'x, 'y>(s: &'x str, arg: &'y str, num: usize, mut f: impl FnMut(&'x str)) {\n        let mut s = s;\n        for _ in 0..num {\n            if let Some((a, b)) = s.split_once(arg) {\n                f(a);\n                s = b;\n            } else {\n                break;\n            }\n        }\n        f(s);\n    }\n\n    let mut result = Vec::new();\n    match value {\n        StringCow::Borrowed(s) => split_n(s, arg.as_str(), num, |s| result.push(Variable::from(s))),\n        StringCow::Owned(s) => split_n(&s, arg.as_str(), num, |s| {\n            result.push(Variable::from(CompactString::new(s)))\n        }),\n    }\n\n    result.into()\n}\n\npub(crate) fn fn_split_once(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap().into_string();\n    let arg = v.next().unwrap().into_string();\n\n    match value {\n        StringCow::Borrowed(s) => s\n            .split_once(arg.as_str())\n            .map(|(a, b)| Variable::Array(vec![Variable::from(a), Variable::from(b)]))\n            .unwrap_or_default(),\n        StringCow::Owned(s) => s\n            .split_once(arg.as_str())\n            .map(|(a, b)| {\n                Variable::Array(vec![\n                    Variable::from(CompactString::new(a)),\n                    Variable::from(CompactString::new(b)),\n                ])\n            })\n            .unwrap_or_default(),\n    }\n}\n\npub(crate) fn fn_rsplit_once(v: Vec<Variable>) -> Variable {\n    let mut v = v.into_iter();\n    let value = v.next().unwrap().into_string();\n    let arg = v.next().unwrap().into_string();\n\n    match value {\n        StringCow::Borrowed(s) => s\n            .rsplit_once(arg.as_str())\n            .map(|(a, b)| Variable::Array(vec![Variable::from(a), Variable::from(b)]))\n            .unwrap_or_default(),\n        StringCow::Owned(s) => s\n            .rsplit_once(arg.as_str())\n            .map(|(a, b)| {\n                Variable::Array(vec![\n                    Variable::from(CompactString::new(a)),\n                    Variable::from(CompactString::new(b)),\n                ])\n            })\n            .unwrap_or_default(),\n    }\n}\n\npub(crate) fn fn_hash(v: Vec<Variable>) -> Variable {\n    use sha1::Digest;\n    let mut v = v.into_iter();\n    let value = v.next().unwrap().into_string();\n    let algo = v.next().unwrap().into_string();\n\n    match algo.as_str() {\n        \"md5\" => format_compact!(\"{:x}\", md5::compute(value.as_bytes())).into(),\n        \"sha1\" => {\n            let mut hasher = Sha1::new();\n            hasher.update(value.as_bytes());\n            format_compact!(\"{:x}\", hasher.finalize()).into()\n        }\n        \"sha256\" => {\n            let mut hasher = Sha256::new();\n            hasher.update(value.as_bytes());\n            format_compact!(\"{:x}\", hasher.finalize()).into()\n        }\n        \"sha512\" => {\n            let mut hasher = Sha512::new();\n            hasher.update(value.as_bytes());\n            format_compact!(\"{:x}\", hasher.finalize()).into()\n        }\n        _ => Variable::default(),\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/if_block.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::CompactString;\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::expr::{Constant, Expression};\n\nuse super::{\n    ConstantValue, ExpressionItem,\n    parser::ExpressionParser,\n    tokenizer::{TokenMap, Tokenizer},\n};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct IfThen {\n    pub expr: Expression,\n    pub then: Expression,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct IfBlock {\n    pub key: String,\n    pub if_then: Vec<IfThen>,\n    pub default: Expression,\n}\n\nimpl IfBlock {\n    pub fn new<T: ConstantValue>(\n        key: impl Into<String>,\n        if_thens: impl IntoIterator<Item = (&'static str, &'static str)>,\n        default: impl AsRef<str>,\n    ) -> Self {\n        let token_map = TokenMap::default()\n            .with_all_variables()\n            .with_constants::<T>();\n\n        Self {\n            key: key.into(),\n            if_then: if_thens\n                .into_iter()\n                .map(|(if_, then)| IfThen {\n                    expr: Expression::parse(&token_map, if_),\n                    then: Expression::parse(&token_map, then),\n                })\n                .collect(),\n            default: Expression::parse(&token_map, default.as_ref()),\n        }\n    }\n\n    pub fn empty(key: impl Into<String>) -> Self {\n        Self {\n            key: key.into(),\n            if_then: Default::default(),\n            default: Expression {\n                items: Default::default(),\n            },\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.default.is_empty() && self.if_then.is_empty()\n    }\n}\n\nimpl Expression {\n    pub fn try_parse(\n        config: &mut Config,\n        key: impl AsKey,\n        token_map: &TokenMap,\n    ) -> Option<Expression> {\n        if let Some(expr) = config.value(key.as_key()) {\n            match ExpressionParser::new(Tokenizer::new(expr, token_map)).parse() {\n                Ok(expr) => Some(expr),\n                Err(err) => {\n                    config.new_parse_error(key, err);\n                    None\n                }\n            }\n        } else {\n            None\n        }\n    }\n\n    fn parse(token_map: &TokenMap, expr: &str) -> Self {\n        ExpressionParser::new(Tokenizer::new(expr, token_map))\n            .parse()\n            .unwrap()\n    }\n}\n\nimpl IfBlock {\n    pub fn try_parse(\n        config: &mut Config,\n        prefix: impl AsKey,\n        token_map: &TokenMap,\n    ) -> Option<IfBlock> {\n        let key = prefix.as_key();\n\n        // Parse conditions\n        let mut if_block = IfBlock {\n            key,\n            if_then: Default::default(),\n            default: Expression {\n                items: Default::default(),\n            },\n        };\n\n        // Try first with a single value\n        if config.contains_key(if_block.key.as_str()) {\n            if_block.default = Expression::try_parse(config, &if_block.key, token_map)?;\n            return Some(if_block);\n        }\n\n        // Collect prefixes\n        let prefix = prefix.as_prefix();\n        let keys = config\n            .keys\n            .keys()\n            .filter(|k| k.starts_with(&prefix))\n            .cloned()\n            .collect::<Vec<_>>();\n        let mut found_if = false;\n        let mut found_else = \"\";\n        let mut found_then = false;\n        let mut last_array_pos = \"\";\n\n        for item in &keys {\n            let suffix_ = item.strip_prefix(&prefix).unwrap();\n\n            if let Some((array_pos, suffix)) = suffix_.split_once('.') {\n                let if_key = suffix.split_once('.').map(|(v, _)| v).unwrap_or(suffix);\n                if if_key == \"if\" {\n                    if array_pos != last_array_pos {\n                        if !last_array_pos.is_empty() && !found_then {\n                            config.new_parse_error(\n                                if_block.key,\n                                format!(\n                                    \"Missing 'then' in 'if' condition {}.\",\n                                    last_array_pos.parse().unwrap_or(0) + 1,\n                                ),\n                            );\n                            return None;\n                        }\n\n                        if_block.if_then.push(IfThen {\n                            expr: Expression::try_parse(config, item, token_map)?,\n                            then: Expression::default(),\n                        });\n\n                        found_then = false;\n                        last_array_pos = array_pos;\n                    }\n\n                    found_if = true;\n                } else if if_key == \"else\" {\n                    if found_else.is_empty() {\n                        if found_if {\n                            if_block.default = Expression::try_parse(config, item, token_map)?;\n                            found_else = array_pos;\n                        } else {\n                            config.new_parse_error(if_block.key, \"Found 'else' before 'if'\");\n                            return None;\n                        }\n                    } else if array_pos != found_else {\n                        config.new_parse_error(if_block.key, \"Multiple 'else' found\");\n                        return None;\n                    }\n                } else if if_key == \"then\" {\n                    if found_else.is_empty() {\n                        if array_pos == last_array_pos {\n                            if !found_then {\n                                if_block.if_then.last_mut().unwrap().then =\n                                    Expression::try_parse(config, item, token_map)?;\n                                found_then = true;\n                            }\n                        } else {\n                            config.new_parse_error(if_block.key, \"Found 'then' without 'if'\");\n                            return None;\n                        }\n                    } else {\n                        config.new_parse_error(if_block.key, \"Found 'then' in 'else' block\");\n                        return None;\n                    }\n                }\n            } else {\n                config.new_parse_error(\n                    if_block.key,\n                    format!(\"Invalid property {item:?} found in 'if' block.\"),\n                );\n                return None;\n            }\n        }\n\n        if !found_if {\n            None\n        } else if !found_then {\n            config.new_parse_error(\n                if_block.key,\n                format!(\n                    \"Missing 'then' in 'if' condition {}\",\n                    last_array_pos.parse().unwrap_or(0) + 1,\n                ),\n            );\n            None\n        } else if found_else.is_empty() {\n            config.new_parse_error(if_block.key, \"Missing 'else'\");\n            None\n        } else {\n            Some(if_block)\n        }\n    }\n\n    pub fn into_default(self, key: impl Into<String>) -> IfBlock {\n        IfBlock {\n            key: key.into(),\n            if_then: Default::default(),\n            default: self.default,\n        }\n    }\n\n    pub fn default_string(&self) -> Option<&str> {\n        for expr_item in &self.default.items {\n            if let ExpressionItem::Constant(Constant::String(value)) = expr_item {\n                return Some(value.as_str());\n            }\n        }\n\n        None\n    }\n\n    pub fn into_default_string(self) -> Option<CompactString> {\n        for expr_item in self.default.items {\n            if let ExpressionItem::Constant(Constant::String(value)) = expr_item {\n                return Some(value);\n            }\n        }\n\n        None\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse self::tokenizer::TokenMap;\nuse compact_str::CompactString;\nuse regex::Regex;\nuse std::{\n    borrow::Cow,\n    fmt::{Display, Formatter},\n    net::{IpAddr, Ipv4Addr, Ipv6Addr},\n    time::Duration,\n};\nuse utils::config::{Rate, utils::ParseValue};\n\npub const V_RECIPIENT: u32 = 0;\npub const V_RECIPIENT_DOMAIN: u32 = 1;\npub const V_SENDER: u32 = 2;\npub const V_SENDER_DOMAIN: u32 = 3;\npub const V_MX: u32 = 4;\npub const V_HELO_DOMAIN: u32 = 5;\npub const V_AUTHENTICATED_AS: u32 = 6;\npub const V_LISTENER: u32 = 7;\npub const V_REMOTE_IP: u32 = 8;\npub const V_REMOTE_PORT: u32 = 9;\npub const V_LOCAL_IP: u32 = 10;\npub const V_LOCAL_PORT: u32 = 11;\npub const V_PRIORITY: u32 = 12;\npub const V_PROTOCOL: u32 = 13;\npub const V_TLS: u32 = 14;\npub const V_RECIPIENTS: u32 = 15;\npub const V_QUEUE_RETRY_NUM: u32 = 16;\npub const V_QUEUE_NOTIFY_NUM: u32 = 17;\npub const V_QUEUE_EXPIRES_IN: u32 = 18;\npub const V_QUEUE_LAST_STATUS: u32 = 19;\npub const V_QUEUE_LAST_ERROR: u32 = 20;\npub const V_URL: u32 = 21;\npub const V_URL_PATH: u32 = 22;\npub const V_HEADERS: u32 = 23;\npub const V_METHOD: u32 = 24;\npub const V_ASN: u32 = 25;\npub const V_COUNTRY: u32 = 26;\npub const V_RECEIVED_VIA_PORT: u32 = 27;\npub const V_RECEIVED_FROM_IP: u32 = 28;\npub const V_QUEUE_NAME: u32 = 29;\npub const V_SOURCE: u32 = 30;\npub const V_SIZE: u32 = 31;\npub const V_QUEUE_AGE: u32 = 32;\n\npub const VARIABLES_MAP: &[(&str, u32)] = &[\n    (\"rcpt\", V_RECIPIENT),\n    (\"rcpt_domain\", V_RECIPIENT_DOMAIN),\n    (\"sender\", V_SENDER),\n    (\"sender_domain\", V_SENDER_DOMAIN),\n    (\"mx\", V_MX),\n    (\"helo_domain\", V_HELO_DOMAIN),\n    (\"authenticated_as\", V_AUTHENTICATED_AS),\n    (\"listener\", V_LISTENER),\n    (\"remote_ip\", V_REMOTE_IP),\n    (\"local_ip\", V_LOCAL_IP),\n    (\"priority\", V_PRIORITY),\n    (\"local_port\", V_LOCAL_PORT),\n    (\"remote_port\", V_REMOTE_PORT),\n    (\"protocol\", V_PROTOCOL),\n    (\"is_tls\", V_TLS),\n    (\"recipients\", V_RECIPIENTS),\n    (\"retry_num\", V_QUEUE_RETRY_NUM),\n    (\"notify_num\", V_QUEUE_NOTIFY_NUM),\n    (\"expires_in\", V_QUEUE_EXPIRES_IN),\n    (\"last_status\", V_QUEUE_LAST_STATUS),\n    (\"last_error\", V_QUEUE_LAST_ERROR),\n    (\"url\", V_URL),\n    (\"url_path\", V_URL_PATH),\n    (\"headers\", V_HEADERS),\n    (\"method\", V_METHOD),\n    (\"asn\", V_ASN),\n    (\"country\", V_COUNTRY),\n    (\"received_via_port\", V_RECEIVED_VIA_PORT),\n    (\"received_from_ip\", V_RECEIVED_FROM_IP),\n    (\"queue_name\", V_QUEUE_NAME),\n    (\"source\", V_SOURCE),\n    (\"size\", V_SIZE),\n    (\"queue_age\", V_QUEUE_AGE),\n];\n\npub mod eval;\npub mod functions;\npub mod if_block;\npub mod parser;\npub mod tokenizer;\n\n#[derive(Debug, PartialEq, Eq, Clone, Default)]\npub struct Expression {\n    pub items: Vec<ExpressionItem>,\n}\n\n#[derive(Debug, Clone)]\npub enum ExpressionItem {\n    Variable(u32),\n    Global(CompactString),\n    Setting(Setting),\n    Capture(u32),\n    Constant(Constant),\n    BinaryOperator(BinaryOperator),\n    UnaryOperator(UnaryOperator),\n    Regex(Regex),\n    JmpIf { val: bool, pos: u32 },\n    Function { id: u32, num_args: u32 },\n    ArrayAccess,\n    ArrayBuild(u32),\n}\n\n#[derive(Debug, Clone)]\npub enum Variable<'x> {\n    String(StringCow<'x>),\n    Integer(i64),\n    Float(f64),\n    Array(Vec<Variable<'x>>),\n}\n\n#[derive(Debug, Clone)]\npub enum StringCow<'x> {\n    Owned(CompactString),\n    Borrowed(&'x str),\n}\n\nimpl Default for Variable<'_> {\n    fn default() -> Self {\n        Variable::String(StringCow::Borrowed(\"\"))\n    }\n}\n\n#[derive(Debug, PartialEq, Clone)]\npub enum Constant {\n    Integer(i64),\n    Float(f64),\n    String(CompactString),\n}\n\nimpl Eq for Constant {}\n\nimpl From<CompactString> for Constant {\n    fn from(value: CompactString) -> Self {\n        Constant::String(value)\n    }\n}\n\nimpl From<bool> for Constant {\n    fn from(value: bool) -> Self {\n        Constant::Integer(value as i64)\n    }\n}\n\nimpl From<i64> for Constant {\n    fn from(value: i64) -> Self {\n        Constant::Integer(value)\n    }\n}\n\nimpl From<i32> for Constant {\n    fn from(value: i32) -> Self {\n        Constant::Integer(value as i64)\n    }\n}\n\nimpl From<i16> for Constant {\n    fn from(value: i16) -> Self {\n        Constant::Integer(value as i64)\n    }\n}\n\nimpl From<f64> for Constant {\n    fn from(value: f64) -> Self {\n        Constant::Float(value)\n    }\n}\n\nimpl From<usize> for Constant {\n    fn from(value: usize) -> Self {\n        Constant::Integer(value as i64)\n    }\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, Copy)]\npub enum BinaryOperator {\n    Add,\n    Subtract,\n    Multiply,\n    Divide,\n\n    And,\n    Or,\n    Xor,\n\n    Eq,\n    Ne,\n    Lt,\n    Le,\n    Gt,\n    Ge,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, Copy)]\npub enum UnaryOperator {\n    Not,\n    Minus,\n}\n\n#[derive(Debug, Clone)]\npub enum Token {\n    Variable(u32),\n    Global(CompactString),\n    Capture(u32),\n    Function {\n        name: Cow<'static, str>,\n        id: u32,\n        num_args: u32,\n    },\n    Constant(Constant),\n    Setting(Setting),\n    Regex(Regex),\n    BinaryOperator(BinaryOperator),\n    UnaryOperator(UnaryOperator),\n    OpenParen,\n    CloseParen,\n    OpenBracket,\n    CloseBracket,\n    Comma,\n}\n\n#[derive(Debug, Clone)]\npub enum Setting {\n    Hostname,\n    ReportDomain,\n    NodeId,\n    Other(CompactString),\n}\n\nimpl From<CompactString> for Setting {\n    fn from(value: CompactString) -> Self {\n        match value.as_str() {\n            \"server.hostname\" => Setting::Hostname,\n            \"report.domain\" => Setting::ReportDomain,\n            \"cluster.node-id\" => Setting::NodeId,\n            _ => Setting::Other(value),\n        }\n    }\n}\n\nimpl From<usize> for Variable<'_> {\n    fn from(value: usize) -> Self {\n        Variable::Integer(value as i64)\n    }\n}\n\nimpl From<i64> for Variable<'_> {\n    fn from(value: i64) -> Self {\n        Variable::Integer(value)\n    }\n}\n\nimpl From<u64> for Variable<'_> {\n    fn from(value: u64) -> Self {\n        Variable::Integer(value as i64)\n    }\n}\n\nimpl From<i32> for Variable<'_> {\n    fn from(value: i32) -> Self {\n        Variable::Integer(value as i64)\n    }\n}\n\nimpl From<u32> for Variable<'_> {\n    fn from(value: u32) -> Self {\n        Variable::Integer(value as i64)\n    }\n}\n\nimpl From<u16> for Variable<'_> {\n    fn from(value: u16) -> Self {\n        Variable::Integer(value as i64)\n    }\n}\n\nimpl From<i16> for Variable<'_> {\n    fn from(value: i16) -> Self {\n        Variable::Integer(value as i64)\n    }\n}\n\nimpl From<f64> for Variable<'_> {\n    fn from(value: f64) -> Self {\n        Variable::Float(value)\n    }\n}\n\nimpl<'x> From<&'x str> for Variable<'x> {\n    fn from(value: &'x str) -> Self {\n        Variable::String(StringCow::Borrowed(value))\n    }\n}\n\nimpl From<CompactString> for Variable<'_> {\n    fn from(value: CompactString) -> Self {\n        Variable::String(StringCow::Owned(value))\n    }\n}\n\nimpl<'x> From<Vec<Variable<'x>>> for Variable<'x> {\n    fn from(value: Vec<Variable<'x>>) -> Self {\n        Variable::Array(value)\n    }\n}\n\nimpl From<bool> for Variable<'_> {\n    fn from(value: bool) -> Self {\n        Variable::Integer(value as i64)\n    }\n}\n\nimpl<T: Into<Constant>> From<T> for Expression {\n    fn from(value: T) -> Self {\n        Expression {\n            items: vec![ExpressionItem::Constant(value.into())],\n        }\n    }\n}\n\nimpl PartialEq for ExpressionItem {\n    fn eq(&self, other: &Self) -> bool {\n        match (self, other) {\n            (Self::Variable(l0), Self::Variable(r0)) => l0 == r0,\n            (Self::Constant(l0), Self::Constant(r0)) => l0 == r0,\n            (Self::BinaryOperator(l0), Self::BinaryOperator(r0)) => l0 == r0,\n            (Self::UnaryOperator(l0), Self::UnaryOperator(r0)) => l0 == r0,\n            (Self::Regex(_), Self::Regex(_)) => true,\n            (\n                Self::JmpIf {\n                    val: l_val,\n                    pos: l_pos,\n                },\n                Self::JmpIf {\n                    val: r_val,\n                    pos: r_pos,\n                },\n            ) => l_val == r_val && l_pos == r_pos,\n            (\n                Self::Function {\n                    id: l_id,\n                    num_args: l_num_args,\n                },\n                Self::Function {\n                    id: r_id,\n                    num_args: r_num_args,\n                },\n            ) => l_id == r_id && l_num_args == r_num_args,\n            (Self::ArrayBuild(l0), Self::ArrayBuild(r0)) => l0 == r0,\n            _ => core::mem::discriminant(self) == core::mem::discriminant(other),\n        }\n    }\n}\n\nimpl Eq for ExpressionItem {}\n\nimpl PartialEq for Token {\n    fn eq(&self, other: &Self) -> bool {\n        match (self, other) {\n            (Self::Variable(l0), Self::Variable(r0)) => l0 == r0,\n            (\n                Self::Function {\n                    name: l_name,\n                    id: l_id,\n                    num_args: l_num_args,\n                },\n                Self::Function {\n                    name: r_name,\n                    id: r_id,\n                    num_args: r_num_args,\n                },\n            ) => l_name == r_name && l_id == r_id && l_num_args == r_num_args,\n            (Self::Constant(l0), Self::Constant(r0)) => l0 == r0,\n            (Self::Regex(_), Self::Regex(_)) => true,\n            (Self::BinaryOperator(l0), Self::BinaryOperator(r0)) => l0 == r0,\n            (Self::UnaryOperator(l0), Self::UnaryOperator(r0)) => l0 == r0,\n            _ => core::mem::discriminant(self) == core::mem::discriminant(other),\n        }\n    }\n}\n\nimpl Eq for Token {}\n\npub struct NoConstants;\n\npub trait ConstantValue:\n    ParseValue + for<'x> TryFrom<Variable<'x>> + Into<Constant> + Sized\n{\n    fn add_constants(token_map: &mut TokenMap);\n}\n\nimpl ConstantValue for () {\n    fn add_constants(_: &mut TokenMap) {}\n}\n\nimpl From<()> for Constant {\n    fn from(_: ()) -> Self {\n        Constant::Integer(0)\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for () {\n    type Error = ();\n\n    fn try_from(_: Variable<'x>) -> Result<Self, Self::Error> {\n        Ok(())\n    }\n}\n\nimpl ConstantValue for Duration {\n    fn add_constants(_: &mut TokenMap) {}\n}\n\nimpl<'x> TryFrom<Variable<'x>> for Duration {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::Integer(value) if value > 0 => Ok(Duration::from_millis(value as u64)),\n            Variable::Float(value) if value > 0.0 => Ok(Duration::from_millis(value as u64)),\n            Variable::String(value) if !value.is_empty() => {\n                Duration::parse_value(value.as_str()).map_err(|_| ())\n            }\n            _ => Err(()),\n        }\n    }\n}\n\nimpl StringCow<'_> {\n    pub fn as_str(&self) -> &str {\n        match self {\n            StringCow::Owned(s) => s.as_str(),\n            StringCow::Borrowed(s) => s,\n        }\n    }\n\n    pub fn as_bytes(&self) -> &[u8] {\n        match self {\n            StringCow::Owned(s) => s.as_bytes(),\n            StringCow::Borrowed(s) => s.as_bytes(),\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        match self {\n            StringCow::Owned(s) => s.is_empty(),\n            StringCow::Borrowed(s) => s.is_empty(),\n        }\n    }\n\n    pub fn len(&self) -> usize {\n        match self {\n            StringCow::Owned(s) => s.len(),\n            StringCow::Borrowed(s) => s.len(),\n        }\n    }\n\n    pub fn into_owned(self) -> CompactString {\n        match self {\n            StringCow::Owned(s) => s,\n            StringCow::Borrowed(s) => s.into(),\n        }\n    }\n}\n\nimpl<'x> From<Cow<'x, str>> for StringCow<'x> {\n    fn from(value: Cow<'x, str>) -> Self {\n        match value {\n            Cow::Borrowed(s) => StringCow::Borrowed(s),\n            Cow::Owned(s) => StringCow::Owned(s.into()),\n        }\n    }\n}\n\nimpl From<CompactString> for StringCow<'_> {\n    fn from(value: CompactString) -> Self {\n        StringCow::Owned(value)\n    }\n}\n\nimpl AsRef<str> for StringCow<'_> {\n    fn as_ref(&self) -> &str {\n        self.as_str()\n    }\n}\n\nimpl AsRef<[u8]> for StringCow<'_> {\n    fn as_ref(&self) -> &[u8] {\n        self.as_str().as_bytes()\n    }\n}\n\nimpl Display for StringCow<'_> {\n    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {\n        match self {\n            StringCow::Owned(s) => write!(f, \"{}\", s),\n            StringCow::Borrowed(s) => write!(f, \"{}\", s),\n        }\n    }\n}\n\nimpl From<Duration> for Constant {\n    fn from(value: Duration) -> Self {\n        Constant::Integer(value.as_millis() as i64)\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for Rate {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::Array(items) if items.len() == 2 => {\n                let requests = items[0].to_integer().ok_or(())?;\n                let period = items[1].to_integer().ok_or(())?;\n\n                if requests > 0 && period > 0 {\n                    Ok(Rate {\n                        requests: requests as u64,\n                        period: Duration::from_millis(period as u64),\n                    })\n                } else {\n                    Err(())\n                }\n            }\n            _ => Err(()),\n        }\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for Ipv4Addr {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::String(value) => value.as_str().parse().map_err(|_| ()),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for Ipv6Addr {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::String(value) => value.as_str().parse().map_err(|_| ()),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl<'x> TryFrom<Variable<'x>> for IpAddr {\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        match value {\n            Variable::String(value) => value.as_str().parse().map_err(|_| ()),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl<'x, T: TryFrom<Variable<'x>>> TryFrom<Variable<'x>> for Vec<T>\nwhere\n    Result<Vec<T>, ()>: FromIterator<Result<T, <T as TryFrom<Variable<'x>>>::Error>>,\n{\n    type Error = ();\n\n    fn try_from(value: Variable<'x>) -> Result<Self, Self::Error> {\n        value\n            .into_array()\n            .into_iter()\n            .map(|v| T::try_from(v))\n            .collect()\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/parser.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{BinaryOperator, Expression, ExpressionItem, Token, tokenizer::Tokenizer};\n\npub struct ExpressionParser<'x> {\n    pub(crate) tokenizer: Tokenizer<'x>,\n    pub(crate) output: Vec<ExpressionItem>,\n    operator_stack: Vec<(Token, Option<usize>)>,\n    arg_count: Vec<i32>,\n}\n\npub(crate) const ID_ARRAY_ACCESS: u32 = u32::MAX;\npub(crate) const ID_ARRAY_BUILD: u32 = u32::MAX - 1;\n\nimpl<'x> ExpressionParser<'x> {\n    pub fn new(tokenizer: Tokenizer<'x>) -> Self {\n        Self {\n            tokenizer,\n            output: Vec::new(),\n            operator_stack: Vec::new(),\n            arg_count: Vec::new(),\n        }\n    }\n\n    pub fn parse(mut self) -> Result<Expression, String> {\n        let mut last_is_var_or_fnc = false;\n\n        while let Some(token) = self.tokenizer.next()? {\n            let mut is_var_or_fnc = false;\n            match token {\n                Token::Variable(v) => {\n                    self.inc_arg_count();\n                    is_var_or_fnc = true;\n                    self.output.push(ExpressionItem::Variable(v))\n                }\n                Token::Constant(c) => {\n                    self.inc_arg_count();\n                    self.output.push(ExpressionItem::Constant(c))\n                }\n                Token::Global(g) => {\n                    self.inc_arg_count();\n                    self.output.push(ExpressionItem::Global(g))\n                }\n                Token::Capture(c) => {\n                    self.inc_arg_count();\n                    self.output.push(ExpressionItem::Capture(c))\n                }\n                Token::UnaryOperator(uop) => {\n                    self.operator_stack.push((Token::UnaryOperator(uop), None))\n                }\n                Token::OpenParen => self.operator_stack.push((token, None)),\n                Token::CloseParen | Token::CloseBracket => {\n                    let expect_token = if matches!(token, Token::CloseParen) {\n                        Token::OpenParen\n                    } else {\n                        Token::OpenBracket\n                    };\n                    loop {\n                        match self.operator_stack.pop() {\n                            Some((t, _)) if t == expect_token => {\n                                break;\n                            }\n                            Some((Token::BinaryOperator(bop), jmp_pos)) => {\n                                self.update_jmp_pos(jmp_pos);\n                                self.output.push(ExpressionItem::BinaryOperator(bop))\n                            }\n                            Some((Token::UnaryOperator(uop), _)) => {\n                                self.output.push(ExpressionItem::UnaryOperator(uop))\n                            }\n                            _ => return Err(\"Mismatched parentheses\".to_string()),\n                        }\n                    }\n\n                    match self.operator_stack.last() {\n                        Some((Token::Function { id, num_args, name }, _)) => {\n                            let got_args = self.arg_count.pop().unwrap();\n                            if got_args != *num_args as i32 {\n                                return Err(if *id != u32::MAX {\n                                    format!(\n                                        \"Expression function {:?} expected {} arguments, got {}\",\n                                        name, num_args, got_args\n                                    )\n                                } else {\n                                    \"Missing array index\".to_string()\n                                });\n                            }\n\n                            let expr = match *id {\n                                ID_ARRAY_ACCESS => ExpressionItem::ArrayAccess,\n                                ID_ARRAY_BUILD => ExpressionItem::ArrayBuild(*num_args),\n                                id => ExpressionItem::Function {\n                                    id,\n                                    num_args: *num_args,\n                                },\n                            };\n\n                            self.operator_stack.pop();\n                            self.output.push(expr);\n                        }\n                        Some((Token::Regex(regex), _)) => {\n                            if self.arg_count.pop().unwrap() != 1 {\n                                return Err(\"Expression function \\\"matches\\\" expected 2 arguments\"\n                                    .to_string());\n                            }\n                            self.output.push(ExpressionItem::Regex(regex.clone()));\n                            self.operator_stack.pop();\n                        }\n                        Some((Token::Setting(setting), _)) => {\n                            if self.arg_count.pop().unwrap() != 0 {\n                                return Err(\n                                    \"Expression function \\\"config_get\\\" expected 1 argument\"\n                                        .to_string(),\n                                );\n                            }\n                            self.output.push(ExpressionItem::Setting(setting.clone()));\n                            self.operator_stack.pop();\n                        }\n                        _ => {}\n                    }\n\n                    is_var_or_fnc = true;\n                }\n                Token::BinaryOperator(bop) => {\n                    self.dec_arg_count();\n                    while let Some((top_token, prev_jmp_pos)) = self.operator_stack.last() {\n                        match top_token {\n                            Token::BinaryOperator(top_bop) => {\n                                if bop.precedence() <= top_bop.precedence() {\n                                    let top_bop = *top_bop;\n                                    let jmp_pos = *prev_jmp_pos;\n                                    self.update_jmp_pos(jmp_pos);\n                                    self.operator_stack.pop();\n                                    self.output.push(ExpressionItem::BinaryOperator(top_bop));\n                                } else {\n                                    break;\n                                }\n                            }\n                            Token::UnaryOperator(top_uop) => {\n                                let top_uop = *top_uop;\n                                self.operator_stack.pop();\n                                self.output.push(ExpressionItem::UnaryOperator(top_uop));\n                            }\n                            _ => break,\n                        }\n                    }\n\n                    // Add jump instruction for short-circuiting\n                    let jmp_pos = match bop {\n                        BinaryOperator::And => {\n                            self.output\n                                .push(ExpressionItem::JmpIf { val: false, pos: 0 });\n                            Some(self.output.len() - 1)\n                        }\n                        BinaryOperator::Or => {\n                            self.output\n                                .push(ExpressionItem::JmpIf { val: true, pos: 0 });\n                            Some(self.output.len() - 1)\n                        }\n                        _ => None,\n                    };\n\n                    self.operator_stack\n                        .push((Token::BinaryOperator(bop), jmp_pos));\n                }\n                token @ (Token::Function { .. } | Token::Regex(_) | Token::Setting(_)) => {\n                    self.inc_arg_count();\n                    self.arg_count.push(0);\n                    self.operator_stack.push((token, None))\n                }\n                Token::OpenBracket => {\n                    // Array functions\n                    let (id, num_args, arg_count) = if last_is_var_or_fnc {\n                        (ID_ARRAY_ACCESS, 2, 1)\n                    } else {\n                        self.inc_arg_count();\n                        (ID_ARRAY_BUILD, 0, 0)\n                    };\n                    self.arg_count.push(arg_count);\n                    self.operator_stack.push((\n                        Token::Function {\n                            id,\n                            name: \"array\".into(),\n                            num_args,\n                        },\n                        None,\n                    ));\n                    self.operator_stack.push((token, None));\n                }\n                Token::Comma => {\n                    while let Some((token, jmp_pos)) = self.operator_stack.last() {\n                        match token {\n                            Token::OpenParen => break,\n                            Token::BinaryOperator(bop) => {\n                                let bop = *bop;\n                                let jmp_pos = *jmp_pos;\n                                self.update_jmp_pos(jmp_pos);\n                                self.output.push(ExpressionItem::BinaryOperator(bop));\n                                self.operator_stack.pop();\n                            }\n                            Token::UnaryOperator(uop) => {\n                                self.output.push(ExpressionItem::UnaryOperator(*uop));\n                                self.operator_stack.pop();\n                            }\n                            _ => break,\n                        }\n                    }\n                }\n            }\n            last_is_var_or_fnc = is_var_or_fnc;\n        }\n\n        while let Some((token, jmp_pos)) = self.operator_stack.pop() {\n            match token {\n                Token::BinaryOperator(bop) => {\n                    self.update_jmp_pos(jmp_pos);\n                    self.output.push(ExpressionItem::BinaryOperator(bop))\n                }\n                Token::UnaryOperator(uop) => self.output.push(ExpressionItem::UnaryOperator(uop)),\n                _ => return Err(\"Invalid token on the operator stack\".to_string()),\n            }\n        }\n\n        if self.operator_stack.is_empty() {\n            Ok(Expression { items: self.output })\n        } else {\n            Err(\"Invalid expression\".to_string())\n        }\n    }\n\n    fn inc_arg_count(&mut self) {\n        if let Some(x) = self.arg_count.last_mut() {\n            *x = x.saturating_add(1);\n            let op_pos = self.operator_stack.len().saturating_sub(2);\n            match self.operator_stack.get_mut(op_pos) {\n                Some((Token::Function { num_args, id, .. }, _)) if *id == ID_ARRAY_BUILD => {\n                    *num_args += 1;\n                }\n                _ => {}\n            }\n        }\n    }\n\n    fn dec_arg_count(&mut self) {\n        if let Some(x) = self.arg_count.last_mut() {\n            *x = x.saturating_sub(1);\n        }\n    }\n\n    fn update_jmp_pos(&mut self, jmp_pos: Option<usize>) {\n        if let Some(jmp_pos) = jmp_pos {\n            let cur_pos = self.output.len();\n            if let ExpressionItem::JmpIf { pos, .. } = &mut self.output[jmp_pos] {\n                *pos = (cur_pos - jmp_pos) as u32;\n            } else {\n                #[cfg(test)]\n                panic!(\"Invalid jump position\");\n            }\n        }\n    }\n}\n\nimpl BinaryOperator {\n    fn precedence(&self) -> i32 {\n        match self {\n            BinaryOperator::Multiply | BinaryOperator::Divide => 7,\n            BinaryOperator::Add | BinaryOperator::Subtract => 6,\n            BinaryOperator::Gt | BinaryOperator::Ge | BinaryOperator::Lt | BinaryOperator::Le => 5,\n            BinaryOperator::Eq | BinaryOperator::Ne => 4,\n            BinaryOperator::Xor => 3,\n            BinaryOperator::And => 2,\n            BinaryOperator::Or => 1,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/expr/tokenizer.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, iter::Peekable, slice::Iter, time::Duration};\n\nuse ahash::AHashMap;\nuse regex::Regex;\nuse utils::config::utils::ParseValue;\n\nuse super::{\n    functions::{ASYNC_FUNCTIONS, FUNCTIONS},\n    *,\n};\n\npub struct Tokenizer<'x> {\n    pub(crate) iter: Peekable<Iter<'x, u8>>,\n    token_map: &'x TokenMap,\n    buf: Vec<u8>,\n    depth: u32,\n    next_token: Vec<Token>,\n    has_number: bool,\n    has_dot: bool,\n    has_alpha: bool,\n    is_start: bool,\n    is_eof: bool,\n}\n\n#[derive(Debug, Default, Clone)]\npub struct TokenMap {\n    pub tokens: AHashMap<Cow<'static, str>, Token>,\n}\n\nimpl<'x> Tokenizer<'x> {\n    #[allow(clippy::should_implement_trait)]\n    pub fn new(expr: &'x str, token_map: &'x TokenMap) -> Self {\n        Self {\n            iter: expr.as_bytes().iter().peekable(),\n            buf: Vec::new(),\n            depth: 0,\n            next_token: Vec::with_capacity(2),\n            has_number: false,\n            has_dot: false,\n            has_alpha: false,\n            is_start: true,\n            is_eof: false,\n            token_map,\n        }\n    }\n\n    #[allow(clippy::should_implement_trait)]\n    pub fn next(&mut self) -> Result<Option<Token>, String> {\n        if let Some(token) = self.next_token.pop() {\n            return Ok(Some(token));\n        } else if self.is_eof {\n            return Ok(None);\n        }\n\n        while let Some(&ch) = self.iter.next() {\n            match ch {\n                b'A'..=b'Z' | b'a'..=b'z' | b'_' | b'$' => {\n                    self.buf.push(ch);\n                    self.has_alpha = true;\n                }\n                b'0'..=b'9' => {\n                    self.buf.push(ch);\n                    self.has_number = true;\n                }\n                b'.' => {\n                    self.buf.push(ch);\n                    self.has_dot = true;\n                }\n                b'}' => {\n                    self.is_eof = true;\n                    break;\n                }\n                b'-' if self.buf.last().is_some_and(|c| *c == b'[') => {\n                    self.buf.push(ch);\n                }\n                b':' if self.buf.contains(&b'.') => {\n                    self.buf.push(ch);\n                }\n                b']' if self.buf.contains(&b'[') => {\n                    self.buf.push(b']');\n                }\n                b'*' if self.buf.last().is_some_and(|&c| c == b'[' || c == b'.') => {\n                    self.buf.push(ch);\n                }\n                _ => {\n                    let (prev_token, ch) = if ch == b'(' && self.buf.eq(b\"matches\") {\n                        // Parse regular expressions\n                        let stop_ch = self.find_char(b\"\\\"'\")?;\n                        let regex_str = self.parse_string(stop_ch)?;\n                        let regex = Regex::new(&regex_str).map_err(|e| {\n                            format!(\"Invalid regular expression {:?}: {}\", regex_str, e)\n                        })?;\n                        self.has_alpha = false;\n                        self.buf.clear();\n                        self.find_char(b\",\")?;\n                        (Token::Regex(regex).into(), b'(')\n                    } else if ch == b'(' && self.buf.eq(b\"config_get\") {\n                        // Parse setting\n                        let stop_ch = self.find_char(b\"\\\"'\")?;\n                        let setting_str = self.parse_string(stop_ch)?;\n                        self.has_alpha = false;\n                        self.buf.clear();\n                        (Token::Setting(Setting::from(setting_str)).into(), b'(')\n                    } else if !self.buf.is_empty() {\n                        self.is_start = false;\n                        (self.parse_buf()?.into(), ch)\n                    } else {\n                        (None, ch)\n                    };\n                    let token = match ch {\n                        b'&' => {\n                            if matches!(self.iter.peek(), Some(b'&')) {\n                                self.iter.next();\n                            }\n                            Token::BinaryOperator(BinaryOperator::And)\n                        }\n                        b'|' => {\n                            if matches!(self.iter.peek(), Some(b'|')) {\n                                self.iter.next();\n                            }\n                            Token::BinaryOperator(BinaryOperator::Or)\n                        }\n                        b'!' => {\n                            if matches!(self.iter.peek(), Some(b'=')) {\n                                self.iter.next();\n                                Token::BinaryOperator(BinaryOperator::Ne)\n                            } else {\n                                Token::UnaryOperator(UnaryOperator::Not)\n                            }\n                        }\n                        b'^' => Token::BinaryOperator(BinaryOperator::Xor),\n                        b'(' => {\n                            self.depth += 1;\n                            Token::OpenParen\n                        }\n                        b')' => {\n                            if self.depth == 0 {\n                                return Err(\"Unmatched close parenthesis\".to_string());\n                            }\n                            self.depth -= 1;\n                            Token::CloseParen\n                        }\n                        b'+' => Token::BinaryOperator(BinaryOperator::Add),\n                        b'*' => Token::BinaryOperator(BinaryOperator::Multiply),\n                        b'/' => Token::BinaryOperator(BinaryOperator::Divide),\n                        b'-' => {\n                            if self.is_start {\n                                Token::UnaryOperator(UnaryOperator::Minus)\n                            } else {\n                                Token::BinaryOperator(BinaryOperator::Subtract)\n                            }\n                        }\n                        b'=' => match self.iter.next() {\n                            Some(b'=') => Token::BinaryOperator(BinaryOperator::Eq),\n                            Some(b'>') => Token::BinaryOperator(BinaryOperator::Ge),\n                            Some(b'<') => Token::BinaryOperator(BinaryOperator::Le),\n                            _ => Token::BinaryOperator(BinaryOperator::Eq),\n                        },\n                        b'>' => match self.iter.peek() {\n                            Some(b'=') => {\n                                self.iter.next();\n                                Token::BinaryOperator(BinaryOperator::Ge)\n                            }\n                            _ => Token::BinaryOperator(BinaryOperator::Gt),\n                        },\n                        b'<' => match self.iter.peek() {\n                            Some(b'=') => {\n                                self.iter.next();\n                                Token::BinaryOperator(BinaryOperator::Le)\n                            }\n                            _ => Token::BinaryOperator(BinaryOperator::Lt),\n                        },\n                        b',' => Token::Comma,\n                        b'[' => Token::OpenBracket,\n                        b']' => Token::CloseBracket,\n                        b' ' | b'\\r' | b'\\n' => {\n                            if prev_token.is_some() {\n                                return Ok(prev_token);\n                            } else {\n                                continue;\n                            }\n                        }\n                        b'\\\"' | b'\\'' => Token::Constant(Constant::String(self.parse_string(ch)?)),\n                        _ => {\n                            return Err(format!(\"Invalid character {:?}\", char::from(ch),));\n                        }\n                    };\n                    self.is_start = matches!(\n                        token,\n                        Token::OpenParen | Token::Comma | Token::BinaryOperator(_)\n                    );\n\n                    return if prev_token.is_some() {\n                        self.next_token.push(token);\n                        Ok(prev_token)\n                    } else {\n                        Ok(Some(token))\n                    };\n                }\n            }\n        }\n\n        if self.depth > 0 {\n            Err(\"Unmatched open parenthesis\".to_string())\n        } else if !self.buf.is_empty() {\n            self.parse_buf().map(Some)\n        } else {\n            Ok(None)\n        }\n    }\n\n    fn find_char(&mut self, chars: &[u8]) -> Result<u8, String> {\n        for &ch in self.iter.by_ref() {\n            if !ch.is_ascii_whitespace() {\n                return if chars.contains(&ch) {\n                    Ok(ch)\n                } else {\n                    Err(format!(\n                        \"Expected {:?}, found invalid character {:?}\",\n                        char::from(chars[0]),\n                        char::from(ch),\n                    ))\n                };\n            }\n        }\n\n        Err(\"Unexpected end of expression\".to_string())\n    }\n\n    fn parse_string(&mut self, stop_ch: u8) -> Result<CompactString, String> {\n        let mut buf = Vec::with_capacity(16);\n        let mut last_ch = 0;\n        let mut found_end = false;\n\n        for &ch in self.iter.by_ref() {\n            if last_ch != b'\\\\' {\n                if ch != stop_ch {\n                    buf.push(ch);\n                } else {\n                    found_end = true;\n                    break;\n                }\n            } else {\n                match ch {\n                    b'n' => {\n                        buf.push(b'\\n');\n                    }\n                    b'r' => {\n                        buf.push(b'\\r');\n                    }\n                    b't' => {\n                        buf.push(b'\\t');\n                    }\n                    _ => {\n                        buf.push(ch);\n                    }\n                }\n            }\n\n            last_ch = ch;\n        }\n\n        if found_end {\n            CompactString::from_utf8(buf).map_err(|_| \"Invalid UTF-8\".into())\n        } else {\n            Err(\"Unterminated string\".to_string())\n        }\n    }\n\n    fn parse_buf(&mut self) -> Result<Token, String> {\n        let buf = String::from_utf8(std::mem::take(&mut self.buf)).unwrap_or_default();\n        if self.has_number && !self.has_alpha {\n            self.has_number = false;\n            if self.has_dot {\n                self.has_dot = false;\n\n                buf.parse::<f64>()\n                    .map(|f| Token::Constant(Constant::Float(f)))\n                    .map_err(|_| format!(\"Invalid float value {}\", buf,))\n            } else {\n                buf.parse::<i64>()\n                    .map(|i| Token::Constant(Constant::Integer(i)))\n                    .map_err(|_| format!(\"Invalid integer value {}\", buf,))\n            }\n        } else {\n            let has_dot = self.has_dot;\n            let has_number = self.has_number;\n\n            self.has_alpha = false;\n            self.has_number = false;\n            self.has_dot = false;\n\n            if !has_number && !has_dot && [4, 5].contains(&buf.len()) {\n                if buf == \"true\" {\n                    return Ok(Token::Constant(Constant::Integer(1)));\n                } else if buf == \"false\" {\n                    return Ok(Token::Constant(Constant::Integer(0)));\n                }\n            }\n\n            if let Some(variable) = buf.strip_prefix('$').filter(|s| !s.is_empty()) {\n                if variable.chars().all(|c| c.is_ascii_digit()) {\n                    Ok(variable\n                        .parse::<u32>()\n                        .map(Token::Capture)\n                        .unwrap_or_else(|_| Token::Global(variable.into())))\n                } else {\n                    Ok(Token::Global(variable.into()))\n                }\n            } else if let Some((idx, (name, _, num_args))) = FUNCTIONS\n                .iter()\n                .enumerate()\n                .find(|(_, (name, _, _))| name == &buf)\n            {\n                Ok(Token::Function {\n                    name: Cow::Borrowed(*name),\n                    id: idx as u32,\n                    num_args: *num_args,\n                })\n            } else if let Some((name, idx, num_args)) =\n                ASYNC_FUNCTIONS.iter().find(|(name, _, _)| name == &buf)\n            {\n                Ok(Token::Function {\n                    name: Cow::Borrowed(*name),\n                    id: *idx + FUNCTIONS.len() as u32,\n                    num_args: *num_args,\n                })\n            } else if let Some(token) = self.token_map.tokens.get(buf.as_str()) {\n                Ok(token.clone())\n            } else if let Ok(duration) = Duration::parse_value(&buf) {\n                Ok(Token::Constant(Constant::Integer(\n                    duration.as_millis() as i64\n                )))\n            } else {\n                Err(format!(\"Invalid variable or constant {buf:?}\"))\n            }\n        }\n    }\n}\n\nimpl TokenMap {\n    pub fn with_all_variables(self) -> Self {\n        self.with_variables(&[\n            V_RECIPIENT,\n            V_RECIPIENT_DOMAIN,\n            V_SENDER,\n            V_SENDER_DOMAIN,\n            V_MX,\n            V_HELO_DOMAIN,\n            V_AUTHENTICATED_AS,\n            V_LISTENER,\n            V_REMOTE_IP,\n            V_REMOTE_PORT,\n            V_LOCAL_IP,\n            V_LOCAL_PORT,\n            V_PRIORITY,\n            V_PROTOCOL,\n            V_TLS,\n            V_QUEUE_RETRY_NUM,\n            V_QUEUE_NOTIFY_NUM,\n            V_QUEUE_EXPIRES_IN,\n            V_QUEUE_LAST_STATUS,\n            V_QUEUE_LAST_ERROR,\n            V_QUEUE_NAME,\n            V_QUEUE_AGE,\n            V_ASN,\n            V_COUNTRY,\n            V_RECEIVED_FROM_IP,\n            V_RECEIVED_VIA_PORT,\n            V_SOURCE,\n            V_SIZE,\n        ])\n    }\n\n    pub fn with_variables(mut self, variables: &[u32]) -> Self {\n        for (name, idx) in VARIABLES_MAP {\n            if variables.contains(idx) {\n                self.tokens\n                    .insert(Cow::Borrowed(name), Token::Variable(*idx));\n            }\n        }\n\n        self\n    }\n\n    pub fn with_variables_map<I, V>(mut self, vars: I) -> Self\n    where\n        I: IntoIterator<Item = (V, u32)>,\n        V: Into<Cow<'static, str>>,\n    {\n        for (name, idx) in vars {\n            self.tokens.insert(name.into(), Token::Variable(idx));\n        }\n\n        self\n    }\n\n    pub fn set_constants<I, T>(mut self, consts: I) -> Self\n    where\n        I: IntoIterator<Item = (&'static str, T)>,\n        T: Into<Constant>,\n    {\n        for (name, constant) in consts {\n            self.tokens\n                .insert(Cow::Borrowed(name), Token::Constant(constant.into()));\n        }\n\n        self\n    }\n\n    pub fn with_constants<T: ConstantValue>(mut self) -> Self {\n        T::add_constants(&mut self);\n        self\n    }\n\n    pub fn add_constant(&mut self, name: &'static str, constant: impl Into<Constant>) -> &mut Self {\n        self.tokens\n            .insert(Cow::Borrowed(name), Token::Constant(constant.into()));\n        self\n    }\n}\n"
  },
  {
    "path": "crates/common/src/i18n.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\ninclude!(concat!(env!(\"OUT_DIR\"), \"/locales.rs\"));\n\npub fn locale_or_default(name: &str) -> &'static Locale {\n    locale(name)\n        .or_else(|| name.split_once('_').and_then(|(lang, _)| locale(lang)))\n        .unwrap_or(&EN_LOCALES)\n}\n\n#[cfg(test)]\nmod tests {\n    use super::locale;\n\n    #[test]\n    fn calendar_templates_include_minutes() {\n        for lang in [\"en\", \"es\", \"fr\", \"de\", \"it\", \"pt\", \"nl\", \"da\", \"ca\", \"el\", \"sv\", \"pl\"] {\n            let locale = locale(lang).expect(\"locale must exist\");\n            assert!(\n                locale.calendar_date_template.contains(\"%M\"),\n                \"{lang} calendar.date_template must include minutes\"\n            );\n            assert!(\n                locale.calendar_date_template_long.contains(\"%M\"),\n                \"{lang} calendar.date_template_long must include minutes\"\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/ipc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::config::smtp::{\n    queue::QueueName,\n    report::AggregateFrequency,\n    resolver::{Policy, Tlsa},\n};\nuse ahash::RandomState;\nuse mail_auth::{\n    dmarc::Dmarc,\n    mta_sts::TlsRpt,\n    report::{Record, tlsrpt::FailureDetails},\n};\nuse std::{\n    sync::{\n        Arc,\n        atomic::{AtomicBool, Ordering},\n    },\n    time::Instant,\n};\nuse store::{BlobStore, InMemoryStore, Store};\nuse tokio::sync::{Semaphore, SemaphorePermit, mpsc};\nuse types::type_state::{DataType, StateChange};\nuse utils::map::bitmap::Bitmap;\n\npub enum HousekeeperEvent {\n    AcmeReschedule {\n        provider_id: String,\n        renew_at: Instant,\n    },\n    Purge(PurgeType),\n    ReloadSettings,\n    Exit,\n}\n\npub enum PurgeType {\n    Data(Store),\n    Blobs {\n        store: Store,\n        blob_store: BlobStore,\n    },\n    Lookup {\n        store: InMemoryStore,\n        prefix: Option<Vec<u8>>,\n    },\n    Account {\n        account_id: Option<u32>,\n        use_roles: bool,\n    },\n}\n\n#[derive(Debug)]\npub enum PushEvent {\n    Subscribe {\n        account_ids: Vec<u32>,\n        types: Bitmap<DataType>,\n        tx: mpsc::Sender<PushNotification>,\n    },\n    Publish {\n        notification: PushNotification,\n        broadcast: bool,\n    },\n    PushServerRegister {\n        activate: Vec<u32>,\n        expired: Vec<u32>,\n    },\n    PushServerUpdate {\n        account_id: u32,\n        broadcast: bool,\n    },\n    Stop,\n}\n\n#[derive(Debug, Clone)]\npub enum PushNotification {\n    StateChange(StateChange),\n    CalendarAlert(CalendarAlert),\n    EmailPush(EmailPush),\n}\n\n#[derive(Debug, Clone)]\npub struct EmailPush {\n    pub account_id: u32,\n    pub email_id: u32,\n    pub change_id: u64,\n}\n\n#[derive(Debug, Clone)]\npub struct CalendarAlert {\n    pub account_id: u32,\n    pub event_id: u32,\n    pub recurrence_id: Option<i64>,\n    pub uid: String,\n    pub alert_id: String,\n}\n\n#[derive(Debug)]\npub enum BroadcastEvent {\n    PushNotification(PushNotification),\n    InvalidateAccessTokens(Vec<u32>),\n    InvalidateGroupwareCache(Vec<u32>),\n    ReloadPushServers(u32),\n    ReloadSettings,\n    ReloadBlockedIps,\n    ReloadSpamFilter,\n}\n\n#[derive(Debug)]\npub enum QueueEvent {\n    Refresh,\n    WorkerDone {\n        queue_id: u64,\n        queue_name: QueueName,\n        status: QueueEventStatus,\n    },\n    Paused(bool),\n    ReloadSettings,\n    Stop,\n}\n\n#[derive(Debug)]\npub enum QueueEventStatus {\n    Completed,\n    Locked,\n    Deferred,\n}\n\n#[derive(Debug)]\npub enum ReportingEvent {\n    Dmarc(Box<DmarcEvent>),\n    Tls(Box<TlsEvent>),\n    Stop,\n}\n\n#[derive(Debug)]\npub struct DmarcEvent {\n    pub domain: String,\n    pub report_record: Record,\n    pub dmarc_record: Arc<Dmarc>,\n    pub interval: AggregateFrequency,\n}\n\n#[derive(Debug)]\npub struct TlsEvent {\n    pub domain: String,\n    pub policy: PolicyType,\n    pub failure: Option<FailureDetails>,\n    pub tls_record: Arc<TlsRpt>,\n    pub interval: AggregateFrequency,\n}\n\n#[derive(Debug, Hash, PartialEq, Eq)]\npub enum PolicyType {\n    Tlsa(Option<Arc<Tlsa>>),\n    Sts(Option<Arc<Policy>>),\n    None,\n}\n\npub struct TrainTaskController {\n    semaphore: Semaphore,\n    stop_flag: AtomicBool,\n}\n\nimpl Default for TrainTaskController {\n    fn default() -> Self {\n        Self {\n            semaphore: Semaphore::new(1),\n            stop_flag: AtomicBool::new(false),\n        }\n    }\n}\n\nimpl TrainTaskController {\n    pub fn try_run(&self) -> Option<SemaphorePermit<'_>> {\n        let permit = self.semaphore.try_acquire().ok()?;\n\n        self.stop_flag.store(false, Ordering::SeqCst);\n\n        Some(permit)\n    }\n\n    pub fn is_running(&self) -> bool {\n        self.semaphore.available_permits() == 0\n    }\n\n    pub fn stop(&self) {\n        self.stop_flag.store(true, Ordering::SeqCst);\n    }\n\n    pub fn should_stop(&self) -> bool {\n        self.stop_flag.load(Ordering::SeqCst)\n    }\n}\n\npub trait ToHash {\n    fn to_hash(&self) -> u64;\n}\n\nimpl ToHash for Dmarc {\n    fn to_hash(&self) -> u64 {\n        RandomState::with_seeds(1, 9, 7, 9).hash_one(self)\n    }\n}\n\nimpl ToHash for PolicyType {\n    fn to_hash(&self) -> u64 {\n        RandomState::with_seeds(1, 9, 7, 9).hash_one(self)\n    }\n}\n\nimpl From<DmarcEvent> for ReportingEvent {\n    fn from(value: DmarcEvent) -> Self {\n        ReportingEvent::Dmarc(Box::new(value))\n    }\n}\n\nimpl From<TlsEvent> for ReportingEvent {\n    fn from(value: TlsEvent) -> Self {\n        ReportingEvent::Tls(Box::new(value))\n    }\n}\n\nimpl From<Arc<Tlsa>> for PolicyType {\n    fn from(value: Arc<Tlsa>) -> Self {\n        PolicyType::Tlsa(Some(value))\n    }\n}\n\nimpl From<Arc<Policy>> for PolicyType {\n    fn from(value: Arc<Policy>) -> Self {\n        PolicyType::Sts(Some(value))\n    }\n}\n\nimpl From<&Arc<Tlsa>> for PolicyType {\n    fn from(value: &Arc<Tlsa>) -> Self {\n        PolicyType::Tlsa(Some(value.clone()))\n    }\n}\n\nimpl From<&Arc<Policy>> for PolicyType {\n    fn from(value: &Arc<Policy>) -> Self {\n        PolicyType::Sts(Some(value.clone()))\n    }\n}\n\nimpl From<(&Option<Arc<Policy>>, &Option<Arc<Tlsa>>)> for PolicyType {\n    fn from(value: (&Option<Arc<Policy>>, &Option<Arc<Tlsa>>)) -> Self {\n        match value {\n            (Some(value), _) => PolicyType::Sts(Some(value.clone())),\n            (_, Some(value)) => PolicyType::Tlsa(Some(value.clone())),\n            _ => PolicyType::None,\n        }\n    }\n}\n\nimpl PushNotification {\n    pub fn account_id(&self) -> u32 {\n        match self {\n            PushNotification::StateChange(state_change) => state_change.account_id,\n            PushNotification::CalendarAlert(calendar_alert) => calendar_alert.account_id,\n            PushNotification::EmailPush(email_push) => email_push.account_id,\n        }\n    }\n\n    pub fn filter_types(&self, types: &Bitmap<DataType>) -> Option<PushNotification> {\n        match self {\n            PushNotification::StateChange(state_change) => {\n                let mut filtered_types = state_change.types;\n                filtered_types.intersection(types);\n                if !filtered_types.is_empty() {\n                    Some(PushNotification::StateChange(StateChange {\n                        account_id: state_change.account_id,\n                        change_id: state_change.change_id,\n                        types: filtered_types,\n                    }))\n                } else {\n                    None\n                }\n            }\n            PushNotification::CalendarAlert(_) => {\n                if types.contains(DataType::CalendarAlert) {\n                    Some(self.clone())\n                } else {\n                    None\n                }\n            }\n            PushNotification::EmailPush(_) => {\n                if types.contains_any(\n                    [\n                        DataType::EmailDelivery,\n                        DataType::Email,\n                        DataType::Mailbox,\n                        DataType::Thread,\n                    ]\n                    .into_iter(),\n                ) {\n                    Some(self.clone())\n                } else {\n                    None\n                }\n            }\n        }\n    }\n}\n\nimpl EmailPush {\n    pub fn to_state_change(&self) -> StateChange {\n        StateChange {\n            account_id: self.account_id,\n            change_id: self.change_id,\n            types: Bitmap::from_iter([\n                DataType::EmailDelivery,\n                DataType::Email,\n                DataType::Mailbox,\n                DataType::Thread,\n            ]),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#![warn(clippy::large_futures)]\n\nuse ahash::{AHashMap, AHashSet};\nuse arc_swap::ArcSwap;\nuse auth::{AccessToken, oauth::config::OAuthConfig, roles::RolePermissions};\nuse calcard::common::timezone::Tz;\nuse config::{\n    groupware::GroupwareConfig,\n    imap::ImapConfig,\n    jmap::settings::JmapConfig,\n    network::Network,\n    scripts::Scripting,\n    smtp::{\n        SmtpConfig,\n        resolver::{Policy, Tlsa},\n    },\n    spamfilter::{IpResolver, SpamFilterConfig},\n    storage::Storage,\n    telemetry::Metrics,\n};\nuse ipc::{BroadcastEvent, HousekeeperEvent, PushEvent, QueueEvent, ReportingEvent};\nuse listener::{asn::AsnGeoLookupData, blocked::Security, tls::AcmeProviders};\nuse mail_auth::{MX, Txt};\nuse manager::webadmin::{Resource, WebAdminManager};\nuse parking_lot::{Mutex, RwLock};\nuse rustls::sign::CertifiedKey;\nuse std::{\n    hash::{BuildHasher, Hash, Hasher},\n    net::{IpAddr, Ipv4Addr, Ipv6Addr},\n    sync::{Arc, atomic::AtomicBool},\n    time::{Duration, Instant},\n};\nuse store::rand::{Rng, distr::Alphanumeric};\nuse tinyvec::TinyVec;\nuse tokio::sync::{Notify, Semaphore, mpsc};\nuse tokio_rustls::TlsConnector;\nuse types::{acl::AclGrant, special_use::SpecialUse};\nuse utils::{\n    cache::{Cache, CacheItemWeight, CacheWithTtl},\n    snowflake::SnowflakeIdGenerator,\n};\n\npub mod addresses;\npub mod auth;\npub mod config;\npub mod core;\npub mod dns;\npub mod expr;\npub mod i18n;\npub mod ipc;\npub mod listener;\npub mod manager;\npub mod scripts;\npub mod sharing;\npub mod storage;\npub mod telemetry;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n\n#[cfg(feature = \"enterprise\")]\npub mod enterprise;\n\n// SPDX-SnippetEnd\n\npub use psl;\n\nuse crate::{config::spamfilter::SpamClassifier, ipc::TrainTaskController};\n\npub static VERSION_PRIVATE: &str = env!(\"CARGO_PKG_VERSION\");\npub static VERSION_PUBLIC: &str = \"1.0.0\";\n\npub static USER_AGENT: &str = \"Stalwart/1.0.0\";\npub static DAEMON_NAME: &str = concat!(\"Stalwart v\", env!(\"CARGO_PKG_VERSION\"),);\npub static PROD_ID: &str = \"-//Stalwart Labs LLC//Stalwart Server//EN\";\n\n/*\n\nSchema history:\n\n1 - v0.12.0\n2 - v0.12.4\n3 - v0.13.0\n4 - v0.14.0\n5 - v0.15.0\n\n*/\n\npub const DATABASE_SCHEMA_VERSION: u32 = 5;\n\npub const LONG_1D_SLUMBER: Duration = Duration::from_secs(60 * 60 * 24);\npub const LONG_1Y_SLUMBER: Duration = Duration::from_secs(60 * 60 * 24 * 365);\n\npub const IPC_CHANNEL_BUFFER: usize = 1024;\n\npub const KV_ACME: u8 = 0;\npub const KV_OAUTH: u8 = 1;\npub const KV_RATE_LIMIT_RCPT: u8 = 2;\npub const KV_RATE_LIMIT_SCAN: u8 = 3;\npub const KV_RATE_LIMIT_LOITER: u8 = 4;\npub const KV_RATE_LIMIT_AUTH: u8 = 5;\npub const KV_RATE_LIMIT_SMTP: u8 = 6;\npub const KV_RATE_LIMIT_CONTACT: u8 = 7;\npub const KV_RATE_LIMIT_HTTP_AUTHENTICATED: u8 = 8;\npub const KV_RATE_LIMIT_HTTP_ANONYMOUS: u8 = 9;\npub const KV_RATE_LIMIT_IMAP: u8 = 10;\npub const KV_GREYLIST: u8 = 16;\npub const KV_LOCK_PURGE_ACCOUNT: u8 = 20;\npub const KV_LOCK_QUEUE_MESSAGE: u8 = 21;\npub const KV_LOCK_QUEUE_REPORT: u8 = 22;\npub const KV_LOCK_TASK: u8 = 23;\npub const KV_LOCK_HOUSEKEEPER: u8 = 24;\npub const KV_LOCK_DAV: u8 = 25;\npub const KV_SIEVE_ID: u8 = 26;\n\n#[derive(Clone)]\npub struct Server {\n    pub inner: Arc<Inner>,\n    pub core: Arc<Core>,\n}\n\npub struct Inner {\n    pub shared_core: ArcSwap<Core>,\n    pub data: Data,\n    pub cache: Caches,\n    pub ipc: Ipc,\n}\n\npub struct Data {\n    pub spam_classifier: ArcSwap<SpamClassifier>,\n\n    pub tls_certificates: ArcSwap<AHashMap<String, Arc<CertifiedKey>>>,\n    pub tls_self_signed_cert: Option<Arc<CertifiedKey>>,\n\n    pub blocked_ips: RwLock<AHashSet<IpAddr>>,\n\n    pub asn_geo_data: AsnGeoLookupData,\n\n    pub jmap_id_gen: SnowflakeIdGenerator,\n    pub queue_id_gen: SnowflakeIdGenerator,\n    pub span_id_gen: SnowflakeIdGenerator,\n    pub queue_status: AtomicBool,\n\n    pub webadmin: WebAdminManager,\n    pub logos: Mutex<AHashMap<String, Option<Resource<Vec<u8>>>>>,\n\n    pub smtp_connectors: TlsConnectors,\n}\n\npub struct Caches {\n    pub access_tokens: Cache<u32, Arc<AccessToken>>,\n    pub http_auth: Cache<String, HttpAuthCache>,\n    pub permissions: Cache<u32, Arc<RolePermissions>>,\n\n    pub messages: Cache<u32, CacheSwap<MessageStoreCache>>,\n    pub files: Cache<u32, CacheSwap<DavResources>>,\n    pub contacts: Cache<u32, CacheSwap<DavResources>>,\n    pub events: Cache<u32, CacheSwap<DavResources>>,\n    pub scheduling: Cache<u32, CacheSwap<DavResources>>,\n\n    pub dns_txt: CacheWithTtl<String, Txt>,\n    pub dns_mx: CacheWithTtl<String, Arc<Vec<MX>>>,\n    pub dns_ptr: CacheWithTtl<IpAddr, Arc<Vec<String>>>,\n    pub dns_ipv4: CacheWithTtl<String, Arc<Vec<Ipv4Addr>>>,\n    pub dns_ipv6: CacheWithTtl<String, Arc<Vec<Ipv6Addr>>>,\n    pub dns_tlsa: CacheWithTtl<String, Arc<Tlsa>>,\n    pub dbs_mta_sts: CacheWithTtl<String, Arc<Policy>>,\n    pub dns_rbl: CacheWithTtl<String, Option<Arc<IpResolver>>>,\n}\n\n#[derive(Debug, Clone)]\npub struct CacheSwap<T>(pub Arc<ArcSwap<T>>);\n\n#[derive(Debug, Clone)]\npub struct MessageStoreCache {\n    pub emails: Arc<MessagesCache>,\n    pub mailboxes: Arc<MailboxesCache>,\n    pub update_lock: Arc<Semaphore>,\n    pub last_change_id: u64,\n    pub size: u64,\n}\n\n#[derive(Debug, Clone)]\npub struct MailboxesCache {\n    pub change_id: u64,\n    pub index: AHashMap<u32, u32>,\n    pub items: Box<[MailboxCache]>,\n    pub size: u64,\n}\n\n#[derive(Debug, Clone)]\npub struct MessagesCache {\n    pub change_id: u64,\n    pub items: Box<[MessageCache]>,\n    pub index: AHashMap<u32, u32>,\n    pub keywords: Box<[Box<str>]>,\n    pub size: u64,\n}\n\n#[derive(Debug, Clone)]\npub struct MessageCache {\n    pub document_id: u32,\n    pub mailboxes: TinyVec<[MessageUidCache; 2]>,\n    pub keywords: u128,\n    pub thread_id: u32,\n    pub change_id: u64,\n    pub size: u32,\n}\n\n#[derive(Debug, Default, Clone, Copy)]\npub struct MessageUidCache {\n    pub mailbox_id: u32,\n    pub uid: u32,\n}\n\n#[derive(Debug, Clone)]\npub struct MailboxCache {\n    pub document_id: u32,\n    pub name: String,\n    pub path: String,\n    pub role: SpecialUse,\n    pub parent_id: u32,\n    pub sort_order: u32,\n    pub subscribers: TinyVec<[u32; 4]>,\n    pub uid_validity: u32,\n    pub acls: TinyVec<[AclGrant; 2]>,\n}\n\n#[derive(Debug, Clone)]\npub struct HttpAuthCache {\n    pub account_id: u32,\n    pub revision: u64,\n    pub expires: Instant,\n}\n\npub struct Ipc {\n    pub push_tx: mpsc::Sender<PushEvent>,\n    pub housekeeper_tx: mpsc::Sender<HousekeeperEvent>,\n    pub task_tx: Arc<Notify>,\n    pub queue_tx: mpsc::Sender<QueueEvent>,\n    pub report_tx: mpsc::Sender<ReportingEvent>,\n    pub broadcast_tx: Option<mpsc::Sender<BroadcastEvent>>,\n    pub train_task_controller: Arc<TrainTaskController>,\n}\n\npub struct TlsConnectors {\n    pub pki_verify: TlsConnector,\n    pub dummy_verify: TlsConnector,\n}\n\npub struct NameWrapper(pub String);\n\n#[derive(Debug, Clone)]\npub struct DavResources {\n    pub base_path: String,\n    pub paths: AHashSet<DavPath>,\n    pub resources: Vec<DavResource>,\n    pub item_change_id: u64,\n    pub container_change_id: u64,\n    pub highest_change_id: u64,\n    pub size: u64,\n    pub update_lock: Arc<Semaphore>,\n}\n\n#[derive(Debug, Clone)]\npub struct DavPath {\n    pub path: String,\n    pub parent_id: Option<u32>,\n    pub hierarchy_seq: u32,\n    pub resource_idx: usize,\n}\n\n#[derive(Debug, Clone)]\npub struct DavResource {\n    pub document_id: u32,\n    pub data: DavResourceMetadata,\n}\n\n#[derive(Debug, Clone, Copy)]\npub struct DavResourcePath<'x> {\n    pub path: &'x DavPath,\n    pub resource: &'x DavResource,\n}\n\n#[derive(Debug, Clone)]\npub enum DavResourceMetadata {\n    File {\n        name: String,\n        size: Option<u32>,\n        parent_id: Option<u32>,\n        acls: TinyVec<[AclGrant; 2]>,\n    },\n    Calendar {\n        name: String,\n        acls: TinyVec<[AclGrant; 2]>,\n        preferences: TinyVec<[TinyCalendarPreferences; 2]>,\n    },\n    CalendarEvent {\n        names: TinyVec<[DavName; 2]>,\n        start: i64,\n        duration: u32,\n    },\n    CalendarEventNotification {\n        names: TinyVec<[DavName; 2]>,\n    },\n    AddressBook {\n        name: String,\n        acls: TinyVec<[AclGrant; 2]>,\n    },\n    ContactCard {\n        names: TinyVec<[DavName; 2]>,\n    },\n}\n\n#[derive(Debug, Clone, Default)]\npub struct TinyCalendarPreferences {\n    pub account_id: u32,\n    pub tz: Tz,\n    pub flags: u16,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct DavName {\n    pub name: String,\n    pub parent_id: u32,\n}\n\n#[derive(Clone, Default)]\npub struct Core {\n    pub storage: Storage,\n    pub sieve: Scripting,\n    pub network: Network,\n    pub acme: AcmeProviders,\n    pub oauth: OAuthConfig,\n    pub smtp: SmtpConfig,\n    pub jmap: JmapConfig,\n    pub groupware: GroupwareConfig,\n    pub spam: SpamFilterConfig,\n    pub imap: ImapConfig,\n    pub metrics: Metrics,\n\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    pub enterprise: Option<enterprise::Enterprise>,\n    // SPDX-SnippetEnd\n}\n\nimpl<T: CacheItemWeight> CacheItemWeight for CacheSwap<T> {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<CacheSwap<T>>() as u64 + self.0.load().weight()\n    }\n}\n\nimpl CacheItemWeight for MessageStoreCache {\n    fn weight(&self) -> u64 {\n        self.size\n    }\n}\n\nimpl CacheItemWeight for HttpAuthCache {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<HttpAuthCache>() as u64\n    }\n}\n\nimpl CacheItemWeight for DavResources {\n    fn weight(&self) -> u64 {\n        self.size\n    }\n}\n\npub trait IntoString: Sized {\n    fn into_string(self) -> String;\n}\n\nimpl IntoString for Vec<u8> {\n    fn into_string(self) -> String {\n        String::from_utf8(self)\n            .unwrap_or_else(|err| String::from_utf8_lossy(err.as_bytes()).into_owned())\n    }\n}\n\n#[derive(Debug, Clone, Eq)]\npub struct ThrottleKey {\n    pub hash: [u8; 32],\n}\n\nimpl PartialEq for ThrottleKey {\n    fn eq(&self, other: &Self) -> bool {\n        self.hash == other.hash\n    }\n}\n\nimpl std::hash::Hash for ThrottleKey {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.hash.hash(state);\n    }\n}\n\nimpl AsRef<[u8]> for ThrottleKey {\n    fn as_ref(&self) -> &[u8] {\n        &self.hash\n    }\n}\n\n#[derive(Default)]\npub struct ThrottleKeyHasher {\n    hash: u64,\n}\n\nimpl Hasher for ThrottleKeyHasher {\n    fn finish(&self) -> u64 {\n        self.hash\n    }\n\n    fn write(&mut self, bytes: &[u8]) {\n        debug_assert!(\n            bytes.len() >= std::mem::size_of::<u64>(),\n            \"ThrottleKeyHasher: input too short {bytes:?}\"\n        );\n        self.hash = bytes\n            .get(0..std::mem::size_of::<u64>())\n            .map_or(0, |b| u64::from_ne_bytes(b.try_into().unwrap()));\n    }\n}\n\n#[derive(Clone, Default)]\npub struct ThrottleKeyHasherBuilder {}\n\nimpl BuildHasher for ThrottleKeyHasherBuilder {\n    type Hasher = ThrottleKeyHasher;\n\n    fn build_hasher(&self) -> Self::Hasher {\n        ThrottleKeyHasher::default()\n    }\n}\n\n#[cfg(feature = \"test_mode\")]\n#[allow(clippy::derivable_impls)]\nimpl Default for Server {\n    fn default() -> Self {\n        Self {\n            inner: Default::default(),\n            core: Default::default(),\n        }\n    }\n}\n\n#[cfg(feature = \"test_mode\")]\n#[allow(clippy::derivable_impls)]\nimpl Default for Inner {\n    fn default() -> Self {\n        Self {\n            shared_core: Default::default(),\n            data: Default::default(),\n            ipc: Default::default(),\n            cache: Default::default(),\n        }\n    }\n}\n\n#[cfg(feature = \"test_mode\")]\n#[allow(clippy::derivable_impls)]\nimpl Default for Caches {\n    fn default() -> Self {\n        Self {\n            access_tokens: Cache::new(1024, 10 * 1024 * 1024),\n            http_auth: Cache::new(1024, 10 * 1024 * 1024),\n            permissions: Cache::new(1024, 10 * 1024 * 1024),\n            messages: Cache::new(1024, 25 * 1024 * 1024),\n            files: Cache::new(1024, 10 * 1024 * 1024),\n            contacts: Cache::new(1024, 10 * 1024 * 1024),\n            events: Cache::new(1024, 10 * 1024 * 1024),\n            scheduling: Cache::new(1024, 10 * 1024 * 1024),\n            dns_rbl: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n            dns_txt: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n            dns_mx: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n            dns_ptr: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n            dns_ipv4: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n            dns_ipv6: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n            dns_tlsa: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n            dbs_mta_sts: CacheWithTtl::new(1024, 10 * 1024 * 1024),\n        }\n    }\n}\n\n#[cfg(feature = \"test_mode\")]\nimpl Default for Ipc {\n    fn default() -> Self {\n        Self {\n            push_tx: mpsc::channel(IPC_CHANNEL_BUFFER).0,\n            housekeeper_tx: mpsc::channel(IPC_CHANNEL_BUFFER).0,\n            task_tx: Default::default(),\n            queue_tx: mpsc::channel(IPC_CHANNEL_BUFFER).0,\n            report_tx: mpsc::channel(IPC_CHANNEL_BUFFER).0,\n            broadcast_tx: None,\n            train_task_controller: Arc::new(TrainTaskController::default()),\n        }\n    }\n}\n\npub fn ip_to_bytes(ip: &IpAddr) -> Vec<u8> {\n    match ip {\n        IpAddr::V4(ip) => ip.octets().to_vec(),\n        IpAddr::V6(ip) => ip.octets().to_vec(),\n    }\n}\n\npub fn ip_to_bytes_prefix(prefix: u8, ip: &IpAddr) -> Vec<u8> {\n    match ip {\n        IpAddr::V4(ip) => {\n            let mut buf = Vec::with_capacity(5);\n            buf.push(prefix);\n            buf.extend_from_slice(&ip.octets());\n            buf\n        }\n        IpAddr::V6(ip) => {\n            let mut buf = Vec::with_capacity(17);\n            buf.push(prefix);\n            buf.extend_from_slice(&ip.octets());\n            buf\n        }\n    }\n}\n\nimpl DavResourcePath<'_> {\n    #[inline(always)]\n    pub fn document_id(&self) -> u32 {\n        self.resource.document_id\n    }\n\n    #[inline(always)]\n    pub fn parent_id(&self) -> Option<u32> {\n        self.path.parent_id\n    }\n\n    #[inline(always)]\n    pub fn path(&self) -> &str {\n        self.path.path.as_str()\n    }\n\n    #[inline(always)]\n    pub fn is_container(&self) -> bool {\n        self.resource.is_container()\n    }\n\n    #[inline(always)]\n    pub fn hierarchy_seq(&self) -> u32 {\n        self.path.hierarchy_seq\n    }\n\n    #[inline(always)]\n    pub fn size(&self) -> u32 {\n        self.resource.size().unwrap_or_default()\n    }\n}\n\nimpl DavResources {\n    pub fn by_path(&self, name: &str) -> Option<DavResourcePath<'_>> {\n        self.paths.get(name).map(|path| DavResourcePath {\n            path,\n            resource: &self.resources[path.resource_idx],\n        })\n    }\n\n    pub fn container_resource_by_id(&self, id: u32) -> Option<&DavResource> {\n        self.resources\n            .iter()\n            .find(|res| res.document_id == id && res.is_container())\n    }\n\n    pub fn container_resource_path_by_id(&self, id: u32) -> Option<DavResourcePath<'_>> {\n        self.resources\n            .iter()\n            .enumerate()\n            .find(|(_, resource)| resource.document_id == id && resource.is_container())\n            .and_then(|(idx, resource)| {\n                self.paths\n                    .iter()\n                    .find(|path| path.resource_idx == idx)\n                    .map(|path| DavResourcePath { path, resource })\n            })\n    }\n\n    pub fn any_resource_path_by_id(&self, id: u32) -> Option<DavResourcePath<'_>> {\n        self.resources\n            .iter()\n            .enumerate()\n            .find(|(_, resource)| resource.document_id == id)\n            .and_then(|(idx, resource)| {\n                self.paths\n                    .iter()\n                    .find(|path| path.resource_idx == idx)\n                    .map(|path| DavResourcePath { path, resource })\n            })\n    }\n\n    pub fn subtree(&self, search_path: &str) -> impl Iterator<Item = DavResourcePath<'_>> {\n        let prefix = format!(\"{search_path}/\");\n        self.paths.iter().filter_map(move |path| {\n            if path.path.starts_with(&prefix) || path.path == search_path {\n                Some(DavResourcePath {\n                    path,\n                    resource: &self.resources[path.resource_idx],\n                })\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn subtree_with_depth(\n        &self,\n        search_path: &str,\n        depth: usize,\n    ) -> impl Iterator<Item = DavResourcePath<'_>> {\n        let prefix = format!(\"{search_path}/\");\n        self.paths.iter().filter_map(move |path| {\n            if path\n                .path\n                .strip_prefix(&prefix)\n                .is_some_and(|name| name.as_bytes().iter().filter(|&&c| c == b'/').count() < depth)\n                || path.path.as_str() == search_path\n            {\n                Some(DavResourcePath {\n                    path,\n                    resource: &self.resources[path.resource_idx],\n                })\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn tree_with_depth(&self, depth: usize) -> impl Iterator<Item = DavResourcePath<'_>> {\n        self.paths.iter().filter_map(move |path| {\n            if path.path.as_bytes().iter().filter(|&&c| c == b'/').count() <= depth {\n                Some(DavResourcePath {\n                    path,\n                    resource: &self.resources[path.resource_idx],\n                })\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn children(&self, parent_id: u32) -> impl Iterator<Item = DavResourcePath<'_>> {\n        self.paths\n            .iter()\n            .filter(move |item| item.parent_id.is_some_and(|id| id == parent_id))\n            .map(|path| DavResourcePath {\n                path,\n                resource: &self.resources[path.resource_idx],\n            })\n    }\n\n    pub fn children_ids(&self, parent_id: u32) -> impl Iterator<Item = u32> {\n        self.paths\n            .iter()\n            .filter(move |item| item.parent_id.is_some_and(|id| id == parent_id))\n            .map(|path| self.resources[path.resource_idx].document_id)\n    }\n\n    pub fn format_resource(&self, resource: DavResourcePath<'_>) -> String {\n        if resource.resource.is_container() {\n            format!(\"{}{}/\", self.base_path, resource.path.path)\n        } else {\n            format!(\"{}{}\", self.base_path, resource.path.path)\n        }\n    }\n\n    pub fn format_collection(&self, name: &str) -> String {\n        format!(\"{}{name}/\", self.base_path)\n    }\n\n    pub fn format_item(&self, name: &str) -> String {\n        format!(\"{}{}\", self.base_path, name)\n    }\n}\n\nconst SCHEDULE_INBOX_ID: u32 = u32::MAX - 1;\n\nimpl DavResource {\n    pub fn is_child_of(&self, parent_id: u32) -> bool {\n        match &self.data {\n            DavResourceMetadata::File { parent_id: id, .. } => id.is_some_and(|id| id == parent_id),\n            DavResourceMetadata::CalendarEvent { names, .. } => {\n                names.iter().any(|name| name.parent_id == parent_id)\n            }\n            DavResourceMetadata::ContactCard { names } => {\n                names.iter().any(|name| name.parent_id == parent_id)\n            }\n            DavResourceMetadata::CalendarEventNotification { names } => {\n                names.is_empty() && parent_id == SCHEDULE_INBOX_ID\n            }\n            _ => false,\n        }\n    }\n\n    pub fn parent_id(&self) -> Option<u32> {\n        match &self.data {\n            DavResourceMetadata::File { parent_id, .. } => *parent_id,\n            DavResourceMetadata::CalendarEvent { names, .. } => {\n                names.first().map(|name| name.parent_id)\n            }\n            DavResourceMetadata::ContactCard { names } => names.first().map(|name| name.parent_id),\n            DavResourceMetadata::CalendarEventNotification { names } if names.is_empty() => {\n                Some(SCHEDULE_INBOX_ID)\n            }\n            _ => None,\n        }\n    }\n\n    pub fn child_names(&self) -> Option<&[DavName]> {\n        match &self.data {\n            DavResourceMetadata::CalendarEvent { names, .. } => Some(names.as_slice()),\n            DavResourceMetadata::ContactCard { names } => Some(names.as_slice()),\n            DavResourceMetadata::CalendarEventNotification { names } if !names.is_empty() => {\n                Some(names.as_slice())\n            }\n            _ => None,\n        }\n    }\n\n    pub fn container_name(&self) -> Option<&str> {\n        match &self.data {\n            DavResourceMetadata::File { name, .. } => Some(name.as_str()),\n            DavResourceMetadata::Calendar { name, .. } => Some(name.as_str()),\n            DavResourceMetadata::AddressBook { name, .. } => Some(name.as_str()),\n            DavResourceMetadata::CalendarEventNotification { names } if names.is_empty() => {\n                Some(if self.document_id == SCHEDULE_INBOX_ID {\n                    \"inbox\"\n                } else {\n                    \"outbox\"\n                })\n            }\n            _ => None,\n        }\n    }\n\n    pub fn has_hierarchy_changes(&self, other: &DavResource) -> bool {\n        match (&self.data, &other.data) {\n            (\n                DavResourceMetadata::File {\n                    name: a,\n                    parent_id: c,\n                    ..\n                },\n                DavResourceMetadata::File {\n                    name: b,\n                    parent_id: d,\n                    ..\n                },\n            ) => a != b || c != d,\n            (\n                DavResourceMetadata::Calendar { name: a, .. },\n                DavResourceMetadata::Calendar { name: b, .. },\n            ) => a != b,\n            (\n                DavResourceMetadata::AddressBook { name: a, .. },\n                DavResourceMetadata::AddressBook { name: b, .. },\n            ) => a != b,\n            (\n                DavResourceMetadata::CalendarEvent { names: a, .. },\n                DavResourceMetadata::CalendarEvent { names: b, .. },\n            ) => a != b,\n            (\n                DavResourceMetadata::ContactCard { names: a, .. },\n                DavResourceMetadata::ContactCard { names: b, .. },\n            ) => a != b,\n            (\n                DavResourceMetadata::CalendarEventNotification { names: a, .. },\n                DavResourceMetadata::CalendarEventNotification { names: b, .. },\n            ) => a != b,\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn event_time_range(&self) -> Option<(i64, i64)> {\n        match &self.data {\n            DavResourceMetadata::CalendarEvent {\n                start, duration, ..\n            } => Some((*start, *start + *duration as i64)),\n            _ => None,\n        }\n    }\n\n    pub fn calendar_preferences(&self, account_id: u32) -> Option<&TinyCalendarPreferences> {\n        match &self.data {\n            DavResourceMetadata::Calendar { preferences, .. } => preferences\n                .iter()\n                .find(|pref| pref.account_id == account_id)\n                .or_else(|| preferences.first()),\n            _ => None,\n        }\n    }\n\n    pub fn is_container(&self) -> bool {\n        match &self.data {\n            DavResourceMetadata::File { size, .. } => size.is_none(),\n            DavResourceMetadata::Calendar { .. } | DavResourceMetadata::AddressBook { .. } => true,\n            DavResourceMetadata::CalendarEventNotification { names } => names.is_empty(),\n            _ => false,\n        }\n    }\n\n    pub fn size(&self) -> Option<u32> {\n        match &self.data {\n            DavResourceMetadata::File { size, .. } => *size,\n            _ => None,\n        }\n    }\n\n    pub fn acls(&self) -> Option<&[AclGrant]> {\n        match &self.data {\n            DavResourceMetadata::File { acls, .. } => Some(acls.as_slice()),\n            DavResourceMetadata::Calendar { acls, .. } => Some(acls.as_slice()),\n            DavResourceMetadata::AddressBook { acls, .. } => Some(acls.as_slice()),\n            _ => None,\n        }\n    }\n}\n\nimpl Hash for DavPath {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.path.hash(state);\n    }\n}\n\nimpl PartialEq for DavPath {\n    fn eq(&self, other: &Self) -> bool {\n        self.path == other.path\n    }\n}\n\nimpl Eq for DavPath {}\n\nimpl std::borrow::Borrow<str> for DavPath {\n    fn borrow(&self) -> &str {\n        &self.path\n    }\n}\n\nimpl std::hash::Hash for DavResource {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.document_id.hash(state);\n    }\n}\n\nimpl PartialEq for DavResource {\n    fn eq(&self, other: &Self) -> bool {\n        self.document_id == other.document_id\n    }\n}\n\nimpl Eq for DavResource {}\n\nimpl std::borrow::Borrow<u32> for DavResource {\n    fn borrow(&self) -> &u32 {\n        &self.document_id\n    }\n}\n\nimpl DavName {\n    pub fn new(name: String, parent_id: u32) -> Self {\n        Self { name, parent_id }\n    }\n\n    pub fn new_with_rand_name(parent_id: u32) -> Self {\n        Self {\n            name: store::rand::rng()\n                .sample_iter(Alphanumeric)\n                .take(10)\n                .map(char::from)\n                .collect::<String>(),\n            parent_id,\n        }\n    }\n}\n\nimpl<T> CacheSwap<T> {\n    pub fn new(value: Arc<T>) -> Self {\n        Self(Arc::new(ArcSwap::new(value)))\n    }\n\n    pub fn load_full(&self) -> Arc<T> {\n        self.0.load_full()\n    }\n\n    pub fn update(&self, value: Arc<T>) {\n        self.0.store(value);\n    }\n}\n\nimpl MailboxCache {\n    pub fn parent_id(&self) -> Option<u32> {\n        if self.parent_id != u32::MAX {\n            Some(self.parent_id)\n        } else {\n            None\n        }\n    }\n\n    pub fn sort_order(&self) -> Option<u32> {\n        if self.sort_order != u32::MAX {\n            Some(self.sort_order)\n        } else {\n            None\n        }\n    }\n\n    pub fn is_root(&self) -> bool {\n        self.parent_id == u32::MAX\n    }\n}\n\npub const DEFAULT_LOGO_RAW: &str = r#\"<svg xmlns=\"http://www.w3.org/2000/svg\" xml:space=\"preserve\" id=\"Layer_1\" x=\"0\" y=\"0\" style=\"enable-background:new 0 0 680.5 252.1\" version=\"1.1\" viewBox=\"0 0 680.5 252.1\">\n<style>\n .st0{fill:#100e42}.st1{fill:#db2d54}\n</style>\n<path d=\"M227.8 143.6c.3 4.2 2.1 7.6 5.1 10.1 3.1 2.5 7.1 3.8 12.1 3.8 4.3 0 7.9-.9 10.5-2.8 2.7-1.9 4-4.5 4-7.8 0-2.4-.7-4.3-2.2-5.7-1.5-1.4-3.4-2.5-6-3.2-2.5-.7-6-1.5-10.6-2.3-4.6-.8-8.6-1.9-11.9-3.2-3.3-1.3-6-3.3-8.1-6.1-2.1-2.7-3.1-6.3-3.1-10.7 0-4.1 1.1-7.7 3.2-10.9s5.1-5.7 9-7.4c3.8-1.8 8.2-2.6 13.2-2.6 5.1 0 9.6 1 13.7 2.9 4 1.9 7.2 4.5 9.5 7.8s3.6 7.1 3.8 11.4h-11.5c-.4-3.7-2-6.6-4.8-8.9-2.8-2.2-6.3-3.4-10.6-3.4-4.1 0-7.5.9-9.9 2.7-2.5 1.8-3.7 4.3-3.7 7.6 0 2.3.7 4.1 2.2 5.5 1.5 1.4 3.4 2.4 5.9 3.1 2.4.7 5.9 1.4 10.5 2.2 4.6.8 8.6 1.9 11.9 3.3 3.3 1.4 6 3.4 8.2 6 2.1 2.6 3.2 6.1 3.2 10.5 0 4.2-1.1 8-3.4 11.3-2.2 3.3-5.4 5.9-9.4 7.8-4 1.9-8.6 2.8-13.7 2.8-5.6 0-10.6-1-14.9-3.1-4.3-2-7.6-4.9-10-8.5-2.4-3.6-3.7-7.8-3.7-12.5l11.5.3zM278.5 102.1l11-2.1v14.6h12.6v9.7h-12.6v27.2c0 2 .4 3.5 1.2 4.3.8.9 2.2 1.3 4.2 1.3h8.4v9.7h-10.6c-5 0-8.6-1.2-10.8-3.5-2.2-2.3-3.4-5.9-3.4-10.7v-50.5zM356.8 114.6v52.2h-9.7l-1.2-7.9c-1.8 2.6-4.2 4.7-7 6.2-2.9 1.6-6.2 2.3-10 2.3-4.8 0-9-1.1-12.7-3.2-3.7-2.1-6.7-5.2-8.8-9.3-2.1-4-3.2-8.8-3.2-14.2 0-5.3 1.1-10 3.2-14s5.1-7.2 8.8-9.4c3.7-2.2 7.9-3.3 12.6-3.3 3.9 0 7.2.7 10.1 2.2 2.9 1.5 5.2 3.5 6.9 6.1l1.3-7.6h9.7zm-15.1 38.7c2.8-3.2 4.2-7.3 4.2-12.4 0-5.2-1.4-9.4-4.2-12.6-2.8-3.3-6.5-4.9-11-4.9-4.6 0-8.2 1.6-11 4.8-2.8 3.2-4.2 7.4-4.2 12.5 0 5.2 1.4 9.4 4.2 12.6 2.8 3.2 6.5 4.8 11 4.8s8.2-1.6 11-4.8zM365.5 97.5l11-2.1v71.3h-11V97.5zM380.3 114.6h11.6l11.9 39.9 11.9-39.9h10.1l11.4 39.9 12.3-39.9h11.2l-17.3 52.2h-11.8l-11-35.5-11.4 35.5-11.9.1-17-52.3zM513.7 114.6v52.2H504l-1.2-7.9c-1.8 2.6-4.2 4.7-7 6.2-2.9 1.6-6.2 2.3-10 2.3-4.8 0-9-1.1-12.7-3.2-3.7-2.1-6.7-5.2-8.8-9.3-2.1-4-3.2-8.8-3.2-14.2 0-5.3 1.1-10 3.2-14s5.1-7.2 8.8-9.4c3.7-2.2 7.9-3.3 12.6-3.3 3.9 0 7.2.7 10.1 2.2 2.9 1.5 5.2 3.5 6.9 6.1l1.3-7.6h9.7zm-15.1 38.7c2.8-3.2 4.2-7.3 4.2-12.4 0-5.2-1.4-9.4-4.2-12.6-2.8-3.3-6.5-4.9-11-4.9-4.6 0-8.2 1.6-11 4.8-2.8 3.2-4.2 7.4-4.2 12.5 0 5.2 1.4 9.4 4.2 12.6 2.8 3.2 6.5 4.8 11 4.8 4.6 0 8.2-1.6 11-4.8zM551.3 114.6v10.3h-4.9c-4.6 0-7.8 1.5-9.9 4.4-2 3-3.1 6.7-3.1 11.3v26.2h-11v-52.2h9.8l1.2 7.8c1.5-2.4 3.4-4.4 5.8-5.8 2.4-1.4 5.6-2.1 9.6-2.1h2.5zM556.3 102.1l11-2.1v14.6h12.6v9.7h-12.6v27.2c0 2 .4 3.5 1.2 4.3.8.9 2.2 1.3 4.2 1.3h8.4v9.7h-10.6c-5 0-8.6-1.2-10.8-3.5s-3.4-5.9-3.4-10.7v-50.5z\" class=\"st0\"/>\n<path d=\"M149.1 84.7h-4.8l-44.8 25.9v8.3l44.8 25.9h4.8l44.8-25.9v-8.3l-44.8-25.9zm32.9 30h-35.3V94.4l35.3 20.3zm-35.3 20.4-35.3-20.4 27-15.6v20.2l6.3 3.6h22.9l-20.9 12.2z\" class=\"st1\"/>\n<path d=\"M99.5 129.9v11l44.8 25.9h4.8l44.8-25.9v-11l-47.2 27.3zM187.3 166.8l6.6-3.8v-11l-25.7 14.8zM99.5 163l6.6 3.8h19.1L99.5 152z\" class=\"st1\"/>\n</svg>\n\"#;\n\npub const DEFAULT_LOGO_BASE64: &str =\n    \"iVBORw0KGgoAAAANSUhEUgAAAMgAAAAnCAMAAAB9lPf7AAABOFBMVEUAAAAAADoAAEkPDkIPDkIQ\nDkIQDkLcLVTYMVTbLVTbLVQPDkLWM2YQDkIPD0IODEHaJlPbLVQWDT8MC0IQDkIQDkIPDkIODkEQ\nD0ITDEAQDkIPDkIODkIPDkIPDkIRDUIQDkIQDUIPDkLcLVQQDkIQDkIQDkIQDULbLVQQDUIPDkIQ\nEEIPDkIPD0EPDkHcLlUPDkLbLFQPDULbLFPbLVQQDkLbLFPcK1TbLVTbLVQQDUIPDkIPDkIPDULc\nLVTcLVQQDkLbLFTcLFTbLFQPDkIQDkIQDULbLVUSEkPcLVTbLVQQDULbLFTcLVTbLVTbLVPbLFQP\nDUHbLVMPDkPbLVTdLFXbLFQNDULcLVTcLlMRD0cQDkIQDkXpMFnrMFrtMFvlL1jjLlfdLVThLlYS\nEErnL1gRD0n0Ml2YjG1wAAAAWnRSTlMABAb59/379wr7/lMF9HgoBvQJF/BtZSQwGu7JNulAO95x\nWD3TsF1M3b6kH5MRiRe5saujyH9oHezk4tjGmn9fwkc1vrVqYA8N19DPqnFQkYh1V0a4LSITmSiJ\nLN30AAAKZUlEQVRYw91ZCVvbRhCVrcuWD7AxYLANtrEx5opDIBCOQBLO5qTQ0uyubyD//x/0za4O\nRIG0/dLvazu0tbTaWc3bOd6sqv2PJen9/LcF9o++fnf58r8OBcb/vNBttzvrc0Ck/VcFMN6+a/fs\nuG23+x9+fcopkUiE/ks/35ew2j8ucMCz3wbteNwZtIcZp2O/jj3mlWjo+t8nHzNdx3ac/sLrTDee\nGXRf7cMpD+OYnF9dO6zvnc/gOuLv93dBRScTkz/KWrwu+mBUfTrrwhHtod1b0J696N/E7T5S5UEo\nxbrBlIidyp90CvCO1cu56you/jFBAP1y2evZdq/z4suwv3CiaXPr7X7Gbg9fbP4xvtKccSNlGEYq\nZQpWG3GRTM2ki49biUkjgpks8aOAZCcT6ZX7ME4+Izkyw85Pn7T9m+FCjMb2X3UHSJXM62TIKxFt\nlXHdEPAG/jUNnelFQjJVM8T1BK4e98hIiqd+DBCskcgZjE/dW+zlFiVHd+sjXfcICEnsS7wTtwfd\nhZ8DJFBsMNPirLxabSQqTSFMQ5RWyErBDTH/NBCdGz8MyPm1wUtjocXe/NQZZuLtwednMN4FkpSO\n+vqhh1Tp3bx7qwXyXuhcLLsLNHJAwvZg/6TBLRYGEoE8AOTplAqrPx6nR8wQ01N3Fht90evF4+2b\ny1+k7QEQLTkqQfbt+M3wi29NUZgWO8RSJLg1ucVTY6hj5j0geCh//pZHAvX7s+nNtOr4fSCbw0Hc\ncdbd8ElKIIESMf3AsftbSW8nJphusqqWdTMOGQNgVRdINUQvkZWrLP24Ix6Q74qn46kHEtwqIKda\nIJtbDgQ4YpoLxMk89+Xy+W/rAyc+OPOBpBki6wIX7v2s4CZblUBMXjtstgoNLYK/6PxarVQqt8Zh\nzEThEKMKSCPaah4WKporK/Vm6yCP+SqDC61ma0paW10k9SYVpirUq3Jqcw23F8tr0K8wg+sHrSbN\nl/LMadvxwfDyKyKL/vn4DQSipNdG19XtO3G7+8rf6Ap5pKJlPZ9H8/l8ccoFQrXsukIoG2Vcco6R\nUkNrXfPrI83zSA135YhXOm6hseHt8yEe1Wh4puapT89rixjdQF3kGFvKHtA7FskOeow/RbGKCG2n\n7Xw5kdG1+bzXaUvp9uIkjiRGH0iCWSafzitmDUhdATFNpP4RZm0wYRm6EFTKxHxBpMSEC6QKE1I6\nB+NItWWWMsSupmRsmudok7QjqoaGVOdsoiVS7Fw+tfhx4dpI5cSyAoL3cWPyTmvScWx0JC9din+h\n5POH+MBx4rJVCWTKgLrQV4tZt7QADWxSQCAWgCCATcsSorxbKDGMUUELgOQt/KSBltR3hMW5vuRG\nFowz84glMJUl2PbuLtRNqOssDSA5WJ7CdeAREssDEqNmcQg+7Ldp42PJoCwPHLgq/iWG0WDrl29T\nWJzx7cONxEjUTz9KdpFeQqCNaXm80WKFGWBdmS8JHdABxE/2XWGwA68GcuBgx4CFu0WWYvDOEqw1\nWe0C6qeJMtR9ILRVopmeqMxQsvPczCTeB9UACrXvGftGNu8xd6SD0PJHAokc3ILPLcoHxsuL1RUM\nuUC8klRnusXqXryUsekhIGkGnyxBi0JQJycuyiWy20JavMcMizW9TgS8FQJSfaRqPfuEFB+VB6qB\nHVfNu+sjGz3LG8DA0093XBJdRIaZOgmhKW1g0OeRLHEjDBZlTJSGSP+EgOTJdtXN7FJkmWI7S+l3\nIWBoXpvKcZ26haxSh39MH4iOZEFqRh7ikcE7UGGMjrh2x3Gb949bXVx2tl5iGI/mzohHgkI+u1gS\nsqpYhoFYLqxo0YAQI0Q1iJagQi8y/S6QCJoDg61pEQlZ7BSELmaI5pZhW0GT5QSGZz31VWZ5QHRA\nVlXCAxIJPJLpqOYEmF4M2zZS5d16Fz1Lx/l8Iod//TDsnmlhOb3YqO+kBMBYpkFxfRfIHkuJ3NSd\n7iwMJIuAMtSWTyApNiZuU2yV7N4RBhm8qlI+YKogRwwZhA8D2cygXfS3fl12JNTQ954rR8Veo3V0\nFkbvdkBu2V1qrL5HncTy46EW5RDm1QLQ8lEABFPyVPgasL3ODFacFJbYpcTnkBGNHCi2Tz0TI1R2\nfY/IaveYR7po4LuUDKNu827b6j6WdJv5+Dd4JCwemEZJwKhCCMgBgBz4M5UFodBCbKXYsqZdlbjY\njmo1IclgA2oEqA4gO8GLqAYEQMYfBYIjleuBrzBdNu/tNjX0SSpdOF5RMcPxypVssUgFNjhrFok/\nSlfaiOUDacKiwl3qSYWBRMhmUctqDYHOmWLJYMdIfKhRaqwBSPlOr3la+nNAvENuB9wuoXzd3/dy\nBgfeTMDrpL9kCo5AjfqOgdk6zy3dBbIGM8ungQ5VqRAQFUVFmd3owWYo92kB1CwA2aMFx4LQWjL4\nnwIympSfHRS3K0dIGMnX7uA+BZ3vEbCCKAdNaJQ2kE+PBUAitMMwM+h6j8LJTuM12XXsCF46pTW5\nKMnEL+BGcrZoBMleFeb3gYS5XW1+LBaTvNIhN4V5XUWwicYp6582doCshhxxgeCv4R22ICojrDCQ\nqIymwxGhUxWKkg9EHrROqeyeeOq4UrUFLrceBxINkv1jUkv63N6TTC7v+rbP9JgUC5pGHXVqlq6j\ntEqF9mtZJrvOqpIQT7cFtfowEUJJbPH7QGaJBfdAIChiWFMYYrEMXlJFt0bq85qm1I+Y+Vho5a5k\nBCjZ7C/c5/ZRfKgL8fp+JqP5sssMU5jpK1WQVgU3qWhSJFOkS0lLsG6/soGyFgaimkWTQ6ZX5KFk\nmq5l8dMUoeLhsVI/4vxBIJjExXHoYNXuXXqUobh94VWY19fbnbNk0K8bwtA5m24try63cozrqrhH\ny7SP7+v11h7CDWC5aI03EukaQ3m4D4RYXPaCa3QXkXkWkIR2INUPKonEeUFA/SEgM7Q/VrNeb07g\njoDEbfrwcBJwu90P83q/ZzuvRoMzbDHHcFjgjISncIHAIj6/zYFRmLgFM0xOM90wGROMERuDskNA\nlB2cYlG1vccEhCIroihwG+oW1AWDZr0mjHtAVIeZMjm19Mtq6GSBPgXJ7deSHrcrXh9VTsInoW/v\n7n7AWGoJOjfRNzpYnhpXWzRVuuVwj8Fr5Lb3DCwH0QXbI4aUQAw/tKQDLaqyEBmXpi4K/hvGcAz0\n1NdQX1J+izLu15AqY1DSU2LVHXr22ekPB+1vZ59wI7m815M8j7uXW996g2Evg5Y4EKhdrJWE8sjO\nxpRf79emucAmluWU8RqXE94nNDqr3tJRlwt+W/WOhtecX9e9NZu3uEsH3KEdF6S62DnWaOo1AdGh\nXgnqVKNg4H3c8wjk69zc3Nu3b+ZG3c+Oc3SVJG+9efP2LR5u/vFDxkqxOn5+lMhTwPujK/nZ2dmZ\nvDslX61U5vMS4szsDPY+S0+vvL7lAoNeZ4kZeHQaesNIYvz8uCg7AzUzWpwNNJRWkVZceur/vSUf\nGAtDCZrI0KAvoeG/Lt9Xx5MfIhFiicj9QSmhGd649zg098nPik+rB+/T/k/yO9A7bEvKcQkCAAAA\nAElFTkSuQmCC\";\n"
  },
  {
    "path": "crates/common/src/listener/acme/cache.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};\nuse trc::AddContext;\nuse utils::config::ConfigKey;\n\nuse crate::Server;\n\nuse super::AcmeProvider;\n\nimpl Server {\n    pub(crate) async fn load_cert(&self, provider: &AcmeProvider) -> trc::Result<Option<Vec<u8>>> {\n        self.read_if_exists(provider, \"cert\", provider.domains.as_slice())\n            .await\n            .add_context(|err| {\n                err.caused_by(trc::location!())\n                    .details(\"Failed to load certificates\")\n            })\n    }\n\n    pub(crate) async fn store_cert(&self, provider: &AcmeProvider, cert: &[u8]) -> trc::Result<()> {\n        self.write(provider, \"cert\", provider.domains.as_slice(), cert)\n            .await\n            .add_context(|err| {\n                err.caused_by(trc::location!())\n                    .details(\"Failed to store certificate\")\n            })\n    }\n\n    pub(crate) async fn load_account(\n        &self,\n        provider: &AcmeProvider,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        self.read_if_exists(provider, \"account-key\", provider.contact.as_slice())\n            .await\n            .add_context(|err| {\n                err.caused_by(trc::location!())\n                    .details(\"Failed to load account\")\n            })\n    }\n\n    pub(crate) async fn store_account(\n        &self,\n        provider: &AcmeProvider,\n        account: &[u8],\n    ) -> trc::Result<()> {\n        self.write(\n            provider,\n            \"account-key\",\n            provider.contact.as_slice(),\n            account,\n        )\n        .await\n        .add_context(|err| {\n            err.caused_by(trc::location!())\n                .details(\"Failed to store account\")\n        })\n    }\n\n    async fn read_if_exists(\n        &self,\n        provider: &AcmeProvider,\n        class: &str,\n        items: &[String],\n    ) -> trc::Result<Option<Vec<u8>>> {\n        if let Some(content) = self\n            .core\n            .storage\n            .config\n            .get(self.build_key(provider, class, items))\n            .await?\n        {\n            URL_SAFE_NO_PAD\n                .decode(content.as_bytes())\n                .map_err(|err| {\n                    trc::EventType::Acme(trc::AcmeEvent::Error)\n                        .caused_by(trc::location!())\n                        .reason(err)\n                        .details(\"failed to decode certificate\")\n                })\n                .map(Some)\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn write(\n        &self,\n        provider: &AcmeProvider,\n        class: &str,\n        items: &[String],\n        contents: impl AsRef<[u8]>,\n    ) -> trc::Result<()> {\n        self.core\n            .storage\n            .config\n            .set(\n                [ConfigKey {\n                    key: self.build_key(provider, class, items),\n                    value: URL_SAFE_NO_PAD.encode(contents.as_ref()),\n                }],\n                true,\n            )\n            .await\n    }\n\n    fn build_key(&self, provider: &AcmeProvider, class: &str, _: &[String]) -> String {\n        /*let mut ctx = Context::new(&SHA512);\n        for el in items {\n            ctx.update(el.as_ref());\n            ctx.update(&[0])\n        }\n        ctx.update(provider.directory_url.as_bytes());\n\n        format!(\n            \"certificate.acme-{}-{}.{}\",\n            provider.id,\n            URL_SAFE_NO_PAD.encode(ctx.finish()),\n            class\n        )*/\n\n        format!(\"acme.{}.{}\", provider.id, class)\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/acme/directory.rs",
    "content": "// Adapted from rustls-acme (https://github.com/FlorianUekermann/rustls-acme), licensed under MIT/Apache-2.0.\n\nuse super::AcmeProvider;\nuse super::jose::{\n    Body, eab_sign, key_authorization, key_authorization_sha256, key_authorization_sha256_base64,\n    sign,\n};\nuse base64::Engine;\nuse base64::engine::general_purpose::URL_SAFE_NO_PAD;\nuse hyper::header::USER_AGENT;\nuse rcgen::{Certificate, CustomExtension, PKCS_ECDSA_P256_SHA256};\nuse reqwest::header::CONTENT_TYPE;\nuse reqwest::{Method, Response};\nuse ring::rand::SystemRandom;\nuse ring::signature::{ECDSA_P256_SHA256_FIXED_SIGNING, EcdsaKeyPair, EcdsaSigningAlgorithm};\nuse serde::Deserialize;\nuse std::time::Duration;\nuse store::Serialize;\nuse store::write::Archiver;\nuse trc::AddContext;\nuse trc::event::conv::AssertSuccess;\n\npub const LETS_ENCRYPT_STAGING_DIRECTORY: &str =\n    \"https://acme-staging-v02.api.letsencrypt.org/directory\";\npub const LETS_ENCRYPT_PRODUCTION_DIRECTORY: &str =\n    \"https://acme-v02.api.letsencrypt.org/directory\";\npub const ACME_TLS_ALPN_NAME: &[u8] = b\"acme-tls/1\";\n\n#[derive(Debug)]\npub struct Account {\n    pub key_pair: EcdsaKeyPair,\n    pub directory: Directory,\n    pub kid: String,\n}\n\n#[derive(Debug, serde::Serialize)]\npub struct NewAccountPayload<'x> {\n    #[serde(rename = \"termsOfServiceAgreed\")]\n    tos_agreed: bool,\n    contact: &'x [String],\n    #[serde(rename = \"externalAccountBinding\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    eab: Option<Body>,\n}\n\nstatic ALG: &EcdsaSigningAlgorithm = &ECDSA_P256_SHA256_FIXED_SIGNING;\n\nimpl Account {\n    pub fn generate_key_pair() -> Vec<u8> {\n        EcdsaKeyPair::generate_pkcs8(ALG, &SystemRandom::new())\n            .unwrap()\n            .as_ref()\n            .to_vec()\n    }\n\n    pub async fn create(directory: Directory, provider: &AcmeProvider) -> trc::Result<Self> {\n        Self::create_with_keypair(directory, provider).await\n    }\n\n    pub async fn create_with_keypair(\n        directory: Directory,\n        provider: &AcmeProvider,\n    ) -> trc::Result<Self> {\n        let key_pair = EcdsaKeyPair::from_pkcs8(\n            ALG,\n            provider.account_key.load().as_slice(),\n            &SystemRandom::new(),\n        )\n        .map_err(|err| {\n            trc::EventType::Acme(trc::AcmeEvent::Error)\n                .reason(err)\n                .caused_by(trc::location!())\n        })?;\n        let eab = if let Some(eab) = &provider.eab {\n            eab_sign(&key_pair, &eab.kid, &eab.hmac_key, &directory.new_account)\n                .caused_by(trc::location!())?\n                .into()\n        } else {\n            None\n        };\n\n        let payload = serde_json::to_string(&NewAccountPayload {\n            tos_agreed: true,\n            contact: &provider.contact,\n            eab,\n        })\n        .unwrap_or_default();\n\n        let body = sign(\n            &key_pair,\n            None,\n            directory.nonce().await?,\n            &directory.new_account,\n            &payload,\n        )?;\n        let response = https(&directory.new_account, Method::POST, Some(body)).await?;\n        let kid = get_header(&response, \"Location\")?;\n        Ok(Account {\n            key_pair,\n            kid,\n            directory,\n        })\n    }\n\n    async fn request(\n        &self,\n        url: impl AsRef<str>,\n        payload: &str,\n    ) -> trc::Result<(Option<String>, String)> {\n        let body = sign(\n            &self.key_pair,\n            Some(&self.kid),\n            self.directory.nonce().await?,\n            url.as_ref(),\n            payload,\n        )?;\n        let response = https(url.as_ref(), Method::POST, Some(body)).await?;\n        let location = get_header(&response, \"Location\").ok();\n        let body = response\n            .text()\n            .await\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_http_error(err))?;\n        Ok((location, body))\n    }\n\n    pub async fn new_order(&self, domains: Vec<String>) -> trc::Result<(String, Order)> {\n        let domains: Vec<Identifier> = domains.into_iter().map(Identifier::Dns).collect();\n        let payload = format!(\n            \"{{\\\"identifiers\\\":{}}}\",\n            serde_json::to_string(&domains)\n                .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))?\n        );\n        let response = self.request(&self.directory.new_order, &payload).await?;\n        let url = response.0.ok_or(\n            trc::EventType::Acme(trc::AcmeEvent::Error)\n                .caused_by(trc::location!())\n                .details(\"Missing header\")\n                .ctx(trc::Key::Id, \"Location\"),\n        )?;\n        let order = serde_json::from_str(&response.1)\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))?;\n        Ok((url, order))\n    }\n\n    pub async fn auth(&self, url: impl AsRef<str>) -> trc::Result<Auth> {\n        let response = self.request(url, \"\").await?;\n        serde_json::from_str(&response.1)\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))\n    }\n\n    pub async fn challenge(&self, url: impl AsRef<str>) -> trc::Result<()> {\n        self.request(&url, \"{}\").await.map(|_| ())\n    }\n\n    pub async fn order(&self, url: impl AsRef<str>) -> trc::Result<Order> {\n        let response = self.request(&url, \"\").await?;\n        serde_json::from_str(&response.1)\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))\n    }\n\n    pub async fn finalize(&self, url: impl AsRef<str>, csr: Vec<u8>) -> trc::Result<Order> {\n        let payload = format!(\"{{\\\"csr\\\":\\\"{}\\\"}}\", URL_SAFE_NO_PAD.encode(csr));\n        let response = self.request(&url, &payload).await?;\n        serde_json::from_str(&response.1)\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))\n    }\n\n    pub async fn certificate(&self, url: impl AsRef<str>) -> trc::Result<String> {\n        Ok(self.request(&url, \"\").await?.1)\n    }\n\n    pub fn http_proof(&self, challenge: &Challenge) -> trc::Result<Vec<u8>> {\n        key_authorization(&self.key_pair, &challenge.token).map(|key| key.into_bytes())\n    }\n\n    pub fn dns_proof(&self, challenge: &Challenge) -> trc::Result<String> {\n        key_authorization_sha256_base64(&self.key_pair, &challenge.token)\n    }\n\n    pub fn tls_alpn_key(&self, challenge: &Challenge, domain: String) -> trc::Result<Vec<u8>> {\n        let mut params = rcgen::CertificateParams::new(vec![domain]);\n        let key_auth = key_authorization_sha256(&self.key_pair, &challenge.token)?;\n        params.alg = &PKCS_ECDSA_P256_SHA256;\n        params.custom_extensions = vec![CustomExtension::new_acme_identifier(key_auth.as_ref())];\n        let cert = Certificate::from_params(params).map_err(|err| {\n            trc::EventType::Acme(trc::AcmeEvent::Error)\n                .caused_by(trc::location!())\n                .reason(err)\n        })?;\n\n        Archiver::new(SerializedCert {\n            certificate: cert.serialize_der().map_err(|err| {\n                trc::EventType::Acme(trc::AcmeEvent::Error)\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })?,\n            private_key: cert.serialize_private_key_der(),\n        })\n        .untrusted()\n        .serialize()\n    }\n}\n\n#[derive(\n    rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug, Clone, serde::Serialize, Deserialize,\n)]\npub struct SerializedCert {\n    pub certificate: Vec<u8>,\n    pub private_key: Vec<u8>,\n}\n\n#[derive(Debug, Clone, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Directory {\n    pub new_nonce: String,\n    pub new_account: String,\n    pub new_order: String,\n}\n\nimpl Directory {\n    pub async fn discover(url: impl AsRef<str>) -> trc::Result<Self> {\n        serde_json::from_str(\n            &https(url, Method::GET, None)\n                .await?\n                .text()\n                .await\n                .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_http_error(err))?,\n        )\n        .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))\n    }\n    pub async fn nonce(&self) -> trc::Result<String> {\n        get_header(\n            &https(&self.new_nonce.as_str(), Method::HEAD, None).await?,\n            \"replay-nonce\",\n        )\n    }\n}\n\n#[derive(Debug, Deserialize, Eq, PartialEq, Clone, Copy)]\npub enum ChallengeType {\n    #[serde(rename = \"http-01\")]\n    Http01,\n    #[serde(rename = \"dns-01\")]\n    Dns01,\n    #[serde(rename = \"tls-alpn-01\")]\n    TlsAlpn01,\n    #[serde(other)]\n    Unknown,\n}\n\n#[derive(Debug, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Order {\n    #[serde(flatten)]\n    pub status: OrderStatus,\n    pub authorizations: Vec<String>,\n    pub finalize: String,\n    pub error: Option<Problem>,\n}\n\n#[derive(Debug, Deserialize, Clone, PartialEq, Eq)]\n#[serde(tag = \"status\", rename_all = \"camelCase\")]\npub enum OrderStatus {\n    Pending,\n    Ready,\n    Valid { certificate: String },\n    Invalid,\n    Processing,\n}\n\n#[derive(Debug, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Auth {\n    pub status: AuthStatus,\n    pub identifier: Identifier,\n    pub challenges: Vec<Challenge>,\n    pub wildcard: Option<bool>,\n}\n\n#[derive(Debug, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub enum AuthStatus {\n    Pending,\n    Valid,\n    Invalid,\n    Revoked,\n    Expired,\n    Deactivated,\n}\n\n#[derive(Clone, Debug, serde::Serialize, Deserialize)]\n#[serde(tag = \"type\", content = \"value\", rename_all = \"camelCase\")]\npub enum Identifier {\n    Dns(String),\n}\n\n#[derive(Debug, Deserialize)]\npub struct Challenge {\n    #[serde(rename = \"type\")]\n    pub typ: ChallengeType,\n    pub url: String,\n    pub token: String,\n    pub error: Option<Problem>,\n}\n\n#[derive(Clone, Debug, serde::Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Problem {\n    #[serde(rename = \"type\")]\n    pub typ: Option<String>,\n    pub detail: Option<String>,\n}\n\n#[allow(unused_mut)]\nasync fn https(\n    url: impl AsRef<str>,\n    method: Method,\n    body: Option<String>,\n) -> trc::Result<Response> {\n    let url = url.as_ref();\n    let mut builder = reqwest::Client::builder()\n        .timeout(Duration::from_secs(30))\n        .http1_only();\n\n    #[cfg(debug_assertions)]\n    {\n        builder = builder.danger_accept_invalid_certs(\n            url.starts_with(\"https://localhost\") || url.starts_with(\"https://127.0.0.1\"),\n        );\n    }\n\n    let mut request = builder\n        .build()\n        .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_http_error(err))?\n        .request(method, url)\n        .header(USER_AGENT, crate::USER_AGENT);\n\n    if let Some(body) = body {\n        request = request\n            .header(CONTENT_TYPE, \"application/jose+json\")\n            .body(body);\n    }\n\n    request\n        .send()\n        .await\n        .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_http_error(err))?\n        .assert_success(trc::EventType::Acme(trc::AcmeEvent::Error))\n        .await\n}\n\nfn get_header(response: &Response, header: &'static str) -> trc::Result<String> {\n    match response.headers().get_all(header).iter().next_back() {\n        Some(value) => Ok(value\n            .to_str()\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_http_str_error(err))?\n            .to_string()),\n        None => Err(trc::EventType::Acme(trc::AcmeEvent::Error)\n            .caused_by(trc::location!())\n            .details(\"Missing header\")\n            .ctx(trc::Key::Id, header)),\n    }\n}\n\nimpl ChallengeType {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Self::Http01 => \"http-01\",\n            Self::Dns01 => \"dns-01\",\n            Self::TlsAlpn01 => \"tls-alpn-01\",\n            Self::Unknown => \"unknown\",\n        }\n    }\n}\n\nimpl AuthStatus {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Self::Pending => \"pending\",\n            Self::Valid => \"valid\",\n            Self::Invalid => \"invalid\",\n            Self::Revoked => \"revoked\",\n            Self::Expired => \"expired\",\n            Self::Deactivated => \"deactivated\",\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/acme/jose.rs",
    "content": "// Adapted from rustls-acme (https://github.com/FlorianUekermann/rustls-acme), licensed under MIT/Apache-2.0.\n\nuse base64::Engine;\nuse base64::engine::general_purpose::URL_SAFE_NO_PAD;\nuse ring::digest::{Digest, SHA256, digest};\nuse ring::hmac;\nuse ring::rand::SystemRandom;\nuse ring::signature::{EcdsaKeyPair, KeyPair};\nuse serde::Serialize;\n\npub(crate) fn sign(\n    key: &EcdsaKeyPair,\n    kid: Option<&str>,\n    nonce: String,\n    url: &str,\n    payload: &str,\n) -> trc::Result<String> {\n    let jwk = match kid {\n        None => Some(Jwk::new(key)),\n        Some(_) => None,\n    };\n    let protected = Protected::encode(\"ES256\", jwk, kid, nonce.into(), url)?;\n    let payload = URL_SAFE_NO_PAD.encode(payload);\n    let combined = format!(\"{}.{}\", &protected, &payload);\n    let signature = key\n        .sign(&SystemRandom::new(), combined.as_bytes())\n        .map_err(|err| {\n            trc::EventType::Acme(trc::AcmeEvent::Error)\n                .caused_by(trc::location!())\n                .reason(err)\n        })?;\n\n    serde_json::to_string(&Body {\n        protected,\n        payload,\n        signature: URL_SAFE_NO_PAD.encode(signature.as_ref()),\n    })\n    .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))\n}\n\npub(crate) fn eab_sign(\n    key: &EcdsaKeyPair,\n    kid: &str,\n    hmac_key: &[u8],\n    url: &str,\n) -> trc::Result<Body> {\n    let protected = Protected::encode(\"HS256\", None, kid.into(), None, url)?;\n    let payload = Jwk::new(key).base64()?;\n    let combined = format!(\"{}.{}\", &protected, &payload);\n\n    let key = hmac::Key::new(hmac::HMAC_SHA256, hmac_key);\n    let tag = hmac::sign(&key, combined.as_bytes());\n    let signature = URL_SAFE_NO_PAD.encode(tag.as_ref());\n\n    Ok(Body {\n        protected,\n        payload,\n        signature,\n    })\n}\n\npub(crate) fn key_authorization(key: &EcdsaKeyPair, token: &str) -> trc::Result<String> {\n    Ok(format!(\n        \"{}.{}\",\n        token,\n        Jwk::new(key).thumb_sha256_base64()?\n    ))\n}\n\npub(crate) fn key_authorization_sha256(key: &EcdsaKeyPair, token: &str) -> trc::Result<Digest> {\n    key_authorization(key, token).map(|s| digest(&SHA256, s.as_bytes()))\n}\n\npub(crate) fn key_authorization_sha256_base64(\n    key: &EcdsaKeyPair,\n    token: &str,\n) -> trc::Result<String> {\n    key_authorization_sha256(key, token).map(|s| URL_SAFE_NO_PAD.encode(s.as_ref()))\n}\n\n#[derive(Debug, Serialize)]\npub(crate) struct Body {\n    protected: String,\n    payload: String,\n    signature: String,\n}\n\n#[derive(Serialize)]\nstruct Protected<'a> {\n    alg: &'static str,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    jwk: Option<Jwk>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    kid: Option<&'a str>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    nonce: Option<String>,\n    url: &'a str,\n}\n\nimpl<'a> Protected<'a> {\n    fn encode(\n        alg: &'static str,\n        jwk: Option<Jwk>,\n        kid: Option<&'a str>,\n        nonce: Option<String>,\n        url: &'a str,\n    ) -> trc::Result<String> {\n        serde_json::to_vec(&Protected {\n            alg,\n            jwk,\n            kid,\n            nonce,\n            url,\n        })\n        .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))\n        .map(|v| URL_SAFE_NO_PAD.encode(v.as_slice()))\n    }\n}\n\n#[derive(Serialize)]\nstruct Jwk {\n    alg: &'static str,\n    crv: &'static str,\n    kty: &'static str,\n    #[serde(rename = \"use\")]\n    u: &'static str,\n    x: String,\n    y: String,\n}\n\nimpl Jwk {\n    pub(crate) fn new(key: &EcdsaKeyPair) -> Self {\n        let (x, y) = key.public_key().as_ref()[1..].split_at(32);\n        Self {\n            alg: \"ES256\",\n            crv: \"P-256\",\n            kty: \"EC\",\n            u: \"sig\",\n            x: URL_SAFE_NO_PAD.encode(x),\n            y: URL_SAFE_NO_PAD.encode(y),\n        }\n    }\n\n    pub(crate) fn base64(&self) -> trc::Result<String> {\n        serde_json::to_vec(self)\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))\n            .map(|v| URL_SAFE_NO_PAD.encode(v.as_slice()))\n    }\n\n    pub(crate) fn thumb_sha256_base64(&self) -> trc::Result<String> {\n        Ok(URL_SAFE_NO_PAD.encode(digest(\n            &SHA256,\n            &serde_json::to_vec(&JwkThumb {\n                crv: self.crv,\n                kty: self.kty,\n                x: &self.x,\n                y: &self.y,\n            })\n            .map_err(|err| trc::EventType::Acme(trc::AcmeEvent::Error).from_json_error(err))?,\n        )))\n    }\n}\n\n#[derive(Serialize)]\nstruct JwkThumb<'a> {\n    crv: &'a str,\n    kty: &'a str,\n    x: &'a str,\n    y: &'a str,\n}\n"
  },
  {
    "path": "crates/common/src/listener/acme/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod cache;\npub mod directory;\npub mod jose;\npub mod order;\npub mod resolver;\n\nuse std::{fmt::Debug, sync::Arc, time::Duration};\n\nuse arc_swap::ArcSwap;\nuse dns_update::DnsUpdater;\nuse rustls::sign::CertifiedKey;\n\nuse crate::Server;\n\nuse self::directory::{Account, ChallengeType};\n\npub struct AcmeProvider {\n    pub id: String,\n    pub directory_url: String,\n    pub domains: Vec<String>,\n    pub contact: Vec<String>,\n    pub challenge: ChallengeSettings,\n    pub eab: Option<EabSettings>,\n    renew_before: chrono::Duration,\n    account_key: ArcSwap<Vec<u8>>,\n    default: bool,\n}\n\n#[derive(Clone)]\npub struct EabSettings {\n    pub kid: String,\n    pub hmac_key: Vec<u8>,\n}\n\n#[derive(Clone)]\npub enum ChallengeSettings {\n    Http01,\n    TlsAlpn01,\n    Dns01 {\n        updater: DnsUpdater,\n        origin: Option<String>,\n        polling_interval: Duration,\n        propagation_timeout: Duration,\n        ttl: u32,\n    },\n}\n\npub struct StaticResolver {\n    pub key: Option<Arc<CertifiedKey>>,\n}\n\nimpl AcmeProvider {\n    #[allow(clippy::too_many_arguments)]\n    pub fn new(\n        id: String,\n        directory_url: String,\n        domains: Vec<String>,\n        contact: Vec<String>,\n        challenge: ChallengeSettings,\n        eab: Option<EabSettings>,\n        renew_before: Duration,\n        default: bool,\n    ) -> trc::Result<Self> {\n        Ok(AcmeProvider {\n            id,\n            directory_url,\n            contact: contact\n                .into_iter()\n                .map(|c| {\n                    if !c.starts_with(\"mailto:\") {\n                        format!(\"mailto:{}\", c)\n                    } else {\n                        c\n                    }\n                })\n                .collect(),\n            renew_before: chrono::Duration::from_std(renew_before).unwrap(),\n            domains,\n            account_key: Default::default(),\n            challenge,\n            eab,\n            default,\n        })\n    }\n}\n\nimpl Server {\n    pub async fn init_acme(&self, provider: &AcmeProvider) -> trc::Result<Duration> {\n        // Load account key from cache or generate a new one\n        if let Some(account_key) = self.load_account(provider).await? {\n            provider.account_key.store(Arc::new(account_key));\n        } else {\n            let account_key = Account::generate_key_pair();\n            self.store_account(provider, &account_key).await?;\n            provider.account_key.store(Arc::new(account_key));\n        }\n\n        // Load certificate from cache or request a new one\n        Ok(if let Some(pem) = self.load_cert(provider).await? {\n            self.process_cert(provider, pem, true).await?\n        } else {\n            Duration::from_millis(1000)\n        })\n    }\n\n    pub fn has_acme_tls_providers(&self) -> bool {\n        self.core\n            .acme\n            .providers\n            .values()\n            .any(|p| matches!(p.challenge, ChallengeSettings::TlsAlpn01))\n    }\n\n    pub fn has_acme_http_providers(&self) -> bool {\n        self.core\n            .acme\n            .providers\n            .values()\n            .any(|p| matches!(p.challenge, ChallengeSettings::Http01))\n    }\n}\n\nimpl ChallengeSettings {\n    pub fn challenge_type(&self) -> ChallengeType {\n        match self {\n            ChallengeSettings::Http01 => ChallengeType::Http01,\n            ChallengeSettings::TlsAlpn01 => ChallengeType::TlsAlpn01,\n            ChallengeSettings::Dns01 { .. } => ChallengeType::Dns01,\n        }\n    }\n}\n\nimpl Debug for StaticResolver {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.debug_struct(\"StaticResolver\").finish()\n    }\n}\n\nimpl Clone for AcmeProvider {\n    fn clone(&self) -> Self {\n        Self {\n            id: self.id.clone(),\n            directory_url: self.directory_url.clone(),\n            domains: self.domains.clone(),\n            contact: self.contact.clone(),\n            challenge: self.challenge.clone(),\n            renew_before: self.renew_before,\n            account_key: ArcSwap::from_pointee(self.account_key.load().as_ref().clone()),\n            eab: self.eab.clone(),\n            default: self.default,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/acme/order.rs",
    "content": "// Adapted from rustls-acme (https://github.com/FlorianUekermann/rustls-acme), licensed under MIT/Apache-2.0.\n\nuse chrono::{DateTime, TimeZone, Utc};\n\nuse compact_str::CompactString;\nuse dns_update::{DnsRecord, DnsRecordType};\nuse futures::future::try_join_all;\nuse rcgen::{CertificateParams, DistinguishedName, PKCS_ECDSA_P256_SHA256};\nuse rustls::crypto::ring::sign::any_ecdsa_type;\nuse rustls::sign::CertifiedKey;\nuse rustls_pki_types::{CertificateDer, PrivateKeyDer, PrivatePkcs8KeyDer};\nuse std::sync::Arc;\nuse std::time::{Duration, Instant};\nuse store::dispatch::lookup::KeyValue;\nuse trc::{AcmeEvent, EventType};\nuse x509_parser::parse_x509_certificate;\n\nuse crate::listener::acme::ChallengeSettings;\nuse crate::listener::acme::directory::Identifier;\nuse crate::{KV_ACME, Server};\n\nuse super::AcmeProvider;\nuse super::directory::{Account, AuthStatus, Directory, OrderStatus};\n\nimpl Server {\n    pub(crate) async fn process_cert(\n        &self,\n        provider: &AcmeProvider,\n        pem: Vec<u8>,\n        cached: bool,\n    ) -> trc::Result<Duration> {\n        let (cert, validity) = parse_cert(&pem)?;\n\n        self.set_cert(provider, Arc::new(cert));\n\n        let renew_at = (validity[1] - provider.renew_before - Utc::now())\n            .max(chrono::Duration::zero())\n            .to_std()\n            .unwrap_or_default();\n        let renewal_date = validity[1] - provider.renew_before;\n\n        trc::event!(\n            Acme(AcmeEvent::ProcessCert),\n            Id = provider.id.to_string(),\n            Hostname = provider.domains.as_slice(),\n            ValidFrom = trc::Value::Timestamp(validity[0].timestamp() as u64),\n            ValidTo = trc::Value::Timestamp(validity[1].timestamp() as u64),\n            Due = trc::Value::Timestamp(renewal_date.timestamp() as u64),\n        );\n\n        if !cached {\n            self.store_cert(provider, &pem).await?;\n        }\n\n        Ok(renew_at)\n    }\n\n    pub async fn renew(&self, provider: &AcmeProvider) -> trc::Result<Duration> {\n        let mut backoff = 0;\n        loop {\n            match self.order(provider).await {\n                Ok(pem) => return self.process_cert(provider, pem, false).await,\n                Err(err)\n                    if !err.matches(EventType::Acme(AcmeEvent::OrderInvalid)) && backoff < 9 =>\n                {\n                    trc::event!(\n                        Acme(AcmeEvent::RenewBackoff),\n                        Id = provider.id.to_string(),\n                        Hostname = provider.domains.as_slice(),\n                        Total = backoff,\n                        NextRetry = 1 << backoff,\n                        CausedBy = err,\n                    );\n                    backoff += 1;\n                    tokio::time::sleep(Duration::from_secs(1 << backoff)).await;\n                }\n                Err(err) => {\n                    return Err(err\n                        .details(\"Failed to renew certificate\")\n                        .ctx_unique(trc::Key::Id, provider.id.to_string())\n                        .ctx_unique(trc::Key::Hostname, provider.domains.as_slice()));\n                }\n            }\n        }\n    }\n\n    async fn order(&self, provider: &AcmeProvider) -> trc::Result<Vec<u8>> {\n        let directory = Directory::discover(&provider.directory_url).await?;\n        let account = Account::create_with_keypair(directory, provider).await?;\n\n        let mut params = CertificateParams::new(provider.domains.clone());\n        params.distinguished_name = DistinguishedName::new();\n        params.alg = &PKCS_ECDSA_P256_SHA256;\n        let cert = rcgen::Certificate::from_params(params).map_err(|err| {\n            EventType::Acme(AcmeEvent::Error)\n                .caused_by(trc::location!())\n                .reason(err)\n        })?;\n\n        let (order_url, mut order) = account.new_order(provider.domains.clone()).await?;\n        loop {\n            match order.status {\n                OrderStatus::Pending => {\n                    let auth_futures = order\n                        .authorizations\n                        .iter()\n                        .map(|url| self.authorize(provider, &account, url));\n                    try_join_all(auth_futures).await?;\n                    trc::event!(\n                        Acme(AcmeEvent::AuthCompleted),\n                        Id = provider.id.to_string(),\n                        Hostname = provider.domains.as_slice(),\n                    );\n                    order = account.order(&order_url).await?;\n                }\n                OrderStatus::Processing => {\n                    for i in 0u64..10 {\n                        trc::event!(\n                            Acme(AcmeEvent::OrderProcessing),\n                            Id = provider.id.to_string(),\n                            Hostname = provider.domains.as_slice(),\n                            Total = i,\n                        );\n\n                        tokio::time::sleep(Duration::from_secs(1u64 << i)).await;\n                        order = account.order(&order_url).await?;\n                        if order.status != OrderStatus::Processing {\n                            break;\n                        }\n                    }\n                    if order.status == OrderStatus::Processing {\n                        return Err(EventType::Acme(AcmeEvent::Error)\n                            .caused_by(trc::location!())\n                            .details(\"Order processing timed out\"));\n                    }\n                }\n                OrderStatus::Ready => {\n                    trc::event!(\n                        Acme(AcmeEvent::OrderReady),\n                        Id = provider.id.to_string(),\n                        Hostname = provider.domains.as_slice(),\n                    );\n\n                    let csr = cert.serialize_request_der().map_err(|err| {\n                        EventType::Acme(AcmeEvent::Error)\n                            .caused_by(trc::location!())\n                            .reason(err)\n                    })?;\n                    order = account.finalize(order.finalize, csr).await?\n                }\n                OrderStatus::Valid { certificate } => {\n                    trc::event!(\n                        Acme(AcmeEvent::OrderValid),\n                        Id = provider.id.to_string(),\n                        Hostname = provider.domains.as_slice(),\n                    );\n\n                    let pem = [\n                        &cert.serialize_private_key_pem(),\n                        \"\\n\",\n                        &account.certificate(certificate).await?,\n                    ]\n                    .concat();\n                    return Ok(pem.into_bytes());\n                }\n                OrderStatus::Invalid => {\n                    return Err(EventType::Acme(AcmeEvent::OrderInvalid).into_err());\n                }\n            }\n        }\n    }\n\n    async fn authorize(\n        &self,\n        provider: &AcmeProvider,\n        account: &Account,\n        url: &String,\n    ) -> trc::Result<()> {\n        let auth = account.auth(url).await?;\n        let (domain, challenge_url) = match auth.status {\n            AuthStatus::Pending => {\n                let Identifier::Dns(domain) = auth.identifier;\n                let challenge_type = provider.challenge.challenge_type();\n\n                trc::event!(\n                    Acme(AcmeEvent::AuthStart),\n                    Hostname = domain.to_string(),\n                    Type = challenge_type.as_str(),\n                    Id = provider.id.to_string(),\n                );\n\n                let challenge = auth\n                    .challenges\n                    .iter()\n                    .find(|c| c.typ == challenge_type)\n                    .ok_or(\n                        EventType::Acme(AcmeEvent::OrderInvalid)\n                            .into_err()\n                            .details(\"Challenge not supported by ACME provider\")\n                            .ctx(trc::Key::Id, provider.id.to_string())\n                            .ctx(trc::Key::Type, challenge_type.as_str())\n                            .ctx(\n                                trc::Key::Contents,\n                                auth.challenges\n                                    .iter()\n                                    .map(|c| {\n                                        trc::Value::String(CompactString::const_new(c.typ.as_str()))\n                                    })\n                                    .collect::<Vec<_>>(),\n                            ),\n                    )?;\n\n                match &provider.challenge {\n                    ChallengeSettings::TlsAlpn01 => {\n                        self.in_memory_store()\n                            .key_set(\n                                KeyValue::with_prefix(\n                                    KV_ACME,\n                                    &domain,\n                                    account.tls_alpn_key(challenge, domain.clone())?,\n                                )\n                                .expires(3600),\n                            )\n                            .await?;\n                    }\n                    ChallengeSettings::Http01 => {\n                        self.in_memory_store()\n                            .key_set(\n                                KeyValue::with_prefix(\n                                    KV_ACME,\n                                    &challenge.token,\n                                    account.http_proof(challenge)?,\n                                )\n                                .expires(3600),\n                            )\n                            .await?;\n                    }\n                    ChallengeSettings::Dns01 {\n                        updater,\n                        origin,\n                        polling_interval,\n                        propagation_timeout,\n                        ttl,\n                    } => {\n                        let dns_proof = account.dns_proof(challenge)?;\n                        let domain = domain.strip_prefix(\"*.\").unwrap_or(&domain);\n                        let name = format!(\"_acme-challenge.{}\", domain);\n                        let origin = origin\n                            .as_deref()\n                            .or_else(|| psl::domain_str(domain))\n                            .unwrap_or(domain)\n                            .to_string();\n\n                        // First try deleting the record\n                        if let Err(err) = updater.delete(&name, &origin, DnsRecordType::TXT).await {\n                            // Errors are expected if the record does not exist\n                            trc::event!(\n                                Acme(AcmeEvent::DnsRecordDeletionFailed),\n                                Hostname = name.to_string(),\n                                Reason = err.to_string(),\n                                Details = origin.to_string(),\n                                Id = provider.id.to_string(),\n                            );\n                        }\n\n                        // Create the record\n                        if let Err(err) = updater\n                            .create(\n                                &name,\n                                DnsRecord::TXT {\n                                    content: dns_proof.clone(),\n                                },\n                                *ttl,\n                                &origin,\n                            )\n                            .await\n                        {\n                            return Err(EventType::Acme(AcmeEvent::DnsRecordCreationFailed)\n                                .ctx(trc::Key::Id, provider.id.to_string())\n                                .ctx(trc::Key::Hostname, name)\n                                .ctx(trc::Key::Details, origin)\n                                .reason(err));\n                        }\n\n                        trc::event!(\n                            Acme(AcmeEvent::DnsRecordCreated),\n                            Hostname = name.to_string(),\n                            Details = origin.to_string(),\n                            Id = provider.id.to_string(),\n                        );\n\n                        // Wait for changes to propagate\n                        let wait_until = Instant::now() + *propagation_timeout;\n                        let mut did_propagate = false;\n                        while Instant::now() < wait_until {\n                            match self.core.smtp.resolvers.dns.txt_raw_lookup(&name).await {\n                                Ok(result) => {\n                                    let result = std::str::from_utf8(&result).unwrap_or_default();\n                                    if result.contains(&dns_proof) {\n                                        did_propagate = true;\n                                        break;\n                                    } else {\n                                        trc::event!(\n                                            Acme(AcmeEvent::DnsRecordNotPropagated),\n                                            Id = provider.id.to_string(),\n                                            Hostname = name.to_string(),\n                                            Details = origin.to_string(),\n                                            Result = result.to_string(),\n                                            Value = dns_proof.to_string(),\n                                        );\n                                    }\n                                }\n                                Err(err) => {\n                                    trc::event!(\n                                        Acme(AcmeEvent::DnsRecordLookupFailed),\n                                        Id = provider.id.to_string(),\n                                        Hostname = name.to_string(),\n                                        Details = origin.to_string(),\n                                        Reason = err.to_string(),\n                                    );\n                                }\n                            }\n\n                            tokio::time::sleep(*polling_interval).await;\n                        }\n\n                        if did_propagate {\n                            trc::event!(\n                                Acme(AcmeEvent::DnsRecordPropagated),\n                                Id = provider.id.to_string(),\n                                Hostname = name.to_string(),\n                                Details = origin.to_string(),\n                            );\n                        } else {\n                            trc::event!(\n                                Acme(AcmeEvent::DnsRecordPropagationTimeout),\n                                Id = provider.id.to_string(),\n                                Hostname = name.to_string(),\n                                Details = origin.to_string(),\n                            );\n                        }\n                    }\n                }\n\n                account.challenge(&challenge.url).await?;\n                (domain, challenge.url.clone())\n            }\n            AuthStatus::Valid => return Ok(()),\n            _ => {\n                return Err(EventType::Acme(AcmeEvent::AuthError)\n                    .into_err()\n                    .ctx(trc::Key::Id, provider.id.to_string())\n                    .ctx(trc::Key::Details, auth.status.as_str()));\n            }\n        };\n\n        for i in 0u64..5 {\n            tokio::time::sleep(Duration::from_secs(1u64 << i)).await;\n            let auth = account.auth(url).await?;\n            match auth.status {\n                AuthStatus::Pending => {\n                    trc::event!(\n                        Acme(AcmeEvent::AuthPending),\n                        Hostname = domain.to_string(),\n                        Id = provider.id.to_string(),\n                        Total = i,\n                    );\n\n                    account.challenge(&challenge_url).await?\n                }\n                AuthStatus::Valid => {\n                    trc::event!(\n                        Acme(AcmeEvent::AuthValid),\n                        Hostname = domain.to_string(),\n                        Id = provider.id.to_string(),\n                    );\n\n                    return Ok(());\n                }\n                _ => {\n                    return Err(EventType::Acme(AcmeEvent::AuthError)\n                        .into_err()\n                        .ctx(trc::Key::Id, provider.id.to_string())\n                        .ctx(trc::Key::Details, auth.status.as_str()));\n                }\n            }\n        }\n        Err(EventType::Acme(AcmeEvent::AuthTooManyAttempts)\n            .into_err()\n            .ctx(trc::Key::Id, provider.id.to_string())\n            .ctx(trc::Key::Hostname, domain))\n    }\n}\n\nfn parse_cert(pem: &[u8]) -> trc::Result<(CertifiedKey, [DateTime<Utc>; 2])> {\n    let mut pems = pem::parse_many(pem).map_err(|err| {\n        EventType::Acme(AcmeEvent::Error)\n            .reason(err)\n            .caused_by(trc::location!())\n    })?;\n    if pems.len() < 2 {\n        return Err(EventType::Acme(AcmeEvent::Error)\n            .caused_by(trc::location!())\n            .ctx(trc::Key::Size, pems.len())\n            .details(\"Too few PEMs\"));\n    }\n    let pk = match any_ecdsa_type(&PrivateKeyDer::Pkcs8(PrivatePkcs8KeyDer::from(\n        pems.remove(0).contents(),\n    ))) {\n        Ok(pk) => pk,\n        Err(err) => {\n            return Err(EventType::Acme(AcmeEvent::Error)\n                .reason(err)\n                .caused_by(trc::location!()));\n        }\n    };\n    let cert_chain: Vec<CertificateDer> = pems\n        .into_iter()\n        .map(|p| CertificateDer::from(p.into_contents()))\n        .collect();\n    let validity = match parse_x509_certificate(&cert_chain[0]) {\n        Ok((_, cert)) => {\n            let validity = cert.validity();\n            [validity.not_before, validity.not_after].map(|t| {\n                Utc.timestamp_opt(t.timestamp(), 0)\n                    .earliest()\n                    .unwrap_or_default()\n            })\n        }\n        Err(err) => {\n            return Err(EventType::Acme(AcmeEvent::Error)\n                .reason(err)\n                .caused_by(trc::location!()));\n        }\n    };\n    let cert = CertifiedKey::new(cert_chain, pk);\n    Ok((cert, validity))\n}\n"
  },
  {
    "path": "crates/common/src/listener/acme/resolver.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    AcmeProvider, StaticResolver,\n    directory::{ACME_TLS_ALPN_NAME, SerializedCert},\n};\nuse crate::{KV_ACME, Server};\nuse rustls::{\n    ServerConfig,\n    crypto::ring::sign::any_ecdsa_type,\n    server::{ClientHello, ResolvesServerCert},\n    sign::CertifiedKey,\n};\nuse rustls_pki_types::{CertificateDer, PrivateKeyDer, PrivatePkcs8KeyDer};\nuse std::sync::Arc;\nuse store::{\n    dispatch::lookup::KeyValue,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AcmeEvent;\n\nimpl Server {\n    pub(crate) fn set_cert(&self, provider: &AcmeProvider, cert: Arc<CertifiedKey>) {\n        // Add certificates\n        let mut certificates = self.inner.data.tls_certificates.load().as_ref().clone();\n        for domain in provider.domains.iter() {\n            certificates.insert(\n                domain\n                    .strip_prefix(\"*.\")\n                    .unwrap_or(domain.as_str())\n                    .to_string(),\n                cert.clone(),\n            );\n        }\n\n        // Add default certificate\n        if provider.default {\n            certificates.insert(\"*\".to_string(), cert);\n        }\n\n        self.inner.data.tls_certificates.store(certificates.into());\n    }\n\n    pub(crate) async fn build_acme_certificate(&self, domain: &str) -> Option<Arc<CertifiedKey>> {\n        match self\n            .in_memory_store()\n            .key_get::<Archive<AlignedBytes>>(KeyValue::<()>::build_key(KV_ACME, domain))\n            .await\n        {\n            Ok(Some(cert_)) => match cert_.unarchive::<SerializedCert>() {\n                Ok(cert) => {\n                    match any_ecdsa_type(&PrivateKeyDer::Pkcs8(PrivatePkcs8KeyDer::from(\n                        cert.private_key.as_ref(),\n                    ))) {\n                        Ok(key) => Some(Arc::new(CertifiedKey::new(\n                            vec![CertificateDer::from(cert.certificate.to_vec())],\n                            key,\n                        ))),\n                        Err(err) => {\n                            trc::event!(\n                                Acme(AcmeEvent::Error),\n                                Domain = domain.to_string(),\n                                Reason = err.to_string(),\n                                Details = \"Failed to parse private key\"\n                            );\n                            None\n                        }\n                    }\n                }\n\n                Err(err) => {\n                    trc::event!(\n                        Acme(AcmeEvent::Error),\n                        Domain = domain.to_string(),\n                        CausedBy = err,\n                        Details = \"Failed to unarchive certificate\"\n                    );\n                    None\n                }\n            },\n            Err(err) => {\n                trc::event!(\n                    Acme(AcmeEvent::Error),\n                    Domain = domain.to_string(),\n                    CausedBy = err\n                );\n                None\n            }\n            Ok(None) => {\n                trc::event!(Acme(AcmeEvent::TokenNotFound), Domain = domain.to_string());\n                None\n            }\n        }\n    }\n}\n\nimpl ResolvesServerCert for StaticResolver {\n    fn resolve(&self, _: ClientHello) -> Option<Arc<CertifiedKey>> {\n        self.key.clone()\n    }\n}\n\npub(crate) fn build_acme_static_resolver(key: Option<Arc<CertifiedKey>>) -> Arc<ServerConfig> {\n    let mut challenge = ServerConfig::builder()\n        .with_no_client_auth()\n        .with_cert_resolver(Arc::new(StaticResolver { key }));\n    challenge.alpn_protocols.push(ACME_TLS_ALPN_NAME.to_vec());\n    Arc::new(challenge)\n}\n\npub trait IsTlsAlpnChallenge {\n    fn is_tls_alpn_challenge(&self) -> bool;\n}\n\nimpl IsTlsAlpnChallenge for ClientHello<'_> {\n    fn is_tls_alpn_challenge(&self) -> bool {\n        self.alpn().into_iter().flatten().eq([ACME_TLS_ALPN_NAME])\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/asn.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    net::IpAddr,\n    sync::{Arc, atomic::AtomicU64},\n    time::{Duration, Instant},\n};\n\nuse ahash::AHashMap;\nuse arc_swap::ArcSwap;\nuse mail_auth::common::resolver::ToReverseName;\nuse store::write::now;\nuse tokio::sync::Semaphore;\n\nuse crate::{Server, config::network::AsnGeoLookupConfig, manager::fetch_resource};\n\npub struct AsnGeoLookupData {\n    pub lock: Semaphore,\n    expires: AtomicU64,\n    asn: ArcSwap<Data<Arc<AsnData>>>,\n    country: ArcSwap<Data<Arc<String>>>,\n}\n\n#[derive(Clone, Default, Debug)]\npub struct AsnData {\n    pub id: u32,\n    pub name: Option<String>,\n}\n\n#[derive(Clone, Default, Debug)]\npub struct AsnGeoLookupResult {\n    pub asn: Option<Arc<AsnData>>,\n    pub country: Option<Arc<String>>,\n}\n\nstruct Data<T> {\n    ip4_ranges: Vec<IpRange<u32, T>>,\n    ip6_ranges: Vec<IpRange<u128, T>>,\n}\n\npub struct IpRange<I: Ord, T> {\n    pub start: I,\n    pub end: I,\n    pub data: T,\n}\n\nimpl Server {\n    pub async fn lookup_asn_country(&self, ip: IpAddr) -> AsnGeoLookupResult {\n        let mut result = AsnGeoLookupResult::default();\n\n        match &self.core.network.asn_geo_lookup {\n            AsnGeoLookupConfig::Resource { .. } if !ip.is_loopback() => {\n                let asn_geo = &self.inner.data.asn_geo_data;\n\n                if asn_geo.expires.load(std::sync::atomic::Ordering::Relaxed) <= now()\n                    && asn_geo.lock.available_permits() > 0\n                {\n                    self.refresh_asn_geo_tables();\n                }\n\n                result.asn = asn_geo.asn.load().lookup(ip).cloned();\n                result.country = asn_geo.country.load().lookup(ip).cloned();\n            }\n            AsnGeoLookupConfig::Dns {\n                zone_ipv4,\n                zone_ipv6,\n                separator,\n                index_asn,\n                index_asn_name,\n                index_country,\n            } if !ip.is_loopback() => {\n                let zone = if ip.is_ipv4() { zone_ipv4 } else { zone_ipv6 };\n                match self\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .txt_raw_lookup(format!(\"{}.{}.\", ip.to_reverse_name(), zone))\n                    .await\n                    .map(String::from_utf8)\n                {\n                    Ok(Ok(entry)) => {\n                        let mut asn = None;\n                        let mut asn_name = None;\n                        let mut country = None;\n\n                        for (idx, part) in entry.split(separator).enumerate() {\n                            let part = part.trim();\n                            if !part.is_empty() {\n                                if idx == *index_asn {\n                                    asn = part.parse::<u32>().ok();\n                                } else if index_asn_name.is_some_and(|i| i == idx) {\n                                    asn_name = Some(part.to_string());\n                                } else if index_country.is_some_and(|i| i == idx) {\n                                    country = Some(part.to_string());\n                                }\n                            }\n                        }\n\n                        if let Some(asn) = asn {\n                            result.asn = Some(Arc::new(AsnData {\n                                id: asn,\n                                name: asn_name,\n                            }));\n                        }\n\n                        if let Some(country) = country {\n                            result.country = Some(Arc::new(country));\n                        }\n                    }\n                    Ok(Err(_)) => {\n                        trc::event!(\n                            Resource(trc::ResourceEvent::Error),\n                            Details = \"Failed to UTF-8 decode ASN/Geo data\",\n                            Hostname = format!(\"{}.{}.\", ip.to_reverse_name(), zone),\n                        );\n                    }\n                    Err(err) => {\n                        trc::event!(\n                            Resource(trc::ResourceEvent::Error),\n                            Details = \"Failed to lookup ASN/Geo data\",\n                            Hostname = format!(\"{}.{}.\", ip.to_reverse_name(), zone),\n                            CausedBy = err.to_string()\n                        );\n                    }\n                }\n            }\n            _ => (),\n        }\n\n        result\n    }\n\n    fn refresh_asn_geo_tables(&self) {\n        let server = self.clone();\n        tokio::spawn(async move {\n            let asn_geo = &server.inner.data.asn_geo_data;\n            let _permit = asn_geo.lock.acquire().await;\n\n            if asn_geo.expires.load(std::sync::atomic::Ordering::Relaxed) > now() {\n                return;\n            }\n\n            if let AsnGeoLookupConfig::Resource {\n                expires,\n                timeout,\n                max_size,\n                asn_resources,\n                geo_resources,\n                headers,\n            } = &server.core.network.asn_geo_lookup\n            {\n                let mut asn_data = Data::new();\n                let mut country_data = Data::new();\n\n                for (is_asn, url) in asn_resources\n                    .iter()\n                    .map(|url| (true, url))\n                    .chain(geo_resources.iter().map(|url| (false, url)))\n                {\n                    let time = Instant::now();\n                    match fetch_resource(url, headers.clone().into(), *timeout, *max_size)\n                        .await\n                        .map(String::from_utf8)\n                    {\n                        Ok(Ok(data)) => {\n                            let mut has_errors = false;\n                            let mut asn_mappings = AHashMap::new();\n                            let mut geo_mappings = AHashMap::new();\n\n                            let mut from_ip = None;\n                            let mut to_ip = None;\n                            let mut asn = None;\n                            let mut details = None;\n\n                            let mut in_quote = false;\n                            let mut col_num = 0;\n                            let mut col_start = 0;\n                            let mut line_start = 0;\n\n                            for (idx, ch) in data.char_indices() {\n                                match ch {\n                                    '\"' => in_quote = !in_quote,\n                                    ',' | '\\n' if !in_quote => {\n                                        let column =\n                                            data.get(col_start..idx).unwrap_or_default().trim();\n                                        match col_num {\n                                            0 => from_ip = column.parse::<IpAddr>().ok(),\n                                            1 => to_ip = column.parse::<IpAddr>().ok(),\n                                            2 if is_asn => asn = column.parse::<u32>().ok(),\n                                            2 | 3 => {\n                                                let column = column\n                                                    .strip_prefix('\"')\n                                                    .and_then(|s| s.strip_suffix('\"'))\n                                                    .unwrap_or(column);\n                                                if !column.is_empty() || details.is_none() {\n                                                    details = Some(column);\n                                                }\n                                            }\n                                            _ => break,\n                                        }\n\n                                        if ch == '\\n' {\n                                            let is_success = match (from_ip, to_ip, asn, details) {\n                                                (\n                                                    Some(from_ip),\n                                                    Some(to_ip),\n                                                    Some(asn),\n                                                    asn_name,\n                                                ) if is_asn => {\n                                                    let data = asn_mappings\n                                                        .entry(asn)\n                                                        .or_insert_with(|| {\n                                                            Arc::new(AsnData {\n                                                                id: asn,\n                                                                name: asn_name.map(String::from),\n                                                            })\n                                                        })\n                                                        .clone();\n                                                    asn_data.insert(from_ip, to_ip, data)\n                                                }\n                                                (Some(from_ip), Some(to_ip), _, Some(code))\n                                                    if !is_asn && [2, 3].contains(&code.len()) =>\n                                                {\n                                                    let code = code.to_uppercase();\n                                                    let data = geo_mappings\n                                                        .entry(code.clone())\n                                                        .or_insert_with(|| Arc::new(code))\n                                                        .clone();\n                                                    country_data.insert(from_ip, to_ip, data)\n                                                }\n                                                (None, None, _, _) => true, // Ignore empty rows\n                                                _ => false,\n                                            };\n\n                                            if !is_success && !has_errors {\n                                                trc::event!(\n                                                    Resource(trc::ResourceEvent::Error),\n                                                    Details = \"Invalid ASN/Geo data\",\n                                                    Url = url.clone(),\n                                                    Details = data\n                                                        .get(line_start..idx)\n                                                        .unwrap_or_default()\n                                                        .to_string(),\n                                                );\n                                                has_errors = true;\n                                            }\n\n                                            col_num = 0;\n                                            from_ip = None;\n                                            to_ip = None;\n                                            asn = None;\n                                            details = None;\n                                            line_start = idx + 1;\n                                        } else {\n                                            col_num += 1;\n                                        }\n                                        col_start = idx + 1;\n                                    }\n                                    _ => {}\n                                }\n                            }\n\n                            trc::event!(\n                                Resource(trc::ResourceEvent::DownloadExternal),\n                                Details = \"Downloaded ASN/Geo data\",\n                                Url = url.clone(),\n                                Elapsed = time.elapsed()\n                            );\n                        }\n                        Ok(Err(_)) => {\n                            trc::event!(\n                                Resource(trc::ResourceEvent::Error),\n                                Details = \"Failed to UTF-8 decode ASN/Geo data\",\n                                Url = url.clone(),\n                            );\n                        }\n                        Err(err) => {\n                            trc::event!(\n                                Resource(trc::ResourceEvent::Error),\n                                Details = \"Failed to download ASN/Geo data\",\n                                Url = url.clone(),\n                                CausedBy = err\n                            );\n                        }\n                    }\n                }\n\n                let expires = if !asn_data.is_empty() || !country_data.is_empty() {\n                    *expires\n                } else {\n                    Duration::from_secs(60)\n                };\n\n                if !asn_data.is_empty() {\n                    asn_geo.asn.store(Arc::new(asn_data.sorted()));\n                }\n                if !country_data.is_empty() {\n                    asn_geo.country.store(Arc::new(country_data.sorted()));\n                }\n\n                asn_geo.expires.store(\n                    now() + expires.as_secs(),\n                    std::sync::atomic::Ordering::Relaxed,\n                );\n            }\n        });\n    }\n}\n\nimpl<T> Data<T> {\n    fn new() -> Self {\n        Self {\n            ip4_ranges: Vec::new(),\n            ip6_ranges: Vec::new(),\n        }\n    }\n\n    pub fn lookup(&self, ip: IpAddr) -> Option<&T> {\n        match ip {\n            IpAddr::V4(ip) => {\n                let ip = u32::from(ip);\n                match self.ip4_ranges.binary_search_by(|range| {\n                    if ip < range.start {\n                        std::cmp::Ordering::Greater\n                    } else if ip > range.end {\n                        std::cmp::Ordering::Less\n                    } else {\n                        std::cmp::Ordering::Equal\n                    }\n                }) {\n                    Ok(idx) => Some(&self.ip4_ranges[idx].data),\n                    Err(_) => None,\n                }\n            }\n            IpAddr::V6(ip) => {\n                let ip = u128::from(ip);\n                match self.ip6_ranges.binary_search_by(|range| {\n                    if ip < range.start {\n                        std::cmp::Ordering::Greater\n                    } else if ip > range.end {\n                        std::cmp::Ordering::Less\n                    } else {\n                        std::cmp::Ordering::Equal\n                    }\n                }) {\n                    Ok(idx) => Some(&self.ip6_ranges[idx].data),\n                    Err(_) => None,\n                }\n            }\n        }\n    }\n\n    pub fn insert(&mut self, from_ip: IpAddr, to_ip: IpAddr, data: T) -> bool {\n        match (from_ip, to_ip) {\n            (IpAddr::V4(from), IpAddr::V4(to)) => {\n                self.ip4_ranges.push(IpRange {\n                    start: u32::from(from),\n                    end: u32::from(to),\n                    data,\n                });\n                true\n            }\n            (IpAddr::V6(from), IpAddr::V6(to)) => {\n                self.ip6_ranges.push(IpRange {\n                    start: u128::from(from),\n                    end: u128::from(to),\n                    data,\n                });\n                true\n            }\n            _ => false,\n        }\n    }\n\n    pub fn sorted(mut self) -> Self {\n        self.ip4_ranges.sort_unstable_by_key(|range| range.start);\n        self.ip6_ranges.sort_unstable_by_key(|range| range.start);\n        self\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.ip4_ranges.is_empty() && self.ip6_ranges.is_empty()\n    }\n}\n\nimpl Default for AsnGeoLookupData {\n    fn default() -> Self {\n        Self {\n            lock: Semaphore::new(1),\n            expires: AtomicU64::new(0),\n            asn: ArcSwap::new(Arc::new(Data::new())),\n            country: ArcSwap::new(Arc::new(Data::new())),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/blocked.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fmt::Debug, net::IpAddr};\n\nuse ahash::AHashSet;\nuse utils::{\n    config::{\n        Config, ConfigKey, Rate,\n        ipmask::{IpAddrMask, IpAddrOrMask},\n        utils::ParseValue,\n    },\n    glob::GlobPattern,\n};\n\nuse crate::{\n    KV_RATE_LIMIT_AUTH, KV_RATE_LIMIT_LOITER, KV_RATE_LIMIT_RCPT, KV_RATE_LIMIT_SCAN, Server,\n    ip_to_bytes, ipc::BroadcastEvent, manager::config::MatchType,\n};\n\n#[derive(Debug, Clone)]\npub struct Security {\n    blocked_ip_networks: Vec<IpAddrMask>,\n    has_blocked_networks: bool,\n\n    allowed_ip_addresses: AHashSet<IpAddr>,\n    allowed_ip_networks: Vec<IpAddrMask>,\n    has_allowed_networks: bool,\n\n    http_banned_paths: Vec<MatchType>,\n    scanner_fail_rate: Option<Rate>,\n\n    auth_fail_rate: Option<Rate>,\n    rcpt_fail_rate: Option<Rate>,\n    loiter_fail_rate: Option<Rate>,\n}\n\npub const BLOCKED_IP_KEY: &str = \"server.blocked-ip\";\npub const BLOCKED_IP_PREFIX: &str = \"server.blocked-ip.\";\npub const ALLOWED_IP_KEY: &str = \"server.allowed-ip\";\npub const ALLOWED_IP_PREFIX: &str = \"server.allowed-ip.\";\n\npub struct BlockedIps {\n    pub blocked_ip_addresses: AHashSet<IpAddr>,\n    pub blocked_ip_networks: Vec<IpAddrMask>,\n}\n\nimpl Security {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut allowed_ip_addresses = AHashSet::new();\n        let mut allowed_ip_networks = Vec::new();\n\n        for ip in config\n            .set_values(ALLOWED_IP_KEY)\n            .map(IpAddrOrMask::parse_value)\n            .collect::<Vec<_>>()\n        {\n            match ip {\n                Ok(IpAddrOrMask::Ip(ip)) => {\n                    allowed_ip_addresses.insert(ip);\n                }\n                Ok(IpAddrOrMask::Mask(ip)) => {\n                    allowed_ip_networks.push(ip);\n                }\n                Err(err) => {\n                    config.new_parse_error(ALLOWED_IP_KEY, err);\n                }\n            }\n        }\n\n        #[cfg(not(feature = \"test_mode\"))]\n        {\n            // Add loopback addresses\n            allowed_ip_addresses.insert(IpAddr::V4(std::net::Ipv4Addr::LOCALHOST));\n            allowed_ip_addresses.insert(IpAddr::V6(std::net::Ipv6Addr::LOCALHOST));\n        }\n\n        let blocked = BlockedIps::parse(config);\n\n        // Parse blocked HTTP paths\n        let mut http_banned_paths = config\n            .values(\"server.auto-ban.scan.paths\")\n            .filter_map(|(_, v)| {\n                let v = v.trim();\n                if !v.is_empty() {\n                    MatchType::parse(v).into()\n                } else {\n                    None\n                }\n            })\n            .collect::<Vec<_>>();\n        if http_banned_paths.is_empty() {\n            for pattern in [\n                \"*.php*\",\n                \"*.cgi*\",\n                \"*.asp*\",\n                \"*/wp-*\",\n                \"*/php*\",\n                \"*/cgi-bin*\",\n                \"*xmlrpc*\",\n                \"*../*\",\n                \"*/..*\",\n                \"*joomla*\",\n                \"*wordpress*\",\n                \"*drupal*\",\n            ]\n            .iter()\n            {\n                http_banned_paths.push(MatchType::Matches(GlobPattern::compile(pattern, true)));\n            }\n        }\n\n        Security {\n            has_blocked_networks: !blocked.blocked_ip_networks.is_empty(),\n            blocked_ip_networks: blocked.blocked_ip_networks,\n            has_allowed_networks: !allowed_ip_networks.is_empty(),\n            allowed_ip_addresses,\n            allowed_ip_networks,\n            auth_fail_rate: config\n                .property_or_default::<Option<Rate>>(\"server.auto-ban.auth.rate\", \"100/1d\")\n                .unwrap_or_default(),\n            rcpt_fail_rate: config\n                .property_or_default::<Option<Rate>>(\"server.auto-ban.abuse.rate\", \"35/1d\")\n                .unwrap_or_default(),\n            loiter_fail_rate: config\n                .property_or_default::<Option<Rate>>(\"server.auto-ban.loiter.rate\", \"150/1d\")\n                .unwrap_or_default(),\n            http_banned_paths,\n            scanner_fail_rate: config\n                .property_or_default::<Option<Rate>>(\"server.auto-ban.scan.rate\", \"30/1d\")\n                .unwrap_or_default(),\n        }\n    }\n}\n\nimpl Server {\n    pub async fn is_rcpt_fail2banned(&self, ip: IpAddr, rcpt: &str) -> trc::Result<bool> {\n        if let Some(rate) = &self.core.network.security.rcpt_fail_rate {\n            let is_allowed = self.is_ip_allowed(&ip)\n                || (self\n                    .in_memory_store()\n                    .is_rate_allowed(KV_RATE_LIMIT_RCPT, &ip_to_bytes(&ip), rate, false)\n                    .await?\n                    .is_none()\n                    && self\n                        .in_memory_store()\n                        .is_rate_allowed(KV_RATE_LIMIT_RCPT, rcpt.as_bytes(), rate, false)\n                        .await?\n                        .is_none());\n\n            if !is_allowed {\n                return self.block_ip(ip).await.map(|_| true);\n            }\n        }\n\n        Ok(false)\n    }\n\n    pub async fn is_scanner_fail2banned(&self, ip: IpAddr) -> trc::Result<bool> {\n        if let Some(rate) = &self.core.network.security.scanner_fail_rate {\n            let is_allowed = self.is_ip_allowed(&ip)\n                || self\n                    .in_memory_store()\n                    .is_rate_allowed(KV_RATE_LIMIT_SCAN, &ip_to_bytes(&ip), rate, false)\n                    .await?\n                    .is_none();\n\n            if !is_allowed {\n                return self.block_ip(ip).await.map(|_| true);\n            }\n        }\n\n        Ok(false)\n    }\n\n    pub async fn is_http_banned_path(&self, path: &str, ip: IpAddr) -> trc::Result<bool> {\n        let paths = &self.core.network.security.http_banned_paths;\n\n        if !paths.is_empty() && paths.iter().any(|p| p.matches(path)) && !self.is_ip_allowed(&ip) {\n            self.block_ip(ip).await.map(|_| true)\n        } else {\n            Ok(false)\n        }\n    }\n\n    pub async fn is_loiter_fail2banned(&self, ip: IpAddr) -> trc::Result<bool> {\n        if let Some(rate) = &self.core.network.security.loiter_fail_rate {\n            let is_allowed = self.is_ip_allowed(&ip)\n                || self\n                    .in_memory_store()\n                    .is_rate_allowed(KV_RATE_LIMIT_LOITER, &ip_to_bytes(&ip), rate, false)\n                    .await?\n                    .is_none();\n\n            if !is_allowed {\n                return self.block_ip(ip).await.map(|_| true);\n            }\n        }\n\n        Ok(false)\n    }\n\n    pub async fn is_auth_fail2banned(&self, ip: IpAddr, login: Option<&str>) -> trc::Result<bool> {\n        if let Some(rate) = &self.core.network.security.auth_fail_rate {\n            let login = login.unwrap_or_default();\n            let is_allowed = self.is_ip_allowed(&ip)\n                || (self\n                    .in_memory_store()\n                    .is_rate_allowed(KV_RATE_LIMIT_AUTH, &ip_to_bytes(&ip), rate, false)\n                    .await?\n                    .is_none()\n                    && (login.is_empty()\n                        || self\n                            .in_memory_store()\n                            .is_rate_allowed(KV_RATE_LIMIT_AUTH, login.as_bytes(), rate, false)\n                            .await?\n                            .is_none()));\n            if !is_allowed {\n                return self.block_ip(ip).await.map(|_| true);\n            }\n        }\n\n        Ok(false)\n    }\n\n    pub async fn block_ip(&self, ip: IpAddr) -> trc::Result<()> {\n        // Add IP to blocked list\n        self.inner.data.blocked_ips.write().insert(ip);\n\n        // Write blocked IP to config\n        self.core\n            .storage\n            .config\n            .set(\n                [ConfigKey {\n                    key: format!(\"{}.{}\", BLOCKED_IP_KEY, ip),\n                    value: String::new(),\n                }],\n                true,\n            )\n            .await?;\n\n        // Increment version\n        self.cluster_broadcast(BroadcastEvent::ReloadBlockedIps)\n            .await;\n\n        Ok(())\n    }\n\n    pub fn has_auth_fail2ban(&self) -> bool {\n        self.core.network.security.auth_fail_rate.is_some()\n    }\n\n    pub fn is_ip_blocked(&self, ip: &IpAddr) -> bool {\n        self.inner.data.blocked_ips.read().contains(ip)\n            || (self.core.network.security.has_blocked_networks\n                && self\n                    .core\n                    .network\n                    .security\n                    .blocked_ip_networks\n                    .iter()\n                    .any(|network| network.matches(ip)))\n    }\n\n    pub fn is_ip_allowed(&self, ip: &IpAddr) -> bool {\n        self.core.network.security.allowed_ip_addresses.contains(ip)\n            || (self.core.network.security.has_allowed_networks\n                && self\n                    .core\n                    .network\n                    .security\n                    .allowed_ip_networks\n                    .iter()\n                    .any(|network| network.matches(ip)))\n    }\n}\n\nimpl BlockedIps {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut blocked_ip_addresses = AHashSet::new();\n        let mut blocked_ip_networks = Vec::new();\n\n        for ip in config\n            .set_values(BLOCKED_IP_KEY)\n            .map(IpAddrOrMask::parse_value)\n            .collect::<Vec<_>>()\n        {\n            match ip {\n                Ok(IpAddrOrMask::Ip(ip)) => {\n                    blocked_ip_addresses.insert(ip);\n                }\n                Ok(IpAddrOrMask::Mask(ip)) => {\n                    blocked_ip_networks.push(ip);\n                }\n                Err(err) => {\n                    config.new_parse_error(BLOCKED_IP_KEY, err);\n                }\n            }\n        }\n\n        Self {\n            blocked_ip_addresses,\n            blocked_ip_networks,\n        }\n    }\n}\n\n#[allow(clippy::derivable_impls)]\nimpl Default for Security {\n    fn default() -> Self {\n        // Add IPv4 and IPv6 loopback addresses\n        Self {\n            #[cfg(not(feature = \"test_mode\"))]\n            allowed_ip_addresses: AHashSet::from_iter([\n                IpAddr::V4(std::net::Ipv4Addr::LOCALHOST),\n                IpAddr::V6(std::net::Ipv6Addr::LOCALHOST),\n            ]),\n            #[cfg(feature = \"test_mode\")]\n            allowed_ip_addresses: Default::default(),\n            allowed_ip_networks: Default::default(),\n            has_allowed_networks: Default::default(),\n            blocked_ip_networks: Default::default(),\n            has_blocked_networks: Default::default(),\n            auth_fail_rate: Default::default(),\n            rcpt_fail_rate: Default::default(),\n            loiter_fail_rate: Default::default(),\n            scanner_fail_rate: Default::default(),\n            http_banned_paths: Default::default(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/limiter.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::{\n    Arc,\n    atomic::{AtomicU64, Ordering},\n};\n\n#[derive(Debug, Clone)]\npub struct ConcurrencyLimiter {\n    pub max_concurrent: u64,\n    pub concurrent: Arc<AtomicU64>,\n}\n\n#[derive(Default)]\npub struct InFlight {\n    concurrent: Arc<AtomicU64>,\n}\n\npub enum LimiterResult {\n    Allowed(InFlight),\n    Forbidden,\n    Disabled,\n}\n\nimpl Drop for InFlight {\n    fn drop(&mut self) {\n        self.concurrent.fetch_sub(1, Ordering::Relaxed);\n    }\n}\n\nimpl ConcurrencyLimiter {\n    pub fn new(max_concurrent: u64) -> Self {\n        ConcurrencyLimiter {\n            max_concurrent,\n            concurrent: Arc::new(0.into()),\n        }\n    }\n\n    pub fn is_allowed(&self) -> LimiterResult {\n        if self.concurrent.load(Ordering::Relaxed) < self.max_concurrent {\n            // Return in-flight request\n            self.concurrent.fetch_add(1, Ordering::Relaxed);\n            LimiterResult::Allowed(InFlight {\n                concurrent: self.concurrent.clone(),\n            })\n        } else {\n            LimiterResult::Forbidden\n        }\n    }\n\n    pub fn check_is_allowed(&self) -> bool {\n        self.concurrent.load(Ordering::Relaxed) < self.max_concurrent\n    }\n\n    pub fn is_active(&self) -> bool {\n        self.concurrent.load(Ordering::Relaxed) > 0\n    }\n}\n\nimpl InFlight {\n    pub fn num_concurrent(&self) -> u64 {\n        self.concurrent.load(Ordering::Relaxed)\n    }\n}\n\nimpl From<LimiterResult> for Option<InFlight> {\n    fn from(result: LimiterResult) -> Self {\n        match result {\n            LimiterResult::Allowed(in_flight) => Some(in_flight),\n            LimiterResult::Forbidden => None,\n            LimiterResult::Disabled => Some(InFlight::default()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/listen.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    net::{IpAddr, SocketAddr},\n    sync::Arc,\n    time::Duration,\n};\n\nuse proxy_header::io::ProxiedStream;\nuse rustls::crypto::ring::cipher_suite::TLS13_AES_128_GCM_SHA256;\nuse tokio::{net::TcpStream, sync::watch};\nuse tokio_rustls::server::TlsStream;\nuse trc::{EventType, HttpEvent, ImapEvent, ManageSieveEvent, Pop3Event, SmtpEvent};\nuse utils::{UnwrapFailure, config::Config};\n\nuse crate::{\n    Inner, Server,\n    config::server::{Listener, Listeners, ServerProtocol, TcpListener},\n    core::BuildServer,\n};\n\nuse super::{\n    ServerInstance, SessionData, SessionManager, SessionStream, TcpAcceptor,\n    limiter::{ConcurrencyLimiter, LimiterResult},\n};\n\nimpl Listener {\n    pub fn spawn(\n        self,\n        manager: impl SessionManager,\n        inner: Arc<Inner>,\n        acceptor: TcpAcceptor,\n        shutdown_rx: watch::Receiver<bool>,\n    ) {\n        // Prepare instance\n        let instance = Arc::new(ServerInstance {\n            id: self.id,\n            protocol: self.protocol,\n            proxy_networks: self.proxy_networks,\n            limiter: ConcurrencyLimiter::new(self.max_connections),\n            acceptor,\n            shutdown_rx,\n            span_id_gen: self.span_id_gen,\n        });\n        let is_tls = matches!(instance.acceptor, TcpAcceptor::Tls { implicit, .. } if implicit);\n        let is_https = is_tls && self.protocol == ServerProtocol::Http;\n        let has_proxies = !instance.proxy_networks.is_empty();\n\n        // Spawn listeners\n        for listener in self.listeners {\n            let local_addr = listener.addr;\n\n            // Obtain TCP options\n            let opts = SocketOpts {\n                nodelay: listener.nodelay,\n                ttl: listener.ttl,\n                linger: listener.linger,\n            };\n\n            // Bind socket\n            let listener = match listener.listen() {\n                Ok(listener) => {\n                    trc::event!(\n                        Network(trc::NetworkEvent::ListenStart),\n                        ListenerId = instance.id.clone(),\n                        LocalIp = local_addr.ip(),\n                        LocalPort = local_addr.port(),\n                        Tls = is_tls,\n                    );\n\n                    listener\n                }\n                Err(err) => {\n                    trc::event!(\n                        Network(trc::NetworkEvent::ListenError),\n                        ListenerId = instance.id.clone(),\n                        LocalIp = local_addr.ip(),\n                        LocalPort = local_addr.port(),\n                        Tls = is_tls,\n                        Reason = err,\n                    );\n\n                    continue;\n                }\n            };\n\n            // Spawn listener\n            let mut shutdown_rx = instance.shutdown_rx.clone();\n            let manager = manager.clone();\n            let instance = instance.clone();\n            let inner = inner.clone();\n            tokio::spawn(async move {\n                let (span_start, span_end) = match self.protocol {\n                    ServerProtocol::Smtp | ServerProtocol::Lmtp => (\n                        EventType::Smtp(SmtpEvent::ConnectionStart),\n                        EventType::Smtp(SmtpEvent::ConnectionEnd),\n                    ),\n                    ServerProtocol::Imap => (\n                        EventType::Imap(ImapEvent::ConnectionStart),\n                        EventType::Imap(ImapEvent::ConnectionEnd),\n                    ),\n                    ServerProtocol::Pop3 => (\n                        EventType::Pop3(Pop3Event::ConnectionStart),\n                        EventType::Pop3(Pop3Event::ConnectionEnd),\n                    ),\n                    ServerProtocol::Http => (\n                        EventType::Http(HttpEvent::ConnectionStart),\n                        EventType::Http(HttpEvent::ConnectionEnd),\n                    ),\n                    ServerProtocol::ManageSieve => (\n                        EventType::ManageSieve(ManageSieveEvent::ConnectionStart),\n                        EventType::ManageSieve(ManageSieveEvent::ConnectionEnd),\n                    ),\n                };\n\n                loop {\n                    tokio::select! {\n                        stream = listener.accept() => {\n                            match stream {\n                                Ok((stream, remote_addr)) => {\n                                    let server = inner.build_server();\n                                    let enable_acme = (is_https && server.has_acme_tls_providers()).then(|| server.clone());\n\n                                    if has_proxies && instance.proxy_networks.iter().any(|network| network.matches(&remote_addr.ip())) {\n                                        let instance = instance.clone();\n                                        let manager = manager.clone();\n\n                                        // Set socket options\n                                        opts.apply(&stream);\n\n                                        tokio::spawn(async move {\n                                            match ProxiedStream::create_from_tokio(stream, Default::default()).await {\n                                                Ok(stream) =>{\n                                                    let remote_addr = stream.proxy_header()\n                                                                            .proxied_address()\n                                                                            .map(|addr| addr.source)\n                                                                            .unwrap_or(remote_addr);\n                                                    if let Some(session) = instance.build_session(stream, local_addr, remote_addr, &server) {\n                                                        // Spawn session\n                                                        manager.spawn(session, is_tls, enable_acme, span_start, span_end);\n                                                    }\n                                                }\n                                                Err(err) => {\n                                                    trc::event!(\n                                                        Network(trc::NetworkEvent::ProxyError),\n                                                        ListenerId = instance.id.clone(),\n                                                        LocalIp = local_addr.ip(),\n                                                        LocalPort = local_addr.port(),\n                                                        Tls = is_tls,\n                                                        Reason = err.to_string(),\n                                                    );\n                                                }\n                                            }\n                                        });\n                                    } else if let Some(session) = instance.build_session(stream, local_addr, remote_addr, &server) {\n                                        // Set socket options\n                                        opts.apply(&session.stream);\n\n                                        // Spawn session\n                                        manager.spawn(session, is_tls, enable_acme, span_start, span_end);\n                                    }\n                                }\n                                Err(err) => {\n                                    trc::event!(\n                                        Network(trc::NetworkEvent::AcceptError),\n                                        ListenerId = instance.id.clone(),\n                                        LocalIp = local_addr.ip(),\n                                        LocalPort = local_addr.port(),\n                                        Tls = is_tls,\n                                        Reason = err.to_string(),\n                                    );\n                                }\n                            }\n                        },\n                        _ = shutdown_rx.changed() => {\n\n                            trc::event!(\n                                Network(trc::NetworkEvent::ListenStop),\n                                ListenerId = instance.id.clone(),\n                                LocalIp = local_addr.ip(),\n                                Tls = is_tls,\n                                LocalPort = local_addr.port(),\n                            );\n\n                            manager.shutdown().await;\n                            break;\n                        }\n                    };\n                }\n            });\n        }\n    }\n}\n\ntrait BuildSession {\n    fn build_session<T: SessionStream>(\n        &self,\n        stream: T,\n        local_addr: SocketAddr,\n        remote_addr: SocketAddr,\n        server: &Server,\n    ) -> Option<SessionData<T>>;\n}\n\nimpl BuildSession for Arc<ServerInstance> {\n    fn build_session<T: SessionStream>(\n        &self,\n        stream: T,\n        local_addr: SocketAddr,\n        remote_addr: SocketAddr,\n        server: &Server,\n    ) -> Option<SessionData<T>> {\n        // Convert mapped IPv6 addresses to IPv4\n        let remote_ip = match remote_addr.ip() {\n            IpAddr::V6(ip) => ip\n                .to_ipv4_mapped()\n                .map(IpAddr::V4)\n                .unwrap_or(IpAddr::V6(ip)),\n            remote_ip => remote_ip,\n        };\n        let remote_port = remote_addr.port();\n\n        // Check if blocked\n        if server.is_ip_blocked(&remote_ip) {\n            trc::event!(\n                Security(trc::SecurityEvent::IpBlocked),\n                ListenerId = self.id.clone(),\n                LocalPort = local_addr.port(),\n                RemoteIp = remote_ip,\n                RemotePort = remote_port,\n            );\n            None\n        } else if let LimiterResult::Allowed(in_flight) = self.limiter.is_allowed() {\n            // Enforce concurrency\n            SessionData {\n                stream,\n                in_flight,\n                local_ip: local_addr.ip(),\n                local_port: local_addr.port(),\n                session_id: 0,\n                remote_ip,\n                remote_port,\n                protocol: self.protocol,\n                instance: self.clone(),\n            }\n            .into()\n        } else {\n            trc::event!(\n                Limit(trc::LimitEvent::ConcurrentConnection),\n                ListenerId = self.id.clone(),\n                LocalPort = local_addr.port(),\n                RemoteIp = remote_ip,\n                RemotePort = remote_port,\n                Limit = self.limiter.max_concurrent,\n            );\n\n            None\n        }\n    }\n}\n\npub struct SocketOpts {\n    pub nodelay: bool,\n    pub ttl: Option<u32>,\n    pub linger: Option<Duration>,\n}\n\nimpl SocketOpts {\n    pub fn apply(&self, stream: &TcpStream) {\n        // Set TCP options\n        if let Err(err) = stream.set_nodelay(self.nodelay) {\n            trc::event!(\n                Network(trc::NetworkEvent::SetOptError),\n                Reason = err.to_string(),\n                Details = \"Failed to set TCP_NODELAY\",\n            );\n        }\n        if let Some(ttl) = self.ttl\n            && let Err(err) = stream.set_ttl(ttl)\n        {\n            trc::event!(\n                Network(trc::NetworkEvent::SetOptError),\n                Reason = err.to_string(),\n                Details = \"Failed to set TTL\",\n            );\n        }\n        if self.linger.is_some()\n            && let Err(err) = stream.set_linger(self.linger)\n        {\n            trc::event!(\n                Network(trc::NetworkEvent::SetOptError),\n                Reason = err.to_string(),\n                Details = \"Failed to set LINGER\",\n            );\n        }\n    }\n}\n\nimpl Listeners {\n    pub fn bind_and_drop_priv(&self, config: &mut Config) {\n        // Bind as root\n        for server in &self.servers {\n            for listener in &server.listeners {\n                if let Err(err) = listener.socket.bind(listener.addr) {\n                    config.new_build_error(\n                        format!(\"server.listener.{}\", server.id),\n                        format!(\"Failed to bind to {}: {}\", listener.addr, err),\n                    );\n                }\n            }\n        }\n\n        // Drop privileges\n        #[cfg(not(target_env = \"msvc\"))]\n        {\n            if let Ok(run_as_user) = std::env::var(\"RUN_AS_USER\") {\n                let mut pd = privdrop::PrivDrop::default()\n                    .user(run_as_user)\n                    .fallback_to_ids_if_names_are_numeric();\n                if let Ok(run_as_group) = std::env::var(\"RUN_AS_GROUP\") {\n                    pd = pd\n                        .group(run_as_group)\n                        .fallback_to_ids_if_names_are_numeric();\n                }\n                pd.apply().failed(\"Failed to drop privileges\");\n            }\n        }\n    }\n\n    pub fn spawn(\n        mut self,\n        spawn: impl Fn(Listener, TcpAcceptor, watch::Receiver<bool>),\n    ) -> (watch::Sender<bool>, watch::Receiver<bool>) {\n        // Spawn listeners\n        let (shutdown_tx, shutdown_rx) = watch::channel(false);\n        for server in self.servers {\n            let acceptor = self\n                .tcp_acceptors\n                .remove(&server.id)\n                .unwrap_or(TcpAcceptor::Plain);\n\n            spawn(server, acceptor, shutdown_rx.clone());\n        }\n        (shutdown_tx, shutdown_rx)\n    }\n}\n\nimpl TcpListener {\n    pub fn listen(self) -> Result<tokio::net::TcpListener, String> {\n        self.socket\n            .listen(self.backlog.unwrap_or(1024))\n            .map_err(|err| format!(\"Failed to listen on {}: {}\", self.addr, err))\n    }\n}\n\nimpl ServerInstance {\n    pub async fn tls_accept<T: SessionStream>(\n        &self,\n        stream: T,\n        session_id: u64,\n    ) -> Result<TlsStream<T>, ()> {\n        match &self.acceptor {\n            TcpAcceptor::Tls { acceptor, .. } => match acceptor.accept(stream).await {\n                Ok(stream) => {\n                    trc::event!(\n                        Tls(trc::TlsEvent::Handshake),\n                        ListenerId = self.id.clone(),\n                        SpanId = session_id,\n                        Version = format!(\n                            \"{:?}\",\n                            stream\n                                .get_ref()\n                                .1\n                                .protocol_version()\n                                .unwrap_or(rustls::ProtocolVersion::TLSv1_3)\n                        ),\n                        Details = format!(\n                            \"{:?}\",\n                            stream\n                                .get_ref()\n                                .1\n                                .negotiated_cipher_suite()\n                                .unwrap_or(TLS13_AES_128_GCM_SHA256)\n                        )\n                    );\n                    Ok(stream)\n                }\n                Err(err) => {\n                    trc::event!(\n                        Tls(trc::TlsEvent::HandshakeError),\n                        ListenerId = self.id.clone(),\n                        SpanId = session_id,\n                        Reason = err.to_string(),\n                    );\n                    Err(())\n                }\n            },\n            TcpAcceptor::Plain => {\n                trc::event!(\n                    Tls(trc::TlsEvent::NotConfigured),\n                    ListenerId = self.id.clone(),\n                    SpanId = session_id,\n                );\n                Err(())\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, net::IpAddr, sync::Arc, time::Instant};\n\nuse compact_str::ToCompactString;\nuse rustls::ServerConfig;\nuse std::fmt::Debug;\nuse tokio::{\n    io::{AsyncRead, AsyncWrite},\n    sync::watch,\n};\nuse tokio_rustls::{Accept, TlsAcceptor};\nuse trc::{Event, EventType, Key};\nuse utils::{config::ipmask::IpAddrMask, snowflake::SnowflakeIdGenerator};\n\nuse crate::{\n    Server,\n    config::server::ServerProtocol,\n    expr::{functions::ResolveVariable, *},\n};\n\nuse self::limiter::{ConcurrencyLimiter, InFlight};\n\npub mod acme;\npub mod asn;\npub mod blocked;\npub mod limiter;\npub mod listen;\npub mod stream;\npub mod tls;\n\npub struct ServerInstance {\n    pub id: String,\n    pub protocol: ServerProtocol,\n    pub acceptor: TcpAcceptor,\n    pub limiter: ConcurrencyLimiter,\n    pub proxy_networks: Vec<IpAddrMask>,\n    pub shutdown_rx: watch::Receiver<bool>,\n    pub span_id_gen: Arc<SnowflakeIdGenerator>,\n}\n\n#[derive(Default)]\npub enum TcpAcceptor {\n    Tls {\n        config: Arc<ServerConfig>,\n        acceptor: TlsAcceptor,\n        implicit: bool,\n    },\n    #[default]\n    Plain,\n}\n\n#[allow(clippy::large_enum_variant)]\npub enum TcpAcceptorResult<IO>\nwhere\n    IO: AsyncRead + AsyncWrite + Unpin,\n{\n    Tls(Accept<IO>),\n    Plain(IO),\n    Close,\n}\n\npub struct SessionData<T: SessionStream> {\n    pub stream: T,\n    pub local_ip: IpAddr,\n    pub local_port: u16,\n    pub remote_ip: IpAddr,\n    pub remote_port: u16,\n    pub protocol: ServerProtocol,\n    pub session_id: u64,\n    pub in_flight: InFlight,\n    pub instance: Arc<ServerInstance>,\n}\n\npub trait SessionStream: AsyncRead + AsyncWrite + Unpin + 'static + Sync + Send {\n    fn is_tls(&self) -> bool;\n    fn tls_version_and_cipher(&self) -> (Cow<'static, str>, Cow<'static, str>);\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum SessionResult {\n    Continue,\n    Close,\n    UpgradeTls,\n}\n\npub trait SessionManager: Sync + Send + 'static + Clone {\n    fn spawn<T: SessionStream>(\n        &self,\n        mut session: SessionData<T>,\n        is_tls: bool,\n        acme_core: Option<Server>,\n        span_start: EventType,\n        span_end: EventType,\n    ) {\n        let manager = self.clone();\n\n        tokio::spawn(async move {\n            let start_time = Instant::now();\n            let local_port = session.local_port;\n            let session_id;\n\n            if is_tls {\n                match session\n                    .instance\n                    .acceptor\n                    .accept(session.stream, acme_core, &session.instance)\n                    .await\n                {\n                    TcpAcceptorResult::Tls(accept) => match accept.await {\n                        Ok(stream) => {\n                            // Generate sessionId\n                            session.session_id = session.instance.span_id_gen.generate();\n                            session_id = session.session_id;\n\n                            // Send span\n                            Event::with_keys(\n                                span_start,\n                                vec![\n                                    (Key::ListenerId, session.instance.id.clone().into()),\n                                    (Key::LocalPort, session.local_port.into()),\n                                    (Key::RemoteIp, session.remote_ip.into()),\n                                    (Key::RemotePort, session.remote_port.into()),\n                                    (Key::SpanId, session.session_id.into()),\n                                ],\n                            )\n                            .send_with_metrics();\n\n                            manager\n                                .handle(SessionData {\n                                    stream,\n                                    local_ip: session.local_ip,\n                                    local_port: session.local_port,\n                                    remote_ip: session.remote_ip,\n                                    remote_port: session.remote_port,\n                                    protocol: session.protocol,\n                                    session_id: session.session_id,\n                                    in_flight: session.in_flight,\n                                    instance: session.instance,\n                                })\n                                .await;\n                        }\n                        Err(err) => {\n                            trc::event!(\n                                Tls(trc::TlsEvent::HandshakeError),\n                                ListenerId = session.instance.id.clone(),\n                                LocalPort = local_port,\n                                RemoteIp = session.remote_ip,\n                                RemotePort = session.remote_port,\n                                Reason = err.to_string(),\n                            );\n\n                            return;\n                        }\n                    },\n                    TcpAcceptorResult::Plain(stream) => {\n                        // Generate sessionId\n                        session.session_id = session.instance.span_id_gen.generate();\n                        session_id = session.session_id;\n\n                        // Send span\n                        Event::with_keys(\n                            span_start,\n                            vec![\n                                (Key::ListenerId, session.instance.id.clone().into()),\n                                (Key::LocalPort, session.local_port.into()),\n                                (Key::RemoteIp, session.remote_ip.into()),\n                                (Key::RemotePort, session.remote_port.into()),\n                                (Key::SpanId, session.session_id.into()),\n                            ],\n                        )\n                        .send_with_metrics();\n\n                        session.stream = stream;\n                        manager.handle(session).await;\n                    }\n                    TcpAcceptorResult::Close => return,\n                }\n            } else {\n                // Generate sessionId\n                session.session_id = session.instance.span_id_gen.generate();\n                session_id = session.session_id;\n\n                // Send span\n                Event::with_keys(\n                    span_start,\n                    vec![\n                        (Key::ListenerId, session.instance.id.clone().into()),\n                        (Key::LocalPort, session.local_port.into()),\n                        (Key::RemoteIp, session.remote_ip.into()),\n                        (Key::RemotePort, session.remote_port.into()),\n                        (Key::SpanId, session.session_id.into()),\n                    ],\n                )\n                .send_with_metrics();\n\n                manager.handle(session).await;\n            }\n\n            // End span\n            Event::with_keys(\n                span_end,\n                vec![\n                    (Key::SpanId, session_id.into()),\n                    (Key::Elapsed, start_time.elapsed().into()),\n                ],\n            )\n            .send_with_metrics();\n        });\n    }\n\n    fn handle<T: SessionStream>(\n        self,\n        session: SessionData<T>,\n    ) -> impl std::future::Future<Output = ()> + Send;\n\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send;\n}\n\nimpl<T: SessionStream> ResolveVariable for SessionData<T> {\n    fn resolve_variable(&self, variable: u32) -> crate::expr::Variable<'_> {\n        match variable {\n            V_REMOTE_IP => self.remote_ip.to_compact_string().into(),\n            V_REMOTE_PORT => self.remote_port.into(),\n            V_LOCAL_IP => self.local_ip.to_compact_string().into(),\n            V_LOCAL_PORT => self.local_port.into(),\n            V_LISTENER => self.instance.id.as_str().into(),\n            V_PROTOCOL => self.protocol.as_str().into(),\n            V_TLS => self.stream.is_tls().into(),\n            _ => crate::expr::Variable::default(),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl Debug for TcpAcceptor {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Self::Tls {\n                config, implicit, ..\n            } => f\n                .debug_struct(\"Tls\")\n                .field(\"config\", config)\n                .field(\"implicit\", implicit)\n                .finish(),\n            Self::Plain => write!(f, \"Plain\"),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/stream.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse proxy_header::io::ProxiedStream;\nuse tokio::{\n    io::{AsyncRead, AsyncWrite},\n    net::TcpStream,\n};\nuse tokio_rustls::server::TlsStream;\n\nuse super::SessionStream;\n\nimpl SessionStream for TcpStream {\n    fn is_tls(&self) -> bool {\n        false\n    }\n\n    fn tls_version_and_cipher(&self) -> (Cow<'static, str>, Cow<'static, str>) {\n        (Cow::Borrowed(\"\"), Cow::Borrowed(\"\"))\n    }\n}\n\nimpl<T: SessionStream> SessionStream for TlsStream<T> {\n    fn is_tls(&self) -> bool {\n        true\n    }\n\n    fn tls_version_and_cipher(&self) -> (Cow<'static, str>, Cow<'static, str>) {\n        let (_, conn) = self.get_ref();\n\n        (\n            match conn\n                .protocol_version()\n                .unwrap_or(rustls::ProtocolVersion::Unknown(0))\n            {\n                rustls::ProtocolVersion::SSLv2 => \"SSLv2\",\n                rustls::ProtocolVersion::SSLv3 => \"SSLv3\",\n                rustls::ProtocolVersion::TLSv1_0 => \"TLSv1.0\",\n                rustls::ProtocolVersion::TLSv1_1 => \"TLSv1.1\",\n                rustls::ProtocolVersion::TLSv1_2 => \"TLSv1.2\",\n                rustls::ProtocolVersion::TLSv1_3 => \"TLSv1.3\",\n                rustls::ProtocolVersion::DTLSv1_0 => \"DTLSv1.0\",\n                rustls::ProtocolVersion::DTLSv1_2 => \"DTLSv1.2\",\n                rustls::ProtocolVersion::DTLSv1_3 => \"DTLSv1.3\",\n                _ => \"unknown\",\n            }\n            .into(),\n            match conn.negotiated_cipher_suite() {\n                Some(rustls::SupportedCipherSuite::Tls13(cs)) => {\n                    cs.common.suite.as_str().unwrap_or(\"unknown\")\n                }\n                Some(rustls::SupportedCipherSuite::Tls12(cs)) => {\n                    cs.common.suite.as_str().unwrap_or(\"unknown\")\n                }\n                None => \"unknown\",\n            }\n            .into(),\n        )\n    }\n}\n\nimpl SessionStream for ProxiedStream<TcpStream> {\n    fn is_tls(&self) -> bool {\n        self.proxy_header()\n            .ssl()\n            .is_some_and(|ssl| ssl.client_ssl())\n    }\n\n    fn tls_version_and_cipher(&self) -> (Cow<'static, str>, Cow<'static, str>) {\n        self.proxy_header()\n            .ssl()\n            .map(|ssl| {\n                (\n                    ssl.version().unwrap_or(\"unknown\").to_string().into(),\n                    ssl.cipher().unwrap_or(\"unknown\").to_string().into(),\n                )\n            })\n            .unwrap_or((Cow::Borrowed(\"unknown\"), Cow::Borrowed(\"unknown\")))\n    }\n}\n\n#[derive(Default)]\npub struct NullIo {\n    pub tx_buf: Vec<u8>,\n}\n\nimpl AsyncWrite for NullIo {\n    fn poll_write(\n        mut self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n        buf: &[u8],\n    ) -> std::task::Poll<Result<usize, std::io::Error>> {\n        self.tx_buf.extend_from_slice(buf);\n        std::task::Poll::Ready(Ok(buf.len()))\n    }\n\n    fn poll_flush(\n        self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n    ) -> std::task::Poll<Result<(), std::io::Error>> {\n        std::task::Poll::Ready(Ok(()))\n    }\n\n    fn poll_shutdown(\n        self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n    ) -> std::task::Poll<Result<(), std::io::Error>> {\n        std::task::Poll::Ready(Ok(()))\n    }\n}\n\nimpl AsyncRead for NullIo {\n    fn poll_read(\n        self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n        _buf: &mut tokio::io::ReadBuf<'_>,\n    ) -> std::task::Poll<std::io::Result<()>> {\n        unreachable!()\n    }\n}\n\nimpl SessionStream for NullIo {\n    fn is_tls(&self) -> bool {\n        true\n    }\n\n    fn tls_version_and_cipher(\n        &self,\n    ) -> (\n        std::borrow::Cow<'static, str>,\n        std::borrow::Cow<'static, str>,\n    ) {\n        (\n            std::borrow::Cow::Borrowed(\"\"),\n            std::borrow::Cow::Borrowed(\"\"),\n        )\n    }\n}\n"
  },
  {
    "path": "crates/common/src/listener/tls.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    cmp::Ordering,\n    fmt::{self, Formatter},\n    sync::Arc,\n};\n\nuse ahash::AHashMap;\nuse rustls::{\n    SupportedProtocolVersion,\n    server::{ClientHello, ResolvesServerCert},\n    sign::CertifiedKey,\n    version::{TLS12, TLS13},\n};\nuse tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt};\nuse tokio_rustls::{Accept, LazyConfigAcceptor};\n\nuse crate::{Inner, Server};\n\nuse super::{\n    ServerInstance, SessionStream, TcpAcceptor, TcpAcceptorResult,\n    acme::{\n        AcmeProvider,\n        resolver::{IsTlsAlpnChallenge, build_acme_static_resolver},\n    },\n};\n\npub static TLS13_VERSION: &[&SupportedProtocolVersion] = &[&TLS13];\npub static TLS12_VERSION: &[&SupportedProtocolVersion] = &[&TLS12];\n\n#[derive(Default, Clone)]\npub struct AcmeProviders {\n    pub providers: AHashMap<String, AcmeProvider>,\n}\n\n#[derive(Clone)]\npub struct CertificateResolver {\n    pub inner: Arc<Inner>,\n}\n\nimpl CertificateResolver {\n    pub fn new(inner: Arc<Inner>) -> Self {\n        Self { inner }\n    }\n}\n\nimpl ResolvesServerCert for CertificateResolver {\n    fn resolve(&self, hello: ClientHello<'_>) -> Option<Arc<CertifiedKey>> {\n        self.resolve_certificate(hello.server_name())\n    }\n}\n\nimpl CertificateResolver {\n    pub(crate) fn resolve_certificate(&self, name: Option<&str>) -> Option<Arc<CertifiedKey>> {\n        let certs = self.inner.data.tls_certificates.load();\n\n        name.map_or_else(\n            || certs.get(\"*\"),\n            |name| {\n                certs\n                    .get(name)\n                    .or_else(|| {\n                        // Try with a wildcard certificate\n                        name.split_once('.')\n                            .and_then(|(_, domain)| certs.get(domain))\n                    })\n                    .or_else(|| {\n                        trc::event!(\n                            Tls(trc::TlsEvent::CertificateNotFound),\n                            Hostname = name.to_string(),\n                        );\n                        certs.get(\"*\")\n                    })\n            },\n        )\n        .or_else(|| match certs.len().cmp(&1) {\n            Ordering::Equal => certs.values().next(),\n            Ordering::Greater => {\n                trc::event!(\n                    Tls(trc::TlsEvent::MultipleCertificatesAvailable),\n                    Total = certs.len(),\n                );\n                certs.values().next()\n            }\n            Ordering::Less => {\n                trc::event!(\n                    Tls(trc::TlsEvent::NoCertificatesAvailable),\n                    Total = certs.len(),\n                );\n                self.inner.data.tls_self_signed_cert.as_ref()\n            }\n        })\n        .cloned()\n    }\n}\n\nimpl TcpAcceptor {\n    pub async fn accept<IO>(\n        &self,\n        stream: IO,\n        enable_acme: Option<Server>,\n        instance: &ServerInstance,\n    ) -> TcpAcceptorResult<IO>\n    where\n        IO: SessionStream,\n    {\n        match self {\n            TcpAcceptor::Tls {\n                config,\n                acceptor,\n                implicit,\n            } if *implicit => match enable_acme {\n                None => TcpAcceptorResult::Tls(acceptor.accept(stream)),\n                Some(core) => {\n                    match LazyConfigAcceptor::new(Default::default(), stream).await {\n                        Ok(start_handshake) => {\n                            if core.has_acme_tls_providers()\n                                && start_handshake.client_hello().is_tls_alpn_challenge()\n                            {\n                                let key = match start_handshake.client_hello().server_name() {\n                                    Some(domain) => {\n                                        let key = core.build_acme_certificate(domain).await;\n\n                                        trc::event!(\n                                            Acme(trc::AcmeEvent::ClientSuppliedSni),\n                                            ListenerId = instance.id.clone(),\n                                            Domain = domain.to_string(),\n                                            Result = key.is_some(),\n                                        );\n\n                                        key\n                                    }\n                                    None => {\n                                        trc::event!(\n                                            Acme(trc::AcmeEvent::ClientMissingSni),\n                                            ListenerId = instance.id.clone(),\n                                        );\n\n                                        None\n                                    }\n                                };\n\n                                match start_handshake\n                                    .into_stream(build_acme_static_resolver(key))\n                                    .await\n                                {\n                                    Ok(mut tls) => {\n                                        trc::event!(\n                                            Acme(trc::AcmeEvent::TlsAlpnReceived),\n                                            ListenerId = instance.id.clone(),\n                                        );\n\n                                        let _ = tls.shutdown().await;\n                                    }\n                                    Err(err) => {\n                                        trc::event!(\n                                            Acme(trc::AcmeEvent::TlsAlpnError),\n                                            ListenerId = instance.id.clone(),\n                                            Reason = err.to_string(),\n                                        );\n                                    }\n                                }\n                            } else {\n                                return TcpAcceptorResult::Tls(\n                                    start_handshake.into_stream(config.clone()),\n                                );\n                            }\n                        }\n                        Err(err) => {\n                            trc::event!(\n                                Tls(trc::TlsEvent::HandshakeError),\n                                ListenerId = instance.id.clone(),\n                                Reason = err.to_string(),\n                            );\n                        }\n                    }\n\n                    TcpAcceptorResult::Close\n                }\n            },\n            _ => TcpAcceptorResult::Plain(stream),\n        }\n    }\n\n    pub fn is_tls(&self) -> bool {\n        matches!(self, TcpAcceptor::Tls { .. })\n    }\n}\n\nimpl<IO> TcpAcceptorResult<IO>\nwhere\n    IO: AsyncRead + AsyncWrite + Unpin,\n{\n    pub fn unwrap_tls(self) -> Accept<IO> {\n        match self {\n            TcpAcceptorResult::Tls(accept) => accept,\n            _ => panic!(\"unwrap_tls called on non-TLS acceptor\"),\n        }\n    }\n}\n\nimpl std::fmt::Debug for CertificateResolver {\n    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {\n        f.debug_struct(\"CertificateResolver\").finish()\n    }\n}\n"
  },
  {
    "path": "crates/common/src/manager/backup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::Core;\nuse ahash::AHashSet;\nuse lz4_flex::frame::FrameEncoder;\nuse std::{\n    io::{BufWriter, Write},\n    path::{Path, PathBuf},\n    sync::mpsc::{self, SyncSender},\n};\nuse store::{\n    write::{AnyClass, AnyKey, ValueClass},\n    *,\n};\nuse types::blob_hash::{BLOB_HASH_LEN, BlobHash};\nuse utils::{UnwrapFailure, codec::leb128::Leb128_};\n\npub(super) const MAGIC_MARKER: u8 = 123;\n\n#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq)]\npub(super) enum Family {\n    Data = 0,\n    Directory = 1,\n    Blob = 2,\n    Config = 3,\n    Changelog = 4,\n    Queue = 5,\n    Report = 6,\n    Telemetry = 7,\n    Tasks = 8,\n}\n\ntype TaskHandle = (tokio::task::JoinHandle<()>, std::thread::JoinHandle<()>);\n\n#[derive(Debug, Default, PartialEq, Eq)]\npub struct BackupParams {\n    dest: PathBuf,\n    families: AHashSet<Family>,\n}\n\nimpl Core {\n    pub async fn backup(&self, mut params: BackupParams) {\n        if !params.dest.exists() {\n            std::fs::create_dir_all(&params.dest).failed(\"Failed to create backup directory\");\n        } else if !params.dest.is_dir() {\n            eprintln!(\"Backup destination {:?} is not a directory.\", params.dest);\n            std::process::exit(1);\n        }\n\n        let mut sync_handles = Vec::new();\n        let schema_version = self\n            .storage\n            .data\n            .get_value::<u32>(AnyKey {\n                subspace: SUBSPACE_PROPERTY,\n                key: vec![0u8],\n            })\n            .await\n            .failed(\"Could not retrieve database schema version.\")\n            .failed(\"Could not retrieve database schema version.\");\n\n        if params.families.is_empty() {\n            params.families = [\n                Family::Data,\n                Family::Directory,\n                Family::Blob,\n                Family::Config,\n                Family::Changelog,\n                Family::Queue,\n                Family::Report,\n                Family::Telemetry,\n                Family::Tasks,\n            ]\n            .into_iter()\n            .collect();\n        }\n\n        for subspace in params\n            .families\n            .into_iter()\n            .flat_map(|f| f.subspaces())\n            .copied()\n        {\n            let (async_handle, sync_handle) = if subspace == SUBSPACE_BLOBS {\n                self.backup_blobs(&params.dest, subspace, schema_version)\n            } else {\n                self.backup_subspace(&params.dest, subspace, schema_version)\n            };\n            async_handle.await.failed(\"Task failed\");\n            sync_handles.push(sync_handle);\n        }\n\n        for handle in sync_handles {\n            handle.join().expect(\"Failed to join thread\");\n        }\n    }\n\n    fn backup_blobs(&self, dest: &Path, subspace: u8, schema_version: u32) -> TaskHandle {\n        let store = self.storage.data.clone();\n        let blob_store = self.storage.blob.clone();\n        let (handle, writer) = spawn_writer(\n            dest.join(format!(\"subspace_{}\", char::from(subspace))),\n            subspace,\n            schema_version,\n        );\n        (\n            tokio::spawn(async move {\n                let mut blobs = Vec::new();\n                let mut last_hash = BlobHash::default();\n                store\n                    .iterate(\n                        IterateParams::new(\n                            AnyKey {\n                                subspace: SUBSPACE_BLOB_LINK,\n                                key: vec![0u8],\n                            },\n                            AnyKey {\n                                subspace: SUBSPACE_BLOB_LINK,\n                                key: vec![u8::MAX; 32],\n                            },\n                        )\n                        .no_values(),\n                        |key, _| {\n                            let hash = BlobHash::try_from_hash_slice(\n                                key.get(0..BLOB_HASH_LEN).ok_or_else(|| {\n                                    trc::Error::corrupted_key(key, None, trc::location!())\n                                })?,\n                            )\n                            .unwrap();\n\n                            if last_hash != hash {\n                                blobs.push(hash.clone());\n                                last_hash = hash;\n                            }\n\n                            Ok(true)\n                        },\n                    )\n                    .await\n                    .failed(\"Failed to iterate over data store\");\n\n                for hash in blobs {\n                    if let Some(blob) = blob_store\n                        .get_blob(hash.as_slice(), 0..usize::MAX)\n                        .await\n                        .failed(\"Failed to get blob\")\n                    {\n                        writer\n                            .send((hash.as_slice().to_vec(), blob))\n                            .failed(\"Failed to send key\");\n                    }\n                }\n            }),\n            handle,\n        )\n    }\n\n    fn backup_subspace(&self, dest: &Path, subspace: u8, schema_version: u32) -> TaskHandle {\n        let store = self.storage.data.clone();\n        let (handle, writer) = spawn_writer(\n            dest.join(format!(\"subspace_{}\", char::from(subspace))),\n            subspace,\n            schema_version,\n        );\n        (\n            tokio::spawn(async move {\n                if !store.is_sql() || (subspace != SUBSPACE_COUNTER && subspace != SUBSPACE_QUOTA) {\n                    store\n                        .iterate(\n                            IterateParams::new(\n                                AnyKey {\n                                    subspace,\n                                    key: vec![0u8],\n                                },\n                                AnyKey {\n                                    subspace,\n                                    key: vec![u8::MAX; 32],\n                                },\n                            )\n                            .set_values(subspace != SUBSPACE_INDEXES),\n                            |key, value| {\n                                writer\n                                    .send((key.to_vec(), value.to_vec()))\n                                    .failed(\"Failed to send key\");\n\n                                Ok(true)\n                            },\n                        )\n                        .await\n                        .failed(\"Failed to iterate over data store\");\n                } else {\n                    let mut keys = Vec::with_capacity(128);\n                    store\n                        .iterate(\n                            IterateParams::new(\n                                AnyKey {\n                                    subspace,\n                                    key: vec![0u8],\n                                },\n                                AnyKey {\n                                    subspace,\n                                    key: vec![u8::MAX; 32],\n                                },\n                            )\n                            .no_values(),\n                            |key, _| {\n                                keys.push(key.to_vec());\n\n                                Ok(true)\n                            },\n                        )\n                        .await\n                        .failed(\"Failed to iterate over data store\");\n\n                    for key in keys {\n                        let counter = store\n                            .get_counter(ValueClass::Any(AnyClass {\n                                subspace,\n                                key: key.clone(),\n                            }))\n                            .await\n                            .failed(\"Failed to get counter\");\n                        writer\n                            .send((key.to_vec(), (counter as u64).to_le_bytes().to_vec()))\n                            .failed(\"Failed to send key\");\n                    }\n                }\n            }),\n            handle,\n        )\n    }\n}\n\n#[allow(clippy::type_complexity)]\nfn spawn_writer(\n    path: PathBuf,\n    subspace: u8,\n    version: u32,\n) -> (std::thread::JoinHandle<()>, SyncSender<(Vec<u8>, Vec<u8>)>) {\n    let (tx, rx) = mpsc::sync_channel::<(Vec<u8>, Vec<u8>)>(10);\n\n    let handle = std::thread::spawn(move || {\n        println!(\"Exporting database to {}.\", path.to_str().unwrap());\n\n        let mut file = FrameEncoder::new(BufWriter::new(\n            std::fs::File::create(path).failed(\"Failed to create backup file\"),\n        ));\n        file.write_all(&[MAGIC_MARKER, subspace])\n            .failed(\"Failed to write version\");\n        file.write_all(&version.to_le_bytes())\n            .failed(\"Failed to write version\");\n\n        while let Ok((key, value)) = rx.recv() {\n            key.len()\n                .to_leb128_writer(&mut file)\n                .failed(\"Failed to write key value\");\n            file.write_all(&key).failed(\"Failed to write key\");\n            value\n                .len()\n                .to_leb128_writer(&mut file)\n                .failed(\"Failed to write key value\");\n            if !value.is_empty() {\n                file.write_all(&value).failed(\"Failed to write key value\");\n            }\n        }\n\n        file.flush().failed(\"Failed to flush backup file\");\n    });\n\n    (handle, tx)\n}\n\nimpl BackupParams {\n    pub fn new(dest: PathBuf) -> Self {\n        let mut params = Self {\n            dest,\n            families: AHashSet::new(),\n        };\n\n        if let Ok(families) = std::env::var(\"EXPORT_TYPES\") {\n            params.parse_families(&families);\n        }\n\n        params\n    }\n\n    fn parse_families(&mut self, families: &str) {\n        for family in families.split(',') {\n            let family = family.trim();\n            match Family::parse(family) {\n                Ok(family) => {\n                    self.families.insert(family);\n                }\n                Err(err) => {\n                    eprintln!(\"Backup failed: {err}.\");\n                    std::process::exit(1);\n                }\n            }\n        }\n    }\n}\n\nimpl Family {\n    pub fn subspaces(&self) -> &'static [u8] {\n        match self {\n            Family::Data => &[\n                SUBSPACE_ACL,\n                SUBSPACE_INDEXES,\n                SUBSPACE_QUOTA,\n                SUBSPACE_COUNTER,\n                SUBSPACE_PROPERTY,\n            ],\n            Family::Directory => &[SUBSPACE_DIRECTORY],\n            Family::Blob => &[SUBSPACE_BLOBS, SUBSPACE_BLOB_EXTRA, SUBSPACE_BLOB_LINK],\n            Family::Config => &[SUBSPACE_SETTINGS],\n            Family::Changelog => &[SUBSPACE_LOGS],\n            Family::Queue => &[SUBSPACE_QUEUE_MESSAGE, SUBSPACE_QUEUE_EVENT],\n            Family::Report => &[SUBSPACE_REPORT_OUT, SUBSPACE_REPORT_IN],\n            Family::Telemetry => &[SUBSPACE_TELEMETRY_SPAN, SUBSPACE_TELEMETRY_METRIC],\n            Family::Tasks => &[SUBSPACE_TASK_QUEUE],\n        }\n    }\n\n    pub fn parse(family: &str) -> Result<Self, String> {\n        match family {\n            \"data\" => Ok(Family::Data),\n            \"directory\" => Ok(Family::Directory),\n            \"blob\" => Ok(Family::Blob),\n            \"config\" => Ok(Family::Config),\n            \"changelog\" => Ok(Family::Changelog),\n            \"queue\" => Ok(Family::Queue),\n            \"report\" => Ok(Family::Report),\n            \"telemetry\" => Ok(Family::Telemetry),\n            \"tasks\" => Ok(Family::Tasks),\n            _ => Err(format!(\"Unknown family {}\", family)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/manager/boot.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    WEBADMIN_KEY,\n    backup::BackupParams,\n    config::{ConfigManager, Patterns},\n    console::store_console,\n};\nuse crate::{\n    Caches, Core, Data, IPC_CHANNEL_BUFFER, Inner, Ipc,\n    config::{network::AsnGeoLookupConfig, server::Listeners, telemetry::Telemetry},\n    core::BuildServer,\n    ipc::{\n        BroadcastEvent, HousekeeperEvent, PushEvent, QueueEvent, ReportingEvent,\n        TrainTaskController,\n    },\n};\nuse arc_swap::ArcSwap;\nuse pwhash::sha512_crypt;\nuse std::{\n    net::{IpAddr, Ipv4Addr},\n    path::PathBuf,\n    sync::Arc,\n};\nuse store::{\n    Stores,\n    rand::{Rng, distr::Alphanumeric, rng},\n};\nuse tokio::sync::{Notify, mpsc};\nuse utils::{\n    UnwrapFailure,\n    config::{Config, ConfigKey},\n    failed,\n};\n\npub struct BootManager {\n    pub config: Config,\n    pub inner: Arc<Inner>,\n    pub servers: Listeners,\n    pub ipc_rxs: IpcReceivers,\n}\n\npub struct IpcReceivers {\n    pub push_rx: Option<mpsc::Receiver<PushEvent>>,\n    pub housekeeper_rx: Option<mpsc::Receiver<HousekeeperEvent>>,\n    pub queue_rx: Option<mpsc::Receiver<QueueEvent>>,\n    pub report_rx: Option<mpsc::Receiver<ReportingEvent>>,\n    pub broadcast_rx: Option<mpsc::Receiver<BroadcastEvent>>,\n}\n\nconst HELP: &str = concat!(\n    \"Stalwart Server v\",\n    env!(\"CARGO_PKG_VERSION\"),\n    r#\"\n\nUsage: stalwart [OPTIONS]\n\nOptions:\n  -c, --config <PATH>              Start server with the specified configuration file\n  -e, --export <PATH>              Export all store data to a specific path\n  -i, --import <PATH>              Import store data from a specific path\n  -o, --console                    Open the store console\n  -I, --init <PATH>                Initialize a new server at a specific path\n  -h, --help                       Print help\n  -V, --version                    Print version\n\"#\n);\n\n#[derive(PartialEq, Eq)]\nenum StoreOp {\n    Export(BackupParams),\n    Import(PathBuf),\n    Console,\n    None,\n}\n\npub const DEFAULT_SETTINGS: &[(&str, &str)] = &[\n    (\"queue.quota.size.messages\", \"100000\"),\n    (\"queue.quota.size.size\", \"10737418240\"),\n    (\"queue.quota.size.enable\", \"true\"),\n    (\"queue.limiter.inbound.ip.key\", \"remote_ip\"),\n    (\"queue.limiter.inbound.ip.rate\", \"5/1s\"),\n    (\"queue.limiter.inbound.ip.enable\", \"true\"),\n    (\"queue.limiter.inbound.sender.key.0\", \"sender_domain\"),\n    (\"queue.limiter.inbound.sender.key.1\", \"rcpt\"),\n    (\"queue.limiter.inbound.sender.rate\", \"25/1h\"),\n    (\"queue.limiter.inbound.sender.enable\", \"true\"),\n    (\"report.analysis.addresses\", \"postmaster@*\"),\n    (\"queue.virtual.local.threads-per-node\", \"25\"),\n    (\"queue.virtual.local.description\", \"Local delivery queue\"),\n    (\"queue.virtual.remote.threads-per-node\", \"50\"),\n    (\"queue.virtual.remote.description\", \"Remote delivery queue\"),\n    (\"queue.virtual.dsn.threads-per-node\", \"5\"),\n    (\n        \"queue.virtual.dsn.description\",\n        \"Delivery Status Notification delivery queue\",\n    ),\n    (\"queue.virtual.report.threads-per-node\", \"5\"),\n    (\n        \"queue.virtual.report.description\",\n        \"DMARC and TLS report delivery queue\",\n    ),\n    (\"queue.schedule.local.queue-name\", \"local\"),\n    (\"queue.schedule.local.retry.0\", \"2m\"),\n    (\"queue.schedule.local.retry.1\", \"5m\"),\n    (\"queue.schedule.local.retry.2\", \"10m\"),\n    (\"queue.schedule.local.retry.3\", \"15m\"),\n    (\"queue.schedule.local.retry.4\", \"30m\"),\n    (\"queue.schedule.local.retry.5\", \"1h\"),\n    (\"queue.schedule.local.retry.6\", \"2h\"),\n    (\"queue.schedule.local.notify.0\", \"1d\"),\n    (\"queue.schedule.local.notify.1\", \"3d\"),\n    (\"queue.schedule.local.expire-type\", \"ttl\"),\n    (\"queue.schedule.local.expire\", \"3d\"),\n    (\n        \"queue.schedule.local.description\",\n        \"Local delivery schedule\",\n    ),\n    (\"queue.schedule.remote.queue-name\", \"remote\"),\n    (\"queue.schedule.remote.retry.0\", \"2m\"),\n    (\"queue.schedule.remote.retry.1\", \"5m\"),\n    (\"queue.schedule.remote.retry.2\", \"10m\"),\n    (\"queue.schedule.remote.retry.3\", \"15m\"),\n    (\"queue.schedule.remote.retry.4\", \"30m\"),\n    (\"queue.schedule.remote.retry.5\", \"1h\"),\n    (\"queue.schedule.remote.retry.6\", \"2h\"),\n    (\"queue.schedule.remote.notify.0\", \"1d\"),\n    (\"queue.schedule.remote.notify.1\", \"3d\"),\n    (\"queue.schedule.remote.expire-type\", \"ttl\"),\n    (\"queue.schedule.remote.expire\", \"3d\"),\n    (\n        \"queue.schedule.remote.description\",\n        \"Remote delivery schedule\",\n    ),\n    (\"queue.schedule.dsn.queue-name\", \"dsn\"),\n    (\"queue.schedule.dsn.retry.0\", \"15m\"),\n    (\"queue.schedule.dsn.retry.1\", \"30m\"),\n    (\"queue.schedule.dsn.retry.2\", \"1h\"),\n    (\"queue.schedule.dsn.retry.3\", \"2h\"),\n    (\"queue.schedule.dsn.expire-type\", \"attempts\"),\n    (\"queue.schedule.dsn.max-attempts\", \"10\"),\n    (\n        \"queue.schedule.dsn.description\",\n        \"Delivery Status Notification delivery schedule\",\n    ),\n    (\"queue.schedule.report.queue-name\", \"report\"),\n    (\"queue.schedule.report.retry.0\", \"30m\"),\n    (\"queue.schedule.report.retry.1\", \"1h\"),\n    (\"queue.schedule.report.retry.2\", \"2h\"),\n    (\"queue.schedule.report.expire-type\", \"attempts\"),\n    (\"queue.schedule.report.max-attempts\", \"8\"),\n    (\n        \"queue.schedule.report.description\",\n        \"DMARC and TLS report delivery schedule\",\n    ),\n    (\"queue.tls.invalid-tls.allow-invalid-certs\", \"true\"),\n    (\n        \"queue.tls.invalid-tls.description\",\n        \"Allow invalid TLS certificates\",\n    ),\n    (\"queue.tls.default.allow-invalid-certs\", \"false\"),\n    (\"queue.tls.default.description\", \"Default TLS settings\"),\n    (\"queue.route.local.type\", \"local\"),\n    (\"queue.route.local.description\", \"Local delivery route\"),\n    (\"queue.route.mx.type\", \"mx\"),\n    (\"queue.route.mx.limits.multihomed\", \"2\"),\n    (\"queue.route.mx.limits.mx\", \"5\"),\n    (\"queue.route.mx.ip-lookup\", \"ipv4_then_ipv6\"),\n    (\"queue.route.mx.description\", \"MX delivery route\"),\n    (\"queue.connection.default.timeout.connect\", \"5m\"),\n    (\n        \"queue.connection.default.description\",\n        \"Default connection settings\",\n    ),\n];\n\nimpl BootManager {\n    pub async fn init() -> Self {\n        let mut config_path = std::env::var(\"CONFIG_PATH\").ok();\n        let mut import_export = StoreOp::None;\n\n        if config_path.is_none() {\n            let mut args = std::env::args().skip(1);\n\n            while let Some(arg) = args.next().and_then(|arg| {\n                arg.strip_prefix(\"--\")\n                    .or_else(|| arg.strip_prefix('-'))\n                    .map(|arg| arg.to_string())\n            }) {\n                let (key, value) = if let Some((key, value)) = arg.split_once('=') {\n                    (key.to_string(), Some(value.trim().to_string()))\n                } else {\n                    (arg, args.next())\n                };\n\n                match (key.as_str(), value) {\n                    (\"help\" | \"h\", _) => {\n                        eprintln!(\"{HELP}\");\n                        std::process::exit(0);\n                    }\n                    (\"version\" | \"V\", _) => {\n                        println!(\"{}\", env!(\"CARGO_PKG_VERSION\"));\n                        std::process::exit(0);\n                    }\n                    (\"config\" | \"c\", Some(value)) => {\n                        config_path = Some(value);\n                    }\n                    (\"init\" | \"I\", Some(value)) => {\n                        quickstart(value);\n                        std::process::exit(0);\n                    }\n                    (\"export\" | \"e\", Some(value)) => {\n                        import_export = StoreOp::Export(BackupParams::new(value.into()));\n                    }\n                    (\"import\" | \"i\", Some(value)) => {\n                        import_export = StoreOp::Import(value.into());\n                    }\n                    (\"console\" | \"o\", None) => {\n                        import_export = StoreOp::Console;\n                    }\n                    (_, None) => {\n                        failed(&format!(\"Unrecognized command '{key}', try '--help'.\"));\n                    }\n                    (_, Some(_)) => failed(&format!(\n                        \"Missing value for argument '{key}', try '--help'.\"\n                    )),\n                }\n            }\n\n            if config_path.is_none() {\n                if import_export == StoreOp::None {\n                    eprintln!(\"{HELP}\");\n                } else {\n                    eprintln!(\"Missing '--config' argument for import/export.\")\n                }\n                std::process::exit(0);\n            }\n        }\n\n        // Read main configuration file\n        let cfg_local_path = PathBuf::from(config_path.unwrap());\n        let mut config = Config::default();\n        match std::fs::read_to_string(&cfg_local_path) {\n            Ok(value) => {\n                config.parse(&value).failed(\"Invalid configuration file\");\n            }\n            Err(err) => {\n                config.new_build_error(\"*\", format!(\"Could not read configuration file: {err}\"));\n            }\n        }\n        let cfg_local = config.keys.clone();\n\n        // Resolve environment macros\n        config.resolve_macros(&[\"env\"]).await;\n\n        // Parser servers\n        let mut servers = Listeners::parse(&mut config);\n\n        // Bind ports and drop privileges\n        servers.bind_and_drop_priv(&mut config);\n\n        // Resolve file and configuration macros\n        config.resolve_macros(&[\"file\", \"cfg\"]).await;\n\n        // Load stores\n        let mut stores = Stores::parse(&mut config).await;\n        let local_patterns = Patterns::parse(&mut config);\n\n        // Build local keys and warn about database keys defined in the local configuration\n        let mut warn_keys = Vec::new();\n        for key in config.keys.keys() {\n            if !local_patterns.is_local_key(key) {\n                warn_keys.push(key.clone());\n            }\n        }\n        for warn_key in warn_keys {\n            config.new_build_warning(\n                warn_key,\n                concat!(\n                    \"Database key defined in local configuration, this might cause issues. \",\n                    \"See https://stalw.art/docs/configuration/overview/#loc\",\n                    \"al-and-database-settings\"\n                ),\n            );\n        }\n\n        // Build manager\n        let manager = ConfigManager {\n            cfg_local: ArcSwap::from_pointee(cfg_local),\n            cfg_local_path,\n            cfg_local_patterns: local_patterns.into(),\n            cfg_store: config\n                .value(\"storage.data\")\n                .and_then(|id| stores.stores.get(id))\n                .cloned()\n                .unwrap_or_default(),\n        };\n\n        // Extend configuration with settings stored in the db\n        if !manager.cfg_store.is_none() {\n            for (key, value) in manager\n                .db_list(\"\", false)\n                .await\n                .failed(\"Failed to read database configuration\")\n            {\n                if manager.cfg_local_patterns.is_local_key(&key) {\n                    config.new_build_warning(\n                        &key,\n                        concat!(\n                            \"Local key defined in database, this might cause issues. \",\n                            \"See https://stalw.art/docs/configuration/overview/#loc\",\n                            \"al-and-database-settings\"\n                        ),\n                    );\n                }\n\n                config.keys.entry(key).or_insert(value);\n            }\n        }\n\n        // Parse telemetry\n        let telemetry = Telemetry::parse(&mut config, &stores);\n\n        match import_export {\n            StoreOp::None => {\n                // Add hostname lookup if missing\n                let mut insert_keys = Vec::new();\n\n                // Generate an OAuth key if missing\n                if config\n                    .value(\"oauth.key\")\n                    .filter(|v| !v.is_empty())\n                    .is_none()\n                {\n                    insert_keys.push(ConfigKey::from((\n                        \"oauth.key\",\n                        rng()\n                            .sample_iter(Alphanumeric)\n                            .take(64)\n                            .map(char::from)\n                            .collect::<String>(),\n                    )));\n                }\n\n                // Download Spam filter rules if missing\n                if config.value(\"version.spam-filter\").is_none() {\n                    match manager.fetch_spam_rules().await {\n                        Ok(external_config) => {\n                            trc::event!(\n                                Config(trc::ConfigEvent::ImportExternal),\n                                Version = external_config.version.to_string(),\n                                Id = \"spam-filter\"\n                            );\n                            insert_keys.extend(external_config.keys);\n                        }\n                        Err(err) => {\n                            config.new_build_error(\n                                \"*\",\n                                format!(\"Failed to fetch spam filter: {err}\"),\n                            );\n                        }\n                    }\n\n                    // Add default settings\n                    for key in DEFAULT_SETTINGS {\n                        insert_keys.push(ConfigKey::from(*key));\n                    }\n                }\n\n                // Download webadmin if missing\n                if let Some(blob_store) = config\n                    .value(\"storage.blob\")\n                    .and_then(|id| stores.blob_stores.get(id))\n                {\n                    match blob_store.get_blob(WEBADMIN_KEY, 0..usize::MAX).await {\n                        Ok(Some(_)) => (),\n                        Ok(None) => match manager.fetch_resource(\"webadmin\").await {\n                            Ok(bytes) => match blob_store.put_blob(WEBADMIN_KEY, &bytes).await {\n                                Ok(_) => {\n                                    trc::event!(\n                                        Resource(trc::ResourceEvent::DownloadExternal),\n                                        Id = \"webadmin\"\n                                    );\n                                }\n                                Err(err) => {\n                                    config.new_build_error(\n                                        \"*\",\n                                        format!(\"Failed to store webadmin blob: {err}\"),\n                                    );\n                                }\n                            },\n                            Err(err) => {\n                                config.new_build_error(\n                                    \"*\",\n                                    format!(\"Failed to download webadmin: {err}\"),\n                                );\n                            }\n                        },\n                        Err(err) => config\n                            .new_build_error(\"*\", format!(\"Failed to access webadmin blob: {err}\")),\n                    }\n                }\n\n                // Add missing settings\n                if !insert_keys.is_empty() {\n                    for item in &insert_keys {\n                        config.keys.insert(item.key.clone(), item.value.clone());\n                    }\n\n                    if let Err(err) = manager.set(insert_keys, true).await {\n                        config\n                            .new_build_error(\"*\", format!(\"Failed to update configuration: {err}\"));\n                    }\n                }\n\n                // Parse in-memory stores\n                stores.parse_in_memory(&mut config, false).await;\n\n                // Parse settings\n                let core = Box::pin(Core::parse(&mut config, stores, manager)).await;\n\n                // Parse data\n                let data = Data::parse(&mut config);\n\n                // Parse caches\n                let cache = Caches::parse(&mut config);\n\n                // Enable telemetry\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                telemetry.enable(core.is_enterprise_edition());\n                // SPDX-SnippetEnd\n\n                #[cfg(not(feature = \"enterprise\"))]\n                telemetry.enable(false);\n\n                trc::event!(\n                    Server(trc::ServerEvent::Startup),\n                    Version = env!(\"CARGO_PKG_VERSION\"),\n                );\n\n                // Webadmin auto-update\n                // Disabled temporarily until selective updates are implemented\n                /*if config\n                    .property_or_default::<bool>(\"webadmin.auto-update\", \"false\")\n                    .unwrap_or_default()\n                {\n                    if let Err(err) = data.webadmin.update(&core).await {\n                        trc::event!(\n                            Resource(trc::ResourceEvent::Error),\n                            Details = \"Failed to update webadmin\",\n                            CausedBy = err\n                        );\n                    }\n                }*/\n\n                // Spam filter auto-update\n                if config\n                    .property_or_default::<bool>(\"spam-filter.auto-update\", \"false\")\n                    .unwrap_or_default()\n                    && let Err(err) = core.storage.config.update_spam_rules(false, false).await\n                {\n                    trc::event!(\n                        Resource(trc::ResourceEvent::Error),\n                        Details = \"Failed to update spam-filter\",\n                        CausedBy = err\n                    );\n                }\n\n                // Build shared inner\n                let has_remote_asn = matches!(\n                    core.network.asn_geo_lookup,\n                    AsnGeoLookupConfig::Resource { .. }\n                );\n                let (ipc, ipc_rxs) = build_ipc(!core.storage.pubsub.is_none());\n                let inner = Arc::new(Inner {\n                    shared_core: ArcSwap::from_pointee(core),\n                    data,\n                    ipc,\n                    cache,\n                });\n\n                // Load spam model\n                if let Err(err) = inner.build_server().spam_model_reload().await {\n                    trc::error!(\n                        err.details(\"Failed to load spam filter model\")\n                            .caused_by(trc::location!())\n                    );\n                }\n\n                // Fetch ASN database\n                if has_remote_asn {\n                    inner\n                        .build_server()\n                        .lookup_asn_country(IpAddr::V4(Ipv4Addr::new(8, 8, 8, 8)))\n                        .await;\n                }\n\n                // Parse TCP acceptors\n                servers.parse_tcp_acceptors(&mut config, inner.clone());\n\n                BootManager {\n                    inner,\n                    config,\n                    servers,\n                    ipc_rxs,\n                }\n            }\n            StoreOp::Export(path) => {\n                // Enable telemetry\n                telemetry.enable(false);\n\n                // Parse settings and backup\n                Box::pin(Core::parse(&mut config, stores, manager))\n                    .await\n                    .backup(path)\n                    .await;\n                std::process::exit(0);\n            }\n            StoreOp::Import(path) => {\n                // Enable telemetry\n                telemetry.enable(false);\n\n                // Parse settings and restore\n                Box::pin(Core::parse(&mut config, stores, manager))\n                    .await\n                    .restore(path)\n                    .await;\n                std::process::exit(0);\n            }\n            StoreOp::Console => {\n                // Store console\n                store_console(\n                    Box::pin(Core::parse(&mut config, stores, manager))\n                        .await\n                        .storage\n                        .data,\n                )\n                .await;\n                std::process::exit(0);\n            }\n        }\n    }\n}\n\npub fn build_ipc(has_pubsub: bool) -> (Ipc, IpcReceivers) {\n    // Build ipc receivers\n    let (push_tx, push_rx) = mpsc::channel(IPC_CHANNEL_BUFFER);\n    let (housekeeper_tx, housekeeper_rx) = mpsc::channel(IPC_CHANNEL_BUFFER);\n    let (queue_tx, queue_rx) = mpsc::channel(IPC_CHANNEL_BUFFER);\n    let (report_tx, report_rx) = mpsc::channel(IPC_CHANNEL_BUFFER);\n    let (broadcast_tx, broadcast_rx) = mpsc::channel(IPC_CHANNEL_BUFFER);\n    (\n        Ipc {\n            push_tx,\n            housekeeper_tx,\n            queue_tx,\n            report_tx,\n            broadcast_tx: has_pubsub.then_some(broadcast_tx),\n            task_tx: Arc::new(Notify::new()),\n            train_task_controller: Arc::new(TrainTaskController::default()),\n        },\n        IpcReceivers {\n            push_rx: Some(push_rx),\n            housekeeper_rx: Some(housekeeper_rx),\n            queue_rx: Some(queue_rx),\n            report_rx: Some(report_rx),\n            broadcast_rx: has_pubsub.then_some(broadcast_rx),\n        },\n    )\n}\n\nfn quickstart(path: impl Into<PathBuf>) {\n    let path = path.into();\n\n    if !path.exists() {\n        std::fs::create_dir_all(&path).failed(\"Failed to create directory\");\n    }\n\n    for dir in &[\"etc\", \"data\", \"logs\"] {\n        let sub_path = path.join(dir);\n        if !sub_path.exists() {\n            std::fs::create_dir(sub_path).failed(&format!(\"Failed to create {dir} directory\"));\n        }\n    }\n\n    let admin_pass = std::env::var(\"STALWART_ADMIN_PASSWORD\").unwrap_or_else(|_| {\n        rng()\n            .sample_iter(Alphanumeric)\n            .take(10)\n            .map(char::from)\n            .collect::<String>()\n    });\n\n    std::fs::write(\n        path.join(\"etc\").join(\"config.toml\"),\n        QUICKSTART_CONFIG\n            .replace(\"_P_\", &path.to_string_lossy())\n            .replace(\"_S_\", &sha512_crypt::hash(&admin_pass).unwrap()),\n    )\n    .failed(\"Failed to write configuration file\");\n\n    eprintln!(\n        \"✅ Configuration file written to {}/etc/config.toml\",\n        path.to_string_lossy()\n    );\n    eprintln!(\"🔑 Your administrator account is 'admin' with password '{admin_pass}'.\");\n}\n\n#[cfg(not(feature = \"foundation\"))]\nconst QUICKSTART_CONFIG: &str = r#\"[server.listener.smtp]\nbind = \"[::]:25\"\nprotocol = \"smtp\"\n\n[server.listener.submission]\nbind = \"[::]:587\"\nprotocol = \"smtp\"\n\n[server.listener.submissions]\nbind = \"[::]:465\"\nprotocol = \"smtp\"\ntls.implicit = true\n\n[server.listener.imap]\nbind = \"[::]:143\"\nprotocol = \"imap\"\n\n[server.listener.imaptls]\nbind = \"[::]:993\"\nprotocol = \"imap\"\ntls.implicit = true\n\n[server.listener.pop3]\nbind = \"[::]:110\"\nprotocol = \"pop3\"\n\n[server.listener.pop3s]\nbind = \"[::]:995\"\nprotocol = \"pop3\"\ntls.implicit = true\n\n[server.listener.sieve]\nbind = \"[::]:4190\"\nprotocol = \"managesieve\"\n\n[server.listener.https]\nprotocol = \"http\"\nbind = \"[::]:443\"\ntls.implicit = true\n\n[server.listener.http]\nprotocol = \"http\"\nbind = \"[::]:8080\"\n\n[storage]\ndata = \"rocksdb\"\nfts = \"rocksdb\"\nblob = \"rocksdb\"\nlookup = \"rocksdb\"\ndirectory = \"internal\"\n\n[store.rocksdb]\ntype = \"rocksdb\"\npath = \"_P_/data\"\ncompression = \"lz4\"\n\n[directory.internal]\ntype = \"internal\"\nstore = \"rocksdb\"\n\n[tracer.log]\ntype = \"log\"\nlevel = \"info\"\npath = \"_P_/logs\"\nprefix = \"stalwart.log\"\nrotate = \"daily\"\nansi = false\nenable = true\n\n[authentication.fallback-admin]\nuser = \"admin\"\nsecret = \"_S_\"\n\"#;\n\n#[cfg(feature = \"foundation\")]\nconst QUICKSTART_CONFIG: &str = r#\"[server.listener.smtp]\nbind = \"[::]:25\"\nprotocol = \"smtp\"\n\n[server.listener.submission]\nbind = \"[::]:587\"\nprotocol = \"smtp\"\n\n[server.listener.submissions]\nbind = \"[::]:465\"\nprotocol = \"smtp\"\ntls.implicit = true\n\n[server.listener.imap]\nbind = \"[::]:143\"\nprotocol = \"imap\"\n\n[server.listener.imaptls]\nbind = \"[::]:993\"\nprotocol = \"imap\"\ntls.implicit = true\n\n[server.listener.pop3]\nbind = \"[::]:110\"\nprotocol = \"pop3\"\n\n[server.listener.pop3s]\nbind = \"[::]:995\"\nprotocol = \"pop3\"\ntls.implicit = true\n\n[server.listener.sieve]\nbind = \"[::]:4190\"\nprotocol = \"managesieve\"\n\n[server.listener.https]\nprotocol = \"http\"\nbind = \"[::]:443\"\ntls.implicit = true\n\n[server.listener.http]\nprotocol = \"http\"\nbind = \"[::]:8080\"\n\n[storage]\ndata = \"foundation-db\"\nfts = \"foundation-db\"\nblob = \"foundation-db\"\nlookup = \"foundation-db\"\ndirectory = \"internal\"\n\n[store.foundation-db]\ntype = \"foundationdb\"\ncompression = \"lz4\"\n\n[directory.internal]\ntype = \"internal\"\nstore = \"foundation-db\"\n\n[tracer.log]\ntype = \"log\"\nlevel = \"info\"\npath = \"_P_/logs\"\nprefix = \"stalwart.log\"\nrotate = \"daily\"\nansi = false\nenable = true\n\n[authentication.fallback-admin]\nuser = \"admin\"\nsecret = \"_S_\"\n\"#;\n"
  },
  {
    "path": "crates/common/src/manager/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    collections::{BTreeMap, btree_map::Entry},\n    path::PathBuf,\n    sync::Arc,\n};\n\nuse ahash::AHashMap;\nuse arc_swap::ArcSwap;\nuse store::{\n    Deserialize, IterateParams, Store, ValueKey,\n    write::{BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::semver::Semver;\nuse utils::{\n    config::{Config, ConfigKey},\n    glob::GlobPattern,\n};\n\n#[derive(Default)]\npub struct ConfigManager {\n    pub cfg_local: ArcSwap<BTreeMap<String, String>>,\n    pub cfg_local_path: PathBuf,\n    pub cfg_local_patterns: Arc<Patterns>,\n    pub cfg_store: Store,\n}\n\n#[derive(Default)]\npub struct Patterns {\n    patterns: Vec<Pattern>,\n}\n\n#[derive(Debug, PartialEq, Eq)]\nenum Pattern {\n    Include(MatchType),\n    Exclude(MatchType),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum MatchType {\n    Equal(String),\n    StartsWith(String),\n    EndsWith(String),\n    Matches(GlobPattern),\n    All,\n}\n\npub(crate) struct ExternalSpamRules {\n    pub version: Semver,\n    pub keys: Vec<ConfigKey>,\n}\n\nimpl ConfigManager {\n    pub async fn build_config(&self, prefix: &str) -> trc::Result<Config> {\n        let mut config = Config {\n            keys: self.cfg_local.load().as_ref().clone(),\n            ..Default::default()\n        };\n        config.resolve_all_macros().await;\n        self.extend_config(&mut config, prefix)\n            .await\n            .map(|_| config)\n    }\n\n    pub(crate) async fn extend_config(&self, config: &mut Config, prefix: &str) -> trc::Result<()> {\n        for (key, value) in self.db_list(prefix, false).await? {\n            config.keys.entry(key).or_insert(value);\n        }\n\n        Ok(())\n    }\n\n    pub async fn get(&self, key: impl AsRef<str>) -> trc::Result<Option<String>> {\n        let key = key.as_ref();\n        match self.cfg_local.load().get(key) {\n            Some(value) => Ok(Some(value.to_string())),\n            None => {\n                self.cfg_store\n                    .get_value(ValueKey::from(ValueClass::Config(\n                        key.to_string().into_bytes(),\n                    )))\n                    .await\n            }\n        }\n    }\n\n    pub async fn list(\n        &self,\n        prefix: &str,\n        strip_prefix: bool,\n    ) -> trc::Result<BTreeMap<String, String>> {\n        let mut results = self.db_list(prefix, strip_prefix).await?;\n        for (key, value) in self.cfg_local.load().iter() {\n            if prefix.is_empty() || (!strip_prefix && key.starts_with(prefix)) {\n                results.insert(key.clone(), value.clone());\n            } else if let Some(key) = key.strip_prefix(prefix) {\n                results.insert(key.to_string(), value.clone());\n            }\n        }\n\n        Ok(results)\n    }\n\n    pub async fn group(\n        &self,\n        prefix: &str,\n        suffix: &str,\n    ) -> trc::Result<AHashMap<String, AHashMap<String, String>>> {\n        let mut grouped = AHashMap::new();\n\n        let mut list = self.list(prefix, true).await?;\n        for key in list.keys() {\n            if let Some(key) = key.strip_suffix(suffix) {\n                grouped.insert(key.to_string(), AHashMap::new());\n            }\n        }\n\n        for (name, entries) in &mut grouped {\n            let prefix = format!(\"{name}.\");\n            for (key, value) in &mut list {\n                if let Some(key) = key.strip_prefix(&prefix) {\n                    entries.insert(key.to_string(), std::mem::take(value));\n                }\n            }\n        }\n\n        Ok(grouped)\n    }\n\n    pub async fn db_list(\n        &self,\n        prefix: &str,\n        strip_prefix: bool,\n    ) -> trc::Result<BTreeMap<String, String>> {\n        let key = prefix.as_bytes();\n        let from_key = ValueKey::from(ValueClass::Config(key.to_vec()));\n        let to_key = ValueKey::from(ValueClass::Config(\n            key.iter()\n                .copied()\n                .chain([u8::MAX, u8::MAX, u8::MAX, u8::MAX, u8::MAX])\n                .collect::<Vec<_>>(),\n        ));\n        let mut results = BTreeMap::new();\n        let patterns = self.cfg_local_patterns.clone();\n        self.cfg_store\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending(),\n                |key, value| {\n                    let mut key = std::str::from_utf8(key).map_err(|_| {\n                        trc::Error::corrupted_key(key, value.into(), trc::location!())\n                    })?;\n\n                    if !patterns.is_local_key(key) {\n                        if strip_prefix && !prefix.is_empty() {\n                            key = key.strip_prefix(prefix).unwrap_or(key);\n                        }\n\n                        results.insert(key.to_string(), String::deserialize(value)?);\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(results)\n    }\n\n    pub async fn set<I, T>(&self, keys: I, overwrite: bool) -> trc::Result<()>\n    where\n        I: IntoIterator<Item = T>,\n        T: Into<ConfigKey>,\n    {\n        let mut batch = BatchBuilder::new();\n        let mut local_batch = Vec::new();\n\n        for key in keys {\n            let key = key.into();\n\n            if overwrite || self.get(&key.key).await?.is_none() || key.key.starts_with(\"version.\") {\n                if self.cfg_local_patterns.is_local_key(&key.key) {\n                    local_batch.push(key);\n                } else {\n                    batch.set(ValueClass::Config(key.key.into_bytes()), key.value);\n                }\n            }\n        }\n\n        if !batch.is_empty() {\n            self.cfg_store.write(batch.build_all()).await?;\n        }\n\n        if !local_batch.is_empty() {\n            let mut local = self.cfg_local.load().as_ref().clone();\n            let mut has_changes = false;\n\n            for key in local_batch {\n                match local.entry(key.key) {\n                    Entry::Vacant(v) => {\n                        v.insert(key.value);\n                        has_changes = true;\n                    }\n                    Entry::Occupied(mut v) => {\n                        if v.get() != &key.value {\n                            v.insert(key.value);\n                            has_changes = true;\n                        }\n                    }\n                }\n            }\n            if has_changes {\n                self.update_local(local).await?;\n            }\n        }\n\n        Ok(())\n    }\n\n    pub async fn clear(&self, key: impl AsRef<str>) -> trc::Result<()> {\n        let key = key.as_ref();\n\n        if self.cfg_local_patterns.is_local_key(key) {\n            let mut local = self.cfg_local.load().as_ref().clone();\n            if local.remove(key).is_some() {\n                self.update_local(local).await\n            } else {\n                Ok(())\n            }\n        } else {\n            let mut batch = BatchBuilder::new();\n            batch.clear(ValueClass::Config(key.to_string().into_bytes()));\n            self.cfg_store.write(batch.build_all()).await.map(|_| ())\n        }\n    }\n\n    pub async fn clear_prefix(&self, key: impl AsRef<str>) -> trc::Result<()> {\n        let key = key.as_ref();\n\n        // Delete local keys\n        let local = self.cfg_local.load();\n        if local.keys().any(|k| k.starts_with(key)) {\n            let mut local = local.as_ref().clone();\n            local.retain(|k, _| !k.starts_with(key));\n            self.update_local(local).await?;\n        }\n\n        // Delete db keys\n        self.cfg_store\n            .delete_range(\n                ValueKey::from(ValueClass::Config(key.as_bytes().to_vec())),\n                ValueKey::from(ValueClass::Config(\n                    key.as_bytes()\n                        .iter()\n                        .copied()\n                        .chain([u8::MAX, u8::MAX, u8::MAX, u8::MAX, u8::MAX])\n                        .collect::<Vec<_>>(),\n                )),\n            )\n            .await\n    }\n\n    async fn update_local(&self, map: BTreeMap<String, String>) -> trc::Result<()> {\n        let mut cfg_text = String::with_capacity(1024);\n        for (key, value) in &map {\n            cfg_text.push_str(key);\n            cfg_text.push_str(\" = \");\n            if value == \"true\" || value == \"false\" || value.parse::<f64>().is_ok() {\n                cfg_text.push_str(value);\n            } else {\n                let mut needs_escape = false;\n                let mut has_lf = false;\n\n                for ch in value.chars() {\n                    match ch {\n                        '\"' | '\\\\' => {\n                            needs_escape = true;\n                            if has_lf {\n                                break;\n                            }\n                        }\n                        '\\n' => {\n                            has_lf = true;\n                            if needs_escape {\n                                break;\n                            }\n                        }\n                        _ => {}\n                    }\n                }\n\n                if has_lf || (value.len() > 50 && needs_escape) {\n                    cfg_text.push_str(\"'''\");\n                    cfg_text.push_str(value);\n                    cfg_text.push_str(\"'''\");\n                } else {\n                    cfg_text.push('\"');\n                    if needs_escape {\n                        for ch in value.chars() {\n                            if ch == '\\\\' || ch == '\"' {\n                                cfg_text.push('\\\\');\n                            }\n                            cfg_text.push(ch);\n                        }\n                    } else {\n                        cfg_text.push_str(value);\n                    }\n                    cfg_text.push('\"');\n                }\n            }\n            cfg_text.push('\\n');\n        }\n\n        self.cfg_local.store(map.into());\n\n        tokio::fs::write(&self.cfg_local_path, cfg_text)\n            .await\n            .map_err(|err| {\n                trc::EventType::Config(trc::ConfigEvent::WriteError)\n                    .reason(err)\n                    .details(\"Failed to write local configuration\")\n                    .ctx(trc::Key::Path, self.cfg_local_path.display().to_string())\n            })\n    }\n\n    pub async fn update_spam_rules(\n        &self,\n        force_update: bool,\n        overwrite: bool,\n    ) -> trc::Result<Option<Semver>> {\n        let current_version = self\n            .get(\"version.spam-filter\")\n            .await?\n            .and_then(|v| Semver::try_from(v.as_str()).ok());\n        let is_update = current_version.is_some();\n\n        let mut external = self.fetch_spam_rules().await.map_err(|reason| {\n            trc::EventType::Config(trc::ConfigEvent::FetchError)\n                .caused_by(trc::location!())\n                .details(\"Failed to update spam filter rules\")\n                .ctx(trc::Key::Reason, reason)\n        })?;\n\n        if current_version.is_none_or(|v| external.version > v || force_update) {\n            if is_update {\n                // Delete previous STWT_* rules\n                let mut rule_settings = AHashMap::new();\n                for prefix in [\n                    \"spam-filter.rule.stwt_\",\n                    \"spam-filter.dnsbl.server.stwt_\",\n                    \"http-lookup.stwt_\",\n                ] {\n                    for (key, value) in self.list(prefix, false).await? {\n                        if key.ends_with(\".enable\") {\n                            rule_settings.insert(key, value);\n                        }\n                    }\n\n                    self.clear_prefix(prefix).await?;\n                }\n\n                // Update keys\n                if !rule_settings.is_empty() {\n                    for key in &mut external.keys {\n                        if let Some(value) = rule_settings.remove(&key.key) {\n                            key.value = value;\n                        }\n                    }\n                }\n\n                if !overwrite {\n                    // Do not overwrite ASN or LLM settings\n                    external.keys.retain(|key| {\n                        !key.key.starts_with(\"spam-filter.llm.\") && !key.key.starts_with(\"asn.\")\n                    });\n                }\n            }\n\n            self.set(external.keys, overwrite).await?;\n\n            trc::event!(\n                Config(trc::ConfigEvent::ImportExternal),\n                Version = external.version.to_string(),\n                Id = \"spam-filter\",\n            );\n\n            Ok(Some(external.version))\n        } else {\n            trc::event!(\n                Config(trc::ConfigEvent::AlreadyUpToDate),\n                Version = external.version.to_string(),\n                Id = \"spam-filter\",\n            );\n\n            Ok(None)\n        }\n    }\n\n    pub(crate) async fn fetch_spam_rules(&self) -> Result<ExternalSpamRules, String> {\n        let config = String::from_utf8(self.fetch_resource(\"spam-filter\").await?)\n            .map_err(|err| format!(\"Configuration file has invalid UTF-8: {err}\"))?;\n        let config = Config::new(config)\n            .map_err(|err| format!(\"Failed to parse external configuration: {err}\"))?;\n\n        // Import configuration\n        let mut external = ExternalSpamRules {\n            version: Semver::default(),\n            keys: Vec::new(),\n        };\n        let mut required_semver = Semver::default();\n        let server_semver: Semver = env!(\"CARGO_PKG_VERSION\").try_into().unwrap();\n        for (key, value) in config.keys {\n            if key == \"version.spam-filter\" {\n                external.version = value.as_str().try_into().unwrap_or_default();\n                external.keys.push(ConfigKey::from((key, value)));\n            } else if key == \"version.server\" {\n                required_semver = value.as_str().try_into().unwrap_or_default();\n            } else if key.starts_with(\"spam-filter.\")\n                || key.starts_with(\"http-lookup.\")\n                || key.starts_with(\"lookup.\")\n                || key.starts_with(\"asn.\")\n            {\n                external.keys.push(ConfigKey::from((key, value)));\n            }\n        }\n\n        if !required_semver.is_valid() {\n            Err(\"External spam filter rules do not contain a valid server version\".to_string())\n        } else if required_semver > server_semver {\n            Err(format!(\n                \"External spam filter rules require server version {required_semver}, but this is version {server_semver}\",\n            ))\n        } else if external.version.is_valid() {\n            Ok(external)\n        } else {\n            Err(\"External spam filter rules do not contain a version key\".to_string())\n        }\n    }\n\n    pub async fn get_services(&self) -> trc::Result<Vec<(String, u16, bool)>> {\n        let mut result = Vec::new();\n\n        for listener in self\n            .group(\"server.listener.\", \".protocol\")\n            .await\n            .unwrap_or_default()\n            .into_values()\n        {\n            let is_tls = listener\n                .get(\"tls.implicit\")\n                .is_some_and(|tls| tls == \"true\");\n            let protocol = listener\n                .get(\"protocol\")\n                .map(|s| s.as_str())\n                .unwrap_or_default();\n            let port = listener\n                .get(\"bind\")\n                .or_else(|| {\n                    listener.iter().find_map(|(key, value)| {\n                        if key.starts_with(\"bind.\") {\n                            Some(value)\n                        } else {\n                            None\n                        }\n                    })\n                })\n                .and_then(|s| s.rsplit_once(':').and_then(|(_, p)| p.parse::<u16>().ok()))\n                .unwrap_or_default();\n\n            if port > 0 {\n                result.push((protocol.to_string(), port, is_tls));\n            }\n        }\n\n        // Sort by name, then tls and finally port\n        result.sort_unstable_by(|a, b| {\n            a.0.cmp(&b.0)\n                .then_with(|| b.2.cmp(&a.2))\n                .then_with(|| a.1.cmp(&b.1))\n        });\n\n        Ok(result)\n    }\n}\n\nimpl Patterns {\n    pub fn parse(config: &mut Config) -> Self {\n        let mut cfg_local_patterns = Vec::new();\n        for (key, value) in &config.keys {\n            if !key.starts_with(\"config.local-keys\") {\n                if cfg_local_patterns.is_empty() {\n                    continue;\n                } else {\n                    break;\n                }\n            };\n            let value = value.trim();\n            let (value, is_include) = value\n                .strip_prefix('!')\n                .map_or((value, true), |value| (value, false));\n            let value = value.trim().to_ascii_lowercase();\n            if value.is_empty() {\n                continue;\n            }\n            let match_type = MatchType::parse(&value);\n\n            cfg_local_patterns.push(if is_include {\n                Pattern::Include(match_type)\n            } else {\n                Pattern::Exclude(match_type)\n            });\n        }\n\n        if cfg_local_patterns.is_empty() {\n            cfg_local_patterns = vec![\n                Pattern::Include(MatchType::StartsWith(\"store.\".to_string())),\n                Pattern::Include(MatchType::StartsWith(\"directory.\".to_string())),\n                Pattern::Include(MatchType::StartsWith(\"tracer.\".to_string())),\n                Pattern::Exclude(MatchType::StartsWith(\"server.blocked-ip.\".to_string())),\n                Pattern::Exclude(MatchType::StartsWith(\"server.allowed-ip.\".to_string())),\n                Pattern::Include(MatchType::StartsWith(\"server.\".to_string())),\n                Pattern::Include(MatchType::StartsWith(\"certificate.\".to_string())),\n                Pattern::Include(MatchType::StartsWith(\"config.local-keys.\".to_string())),\n                Pattern::Include(MatchType::StartsWith(\n                    \"authentication.fallback-admin.\".to_string(),\n                )),\n                Pattern::Include(MatchType::StartsWith(\"cluster.\".to_string())),\n                Pattern::Include(MatchType::Equal(\"storage.data\".to_string())),\n                Pattern::Include(MatchType::Equal(\"storage.blob\".to_string())),\n                Pattern::Include(MatchType::Equal(\"storage.lookup\".to_string())),\n                Pattern::Include(MatchType::Equal(\"storage.fts\".to_string())),\n                Pattern::Include(MatchType::Equal(\"storage.directory\".to_string())),\n                Pattern::Include(MatchType::Equal(\"enterprise.license-key\".to_string())),\n            ];\n        } else if !cfg_local_patterns.contains(&Pattern::Include(MatchType::StartsWith(\n            \"config.local-keys.\".to_string(),\n        ))) {\n            cfg_local_patterns.push(Pattern::Include(MatchType::StartsWith(\n                \"config.local-keys.\".to_string(),\n            )));\n        }\n\n        Patterns {\n            patterns: cfg_local_patterns,\n        }\n    }\n\n    pub fn is_local_key(&self, key: &str) -> bool {\n        let mut is_local = false;\n\n        for pattern in &self.patterns {\n            match pattern {\n                Pattern::Include(pattern) => {\n                    if !is_local && pattern.matches(key) {\n                        is_local = true;\n                    }\n                }\n                Pattern::Exclude(pattern) => {\n                    if pattern.matches(key) {\n                        return false;\n                    }\n                }\n            }\n        }\n\n        is_local\n    }\n}\n\nimpl MatchType {\n    pub fn parse(value: &str) -> Self {\n        if value == \"*\" {\n            MatchType::All\n        } else if let Some(value) = value.strip_suffix('*') {\n            MatchType::StartsWith(value.to_string())\n        } else if let Some(value) = value.strip_prefix('*') {\n            MatchType::EndsWith(value.to_string())\n        } else if value.contains('*') {\n            MatchType::Matches(GlobPattern::compile(value, false))\n        } else {\n            MatchType::Equal(value.to_string())\n        }\n    }\n\n    pub fn matches(&self, value: &str) -> bool {\n        match self {\n            MatchType::Equal(pattern) => value == pattern,\n            MatchType::StartsWith(pattern) => value.starts_with(pattern),\n            MatchType::EndsWith(pattern) => value.ends_with(pattern),\n            MatchType::Matches(pattern) => pattern.matches(value),\n            MatchType::All => true,\n        }\n    }\n}\n\nimpl Clone for ConfigManager {\n    fn clone(&self) -> Self {\n        Self {\n            cfg_local: ArcSwap::from_pointee(self.cfg_local.load().as_ref().clone()),\n            cfg_local_path: self.cfg_local_path.clone(),\n            cfg_local_patterns: self.cfg_local_patterns.clone(),\n            cfg_store: self.cfg_store.clone(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/manager/console.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse base64::Engine;\nuse base64::engine::general_purpose;\nuse std::env;\nuse std::io::{self, Write};\nuse store::write::{AnyClass, AnyKey, BatchBuilder, ValueClass};\nuse store::{Deserialize, IterateParams, SUBSPACE_INDEXES, Store};\n\nconst HELP: &str = concat!(\n    \"Stalwart Server v\",\n    env!(\"CARGO_PKG_VERSION\"),\n    r#\" Data Store CLI\n\nEnter commands (type 'help' for available commands).\n\"#\n);\n\npub async fn store_console(store: Store) {\n    print!(\"{HELP}\");\n\n    if matches!(store, Store::None) {\n        println!(\"No store available. Verify your configuration.\");\n        return;\n    }\n\n    loop {\n        print!(\"> \");\n        io::stdout().flush().unwrap();\n\n        let mut input = String::new();\n        io::stdin().read_line(&mut input).unwrap();\n        let input = input.trim();\n\n        let parts: Vec<&str> = input.split_whitespace().collect();\n\n        if parts.is_empty() {\n            continue;\n        }\n\n        match parts[0] {\n            \"scan\" => {\n                if parts.len() != 3 {\n                    println!(\"Usage: scan <from_key> <to_key>\");\n                } else if let (Some(from_key), Some(to_key)) =\n                    (parse_key(parts[1]), parse_key(parts[2]))\n                {\n                    println!(\"Scanning from {:?} to {:?}\", from_key, to_key);\n                    let mut from_key = from_key.into_iter();\n                    let mut to_key = to_key.into_iter();\n                    let from_subspace = from_key.next().unwrap();\n                    let to_subspace = to_key.next().unwrap();\n\n                    if from_subspace != to_subspace {\n                        println!(\"Keys must be in the same subspace.\");\n                        return;\n                    }\n\n                    store\n                        .iterate(\n                            IterateParams::new(\n                                AnyKey {\n                                    subspace: from_subspace,\n                                    key: from_key.collect::<Vec<_>>(),\n                                },\n                                AnyKey {\n                                    subspace: to_subspace,\n                                    key: to_key.collect::<Vec<_>>(),\n                                },\n                            )\n                            .set_values(![SUBSPACE_INDEXES].contains(&from_subspace)),\n                            |key, value| {\n                                print!(\"{}\", char::from(from_subspace));\n                                print_escaped(key);\n                                print!(\" : \");\n                                print_escaped(value);\n                                println!();\n                                Ok(true)\n                            },\n                        )\n                        .await\n                        .expect(\"Failed to scan keys\");\n                }\n            }\n            \"delete\" => match (parts.get(1), parts.get(2)) {\n                (Some(from_key), Some(to_key)) => {\n                    if let (Some(from_key), Some(to_key)) = (parse_key(from_key), parse_key(to_key))\n                    {\n                        let mut from_key = from_key.into_iter();\n                        let mut to_key = to_key.into_iter();\n\n                        let from_key = AnyKey {\n                            subspace: from_key.next().unwrap(),\n                            key: from_key.collect::<Vec<_>>(),\n                        };\n                        let to_key = AnyKey {\n                            subspace: to_key.next().unwrap(),\n                            key: to_key.collect::<Vec<_>>(),\n                        };\n\n                        if from_key.subspace != to_key.subspace {\n                            println!(\"Keys must be in the same subspace.\");\n                            return;\n                        }\n\n                        let mut total = 0;\n                        store\n                            .iterate(\n                                IterateParams::new(from_key.clone(), to_key.clone()).no_values(),\n                                |_, _| {\n                                    total += 1;\n                                    Ok(true)\n                                },\n                            )\n                            .await\n                            .expect(\"Failed to scan keys\");\n\n                        if total > 0 {\n                            print!(\"Are you sure you want to delete {total} keys? (y/N): \");\n                            io::stdout().flush().unwrap();\n                            let mut response = String::new();\n                            io::stdin().read_line(&mut response).unwrap();\n                            if !response.trim().eq_ignore_ascii_case(\"y\") {\n                                println!(\"Aborted.\");\n                                return;\n                            }\n\n                            store\n                                .delete_range(from_key, to_key)\n                                .await\n                                .expect(\"Failed to delete keys\");\n                            println!(\"Deleted {total} keys.\");\n                        } else {\n                            println!(\"No keys found.\");\n                        }\n                    }\n                }\n                (Some(key), None) => {\n                    if let Some(key) = parse_key(key) {\n                        println!(\"Deleting key: {:?}\", key);\n                        let mut key = key.into_iter();\n                        let mut batch = BatchBuilder::new();\n                        batch.clear(ValueClass::Any(AnyClass {\n                            subspace: key.next().unwrap(),\n                            key: key.collect(),\n                        }));\n                        if let Err(err) = store.write(batch.build_all()).await {\n                            println!(\"Failed to delete key: {}\", err);\n                        }\n                    }\n                }\n                _ => {\n                    println!(\"Usage: delete <from_key> [<to_key>]\");\n                }\n            },\n            \"get\" => {\n                if parts.len() != 2 {\n                    println!(\"Usage: get <key>\");\n                } else if let Some(key) = parse_key(parts[1]) {\n                    let mut key = key.into_iter();\n                    match store\n                        .get_value::<RawValue>(AnyKey {\n                            subspace: key.next().unwrap(),\n                            key: key.collect::<Vec<_>>(),\n                        })\n                        .await\n                    {\n                        Ok(Some(data)) => {\n                            print_escaped(&data.0);\n                            println!();\n                        }\n                        Ok(None) => {\n                            println!(\"Key not found.\");\n                        }\n                        Err(err) => {\n                            println!(\"Failed to retrieve key: {}\", err);\n                        }\n                    }\n                }\n            }\n            \"put\" => {\n                if parts.len() < 2 {\n                    println!(\"Usage: put <key> [<value>]\");\n                } else if let Some(key) = parse_key(parts[1]) {\n                    let value = parts.get(2).map(|v| parse_value(v)).unwrap_or_default();\n                    println!(\"Putting key: {key:?}\");\n\n                    let mut key = key.into_iter();\n                    let mut batch = BatchBuilder::new();\n                    batch.set(\n                        ValueClass::Any(AnyClass {\n                            subspace: key.next().unwrap(),\n                            key: key.collect(),\n                        }),\n                        value,\n                    );\n                    if let Err(err) = store.write(batch.build_all()).await {\n                        println!(\"Failed to insert key: {}\", err);\n                    }\n                }\n            }\n            \"help\" => {\n                print_help();\n            }\n            \"exit\" | \"quit\" => {\n                println!(\"Exiting...\");\n                break;\n            }\n            _ => {\n                println!(\"Unknown command. Type 'help' for available commands.\");\n            }\n        }\n    }\n}\n\nfn parse_key(input: &str) -> Option<Vec<u8>> {\n    let result = if let Some(key) = input.strip_prefix(\"base64:\") {\n        base64_decode(key)\n    } else {\n        parse_binary(input)\n    };\n    if matches!(result.first(), Some(ch) if ch.is_ascii_alphabetic() && ch.is_ascii_lowercase()) {\n        Some(result)\n    } else {\n        println!(\"Invalid key: {result:?}\");\n        None\n    }\n}\n\nfn parse_value(input: &str) -> Vec<u8> {\n    if let Some(key) = input.strip_prefix(\"base64:\") {\n        base64_decode(key)\n    } else {\n        parse_binary(input)\n    }\n}\n\nfn base64_decode(input: &str) -> Vec<u8> {\n    general_purpose::STANDARD\n        .decode(input)\n        .expect(\"Failed to decode base64\")\n}\n\nfn parse_binary(input: &str) -> Vec<u8> {\n    let mut result = Vec::new();\n    let mut chars = input.chars().peekable();\n\n    while let Some(c) = chars.next() {\n        if c == '\\\\' {\n            match chars.next() {\n                Some('x') => {\n                    let hex: String = chars.by_ref().take(2).collect();\n                    if hex.len() == 2 {\n                        if let Ok(byte) = u8::from_str_radix(&hex, 16) {\n                            result.push(byte);\n                        } else {\n                            result.extend_from_slice(b\"\\\\x\");\n                            result.extend_from_slice(hex.as_bytes());\n                        }\n                    } else {\n                        result.push(b'\\\\');\n                        result.push(b'x');\n                        result.extend_from_slice(hex.as_bytes());\n                    }\n                }\n                Some(other) => {\n                    result.push(b'\\\\');\n                    result.push(other as u8);\n                }\n                None => {\n                    result.push(b'\\\\');\n                }\n            }\n        } else {\n            result.push(c as u8);\n        }\n    }\n\n    result\n}\n\nfn print_escaped(bytes: &[u8]) {\n    for ch in bytes {\n        if ch.is_ascii() && !ch.is_ascii_control() && *ch != b'\\\\' {\n            print!(\"{}\", *ch as char);\n        } else {\n            print!(\"\\\\x{:02x}\", ch);\n        }\n    }\n}\n\nfn print_help() {\n    println!(\"Available commands:\");\n    println!(\"  scan <from_key> <to_key>\");\n    println!(\"  delete <from_key> [<to_key>]\");\n    println!(\"  get <key>\");\n    println!(\"  put <key> [<value>]\");\n    println!(\"  help\");\n    println!(\"  exit/quit\");\n    println!(\"Note: Keys and values can be prefixed with 'base64:' for base64 encoding\");\n    println!(\"      or use escaped hex values (e.g., \\\\x41 for 'A')\");\n}\n\nstruct RawValue(Vec<u8>);\n\nimpl Deserialize for RawValue {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(RawValue(bytes.to_vec()))\n    }\n}\n"
  },
  {
    "path": "crates/common/src/manager/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse self::config::ConfigManager;\nuse crate::USER_AGENT;\nuse hyper::HeaderMap;\nuse std::time::Duration;\nuse utils::HttpLimitResponse;\n\npub mod backup;\npub mod boot;\npub mod config;\npub mod console;\npub mod reload;\npub mod restore;\npub mod webadmin;\n\nconst DEFAULT_SPAMFILTER_URL: &str =\n    \"https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter.toml\";\npub const WEBADMIN_KEY: &[u8] = \"STALWART_WEBADMIN\".as_bytes();\npub const SPAM_TRAINER_KEY: &[u8] = \"STALWART_SPAM_TRAIN_DATA.lz4\".as_bytes();\npub const SPAM_CLASSIFIER_KEY: &[u8] = \"STALWART_SPAM_CLASSIFIER_MODEL.lz4\".as_bytes();\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\nconst DEFAULT_WEBADMIN_URL: &str =\n    \"https://github.com/stalwartlabs/webadmin/releases/latest/download/webadmin.zip\";\n// SPDX-SnippetEnd\n\n#[cfg(not(feature = \"enterprise\"))]\nconst DEFAULT_WEBADMIN_URL: &str =\n    \"https://github.com/stalwartlabs/webadmin/releases/latest/download/webadmin-oss.zip\";\n\nimpl ConfigManager {\n    pub async fn fetch_resource(&self, resource_id: &str) -> Result<Vec<u8>, String> {\n        if let Some(url) = self\n            .get(&format!(\"{resource_id}.resource\"))\n            .await\n            .map_err(|err| {\n                format!(\"Failed to fetch configuration key '{resource_id}.resource': {err}\",)\n            })?\n        {\n            fetch_resource(&url, None, Duration::from_secs(60), MAX_SIZE).await\n        } else {\n            match resource_id {\n                \"spam-filter\" => {\n                    fetch_resource(\n                        DEFAULT_SPAMFILTER_URL,\n                        None,\n                        Duration::from_secs(60),\n                        MAX_SIZE,\n                    )\n                    .await\n                }\n                \"webadmin\" => {\n                    fetch_resource(\n                        DEFAULT_WEBADMIN_URL,\n                        None,\n                        Duration::from_secs(60),\n                        MAX_SIZE,\n                    )\n                    .await\n                }\n                _ => Err(format!(\"Unknown resource: {resource_id}\")),\n            }\n        }\n    }\n}\n\nconst MAX_SIZE: usize = 100 * 1024 * 1024;\n\npub async fn fetch_resource(\n    url: &str,\n    headers: Option<HeaderMap>,\n    timeout: Duration,\n    max_size: usize,\n) -> Result<Vec<u8>, String> {\n    if let Some(path) = url.strip_prefix(\"file://\") {\n        tokio::fs::read(path)\n            .await\n            .map_err(|err| format!(\"Failed to read {path}: {err}\"))\n    } else {\n        let response = reqwest::Client::builder()\n            .timeout(timeout)\n            .danger_accept_invalid_certs(is_localhost_url(url))\n            .user_agent(USER_AGENT)\n            .build()\n            .unwrap_or_default()\n            .get(url)\n            .headers(headers.unwrap_or_default())\n            .send()\n            .await\n            .map_err(|err| format!(\"Failed to fetch {url}: {err}\"))?;\n\n        if response.status().is_success() {\n            response\n                .bytes_with_limit(max_size)\n                .await\n                .map_err(|err| format!(\"Failed to fetch {url}: {err}\"))\n                .and_then(|bytes| bytes.ok_or_else(|| format!(\"Resource too large: {url}\")))\n        } else {\n            let code = response.status().canonical_reason().unwrap_or_default();\n            let reason = response.text().await.unwrap_or_default();\n\n            Err(format!(\n                \"Failed to fetch {url}: Code: {code}, Details: {reason}\",\n            ))\n        }\n    }\n}\n\npub fn is_localhost_url(url: &str) -> bool {\n    url.split_once(\"://\")\n        .map(|(_, url)| url.split_once('/').map_or(url, |(host, _)| host))\n        .is_some_and(|host| {\n            let host = host.rsplit_once(':').map_or(host, |(host, _)| host);\n            host == \"localhost\" || host == \"127.0.0.1\" || host == \"[::1]\"\n        })\n}\n"
  },
  {
    "path": "crates/common/src/manager/reload.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashMap;\nuse arc_swap::ArcSwap;\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::{\n    Core, Server,\n    config::{\n        server::{Listeners, tls::parse_certificates},\n        telemetry::Telemetry,\n    },\n    listener::blocked::{BLOCKED_IP_KEY, BlockedIps},\n};\n\nuse super::config::{ConfigManager, Patterns};\n\npub struct ReloadResult {\n    pub config: Config,\n    pub new_core: Option<Core>,\n    pub tracers: Option<Telemetry>,\n}\n\nimpl Server {\n    pub async fn reload_blocked_ips(&self) -> trc::Result<ReloadResult> {\n        let mut config = self\n            .core\n            .storage\n            .config\n            .build_config(BLOCKED_IP_KEY)\n            .await?;\n        *self.inner.data.blocked_ips.write() = BlockedIps::parse(&mut config).blocked_ip_addresses;\n\n        Ok(config.into())\n    }\n\n    pub async fn reload_certificates(&self) -> trc::Result<ReloadResult> {\n        let mut config = self.core.storage.config.build_config(\"certificate\").await?;\n        let mut certificates = self.inner.data.tls_certificates.load().as_ref().clone();\n\n        parse_certificates(&mut config, &mut certificates, &mut Default::default());\n\n        self.inner.data.tls_certificates.store(certificates.into());\n\n        Ok(config.into())\n    }\n\n    pub async fn reload_lookups(&self) -> trc::Result<ReloadResult> {\n        let mut config = self.core.storage.config.build_config(\"lookup\").await?;\n        let mut stores = Stores::default();\n        stores.parse_static_stores(&mut config, true);\n\n        let mut core = self.core.as_ref().clone();\n        for (id, store) in stores.in_memory_stores {\n            core.storage.lookups.insert(id, store);\n        }\n\n        Ok(ReloadResult {\n            config,\n            new_core: core.into(),\n            tracers: None,\n        })\n    }\n\n    pub async fn reload(&self) -> trc::Result<ReloadResult> {\n        let mut config = self.core.storage.config.build_config(\"\").await?;\n\n        // Load stores\n        let mut stores = Stores {\n            stores: self.core.storage.stores.clone(),\n            blob_stores: self.core.storage.blobs.clone(),\n            search_stores: self.core.storage.ftss.clone(),\n            in_memory_stores: self.core.storage.lookups.clone(),\n            pubsub_stores: Default::default(),\n            purge_schedules: Default::default(),\n        };\n        stores.parse_stores(&mut config).await;\n        stores.parse_in_memory(&mut config, true).await;\n\n        // Parse tracers\n        let tracers = Telemetry::parse(&mut config, &stores);\n\n        if !config.errors.is_empty() {\n            return Ok(config.into());\n        }\n\n        // Build manager\n        let manager = ConfigManager {\n            cfg_local: ArcSwap::from_pointee(\n                self.core.storage.config.cfg_local.load().as_ref().clone(),\n            ),\n            cfg_local_path: self.core.storage.config.cfg_local_path.clone(),\n            cfg_local_patterns: Patterns::parse(&mut config).into(),\n            cfg_store: config\n                .value(\"storage.data\")\n                .and_then(|id| stores.stores.get(id))\n                .cloned()\n                .unwrap_or_default(),\n        };\n\n        // Parse settings and build shared core\n        let core = Box::pin(Core::parse(&mut config, stores, manager)).await;\n        if !config.errors.is_empty() {\n            return Ok(config.into());\n        }\n\n        // Update TLS certificates\n        let mut new_certificates = AHashMap::new();\n        parse_certificates(&mut config, &mut new_certificates, &mut Default::default());\n        let mut current_certificates = self.inner.data.tls_certificates.load().as_ref().clone();\n        for (cert_id, cert) in new_certificates {\n            current_certificates.insert(cert_id, cert);\n        }\n        self.inner\n            .data\n            .tls_certificates\n            .store(current_certificates.into());\n\n        // Update blocked IPs\n        *self.inner.data.blocked_ips.write() = BlockedIps::parse(&mut config).blocked_ip_addresses;\n\n        // Parser servers\n        let mut servers = Listeners::parse(&mut config);\n        servers.parse_tcp_acceptors(&mut config, self.inner.clone());\n\n        Ok(if config.errors.is_empty() {\n            ReloadResult {\n                config,\n                new_core: core.into(),\n                tracers: tracers.into(),\n            }\n        } else {\n            config.into()\n        })\n    }\n}\n\nimpl From<Config> for ReloadResult {\n    fn from(config: Config) -> Self {\n        Self {\n            config,\n            new_core: None,\n            tracers: None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/manager/restore.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::backup::MAGIC_MARKER;\nuse crate::{Core, DATABASE_SCHEMA_VERSION};\nuse lz4_flex::frame::FrameDecoder;\nuse std::{\n    fs::File,\n    io::{BufReader, ErrorKind, Read},\n    path::{Path, PathBuf},\n};\nuse store::{\n    BlobStore, SUBSPACE_BLOBS, SUBSPACE_COUNTER, SUBSPACE_INDEXES, SUBSPACE_QUOTA, Store, U32_LEN,\n    write::{AnyClass, BatchBuilder, ValueClass, key::DeserializeBigEndian},\n};\nuse types::{collection::Collection, field::Field};\nuse utils::{UnwrapFailure, failed};\n\nimpl Core {\n    pub async fn restore(&self, src: PathBuf) {\n        // Backup the core\n        if src.is_dir() {\n            // Iterate directory and spawn a task for each file\n            let mut tasks = Vec::new();\n            for entry in std::fs::read_dir(&src).failed(\"Failed to read directory\") {\n                let entry = entry.failed(\"Failed to read entry\");\n                let path = entry.path();\n                if path.is_file() {\n                    let storage = self.storage.clone();\n                    let blob_store = self.storage.blob.clone();\n                    tasks.push(tokio::spawn(async move {\n                        restore_file(storage.data, blob_store, &path).await;\n                    }));\n                }\n            }\n\n            for task in tasks {\n                task.await.failed(\"Failed to wait for task\");\n            }\n        } else {\n            restore_file(self.storage.data.clone(), self.storage.blob.clone(), &src).await;\n        }\n    }\n}\n\nasync fn restore_file(store: Store, blob_store: BlobStore, path: &Path) {\n    println!(\"Importing database dump from {}.\", path.to_str().unwrap());\n\n    let mut reader = KeyValueReader::new(path);\n    let mut batch = BatchBuilder::new();\n\n    match reader.subspace {\n        SUBSPACE_BLOBS => {\n            while let Some((key, value)) = reader.next() {\n                blob_store\n                    .put_blob(&key, &value)\n                    .await\n                    .failed(\"Failed to write blob\");\n            }\n        }\n        SUBSPACE_COUNTER | SUBSPACE_QUOTA => {\n            while let Some((key, value)) = reader.next() {\n                batch.add(\n                    ValueClass::Any(AnyClass {\n                        subspace: reader.subspace,\n                        key,\n                    }),\n                    u64::from_le_bytes(\n                        value\n                            .try_into()\n                            .expect(\"Failed to deserialize counter/quota\"),\n                    ) as i64,\n                );\n                if batch.is_large_batch() {\n                    store\n                        .write(batch.build_all())\n                        .await\n                        .failed(\"Failed to write batch\");\n                    batch = BatchBuilder::new();\n                }\n            }\n        }\n        SUBSPACE_INDEXES => {\n            while let Some((key, _)) = reader.next() {\n                let account_id = key\n                    .as_slice()\n                    .deserialize_be_u32(0)\n                    .failed(\"Failed to deserialize account ID\");\n                let collection = *key.get(U32_LEN).failed(\"Missing collection byte\");\n                let field = *key.get(U32_LEN + 1).failed(\"Missing field byte\");\n                let value = key\n                    .get(U32_LEN + 2..key.len() - U32_LEN)\n                    .failed(\"Missing index key\")\n                    .to_vec();\n                let document_id = key\n                    .as_slice()\n                    .deserialize_be_u32(key.len() - U32_LEN)\n                    .failed(\"Failed to deserialize document ID\");\n\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::from(collection))\n                    .with_document(document_id)\n                    .index(Field::new(field), value);\n\n                if batch.is_large_batch() {\n                    store\n                        .write(batch.build_all())\n                        .await\n                        .failed(\"Failed to write batch\");\n                    batch = BatchBuilder::new();\n                }\n            }\n        }\n        _ => {\n            while let Some((key, value)) = reader.next() {\n                batch.set(\n                    ValueClass::Any(AnyClass {\n                        subspace: reader.subspace,\n                        key,\n                    }),\n                    value,\n                );\n                if batch.is_large_batch() {\n                    store\n                        .write(batch.build_all())\n                        .await\n                        .failed(\"Failed to write batch\");\n                    batch = BatchBuilder::new();\n                }\n            }\n        }\n    }\n\n    if !batch.is_empty() {\n        store\n            .write(batch.build_all())\n            .await\n            .failed(\"Failed to write batch\");\n    }\n}\n\nstruct KeyValueReader {\n    subspace: u8,\n    file: FrameDecoder<BufReader<File>>,\n}\n\nimpl KeyValueReader {\n    fn new(path: &Path) -> Self {\n        let mut file = FrameDecoder::new(BufReader::new(\n            File::open(path).failed(\"Failed to open file\"),\n        ));\n        let mut buf = [0u8; 1];\n        file.read_exact(&mut buf)\n            .failed(&format!(\"Failed to read magic marker from {path:?}\"));\n\n        if buf[0] != MAGIC_MARKER {\n            failed(&format!(\"Invalid magic marker in {path:?}\"));\n        }\n\n        file.read_exact(&mut buf)\n            .failed(&format!(\"Failed to read subspace from {path:?}\"));\n        let subspace = buf[0];\n\n        let mut buf = [0u8; 4];\n        file.read_exact(&mut buf)\n            .failed(&format!(\"Failed to read version from {path:?}\"));\n        let version = u32::from_le_bytes(buf);\n\n        if version != DATABASE_SCHEMA_VERSION {\n            failed(&format!(\n                \"Invalid database schema version in {path:?}: Expected {DATABASE_SCHEMA_VERSION}, found {version}\"\n            ));\n        }\n\n        Self { file, subspace }\n    }\n\n    fn next(&mut self) -> Option<(Vec<u8>, Vec<u8>)> {\n        let size = self.read_size()?;\n\n        let mut key = vec![0; size as usize];\n        self.file\n            .read_exact(&mut key)\n            .failed(\"Failed to read bytes\");\n        let value = self.expect_sized_bytes();\n\n        Some((key, value))\n    }\n\n    fn read_size(&mut self) -> Option<u32> {\n        let mut result = 0;\n        let mut buf = [0u8; 1];\n\n        for shift in [0, 7, 14, 21, 28] {\n            if let Err(err) = self.file.read_exact(&mut buf) {\n                if err.kind() == ErrorKind::UnexpectedEof {\n                    return None;\n                } else {\n                    failed(&format!(\"Failed to read file: {err:?}\"));\n                }\n            }\n\n            let byte = buf[0];\n            if (byte & 0x80) == 0 {\n                result |= (byte as u32) << shift;\n                return Some(result);\n            } else {\n                result |= ((byte & 0x7F) as u32) << shift;\n            }\n        }\n\n        failed(\"Invalid leb128 sequence\")\n    }\n\n    fn expect_sized_bytes(&mut self) -> Vec<u8> {\n        let len = self.read_size().failed(\"Missing leb128 value sequence\") as usize;\n        let mut bytes = vec![0; len];\n        self.file\n            .read_exact(&mut bytes)\n            .failed(\"Failed to read bytes\");\n        bytes\n    }\n}\n"
  },
  {
    "path": "crates/common/src/manager/webadmin.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    borrow::Cow,\n    io::{self, Cursor, Read},\n    path::PathBuf,\n};\n\nuse ahash::AHashMap;\nuse arc_swap::ArcSwap;\nuse store::BlobStore;\n\nuse crate::Core;\n\nuse super::WEBADMIN_KEY;\n\npub struct WebAdminManager {\n    bundle_path: TempDir,\n    routes: ArcSwap<AHashMap<String, Resource<PathBuf>>>,\n}\n\n#[derive(Default, Clone)]\npub struct Resource<T> {\n    pub content_type: Cow<'static, str>,\n    pub contents: T,\n}\n\nimpl<T> Resource<T> {\n    pub fn new(content_type: impl Into<Cow<'static, str>>, contents: T) -> Self {\n        Self {\n            content_type: content_type.into(),\n            contents,\n        }\n    }\n}\n\nimpl WebAdminManager {\n    pub fn new(base_path: PathBuf) -> Self {\n        Self {\n            bundle_path: TempDir::new(base_path),\n            routes: ArcSwap::from_pointee(Default::default()),\n        }\n    }\n\n    pub async fn get(&self, path: &str) -> trc::Result<Resource<Vec<u8>>> {\n        let routes = self.routes.load();\n        if let Some(resource) = routes.get(path).or_else(|| routes.get(\"index.html\")) {\n            tokio::fs::read(&resource.contents)\n                .await\n                .map(|contents| Resource {\n                    content_type: resource.content_type.clone(),\n                    contents,\n                })\n                .map_err(|err| {\n                    trc::ResourceEvent::Error\n                        .reason(err)\n                        .ctx(trc::Key::Path, path.to_string())\n                        .caused_by(trc::location!())\n                })\n        } else {\n            Ok(Resource::default())\n        }\n    }\n\n    pub async fn unpack(&self, blob_store: &BlobStore) -> trc::Result<()> {\n        // Delete any existing bundles\n        self.bundle_path.clean().await.map_err(unpack_error)?;\n\n        // Obtain webadmin bundle\n        let bundle = blob_store\n            .get_blob(WEBADMIN_KEY, 0..usize::MAX)\n            .await?\n            .ok_or_else(|| {\n                trc::ResourceEvent::NotFound\n                    .caused_by(trc::location!())\n                    .details(\"Webadmin bundle not found\")\n            })?;\n\n        // Uncompress\n        let mut bundle = zip::ZipArchive::new(Cursor::new(bundle)).map_err(|err| {\n            trc::ResourceEvent::Error\n                .caused_by(trc::location!())\n                .reason(err)\n                .details(\"Failed to decompress webadmin bundle\")\n        })?;\n        let mut routes = AHashMap::new();\n        for i in 0..bundle.len() {\n            let (file_name, contents) = {\n                let mut file = bundle.by_index(i).map_err(|err| {\n                    trc::ResourceEvent::Error\n                        .caused_by(trc::location!())\n                        .reason(err)\n                        .details(\"Failed to read file from webadmin bundle\")\n                })?;\n                if file.is_dir() {\n                    continue;\n                }\n\n                let mut contents = Vec::new();\n                file.read_to_end(&mut contents).map_err(unpack_error)?;\n                (file.name().to_string(), contents)\n            };\n            let path = self.bundle_path.path.join(format!(\"{i:02}\"));\n            tokio::fs::write(&path, contents)\n                .await\n                .map_err(unpack_error)?;\n\n            let resource = Resource {\n                content_type: match file_name\n                    .rsplit_once('.')\n                    .map(|(_, ext)| ext)\n                    .unwrap_or_default()\n                {\n                    \"html\" => \"text/html\",\n                    \"css\" => \"text/css\",\n                    \"wasm\" => \"application/wasm\",\n                    \"js\" => \"application/javascript\",\n                    \"json\" => \"application/json\",\n                    \"png\" => \"image/png\",\n                    \"svg\" => \"image/svg+xml\",\n                    \"ico\" => \"image/x-icon\",\n                    _ => \"application/octet-stream\",\n                }\n                .into(),\n                contents: path,\n            };\n\n            routes.insert(file_name, resource);\n        }\n\n        // Update routes\n        self.routes.store(routes.into());\n\n        trc::event!(\n            Resource(trc::ResourceEvent::WebadminUnpacked),\n            Path = self.bundle_path.path.to_string_lossy().into_owned(),\n        );\n\n        Ok(())\n    }\n\n    pub async fn update(&self, core: &Core) -> trc::Result<()> {\n        let bytes = core\n            .storage\n            .config\n            .fetch_resource(\"webadmin\")\n            .await\n            .map_err(|err| {\n                trc::ResourceEvent::Error\n                    .caused_by(trc::location!())\n                    .reason(err)\n                    .details(\"Failed to download webadmin\")\n            })?;\n        core.storage.blob.put_blob(WEBADMIN_KEY, &bytes).await\n    }\n\n    pub async fn update_and_unpack(&self, core: &Core) -> trc::Result<()> {\n        self.update(core).await?;\n        self.unpack(&core.storage.blob).await\n    }\n}\n\nimpl Resource<Vec<u8>> {\n    pub fn is_empty(&self) -> bool {\n        self.content_type.is_empty() && self.contents.is_empty()\n    }\n}\n\npub struct TempDir {\n    pub path: PathBuf,\n}\n\nimpl TempDir {\n    pub fn new(path: PathBuf) -> TempDir {\n        TempDir {\n            path: path.join(std::str::from_utf8(WEBADMIN_KEY).unwrap()),\n        }\n    }\n\n    pub async fn clean(&self) -> io::Result<()> {\n        if tokio::fs::metadata(&self.path).await.is_ok() {\n            let _ = tokio::fs::remove_dir_all(&self.path).await;\n        }\n        tokio::fs::create_dir(&self.path).await\n    }\n}\n\nfn unpack_error(err: std::io::Error) -> trc::Error {\n    trc::ResourceEvent::Error\n        .reason(err)\n        .details(\"Failed to unpack webadmin bundle\")\n}\n\nimpl Default for WebAdminManager {\n    fn default() -> Self {\n        Self::new(std::env::temp_dir())\n    }\n}\n\nimpl Default for TempDir {\n    fn default() -> Self {\n        Self::new(std::env::temp_dir())\n    }\n}\n\nimpl Drop for TempDir {\n    fn drop(&mut self) {\n        let _ = std::fs::remove_dir_all(&self.path);\n    }\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/array.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::collections::{HashMap, HashSet};\n\nuse sieve::{Context, runtime::Variable};\n\npub fn fn_count<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::Array(a) => a.len(),\n        v => {\n            if !v.is_empty() {\n                1\n            } else {\n                0\n            }\n        }\n    }\n    .into()\n}\n\npub fn fn_sort<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let is_asc = v[1].to_bool();\n    let mut arr = (*v[0].to_array()).clone();\n    if is_asc {\n        arr.sort_unstable_by(|a, b| b.cmp(a));\n    } else {\n        arr.sort_unstable();\n    }\n    arr.into()\n}\n\npub fn fn_dedup<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let arr = v[0].to_array();\n    let mut result = Vec::with_capacity(arr.len());\n\n    for item in arr.iter() {\n        if !result.contains(item) {\n            result.push(item.clone());\n        }\n    }\n\n    result.into()\n}\n\npub fn fn_cosine_similarity<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let mut word_freq: HashMap<Variable, [u32; 2]> = HashMap::new();\n\n    for (idx, var) in v.into_iter().enumerate() {\n        match var {\n            Variable::Array(l) => {\n                for item in l.iter() {\n                    word_freq.entry(item.clone()).or_insert([0, 0])[idx] += 1;\n                }\n            }\n            _ => {\n                for char in var.to_string().chars() {\n                    word_freq.entry(char.to_string().into()).or_insert([0, 0])[idx] += 1;\n                }\n            }\n        }\n    }\n\n    let mut dot_product = 0;\n    let mut magnitude_a = 0;\n    let mut magnitude_b = 0;\n\n    for (_word, count) in word_freq.iter() {\n        dot_product += count[0] * count[1];\n        magnitude_a += count[0] * count[0];\n        magnitude_b += count[1] * count[1];\n    }\n\n    if magnitude_a != 0 && magnitude_b != 0 {\n        dot_product as f64 / (magnitude_a as f64).sqrt() / (magnitude_b as f64).sqrt()\n    } else {\n        0.0\n    }\n    .into()\n}\n\npub fn cosine_similarity(a: &[&str], b: &[&str]) -> f64 {\n    let mut word_freq: HashMap<&str, [u32; 2]> = HashMap::new();\n\n    for (idx, items) in [a, b].into_iter().enumerate() {\n        for item in items {\n            word_freq.entry(item).or_insert([0, 0])[idx] += 1;\n        }\n    }\n\n    let mut dot_product = 0;\n    let mut magnitude_a = 0;\n    let mut magnitude_b = 0;\n\n    for (_word, count) in word_freq.iter() {\n        dot_product += count[0] * count[1];\n        magnitude_a += count[0] * count[0];\n        magnitude_b += count[1] * count[1];\n    }\n\n    if magnitude_a != 0 && magnitude_b != 0 {\n        dot_product as f64 / (magnitude_a as f64).sqrt() / (magnitude_b as f64).sqrt()\n    } else {\n        0.0\n    }\n}\n\npub fn fn_jaccard_similarity<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let mut word_freq = [HashSet::new(), HashSet::new()];\n\n    for (idx, var) in v.into_iter().enumerate() {\n        match var {\n            Variable::Array(l) => {\n                for item in l.iter() {\n                    word_freq[idx].insert(item.clone());\n                }\n            }\n            _ => {\n                for char in var.to_string().chars() {\n                    word_freq[idx].insert(char.to_string().into());\n                }\n            }\n        }\n    }\n\n    let intersection_size = word_freq[0].intersection(&word_freq[1]).count();\n    let union_size = word_freq[0].union(&word_freq[1]).count();\n\n    if union_size != 0 {\n        intersection_size as f64 / union_size as f64\n    } else {\n        0.0\n    }\n    .into()\n}\n\npub fn fn_is_intersect<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match (&v[0], &v[1]) {\n        (Variable::Array(a), Variable::Array(b)) => a.iter().any(|x| b.contains(x)),\n        (Variable::Array(a), item) | (item, Variable::Array(a)) => a.contains(item),\n        _ => false,\n    }\n    .into()\n}\n\npub fn fn_winnow<'x>(_: &'x Context<'x>, mut v: Vec<Variable>) -> Variable {\n    match v.remove(0) {\n        Variable::Array(a) => a\n            .iter()\n            .filter(|i| !i.is_empty())\n            .cloned()\n            .collect::<Vec<_>>()\n            .into(),\n        v => v,\n    }\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/email.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse sieve::{Context, runtime::Variable};\n\nuse super::ApplyString;\n\npub fn fn_is_email<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let mut last_ch = 0;\n    let mut in_quote = false;\n    let mut at_count = 0;\n    let mut dot_count = 0;\n    let mut lp_len = 0;\n    let mut value = 0;\n\n    for ch in v[0].to_string().bytes() {\n        match ch {\n            b'0'..=b'9'\n            | b'a'..=b'z'\n            | b'A'..=b'Z'\n            | b'!'\n            | b'#'\n            | b'$'\n            | b'%'\n            | b'&'\n            | b'\\''\n            | b'*'\n            | b'+'\n            | b'-'\n            | b'/'\n            | b'='\n            | b'?'\n            | b'^'\n            | b'_'\n            | b'`'\n            | b'{'\n            | b'|'\n            | b'}'\n            | b'~'\n            | 0x7f..=u8::MAX => {\n                value += 1;\n            }\n            b'.' if !in_quote => {\n                if last_ch != b'.' && last_ch != b'@' && value != 0 {\n                    value += 1;\n                    if at_count == 1 {\n                        dot_count += 1;\n                    }\n                } else {\n                    return false.into();\n                }\n            }\n            b'@' if !in_quote => {\n                at_count += 1;\n                lp_len = value;\n                value = 0;\n            }\n            b'>' | b':' | b',' | b' ' if in_quote => {\n                value += 1;\n            }\n            b'\\\"' if !in_quote || last_ch != b'\\\\' => {\n                in_quote = !in_quote;\n            }\n            b'\\\\' if in_quote && last_ch != b'\\\\' => (),\n            _ => {\n                if !in_quote {\n                    return false.into();\n                }\n            }\n        }\n\n        last_ch = ch;\n    }\n\n    (at_count == 1 && dot_count > 0 && lp_len > 0 && value > 0).into()\n}\n\npub fn fn_email_part<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| {\n        s.rsplit_once('@')\n            .map(|(u, d)| match v[1].to_string().as_ref() {\n                \"local\" => Variable::from(u.trim()),\n                \"domain\" => Variable::from(d.trim()),\n                _ => Variable::default(),\n            })\n            .unwrap_or_default()\n    })\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/header.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mail_parser::{HeaderName, HeaderValue, MimeHeaders, parsers::fields::thread::thread_name};\nuse sieve::{Context, compiler::ReceivedPart, runtime::Variable};\n\nuse super::ApplyString;\n\npub fn fn_received_part<'x>(ctx: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    if let (Ok(part), Some(HeaderValue::Received(rcvd))) = (\n        ReceivedPart::try_from(v[1].to_string().as_ref()),\n        ctx.message()\n            .part(ctx.part())\n            .and_then(|p| {\n                p.headers\n                    .iter()\n                    .filter(|h| h.name == HeaderName::Received)\n                    .nth((v[0].to_integer() as usize).saturating_sub(1))\n            })\n            .map(|h| &h.value),\n    ) {\n        part.eval(rcvd).unwrap_or_default()\n    } else {\n        Variable::default()\n    }\n}\n\npub fn fn_is_encoding_problem<'x>(ctx: &'x Context<'x>, _: Vec<Variable>) -> Variable {\n    ctx.message()\n        .part(ctx.part())\n        .map(|p| p.is_encoding_problem)\n        .unwrap_or_default()\n        .into()\n}\n\npub fn fn_is_attachment<'x>(ctx: &'x Context<'x>, _: Vec<Variable>) -> Variable {\n    ctx.message().attachments.contains(&ctx.part()).into()\n}\n\npub fn fn_is_body<'x>(ctx: &'x Context<'x>, _: Vec<Variable>) -> Variable {\n    (ctx.message().text_body.contains(&ctx.part()) || ctx.message().html_body.contains(&ctx.part()))\n        .into()\n}\n\npub fn fn_attachment_name<'x>(ctx: &'x Context<'x>, _: Vec<Variable>) -> Variable {\n    ctx.message()\n        .part(ctx.part())\n        .and_then(|p| p.attachment_name())\n        .unwrap_or_default()\n        .into()\n}\n\npub fn fn_mime_part_len<'x>(ctx: &'x Context<'x>, _: Vec<Variable>) -> Variable {\n    ctx.message()\n        .part(ctx.part())\n        .map(|p| p.len())\n        .unwrap_or_default()\n        .into()\n}\n\npub fn fn_thread_name<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| thread_name(s).into())\n}\n\npub fn fn_is_header_utf8_valid<'x>(ctx: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    ctx.message()\n        .part(ctx.part())\n        .map(|p| {\n            let raw = ctx.message().raw_message();\n            let mut is_valid = true;\n            if let Some(header_name) = HeaderName::parse(v[0].to_string().as_ref()) {\n                for header in &p.headers {\n                    if header.name == header_name\n                        && raw\n                            .get(header.offset_start() as usize..header.offset_end() as usize)\n                            .and_then(|raw| std::str::from_utf8(raw).ok())\n                            .is_none()\n                    {\n                        is_valid = false;\n                        break;\n                    }\n                }\n            } else {\n                is_valid = raw\n                    .get(p.raw_header_offset() as usize..p.raw_body_offset() as usize)\n                    .and_then(|raw| std::str::from_utf8(raw).ok())\n                    .is_some();\n            }\n\n            Variable::from(is_valid)\n        })\n        .unwrap_or(Variable::Integer(1))\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/image.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse sieve::{Context, runtime::Variable};\n\npub fn fn_img_metadata<'x>(ctx: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    ctx.message()\n        .part(ctx.part())\n        .map(|p| p.contents())\n        .and_then(|bytes| {\n            let arg = v[1].to_string();\n            match arg.as_ref() {\n                \"type\" => imagesize::image_type(bytes).ok().map(|t| {\n                    Variable::from(match t {\n                        imagesize::ImageType::Aseprite => \"aseprite\",\n                        imagesize::ImageType::Bmp => \"bmp\",\n                        imagesize::ImageType::Dds => \"dds\",\n                        imagesize::ImageType::Exr => \"exr\",\n                        imagesize::ImageType::Farbfeld => \"farbfeld\",\n                        imagesize::ImageType::Gif => \"gif\",\n                        imagesize::ImageType::Hdr => \"hdr\",\n                        imagesize::ImageType::Heif(_) => \"heif\",\n                        imagesize::ImageType::Ico => \"ico\",\n                        imagesize::ImageType::Jpeg => \"jpeg\",\n                        imagesize::ImageType::Jxl => \"jxl\",\n                        imagesize::ImageType::Ktx2 => \"ktx2\",\n                        imagesize::ImageType::Png => \"png\",\n                        imagesize::ImageType::Pnm => \"pnm\",\n                        imagesize::ImageType::Psd => \"psd\",\n                        imagesize::ImageType::Qoi => \"qoi\",\n                        imagesize::ImageType::Tga => \"tga\",\n                        imagesize::ImageType::Tiff => \"tiff\",\n                        imagesize::ImageType::Vtf => \"vtf\",\n                        imagesize::ImageType::Webp => \"webp\",\n                        imagesize::ImageType::Ilbm => \"ilbm\",\n                        _ => \"unknown\",\n                    })\n                }),\n                \"width\" => imagesize::blob_size(bytes)\n                    .ok()\n                    .map(|s| Variable::Integer(s.width as i64)),\n                \"height\" => imagesize::blob_size(bytes)\n                    .ok()\n                    .map(|s| Variable::Integer(s.height as i64)),\n                \"area\" => imagesize::blob_size(bytes)\n                    .ok()\n                    .map(|s| Variable::Integer(s.width.saturating_mul(s.height) as i64)),\n                \"dimension\" => imagesize::blob_size(bytes)\n                    .ok()\n                    .map(|s| Variable::Integer(s.width.saturating_add(s.height) as i64)),\n                _ => None,\n            }\n        })\n        .unwrap_or_default()\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/misc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse mail_auth::common::resolver::ToReverseName;\nuse sha1::Sha1;\nuse sha2::{Sha256, Sha512};\nuse sieve::{Context, runtime::Variable};\n\nuse super::ApplyString;\n\npub fn fn_is_empty<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.is_empty(),\n        Variable::Integer(_) | Variable::Float(_) => false,\n        Variable::Array(a) => a.is_empty(),\n    }\n    .into()\n}\n\npub fn fn_is_number<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    matches!(&v[0], Variable::Integer(_) | Variable::Float(_)).into()\n}\n\npub fn fn_is_ip_addr<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string().parse::<std::net::IpAddr>().is_ok().into()\n}\n\npub fn fn_is_ipv4_addr<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .parse::<std::net::IpAddr>()\n        .is_ok_and(|ip| matches!(ip, IpAddr::V4(_)))\n        .into()\n}\n\npub fn fn_is_ipv6_addr<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .parse::<std::net::IpAddr>()\n        .is_ok_and(|ip| matches!(ip, IpAddr::V6(_)))\n        .into()\n}\n\npub fn fn_ip_reverse_name<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .parse::<std::net::IpAddr>()\n        .map(|ip| ip.to_reverse_name())\n        .unwrap_or_default()\n        .into()\n}\n\npub fn fn_detect_file_type<'x>(ctx: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    ctx.message()\n        .part(ctx.part())\n        .and_then(|p| infer::get(p.contents()))\n        .map(|t| {\n            Variable::from(\n                if v[0].to_string() != \"ext\" {\n                    t.mime_type()\n                } else {\n                    t.extension()\n                }\n                .to_string(),\n            )\n        })\n        .unwrap_or_default()\n}\n\npub fn fn_hash<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    use sha1::Digest;\n    let hash = v[1].to_string();\n\n    v[0].transform(|value| match hash.as_ref() {\n        \"md5\" => format!(\"{:x}\", md5::compute(value.as_bytes())).into(),\n        \"sha1\" => {\n            let mut hasher = Sha1::new();\n            hasher.update(value.as_bytes());\n            format!(\"{:x}\", hasher.finalize()).into()\n        }\n        \"sha256\" => {\n            let mut hasher = Sha256::new();\n            hasher.update(value.as_bytes());\n            format!(\"{:x}\", hasher.finalize()).into()\n        }\n        \"sha512\" => {\n            let mut hasher = Sha512::new();\n            hasher.update(value.as_bytes());\n            format!(\"{:x}\", hasher.finalize()).into()\n        }\n        _ => Variable::default(),\n    })\n}\n\npub fn fn_get_var_names<'x>(ctx: &'x Context<'x>, _: Vec<Variable>) -> Variable {\n    Variable::Array(\n        ctx.global_variable_names()\n            .map(|v| Variable::from(v.to_uppercase()))\n            .collect::<Vec<_>>()\n            .into(),\n    )\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod array;\nmod email;\nmod header;\npub mod image;\npub mod misc;\npub mod text;\npub mod unicode;\npub mod url;\n\nuse sieve::{FunctionMap, runtime::Variable};\n\nuse self::{array::*, email::*, header::*, image::*, misc::*, text::*, unicode::*, url::*};\n\npub fn register_functions_trusted() -> FunctionMap {\n    FunctionMap::new()\n        .with_function(\"trim\", fn_trim)\n        .with_function(\"trim_start\", fn_trim_start)\n        .with_function(\"trim_end\", fn_trim_end)\n        .with_function(\"len\", fn_len)\n        .with_function(\"count\", fn_count)\n        .with_function(\"is_empty\", fn_is_empty)\n        .with_function(\"is_number\", fn_is_number)\n        .with_function(\"is_ascii\", fn_is_ascii)\n        .with_function(\"to_lowercase\", fn_to_lowercase)\n        .with_function(\"to_uppercase\", fn_to_uppercase)\n        .with_function(\"detect_language\", fn_detect_language)\n        .with_function(\"is_email\", fn_is_email)\n        .with_function(\"thread_name\", fn_thread_name)\n        .with_function(\"html_to_text\", fn_html_to_text)\n        .with_function(\"is_uppercase\", fn_is_uppercase)\n        .with_function(\"is_lowercase\", fn_is_lowercase)\n        .with_function(\"has_digits\", fn_has_digits)\n        .with_function(\"count_spaces\", fn_count_spaces)\n        .with_function(\"count_uppercase\", fn_count_uppercase)\n        .with_function(\"count_lowercase\", fn_count_lowercase)\n        .with_function(\"count_chars\", fn_count_chars)\n        .with_function(\"dedup\", fn_dedup)\n        .with_function(\"lines\", fn_lines)\n        .with_function(\"is_header_utf8_valid\", fn_is_header_utf8_valid)\n        .with_function(\"img_metadata\", fn_img_metadata)\n        .with_function(\"is_ip_addr\", fn_is_ip_addr)\n        .with_function(\"is_ipv4_addr\", fn_is_ipv4_addr)\n        .with_function(\"is_ipv6_addr\", fn_is_ipv6_addr)\n        .with_function(\"ip_reverse_name\", fn_ip_reverse_name)\n        .with_function(\"winnow\", fn_winnow)\n        .with_function(\"has_zwsp\", fn_has_zwsp)\n        .with_function(\"has_obscured\", fn_has_obscured)\n        .with_function(\"is_mixed_charset\", fn_is_mixed_charset)\n        .with_function(\"puny_decode\", fn_puny_decode)\n        .with_function(\"unicode_skeleton\", fn_unicode_skeleton)\n        .with_function(\"cure_text\", fn_cure_text)\n        .with_function(\"detect_file_type\", fn_detect_file_type)\n        .with_function_args(\"sort\", fn_sort, 2)\n        .with_function_args(\"email_part\", fn_email_part, 2)\n        .with_function_args(\"eq_ignore_case\", fn_eq_ignore_case, 2)\n        .with_function_args(\"contains\", fn_contains, 2)\n        .with_function_args(\"contains_ignore_case\", fn_contains_ignore_case, 2)\n        .with_function_args(\"starts_with\", fn_starts_with, 2)\n        .with_function_args(\"ends_with\", fn_ends_with, 2)\n        .with_function_args(\"received_part\", fn_received_part, 2)\n        .with_function_args(\"cosine_similarity\", fn_cosine_similarity, 2)\n        .with_function_args(\"jaccard_similarity\", fn_jaccard_similarity, 2)\n        .with_function_args(\"levenshtein_distance\", fn_levenshtein_distance, 2)\n        .with_function_args(\"uri_part\", fn_uri_part, 2)\n        .with_function_args(\"substring\", fn_substring, 3)\n        .with_function_args(\"split\", fn_split, 2)\n        .with_function_args(\"rsplit\", fn_rsplit, 2)\n        .with_function_args(\"split_once\", fn_split_once, 2)\n        .with_function_args(\"rsplit_once\", fn_rsplit_once, 2)\n        .with_function_args(\"split_n\", fn_split_n, 3)\n        .with_function_args(\"strip_prefix\", fn_strip_prefix, 2)\n        .with_function_args(\"strip_suffix\", fn_strip_suffix, 2)\n        .with_function_args(\"is_intersect\", fn_is_intersect, 2)\n        .with_function_args(\"hash\", fn_hash, 2)\n        .with_function_no_args(\"is_encoding_problem\", fn_is_encoding_problem)\n        .with_function_no_args(\"is_attachment\", fn_is_attachment)\n        .with_function_no_args(\"is_body\", fn_is_body)\n        .with_function_no_args(\"var_names\", fn_get_var_names)\n        .with_function_no_args(\"attachment_name\", fn_attachment_name)\n        .with_function_no_args(\"mime_part_len\", fn_mime_part_len)\n}\n\npub fn register_functions_untrusted() -> FunctionMap {\n    FunctionMap::new()\n        .with_function(\"trim\", fn_trim)\n        .with_function(\"trim_start\", fn_trim_start)\n        .with_function(\"trim_end\", fn_trim_end)\n        .with_function(\"len\", fn_len)\n        .with_function(\"count\", fn_count)\n        .with_function(\"is_empty\", fn_is_empty)\n        .with_function(\"is_number\", fn_is_number)\n        .with_function(\"is_ascii\", fn_is_ascii)\n        .with_function(\"to_lowercase\", fn_to_lowercase)\n        .with_function(\"to_uppercase\", fn_to_uppercase)\n        .with_function(\"is_email\", fn_is_email)\n        .with_function(\"thread_name\", fn_thread_name)\n        .with_function(\"html_to_text\", fn_html_to_text)\n        .with_function(\"is_uppercase\", fn_is_uppercase)\n        .with_function(\"is_lowercase\", fn_is_lowercase)\n        .with_function(\"has_digits\", fn_has_digits)\n        .with_function(\"count_spaces\", fn_count_spaces)\n        .with_function(\"count_uppercase\", fn_count_uppercase)\n        .with_function(\"count_lowercase\", fn_count_lowercase)\n        .with_function(\"count_chars\", fn_count_chars)\n        .with_function(\"dedup\", fn_dedup)\n        .with_function(\"lines\", fn_lines)\n        .with_function(\"is_ip_addr\", fn_is_ip_addr)\n        .with_function(\"is_ipv4_addr\", fn_is_ipv4_addr)\n        .with_function(\"is_ipv6_addr\", fn_is_ipv6_addr)\n        .with_function(\"winnow\", fn_winnow)\n        .with_function_args(\"sort\", fn_sort, 2)\n        .with_function_args(\"email_part\", fn_email_part, 2)\n        .with_function_args(\"eq_ignore_case\", fn_eq_ignore_case, 2)\n        .with_function_args(\"contains\", fn_contains, 2)\n        .with_function_args(\"contains_ignore_case\", fn_contains_ignore_case, 2)\n        .with_function_args(\"starts_with\", fn_starts_with, 2)\n        .with_function_args(\"ends_with\", fn_ends_with, 2)\n        .with_function_args(\"uri_part\", fn_uri_part, 2)\n        .with_function_args(\"substring\", fn_substring, 3)\n        .with_function_args(\"split\", fn_split, 2)\n        .with_function_args(\"rsplit\", fn_rsplit, 2)\n        .with_function_args(\"split_once\", fn_split_once, 2)\n        .with_function_args(\"rsplit_once\", fn_rsplit_once, 2)\n        .with_function_args(\"split_n\", fn_split_n, 3)\n        .with_function_args(\"strip_prefix\", fn_strip_prefix, 2)\n        .with_function_args(\"strip_suffix\", fn_strip_suffix, 2)\n        .with_function_args(\"is_intersect\", fn_is_intersect, 2)\n}\n\npub trait ApplyString<'x> {\n    fn transform(&self, f: impl Fn(&'_ str) -> Variable) -> Variable;\n}\n\nimpl ApplyString<'_> for Variable {\n    fn transform(&self, f: impl Fn(&'_ str) -> Variable) -> Variable {\n        match self {\n            Variable::String(s) => f(s),\n            Variable::Array(list) => list\n                .iter()\n                .map(|v| match v {\n                    Variable::String(s) => f(s),\n                    v => f(v.to_string().as_ref()),\n                })\n                .collect::<Vec<_>>()\n                .into(),\n            v => f(v.to_string().as_ref()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/text.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mail_parser::decoders::html::html_to_text;\nuse sieve::{Context, runtime::Variable};\n\nuse super::ApplyString;\n\npub fn fn_trim<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| Variable::from(s.trim()))\n}\n\npub fn fn_trim_end<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| Variable::from(s.trim_end()))\n}\n\npub fn fn_trim_start<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| Variable::from(s.trim_start()))\n}\n\npub fn fn_len<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.len(),\n        Variable::Array(a) => a.len(),\n        v => v.to_string().len(),\n    }\n    .into()\n}\n\npub fn fn_to_lowercase<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| Variable::from(s.to_lowercase()))\n}\n\npub fn fn_to_uppercase<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| Variable::from(s.to_uppercase()))\n}\n\npub fn fn_is_uppercase<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| {\n        s.chars()\n            .filter(|c| c.is_alphabetic())\n            .all(|c| c.is_uppercase())\n            .into()\n    })\n}\n\npub fn fn_is_lowercase<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| {\n        s.chars()\n            .filter(|c| c.is_alphabetic())\n            .all(|c| c.is_lowercase())\n            .into()\n    })\n}\n\npub fn fn_has_digits<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|s| s.chars().any(|c| c.is_ascii_digit()).into())\n}\n\npub fn tokenize_words(v: &Variable) -> Variable {\n    v.to_string()\n        .split_whitespace()\n        .filter(|word| word.chars().all(|c| c.is_alphanumeric()))\n        .map(|word| Variable::from(word.to_string()))\n        .collect::<Vec<_>>()\n        .into()\n}\n\npub fn fn_count_spaces<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_ref()\n        .chars()\n        .filter(|c| c.is_whitespace())\n        .count()\n        .into()\n}\n\npub fn fn_count_uppercase<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_ref()\n        .chars()\n        .filter(|c| c.is_alphabetic() && c.is_uppercase())\n        .count()\n        .into()\n}\n\npub fn fn_count_lowercase<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .as_ref()\n        .chars()\n        .filter(|c| c.is_alphabetic() && c.is_lowercase())\n        .count()\n        .into()\n}\n\npub fn fn_count_chars<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string().as_ref().chars().count().into()\n}\n\npub fn fn_eq_ignore_case<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .eq_ignore_ascii_case(v[1].to_string().as_ref())\n        .into()\n}\n\npub fn fn_contains<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.contains(v[1].to_string().as_ref()),\n        Variable::Array(arr) => arr.contains(&v[1]),\n        val => val.to_string().contains(v[1].to_string().as_ref()),\n    }\n    .into()\n}\n\npub fn fn_contains_ignore_case<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let needle = v[1].to_string();\n    match &v[0] {\n        Variable::String(s) => s.to_lowercase().contains(&needle.to_lowercase()),\n        Variable::Array(arr) => arr.iter().any(|v| match v {\n            Variable::String(s) => s.eq_ignore_ascii_case(needle.as_ref()),\n            _ => false,\n        }),\n        val => val.to_string().contains(needle.as_ref()),\n    }\n    .into()\n}\n\npub fn fn_starts_with<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .starts_with(v[1].to_string().as_ref())\n        .into()\n}\n\npub fn fn_ends_with<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string().ends_with(v[1].to_string().as_ref()).into()\n}\n\npub fn fn_lines<'x>(_: &'x Context<'x>, mut v: Vec<Variable>) -> Variable {\n    match v.remove(0) {\n        Variable::String(s) => s\n            .lines()\n            .map(|s| Variable::from(s.to_string()))\n            .collect::<Vec<_>>()\n            .into(),\n        val => val,\n    }\n}\n\npub fn fn_substring<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .chars()\n        .skip(v[1].to_usize())\n        .take(v[2].to_usize())\n        .collect::<String>()\n        .into()\n}\n\npub fn fn_strip_prefix<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let prefix = v[1].to_string();\n    v[0].transform(|s| {\n        s.strip_prefix(prefix.as_ref())\n            .map(Variable::from)\n            .unwrap_or_default()\n    })\n}\n\npub fn fn_strip_suffix<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let suffix = v[1].to_string();\n    v[0].transform(|s| {\n        s.strip_suffix(suffix.as_ref())\n            .map(Variable::from)\n            .unwrap_or_default()\n    })\n}\n\npub fn fn_split<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .split(v[1].to_string().as_ref())\n        .map(|s| Variable::from(s.to_string()))\n        .collect::<Vec<_>>()\n        .into()\n}\n\npub fn fn_rsplit<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .rsplit(v[1].to_string().as_ref())\n        .map(|s| Variable::from(s.to_string()))\n        .collect::<Vec<_>>()\n        .into()\n}\n\npub fn fn_split_n<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let value = v[0].to_string();\n    let arg = v[1].to_string();\n    let num = v[2].to_integer() as usize;\n    let mut result = Vec::new();\n\n    let mut s = value.as_ref();\n    for _ in 0..num {\n        if let Some((a, b)) = s.split_once(arg.as_ref()) {\n            result.push(Variable::from(a.to_string()));\n            s = b;\n        } else {\n            break;\n        }\n    }\n    result.push(Variable::from(s.to_string()));\n    result.into()\n}\n\npub fn fn_split_once<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .split_once(v[1].to_string().as_ref())\n        .map(|(a, b)| {\n            Variable::Array(\n                vec![Variable::from(a.to_string()), Variable::from(b.to_string())].into(),\n            )\n        })\n        .unwrap_or_default()\n}\n\npub fn fn_rsplit_once<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].to_string()\n        .rsplit_once(v[1].to_string().as_ref())\n        .map(|(a, b)| {\n            Variable::Array(\n                vec![Variable::from(a.to_string()), Variable::from(b.to_string())].into(),\n            )\n        })\n        .unwrap_or_default()\n}\n\n/**\n * `levenshtein-rs` - levenshtein\n *\n * MIT licensed.\n *\n * Copyright (c) 2016 Titus Wormer <tituswormer@gmail.com>\n */\npub fn fn_levenshtein_distance<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let a = v[0].to_string();\n    let b = v[1].to_string();\n\n    levenshtein_distance(a.as_ref(), b.as_ref()).into()\n}\n\npub fn levenshtein_distance(a: &str, b: &str) -> usize {\n    let mut result = 0;\n\n    /* Shortcut optimizations / degenerate cases. */\n    if a == b {\n        return result;\n    }\n\n    let length_a = a.chars().count();\n    let length_b = b.chars().count();\n\n    if length_a == 0 {\n        return length_b;\n    } else if length_b == 0 {\n        return length_a;\n    }\n\n    /* Initialize the vector.\n     *\n     * This is why it’s fast, normally a matrix is used,\n     * here we use a single vector. */\n    let mut cache: Vec<usize> = (1..).take(length_a).collect();\n    let mut distance_a;\n    let mut distance_b;\n\n    /* Loop. */\n    for (index_b, code_b) in b.chars().enumerate() {\n        result = index_b;\n        distance_a = index_b;\n\n        for (index_a, code_a) in a.chars().enumerate() {\n            distance_b = if code_a == code_b {\n                distance_a\n            } else {\n                distance_a + 1\n            };\n\n            distance_a = cache[index_a];\n\n            result = if distance_a > result {\n                if distance_b > result {\n                    result + 1\n                } else {\n                    distance_b\n                }\n            } else if distance_b > distance_a {\n                distance_a + 1\n            } else {\n                distance_b\n            };\n\n            cache[index_a] = result;\n        }\n    }\n\n    result\n}\n\npub fn fn_detect_language<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    whatlang::detect_lang(v[0].to_string().as_ref())\n        .map(|l| l.code())\n        .unwrap_or(\"unknown\")\n        .into()\n}\n\npub fn fn_html_to_text<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    html_to_text(v[0].to_string().as_ref()).into()\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/unicode.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse sieve::{Context, runtime::Variable};\n\nuse crate::scripts::IsMixedCharset;\n\npub fn fn_is_ascii<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.is_ascii(),\n        Variable::Integer(_) | Variable::Float(_) => true,\n        Variable::Array(a) => a.iter().all(|v| match v {\n            Variable::String(s) => s.is_ascii(),\n            _ => true,\n        }),\n    }\n    .into()\n}\n\npub fn fn_has_zwsp<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.chars().any(|c| c.is_zwsp()),\n        Variable::Array(a) => a.iter().any(|v| match v {\n            Variable::String(s) => s.chars().any(|c| c.is_zwsp()),\n            _ => true,\n        }),\n        Variable::Integer(_) | Variable::Float(_) => false,\n    }\n    .into()\n}\n\npub fn fn_has_obscured<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    match &v[0] {\n        Variable::String(s) => s.chars().any(|c| c.is_obscured()),\n        Variable::Array(a) => a.iter().any(|v| match v {\n            Variable::String(s) => s.chars().any(|c| c.is_obscured()),\n            _ => true,\n        }),\n        Variable::Integer(_) | Variable::Float(_) => false,\n    }\n    .into()\n}\n\npub trait CharUtils {\n    fn is_zwsp(&self) -> bool;\n    fn is_obscured(&self) -> bool;\n}\n\nimpl CharUtils for char {\n    fn is_zwsp(&self) -> bool {\n        matches!(\n            self,\n            '\\u{200B}' | '\\u{200C}' | '\\u{200D}' | '\\u{FEFF}' | '\\u{00AD}'\n        )\n    }\n\n    fn is_obscured(&self) -> bool {\n        matches!(\n            self,\n            '\\u{200B}'..='\\u{200F}'\n                | '\\u{2028}'..='\\u{202F}'\n                | '\\u{205F}'..='\\u{206F}'\n                | '\\u{FEFF}'\n        )\n    }\n}\n\npub fn fn_cure_text<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    decancer::cure(v[0].to_string().as_ref(), decancer::Options::default())\n        .map(String::from)\n        .unwrap_or_default()\n        .into()\n}\n\npub fn fn_unicode_skeleton<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    unicode_security::skeleton(v[0].to_string().as_ref())\n        .collect::<String>()\n        .into()\n}\n\npub fn fn_is_mixed_charset<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let text = v[0].to_string();\n    if !text.is_empty() {\n        text.as_ref().is_mixed_charset()\n    } else {\n        false\n    }\n    .into()\n}\n"
  },
  {
    "path": "crates/common/src/scripts/functions/url.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse hyper::Uri;\nuse sieve::{Context, runtime::Variable};\n\nuse super::ApplyString;\n\npub fn fn_uri_part<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    let part = v[1].to_string();\n    v[0].transform(|uri| {\n        uri.parse::<Uri>()\n            .ok()\n            .and_then(|uri| match part.as_ref() {\n                \"scheme\" => uri.scheme_str().map(|s| Variable::from(s.to_string())),\n                \"host\" => uri.host().map(|s| Variable::from(s.to_string())),\n                \"scheme_host\" => uri\n                    .scheme_str()\n                    .and_then(|s| (s, uri.host()?).into())\n                    .map(|(s, h)| Variable::from(format!(\"{}://{}\", s, h))),\n                \"path\" => Variable::from(uri.path().to_string()).into(),\n                \"port\" => uri.port_u16().map(|port| Variable::Integer(port as i64)),\n                \"query\" => uri.query().map(|s| Variable::from(s.to_string())),\n                \"path_query\" => uri.path_and_query().map(|s| Variable::from(s.to_string())),\n                \"authority\" => uri.authority().map(|s| Variable::from(s.to_string())),\n                _ => None,\n            })\n            .unwrap_or_default()\n    })\n}\n\npub fn fn_puny_decode<'x>(_: &'x Context<'x>, v: Vec<Variable>) -> Variable {\n    v[0].transform(|domain| {\n        if domain.contains(\"xn--\") {\n            let mut decoded = String::with_capacity(domain.len());\n            for part in domain.split('.') {\n                if !decoded.is_empty() {\n                    decoded.push('.');\n                }\n\n                if let Some(puny) = part\n                    .strip_prefix(\"xn--\")\n                    .and_then(idna::punycode::decode_to_string)\n                {\n                    decoded.push_str(&puny);\n                } else {\n                    decoded.push_str(part);\n                }\n            }\n            decoded.into()\n        } else {\n            domain.into()\n        }\n    })\n}\n"
  },
  {
    "path": "crates/common/src/scripts/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse sieve::{Envelope, runtime::Variable};\nuse store::Value;\nuse unicode_security::mixed_script::AugmentedScriptSet;\n\nuse crate::IntoString;\n\npub mod functions;\npub mod plugins;\n\n#[derive(Debug, serde::Serialize)]\n#[serde(tag = \"action\")]\n#[serde(rename_all = \"camelCase\")]\npub enum ScriptModification {\n    SetEnvelope {\n        name: Envelope,\n        value: String,\n    },\n    AddHeader {\n        name: Arc<String>,\n        value: Arc<String>,\n    },\n}\n\npub fn into_sieve_value(value: Value) -> Variable {\n    match value {\n        Value::Integer(v) => Variable::Integer(v),\n        Value::Bool(v) => Variable::Integer(i64::from(v)),\n        Value::Float(v) => Variable::Float(v),\n        Value::Text(v) => Variable::String(v.into_owned().into()),\n        Value::Blob(v) => Variable::String(v.into_owned().into_string().into()),\n        Value::Null => Variable::default(),\n    }\n}\n\npub fn into_store_value(value: Variable) -> Value<'static> {\n    match value {\n        Variable::String(v) => Value::Text(v.to_string().into()),\n        Variable::Integer(v) => Value::Integer(v),\n        Variable::Float(v) => Value::Float(v),\n        v => Value::Text(v.to_string().into_owned().into()),\n    }\n}\n\npub fn to_store_value(value: &Variable) -> Value<'static> {\n    match value {\n        Variable::String(v) => Value::Text(v.to_string().into()),\n        Variable::Integer(v) => Value::Integer(*v),\n        Variable::Float(v) => Value::Float(*v),\n        v => Value::Text(v.to_string().into_owned().into()),\n    }\n}\n\npub trait IsMixedCharset {\n    fn is_mixed_charset(&self) -> bool;\n}\n\nimpl<T: AsRef<str>> IsMixedCharset for T {\n    fn is_mixed_charset(&self) -> bool {\n        let mut set: Option<AugmentedScriptSet> = None;\n\n        for ch in self.as_ref().chars() {\n            if !ch.is_ascii() {\n                set.get_or_insert_default().intersect_with(ch.into());\n            }\n        }\n\n        set.is_some_and(|set| set.is_empty())\n    }\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/dns.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse mail_auth::IpLookupStrategy;\nuse sieve::{FunctionMap, runtime::Variable};\n\nuse super::PluginContext;\n\npub fn register(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"dns_query\", plugin_id, 2);\n}\n\npub fn register_exists(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"dns_exists\", plugin_id, 2);\n}\n\npub async fn exec(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let entry = ctx.arguments[0].to_string();\n    let record_type = ctx.arguments[1].to_string();\n\n    Ok(if record_type.eq_ignore_ascii_case(\"ip\") {\n        match ctx\n            .server\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .ip_lookup(\n                entry.as_ref(),\n                IpLookupStrategy::Ipv4thenIpv6,\n                10,\n                Some(&ctx.server.inner.cache.dns_ipv4),\n                Some(&ctx.server.inner.cache.dns_ipv6),\n            )\n            .await\n        {\n            Ok(result) => result\n                .iter()\n                .map(|ip| Variable::from(ip.to_string()))\n                .collect::<Vec<_>>()\n                .into(),\n            Err(err) => err.short_error().into(),\n        }\n    } else if record_type.eq_ignore_ascii_case(\"mx\") {\n        match ctx\n            .server\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .mx_lookup(entry.as_ref(), Some(&ctx.server.inner.cache.dns_mx))\n            .await\n        {\n            Ok(result) => result\n                .iter()\n                .flat_map(|mx| {\n                    mx.exchanges\n                        .iter()\n                        .map(|host| Variable::from(format!(\"{} {}\", mx.preference, host)))\n                })\n                .collect::<Vec<_>>()\n                .into(),\n            Err(err) => err.short_error().into(),\n        }\n    } else if record_type.eq_ignore_ascii_case(\"txt\") {\n        #[cfg(feature = \"test_mode\")]\n        {\n            if entry.contains(\"origin\") {\n                return Ok(Variable::from(\"23028|US|arin|2002-01-04\".to_string()));\n            }\n        }\n\n        match ctx\n            .server\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .txt_raw_lookup(entry.as_ref())\n            .await\n        {\n            Ok(result) => Variable::from(String::from_utf8(result).unwrap_or_default()),\n            Err(err) => err.short_error().into(),\n        }\n    } else if record_type.eq_ignore_ascii_case(\"ptr\") {\n        if let Ok(addr) = entry.parse::<IpAddr>() {\n            match ctx\n                .server\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .ptr_lookup(addr, Some(&ctx.server.inner.cache.dns_ptr))\n                .await\n            {\n                Ok(result) => result\n                    .iter()\n                    .map(|host| Variable::from(host.to_string()))\n                    .collect::<Vec<_>>()\n                    .into(),\n                Err(err) => err.short_error().into(),\n            }\n        } else {\n            Variable::default()\n        }\n    } else if record_type.eq_ignore_ascii_case(\"ipv4\") {\n        #[cfg(feature = \"test_mode\")]\n        {\n            if entry.contains(\".168.192.\") {\n                let parts = entry.split('.').collect::<Vec<_>>();\n                return Ok(vec![Variable::from(format!(\"127.0.{}.{}\", parts[1], parts[0]))].into());\n            }\n        }\n\n        match ctx\n            .server\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .ipv4_lookup(entry.as_ref(), Some(&ctx.server.inner.cache.dns_ipv4))\n            .await\n        {\n            Ok(result) => result\n                .iter()\n                .map(|ip| Variable::from(ip.to_string()))\n                .collect::<Vec<_>>()\n                .into(),\n            Err(err) => err.short_error().into(),\n        }\n    } else if record_type.eq_ignore_ascii_case(\"ipv6\") {\n        match ctx\n            .server\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .ipv6_lookup(entry.as_ref(), Some(&ctx.server.inner.cache.dns_ipv6))\n            .await\n        {\n            Ok(result) => result\n                .iter()\n                .map(|ip| Variable::from(ip.to_string()))\n                .collect::<Vec<_>>()\n                .into(),\n            Err(err) => err.short_error().into(),\n        }\n    } else {\n        Variable::default()\n    })\n}\n\npub async fn exec_exists(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let entry = ctx.arguments[0].to_string();\n    let record_type = ctx.arguments[1].to_string();\n\n    let result = if record_type.eq_ignore_ascii_case(\"ip\") {\n        ctx.server.dns_exists_ip(entry.as_ref()).await\n    } else if record_type.eq_ignore_ascii_case(\"mx\") {\n        ctx.server.dns_exists_mx(entry.as_ref()).await\n    } else if record_type.eq_ignore_ascii_case(\"ptr\") {\n        ctx.server.dns_exists_ptr(entry.as_ref()).await\n    } else if record_type.eq_ignore_ascii_case(\"ipv4\") {\n        #[cfg(feature = \"test_mode\")]\n        {\n            if entry.starts_with(\"2.0.168.192.\") {\n                return Ok(1.into());\n            }\n        }\n\n        ctx.server.dns_exists_ipv4(entry.as_ref()).await\n    } else if record_type.eq_ignore_ascii_case(\"ipv6\") {\n        ctx.server.dns_exists_ipv6(entry.as_ref()).await\n    } else {\n        return Ok((-1).into());\n    };\n\n    Ok(result.map(i64::from).unwrap_or(-1).into())\n}\n\ntrait ShortError {\n    fn short_error(&self) -> &'static str;\n}\n\nimpl ShortError for mail_auth::Error {\n    fn short_error(&self) -> &'static str {\n        match self {\n            mail_auth::Error::DnsError(_) => \"temp_fail\",\n            mail_auth::Error::DnsRecordNotFound(_) => \"not_found\",\n            mail_auth::Error::Io(_) => \"io_error\",\n            mail_auth::Error::InvalidRecordType => \"invalid_record\",\n            _ => \"unknown_error\",\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/exec.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::process::Command;\n\nuse sieve::{FunctionMap, runtime::Variable};\n\nuse super::PluginContext;\n\npub fn register(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"exec\", plugin_id, 2);\n}\n\npub async fn exec(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let mut arguments = ctx.arguments.into_iter();\n\n    tokio::task::spawn_blocking(move || {\n        let command = arguments\n            .next()\n            .map(|a| a.to_string().into_owned())\n            .unwrap_or_default();\n\n        match Command::new(&command)\n            .args(\n                arguments\n                    .next()\n                    .map(|a| a.into_string_array())\n                    .unwrap_or_default(),\n            )\n            .output()\n        {\n            Ok(result) => Ok(result.status.success()),\n            Err(err) => Err(trc::SieveEvent::RuntimeError\n                .ctx(trc::Key::Path, command)\n                .reason(err)\n                .details(\"Failed to execute command\")),\n        }\n    })\n    .await\n    .map_err(|err| {\n        trc::EventType::Server(trc::ServerEvent::ThreadError)\n            .reason(err)\n            .caused_by(trc::location!())\n            .details(\"Join Error\")\n    })?\n    .map(Into::into)\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/headers.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse sieve::{FunctionMap, runtime::Variable};\n\nuse crate::scripts::ScriptModification;\n\nuse super::PluginContext;\n\npub fn register(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"add_header\", plugin_id, 2);\n}\n\npub fn exec(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    Ok(if let (Variable::String(name), Variable::String(value)) =\n        (&ctx.arguments[0], &ctx.arguments[1])\n    {\n        ctx.modifications.push(ScriptModification::AddHeader {\n            name: name.clone(),\n            value: value.clone(),\n        });\n        true\n    } else {\n        false\n    }\n    .into())\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/http.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse reqwest::redirect::Policy;\nuse sieve::{FunctionMap, runtime::Variable};\n\nuse super::PluginContext;\n\npub fn register_header(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"http_header\", plugin_id, 4);\n}\n\npub async fn exec_header(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let url = ctx.arguments[0].to_string();\n    let header = ctx.arguments[1].to_string();\n    let agent = ctx.arguments[2].to_string();\n    let timeout = ctx.arguments[3].to_string().parse::<u64>().unwrap_or(5000);\n\n    #[cfg(feature = \"test_mode\")]\n    if url.contains(\"redirect.\") {\n        return Ok(Variable::from(url.split_once(\"/?\").unwrap().1.to_string()));\n    }\n\n    reqwest::Client::builder()\n        .user_agent(agent.as_ref())\n        .timeout(Duration::from_millis(timeout))\n        .redirect(Policy::none())\n        .danger_accept_invalid_certs(true)\n        .build()\n        .map_err(|err| {\n            trc::SieveEvent::RuntimeError\n                .into_err()\n                .reason(err)\n                .details(\"Failed to build request\")\n        })?\n        .get(url.as_ref())\n        .send()\n        .await\n        .map_err(|err| {\n            trc::SieveEvent::RuntimeError\n                .into_err()\n                .reason(err)\n                .details(\"Failed to send request\")\n        })\n        .map(|response| {\n            response\n                .headers()\n                .get(header.as_ref())\n                .and_then(|h| h.to_str().ok())\n                .map(|h| Variable::from(h.to_string()))\n                .unwrap_or_default()\n        })\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/llm_prompt.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse directory::Permission;\nuse sieve::{FunctionMap, compiler::Number, runtime::Variable};\nuse trc::{AiEvent, SecurityEvent};\n\nuse super::PluginContext;\n\npub fn register(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"llm_prompt\", plugin_id, 3);\n}\n\npub async fn exec(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n\n    #[cfg(feature = \"enterprise\")]\n    if let (Variable::String(name), Variable::String(prompt)) =\n        (&ctx.arguments[0], &ctx.arguments[1])\n    {\n        #[cfg(feature = \"test_mode\")]\n        if name.as_ref() == \"echo-test\" {\n            return Ok(prompt.to_string().into());\n        }\n        let temperature = ctx.arguments[2].to_number_checked().map(|n| match n {\n            Number::Integer(n) => (n as f64).clamp(0.0, 1.0),\n            Number::Float(n) => n.clamp(0.0, 1.0),\n        });\n\n        if let Some(ai_api) = ctx.server.core.enterprise.as_ref().and_then(|e| {\n            if ctx.access_token.is_none_or(|token| {\n                if token.has_permission(Permission::AiModelInteract) {\n                    true\n                } else {\n                    trc::event!(\n                        Security(SecurityEvent::Unauthorized),\n                        AccountId = token.primary_id(),\n                        Details = Permission::AiModelInteract.name(),\n                        SpanId = ctx.session_id,\n                    );\n                    false\n                }\n            }) {\n                if e.ai_apis.len() == 1 && name.is_empty() {\n                    e.ai_apis.values().next()\n                } else {\n                    e.ai_apis.get(name.as_ref())\n                }\n            } else {\n                None\n            }\n        }) {\n            let time = Instant::now();\n            match ai_api.send_request(prompt.as_ref(), temperature).await {\n                Ok(response) => {\n                    trc::event!(\n                        Ai(AiEvent::LlmResponse),\n                        Id = ai_api.id.clone(),\n                        Value = prompt.to_string(),\n                        Details = response.clone(),\n                        Elapsed = time.elapsed(),\n                        SpanId = ctx.session_id,\n                    );\n\n                    return Ok(response.into());\n                }\n                Err(err) => {\n                    trc::error!(err.span_id(ctx.session_id));\n                }\n            }\n        }\n    }\n\n    // SPDX-SnippetEnd\n\n    Ok(false.into())\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::PluginContext;\nuse crate::scripts::into_sieve_value;\nuse sieve::{FunctionMap, runtime::Variable};\nuse store::{Deserialize, Value, dispatch::lookup::KeyValue};\n\npub fn register(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"key_exists\", plugin_id, 2);\n}\n\npub fn register_get(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"key_get\", plugin_id, 2);\n}\n\npub fn register_set(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"key_set\", plugin_id, 4);\n}\n\npub fn register_local_domain(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"is_local_domain\", plugin_id, 2);\n}\n\npub async fn exec(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let store = match &ctx.arguments[0] {\n        Variable::String(v) if !v.is_empty() => ctx.server.core.storage.lookups.get(v.as_ref()),\n        _ => Some(&ctx.server.core.storage.lookup),\n    }\n    .ok_or_else(|| {\n        trc::SieveEvent::RuntimeError\n            .ctx(trc::Key::Id, ctx.arguments[0].to_string().into_owned())\n            .details(\"Unknown store\")\n    })?;\n\n    Ok(match &ctx.arguments[1] {\n        Variable::Array(items) => {\n            for item in items.iter() {\n                if !item.is_empty() && store.key_exists(item.to_string()).await? {\n                    return Ok(true.into());\n                }\n            }\n            false\n        }\n        v if !v.is_empty() => store.key_exists(v.to_string()).await?,\n        _ => false,\n    }\n    .into())\n}\n\npub async fn exec_get(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    match &ctx.arguments[0] {\n        Variable::String(v) if !v.is_empty() => ctx.server.core.storage.lookups.get(v.as_ref()),\n        _ => Some(&ctx.server.core.storage.lookup),\n    }\n    .ok_or_else(|| {\n        trc::SieveEvent::RuntimeError\n            .ctx(trc::Key::Id, ctx.arguments[0].to_string().into_owned())\n            .details(\"Unknown store\")\n    })?\n    .key_get::<VariableWrapper>(ctx.arguments[1].to_string())\n    .await\n    .map(|v| v.map(|v| v.into_inner()).unwrap_or_default())\n}\n\npub async fn exec_set(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let expires = match &ctx.arguments[3] {\n        Variable::Integer(v) => Some(*v as u64),\n        Variable::Float(v) => Some(*v as u64),\n        _ => None,\n    };\n\n    match &ctx.arguments[0] {\n        Variable::String(v) if !v.is_empty() => ctx.server.core.storage.lookups.get(v.as_ref()),\n        _ => Some(&ctx.server.core.storage.lookup),\n    }\n    .ok_or_else(|| {\n        trc::SieveEvent::RuntimeError\n            .ctx(trc::Key::Id, ctx.arguments[0].to_string().into_owned())\n            .details(\"Unknown store\")\n    })?\n    .key_set(\n        KeyValue::new(\n            ctx.arguments[1].to_string().into_owned().into_bytes(),\n            if !ctx.arguments[2].is_empty() {\n                bincode::serde::encode_to_vec(&ctx.arguments[2], bincode::config::standard())\n                    .unwrap_or_default()\n            } else {\n                vec![]\n            },\n        )\n        .expires_opt(expires),\n    )\n    .await\n    .map(|_| true.into())\n}\n\npub async fn exec_local_domain(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let domain = ctx.arguments[1].to_string();\n\n    if !domain.is_empty() {\n        return match &ctx.arguments[0] {\n            Variable::String(v) if !v.is_empty() => {\n                ctx.server.core.storage.directories.get(v.as_ref())\n            }\n            _ => Some(&ctx.server.core.storage.directory),\n        }\n        .ok_or_else(|| {\n            trc::SieveEvent::RuntimeError\n                .ctx(trc::Key::Id, ctx.arguments[0].to_string().into_owned())\n                .details(\"Unknown directory\")\n        })?\n        .is_local_domain(domain.as_ref())\n        .await\n        .map(Into::into);\n    }\n\n    Ok(Variable::default())\n}\n\n#[derive(Debug, PartialEq, Eq)]\npub struct VariableWrapper(Variable);\n\nimpl Deserialize for VariableWrapper {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(VariableWrapper(\n            bincode::serde::decode_from_slice::<Variable, _>(bytes, bincode::config::standard())\n                .map(|v| v.0)\n                .unwrap_or_else(|_| {\n                    Variable::String(String::from_utf8_lossy(bytes).into_owned().into())\n                }),\n        ))\n    }\n}\n\nimpl From<i64> for VariableWrapper {\n    fn from(value: i64) -> Self {\n        VariableWrapper(value.into())\n    }\n}\n\nimpl VariableWrapper {\n    pub fn into_inner(self) -> Variable {\n        self.0\n    }\n}\n\nimpl From<Value<'static>> for VariableWrapper {\n    fn from(value: Value<'static>) -> Self {\n        VariableWrapper(into_sieve_value(value))\n    }\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod dns;\npub mod exec;\npub mod headers;\npub mod http;\npub mod llm_prompt;\npub mod lookup;\npub mod query;\npub mod text;\n\nuse mail_parser::Message;\nuse sieve::{FunctionMap, Input, runtime::Variable};\n\nuse crate::{Core, Server, auth::AccessToken};\n\nuse super::ScriptModification;\n\ntype RegisterPluginFnc = fn(u32, &mut FunctionMap) -> ();\n\npub struct PluginContext<'x> {\n    pub session_id: u64,\n    pub access_token: Option<&'x AccessToken>,\n    pub server: &'x Server,\n    pub message: &'x Message<'x>,\n    pub modifications: &'x mut Vec<ScriptModification>,\n    pub arguments: Vec<Variable>,\n}\n\nconst PLUGINS_REGISTER: [RegisterPluginFnc; 13] = [\n    query::register,\n    exec::register,\n    lookup::register,\n    lookup::register_get,\n    lookup::register_set,\n    lookup::register_local_domain,\n    dns::register,\n    dns::register_exists,\n    http::register_header,\n    headers::register,\n    text::register_tokenize,\n    text::register_domain_part,\n    llm_prompt::register,\n];\n\npub trait RegisterSievePlugins {\n    fn register_plugins_trusted(self) -> Self;\n    fn register_plugins_untrusted(self) -> Self;\n}\n\nimpl RegisterSievePlugins for FunctionMap {\n    fn register_plugins_trusted(mut self) -> Self {\n        #[cfg(feature = \"test_mode\")]\n        {\n            self.set_external_function(\"print\", PLUGINS_REGISTER.len() as u32, 1)\n        }\n\n        for (i, fnc) in PLUGINS_REGISTER.iter().enumerate() {\n            fnc(i as u32, &mut self);\n        }\n        self\n    }\n\n    fn register_plugins_untrusted(mut self) -> Self {\n        llm_prompt::register(12, &mut self);\n        self\n    }\n}\n\nimpl Core {\n    pub async fn run_plugin(&self, id: u32, ctx: PluginContext<'_>) -> Input {\n        #[cfg(feature = \"test_mode\")]\n        if id == PLUGINS_REGISTER.len() as u32 {\n            return test_print(ctx);\n        }\n\n        let session_id = ctx.session_id;\n        let result = match id {\n            0 => query::exec(ctx).await,\n            1 => exec::exec(ctx).await,\n            2 => lookup::exec(ctx).await,\n            3 => lookup::exec_get(ctx).await,\n            4 => lookup::exec_set(ctx).await,\n            5 => lookup::exec_local_domain(ctx).await,\n            6 => dns::exec(ctx).await,\n            7 => dns::exec_exists(ctx).await,\n            8 => http::exec_header(ctx).await,\n            9 => headers::exec(ctx),\n            10 => text::exec_tokenize(ctx),\n            11 => text::exec_domain_part(ctx),\n            12 => llm_prompt::exec(ctx).await,\n            _ => unreachable!(),\n        };\n\n        match result {\n            Ok(result) => result.into(),\n            Err(err) => {\n                trc::error!(err.span_id(session_id).details(\"Sieve runtime error\"));\n                Input::FncResult(Variable::default())\n            }\n        }\n    }\n}\n\n#[cfg(feature = \"test_mode\")]\npub fn test_print(ctx: PluginContext<'_>) -> Input {\n    println!(\"{}\", ctx.arguments[0].to_string());\n    Input::True\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::cmp::Ordering;\n\nuse crate::scripts::{into_sieve_value, to_store_value};\nuse sieve::{FunctionMap, runtime::Variable};\nuse store::{Rows, Value};\n\nuse super::PluginContext;\n\npub fn register(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"query\", plugin_id, 3);\n}\n\npub async fn exec(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    // Obtain store name\n    let store = match &ctx.arguments[0] {\n        Variable::String(v) if !v.is_empty() => ctx.server.core.storage.stores.get(v.as_ref()),\n        _ => Some(&ctx.server.core.storage.data),\n    }\n    .ok_or_else(|| {\n        trc::SieveEvent::RuntimeError\n            .ctx(trc::Key::Id, ctx.arguments[0].to_string().into_owned())\n            .details(\"Unknown store\")\n    })?;\n\n    // Obtain query string\n    let query = ctx.arguments[1].to_string();\n    if query.is_empty() {\n        trc::bail!(\n            trc::SieveEvent::RuntimeError\n                .ctx(trc::Key::Id, ctx.arguments[0].to_string().into_owned())\n                .details(\"Empty query string\")\n        );\n    }\n\n    // Obtain arguments\n    let arguments = match &ctx.arguments[2] {\n        Variable::Array(l) => l.iter().map(to_store_value).collect(),\n        v => vec![to_store_value(v)],\n    };\n\n    // Run query\n    if query\n        .as_bytes()\n        .get(..6)\n        .is_some_and(|q| q.eq_ignore_ascii_case(b\"SELECT\"))\n    {\n        let mut rows = store.sql_query::<Rows>(&query, arguments).await?;\n        Ok(match rows.rows.len().cmp(&1) {\n            Ordering::Equal => {\n                let mut row = rows.rows.pop().unwrap().values;\n                match row.len().cmp(&1) {\n                    Ordering::Equal if !matches!(row.first(), Some(Value::Null)) => {\n                        row.pop().map(into_sieve_value).unwrap()\n                    }\n                    Ordering::Less => Variable::default(),\n                    _ => Variable::Array(\n                        row.into_iter()\n                            .map(into_sieve_value)\n                            .collect::<Vec<_>>()\n                            .into(),\n                    ),\n                }\n            }\n            Ordering::Less => Variable::default(),\n            Ordering::Greater => rows\n                .rows\n                .into_iter()\n                .map(|r| {\n                    Variable::Array(\n                        r.values\n                            .into_iter()\n                            .map(into_sieve_value)\n                            .collect::<Vec<_>>()\n                            .into(),\n                    )\n                })\n                .collect::<Vec<_>>()\n                .into(),\n        })\n    } else {\n        Ok(store\n            .sql_query::<usize>(&query, arguments)\n            .await\n            .is_ok()\n            .into())\n    }\n}\n"
  },
  {
    "path": "crates/common/src/scripts/plugins/text.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse nlp::tokenizers::types::{TokenType, TypesTokenizer};\nuse sieve::{FunctionMap, runtime::Variable};\n\nuse crate::scripts::functions::{ApplyString, text::tokenize_words};\n\nuse super::PluginContext;\n\npub fn register_tokenize(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"tokenize\", plugin_id, 2);\n}\n\npub fn register_domain_part(plugin_id: u32, fnc_map: &mut FunctionMap) {\n    fnc_map.set_external_function(\"domain_part\", plugin_id, 2);\n}\n\npub fn exec_tokenize(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let mut v = ctx.arguments;\n    let (urls, urls_without_scheme, emails) = match v[1].to_string().as_ref() {\n        \"words\" => return Ok(tokenize_words(&v[0])),\n        \"uri\" | \"url\" => (true, true, true),\n        \"uri_strict\" | \"url_strict\" => (true, false, false),\n        \"email\" => (false, false, true),\n        _ => return Ok(Variable::default()),\n    };\n\n    Ok(match v.remove(0) {\n        v @ (Variable::String(_) | Variable::Array(_)) => {\n            TypesTokenizer::new(v.to_string().as_ref())\n                .tokenize_numbers(false)\n                .tokenize_urls(urls)\n                .tokenize_urls_without_scheme(urls_without_scheme)\n                .tokenize_emails(emails)\n                .filter_map(|t| match t.word {\n                    TokenType::Url(text) if urls => Variable::from(text.to_string()).into(),\n                    TokenType::UrlNoScheme(text) if urls_without_scheme => {\n                        Variable::from(format!(\"https://{text}\")).into()\n                    }\n                    TokenType::Email(text) if emails => Variable::from(text.to_string()).into(),\n                    _ => None,\n                })\n                .collect::<Vec<_>>()\n                .into()\n        }\n        v => v,\n    })\n}\n\nenum DomainPart {\n    Sld,\n    Tld,\n    Host,\n}\n\npub fn exec_domain_part(ctx: PluginContext<'_>) -> trc::Result<Variable> {\n    let v = ctx.arguments;\n    let part = match v[1].to_string().as_ref() {\n        \"sld\" => DomainPart::Sld,\n        \"tld\" => DomainPart::Tld,\n        \"host\" => DomainPart::Host,\n        _ => return Ok(Variable::default()),\n    };\n\n    Ok(v[0].transform(|domain| {\n        match part {\n            DomainPart::Sld => psl::domain_str(domain),\n            DomainPart::Tld => domain.rsplit_once('.').map(|(_, tld)| tld),\n            DomainPart::Host => domain.split_once('.').map(|(host, _)| host),\n        }\n        .map(Variable::from)\n        .unwrap_or_default()\n    }))\n}\n"
  },
  {
    "path": "crates/common/src/sharing/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::Server;\nuse directory::{\n    Type,\n    backend::internal::{PrincipalField, manage::ChangedPrincipals},\n};\nuse types::acl::{AclGrant, ArchivedAclGrant};\n\nimpl Server {\n    pub async fn refresh_acls(&self, acl_changes: &[AclGrant], current: Option<&[AclGrant]>) {\n        let mut changed_principals = ChangedPrincipals::new();\n        if let Some(acl_current) = current {\n            for current_item in acl_current {\n                let mut invalidate = true;\n                for change_item in acl_changes {\n                    if change_item.account_id == current_item.account_id {\n                        invalidate = change_item.grants != current_item.grants;\n                        break;\n                    }\n                }\n                if invalidate {\n                    changed_principals.add_change(\n                        current_item.account_id,\n                        Type::Individual,\n                        PrincipalField::EnabledPermissions,\n                    );\n                }\n            }\n\n            for change_item in acl_changes {\n                let mut invalidate = true;\n                for current_item in acl_current {\n                    if change_item.account_id == current_item.account_id {\n                        invalidate = change_item.grants != current_item.grants;\n                        break;\n                    }\n                }\n                if invalidate {\n                    changed_principals.add_change(\n                        change_item.account_id,\n                        Type::Individual,\n                        PrincipalField::EnabledPermissions,\n                    );\n                }\n            }\n        } else {\n            for value in acl_changes {\n                changed_principals.add_change(\n                    value.account_id,\n                    Type::Individual,\n                    PrincipalField::EnabledPermissions,\n                );\n            }\n        }\n\n        self.invalidate_principal_caches(changed_principals).await;\n    }\n\n    pub async fn refresh_archived_acls(\n        &self,\n        acl_changes: &[AclGrant],\n        acl_current: &[ArchivedAclGrant],\n    ) {\n        let mut changed_principals = ChangedPrincipals::new();\n        for current_item in acl_current.iter() {\n            let mut invalidate = true;\n            for change_item in acl_changes {\n                if change_item.account_id == current_item.account_id {\n                    invalidate = change_item.grants != current_item.grants;\n                    break;\n                }\n            }\n            if invalidate {\n                changed_principals.add_change(\n                    current_item.account_id.to_native(),\n                    Type::Individual,\n                    PrincipalField::EnabledPermissions,\n                );\n            }\n        }\n\n        for change_item in acl_changes {\n            let mut invalidate = true;\n            for current_item in acl_current.iter() {\n                if change_item.account_id == current_item.account_id {\n                    invalidate = change_item.grants != current_item.grants;\n                    break;\n                }\n            }\n            if invalidate {\n                changed_principals.add_change(\n                    change_item.account_id,\n                    Type::Individual,\n                    PrincipalField::EnabledPermissions,\n                );\n            }\n        }\n\n        self.invalidate_principal_caches(changed_principals).await;\n    }\n}\n"
  },
  {
    "path": "crates/common/src/sharing/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::auth::AccessToken;\nuse rkyv::vec::ArchivedVec;\nuse types::acl::{Acl, AclGrant, ArchivedAclGrant};\nuse utils::map::bitmap::Bitmap;\n\npub mod acl;\npub mod notification;\npub mod resources;\n\npub trait EffectiveAcl {\n    fn effective_acl(&self, access_token: &AccessToken) -> Bitmap<Acl>;\n}\n\nimpl EffectiveAcl for Vec<AclGrant> {\n    fn effective_acl(&self, access_token: &AccessToken) -> Bitmap<Acl> {\n        self.as_slice().effective_acl(access_token)\n    }\n}\n\nimpl EffectiveAcl for &[AclGrant] {\n    fn effective_acl(&self, access_token: &AccessToken) -> Bitmap<Acl> {\n        let mut acl = Bitmap::<Acl>::new();\n        for item in self.iter() {\n            if access_token.is_member(item.account_id) {\n                acl.union(&item.grants);\n            }\n        }\n\n        acl\n    }\n}\n\nimpl EffectiveAcl for ArchivedVec<ArchivedAclGrant> {\n    fn effective_acl(&self, access_token: &AccessToken) -> Bitmap<Acl> {\n        let mut acl = Bitmap::<Acl>::new();\n        for item in self.iter() {\n            if access_token.is_member(item.account_id.into()) {\n                acl.union_raw(item.grants.bitmap);\n            }\n        }\n\n        acl\n    }\n}\n"
  },
  {
    "path": "crates/common/src/sharing/notification.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse store::{Deserialize, SerializeInfallible, U32_LEN, U64_LEN, write::key::KeySerializer};\nuse types::{acl::Acl, collection::Collection};\nuse utils::map::bitmap::Bitmap;\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\npub struct ShareNotification {\n    pub object_account_id: u32,\n    pub object_id: u32,\n    pub object_type: Collection,\n    pub changed_by: u32,\n    pub old_rights: Bitmap<Acl>,\n    pub new_rights: Bitmap<Acl>,\n    pub name: String,\n}\n\nimpl SerializeInfallible for ShareNotification {\n    fn serialize(&self) -> Vec<u8> {\n        KeySerializer::new(U64_LEN * 2 + U32_LEN * 3 + 1 + self.name.len())\n            .write(self.object_account_id)\n            .write(self.object_id)\n            .write(self.object_type as u8)\n            .write(self.changed_by)\n            .write(self.old_rights.bitmap)\n            .write(self.new_rights.bitmap)\n            .write(self.name.as_bytes())\n            .finalize()\n    }\n}\n\nimpl Deserialize for ShareNotification {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Self::deserialize_from_slice(bytes)\n            .ok_or(trc::StoreEvent::DataCorruption.caused_by(trc::location!()))\n    }\n}\n\nimpl ShareNotification {\n    fn deserialize_from_slice(bytes: &[u8]) -> Option<Self> {\n        Some(Self {\n            object_account_id: bytes\n                .get(..U32_LEN)\n                .and_then(|b| b.try_into().ok())\n                .map(u32::from_be_bytes)?,\n            object_id: bytes\n                .get(U32_LEN..U32_LEN * 2)\n                .and_then(|b| b.try_into().ok())\n                .map(u32::from_be_bytes)?,\n            object_type: bytes.get(U32_LEN * 2).copied().map(Collection::from)?,\n            changed_by: bytes\n                .get(U32_LEN * 2 + 1..U32_LEN * 3 + 1)\n                .and_then(|b| b.try_into().ok())\n                .map(u32::from_be_bytes)?,\n            old_rights: bytes\n                .get(U32_LEN * 3 + 1..U32_LEN * 3 + U64_LEN + 1)\n                .and_then(|b| b.try_into().ok())\n                .map(u64::from_be_bytes)\n                .map(Bitmap::from)?,\n            new_rights: bytes\n                .get(U32_LEN * 3 + U64_LEN + 1..U32_LEN * 3 + U64_LEN * 2 + 1)\n                .and_then(|b| b.try_into().ok())\n                .map(u64::from_be_bytes)\n                .map(Bitmap::from)?,\n            name: bytes\n                .get(U32_LEN * 3 + U64_LEN * 2 + 1..)\n                .and_then(|b| String::from_utf8(b.to_vec()).ok())\n                .unwrap_or_default(),\n        })\n    }\n}\n"
  },
  {
    "path": "crates/common/src/sharing/resources.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{DavResources, auth::AccessToken};\nuse store::roaring::RoaringBitmap;\nuse types::acl::Acl;\nuse utils::map::bitmap::Bitmap;\n\nimpl DavResources {\n    pub fn shared_containers(\n        &self,\n        access_token: &AccessToken,\n        check_acls: impl IntoIterator<Item = Acl>,\n        match_any: bool,\n    ) -> RoaringBitmap {\n        let check_acls = Bitmap::<Acl>::from_iter(check_acls);\n        let mut document_ids = RoaringBitmap::new();\n\n        for resource in &self.resources {\n            if let Some(acls) = resource.acls() {\n                for acl in acls {\n                    if access_token.is_member(acl.account_id) {\n                        let mut grants = acl.grants;\n                        grants.intersection(&check_acls);\n                        if grants == check_acls || (match_any && !grants.is_empty()) {\n                            document_ids.insert(resource.document_id);\n                        }\n                    }\n                }\n            }\n        }\n\n        document_ids\n    }\n\n    pub fn shared_items(\n        &self,\n        access_token: &AccessToken,\n        check_acls: impl IntoIterator<Item = Acl>,\n        match_any: bool,\n    ) -> RoaringBitmap {\n        let shared_containers = self.shared_containers(access_token, check_acls, match_any);\n\n        if !shared_containers.is_empty() {\n            let mut document_ids = RoaringBitmap::new();\n\n            for path in &self.paths {\n                if let Some(parent_id) = path.parent_id\n                    && shared_containers.contains(parent_id)\n                {\n                    document_ids.insert(self.resources[path.resource_idx].document_id);\n                }\n            }\n\n            document_ids\n        } else {\n            shared_containers\n        }\n    }\n\n    pub fn has_access_to_container(\n        &self,\n        access_token: &AccessToken,\n        document_id: u32,\n        check_acls: impl Into<Bitmap<Acl>>,\n    ) -> bool {\n        let check_acls = check_acls.into();\n\n        for resource in &self.resources {\n            if resource.document_id == document_id\n                && let Some(acls) = resource.acls()\n            {\n                for acl in acls {\n                    if access_token.is_member(acl.account_id) {\n                        let mut grants = acl.grants;\n                        grants.intersection(&check_acls);\n                        return !grants.is_empty();\n                    }\n                }\n                break;\n            }\n        }\n\n        false\n    }\n\n    pub fn container_acl(&self, access_token: &AccessToken, document_id: u32) -> Bitmap<Acl> {\n        let mut account_acls = Bitmap::<Acl>::new();\n\n        for resource in &self.resources {\n            if resource.document_id == document_id\n                && let Some(acls) = resource.acls()\n            {\n                for acl in acls {\n                    if access_token.is_member(acl.account_id) {\n                        account_acls.union(&acl.grants);\n                    }\n                }\n                break;\n            }\n        }\n\n        account_acls\n    }\n\n    pub fn document_ids(&self, is_container: bool) -> impl Iterator<Item = u32> {\n        self.resources.iter().filter_map(move |resource| {\n            if resource.is_container() == is_container {\n                Some(resource.document_id)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn has_container_id(&self, id: &u32) -> bool {\n        self.resources\n            .iter()\n            .any(|r| r.document_id == *id && r.is_container())\n    }\n\n    pub fn has_item_id(&self, id: &u32) -> bool {\n        self.resources\n            .iter()\n            .any(|r| r.document_id == *id && !r.is_container())\n    }\n}\n"
  },
  {
    "path": "crates/common/src/storage/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::Server;\nuse mail_parser::{\n    Encoding,\n    decoders::{base64::base64_decode, quoted_printable::quoted_printable_decode},\n};\nuse types::{blob::BlobSection, blob_hash::BlobHash};\n\nimpl Server {\n    pub async fn get_blob_section(\n        &self,\n        hash: &BlobHash,\n        section: &BlobSection,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        Ok(self\n            .blob_store()\n            .get_blob(\n                hash.as_slice(),\n                (section.offset_start)..(section.offset_start.saturating_add(section.size)),\n            )\n            .await?\n            .and_then(|bytes| match Encoding::from(section.encoding) {\n                Encoding::None => Some(bytes),\n                Encoding::Base64 => base64_decode(&bytes),\n                Encoding::QuotedPrintable => quoted_printable_decode(&bytes),\n            }))\n    }\n}\n"
  },
  {
    "path": "crates/common/src/storage/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{auth::AccessToken, sharing::notification::ShareNotification};\nuse rkyv::{\n    option::ArchivedOption,\n    primitive::{ArchivedU32, ArchivedU64},\n    string::ArchivedString,\n};\nuse std::{borrow::Cow, fmt::Debug};\nuse store::{\n    Serialize, SerializeInfallible,\n    write::{\n        Archive, Archiver, BatchBuilder, BlobLink, BlobOp, DirectoryClass, IntoOperations, Params,\n        SearchIndex, TaskEpoch, TaskQueueClass, ValueClass,\n    },\n};\nuse types::{\n    acl::AclGrant,\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection},\n    field::Field,\n};\nuse utils::{cheeky_hash::CheekyHash, map::bitmap::Bitmap, snowflake::SnowflakeIdGenerator};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum IndexValue<'x> {\n    Index {\n        field: Field,\n        value: IndexItem<'x>,\n    },\n    Property {\n        field: ValueClass,\n        value: IndexItem<'x>,\n    },\n    SearchIndex {\n        index: SearchIndex,\n        hash: u64,\n    },\n    Blob {\n        value: BlobHash,\n    },\n    Quota {\n        used: u32,\n    },\n    LogContainer {\n        sync_collection: SyncCollection,\n    },\n    LogContainerProperty {\n        sync_collection: SyncCollection,\n        ids: Vec<u32>,\n    },\n    LogItem {\n        sync_collection: SyncCollection,\n        prefix: Option<u32>,\n    },\n    Acl {\n        value: Cow<'x, [AclGrant]>,\n    },\n}\n\n#[derive(Debug, Clone)]\npub enum IndexItem<'x> {\n    Vec(Vec<u8>),\n    Slice(&'x [u8]),\n    ShortInt([u8; std::mem::size_of::<u32>()]),\n    LongInt([u8; std::mem::size_of::<u64>()]),\n    Hash(CheekyHash),\n    None,\n}\n\nimpl IndexItem<'_> {\n    pub fn as_slice(&self) -> &[u8] {\n        match self {\n            IndexItem::Vec(v) => v,\n            IndexItem::Slice(s) => s,\n            IndexItem::ShortInt(s) => s,\n            IndexItem::LongInt(s) => s,\n            IndexItem::Hash(h) => h.as_bytes(),\n            IndexItem::None => &[],\n        }\n    }\n\n    pub fn into_owned(self) -> Vec<u8> {\n        match self {\n            IndexItem::Vec(v) => v,\n            IndexItem::Slice(s) => s.to_vec(),\n            IndexItem::ShortInt(s) => s.to_vec(),\n            IndexItem::LongInt(s) => s.to_vec(),\n            IndexItem::Hash(h) => h.as_bytes().to_vec(),\n            IndexItem::None => vec![],\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        match self {\n            IndexItem::Vec(v) => v.is_empty(),\n            IndexItem::Slice(s) => s.is_empty(),\n            IndexItem::None => true,\n            _ => false,\n        }\n    }\n\n    pub fn is_none(&self) -> bool {\n        matches!(self, IndexItem::None)\n    }\n\n    pub fn is_some(&self) -> bool {\n        !self.is_none()\n    }\n}\n\nimpl PartialEq for IndexItem<'_> {\n    fn eq(&self, other: &Self) -> bool {\n        self.as_slice() == other.as_slice()\n    }\n}\n\nimpl Eq for IndexItem<'_> {}\n\nimpl std::hash::Hash for IndexItem<'_> {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        match self {\n            IndexItem::Vec(v) => v.as_slice().hash(state),\n            IndexItem::Slice(s) => s.hash(state),\n            IndexItem::ShortInt(s) => s.as_slice().hash(state),\n            IndexItem::LongInt(s) => s.as_slice().hash(state),\n            IndexItem::Hash(h) => h.hash(state),\n            IndexItem::None => 0.hash(state),\n        }\n    }\n}\n\nimpl From<u32> for IndexItem<'_> {\n    fn from(value: u32) -> Self {\n        IndexItem::ShortInt(value.to_be_bytes())\n    }\n}\n\nimpl From<&u32> for IndexItem<'_> {\n    fn from(value: &u32) -> Self {\n        IndexItem::ShortInt(value.to_be_bytes())\n    }\n}\n\nimpl From<u64> for IndexItem<'_> {\n    fn from(value: u64) -> Self {\n        IndexItem::LongInt(value.to_be_bytes())\n    }\n}\n\nimpl From<i64> for IndexItem<'_> {\n    fn from(value: i64) -> Self {\n        IndexItem::LongInt(value.to_be_bytes())\n    }\n}\n\nimpl<'x> From<&'x [u8]> for IndexItem<'x> {\n    fn from(value: &'x [u8]) -> Self {\n        IndexItem::Slice(value)\n    }\n}\n\nimpl From<Vec<u8>> for IndexItem<'_> {\n    fn from(value: Vec<u8>) -> Self {\n        IndexItem::Vec(value)\n    }\n}\n\nimpl<'x> From<&'x str> for IndexItem<'x> {\n    fn from(value: &'x str) -> Self {\n        IndexItem::Slice(value.as_bytes())\n    }\n}\n\nimpl<'x> From<&'x String> for IndexItem<'x> {\n    fn from(value: &'x String) -> Self {\n        IndexItem::Slice(value.as_bytes())\n    }\n}\n\nimpl From<String> for IndexItem<'_> {\n    fn from(value: String) -> Self {\n        IndexItem::Vec(value.into_bytes())\n    }\n}\n\nimpl<'x> From<&'x ArchivedString> for IndexItem<'x> {\n    fn from(value: &'x ArchivedString) -> Self {\n        IndexItem::Slice(value.as_bytes())\n    }\n}\n\nimpl From<ArchivedU32> for IndexItem<'_> {\n    fn from(value: ArchivedU32) -> Self {\n        IndexItem::ShortInt(value.to_native().to_be_bytes())\n    }\n}\n\nimpl From<&ArchivedU32> for IndexItem<'_> {\n    fn from(value: &ArchivedU32) -> Self {\n        IndexItem::ShortInt(value.to_native().to_be_bytes())\n    }\n}\n\nimpl From<ArchivedU64> for IndexItem<'_> {\n    fn from(value: ArchivedU64) -> Self {\n        IndexItem::LongInt(value.to_native().to_be_bytes())\n    }\n}\n\nimpl<'x, T: Into<IndexItem<'x>>> From<Option<T>> for IndexItem<'x> {\n    fn from(value: Option<T>) -> Self {\n        match value {\n            Some(v) => v.into(),\n            None => IndexItem::None,\n        }\n    }\n}\n\nimpl<'x, T: Into<IndexItem<'x>>> From<ArchivedOption<T>> for IndexItem<'x> {\n    fn from(value: ArchivedOption<T>) -> Self {\n        match value {\n            ArchivedOption::Some(v) => v.into(),\n            ArchivedOption::None => IndexItem::None,\n        }\n    }\n}\n\npub trait IndexableObject: Sync + Send {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>>;\n}\n\npub trait IndexableAndSerializableObject:\n    IndexableObject\n    + rkyv::Archive\n    + for<'a> rkyv::Serialize<\n        rkyv::api::high::HighSerializer<\n            rkyv::util::AlignedVec,\n            rkyv::ser::allocator::ArenaHandle<'a>,\n            rkyv::rancor::Error,\n        >,\n    >\n{\n    fn is_versioned() -> bool;\n}\n\n#[derive(Debug)]\npub struct ObjectIndexBuilder<C: IndexableObject, N: IndexableAndSerializableObject> {\n    changed_by: u32,\n    tenant_id: Option<u32>,\n    current: Option<Archive<C>>,\n    changes: Option<N>,\n}\n\nimpl<C: IndexableObject, N: IndexableAndSerializableObject> Default for ObjectIndexBuilder<C, N> {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n\nimpl<C: IndexableObject, N: IndexableAndSerializableObject> ObjectIndexBuilder<C, N> {\n    pub fn new() -> Self {\n        Self {\n            current: None,\n            changes: None,\n            tenant_id: None,\n            changed_by: u32::MAX,\n        }\n    }\n\n    pub fn with_current(mut self, current: Archive<C>) -> Self {\n        self.current = Some(current);\n        self\n    }\n\n    pub fn with_changes(mut self, changes: N) -> Self {\n        self.changes = Some(changes);\n        self\n    }\n\n    pub fn with_current_opt(mut self, current: Option<Archive<C>>) -> Self {\n        self.current = current;\n        self\n    }\n\n    pub fn changes(&self) -> Option<&N> {\n        self.changes.as_ref()\n    }\n\n    pub fn changes_mut(&mut self) -> Option<&mut N> {\n        self.changes.as_mut()\n    }\n\n    pub fn current(&self) -> Option<&Archive<C>> {\n        self.current.as_ref()\n    }\n\n    pub fn with_access_token(mut self, access_token: &AccessToken) -> Self {\n        self.tenant_id = access_token.tenant.as_ref().map(|t| t.id);\n        self.changed_by = access_token.primary_id();\n        self\n    }\n\n    pub fn with_tenant_id(mut self, tenant_id: Option<u32>) -> Self {\n        self.tenant_id = tenant_id;\n        self\n    }\n}\n\nimpl<C: IndexableObject, N: IndexableAndSerializableObject> IntoOperations\n    for ObjectIndexBuilder<C, N>\n{\n    fn build(self, batch: &mut BatchBuilder) -> trc::Result<()> {\n        match (self.current, self.changes) {\n            (None, Some(changes)) => {\n                // Insertion\n                for item in changes.index_values() {\n                    build_index(batch, item, self.changed_by, self.tenant_id, true);\n                }\n                if N::is_versioned() {\n                    let (offset, bytes) = Archiver::new(changes).serialize_versioned()?;\n                    batch.set_fnc(\n                        Field::ARCHIVE,\n                        Params::with_capacity(2).with_bytes(bytes).with_u64(offset),\n                        |params, ids| {\n                            let change_id = ids.current_change_id()?;\n                            let archive = params.bytes(0);\n                            let offset = params.u64(1);\n\n                            let mut bytes = Vec::with_capacity(archive.len());\n                            bytes.extend_from_slice(&archive[..offset as usize]);\n                            bytes.extend_from_slice(&change_id.to_be_bytes()[..]);\n                            bytes.push(archive.last().copied().unwrap()); // Marker\n                            Ok(bytes)\n                        },\n                    );\n                } else {\n                    batch.set(Field::ARCHIVE, Archiver::new(changes).serialize()?);\n                }\n            }\n            (Some(current), Some(changes)) => {\n                // Update\n                batch.assert_value(Field::ARCHIVE, &current);\n                for (current, change) in current.inner.index_values().zip(changes.index_values()) {\n                    if current != change {\n                        merge_index(batch, current, change, self.changed_by, self.tenant_id)?;\n                    } else {\n                        match current {\n                            IndexValue::LogContainer { sync_collection } => {\n                                batch.log_container_update(sync_collection);\n                            }\n                            IndexValue::LogItem {\n                                sync_collection,\n                                prefix,\n                            } => {\n                                batch.log_item_update(sync_collection, prefix);\n                            }\n                            _ => (),\n                        }\n                    }\n                }\n                if N::is_versioned() {\n                    let (offset, bytes) = Archiver::new(changes).serialize_versioned()?;\n                    batch.set_fnc(\n                        Field::ARCHIVE,\n                        Params::with_capacity(2).with_bytes(bytes).with_u64(offset),\n                        |params, ids| {\n                            let change_id = ids.current_change_id()?;\n                            let archive = params.bytes(0);\n                            let offset = params.u64(1);\n\n                            let mut bytes = Vec::with_capacity(archive.len());\n                            bytes.extend_from_slice(&archive[..offset as usize]);\n                            bytes.extend_from_slice(&change_id.to_be_bytes()[..]);\n                            bytes.push(archive.last().copied().unwrap()); // Marker\n                            Ok(bytes)\n                        },\n                    );\n                } else {\n                    batch.set(Field::ARCHIVE, Archiver::new(changes).serialize()?);\n                }\n            }\n            (Some(current), None) => {\n                // Deletion\n                batch.assert_value(Field::ARCHIVE, &current);\n                for item in current.inner.index_values() {\n                    build_index(batch, item, self.changed_by, self.tenant_id, false);\n                }\n\n                batch.clear(Field::ARCHIVE);\n            }\n            (None, None) => unreachable!(),\n        }\n\n        Ok(())\n    }\n}\n\nfn build_index(\n    batch: &mut BatchBuilder,\n    item: IndexValue<'_>,\n    changed_by: u32,\n    tenant_id: Option<u32>,\n    set: bool,\n) {\n    match item {\n        IndexValue::Index { field, value } => {\n            if !value.is_empty() {\n                if set {\n                    batch.index(field, value.into_owned());\n                } else {\n                    batch.unindex(field, value.into_owned());\n                }\n            }\n        }\n        IndexValue::SearchIndex { index, .. } => {\n            batch.set(\n                ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                    due: TaskEpoch::now().with_random_sequence_id(),\n                    index,\n                    is_insert: set,\n                }),\n                vec![],\n            );\n        }\n        IndexValue::Property { field, value } => {\n            if !value.is_none() {\n                if set {\n                    batch.set(field, value.into_owned());\n                } else {\n                    batch.clear(field);\n                }\n            }\n        }\n        IndexValue::Blob { value } => {\n            if set {\n                batch.set(\n                    BlobOp::Link {\n                        hash: value,\n                        to: BlobLink::Document,\n                    },\n                    vec![],\n                );\n            } else {\n                batch.clear(BlobOp::Link {\n                    hash: value,\n                    to: BlobLink::Document,\n                });\n            }\n        }\n        IndexValue::Acl { value } => {\n            let object_account_id = batch.last_account_id().unwrap_or_default();\n            let object_type = batch.last_collection().unwrap_or(Collection::None);\n            let object_id = batch.last_document_id().unwrap_or_default();\n            let notification_id = SnowflakeIdGenerator::from_sequence_and_node_id(\n                object_type as u64 ^ object_account_id as u64,\n                None,\n            )\n            .unwrap_or_default();\n\n            for item in value.as_ref() {\n                if set {\n                    batch.acl_grant(item.account_id, item.grants.bitmap.serialize());\n                    batch.log_share_notification(\n                        notification_id,\n                        item.account_id,\n                        ShareNotification {\n                            object_account_id,\n                            object_id,\n                            object_type,\n                            changed_by,\n                            old_rights: Default::default(),\n                            new_rights: item.grants,\n                            name: Default::default(),\n                        },\n                    );\n                } else {\n                    batch.acl_revoke(item.account_id);\n                    batch.log_share_notification(\n                        notification_id,\n                        item.account_id,\n                        ShareNotification {\n                            object_account_id,\n                            object_id,\n                            object_type,\n                            changed_by,\n                            old_rights: item.grants,\n                            new_rights: Default::default(),\n                            name: Default::default(),\n                        },\n                    );\n                }\n            }\n        }\n        IndexValue::Quota { used } => {\n            let value = if set { used as i64 } else { -(used as i64) };\n\n            if let Some(account_id) = batch.last_account_id() {\n                batch.add(DirectoryClass::UsedQuota(account_id), value);\n            }\n\n            if let Some(tenant_id) = tenant_id {\n                batch.add(DirectoryClass::UsedQuota(tenant_id), value);\n            }\n        }\n        IndexValue::LogItem {\n            sync_collection,\n            prefix,\n        } => {\n            if set {\n                batch.log_item_insert(sync_collection, prefix);\n            } else {\n                batch.log_item_delete(sync_collection, prefix);\n            }\n        }\n        IndexValue::LogContainer { sync_collection } => {\n            if set {\n                batch.log_container_insert(sync_collection);\n            } else {\n                batch.log_container_delete(sync_collection);\n            }\n        }\n        IndexValue::LogContainerProperty {\n            sync_collection,\n            ids,\n        } => {\n            for parent_id in ids {\n                batch.log_container_property_change(sync_collection, parent_id);\n            }\n        }\n    }\n}\n\nfn merge_index(\n    batch: &mut BatchBuilder,\n    current: IndexValue<'_>,\n    change: IndexValue<'_>,\n    changed_by: u32,\n    tenant_id: Option<u32>,\n) -> trc::Result<()> {\n    match (current, change) {\n        (\n            IndexValue::Index {\n                field,\n                value: old_value,\n            },\n            IndexValue::Index {\n                value: new_value, ..\n            },\n        ) => {\n            if !old_value.is_empty() {\n                batch.unindex(field, old_value.into_owned());\n            }\n\n            if !new_value.is_empty() {\n                batch.index(field, new_value.into_owned());\n            }\n        }\n        (IndexValue::SearchIndex { index, .. }, IndexValue::SearchIndex { .. }) => {\n            batch.set(\n                ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                    due: TaskEpoch::now().with_random_sequence_id(),\n                    index,\n                    is_insert: true,\n                }),\n                vec![],\n            );\n        }\n        (\n            IndexValue::Property {\n                field: old_field,\n                value: old_value,\n            },\n            IndexValue::Property {\n                field: new_field,\n                value: new_value,\n                ..\n            },\n        ) => {\n            if old_field != new_field {\n                batch.clear(old_field);\n                batch.set(new_field, new_value.into_owned());\n            } else if new_value != old_value {\n                if new_value.is_some() {\n                    batch.set(old_field, new_value.into_owned());\n                } else {\n                    batch.clear(old_field);\n                }\n            }\n        }\n        (IndexValue::Blob { value: old_hash }, IndexValue::Blob { value: new_hash }) => {\n            batch.clear(BlobOp::Link {\n                hash: old_hash,\n                to: BlobLink::Document,\n            });\n            batch.set(\n                BlobOp::Link {\n                    hash: new_hash,\n                    to: BlobLink::Document,\n                },\n                vec![],\n            );\n        }\n        (IndexValue::Acl { value: old_acl }, IndexValue::Acl { value: new_acl }) => {\n            let has_old_acl = !old_acl.is_empty();\n            let has_new_acl = !new_acl.is_empty();\n\n            if !has_old_acl && !has_new_acl {\n                return Ok(());\n            }\n\n            let object_account_id = batch.last_account_id().unwrap_or_default();\n            let object_type = batch.last_collection().unwrap_or(Collection::None);\n            let object_id = batch.last_document_id().unwrap_or_default();\n            let notification_id = SnowflakeIdGenerator::from_sequence_and_node_id(\n                object_type as u64 ^ object_account_id as u64,\n                None,\n            )\n            .unwrap_or_default();\n\n            match (has_old_acl, has_new_acl) {\n                (true, true) => {\n                    // Remove deleted ACLs\n                    for current_item in old_acl.as_ref() {\n                        if !new_acl\n                            .iter()\n                            .any(|item| item.account_id == current_item.account_id)\n                        {\n                            batch.acl_revoke(current_item.account_id);\n                            batch.log_share_notification(\n                                notification_id,\n                                current_item.account_id,\n                                ShareNotification {\n                                    object_account_id,\n                                    object_id,\n                                    object_type,\n                                    changed_by,\n                                    old_rights: current_item.grants,\n                                    new_rights: Default::default(),\n                                    name: Default::default(),\n                                },\n                            );\n                        }\n                    }\n\n                    // Update ACLs\n                    for item in new_acl.as_ref() {\n                        let mut add_item = true;\n                        let mut old_rights = Bitmap::default();\n                        for current_item in old_acl.as_ref() {\n                            if item.account_id == current_item.account_id {\n                                if item.grants == current_item.grants {\n                                    add_item = false;\n                                } else {\n                                    old_rights = current_item.grants;\n                                }\n                                break;\n                            }\n                        }\n                        if add_item {\n                            batch.acl_grant(item.account_id, item.grants.bitmap.serialize());\n                            batch.log_share_notification(\n                                notification_id,\n                                item.account_id,\n                                ShareNotification {\n                                    object_account_id,\n                                    object_id,\n                                    object_type,\n                                    changed_by,\n                                    old_rights,\n                                    new_rights: item.grants,\n                                    name: Default::default(),\n                                },\n                            );\n                        }\n                    }\n                }\n                (false, true) => {\n                    // Add all ACLs\n                    for item in new_acl.as_ref() {\n                        batch.acl_grant(item.account_id, item.grants.bitmap.serialize());\n                        batch.log_share_notification(\n                            notification_id,\n                            item.account_id,\n                            ShareNotification {\n                                object_account_id,\n                                object_id,\n                                object_type,\n                                changed_by,\n                                old_rights: Default::default(),\n                                new_rights: item.grants,\n                                name: Default::default(),\n                            },\n                        );\n                    }\n                }\n                (true, false) => {\n                    // Remove all ACLs\n                    for item in old_acl.as_ref() {\n                        batch.acl_revoke(item.account_id);\n                        batch.log_share_notification(\n                            notification_id,\n                            item.account_id,\n                            ShareNotification {\n                                object_account_id,\n                                object_id,\n                                object_type,\n                                changed_by,\n                                old_rights: item.grants,\n                                new_rights: Default::default(),\n                                name: Default::default(),\n                            },\n                        );\n                    }\n                }\n                _ => {}\n            }\n        }\n        (IndexValue::Quota { used: old_used }, IndexValue::Quota { used: new_used }) => {\n            let value = new_used as i64 - old_used as i64;\n            if let Some(account_id) = batch.last_account_id() {\n                batch.add(DirectoryClass::UsedQuota(account_id), value);\n            }\n\n            if let Some(tenant_id) = tenant_id {\n                batch.add(DirectoryClass::UsedQuota(tenant_id), value);\n            }\n        }\n        (\n            IndexValue::LogItem {\n                sync_collection,\n                prefix: old_prefix,\n            },\n            IndexValue::LogItem {\n                prefix: new_prefix, ..\n            },\n        ) => {\n            batch.log_item_delete(sync_collection, old_prefix);\n            batch.log_item_insert(sync_collection, new_prefix);\n        }\n        (\n            IndexValue::LogContainerProperty {\n                sync_collection,\n                ids: old_ids,\n            },\n            IndexValue::LogContainerProperty { ids: new_ids, .. },\n        ) => {\n            for parent_id in &old_ids {\n                if !new_ids.contains(parent_id) {\n                    batch.log_container_property_change(sync_collection, *parent_id);\n                }\n            }\n            for parent_id in new_ids {\n                if !old_ids.contains(&parent_id) {\n                    batch.log_container_property_change(sync_collection, parent_id);\n                }\n            }\n        }\n        _ => unreachable!(),\n    }\n\n    Ok(())\n}\n\nimpl IndexableObject for () {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        std::iter::empty()\n    }\n}\n\nimpl IndexableAndSerializableObject for () {\n    fn is_versioned() -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/common/src/storage/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod blob;\npub mod index;\npub mod state;\n"
  },
  {
    "path": "crates/common/src/storage/state.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    IPC_CHANNEL_BUFFER, Server,\n    auth::AccessToken,\n    ipc::{PushEvent, PushNotification},\n};\nuse tokio::sync::mpsc;\nuse types::type_state::DataType;\nuse utils::map::bitmap::Bitmap;\n\nimpl Server {\n    pub async fn subscribe_push_manager(\n        &self,\n        access_token: &AccessToken,\n        types: Bitmap<DataType>,\n    ) -> trc::Result<mpsc::Receiver<PushNotification>> {\n        let (tx, rx) = mpsc::channel::<PushNotification>(IPC_CHANNEL_BUFFER);\n        let push_tx = self.inner.ipc.push_tx.clone();\n\n        push_tx\n            .send(PushEvent::Subscribe {\n                account_ids: access_token.member_ids().collect(),\n                types,\n                tx,\n            })\n            .await\n            .map_err(|err| {\n                trc::EventType::Server(trc::ServerEvent::ThreadError)\n                    .reason(err)\n                    .caused_by(trc::location!())\n            })?;\n\n        Ok(rx)\n    }\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/metrics/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod otel;\npub mod prometheus;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\npub mod store;\n// SPDX-SnippetEnd\n"
  },
  {
    "path": "crates/common/src/telemetry/metrics/otel.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::config::telemetry::OtelMetrics;\nuse opentelemetry_sdk::metrics::{\n    Temporality,\n    data::{\n        Gauge, GaugeDataPoint, Histogram, HistogramDataPoint, Metric, ResourceMetrics,\n        ScopeMetrics, Sum, SumDataPoint,\n    },\n    exporter::PushMetricExporter,\n};\nuse std::time::SystemTime;\nuse trc::{Collector, TelemetryEvent};\n\nimpl OtelMetrics {\n    pub async fn push_metrics(&self, is_enterprise: bool, start_time: SystemTime) {\n        let mut metrics = Vec::with_capacity(256);\n        let time = SystemTime::now();\n\n        // Add counters\n        for counter in Collector::collect_counters(is_enterprise) {\n            metrics.push(Metric {\n                name: counter.id().name().into(),\n                description: counter.id().description().into(),\n                unit: \"events\".into(),\n                data: Box::new(Sum {\n                    data_points: vec![SumDataPoint {\n                        attributes: vec![],\n                        value: counter.value(),\n                        exemplars: vec![],\n                    }],\n                    temporality: Temporality::Cumulative,\n                    is_monotonic: true,\n                    start_time,\n                    time,\n                }),\n            });\n        }\n\n        // Add gauges\n        for gauge in Collector::collect_gauges(is_enterprise) {\n            metrics.push(Metric {\n                name: gauge.id().name().into(),\n                description: gauge.id().description().into(),\n                unit: gauge.id().unit().into(),\n                data: Box::new(Gauge {\n                    data_points: vec![GaugeDataPoint {\n                        attributes: vec![],\n                        value: gauge.get(),\n                        exemplars: vec![],\n                    }],\n                    start_time: start_time.into(),\n                    time,\n                }),\n            });\n        }\n\n        // Add histograms\n        for histogram in Collector::collect_histograms(is_enterprise) {\n            metrics.push(Metric {\n                name: histogram.id().name().into(),\n                description: histogram.id().description().into(),\n                unit: histogram.id().unit().into(),\n                data: Box::new(Histogram {\n                    data_points: vec![HistogramDataPoint {\n                        attributes: vec![],\n                        count: histogram.count(),\n                        bounds: histogram.upper_bounds_vec(),\n                        bucket_counts: histogram.buckets_vec(),\n                        min: histogram.min(),\n                        max: histogram.max(),\n                        sum: histogram.sum(),\n                        exemplars: vec![],\n                    }],\n                    temporality: Temporality::Cumulative,\n                    start_time,\n                    time,\n                }),\n            });\n        }\n\n        // Export metrics\n        if let Err(err) = self\n            .exporter\n            .export(&mut ResourceMetrics {\n                resource: self.resource.clone(),\n                scope_metrics: vec![ScopeMetrics {\n                    scope: self.instrumentation.clone(),\n                    metrics,\n                }],\n            })\n            .await\n        {\n            trc::event!(\n                Telemetry(TelemetryEvent::OtelMetricsExporterError),\n                Reason = err.to_string(),\n            );\n        }\n    }\n\n    pub fn enable_errors() {\n        // TODO: Remove this when the OpenTelemetry SDK supports error handling\n        /*let _ = set_error_handler(|error| {\n            trc::event!(\n                Telemetry(TelemetryEvent::OtelMetricsExporterError),\n                Reason = error.to_string(),\n            );\n        });*/\n    }\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/metrics/prometheus.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse prometheus::{\n    TextEncoder,\n    proto::{Bucket, Counter, Gauge, Histogram, Metric, MetricFamily, MetricType},\n};\nuse trc::{Collector, atomics::histogram::AtomicHistogram};\n\nuse crate::Server;\n\nimpl Server {\n    pub async fn export_prometheus_metrics(&self) -> trc::Result<String> {\n        let mut metrics = Vec::new();\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n        #[cfg(feature = \"enterprise\")]\n        let is_enterprise = self.is_enterprise_edition();\n        // SPDX-SnippetEnd\n\n        #[cfg(not(feature = \"enterprise\"))]\n        let is_enterprise = false;\n\n        // Add counters\n        for counter in Collector::collect_counters(is_enterprise) {\n            let mut metric = MetricFamily::default();\n            metric.set_name(metric_name(counter.id().name()));\n            metric.set_help(counter.id().description().into());\n            metric.set_field_type(MetricType::COUNTER);\n            metric.set_metric(vec![new_counter(counter.value())]);\n            metrics.push(metric);\n        }\n\n        // Add gauges\n        for gauge in Collector::collect_gauges(is_enterprise) {\n            let mut metric = MetricFamily::default();\n            metric.set_name(metric_name(gauge.id().name()));\n            metric.set_help(gauge.id().description().into());\n            metric.set_field_type(MetricType::GAUGE);\n            metric.set_metric(vec![new_gauge(gauge.get())]);\n            metrics.push(metric);\n        }\n\n        // Add histograms\n        for histogram in Collector::collect_histograms(is_enterprise) {\n            let mut metric = MetricFamily::default();\n            metric.set_name(metric_name(histogram.id().name()));\n            metric.set_help(histogram.id().description().into());\n            metric.set_field_type(MetricType::HISTOGRAM);\n            metric.set_metric(vec![new_histogram(histogram)]);\n            metrics.push(metric);\n        }\n\n        TextEncoder::new().encode_to_string(&metrics).map_err(|e| {\n            trc::EventType::Telemetry(trc::TelemetryEvent::OtelExporterError).reason(e)\n        })\n    }\n}\n\nfn metric_name(id: impl AsRef<str>) -> String {\n    let id = id.as_ref();\n    let mut name = String::with_capacity(id.len());\n    for c in id.chars() {\n        if c.is_ascii_alphanumeric() {\n            name.push(c);\n        } else {\n            name.push('_');\n        }\n    }\n    name\n}\n\nfn new_counter(value: u64) -> Metric {\n    let mut m = Metric::default();\n    let mut counter = Counter::default();\n    counter.set_value(value as f64);\n    m.set_counter(counter);\n    m\n}\n\nfn new_gauge(value: u64) -> Metric {\n    let mut m = Metric::default();\n    let mut gauge = Gauge::default();\n    gauge.set_value(value as f64);\n    m.set_gauge(gauge);\n    m\n}\n\nfn new_histogram(histogram: &AtomicHistogram<12>) -> Metric {\n    let mut m = Metric::default();\n    let mut h = Histogram::default();\n    h.set_sample_count(histogram.count());\n    h.set_sample_sum(histogram.sum() as f64);\n    h.set_bucket(\n        histogram\n            .buckets_iter()\n            .into_iter()\n            .zip(histogram.upper_bounds_iter())\n            .map(|(count, upper_bound)| {\n                let mut b = Bucket::default();\n                b.set_cumulative_count(count);\n                b.set_upper_bound(if upper_bound != u64::MAX {\n                    upper_bound as f64\n                } else {\n                    f64::INFINITY\n                });\n                b\n            })\n            .collect(),\n    );\n    m.set_histogram(h);\n    m\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/metrics/store.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse std::{future::Future, sync::Arc, time::Duration};\n\nuse ahash::AHashMap;\nuse parking_lot::Mutex;\nuse serde::{Deserialize, Serialize};\nuse store::{\n    IterateParams, Store, U32_LEN, U64_LEN, ValueKey,\n    write::{\n        BatchBuilder, TelemetryClass, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n        now,\n    },\n};\nuse trc::*;\nuse utils::codec::leb128::Leb128Reader;\n\nuse crate::Core;\n\npub trait MetricsStore: Sync + Send {\n    fn write_metrics(\n        &self,\n        core: Arc<Core>,\n        timestamp: u64,\n        history: SharedMetricHistory,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n    fn query_metrics(\n        &self,\n        from_timestamp: u64,\n        to_timestamp: u64,\n    ) -> impl Future<Output = trc::Result<Vec<Metric<EventType, MetricType, u64>>>> + Send;\n    fn purge_metrics(&self, period: Duration) -> impl Future<Output = trc::Result<()>> + Send;\n}\n\n#[derive(Default)]\npub struct MetricsHistory {\n    events: AHashMap<EventType, u32>,\n    histograms: AHashMap<MetricType, HistogramHistory>,\n}\n\n#[derive(Default)]\nstruct HistogramHistory {\n    sum: u64,\n    count: u64,\n}\n\n#[derive(Debug, Serialize, Deserialize, PartialEq, Eq)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum Metric<CI, MI, T> {\n    Counter {\n        id: CI,\n        timestamp: T,\n        value: u64,\n    },\n    Gauge {\n        id: MI,\n        timestamp: T,\n        value: u64,\n    },\n    Histogram {\n        id: MI,\n        timestamp: T,\n        count: u64,\n        sum: u64,\n    },\n}\n\npub type SharedMetricHistory = Arc<Mutex<MetricsHistory>>;\n\nconst TYPE_COUNTER: u64 = 0x00;\nconst TYPE_HISTOGRAM: u64 = 0x01;\nconst TYPE_GAUGE: u64 = 0x02;\n\nimpl MetricsStore for Store {\n    async fn write_metrics(\n        &self,\n        core: Arc<Core>,\n        timestamp: u64,\n        history_: SharedMetricHistory,\n    ) -> trc::Result<()> {\n        let mut batch = BatchBuilder::new();\n        {\n            let node_id = core.network.node_id;\n            let mut history = history_.lock();\n            for event in [\n                EventType::Smtp(SmtpEvent::ConnectionStart),\n                EventType::Imap(ImapEvent::ConnectionStart),\n                EventType::Pop3(Pop3Event::ConnectionStart),\n                EventType::ManageSieve(ManageSieveEvent::ConnectionStart),\n                EventType::Http(HttpEvent::ConnectionStart),\n                EventType::Delivery(DeliveryEvent::AttemptStart),\n                EventType::Queue(QueueEvent::QueueMessage),\n                EventType::Queue(QueueEvent::QueueMessageAuthenticated),\n                EventType::Queue(QueueEvent::QueueDsn),\n                EventType::Queue(QueueEvent::QueueReport),\n                EventType::MessageIngest(MessageIngestEvent::Ham),\n                EventType::MessageIngest(MessageIngestEvent::Spam),\n                EventType::Auth(AuthEvent::Failed),\n                EventType::Security(SecurityEvent::AuthenticationBan),\n                EventType::Security(SecurityEvent::ScanBan),\n                EventType::Security(SecurityEvent::AbuseBan),\n                EventType::Security(SecurityEvent::LoiterBan),\n                EventType::Security(SecurityEvent::IpBlocked),\n                EventType::IncomingReport(IncomingReportEvent::DmarcReport),\n                EventType::IncomingReport(IncomingReportEvent::DmarcReportWithWarnings),\n                EventType::IncomingReport(IncomingReportEvent::TlsReport),\n                EventType::IncomingReport(IncomingReportEvent::TlsReportWithWarnings),\n            ] {\n                let reading = Collector::read_event_metric(event.id());\n                if reading > 0 {\n                    let history = history.events.entry(event).or_insert(0);\n                    let diff = reading - *history;\n                    if diff > 0 {\n                        batch.set(\n                            ValueClass::Telemetry(TelemetryClass::Metric {\n                                timestamp,\n                                metric_id: (event.code() << 2) | TYPE_COUNTER,\n                                node_id,\n                            }),\n                            KeySerializer::new(U32_LEN).write_leb128(diff).finalize(),\n                        );\n                    }\n                    *history = reading;\n                }\n            }\n\n            for gauge in Collector::collect_gauges(true) {\n                let gauge_id = gauge.id();\n                if matches!(gauge_id, MetricType::QueueCount | MetricType::ServerMemory) {\n                    let value = gauge.get();\n                    if value > 0 {\n                        batch.set(\n                            ValueClass::Telemetry(TelemetryClass::Metric {\n                                timestamp,\n                                metric_id: (gauge_id.code() << 2) | TYPE_GAUGE,\n                                node_id,\n                            }),\n                            KeySerializer::new(U32_LEN).write_leb128(value).finalize(),\n                        );\n                    }\n                }\n            }\n\n            for histogram in Collector::collect_histograms(true) {\n                let histogram_id = histogram.id();\n                if matches!(\n                    histogram_id,\n                    MetricType::MessageIngestionTime\n                        | MetricType::MessageFtsIndexTime\n                        | MetricType::DeliveryTotalTime\n                        | MetricType::DeliveryTime\n                        | MetricType::DnsLookupTime\n                ) {\n                    let history = history.histograms.entry(histogram_id).or_default();\n                    let sum = histogram.sum();\n                    let count = histogram.count();\n                    let diff_sum = sum - history.sum;\n                    let diff_count = count - history.count;\n                    if diff_sum > 0 || diff_count > 0 {\n                        batch.set(\n                            ValueClass::Telemetry(TelemetryClass::Metric {\n                                timestamp,\n                                metric_id: (histogram_id.code() << 2) | TYPE_HISTOGRAM,\n                                node_id,\n                            }),\n                            KeySerializer::new(U32_LEN)\n                                .write_leb128(diff_count)\n                                .write_leb128(diff_sum)\n                                .finalize(),\n                        );\n                    }\n                    history.sum = sum;\n                    history.count = count;\n                }\n            }\n        }\n\n        if !batch.is_empty() {\n            self.write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(())\n    }\n\n    async fn query_metrics(\n        &self,\n        from_timestamp: u64,\n        to_timestamp: u64,\n    ) -> trc::Result<Vec<Metric<EventType, MetricType, u64>>> {\n        let mut metrics = Vec::new();\n        self.iterate(\n            IterateParams::new(\n                ValueKey::from(ValueClass::Telemetry(TelemetryClass::Metric {\n                    timestamp: from_timestamp,\n                    metric_id: 0,\n                    node_id: 0,\n                })),\n                ValueKey::from(ValueClass::Telemetry(TelemetryClass::Metric {\n                    timestamp: to_timestamp,\n                    metric_id: 0,\n                    node_id: 0,\n                })),\n            ),\n            |key, value| {\n                let timestamp = key.deserialize_be_u64(0).caused_by(trc::location!())?;\n                let (metric_type, _) = key\n                    .get(U64_LEN..)\n                    .and_then(|bytes| bytes.read_leb128::<u64>())\n                    .ok_or_else(|| trc::Error::corrupted_key(key, None, trc::location!()))?;\n                match metric_type & 0x03 {\n                    TYPE_COUNTER => {\n                        let id = EventType::from_code(metric_type >> 2).ok_or_else(|| {\n                            trc::Error::corrupted_key(key, None, trc::location!())\n                        })?;\n                        let (value, _) = value.read_leb128::<u64>().ok_or_else(|| {\n                            trc::Error::corrupted_key(key, value.into(), trc::location!())\n                        })?;\n                        metrics.push(Metric::Counter {\n                            id,\n                            timestamp,\n                            value,\n                        });\n                    }\n                    TYPE_HISTOGRAM => {\n                        let id = MetricType::from_code(metric_type >> 2).ok_or_else(|| {\n                            trc::Error::corrupted_key(key, None, trc::location!())\n                        })?;\n                        let (count, bytes_read) = value.read_leb128::<u64>().ok_or_else(|| {\n                            trc::Error::corrupted_key(key, value.into(), trc::location!())\n                        })?;\n                        let (sum, _) = value\n                            .get(bytes_read..)\n                            .and_then(|bytes| bytes.read_leb128::<u64>())\n                            .ok_or_else(|| {\n                                trc::Error::corrupted_key(key, value.into(), trc::location!())\n                            })?;\n                        metrics.push(Metric::Histogram {\n                            id,\n                            timestamp,\n                            count,\n                            sum,\n                        });\n                    }\n                    TYPE_GAUGE => {\n                        let id = MetricType::from_code(metric_type >> 2).ok_or_else(|| {\n                            trc::Error::corrupted_key(key, None, trc::location!())\n                        })?;\n                        let (value, _) = value.read_leb128::<u64>().ok_or_else(|| {\n                            trc::Error::corrupted_key(key, value.into(), trc::location!())\n                        })?;\n                        metrics.push(Metric::Gauge {\n                            id,\n                            timestamp,\n                            value,\n                        });\n                    }\n                    _ => return Err(trc::Error::corrupted_key(key, None, trc::location!())),\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        Ok(metrics)\n    }\n\n    async fn purge_metrics(&self, period: Duration) -> trc::Result<()> {\n        self.delete_range(\n            ValueKey::from(ValueClass::Telemetry(TelemetryClass::Metric {\n                timestamp: 0,\n                metric_id: 0,\n                node_id: 0,\n            })),\n            ValueKey::from(ValueClass::Telemetry(TelemetryClass::Metric {\n                timestamp: now() - period.as_secs(),\n                metric_id: 0,\n                node_id: 0,\n            })),\n        )\n        .await\n        .caused_by(trc::location!())\n    }\n}\n\nimpl MetricsHistory {\n    pub fn init() -> SharedMetricHistory {\n        Arc::new(Mutex::new(Self::default()))\n    }\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod metrics;\npub mod tracers;\npub mod webhooks;\n\nuse tracers::log::spawn_log_tracer;\nuse tracers::otel::spawn_otel_tracer;\nuse tracers::stdout::spawn_console_tracer;\nuse trc::{Collector, ipc::subscriber::SubscriberBuilder};\nuse webhooks::spawn_webhook_tracer;\n\nuse crate::config::telemetry::{Telemetry, TelemetrySubscriberType};\n\nimpl Telemetry {\n    pub fn enable(self, is_enterprise: bool) {\n        // Spawn tracers\n        for tracer in self.tracers.subscribers {\n            tracer.typ.spawn(\n                SubscriberBuilder::new(tracer.id)\n                    .with_interests(tracer.interests)\n                    .with_lossy(tracer.lossy),\n                is_enterprise,\n            );\n        }\n\n        // Update global collector\n        Collector::set_interests(self.tracers.interests);\n        Collector::update_custom_levels(self.tracers.levels);\n        Collector::set_metrics(self.metrics);\n        Collector::reload();\n    }\n\n    pub fn update(self, is_enterprise: bool) {\n        // Remove tracers that are no longer active\n        let active_subscribers = Collector::get_subscribers();\n        for subscribed_id in &active_subscribers {\n            if !self\n                .tracers\n                .subscribers\n                .iter()\n                .any(|tracer| tracer.id == *subscribed_id)\n            {\n                Collector::remove_subscriber(subscribed_id.clone());\n            }\n        }\n\n        // Activate new tracers or update existing ones\n        for tracer in self.tracers.subscribers {\n            if active_subscribers.contains(&tracer.id) {\n                Collector::update_subscriber(tracer.id, tracer.interests, tracer.lossy);\n            } else {\n                tracer.typ.spawn(\n                    SubscriberBuilder::new(tracer.id)\n                        .with_interests(tracer.interests)\n                        .with_lossy(tracer.lossy),\n                    is_enterprise,\n                );\n            }\n        }\n\n        // Update global collector\n        Collector::set_interests(self.tracers.interests);\n        Collector::update_custom_levels(self.tracers.levels);\n        Collector::set_metrics(self.metrics);\n        Collector::reload();\n    }\n\n    #[cfg(feature = \"test_mode\")]\n    pub fn test_tracer(level: trc::Level) {\n        let mut interests = trc::ipc::subscriber::Interests::default();\n        for event in trc::EventType::variants() {\n            if level.is_contained(event.level()) {\n                interests.set(event);\n            }\n        }\n\n        spawn_console_tracer(\n            SubscriberBuilder::new(\"stderr\".to_string())\n                .with_interests(interests.clone())\n                .with_lossy(false),\n            crate::config::telemetry::ConsoleTracer {\n                ansi: true,\n                multiline: false,\n                buffered: false,\n            },\n        );\n\n        Collector::union_interests(interests);\n        Collector::reload();\n    }\n}\n\nimpl TelemetrySubscriberType {\n    pub fn spawn(self, builder: SubscriberBuilder, is_enterprise: bool) {\n        match self {\n            TelemetrySubscriberType::ConsoleTracer(settings) => {\n                spawn_console_tracer(builder, settings)\n            }\n            TelemetrySubscriberType::LogTracer(settings) => spawn_log_tracer(builder, settings),\n            TelemetrySubscriberType::Webhook(settings) => spawn_webhook_tracer(builder, settings),\n            TelemetrySubscriberType::OtelTracer(settings) => spawn_otel_tracer(builder, settings),\n            #[cfg(unix)]\n            TelemetrySubscriberType::JournalTracer(subscriber) => {\n                tracers::journald::spawn_journald_tracer(builder, subscriber)\n            }\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            TelemetrySubscriberType::StoreTracer(subscriber) => {\n                if is_enterprise {\n                    tracers::store::spawn_store_tracer(builder, subscriber)\n                }\n            } // SPDX-SnippetEnd\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/tracers/journald.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashSet;\nuse std::io::Write;\nuse trc::ipc::subscriber::SubscriberBuilder;\nuse trc::{Event, EventDetails, Level, TelemetryEvent};\n\npub(crate) fn spawn_journald_tracer(builder: SubscriberBuilder, subscriber: Subscriber) {\n    let (_, mut rx) = builder.register();\n    tokio::spawn(async move {\n        while let Some(events) = rx.recv().await {\n            for event in events {\n                subscriber.send_event(&event);\n            }\n        }\n    });\n}\n\nimpl Subscriber {\n    fn send_event(&self, event: &Event<EventDetails>) {\n        let mut buf = Vec::with_capacity(256);\n        put_field_wellformed(\n            &mut buf,\n            \"PRIORITY\",\n            &[match event.inner.level {\n                Level::Error => self.priority_mappings.error as u8,\n                Level::Warn => self.priority_mappings.warn as u8,\n                Level::Info => self.priority_mappings.info as u8,\n                Level::Debug => self.priority_mappings.debug as u8,\n                Level::Trace | Level::Disable => self.priority_mappings.trace as u8,\n            }],\n        );\n        put_field_length_encoded(&mut buf, \"SYSLOG_IDENTIFIER\", |buf| {\n            write!(buf, \"{}\", self.syslog_identifier).unwrap()\n        });\n        put_field_length_encoded(&mut buf, \"MESSAGE\", |buf| {\n            write!(buf, \"{}\", event.inner.typ.description()).unwrap()\n        });\n\n        let mut seen_keys = AHashSet::new();\n        for (key, value) in &event.keys {\n            if seen_keys.insert(*key) {\n                put_field_length_encoded(&mut buf, key.name(), |buf| {\n                    write!(buf, \"{value}\").unwrap()\n                });\n            }\n        }\n\n        if let Err(err) = self.send_payload(&buf) {\n            trc::event!(\n                Telemetry(TelemetryEvent::JournalError),\n                Details = \"Failed to send event to journald\",\n                Reason = err.to_string()\n            );\n        }\n    }\n}\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2018 Benjamin Saunders <ben.e.saunders@gmail.com>\n// SPDX-License-Identifier: MIT\n\n#[cfg(target_os = \"linux\")]\nuse std::fs::File;\nuse std::io::{self, Error, Result};\nuse std::mem::{size_of, zeroed};\n#[cfg(target_os = \"linux\")]\nuse std::os::raw::c_uint;\nuse std::os::unix::ffi::OsStrExt;\nuse std::os::unix::net::UnixDatagram;\n#[cfg(target_os = \"linux\")]\nuse std::os::unix::prelude::FromRawFd;\nuse std::os::unix::prelude::{AsRawFd, RawFd};\nuse std::path::Path;\nuse std::ptr;\n\nuse libc::*;\n\n#[cfg(unix)]\nconst JOURNALD_PATH: &str = \"/run/systemd/journal/socket\";\nconst CMSG_BUFSIZE: usize = 64;\n\npub struct Subscriber {\n    #[cfg(unix)]\n    socket: UnixDatagram,\n    syslog_identifier: String,\n    priority_mappings: PriorityMappings,\n}\n\n#[derive(Debug, Clone)]\npub struct PriorityMappings {\n    /// Priority mapped to the `ERROR` level\n    pub error: Priority,\n    /// Priority mapped to the `WARN` level\n    pub warn: Priority,\n    /// Priority mapped to the `INFO` level\n    pub info: Priority,\n    /// Priority mapped to the `DEBUG` level\n    pub debug: Priority,\n    /// Priority mapped to the `TRACE` level\n    pub trace: Priority,\n}\n\n#[repr(C)]\nunion AlignedBuffer<T: Copy + Clone> {\n    buffer: T,\n    align: cmsghdr,\n}\n\n#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq)]\n#[repr(u8)]\npub enum Priority {\n    /// System is unusable.\n    ///\n    /// Examples:\n    ///\n    /// - severe Kernel BUG\n    /// - systemd dumped core\n    ///\n    /// This level should not be used by applications.\n    Emergency = b'0',\n    /// Should be corrected immediately.\n    ///\n    /// Examples:\n    ///\n    /// - Vital subsystem goes out of work, data loss:\n    /// - `kernel: BUG: unable to handle kernel paging request at ffffc90403238ffc`\n    Alert = b'1',\n    /// Critical conditions\n    ///\n    /// Examples:\n    ///\n    /// - Crashe, coredumps\n    /// - `systemd-coredump[25319]: Process 25310 (plugin-container) of user 1000 dumped core`\n    Critical = b'2',\n    /// Error conditions\n    ///\n    /// Examples:\n    ///\n    /// - Not severe error reported\n    /// - `kernel: usb 1-3: 3:1: cannot get freq at ep 0x84, systemd[1]: Failed unmounting /var`\n    /// - `libvirtd[1720]: internal error: Failed to initialize a valid firewall backend`\n    Error = b'3',\n    /// May indicate that an error will occur if action is not taken.\n    ///\n    /// Examples:\n    ///\n    /// - a non-root file system has only 1GB free\n    /// - `org.freedesktop. Notifications[1860]: (process:5999): Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale`\n    Warning = b'4',\n    /// Events that are unusual, but not error conditions.\n    ///\n    /// Examples:\n    ///\n    /// - `systemd[1]: var.mount: Directory /var to mount over is not empty, mounting anyway`\n    /// - `gcr-prompter[4997]: Gtk: GtkDialog mapped without a transient parent. This is discouraged`\n    Notice = b'5',\n    /// Normal operational messages that require no action.\n    ///\n    /// Example: `lvm[585]: 7 logical volume(s) in volume group \"archvg\" now active`\n    Informational = b'6',\n    /// Information useful to developers for debugging the\n    /// application.\n    ///\n    /// Example: `kdeinit5[1900]: powerdevil: Scheduling inhibition from \":1.14\" \"firefox\" with cookie 13 and reason \"screen\"`\n    Debug = b'7',\n}\n\nimpl Subscriber {\n    /// Construct a journald subscriber\n    ///\n    /// Fails if the journald socket couldn't be opened. Returns a `NotFound` error unconditionally\n    /// in non-Unix environments.\n    pub fn new() -> io::Result<Self> {\n        #[cfg(unix)]\n        {\n            let socket = UnixDatagram::unbound()?;\n            let sub = Self {\n                socket,\n                syslog_identifier: std::env::current_exe()\n                    .ok()\n                    .as_ref()\n                    .and_then(|p| p.file_name())\n                    .map(|n| n.to_string_lossy().into_owned())\n                    // If we fail to get the name of the current executable fall back to an empty string.\n                    .unwrap_or_default(),\n                priority_mappings: PriorityMappings::new(),\n            };\n            // Check that we can talk to journald, by sending empty payload which journald discards.\n            // However if the socket didn't exist or if none listened we'd get an error here.\n            sub.send_payload(&[])?;\n            Ok(sub)\n        }\n        #[cfg(not(unix))]\n        Err(io::Error::new(\n            io::ErrorKind::NotFound,\n            \"journald does not exist in this environment\",\n        ))\n    }\n\n    /// Sets how [`tracing_core::Level`]s are mapped to [journald priorities](Priority).\n    ///\n    pub fn with_priority_mappings(mut self, mappings: PriorityMappings) -> Self {\n        self.priority_mappings = mappings;\n        self\n    }\n\n    /// Sets the syslog identifier for this logger.\n    ///\n    /// The syslog identifier comes from the classic syslog interface (`openlog()`\n    /// and `syslog()`) and tags log entries with a given identifier.\n    /// Systemd exposes it in the `SYSLOG_IDENTIFIER` journal field, and allows\n    /// filtering log messages by syslog identifier with `journalctl -t`.\n    /// Unlike the unit (`journalctl -u`) this field is not trusted, i.e. applications\n    /// can set it freely, and use it e.g. to further categorize log entries emitted under\n    /// the same systemd unit or in the same process.  It also allows to filter for log\n    /// entries of processes not started in their own unit.\n    ///\n    /// See [Journal Fields](https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html)\n    /// and [journalctl](https://www.freedesktop.org/software/systemd/man/journalctl.html)\n    /// for more information.\n    ///\n    /// Defaults to the file name of the executable of the current process, if any.\n    pub fn with_syslog_identifier(mut self, identifier: String) -> Self {\n        self.syslog_identifier = identifier;\n        self\n    }\n\n    /// Returns the syslog identifier in use.\n    pub fn syslog_identifier(&self) -> &str {\n        &self.syslog_identifier\n    }\n\n    #[cfg(not(unix))]\n    fn send_payload(&self, _opayload: &[u8]) -> io::Result<()> {\n        Err(io::Error::new(\n            io::ErrorKind::Other,\n            \"journald not supported on non-Unix\",\n        ))\n    }\n\n    #[cfg(unix)]\n    fn send_payload(&self, payload: &[u8]) -> io::Result<usize> {\n        self.socket\n            .send_to(payload, JOURNALD_PATH)\n            .or_else(|error| {\n                if Some(libc::EMSGSIZE) == error.raw_os_error() {\n                    self.send_large_payload(payload)\n                } else {\n                    Err(error)\n                }\n            })\n    }\n\n    #[cfg(all(unix, not(target_os = \"linux\")))]\n    fn send_large_payload(&self, _payload: &[u8]) -> io::Result<usize> {\n        Err(std::io::Error::other(\n            \"Large payloads not supported on non-Linux OS\",\n        ))\n    }\n\n    /// Send large payloads to journald via a memfd.\n    #[cfg(target_os = \"linux\")]\n    fn send_large_payload(&self, payload: &[u8]) -> io::Result<usize> {\n        // If the payload's too large for a single datagram, send it through a memfd, see\n        // https://systemd.io/JOURNAL_NATIVE_PROTOCOL/\n        use std::os::unix::prelude::AsRawFd;\n        // Write the whole payload to a memfd\n        let mut mem = create_sealable()?;\n        mem.write_all(payload)?;\n        // Fully seal the memfd to signal journald that its backing data won't resize anymore\n        // and so is safe to mmap.\n        seal_fully(mem.as_raw_fd())?;\n        send_one_fd_to(&self.socket, mem.as_raw_fd(), JOURNALD_PATH)\n    }\n}\n\nimpl PriorityMappings {\n    /// Returns the default priority mappings:\n    ///\n    pub fn new() -> PriorityMappings {\n        Self {\n            error: Priority::Error,\n            warn: Priority::Warning,\n            info: Priority::Notice,\n            debug: Priority::Informational,\n            trace: Priority::Debug,\n        }\n    }\n}\n\nimpl Default for PriorityMappings {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n\nimpl std::fmt::Debug for Subscriber {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.debug_struct(\"Subscriber\")\n            .field(\"socket\", &self.socket)\n            .field(\"syslog_identifier\", &self.syslog_identifier)\n            .field(\"priority_mappings\", &self.priority_mappings)\n            .finish()\n    }\n}\n\n/// Append a sanitized and length-encoded field into `buf`.\n///\n/// Unlike `put_field_wellformed` this function handles arbitrary field names and values.\n///\n/// `name` denotes the field name. It gets sanitized before being appended to `buf`.\n///\n/// `write_value` is invoked with `buf` as argument to append the value data to `buf`.  It must\n/// not delete from `buf`, but may append arbitrary data.  This function then determines the length\n/// of the data written and adds it in the appropriate place in `buf`.\nfn put_field_length_encoded(buf: &mut Vec<u8>, name: &str, write_value: impl FnOnce(&mut Vec<u8>)) {\n    for ch in name.as_bytes() {\n        buf.push(ch.to_ascii_uppercase());\n    }\n    buf.push(b'\\n');\n    buf.extend_from_slice(&[0; 8]); // Length tag, to be populated\n    let start = buf.len();\n    write_value(buf);\n    let end = buf.len();\n    buf[start - 8..start].copy_from_slice(&((end - start) as u64).to_le_bytes());\n    buf.push(b'\\n');\n}\n\n/// Append arbitrary data with a well-formed name and value.\n///\n/// `value` must not contain an internal newline, because this function writes\n/// `value` in the new-line separated format.\n///\n/// For a \"newline-safe\" variant, see `put_field_length_encoded`.\nfn put_field_wellformed(buf: &mut Vec<u8>, name: &str, value: &[u8]) {\n    buf.extend_from_slice(name.as_bytes());\n    buf.push(b'\\n');\n    put_value(buf, value);\n}\n\n/// Write the value portion of a key-value pair, in newline separated format.\n///\n/// `value` must not contain an internal newline.\n///\n/// For a \"newline-safe\" variant, see `put_field_length_encoded`.\nfn put_value(buf: &mut Vec<u8>, value: &[u8]) {\n    buf.extend_from_slice(&(value.len() as u64).to_le_bytes());\n    buf.extend_from_slice(value);\n    buf.push(b'\\n');\n}\n\nfn assert_cmsg_bufsize() {\n    let space_one_fd = unsafe { CMSG_SPACE(size_of::<RawFd>() as u32) };\n    assert!(\n        space_one_fd <= CMSG_BUFSIZE as u32,\n        \"cmsghdr buffer too small (< {}) to hold a single fd\",\n        space_one_fd\n    );\n}\n\npub fn send_one_fd_to<P: AsRef<Path>>(socket: &UnixDatagram, fd: RawFd, path: P) -> Result<usize> {\n    assert_cmsg_bufsize();\n\n    let mut addr: sockaddr_un = unsafe { zeroed() };\n    let path_bytes = path.as_ref().as_os_str().as_bytes();\n    // path_bytes may have at most sun_path + 1 bytes, to account for the trailing NUL byte.\n    if addr.sun_path.len() <= path_bytes.len() {\n        return Err(Error::from_raw_os_error(ENAMETOOLONG));\n    }\n\n    addr.sun_family = AF_UNIX as _;\n    unsafe {\n        std::ptr::copy_nonoverlapping(\n            path_bytes.as_ptr(),\n            addr.sun_path.as_mut_ptr() as *mut u8,\n            path_bytes.len(),\n        )\n    };\n\n    let mut msg: msghdr = unsafe { zeroed() };\n    // Set the target address.\n    msg.msg_name = &mut addr as *mut _ as *mut c_void;\n    msg.msg_namelen = size_of::<sockaddr_un>() as socklen_t;\n\n    // We send no data body with this message.\n    msg.msg_iov = ptr::null_mut();\n    msg.msg_iovlen = 0;\n\n    // Create and fill the control message buffer with our file descriptor\n    let mut cmsg_buffer = AlignedBuffer {\n        buffer: ([0u8; CMSG_BUFSIZE]),\n    };\n    msg.msg_control = unsafe { cmsg_buffer.buffer.as_mut_ptr() as _ };\n    msg.msg_controllen = unsafe { CMSG_SPACE(size_of::<RawFd>() as _) as _ };\n\n    let cmsg: &mut cmsghdr =\n        unsafe { CMSG_FIRSTHDR(&msg).as_mut() }.expect(\"Control message buffer exhausted\");\n\n    cmsg.cmsg_level = SOL_SOCKET;\n    cmsg.cmsg_type = SCM_RIGHTS;\n    cmsg.cmsg_len = unsafe { CMSG_LEN(size_of::<RawFd>() as _) as _ };\n\n    unsafe { ptr::write(CMSG_DATA(cmsg) as *mut RawFd, fd) };\n\n    let result = unsafe { sendmsg(socket.as_raw_fd(), &msg, libc::MSG_NOSIGNAL) };\n\n    if result < 0 {\n        Err(Error::last_os_error())\n    } else {\n        // sendmsg returns the number of bytes written\n        Ok(result as usize)\n    }\n}\n\n#[cfg(target_os = \"linux\")]\nfn create(flags: c_uint) -> Result<File> {\n    let fd = memfd_create_syscall(flags);\n    if fd < 0 {\n        Err(Error::last_os_error())\n    } else {\n        Ok(unsafe { File::from_raw_fd(fd as RawFd) })\n    }\n}\n\n/// Make the `memfd_create` syscall ourself instead of going through `libc`;\n/// `memfd_create` isn't supported on `glibc<2.27` so this allows us to\n/// support old-but-still-used distros like Ubuntu Xenial, Debian Stretch,\n/// RHEL 7, etc.\n///\n/// See: https://github.com/tokio-rs/tracing/issues/1879\n#[cfg(target_os = \"linux\")]\nfn memfd_create_syscall(flags: c_uint) -> c_int {\n    unsafe {\n        syscall(\n            SYS_memfd_create,\n            \"tracing-journald\\0\".as_ptr() as *const c_char,\n            flags,\n        ) as c_int\n    }\n}\n\n#[cfg(target_os = \"linux\")]\npub fn create_sealable() -> Result<File> {\n    create(MFD_ALLOW_SEALING | MFD_CLOEXEC)\n}\n\n#[cfg(target_os = \"linux\")]\npub fn seal_fully(fd: RawFd) -> Result<()> {\n    let all_seals = F_SEAL_SHRINK | F_SEAL_GROW | F_SEAL_WRITE | F_SEAL_SEAL;\n    let result = unsafe { fcntl(fd, F_ADD_SEALS, all_seals) };\n    if result < 0 {\n        Err(Error::last_os_error())\n    } else {\n        Ok(())\n    }\n}\n// SPDX-SnippetEnd\n"
  },
  {
    "path": "crates/common/src/telemetry/tracers/log.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{path::PathBuf, time::SystemTime};\n\nuse crate::config::telemetry::{LogTracer, RotationStrategy};\n\nuse mail_parser::DateTime;\nuse tokio::{\n    fs::{File, OpenOptions},\n    io::BufWriter,\n};\nuse trc::{TelemetryEvent, ipc::subscriber::SubscriberBuilder, serializers::text::FmtWriter};\n\npub(crate) fn spawn_log_tracer(builder: SubscriberBuilder, settings: LogTracer) {\n    let (_, mut rx) = builder.register();\n    tokio::spawn(async move {\n        if let Some(writer) = settings.build_writer().await {\n            let mut buf = FmtWriter::new(writer)\n                .with_ansi(settings.ansi)\n                .with_multiline(settings.multiline);\n            let mut roatation_timestamp = settings.next_rotation();\n\n            while let Some(events) = rx.recv().await {\n                for event in events {\n                    // Check if we need to rotate the log file\n                    if roatation_timestamp != 0 && event.inner.timestamp > roatation_timestamp {\n                        if let Err(err) = buf.flush().await {\n                            trc::event!(\n                                Telemetry(TelemetryEvent::LogError),\n                                Reason = err.to_string(),\n                                Details = \"Failed to flush log buffer\"\n                            );\n                        }\n\n                        if let Some(writer) = settings.build_writer().await {\n                            buf.update_writer(writer);\n                            roatation_timestamp = settings.next_rotation();\n                        } else {\n                            return;\n                        };\n                    }\n\n                    if let Err(err) = buf.write(&event).await {\n                        trc::event!(\n                            Telemetry(TelemetryEvent::LogError),\n                            Reason = err.to_string(),\n                            Details = \"Failed to write event to log\"\n                        );\n                        return;\n                    }\n                }\n\n                if let Err(err) = buf.flush().await {\n                    trc::event!(\n                        Telemetry(TelemetryEvent::LogError),\n                        Reason = err.to_string(),\n                        Details = \"Failed to flush log buffer\"\n                    );\n                }\n            }\n        }\n    });\n}\n\nimpl LogTracer {\n    pub async fn build_writer(&self) -> Option<BufWriter<File>> {\n        let now = DateTime::from_timestamp(\n            SystemTime::now()\n                .duration_since(SystemTime::UNIX_EPOCH)\n                .map_or(0, |d| d.as_secs()) as i64,\n        );\n        let file_name = match self.rotate {\n            RotationStrategy::Daily => {\n                format!(\n                    \"{}.{:04}-{:02}-{:02}\",\n                    self.prefix, now.year, now.month, now.day\n                )\n            }\n            RotationStrategy::Hourly => {\n                format!(\n                    \"{}.{:04}-{:02}-{:02}T{:02}\",\n                    self.prefix, now.year, now.month, now.day, now.hour\n                )\n            }\n            RotationStrategy::Minutely => {\n                format!(\n                    \"{}.{:04}-{:02}-{:02}T{:02}:{:02}\",\n                    self.prefix, now.year, now.month, now.day, now.hour, now.minute\n                )\n            }\n            RotationStrategy::Never => self.prefix.clone(),\n        };\n        let path = PathBuf::from(&self.path).join(file_name);\n\n        match OpenOptions::new()\n            .create(true)\n            .append(true)\n            .open(&path)\n            .await\n        {\n            Ok(writer) => Some(BufWriter::new(writer)),\n            Err(err) => {\n                trc::event!(\n                    Telemetry(TelemetryEvent::LogError),\n                    Details = \"Failed to create log file\",\n                    Path = path.to_string_lossy().into_owned(),\n                    Reason = err.to_string(),\n                );\n                None\n            }\n        }\n    }\n\n    pub fn next_rotation(&self) -> u64 {\n        let mut now = DateTime::from_timestamp(\n            SystemTime::now()\n                .duration_since(SystemTime::UNIX_EPOCH)\n                .map_or(0, |d| d.as_secs()) as i64,\n        );\n\n        now.second = 0;\n\n        match self.rotate {\n            RotationStrategy::Daily => {\n                now.hour = 0;\n                now.minute = 0;\n                now.to_timestamp() as u64 + 86400\n            }\n            RotationStrategy::Hourly => {\n                now.minute = 0;\n                now.to_timestamp() as u64 + 3600\n            }\n            RotationStrategy::Minutely => now.to_timestamp() as u64 + 60,\n            RotationStrategy::Never => 0,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/tracers/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[cfg(unix)]\npub mod journald;\npub mod log;\npub mod otel;\npub mod stdout;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\npub mod store;\n// SPDX-SnippetEnd\n"
  },
  {
    "path": "crates/common/src/telemetry/tracers/otel.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{LONG_1Y_SLUMBER, config::telemetry::OtelTracer};\nuse ahash::AHashMap;\nuse mail_parser::DateTime;\nuse opentelemetry::{\n    InstrumentationScope, Key, KeyValue, Value,\n    logs::{AnyValue, Severity},\n    trace::{SpanContext, SpanKind, Status, TraceFlags, TraceState},\n};\nuse opentelemetry_sdk::{\n    Resource,\n    logs::{LogBatch, LogExporter, SdkLogRecord},\n    trace::{SpanData, SpanEvents, SpanExporter, SpanLinks},\n};\nuse opentelemetry_semantic_conventions::resource::SERVICE_VERSION;\nuse std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};\nuse trc::{Event, EventDetails, Level, TelemetryEvent, ipc::subscriber::SubscriberBuilder};\n\nconst MAX_EVENTS: usize = 2048;\n\npub(crate) fn spawn_otel_tracer(builder: SubscriberBuilder, mut otel: OtelTracer) {\n    let (_, mut rx) = builder.register();\n    tokio::spawn(async move {\n        let resource = Resource::builder()\n            .with_service_name(\"stalwart\")\n            .with_attribute(KeyValue::new(SERVICE_VERSION, env!(\"CARGO_PKG_VERSION\")))\n            .build();\n\n        let instrumentation = InstrumentationScope::builder(\"stalwart\")\n            .with_version(env!(\"CARGO_PKG_VERSION\"))\n            .build();\n\n        otel.log_exporter.set_resource(&resource);\n        otel.span_exporter.set_resource(&resource);\n\n        let mut wakeup_time = LONG_1Y_SLUMBER;\n        let mut next_delivery = Instant::now();\n\n        let mut pending_logs = Vec::new();\n        let mut pending_spans = Vec::new();\n\n        let mut active_spans = AHashMap::new();\n\n        loop {\n            // Wait for the next event or timeout\n            let event_or_timeout = tokio::time::timeout(wakeup_time, rx.recv()).await;\n\n            match event_or_timeout {\n                Ok(Some(events)) => {\n                    for event in events {\n                        if otel.log_exporter_enable {\n                            pending_logs.push(otel.build_log_record(&event));\n                        }\n\n                        if otel.span_exporter_enable\n                            && let Some(span) = event.inner.span.as_ref()\n                        {\n                            let span_id = span.span_id().unwrap();\n                            if !event.inner.typ.is_span_end() {\n                                let events = active_spans.entry(span_id).or_insert_with(Vec::new);\n                                if events.len() < MAX_EVENTS {\n                                    events.push(event);\n                                }\n                            } else if let Some(events) = active_spans.remove(&span_id) {\n                                pending_spans.push(build_span_data(\n                                    span,\n                                    &event,\n                                    events.iter().chain(std::iter::once(&event)),\n                                    &instrumentation,\n                                ));\n                            }\n                        }\n                    }\n                }\n                Ok(None) => {\n                    break;\n                }\n                Err(_) => (),\n            }\n\n            // Process events\n            let mut next_retry = None;\n            let now = Instant::now();\n            if next_delivery <= now {\n                if !pending_spans.is_empty() || !pending_logs.is_empty() {\n                    next_delivery = now + otel.throttle;\n\n                    if !pending_spans.is_empty()\n                        && let Err(err) = otel\n                            .span_exporter\n                            .export(std::mem::take(&mut pending_spans))\n                            .await\n                    {\n                        trc::event!(\n                            Telemetry(TelemetryEvent::OtelExporterError),\n                            Details = \"Failed to export spans\",\n                            Reason = err.to_string()\n                        );\n                    }\n\n                    if !pending_logs.is_empty() {\n                        let logs = pending_logs\n                            .iter()\n                            .map(|log| (log, &instrumentation))\n                            .collect::<Vec<_>>();\n\n                        if let Err(err) = otel.log_exporter.export(LogBatch::new(&logs)).await {\n                            trc::event!(\n                                Telemetry(TelemetryEvent::OtelExporterError),\n                                Details = \"Failed to export logs\",\n                                Reason = err.to_string()\n                            );\n                        }\n                        pending_logs.clear();\n                    }\n                }\n            } else if !pending_logs.is_empty() || !pending_spans.is_empty() {\n                // Retry later\n                let this_retry = next_delivery - now;\n                match next_retry {\n                    Some(next_retry) if this_retry >= next_retry => {}\n                    _ => {\n                        next_retry = Some(this_retry);\n                    }\n                }\n            }\n            wakeup_time = next_retry.unwrap_or(LONG_1Y_SLUMBER);\n        }\n    });\n}\n\nfn build_span_data<I, T>(\n    start_span: &Event<EventDetails>,\n    end_span: &Event<EventDetails>,\n    span_events: I,\n    instrumentation: &InstrumentationScope,\n) -> SpanData\nwhere\n    I: IntoIterator<Item = T>,\n    T: AsRef<Event<EventDetails>>,\n{\n    let span_id = start_span.span_id().unwrap();\n\n    let mut events = SpanEvents::default();\n    events.events = span_events\n        .into_iter()\n        .map(|event| {\n            let event = event.as_ref();\n\n            opentelemetry::trace::Event::new(\n                event.inner.typ.name(),\n                UNIX_EPOCH + Duration::from_secs(event.inner.timestamp),\n                event.keys.iter().filter_map(build_key_value).collect(),\n                0,\n            )\n        })\n        .collect();\n\n    SpanData {\n        span_context: SpanContext::new(\n            (span_id as u128).into(),\n            span_id.into(),\n            TraceFlags::default(),\n            false,\n            TraceState::default(),\n        ),\n        dropped_attributes_count: 0,\n        parent_span_id: 0.into(),\n        name: start_span.inner.typ.name().into(),\n        start_time: UNIX_EPOCH + Duration::from_secs(start_span.inner.timestamp),\n        end_time: UNIX_EPOCH + Duration::from_secs(end_span.inner.timestamp),\n        attributes: start_span.keys.iter().filter_map(build_key_value).collect(),\n        events,\n        links: SpanLinks::default(),\n        status: Status::default(),\n        span_kind: SpanKind::Server,\n        instrumentation_scope: instrumentation.clone(),\n    }\n}\n\nimpl OtelTracer {\n    fn build_log_record(&self, event: &Event<EventDetails>) -> SdkLogRecord {\n        use opentelemetry::logs::LogRecord;\n        use opentelemetry::logs::Logger;\n\n        let mut record = self.log_provider.create_log_record();\n        record.set_event_name(event.inner.typ.name());\n        record.set_severity_number(match event.inner.level {\n            Level::Trace => Severity::Trace,\n            Level::Debug => Severity::Debug,\n            Level::Info => Severity::Info,\n            Level::Warn => Severity::Warn,\n            Level::Error => Severity::Error,\n            Level::Disable => Severity::Error,\n        });\n        record.set_severity_text(event.inner.level.as_str());\n        record.set_body(AnyValue::String(event.inner.typ.description().into()));\n        record.set_timestamp(UNIX_EPOCH + Duration::from_secs(event.inner.timestamp));\n        record.set_observed_timestamp(SystemTime::now());\n        for (k, v) in &event.keys {\n            record.add_attribute(k.name(), build_any_value(v));\n        }\n        record\n    }\n}\n\nfn build_key_value(key_value: &(trc::Key, trc::Value)) -> Option<KeyValue> {\n    (key_value.0 != trc::Key::SpanId).then(|| {\n        KeyValue::new(\n            build_key(&key_value.0),\n            match &key_value.1 {\n                trc::Value::String(v) => Value::String(v.to_string().into()),\n                trc::Value::UInt(v) => Value::I64(*v as i64),\n                trc::Value::Int(v) => Value::I64(*v),\n                trc::Value::Float(v) => Value::F64(*v),\n                trc::Value::Timestamp(v) => {\n                    Value::String(DateTime::from_timestamp(*v as i64).to_rfc3339().into())\n                }\n                trc::Value::Duration(v) => Value::I64(*v as i64),\n                trc::Value::Bytes(_) => Value::String(\"[binary data]\".into()),\n                trc::Value::Bool(v) => Value::Bool(*v),\n                trc::Value::Ipv4(v) => Value::String(v.to_string().into()),\n                trc::Value::Ipv6(v) => Value::String(v.to_string().into()),\n                trc::Value::Event(_) => Value::String(\"[event data]\".into()),\n                trc::Value::Array(_) => Value::String(\"[array]\".into()),\n                trc::Value::None => Value::Bool(false),\n            },\n        )\n    })\n}\n\nfn build_key(key: &trc::Key) -> Key {\n    Key::from_static_str(key.name())\n}\n\nfn build_any_value(value: &trc::Value) -> AnyValue {\n    match value {\n        trc::Value::String(v) => AnyValue::String(v.to_string().into()),\n        trc::Value::UInt(v) => AnyValue::Int(*v as i64),\n        trc::Value::Int(v) => AnyValue::Int(*v),\n        trc::Value::Float(v) => AnyValue::Double(*v),\n        trc::Value::Timestamp(v) => {\n            AnyValue::String(DateTime::from_timestamp(*v as i64).to_rfc3339().into())\n        }\n        trc::Value::Duration(v) => AnyValue::Int(*v as i64),\n        trc::Value::Bytes(v) => AnyValue::Bytes(Box::new(v.clone())),\n        trc::Value::Bool(v) => AnyValue::Boolean(*v),\n        trc::Value::Ipv4(v) => AnyValue::String(v.to_string().into()),\n        trc::Value::Ipv6(v) => AnyValue::String(v.to_string().into()),\n        trc::Value::Event(v) => AnyValue::Map(Box::new(\n            [(\n                Key::from_static_str(\"eventName\"),\n                AnyValue::String(v.event_type().name().into()),\n            )]\n            .into_iter()\n            .chain(\n                v.keys()\n                    .iter()\n                    .map(|(k, v)| (build_key(k), build_any_value(v))),\n            )\n            .collect(),\n        )),\n        trc::Value::Array(v) => {\n            AnyValue::ListAny(Box::new(v.iter().map(build_any_value).collect()))\n        }\n        trc::Value::None => AnyValue::Boolean(false),\n    }\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/tracers/stdout.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    io::{Error, stderr},\n    pin::Pin,\n    task::{Context, Poll},\n};\n\nuse crate::config::telemetry::ConsoleTracer;\nuse std::io::Write;\nuse tokio::io::AsyncWrite;\nuse trc::{ipc::subscriber::SubscriberBuilder, serializers::text::FmtWriter};\n\npub(crate) fn spawn_console_tracer(builder: SubscriberBuilder, settings: ConsoleTracer) {\n    let (_, mut rx) = builder.register();\n    tokio::spawn(async move {\n        let mut buf = FmtWriter::new(StdErrWriter::default())\n            .with_ansi(settings.ansi)\n            .with_multiline(settings.multiline);\n\n        while let Some(events) = rx.recv().await {\n            for event in events {\n                let _ = buf.write(&event).await;\n\n                if !settings.buffered {\n                    let _ = buf.flush().await;\n                }\n            }\n\n            if settings.buffered {\n                let _ = buf.flush().await;\n            }\n        }\n    });\n}\n\nconst BUFFER_CAPACITY: usize = 4096;\n\npub struct StdErrWriter {\n    buffer: Vec<u8>,\n}\n\nimpl AsyncWrite for StdErrWriter {\n    fn poll_write(\n        mut self: Pin<&mut Self>,\n        _: &mut Context<'_>,\n        bytes: &[u8],\n    ) -> Poll<Result<usize, Error>> {\n        let bytes_len = bytes.len();\n        let buffer_len = self.buffer.len();\n\n        if buffer_len + bytes_len < BUFFER_CAPACITY {\n            self.buffer.extend_from_slice(bytes);\n            Poll::Ready(Ok(bytes_len))\n        } else if bytes_len > BUFFER_CAPACITY {\n            let result = stderr()\n                .write_all(&self.buffer)\n                .and_then(|_| stderr().write_all(bytes));\n            self.buffer.clear();\n            Poll::Ready(result.map(|_| bytes_len))\n        } else {\n            let result = stderr().write_all(&self.buffer);\n            self.buffer.clear();\n            self.buffer.extend_from_slice(bytes);\n            Poll::Ready(result.map(|_| bytes_len))\n        }\n    }\n\n    fn poll_flush(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Result<(), Error>> {\n        Poll::Ready(if !self.buffer.is_empty() {\n            let result = stderr().write_all(&self.buffer);\n            self.buffer.clear();\n            result\n        } else {\n            Ok(())\n        })\n    }\n\n    fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Result<(), Error>> {\n        Poll::Ready(Ok(()))\n    }\n}\n\nimpl Default for StdErrWriter {\n    fn default() -> Self {\n        Self {\n            buffer: Vec::with_capacity(BUFFER_CAPACITY),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/tracers/store.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse crate::config::telemetry::StoreTracer;\nuse ahash::{AHashMap, AHashSet};\nuse std::{collections::HashSet, future::Future, time::Duration};\nuse store::{\n    Deserialize, SearchStore, Store, ValueKey,\n    search::{IndexDocument, SearchField, SearchFilter, SearchQuery, TracingSearchField},\n    write::{BatchBuilder, SearchIndex, TaskEpoch, TaskQueueClass, TelemetryClass, ValueClass},\n};\nuse trc::{\n    AddContext, AuthEvent, Event, EventDetails, EventType, Key, MessageIngestEvent,\n    OutgoingReportEvent, QueueEvent, Value,\n    ipc::subscriber::SubscriberBuilder,\n    serializers::binary::{deserialize_events, serialize_events},\n};\nuse utils::snowflake::SnowflakeIdGenerator;\n\nconst MAX_EVENTS: usize = 2048;\n\npub(crate) fn spawn_store_tracer(builder: SubscriberBuilder, settings: StoreTracer) {\n    let (_, mut rx) = builder.register();\n    tokio::spawn(async move {\n        let mut active_spans = AHashMap::new();\n        let store = settings.store;\n        let mut batch = BatchBuilder::new();\n\n        while let Some(events) = rx.recv().await {\n            for event in events {\n                if let Some(span) = &event.inner.span {\n                    let span_id = span.span_id().unwrap();\n                    if !event.inner.typ.is_span_end() {\n                        let events = active_spans.entry(span_id).or_insert_with(Vec::new);\n                        if events.len() < MAX_EVENTS {\n                            events.push(event);\n                        }\n                    } else if let Some(events) = active_spans.remove(&span_id)\n                        && events\n                            .iter()\n                            .chain([span, &event])\n                            .flat_map(|event| event.keys.iter())\n                            .any(|(k, v)| matches!((k, v), (Key::QueueId, Value::UInt(_))))\n                    {\n                        // Serialize events\n                        batch\n                            .set(\n                                ValueClass::Telemetry(TelemetryClass::Span { span_id }),\n                                serialize_events(\n                                    [span.as_ref()]\n                                        .into_iter()\n                                        .chain(events.iter().map(|event| event.as_ref()))\n                                        .chain([event.as_ref()].into_iter()),\n                                    events.len() + 2,\n                                ),\n                            )\n                            .with_account_id((span_id >> 32) as u32) // TODO: This is hacky, improve\n                            .with_document(span_id as u32)\n                            .set(\n                                ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                                    due: TaskEpoch::now(),\n                                    index: SearchIndex::Tracing,\n                                    is_insert: true,\n                                }),\n                                vec![],\n                            );\n                    }\n                }\n            }\n\n            if !batch.is_empty() {\n                if let Err(err) = store.write(batch.build_all()).await {\n                    trc::error!(err.caused_by(trc::location!()));\n                }\n                batch = BatchBuilder::new();\n            }\n        }\n    });\n}\n\npub trait TracingStore: Sync + Send {\n    fn get_span(\n        &self,\n        span_id: u64,\n    ) -> impl Future<Output = trc::Result<Vec<Event<EventDetails>>>> + Send;\n    fn get_raw_span(\n        &self,\n        span_id: u64,\n    ) -> impl Future<Output = trc::Result<Option<Vec<u8>>>> + Send;\n    fn purge_spans(\n        &self,\n        period: Duration,\n        search_store: Option<&SearchStore>,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n}\n\nimpl TracingStore for Store {\n    async fn get_span(&self, span_id: u64) -> trc::Result<Vec<Event<EventDetails>>> {\n        self.get_value::<Span>(ValueKey::from(ValueClass::Telemetry(\n            TelemetryClass::Span { span_id },\n        )))\n        .await\n        .caused_by(trc::location!())\n        .map(|span| span.map(|span| span.0).unwrap_or_default())\n    }\n\n    async fn get_raw_span(&self, span_id: u64) -> trc::Result<Option<Vec<u8>>> {\n        self.get_value::<RawSpan>(ValueKey::from(ValueClass::Telemetry(\n            TelemetryClass::Span { span_id },\n        )))\n        .await\n        .caused_by(trc::location!())\n        .map(|span| span.map(|span| span.0))\n    }\n\n    async fn purge_spans(\n        &self,\n        period: Duration,\n        search_store: Option<&SearchStore>,\n    ) -> trc::Result<()> {\n        let until_span_id = SnowflakeIdGenerator::from_duration(period).ok_or_else(|| {\n            trc::StoreEvent::UnexpectedError\n                .caused_by(trc::location!())\n                .ctx(trc::Key::Reason, \"Failed to generate reference span id.\")\n        })?;\n\n        self.delete_range(\n            ValueKey::from(ValueClass::Telemetry(TelemetryClass::Span { span_id: 0 })),\n            ValueKey::from(ValueClass::Telemetry(TelemetryClass::Span {\n                span_id: until_span_id,\n            })),\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        if let Some(search_store) = search_store {\n            search_store\n                .unindex(\n                    SearchQuery::new(SearchIndex::Tracing)\n                        .with_filter(SearchFilter::lt(SearchField::Id, until_span_id)),\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl StoreTracer {\n    pub fn default_events() -> impl IntoIterator<Item = EventType> {\n        EventType::variants().into_iter().filter(|event| {\n            !event.is_raw_io()\n                && matches!(\n                    event,\n                    EventType::MessageIngest(\n                        MessageIngestEvent::Ham\n                            | MessageIngestEvent::Spam\n                            | MessageIngestEvent::Duplicate\n                            | MessageIngestEvent::Error\n                    ) | EventType::Smtp(_)\n                        | EventType::Delivery(_)\n                        | EventType::MtaSts(_)\n                        | EventType::TlsRpt(_)\n                        | EventType::Dane(_)\n                        | EventType::Iprev(_)\n                        | EventType::Spf(_)\n                        | EventType::Dmarc(_)\n                        | EventType::Dkim(_)\n                        | EventType::MailAuth(_)\n                        | EventType::Queue(\n                            QueueEvent::QueueMessage\n                                | QueueEvent::QueueMessageAuthenticated\n                                | QueueEvent::QueueReport\n                                | QueueEvent::QueueDsn\n                                | QueueEvent::QueueAutogenerated\n                                | QueueEvent::Rescheduled\n                                | QueueEvent::RateLimitExceeded\n                                | QueueEvent::ConcurrencyLimitExceeded\n                                | QueueEvent::QuotaExceeded\n                        )\n                        | EventType::Limit(_)\n                        | EventType::Tls(_)\n                        | EventType::IncomingReport(_)\n                        | EventType::OutgoingReport(\n                            OutgoingReportEvent::SpfReport\n                                | OutgoingReportEvent::SpfRateLimited\n                                | OutgoingReportEvent::DkimReport\n                                | OutgoingReportEvent::DkimRateLimited\n                                | OutgoingReportEvent::DmarcReport\n                                | OutgoingReportEvent::DmarcRateLimited\n                                | OutgoingReportEvent::DmarcAggregateReport\n                                | OutgoingReportEvent::TlsAggregate\n                                | OutgoingReportEvent::HttpSubmission\n                                | OutgoingReportEvent::UnauthorizedReportingAddress\n                                | OutgoingReportEvent::ReportingAddressValidationError\n                                | OutgoingReportEvent::NotFound\n                                | OutgoingReportEvent::SubmissionError\n                                | OutgoingReportEvent::NoRecipientsFound\n                        )\n                        | EventType::Auth(\n                            AuthEvent::Success\n                                | AuthEvent::Failed\n                                | AuthEvent::TooManyAttempts\n                                | AuthEvent::Error\n                        )\n                        | EventType::Sieve(_)\n                        | EventType::Milter(_)\n                        | EventType::MtaHook(_)\n                        | EventType::Security(_)\n                )\n        })\n    }\n}\n\nstruct RawSpan(Vec<u8>);\nstruct Span(Vec<Event<EventDetails>>);\n\nimpl Deserialize for Span {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        deserialize_events(bytes).map(Self)\n    }\n}\n\nimpl Deserialize for RawSpan {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(Self(bytes.to_vec()))\n    }\n}\n\npub fn build_span_document(\n    span_id: u64,\n    events: Vec<Event<EventDetails>>,\n    index_fields: &AHashSet<SearchField>,\n) -> IndexDocument {\n    let mut document = IndexDocument::new(SearchIndex::Tracing).with_id(span_id);\n    let mut keywords = HashSet::new();\n\n    for (idx, event) in events.into_iter().enumerate() {\n        if idx == 0\n            && (index_fields.is_empty()\n                || index_fields.contains(&TracingSearchField::EventType.into()))\n        {\n            document.index_unsigned(TracingSearchField::EventType, event.inner.typ.code());\n        }\n\n        for (key, value) in event.keys {\n            match (key, value) {\n                (Key::QueueId, Value::UInt(queue_id)) => {\n                    if index_fields.is_empty()\n                        || index_fields.contains(&TracingSearchField::QueueId.into())\n                    {\n                        document.index_unsigned(TracingSearchField::QueueId, queue_id);\n                    }\n                }\n                (Key::From | Key::To | Key::Domain | Key::Hostname, Value::String(address)) => {\n                    if index_fields.is_empty()\n                        || index_fields.contains(&TracingSearchField::Keywords.into())\n                    {\n                        keywords.insert(address.to_string());\n                    }\n                }\n                (Key::To, Value::Array(value)) => {\n                    if index_fields.is_empty()\n                        || index_fields.contains(&TracingSearchField::Keywords.into())\n                    {\n                        for value in value {\n                            if let Value::String(address) = value {\n                                keywords.insert(address.to_string());\n                            }\n                        }\n                    }\n                }\n                (Key::RemoteIp, Value::Ipv4(ip)) => {\n                    if index_fields.is_empty()\n                        || index_fields.contains(&TracingSearchField::Keywords.into())\n                    {\n                        keywords.insert(ip.to_string());\n                    }\n                }\n                (Key::RemoteIp, Value::Ipv6(ip)) => {\n                    if index_fields.is_empty()\n                        || index_fields.contains(&TracingSearchField::Keywords.into())\n                    {\n                        keywords.insert(ip.to_string());\n                    }\n                }\n\n                _ => {}\n            }\n        }\n    }\n\n    if !keywords.is_empty() {\n        let mut keyword_str = String::new();\n        for keyword in keywords {\n            if !keyword_str.is_empty() {\n                keyword_str.push(' ');\n            }\n            keyword_str.push_str(&keyword);\n        }\n\n        document.index_keyword(TracingSearchField::Keywords, keyword_str);\n    }\n\n    document\n}\n"
  },
  {
    "path": "crates/common/src/telemetry/webhooks/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    sync::{\n        Arc,\n        atomic::{AtomicBool, Ordering},\n    },\n    time::Instant,\n};\n\nuse crate::{LONG_1Y_SLUMBER, config::telemetry::WebhookTracer};\nuse base64::{Engine, engine::general_purpose::STANDARD};\nuse ring::hmac;\nuse serde::Serialize;\nuse store::write::now;\nuse tokio::sync::mpsc;\nuse trc::{\n    Event, EventDetails, ServerEvent, TelemetryEvent,\n    ipc::subscriber::{EventBatch, SubscriberBuilder},\n    serializers::json::JsonEventSerializer,\n};\n\npub(crate) fn spawn_webhook_tracer(builder: SubscriberBuilder, settings: WebhookTracer) {\n    let (tx, mut rx) = builder.register();\n    tokio::spawn(async move {\n        let settings = Arc::new(settings);\n        let mut wakeup_time = LONG_1Y_SLUMBER;\n        let discard_after = settings.discard_after.as_secs();\n        let mut pending_events = Vec::new();\n        let mut next_delivery = Instant::now();\n        let in_flight = Arc::new(AtomicBool::new(false));\n\n        loop {\n            // Wait for the next event or timeout\n            let event_or_timeout = tokio::time::timeout(wakeup_time, rx.recv()).await;\n            let now = now();\n\n            match event_or_timeout {\n                Ok(Some(events)) => {\n                    let mut discard_count = 0;\n                    for event in events {\n                        if now.saturating_sub(event.inner.timestamp) < discard_after {\n                            pending_events.push(event)\n                        } else {\n                            discard_count += 1;\n                        }\n                    }\n\n                    if discard_count > 0 {\n                        trc::event!(\n                            Telemetry(TelemetryEvent::WebhookError),\n                            Details = \"Discarded stale events\",\n                            Total = discard_count\n                        );\n                    }\n                }\n                Ok(None) => {\n                    break;\n                }\n                Err(_) => (),\n            }\n\n            // Process events\n            let mut next_retry = None;\n            let now = Instant::now();\n            if next_delivery <= now {\n                if !pending_events.is_empty() {\n                    next_delivery = now + settings.throttle;\n                    if !in_flight.load(Ordering::Relaxed) {\n                        spawn_webhook_handler(\n                            settings.clone(),\n                            in_flight.clone(),\n                            std::mem::take(&mut pending_events),\n                            tx.clone(),\n                        );\n                    }\n                }\n            } else if !pending_events.is_empty() {\n                // Retry later\n                let this_retry = next_delivery - now;\n                match next_retry {\n                    Some(next_retry) if this_retry >= next_retry => {}\n                    _ => {\n                        next_retry = Some(this_retry);\n                    }\n                }\n            }\n            wakeup_time = next_retry.unwrap_or(LONG_1Y_SLUMBER);\n        }\n    });\n}\n\n#[derive(Serialize)]\nstruct EventWrapper {\n    events: JsonEventSerializer<Vec<Arc<Event<EventDetails>>>>,\n}\n\nfn spawn_webhook_handler(\n    settings: Arc<WebhookTracer>,\n    in_flight: Arc<AtomicBool>,\n    events: EventBatch,\n    webhook_tx: mpsc::Sender<EventBatch>,\n) {\n    tokio::spawn(async move {\n        in_flight.store(true, Ordering::Relaxed);\n        let wrapper = EventWrapper {\n            events: JsonEventSerializer::new(events).with_id().with_spans(),\n        };\n\n        if let Err(err) = post_webhook_events(&settings, &wrapper).await {\n            trc::event!(Telemetry(TelemetryEvent::WebhookError), Details = err);\n\n            if webhook_tx.send(wrapper.events.into_inner()).await.is_err() {\n                trc::event!(\n                    Server(ServerEvent::ThreadError),\n                    Details = \"Failed to send failed webhook events back to main thread\",\n                    CausedBy = trc::location!()\n                );\n            }\n        }\n\n        in_flight.store(false, Ordering::Relaxed);\n    });\n}\n\nasync fn post_webhook_events(\n    settings: &WebhookTracer,\n    events: &EventWrapper,\n) -> Result<(), String> {\n    // Serialize body\n    let body = serde_json::to_string(events)\n        .map_err(|err| format!(\"Failed to serialize events: {}\", err))?;\n\n    // Add HMAC-SHA256 signature\n    let mut headers = settings.headers.clone();\n    if !settings.key.is_empty() {\n        let key = hmac::Key::new(hmac::HMAC_SHA256, settings.key.as_bytes());\n        let tag = hmac::sign(&key, body.as_bytes());\n\n        headers.insert(\n            \"X-Signature\",\n            STANDARD.encode(tag.as_ref()).parse().unwrap(),\n        );\n    }\n\n    // Send request\n    let response = reqwest::Client::builder()\n        .timeout(settings.timeout)\n        .danger_accept_invalid_certs(settings.tls_allow_invalid_certs)\n        .build()\n        .map_err(|err| format!(\"Failed to create HTTP client: {}\", err))?\n        .post(&settings.url)\n        .headers(headers)\n        .body(body)\n        .send()\n        .await\n        .map_err(|err| format!(\"Webhook request to {} failed: {err}\", settings.url))?;\n\n    if response.status().is_success() {\n        Ok(())\n    } else {\n        Err(format!(\n            \"Webhook request to {} failed with code {}: {}\",\n            settings.url,\n            response.status().as_u16(),\n            response.status().canonical_reason().unwrap_or(\"Unknown\")\n        ))\n    }\n}\n"
  },
  {
    "path": "crates/dav/Cargo.toml",
    "content": "[package]\nname = \"dav\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\ndav-proto = { path =  \"../dav-proto\" }\ncommon = { path = \"../common\" }\nstore = { path = \"../store\" }\nutils = { path = \"../utils\" }\ngroupware = { path = \"../groupware\" }\ndirectory = { path = \"../directory\" }\nhttp_proto = { path = \"../http-proto\" }\ntypes = { path = \"../types\" }\ntrc = { path = \"../trc\" }\ncalcard = { version = \"0.3\", features = [\"rkyv\"] }\nhashify = { version = \"0.2\" }\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\npercent-encoding = \"2.3.1\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\nchrono = \"0.4.40\"\n\n[dev-dependencies]\n\n[features]\ntest_mode = []\nenterprise = []\n"
  },
  {
    "path": "crates/dav/src/calendar/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::assert_is_unique_uid;\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n};\nuse calcard::common::timezone::Tz;\nuse common::{DavName, Server, auth::AccessToken};\nuse dav_proto::{Depth, RequestHeaders};\nuse groupware::{\n    DestroyArchive,\n    cache::GroupwareCache,\n    calendar::{Calendar, CalendarEvent, CalendarPreferences, Timezone},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::{BatchBuilder, now};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection, VanishedCollection},\n};\n\npub(crate) trait CalendarCopyMoveRequestHandler: Sync + Send {\n    fn handle_calendar_copy_move_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_move: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CalendarCopyMoveRequestHandler for Server {\n    async fn handle_calendar_copy_move_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_move: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate source\n        let from_resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let from_account_id = from_resource_.account_id;\n        let from_resources = self\n            .fetch_dav_resources(access_token, from_account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let from_resource_name = from_resource_\n            .resource\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        let from_resource = from_resources\n            .by_path(from_resource_name)\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        #[cfg(not(debug_assertions))]\n        if is_move\n            && from_resource.is_container()\n            && self\n                .core\n                .groupware\n                .default_calendar_name\n                .as_ref()\n                .is_some_and(|name| name == from_resource_name)\n        {\n            return Err(DavError::Condition(crate::DavErrorCondition::new(\n                StatusCode::FORBIDDEN,\n                dav_proto::schema::response::CalCondition::DefaultCalendarNeeded,\n            )));\n        }\n\n        // Validate ACL\n        if !access_token.is_member(from_account_id)\n            && !from_resources.has_access_to_container(\n                access_token,\n                if from_resource.is_container() {\n                    from_resource.document_id()\n                } else {\n                    from_resource.parent_id().unwrap()\n                },\n                Acl::ReadItems,\n            )\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Validate destination\n        let destination = self\n            .validate_uri_with_status(\n                access_token,\n                headers\n                    .destination\n                    .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?,\n                StatusCode::BAD_GATEWAY,\n            )\n            .await?;\n        if destination.collection != Collection::Calendar {\n            return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n        }\n        let to_account_id = destination\n            .account_id\n            .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?;\n        let to_resources = if to_account_id == from_account_id {\n            from_resources.clone()\n        } else {\n            self.fetch_dav_resources(access_token, to_account_id, SyncCollection::Calendar)\n                .await\n                .caused_by(trc::location!())?\n        };\n\n        // Validate headers\n        let destination_resource_name = destination\n            .resource\n            .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?;\n        let to_resource = to_resources.by_path(destination_resource_name);\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![\n                ResourceState {\n                    account_id: from_account_id,\n                    collection: if from_resource.is_container() {\n                        Collection::Calendar\n                    } else {\n                        Collection::CalendarEvent\n                    },\n                    document_id: Some(from_resource.document_id()),\n                    path: from_resource_name,\n                    ..Default::default()\n                },\n                ResourceState {\n                    account_id: to_account_id,\n                    collection: to_resource\n                        .map(|r| {\n                            if r.is_container() {\n                                Collection::Calendar\n                            } else {\n                                Collection::CalendarEvent\n                            }\n                        })\n                        .unwrap_or(Collection::Calendar),\n                    document_id: Some(to_resource.map(|r| r.document_id()).unwrap_or(u32::MAX)),\n                    path: destination_resource_name,\n                    ..Default::default()\n                },\n            ],\n            Default::default(),\n            if is_move {\n                DavMethod::MOVE\n            } else {\n                DavMethod::COPY\n            },\n        )\n        .await?;\n\n        // Map destination\n        if let Some(to_resource) = to_resource {\n            if from_resource.path() == to_resource.path() {\n                // Same resource\n                return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n            }\n            let new_name = destination_resource_name\n                .rsplit_once('/')\n                .map(|(_, name)| name)\n                .unwrap_or(destination_resource_name);\n\n            match (from_resource.is_container(), to_resource.is_container()) {\n                (true, true) => {\n                    let from_children_ids = from_resources\n                        .subtree(from_resource_name)\n                        .filter(|r| !r.is_container())\n                        .map(|r| r.document_id())\n                        .collect::<Vec<_>>();\n                    let to_document_ids = to_resources\n                        .subtree(destination_resource_name)\n                        .filter(|r| !r.is_container())\n                        .map(|r| r.document_id())\n                        .collect::<Vec<_>>();\n\n                    // Validate ACLs\n                    if !access_token.is_member(to_account_id)\n                        || (!access_token.is_member(from_account_id)\n                            && !from_resources.has_access_to_container(\n                                access_token,\n                                from_resource.document_id(),\n                                if is_move {\n                                    Acl::RemoveItems\n                                } else {\n                                    Acl::ReadItems\n                                },\n                            ))\n                    {\n                        return Err(DavError::Code(StatusCode::FORBIDDEN));\n                    }\n\n                    // Overwrite container\n                    copy_container(\n                        self,\n                        access_token,\n                        from_account_id,\n                        from_resource.document_id(),\n                        from_children_ids,\n                        from_resources.format_collection(from_resource_name),\n                        to_account_id,\n                        to_resource.document_id().into(),\n                        to_document_ids,\n                        new_name,\n                        is_move,\n                    )\n                    .await\n                }\n                (false, false) => {\n                    // Overwrite event\n                    let from_calendar_id = from_resource.parent_id().unwrap();\n                    let to_calendar_id = to_resource.parent_id().unwrap();\n\n                    // Validate ACL\n                    if (!access_token.is_member(from_account_id)\n                        && !from_resources.has_access_to_container(\n                            access_token,\n                            from_calendar_id,\n                            if is_move {\n                                Acl::RemoveItems\n                            } else {\n                                Acl::ReadItems\n                            },\n                        ))\n                        || (!access_token.is_member(to_account_id)\n                            && !to_resources.has_access_to_container(\n                                access_token,\n                                to_calendar_id,\n                                Acl::RemoveItems,\n                            ))\n                    {\n                        return Err(DavError::Code(StatusCode::FORBIDDEN));\n                    }\n\n                    if is_move {\n                        move_event(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            from_calendar_id,\n                            from_resources.format_item(from_resource_name),\n                            to_account_id,\n                            to_resource.document_id().into(),\n                            to_calendar_id,\n                            new_name,\n                            headers.if_schedule_tag,\n                        )\n                        .await\n                    } else {\n                        copy_event(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            to_account_id,\n                            to_resource.document_id().into(),\n                            to_calendar_id,\n                            new_name,\n                        )\n                        .await\n                    }\n                }\n                _ => Err(DavError::Code(StatusCode::BAD_GATEWAY)),\n            }\n        } else if let Some((parent_resource, new_name)) =\n            to_resources.map_parent(destination_resource_name)\n        {\n            if let Some(parent_resource) = parent_resource {\n                // Creating items under an event is not allowed\n                // Copying/moving containers under a container is not allowed\n                if !parent_resource.is_container() || from_resource.is_container() {\n                    return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n                }\n\n                // Validate ACL\n                let from_calendar_id = from_resource.parent_id().unwrap();\n                let to_calendar_id = parent_resource.document_id();\n                if (!access_token.is_member(from_account_id)\n                    && !from_resources.has_access_to_container(\n                        access_token,\n                        from_calendar_id,\n                        if is_move {\n                            Acl::RemoveItems\n                        } else {\n                            Acl::ReadItems\n                        },\n                    ))\n                    || (!access_token.is_member(to_account_id)\n                        && !to_resources.has_access_to_container(\n                            access_token,\n                            to_calendar_id,\n                            Acl::AddItems,\n                        ))\n                {\n                    return Err(DavError::Code(StatusCode::FORBIDDEN));\n                }\n\n                // Copy/move event\n                if is_move {\n                    if from_account_id != to_account_id\n                        || parent_resource.document_id() != from_calendar_id\n                    {\n                        move_event(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            from_calendar_id,\n                            from_resources.format_item(from_resource_name),\n                            to_account_id,\n                            None,\n                            to_calendar_id,\n                            new_name,\n                            headers.if_schedule_tag,\n                        )\n                        .await\n                    } else {\n                        rename_event(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            from_calendar_id,\n                            new_name,\n                            from_resources.format_item(from_resource_name),\n                        )\n                        .await\n                    }\n                } else {\n                    copy_event(\n                        self,\n                        access_token,\n                        from_account_id,\n                        from_resource.document_id(),\n                        to_account_id,\n                        None,\n                        to_calendar_id,\n                        new_name,\n                    )\n                    .await\n                }\n            } else {\n                // Copying/moving events to the root is not allowed\n                if !from_resource.is_container() {\n                    return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n                }\n\n                // Shared users cannot create containers\n                if !access_token.is_member(to_account_id) {\n                    return Err(DavError::Code(StatusCode::FORBIDDEN));\n                }\n\n                // Validate ACLs\n                if !access_token.is_member(from_account_id)\n                    && !from_resources.has_access_to_container(\n                        access_token,\n                        from_resource.document_id(),\n                        if is_move {\n                            Acl::RemoveItems\n                        } else {\n                            Acl::ReadItems\n                        },\n                    )\n                {\n                    return Err(DavError::Code(StatusCode::FORBIDDEN));\n                }\n\n                // Copy/move container\n                let from_children_ids = from_resources\n                    .subtree(from_resource_name)\n                    .filter(|r| !r.is_container())\n                    .map(|r| r.document_id())\n                    .collect::<Vec<_>>();\n                if is_move {\n                    if from_account_id != to_account_id {\n                        copy_container(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            if headers.depth != Depth::Zero {\n                                from_children_ids\n                            } else {\n                                return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n                            },\n                            from_resources.format_collection(from_resource_name),\n                            to_account_id,\n                            None,\n                            vec![],\n                            new_name,\n                            true,\n                        )\n                        .await\n                    } else {\n                        rename_container(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            new_name,\n                            from_resources.format_collection(from_resource_name),\n                        )\n                        .await\n                    }\n                } else {\n                    copy_container(\n                        self,\n                        access_token,\n                        from_account_id,\n                        from_resource.document_id(),\n                        if headers.depth != Depth::Zero {\n                            from_children_ids\n                        } else {\n                            vec![]\n                        },\n                        from_resources.format_collection(from_resource_name),\n                        to_account_id,\n                        None,\n                        vec![],\n                        new_name,\n                        false,\n                    )\n                    .await\n                }\n            }\n        } else {\n            Err(DavError::Code(StatusCode::CONFLICT))\n        }\n    }\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn copy_event(\n    server: &Server,\n    access_token: &AccessToken,\n    from_account_id: u32,\n    from_document_id: u32,\n    to_account_id: u32,\n    to_document_id: Option<u32>,\n    to_calendar_id: u32,\n    new_name: &str,\n) -> crate::Result<HttpResponse> {\n    // Fetch event\n    let event_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::CalendarEvent,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let event = event_\n        .to_unarchived::<CalendarEvent>()\n        .caused_by(trc::location!())?;\n    let mut batch = BatchBuilder::new();\n\n    // Validate UID\n    assert_is_unique_uid(\n        server,\n        server\n            .fetch_dav_resources(access_token, to_account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?\n            .as_ref(),\n        to_account_id,\n        to_calendar_id,\n        event.inner.data.event.uids().next(),\n    )\n    .await?;\n\n    if from_account_id == to_account_id {\n        let mut new_event = event\n            .deserialize::<CalendarEvent>()\n            .caused_by(trc::location!())?;\n        new_event.names.push(DavName {\n            name: new_name.to_string(),\n            parent_id: to_calendar_id,\n        });\n        new_event\n            .update(\n                access_token,\n                event,\n                from_account_id,\n                from_document_id,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n    } else {\n        let next_email_alarm = event.inner.data.next_alarm(now() as i64, Tz::Floating);\n        let mut new_event = event\n            .deserialize::<CalendarEvent>()\n            .caused_by(trc::location!())?;\n        new_event.names = vec![DavName {\n            name: new_name.to_string(),\n            parent_id: to_calendar_id,\n        }];\n        let to_document_id = server\n            .store()\n            .assign_document_ids(to_account_id, Collection::CalendarEvent, 1)\n            .await\n            .caused_by(trc::location!())?;\n        new_event\n            .insert(\n                access_token,\n                to_account_id,\n                to_document_id,\n                next_email_alarm,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n    }\n\n    let response = if let Some(to_document_id) = to_document_id {\n        // Overwrite event on destination\n        let event_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                to_account_id,\n                Collection::CalendarEvent,\n                to_document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?;\n        if let Some(event_) = event_ {\n            let event = event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            DestroyArchive(event)\n                .delete(\n                    access_token,\n                    to_account_id,\n                    to_document_id,\n                    to_calendar_id,\n                    None,\n                    false,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    } else {\n        Ok(HttpResponse::new(StatusCode::CREATED))\n    };\n\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    response\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn move_event(\n    server: &Server,\n    access_token: &AccessToken,\n    from_account_id: u32,\n    from_document_id: u32,\n    from_calendar_id: u32,\n    from_resource_path: String,\n    to_account_id: u32,\n    to_document_id: Option<u32>,\n    to_calendar_id: u32,\n    new_name: &str,\n    if_schedule_tag: Option<u32>,\n) -> crate::Result<HttpResponse> {\n    // Fetch event\n    let event_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::CalendarEvent,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let event = event_\n        .to_unarchived::<CalendarEvent>()\n        .caused_by(trc::location!())?;\n\n    // Validate headers\n    if if_schedule_tag.is_some()\n        && event.inner.schedule_tag.as_ref().map(|t| t.to_native()) != if_schedule_tag\n    {\n        return Err(DavError::Code(StatusCode::PRECONDITION_FAILED));\n    }\n\n    // Validate UID\n    if from_account_id != to_account_id\n        || from_calendar_id != to_calendar_id\n        || to_document_id.is_none()\n    {\n        assert_is_unique_uid(\n            server,\n            server\n                .fetch_dav_resources(access_token, to_account_id, SyncCollection::Calendar)\n                .await\n                .caused_by(trc::location!())?\n                .as_ref(),\n            to_account_id,\n            to_calendar_id,\n            event.inner.data.event.uids().next(),\n        )\n        .await?;\n    }\n\n    let mut batch = BatchBuilder::new();\n    if from_account_id == to_account_id {\n        let mut name_idx = None;\n        for (idx, name) in event.inner.names.iter().enumerate() {\n            if name.parent_id == from_calendar_id {\n                name_idx = Some(idx);\n                break;\n            }\n        }\n\n        let name_idx = if let Some(name_idx) = name_idx {\n            name_idx\n        } else {\n            return Err(DavError::Code(StatusCode::NOT_FOUND));\n        };\n\n        let mut new_event = event\n            .deserialize::<CalendarEvent>()\n            .caused_by(trc::location!())?;\n        new_event.names.swap_remove(name_idx);\n        new_event.names.push(DavName {\n            name: new_name.to_string(),\n            parent_id: to_calendar_id,\n        });\n        new_event\n            .update(\n                access_token,\n                event.clone(),\n                from_account_id,\n                from_document_id,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n        batch.log_vanished_item(VanishedCollection::Calendar, from_resource_path);\n    } else {\n        let next_email_alarm = event.inner.data.next_alarm(now() as i64, Tz::Floating);\n        let mut new_event = event\n            .deserialize::<CalendarEvent>()\n            .caused_by(trc::location!())?;\n        new_event.names = vec![DavName {\n            name: new_name.to_string(),\n            parent_id: to_calendar_id,\n        }];\n\n        DestroyArchive(event)\n            .delete(\n                access_token,\n                from_account_id,\n                from_document_id,\n                from_calendar_id,\n                from_resource_path.into(),\n                false,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n\n        let to_document_id = server\n            .store()\n            .assign_document_ids(to_account_id, Collection::CalendarEvent, 1)\n            .await\n            .caused_by(trc::location!())?;\n        new_event\n            .insert(\n                access_token,\n                to_account_id,\n                to_document_id,\n                next_email_alarm,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n    }\n\n    let response = if let Some(to_document_id) = to_document_id {\n        // Overwrite event on destination\n        let event_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                to_account_id,\n                Collection::CalendarEvent,\n                to_document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?;\n        if let Some(event_) = event_ {\n            let event = event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            DestroyArchive(event)\n                .delete(\n                    access_token,\n                    to_account_id,\n                    to_document_id,\n                    to_calendar_id,\n                    None,\n                    false,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    } else {\n        Ok(HttpResponse::new(StatusCode::CREATED))\n    };\n\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    response\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn rename_event(\n    server: &Server,\n    access_token: &AccessToken,\n    account_id: u32,\n    document_id: u32,\n    calendar_id: u32,\n    new_name: &str,\n    from_resource_path: String,\n) -> crate::Result<HttpResponse> {\n    // Fetch event\n    let event_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::CalendarEvent,\n            document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let event = event_\n        .to_unarchived::<CalendarEvent>()\n        .caused_by(trc::location!())?;\n\n    let name_idx = event\n        .inner\n        .names\n        .iter()\n        .position(|n| n.parent_id == calendar_id)\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let mut new_event = event\n        .deserialize::<CalendarEvent>()\n        .caused_by(trc::location!())?;\n    new_event.names[name_idx].name = new_name.to_string();\n\n    let mut batch = BatchBuilder::new();\n    new_event\n        .update(access_token, event, account_id, document_id, &mut batch)\n        .caused_by(trc::location!())?;\n    batch.log_vanished_item(VanishedCollection::Calendar, from_resource_path);\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    Ok(HttpResponse::new(StatusCode::CREATED))\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn copy_container(\n    server: &Server,\n    access_token: &AccessToken,\n    from_account_id: u32,\n    from_document_id: u32,\n    from_children_ids: Vec<u32>,\n    from_resource_path: String,\n    to_account_id: u32,\n    to_document_id: Option<u32>,\n    to_children_ids: Vec<u32>,\n    new_name: &str,\n    remove_source: bool,\n) -> crate::Result<HttpResponse> {\n    // Fetch calendar\n    let calendar_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::Calendar,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let old_calendar = calendar_\n        .to_unarchived::<Calendar>()\n        .caused_by(trc::location!())?;\n    let mut calendar = old_calendar\n        .deserialize::<Calendar>()\n        .caused_by(trc::location!())?;\n\n    // Prepare write batch\n    let mut batch = BatchBuilder::new();\n\n    if remove_source {\n        DestroyArchive(old_calendar)\n            .delete(\n                access_token,\n                from_account_id,\n                from_document_id,\n                from_resource_path.into(),\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n    }\n\n    let preference = calendar.preferences.into_iter().next().unwrap();\n    calendar.name = new_name.to_string();\n    calendar.acls.clear();\n    calendar.preferences = vec![CalendarPreferences {\n        account_id: to_account_id,\n        name: preference.name,\n        description: preference.description,\n        default_alerts: preference.default_alerts,\n        sort_order: 0,\n        color: preference.color,\n        flags: 0,\n        time_zone: Timezone::Default,\n    }];\n\n    let is_overwrite = to_document_id.is_some();\n    let to_document_id = if let Some(to_document_id) = to_document_id {\n        // Overwrite destination\n        let calendar_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                to_account_id,\n                Collection::Calendar,\n                to_document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?;\n        if let Some(calendar_) = calendar_ {\n            let calendar = calendar_\n                .to_unarchived::<Calendar>()\n                .caused_by(trc::location!())?;\n\n            DestroyArchive(calendar)\n                .delete_with_events(\n                    server,\n                    access_token,\n                    to_account_id,\n                    to_document_id,\n                    to_children_ids,\n                    None,\n                    false,\n                    &mut batch,\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        to_document_id\n    } else {\n        server\n            .store()\n            .assign_document_ids(to_account_id, Collection::Calendar, 1)\n            .await\n            .caused_by(trc::location!())?\n    };\n    calendar\n        .insert(access_token, to_account_id, to_document_id, &mut batch)\n        .caused_by(trc::location!())?;\n\n    // Copy children\n    let mut required_space = 0;\n    for from_child_document_id in from_children_ids {\n        if let Some(event_) = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                from_account_id,\n                Collection::CalendarEvent,\n                from_child_document_id,\n            ))\n            .await?\n        {\n            let event = event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n            let mut new_name = None;\n\n            for name in event.inner.names.iter() {\n                if name.parent_id == to_document_id {\n                    continue;\n                } else if name.parent_id == from_document_id {\n                    new_name = Some(name.name.to_string());\n                }\n            }\n            let new_name = if let Some(new_name) = new_name {\n                DavName {\n                    name: new_name,\n                    parent_id: to_document_id,\n                }\n            } else {\n                continue;\n            };\n            let event = event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n            let mut new_event = event\n                .deserialize::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            if from_account_id == to_account_id {\n                if remove_source {\n                    new_event\n                        .names\n                        .retain(|name| name.parent_id != from_document_id);\n                }\n\n                new_event.names.push(new_name);\n                new_event\n                    .update(\n                        access_token,\n                        event,\n                        from_account_id,\n                        from_child_document_id,\n                        &mut batch,\n                    )\n                    .caused_by(trc::location!())?;\n            } else {\n                let next_email_alarm = event.inner.data.next_alarm(now() as i64, Tz::Floating);\n                if remove_source {\n                    DestroyArchive(event)\n                        .delete(\n                            access_token,\n                            from_account_id,\n                            from_child_document_id,\n                            from_document_id,\n                            None,\n                            false,\n                            &mut batch,\n                        )\n                        .caused_by(trc::location!())?;\n                }\n                let to_document_id = server\n                    .store()\n                    .assign_document_ids(to_account_id, Collection::CalendarEvent, 1)\n                    .await\n                    .caused_by(trc::location!())?;\n                new_event.names = vec![new_name];\n                required_space += new_event.size as u64;\n                new_event\n                    .insert(\n                        access_token,\n                        to_account_id,\n                        to_document_id,\n                        next_email_alarm,\n                        &mut batch,\n                    )\n                    .caused_by(trc::location!())?;\n            }\n        }\n    }\n\n    if from_account_id != to_account_id && required_space > 0 {\n        server\n            .has_available_quota(\n                &server\n                    .get_resource_token(access_token, to_account_id)\n                    .await?,\n                required_space,\n            )\n            .await?;\n    }\n\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    if !is_overwrite {\n        Ok(HttpResponse::new(StatusCode::CREATED))\n    } else {\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    }\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn rename_container(\n    server: &Server,\n    access_token: &AccessToken,\n    account_id: u32,\n    document_id: u32,\n    new_name: &str,\n    from_resource_path: String,\n) -> crate::Result<HttpResponse> {\n    // Fetch calendar\n    let calendar_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::Calendar,\n            document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let calendar = calendar_\n        .to_unarchived::<Calendar>()\n        .caused_by(trc::location!())?;\n    let mut new_calendar = calendar\n        .deserialize::<Calendar>()\n        .caused_by(trc::location!())?;\n    new_calendar.name = new_name.to_string();\n\n    let mut batch = BatchBuilder::new();\n    new_calendar\n        .update(access_token, calendar, account_id, document_id, &mut batch)\n        .caused_by(trc::location!())?;\n    batch.log_vanished_item(VanishedCollection::Calendar, from_resource_path);\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    Ok(HttpResponse::new(StatusCode::CREATED))\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        ETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse dav_proto::RequestHeaders;\nuse directory::Permission;\nuse groupware::{\n    DestroyArchive,\n    cache::GroupwareCache,\n    calendar::{Calendar, CalendarEvent},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::{BatchBuilder, ValueClass};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n};\n\npub(crate) trait CalendarDeleteRequestHandler: Sync + Send {\n    fn handle_calendar_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CalendarDeleteRequestHandler for Server {\n    async fn handle_calendar_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let delete_path = resource\n            .resource\n            .filter(|r| !r.is_empty())\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Check resource type\n        let delete_resource = resources\n            .by_path(delete_path)\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let document_id = delete_resource.document_id();\n        let send_itip = self.core.groupware.itip_enabled\n            && !headers.no_schedule_reply\n            && !access_token.emails.is_empty()\n            && access_token.has_permission(Permission::CalendarSchedulingSend);\n\n        // Fetch entry\n        let mut batch = BatchBuilder::new();\n        if delete_resource.is_container() {\n            // Deleting the default calendar is not allowed\n            #[cfg(not(debug_assertions))]\n            if self\n                .core\n                .groupware\n                .default_calendar_name\n                .as_ref()\n                .is_some_and(|name| name == delete_path)\n            {\n                return Err(DavError::Condition(crate::DavErrorCondition::new(\n                    StatusCode::FORBIDDEN,\n                    dav_proto::schema::response::CalCondition::DefaultCalendarNeeded,\n                )));\n            }\n\n            let calendar_ = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Calendar,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n            let calendar = calendar_\n                .to_unarchived::<Calendar>()\n                .caused_by(trc::location!())?;\n\n            // Validate ACL\n            if !access_token.is_member(account_id)\n                && !calendar\n                    .inner\n                    .acls\n                    .effective_acl(access_token)\n                    .contains_all([Acl::Delete, Acl::RemoveItems].into_iter())\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Validate headers\n            self.validate_headers(\n                access_token,\n                headers,\n                vec![ResourceState {\n                    account_id,\n                    collection: Collection::Calendar,\n                    document_id: document_id.into(),\n                    etag: calendar.etag().into(),\n                    path: delete_path,\n                    ..Default::default()\n                }],\n                Default::default(),\n                DavMethod::DELETE,\n            )\n            .await?;\n\n            // Delete calendar and events\n            DestroyArchive(calendar)\n                .delete_with_events(\n                    self,\n                    access_token,\n                    account_id,\n                    document_id,\n                    resources\n                        .subtree(delete_path)\n                        .filter(|r| !r.is_container())\n                        .map(|r| r.document_id())\n                        .collect::<Vec<_>>(),\n                    resources.format_resource(delete_resource).into(),\n                    send_itip,\n                    &mut batch,\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n            // Reset default calendar id\n            let default_calendar_id = self\n                .store()\n                .get_value::<u32>(ValueKey {\n                    account_id,\n                    collection: Collection::Principal.into(),\n                    document_id: 0,\n                    class: ValueClass::Property(PrincipalField::DefaultCalendarId.into()),\n                })\n                .await\n                .caused_by(trc::location!())?;\n            if default_calendar_id.is_some_and(|id| id == document_id) {\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .clear(PrincipalField::DefaultCalendarId);\n            }\n        } else {\n            // Validate ACL\n            let calendar_id = delete_resource.parent_id().unwrap();\n            if !access_token.is_member(account_id)\n                && !resources.has_access_to_container(access_token, calendar_id, Acl::RemoveItems)\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            let event_ = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n            // Validate headers\n            self.validate_headers(\n                access_token,\n                headers,\n                vec![ResourceState {\n                    account_id,\n                    collection: Collection::CalendarEvent,\n                    document_id: document_id.into(),\n                    etag: event_.etag().into(),\n                    path: delete_path,\n                    ..Default::default()\n                }],\n                Default::default(),\n                DavMethod::DELETE,\n            )\n            .await?;\n\n            // Validate schedule tag\n            let event = event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n            if headers.if_schedule_tag.is_some()\n                && event.inner.schedule_tag.as_ref().map(|t| t.to_native())\n                    != headers.if_schedule_tag\n            {\n                return Err(DavError::Code(StatusCode::PRECONDITION_FAILED));\n            }\n\n            // Delete event\n            DestroyArchive(event)\n                .delete(\n                    access_token,\n                    account_id,\n                    document_id,\n                    calendar_id,\n                    resources.format_resource(delete_resource).into(),\n                    send_itip,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n        }\n\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n        self.notify_task_queue();\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/freebusy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::query::CalendarQueryHandler;\nuse crate::{DavError, calendar::query::is_resource_in_time_range, common::uri::DavUriResource};\nuse calcard::{\n    common::{PartialDateTime, timezone::Tz},\n    icalendar::{\n        ArchivedICalendarComponentType, ArchivedICalendarEntry, ArchivedICalendarParameterName,\n        ArchivedICalendarParameterValue, ArchivedICalendarProperty, ArchivedICalendarStatus,\n        ArchivedICalendarValue, ICalendar, ICalendarComponent, ICalendarComponentType,\n        ICalendarEntry, ICalendarFreeBusyType, ICalendarParameter, ICalendarPeriod,\n        ICalendarProperty, ICalendarTransparency, ICalendarValue,\n    },\n};\nuse common::{DavResourcePath, DavResources, PROD_ID, Server, auth::AccessToken};\nuse dav_proto::{RequestHeaders, schema::request::FreeBusyQuery};\nuse groupware::{cache::GroupwareCache, calendar::CalendarEvent};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::str::FromStr;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse store::{\n    ahash::AHashMap,\n    write::{now, serialize::rkyv_deserialize},\n};\nuse trc::AddContext;\nuse types::{\n    TimeRange,\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait CalendarFreebusyRequestHandler: Sync + Send {\n    fn handle_calendar_freebusy_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: FreeBusyQuery,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn build_freebusy_object(\n        &self,\n        access_token: &AccessToken,\n        request: FreeBusyQuery,\n        resources: &DavResources,\n        account_id: u32,\n        resource: DavResourcePath<'_>,\n    ) -> impl Future<Output = crate::Result<ICalendar>> + Send;\n}\n\nimpl CalendarFreebusyRequestHandler for Server {\n    async fn handle_calendar_freebusy_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: FreeBusyQuery,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resources\n            .by_path(\n                resource_\n                    .resource\n                    .ok_or(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))?,\n            )\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        if !resource.is_container() {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        self.build_freebusy_object(access_token, request, &resources, account_id, resource)\n            .await\n            .map(|ical| {\n                HttpResponse::new(StatusCode::OK)\n                    .with_content_type(\"text/calendar; charset=utf-8\")\n                    .with_text_body(ical.to_string())\n            })\n    }\n\n    async fn build_freebusy_object(\n        &self,\n        access_token: &AccessToken,\n        request: FreeBusyQuery,\n        resources: &DavResources,\n        account_id: u32,\n        resource: DavResourcePath<'_>,\n    ) -> crate::Result<ICalendar> {\n        // Obtain shared ids\n        let shared_ids = if !access_token.is_member(account_id) {\n            resources\n                .shared_containers(\n                    access_token,\n                    [Acl::ReadItems, Acl::SchedulingReadFreeBusy],\n                    false,\n                )\n                .into()\n        } else {\n            None\n        };\n\n        // Build FreeBusy component\n        let default_tz = resource\n            .resource\n            .calendar_preferences(account_id)\n            .map(|p| p.tz)\n            .unwrap_or(Tz::UTC);\n        let mut entries = Vec::with_capacity(6);\n        if let Some(range) = request.range {\n            entries.push(ICalendarEntry {\n                name: ICalendarProperty::Dtstart,\n                params: vec![],\n                values: vec![ICalendarValue::PartialDateTime(Box::new(\n                    PartialDateTime::from_utc_timestamp(range.start),\n                ))],\n            });\n            entries.push(ICalendarEntry {\n                name: ICalendarProperty::Dtend,\n                params: vec![],\n                values: vec![ICalendarValue::PartialDateTime(Box::new(\n                    PartialDateTime::from_utc_timestamp(range.end),\n                ))],\n            });\n            entries.push(ICalendarEntry {\n                name: ICalendarProperty::Dtstamp,\n                params: vec![],\n                values: vec![ICalendarValue::PartialDateTime(Box::new(\n                    PartialDateTime::from_utc_timestamp(now() as i64),\n                ))],\n            });\n\n            let document_ids = resources\n                .children(resource.document_id())\n                .filter(|resource| {\n                    shared_ids\n                        .as_ref()\n                        .is_none_or(|ids| ids.contains(resource.document_id()))\n                        && is_resource_in_time_range(resource.resource, &range)\n                })\n                .map(|resource| resource.document_id())\n                .collect::<Vec<_>>();\n\n            let mut fb_entries: AHashMap<ICalendarFreeBusyType, Vec<(i64, i64)>> =\n                AHashMap::with_capacity(document_ids.len());\n\n            for document_id in document_ids {\n                let Some(archive) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::CalendarEvent,\n                        document_id,\n                    ))\n                    .await\n                    .caused_by(trc::location!())?\n                else {\n                    continue;\n                };\n                let event = archive\n                    .unarchive::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n\n                /*\n                   Only VEVENT components without a TRANSP property or with the TRANSP\n                   property set to OPAQUE, and VFREEBUSY components SHOULD be considered\n                   in generating the free busy time information.\n                */\n                let mut components = event\n                    .data\n                    .event\n                    .components\n                    .iter()\n                    .enumerate()\n                    .filter(|(_, comp)| {\n                        (matches!(comp.component_type, ArchivedICalendarComponentType::VEvent)\n                            && comp\n                                .transparency()\n                                .is_none_or(|t| t == &ICalendarTransparency::Opaque))\n                            || matches!(\n                                comp.component_type,\n                                ArchivedICalendarComponentType::VFreebusy\n                            )\n                    })\n                    .peekable();\n\n                if components.peek().is_none() {\n                    continue;\n                }\n\n                let events =\n                    CalendarQueryHandler::new(event, Some(range), default_tz).into_expanded_times();\n\n                if events.is_empty() {\n                    continue;\n                }\n\n                for (component_id, component) in components {\n                    let component_id = component_id as u32;\n                    match component.component_type {\n                        ArchivedICalendarComponentType::VEvent => {\n                            let fbtype = match component.status() {\n                                Some(ArchivedICalendarStatus::Cancelled) => continue,\n                                Some(ArchivedICalendarStatus::Tentative) => {\n                                    ICalendarFreeBusyType::BusyTentative\n                                }\n                                _ => ICalendarFreeBusyType::Busy,\n                            };\n\n                            let mut events_in_range = Vec::new();\n                            for event in &events {\n                                if event.comp_id == component_id\n                                    && range.is_in_range(false, event.start, event.end)\n                                {\n                                    events_in_range.push((event.start, event.end));\n                                }\n                            }\n\n                            if !events_in_range.is_empty() {\n                                fb_entries\n                                    .entry(fbtype)\n                                    .or_default()\n                                    .extend(events_in_range);\n                            }\n                        }\n                        ArchivedICalendarComponentType::VFreebusy => {\n                            for entry in component.entries.iter() {\n                                if matches!(entry.name, ArchivedICalendarProperty::Freebusy) {\n                                    let mut fb_in_range =\n                                        freebusy_in_range_utc(entry, &range, default_tz).peekable();\n                                    if fb_in_range.peek().is_some() {\n                                        let fb_type = entry\n                                            .params\n                                            .iter()\n                                            .find_map(|param| {\n                                                if let (\n                                                    ArchivedICalendarParameterName::Fbtype,\n                                                    ArchivedICalendarParameterValue::Fbtype(param),\n                                                ) = (&param.name, &param.value)\n                                                {\n                                                    rkyv_deserialize(param).ok()\n                                                } else {\n                                                    None\n                                                }\n                                            })\n                                            .unwrap_or(ICalendarFreeBusyType::Busy);\n\n                                        fb_entries.entry(fb_type).or_default().extend(fb_in_range);\n                                    }\n                                }\n                            }\n                        }\n                        _ => {}\n                    }\n                }\n            }\n\n            for (fbtype, events_in_range) in fb_entries {\n                entries.push(ICalendarEntry {\n                    name: ICalendarProperty::Freebusy,\n                    params: vec![ICalendarParameter::fbtype(fbtype)],\n                    values: merge_intervals(events_in_range),\n                });\n            }\n        }\n\n        // Build ICalendar\n        Ok(ICalendar {\n            components: vec![\n                ICalendarComponent {\n                    component_type: ICalendarComponentType::VCalendar,\n                    entries: vec![\n                        ICalendarEntry {\n                            name: ICalendarProperty::Version,\n                            params: vec![],\n                            values: vec![ICalendarValue::Text(\"2.0\".to_string())],\n                        },\n                        ICalendarEntry {\n                            name: ICalendarProperty::Prodid,\n                            params: vec![],\n                            values: vec![ICalendarValue::Text(PROD_ID.to_string())],\n                        },\n                    ],\n                    component_ids: vec![1],\n                },\n                ICalendarComponent {\n                    component_type: ICalendarComponentType::VFreebusy,\n                    entries,\n                    component_ids: vec![],\n                },\n            ],\n        })\n    }\n}\n\nfn merge_intervals(mut intervals: Vec<(i64, i64)>) -> Vec<ICalendarValue> {\n    if intervals.len() > 1 {\n        intervals.sort_unstable_by(|a, b| a.0.cmp(&b.0));\n\n        let mut unique_intervals = Vec::new();\n        let mut start_time = intervals[0].0;\n        let mut end_time = intervals[0].1;\n\n        for &(curr_start, curr_end) in intervals.iter().skip(1) {\n            if curr_start <= end_time {\n                end_time = end_time.max(curr_end);\n            } else {\n                unique_intervals.push(build_ical_value(start_time, end_time));\n                start_time = curr_start;\n                end_time = curr_end;\n            }\n        }\n\n        unique_intervals.push(build_ical_value(start_time, end_time));\n        unique_intervals\n    } else {\n        intervals\n            .into_iter()\n            .map(|(start, end)| build_ical_value(start, end))\n            .collect()\n    }\n}\n\nfn build_ical_value(from: i64, to: i64) -> ICalendarValue {\n    ICalendarValue::Period(ICalendarPeriod::Range {\n        start: PartialDateTime::from_utc_timestamp(from),\n        end: PartialDateTime::from_utc_timestamp(to),\n    })\n}\n\npub(crate) fn freebusy_in_range(\n    entry: &ArchivedICalendarEntry,\n    range: &TimeRange,\n    default_tz: Tz,\n) -> impl Iterator<Item = ICalendarValue> {\n    let tz = entry\n        .tz_id()\n        .and_then(|tz_id| Tz::from_str(tz_id).ok())\n        .unwrap_or(default_tz);\n\n    entry.values.iter().filter_map(move |value| {\n        if let ArchivedICalendarValue::Period(period) = &value {\n            period.time_range(tz).and_then(|(start, end)| {\n                let start = start.timestamp();\n                let end = end.timestamp();\n                if range.is_in_range(false, start, end) {\n                    rkyv_deserialize(value).ok()\n                } else {\n                    None\n                }\n            })\n        } else {\n            None\n        }\n    })\n}\n\nfn freebusy_in_range_utc(\n    entry: &ArchivedICalendarEntry,\n    range: &TimeRange,\n    default_tz: Tz,\n) -> impl Iterator<Item = (i64, i64)> {\n    let tz = entry\n        .tz_id()\n        .and_then(|tz_id| Tz::from_str(tz_id).ok())\n        .unwrap_or(default_tz);\n\n    entry.values.iter().filter_map(move |value| {\n        if let ArchivedICalendarValue::Period(period) = &value {\n            period.time_range(tz).and_then(|(start, end)| {\n                let start = start.timestamp();\n                let end = end.timestamp();\n                if range.is_in_range(false, start, end) {\n                    Some((start, end))\n                } else {\n                    None\n                }\n            })\n        } else {\n            None\n        }\n    })\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        ETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{RequestHeaders, schema::property::Rfc1123DateTime};\nuse groupware::{cache::GroupwareCache, calendar::CalendarEvent};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait CalendarGetRequestHandler: Sync + Send {\n    fn handle_calendar_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CalendarGetRequestHandler for Server {\n    async fn handle_calendar_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resources\n            .by_path(\n                resource_\n                    .resource\n                    .ok_or(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))?,\n            )\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        if resource.is_container() {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Validate ACL\n        if !access_token.is_member(account_id)\n            && !resources.has_access_to_container(\n                access_token,\n                resource.parent_id().unwrap(),\n                Acl::ReadItems,\n            )\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Fetch event\n        let event_ = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::CalendarEvent,\n                resource.document_id(),\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let event = event_\n            .unarchive::<CalendarEvent>()\n            .caused_by(trc::location!())?;\n\n        // Validate headers\n        let etag = event_.etag();\n        let schedule_tag = event.schedule_tag.as_ref().map(|tag| tag.to_native());\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: Collection::CalendarEvent,\n                document_id: resource.document_id().into(),\n                etag: etag.clone().into(),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::GET,\n        )\n        .await?;\n\n        let response = HttpResponse::new(StatusCode::OK)\n            .with_content_type(\"text/calendar; charset=utf-8\")\n            .with_etag(etag)\n            .with_schedule_tag_opt(schedule_tag)\n            .with_last_modified(Rfc1123DateTime::new(i64::from(event.modified)).to_string());\n\n        let ical = event.data.event.to_string();\n\n        if !is_head {\n            Ok(response.with_binary_body(ical))\n        } else {\n            Ok(response.with_content_length(ical.len()))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/mkcol.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::proppatch::CalendarPropPatchRequestHandler;\nuse crate::{\n    DavError, DavMethod, PropStatBuilder,\n    common::{\n        ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{Namespace, request::MkCol, response::MkColResponse},\n};\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{Calendar, CalendarPreferences},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::BatchBuilder;\nuse trc::AddContext;\nuse types::collection::{Collection, SyncCollection};\n\npub(crate) trait CalendarMkColRequestHandler: Sync + Send {\n    fn handle_calendar_mkcol_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: Option<MkCol>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CalendarMkColRequestHandler for Server {\n    async fn handle_calendar_mkcol_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: Option<MkCol>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let name = resource\n            .resource\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        if !access_token.is_member(account_id) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        } else if name.contains('/')\n            || self\n                .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n                .await\n                .caused_by(trc::location!())?\n                .by_path(name)\n                .is_some()\n        {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: resource.collection,\n                document_id: Some(u32::MAX),\n                path: name,\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::MKCOL,\n        )\n        .await?;\n\n        // Build file container\n        let mut calendar = Calendar {\n            name: name.to_string(),\n            preferences: vec![CalendarPreferences {\n                account_id,\n                name: name.to_string(),\n                ..Default::default()\n            }],\n            ..Default::default()\n        };\n\n        // Apply MKCOL properties\n        let mut return_prop_stat = None;\n        let mut is_mkcalendar = false;\n        if let Some(mkcol) = request {\n            let mut prop_stat = PropStatBuilder::default();\n            is_mkcalendar = mkcol.is_mkcalendar;\n            if !self.apply_calendar_properties(\n                access_token,\n                &mut calendar,\n                false,\n                mkcol.props,\n                &mut prop_stat,\n            ) {\n                return Ok(HttpResponse::new(StatusCode::FORBIDDEN).with_xml_body(\n                    MkColResponse::new(prop_stat.build())\n                        .with_namespace(Namespace::CalDav)\n                        .with_mkcalendar(is_mkcalendar)\n                        .to_string(),\n                ));\n            }\n            if headers.ret != Return::Minimal {\n                return_prop_stat = Some(prop_stat);\n            }\n        }\n\n        // Prepare write batch\n        let mut batch = BatchBuilder::new();\n        let document_id = self\n            .store()\n            .assign_document_ids(account_id, Collection::Calendar, 1)\n            .await\n            .caused_by(trc::location!())?;\n        calendar\n            .insert(access_token, account_id, document_id, &mut batch)\n            .caused_by(trc::location!())?;\n        let etag = batch.etag();\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n        if let Some(prop_stat) = return_prop_stat {\n            Ok(HttpResponse::new(StatusCode::CREATED)\n                .with_xml_body(\n                    MkColResponse::new(prop_stat.build())\n                        .with_namespace(Namespace::CalDav)\n                        .with_mkcalendar(is_mkcalendar)\n                        .to_string(),\n                )\n                .with_etag_opt(etag))\n        } else {\n            Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod copy_move;\npub mod delete;\npub mod freebusy;\npub mod get;\npub mod mkcol;\npub mod proppatch;\npub mod query;\npub mod scheduling;\npub mod update;\n\nuse crate::{DavError, DavErrorCondition};\nuse common::{DavResources, Server};\nuse dav_proto::schema::{\n    property::{CalDavProperty, CalendarData, DavProperty, WebDavProperty},\n    response::CalCondition,\n};\nuse groupware::scheduling::ItipError;\nuse hyper::StatusCode;\nuse trc::AddContext;\nuse types::{collection::Collection, field::CalendarEventField};\n\npub(crate) static CALENDAR_CONTAINER_PROPS: [DavProperty; 31] = [\n    DavProperty::WebDav(WebDavProperty::CreationDate),\n    DavProperty::WebDav(WebDavProperty::DisplayName),\n    DavProperty::WebDav(WebDavProperty::GetETag),\n    DavProperty::WebDav(WebDavProperty::GetLastModified),\n    DavProperty::WebDav(WebDavProperty::ResourceType),\n    DavProperty::WebDav(WebDavProperty::LockDiscovery),\n    DavProperty::WebDav(WebDavProperty::SupportedLock),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n    DavProperty::WebDav(WebDavProperty::SyncToken),\n    DavProperty::WebDav(WebDavProperty::Owner),\n    DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::Acl),\n    DavProperty::WebDav(WebDavProperty::AclRestrictions),\n    DavProperty::WebDav(WebDavProperty::InheritedAclSet),\n    DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n    DavProperty::WebDav(WebDavProperty::SupportedReportSet),\n    DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes),\n    DavProperty::WebDav(WebDavProperty::QuotaUsedBytes),\n    DavProperty::CalDav(CalDavProperty::CalendarDescription),\n    DavProperty::CalDav(CalDavProperty::SupportedCalendarData),\n    DavProperty::CalDav(CalDavProperty::SupportedCollationSet),\n    DavProperty::CalDav(CalDavProperty::SupportedCalendarComponentSet),\n    DavProperty::CalDav(CalDavProperty::CalendarTimezone),\n    DavProperty::CalDav(CalDavProperty::MaxResourceSize),\n    DavProperty::CalDav(CalDavProperty::MinDateTime),\n    DavProperty::CalDav(CalDavProperty::MaxDateTime),\n    DavProperty::CalDav(CalDavProperty::MaxInstances),\n    DavProperty::CalDav(CalDavProperty::MaxAttendeesPerInstance),\n    DavProperty::CalDav(CalDavProperty::TimezoneServiceSet),\n    DavProperty::CalDav(CalDavProperty::TimezoneId),\n];\n\npub(crate) static CALENDAR_ITEM_PROPS: [DavProperty; 20] = [\n    DavProperty::WebDav(WebDavProperty::CreationDate),\n    DavProperty::WebDav(WebDavProperty::DisplayName),\n    DavProperty::WebDav(WebDavProperty::GetETag),\n    DavProperty::WebDav(WebDavProperty::GetLastModified),\n    DavProperty::WebDav(WebDavProperty::ResourceType),\n    DavProperty::WebDav(WebDavProperty::LockDiscovery),\n    DavProperty::WebDav(WebDavProperty::SupportedLock),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n    DavProperty::WebDav(WebDavProperty::SyncToken),\n    DavProperty::WebDav(WebDavProperty::Owner),\n    DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::Acl),\n    DavProperty::WebDav(WebDavProperty::AclRestrictions),\n    DavProperty::WebDav(WebDavProperty::InheritedAclSet),\n    DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n    DavProperty::WebDav(WebDavProperty::GetContentLanguage),\n    DavProperty::WebDav(WebDavProperty::GetContentLength),\n    DavProperty::WebDav(WebDavProperty::GetContentType),\n    DavProperty::CalDav(CalDavProperty::CalendarData(CalendarData {\n        properties: vec![],\n        expand: None,\n        limit_recurrence: None,\n        limit_freebusy: None,\n    })),\n];\n\npub(crate) async fn assert_is_unique_uid(\n    server: &Server,\n    resources: &DavResources,\n    account_id: u32,\n    calendar_id: u32,\n    uid: Option<&str>,\n) -> crate::Result<()> {\n    if let Some(uid) = uid {\n        let hits = server\n            .document_ids_matching(\n                account_id,\n                Collection::CalendarEvent,\n                CalendarEventField::Uid,\n                uid.as_bytes(),\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if !hits.is_empty() {\n            for path in resources.children(calendar_id) {\n                if hits.contains(path.document_id()) {\n                    return Err(DavError::Condition(DavErrorCondition::new(\n                        StatusCode::PRECONDITION_FAILED,\n                        CalCondition::NoUidConflict(resources.format_resource(path).into()),\n                    )));\n                }\n            }\n        }\n    }\n\n    Ok(())\n}\n\npub(crate) trait ItipPrecondition {\n    fn failed_precondition(&self) -> Option<CalCondition>;\n}\n\nimpl ItipPrecondition for ItipError {\n    fn failed_precondition(&self) -> Option<CalCondition> {\n        match self {\n            ItipError::MultipleOrganizer => Some(CalCondition::SameOrganizerInAllComponents),\n            ItipError::OrganizerIsLocalAddress\n            | ItipError::SenderIsNotParticipant(_)\n            | ItipError::OrganizerMismatch => Some(CalCondition::ValidOrganizer),\n            ItipError::CannotModifyProperty(_)\n            | ItipError::CannotModifyInstance\n            | ItipError::CannotModifyAddress => Some(CalCondition::AllowedAttendeeObjectChange),\n            ItipError::MissingUid\n            | ItipError::MultipleUid\n            | ItipError::MultipleObjectTypes\n            | ItipError::MultipleObjectInstances\n            | ItipError::MissingMethod\n            | ItipError::InvalidComponentType\n            | ItipError::OutOfSequence\n            | ItipError::UnknownParticipant(_)\n            | ItipError::UnsupportedMethod(_) => Some(CalCondition::ValidSchedulingMessage),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/proppatch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod, PropStatBuilder,\n    common::{\n        ETag, ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse calcard::common::timezone::Tz;\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{\n        Namespace,\n        property::{CalDavProperty, DavProperty, DavValue, ResourceType, WebDavProperty},\n        request::{DavPropertyValue, PropertyUpdate},\n        response::{BaseCondition, CalCondition, MultiStatus, Response},\n    },\n};\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{Calendar, CalendarEvent, SupportedComponent, Timezone},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::str::FromStr;\nuse store::write::BatchBuilder;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\nuse utils::map::bitmap::Bitmap;\n\npub(crate) trait CalendarPropPatchRequestHandler: Sync + Send {\n    fn handle_calendar_proppatch_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: PropertyUpdate,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn apply_calendar_properties(\n        &self,\n        access_token: &AccessToken,\n        calendar: &mut Calendar,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool;\n\n    fn apply_event_properties(\n        &self,\n        event: &mut CalendarEvent,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool;\n}\n\nimpl CalendarPropPatchRequestHandler for Server {\n    async fn handle_calendar_proppatch_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        mut request: PropertyUpdate,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let uri = headers.uri;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resource_\n            .resource\n            .and_then(|r| resources.by_path(r))\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let document_id = resource.document_id();\n        let collection = if resource.is_container() {\n            Collection::Calendar\n        } else {\n            Collection::CalendarEvent\n        };\n\n        if !request.has_changes() {\n            return Ok(HttpResponse::new(StatusCode::NO_CONTENT));\n        }\n\n        // Verify ACL\n        if !access_token.is_member(account_id) {\n            let (acl, document_id) = if resource.is_container() {\n                (Acl::Modify, resource.document_id())\n            } else {\n                (Acl::ModifyItems, resource.parent_id().unwrap())\n            };\n\n            if !resources.has_access_to_container(access_token, document_id, acl) {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n        }\n\n        // Fetch archive\n        let archive = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                collection,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection,\n                document_id: document_id.into(),\n                etag: archive.etag().into(),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::PROPPATCH,\n        )\n        .await?;\n\n        let is_success;\n        let mut batch = BatchBuilder::new();\n        let mut items = PropStatBuilder::default();\n\n        let etag = if resource.is_container() {\n            // Deserialize\n            let calendar = archive\n                .to_unarchived::<Calendar>()\n                .caused_by(trc::location!())?;\n            let mut new_calendar = archive\n                .deserialize::<Calendar>()\n                .caused_by(trc::location!())?;\n\n            // Remove properties\n            if !request.set_first && !request.remove.is_empty() {\n                remove_calendar_properties(\n                    access_token,\n                    &mut new_calendar,\n                    std::mem::take(&mut request.remove),\n                    &mut items,\n                );\n            }\n\n            // Set properties\n            is_success = self.apply_calendar_properties(\n                access_token,\n                &mut new_calendar,\n                true,\n                request.set,\n                &mut items,\n            );\n\n            // Remove properties\n            if is_success && !request.remove.is_empty() {\n                remove_calendar_properties(\n                    access_token,\n                    &mut new_calendar,\n                    request.remove,\n                    &mut items,\n                );\n            }\n\n            if is_success {\n                new_calendar\n                    .update(access_token, calendar, account_id, document_id, &mut batch)\n                    .caused_by(trc::location!())?\n                    .etag()\n            } else {\n                calendar.etag().into()\n            }\n        } else {\n            // Deserialize\n            let event = archive\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n            let mut new_event = archive\n                .deserialize::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            // Remove properties\n            if !request.set_first && !request.remove.is_empty() {\n                remove_event_properties(\n                    &mut new_event,\n                    std::mem::take(&mut request.remove),\n                    &mut items,\n                );\n            }\n\n            // Set properties\n            is_success = self.apply_event_properties(&mut new_event, true, request.set, &mut items);\n\n            // Remove properties\n            if is_success && !request.remove.is_empty() {\n                remove_event_properties(&mut new_event, request.remove, &mut items);\n            }\n\n            if is_success {\n                new_event\n                    .update(access_token, event, account_id, document_id, &mut batch)\n                    .caused_by(trc::location!())?\n                    .etag()\n            } else {\n                event.etag().into()\n            }\n        };\n\n        if is_success {\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n        }\n\n        if headers.ret != Return::Minimal || !is_success {\n            Ok(HttpResponse::new(StatusCode::MULTI_STATUS)\n                .with_xml_body(\n                    MultiStatus::new(vec![Response::new_propstat(uri, items.build())])\n                        .with_namespace(Namespace::CalDav)\n                        .to_string(),\n                )\n                .with_etag_opt(etag))\n        } else {\n            Ok(HttpResponse::new(StatusCode::NO_CONTENT).with_etag_opt(etag))\n        }\n    }\n\n    fn apply_calendar_properties(\n        &self,\n        access_token: &AccessToken,\n        calendar: &mut Calendar,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool {\n        let mut has_errors = false;\n\n        for property in properties {\n            match (&property.property, property.value) {\n                (DavProperty::WebDav(WebDavProperty::DisplayName), DavValue::String(name)) => {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        calendar.preferences_mut(access_token).name = name;\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (\n                    DavProperty::CalDav(CalDavProperty::CalendarDescription),\n                    DavValue::String(name),\n                ) => {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        calendar.preferences_mut(access_token).description = Some(name);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n\n                        has_errors = true;\n                    }\n                }\n                (\n                    DavProperty::CalDav(CalDavProperty::CalendarTimezone),\n                    DavValue::ICalendar(ical),\n                ) => {\n                    if ical.size() > self.core.groupware.max_ical_size {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    } else if !ical.is_timezone() {\n                        items.insert_precondition_failed_with_description(\n                            property.property,\n                            StatusCode::PRECONDITION_FAILED,\n                            CalCondition::ValidCalendarData,\n                            \"Invalid calendar timezone\",\n                        );\n                        has_errors = true;\n                    } else {\n                        calendar.preferences_mut(access_token).time_zone = Timezone::Custom(ical);\n                        items.insert_ok(property.property);\n                    }\n                }\n                (DavProperty::CalDav(CalDavProperty::TimezoneId), DavValue::String(tz_id)) => {\n                    if let Ok(tz) = Tz::from_str(&tz_id) {\n                        calendar.preferences_mut(access_token).time_zone =\n                            Timezone::IANA(tz.as_id());\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_precondition_failed_with_description(\n                            property.property,\n                            StatusCode::PRECONDITION_FAILED,\n                            CalCondition::ValidTimezone,\n                            \"Invalid timezone ID\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (DavProperty::WebDav(WebDavProperty::CreationDate), DavValue::Timestamp(dt)) => {\n                    calendar.created = dt;\n                    items.insert_ok(property.property);\n                }\n                (\n                    DavProperty::WebDav(WebDavProperty::ResourceType),\n                    DavValue::ResourceTypes(types),\n                ) => {\n                    if !types\n                        .0\n                        .iter()\n                        .all(|rt| matches!(rt, ResourceType::Collection | ResourceType::Calendar))\n                    {\n                        items.insert_precondition_failed(\n                            property.property,\n                            StatusCode::FORBIDDEN,\n                            BaseCondition::ValidResourceType,\n                        );\n                        has_errors = true;\n                    } else {\n                        items.insert_ok(property.property);\n                    }\n                }\n                (\n                    DavProperty::CalDav(CalDavProperty::SupportedCalendarComponentSet),\n                    DavValue::Components(components),\n                ) => {\n                    if !is_update {\n                        calendar.supported_components = Bitmap::<SupportedComponent>::from_iter(\n                            components\n                                .0\n                                .into_iter()\n                                .map(|v| SupportedComponent::from(v.0)),\n                        )\n                        .into_inner();\n                        if calendar.supported_components != 0 {\n                            items.insert_ok(property.property);\n                        } else {\n                            items.insert_precondition_failed_with_description(\n                                property.property,\n                                StatusCode::PRECONDITION_FAILED,\n                                CalCondition::SupportedCalendarComponent,\n                                \"At least one supported component must be specified\",\n                            );\n                            has_errors = true;\n                        }\n                    } else {\n                        items.insert_precondition_failed_with_description(\n                            property.property,\n                            StatusCode::PRECONDITION_FAILED,\n                            CalCondition::SupportedCalendarComponent,\n                            \"Property cannot be modified\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (DavProperty::DeadProperty(dead), DavValue::DeadProperty(values))\n                    if self.core.groupware.dead_property_size.is_some() =>\n                {\n                    if is_update {\n                        calendar.dead_properties.remove_element(dead);\n                    }\n\n                    if calendar.dead_properties.size() + values.size() + dead.size()\n                        < self.core.groupware.dead_property_size.unwrap()\n                    {\n                        calendar.dead_properties.add_element(dead.clone(), values.0);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n\n                        has_errors = true;\n                    }\n                }\n                (_, DavValue::Null) => {\n                    items.insert_ok(property.property);\n                }\n                _ => {\n                    items.insert_error_with_description(\n                        property.property,\n                        StatusCode::CONFLICT,\n                        \"Property cannot be modified\",\n                    );\n                    has_errors = true;\n                }\n            }\n        }\n\n        !has_errors\n    }\n\n    fn apply_event_properties(\n        &self,\n        event: &mut CalendarEvent,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool {\n        let mut has_errors = false;\n\n        for property in properties {\n            match (&property.property, property.value) {\n                (DavProperty::WebDav(WebDavProperty::DisplayName), DavValue::String(name)) => {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        event.display_name = Some(name);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (DavProperty::WebDav(WebDavProperty::CreationDate), DavValue::Timestamp(dt)) => {\n                    event.created = dt;\n                    items.insert_ok(property.property);\n                }\n                (DavProperty::DeadProperty(dead), DavValue::DeadProperty(values))\n                    if self.core.groupware.dead_property_size.is_some() =>\n                {\n                    if is_update {\n                        event.dead_properties.remove_element(dead);\n                    }\n\n                    if event.dead_properties.size() + values.size() + dead.size()\n                        < self.core.groupware.dead_property_size.unwrap()\n                    {\n                        event.dead_properties.add_element(dead.clone(), values.0);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (_, DavValue::Null) => {\n                    items.insert_ok(property.property);\n                }\n                _ => {\n                    items.insert_error_with_description(\n                        property.property,\n                        StatusCode::CONFLICT,\n                        \"Property cannot be modified\",\n                    );\n                    has_errors = true;\n                }\n            }\n        }\n\n        !has_errors\n    }\n}\n\nfn remove_event_properties(\n    event: &mut CalendarEvent,\n    properties: Vec<DavProperty>,\n    items: &mut PropStatBuilder,\n) {\n    for property in properties {\n        match &property {\n            DavProperty::WebDav(WebDavProperty::DisplayName) => {\n                event.display_name = None;\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::DeadProperty(dead) => {\n                event.dead_properties.remove_element(dead);\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            _ => {\n                items.insert_error_with_description(\n                    property,\n                    StatusCode::CONFLICT,\n                    \"Property cannot be deleted\",\n                );\n            }\n        }\n    }\n}\n\nfn remove_calendar_properties(\n    access_token: &AccessToken,\n    calendar: &mut Calendar,\n    properties: Vec<DavProperty>,\n    items: &mut PropStatBuilder,\n) {\n    for property in properties {\n        match &property {\n            DavProperty::CalDav(CalDavProperty::CalendarDescription) => {\n                calendar.preferences_mut(access_token).description = None;\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::CalDav(CalDavProperty::CalendarTimezone)\n            | DavProperty::CalDav(CalDavProperty::TimezoneId) => {\n                calendar.preferences_mut(access_token).time_zone = Timezone::Default;\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::DeadProperty(dead) => {\n                calendar.dead_properties.remove_element(dead);\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            _ => {\n                items.insert_error_with_description(\n                    property,\n                    StatusCode::CONFLICT,\n                    \"Property cannot be deleted\",\n                );\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::freebusy::freebusy_in_range;\nuse crate::{\n    DavError,\n    common::{\n        CalendarFilter, DavQuery,\n        propfind::{PropFindItem, PropFindRequestHandler},\n        uri::DavUriResource,\n    },\n};\nuse calcard::{\n    common::{PartialDateTime, timezone::Tz},\n    icalendar::{\n        ArchivedICalendar, ArchivedICalendarComponent, ArchivedICalendarEntry,\n        ArchivedICalendarParameter, ArchivedICalendarProperty, ArchivedICalendarValue,\n        ICalendarComponentType, ICalendarEntry, ICalendarParameterName, ICalendarProperty,\n        ICalendarValue,\n    },\n};\nuse common::{DavResource, Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders,\n    schema::{\n        property::{CalDavProperty, CalendarData, DavProperty},\n        request::{CalendarQuery, Filter, FilterOp, PropFind, Timezone},\n        response::MultiStatus,\n    },\n};\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{ArchivedCalendarEvent, expand::CalendarEventExpansion},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::{fmt::Write, slice::Iter, str::FromStr};\nuse store::{\n    ahash::{AHashMap, AHashSet},\n    write::serialize::rkyv_deserialize,\n};\nuse trc::AddContext;\nuse types::{TimeRange, acl::Acl, collection::SyncCollection};\n\npub(crate) trait CalendarQueryRequestHandler: Sync + Send {\n    fn handle_calendar_query_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: CalendarQuery,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CalendarQueryRequestHandler for Server {\n    async fn handle_calendar_query_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: CalendarQuery,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let Some(resource) = resources.by_path(\n            resource_\n                .resource\n                .ok_or(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))?,\n        ) else {\n            return Ok(HttpResponse::new(StatusCode::MULTI_STATUS)\n                .with_xml_body(MultiStatus::not_found(headers.uri).to_string()));\n        };\n        if !resource.is_container() {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Obtain shared ids\n        let shared_ids = if !access_token.is_member(account_id) {\n            resources\n                .shared_containers(access_token, [Acl::ReadItems], false)\n                .into()\n        } else {\n            None\n        };\n\n        // Pre-filter by date range\n        let filter_range = extract_filter_range(&request);\n\n        // Obtain document ids in folder\n        let mut items = Vec::with_capacity(16);\n        for resource in resources.children(resource.document_id()) {\n            if shared_ids\n                .as_ref()\n                .is_none_or(|ids| ids.contains(resource.document_id()))\n                && filter_range\n                    .as_ref()\n                    .is_none_or(|range| is_resource_in_time_range(resource.resource, range))\n            {\n                items.push(PropFindItem::new(\n                    resources.format_resource(resource),\n                    account_id,\n                    resource,\n                ));\n            }\n        }\n\n        // Extract the time range from the request\n        let max_time_range = extract_data_range(&request.properties, filter_range);\n\n        self.handle_dav_query(\n            access_token,\n            DavQuery::calendar_query(request, max_time_range, items, headers),\n        )\n        .await\n    }\n}\n\npub(crate) fn is_resource_in_time_range(resource: &DavResource, filter: &TimeRange) -> bool {\n    // Check whether the resource has a time range and if it overlaps with the filter\n    if let Some((start, end)) = resource.event_time_range() {\n        ((filter.start < end) || (filter.start <= start))\n            && (filter.end > start || filter.end >= end)\n    } else {\n        // If the resource does not have a time range, it is not in the range\n        false\n    }\n}\n\nfn extract_filter_range(query: &CalendarQuery) -> Option<TimeRange> {\n    let mut range = TimeRange {\n        start: i64::MAX,\n        end: i64::MIN,\n    };\n\n    for filter in &query.filters {\n        let op = match filter {\n            Filter::Component { op, .. } => op,\n            Filter::Property { op, .. } => op,\n            Filter::Parameter { op, .. } => op,\n            _ => continue,\n        };\n        if let FilterOp::TimeRange(date_range) = op {\n            if date_range.start < range.start {\n                range.start = date_range.start;\n            }\n            if date_range.end > range.end {\n                range.end = date_range.end;\n            }\n        }\n    }\n\n    if range.start != i64::MAX {\n        Some(range)\n    } else {\n        None\n    }\n}\n\nfn extract_data_range(propfind: &PropFind, filter_range: Option<TimeRange>) -> Option<TimeRange> {\n    let props = match propfind {\n        PropFind::AllProp(props) | PropFind::Prop(props) => props,\n        PropFind::PropName => &[][..],\n    };\n\n    for prop in props {\n        if let DavProperty::CalDav(CalDavProperty::CalendarData(data)) = prop {\n            let mut range = filter_range.unwrap_or(TimeRange {\n                start: i64::MAX,\n                end: i64::MIN,\n            });\n\n            for data_range in [&data.expand, &data.limit_recurrence, &data.limit_freebusy]\n                .into_iter()\n                .flatten()\n            {\n                if data_range.start < range.start {\n                    range.start = data_range.start;\n                }\n                if data_range.end > range.end {\n                    range.end = data_range.end;\n                }\n            }\n\n            return if range.start != i64::MAX {\n                Some(range)\n            } else {\n                None\n            };\n        }\n    }\n\n    filter_range\n}\n\npub fn try_parse_tz(tz: &Timezone) -> Option<Tz> {\n    match tz {\n        Timezone::Name(value) | Timezone::Id(value) => Tz::from_str(value).ok(),\n        Timezone::None => None,\n    }\n}\n\npub(crate) struct CalendarQueryHandler {\n    default_tz: Tz,\n    expanded_times: Vec<CalendarEventExpansion>,\n}\n\nimpl CalendarQueryHandler {\n    pub fn new(\n        event: &ArchivedCalendarEvent,\n        max_time_range: Option<TimeRange>,\n        default_tz: Tz,\n    ) -> Self {\n        Self {\n            default_tz,\n            expanded_times: max_time_range\n                .map(|max_time_range| {\n                    event\n                        .data\n                        .expand(default_tz, max_time_range)\n                        .unwrap_or_else(|| {\n                            trc::event!(\n                                Calendar(trc::CalendarEvent::RuleExpansionError),\n                                Reason = \"chrono error\",\n                                Details = event.data.event.to_string(),\n                            );\n                            vec![]\n                        })\n                })\n                .unwrap_or_default(),\n        }\n    }\n\n    pub fn filter(&mut self, event: &ArchivedCalendarEvent, filters: &CalendarFilter) -> bool {\n        let ical = &event.data.event;\n        let mut is_all = true;\n        let mut matches_one = false;\n\n        for filter in filters {\n            match filter {\n                Filter::AnyOf => {\n                    is_all = false;\n                }\n                Filter::AllOf => {\n                    is_all = true;\n                }\n                Filter::Property { prop, op, comp } => {\n                    let mut properties = find_components(ical, comp)\n                        .flat_map(|(_, comp)| find_properties(comp, prop))\n                        .peekable();\n\n                    let result = if properties.peek().is_some() {\n                        properties.any(|entry| {\n                            match op {\n                                FilterOp::Exists => true,\n                                FilterOp::Undefined => false,\n                                FilterOp::TextMatch(text_match) => {\n                                    let mut matched_any = false;\n\n                                    for value in entry.values.iter() {\n                                        if let Some(text) = value.as_text()\n                                            && text_match.matches(text)\n                                        {\n                                            matched_any = true;\n                                            break;\n                                        }\n                                    }\n\n                                    matched_any\n                                }\n                                FilterOp::TimeRange(range) => {\n                                    if let Some(ArchivedICalendarValue::PartialDateTime(date)) =\n                                        entry.values.first()\n                                    {\n                                        let tz = entry\n                                            .tz_id()\n                                            .and_then(|tz_id| Tz::from_str(tz_id).ok())\n                                            .unwrap_or(self.default_tz);\n\n                                        if let Some(date) = date\n                                            .to_date_time()\n                                            .and_then(|date| date.to_date_time_with_tz(tz))\n                                        {\n                                            let timestamp = date.timestamp();\n                                            // RFC4791#9.9: start <= DTSTART AND end > DTSTART\n                                            range.start <= timestamp && range.end > timestamp\n                                        } else {\n                                            false\n                                        }\n                                    } else {\n                                        false\n                                    }\n                                }\n                            }\n                        })\n                    } else {\n                        matches!(op, FilterOp::Undefined)\n                    };\n\n                    if result {\n                        matches_one = true;\n                    } else if is_all {\n                        return false;\n                    }\n                }\n                Filter::Parameter {\n                    prop,\n                    param,\n                    op,\n                    comp,\n                } => {\n                    let mut parameters = find_components(ical, comp)\n                        .flat_map(|(_, comp)| {\n                            find_properties(comp, prop)\n                                .filter_map(|entry| find_parameter(entry, param))\n                        })\n                        .peekable();\n\n                    let result = if parameters.peek().is_some() {\n                        parameters.any(|entry| match op {\n                            FilterOp::Exists => true,\n                            FilterOp::Undefined => false,\n                            FilterOp::TextMatch(text_match) => {\n                                if let Some(text) = entry.value.as_text() {\n                                    text_match.matches(text)\n                                } else {\n                                    false\n                                }\n                            }\n                            FilterOp::TimeRange(_) => false,\n                        })\n                    } else {\n                        matches!(op, FilterOp::Undefined)\n                    };\n\n                    if result {\n                        matches_one = true;\n                    } else if is_all {\n                        return false;\n                    }\n                }\n                Filter::Component { comp, op } => {\n                    let result = match op {\n                        FilterOp::Exists => find_components(ical, comp).next().is_some(),\n                        FilterOp::Undefined => find_components(ical, comp).next().is_none(),\n                        FilterOp::TimeRange(range) => {\n                            if !matches!(comp.last(), Some(ICalendarComponentType::VAlarm)) {\n                                let matching_comp_ids = find_components(ical, comp)\n                                    .map(|(id, comp)| (id as u32, &comp.component_type))\n                                    .collect::<AHashMap<_, _>>();\n\n                                !matching_comp_ids.is_empty()\n                                    && self.expanded_times.iter().any(|event| {\n                                        matching_comp_ids.get(&event.comp_id).is_some_and(|ct| {\n                                            range.is_in_range(\n                                                ct == &&ICalendarComponentType::VTodo,\n                                                event.start,\n                                                event.end,\n                                            )\n                                        })\n                                    })\n                            } else {\n                                let matching_comp_ids = event\n                                    .data\n                                    .alarms\n                                    .iter()\n                                    .map(|alarm| alarm.parent_id.to_native() as u32)\n                                    .collect::<AHashSet<_>>();\n\n                                !matching_comp_ids.is_empty()\n                                    && self.expanded_times.iter().any(|time| {\n                                        matching_comp_ids.contains(&time.comp_id)\n                                            && event.data.alarms.iter().any(|alarm| {\n                                                alarm.parent_id.to_native() as u32 == time.comp_id\n                                                    && alarm\n                                                        .delta\n                                                        .to_timestamp(\n                                                            time.start,\n                                                            time.end,\n                                                            self.default_tz,\n                                                        )\n                                                        .is_some_and(|timestamp| {\n                                                            range.is_in_range(\n                                                                false, timestamp, timestamp,\n                                                            )\n                                                        })\n                                            })\n                                    })\n                            }\n                        }\n                        FilterOp::TextMatch(_) => false,\n                    };\n\n                    if result {\n                        matches_one = true;\n                    } else if is_all {\n                        return false;\n                    }\n                }\n            }\n        }\n\n        is_all || matches_one\n    }\n\n    pub fn serialize_ical(\n        &mut self,\n        event: &ArchivedCalendarEvent,\n        data: &CalendarData,\n        instances_limit: &mut usize,\n    ) -> Option<String> {\n        let mut out = String::with_capacity(event.size.to_native() as usize);\n        let _v = [0.into()];\n        let mut component_iter: Iter<'_, rkyv::rend::u32_le> = _v.iter();\n        let mut component_stack: Vec<(&ArchivedICalendarComponent, Iter<'_, rkyv::rend::u32_le>)> =\n            Vec::with_capacity(4);\n\n        if data.expand.is_some() {\n            self.expanded_times\n                .sort_unstable_by(|a, b| a.start.cmp(&b.start));\n        }\n\n        loop {\n            if let Some(component_id) = component_iter.next() {\n                let component_id = component_id.to_native();\n                let component = event\n                    .data\n                    .event\n                    .components\n                    .get(component_id as usize)\n                    .unwrap();\n\n                // Limit recurrence override\n                if let Some(limit_recurrence) = &data.limit_recurrence\n                    && component.is_recurrence_override()\n                    && !self.expanded_times.iter().any(|event| {\n                        event.comp_id == component_id\n                            && limit_recurrence.is_in_range(\n                                component.component_type == ICalendarComponentType::VTodo,\n                                event.start,\n                                event.end,\n                            )\n                    })\n                {\n                    continue;\n                }\n\n                // Limit freebusy\n                if let Some(limit_recurrence) = &data.limit_freebusy\n                    && component.component_type == ICalendarComponentType::VFreebusy\n                    && !self.expanded_times.iter().any(|event| {\n                        event.comp_id == component_id\n                            && limit_recurrence.is_in_range(false, event.start, event.end)\n                    })\n                {\n                    continue;\n                }\n\n                // Filter entries\n                let mut entries = component\n                    .entries\n                    .iter()\n                    .filter_map(|entry| {\n                        if data.properties.is_empty()\n                            || component.component_type == ICalendarComponentType::VCalendar\n                        {\n                            Some((entry, true))\n                        } else {\n                            data.properties\n                                .iter()\n                                .find(|prop| {\n                                    prop.component.as_ref().is_none_or(|comp| {\n                                        comp == &component.component_type\n                                            || component_stack.iter().any(|(parent_comp, _)| {\n                                                comp == &parent_comp.component_type\n                                            })\n                                    }) && prop.name.as_ref().is_none_or(|name| name == &entry.name)\n                                })\n                                .map(|prop| (entry, !prop.no_value))\n                        }\n                    })\n                    .peekable();\n\n                // Expand recurrences\n                let component_name = component.component_type.as_str();\n                if let Some(expand) = &data\n                    .expand\n                    .filter(|_| component.component_type.has_time_ranges())\n                {\n                    let is_recurrent = component.is_recurrent();\n                    let is_recurrent_or_override =\n                        is_recurrent || component.is_recurrence_override();\n                    let is_todo = component.component_type == ICalendarComponentType::VTodo;\n                    let mut has_duration = false;\n                    let entries = entries\n                        .filter(|(entry, _)| match &entry.name {\n                            ArchivedICalendarProperty::Dtstart\n                            | ArchivedICalendarProperty::Dtend\n                            | ArchivedICalendarProperty::Exdate\n                            | ArchivedICalendarProperty::Exrule\n                            | ArchivedICalendarProperty::Rdate\n                            | ArchivedICalendarProperty::Rrule\n                            | ArchivedICalendarProperty::RecurrenceId => false,\n                            ArchivedICalendarProperty::Due\n                            | ArchivedICalendarProperty::Completed\n                            | ArchivedICalendarProperty::Created => is_recurrent,\n                            ArchivedICalendarProperty::Duration => {\n                                has_duration = true;\n                                true\n                            }\n                            _ => true,\n                        })\n                        .collect::<Vec<_>>();\n                    for event in &self.expanded_times {\n                        if event.comp_id == component_id\n                            && (!is_recurrent_or_override\n                                || expand.is_in_range(is_todo, event.start, event.end))\n                        {\n                            if *instances_limit > 0 {\n                                *instances_limit -= 1;\n                            } else {\n                                return None;\n                            }\n                            let _ = write!(&mut out, \"BEGIN:{component_name}\\r\\n\");\n\n                            // Write DTSTART, DTEND and RECURRENCE-ID\n                            let mut entry = ICalendarEntry {\n                                name: ICalendarProperty::Dtstart,\n                                params: vec![],\n                                values: vec![ICalendarValue::PartialDateTime(Box::new(\n                                    PartialDateTime::from_utc_timestamp(event.start),\n                                ))],\n                            };\n                            let _ = entry.write_to(&mut out);\n                            if is_recurrent_or_override {\n                                entry.name = ICalendarProperty::RecurrenceId;\n                                let _ = entry.write_to(&mut out);\n                            }\n                            if !has_duration {\n                                entry.name = ICalendarProperty::Dtend;\n                                entry.values = vec![ICalendarValue::PartialDateTime(Box::new(\n                                    PartialDateTime::from_utc_timestamp(event.end),\n                                ))];\n                                let _ = entry.write_to(&mut out);\n                            }\n\n                            // Write other component entries\n                            for (entry, with_value) in &entries {\n                                let _ = entry.write_to(&mut out, *with_value);\n                            }\n                            let _ = write!(&mut out, \"END:{component_name}\\r\\n\");\n                        }\n                    }\n                } else if entries.peek().is_some() {\n                    let _ = write!(&mut out, \"BEGIN:{component_name}\\r\\n\");\n\n                    if data.limit_freebusy.is_none()\n                        || component.component_type != ICalendarComponentType::VFreebusy\n                    {\n                        for (entry, with_value) in entries {\n                            let _ = entry.write_to(&mut out, with_value);\n                        }\n                    } else {\n                        // Filter freebusy\n                        let range = data.limit_freebusy.unwrap();\n                        for (entry, with_value) in entries {\n                            if matches!(entry.name, ArchivedICalendarProperty::Freebusy) {\n                                let mut fb_in_range =\n                                    freebusy_in_range(entry, &range, self.default_tz).peekable();\n                                if fb_in_range.peek().is_none() {\n                                    continue;\n                                } else {\n                                    let _ = ICalendarEntry {\n                                        name: ICalendarProperty::Freebusy,\n                                        params: rkyv_deserialize(&entry.params)\n                                            .ok()\n                                            .unwrap_or_default(),\n                                        values: fb_in_range.collect(),\n                                    }\n                                    .write_to(&mut out);\n                                }\n                            } else {\n                                let _ = entry.write_to(&mut out, with_value);\n                            }\n                        }\n                    }\n\n                    if !component.component_ids.is_empty() {\n                        component_stack.push((component, component_iter));\n                        component_iter = component.component_ids.iter();\n                    } else if component.component_ids.is_empty() {\n                        let _ = write!(&mut out, \"END:{component_name}\\r\\n\");\n                    }\n                }\n            } else if let Some((component, iter)) = component_stack.pop() {\n                let _ = write!(&mut out, \"END:{}\\r\\n\", component.component_type.as_str());\n                component_iter = iter;\n            } else {\n                break;\n            }\n        }\n\n        Some(out)\n    }\n\n    pub fn into_expanded_times(self) -> Vec<CalendarEventExpansion> {\n        self.expanded_times\n    }\n}\n\n#[inline(always)]\nfn find_components<'x>(\n    ical: &'x ArchivedICalendar,\n    comp: &[ICalendarComponentType],\n) -> impl Iterator<Item = (usize, &'x ArchivedICalendarComponent)> {\n    // TODO: Properly expand the component type path\n    let comp = comp.last().unwrap_or(&ICalendarComponentType::VCalendar);\n    ical.components\n        .iter()\n        .enumerate()\n        .filter(move |(_, entry)| {\n            comp == &ICalendarComponentType::VCalendar || &entry.component_type == comp\n        })\n}\n\n#[inline(always)]\nfn find_properties<'x>(\n    comp: &'x ArchivedICalendarComponent,\n    prop: &ICalendarProperty,\n) -> impl Iterator<Item = &'x ArchivedICalendarEntry> {\n    comp.entries.iter().filter(move |entry| &entry.name == prop)\n}\n\n#[inline(always)]\nfn find_parameter<'x>(\n    entry: &'x ArchivedICalendarEntry,\n    name: &ICalendarParameterName,\n) -> Option<&'x ArchivedICalendarParameter> {\n    entry.params.iter().find(|param| param.name == *name)\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/scheduling.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavErrorCondition, DavMethod,\n    calendar::freebusy::CalendarFreebusyRequestHandler,\n    common::{\n        ETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse calcard::{\n    Entry, Parser,\n    icalendar::{\n        ICalendarComponentType, ICalendarEntry, ICalendarMethod, ICalendarProperty, ICalendarValue,\n        Uri,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders,\n    schema::{\n        property::Rfc1123DateTime,\n        request::FreeBusyQuery,\n        response::{CalCondition, Href, ScheduleResponse, ScheduleResponseItem},\n    },\n};\nuse groupware::{DestroyArchive, cache::GroupwareCache, calendar::CalendarEventNotification};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse store::{ahash::AHashMap, write::BatchBuilder};\nuse trc::AddContext;\nuse types::collection::{Collection, SyncCollection};\nuse utils::sanitize_email;\n\npub(crate) trait CalendarEventNotificationHandler: Sync + Send {\n    fn handle_scheduling_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn handle_scheduling_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn handle_scheduling_post_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CalendarEventNotificationHandler for Server {\n    async fn handle_scheduling_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(\n                access_token,\n                account_id,\n                SyncCollection::CalendarEventNotification,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resources\n            .by_path(\n                resource_\n                    .resource\n                    .ok_or(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))?,\n            )\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        if resource.is_container() {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Validate ACL\n        if !access_token.is_member(account_id) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Fetch event\n        let event_ = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::CalendarEventNotification,\n                resource.document_id(),\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let event = event_\n            .unarchive::<CalendarEventNotification>()\n            .caused_by(trc::location!())?;\n\n        // Validate headers\n        let etag = event_.etag();\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: Collection::CalendarEventNotification,\n                document_id: resource.document_id().into(),\n                etag: etag.clone().into(),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::GET,\n        )\n        .await?;\n\n        let response = HttpResponse::new(StatusCode::OK)\n            .with_content_type(\"text/calendar; charset=utf-8\")\n            .with_etag(etag)\n            .with_last_modified(Rfc1123DateTime::new(i64::from(event.modified)).to_string());\n\n        let ical = event.event.to_string();\n\n        if !is_head {\n            Ok(response.with_binary_body(ical))\n        } else {\n            Ok(response.with_content_length(ical.len()))\n        }\n    }\n\n    async fn handle_scheduling_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let delete_path = resource\n            .resource\n            .filter(|r| !r.is_empty())\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        let resources = self\n            .fetch_dav_resources(\n                access_token,\n                account_id,\n                SyncCollection::CalendarEventNotification,\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        // Check resource type\n        let resource = resources\n            .by_path(delete_path)\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        if resource.is_container() {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Validate ACL\n        if !access_token.is_member(account_id) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        let document_id = resource.document_id();\n        let event_ = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::CalendarEventNotification,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: Collection::CalendarEventNotification,\n                document_id: document_id.into(),\n                etag: event_.etag().into(),\n                path: delete_path,\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::DELETE,\n        )\n        .await?;\n\n        let event = event_\n            .to_unarchived::<CalendarEventNotification>()\n            .caused_by(trc::location!())?;\n\n        // Delete event\n        let mut batch = BatchBuilder::new();\n        DestroyArchive(event)\n            .delete(access_token, account_id, document_id, &mut batch)\n            .caused_by(trc::location!())?;\n\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    }\n\n    async fn handle_scheduling_post_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        if resource.resource.is_none_or(|r| r != \"outbox\") {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Parse iTIP message\n        if bytes.len() > self.core.groupware.max_ical_size {\n            return Err(DavError::Condition(DavErrorCondition::new(\n                StatusCode::PRECONDITION_FAILED,\n                CalCondition::MaxResourceSize(self.core.groupware.max_ical_size as u32),\n            )));\n        }\n        let itip_raw = std::str::from_utf8(&bytes).map_err(|_| {\n            DavError::Condition(\n                DavErrorCondition::new(\n                    StatusCode::BAD_REQUEST,\n                    CalCondition::ValidSchedulingMessage,\n                )\n                .with_details(\"Invalid UTF-8 in iCalendar data\"),\n            )\n        })?;\n        let itip = match Parser::new(itip_raw).entry() {\n            Entry::ICalendar(ical) if ical.components.len() > 1 => ical,\n            _ => {\n                return Err(DavError::Condition(\n                    DavErrorCondition::new(\n                        StatusCode::BAD_REQUEST,\n                        CalCondition::ValidSchedulingMessage,\n                    )\n                    .with_details(\"Failed to parse iCalendar data\"),\n                ));\n            }\n        };\n\n        // Parse request\n        let mut from_date = None;\n        let mut to_date = None;\n        let mut organizer = None;\n        let mut attendees = AHashMap::new();\n        let mut uid = None;\n        let tz_resolver = itip.build_tz_resolver();\n        let mut found_freebusy = false;\n\n        for component in &itip.components {\n            if component.component_type != ICalendarComponentType::VFreebusy {\n                continue;\n            } else if !found_freebusy {\n                found_freebusy = true;\n            } else {\n                return Err(DavError::Condition(\n                    DavErrorCondition::new(\n                        StatusCode::BAD_REQUEST,\n                        CalCondition::ValidSchedulingMessage,\n                    )\n                    .with_details(\"Multiple VFREEBUSY components found\"),\n                ));\n            }\n\n            for entry in &component.entries {\n                let tz_id = entry.tz_id();\n                match (&entry.name, entry.values.first()) {\n                    (ICalendarProperty::Dtstart, Some(ICalendarValue::PartialDateTime(dt))) => {\n                        from_date = dt.to_date_time_with_tz(tz_resolver.resolve_or_default(tz_id));\n                    }\n                    (ICalendarProperty::Dtend, Some(ICalendarValue::PartialDateTime(dt))) => {\n                        to_date = dt.to_date_time_with_tz(tz_resolver.resolve_or_default(tz_id));\n                    }\n                    (ICalendarProperty::Uid, Some(ICalendarValue::Text(_))) => {\n                        uid = Some(entry);\n                    }\n                    (\n                        ICalendarProperty::Organizer,\n                        Some(ICalendarValue::Text(_) | ICalendarValue::Uri(Uri::Location(_))),\n                    ) => {\n                        organizer = Some(entry);\n                    }\n                    (\n                        ICalendarProperty::Attendee,\n                        Some(\n                            ICalendarValue::Text(value) | ICalendarValue::Uri(Uri::Location(value)),\n                        ),\n                    ) => {\n                        if let Some(email) =\n                            sanitize_email(value.strip_prefix(\"mailto:\").unwrap_or(value.as_str()))\n                        {\n                            attendees.insert(email, entry);\n                        }\n                    }\n                    _ => {}\n                }\n            }\n        }\n\n        let (Some(from_date), Some(to_date)) = (from_date, to_date) else {\n            return Err(DavError::Condition(\n                DavErrorCondition::new(\n                    StatusCode::BAD_REQUEST,\n                    CalCondition::ValidSchedulingMessage,\n                )\n                .with_details(\"Missing DTSTART or DTEND in VFREEBUSY component\"),\n            ));\n        };\n        let Some(organizer) = organizer else {\n            return Err(DavError::Condition(\n                DavErrorCondition::new(\n                    StatusCode::BAD_REQUEST,\n                    CalCondition::ValidSchedulingMessage,\n                )\n                .with_details(\"Missing ORGANIZER in VFREEBUSY component\"),\n            ));\n        };\n        if attendees.is_empty() {\n            return Err(DavError::Condition(\n                DavErrorCondition::new(\n                    StatusCode::BAD_REQUEST,\n                    CalCondition::ValidSchedulingMessage,\n                )\n                .with_details(\"Missing ATTENDEE in VFREEBUSY component\"),\n            ));\n        }\n\n        let mut response = ScheduleResponse::default();\n\n        for (email, attendee) in attendees {\n            if let Some(account_id) = self\n                .directory()\n                .email_to_id(&email)\n                .await\n                .caused_by(trc::location!())?\n            {\n                let resources = self\n                    .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n                    .await\n                    .caused_by(trc::location!())?;\n                if let Some(resource) = self\n                    .core\n                    .groupware\n                    .default_calendar_name\n                    .as_ref()\n                    .and_then(|name| resources.by_path(name))\n                {\n                    let mut free_busy = self\n                        .build_freebusy_object(\n                            access_token,\n                            FreeBusyQuery::new(from_date.timestamp(), to_date.timestamp()),\n                            &resources,\n                            account_id,\n                            resource,\n                        )\n                        .await?;\n\n                    // Add iTIP method\n                    free_busy.components[0].entries.push(ICalendarEntry {\n                        name: ICalendarProperty::Method,\n                        params: vec![],\n                        values: vec![ICalendarValue::Method(ICalendarMethod::Reply)],\n                    });\n\n                    // Add properties\n                    let component = &mut free_busy.components[1];\n                    component.entries.push(organizer.clone());\n                    component.entries.push(attendee.clone());\n                    if let Some(uid) = uid {\n                        component.entries.push(uid.clone());\n                    }\n\n                    response.items.0.push(ScheduleResponseItem {\n                        recipient: Href(format!(\"mailto:{email}\")),\n                        request_status: \"2.0;Success\".into(),\n                        calendar_data: Some(free_busy.to_string()),\n                    });\n                } else {\n                    response.items.0.push(ScheduleResponseItem {\n                        recipient: Href(format!(\"mailto:{email}\")),\n                        request_status: \"3.7;Default calendar not found\".into(),\n                        calendar_data: None,\n                    });\n                }\n            } else {\n                response.items.0.push(ScheduleResponseItem {\n                    recipient: Href(format!(\"mailto:{email}\")),\n                    request_status: \"3.7;Invalid calendar user or insufficient permissions\".into(),\n                    calendar_data: None,\n                });\n            }\n        }\n\n        Ok(HttpResponse::new(StatusCode::OK).with_xml_body(response.to_string()))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/calendar/update.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::assert_is_unique_uid;\nuse crate::{\n    DavError, DavErrorCondition, DavMethod,\n    calendar::ItipPrecondition,\n    common::{\n        ETag, ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n    fix_percent_encoding,\n};\nuse calcard::{\n    Entry, Parser,\n    common::timezone::Tz,\n    icalendar::{ICalendar, ICalendarComponentType},\n};\nuse common::{DavName, Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{property::Rfc1123DateTime, response::CalCondition},\n};\nuse directory::Permission;\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{CalendarEvent, CalendarEventData},\n    scheduling::{ItipMessages, event_create::itip_create, event_update::itip_update},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::collections::HashSet;\nuse store::write::{BatchBuilder, now};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait CalendarUpdateRequestHandler: Sync + Send {\n    fn handle_calendar_update_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n        is_patch: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CalendarUpdateRequestHandler for Server {\n    async fn handle_calendar_update_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n        _is_patch: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let resource_name = fix_percent_encoding(\n            resource\n                .resource\n                .ok_or(DavError::Code(StatusCode::CONFLICT))?,\n        );\n\n        if bytes.len() > self.core.groupware.max_ical_size {\n            return Err(DavError::Condition(DavErrorCondition::new(\n                StatusCode::PRECONDITION_FAILED,\n                CalCondition::MaxResourceSize(self.core.groupware.max_ical_size as u32),\n            )));\n        }\n        let ical_raw = std::str::from_utf8(&bytes).map_err(|_| {\n            DavError::Condition(\n                DavErrorCondition::new(\n                    StatusCode::PRECONDITION_FAILED,\n                    CalCondition::SupportedCalendarData,\n                )\n                .with_details(\"Invalid UTF-8 in iCalendar data\"),\n            )\n        })?;\n\n        let ical = match Parser::new(ical_raw).entry() {\n            Entry::ICalendar(ical) => ical,\n            _ => {\n                return Err(DavError::Condition(\n                    DavErrorCondition::new(\n                        StatusCode::PRECONDITION_FAILED,\n                        CalCondition::SupportedCalendarData,\n                    )\n                    .with_details(\"Failed to parse iCalendar data\"),\n                ));\n            }\n        };\n\n        if let Some(resource) = resources.by_path(resource_name.as_ref()) {\n            if resource.is_container() {\n                return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n            }\n\n            // Validate ACL\n            let parent_id = resource.parent_id().unwrap();\n            let document_id = resource.document_id();\n            if !access_token.is_member(account_id)\n                && !resources.has_access_to_container(access_token, parent_id, Acl::ModifyItems)\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Update\n            let event_ = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n            let event = event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            // Validate headers\n            match self\n                .validate_headers(\n                    access_token,\n                    headers,\n                    vec![ResourceState {\n                        account_id,\n                        collection: Collection::CalendarEvent,\n                        document_id: Some(document_id),\n                        etag: event.etag().into(),\n                        path: resource_name.as_ref(),\n                        ..Default::default()\n                    }],\n                    Default::default(),\n                    DavMethod::PUT,\n                )\n                .await\n            {\n                Ok(_) => {}\n                Err(DavError::Code(StatusCode::PRECONDITION_FAILED))\n                    if headers.ret == Return::Representation =>\n                {\n                    return Ok(HttpResponse::new(StatusCode::PRECONDITION_FAILED)\n                        .with_content_type(\"text/calendar; charset=utf-8\")\n                        .with_etag(event.etag())\n                        .with_last_modified(\n                            Rfc1123DateTime::new(i64::from(event.inner.modified)).to_string(),\n                        )\n                        .with_header(\"Preference-Applied\", \"return=representation\")\n                        .with_binary_body(event.inner.data.event.to_string()));\n                }\n                Err(e) => return Err(e),\n            }\n\n            if ical == event.inner.data.event {\n                // No changes, return existing event\n                return Ok(HttpResponse::new(StatusCode::NO_CONTENT));\n            }\n\n            // Validate iCal\n            if event.inner.data.event.uids().next().unwrap_or_default() != validate_ical(&ical)? {\n                return Err(DavError::Condition(DavErrorCondition::new(\n                    StatusCode::PRECONDITION_FAILED,\n                    CalCondition::NoUidConflict(resources.format_resource(resource).into()),\n                )));\n            }\n\n            // Validate schedule tag\n            if headers.if_schedule_tag.is_some()\n                && event.inner.schedule_tag.as_ref().map(|t| t.to_native())\n                    != headers.if_schedule_tag\n            {\n                return Err(DavError::Code(StatusCode::PRECONDITION_FAILED));\n            }\n\n            // Obtain previous alarm\n            let now = now() as i64;\n            let prev_email_alarm = event.inner.data.next_alarm(now, Tz::Floating);\n\n            // Build event\n            let mut next_email_alarm = None;\n            let mut new_event = event\n                .deserialize::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n            let old_ical = new_event.data.event;\n            new_event.size = bytes.len() as u32;\n            new_event.data = CalendarEventData::new(\n                ical,\n                Tz::Floating,\n                self.core.groupware.max_ical_instances,\n                &mut next_email_alarm,\n            );\n\n            // Scheduling\n            let mut itip_messages = None;\n            if self.core.groupware.itip_enabled\n                && !access_token.emails.is_empty()\n                && access_token.has_permission(Permission::CalendarSchedulingSend)\n                && new_event.data.event_range_end() > now\n            {\n                let result = if new_event.schedule_tag.is_some() {\n                    itip_update(\n                        &mut new_event.data.event,\n                        &old_ical,\n                        access_token.emails.as_slice(),\n                    )\n                } else {\n                    itip_create(&mut new_event.data.event, access_token.emails.as_slice())\n                };\n\n                match result {\n                    Ok(messages) => {\n                        let mut is_organizer = false;\n                        if messages\n                            .iter()\n                            .map(|r| {\n                                is_organizer = r.from_organizer;\n                                r.to.len()\n                            })\n                            .sum::<usize>()\n                            < self.core.groupware.itip_outbound_max_recipients\n                        {\n                            // Only update schedule tag if the user is the organizer\n                            if is_organizer {\n                                if let Some(schedule_tag) = &mut new_event.schedule_tag {\n                                    *schedule_tag += 1;\n                                } else {\n                                    new_event.schedule_tag = Some(1);\n                                }\n                            }\n\n                            itip_messages = Some(ItipMessages::new(messages));\n                        } else {\n                            return Err(DavError::Condition(DavErrorCondition::new(\n                                StatusCode::PRECONDITION_FAILED,\n                                CalCondition::MaxAttendeesPerInstance,\n                            )));\n                        }\n                    }\n                    Err(err) => {\n                        if let Some(failed_precondition) = err.failed_precondition() {\n                            return Err(DavError::Condition(\n                                DavErrorCondition::new(\n                                    StatusCode::PRECONDITION_FAILED,\n                                    failed_precondition,\n                                )\n                                .with_details(err.to_string()),\n                            ));\n                        }\n\n                        // Event changed, but there are no iTIP messages to send\n                        if let Some(schedule_tag) = &mut new_event.schedule_tag {\n                            *schedule_tag += 1;\n                        }\n                    }\n                }\n            }\n            // Validate quota\n            let extra_bytes =\n                (bytes.len() as u64).saturating_sub(u32::from(event.inner.size) as u64);\n            if extra_bytes > 0 {\n                self.has_available_quota(\n                    &self.get_resource_token(access_token, account_id).await?,\n                    extra_bytes,\n                )\n                .await?;\n            }\n\n            // Prepare write batch\n            let mut batch = BatchBuilder::new();\n            let schedule_tag = new_event.schedule_tag;\n            let etag = new_event\n                .update(access_token, event, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?\n                .etag();\n            if prev_email_alarm != next_email_alarm {\n                if let Some(prev_alarm) = prev_email_alarm {\n                    prev_alarm.delete_task(&mut batch);\n                }\n                if let Some(next_alarm) = next_email_alarm {\n                    next_alarm.write_task(&mut batch);\n                }\n            }\n            if let Some(itip_messages) = itip_messages {\n                itip_messages\n                    .queue(&mut batch)\n                    .caused_by(trc::location!())?;\n            }\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n            self.notify_task_queue();\n\n            Ok(HttpResponse::new(StatusCode::NO_CONTENT)\n                .with_etag_opt(etag)\n                .with_schedule_tag_opt(schedule_tag))\n        } else if let Some((Some(parent), name)) = resources.map_parent(resource_name.as_ref()) {\n            if !parent.is_container() {\n                return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n            }\n\n            // Validate ACL\n            if !access_token.is_member(account_id)\n                && !resources.has_access_to_container(\n                    access_token,\n                    parent.document_id(),\n                    Acl::AddItems,\n                )\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Validate headers\n            self.validate_headers(\n                access_token,\n                headers,\n                vec![ResourceState {\n                    account_id,\n                    collection: resource.collection,\n                    document_id: Some(u32::MAX),\n                    path: resource_name.as_ref(),\n                    ..Default::default()\n                }],\n                Default::default(),\n                DavMethod::PUT,\n            )\n            .await?;\n\n            // Validate ical object\n            assert_is_unique_uid(\n                self,\n                &resources,\n                account_id,\n                parent.document_id(),\n                validate_ical(&ical)?.into(),\n            )\n            .await?;\n\n            // Build event\n            let mut next_email_alarm = None;\n            let mut event = CalendarEvent {\n                names: vec![DavName {\n                    name: name.to_string(),\n                    parent_id: parent.document_id(),\n                }],\n                data: CalendarEventData::new(\n                    ical,\n                    Tz::Floating,\n                    self.core.groupware.max_ical_instances,\n                    &mut next_email_alarm,\n                ),\n                size: bytes.len() as u32,\n                ..Default::default()\n            };\n\n            // Scheduling\n            let mut itip_messages = None;\n            if self.core.groupware.itip_enabled\n                && !access_token.emails.is_empty()\n                && access_token.has_permission(Permission::CalendarSchedulingSend)\n                && event.data.event_range_end() > now() as i64\n            {\n                match itip_create(&mut event.data.event, access_token.emails.as_slice()) {\n                    Ok(messages) => {\n                        if messages.iter().map(|r| r.to.len()).sum::<usize>()\n                            < self.core.groupware.itip_outbound_max_recipients\n                        {\n                            event.schedule_tag = Some(1);\n                            itip_messages = Some(ItipMessages::new(messages));\n                        } else {\n                            return Err(DavError::Condition(DavErrorCondition::new(\n                                StatusCode::PRECONDITION_FAILED,\n                                CalCondition::MaxAttendeesPerInstance,\n                            )));\n                        }\n                    }\n                    Err(err) => {\n                        if let Some(failed_precondition) = err.failed_precondition() {\n                            return Err(DavError::Condition(\n                                DavErrorCondition::new(\n                                    StatusCode::PRECONDITION_FAILED,\n                                    failed_precondition,\n                                )\n                                .with_details(err.to_string()),\n                            ));\n                        }\n                    }\n                }\n            }\n\n            // Validate quota\n            if !bytes.is_empty() {\n                self.has_available_quota(\n                    &self.get_resource_token(access_token, account_id).await?,\n                    bytes.len() as u64,\n                )\n                .await?;\n            }\n\n            // Prepare write batch\n            let mut batch = BatchBuilder::new();\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::CalendarEvent, 1)\n                .await\n                .caused_by(trc::location!())?;\n            let schedule_tag = event.schedule_tag;\n            let etag = event\n                .insert(\n                    access_token,\n                    account_id,\n                    document_id,\n                    next_email_alarm,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?\n                .etag();\n            if let Some(itip_messages) = itip_messages {\n                itip_messages\n                    .queue(&mut batch)\n                    .caused_by(trc::location!())?;\n            }\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n            self.notify_task_queue();\n\n            Ok(HttpResponse::new(StatusCode::CREATED)\n                .with_etag_opt(etag)\n                .with_schedule_tag_opt(schedule_tag))\n        } else {\n            Err(DavError::Code(StatusCode::CONFLICT))?\n        }\n    }\n}\n\nfn validate_ical(ical: &ICalendar) -> crate::Result<&str> {\n    // Validate UIDs\n    let mut uids = HashSet::with_capacity(1);\n\n    // Validate component types\n    let mut types: [u8; 5] = [0; 5];\n    for comp in &ical.components {\n        *(match comp.component_type {\n            ICalendarComponentType::VEvent => &mut types[0],\n            ICalendarComponentType::VTodo => &mut types[1],\n            ICalendarComponentType::VJournal => &mut types[2],\n            ICalendarComponentType::VFreebusy => &mut types[3],\n            ICalendarComponentType::VAvailability => &mut types[4],\n            _ => {\n                continue;\n            }\n        }) += 1;\n\n        if let Some(uid) = comp.uid() {\n            uids.insert(uid);\n        }\n    }\n\n    if uids.len() == 1 && types.iter().filter(|&&v| v == 0).count() == 4 {\n        Ok(uids.iter().next().unwrap())\n    } else {\n        Err(DavError::Condition(\n            DavErrorCondition::new(\n                StatusCode::PRECONDITION_FAILED,\n                CalCondition::ValidCalendarObjectResource,\n            )\n            .with_details(\"iCalendar must contain exactly one UID and same component types\"),\n        ))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/card/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::assert_is_unique_uid;\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n};\nuse common::{DavName, Server, auth::AccessToken};\nuse dav_proto::{Depth, RequestHeaders};\nuse groupware::{\n    DestroyArchive,\n    cache::GroupwareCache,\n    contact::{AddressBook, AddressBookPreferences, ContactCard},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::BatchBuilder;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection, VanishedCollection},\n};\n\npub(crate) trait CardCopyMoveRequestHandler: Sync + Send {\n    fn handle_card_copy_move_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_move: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CardCopyMoveRequestHandler for Server {\n    async fn handle_card_copy_move_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_move: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate source\n        let from_resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let from_account_id = from_resource_.account_id;\n        let from_resources = self\n            .fetch_dav_resources(access_token, from_account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n        let from_resource_name = from_resource_\n            .resource\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        let from_resource = from_resources\n            .by_path(from_resource_name)\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n        // Validate ACL\n        if !access_token.is_member(from_account_id)\n            && !from_resources.has_access_to_container(\n                access_token,\n                if from_resource.is_container() {\n                    from_resource.document_id()\n                } else {\n                    from_resource.parent_id().unwrap()\n                },\n                Acl::ReadItems,\n            )\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Validate destination\n        let destination = self\n            .validate_uri_with_status(\n                access_token,\n                headers\n                    .destination\n                    .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?,\n                StatusCode::BAD_GATEWAY,\n            )\n            .await?;\n        if destination.collection != Collection::AddressBook {\n            return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n        }\n        let to_account_id = destination\n            .account_id\n            .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?;\n        let to_resources = if to_account_id == from_account_id {\n            from_resources.clone()\n        } else {\n            self.fetch_dav_resources(access_token, to_account_id, SyncCollection::AddressBook)\n                .await\n                .caused_by(trc::location!())?\n        };\n\n        // Validate headers\n        let destination_resource_name = destination\n            .resource\n            .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?;\n        let to_resource = to_resources.by_path(destination_resource_name);\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![\n                ResourceState {\n                    account_id: from_account_id,\n                    collection: if from_resource.is_container() {\n                        Collection::AddressBook\n                    } else {\n                        Collection::ContactCard\n                    },\n                    document_id: Some(from_resource.document_id()),\n                    path: from_resource_name,\n                    ..Default::default()\n                },\n                ResourceState {\n                    account_id: to_account_id,\n                    collection: to_resource\n                        .map(|r| {\n                            if r.is_container() {\n                                Collection::AddressBook\n                            } else {\n                                Collection::ContactCard\n                            }\n                        })\n                        .unwrap_or(Collection::AddressBook),\n                    document_id: Some(to_resource.map(|r| r.document_id()).unwrap_or(u32::MAX)),\n                    path: destination_resource_name,\n                    ..Default::default()\n                },\n            ],\n            Default::default(),\n            if is_move {\n                DavMethod::MOVE\n            } else {\n                DavMethod::COPY\n            },\n        )\n        .await?;\n\n        // Map destination\n        if let Some(to_resource) = to_resource {\n            if from_resource.path() == to_resource.path() {\n                // Same resource\n                return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n            }\n            let new_name = destination_resource_name\n                .rsplit_once('/')\n                .map(|(_, name)| name)\n                .unwrap_or(destination_resource_name);\n\n            match (from_resource.is_container(), to_resource.is_container()) {\n                (true, true) => {\n                    let from_children_ids = from_resources\n                        .subtree(from_resource_name)\n                        .filter(|r| !r.is_container())\n                        .map(|r| r.document_id())\n                        .collect::<Vec<_>>();\n                    let to_document_ids = to_resources\n                        .subtree(destination_resource_name)\n                        .filter(|r| !r.is_container())\n                        .map(|r| r.document_id())\n                        .collect::<Vec<_>>();\n\n                    // Validate ACLs\n                    if !access_token.is_member(to_account_id)\n                        || (!access_token.is_member(from_account_id)\n                            && !from_resources.has_access_to_container(\n                                access_token,\n                                from_resource.document_id(),\n                                if is_move {\n                                    Acl::RemoveItems\n                                } else {\n                                    Acl::ReadItems\n                                },\n                            ))\n                    {\n                        return Err(DavError::Code(StatusCode::FORBIDDEN));\n                    }\n\n                    // Overwrite container\n                    copy_container(\n                        self,\n                        access_token,\n                        from_account_id,\n                        from_resource.document_id(),\n                        from_children_ids,\n                        from_resources.format_collection(from_resource_name),\n                        to_account_id,\n                        to_resource.document_id().into(),\n                        to_document_ids,\n                        new_name,\n                        is_move,\n                    )\n                    .await\n                }\n                (false, false) => {\n                    // Overwrite card\n                    let from_addressbook_id = from_resource.parent_id().unwrap();\n                    let to_addressbook_id = to_resource.parent_id().unwrap();\n\n                    // Validate ACL\n                    if (!access_token.is_member(from_account_id)\n                        && !from_resources.has_access_to_container(\n                            access_token,\n                            from_addressbook_id,\n                            if is_move {\n                                Acl::RemoveItems\n                            } else {\n                                Acl::ReadItems\n                            },\n                        ))\n                        || (!access_token.is_member(to_account_id)\n                            && !to_resources.has_access_to_container(\n                                access_token,\n                                to_addressbook_id,\n                                Acl::RemoveItems,\n                            ))\n                    {\n                        return Err(DavError::Code(StatusCode::FORBIDDEN));\n                    }\n\n                    if is_move {\n                        move_card(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            from_addressbook_id,\n                            from_resources.format_item(from_resource_name),\n                            to_account_id,\n                            to_resource.document_id().into(),\n                            to_addressbook_id,\n                            new_name,\n                        )\n                        .await\n                    } else {\n                        copy_card(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            to_account_id,\n                            to_resource.document_id().into(),\n                            to_addressbook_id,\n                            new_name,\n                        )\n                        .await\n                    }\n                }\n                _ => Err(DavError::Code(StatusCode::BAD_GATEWAY)),\n            }\n        } else if let Some((parent_resource, new_name)) =\n            to_resources.map_parent(destination_resource_name)\n        {\n            if let Some(parent_resource) = parent_resource {\n                // Creating items under a card is not allowed\n                // Copying/moving containers under a container is not allowed\n                if !parent_resource.is_container() || from_resource.is_container() {\n                    return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n                }\n\n                // Validate ACL\n                let from_addressbook_id = from_resource.parent_id().unwrap();\n                let to_addressbook_id = parent_resource.document_id();\n                if (!access_token.is_member(from_account_id)\n                    && !from_resources.has_access_to_container(\n                        access_token,\n                        from_addressbook_id,\n                        if is_move {\n                            Acl::RemoveItems\n                        } else {\n                            Acl::ReadItems\n                        },\n                    ))\n                    || (!access_token.is_member(to_account_id)\n                        && !to_resources.has_access_to_container(\n                            access_token,\n                            to_addressbook_id,\n                            Acl::AddItems,\n                        ))\n                {\n                    return Err(DavError::Code(StatusCode::FORBIDDEN));\n                }\n\n                // Copy/move card\n                if is_move {\n                    if from_account_id != to_account_id\n                        || parent_resource.document_id() != from_addressbook_id\n                    {\n                        move_card(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            from_addressbook_id,\n                            from_resources.format_item(from_resource_name),\n                            to_account_id,\n                            None,\n                            to_addressbook_id,\n                            new_name,\n                        )\n                        .await\n                    } else {\n                        rename_card(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            from_addressbook_id,\n                            new_name,\n                            from_resources.format_item(from_resource_name),\n                        )\n                        .await\n                    }\n                } else {\n                    copy_card(\n                        self,\n                        access_token,\n                        from_account_id,\n                        from_resource.document_id(),\n                        to_account_id,\n                        None,\n                        to_addressbook_id,\n                        new_name,\n                    )\n                    .await\n                }\n            } else {\n                // Copying/moving cards to the root is not allowed\n                if !from_resource.is_container() {\n                    return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n                }\n\n                // Shared users cannot create containers\n                if !access_token.is_member(to_account_id) {\n                    return Err(DavError::Code(StatusCode::FORBIDDEN));\n                }\n\n                // Validate ACLs\n                if !access_token.is_member(from_account_id)\n                    && !from_resources.has_access_to_container(\n                        access_token,\n                        from_resource.document_id(),\n                        if is_move {\n                            Acl::RemoveItems\n                        } else {\n                            Acl::ReadItems\n                        },\n                    )\n                {\n                    return Err(DavError::Code(StatusCode::FORBIDDEN));\n                }\n\n                // Copy/move container\n                let from_children_ids = from_resources\n                    .subtree(from_resource_name)\n                    .filter(|r| !r.is_container())\n                    .map(|r| r.document_id())\n                    .collect::<Vec<_>>();\n                if is_move {\n                    if from_account_id != to_account_id {\n                        copy_container(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            if headers.depth != Depth::Zero {\n                                from_children_ids\n                            } else {\n                                return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n                            },\n                            from_resources.format_collection(from_resource_name),\n                            to_account_id,\n                            None,\n                            vec![],\n                            new_name,\n                            true,\n                        )\n                        .await\n                    } else {\n                        rename_container(\n                            self,\n                            access_token,\n                            from_account_id,\n                            from_resource.document_id(),\n                            new_name,\n                            from_resources.format_collection(from_resource_name),\n                        )\n                        .await\n                    }\n                } else {\n                    copy_container(\n                        self,\n                        access_token,\n                        from_account_id,\n                        from_resource.document_id(),\n                        if headers.depth != Depth::Zero {\n                            from_children_ids\n                        } else {\n                            vec![]\n                        },\n                        from_resources.format_collection(from_resource_name),\n                        to_account_id,\n                        None,\n                        vec![],\n                        new_name,\n                        false,\n                    )\n                    .await\n                }\n            }\n        } else {\n            Err(DavError::Code(StatusCode::CONFLICT))\n        }\n    }\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn copy_card(\n    server: &Server,\n    access_token: &AccessToken,\n    from_account_id: u32,\n    from_document_id: u32,\n    to_account_id: u32,\n    to_document_id: Option<u32>,\n    to_addressbook_id: u32,\n    new_name: &str,\n) -> crate::Result<HttpResponse> {\n    // Fetch card\n    let card_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::ContactCard,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let card = card_\n        .to_unarchived::<ContactCard>()\n        .caused_by(trc::location!())?;\n    let mut batch = BatchBuilder::new();\n\n    // Validate UID\n    assert_is_unique_uid(\n        server,\n        server\n            .fetch_dav_resources(access_token, to_account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?\n            .as_ref(),\n        to_account_id,\n        to_addressbook_id,\n        card.inner.card.uid(),\n    )\n    .await?;\n\n    if from_account_id == to_account_id {\n        let mut new_card = card\n            .deserialize::<ContactCard>()\n            .caused_by(trc::location!())?;\n        new_card.names.push(DavName {\n            name: new_name.to_string(),\n            parent_id: to_addressbook_id,\n        });\n        new_card\n            .update(\n                access_token,\n                card,\n                from_account_id,\n                from_document_id,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n    } else {\n        let mut new_card = card\n            .deserialize::<ContactCard>()\n            .caused_by(trc::location!())?;\n        new_card.names = vec![DavName {\n            name: new_name.to_string(),\n            parent_id: to_addressbook_id,\n        }];\n        let to_document_id = server\n            .store()\n            .assign_document_ids(to_account_id, Collection::ContactCard, 1)\n            .await\n            .caused_by(trc::location!())?;\n        new_card\n            .insert(access_token, to_account_id, to_document_id, &mut batch)\n            .caused_by(trc::location!())?;\n    }\n\n    let response = if let Some(to_document_id) = to_document_id {\n        // Overwrite card on destination\n        let card_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                to_account_id,\n                Collection::ContactCard,\n                to_document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?;\n        if let Some(card_) = card_ {\n            let card = card_\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            DestroyArchive(card)\n                .delete(\n                    access_token,\n                    to_account_id,\n                    to_document_id,\n                    to_addressbook_id,\n                    None,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    } else {\n        Ok(HttpResponse::new(StatusCode::CREATED))\n    };\n\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    response\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn move_card(\n    server: &Server,\n    access_token: &AccessToken,\n    from_account_id: u32,\n    from_document_id: u32,\n    from_addressbook_id: u32,\n    from_resource_path: String,\n    to_account_id: u32,\n    to_document_id: Option<u32>,\n    to_addressbook_id: u32,\n    new_name: &str,\n) -> crate::Result<HttpResponse> {\n    // Fetch card\n    let card_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::ContactCard,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let card = card_\n        .to_unarchived::<ContactCard>()\n        .caused_by(trc::location!())?;\n\n    // Validate UID\n    if from_account_id != to_account_id\n        || from_addressbook_id != to_addressbook_id\n        || to_document_id.is_none()\n    {\n        assert_is_unique_uid(\n            server,\n            server\n                .fetch_dav_resources(access_token, to_account_id, SyncCollection::AddressBook)\n                .await\n                .caused_by(trc::location!())?\n                .as_ref(),\n            to_account_id,\n            to_addressbook_id,\n            card.inner.card.uid(),\n        )\n        .await?;\n    }\n\n    let mut batch = BatchBuilder::new();\n    if from_account_id == to_account_id {\n        let mut name_idx = None;\n        for (idx, name) in card.inner.names.iter().enumerate() {\n            if name.parent_id == from_addressbook_id {\n                name_idx = Some(idx);\n                break;\n            }\n        }\n\n        let name_idx = if let Some(name_idx) = name_idx {\n            name_idx\n        } else {\n            return Err(DavError::Code(StatusCode::NOT_FOUND));\n        };\n\n        let mut new_card = card\n            .deserialize::<ContactCard>()\n            .caused_by(trc::location!())?;\n        new_card.names.swap_remove(name_idx);\n        new_card.names.push(DavName {\n            name: new_name.to_string(),\n            parent_id: to_addressbook_id,\n        });\n        new_card\n            .update(\n                access_token,\n                card.clone(),\n                from_account_id,\n                from_document_id,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n        batch.log_vanished_item(VanishedCollection::AddressBook, from_resource_path);\n    } else {\n        let mut new_card = card\n            .deserialize::<ContactCard>()\n            .caused_by(trc::location!())?;\n        new_card.names = vec![DavName {\n            name: new_name.to_string(),\n            parent_id: to_addressbook_id,\n        }];\n\n        DestroyArchive(card)\n            .delete(\n                access_token,\n                from_account_id,\n                from_document_id,\n                from_addressbook_id,\n                from_resource_path.into(),\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n\n        let to_document_id = server\n            .store()\n            .assign_document_ids(to_account_id, Collection::ContactCard, 1)\n            .await\n            .caused_by(trc::location!())?;\n        new_card\n            .insert(access_token, to_account_id, to_document_id, &mut batch)\n            .caused_by(trc::location!())?;\n    }\n\n    let response = if let Some(to_document_id) = to_document_id {\n        // Overwrite card on destination\n        let card_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                to_account_id,\n                Collection::ContactCard,\n                to_document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?;\n        if let Some(card_) = card_ {\n            let card = card_\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            DestroyArchive(card)\n                .delete(\n                    access_token,\n                    to_account_id,\n                    to_document_id,\n                    to_addressbook_id,\n                    None,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    } else {\n        Ok(HttpResponse::new(StatusCode::CREATED))\n    };\n\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    response\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn rename_card(\n    server: &Server,\n    access_token: &AccessToken,\n    account_id: u32,\n    document_id: u32,\n    addressbook_id: u32,\n    new_name: &str,\n    from_resource_path: String,\n) -> crate::Result<HttpResponse> {\n    // Fetch card\n    let card_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::ContactCard,\n            document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let card = card_\n        .to_unarchived::<ContactCard>()\n        .caused_by(trc::location!())?;\n\n    let name_idx = card\n        .inner\n        .names\n        .iter()\n        .position(|n| n.parent_id == addressbook_id)\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let mut new_card = card\n        .deserialize::<ContactCard>()\n        .caused_by(trc::location!())?;\n    new_card.names[name_idx].name = new_name.to_string();\n\n    let mut batch = BatchBuilder::new();\n    new_card\n        .update(access_token, card, account_id, document_id, &mut batch)\n        .caused_by(trc::location!())?;\n    batch.log_vanished_item(VanishedCollection::AddressBook, from_resource_path);\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    Ok(HttpResponse::new(StatusCode::CREATED))\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn copy_container(\n    server: &Server,\n    access_token: &AccessToken,\n    from_account_id: u32,\n    from_document_id: u32,\n    from_children_ids: Vec<u32>,\n    from_resource_path: String,\n    to_account_id: u32,\n    to_document_id: Option<u32>,\n    to_children_ids: Vec<u32>,\n    new_name: &str,\n    remove_source: bool,\n) -> crate::Result<HttpResponse> {\n    // Fetch book\n    let book_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::AddressBook,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let old_book = book_\n        .to_unarchived::<AddressBook>()\n        .caused_by(trc::location!())?;\n    let mut book = old_book\n        .deserialize::<AddressBook>()\n        .caused_by(trc::location!())?;\n\n    // Prepare write batch\n    let mut batch = BatchBuilder::new();\n\n    if remove_source {\n        DestroyArchive(old_book)\n            .delete(\n                access_token,\n                from_account_id,\n                from_document_id,\n                from_resource_path.into(),\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n    }\n\n    let preference = book.preferences.into_iter().next().unwrap();\n    book.name = new_name.to_string();\n    book.subscribers.clear();\n    book.acls.clear();\n    book.preferences = vec![AddressBookPreferences {\n        account_id: to_account_id,\n        name: preference.name,\n        description: preference.description,\n        sort_order: 0,\n    }];\n\n    let is_overwrite = to_document_id.is_some();\n    let to_document_id = if let Some(to_document_id) = to_document_id {\n        // Overwrite destination\n        let book_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                to_account_id,\n                Collection::AddressBook,\n                to_document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?;\n        if let Some(book_) = book_ {\n            let book = book_\n                .to_unarchived::<AddressBook>()\n                .caused_by(trc::location!())?;\n\n            DestroyArchive(book)\n                .delete_with_cards(\n                    server,\n                    access_token,\n                    to_account_id,\n                    to_document_id,\n                    to_children_ids,\n                    None,\n                    &mut batch,\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        to_document_id\n    } else {\n        server\n            .store()\n            .assign_document_ids(to_account_id, Collection::AddressBook, 1)\n            .await\n            .caused_by(trc::location!())?\n    };\n    book.insert(access_token, to_account_id, to_document_id, &mut batch)\n        .caused_by(trc::location!())?;\n\n    // Copy children\n    let mut required_space = 0;\n    for from_child_document_id in from_children_ids {\n        if let Some(card_) = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                from_account_id,\n                Collection::ContactCard,\n                from_child_document_id,\n            ))\n            .await?\n        {\n            let card = card_\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n            let mut new_name = None;\n\n            for name in card.inner.names.iter() {\n                if name.parent_id == to_document_id {\n                    continue;\n                } else if name.parent_id == from_document_id {\n                    new_name = Some(name.name.to_string());\n                }\n            }\n            let new_name = if let Some(new_name) = new_name {\n                DavName {\n                    name: new_name,\n                    parent_id: to_document_id,\n                }\n            } else {\n                continue;\n            };\n            let card = card_\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n            let mut new_card = card\n                .deserialize::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            if from_account_id == to_account_id {\n                if remove_source {\n                    new_card\n                        .names\n                        .retain(|name| name.parent_id != from_document_id);\n                }\n\n                new_card.names.push(new_name);\n                new_card\n                    .update(\n                        access_token,\n                        card,\n                        from_account_id,\n                        from_child_document_id,\n                        &mut batch,\n                    )\n                    .caused_by(trc::location!())?;\n            } else {\n                if remove_source {\n                    DestroyArchive(card)\n                        .delete(\n                            access_token,\n                            from_account_id,\n                            from_child_document_id,\n                            from_document_id,\n                            None,\n                            &mut batch,\n                        )\n                        .caused_by(trc::location!())?;\n                }\n\n                let to_document_id = server\n                    .store()\n                    .assign_document_ids(to_account_id, Collection::ContactCard, 1)\n                    .await\n                    .caused_by(trc::location!())?;\n                new_card.names = vec![new_name];\n                required_space += new_card.size as u64;\n                new_card\n                    .insert(access_token, to_account_id, to_document_id, &mut batch)\n                    .caused_by(trc::location!())?;\n            }\n        }\n    }\n\n    if from_account_id != to_account_id && required_space > 0 {\n        server\n            .has_available_quota(\n                &server\n                    .get_resource_token(access_token, to_account_id)\n                    .await?,\n                required_space,\n            )\n            .await?;\n    }\n\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    if !is_overwrite {\n        Ok(HttpResponse::new(StatusCode::CREATED))\n    } else {\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    }\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn rename_container(\n    server: &Server,\n    access_token: &AccessToken,\n    account_id: u32,\n    document_id: u32,\n    new_name: &str,\n    from_resource_path: String,\n) -> crate::Result<HttpResponse> {\n    // Fetch book\n    let book_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::AddressBook,\n            document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let book = book_\n        .to_unarchived::<AddressBook>()\n        .caused_by(trc::location!())?;\n    let mut new_book = book\n        .deserialize::<AddressBook>()\n        .caused_by(trc::location!())?;\n    new_book.name = new_name.to_string();\n\n    let mut batch = BatchBuilder::new();\n    new_book\n        .update(access_token, book, account_id, document_id, &mut batch)\n        .caused_by(trc::location!())?;\n    batch.log_vanished_item(VanishedCollection::AddressBook, from_resource_path);\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n    server.notify_task_queue();\n\n    Ok(HttpResponse::new(StatusCode::CREATED))\n}\n"
  },
  {
    "path": "crates/dav/src/card/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        ETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse dav_proto::RequestHeaders;\nuse groupware::{\n    DestroyArchive,\n    cache::GroupwareCache,\n    contact::{AddressBook, ContactCard},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::{BatchBuilder, ValueClass};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n};\n\npub(crate) trait CardDeleteRequestHandler: Sync + Send {\n    fn handle_card_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CardDeleteRequestHandler for Server {\n    async fn handle_card_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let delete_path = resource\n            .resource\n            .filter(|r| !r.is_empty())\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Check resource type\n        let delete_resource = resources\n            .by_path(delete_path)\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let document_id = delete_resource.document_id();\n\n        // Fetch entry\n        let mut batch = BatchBuilder::new();\n        if delete_resource.is_container() {\n            let book_ = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::AddressBook,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n            let book = book_\n                .to_unarchived::<AddressBook>()\n                .caused_by(trc::location!())?;\n\n            // Validate ACL\n            if !access_token.is_member(account_id)\n                && !book\n                    .inner\n                    .acls\n                    .effective_acl(access_token)\n                    .contains_all([Acl::Delete, Acl::RemoveItems].into_iter())\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Validate headers\n            self.validate_headers(\n                access_token,\n                headers,\n                vec![ResourceState {\n                    account_id,\n                    collection: Collection::AddressBook,\n                    document_id: document_id.into(),\n                    etag: book.etag().into(),\n                    path: delete_path,\n                    ..Default::default()\n                }],\n                Default::default(),\n                DavMethod::DELETE,\n            )\n            .await?;\n\n            // Delete addressbook and cards\n            DestroyArchive(book)\n                .delete_with_cards(\n                    self,\n                    access_token,\n                    account_id,\n                    document_id,\n                    resources\n                        .subtree(delete_path)\n                        .filter(|r| !r.is_container())\n                        .map(|r| r.document_id())\n                        .collect::<Vec<_>>(),\n                    resources.format_resource(delete_resource).into(),\n                    &mut batch,\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n            // Reset default address book id\n            let default_book_id = self\n                .store()\n                .get_value::<u32>(ValueKey {\n                    account_id,\n                    collection: Collection::Principal.into(),\n                    document_id: 0,\n                    class: ValueClass::Property(PrincipalField::DefaultAddressBookId.into()),\n                })\n                .await\n                .caused_by(trc::location!())?;\n            if default_book_id.is_some_and(|id| id == document_id) {\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .clear(PrincipalField::DefaultAddressBookId);\n            }\n        } else {\n            // Validate ACL\n            let addressbook_id = delete_resource.parent_id().unwrap();\n            if !access_token.is_member(account_id)\n                && !resources.has_access_to_container(\n                    access_token,\n                    addressbook_id,\n                    Acl::RemoveItems,\n                )\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            let card_ = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::ContactCard,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n            // Validate headers\n            self.validate_headers(\n                access_token,\n                headers,\n                vec![ResourceState {\n                    account_id,\n                    collection: Collection::ContactCard,\n                    document_id: document_id.into(),\n                    etag: card_.etag().into(),\n                    path: delete_path,\n                    ..Default::default()\n                }],\n                Default::default(),\n                DavMethod::DELETE,\n            )\n            .await?;\n\n            // Delete card\n            DestroyArchive(\n                card_\n                    .to_unarchived::<ContactCard>()\n                    .caused_by(trc::location!())?,\n            )\n            .delete(\n                access_token,\n                account_id,\n                document_id,\n                addressbook_id,\n                resources.format_resource(delete_resource).into(),\n                &mut batch,\n            )\n            .caused_by(trc::location!())?;\n        }\n\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n        self.notify_task_queue();\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/card/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        ETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{RequestHeaders, schema::property::Rfc1123DateTime};\nuse groupware::{cache::GroupwareCache, contact::ContactCard};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait CardGetRequestHandler: Sync + Send {\n    fn handle_card_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CardGetRequestHandler for Server {\n    async fn handle_card_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resources\n            .by_path(\n                resource_\n                    .resource\n                    .ok_or(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))?,\n            )\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        if resource.is_container() {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Validate ACL\n        if !access_token.is_member(account_id)\n            && !resources.has_access_to_container(\n                access_token,\n                resource.parent_id().unwrap(),\n                Acl::ReadItems,\n            )\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Fetch card\n        let card_ = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::ContactCard,\n                resource.document_id(),\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let card = card_\n            .unarchive::<ContactCard>()\n            .caused_by(trc::location!())?;\n\n        // Validate headers\n        let etag = card_.etag();\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: Collection::ContactCard,\n                document_id: resource.document_id().into(),\n                etag: etag.clone().into(),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::GET,\n        )\n        .await?;\n\n        let response = HttpResponse::new(StatusCode::OK)\n            .with_content_type(\"text/vcard; charset=utf-8\")\n            .with_etag(etag)\n            .with_last_modified(Rfc1123DateTime::new(i64::from(card.modified)).to_string());\n\n        let mut vcard = String::with_capacity(128);\n        let _ = card.card.write_to(\n            &mut vcard,\n            headers\n                .max_vcard_version\n                .or_else(|| card.card.version())\n                .unwrap_or_default(),\n        );\n\n        if !is_head {\n            Ok(response.with_binary_body(vcard))\n        } else {\n            Ok(response.with_content_length(vcard.len()))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/card/mkcol.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::proppatch::CardPropPatchRequestHandler;\nuse crate::{\n    DavError, DavMethod, PropStatBuilder,\n    common::{\n        ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{Namespace, request::MkCol, response::MkColResponse},\n};\nuse groupware::{\n    cache::GroupwareCache,\n    contact::{AddressBook, AddressBookPreferences},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::BatchBuilder;\nuse trc::AddContext;\nuse types::collection::{Collection, SyncCollection};\n\npub(crate) trait CardMkColRequestHandler: Sync + Send {\n    fn handle_card_mkcol_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: Option<MkCol>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CardMkColRequestHandler for Server {\n    async fn handle_card_mkcol_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: Option<MkCol>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let name = resource\n            .resource\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        if !access_token.is_member(account_id) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        } else if name.contains('/')\n            || self\n                .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n                .await\n                .caused_by(trc::location!())?\n                .by_path(name)\n                .is_some()\n        {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: resource.collection,\n                document_id: Some(u32::MAX),\n                path: name,\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::MKCOL,\n        )\n        .await?;\n\n        // Build file container\n        let mut book = AddressBook {\n            name: name.to_string(),\n            preferences: vec![AddressBookPreferences {\n                account_id,\n                name: \"Address Book\".to_string(),\n                ..Default::default()\n            }],\n            ..Default::default()\n        };\n\n        // Apply MKCOL properties\n        let mut return_prop_stat = None;\n        if let Some(mkcol) = request {\n            let mut prop_stat = PropStatBuilder::default();\n            if !self.apply_addressbook_properties(\n                access_token,\n                &mut book,\n                false,\n                mkcol.props,\n                &mut prop_stat,\n            ) {\n                return Ok(HttpResponse::new(StatusCode::FORBIDDEN).with_xml_body(\n                    MkColResponse::new(prop_stat.build())\n                        .with_namespace(Namespace::CardDav)\n                        .to_string(),\n                ));\n            }\n            if headers.ret != Return::Minimal {\n                return_prop_stat = Some(prop_stat);\n            }\n        }\n\n        // Prepare write batch\n        let mut batch = BatchBuilder::new();\n        let document_id = self\n            .store()\n            .assign_document_ids(account_id, Collection::AddressBook, 1)\n            .await\n            .caused_by(trc::location!())?;\n        book.insert(access_token, account_id, document_id, &mut batch)\n            .caused_by(trc::location!())?;\n        let etag = batch.etag();\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n        if let Some(prop_stat) = return_prop_stat {\n            Ok(HttpResponse::new(StatusCode::CREATED)\n                .with_xml_body(\n                    MkColResponse::new(prop_stat.build())\n                        .with_namespace(Namespace::CardDav)\n                        .to_string(),\n                )\n                .with_etag_opt(etag))\n        } else {\n            Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/card/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{DavError, DavErrorCondition};\nuse common::{DavResources, Server};\nuse dav_proto::schema::{\n    property::{CardDavProperty, DavProperty, WebDavProperty},\n    response::CardCondition,\n};\nuse hyper::StatusCode;\nuse trc::AddContext;\nuse types::{collection::Collection, field::ContactField};\n\npub mod copy_move;\npub mod delete;\npub mod get;\npub mod mkcol;\npub mod proppatch;\npub mod query;\npub mod update;\n\npub(crate) static CARD_CONTAINER_PROPS: [DavProperty; 23] = [\n    DavProperty::WebDav(WebDavProperty::CreationDate),\n    DavProperty::WebDav(WebDavProperty::DisplayName),\n    DavProperty::WebDav(WebDavProperty::GetETag),\n    DavProperty::WebDav(WebDavProperty::GetLastModified),\n    DavProperty::WebDav(WebDavProperty::ResourceType),\n    DavProperty::WebDav(WebDavProperty::LockDiscovery),\n    DavProperty::WebDav(WebDavProperty::SupportedLock),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n    DavProperty::WebDav(WebDavProperty::SyncToken),\n    DavProperty::WebDav(WebDavProperty::Owner),\n    DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::Acl),\n    DavProperty::WebDav(WebDavProperty::AclRestrictions),\n    DavProperty::WebDav(WebDavProperty::InheritedAclSet),\n    DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n    DavProperty::WebDav(WebDavProperty::SupportedReportSet),\n    DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes),\n    DavProperty::WebDav(WebDavProperty::QuotaUsedBytes),\n    DavProperty::CardDav(CardDavProperty::AddressbookDescription),\n    DavProperty::CardDav(CardDavProperty::SupportedAddressData),\n    DavProperty::CardDav(CardDavProperty::SupportedCollationSet),\n    DavProperty::CardDav(CardDavProperty::MaxResourceSize),\n];\n\npub(crate) static CARD_ITEM_PROPS: [DavProperty; 20] = [\n    DavProperty::WebDav(WebDavProperty::CreationDate),\n    DavProperty::WebDav(WebDavProperty::DisplayName),\n    DavProperty::WebDav(WebDavProperty::GetETag),\n    DavProperty::WebDav(WebDavProperty::GetLastModified),\n    DavProperty::WebDav(WebDavProperty::ResourceType),\n    DavProperty::WebDav(WebDavProperty::LockDiscovery),\n    DavProperty::WebDav(WebDavProperty::SupportedLock),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n    DavProperty::WebDav(WebDavProperty::SyncToken),\n    DavProperty::WebDav(WebDavProperty::Owner),\n    DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::Acl),\n    DavProperty::WebDav(WebDavProperty::AclRestrictions),\n    DavProperty::WebDav(WebDavProperty::InheritedAclSet),\n    DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n    DavProperty::WebDav(WebDavProperty::GetContentLanguage),\n    DavProperty::WebDav(WebDavProperty::GetContentLength),\n    DavProperty::WebDav(WebDavProperty::GetContentType),\n    DavProperty::CardDav(CardDavProperty::AddressData(vec![])),\n];\n\npub(crate) async fn assert_is_unique_uid(\n    server: &Server,\n    resources: &DavResources,\n    account_id: u32,\n    addressbook_id: u32,\n    uid: Option<&str>,\n) -> crate::Result<()> {\n    if let Some(uid) = uid {\n        let hits = server\n            .document_ids_matching(\n                account_id,\n                Collection::ContactCard,\n                ContactField::Uid,\n                uid.as_bytes(),\n            )\n            .await\n            .caused_by(trc::location!())?;\n        if !hits.is_empty() {\n            for path in resources.children(addressbook_id) {\n                if hits.contains(path.document_id()) {\n                    return Err(DavError::Condition(DavErrorCondition::new(\n                        StatusCode::PRECONDITION_FAILED,\n                        CardCondition::NoUidConflict(resources.format_resource(path).into()),\n                    )));\n                }\n            }\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/dav/src/card/proppatch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod, PropStatBuilder,\n    common::{\n        ETag, ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{\n        Namespace,\n        property::{CardDavProperty, DavProperty, DavValue, ResourceType, WebDavProperty},\n        request::{DavPropertyValue, PropertyUpdate},\n        response::{BaseCondition, MultiStatus, Response},\n    },\n};\nuse groupware::{\n    cache::GroupwareCache,\n    contact::{AddressBook, ContactCard},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::BatchBuilder;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait CardPropPatchRequestHandler: Sync + Send {\n    fn handle_card_proppatch_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: PropertyUpdate,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn apply_addressbook_properties(\n        &self,\n        access_token: &AccessToken,\n        address_book: &mut AddressBook,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool;\n\n    fn apply_card_properties(\n        &self,\n        card: &mut ContactCard,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool;\n}\n\nimpl CardPropPatchRequestHandler for Server {\n    async fn handle_card_proppatch_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        mut request: PropertyUpdate,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let uri = headers.uri;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resource_\n            .resource\n            .and_then(|r| resources.by_path(r))\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let document_id = resource.document_id();\n        let collection = if resource.is_container() {\n            Collection::AddressBook\n        } else {\n            Collection::ContactCard\n        };\n\n        if !request.has_changes() {\n            return Ok(HttpResponse::new(StatusCode::NO_CONTENT));\n        }\n\n        // Verify ACL\n        if !access_token.is_member(account_id) {\n            let (acl, document_id) = if resource.is_container() {\n                (Acl::Modify, resource.document_id())\n            } else {\n                (Acl::ModifyItems, resource.parent_id().unwrap())\n            };\n\n            if !resources.has_access_to_container(access_token, document_id, acl) {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n        }\n\n        // Fetch archive\n        let archive = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                collection,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection,\n                document_id: document_id.into(),\n                etag: archive.etag().into(),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::PROPPATCH,\n        )\n        .await?;\n\n        let is_success;\n        let mut batch = BatchBuilder::new();\n        let mut items = PropStatBuilder::default();\n\n        let etag = if resource.is_container() {\n            // Deserialize\n            let book = archive\n                .to_unarchived::<AddressBook>()\n                .caused_by(trc::location!())?;\n            let mut new_book = archive\n                .deserialize::<AddressBook>()\n                .caused_by(trc::location!())?;\n\n            // Remove properties\n            if !request.set_first && !request.remove.is_empty() {\n                remove_addressbook_properties(\n                    access_token,\n                    &mut new_book,\n                    std::mem::take(&mut request.remove),\n                    &mut items,\n                );\n            }\n\n            // Set properties\n            is_success = self.apply_addressbook_properties(\n                access_token,\n                &mut new_book,\n                true,\n                request.set,\n                &mut items,\n            );\n\n            // Remove properties\n            if is_success && !request.remove.is_empty() {\n                remove_addressbook_properties(\n                    access_token,\n                    &mut new_book,\n                    request.remove,\n                    &mut items,\n                );\n            }\n\n            if is_success {\n                new_book\n                    .update(access_token, book, account_id, document_id, &mut batch)\n                    .caused_by(trc::location!())?\n                    .etag()\n            } else {\n                book.etag().into()\n            }\n        } else {\n            // Deserialize\n            let card = archive\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n            let mut new_card = archive\n                .deserialize::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            // Remove properties\n            if !request.set_first && !request.remove.is_empty() {\n                remove_card_properties(\n                    &mut new_card,\n                    std::mem::take(&mut request.remove),\n                    &mut items,\n                );\n            }\n\n            // Set properties\n            is_success = self.apply_card_properties(&mut new_card, true, request.set, &mut items);\n\n            // Remove properties\n            if is_success && !request.remove.is_empty() {\n                remove_card_properties(&mut new_card, request.remove, &mut items);\n            }\n\n            if is_success {\n                new_card\n                    .update(access_token, card, account_id, document_id, &mut batch)\n                    .caused_by(trc::location!())?\n                    .etag()\n            } else {\n                card.etag().into()\n            }\n        };\n\n        if is_success {\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n        }\n\n        if headers.ret != Return::Minimal || !is_success {\n            Ok(HttpResponse::new(StatusCode::MULTI_STATUS)\n                .with_xml_body(\n                    MultiStatus::new(vec![Response::new_propstat(uri, items.build())])\n                        .with_namespace(Namespace::CardDav)\n                        .to_string(),\n                )\n                .with_etag_opt(etag))\n        } else {\n            Ok(HttpResponse::new(StatusCode::NO_CONTENT).with_etag_opt(etag))\n        }\n    }\n\n    fn apply_addressbook_properties(\n        &self,\n        access_token: &AccessToken,\n        address_book: &mut AddressBook,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool {\n        let mut has_errors = false;\n\n        for property in properties {\n            match (&property.property, property.value) {\n                (DavProperty::WebDav(WebDavProperty::DisplayName), DavValue::String(name)) => {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        address_book.preferences_mut(access_token).name = name;\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (\n                    DavProperty::CardDav(CardDavProperty::AddressbookDescription),\n                    DavValue::String(name),\n                ) => {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        address_book.preferences_mut(access_token).description = Some(name);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n\n                        has_errors = true;\n                    }\n                }\n                (DavProperty::WebDav(WebDavProperty::CreationDate), DavValue::Timestamp(dt)) => {\n                    address_book.created = dt;\n                    items.insert_ok(property.property);\n                }\n                (\n                    DavProperty::WebDav(WebDavProperty::ResourceType),\n                    DavValue::ResourceTypes(types),\n                ) => {\n                    if !types.0.iter().all(|rt| {\n                        matches!(rt, ResourceType::Collection | ResourceType::AddressBook)\n                    }) {\n                        items.insert_precondition_failed(\n                            property.property,\n                            StatusCode::FORBIDDEN,\n                            BaseCondition::ValidResourceType,\n                        );\n                        has_errors = true;\n                    } else {\n                        items.insert_ok(property.property);\n                    }\n                }\n                (DavProperty::DeadProperty(dead), DavValue::DeadProperty(values))\n                    if self.core.groupware.dead_property_size.is_some() =>\n                {\n                    if is_update {\n                        address_book.dead_properties.remove_element(dead);\n                    }\n\n                    if address_book.dead_properties.size() + values.size() + dead.size()\n                        < self.core.groupware.dead_property_size.unwrap()\n                    {\n                        address_book\n                            .dead_properties\n                            .add_element(dead.clone(), values.0);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (_, DavValue::Null) => {\n                    items.insert_ok(property.property);\n                }\n                _ => {\n                    items.insert_error_with_description(\n                        property.property,\n                        StatusCode::CONFLICT,\n                        \"Property cannot be modified\",\n                    );\n                    has_errors = true;\n                }\n            }\n        }\n\n        !has_errors\n    }\n\n    fn apply_card_properties(\n        &self,\n        card: &mut ContactCard,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool {\n        let mut has_errors = false;\n\n        for property in properties {\n            match (&property.property, property.value) {\n                (DavProperty::WebDav(WebDavProperty::DisplayName), DavValue::String(name)) => {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        card.display_name = Some(name);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (DavProperty::WebDav(WebDavProperty::CreationDate), DavValue::Timestamp(dt)) => {\n                    card.created = dt;\n                    items.insert_ok(property.property);\n                }\n                (DavProperty::DeadProperty(dead), DavValue::DeadProperty(values))\n                    if self.core.groupware.dead_property_size.is_some() =>\n                {\n                    if is_update {\n                        card.dead_properties.remove_element(dead);\n                    }\n\n                    if card.dead_properties.size() + values.size() + dead.size()\n                        < self.core.groupware.dead_property_size.unwrap()\n                    {\n                        card.dead_properties.add_element(dead.clone(), values.0);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (_, DavValue::Null) => {\n                    items.insert_ok(property.property);\n                }\n                _ => {\n                    items.insert_error_with_description(\n                        property.property,\n                        StatusCode::CONFLICT,\n                        \"Property cannot be modified\",\n                    );\n                    has_errors = true;\n                }\n            }\n        }\n\n        !has_errors\n    }\n}\n\nfn remove_card_properties(\n    card: &mut ContactCard,\n    properties: Vec<DavProperty>,\n    items: &mut PropStatBuilder,\n) {\n    for property in properties {\n        match &property {\n            DavProperty::WebDav(WebDavProperty::DisplayName) => {\n                card.display_name = None;\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::DeadProperty(dead) => {\n                card.dead_properties.remove_element(dead);\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            _ => {\n                items.insert_error_with_description(\n                    property,\n                    StatusCode::CONFLICT,\n                    \"Property cannot be deleted\",\n                );\n            }\n        }\n    }\n}\n\nfn remove_addressbook_properties(\n    access_token: &AccessToken,\n    book: &mut AddressBook,\n    properties: Vec<DavProperty>,\n    items: &mut PropStatBuilder,\n) {\n    for property in properties {\n        match &property {\n            DavProperty::CardDav(CardDavProperty::AddressbookDescription) => {\n                book.preferences_mut(access_token).description = None;\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::WebDav(WebDavProperty::DisplayName) => {\n                book.preferences_mut(access_token).name.clear();\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::DeadProperty(dead) => {\n                book.dead_properties.remove_element(dead);\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            _ => {\n                items.insert_error_with_description(\n                    property,\n                    StatusCode::CONFLICT,\n                    \"Property cannot be deleted\",\n                );\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/card/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError,\n    common::{\n        AddressbookFilter, DavQuery,\n        propfind::{PropFindItem, PropFindRequestHandler},\n        uri::DavUriResource,\n    },\n};\nuse calcard::vcard::{\n    ArchivedVCard, ArchivedVCardEntry, ArchivedVCardParameter, VCardParameterName, VCardProperty,\n    VCardVersion,\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders,\n    schema::{\n        property::CardDavPropertyName,\n        request::{AddressbookQuery, Filter, FilterOp, VCardPropertyWithGroup},\n        response::MultiStatus,\n    },\n};\nuse groupware::cache::GroupwareCache;\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::fmt::Write;\nuse trc::AddContext;\nuse types::{acl::Acl, collection::SyncCollection};\n\npub(crate) trait CardQueryRequestHandler: Sync + Send {\n    fn handle_card_query_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: AddressbookQuery,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CardQueryRequestHandler for Server {\n    async fn handle_card_query_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: AddressbookQuery,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n        let Some(resource) = resources.by_path(\n            resource_\n                .resource\n                .ok_or(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))?,\n        ) else {\n            return Ok(HttpResponse::new(StatusCode::MULTI_STATUS)\n                .with_xml_body(MultiStatus::not_found(headers.uri).to_string()));\n        };\n        if !resource.is_container() {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        }\n\n        // Obtain shared ids\n        let shared_ids = if !access_token.is_member(account_id) {\n            resources\n                .shared_containers(access_token, [Acl::ReadItems], false)\n                .into()\n        } else {\n            None\n        };\n\n        // Obtain document ids in folder\n        let mut items = Vec::with_capacity(16);\n        for resource in resources.children(resource.document_id()) {\n            if shared_ids\n                .as_ref()\n                .is_none_or(|ids| ids.contains(resource.document_id()))\n            {\n                items.push(PropFindItem::new(\n                    resources.format_resource(resource),\n                    account_id,\n                    resource,\n                ));\n            }\n        }\n\n        self.handle_dav_query(\n            access_token,\n            DavQuery::addressbook_query(request, items, headers),\n        )\n        .await\n    }\n}\n\npub(crate) fn vcard_query(card: &ArchivedVCard, filters: &AddressbookFilter) -> bool {\n    let mut is_all = true;\n    let mut matches_one = false;\n\n    for filter in filters {\n        match filter {\n            Filter::AnyOf => {\n                is_all = false;\n            }\n            Filter::AllOf => {\n                is_all = true;\n            }\n            Filter::Property { prop, op, .. } => {\n                let mut properties = find_properties(card, prop).peekable();\n                let result = if properties.peek().is_some() {\n                    properties.any(|entry| match op {\n                        FilterOp::Exists => true,\n                        FilterOp::Undefined => false,\n                        FilterOp::TextMatch(text_match) => {\n                            let mut matched_any = false;\n\n                            for value in entry.values.iter() {\n                                if let Some(text) = value.as_text()\n                                    && text_match.matches(text)\n                                {\n                                    matched_any = true;\n                                    break;\n                                }\n                            }\n\n                            matched_any\n                        }\n                        FilterOp::TimeRange(_) => false,\n                    })\n                } else {\n                    matches!(op, FilterOp::Undefined)\n                };\n\n                if result {\n                    matches_one = true;\n                } else if is_all {\n                    return false;\n                }\n            }\n            Filter::Parameter {\n                prop, param, op, ..\n            } => {\n                let mut properties = find_properties(card, prop)\n                    .filter_map(|entry| find_parameter(entry, param))\n                    .peekable();\n                let result = if properties.peek().is_some() {\n                    properties.any(|entry| match op {\n                        FilterOp::Exists => true,\n                        FilterOp::Undefined => false,\n                        FilterOp::TextMatch(text_match) => {\n                            if let Some(text) = entry.value.as_text() {\n                                text_match.matches(text)\n                            } else {\n                                false\n                            }\n                        }\n                        FilterOp::TimeRange(_) => false,\n                    })\n                } else {\n                    matches!(op, FilterOp::Undefined)\n                };\n\n                if result {\n                    matches_one = true;\n                } else if is_all {\n                    return false;\n                }\n            }\n            Filter::Component { .. } => {}\n        }\n    }\n\n    is_all || matches_one\n}\n\n#[inline(always)]\nfn find_properties<'x>(\n    card: &'x ArchivedVCard,\n    prop: &VCardPropertyWithGroup,\n) -> impl Iterator<Item = &'x ArchivedVCardEntry> {\n    card.entries\n        .iter()\n        .filter(move |entry| entry.name == prop.name && entry.group == prop.group)\n}\n\n#[inline(always)]\nfn find_parameter<'x>(\n    entry: &'x ArchivedVCardEntry,\n    name: &VCardParameterName,\n) -> Option<&'x ArchivedVCardParameter> {\n    entry.params.iter().find(|param| param.name == *name)\n}\n\npub(crate) fn serialize_vcard_with_props(\n    card: &ArchivedVCard,\n    props: &[CardDavPropertyName],\n    version: Option<VCardVersion>,\n) -> String {\n    let mut vcard = String::with_capacity(128);\n    let version = version.or_else(|| card.version()).unwrap_or_default();\n    if !props.is_empty() {\n        let _ = write!(&mut vcard, \"BEGIN:VCARD\\r\\n\");\n        let is_v4 = matches!(version, VCardVersion::V4_0);\n\n        for entry in card.entries.iter() {\n            for item in props {\n                if entry.name == item.name && entry.group == item.group {\n                    if item.name != VCardProperty::Version {\n                        let _ = entry.write_to(&mut vcard, !item.no_value, is_v4);\n                    } else {\n                        let _ = write!(&mut vcard, \"VERSION:{version}\\r\\n\");\n                    }\n                    break;\n                }\n            }\n        }\n        let _ = write!(&mut vcard, \"END:VCARD\\r\\n\");\n    } else {\n        let _ = card.write_to(&mut vcard, version);\n    }\n\n    vcard\n}\n"
  },
  {
    "path": "crates/dav/src/card/update.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::assert_is_unique_uid;\nuse crate::{\n    DavError, DavErrorCondition, DavMethod,\n    common::{\n        ETag, ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n    fix_percent_encoding,\n};\nuse calcard::{Entry, Parser};\nuse common::{DavName, Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{property::Rfc1123DateTime, response::CardCondition},\n};\nuse groupware::{cache::GroupwareCache, contact::ContactCard};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::BatchBuilder;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait CardUpdateRequestHandler: Sync + Send {\n    fn handle_card_update_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n        is_patch: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl CardUpdateRequestHandler for Server {\n    async fn handle_card_update_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n        _is_patch: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n        let resource_name = fix_percent_encoding(\n            resource\n                .resource\n                .ok_or(DavError::Code(StatusCode::CONFLICT))?,\n        );\n\n        if bytes.len() > self.core.groupware.max_vcard_size {\n            return Err(DavError::Condition(DavErrorCondition::new(\n                StatusCode::PRECONDITION_FAILED,\n                CardCondition::MaxResourceSize(self.core.groupware.max_vcard_size as u32),\n            )));\n        }\n        let vcard_raw = std::str::from_utf8(&bytes).map_err(|_| {\n            DavError::Condition(\n                DavErrorCondition::new(\n                    StatusCode::PRECONDITION_FAILED,\n                    CardCondition::SupportedAddressData,\n                )\n                .with_details(\"The request body is not valid UTF-8.\"),\n            )\n        })?;\n\n        let vcard = match Parser::new(vcard_raw).strict().entry() {\n            Entry::VCard(vcard) => vcard,\n            _ => {\n                return Err(DavError::Condition(\n                    DavErrorCondition::new(\n                        StatusCode::PRECONDITION_FAILED,\n                        CardCondition::SupportedAddressData,\n                    )\n                    .with_details(\"Failed to parse vCard data.\"),\n                ));\n            }\n        };\n\n        if let Some(resource) = resources.by_path(resource_name.as_ref()) {\n            if resource.is_container() {\n                return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n            }\n\n            // Validate ACL\n            let parent_id = resource.parent_id().unwrap();\n            let document_id = resource.document_id();\n            if !access_token.is_member(account_id)\n                && !resources.has_access_to_container(access_token, parent_id, Acl::ModifyItems)\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Update\n            let card_ = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::ContactCard,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n            let card = card_\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            // Validate headers\n            match self\n                .validate_headers(\n                    access_token,\n                    headers,\n                    vec![ResourceState {\n                        account_id,\n                        collection: Collection::ContactCard,\n                        document_id: Some(document_id),\n                        etag: card.etag().into(),\n                        path: resource_name.as_ref(),\n                        ..Default::default()\n                    }],\n                    Default::default(),\n                    DavMethod::PUT,\n                )\n                .await\n            {\n                Ok(_) => {}\n                Err(DavError::Code(StatusCode::PRECONDITION_FAILED))\n                    if headers.ret == Return::Representation =>\n                {\n                    return Ok(HttpResponse::new(StatusCode::PRECONDITION_FAILED)\n                        .with_content_type(\"text/vcard; charset=utf-8\")\n                        .with_etag(card.etag())\n                        .with_last_modified(\n                            Rfc1123DateTime::new(i64::from(card.inner.modified)).to_string(),\n                        )\n                        .with_header(\"Preference-Applied\", \"return=representation\")\n                        .with_binary_body(card.inner.card.to_string()));\n                }\n                Err(e) => return Err(e),\n            }\n\n            // Validate UID\n            match (card.inner.card.uid(), vcard.uid()) {\n                (Some(old_uid), Some(new_uid)) if old_uid == new_uid => {}\n                (None, None) | (None, Some(_)) => {}\n                _ => {\n                    return Err(DavError::Condition(DavErrorCondition::new(\n                        StatusCode::PRECONDITION_FAILED,\n                        CardCondition::NoUidConflict(resources.format_resource(resource).into()),\n                    )));\n                }\n            }\n\n            // Validate quota\n            let extra_bytes =\n                (bytes.len() as u64).saturating_sub(u32::from(card.inner.size) as u64);\n            if extra_bytes > 0 {\n                self.has_available_quota(\n                    &self.get_resource_token(access_token, account_id).await?,\n                    extra_bytes,\n                )\n                .await?;\n            }\n\n            // Build node\n            let mut new_card = card\n                .deserialize::<ContactCard>()\n                .caused_by(trc::location!())?;\n            new_card.size = bytes.len() as u32;\n            new_card.card = vcard;\n\n            // Prepare write batch\n            let mut batch = BatchBuilder::new();\n            let etag = new_card\n                .update(access_token, card, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?\n                .etag();\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n            self.notify_task_queue();\n\n            Ok(HttpResponse::new(StatusCode::NO_CONTENT).with_etag_opt(etag))\n        } else if let Some((Some(parent), name)) = resources.map_parent(resource_name.as_ref()) {\n            if !parent.is_container() {\n                return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n            }\n\n            // Validate ACL\n            if !access_token.is_member(account_id)\n                && !resources.has_access_to_container(\n                    access_token,\n                    parent.document_id(),\n                    Acl::AddItems,\n                )\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Validate headers\n            self.validate_headers(\n                access_token,\n                headers,\n                vec![ResourceState {\n                    account_id,\n                    collection: resource.collection,\n                    document_id: Some(u32::MAX),\n                    path: resource_name.as_ref(),\n                    ..Default::default()\n                }],\n                Default::default(),\n                DavMethod::PUT,\n            )\n            .await?;\n\n            // Validate UID\n            assert_is_unique_uid(\n                self,\n                &resources,\n                account_id,\n                parent.document_id(),\n                vcard.uid(),\n            )\n            .await?;\n\n            // Validate quota\n            if !bytes.is_empty() {\n                self.has_available_quota(\n                    &self.get_resource_token(access_token, account_id).await?,\n                    bytes.len() as u64,\n                )\n                .await?;\n            }\n\n            // Build node\n            let card = ContactCard {\n                names: vec![DavName {\n                    name: name.to_string(),\n                    parent_id: parent.document_id(),\n                }],\n                card: vcard,\n                size: bytes.len() as u32,\n                ..Default::default()\n            };\n\n            // Prepare write batch\n            let mut batch = BatchBuilder::new();\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::ContactCard, 1)\n                .await\n                .caused_by(trc::location!())?;\n            let etag = card\n                .insert(access_token, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?\n                .etag();\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n            self.notify_task_queue();\n\n            Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n        } else {\n            Err(DavError::Code(StatusCode::CONFLICT))?\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/common/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ArchivedResource;\nuse crate::{\n    DavError, DavErrorCondition, DavResourceName, common::uri::DavUriResource,\n    principal::propfind::PrincipalPropFind,\n};\nuse common::{DavResources, Server, auth::AccessToken, sharing::EffectiveAcl};\nuse dav_proto::{\n    RequestHeaders,\n    schema::{\n        property::{DavProperty, Privilege, WebDavProperty},\n        request::{AclPrincipalPropSet, PropFind},\n        response::{Ace, BaseCondition, GrantDeny, Href, MultiStatus, Principal},\n    },\n};\nuse directory::{QueryParams, Type, backend::internal::manage::ManageDirectory};\nuse groupware::RFC_3986;\nuse groupware::{cache::GroupwareCache, calendar::Calendar, contact::AddressBook, file::FileNode};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse rkyv::vec::ArchivedVec;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse store::{ahash::AHashSet, roaring::RoaringBitmap, write::BatchBuilder};\nuse trc::AddContext;\nuse types::{\n    acl::{Acl, AclGrant, ArchivedAclGrant},\n    collection::Collection,\n};\nuse utils::map::bitmap::Bitmap;\n\npub(crate) trait DavAclHandler: Sync + Send {\n    fn handle_acl_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: dav_proto::schema::request::Acl,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn handle_acl_prop_set(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: AclPrincipalPropSet,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn validate_and_map_aces(\n        &self,\n        access_token: &AccessToken,\n        acl: dav_proto::schema::request::Acl,\n        collection: Collection,\n    ) -> impl Future<Output = crate::Result<Vec<AclGrant>>> + Send;\n\n    fn resolve_ace(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        grants: &ArchivedVec<ArchivedAclGrant>,\n        expand: Option<&PropFind>,\n    ) -> impl Future<Output = crate::Result<Vec<Ace>>> + Send;\n}\n\npub(crate) trait ResourceAcl {\n    fn validate_and_map_parent_acl(\n        &self,\n        access_token: &AccessToken,\n        is_member: bool,\n        parent_id: Option<u32>,\n        check_acls: impl Into<Bitmap<Acl>> + Send,\n    ) -> crate::Result<u32>;\n}\n\nimpl DavAclHandler for Server {\n    async fn handle_acl_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: dav_proto::schema::request::Acl,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let collection = resource_.collection;\n\n        if !matches!(\n            collection,\n            Collection::AddressBook | Collection::Calendar | Collection::FileNode\n        ) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, collection.into())\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resource_\n            .resource\n            .and_then(|r| resources.by_path(r))\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        if !resource.resource.is_container() && !matches!(collection, Collection::FileNode) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Fetch node\n        let archive = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                collection,\n                resource.document_id(),\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n        let container =\n            ArchivedResource::from_archive(&archive, collection).caused_by(trc::location!())?;\n\n        // Validate ACL\n        let acls = container.acls().unwrap();\n        if !access_token.is_member(account_id)\n            && !acls.effective_acl(access_token).contains(Acl::Share)\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Validate ACEs\n        let grants = self\n            .validate_and_map_aces(access_token, request, collection)\n            .await?;\n\n        if grants.len() != acls.len() || acls.iter().zip(grants.iter()).any(|(a, b)| a != b) {\n            // Refresh ACLs\n            self.refresh_archived_acls(&grants, acls).await;\n\n            let mut batch = BatchBuilder::new();\n            match container {\n                ArchivedResource::Calendar(calendar) => {\n                    let mut new_calendar = calendar\n                        .deserialize::<Calendar>()\n                        .caused_by(trc::location!())?;\n                    new_calendar.acls = grants;\n                    new_calendar\n                        .update(\n                            access_token,\n                            calendar,\n                            account_id,\n                            resource.document_id(),\n                            &mut batch,\n                        )\n                        .caused_by(trc::location!())?;\n                }\n                ArchivedResource::AddressBook(book) => {\n                    let mut new_book = book\n                        .deserialize::<AddressBook>()\n                        .caused_by(trc::location!())?;\n                    new_book.acls = grants;\n                    new_book\n                        .update(\n                            access_token,\n                            book,\n                            account_id,\n                            resource.document_id(),\n                            &mut batch,\n                        )\n                        .caused_by(trc::location!())?;\n                }\n                ArchivedResource::FileNode(node) => {\n                    let mut new_node =\n                        node.deserialize::<FileNode>().caused_by(trc::location!())?;\n                    new_node.acls = grants;\n                    new_node\n                        .update(\n                            access_token,\n                            node,\n                            account_id,\n                            resource.document_id(),\n                            &mut batch,\n                        )\n                        .caused_by(trc::location!())?;\n                }\n                _ => unreachable!(),\n            }\n\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n        }\n\n        Ok(HttpResponse::new(StatusCode::OK))\n    }\n\n    async fn handle_acl_prop_set(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        mut request: AclPrincipalPropSet,\n    ) -> crate::Result<HttpResponse> {\n        let uri = self\n            .validate_uri(access_token, headers.uri)\n            .await\n            .and_then(|uri| uri.into_owned_uri())?;\n        let uri = self\n            .map_uri_resource(access_token, uri)\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n        if !matches!(\n            uri.collection,\n            Collection::Calendar | Collection::AddressBook | Collection::FileNode\n        ) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        let archive = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                uri.account_id,\n                uri.collection,\n                uri.resource,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n        let acls = match uri.collection {\n            Collection::FileNode => {\n                &archive\n                    .unarchive::<FileNode>()\n                    .caused_by(trc::location!())?\n                    .acls\n            }\n            Collection::AddressBook => {\n                &archive\n                    .unarchive::<AddressBook>()\n                    .caused_by(trc::location!())?\n                    .acls\n            }\n            Collection::Calendar => {\n                &archive\n                    .unarchive::<Calendar>()\n                    .caused_by(trc::location!())?\n                    .acls\n            }\n            _ => unreachable!(),\n        };\n\n        // Validate ACLs\n        if !access_token.is_member(uri.account_id)\n            && !acls.effective_acl(access_token).contains(Acl::Read)\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Validate\n        let account_ids = RoaringBitmap::from_iter(acls.iter().map(|a| u32::from(a.account_id)));\n        let mut response = MultiStatus::new(Vec::with_capacity(16));\n\n        if !account_ids.is_empty() {\n            if request.properties.is_empty() {\n                request\n                    .properties\n                    .push(DavProperty::WebDav(WebDavProperty::DisplayName));\n            }\n            let request = PropFind::Prop(request.properties);\n            self.prepare_principal_propfind_response(\n                access_token,\n                Collection::Principal,\n                account_ids.into_iter(),\n                &request,\n                &mut response,\n            )\n            .await?;\n        }\n\n        Ok(HttpResponse::new(StatusCode::MULTI_STATUS).with_xml_body(response.to_string()))\n    }\n\n    async fn validate_and_map_aces(\n        &self,\n        access_token: &AccessToken,\n        acl: dav_proto::schema::request::Acl,\n        collection: Collection,\n    ) -> crate::Result<Vec<AclGrant>> {\n        let mut grants = Vec::with_capacity(acl.aces.len());\n        for ace in acl.aces {\n            if ace.invert {\n                return Err(DavError::Condition(DavErrorCondition::new(\n                    StatusCode::FORBIDDEN,\n                    BaseCondition::NoInvert,\n                )));\n            }\n            let privileges = match ace.grant_deny {\n                GrantDeny::Grant(list) => list.0,\n                GrantDeny::Deny(_) => {\n                    return Err(DavError::Condition(DavErrorCondition::new(\n                        StatusCode::FORBIDDEN,\n                        BaseCondition::GrantOnly,\n                    )));\n                }\n            };\n            let principal_uri = match ace.principal {\n                Principal::Href(href) => href.0,\n                _ => {\n                    return Err(DavError::Condition(DavErrorCondition::new(\n                        StatusCode::FORBIDDEN,\n                        BaseCondition::AllowedPrincipal,\n                    )));\n                }\n            };\n\n            let mut acls = Bitmap::<Acl>::default();\n            for privilege in privileges {\n                match privilege {\n                    Privilege::Read => {\n                        acls.insert(Acl::Read);\n                        acls.insert(Acl::ReadItems);\n                    }\n                    Privilege::Write => {\n                        acls.insert(Acl::Modify);\n                        acls.insert(Acl::Delete);\n                        acls.insert(Acl::AddItems);\n                        acls.insert(Acl::ModifyItems);\n                        acls.insert(Acl::RemoveItems);\n                    }\n                    Privilege::WriteContent => {\n                        acls.insert(Acl::AddItems);\n                        acls.insert(Acl::Modify);\n                        acls.insert(Acl::ModifyItems);\n                    }\n                    Privilege::WriteProperties => {\n                        acls.insert(Acl::Modify);\n                    }\n                    Privilege::ReadCurrentUserPrivilegeSet\n                    | Privilege::Unlock\n                    | Privilege::Bind\n                    | Privilege::Unbind => {}\n                    Privilege::All => {\n                        return Err(DavError::Condition(DavErrorCondition::new(\n                            StatusCode::FORBIDDEN,\n                            BaseCondition::NoAbstract,\n                        )));\n                    }\n                    Privilege::ReadAcl => {}\n                    Privilege::WriteAcl => {\n                        acls.insert(Acl::Share);\n                    }\n                    Privilege::ReadFreeBusy\n                    | Privilege::ScheduleQueryFreeBusy\n                    | Privilege::ScheduleSendFreeBusy => {\n                        if collection == Collection::Calendar {\n                            acls.insert(Acl::SchedulingReadFreeBusy);\n                        } else {\n                            return Err(DavError::Condition(DavErrorCondition::new(\n                                StatusCode::FORBIDDEN,\n                                BaseCondition::NotSupportedPrivilege,\n                            )));\n                        }\n                    }\n                    Privilege::ScheduleDeliver | Privilege::ScheduleSend => {\n                        if collection == Collection::Calendar {\n                            acls.insert(Acl::SchedulingReadFreeBusy);\n                            acls.insert(Acl::SchedulingInvite);\n                            acls.insert(Acl::SchedulingReply);\n                        } else {\n                            return Err(DavError::Condition(DavErrorCondition::new(\n                                StatusCode::FORBIDDEN,\n                                BaseCondition::NotSupportedPrivilege,\n                            )));\n                        }\n                    }\n                    Privilege::ScheduleDeliverInvite | Privilege::ScheduleSendInvite => {\n                        if collection == Collection::Calendar {\n                            acls.insert(Acl::SchedulingInvite);\n                        } else {\n                            return Err(DavError::Condition(DavErrorCondition::new(\n                                StatusCode::FORBIDDEN,\n                                BaseCondition::NotSupportedPrivilege,\n                            )));\n                        }\n                    }\n                    Privilege::ScheduleDeliverReply | Privilege::ScheduleSendReply => {\n                        if collection == Collection::Calendar {\n                            acls.insert(Acl::SchedulingReply);\n                        } else {\n                            return Err(DavError::Condition(DavErrorCondition::new(\n                                StatusCode::FORBIDDEN,\n                                BaseCondition::NotSupportedPrivilege,\n                            )));\n                        }\n                    }\n                }\n            }\n\n            if acls.is_empty() {\n                continue;\n            }\n\n            let principal_id = self\n                .validate_uri(access_token, &principal_uri)\n                .await\n                .map_err(|_| {\n                    DavError::Condition(DavErrorCondition::new(\n                        StatusCode::FORBIDDEN,\n                        BaseCondition::AllowedPrincipal,\n                    ))\n                })?\n                .account_id\n                .ok_or_else(|| {\n                    DavError::Condition(DavErrorCondition::new(\n                        StatusCode::FORBIDDEN,\n                        BaseCondition::AllowedPrincipal,\n                    ))\n                })?;\n\n            // Verify that the principal is a valid principal\n            let principal = self\n                .directory()\n                .query(QueryParams::id(principal_id).with_return_member_of(false))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| {\n                    DavError::Condition(DavErrorCondition::new(\n                        StatusCode::FORBIDDEN,\n                        BaseCondition::AllowedPrincipal,\n                    ))\n                })?;\n            if !matches!(principal.typ(), Type::Individual | Type::Group) {\n                return Err(DavError::Condition(DavErrorCondition::new(\n                    StatusCode::FORBIDDEN,\n                    BaseCondition::AllowedPrincipal,\n                )));\n            }\n\n            grants.push(AclGrant {\n                account_id: principal_id,\n                grants: acls,\n            });\n        }\n\n        Ok(grants)\n    }\n\n    async fn resolve_ace(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        grants: &ArchivedVec<ArchivedAclGrant>,\n        expand: Option<&PropFind>,\n    ) -> crate::Result<Vec<Ace>> {\n        let mut aces = Vec::with_capacity(grants.len());\n        if access_token.is_member(account_id)\n            || grants.effective_acl(access_token).contains(Acl::Share)\n        {\n            for grant in grants.iter() {\n                let grant_account_id = u32::from(grant.account_id);\n                let principal = if let Some(expand) = expand {\n                    self.expand_principal(access_token, grant_account_id, expand)\n                        .await?\n                        .map(Principal::Response)\n                        .unwrap_or_else(|| {\n                            Principal::Href(Href(format!(\n                                \"{}/_{grant_account_id}/\",\n                                DavResourceName::Principal.base_path(),\n                            )))\n                        })\n                } else {\n                    let grant_account_name = self\n                        .store()\n                        .get_principal_name(grant_account_id)\n                        .await\n                        .caused_by(trc::location!())?\n                        .unwrap_or_else(|| format!(\"_{grant_account_id}\"));\n\n                    Principal::Href(Href(format!(\n                        \"{}/{}/\",\n                        DavResourceName::Principal.base_path(),\n                        percent_encoding::utf8_percent_encode(&grant_account_name, RFC_3986),\n                    )))\n                };\n\n                aces.push(Ace::new(\n                    principal,\n                    GrantDeny::grant(current_user_privilege_set(Bitmap::<Acl>::from(\n                        &grant.grants,\n                    ))),\n                ));\n            }\n        }\n\n        Ok(aces)\n    }\n}\n\nimpl ResourceAcl for DavResources {\n    fn validate_and_map_parent_acl(\n        &self,\n        access_token: &AccessToken,\n        is_member: bool,\n        parent_id: Option<u32>,\n        check_acls: impl Into<Bitmap<Acl>> + Send,\n    ) -> crate::Result<u32> {\n        match parent_id {\n            Some(parent_id) => {\n                if is_member || self.has_access_to_container(access_token, parent_id, check_acls) {\n                    Ok(parent_id + 1)\n                } else {\n                    Err(DavError::Code(StatusCode::FORBIDDEN))\n                }\n            }\n            None => {\n                if is_member {\n                    Ok(0)\n                } else {\n                    Err(DavError::Code(StatusCode::FORBIDDEN))\n                }\n            }\n        }\n    }\n}\n\npub(crate) trait Privileges {\n    fn current_privilege_set(\n        &self,\n        account_id: u32,\n        grants: &ArchivedVec<ArchivedAclGrant>,\n        is_calendar: bool,\n    ) -> Vec<Privilege>;\n}\n\nimpl Privileges for AccessToken {\n    fn current_privilege_set(\n        &self,\n        account_id: u32,\n        grants: &ArchivedVec<ArchivedAclGrant>,\n        is_calendar: bool,\n    ) -> Vec<Privilege> {\n        if self.is_member(account_id) {\n            Privilege::all(is_calendar)\n        } else {\n            current_user_privilege_set(grants.effective_acl(self))\n        }\n    }\n}\n\npub(crate) fn current_user_privilege_set(acl_bitmap: Bitmap<Acl>) -> Vec<Privilege> {\n    let mut acls = AHashSet::with_capacity(16);\n    for grant in acl_bitmap {\n        match grant {\n            Acl::Read | Acl::ReadItems => {\n                acls.insert(Privilege::Read);\n                acls.insert(Privilege::ReadCurrentUserPrivilegeSet);\n            }\n            Acl::Modify => {\n                acls.insert(Privilege::WriteProperties);\n            }\n            Acl::ModifyItems => {\n                acls.insert(Privilege::WriteContent);\n            }\n            Acl::Delete | Acl::RemoveItems => {\n                acls.insert(Privilege::Write);\n            }\n            Acl::Share => {\n                acls.insert(Privilege::ReadAcl);\n                acls.insert(Privilege::WriteAcl);\n            }\n            Acl::SchedulingReadFreeBusy => {\n                acls.insert(Privilege::ReadFreeBusy);\n            }\n            _ => {}\n        }\n    }\n    acls.into_iter().collect()\n}\n"
  },
  {
    "path": "crates/dav/src/common/lock.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ETag;\nuse super::uri::{DavUriResource, OwnedUri, UriResource, Urn};\nuse crate::{DavError, DavErrorCondition, DavMethod};\nuse common::KV_LOCK_DAV;\nuse common::{Server, auth::AccessToken};\nuse dav_proto::schema::property::{ActiveLock, LockScope, WebDavProperty};\nuse dav_proto::schema::request::DavPropertyValue;\nuse dav_proto::schema::response::{BaseCondition, List, PropResponse};\nuse dav_proto::{Condition, Depth, Timeout};\nuse dav_proto::{RequestHeaders, schema::request::LockInfo};\nuse groupware::cache::GroupwareCache;\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::collections::HashMap;\nuse store::ValueKey;\nuse store::dispatch::lookup::KeyValue;\nuse store::write::serialize::rkyv_deserialize;\nuse store::write::{AlignedBytes, Archive, Archiver, now};\nuse store::{Serialize, U32_LEN};\nuse trc::AddContext;\nuse types::collection::Collection;\nuse types::dead_property::DeadProperty;\n\n#[derive(Debug, Default, Clone)]\npub struct ResourceState<'x> {\n    pub account_id: u32,\n    pub collection: Collection,\n    pub document_id: Option<u32>,\n    pub etag: Option<String>,\n    pub lock_tokens: Vec<String>,\n    pub sync_token: Option<String>,\n    pub path: &'x str,\n}\n\n#[derive(Debug, Default, Clone, rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)]\npub(crate) struct LockData {\n    locks: HashMap<String, LockItems>,\n}\n\n#[derive(Debug, Default, Clone, rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)]\n#[repr(transparent)]\npub(crate) struct LockItems(Vec<LockItem>);\n\n#[derive(Debug, Default, Clone, rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)]\npub(crate) struct LockItem {\n    lock_id: u64,\n    owner: u32,\n    expires: u64,\n    depth_infinity: bool,\n    exclusive: bool,\n    owner_dav: Option<DeadProperty>,\n}\n\nstruct LockCache<'x> {\n    account_id: u32,\n    collection: Collection,\n    lock_archive: LockArchive<'x>,\n}\n\nenum LockArchive<'x> {\n    Unarchived(&'x ArchivedLockData),\n    Archived(Archive<AlignedBytes>),\n}\n\n#[derive(Default)]\npub(crate) struct LockCaches<'x> {\n    caches: Vec<LockCache<'x>>,\n}\n\npub(crate) trait LockRequestHandler: Sync + Send {\n    fn handle_lock_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        lock_info: LockRequest,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn validate_headers(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        resources: Vec<ResourceState<'_>>,\n        locks: LockCaches<'_>,\n        method: DavMethod,\n    ) -> impl Future<Output = crate::Result<()>> + Send;\n}\n\npub(crate) enum LockRequest {\n    Lock(LockInfo),\n    Unlock,\n    Refresh,\n}\n\nimpl LockRequestHandler for Server {\n    async fn handle_lock_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        lock_info: LockRequest,\n    ) -> crate::Result<HttpResponse> {\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let resource_hash = resource.lock_key();\n        let resource_path = resource\n            .resource\n            .ok_or(DavError::Code(StatusCode::CONFLICT))?;\n        let account_id = resource.account_id;\n        if !access_token.is_member(account_id) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        let resources = vec![ResourceState {\n            account_id,\n            collection: resource.collection,\n            path: resource_path,\n            ..Default::default()\n        }];\n\n        let mut base_path = None;\n        let is_lock_request = !matches!(lock_info, LockRequest::Unlock);\n        let if_lock_token = headers\n            .if_\n            .iter()\n            .flat_map(|if_| if_.list.iter())\n            .find_map(|cond| {\n                if let Condition::StateToken { token, .. } = cond {\n                    Urn::parse(token).and_then(|u| u.try_unwrap_lock())\n                } else {\n                    None\n                }\n            })\n            .unwrap_or_default();\n        let mut lock_data = if let Some(lock_data) = self\n            .in_memory_store()\n            .key_get::<Archive<AlignedBytes>>(resource_hash.as_slice())\n            .await\n            .caused_by(trc::location!())?\n        {\n            let lock_data = lock_data\n                .unarchive::<LockData>()\n                .caused_by(trc::location!())?;\n\n            self.validate_headers(\n                access_token,\n                headers,\n                resources,\n                LockCaches::new_shared(account_id, resource.collection, lock_data),\n                if is_lock_request {\n                    DavMethod::LOCK\n                } else {\n                    DavMethod::UNLOCK\n                },\n            )\n            .await?;\n\n            if let LockRequest::Lock(lock_info) = &lock_info {\n                let mut failed_locks = Vec::new();\n                let is_exclusive = matches!(lock_info.lock_scope, LockScope::Exclusive);\n                let is_infinity = matches!(headers.depth, Depth::Infinity);\n\n                for (lock_path, lock_item) in lock_data.find_locks(resource_path, true) {\n                    if if_lock_token != lock_item.lock_id\n                        && (lock_item.exclusive || is_exclusive)\n                        && (lock_path.len() == resource_path.len()\n                            || lock_item.depth_infinity && resource_path.len() > lock_path.len()\n                            || is_infinity && lock_path.len() > resource_path.len())\n                    {\n                        let base_path =\n                            base_path.get_or_insert_with(|| headers.base_uri().unwrap_or_default());\n                        failed_locks.push(format!(\"{base_path}/{lock_path}\").into());\n                    }\n                }\n\n                if !failed_locks.is_empty() {\n                    return Err(DavErrorCondition::new(\n                        StatusCode::LOCKED,\n                        BaseCondition::LockTokenSubmitted(List(failed_locks)),\n                    )\n                    .into());\n                }\n\n                // Validate lock_info\n                if lock_info.owner.as_ref().is_some_and(|o| {\n                    o.size() > self.core.groupware.dead_property_size.unwrap_or(512)\n                }) {\n                    return Err(DavError::Code(StatusCode::PAYLOAD_TOO_LARGE));\n                }\n\n                if self.core.groupware.max_locks_per_user > 0\n                    && lock_data\n                        .locks\n                        .values()\n                        .flat_map(|locks| {\n                            locks\n                                .0\n                                .iter()\n                                .filter(|lock| lock.owner == access_token.primary_id)\n                        })\n                        .count()\n                        >= self.core.groupware.max_locks_per_user\n                {\n                    return Err(DavError::Code(StatusCode::TOO_MANY_REQUESTS));\n                }\n            }\n\n            rkyv_deserialize(lock_data).caused_by(trc::location!())?\n        } else if is_lock_request {\n            self.validate_headers(\n                access_token,\n                headers,\n                resources,\n                Default::default(),\n                DavMethod::LOCK,\n            )\n            .await?;\n\n            LockData::default()\n        } else {\n            return Err(DavErrorCondition::new(\n                StatusCode::CONFLICT,\n                BaseCondition::LockTokenMatchesRequestUri,\n            )\n            .into());\n        };\n\n        let now = now();\n        let response = if is_lock_request {\n            let timeout = if let Timeout::Second(seconds) = headers.timeout {\n                std::cmp::min(seconds, self.core.groupware.max_lock_timeout)\n            } else {\n                self.core.groupware.max_lock_timeout\n            };\n            let expires = now + timeout;\n\n            let lock_item = if if_lock_token > 0 {\n                if let Some(lock_item) = lock_data\n                    .locks\n                    .values_mut()\n                    .flat_map(|locks| locks.0.iter_mut())\n                    .find(|lock| lock.lock_id == if_lock_token)\n                {\n                    lock_item\n                } else {\n                    return Err(DavError::Code(StatusCode::PRECONDITION_FAILED));\n                }\n            } else {\n                let locks = lock_data\n                    .locks\n                    .entry(resource_path.to_string())\n                    .or_insert_with(Default::default);\n                locks.0.push(LockItem::default());\n                locks.0.last_mut().unwrap()\n            };\n\n            lock_item.expires = expires;\n            if let LockRequest::Lock(lock_info) = lock_info {\n                // Validate lock_info\n                if lock_info.owner.as_ref().is_some_and(|o| {\n                    o.size() > self.core.groupware.dead_property_size.unwrap_or(512)\n                }) {\n                    return Err(DavError::Code(StatusCode::PAYLOAD_TOO_LARGE));\n                }\n\n                lock_item.lock_id = store::rand::random::<u64>() ^ expires;\n                lock_item.owner = access_token.primary_id;\n                lock_item.depth_infinity = matches!(headers.depth, Depth::Infinity);\n                lock_item.owner_dav = lock_info.owner;\n                lock_item.exclusive = matches!(lock_info.lock_scope, LockScope::Exclusive);\n            }\n\n            let base_path = base_path.get_or_insert_with(|| headers.base_uri().unwrap_or_default());\n            let active_lock = lock_item.to_active_lock(format!(\"{base_path}/{resource_path}\"));\n\n            HttpResponse::new(if if_lock_token == 0 {\n                StatusCode::CREATED\n            } else {\n                StatusCode::OK\n            })\n            .with_lock_token(&active_lock.lock_token.as_ref().unwrap().0)\n            .with_xml_body(\n                PropResponse::new(vec![DavPropertyValue::new(\n                    WebDavProperty::LockDiscovery,\n                    vec![active_lock],\n                )])\n                .to_string(),\n            )\n        } else {\n            let lock_id = headers\n                .lock_token\n                .and_then(Urn::parse)\n                .and_then(|urn| urn.try_unwrap_lock())\n                .ok_or(DavError::Code(StatusCode::BAD_REQUEST))?;\n\n            if lock_data.remove_lock(lock_id) {\n                HttpResponse::new(StatusCode::NO_CONTENT)\n            } else {\n                return Err(DavErrorCondition::new(\n                    StatusCode::CONFLICT,\n                    BaseCondition::LockTokenMatchesRequestUri,\n                )\n                .into());\n            }\n        };\n\n        // Remove expired locks\n        let max_expire = lock_data.remove_expired();\n        if max_expire > 0 {\n            self.in_memory_store()\n                .key_set(\n                    KeyValue::new(\n                        resource_hash,\n                        Archiver::new(lock_data)\n                            .untrusted()\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    )\n                    .expires(max_expire),\n                )\n                .await\n                .caused_by(trc::location!())?;\n        } else {\n            self.in_memory_store()\n                .key_delete(resource_hash)\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(response)\n    }\n\n    async fn validate_headers(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        mut resources: Vec<ResourceState<'_>>,\n        mut locks_: LockCaches<'_>,\n        method: DavMethod,\n    ) -> crate::Result<()> {\n        let no_if_headers = headers.if_.is_empty();\n        match method {\n            DavMethod::GET | DavMethod::HEAD => {\n                // Return early for GET/HEAD requests without If headers\n                if no_if_headers {\n                    return Ok(());\n                }\n            }\n            DavMethod::COPY\n            | DavMethod::MOVE\n            | DavMethod::POST\n            | DavMethod::PUT\n            | DavMethod::PATCH => {\n                if headers.overwrite_fail\n                    && resources.last().is_some_and(|r| {\n                        r.etag.is_some() || r.document_id.is_some_and(|id| id != u32::MAX)\n                    })\n                {\n                    return Err(DavError::Code(StatusCode::PRECONDITION_FAILED));\n                }\n            }\n            _ => {}\n        }\n\n        // Add lock data to the cache\n        for resource in &resources {\n            if locks_.is_cached(resource).is_none() {\n                locks_.insert_lock_data(self, resource).await?;\n            }\n        }\n\n        // Unarchive lock data\n        let mut locks = locks_.to_unarchived().caused_by(trc::location!())?;\n\n        // Validate locks for write operations\n        let mut lock_response = Ok(());\n        if !matches!(\n            method,\n            DavMethod::GET | DavMethod::HEAD | DavMethod::LOCK | DavMethod::UNLOCK\n        ) {\n            let mut base_path = None;\n\n            'outer: for (pos, resource) in resources.iter().enumerate() {\n                if pos == 0 && matches!(method, DavMethod::COPY) {\n                    continue;\n                }\n\n                if let Some(idx) = locks.find_cache_pos(self, resource).await? {\n                    let mut failed_locks = Vec::new();\n\n                    for (lock_path, lock_item) in locks.find_locks_by_pos(idx, resource, true)? {\n                        let lock_token = lock_item.urn().to_string();\n                        if headers.if_.iter().any(|if_| {\n                            if_.resource\n                                .is_none_or(|r| {\n                                    r.trim_end_matches('/').ends_with(lock_path)})\n                                && if_.list.iter().any(|cond| matches!(cond, Condition::StateToken { token, .. } if token == &lock_token))\n                        }) {\n                            break 'outer;\n                        } else {\n                            let base_path = base_path.get_or_insert_with(|| {\n                                headers.base_uri()\n                                    .unwrap_or_default()\n                            });\n                            failed_locks.push(format!(\"{base_path}/{lock_path}\").into());\n                        }\n                    }\n\n                    if !failed_locks.is_empty() {\n                        lock_response = Err(DavErrorCondition::new(\n                            StatusCode::LOCKED,\n                            BaseCondition::LockTokenSubmitted(List(failed_locks)),\n                        )\n                        .into());\n                        break;\n                    }\n                }\n            }\n        }\n\n        // There are no If headers, so we can return early\n        if no_if_headers {\n            return lock_response;\n        }\n\n        let mut resource_not_found = ResourceState {\n            account_id: u32::MAX,\n            collection: Collection::None,\n            path: \"\",\n            ..Default::default()\n        };\n\n        'outer: for if_ in &headers.if_ {\n            if if_.list.is_empty() {\n                continue;\n            }\n\n            let mut resource_state = &mut resource_not_found;\n\n            if let Some(resource) = if_.resource {\n                if let Some(resource) = self\n                    .validate_uri(access_token, resource)\n                    .await\n                    .ok()\n                    .and_then(|r| {\n                        let path = r.resource?;\n\n                        Some(ResourceState {\n                            account_id: r.account_id?,\n                            collection: if !matches!(r.collection, Collection::FileNode)\n                                && path.contains('/')\n                            {\n                                r.collection.child_collection().unwrap_or(r.collection)\n                            } else {\n                                r.collection\n                            },\n                            path,\n                            ..Default::default()\n                        })\n                    })\n                {\n                    if let Some(known_resource) = resources.iter_mut().find(|r| {\n                        r.account_id == resource.account_id\n                            && r.collection == resource.collection\n                            && r.path == resource.path\n                    }) {\n                        resource_state = known_resource;\n                    } else if access_token.has_access(resource.account_id, resource.collection) {\n                        resources.push(resource);\n                        resource_state = resources.last_mut().unwrap();\n                    }\n                }\n            } else if let Some(resource) = resources.first_mut() {\n                resource_state = resource;\n            };\n\n            // Fill missing data for resource\n            if resource_state.collection != Collection::None\n                && (resource_state.etag.is_none()\n                    || resource_state.lock_tokens.is_empty()\n                    || resource_state.sync_token.is_none())\n            {\n                let mut needs_lock_token = false;\n                let mut needs_sync_token = false;\n                let mut needs_etag = false;\n\n                for cond in &if_.list {\n                    match cond {\n                        Condition::StateToken { token, .. } => {\n                            if token.starts_with(\"urn:stalwart:davsync:\") {\n                                needs_sync_token = true;\n                            } else {\n                                needs_lock_token = true;\n                            }\n                        }\n                        Condition::ETag { .. } | Condition::Exists { .. } => {\n                            needs_etag = true;\n                        }\n                    }\n                }\n\n                // Fetch eTag\n                if needs_etag && resource_state.etag.is_none() {\n                    if resource_state.document_id.is_none() {\n                        resource_state.document_id = self\n                            .map_uri_resource(\n                                access_token,\n                                UriResource {\n                                    collection: resource_state.collection,\n                                    account_id: resource_state.account_id,\n                                    resource: resource_state.path.into(),\n                                },\n                            )\n                            .await\n                            .caused_by(trc::location!())?\n                            .map(|uri| uri.resource)\n                            .unwrap_or(u32::MAX)\n                            .into();\n                    }\n\n                    if let Some(document_id) =\n                        resource_state.document_id.filter(|&id| id != u32::MAX)\n                        && let Some(archive) = self\n                            .store()\n                            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                                resource_state.account_id,\n                                resource_state.collection,\n                                document_id,\n                            ))\n                            .await\n                            .caused_by(trc::location!())?\n                    {\n                        resource_state.etag = archive.etag().into();\n                    }\n                }\n\n                // Fetch lock token\n                if needs_lock_token\n                    && resource_state.lock_tokens.is_empty()\n                    && let Some(idx) = locks.find_cache_pos(self, resource_state).await?\n                {\n                    let found_locks = locks\n                        .find_locks_by_pos(idx, resource_state, false)?\n                        .iter()\n                        .map(|(_, lock)| lock.urn().to_string())\n                        .collect::<Vec<_>>();\n                    resource_state.lock_tokens = found_locks;\n                }\n\n                // Fetch sync token\n                if needs_sync_token && resource_state.sync_token.is_none() {\n                    let id = self\n                        .fetch_dav_resources(\n                            access_token,\n                            resource_state.account_id,\n                            resource_state.collection.into(),\n                        )\n                        .await\n                        .caused_by(trc::location!())?\n                        .highest_change_id;\n                    resource_state.sync_token = Some(Urn::Sync { id, seq: 0 }.to_string());\n                }\n            }\n\n            for cond in &if_.list {\n                match cond {\n                    Condition::StateToken { is_not, token } => {\n                        if let Some(token) = Urn::try_extract_sync_id(token) {\n                            if !((resource_state\n                                .sync_token\n                                .as_deref()\n                                .and_then(Urn::try_extract_sync_id)\n                                .is_some_and(|sync_token| sync_token == token))\n                                ^ is_not)\n                            {\n                                continue 'outer;\n                            }\n                        } else if !((resource_state.lock_tokens.iter().any(|t| t == token))\n                            ^ is_not)\n                        {\n                            continue 'outer;\n                        }\n                    }\n                    Condition::ETag { is_not, tag } => {\n                        if !((resource_state.etag.as_ref().is_some_and(|etag| etag == tag))\n                            ^ is_not)\n                        {\n                            continue 'outer;\n                        }\n                    }\n                    Condition::Exists { is_not } => {\n                        if !((resource_state.etag.is_some()) ^ is_not) {\n                            continue 'outer;\n                        }\n                    }\n                }\n            }\n\n            return lock_response;\n        }\n\n        Err(DavError::Code(\n            if matches!(method, DavMethod::GET | DavMethod::HEAD)\n                && headers\n                    .if_\n                    .iter()\n                    .any(|if_| if_.list.iter().any(|cond| cond.is_none_match()))\n            {\n                StatusCode::NOT_MODIFIED\n            } else {\n                StatusCode::PRECONDITION_FAILED\n            },\n        ))\n    }\n}\n\nimpl LockData {\n    pub fn remove_lock(&mut self, lock_id: u64) -> bool {\n        for (lock_path, lock_items) in self.locks.iter_mut() {\n            for (idx, lock_item) in lock_items.0.iter().enumerate() {\n                if lock_item.lock_id == lock_id {\n                    lock_items.0.swap_remove(idx);\n                    if lock_items.0.is_empty() {\n                        let lock_path = lock_path.clone();\n                        self.locks.remove(&lock_path);\n                    }\n                    return true;\n                }\n            }\n        }\n\n        false\n    }\n\n    pub fn remove_expired(&mut self) -> u64 {\n        let mut max_expire = 0;\n        let now = now();\n\n        self.locks.retain(|_, locks| {\n            locks.0.retain(|lock| {\n                if lock.expires > now {\n                    max_expire = std::cmp::max(max_expire, lock.expires);\n                    true\n                } else {\n                    false\n                }\n            });\n\n            !locks.0.is_empty()\n        });\n\n        max_expire\n    }\n}\n\nimpl<'x> LockArchive<'x> {\n    fn unarchive(&'x self) -> trc::Result<&'x ArchivedLockData> {\n        match self {\n            LockArchive::Unarchived(archived_lock_data) => Ok(archived_lock_data),\n            LockArchive::Archived(archive) => {\n                archive.unarchive::<LockData>().caused_by(trc::location!())\n            }\n        }\n    }\n}\n\nimpl<'x> LockCaches<'x> {\n    pub(self) fn new_shared(\n        account_id: u32,\n        collection: Collection,\n        lock_data: &'x ArchivedLockData,\n    ) -> Self {\n        Self {\n            caches: vec![LockCache {\n                account_id,\n                collection,\n                lock_archive: LockArchive::Unarchived(lock_data),\n            }],\n        }\n    }\n\n    pub fn to_unarchived(&'x self) -> trc::Result<LockCaches<'x>> {\n        let caches = self\n            .caches\n            .iter()\n            .map(|cache| {\n                Ok(LockCache {\n                    account_id: cache.account_id,\n                    collection: cache.collection,\n                    lock_archive: LockArchive::Unarchived(\n                        cache.lock_archive.unarchive().caused_by(trc::location!())?,\n                    ),\n                })\n            })\n            .collect::<trc::Result<Vec<_>>>()?;\n\n        Ok(LockCaches { caches })\n    }\n\n    #[inline]\n    pub fn is_cached(&self, resource_state: &ResourceState<'_>) -> Option<usize> {\n        self.caches.iter().position(|cache| {\n            resource_state.account_id == cache.account_id\n                && resource_state.collection.main_collection() == cache.collection.main_collection()\n        })\n    }\n\n    pub async fn find_cache_pos(\n        &mut self,\n        server: &Server,\n        resource_state: &ResourceState<'_>,\n    ) -> trc::Result<Option<usize>> {\n        if let Some(idx) = self.is_cached(resource_state) {\n            Ok(Some(idx))\n        } else if resource_state.collection != Collection::None {\n            if self.insert_lock_data(server, resource_state).await? {\n                Ok(Some(self.caches.len() - 1))\n            } else {\n                Ok(None)\n            }\n        } else {\n            Ok(None)\n        }\n    }\n\n    fn find_locks_by_pos(\n        &'x self,\n        pos: usize,\n        resource_state: &'x ResourceState<'_>,\n        include_children: bool,\n    ) -> trc::Result<Vec<(&'x str, &'x ArchivedLockItem)>> {\n        self.caches[pos]\n            .lock_archive\n            .unarchive()\n            .map(|l| l.find_locks(resource_state.path, include_children))\n    }\n\n    async fn insert_lock_data(\n        &mut self,\n        server: &Server,\n        resource_state: &ResourceState<'_>,\n    ) -> trc::Result<bool> {\n        if let Some(lock_archive) = server\n            .in_memory_store()\n            .key_get::<Archive<AlignedBytes>>(resource_state.lock_key().as_slice())\n            .await\n            .caused_by(trc::location!())?\n        {\n            self.caches.push(LockCache {\n                account_id: resource_state.account_id,\n                collection: resource_state.collection,\n                lock_archive: LockArchive::Archived(lock_archive),\n            });\n\n            Ok(true)\n        } else {\n            Ok(false)\n        }\n    }\n}\n\nimpl LockItem {\n    pub fn to_active_lock(&self, href: String) -> ActiveLock {\n        ActiveLock::new(\n            href,\n            if self.exclusive {\n                LockScope::Exclusive\n            } else {\n                LockScope::Shared\n            },\n        )\n        .with_depth(if self.depth_infinity {\n            Depth::Infinity\n        } else {\n            Depth::Zero\n        })\n        .with_owner_opt(self.owner_dav.clone())\n        .with_timeout(self.expires.saturating_sub(now()))\n        .with_lock_token(self.urn().to_string())\n    }\n\n    pub fn urn(&self) -> Urn {\n        Urn::Lock(self.lock_id)\n    }\n}\n\nimpl ArchivedLockData {\n    pub fn find_locks<'x: 'y, 'y>(\n        &'x self,\n        resource: &'y str,\n        include_children: bool,\n    ) -> Vec<(&'y str, &'x ArchivedLockItem)> {\n        let now = now();\n        let mut resource_part = resource;\n        let mut found_locks = Vec::new();\n\n        loop {\n            if let Some(locks) = self.locks.get(resource_part) {\n                found_locks.extend(\n                    locks\n                        .0\n                        .iter()\n                        .filter(|lock| {\n                            lock.expires > now && (resource == resource_part || lock.depth_infinity)\n                        })\n                        .map(|lock| (resource_part, lock)),\n                );\n            }\n\n            if let Some((resource_part_, _)) = resource_part.rsplit_once('/') {\n                resource_part = resource_part_;\n            } else {\n                break;\n            }\n        }\n\n        if include_children {\n            let prefix = format!(\"{}/\", resource);\n            for (resource_part, locks) in self.locks.iter() {\n                if resource_part.starts_with(&prefix) {\n                    found_locks.extend(\n                        locks\n                            .0\n                            .iter()\n                            .filter(|lock| lock.expires > now)\n                            .map(|lock| (resource_part.as_str(), lock)),\n                    );\n                }\n            }\n        }\n\n        found_locks\n    }\n}\n\nimpl ArchivedLockItem {\n    pub fn to_active_lock(&self, href: String) -> ActiveLock {\n        ActiveLock::new(\n            href,\n            if self.exclusive {\n                LockScope::Exclusive\n            } else {\n                LockScope::Shared\n            },\n        )\n        .with_depth(if self.depth_infinity {\n            Depth::Infinity\n        } else {\n            Depth::Zero\n        })\n        .with_owner_opt(self.owner_dav.as_ref().map(Into::into))\n        .with_timeout(u64::from(self.expires).saturating_sub(now()))\n        .with_lock_token(self.urn().to_string())\n    }\n\n    pub fn urn(&self) -> Urn {\n        Urn::Lock(self.lock_id.into())\n    }\n}\n\nimpl OwnedUri<'_> {\n    pub fn lock_key(&self) -> Vec<u8> {\n        build_lock_key(self.account_id, self.collection.main_collection())\n    }\n}\n\nimpl ResourceState<'_> {\n    pub fn lock_key(&self) -> Vec<u8> {\n        build_lock_key(self.account_id, self.collection.main_collection())\n    }\n}\n\npub(crate) fn build_lock_key(account_id: u32, collection: Collection) -> Vec<u8> {\n    let mut result = Vec::with_capacity(U32_LEN + 2);\n    result.push(KV_LOCK_DAV);\n    result.extend_from_slice(account_id.to_be_bytes().as_slice());\n    result.push(u8::from(collection));\n    result\n}\n\nimpl PartialEq for ResourceState<'_> {\n    fn eq(&self, other: &Self) -> bool {\n        self.account_id == other.account_id\n            && self.collection == other.collection\n            && self.document_id == other.document_id\n    }\n}\n\nimpl Eq for ResourceState<'_> {}\n"
  },
  {
    "path": "crates/dav/src/common/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse calcard::{\n    icalendar::{ICalendarComponentType, ICalendarParameterName, ICalendarProperty},\n    vcard::{VCardParameterName, VCardVersion},\n};\nuse common::auth::AccessToken;\nuse dav_proto::{\n    Depth, RequestHeaders, Return,\n    schema::{\n        Namespace,\n        property::{DavProperty, ReportSet, ResourceType},\n        request::{\n            AddressbookQuery, CalendarQuery, ExpandProperty, Filter, MultiGet, PropFind,\n            SyncCollection, Timezone, VCardPropertyWithGroup,\n        },\n    },\n};\nuse groupware::{\n    calendar::{\n        ArchivedCalendar, ArchivedCalendarEvent, ArchivedCalendarEventNotification, Calendar,\n        CalendarEvent, CalendarEventNotification,\n    },\n    contact::{AddressBook, ArchivedAddressBook, ArchivedContactCard, ContactCard},\n    file::{ArchivedFileNode, FileNode},\n};\nuse propfind::PropFindItem;\nuse rkyv::vec::ArchivedVec;\nuse store::write::{AlignedBytes, Archive, BatchBuilder, Operation, ValueClass, ValueOp};\nuse types::{\n    TimeRange, acl::ArchivedAclGrant, collection::Collection, dead_property::ArchivedDeadProperty,\n    field::Field,\n};\nuse uri::{OwnedUri, Urn};\n\npub mod acl;\npub mod lock;\npub mod propfind;\npub mod uri;\n\n#[derive(Debug)]\npub(crate) struct DavQuery<'x> {\n    pub uri: &'x str,\n    pub resource: DavQueryResource<'x>,\n    pub propfind: PropFind,\n    pub sync_type: SyncType,\n    pub depth: usize,\n    pub limit: Option<u32>,\n    pub max_vcard_version: Option<VCardVersion>,\n    pub ret: Return,\n    pub depth_no_root: bool,\n    pub expand: bool,\n}\n\n#[derive(Default, Debug)]\npub(crate) enum SyncType {\n    #[default]\n    None,\n    Initial,\n    From {\n        id: u64,\n        seq: u32,\n    },\n}\n\n#[derive(Default, Debug)]\npub(crate) enum DavQueryResource<'x> {\n    Uri(OwnedUri<'x>),\n    Multiget {\n        parent_collection: Collection,\n        hrefs: Vec<String>,\n    },\n    Query {\n        filter: DavQueryFilter,\n        parent_collection: Collection,\n        items: Vec<PropFindItem>,\n    },\n    #[default]\n    None,\n}\n\npub(crate) type AddressbookFilter = Vec<Filter<(), VCardPropertyWithGroup, VCardParameterName>>;\npub(crate) type CalendarFilter =\n    Vec<Filter<Vec<ICalendarComponentType>, ICalendarProperty, ICalendarParameterName>>;\n\n#[derive(Debug)]\npub(crate) enum DavQueryFilter {\n    Addressbook(AddressbookFilter),\n    Calendar {\n        filter: CalendarFilter,\n        max_time_range: Option<TimeRange>,\n        timezone: Timezone,\n    },\n}\n\npub(crate) trait ETag {\n    fn etag(&self) -> String;\n}\n\npub(crate) trait ExtractETag {\n    fn etag(&self) -> Option<String>;\n}\n\nimpl<T> ETag for Archive<T> {\n    fn etag(&self) -> String {\n        format!(\"\\\"{}\\\"\", self.version.hash().unwrap_or_default())\n    }\n}\n\nimpl ExtractETag for BatchBuilder {\n    fn etag(&self) -> Option<String> {\n        let p_value = u8::from(Field::ARCHIVE);\n        for op in self.ops().iter().rev() {\n            match op {\n                Operation::Value {\n                    class: ValueClass::Property(p_id),\n                    op: ValueOp::Set(value),\n                } if *p_id == p_value => {\n                    return Archive::<AlignedBytes>::extract_hash(value)\n                        .map(|hash| format!(\"\\\"{}\\\"\", hash));\n                }\n                Operation::Value {\n                    class: ValueClass::Property(p_id),\n                    op: ValueOp::SetFnc(set_fnc),\n                } if *p_id == p_value => {\n                    return Archive::<AlignedBytes>::extract_hash(set_fnc.params().bytes(0))\n                        .map(|hash| format!(\"\\\"{}\\\"\", hash));\n                }\n                _ => {}\n            }\n        }\n\n        None\n    }\n}\n\npub(crate) trait DavCollection {\n    fn namespace(&self) -> Namespace;\n}\n\nimpl DavCollection for Collection {\n    fn namespace(&self) -> Namespace {\n        match self {\n            Collection::Calendar\n            | Collection::CalendarEvent\n            | Collection::CalendarEventNotification => Namespace::CalDav,\n            Collection::AddressBook | Collection::ContactCard => Namespace::CardDav,\n            _ => Namespace::Dav,\n        }\n    }\n}\n\nimpl<'x> DavQuery<'x> {\n    pub fn propfind(\n        resource: OwnedUri<'x>,\n        propfind: PropFind,\n        headers: &RequestHeaders<'x>,\n    ) -> Self {\n        Self {\n            resource: DavQueryResource::Uri(resource),\n            propfind,\n            depth: match headers.depth {\n                Depth::Zero => 0,\n                _ => 1,\n            },\n            ret: headers.ret,\n            depth_no_root: headers.depth_no_root,\n            uri: headers.uri,\n            max_vcard_version: headers.max_vcard_version,\n            sync_type: Default::default(),\n            limit: Default::default(),\n            expand: Default::default(),\n        }\n    }\n\n    pub fn multiget(\n        multiget: MultiGet,\n        collection: Collection,\n        headers: &RequestHeaders<'x>,\n    ) -> Self {\n        Self {\n            resource: DavQueryResource::Multiget {\n                hrefs: multiget.hrefs,\n                parent_collection: collection,\n            },\n            propfind: multiget.properties,\n            ret: headers.ret,\n            depth_no_root: headers.depth_no_root,\n            uri: headers.uri,\n            max_vcard_version: headers.max_vcard_version,\n            sync_type: Default::default(),\n            depth: Default::default(),\n            limit: Default::default(),\n            expand: Default::default(),\n        }\n    }\n\n    pub fn addressbook_query(\n        query: AddressbookQuery,\n        items: Vec<PropFindItem>,\n        headers: &RequestHeaders<'x>,\n    ) -> Self {\n        Self {\n            resource: DavQueryResource::Query {\n                filter: DavQueryFilter::Addressbook(query.filters),\n                parent_collection: Collection::AddressBook,\n                items,\n            },\n            propfind: query.properties,\n            limit: query.limit,\n            ret: headers.ret,\n            depth_no_root: headers.depth_no_root,\n            uri: headers.uri,\n            max_vcard_version: headers.max_vcard_version,\n            sync_type: Default::default(),\n            depth: Default::default(),\n            expand: Default::default(),\n        }\n    }\n\n    pub fn calendar_query(\n        query: CalendarQuery,\n        max_time_range: Option<TimeRange>,\n        items: Vec<PropFindItem>,\n        headers: &RequestHeaders<'x>,\n    ) -> Self {\n        Self {\n            resource: DavQueryResource::Query {\n                filter: DavQueryFilter::Calendar {\n                    filter: query.filters,\n                    timezone: query.timezone,\n                    max_time_range,\n                },\n                parent_collection: Collection::Calendar,\n                items,\n            },\n            propfind: query.properties,\n            ret: headers.ret,\n            depth_no_root: headers.depth_no_root,\n            uri: headers.uri,\n            sync_type: Default::default(),\n            depth: Default::default(),\n            limit: Default::default(),\n            max_vcard_version: Default::default(),\n            expand: Default::default(),\n        }\n    }\n\n    pub fn changes(\n        resource: OwnedUri<'x>,\n        changes: SyncCollection,\n        headers: &RequestHeaders<'x>,\n    ) -> Self {\n        Self {\n            resource: DavQueryResource::Uri(resource),\n            propfind: changes.properties,\n            sync_type: changes\n                .sync_token\n                .as_deref()\n                .and_then(Urn::parse)\n                .and_then(|urn| urn.try_unwrap_sync())\n                .map(|(id, seq)| SyncType::From { id, seq })\n                .unwrap_or(SyncType::Initial),\n            depth: match changes.depth {\n                Depth::One => 1,\n                Depth::Infinity => usize::MAX,\n                _ => 0,\n            },\n            limit: changes.limit,\n            ret: headers.ret,\n            depth_no_root: headers.depth_no_root,\n            expand: false,\n            uri: headers.uri,\n            max_vcard_version: headers.max_vcard_version,\n        }\n    }\n\n    pub fn expand(\n        resource: OwnedUri<'x>,\n        expand: ExpandProperty,\n        headers: &RequestHeaders<'x>,\n    ) -> Self {\n        let mut props = Vec::with_capacity(expand.properties.len());\n        for item in expand.properties {\n            if !matches!(item.property, DavProperty::DeadProperty(_))\n                && !props.contains(&item.property)\n            {\n                props.push(item.property);\n            }\n        }\n\n        Self {\n            resource: DavQueryResource::Uri(resource),\n            propfind: PropFind::Prop(props),\n            depth: match headers.depth {\n                Depth::Zero => 0,\n                _ => 1,\n            },\n            ret: headers.ret,\n            depth_no_root: headers.depth_no_root,\n            expand: true,\n            uri: headers.uri,\n            sync_type: Default::default(),\n            limit: Default::default(),\n            max_vcard_version: headers.max_vcard_version,\n        }\n    }\n\n    pub fn is_minimal(&self) -> bool {\n        self.ret == Return::Minimal\n    }\n}\n\npub(crate) enum ArchivedResource<'x> {\n    Calendar(Archive<&'x ArchivedCalendar>),\n    CalendarEvent(Archive<&'x ArchivedCalendarEvent>),\n    CalendarEventNotification(Archive<&'x ArchivedCalendarEventNotification>),\n    CalendarEventNotificationCollection(bool),\n    AddressBook(Archive<&'x ArchivedAddressBook>),\n    ContactCard(Archive<&'x ArchivedContactCard>),\n    FileNode(Archive<&'x ArchivedFileNode>),\n}\n\nimpl<'x> ArchivedResource<'x> {\n    pub fn from_archive(\n        archive: &'x Archive<AlignedBytes>,\n        collection: Collection,\n    ) -> trc::Result<Self> {\n        match collection {\n            Collection::Calendar => archive\n                .to_unarchived::<Calendar>()\n                .map(ArchivedResource::Calendar),\n            Collection::CalendarEvent => archive\n                .to_unarchived::<CalendarEvent>()\n                .map(ArchivedResource::CalendarEvent),\n            Collection::CalendarEventNotification => archive\n                .to_unarchived::<CalendarEventNotification>()\n                .map(ArchivedResource::CalendarEventNotification),\n            Collection::AddressBook => archive\n                .to_unarchived::<AddressBook>()\n                .map(ArchivedResource::AddressBook),\n            Collection::FileNode => archive\n                .to_unarchived::<FileNode>()\n                .map(ArchivedResource::FileNode),\n            Collection::ContactCard => archive\n                .to_unarchived::<ContactCard>()\n                .map(ArchivedResource::ContactCard),\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn acls(&self) -> Option<&ArchivedVec<ArchivedAclGrant>> {\n        match self {\n            Self::Calendar(archive) => Some(&archive.inner.acls),\n            Self::AddressBook(archive) => Some(&archive.inner.acls),\n            Self::FileNode(archive) => Some(&archive.inner.acls),\n            _ => None,\n        }\n    }\n\n    pub fn created(&self) -> i64 {\n        match self {\n            ArchivedResource::Calendar(archive) => archive.inner.created.to_native(),\n            ArchivedResource::CalendarEvent(archive) => archive.inner.created.to_native(),\n            ArchivedResource::AddressBook(archive) => archive.inner.created.to_native(),\n            ArchivedResource::ContactCard(archive) => archive.inner.created.to_native(),\n            ArchivedResource::FileNode(archive) => archive.inner.created.to_native(),\n            ArchivedResource::CalendarEventNotification(archive) => {\n                archive.inner.created.to_native()\n            }\n            ArchivedResource::CalendarEventNotificationCollection(_) => 1634515200,\n        }\n    }\n\n    pub fn modified(&self) -> i64 {\n        match self {\n            ArchivedResource::Calendar(archive) => archive.inner.modified.to_native(),\n            ArchivedResource::CalendarEvent(archive) => archive.inner.modified.to_native(),\n            ArchivedResource::AddressBook(archive) => archive.inner.modified.to_native(),\n            ArchivedResource::ContactCard(archive) => archive.inner.modified.to_native(),\n            ArchivedResource::FileNode(archive) => archive.inner.modified.to_native(),\n            ArchivedResource::CalendarEventNotification(archive) => {\n                archive.inner.modified.to_native()\n            }\n            ArchivedResource::CalendarEventNotificationCollection(_) => 1634515200,\n        }\n    }\n\n    pub fn dead_properties(&self) -> Option<&ArchivedDeadProperty> {\n        match self {\n            ArchivedResource::Calendar(archive) => Some(&archive.inner.dead_properties),\n            ArchivedResource::CalendarEvent(archive) => Some(&archive.inner.dead_properties),\n            ArchivedResource::AddressBook(archive) => Some(&archive.inner.dead_properties),\n            ArchivedResource::ContactCard(archive) => Some(&archive.inner.dead_properties),\n            ArchivedResource::FileNode(archive) => Some(&archive.inner.dead_properties),\n            ArchivedResource::CalendarEventNotification(_)\n            | ArchivedResource::CalendarEventNotificationCollection(_) => None,\n        }\n    }\n\n    pub fn content_length(&self) -> Option<u32> {\n        match self {\n            ArchivedResource::FileNode(archive) => {\n                archive.inner.file.as_ref().map(|f| f.size.to_native())\n            }\n            ArchivedResource::CalendarEvent(archive) => archive.inner.size.to_native().into(),\n            ArchivedResource::CalendarEventNotification(archive) => {\n                archive.inner.size.to_native().into()\n            }\n            ArchivedResource::ContactCard(archive) => archive.inner.size.to_native().into(),\n            ArchivedResource::AddressBook(_)\n            | ArchivedResource::Calendar(_)\n            | ArchivedResource::CalendarEventNotificationCollection(_) => None,\n        }\n    }\n\n    pub fn content_type(&self) -> Option<&str> {\n        match self {\n            ArchivedResource::FileNode(archive) => archive\n                .inner\n                .file\n                .as_ref()\n                .and_then(|f| f.media_type.as_deref()),\n            ArchivedResource::CalendarEvent(_) | ArchivedResource::CalendarEventNotification(_) => {\n                \"text/calendar\".into()\n            }\n            ArchivedResource::ContactCard(_) => \"text/vcard\".into(),\n            ArchivedResource::AddressBook(_)\n            | ArchivedResource::Calendar(_)\n            | ArchivedResource::CalendarEventNotificationCollection(_) => None,\n        }\n    }\n\n    pub fn display_name(&self, access_token: &AccessToken) -> Option<&str> {\n        match self {\n            ArchivedResource::Calendar(archive) => {\n                Some(archive.inner.preferences(access_token).name.as_str())\n            }\n            ArchivedResource::CalendarEvent(archive) => archive.inner.display_name.as_deref(),\n            ArchivedResource::AddressBook(archive) => {\n                Some(archive.inner.preferences(access_token).name.as_str())\n            }\n            ArchivedResource::ContactCard(archive) => archive.inner.display_name.as_deref(),\n            ArchivedResource::FileNode(archive) => archive.inner.display_name.as_deref(),\n            ArchivedResource::CalendarEventNotification(_)\n            | ArchivedResource::CalendarEventNotificationCollection(_) => None,\n        }\n    }\n\n    pub fn supported_report_set(&self) -> Option<Vec<ReportSet>> {\n        match self {\n            ArchivedResource::Calendar(_) => vec![\n                ReportSet::SyncCollection,\n                ReportSet::AclPrincipalPropSet,\n                ReportSet::PrincipalMatch,\n                ReportSet::ExpandProperty,\n                ReportSet::CalendarQuery,\n                ReportSet::CalendarMultiGet,\n                ReportSet::FreeBusyQuery,\n            ]\n            .into(),\n            ArchivedResource::AddressBook(_) => vec![\n                ReportSet::SyncCollection,\n                ReportSet::AclPrincipalPropSet,\n                ReportSet::PrincipalMatch,\n                ReportSet::ExpandProperty,\n                ReportSet::AddressbookQuery,\n                ReportSet::AddressbookMultiGet,\n            ]\n            .into(),\n            ArchivedResource::FileNode(archive) if archive.inner.file.is_none() => vec![\n                ReportSet::SyncCollection,\n                ReportSet::AclPrincipalPropSet,\n                ReportSet::PrincipalMatch,\n            ]\n            .into(),\n            ArchivedResource::CalendarEventNotificationCollection(_) => vec![\n                ReportSet::SyncCollection,\n                ReportSet::CalendarQuery,\n                ReportSet::CalendarMultiGet,\n            ]\n            .into(),\n            _ => None,\n        }\n    }\n\n    pub fn resource_type(&self) -> Option<Vec<ResourceType>> {\n        match self {\n            ArchivedResource::Calendar(_) => {\n                vec![ResourceType::Collection, ResourceType::Calendar].into()\n            }\n            ArchivedResource::AddressBook(_) => {\n                vec![ResourceType::Collection, ResourceType::AddressBook].into()\n            }\n            ArchivedResource::FileNode(archive) if archive.inner.file.is_none() => {\n                vec![ResourceType::Collection].into()\n            }\n            ArchivedResource::CalendarEventNotificationCollection(true) => {\n                vec![ResourceType::Collection, ResourceType::ScheduleInbox].into()\n            }\n            ArchivedResource::CalendarEventNotificationCollection(false) => {\n                vec![ResourceType::Collection, ResourceType::ScheduleOutbox].into()\n            }\n            _ => None,\n        }\n    }\n}\n\nimpl SyncType {\n    pub fn is_none(&self) -> bool {\n        matches!(self, SyncType::None)\n    }\n\n    pub fn is_none_or_initial(&self) -> bool {\n        matches!(self, SyncType::None | SyncType::Initial)\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/common/propfind.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ArchivedResource, DavCollection, DavQuery, DavQueryFilter, ETag, SyncType,\n    acl::{DavAclHandler, Privileges},\n    lock::{LockData, build_lock_key},\n    uri::{UriResource, Urn},\n};\nuse crate::{\n    DavError, DavErrorCondition,\n    calendar::{\n        CALENDAR_CONTAINER_PROPS, CALENDAR_ITEM_PROPS,\n        query::{CalendarQueryHandler, try_parse_tz},\n    },\n    card::{\n        CARD_CONTAINER_PROPS, CARD_ITEM_PROPS,\n        query::{serialize_vcard_with_props, vcard_query},\n    },\n    common::{DavQueryResource, acl::current_user_privilege_set, uri::DavUriResource},\n    file::{FILE_CONTAINER_PROPS, FILE_ITEM_PROPS},\n    principal::{\n        CurrentUserPrincipal,\n        propfind::{PrincipalPropFind, build_home_set},\n    },\n};\nuse calcard::{common::timezone::Tz, icalendar::ICalendarComponentType};\nuse common::{DavResourcePath, DavResources, Server, auth::AccessToken};\nuse dav_proto::{\n    Depth, RequestHeaders,\n    parser::header::dav_base_uri,\n    requests::NsDeadProperty,\n    schema::{\n        Collation, Namespace,\n        property::{\n            ActiveLock, CalDavProperty, CardDavProperty, Comp, DavProperty, DavValue,\n            PrincipalProperty, Privilege, ReportSet, ResourceType, Rfc1123DateTime,\n            SupportedCollation, SupportedLock, WebDavProperty,\n        },\n        request::{DavDeadProperty, DavPropertyValue, PropFind},\n        response::{\n            AclRestrictions, BaseCondition, Href, List, MultiStatus, PropStat, Response,\n            SupportedPrivilege,\n        },\n    },\n};\nuse directory::{Permission, Type, backend::internal::manage::ManageDirectory};\nuse groupware::calendar::{SCHEDULE_INBOX_ID, SupportedComponent};\nuse groupware::{\n    DavCalendarResource, DavResourceName, cache::GroupwareCache, calendar::ArchivedTimezone,\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::sync::Arc;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse store::{\n    ahash::AHashMap,\n    query::log::{Change, Query},\n    roaring::RoaringBitmap,\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    dead_property::DeadProperty,\n};\nuse utils::map::bitmap::Bitmap;\n\npub(crate) trait PropFindRequestHandler: Sync + Send {\n    fn handle_propfind_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: PropFind,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn handle_dav_query(\n        &self,\n        access_token: &AccessToken,\n        query: DavQuery<'_>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn dav_quota(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<PropFindAccountQuota>> + Send;\n}\n\npub(crate) struct PropFindData {\n    pub accounts: AHashMap<u32, PropFindAccountData>,\n}\n\n#[derive(Default)]\npub(crate) struct PropFindAccountData {\n    pub resources: Option<Arc<DavResources>>,\n    pub quota: Option<PropFindAccountQuota>,\n    pub owner: Option<Href>,\n    pub locks: Option<Archive<AlignedBytes>>,\n    pub locks_not_found: bool,\n}\n\n#[derive(Clone, Default)]\npub(crate) struct PropFindAccountQuota {\n    pub used: u64,\n    pub available: u64,\n}\n\n#[derive(Debug)]\npub(crate) struct PropFindItem {\n    pub name: String,\n    pub account_id: u32,\n    pub document_id: u32,\n    pub parent_id: Option<u32>,\n    pub is_container: bool,\n}\n\nimpl PropFindRequestHandler for Server {\n    async fn handle_propfind_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: PropFind,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self.validate_uri(access_token, headers.uri).await?;\n\n        // Reject Infinity depth for certain queries\n        let return_children = match headers.depth {\n            Depth::One | Depth::None => true,\n            Depth::Zero => false,\n            Depth::Infinity => match resource.collection {\n                Collection::Principal => true,\n                Collection::Calendar | Collection::AddressBook\n                    if resource.account_id.is_some() && resource.resource.is_some() =>\n                {\n                    true\n                }\n                Collection::CalendarEventNotification if resource.account_id.is_some() => true,\n                _ => {\n                    return Err(DavErrorCondition::new(\n                        StatusCode::FORBIDDEN,\n                        BaseCondition::PropFindFiniteDepth,\n                    )\n                    .into());\n                }\n            },\n        };\n\n        // List shared resources\n        if let Some(account_id) = resource.account_id {\n            match resource.collection {\n                Collection::FileNode\n                | Collection::Calendar\n                | Collection::AddressBook\n                | Collection::CalendarEventNotification => {\n                    // Validate permissions\n                    access_token.assert_has_permission(match resource.collection {\n                        Collection::FileNode => Permission::DavFilePropFind,\n                        Collection::Calendar\n                        | Collection::CalendarEvent\n                        | Collection::CalendarEventNotification => Permission::DavCalPropFind,\n                        Collection::AddressBook | Collection::ContactCard => {\n                            Permission::DavCardPropFind\n                        }\n                        _ => unreachable!(),\n                    })?;\n\n                    self.handle_dav_query(\n                        access_token,\n                        DavQuery::propfind(\n                            UriResource::new_owned(\n                                resource.collection,\n                                account_id,\n                                resource.resource,\n                            ),\n                            request,\n                            headers,\n                        ),\n                    )\n                    .await\n                }\n                Collection::Principal => {\n                    let mut response = MultiStatus::new(Vec::with_capacity(16));\n\n                    if resource.resource.is_some() {\n                        response.add_response(Response::new_status(\n                            [headers.uri.to_string()],\n                            StatusCode::NOT_FOUND,\n                        ));\n                    } else if access_token.has_account_access(account_id)\n                        || (self.core.groupware.allow_directory_query\n                            && access_token.has_permission(Permission::DavPrincipalList))\n                        || access_token.has_permission(Permission::IndividualList)\n                    {\n                        self.prepare_principal_propfind_response(\n                            access_token,\n                            Collection::Principal,\n                            [account_id].into_iter(),\n                            &request,\n                            &mut response,\n                        )\n                        .await?;\n                    } else {\n                        response.add_response(Response::new_status(\n                            [headers.uri.to_string()],\n                            StatusCode::FORBIDDEN,\n                        ));\n                    }\n\n                    Ok(HttpResponse::new(StatusCode::MULTI_STATUS)\n                        .with_xml_body(response.to_string()))\n                }\n                _ => unreachable!(),\n            }\n        } else {\n            let mut response = MultiStatus::new(Vec::with_capacity(16));\n\n            // Add container info\n            if !headers.depth_no_root {\n                add_base_collection_response(\n                    self,\n                    &request,\n                    resource.collection,\n                    access_token,\n                    &mut response,\n                )\n                .await?;\n            }\n\n            if return_children {\n                let ids = if !matches!(resource.collection, Collection::Principal) {\n                    // Validate permissions\n                    access_token.assert_has_permission(match resource.collection {\n                        Collection::FileNode => Permission::DavFilePropFind,\n                        Collection::Calendar\n                        | Collection::CalendarEvent\n                        | Collection::CalendarEventNotification => Permission::DavCalPropFind,\n                        Collection::AddressBook | Collection::ContactCard => {\n                            Permission::DavCardPropFind\n                        }\n                        _ => unreachable!(),\n                    })?;\n                    RoaringBitmap::from_iter(\n                        access_token.all_ids_by_collection(resource.collection),\n                    )\n                } else if (self.core.groupware.allow_directory_query\n                    && access_token.has_permission(Permission::DavPrincipalList))\n                    || access_token.has_permission(Permission::IndividualList)\n                {\n                    // Return all principals\n                    let principals = self\n                        .store()\n                        .list_principals(\n                            None,\n                            access_token.tenant_id(),\n                            &[Type::Individual, Type::Group],\n                            false,\n                            0,\n                            0,\n                        )\n                        .await\n                        .caused_by(trc::location!())?;\n\n                    RoaringBitmap::from_iter(principals.items.into_iter().map(|p| p.id()))\n                } else {\n                    RoaringBitmap::from_iter(access_token.all_ids())\n                };\n\n                self.prepare_principal_propfind_response(\n                    access_token,\n                    resource.collection,\n                    ids.into_iter(),\n                    &request,\n                    &mut response,\n                )\n                .await?;\n            }\n\n            Ok(HttpResponse::new(StatusCode::MULTI_STATUS).with_xml_body(response.to_string()))\n        }\n    }\n\n    async fn handle_dav_query(\n        &self,\n        access_token: &AccessToken,\n        mut query: DavQuery<'_>,\n    ) -> crate::Result<HttpResponse> {\n        let mut response = MultiStatus::new(Vec::with_capacity(16));\n        let mut data = PropFindData::new();\n        let collection_container;\n        let collection_children;\n        let sync_collection;\n        let mut query_filter = None;\n        let mut limit = std::cmp::min(\n            query.limit.unwrap_or(u32::MAX) as usize,\n            self.core.groupware.max_results,\n        );\n        let mut is_sync_limited = false;\n        let mut is_propfind = false;\n        let mut ical_instances_limit = self.core.groupware.max_ical_instances;\n\n        let paths = match std::mem::take(&mut query.resource) {\n            DavQueryResource::Uri(resource) => {\n                collection_container = resource.collection;\n                collection_children = collection_container.child_collection().unwrap();\n                sync_collection = SyncCollection::from(collection_container);\n                is_propfind = true;\n\n                get(\n                    self,\n                    access_token,\n                    collection_container,\n                    collection_children,\n                    sync_collection,\n                    &query,\n                    &mut data,\n                    &mut response,\n                    resource,\n                    limit,\n                    &mut is_sync_limited,\n                )\n                .await?\n            }\n            DavQueryResource::Multiget {\n                hrefs,\n                parent_collection,\n            } => {\n                collection_container = parent_collection;\n                collection_children = collection_container.child_collection().unwrap();\n                sync_collection = SyncCollection::from(collection_container);\n\n                multiget(\n                    self,\n                    access_token,\n                    collection_container,\n                    collection_children,\n                    sync_collection,\n                    &mut data,\n                    &mut response,\n                    hrefs,\n                )\n                .await?\n            }\n            DavQueryResource::Query {\n                filter,\n                parent_collection,\n                items,\n            } => {\n                query_filter = Some(filter);\n                collection_container = parent_collection;\n                collection_children = collection_container.child_collection().unwrap();\n                sync_collection = SyncCollection::from(collection_container);\n\n                items\n            }\n            DavQueryResource::None => unreachable!(),\n        };\n        response.set_namespace(collection_container.namespace());\n\n        let mut skip_not_found = query.expand;\n        let properties = match &query.propfind {\n            PropFind::PropName => {\n                let (container_props, children_props) = match collection_container {\n                    Collection::FileNode => {\n                        (FILE_CONTAINER_PROPS.as_slice(), FILE_ITEM_PROPS.as_slice())\n                    }\n                    Collection::Calendar | Collection::CalendarEventNotification => (\n                        CALENDAR_CONTAINER_PROPS.as_slice(),\n                        CALENDAR_ITEM_PROPS.as_slice(),\n                    ),\n                    Collection::AddressBook => {\n                        (CARD_CONTAINER_PROPS.as_slice(), CARD_ITEM_PROPS.as_slice())\n                    }\n                    _ => unreachable!(),\n                };\n\n                for item in paths {\n                    let props = if item.is_container {\n                        container_props\n                            .iter()\n                            .cloned()\n                            .map(DavPropertyValue::empty)\n                            .collect::<Vec<_>>()\n                    } else {\n                        children_props\n                            .iter()\n                            .cloned()\n                            .map(DavPropertyValue::empty)\n                            .collect::<Vec<_>>()\n                    };\n\n                    response.add_response(Response::new_propstat(\n                        item.name,\n                        vec![PropStat::new_list(props)],\n                    ));\n                }\n\n                return Ok(\n                    HttpResponse::new(StatusCode::MULTI_STATUS).with_xml_body(response.to_string())\n                );\n            }\n            PropFind::AllProp(items) => {\n                skip_not_found = true;\n                let mut result = Vec::with_capacity(items.len() + DavProperty::ALL_PROPS.len());\n                result.extend(DavProperty::ALL_PROPS);\n                result.extend(items.iter().filter(|field| !field.is_all_prop()).cloned());\n                result\n            }\n            PropFind::Prop(items) => items.clone(),\n        };\n\n        let is_scheduling = collection_container == Collection::CalendarEventNotification;\n        'outer: for item in paths {\n            let account_id = item.account_id;\n            let document_id = item.document_id;\n            let collection = if item.is_container {\n                collection_container\n            } else {\n                collection_children\n            };\n\n            // Unarchive resource\n            let archive_;\n            let archive = if is_scheduling && item.is_container {\n                archive_ = Archive::default();\n                ArchivedResource::CalendarEventNotificationCollection(\n                    item.document_id == SCHEDULE_INBOX_ID,\n                )\n            } else if let Some(archive) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    collection,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                archive_ = archive;\n                ArchivedResource::from_archive(&archive_, collection).caused_by(trc::location!())?\n            } else {\n                response.add_response(Response::new_status([item.name], StatusCode::NOT_FOUND));\n                continue;\n            };\n\n            // Filter\n            let mut calendar_filter = None;\n            if let Some(query_filter) = &query_filter {\n                match (query_filter, &archive) {\n                    (DavQueryFilter::Addressbook(filter), ArchivedResource::ContactCard(card)) => {\n                        if !vcard_query(&card.inner.card, filter) {\n                            continue;\n                        }\n                    }\n                    (\n                        DavQueryFilter::Calendar {\n                            filter,\n                            timezone,\n                            max_time_range,\n                        },\n                        ArchivedResource::CalendarEvent(event),\n                    ) => {\n                        let default_tz = if let Some(tz) = try_parse_tz(timezone) {\n                            tz\n                        } else if let Some(calendar_id) = item.parent_id {\n                            data.resources(self, access_token, account_id, SyncCollection::Calendar)\n                                .await\n                                .caused_by(trc::location!())?\n                                .calendar_default_tz(calendar_id, account_id)\n                                .unwrap_or(Tz::UTC)\n                        } else {\n                            Tz::UTC\n                        };\n                        let mut query_handler =\n                            CalendarQueryHandler::new(event.inner, *max_time_range, default_tz);\n                        if !query_handler.filter(event.inner, filter) {\n                            continue;\n                        }\n                        calendar_filter = Some(query_handler);\n                    }\n                    _ => (),\n                }\n            }\n\n            // Fill properties\n            let dead_properties = archive.dead_properties();\n            let mut fields = Vec::with_capacity(properties.len());\n            let mut fields_not_found = Vec::new();\n            for property in &properties {\n                match property {\n                    DavProperty::WebDav(dav_property) => match dav_property {\n                        WebDavProperty::CreationDate => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::Timestamp(archive.created()),\n                            ));\n                        }\n                        WebDavProperty::DisplayName => {\n                            if let Some(name) = archive.display_name(access_token) {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    DavValue::String(name.to_string()),\n                                ));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::GetContentLanguage => {\n                            if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::GetContentLength => {\n                            if let Some(value) = archive.content_length() {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    DavValue::Uint64(value as u64),\n                                ));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::GetContentType => {\n                            if let Some(value) = archive.content_type() {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    DavValue::String(value.to_string()),\n                                ));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::GetETag => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::String(archive_.etag()),\n                            ));\n                        }\n                        WebDavProperty::GetCTag => {\n                            if item.is_container {\n                                let ctag = data\n                                    .resources(self, access_token, account_id, sync_collection)\n                                    .await\n                                    .caused_by(trc::location!())?\n                                    .highest_change_id;\n\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    DavValue::String(format!(\"\\\"{ctag}\\\"\")),\n                                ));\n                            } else {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                            response.set_namespace(Namespace::CalendarServer);\n                        }\n                        WebDavProperty::GetLastModified => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::Rfc1123Date(Rfc1123DateTime::new(archive.modified())),\n                            ));\n                        }\n                        WebDavProperty::ResourceType => {\n                            if let Some(resource_type) = archive.resource_type() {\n                                fields.push(DavPropertyValue::new(property.clone(), resource_type));\n                            } else {\n                                fields.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::LockDiscovery => {\n                            if let Some(locks) = data\n                                .locks(self, account_id, collection_container, &item)\n                                .await\n                                .caused_by(trc::location!())?\n                            {\n                                fields.push(DavPropertyValue::new(property.clone(), locks));\n                            } else {\n                                fields.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::SupportedLock => {\n                            if !is_scheduling {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    SupportedLock::default(),\n                                ));\n                            } else {\n                                fields.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::SupportedReportSet => {\n                            if let Some(report_set) = archive.supported_report_set() {\n                                fields.push(DavPropertyValue::new(property.clone(), report_set));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::SyncToken => {\n                            let sync_token = data\n                                .resources(self, access_token, account_id, sync_collection)\n                                .await\n                                .caused_by(trc::location!())?\n                                .sync_token();\n\n                            fields.push(DavPropertyValue::new(property.clone(), sync_token));\n                        }\n                        WebDavProperty::CurrentUserPrincipal => {\n                            if !query.expand {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    vec![access_token.current_user_principal()],\n                                ));\n                            } else {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    self.expand_principal(\n                                        access_token,\n                                        access_token.primary_id(),\n                                        &query.propfind,\n                                    )\n                                    .await?\n                                    .map(DavValue::Response)\n                                    .unwrap_or(DavValue::Null),\n                                ));\n                            }\n                        }\n                        WebDavProperty::QuotaAvailableBytes => {\n                            if item.is_container {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    data.quota(self, access_token, account_id)\n                                        .await\n                                        .caused_by(trc::location!())?\n                                        .available,\n                                ));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::QuotaUsedBytes => {\n                            if item.is_container {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    data.quota(self, access_token, account_id)\n                                        .await\n                                        .caused_by(trc::location!())?\n                                        .used,\n                                ));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::Owner => {\n                            if !query.expand {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    vec![\n                                        data.owner(self, access_token, account_id)\n                                            .await\n                                            .caused_by(trc::location!())?,\n                                    ],\n                                ));\n                            } else {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    self.expand_principal(\n                                        access_token,\n                                        account_id,\n                                        &query.propfind,\n                                    )\n                                    .await?\n                                    .map(DavValue::Response)\n                                    .unwrap_or(DavValue::Null),\n                                ));\n                            }\n                        }\n                        WebDavProperty::Group => {\n                            fields.push(DavPropertyValue::empty(property.clone()));\n                        }\n                        WebDavProperty::SupportedPrivilegeSet => {\n                            if !is_scheduling {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    vec![SupportedPrivilege::all_privileges(\n                                        collection_container == Collection::Calendar,\n                                    )],\n                                ));\n                            } else {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    vec![SupportedPrivilege::all_scheduling_privileges(matches!(\n                                        archive,\n                                        ArchivedResource::CalendarEventNotification(_)\n                                            | ArchivedResource::CalendarEventNotificationCollection(\n                                                true\n                                            )\n                                    ))],\n                                ));\n                            }\n                        }\n                        WebDavProperty::CurrentUserPrivilegeSet => {\n                            let privileges = if is_scheduling {\n                                Privilege::scheduling(\n                                    matches!(\n                                        archive,\n                                        ArchivedResource::CalendarEventNotification(_)\n                                            | ArchivedResource::CalendarEventNotificationCollection(\n                                                true\n                                            )\n                                    ),\n                                    access_token.is_member(account_id),\n                                )\n                            } else if access_token.is_member(account_id) {\n                                Privilege::all(matches!(\n                                    collection,\n                                    Collection::Calendar | Collection::CalendarEvent\n                                ))\n                            } else if let Some(acls) = archive.acls() {\n                                access_token.current_privilege_set(\n                                    account_id,\n                                    acls,\n                                    collection_container == Collection::Calendar,\n                                )\n                            } else if let Some(parent_id) = item.parent_id {\n                                current_user_privilege_set(\n                                    data.resources(self, access_token, account_id, sync_collection)\n                                        .await\n                                        .caused_by(trc::location!())?\n                                        .container_acl(access_token, parent_id),\n                                )\n                            } else {\n                                vec![]\n                            };\n\n                            if !privileges.is_empty() {\n                                fields.push(DavPropertyValue::new(property.clone(), privileges));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::Acl => {\n                            if let Some(acls) = archive.acls() {\n                                let aces = self\n                                    .resolve_ace(\n                                        access_token,\n                                        account_id,\n                                        acls,\n                                        query.expand.then_some(&query.propfind),\n                                    )\n                                    .await?;\n\n                                fields.push(DavPropertyValue::new(property.clone(), aces));\n                            } else if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        WebDavProperty::AclRestrictions => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                AclRestrictions::default()\n                                    .with_no_invert()\n                                    .with_grant_only(),\n                            ));\n                        }\n                        WebDavProperty::InheritedAclSet => {\n                            fields.push(DavPropertyValue::empty(property.clone()));\n                        }\n                        WebDavProperty::PrincipalCollectionSet => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                vec![Href(\n                                    DavResourceName::Principal.collection_path().to_string(),\n                                )],\n                            ));\n                        }\n                    },\n                    DavProperty::DeadProperty(tag) => {\n                        if let Some(value) =\n                            dead_properties.and_then(|props| props.find_tag(&tag.name))\n                        {\n                            fields.push(DavPropertyValue::new(property.clone(), value));\n                        } else {\n                            fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                        }\n                    }\n                    DavProperty::CardDav(card_property) => match (card_property, &archive) {\n                        (\n                            CardDavProperty::AddressbookDescription,\n                            ArchivedResource::AddressBook(book),\n                        ) => {\n                            if let Some(desc) =\n                                book.inner.preferences(access_token).description.as_deref()\n                            {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    desc.to_string(),\n                                ));\n                            } else {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        (\n                            CardDavProperty::SupportedAddressData,\n                            ArchivedResource::AddressBook(_),\n                        ) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::SupportedAddressData,\n                            ));\n                        }\n                        (\n                            CardDavProperty::SupportedCollationSet,\n                            ArchivedResource::AddressBook(_),\n                        ) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::Collations(List(vec![\n                                    SupportedCollation {\n                                        collation: Collation::AsciiCasemap,\n                                        namespace: Namespace::CardDav,\n                                    },\n                                    SupportedCollation {\n                                        collation: Collation::UnicodeCasemap,\n                                        namespace: Namespace::CardDav,\n                                    },\n                                ])),\n                            ));\n                        }\n                        (CardDavProperty::MaxResourceSize, ArchivedResource::AddressBook(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                self.core.groupware.max_vcard_size as u64,\n                            ));\n                        }\n                        (\n                            CardDavProperty::AddressData(items),\n                            ArchivedResource::ContactCard(card),\n                        ) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::CData(serialize_vcard_with_props(\n                                    &card.inner.card,\n                                    items,\n                                    query\n                                        .max_vcard_version\n                                        .or_else(|| card.inner.card.version()),\n                                )),\n                            ));\n                        }\n                        _ => {\n                            if !skip_not_found {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                    },\n                    DavProperty::CalDav(cal_property) => match (cal_property, &archive) {\n                        (\n                            CalDavProperty::CalendarDescription,\n                            ArchivedResource::Calendar(calendar),\n                        ) => {\n                            if let Some(desc) = calendar\n                                .inner\n                                .preferences(access_token)\n                                .description\n                                .as_deref()\n                            {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    desc.to_string(),\n                                ));\n                            } else {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        (\n                            CalDavProperty::CalendarTimezone,\n                            ArchivedResource::Calendar(calendar),\n                        ) => {\n                            if let ArchivedTimezone::Custom(tz) =\n                                &calendar.inner.preferences(access_token).time_zone\n                            {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    DavValue::CData(tz.to_string()),\n                                ));\n                            } else {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        (CalDavProperty::TimezoneId, ArchivedResource::Calendar(calendar)) => {\n                            if let ArchivedTimezone::IANA(tz) =\n                                &calendar.inner.preferences(access_token).time_zone\n                            {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    Tz::from_id(tz.to_native()).unwrap_or(Tz::UTC).to_string(),\n                                ));\n                            } else {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                        (\n                            CalDavProperty::SupportedCalendarComponentSet,\n                            ArchivedResource::Calendar(calendar),\n                        ) => {\n                            let supported_components =\n                                calendar.inner.supported_components.to_native();\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                if supported_components != 0 {\n                                    DavValue::Components(List(\n                                        Bitmap::<SupportedComponent>::from(supported_components)\n                                            .into_iter()\n                                            .map(ICalendarComponentType::from)\n                                            .map(Comp)\n                                            .collect(),\n                                    ))\n                                } else {\n                                    DavValue::all_calendar_components()\n                                },\n                            ));\n                        }\n                        (CalDavProperty::SupportedCalendarData, ArchivedResource::Calendar(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::SupportedCalendarData,\n                            ));\n                        }\n                        (CalDavProperty::SupportedCollationSet, ArchivedResource::Calendar(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::Collations(List(vec![\n                                    SupportedCollation {\n                                        collation: Collation::AsciiCasemap,\n                                        namespace: Namespace::CalDav,\n                                    },\n                                    SupportedCollation {\n                                        collation: Collation::UnicodeCasemap,\n                                        namespace: Namespace::CalDav,\n                                    },\n                                ])),\n                            ));\n                        }\n                        (CalDavProperty::MaxResourceSize, ArchivedResource::Calendar(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                self.core.groupware.max_ical_size as u64,\n                            ));\n                        }\n                        (CalDavProperty::MinDateTime, ArchivedResource::Calendar(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::String(\"0001-01-01T00:00:00Z\".to_string()),\n                            ));\n                        }\n                        (CalDavProperty::MaxDateTime, ArchivedResource::Calendar(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::String(\"9999-12-31T23:59:59Z\".to_string()),\n                            ));\n                        }\n                        (CalDavProperty::MaxInstances, ArchivedResource::Calendar(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                self.core.groupware.max_ical_instances as u64,\n                            ));\n                        }\n                        (\n                            CalDavProperty::MaxAttendeesPerInstance,\n                            ArchivedResource::Calendar(_),\n                        ) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                self.core.groupware.max_ical_attendees_per_instance as u64,\n                            ));\n                        }\n                        (\n                            CalDavProperty::CalendarData(data),\n                            ArchivedResource::CalendarEvent(event),\n                        ) => {\n                            if calendar_filter.is_some() || !data.properties.is_empty() {\n                                if let Some(ical) = calendar_filter\n                                    .get_or_insert_with(|| {\n                                        CalendarQueryHandler::new(event.inner, None, Tz::UTC)\n                                    })\n                                    .serialize_ical(event.inner, data, &mut ical_instances_limit)\n                                {\n                                    fields.push(DavPropertyValue::new(\n                                        property.clone(),\n                                        DavValue::CData(ical),\n                                    ));\n                                } else {\n                                    limit = 0;\n                                    break 'outer;\n                                }\n                            } else {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    DavValue::CData(event.inner.data.event.to_string()),\n                                ));\n                            }\n                        }\n                        (\n                            CalDavProperty::CalendarData(_),\n                            ArchivedResource::CalendarEventNotification(event),\n                        ) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::CData(event.inner.event.to_string()),\n                            ));\n                        }\n                        (CalDavProperty::ScheduleTag, ArchivedResource::CalendarEvent(event))\n                            if event.inner.schedule_tag.is_some() =>\n                        {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::String(format!(\n                                    \"\\\"{}\\\"\",\n                                    event.inner.schedule_tag.as_ref().unwrap()\n                                )),\n                            ));\n                        }\n                        (CalDavProperty::ScheduleCalendarTransp, ArchivedResource::Calendar(_)) => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::DeadProperty(DeadProperty::single_with_ns(\n                                    Namespace::CalDav,\n                                    \"opaque\",\n                                )),\n                            ));\n                        }\n                        (\n                            CalDavProperty::ScheduleDefaultCalendarURL,\n                            ArchivedResource::CalendarEventNotificationCollection(true),\n                        ) => {\n                            if let Some(default_cal) = &self.core.groupware.default_calendar_name {\n                                fields.push(DavPropertyValue::new(\n                                    property.clone(),\n                                    vec![Href(format!(\n                                        \"{}/{}/{default_cal}/\",\n                                        DavResourceName::Cal.base_path(),\n                                        item.name.split('/').nth(3).unwrap_or_default()\n                                    ))],\n                                ));\n                            } else {\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n\n                        _ => {\n                            if !skip_not_found {\n                                response.set_namespace(property.namespace());\n                                fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            }\n                        }\n                    },\n\n                    property => {\n                        if !skip_not_found {\n                            response.set_namespace(property.namespace());\n                            fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                        }\n                    }\n                }\n            }\n\n            // Add dead properties\n            if skip_not_found\n                && let Some(dead_properties) =\n                    dead_properties.filter(|dead_properties| !dead_properties.0.is_empty())\n            {\n                dead_properties.to_dav_values(&mut fields);\n            }\n\n            // Add response\n            let mut prop_stat = Vec::with_capacity(2);\n            if !fields.is_empty() {\n                prop_stat.push(PropStat::new_list(fields));\n            }\n            if !fields_not_found.is_empty() && !query.is_minimal() {\n                prop_stat\n                    .push(PropStat::new_list(fields_not_found).with_status(StatusCode::NOT_FOUND));\n            }\n            if prop_stat.is_empty() {\n                prop_stat.push(PropStat::new_list(vec![]));\n            }\n            response.add_response(Response::new_propstat(item.name, prop_stat));\n\n            limit -= 1;\n            if limit == 0 {\n                break;\n            }\n        }\n\n        if limit == 0 || is_sync_limited {\n            response.add_response(\n                Response::new_status([query.uri], StatusCode::INSUFFICIENT_STORAGE)\n                    .with_error(BaseCondition::NumberOfMatchesWithinLimit)\n                    .with_response_description(if ical_instances_limit > 0 {\n                        format!(\n                            \"The number of matches exceeds the limit of {}\",\n                            query\n                                .limit\n                                .unwrap_or(self.core.groupware.max_results as u32)\n                        )\n                    } else {\n                        format!(\n                            \"The number of recurrence instances exceeds the limit of {}\",\n                            query\n                                .limit\n                                .unwrap_or(self.core.groupware.max_ical_instances as u32)\n                        )\n                    }),\n            );\n        }\n\n        if !response.response.0.is_empty() || !query.sync_type.is_none() {\n            Ok(HttpResponse::new(StatusCode::MULTI_STATUS).with_xml_body(response.to_string()))\n        } else if !is_propfind {\n            Ok(HttpResponse::new(StatusCode::MULTI_STATUS)\n                .with_xml_body(MultiStatus::not_found(query.uri).to_string()))\n        } else {\n            Ok(HttpResponse::new(StatusCode::NOT_FOUND))\n        }\n    }\n\n    async fn dav_quota(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> trc::Result<PropFindAccountQuota> {\n        let resource_token = self\n            .get_resource_token(access_token, account_id)\n            .await\n            .caused_by(trc::location!())?;\n        let quota = if resource_token.quota > 0 {\n            resource_token.quota\n        } else if let Some(tenant) = resource_token.tenant.filter(|t| t.quota > 0) {\n            tenant.quota\n        } else {\n            u32::MAX as u64\n        };\n        let used = self\n            .get_used_quota(account_id)\n            .await\n            .caused_by(trc::location!())? as u64;\n\n        Ok(PropFindAccountQuota {\n            used,\n            available: quota.saturating_sub(used),\n        })\n    }\n}\n#[allow(clippy::too_many_arguments)]\nasync fn get(\n    server: &Server,\n    access_token: &AccessToken,\n    collection_container: Collection,\n    collection_children: Collection,\n    sync_collection: SyncCollection,\n    query: &DavQuery<'_>,\n    data: &mut PropFindData,\n    response: &mut MultiStatus,\n    resource: UriResource<u32, Option<&str>>,\n    limit: usize,\n    is_sync_limited: &mut bool,\n) -> crate::Result<Vec<PropFindItem>> {\n    let container_has_children = collection_children != collection_container;\n    response.set_namespace(collection_container.namespace());\n\n    let account_id = resource.account_id;\n    let resources = data\n        .resources(server, access_token, account_id, sync_collection)\n        .await\n        .caused_by(trc::location!())?;\n\n    // Obtain document ids\n    let mut display_containers = if !access_token.is_member(account_id) {\n        resources\n            .shared_containers(\n                access_token,\n                [if container_has_children {\n                    Acl::ReadItems\n                } else {\n                    Acl::Read\n                }],\n                true,\n            )\n            .into()\n    } else {\n        None\n    };\n    let mut display_children = display_containers\n        .as_ref()\n        .filter(|_| container_has_children)\n        .map(|containers| {\n            RoaringBitmap::from_iter(resources.resources.iter().filter_map(|r| {\n                if r.child_names()\n                    .is_some_and(|n| n.iter().any(|n| containers.contains(n.parent_id)))\n                {\n                    Some(r.document_id)\n                } else {\n                    None\n                }\n            }))\n        });\n\n    // Filter by changelog\n    let is_sync = match query.sync_type {\n        SyncType::From { id, seq } => {\n            let changes = server\n                .store()\n                .changes(account_id, sync_collection.into(), Query::Since(id))\n                .await\n                .caused_by(trc::location!())?;\n            let mut vanished: Vec<String> = Vec::new();\n\n            // Merge changes\n            let mut total_changes = 0;\n            let mut maybe_has_vanished = false;\n            if container_has_children {\n                let mut container_changes = RoaringBitmap::new();\n                let mut item_changes = RoaringBitmap::new();\n\n                for change in changes.changes {\n                    match change {\n                        Change::InsertItem(id) => {\n                            item_changes.insert(id as u32);\n                        }\n                        Change::UpdateItem(id) => {\n                            maybe_has_vanished = true;\n                            item_changes.insert(id as u32);\n                        }\n                        Change::InsertContainer(id) => {\n                            container_changes.insert(id as u32);\n                        }\n                        Change::UpdateContainer(id) => {\n                            maybe_has_vanished = true;\n                            container_changes.insert(id as u32);\n                        }\n                        Change::DeleteContainer(_) | Change::DeleteItem(_) => {\n                            maybe_has_vanished = true;\n                        }\n                        Change::UpdateContainerProperty(_) => (),\n                    }\n                }\n\n                for (document_ids, changes) in [\n                    (&mut display_containers, container_changes),\n                    (&mut display_children, item_changes),\n                ] {\n                    if let Some(document_ids) = document_ids {\n                        *document_ids &= changes;\n                        total_changes += document_ids.len() as usize;\n                    } else {\n                        total_changes += changes.len() as usize;\n                        *document_ids = Some(changes);\n                    }\n                }\n            } else {\n                let changes = RoaringBitmap::from_iter(changes.changes.iter().filter_map(\n                    |change| match change {\n                        Change::InsertItem(id) | Change::InsertContainer(id) => Some(*id as u32),\n                        Change::UpdateItem(id) | Change::UpdateContainer(id) => {\n                            maybe_has_vanished = true;\n                            Some(*id as u32)\n                        }\n                        Change::DeleteContainer(_) | Change::DeleteItem(_) => {\n                            maybe_has_vanished = true;\n                            None\n                        }\n                        _ => None,\n                    },\n                ));\n                if let Some(document_ids) = &mut display_containers {\n                    *document_ids &= changes;\n                    total_changes += document_ids.len() as usize;\n                } else {\n                    total_changes += changes.len() as usize;\n                    display_containers = Some(changes);\n                }\n            }\n\n            if maybe_has_vanished\n                && let Some(vanished_collection) = sync_collection.vanished_collection()\n            {\n                vanished = server\n                    .store()\n                    .vanished(account_id, vanished_collection.into(), Query::Since(id))\n                    .await\n                    .caused_by(trc::location!())?;\n                total_changes += vanished.len();\n            }\n\n            // Truncate changes\n            if total_changes > limit {\n                let mut offset = limit * seq as usize;\n                let mut total_changes = 0;\n\n                // Add vanished items to response\n                for item in vanished {\n                    if offset > 0 {\n                        offset -= 1;\n                    } else if total_changes < limit {\n                        response.add_response(Response::new_status([item], StatusCode::NOT_FOUND));\n                        total_changes += 1;\n                    } else {\n                        *is_sync_limited = true;\n                    }\n                }\n\n                // Add items to document set\n                for document_ids in [&mut display_containers, &mut display_children]\n                    .into_iter()\n                    .flatten()\n                {\n                    let mut new_document_ids = RoaringBitmap::new();\n                    for id in document_ids.iter() {\n                        if offset > 0 {\n                            offset -= 1;\n                        } else if total_changes < limit {\n                            new_document_ids.insert(id);\n                            total_changes += 1;\n                        } else {\n                            *is_sync_limited = true;\n                        }\n                    }\n                    *document_ids = new_document_ids;\n                }\n\n                if *is_sync_limited {\n                    response.set_sync_token(Urn::Sync { id, seq: seq + 1 }.to_string());\n                }\n            } else {\n                // Add vanished items to response\n                for item in vanished {\n                    response.add_response(Response::new_status([item], StatusCode::NOT_FOUND));\n                }\n            }\n\n            if !*is_sync_limited {\n                response.set_sync_token(resources.sync_token());\n            }\n\n            true\n        }\n        SyncType::Initial => {\n            response.set_sync_token(resources.sync_token());\n            false\n        }\n        SyncType::None => false,\n    };\n\n    let mut results = Vec::new();\n    if let Some(resource) = resource.resource {\n        results = resources\n            .subtree_with_depth(resource, query.depth)\n            .filter(|item| {\n                display_containers.as_ref().is_none_or(|containers| {\n                    if container_has_children {\n                        if item.is_container() {\n                            containers.contains(item.document_id())\n                        } else {\n                            display_children\n                                .as_ref()\n                                .is_some_and(|children| children.contains(item.document_id()))\n                        }\n                    } else {\n                        containers.contains(item.document_id())\n                    }\n                }) && (!query.depth_no_root || item.path() != resource)\n            })\n            .map(|item| PropFindItem::new(resources.format_resource(item), account_id, item))\n            .collect::<Vec<_>>();\n    } else {\n        if !query.depth_no_root && query.sync_type.is_none_or_initial() {\n            server\n                .prepare_principal_propfind_response(\n                    access_token,\n                    collection_container,\n                    [account_id].into_iter(),\n                    &query.propfind,\n                    response,\n                )\n                .await?;\n        }\n\n        if query.depth != 0 {\n            results = resources\n                .tree_with_depth(query.depth - 1)\n                .filter(|item| {\n                    display_containers.as_ref().is_none_or(|containers| {\n                        if container_has_children {\n                            if item.is_container() {\n                                containers.contains(item.document_id())\n                            } else {\n                                display_children\n                                    .as_ref()\n                                    .is_some_and(|children| children.contains(item.document_id()))\n                            }\n                        } else {\n                            containers.contains(item.document_id())\n                        }\n                    })\n                })\n                .map(|item| PropFindItem::new(resources.format_resource(item), account_id, item))\n                .collect::<Vec<_>>();\n\n            // Assisted discovery:\n            // If 'bob' has access to 'jane' and `bill` calendars, a query to '/dav/cal/bob' will return:\n            //    - /dav/cal/bob/default\n            //    - /dav/cal/jane/default\n            //    - /dav/cal/bill/default\n            // This is invalid but it's the only workaround for clients which do not support multiple home-sets\n            if server.core.groupware.assisted_discovery\n                && !is_sync\n                && account_id == access_token.primary_id()\n                && matches!(\n                    sync_collection,\n                    SyncCollection::Calendar | SyncCollection::AddressBook\n                )\n            {\n                for shared_account_id in access_token.all_ids_by_collection(collection_container) {\n                    if shared_account_id == access_token.primary_id() {\n                        continue;\n                    }\n                    let shared_resources = data\n                        .resources(server, access_token, shared_account_id, sync_collection)\n                        .await\n                        .caused_by(trc::location!())?;\n                    let shared_containers =\n                        (!access_token.is_member(shared_account_id)).then(|| {\n                            shared_resources.shared_containers(\n                                access_token,\n                                [if container_has_children {\n                                    Acl::ReadItems\n                                } else {\n                                    Acl::Read\n                                }],\n                                true,\n                            )\n                        });\n                    if shared_containers\n                        .as_ref()\n                        .is_none_or(|containers| !containers.is_empty())\n                    {\n                        results.extend(\n                            shared_resources\n                                .tree_with_depth(query.depth - 1)\n                                .filter(|item| {\n                                    item.is_container()\n                                        && shared_containers.as_ref().is_none_or(|containers| {\n                                            containers.contains(item.document_id())\n                                        })\n                                })\n                                .map(|item| {\n                                    PropFindItem::new(\n                                        shared_resources.format_resource(item),\n                                        shared_account_id,\n                                        item,\n                                    )\n                                }),\n                        );\n                    }\n                }\n            }\n        }\n    }\n\n    Ok(results)\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn multiget(\n    server: &Server,\n    access_token: &AccessToken,\n    collection_container: Collection,\n    collection_children: Collection,\n    sync_collection: SyncCollection,\n    data: &mut PropFindData,\n    response: &mut MultiStatus,\n    hrefs: Vec<String>,\n) -> crate::Result<Vec<PropFindItem>> {\n    let mut paths = Vec::with_capacity(hrefs.len() * 2);\n    let mut shared_folders_by_account: AHashMap<u32, Arc<RoaringBitmap>> =\n        AHashMap::with_capacity(3);\n\n    for item in hrefs {\n        let resource = match server\n            .validate_uri(access_token, &item)\n            .await\n            .and_then(|r| r.into_owned_uri())\n        {\n            Ok(resource) => resource,\n            Err(DavError::Code(code)) => {\n                response.add_response(Response::new_status([item], code));\n                continue;\n            }\n            Err(err) => {\n                return Err(err);\n            }\n        };\n\n        let account_id = resource.account_id;\n        let resources = data\n            .resources(server, access_token, account_id, sync_collection)\n            .await\n            .caused_by(trc::location!())?;\n\n        let document_ids = if !access_token.is_member(account_id) {\n            if let Some(document_ids) = shared_folders_by_account.get(&account_id) {\n                document_ids.clone().into()\n            } else {\n                let document_ids = Arc::new(resources.shared_containers(\n                    access_token,\n                    [if collection_children == collection_container {\n                        Acl::ReadItems\n                    } else {\n                        Acl::Read\n                    }],\n                    true,\n                ));\n                shared_folders_by_account.insert(account_id, document_ids.clone());\n                document_ids.into()\n            }\n        } else {\n            None\n        };\n\n        if let Some(resource) = resource.resource.and_then(|name| resources.by_path(name)) {\n            if !resource.is_container() {\n                if document_ids\n                    .as_ref()\n                    .is_none_or(|docs| docs.contains(resource.parent_id().unwrap()))\n                {\n                    paths.push(PropFindItem::new(\n                        resources.format_resource(resource),\n                        account_id,\n                        resource,\n                    ));\n                } else {\n                    response.add_response(\n                        Response::new_status([item], StatusCode::FORBIDDEN)\n                            .with_response_description(\n                                \"Not enough permissions to access this shared resource\",\n                            ),\n                    );\n                }\n            } else {\n                response.add_response(\n                    Response::new_status([item], StatusCode::FORBIDDEN)\n                        .with_response_description(\"Multiget not allowed for collections\"),\n                );\n            }\n        } else {\n            response.add_response(Response::new_status([item], StatusCode::NOT_FOUND));\n        }\n    }\n\n    Ok(paths)\n}\n\nimpl PropFindItem {\n    pub fn new(name: String, account_id: u32, resource: DavResourcePath<'_>) -> Self {\n        Self {\n            name,\n            account_id,\n            document_id: resource.document_id(),\n            parent_id: resource.parent_id(),\n            is_container: resource.is_container(),\n        }\n    }\n}\n\nimpl PropFindData {\n    pub fn new() -> Self {\n        Self {\n            accounts: AHashMap::with_capacity(2),\n        }\n    }\n\n    pub async fn quota(\n        &mut self,\n        server: &Server,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> trc::Result<PropFindAccountQuota> {\n        let data = self.accounts.entry(account_id).or_default();\n\n        if data.quota.is_none() {\n            data.quota = server.dav_quota(access_token, account_id).await?.into();\n        }\n\n        Ok(data.quota.clone().unwrap())\n    }\n\n    pub async fn owner(\n        &mut self,\n        server: &Server,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> trc::Result<Href> {\n        let data = self.accounts.entry(account_id).or_default();\n\n        if data.owner.is_none() {\n            data.owner = server\n                .owner_href(access_token, account_id)\n                .await\n                .caused_by(trc::location!())?\n                .into();\n        }\n\n        Ok(data.owner.clone().unwrap())\n    }\n\n    pub async fn resources(\n        &mut self,\n        server: &Server,\n        access_token: &AccessToken,\n        account_id: u32,\n        sync_collection: SyncCollection,\n    ) -> trc::Result<Arc<DavResources>> {\n        let data = self.accounts.entry(account_id).or_default();\n\n        if data.resources.is_none() {\n            let resources = server\n                .fetch_dav_resources(access_token, account_id, sync_collection)\n                .await\n                .caused_by(trc::location!())?;\n            data.resources = resources.into();\n        }\n\n        Ok(data.resources.clone().unwrap())\n    }\n\n    pub async fn locks(\n        &mut self,\n        server: &Server,\n        account_id: u32,\n        collection_container: Collection,\n        item: &PropFindItem,\n    ) -> trc::Result<Option<Vec<ActiveLock>>> {\n        let data = self.accounts.entry(account_id).or_default();\n\n        if data.locks.is_none() && !data.locks_not_found {\n            data.locks = server\n                .in_memory_store()\n                .key_get::<Archive<AlignedBytes>>(\n                    build_lock_key(account_id, collection_container).as_slice(),\n                )\n                .await\n                .caused_by(trc::location!())?;\n            if data.locks.is_none() {\n                data.locks_not_found = true;\n            }\n        }\n\n        if let Some(lock_data) = &data.locks {\n            let base_uri = dav_base_uri(&item.name).unwrap_or_default();\n            lock_data.unarchive::<LockData>().map(|locks| {\n                locks\n                    .find_locks(&item.name.strip_prefix(base_uri).unwrap()[1..], false)\n                    .iter()\n                    .map(|(path, lock)| lock.to_active_lock(format!(\"{base_uri}/{path}\")))\n                    .collect::<Vec<_>>()\n                    .into()\n            })\n        } else {\n            Ok(None)\n        }\n    }\n}\n\npub(crate) trait SyncTokenUrn {\n    fn sync_token(&self) -> String;\n}\n\nimpl SyncTokenUrn for DavResources {\n    fn sync_token(&self) -> String {\n        Urn::Sync {\n            id: self.highest_change_id,\n            seq: 0,\n        }\n        .to_string()\n    }\n}\n\nasync fn add_base_collection_response(\n    server: &Server,\n    request: &PropFind,\n    collection: Collection,\n    access_token: &AccessToken,\n    response: &mut MultiStatus,\n) -> trc::Result<()> {\n    let properties = match request {\n        PropFind::PropName => {\n            response.add_response(Response::new_propstat(\n                DavResourceName::from(collection).collection_path(),\n                vec![PropStat::new_list(vec![\n                    DavPropertyValue::empty(DavProperty::WebDav(WebDavProperty::ResourceType)),\n                    DavPropertyValue::empty(DavProperty::WebDav(\n                        WebDavProperty::CurrentUserPrincipal,\n                    )),\n                    DavPropertyValue::empty(DavProperty::WebDav(\n                        WebDavProperty::SupportedReportSet,\n                    )),\n                ])],\n            ));\n            return Ok(());\n        }\n        PropFind::AllProp(_) => [\n            DavProperty::WebDav(WebDavProperty::ResourceType),\n            DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n            DavProperty::WebDav(WebDavProperty::SupportedReportSet),\n        ]\n        .as_slice(),\n        PropFind::Prop(items) => items,\n    };\n\n    let mut fields = Vec::with_capacity(properties.len());\n    let mut fields_not_found = Vec::new();\n\n    for prop in properties {\n        match &prop {\n            DavProperty::WebDav(WebDavProperty::ResourceType) => {\n                fields.push(DavPropertyValue::new(\n                    prop.clone(),\n                    vec![ResourceType::Collection],\n                ));\n            }\n            DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal) => {\n                fields.push(DavPropertyValue::new(\n                    prop.clone(),\n                    vec![access_token.current_user_principal()],\n                ));\n            }\n            DavProperty::Principal(PrincipalProperty::CalendarHomeSet) => {\n                let hrefs = build_home_set(\n                    server,\n                    access_token,\n                    &access_token.name,\n                    access_token.primary_id,\n                    true,\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n                fields.push(DavPropertyValue::new(prop.clone(), hrefs));\n                response.set_namespace(Namespace::CalDav);\n            }\n            DavProperty::Principal(PrincipalProperty::AddressbookHomeSet) => {\n                let hrefs = build_home_set(\n                    server,\n                    access_token,\n                    &access_token.name,\n                    access_token.primary_id,\n                    false,\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n                fields.push(DavPropertyValue::new(prop.clone(), hrefs));\n                response.set_namespace(Namespace::CardDav);\n            }\n            DavProperty::WebDav(WebDavProperty::SupportedReportSet) => {\n                let reports = match collection {\n                    Collection::Principal => ReportSet::principal(),\n                    Collection::Calendar | Collection::CalendarEvent => ReportSet::calendar(),\n                    Collection::AddressBook | Collection::ContactCard => ReportSet::addressbook(),\n                    _ => ReportSet::file(),\n                };\n\n                fields.push(DavPropertyValue::new(prop.clone(), reports));\n            }\n            _ => {\n                response.set_namespace(prop.namespace());\n                fields_not_found.push(DavPropertyValue::empty(prop.clone()));\n            }\n        }\n    }\n\n    let mut prop_stat = Vec::with_capacity(2);\n\n    if !fields.is_empty() {\n        prop_stat.push(PropStat::new_list(fields));\n    }\n\n    if !fields_not_found.is_empty() {\n        prop_stat.push(PropStat::new_list(fields_not_found).with_status(StatusCode::NOT_FOUND));\n    }\n\n    response.add_response(Response::new_propstat(\n        DavResourceName::from(collection).collection_path(),\n        prop_stat,\n    ));\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/dav/src/common/uri.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{DavError, DavResourceName};\nuse common::{Server, auth::AccessToken};\nuse directory::backend::internal::manage::ManageDirectory;\nuse groupware::cache::GroupwareCache;\nuse http_proto::request::decode_path_element;\nuse hyper::StatusCode;\nuse std::fmt::Display;\nuse trc::AddContext;\nuse types::collection::Collection;\n\n#[derive(Debug)]\npub(crate) struct UriResource<A, R> {\n    pub collection: Collection,\n    pub account_id: A,\n    pub resource: R,\n}\n\npub(crate) enum Urn {\n    Lock(u64),\n    Sync { id: u64, seq: u32 },\n}\n\npub(crate) type UnresolvedUri<'x> = UriResource<Option<u32>, Option<&'x str>>;\npub(crate) type OwnedUri<'x> = UriResource<u32, Option<&'x str>>;\npub(crate) type DocumentUri = UriResource<u32, u32>;\n\npub(crate) trait DavUriResource: Sync + Send {\n    fn validate_uri_with_status<'x>(\n        &self,\n        access_token: &AccessToken,\n        uri: &'x str,\n        error_status: StatusCode,\n    ) -> impl Future<Output = crate::Result<UnresolvedUri<'x>>> + Send;\n\n    fn validate_uri<'x>(\n        &self,\n        access_token: &AccessToken,\n        uri: &'x str,\n    ) -> impl Future<Output = crate::Result<UnresolvedUri<'x>>> + Send;\n\n    fn map_uri_resource(\n        &self,\n        access_token: &AccessToken,\n        uri: OwnedUri<'_>,\n    ) -> impl Future<Output = trc::Result<Option<DocumentUri>>> + Send;\n}\n\nimpl DavUriResource for Server {\n    async fn validate_uri<'x>(\n        &self,\n        access_token: &AccessToken,\n        uri: &'x str,\n    ) -> crate::Result<UnresolvedUri<'x>> {\n        self.validate_uri_with_status(access_token, uri, StatusCode::NOT_FOUND)\n            .await\n    }\n\n    async fn validate_uri_with_status<'x>(\n        &self,\n        access_token: &AccessToken,\n        uri: &'x str,\n        error_status: StatusCode,\n    ) -> crate::Result<UnresolvedUri<'x>> {\n        let (_, uri_parts) = uri\n            .split_once(\"/dav/\")\n            .ok_or(DavError::Code(error_status))?;\n\n        let mut uri_parts = uri_parts\n            .trim_end_matches('/')\n            .splitn(3, '/')\n            .filter(|x| !x.is_empty());\n        let mut resource = UriResource {\n            collection: uri_parts\n                .next()\n                .and_then(DavResourceName::parse)\n                .ok_or(DavError::Code(error_status))?\n                .into(),\n            account_id: None,\n            resource: None,\n        };\n        if let Some(account) = uri_parts.next() {\n            // Parse account id\n            let account_id = if let Some(account_id) = account.strip_prefix('_') {\n                account_id\n                    .parse::<u32>()\n                    .map_err(|_| DavError::Code(error_status))?\n            } else {\n                let account = decode_path_element(account);\n                if access_token.name == account {\n                    access_token.primary_id\n                } else {\n                    self.store()\n                        .get_principal_id(&account)\n                        .await\n                        .caused_by(trc::location!())?\n                        .ok_or(DavError::Code(error_status))?\n                }\n            };\n\n            // Validate access\n            if resource.collection != Collection::Principal\n                && !access_token.has_access(account_id, resource.collection)\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Obtain remaining path\n            resource.account_id = Some(account_id);\n            resource.resource = uri_parts.next();\n        }\n\n        Ok(resource)\n    }\n\n    async fn map_uri_resource(\n        &self,\n        access_token: &AccessToken,\n        uri: OwnedUri<'_>,\n    ) -> trc::Result<Option<DocumentUri>> {\n        if let Some(resource) = uri.resource {\n            if let Some(resource) = self\n                .fetch_dav_resources(access_token, uri.account_id, uri.collection.into())\n                .await\n                .caused_by(trc::location!())?\n                .by_path(resource)\n            {\n                Ok(Some(DocumentUri {\n                    collection: if resource.is_container() {\n                        uri.collection\n                    } else {\n                        uri.collection.child_collection().unwrap_or(uri.collection)\n                    },\n                    account_id: uri.account_id,\n                    resource: resource.document_id(),\n                }))\n            } else {\n                Ok(None)\n            }\n        } else {\n            Ok(None)\n        }\n    }\n}\n\nimpl<'x> UnresolvedUri<'x> {\n    pub fn into_owned_uri(self) -> crate::Result<OwnedUri<'x>> {\n        Ok(OwnedUri {\n            collection: self.collection,\n            account_id: self\n                .account_id\n                .ok_or(DavError::Code(StatusCode::FORBIDDEN))?,\n            resource: self.resource,\n        })\n    }\n}\n\nimpl OwnedUri<'_> {\n    pub fn new_owned(\n        collection: Collection,\n        account_id: u32,\n        resource: Option<&str>,\n    ) -> OwnedUri<'_> {\n        OwnedUri {\n            collection,\n            account_id,\n            resource,\n        }\n    }\n}\n\n/*impl<A, R> UriResource<A, R> {\n    pub fn collection_path(&self) -> &'static str {\n        DavResourceName::from(self.collection).collection_path()\n    }\n}*/\n\nimpl Urn {\n    pub fn try_extract_sync_id(token: &str) -> Option<&str> {\n        token\n            .strip_prefix(\"urn:stalwart:davsync:\")\n            .map(|x| x.split_once(':').map(|(x, _)| x).unwrap_or(x))\n    }\n\n    pub fn parse(input: &str) -> Option<Self> {\n        let inbox = input.strip_prefix(\"urn:stalwart:\")?;\n        let (kind, id) = inbox.split_once(':')?;\n        match kind {\n            \"davlock\" => u64::from_str_radix(id, 16).ok().map(Urn::Lock),\n            \"davsync\" => {\n                if let Some((id, seq)) = id.split_once(':') {\n                    let id = u64::from_str_radix(id, 16).ok()?;\n                    let seq = u32::from_str_radix(seq, 16).ok()?;\n                    Some(Urn::Sync { id, seq })\n                } else {\n                    u64::from_str_radix(id, 16)\n                        .ok()\n                        .map(|id| Urn::Sync { id, seq: 0 })\n                }\n            }\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_lock(&self) -> Option<u64> {\n        match self {\n            Urn::Lock(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_sync(&self) -> Option<(u64, u32)> {\n        match self {\n            Urn::Sync { id, seq } => Some((*id, *seq)),\n            _ => None,\n        }\n    }\n}\n\nimpl Display for Urn {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Urn::Lock(id) => write!(f, \"urn:stalwart:davlock:{id:x}\",),\n            Urn::Sync { id, seq } => {\n                if *seq == 0 {\n                    write!(f, \"urn:stalwart:davsync:{id:x}\")\n                } else {\n                    write!(f, \"urn:stalwart:davsync:{id:x}:{seq:x}\")\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/file/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::FromDavResource;\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::{DavUriResource, UriResource},\n    },\n    file::{DavFileResource, FileItemId},\n};\nuse common::{\n    DavResourcePath, DavResources, Server, auth::AccessToken, storage::index::ObjectIndexBuilder,\n};\nuse dav_proto::{Depth, RequestHeaders};\nuse groupware::{DestroyArchive, cache::GroupwareCache, file::FileNode};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse std::sync::Arc;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse store::{\n    ahash::AHashMap,\n    write::{BatchBuilder, now},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection, VanishedCollection},\n};\n\npub(crate) trait FileCopyMoveRequestHandler: Sync + Send {\n    fn handle_file_copy_move_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_move: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl FileCopyMoveRequestHandler for Server {\n    async fn handle_file_copy_move_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_move: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate source\n        let from_resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let from_account_id = from_resource_.account_id;\n        let from_resources = self\n            .fetch_dav_resources(access_token, from_account_id, SyncCollection::FileNode)\n            .await\n            .caused_by(trc::location!())?;\n        let from_resource = from_resources.map_resource::<FileItemId>(&from_resource_)?;\n        let from_resource_name = from_resource_.resource.unwrap();\n\n        // Validate source ACLs\n        if !access_token.is_member(from_account_id) {\n            let shared = from_resources.shared_containers(\n                access_token,\n                if is_move {\n                    [Acl::Read, Acl::Delete].as_slice().iter().copied()\n                } else {\n                    [Acl::Read].as_slice().iter().copied()\n                },\n                false,\n            );\n\n            for resource in from_resources.subtree(from_resource_.resource.unwrap()) {\n                if !shared.contains(resource.document_id()) {\n                    return Err(DavError::Code(StatusCode::FORBIDDEN));\n                }\n            }\n        }\n\n        // Validate destination\n        let destination = self\n            .validate_uri_with_status(\n                access_token,\n                headers\n                    .destination\n                    .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?,\n                StatusCode::BAD_GATEWAY,\n            )\n            .await?;\n        if destination.collection != Collection::FileNode {\n            return Err(DavError::Code(StatusCode::BAD_GATEWAY));\n        }\n        let to_account_id = destination\n            .account_id\n            .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?;\n        let to_resources = if to_account_id == from_account_id {\n            from_resources.clone()\n        } else {\n            self.fetch_dav_resources(access_token, to_account_id, SyncCollection::FileNode)\n                .await\n                .caused_by(trc::location!())?\n        };\n\n        // Map file item\n        let destination_resource_name = destination\n            .resource\n            .ok_or(DavError::Code(StatusCode::BAD_GATEWAY))?;\n        if from_account_id == to_account_id\n            && (from_resource_name == destination_resource_name\n                || from_resource_name\n                    .strip_prefix(destination_resource_name)\n                    .is_some_and(|v| v.is_empty() || v.starts_with('/')))\n        {\n            return Ok(HttpResponse::new(StatusCode::BAD_GATEWAY));\n        }\n\n        // Check if the resource exists\n        let mut delete_destination = None;\n        let mut destination = if let Some((destination, new_name)) =\n            to_resources.map_parent(destination_resource_name)\n        {\n            if let Some(mut existing_destination) = to_resources\n                .by_path(destination_resource_name)\n                .map(Destination::from_dav_resource)\n            {\n                if !headers.overwrite_fail {\n                    existing_destination.account_id = to_account_id;\n                    delete_destination = Some(existing_destination);\n                } else {\n                    return Ok(HttpResponse::new(StatusCode::PRECONDITION_FAILED));\n                }\n            }\n\n            let mut destination = destination\n                .map(Destination::from_dav_resource)\n                .unwrap_or_default();\n            destination.new_name = Some(new_name.to_string());\n            destination\n        } else {\n            return Err(DavError::Code(StatusCode::CONFLICT));\n        };\n        destination.account_id = to_account_id;\n\n        // Validate destination ACLs\n        if let Some(document_id) = destination.document_id {\n            if let Some(delete_destination) = &delete_destination\n                && !access_token.is_member(to_account_id)\n                && !from_resources.has_access_to_container(\n                    access_token,\n                    delete_destination.document_id.unwrap(),\n                    Acl::Delete,\n                )\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            if !access_token.is_member(to_account_id)\n                && !from_resources.has_access_to_container(access_token, document_id, Acl::Modify)\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n        } else if !access_token.is_member(to_account_id) {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![\n                ResourceState {\n                    account_id: from_account_id,\n                    collection: Collection::FileNode,\n                    document_id: Some(from_resource.resource.document_id),\n                    path: from_resource_name,\n                    ..Default::default()\n                },\n                ResourceState {\n                    account_id: to_account_id,\n                    collection: Collection::FileNode,\n                    document_id: Some(\n                        delete_destination\n                            .as_ref()\n                            .and_then(|d| d.document_id)\n                            .unwrap_or(u32::MAX),\n                    ),\n                    path: destination_resource_name,\n                    ..Default::default()\n                },\n            ],\n            Default::default(),\n            if is_move {\n                DavMethod::MOVE\n            } else {\n                DavMethod::COPY\n            },\n        )\n        .await?;\n\n        if delete_destination.is_none()\n            && from_account_id == destination.account_id\n            && from_resource.resource.parent_id == destination.document_id\n            && destination.new_name.is_some()\n            && is_move\n        {\n            // Rename\n            let from_resource_path = if from_resource.resource.is_container {\n                from_resources.format_collection(from_resource_name)\n            } else {\n                from_resources.format_item(from_resource_name)\n            };\n            return rename_item(\n                self,\n                access_token,\n                from_resource,\n                from_resource_path,\n                destination,\n            )\n            .await;\n        }\n\n        // Validate quota\n        if !is_move || from_account_id != to_account_id {\n            let space_needed = from_resources\n                .subtree(from_resource_name)\n                .map(|a| a.size() as u64)\n                .sum::<u64>();\n            self.has_available_quota(\n                &self.get_resource_token(access_token, to_account_id).await?,\n                space_needed,\n            )\n            .await?;\n        }\n\n        // Delete collection\n        let is_overwrite = delete_destination\n            .as_ref()\n            .is_some_and(|d| d.is_container || from_resource.resource.is_container);\n        if is_overwrite {\n            delete_destination = None;\n            // Find ids to delete\n            let mut ids = to_resources\n                .subtree(destination_resource_name)\n                .collect::<Vec<_>>();\n            if !ids.is_empty() {\n                ids.sort_unstable_by_key(|b| std::cmp::Reverse(b.hierarchy_seq()));\n                let mut sorted_ids = Vec::with_capacity(ids.len());\n                sorted_ids.extend(ids.into_iter().map(|a| a.document_id()));\n                DestroyArchive(sorted_ids)\n                    .delete(self, access_token, destination.account_id, None)\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n        }\n\n        match (from_resource.resource.is_container, is_move) {\n            (true, true) => {\n                move_container(\n                    self,\n                    access_token,\n                    from_resources,\n                    from_resource,\n                    from_resource_name,\n                    destination,\n                    headers.depth,\n                )\n                .await\n            }\n            (true, false) => {\n                copy_container(\n                    self,\n                    access_token,\n                    from_resources,\n                    from_resource,\n                    from_resource_name,\n                    destination,\n                    headers.depth,\n                    false,\n                )\n                .await\n            }\n            (false, true) => {\n                if let Some(delete_destination) = delete_destination {\n                    overwrite_and_delete_item(\n                        self,\n                        access_token,\n                        from_resource,\n                        from_resources.format_item(from_resource_name),\n                        delete_destination,\n                    )\n                    .await\n                } else {\n                    move_item(\n                        self,\n                        access_token,\n                        from_resource,\n                        from_resources.format_item(from_resource_name),\n                        destination,\n                    )\n                    .await\n                }\n            }\n\n            (false, false) => {\n                if let Some(delete_destination) = delete_destination {\n                    overwrite_item(self, access_token, from_resource, delete_destination).await\n                } else {\n                    copy_item(self, access_token, from_resource, destination).await\n                }\n            }\n        }\n        .map(|r| {\n            if is_overwrite && r.status() == StatusCode::CREATED {\n                r.with_status_code(StatusCode::NO_CONTENT)\n            } else {\n                r\n            }\n        })\n    }\n}\n\n#[derive(Debug)]\npub(crate) struct Destination {\n    pub account_id: u32,\n    pub new_name: Option<String>,\n    pub document_id: Option<u32>,\n    pub is_container: bool,\n}\n\nimpl Default for Destination {\n    fn default() -> Self {\n        Self {\n            account_id: Default::default(),\n            document_id: Default::default(),\n            new_name: Default::default(),\n            is_container: true,\n        }\n    }\n}\n\n// Moves a container under an existing container\nasync fn move_container(\n    server: &Server,\n    access_token: &AccessToken,\n    from_resources: Arc<DavResources>,\n    from_resource: UriResource<u32, FileItemId>,\n    from_resource_name: &str,\n    destination: Destination,\n    depth: Depth,\n) -> crate::Result<HttpResponse> {\n    let from_account_id = from_resource.account_id;\n    let to_account_id = destination.account_id;\n    let from_document_id = from_resource.resource.document_id;\n    let parent_id = destination.document_id.map(|id| id + 1).unwrap_or(0);\n\n    if from_account_id == to_account_id {\n        let node_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                from_account_id,\n                Collection::FileNode,\n                from_document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let node = node_\n            .to_unarchived::<FileNode>()\n            .caused_by(trc::location!())?;\n        let mut new_node = node.deserialize::<FileNode>().caused_by(trc::location!())?;\n        new_node.parent_id = parent_id;\n        if let Some(new_name) = destination.new_name {\n            new_node.name = new_name;\n        }\n        let mut batch = BatchBuilder::new();\n        let etag = new_node\n            .update(\n                access_token,\n                node,\n                from_account_id,\n                from_document_id,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?\n            .etag();\n        batch.with_account_id(from_account_id).log_vanished_item(\n            VanishedCollection::FileNode,\n            from_resources.format_collection(from_resource_name),\n        );\n        server\n            .commit_batch(batch)\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n    } else {\n        copy_container(\n            server,\n            access_token,\n            from_resources,\n            from_resource,\n            from_resource_name,\n            destination,\n            depth,\n            true,\n        )\n        .await\n    }\n}\n\n#[allow(clippy::too_many_arguments)]\nasync fn copy_container(\n    server: &Server,\n    access_token: &AccessToken,\n    from_resources: Arc<DavResources>,\n    from_resource: UriResource<u32, FileItemId>,\n    from_resource_name: &str,\n    mut destination: Destination,\n    depth: Depth,\n    delete_source: bool,\n) -> crate::Result<HttpResponse> {\n    let infinity_copy = match depth {\n        Depth::Zero => {\n            return copy_item(server, access_token, from_resource, destination).await;\n        }\n        Depth::One => false,\n        _ => true,\n    };\n\n    let from_account_id = from_resource.account_id;\n    let to_account_id = destination.account_id;\n    let parent_id = destination.document_id.map(|id| id + 1).unwrap_or(0);\n\n    // Obtain files to copy\n    let mut copy_files = if infinity_copy {\n        from_resources\n            .subtree(from_resource_name)\n            .map(|r| (r.document_id(), r.hierarchy_seq()))\n            .collect::<Vec<_>>()\n    } else {\n        from_resources\n            .subtree_with_depth(from_resource_name, 1)\n            .map(|r| (r.document_id(), r.hierarchy_seq()))\n            .collect::<Vec<_>>()\n    };\n\n    // Top-down copy\n    let mut batch = BatchBuilder::new();\n    let mut id_map = AHashMap::with_capacity(copy_files.len());\n    let mut delete_files = if delete_source {\n        Vec::with_capacity(copy_files.len())\n    } else {\n        Vec::new()\n    };\n    copy_files.sort_unstable_by(|a, b| a.1.cmp(&b.1));\n    let now = now() as i64;\n    let mut next_document_id = server\n        .store()\n        .assign_document_ids(to_account_id, Collection::FileNode, copy_files.len() as u64)\n        .await\n        .caused_by(trc::location!())?;\n    for (document_id, _) in copy_files.into_iter() {\n        let node_ = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                from_account_id,\n                Collection::FileNode,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?\n            .into_deserialized::<FileNode>()\n            .caused_by(trc::location!())?;\n\n        // Build node\n        let mut node = if !delete_source {\n            node_.inner\n        } else {\n            let node = node_.inner.clone();\n            delete_files.push((document_id, node_));\n            node\n        };\n        node.modified = now;\n        node.created = now;\n        if let Some(new_name) = destination.new_name.take() {\n            node.name = new_name;\n        }\n        node.parent_id = if let Some(&prev_document_id) = id_map.get(&node.parent_id) {\n            prev_document_id\n        } else {\n            parent_id\n        };\n\n        // Prepare write batch\n        let new_document_id = next_document_id;\n        next_document_id -= 1;\n        batch\n            .with_account_id(to_account_id)\n            .with_collection(Collection::FileNode)\n            .with_document(new_document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_changes(node)\n                    .with_access_token(access_token),\n            )\n            .caused_by(trc::location!())?\n            .commit_point();\n        id_map.insert(document_id + 1, new_document_id + 1);\n    }\n\n    // Delete nodes\n    if !delete_files.is_empty() {\n        for (document_id, node) in delete_files.into_iter().rev() {\n            // Delete record\n            batch\n                .with_account_id(from_account_id)\n                .with_collection(Collection::FileNode)\n                .with_document(document_id)\n                .custom(\n                    ObjectIndexBuilder::<_, ()>::new()\n                        .with_access_token(access_token)\n                        .with_current(node),\n                )\n                .caused_by(trc::location!())?\n                .commit_point();\n        }\n        batch.with_account_id(from_account_id).log_vanished_item(\n            VanishedCollection::FileNode,\n            from_resources.format_collection(from_resource_name),\n        );\n    }\n\n    // Write changes\n    if !batch.is_empty() {\n        server\n            .commit_batch(batch)\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    Ok(HttpResponse::new(StatusCode::CREATED))\n}\n\n// Overwrites the contents of one file with another, then deletes the original\nasync fn overwrite_and_delete_item(\n    server: &Server,\n    access_token: &AccessToken,\n    from_resource: UriResource<u32, FileItemId>,\n    from_resource_path: String,\n    destination: Destination,\n) -> crate::Result<HttpResponse> {\n    let from_account_id = from_resource.account_id;\n    let to_account_id = destination.account_id;\n    let from_document_id = from_resource.resource.document_id;\n    let to_document_id = destination.document_id.unwrap();\n\n    // dest_node is the current file at the destination\n    let dest_node_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            to_account_id,\n            Collection::FileNode,\n            to_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n    let dest_node = dest_node_\n        .to_unarchived::<FileNode>()\n        .caused_by(trc::location!())?;\n\n    // source_node is the file to be copied\n    let source_node__ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::FileNode,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let source_node_ = source_node__\n        .to_unarchived::<FileNode>()\n        .caused_by(trc::location!())?;\n    let mut source_node = source_node_\n        .deserialize::<FileNode>()\n        .caused_by(trc::location!())?;\n    source_node.name = if let Some(new_name) = destination.new_name {\n        new_name\n    } else {\n        dest_node.inner.name.to_string()\n    };\n    source_node.parent_id = dest_node.inner.parent_id.into();\n\n    let mut batch = BatchBuilder::new();\n    let etag = source_node\n        .update(\n            access_token,\n            dest_node,\n            to_account_id,\n            to_document_id,\n            &mut batch,\n        )\n        .caused_by(trc::location!())?\n        .etag();\n    DestroyArchive(source_node_)\n        .delete(\n            access_token,\n            from_account_id,\n            from_document_id,\n            &mut batch,\n            from_resource_path,\n        )\n        .caused_by(trc::location!())?;\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(HttpResponse::new(StatusCode::NO_CONTENT).with_etag_opt(etag))\n}\n\n// Overwrites the contents of one file with another\nasync fn overwrite_item(\n    server: &Server,\n    access_token: &AccessToken,\n    from_resource: UriResource<u32, FileItemId>,\n    destination: Destination,\n) -> crate::Result<HttpResponse> {\n    let from_account_id = from_resource.account_id;\n    let to_account_id = destination.account_id;\n    let from_document_id = from_resource.resource.document_id;\n    let to_document_id = destination.document_id.unwrap();\n\n    // dest_node is the current file at the destination\n    let dest_node_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            to_account_id,\n            Collection::FileNode,\n            to_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n\n    let dest_node = dest_node_\n        .to_unarchived::<FileNode>()\n        .caused_by(trc::location!())?;\n\n    // source_node is the file to be copied\n    let mut source_node = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::FileNode,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?\n        .deserialize::<FileNode>()\n        .caused_by(trc::location!())?;\n    source_node.name = if let Some(new_name) = destination.new_name {\n        new_name\n    } else {\n        dest_node.inner.name.to_string()\n    };\n    source_node.parent_id = dest_node.inner.parent_id.into();\n    let mut batch = BatchBuilder::new();\n    let etag = source_node\n        .update(\n            access_token,\n            dest_node,\n            to_account_id,\n            to_document_id,\n            &mut batch,\n        )\n        .caused_by(trc::location!())?\n        .etag();\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(HttpResponse::new(StatusCode::NO_CONTENT).with_etag_opt(etag))\n}\n\n// Moves an item under an existing container\nasync fn move_item(\n    server: &Server,\n    access_token: &AccessToken,\n    from_resource: UriResource<u32, FileItemId>,\n    from_resource_path: String,\n    destination: Destination,\n) -> crate::Result<HttpResponse> {\n    let from_account_id = from_resource.account_id;\n    let to_account_id = destination.account_id;\n    let from_document_id = from_resource.resource.document_id;\n    let parent_id = destination.document_id.map(|id| id + 1).unwrap_or(0);\n\n    let node_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::FileNode,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let node = node_\n        .to_unarchived::<FileNode>()\n        .caused_by(trc::location!())?;\n    let mut new_node = node.deserialize::<FileNode>().caused_by(trc::location!())?;\n    new_node.parent_id = parent_id;\n    if let Some(new_name) = destination.new_name {\n        new_node.name = new_name;\n    }\n\n    let mut batch = BatchBuilder::new();\n    let etag = if from_account_id == to_account_id {\n        // Destination is in the same account: just update the parent id\n        batch.log_vanished_item(VanishedCollection::FileNode, from_resource_path);\n        new_node\n            .update(\n                access_token,\n                node,\n                from_account_id,\n                from_document_id,\n                &mut batch,\n            )\n            .caused_by(trc::location!())?\n            .etag()\n    } else {\n        // Destination is in a different account: insert a new node, then delete the old one\n        let to_document_id = server\n            .store()\n            .assign_document_ids(to_account_id, Collection::FileNode, 1)\n            .await\n            .caused_by(trc::location!())?;\n        let etag = new_node\n            .insert(access_token, to_account_id, to_document_id, &mut batch)\n            .caused_by(trc::location!())?\n            .etag();\n        DestroyArchive(node)\n            .delete(\n                access_token,\n                from_account_id,\n                from_document_id,\n                &mut batch,\n                from_resource_path,\n            )\n            .caused_by(trc::location!())?;\n        etag\n    };\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n}\n\n// Copies an item under an existing container\nasync fn copy_item(\n    server: &Server,\n    access_token: &AccessToken,\n    from_resource: UriResource<u32, FileItemId>,\n    destination: Destination,\n) -> crate::Result<HttpResponse> {\n    let from_account_id = from_resource.account_id;\n    let to_account_id = destination.account_id;\n    let from_document_id = from_resource.resource.document_id;\n    let parent_id = destination.document_id.map(|id| id + 1).unwrap_or(0);\n\n    let mut node = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::FileNode,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?\n        .deserialize::<FileNode>()\n        .caused_by(trc::location!())?;\n    node.parent_id = parent_id;\n    if let Some(new_name) = destination.new_name {\n        node.name = new_name;\n    }\n    let mut batch = BatchBuilder::new();\n    let to_document_id = server\n        .store()\n        .assign_document_ids(to_account_id, Collection::FileNode, 1)\n        .await\n        .caused_by(trc::location!())?;\n    let etag = node\n        .insert(access_token, to_account_id, to_document_id, &mut batch)\n        .caused_by(trc::location!())?\n        .etag();\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n}\n\n// Renames an item\nasync fn rename_item(\n    server: &Server,\n    access_token: &AccessToken,\n    from_resource: UriResource<u32, FileItemId>,\n    from_resource_path: String,\n    destination: Destination,\n) -> crate::Result<HttpResponse> {\n    let from_account_id = from_resource.account_id;\n    let from_document_id = from_resource.resource.document_id;\n\n    let node_ = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            from_account_id,\n            Collection::FileNode,\n            from_document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n        .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n    let node = node_\n        .to_unarchived::<FileNode>()\n        .caused_by(trc::location!())?;\n    let mut new_node = node.deserialize::<FileNode>().caused_by(trc::location!())?;\n    if let Some(new_name) = destination.new_name {\n        new_node.name = new_name;\n    }\n    let mut batch = BatchBuilder::new();\n    let etag = new_node\n        .update(\n            access_token,\n            node,\n            from_account_id,\n            from_document_id,\n            &mut batch,\n        )\n        .caused_by(trc::location!())?\n        .etag();\n    batch.log_vanished_item(VanishedCollection::FileNode, from_resource_path);\n    server\n        .commit_batch(batch)\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n}\n\nimpl FromDavResource for Destination {\n    fn from_dav_resource(item: DavResourcePath<'_>) -> Self {\n        Destination {\n            account_id: u32::MAX,\n            document_id: Some(item.document_id()),\n            is_container: item.is_container(),\n            new_name: None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/file/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::RequestHeaders;\nuse groupware::{DestroyArchive, cache::GroupwareCache};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse trc::AddContext;\nuse types::{acl::Acl, collection::SyncCollection};\n\npub(crate) trait FileDeleteRequestHandler: Sync + Send {\n    fn handle_file_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl FileDeleteRequestHandler for Server {\n    async fn handle_file_delete_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let delete_path = resource\n            .resource\n            .filter(|r| !r.is_empty())\n            .ok_or(DavError::Code(StatusCode::FORBIDDEN))?;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Find ids to delete\n        let mut ids = resources.subtree(delete_path).collect::<Vec<_>>();\n        if ids.is_empty() {\n            return Err(DavError::Code(StatusCode::NOT_FOUND));\n        }\n\n        // Sort ids descending from the deepest to the root\n        ids.sort_unstable_by_key(|b| std::cmp::Reverse(b.hierarchy_seq()));\n        let (document_id, full_delete_path) = ids\n            .last()\n            .map(|a| (a.document_id(), resources.format_resource(*a)))\n            .unwrap();\n        let mut sorted_ids = Vec::with_capacity(ids.len());\n        sorted_ids.extend(ids.into_iter().map(|a| a.document_id()));\n\n        // Validate ACLs\n        if !access_token.is_member(account_id) {\n            let permissions = resources.shared_containers(access_token, [Acl::Delete], false);\n            if permissions.len() < sorted_ids.len() as u64\n                || !sorted_ids.iter().all(|id| permissions.contains(*id))\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n        }\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: resource.collection,\n                document_id: document_id.into(),\n                path: delete_path,\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::DELETE,\n        )\n        .await?;\n\n        DestroyArchive(sorted_ids)\n            .delete(self, access_token, account_id, full_delete_path.into())\n            .await?;\n\n        Ok(HttpResponse::new(StatusCode::NO_CONTENT))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/file/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        ETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse dav_proto::{RequestHeaders, schema::property::Rfc1123DateTime};\nuse groupware::{cache::GroupwareCache, file::FileNode};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait FileGetRequestHandler: Sync + Send {\n    fn handle_file_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl FileGetRequestHandler for Server {\n    async fn handle_file_get_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        is_head: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let files = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = files.map_resource(&resource_)?;\n\n        // Fetch node\n        let node_ = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::FileNode,\n                resource.resource,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let node = node_.unarchive::<FileNode>().caused_by(trc::location!())?;\n\n        // Validate ACL\n        if !access_token.is_member(account_id)\n            && !node.acls.effective_acl(access_token).contains(Acl::Read)\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        let (hash, size, content_type) = if let Some(file) = node.file.as_ref() {\n            (\n                file.blob_hash.0.as_ref(),\n                u32::from(file.size) as usize,\n                file.media_type.as_ref().map(|s| s.as_str()),\n            )\n        } else {\n            return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n        };\n\n        // Validate headers\n        let etag = node_.etag();\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: resource.collection,\n                document_id: resource.resource.into(),\n                etag: etag.clone().into(),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::GET,\n        )\n        .await?;\n\n        let response = HttpResponse::new(StatusCode::OK)\n            .with_content_type(content_type.unwrap_or(\"application/octet-stream\"))\n            .with_etag(etag)\n            .with_last_modified(Rfc1123DateTime::new(i64::from(node.modified)).to_string());\n\n        if !is_head {\n            Ok(response.with_binary_body(\n                self.blob_store()\n                    .get_blob(hash, 0..usize::MAX)\n                    .await\n                    .caused_by(trc::location!())?\n                    .ok_or(DavError::Code(StatusCode::NOT_FOUND))?,\n            ))\n        } else {\n            Ok(response.with_content_length(size))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/file/mkcol.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::proppatch::FilePropPatchRequestHandler;\nuse crate::{\n    DavMethod, PropStatBuilder,\n    common::{\n        ExtractETag,\n        acl::ResourceAcl,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n};\nuse common::{Server, auth::AccessToken, storage::index::ObjectIndexBuilder};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{Namespace, request::MkCol, response::MkColResponse},\n};\nuse groupware::{cache::GroupwareCache, file::FileNode};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::{BatchBuilder, now};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait FileMkColRequestHandler: Sync + Send {\n    fn handle_file_mkcol_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: Option<MkCol>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl FileMkColRequestHandler for Server {\n    async fn handle_file_mkcol_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: Option<MkCol>,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource_.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = resources.map_parent_resource(&resource_)?;\n\n        // Validate and map parent ACL\n        let parent_id = resources.validate_and_map_parent_acl(\n            access_token,\n            access_token.is_member(account_id),\n            resource.resource.0,\n            Acl::AddItems,\n        )?;\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: resource.collection,\n                document_id: Some(u32::MAX),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::MKCOL,\n        )\n        .await?;\n\n        // Build file container\n        let now = now();\n        let mut node = FileNode {\n            parent_id,\n            name: resource.resource.1.to_string(),\n            display_name: None,\n            file: None,\n            created: now as i64,\n            modified: now as i64,\n            dead_properties: Default::default(),\n            acls: Default::default(),\n        };\n\n        // Apply MKCOL properties\n        let mut return_prop_stat = None;\n        if let Some(mkcol) = request {\n            let mut prop_stat = PropStatBuilder::default();\n            if !self.apply_file_properties(&mut node, false, mkcol.props, &mut prop_stat) {\n                return Ok(HttpResponse::new(StatusCode::FORBIDDEN).with_xml_body(\n                    MkColResponse::new(prop_stat.build())\n                        .with_namespace(Namespace::Dav)\n                        .to_string(),\n                ));\n            }\n            if headers.ret != Return::Minimal {\n                return_prop_stat = Some(prop_stat);\n            }\n        }\n\n        // Prepare write batch\n        let document_id = self\n            .store()\n            .assign_document_ids(account_id, Collection::FileNode, 1)\n            .await\n            .caused_by(trc::location!())?;\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::FileNode)\n            .with_document(document_id)\n            .custom(ObjectIndexBuilder::<(), _>::new().with_changes(node))\n            .caused_by(trc::location!())?;\n        let etag = batch.etag();\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n        if let Some(prop_stat) = return_prop_stat {\n            Ok(HttpResponse::new(StatusCode::CREATED)\n                .with_xml_body(\n                    MkColResponse::new(prop_stat.build())\n                        .with_namespace(Namespace::Dav)\n                        .to_string(),\n                )\n                .with_etag_opt(etag))\n        } else {\n            Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/file/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError,\n    common::uri::{OwnedUri, UriResource},\n};\nuse common::{DavResourcePath, DavResources};\nuse dav_proto::schema::property::{DavProperty, WebDavProperty};\nuse hyper::StatusCode;\n\npub mod copy_move;\npub mod delete;\npub mod get;\npub mod mkcol;\npub mod proppatch;\npub mod update;\n\npub(crate) static FILE_CONTAINER_PROPS: [DavProperty; 19] = [\n    DavProperty::WebDav(WebDavProperty::CreationDate),\n    DavProperty::WebDav(WebDavProperty::DisplayName),\n    DavProperty::WebDav(WebDavProperty::GetETag),\n    DavProperty::WebDav(WebDavProperty::GetLastModified),\n    DavProperty::WebDav(WebDavProperty::ResourceType),\n    DavProperty::WebDav(WebDavProperty::LockDiscovery),\n    DavProperty::WebDav(WebDavProperty::SupportedLock),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n    DavProperty::WebDav(WebDavProperty::SyncToken),\n    DavProperty::WebDav(WebDavProperty::Owner),\n    DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::Acl),\n    DavProperty::WebDav(WebDavProperty::AclRestrictions),\n    DavProperty::WebDav(WebDavProperty::InheritedAclSet),\n    DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n    DavProperty::WebDav(WebDavProperty::SupportedReportSet),\n    DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes),\n    DavProperty::WebDav(WebDavProperty::QuotaUsedBytes),\n];\n\npub(crate) static FILE_ITEM_PROPS: [DavProperty; 19] = [\n    DavProperty::WebDav(WebDavProperty::CreationDate),\n    DavProperty::WebDav(WebDavProperty::DisplayName),\n    DavProperty::WebDav(WebDavProperty::GetETag),\n    DavProperty::WebDav(WebDavProperty::GetLastModified),\n    DavProperty::WebDav(WebDavProperty::ResourceType),\n    DavProperty::WebDav(WebDavProperty::LockDiscovery),\n    DavProperty::WebDav(WebDavProperty::SupportedLock),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n    DavProperty::WebDav(WebDavProperty::SyncToken),\n    DavProperty::WebDav(WebDavProperty::Owner),\n    DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::Acl),\n    DavProperty::WebDav(WebDavProperty::AclRestrictions),\n    DavProperty::WebDav(WebDavProperty::InheritedAclSet),\n    DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n    DavProperty::WebDav(WebDavProperty::GetContentLanguage),\n    DavProperty::WebDav(WebDavProperty::GetContentLength),\n    DavProperty::WebDav(WebDavProperty::GetContentType),\n];\n\npub(crate) trait FromDavResource {\n    fn from_dav_resource(item: DavResourcePath<'_>) -> Self;\n}\n\npub(crate) struct FileItemId {\n    pub document_id: u32,\n    pub parent_id: Option<u32>,\n    pub is_container: bool,\n}\n\npub(crate) trait DavFileResource {\n    fn map_resource<T: FromDavResource>(\n        &self,\n        resource: &OwnedUri<'_>,\n    ) -> crate::Result<UriResource<u32, T>>;\n\n    fn map_parent<'x>(&self, resource: &'x str) -> Option<(Option<DavResourcePath<'_>>, &'x str)>;\n\n    #[allow(clippy::type_complexity)]\n    fn map_parent_resource<'x, T: FromDavResource>(\n        &self,\n        resource: &OwnedUri<'x>,\n    ) -> crate::Result<UriResource<u32, (Option<T>, &'x str)>>;\n}\n\nimpl DavFileResource for DavResources {\n    fn map_resource<T: FromDavResource>(\n        &self,\n        resource: &OwnedUri<'_>,\n    ) -> crate::Result<UriResource<u32, T>> {\n        resource\n            .resource\n            .and_then(|r| self.by_path(r))\n            .map(|r| UriResource {\n                collection: resource.collection,\n                account_id: resource.account_id,\n                resource: T::from_dav_resource(r),\n            })\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))\n    }\n\n    fn map_parent<'x>(&self, resource: &'x str) -> Option<(Option<DavResourcePath<'_>>, &'x str)> {\n        let (parent, child) = if let Some((parent, child)) = resource.rsplit_once('/') {\n            (Some(self.by_path(parent)?), child)\n        } else {\n            (None, resource)\n        };\n\n        Some((parent, child))\n    }\n\n    fn map_parent_resource<'x, T: FromDavResource>(\n        &self,\n        resource: &OwnedUri<'x>,\n    ) -> crate::Result<UriResource<u32, (Option<T>, &'x str)>> {\n        if let Some(r) = resource.resource {\n            if self.by_path(r).is_none() {\n                self.map_parent(r)\n                    .map(|(parent, child)| UriResource {\n                        collection: resource.collection,\n                        account_id: resource.account_id,\n                        resource: (parent.map(T::from_dav_resource), child),\n                    })\n                    .ok_or(DavError::Code(StatusCode::CONFLICT))\n            } else {\n                Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n            }\n        } else {\n            Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n        }\n    }\n}\n\nimpl FromDavResource for u32 {\n    fn from_dav_resource(item: DavResourcePath) -> Self {\n        item.document_id()\n    }\n}\n\nimpl FromDavResource for FileItemId {\n    fn from_dav_resource(item: DavResourcePath) -> Self {\n        FileItemId {\n            document_id: item.document_id(),\n            parent_id: item.parent_id(),\n            is_container: item.is_container(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/file/proppatch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod, PropStatBuilder,\n    common::{\n        ETag, ExtractETag,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse dav_proto::{\n    RequestHeaders, Return,\n    schema::{\n        property::{DavProperty, DavValue, ResourceType, WebDavProperty},\n        request::{DavPropertyValue, PropertyUpdate},\n        response::{BaseCondition, MultiStatus, Response},\n    },\n};\nuse groupware::{cache::GroupwareCache, file::FileNode};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::BatchBuilder;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait FilePropPatchRequestHandler: Sync + Send {\n    fn handle_file_proppatch_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: PropertyUpdate,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n\n    fn apply_file_properties(\n        &self,\n        file: &mut FileNode,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool;\n}\n\nimpl FilePropPatchRequestHandler for Server {\n    async fn handle_file_proppatch_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        mut request: PropertyUpdate,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource_ = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let uri = headers.uri;\n        let account_id = resource_.account_id;\n        let files = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await\n            .caused_by(trc::location!())?;\n        let resource = files.map_resource(&resource_)?;\n\n        if !request.has_changes() {\n            return Ok(HttpResponse::new(StatusCode::NO_CONTENT));\n        }\n\n        // Fetch node\n        let node_ = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::FileNode,\n                resource.resource,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n        let node = node_\n            .to_unarchived::<FileNode>()\n            .caused_by(trc::location!())?;\n\n        // Validate ACL\n        if !access_token.is_member(account_id)\n            && !node\n                .inner\n                .acls\n                .effective_acl(access_token)\n                .contains(Acl::Modify)\n        {\n            return Err(DavError::Code(StatusCode::FORBIDDEN));\n        }\n\n        // Validate headers\n        self.validate_headers(\n            access_token,\n            headers,\n            vec![ResourceState {\n                account_id,\n                collection: resource.collection,\n                document_id: resource.resource.into(),\n                etag: node_.etag().into(),\n                path: resource_.resource.unwrap(),\n                ..Default::default()\n            }],\n            Default::default(),\n            DavMethod::PROPPATCH,\n        )\n        .await?;\n\n        // Deserialize\n        let mut new_node = node.deserialize::<FileNode>().caused_by(trc::location!())?;\n\n        // Remove properties\n        let mut items = PropStatBuilder::default();\n        if !request.set_first && !request.remove.is_empty() {\n            remove_file_properties(\n                &mut new_node,\n                std::mem::take(&mut request.remove),\n                &mut items,\n            );\n        }\n\n        // Set properties\n        let is_success = self.apply_file_properties(&mut new_node, true, request.set, &mut items);\n\n        // Remove properties\n        if is_success && !request.remove.is_empty() {\n            remove_file_properties(&mut new_node, request.remove, &mut items);\n        }\n\n        let etag = if is_success {\n            let mut batch = BatchBuilder::new();\n            let etag = new_node\n                .update(\n                    access_token,\n                    node,\n                    account_id,\n                    resource.resource,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?\n                .etag();\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n            etag\n        } else {\n            node_.etag().into()\n        };\n\n        if headers.ret != Return::Minimal || !is_success {\n            Ok(HttpResponse::new(StatusCode::MULTI_STATUS)\n                .with_xml_body(\n                    MultiStatus::new(vec![Response::new_propstat(uri, items.build())]).to_string(),\n                )\n                .with_etag_opt(etag))\n        } else {\n            Ok(HttpResponse::new(StatusCode::NO_CONTENT).with_etag_opt(etag))\n        }\n    }\n\n    fn apply_file_properties(\n        &self,\n        file: &mut FileNode,\n        is_update: bool,\n        properties: Vec<DavPropertyValue>,\n        items: &mut PropStatBuilder,\n    ) -> bool {\n        let mut has_errors = false;\n\n        for property in properties {\n            match (&property.property, property.value) {\n                (DavProperty::WebDav(WebDavProperty::DisplayName), DavValue::String(name)) => {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        file.display_name = Some(name);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n\n                        has_errors = true;\n                    }\n                }\n                (DavProperty::WebDav(WebDavProperty::CreationDate), DavValue::Timestamp(dt)) => {\n                    file.created = dt;\n                    items.insert_ok(property.property);\n                }\n                (DavProperty::WebDav(WebDavProperty::GetContentType), DavValue::String(name))\n                    if file.file.is_some() =>\n                {\n                    if name.len() <= self.core.groupware.live_property_size {\n                        file.file.as_mut().unwrap().media_type = Some(name);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (\n                    DavProperty::WebDav(WebDavProperty::ResourceType),\n                    DavValue::ResourceTypes(types),\n                ) if file.file.is_none() => {\n                    if types.0.len() != 1 || types.0.first() != Some(&ResourceType::Collection) {\n                        items.insert_precondition_failed(\n                            property.property,\n                            StatusCode::FORBIDDEN,\n                            BaseCondition::ValidResourceType,\n                        );\n                        has_errors = true;\n                    } else {\n                        items.insert_ok(property.property);\n                    }\n                }\n                (DavProperty::DeadProperty(dead), DavValue::DeadProperty(values))\n                    if self.core.groupware.dead_property_size.is_some() =>\n                {\n                    if is_update {\n                        file.dead_properties.remove_element(dead);\n                    }\n\n                    if file.dead_properties.size() + values.size() + dead.size()\n                        < self.core.groupware.dead_property_size.unwrap()\n                    {\n                        file.dead_properties.add_element(dead.clone(), values.0);\n                        items.insert_ok(property.property);\n                    } else {\n                        items.insert_error_with_description(\n                            property.property,\n                            StatusCode::INSUFFICIENT_STORAGE,\n                            \"Property value is too long\",\n                        );\n                        has_errors = true;\n                    }\n                }\n                (_, DavValue::Null) => {\n                    items.insert_ok(property.property);\n                }\n                _ => {\n                    items.insert_error_with_description(\n                        property.property,\n                        StatusCode::CONFLICT,\n                        \"Property cannot be modified\",\n                    );\n                    has_errors = true;\n                }\n            }\n        }\n\n        !has_errors\n    }\n}\n\nfn remove_file_properties(\n    node: &mut FileNode,\n    properties: Vec<DavProperty>,\n    items: &mut PropStatBuilder,\n) {\n    for property in properties {\n        match &property {\n            DavProperty::WebDav(WebDavProperty::DisplayName) => {\n                node.display_name = None;\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::WebDav(WebDavProperty::GetContentType) if node.file.is_some() => {\n                node.file.as_mut().unwrap().media_type = None;\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            DavProperty::DeadProperty(dead) => {\n                node.dead_properties.remove_element(dead);\n                items.insert_with_status(property, StatusCode::NO_CONTENT);\n            }\n            _ => {\n                items.insert_error_with_description(\n                    property,\n                    StatusCode::CONFLICT,\n                    \"Property cannot be deleted\",\n                );\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/file/update.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavMethod,\n    common::{\n        ETag, ExtractETag,\n        acl::ResourceAcl,\n        lock::{LockRequestHandler, ResourceState},\n        uri::DavUriResource,\n    },\n    file::DavFileResource,\n};\nuse common::{\n    Server, auth::AccessToken, sharing::EffectiveAcl, storage::index::ObjectIndexBuilder,\n};\nuse dav_proto::{RequestHeaders, Return, schema::property::Rfc1123DateTime};\nuse groupware::{\n    cache::GroupwareCache,\n    file::{FileNode, FileProperties},\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::write::{BatchBuilder, now};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection},\n};\n\npub(crate) trait FileUpdateRequestHandler: Sync + Send {\n    fn handle_file_update_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n        is_patch: bool,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl FileUpdateRequestHandler for Server {\n    async fn handle_file_update_request(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        bytes: Vec<u8>,\n        _is_patch: bool,\n    ) -> crate::Result<HttpResponse> {\n        // Validate URI\n        let resource = self\n            .validate_uri(access_token, headers.uri)\n            .await?\n            .into_owned_uri()?;\n        let account_id = resource.account_id;\n        let resources = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await\n            .caused_by(trc::location!())?;\n        let resource_name = resource\n            .resource\n            .ok_or(DavError::Code(StatusCode::CONFLICT))?;\n\n        if bytes.len() > self.core.groupware.max_file_size {\n            return Err(DavError::Code(StatusCode::PAYLOAD_TOO_LARGE));\n        }\n\n        if let Some(document_id) = resources\n            .by_path(resource_name.as_ref())\n            .map(|r| r.document_id())\n        {\n            // Update\n            let node_ = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::FileNode,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or(DavError::Code(StatusCode::NOT_FOUND))?;\n            let node = node_\n                .to_unarchived::<FileNode>()\n                .caused_by(trc::location!())?;\n\n            // Validate ACL\n            if !access_token.is_member(account_id)\n                && !node\n                    .inner\n                    .acls\n                    .effective_acl(access_token)\n                    .contains(Acl::Modify)\n            {\n                return Err(DavError::Code(StatusCode::FORBIDDEN));\n            }\n\n            // Validate headers\n            match self\n                .validate_headers(\n                    access_token,\n                    headers,\n                    vec![ResourceState {\n                        account_id,\n                        collection: resource.collection,\n                        document_id: Some(document_id),\n                        etag: node.etag().into(),\n                        path: resource_name,\n                        ..Default::default()\n                    }],\n                    Default::default(),\n                    DavMethod::PUT,\n                )\n                .await\n            {\n                Ok(_) => {}\n                Err(DavError::Code(StatusCode::PRECONDITION_FAILED))\n                    if headers.ret == Return::Representation =>\n                {\n                    let file = node.inner.file.as_ref().unwrap();\n                    let contents = self\n                        .blob_store()\n                        .get_blob(file.blob_hash.0.as_slice(), 0..usize::MAX)\n                        .await\n                        .caused_by(trc::location!())?\n                        .ok_or(DavError::Code(StatusCode::PRECONDITION_FAILED))?;\n\n                    return Ok(HttpResponse::new(StatusCode::PRECONDITION_FAILED)\n                        .with_content_type(\n                            file.media_type\n                                .as_ref()\n                                .map(|v| v.as_str())\n                                .unwrap_or(\"application/octet-stream\"),\n                        )\n                        .with_etag(node.etag())\n                        .with_last_modified(\n                            Rfc1123DateTime::new(i64::from(node.inner.modified)).to_string(),\n                        )\n                        .with_header(\"Preference-Applied\", \"return=representation\")\n                        .with_binary_body(contents));\n                }\n                Err(e) => return Err(e),\n            }\n\n            // Verify that the node is a file\n            if let Some(file) = node.inner.file.as_ref() {\n                if BlobHash::generate(&bytes).as_slice() == file.blob_hash.0.as_slice() {\n                    return Ok(HttpResponse::new(StatusCode::NO_CONTENT));\n                }\n            } else {\n                return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n            }\n\n            // Validate quota\n            let extra_bytes = (bytes.len() as u64)\n                .saturating_sub(u32::from(node.inner.file.as_ref().unwrap().size) as u64);\n            if extra_bytes > 0 {\n                self.has_available_quota(\n                    &self.get_resource_token(access_token, account_id).await?,\n                    extra_bytes,\n                )\n                .await?;\n            }\n\n            // Write blob\n            let (blob_hash, blob_hold) = self\n                .put_temporary_blob(account_id, &bytes, 60)\n                .await\n                .caused_by(trc::location!())?;\n\n            // Build node\n            let mut new_node = node.deserialize::<FileNode>().caused_by(trc::location!())?;\n            let new_file = new_node.file.as_mut().unwrap();\n            new_file.blob_hash = blob_hash;\n            new_file.media_type = headers\n                .content_type\n                .filter(|ct| !ct.is_empty() && *ct != \"application/octet-stream\")\n                .map(|v| v.to_string());\n            new_file.size = bytes.len() as u32;\n            new_node.modified = now() as i64;\n\n            // Prepare write batch\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::FileNode)\n                .with_document(document_id)\n                .clear(blob_hold)\n                .custom(\n                    ObjectIndexBuilder::new()\n                        .with_current(node)\n                        .with_changes(new_node)\n                        .with_access_token(access_token),\n                )\n                .caused_by(trc::location!())?;\n            let etag = batch.etag();\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n            Ok(HttpResponse::new(StatusCode::NO_CONTENT).with_etag_opt(etag))\n        } else {\n            // Insert\n            let orig_resource_name = resource_name;\n            let (parent, resource_name) = resources\n                .map_parent(orig_resource_name.as_ref())\n                .ok_or(DavError::Code(StatusCode::CONFLICT))?;\n\n            // Validate ACL\n            let parent_id = resources.validate_and_map_parent_acl(\n                access_token,\n                access_token.is_member(account_id),\n                parent.map(|r| r.document_id()),\n                Acl::AddItems,\n            )?;\n\n            // Verify that parent is a collection\n            if parent.as_ref().is_some_and(|r| !r.is_container()) {\n                return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED));\n            }\n\n            // Validate headers\n            self.validate_headers(\n                access_token,\n                headers,\n                vec![ResourceState {\n                    account_id,\n                    collection: resource.collection,\n                    document_id: Some(u32::MAX),\n                    path: orig_resource_name,\n                    ..Default::default()\n                }],\n                Default::default(),\n                DavMethod::PUT,\n            )\n            .await?;\n\n            // Validate quota\n            if !bytes.is_empty() {\n                self.has_available_quota(\n                    &self.get_resource_token(access_token, account_id).await?,\n                    bytes.len() as u64,\n                )\n                .await?;\n            }\n\n            // Write blob\n            let (blob_hash, blob_hold) = self\n                .put_temporary_blob(account_id, &bytes, 60)\n                .await\n                .caused_by(trc::location!())?;\n\n            // Build node\n            let now = now();\n            let node = FileNode {\n                parent_id,\n                name: resource_name.to_string(),\n                display_name: None,\n                file: Some(FileProperties {\n                    blob_hash,\n                    size: bytes.len() as u32,\n                    media_type: headers.content_type.map(|v| v.to_string()),\n                    executable: false,\n                }),\n                created: now as i64,\n                modified: now as i64,\n                dead_properties: Default::default(),\n                acls: parent\n                    .as_ref()\n                    .and_then(|p| p.resource.acls())\n                    .map(|acls| acls.to_vec())\n                    .unwrap_or_default(),\n            };\n\n            // Prepare write batch\n            let mut batch = BatchBuilder::new();\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::FileNode, 1)\n                .await\n                .caused_by(trc::location!())?;\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::FileNode)\n                .with_document(document_id)\n                .clear(blob_hold)\n                .custom(\n                    ObjectIndexBuilder::<(), _>::new()\n                        .with_changes(node)\n                        .with_access_token(access_token),\n                )\n                .caused_by(trc::location!())?;\n            let etag = batch.etag();\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n            Ok(HttpResponse::new(StatusCode::CREATED).with_etag_opt(etag))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n#![warn(clippy::large_futures)]\n\npub mod calendar;\npub mod card;\npub mod common;\npub mod file;\npub mod principal;\npub mod request;\n\nuse dav_proto::schema::{\n    request::DavPropertyValue,\n    response::{Condition, List, Prop, PropStat, ResponseDescription, Status},\n};\nuse groupware::{DavResourceName, RFC_3986};\nuse hyper::{Method, StatusCode};\nuse std::borrow::Cow;\nuse store::ahash::AHashMap;\npub(crate) type Result<T> = std::result::Result<T, DavError>;\n\n#[derive(Debug, Clone, Copy)]\npub enum DavMethod {\n    GET,\n    PUT,\n    POST,\n    DELETE,\n    HEAD,\n    PATCH,\n    PROPFIND,\n    PROPPATCH,\n    REPORT,\n    MKCOL,\n    MKCALENDAR,\n    COPY,\n    MOVE,\n    LOCK,\n    UNLOCK,\n    OPTIONS,\n    ACL,\n}\n\nimpl From<DavMethod> for trc::WebDavEvent {\n    fn from(value: DavMethod) -> Self {\n        match value {\n            DavMethod::GET => trc::WebDavEvent::Get,\n            DavMethod::PUT => trc::WebDavEvent::Put,\n            DavMethod::POST => trc::WebDavEvent::Post,\n            DavMethod::DELETE => trc::WebDavEvent::Delete,\n            DavMethod::HEAD => trc::WebDavEvent::Head,\n            DavMethod::PATCH => trc::WebDavEvent::Patch,\n            DavMethod::PROPFIND => trc::WebDavEvent::Propfind,\n            DavMethod::PROPPATCH => trc::WebDavEvent::Proppatch,\n            DavMethod::REPORT => trc::WebDavEvent::Report,\n            DavMethod::MKCOL => trc::WebDavEvent::Mkcol,\n            DavMethod::MKCALENDAR => trc::WebDavEvent::Mkcalendar,\n            DavMethod::COPY => trc::WebDavEvent::Copy,\n            DavMethod::MOVE => trc::WebDavEvent::Move,\n            DavMethod::LOCK => trc::WebDavEvent::Lock,\n            DavMethod::UNLOCK => trc::WebDavEvent::Unlock,\n            DavMethod::OPTIONS => trc::WebDavEvent::Options,\n            DavMethod::ACL => trc::WebDavEvent::Acl,\n        }\n    }\n}\n\npub(crate) enum DavError {\n    Parse(dav_proto::parser::Error),\n    Internal(trc::Error),\n    Condition(DavErrorCondition),\n    Code(StatusCode),\n}\n\nstruct DavErrorCondition {\n    pub code: StatusCode,\n    pub condition: Condition,\n    pub details: Option<String>,\n}\n\nimpl From<DavErrorCondition> for DavError {\n    fn from(value: DavErrorCondition) -> Self {\n        DavError::Condition(value)\n    }\n}\n\nimpl From<Condition> for DavErrorCondition {\n    fn from(value: Condition) -> Self {\n        DavErrorCondition {\n            code: StatusCode::CONFLICT,\n            condition: value,\n            details: None,\n        }\n    }\n}\n\nimpl DavErrorCondition {\n    pub fn new(code: StatusCode, condition: impl Into<Condition>) -> Self {\n        DavErrorCondition {\n            code,\n            condition: condition.into(),\n            details: None,\n        }\n    }\n\n    pub fn with_details(mut self, details: impl Into<String>) -> Self {\n        self.details = Some(details.into());\n        self\n    }\n}\n\nimpl DavMethod {\n    pub fn parse(method: &Method) -> Option<Self> {\n        match *method {\n            Method::GET => Some(DavMethod::GET),\n            Method::PUT => Some(DavMethod::PUT),\n            Method::DELETE => Some(DavMethod::DELETE),\n            Method::OPTIONS => Some(DavMethod::OPTIONS),\n            Method::POST => Some(DavMethod::POST),\n            Method::PATCH => Some(DavMethod::PATCH),\n            Method::HEAD => Some(DavMethod::HEAD),\n            _ => {\n                hashify::tiny_map!(method.as_str().as_bytes(),\n                    \"PROPFIND\" => DavMethod::PROPFIND,\n                    \"PROPPATCH\" => DavMethod::PROPPATCH,\n                    \"REPORT\" => DavMethod::REPORT,\n                    \"MKCOL\" => DavMethod::MKCOL,\n                    \"MKCALENDAR\" => DavMethod::MKCALENDAR,\n                    \"COPY\" => DavMethod::COPY,\n                    \"MOVE\" => DavMethod::MOVE,\n                    \"LOCK\" => DavMethod::LOCK,\n                    \"UNLOCK\" => DavMethod::UNLOCK,\n                    \"ACL\" => DavMethod::ACL\n                )\n            }\n        }\n    }\n\n    #[inline]\n    pub fn has_body(self) -> bool {\n        matches!(\n            self,\n            DavMethod::PUT\n                | DavMethod::POST\n                | DavMethod::PATCH\n                | DavMethod::PROPPATCH\n                | DavMethod::PROPFIND\n                | DavMethod::REPORT\n                | DavMethod::LOCK\n                | DavMethod::ACL\n                | DavMethod::MKCALENDAR\n        )\n    }\n}\n\n#[derive(Debug, Default)]\npub struct PropStatBuilder {\n    propstats: AHashMap<(StatusCode, Option<Condition>, Option<String>), Vec<DavPropertyValue>>,\n}\n\nimpl PropStatBuilder {\n    pub fn insert_ok(&mut self, prop: impl Into<DavPropertyValue>) -> &mut Self {\n        self.propstats\n            .entry((StatusCode::OK, None, None))\n            .or_default()\n            .push(prop.into());\n        self\n    }\n\n    pub fn insert_with_status(\n        &mut self,\n        prop: impl Into<DavPropertyValue>,\n        status: StatusCode,\n    ) -> &mut Self {\n        self.propstats\n            .entry((status, None, None))\n            .or_default()\n            .push(prop.into());\n        self\n    }\n\n    pub fn insert_error_with_description(\n        &mut self,\n        prop: impl Into<DavPropertyValue>,\n        status: StatusCode,\n        description: impl Into<String>,\n    ) -> &mut Self {\n        self.propstats\n            .entry((status, None, Some(description.into())))\n            .or_default()\n            .push(prop.into());\n        self\n    }\n\n    pub fn insert_precondition_failed(\n        &mut self,\n        prop: impl Into<DavPropertyValue>,\n        status: StatusCode,\n        condition: impl Into<Condition>,\n    ) -> &mut Self {\n        self.propstats\n            .entry((status, Some(condition.into()), None))\n            .or_default()\n            .push(prop.into());\n        self\n    }\n\n    pub fn insert_precondition_failed_with_description(\n        &mut self,\n        prop: impl Into<DavPropertyValue>,\n        status: StatusCode,\n        condition: impl Into<Condition>,\n        description: impl Into<String>,\n    ) -> &mut Self {\n        self.propstats\n            .entry((status, Some(condition.into()), Some(description.into())))\n            .or_default()\n            .push(prop.into());\n        self\n    }\n\n    pub fn build(self) -> Vec<PropStat> {\n        self.propstats\n            .into_iter()\n            .map(|((status, condition, description), props)| PropStat {\n                prop: Prop(List(props)),\n                status: Status(status),\n                error: condition,\n                response_description: description.map(ResponseDescription),\n            })\n            .collect()\n    }\n}\n\n// Workaround for Apple bug with missing percent encoding in paths\npub(crate) fn fix_percent_encoding(path: &'_ str) -> Cow<'_, str> {\n    let (parent, name) = if let Some((parent, name)) = path.rsplit_once('/') {\n        (Some(parent), name)\n    } else {\n        (None, path)\n    };\n\n    for &ch in name.as_bytes() {\n        if !matches!(ch, b'0'..=b'9' | b'A'..=b'Z' | b'a'..=b'z' | b'-' | b'.' | b'_' | b'~' | b'%')\n        {\n            let name = percent_encoding::percent_encode(name.as_bytes(), RFC_3986);\n\n            return if let Some(parent) = parent {\n                Cow::Owned(format!(\"{parent}/{name}\"))\n            } else {\n                Cow::Owned(name.to_string())\n            };\n        }\n    }\n\n    path.into()\n}\n"
  },
  {
    "path": "crates/dav/src/principal/matching.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::propfind::PrincipalPropFind;\nuse crate::{\n    DavError,\n    common::{\n        DavQuery, DavQueryResource,\n        propfind::PropFindRequestHandler,\n        uri::{DavUriResource, UriResource},\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::{\n    RequestHeaders,\n    schema::{\n        property::{DavProperty, WebDavProperty},\n        request::{PrincipalMatch, PropFind},\n        response::MultiStatus,\n    },\n};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::roaring::RoaringBitmap;\nuse types::collection::Collection;\n\npub(crate) trait PrincipalMatching: Sync + Send {\n    fn handle_principal_match(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        request: PrincipalMatch,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl PrincipalMatching for Server {\n    async fn handle_principal_match(\n        &self,\n        access_token: &AccessToken,\n        headers: &RequestHeaders<'_>,\n        mut request: PrincipalMatch,\n    ) -> crate::Result<HttpResponse> {\n        let resource = self.validate_uri(access_token, headers.uri).await?;\n\n        match resource.collection {\n            Collection::AddressBook | Collection::Calendar | Collection::FileNode => {\n                if request.properties.is_empty() {\n                    request\n                        .properties\n                        .push(DavProperty::WebDav(WebDavProperty::Owner));\n                }\n                if let Some(account_id) = resource.account_id {\n                    return self\n                        .handle_dav_query(\n                            access_token,\n                            DavQuery {\n                                resource: DavQueryResource::Uri(UriResource {\n                                    collection: resource.collection,\n                                    account_id,\n                                    resource: resource.resource,\n                                }),\n                                propfind: PropFind::Prop(request.properties),\n                                depth: usize::MAX,\n                                ret: headers.ret,\n                                depth_no_root: headers.depth_no_root,\n                                uri: headers.uri,\n                                sync_type: Default::default(),\n                                limit: Default::default(),\n                                max_vcard_version: Default::default(),\n                                expand: Default::default(),\n                            },\n                        )\n                        .await;\n                }\n            }\n            Collection::Principal => {}\n            _ => return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n        }\n\n        let mut response = MultiStatus::new(Vec::with_capacity(16));\n        if request.properties.is_empty() {\n            request\n                .properties\n                .push(DavProperty::WebDav(WebDavProperty::DisplayName));\n        }\n        let request = PropFind::Prop(request.properties);\n        self.prepare_principal_propfind_response(\n            access_token,\n            resource.collection,\n            RoaringBitmap::from_iter(access_token.all_ids()).into_iter(),\n            &request,\n            &mut response,\n        )\n        .await?;\n        Ok(HttpResponse::new(StatusCode::MULTI_STATUS).with_xml_body(response.to_string()))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/principal/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::auth::AccessToken;\nuse dav_proto::schema::response::Href;\nuse groupware::RFC_3986;\n\nuse crate::DavResourceName;\n\npub mod matching;\npub mod propfind;\npub mod propsearch;\n\npub trait CurrentUserPrincipal {\n    fn current_user_principal(&self) -> Href;\n}\n\nimpl CurrentUserPrincipal for AccessToken {\n    fn current_user_principal(&self) -> Href {\n        Href(format!(\n            \"{}/{}/\",\n            DavResourceName::Principal.base_path(),\n            percent_encoding::utf8_percent_encode(&self.name, RFC_3986)\n        ))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/principal/propfind.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::CurrentUserPrincipal;\nuse crate::{\n    DavResourceName,\n    common::propfind::{PropFindRequestHandler, SyncTokenUrn},\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::schema::{\n    Namespace,\n    property::{\n        DavProperty, DavValue, PrincipalProperty, Privilege, ReportSet, ResourceType,\n        WebDavProperty,\n    },\n    request::{DavPropertyValue, PropFind},\n    response::{Href, MultiStatus, PropStat, Response},\n};\nuse directory::{PrincipalData, QueryParams, Type, backend::internal::manage::ManageDirectory};\nuse groupware::RFC_3986;\nuse groupware::cache::GroupwareCache;\nuse hyper::StatusCode;\nuse std::borrow::Cow;\nuse trc::AddContext;\nuse types::collection::Collection;\n\npub(crate) trait PrincipalPropFind: Sync + Send {\n    fn prepare_principal_propfind_response(\n        &self,\n        access_token: &AccessToken,\n        collection: Collection,\n        documents: impl Iterator<Item = u32> + Sync + Send,\n        request: &PropFind,\n        response: &mut MultiStatus,\n    ) -> impl Future<Output = crate::Result<()>> + Send;\n\n    fn expand_principal(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        propfind: &PropFind,\n    ) -> impl Future<Output = crate::Result<Option<Response>>> + Send;\n\n    fn owner_href(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Href>> + Send;\n}\n\nimpl PrincipalPropFind for Server {\n    async fn prepare_principal_propfind_response(\n        &self,\n        access_token: &AccessToken,\n        collection: Collection,\n        account_ids: impl Iterator<Item = u32> + Sync + Send,\n        request: &PropFind,\n        response: &mut MultiStatus,\n    ) -> crate::Result<()> {\n        let properties = match request {\n            PropFind::PropName => {\n                let props = all_props(collection, None);\n                for account_id in account_ids {\n                    response.add_response(Response::new_propstat(\n                        self.owner_href(access_token, account_id)\n                            .await\n                            .caused_by(trc::location!())?,\n                        vec![PropStat::new_list(\n                            props.iter().cloned().map(DavPropertyValue::empty).collect(),\n                        )],\n                    ));\n                }\n                return Ok(());\n            }\n            PropFind::AllProp(items) => Cow::Owned(all_props(collection, items.as_slice().into())),\n            PropFind::Prop(items) => Cow::Borrowed(items),\n        };\n        let is_principal = match collection {\n            Collection::AddressBook | Collection::ContactCard => {\n                response.set_namespace(Namespace::CardDav);\n                false\n            }\n            Collection::Calendar\n            | Collection::CalendarEvent\n            | Collection::CalendarEventNotification => {\n                response.set_namespace(Namespace::CalDav);\n                false\n            }\n            Collection::Principal => true,\n            _ => false,\n        };\n        let base_path = DavResourceName::from(collection).base_path();\n        let needs_quota = properties.iter().any(|property| {\n            matches!(\n                property,\n                DavProperty::WebDav(\n                    WebDavProperty::QuotaAvailableBytes | WebDavProperty::QuotaUsedBytes\n                )\n            )\n        });\n\n        for account_id in account_ids {\n            let mut fields = Vec::with_capacity(properties.len());\n            let mut fields_not_found = Vec::new();\n\n            let (name, description, emails, typ) = if access_token.primary_id() == account_id {\n                (\n                    Cow::Borrowed(access_token.name.as_str()),\n                    access_token\n                        .description\n                        .as_deref()\n                        .unwrap_or(&access_token.name)\n                        .to_string(),\n                    Cow::Borrowed(access_token.emails.as_slice()),\n                    Type::Individual,\n                )\n            } else {\n                self.directory()\n                    .query(QueryParams::id(account_id).with_return_member_of(false))\n                    .await\n                    .caused_by(trc::location!())?\n                    .map(|p| {\n                        let name = p.name;\n                        let mut description = None;\n                        let mut emails = Vec::new();\n                        for data in p.data {\n                            match data {\n                                PrincipalData::Description(desc) => {\n                                    description = Some(desc);\n                                }\n                                PrincipalData::PrimaryEmail(email) => {\n                                    if emails.is_empty() {\n                                        emails.push(email);\n                                    } else {\n                                        emails.insert(0, email);\n                                    }\n                                }\n                                PrincipalData::EmailAlias(email) => {\n                                    emails.push(email);\n                                }\n                                _ => {}\n                            }\n                        }\n\n                        let description = description.unwrap_or_else(|| name.clone());\n                        (\n                            Cow::Owned(name.to_string()),\n                            description,\n                            Cow::Owned(emails),\n                            p.typ,\n                        )\n                    })\n                    .unwrap_or_else(|| {\n                        (\n                            Cow::Owned(format!(\"_{}\", account_id)),\n                            format!(\"_{}\", account_id),\n                            Cow::Owned(vec![]),\n                            Type::Individual,\n                        )\n                    })\n            };\n\n            // Fetch quota\n            let quota = if needs_quota {\n                self.dav_quota(access_token, account_id)\n                    .await\n                    .caused_by(trc::location!())?\n            } else {\n                Default::default()\n            };\n\n            for property in properties.as_slice() {\n                match property {\n                    DavProperty::WebDav(dav_property) => match dav_property {\n                        WebDavProperty::DisplayName => {\n                            fields\n                                .push(DavPropertyValue::new(property.clone(), description.clone()));\n                        }\n                        WebDavProperty::ResourceType => {\n                            let resource_type = if !is_principal {\n                                vec![ResourceType::Collection]\n                            } else {\n                                vec![ResourceType::Principal, ResourceType::Collection]\n                            };\n\n                            fields.push(DavPropertyValue::new(property.clone(), resource_type));\n                        }\n                        WebDavProperty::SupportedReportSet => {\n                            let reports = match collection {\n                                Collection::Principal => ReportSet::principal(),\n                                Collection::Calendar | Collection::CalendarEvent => {\n                                    ReportSet::calendar()\n                                }\n                                Collection::AddressBook | Collection::ContactCard => {\n                                    ReportSet::addressbook()\n                                }\n                                _ => ReportSet::file(),\n                            };\n\n                            fields.push(DavPropertyValue::new(property.clone(), reports));\n                        }\n                        WebDavProperty::CurrentUserPrincipal => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                vec![access_token.current_user_principal()],\n                            ));\n                        }\n                        WebDavProperty::QuotaAvailableBytes if !is_principal => {\n                            fields.push(DavPropertyValue::new(property.clone(), quota.available));\n                        }\n                        WebDavProperty::QuotaUsedBytes if !is_principal => {\n                            fields.push(DavPropertyValue::new(property.clone(), quota.used));\n                        }\n                        WebDavProperty::SyncToken if !is_principal => {\n                            let sync_token = self\n                                .fetch_dav_resources(access_token, account_id, collection.into())\n                                .await\n                                .caused_by(trc::location!())?\n                                .sync_token();\n\n                            fields.push(DavPropertyValue::new(property.clone(), sync_token));\n                        }\n                        WebDavProperty::GetCTag if !is_principal => {\n                            let ctag = self\n                                .fetch_dav_resources(access_token, account_id, collection.into())\n                                .await\n                                .caused_by(trc::location!())?\n                                .highest_change_id;\n\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                DavValue::String(format!(\"\\\"{ctag}\\\"\")),\n                            ));\n                        }\n                        WebDavProperty::Owner => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                vec![Href(format!(\n                                    \"{}/{}/\",\n                                    DavResourceName::Principal.base_path(),\n                                    percent_encoding::utf8_percent_encode(&name, RFC_3986),\n                                ))],\n                            ));\n                        }\n                        WebDavProperty::Group if !is_principal => {\n                            fields.push(DavPropertyValue::empty(property.clone()));\n                        }\n                        WebDavProperty::CurrentUserPrivilegeSet if !is_principal => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                if access_token.is_member(account_id) {\n                                    Privilege::all(matches!(\n                                        collection,\n                                        Collection::Calendar | Collection::CalendarEvent\n                                    ))\n                                } else {\n                                    vec![Privilege::Read]\n                                },\n                            ));\n                        }\n                        WebDavProperty::PrincipalCollectionSet => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                vec![Href(\n                                    DavResourceName::Principal.collection_path().to_string(),\n                                )],\n                            ));\n                        }\n                        _ => {\n                            response.set_namespace(property.namespace());\n                            fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                        }\n                    },\n                    DavProperty::Principal(principal_property) => match principal_property {\n                        PrincipalProperty::AlternateURISet\n                        | PrincipalProperty::GroupMemberSet\n                        | PrincipalProperty::GroupMembership => {\n                            fields.push(DavPropertyValue::empty(property.clone()));\n                        }\n                        PrincipalProperty::PrincipalURL => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                vec![Href(format!(\n                                    \"{}/{}/\",\n                                    DavResourceName::Principal.base_path(),\n                                    percent_encoding::utf8_percent_encode(&name, RFC_3986),\n                                ))],\n                            ));\n                        }\n                        PrincipalProperty::CalendarHomeSet => {\n                            let hrefs =\n                                build_home_set(self, access_token, name.as_ref(), account_id, true)\n                                    .await\n                                    .caused_by(trc::location!())?;\n\n                            fields.push(DavPropertyValue::new(property.clone(), hrefs));\n                            response.set_namespace(Namespace::CalDav);\n                        }\n                        PrincipalProperty::AddressbookHomeSet => {\n                            let hrefs = build_home_set(\n                                self,\n                                access_token,\n                                name.as_ref(),\n                                account_id,\n                                false,\n                            )\n                            .await\n                            .caused_by(trc::location!())?;\n\n                            fields.push(DavPropertyValue::new(property.clone(), hrefs));\n                            response.set_namespace(Namespace::CardDav);\n                        }\n\n                        PrincipalProperty::PrincipalAddress => {\n                            fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                            response.set_namespace(Namespace::CardDav);\n                        }\n                        PrincipalProperty::CalendarUserAddressSet => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                emails\n                                    .iter()\n                                    .filter(|email| !email.starts_with(\"@\"))\n                                    .take(1)\n                                    .map(|email| Href(format!(\"mailto:{email}\",)))\n                                    .collect::<Vec<_>>(),\n                            ));\n                            response.set_namespace(Namespace::CalDav);\n                        }\n                        PrincipalProperty::CalendarUserType => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                typ.as_str().to_uppercase(),\n                            ));\n                            response.set_namespace(Namespace::CalDav);\n                        }\n                        PrincipalProperty::ScheduleInboxURL => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                vec![Href(format!(\n                                    \"{}/{}/inbox/\",\n                                    DavResourceName::Scheduling.base_path(),\n                                    percent_encoding::utf8_percent_encode(&name, RFC_3986),\n                                ))],\n                            ));\n                            response.set_namespace(Namespace::CalDav);\n                        }\n                        PrincipalProperty::ScheduleOutboxURL => {\n                            fields.push(DavPropertyValue::new(\n                                property.clone(),\n                                vec![Href(format!(\n                                    \"{}/{}/outbox/\",\n                                    DavResourceName::Scheduling.base_path(),\n                                    percent_encoding::utf8_percent_encode(&name, RFC_3986),\n                                ))],\n                            ));\n                            response.set_namespace(Namespace::CalDav);\n                        }\n                    },\n                    _ => {\n                        response.set_namespace(property.namespace());\n                        fields_not_found.push(DavPropertyValue::empty(property.clone()));\n                    }\n                }\n            }\n\n            let mut prop_stats = Vec::with_capacity(2);\n\n            if !fields_not_found.is_empty() {\n                prop_stats\n                    .push(PropStat::new_list(fields_not_found).with_status(StatusCode::NOT_FOUND));\n            }\n\n            if !fields.is_empty() || prop_stats.is_empty() {\n                prop_stats.push(PropStat::new_list(fields));\n            }\n\n            response.add_response(Response::new_propstat(\n                Href(format!(\n                    \"{}/{}/\",\n                    base_path,\n                    percent_encoding::utf8_percent_encode(&name, RFC_3986),\n                )),\n                prop_stats,\n            ));\n        }\n\n        Ok(())\n    }\n\n    async fn expand_principal(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        propfind: &PropFind,\n    ) -> crate::Result<Option<Response>> {\n        let mut status = MultiStatus::new(vec![]);\n        self.prepare_principal_propfind_response(\n            access_token,\n            Collection::Principal,\n            [account_id].into_iter(),\n            propfind,\n            &mut status,\n        )\n        .await?;\n\n        Ok(status.response.0.into_iter().next())\n    }\n\n    async fn owner_href(&self, access_token: &AccessToken, account_id: u32) -> trc::Result<Href> {\n        if access_token.primary_id() == account_id {\n            Ok(access_token.current_user_principal())\n        } else {\n            let name = self\n                .store()\n                .get_principal_name(account_id)\n                .await\n                .caused_by(trc::location!())?\n                .unwrap_or_else(|| format!(\"_{account_id}\"));\n            Ok(Href(format!(\n                \"{}/{}/\",\n                DavResourceName::Principal.base_path(),\n                percent_encoding::utf8_percent_encode(&name, RFC_3986),\n            )))\n        }\n    }\n}\n\npub(crate) async fn build_home_set(\n    server: &Server,\n    access_token: &AccessToken,\n    name: &str,\n    account_id: u32,\n    is_calendar: bool,\n) -> trc::Result<Vec<Href>> {\n    let (collection, resource_name) = if is_calendar {\n        (Collection::Calendar, DavResourceName::Cal)\n    } else {\n        (Collection::AddressBook, DavResourceName::Card)\n    };\n\n    let mut hrefs = Vec::new();\n    hrefs.push(Href(format!(\n        \"{}/{}/\",\n        resource_name.base_path(),\n        percent_encoding::utf8_percent_encode(name, RFC_3986),\n    )));\n\n    if !server.core.groupware.assisted_discovery && account_id == access_token.primary_id() {\n        for account_id in access_token.all_ids_by_collection(collection) {\n            if account_id != access_token.primary_id() {\n                let other_name = server\n                    .store()\n                    .get_principal_name(account_id)\n                    .await\n                    .caused_by(trc::location!())?\n                    .unwrap_or_else(|| format!(\"_{account_id}\"));\n\n                hrefs.push(Href(format!(\n                    \"{}/{}/\",\n                    resource_name.base_path(),\n                    percent_encoding::utf8_percent_encode(&other_name, RFC_3986),\n                )));\n            }\n        }\n    }\n\n    Ok(hrefs)\n}\n\nfn all_props(collection: Collection, all_props: Option<&[DavProperty]>) -> Vec<DavProperty> {\n    if collection == Collection::Principal {\n        vec![\n            DavProperty::WebDav(WebDavProperty::DisplayName),\n            DavProperty::WebDav(WebDavProperty::ResourceType),\n            DavProperty::WebDav(WebDavProperty::SupportedReportSet),\n            DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n            DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n            DavProperty::Principal(PrincipalProperty::AlternateURISet),\n            DavProperty::Principal(PrincipalProperty::PrincipalURL),\n            DavProperty::Principal(PrincipalProperty::GroupMemberSet),\n            DavProperty::Principal(PrincipalProperty::GroupMembership),\n        ]\n    } else {\n        let mut props = vec![\n            DavProperty::WebDav(WebDavProperty::DisplayName),\n            DavProperty::WebDav(WebDavProperty::ResourceType),\n            DavProperty::WebDav(WebDavProperty::SupportedReportSet),\n            DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n            DavProperty::WebDav(WebDavProperty::SyncToken),\n            DavProperty::WebDav(WebDavProperty::Owner),\n            DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n        ];\n\n        if let Some(all_props) = all_props {\n            props.extend(all_props.iter().filter(|p| !p.is_all_prop()).cloned());\n            props\n        } else {\n            props\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/principal/propsearch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::propfind::PrincipalPropFind;\nuse common::{Server, auth::AccessToken};\nuse dav_proto::schema::{\n    property::{DavProperty, WebDavProperty},\n    request::{PrincipalPropertySearch, PropFind},\n    response::MultiStatus,\n};\nuse directory::{Type, backend::internal::manage::ManageDirectory};\nuse http_proto::HttpResponse;\nuse hyper::StatusCode;\nuse store::roaring::RoaringBitmap;\nuse trc::AddContext;\nuse types::collection::Collection;\n\npub(crate) trait PrincipalPropSearch: Sync + Send {\n    fn handle_principal_property_search(\n        &self,\n        access_token: &AccessToken,\n        request: PrincipalPropertySearch,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl PrincipalPropSearch for Server {\n    async fn handle_principal_property_search(\n        &self,\n        access_token: &AccessToken,\n        mut request: PrincipalPropertySearch,\n    ) -> crate::Result<HttpResponse> {\n        let mut search_for = None;\n\n        for prop_search in request.property_search {\n            if matches!(\n                prop_search.property,\n                DavProperty::WebDav(WebDavProperty::DisplayName)\n            ) && !prop_search.match_.is_empty()\n            {\n                search_for = Some(prop_search.match_);\n            }\n        }\n\n        let mut response = MultiStatus::new(Vec::with_capacity(16));\n        if let Some(search_for) = search_for {\n            // Return all principals\n            let principals = self\n                .store()\n                .list_principals(\n                    search_for.as_str().into(),\n                    access_token.tenant_id(),\n                    &[Type::Individual, Type::Group],\n                    false,\n                    0,\n                    0,\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n            let ids = RoaringBitmap::from_iter(principals.items.into_iter().map(|p| p.id()));\n\n            if !ids.is_empty() {\n                if request.properties.is_empty() {\n                    request\n                        .properties\n                        .push(DavProperty::WebDav(WebDavProperty::DisplayName));\n                }\n                let request = PropFind::Prop(request.properties);\n                self.prepare_principal_propfind_response(\n                    access_token,\n                    Collection::Principal,\n                    ids.into_iter(),\n                    &request,\n                    &mut response,\n                )\n                .await?;\n            }\n        }\n\n        Ok(HttpResponse::new(StatusCode::MULTI_STATUS).with_xml_body(response.to_string()))\n    }\n}\n"
  },
  {
    "path": "crates/dav/src/request.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavError, DavErrorCondition, DavMethod, DavResourceName,\n    calendar::{\n        copy_move::CalendarCopyMoveRequestHandler, delete::CalendarDeleteRequestHandler,\n        freebusy::CalendarFreebusyRequestHandler, get::CalendarGetRequestHandler,\n        mkcol::CalendarMkColRequestHandler, proppatch::CalendarPropPatchRequestHandler,\n        query::CalendarQueryRequestHandler, scheduling::CalendarEventNotificationHandler,\n        update::CalendarUpdateRequestHandler,\n    },\n    card::{\n        copy_move::CardCopyMoveRequestHandler, delete::CardDeleteRequestHandler,\n        get::CardGetRequestHandler, mkcol::CardMkColRequestHandler,\n        proppatch::CardPropPatchRequestHandler, query::CardQueryRequestHandler,\n        update::CardUpdateRequestHandler,\n    },\n    common::{\n        DavQuery,\n        acl::DavAclHandler,\n        lock::{LockRequest, LockRequestHandler},\n        propfind::PropFindRequestHandler,\n        uri::DavUriResource,\n    },\n    file::{\n        copy_move::FileCopyMoveRequestHandler, delete::FileDeleteRequestHandler,\n        get::FileGetRequestHandler, mkcol::FileMkColRequestHandler,\n        proppatch::FilePropPatchRequestHandler, update::FileUpdateRequestHandler,\n    },\n    principal::{matching::PrincipalMatching, propsearch::PrincipalPropSearch},\n};\nuse common::{Server, auth::AccessToken};\nuse compact_str::{CompactString, ToCompactString};\nuse dav_proto::{\n    RequestHeaders,\n    parser::{DavParser, tokenizer::Tokenizer},\n    schema::{\n        Namespace,\n        property::WebDavProperty,\n        request::{Acl, LockInfo, MkCol, PropFind, PropertyUpdate, Report},\n        response::{\n            BaseCondition, ErrorResponse, List, PrincipalSearchProperty, PrincipalSearchPropertySet,\n        },\n    },\n};\nuse directory::Permission;\nuse http_proto::{HttpRequest, HttpResponse, HttpSessionData, request::fetch_body};\nuse hyper::{StatusCode, header};\nuse std::{sync::Arc, time::Instant};\nuse trc::{EventType, LimitEvent, StoreEvent, WebDavEvent};\nuse types::collection::Collection;\n\npub trait DavRequestHandler: Sync + Send {\n    fn handle_dav_request(\n        &self,\n        request: HttpRequest,\n        access_token: Arc<AccessToken>,\n        session: &HttpSessionData,\n        resource: DavResourceName,\n        method: DavMethod,\n    ) -> impl Future<Output = HttpResponse> + Send;\n}\n\npub(crate) trait DavRequestDispatcher: Sync + Send {\n    fn dispatch_dav_request(\n        &self,\n        headers: &RequestHeaders<'_>,\n        access_token: Arc<AccessToken>,\n        resource: DavResourceName,\n        method: DavMethod,\n        body: Vec<u8>,\n    ) -> impl Future<Output = crate::Result<HttpResponse>> + Send;\n}\n\nimpl DavRequestDispatcher for Server {\n    async fn dispatch_dav_request(\n        &self,\n        headers: &RequestHeaders<'_>,\n        access_token: Arc<AccessToken>,\n        resource: DavResourceName,\n        method: DavMethod,\n        body: Vec<u8>,\n    ) -> crate::Result<HttpResponse> {\n        // Dispatch\n        match method {\n            DavMethod::PROPFIND => {\n                let request = PropFind::parse(&mut Tokenizer::new(&body))?;\n\n                self.handle_propfind_request(&access_token, headers, request)\n                    .await\n            }\n            DavMethod::GET | DavMethod::HEAD => match resource {\n                DavResourceName::Card => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCardGet)?;\n\n                    self.handle_card_get_request(\n                        &access_token,\n                        headers,\n                        matches!(method, DavMethod::HEAD),\n                    )\n                    .await\n                }\n                DavResourceName::Cal => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalGet)?;\n\n                    self.handle_calendar_get_request(\n                        &access_token,\n                        headers,\n                        matches!(method, DavMethod::HEAD),\n                    )\n                    .await\n                }\n                DavResourceName::File => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavFileGet)?;\n\n                    // Deal with Litmus bug\n                    /*self.handle_file_get_request(\n                        &access_token,\n                        headers,\n                        matches!(method, DavMethod::HEAD)\n                            && !request.headers().contains_key(\"x-litmus\"),\n                    )\n                    .await*/\n                    self.handle_file_get_request(\n                        &access_token,\n                        headers,\n                        matches!(method, DavMethod::HEAD),\n                    )\n                    .await\n                }\n                DavResourceName::Scheduling => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalGet)?;\n\n                    self.handle_scheduling_get_request(\n                        &access_token,\n                        headers,\n                        matches!(method, DavMethod::HEAD),\n                    )\n                    .await\n                }\n                DavResourceName::Principal => Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n            },\n            DavMethod::REPORT => match Report::parse(&mut Tokenizer::new(&body))? {\n                Report::SyncCollection(sync_collection) => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavSyncCollection)?;\n\n                    let uri = self\n                        .validate_uri(&access_token, headers.uri)\n                        .await\n                        .and_then(|d| d.into_owned_uri())?;\n                    match resource {\n                        DavResourceName::Card\n                        | DavResourceName::Cal\n                        | DavResourceName::File\n                        | DavResourceName::Scheduling => {\n                            self.handle_dav_query(\n                                &access_token,\n                                DavQuery::changes(uri, sync_collection, headers),\n                            )\n                            .await\n                        }\n                        DavResourceName::Principal => {\n                            Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n                        }\n                    }\n                }\n                Report::AclPrincipalPropSet(report) => {\n                    // Validate permissions\n                    if !self.core.groupware.allow_directory_query\n                        && !access_token.has_permission(Permission::IndividualList)\n                    {\n                        return Err(DavError::Condition(\n                            DavErrorCondition::new(\n                                StatusCode::FORBIDDEN,\n                                BaseCondition::NeedPrivileges(List(Default::default())),\n                            )\n                            .with_details(\"The administrator has disabled directory queries.\"),\n                        ));\n                    }\n                    access_token.assert_has_permission(Permission::DavPrincipalAcl)?;\n\n                    self.handle_acl_prop_set(&access_token, headers, report)\n                        .await\n                }\n                Report::PrincipalMatch(report) => {\n                    // Validate permissions\n                    if !self.core.groupware.allow_directory_query\n                        && !access_token.has_permission(Permission::IndividualList)\n                    {\n                        return Err(DavError::Condition(\n                            DavErrorCondition::new(\n                                StatusCode::FORBIDDEN,\n                                BaseCondition::NeedPrivileges(List(Default::default())),\n                            )\n                            .with_details(\"The administrator has disabled directory queries.\"),\n                        ));\n                    }\n                    access_token.assert_has_permission(Permission::DavPrincipalMatch)?;\n\n                    self.handle_principal_match(&access_token, headers, report)\n                        .await\n                }\n                Report::PrincipalPropertySearch(report) => {\n                    if resource == DavResourceName::Principal {\n                        // Validate permissions\n                        if !self.core.groupware.allow_directory_query\n                            && !access_token.has_permission(Permission::IndividualList)\n                        {\n                            return Err(DavError::Condition(\n                                DavErrorCondition::new(\n                                    StatusCode::FORBIDDEN,\n                                    BaseCondition::NeedPrivileges(List(Default::default())),\n                                )\n                                .with_details(\"The administrator has disabled directory queries.\"),\n                            ));\n                        }\n\n                        access_token.assert_has_permission(Permission::DavPrincipalSearch)?;\n\n                        self.handle_principal_property_search(&access_token, report)\n                            .await\n                    } else {\n                        Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n                    }\n                }\n                Report::PrincipalSearchPropertySet => {\n                    if resource == DavResourceName::Principal {\n                        // Validate permissions\n                        access_token\n                            .assert_has_permission(Permission::DavPrincipalSearchPropSet)?;\n\n                        Ok(HttpResponse::new(StatusCode::OK).with_xml_body(\n                            PrincipalSearchPropertySet::new(vec![PrincipalSearchProperty::new(\n                                WebDavProperty::DisplayName,\n                                \"Account or Group name\",\n                            )])\n                            .to_string(),\n                        ))\n                    } else {\n                        Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n                    }\n                }\n                Report::AddressbookQuery(report) => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCardQuery)?;\n\n                    self.handle_card_query_request(&access_token, headers, report)\n                        .await\n                }\n                Report::AddressbookMultiGet(report) => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCardMultiGet)?;\n\n                    self.handle_dav_query(\n                        &access_token,\n                        DavQuery::multiget(report, Collection::AddressBook, headers),\n                    )\n                    .await\n                }\n                Report::CalendarQuery(report) => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalQuery)?;\n\n                    self.handle_calendar_query_request(&access_token, headers, report)\n                        .await\n                }\n                Report::CalendarMultiGet(report) => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalMultiGet)?;\n\n                    self.handle_dav_query(\n                        &access_token,\n                        DavQuery::multiget(report, Collection::Calendar, headers),\n                    )\n                    .await\n                }\n                Report::FreeBusyQuery(report) => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalFreeBusyQuery)?;\n\n                    self.handle_calendar_freebusy_request(&access_token, headers, report)\n                        .await\n                }\n                Report::ExpandProperty(report) => {\n                    let uri = self\n                        .validate_uri(&access_token, headers.uri)\n                        .await\n                        .and_then(|d| d.into_owned_uri())?;\n\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavExpandProperty)?;\n\n                    match resource {\n                        DavResourceName::Card | DavResourceName::Cal | DavResourceName::File => {\n                            self.handle_dav_query(\n                                &access_token,\n                                DavQuery::expand(uri, report, headers),\n                            )\n                            .await\n                        }\n                        DavResourceName::Principal | DavResourceName::Scheduling => {\n                            Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n                        }\n                    }\n                }\n            },\n            DavMethod::PROPPATCH => {\n                let request = PropertyUpdate::parse(&mut Tokenizer::new(&body))?;\n                match resource {\n                    DavResourceName::Card => {\n                        // Validate permissions\n                        access_token.assert_has_permission(Permission::DavCardPropPatch)?;\n\n                        self.handle_card_proppatch_request(&access_token, headers, request)\n                            .await\n                    }\n                    DavResourceName::Cal => {\n                        // Validate permissions\n                        access_token.assert_has_permission(Permission::DavCalPropPatch)?;\n\n                        self.handle_calendar_proppatch_request(&access_token, headers, request)\n                            .await\n                    }\n                    DavResourceName::File => {\n                        // Validate permissions\n                        access_token.assert_has_permission(Permission::DavFilePropPatch)?;\n\n                        self.handle_file_proppatch_request(&access_token, headers, request)\n                            .await\n                    }\n                    DavResourceName::Principal | DavResourceName::Scheduling => {\n                        Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n                    }\n                }\n            }\n            DavMethod::MKCOL => {\n                let request = if !body.is_empty() {\n                    Some(MkCol::parse(&mut Tokenizer::new(&body))?)\n                } else {\n                    None\n                };\n\n                match resource {\n                    DavResourceName::Card => {\n                        // Validate permissions\n                        access_token.assert_has_permission(Permission::DavCardMkCol)?;\n\n                        self.handle_card_mkcol_request(&access_token, headers, request)\n                            .await\n                    }\n                    DavResourceName::Cal => {\n                        // Validate permissions\n                        access_token.assert_has_permission(Permission::DavCalMkCol)?;\n\n                        self.handle_calendar_mkcol_request(&access_token, headers, request)\n                            .await\n                    }\n                    DavResourceName::File => {\n                        // Validate permissions\n                        access_token.assert_has_permission(Permission::DavFileMkCol)?;\n\n                        self.handle_file_mkcol_request(&access_token, headers, request)\n                            .await\n                    }\n                    DavResourceName::Principal | DavResourceName::Scheduling => {\n                        Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n                    }\n                }\n            }\n            DavMethod::DELETE => match resource {\n                DavResourceName::Card => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCardDelete)?;\n\n                    self.handle_card_delete_request(&access_token, headers)\n                        .await\n                }\n                DavResourceName::Cal => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalDelete)?;\n\n                    self.handle_calendar_delete_request(&access_token, headers)\n                        .await\n                }\n                DavResourceName::File => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavFileDelete)?;\n\n                    self.handle_file_delete_request(&access_token, headers)\n                        .await\n                }\n                DavResourceName::Scheduling => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalDelete)?;\n\n                    self.handle_scheduling_delete_request(&access_token, headers)\n                        .await\n                }\n                DavResourceName::Principal => Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n            },\n            DavMethod::PUT | DavMethod::POST | DavMethod::PATCH => match resource {\n                DavResourceName::Card => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCardPut)?;\n\n                    self.handle_card_update_request(\n                        &access_token,\n                        headers,\n                        body,\n                        matches!(method, DavMethod::PATCH),\n                    )\n                    .await\n                }\n                DavResourceName::Cal => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalPut)?;\n\n                    self.handle_calendar_update_request(\n                        &access_token,\n                        headers,\n                        body,\n                        matches!(method, DavMethod::PATCH),\n                    )\n                    .await\n                }\n                DavResourceName::File => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavFilePut)?;\n\n                    self.handle_file_update_request(\n                        &access_token,\n                        headers,\n                        body,\n                        matches!(method, DavMethod::PATCH),\n                    )\n                    .await\n                }\n                DavResourceName::Scheduling => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalFreeBusyQuery)?;\n\n                    self.handle_scheduling_post_request(&access_token, headers, body)\n                        .await\n                }\n                DavResourceName::Principal => Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n            },\n            DavMethod::COPY | DavMethod::MOVE => {\n                let is_move = matches!(method, DavMethod::MOVE);\n                match resource {\n                    DavResourceName::Card => {\n                        // Validate permissions\n                        access_token.assert_has_permission(if is_move {\n                            Permission::DavCardMove\n                        } else {\n                            Permission::DavCardCopy\n                        })?;\n\n                        self.handle_card_copy_move_request(&access_token, headers, is_move)\n                            .await\n                    }\n                    DavResourceName::Cal => {\n                        // Validate permissions\n                        access_token.assert_has_permission(if is_move {\n                            Permission::DavCalMove\n                        } else {\n                            Permission::DavCalCopy\n                        })?;\n                        self.handle_calendar_copy_move_request(&access_token, headers, is_move)\n                            .await\n                    }\n                    DavResourceName::File => {\n                        // Validate permissions\n                        access_token.assert_has_permission(if is_move {\n                            Permission::DavFileMove\n                        } else {\n                            Permission::DavFileCopy\n                        })?;\n\n                        self.handle_file_copy_move_request(&access_token, headers, is_move)\n                            .await\n                    }\n                    DavResourceName::Principal | DavResourceName::Scheduling => {\n                        Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED))\n                    }\n                }\n            }\n            DavMethod::MKCALENDAR => match resource {\n                DavResourceName::Cal => {\n                    // Validate permissions\n                    access_token.assert_has_permission(Permission::DavCalMkCol)?;\n\n                    self.handle_calendar_mkcol_request(\n                        &access_token,\n                        headers,\n                        Some(MkCol::parse(&mut Tokenizer::new(&body))?),\n                    )\n                    .await\n                }\n                _ => Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n            },\n            DavMethod::LOCK => {\n                // Validate permissions\n                access_token.assert_has_permission(match resource {\n                    DavResourceName::File => Permission::DavFileLock,\n                    DavResourceName::Cal => Permission::DavCalLock,\n                    DavResourceName::Card => Permission::DavCardLock,\n                    _ => return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n                })?;\n\n                self.handle_lock_request(\n                    &access_token,\n                    headers,\n                    if !body.is_empty() {\n                        LockRequest::Lock(LockInfo::parse(&mut Tokenizer::new(&body))?)\n                    } else {\n                        LockRequest::Refresh\n                    },\n                )\n                .await\n            }\n            DavMethod::UNLOCK => {\n                // Validate permissions\n                access_token.assert_has_permission(match resource {\n                    DavResourceName::File => Permission::DavFileLock,\n                    DavResourceName::Cal => Permission::DavCalLock,\n                    DavResourceName::Card => Permission::DavCardLock,\n                    _ => return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n                })?;\n\n                self.handle_lock_request(&access_token, headers, LockRequest::Unlock)\n                    .await\n            }\n            DavMethod::ACL => {\n                // Validate permissions\n                access_token.assert_has_permission(match resource {\n                    DavResourceName::File => Permission::DavFileAcl,\n                    DavResourceName::Cal => Permission::DavCalAcl,\n                    DavResourceName::Card => Permission::DavCardAcl,\n                    _ => return Err(DavError::Code(StatusCode::METHOD_NOT_ALLOWED)),\n                })?;\n\n                self.handle_acl_request(\n                    &access_token,\n                    headers,\n                    Acl::parse(&mut Tokenizer::new(&body))?,\n                )\n                .await\n            }\n            DavMethod::OPTIONS => unreachable!(),\n        }\n    }\n}\n\nimpl DavRequestHandler for Server {\n    async fn handle_dav_request(\n        &self,\n        mut request: HttpRequest,\n        access_token: Arc<AccessToken>,\n        session: &HttpSessionData,\n        resource: DavResourceName,\n        method: DavMethod,\n    ) -> HttpResponse {\n        let body = if method.has_body()\n            || request\n                .headers()\n                .get(header::CONTENT_LENGTH)\n                .and_then(|v| v.to_str().ok())\n                .and_then(|v| v.parse::<u64>().ok())\n                .is_some_and(|len| len > 0)\n        {\n            if let Some(body) = fetch_body(\n                &mut request,\n                if !access_token.has_permission(Permission::UnlimitedUploads) {\n                    self.core.groupware.max_request_size\n                } else {\n                    0\n                },\n                session.session_id,\n            )\n            .await\n            {\n                body\n            } else {\n                trc::event!(\n                    Limit(trc::LimitEvent::SizeRequest),\n                    SpanId = session.session_id,\n                    Contents = \"Request body too large\",\n                );\n\n                return HttpResponse::new(StatusCode::PAYLOAD_TOO_LARGE);\n            }\n        } else {\n            Vec::new()\n        };\n\n        // Parse headers\n        let mut headers = RequestHeaders::new(request.uri().path());\n        for (key, value) in request.headers() {\n            headers.parse(key.as_str(), value.to_str().unwrap_or_default());\n        }\n\n        let start_time = Instant::now();\n        match self\n            .dispatch_dav_request(&headers, access_token, resource, method, body)\n            .await\n        {\n            Ok(response) => {\n                let event = WebDavEvent::from(method);\n\n                trc::event!(\n                    WebDav(event),\n                    SpanId = session.session_id,\n                    Url = headers.uri.to_compact_string(),\n                    Type = resource.name(),\n                    Details = &headers,\n                    Result = response.status().as_u16(),\n                    Elapsed = start_time.elapsed(),\n                );\n\n                response\n            }\n            Err(DavError::Internal(err)) => {\n                let err_type = err.event_type();\n\n                trc::error!(\n                    err.span_id(session.session_id)\n                        .ctx(trc::Key::Url, headers.uri.to_compact_string())\n                        .ctx(trc::Key::Type, resource.name())\n                        .ctx(trc::Key::Elapsed, start_time.elapsed())\n                );\n\n                match err_type {\n                    EventType::Limit(LimitEvent::Quota | LimitEvent::TenantQuota) => {\n                        HttpResponse::new(StatusCode::PRECONDITION_FAILED)\n                            .with_xml_body(\n                                ErrorResponse::new(BaseCondition::QuotaNotExceeded)\n                                    .with_namespace(match resource {\n                                        DavResourceName::Card => Namespace::CardDav,\n                                        DavResourceName::Cal | DavResourceName::Scheduling => {\n                                            Namespace::CalDav\n                                        }\n                                        DavResourceName::File | DavResourceName::Principal => {\n                                            Namespace::Dav\n                                        }\n                                    })\n                                    .to_string(),\n                            )\n                            .with_no_cache()\n                    }\n                    EventType::Store(StoreEvent::AssertValueFailed) => {\n                        HttpResponse::new(StatusCode::CONFLICT)\n                    }\n                    EventType::Security(_) => HttpResponse::new(StatusCode::FORBIDDEN),\n                    _ => HttpResponse::new(StatusCode::INTERNAL_SERVER_ERROR),\n                }\n            }\n            Err(DavError::Parse(err)) => {\n                let result = if headers.content_type.is_some_and(|h| h.contains(\"/xml\")) {\n                    StatusCode::BAD_REQUEST\n                } else {\n                    StatusCode::UNSUPPORTED_MEDIA_TYPE\n                };\n\n                trc::event!(\n                    WebDav(WebDavEvent::Error),\n                    SpanId = session.session_id,\n                    Url = headers.uri.to_compact_string(),\n                    Type = resource.name(),\n                    Details = &headers,\n                    Result = result.as_u16(),\n                    Reason = err.to_compact_string(),\n                    Elapsed = start_time.elapsed(),\n                );\n\n                HttpResponse::new(result)\n            }\n            Err(DavError::Condition(condition)) => {\n                let event = WebDavEvent::from(method);\n\n                trc::event!(\n                    WebDav(event),\n                    SpanId = session.session_id,\n                    Url = headers.uri.to_compact_string(),\n                    Type = resource.name(),\n                    Details = &headers,\n                    Code = condition.code.as_u16(),\n                    Result = CompactString::const_new(condition.condition.display_name()),\n                    Reason = condition.details,\n                    Elapsed = start_time.elapsed(),\n                );\n\n                HttpResponse::new(condition.code)\n                    .with_xml_body(\n                        ErrorResponse::new(condition.condition)\n                            .with_namespace(match resource {\n                                DavResourceName::Card => Namespace::CardDav,\n                                DavResourceName::Cal | DavResourceName::Scheduling => {\n                                    Namespace::CalDav\n                                }\n                                DavResourceName::File | DavResourceName::Principal => {\n                                    Namespace::Dav\n                                }\n                            })\n                            .to_string(),\n                    )\n                    .with_no_cache()\n            }\n            Err(DavError::Code(code)) => {\n                let event = WebDavEvent::from(method);\n\n                trc::event!(\n                    WebDav(event),\n                    SpanId = session.session_id,\n                    Url = headers.uri.to_compact_string(),\n                    Type = resource.name(),\n                    Details = &headers,\n                    Result = code.as_u16(),\n                    Elapsed = start_time.elapsed(),\n                );\n\n                HttpResponse::new(code)\n            }\n        }\n    }\n}\n\nimpl From<dav_proto::parser::Error> for DavError {\n    fn from(err: dav_proto::parser::Error) -> Self {\n        DavError::Parse(err)\n    }\n}\n\nimpl From<trc::Error> for DavError {\n    fn from(err: trc::Error) -> Self {\n        DavError::Internal(err)\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/Cargo.toml",
    "content": "[package]\nname = \"dav-proto\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\ntrc = { path = \"../trc\" }\ntypes = { path = \"../types\" }\nhashify = \"0.2.6\"\nquick-xml = { version = \"0.38\" }\ncalcard = { version = \"0.3\", features = [\"rkyv\"] }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] }\nhyper = \"1.6.0\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\nchrono = { version = \"0.4.40\", features = [\"serde\"], optional = true }\ncompact_str = \"0.9.0\"\n\n[dev-dependencies]\ncalcard = { version = \"0.3\", features = [\"serde\", \"rkyv\"] }\ntypes = { path = \"../types\", features = [\"test_mode\"] }\nserde = { version = \"1.0.217\", features = [\"derive\"] }\nserde_json = \"1.0.138\"\nchrono = { version = \"0.4.40\", features = [\"serde\"] }\n\n[features]\ntest_mode = [\"chrono\"]\nenterprise = []\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-001.json",
    "content": "{\n  \"aces\": [\n    {\n      \"principal\": {\n        \"Href\": \"http://www.example.com/users/friends\"\n      },\n      \"invert\": false,\n      \"grant_deny\": {\n        \"Grant\": [\n          \"Read\"\n        ]\n      },\n      \"protected\": false,\n      \"inherited\": null\n    },\n    {\n      \"principal\": {\n        \"Href\": \"http://www.example.com/users/ygoland-so\"\n      },\n      \"invert\": false,\n      \"grant_deny\": {\n        \"Deny\": [\n          \"Read\"\n        ]\n      },\n      \"protected\": false,\n      \"inherited\": null\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-001.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:acl xmlns:D=\"DAV:\">\n     <D:ace>\n       <D:principal>\n         <D:href>http://www.example.com/users/friends</D:href>\n       </D:principal>\n       <D:grant><D:read/></D:grant>\n      </D:ace>\n       <D:ace>\n       <D:principal>\n         <D:href>http://www.example.com/users/ygoland-so</D:href>\n       </D:principal>\n       <D:deny><D:read/></D:deny>\n     </D:ace>\n   </D:acl>"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-002.json",
    "content": "{\n  \"aces\": [\n    {\n      \"principal\": {\n        \"Href\": \"http://www.example.com/users/ejw\"\n      },\n      \"invert\": false,\n      \"grant_deny\": {\n        \"Grant\": [\n          \"Write\"\n        ]\n      },\n      \"protected\": false,\n      \"inherited\": null\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-002.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:acl xmlns:D=\"DAV:\" xmlns:F=\"http://www.example.com/privs/\">\n     <D:ace>\n       <D:principal>\n          <D:href>http://www.example.com/users/ejw</D:href>\n       </D:principal>\n       <D:grant><D:write/></D:grant>\n     </D:ace>\n   </D:acl>"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-003.json",
    "content": "{\n  \"aces\": [\n    {\n      \"principal\": {\n        \"Href\": \"http://www.example.com/users/esedlar\"\n      },\n      \"invert\": true,\n      \"grant_deny\": {\n        \"Deny\": [\n          \"Write\"\n        ]\n      },\n      \"protected\": true,\n      \"inherited\": \"http://www.example.com/container/\"\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-003.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:acl xmlns:D=\"DAV:\">\n     <D:ace>\n       <D:invert><D:principal>\n         <D:href>http://www.example.com/users/esedlar</D:href>\n       </D:principal></D:invert>\n       <D:deny>\n         <D:privilege><D:write/></D:privilege>\n       </D:deny>\n       <D:protected/>\n       <D:inherited>\n         <D:href>http://www.example.com/container/</D:href>\n        </D:inherited>\n     </D:ace>\n   </D:acl>"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-004.json",
    "content": "{\n  \"aces\": [\n    {\n      \"principal\": {\n        \"Href\": \"http://www.example.com/users/esedlar\"\n      },\n      \"invert\": false,\n      \"grant_deny\": {\n        \"Grant\": [\n          \"Read\",\n          \"Write\"\n        ]\n      },\n      \"protected\": false,\n      \"inherited\": null\n    },\n    {\n      \"principal\": {\n        \"Property\": [\n          {\n            \"property\": {\n              \"type\": \"WebDav\",\n              \"data\": {\n                \"type\": \"Owner\"\n              }\n            },\n            \"value\": \"Null\"\n          }\n        ]\n      },\n      \"invert\": false,\n      \"grant_deny\": {\n        \"Grant\": [\n          \"ReadAcl\",\n          \"WriteAcl\"\n        ]\n      },\n      \"protected\": false,\n      \"inherited\": null\n    },\n    {\n      \"principal\": \"All\",\n      \"invert\": false,\n      \"grant_deny\": {\n        \"Grant\": [\n          \"Read\"\n        ]\n      },\n      \"protected\": false,\n      \"inherited\": null\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/acl-004.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:acl xmlns:D=\"DAV:\">\n     <D:ace>\n       <D:principal>\n         <D:href>http://www.example.com/users/esedlar</D:href>\n       </D:principal>\n       <D:grant>\n         <D:privilege><D:read/></D:privilege>\n         <D:privilege><D:write/></D:privilege>\n       </D:grant>\n     </D:ace>\n     <D:ace>\n       <D:principal>\n         <D:property><D:owner/></D:property>\n       </D:principal>\n       <D:grant>\n         <D:privilege><D:read-acl/></D:privilege>\n         <D:privilege><D:write-acl/></D:privilege>\n       </D:grant>\n     </D:ace>\n     <D:ace>\n       <D:principal><D:all/></D:principal>\n       <D:grant>\n         <D:privilege><D:read/></D:privilege>\n       </D:grant>\n     </D:ace>\n   </D:acl>"
  },
  {
    "path": "crates/dav-proto/resources/requests/lockinfo-001.json",
    "content": "{\n  \"lock_scope\": \"Shared\",\n  \"lock_type\": \"Write\",\n  \"owner\": [\n    {\n      \"type\": \"ElementStart\",\n      \"data\": {\n        \"name\": \"href\",\n        \"attrs\": \"xmlns=\\\"DAV:\\\"\"\n      }\n    },\n    {\n      \"type\": \"Text\",\n      \"data\": \"http://example.org/~ejw/contact.html\"\n    },\n    {\n      \"type\": \"ElementEnd\"\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/lockinfo-001.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:lockinfo xmlns:D='DAV:'>\n       <D:lockscope><D:shared/></D:lockscope>\n       <D:locktype><D:write/></D:locktype>\n       <D:owner>\n         <D:href>http://example.org/~ejw/contact.html</D:href>\n       </D:owner>\n     </D:lockinfo>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/lockinfo-002.json",
    "content": "{\n  \"lock_scope\": \"Exclusive\",\n  \"lock_type\": \"Write\",\n  \"owner\": [\n    {\n      \"type\": \"ElementStart\",\n      \"data\": {\n        \"name\": \"href\",\n        \"attrs\": \"xmlns=\\\"DAV:\\\"\"\n      }\n    },\n    {\n      \"type\": \"Text\",\n      \"data\": \"http://example.org/~ejw/contact.html\"\n    },\n    {\n      \"type\": \"ElementEnd\"\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/lockinfo-002.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:lockinfo xmlns:D=\"DAV:\">\n       <D:locktype><D:write/></D:locktype>\n       <D:lockscope><D:exclusive/></D:lockscope>\n       <D:owner>\n         <D:href>http://example.org/~ejw/contact.html</D:href>\n       </D:owner>\n     </D:lockinfo>\n     "
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-001.json",
    "content": "{\n  \"is_mkcalendar\": false,\n  \"props\": [\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"ResourceType\"\n        }\n      },\n      \"value\": {\n        \"ResourceTypes\": [\n          \"Collection\",\n          \"AddressBook\"\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"DisplayName\"\n        }\n      },\n      \"value\": {\n        \"String\": \"Lisa's Contacts\"\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"CardDav\",\n        \"data\": {\n          \"type\": \"AddressbookDescription\"\n        }\n      },\n      \"value\": {\n        \"String\": \"My primary address book.\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-001.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:mkcol xmlns:D=\"DAV:\"\n                 xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:set>\n       <D:prop>\n         <D:resourcetype>\n           <D:collection/>\n           <C:addressbook/>\n         </D:resourcetype>\n         <D:displayname>Lisa's Contacts</D:displayname>\n         <C:addressbook-description xml:lang=\"en\"\n   >My primary address book.</C:addressbook-description>\n       </D:prop>\n     </D:set>\n   </D:mkcol>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-002.json",
    "content": "{\n  \"is_mkcalendar\": true,\n  \"props\": [\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"DisplayName\"\n        }\n      },\n      \"value\": {\n        \"String\": \"Lisa's Events\"\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarDescription\"\n        }\n      },\n      \"value\": {\n        \"String\": \"Calendar restricted to events.\"\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"SupportedCalendarComponentSet\"\n        }\n      },\n      \"value\": {\n        \"Components\": [\n          \"VEvent\"\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarTimezone\"\n        }\n      },\n      \"value\": {\n        \"ICalendar\": {\n          \"components\": [\n            {\n              \"component_type\": \"VCalendar\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Prodid\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"-//Example Corp.//CalDAV Client//EN\"\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Version\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"2.0\"\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": [\n                1\n              ]\n            },\n            {\n              \"component_type\": \"VTimezone\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Tzid\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"US-Eastern\"\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"LastModified\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": 1987,\n                        \"month\": 1,\n                        \"day\": 1,\n                        \"hour\": 0,\n                        \"minute\": 0,\n                        \"second\": 0,\n                        \"tz_hour\": 0,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": false\n                      }\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": [\n                2,\n                3\n              ]\n            },\n            {\n              \"component_type\": \"Standard\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Dtstart\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": 1967,\n                        \"month\": 10,\n                        \"day\": 29,\n                        \"hour\": 2,\n                        \"minute\": 0,\n                        \"second\": 0,\n                        \"tz_hour\": null,\n                        \"tz_minute\": null,\n                        \"tz_minus\": false\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Rrule\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"RecurrenceRule\",\n                      \"data\": {\n                        \"freq\": \"Yearly\",\n                        \"until\": null,\n                        \"count\": null,\n                        \"interval\": null,\n                        \"bysecond\": [],\n                        \"byminute\": [],\n                        \"byhour\": [],\n                        \"byday\": [\n                          {\n                            \"ordwk\": -1,\n                            \"weekday\": \"Sunday\"\n                          }\n                        ],\n                        \"bymonthday\": [],\n                        \"byyearday\": [],\n                        \"byweekno\": [],\n                        \"bymonth\": [\n                          10\n                        ],\n                        \"bysetpos\": [],\n                        \"wkst\": null\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetfrom\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 4,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetto\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 5,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzname\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"Eastern Standard Time (US & Canada)\"\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": []\n            },\n            {\n              \"component_type\": \"Daylight\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Dtstart\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": 1987,\n                        \"month\": 4,\n                        \"day\": 5,\n                        \"hour\": 2,\n                        \"minute\": 0,\n                        \"second\": 0,\n                        \"tz_hour\": null,\n                        \"tz_minute\": null,\n                        \"tz_minus\": false\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Rrule\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"RecurrenceRule\",\n                      \"data\": {\n                        \"freq\": \"Yearly\",\n                        \"until\": null,\n                        \"count\": null,\n                        \"interval\": null,\n                        \"bysecond\": [],\n                        \"byminute\": [],\n                        \"byhour\": [],\n                        \"byday\": [\n                          {\n                            \"ordwk\": 1,\n                            \"weekday\": \"Sunday\"\n                          }\n                        ],\n                        \"bymonthday\": [],\n                        \"byyearday\": [],\n                        \"byweekno\": [],\n                        \"bymonth\": [\n                          4\n                        ],\n                        \"bysetpos\": [],\n                        \"wkst\": null\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetfrom\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 5,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetto\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 4,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzname\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"Eastern Daylight Time (US & Canada)\"\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": []\n            }\n          ]\n        }\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-002.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:mkcalendar xmlns:D=\"DAV:\"\n                 xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:set>\n       <D:prop>\n         <D:displayname>Lisa's Events</D:displayname>\n         <C:calendar-description xml:lang=\"en\"\n   >Calendar restricted to events.</C:calendar-description>\n         <C:supported-calendar-component-set>\n           <C:comp name=\"VEVENT\"/>\n         </C:supported-calendar-component-set>\n         <C:calendar-timezone><![CDATA[BEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VTIMEZONE\nTZID:US-Eastern\nLAST-MODIFIED:19870101T000000Z\nBEGIN:STANDARD\nDTSTART:19671029T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nTZNAME:Eastern Standard Time (US & Canada)\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:19870405T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nTZNAME:Eastern Daylight Time (US & Canada)\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n]]></C:calendar-timezone>\n       </D:prop>\n     </D:set>\n   </C:mkcalendar>"
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-003.json",
    "content": "{\n  \"is_mkcalendar\": false,\n  \"props\": [\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"ResourceType\"\n        }\n      },\n      \"value\": {\n        \"ResourceTypes\": [\n          \"Collection\"\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"DisplayName\"\n        }\n      },\n      \"value\": {\n        \"String\": \"Special Resource\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-003.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:mkcol xmlns:D=\"DAV:\"\n                 xmlns:E=\"http://example.com/ns/\">\n     <D:set>\n       <D:prop>\n         <D:resourcetype>\n           <D:collection/>\n           <E:special-resource/>\n         </D:resourcetype>\n         <D:displayname>Special Resource</D:displayname>\n       </D:prop>\n     </D:set>\n   </D:mkcol>\n   "
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-004.json",
    "content": "{\n  \"is_mkcalendar\": false,\n  \"props\": [\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"ResourceType\"\n        }\n      },\n      \"value\": {\n        \"ResourceTypes\": [\n          \"Collection\",\n          \"Calendar\"\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"DisplayName\"\n        }\n      },\n      \"value\": {\n        \"String\": \"Lisa's Events\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/mkcol-004.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:mkcol xmlns:D=\"DAV:\"\n                 xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:set>\n       <D:prop>\n         <D:resourcetype>\n           <D:collection/>\n           <C:calendar/>\n         </D:resourcetype>\n         <D:displayname>Lisa's Events</D:displayname>\n       </D:prop>\n     </D:set>\n   </D:mkcol>"
  },
  {
    "path": "crates/dav-proto/resources/requests/propertyupdate-001.json",
    "content": "{\n  \"set\": [\n    {\n      \"property\": {\n        \"type\": \"CardDav\",\n        \"data\": {\n          \"type\": \"AddressbookDescription\"\n        }\n      },\n      \"value\": {\n        \"String\": \"Adresses de Oliver Daboo\"\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarDescription\"\n        }\n      },\n      \"value\": {\n        \"String\": \"Calendrier de Mathilde Desruisseaux\"\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"SupportedCalendarComponentSet\"\n        }\n      },\n      \"value\": {\n        \"Components\": [\n          \"VEvent\",\n          \"VTodo\"\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarTimezone\"\n        }\n      },\n      \"value\": {\n        \"ICalendar\": {\n          \"components\": [\n            {\n              \"component_type\": \"VCalendar\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Prodid\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"-//Example Corp.//CalDAV Client//EN\"\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Version\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"2.0\"\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": [\n                1\n              ]\n            },\n            {\n              \"component_type\": \"VTimezone\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Tzid\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"US-Eastern\"\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"LastModified\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": 1987,\n                        \"month\": 1,\n                        \"day\": 1,\n                        \"hour\": 0,\n                        \"minute\": 0,\n                        \"second\": 0,\n                        \"tz_hour\": 0,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": false\n                      }\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": [\n                2,\n                3\n              ]\n            },\n            {\n              \"component_type\": \"Standard\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Dtstart\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": 1967,\n                        \"month\": 10,\n                        \"day\": 29,\n                        \"hour\": 2,\n                        \"minute\": 0,\n                        \"second\": 0,\n                        \"tz_hour\": null,\n                        \"tz_minute\": null,\n                        \"tz_minus\": false\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Rrule\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"RecurrenceRule\",\n                      \"data\": {\n                        \"freq\": \"Yearly\",\n                        \"until\": null,\n                        \"count\": null,\n                        \"interval\": null,\n                        \"bysecond\": [],\n                        \"byminute\": [],\n                        \"byhour\": [],\n                        \"byday\": [\n                          {\n                            \"ordwk\": -1,\n                            \"weekday\": \"Sunday\"\n                          }\n                        ],\n                        \"bymonthday\": [],\n                        \"byyearday\": [],\n                        \"byweekno\": [],\n                        \"bymonth\": [\n                          10\n                        ],\n                        \"bysetpos\": [],\n                        \"wkst\": null\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetfrom\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 4,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetto\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 5,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzname\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"Eastern Standard Time (US & Canada)\"\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": []\n            },\n            {\n              \"component_type\": \"Daylight\",\n              \"entries\": [\n                {\n                  \"name\": {\n                    \"type\": \"Dtstart\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": 1987,\n                        \"month\": 4,\n                        \"day\": 5,\n                        \"hour\": 2,\n                        \"minute\": 0,\n                        \"second\": 0,\n                        \"tz_hour\": null,\n                        \"tz_minute\": null,\n                        \"tz_minus\": false\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Rrule\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"RecurrenceRule\",\n                      \"data\": {\n                        \"freq\": \"Yearly\",\n                        \"until\": null,\n                        \"count\": null,\n                        \"interval\": null,\n                        \"bysecond\": [],\n                        \"byminute\": [],\n                        \"byhour\": [],\n                        \"byday\": [\n                          {\n                            \"ordwk\": 1,\n                            \"weekday\": \"Sunday\"\n                          }\n                        ],\n                        \"bymonthday\": [],\n                        \"byyearday\": [],\n                        \"byweekno\": [],\n                        \"bymonth\": [\n                          4\n                        ],\n                        \"bysetpos\": [],\n                        \"wkst\": null\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetfrom\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 5,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzoffsetto\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"PartialDateTime\",\n                      \"data\": {\n                        \"year\": null,\n                        \"month\": null,\n                        \"day\": null,\n                        \"hour\": null,\n                        \"minute\": null,\n                        \"second\": null,\n                        \"tz_hour\": 4,\n                        \"tz_minute\": 0,\n                        \"tz_minus\": true\n                      }\n                    }\n                  ]\n                },\n                {\n                  \"name\": {\n                    \"type\": \"Tzname\"\n                  },\n                  \"params\": [],\n                  \"values\": [\n                    {\n                      \"type\": \"Text\",\n                      \"data\": \"Eastern Daylight Time (US & Canada)\"\n                    }\n                  ]\n                }\n              ],\n              \"component_ids\": []\n            }\n          ]\n        }\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"ResourceType\"\n        }\n      },\n      \"value\": {\n        \"ResourceTypes\": [\n          \"Collection\",\n          \"AddressBook\"\n        ]\n      }\n    }\n  ],\n  \"remove\": [\n    {\n      \"type\": \"CalDav\",\n      \"data\": {\n        \"type\": \"CalendarTimezone\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"ResourceType\"\n      }\n    }\n  ],\n  \"set_first\": true\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propertyupdate-001.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propertyupdate xmlns:D=\"DAV:\"\n        xmlns:C=\"\">\n<D:set>\n    <D:prop>\n    <C:addressbook-description xml:lang=\"fr-CA\"\n          xmlns:C=\"urn:ietf:params:xml:ns:carddav\"\n       >Adresses de Oliver Daboo</C:addressbook-description>\n   \n    <C:calendar-description xml:lang=\"fr-CA\"\n            xmlns:C=\"urn:ietf:params:xml:ns:caldav\"\n         >Calendrier de Mathilde Desruisseaux</C:calendar-description>\n    <C:supported-calendar-component-set\n             xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n           <C:comp name=\"VEVENT\"/>\n           <C:comp name=\"VTODO\"/>\n         </C:supported-calendar-component-set>\n    <C:calendar-timezone\n       xmlns:C=\"urn:ietf:params:xml:ns:caldav\">BEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VTIMEZONE\nTZID:US-Eastern\nLAST-MODIFIED:19870101T000000Z\nBEGIN:STANDARD\nDTSTART:19671029T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nTZNAME:Eastern Standard Time (US &amp; Canada)\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:19870405T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nTZNAME:Eastern Daylight Time (US &amp; Canada)\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n   </C:calendar-timezone>\n    <D:resourcetype xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n           <D:collection/>\n           <C:addressbook/>\n    </D:resourcetype>\n     </D:prop>\n</D:set>\n<D:remove>\n      <D:prop>\n    <C:calendar-timezone\n       xmlns:C=\"urn:ietf:params:xml:ns:caldav\"/>\n    <D:resourcetype/>\n     </D:prop>\n</D:remove>\n</D:propertyupdate>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propertyupdate-002.json",
    "content": "{\n  \"set\": [\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop0\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value0\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop1\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value1\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop2\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value2\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop3\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value3\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop4\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value4\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop5\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value5\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop6\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value6\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop7\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value7\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop8\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value8\"\n          }\n        ]\n      }\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"prop9\",\n          \"attrs\": \"xmlns=\\\"http://example.com/neon/litmus/\\\"\"\n        }\n      },\n      \"value\": {\n        \"DeadProperty\": [\n          {\n            \"type\": \"Text\",\n            \"data\": \"value9\"\n          }\n        ]\n      }\n    }\n  ],\n  \"remove\": [],\n  \"set_first\": true\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propertyupdate-002.xml",
    "content": "<D:propertyupdate xmlns:D=\"DAV:\"><D:set><D:prop><prop0 xmlns=\"http://example.com/neon/litmus/\">value0</prop0></D:prop></D:set>\n<D:set><D:prop><prop1 xmlns=\"http://example.com/neon/litmus/\">value1</prop1></D:prop></D:set>\n<D:set><D:prop><prop2 xmlns=\"http://example.com/neon/litmus/\">value2</prop2></D:prop></D:set>\n<D:set><D:prop><prop3 xmlns=\"http://example.com/neon/litmus/\">value3</prop3></D:prop></D:set>\n<D:set><D:prop><prop4 xmlns=\"http://example.com/neon/litmus/\">value4</prop4></D:prop></D:set>\n<D:set><D:prop><prop5 xmlns=\"http://example.com/neon/litmus/\">value5</prop5></D:prop></D:set>\n<D:set><D:prop><prop6 xmlns=\"http://example.com/neon/litmus/\">value6</prop6></D:prop></D:set>\n<D:set><D:prop><prop7 xmlns=\"http://example.com/neon/litmus/\">value7</prop7></D:prop></D:set>\n<D:set><D:prop><prop8 xmlns=\"http://example.com/neon/litmus/\">value8</prop8></D:prop></D:set>\n<D:set><D:prop><prop9 xmlns=\"http://example.com/neon/litmus/\">value9</prop9></D:prop></D:set>\n</D:propertyupdate>"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-001.json",
    "content": "{\n  \"type\": \"Prop\",\n  \"data\": [\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"bigbox\",\n        \"attrs\": \"xmlns=\\\"http://ns.example.com/boxschema/\\\"\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"author\",\n        \"attrs\": \"xmlns=\\\"http://ns.example.com/boxschema/\\\"\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"DingALing\",\n        \"attrs\": \"xmlns=\\\"http://ns.example.com/boxschema/\\\"\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"Random\",\n        \"attrs\": \"xmlns=\\\"http://ns.example.com/boxschema/\\\"\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-001.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:prop xmlns:R=\"http://ns.example.com/boxschema/\">\n    <R:bigbox/>\n    <R:author/>\n    <R:DingALing/>\n    <R:Random/>\n</D:prop>\n</D:propfind>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-002.json",
    "content": "{\n  \"type\": \"PropName\"\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-002.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<propfind xmlns=\"DAV:\">\n<propname/>\n</propfind>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-003.json",
    "content": "{\n  \"type\": \"AllProp\",\n  \"data\": []\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-003.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:allprop/>\n</D:propfind>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-004.json",
    "content": "{\n  \"type\": \"AllProp\",\n  \"data\": [\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"SupportedReportSet\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-004.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:allprop/>\n<D:include>\n<D:supported-live-property-set/>\n<D:supported-report-set/>\n</D:include>\n</D:propfind>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-005.json",
    "content": "{\n  \"type\": \"Prop\",\n  \"data\": [\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"CurrentUserPrincipal\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-005.xml",
    "content": "<d:propfind xmlns:d=\"DAV:\">\n  <d:prop>\n    <d:current-user-principal/>\n  </d:prop>\n</d:propfind>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-006.json",
    "content": "{\n  \"type\": \"Prop\",\n  \"data\": [\n    {\n      \"type\": \"Principal\",\n      \"data\": {\n        \"type\": \"CalendarHomeSet\"\n      }\n    },\n    {\n      \"type\": \"Principal\",\n      \"data\": {\n        \"type\": \"GroupMembership\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-006.xml",
    "content": "<d:propfind xmlns:d=\"DAV:\" xmlns:cal=\"urn:ietf:params:xml:ns:caldav\" xmlns:cs=\"http://calendarserver.org/ns/\">\n  <d:prop>\n    <cal:calendar-home-set/>\n    <d:group-membership/>\n  </d:prop>\n</d:propfind>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-007.json",
    "content": "{\n  \"type\": \"Prop\",\n  \"data\": [\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"CurrentUserPrivilegeSet\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"ResourceType\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"DisplayName\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"calendar-color\",\n        \"attrs\": \"xmlns=\\\"http://apple.com/ns/ical/\\\"\"\n      }\n    },\n    {\n      \"type\": \"CalDav\",\n      \"data\": {\n        \"type\": \"SupportedCalendarComponentSet\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-007.xml",
    "content": "<d:propfind xmlns:d=\"DAV:\" xmlns:apple=\"http://apple.com/ns/ical/\" xmlns:cs=\"http://calendarserver.org/ns/\" xmlns:cal=\"urn:ietf:params:xml:ns:caldav\">\n  <d:prop>\n    <d:current-user-privilege-set/>\n    <d:resourcetype />\n    <d:displayname />\n    <apple:calendar-color/>\n    <cs:source/>\n    <cal:supported-calendar-component-set/>\n  </d:prop>\n</d:propfind>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-008.json",
    "content": "{\n  \"type\": \"Prop\",\n  \"data\": [\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"ResourceType\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"DisplayName\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"GetCTag\"\n      }\n    },\n    {\n      \"type\": \"CalDav\",\n      \"data\": {\n        \"type\": \"SupportedCalendarComponentSet\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-008.xml",
    "content": "<d:propfind xmlns:d=\"DAV:\" xmlns:cs=\"http://calendarserver.org/ns/\" xmlns:c=\"urn:ietf:params:xml:ns:caldav\">\n  <d:prop>\n     <d:resourcetype />\n     <d:displayname />\n     <cs:getctag />\n     <c:supported-calendar-component-set />\n  </d:prop>\n</d:propfind>"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-009.json",
    "content": ""
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-009.xml",
    "content": "<D:propfind xmlns:D=\"DAV:\"><D:prop><bar:foo xmlns:bar=\"\"/></D:prop></D:propfind>"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-010.json",
    "content": "{\n  \"type\": \"Prop\",\n  \"data\": [\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"ResourceType\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"DisplayName\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"SyncToken\"\n      }\n    },\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"GetCTag\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"me-card\",\n        \"attrs\": \"xmlns=\\\"http://calendarserver.org/ns/\\\" hello=\\\"world &amp; test\\\"\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/propfind-010.xml",
    "content": "<d:propfind xmlns:d=\"DAV:\" xmlns:cs=\"http://calendarserver.org/ns/\" xmlns:c=\"urn:ietf:params:xml:ns:carddav\"><d:prop><d:resourcetype /><d:displayname/><d:sync-token></d:sync-token><cs:getctag /><cs:me-card hello=\"world &amp; test\"/></d:prop></d:propfind>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-001.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [\n              {\n                \"component\": \"VCalendar\",\n                \"name\": {\n                  \"type\": \"Version\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Summary\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Uid\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Dtstart\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Dtend\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Duration\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Rrule\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Rdate\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Exrule\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"Exdate\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VEvent\",\n                \"name\": {\n                  \"type\": \"RecurrenceId\"\n                },\n                \"no_value\": false\n              },\n              {\n                \"component\": \"VTimezone\",\n                \"name\": null,\n                \"no_value\": false\n              }\n            ],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Component\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"op\": {\n        \"type\": \"TimeRange\",\n        \"data\": {\n          \"start\": 1136332800,\n          \"end\": 1136419200\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-001.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                 xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <D:getetag/>\n       <C:calendar-data>\n         <C:comp name=\"VCALENDAR\">\n           <C:prop name=\"VERSION\"/>\n           <C:comp name=\"VEVENT\">\n             <C:prop name=\"SUMMARY\"/>\n             <C:prop name=\"UID\"/>\n             <C:prop name=\"DTSTART\"/>\n             <C:prop name=\"DTEND\"/>\n             <C:prop name=\"DURATION\"/>\n             <C:prop name=\"RRULE\"/>\n             <C:prop name=\"RDATE\"/>\n             <C:prop name=\"EXRULE\"/>\n             <C:prop name=\"EXDATE\"/>\n             <C:prop name=\"RECURRENCE-ID\"/>\n           </C:comp>\n           <C:comp name=\"VTIMEZONE\"/>\n         </C:comp>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:time-range start=\"20060104T000000Z\"\n                         end=\"20060105T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n   "
  },
  {
    "path": "crates/dav-proto/resources/requests/report-002.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": {\n              \"start\": 1136246400,\n              \"end\": 1136419200\n            },\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Component\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"op\": {\n        \"type\": \"TimeRange\",\n        \"data\": {\n          \"start\": 1136246400,\n          \"end\": 1136419200\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-002.xml",
    "content": "   <C:calendar-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <C:calendar-data>\n         <C:limit-recurrence-set start=\"20060103T000000Z\"\n                                 end=\"20060105T000000Z\"/>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:time-range start=\"20060103T000000Z\"\n                         end=\"20060105T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-003.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": {\n              \"start\": 1136246400,\n              \"end\": 1136419200\n            },\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Component\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"op\": {\n        \"type\": \"TimeRange\",\n        \"data\": {\n          \"start\": 1136246400,\n          \"end\": 1136419200\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-003.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <C:calendar-data>\n         <C:expand start=\"20060103T000000Z\"\n                   end=\"20060105T000000Z\"/>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:time-range start=\"20060103T000000Z\"\n                         end=\"20060105T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n   "
  },
  {
    "path": "crates/dav-proto/resources/requests/report-004.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": {\n              \"start\": 1136160000,\n              \"end\": 1136246400\n            }\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Component\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VFreebusy\"\n      ],\n      \"op\": {\n        \"type\": \"TimeRange\",\n        \"data\": {\n          \"start\": 1136160000,\n          \"end\": 1136246400\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-004.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                 xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <C:calendar-data>\n         <C:limit-freebusy-set start=\"20060102T000000Z\"\n                                 end=\"20060103T000000Z\"/>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VFREEBUSY\">\n           <C:time-range start=\"20060102T000000Z\"\n                           end=\"20060103T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-005.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Component\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VTodo\",\n        \"VAlarm\"\n      ],\n      \"op\": {\n        \"type\": \"TimeRange\",\n        \"data\": {\n          \"start\": 1136541600,\n          \"end\": 1136628000\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-005.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VTODO\">\n           <C:comp-filter name=\"VALARM\">\n             <C:time-range start=\"20060106T100000Z\"\n                             end=\"20060107T100000Z\"/>\n           </C:comp-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-006.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Property\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"prop\": {\n        \"type\": \"Uid\"\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"DC6C50A017428C5216A2F1CD@example.com\",\n          \"collation\": \"Octet\",\n          \"negate\": false\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-006.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:prop-filter name=\"UID\">\n             <C:text-match collation=\"i;octet\"\n             >DC6C50A017428C5216A2F1CD@example.com</C:text-match>\n           </C:prop-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-007.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Property\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"prop\": {\n        \"type\": \"Attendee\"\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"mailto:lisa@example.com\",\n          \"collation\": \"AsciiCasemap\",\n          \"negate\": false\n        }\n      }\n    },\n    {\n      \"type\": \"Parameter\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"prop\": {\n        \"type\": \"Attendee\"\n      },\n      \"param\": \"Partstat\",\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"NEEDS-ACTION\",\n          \"collation\": \"AsciiCasemap\",\n          \"negate\": false\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-007.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:prop-filter name=\"ATTENDEE\">\n             <C:text-match collation=\"i;ascii-casemap\"\n              >mailto:lisa@example.com</C:text-match>\n             <C:param-filter name=\"PARTSTAT\">\n               <C:text-match collation=\"i;ascii-casemap\"\n                >NEEDS-ACTION</C:text-match>\n             </C:param-filter>\n           </C:prop-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-008.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Component\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"op\": {\n        \"type\": \"Exists\"\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-008.xml",
    "content": "\n   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\"/>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n   "
  },
  {
    "path": "crates/dav-proto/resources/requests/report-009.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Property\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VTodo\"\n      ],\n      \"prop\": {\n        \"type\": \"Completed\"\n      },\n      \"op\": {\n        \"type\": \"Undefined\"\n      }\n    },\n    {\n      \"type\": \"Property\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VTodo\"\n      ],\n      \"prop\": {\n        \"type\": \"Status\"\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"CANCELLED\",\n          \"collation\": \"AsciiCasemap\",\n          \"negate\": true\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-009.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VTODO\">\n           <C:prop-filter name=\"COMPLETED\">\n             <C:is-not-defined/>\n           </C:prop-filter>\n           <C:prop-filter name=\"STATUS\">\n             <C:text-match\n                negate-condition=\"yes\">CANCELLED</C:text-match>\n           </C:prop-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-010.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Property\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"prop\": {\n        \"type\": \"Other\",\n        \"data\": \"X-ABC-GUID\"\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"ABC\",\n          \"collation\": \"AsciiCasemap\",\n          \"negate\": false\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-010.xml",
    "content": "\n   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:prop-filter name=\"X-ABC-GUID\">\n             <C:text-match>ABC</C:text-match>\n           </C:prop-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-011.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Component\",\n      \"comp\": [\n        \"VCalendar\",\n        \"VEvent\"\n      ],\n      \"op\": {\n        \"type\": \"TimeRange\",\n        \"data\": {\n          \"start\": 1094083200,\n          \"end\": 1094169600\n        }\n      }\n    }\n  ],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-011.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <D:getetag/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:time-range start=\"20040902T000000Z\"\n                           end=\"20040903T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-012.json",
    "content": "{\n  \"type\": \"CalendarQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"filters\": [],\n  \"timezone\": {\n    \"type\": \"None\"\n  }\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-012.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                    xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n    <D:prop>\n      <D:getetag/>\n      <C:calendar-data/>\n    </D:prop>\n    <C:filter>\n      <C:comp-filter name=\"VCALENDAR\"/>\n    </C:filter>\n   </C:calendar-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-013.json",
    "content": "{\n  \"type\": \"CalendarMultiGet\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CalDav\",\n        \"data\": {\n          \"type\": \"CalendarData\",\n          \"data\": {\n            \"properties\": [],\n            \"expand\": null,\n            \"limit_recurrence\": null,\n            \"limit_freebusy\": null\n          }\n        }\n      }\n    ]\n  },\n  \"hrefs\": [\n    \"/bernard/work/abcd1.ics\",\n    \"/bernard/work/mtg1.ics\"\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-013.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-multiget xmlns:D=\"DAV:\"\n                    xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <D:href>/bernard/work/abcd1.ics</D:href>\n     <D:href>/bernard/work/mtg1.ics</D:href>\n   </C:calendar-multiget>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-014.json",
    "content": "{\n  \"type\": \"AddressbookQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CardDav\",\n        \"data\": {\n          \"type\": \"AddressData\",\n          \"data\": [\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Version\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Uid\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Nickname\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Email\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Fn\"\n              },\n              \"no_value\": false\n            }\n          ]\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"Property\",\n      \"comp\": null,\n      \"prop\": {\n        \"name\": {\n          \"type\": \"Nickname\"\n        },\n        \"group\": null\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Equals\",\n          \"value\": \"me\",\n          \"collation\": \"UnicodeCasemap\",\n          \"negate\": false\n        }\n      }\n    }\n  ],\n  \"limit\": null\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-014.xml",
    "content": "\n   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data>\n         <C:prop name=\"VERSION\"/>\n         <C:prop name=\"UID\"/>\n         <C:prop name=\"NICKNAME\"/>\n         <C:prop name=\"EMAIL\"/>\n         <C:prop name=\"FN\"/>\n       </C:address-data>\n     </D:prop>\n     <C:filter>\n       <C:prop-filter name=\"NICKNAME\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"equals\"\n         >me</C:text-match>\n       </C:prop-filter>\n     </C:filter>\n   </C:addressbook-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-015.json",
    "content": "{\n  \"type\": \"AddressbookQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CardDav\",\n        \"data\": {\n          \"type\": \"AddressData\",\n          \"data\": [\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Version\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Uid\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Nickname\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Email\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Fn\"\n              },\n              \"no_value\": false\n            }\n          ]\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"AnyOf\"\n    },\n    {\n      \"type\": \"Property\",\n      \"comp\": null,\n      \"prop\": {\n        \"name\": {\n          \"type\": \"Fn\"\n        },\n        \"group\": null\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"daboo\",\n          \"collation\": \"UnicodeCasemap\",\n          \"negate\": false\n        }\n      }\n    },\n    {\n      \"type\": \"Property\",\n      \"comp\": null,\n      \"prop\": {\n        \"name\": {\n          \"type\": \"Email\"\n        },\n        \"group\": null\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"daboo\",\n          \"collation\": \"UnicodeCasemap\",\n          \"negate\": false\n        }\n      }\n    }\n  ],\n  \"limit\": null\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-015.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data>\n         <C:prop name=\"VERSION\"/>\n         <C:prop name=\"UID\"/>\n         <C:prop name=\"NICKNAME\"/>\n         <C:prop name=\"EMAIL\"/>\n         <C:prop name=\"FN\"/>\n       </C:address-data>\n     </D:prop>\n     <C:filter test=\"anyof\">\n       <C:prop-filter name=\"FN\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"contains\"\n         >daboo</C:text-match>\n       </C:prop-filter>\n       <C:prop-filter name=\"EMAIL\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"contains\"\n         >daboo</C:text-match>\n       </C:prop-filter>\n     </C:filter>\n   </C:addressbook-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-016.json",
    "content": "{\n  \"type\": \"AddressbookQuery\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      }\n    ]\n  },\n  \"filters\": [\n    {\n      \"type\": \"AnyOf\"\n    },\n    {\n      \"type\": \"Property\",\n      \"comp\": null,\n      \"prop\": {\n        \"name\": {\n          \"type\": \"Fn\"\n        },\n        \"group\": null\n      },\n      \"op\": {\n        \"type\": \"TextMatch\",\n        \"data\": {\n          \"type\": \"TextMatch\",\n          \"match_type\": \"Contains\",\n          \"value\": \"daboo\",\n          \"collation\": \"UnicodeCasemap\",\n          \"negate\": false\n        }\n      }\n    }\n  ],\n  \"limit\": 2\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-016.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n     </D:prop>\n     <C:filter test=\"anyof\">\n       <C:prop-filter name=\"FN\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"contains\"\n         >daboo</C:text-match>\n       </C:prop-filter>\n     </C:filter>\n     <C:limit>\n       <C:nresults>2</C:nresults>\n     </C:limit>\n   </C:addressbook-query>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-017.json",
    "content": "{\n  \"type\": \"AddressbookMultiGet\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CardDav\",\n        \"data\": {\n          \"type\": \"AddressData\",\n          \"data\": [\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Version\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Uid\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Nickname\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Email\"\n              },\n              \"no_value\": false\n            },\n            {\n              \"group\": null,\n              \"name\": {\n                \"type\": \"Fn\"\n              },\n              \"no_value\": false\n            }\n          ]\n        }\n      }\n    ]\n  },\n  \"hrefs\": [\n    \"/home/bernard/addressbook/vcf102.vcf\",\n    \"/home/bernard/addressbook/vcf1.vcf\"\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-017.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-multiget xmlns:D=\"DAV:\"\n                        xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data>\n         <C:prop name=\"VERSION\"/>\n         <C:prop name=\"UID\"/>\n         <C:prop name=\"NICKNAME\"/>\n         <C:prop name=\"EMAIL\"/>\n         <C:prop name=\"FN\"/>\n       </C:address-data>\n     </D:prop>\n     <D:href>/home/bernard/addressbook/vcf102.vcf</D:href>\n     <D:href>/home/bernard/addressbook/vcf1.vcf</D:href>\n   </C:addressbook-multiget>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-018.json",
    "content": "{\n  \"type\": \"AddressbookMultiGet\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      },\n      {\n        \"type\": \"CardDav\",\n        \"data\": {\n          \"type\": \"AddressData\",\n          \"data\": []\n        }\n      }\n    ]\n  },\n  \"hrefs\": [\n    \"/home/bernard/addressbook/vcf3.vcf\"\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-018.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-multiget xmlns:D=\"DAV:\"\n                        xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data content-type='text/vcard' version='4.0'/>\n     </D:prop>\n     <D:href>/home/bernard/addressbook/vcf3.vcf</D:href>\n   </C:addressbook-multiget>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-019.json",
    "content": "{\n  \"type\": \"SyncCollection\",\n  \"sync_token\": \"abc\",\n  \"properties\": {\n    \"type\": \"Prop\",\n    \"data\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"GetETag\"\n        }\n      }\n    ]\n  },\n  \"depth\": \"Infinity\",\n  \"limit\": 9\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-019.xml",
    "content": "\n   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:sync-collection xmlns:D=\"DAV:\">\n     <D:sync-token>abc</D:sync-token>\n     <D:sync-level>infinite</D:sync-level>\n     <D:limit><D:nresults>9</D:nresults></D:limit>\n     <D:prop>\n       <D:getetag/>\n     </D:prop>\n   </D:sync-collection>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-020.json",
    "content": "{\n  \"type\": \"AclPrincipalPropSet\",\n  \"properties\": [\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"DisplayName\"\n      }\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-020.xml",
    "content": "  <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:acl-principal-prop-set xmlns:D=\"DAV:\">\n     <D:prop>\n       <D:displayname/>\n     </D:prop>\n   </D:acl-principal-prop-set>\n"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-021.json",
    "content": "{\n  \"type\": \"PrincipalMatch\",\n  \"principal_properties\": {\n    \"Properties\": [\n      {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"Owner\"\n        }\n      }\n    ]\n  },\n  \"properties\": []\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-021.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:principal-match xmlns:D=\"DAV:\">\n     <D:principal-property>\n       <D:owner/>\n     </D:principal-property>\n   </D:principal-match>\n   "
  },
  {
    "path": "crates/dav-proto/resources/requests/report-022.json",
    "content": "{\n  \"type\": \"PrincipalPropertySearch\",\n  \"property_search\": [\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"DisplayName\"\n        }\n      },\n      \"match_\": \"doE\"\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"title\",\n          \"attrs\": \"xmlns=\\\"http://www.example.com/ns/\\\"\"\n        }\n      },\n      \"match_\": \"Sales\"\n    }\n  ],\n  \"properties\": [\n    {\n      \"type\": \"WebDav\",\n      \"data\": {\n        \"type\": \"DisplayName\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"department\",\n        \"attrs\": \"xmlns=\\\"http://www.example.com/ns/\\\"\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"phone\",\n        \"attrs\": \"xmlns=\\\"http://www.example.com/ns/\\\"\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"office\",\n        \"attrs\": \"xmlns=\\\"http://www.example.com/ns/\\\"\"\n      }\n    },\n    {\n      \"type\": \"DeadProperty\",\n      \"data\": {\n        \"name\": \"salary\",\n        \"attrs\": \"xmlns=\\\"http://www.example.com/ns/\\\"\"\n      }\n    }\n  ],\n  \"apply_to_principal_collection_set\": false\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-022.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:principal-property-search xmlns:D=\"DAV:\">\n     <D:property-search>\n       <D:prop>\n         <D:displayname/>\n       </D:prop>\n       <D:match>doE</D:match>\n     </D:property-search>\n     <D:property-search>\n       <D:prop xmlns:B=\"http://www.example.com/ns/\">\n         <B:title/>\n       </D:prop>\n       <D:match>Sales</D:match>\n     </D:property-search>\n     <D:prop xmlns:B=\"http://www.example.com/ns/\">\n       <D:displayname/>\n       <B:department/>\n       <B:phone/>\n       <B:office/>\n       <B:salary/>\n     </D:prop>\n   </D:principal-property-search>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-023.json",
    "content": "{\n  \"type\": \"PrincipalSearchPropertySet\"\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-023.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:principal-search-property-set xmlns:D=\"DAV:\"/>\n   "
  },
  {
    "path": "crates/dav-proto/resources/requests/report-024.json",
    "content": "{\n  \"type\": \"ExpandProperty\",\n  \"properties\": [\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"version-history\",\n          \"attrs\": \"name=\\\"version-history\\\"\"\n        }\n      },\n      \"depth\": 0\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"version-set\",\n          \"attrs\": \"name=\\\"version-set\\\"\"\n        }\n      },\n      \"depth\": 1\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"creator-displayname\",\n          \"attrs\": \"name=\\\"creator-displayname\\\"\"\n        }\n      },\n      \"depth\": 2\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"activity-set\",\n          \"attrs\": \"name=\\\"activity-set\\\"\"\n        }\n      },\n      \"depth\": 2\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-024.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:expand-property xmlns:D=\"DAV:\">\n    <D:property name=\"version-history\">\n        <D:property name=\"version-set\">\n            <D:property name=\"creator-displayname\"/>\n            <D:property name=\"activity-set\"/>\n        </D:property>\n    </D:property>\n</D:expand-property>"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-025.json",
    "content": "{\n  \"type\": \"ExpandProperty\",\n  \"properties\": [\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"calendar-proxy-read-for\",\n          \"attrs\": \"name=\\\"calendar-proxy-read-for\\\" namespace=\\\"\\nhttp://calendarserver.org/ns/\\\"\"\n        }\n      },\n      \"depth\": 0\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"email-address-set\",\n          \"attrs\": \"name=\\\"email-address-set\\\"\\nnamespace=\\\"http://calendarserver.org/ns/\\\"\"\n        }\n      },\n      \"depth\": 1\n    },\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"DisplayName\"\n        }\n      },\n      \"depth\": 1\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"calendar-user-address-set\",\n          \"attrs\": \"name=\\\"calendar-user-address-set\\\"\\nnamespace=\\\"urn:ietf:params:xml:ns:caldav\\\"\"\n        }\n      },\n      \"depth\": 1\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"calendar-proxy-write-for\",\n          \"attrs\": \"name=\\\"calendar-proxy-write-for\\\"\\nnamespace=\\\"http://calendarserver.org/ns/\\\"\"\n        }\n      },\n      \"depth\": 0\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"email-address-set\",\n          \"attrs\": \"name=\\\"email-address-set\\\" namespace=\\\"http://calendarserver.org/ns/\\\"\"\n        }\n      },\n      \"depth\": 1\n    },\n    {\n      \"property\": {\n        \"type\": \"WebDav\",\n        \"data\": {\n          \"type\": \"DisplayName\"\n        }\n      },\n      \"depth\": 1\n    },\n    {\n      \"property\": {\n        \"type\": \"DeadProperty\",\n        \"data\": {\n          \"name\": \"calendar-user-address-set\",\n          \"attrs\": \"name=\\\"calendar-user-address-set\\\"\\nnamespace=\\\"urn:ietf:params:xml:ns:caldav\\\"\"\n        }\n      },\n      \"depth\": 1\n    }\n  ]\n}"
  },
  {
    "path": "crates/dav-proto/resources/requests/report-025.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\"?><A:expand-property\nxmlns:A=\"DAV:\"><A:property name=\"calendar-proxy-read-for\" namespace=\"\nhttp://calendarserver.org/ns/\"><A:property name=\"email-address-set\"\nnamespace=\"http://calendarserver.org/ns/\"/><A:property name=\"displayname\"\nnamespace=\"DAV:\"/><A:property name=\"calendar-user-address-set\"\nnamespace=\"urn:ietf:params:xml:ns:caldav\"/></A:property><A:property\nname=\"calendar-proxy-write-for\"\nnamespace=\"http://calendarserver.org/ns/\"><A:property\nname=\"email-address-set\" namespace=\"http://calendarserver.org/ns/\"/><A:property\nname=\"displayname\" namespace=\"DAV:\"/><A:property\nname=\"calendar-user-address-set\"\nnamespace=\"urn:ietf:params:xml:ns:caldav\"/></A:property></A:expand-property>"
  },
  {
    "path": "crates/dav-proto/resources/responses/001.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:error xmlns:D=\"DAV:\">\n       <D:lock-token-submitted>\n         <D:href>/locked/</D:href>\n       </D:lock-token-submitted>\n     </D:error>"
  },
  {
    "path": "crates/dav-proto/resources/responses/002.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:multistatus xmlns:D=\"DAV:\">\n       <D:response>\n         <D:href>http://www.example.com/file</D:href>\n         <D:propstat>\n           <D:prop>\n               <D:displayname>Box type A</D:displayname>\n           </D:prop>\n           <D:status>HTTP/1.1 200 OK</D:status>\n         </D:propstat>\n         <D:propstat>\n           <D:prop><D:displayname>Box type B</D:displayname></D:prop>\n           <D:status>HTTP/1.1 403 Forbidden</D:status>\n           <D:responsedescription>The user does not have access to the DingALing property.</D:responsedescription>\n         </D:propstat>\n       </D:response>\n       <D:responsedescription>There has been an access violation error.</D:responsedescription>\n     </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/003.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:multistatus xmlns:D=\"DAV:\">\n       <D:response>\n         <D:href>/container/</D:href>\n         <D:propstat>\n           <D:prop>\n             <D:creationdate>1997-12-02T01:42:21Z</D:creationdate>\n             <D:displayname>Example collection</D:displayname>\n             <D:resourcetype><D:collection/></D:resourcetype>\n             <D:supportedlock>\n               <D:lockentry>\n                 <D:lockscope><D:exclusive/></D:lockscope>\n                 <D:locktype><D:write/></D:locktype>\n               </D:lockentry>\n               <D:lockentry>\n                 <D:lockscope><D:shared/></D:lockscope>\n                 <D:locktype><D:write/></D:locktype>\n               </D:lockentry>\n             </D:supportedlock>\n           </D:prop>\n           <D:status>HTTP/1.1 200 OK</D:status>\n         </D:propstat>\n       </D:response>\n       <D:response>\n         <D:href>/container/front.html</D:href>\n         <D:propstat>\n           <D:prop>\n             <D:creationdate>1997-12-02T02:27:21Z</D:creationdate>\n             <D:displayname>Example HTML resource</D:displayname>\n             <D:getcontentlength>4525</D:getcontentlength>\n             <D:getcontenttype>text/html</D:getcontenttype>\n             <D:getetag>\"zzyzx\"</D:getetag>\n             <D:getlastmodified\n               >Mon, 12 Jan 1998 09:25:56 GMT</D:getlastmodified>\n             <D:resourcetype/>\n             <D:supportedlock>\n               <D:lockentry>\n                 <D:lockscope><D:exclusive/></D:lockscope>\n                 <D:locktype><D:write/></D:locktype>\n               </D:lockentry>\n               <D:lockentry>\n                 <D:lockscope><D:shared/></D:lockscope>\n                 <D:locktype><D:write/></D:locktype>\n               </D:lockentry>\n             </D:supportedlock>\n           </D:prop>\n           <D:status>HTTP/1.1 200 OK</D:status>\n         </D:propstat>\n       </D:response>\n     </D:multistatus>\n"
  },
  {
    "path": "crates/dav-proto/resources/responses/004.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <d:multistatus xmlns:d=\"DAV:\">\n       <d:response>\n         <d:href>http://www.example.com/container/resource3</d:href>\n         <d:status>HTTP/1.1 423 Locked</d:status>\n         <d:error><d:lock-token-submitted/></d:error>\n       </d:response>\n     </d:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/005.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:prop xmlns:D=\"DAV:\">\n       <D:lockdiscovery>\n         <D:activelock>\n           <D:lockscope><D:exclusive/></D:lockscope>\n           <D:locktype><D:write/></D:locktype>\n           <D:depth>infinity</D:depth>\n           <D:owner>\n             <D:href>http://example.org/~ejw/contact.html</D:href>\n           </D:owner>\n           <D:timeout>Second-604800</D:timeout>\n           <D:locktoken>\n             <D:href\n             >urn:uuid:e71d4fae-5dec-22d6-fea5-00a0c91e6be4</D:href>\n           </D:locktoken>\n           <D:lockroot>\n             <D:href\n             >http://example.com/workspace/webdav/proposal.doc</D:href>\n           </D:lockroot>\n         </D:activelock>\n       </D:lockdiscovery>\n     </D:prop>"
  },
  {
    "path": "crates/dav-proto/resources/responses/006.xml",
    "content": "     <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:multistatus xmlns:D=\"DAV:\">\n       <D:response>\n         <D:href>http://www.example.com/container/</D:href>\n         <D:propstat>\n           <D:prop>\n             <D:lockdiscovery>\n              <D:activelock>\n               <D:lockscope><D:shared/></D:lockscope>\n               <D:locktype><D:write/></D:locktype>\n               <D:depth>0</D:depth>\n               <D:owner>Jane Smith</D:owner>\n               <D:timeout>Infinite</D:timeout>\n               <D:locktoken>\n                 <D:href\n             >urn:uuid:f81de2ad-7f3d-a1b2-4f3c-00a0c91a9d76</D:href>\n               </D:locktoken>\n               <D:lockroot>\n                 <D:href>http://www.example.com/container/</D:href>\n               </D:lockroot>\n              </D:activelock>\n             </D:lockdiscovery>\n           </D:prop>\n           <D:status>HTTP/1.1 200 OK</D:status>\n         </D:propstat>\n       </D:response>\n     </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/007.xml",
    "content": "      <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n      <D:error xmlns:D=\"DAV:\">\n        <D:lock-token-submitted>\n          <D:href>/workspace/webdav/</D:href>\n        </D:lock-token-submitted>\n      </D:error>"
  },
  {
    "path": "crates/dav-proto/resources/responses/008.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\" xmlns:A=\"urn:ietf:params:xml:ns:caldav\">\n     <D:response>\n       <D:href>http://cal.example.com/bernard/work/abcd2.ics</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:getetag>\"fffff-abcd2\"</D:getetag>\n        <A:calendar-data><![CDATA[BEGIN:VCALENDAR\nVERSION:2.0\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060106T140000\nDURATION:PT1H\nRECURRENCE-ID;TZID=US/Eastern:20060106T120000\nSUMMARY:Event #2 bis bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nEND:VCALENDAR\n]]></A:calendar-data>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n     <D:response>\n       <D:href>http://cal.example.com/bernard/work/abcd3.ics</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:getetag>\"fffff-abcd3\"</D:getetag>\n        <A:calendar-data><![CDATA[BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060104T100000\nDURATION:PT1H\nSUMMARY:Event #3\nUID:DC6C50A017428C5216A2F1CD@example.com\nEND:VEVENT\nEND:VCALENDAR\n]]></A:calendar-data>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/009.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:mkcol-response xmlns:D=\"DAV:\" xmlns:B=\"urn:ietf:params:xml:ns:carddav\">\n     <D:propstat>\n       <D:prop>\n         <D:resourcetype/>\n         <D:displayname/>\n         <B:addressbook-description/>\n       </D:prop>\n       <D:status>HTTP/1.1 200 OK</D:status>\n     </D:propstat>\n   </D:mkcol-response>"
  },
  {
    "path": "crates/dav-proto/resources/responses/010.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\" xmlns:B=\"urn:ietf:params:xml:ns:carddav\">\n     <D:response>\n       <D:href>/home/bernard/addressbook/v102.vcf</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:getetag>\"23ba4d-ff11fb\"</D:getetag>\n        <B:address-data><![CDATA[BEGIN:VCARD\nVERSION:3.0\nNICKNAME:me\nUID:34222-232@example.com\nFN:Cyrus Daboo\nEMAIL:daboo@example.com\nEND:VCARD\n]]></B:address-data>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/011.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\" xmlns:B=\"urn:ietf:params:xml:ns:carddav\">\n     <D:response>\n       <D:href>/home/bernard/addressbook/</D:href>\n       <D:status>HTTP/1.1 507 Insufficient Storage</D:status>\n       <D:error><D:number-of-matches-within-limits/></D:error>\n       <D:responsedescription>Only two matching records were returned</D:responsedescription>\n     </D:response>\n     <D:response>\n       <D:href>/home/bernard/addressbook/v102.vcf</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:getetag>\"23ba4d-ff11fb\"</D:getetag>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n     <D:response>\n       <D:href>/home/bernard/addressbook/v104.vcf</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:getetag>\"23ba4d-ff11fc\"</D:getetag>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/012.xml",
    "content": "   <D:error xmlns:D=\"DAV:\">\n     <D:need-privileges>\n       <D:resource>\n         <D:href>/a</D:href>\n         <D:privilege><D:unbind/></D:privilege>\n       </D:resource>\n       <D:resource>\n         <D:href>/c</D:href>\n         <D:privilege><D:bind/></D:privilege>\n       </D:resource>\n     </D:need-privileges>\n   </D:error>"
  },
  {
    "path": "crates/dav-proto/resources/responses/013.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:principal-search-property-set xmlns:D=\"DAV:\">\n     <D:principal-search-property>\n       <D:prop>\n         <D:displayname/>\n       </D:prop>\n       <D:description>Full name</D:description>\n     </D:principal-search-property>\n     <D:principal-search-property>\n       <D:prop>\n         <D:displayname/>\n       </D:prop>\n       <D:description>Job title</D:description>\n     </D:principal-search-property>\n   </D:principal-search-property-set>"
  },
  {
    "path": "crates/dav-proto/resources/responses/014.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\">\n     <D:response>\n       <D:href>http://www.example.com/papers/</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:supported-privilege-set>\n             <D:supported-privilege>\n               <D:privilege><D:all/></D:privilege>\n              <D:abstract/>\n               <D:description>Any operation</D:description>\n               <D:supported-privilege>\n                 <D:privilege><D:read/></D:privilege>\n                 <D:description>Read any object</D:description>\n                 <D:supported-privilege>\n                   <D:privilege><D:read-acl/></D:privilege>\n                   <D:abstract/>\n                   <D:description>Read ACL</D:description>\n                 </D:supported-privilege>\n                 <D:supported-privilege>\n                   <D:privilege>\n                     <D:read-current-user-privilege-set/>\n                   </D:privilege>\n                   <D:abstract/>\n                   <D:description>Read current user privilege set property</D:description>\n                 </D:supported-privilege>\n               </D:supported-privilege>\n               <D:supported-privilege>\n                 <D:privilege><D:write/></D:privilege>\n                 <D:description>Write any object</D:description>\n                 <D:supported-privilege>\n                   <D:privilege><D:write-acl/></D:privilege>\n                   <D:abstract/>\n                   <D:description>Write ACL</D:description>\n                 </D:supported-privilege>\n                 <D:supported-privilege>\n                   <D:privilege><D:write-properties/></D:privilege>\n                   <D:description>Write properties</D:description>\n                 </D:supported-privilege>\n                 <D:supported-privilege>\n                   <D:privilege><D:write-content/></D:privilege>\n                   <D:description>Write resource content</D:description>\n                 </D:supported-privilege>\n               </D:supported-privilege>\n               <D:supported-privilege>\n                 <D:privilege><D:unlock/></D:privilege>\n                 <D:description>Unlock resource</D:description>\n               </D:supported-privilege>\n             </D:supported-privilege>\n           </D:supported-privilege-set>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/015.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\">\n     <D:response>\n     <D:href>http://www.example.com/papers/</D:href>\n     <D:propstat>\n       <D:prop>\n         <D:current-user-privilege-set>\n           <D:privilege><D:read/></D:privilege>\n         </D:current-user-privilege-set>\n       </D:prop>\n       <D:status>HTTP/1.1 200 OK</D:status>\n     </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/016.xml",
    "content": "   <D:multistatus xmlns:D=\"DAV:\">\n     <D:response>\n       <D:href>http://www.example.com/papers/</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:acl>\n           <D:ace>\n             <D:principal>\n               <D:href\n               >http://www.example.com/acl/groups/maintainers</D:href>\n             </D:principal>\n             <D:grant>\n               <D:privilege><D:write/></D:privilege>\n             </D:grant>\n           </D:ace>\n           <D:ace>\n             <D:principal>\n               <D:all/>\n             </D:principal>\n             <D:grant>\n               <D:privilege><D:read/></D:privilege>\n             </D:grant>\n           </D:ace>\n         </D:acl>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/017.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\">\n     <D:response>\n       <D:href>http://www.example.com/papers/</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:acl-restrictions>\n             <D:grant-only/>\n             <D:required-principal>\n               <D:all/>\n             </D:required-principal>\n           </D:acl-restrictions>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/018.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\">\n     <D:response>\n       <D:href>http://www.example.com/papers/</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:principal-collection-set>\n             <D:href>http://www.example.com/acl/users/</D:href>\n             <D:href>http://www.example.com/acl/groups/</D:href>\n           </D:principal-collection-set>\n         </D:prop>\n       <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>"
  },
  {
    "path": "crates/dav-proto/resources/responses/019.xml",
    "content": "   <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:multistatus xmlns:D=\"DAV:\">\n     <D:response>\n       <D:href>http://www.example.com/top/container/</D:href>\n       <D:propstat>\n         <D:prop>\n           <D:owner>\n             <D:href>http://www.example.com/users/gclemm</D:href>\n           </D:owner>\n           <D:supported-privilege-set>\n             <D:supported-privilege>\n               <D:privilege><D:all/></D:privilege>\n               <D:abstract/>\n               <D:description>Any operation</D:description>\n               <D:supported-privilege>\n                 <D:privilege><D:read/></D:privilege>\n                 <D:description>Read any object</D:description>\n               </D:supported-privilege>\n               <D:supported-privilege>\n                 <D:privilege><D:write/></D:privilege>\n                 <D:abstract/>\n                 <D:description>Write any object</D:description>\n               </D:supported-privilege>\n               <D:supported-privilege>\n                 <D:privilege><D:read-acl/></D:privilege>\n                 <D:description>Read the ACL</D:description>\n               </D:supported-privilege>\n               <D:supported-privilege>\n                 <D:privilege><D:write-acl/></D:privilege>\n                 <D:description>Write the ACL</D:description>\n               </D:supported-privilege>\n             </D:supported-privilege>\n           </D:supported-privilege-set>\n           <D:current-user-privilege-set>\n             <D:privilege><D:read/></D:privilege>\n             <D:privilege><D:read-acl/></D:privilege>\n           </D:current-user-privilege-set>\n           <D:acl>\n             <D:ace>\n               <D:principal>\n                 <D:href>http://www.example.com/users/esedlar</D:href>\n               </D:principal>\n               <D:grant>\n                 <D:privilege><D:read/></D:privilege>\n                 <D:privilege><D:write/></D:privilege>\n                 <D:privilege><D:read-acl/></D:privilege>\n               </D:grant>\n             </D:ace>\n             <D:ace>\n               <D:principal>\n                 <D:href>http://www.example.com/groups/mrktng</D:href>\n               </D:principal>\n               <D:deny>\n                 <D:privilege><D:read/></D:privilege>\n               </D:deny>\n             </D:ace>\n             <D:ace>\n               <D:principal>\n                 <D:property><D:owner/></D:property>\n               </D:principal>\n               <D:grant>\n                 <D:privilege><D:read-acl/></D:privilege>\n                 <D:privilege><D:write-acl/></D:privilege>\n               </D:grant>\n             </D:ace>\n             <D:ace>\n               <D:principal><D:all/></D:principal>\n               <D:grant>\n                 <D:privilege><D:read/></D:privilege>\n               </D:grant>\n               <D:inherited>\n                 <D:href>http://www.example.com/top</D:href>\n               </D:inherited>\n             </D:ace>\n           </D:acl>\n         </D:prop>\n         <D:status>HTTP/1.1 200 OK</D:status>\n       </D:propstat>\n     </D:response>\n   </D:multistatus>\n"
  },
  {
    "path": "crates/dav-proto/resources/responses/020.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<A:schedule-response xmlns:D=\"DAV:\" xmlns:A=\"urn:ietf:params:xml:ns:caldav\">\n<A:response>\n<A:recipient>\n<D:href>mailto:wilfredo@example.com</D:href>\n</A:recipient>\n<A:request-status>2.0;Success</A:request-status>\n<A:calendar-data><![CDATA[BEGIN:VCALENDAR]]></A:calendar-data>\n</A:response>\n<A:response>\n<A:recipient>\n<D:href>mailto:bernard@example.net</D:href>\n</A:recipient>\n<A:request-status>2.0;Success</A:request-status>\n<A:calendar-data><![CDATA[END:VCALENDAR]]></A:calendar-data>\n</A:response>\n<A:response>\n<A:recipient>\n<D:href>mailto:mike@example.org</D:href>\n</A:recipient>\n<A:request-status>3.7;Invalid calendar user</A:request-status>\n</A:response>\n</A:schedule-response>\n"
  },
  {
    "path": "crates/dav-proto/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse calcard::vcard::VCardVersion;\nuse compact_str::{CompactString, ToCompactString};\nuse trc::Value;\n\npub mod parser;\npub mod requests;\npub mod responses;\npub mod schema;\n\npub fn xml_pretty_print(xml_string: &str) -> String {\n    // Create a reader\n    let mut reader = quick_xml::Reader::from_str(xml_string);\n    let mut writer = quick_xml::Writer::new_with_indent(std::io::Cursor::new(Vec::new()), b' ', 2);\n    let mut buf = Vec::new();\n    loop {\n        match reader.read_event_into(&mut buf) {\n            Ok(quick_xml::events::Event::Eof) => break,\n            Ok(event) => {\n                writer.write_event(event).unwrap();\n            }\n            Err(e) => panic!(\"Error at position {}: {:?}\", reader.buffer_position(), e),\n        }\n        buf.clear();\n    }\n\n    let result = writer.into_inner().into_inner();\n    String::from_utf8(result).unwrap()\n}\n\n#[derive(Debug, Default, PartialEq, Eq)]\npub struct RequestHeaders<'x> {\n    pub uri: &'x str,\n    pub depth: Depth,\n    pub timeout: Timeout,\n    pub content_type: Option<&'x str>,\n    pub destination: Option<&'x str>,\n    pub lock_token: Option<&'x str>,\n    pub max_vcard_version: Option<VCardVersion>,\n    pub no_schedule_reply: bool,\n    pub if_schedule_tag: Option<u32>,\n    pub overwrite_fail: bool,\n    pub no_timezones: bool,\n    pub ret: Return,\n    pub depth_no_root: bool,\n    pub if_: Vec<If<'x>>,\n}\n\npub struct ResourceState<T: AsRef<str>> {\n    pub resource: Option<T>,\n    pub etag: T,\n    pub state_token: T,\n}\n\n#[derive(Debug, Default, PartialEq, Eq, Clone, Copy)]\npub enum Return {\n    Minimal,\n    Representation,\n    #[default]\n    Default,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone)]\npub struct If<'x> {\n    pub resource: Option<&'x str>,\n    pub list: Vec<Condition<'x>>,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone)]\npub enum Condition<'x> {\n    StateToken { is_not: bool, token: &'x str },\n    ETag { is_not: bool, tag: &'x str },\n    Exists { is_not: bool },\n}\n\n#[derive(Debug, Default, PartialEq, Eq, Clone, Copy)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum Timeout {\n    Infinite,\n    Second(u64),\n    #[default]\n    None,\n}\n\n#[derive(Debug, Default, PartialEq, Eq, Clone, Copy)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum Depth {\n    Zero,\n    One,\n    Infinity,\n    #[default]\n    None,\n}\n\nimpl From<&RequestHeaders<'_>> for Value {\n    fn from(headers: &RequestHeaders<'_>) -> Self {\n        let mut values = Vec::with_capacity(4);\n        if headers.depth != Depth::None {\n            values.push(Value::String(CompactString::const_new(\"Depth\")));\n            values.push(match headers.depth {\n                Depth::Zero => Value::Int(0),\n                Depth::One => Value::Int(1),\n                Depth::Infinity => Value::String(CompactString::const_new(\"infinity\")),\n                Depth::None => Value::None,\n            });\n        }\n        if headers.timeout != Timeout::None {\n            values.push(Value::String(CompactString::const_new(\"Timeout\")));\n            values.push(match headers.timeout {\n                Timeout::Infinite => Value::String(CompactString::const_new(\"infinite\")),\n                Timeout::Second(n) => Value::Int(n as i64),\n                Timeout::None => Value::None,\n            });\n        }\n        for (name, header_value) in [\n            (\"Content-Type\", headers.content_type),\n            (\"Destination\", headers.destination),\n            (\"Lock-Token\", headers.lock_token),\n        ] {\n            if let Some(value) = header_value {\n                values.push(CompactString::const_new(name).into());\n                values.push(value.to_compact_string().into());\n            }\n        }\n        for (name, is_set) in [\n            (\"Overwrite\", headers.overwrite_fail),\n            (\"No-Timezones\", headers.no_timezones),\n            (\"Depth-No-Root\", headers.depth_no_root),\n        ] {\n            if is_set {\n                values.push(CompactString::const_new(name).into());\n            }\n        }\n        for if_ in &headers.if_ {\n            values.push(CompactString::const_new(\"If\").into());\n            let mut if_values = Vec::with_capacity(if_.list.len() * 2 + 1);\n            if let Some(resource) = if_.resource {\n                if_values.push(Value::String(resource.to_compact_string()));\n            }\n            for condition in &if_.list {\n                match condition {\n                    Condition::StateToken { is_not, token } => {\n                        if *is_not {\n                            if_values.push(Value::String(CompactString::const_new(\"!State-Token\")));\n                        } else {\n                            if_values.push(Value::String(CompactString::const_new(\"State-Token\")));\n                        }\n                        if_values.push(Value::String(token.to_compact_string()));\n                    }\n                    Condition::ETag { is_not, tag } => {\n                        if *is_not {\n                            if_values.push(Value::String(CompactString::const_new(\"!ETag\")));\n                        } else {\n                            if_values.push(Value::String(CompactString::const_new(\"ETag\")));\n                        }\n                        if_values.push(Value::String(tag.to_compact_string()));\n                    }\n                    Condition::Exists { is_not } => {\n                        if *is_not {\n                            if_values.push(Value::String(CompactString::const_new(\"!Exists\")));\n                        } else {\n                            if_values.push(Value::String(CompactString::const_new(\"Exists\")));\n                        }\n                    }\n                }\n            }\n            values.push(Value::Array(if_values));\n        }\n\n        Value::Array(values)\n    }\n}\n\n/*\n\n\nImplemented:\n\nRFC4918 - HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV)\nRFC5689 - Extended MKCOL for Web Distributed Authoring and Versioning (WebDAV)\nRFC6578 - Collection Synchronization for Web Distributed Authoring and Versioning (WebDAV)\nRFC3744 - Web Distributed Authoring and Versioning (WebDAV) Access Control Protocol\nRFC4331 - Quota and Size Properties for Distributed Authoring and Versioning (DAV) Collections\nRFC5397 - WebDAV Current Principal Extension\nRFC8144 - Use of the Prefer Header Field in Web Distributed Authoring and Versioning (WebDAV)\nRFC4791 - Calendaring Extensions to WebDAV (CalDAV)\nRFC7809 - Calendaring Extensions to WebDAV (CalDAV) Time Zones by Reference\nRFC6638 - Scheduling Extensions to CalDAV\nRFC6352 - CardDAV vCard Extensions to Web Distributed Authoring and Versioning (WebDAV)\nRFC6764 - Locating Services for Calendaring Extensions to WebDAV (CalDAV) and vCard Extensions to WebDAV (CardDAV)\n\nOut of scope:\n\nRFC5842 - Binding Extensions to Web Distributed Authoring and Versioning (WebDAV)\nRFC4316 - Datatypes for Web Distributed Authoring and Versioning (WebDAV) Properties\nRFC4709 - Mounting Web Distributed Authoring and Versioning (WebDAV) Servers\nRFC3648 - Web Distributed Authoring and Versioning (WebDAV) Ordered Collections Protocol\nRFC4437 - Web Distributed Authoring and Versioning (WebDAV) Redirect Reference Resources\nRFC8607 - Calendaring Extensions to WebDAV (CalDAV) Managed Attachments\nRFC5995 - Using POST to Add Members to Web Distributed Authoring and Versioning (WebDAV) Collections\nRFC3253 - Versioning Extensions to WebDAV (Web Distributed Authoring and Versioning)\nRFC5323 - Web Distributed Authoring and Versioning (WebDAV) SEARCH\n\n\n*/\n"
  },
  {
    "path": "crates/dav-proto/src/parser/header.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{Condition, Depth, If, RequestHeaders, ResourceState, Return, Timeout};\nuse calcard::vcard::VCardVersion;\n\nimpl<'x> RequestHeaders<'x> {\n    pub fn new(uri: &'x str) -> Self {\n        RequestHeaders {\n            uri,\n            ..Default::default()\n        }\n    }\n\n    pub fn parse(&mut self, key: &str, value: &'x str) -> bool {\n        hashify::fnc_map_ignore_case!(key.as_bytes(),\n            \"Depth\" => {\n                if let Some(depth) = Depth::parse(value.as_bytes()) {\n                    self.depth = depth;\n                    return true;\n                }\n            },\n            \"Destination\" => {\n                self.destination = Some(value);\n                return true;\n            },\n            \"Lock-Token\" => {\n                self.lock_token = Some(try_unwrap_coded_url(value));\n                return true;\n            },\n            \"If\" => {\n                let num = self.if_.len();\n                self.parse_if(value);\n                return self.if_.len() != num;\n            },\n            \"If-Match\" => {\n                let num = self.if_.len();\n                self.parse_if_match(value, false);\n                return self.if_.len() != num;\n            },\n            \"If-None-Match\" => {\n                let num = self.if_.len();\n                self.parse_if_match(value, true);\n                return self.if_.len() != num;\n            },\n            \"Timeout\" => {\n                let value = value.split_once(',').map(|(first, _)| first).unwrap_or(value).trim();\n                if let Some(seconds) = value.strip_prefix(\"Second-\") {\n                    if let Ok(seconds) = seconds.parse() {\n                        self.timeout = Timeout::Second(seconds);\n                        return true;\n                    }\n                } else if value == \"Infinite\" {\n                    self.timeout = Timeout::Infinite;\n                    return true;\n                }\n            },\n            \"Overwrite\" => {\n                self.overwrite_fail = value == \"F\";\n                return true;\n            },\n            \"CalDAV-Timezones\" => {\n                self.no_timezones = value == \"F\";\n                return true;\n            },\n            \"Prefer\" => {\n                for value in value.split(&[',', ';']) {\n                    match value.trim() {\n                        \"return=minimal\" => self.ret = Return::Minimal,\n                        \"return=representation\" => self.ret = Return::Representation,\n                        \"depth-noroot\" => self.depth_no_root = true,\n                        _ => {}\n                    }\n                }\n            },\n            \"Content-Type\" => {\n                let value = value.trim();\n                if (2..=127).contains(&value.len()) {\n                    self.content_type = Some(value);\n                }\n                return true;\n            },\n            \"Accept\" => {\n                for value in value.split(',') {\n                    if value.trim().starts_with(\"text/vcard\") && let Some(version) = value.split_once(\"version=\")\n                                .and_then(|(_, version)| VCardVersion::try_parse(version.trim())) {\n                        if let Some(max_vcard_version) = &mut self.max_vcard_version {\n                            if version > *max_vcard_version {\n                                *max_vcard_version = version;\n                            }\n                        } else {\n                            self.max_vcard_version = Some(version);\n                        }\n                    }\n                }\n                return true;\n            },\n            \"If-Schedule-Tag-Match\" => {\n                self.if_schedule_tag = value.trim().trim_matches('\"').parse().ok();\n                return true;\n            },\n            \"Schedule-Reply\" => {\n                self.no_schedule_reply = value == \"F\";\n                return true;\n            },\n            _ => {}\n        );\n\n        false\n    }\n\n    pub fn has_if(&self) -> bool {\n        !self.if_.is_empty()\n    }\n\n    pub fn eval_if_resources(&self) -> impl Iterator<Item = &str> {\n        self.if_.iter().filter_map(|if_| if_.resource)\n    }\n\n    pub fn eval_if<T>(&self, resources: &[ResourceState<T>]) -> bool\n    where\n        T: AsRef<str>,\n    {\n        if self.if_.is_empty() {\n            return true;\n        }\n\n        'outer: for if_ in &self.if_ {\n            if if_.list.is_empty() {\n                continue;\n            }\n\n            let (current_token, current_etag) = resources\n                .iter()\n                .find_map(|r| {\n                    if if_.resource == r.resource.as_ref().map(|v| v.as_ref()) {\n                        Some((r.state_token.as_ref(), r.etag.as_ref()))\n                    } else {\n                        None\n                    }\n                })\n                .unwrap_or_default();\n\n            for cond in if_.list.iter() {\n                match cond {\n                    Condition::StateToken { is_not, token } => {\n                        if !((current_token == *token) ^ is_not) {\n                            continue 'outer;\n                        }\n                    }\n                    Condition::ETag { is_not, tag } => {\n                        if !((current_etag == *tag) ^ is_not) {\n                            continue 'outer;\n                        }\n                    }\n                    Condition::Exists { is_not } => {\n                        if !((current_etag.is_empty()) ^ is_not) {\n                            continue 'outer;\n                        }\n                    }\n                }\n            }\n\n            return true;\n        }\n\n        false\n    }\n\n    fn parse_if(&mut self, value: &'x str) {\n        let value = value.as_bytes();\n        let mut iter = value.iter().enumerate();\n        let mut resource = None;\n\n        while let Some((idx, ch)) = iter.next() {\n            match ch {\n                b'<' if resource.is_none() => {\n                    for (to_idx, ch) in iter.by_ref() {\n                        if *ch == b'>' {\n                            resource = Some(std::str::from_utf8(&value[idx + 1..to_idx]).unwrap());\n                            break;\n                        }\n                    }\n                }\n                b'(' => {\n                    let mut is_not = false;\n                    let mut conditions = Vec::new();\n                    while let Some((idx, ch)) = iter.next() {\n                        match ch {\n                            b'N' => {\n                                if matches!(iter.next(), Some((_, b'o')))\n                                    && matches!(iter.next(), Some((_, b't')))\n                                {\n                                    is_not = true;\n                                } else {\n                                    return;\n                                }\n                            }\n                            b'<' | b'[' => {\n                                let (stop_char, is_etag) = match ch {\n                                    b'<' => (b'>', false),\n                                    b'[' => (b']', true),\n                                    _ => unreachable!(),\n                                };\n\n                                for (to_idx, ch) in iter.by_ref() {\n                                    if *ch == stop_char {\n                                        let value =\n                                            std::str::from_utf8(&value[idx + 1..to_idx]).unwrap();\n                                        let condition = if is_etag {\n                                            Condition::ETag { is_not, tag: value }\n                                        } else {\n                                            Condition::StateToken {\n                                                is_not,\n                                                token: value,\n                                            }\n                                        };\n                                        conditions.push(condition);\n                                        is_not = false;\n                                        break;\n                                    }\n                                }\n                            }\n                            b')' => {\n                                self.if_.push(If {\n                                    resource: resource.take(),\n                                    list: conditions,\n                                });\n                                break;\n                            }\n                            _ => {\n                                if !ch.is_ascii_whitespace() {\n                                    return;\n                                }\n                            }\n                        }\n                    }\n                }\n                _ => {\n                    if !ch.is_ascii_whitespace() {\n                        return;\n                    }\n                }\n            }\n        }\n    }\n\n    pub fn parse_if_match(&mut self, value: &'x str, is_not: bool) {\n        if value == \"*\" {\n            self.if_.push(If {\n                resource: None,\n                list: vec![Condition::Exists { is_not }],\n            });\n        } else if !is_not {\n            for etag in value.split(',') {\n                self.if_.push(If {\n                    resource: None,\n                    list: vec![Condition::ETag {\n                        is_not,\n                        tag: etag.trim(),\n                    }],\n                });\n            }\n        } else {\n            let mut etags = Vec::new();\n            for etag in value.split(',') {\n                etags.push(Condition::ETag {\n                    is_not,\n                    tag: etag.trim(),\n                });\n            }\n            self.if_.push(If {\n                resource: None,\n                list: etags,\n            });\n        }\n    }\n\n    pub fn base_uri(&self) -> Option<&str> {\n        dav_base_uri(self.uri)\n    }\n}\n\npub fn dav_base_uri(uri: &str) -> Option<&str> {\n    // From a path ../dav/collection/account/..\n    // returns ../dav/collection/account without the trailing slash\n\n    let uri = uri.as_bytes();\n    let mut found_dav = false;\n    let mut last_idx = 0;\n    let mut sep_count = 0;\n\n    for (idx, ch) in uri.iter().enumerate() {\n        if *ch == b'/' {\n            if !found_dav {\n                found_dav = uri.get(idx + 1..idx + 5).is_some_and(|s| s == b\"dav/\");\n            } else if found_dav {\n                if sep_count == 2 {\n                    break;\n                }\n                sep_count += 1;\n            }\n        }\n        last_idx = idx;\n    }\n\n    if sep_count == 2 {\n        uri.get(..last_idx + 1)\n            .map(|uri| std::str::from_utf8(uri).unwrap())\n    } else {\n        None\n    }\n}\n\nimpl Depth {\n    pub fn parse(value: &[u8]) -> Option<Self> {\n        hashify::tiny_map!(value,\n            \"0\" => Depth::Zero,\n            \"1\" => Depth::One,\n            \"infinity\" => Depth::Infinity,\n            \"infinite\" => Depth::Infinity,\n        )\n    }\n}\n\nfn try_unwrap_coded_url(url: &str) -> &str {\n    url.strip_prefix(\"<\")\n        .and_then(|url| url.strip_suffix(\">\"))\n        .unwrap_or(url)\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn base_uri() {\n        for (uri, expected_base) in [\n            (\n                \"http://host/dav/collection/account/test/\",\n                Some(\"http://host/dav/collection/account\"),\n            ),\n            (\n                \"http://host/dav/collection/account/test\",\n                Some(\"http://host/dav/collection/account\"),\n            ),\n            (\n                \"http://host/dav/collection/account/\",\n                Some(\"http://host/dav/collection/account\"),\n            ),\n            (\n                \"http://host/dav/collection/account\",\n                Some(\"http://host/dav/collection/account\"),\n            ),\n            (\n                \"http://host/dev/dav/collection/account/test/\",\n                Some(\"http://host/dev/dav/collection/account\"),\n            ),\n            (\n                \"http://host/dev/dav/collection/account/test\",\n                Some(\"http://host/dev/dav/collection/account\"),\n            ),\n            (\n                \"http://host/dev/dav/collection/account/\",\n                Some(\"http://host/dev/dav/collection/account\"),\n            ),\n            (\n                \"http://host/dev/dav/collection/account\",\n                Some(\"http://host/dev/dav/collection/account\"),\n            ),\n            (\n                \"/dav/collection/account/test/\",\n                Some(\"/dav/collection/account\"),\n            ),\n            (\n                \"/dav/collection/account/test\",\n                Some(\"/dav/collection/account\"),\n            ),\n            (\"/dav/collection/account/\", Some(\"/dav/collection/account\")),\n            (\"/dav/collection/account\", Some(\"/dav/collection/account\")),\n        ] {\n            assert_eq!(RequestHeaders::new(uri).base_uri(), expected_base);\n        }\n    }\n\n    #[test]\n    fn eval_if_header() {\n        let mut headers = RequestHeaders::default();\n        assert!(headers.parse(\n            \"If\",\n            r#\"(<urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>\n   [\"I am an ETag\"])\n   ([\"I am another ETag\"])\"#,\n        ));\n\n        assert!(headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n            etag: \"\\\"I am an ETag\\\"\"\n        }]));\n        assert!(headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"\",\n            etag: \"\\\"I am another ETag\\\"\"\n        }]));\n        assert!(!headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"\",\n            etag: \"\\\"Unknown ETag\\\"\"\n        }]));\n        assert!(!headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n            etag: \"\"\n        }]));\n        assert!(!headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n            etag: \"\\\"Other ETag\\\"\"\n        }]));\n        assert!(!headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"\",\n            etag: \"\\\"I am an ETag\\\"\"\n        }]));\n        assert!(!headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:blah\",\n            etag: \"\\\"I am an ETag\\\"\"\n        }]));\n\n        assert!(headers.parse(\n            \"If\",\n            r#\"(Not <urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>\n     <urn:uuid:58f202ac-22cf-11d1-b12d-002035b29092>)\"#,\n        ));\n        assert!(headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:uuid:58f202ac-22cf-11d1-b12d-002035b29092\",\n            etag: \"\"\n        }]));\n        assert!(!headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n            etag: \"\"\n        }]));\n\n        assert!(headers.parse(\n            \"If\",\n            r#\"(<urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>)\n       (Not <DAV:no-lock>)\"#\n        ));\n        assert!(headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n            etag: \"\"\n        }]));\n        assert!(headers.eval_if(&[ResourceState {\n            resource: None,\n            state_token: \"urn:other-token\",\n            etag: \"\"\n        }]));\n    }\n\n    #[test]\n    fn parse_headers() {\n        let mut headers = RequestHeaders::default();\n        assert!(headers.parse(\"Depth\", \"0\"));\n        assert_eq!(headers.depth, Depth::Zero);\n\n        assert!(headers.parse(\"Destination\", \"/path/to/destination\"));\n        assert_eq!(headers.destination, Some(\"/path/to/destination\"));\n\n        assert!(headers.parse(\"Lock-Token\", \"<urn:uuid:1234>\"));\n        assert_eq!(headers.lock_token, Some(\"urn:uuid:1234\"));\n\n        for (input, expected) in [\n            (\n                \"<urn:uuid:1234>(<urn:uuid:1234>)\",\n                vec![If {\n                    resource: \"urn:uuid:1234\".into(),\n                    list: vec![Condition::StateToken {\n                        is_not: false,\n                        token: \"urn:uuid:1234\",\n                    }],\n                }],\n            ),\n            (\n                \"<>(<>)\",\n                vec![If {\n                    resource: \"\".into(),\n                    list: vec![Condition::StateToken {\n                        is_not: false,\n                        token: \"\",\n                    }],\n                }],\n            ),\n            (\n                r#\"(<urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>\n       [\"I am an ETag\"])\n       ([\"I am another ETag\"])\"#,\n                vec![\n                    If {\n                        resource: None,\n                        list: vec![\n                            Condition::StateToken {\n                                is_not: false,\n                                token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n                            },\n                            Condition::ETag {\n                                is_not: false,\n                                tag: \"\\\"I am an ETag\\\"\",\n                            },\n                        ],\n                    },\n                    If {\n                        resource: None,\n                        list: vec![Condition::ETag {\n                            is_not: false,\n                            tag: \"\\\"I am another ETag\\\"\",\n                        }],\n                    },\n                ],\n            ),\n            (\n                r#\"(Not <urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>\n     <urn:uuid:58f202ac-22cf-11d1-b12d-002035b29092>)\"#,\n                vec![If {\n                    resource: None,\n                    list: vec![\n                        Condition::StateToken {\n                            is_not: true,\n                            token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n                        },\n                        Condition::StateToken {\n                            is_not: false,\n                            token: \"urn:uuid:58f202ac-22cf-11d1-b12d-002035b29092\",\n                        },\n                    ],\n                }],\n            ),\n            (\n                r#\"(<urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>)\n       (Not <DAV:no-lock>)\"#,\n                vec![\n                    If {\n                        resource: None,\n                        list: vec![Condition::StateToken {\n                            is_not: false,\n                            token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n                        }],\n                    },\n                    If {\n                        resource: None,\n                        list: vec![Condition::StateToken {\n                            is_not: true,\n                            token: \"DAV:no-lock\",\n                        }],\n                    },\n                ],\n            ),\n            (\n                r#\"</resource1>\n                 (<urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>\n                 [W/\"A weak ETag\"]) ([\"strong ETag\"])\"#,\n                vec![\n                    If {\n                        resource: \"/resource1\".into(),\n                        list: vec![\n                            Condition::StateToken {\n                                is_not: false,\n                                token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n                            },\n                            Condition::ETag {\n                                is_not: false,\n                                tag: \"W/\\\"A weak ETag\\\"\",\n                            },\n                        ],\n                    },\n                    If {\n                        resource: None,\n                        list: vec![Condition::ETag {\n                            is_not: false,\n                            tag: \"\\\"strong ETag\\\"\",\n                        }],\n                    },\n                ],\n            ),\n            (\n                r#\"<http://www.example.com/specs/>\n            (<urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2>)\"#,\n                vec![If {\n                    resource: \"http://www.example.com/specs/\".into(),\n                    list: vec![Condition::StateToken {\n                        is_not: false,\n                        token: \"urn:uuid:181d4fae-7d8c-11d0-a765-00a0c91e6bf2\",\n                    }],\n                }],\n            ),\n            (\n                r#\"</specs/rfc2518.doc> ([\"4217\"])\"#,\n                vec![If {\n                    resource: \"/specs/rfc2518.doc\".into(),\n                    list: vec![Condition::ETag {\n                        is_not: false,\n                        tag: \"\\\"4217\\\"\",\n                    }],\n                }],\n            ),\n            (\n                r#\"</specs/rfc2518.doc> (Not [\"4217\"])\"#,\n                vec![If {\n                    resource: \"/specs/rfc2518.doc\".into(),\n                    list: vec![Condition::ETag {\n                        is_not: true,\n                        tag: \"\\\"4217\\\"\",\n                    }],\n                }],\n            ),\n            (\n                r#\"</test/file.txt> ([\"1234\"]) </specs/rfc2518.doc> (Not [\"4217\"])\"#,\n                vec![\n                    If {\n                        resource: \"/test/file.txt\".into(),\n                        list: vec![Condition::ETag {\n                            is_not: false,\n                            tag: \"\\\"1234\\\"\",\n                        }],\n                    },\n                    If {\n                        resource: \"/specs/rfc2518.doc\".into(),\n                        list: vec![Condition::ETag {\n                            is_not: true,\n                            tag: \"\\\"4217\\\"\",\n                        }],\n                    },\n                ],\n            ),\n        ] {\n            assert!(headers.parse(\"If\", input));\n            assert_eq!(headers.if_, expected, \"Failed for input: {}\", input);\n            headers.if_.clear();\n        }\n\n        assert!(headers.parse(\"If-Match\", \"*\"));\n        assert_eq!(\n            headers.if_,\n            vec![If {\n                resource: None,\n                list: vec![Condition::Exists { is_not: false }],\n            }]\n        );\n        headers.if_.clear();\n\n        assert!(headers.parse(\"If-None-Match\", \"etag1, etag2\"));\n        assert_eq!(\n            headers.if_,\n            vec![If {\n                resource: None,\n                list: vec![\n                    Condition::ETag {\n                        is_not: true,\n                        tag: \"etag1\",\n                    },\n                    Condition::ETag {\n                        is_not: true,\n                        tag: \"etag2\",\n                    }\n                ],\n            },]\n        );\n\n        assert!(headers.parse(\"Timeout\", \"Second-10\"));\n        assert_eq!(headers.timeout, Timeout::Second(10));\n\n        assert!(headers.parse(\"Timeout\", \"Infinite, Second-4100000000\"));\n        assert_eq!(headers.timeout, Timeout::Infinite);\n\n        assert!(headers.parse(\"Overwrite\", \"F\"));\n        assert!(headers.overwrite_fail);\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/parser/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    borrow::Cow,\n    fmt::{Display, Formatter},\n};\n\nuse quick_xml::events::BytesStart;\nuse tokenizer::Tokenizer;\n\nuse crate::schema::{Element, NamedElement, Namespace};\n\npub mod header;\npub mod property;\npub mod tokenizer;\n\n#[derive(Debug, Clone)]\npub enum Error {\n    Xml(Box<quick_xml::Error>),\n    UnexpectedToken(Box<UnexpectedToken>),\n}\n\n#[derive(Debug, Clone)]\npub struct UnexpectedToken {\n    pub expected: Option<Token<'static>>,\n    pub found: Token<'static>,\n}\n\npub type Result<T> = std::result::Result<T, Error>;\n\n#[derive(Debug, Clone)]\npub enum Token<'x> {\n    ElementStart {\n        name: NamedElement,\n        raw: RawElement<'x>,\n    },\n    ElementEnd,\n    Bytes(Cow<'x, [u8]>),\n    Text(Cow<'x, str>),\n    UnknownElement(RawElement<'x>),\n    Eof,\n}\n\n#[derive(Debug, Clone)]\npub struct RawElement<'x> {\n    pub element: BytesStart<'x>,\n    pub namespace: Option<Cow<'static, [u8]>>,\n}\n\npub trait DavParser: Sized {\n    fn parse(stream: &mut Tokenizer<'_>) -> Result<Self>;\n}\n\npub trait XmlValueParser: Sized {\n    fn parse_bytes(bytes: &[u8]) -> Option<Self>;\n    fn parse_str(text: &str) -> Option<Self>;\n}\n\nimpl NamedElement {\n    pub fn dav(element: Element) -> NamedElement {\n        NamedElement {\n            ns: Namespace::Dav,\n            element,\n        }\n    }\n\n    pub fn caldav(element: Element) -> NamedElement {\n        NamedElement {\n            ns: Namespace::CalDav,\n            element,\n        }\n    }\n\n    pub fn carddav(element: Element) -> NamedElement {\n        NamedElement {\n            ns: Namespace::CardDav,\n            element,\n        }\n    }\n\n    pub fn calendarserver(element: Element) -> NamedElement {\n        NamedElement {\n            ns: Namespace::CalendarServer,\n            element,\n        }\n    }\n}\n\nimpl Token<'_> {\n    pub fn into_owned(self) -> Token<'static> {\n        match self {\n            Token::ElementStart { name, raw } => Token::ElementStart {\n                name,\n                raw: raw.into_owned(),\n            },\n            Token::ElementEnd => Token::ElementEnd,\n            Token::Bytes(bytes) => Token::Bytes(bytes.into_owned().into()),\n            Token::Text(text) => Token::Text(text.into_owned().into()),\n            Token::UnknownElement(raw) => Token::UnknownElement(raw.into_owned()),\n            Token::Eof => Token::Eof,\n        }\n    }\n\n    pub fn into_unexpected(self) -> Error {\n        Error::UnexpectedToken(Box::new(UnexpectedToken {\n            expected: None,\n            found: self.into_owned(),\n        }))\n    }\n}\n\nimpl<'x> RawElement<'x> {\n    pub fn new(element: BytesStart<'x>) -> Self {\n        RawElement {\n            element,\n            namespace: None,\n        }\n    }\n\n    pub fn with_namespace(self, namespace: quick_xml::name::Namespace<'_>) -> Self {\n        RawElement {\n            element: self.element,\n            namespace: Some(Cow::Owned(namespace.into_inner().to_vec())),\n        }\n    }\n\n    pub fn with_namespace_static(self, namespace: &'static [u8]) -> Self {\n        RawElement {\n            element: self.element,\n            namespace: Some(Cow::Borrowed(namespace)),\n        }\n    }\n\n    pub fn into_owned(self) -> RawElement<'static> {\n        RawElement {\n            element: self.element.into_owned(),\n            namespace: self.namespace,\n        }\n    }\n}\n\n#[cfg(test)]\nimpl PartialEq for Token<'_> {\n    fn eq(&self, other: &Self) -> bool {\n        match (self, other) {\n            (\n                Self::ElementStart {\n                    name: l_name,\n                    raw: l_raw,\n                },\n                Self::ElementStart {\n                    name: r_name,\n                    raw: r_raw,\n                },\n            ) => {\n                l_name == r_name\n                    && l_raw\n                        .element\n                        .attributes_raw()\n                        .trim_ascii()\n                        .eq_ignore_ascii_case(r_raw.element.attributes_raw().trim_ascii())\n            }\n            (Self::Bytes(l0), Self::Bytes(r0)) => l0 == r0,\n            (Self::Text(l0), Self::Text(r0)) => l0 == r0,\n            (Self::UnknownElement(l0), Self::UnknownElement(r0)) => l0\n                .element\n                .as_ref()\n                .eq_ignore_ascii_case(r0.element.as_ref()),\n            _ => core::mem::discriminant(self) == core::mem::discriminant(other),\n        }\n    }\n}\n\nimpl NamedElement {\n    pub fn into_unexpected(self) -> Error {\n        Error::UnexpectedToken(Box::new(UnexpectedToken {\n            expected: None,\n            found: Token::ElementStart {\n                name: self,\n                raw: RawElement::new(BytesStart::new(\"\")),\n            },\n        }))\n    }\n}\n\nimpl Default for RawElement<'_> {\n    fn default() -> Self {\n        RawElement::new(BytesStart::new(\"\"))\n    }\n}\n\nimpl Display for Error {\n    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Error::Xml(err) => write!(f, \"XML error: {}\", err),\n            Error::UnexpectedToken(err) => {\n                write!(f, \"Unexpected token: {:?}\", err.found)?;\n                if let Some(expected) = &err.expected {\n                    write!(f, \", expected: {expected:?}\")?;\n                }\n                Ok(())\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/parser/property.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{DavParser, RawElement, Token, XmlValueParser, tokenizer::Tokenizer};\nuse crate::schema::{\n    Attribute, AttributeValue, Element, NamedElement, Namespace,\n    property::{\n        CalDavProperty, CalDavPropertyName, CalendarData, CardDavProperty, CardDavPropertyName,\n        Comp, DavProperty, DavValue, PrincipalProperty, ResourceType, WebDavProperty,\n    },\n    request::{DavPropertyValue, VCardPropertyWithGroup},\n    response::List,\n};\nuse calcard::{\n    Entry, Parser,\n    common::{IanaParse, PartialDateTime},\n    icalendar::{ICalendar, ICalendarComponentType, ICalendarParameterName, ICalendarProperty},\n    vcard::{VCardParameterName, VCardProperty},\n};\nuse mail_parser::DateTime;\nuse types::{TimeRange, dead_property::DeadProperty};\n\nimpl Tokenizer<'_> {\n    pub(crate) fn collect_properties(\n        &mut self,\n        mut elements: Vec<DavProperty>,\n    ) -> crate::parser::Result<Vec<DavProperty>> {\n        loop {\n            match self.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::CalendarData,\n                        },\n                    ..\n                } => {\n                    elements.push(DavProperty::CalDav(CalDavProperty::CalendarData(\n                        self.collect_calendar_data()?,\n                    )));\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CardDav,\n                            element: Element::AddressData,\n                        },\n                    ..\n                } => {\n                    elements.push(DavProperty::CardDav(CardDavProperty::AddressData(\n                        self.collect_address_data()?,\n                    )));\n                }\n                Token::ElementStart { name, .. } => {\n                    if let Some(property) = DavProperty::from_element(name) {\n                        elements.push(property);\n                    }\n                    self.expect_element_end()?;\n                }\n                Token::ElementEnd => {\n                    break;\n                }\n                Token::UnknownElement(name) => {\n                    elements.push(DavProperty::DeadProperty((&name).into()));\n                    self.expect_element_end()?;\n                }\n                token => return Err(token.into_unexpected()),\n            }\n        }\n\n        Ok(elements)\n    }\n\n    pub(crate) fn collect_calendar_data(&mut self) -> crate::parser::Result<CalendarData> {\n        let mut depth = 1;\n        let mut data = CalendarData {\n            properties: Vec::with_capacity(4),\n            expand: None,\n            limit_recurrence: None,\n            limit_freebusy: None,\n        };\n        let mut components: Vec<ICalendarComponentType> = Vec::new();\n\n        loop {\n            match self.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::Allcomp,\n                        },\n                    ..\n                } => {\n                    self.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::Allprop,\n                        },\n                    ..\n                } => {\n                    if let Some(component) = components.last().cloned() {\n                        data.properties.push(CalDavPropertyName {\n                            component: Some(component),\n                            name: None,\n                            no_value: false,\n                        });\n                    }\n                    self.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::Comp,\n                        },\n                    raw,\n                } => {\n                    depth += 1;\n\n                    for attribute in raw.attributes::<ICalendarComponentType>() {\n                        if let Attribute::Name(name) = attribute? {\n                            components.push(name);\n                        }\n                    }\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::Prop,\n                        },\n                    raw,\n                } => {\n                    let mut name = None;\n                    let mut no_value = false;\n\n                    for attribute in raw.attributes::<ICalendarProperty>() {\n                        match attribute? {\n                            Attribute::Name(name_) => {\n                                name = Some(name_);\n                            }\n                            Attribute::NoValue(no_value_) => {\n                                no_value = no_value_;\n                            }\n                            _ => {}\n                        }\n                    }\n\n                    if let Some(name) = name {\n                        data.properties.push(CalDavPropertyName {\n                            component: components.last().cloned(),\n                            name: Some(name),\n                            no_value,\n                        });\n                    }\n\n                    self.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::Expand,\n                        },\n                    raw,\n                } => {\n                    data.expand = TimeRange::from_raw(&raw)?;\n                    self.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::LimitRecurrenceSet,\n                        },\n                    raw,\n                } => {\n                    data.limit_recurrence = TimeRange::from_raw(&raw)?;\n                    self.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CalDav,\n                            element: Element::LimitFreebusySet,\n                        },\n                    raw,\n                } => {\n                    data.limit_freebusy = TimeRange::from_raw(&raw)?;\n                    self.expect_element_end()?;\n                }\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                    if let Some(last_component) = components.pop()\n                        && last_component != ICalendarComponentType::VCalendar\n                        && !matches!(data.properties.last(), Some(CalDavPropertyName { component: Some(component), .. }) if component == &last_component)\n                    {\n                        data.properties.push(CalDavPropertyName {\n                            component: Some(last_component),\n                            name: None,\n                            no_value: false,\n                        });\n                    }\n                }\n                Token::Eof => {\n                    break;\n                }\n                token => return Err(token.into_unexpected()),\n            }\n        }\n\n        Ok(data)\n    }\n\n    pub(crate) fn collect_address_data(\n        &mut self,\n    ) -> crate::parser::Result<Vec<CardDavPropertyName>> {\n        let mut items = Vec::with_capacity(4);\n        loop {\n            match self.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CardDav,\n                            element: Element::Allprop,\n                        },\n                    ..\n                } => {\n                    self.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::CardDav,\n                            element: Element::Prop,\n                        },\n                    raw,\n                } => {\n                    let mut name = None;\n                    let mut group = None;\n                    let mut no_value = false;\n\n                    for attribute in raw.attributes::<VCardPropertyWithGroup>() {\n                        match attribute? {\n                            Attribute::Name(name_) => {\n                                name = Some(name_.name);\n                                group = name_.group;\n                            }\n                            Attribute::NoValue(no_value_) => {\n                                no_value = no_value_;\n                            }\n                            _ => {}\n                        }\n                    }\n\n                    if let Some(name) = name {\n                        items.push(CardDavPropertyName {\n                            name,\n                            group,\n                            no_value,\n                        });\n                    }\n\n                    self.expect_element_end()?;\n                }\n                Token::ElementEnd | Token::Eof => {\n                    break;\n                }\n                token => return Err(token.into_unexpected()),\n            }\n        }\n\n        Ok(items)\n    }\n}\n\nimpl Tokenizer<'_> {\n    pub(crate) fn collect_property_values(\n        &mut self,\n        elements: &mut Vec<DavPropertyValue>,\n    ) -> crate::parser::Result<()> {\n        loop {\n            match self.token()? {\n                Token::ElementStart { name, .. } => {\n                    if let Some(property) = DavProperty::from_element(name) {\n                        let value = match property {\n                            DavProperty::WebDav(WebDavProperty::ResourceType) => {\n                                DavValue::ResourceTypes(List(self.collect_elements()?))\n                            }\n                            DavProperty::WebDav(WebDavProperty::CreationDate) => {\n                                match self.parse_value::<DateTime>()? {\n                                    Some(Ok(value)) => DavValue::Timestamp(value.to_timestamp()),\n                                    Some(Err(value)) => DavValue::String(value),\n                                    None => DavValue::Null,\n                                }\n                            }\n                            DavProperty::CalDav(CalDavProperty::CalendarTimezone) => {\n                                match self\n                                    .collect_string_value()?\n                                    .map(|v| ICalendar::parse(&v).map_err(|_| v))\n                                {\n                                    Some(Ok(value)) => DavValue::ICalendar(value),\n                                    Some(Err(value)) => DavValue::String(value),\n                                    None => DavValue::Null,\n                                }\n                            }\n                            DavProperty::CalDav(CalDavProperty::SupportedCalendarComponentSet) => {\n                                let mut components = Vec::new();\n\n                                loop {\n                                    match self.token()? {\n                                        Token::ElementStart { name, raw } => {\n                                            if name.ns == Namespace::CalDav\n                                                && name.element == Element::Comp\n                                            {\n                                                for component in\n                                                    raw.attributes::<ICalendarComponentType>()\n                                                {\n                                                    if let Attribute::Name(name) = component? {\n                                                        components.push(Comp(name));\n                                                    }\n                                                }\n                                            }\n                                            self.seek_element_end()?;\n                                        }\n                                        Token::UnknownElement(_) => {\n                                            // Ignore unknown elements\n                                            self.seek_element_end()?;\n                                        }\n                                        Token::ElementEnd | Token::Eof => {\n                                            break;\n                                        }\n                                        _ => {}\n                                    }\n                                }\n\n                                DavValue::Components(List(components))\n                            }\n                            DavProperty::CalDav(\n                                CalDavProperty::MaxInstances\n                                | CalDavProperty::MaxAttendeesPerInstance,\n                            ) => match self.parse_value()? {\n                                Some(Ok(value)) => DavValue::Uint64(value),\n                                Some(Err(value)) => DavValue::String(value),\n                                None => DavValue::Null,\n                            },\n                            _ => self\n                                .collect_string_value()?\n                                .map(DavValue::String)\n                                .unwrap_or(DavValue::Null),\n                        };\n\n                        elements.push(DavPropertyValue { property, value });\n                    } else {\n                        // Ignore unknown elements\n                        self.seek_element_end()?;\n                    }\n                }\n                Token::ElementEnd | Token::Eof => {\n                    break;\n                }\n                Token::UnknownElement(raw) => {\n                    elements.push(DavPropertyValue {\n                        property: DavProperty::DeadProperty((&raw).into()),\n                        value: DavValue::DeadProperty(DeadProperty::parse(self)?),\n                    });\n                }\n                token => return Err(token.into_unexpected()),\n            }\n        }\n\n        Ok(())\n    }\n}\n\npub(crate) trait TimeRangeFromRaw {\n    fn from_raw(raw: &RawElement<'_>) -> super::Result<Option<TimeRange>>;\n}\n\nimpl TimeRangeFromRaw for TimeRange {\n    fn from_raw(raw: &RawElement<'_>) -> super::Result<Option<Self>> {\n        let mut range = TimeRange {\n            start: i64::MIN,\n            end: i64::MAX,\n        };\n\n        for attribute in raw.attributes::<ICalendarDateTime>() {\n            match attribute? {\n                Attribute::Start(start) => {\n                    range.start = start.0;\n                }\n                Attribute::End(end) => {\n                    range.end = end.0;\n                }\n                _ => {}\n            }\n        }\n\n        if range.end < range.start {\n            range.end = i64::MAX;\n        }\n\n        if range.start != i64::MIN || range.end != i64::MAX {\n            Ok(Some(range))\n        } else {\n            Ok(None)\n        }\n    }\n}\n\nimpl DavProperty {\n    pub(crate) fn from_element(element: NamedElement) -> Option<Self> {\n        match (element.ns, element.element) {\n            (Namespace::Dav, Element::Creationdate) => {\n                Some(DavProperty::WebDav(WebDavProperty::CreationDate))\n            }\n            (Namespace::Dav, Element::Displayname) => {\n                Some(DavProperty::WebDav(WebDavProperty::DisplayName))\n            }\n            (Namespace::Dav, Element::Getcontentlanguage) => {\n                Some(DavProperty::WebDav(WebDavProperty::GetContentLanguage))\n            }\n            (Namespace::Dav, Element::Getcontentlength) => {\n                Some(DavProperty::WebDav(WebDavProperty::GetContentLength))\n            }\n            (Namespace::Dav, Element::Getcontenttype) => {\n                Some(DavProperty::WebDav(WebDavProperty::GetContentType))\n            }\n            (Namespace::Dav, Element::Getetag) => {\n                Some(DavProperty::WebDav(WebDavProperty::GetETag))\n            }\n            (Namespace::Dav, Element::Getlastmodified) => {\n                Some(DavProperty::WebDav(WebDavProperty::GetLastModified))\n            }\n            (Namespace::Dav, Element::Resourcetype) => {\n                Some(DavProperty::WebDav(WebDavProperty::ResourceType))\n            }\n            (Namespace::Dav, Element::Lockdiscovery) => {\n                Some(DavProperty::WebDav(WebDavProperty::LockDiscovery))\n            }\n            (Namespace::Dav, Element::Supportedlock) => {\n                Some(DavProperty::WebDav(WebDavProperty::SupportedLock))\n            }\n            (Namespace::Dav, Element::CurrentUserPrincipal) => {\n                Some(DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal))\n            }\n            (Namespace::Dav, Element::QuotaAvailableBytes) => {\n                Some(DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes))\n            }\n            (Namespace::Dav, Element::QuotaUsedBytes) => {\n                Some(DavProperty::WebDav(WebDavProperty::QuotaUsedBytes))\n            }\n            (Namespace::Dav, Element::SupportedReportSet) => {\n                Some(DavProperty::WebDav(WebDavProperty::SupportedReportSet))\n            }\n            (Namespace::Dav, Element::SyncToken) => {\n                Some(DavProperty::WebDav(WebDavProperty::SyncToken))\n            }\n            (Namespace::Dav, Element::AlternateUriSet) => {\n                Some(DavProperty::Principal(PrincipalProperty::AlternateURISet))\n            }\n            (Namespace::Dav, Element::PrincipalUrl) => {\n                Some(DavProperty::Principal(PrincipalProperty::PrincipalURL))\n            }\n            (Namespace::Dav, Element::GroupMemberSet) => {\n                Some(DavProperty::Principal(PrincipalProperty::GroupMemberSet))\n            }\n            (Namespace::Dav, Element::GroupMembership) => {\n                Some(DavProperty::Principal(PrincipalProperty::GroupMembership))\n            }\n            (Namespace::Dav, Element::Owner) => Some(DavProperty::WebDav(WebDavProperty::Owner)),\n            (Namespace::Dav, Element::Group) => Some(DavProperty::WebDav(WebDavProperty::Group)),\n            (Namespace::Dav, Element::SupportedPrivilegeSet) => {\n                Some(DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet))\n            }\n            (Namespace::Dav, Element::CurrentUserPrivilegeSet) => {\n                Some(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n            }\n            (Namespace::Dav, Element::Acl) => Some(DavProperty::WebDav(WebDavProperty::Acl)),\n            (Namespace::Dav, Element::AclRestrictions) => {\n                Some(DavProperty::WebDav(WebDavProperty::AclRestrictions))\n            }\n            (Namespace::Dav, Element::InheritedAclSet) => {\n                Some(DavProperty::WebDav(WebDavProperty::InheritedAclSet))\n            }\n            (Namespace::Dav, Element::PrincipalCollectionSet) => {\n                Some(DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet))\n            }\n            (Namespace::CardDav, Element::AddressbookDescription) => Some(DavProperty::CardDav(\n                CardDavProperty::AddressbookDescription,\n            )),\n            (Namespace::CardDav, Element::SupportedAddressData) => {\n                Some(DavProperty::CardDav(CardDavProperty::SupportedAddressData))\n            }\n            (Namespace::CardDav, Element::SupportedCollationSet) => {\n                Some(DavProperty::CardDav(CardDavProperty::SupportedCollationSet))\n            }\n            (Namespace::CardDav, Element::AddressbookHomeSet) => Some(DavProperty::Principal(\n                PrincipalProperty::AddressbookHomeSet,\n            )),\n            (Namespace::CardDav, Element::PrincipalAddress) => {\n                Some(DavProperty::Principal(PrincipalProperty::PrincipalAddress))\n            }\n            (Namespace::CardDav, Element::AddressData) => Some(DavProperty::CardDav(\n                CardDavProperty::AddressData(Default::default()),\n            )),\n            (Namespace::CardDav, Element::MaxResourceSize) => {\n                Some(DavProperty::CardDav(CardDavProperty::MaxResourceSize))\n            }\n            (Namespace::CalDav, Element::CalendarDescription) => {\n                Some(DavProperty::CalDav(CalDavProperty::CalendarDescription))\n            }\n            (Namespace::CalDav, Element::CalendarTimezone) => {\n                Some(DavProperty::CalDav(CalDavProperty::CalendarTimezone))\n            }\n            (Namespace::CalDav, Element::SupportedCalendarComponentSet) => Some(\n                DavProperty::CalDav(CalDavProperty::SupportedCalendarComponentSet),\n            ),\n            (Namespace::CalDav, Element::SupportedCollationSet) => {\n                Some(DavProperty::CalDav(CalDavProperty::SupportedCollationSet))\n            }\n            (Namespace::CalDav, Element::SupportedCalendarData) => {\n                Some(DavProperty::CalDav(CalDavProperty::SupportedCalendarData))\n            }\n            (Namespace::CalDav, Element::MaxResourceSize) => {\n                Some(DavProperty::CalDav(CalDavProperty::MaxResourceSize))\n            }\n            (Namespace::CalDav, Element::MinDateTime) => {\n                Some(DavProperty::CalDav(CalDavProperty::MinDateTime))\n            }\n            (Namespace::CalDav, Element::MaxDateTime) => {\n                Some(DavProperty::CalDav(CalDavProperty::MaxDateTime))\n            }\n            (Namespace::CalDav, Element::MaxInstances) => {\n                Some(DavProperty::CalDav(CalDavProperty::MaxInstances))\n            }\n            (Namespace::CalDav, Element::MaxAttendeesPerInstance) => {\n                Some(DavProperty::CalDav(CalDavProperty::MaxAttendeesPerInstance))\n            }\n            (Namespace::CalDav, Element::ScheduleDefaultCalendarUrl) => Some(DavProperty::CalDav(\n                CalDavProperty::ScheduleDefaultCalendarURL,\n            )),\n            (Namespace::CalDav, Element::ScheduleTag) => {\n                Some(DavProperty::CalDav(CalDavProperty::ScheduleTag))\n            }\n            (Namespace::CalDav, Element::ScheduleCalendarTransp) => {\n                Some(DavProperty::CalDav(CalDavProperty::ScheduleCalendarTransp))\n            }\n            (Namespace::CalDav, Element::CalendarHomeSet) => {\n                Some(DavProperty::Principal(PrincipalProperty::CalendarHomeSet))\n            }\n            (Namespace::CalDav, Element::CalendarUserAddressSet) => Some(DavProperty::Principal(\n                PrincipalProperty::CalendarUserAddressSet,\n            )),\n            (Namespace::CalDav, Element::CalendarUserType) => {\n                Some(DavProperty::Principal(PrincipalProperty::CalendarUserType))\n            }\n            (Namespace::CalDav, Element::ScheduleInboxUrl) => {\n                Some(DavProperty::Principal(PrincipalProperty::ScheduleInboxURL))\n            }\n            (Namespace::CalDav, Element::ScheduleOutboxUrl) => {\n                Some(DavProperty::Principal(PrincipalProperty::ScheduleOutboxURL))\n            }\n            (Namespace::CalDav, Element::CalendarData) => Some(DavProperty::CalDav(\n                CalDavProperty::CalendarData(Default::default()),\n            )),\n            (Namespace::CalDav, Element::TimezoneServiceSet) => {\n                Some(DavProperty::CalDav(CalDavProperty::TimezoneServiceSet))\n            }\n            (Namespace::CalDav, Element::CalendarTimezoneId) => {\n                Some(DavProperty::CalDav(CalDavProperty::TimezoneId))\n            }\n            (Namespace::CalendarServer, Element::Getctag) => {\n                Some(DavProperty::WebDav(WebDavProperty::GetCTag))\n            }\n            _ => None,\n        }\n    }\n}\n\nimpl TryFrom<NamedElement> for ResourceType {\n    type Error = ();\n\n    fn try_from(value: NamedElement) -> Result<Self, Self::Error> {\n        match (value.ns, value.element) {\n            (Namespace::Dav, Element::Collection) => Ok(ResourceType::Collection),\n            (Namespace::Dav, Element::Principal) => Ok(ResourceType::Principal),\n            (Namespace::CardDav, Element::Addressbook) => Ok(ResourceType::AddressBook),\n            (Namespace::CalDav, Element::Calendar) => Ok(ResourceType::Calendar),\n            (Namespace::CalDav, Element::ScheduleInbox) => Ok(ResourceType::ScheduleInbox),\n            (Namespace::CalDav, Element::ScheduleOutbox) => Ok(ResourceType::ScheduleOutbox),\n            _ => Err(()),\n        }\n    }\n}\n\nstruct ICalendarDateTime(i64);\n\nimpl AttributeValue for ICalendarDateTime {\n    fn from_str(s: &str) -> Option<Self>\n    where\n        Self: Sized,\n    {\n        let mut dt = PartialDateTime::default();\n        dt.parse_timestamp(&mut s.as_bytes().iter().peekable(), true);\n        dt.to_timestamp().map(ICalendarDateTime)\n    }\n}\n\nimpl AttributeValue for ICalendarComponentType {\n    fn from_str(s: &str) -> Option<Self>\n    where\n        Self: Sized,\n    {\n        ICalendarComponentType::parse(s.as_bytes())\n    }\n}\n\nimpl AttributeValue for ICalendarProperty {\n    fn from_str(s: &str) -> Option<Self>\n    where\n        Self: Sized,\n    {\n        ICalendarProperty::parse(s.as_bytes())\n            .unwrap_or_else(|| ICalendarProperty::Other(s.to_string()))\n            .into()\n    }\n}\n\nimpl AttributeValue for ICalendarParameterName {\n    fn from_str(s: &str) -> Option<Self>\n    where\n        Self: Sized,\n    {\n        ICalendarParameterName::parse(s).into()\n    }\n}\n\nimpl AttributeValue for VCardPropertyWithGroup {\n    fn from_str(s: &str) -> Option<Self>\n    where\n        Self: Sized,\n    {\n        if let Some((group, s)) = s.split_once('.') {\n            VCardPropertyWithGroup {\n                name: VCardProperty::parse(s.as_bytes())\n                    .unwrap_or_else(|| VCardProperty::Other(s.to_string())),\n                group: group.to_string().into(),\n            }\n            .into()\n        } else {\n            VCardPropertyWithGroup {\n                name: VCardProperty::parse(s.as_bytes())\n                    .unwrap_or_else(|| VCardProperty::Other(s.to_string())),\n                group: None,\n            }\n            .into()\n        }\n    }\n}\n\nimpl AttributeValue for VCardParameterName {\n    fn from_str(s: &str) -> Option<Self>\n    where\n        Self: Sized,\n    {\n        VCardParameterName::parse(s).into()\n    }\n}\n\nimpl XmlValueParser for ICalendar {\n    fn parse_bytes(bytes: &[u8]) -> Option<Self> {\n        let text = String::from_utf8_lossy(bytes);\n        let mut parser = Parser::new(&text);\n        if let Entry::ICalendar(ical) = parser.entry() {\n            Some(ical)\n        } else {\n            None\n        }\n    }\n\n    fn parse_str(text: &str) -> Option<Self> {\n        let mut parser = Parser::new(text);\n        if let Entry::ICalendar(ical) = parser.entry() {\n            Some(ical)\n        } else {\n            None\n        }\n    }\n}\n\nimpl XmlValueParser for u64 {\n    fn parse_bytes(bytes: &[u8]) -> Option<Self> {\n        std::str::from_utf8(bytes).ok().and_then(|s| s.parse().ok())\n    }\n\n    fn parse_str(text: &str) -> Option<Self> {\n        text.parse().ok()\n    }\n}\n\nimpl XmlValueParser for u32 {\n    fn parse_bytes(bytes: &[u8]) -> Option<Self> {\n        std::str::from_utf8(bytes).ok().and_then(|s| s.parse().ok())\n    }\n\n    fn parse_str(text: &str) -> Option<Self> {\n        text.parse().ok()\n    }\n}\n\nimpl XmlValueParser for DateTime {\n    fn parse_bytes(bytes: &[u8]) -> Option<Self> {\n        std::str::from_utf8(bytes)\n            .ok()\n            .and_then(DateTime::parse_rfc3339)\n    }\n\n    fn parse_str(text: &str) -> Option<Self> {\n        DateTime::parse_rfc3339(text)\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/parser/tokenizer.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Error, RawElement, Token, UnexpectedToken, XmlValueParser};\nuse crate::schema::{Attribute, AttributeValue, Element, NamedElement, Namespace};\nuse quick_xml::{\n    NsReader,\n    events::{Event, attributes::AttrError},\n    name::ResolveResult,\n};\n\npub struct Tokenizer<'x> {\n    xml: NsReader<&'x [u8]>,\n    last_is_end: bool,\n}\n\nimpl<'x> Tokenizer<'x> {\n    pub fn new(input: &'x [u8]) -> Self {\n        let mut xml = NsReader::from_reader(input);\n        xml.config_mut();\n        Self {\n            xml,\n            last_is_end: false,\n        }\n    }\n\n    pub fn token(&'_ mut self) -> super::Result<Token<'_>> {\n        loop {\n            if self.last_is_end {\n                self.last_is_end = false;\n                return Ok(Token::ElementEnd);\n            }\n\n            let (resolve_result, event) = self.xml.read_resolved_event()?;\n            let tag = match event {\n                Event::Start(tag) => tag,\n                Event::Empty(tag) => {\n                    self.last_is_end = true;\n                    tag\n                }\n                Event::End(_) => {\n                    return Ok(Token::ElementEnd);\n                }\n                Event::Text(text) if text.iter().any(|ch| !ch.is_ascii_whitespace()) => {\n                    return text\n                        .xml_content()\n                        .map(Token::Text)\n                        .map_err(|err| Error::Xml(Box::new(err.into())));\n                }\n                Event::GeneralRef(entity) => {\n                    hashify::fnc_map!(entity.as_ref(),\n                        b\"lt\" => { return Ok(Token::Text(\"<\".into())); },\n                        b\"gt\" => { return Ok(Token::Text(\">\".into())); },\n                        b\"amp\" => { return Ok(Token::Text(\"&\".into())); },\n                        b\"apos\" => { return Ok(Token::Text(\"'\".into())); },\n                        b\"quot\" => { return Ok(Token::Text(\"\\\"\".into())); },\n                        _ => {\n                            if let Ok(Some(gr)) = entity.resolve_char_ref() {\n                                return Ok(Token::Text(gr.to_string().into()));\n                            }\n                        }\n                    );\n\n                    return entity\n                        .xml_content()\n                        .map(Token::Text)\n                        .map_err(|err| Error::Xml(Box::new(err.into())));\n                }\n                Event::CData(bytes) => return Ok(Token::Bytes(bytes.into_inner())),\n                Event::Eof => return Ok(Token::Eof),\n                _ => {\n                    continue;\n                }\n            };\n\n            // Parse element\n            let name = tag.name();\n            match resolve_result {\n                ResolveResult::Bound(raw_ns) if !raw_ns.as_ref().is_empty() => {\n                    if let (Some(ns), Some(element)) = (\n                        Namespace::try_parse(raw_ns.as_ref()),\n                        Element::try_parse(name.local_name().as_ref()).copied(),\n                    ) {\n                        return Ok(Token::ElementStart {\n                            name: NamedElement { ns, element },\n                            raw: RawElement::new(tag)\n                                .with_namespace_static(ns.namespace().as_bytes()),\n                        });\n                    } else {\n                        return Ok(Token::UnknownElement(\n                            RawElement::new(tag).with_namespace(raw_ns),\n                        ));\n                    }\n                }\n                ResolveResult::Unknown(p) => {\n                    return Err(Error::Xml(Box::new(quick_xml::Error::Namespace(\n                        quick_xml::name::NamespaceError::UnknownPrefix(p),\n                    ))));\n                }\n                _ => {\n                    return Ok(Token::UnknownElement(RawElement::new(tag)));\n                }\n            }\n        }\n    }\n\n    pub fn unwrap_named_element(&mut self) -> super::Result<NamedElement> {\n        match self.token()? {\n            Token::ElementStart { name, .. } => Ok(name),\n            found => Err(Error::UnexpectedToken(Box::new(UnexpectedToken {\n                expected: None,\n                found: found.into_owned(),\n            }))),\n        }\n    }\n\n    pub fn expect_named_element(&mut self, expected: NamedElement) -> super::Result<()> {\n        match self.token()? {\n            Token::ElementStart { name, .. } if name == expected => Ok(()),\n            found => Err(Error::UnexpectedToken(Box::new(UnexpectedToken {\n                expected: Token::ElementStart {\n                    name: expected,\n                    raw: RawElement::default(),\n                }\n                .into(),\n                found: found.into_owned(),\n            }))),\n        }\n    }\n\n    pub fn expect_named_element_or_eof(&mut self, expected: NamedElement) -> super::Result<bool> {\n        match self.token()? {\n            Token::ElementStart { name, .. } if name == expected => Ok(true),\n            Token::Eof => Ok(false),\n            found => Err(Error::UnexpectedToken(Box::new(UnexpectedToken {\n                expected: Token::ElementStart {\n                    name: expected,\n                    raw: RawElement::default(),\n                }\n                .into(),\n                found: found.into_owned(),\n            }))),\n        }\n    }\n\n    pub fn expect_element_end(&mut self) -> super::Result<()> {\n        match self.token()? {\n            Token::ElementEnd => Ok(()),\n            found => Err(Error::UnexpectedToken(Box::new(UnexpectedToken {\n                expected: Token::ElementEnd.into(),\n                found: found.into_owned(),\n            }))),\n        }\n    }\n\n    pub fn seek_element_end(&mut self) -> super::Result<()> {\n        let mut depth = 1;\n        loop {\n            match self.token()? {\n                Token::ElementStart { .. } | Token::UnknownElement(_) => depth += 1,\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        return Ok(());\n                    }\n                }\n                Token::Eof => return Err(Token::Eof.into_unexpected()),\n                _ => {}\n            }\n        }\n    }\n\n    pub fn collect_string_value(&mut self) -> super::Result<Option<String>> {\n        let mut depth = 1;\n        let mut value: Option<String> = None;\n\n        loop {\n            match self.token()? {\n                Token::ElementStart { .. } | Token::UnknownElement(_) => depth += 1,\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                }\n                Token::Text(text) => {\n                    if let Some(ref mut v) = value {\n                        v.push_str(&text);\n                    } else {\n                        value = Some(text.into_owned());\n                    }\n                }\n                Token::Bytes(bytes) => {\n                    if let Some(ref mut v) = value {\n                        v.push_str(&String::from_utf8_lossy(&bytes));\n                    } else {\n                        value = Some(String::from_utf8_lossy(&bytes).into_owned());\n                    }\n                }\n                Token::Eof => return Err(Token::Eof.into_unexpected()),\n            }\n        }\n\n        Ok(value)\n    }\n\n    pub fn parse_value<T: XmlValueParser>(&mut self) -> super::Result<Option<Result<T, String>>> {\n        let mut depth = 1;\n        let mut result: Option<Result<T, String>> = None;\n\n        loop {\n            match self.token()? {\n                Token::ElementStart { .. } | Token::UnknownElement(_) => depth += 1,\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                }\n                Token::Text(text) => {\n                    if let Some(value) = T::parse_str(&text) {\n                        result = Some(Ok(value));\n                    } else {\n                        result = Some(Err(text.into_owned()));\n                    }\n                }\n                Token::Bytes(bytes) => {\n                    if let Some(value) = T::parse_bytes(&bytes) {\n                        result = Some(Ok(value));\n                    } else {\n                        result = Some(Err(String::from_utf8_lossy(&bytes).into_owned()));\n                    }\n                }\n                Token::Eof => return Err(Token::Eof.into_unexpected()),\n            }\n        }\n\n        Ok(result)\n    }\n\n    pub fn collect_elements<T>(&mut self) -> super::Result<Vec<T>>\n    where\n        T: TryFrom<NamedElement>,\n    {\n        let mut elements = Vec::with_capacity(2);\n        let mut depth = 1;\n\n        loop {\n            match self.token()? {\n                Token::ElementStart { name, .. } => {\n                    if depth == 1\n                        && let Ok(element) = T::try_from(name)\n                    {\n                        elements.push(element);\n                    }\n\n                    depth += 1;\n                }\n                Token::UnknownElement(_) => {\n                    depth += 1;\n                }\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                }\n                Token::Eof => break,\n                _ => {}\n            }\n        }\n        Ok(elements)\n    }\n}\n\nimpl RawElement<'_> {\n    pub fn attributes<T: AttributeValue>(\n        &self,\n    ) -> impl Iterator<Item = super::Result<Attribute<T>>> + '_ {\n        self.element.attributes().filter_map(|attr| match attr {\n            Ok(attr) => match attr.unescape_value() {\n                Ok(value) => Attribute::from_param(attr.key.as_ref(), value).map(Ok),\n                Err(err) => Some(Err(err.into())),\n            },\n            Err(err) => Some(Err(err.into())),\n        })\n    }\n}\n\nimpl From<quick_xml::Error> for Error {\n    fn from(err: quick_xml::Error) -> Self {\n        Error::Xml(Box::new(err))\n    }\n}\n\nimpl From<AttrError> for Error {\n    fn from(err: AttrError) -> Self {\n        Error::Xml(Box::new(err.into()))\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use std::borrow::Cow;\n\n    use crate::schema::{Collation, MatchType};\n\n    use super::*;\n\n    #[derive(Debug, PartialEq, Eq)]\n    pub enum TestToken<'x> {\n        ElementStart(NamedElement),\n        ElementEnd,\n        Attribute(Attribute<String>),\n        Bytes(Cow<'x, [u8]>),\n        Text(Cow<'x, str>),\n    }\n\n    #[test]\n    fn test_tokenizer() {\n        for (input, expected) in [\n            (\n                r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                    xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n    <D:prop>\n      <D:getetag/>\n      <C:calendar-data/>\n    </D:prop>\n    <C:filter>\n      <C:comp-filter name=\"VCALENDAR\"/>\n    </C:filter>\n   </C:calendar-query>\"#,\n                vec![\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::CalendarQuery,\n                    }),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Prop,\n                    }),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Getetag,\n                    }),\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::CalendarData,\n                    }),\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::Filter,\n                    }),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::CompFilter,\n                    }),\n                    TestToken::Attribute(Attribute::Name(\"VCALENDAR\".to_string())),\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                ],\n            ),\n            (\n                r#\" <?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data>\n         <C:prop name=\"VERSION\"/>\n         <C:prop name=\"UID\"/>\n         <C:prop name=\"NICKNAME\"/>\n         <C:prop name=\"EMAIL\"/>\n         <C:prop name=\"FN\"/>\n       </C:address-data>\n     </D:prop>\n     <C:filter>\n       <C:prop-filter name=\"NICKNAME\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"equals\"\n         >me</C:text-match>\n       </C:prop-filter>\n     </C:filter>\n   </C:addressbook-query>\"#,\n                vec![\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::AddressbookQuery,\n                    }),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Prop,\n                    }),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Getetag,\n                    }),\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::AddressData,\n                    }),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Prop,\n                    }),\n                    TestToken::Attribute(Attribute::Name(\"VERSION\".to_string())),\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Prop,\n                    }),\n                    TestToken::Attribute(Attribute::Name(\"UID\".to_string())),\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Prop,\n                    }),\n                    TestToken::Attribute(Attribute::Name(\"NICKNAME\".to_string())),\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Prop,\n                    }),\n                    TestToken::Attribute(Attribute::Name(\"EMAIL\".to_string())),\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Prop,\n                    }),\n                    TestToken::Attribute(Attribute::Name(\"FN\".to_string())),\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Filter,\n                    }),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::PropFilter,\n                    }),\n                    TestToken::Attribute(Attribute::Name(\"NICKNAME\".to_string())),\n                    TestToken::ElementStart(NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::TextMatch,\n                    }),\n                    TestToken::Attribute(Attribute::Collation(Collation::UnicodeCasemap)),\n                    TestToken::Attribute(Attribute::MatchType(MatchType::Equals)),\n                    TestToken::Text(\"me\".into()),\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                    TestToken::ElementEnd,\n                ],\n            ),\n        ] {\n            let mut tokenizer = Tokenizer::new(input.as_bytes());\n            let mut result = vec![];\n\n            loop {\n                match tokenizer.token() {\n                    Ok(token) => match token {\n                        Token::ElementStart { name, raw } => {\n                            result.push(TestToken::ElementStart(name));\n                            for attr in raw.attributes::<String>() {\n                                result.push(TestToken::Attribute(attr.unwrap()));\n                            }\n                        }\n                        Token::ElementEnd => {\n                            result.push(TestToken::ElementEnd);\n                        }\n                        Token::Bytes(cow) => {\n                            result.push(TestToken::Bytes(cow.into_owned().into()));\n                        }\n                        Token::Text(cow) => {\n                            result.push(TestToken::Text(cow.into_owned().into()));\n                        }\n                        Token::UnknownElement(_) => {\n                            //result.push(TestToken::UnknownElement(unknown_element));\n                        }\n                        Token::Eof => break,\n                    },\n                    Err(err) => {\n                        panic!(\"Error: {:?}\", err);\n                    }\n                }\n            }\n\n            assert_eq!(result, expected);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/requests/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    parser::{tokenizer::Tokenizer, DavParser, Token},\n    schema::{\n        property::{DavValue, Privilege},\n        request::{\n            Acl, AclPrincipalPropSet, DavPropertyValue, PrincipalMatch, PrincipalMatchProperties,\n            PrincipalPropertySearch, PropertySearch,\n        },\n        response::{Ace, GrantDeny, Href, List, Principal},\n        Element, NamedElement, Namespace,\n    },\n};\n\nimpl DavParser for Acl {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        stream.expect_named_element(NamedElement::dav(Element::Acl))?;\n\n        let mut acl = Acl { aces: vec![] };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Ace,\n                        },\n                    ..\n                } => {\n                    acl.aces.push(Ace::parse(stream)?);\n                }\n                Token::ElementEnd => {\n                    break;\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                other => {\n                    return Err(other.into_unexpected());\n                }\n            }\n        }\n\n        Ok(acl)\n    }\n}\n\nimpl DavParser for Ace {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut ace = Ace {\n            principal: Principal::All,\n            invert: false,\n            grant_deny: GrantDeny::Grant(List(vec![])),\n            protected: false,\n            inherited: None,\n        };\n        let mut depth = 1;\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Principal,\n                        },\n                    ..\n                } => {\n                    ace.principal = Principal::parse(stream)?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Invert,\n                        },\n                    ..\n                } if depth == 1 => {\n                    ace.invert = true;\n                    depth += 1;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Protected,\n                        },\n                    ..\n                } if depth == 1 => {\n                    ace.protected = true;\n                    stream.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Inherited,\n                        },\n                    ..\n                } if depth == 1 => {\n                    stream.expect_named_element(NamedElement::dav(Element::Href))?;\n                    ace.inherited = stream.collect_string_value()?.map(Href);\n                    stream.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Grant,\n                        },\n                    ..\n                } if depth == 1 => {\n                    ace.grant_deny = GrantDeny::Grant(List(stream.collect_privileges()?));\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Deny,\n                        },\n                    ..\n                } if depth == 1 => {\n                    ace.grant_deny = GrantDeny::Deny(List(stream.collect_privileges()?));\n                }\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                other => {\n                    return Err(other.into_unexpected());\n                }\n            }\n        }\n\n        Ok(ace)\n    }\n}\n\nimpl DavParser for Principal {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let result = match stream.unwrap_named_element()? {\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Href,\n            } => Principal::Href(Href(stream.collect_string_value()?.unwrap_or_default())),\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::All,\n            } => {\n                stream.expect_element_end()?;\n                Principal::All\n            }\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Authenticated,\n            } => {\n                stream.expect_element_end()?;\n                Principal::Authenticated\n            }\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Unauthenticated,\n            } => {\n                stream.expect_element_end()?;\n                Principal::Unauthenticated\n            }\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Property,\n            } => {\n                let property = stream.collect_properties(Vec::new())?;\n                Principal::Property(List(\n                    property\n                        .into_iter()\n                        .map(|prop| DavPropertyValue::new(prop, DavValue::Null))\n                        .collect(),\n                ))\n            }\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Self_,\n            } => {\n                stream.expect_element_end()?;\n                Principal::Self_\n            }\n            other => return Err(other.into_unexpected()),\n        };\n        stream.expect_element_end()?;\n        Ok(result)\n    }\n}\n\nimpl Tokenizer<'_> {\n    pub fn collect_privileges(&mut self) -> crate::parser::Result<Vec<Privilege>> {\n        let mut privileges = Vec::new();\n        let mut depth = 1;\n\n        loop {\n            match self.token()? {\n                Token::ElementStart { name, .. } => {\n                    if let Some(privilege) = Privilege::from_element(name) {\n                        privileges.push(privilege);\n                        self.expect_element_end()?;\n                    } else {\n                        depth += 1;\n                    }\n                }\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                }\n                Token::UnknownElement(_) => {\n                    self.seek_element_end()?;\n                }\n                other => {\n                    return Err(other.into_unexpected());\n                }\n            }\n        }\n\n        Ok(privileges)\n    }\n}\n\nimpl Privilege {\n    pub fn from_element(element: NamedElement) -> Option<Self> {\n        match (element.ns, element.element) {\n            (Namespace::Dav, Element::Read) => Some(Privilege::Read),\n            (Namespace::Dav, Element::Write) => Some(Privilege::Write),\n            (Namespace::Dav, Element::WriteProperties) => Some(Privilege::WriteProperties),\n            (Namespace::Dav, Element::WriteContent) => Some(Privilege::WriteContent),\n            (Namespace::Dav, Element::Unlock) => Some(Privilege::Unlock),\n            (Namespace::Dav, Element::ReadAcl) => Some(Privilege::ReadAcl),\n            (Namespace::Dav, Element::ReadCurrentUserPrivilegeSet) => {\n                Some(Privilege::ReadCurrentUserPrivilegeSet)\n            }\n            (Namespace::Dav, Element::WriteAcl) => Some(Privilege::WriteAcl),\n            (Namespace::Dav, Element::Bind) => Some(Privilege::Bind),\n            (Namespace::Dav, Element::Unbind) => Some(Privilege::Unbind),\n            (Namespace::Dav, Element::All) => Some(Privilege::All),\n            (Namespace::CalDav, Element::ReadFreeBusy) => Some(Privilege::ReadFreeBusy),\n            (Namespace::CalDav, Element::ScheduleDeliver) => Some(Privilege::ScheduleDeliver),\n            (Namespace::CalDav, Element::ScheduleDeliverInvite) => {\n                Some(Privilege::ScheduleDeliverInvite)\n            }\n            (Namespace::CalDav, Element::ScheduleDeliverReply) => {\n                Some(Privilege::ScheduleDeliverReply)\n            }\n            (Namespace::CalDav, Element::ScheduleQueryFreebusy) => {\n                Some(Privilege::ScheduleQueryFreeBusy)\n            }\n            (Namespace::CalDav, Element::ScheduleSend) => Some(Privilege::ScheduleSend),\n            (Namespace::CalDav, Element::ScheduleSendInvite) => Some(Privilege::ScheduleSendInvite),\n            (Namespace::CalDav, Element::ScheduleSendReply) => Some(Privilege::ScheduleSendReply),\n            (Namespace::CalDav, Element::ScheduleSendFreebusy) => {\n                Some(Privilege::ScheduleSendFreeBusy)\n            }\n            _ => None,\n        }\n    }\n}\n\nimpl DavParser for AclPrincipalPropSet {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut acps = AclPrincipalPropSet { properties: vec![] };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Prop,\n                        },\n                    ..\n                } => {\n                    acps.properties = stream.collect_properties(acps.properties)?;\n                }\n                Token::ElementEnd => {\n                    break;\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                other => {\n                    return Err(other.into_unexpected());\n                }\n            }\n        }\n\n        Ok(acps)\n    }\n}\n\nimpl DavParser for PrincipalMatch {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut pm = PrincipalMatch {\n            principal_properties: PrincipalMatchProperties::Self_,\n            properties: vec![],\n        };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::PrincipalProperty,\n                        },\n                    ..\n                } => {\n                    pm.principal_properties = PrincipalMatchProperties::Properties(\n                        stream.collect_properties(Vec::new())?,\n                    );\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Self_,\n                        },\n                    ..\n                } => {\n                    pm.principal_properties = PrincipalMatchProperties::Self_;\n                    stream.expect_element_end()?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Prop,\n                        },\n                    ..\n                } => {\n                    pm.properties = stream.collect_properties(pm.properties)?;\n                }\n                Token::ElementEnd => {\n                    break;\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                other => {\n                    return Err(other.into_unexpected());\n                }\n            }\n        }\n\n        Ok(pm)\n    }\n}\n\nimpl DavParser for PrincipalPropertySearch {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut pps = PrincipalPropertySearch {\n            property_search: vec![],\n            properties: vec![],\n            apply_to_principal_collection_set: false,\n        };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::PropertySearch,\n                        },\n                    ..\n                } => {\n                    if let Some(prop) = PropertySearch::parse(stream)? {\n                        pps.property_search.push(prop);\n                    }\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Prop,\n                        },\n                    ..\n                } => {\n                    pps.properties = stream.collect_properties(pps.properties)?;\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::ApplyToPrincipalCollectionSet,\n                        },\n                    ..\n                } => {\n                    stream.expect_element_end()?;\n                    pps.apply_to_principal_collection_set = true;\n                }\n                Token::ElementEnd => {\n                    break;\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                other => {\n                    return Err(other.into_unexpected());\n                }\n            }\n        }\n\n        Ok(pps)\n    }\n}\n\nimpl PropertySearch {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Option<Self>> {\n        let mut property = None;\n        let mut match_ = None;\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Prop,\n                        },\n                    ..\n                } => {\n                    property = stream.collect_properties(Vec::new())?.into_iter().next();\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Match,\n                        },\n                    ..\n                } => {\n                    match_ = stream.collect_string_value()?;\n                }\n                Token::ElementEnd => {\n                    break;\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                other => {\n                    return Err(other.into_unexpected());\n                }\n            }\n        }\n\n        Ok(property.map(|property| PropertySearch {\n            property,\n            match_: match_.unwrap_or_default(),\n        }))\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/requests/lockinfo.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse types::dead_property::DeadProperty;\n\nuse crate::{\n    parser::{tokenizer::Tokenizer, DavParser, Token},\n    schema::{\n        property::{LockScope, LockType},\n        request::LockInfo,\n        Element, NamedElement, Namespace,\n    },\n};\n\nimpl DavParser for LockInfo {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut lockinfo = LockInfo {\n            lock_scope: LockScope::Exclusive,\n            lock_type: LockType::Write,\n            owner: None,\n        };\n\n        if stream.expect_named_element_or_eof(NamedElement::dav(Element::Lockinfo))? {\n            loop {\n                match stream.token()? {\n                    Token::ElementStart {\n                        name:\n                            NamedElement {\n                                ns: Namespace::Dav,\n                                element: Element::Lockscope,\n                            },\n                        ..\n                    } => {\n                        lockinfo.lock_scope = LockScope::parse(stream)?;\n                    }\n                    Token::ElementStart {\n                        name:\n                            NamedElement {\n                                ns: Namespace::Dav,\n                                element: Element::Locktype,\n                            },\n                        ..\n                    } => {\n                        lockinfo.lock_type = LockType::parse(stream)?;\n                    }\n                    Token::ElementStart {\n                        name:\n                            NamedElement {\n                                ns: Namespace::Dav,\n                                element: Element::Owner,\n                            },\n                        ..\n                    } => {\n                        lockinfo.owner = Some(DeadProperty::parse(stream)?);\n                    }\n                    Token::ElementEnd | Token::Eof => {\n                        break;\n                    }\n                    other => {\n                        return Err(other.into_unexpected());\n                    }\n                }\n            }\n        }\n\n        Ok(lockinfo)\n    }\n}\n\nimpl DavParser for LockScope {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        match stream.unwrap_named_element()? {\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Exclusive,\n            } => {\n                stream.expect_element_end()?;\n                stream.expect_element_end()?;\n                Ok(LockScope::Exclusive)\n            }\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Shared,\n            } => {\n                stream.expect_element_end()?;\n                stream.expect_element_end()?;\n                Ok(LockScope::Shared)\n            }\n            other => Err(other.into_unexpected()),\n        }\n    }\n}\n\nimpl DavParser for LockType {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        match stream.unwrap_named_element()? {\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::Write,\n            } => {\n                stream.expect_element_end()?;\n                stream.expect_element_end()?;\n                Ok(LockType::Write)\n            }\n            other => Err(other.into_unexpected()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/requests/mkcol.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    parser::{tokenizer::Tokenizer, DavParser, Token},\n    schema::{request::MkCol, Element, NamedElement, Namespace},\n};\n\nimpl DavParser for MkCol {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut mkcol = MkCol {\n            is_mkcalendar: false,\n            props: Vec::new(),\n        };\n        match stream.token()? {\n            Token::ElementStart {\n                name:\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Mkcol,\n                    },\n                ..\n            } => {}\n            Token::ElementStart {\n                name:\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::Mkcalendar,\n                    },\n                ..\n            } => {\n                mkcol.is_mkcalendar = true;\n            }\n            Token::Eof => {\n                return Ok(mkcol);\n            }\n            other => return Err(other.into_unexpected()),\n        };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Set,\n                        },\n                    ..\n                } => {\n                    stream.expect_named_element(NamedElement::dav(Element::Prop))?;\n                    stream.collect_property_values(&mut mkcol.props)?;\n                    stream.expect_element_end()?;\n                }\n                Token::ElementEnd | Token::Eof => {\n                    break;\n                }\n                token => return Err(token.into_unexpected()),\n            }\n        }\n\n        Ok(mkcol)\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/requests/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    parser::{tokenizer::Tokenizer, DavParser, RawElement, Token},\n    schema::Namespace,\n};\nuse types::dead_property::{DeadElementTag, DeadProperty, DeadPropertyTag};\n\npub mod acl;\npub mod lockinfo;\npub mod mkcol;\npub mod propertyupdate;\npub mod propfind;\npub mod report;\n\nimpl DavParser for DeadProperty {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut depth = 1;\n        let mut items = DeadProperty::default();\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart { raw, .. } | Token::UnknownElement(raw) => {\n                    items.0.push(DeadPropertyTag::ElementStart((&raw).into()));\n                    depth += 1;\n                }\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                    items.0.push(DeadPropertyTag::ElementEnd);\n                }\n                Token::Text(text) => {\n                    items.0.push(DeadPropertyTag::Text(text.into_owned()));\n                }\n                Token::Bytes(bytes) => {\n                    items.0.push(DeadPropertyTag::Text(\n                        String::from_utf8_lossy(&bytes).into_owned(),\n                    ));\n                }\n                Token::Eof => {\n                    break;\n                }\n            }\n        }\n\n        Ok(items)\n    }\n}\n\npub trait NsDeadProperty {\n    fn single_with_ns(namespace: Namespace, name: &str) -> Self;\n}\n\nimpl NsDeadProperty for DeadProperty {\n    fn single_with_ns(namespace: Namespace, name: &str) -> Self {\n        DeadProperty(vec![\n            DeadPropertyTag::ElementStart(DeadElementTag {\n                name: format!(\"{}:{name}\", namespace.prefix()),\n                attrs: None,\n            }),\n            DeadPropertyTag::ElementEnd,\n        ])\n    }\n}\n\nimpl From<&RawElement<'_>> for DeadElementTag {\n    fn from(raw: &RawElement<'_>) -> Self {\n        let name = std::str::from_utf8(raw.element.local_name().as_ref())\n            .unwrap_or(\"invalid-utf8\")\n            .trim_ascii()\n            .to_string();\n        let mut attrs = String::with_capacity(raw.element.attributes_raw().len());\n        if let Some(namespace) = &raw.namespace {\n            attrs.push_str(\"xmlns=\\\"\");\n            attrs.push_str(std::str::from_utf8(namespace).unwrap_or(\"invalid-utf8\"));\n            attrs.push('\"');\n        }\n\n        for attr in raw.element.attributes().flatten() {\n            if attr.key.as_ref() == b\"xmlns\" || attr.key.as_ref().starts_with(b\"xmlns:\") {\n                // Skip namespace attributes\n                continue;\n            }\n            if let (Ok(key), Ok(value)) = (\n                std::str::from_utf8(attr.key.as_ref()),\n                std::str::from_utf8(attr.value.as_ref()),\n            ) {\n                if !attrs.is_empty() {\n                    attrs.push(' ');\n                }\n                attrs.push_str(key);\n                attrs.push('=');\n                attrs.push('\"');\n                attrs.push_str(value);\n                attrs.push('\"');\n            }\n        }\n\n        DeadElementTag {\n            name,\n            attrs: (!attrs.is_empty()).then_some(attrs),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        parser::{tokenizer::Tokenizer, DavParser},\n        schema::request::{Acl, LockInfo, MkCol, PropFind, PropertyUpdate, Report},\n    };\n\n    #[test]\n    fn parse_requests() {\n        for entry in std::fs::read_dir(\"resources/requests\").unwrap() {\n            let entry = entry.unwrap();\n            let path = entry.path();\n\n            if path.extension().map(|ext| ext == \"xml\").unwrap_or(false) {\n                println!(\"Parsing: {:?}\", path);\n                let filename = path.file_name().unwrap().to_str().unwrap();\n                let xml = std::fs::read_to_string(&path).unwrap();\n                let mut tokenizer = Tokenizer::new(xml.as_bytes());\n\n                let json_path = path.with_extension(\"json\");\n                let json_output = match filename.split_once('-').unwrap().0 {\n                    \"propfind\" => match PropFind::parse(&mut tokenizer) {\n                        Ok(propfind) => serde_json::to_string_pretty(&propfind).unwrap(),\n                        Err(_) => String::new(),\n                    },\n                    \"propertyupdate\" => serde_json::to_string_pretty(\n                        &PropertyUpdate::parse(&mut tokenizer).unwrap(),\n                    )\n                    .unwrap(),\n                    \"mkcol\" => serde_json::to_string_pretty(&MkCol::parse(&mut tokenizer).unwrap())\n                        .unwrap(),\n                    \"lockinfo\" => {\n                        serde_json::to_string_pretty(&LockInfo::parse(&mut tokenizer).unwrap())\n                            .unwrap()\n                    }\n                    \"report\" => {\n                        serde_json::to_string_pretty(&Report::parse(&mut tokenizer).unwrap())\n                            .unwrap()\n                    }\n                    \"acl\" => {\n                        serde_json::to_string_pretty(&Acl::parse(&mut tokenizer).unwrap()).unwrap()\n                    }\n                    _ => {\n                        panic!(\"Unknown method: {}\", filename);\n                    }\n                };\n\n                /*if json_path.exists() {\n                    let expected = std::fs::read_to_string(json_path).unwrap();\n                    assert_eq!(json_output, expected);\n                } else {*/\n                std::fs::write(json_path, json_output).unwrap();\n                //}\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/requests/propertyupdate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    parser::{tokenizer::Tokenizer, DavParser, Token},\n    schema::{request::PropertyUpdate, Element, NamedElement, Namespace},\n};\n\nimpl DavParser for PropertyUpdate {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        stream.expect_named_element(NamedElement::dav(Element::Propertyupdate))?;\n        let mut update = PropertyUpdate {\n            set: Vec::with_capacity(4),\n            remove: Vec::with_capacity(4),\n            set_first: true,\n        };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Set,\n                        },\n                    ..\n                } => {\n                    stream.expect_named_element(NamedElement::dav(Element::Prop))?;\n                    stream.collect_property_values(&mut update.set)?;\n                    stream.expect_element_end()?;\n                    update.set_first = update.remove.is_empty();\n                }\n                Token::ElementStart {\n                    name:\n                        NamedElement {\n                            ns: Namespace::Dav,\n                            element: Element::Remove,\n                        },\n                    ..\n                } => {\n                    stream.expect_named_element(NamedElement::dav(Element::Prop))?;\n                    update.remove = stream.collect_properties(update.remove)?;\n                    stream.expect_element_end()?;\n                }\n                Token::ElementEnd | Token::Eof => {\n                    break;\n                }\n                Token::UnknownElement(_) => {\n                    // Ignore unknown elements\n                    stream.seek_element_end()?;\n                }\n                token => return Err(token.into_unexpected()),\n            }\n        }\n\n        Ok(update)\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/requests/propfind.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    parser::{tokenizer::Tokenizer, DavParser, Token},\n    schema::{request::PropFind, Element, NamedElement, Namespace},\n};\n\nimpl DavParser for PropFind {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        if stream.expect_named_element_or_eof(NamedElement::dav(Element::Propfind))? {\n            match stream.unwrap_named_element()? {\n                NamedElement {\n                    ns: Namespace::Dav,\n                    element: Element::Propname,\n                } => Ok(PropFind::PropName),\n                NamedElement {\n                    ns: Namespace::Dav,\n                    element: Element::Allprop,\n                } => {\n                    stream.expect_element_end()?;\n                    if matches!(\n                        stream.token()?,\n                        Token::ElementStart {\n                            name: NamedElement {\n                                ns: Namespace::Dav,\n                                element: Element::Include\n                            },\n                            ..\n                        }\n                    ) {\n                        stream.collect_properties(Vec::new()).map(PropFind::AllProp)\n                    } else {\n                        Ok(PropFind::AllProp(vec![]))\n                    }\n                }\n                NamedElement {\n                    ns: Namespace::Dav,\n                    element: Element::Prop,\n                } => stream.collect_properties(Vec::new()).map(PropFind::Prop),\n                element => Err(element.into_unexpected()),\n            }\n        } else {\n            Ok(PropFind::AllProp(vec![]))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/requests/report.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    parser::{\n        property::TimeRangeFromRaw, tokenizer::Tokenizer, DavParser, RawElement, Token,\n        XmlValueParser,\n    },\n    schema::{\n        property::DavProperty,\n        request::{\n            AclPrincipalPropSet, AddressbookQuery, CalendarQuery, ExpandProperty,\n            ExpandPropertyItem, Filter, FilterOp, FreeBusyQuery, MultiGet, PrincipalMatch,\n            PrincipalPropertySearch, PropFind, Report, SyncCollection, TextMatch, Timezone,\n            VCardPropertyWithGroup,\n        },\n        Attribute, Collation, Element, MatchType, NamedElement, Namespace,\n    },\n    Depth,\n};\nuse calcard::{\n    icalendar::{ICalendarComponentType, ICalendarParameterName, ICalendarProperty},\n    vcard::VCardParameterName,\n};\nuse types::{dead_property::DeadElementTag, TimeRange};\n\nimpl DavParser for Report {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        match stream.unwrap_named_element()? {\n            NamedElement {\n                ns: Namespace::CalDav,\n                element: Element::CalendarQuery,\n            } => CalendarQuery::parse(stream).map(Report::CalendarQuery),\n            NamedElement {\n                ns: Namespace::CalDav,\n                element: Element::FreeBusyQuery,\n            } => FreeBusyQuery::parse(stream).map(Report::FreeBusyQuery),\n            NamedElement {\n                ns: Namespace::CalDav,\n                element: Element::CalendarMultiget,\n            } => MultiGet::parse(stream).map(Report::CalendarMultiGet),\n            NamedElement {\n                ns: Namespace::CardDav,\n                element: Element::AddressbookQuery,\n            } => AddressbookQuery::parse(stream).map(Report::AddressbookQuery),\n            NamedElement {\n                ns: Namespace::CardDav,\n                element: Element::AddressbookMultiget,\n            } => MultiGet::parse(stream).map(Report::AddressbookMultiGet),\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::SyncCollection,\n            } => SyncCollection::parse(stream).map(Report::SyncCollection),\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::AclPrincipalPropSet,\n            } => AclPrincipalPropSet::parse(stream).map(Report::AclPrincipalPropSet),\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::PrincipalMatch,\n            } => PrincipalMatch::parse(stream).map(Report::PrincipalMatch),\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::PrincipalPropertySearch,\n            } => PrincipalPropertySearch::parse(stream).map(Report::PrincipalPropertySearch),\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::PrincipalSearchPropertySet,\n            } => stream\n                .expect_element_end()\n                .map(|_| Report::PrincipalSearchPropertySet),\n            NamedElement {\n                ns: Namespace::Dav,\n                element: Element::ExpandProperty,\n            } => ExpandProperty::parse(stream).map(Report::ExpandProperty),\n            other => Err(other.into_unexpected()),\n        }\n    }\n}\n\nimpl DavParser for CalendarQuery {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut cq = CalendarQuery {\n            properties: PropFind::AllProp(vec![]),\n            filters: vec![],\n            timezone: Timezone::None,\n        };\n        let mut depth = 1;\n        let mut components = Vec::with_capacity(3);\n        let mut property = None;\n        let mut parameter = None;\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart { name, raw } => match name {\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Propname,\n                    } if depth == 1 => {\n                        cq.properties = PropFind::PropName;\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Allprop,\n                    } if depth == 1 => {\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Prop,\n                    } if depth == 1 => {\n                        cq.properties = PropFind::Prop(stream.collect_properties(Vec::new())?);\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::Filter,\n                    } if depth == 1 => {\n                        depth += 1;\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::Timezone,\n                    } if depth == 1 => {\n                        cq.timezone =\n                            Timezone::Name(stream.collect_string_value()?.unwrap_or_default());\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::TimezoneId,\n                    } if depth == 1 => {\n                        cq.timezone =\n                            Timezone::Id(stream.collect_string_value()?.unwrap_or_default());\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::CompFilter,\n                    } if depth >= 2 => {\n                        for attribute in raw.attributes::<ICalendarComponentType>() {\n                            if let Attribute::Name(name) = attribute? {\n                                components.push((name, depth));\n                            }\n                        }\n                        depth += 1;\n                    }\n\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::PropFilter,\n                    } if depth >= 3 => {\n                        for attribute in raw.attributes::<ICalendarProperty>() {\n                            if let Attribute::Name(name) = attribute? {\n                                property = Some(name);\n                            }\n                        }\n                        depth += 1;\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::ParamFilter,\n                    } if depth >= 4 => {\n                        for attribute in raw.attributes::<ICalendarParameterName>() {\n                            if let Attribute::Name(name) = attribute? {\n                                parameter = Some(name);\n                            }\n                        }\n                        depth += 1;\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::IsNotDefined,\n                    } => {\n                        stream.expect_element_end()?;\n                        if let Some(filter) = Filter::from_parts(\n                            components.iter().map(|(c, _)| c.clone()).collect(),\n                            property.clone(),\n                            parameter.clone(),\n                            FilterOp::Undefined,\n                        ) {\n                            cq.filters.push(filter);\n                        }\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::TextMatch,\n                    } => {\n                        let mut tm = TextMatch::parse(raw)?;\n                        tm.value = stream.collect_string_value()?.unwrap_or_default();\n                        if let Some(filter) = Filter::from_parts(\n                            components.iter().map(|(c, _)| c.clone()).collect(),\n                            property.clone(),\n                            parameter.clone(),\n                            FilterOp::TextMatch(tm),\n                        ) {\n                            cq.filters.push(filter);\n                        }\n                    }\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::TimeRange,\n                    } => {\n                        let range = TimeRange::from_raw(&raw)?;\n                        stream.expect_element_end()?;\n                        if let Some(filter) = range.and_then(|range| {\n                            Filter::from_parts(\n                                components.iter().map(|(c, _)| c.clone()).collect(),\n                                property.clone(),\n                                parameter.clone(),\n                                FilterOp::TimeRange(range),\n                            )\n                        }) {\n                            cq.filters.push(filter);\n                        }\n                    }\n                    name => return Err(name.into_unexpected()),\n                },\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                    if matches!(components.last(), Some((_, d)) if *d == depth) {\n                        if components.len() > 1\n                            && cq\n                                .filters\n                                .last()\n                                .and_then(|c| c.components())\n                                .is_none_or(|c| c.len() < components.len())\n                        {\n                            cq.filters.push(Filter::Component {\n                                comp: components.iter().map(|(c, _)| c.clone()).collect(),\n                                op: FilterOp::Exists,\n                            });\n                        }\n                        components.pop();\n                    }\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                element => return Err(element.into_unexpected()),\n            }\n        }\n\n        Ok(cq)\n    }\n}\n\nimpl DavParser for AddressbookQuery {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut aq = AddressbookQuery {\n            properties: PropFind::AllProp(vec![]),\n            filters: vec![],\n            limit: None,\n        };\n        let mut depth = 1;\n        let mut property = None;\n        let mut parameter = None;\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart { name, raw } => match name {\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Propname,\n                    } if depth == 1 => {\n                        aq.properties = PropFind::PropName;\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Allprop,\n                    } if depth == 1 => {\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Prop,\n                    } if depth == 1 => {\n                        aq.properties = PropFind::Prop(stream.collect_properties(Vec::new())?);\n                    }\n                    NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Filter,\n                    } if depth == 1 => {\n                        if let Some(filter) = Filter::parse(raw)? {\n                            aq.filters.push(filter);\n                        }\n                        depth += 1;\n                    }\n                    NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::Limit,\n                    } if depth == 1 => {\n                        stream.expect_named_element(NamedElement::carddav(Element::Nresults))?;\n                        if let Some(Ok(limit)) = stream.parse_value::<u32>()? {\n                            aq.limit = limit.into();\n                        }\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::PropFilter,\n                    } if depth == 2 => {\n                        let mut filter = None;\n                        for attribute in raw.attributes::<VCardPropertyWithGroup>() {\n                            match attribute? {\n                                Attribute::Name(name) => {\n                                    property = Some(name);\n                                }\n                                Attribute::TestAllOf(all_of) => {\n                                    filter =\n                                        (if all_of { Filter::AllOf } else { Filter::AnyOf }).into();\n                                }\n                                _ => {}\n                            }\n                        }\n                        if let Some(filter) = filter {\n                            aq.filters.push(filter);\n                        }\n                        depth += 1;\n                    }\n                    NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::ParamFilter,\n                    } if depth == 3 => {\n                        for attribute in raw.attributes::<VCardParameterName>() {\n                            if let Attribute::Name(name) = attribute? {\n                                parameter = Some(name);\n                            }\n                        }\n                        depth += 1;\n                    }\n                    NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::IsNotDefined,\n                    } => {\n                        stream.expect_element_end()?;\n                        if let Some(filter) = Filter::from_parts(\n                            (),\n                            property.clone(),\n                            parameter.clone(),\n                            FilterOp::Undefined,\n                        ) {\n                            aq.filters.push(filter);\n                        }\n                    }\n                    NamedElement {\n                        ns: Namespace::CardDav,\n                        element: Element::TextMatch,\n                    } => {\n                        let mut tm = TextMatch::parse(raw)?;\n                        tm.value = stream.collect_string_value()?.unwrap_or_default();\n                        if let Some(filter) = Filter::from_parts(\n                            (),\n                            property.clone(),\n                            parameter.clone(),\n                            FilterOp::TextMatch(tm),\n                        ) {\n                            aq.filters.push(filter);\n                        }\n                    }\n                    name => return Err(name.into_unexpected()),\n                },\n                Token::ElementEnd => {\n                    depth -= 1;\n                    if depth == 0 {\n                        break;\n                    }\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                element => return Err(element.into_unexpected()),\n            }\n        }\n\n        Ok(aq)\n    }\n}\n\nimpl DavParser for FreeBusyQuery {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        match stream.token()? {\n            Token::ElementStart {\n                name:\n                    NamedElement {\n                        ns: Namespace::CalDav,\n                        element: Element::TimeRange,\n                    },\n                raw,\n            } => TimeRange::from_raw(&raw).map(|range| FreeBusyQuery { range }),\n            other => Err(other.into_unexpected()),\n        }\n    }\n}\n\nimpl DavParser for MultiGet {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut mg = MultiGet {\n            properties: PropFind::AllProp(vec![]),\n            hrefs: vec![],\n        };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart { name, .. } => match name {\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Propname,\n                    } => {\n                        mg.properties = PropFind::PropName;\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Allprop,\n                    } => {\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Prop,\n                    } => {\n                        mg.properties = PropFind::Prop(stream.collect_properties(Vec::new())?);\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Href,\n                    } => {\n                        if let Some(href) = stream.collect_string_value()? {\n                            mg.hrefs.push(href);\n                        }\n                    }\n                    name => return Err(name.into_unexpected()),\n                },\n                Token::ElementEnd => {\n                    break;\n                }\n                element => return Err(element.into_unexpected()),\n            }\n        }\n\n        Ok(mg)\n    }\n}\n\nimpl DavParser for SyncCollection {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut sc = SyncCollection {\n            properties: PropFind::AllProp(vec![]),\n            limit: None,\n            sync_token: None,\n            depth: Depth::None,\n        };\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart { name, .. } => match name {\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Prop,\n                    } => {\n                        sc.properties = PropFind::Prop(stream.collect_properties(Vec::new())?);\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::Limit,\n                    } => {\n                        stream.expect_named_element(NamedElement::dav(Element::Nresults))?;\n                        if let Some(Ok(limit)) = stream.parse_value::<u32>()? {\n                            sc.limit = limit.into();\n                        }\n                        stream.expect_element_end()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::SyncToken,\n                    } => {\n                        sc.sync_token = stream.collect_string_value()?;\n                    }\n                    NamedElement {\n                        ns: Namespace::Dav,\n                        element: Element::SyncLevel,\n                    } => {\n                        if let Some(Ok(depth)) = stream.parse_value::<Depth>()? {\n                            sc.depth = depth;\n                        }\n                    }\n                    name => return Err(name.into_unexpected()),\n                },\n                Token::ElementEnd => {\n                    break;\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                element => return Err(element.into_unexpected()),\n            }\n        }\n\n        Ok(sc)\n    }\n}\n\nimpl DavParser for ExpandProperty {\n    fn parse(stream: &mut Tokenizer<'_>) -> crate::parser::Result<Self> {\n        let mut ep = ExpandProperty { properties: vec![] };\n        let mut depth = 1;\n\n        loop {\n            match stream.token()? {\n                Token::ElementStart { name, raw } => match name {\n                    NamedElement {\n                        ns,\n                        element: Element::Property,\n                    } => {\n                        for attribute in raw.attributes::<String>() {\n                            if let Attribute::Name(name) = attribute? {\n                                if let Some(property) = Element::try_parse(name.as_bytes())\n                                    .copied()\n                                    .and_then(|element| {\n                                        DavProperty::from_element(NamedElement { ns, element })\n                                    })\n                                {\n                                    ep.properties.push(ExpandPropertyItem {\n                                        property,\n                                        depth: depth - 1,\n                                    });\n                                } else {\n                                    let attrs = raw.element.attributes_raw().trim_ascii();\n                                    ep.properties.push(ExpandPropertyItem {\n                                        property: DavProperty::DeadProperty(DeadElementTag {\n                                            name,\n                                            attrs: (!attrs.is_empty()).then(|| {\n                                                String::from_utf8_lossy(attrs).into_owned()\n                                            }),\n                                        }),\n                                        depth: depth - 1,\n                                    });\n                                }\n                                break;\n                            }\n                        }\n                        depth += 1;\n                    }\n                    name => return Err(name.into_unexpected()),\n                },\n                Token::ElementEnd => {\n                    depth -= 1;\n\n                    if depth == 0 {\n                        break;\n                    }\n                }\n                Token::UnknownElement(_) => {\n                    stream.seek_element_end()?;\n                }\n                element => return Err(element.into_unexpected()),\n            }\n        }\n\n        Ok(ep)\n    }\n}\n\nimpl TextMatch {\n    fn parse(raw: RawElement<'_>) -> crate::parser::Result<Self> {\n        let mut tm = TextMatch {\n            match_type: MatchType::Contains,\n            value: String::new(),\n            collation: Collation::AsciiCasemap,\n            negate: false,\n        };\n\n        for attribute in raw.attributes::<String>() {\n            match attribute? {\n                Attribute::MatchType(match_type) => {\n                    tm.match_type = match_type;\n                }\n                Attribute::NegateCondition(negate) => {\n                    tm.negate = negate;\n                }\n                Attribute::Collation(collation) => {\n                    tm.collation = collation;\n                }\n                _ => {}\n            }\n        }\n\n        Ok(tm)\n    }\n}\n\nimpl<A, B, C> Filter<A, B, C> {\n    fn from_parts(comp: A, prop: Option<B>, param: Option<C>, op: FilterOp) -> Option<Self> {\n        match (prop, param) {\n            (Some(prop), Some(param)) => Some(Filter::Parameter {\n                comp,\n                prop,\n                param,\n                op,\n            }),\n            (Some(prop), None) => Some(Filter::Property { comp, prop, op }),\n            (None, None) => Some(Filter::Component { comp, op }),\n            _ => None,\n        }\n    }\n\n    fn components(&self) -> Option<&A> {\n        match self {\n            Filter::Component { comp, .. } => Some(comp),\n            Filter::Property { comp, .. } => Some(comp),\n            Filter::Parameter { comp, .. } => Some(comp),\n            _ => None,\n        }\n    }\n\n    fn parse(raw: RawElement<'_>) -> crate::parser::Result<Option<Self>> {\n        for attribute in raw.attributes::<String>() {\n            if let Attribute::TestAllOf(all_of) = attribute? {\n                return Ok(Some(if all_of { Filter::AllOf } else { Filter::AnyOf }));\n            }\n        }\n\n        Ok(None)\n    }\n}\n\nimpl XmlValueParser for Depth {\n    fn parse_bytes(bytes: &[u8]) -> Option<Self> {\n        Depth::parse(bytes)\n    }\n\n    fn parse_str(text: &str) -> Option<Self> {\n        Depth::parse(text.as_bytes())\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse crate::{\n    responses::XmlEscape,\n    schema::{\n        property::{DavProperty, Privilege},\n        response::{\n            Ace, AclRestrictions, GrantDeny, Href, List, Principal, PrincipalSearchProperty,\n            PrincipalSearchPropertySet, RequiredPrincipal, Resource, SupportedPrivilege,\n        },\n        Namespace, Namespaces,\n    },\n};\n\nimpl Display for SupportedPrivilege {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:supported-privilege>{}\", self.privilege)?;\n        if self.abstract_ {\n            write!(f, \"<D:abstract/>\")?;\n        }\n        write!(f, \"<D:description>\")?;\n        self.description.write_escaped_to(f)?;\n        write!(\n            f,\n            \"</D:description>{}</D:supported-privilege>\",\n            self.supported_privilege\n        )\n    }\n}\n\nimpl SupportedPrivilege {\n    pub fn new(privilege: Privilege, description: impl Into<String>) -> Self {\n        SupportedPrivilege {\n            privilege,\n            abstract_: false,\n            description: description.into(),\n            supported_privilege: List(vec![]),\n        }\n    }\n\n    pub fn with_abstract(mut self) -> Self {\n        self.abstract_ = true;\n        self\n    }\n\n    pub fn with_supported_privilege(mut self, supported_privilege: SupportedPrivilege) -> Self {\n        self.supported_privilege.0.push(supported_privilege);\n        self\n    }\n\n    pub fn with_opt_supported_privilege(\n        mut self,\n        supported_privilege: Option<SupportedPrivilege>,\n    ) -> Self {\n        if let Some(supported_privilege) = supported_privilege {\n            self.supported_privilege.0.push(supported_privilege);\n        }\n        self\n    }\n\n    pub fn all_privileges(is_calendar: bool) -> SupportedPrivilege {\n        SupportedPrivilege::new(Privilege::All, \"Any operation\")\n            .with_abstract()\n            .with_supported_privilege(\n                SupportedPrivilege::new(Privilege::Read, \"Read objects\").with_supported_privilege(\n                    SupportedPrivilege::new(\n                        Privilege::ReadCurrentUserPrivilegeSet,\n                        \"Read current user privileges\",\n                    ),\n                ),\n            )\n            .with_supported_privilege(\n                SupportedPrivilege::new(Privilege::Write, \"Write objects\")\n                    .with_supported_privilege(SupportedPrivilege::new(\n                        Privilege::WriteProperties,\n                        \"Write properties\",\n                    ))\n                    .with_supported_privilege(SupportedPrivilege::new(\n                        Privilege::WriteContent,\n                        \"Write object contents\",\n                    ))\n                    .with_supported_privilege(SupportedPrivilege::new(\n                        Privilege::Bind,\n                        \"Add resources to a collection\",\n                    ))\n                    .with_supported_privilege(SupportedPrivilege::new(\n                        Privilege::Unbind,\n                        \"Remove resources from a collection\",\n                    ))\n                    .with_supported_privilege(SupportedPrivilege::new(\n                        Privilege::Unlock,\n                        \"Unlock resources\",\n                    )),\n            )\n            .with_supported_privilege(SupportedPrivilege::new(Privilege::ReadAcl, \"Read ACL\"))\n            .with_supported_privilege(SupportedPrivilege::new(Privilege::WriteAcl, \"Write ACL\"))\n            .with_opt_supported_privilege((is_calendar).then(|| {\n                SupportedPrivilege::new(Privilege::ReadFreeBusy, \"Read free/busy information\")\n            }))\n    }\n\n    pub fn all_scheduling_privileges(is_inbox: bool) -> SupportedPrivilege {\n        let privilege = SupportedPrivilege::new(Privilege::All, \"Any operation\")\n            .with_abstract()\n            .with_supported_privilege(\n                SupportedPrivilege::new(Privilege::Read, \"Read objects\").with_supported_privilege(\n                    SupportedPrivilege::new(\n                        Privilege::ReadCurrentUserPrivilegeSet,\n                        \"Read current user privileges\",\n                    ),\n                ),\n            );\n\n        if is_inbox {\n            privilege.with_supported_privilege(\n                SupportedPrivilege::new(\n                    Privilege::ScheduleDeliver,\n                    \"Deliver calendar scheduling messages\",\n                )\n                .with_supported_privilege(SupportedPrivilege::new(\n                    Privilege::ScheduleDeliverInvite,\n                    \"Deliver calendar scheduling invites\",\n                ))\n                .with_supported_privilege(SupportedPrivilege::new(\n                    Privilege::ScheduleDeliverReply,\n                    \"Deliver calendar scheduling replies\",\n                ))\n                .with_supported_privilege(SupportedPrivilege::new(\n                    Privilege::ScheduleQueryFreeBusy,\n                    \"Query free/busy information\",\n                )),\n            )\n        } else {\n            privilege.with_supported_privilege(\n                SupportedPrivilege::new(\n                    Privilege::ScheduleSend,\n                    \"Send calendar scheduling messages\",\n                )\n                .with_supported_privilege(SupportedPrivilege::new(\n                    Privilege::ScheduleSendInvite,\n                    \"Send calendar scheduling invites\",\n                ))\n                .with_supported_privilege(SupportedPrivilege::new(\n                    Privilege::ScheduleSendReply,\n                    \"Send calendar scheduling replies\",\n                ))\n                .with_supported_privilege(SupportedPrivilege::new(\n                    Privilege::ScheduleSendFreeBusy,\n                    \"Send free/busy information\",\n                )),\n            )\n        }\n    }\n}\n\nimpl Display for Ace {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:ace>\")?;\n        if self.invert {\n            write!(f, \"<D:invert>\")?;\n        }\n        self.principal.fmt(f)?;\n        if self.invert {\n            write!(f, \"</D:invert>\")?;\n        }\n        self.grant_deny.fmt(f)?;\n        if self.protected {\n            write!(f, \"<D:protected/>\")?;\n        }\n        if let Some(inherited) = &self.inherited {\n            write!(f, \"<D:inherited>\")?;\n            inherited.fmt(f)?;\n            write!(f, \"</D:inherited>\")?;\n        }\n        write!(f, \"</D:ace>\")\n    }\n}\n\nimpl Display for Principal {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:principal>\")?;\n        match self {\n            Principal::Href(href) => href.fmt(f),\n            Principal::Response(response) => response.fmt(f),\n            Principal::All => \"<D:all/>\".fmt(f),\n            Principal::Authenticated => \"<D:authenticated/>\".fmt(f),\n            Principal::Unauthenticated => \"<D:unauthenticated/>\".fmt(f),\n            Principal::Property(property) => {\n                write!(f, \"<D:property>{}</D:property>\", property)\n            }\n            Principal::Self_ => \"<D:self/>\".fmt(f),\n        }?;\n        write!(f, \"</D:principal>\")\n    }\n}\n\nimpl Display for GrantDeny {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            GrantDeny::Grant(privileges) => {\n                write!(f, \"<D:grant>\")?;\n                privileges.fmt(f)?;\n                write!(f, \"</D:grant>\")\n            }\n            GrantDeny::Deny(privileges) => {\n                write!(f, \"<D:deny>\")?;\n                privileges.fmt(f)?;\n                write!(f, \"</D:deny>\")\n            }\n        }\n    }\n}\n\nimpl Display for AclRestrictions {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        if self.grant_only {\n            write!(f, \"<D:grant-only/>\")?;\n        }\n        if self.no_invert {\n            write!(f, \"<D:no-invert/>\")?;\n        }\n        if self.deny_before_grant {\n            write!(f, \"<D:deny-before-grant/>\")?;\n        }\n        if let Some(required_principal) = &self.required_principal {\n            required_principal.fmt(f)?;\n        }\n        Ok(())\n    }\n}\n\nimpl Display for RequiredPrincipal {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:required-principal>\")?;\n        match self {\n            RequiredPrincipal::All => \"<D:all/>\".fmt(f)?,\n            RequiredPrincipal::Authenticated => \"<D:authenticated/>\".fmt(f)?,\n            RequiredPrincipal::Unauthenticated => \"<D:unauthenticated/>\".fmt(f)?,\n            RequiredPrincipal::Self_ => \"<D:self/>\".fmt(f)?,\n            RequiredPrincipal::Href(hrefs) => hrefs.fmt(f)?,\n            RequiredPrincipal::Property(properties) => {\n                for property in properties {\n                    write!(f, \"<D:property>{}</D:property>\", property)?;\n                }\n            }\n        }\n        write!(f, \"</D:required-principal>\")\n    }\n}\n\nimpl Display for Privilege {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Privilege::Read => \"<D:privilege><D:read/></D:privilege>\".fmt(f),\n            Privilege::Write => \"<D:privilege><D:write/></D:privilege>\".fmt(f),\n            Privilege::WriteProperties => \"<D:privilege><D:write-properties/></D:privilege>\".fmt(f),\n            Privilege::WriteContent => \"<D:privilege><D:write-content/></D:privilege>\".fmt(f),\n            Privilege::Unlock => \"<D:privilege><D:unlock/></D:privilege>\".fmt(f),\n            Privilege::ReadAcl => \"<D:privilege><D:read-acl/></D:privilege>\".fmt(f),\n            Privilege::ReadCurrentUserPrivilegeSet => {\n                \"<D:privilege><D:read-current-user-privilege-set/></D:privilege>\".fmt(f)\n            }\n            Privilege::WriteAcl => \"<D:privilege><D:write-acl/></D:privilege>\".fmt(f),\n            Privilege::Bind => \"<D:privilege><D:bind/></D:privilege>\".fmt(f),\n            Privilege::Unbind => \"<D:privilege><D:unbind/></D:privilege>\".fmt(f),\n            Privilege::All => \"<D:privilege><D:all/></D:privilege>\".fmt(f),\n            Privilege::ReadFreeBusy => \"<D:privilege><A:read-free-busy/></D:privilege>\".fmt(f),\n            Privilege::ScheduleDeliver => \"<D:privilege><A:schedule-deliver/></D:privilege>\".fmt(f),\n            Privilege::ScheduleDeliverInvite => {\n                \"<D:privilege><A:schedule-deliver-invite/></D:privilege>\".fmt(f)\n            }\n            Privilege::ScheduleDeliverReply => {\n                \"<D:privilege><A:schedule-deliver-reply/></D:privilege>\".fmt(f)\n            }\n            Privilege::ScheduleQueryFreeBusy => {\n                \"<D:privilege><A:schedule-query-freebusy/></D:privilege>\".fmt(f)\n            }\n            Privilege::ScheduleSend => \"<D:privilege><A:schedule-send/></D:privilege>\".fmt(f),\n            Privilege::ScheduleSendInvite => {\n                \"<D:privilege><A:schedule-send-invite/></D:privilege>\".fmt(f)\n            }\n            Privilege::ScheduleSendReply => {\n                \"<D:privilege><A:schedule-send-reply/></D:privilege>\".fmt(f)\n            }\n            Privilege::ScheduleSendFreeBusy => {\n                \"<D:privilege><A:schedule-send-freebusy/></D:privilege>\".fmt(f)\n            }\n        }\n    }\n}\n\nimpl Display for Resource {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"<D:resource>{}{}</D:resource>\",\n            self.href, self.privilege\n        )\n    }\n}\n\nimpl Display for PrincipalSearchPropertySet {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?>\")?;\n        write!(\n            f,\n            \"<D:principal-search-property-set {}>{}</D:principal-search-property-set>\",\n            self.namespaces, self.properties\n        )\n    }\n}\n\nimpl Display for PrincipalSearchProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"<D:principal-search-property><D:prop>{}</D:prop>\",\n            self.name\n        )?;\n        write!(\n            f,\n            \"<D:description>{}</D:description></D:principal-search-property>\",\n            self.description\n        )\n    }\n}\n\nimpl Resource {\n    pub fn new(href: impl Into<String>, privilege: Privilege) -> Self {\n        Resource {\n            href: Href(href.into()),\n            privilege,\n        }\n    }\n}\n\nimpl PrincipalSearchPropertySet {\n    pub fn new(properties: Vec<PrincipalSearchProperty>) -> Self {\n        PrincipalSearchPropertySet {\n            namespaces: Namespaces::default(),\n            properties: List(properties),\n        }\n    }\n\n    pub fn with_namespace(mut self, namespace: Namespace) -> Self {\n        self.namespaces.set(namespace);\n        self\n    }\n}\n\nimpl PrincipalSearchProperty {\n    pub fn new(name: impl Into<DavProperty>, description: impl Into<String>) -> Self {\n        PrincipalSearchProperty {\n            name: name.into(),\n            description: description.into(),\n        }\n    }\n}\n\nimpl Ace {\n    pub fn new(principal: Principal, grant_deny: GrantDeny) -> Self {\n        Ace {\n            principal,\n            invert: false,\n            grant_deny,\n            protected: false,\n            inherited: None,\n        }\n    }\n\n    pub fn with_invert(mut self) -> Self {\n        self.invert = true;\n        self\n    }\n\n    pub fn with_protected(mut self) -> Self {\n        self.protected = true;\n        self\n    }\n\n    pub fn with_inherited(mut self, inherited: impl Into<String>) -> Self {\n        self.inherited = Some(Href(inherited.into()));\n        self\n    }\n}\n\nimpl GrantDeny {\n    pub fn grant(privileges: Vec<Privilege>) -> Self {\n        GrantDeny::Grant(List(privileges))\n    }\n\n    pub fn deny(privileges: Vec<Privilege>) -> Self {\n        GrantDeny::Deny(List(privileges))\n    }\n}\n\nimpl AclRestrictions {\n    pub fn new() -> Self {\n        Self::default()\n    }\n\n    pub fn with_grant_only(mut self) -> Self {\n        self.grant_only = true;\n        self\n    }\n\n    pub fn with_no_invert(mut self) -> Self {\n        self.no_invert = true;\n        self\n    }\n\n    pub fn with_deny_before_grant(mut self) -> Self {\n        self.deny_before_grant = true;\n        self\n    }\n\n    pub fn with_required_principal(mut self, required_principal: RequiredPrincipal) -> Self {\n        self.required_principal = Some(required_principal);\n        self\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/error.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse crate::schema::{\n    response::{BaseCondition, CalCondition, CardCondition, Condition, ErrorResponse},\n    Namespace, Namespaces,\n};\n\nimpl Display for ErrorResponse {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><D:error {}>\",\n            self.namespaces\n        )?;\n\n        match &self.error {\n            Condition::Base(e) => e.fmt(f)?,\n            Condition::Cal(e) => e.fmt(f)?,\n            Condition::Card(e) => e.fmt(f)?,\n        }\n\n        write!(f, \"</D:error>\")\n    }\n}\n\nimpl Display for Condition {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:error>\")?;\n\n        match self {\n            Condition::Base(e) => e.fmt(f)?,\n            Condition::Cal(e) => e.fmt(f)?,\n            Condition::Card(e) => e.fmt(f)?,\n        }\n\n        write!(f, \"</D:error>\")\n    }\n}\n\nimpl Display for BaseCondition {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            BaseCondition::NoConflictingLock(items) => {\n                write!(f, \"<D:no-conflicting-lock>{items}</D:no-conflicting-lock>\")\n            }\n            BaseCondition::LockTokenSubmitted(items) => write!(\n                f,\n                \"<D:lock-token-submitted>{items}</D:lock-token-submitted>\"\n            ),\n            BaseCondition::LockTokenMatchesRequestUri => {\n                write!(f, \"<D:lock-token-matches-request-uri/>\")\n            }\n            BaseCondition::CannotModifyProtectedProperty => {\n                write!(f, \"<D:cannot-modify-protected-property/>\")\n            }\n            BaseCondition::NoExternalEntities => write!(f, \"<D:no-external-entities/>\"),\n            BaseCondition::PreservedLiveProperties => write!(f, \"<D:preserved-live-properties/>\"),\n            BaseCondition::PropFindFiniteDepth => write!(f, \"<D:propfind-finite-depth/>\"),\n            BaseCondition::ResourceMustBeNull => write!(f, \"<D:resource-must-be-null/>\"),\n            BaseCondition::NeedPrivileges(resources) => {\n                write!(f, \"<D:need-privileges>{resources}</D:need-privileges>\")\n            }\n            BaseCondition::NumberOfMatchesWithinLimit => {\n                write!(f, \"<D:number-of-matches-within-limits/>\")\n            }\n            BaseCondition::QuotaNotExceeded => write!(f, \"<D:quota-not-exceeded/>\"),\n            BaseCondition::ValidResourceType => write!(f, \"<D:valid-resourcetype/>\"),\n            BaseCondition::ValidSyncToken => write!(f, \"<D:valid-sync-token/>\"),\n            BaseCondition::NoAceConflict => write!(f, \"<D:no-ace-conflict/>\"),\n            BaseCondition::NoProtectedAceConflict => write!(f, \"<D:no-protected-ace-conflict/>\"),\n            BaseCondition::NoInheritedAceConflict => write!(f, \"<D:no-inherited-ace-conflict/>\"),\n            BaseCondition::LimitedNumberOfAces => write!(f, \"<D:limited-number-of-aces/>\"),\n            BaseCondition::DenyBeforeGrant => write!(f, \"<D:deny-before-grant/>\"),\n            BaseCondition::GrantOnly => write!(f, \"<D:grant-only/>\"),\n            BaseCondition::NoInvert => write!(f, \"<D:no-invert/>\"),\n            BaseCondition::NoAbstract => write!(f, \"<D:no-abstract/>\"),\n            BaseCondition::NotSupportedPrivilege => write!(f, \"<D:not-supported-privilege/>\"),\n            BaseCondition::MissingRequiredPrincipal => write!(f, \"<D:missing-required-principal/>\"),\n            BaseCondition::RecognizedPrincipal => write!(f, \"<D:recognized-principal/>\"),\n            BaseCondition::AllowedPrincipal => write!(f, \"<D:allowed-principal/>\"),\n        }\n    }\n}\n\nimpl Display for CalCondition {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            CalCondition::CalendarCollectionLocationOk => {\n                write!(f, \"<A:calendar-collection-location-ok/>\")\n            }\n            CalCondition::ValidCalendarData => write!(f, \"<A:valid-calendar-data/>\"),\n            CalCondition::ValidFilter => write!(f, \"<A:valid-filter/>\"),\n            CalCondition::ValidTimezone => write!(f, \"<A:valid-timezone/>\"),\n            CalCondition::ValidCalendarObjectResource => {\n                write!(f, \"<A:valid-calendar-object-resource/>\")\n            }\n            CalCondition::NoUidConflict(uid) => {\n                write!(f, \"<A:no-uid-conflict>{uid}</A:no-uid-conflict>\")\n            }\n            CalCondition::InitializeCalendarCollection => {\n                write!(f, \"<A:initialize-calendar-collection/>\")\n            }\n            CalCondition::SupportedCalendarData => write!(f, \"<A:supported-calendar-data/>\"),\n            CalCondition::SupportedFilter(_) => write!(f, \"<A:supported-filter/>\"),\n            CalCondition::SupportedCollation(c) => {\n                write!(f, \"<A:supported-collation>{c}</A:supported-collation>\")\n            }\n            CalCondition::MinDateTime => write!(f, \"<A:min-date-time/>\"),\n            CalCondition::MaxDateTime => write!(f, \"<A:max-date-time/>\"),\n            CalCondition::MaxResourceSize(l) => {\n                write!(f, \"<A:max-resource-size>{l}</A:max-resource-size>\")\n            }\n            CalCondition::MaxInstances => write!(f, \"<A:max-instances/>\"),\n            CalCondition::MaxAttendeesPerInstance => write!(f, \"<A:max-attendees-per-instance/>\"),\n            CalCondition::UniqueSchedulingObjectResource(href) => write!(\n                f,\n                \"<A:unique-scheduling-object-resource>{href}</A:unique-scheduling-object-resource>\"\n            ),\n            CalCondition::SameOrganizerInAllComponents => {\n                write!(f, \"<A:same-organizer-in-all-components/>\")\n            }\n            CalCondition::AllowedOrganizerObjectChange => {\n                write!(f, \"<A:allowed-organizer-scheduling-object-change/>\")\n            }\n            CalCondition::AllowedAttendeeObjectChange => {\n                write!(f, \"<A:allowed-attendee-scheduling-object-change/>\")\n            }\n            CalCondition::DefaultCalendarNeeded => write!(f, \"<A:default-calendar-needed/>\"),\n            CalCondition::ValidScheduleDefaultCalendarUrl => {\n                write!(f, \"<A:valid-schedule-default-calendar-URL/>\")\n            }\n            CalCondition::ValidSchedulingMessage => write!(f, \"<A:valid-scheduling-message/>\"),\n            CalCondition::ValidOrganizer => write!(f, \"<A:valid-organizer/>\"),\n            CalCondition::SupportedCalendarComponent => {\n                write!(f, \"<A:supported-calendar-component/>\")\n            }\n        }\n    }\n}\n\nimpl Display for CardCondition {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            CardCondition::SupportedAddressData => write!(f, \"<B:supported-address-data/>\"),\n            CardCondition::SupportedAddressDataConversion => {\n                write!(f, \"<B:supported-address-data-conversion/>\")\n            }\n            CardCondition::SupportedFilter(_) => write!(f, \"<B:supported-filter/>\"),\n            CardCondition::SupportedCollation(c) => {\n                write!(f, \"<B:supported-collation>{c}</B:supported-collation>\")\n            }\n            CardCondition::ValidAddressData => write!(f, \"<B:valid-address-data/>\"),\n            CardCondition::NoUidConflict(uid) => {\n                write!(f, \"<B:no-uid-conflict>{uid}</B:no-uid-conflict>\")\n            }\n            CardCondition::MaxResourceSize(l) => {\n                write!(f, \"<B:max-resource-size>{l}</B:max-resource-size>\")\n            }\n            CardCondition::AddressBookCollectionLocationOk => {\n                write!(f, \"<B:addressbook-collection-location-ok/>\")\n            }\n        }\n    }\n}\n\nimpl From<CalCondition> for Condition {\n    fn from(error: CalCondition) -> Self {\n        Condition::Cal(error)\n    }\n}\n\nimpl From<CardCondition> for Condition {\n    fn from(error: CardCondition) -> Self {\n        Condition::Card(error)\n    }\n}\n\nimpl From<BaseCondition> for Condition {\n    fn from(error: BaseCondition) -> Self {\n        Condition::Base(error)\n    }\n}\n\nimpl ErrorResponse {\n    pub fn new(error: impl Into<Condition>) -> Self {\n        ErrorResponse {\n            namespaces: Namespaces::default(),\n            error: error.into(),\n        }\n    }\n\n    pub fn with_namespace(mut self, namespace: impl Into<Namespace>) -> Self {\n        self.namespaces.set(namespace.into());\n        self\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/lock.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse types::dead_property::DeadProperty;\n\nuse crate::{\n    responses::DeadPropertyFormat,\n    schema::{\n        property::{ActiveLock, LockDiscovery, LockEntry, LockScope, LockType, SupportedLock},\n        request::LockInfo,\n        response::{Href, List},\n    },\n    Depth, Timeout,\n};\n\nimpl Display for SupportedLock {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:supportedlock>{}</D:supportedlock>\", self.0)\n    }\n}\n\nimpl Display for LockDiscovery {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:lockdiscovery>{}</D:lockdiscovery>\", self.0)\n    }\n}\n\nimpl Display for ActiveLock {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"<D:activelock>{}{}{}\",\n            self.lock_scope, self.lock_type, self.depth\n        )?;\n\n        if let Some(owner) = &self.owner {\n            f.write_str(\"<D:owner>\")?;\n            owner.fmt(f)?;\n            f.write_str(\"</D:owner>\")?;\n        }\n\n        write!(f, \"{}\", self.timeout)?;\n\n        if let Some(lock_token) = &self.lock_token {\n            write!(f, \"<D:locktoken>{}</D:locktoken>\", lock_token)?;\n        }\n\n        write!(\n            f,\n            \"<D:lockroot>{}</D:lockroot></D:activelock>\",\n            self.lock_root\n        )\n    }\n}\n\nimpl Display for Depth {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Depth::Zero => write!(f, \"<D:depth>0</D:depth>\"),\n            Depth::One => write!(f, \"<D:depth>1</D:depth>\"),\n            Depth::Infinity => write!(f, \"<D:depth>infinity</D:depth>\"),\n            Depth::None => write!(f, \"<D:depth/>\"),\n        }\n    }\n}\n\nimpl Display for Timeout {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Timeout::Infinite => write!(f, \"<D:timeout>Infinite</D:timeout>\"),\n            Timeout::Second(s) => write!(f, \"<D:timeout>Second-{}</D:timeout>\", s),\n            Timeout::None => Ok(()),\n        }\n    }\n}\n\nimpl Display for LockInfo {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:lockinfo>{}{}\", self.lock_scope, self.lock_type)?;\n\n        if let Some(owner) = &self.owner {\n            f.write_str(\"<D:owner>\")?;\n            owner.fmt(f)?;\n            f.write_str(\"</D:owner>\")?;\n        }\n\n        write!(f, \"</D:lockinfo>\",)\n    }\n}\n\nimpl Display for LockEntry {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"<D:lockentry>{}{}</D:lockentry>\",\n            self.lock_scope, self.lock_type\n        )\n    }\n}\n\nimpl Display for LockScope {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            LockScope::Exclusive => write!(f, \"<D:lockscope><D:exclusive/></D:lockscope>\"),\n            LockScope::Shared => write!(f, \"<D:lockscope><D:shared/></D:lockscope>\"),\n        }\n    }\n}\n\nimpl Display for LockType {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            LockType::Write => write!(f, \"<D:locktype><D:write/></D:locktype>\"),\n            LockType::Other => write!(f, \"<D:locktype><D:other/></D:locktype>\"),\n        }\n    }\n}\n\nimpl ActiveLock {\n    pub fn new(href: impl Into<String>, lock_scope: LockScope) -> Self {\n        Self {\n            lock_scope,\n            lock_type: LockType::Write,\n            depth: Depth::Infinity,\n            owner: None,\n            timeout: Timeout::Infinite,\n            lock_token: None,\n            lock_root: Href(href.into()),\n        }\n    }\n\n    pub fn with_depth(mut self, depth: Depth) -> Self {\n        self.depth = depth;\n        self\n    }\n\n    pub fn with_timeout(mut self, timeout: u64) -> Self {\n        self.timeout = Timeout::Second(timeout);\n        self\n    }\n\n    pub fn with_owner_opt(mut self, owner: Option<DeadProperty>) -> Self {\n        self.owner = owner;\n        self\n    }\n\n    pub fn with_owner(mut self, owner: DeadProperty) -> Self {\n        self.owner = Some(owner);\n        self\n    }\n\n    pub fn with_lock_token(mut self, token: impl Into<String>) -> Self {\n        self.lock_token = Some(Href(token.into()));\n        self\n    }\n}\n\nimpl Default for SupportedLock {\n    fn default() -> Self {\n        Self(List(vec![\n            LockEntry {\n                lock_scope: LockScope::Exclusive,\n                lock_type: LockType::Write,\n            },\n            LockEntry {\n                lock_scope: LockScope::Shared,\n                lock_type: LockType::Write,\n            },\n        ]))\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/mkcol.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse crate::schema::{\n    response::{List, MkColResponse, PropStat},\n    Namespace, Namespaces,\n};\n\nimpl Display for MkColResponse {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?>\")?;\n        if !self.mkcalendar {\n            write!(\n                f,\n                \"<D:mkcol-response {}>{}</D:mkcol-response>\",\n                self.namespaces, self.propstat\n            )\n        } else {\n            write!(\n                f,\n                \"<A:mkcalendar-response {}>{}</A:mkcalendar-response>\",\n                self.namespaces, self.propstat\n            )\n        }\n    }\n}\n\nimpl MkColResponse {\n    pub fn new(propstat: Vec<PropStat>) -> Self {\n        Self {\n            namespaces: Namespaces::default(),\n            propstat: List(propstat),\n            mkcalendar: false,\n        }\n    }\n\n    pub fn with_mkcalendar(mut self, mkcalendar: bool) -> Self {\n        self.mkcalendar = mkcalendar;\n        if mkcalendar {\n            self.namespaces.set(Namespace::CalDav);\n        }\n        self\n    }\n\n    pub fn with_namespace(mut self, namespace: Namespace) -> Self {\n        self.namespaces.set(namespace);\n        self\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod error;\npub mod lock;\npub mod mkcol;\npub mod multistatus;\npub mod property;\npub mod propstat;\npub mod schedule;\n\nuse types::dead_property::{DeadProperty, DeadPropertyTag};\n\nuse crate::schema::{\n    property::{Comp, ResourceType, SupportedCollation},\n    response::{Href, List, Location, ResponseDescription, Status, SyncToken},\n    Namespaces,\n};\nuse std::fmt::{Display, Write};\n\ntrait XmlEscape {\n    fn write_escaped_to(&self, out: &mut impl Write) -> std::fmt::Result;\n}\n\ntrait XmlCdataEscape {\n    fn write_cdata_escaped_to(&self, out: &mut impl Write) -> std::fmt::Result;\n}\n\nimpl<T: AsRef<str>> XmlEscape for T {\n    fn write_escaped_to(&self, out: &mut impl Write) -> std::fmt::Result {\n        let str = self.as_ref();\n\n        for c in str.chars() {\n            match c {\n                '<' => out.write_str(\"&lt;\")?,\n                '>' => out.write_str(\"&gt;\")?,\n                '&' => out.write_str(\"&amp;\")?,\n                '\"' => out.write_str(\"&quot;\")?,\n                '\\'' => out.write_str(\"&apos;\")?,\n                _ => out.write_char(c)?,\n            }\n        }\n\n        Ok(())\n    }\n}\n\nimpl<T: AsRef<str>> XmlCdataEscape for T {\n    fn write_cdata_escaped_to(&self, out: &mut impl Write) -> std::fmt::Result {\n        let str = self.as_ref();\n        let mut last_ch = '\\0';\n        let mut last_ch2 = '\\0';\n\n        out.write_str(\"<![CDATA[\")?;\n\n        for ch in str.chars() {\n            match ch {\n                '>' if last_ch == ']' && last_ch2 == ']' => {\n                    out.write_str(\"]]><![CDATA[>\")?;\n                }\n                _ => out.write_char(ch)?,\n            }\n\n            last_ch2 = last_ch;\n            last_ch = ch;\n        }\n\n        out.write_str(\"]]>\")\n    }\n}\n\nimpl Display for Namespaces {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(\"xmlns:D=\\\"DAV:\\\"\")?;\n        if self.cal {\n            f.write_str(\" xmlns:A=\\\"urn:ietf:params:xml:ns:caldav\\\"\")?;\n        }\n        if self.card {\n            f.write_str(\" xmlns:B=\\\"urn:ietf:params:xml:ns:carddav\\\"\")?;\n        }\n        if self.cs {\n            f.write_str(\" xmlns:C=\\\"http://calendarserver.org/ns/\\\"\")?;\n        }\n        Ok(())\n    }\n}\n\nimpl Display for Href {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:href>\")?;\n        self.0.write_escaped_to(f)?;\n        write!(f, \"</D:href>\")\n    }\n}\n\nimpl<T: Display> Display for List<T> {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        for item in &self.0 {\n            item.fmt(f)?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Display for Status {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:status>\")?;\n        write!(f, \"HTTP/1.1 {}\", self.0)?;\n        write!(f, \"</D:status>\")\n    }\n}\n\nimpl Display for ResponseDescription {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:responsedescription>\")?;\n        self.0.write_escaped_to(f)?;\n        write!(f, \"</D:responsedescription>\")\n    }\n}\n\nimpl Display for Location {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:location>\")?;\n        self.0.fmt(f)?;\n        write!(f, \"</D:location>\")\n    }\n}\n\nimpl Display for SyncToken {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:sync-token>\")?;\n        self.0.write_escaped_to(f)?;\n        write!(f, \"</D:sync-token>\")\n    }\n}\n\nimpl Display for Comp {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<A:comp name=\\\"{}\\\"/>\", self.0.as_str())\n    }\n}\n\nimpl Display for ResourceType {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            ResourceType::Collection => write!(f, \"<D:collection/>\"),\n            ResourceType::Principal => write!(f, \"<D:principal/>\"),\n            ResourceType::AddressBook => write!(f, \"<B:addressbook/>\"),\n            ResourceType::Calendar => write!(f, \"<A:calendar/>\"),\n            ResourceType::ScheduleInbox => write!(f, \"<A:schedule-inbox/>\"),\n            ResourceType::ScheduleOutbox => write!(f, \"<A:schedule-outbox/>\"),\n        }\n    }\n}\n\nimpl Display for SupportedCollation {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let ns = self.namespace.prefix();\n        write!(\n            f,\n            \"<{ns}:supported-collation>{}</{ns}:supported-collation>\",\n            self.collation.as_str()\n        )\n    }\n}\n\npub trait DeadPropertyFormat {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result;\n}\n\nimpl DeadPropertyFormat for DeadProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let mut last_tag = \"\";\n\n        for item in &self.0 {\n            match item {\n                DeadPropertyTag::ElementStart(tag) => {\n                    let name = &tag.name;\n                    if let Some(attrs) = &tag.attrs {\n                        write!(f, \"<{name} {attrs}>\")?;\n                    } else {\n                        write!(f, \"<{name}>\")?;\n                    }\n                    last_tag = name;\n                }\n                DeadPropertyTag::ElementEnd => {\n                    write!(f, \"</{}>\", last_tag)?;\n                }\n                DeadPropertyTag::Text(text) => {\n                    text.write_escaped_to(f)?;\n                }\n            }\n        }\n\n        Ok(())\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use std::fmt::Display;\n\n    use calcard::{icalendar::ICalendar, vcard::VCard};\n    use hyper::StatusCode;\n    use mail_parser::DateTime;\n    use types::dead_property::{DeadElementTag, DeadProperty, DeadPropertyTag};\n\n    use crate::{\n        parser::{tokenizer::Tokenizer, Token},\n        responses::XmlCdataEscape,\n        schema::{\n            property::{\n                ActiveLock, CalDavProperty, CardDavProperty, DavValue, LockScope, Privilege,\n                ResourceType, Rfc1123DateTime, SupportedLock, WebDavProperty,\n            },\n            request::DavPropertyValue,\n            response::{\n                Ace, AclRestrictions, BaseCondition, ErrorResponse, GrantDeny, Href, List,\n                MkColResponse, MultiStatus, Principal, PrincipalSearchProperty,\n                PrincipalSearchPropertySet, PropResponse, PropStat, RequiredPrincipal, Resource,\n                Response, ScheduleResponse, ScheduleResponseItem, SupportedPrivilege,\n            },\n            Namespace,\n        },\n        Depth,\n    };\n\n    impl<T: Display> List<T> {\n        pub fn new(vec: impl IntoIterator<Item = T>) -> Self {\n            List(vec.into_iter().collect())\n        }\n    }\n\n    impl From<ICalendar> for DavValue {\n        fn from(v: ICalendar) -> Self {\n            DavValue::ICalendar(v)\n        }\n    }\n\n    impl From<VCard> for DavValue {\n        fn from(v: VCard) -> Self {\n            DavValue::VCard(v)\n        }\n    }\n\n    #[test]\n    fn parse_responses() {\n        for (num, test) in [\n            // 001.xml\n            ErrorResponse::new(BaseCondition::LockTokenSubmitted(List::new([Href(\n                \"/locked/\".to_string(),\n            )])))\n            .to_string(),\n            // 002.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/file\",\n                vec![\n                    PropStat::new(DavPropertyValue::new(\n                        WebDavProperty::DisplayName,\n                        \"Box type A\",\n                    )),\n                    PropStat::new(DavPropertyValue::new(\n                        WebDavProperty::DisplayName,\n                        \"Box type B\",\n                    ))\n                    .with_status(StatusCode::FORBIDDEN)\n                    .with_response_description(\n                        \"The user does not have access to the DingALing property.\",\n                    ),\n                ],\n            )])\n            .with_response_description(\"There has been an access violation error.\")\n            .to_string(),\n            // 003.xml\n            MultiStatus::new(vec![\n                Response::new_propstat(\n                    \"/container/\",\n                    vec![PropStat::new_list(vec![\n                        DavPropertyValue::new(\n                            WebDavProperty::CreationDate,\n                            DateTime::parse_rfc3339(\"1997-12-01T17:42:21-08:00Z\").unwrap(),\n                        ),\n                        DavPropertyValue::new(WebDavProperty::DisplayName, \"Example collection\"),\n                        DavPropertyValue::new(\n                            WebDavProperty::ResourceType,\n                            vec![ResourceType::Collection],\n                        ),\n                        DavPropertyValue::new(\n                            WebDavProperty::SupportedLock,\n                            SupportedLock::default(),\n                        ),\n                    ])],\n                ),\n                Response::new_propstat(\n                    \"/container/front.html\",\n                    vec![PropStat::new_list(vec![\n                        DavPropertyValue::new(\n                            WebDavProperty::CreationDate,\n                            DateTime::parse_rfc3339(\"1997-12-01T18:27:21-08:00\").unwrap(),\n                        ),\n                        DavPropertyValue::new(WebDavProperty::DisplayName, \"Example HTML resource\"),\n                        DavPropertyValue::new(WebDavProperty::GetContentLength, 4525u64),\n                        DavPropertyValue::new(WebDavProperty::GetContentType, \"text/html\"),\n                        DavPropertyValue::new(WebDavProperty::GetETag, \"\\\"zzyzx\\\"\"),\n                        DavPropertyValue::new(\n                            WebDavProperty::GetLastModified,\n                            DavValue::Rfc1123Date(Rfc1123DateTime::new(\n                                DateTime::parse_rfc822(\"Mon, 12 Jan 1998 09:25:56 GMT\")\n                                    .unwrap()\n                                    .to_timestamp(),\n                            )),\n                        ),\n                        DavPropertyValue::new(WebDavProperty::ResourceType, DavValue::Null),\n                        DavPropertyValue::new(\n                            WebDavProperty::SupportedLock,\n                            SupportedLock::default(),\n                        ),\n                    ])],\n                ),\n            ])\n            .to_string(),\n            // 004.xml\n            MultiStatus::new(vec![Response::new_status(\n                [\"http://www.example.com/container/resource3\"],\n                StatusCode::LOCKED,\n            )\n            .with_error(BaseCondition::LockTokenSubmitted(List(vec![])))])\n            .to_string(),\n            // 005.xml\n            PropResponse::new(vec![DavPropertyValue::new(\n                WebDavProperty::LockDiscovery,\n                vec![ActiveLock::new(\n                    \"http://example.com/workspace/webdav/proposal.doc\",\n                    LockScope::Exclusive,\n                )\n                .with_owner(DeadProperty(vec![\n                    DeadPropertyTag::ElementStart(DeadElementTag {\n                        name: \"D:href\".to_string(),\n                        attrs: None,\n                    }),\n                    DeadPropertyTag::Text(\"http://example.org/~ejw/contact.html\".to_string()),\n                    DeadPropertyTag::ElementEnd,\n                ]))\n                .with_timeout(604800)\n                .with_lock_token(\"urn:uuid:e71d4fae-5dec-22d6-fea5-00a0c91e6be4\")],\n            )])\n            .to_string(),\n            // 006.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/container/\",\n                vec![PropStat::new_list(vec![DavPropertyValue::new(\n                    WebDavProperty::LockDiscovery,\n                    vec![\n                        ActiveLock::new(\"http://www.example.com/container/\", LockScope::Shared)\n                            .with_owner(DeadProperty(vec![DeadPropertyTag::Text(\n                                \"Jane Smith\".to_string(),\n                            )]))\n                            .with_depth(Depth::Zero)\n                            .with_lock_token(\"urn:uuid:f81de2ad-7f3d-a1b2-4f3c-00a0c91a9d76\"),\n                    ],\n                )])],\n            )])\n            .to_string(),\n            // 007.xml\n            ErrorResponse::new(BaseCondition::LockTokenSubmitted(List(vec![Href(\n                \"/workspace/webdav/\".to_string(),\n            )])))\n            .to_string(),\n            // 008.xml\n            MultiStatus::new(vec![\n                Response::new_propstat(\n                    \"http://cal.example.com/bernard/work/abcd2.ics\",\n                    vec![PropStat::new_list(vec![\n                        DavPropertyValue::new(WebDavProperty::GetETag, \"\\\"fffff-abcd2\\\"\"),\n                        DavPropertyValue::new(\n                            CalDavProperty::CalendarData(Default::default()),\n                            DavValue::CData(\n                                r#\"BEGIN:VCALENDAR\nVERSION:2.0\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060106T140000\nDURATION:PT1H\nRECURRENCE-ID;TZID=US/Eastern:20060106T120000\nSUMMARY:Event #2 bis bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#\n                                .to_string(),\n                            ),\n                        ),\n                    ])],\n                ),\n                Response::new_propstat(\n                    \"http://cal.example.com/bernard/work/abcd3.ics\",\n                    vec![PropStat::new_list(vec![\n                        DavPropertyValue::new(WebDavProperty::GetETag, \"\\\"fffff-abcd3\\\"\"),\n                        DavPropertyValue::new(\n                            CalDavProperty::CalendarData(Default::default()),\n                            DavValue::CData(\n                                r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060104T100000\nDURATION:PT1H\nSUMMARY:Event #3\nUID:DC6C50A017428C5216A2F1CD@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#\n                                .to_string(),\n                            ),\n                        ),\n                    ])],\n                ),\n            ])\n            .with_namespace(Namespace::CalDav)\n            .to_string(),\n            // 009.xml\n            MkColResponse::new(vec![PropStat::new_list(vec![\n                DavPropertyValue::new(WebDavProperty::ResourceType, DavValue::Null),\n                DavPropertyValue::new(WebDavProperty::DisplayName, DavValue::Null),\n                DavPropertyValue::new(CardDavProperty::AddressbookDescription, DavValue::Null),\n            ])])\n            .with_namespace(Namespace::CardDav)\n            .to_string(),\n            // 010.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"/home/bernard/addressbook/v102.vcf\",\n                vec![PropStat::new_list(vec![\n                    DavPropertyValue::new(WebDavProperty::GetETag, \"\\\"23ba4d-ff11fb\\\"\"),\n                    DavPropertyValue::new(\n                        CardDavProperty::AddressData(Default::default()),\n                        DavValue::CData(\n                            r#\"BEGIN:VCARD\nVERSION:3.0\nNICKNAME:me\nUID:34222-232@example.com\nFN:Cyrus Daboo\nEMAIL:daboo@example.com\nEND:VCARD\n\"#\n                            .to_string(),\n                        ),\n                    ),\n                ])],\n            )])\n            .with_namespace(Namespace::CardDav)\n            .to_string(),\n            // 011.xml\n            MultiStatus::new(vec![\n                Response::new_status(\n                    [\"/home/bernard/addressbook/\"],\n                    StatusCode::INSUFFICIENT_STORAGE,\n                )\n                .with_error(BaseCondition::NumberOfMatchesWithinLimit)\n                .with_response_description(\"Only two matching records were returned\"),\n                Response::new_propstat(\n                    \"/home/bernard/addressbook/v102.vcf\",\n                    vec![PropStat::new_list(vec![DavPropertyValue::new(\n                        WebDavProperty::GetETag,\n                        \"\\\"23ba4d-ff11fb\\\"\",\n                    )])],\n                ),\n                Response::new_propstat(\n                    \"/home/bernard/addressbook/v104.vcf\",\n                    vec![PropStat::new_list(vec![DavPropertyValue::new(\n                        WebDavProperty::GetETag,\n                        \"\\\"23ba4d-ff11fc\\\"\",\n                    )])],\n                ),\n            ])\n            .with_namespace(Namespace::CardDav)\n            .to_string(),\n            // 012.xml\n            ErrorResponse::new(BaseCondition::NeedPrivileges(List(vec![\n                Resource::new(\"/a\", Privilege::Unbind),\n                Resource::new(\"/c\", Privilege::Bind),\n            ])))\n            .to_string(),\n            // 013.xml\n            PrincipalSearchPropertySet::new(vec![\n                PrincipalSearchProperty::new(WebDavProperty::DisplayName, \"Full name\"),\n                PrincipalSearchProperty::new(WebDavProperty::DisplayName, \"Job title\"),\n            ])\n            .to_string(),\n            // 014.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/papers/\",\n                vec![PropStat::new_list(vec![DavPropertyValue::new(\n                    WebDavProperty::SupportedPrivilegeSet,\n                    vec![SupportedPrivilege::new(Privilege::All, \"Any operation\")\n                        .with_abstract()\n                        .with_supported_privilege(\n                            SupportedPrivilege::new(Privilege::Read, \"Read any object\")\n                                .with_supported_privilege(\n                                    SupportedPrivilege::new(Privilege::ReadAcl, \"Read ACL\")\n                                        .with_abstract(),\n                                )\n                                .with_supported_privilege(\n                                    SupportedPrivilege::new(\n                                        Privilege::ReadCurrentUserPrivilegeSet,\n                                        \"Read current user privilege set property\",\n                                    )\n                                    .with_abstract(),\n                                ),\n                        )\n                        .with_supported_privilege(\n                            SupportedPrivilege::new(Privilege::Write, \"Write any object\")\n                                .with_supported_privilege(\n                                    SupportedPrivilege::new(Privilege::WriteAcl, \"Write ACL\")\n                                        .with_abstract(),\n                                )\n                                .with_supported_privilege(SupportedPrivilege::new(\n                                    Privilege::WriteProperties,\n                                    \"Write properties\",\n                                ))\n                                .with_supported_privilege(SupportedPrivilege::new(\n                                    Privilege::WriteContent,\n                                    \"Write resource content\",\n                                )),\n                        )\n                        .with_supported_privilege(SupportedPrivilege::new(\n                            Privilege::Unlock,\n                            \"Unlock resource\",\n                        ))],\n                )])],\n            )])\n            .to_string(),\n            // 015.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/papers/\",\n                vec![PropStat::new_list(vec![DavPropertyValue::new(\n                    WebDavProperty::CurrentUserPrivilegeSet,\n                    vec![Privilege::Read],\n                )])],\n            )])\n            .to_string(),\n            // 016.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/papers/\",\n                vec![PropStat::new_list(vec![DavPropertyValue::new(\n                    WebDavProperty::Acl,\n                    vec![\n                        Ace::new(\n                            Principal::Href(Href(\n                                \"http://www.example.com/acl/groups/maintainers\".to_string(),\n                            )),\n                            GrantDeny::grant(vec![Privilege::Write]),\n                        ),\n                        Ace::new(Principal::All, GrantDeny::grant(vec![Privilege::Read])),\n                    ],\n                )])],\n            )])\n            .to_string(),\n            // 017.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/papers/\",\n                vec![PropStat::new_list(vec![DavPropertyValue::new(\n                    WebDavProperty::AclRestrictions,\n                    AclRestrictions::new()\n                        .with_grant_only()\n                        .with_required_principal(RequiredPrincipal::All),\n                )])],\n            )])\n            .to_string(),\n            // 018.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/papers/\",\n                vec![PropStat::new_list(vec![DavPropertyValue::new(\n                    WebDavProperty::PrincipalCollectionSet,\n                    vec![\n                        Href(\"http://www.example.com/acl/users/\".to_string()),\n                        Href(\"http://www.example.com/acl/groups/\".to_string()),\n                    ],\n                )])],\n            )])\n            .to_string(),\n            // 019.xml\n            MultiStatus::new(vec![Response::new_propstat(\n                \"http://www.example.com/top/container/\",\n                vec![PropStat::new_list(vec![\n                    DavPropertyValue::new(\n                        WebDavProperty::Owner,\n                        vec![Href(\"http://www.example.com/users/gclemm\".to_string())],\n                    ),\n                    DavPropertyValue::new(\n                        WebDavProperty::SupportedPrivilegeSet,\n                        vec![SupportedPrivilege::new(Privilege::All, \"Any operation\")\n                            .with_abstract()\n                            .with_supported_privilege(SupportedPrivilege::new(\n                                Privilege::Read,\n                                \"Read any object\",\n                            ))\n                            .with_supported_privilege(\n                                SupportedPrivilege::new(Privilege::Write, \"Write any object\")\n                                    .with_abstract(),\n                            )\n                            .with_supported_privilege(SupportedPrivilege::new(\n                                Privilege::ReadAcl,\n                                \"Read the ACL\",\n                            ))\n                            .with_supported_privilege(SupportedPrivilege::new(\n                                Privilege::WriteAcl,\n                                \"Write the ACL\",\n                            ))],\n                    ),\n                    DavPropertyValue::new(\n                        WebDavProperty::CurrentUserPrivilegeSet,\n                        vec![Privilege::Read, Privilege::ReadAcl],\n                    ),\n                    DavPropertyValue::new(\n                        WebDavProperty::Acl,\n                        vec![\n                            Ace::new(\n                                Principal::Href(Href(\n                                    \"http://www.example.com/users/esedlar\".to_string(),\n                                )),\n                                GrantDeny::grant(vec![\n                                    Privilege::Read,\n                                    Privilege::Write,\n                                    Privilege::ReadAcl,\n                                ]),\n                            ),\n                            Ace::new(\n                                Principal::Href(Href(\n                                    \"http://www.example.com/groups/mrktng\".to_string(),\n                                )),\n                                GrantDeny::deny(vec![Privilege::Read]),\n                            ),\n                            Ace::new(\n                                Principal::Property(List(vec![DavPropertyValue::new(\n                                    WebDavProperty::Owner,\n                                    DavValue::Null,\n                                )])),\n                                GrantDeny::grant(vec![Privilege::ReadAcl, Privilege::WriteAcl]),\n                            ),\n                            Ace::new(Principal::All, GrantDeny::grant(vec![Privilege::Read]))\n                                .with_inherited(\"http://www.example.com/top\"),\n                        ],\n                    ),\n                ])],\n            )])\n            .to_string(),\n            // 020.xml\n            ScheduleResponse {\n                items: List(vec![\n                    ScheduleResponseItem {\n                        recipient: Href(\"mailto:wilfredo@example.com\".to_string()),\n                        request_status: \"2.0;Success\".into(),\n                        calendar_data: Some(\"BEGIN:VCALENDAR\".to_string()),\n                    },\n                    ScheduleResponseItem {\n                        recipient: Href(\"mailto:bernard@example.net\".to_string()),\n                        request_status: \"2.0;Success\".into(),\n                        calendar_data: Some(\"END:VCALENDAR\".to_string()),\n                    },\n                    ScheduleResponseItem {\n                        recipient: Href(\"mailto:mike@example.org\".to_string()),\n                        request_status: \"3.7;Invalid calendar user\".into(),\n                        calendar_data: None,\n                    },\n                ]),\n            }\n            .to_string(),\n        ]\n        .into_iter()\n        .enumerate()\n        {\n            let xml =\n                std::fs::read_to_string(format!(\"resources/responses/{:03}.xml\", num + 1)).unwrap();\n            let mut output_token = Tokenizer::new(test.as_bytes());\n            let mut expected_token = Tokenizer::new(xml.as_bytes());\n            let mut output_tokens = Vec::new();\n            let mut expected_tokens = Vec::new();\n\n            for (tokens, tokenizer) in [\n                (&mut output_tokens, &mut output_token),\n                (&mut expected_tokens, &mut expected_token),\n            ] {\n                while let Ok(token) = tokenizer.token() {\n                    if token == Token::Eof {\n                        break;\n                    }\n                    match (tokens.last_mut(), token) {\n                        (Some(Token::Text(text)), Token::Text(new_text)) => {\n                            *text = format!(\"{}{}\", text, new_text).into();\n                        }\n                        (_, element) => {\n                            tokens.push(element.into_owned());\n                        }\n                    }\n                }\n            }\n\n            assert!(!output_tokens.is_empty());\n            assert!(!expected_tokens.is_empty());\n            assert_eq!(output_tokens.len(), expected_tokens.len());\n\n            for (output, expected) in output_tokens.iter().zip(expected_tokens.iter()) {\n                if output != expected {\n                    eprintln!(\"{test}\");\n                }\n                assert_eq!(output, expected, \"failed for {:03}.xml\", num + 1);\n            }\n        }\n    }\n\n    #[test]\n    fn escape_cdata() {\n        for (test, expected) in [\n            (\"\", \"<![CDATA[]]>\"),\n            (\"hello\", \"<![CDATA[hello]]>\"),\n            (\"hello world\", \"<![CDATA[hello world]]>\"),\n            (\"<hello>\", \"<![CDATA[<hello>]]>\"),\n            (\"&hello;\", \"<![CDATA[&hello;]]>\"),\n            (\"'hello'\", \"<![CDATA['hello']]>\"),\n            (\"\\\"hello\\\"\", \"<![CDATA[\\\"hello\\\"]]>\"),\n            (\"<>&'\\\"\", \"<![CDATA[<>&'\\\"]]>\"),\n            (\">\", \"<![CDATA[>]]>\"),\n            (\"]]>]\", \"<![CDATA[]]]]><![CDATA[>]]]>\"),\n            (\"]]>\", \"<![CDATA[]]]]><![CDATA[>]]>\"),\n            (\"hello]]>world\", \"<![CDATA[hello]]]]><![CDATA[>world]]>\"),\n            (\n                \"hello]]><nasty-xml>pure-evil</nasty-xml>\",\n                \"<![CDATA[hello]]]]><![CDATA[><nasty-xml>pure-evil</nasty-xml>]]>\",\n            ),\n        ] {\n            let mut output = String::new();\n            test.write_cdata_escaped_to(&mut output).unwrap();\n            assert_eq!(output, expected, \"failed for input: {test:?}\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/multistatus.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse hyper::StatusCode;\n\nuse crate::schema::{\n    response::{\n        Condition, Href, List, Location, MultiStatus, PropStat, Response, ResponseDescription,\n        ResponseType, Status, SyncToken,\n    },\n    Namespace, Namespaces,\n};\n\nimpl Display for MultiStatus {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><D:multistatus {}>{}\",\n            self.namespaces, self.response\n        )?;\n        if let Some(response_description) = &self.response_description {\n            write!(f, \"{response_description}\")?;\n        }\n\n        if let Some(sync_token) = &self.sync_token {\n            write!(f, \"{sync_token}\")?;\n        }\n\n        write!(f, \"</D:multistatus>\")\n    }\n}\n\nimpl Display for Response {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:response>\")?;\n        self.href.fmt(f)?;\n        self.typ.fmt(f)?;\n        if let Some(error) = &self.error {\n            error.fmt(f)?;\n        }\n        if let Some(response_description) = &self.response_description {\n            response_description.fmt(f)?;\n        }\n        if let Some(location) = &self.location {\n            location.fmt(f)?;\n        }\n        write!(f, \"</D:response>\")\n    }\n}\n\nimpl Display for ResponseType {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            ResponseType::PropStat(list) => list.fmt(f),\n            ResponseType::Status { href, status } => {\n                href.fmt(f)?;\n                status.fmt(f)\n            }\n        }\n    }\n}\n\nimpl MultiStatus {\n    pub fn new(response: Vec<Response>) -> Self {\n        MultiStatus {\n            namespaces: Namespaces::default(),\n            response: List(response),\n            response_description: None,\n            sync_token: None,\n        }\n    }\n\n    pub fn with_response(mut self, response: Response) -> Self {\n        self.response.0.push(response);\n        self\n    }\n\n    pub fn not_found(href: impl Into<String>) -> Self {\n        let mut response = Self::new(Vec::with_capacity(1));\n        response.response.0.push(\n            Response::new_status([href], StatusCode::NOT_FOUND)\n                .with_response_description(\"No resources found\"),\n        );\n        response\n    }\n\n    pub fn add_response(&mut self, response: Response) {\n        self.response.0.push(response);\n    }\n\n    pub fn with_response_description(mut self, response_description: impl Into<String>) -> Self {\n        self.response_description = Some(ResponseDescription(response_description.into()));\n        self\n    }\n\n    pub fn with_namespace(mut self, namespace: Namespace) -> Self {\n        self.namespaces.set(namespace);\n        self\n    }\n\n    pub fn set_namespace(&mut self, namespace: Namespace) {\n        self.namespaces.set(namespace);\n    }\n\n    pub fn with_sync_token(mut self, sync_token: impl Into<String>) -> Self {\n        self.sync_token = Some(SyncToken(sync_token.into()));\n        self\n    }\n\n    pub fn set_sync_token(&mut self, sync_token: impl Into<String>) {\n        self.sync_token = Some(SyncToken(sync_token.into()));\n    }\n}\n\nimpl Response {\n    pub fn new_propstat(href: impl Into<Href>, propstat: Vec<PropStat>) -> Self {\n        Response {\n            href: href.into(),\n            typ: ResponseType::PropStat(List(propstat)),\n            error: None,\n            response_description: None,\n            location: None,\n        }\n    }\n\n    pub fn new_status<T, H>(href: T, status: StatusCode) -> Self\n    where\n        T: IntoIterator<Item = H>,\n        H: Into<String>,\n    {\n        let mut href = href.into_iter().map(|h| Href(h.into()));\n        Response {\n            href: href.next().unwrap(),\n            typ: ResponseType::Status {\n                href: List(href.collect()),\n                status: Status(status),\n            },\n            error: None,\n            response_description: None,\n            location: None,\n        }\n    }\n\n    pub fn with_error(mut self, error: impl Into<Condition>) -> Self {\n        self.error = Some(error.into());\n        self\n    }\n\n    pub fn with_response_description(mut self, response_description: impl Into<String>) -> Self {\n        self.response_description = Some(ResponseDescription(response_description.into()));\n        self\n    }\n\n    pub fn with_location(mut self, location: impl Into<String>) -> Self {\n        self.location = Some(Location(Href(location.into())));\n        self\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/property.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{XmlCdataEscape, XmlEscape};\nuse crate::{\n    responses::DeadPropertyFormat,\n    schema::{\n        property::{\n            ActiveLock, CalDavProperty, CardDavProperty, Comp, DavProperty, DavValue,\n            LockDiscovery, LockEntry, PrincipalProperty, Privilege, ReportSet, ResourceType,\n            Rfc1123DateTime, SupportedCollation, SupportedLock, WebDavProperty,\n        },\n        request::DavPropertyValue,\n        response::{Ace, AclRestrictions, Href, List, PropResponse, SupportedPrivilege},\n        Namespace, Namespaces,\n    },\n};\nuse calcard::icalendar::ICalendarComponentType;\nuse mail_parser::{\n    parsers::fields::date::{DOW, MONTH},\n    DateTime,\n};\nuse std::fmt::Display;\nuse types::dead_property::DeadProperty;\n\nimpl Display for PropResponse {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?><D:prop {}>{}</D:prop>\",\n            self.namespaces, self.properties\n        )\n    }\n}\n\nimpl Display for DavPropertyValue {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let (name, attrs) = self.property.tag_name();\n\n        write!(f, \"<{}\", name)?;\n\n        if let Some(attrs) = attrs {\n            write!(f, \" {attrs}\")?;\n        }\n\n        if !matches!(self.value, DavValue::Null) {\n            write!(f, \">{}</{}>\", self.value, name)\n        } else {\n            write!(f, \"/>\")\n        }\n    }\n}\n\nimpl Display for Rfc1123DateTime {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let dt = DateTime::from_timestamp(self.0);\n        write!(\n            f,\n            \"{}, {} {} {:04} {:02}:{:02}:{:02} GMT\",\n            DOW[dt.day_of_week() as usize],\n            dt.day,\n            MONTH\n                .get(dt.month.saturating_sub(1) as usize)\n                .copied()\n                .unwrap_or_default(),\n            dt.year,\n            dt.hour,\n            dt.minute,\n            dt.second,\n        )\n    }\n}\n\nimpl Display for DavValue {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            DavValue::Timestamp(v) => {\n                let dt = DateTime::from_timestamp(*v);\n                write!(\n                    f,\n                    \"{:04}-{:02}-{:02}T{:02}:{:02}:{:02}Z\",\n                    dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second,\n                )\n            }\n            DavValue::Rfc1123Date(v) => v.fmt(f),\n            DavValue::Uint64(v) => v.fmt(f),\n            DavValue::String(v) => v.write_escaped_to(f),\n            DavValue::ResourceTypes(v) => v.fmt(f),\n            DavValue::ActiveLocks(v) => v.fmt(f),\n            DavValue::LockEntries(v) => v.fmt(f),\n            DavValue::ReportSets(v) => v.fmt(f),\n            DavValue::CData(v) => v.write_cdata_escaped_to(f),\n            DavValue::Components(v) => v.fmt(f),\n            DavValue::Collations(v) => v.fmt(f),\n            DavValue::Href(v) => v.fmt(f),\n            DavValue::PrivilegeSet(v) => v.fmt(f),\n            DavValue::Privileges(v) => v.fmt(f),\n            DavValue::Acl(v) => v.fmt(f),\n            DavValue::AclRestrictions(v) => v.fmt(f),\n            DavValue::DeadProperty(v) => v.fmt(f),\n            DavValue::SupportedAddressData => {\n                write!(\n                    f,\n                    concat!(\n                        \"<B:address-data-type content-type=\\\"text/vcard\\\" version=\\\"4.0\\\"/>\",\n                        \"<B:address-data-type content-type=\\\"text/vcard\\\" version=\\\"3.0\\\"/>\",\n                        \"<B:address-data-type content-type=\\\"text/vcard\\\" version=\\\"2.1\\\"/>\",\n                    )\n                )\n            }\n            DavValue::SupportedCalendarData => {\n                write!(\n                    f,\n                    concat!(\n                        \"<A:calendar-data-type content-type=\\\"text/calendar\\\" version=\\\"2.0\\\"/>\",\n                        \"<A:calendar-data-type content-type=\\\"text/calendar\\\" version=\\\"1.0\\\"/>\",\n                    )\n                )\n            }\n            DavValue::Response(v) => v.fmt(f),\n            DavValue::VCard(_) | DavValue::ICalendar(_) | DavValue::Null => Ok(()),\n        }\n    }\n}\n\nimpl DavValue {\n    pub fn all_calendar_components() -> Self {\n        DavValue::Components(List(vec![\n            Comp(ICalendarComponentType::VEvent),\n            Comp(ICalendarComponentType::VTodo),\n            Comp(ICalendarComponentType::VJournal),\n            Comp(ICalendarComponentType::VFreebusy),\n            Comp(ICalendarComponentType::VTimezone),\n            Comp(ICalendarComponentType::VAlarm),\n            Comp(ICalendarComponentType::Standard),\n            Comp(ICalendarComponentType::Daylight),\n            Comp(ICalendarComponentType::VAvailability),\n            Comp(ICalendarComponentType::Available),\n            Comp(ICalendarComponentType::Participant),\n            Comp(ICalendarComponentType::VLocation),\n            Comp(ICalendarComponentType::VResource),\n        ]))\n    }\n}\n\nimpl DavProperty {\n    fn tag_name(&self) -> (&str, Option<&str>) {\n        (\n            match self {\n                DavProperty::WebDav(prop) => match prop {\n                    WebDavProperty::CreationDate => \"D:creationdate\",\n                    WebDavProperty::DisplayName => \"D:displayname\",\n                    WebDavProperty::GetContentLanguage => \"D:getcontentlanguage\",\n                    WebDavProperty::GetContentLength => \"D:getcontentlength\",\n                    WebDavProperty::GetContentType => \"D:getcontenttype\",\n                    WebDavProperty::GetETag => \"D:getetag\",\n                    WebDavProperty::GetLastModified => \"D:getlastmodified\",\n                    WebDavProperty::ResourceType => \"D:resourcetype\",\n                    WebDavProperty::LockDiscovery => \"D:lockdiscovery\",\n                    WebDavProperty::SupportedLock => \"D:supportedlock\",\n                    WebDavProperty::CurrentUserPrincipal => \"D:current-user-principal\",\n                    WebDavProperty::QuotaAvailableBytes => \"D:quota-available-bytes\",\n                    WebDavProperty::QuotaUsedBytes => \"D:quota-used-bytes\",\n                    WebDavProperty::SupportedReportSet => \"D:supported-report-set\",\n                    WebDavProperty::SyncToken => \"D:sync-token\",\n                    WebDavProperty::Owner => \"D:owner\",\n                    WebDavProperty::Group => \"D:group\",\n                    WebDavProperty::SupportedPrivilegeSet => \"D:supported-privilege-set\",\n                    WebDavProperty::CurrentUserPrivilegeSet => \"D:current-user-privilege-set\",\n                    WebDavProperty::Acl => \"D:acl\",\n                    WebDavProperty::AclRestrictions => \"D:acl-restrictions\",\n                    WebDavProperty::InheritedAclSet => \"D:inherited-acl-set\",\n                    WebDavProperty::PrincipalCollectionSet => \"D:principal-collection-set\",\n                    WebDavProperty::GetCTag => \"C:getctag\",\n                },\n                DavProperty::CardDav(prop) => match prop {\n                    CardDavProperty::AddressbookDescription => \"B:addressbook-description\",\n                    CardDavProperty::SupportedAddressData => \"B:supported-address-data\",\n                    CardDavProperty::SupportedCollationSet => \"B:supported-collation-set\",\n                    CardDavProperty::MaxResourceSize => \"B:max-resource-size\",\n                    CardDavProperty::AddressData(_) => \"B:address-data\",\n                },\n                DavProperty::CalDav(prop) => match prop {\n                    CalDavProperty::CalendarDescription => \"A:calendar-description\",\n                    CalDavProperty::CalendarTimezone => \"A:calendar-timezone\",\n                    CalDavProperty::SupportedCalendarComponentSet => {\n                        \"A:supported-calendar-component-set\"\n                    }\n                    CalDavProperty::SupportedCalendarData => \"A:supported-calendar-data\",\n                    CalDavProperty::SupportedCollationSet => \"A:supported-collation-set\",\n                    CalDavProperty::MaxResourceSize => \"A:max-resource-size\",\n                    CalDavProperty::MinDateTime => \"A:min-date-time\",\n                    CalDavProperty::MaxDateTime => \"A:max-date-time\",\n                    CalDavProperty::MaxInstances => \"A:max-instances\",\n                    CalDavProperty::MaxAttendeesPerInstance => \"A:max-attendees-per-instance\",\n                    CalDavProperty::CalendarData(_) => \"A:calendar-data\",\n                    CalDavProperty::TimezoneServiceSet => \"A:timezone-service-set\",\n                    CalDavProperty::TimezoneId => \"A:calendar-timezone-id\",\n                    CalDavProperty::ScheduleDefaultCalendarURL => \"A:schedule-default-calendar-URL\",\n                    CalDavProperty::ScheduleTag => \"A:schedule-tag\",\n                    CalDavProperty::ScheduleCalendarTransp => \"A:schedule-calendar-transp\",\n                },\n                DavProperty::Principal(prop) => match prop {\n                    PrincipalProperty::AlternateURISet => \"D:alternate-URI-set\",\n                    PrincipalProperty::PrincipalURL => \"D:principal-URL\",\n                    PrincipalProperty::GroupMemberSet => \"D:group-member-set\",\n                    PrincipalProperty::GroupMembership => \"D:group-membership\",\n                    PrincipalProperty::CalendarHomeSet => \"A:calendar-home-set\",\n                    PrincipalProperty::AddressbookHomeSet => \"B:addressbook-home-set\",\n                    PrincipalProperty::PrincipalAddress => \"B:principal-address\",\n                    PrincipalProperty::CalendarUserAddressSet => \"A:calendar-user-address-set\",\n                    PrincipalProperty::CalendarUserType => \"A:calendar-user-type\",\n                    PrincipalProperty::ScheduleInboxURL => \"A:schedule-inbox-URL\",\n                    PrincipalProperty::ScheduleOutboxURL => \"A:schedule-outbox-URL\",\n                },\n                DavProperty::DeadProperty(dead) => {\n                    return (dead.name.as_str(), dead.attrs.as_deref())\n                }\n            },\n            None,\n        )\n    }\n\n    pub fn namespace(&self) -> Namespace {\n        match self {\n            DavProperty::WebDav(WebDavProperty::GetCTag) => Namespace::CalendarServer,\n            DavProperty::CardDav(_)\n            | DavProperty::Principal(PrincipalProperty::AddressbookHomeSet) => Namespace::CardDav,\n            DavProperty::CalDav(_)\n            | DavProperty::Principal(\n                PrincipalProperty::CalendarHomeSet\n                | PrincipalProperty::CalendarUserAddressSet\n                | PrincipalProperty::CalendarUserType\n                | PrincipalProperty::ScheduleInboxURL\n                | PrincipalProperty::ScheduleOutboxURL,\n            ) => Namespace::CalDav,\n            _ => Namespace::Dav,\n        }\n    }\n}\n\nimpl AsRef<str> for DavProperty {\n    fn as_ref(&self) -> &str {\n        self.tag_name().0\n    }\n}\n\nimpl Display for ReportSet {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(\"<D:supported-report><D:report>\")?;\n        match self {\n            ReportSet::SyncCollection => write!(f, \"<D:sync-collection/>\"),\n            ReportSet::ExpandProperty => write!(f, \"<D:expand-property/>\"),\n            ReportSet::AddressbookQuery => write!(f, \"<B:addressbook-query/>\"),\n            ReportSet::AddressbookMultiGet => write!(f, \"<B:addressbook-multiget/>\"),\n            ReportSet::CalendarQuery => write!(f, \"<A:calendar-query/>\"),\n            ReportSet::CalendarMultiGet => write!(f, \"<A:calendar-multiget/>\"),\n            ReportSet::FreeBusyQuery => write!(f, \"<A:free-busy-query/>\"),\n            ReportSet::AclPrincipalPropSet => write!(f, \"<D:acl-principal-prop-set/>\"),\n            ReportSet::PrincipalMatch => write!(f, \"<D:principal-match/>\"),\n            ReportSet::PrincipalPropertySearch => write!(f, \"<D:principal-property-search/>\"),\n            ReportSet::PrincipalSearchPropertySet => {\n                write!(f, \"<D:principal-search-property-set/>\")\n            }\n        }?;\n        f.write_str(\"</D:report></D:supported-report>\")\n    }\n}\n\nimpl Display for DavProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let (name, attrs) = self.tag_name();\n        if let Some(attrs) = attrs {\n            write!(f, \"<{name} {attrs}/>\")\n        } else {\n            write!(f, \"<{name}/>\")\n        }\n    }\n}\n\nimpl PropResponse {\n    pub fn new(properties: Vec<DavPropertyValue>) -> Self {\n        PropResponse {\n            namespaces: Namespaces::default(),\n            properties: List(properties),\n        }\n    }\n\n    pub fn with_namespace(mut self, namespace: Namespace) -> Self {\n        self.namespaces.set(namespace);\n        self\n    }\n}\n\nimpl From<WebDavProperty> for DavProperty {\n    fn from(prop: WebDavProperty) -> Self {\n        DavProperty::WebDav(prop)\n    }\n}\n\nimpl From<CardDavProperty> for DavProperty {\n    fn from(prop: CardDavProperty) -> Self {\n        DavProperty::CardDav(prop)\n    }\n}\n\nimpl From<CalDavProperty> for DavProperty {\n    fn from(prop: CalDavProperty) -> Self {\n        DavProperty::CalDav(prop)\n    }\n}\n\nimpl From<String> for DavValue {\n    fn from(v: String) -> Self {\n        DavValue::String(v)\n    }\n}\n\nimpl From<&str> for DavValue {\n    fn from(v: &str) -> Self {\n        DavValue::String(v.to_string())\n    }\n}\n\nimpl From<u64> for DavValue {\n    fn from(v: u64) -> Self {\n        DavValue::Uint64(v)\n    }\n}\n\nimpl From<DateTime> for DavValue {\n    fn from(v: DateTime) -> Self {\n        DavValue::Timestamp(v.to_timestamp())\n    }\n}\n\nimpl From<Vec<ResourceType>> for DavValue {\n    fn from(v: Vec<ResourceType>) -> Self {\n        DavValue::ResourceTypes(List(v))\n    }\n}\n\nimpl From<Vec<ReportSet>> for DavValue {\n    fn from(v: Vec<ReportSet>) -> Self {\n        DavValue::ReportSets(List(v))\n    }\n}\n\nimpl From<Vec<Comp>> for DavValue {\n    fn from(v: Vec<Comp>) -> Self {\n        DavValue::Components(List(v))\n    }\n}\n\nimpl From<Vec<SupportedCollation>> for DavValue {\n    fn from(v: Vec<SupportedCollation>) -> Self {\n        DavValue::Collations(List(v))\n    }\n}\n\nimpl From<SupportedLock> for DavValue {\n    fn from(v: SupportedLock) -> Self {\n        DavValue::LockEntries(v.0)\n    }\n}\n\nimpl From<Vec<LockEntry>> for DavValue {\n    fn from(v: Vec<LockEntry>) -> Self {\n        DavValue::LockEntries(List(v))\n    }\n}\n\nimpl From<Vec<ActiveLock>> for DavValue {\n    fn from(v: Vec<ActiveLock>) -> Self {\n        DavValue::ActiveLocks(List(v))\n    }\n}\n\nimpl From<LockDiscovery> for DavValue {\n    fn from(v: LockDiscovery) -> Self {\n        DavValue::ActiveLocks(v.0)\n    }\n}\n\nimpl From<Vec<SupportedPrivilege>> for DavValue {\n    fn from(v: Vec<SupportedPrivilege>) -> Self {\n        DavValue::PrivilegeSet(List(v))\n    }\n}\n\nimpl From<Vec<Privilege>> for DavValue {\n    fn from(v: Vec<Privilege>) -> Self {\n        DavValue::Privileges(List(v))\n    }\n}\n\nimpl From<Vec<Href>> for DavValue {\n    fn from(v: Vec<Href>) -> Self {\n        DavValue::Href(List(v))\n    }\n}\n\nimpl From<Vec<Ace>> for DavValue {\n    fn from(v: Vec<Ace>) -> Self {\n        DavValue::Acl(List(v))\n    }\n}\n\nimpl From<AclRestrictions> for DavValue {\n    fn from(v: AclRestrictions) -> Self {\n        DavValue::AclRestrictions(v)\n    }\n}\n\nimpl From<DeadProperty> for DavValue {\n    fn from(v: DeadProperty) -> Self {\n        DavValue::DeadProperty(v)\n    }\n}\n\nimpl DavPropertyValue {\n    pub fn new(property: impl Into<DavProperty>, value: impl Into<DavValue>) -> Self {\n        DavPropertyValue {\n            property: property.into(),\n            value: value.into(),\n        }\n    }\n\n    pub fn empty(property: impl Into<DavProperty>) -> Self {\n        DavPropertyValue {\n            property: property.into(),\n            value: DavValue::Null,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/propstat.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse hyper::StatusCode;\n\nuse crate::schema::{\n    request::DavPropertyValue,\n    response::{Condition, List, Prop, PropStat, ResponseDescription, Status},\n};\n\nimpl Display for PropStat {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:propstat>\")?;\n        self.prop.fmt(f)?;\n        self.status.fmt(f)?;\n        if let Some(error) = &self.error {\n            error.fmt(f)?;\n        }\n        if let Some(response_description) = &self.response_description {\n            response_description.fmt(f)?;\n        }\n        write!(f, \"</D:propstat>\")\n    }\n}\n\nimpl Display for Prop {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<D:prop>{}</D:prop>\", self.0)\n    }\n}\n\nimpl PropStat {\n    #[cfg(test)]\n    pub(crate) fn new(prop: impl Into<DavPropertyValue>) -> Self {\n        PropStat {\n            prop: Prop(List(vec![prop.into()])),\n            status: Status(StatusCode::OK),\n            error: None,\n            response_description: None,\n        }\n    }\n\n    pub fn new_list(props: Vec<DavPropertyValue>) -> Self {\n        PropStat {\n            prop: Prop(List(props)),\n            status: Status(StatusCode::OK),\n            error: None,\n            response_description: None,\n        }\n    }\n\n    pub fn with_prop(mut self, prop: impl Into<DavPropertyValue>) -> Self {\n        self.prop.0 .0.push(prop.into());\n        self\n    }\n\n    pub fn with_status(mut self, status: StatusCode) -> Self {\n        self.status = Status(status);\n        self\n    }\n\n    pub fn with_error(mut self, error: impl Into<Condition>) -> Self {\n        self.error = Some(error.into());\n        self\n    }\n\n    pub fn with_response_description(mut self, response_description: impl Into<String>) -> Self {\n        self.response_description = Some(ResponseDescription(response_description.into()));\n        self\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/responses/schedule.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    responses::{XmlCdataEscape, XmlEscape},\n    schema::{\n        response::{ScheduleResponse, ScheduleResponseItem},\n        Namespaces,\n    },\n};\nuse std::fmt::Display;\n\nconst NAMESPACE: Namespaces = Namespaces {\n    cal: true,\n    card: false,\n    cs: false,\n};\n\nimpl Display for ScheduleResponse {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?>\")?;\n        write!(\n            f,\n            \"<A:schedule-response {NAMESPACE}>{}</A:schedule-response>\",\n            self.items\n        )\n    }\n}\n\nimpl Display for ScheduleResponseItem {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"<A:response>\")?;\n        write!(f, \"<A:recipient>{}</A:recipient>\", self.recipient)?;\n\n        write!(f, \"<A:request-status>\")?;\n        self.request_status.write_escaped_to(f)?;\n        write!(f, \"</A:request-status>\")?;\n\n        if let Some(calendar_data) = &self.calendar_data {\n            write!(f, \"<A:calendar-data>\")?;\n            calendar_data.write_cdata_escaped_to(f)?;\n            write!(f, \"</A:calendar-data>\")?;\n        }\n        write!(f, \"</A:response>\")\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/schema/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse request::TextMatch;\npub mod property;\npub mod request;\npub mod response;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct NamedElement {\n    pub ns: Namespace,\n    pub element: Element,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[repr(u8)]\npub enum Namespace {\n    Dav,\n    CalDav,\n    CardDav,\n    CalendarServer,\n}\n\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct Namespaces {\n    pub(crate) cal: bool,\n    pub(crate) card: bool,\n    pub(crate) cs: bool,\n}\n\nimpl Namespaces {\n    pub fn set(&mut self, ns: Namespace) {\n        match ns {\n            Namespace::CalDav => self.cal = true,\n            Namespace::CardDav => self.card = true,\n            Namespace::CalendarServer => self.cs = true,\n            Namespace::Dav => {}\n        }\n    }\n}\n\nimpl Namespace {\n    pub fn try_parse(value: &[u8]) -> Option<Self> {\n        hashify::tiny_map!(value,\n            \"DAV:\" => Namespace::Dav,\n            \"urn:ietf:params:xml:ns:caldav\" => Namespace::CalDav,\n            \"urn:ietf:params:xml:ns:carddav\" => Namespace::CardDav,\n            \"http://calendarserver.org/ns/\" => Namespace::CalendarServer,\n            \"http://calendarserver.org/ns\" => Namespace::CalendarServer\n        )\n    }\n\n    pub fn prefix(&self) -> &str {\n        match self {\n            Namespace::Dav => \"D\",\n            Namespace::CalDav => \"A\",\n            Namespace::CardDav => \"B\",\n            Namespace::CalendarServer => \"C\",\n        }\n    }\n\n    pub fn namespace(&self) -> &'static str {\n        match self {\n            Namespace::Dav => \"DAV:\",\n            Namespace::CalDav => \"urn:ietf:params:xml:ns:caldav\",\n            Namespace::CardDav => \"urn:ietf:params:xml:ns:carddav\",\n            Namespace::CalendarServer => \"http://calendarserver.org/ns/\",\n        }\n    }\n}\n\nimpl AsRef<str> for Namespace {\n    fn as_ref(&self) -> &str {\n        self.namespace()\n    }\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum Element {\n    Abstract,\n    Ace,\n    Acl,\n    AclPrincipalPropSet,\n    AclRestrictions,\n    Activelock,\n    ActivityCheckoutSet,\n    ActivityCollectionSet,\n    ActivitySet,\n    ActivityVersionSet,\n    Add,\n    AddMember,\n    AddedVersion,\n    AddressData,\n    AddressDataType,\n    Addressbook,\n    AddressbookDescription,\n    AddressbookHomeSet,\n    AddressbookMultiget,\n    AddressbookQuery,\n    After,\n    All,\n    Allcomp,\n    AllowClientDefinedUri,\n    AllowedAttendeeSchedulingObjectChange,\n    AllowedOrganizerSchedulingObjectChange,\n    AllowedPrincipal,\n    Allprop,\n    AlternateUriSet,\n    And,\n    AnyOtherProperty,\n    ApplyToVersion,\n    ApplyToPrincipalCollectionSet,\n    Ascending,\n    Authenticated,\n    AutoMergeSet,\n    AutoUpdate,\n    AutoVersion,\n    Baseline,\n    BaselineCollection,\n    BaselineControl,\n    BaselineControlResponse,\n    BaselineControlledCollection,\n    BaselineControlledCollectionSet,\n    Basicsearch,\n    Basicsearchschema,\n    Before,\n    Bind,\n    BindResponse,\n    BindingName,\n    Calendar,\n    CalendarAvailability,\n    CalendarData,\n    CalendarDescription,\n    CalendarHomeSet,\n    CalendarMultiget,\n    CalendarQuery,\n    CalendarTimezone,\n    CalendarTimezoneId,\n    CalendarUserAddressSet,\n    CalendarUserType,\n    Caseless,\n    ChangedVersion,\n    CheckedIn,\n    CheckedOut,\n    Checkin,\n    CheckinActivity,\n    CheckinFork,\n    CheckinResponse,\n    Checkout,\n    CheckoutCheckin,\n    CheckoutFork,\n    CheckoutResponse,\n    CheckoutSet,\n    CheckoutUnlockedCheckin,\n    Collection,\n    Comment,\n    CommonAncestor,\n    Comp,\n    CompFilter,\n    CompareBaseline,\n    CompareBaselineReport,\n    ConflictPreview,\n    Contains,\n    Creationdate,\n    CreatorDisplayname,\n    CurrentActivitySet,\n    CurrentUserPrincipal,\n    CurrentUserPrivilegeSet,\n    CurrentWorkspaceSet,\n    Datatype,\n    DefaultCalendarNeeded,\n    DeletedVersion,\n    Deny,\n    DenyBeforeGrant,\n    Depth,\n    Descending,\n    Description,\n    Discouraged,\n    Displayname,\n    Eq,\n    Error,\n    Exclusive,\n    Expand,\n    ExpandProperty,\n    Filter,\n    First,\n    Forbidden,\n    ForkOk,\n    FreeBusyQuery,\n    From,\n    Getcontentlanguage,\n    Getcontentlength,\n    Getcontenttype,\n    Getctag,\n    Getetag,\n    Getlastmodified,\n    Grammar,\n    Grant,\n    GrantOnly,\n    Group,\n    GroupMemberSet,\n    GroupMembership,\n    Gt,\n    Gte,\n    Href,\n    IgnorePreview,\n    Include,\n    IncludeVersions,\n    Inherited,\n    InheritedAclSet,\n    Invert,\n    IsCollection,\n    IsDefined,\n    IsNotDefined,\n    KeepCheckedOut,\n    Label,\n    LabelName,\n    LabelNameSet,\n    LabelResponse,\n    LanguageDefined,\n    LanguageMatches,\n    Last,\n    LatestActivityVersion,\n    LatestActivityVersionReport,\n    Like,\n    Limit,\n    LimitFreebusySet,\n    LimitRecurrenceSet,\n    LimitedNumberOfAces,\n    Literal,\n    LocateByHistory,\n    Location,\n    LockTokenSubmitted,\n    Lockdiscovery,\n    LockedCheckout,\n    Lockentry,\n    Lockinfo,\n    Lockroot,\n    Lockscope,\n    Locktoken,\n    Locktype,\n    Lt,\n    Lte,\n    ManagedAttachmentsServerUrl,\n    Match,\n    MaxAttachmentSize,\n    MaxAttachmentsPerResource,\n    MaxAttendeesPerInstance,\n    MaxDateTime,\n    MaxInstances,\n    MaxResourceSize,\n    Merge,\n    MergePreview,\n    MergePreviewReport,\n    MergeSet,\n    MinDateTime,\n    MissingRequiredPrincipal,\n    Mkactivity,\n    MkactivityResponse,\n    Mkcalendar,\n    MkcalendarResponse,\n    Mkcol,\n    MkcolResponse,\n    Mkredirectref,\n    MkredirectrefResponse,\n    Mkworkspace,\n    MkworkspaceResponse,\n    Mount,\n    Multistatus,\n    NeedPrivileges,\n    New,\n    NoAbstract,\n    NoAceConflict,\n    NoAutoMerge,\n    NoCheckout,\n    NoConflictingLock,\n    NoInheritedAceConflict,\n    NoInvert,\n    NoProtectedAceConflict,\n    NoUidConflict,\n    Not,\n    NotSupportedPrivilege,\n    Nresults,\n    Opaque,\n    Opdesc,\n    Open,\n    OperandLiteral,\n    OperandProperty,\n    OperandTypedLiteral,\n    Operators,\n    Options,\n    OptionsResponse,\n    Or,\n    Order,\n    OrderMember,\n    Orderby,\n    OrderingType,\n    Orderpatch,\n    OrderpatchResponse,\n    Owner,\n    ParamFilter,\n    Parent,\n    ParentSet,\n    Permanent,\n    Position,\n    PredecessorSet,\n    Principal,\n    PrincipalUrl,\n    PrincipalAddress,\n    PrincipalCollectionSet,\n    PrincipalMatch,\n    PrincipalProperty,\n    PrincipalPropertySearch,\n    PrincipalSearchProperty,\n    PrincipalSearchPropertySet,\n    Privilege,\n    Prop,\n    PropFilter,\n    Propdesc,\n    Properties,\n    Property,\n    PropertySearch,\n    Propertyupdate,\n    Propfind,\n    Propname,\n    Propstat,\n    Protected,\n    QuerySchema,\n    QuerySchemaDiscovery,\n    QuotaAvailableBytes,\n    QuotaUsedBytes,\n    Read,\n    ReadAcl,\n    ReadCurrentUserPrivilegeSet,\n    ReadFreeBusy,\n    Rebind,\n    RebindResponse,\n    Recipient,\n    RecognizedPrincipal,\n    RedirectLifetime,\n    Redirectref,\n    Reftarget,\n    Remove,\n    Report,\n    RequestStatus,\n    RequiredPrincipal,\n    Resource,\n    ResourceId,\n    Resourcetype,\n    Response,\n    Responsedescription,\n    RootVersion,\n    SameOrganizerInAllComponents,\n    ScheduleCalendarTransp,\n    ScheduleDefaultCalendarUrl,\n    ScheduleDeliver,\n    ScheduleDeliverInvite,\n    ScheduleDeliverReply,\n    ScheduleInbox,\n    ScheduleInboxUrl,\n    ScheduleOutbox,\n    ScheduleOutboxUrl,\n    ScheduleQueryFreebusy,\n    ScheduleResponse,\n    ScheduleSend,\n    ScheduleSendFreebusy,\n    ScheduleSendInvite,\n    ScheduleSendReply,\n    ScheduleTag,\n    Scope,\n    Score,\n    Searchable,\n    Segment,\n    Select,\n    Selectable,\n    Self_,\n    Set,\n    Shared,\n    Sortable,\n    Source,\n    Status,\n    SubactivitySet,\n    SubbaselineSet,\n    SuccessorSet,\n    SupportedAddressData,\n    SupportedCalendarComponentSet,\n    SupportedCalendarData,\n    SupportedCollation,\n    SupportedCollationSet,\n    SupportedFilter,\n    SupportedLiveProperty,\n    SupportedLivePropertySet,\n    SupportedMethod,\n    SupportedMethodSet,\n    SupportedPrivilege,\n    SupportedPrivilegeSet,\n    SupportedQueryGrammar,\n    SupportedQueryGrammarSet,\n    SupportedReport,\n    SupportedReportSet,\n    SupportedRscale,\n    SupportedRscaleSet,\n    Supportedlock,\n    SyncCollection,\n    SyncLevel,\n    SyncToken,\n    Target,\n    Temporary,\n    TextMatch,\n    TimeRange,\n    Timeout,\n    Timezone,\n    TimezoneId,\n    TimezoneServiceSet,\n    Transparent,\n    TypedLiteral,\n    Unauthenticated,\n    Unbind,\n    UnbindResponse,\n    Uncheckout,\n    UncheckoutResponse,\n    UniqueSchedulingObjectResource,\n    Unlock,\n    Unreserved,\n    Update,\n    UpdatePreview,\n    Updateredirectref,\n    UpdateredirectrefResponse,\n    Url,\n    Username,\n    ValidOrganizer,\n    ValidScheduleDefaultCalendarUrl,\n    ValidSchedulingMessage,\n    Version,\n    VersionControl,\n    VersionControlResponse,\n    VersionControlledBinding,\n    VersionControlledBindingSet,\n    VersionControlledConfiguration,\n    VersionHistory,\n    VersionHistoryCollectionSet,\n    VersionHistorySet,\n    VersionName,\n    VersionSet,\n    VersionTree,\n    Where,\n    Workspace,\n    WorkspaceCheckoutSet,\n    WorkspaceCollectionSet,\n    Write,\n    WriteAcl,\n    WriteContent,\n    WriteProperties,\n}\n\nimpl Element {\n    pub fn try_parse(value: &[u8]) -> Option<&Self> {\n        hashify::map!(value,\n            Element,\n            \"abstract\" => Element::Abstract,\n            \"ace\" => Element::Ace,\n            \"acl\" => Element::Acl,\n            \"acl-principal-prop-set\" => Element::AclPrincipalPropSet,\n            \"acl-restrictions\" => Element::AclRestrictions,\n            \"activelock\" => Element::Activelock,\n            \"activity-checkout-set\" => Element::ActivityCheckoutSet,\n            \"activity-collection-set\" => Element::ActivityCollectionSet,\n            \"activity-set\" => Element::ActivitySet,\n            \"activity-version-set\" => Element::ActivityVersionSet,\n            \"add\" => Element::Add,\n            \"add-member\" => Element::AddMember,\n            \"added-version\" => Element::AddedVersion,\n            \"address-data\" => Element::AddressData,\n            \"address-data-type\" => Element::AddressDataType,\n            \"addressbook\" => Element::Addressbook,\n            \"addressbook-description\" => Element::AddressbookDescription,\n            \"addressbook-home-set\" => Element::AddressbookHomeSet,\n            \"addressbook-multiget\" => Element::AddressbookMultiget,\n            \"addressbook-query\" => Element::AddressbookQuery,\n            \"after\" => Element::After,\n            \"all\" => Element::All,\n            \"allcomp\" => Element::Allcomp,\n            \"allow-client-defined-uri\" => Element::AllowClientDefinedUri,\n            \"allowed-attendee-scheduling-object-change\" => Element::AllowedAttendeeSchedulingObjectChange,\n            \"allowed-organizer-scheduling-object-change\" => Element::AllowedOrganizerSchedulingObjectChange,\n            \"allowed-principal\" => Element::AllowedPrincipal,\n            \"allprop\" => Element::Allprop,\n            \"alternate-URI-set\" => Element::AlternateUriSet,\n            \"and\" => Element::And,\n            \"any-other-property\" => Element::AnyOtherProperty,\n            \"apply-to-version\" => Element::ApplyToVersion,\n            \"apply-to-principal-collection-set\" => Element::ApplyToPrincipalCollectionSet,\n            \"ascending\" => Element::Ascending,\n            \"authenticated\" => Element::Authenticated,\n            \"auto-merge-set\" => Element::AutoMergeSet,\n            \"auto-update\" => Element::AutoUpdate,\n            \"auto-version\" => Element::AutoVersion,\n            \"baseline\" => Element::Baseline,\n            \"baseline-collection\" => Element::BaselineCollection,\n            \"baseline-control\" => Element::BaselineControl,\n            \"baseline-control-response\" => Element::BaselineControlResponse,\n            \"baseline-controlled-collection\" => Element::BaselineControlledCollection,\n            \"baseline-controlled-collection-set\" => Element::BaselineControlledCollectionSet,\n            \"basicsearch\" => Element::Basicsearch,\n            \"basicsearchschema\" => Element::Basicsearchschema,\n            \"before\" => Element::Before,\n            \"bind\" => Element::Bind,\n            \"bind-response\" => Element::BindResponse,\n            \"binding-name\" => Element::BindingName,\n            \"calendar\" => Element::Calendar,\n            \"calendar-availability\" => Element::CalendarAvailability,\n            \"calendar-data\" => Element::CalendarData,\n            \"calendar-description\" => Element::CalendarDescription,\n            \"calendar-home-set\" => Element::CalendarHomeSet,\n            \"calendar-multiget\" => Element::CalendarMultiget,\n            \"calendar-query\" => Element::CalendarQuery,\n            \"calendar-timezone\" => Element::CalendarTimezone,\n            \"calendar-timezone-id\" => Element::CalendarTimezoneId,\n            \"calendar-user-address-set\" => Element::CalendarUserAddressSet,\n            \"calendar-user-type\" => Element::CalendarUserType,\n            \"caseless\" => Element::Caseless,\n            \"changed-version\" => Element::ChangedVersion,\n            \"checked-in\" => Element::CheckedIn,\n            \"checked-out\" => Element::CheckedOut,\n            \"checkin\" => Element::Checkin,\n            \"checkin-activity\" => Element::CheckinActivity,\n            \"checkin-fork\" => Element::CheckinFork,\n            \"checkin-response\" => Element::CheckinResponse,\n            \"checkout\" => Element::Checkout,\n            \"checkout-checkin\" => Element::CheckoutCheckin,\n            \"checkout-fork\" => Element::CheckoutFork,\n            \"checkout-response\" => Element::CheckoutResponse,\n            \"checkout-set\" => Element::CheckoutSet,\n            \"checkout-unlocked-checkin\" => Element::CheckoutUnlockedCheckin,\n            \"collection\" => Element::Collection,\n            \"comment\" => Element::Comment,\n            \"common-ancestor\" => Element::CommonAncestor,\n            \"comp\" => Element::Comp,\n            \"comp-filter\" => Element::CompFilter,\n            \"compare-baseline\" => Element::CompareBaseline,\n            \"compare-baseline-report\" => Element::CompareBaselineReport,\n            \"conflict-preview\" => Element::ConflictPreview,\n            \"contains\" => Element::Contains,\n            \"creationdate\" => Element::Creationdate,\n            \"creator-displayname\" => Element::CreatorDisplayname,\n            \"current-activity-set\" => Element::CurrentActivitySet,\n            \"current-user-principal\" => Element::CurrentUserPrincipal,\n            \"current-user-privilege-set\" => Element::CurrentUserPrivilegeSet,\n            \"current-workspace-set\" => Element::CurrentWorkspaceSet,\n            \"datatype\" => Element::Datatype,\n            \"default-calendar-needed\" => Element::DefaultCalendarNeeded,\n            \"deleted-version\" => Element::DeletedVersion,\n            \"deny\" => Element::Deny,\n            \"deny-before-grant\" => Element::DenyBeforeGrant,\n            \"depth\" => Element::Depth,\n            \"descending\" => Element::Descending,\n            \"description\" => Element::Description,\n            \"discouraged\" => Element::Discouraged,\n            \"displayname\" => Element::Displayname,\n            \"eq\" => Element::Eq,\n            \"error\" => Element::Error,\n            \"exclusive\" => Element::Exclusive,\n            \"expand\" => Element::Expand,\n            \"expand-property\" => Element::ExpandProperty,\n            \"filter\" => Element::Filter,\n            \"first\" => Element::First,\n            \"forbidden\" => Element::Forbidden,\n            \"fork-ok\" => Element::ForkOk,\n            \"free-busy-query\" => Element::FreeBusyQuery,\n            \"from\" => Element::From,\n            \"getcontentlanguage\" => Element::Getcontentlanguage,\n            \"getcontentlength\" => Element::Getcontentlength,\n            \"getcontenttype\" => Element::Getcontenttype,\n            \"getetag\" => Element::Getetag,\n            \"getctag\" => Element::Getctag,\n            \"getlastmodified\" => Element::Getlastmodified,\n            \"grammar\" => Element::Grammar,\n            \"grant\" => Element::Grant,\n            \"grant-only\" => Element::GrantOnly,\n            \"group\" => Element::Group,\n            \"group-member-set\" => Element::GroupMemberSet,\n            \"group-membership\" => Element::GroupMembership,\n            \"gt\" => Element::Gt,\n            \"gte\" => Element::Gte,\n            \"href\" => Element::Href,\n            \"ignore-preview\" => Element::IgnorePreview,\n            \"include\" => Element::Include,\n            \"include-versions\" => Element::IncludeVersions,\n            \"inherited\" => Element::Inherited,\n            \"inherited-acl-set\" => Element::InheritedAclSet,\n            \"invert\" => Element::Invert,\n            \"is-collection\" => Element::IsCollection,\n            \"is-defined\" => Element::IsDefined,\n            \"is-not-defined\" => Element::IsNotDefined,\n            \"keep-checked-out\" => Element::KeepCheckedOut,\n            \"label\" => Element::Label,\n            \"label-name\" => Element::LabelName,\n            \"label-name-set\" => Element::LabelNameSet,\n            \"label-response\" => Element::LabelResponse,\n            \"language-defined\" => Element::LanguageDefined,\n            \"language-matches\" => Element::LanguageMatches,\n            \"last\" => Element::Last,\n            \"latest-activity-version\" => Element::LatestActivityVersion,\n            \"latest-activity-version-report\" => Element::LatestActivityVersionReport,\n            \"like\" => Element::Like,\n            \"limit\" => Element::Limit,\n            \"limit-freebusy-set\" => Element::LimitFreebusySet,\n            \"limit-recurrence-set\" => Element::LimitRecurrenceSet,\n            \"limited-number-of-aces\" => Element::LimitedNumberOfAces,\n            \"literal\" => Element::Literal,\n            \"locate-by-history\" => Element::LocateByHistory,\n            \"location\" => Element::Location,\n            \"lock-token-submitted\" => Element::LockTokenSubmitted,\n            \"lockdiscovery\" => Element::Lockdiscovery,\n            \"locked-checkout\" => Element::LockedCheckout,\n            \"lockentry\" => Element::Lockentry,\n            \"lockinfo\" => Element::Lockinfo,\n            \"lockroot\" => Element::Lockroot,\n            \"lockscope\" => Element::Lockscope,\n            \"locktoken\" => Element::Locktoken,\n            \"locktype\" => Element::Locktype,\n            \"lt\" => Element::Lt,\n            \"lte\" => Element::Lte,\n            \"managed-attachments-server-URL\" => Element::ManagedAttachmentsServerUrl,\n            \"match\" => Element::Match,\n            \"max-attachment-size\" => Element::MaxAttachmentSize,\n            \"max-attachments-per-resource\" => Element::MaxAttachmentsPerResource,\n            \"max-attendees-per-instance\" => Element::MaxAttendeesPerInstance,\n            \"max-date-time\" => Element::MaxDateTime,\n            \"max-instances\" => Element::MaxInstances,\n            \"max-resource-size\" => Element::MaxResourceSize,\n            \"merge\" => Element::Merge,\n            \"merge-preview\" => Element::MergePreview,\n            \"merge-preview-report\" => Element::MergePreviewReport,\n            \"merge-set\" => Element::MergeSet,\n            \"min-date-time\" => Element::MinDateTime,\n            \"missing-required-principal\" => Element::MissingRequiredPrincipal,\n            \"mkactivity\" => Element::Mkactivity,\n            \"mkactivity-response\" => Element::MkactivityResponse,\n            \"mkcalendar\" => Element::Mkcalendar,\n            \"mkcalendar-response\" => Element::MkcalendarResponse,\n            \"mkcol\" => Element::Mkcol,\n            \"mkcol-response\" => Element::MkcolResponse,\n            \"mkredirectref\" => Element::Mkredirectref,\n            \"mkredirectref-response\" => Element::MkredirectrefResponse,\n            \"mkworkspace\" => Element::Mkworkspace,\n            \"mkworkspace-response\" => Element::MkworkspaceResponse,\n            \"mount\" => Element::Mount,\n            \"multistatus\" => Element::Multistatus,\n            \"need-privileges\" => Element::NeedPrivileges,\n            \"new\" => Element::New,\n            \"no-abstract\" => Element::NoAbstract,\n            \"no-ace-conflict\" => Element::NoAceConflict,\n            \"no-auto-merge\" => Element::NoAutoMerge,\n            \"no-checkout\" => Element::NoCheckout,\n            \"no-conflicting-lock\" => Element::NoConflictingLock,\n            \"no-inherited-ace-conflict\" => Element::NoInheritedAceConflict,\n            \"no-invert\" => Element::NoInvert,\n            \"no-protected-ace-conflict\" => Element::NoProtectedAceConflict,\n            \"no-uid-conflict\" => Element::NoUidConflict,\n            \"not\" => Element::Not,\n            \"not-supported-privilege\" => Element::NotSupportedPrivilege,\n            \"nresults\" => Element::Nresults,\n            \"opaque\" => Element::Opaque,\n            \"opdesc\" => Element::Opdesc,\n            \"open\" => Element::Open,\n            \"operand-literal\" => Element::OperandLiteral,\n            \"operand-property\" => Element::OperandProperty,\n            \"operand-typed-literal\" => Element::OperandTypedLiteral,\n            \"operators\" => Element::Operators,\n            \"options\" => Element::Options,\n            \"options-response\" => Element::OptionsResponse,\n            \"or\" => Element::Or,\n            \"order\" => Element::Order,\n            \"order-member\" => Element::OrderMember,\n            \"orderby\" => Element::Orderby,\n            \"ordering-type\" => Element::OrderingType,\n            \"orderpatch\" => Element::Orderpatch,\n            \"orderpatch-response\" => Element::OrderpatchResponse,\n            \"owner\" => Element::Owner,\n            \"param-filter\" => Element::ParamFilter,\n            \"parent\" => Element::Parent,\n            \"parent-set\" => Element::ParentSet,\n            \"permanent\" => Element::Permanent,\n            \"position\" => Element::Position,\n            \"predecessor-set\" => Element::PredecessorSet,\n            \"principal\" => Element::Principal,\n            \"principal-URL\" => Element::PrincipalUrl,\n            \"principal-address\" => Element::PrincipalAddress,\n            \"principal-collection-set\" => Element::PrincipalCollectionSet,\n            \"principal-match\" => Element::PrincipalMatch,\n            \"principal-property\" => Element::PrincipalProperty,\n            \"principal-property-search\" => Element::PrincipalPropertySearch,\n            \"principal-search-property\" => Element::PrincipalSearchProperty,\n            \"principal-search-property-set\" => Element::PrincipalSearchPropertySet,\n            \"privilege\" => Element::Privilege,\n            \"prop\" => Element::Prop,\n            \"prop-filter\" => Element::PropFilter,\n            \"propdesc\" => Element::Propdesc,\n            \"properties\" => Element::Properties,\n            \"property\" => Element::Property,\n            \"property-search\" => Element::PropertySearch,\n            \"propertyupdate\" => Element::Propertyupdate,\n            \"propfind\" => Element::Propfind,\n            \"propname\" => Element::Propname,\n            \"propstat\" => Element::Propstat,\n            \"protected\" => Element::Protected,\n            \"query-schema\" => Element::QuerySchema,\n            \"query-schema-discovery\" => Element::QuerySchemaDiscovery,\n            \"quota-available-bytes\" => Element::QuotaAvailableBytes,\n            \"quota-used-bytes\" => Element::QuotaUsedBytes,\n            \"read\" => Element::Read,\n            \"read-acl\" => Element::ReadAcl,\n            \"read-current-user-privilege-set\" => Element::ReadCurrentUserPrivilegeSet,\n            \"read-free-busy\" => Element::ReadFreeBusy,\n            \"rebind\" => Element::Rebind,\n            \"rebind-response\" => Element::RebindResponse,\n            \"recipient\" => Element::Recipient,\n            \"recognized-principal\" => Element::RecognizedPrincipal,\n            \"redirect-lifetime\" => Element::RedirectLifetime,\n            \"redirectref\" => Element::Redirectref,\n            \"reftarget\" => Element::Reftarget,\n            \"remove\" => Element::Remove,\n            \"report\" => Element::Report,\n            \"request-status\" => Element::RequestStatus,\n            \"required-principal\" => Element::RequiredPrincipal,\n            \"resource\" => Element::Resource,\n            \"resource-id\" => Element::ResourceId,\n            \"resourcetype\" => Element::Resourcetype,\n            \"response\" => Element::Response,\n            \"responsedescription\" => Element::Responsedescription,\n            \"root-version\" => Element::RootVersion,\n            \"same-organizer-in-all-components\" => Element::SameOrganizerInAllComponents,\n            \"schedule-calendar-transp\" => Element::ScheduleCalendarTransp,\n            \"schedule-default-calendar-URL\" => Element::ScheduleDefaultCalendarUrl,\n            \"schedule-deliver\" => Element::ScheduleDeliver,\n            \"schedule-deliver-invite\" => Element::ScheduleDeliverInvite,\n            \"schedule-deliver-reply\" => Element::ScheduleDeliverReply,\n            \"schedule-inbox\" => Element::ScheduleInbox,\n            \"schedule-inbox-URL\" => Element::ScheduleInboxUrl,\n            \"schedule-outbox\" => Element::ScheduleOutbox,\n            \"schedule-outbox-URL\" => Element::ScheduleOutboxUrl,\n            \"schedule-query-freebusy\" => Element::ScheduleQueryFreebusy,\n            \"schedule-response\" => Element::ScheduleResponse,\n            \"schedule-send\" => Element::ScheduleSend,\n            \"schedule-send-freebusy\" => Element::ScheduleSendFreebusy,\n            \"schedule-send-invite\" => Element::ScheduleSendInvite,\n            \"schedule-send-reply\" => Element::ScheduleSendReply,\n            \"schedule-tag\" => Element::ScheduleTag,\n            \"scope\" => Element::Scope,\n            \"score\" => Element::Score,\n            \"searchable\" => Element::Searchable,\n            \"segment\" => Element::Segment,\n            \"select\" => Element::Select,\n            \"selectable\" => Element::Selectable,\n            \"self\" => Element::Self_,\n            \"set\" => Element::Set,\n            \"shared\" => Element::Shared,\n            \"sortable\" => Element::Sortable,\n            \"source\" => Element::Source,\n            \"status\" => Element::Status,\n            \"subactivity-set\" => Element::SubactivitySet,\n            \"subbaseline-set\" => Element::SubbaselineSet,\n            \"successor-set\" => Element::SuccessorSet,\n            \"supported-address-data\" => Element::SupportedAddressData,\n            \"supported-calendar-component-set\" => Element::SupportedCalendarComponentSet,\n            \"supported-calendar-data\" => Element::SupportedCalendarData,\n            \"supported-collation\" => Element::SupportedCollation,\n            \"supported-collation-set\" => Element::SupportedCollationSet,\n            \"supported-filter\" => Element::SupportedFilter,\n            \"supported-live-property\" => Element::SupportedLiveProperty,\n            \"supported-live-property-set\" => Element::SupportedLivePropertySet,\n            \"supported-method\" => Element::SupportedMethod,\n            \"supported-method-set\" => Element::SupportedMethodSet,\n            \"supported-privilege\" => Element::SupportedPrivilege,\n            \"supported-privilege-set\" => Element::SupportedPrivilegeSet,\n            \"supported-query-grammar\" => Element::SupportedQueryGrammar,\n            \"supported-query-grammar-set\" => Element::SupportedQueryGrammarSet,\n            \"supported-report\" => Element::SupportedReport,\n            \"supported-report-set\" => Element::SupportedReportSet,\n            \"supported-rscale\" => Element::SupportedRscale,\n            \"supported-rscale-set\" => Element::SupportedRscaleSet,\n            \"supportedlock\" => Element::Supportedlock,\n            \"sync-collection\" => Element::SyncCollection,\n            \"sync-level\" => Element::SyncLevel,\n            \"sync-token\" => Element::SyncToken,\n            \"target\" => Element::Target,\n            \"temporary\" => Element::Temporary,\n            \"text-match\" => Element::TextMatch,\n            \"time-range\" => Element::TimeRange,\n            \"timeout\" => Element::Timeout,\n            \"timezone\" => Element::Timezone,\n            \"timezone-id\" => Element::TimezoneId,\n            \"timezone-service-set\" => Element::TimezoneServiceSet,\n            \"transparent\" => Element::Transparent,\n            \"typed-literal\" => Element::TypedLiteral,\n            \"unauthenticated\" => Element::Unauthenticated,\n            \"unbind\" => Element::Unbind,\n            \"unbind-response\" => Element::UnbindResponse,\n            \"uncheckout\" => Element::Uncheckout,\n            \"uncheckout-response\" => Element::UncheckoutResponse,\n            \"unique-scheduling-object-resource\" => Element::UniqueSchedulingObjectResource,\n            \"unlock\" => Element::Unlock,\n            \"unreserved\" => Element::Unreserved,\n            \"update\" => Element::Update,\n            \"update-preview\" => Element::UpdatePreview,\n            \"updateredirectref\" => Element::Updateredirectref,\n            \"updateredirectref-response\" => Element::UpdateredirectrefResponse,\n            \"url\" => Element::Url,\n            \"username\" => Element::Username,\n            \"valid-organizer\" => Element::ValidOrganizer,\n            \"valid-schedule-default-calendar-URL\" => Element::ValidScheduleDefaultCalendarUrl,\n            \"valid-scheduling-message\" => Element::ValidSchedulingMessage,\n            \"version\" => Element::Version,\n            \"version-control\" => Element::VersionControl,\n            \"version-control-response\" => Element::VersionControlResponse,\n            \"version-controlled-binding\" => Element::VersionControlledBinding,\n            \"version-controlled-binding-set\" => Element::VersionControlledBindingSet,\n            \"version-controlled-configuration\" => Element::VersionControlledConfiguration,\n            \"version-history\" => Element::VersionHistory,\n            \"version-history-collection-set\" => Element::VersionHistoryCollectionSet,\n            \"version-history-set\" => Element::VersionHistorySet,\n            \"version-name\" => Element::VersionName,\n            \"version-set\" => Element::VersionSet,\n            \"version-tree\" => Element::VersionTree,\n            \"where\" => Element::Where,\n            \"workspace\" => Element::Workspace,\n            \"workspace-checkout-set\" => Element::WorkspaceCheckoutSet,\n            \"workspace-collection-set\" => Element::WorkspaceCollectionSet,\n            \"write\" => Element::Write,\n            \"write-acl\" => Element::WriteAcl,\n            \"write-content\" => Element::WriteContent,\n            \"write-properties\" => Element::WriteProperties,\n        )\n    }\n}\n\nimpl AsRef<str> for Element {\n    fn as_ref(&self) -> &str {\n        match self {\n            Element::Abstract => \"abstract\",\n            Element::Ace => \"ace\",\n            Element::Acl => \"acl\",\n            Element::AclPrincipalPropSet => \"acl-principal-prop-set\",\n            Element::AclRestrictions => \"acl-restrictions\",\n            Element::Activelock => \"activelock\",\n            Element::ActivityCheckoutSet => \"activity-checkout-set\",\n            Element::ActivityCollectionSet => \"activity-collection-set\",\n            Element::ActivitySet => \"activity-set\",\n            Element::ActivityVersionSet => \"activity-version-set\",\n            Element::Add => \"add\",\n            Element::AddMember => \"add-member\",\n            Element::AddedVersion => \"added-version\",\n            Element::AddressData => \"address-data\",\n            Element::AddressDataType => \"address-data-type\",\n            Element::Addressbook => \"addressbook\",\n            Element::AddressbookDescription => \"addressbook-description\",\n            Element::AddressbookHomeSet => \"addressbook-home-set\",\n            Element::AddressbookMultiget => \"addressbook-multiget\",\n            Element::AddressbookQuery => \"addressbook-query\",\n            Element::After => \"after\",\n            Element::All => \"all\",\n            Element::Allcomp => \"allcomp\",\n            Element::AllowClientDefinedUri => \"allow-client-defined-uri\",\n            Element::AllowedAttendeeSchedulingObjectChange => {\n                \"allowed-attendee-scheduling-object-change\"\n            }\n            Element::AllowedOrganizerSchedulingObjectChange => {\n                \"allowed-organizer-scheduling-object-change\"\n            }\n            Element::AllowedPrincipal => \"allowed-principal\",\n            Element::Allprop => \"allprop\",\n            Element::AlternateUriSet => \"alternate-URI-set\",\n            Element::And => \"and\",\n            Element::AnyOtherProperty => \"any-other-property\",\n            Element::ApplyToVersion => \"apply-to-version\",\n            Element::ApplyToPrincipalCollectionSet => \"apply-to-principal-collection-set\",\n            Element::Ascending => \"ascending\",\n            Element::Authenticated => \"authenticated\",\n            Element::AutoMergeSet => \"auto-merge-set\",\n            Element::AutoUpdate => \"auto-update\",\n            Element::AutoVersion => \"auto-version\",\n            Element::Baseline => \"baseline\",\n            Element::BaselineCollection => \"baseline-collection\",\n            Element::BaselineControl => \"baseline-control\",\n            Element::BaselineControlResponse => \"baseline-control-response\",\n            Element::BaselineControlledCollection => \"baseline-controlled-collection\",\n            Element::BaselineControlledCollectionSet => \"baseline-controlled-collection-set\",\n            Element::Basicsearch => \"basicsearch\",\n            Element::Basicsearchschema => \"basicsearchschema\",\n            Element::Before => \"before\",\n            Element::Bind => \"bind\",\n            Element::BindResponse => \"bind-response\",\n            Element::BindingName => \"binding-name\",\n            Element::Calendar => \"calendar\",\n            Element::CalendarAvailability => \"calendar-availability\",\n            Element::CalendarData => \"calendar-data\",\n            Element::CalendarDescription => \"calendar-description\",\n            Element::CalendarHomeSet => \"calendar-home-set\",\n            Element::CalendarMultiget => \"calendar-multiget\",\n            Element::CalendarQuery => \"calendar-query\",\n            Element::CalendarTimezone => \"calendar-timezone\",\n            Element::CalendarTimezoneId => \"calendar-timezone-id\",\n            Element::CalendarUserAddressSet => \"calendar-user-address-set\",\n            Element::CalendarUserType => \"calendar-user-type\",\n            Element::Caseless => \"caseless\",\n            Element::ChangedVersion => \"changed-version\",\n            Element::CheckedIn => \"checked-in\",\n            Element::CheckedOut => \"checked-out\",\n            Element::Checkin => \"checkin\",\n            Element::CheckinActivity => \"checkin-activity\",\n            Element::CheckinFork => \"checkin-fork\",\n            Element::CheckinResponse => \"checkin-response\",\n            Element::Checkout => \"checkout\",\n            Element::CheckoutCheckin => \"checkout-checkin\",\n            Element::CheckoutFork => \"checkout-fork\",\n            Element::CheckoutResponse => \"checkout-response\",\n            Element::CheckoutSet => \"checkout-set\",\n            Element::CheckoutUnlockedCheckin => \"checkout-unlocked-checkin\",\n            Element::Collection => \"collection\",\n            Element::Comment => \"comment\",\n            Element::CommonAncestor => \"common-ancestor\",\n            Element::Comp => \"comp\",\n            Element::CompFilter => \"comp-filter\",\n            Element::CompareBaseline => \"compare-baseline\",\n            Element::CompareBaselineReport => \"compare-baseline-report\",\n            Element::ConflictPreview => \"conflict-preview\",\n            Element::Contains => \"contains\",\n            Element::Creationdate => \"creationdate\",\n            Element::CreatorDisplayname => \"creator-displayname\",\n            Element::CurrentActivitySet => \"current-activity-set\",\n            Element::CurrentUserPrincipal => \"current-user-principal\",\n            Element::CurrentUserPrivilegeSet => \"current-user-privilege-set\",\n            Element::CurrentWorkspaceSet => \"current-workspace-set\",\n            Element::Datatype => \"datatype\",\n            Element::DefaultCalendarNeeded => \"default-calendar-needed\",\n            Element::DeletedVersion => \"deleted-version\",\n            Element::Deny => \"deny\",\n            Element::DenyBeforeGrant => \"deny-before-grant\",\n            Element::Depth => \"depth\",\n            Element::Descending => \"descending\",\n            Element::Description => \"description\",\n            Element::Discouraged => \"discouraged\",\n            Element::Displayname => \"displayname\",\n            Element::Eq => \"eq\",\n            Element::Error => \"error\",\n            Element::Exclusive => \"exclusive\",\n            Element::Expand => \"expand\",\n            Element::ExpandProperty => \"expand-property\",\n            Element::Filter => \"filter\",\n            Element::First => \"first\",\n            Element::Forbidden => \"forbidden\",\n            Element::ForkOk => \"fork-ok\",\n            Element::FreeBusyQuery => \"free-busy-query\",\n            Element::From => \"from\",\n            Element::Getcontentlanguage => \"getcontentlanguage\",\n            Element::Getcontentlength => \"getcontentlength\",\n            Element::Getcontenttype => \"getcontenttype\",\n            Element::Getetag => \"getetag\",\n            Element::Getctag => \"getctag\",\n            Element::Getlastmodified => \"getlastmodified\",\n            Element::Grammar => \"grammar\",\n            Element::Grant => \"grant\",\n            Element::GrantOnly => \"grant-only\",\n            Element::Group => \"group\",\n            Element::GroupMemberSet => \"group-member-set\",\n            Element::GroupMembership => \"group-membership\",\n            Element::Gt => \"gt\",\n            Element::Gte => \"gte\",\n            Element::Href => \"href\",\n            Element::IgnorePreview => \"ignore-preview\",\n            Element::Include => \"include\",\n            Element::IncludeVersions => \"include-versions\",\n            Element::Inherited => \"inherited\",\n            Element::InheritedAclSet => \"inherited-acl-set\",\n            Element::Invert => \"invert\",\n            Element::IsCollection => \"is-collection\",\n            Element::IsDefined => \"is-defined\",\n            Element::IsNotDefined => \"is-not-defined\",\n            Element::KeepCheckedOut => \"keep-checked-out\",\n            Element::Label => \"label\",\n            Element::LabelName => \"label-name\",\n            Element::LabelNameSet => \"label-name-set\",\n            Element::LabelResponse => \"label-response\",\n            Element::LanguageDefined => \"language-defined\",\n            Element::LanguageMatches => \"language-matches\",\n            Element::Last => \"last\",\n            Element::LatestActivityVersion => \"latest-activity-version\",\n            Element::LatestActivityVersionReport => \"latest-activity-version-report\",\n            Element::Like => \"like\",\n            Element::Limit => \"limit\",\n            Element::LimitFreebusySet => \"limit-freebusy-set\",\n            Element::LimitRecurrenceSet => \"limit-recurrence-set\",\n            Element::LimitedNumberOfAces => \"limited-number-of-aces\",\n            Element::Literal => \"literal\",\n            Element::LocateByHistory => \"locate-by-history\",\n            Element::Location => \"location\",\n            Element::LockTokenSubmitted => \"lock-token-submitted\",\n            Element::Lockdiscovery => \"lockdiscovery\",\n            Element::LockedCheckout => \"locked-checkout\",\n            Element::Lockentry => \"lockentry\",\n            Element::Lockinfo => \"lockinfo\",\n            Element::Lockroot => \"lockroot\",\n            Element::Lockscope => \"lockscope\",\n            Element::Locktoken => \"locktoken\",\n            Element::Locktype => \"locktype\",\n            Element::Lt => \"lt\",\n            Element::Lte => \"lte\",\n            Element::ManagedAttachmentsServerUrl => \"managed-attachments-server-URL\",\n            Element::Match => \"match\",\n            Element::MaxAttachmentSize => \"max-attachment-size\",\n            Element::MaxAttachmentsPerResource => \"max-attachments-per-resource\",\n            Element::MaxAttendeesPerInstance => \"max-attendees-per-instance\",\n            Element::MaxDateTime => \"max-date-time\",\n            Element::MaxInstances => \"max-instances\",\n            Element::MaxResourceSize => \"max-resource-size\",\n            Element::Merge => \"merge\",\n            Element::MergePreview => \"merge-preview\",\n            Element::MergePreviewReport => \"merge-preview-report\",\n            Element::MergeSet => \"merge-set\",\n            Element::MinDateTime => \"min-date-time\",\n            Element::MissingRequiredPrincipal => \"missing-required-principal\",\n            Element::Mkactivity => \"mkactivity\",\n            Element::MkactivityResponse => \"mkactivity-response\",\n            Element::Mkcalendar => \"mkcalendar\",\n            Element::MkcalendarResponse => \"mkcalendar-response\",\n            Element::Mkcol => \"mkcol\",\n            Element::MkcolResponse => \"mkcol-response\",\n            Element::Mkredirectref => \"mkredirectref\",\n            Element::MkredirectrefResponse => \"mkredirectref-response\",\n            Element::Mkworkspace => \"mkworkspace\",\n            Element::MkworkspaceResponse => \"mkworkspace-response\",\n            Element::Mount => \"mount\",\n            Element::Multistatus => \"multistatus\",\n            Element::NeedPrivileges => \"need-privileges\",\n            Element::New => \"new\",\n            Element::NoAbstract => \"no-abstract\",\n            Element::NoAceConflict => \"no-ace-conflict\",\n            Element::NoAutoMerge => \"no-auto-merge\",\n            Element::NoCheckout => \"no-checkout\",\n            Element::NoConflictingLock => \"no-conflicting-lock\",\n            Element::NoInheritedAceConflict => \"no-inherited-ace-conflict\",\n            Element::NoInvert => \"no-invert\",\n            Element::NoProtectedAceConflict => \"no-protected-ace-conflict\",\n            Element::NoUidConflict => \"no-uid-conflict\",\n            Element::Not => \"not\",\n            Element::NotSupportedPrivilege => \"not-supported-privilege\",\n            Element::Nresults => \"nresults\",\n            Element::Opaque => \"opaque\",\n            Element::Opdesc => \"opdesc\",\n            Element::Open => \"open\",\n            Element::OperandLiteral => \"operand-literal\",\n            Element::OperandProperty => \"operand-property\",\n            Element::OperandTypedLiteral => \"operand-typed-literal\",\n            Element::Operators => \"operators\",\n            Element::Options => \"options\",\n            Element::OptionsResponse => \"options-response\",\n            Element::Or => \"or\",\n            Element::Order => \"order\",\n            Element::OrderMember => \"order-member\",\n            Element::Orderby => \"orderby\",\n            Element::OrderingType => \"ordering-type\",\n            Element::Orderpatch => \"orderpatch\",\n            Element::OrderpatchResponse => \"orderpatch-response\",\n            Element::Owner => \"owner\",\n            Element::ParamFilter => \"param-filter\",\n            Element::Parent => \"parent\",\n            Element::ParentSet => \"parent-set\",\n            Element::Permanent => \"permanent\",\n            Element::Position => \"position\",\n            Element::PredecessorSet => \"predecessor-set\",\n            Element::Principal => \"principal\",\n            Element::PrincipalUrl => \"principal-URL\",\n            Element::PrincipalAddress => \"principal-address\",\n            Element::PrincipalCollectionSet => \"principal-collection-set\",\n            Element::PrincipalMatch => \"principal-match\",\n            Element::PrincipalProperty => \"principal-property\",\n            Element::PrincipalPropertySearch => \"principal-property-search\",\n            Element::PrincipalSearchProperty => \"principal-search-property\",\n            Element::PrincipalSearchPropertySet => \"principal-search-property-set\",\n            Element::Privilege => \"privilege\",\n            Element::Prop => \"prop\",\n            Element::PropFilter => \"prop-filter\",\n            Element::Propdesc => \"propdesc\",\n            Element::Properties => \"properties\",\n            Element::Property => \"property\",\n            Element::PropertySearch => \"property-search\",\n            Element::Propertyupdate => \"propertyupdate\",\n            Element::Propfind => \"propfind\",\n            Element::Propname => \"propname\",\n            Element::Propstat => \"propstat\",\n            Element::Protected => \"protected\",\n            Element::QuerySchema => \"query-schema\",\n            Element::QuerySchemaDiscovery => \"query-schema-discovery\",\n            Element::QuotaAvailableBytes => \"quota-available-bytes\",\n            Element::QuotaUsedBytes => \"quota-used-bytes\",\n            Element::Read => \"read\",\n            Element::ReadAcl => \"read-acl\",\n            Element::ReadCurrentUserPrivilegeSet => \"read-current-user-privilege-set\",\n            Element::ReadFreeBusy => \"read-free-busy\",\n            Element::Rebind => \"rebind\",\n            Element::RebindResponse => \"rebind-response\",\n            Element::Recipient => \"recipient\",\n            Element::RecognizedPrincipal => \"recognized-principal\",\n            Element::RedirectLifetime => \"redirect-lifetime\",\n            Element::Redirectref => \"redirectref\",\n            Element::Reftarget => \"reftarget\",\n            Element::Remove => \"remove\",\n            Element::Report => \"report\",\n            Element::RequestStatus => \"request-status\",\n            Element::RequiredPrincipal => \"required-principal\",\n            Element::Resource => \"resource\",\n            Element::ResourceId => \"resource-id\",\n            Element::Resourcetype => \"resourcetype\",\n            Element::Response => \"response\",\n            Element::Responsedescription => \"responsedescription\",\n            Element::RootVersion => \"root-version\",\n            Element::SameOrganizerInAllComponents => \"same-organizer-in-all-components\",\n            Element::ScheduleCalendarTransp => \"schedule-calendar-transp\",\n            Element::ScheduleDefaultCalendarUrl => \"schedule-default-calendar-URL\",\n            Element::ScheduleDeliver => \"schedule-deliver\",\n            Element::ScheduleDeliverInvite => \"schedule-deliver-invite\",\n            Element::ScheduleDeliverReply => \"schedule-deliver-reply\",\n            Element::ScheduleInbox => \"schedule-inbox\",\n            Element::ScheduleInboxUrl => \"schedule-inbox-URL\",\n            Element::ScheduleOutbox => \"schedule-outbox\",\n            Element::ScheduleOutboxUrl => \"schedule-outbox-URL\",\n            Element::ScheduleQueryFreebusy => \"schedule-query-freebusy\",\n            Element::ScheduleResponse => \"schedule-response\",\n            Element::ScheduleSend => \"schedule-send\",\n            Element::ScheduleSendFreebusy => \"schedule-send-freebusy\",\n            Element::ScheduleSendInvite => \"schedule-send-invite\",\n            Element::ScheduleSendReply => \"schedule-send-reply\",\n            Element::ScheduleTag => \"schedule-tag\",\n            Element::Scope => \"scope\",\n            Element::Score => \"score\",\n            Element::Searchable => \"searchable\",\n            Element::Segment => \"segment\",\n            Element::Select => \"select\",\n            Element::Selectable => \"selectable\",\n            Element::Self_ => \"self\",\n            Element::Set => \"set\",\n            Element::Shared => \"shared\",\n            Element::Sortable => \"sortable\",\n            Element::Source => \"source\",\n            Element::Status => \"status\",\n            Element::SubactivitySet => \"subactivity-set\",\n            Element::SubbaselineSet => \"subbaseline-set\",\n            Element::SuccessorSet => \"successor-set\",\n            Element::SupportedAddressData => \"supported-address-data\",\n            Element::SupportedCalendarComponentSet => \"supported-calendar-component-set\",\n            Element::SupportedCalendarData => \"supported-calendar-data\",\n            Element::SupportedCollation => \"supported-collation\",\n            Element::SupportedCollationSet => \"supported-collation-set\",\n            Element::SupportedFilter => \"supported-filter\",\n            Element::SupportedLiveProperty => \"supported-live-property\",\n            Element::SupportedLivePropertySet => \"supported-live-property-set\",\n            Element::SupportedMethod => \"supported-method\",\n            Element::SupportedMethodSet => \"supported-method-set\",\n            Element::SupportedPrivilege => \"supported-privilege\",\n            Element::SupportedPrivilegeSet => \"supported-privilege-set\",\n            Element::SupportedQueryGrammar => \"supported-query-grammar\",\n            Element::SupportedQueryGrammarSet => \"supported-query-grammar-set\",\n            Element::SupportedReport => \"supported-report\",\n            Element::SupportedReportSet => \"supported-report-set\",\n            Element::SupportedRscale => \"supported-rscale\",\n            Element::SupportedRscaleSet => \"supported-rscale-set\",\n            Element::Supportedlock => \"supportedlock\",\n            Element::SyncCollection => \"sync-collection\",\n            Element::SyncLevel => \"sync-level\",\n            Element::SyncToken => \"sync-token\",\n            Element::Target => \"target\",\n            Element::Temporary => \"temporary\",\n            Element::TextMatch => \"text-match\",\n            Element::TimeRange => \"time-range\",\n            Element::Timeout => \"timeout\",\n            Element::Timezone => \"timezone\",\n            Element::TimezoneId => \"timezone-id\",\n            Element::TimezoneServiceSet => \"timezone-service-set\",\n            Element::Transparent => \"transparent\",\n            Element::TypedLiteral => \"typed-literal\",\n            Element::Unauthenticated => \"unauthenticated\",\n            Element::Unbind => \"unbind\",\n            Element::UnbindResponse => \"unbind-response\",\n            Element::Uncheckout => \"uncheckout\",\n            Element::UncheckoutResponse => \"uncheckout-response\",\n            Element::UniqueSchedulingObjectResource => \"unique-scheduling-object-resource\",\n            Element::Unlock => \"unlock\",\n            Element::Unreserved => \"unreserved\",\n            Element::Update => \"update\",\n            Element::UpdatePreview => \"update-preview\",\n            Element::Updateredirectref => \"updateredirectref\",\n            Element::UpdateredirectrefResponse => \"updateredirectref-response\",\n            Element::Url => \"url\",\n            Element::Username => \"username\",\n            Element::ValidOrganizer => \"valid-organizer\",\n            Element::ValidScheduleDefaultCalendarUrl => \"valid-schedule-default-calendar-URL\",\n            Element::ValidSchedulingMessage => \"valid-scheduling-message\",\n            Element::Version => \"version\",\n            Element::VersionControl => \"version-control\",\n            Element::VersionControlResponse => \"version-control-response\",\n            Element::VersionControlledBinding => \"version-controlled-binding\",\n            Element::VersionControlledBindingSet => \"version-controlled-binding-set\",\n            Element::VersionControlledConfiguration => \"version-controlled-configuration\",\n            Element::VersionHistory => \"version-history\",\n            Element::VersionHistoryCollectionSet => \"version-history-collection-set\",\n            Element::VersionHistorySet => \"version-history-set\",\n            Element::VersionName => \"version-name\",\n            Element::VersionSet => \"version-set\",\n            Element::VersionTree => \"version-tree\",\n            Element::Where => \"where\",\n            Element::Workspace => \"workspace\",\n            Element::WorkspaceCheckoutSet => \"workspace-checkout-set\",\n            Element::WorkspaceCollectionSet => \"workspace-collection-set\",\n            Element::Write => \"write\",\n            Element::WriteAcl => \"write-acl\",\n            Element::WriteContent => \"write-content\",\n            Element::WriteProperties => \"write-properties\",\n        }\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Attribute<T: AttributeValue> {\n    Caseless(bool),\n    XsiType(XsiType),\n    AllowPCData(bool),\n    Name(T),\n    Namespace(Namespace),\n    ContentType(String),\n    XmlLanguage(String),\n    Version(String),\n    NoValue(bool),\n    TestAllOf(bool),\n    MatchType(MatchType),\n    NegateCondition(bool),\n    Collation(Collation),\n    Start(T),\n    End(T),\n    Unknown { param: String, value: String },\n}\n\npub trait AttributeValue {\n    fn from_str(s: &str) -> Option<Self>\n    where\n        Self: Sized;\n}\n\nimpl<T: AttributeValue> Attribute<T> {\n    pub fn from_param(param: &[u8], value: Cow<'_, str>) -> Option<Attribute<T>> {\n        hashify::fnc_map!(param,\n            \"caseless\" => {\n                if let Some(b) = YesNo::from_str(value.as_ref()) {\n                    return Some(Attribute::Caseless(b));\n                }\n            },\n            \"xsi:type\" => {\n                return Some(Attribute::XsiType(XsiType::from_str(value.as_ref()).unwrap_or(XsiType::Unsupported)));\n            },\n            \"allow-pcdata\" => {\n                if let Some(b) = YesNo::from_str(value.as_ref()) {\n                    return Some(Attribute::AllowPCData(b));\n                }\n            },\n            \"novalue\" => {\n                if let Some(b) = YesNo::from_str(value.as_ref()) {\n                    return Some(Attribute::NoValue(b));\n                }\n            },\n            \"negate-condition\" => {\n                if let Some(b) = YesNo::from_str(value.as_ref()) {\n                    return Some(Attribute::NegateCondition(b));\n                }\n            },\n            \"name\" => {\n                if let Some(value) = T::from_str(value.as_ref()) {\n                    return Some(Attribute::Name(value));\n                }\n            },\n            \"namespace\" => {\n                if let Some(ns) = Namespace::try_parse(value.as_bytes()) {\n                    return Some(Attribute::Namespace(ns));\n                }\n            },\n            \"content-type\" => {\n                return Some(Attribute::ContentType(value.into_owned()));\n            },\n            \"version\" => {\n                return Some(Attribute::Version(value.into_owned()));\n            },\n            \"test\" => {\n                return Some(Attribute::TestAllOf(value.eq(\"allof\")));\n            },\n            \"match-type\" => {\n                if let Some(mt) = MatchType::try_parse(value.as_ref()) {\n                    return Some(Attribute::MatchType(mt));\n                }\n            },\n            \"collation\" => {\n                if let Some(c) = Collation::try_parse(value.as_ref()) {\n                    return Some(Attribute::Collation(c));\n                }\n            },\n            \"start\" => {\n                if let Some(value) = T::from_str(value.as_ref()) {\n                    return Some(Attribute::Start(value));\n                }\n            },\n            \"end\" => {\n                if let Some(value) = T::from_str(value.as_ref()) {\n                    return Some(Attribute::End(value));\n                }\n            },\n            \"xml:lang\" => {\n                return Some(Attribute::XmlLanguage(value.into_owned()));\n            },\n            \"xmlns\" => {\n                return None;\n            },\n            _ => {\n                if param.starts_with(b\"xmlns:\") {\n                    return None;\n                }\n            }\n        );\n\n        Some(Attribute::Unknown {\n            param: String::from_utf8_lossy(param).into_owned(),\n            value: value.into_owned(),\n        })\n    }\n}\n\nimpl AttributeValue for String {\n    fn from_str(s: &str) -> Option<Self> {\n        Some(s.to_string())\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum Collation {\n    AsciiNumeric,\n    AsciiCasemap,\n    Octet,\n    UnicodeCasemap,\n}\n\nimpl Collation {\n    pub fn try_parse(s: &str) -> Option<Self> {\n        hashify::tiny_map!(s.as_bytes(),\n            \"i;ascii-numeric\" => Collation::AsciiNumeric,\n            \"i;ascii-casemap\" => Collation::AsciiCasemap,\n            \"i;octet\" => Collation::Octet,\n            \"i;unicode-casemap\" => Collation::UnicodeCasemap,\n        )\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Collation::AsciiNumeric => \"i;ascii-numeric\",\n            Collation::AsciiCasemap => \"i;ascii-casemap\",\n            Collation::Octet => \"i;octet\",\n            Collation::UnicodeCasemap => \"i;unicode-casemap\",\n        }\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum MatchType {\n    Equals,\n    Contains,\n    StartsWith,\n    EndsWith,\n}\n\nimpl MatchType {\n    pub fn try_parse(s: &str) -> Option<Self> {\n        hashify::tiny_map!(s.as_bytes(),\n            \"equals\" => MatchType::Equals,\n            \"contains\" => MatchType::Contains,\n            \"starts-with\" => MatchType::StartsWith,\n            \"ends-with\" => MatchType::EndsWith,\n        )\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum XsiType {\n    String,\n    Boolean,\n    Decimal,\n    Float,\n    Double,\n    Duration,\n    DateTime,\n    Time,\n    Date,\n    GYearMonth,\n    GYear,\n    GMonthDay,\n    GDay,\n    GMonth,\n    HexBinary,\n    Base64Binary,\n    AnyUri,\n    QName,\n    Notation,\n    Unsupported,\n}\n\nimpl XsiType {\n    fn from_str(s: &str) -> Option<Self> {\n        hashify::tiny_map!(s.as_bytes(),\n            \"xs:string\" => XsiType::String,\n            \"xs:boolean\" => XsiType::Boolean,\n            \"xs:decimal\" => XsiType::Decimal,\n            \"xs:float\" => XsiType::Float,\n            \"xs:double\" => XsiType::Double,\n            \"xs:duration\" => XsiType::Duration,\n            \"xs:dateTime\" => XsiType::DateTime,\n            \"xs:time\" => XsiType::Time,\n            \"xs:date\" => XsiType::Date,\n            \"xs:gYearMonth\" => XsiType::GYearMonth,\n            \"xs:gYear\" => XsiType::GYear,\n            \"xs:gMonthDay\" => XsiType::GMonthDay,\n            \"xs:gDay\" => XsiType::GDay,\n            \"xs:gMonth\" => XsiType::GMonth,\n            \"xs:hexBinary\" => XsiType::HexBinary,\n            \"xs:base64Binary\" => XsiType::Base64Binary,\n            \"xs:anyURI\" => XsiType::AnyUri,\n            \"xs:QName\" => XsiType::QName,\n            \"xs:NOTATION\" => XsiType::Notation,\n        )\n    }\n}\n\nstruct YesNo;\n\nimpl YesNo {\n    fn from_str(s: &str) -> Option<bool> {\n        hashify::tiny_map!(s.as_bytes(),\n            \"yes\" => true,\n            \"no\" => false,\n        )\n    }\n}\n\nimpl TextMatch {\n    pub fn matches(&self, text: &str) -> bool {\n        match self.collation {\n            Collation::Octet => {\n                (match self.match_type {\n                    MatchType::Equals => text == self.value,\n                    MatchType::Contains => text.contains(&self.value),\n                    MatchType::StartsWith => text.starts_with(&self.value),\n                    MatchType::EndsWith => text.ends_with(&self.value),\n                }) ^ self.negate\n            }\n            _ => {\n                (match self.match_type {\n                    MatchType::Equals => text.to_lowercase() == self.value.to_lowercase(),\n                    MatchType::Contains => text.to_lowercase().contains(&self.value.to_lowercase()),\n                    MatchType::StartsWith => {\n                        text.to_lowercase().starts_with(&self.value.to_lowercase())\n                    }\n                    MatchType::EndsWith => {\n                        text.to_lowercase().ends_with(&self.value.to_lowercase())\n                    }\n                }) ^ self.negate\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/schema/property.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    request::DavPropertyValue,\n    response::{Ace, AclRestrictions, Href, List, Response, SupportedPrivilege},\n    Collation, Namespace,\n};\nuse crate::{Depth, Timeout};\nuse calcard::{\n    icalendar::{ICalendar, ICalendarComponentType, ICalendarProperty},\n    vcard::{VCard, VCardProperty},\n};\nuse types::{\n    dead_property::{DeadElementTag, DeadProperty},\n    TimeRange,\n};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum DavProperty {\n    WebDav(WebDavProperty),\n    CardDav(CardDavProperty),\n    CalDav(CalDavProperty),\n    Principal(PrincipalProperty),\n    DeadProperty(DeadElementTag),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum WebDavProperty {\n    CreationDate,\n    DisplayName,\n    GetContentLanguage,\n    GetContentLength,\n    GetContentType,\n    GetETag,\n    GetLastModified,\n    ResourceType,\n    LockDiscovery,\n    SupportedLock,\n    SupportedReportSet,\n    CurrentUserPrincipal,\n    // Quota properties\n    QuotaAvailableBytes,\n    QuotaUsedBytes,\n    // Sync properties\n    SyncToken,\n    // ACL properties (all protected)\n    Owner,\n    Group,\n    SupportedPrivilegeSet,\n    CurrentUserPrivilegeSet,\n    Acl,\n    AclRestrictions,\n    InheritedAclSet,\n    PrincipalCollectionSet,\n    // Apple proprietary properties\n    GetCTag,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum CardDavProperty {\n    AddressbookDescription,\n    SupportedAddressData,\n    SupportedCollationSet,\n    MaxResourceSize,\n    AddressData(Vec<CardDavPropertyName>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct CardDavPropertyName {\n    pub group: Option<String>,\n    pub name: VCardProperty,\n    pub no_value: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum CalDavProperty {\n    CalendarDescription,\n    CalendarTimezone,\n    SupportedCalendarComponentSet,\n    SupportedCalendarData,\n    SupportedCollationSet,\n    MaxResourceSize,\n    MinDateTime,\n    MaxDateTime,\n    MaxInstances,\n    MaxAttendeesPerInstance,\n    CalendarData(CalendarData),\n    TimezoneServiceSet,\n    TimezoneId,\n    ScheduleDefaultCalendarURL,\n    ScheduleTag,\n    ScheduleCalendarTransp,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum PrincipalProperty {\n    AlternateURISet,\n    PrincipalURL,\n    GroupMemberSet,\n    GroupMembership,\n    CalendarHomeSet,\n    AddressbookHomeSet,\n    PrincipalAddress,\n    CalendarUserAddressSet,\n    CalendarUserType,\n    ScheduleInboxURL,\n    ScheduleOutboxURL,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct CalendarData {\n    pub properties: Vec<CalDavPropertyName>,\n    pub expand: Option<TimeRange>,\n    pub limit_recurrence: Option<TimeRange>,\n    pub limit_freebusy: Option<TimeRange>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct CalDavPropertyName {\n    pub component: Option<ICalendarComponentType>,\n    pub name: Option<ICalendarProperty>,\n    pub no_value: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[repr(transparent)]\npub struct Rfc1123DateTime(pub(crate) i64);\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum DavValue {\n    Timestamp(i64),\n    Rfc1123Date(Rfc1123DateTime),\n    Uint64(u64),\n    String(String),\n    CData(String),\n    ResourceTypes(List<ResourceType>),\n    ActiveLocks(List<ActiveLock>),\n    LockEntries(List<LockEntry>),\n    ReportSets(List<ReportSet>),\n    ICalendar(ICalendar),\n    VCard(VCard),\n    Components(List<Comp>),\n    Collations(List<SupportedCollation>),\n    PrivilegeSet(List<SupportedPrivilege>),\n    Privileges(List<Privilege>),\n    Href(List<Href>),\n    Acl(List<Ace>),\n    AclRestrictions(AclRestrictions),\n    Response(Response),\n    DeadProperty(DeadProperty),\n    SupportedAddressData,\n    SupportedCalendarData,\n    Null,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum ReportSet {\n    SyncCollection,\n    ExpandProperty,\n    AddressbookQuery,\n    AddressbookMultiGet,\n    CalendarQuery,\n    CalendarMultiGet,\n    FreeBusyQuery,\n    AclPrincipalPropSet,\n    PrincipalMatch,\n    PrincipalPropertySearch,\n    PrincipalSearchPropertySet,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct Comp(pub ICalendarComponentType);\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct SupportedCollation {\n    pub collation: Collation,\n    pub namespace: Namespace,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum ResourceType {\n    Collection,\n    Principal,\n    AddressBook,\n    Calendar,\n    ScheduleInbox,\n    ScheduleOutbox,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct LockDiscovery(pub List<ActiveLock>);\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct ActiveLock {\n    pub lock_scope: LockScope,\n    pub lock_type: LockType,\n    pub depth: Depth,\n    pub owner: Option<DeadProperty>,\n    pub timeout: Timeout,\n    pub lock_token: Option<Href>,\n    pub lock_root: Href,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct SupportedLock(pub List<LockEntry>);\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct LockEntry {\n    pub lock_scope: LockScope,\n    pub lock_type: LockType,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum LockType {\n    Write,\n    Other,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum LockScope {\n    Exclusive,\n    Shared,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum Privilege {\n    Read,\n    Write,\n    WriteProperties,\n    WriteContent,\n    Unlock,\n    ReadAcl,\n    ReadCurrentUserPrivilegeSet,\n    WriteAcl,\n    Bind,\n    Unbind,\n    All,\n    ReadFreeBusy,\n    ScheduleDeliver,\n    ScheduleDeliverInvite,\n    ScheduleDeliverReply,\n    ScheduleQueryFreeBusy,\n    ScheduleSend,\n    ScheduleSendInvite,\n    ScheduleSendReply,\n    ScheduleSendFreeBusy,\n}\n\nimpl Privilege {\n    pub fn all(is_calendar: bool) -> Vec<Privilege> {\n        if is_calendar {\n            vec![\n                Privilege::All,\n                Privilege::Read,\n                Privilege::Write,\n                Privilege::WriteProperties,\n                Privilege::WriteContent,\n                Privilege::Unlock,\n                Privilege::ReadAcl,\n                Privilege::ReadCurrentUserPrivilegeSet,\n                Privilege::WriteAcl,\n                Privilege::Bind,\n                Privilege::Unbind,\n                Privilege::ReadFreeBusy,\n            ]\n        } else {\n            vec![\n                Privilege::All,\n                Privilege::Read,\n                Privilege::Write,\n                Privilege::WriteProperties,\n                Privilege::WriteContent,\n                Privilege::Unlock,\n                Privilege::ReadAcl,\n                Privilege::ReadCurrentUserPrivilegeSet,\n                Privilege::WriteAcl,\n                Privilege::Bind,\n                Privilege::Unbind,\n            ]\n        }\n    }\n\n    pub fn scheduling(is_inbox: bool, is_owner: bool) -> Vec<Privilege> {\n        let mut privileges = if is_inbox {\n            vec![\n                Privilege::Read,\n                Privilege::ReadCurrentUserPrivilegeSet,\n                Privilege::ScheduleDeliver,\n                Privilege::ScheduleDeliverInvite,\n                Privilege::ScheduleDeliverReply,\n                Privilege::ScheduleQueryFreeBusy,\n            ]\n        } else {\n            vec![\n                Privilege::Read,\n                Privilege::ReadCurrentUserPrivilegeSet,\n                Privilege::ScheduleSend,\n                Privilege::ScheduleSendInvite,\n                Privilege::ScheduleSendReply,\n                Privilege::ScheduleSendFreeBusy,\n            ]\n        };\n\n        if is_owner {\n            privileges.extend([\n                Privilege::All,\n                Privilege::Write,\n                Privilege::WriteProperties,\n                Privilege::WriteContent,\n                Privilege::ReadAcl,\n                Privilege::WriteAcl,\n            ]);\n        }\n\n        privileges\n    }\n}\n\nimpl From<DavProperty> for DavPropertyValue {\n    fn from(value: DavProperty) -> Self {\n        DavPropertyValue {\n            property: value,\n            value: DavValue::Null,\n        }\n    }\n}\n\nimpl Rfc1123DateTime {\n    pub fn new(timestamp: i64) -> Self {\n        Self(timestamp)\n    }\n}\n\nimpl DavProperty {\n    pub const ALL_PROPS: [DavProperty; 11] = [\n        DavProperty::WebDav(WebDavProperty::CreationDate),\n        DavProperty::WebDav(WebDavProperty::DisplayName),\n        DavProperty::WebDav(WebDavProperty::GetETag),\n        DavProperty::WebDav(WebDavProperty::GetLastModified),\n        DavProperty::WebDav(WebDavProperty::ResourceType),\n        DavProperty::WebDav(WebDavProperty::LockDiscovery),\n        DavProperty::WebDav(WebDavProperty::SupportedLock),\n        DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n        DavProperty::WebDav(WebDavProperty::GetContentLanguage),\n        DavProperty::WebDav(WebDavProperty::GetContentLength),\n        DavProperty::WebDav(WebDavProperty::GetContentType),\n    ];\n\n    pub fn is_all_prop(&self) -> bool {\n        matches!(\n            self,\n            DavProperty::WebDav(WebDavProperty::CreationDate)\n                | DavProperty::WebDav(WebDavProperty::DisplayName)\n                | DavProperty::WebDav(WebDavProperty::GetETag)\n                | DavProperty::WebDav(WebDavProperty::GetLastModified)\n                | DavProperty::WebDav(WebDavProperty::ResourceType)\n                | DavProperty::WebDav(WebDavProperty::LockDiscovery)\n                | DavProperty::WebDav(WebDavProperty::SupportedLock)\n                | DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal)\n                | DavProperty::WebDav(WebDavProperty::GetContentLanguage)\n                | DavProperty::WebDav(WebDavProperty::GetContentLength)\n                | DavProperty::WebDav(WebDavProperty::GetContentType)\n                | DavProperty::DeadProperty(_)\n        )\n    }\n}\n\nimpl ReportSet {\n    pub fn calendar() -> Vec<ReportSet> {\n        vec![\n            ReportSet::SyncCollection,\n            ReportSet::AclPrincipalPropSet,\n            ReportSet::PrincipalMatch,\n            ReportSet::ExpandProperty,\n            ReportSet::CalendarQuery,\n            ReportSet::CalendarMultiGet,\n            ReportSet::FreeBusyQuery,\n        ]\n    }\n\n    pub fn addressbook() -> Vec<ReportSet> {\n        vec![\n            ReportSet::SyncCollection,\n            ReportSet::AclPrincipalPropSet,\n            ReportSet::PrincipalMatch,\n            ReportSet::ExpandProperty,\n            ReportSet::AddressbookQuery,\n            ReportSet::AddressbookMultiGet,\n        ]\n    }\n\n    pub fn file() -> Vec<ReportSet> {\n        vec![\n            ReportSet::SyncCollection,\n            ReportSet::AclPrincipalPropSet,\n            ReportSet::PrincipalMatch,\n        ]\n    }\n\n    pub fn principal() -> Vec<ReportSet> {\n        vec![\n            ReportSet::PrincipalPropertySearch,\n            ReportSet::PrincipalSearchPropertySet,\n            ReportSet::PrincipalMatch,\n        ]\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/schema/request.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    Collation, MatchType,\n    property::{DavProperty, DavValue, LockScope, LockType},\n    response::Ace,\n};\nuse crate::{Condition, Depth};\nuse calcard::{\n    icalendar::{ICalendarComponentType, ICalendarParameterName, ICalendarProperty},\n    vcard::{VCardParameterName, VCardProperty},\n};\nuse types::{\n    TimeRange,\n    dead_property::{ArchivedDeadProperty, ArchivedDeadPropertyTag, DeadElementTag, DeadProperty},\n};\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum PropFind {\n    #[default]\n    PropName,\n    AllProp(Vec<DavProperty>),\n    Prop(Vec<DavProperty>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct PropertyUpdate {\n    pub set: Vec<DavPropertyValue>,\n    pub remove: Vec<DavProperty>,\n    pub set_first: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct DavPropertyValue {\n    pub property: DavProperty,\n    pub value: DavValue,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct MkCol {\n    pub is_mkcalendar: bool,\n    pub props: Vec<DavPropertyValue>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct LockInfo {\n    pub lock_scope: LockScope,\n    pub lock_type: LockType,\n    pub owner: Option<DeadProperty>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\"))]\npub enum Report {\n    AddressbookQuery(AddressbookQuery),\n    AddressbookMultiGet(MultiGet),\n    CalendarQuery(CalendarQuery),\n    CalendarMultiGet(MultiGet),\n    FreeBusyQuery(FreeBusyQuery),\n    SyncCollection(SyncCollection),\n    ExpandProperty(ExpandProperty),\n    AclPrincipalPropSet(AclPrincipalPropSet),\n    PrincipalMatch(PrincipalMatch),\n    PrincipalPropertySearch(PrincipalPropertySearch),\n    PrincipalSearchPropertySet,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct ExpandProperty {\n    pub properties: Vec<ExpandPropertyItem>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct ExpandPropertyItem {\n    pub property: DavProperty,\n    pub depth: u32,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct AddressbookQuery {\n    pub properties: PropFind,\n    pub filters: Vec<Filter<(), VCardPropertyWithGroup, VCardParameterName>>,\n    pub limit: Option<u32>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct VCardPropertyWithGroup {\n    pub name: VCardProperty,\n    pub group: Option<String>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct CalendarQuery {\n    pub properties: PropFind,\n    pub filters:\n        Vec<Filter<Vec<ICalendarComponentType>, ICalendarProperty, ICalendarParameterName>>,\n    pub timezone: Timezone,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\"))]\npub enum Timezone {\n    Name(String),\n    Id(String),\n    None,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct FreeBusyQuery {\n    pub range: Option<TimeRange>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct MultiGet {\n    pub properties: PropFind,\n    pub hrefs: Vec<String>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct SyncCollection {\n    pub sync_token: Option<String>,\n    pub properties: PropFind,\n    pub depth: Depth,\n    pub limit: Option<u32>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\"))]\npub enum Filter<A, B, C> {\n    AnyOf,\n    AllOf,\n    Component {\n        comp: A,\n        op: FilterOp,\n    },\n    Property {\n        comp: A,\n        prop: B,\n        op: FilterOp,\n    },\n    Parameter {\n        comp: A,\n        prop: B,\n        param: C,\n        op: FilterOp,\n    },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\", content = \"data\"))]\npub enum FilterOp {\n    Exists,\n    Undefined,\n    TimeRange(TimeRange),\n    TextMatch(TextMatch),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(test, serde(tag = \"type\"))]\npub struct TextMatch {\n    pub match_type: MatchType,\n    pub value: String,\n    pub collation: Collation,\n    pub negate: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct Acl {\n    pub aces: Vec<Ace>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct AclPrincipalPropSet {\n    pub properties: Vec<DavProperty>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct PrincipalMatch {\n    pub principal_properties: PrincipalMatchProperties,\n    pub properties: Vec<DavProperty>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum PrincipalMatchProperties {\n    Properties(Vec<DavProperty>),\n    Self_,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct PrincipalPropertySearch {\n    pub property_search: Vec<PropertySearch>,\n    pub properties: Vec<DavProperty>,\n    pub apply_to_principal_collection_set: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct PropertySearch {\n    pub property: DavProperty,\n    pub match_: String,\n}\n\nimpl PropertyUpdate {\n    pub fn has_changes(&self) -> bool {\n        !self.set.is_empty() || !self.remove.is_empty()\n    }\n}\n\nimpl FreeBusyQuery {\n    pub fn new(start: i64, end: i64) -> Self {\n        FreeBusyQuery {\n            range: Some(TimeRange { start, end }),\n        }\n    }\n}\n\npub trait DavDeadProperty {\n    fn to_dav_values(&self, output: &mut Vec<DavPropertyValue>);\n}\n\nimpl DavDeadProperty for ArchivedDeadProperty {\n    fn to_dav_values(&self, output: &mut Vec<DavPropertyValue>) {\n        let mut depth: u32 = 0;\n        let mut tags = Vec::new();\n        let mut tag_start = None;\n\n        for tag in self.0.iter() {\n            match tag {\n                ArchivedDeadPropertyTag::ElementStart(start) => {\n                    if depth == 0 {\n                        tag_start = Some(DeadElementTag::from(start));\n                    } else {\n                        tags.push(tag.into());\n                    }\n\n                    depth += 1;\n                }\n                ArchivedDeadPropertyTag::ElementEnd => {\n                    depth = depth.saturating_sub(1);\n\n                    if depth > 0 {\n                        tags.push(tag.into());\n                    } else if let Some(tag_start) = tag_start.take() {\n                        output.push(DavPropertyValue::new(\n                            DavProperty::DeadProperty(tag_start),\n                            DavValue::DeadProperty(DeadProperty(std::mem::take(&mut tags))),\n                        ));\n                    }\n                }\n                ArchivedDeadPropertyTag::Text(_) => {\n                    if tag_start.is_some() {\n                        tags.push(tag.into());\n                    }\n                }\n            }\n        }\n    }\n}\n\nimpl Condition<'_> {\n    pub fn is_none_match(&self) -> bool {\n        match self {\n            Condition::ETag { is_not, .. } | Condition::Exists { is_not } => *is_not,\n            Condition::StateToken { .. } => false,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/dav-proto/src/schema/response.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, fmt::Display};\n\nuse calcard::{\n    icalendar::{ICalendarComponentType, ICalendarParameterName, ICalendarProperty},\n    vcard::{VCardParameterName, VCardProperty},\n};\nuse hyper::StatusCode;\n\nuse super::{\n    property::{DavProperty, Privilege},\n    request::{DavPropertyValue, Filter},\n    Namespaces,\n};\n\npub struct MultiStatus {\n    pub namespaces: Namespaces,\n    pub response: List<Response>,\n    pub response_description: Option<ResponseDescription>,\n    pub sync_token: Option<SyncToken>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct Response {\n    pub href: Href,\n    pub typ: ResponseType,\n    pub error: Option<Condition>,\n    pub response_description: Option<ResponseDescription>,\n    pub location: Option<Location>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum ResponseType {\n    PropStat(List<PropStat>),\n    Status { href: List<Href>, status: Status },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[repr(transparent)]\npub struct Status(pub StatusCode);\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[repr(transparent)]\npub struct Location(pub Href);\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[repr(transparent)]\npub struct ResponseDescription(pub String);\n\n#[repr(transparent)]\npub struct SyncToken(pub String);\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash, Default)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[repr(transparent)]\npub struct Href(pub String);\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[repr(transparent)]\npub struct List<T: Display>(pub Vec<T>);\n\npub struct MkColResponse {\n    pub namespaces: Namespaces,\n    pub propstat: List<PropStat>,\n    pub mkcalendar: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct PropStat {\n    pub prop: Prop,\n    pub status: Status,\n    pub error: Option<Condition>,\n    pub response_description: Option<ResponseDescription>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\n#[repr(transparent)]\npub struct Prop(pub List<DavPropertyValue>);\n\npub struct PropResponse {\n    pub namespaces: Namespaces,\n    pub properties: List<DavPropertyValue>,\n}\n\n#[derive(Default)]\npub struct ScheduleResponse {\n    pub items: List<ScheduleResponseItem>,\n}\n\n#[derive(Default)]\npub struct ScheduleResponseItem {\n    pub recipient: Href,\n    pub request_status: Cow<'static, str>,\n    pub calendar_data: Option<String>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct SupportedPrivilege {\n    pub privilege: Privilege,\n    pub abstract_: bool,\n    pub description: String,\n    pub supported_privilege: List<SupportedPrivilege>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct Ace {\n    pub principal: Principal,\n    pub invert: bool,\n    pub grant_deny: GrantDeny,\n    pub protected: bool,\n    pub inherited: Option<Href>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum GrantDeny {\n    Grant(List<Privilege>),\n    Deny(List<Privilege>),\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum Principal {\n    Href(Href),\n    Response(Response),\n    All,\n    #[default]\n    Authenticated,\n    Unauthenticated,\n    Property(List<DavPropertyValue>),\n    Self_,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct AclRestrictions {\n    pub grant_only: bool,\n    pub no_invert: bool,\n    pub deny_before_grant: bool,\n    pub required_principal: Option<RequiredPrincipal>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum RequiredPrincipal {\n    All,\n    Authenticated,\n    Unauthenticated,\n    Self_,\n    Href(List<Href>),\n    Property(Vec<DavPropertyValue>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct PrincipalSearchPropertySet {\n    pub namespaces: Namespaces,\n    pub properties: List<PrincipalSearchProperty>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct PrincipalSearchProperty {\n    pub name: DavProperty,\n    pub description: String,\n}\n\npub struct ErrorResponse {\n    pub namespaces: Namespaces,\n    pub error: Condition,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum Condition {\n    Base(BaseCondition),\n    Cal(CalCondition),\n    Card(CardCondition),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum BaseCondition {\n    NoConflictingLock(List<Href>),\n    LockTokenSubmitted(List<Href>),\n    LockTokenMatchesRequestUri,\n    CannotModifyProtectedProperty,\n    NoExternalEntities,\n    PreservedLiveProperties,\n    PropFindFiniteDepth,\n    ResourceMustBeNull,\n    NeedPrivileges(List<Resource>),\n    NoAceConflict,\n    NoProtectedAceConflict,\n    NoInheritedAceConflict,\n    LimitedNumberOfAces,\n    DenyBeforeGrant,\n    GrantOnly,\n    NoInvert,\n    NoAbstract,\n    NotSupportedPrivilege,\n    MissingRequiredPrincipal,\n    RecognizedPrincipal,\n    AllowedPrincipal,\n    NumberOfMatchesWithinLimit,\n    QuotaNotExceeded,\n    ValidResourceType,\n    ValidSyncToken,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub struct Resource {\n    pub href: Href,\n    pub privilege: Privilege,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum CalCondition {\n    CalendarCollectionLocationOk,\n    ValidCalendarData,\n    ValidFilter,\n    ValidCalendarObjectResource,\n    ValidTimezone,\n    NoUidConflict(Href),\n    InitializeCalendarCollection,\n    SupportedCalendarData,\n    SupportedFilter(\n        Vec<Filter<Vec<ICalendarComponentType>, ICalendarProperty, ICalendarParameterName>>,\n    ),\n    SupportedCollation(String),\n    SupportedCalendarComponent,\n    MinDateTime,\n    MaxDateTime,\n    MaxResourceSize(u32),\n    MaxInstances,\n    MaxAttendeesPerInstance,\n    UniqueSchedulingObjectResource(Href),\n    SameOrganizerInAllComponents,\n    AllowedOrganizerObjectChange,\n    AllowedAttendeeObjectChange,\n    DefaultCalendarNeeded,\n    ValidScheduleDefaultCalendarUrl,\n    ValidSchedulingMessage,\n    ValidOrganizer,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[cfg_attr(test, derive(serde::Serialize, serde::Deserialize))]\npub enum CardCondition {\n    SupportedAddressData,\n    SupportedAddressDataConversion,\n    SupportedFilter(Vec<Filter<(), VCardProperty, VCardParameterName>>),\n    SupportedCollation(String),\n    ValidAddressData,\n    NoUidConflict(Href),\n    MaxResourceSize(u32),\n    AddressBookCollectionLocationOk,\n}\n\nimpl BaseCondition {\n    pub fn status(&self) -> StatusCode {\n        match self {\n            BaseCondition::NoConflictingLock(_) => StatusCode::LOCKED,\n            BaseCondition::CannotModifyProtectedProperty => StatusCode::FORBIDDEN,\n            BaseCondition::LockTokenSubmitted(_) => StatusCode::LOCKED,\n            BaseCondition::LockTokenMatchesRequestUri => StatusCode::CONFLICT,\n            BaseCondition::NoExternalEntities => StatusCode::FORBIDDEN,\n            BaseCondition::PreservedLiveProperties => StatusCode::CONFLICT,\n            BaseCondition::PropFindFiniteDepth => StatusCode::FORBIDDEN,\n            BaseCondition::ResourceMustBeNull => StatusCode::CONFLICT,\n            BaseCondition::NeedPrivileges(_) => StatusCode::FORBIDDEN,\n            BaseCondition::NumberOfMatchesWithinLimit => StatusCode::FORBIDDEN,\n            _ => StatusCode::FORBIDDEN,\n        }\n    }\n}\n\nimpl From<String> for Href {\n    fn from(value: String) -> Self {\n        Self(value)\n    }\n}\n\nimpl From<&str> for Href {\n    fn from(value: &str) -> Self {\n        Self(value.to_string())\n    }\n}\n\nimpl MultiStatus {\n    pub fn is_empty(&self) -> bool {\n        self.response.0.is_empty()\n    }\n}\n\nimpl BaseCondition {\n    pub fn display_name(&self) -> &'static str {\n        match self {\n            BaseCondition::NoConflictingLock(_) => \"NoConflictingLock\",\n            BaseCondition::CannotModifyProtectedProperty => \"CannotModifyProtectedProperty\",\n            BaseCondition::LockTokenSubmitted(_) => \"LockTokenSubmitted\",\n            BaseCondition::LockTokenMatchesRequestUri => \"LockTokenMatchesRequestUri\",\n            BaseCondition::NoExternalEntities => \"NoExternalEntities\",\n            BaseCondition::PreservedLiveProperties => \"PreservedLiveProperties\",\n            BaseCondition::PropFindFiniteDepth => \"PropFindFiniteDepth\",\n            BaseCondition::ResourceMustBeNull => \"ResourceMustBeNull\",\n            BaseCondition::NeedPrivileges(_) => \"NeedPrivileges\",\n            BaseCondition::NoAceConflict => \"NoAceConflict\",\n            BaseCondition::NoProtectedAceConflict => \"NoProtectedAceConflict\",\n            BaseCondition::NoInheritedAceConflict => \"NoInheritedAceConflict\",\n            BaseCondition::LimitedNumberOfAces => \"LimitedNumberOfAces\",\n            BaseCondition::DenyBeforeGrant => \"DenyBeforeGrant\",\n            BaseCondition::GrantOnly => \"GrantOnly\",\n            BaseCondition::NoInvert => \"NoInvert\",\n            BaseCondition::NoAbstract => \"NoAbstract\",\n            BaseCondition::NotSupportedPrivilege => \"NotSupportedPrivilege\",\n            BaseCondition::MissingRequiredPrincipal => \"MissingRequiredPrincipal\",\n            BaseCondition::RecognizedPrincipal => \"RecognizedPrincipal\",\n            BaseCondition::AllowedPrincipal => \"AllowedPrincipal\",\n            BaseCondition::NumberOfMatchesWithinLimit => \"NumberOfMatchesWithinLimit\",\n            BaseCondition::QuotaNotExceeded => \"QuotaNotExceeded\",\n            BaseCondition::ValidResourceType => \"ValidResourceType\",\n            BaseCondition::ValidSyncToken => \"ValidSyncToken\",\n        }\n    }\n}\n\nimpl CalCondition {\n    pub fn display_name(&self) -> &'static str {\n        match self {\n            CalCondition::CalendarCollectionLocationOk => \"CalendarCollectionLocationOk\",\n            CalCondition::ValidCalendarData => \"ValidCalendarData\",\n            CalCondition::ValidFilter => \"ValidFilter\",\n            CalCondition::ValidCalendarObjectResource => \"ValidCalendarObjectResource\",\n            CalCondition::ValidTimezone => \"ValidTimezone\",\n            CalCondition::NoUidConflict(_) => \"NoUidConflict\",\n            CalCondition::InitializeCalendarCollection => \"InitializeCalendarCollection\",\n            CalCondition::SupportedCalendarData => \"SupportedCalendarData\",\n            CalCondition::SupportedFilter(_) => \"SupportedFilter\",\n            CalCondition::SupportedCollation(_) => \"SupportedCollation\",\n            CalCondition::MinDateTime => \"MinDateTime\",\n            CalCondition::MaxDateTime => \"MaxDateTime\",\n            CalCondition::MaxResourceSize(_) => \"MaxResourceSize\",\n            CalCondition::MaxInstances => \"MaxInstances\",\n            CalCondition::MaxAttendeesPerInstance => \"MaxAttendeesPerInstance\",\n            CalCondition::UniqueSchedulingObjectResource(_) => \"UniqueSchedulingObjectResource\",\n            CalCondition::SameOrganizerInAllComponents => \"SameOrganizerInAllComponents\",\n            CalCondition::AllowedOrganizerObjectChange => \"AllowedOrganizerObjectChange\",\n            CalCondition::AllowedAttendeeObjectChange => \"AllowedAttendeeObjectChange\",\n            CalCondition::DefaultCalendarNeeded => \"DefaultCalendarNeeded\",\n            CalCondition::ValidScheduleDefaultCalendarUrl => \"ValidScheduleDefaultCalendarUrl\",\n            CalCondition::ValidSchedulingMessage => \"ValidSchedulingMessage\",\n            CalCondition::ValidOrganizer => \"ValidOrganizer\",\n            CalCondition::SupportedCalendarComponent => \"SupportedCalendarComponent\",\n        }\n    }\n}\n\nimpl CardCondition {\n    pub fn display_name(&self) -> &'static str {\n        match self {\n            CardCondition::SupportedAddressData => \"SupportedAddressData\",\n            CardCondition::SupportedAddressDataConversion => \"SupportedAddressDataConversion\",\n            CardCondition::SupportedFilter(_) => \"SupportedFilter\",\n            CardCondition::SupportedCollation(_) => \"SupportedCollation\",\n            CardCondition::ValidAddressData => \"ValidAddressData\",\n            CardCondition::NoUidConflict(_) => \"NoUidConflict\",\n            CardCondition::MaxResourceSize(_) => \"MaxResourceSize\",\n            CardCondition::AddressBookCollectionLocationOk => \"AddressBookCollectionLocationOk\",\n        }\n    }\n}\n\nimpl Condition {\n    pub fn display_name(&self) -> &'static str {\n        match self {\n            Condition::Base(base) => base.display_name(),\n            Condition::Cal(cal) => cal.display_name(),\n            Condition::Card(card) => card.display_name(),\n        }\n    }\n}\n\n#[cfg(test)]\nmod serde_impl {\n    use super::Status;\n    use hyper::StatusCode;\n    use serde::{Deserialize, Deserializer, Serialize, Serializer};\n\n    impl Serialize for Status {\n        fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n        where\n            S: Serializer,\n        {\n            // Serialize the status code as a u16\n            serializer.serialize_u16(self.0.as_u16())\n        }\n    }\n\n    impl<'de> Deserialize<'de> for Status {\n        fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n        where\n            D: Deserializer<'de>,\n        {\n            // Deserialize as u16\n            let status_value = u16::deserialize(deserializer)?;\n\n            // Convert u16 to StatusCode\n            let status_code = StatusCode::try_from(status_value).map_err(|_| {\n                serde::de::Error::custom(format!(\"Invalid status code: {}\", status_value))\n            })?;\n\n            Ok(Status(status_code))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/Cargo.toml",
    "content": "[package]\nname = \"directory\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path =  \"../utils\" }\nproc_macros = { path =  \"../utils/proc-macros\" }\nstore = { path =  \"../store\" }\ntrc = { path = \"../trc\" }\nnlp = { path = \"../nlp\" }\ntypes = { path = \"../types\" }\nsmtp-proto = { version = \"0.2\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] } \nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nmail-builder = { version = \"0.4\" }\ntokio = { version = \"1.47\", features = [\"net\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pki-types = { version = \"1\" }\nldap3 = { version = \"0.12\", default-features = false, features = [\"tls-rustls-ring\"] }\ndeadpool = { version = \"0.10\", features = [\"managed\", \"rt_tokio_1\"] }\nasync-trait = \"0.1.68\"\nahash = { version = \"0.8\" }\npwhash = \"1\"\npassword-hash = \"0.5.0\"\nargon2 = \"0.5.0\"\npbkdf2 = {version = \"0.12.1\", features = [\"simple\"] }\nscrypt = \"0.11.0\"\nsha1 = \"0.10.5\"\nsha2 = \"0.10.6\"\nmd5 = \"0.8.0\"\nfutures = \"0.3\"\nregex = \"1.7.0\"\nserde = { version = \"1.0\", features = [\"derive\"]}\ntotp-rs = { version = \"5.5.1\", features = [\"otpauth\"] }\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\"] }\nserde_json = \"1.0\"\nbase64 = \"0.22\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = { version = \"0.9.0\", features = [\"rkyv\", \"serde\"] }\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n\n[features]\ntest_mode = []\nenterprise = []\n"
  },
  {
    "path": "crates/directory/src/backend/imap/client.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mail_send::Credentials;\nuse smtp_proto::{\n    AUTH_OAUTHBEARER, AUTH_PLAIN, AUTH_XOAUTH2, IntoString,\n    request::{AUTH, parser::Rfc5321Parser},\n    response::generate::BitToString,\n};\nuse tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt};\n\nuse super::{ImapClient, ImapError};\n\nimpl<T: AsyncRead + AsyncWrite + Unpin> ImapClient<T> {\n    pub async fn authenticate(\n        &mut self,\n        mechanism: u64,\n        credentials: &Credentials<String>,\n    ) -> Result<(), ImapError> {\n        if (mechanism & (AUTH_PLAIN | AUTH_XOAUTH2 | AUTH_OAUTHBEARER)) != 0 {\n            self.write(\n                format!(\n                    \"C3 AUTHENTICATE {} {}\\r\\n\",\n                    mechanism.to_mechanism(),\n                    credentials\n                        .encode(mechanism, \"\")\n                        .map_err(|err| ImapError::InvalidChallenge(err.to_string()))?\n                )\n                .as_bytes(),\n            )\n            .await?;\n        } else {\n            self.write(format!(\"C3 AUTHENTICATE {}\\r\\n\", mechanism.to_mechanism()).as_bytes())\n                .await?;\n        }\n        let mut line = self.read_line().await?;\n\n        for _ in 0..3 {\n            if matches!(line.first(), Some(b'+')) {\n                self.write(\n                    format!(\n                        \"{}\\r\\n\",\n                        credentials\n                            .encode(\n                                mechanism,\n                                std::str::from_utf8(line.get(2..).unwrap_or_default())\n                                    .unwrap_or_default()\n                            )\n                            .map_err(|err| ImapError::InvalidChallenge(err.to_string()))?\n                    )\n                    .as_bytes(),\n                )\n                .await?;\n                line = self.read_line().await?;\n            } else if matches!(line.get(..5), Some(b\"C3 OK\")) {\n                return Ok(());\n            } else if matches!(line.get(..5), Some(b\"C3 NO\"))\n                || matches!(line.get(..6), Some(b\"C3 BAD\"))\n            {\n                return Err(ImapError::AuthenticationFailed);\n            } else {\n                return Err(ImapError::InvalidResponse(line.into_string()));\n            }\n        }\n\n        Err(ImapError::InvalidResponse(line.into_string()))\n    }\n\n    pub async fn authentication_mechanisms(&mut self) -> Result<u64, ImapError> {\n        tokio::time::timeout(self.timeout, async {\n            self.write(b\"C0 CAPABILITY\\r\\n\").await?;\n\n            let mut line = self.read_line().await?.into_string();\n            if !line.starts_with(\"* CAPABILITY\") {\n                return Err(ImapError::InvalidResponse(line));\n            }\n            while !line.contains(\"C0 \") {\n                line.push_str(&self.read_line().await?.into_string());\n            }\n\n            let mut line_iter = line.as_bytes().iter();\n            let mut parser = Rfc5321Parser::new(&mut line_iter);\n            let mut mechanisms = 0;\n\n            'outer: while let Ok(ch) = parser.read_char() {\n                if ch == b' ' {\n                    loop {\n                        if parser.hashed_value().unwrap_or(0) == AUTH && parser.stop_char == b'=' {\n                            if let Ok(Some(mechanism)) = parser.mechanism() {\n                                mechanisms |= mechanism;\n                            }\n                            match parser.stop_char {\n                                b' ' => (),\n                                b'\\n' => break 'outer,\n                                _ => break,\n                            }\n                        }\n                    }\n                } else if ch == b'\\n' {\n                    break;\n                }\n            }\n\n            Ok(mechanisms)\n        })\n        .await\n        .map_err(|_| ImapError::Timeout)?\n    }\n\n    pub async fn noop(&mut self) -> Result<(), ImapError> {\n        tokio::time::timeout(self.timeout, async {\n            self.write(b\"C8 NOOP\\r\\n\").await?;\n            self.read_line().await?;\n            Ok(())\n        })\n        .await\n        .map_err(|_| ImapError::Timeout)?\n    }\n\n    pub async fn logout(&mut self) -> Result<(), ImapError> {\n        tokio::time::timeout(self.timeout, async {\n            self.write(b\"C9 LOGOUT\\r\\n\").await?;\n            Ok(())\n        })\n        .await\n        .map_err(|_| ImapError::Timeout)?\n    }\n\n    pub async fn expect_greeting(&mut self) -> Result<(), ImapError> {\n        tokio::time::timeout(self.timeout, async {\n            let line = self.read_line().await?;\n            if matches!(line.get(..4), Some(b\"* OK\")) {\n                Ok(())\n            } else {\n                Err(ImapError::InvalidResponse(line.into_string()))\n            }\n        })\n        .await\n        .map_err(|_| ImapError::Timeout)?\n    }\n\n    pub async fn read_line(&mut self) -> Result<Vec<u8>, ImapError> {\n        let mut buf = vec![0u8; 1024];\n        let mut buf_extended = Vec::with_capacity(0);\n\n        loop {\n            let br = self.stream.read(&mut buf).await?;\n\n            if br > 0 {\n                if matches!(buf.get(br - 1), Some(b'\\n')) {\n                    //println!(\"{:?}\", std::str::from_utf8(&buf[..br]).unwrap());\n                    return Ok(if buf_extended.is_empty() {\n                        buf.truncate(br);\n                        buf\n                    } else {\n                        buf_extended.extend_from_slice(&buf[..br]);\n                        buf_extended\n                    });\n                } else if buf_extended.is_empty() {\n                    buf_extended = buf[..br].to_vec();\n                } else {\n                    buf_extended.extend_from_slice(&buf[..br]);\n                }\n            } else {\n                return Err(ImapError::Disconnected);\n            }\n        }\n    }\n\n    pub async fn write(&mut self, bytes: &[u8]) -> Result<(), std::io::Error> {\n        self.stream.write_all(bytes).await?;\n        self.stream.flush().await\n    }\n}\n\n#[cfg(test)]\nmod test {\n    use mail_send::smtp::tls::build_tls_connector;\n    use smtp_proto::{AUTH_OAUTHBEARER, AUTH_PLAIN, AUTH_XOAUTH, AUTH_XOAUTH2};\n    use std::time::Duration;\n\n    use crate::backend::imap::ImapClient;\n\n    #[ignore]\n    #[tokio::test]\n    async fn imap_auth() {\n        let connector = build_tls_connector(false);\n\n        let mut client = ImapClient::connect(\n            \"imap.gmail.com:993\",\n            Duration::from_secs(5),\n            &connector,\n            \"imap.gmail.com\",\n            true,\n        )\n        .await\n        .unwrap();\n        assert_eq!(\n            AUTH_PLAIN | AUTH_XOAUTH | AUTH_XOAUTH2 | AUTH_OAUTHBEARER,\n            client.authentication_mechanisms().await.unwrap()\n        );\n        client.logout().await.unwrap();\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/imap/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse mail_send::smtp::tls::build_tls_connector;\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::core::config::build_pool;\n\nuse super::{ImapConnectionManager, ImapDirectory};\n\nimpl ImapDirectory {\n    pub fn from_config(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let address = config.value_require((&prefix, \"host\"))?.to_string();\n        let tls_implicit: bool = config\n            .property_or_default((&prefix, \"tls.enable\"), \"false\")\n            .unwrap_or_default();\n        let port: u16 = config\n            .property_or_default((&prefix, \"port\"), if tls_implicit { \"993\" } else { \"143\" })\n            .unwrap_or(if tls_implicit { 993 } else { 143 });\n\n        let manager = ImapConnectionManager {\n            addr: format!(\"{address}:{port}\"),\n            timeout: config\n                .property_or_default((&prefix, \"timeout\"), \"30s\")\n                .unwrap_or_else(|| Duration::from_secs(30)),\n            tls_connector: build_tls_connector(\n                config\n                    .property_or_default((&prefix, \"tls.allow-invalid-certs\"), \"false\")\n                    .unwrap_or_default(),\n            ),\n            tls_hostname: address.to_string(),\n            tls_implicit,\n            mechanisms: 0.into(),\n        };\n\n        Some(ImapDirectory {\n            pool: build_pool(config, &prefix, manager)\n                .map_err(|e| {\n                    config.new_parse_error(\n                        prefix.as_str(),\n                        format!(\"Failed to build IMAP pool: {e:?}\"),\n                    )\n                })\n                .ok()?,\n            domains: config\n                .values((&prefix, \"lookup.domains\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n        })\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/imap/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mail_send::Credentials;\nuse smtp_proto::{AUTH_CRAM_MD5, AUTH_LOGIN, AUTH_OAUTHBEARER, AUTH_PLAIN, AUTH_XOAUTH2};\n\nuse crate::{IntoError, Principal, QueryBy, Type, backend::RcptType};\n\nuse super::{ImapDirectory, ImapError};\n\nimpl ImapDirectory {\n    pub async fn query(&self, query: QueryBy<'_>) -> trc::Result<Option<Principal>> {\n        if let QueryBy::Credentials(credentials) = query {\n            let mut client = self\n                .pool\n                .get()\n                .await\n                .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n            let mechanism = match credentials {\n                Credentials::Plain { .. }\n                    if (client.mechanisms & (AUTH_PLAIN | AUTH_LOGIN | AUTH_CRAM_MD5)) != 0 =>\n                {\n                    if client.mechanisms & AUTH_CRAM_MD5 != 0 {\n                        AUTH_CRAM_MD5\n                    } else if client.mechanisms & AUTH_PLAIN != 0 {\n                        AUTH_PLAIN\n                    } else {\n                        AUTH_LOGIN\n                    }\n                }\n                Credentials::OAuthBearer { .. } if client.mechanisms & AUTH_OAUTHBEARER != 0 => {\n                    AUTH_OAUTHBEARER\n                }\n                Credentials::XOauth2 { .. } if client.mechanisms & AUTH_XOAUTH2 != 0 => {\n                    AUTH_XOAUTH2\n                }\n                _ => {\n                    trc::bail!(trc::StoreEvent::NotSupported.ctx(\n                        trc::Key::Reason,\n                        \"IMAP server does not offer any supported auth mechanisms.\"\n                    ));\n                }\n            };\n\n            match client.authenticate(mechanism, credentials).await {\n                Ok(_) => {\n                    client.is_valid = false;\n                    Ok(Some(Principal::new(u32::MAX, Type::Individual)))\n                }\n                Err(err) => match &err {\n                    ImapError::AuthenticationFailed => Ok(None),\n                    _ => Err(err.into_error()),\n                },\n            }\n        } else {\n            Err(trc::StoreEvent::NotSupported.caused_by(trc::location!()))\n        }\n    }\n\n    pub async fn email_to_id(&self, _address: &str) -> trc::Result<Option<u32>> {\n        Err(trc::StoreEvent::NotSupported.caused_by(trc::location!()))\n    }\n\n    pub async fn rcpt(&self, _address: &str) -> trc::Result<RcptType> {\n        Err(trc::StoreEvent::NotSupported.caused_by(trc::location!()))\n    }\n\n    pub async fn vrfy(&self, _address: &str) -> trc::Result<Vec<String>> {\n        Err(trc::StoreEvent::NotSupported.caused_by(trc::location!()))\n    }\n\n    pub async fn expn(&self, _address: &str) -> trc::Result<Vec<String>> {\n        Err(trc::StoreEvent::NotSupported.caused_by(trc::location!()))\n    }\n\n    pub async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        Ok(self.domains.contains(domain))\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/imap/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod client;\npub mod config;\npub mod lookup;\npub mod pool;\npub mod tls;\n\nuse std::{fmt::Display, sync::atomic::AtomicU64, time::Duration};\n\nuse ahash::AHashSet;\nuse deadpool::managed::Pool;\nuse tokio::io::{AsyncRead, AsyncWrite};\nuse tokio_rustls::TlsConnector;\n\npub struct ImapDirectory {\n    pool: Pool<ImapConnectionManager>,\n    domains: AHashSet<String>,\n}\n\npub struct ImapConnectionManager {\n    addr: String,\n    timeout: Duration,\n    tls_connector: TlsConnector,\n    tls_hostname: String,\n    tls_implicit: bool,\n    mechanisms: AtomicU64,\n}\n\npub struct ImapClient<T: AsyncRead + AsyncWrite> {\n    stream: T,\n    mechanisms: u64,\n    is_valid: bool,\n    timeout: Duration,\n}\n\n#[derive(Debug)]\npub enum ImapError {\n    Io(std::io::Error),\n    Timeout,\n    InvalidResponse(String),\n    InvalidChallenge(String),\n    AuthenticationFailed,\n    TLSInvalidName,\n    Disconnected,\n}\n\nimpl From<std::io::Error> for ImapError {\n    fn from(error: std::io::Error) -> Self {\n        ImapError::Io(error)\n    }\n}\n\nimpl Display for ImapError {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            ImapError::Io(io) => write!(f, \"I/O error: {io}\"),\n            ImapError::Timeout => f.write_str(\"Connection time-out\"),\n            ImapError::InvalidResponse(response) => write!(f, \"Unexpected response: {response:?}\"),\n            ImapError::InvalidChallenge(response) => {\n                write!(f, \"Invalid auth challenge: {response}\")\n            }\n            ImapError::TLSInvalidName => f.write_str(\"Invalid TLS name\"),\n            ImapError::Disconnected => f.write_str(\"Connection disconnected by peer\"),\n            ImapError::AuthenticationFailed => f.write_str(\"Authentication failed\"),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/imap/pool.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::atomic::Ordering;\n\nuse async_trait::async_trait;\nuse deadpool::managed;\nuse tokio::net::TcpStream;\nuse tokio_rustls::client::TlsStream;\n\nuse super::{ImapClient, ImapConnectionManager, ImapError};\n\n#[async_trait]\nimpl managed::Manager for ImapConnectionManager {\n    type Type = ImapClient<TlsStream<TcpStream>>;\n    type Error = ImapError;\n\n    async fn create(&self) -> Result<ImapClient<TlsStream<TcpStream>>, ImapError> {\n        let mut conn = ImapClient::connect(\n            &self.addr,\n            self.timeout,\n            &self.tls_connector,\n            &self.tls_hostname,\n            self.tls_implicit,\n        )\n        .await?;\n\n        // Obtain the list of supported authentication mechanisms.\n        conn.mechanisms = self.mechanisms.load(Ordering::Relaxed);\n        if conn.mechanisms == 0 {\n            conn.mechanisms = conn.authentication_mechanisms().await?;\n            self.mechanisms.store(conn.mechanisms, Ordering::Relaxed);\n        }\n\n        Ok(conn)\n    }\n\n    async fn recycle(\n        &self,\n        conn: &mut ImapClient<TlsStream<TcpStream>>,\n        _: &managed::Metrics,\n    ) -> managed::RecycleResult<ImapError> {\n        conn.noop()\n            .await\n            .map(|_| ())\n            .map_err(managed::RecycleError::Backend)\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/imap/tls.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse rustls_pki_types::ServerName;\nuse smtp_proto::IntoString;\nuse tokio::net::{TcpStream, ToSocketAddrs};\nuse tokio_rustls::{TlsConnector, client::TlsStream};\n\nuse super::{ImapClient, ImapError};\n\nimpl ImapClient<TcpStream> {\n    async fn start_tls(\n        mut self,\n        tls_connector: &TlsConnector,\n        tls_hostname: &str,\n    ) -> Result<ImapClient<TlsStream<TcpStream>>, ImapError> {\n        let line = tokio::time::timeout(self.timeout, async {\n            self.write(b\"C7 STARTTLS\\r\\n\").await?;\n\n            self.read_line().await\n        })\n        .await\n        .map_err(|_| ImapError::Timeout)??;\n\n        if matches!(line.get(..5), Some(b\"C7 OK\")) {\n            self.into_tls(tls_connector, tls_hostname).await\n        } else {\n            Err(ImapError::InvalidResponse(line.into_string()))\n        }\n    }\n\n    async fn into_tls(\n        self,\n        tls_connector: &TlsConnector,\n        tls_hostname: &str,\n    ) -> Result<ImapClient<TlsStream<TcpStream>>, ImapError> {\n        tokio::time::timeout(self.timeout, async {\n            Ok(ImapClient {\n                stream: tls_connector\n                    .connect(\n                        ServerName::try_from(tls_hostname.to_string())\n                            .map_err(|_| ImapError::TLSInvalidName)?,\n                        self.stream,\n                    )\n                    .await?,\n                timeout: self.timeout,\n                mechanisms: self.mechanisms,\n                is_valid: true,\n            })\n        })\n        .await\n        .map_err(|_| ImapError::Timeout)?\n    }\n}\n\nimpl ImapClient<TlsStream<TcpStream>> {\n    pub async fn connect(\n        addr: impl ToSocketAddrs,\n        timeout: Duration,\n        tls_connector: &TlsConnector,\n        tls_hostname: &str,\n        tls_implicit: bool,\n    ) -> Result<Self, ImapError> {\n        let mut client: ImapClient<TcpStream> = tokio::time::timeout(timeout, async {\n            match TcpStream::connect(addr).await {\n                Ok(stream) => Ok(ImapClient {\n                    stream,\n                    timeout,\n                    mechanisms: 0,\n                    is_valid: true,\n                }),\n                Err(err) => Err(ImapError::Io(err)),\n            }\n        })\n        .await\n        .map_err(|_| ImapError::Timeout)??;\n\n        if tls_implicit {\n            let mut client = client.into_tls(tls_connector, tls_hostname).await?;\n            client.expect_greeting().await?;\n            Ok(client)\n        } else {\n            client.expect_greeting().await?;\n            client.start_tls(tls_connector, tls_hostname).await\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/internal/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{PrincipalInfo, manage::ManageDirectory};\nuse crate::{Principal, PrincipalData, QueryBy, QueryParams, Type, backend::RcptType};\nuse mail_send::Credentials;\nuse store::{\n    Deserialize, IterateParams, Store, ValueKey,\n    write::{DirectoryClass, ValueClass},\n};\nuse trc::AddContext;\nuse utils::DomainPart;\n\n#[allow(async_fn_in_trait)]\npub trait DirectoryStore: Sync + Send {\n    async fn query(&self, by: QueryParams<'_>) -> trc::Result<Option<Principal>>;\n    async fn email_to_id(&self, address: &str) -> trc::Result<Option<u32>>;\n    async fn is_local_domain(&self, domain: &str) -> trc::Result<bool>;\n    async fn rcpt(&self, address: &str) -> trc::Result<RcptType>;\n    async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>>;\n    async fn expn(&self, address: &str) -> trc::Result<Vec<String>>;\n    async fn expn_by_id(&self, id: u32) -> trc::Result<Vec<String>>;\n}\n\nimpl DirectoryStore for Store {\n    async fn query(&self, by: QueryParams<'_>) -> trc::Result<Option<Principal>> {\n        let (account_id, secret) = match by.by {\n            QueryBy::Name(name) => (self.get_principal_id(name).await?, None),\n            QueryBy::Id(account_id) => (account_id.into(), None),\n            QueryBy::Credentials(credentials) => match credentials {\n                Credentials::Plain { username, secret } => (\n                    self.get_principal_id(username).await?,\n                    secret.as_str().into(),\n                ),\n                Credentials::OAuthBearer { token } => {\n                    (self.get_principal_id(token).await?, token.as_str().into())\n                }\n                Credentials::XOauth2 { username, secret } => (\n                    self.get_principal_id(username).await?,\n                    secret.as_str().into(),\n                ),\n            },\n        };\n\n        if let Some(account_id) = account_id\n            && let Some(mut principal) = self.get_principal(account_id).await?\n        {\n            if let Some(secret) = secret\n                && !principal\n                    .verify_secret(secret, by.only_app_pass, true)\n                    .await?\n            {\n                return Ok(None);\n            }\n\n            if by.return_member_of {\n                for member in self.get_member_of(principal.id).await? {\n                    match member.typ {\n                        Type::List => principal\n                            .data\n                            .push(PrincipalData::List(member.principal_id)),\n                        Type::Role => principal\n                            .data\n                            .push(PrincipalData::Role(member.principal_id)),\n                        _ => principal\n                            .data\n                            .push(PrincipalData::MemberOf(member.principal_id)),\n                    }\n                }\n            }\n            return Ok(Some(principal));\n        }\n        Ok(None)\n    }\n\n    async fn email_to_id(&self, address: &str) -> trc::Result<Option<u32>> {\n        self.get_value::<PrincipalInfo>(ValueKey::from(ValueClass::Directory(\n            DirectoryClass::EmailToId(address.as_bytes().to_vec()),\n        )))\n        .await\n        .map(|ptype| ptype.map(|ptype| ptype.id))\n    }\n\n    async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        self.get_value::<PrincipalInfo>(ValueKey::from(ValueClass::Directory(\n            DirectoryClass::NameToId(domain.as_bytes().to_vec()),\n        )))\n        .await\n        .map(|p| p.is_some_and(|p| p.typ == Type::Domain))\n    }\n\n    async fn rcpt(&self, address: &str) -> trc::Result<RcptType> {\n        if let Some(pinfo) = self\n            .get_value::<PrincipalInfo>(ValueKey::from(ValueClass::Directory(\n                DirectoryClass::EmailToId(address.as_bytes().to_vec()),\n            )))\n            .await?\n        {\n            if pinfo.typ != Type::List {\n                Ok(RcptType::Mailbox)\n            } else {\n                self.expn_by_id(pinfo.id).await.map(RcptType::List)\n            }\n        } else {\n            Ok(RcptType::Invalid)\n        }\n    }\n\n    async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>> {\n        let mut results = Vec::new();\n        let address = address.try_local_part().unwrap_or(address);\n        if address.len() > 3 {\n            self.iterate(\n                IterateParams::new(\n                    ValueKey::from(ValueClass::Directory(DirectoryClass::EmailToId(vec![0u8]))),\n                    ValueKey::from(ValueClass::Directory(DirectoryClass::EmailToId(\n                        vec![u8::MAX; 10],\n                    ))),\n                ),\n                |key, value| {\n                    let key =\n                        std::str::from_utf8(key.get(1..).unwrap_or_default()).unwrap_or_default();\n                    if key.try_local_part().unwrap_or(key).contains(address)\n                        && PrincipalInfo::deserialize(value)\n                            .caused_by(trc::location!())?\n                            .typ\n                            != Type::List\n                    {\n                        results.push(key.into());\n                    }\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n        }\n\n        Ok(results)\n    }\n\n    async fn expn(&self, address: &str) -> trc::Result<Vec<String>> {\n        if let Some(ptype) = self\n            .get_value::<PrincipalInfo>(ValueKey::from(ValueClass::Directory(\n                DirectoryClass::EmailToId(address.as_bytes().to_vec()),\n            )))\n            .await?\n            .filter(|p| p.typ == Type::List)\n        {\n            self.expn_by_id(ptype.id).await\n        } else {\n            Ok(vec![])\n        }\n    }\n\n    async fn expn_by_id(&self, list_id: u32) -> trc::Result<Vec<String>> {\n        let mut results = Vec::new();\n        for account_id in self.get_members(list_id).await? {\n            if let Some(email) = self.get_principal(account_id).await?.and_then(|p| {\n                p.data.into_iter().find_map(|data| {\n                    if let PrincipalData::PrimaryEmail(email) | PrincipalData::EmailAlias(email) =\n                        data\n                    {\n                        Some(email)\n                    } else {\n                        None\n                    }\n                })\n            }) {\n                results.push(email);\n            }\n        }\n\n        if let Some(principal) = self.get_principal(list_id).await? {\n            results.extend(principal.data.into_iter().filter_map(|data| {\n                if let PrincipalData::ExternalMember(member) = data {\n                    Some(member)\n                } else {\n                    None\n                }\n            }));\n        }\n\n        Ok(results)\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/internal/manage.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    PrincipalAction, PrincipalField, PrincipalInfo, PrincipalSet, PrincipalUpdate, PrincipalValue,\n    SpecialSecrets, lookup::DirectoryStore,\n};\nuse crate::{\n    ArchivedPrincipalData, FALLBACK_ADMIN_ID, MemberOf, Permission, PermissionGrant, Permissions,\n    Principal, PrincipalData, QueryBy, QueryParams, ROLE_ADMIN, ROLE_TENANT_ADMIN, ROLE_USER, Type,\n    backend::RcptType, core::principal::build_search_index,\n};\nuse ahash::{AHashMap, AHashSet};\nuse compact_str::CompactString;\nuse nlp::tokenizers::word::WordTokenizer;\nuse store::{\n    Deserialize, IterateParams, Serialize, SerializeInfallible, Store, U32_LEN, ValueKey,\n    backend::MAX_TOKEN_LENGTH,\n    roaring::RoaringBitmap,\n    write::{\n        AlignedBytes, Archive, Archiver, BatchBuilder, DirectoryClass, ValueClass,\n        key::DeserializeBigEndian,\n    },\n};\nuse trc::AddContext;\nuse types::{\n    collection::Collection,\n    field::{self},\n};\nuse utils::{DomainPart, sanitize_email};\n\n#[derive(Debug, Default, serde::Serialize, serde::Deserialize)]\npub struct PrincipalList<T> {\n    pub items: Vec<T>,\n    pub total: u64,\n}\n\npub struct UpdatePrincipal<'x> {\n    query: QueryBy<'x>,\n    allowed_permissions: Option<&'x Permissions>,\n    changes: Vec<PrincipalUpdate>,\n    tenant_id: Option<u32>,\n    create_domains: bool,\n}\n\n#[derive(Debug, Default, PartialEq, Eq)]\n#[repr(transparent)]\npub struct ChangedPrincipals(AHashMap<u32, ChangedPrincipal>);\n\n#[derive(Debug, Default, PartialEq, Eq)]\npub struct ChangedPrincipal {\n    pub typ: Type,\n    pub name_change: bool,\n    pub member_change: bool,\n}\n\n#[derive(Debug, Default, PartialEq, Eq)]\npub struct CreatedPrincipal {\n    pub id: u32,\n    pub changed_principals: ChangedPrincipals,\n}\n\n#[allow(async_fn_in_trait)]\npub trait ManageDirectory: Sized {\n    async fn get_principal_id(&self, name: &str) -> trc::Result<Option<u32>>;\n    async fn get_principal_info(&self, name: &str) -> trc::Result<Option<PrincipalInfo>>;\n    async fn get_or_create_principal_id(&self, name: &str, typ: Type) -> trc::Result<u32>;\n    async fn get_principal(&self, principal_id: u32) -> trc::Result<Option<Principal>>;\n    async fn get_principal_name(&self, principal_id: u32) -> trc::Result<Option<String>>;\n    async fn get_member_of(&self, principal_id: u32) -> trc::Result<Vec<MemberOf>>;\n    async fn get_members(&self, principal_id: u32) -> trc::Result<Vec<u32>>;\n    async fn create_principal(\n        &self,\n        principal: PrincipalSet,\n        tenant_id: Option<u32>,\n        allowed_permissions: Option<&Permissions>,\n    ) -> trc::Result<CreatedPrincipal>;\n    async fn update_principal(&self, params: UpdatePrincipal<'_>)\n    -> trc::Result<ChangedPrincipals>;\n    async fn delete_principal(&self, by: QueryBy<'_>) -> trc::Result<ChangedPrincipals>;\n    async fn list_principals(\n        &self,\n        filter: Option<&str>,\n        tenant_id: Option<u32>,\n        types: &[Type],\n        fetch: bool,\n        page: usize,\n        limit: usize,\n    ) -> trc::Result<PrincipalList<Principal>>;\n    async fn count_principals(\n        &self,\n        filter: Option<&str>,\n        typ: Option<Type>,\n        tenant_id: Option<u32>,\n    ) -> trc::Result<u64>;\n    async fn principal_ids(\n        &self,\n        typ: Option<Type>,\n        tenant_id: Option<u32>,\n    ) -> trc::Result<RoaringBitmap>;\n    async fn map_principal(\n        &self,\n        principal: Principal,\n        fields: &[PrincipalField],\n    ) -> trc::Result<PrincipalSet>;\n}\n\n#[allow(async_fn_in_trait)]\ntrait ValidateDirectory: Sized {\n    async fn validate_email(\n        &self,\n        email: &str,\n        tenant_id: Option<u32>,\n        create_if_missing: bool,\n    ) -> trc::Result<()>;\n}\n\nimpl ManageDirectory for Store {\n    async fn get_principal(&self, principal_id: u32) -> trc::Result<Option<Principal>> {\n        let archive = self\n            .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Directory(\n                DirectoryClass::Principal(principal_id),\n            )))\n            .await\n            .caused_by(trc::location!())?;\n\n        if let Some(archive) = archive {\n            let mut principal = archive\n                .deserialize::<Principal>()\n                .caused_by(trc::location!())?;\n            principal.id = principal_id;\n            Ok(Some(principal))\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn get_principal_name(&self, principal_id: u32) -> trc::Result<Option<String>> {\n        let archive = self\n            .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Directory(\n                DirectoryClass::Principal(principal_id),\n            )))\n            .await\n            .caused_by(trc::location!())?;\n\n        if let Some(archive) = archive {\n            let principal = archive\n                .unarchive::<Principal>()\n                .caused_by(trc::location!())?;\n            Ok(Some(principal.name.as_str().into()))\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn get_principal_id(&self, name: &str) -> trc::Result<Option<u32>> {\n        self.get_principal_info(name).await.map(|v| v.map(|v| v.id))\n    }\n\n    async fn get_principal_info(&self, name: &str) -> trc::Result<Option<PrincipalInfo>> {\n        self.get_value::<PrincipalInfo>(ValueKey::from(ValueClass::Directory(\n            DirectoryClass::NameToId(name.as_bytes().to_vec()),\n        )))\n        .await\n        .caused_by(trc::location!())\n    }\n\n    // Used by all directories except internal\n    async fn get_or_create_principal_id(&self, name: &str, typ: Type) -> trc::Result<u32> {\n        let mut try_count = 0;\n        let name = name.to_lowercase();\n        let mut principal_id = None;\n\n        loop {\n            // Try to obtain ID\n            if let Some(principal_id) = self\n                .get_principal_id(&name)\n                .await\n                .caused_by(trc::location!())?\n            {\n                return Ok(principal_id);\n            }\n\n            let principal_id = if let Some(principal_id) = principal_id {\n                principal_id\n            } else {\n                let principal_id_ = self\n                    .assign_document_ids(u32::MAX, Collection::Principal, 1)\n                    .await\n                    .caused_by(trc::location!())?;\n                if principal_id_ == FALLBACK_ADMIN_ID {\n                    return Err(trc::StoreEvent::UnexpectedError\n                        .into_err()\n                        .details(\"ID assignment failed\")\n                        .caused_by(trc::location!()));\n                }\n                principal_id = Some(principal_id_);\n                principal_id_\n            };\n\n            // Prepare principal\n            let mut principal = Principal::new(principal_id, typ);\n            principal.name = name.as_str().into();\n\n            // Write principal ID\n            let name_key =\n                ValueClass::Directory(DirectoryClass::NameToId(name.as_bytes().to_vec()));\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(u32::MAX)\n                .with_collection(Collection::Principal)\n                .assert_value(name_key.clone(), ())\n                .with_document(principal_id);\n            build_search_index(&mut batch, principal_id, None, Some(&principal));\n            principal.sort();\n            batch\n                .set(\n                    name_key,\n                    PrincipalInfo::new(principal_id, typ, None).serialize(),\n                )\n                .set(\n                    ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n                    Archiver::new(principal)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n\n            // Add default user role\n            if typ == Type::Individual {\n                batch\n                    .set(\n                        ValueClass::Directory(DirectoryClass::MemberOf {\n                            principal_id,\n                            member_of: ROLE_USER,\n                        }),\n                        vec![Type::Role as u8],\n                    )\n                    .set(\n                        ValueClass::Directory(DirectoryClass::Members {\n                            principal_id: ROLE_USER,\n                            has_member: principal_id,\n                        }),\n                        vec![],\n                    );\n            }\n\n            match self.write(batch.build_all()).await {\n                Ok(_) => {\n                    return Ok(principal_id);\n                }\n                Err(err) => {\n                    if err.is_assertion_failure() && try_count < 3 {\n                        try_count += 1;\n                        continue;\n                    } else {\n                        return Err(err.caused_by(trc::location!()));\n                    }\n                }\n            }\n        }\n    }\n\n    async fn create_principal(\n        &self,\n        mut principal_set: PrincipalSet,\n        mut tenant_id: Option<u32>,\n        allowed_permissions: Option<&Permissions>,\n    ) -> trc::Result<CreatedPrincipal> {\n        // Make sure the principal has a name\n        let name = principal_set.name().to_lowercase();\n        if name.is_empty() {\n            return Err(err_missing(PrincipalField::Name));\n        }\n        let mut valid_domains: AHashSet<String> = AHashSet::new();\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Validate tenant\n        #[cfg(feature = \"enterprise\")]\n        if let Some(tenant_id) = tenant_id {\n            let tenant = self\n                .query(crate::QueryParams::id(tenant_id).with_return_member_of(false))\n                .await?\n                .ok_or_else(|| {\n                    trc::ManageEvent::NotFound\n                        .into_err()\n                        .id(tenant_id)\n                        .details(\"Tenant not found\")\n                        .caused_by(trc::location!())\n                })?;\n\n            // Enforce tenant quotas\n            if let Some(limit) = tenant\n                .directory_quota(&principal_set.typ())\n                .filter(|q| *q > 0)\n            {\n                // Obtain number of principals\n                let total = self\n                    .count_principals(None, principal_set.typ().into(), tenant_id.into())\n                    .await\n                    .caused_by(trc::location!())? as u32;\n\n                if total >= limit {\n                    trc::bail!(\n                        trc::LimitEvent::TenantQuota\n                            .into_err()\n                            .details(\"Tenant principal quota exceeded\")\n                            .ctx(trc::Key::Details, principal_set.typ().description())\n                            .ctx(trc::Key::Limit, limit)\n                            .ctx(trc::Key::Total, total)\n                    );\n                }\n            }\n        }\n\n        // SPDX-SnippetEnd\n\n        // Make sure new name is not taken\n        if self\n            .get_principal_id(&name)\n            .await\n            .caused_by(trc::location!())?\n            .is_some()\n        {\n            return Err(err_exists(PrincipalField::Name, name));\n        }\n\n        let mut create_principal = Principal::new(0, principal_set.typ());\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Obtain tenant id, only if no default tenant is provided\n        #[cfg(feature = \"enterprise\")]\n        if let (Some(tenant_name), None) =\n            (principal_set.take_str(PrincipalField::Tenant), tenant_id)\n        {\n            tenant_id = self\n                .get_principal_info(&tenant_name)\n                .await\n                .caused_by(trc::location!())?\n                .filter(|v| v.typ == Type::Tenant)\n                .ok_or_else(|| not_found(tenant_name.clone()))?\n                .id\n                .into();\n        }\n\n        // Tenants must provide principal names including a valid domain\n        #[cfg(feature = \"enterprise\")]\n        if let Some(tenant_id) = tenant_id {\n            if matches!(principal_set.typ, Type::Tenant) {\n                return Err(error(\n                    \"Invalid field\",\n                    \"Tenants cannot contain a tenant field\".into(),\n                ));\n            }\n\n            create_principal.data.push(PrincipalData::Tenant(tenant_id));\n\n            if !matches!(create_principal.typ, Type::Tenant | Type::Domain) {\n                if let Some(domain) = name.try_domain_part()\n                    && self\n                        .get_principal_info(domain)\n                        .await\n                        .caused_by(trc::location!())?\n                        .filter(|v| v.typ == Type::Domain && v.has_tenant_access(tenant_id.into()))\n                        .is_some()\n                {\n                    valid_domains.insert(domain.into());\n                }\n\n                if valid_domains.is_empty() {\n                    return Err(error(\n                        \"Invalid principal name\",\n                        \"Principal name must include a valid domain assigned to the tenant\".into(),\n                    ));\n                }\n            }\n        }\n        // SPDX-SnippetEnd\n\n        // Set fields\n        create_principal.name = name;\n        let mut has_secret = false;\n        for secret in principal_set\n            .take_str_array(PrincipalField::Secrets)\n            .unwrap_or_default()\n        {\n            if secret.is_otp_secret() {\n                create_principal.data.push(PrincipalData::OtpAuth(secret));\n            } else if secret.is_app_secret() {\n                create_principal\n                    .data\n                    .push(PrincipalData::AppPassword(secret));\n            } else if !has_secret {\n                has_secret = true;\n                create_principal.data.push(PrincipalData::Password(secret));\n            }\n        }\n\n        if let Some(description) = principal_set.take_str(PrincipalField::Description) {\n            create_principal\n                .data\n                .push(PrincipalData::Description(description));\n        }\n\n        if let Some(picture) = principal_set.take_str(PrincipalField::Picture) {\n            create_principal.data.push(PrincipalData::Picture(picture));\n        }\n        if let Some(picture) = principal_set.take_str(PrincipalField::Locale) {\n            create_principal.data.push(PrincipalData::Locale(picture));\n        }\n        for url in principal_set\n            .take_str_array(PrincipalField::Urls)\n            .unwrap_or_default()\n        {\n            create_principal.data.push(PrincipalData::Url(url));\n        }\n        for member in principal_set\n            .take_str_array(PrincipalField::ExternalMembers)\n            .unwrap_or_default()\n        {\n            create_principal\n                .data\n                .push(PrincipalData::ExternalMember(member));\n        }\n        if let Some(quotas) = principal_set.take_int_array(PrincipalField::Quota) {\n            for (idx, quota) in quotas.into_iter().take(Type::MAX_ID + 2).enumerate() {\n                if quota != 0 {\n                    if idx != 0 {\n                        create_principal.data.push(PrincipalData::DirectoryQuota {\n                            quota: quota as u32,\n                            typ: Type::from_u8((idx - 1) as u8),\n                        });\n                    } else {\n                        create_principal.data.push(PrincipalData::DiskQuota(quota));\n                    }\n                }\n            }\n        }\n\n        // Map member names\n        let mut members = Vec::new();\n        let mut member_of = Vec::new();\n        let mut changed_principals = ChangedPrincipals::default();\n        for (field, expected_type) in [\n            (PrincipalField::Members, None),\n            (PrincipalField::MemberOf, Some(Type::Group)),\n            (PrincipalField::Lists, Some(Type::List)),\n            (PrincipalField::Roles, Some(Type::Role)),\n        ] {\n            if let Some(names) = principal_set.take_str_array(field) {\n                let list = if field == PrincipalField::Members {\n                    &mut members\n                } else {\n                    &mut member_of\n                };\n\n                for name in names {\n                    let item = match (\n                        self.get_principal_info(&name)\n                            .await\n                            .caused_by(trc::location!())?\n                            .filter(|v| {\n                                expected_type.is_none_or(|t| v.typ == t)\n                                    && v.has_tenant_access(tenant_id)\n                            }),\n                        field.map_internal_roles(&name),\n                    ) {\n                        (_, Some(v)) => v,\n                        (Some(v), _) => {\n                            if field == PrincipalField::Members {\n                                // Update principal members\n                                changed_principals.add_change(\n                                    v.id,\n                                    v.typ,\n                                    PrincipalField::MemberOf,\n                                );\n                            }\n                            v\n                        }\n                        _ => {\n                            return Err(not_found(name));\n                        }\n                    };\n\n                    list.push(item);\n                }\n            }\n        }\n\n        // Map permissions\n        let mut permissions = AHashMap::new();\n        for field in [\n            PrincipalField::EnabledPermissions,\n            PrincipalField::DisabledPermissions,\n        ] {\n            let is_disabled = field == PrincipalField::DisabledPermissions;\n            if let Some(names) = principal_set.take_str_array(field) {\n                for name in names {\n                    let permission = Permission::from_name(&name).ok_or_else(|| {\n                        error(\n                            format!(\"Invalid {} value\", field.as_str()),\n                            format!(\"Permission {name:?} is invalid\").into(),\n                        )\n                    })?;\n\n                    if !permissions.contains_key(&permission) {\n                        if allowed_permissions\n                            .as_ref()\n                            .is_none_or(|p| p.get(permission as usize))\n                            || is_disabled\n                        {\n                            permissions.insert(permission, is_disabled);\n                        } else {\n                            return Err(error(\n                                \"Invalid permission\",\n                                format!(\"Your account cannot grant the {name:?} permission\").into(),\n                            ));\n                        }\n                    }\n                }\n            }\n        }\n        if !permissions.is_empty() {\n            for (permission, v) in permissions {\n                create_principal.data.push(PrincipalData::Permission {\n                    permission_id: permission.id(),\n                    grant: !v,\n                });\n            }\n        }\n\n        // Make sure the e-mail is not taken and validate domain\n        if create_principal.typ != Type::OauthClient {\n            for (idx, email) in principal_set\n                .take_str_array(PrincipalField::Emails)\n                .unwrap_or_default()\n                .into_iter()\n                .enumerate()\n            {\n                let email = email.to_lowercase();\n                if self.rcpt(&email).await.caused_by(trc::location!())? != RcptType::Invalid {\n                    return Err(err_exists(PrincipalField::Emails, email.to_string()));\n                }\n                if let Some(domain) = email.try_domain_part()\n                    && valid_domains.insert(domain.into())\n                {\n                    self.get_principal_info(domain)\n                        .await\n                        .caused_by(trc::location!())?\n                        .filter(|v| v.typ == Type::Domain && v.has_tenant_access(tenant_id))\n                        .ok_or_else(|| not_found(domain.to_string()))?;\n                }\n                if idx == 0 {\n                    create_principal\n                        .data\n                        .push(PrincipalData::PrimaryEmail(email));\n                } else {\n                    create_principal.data.push(PrincipalData::EmailAlias(email));\n                }\n            }\n        }\n\n        // Write principal\n        let principal_id = self\n            .assign_document_ids(u32::MAX, Collection::Principal, 1)\n            .await\n            .caused_by(trc::location!())?;\n        if principal_id == FALLBACK_ADMIN_ID {\n            return Err(trc::StoreEvent::UnexpectedError\n                .into_err()\n                .details(\"ID assignment failed\")\n                .caused_by(trc::location!()));\n        }\n        create_principal.id = principal_id;\n        let mut batch = BatchBuilder::new();\n        let pinfo_name = PrincipalInfo::new(principal_id, create_principal.typ, tenant_id);\n        let pinfo_email = PrincipalInfo::new(principal_id, create_principal.typ, None);\n\n        // Validate object size\n        if create_principal.object_size() > 100_000 {\n            return Err(error(\n                \"Invalid parameter\",\n                \"Principal object size exceeds 100kb safety limit.\".into(),\n            ));\n        }\n\n        // Serialize\n        create_principal.sort();\n        let archiver = Archiver::new(create_principal);\n        let principal_bytes = archiver.serialize().caused_by(trc::location!())?;\n        let create_principal = archiver.into_inner();\n\n        batch\n            .with_account_id(u32::MAX)\n            .with_collection(Collection::Principal)\n            .with_document(principal_id)\n            .assert_value(\n                ValueClass::Directory(DirectoryClass::NameToId(\n                    create_principal.name().as_bytes().to_vec(),\n                )),\n                (),\n            );\n        build_search_index(&mut batch, principal_id, None, Some(&create_principal));\n        batch\n            .set(\n                ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n                principal_bytes,\n            )\n            .set(\n                ValueClass::Directory(DirectoryClass::NameToId(\n                    create_principal.name.as_bytes().to_vec(),\n                )),\n                pinfo_name.serialize(),\n            );\n\n        // Write email to id mapping\n        for email in create_principal.email_addresses() {\n            batch.set(\n                ValueClass::Directory(DirectoryClass::EmailToId(email.as_bytes().to_vec())),\n                pinfo_email.serialize(),\n            );\n        }\n\n        // Write membership\n        for member_of in member_of {\n            batch.set(\n                ValueClass::Directory(DirectoryClass::MemberOf {\n                    principal_id,\n                    member_of: member_of.id,\n                }),\n                vec![member_of.typ as u8],\n            );\n            batch.set(\n                ValueClass::Directory(DirectoryClass::Members {\n                    principal_id: member_of.id,\n                    has_member: principal_id,\n                }),\n                vec![],\n            );\n        }\n        for member in members {\n            batch.set(\n                ValueClass::Directory(DirectoryClass::MemberOf {\n                    principal_id: member.id,\n                    member_of: principal_id,\n                }),\n                vec![create_principal.typ as u8],\n            );\n            batch.set(\n                ValueClass::Directory(DirectoryClass::Members {\n                    principal_id,\n                    has_member: member.id,\n                }),\n                vec![],\n            );\n        }\n\n        self.write(batch.build_all())\n            .await\n            .map(|_| CreatedPrincipal {\n                id: principal_id,\n                changed_principals,\n            })\n    }\n\n    async fn delete_principal(&self, by: QueryBy<'_>) -> trc::Result<ChangedPrincipals> {\n        // Obtain principal\n        let principal_id = match by {\n            QueryBy::Name(name) => self\n                .get_principal_id(name)\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| not_found(name.to_string()))?,\n            QueryBy::Id(principal_id) => principal_id,\n            QueryBy::Credentials(_) => unreachable!(),\n        };\n\n        let principal_ = self\n            .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Directory(\n                DirectoryClass::Principal(principal_id),\n            )))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or_else(|| not_found(principal_id.to_string()))?;\n        let principal = principal_\n            .unarchive::<Principal>()\n            .caused_by(trc::location!())?;\n        let typ = Type::from(&principal.typ);\n\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(u32::MAX)\n            .with_collection(Collection::Principal);\n\n        let tenant = principal.data.iter().find_map(|data| {\n            if let ArchivedPrincipalData::Tenant(tenant_id) = data {\n                Some(tenant_id.to_native())\n            } else {\n                None\n            }\n        });\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Make sure tenant has no data\n        #[cfg(feature = \"enterprise\")]\n        match typ {\n            Type::Individual | Type::Group => {\n                // Update tenant quota\n                if let Some(tenant_id) = tenant {\n                    let quota = self\n                        .get_counter(DirectoryClass::UsedQuota(principal_id))\n                        .await\n                        .caused_by(trc::location!())?;\n                    if quota > 0 {\n                        batch.add(DirectoryClass::UsedQuota(tenant_id), -quota);\n                    }\n                }\n            }\n            Type::Tenant => {\n                let tenant_members = self\n                    .list_principals(\n                        None,\n                        principal_id.into(),\n                        &[\n                            Type::Individual,\n                            Type::Group,\n                            Type::Role,\n                            Type::List,\n                            Type::Resource,\n                            Type::Other,\n                            Type::Location,\n                            Type::Domain,\n                            Type::ApiKey,\n                        ],\n                        false,\n                        0,\n                        0,\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n\n                if tenant_members.total > 0 {\n                    let mut message =\n                        String::from(\"Tenant must have no members to be deleted: Found: \");\n\n                    for (num, principal) in tenant_members.items.iter().enumerate() {\n                        if num > 0 {\n                            message.push_str(\", \");\n                        }\n                        message.push_str(principal.name());\n                    }\n\n                    if tenant_members.total > 5 {\n                        message.push_str(\" and \");\n                        message.push_str(&(tenant_members.total - 5).to_string());\n                        message.push_str(\" others\");\n                    }\n\n                    return Err(error(\"Tenant has members\", message.into()));\n                }\n            }\n            Type::Domain => {\n                if let Some(tenant_id) = tenant {\n                    let name = principal.name.as_str();\n                    let tenant_members = self\n                        .list_principals(\n                            None,\n                            tenant_id.into(),\n                            &[\n                                Type::Individual,\n                                Type::Group,\n                                Type::Role,\n                                Type::List,\n                                Type::Resource,\n                                Type::Other,\n                                Type::Location,\n                            ],\n                            false,\n                            0,\n                            0,\n                        )\n                        .await\n                        .caused_by(trc::location!())?;\n                    let domain_members = tenant_members\n                        .items\n                        .iter()\n                        .filter(|v| {\n                            v.name()\n                                .rsplit_once('@')\n                                .is_some_and(|(_, d)| d.eq_ignore_ascii_case(name))\n                        })\n                        .collect::<Vec<_>>();\n                    let total_domain_members = domain_members.len();\n\n                    if total_domain_members > 0 {\n                        let mut message =\n                            String::from(\"Domains must have no members to be deleted: Found: \");\n\n                        for (num, principal) in domain_members.iter().enumerate() {\n                            if num > 0 {\n                                message.push_str(\", \");\n                            }\n                            message.push_str(principal.name());\n                        }\n\n                        if total_domain_members > 5 {\n                            message.push_str(\" and \");\n                            message.push_str(&(total_domain_members - 5).to_string());\n                            message.push_str(\" others\");\n                        }\n\n                        return Err(error(\"Domain has members\", message.into()));\n                    }\n                }\n            }\n\n            _ => {}\n        }\n        // SPDX-SnippetEnd\n\n        // Revoke ACLs, obtain all changed principals\n        let mut changed_principals = ChangedPrincipals::default();\n\n        for member_id in self\n            .acl_revoke_all(principal_id)\n            .await\n            .caused_by(trc::location!())?\n        {\n            changed_principals.add_change(\n                member_id,\n                Type::Individual,\n                PrincipalField::EnabledPermissions,\n            );\n        }\n\n        // Delete principal\n        batch\n            .with_document(principal_id)\n            .clear(DirectoryClass::NameToId(principal.name.as_bytes().to_vec()))\n            .clear(DirectoryClass::Principal(principal_id))\n            .clear(DirectoryClass::UsedQuota(principal_id));\n\n        for email in principal.data.iter() {\n            if let ArchivedPrincipalData::PrimaryEmail(email)\n            | ArchivedPrincipalData::EmailAlias(email) = email\n            {\n                batch.clear(DirectoryClass::EmailToId(email.as_bytes().to_vec()));\n            }\n        }\n\n        build_search_index(&mut batch, principal_id, Some(principal), None);\n\n        for member in self\n            .get_member_of(principal_id)\n            .await\n            .caused_by(trc::location!())?\n        {\n            // Update changed principals\n            changed_principals.add_member_change(\n                principal_id,\n                typ,\n                member.principal_id,\n                member.typ,\n            );\n\n            // Remove memberOf\n            batch.clear(DirectoryClass::MemberOf {\n                principal_id,\n                member_of: member.principal_id,\n            });\n            batch.clear(DirectoryClass::Members {\n                principal_id: member.principal_id,\n                has_member: principal_id,\n            });\n        }\n\n        for member_id in self\n            .get_members(principal_id)\n            .await\n            .caused_by(trc::location!())?\n        {\n            // Update changed principals\n            if let Some(member_info) = self\n                .get_principal(member_id)\n                .await\n                .caused_by(trc::location!())?\n            {\n                changed_principals.add_member_change(member_id, member_info.typ, principal_id, typ);\n            }\n\n            // Remove members\n            batch.clear(DirectoryClass::MemberOf {\n                principal_id: member_id,\n                member_of: principal_id,\n            });\n            batch.clear(DirectoryClass::Members {\n                principal_id,\n                has_member: member_id,\n            });\n        }\n\n        // Delete push subscriptions\n        if matches!(typ, Type::Individual) {\n            batch.untag(field::PrincipalField::PushSubscriptions);\n        }\n\n        self.write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n\n        changed_principals.add_deletion(principal_id, typ);\n\n        Ok(changed_principals)\n    }\n\n    async fn update_principal(\n        &self,\n        params: UpdatePrincipal<'_>,\n    ) -> trc::Result<ChangedPrincipals> {\n        let principal_id = match params.query {\n            QueryBy::Name(name) => self\n                .get_principal_id(name)\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| not_found(name.to_string()))?,\n            QueryBy::Id(principal_id) => principal_id,\n            QueryBy::Credentials(_) => unreachable!(),\n        };\n        let changes = params.changes;\n        let tenant_id = params.tenant_id;\n\n        // Fetch principal\n        let principal_ = self\n            .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Directory(\n                DirectoryClass::Principal(principal_id),\n            )))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or_else(|| not_found(principal_id))?;\n        let prev_principal = principal_\n            .to_unarchived::<Principal>()\n            .caused_by(trc::location!())?;\n        let mut principal = prev_principal\n            .deserialize::<Principal>()\n            .caused_by(trc::location!())?;\n        principal.id = principal_id;\n        let principal_type = principal.typ;\n        let validate_emails = principal_type != Type::OauthClient;\n\n        // Keep track of changed principals\n        let mut changed_principals = ChangedPrincipals::default();\n\n        // Obtain members and memberOf\n        let mut member_of = self\n            .get_member_of(principal_id)\n            .await\n            .caused_by(trc::location!())?;\n        let mut members = self\n            .get_members(principal_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Prepare changes\n        let mut batch = BatchBuilder::new();\n        let mut pinfo_name =\n            PrincipalInfo::new(principal_id, principal_type, principal.tenant()).serialize();\n        let pinfo_email = PrincipalInfo::new(principal_id, principal_type, None).serialize();\n        let update_principal = !changes.is_empty()\n            && !changes.iter().all(|c| {\n                matches!(\n                    c.field,\n                    PrincipalField::MemberOf\n                        | PrincipalField::Members\n                        | PrincipalField::Lists\n                        | PrincipalField::Roles\n                )\n            });\n\n        let mut used_quota: Option<i64> = None;\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Obtain used quota\n        #[cfg(feature = \"enterprise\")]\n        if tenant_id.is_none()\n            && changes\n                .iter()\n                .any(|c| matches!(c.field, PrincipalField::Tenant))\n        {\n            let quota = self\n                .get_counter(DirectoryClass::UsedQuota(principal_id))\n                .await\n                .caused_by(trc::location!())?;\n            if quota > 0 {\n                used_quota = Some(quota);\n            }\n        }\n\n        // SPDX-SnippetEnd\n\n        // Allowed principal types for Member fields\n        let allowed_member_types = match principal_type {\n            Type::Group => &[Type::Individual, Type::Group][..],\n            Type::Resource => &[Type::Resource][..],\n            Type::Location => &[\n                Type::Location,\n                Type::Resource,\n                Type::Individual,\n                Type::Group,\n                Type::Other,\n            ][..],\n            Type::List => &[Type::Individual, Type::Group][..],\n            Type::Other\n            | Type::Domain\n            | Type::Tenant\n            | Type::Individual\n            | Type::ApiKey\n            | Type::OauthClient => &[][..],\n            Type::Role => &[Type::Role][..],\n        };\n        let mut valid_domains = AHashSet::new();\n\n        // Process changes\n        for change in changes {\n            match (change.action, change.field, change.value) {\n                (PrincipalAction::Set, PrincipalField::Name, PrincipalValue::String(new_name)) => {\n                    // Make sure new name is not taken\n                    let new_name = new_name.to_lowercase();\n                    if principal.name() != new_name {\n                        if tenant_id.is_some()\n                            && !matches!(principal_type, Type::Tenant | Type::Domain)\n                        {\n                            if let Some(domain) = new_name.try_domain_part()\n                                && self\n                                    .get_principal_info(domain)\n                                    .await\n                                    .caused_by(trc::location!())?\n                                    .filter(|v| {\n                                        v.typ == Type::Domain && v.has_tenant_access(tenant_id)\n                                    })\n                                    .is_some()\n                            {\n                                valid_domains.insert(domain.to_string());\n                            }\n\n                            if valid_domains.is_empty() {\n                                return Err(error(\n                                    \"Invalid principal name\",\n                                    \"Principal name must include a valid domain assigned to the tenant\".into(),\n                                ));\n                            }\n                        }\n\n                        if self\n                            .get_principal_id(&new_name)\n                            .await\n                            .caused_by(trc::location!())?\n                            .is_some()\n                        {\n                            return Err(err_exists(PrincipalField::Name, new_name));\n                        }\n\n                        batch.clear(ValueClass::Directory(DirectoryClass::NameToId(\n                            principal.name().as_bytes().to_vec(),\n                        )));\n\n                        batch.set(\n                            ValueClass::Directory(DirectoryClass::NameToId(\n                                new_name.as_bytes().to_vec(),\n                            )),\n                            pinfo_name.clone(),\n                        );\n                        principal.name = new_name;\n\n                        // Name changed, update changed principals\n                        changed_principals.add_change(principal_id, principal_type, change.field);\n                    }\n                }\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::Tenant,\n                    PrincipalValue::String(tenant_name),\n                ) if tenant_id.is_none() => {\n                    if !tenant_name.is_empty() {\n                        let tenant_info = self\n                            .get_principal_info(&tenant_name)\n                            .await\n                            .caused_by(trc::location!())?\n                            .ok_or_else(|| not_found(tenant_name.clone()))?;\n\n                        if tenant_info.typ != Type::Tenant {\n                            return Err(error(\n                                \"Not a tenant\",\n                                format!(\"Principal {tenant_name:?} is not a tenant\").into(),\n                            ));\n                        }\n\n                        if principal.tenant() == Some(tenant_info.id) {\n                            continue;\n                        }\n\n                        // Update quota\n                        if let Some(used_quota) = used_quota {\n                            if let Some(old_tenant_id) = principal.tenant() {\n                                batch.add(DirectoryClass::UsedQuota(old_tenant_id), -used_quota);\n                            }\n                            batch.add(DirectoryClass::UsedQuota(tenant_info.id), used_quota);\n                        }\n\n                        // Tenant changed, update changed principals\n                        changed_principals.add_change(principal_id, principal_type, change.field);\n\n                        principal\n                            .data\n                            .retain(|v| !matches!(v, PrincipalData::Tenant(_)));\n                        principal.data.push(PrincipalData::Tenant(tenant_info.id));\n                        pinfo_name =\n                            PrincipalInfo::new(principal_id, principal_type, tenant_info.id.into())\n                                .serialize();\n                    } else if let Some(tenant_id) = principal.tenant() {\n                        // Update quota\n                        if let Some(used_quota) = used_quota {\n                            batch.add(DirectoryClass::UsedQuota(tenant_id), -used_quota);\n                        }\n\n                        // Tenant changed, update changed principals\n                        changed_principals.add_change(principal_id, principal_type, change.field);\n\n                        principal\n                            .data\n                            .retain(|v| !matches!(v, PrincipalData::Tenant(_)));\n                        pinfo_name =\n                            PrincipalInfo::new(principal_id, principal_type, None).serialize();\n                    } else {\n                        continue;\n                    }\n\n                    batch.set(\n                        ValueClass::Directory(DirectoryClass::NameToId(\n                            principal.name().as_bytes().to_vec(),\n                        )),\n                        pinfo_name.clone(),\n                    );\n                }\n\n                // SPDX-SnippetEnd\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::Secrets,\n                    value @ (PrincipalValue::StringList(_) | PrincipalValue::String(_)),\n                ) => {\n                    // Password changed, update changed principals\n                    changed_principals.add_change(principal_id, principal_type, change.field);\n                    principal.data.retain(|v| {\n                        !matches!(\n                            v,\n                            PrincipalData::Password(_)\n                                | PrincipalData::AppPassword(_)\n                                | PrincipalData::OtpAuth(_)\n                        )\n                    });\n                    let mut has_secret = false;\n                    for secret in value.into_str_array() {\n                        if secret.is_otp_secret() {\n                            principal.data.push(PrincipalData::OtpAuth(secret));\n                        } else if secret.is_app_secret() {\n                            principal.data.push(PrincipalData::AppPassword(secret));\n                        } else if !has_secret {\n                            has_secret = true;\n                            principal.data.push(PrincipalData::Password(secret));\n                        }\n                    }\n                }\n                (\n                    PrincipalAction::AddItem,\n                    PrincipalField::Secrets,\n                    PrincipalValue::String(secret),\n                ) => {\n                    if !principal.data.iter().any(|v| match v {\n                        PrincipalData::Password(v)\n                        | PrincipalData::AppPassword(v)\n                        | PrincipalData::OtpAuth(v) => *v == secret,\n                        _ => false,\n                    }) {\n                        if secret.is_app_secret() {\n                            principal.data.push(PrincipalData::AppPassword(secret));\n                        } else if secret.is_otp_secret() {\n                            principal\n                                .data\n                                .retain(|v| !matches!(v, PrincipalData::OtpAuth(_)));\n                            principal.data.push(PrincipalData::OtpAuth(secret));\n                        } else {\n                            principal\n                                .data\n                                .retain(|v| !matches!(v, PrincipalData::Password(_)));\n                            principal.data.push(PrincipalData::Password(secret));\n                        }\n\n                        // Password changed, update changed principals\n                        changed_principals.add_change(principal_id, principal_type, change.field);\n                    }\n                }\n                (\n                    PrincipalAction::RemoveItem,\n                    PrincipalField::Secrets,\n                    PrincipalValue::String(secret),\n                ) => {\n                    // Password changed, update changed principals\n                    changed_principals.add_change(principal_id, principal_type, change.field);\n\n                    if secret.is_app_secret() || secret.is_otp_secret() {\n                        principal.data.retain(|v| match v {\n                            PrincipalData::AppPassword(v) | PrincipalData::OtpAuth(v) => {\n                                *v != secret && !v.starts_with(secret.as_str())\n                            }\n                            _ => true,\n                        });\n                    } else if !secret.is_empty() {\n                        principal.data.retain(|v| match v {\n                            PrincipalData::Password(v) => *v != secret,\n                            _ => true,\n                        });\n                    } else {\n                        principal.data.retain(|v| {\n                            !matches!(v, PrincipalData::AppPassword(_) | PrincipalData::OtpAuth(_))\n                        });\n                    }\n                }\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::Description,\n                    PrincipalValue::String(value),\n                ) => {\n                    principal\n                        .data\n                        .retain(|v| !matches!(v, PrincipalData::Description(_)));\n                    if !value.is_empty() {\n                        principal.data.push(PrincipalData::Description(value));\n                    }\n                }\n                (PrincipalAction::Set, PrincipalField::Picture, PrincipalValue::String(value)) => {\n                    principal\n                        .data\n                        .retain(|v| !matches!(v, PrincipalData::Picture(_)));\n                    if !value.is_empty() {\n                        principal.data.push(PrincipalData::Picture(value));\n                    }\n                }\n                (PrincipalAction::Set, PrincipalField::Locale, PrincipalValue::String(value)) => {\n                    principal\n                        .data\n                        .retain(|v| !matches!(v, PrincipalData::Locale(_)));\n                    if !value.is_empty() {\n                        principal.data.push(PrincipalData::Locale(value));\n                    }\n                }\n                (PrincipalAction::Set, PrincipalField::Quota, PrincipalValue::Integer(quota))\n                    if matches!(\n                        principal_type,\n                        Type::Individual | Type::Group | Type::Tenant\n                    ) =>\n                {\n                    // Quota changed, update changed principals\n                    changed_principals.add_change(principal_id, principal_type, change.field);\n                    principal\n                        .data\n                        .retain(|v| !matches!(v, PrincipalData::DiskQuota(_)));\n                    principal.data.push(PrincipalData::DiskQuota(quota));\n                }\n                (PrincipalAction::Set, PrincipalField::Quota, PrincipalValue::String(quota))\n                    if matches!(\n                        principal_type,\n                        Type::Individual | Type::Group | Type::Tenant\n                    ) && quota.is_empty() =>\n                {\n                    // Quota changed, update changed principals\n                    changed_principals.add_change(principal_id, principal_type, change.field);\n                    principal\n                        .data\n                        .retain(|v| !matches!(v, PrincipalData::DiskQuota(_)));\n                }\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::Quota,\n                    PrincipalValue::IntegerList(quotas),\n                ) if matches!(principal_type, Type::Tenant)\n                    && quotas.len() <= (Type::MAX_ID + 2) =>\n                {\n                    let mut new_quota = None;\n\n                    principal.data.retain(|v| {\n                        !matches!(\n                            v,\n                            PrincipalData::DiskQuota(_) | PrincipalData::DirectoryQuota { .. }\n                        )\n                    });\n\n                    for (idx, quota) in quotas.into_iter().enumerate() {\n                        if quota != 0 {\n                            if idx != 0 {\n                                principal.data.push(PrincipalData::DirectoryQuota {\n                                    quota: quota as u32,\n                                    typ: Type::from_u8((idx - 1) as u8),\n                                });\n                            } else {\n                                new_quota = Some(quota);\n                            }\n                        }\n                    }\n\n                    if let Some(new_quota) = new_quota {\n                        principal.data.push(PrincipalData::DiskQuota(new_quota));\n                    }\n                }\n\n                // Emails\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::Emails,\n                    PrincipalValue::StringList(emails),\n                ) => {\n                    // Validate unique emails\n                    let emails = emails\n                        .into_iter()\n                        .map(|v| v.to_lowercase())\n                        .collect::<Vec<_>>();\n                    for email in &emails {\n                        if !principal.email_addresses().any(|v| v == email) {\n                            if validate_emails {\n                                self.validate_email(email, tenant_id, params.create_domains)\n                                    .await?;\n                            }\n                            batch.set(\n                                ValueClass::Directory(DirectoryClass::EmailToId(\n                                    email.as_bytes().to_vec(),\n                                )),\n                                pinfo_email.clone(),\n                            );\n                        }\n                    }\n\n                    for email in principal.email_addresses() {\n                        if !emails.iter().any(|v| v == email) {\n                            batch.clear(ValueClass::Directory(DirectoryClass::EmailToId(\n                                email.as_bytes().to_vec(),\n                            )));\n                        }\n                    }\n\n                    // Emails changed, update changed principals\n                    changed_principals.add_change(principal_id, principal_type, change.field);\n\n                    principal.data.retain(|v| {\n                        !matches!(\n                            v,\n                            PrincipalData::PrimaryEmail(_) | PrincipalData::EmailAlias(_)\n                        )\n                    });\n                    for (idx, email) in emails.into_iter().enumerate() {\n                        if idx == 0 {\n                            principal.data.push(PrincipalData::PrimaryEmail(email));\n                        } else {\n                            principal.data.push(PrincipalData::EmailAlias(email));\n                        }\n                    }\n                }\n                (\n                    PrincipalAction::AddItem,\n                    PrincipalField::Emails,\n                    PrincipalValue::String(email),\n                ) => {\n                    let email = email.to_lowercase();\n                    let mut emails_iter = principal.email_addresses().peekable();\n                    let has_emails = emails_iter.peek().is_some();\n                    let email_exists = emails_iter.any(|v| v == email);\n                    drop(emails_iter);\n                    if !email_exists {\n                        if validate_emails {\n                            self.validate_email(&email, tenant_id, params.create_domains)\n                                .await?;\n                        }\n                        batch.set(\n                            ValueClass::Directory(DirectoryClass::EmailToId(\n                                email.as_bytes().to_vec(),\n                            )),\n                            pinfo_email.clone(),\n                        );\n                        if has_emails {\n                            principal.data.push(PrincipalData::EmailAlias(email));\n                        } else {\n                            principal.data.push(PrincipalData::PrimaryEmail(email));\n                        }\n\n                        // Emails changed, update changed principals\n                        changed_principals.add_change(principal_id, principal_type, change.field);\n                    }\n                }\n                (\n                    PrincipalAction::RemoveItem,\n                    PrincipalField::Emails,\n                    PrincipalValue::String(email),\n                ) => {\n                    let email = email.to_lowercase();\n                    if principal.email_addresses().any(|v| v == email) {\n                        let mut deleted_primary = false;\n                        principal.data.retain(|v| match v {\n                            PrincipalData::EmailAlias(v) => v != &email,\n                            PrincipalData::PrimaryEmail(v) => {\n                                if v == &email {\n                                    deleted_primary = true;\n                                    false\n                                } else {\n                                    true\n                                }\n                            }\n                            _ => true,\n                        });\n                        batch.clear(ValueClass::Directory(DirectoryClass::EmailToId(\n                            email.as_bytes().to_vec(),\n                        )));\n\n                        if deleted_primary {\n                            for data in &mut principal.data {\n                                if let PrincipalData::EmailAlias(email) = data {\n                                    *data = PrincipalData::PrimaryEmail(std::mem::take(email));\n                                    break;\n                                }\n                            }\n                        }\n\n                        // Emails changed, update changed principals\n                        changed_principals.add_change(principal_id, principal_type, change.field);\n                    }\n                }\n\n                // MemberOf\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::MemberOf | PrincipalField::Lists | PrincipalField::Roles,\n                    PrincipalValue::StringList(members),\n                ) => {\n                    let mut new_member_of = Vec::new();\n                    for member in members {\n                        let member_info = match (\n                            self.get_principal_info(&member)\n                                .await\n                                .caused_by(trc::location!())?\n                                .filter(|p| p.has_tenant_access(tenant_id)),\n                            change.field.map_internal_roles(&member),\n                        ) {\n                            (_, Some(v)) => v,\n                            (Some(v), _) => v,\n                            _ => {\n                                return Err(not_found(member.clone()));\n                            }\n                        };\n\n                        validate_member_of(change.field, principal_type, member_info.typ, &member)?;\n\n                        if !member_of.iter().any(|v| v.principal_id == member_info.id) {\n                            // Update changed principal ids\n                            changed_principals.add_member_change(\n                                principal_id,\n                                principal_type,\n                                member_info.id,\n                                member_info.typ,\n                            );\n\n                            batch.set(\n                                ValueClass::Directory(DirectoryClass::MemberOf {\n                                    principal_id,\n                                    member_of: member_info.id,\n                                }),\n                                vec![member_info.typ as u8],\n                            );\n                            batch.set(\n                                ValueClass::Directory(DirectoryClass::Members {\n                                    principal_id: member_info.id,\n                                    has_member: principal_id,\n                                }),\n                                vec![],\n                            );\n                        }\n\n                        new_member_of.push(MemberOf {\n                            principal_id: member_info.id,\n                            typ: member_info.typ,\n                        });\n                    }\n\n                    for member in &member_of {\n                        if !new_member_of\n                            .iter()\n                            .any(|v| v.principal_id == member.principal_id)\n                        {\n                            // Update changed principal ids\n                            changed_principals.add_member_change(\n                                principal_id,\n                                principal_type,\n                                member.principal_id,\n                                member.typ,\n                            );\n\n                            batch.clear(ValueClass::Directory(DirectoryClass::MemberOf {\n                                principal_id,\n                                member_of: member.principal_id,\n                            }));\n                            batch.clear(ValueClass::Directory(DirectoryClass::Members {\n                                principal_id: member.principal_id,\n                                has_member: principal_id,\n                            }));\n                        }\n                    }\n\n                    member_of = new_member_of;\n                }\n                (\n                    PrincipalAction::AddItem,\n                    PrincipalField::MemberOf | PrincipalField::Lists | PrincipalField::Roles,\n                    PrincipalValue::String(member),\n                ) => {\n                    let member_info = match (\n                        self.get_principal_info(&member)\n                            .await\n                            .caused_by(trc::location!())?\n                            .filter(|p| p.has_tenant_access(tenant_id)),\n                        change.field.map_internal_roles(&member),\n                    ) {\n                        (_, Some(v)) => v,\n                        (Some(v), _) => v,\n                        _ => {\n                            return Err(not_found(member.clone()));\n                        }\n                    };\n\n                    if !member_of.iter().any(|v| v.principal_id == member_info.id) {\n                        validate_member_of(change.field, principal_type, member_info.typ, &member)?;\n\n                        // Update changed principal ids\n                        changed_principals.add_member_change(\n                            principal_id,\n                            principal_type,\n                            member_info.id,\n                            member_info.typ,\n                        );\n\n                        batch.set(\n                            ValueClass::Directory(DirectoryClass::MemberOf {\n                                principal_id,\n                                member_of: member_info.id,\n                            }),\n                            vec![member_info.typ as u8],\n                        );\n\n                        batch.set(\n                            ValueClass::Directory(DirectoryClass::Members {\n                                principal_id: member_info.id,\n                                has_member: principal_id,\n                            }),\n                            vec![],\n                        );\n\n                        member_of.push(MemberOf {\n                            principal_id: member_info.id,\n                            typ: member_info.typ,\n                        });\n                    }\n                }\n                (\n                    PrincipalAction::RemoveItem,\n                    PrincipalField::MemberOf | PrincipalField::Lists | PrincipalField::Roles,\n                    PrincipalValue::String(member),\n                ) => {\n                    if let Some(member_info) =\n                        self.get_principal_info(&member)\n                            .await\n                            .caused_by(trc::location!())?\n                            .or_else(|| {\n                                change.field.map_internal_role_name(&member).map(|id| {\n                                    PrincipalInfo {\n                                        id,\n                                        typ: Type::Role,\n                                        tenant: None,\n                                    }\n                                })\n                            })\n                    {\n                        for (pos, member) in member_of.iter().enumerate() {\n                            if member.principal_id == member_info.id {\n                                // Update changed principal ids\n                                changed_principals.add_member_change(\n                                    principal_id,\n                                    principal_type,\n                                    member_info.id,\n                                    member_info.typ,\n                                );\n\n                                batch.clear(ValueClass::Directory(DirectoryClass::MemberOf {\n                                    principal_id,\n                                    member_of: member_info.id,\n                                }));\n\n                                batch.clear(ValueClass::Directory(DirectoryClass::Members {\n                                    principal_id: member_info.id,\n                                    has_member: principal_id,\n                                }));\n\n                                member_of.remove(pos);\n                                break;\n                            }\n                        }\n                    }\n                }\n\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::Members,\n                    PrincipalValue::StringList(members_),\n                ) => {\n                    let mut new_members = Vec::new();\n\n                    for member in members_ {\n                        let member_info = self\n                            .get_principal_info(&member)\n                            .await\n                            .caused_by(trc::location!())?\n                            .filter(|p| p.has_tenant_access(tenant_id))\n                            .ok_or_else(|| not_found(member.clone()))?;\n\n                        if !allowed_member_types.contains(&member_info.typ) {\n                            return Err(error(\n                                \"Invalid members value\",\n                                format!(\n                                    \"Principal {member:?} is not one of {}.\",\n                                    allowed_member_types\n                                        .iter()\n                                        .map(|v| v.description())\n                                        .collect::<Vec<_>>()\n                                        .join(\", \")\n                                )\n                                .into(),\n                            ));\n                        }\n\n                        if !members.contains(&member_info.id) {\n                            // Update changed principal ids\n                            changed_principals.add_member_change(\n                                member_info.id,\n                                member_info.typ,\n                                principal_id,\n                                principal_type,\n                            );\n\n                            batch.set(\n                                ValueClass::Directory(DirectoryClass::MemberOf {\n                                    principal_id: member_info.id,\n                                    member_of: principal_id,\n                                }),\n                                vec![principal_type as u8],\n                            );\n                            batch.set(\n                                ValueClass::Directory(DirectoryClass::Members {\n                                    principal_id,\n                                    has_member: member_info.id,\n                                }),\n                                vec![],\n                            );\n                        }\n\n                        new_members.push(member_info.id);\n                    }\n\n                    for member_id in &members {\n                        if !new_members.contains(member_id) {\n                            // Update changed principal ids\n                            if principal_type != Type::List\n                                && let Some(member_info) = self\n                                    .get_principal(*member_id)\n                                    .await\n                                    .caused_by(trc::location!())?\n                            {\n                                changed_principals.add_member_change(\n                                    *member_id,\n                                    member_info.typ,\n                                    principal_id,\n                                    principal_type,\n                                );\n                            }\n\n                            batch.clear(ValueClass::Directory(DirectoryClass::MemberOf {\n                                principal_id: *member_id,\n                                member_of: principal_id,\n                            }));\n                            batch.clear(ValueClass::Directory(DirectoryClass::Members {\n                                principal_id,\n                                has_member: *member_id,\n                            }));\n                        }\n                    }\n\n                    members = new_members;\n                }\n                (\n                    PrincipalAction::AddItem,\n                    PrincipalField::Members,\n                    PrincipalValue::String(member),\n                ) => {\n                    let member_info = self\n                        .get_principal_info(&member)\n                        .await\n                        .caused_by(trc::location!())?\n                        .filter(|p| p.has_tenant_access(tenant_id))\n                        .ok_or_else(|| not_found(member.clone()))?;\n\n                    if !members.contains(&member_info.id) {\n                        if !allowed_member_types.contains(&member_info.typ) {\n                            return Err(error(\n                                \"Invalid members value\",\n                                format!(\n                                    \"Principal {member:?} is not one of {}.\",\n                                    allowed_member_types\n                                        .iter()\n                                        .map(|v| v.description())\n                                        .collect::<Vec<_>>()\n                                        .join(\", \")\n                                )\n                                .into(),\n                            ));\n                        }\n\n                        // Update changed principal ids\n                        changed_principals.add_member_change(\n                            member_info.id,\n                            member_info.typ,\n                            principal_id,\n                            principal_type,\n                        );\n\n                        batch.set(\n                            ValueClass::Directory(DirectoryClass::MemberOf {\n                                principal_id: member_info.id,\n                                member_of: principal_id,\n                            }),\n                            vec![principal_type as u8],\n                        );\n                        batch.set(\n                            ValueClass::Directory(DirectoryClass::Members {\n                                principal_id,\n                                has_member: member_info.id,\n                            }),\n                            vec![],\n                        );\n                        members.push(member_info.id);\n                    }\n                }\n                (\n                    PrincipalAction::RemoveItem,\n                    PrincipalField::Members,\n                    PrincipalValue::String(member),\n                ) => {\n                    if let Some(member_info) = self\n                        .get_principal_info(&member)\n                        .await\n                        .caused_by(trc::location!())?\n                    {\n                        for (pos, member_id) in members.iter().enumerate() {\n                            if *member_id == member_info.id {\n                                // Update changed principal ids\n                                changed_principals.add_member_change(\n                                    member_info.id,\n                                    member_info.typ,\n                                    principal_id,\n                                    principal_type,\n                                );\n\n                                batch.clear(ValueClass::Directory(DirectoryClass::MemberOf {\n                                    principal_id: member_info.id,\n                                    member_of: principal_id,\n                                }));\n                                batch.clear(ValueClass::Directory(DirectoryClass::Members {\n                                    principal_id,\n                                    has_member: member_info.id,\n                                }));\n                                members.remove(pos);\n                                break;\n                            }\n                        }\n                    }\n                }\n\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::EnabledPermissions | PrincipalField::DisabledPermissions,\n                    PrincipalValue::StringList(names),\n                ) => {\n                    let is_disabled = change.field == PrincipalField::DisabledPermissions;\n                    let mut permissions = AHashSet::with_capacity(names.len());\n                    for name in names {\n                        let permission = Permission::from_name(&name).ok_or_else(|| {\n                            error(\n                                format!(\"Invalid {} value\", change.field.as_str()),\n                                format!(\"Permission {name:?} is invalid\").into(),\n                            )\n                        })?;\n\n                        if !permissions.contains(&permission) {\n                            if params\n                                .allowed_permissions\n                                .as_ref()\n                                .is_none_or(|p| p.get(permission as usize))\n                                || is_disabled\n                            {\n                                permissions.insert(permission);\n                            } else {\n                                return Err(error(\n                                    \"Invalid permission\",\n                                    format!(\"Your account cannot grant the {name:?} permission\")\n                                        .into(),\n                                ));\n                            }\n                        }\n                    }\n\n                    principal.remove_permissions(!is_disabled);\n\n                    if !permissions.is_empty() {\n                        principal.add_permissions(permissions.into_iter().map(|permission| {\n                            PermissionGrant {\n                                permission,\n                                grant: !is_disabled,\n                            }\n                        }));\n                    }\n\n                    // Permissions changed, update changed principals\n                    changed_principals.add_change(principal_id, principal_type, change.field);\n                }\n                (\n                    PrincipalAction::AddItem,\n                    PrincipalField::EnabledPermissions | PrincipalField::DisabledPermissions,\n                    PrincipalValue::String(name),\n                ) => {\n                    let permission = Permission::from_name(&name).ok_or_else(|| {\n                        error(\n                            format!(\"Invalid {} value\", change.field.as_str()),\n                            format!(\"Permission {name:?} is invalid\").into(),\n                        )\n                    })?;\n\n                    if params\n                        .allowed_permissions\n                        .as_ref()\n                        .is_none_or(|p| p.get(permission as usize))\n                        || change.field == PrincipalField::DisabledPermissions\n                    {\n                        principal.add_permission(\n                            permission,\n                            change.field == PrincipalField::EnabledPermissions,\n                        );\n\n                        // Permissions changed, update changed principals\n                        changed_principals.add_change(principal_id, principal_type, change.field);\n                    } else {\n                        return Err(error(\n                            \"Invalid permission\",\n                            format!(\"Your account cannot grant the {name:?} permission\").into(),\n                        ));\n                    }\n                }\n                (\n                    PrincipalAction::RemoveItem,\n                    PrincipalField::EnabledPermissions | PrincipalField::DisabledPermissions,\n                    PrincipalValue::String(name),\n                ) => {\n                    let permission = Permission::from_name(&name).ok_or_else(|| {\n                        error(\n                            format!(\"Invalid {} value\", change.field.as_str()),\n                            format!(\"Permission {name:?} is invalid\").into(),\n                        )\n                    })?;\n\n                    principal.remove_permission(\n                        permission,\n                        change.field == PrincipalField::EnabledPermissions,\n                    );\n\n                    // Permissions changed, update changed principals\n                    changed_principals.add_change(principal_id, principal_type, change.field);\n                }\n                (\n                    PrincipalAction::Set,\n                    PrincipalField::ExternalMembers,\n                    PrincipalValue::StringList(items),\n                ) => {\n                    principal\n                        .data\n                        .retain(|v| !matches!(v, PrincipalData::ExternalMember(_)));\n                    if !items.is_empty() {\n                        principal.data.extend(\n                            items\n                                .into_iter()\n                                .map(|item| {\n                                    sanitize_email(&item)\n                                        .map(PrincipalData::ExternalMember)\n                                        .ok_or_else(|| {\n                                            error(\n                                                \"Invalid email address\",\n                                                format!(\n                                                    \"Invalid value {:?} for {}\",\n                                                    item,\n                                                    change.field.as_str()\n                                                )\n                                                .into(),\n                                            )\n                                        })\n                                })\n                                .collect::<trc::Result<Vec<_>>>()?,\n                        );\n                    }\n                }\n                (PrincipalAction::Set, PrincipalField::Urls, PrincipalValue::StringList(items)) => {\n                    principal\n                        .data\n                        .retain(|v| !matches!(v, PrincipalData::Url(_)));\n\n                    if !items.is_empty() {\n                        principal\n                            .data\n                            .extend(items.into_iter().map(PrincipalData::Url));\n                    }\n                }\n                (\n                    PrincipalAction::AddItem,\n                    PrincipalField::Urls | PrincipalField::ExternalMembers,\n                    PrincipalValue::String(mut item),\n                ) => {\n                    if matches!(change.field, PrincipalField::ExternalMembers) {\n                        item = sanitize_email(&item).ok_or_else(|| {\n                            error(\n                                \"Invalid email address\",\n                                format!(\"Invalid value {:?} for {}\", item, change.field.as_str())\n                                    .into(),\n                            )\n                        })?\n                    }\n\n                    let mut found = false;\n                    for data in &principal.data {\n                        match (data, change.field) {\n                            (PrincipalData::Url(url), PrincipalField::Urls) => {\n                                if url == &item {\n                                    found = true;\n                                    break;\n                                }\n                            }\n                            (\n                                PrincipalData::ExternalMember(email),\n                                PrincipalField::ExternalMembers,\n                            ) => {\n                                if email == &item {\n                                    found = true;\n                                    break;\n                                }\n                            }\n                            _ => {}\n                        }\n                    }\n\n                    if !found {\n                        match change.field {\n                            PrincipalField::Urls => principal.data.push(PrincipalData::Url(item)),\n                            PrincipalField::ExternalMembers => {\n                                principal.data.push(PrincipalData::ExternalMember(item))\n                            }\n                            _ => {}\n                        }\n                    }\n                }\n                (\n                    PrincipalAction::RemoveItem,\n                    PrincipalField::Urls,\n                    PrincipalValue::String(item),\n                ) => {\n                    principal.data.retain(|v| match v {\n                        PrincipalData::Url(v) => v != &item,\n                        _ => true,\n                    });\n                }\n                (\n                    PrincipalAction::RemoveItem,\n                    PrincipalField::ExternalMembers,\n                    PrincipalValue::String(item),\n                ) => {\n                    principal.data.retain(|v| match v {\n                        PrincipalData::ExternalMember(v) => v != &item,\n                        _ => true,\n                    });\n                }\n\n                (_, field, value) => {\n                    return Err(error(\n                        \"Invalid parameter\",\n                        format!(\"Invalid value {:?} for {}\", value, field.as_str()).into(),\n                    ));\n                }\n            }\n        }\n\n        // Validate object size\n        if principal.object_size() > 100_000 {\n            return Err(error(\n                \"Invalid parameter\",\n                \"Principal object size exceeds 100kb safety limit.\".into(),\n            ));\n        }\n\n        if update_principal {\n            principal.sort();\n            build_search_index(\n                &mut batch,\n                principal_id,\n                Some(prev_principal.inner),\n                Some(&principal),\n            );\n\n            batch\n                .assert_value(\n                    ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n                    prev_principal,\n                )\n                .set(\n                    ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n                    Archiver::new(principal)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n        }\n\n        self.write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(changed_principals)\n    }\n\n    async fn list_principals(\n        &self,\n        filter: Option<&str>,\n        tenant_id: Option<u32>,\n        types: &[Type],\n        fetch: bool,\n        page: usize,\n        limit: usize,\n    ) -> trc::Result<PrincipalList<Principal>> {\n        let filter = if let Some(filter) = filter.filter(|f| !f.trim().is_empty()) {\n            let mut matches = RoaringBitmap::new();\n\n            for token in WordTokenizer::new(filter, MAX_TOKEN_LENGTH) {\n                let word_bytes = token.word.as_bytes();\n                let from_key = ValueKey::from(ValueClass::Directory(DirectoryClass::Index {\n                    word: word_bytes.to_vec(),\n                    principal_id: 0,\n                }));\n                let to_key = ValueKey::from(ValueClass::Directory(DirectoryClass::Index {\n                    word: word_bytes.to_vec(),\n                    principal_id: u32::MAX,\n                }));\n\n                let mut word_matches = RoaringBitmap::new();\n                self.iterate(\n                    IterateParams::new(from_key, to_key).no_values(),\n                    |key, _| {\n                        let id_pos = key.len() - U32_LEN;\n                        if key.get(1..id_pos).is_some_and(|v| v == word_bytes) {\n                            word_matches.insert(key.deserialize_be_u32(id_pos)?);\n                            Ok(true)\n                        } else {\n                            Ok(false)\n                        }\n                    },\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n                if matches.is_empty() {\n                    matches = word_matches;\n                } else {\n                    matches &= word_matches;\n                    if matches.is_empty() {\n                        break;\n                    }\n                }\n            }\n\n            if !matches.is_empty() {\n                Some(matches)\n            } else {\n                return Ok(PrincipalList {\n                    total: 0,\n                    items: vec![],\n                });\n            }\n        } else {\n            None\n        };\n\n        let from_key = ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(vec![])));\n        let to_key = ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(vec![\n            u8::MAX;\n            10\n        ])));\n\n        let max_items = if limit > 0 { limit } else { usize::MAX };\n        let mut offset = page.saturating_sub(1) * limit;\n        let mut result = PrincipalList {\n            items: Vec::new(),\n            total: 0,\n        };\n        self.iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let pt = PrincipalInfo::deserialize(value).caused_by(trc::location!())?;\n\n                if (types.is_empty() || types.contains(&pt.typ))\n                    && pt.has_tenant_access(tenant_id)\n                    && filter.as_ref().is_none_or(|filter| filter.contains(pt.id))\n                {\n                    result.total += 1;\n                    if offset == 0 {\n                        if result.items.len() < max_items {\n                            let mut principal = Principal::new(pt.id, pt.typ);\n                            principal.name =\n                                String::from_utf8_lossy(key.get(1..).unwrap_or_default())\n                                    .into_owned();\n                            result.items.push(principal);\n                        }\n                    } else {\n                        offset -= 1;\n                    }\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        if fetch && !result.items.is_empty() {\n            let mut items = Vec::with_capacity(result.items.len());\n\n            for principal in result.items {\n                items.push(\n                    self.query(QueryParams::id(principal.id).with_return_member_of(fetch))\n                        .await\n                        .caused_by(trc::location!())?\n                        .ok_or_else(|| not_found(principal.name().to_string()))?,\n                );\n            }\n            result.items = items;\n\n            Ok(result)\n        } else {\n            Ok(result)\n        }\n    }\n\n    async fn count_principals(\n        &self,\n        filter: Option<&str>,\n        typ: Option<Type>,\n        tenant_id: Option<u32>,\n    ) -> trc::Result<u64> {\n        let from_key = ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(vec![])));\n        let to_key = ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(vec![\n            u8::MAX;\n            10\n        ])));\n\n        let mut count = 0;\n        self.iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let pt = PrincipalInfo::deserialize(value).caused_by(trc::location!())?;\n                let name =\n                    std::str::from_utf8(key.get(1..).unwrap_or_default()).unwrap_or_default();\n\n                if typ.is_none_or(|t| pt.typ == t)\n                    && pt.has_tenant_access(tenant_id)\n                    && filter.is_none_or(|f| name.contains(f))\n                {\n                    count += 1;\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n        .map(|_| count)\n    }\n\n    async fn principal_ids(\n        &self,\n        typ: Option<Type>,\n        tenant_id: Option<u32>,\n    ) -> trc::Result<RoaringBitmap> {\n        let mut results = RoaringBitmap::new();\n        self.iterate(\n            IterateParams::new(\n                ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(vec![0u8]))),\n                ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(vec![\n                    u8::MAX;\n                    10\n                ]))),\n            ),\n            |_, value| {\n                let pt = PrincipalInfo::deserialize(value).caused_by(trc::location!())?;\n                if typ.is_none_or(|t| pt.typ == t) && pt.has_tenant_access(tenant_id) {\n                    results.insert(pt.id);\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n        .map(|_| results)\n    }\n\n    async fn get_member_of(&self, principal_id: u32) -> trc::Result<Vec<MemberOf>> {\n        let from_key = ValueKey::from(ValueClass::Directory(DirectoryClass::MemberOf {\n            principal_id,\n            member_of: 0,\n        }));\n        let to_key = ValueKey::from(ValueClass::Directory(DirectoryClass::MemberOf {\n            principal_id,\n            member_of: u32::MAX,\n        }));\n        let mut results = Vec::new();\n        self.iterate(IterateParams::new(from_key, to_key), |key, value| {\n            results.push(MemberOf {\n                principal_id: key.deserialize_be_u32(key.len() - U32_LEN)?,\n                typ: value\n                    .first()\n                    .map(|v| Type::from_u8(*v))\n                    .unwrap_or(Type::Group),\n            });\n            Ok(true)\n        })\n        .await\n        .caused_by(trc::location!())?;\n        Ok(results)\n    }\n\n    async fn get_members(&self, principal_id: u32) -> trc::Result<Vec<u32>> {\n        let from_key = ValueKey::from(ValueClass::Directory(DirectoryClass::Members {\n            principal_id,\n            has_member: 0,\n        }));\n        let to_key = ValueKey::from(ValueClass::Directory(DirectoryClass::Members {\n            principal_id,\n            has_member: u32::MAX,\n        }));\n        let mut results = Vec::new();\n        self.iterate(\n            IterateParams::new(from_key, to_key).no_values(),\n            |key, _| {\n                results.push(key.deserialize_be_u32(key.len() - U32_LEN)?);\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n        Ok(results)\n    }\n\n    async fn map_principal(\n        &self,\n        principal: Principal,\n        fields: &[PrincipalField],\n    ) -> trc::Result<PrincipalSet> {\n        let mut result = PrincipalSet::new(principal.id, principal.typ);\n\n        let has_enabled = fields.is_empty() || fields.contains(&PrincipalField::EnabledPermissions);\n        let has_disabled =\n            fields.is_empty() || fields.contains(&PrincipalField::DisabledPermissions);\n        let mut directory_quotas = Vec::new();\n        let mut quota = None;\n        let mut tenant_id = None;\n\n        for data in principal.data {\n            match data {\n                PrincipalData::MemberOf(principal_id)\n                    if fields.is_empty() || fields.contains(&PrincipalField::MemberOf) =>\n                {\n                    if let Some(name) = self\n                        .get_principal_name(principal_id)\n                        .await\n                        .caused_by(trc::location!())?\n                    {\n                        result.append_str(PrincipalField::MemberOf, name);\n                    }\n                }\n                PrincipalData::Role(principal_id)\n                    if fields.is_empty() || fields.contains(&PrincipalField::Roles) =>\n                {\n                    match principal_id {\n                        ROLE_ADMIN => {\n                            result.append_str(PrincipalField::Roles, \"admin\");\n                        }\n                        ROLE_TENANT_ADMIN => {\n                            result.append_str(PrincipalField::Roles, \"tenant-admin\");\n                        }\n                        ROLE_USER => {\n                            result.append_str(PrincipalField::Roles, \"user\");\n                        }\n                        principal_id => {\n                            if let Some(name) = self\n                                .get_principal_name(principal_id)\n                                .await\n                                .caused_by(trc::location!())?\n                            {\n                                result.append_str(PrincipalField::Roles, name);\n                            }\n                        }\n                    }\n                }\n                PrincipalData::List(principal_id)\n                    if fields.is_empty() || fields.contains(&PrincipalField::Lists) =>\n                {\n                    if let Some(name) = self\n                        .get_principal_name(principal_id)\n                        .await\n                        .caused_by(trc::location!())?\n                    {\n                        result.append_str(PrincipalField::Lists, name);\n                    }\n                }\n                PrincipalData::Permission {\n                    permission_id,\n                    grant,\n                } if has_enabled || has_disabled => {\n                    if grant {\n                        if has_enabled {\n                            result.append_str(\n                                PrincipalField::EnabledPermissions,\n                                Permission::from_id(permission_id)\n                                    .map(|f| f.name())\n                                    .unwrap_or(\"unknown\"),\n                            );\n                        }\n                    } else if has_disabled {\n                        result.append_str(\n                            PrincipalField::DisabledPermissions,\n                            Permission::from_id(permission_id)\n                                .map(|f| f.name())\n                                .unwrap_or(\"unknown\"),\n                        );\n                    }\n                }\n                PrincipalData::DiskQuota(q) => {\n                    quota = Some(q);\n                }\n                PrincipalData::Tenant(tid) => {\n                    tenant_id = Some(tid);\n                }\n                PrincipalData::Description(description) => {\n                    if fields.is_empty() || fields.contains(&PrincipalField::Description) {\n                        result.set(PrincipalField::Description, description);\n                    }\n                }\n                PrincipalData::Password(secret)\n                | PrincipalData::AppPassword(secret)\n                | PrincipalData::OtpAuth(secret) => {\n                    if fields.is_empty() || fields.contains(&PrincipalField::Secrets) {\n                        result.append_str(PrincipalField::Secrets, secret);\n                    }\n                }\n                PrincipalData::PrimaryEmail(email) | PrincipalData::EmailAlias(email) => {\n                    if fields.is_empty() || fields.contains(&PrincipalField::Emails) {\n                        result.append_str(PrincipalField::Emails, email);\n                    }\n                }\n                PrincipalData::Picture(picture) => {\n                    if fields.is_empty() || fields.contains(&PrincipalField::Picture) {\n                        result.set(PrincipalField::Picture, picture);\n                    }\n                }\n                PrincipalData::Locale(locale) => {\n                    if fields.is_empty() || fields.contains(&PrincipalField::Locale) {\n                        result.set(PrincipalField::Locale, locale);\n                    }\n                }\n                PrincipalData::ExternalMember(member) => {\n                    if fields.is_empty() || fields.contains(&PrincipalField::ExternalMembers) {\n                        result.append_str(PrincipalField::ExternalMembers, member);\n                    }\n                }\n                PrincipalData::Url(url) => {\n                    if fields.is_empty() || fields.contains(&PrincipalField::Urls) {\n                        result.append_str(PrincipalField::Urls, url);\n                    }\n                }\n                PrincipalData::DirectoryQuota { quota, typ } => {\n                    directory_quotas.push((typ, quota));\n                }\n                _ => (),\n            }\n        }\n\n        // Obtain member names\n        if fields.is_empty() || fields.contains(&PrincipalField::Members) {\n            match principal.typ {\n                Type::Group | Type::List | Type::Role => {\n                    for member_id in self.get_members(principal.id).await? {\n                        if let Some(member_principal) = self\n                            .query(QueryParams::id(member_id).with_return_member_of(false))\n                            .await?\n                        {\n                            result.append_str(PrincipalField::Members, member_principal.name);\n                        }\n                    }\n                }\n                Type::Domain => {\n                    let from_key =\n                        ValueKey::from(ValueClass::Directory(DirectoryClass::EmailToId(vec![])));\n                    let to_key = ValueKey::from(ValueClass::Directory(DirectoryClass::EmailToId(\n                        vec![u8::MAX; 10],\n                    )));\n                    let domain_name = &principal.name;\n                    let mut total: u64 = 0;\n                    self.iterate(\n                        IterateParams::new(from_key, to_key).no_values(),\n                        |key, _| {\n                            if std::str::from_utf8(key.get(1..).unwrap_or_default())\n                                .unwrap_or_default()\n                                .rsplit_once('@')\n                                .is_some_and(|(_, domain)| domain == domain_name)\n                            {\n                                total += 1;\n                            }\n                            Ok(true)\n                        },\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n                    result.set(PrincipalField::Members, total);\n                }\n                Type::Tenant => {\n                    let from_key =\n                        ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(vec![])));\n                    let to_key = ValueKey::from(ValueClass::Directory(DirectoryClass::NameToId(\n                        vec![u8::MAX; 10],\n                    )));\n                    let mut total: u64 = 0;\n\n                    self.iterate(IterateParams::new(from_key, to_key), |_, value| {\n                        let pinfo =\n                            PrincipalInfo::deserialize(value).caused_by(trc::location!())?;\n\n                        if pinfo.typ == Type::Individual\n                            && pinfo.has_tenant_access(Some(principal.id))\n                        {\n                            total += 1;\n                        }\n                        Ok(true)\n                    })\n                    .await\n                    .caused_by(trc::location!())?;\n\n                    result.set(PrincipalField::Members, total);\n                }\n                _ => {}\n            }\n        }\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Map tenant name\n        #[cfg(feature = \"enterprise\")]\n        if let Some(tenant_id) = tenant_id\n            && (fields.is_empty() || fields.contains(&PrincipalField::Tenant))\n            && let Some(name) = self\n                .get_principal_name(tenant_id)\n                .await\n                .caused_by(trc::location!())?\n        {\n            result.set(PrincipalField::Tenant, name);\n        }\n\n        // SPDX-SnippetEnd\n\n        // Map fields\n        if fields.is_empty() || fields.contains(&PrincipalField::Name) {\n            result.set(PrincipalField::Name, principal.name);\n        }\n        if fields.is_empty() || fields.contains(&PrincipalField::Quota) {\n            if !directory_quotas.is_empty() {\n                let mut quotas = vec![0u64; Type::MAX_ID + 2];\n                if let Some(quota) = quota {\n                    quotas[0] = quota;\n                }\n                for (typ, quota) in directory_quotas {\n                    quotas[(typ as usize) + 1] = quota as u64;\n                }\n\n                result.set(PrincipalField::Quota, quotas);\n            } else if let Some(quota) = quota {\n                result.set(PrincipalField::Quota, quota);\n            }\n        }\n\n        // Obtain used quota\n        if matches!(principal.typ, Type::Individual | Type::Group | Type::Tenant)\n            && (fields.is_empty() || fields.contains(&PrincipalField::UsedQuota))\n        {\n            let quota = self\n                .get_counter(DirectoryClass::UsedQuota(principal.id))\n                .await\n                .caused_by(trc::location!())?;\n            if quota > 0 {\n                result.set(PrincipalField::UsedQuota, quota as u64);\n            }\n        }\n\n        Ok(result)\n    }\n}\n\nimpl ValidateDirectory for Store {\n    async fn validate_email(\n        &self,\n        email: &str,\n        tenant_id: Option<u32>,\n        create_if_missing: bool,\n    ) -> trc::Result<()> {\n        if self.rcpt(email).await.caused_by(trc::location!())? != RcptType::Invalid {\n            Err(err_exists(PrincipalField::Emails, email.to_string()))\n        } else if let Some(domain) = email.try_domain_part() {\n            match self\n                .get_principal_info(domain)\n                .await\n                .caused_by(trc::location!())?\n            {\n                Some(v) if v.typ == Type::Domain && v.has_tenant_access(tenant_id) => Ok(()),\n                None if create_if_missing => self\n                    .create_principal(\n                        PrincipalSet::new(0, Type::Domain)\n                            .with_field(PrincipalField::Name, domain)\n                            .with_field(PrincipalField::Description, domain),\n                        tenant_id,\n                        None,\n                    )\n                    .await\n                    .caused_by(trc::location!())\n                    .map(|_| ()),\n                _ => Err(not_found(domain.to_string())),\n            }\n        } else {\n            Err(error(\"Invalid email\", \"Email address is invalid\".into()))\n        }\n    }\n}\n\nimpl PrincipalField {\n    pub fn map_internal_role_name(&self, name: &str) -> Option<u32> {\n        match (self, name) {\n            (PrincipalField::Roles, \"admin\") => Some(ROLE_ADMIN),\n            (PrincipalField::Roles, \"tenant-admin\") => Some(ROLE_TENANT_ADMIN),\n            (PrincipalField::Roles, \"user\") => Some(ROLE_USER),\n            _ => None,\n        }\n    }\n\n    pub fn map_internal_roles(&self, name: &str) -> Option<PrincipalInfo> {\n        self.map_internal_role_name(name)\n            .map(|role_id| PrincipalInfo::new(role_id, Type::Role, None))\n    }\n}\n\nimpl<'x> UpdatePrincipal<'x> {\n    pub fn by_id(id: u32) -> Self {\n        Self {\n            query: QueryBy::Id(id),\n            changes: Vec::new(),\n            create_domains: false,\n            tenant_id: None,\n            allowed_permissions: None,\n        }\n    }\n\n    pub fn by_name(name: &'x str) -> Self {\n        Self {\n            query: QueryBy::Name(name),\n            changes: Vec::new(),\n            create_domains: false,\n            tenant_id: None,\n            allowed_permissions: None,\n        }\n    }\n\n    pub fn with_tenant(mut self, tenant_id: Option<u32>) -> Self {\n        self.tenant_id = tenant_id;\n        self\n    }\n\n    pub fn with_updates(mut self, changes: Vec<PrincipalUpdate>) -> Self {\n        self.changes = changes;\n        self\n    }\n\n    pub fn with_allowed_permissions(mut self, permissions: &'x Permissions) -> Self {\n        self.allowed_permissions = permissions.into();\n        self\n    }\n\n    pub fn create_domains(mut self) -> Self {\n        self.create_domains = true;\n        self\n    }\n}\n\nfn validate_member_of(\n    field: PrincipalField,\n    typ: Type,\n    member_type: Type,\n    member_name: &str,\n) -> trc::Result<()> {\n    let expected_types = match (field, typ) {\n        (PrincipalField::MemberOf, Type::Individual) => &[Type::Group, Type::Individual][..],\n        (PrincipalField::MemberOf, Type::Group) => &[Type::Group][..],\n        (PrincipalField::Lists, Type::Individual | Type::Group) => &[Type::List][..],\n        (PrincipalField::Roles, Type::Individual | Type::Tenant | Type::Role) => &[Type::Role][..],\n        _ => &[][..],\n    };\n\n    if expected_types.is_empty() || !expected_types.contains(&member_type) {\n        Err(error(\n            format!(\"Invalid {} value\", field.as_str()),\n            if !expected_types.is_empty() {\n                format!(\n                    \"Principal {member_name:?} is not a {}.\",\n                    expected_types\n                        .iter()\n                        .map(|t| t.description().to_string())\n                        .collect::<Vec<_>>()\n                        .join(\", \")\n                )\n                .into()\n            } else {\n                format!(\"Principal {member_name:?} cannot be added as a member.\").into()\n            },\n        ))\n    } else {\n        Ok(())\n    }\n}\n\nimpl ChangedPrincipals {\n    pub fn new() -> Self {\n        Self::default()\n    }\n\n    pub fn from_change(principal_id: u32, principal_type: Type, field: PrincipalField) -> Self {\n        let mut set = Self::default();\n        set.add_change(principal_id, principal_type, field);\n        set\n    }\n\n    pub fn add_change(&mut self, principal_id: u32, principal_type: Type, field: PrincipalField) {\n        if matches!(\n            (principal_type, field),\n            (\n                Type::Individual | Type::Group,\n                PrincipalField::Name\n                    | PrincipalField::Quota\n                    | PrincipalField::Secrets\n                    | PrincipalField::Emails\n                    | PrincipalField::MemberOf\n                    | PrincipalField::Members\n                    | PrincipalField::Tenant\n                    | PrincipalField::Roles\n                    | PrincipalField::EnabledPermissions\n                    | PrincipalField::DisabledPermissions,\n            ) | (\n                Type::Tenant | Type::Role | Type::ApiKey | Type::OauthClient,\n                PrincipalField::MemberOf\n                    | PrincipalField::Members\n                    | PrincipalField::Secrets\n                    | PrincipalField::Tenant\n                    | PrincipalField::Roles\n                    | PrincipalField::EnabledPermissions\n                    | PrincipalField::DisabledPermissions,\n            )\n        ) && principal_id < ROLE_USER\n        {\n            self.0\n                .entry(principal_id)\n                .or_insert_with(|| ChangedPrincipal::new(principal_type))\n                .update_member_change(matches!(\n                    (field, principal_type),\n                    (\n                        PrincipalField::EnabledPermissions | PrincipalField::DisabledPermissions,\n                        Type::Role | Type::Tenant\n                    )\n                ))\n                .update_name_change(matches!(field, PrincipalField::Name));\n        }\n    }\n\n    pub fn add_member_change(\n        &mut self,\n        principal_id: u32,\n        principal_type: Type,\n        member_id: u32,\n        member_type: Type,\n    ) {\n        match (principal_type, member_type) {\n            (Type::Group | Type::Role, Type::Individual | Type::ApiKey | Type::OauthClient) => {\n                self.0\n                    .entry(member_id)\n                    .or_insert_with(|| ChangedPrincipal::new(member_type));\n            }\n            (Type::Individual | Type::ApiKey | Type::OauthClient, Type::Group | Type::Role) => {\n                self.0\n                    .entry(principal_id)\n                    .or_insert_with(|| ChangedPrincipal::new(principal_type));\n            }\n            (\n                Type::Group | Type::Tenant | Type::Role,\n                Type::Individual | Type::Group | Type::Tenant | Type::Role,\n            ) => {\n                if principal_id < ROLE_USER {\n                    self.0\n                        .entry(principal_id)\n                        .or_insert_with(|| ChangedPrincipal::new(principal_type))\n                        .update_member_change(matches!(member_type, Type::Role));\n                }\n                if member_id < ROLE_USER {\n                    self.0\n                        .entry(member_id)\n                        .or_insert_with(|| ChangedPrincipal::new(member_type))\n                        .update_member_change(matches!(principal_type, Type::Role));\n                }\n            }\n            _ => {}\n        }\n    }\n\n    pub fn add_deletion(&mut self, principal_id: u32, principal_type: Type) {\n        if matches!(\n            principal_type,\n            Type::Individual\n                | Type::Group\n                | Type::Tenant\n                | Type::Role\n                | Type::ApiKey\n                | Type::OauthClient\n        ) {\n            self.0\n                .entry(principal_id)\n                .or_insert_with(|| ChangedPrincipal::new(principal_type));\n        }\n    }\n\n    pub fn contains(&self, principal_id: u32) -> bool {\n        self.0.contains_key(&principal_id)\n    }\n\n    pub fn iter(&'_ self) -> std::collections::hash_map::Iter<'_, u32, ChangedPrincipal> {\n        self.0.iter()\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.0.is_empty()\n    }\n}\n\nimpl ChangedPrincipal {\n    pub fn new(typ: Type) -> Self {\n        Self {\n            typ,\n            member_change: false,\n            name_change: false,\n        }\n    }\n\n    pub fn update_member_change(&mut self, member_change: bool) -> &mut Self {\n        self.member_change |= member_change;\n        self\n    }\n\n    pub fn update_name_change(&mut self, name_change: bool) -> &mut Self {\n        self.name_change |= name_change;\n        self\n    }\n}\n\npub fn err_missing(field: impl Into<trc::Value>) -> trc::Error {\n    trc::ManageEvent::MissingParameter.ctx(trc::Key::Key, field)\n}\n\npub fn err_exists(field: impl Into<trc::Value>, value: impl Into<trc::Value>) -> trc::Error {\n    trc::ManageEvent::AlreadyExists\n        .ctx(trc::Key::Key, field)\n        .ctx(trc::Key::Value, value)\n}\n\npub fn not_found(value: impl Into<trc::Value>) -> trc::Error {\n    trc::ManageEvent::NotFound.ctx(trc::Key::Key, value)\n}\n\npub fn unsupported(details: impl Into<trc::Value>) -> trc::Error {\n    trc::ManageEvent::NotSupported.ctx(trc::Key::Details, details)\n}\n\npub fn enterprise() -> trc::Error {\n    trc::ManageEvent::NotSupported.ctx(trc::Key::Details, \"Enterprise feature\")\n}\n\npub fn error(details: impl Into<trc::Value>, reason: Option<impl Into<trc::Value>>) -> trc::Error {\n    trc::ManageEvent::Error\n        .ctx(trc::Key::Details, details)\n        .ctx_opt(trc::Key::Reason, reason)\n}\n\nimpl From<PrincipalField> for trc::Value {\n    fn from(value: PrincipalField) -> Self {\n        trc::Value::String(CompactString::const_new(value.as_str()))\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/internal/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod lookup;\npub mod manage;\n\nuse crate::Type;\nuse ahash::AHashMap;\n\nuse std::fmt::Display;\nuse store::{Deserialize, SerializeInfallible, U32_LEN, write::key::KeySerializer};\nuse utils::codec::leb128::Leb128Iterator;\n\npub struct PrincipalInfo {\n    pub id: u32,\n    pub typ: Type,\n    pub tenant: Option<u32>,\n}\n\nimpl PrincipalInfo {\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    pub fn has_tenant_access(&self, tenant_id: Option<u32>) -> bool {\n        tenant_id.is_none_or(|tenant_id| {\n            self.tenant.is_some_and(|t| tenant_id == t)\n                || (self.typ == Type::Tenant && self.id == tenant_id)\n        })\n    }\n    // SPDX-SnippetEnd\n\n    #[cfg(not(feature = \"enterprise\"))]\n    pub fn has_tenant_access(&self, _tenant_id: Option<u32>) -> bool {\n        true\n    }\n}\n\nimpl SerializeInfallible for PrincipalInfo {\n    fn serialize(&self) -> Vec<u8> {\n        if let Some(tenant) = self.tenant {\n            KeySerializer::new((U32_LEN * 2) + 1)\n                .write_leb128(self.id)\n                .write(self.typ as u8)\n                .write_leb128(tenant)\n                .finalize()\n        } else {\n            KeySerializer::new(U32_LEN + 1)\n                .write_leb128(self.id)\n                .write(self.typ as u8)\n                .finalize()\n        }\n    }\n}\n\nimpl Deserialize for PrincipalInfo {\n    fn deserialize(bytes_: &[u8]) -> trc::Result<Self> {\n        let mut bytes = bytes_.iter();\n        Ok(PrincipalInfo {\n            id: bytes.next_leb128().ok_or_else(|| {\n                trc::StoreEvent::DataCorruption\n                    .caused_by(trc::location!())\n                    .ctx(trc::Key::Value, bytes_)\n            })?,\n            typ: Type::from_u8(*bytes.next().ok_or_else(|| {\n                trc::StoreEvent::DataCorruption\n                    .caused_by(trc::location!())\n                    .ctx(trc::Key::Value, bytes_)\n            })?),\n            tenant: bytes.next_leb128(),\n        })\n    }\n}\n\nimpl PrincipalInfo {\n    pub fn new(principal_id: u32, typ: Type, tenant: Option<u32>) -> Self {\n        Self {\n            id: principal_id,\n            typ,\n            tenant,\n        }\n    }\n}\n\n#[derive(\n    Debug, Clone, Copy, PartialEq, Hash, Eq, PartialOrd, Ord, serde::Serialize, serde::Deserialize,\n)]\n#[serde(rename_all = \"camelCase\")]\npub enum PrincipalField {\n    Name,\n    Type,\n    Quota,\n    UsedQuota,\n    Description,\n    Secrets,\n    Emails,\n    MemberOf,\n    Members,\n    Tenant,\n    Roles,\n    Lists,\n    EnabledPermissions,\n    DisabledPermissions,\n    Picture,\n    Urls,\n    ExternalMembers,\n    Locale,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\npub struct PrincipalSet {\n    pub id: u32,\n    pub typ: Type,\n    pub fields: AHashMap<PrincipalField, PrincipalValue>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]\npub struct PrincipalUpdate {\n    pub action: PrincipalAction,\n    pub field: PrincipalField,\n    pub value: PrincipalValue,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]\npub enum PrincipalAction {\n    #[serde(rename = \"set\")]\n    Set,\n    #[serde(rename = \"addItem\")]\n    AddItem,\n    #[serde(rename = \"removeItem\")]\n    RemoveItem,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]\n#[serde(untagged)]\npub enum PrincipalValue {\n    String(String),\n    StringList(Vec<String>),\n    Integer(u64),\n    IntegerList(Vec<u64>),\n}\n\nimpl PrincipalUpdate {\n    pub fn set(field: PrincipalField, value: PrincipalValue) -> PrincipalUpdate {\n        PrincipalUpdate {\n            action: PrincipalAction::Set,\n            field,\n            value,\n        }\n    }\n\n    pub fn add_item(field: PrincipalField, value: PrincipalValue) -> PrincipalUpdate {\n        PrincipalUpdate {\n            action: PrincipalAction::AddItem,\n            field,\n            value,\n        }\n    }\n\n    pub fn remove_item(field: PrincipalField, value: PrincipalValue) -> PrincipalUpdate {\n        PrincipalUpdate {\n            action: PrincipalAction::RemoveItem,\n            field,\n            value,\n        }\n    }\n}\n\nimpl Display for PrincipalField {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        self.as_str().fmt(f)\n    }\n}\n\nimpl PrincipalField {\n    pub fn id(&self) -> u8 {\n        match self {\n            PrincipalField::Name => 0,\n            PrincipalField::Type => 1,\n            PrincipalField::Quota => 2,\n            PrincipalField::Description => 3,\n            PrincipalField::Secrets => 4,\n            PrincipalField::Emails => 5,\n            PrincipalField::MemberOf => 6,\n            PrincipalField::Members => 7,\n            PrincipalField::Tenant => 8,\n            PrincipalField::Roles => 9,\n            PrincipalField::Lists => 10,\n            PrincipalField::EnabledPermissions => 11,\n            PrincipalField::DisabledPermissions => 12,\n            PrincipalField::UsedQuota => 13,\n            PrincipalField::Picture => 14,\n            PrincipalField::Urls => 15,\n            PrincipalField::ExternalMembers => 16,\n            PrincipalField::Locale => 17,\n        }\n    }\n\n    pub fn from_id(id: u8) -> Option<Self> {\n        match id {\n            0 => Some(PrincipalField::Name),\n            1 => Some(PrincipalField::Type),\n            2 => Some(PrincipalField::Quota),\n            3 => Some(PrincipalField::Description),\n            4 => Some(PrincipalField::Secrets),\n            5 => Some(PrincipalField::Emails),\n            6 => Some(PrincipalField::MemberOf),\n            7 => Some(PrincipalField::Members),\n            8 => Some(PrincipalField::Tenant),\n            9 => Some(PrincipalField::Roles),\n            10 => Some(PrincipalField::Lists),\n            11 => Some(PrincipalField::EnabledPermissions),\n            12 => Some(PrincipalField::DisabledPermissions),\n            13 => Some(PrincipalField::UsedQuota),\n            14 => Some(PrincipalField::Picture),\n            15 => Some(PrincipalField::Urls),\n            16 => Some(PrincipalField::ExternalMembers),\n            17 => Some(PrincipalField::Locale),\n            _ => None,\n        }\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            PrincipalField::Name => \"name\",\n            PrincipalField::Type => \"type\",\n            PrincipalField::Quota => \"quota\",\n            PrincipalField::UsedQuota => \"usedQuota\",\n            PrincipalField::Description => \"description\",\n            PrincipalField::Secrets => \"secrets\",\n            PrincipalField::Emails => \"emails\",\n            PrincipalField::MemberOf => \"memberOf\",\n            PrincipalField::Members => \"members\",\n            PrincipalField::Tenant => \"tenant\",\n            PrincipalField::Roles => \"roles\",\n            PrincipalField::Lists => \"lists\",\n            PrincipalField::EnabledPermissions => \"enabledPermissions\",\n            PrincipalField::DisabledPermissions => \"disabledPermissions\",\n            PrincipalField::Picture => \"picture\",\n            PrincipalField::Urls => \"urls\",\n            PrincipalField::ExternalMembers => \"externalMembers\",\n            PrincipalField::Locale => \"locale\",\n        }\n    }\n\n    pub fn try_parse(s: &str) -> Option<Self> {\n        match s {\n            \"name\" => Some(PrincipalField::Name),\n            \"type\" => Some(PrincipalField::Type),\n            \"quota\" => Some(PrincipalField::Quota),\n            \"usedQuota\" => Some(PrincipalField::UsedQuota),\n            \"description\" => Some(PrincipalField::Description),\n            \"secrets\" => Some(PrincipalField::Secrets),\n            \"emails\" => Some(PrincipalField::Emails),\n            \"memberOf\" => Some(PrincipalField::MemberOf),\n            \"members\" => Some(PrincipalField::Members),\n            \"tenant\" => Some(PrincipalField::Tenant),\n            \"roles\" => Some(PrincipalField::Roles),\n            \"lists\" => Some(PrincipalField::Lists),\n            \"enabledPermissions\" => Some(PrincipalField::EnabledPermissions),\n            \"disabledPermissions\" => Some(PrincipalField::DisabledPermissions),\n            \"picture\" => Some(PrincipalField::Picture),\n            \"urls\" => Some(PrincipalField::Urls),\n            \"externalMembers\" => Some(PrincipalField::ExternalMembers),\n            \"locale\" => Some(PrincipalField::Locale),\n            _ => None,\n        }\n    }\n}\n\npub trait SpecialSecrets {\n    fn is_otp_secret(&self) -> bool;\n    fn is_app_secret(&self) -> bool;\n}\n\nimpl<T> SpecialSecrets for T\nwhere\n    T: AsRef<str>,\n{\n    fn is_otp_secret(&self) -> bool {\n        self.as_ref().starts_with(\"otpauth://\")\n    }\n\n    fn is_app_secret(&self) -> bool {\n        self.as_ref().starts_with(\"$app$\")\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/ldap/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse ldap3::LdapConnSettings;\nuse store::Store;\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::core::config::build_pool;\n\nuse super::{\n    AuthBind, Bind, LdapConnectionManager, LdapDirectory, LdapFilter, LdapFilterItem, LdapMappings,\n};\n\nimpl LdapDirectory {\n    pub fn from_config(config: &mut Config, prefix: impl AsKey, data_store: Store) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let bind_dn = if let Some(dn) = config.value((&prefix, \"bind.dn\")) {\n            Bind::new(\n                dn.to_string(),\n                config.value_require((&prefix, \"bind.secret\"))?.to_string(),\n            )\n            .into()\n        } else {\n            None\n        };\n\n        let manager = LdapConnectionManager::new(\n            config.value_require((&prefix, \"url\"))?.to_string(),\n            LdapConnSettings::new()\n                .set_conn_timeout(\n                    config\n                        .property_or_default((&prefix, \"timeout\"), \"30s\")\n                        .unwrap_or_else(|| Duration::from_secs(30)),\n                )\n                .set_starttls(\n                    config\n                        .property_or_default((&prefix, \"tls.enable\"), \"false\")\n                        .unwrap_or_default(),\n                )\n                .set_no_tls_verify(\n                    config\n                        .property_or_default((&prefix, \"tls.allow-invalid-certs\"), \"false\")\n                        .unwrap_or_default(),\n                ),\n            bind_dn,\n        );\n\n        let mut mappings = LdapMappings {\n            base_dn: config.value_require((&prefix, \"base-dn\"))?.to_string(),\n            filter_name: LdapFilter::from_config(config, (&prefix, \"filter.name\")),\n            filter_email: LdapFilter::from_config(config, (&prefix, \"filter.email\")),\n            attr_name: config\n                .values((&prefix, \"attributes.name\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_groups: config\n                .values((&prefix, \"attributes.groups\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_type: config\n                .values((&prefix, \"attributes.class\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_description: config\n                .values((&prefix, \"attributes.description\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_secret: config\n                .values((&prefix, \"attributes.secret\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_secret_changed: config\n                .values((&prefix, \"attributes.secret-changed\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_email_address: config\n                .values((&prefix, \"attributes.email\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_quota: config\n                .values((&prefix, \"attributes.quota\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attr_email_alias: config\n                .values((&prefix, \"attributes.email-alias\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n            attrs_principal: vec![\"objectClass\".to_lowercase()],\n        };\n\n        for attr in [\n            &mappings.attr_name,\n            &mappings.attr_type,\n            &mappings.attr_description,\n            &mappings.attr_secret,\n            &mappings.attr_secret_changed,\n            &mappings.attr_quota,\n            &mappings.attr_groups,\n            &mappings.attr_email_address,\n            &mappings.attr_email_alias,\n        ] {\n            mappings\n                .attrs_principal\n                .extend(attr.iter().filter(|a| !a.is_empty()).cloned());\n        }\n\n        let auth_bind = match config\n            .value((&prefix, \"bind.auth.method\"))\n            .unwrap_or(\"default\")\n        {\n            \"template\" => AuthBind::Template {\n                template: LdapFilter::from_config(config, (&prefix, \"bind.auth.template\")),\n                can_search: config\n                    .property_or_default::<bool>((&prefix, \"bind.auth.search\"), \"true\")\n                    .unwrap_or(true),\n            },\n            \"lookup\" => AuthBind::Lookup,\n            \"default\" => AuthBind::None,\n            unknown => {\n                config.new_parse_error(\n                    (&prefix, \"bind.auth.method\"),\n                    format!(\"Unknown LDAP bind method: {unknown}\"),\n                );\n                return None;\n            }\n        };\n\n        Some(LdapDirectory {\n            mappings,\n            pool: build_pool(config, &prefix, manager)\n                .map_err(|e| {\n                    config.new_parse_error(prefix, format!(\"Failed to build LDAP pool: {e:?}\"))\n                })\n                .ok()?,\n            auth_bind,\n            data_store,\n        })\n    }\n}\n\nimpl LdapFilter {\n    fn from_config(config: &mut Config, key: impl AsKey) -> Self {\n        if let Some(value) = config.value(key.clone()) {\n            let mut filter = Vec::new();\n            let mut token = String::new();\n            let mut value = value.chars();\n\n            while let Some(ch) = value.next() {\n                match ch {\n                    '?' => {\n                        // For backwards compatibility, we treat '?' as a placeholder for the full value.\n                        if !token.is_empty() {\n                            filter.push(LdapFilterItem::Static(token));\n                            token = String::new();\n                        }\n                        filter.push(LdapFilterItem::Full);\n                    }\n                    '{' => {\n                        if !token.is_empty() {\n                            filter.push(LdapFilterItem::Static(token));\n                            token = String::new();\n                        }\n                        for ch in value.by_ref() {\n                            if ch == '}' {\n                                break;\n                            } else {\n                                token.push(ch);\n                            }\n                        }\n                        match token.as_str() {\n                            \"user\" | \"username\" | \"email\" => filter.push(LdapFilterItem::Full),\n                            \"local\" => filter.push(LdapFilterItem::LocalPart),\n                            \"domain\" => filter.push(LdapFilterItem::DomainPart),\n                            _ => {\n                                config.new_parse_error(\n                                    key,\n                                    format!(\"Unknown LDAP filter placeholder: {}\", token),\n                                );\n                                return Self::default();\n                            }\n                        }\n                        token.clear();\n                    }\n                    _ => token.push(ch),\n                }\n            }\n\n            if !token.is_empty() {\n                filter.push(LdapFilterItem::Static(token));\n            }\n\n            if filter.len() >= 2 {\n                return LdapFilter { filter };\n            } else {\n                config.new_parse_error(\n                    key,\n                    format!(\"Missing parameter placeholders in value {:?}\", value),\n                );\n            }\n        }\n\n        Self::default()\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/ldap/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{AuthBind, LdapDirectory, LdapMappings};\nuse crate::{\n    IntoError, Principal, PrincipalData, QueryBy, QueryParams, ROLE_ADMIN, ROLE_USER, Type,\n    backend::{\n        RcptType,\n        internal::{\n            SpecialSecrets,\n            lookup::DirectoryStore,\n            manage::{self, ManageDirectory, UpdatePrincipal},\n        },\n    },\n};\nuse ldap3::{Ldap, LdapConnAsync, ResultEntry, Scope, SearchEntry};\nuse mail_send::Credentials;\nuse store::xxhash_rust;\nuse trc::AddContext;\n\nimpl LdapDirectory {\n    pub async fn query(&self, by: QueryParams<'_>) -> trc::Result<Option<Principal>> {\n        let mut conn = self.pool.get().await.map_err(|err| err.into_error())?;\n        let (mut external_principal, member_of, stored_principal) = match by.by {\n            QueryBy::Name(username) => {\n                let filter = self.mappings.filter_name.build(username);\n                if let Some(mut result) = self.find_principal(&mut conn, &filter).await? {\n                    if result.principal.name.is_empty() {\n                        result.principal.name = username.into();\n                    }\n                    (result.principal, result.member_of, None)\n                } else {\n                    trc::event!(\n                        Store(trc::StoreEvent::LdapWarning),\n                        Reason = \"Name filter yielded no results\",\n                        Details = filter\n                    );\n                    return Ok(None);\n                }\n            }\n            QueryBy::Id(uid) => {\n                if let Some(stored_principal_) = self\n                    .data_store\n                    .query(QueryParams::id(uid).with_return_member_of(by.return_member_of))\n                    .await?\n                {\n                    if let Some(result) = self\n                        .find_principal(\n                            &mut conn,\n                            &self.mappings.filter_name.build(stored_principal_.name()),\n                        )\n                        .await?\n                    {\n                        (result.principal, result.member_of, Some(stored_principal_))\n                    } else {\n                        return Ok(None);\n                    }\n                } else {\n                    return Ok(None);\n                }\n            }\n            QueryBy::Credentials(credentials) => {\n                let (username, secret) = match credentials {\n                    Credentials::Plain { username, secret } => (username, secret),\n                    Credentials::OAuthBearer { token } => (token, token),\n                    Credentials::XOauth2 { username, secret } => (username, secret),\n                };\n\n                match &self.auth_bind {\n                    AuthBind::Template {\n                        template,\n                        can_search,\n                    } => {\n                        let (auth_bind_conn, mut ldap) = LdapConnAsync::with_settings(\n                            self.pool.manager().settings.clone(),\n                            &self.pool.manager().address,\n                        )\n                        .await\n                        .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n\n                        ldap3::drive!(auth_bind_conn);\n\n                        let dn = template.build(username);\n\n                        if ldap\n                            .simple_bind(&dn, secret)\n                            .await\n                            .map_err(|err| err.into_error().caused_by(trc::location!()))?\n                            .success()\n                            .is_err()\n                        {\n                            trc::event!(\n                                Store(trc::StoreEvent::LdapWarning),\n                                Reason = \"Secret rejected during auth bind using template\",\n                                Details = dn\n                            );\n                            return Ok(None);\n                        }\n\n                        let filter = self.mappings.filter_name.build(username);\n                        let result = if *can_search {\n                            self.find_principal(&mut ldap, &filter).await\n                        } else {\n                            self.find_principal(&mut conn, &filter).await\n                        };\n\n                        match result {\n                            Ok(Some(mut result)) => {\n                                if result.principal.name.is_empty() {\n                                    result.principal.name = username.into();\n                                }\n                                (result.principal, result.member_of, None)\n                            }\n                            Err(err)\n                                if err\n                                    .matches(trc::EventType::Store(trc::StoreEvent::LdapError))\n                                    && err\n                                        .value(trc::Key::Code)\n                                        .and_then(|v| v.to_uint())\n                                        .is_some_and(|rc| [49, 50].contains(&rc)) =>\n                            {\n                                trc::event!(\n                                    Store(trc::StoreEvent::LdapWarning),\n                                    Reason = \"Error codes 49 or 50 returned by LDAP server\",\n                                    Details = vec![dn, filter]\n                                );\n                                return Ok(None);\n                            }\n                            Ok(None) => {\n                                trc::event!(\n                                    Store(trc::StoreEvent::LdapWarning),\n                                    Reason = \"Auth bind successful but filter yielded no results\",\n                                    Details = vec![dn, filter]\n                                );\n\n                                return Ok(None);\n                            }\n                            Err(err) => return Err(err),\n                        }\n                    }\n                    AuthBind::Lookup => {\n                        let filter = self.mappings.filter_name.build(username);\n                        if let Some(mut result) = self.find_principal(&mut conn, &filter).await? {\n                            // Perform bind auth using the found dn\n                            let (auth_bind_conn, mut ldap) = LdapConnAsync::with_settings(\n                                self.pool.manager().settings.clone(),\n                                &self.pool.manager().address,\n                            )\n                            .await\n                            .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n\n                            ldap3::drive!(auth_bind_conn);\n\n                            if ldap\n                                .simple_bind(&result.dn, secret)\n                                .await\n                                .map_err(|err| err.into_error().caused_by(trc::location!()))?\n                                .success()\n                                .is_ok()\n                            {\n                                if result.principal.name.is_empty() {\n                                    result.principal.name = username.into();\n                                }\n                                (result.principal, result.member_of, None)\n                            } else {\n                                trc::event!(\n                                    Store(trc::StoreEvent::LdapWarning),\n                                    Reason = \"Secret rejected during auth bind using lookup filter\",\n                                    Details = vec![result.dn, filter]\n                                );\n                                return Ok(None);\n                            }\n                        } else {\n                            trc::event!(\n                                Store(trc::StoreEvent::LdapWarning),\n                                Reason = \"Auth bind lookup filter yielded no results\",\n                                Details = filter\n                            );\n                            return Ok(None);\n                        }\n                    }\n                    AuthBind::None => {\n                        let filter = self.mappings.filter_name.build(username);\n                        if let Some(mut result) = self.find_principal(&mut conn, &filter).await? {\n                            if result.principal.verify_secret(secret, false, false).await? {\n                                if result.principal.name.is_empty() {\n                                    result.principal.name = username.into();\n                                }\n                                (result.principal, result.member_of, None)\n                            } else {\n                                trc::event!(\n                                    Store(trc::StoreEvent::LdapWarning),\n                                    Reason = \"Password verification failed\",\n                                    Details = vec![result.dn, filter]\n                                );\n                                return Ok(None);\n                            }\n                        } else {\n                            trc::event!(\n                                Store(trc::StoreEvent::LdapWarning),\n                                Reason = \"Authentication filter yielded no results\",\n                                Details = filter\n                            );\n                            return Ok(None);\n                        }\n                    }\n                }\n            }\n        };\n\n        // Query groups\n        if !member_of.is_empty() && by.return_member_of {\n            for mut name in member_of {\n                if name.contains('=') {\n                    let (rs, _res) = conn\n                        .search(\n                            &name,\n                            Scope::Base,\n                            \"objectClass=*\",\n                            &self.mappings.attr_name,\n                        )\n                        .await\n                        .map_err(|err| err.into_error().caused_by(trc::location!()))?\n                        .success()\n                        .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n                    for entry in rs {\n                        'outer: for (attr, value) in SearchEntry::construct(entry).attrs {\n                            if self.mappings.attr_name.contains(&attr.to_lowercase())\n                                && let Some(group) = value.into_iter().next()\n                                && !group.is_empty()\n                            {\n                                name = group;\n                                break 'outer;\n                            }\n                        }\n                    }\n                }\n\n                let account_id = self\n                    .data_store\n                    .get_or_create_principal_id(&name, Type::Group)\n                    .await\n                    .caused_by(trc::location!())?;\n\n                external_principal\n                    .data\n                    .push(PrincipalData::MemberOf(account_id));\n            }\n        }\n\n        // Obtain account ID if not available\n        let mut principal = if let Some(stored_principal) = stored_principal {\n            stored_principal\n        } else {\n            let id = self\n                .data_store\n                .get_or_create_principal_id(external_principal.name(), Type::Individual)\n                .await\n                .caused_by(trc::location!())?;\n\n            self.data_store\n                .query(QueryParams::id(id).with_return_member_of(by.return_member_of))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| manage::not_found(id).caused_by(trc::location!()))?\n        };\n\n        // Keep the internal store up to date with the LDAP server\n        let changes = principal.update_external(external_principal);\n        if !changes.is_empty() {\n            self.data_store\n                .update_principal(\n                    UpdatePrincipal::by_id(principal.id)\n                        .with_updates(changes)\n                        .create_domains(),\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(Some(principal))\n    }\n\n    pub async fn email_to_id(&self, address: &str) -> trc::Result<Option<u32>> {\n        let filter = self.mappings.filter_email.build(address.as_ref());\n        let rs = self\n            .pool\n            .get()\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?\n            .search(\n                &self.mappings.base_dn,\n                Scope::Subtree,\n                &filter,\n                &self.mappings.attr_name,\n            )\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?\n            .success()\n            .map(|(rs, _res)| rs)\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n\n        trc::event!(\n            Store(trc::StoreEvent::LdapQuery),\n            Details = filter,\n            Result = rs.iter().map(result_to_trace).collect::<Vec<_>>()\n        );\n\n        for entry in rs {\n            for (attr, value) in SearchEntry::construct(entry).attrs {\n                if self.mappings.attr_name.contains(&attr.to_lowercase())\n                    && let Some(name) = value.into_iter().find(|name| !name.is_empty())\n                {\n                    return self\n                        .data_store\n                        .get_or_create_principal_id(&name, Type::Individual)\n                        .await\n                        .map(Some);\n                }\n            }\n        }\n\n        Ok(None)\n    }\n\n    pub async fn rcpt(&self, address: &str) -> trc::Result<RcptType> {\n        let filter = self.mappings.filter_email.build(address.as_ref());\n        let result = self\n            .pool\n            .get()\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?\n            .streaming_search(\n                &self.mappings.base_dn,\n                Scope::Subtree,\n                &filter,\n                &self.mappings.attr_email_address,\n            )\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?\n            .next()\n            .await\n            .map(|entry| {\n                let result = if entry.is_some() {\n                    RcptType::Mailbox\n                } else {\n                    RcptType::Invalid\n                };\n\n                trc::event!(\n                    Store(trc::StoreEvent::LdapQuery),\n                    Details = filter,\n                    Result = entry.as_ref().map(result_to_trace).unwrap_or_default()\n                );\n\n                result\n            })\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n\n        if result != RcptType::Invalid {\n            Ok(result)\n        } else {\n            self.data_store.rcpt(address).await.map(|result| {\n                if matches!(result, RcptType::List(_)) {\n                    result\n                } else {\n                    RcptType::Invalid\n                }\n            })\n        }\n    }\n\n    pub async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.data_store.vrfy(address).await\n    }\n\n    pub async fn expn(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.data_store.expn(address).await\n    }\n\n    pub async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        self.data_store.is_local_domain(domain).await\n    }\n}\n\nimpl LdapDirectory {\n    async fn find_principal(\n        &self,\n        conn: &mut Ldap,\n        filter: &str,\n    ) -> trc::Result<Option<LdapResult>> {\n        conn.search(\n            &self.mappings.base_dn,\n            Scope::Subtree,\n            filter,\n            &self.mappings.attrs_principal,\n        )\n        .await\n        .map_err(|err| err.into_error().caused_by(trc::location!()))?\n        .success()\n        .map(|(rs, _)| {\n            trc::event!(\n                Store(trc::StoreEvent::LdapQuery),\n                Details = filter.to_string(),\n                Result = rs.first().map(result_to_trace).unwrap_or_default()\n            );\n\n            rs.into_iter().next().map(|entry| {\n                self.mappings\n                    .entry_to_principal(SearchEntry::construct(entry))\n            })\n        })\n        .map_err(|err| err.into_error().caused_by(trc::location!()))\n    }\n}\n\nstruct LdapResult {\n    dn: String,\n    principal: Principal,\n    member_of: Vec<String>,\n}\n\nimpl LdapMappings {\n    fn entry_to_principal(&self, entry: SearchEntry) -> LdapResult {\n        let mut principal = Principal::new(0, Type::Individual);\n        let mut role = ROLE_USER;\n        let mut member_of = vec![];\n        let mut description = None;\n        let mut secret = None;\n        let mut otp_secret = None;\n        let mut email = None;\n        let mut email_aliases = Vec::new();\n\n        for (attr, value) in entry.attrs {\n            let attr = attr.to_lowercase();\n            if self.attr_name.contains(&attr) {\n                if !self.attr_email_address.contains(&attr) {\n                    principal.name = value.into_iter().next().unwrap_or_default();\n                } else {\n                    for (idx, item) in value.into_iter().enumerate() {\n                        if email.is_none() {\n                            email = Some(item.to_lowercase());\n                        }\n\n                        if idx == 0 {\n                            principal.name = item;\n                        }\n                    }\n                }\n            } else if self.attr_secret.contains(&attr) {\n                for item in value {\n                    if item.is_otp_secret() {\n                        otp_secret = Some(item);\n                    } else if item.is_app_secret() {\n                        principal.data.push(PrincipalData::AppPassword(item));\n                    } else if secret.is_none() {\n                        secret = Some(item);\n                    }\n                }\n            } else if self.attr_secret_changed.contains(&attr) {\n                // Create a disabled AppPassword, used to indicate that the password has been changed\n                // but cannot be used for authentication.\n                if secret.is_none() {\n                    secret = value.into_iter().next().map(|item| {\n                        format!(\"$app${}$\", xxhash_rust::xxh3::xxh3_64(item.as_bytes()))\n                    });\n                }\n            } else if self.attr_email_address.contains(&attr) {\n                for item in value {\n                    if email.is_some() {\n                        email_aliases.push(item.to_lowercase());\n                    } else {\n                        email = Some(item.to_lowercase());\n                    }\n                }\n            } else if self.attr_email_alias.contains(&attr) {\n                for item in value {\n                    email_aliases.push(item.to_lowercase());\n                }\n            } else if let Some(idx) = self.attr_description.iter().position(|a| a == &attr) {\n                if (description.is_none() || idx == 0)\n                    && let Some(desc) = value.into_iter().next()\n                {\n                    description = Some(desc);\n                }\n            } else if self.attr_groups.contains(&attr) {\n                member_of.extend(value);\n            } else if self.attr_quota.contains(&attr) {\n                if let Ok(quota) = value.into_iter().next().unwrap_or_default().parse::<u64>()\n                    && quota > 0\n                {\n                    principal.data.push(PrincipalData::DiskQuota(quota));\n                }\n            } else if self.attr_type.contains(&attr) {\n                for value in value {\n                    match value.to_ascii_lowercase().as_str() {\n                        \"admin\" | \"administrator\" | \"root\" | \"superuser\" => {\n                            role = ROLE_ADMIN;\n                            principal.typ = Type::Individual\n                        }\n                        \"posixaccount\" | \"individual\" | \"person\" | \"inetorgperson\" => {\n                            principal.typ = Type::Individual\n                        }\n                        \"posixgroup\" | \"groupofuniquenames\" | \"group\" => {\n                            principal.typ = Type::Group\n                        }\n                        _ => continue,\n                    }\n                    break;\n                }\n            }\n        }\n\n        for alias in email_aliases {\n            if email.as_ref().is_none_or(|email| email != &alias) {\n                principal.data.push(PrincipalData::EmailAlias(alias));\n            }\n        }\n\n        if let Some(email) = email {\n            principal.data.push(PrincipalData::PrimaryEmail(email));\n        }\n\n        if let Some(secret) = secret {\n            principal.data.push(PrincipalData::Password(secret));\n        }\n\n        if let Some(otp_secret) = otp_secret {\n            principal.data.push(PrincipalData::OtpAuth(otp_secret));\n        }\n\n        if let Some(desc) = description {\n            principal.data.push(PrincipalData::Description(desc));\n        }\n\n        principal.data.push(PrincipalData::Role(role));\n\n        LdapResult {\n            dn: entry.dn,\n            principal,\n            member_of,\n        }\n    }\n}\n\nfn result_to_trace(rs: &ResultEntry) -> trc::Value {\n    let se = SearchEntry::construct(rs.clone());\n    se.attrs\n        .into_iter()\n        .map(|(k, v)| trc::Value::Array(vec![trc::Value::from(k), trc::Value::from(v.join(\", \"))]))\n        .chain([trc::Value::from(se.dn)])\n        .collect::<Vec<_>>()\n        .into()\n}\n"
  },
  {
    "path": "crates/directory/src/backend/ldap/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse deadpool::managed::Pool;\nuse ldap3::{LdapConnSettings, ldap_escape};\nuse store::Store;\n\npub mod config;\npub mod lookup;\npub mod pool;\n\npub struct LdapDirectory {\n    pool: Pool<LdapConnectionManager>,\n    mappings: LdapMappings,\n    auth_bind: AuthBind,\n    pub(crate) data_store: Store,\n}\n\n#[derive(Debug, Default)]\npub struct LdapMappings {\n    base_dn: String,\n    filter_name: LdapFilter,\n    filter_email: LdapFilter,\n    attr_name: Vec<String>,\n    attr_type: Vec<String>,\n    attr_groups: Vec<String>,\n    attr_description: Vec<String>,\n    attr_secret: Vec<String>,\n    attr_secret_changed: Vec<String>,\n    attr_email_address: Vec<String>,\n    attr_email_alias: Vec<String>,\n    attr_quota: Vec<String>,\n    attrs_principal: Vec<String>,\n}\n\n#[derive(Debug, Default)]\npub(crate) struct LdapFilter {\n    filter: Vec<LdapFilterItem>,\n}\n\n#[derive(Debug)]\nenum LdapFilterItem {\n    Static(String),\n    Full,\n    LocalPart,\n    DomainPart,\n}\n\nimpl LdapFilter {\n    pub fn build(&self, value: &str) -> String {\n        let mut result = String::with_capacity(value.len() + 16);\n\n        for item in &self.filter {\n            match item {\n                LdapFilterItem::Static(s) => result.push_str(s),\n                LdapFilterItem::Full => result.push_str(ldap_escape(value).as_ref()),\n                LdapFilterItem::LocalPart => {\n                    result.push_str(\n                        value\n                            .rsplit_once('@')\n                            .map(|(local, _)| local)\n                            .unwrap_or(value),\n                    );\n                }\n                LdapFilterItem::DomainPart => {\n                    if let Some((_, domain)) = value.rsplit_once('@') {\n                        result.push_str(domain);\n                    }\n                }\n            }\n        }\n\n        result\n    }\n}\n\npub(crate) struct LdapConnectionManager {\n    address: String,\n    settings: LdapConnSettings,\n    bind_dn: Option<Bind>,\n}\n\npub(crate) struct Bind {\n    dn: String,\n    password: String,\n}\n\nimpl LdapConnectionManager {\n    pub fn new(address: String, settings: LdapConnSettings, bind_dn: Option<Bind>) -> Self {\n        Self {\n            address,\n            settings,\n            bind_dn,\n        }\n    }\n}\n\nimpl Bind {\n    pub fn new(dn: String, password: String) -> Self {\n        Self { dn, password }\n    }\n}\n\npub(crate) enum AuthBind {\n    Template {\n        template: LdapFilter,\n        can_search: bool,\n    },\n    Lookup,\n    None,\n}\n"
  },
  {
    "path": "crates/directory/src/backend/ldap/pool.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse async_trait::async_trait;\nuse deadpool::managed;\nuse ldap3::{Ldap, LdapConnAsync, LdapError, exop::WhoAmI};\n\nuse super::LdapConnectionManager;\n\n#[async_trait]\nimpl managed::Manager for LdapConnectionManager {\n    type Type = Ldap;\n    type Error = LdapError;\n\n    async fn create(&self) -> Result<Ldap, LdapError> {\n        let (conn, mut ldap) =\n            LdapConnAsync::with_settings(self.settings.clone(), &self.address).await?;\n\n        ldap3::drive!(conn);\n\n        if let Some(bind) = &self.bind_dn {\n            ldap.simple_bind(&bind.dn, &bind.password)\n                .await?\n                .success()?;\n        }\n\n        Ok(ldap)\n    }\n\n    async fn recycle(\n        &self,\n        conn: &mut Ldap,\n        _: &managed::Metrics,\n    ) -> managed::RecycleResult<LdapError> {\n        conn.extended(WhoAmI)\n            .await\n            .map(|_| ())\n            .map_err(managed::RecycleError::Backend)\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/memory/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse store::Store;\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::{\n    Principal, PrincipalData, ROLE_ADMIN, ROLE_USER, Type,\n    backend::internal::manage::ManageDirectory,\n};\n\nuse super::{EmailType, MemoryDirectory};\n\nimpl MemoryDirectory {\n    pub async fn from_config(\n        config: &mut Config,\n        prefix: impl AsKey,\n        data_store: Store,\n    ) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let mut directory = MemoryDirectory {\n            data_store,\n            principals: Default::default(),\n            emails_to_ids: Default::default(),\n            domains: Default::default(),\n        };\n\n        for lookup_id in config.sub_keys((prefix.as_str(), \"principals\"), \".name\") {\n            let lookup_id = lookup_id.as_str();\n            let name = config\n                .value_require((prefix.as_str(), \"principals\", lookup_id, \"name\"))?\n                .to_string();\n            let (typ, is_superuser) =\n                match config.value((prefix.as_str(), \"principals\", lookup_id, \"class\")) {\n                    Some(\"individual\") => (Type::Individual, false),\n                    Some(\"admin\") => (Type::Individual, true),\n                    Some(\"group\") => (Type::Group, false),\n                    _ => (Type::Individual, false),\n                };\n\n            // Obtain id\n            let id = directory\n                .data_store\n                .get_or_create_principal_id(&name, Type::Individual)\n                .await\n                .map_err(|err| {\n                    config.new_build_error(\n                        prefix.as_str(),\n                        format!(\n                            \"Failed to obtain id for principal {} ({}): {:?}\",\n                            name, lookup_id, err\n                        ),\n                    )\n                })\n                .ok()?;\n\n            // Create principal\n            let mut principal = Principal::new(id, typ);\n            principal.data.push(PrincipalData::Role(if is_superuser {\n                ROLE_ADMIN\n            } else {\n                ROLE_USER\n            }));\n\n            // Obtain group ids\n            for group in config\n                .values((prefix.as_str(), \"principals\", lookup_id, \"member-of\"))\n                .map(|(_, s)| s.to_string())\n                .collect::<Vec<_>>()\n            {\n                principal.data.push(PrincipalData::MemberOf(\n                    directory\n                        .data_store\n                        .get_or_create_principal_id(&group, Type::Group)\n                        .await\n                        .map_err(|err| {\n                            config.new_build_error(\n                                prefix.as_str(),\n                                format!(\n                                    \"Failed to obtain id for principal {} ({}): {:?}\",\n                                    name, lookup_id, err\n                                ),\n                            )\n                        })\n                        .ok()?,\n                ));\n            }\n\n            // Parse email addresses\n            for (pos, (_, email)) in config\n                .values((prefix.as_str(), \"principals\", lookup_id, \"email\"))\n                .enumerate()\n            {\n                directory\n                    .emails_to_ids\n                    .entry(email.to_string())\n                    .or_default()\n                    .push(if pos > 0 {\n                        EmailType::Alias(id)\n                    } else {\n                        EmailType::Primary(id)\n                    });\n\n                if let Some((_, domain)) = email.rsplit_once('@') {\n                    directory.domains.insert(domain.to_lowercase());\n                }\n\n                if pos == 0 {\n                    principal\n                        .data\n                        .push(PrincipalData::PrimaryEmail(email.to_lowercase()));\n                } else {\n                    principal\n                        .data\n                        .push(PrincipalData::EmailAlias(email.to_lowercase()));\n                }\n            }\n\n            // Parse mailing lists\n            for (_, email) in\n                config.values((prefix.as_str(), \"principals\", lookup_id, \"email-list\"))\n            {\n                directory\n                    .emails_to_ids\n                    .entry(email.to_lowercase())\n                    .or_default()\n                    .push(EmailType::List(id));\n                if let Some((_, domain)) = email.rsplit_once('@') {\n                    directory.domains.insert(domain.to_lowercase());\n                }\n            }\n\n            principal.name = name.as_str().into();\n            for (_, secret) in config.values((prefix.as_str(), \"principals\", lookup_id, \"secret\")) {\n                principal.data.push(PrincipalData::Password(secret.into()));\n            }\n            if let Some(description) =\n                config.value((prefix.as_str(), \"principals\", lookup_id, \"description\"))\n            {\n                principal\n                    .data\n                    .push(PrincipalData::Description(description.into()));\n            }\n            if let Some(quota) =\n                config.property::<u64>((prefix.as_str(), \"principals\", lookup_id, \"quota\"))\n            {\n                principal.data.push(PrincipalData::DiskQuota(quota));\n            }\n\n            directory.principals.push(principal);\n        }\n\n        Some(directory)\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/memory/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{EmailType, MemoryDirectory};\nuse crate::{Principal, QueryBy, QueryParams, backend::RcptType};\n\nuse mail_send::Credentials;\n\nimpl MemoryDirectory {\n    pub async fn query(&self, by: QueryParams<'_>) -> trc::Result<Option<Principal>> {\n        match by.by {\n            QueryBy::Name(name) => {\n                for principal in &self.principals {\n                    if principal.name() == name {\n                        return Ok(Some(principal.clone()));\n                    }\n                }\n            }\n            QueryBy::Id(uid) => {\n                for principal in &self.principals {\n                    if principal.id == uid {\n                        return Ok(Some(principal.clone()));\n                    }\n                }\n            }\n            QueryBy::Credentials(credentials) => {\n                let (username, secret) = match credentials {\n                    Credentials::Plain { username, secret } => (username, secret),\n                    Credentials::OAuthBearer { token } => (token, token),\n                    Credentials::XOauth2 { username, secret } => (username, secret),\n                };\n\n                for principal in &self.principals {\n                    if principal.name() == username {\n                        return if principal.verify_secret(secret, false, false).await? {\n                            Ok(Some(principal.clone()))\n                        } else {\n                            Ok(None)\n                        };\n                    }\n                }\n            }\n        }\n        Ok(None)\n    }\n\n    pub async fn email_to_id(&self, address: &str) -> trc::Result<Option<u32>> {\n        Ok(self.emails_to_ids.get(address).and_then(|names| {\n            names\n                .iter()\n                .map(|t| match t {\n                    EmailType::Primary(uid) | EmailType::Alias(uid) | EmailType::List(uid) => *uid,\n                })\n                .next()\n        }))\n    }\n\n    pub async fn rcpt(&self, address: &str) -> trc::Result<RcptType> {\n        Ok(self.emails_to_ids.contains_key(address).into())\n    }\n\n    pub async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>> {\n        let mut result = Vec::new();\n        for (key, value) in &self.emails_to_ids {\n            if key.contains(address) && value.iter().any(|t| matches!(t, EmailType::Primary(_))) {\n                result.push(key.into())\n            }\n        }\n        Ok(result)\n    }\n\n    pub async fn expn(&self, address: &str) -> trc::Result<Vec<String>> {\n        let mut result = Vec::new();\n        for (key, value) in &self.emails_to_ids {\n            if key == address {\n                for item in value {\n                    if let EmailType::List(uid) = item {\n                        for principal in &self.principals {\n                            if principal.id == *uid {\n                                if let Some(addr) = principal.primary_email() {\n                                    result.push(addr.to_string())\n                                }\n                                break;\n                            }\n                        }\n                    }\n                }\n            }\n        }\n        Ok(result)\n    }\n\n    pub async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        Ok(self.domains.contains(domain))\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/memory/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::{AHashMap, AHashSet};\nuse store::Store;\n\nuse crate::Principal;\n\npub mod config;\npub mod lookup;\n\n#[derive(Debug)]\npub struct MemoryDirectory {\n    principals: Vec<Principal>,\n    emails_to_ids: AHashMap<String, Vec<EmailType>>,\n    pub(crate) data_store: Store,\n    domains: AHashSet<String>,\n}\n\n#[derive(Debug)]\nenum EmailType {\n    Primary(u32),\n    Alias(u32),\n    List(u32),\n}\n"
  },
  {
    "path": "crates/directory/src/backend/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod imap;\npub mod internal;\npub mod ldap;\npub mod memory;\npub mod oidc;\npub mod smtp;\npub mod sql;\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, Hash)]\npub enum RcptType {\n    Mailbox,\n    List(Vec<String>),\n    #[default]\n    Invalid,\n}\n\nimpl From<bool> for RcptType {\n    fn from(value: bool) -> Self {\n        if value {\n            RcptType::Mailbox\n        } else {\n            RcptType::Invalid\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/oidc/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse base64::{Engine, engine::general_purpose};\nuse store::Store;\nuse utils::config::{Config, utils::AsKey};\n\nuse super::{Authentication, EndpointType, OpenIdConfig, OpenIdDirectory};\n\nimpl OpenIdDirectory {\n    pub fn from_config(config: &mut Config, prefix: impl AsKey, data_store: Store) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let endpoint_type = match config.value_require((&prefix, \"endpoint.method\"))? {\n            \"introspect\" => match config.value_require((&prefix, \"auth.method\"))? {\n                #[allow(clippy::to_string_in_format_args)]\n                \"basic\" => EndpointType::Introspect(Authentication::Header(format!(\n                    \"Basic {}\",\n                    general_purpose::STANDARD.encode(\n                        format!(\n                            \"{}:{}\",\n                            config\n                                .value_require((&prefix, \"auth.username\"))?\n                                .to_string(),\n                            config.value_require((&prefix, \"auth.secret\"))?\n                        )\n                        .as_bytes()\n                    )\n                ))),\n                \"token\" => EndpointType::Introspect(Authentication::Header(format!(\n                    \"Bearer {}\",\n                    config.value_require((&prefix, \"auth.token\"))?\n                ))),\n                \"user-token\" => EndpointType::Introspect(Authentication::Bearer),\n                \"none\" => EndpointType::Introspect(Authentication::None),\n                _ => {\n                    config.new_build_error(\n                        (&prefix, \"auth.method\"),\n                        \"Invalid authentication method, must be 'header', 'bearer' or 'none'\",\n                    );\n                    return None;\n                }\n            },\n            \"userinfo\" => EndpointType::UserInfo,\n            _ => {\n                config.new_build_error(\n                    (&prefix, \"endpoint.method\"),\n                    \"Invalid endpoint method, must be 'introspect' or 'userinfo'\",\n                );\n                return None;\n            }\n        };\n\n        let email_field = config.value_require((&prefix, \"fields.email\"))?.to_string();\n\n        Some(OpenIdDirectory {\n            config: OpenIdConfig {\n                endpoint: config.value_require((&prefix, \"endpoint.url\"))?.to_string(),\n                endpoint_type,\n                endpoint_timeout: config\n                    .property_or_default::<Duration>((&prefix, \"timeout\"), \"30s\")\n                    .unwrap_or_else(|| Duration::from_secs(30)),\n                username_field: config\n                    .value((&prefix, \"fields.username\"))\n                    .filter(|&v| v != email_field)\n                    .map(|v| v.to_string()),\n                email_field,\n                full_name_field: config\n                    .value((&prefix, \"fields.full-name\"))\n                    .map(|v| v.to_string()),\n            },\n            data_store,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/oidc/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::HashMap;\n\nuse mail_send::Credentials;\nuse reqwest::{StatusCode, header::AUTHORIZATION};\nuse trc::{AddContext, AuthEvent};\n\nuse crate::{\n    Principal, PrincipalData, QueryBy, QueryParams, ROLE_USER, Type,\n    backend::{\n        RcptType,\n        internal::{\n            lookup::DirectoryStore,\n            manage::{self, ManageDirectory, UpdatePrincipal},\n        },\n        oidc::{Authentication, EndpointType},\n    },\n};\n\nuse super::{OpenIdConfig, OpenIdDirectory};\n\ntype OpenIdResponse = HashMap<String, serde_json::Value>;\n\nimpl OpenIdDirectory {\n    pub async fn query(&self, by: QueryParams<'_>) -> trc::Result<Option<Principal>> {\n        match &by.by {\n            QueryBy::Credentials(Credentials::OAuthBearer { token }) => {\n                // Send request\n                #[cfg(feature = \"test_mode\")]\n                let client = reqwest::Client::builder().danger_accept_invalid_certs(true);\n\n                #[cfg(not(feature = \"test_mode\"))]\n                let client = reqwest::Client::builder();\n\n                let client = client\n                    .timeout(self.config.endpoint_timeout)\n                    .build()\n                    .map_err(|err| {\n                        AuthEvent::Error\n                            .into_err()\n                            .reason(err)\n                            .details(\"Failed to build client\")\n                    })?;\n\n                let client = match &self.config.endpoint_type {\n                    EndpointType::UserInfo => client.get(&self.config.endpoint).bearer_auth(token),\n                    EndpointType::Introspect(authentication) => {\n                        let client = client.post(&self.config.endpoint).form(&[\n                            (\"token\", token.as_str()),\n                            (\"token_type_hint\", \"access_token\"),\n                        ]);\n                        match authentication {\n                            Authentication::Header(header) => client.header(AUTHORIZATION, header),\n                            Authentication::Bearer => client.bearer_auth(token),\n                            Authentication::None => client,\n                        }\n                    }\n                };\n\n                let response = client.send().await.map_err(|err| {\n                    AuthEvent::Error\n                        .into_err()\n                        .reason(err)\n                        .details(\"HTTP request failed\")\n                })?;\n\n                match response.status() {\n                    StatusCode::OK => {\n                        // Fetch response\n                        let response = response.bytes().await.map_err(|err| {\n                            AuthEvent::Error\n                                .into_err()\n                                .reason(err)\n                                .details(\"Failed to read OIDC response\")\n                        })?;\n\n                        // Deserialize response\n                        let external_principal =\n                            serde_json::from_slice::<OpenIdResponse>(&response)\n                                .map_err(|err| {\n                                    AuthEvent::Error\n                                        .into_err()\n                                        .reason(err)\n                                        .details(\"Failed to deserialize OIDC response\")\n                                })?\n                                .build_principal(&self.config)?;\n\n                        // Fetch principal\n                        let id = self\n                            .data_store\n                            .get_or_create_principal_id(external_principal.name(), Type::Individual)\n                            .await\n                            .caused_by(trc::location!())?;\n                        let mut principal = self\n                            .data_store\n                            .query(QueryParams::id(id).with_return_member_of(by.return_member_of))\n                            .await\n                            .caused_by(trc::location!())?\n                            .ok_or_else(|| manage::not_found(id).caused_by(trc::location!()))?;\n\n                        // Keep the internal store up to date with the OIDC server\n                        let changes = principal.update_external(external_principal);\n                        if !changes.is_empty() {\n                            self.data_store\n                                .update_principal(\n                                    UpdatePrincipal::by_id(principal.id)\n                                        .with_updates(changes)\n                                        .create_domains(),\n                                )\n                                .await\n                                .caused_by(trc::location!())?;\n                        }\n\n                        Ok(Some(principal))\n                    }\n                    StatusCode::UNAUTHORIZED => Err(trc::AuthEvent::Failed\n                        .into_err()\n                        .code(401)\n                        .details(\"Unauthorized\")),\n                    other => Err(trc::AuthEvent::Error\n                        .into_err()\n                        .code(other.as_u16())\n                        .ctx(trc::Key::Reason, response.text().await.unwrap_or_default())\n                        .details(\"Unexpected status code\")),\n                }\n            }\n            _ => self.data_store.query(by.with_only_app_pass(true)).await,\n        }\n    }\n\n    pub async fn email_to_id(&self, address: &str) -> trc::Result<Option<u32>> {\n        self.data_store.email_to_id(address).await\n    }\n\n    pub async fn rcpt(&self, address: &str) -> trc::Result<RcptType> {\n        self.data_store.rcpt(address).await\n    }\n\n    pub async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.data_store.vrfy(address).await\n    }\n\n    pub async fn expn(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.data_store.expn(address).await\n    }\n\n    pub async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        self.data_store.is_local_domain(domain).await\n    }\n}\n\ntrait BuildPrincipal {\n    fn build_principal(&mut self, config: &OpenIdConfig) -> trc::Result<Principal>;\n    fn take_required_field(&mut self, field: &str) -> trc::Result<String>;\n    fn take_field(&mut self, field: &str) -> Option<String>;\n}\n\nimpl BuildPrincipal for OpenIdResponse {\n    fn build_principal(&mut self, config: &OpenIdConfig) -> trc::Result<Principal> {\n        let email = self\n            .take_required_field(&config.email_field)?\n            .to_lowercase();\n        let username = if let Some(username_field) = &config.username_field {\n            self.take_required_field(username_field)?.to_lowercase()\n        } else {\n            email.clone()\n        };\n        if !email.contains('@') && !email.contains('.') {\n            return Err(AuthEvent::Error\n                .into_err()\n                .details(\"Email field is not valid\")\n                .ctx(trc::Key::Key, email));\n        }\n        let full_name = config\n            .full_name_field\n            .as_ref()\n            .and_then(|field| self.take_field(field));\n\n        // Build principal\n        let mut data = Vec::with_capacity(3);\n        data.push(PrincipalData::PrimaryEmail(email));\n        if let Some(name) = full_name {\n            data.push(PrincipalData::Description(name));\n        }\n        data.push(PrincipalData::Role(ROLE_USER));\n        Ok(Principal {\n            id: u32::MAX,\n            typ: Type::Individual,\n            name: username,\n            data,\n        })\n    }\n\n    fn take_required_field(&mut self, field: &str) -> trc::Result<String> {\n        match self.remove(field) {\n            Some(serde_json::Value::String(value)) if !value.is_empty() => Ok(value),\n            other => Err(trc::AuthEvent::Error\n                .into_err()\n                .details(\"Unexpected field type in OIDC response\")\n                .ctx(trc::Key::Key, field.to_string())\n                .ctx(\n                    trc::Key::Value,\n                    serde_json::to_string(&other.unwrap_or(serde_json::Value::Null))\n                        .unwrap_or_default(),\n                )),\n        }\n    }\n\n    fn take_field(&mut self, field: &str) -> Option<String> {\n        match self.remove(field) {\n            Some(serde_json::Value::String(value)) if !value.is_empty() => Some(value),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/oidc/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod config;\npub mod lookup;\n\nuse std::time::Duration;\n\nuse store::Store;\n\npub struct OpenIdDirectory {\n    config: OpenIdConfig,\n    pub(crate) data_store: Store,\n}\n\nstruct OpenIdConfig {\n    pub endpoint: String,\n    pub endpoint_type: EndpointType,\n    pub endpoint_timeout: Duration,\n    pub email_field: String,\n    pub username_field: Option<String>,\n    pub full_name_field: Option<String>,\n}\n\n#[derive(Debug)]\npub enum EndpointType {\n    Introspect(Authentication),\n    UserInfo,\n}\n\n#[derive(Debug)]\npub enum Authentication {\n    Header(String),\n    Bearer,\n    None,\n}\n"
  },
  {
    "path": "crates/directory/src/backend/smtp/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse mail_send::{SmtpClientBuilder, smtp::tls::build_tls_connector};\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::core::config::build_pool;\n\nuse super::{SmtpConnectionManager, SmtpDirectory};\n\nimpl SmtpDirectory {\n    pub fn from_config(config: &mut Config, prefix: impl AsKey, is_lmtp: bool) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let address = config.value_require((&prefix, \"host\"))?.to_string();\n        let tls_implicit: bool = config\n            .property_or_default((&prefix, \"tls.enable\"), \"false\")\n            .unwrap_or_default();\n        let port: u16 = config\n            .property_or_default((&prefix, \"port\"), if tls_implicit { \"465\" } else { \"25\" })\n            .unwrap_or(if tls_implicit { 465 } else { 25 });\n\n        let manager = SmtpConnectionManager {\n            builder: SmtpClientBuilder {\n                addr: format!(\"{address}:{port}\"),\n                timeout: config\n                    .property_or_default((&prefix, \"timeout\"), \"30s\")\n                    .unwrap_or_else(|| Duration::from_secs(30)),\n                tls_connector: build_tls_connector(\n                    config\n                        .property_or_default((&prefix, \"tls.allow-invalid-certs\"), \"false\")\n                        .unwrap_or_default(),\n                ),\n                tls_hostname: address.to_string(),\n                tls_implicit,\n                is_lmtp,\n                credentials: None,\n                local_host: config\n                    .value(\"server.hostname\")\n                    .unwrap_or(\"[127.0.0.1]\")\n                    .to_string(),\n                say_ehlo: false,\n                local_ip: None,\n            },\n            max_rcpt: config\n                .property_or_default((&prefix, \"limits.rcpt\"), \"10\")\n                .unwrap_or(10),\n            max_auth_errors: config\n                .property_or_default((&prefix, \"limits.auth-errors\"), \"3\")\n                .unwrap_or(10),\n        };\n\n        Some(SmtpDirectory {\n            pool: build_pool(config, &prefix, manager)\n                .map_err(|e| {\n                    config.new_parse_error(\n                        prefix.as_str(),\n                        format!(\"Failed to build SMTP pool: {e:?}\"),\n                    )\n                })\n                .ok()?,\n            domains: config\n                .values((&prefix, \"lookup.domains\"))\n                .map(|(_, v)| v.to_lowercase())\n                .collect(),\n        })\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/smtp/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mail_send::{Credentials, smtp::AssertReply};\nuse smtp_proto::Severity;\n\nuse crate::{IntoError, Principal, QueryBy, Type, backend::RcptType};\n\nuse super::{SmtpClient, SmtpDirectory};\n\nimpl SmtpDirectory {\n    pub async fn query(&self, query: QueryBy<'_>) -> trc::Result<Option<Principal>> {\n        if let QueryBy::Credentials(credentials) = query {\n            self.pool\n                .get()\n                .await\n                .map_err(|err| err.into_error().caused_by(trc::location!()))?\n                .authenticate(credentials)\n                .await\n        } else {\n            Err(trc::StoreEvent::NotSupported.caused_by(trc::location!()))\n        }\n    }\n\n    pub async fn email_to_id(&self, _address: &str) -> trc::Result<Option<u32>> {\n        Err(trc::StoreEvent::NotSupported.caused_by(trc::location!()))\n    }\n\n    pub async fn rcpt(&self, address: &str) -> trc::Result<RcptType> {\n        let mut conn = self\n            .pool\n            .get()\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n        if !conn.sent_mail_from {\n            conn.client\n                .cmd(b\"MAIL FROM:<>\\r\\n\")\n                .await\n                .map_err(|err| err.into_error().caused_by(trc::location!()))?\n                .assert_positive_completion()\n                .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n            conn.sent_mail_from = true;\n        }\n        let reply = conn\n            .client\n            .cmd(format!(\"RCPT TO:<{address}>\\r\\n\").as_bytes())\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n        match reply.severity() {\n            Severity::PositiveCompletion => {\n                conn.num_rcpts += 1;\n                if conn.num_rcpts >= conn.max_rcpt {\n                    let _ = conn.client.rset().await;\n                    conn.num_rcpts = 0;\n                    conn.sent_mail_from = false;\n                }\n                Ok(RcptType::Mailbox)\n            }\n            Severity::PermanentNegativeCompletion => Ok(RcptType::Invalid),\n            _ => Err(trc::StoreEvent::UnexpectedError\n                .ctx(trc::Key::Code, reply.code())\n                .ctx(trc::Key::Details, reply.message)),\n        }\n    }\n\n    pub async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.pool\n            .get()\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?\n            .expand(&format!(\"VRFY {address}\\r\\n\"))\n            .await\n    }\n\n    pub async fn expn(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.pool\n            .get()\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?\n            .expand(&format!(\"EXPN {address}\\r\\n\"))\n            .await\n    }\n\n    pub async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        Ok(self.domains.contains(domain))\n    }\n}\n\nimpl SmtpClient {\n    async fn authenticate(\n        &mut self,\n        credentials: &Credentials<String>,\n    ) -> trc::Result<Option<Principal>> {\n        match self\n            .client\n            .authenticate(credentials, &self.capabilities)\n            .await\n        {\n            Ok(_) => Ok(Some(Principal::new(u32::MAX, Type::Individual))),\n            Err(err) => match &err {\n                mail_send::Error::AuthenticationFailed(err) if err.code() == 535 => {\n                    self.num_auth_failures += 1;\n                    Ok(None)\n                }\n                _ => Err(err.into_error()),\n            },\n        }\n    }\n\n    async fn expand(&mut self, command: &str) -> trc::Result<Vec<String>> {\n        let reply = self\n            .client\n            .cmd(command.as_bytes())\n            .await\n            .map_err(|err| err.into_error().caused_by(trc::location!()))?;\n        match reply.code() {\n            250 | 251 => Ok(reply\n                .message()\n                .split('\\n')\n                .map(|p| p.into())\n                .collect::<Vec<String>>()),\n            code @ (550 | 551 | 553 | 500 | 502) => {\n                Err(trc::StoreEvent::NotSupported.ctx(trc::Key::Code, code))\n            }\n            code => Err(trc::StoreEvent::UnexpectedError\n                .ctx(trc::Key::Code, code)\n                .ctx(trc::Key::Details, reply.message)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/smtp/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod config;\npub mod lookup;\npub mod pool;\n\nuse ahash::AHashSet;\nuse deadpool::managed::Pool;\nuse mail_send::SmtpClientBuilder;\nuse smtp_proto::EhloResponse;\nuse tokio::net::TcpStream;\nuse tokio_rustls::client::TlsStream;\n\npub struct SmtpDirectory {\n    pool: Pool<SmtpConnectionManager>,\n    domains: AHashSet<String>,\n}\n\npub struct SmtpConnectionManager {\n    builder: SmtpClientBuilder<String>,\n    max_rcpt: usize,\n    max_auth_errors: usize,\n}\n\npub struct SmtpClient {\n    client: mail_send::SmtpClient<TlsStream<TcpStream>>,\n    capabilities: EhloResponse<String>,\n    max_rcpt: usize,\n    max_auth_errors: usize,\n    num_rcpts: usize,\n    num_auth_failures: usize,\n    sent_mail_from: bool,\n}\n"
  },
  {
    "path": "crates/directory/src/backend/smtp/pool.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse async_trait::async_trait;\nuse deadpool::managed;\nuse mail_send::{Error, smtp::AssertReply};\n\nuse super::{SmtpClient, SmtpConnectionManager};\n\n#[async_trait]\nimpl managed::Manager for SmtpConnectionManager {\n    type Type = SmtpClient;\n    type Error = Error;\n\n    async fn create(&self) -> Result<SmtpClient, Error> {\n        let mut client = self.builder.connect().await?;\n        let capabilities = client\n            .capabilities(&self.builder.local_host, self.builder.is_lmtp)\n            .await?;\n\n        Ok(SmtpClient {\n            capabilities,\n            client,\n            max_auth_errors: self.max_auth_errors,\n            max_rcpt: self.max_rcpt,\n            num_rcpts: 0,\n            num_auth_failures: 0,\n            sent_mail_from: false,\n        })\n    }\n\n    async fn recycle(\n        &self,\n        conn: &mut SmtpClient,\n        _: &managed::Metrics,\n    ) -> managed::RecycleResult<Error> {\n        if conn.num_auth_failures < conn.max_auth_errors {\n            conn.client\n                .cmd(b\"NOOP\\r\\n\")\n                .await?\n                .assert_positive_completion()\n                .map(|_| ())\n                .map_err(managed::RecycleError::Backend)\n        } else {\n            Err(managed::RecycleError::Message(\n                \"No longer valid: Too many authentication failures\".to_string(),\n            ))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/sql/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse store::{Store, Stores};\nuse utils::config::{Config, utils::AsKey};\n\nuse super::{SqlDirectory, SqlMappings};\n\nimpl SqlDirectory {\n    pub fn from_config(\n        config: &mut Config,\n        prefix: impl AsKey,\n        stores: &Stores,\n        data_store: Store,\n    ) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let store_id = config.value_require((&prefix, \"store\"))?.to_string();\n        let sql_store =\n            if let Some(sql_store) = stores.stores.get(&store_id).filter(|store| store.is_sql()) {\n                sql_store.clone()\n            } else {\n                let err = format!(\"Directory references a non-existent store {store_id:?}\");\n                config.new_build_error((&prefix, \"store\"), err);\n                return None;\n            };\n\n        let mut mappings = SqlMappings {\n            column_description: config\n                .value((&prefix, \"columns.description\"))\n                .unwrap_or_default()\n                .to_string(),\n            column_secret: config\n                .value((&prefix, \"columns.secret\"))\n                .unwrap_or_default()\n                .to_string(),\n            column_email: config\n                .value((&prefix, \"columns.email\"))\n                .unwrap_or_default()\n                .to_string(),\n            column_quota: config\n                .value((&prefix, \"columns.quota\"))\n                .unwrap_or_default()\n                .to_string(),\n            column_type: config\n                .value((&prefix, \"columns.class\"))\n                .unwrap_or_default()\n                .to_string(),\n            ..Default::default()\n        };\n\n        for (query_id, query) in [\n            (\"name\", &mut mappings.query_name),\n            (\"members\", &mut mappings.query_members),\n            (\"emails\", &mut mappings.query_emails),\n            (\"recipients\", &mut mappings.query_recipients),\n            (\"secrets\", &mut mappings.query_secrets),\n        ] {\n            *query = config\n                .value((\"store\", store_id.as_str(), \"query\", query_id))\n                .unwrap_or_default()\n                .to_string();\n        }\n\n        Some(SqlDirectory {\n            sql_store,\n            mappings,\n            data_store,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/sql/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{SqlDirectory, SqlMappings};\nuse crate::{\n    Principal, PrincipalData, QueryBy, QueryParams, ROLE_ADMIN, ROLE_USER, Type,\n    backend::{\n        RcptType,\n        internal::{\n            SpecialSecrets,\n            lookup::DirectoryStore,\n            manage::{self, ManageDirectory, UpdatePrincipal},\n        },\n    },\n};\nuse mail_send::Credentials;\nuse store::{NamedRows, Rows, Value};\nuse trc::AddContext;\n\nimpl SqlDirectory {\n    pub async fn query(&self, by: QueryParams<'_>) -> trc::Result<Option<Principal>> {\n        let (external_principal, stored_principal) = match by.by {\n            QueryBy::Name(username) => (\n                self.mappings\n                    .row_to_principal(\n                        self.sql_store\n                            .sql_query::<NamedRows>(\n                                &self.mappings.query_name,\n                                vec![username.into()],\n                            )\n                            .await\n                            .caused_by(trc::location!())?,\n                    )\n                    .caused_by(trc::location!())?\n                    .map(|mut p| {\n                        p.name = username.into();\n                        p\n                    }),\n                None,\n            ),\n            QueryBy::Id(uid) => {\n                if let Some(principal) = self\n                    .data_store\n                    .query(QueryParams::id(uid).with_return_member_of(by.return_member_of))\n                    .await\n                    .caused_by(trc::location!())?\n                {\n                    (\n                        self.mappings\n                            .row_to_principal(\n                                self.sql_store\n                                    .sql_query::<NamedRows>(\n                                        &self.mappings.query_name,\n                                        vec![principal.name().into()],\n                                    )\n                                    .await\n                                    .caused_by(trc::location!())?,\n                            )\n                            .caused_by(trc::location!())?,\n                        Some(principal),\n                    )\n                } else {\n                    return Ok(None);\n                }\n            }\n            QueryBy::Credentials(credentials) => {\n                let (username, secret) = match credentials {\n                    Credentials::Plain { username, secret } => (username, secret),\n                    Credentials::OAuthBearer { token } => (token, token),\n                    Credentials::XOauth2 { username, secret } => (username, secret),\n                };\n\n                match self\n                    .mappings\n                    .row_to_principal(\n                        self.sql_store\n                            .sql_query::<NamedRows>(\n                                &self.mappings.query_name,\n                                vec![username.into()],\n                            )\n                            .await\n                            .caused_by(trc::location!())?,\n                    )\n                    .caused_by(trc::location!())?\n                {\n                    Some(mut principal) => {\n                        // Obtain secrets\n                        if !self.mappings.query_secrets.is_empty() {\n                            let secrets = self\n                                .sql_store\n                                .sql_query::<Rows>(\n                                    &self.mappings.query_secrets,\n                                    vec![username.into()],\n                                )\n                                .await\n                                .caused_by(trc::location!())?;\n\n                            for row in secrets.rows {\n                                for value in row.values {\n                                    if let Value::Text(secret) = value {\n                                        let secret = secret.into_owned();\n\n                                        if secret.is_otp_secret() {\n                                            if !principal.data.iter().any(|data| {\n                                                matches!(data, PrincipalData::OtpAuth(_))\n                                            }) {\n                                                principal.data.push(PrincipalData::OtpAuth(secret));\n                                            }\n                                        } else if secret.is_app_secret() {\n                                            principal.data.push(PrincipalData::AppPassword(secret));\n                                        } else if !principal\n                                            .data\n                                            .iter()\n                                            .any(|data| matches!(data, PrincipalData::Password(_)))\n                                        {\n                                            principal.data.push(PrincipalData::Password(secret));\n                                        }\n                                    }\n                                }\n                            }\n                        }\n\n                        if principal\n                            .verify_secret(secret, false, false)\n                            .await\n                            .caused_by(trc::location!())?\n                        {\n                            principal.name = username.into();\n                            (Some(principal), None)\n                        } else {\n                            (None, None)\n                        }\n                    }\n\n                    _ => (None, None),\n                }\n            }\n        };\n\n        let mut external_principal = if let Some(external_principal) = external_principal {\n            external_principal\n        } else {\n            return Ok(None);\n        };\n\n        // Obtain members\n        if by.return_member_of && !self.mappings.query_members.is_empty() {\n            for row in self\n                .sql_store\n                .sql_query::<Rows>(\n                    &self.mappings.query_members,\n                    vec![external_principal.name().into()],\n                )\n                .await\n                .caused_by(trc::location!())?\n                .rows\n            {\n                if let Some(Value::Text(account_name)) = row.values.first() {\n                    let account_id = self\n                        .data_store\n                        .get_or_create_principal_id(account_name, Type::Group)\n                        .await\n                        .caused_by(trc::location!())?;\n                    external_principal\n                        .data\n                        .push(PrincipalData::MemberOf(account_id));\n                }\n            }\n        }\n\n        // Obtain emails\n        if !self.mappings.query_emails.is_empty() {\n            let mut rows = self\n                .sql_store\n                .sql_query::<Rows>(\n                    &self.mappings.query_emails,\n                    vec![external_principal.name().into()],\n                )\n                .await\n                .caused_by(trc::location!())?\n                .rows\n                .into_iter()\n                .flat_map(|v| v.values.into_iter().map(|v| v.into_lower_string()));\n\n            if external_principal.primary_email().is_none()\n                && let Some(email) = rows.next()\n            {\n                external_principal\n                    .data\n                    .push(PrincipalData::PrimaryEmail(email));\n            }\n\n            external_principal\n                .data\n                .extend(rows.map(PrincipalData::EmailAlias));\n        }\n\n        // Obtain account ID if not available\n        let mut principal = if let Some(stored_principal) = stored_principal {\n            stored_principal\n        } else {\n            let id = self\n                .data_store\n                .get_or_create_principal_id(external_principal.name(), Type::Individual)\n                .await\n                .caused_by(trc::location!())?;\n\n            self.data_store\n                .query(QueryParams::id(id).with_return_member_of(by.return_member_of))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| manage::not_found(id).caused_by(trc::location!()))?\n        };\n\n        // Keep the internal store up to date with the SQL server\n        let changes = principal.update_external(external_principal);\n        if !changes.is_empty() {\n            self.data_store\n                .update_principal(\n                    UpdatePrincipal::by_id(principal.id)\n                        .with_updates(changes)\n                        .create_domains(),\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(Some(principal))\n    }\n\n    pub async fn email_to_id(&self, address: &str) -> trc::Result<Option<u32>> {\n        let names = self\n            .sql_store\n            .sql_query::<Rows>(&self.mappings.query_recipients, vec![address.into()])\n            .await\n            .caused_by(trc::location!())?;\n\n        for row in names.rows {\n            if let Some(Value::Text(name)) = row.values.first() {\n                return self\n                    .data_store\n                    .get_or_create_principal_id(name, Type::Individual)\n                    .await\n                    .caused_by(trc::location!())\n                    .map(Some);\n            }\n        }\n\n        Ok(None)\n    }\n\n    pub async fn rcpt(&self, address: &str) -> trc::Result<RcptType> {\n        let result = self\n            .sql_store\n            .sql_query::<bool>(\n                &self.mappings.query_recipients,\n                vec![address.to_string().into()],\n            )\n            .await?;\n\n        if result {\n            Ok(RcptType::Mailbox)\n        } else {\n            self.data_store.rcpt(address).await.map(|result| {\n                if matches!(result, RcptType::List(_)) {\n                    result\n                } else {\n                    RcptType::Invalid\n                }\n            })\n        }\n    }\n\n    pub async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.data_store.vrfy(address).await\n    }\n\n    pub async fn expn(&self, address: &str) -> trc::Result<Vec<String>> {\n        self.data_store.expn(address).await\n    }\n\n    pub async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        self.data_store.is_local_domain(domain).await\n    }\n}\n\nimpl SqlMappings {\n    pub fn row_to_principal(&self, rows: NamedRows) -> trc::Result<Option<Principal>> {\n        if rows.rows.is_empty() {\n            return Ok(None);\n        }\n\n        let mut principal = Principal::new(u32::MAX, Type::Individual);\n        let mut role = ROLE_USER;\n        let mut has_primary_email = false;\n        let mut secret = None;\n\n        if let Some(row) = rows.rows.into_iter().next() {\n            for (name, value) in rows.names.into_iter().zip(row.values) {\n                if name.eq_ignore_ascii_case(&self.column_secret) {\n                    if let Value::Text(text) = value {\n                        secret = Some(text.into_owned());\n                    }\n                } else if name.eq_ignore_ascii_case(&self.column_type) {\n                    match value.to_str().as_ref() {\n                        \"individual\" | \"person\" | \"user\" => {\n                            principal.typ = Type::Individual;\n                        }\n                        \"group\" => principal.typ = Type::Group,\n                        \"admin\" | \"superuser\" | \"administrator\" => {\n                            principal.typ = Type::Individual;\n                            role = ROLE_ADMIN;\n                        }\n                        _ => (),\n                    }\n                } else if name.eq_ignore_ascii_case(&self.column_description) {\n                    if let Value::Text(text) = value {\n                        principal\n                            .data\n                            .push(PrincipalData::Description(text.as_ref().into()));\n                    }\n                } else if name.eq_ignore_ascii_case(&self.column_email) {\n                    if let Value::Text(text) = value {\n                        if !has_primary_email {\n                            has_primary_email = true;\n                            principal\n                                .data\n                                .push(PrincipalData::PrimaryEmail(text.to_lowercase()));\n                        } else {\n                            principal\n                                .data\n                                .push(PrincipalData::EmailAlias(text.to_lowercase()));\n                        }\n                    }\n                } else if name.eq_ignore_ascii_case(&self.column_quota)\n                    && let Value::Integer(quota) = value\n                    && quota > 0\n                {\n                    principal.data.push(PrincipalData::DiskQuota(quota as u64));\n                }\n            }\n        }\n\n        if let Some(secret) = secret {\n            principal.data.push(PrincipalData::Password(secret));\n        }\n\n        principal.data.push(PrincipalData::Role(role));\n\n        Ok(Some(principal))\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/backend/sql/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse store::Store;\n\npub mod config;\npub mod lookup;\n\npub struct SqlDirectory {\n    sql_store: Store,\n    mappings: SqlMappings,\n    pub(crate) data_store: Store,\n}\n\n#[derive(Debug, Default)]\npub(crate) struct SqlMappings {\n    query_name: String,\n    query_members: String,\n    query_emails: String,\n    query_recipients: String,\n    query_secrets: String,\n    column_description: String,\n    column_secret: String,\n    column_email: String,\n    column_quota: String,\n    column_type: String,\n}\n"
  },
  {
    "path": "crates/directory/src/core/cache.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse utils::{\n    cache::CacheWithTtl,\n    config::{Config, utils::AsKey},\n};\n\nuse crate::backend::RcptType;\n\npub struct CachedDirectory {\n    cached_domains: CacheWithTtl<String, bool>,\n    cached_rcpts: CacheWithTtl<String, bool>,\n    ttl_pos: Duration,\n    ttl_neg: Duration,\n}\n\nimpl CachedDirectory {\n    pub fn try_from_config(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let cached_size = config\n            .property_or_default::<Option<u64>>((&prefix, \"cache.size\"), \"1048576\")\n            .unwrap_or_default()?;\n\n        Some(CachedDirectory {\n            cached_domains: CacheWithTtl::new(50, cached_size),\n            cached_rcpts: CacheWithTtl::new(100, cached_size),\n            ttl_pos: config\n                .property((&prefix, \"cache.ttl.positive\"))\n                .unwrap_or(Duration::from_secs(86400)),\n            ttl_neg: config\n                .property((&prefix, \"cache.ttl.negative\"))\n                .unwrap_or_else(|| Duration::from_secs(3600)),\n        })\n    }\n\n    pub fn get_rcpt(&self, address: &str) -> Option<RcptType> {\n        self.cached_rcpts.get(address).map(Into::into)\n    }\n\n    pub fn set_rcpt(&self, address: &str, exists: &RcptType) {\n        let (exists, ttl) = match exists {\n            RcptType::Mailbox => (true, self.ttl_pos),\n            RcptType::Invalid => (false, self.ttl_neg),\n            RcptType::List(_) => return,\n        };\n\n        self.cached_rcpts.insert(address.to_string(), exists, ttl);\n    }\n\n    pub fn get_domain(&self, domain: &str) -> Option<bool> {\n        self.cached_domains.get(domain)\n    }\n\n    pub fn set_domain(&self, domain: &str, exists: bool) {\n        self.cached_domains.insert(\n            domain.to_string(),\n            exists,\n            if exists { self.ttl_pos } else { self.ttl_neg },\n        );\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/core/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse deadpool::{\n    Runtime,\n    managed::{Manager, Pool},\n};\nuse std::{sync::Arc, time::Duration};\nuse store::{Store, Stores};\nuse utils::config::Config;\n\nuse ahash::AHashMap;\n\nuse crate::{\n    Directories, Directory, DirectoryInner,\n    backend::{\n        imap::ImapDirectory, ldap::LdapDirectory, memory::MemoryDirectory, oidc::OpenIdDirectory,\n        smtp::SmtpDirectory, sql::SqlDirectory,\n    },\n};\n\nuse super::cache::CachedDirectory;\n\nimpl Directories {\n    pub async fn parse(\n        config: &mut Config,\n        stores: &Stores,\n        data_store: Store,\n        is_enterprise: bool,\n    ) -> Self {\n        let mut directories = AHashMap::new();\n\n        for id in config.sub_keys(\"directory\", \".type\") {\n            // Parse directory\n            let id = id.as_str();\n            #[cfg(feature = \"test_mode\")]\n            {\n                if config\n                    .property_or_default::<bool>((\"directory\", id, \"disable\"), \"false\")\n                    .unwrap_or(false)\n                {\n                    continue;\n                }\n            }\n            let protocol = config\n                .value_require((\"directory\", id, \"type\"))\n                .unwrap()\n                .to_string();\n            let prefix = (\"directory\", id);\n            let store = match protocol.as_str() {\n                \"internal\" => Some(DirectoryInner::Internal(\n                    if let Some(store_id) = config.value_require((\"directory\", id, \"store\")) {\n                        if let Some(data) = stores.stores.get(store_id) {\n                            data.clone()\n                        } else {\n                            config.new_parse_error(\n                                (\"directory\", id, \"store\"),\n                                \"Store does not exist\",\n                            );\n                            continue;\n                        }\n                    } else {\n                        continue;\n                    },\n                )),\n                \"ldap\" => LdapDirectory::from_config(config, prefix, data_store.clone())\n                    .map(DirectoryInner::Ldap),\n                \"sql\" => SqlDirectory::from_config(config, prefix, stores, data_store.clone())\n                    .map(DirectoryInner::Sql),\n                \"imap\" => ImapDirectory::from_config(config, prefix).map(DirectoryInner::Imap),\n                \"smtp\" => {\n                    SmtpDirectory::from_config(config, prefix, false).map(DirectoryInner::Smtp)\n                }\n                \"lmtp\" => {\n                    SmtpDirectory::from_config(config, prefix, true).map(DirectoryInner::Smtp)\n                }\n                \"memory\" => MemoryDirectory::from_config(config, prefix, data_store.clone())\n                    .await\n                    .map(DirectoryInner::Memory),\n                \"oidc\" => OpenIdDirectory::from_config(config, prefix, data_store.clone())\n                    .map(DirectoryInner::OpenId),\n                unknown => {\n                    let err = format!(\"Unknown directory type: {unknown:?}\");\n                    config.new_parse_error((\"directory\", id, \"type\"), err);\n                    continue;\n                }\n            };\n\n            // Build directory\n            if let Some(store) = store {\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                if store.is_enterprise_directory() && !is_enterprise {\n                    let message =\n                        format!(\"Directory {protocol:?} is an Enterprise Edition feature\");\n                    config.new_parse_error((\"directory\", id, \"type\"), message);\n                    continue;\n                }\n                // SPDX-SnippetEnd\n\n                let directory = Arc::new(Directory {\n                    store,\n                    cache: CachedDirectory::try_from_config(config, (\"directory\", id)),\n                });\n\n                // Add directory\n                directories.insert(id.to_string(), directory);\n            }\n        }\n\n        Directories { directories }\n    }\n}\n\npub(crate) fn build_pool<M: Manager>(\n    config: &mut Config,\n    prefix: &str,\n    manager: M,\n) -> Result<Pool<M>, String> {\n    Pool::builder(manager)\n        .runtime(Runtime::Tokio1)\n        .max_size(\n            config\n                .property_or_default((prefix, \"pool.max-connections\"), \"10\")\n                .unwrap_or(10),\n        )\n        .create_timeout(\n            config\n                .property_or_default::<Duration>((prefix, \"pool.timeout.create\"), \"30s\")\n                .unwrap_or_else(|| Duration::from_secs(30))\n                .into(),\n        )\n        .wait_timeout(config.property_or_default((prefix, \"pool.timeout.wait\"), \"30s\"))\n        .recycle_timeout(config.property_or_default((prefix, \"pool.timeout.recycle\"), \"30s\"))\n        .build()\n        .map_err(|err| {\n            format!(\n                \"Failed to build pool for {prefix:?}: {err}\",\n                prefix = prefix,\n                err = err\n            )\n        })\n}\n"
  },
  {
    "path": "crates/directory/src/core/dispatch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse trc::AddContext;\n\nuse crate::{\n    Directory, DirectoryInner, Principal, QueryParams,\n    backend::{RcptType, internal::lookup::DirectoryStore},\n};\n\nimpl Directory {\n    pub async fn query(&self, by: QueryParams<'_>) -> trc::Result<Option<Principal>> {\n        match &self.store {\n            DirectoryInner::Internal(store) => store.query(by).await,\n            DirectoryInner::Ldap(store) => store.query(by).await,\n            DirectoryInner::Sql(store) => store.query(by).await,\n            DirectoryInner::Imap(store) => store.query(by.by).await,\n            DirectoryInner::Smtp(store) => store.query(by.by).await,\n            DirectoryInner::Memory(store) => store.query(by).await,\n            DirectoryInner::OpenId(store) => store.query(by).await,\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn email_to_id(&self, address: &str) -> trc::Result<Option<u32>> {\n        match &self.store {\n            DirectoryInner::Internal(store) => store.email_to_id(address).await,\n            DirectoryInner::Ldap(store) => store.email_to_id(address).await,\n            DirectoryInner::Sql(store) => store.email_to_id(address).await,\n            DirectoryInner::Imap(store) => store.email_to_id(address).await,\n            DirectoryInner::Smtp(store) => store.email_to_id(address).await,\n            DirectoryInner::Memory(store) => store.email_to_id(address).await,\n            DirectoryInner::OpenId(store) => store.email_to_id(address).await,\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn is_local_domain(&self, domain: &str) -> trc::Result<bool> {\n        // Check cache\n        if let Some(cache) = &self.cache\n            && let Some(result) = cache.get_domain(domain)\n        {\n            return Ok(result);\n        }\n\n        let result = match &self.store {\n            DirectoryInner::Internal(store) => store.is_local_domain(domain).await,\n            DirectoryInner::Ldap(store) => store.is_local_domain(domain).await,\n            DirectoryInner::Sql(store) => store.is_local_domain(domain).await,\n            DirectoryInner::Imap(store) => store.is_local_domain(domain).await,\n            DirectoryInner::Smtp(store) => store.is_local_domain(domain).await,\n            DirectoryInner::Memory(store) => store.is_local_domain(domain).await,\n            DirectoryInner::OpenId(store) => store.is_local_domain(domain).await,\n        }\n        .caused_by(trc::location!())?;\n\n        // Update cache\n        if let Some(cache) = &self.cache {\n            cache.set_domain(domain, result);\n        }\n\n        Ok(result)\n    }\n\n    pub async fn rcpt(&self, email: &str) -> trc::Result<RcptType> {\n        // Check cache\n        if let Some(cache) = &self.cache\n            && let Some(result) = cache.get_rcpt(email)\n        {\n            return Ok(result);\n        }\n\n        let result = match &self.store {\n            DirectoryInner::Internal(store) => store.rcpt(email).await,\n            DirectoryInner::Ldap(store) => store.rcpt(email).await,\n            DirectoryInner::Sql(store) => store.rcpt(email).await,\n            DirectoryInner::Imap(store) => store.rcpt(email).await,\n            DirectoryInner::Smtp(store) => store.rcpt(email).await,\n            DirectoryInner::Memory(store) => store.rcpt(email).await,\n            DirectoryInner::OpenId(store) => store.rcpt(email).await,\n        }\n        .caused_by(trc::location!())?;\n\n        // Update cache\n        if let Some(cache) = &self.cache {\n            cache.set_rcpt(email, &result);\n        }\n\n        Ok(result)\n    }\n\n    pub async fn vrfy(&self, address: &str) -> trc::Result<Vec<String>> {\n        match &self.store {\n            DirectoryInner::Internal(store) => store.vrfy(address).await,\n            DirectoryInner::Ldap(store) => store.vrfy(address).await,\n            DirectoryInner::Sql(store) => store.vrfy(address).await,\n            DirectoryInner::Imap(store) => store.vrfy(address).await,\n            DirectoryInner::Smtp(store) => store.vrfy(address).await,\n            DirectoryInner::Memory(store) => store.vrfy(address).await,\n            DirectoryInner::OpenId(store) => store.vrfy(address).await,\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn expn(&self, address: &str) -> trc::Result<Vec<String>> {\n        match &self.store {\n            DirectoryInner::Internal(store) => store.expn(address).await,\n            DirectoryInner::Ldap(store) => store.expn(address).await,\n            DirectoryInner::Sql(store) => store.expn(address).await,\n            DirectoryInner::Imap(store) => store.expn(address).await,\n            DirectoryInner::Smtp(store) => store.expn(address).await,\n            DirectoryInner::Memory(store) => store.expn(address).await,\n            DirectoryInner::OpenId(store) => store.expn(address).await,\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub fn has_bearer_token_support(&self) -> bool {\n        match &self.store {\n            DirectoryInner::Internal(_)\n            | DirectoryInner::Ldap(_)\n            | DirectoryInner::Sql(_)\n            | DirectoryInner::Imap(_)\n            | DirectoryInner::Smtp(_)\n            | DirectoryInner::Memory(_) => false,\n            DirectoryInner::OpenId(_) => true,\n        }\n    }\n}\n\nimpl DirectoryInner {\n    pub fn is_enterprise_directory(&self) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/core/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::Permission;\n\npub mod cache;\npub mod config;\npub mod dispatch;\npub mod principal;\npub mod secret;\n\nimpl Permission {\n    pub fn description(&self) -> &'static str {\n        match self {\n            Permission::Impersonate => \"Act on behalf of another user\",\n            Permission::UnlimitedRequests => \"Perform unlimited requests\",\n            Permission::UnlimitedUploads => \"Upload unlimited data\",\n            Permission::DeleteSystemFolders_ => \"\",\n            Permission::MessageQueueList => \"View message queue\",\n            Permission::MessageQueueGet => \"Retrieve specific messages from the queue\",\n            Permission::MessageQueueUpdate => \"Modify queued messages\",\n            Permission::MessageQueueDelete => \"Remove messages from the queue\",\n            Permission::OutgoingReportList => \"View outgoing DMARC and TLS reports\",\n            Permission::OutgoingReportGet => \"Retrieve specific outgoing DMARC and TLS reports\",\n            Permission::OutgoingReportDelete => \"Remove outgoing DMARC and TLS reports\",\n            Permission::IncomingReportList => \"View incoming DMARC, TLS and ARF reports\",\n            Permission::IncomingReportGet => {\n                \"Retrieve specific incoming DMARC, TLS and ARF reports\"\n            }\n            Permission::IncomingReportDelete => \"Remove incoming DMARC, TLS and ARF reports\",\n            Permission::SettingsList => \"View system settings\",\n            Permission::SettingsUpdate => \"Modify system settings\",\n            Permission::SettingsDelete => \"Remove system settings\",\n            Permission::SettingsReload => \"Refresh system settings\",\n            Permission::IndividualList => \"View list of user accounts\",\n            Permission::IndividualGet => \"Retrieve specific account information\",\n            Permission::IndividualUpdate => \"Modify user account information\",\n            Permission::IndividualDelete => \"Remove user accounts\",\n            Permission::IndividualCreate => \"Add new user accounts\",\n            Permission::GroupList => \"View list of user groups\",\n            Permission::GroupGet => \"Retrieve specific group information\",\n            Permission::GroupUpdate => \"Modify group information\",\n            Permission::GroupDelete => \"Remove user groups\",\n            Permission::GroupCreate => \"Add new user groups\",\n            Permission::DomainList => \"View list of email domains\",\n            Permission::DomainGet => \"Retrieve specific domain information\",\n            Permission::DomainCreate => \"Add new email domains\",\n            Permission::DomainUpdate => \"Modify domain information\",\n            Permission::DomainDelete => \"Remove email domains\",\n            Permission::TenantList => \"View list of tenants\",\n            Permission::TenantGet => \"Retrieve specific tenant information\",\n            Permission::TenantCreate => \"Add new tenants\",\n            Permission::TenantUpdate => \"Modify tenant information\",\n            Permission::TenantDelete => \"Remove tenants\",\n            Permission::MailingListList => \"View list of mailing lists\",\n            Permission::MailingListGet => \"Retrieve specific mailing list information\",\n            Permission::MailingListCreate => \"Create new mailing lists\",\n            Permission::MailingListUpdate => \"Modify mailing list information\",\n            Permission::MailingListDelete => \"Remove mailing lists\",\n            Permission::RoleList => \"View list of roles\",\n            Permission::RoleGet => \"Retrieve specific role information\",\n            Permission::RoleCreate => \"Create new roles\",\n            Permission::RoleUpdate => \"Modify role information\",\n            Permission::RoleDelete => \"Remove roles\",\n            Permission::PrincipalList => \"View list of principals\",\n            Permission::PrincipalGet => \"Retrieve specific principal information\",\n            Permission::PrincipalCreate => \"Create new principals\",\n            Permission::PrincipalUpdate => \"Modify principal information\",\n            Permission::PrincipalDelete => \"Remove principals\",\n            Permission::BlobFetch => \"Retrieve arbitrary blobs\",\n            Permission::PurgeBlobStore => \"Purge the blob storage\",\n            Permission::PurgeDataStore => \"Purge the data storage\",\n            Permission::PurgeInMemoryStore => \"Purge the in-memory storage\",\n            Permission::PurgeAccount => \"Purge user accounts\",\n            Permission::FtsReindex => \"Rebuild the full-text search index\",\n            Permission::Undelete => \"Restore deleted items\",\n            Permission::DkimSignatureCreate => \"Create DKIM signatures for email authentication\",\n            Permission::DkimSignatureGet => \"Retrieve DKIM signature information\",\n            Permission::SpamFilterUpdate => \"Modify spam filter settings\",\n            Permission::WebadminUpdate => \"Modify web admin interface settings\",\n            Permission::LogsView => \"Access system logs\",\n            Permission::SpamFilterTrain => \"Train the spam filter\",\n            Permission::SpamFilterTest => \"Test the spam filter\",\n            Permission::Restart => \"Restart the email server\",\n            Permission::TracingList => \"View stored traces\",\n            Permission::TracingGet => \"Retrieve specific trace information\",\n            Permission::TracingLive => \"Perform real-time tracing\",\n            Permission::MetricsList => \"View stored metrics\",\n            Permission::MetricsLive => \"View real-time metrics\",\n            Permission::Authenticate => \"Authenticate\",\n            Permission::AuthenticateOauth => \"Authenticate via OAuth\",\n            Permission::EmailSend => \"Send emails\",\n            Permission::EmailReceive => \"Receive emails\",\n            Permission::ManageEncryption => \"Manage encryption-at-rest settings\",\n            Permission::ManagePasswords => \"Manage account passwords\",\n            Permission::JmapEmailGet => \"Retrieve emails via JMAP\",\n            Permission::JmapMailboxGet => \"Retrieve mailboxes via JMAP\",\n            Permission::JmapThreadGet => \"Retrieve email threads via JMAP\",\n            Permission::JmapIdentityGet => \"Retrieve user identities via JMAP\",\n            Permission::JmapEmailSubmissionGet => \"Retrieve email submission info via JMAP\",\n            Permission::JmapPushSubscriptionGet => \"Retrieve push subscriptions via JMAP\",\n            Permission::JmapSieveScriptGet => \"Retrieve Sieve scripts via JMAP\",\n            Permission::JmapVacationResponseGet => \"Retrieve vacation responses via JMAP\",\n            Permission::JmapPrincipalGet => \"Retrieve principal information via JMAP\",\n            Permission::JmapQuotaGet => \"Retrieve quota information via JMAP\",\n            Permission::JmapBlobGet => \"Retrieve blobs via JMAP\",\n            Permission::JmapEmailSet => \"Modify emails via JMAP\",\n            Permission::JmapMailboxSet => \"Modify mailboxes via JMAP\",\n            Permission::JmapIdentitySet => \"Modify user identities via JMAP\",\n            Permission::JmapEmailSubmissionSet => \"Modify email submission settings via JMAP\",\n            Permission::JmapPushSubscriptionSet => \"Modify push subscriptions via JMAP\",\n            Permission::JmapSieveScriptSet => \"Modify Sieve scripts via JMAP\",\n            Permission::JmapVacationResponseSet => \"Modify vacation responses via JMAP\",\n            Permission::JmapEmailChanges => \"Track email changes via JMAP\",\n            Permission::JmapMailboxChanges => \"Track mailbox changes via JMAP\",\n            Permission::JmapThreadChanges => \"Track thread changes via JMAP\",\n            Permission::JmapIdentityChanges => \"Track identity changes via JMAP\",\n            Permission::JmapEmailSubmissionChanges => \"Track email submission changes via JMAP\",\n            Permission::JmapQuotaChanges => \"Track quota changes via JMAP\",\n            Permission::JmapEmailCopy => \"Copy emails via JMAP\",\n            Permission::JmapBlobCopy => \"Copy blobs via JMAP\",\n            Permission::JmapEmailImport => \"Import emails via JMAP\",\n            Permission::JmapEmailParse => \"Parse emails via JMAP\",\n            Permission::JmapEmailQueryChanges => \"Track email query changes via JMAP\",\n            Permission::JmapMailboxQueryChanges => \"Track mailbox query changes via JMAP\",\n            Permission::JmapEmailSubmissionQueryChanges => {\n                \"Track email submission query changes via JMAP\"\n            }\n            Permission::JmapSieveScriptQueryChanges => \"Track Sieve script query changes via JMAP\",\n            Permission::JmapPrincipalQueryChanges => \"Track principal query changes via JMAP\",\n            Permission::JmapQuotaQueryChanges => \"Track quota query changes via JMAP\",\n            Permission::JmapEmailQuery => \"Perform email queries via JMAP\",\n            Permission::JmapMailboxQuery => \"Perform mailbox queries via JMAP\",\n            Permission::JmapEmailSubmissionQuery => \"Perform email submission queries via JMAP\",\n            Permission::JmapSieveScriptQuery => \"Perform Sieve script queries via JMAP\",\n            Permission::JmapPrincipalQuery => \"Perform principal queries via JMAP\",\n            Permission::JmapQuotaQuery => \"Perform quota queries via JMAP\",\n            Permission::JmapSearchSnippet => \"Retrieve search snippets via JMAP\",\n            Permission::JmapSieveScriptValidate => \"Validate Sieve scripts via JMAP\",\n            Permission::JmapBlobLookup => \"Look up blobs via JMAP\",\n            Permission::JmapBlobUpload => \"Upload blobs via JMAP\",\n            Permission::JmapEcho => \"Perform JMAP echo requests\",\n            Permission::ImapAuthenticate => \"Authenticate via IMAP\",\n            Permission::ImapAclGet => \"Retrieve ACLs via IMAP\",\n            Permission::ImapAclSet => \"Set ACLs via IMAP\",\n            Permission::ImapMyRights => \"Retrieve own rights via IMAP\",\n            Permission::ImapListRights => \"List rights via IMAP\",\n            Permission::ImapAppend => \"Append messages via IMAP\",\n            Permission::ImapCapability => \"Retrieve server capabilities via IMAP\",\n            Permission::ImapId => \"Retrieve server ID via IMAP\",\n            Permission::ImapCopy => \"Copy messages via IMAP\",\n            Permission::ImapMove => \"Move messages via IMAP\",\n            Permission::ImapCreate => \"Create mailboxes via IMAP\",\n            Permission::ImapDelete => \"Delete mailboxes or messages via IMAP\",\n            Permission::ImapEnable => \"Enable IMAP extensions\",\n            Permission::ImapExpunge => \"Expunge deleted messages via IMAP\",\n            Permission::ImapFetch => \"Fetch messages or metadata via IMAP\",\n            Permission::ImapIdle => \"Use IMAP IDLE command\",\n            Permission::ImapList => \"List mailboxes via IMAP\",\n            Permission::ImapLsub => \"List subscribed mailboxes via IMAP\",\n            Permission::ImapNamespace => \"Retrieve namespaces via IMAP\",\n            Permission::ImapRename => \"Rename mailboxes via IMAP\",\n            Permission::ImapSearch => \"Search messages via IMAP\",\n            Permission::ImapSort => \"Sort messages via IMAP\",\n            Permission::ImapSelect => \"Select mailboxes via IMAP\",\n            Permission::ImapExamine => \"Examine mailboxes via IMAP\",\n            Permission::ImapStatus => \"Retrieve mailbox status via IMAP\",\n            Permission::ImapStore => \"Modify message flags via IMAP\",\n            Permission::ImapSubscribe => \"Subscribe to mailboxes via IMAP\",\n            Permission::ImapThread => \"Thread messages via IMAP\",\n            Permission::Pop3Authenticate => \"Authenticate via POP3\",\n            Permission::Pop3List => \"List messages via POP3\",\n            Permission::Pop3Uidl => \"Retrieve unique IDs via POP3\",\n            Permission::Pop3Stat => \"Retrieve mailbox statistics via POP3\",\n            Permission::Pop3Retr => \"Retrieve messages via POP3\",\n            Permission::Pop3Dele => \"Mark messages for deletion via POP3\",\n            Permission::SieveAuthenticate => \"Authenticate for Sieve script management\",\n            Permission::SieveListScripts => \"List Sieve scripts\",\n            Permission::SieveSetActive => \"Set active Sieve script\",\n            Permission::SieveGetScript => \"Retrieve Sieve scripts\",\n            Permission::SievePutScript => \"Upload Sieve scripts\",\n            Permission::SieveDeleteScript => \"Delete Sieve scripts\",\n            Permission::SieveRenameScript => \"Rename Sieve scripts\",\n            Permission::SieveCheckScript => \"Validate Sieve scripts\",\n            Permission::SieveHaveSpace => \"Check available space for Sieve scripts\",\n            Permission::OauthClientRegistration => \"Register OAuth clients\",\n            Permission::OauthClientOverride => \"Override OAuth client settings\",\n            Permission::ApiKeyList => \"View API keys\",\n            Permission::ApiKeyGet => \"Retrieve specific API keys\",\n            Permission::ApiKeyCreate => \"Create new API keys\",\n            Permission::ApiKeyUpdate => \"Modify API keys\",\n            Permission::ApiKeyDelete => \"Remove API keys\",\n            Permission::OauthClientList => \"View OAuth clients\",\n            Permission::OauthClientGet => \"Retrieve specific OAuth clients\",\n            Permission::OauthClientCreate => \"Create new OAuth clients\",\n            Permission::OauthClientUpdate => \"Modify OAuth clients\",\n            Permission::OauthClientDelete => \"Remove OAuth clients\",\n            Permission::AiModelInteract => \"Interact with AI models\",\n            Permission::Troubleshoot => \"Perform troubleshooting\",\n            Permission::DavSyncCollection => \"Synchronize collection changes with client\",\n            Permission::DavPrincipalAcl => \"Set principal properties for access control\",\n            Permission::DavPrincipalMatch => \"Match principals based on specified criteria\",\n            Permission::DavPrincipalSearch => \"Search for principals by property values\",\n            Permission::DavPrincipalSearchPropSet => \"Define property sets for principal searches\",\n            Permission::DavExpandProperty => \"Expand properties that reference other resources\",\n            Permission::DavPrincipalList => \"List available principals in the system\",\n            Permission::DavFilePropFind => \"Retrieve properties of file resources\",\n            Permission::DavFilePropPatch => \"Modify properties of file resources\",\n            Permission::DavFileGet => \"Download file resources\",\n            Permission::DavFileMkCol => \"Create new file collections or directories\",\n            Permission::DavFileDelete => \"Remove file resources\",\n            Permission::DavFilePut => \"Upload or modify file resources\",\n            Permission::DavFileCopy => \"Copy file resources to new locations\",\n            Permission::DavFileMove => \"Move file resources to new locations\",\n            Permission::DavFileLock => \"Lock file resources to prevent concurrent modifications\",\n            Permission::DavFileAcl => \"Manage access control lists for file resources\",\n            Permission::DavCardPropFind => \"Retrieve properties of address book entries\",\n            Permission::DavCardPropPatch => \"Modify properties of address book entries\",\n            Permission::DavCardGet => \"Download address book entries\",\n            Permission::DavCardMkCol => \"Create new address book collections\",\n            Permission::DavCardDelete => \"Remove address book entries or collections\",\n            Permission::DavCardPut => \"Upload or modify address book entries\",\n            Permission::DavCardCopy => \"Copy address book entries to new locations\",\n            Permission::DavCardMove => \"Move address book entries to new locations\",\n            Permission::DavCardLock => {\n                \"Lock address book entries to prevent concurrent modifications\"\n            }\n            Permission::DavCardAcl => \"Manage access control lists for address book entries\",\n            Permission::DavCardQuery => \"Search for address book entries matching criteria\",\n            Permission::DavCardMultiGet => {\n                \"Retrieve multiple address book entries in a single request\"\n            }\n            Permission::DavCalPropFind => \"Retrieve properties of calendar entries\",\n            Permission::DavCalPropPatch => \"Modify properties of calendar entries\",\n            Permission::DavCalGet => \"Download calendar entries\",\n            Permission::DavCalMkCol => \"Create new calendar collections\",\n            Permission::DavCalDelete => \"Remove calendar entries or collections\",\n            Permission::DavCalPut => \"Upload or modify calendar entries\",\n            Permission::DavCalCopy => \"Copy calendar entries to new locations\",\n            Permission::DavCalMove => \"Move calendar entries to new locations\",\n            Permission::DavCalLock => \"Lock calendar entries to prevent concurrent modifications\",\n            Permission::DavCalAcl => \"Manage access control lists for calendar entries\",\n            Permission::DavCalQuery => \"Search for calendar entries matching criteria\",\n            Permission::DavCalMultiGet => \"Retrieve multiple calendar entries in a single request\",\n            Permission::DavCalFreeBusyQuery => \"Query free/busy time information for scheduling\",\n            Permission::CalendarAlarms => \"Receive calendar alarms via e-mail\",\n            Permission::CalendarSchedulingSend => \"Send calendar scheduling requests via e-mail\",\n            Permission::CalendarSchedulingReceive => {\n                \"Receive calendar scheduling requests via e-mail\"\n            }\n            Permission::JmapAddressBookGet => \"Retrieve address books via JMAP\",\n            Permission::JmapAddressBookSet => \"Create or update address books via JMAP\",\n            Permission::JmapAddressBookChanges => \"Track address book changes via JMAP\",\n            Permission::JmapContactCardGet => \"Retrieve contact cards via JMAP\",\n            Permission::JmapContactCardChanges => \"Track contact card changes via JMAP\",\n            Permission::JmapContactCardQuery => {\n                \"Search for contact cards matching criteria via JMAP\"\n            }\n            Permission::JmapContactCardQueryChanges => \"Track contact card query changes via JMAP\",\n            Permission::JmapContactCardSet => \"Create or update contact cards via JMAP\",\n            Permission::JmapContactCardCopy => \"Copy contact cards to new locations via JMAP\",\n            Permission::JmapContactCardParse => \"Parse contact cards via JMAP\",\n            Permission::JmapFileNodeGet => \"Retrieve file nodes via JMAP\",\n            Permission::JmapFileNodeSet => \"Create or update file nodes via JMAP\",\n            Permission::JmapFileNodeChanges => \"Track file node changes via JMAP\",\n            Permission::JmapFileNodeQuery => \"Search for file nodes matching criteria via JMAP\",\n            Permission::JmapFileNodeQueryChanges => \"Track file node query changes via JMAP\",\n            Permission::JmapPrincipalGetAvailability => {\n                \"Retrieve availability information via JMAP\"\n            }\n            Permission::JmapPrincipalChanges => \"Track principal changes via JMAP\",\n            Permission::JmapShareNotificationGet => \"Retrieve share notifications via JMAP\",\n            Permission::JmapShareNotificationSet => \"Create or update share notifications via JMAP\",\n            Permission::JmapShareNotificationChanges => \"Track share notification changes via JMAP\",\n            Permission::JmapShareNotificationQuery => {\n                \"Search for share notifications matching criteria via JMAP\"\n            }\n            Permission::JmapShareNotificationQueryChanges => {\n                \"Track share notification query changes via JMAP\"\n            }\n            Permission::JmapCalendarGet => \"Retrieve calendars via JMAP\",\n            Permission::JmapCalendarSet => \"Create or update calendars via JMAP\",\n            Permission::JmapCalendarChanges => \"Track calendar changes via JMAP\",\n            Permission::JmapCalendarEventGet => \"Retrieve calendar events via JMAP\",\n            Permission::JmapCalendarEventSet => \"Create or update calendar events via JMAP\",\n            Permission::JmapCalendarEventChanges => \"Track calendar event changes via JMAP\",\n            Permission::JmapCalendarEventQuery => {\n                \"Search for calendar events matching criteria via JMAP\"\n            }\n            Permission::JmapCalendarEventQueryChanges => {\n                \"Track calendar event query changes via JMAP\"\n            }\n            Permission::JmapCalendarEventCopy => \"Copy calendar events to new locations via JMAP\",\n            Permission::JmapCalendarEventParse => \"Parse calendar events via JMAP\",\n            Permission::JmapCalendarEventNotificationGet => {\n                \"Retrieve calendar event notifications via JMAP\"\n            }\n            Permission::JmapCalendarEventNotificationSet => {\n                \"Create or update calendar event notifications via JMAP\"\n            }\n            Permission::JmapCalendarEventNotificationChanges => {\n                \"Track calendar event notification changes via JMAP\"\n            }\n            Permission::JmapCalendarEventNotificationQuery => {\n                \"Search for calendar event notifications matching criteria via JMAP\"\n            }\n            Permission::JmapCalendarEventNotificationQueryChanges => {\n                \"Track calendar event notification query changes via JMAP\"\n            }\n            Permission::JmapParticipantIdentityGet => {\n                \"Retrieve participant identity information via JMAP\"\n            }\n            Permission::JmapParticipantIdentitySet => {\n                \"Create or update participant identities via JMAP\"\n            }\n            Permission::JmapParticipantIdentityChanges => {\n                \"Track participant identity changes via JMAP\"\n            }\n        }\n    }\n}\n\n#[cfg(test)]\nmod test {\n    use crate::Permission;\n\n    #[test]\n    #[ignore]\n    #[allow(clippy::obfuscated_if_else)]\n    fn print_permissions() {\n        const CHECK: &str = \":white_check_mark:\";\n\n        let mut permissions = Permission::all().collect::<Vec<_>>();\n        permissions.sort_by(|a, b| a.name().cmp(b.name()));\n\n        for permission in permissions {\n            println!(\n                \"|`{}`|{}|{}|{}|{}|\",\n                permission.name(),\n                permission.description(),\n                CHECK,\n                permission\n                    .is_tenant_admin_permission()\n                    .then_some(CHECK)\n                    .unwrap_or_default(),\n                permission\n                    .is_user_permission()\n                    .then_some(CHECK)\n                    .unwrap_or_default()\n            );\n            //println!(\"({:?},{:?}),\", permission.name(), permission.description(),);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/core/principal.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    ArchivedPrincipal, ArchivedPrincipalData, FALLBACK_ADMIN_ID, Permission, PermissionGrant,\n    Principal, PrincipalData, ROLE_ADMIN, Type,\n    backend::internal::{PrincipalField, PrincipalSet, PrincipalUpdate, PrincipalValue},\n};\nuse ahash::AHashSet;\nuse nlp::tokenizers::word::WordTokenizer;\nuse serde::{\n    Deserializer, Serializer,\n    de::{self, IgnoredAny, Visitor},\n    ser::SerializeMap,\n};\nuse serde_json::Value;\nuse std::{cmp::Ordering, collections::hash_map::Entry, fmt, str::FromStr};\nuse store::{\n    U32_LEN, U64_LEN,\n    backend::MAX_TOKEN_LENGTH,\n    write::{BatchBuilder, DirectoryClass},\n};\n\nimpl Principal {\n    pub fn new(id: u32, typ: Type) -> Self {\n        Self {\n            id,\n            typ,\n            name: \"\".into(),\n            data: Default::default(),\n        }\n    }\n\n    pub fn id(&self) -> u32 {\n        self.id\n    }\n\n    pub fn typ(&self) -> Type {\n        self.typ\n    }\n\n    pub fn name(&self) -> &str {\n        self.name.as_str()\n    }\n\n    pub fn quota(&self) -> Option<u64> {\n        self.data.iter().find_map(|d| {\n            if let PrincipalData::DiskQuota(quota) = d {\n                if *quota > 0 { Some(*quota) } else { None }\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn directory_quota(&self, typ: &Type) -> Option<u32> {\n        self.data.iter().find_map(|d| {\n            if let PrincipalData::DirectoryQuota { quota, typ: qtyp } = d\n                && qtyp == typ\n            {\n                Some(*quota)\n            } else {\n                None\n            }\n        })\n    }\n\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n\n    #[cfg(feature = \"enterprise\")]\n    pub fn tenant(&self) -> Option<u32> {\n        self.data.iter().find_map(|item| {\n            if let PrincipalData::Tenant(tenant) = item {\n                Some(*tenant)\n            } else {\n                None\n            }\n        })\n    }\n    // SPDX-SnippetEnd\n\n    #[cfg(not(feature = \"enterprise\"))]\n    pub fn tenant(&self) -> Option<u32> {\n        None\n    }\n\n    pub fn description(&self) -> Option<&str> {\n        self.data.iter().find_map(|item| {\n            if let PrincipalData::Description(description) = item {\n                if !description.is_empty() {\n                    Some(description.as_str())\n                } else {\n                    None\n                }\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn secret(&self) -> Option<&str> {\n        if let Some(PrincipalData::Password(password)) = self.data.first() {\n            Some(password.as_str())\n        } else if let Some(PrincipalData::Password(password)) = self.data.get(1) {\n            Some(password.as_str())\n        } else {\n            None\n        }\n    }\n\n    pub fn primary_email(&self) -> Option<&str> {\n        self.data.iter().find_map(|item| {\n            if let PrincipalData::PrimaryEmail(email) = item {\n                Some(email.as_str())\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn email_addresses(&self) -> impl Iterator<Item = &str> {\n        let mut found_email = false;\n        self.data\n            .iter()\n            .take_while(move |item| {\n                if matches!(\n                    item,\n                    PrincipalData::PrimaryEmail(_) | PrincipalData::EmailAlias(_)\n                ) {\n                    found_email = true;\n                    true\n                } else {\n                    !found_email\n                }\n            })\n            .filter_map(|item| {\n                if let PrincipalData::PrimaryEmail(email) | PrincipalData::EmailAlias(email) = item\n                {\n                    Some(email.as_str())\n                } else {\n                    None\n                }\n            })\n    }\n\n    pub fn into_primary_email(self) -> Option<String> {\n        self.data.into_iter().find_map(|item| {\n            if let PrincipalData::PrimaryEmail(email) = item {\n                Some(email)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn into_email_addresses(self) -> impl Iterator<Item = String> {\n        self.data.into_iter().filter_map(|item| {\n            if let PrincipalData::PrimaryEmail(email) | PrincipalData::EmailAlias(email) = item {\n                Some(email)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn member_of(&self) -> impl Iterator<Item = u32> {\n        self.data.iter().filter_map(|item| {\n            if let PrincipalData::MemberOf(item) = item {\n                Some(*item)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn roles(&self) -> impl Iterator<Item = u32> {\n        self.data.iter().filter_map(|item| {\n            if let PrincipalData::Role(item) = item {\n                Some(*item)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn permissions(&self) -> impl Iterator<Item = PermissionGrant> {\n        self.data.iter().filter_map(|item| {\n            if let PrincipalData::Permission {\n                permission_id,\n                grant,\n            } = item\n            {\n                Permission::from_id(*permission_id).map(|permission| PermissionGrant {\n                    permission,\n                    grant: *grant,\n                })\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn urls(&self) -> impl Iterator<Item = &String> {\n        self.data.iter().filter_map(|item| {\n            if let PrincipalData::Url(item) = item {\n                Some(item)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn lists(&self) -> impl Iterator<Item = &u32> {\n        self.data.iter().filter_map(|item| {\n            if let PrincipalData::List(item) = item {\n                Some(item)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn picture(&self) -> Option<&String> {\n        self.data.iter().find_map(|item| {\n            if let PrincipalData::Picture(picture) = item {\n                picture.into()\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn picture_mut(&mut self) -> Option<&mut String> {\n        self.data.iter_mut().find_map(|item| {\n            if let PrincipalData::Picture(picture) = item {\n                picture.into()\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn add_permission(&mut self, permission: Permission, grant: bool) {\n        let permission = permission.id();\n        if let Some(permissions) = self.data.iter_mut().find_map(|item| {\n            if let PrincipalData::Permission {\n                permission_id,\n                grant,\n            } = item\n            {\n                if *permission_id == permission {\n                    Some(grant)\n                } else {\n                    None\n                }\n            } else {\n                None\n            }\n        }) {\n            *permissions = grant;\n        } else {\n            self.data.push(PrincipalData::Permission {\n                permission_id: permission,\n                grant,\n            });\n        }\n    }\n\n    pub fn add_permissions(&mut self, iter: impl Iterator<Item = PermissionGrant>) {\n        for grant in iter {\n            self.add_permission(grant.permission, grant.grant);\n        }\n    }\n\n    pub fn remove_permission(&mut self, permission: Permission, grant: bool) {\n        let permission = permission.id();\n        self.data.retain(|data| {\n            if let PrincipalData::Permission {\n                permission_id: p,\n                grant: g,\n            } = data\n            {\n                *p != permission || *g != grant\n            } else {\n                true\n            }\n        });\n    }\n\n    pub fn remove_permissions(&mut self, grant: bool) {\n        self.data.retain(|data| {\n            if let PrincipalData::Permission { grant: g, .. } = data {\n                *g != grant\n            } else {\n                true\n            }\n        });\n    }\n\n    pub fn update_external(&mut self, external: Principal) -> Vec<PrincipalUpdate> {\n        let mut updates = Vec::new();\n        let mut external_data = AHashSet::with_capacity(external.data.len());\n        let mut has_role = false;\n        let mut has_member_of = false;\n        let mut has_quota = false;\n        let mut has_otp_auth = false;\n        let mut has_app_password = false;\n\n        for item in external.data {\n            match item {\n                PrincipalData::DiskQuota(_) => {\n                    has_quota = true;\n                    external_data.insert(item);\n                }\n                PrincipalData::MemberOf(_) => {\n                    has_member_of = true;\n                    external_data.insert(item);\n                }\n                PrincipalData::Role(_) => {\n                    has_role = true;\n                    external_data.insert(item);\n                }\n                PrincipalData::OtpAuth(_) => {\n                    has_otp_auth = true;\n                    external_data.insert(item);\n                }\n                PrincipalData::AppPassword(_) => {\n                    has_app_password = true;\n                    external_data.insert(item);\n                }\n                PrincipalData::Password(_)\n                | PrincipalData::Description(_)\n                | PrincipalData::PrimaryEmail(_)\n                | PrincipalData::EmailAlias(_) => {\n                    external_data.insert(item);\n                }\n                _ => {}\n            }\n        }\n\n        let mut old_data = Vec::new();\n        let data_len = self.data.len();\n\n        for item in std::mem::replace(&mut self.data, Vec::with_capacity(data_len)) {\n            match item {\n                PrincipalData::Password(_)\n                | PrincipalData::AppPassword(_)\n                | PrincipalData::OtpAuth(_)\n                | PrincipalData::Description(_)\n                | PrincipalData::PrimaryEmail(_)\n                | PrincipalData::EmailAlias(_)\n                | PrincipalData::DiskQuota(_)\n                | PrincipalData::MemberOf(_)\n                | PrincipalData::Role(_) => {\n                    if external_data.remove(&item)\n                        || match item {\n                            PrincipalData::EmailAlias(_) => true,\n                            PrincipalData::AppPassword(_) => !has_app_password,\n                            PrincipalData::OtpAuth(_) => !has_otp_auth,\n                            PrincipalData::Role(_) => !has_role,\n                            PrincipalData::MemberOf(_) => !has_member_of,\n                            PrincipalData::DiskQuota(_) => !has_quota,\n                            _ => false,\n                        }\n                    {\n                        self.data.push(item);\n                    } else if matches!(\n                        item,\n                        PrincipalData::Password(_)\n                            | PrincipalData::AppPassword(_)\n                            | PrincipalData::OtpAuth(_)\n                            | PrincipalData::PrimaryEmail(_)\n                            | PrincipalData::EmailAlias(_)\n                    ) {\n                        old_data.push(item);\n                    }\n                }\n                _ => {\n                    self.data.push(item);\n                }\n            }\n        }\n\n        // Add new data\n        let mut has_password = false;\n        let mut has_email = false;\n        for item in external_data {\n            match &item {\n                PrincipalData::Description(value) => {\n                    updates.push(PrincipalUpdate::set(\n                        PrincipalField::Description,\n                        PrincipalValue::String(value.to_string()),\n                    ));\n                }\n                PrincipalData::DiskQuota(value) => {\n                    updates.push(PrincipalUpdate::set(\n                        PrincipalField::Quota,\n                        PrincipalValue::Integer(*value),\n                    ));\n                }\n                PrincipalData::Password(value)\n                | PrincipalData::AppPassword(value)\n                | PrincipalData::OtpAuth(value) => {\n                    let item = PrincipalUpdate::add_item(\n                        PrincipalField::Secrets,\n                        PrincipalValue::String(value.to_string()),\n                    );\n                    if !has_password && !updates.is_empty() {\n                        updates.insert(0, item);\n                    } else {\n                        updates.push(item);\n                    }\n                    has_password = true;\n                }\n                PrincipalData::PrimaryEmail(value) => {\n                    let item = PrincipalUpdate::add_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(value.to_string()),\n                    );\n                    if !has_email && !updates.is_empty() {\n                        updates.insert(0, item);\n                    } else {\n                        updates.push(item);\n                    }\n                    has_email = true;\n                }\n                PrincipalData::EmailAlias(value) => {\n                    updates.push(PrincipalUpdate::add_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(value.to_string()),\n                    ));\n                }\n                _ => (),\n            }\n            self.data.push(item);\n        }\n\n        // Remove old data\n        for item in old_data {\n            match item {\n                PrincipalData::Password(value)\n                | PrincipalData::AppPassword(value)\n                | PrincipalData::OtpAuth(value) => {\n                    updates.push(PrincipalUpdate::remove_item(\n                        PrincipalField::Secrets,\n                        PrincipalValue::String(value),\n                    ));\n                }\n                PrincipalData::PrimaryEmail(value) | PrincipalData::EmailAlias(value) => {\n                    updates.push(PrincipalUpdate::remove_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(value),\n                    ));\n                }\n                _ => (),\n            }\n        }\n\n        self.sort();\n\n        updates\n    }\n\n    pub fn object_size(&self) -> usize {\n        self.name.len()\n            + self\n                .data\n                .iter()\n                .map(|item| item.object_size())\n                .sum::<usize>()\n    }\n\n    pub fn fallback_admin(fallback_pass: impl Into<String>) -> Self {\n        Principal {\n            id: FALLBACK_ADMIN_ID,\n            typ: Type::Individual,\n            name: \"Fallback Administrator\".into(),\n            data: vec![\n                PrincipalData::Role(ROLE_ADMIN),\n                PrincipalData::Password(fallback_pass.into()),\n            ],\n        }\n    }\n\n    pub fn sort(&mut self) {\n        self.data.sort_unstable();\n    }\n}\n\nimpl PrincipalData {\n    fn rank(&self) -> u8 {\n        match self {\n            PrincipalData::OtpAuth(_) => 0,\n            PrincipalData::Password(_) => 1,\n            PrincipalData::AppPassword(_) => 2,\n            PrincipalData::PrimaryEmail(_) => 3,\n            PrincipalData::EmailAlias(_) => 4,\n            _ => 5,\n        }\n    }\n\n    fn rank_string(&self) -> Option<&str> {\n        match self {\n            PrincipalData::OtpAuth(s)\n            | PrincipalData::Password(s)\n            | PrincipalData::AppPassword(s)\n            | PrincipalData::PrimaryEmail(s)\n            | PrincipalData::EmailAlias(s) => Some(s),\n            _ => None,\n        }\n    }\n}\n\nimpl PartialOrd for PrincipalData {\n    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {\n        Some(self.cmp(other))\n    }\n}\n\nimpl Ord for PrincipalData {\n    fn cmp(&self, other: &Self) -> Ordering {\n        match self.rank().cmp(&other.rank()) {\n            Ordering::Equal => match (self.rank_string(), other.rank_string()) {\n                (Some(a), Some(b)) => a.cmp(b),\n                _ => Ordering::Equal,\n            },\n            other => other,\n        }\n    }\n}\n\nimpl PrincipalData {\n    pub fn object_size(&self) -> usize {\n        match self {\n            PrincipalData::Password(v)\n            | PrincipalData::AppPassword(v)\n            | PrincipalData::OtpAuth(v)\n            | PrincipalData::Description(v)\n            | PrincipalData::PrimaryEmail(v)\n            | PrincipalData::EmailAlias(v)\n            | PrincipalData::Picture(v)\n            | PrincipalData::ExternalMember(v)\n            | PrincipalData::Url(v)\n            | PrincipalData::Locale(v) => v.len(),\n            PrincipalData::DiskQuota(_) => U64_LEN,\n            PrincipalData::Permission { .. } => U32_LEN + 1,\n            PrincipalData::DirectoryQuota { .. } | PrincipalData::ObjectQuota { .. } => U64_LEN + 1,\n            PrincipalData::Tenant(_)\n            | PrincipalData::MemberOf(_)\n            | PrincipalData::Role(_)\n            | PrincipalData::List(_) => U32_LEN,\n        }\n    }\n}\n\nimpl PrincipalSet {\n    pub fn new(id: u32, typ: Type) -> Self {\n        Self {\n            id,\n            typ,\n            ..Default::default()\n        }\n    }\n\n    pub fn id(&self) -> u32 {\n        self.id\n    }\n\n    pub fn typ(&self) -> Type {\n        self.typ\n    }\n\n    pub fn name(&self) -> &str {\n        self.get_str(PrincipalField::Name).unwrap_or_default()\n    }\n\n    pub fn has_name(&self) -> bool {\n        self.fields.contains_key(&PrincipalField::Name)\n    }\n\n    pub fn quota(&self) -> u64 {\n        self.get_int(PrincipalField::Quota).unwrap_or_default()\n    }\n\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    pub fn tenant(&self) -> Option<u32> {\n        self.get_int(PrincipalField::Tenant).map(|v| v as u32)\n    }\n    // SPDX-SnippetEnd\n\n    #[cfg(not(feature = \"enterprise\"))]\n    pub fn tenant(&self) -> Option<u32> {\n        None\n    }\n\n    pub fn description(&self) -> Option<&str> {\n        self.get_str(PrincipalField::Description)\n    }\n\n    pub fn get_str(&self, key: PrincipalField) -> Option<&str> {\n        self.fields.get(&key).and_then(|v| v.as_str())\n    }\n\n    pub fn get_int(&self, key: PrincipalField) -> Option<u64> {\n        self.fields.get(&key).and_then(|v| v.as_int())\n    }\n\n    pub fn get_str_array(&self, key: PrincipalField) -> Option<&[String]> {\n        self.fields.get(&key).and_then(|v| match v {\n            PrincipalValue::StringList(v) => Some(v.as_slice()),\n            PrincipalValue::String(v) => Some(std::slice::from_ref(v)),\n            PrincipalValue::Integer(_) | PrincipalValue::IntegerList(_) => None,\n        })\n    }\n\n    pub fn get_int_array(&self, key: PrincipalField) -> Option<&[u64]> {\n        self.fields.get(&key).and_then(|v| match v {\n            PrincipalValue::IntegerList(v) => Some(v.as_slice()),\n            PrincipalValue::Integer(v) => Some(std::slice::from_ref(v)),\n            PrincipalValue::String(_) | PrincipalValue::StringList(_) => None,\n        })\n    }\n\n    pub fn take(&mut self, key: PrincipalField) -> Option<PrincipalValue> {\n        self.fields.remove(&key)\n    }\n\n    pub fn take_str(&mut self, key: PrincipalField) -> Option<String> {\n        self.take(key).and_then(|v| match v {\n            PrincipalValue::String(s) => Some(s),\n            PrincipalValue::StringList(l) => l.into_iter().next(),\n            PrincipalValue::Integer(i) => Some(i.to_string()),\n            PrincipalValue::IntegerList(l) => l.into_iter().next().map(|i| i.to_string()),\n        })\n    }\n\n    pub fn take_int(&mut self, key: PrincipalField) -> Option<u64> {\n        self.take(key).and_then(|v| match v {\n            PrincipalValue::Integer(i) => Some(i),\n            PrincipalValue::IntegerList(l) => l.into_iter().next(),\n            PrincipalValue::String(s) => s.parse().ok(),\n            PrincipalValue::StringList(l) => l.into_iter().next().and_then(|s| s.parse().ok()),\n        })\n    }\n\n    pub fn take_str_array(&mut self, key: PrincipalField) -> Option<Vec<String>> {\n        self.take(key).map(|v| v.into_str_array())\n    }\n\n    pub fn take_int_array(&mut self, key: PrincipalField) -> Option<Vec<u64>> {\n        self.take(key).map(|v| v.into_int_array())\n    }\n\n    pub fn iter_str(\n        &self,\n        key: PrincipalField,\n    ) -> Box<dyn Iterator<Item = &String> + Sync + Send + '_> {\n        self.fields\n            .get(&key)\n            .map(|v| v.iter_str())\n            .unwrap_or_else(|| Box::new(std::iter::empty()))\n    }\n\n    pub fn iter_mut_str(\n        &mut self,\n        key: PrincipalField,\n    ) -> Box<dyn Iterator<Item = &mut String> + Sync + Send + '_> {\n        self.fields\n            .get_mut(&key)\n            .map(|v| v.iter_mut_str())\n            .unwrap_or_else(|| Box::new(std::iter::empty()))\n    }\n\n    pub fn iter_int(\n        &self,\n        key: PrincipalField,\n    ) -> Box<dyn Iterator<Item = u64> + Sync + Send + '_> {\n        self.fields\n            .get(&key)\n            .map(|v| v.iter_int())\n            .unwrap_or_else(|| Box::new(std::iter::empty()))\n    }\n\n    pub fn iter_mut_int(\n        &mut self,\n        key: PrincipalField,\n    ) -> Box<dyn Iterator<Item = &mut u64> + Sync + Send + '_> {\n        self.fields\n            .get_mut(&key)\n            .map(|v| v.iter_mut_int())\n            .unwrap_or_else(|| Box::new(std::iter::empty()))\n    }\n\n    pub fn append_int(&mut self, key: PrincipalField, value: impl Into<u64>) -> &mut Self {\n        let value = value.into();\n        match self.fields.entry(key) {\n            Entry::Occupied(v) => {\n                let v = v.into_mut();\n\n                match v {\n                    PrincipalValue::IntegerList(v) => {\n                        if !v.contains(&value) {\n                            v.push(value);\n                        }\n                    }\n                    PrincipalValue::Integer(i) => {\n                        if value != *i {\n                            *v = PrincipalValue::IntegerList(vec![*i, value]);\n                        }\n                    }\n                    PrincipalValue::String(s) => {\n                        *v =\n                            PrincipalValue::IntegerList(vec![s.parse().unwrap_or_default(), value]);\n                    }\n                    PrincipalValue::StringList(l) => {\n                        *v = PrincipalValue::IntegerList(\n                            l.iter()\n                                .map(|s| s.parse().unwrap_or_default())\n                                .chain(std::iter::once(value))\n                                .collect(),\n                        );\n                    }\n                }\n            }\n            Entry::Vacant(v) => {\n                v.insert(PrincipalValue::IntegerList(vec![value]));\n            }\n        }\n\n        self\n    }\n\n    pub fn append_str(&mut self, key: PrincipalField, value: impl Into<String>) -> &mut Self {\n        let value = value.into();\n        match self.fields.entry(key) {\n            Entry::Occupied(v) => {\n                let v = v.into_mut();\n\n                match v {\n                    PrincipalValue::StringList(v) => {\n                        if !v.contains(&value) {\n                            v.push(value);\n                        }\n                    }\n                    PrincipalValue::String(s) => {\n                        if s != &value {\n                            *v = PrincipalValue::StringList(vec![std::mem::take(s), value]);\n                        }\n                    }\n                    PrincipalValue::Integer(i) => {\n                        *v = PrincipalValue::StringList(vec![i.to_string(), value]);\n                    }\n                    PrincipalValue::IntegerList(l) => {\n                        *v = PrincipalValue::StringList(\n                            l.iter()\n                                .map(|i| i.to_string())\n                                .chain(std::iter::once(value))\n                                .collect(),\n                        );\n                    }\n                }\n            }\n            Entry::Vacant(v) => {\n                v.insert(PrincipalValue::StringList(vec![value]));\n            }\n        }\n        self\n    }\n\n    pub fn prepend_str(&mut self, key: PrincipalField, value: impl Into<String>) -> &mut Self {\n        let value = value.into();\n        match self.fields.entry(key) {\n            Entry::Occupied(v) => {\n                let v = v.into_mut();\n\n                match v {\n                    PrincipalValue::StringList(v) => {\n                        if !v.contains(&value) {\n                            v.insert(0, value);\n                        }\n                    }\n                    PrincipalValue::String(s) => {\n                        if s != &value {\n                            *v = PrincipalValue::StringList(vec![value, std::mem::take(s)]);\n                        }\n                    }\n                    PrincipalValue::Integer(i) => {\n                        *v = PrincipalValue::StringList(vec![value, i.to_string()]);\n                    }\n                    PrincipalValue::IntegerList(l) => {\n                        *v = PrincipalValue::StringList(\n                            std::iter::once(value)\n                                .chain(l.iter().map(|i| i.to_string()))\n                                .collect(),\n                        );\n                    }\n                }\n            }\n            Entry::Vacant(v) => {\n                v.insert(PrincipalValue::StringList(vec![value]));\n            }\n        }\n        self\n    }\n\n    pub fn set(&mut self, key: PrincipalField, value: impl Into<PrincipalValue>) -> &mut Self {\n        self.fields.insert(key, value.into());\n        self\n    }\n\n    pub fn with_field(mut self, key: PrincipalField, value: impl Into<PrincipalValue>) -> Self {\n        self.set(key, value);\n        self\n    }\n\n    pub fn with_opt_field(\n        mut self,\n        key: PrincipalField,\n        value: Option<impl Into<PrincipalValue>>,\n    ) -> Self {\n        if let Some(value) = value {\n            self.set(key, value);\n        }\n        self\n    }\n\n    pub fn has_field(&self, key: PrincipalField) -> bool {\n        self.fields.contains_key(&key)\n    }\n\n    pub fn has_str_value(&self, key: PrincipalField, value: &str) -> bool {\n        self.fields.get(&key).is_some_and(|v| match v {\n            PrincipalValue::String(v) => v == value,\n            PrincipalValue::StringList(l) => l.iter().any(|v| v == value),\n            PrincipalValue::Integer(_) | PrincipalValue::IntegerList(_) => false,\n        })\n    }\n\n    pub fn has_int_value(&self, key: PrincipalField, value: u64) -> bool {\n        self.fields.get(&key).is_some_and(|v| match v {\n            PrincipalValue::Integer(v) => *v == value,\n            PrincipalValue::IntegerList(l) => l.contains(&value),\n            PrincipalValue::String(_) | PrincipalValue::StringList(_) => false,\n        })\n    }\n\n    pub fn find_str(&self, value: &str) -> bool {\n        self.fields.values().any(|v| v.find_str(value))\n    }\n\n    pub fn field_len(&self, key: PrincipalField) -> usize {\n        self.fields.get(&key).map_or(0, |v| match v {\n            PrincipalValue::String(_) => 1,\n            PrincipalValue::StringList(l) => l.len(),\n            PrincipalValue::Integer(_) => 1,\n            PrincipalValue::IntegerList(l) => l.len(),\n        })\n    }\n\n    pub fn remove(&mut self, key: PrincipalField) -> Option<PrincipalValue> {\n        self.fields.remove(&key)\n    }\n\n    pub fn retain_str<F>(&mut self, key: PrincipalField, mut f: F)\n    where\n        F: FnMut(&String) -> bool,\n    {\n        if let Some(value) = self.fields.get_mut(&key) {\n            match value {\n                PrincipalValue::String(s) => {\n                    if !f(s) {\n                        self.fields.remove(&key);\n                    }\n                }\n                PrincipalValue::StringList(l) => {\n                    l.retain(f);\n                    if l.is_empty() {\n                        self.fields.remove(&key);\n                    }\n                }\n                _ => {}\n            }\n        }\n    }\n\n    pub fn retain_int<F>(&mut self, key: PrincipalField, mut f: F)\n    where\n        F: FnMut(&u64) -> bool,\n    {\n        if let Some(value) = self.fields.get_mut(&key) {\n            match value {\n                PrincipalValue::Integer(i) => {\n                    if !f(i) {\n                        self.fields.remove(&key);\n                    }\n                }\n                PrincipalValue::IntegerList(l) => {\n                    l.retain(f);\n                    if l.is_empty() {\n                        self.fields.remove(&key);\n                    }\n                }\n                _ => {}\n            }\n        }\n    }\n}\n\nimpl PrincipalValue {\n    pub fn as_str(&self) -> Option<&str> {\n        match self {\n            PrincipalValue::String(v) => Some(v.as_str()),\n            PrincipalValue::StringList(v) => v.first().map(|s| s.as_str()),\n            _ => None,\n        }\n    }\n\n    pub fn as_int(&self) -> Option<u64> {\n        match self {\n            PrincipalValue::Integer(v) => Some(*v),\n            PrincipalValue::IntegerList(v) => v.first().copied(),\n            _ => None,\n        }\n    }\n\n    pub fn iter_str(&self) -> Box<dyn Iterator<Item = &String> + Sync + Send + '_> {\n        match self {\n            PrincipalValue::String(v) => Box::new(std::iter::once(v)),\n            PrincipalValue::StringList(v) => Box::new(v.iter()),\n            _ => Box::new(std::iter::empty()),\n        }\n    }\n\n    pub fn iter_mut_str(&mut self) -> Box<dyn Iterator<Item = &mut String> + Sync + Send + '_> {\n        match self {\n            PrincipalValue::String(v) => Box::new(std::iter::once(v)),\n            PrincipalValue::StringList(v) => Box::new(v.iter_mut()),\n            _ => Box::new(std::iter::empty()),\n        }\n    }\n\n    pub fn iter_int(&self) -> Box<dyn Iterator<Item = u64> + Sync + Send + '_> {\n        match self {\n            PrincipalValue::Integer(v) => Box::new(std::iter::once(*v)),\n            PrincipalValue::IntegerList(v) => Box::new(v.iter().copied()),\n            _ => Box::new(std::iter::empty()),\n        }\n    }\n\n    pub fn iter_mut_int(&mut self) -> Box<dyn Iterator<Item = &mut u64> + Sync + Send + '_> {\n        match self {\n            PrincipalValue::Integer(v) => Box::new(std::iter::once(v)),\n            PrincipalValue::IntegerList(v) => Box::new(v.iter_mut()),\n            _ => Box::new(std::iter::empty()),\n        }\n    }\n\n    pub fn into_array(self) -> Self {\n        match self {\n            PrincipalValue::String(v) => PrincipalValue::StringList(vec![v]),\n            PrincipalValue::Integer(v) => PrincipalValue::IntegerList(vec![v]),\n            v => v,\n        }\n    }\n\n    pub fn into_str_array(self) -> Vec<String> {\n        match self {\n            PrincipalValue::StringList(v) => v,\n            PrincipalValue::String(v) => vec![v],\n            PrincipalValue::Integer(v) => vec![v.to_string()],\n            PrincipalValue::IntegerList(v) => v.into_iter().map(|v| v.to_string()).collect(),\n        }\n    }\n\n    pub fn into_int_array(self) -> Vec<u64> {\n        match self {\n            PrincipalValue::IntegerList(v) => v,\n            PrincipalValue::Integer(v) => vec![v],\n            PrincipalValue::String(v) => vec![v.parse().unwrap_or_default()],\n            PrincipalValue::StringList(v) => v\n                .into_iter()\n                .map(|v| v.parse().unwrap_or_default())\n                .collect(),\n        }\n    }\n\n    pub fn serialized_size(&self) -> usize {\n        match self {\n            PrincipalValue::String(s) => s.len() + 2,\n            PrincipalValue::StringList(s) => s.iter().map(|s| s.len() + 2).sum(),\n            PrincipalValue::Integer(_) => U64_LEN,\n            PrincipalValue::IntegerList(l) => l.len() * U64_LEN,\n        }\n    }\n\n    pub fn find_str(&self, value: &str) -> bool {\n        match self {\n            PrincipalValue::String(s) => s.to_lowercase().contains(value),\n            PrincipalValue::StringList(l) => l.iter().any(|s| s.to_lowercase().contains(value)),\n            _ => false,\n        }\n    }\n}\n\nimpl From<u64> for PrincipalValue {\n    fn from(v: u64) -> Self {\n        Self::Integer(v)\n    }\n}\n\nimpl From<String> for PrincipalValue {\n    fn from(v: String) -> Self {\n        Self::String(v)\n    }\n}\n\nimpl From<&str> for PrincipalValue {\n    fn from(v: &str) -> Self {\n        Self::String(v.into())\n    }\n}\n\nimpl From<Vec<String>> for PrincipalValue {\n    fn from(v: Vec<String>) -> Self {\n        Self::StringList(v)\n    }\n}\n\nimpl From<Vec<u64>> for PrincipalValue {\n    fn from(v: Vec<u64>) -> Self {\n        Self::IntegerList(v)\n    }\n}\n\nimpl From<u32> for PrincipalValue {\n    fn from(v: u32) -> Self {\n        Self::Integer(v as u64)\n    }\n}\n\nimpl From<Vec<u32>> for PrincipalValue {\n    fn from(v: Vec<u32>) -> Self {\n        Self::IntegerList(v.into_iter().map(|v| v as u64).collect())\n    }\n}\n\npub(crate) fn build_search_index(\n    batch: &mut BatchBuilder,\n    principal_id: u32,\n    current: Option<&ArchivedPrincipal>,\n    new: Option<&Principal>,\n) {\n    let mut current_words = AHashSet::new();\n    let mut new_words = AHashSet::new();\n\n    if let Some(current) = current {\n        for word in [Some(current.name.as_str())]\n            .into_iter()\n            .chain(current.data.iter().map(|s| match s {\n                ArchivedPrincipalData::Description(v)\n                | ArchivedPrincipalData::PrimaryEmail(v)\n                | ArchivedPrincipalData::EmailAlias(v) => Some(v.as_str()),\n                _ => None,\n            }))\n            .flatten()\n        {\n            current_words.extend(WordTokenizer::new(word, MAX_TOKEN_LENGTH).map(|t| t.word));\n        }\n    }\n\n    if let Some(new) = new {\n        for word in [Some(new.name.as_str())]\n            .into_iter()\n            .chain(new.data.iter().map(|s| match s {\n                PrincipalData::Description(v)\n                | PrincipalData::PrimaryEmail(v)\n                | PrincipalData::EmailAlias(v) => Some(v.as_str()),\n                _ => None,\n            }))\n            .flatten()\n        {\n            new_words.extend(WordTokenizer::new(word, MAX_TOKEN_LENGTH).map(|t| t.word));\n        }\n    }\n\n    for word in new_words.difference(&current_words) {\n        batch.set(\n            DirectoryClass::Index {\n                word: word.as_bytes().to_vec(),\n                principal_id,\n            },\n            vec![],\n        );\n    }\n\n    for word in current_words.difference(&new_words) {\n        batch.clear(DirectoryClass::Index {\n            word: word.as_bytes().to_vec(),\n            principal_id,\n        });\n    }\n}\n\nimpl Type {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Self::Individual => \"individual\",\n            Self::Group => \"group\",\n            Self::Resource => \"resource\",\n            Self::Location => \"location\",\n            Self::Other => \"other\",\n            Self::List => \"list\",\n            Self::Tenant => \"tenant\",\n            Self::Role => \"role\",\n            Self::Domain => \"domain\",\n            Self::ApiKey => \"apiKey\",\n            Self::OauthClient => \"oauthClient\",\n        }\n    }\n\n    pub fn description(&self) -> &'static str {\n        match self {\n            Self::Individual => \"Individual\",\n            Self::Group => \"Group\",\n            Self::Resource => \"Resource\",\n            Self::Location => \"Location\",\n            Self::Tenant => \"Tenant\",\n            Self::List => \"List\",\n            Self::Other => \"Other\",\n            Self::Role => \"Role\",\n            Self::Domain => \"Domain\",\n            Self::ApiKey => \"API Key\",\n            Self::OauthClient => \"OAuth Client\",\n        }\n    }\n\n    pub fn parse(value: &str) -> Option<Self> {\n        match value {\n            \"individual\" => Some(Type::Individual),\n            \"group\" => Some(Type::Group),\n            \"resource\" => Some(Type::Resource),\n            \"location\" => Some(Type::Location),\n            \"list\" => Some(Type::List),\n            \"tenant\" => Some(Type::Tenant),\n            \"superuser\" => Some(Type::Individual), // legacy\n            \"role\" => Some(Type::Role),\n            \"domain\" => Some(Type::Domain),\n            \"apiKey\" => Some(Type::ApiKey),\n            \"oauthClient\" => Some(Type::OauthClient),\n            _ => None,\n        }\n    }\n\n    pub const MAX_ID: usize = 11;\n\n    pub fn from_u8(value: u8) -> Self {\n        match value {\n            0 => Type::Individual,\n            1 => Type::Group,\n            2 => Type::Resource,\n            3 => Type::Location,\n            4 => Type::Other, // legacy\n            5 => Type::List,\n            6 => Type::Other,\n            7 => Type::Domain,\n            8 => Type::Tenant,\n            9 => Type::Role,\n            10 => Type::ApiKey,\n            11 => Type::OauthClient,\n            _ => Type::Other,\n        }\n    }\n}\n\nimpl FromStr for Type {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        Type::parse(s).ok_or(())\n    }\n}\n\nimpl serde::Serialize for PrincipalSet {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        let mut map = serializer.serialize_map(None)?;\n\n        map.serialize_entry(\"id\", &self.id)?;\n        map.serialize_entry(\"type\", &self.typ.as_str())?;\n\n        for (key, value) in &self.fields {\n            match value {\n                PrincipalValue::String(v) => map.serialize_entry(key.as_str(), v)?,\n                PrincipalValue::StringList(v) => map.serialize_entry(key.as_str(), v)?,\n                PrincipalValue::Integer(v) => map.serialize_entry(key.as_str(), v)?,\n                PrincipalValue::IntegerList(v) => map.serialize_entry(key.as_str(), v)?,\n            };\n        }\n\n        map.end()\n    }\n}\n\nconst MAX_STRING_LEN: usize = 512;\n\nimpl<'de> serde::Deserialize<'de> for PrincipalValue {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        struct PrincipalValueVisitor;\n\n        impl<'de> Visitor<'de> for PrincipalValueVisitor {\n            type Value = PrincipalValue;\n\n            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n                formatter.write_str(\"an optional values or a sequence of values\")\n            }\n\n            fn visit_none<E>(self) -> Result<Self::Value, E>\n            where\n                E: de::Error,\n            {\n                Ok(PrincipalValue::String(\"\".into()))\n            }\n\n            fn visit_some<D>(self, deserializer: D) -> Result<Self::Value, D::Error>\n            where\n                D: Deserializer<'de>,\n            {\n                deserializer.deserialize_any(self)\n            }\n\n            fn visit_u64<E>(self, value: u64) -> Result<Self::Value, E>\n            where\n                E: de::Error,\n            {\n                Ok(PrincipalValue::Integer(value))\n            }\n\n            fn visit_string<E>(self, value: String) -> Result<Self::Value, E>\n            where\n                E: de::Error,\n            {\n                if value.len() <= MAX_STRING_LEN {\n                    Ok(PrincipalValue::String(value))\n                } else {\n                    Err(serde::de::Error::custom(\"string too long\"))\n                }\n            }\n\n            fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>\n            where\n                E: de::Error,\n            {\n                if value.len() <= MAX_STRING_LEN {\n                    Ok(PrincipalValue::String(value.into()))\n                } else {\n                    Err(serde::de::Error::custom(\"string too long\"))\n                }\n            }\n\n            fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>\n            where\n                A: de::SeqAccess<'de>,\n            {\n                let mut vec_u64 = Vec::new();\n                let mut vec_string = Vec::new();\n\n                while let Some(value) = seq.next_element::<StringOrU64>()? {\n                    match value {\n                        StringOrU64::String(s) => {\n                            if s.len() <= MAX_STRING_LEN {\n                                vec_string.push(s);\n                            } else {\n                                return Err(serde::de::Error::custom(\"string too long\"));\n                            }\n                        }\n                        StringOrU64::U64(u) => vec_u64.push(u),\n                    }\n                }\n\n                match (vec_u64.is_empty(), vec_string.is_empty()) {\n                    (true, false) => Ok(PrincipalValue::StringList(vec_string)),\n                    (false, true) => Ok(PrincipalValue::IntegerList(vec_u64)),\n                    (true, true) => Ok(PrincipalValue::StringList(vec_string)),\n                    _ => Err(serde::de::Error::custom(\"invalid principal value\")),\n                }\n            }\n        }\n\n        deserializer.deserialize_any(PrincipalValueVisitor)\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for PrincipalSet {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        struct PrincipalVisitor;\n\n        // Deserialize the principal\n        impl<'de> Visitor<'de> for PrincipalVisitor {\n            type Value = PrincipalSet;\n\n            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n                formatter.write_str(\"a valid principal\")\n            }\n\n            fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>\n            where\n                A: de::MapAccess<'de>,\n            {\n                let mut principal = PrincipalSet::default();\n\n                while let Some(key) = map.next_key::<&str>()? {\n                    if key == \"id\" {\n                        // Ignored\n                        map.next_value::<IgnoredAny>()?;\n                        continue;\n                    }\n\n                    let key = PrincipalField::try_parse(key).ok_or_else(|| {\n                        serde::de::Error::custom(format!(\"invalid principal field: {}\", key))\n                    })?;\n\n                    let value = match key {\n                        PrincipalField::Name => {\n                            PrincipalValue::String(map.next_value::<String>().and_then(|v| {\n                                if v.len() <= MAX_STRING_LEN {\n                                    Ok(v)\n                                } else {\n                                    Err(serde::de::Error::custom(\"string too long\"))\n                                }\n                            })?)\n                        }\n                        PrincipalField::Description\n                        | PrincipalField::Tenant\n                        | PrincipalField::Picture\n                        | PrincipalField::Locale => {\n                            if let Some(v) = map.next_value::<Option<String>>()? {\n                                if v.len() <= MAX_STRING_LEN {\n                                    PrincipalValue::String(v)\n                                } else {\n                                    return Err(serde::de::Error::custom(\"string too long\"));\n                                }\n                            } else {\n                                continue;\n                            }\n                        }\n                        PrincipalField::Type => {\n                            principal.typ = Type::parse(map.next_value()?).ok_or_else(|| {\n                                serde::de::Error::custom(\"invalid principal type\")\n                            })?;\n                            continue;\n                        }\n                        PrincipalField::Quota => map.next_value::<PrincipalValue>()?,\n                        PrincipalField::Secrets\n                        | PrincipalField::Emails\n                        | PrincipalField::MemberOf\n                        | PrincipalField::Members\n                        | PrincipalField::Roles\n                        | PrincipalField::Lists\n                        | PrincipalField::EnabledPermissions\n                        | PrincipalField::DisabledPermissions\n                        | PrincipalField::Urls\n                        | PrincipalField::ExternalMembers => match map.next_value::<Value>()? {\n                            Value::String(v) => {\n                                if v.len() <= MAX_STRING_LEN {\n                                    PrincipalValue::StringList(vec![v])\n                                } else {\n                                    return Err(serde::de::Error::custom(\"string too long\"));\n                                }\n                            }\n                            Value::Array(v) => {\n                                if !v.is_empty() {\n                                    PrincipalValue::StringList(\n                                        v.into_iter()\n                                            .filter_map(|item| {\n                                                if let Value::String(s) = item {\n                                                    if s.len() <= MAX_STRING_LEN {\n                                                        Some(s)\n                                                    } else {\n                                                        None\n                                                    }\n                                                } else {\n                                                    None\n                                                }\n                                            })\n                                            .collect(),\n                                    )\n                                } else {\n                                    continue;\n                                }\n                            }\n                            _ => continue,\n                        },\n                        PrincipalField::UsedQuota => {\n                            // consume and ignore\n                            map.next_value::<IgnoredAny>()?;\n                            continue;\n                        }\n                    };\n\n                    principal.fields.insert(key, value);\n                }\n\n                Ok(principal)\n            }\n        }\n\n        deserializer.deserialize_map(PrincipalVisitor)\n    }\n}\n\n#[derive(Debug)]\nenum StringOrU64 {\n    String(String),\n    U64(u64),\n}\n\nimpl<'de> serde::Deserialize<'de> for StringOrU64 {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        struct StringOrU64Visitor;\n\n        impl Visitor<'_> for StringOrU64Visitor {\n            type Value = StringOrU64;\n\n            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n                formatter.write_str(\"a string or u64\")\n            }\n\n            fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>\n            where\n                E: de::Error,\n            {\n                if value.len() <= MAX_STRING_LEN {\n                    Ok(StringOrU64::String(value.to_string()))\n                } else {\n                    Err(serde::de::Error::custom(\"string too long\"))\n                }\n            }\n\n            fn visit_string<E>(self, v: String) -> Result<Self::Value, E>\n            where\n                E: de::Error,\n            {\n                if v.len() <= MAX_STRING_LEN {\n                    Ok(StringOrU64::String(v))\n                } else {\n                    Err(serde::de::Error::custom(\"string too long\"))\n                }\n            }\n\n            fn visit_u64<E>(self, value: u64) -> Result<Self::Value, E>\n            where\n                E: de::Error,\n            {\n                Ok(StringOrU64::U64(value))\n            }\n        }\n\n        deserializer.deserialize_any(StringOrU64Visitor)\n    }\n}\n\nimpl Permission {\n    pub fn all() -> impl Iterator<Item = Permission> {\n        (0..Permission::COUNT as u32).filter_map(Permission::from_id)\n    }\n\n    pub const fn is_user_permission(&self) -> bool {\n        matches!(\n            self,\n            Permission::Authenticate\n                | Permission::AuthenticateOauth\n                | Permission::EmailSend\n                | Permission::EmailReceive\n                | Permission::ManageEncryption\n                | Permission::ManagePasswords\n                | Permission::JmapEmailGet\n                | Permission::JmapMailboxGet\n                | Permission::JmapThreadGet\n                | Permission::JmapIdentityGet\n                | Permission::JmapEmailSubmissionGet\n                | Permission::JmapPushSubscriptionGet\n                | Permission::JmapSieveScriptGet\n                | Permission::JmapVacationResponseGet\n                | Permission::JmapQuotaGet\n                | Permission::JmapBlobGet\n                | Permission::JmapEmailSet\n                | Permission::JmapMailboxSet\n                | Permission::JmapIdentitySet\n                | Permission::JmapEmailSubmissionSet\n                | Permission::JmapPushSubscriptionSet\n                | Permission::JmapSieveScriptSet\n                | Permission::JmapVacationResponseSet\n                | Permission::JmapEmailChanges\n                | Permission::JmapMailboxChanges\n                | Permission::JmapThreadChanges\n                | Permission::JmapIdentityChanges\n                | Permission::JmapEmailSubmissionChanges\n                | Permission::JmapQuotaChanges\n                | Permission::JmapEmailCopy\n                | Permission::JmapBlobCopy\n                | Permission::JmapEmailImport\n                | Permission::JmapEmailParse\n                | Permission::JmapEmailQueryChanges\n                | Permission::JmapMailboxQueryChanges\n                | Permission::JmapEmailSubmissionQueryChanges\n                | Permission::JmapSieveScriptQueryChanges\n                | Permission::JmapQuotaQueryChanges\n                | Permission::JmapEmailQuery\n                | Permission::JmapMailboxQuery\n                | Permission::JmapEmailSubmissionQuery\n                | Permission::JmapSieveScriptQuery\n                | Permission::JmapQuotaQuery\n                | Permission::JmapSearchSnippet\n                | Permission::JmapSieveScriptValidate\n                | Permission::JmapBlobLookup\n                | Permission::JmapBlobUpload\n                | Permission::JmapEcho\n                | Permission::ImapAuthenticate\n                | Permission::ImapAclGet\n                | Permission::ImapAclSet\n                | Permission::ImapMyRights\n                | Permission::ImapListRights\n                | Permission::ImapAppend\n                | Permission::ImapCapability\n                | Permission::ImapId\n                | Permission::ImapCopy\n                | Permission::ImapMove\n                | Permission::ImapCreate\n                | Permission::ImapDelete\n                | Permission::ImapEnable\n                | Permission::ImapExpunge\n                | Permission::ImapFetch\n                | Permission::ImapIdle\n                | Permission::ImapList\n                | Permission::ImapLsub\n                | Permission::ImapNamespace\n                | Permission::ImapRename\n                | Permission::ImapSearch\n                | Permission::ImapSort\n                | Permission::ImapSelect\n                | Permission::ImapExamine\n                | Permission::ImapStatus\n                | Permission::ImapStore\n                | Permission::ImapSubscribe\n                | Permission::ImapThread\n                | Permission::Pop3Authenticate\n                | Permission::Pop3List\n                | Permission::Pop3Uidl\n                | Permission::Pop3Stat\n                | Permission::Pop3Retr\n                | Permission::Pop3Dele\n                | Permission::SieveAuthenticate\n                | Permission::SieveListScripts\n                | Permission::SieveSetActive\n                | Permission::SieveGetScript\n                | Permission::SievePutScript\n                | Permission::SieveDeleteScript\n                | Permission::SieveRenameScript\n                | Permission::SieveCheckScript\n                | Permission::SieveHaveSpace\n                | Permission::DavSyncCollection\n                | Permission::DavExpandProperty\n                | Permission::DavPrincipalAcl\n                | Permission::DavPrincipalList\n                | Permission::DavPrincipalSearch\n                | Permission::DavPrincipalMatch\n                | Permission::DavPrincipalSearchPropSet\n                | Permission::DavFilePropFind\n                | Permission::DavFilePropPatch\n                | Permission::DavFileGet\n                | Permission::DavFileMkCol\n                | Permission::DavFileDelete\n                | Permission::DavFilePut\n                | Permission::DavFileCopy\n                | Permission::DavFileMove\n                | Permission::DavFileLock\n                | Permission::DavFileAcl\n                | Permission::DavCardPropFind\n                | Permission::DavCardPropPatch\n                | Permission::DavCardGet\n                | Permission::DavCardMkCol\n                | Permission::DavCardDelete\n                | Permission::DavCardPut\n                | Permission::DavCardCopy\n                | Permission::DavCardMove\n                | Permission::DavCardLock\n                | Permission::DavCardAcl\n                | Permission::DavCardQuery\n                | Permission::DavCardMultiGet\n                | Permission::DavCalPropFind\n                | Permission::DavCalPropPatch\n                | Permission::DavCalGet\n                | Permission::DavCalMkCol\n                | Permission::DavCalDelete\n                | Permission::DavCalPut\n                | Permission::DavCalCopy\n                | Permission::DavCalMove\n                | Permission::DavCalLock\n                | Permission::DavCalAcl\n                | Permission::DavCalQuery\n                | Permission::DavCalMultiGet\n                | Permission::DavCalFreeBusyQuery\n                | Permission::CalendarAlarms\n                | Permission::CalendarSchedulingSend\n                | Permission::CalendarSchedulingReceive\n                | Permission::JmapAddressBookGet\n                | Permission::JmapAddressBookSet\n                | Permission::JmapAddressBookChanges\n                | Permission::JmapContactCardGet\n                | Permission::JmapContactCardChanges\n                | Permission::JmapContactCardQuery\n                | Permission::JmapContactCardQueryChanges\n                | Permission::JmapContactCardSet\n                | Permission::JmapContactCardCopy\n                | Permission::JmapContactCardParse\n                | Permission::JmapFileNodeGet\n                | Permission::JmapFileNodeSet\n                | Permission::JmapFileNodeChanges\n                | Permission::JmapFileNodeQuery\n                | Permission::JmapFileNodeQueryChanges\n                | Permission::JmapPrincipalGetAvailability\n                | Permission::JmapPrincipalChanges\n                | Permission::JmapPrincipalQuery\n                | Permission::JmapPrincipalGet\n                | Permission::JmapPrincipalQueryChanges\n                | Permission::JmapShareNotificationGet\n                | Permission::JmapShareNotificationSet\n                | Permission::JmapShareNotificationChanges\n                | Permission::JmapShareNotificationQuery\n                | Permission::JmapShareNotificationQueryChanges\n                | Permission::JmapCalendarGet\n                | Permission::JmapCalendarSet\n                | Permission::JmapCalendarChanges\n                | Permission::JmapCalendarEventGet\n                | Permission::JmapCalendarEventSet\n                | Permission::JmapCalendarEventChanges\n                | Permission::JmapCalendarEventQuery\n                | Permission::JmapCalendarEventQueryChanges\n                | Permission::JmapCalendarEventCopy\n                | Permission::JmapCalendarEventParse\n                | Permission::JmapCalendarEventNotificationGet\n                | Permission::JmapCalendarEventNotificationSet\n                | Permission::JmapCalendarEventNotificationChanges\n                | Permission::JmapCalendarEventNotificationQuery\n                | Permission::JmapCalendarEventNotificationQueryChanges\n                | Permission::JmapParticipantIdentityGet\n                | Permission::JmapParticipantIdentitySet\n                | Permission::JmapParticipantIdentityChanges\n        )\n    }\n\n    #[cfg(not(feature = \"enterprise\"))]\n    pub const fn is_tenant_admin_permission(&self) -> bool {\n        false\n    }\n\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n\n    #[cfg(feature = \"enterprise\")]\n    pub const fn is_tenant_admin_permission(&self) -> bool {\n        matches!(\n            self,\n            Permission::MessageQueueList\n                | Permission::MessageQueueGet\n                | Permission::MessageQueueUpdate\n                | Permission::MessageQueueDelete\n                | Permission::OutgoingReportList\n                | Permission::OutgoingReportGet\n                | Permission::OutgoingReportDelete\n                | Permission::IncomingReportList\n                | Permission::IncomingReportGet\n                | Permission::IncomingReportDelete\n                | Permission::IndividualList\n                | Permission::IndividualGet\n                | Permission::IndividualUpdate\n                | Permission::IndividualDelete\n                | Permission::IndividualCreate\n                | Permission::GroupList\n                | Permission::GroupGet\n                | Permission::GroupUpdate\n                | Permission::GroupDelete\n                | Permission::GroupCreate\n                | Permission::DomainList\n                | Permission::DomainGet\n                | Permission::DomainCreate\n                | Permission::DomainUpdate\n                | Permission::DomainDelete\n                | Permission::MailingListList\n                | Permission::MailingListGet\n                | Permission::MailingListCreate\n                | Permission::MailingListUpdate\n                | Permission::MailingListDelete\n                | Permission::RoleList\n                | Permission::RoleGet\n                | Permission::RoleCreate\n                | Permission::RoleUpdate\n                | Permission::RoleDelete\n                | Permission::PrincipalList\n                | Permission::PrincipalGet\n                | Permission::PrincipalCreate\n                | Permission::PrincipalUpdate\n                | Permission::PrincipalDelete\n                | Permission::Undelete\n                | Permission::DkimSignatureCreate\n                | Permission::DkimSignatureGet\n                | Permission::ApiKeyList\n                | Permission::ApiKeyGet\n                | Permission::ApiKeyCreate\n                | Permission::ApiKeyUpdate\n                | Permission::ApiKeyDelete\n                | Permission::SpamFilterTrain\n                | Permission::SpamFilterTest\n        ) || self.is_user_permission()\n    }\n\n    // SPDX-SnippetEnd\n}\n"
  },
  {
    "path": "crates/directory/src/core/secret.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::Principal;\nuse crate::PrincipalData;\nuse argon2::Argon2;\nuse compact_str::ToCompactString;\nuse mail_builder::encoders::base64::base64_encode;\nuse mail_parser::decoders::base64::base64_decode;\nuse password_hash::PasswordHash;\nuse pbkdf2::Pbkdf2;\nuse pwhash::{bcrypt, bsdi_crypt, md5_crypt, sha1_crypt, sha256_crypt, sha512_crypt, unix_crypt};\nuse scrypt::Scrypt;\nuse sha1::Digest;\nuse sha1::Sha1;\nuse sha2::Sha256;\nuse sha2::Sha512;\nuse tokio::sync::oneshot;\nuse totp_rs::TOTP;\n\nimpl Principal {\n    pub async fn verify_secret(\n        &self,\n        code: &str,\n        only_app_pass: bool,\n        is_ordered: bool,\n    ) -> trc::Result<bool> {\n        let mut seen_password = false;\n        let mut password = None;\n        let mut otp_auth = None;\n\n        for item in &self.data {\n            match item {\n                PrincipalData::OtpAuth(secret) => {\n                    if !only_app_pass {\n                        otp_auth = Some(secret);\n                    }\n                    seen_password = true;\n                }\n                PrincipalData::Password(secret) => {\n                    if !only_app_pass {\n                        password = Some(secret);\n                    }\n                    seen_password = true;\n                }\n                PrincipalData::AppPassword(secret) => {\n                    // App passwords do not require TOTP\n                    if let Some((_, app_secret)) =\n                        secret.strip_prefix(\"$app$\").and_then(|s| s.split_once('$'))\n                        && verify_secret_hash(app_secret, code).await?\n                    {\n                        return Ok(true);\n                    }\n\n                    seen_password = true;\n                }\n                _ => {\n                    if seen_password && is_ordered {\n                        // Password-related secrets are expected to be at the beginning of the list\n                        break;\n                    }\n                }\n            }\n        }\n\n        // Validate TOTP\n        match (otp_auth, password) {\n            (Some(otp_auth), Some(password)) => {\n                if let Some((code, totp_token)) = code.rsplit_once('$').filter(|(c, t)| {\n                    !c.is_empty()\n                        && (6..=8).contains(&t.len())\n                        && t.as_bytes().iter().all(|b| b.is_ascii_digit())\n                }) {\n                    let result = verify_secret_hash(password, code).await?\n                        && TOTP::from_url(otp_auth)\n                            .map_err(|err| {\n                                trc::AuthEvent::Error\n                                    .reason(err)\n                                    .details(otp_auth.to_compact_string())\n                            })?\n                            .check_current(totp_token)\n                            .unwrap_or(false);\n                    Ok(result)\n                } else if verify_secret_hash(password, code).await? {\n                    // Only let the client know if the TOTP code is missing\n                    // if the password is correct\n\n                    Err(trc::AuthEvent::MissingTotp.into_err())\n                } else {\n                    Ok(false)\n                }\n            }\n            (None, Some(password)) => verify_secret_hash(password, code).await,\n            _ => Ok(false),\n        }\n    }\n}\n\nasync fn verify_hash_prefix(hashed_secret: &str, secret: &str) -> trc::Result<bool> {\n    if hashed_secret.starts_with(\"$argon2\")\n        || hashed_secret.starts_with(\"$pbkdf2\")\n        || hashed_secret.starts_with(\"$scrypt\")\n    {\n        let (tx, rx) = oneshot::channel();\n        let secret = secret.to_string();\n        let hashed_secret = hashed_secret.to_string();\n\n        tokio::task::spawn_blocking(move || match PasswordHash::new(&hashed_secret) {\n            Ok(hash) => {\n                tx.send(Ok(hash\n                    .verify_password(&[&Argon2::default(), &Pbkdf2, &Scrypt], &secret)\n                    .is_ok()))\n                    .ok();\n            }\n            Err(err) => {\n                tx.send(Err(trc::AuthEvent::Error\n                    .reason(err)\n                    .details(hashed_secret)))\n                    .ok();\n            }\n        });\n\n        match rx.await {\n            Ok(result) => result,\n            Err(err) => Err(trc::EventType::Server(trc::ServerEvent::ThreadError)\n                .caused_by(trc::location!())\n                .reason(err)),\n        }\n    } else if hashed_secret.starts_with(\"$2\") {\n        // Blowfish crypt\n        Ok(bcrypt::verify(secret, hashed_secret))\n    } else if hashed_secret.starts_with(\"$6$\") {\n        // SHA-512 crypt\n        Ok(sha512_crypt::verify(secret, hashed_secret))\n    } else if hashed_secret.starts_with(\"$5$\") {\n        // SHA-256 crypt\n        Ok(sha256_crypt::verify(secret, hashed_secret))\n    } else if hashed_secret.starts_with(\"$sha1\") {\n        // SHA-1 crypt\n        Ok(sha1_crypt::verify(secret, hashed_secret))\n    } else if hashed_secret.starts_with(\"$1\") {\n        // MD5 based hash\n        Ok(md5_crypt::verify(secret, hashed_secret))\n    } else {\n        Err(trc::AuthEvent::Error\n            .into_err()\n            .details(hashed_secret.to_string()))\n    }\n}\n\npub async fn verify_secret_hash(hashed_secret: &str, secret: &str) -> trc::Result<bool> {\n    if hashed_secret.starts_with('$') {\n        verify_hash_prefix(hashed_secret, secret).await\n    } else if hashed_secret.starts_with('_') {\n        // Enhanced DES-based hash\n        Ok(bsdi_crypt::verify(secret, hashed_secret))\n    } else if let Some(hashed_secret) = hashed_secret.strip_prefix('{') {\n        if let Some((algo, hashed_secret)) = hashed_secret.split_once('}') {\n            match algo {\n                \"ARGON2\" | \"ARGON2I\" | \"ARGON2ID\" | \"PBKDF2\" => {\n                    verify_hash_prefix(hashed_secret, secret).await\n                }\n                \"SHA\" => {\n                    // SHA-1\n                    let mut hasher = Sha1::new();\n                    hasher.update(secret.as_bytes());\n                    Ok(\n                        String::from_utf8(\n                            base64_encode(&hasher.finalize()[..]).unwrap_or_default(),\n                        )\n                        .unwrap()\n                            == hashed_secret,\n                    )\n                }\n                \"SSHA\" => {\n                    // Salted SHA-1\n                    let decoded = base64_decode(hashed_secret.as_bytes()).unwrap_or_default();\n                    let hash = decoded.get(..20).unwrap_or_default();\n                    let salt = decoded.get(20..).unwrap_or_default();\n                    let mut hasher = Sha1::new();\n                    hasher.update(secret.as_bytes());\n                    hasher.update(salt);\n                    Ok(&hasher.finalize()[..] == hash)\n                }\n                \"SHA256\" => {\n                    // Verify hash\n                    let mut hasher = Sha256::new();\n                    hasher.update(secret.as_bytes());\n                    Ok(\n                        String::from_utf8(\n                            base64_encode(&hasher.finalize()[..]).unwrap_or_default(),\n                        )\n                        .unwrap()\n                            == hashed_secret,\n                    )\n                }\n                \"SSHA256\" => {\n                    // Salted SHA-256\n                    let decoded = base64_decode(hashed_secret.as_bytes()).unwrap_or_default();\n                    let hash = decoded.get(..32).unwrap_or_default();\n                    let salt = decoded.get(32..).unwrap_or_default();\n                    let mut hasher = Sha256::new();\n                    hasher.update(secret.as_bytes());\n                    hasher.update(salt);\n                    Ok(&hasher.finalize()[..] == hash)\n                }\n                \"SHA512\" => {\n                    // SHA-512\n                    let mut hasher = Sha512::new();\n                    hasher.update(secret.as_bytes());\n                    Ok(\n                        String::from_utf8(\n                            base64_encode(&hasher.finalize()[..]).unwrap_or_default(),\n                        )\n                        .unwrap()\n                            == hashed_secret,\n                    )\n                }\n                \"SSHA512\" => {\n                    // Salted SHA-512\n                    let decoded = base64_decode(hashed_secret.as_bytes()).unwrap_or_default();\n                    let hash = decoded.get(..64).unwrap_or_default();\n                    let salt = decoded.get(64..).unwrap_or_default();\n                    let mut hasher = Sha512::new();\n                    hasher.update(secret.as_bytes());\n                    hasher.update(salt);\n                    Ok(&hasher.finalize()[..] == hash)\n                }\n                \"MD5\" => {\n                    // MD5\n                    let digest = md5::compute(secret.as_bytes());\n                    Ok(\n                        String::from_utf8(base64_encode(&digest[..]).unwrap_or_default()).unwrap()\n                            == hashed_secret,\n                    )\n                }\n                \"CRYPT\" | \"crypt\" => {\n                    if hashed_secret.starts_with('$') {\n                        verify_hash_prefix(hashed_secret, secret).await\n                    } else {\n                        // Unix crypt\n                        Ok(unix_crypt::verify(secret, hashed_secret))\n                    }\n                }\n                \"PLAIN\" | \"plain\" | \"CLEAR\" | \"clear\" => Ok(hashed_secret == secret),\n                _ => Err(trc::AuthEvent::Error\n                    .ctx(trc::Key::Reason, \"Unsupported algorithm\")\n                    .details(hashed_secret.to_string())),\n            }\n        } else {\n            Err(trc::AuthEvent::Error\n                .into_err()\n                .details(hashed_secret.to_string()))\n        }\n    } else if !hashed_secret.is_empty() {\n        Ok(hashed_secret == secret)\n    } else {\n        Ok(false)\n    }\n}\n"
  },
  {
    "path": "crates/directory/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#![warn(clippy::large_futures)]\n\nuse ahash::AHashMap;\nuse backend::{\n    imap::{ImapDirectory, ImapError},\n    ldap::LdapDirectory,\n    memory::MemoryDirectory,\n    smtp::SmtpDirectory,\n    sql::SqlDirectory,\n};\nuse core::cache::CachedDirectory;\nuse deadpool::managed::PoolError;\nuse ldap3::LdapError;\nuse mail_send::Credentials;\nuse proc_macros::EnumMethods;\nuse std::{fmt::Debug, sync::Arc};\nuse store::Store;\nuse trc::ipc::bitset::Bitset;\nuse types::collection::Collection;\n\npub mod backend;\npub mod core;\n\npub struct Directory {\n    pub store: DirectoryInner,\n    pub cache: Option<CachedDirectory>,\n}\n\npub const FALLBACK_ADMIN_ID: u32 = u32::MAX;\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct Principal {\n    pub id: u32,\n    pub typ: Type,\n    pub name: String,\n    pub data: Vec<PrincipalData>,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq, Hash)]\npub enum PrincipalData {\n    Password(String),\n\n    // Permissions and memberships\n    Tenant(u32),\n    MemberOf(u32),\n    Role(u32),\n    List(u32),\n    Permission { permission_id: u32, grant: bool },\n\n    // Quotas\n    DiskQuota(u64),\n    DirectoryQuota { quota: u32, typ: Type },\n    ObjectQuota { quota: u32, typ: Collection },\n\n    // Profile data\n    Description(String),\n    PrimaryEmail(String),\n    EmailAlias(String),\n    Picture(String),\n    ExternalMember(String),\n    Url(String),\n    Locale(String),\n\n    // Secrets\n    AppPassword(String),\n    OtpAuth(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct PermissionGrant {\n    pub permission: Permission,\n    pub grant: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct MemberOf {\n    pub principal_id: u32,\n    pub typ: Type,\n}\n\n#[derive(\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    Debug,\n    Default,\n    Clone,\n    Copy,\n    PartialEq,\n    Eq,\n    serde::Serialize,\n    serde::Deserialize,\n    Hash,\n)]\n#[serde(rename_all = \"camelCase\")]\npub enum Type {\n    #[default]\n    Individual = 0,\n    Group = 1,\n    Resource = 2,\n    Location = 3,\n    List = 5,\n    Other = 6,\n    Domain = 7,\n    Tenant = 8,\n    Role = 9,\n    ApiKey = 10,\n    OauthClient = 11,\n}\n\n#[derive(\n    Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize, EnumMethods,\n)]\n#[serde(rename_all = \"kebab-case\")]\npub enum Permission {\n    // WARNING: add new ids at the end (TODO: use static ids)\n\n    // Admin\n    Impersonate,\n    UnlimitedRequests,\n    UnlimitedUploads,\n    DeleteSystemFolders_,\n    MessageQueueList,\n    MessageQueueGet,\n    MessageQueueUpdate,\n    MessageQueueDelete,\n    OutgoingReportList,\n    OutgoingReportGet,\n    OutgoingReportDelete,\n    IncomingReportList,\n    IncomingReportGet,\n    IncomingReportDelete,\n    SettingsList,\n    SettingsUpdate,\n    SettingsDelete,\n    SettingsReload,\n    IndividualList,\n    IndividualGet,\n    IndividualUpdate,\n    IndividualDelete,\n    IndividualCreate,\n    GroupList,\n    GroupGet,\n    GroupUpdate,\n    GroupDelete,\n    GroupCreate,\n    DomainList,\n    DomainGet,\n    DomainCreate,\n    DomainUpdate,\n    DomainDelete,\n    TenantList,\n    TenantGet,\n    TenantCreate,\n    TenantUpdate,\n    TenantDelete,\n    MailingListList,\n    MailingListGet,\n    MailingListCreate,\n    MailingListUpdate,\n    MailingListDelete,\n    RoleList,\n    RoleGet,\n    RoleCreate,\n    RoleUpdate,\n    RoleDelete,\n    PrincipalList,\n    PrincipalGet,\n    PrincipalCreate,\n    PrincipalUpdate,\n    PrincipalDelete,\n    BlobFetch,\n    PurgeBlobStore,\n    PurgeDataStore,\n    PurgeInMemoryStore,\n    PurgeAccount,\n    FtsReindex,\n    Undelete,\n    DkimSignatureCreate,\n    DkimSignatureGet,\n    SpamFilterUpdate,\n    WebadminUpdate,\n    LogsView,\n    SpamFilterTrain,\n    Restart,\n    TracingList,\n    TracingGet,\n    TracingLive,\n    MetricsList,\n    MetricsLive,\n\n    // Generic\n    Authenticate,\n    AuthenticateOauth,\n    EmailSend,\n    EmailReceive,\n\n    // Account Management\n    ManageEncryption,\n    ManagePasswords,\n\n    // JMAP\n    JmapEmailGet,\n    JmapMailboxGet,\n    JmapThreadGet,\n    JmapIdentityGet,\n    JmapEmailSubmissionGet,\n    JmapPushSubscriptionGet,\n    JmapSieveScriptGet,\n    JmapVacationResponseGet,\n    JmapPrincipalGet,\n    JmapQuotaGet,\n    JmapBlobGet,\n    JmapEmailSet,\n    JmapMailboxSet,\n    JmapIdentitySet,\n    JmapEmailSubmissionSet,\n    JmapPushSubscriptionSet,\n    JmapSieveScriptSet,\n    JmapVacationResponseSet,\n    JmapEmailChanges,\n    JmapMailboxChanges,\n    JmapThreadChanges,\n    JmapIdentityChanges,\n    JmapEmailSubmissionChanges,\n    JmapQuotaChanges,\n    JmapEmailCopy,\n    JmapBlobCopy,\n    JmapEmailImport,\n    JmapEmailParse,\n    JmapEmailQueryChanges,\n    JmapMailboxQueryChanges,\n    JmapEmailSubmissionQueryChanges,\n    JmapSieveScriptQueryChanges,\n    JmapPrincipalQueryChanges,\n    JmapQuotaQueryChanges,\n    JmapEmailQuery,\n    JmapMailboxQuery,\n    JmapEmailSubmissionQuery,\n    JmapSieveScriptQuery,\n    JmapPrincipalQuery,\n    JmapQuotaQuery,\n    JmapSearchSnippet,\n    JmapSieveScriptValidate,\n    JmapBlobLookup,\n    JmapBlobUpload,\n    JmapEcho,\n\n    // IMAP\n    ImapAuthenticate,\n    ImapAclGet,\n    ImapAclSet,\n    ImapMyRights,\n    ImapListRights,\n    ImapAppend,\n    ImapCapability,\n    ImapId,\n    ImapCopy,\n    ImapMove,\n    ImapCreate,\n    ImapDelete,\n    ImapEnable,\n    ImapExpunge,\n    ImapFetch,\n    ImapIdle,\n    ImapList,\n    ImapLsub,\n    ImapNamespace,\n    ImapRename,\n    ImapSearch,\n    ImapSort,\n    ImapSelect,\n    ImapExamine,\n    ImapStatus,\n    ImapStore,\n    ImapSubscribe,\n    ImapThread,\n\n    // POP3\n    Pop3Authenticate,\n    Pop3List,\n    Pop3Uidl,\n    Pop3Stat,\n    Pop3Retr,\n    Pop3Dele,\n\n    // ManageSieve\n    SieveAuthenticate,\n    SieveListScripts,\n    SieveSetActive,\n    SieveGetScript,\n    SievePutScript,\n    SieveDeleteScript,\n    SieveRenameScript,\n    SieveCheckScript,\n    SieveHaveSpace,\n\n    // API keys\n    ApiKeyList,\n    ApiKeyGet,\n    ApiKeyCreate,\n    ApiKeyUpdate,\n    ApiKeyDelete,\n\n    // OAuth clients\n    OauthClientList,\n    OauthClientGet,\n    OauthClientCreate,\n    OauthClientUpdate,\n    OauthClientDelete,\n\n    // OAuth client registration\n    OauthClientRegistration,\n    OauthClientOverride,\n\n    AiModelInteract,\n    Troubleshoot,\n    SpamFilterTest,\n\n    // WebDAV permissions\n    DavSyncCollection,\n    DavExpandProperty,\n\n    DavPrincipalAcl,\n    DavPrincipalList,\n    DavPrincipalMatch,\n    DavPrincipalSearch,\n    DavPrincipalSearchPropSet,\n\n    DavFilePropFind,\n    DavFilePropPatch,\n    DavFileGet,\n    DavFileMkCol,\n    DavFileDelete,\n    DavFilePut,\n    DavFileCopy,\n    DavFileMove,\n    DavFileLock,\n    DavFileAcl,\n\n    DavCardPropFind,\n    DavCardPropPatch,\n    DavCardGet,\n    DavCardMkCol,\n    DavCardDelete,\n    DavCardPut,\n    DavCardCopy,\n    DavCardMove,\n    DavCardLock,\n    DavCardAcl,\n    DavCardQuery,\n    DavCardMultiGet,\n\n    DavCalPropFind,\n    DavCalPropPatch,\n    DavCalGet,\n    DavCalMkCol,\n    DavCalDelete,\n    DavCalPut,\n    DavCalCopy,\n    DavCalMove,\n    DavCalLock,\n    DavCalAcl,\n    DavCalQuery,\n    DavCalMultiGet,\n    DavCalFreeBusyQuery,\n\n    CalendarAlarms,\n    CalendarSchedulingSend,\n    CalendarSchedulingReceive,\n\n    JmapAddressBookGet,\n    JmapAddressBookSet,\n    JmapAddressBookChanges,\n\n    JmapContactCardGet,\n    JmapContactCardChanges,\n    JmapContactCardQuery,\n    JmapContactCardQueryChanges,\n    JmapContactCardSet,\n    JmapContactCardCopy,\n    JmapContactCardParse,\n\n    JmapFileNodeGet,\n    JmapFileNodeSet,\n    JmapFileNodeChanges,\n    JmapFileNodeQuery,\n    JmapFileNodeQueryChanges,\n\n    JmapPrincipalGetAvailability,\n    JmapPrincipalChanges,\n\n    JmapShareNotificationGet,\n    JmapShareNotificationSet,\n    JmapShareNotificationChanges,\n    JmapShareNotificationQuery,\n    JmapShareNotificationQueryChanges,\n\n    JmapCalendarGet,\n    JmapCalendarSet,\n    JmapCalendarChanges,\n\n    JmapCalendarEventGet,\n    JmapCalendarEventSet,\n    JmapCalendarEventChanges,\n    JmapCalendarEventQuery,\n    JmapCalendarEventQueryChanges,\n    JmapCalendarEventCopy,\n    JmapCalendarEventParse,\n\n    JmapCalendarEventNotificationGet,\n    JmapCalendarEventNotificationSet,\n    JmapCalendarEventNotificationChanges,\n    JmapCalendarEventNotificationQuery,\n    JmapCalendarEventNotificationQueryChanges,\n\n    JmapParticipantIdentityGet,\n    JmapParticipantIdentitySet,\n    JmapParticipantIdentityChanges,\n    // TODO: Reuse _ suffixes for new permissions\n    // WARNING: add new ids at the end (TODO: use static ids)\n}\n\npub const PERMISSIONS_BITSET_SIZE: usize = Permission::COUNT.div_ceil(std::mem::size_of::<u32>());\npub type Permissions = Bitset<PERMISSIONS_BITSET_SIZE>;\n\npub const ROLE_ADMIN: u32 = u32::MAX;\npub const ROLE_TENANT_ADMIN: u32 = u32::MAX - 1;\npub const ROLE_USER: u32 = u32::MAX - 2;\n\npub enum DirectoryInner {\n    Internal(Store),\n    Ldap(LdapDirectory),\n    Sql(SqlDirectory),\n    OpenId(backend::oidc::OpenIdDirectory),\n    Imap(ImapDirectory),\n    Smtp(SmtpDirectory),\n    Memory(MemoryDirectory),\n}\n\npub enum QueryBy<'x> {\n    Name(&'x str),\n    Id(u32),\n    Credentials(&'x Credentials<String>),\n}\n\npub struct QueryParams<'x> {\n    pub by: QueryBy<'x>,\n    pub return_member_of: bool,\n    pub only_app_pass: bool,\n}\n\nimpl Default for Directory {\n    fn default() -> Self {\n        Self {\n            store: DirectoryInner::Internal(Store::None),\n            cache: None,\n        }\n    }\n}\n\nimpl Debug for Directory {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.debug_struct(\"Directory\").finish()\n    }\n}\n\n#[derive(Default, Clone, Debug)]\npub struct Directories {\n    pub directories: AHashMap<String, Arc<Directory>>,\n}\n\ntrait IntoError {\n    fn into_error(self) -> trc::Error;\n}\n\nimpl IntoError for PoolError<LdapError> {\n    fn into_error(self) -> trc::Error {\n        match self {\n            PoolError::Backend(error) => error.into_error(),\n            PoolError::Timeout(_) => trc::StoreEvent::PoolError\n                .into_err()\n                .details(\"Connection timed out\"),\n            err => trc::StoreEvent::PoolError.reason(err),\n        }\n    }\n}\n\nimpl IntoError for PoolError<ImapError> {\n    fn into_error(self) -> trc::Error {\n        match self {\n            PoolError::Backend(error) => error.into_error(),\n            PoolError::Timeout(_) => trc::StoreEvent::PoolError\n                .into_err()\n                .details(\"Connection timed out\"),\n            err => trc::StoreEvent::PoolError.reason(err),\n        }\n    }\n}\n\nimpl IntoError for PoolError<mail_send::Error> {\n    fn into_error(self) -> trc::Error {\n        match self {\n            PoolError::Backend(error) => error.into_error(),\n            PoolError::Timeout(_) => trc::StoreEvent::PoolError\n                .into_err()\n                .details(\"Connection timed out\"),\n            err => trc::StoreEvent::PoolError.reason(err),\n        }\n    }\n}\n\nimpl IntoError for ImapError {\n    fn into_error(self) -> trc::Error {\n        trc::ImapEvent::Error.into_err().reason(self)\n    }\n}\n\nimpl IntoError for mail_send::Error {\n    fn into_error(self) -> trc::Error {\n        trc::SmtpEvent::Error.into_err().reason(self)\n    }\n}\n\nimpl IntoError for LdapError {\n    fn into_error(self) -> trc::Error {\n        if let LdapError::LdapResult { result } = &self {\n            trc::StoreEvent::LdapError\n                .ctx(trc::Key::Code, result.rc)\n                .reason(self)\n        } else {\n            trc::StoreEvent::LdapError.reason(self)\n        }\n    }\n}\n\nimpl From<&ArchivedType> for Type {\n    fn from(archived: &ArchivedType) -> Self {\n        match archived {\n            ArchivedType::Individual => Type::Individual,\n            ArchivedType::Group => Type::Group,\n            ArchivedType::Resource => Type::Resource,\n            ArchivedType::Location => Type::Location,\n            ArchivedType::List => Type::List,\n            ArchivedType::Other => Type::Other,\n            ArchivedType::Domain => Type::Domain,\n            ArchivedType::Tenant => Type::Tenant,\n            ArchivedType::Role => Type::Role,\n            ArchivedType::ApiKey => Type::ApiKey,\n            ArchivedType::OauthClient => Type::OauthClient,\n        }\n    }\n}\n\nimpl<'x> QueryParams<'x> {\n    pub fn name(name: &'x str) -> Self {\n        QueryParams {\n            by: QueryBy::Name(name),\n            return_member_of: false,\n            only_app_pass: false,\n        }\n    }\n\n    pub fn credentials(credentials: &'x Credentials<String>) -> Self {\n        QueryParams {\n            by: QueryBy::Credentials(credentials),\n            return_member_of: false,\n            only_app_pass: false,\n        }\n    }\n\n    pub fn id(id: u32) -> Self {\n        QueryParams {\n            by: QueryBy::Id(id),\n            return_member_of: false,\n            only_app_pass: false,\n        }\n    }\n\n    pub fn by(by: QueryBy<'x>) -> Self {\n        QueryParams {\n            by,\n            return_member_of: false,\n            only_app_pass: false,\n        }\n    }\n\n    pub fn with_return_member_of(mut self, return_member_of: bool) -> Self {\n        self.return_member_of = return_member_of;\n        self\n    }\n\n    pub fn with_only_app_pass(mut self, only_app_pass: bool) -> Self {\n        self.only_app_pass = only_app_pass;\n        self\n    }\n}\n"
  },
  {
    "path": "crates/email/Cargo.toml",
    "content": "[package]\nname = \"email\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path = \"../utils\" }\nnlp = { path = \"../nlp\" }\nstore = { path = \"../store\" }\ntrc = { path = \"../trc\" }\ntypes = { path = \"../types\" }\ncommon = { path =  \"../common\" }\ndirectory = { path =  \"../directory\" }\ngroupware = { path =  \"../groupware\" }\nspam-filter = { path =  \"../spam-filter\" }\nsmtp-proto = { version = \"0.2\", features = [\"rkyv\"] }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-builder = { version = \"0.4\" }\nsieve-rs = { version = \"0.7\", features = [\"rkyv\"] } \ntokio = { version = \"1.47\", features = [\"net\", \"macros\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\naes = \"0.8.3\"\naes-gcm = \"0.10.1\"\naes-gcm-siv = \"0.11.1\"\ncbc = { version = \"0.1.2\", features = [\"alloc\"] }\nrasn = \"0.10\"\nrasn-cms = \"0.10\"\nrasn-pkix = \"0.10\"\nrsa = \"0.9.2\"\nrand = \"0.8\"\nsequoia-openpgp = { version = \"2.0\", default-features = false, features = [\"crypto-rust\", \"allow-experimental-crypto\", \"allow-variable-time-crypto\"] }\nhashify = \"0.2\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\n\n[features]\ntest_mode = []\nenterprise = []\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/email/src/cache/email.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::message::metadata::{ArchivedMessageData, MessageData};\nuse common::{\n    MessageCache, MessageStoreCache, MessageUidCache, MessagesCache, Server, auth::AccessToken,\n    sharing::EffectiveAcl,\n};\nuse store::write::{AlignedBytes, Archive};\nuse store::{ValueKey, ahash::AHashMap, roaring::RoaringBitmap};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::Collection,\n    keyword::{Keyword, OTHER},\n};\nuse utils::map::bitmap::Bitmap;\n\nstruct MessagesCacheBuilder {\n    pub change_id: u64,\n    pub items: Vec<MessageCache>,\n    pub index: AHashMap<u32, u32>,\n    pub keywords: Vec<Box<str>>,\n    pub size: u64,\n}\n\npub(crate) async fn update_email_cache(\n    server: &Server,\n    account_id: u32,\n    changed_ids: &AHashMap<u32, bool>,\n    store_cache: &MessageStoreCache,\n) -> trc::Result<MessagesCache> {\n    let mut new_cache = MessagesCacheBuilder {\n        index: AHashMap::with_capacity(store_cache.emails.items.len()),\n        items: Vec::with_capacity(store_cache.emails.items.len()),\n        size: 0,\n        change_id: 0,\n        keywords: store_cache.emails.keywords.to_vec(),\n    };\n\n    for (document_id, is_update) in changed_ids {\n        if *is_update\n            && let Some(archive) = server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Email,\n                    *document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n        {\n            insert_item(\n                &mut new_cache,\n                *document_id,\n                archive.to_unarchived::<MessageData>()?,\n            );\n        }\n    }\n\n    for item in &store_cache.emails.items {\n        if !changed_ids.contains_key(&item.document_id) {\n            email_insert(&mut new_cache, item.clone());\n        }\n    }\n\n    Ok(new_cache.build())\n}\n\npub(crate) async fn full_email_cache_build(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<MessagesCache> {\n    // Build cache\n    let mut cache = MessagesCacheBuilder {\n        items: Vec::with_capacity(16),\n        index: AHashMap::with_capacity(16),\n        keywords: Vec::new(),\n        size: 0,\n        change_id: 0,\n    };\n\n    server\n        .archives(\n            account_id,\n            Collection::Email,\n            &(),\n            |document_id, archive| {\n                insert_item(\n                    &mut cache,\n                    document_id,\n                    archive.to_unarchived::<MessageData>()?,\n                );\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(cache.build())\n}\n\nfn insert_item(\n    cache: &mut MessagesCacheBuilder,\n    document_id: u32,\n    archive: Archive<&ArchivedMessageData>,\n) {\n    let message = archive.inner;\n    let mut item = MessageCache {\n        mailboxes: message\n            .mailboxes\n            .iter()\n            .map(|m| MessageUidCache {\n                mailbox_id: m.mailbox_id.to_native(),\n                uid: m.uid.to_native(),\n            })\n            .collect(),\n        keywords: 0,\n        thread_id: message.thread_id.to_native(),\n        change_id: archive.version.change_id().unwrap_or_default(),\n        document_id,\n        size: message.size.to_native(),\n    };\n    for keyword in message.keywords.iter() {\n        match keyword.id() {\n            Ok(id) => {\n                item.keywords |= 1 << id;\n            }\n            Err(custom) => {\n                if let Some(idx) = cache.keywords.iter().position(|k| **k == *custom) {\n                    item.keywords |= 1 << (OTHER + idx);\n                } else if cache.keywords.len() < (128 - OTHER) {\n                    cache.keywords.push(custom.into());\n                    item.keywords |= 1 << (OTHER + cache.keywords.len() - 1);\n                }\n            }\n        }\n    }\n\n    email_insert(cache, item);\n}\n\nimpl MessagesCacheBuilder {\n    pub fn build(mut self) -> MessagesCache {\n        self.index.shrink_to_fit();\n        MessagesCache {\n            change_id: self.change_id,\n            items: self.items.into_boxed_slice(),\n            index: self.index,\n            keywords: self.keywords.into_boxed_slice(),\n            size: self.size,\n        }\n    }\n}\n\npub trait MessageCacheAccess {\n    fn email_by_id(&self, id: &u32) -> Option<&MessageCache>;\n\n    fn has_email_id(&self, id: &u32) -> bool;\n\n    fn in_mailbox(&self, mailbox_id: u32) -> impl Iterator<Item = &MessageCache>;\n\n    fn in_mailboxes(&self, mailbox_ids: &[u32]) -> impl Iterator<Item = &MessageCache>;\n\n    fn in_thread(&self, thread_id: u32) -> impl Iterator<Item = &MessageCache>;\n\n    fn with_keyword(&self, keyword: &Keyword) -> impl Iterator<Item = &MessageCache>;\n\n    fn without_keyword(&self, keyword: &Keyword) -> impl Iterator<Item = &MessageCache>;\n\n    fn in_mailbox_with_keyword(\n        &self,\n        mailbox_id: u32,\n        keyword: &Keyword,\n    ) -> impl Iterator<Item = &MessageCache>;\n\n    fn in_mailbox_without_keyword(\n        &self,\n        mailbox_id: u32,\n        keyword: &Keyword,\n    ) -> impl Iterator<Item = &MessageCache>;\n\n    fn email_document_ids(&self) -> RoaringBitmap;\n\n    fn shared_messages(\n        &self,\n        access_token: &AccessToken,\n        check_acls: impl Into<Bitmap<Acl>> + Sync + Send,\n    ) -> RoaringBitmap;\n\n    fn expand_keywords(&self, message: &MessageCache) -> impl Iterator<Item = Keyword>;\n\n    fn has_keyword(&self, message: &MessageCache, keyword: &Keyword) -> bool;\n}\n\nimpl MessageCacheAccess for MessageStoreCache {\n    fn in_mailbox(&self, mailbox_id: u32) -> impl Iterator<Item = &MessageCache> {\n        self.emails\n            .items\n            .iter()\n            .filter(move |m| m.mailboxes.iter().any(|m| m.mailbox_id == mailbox_id))\n    }\n\n    fn in_mailboxes(&self, mailbox_ids: &[u32]) -> impl Iterator<Item = &MessageCache> {\n        self.emails.items.iter().filter(move |m| {\n            m.mailboxes\n                .iter()\n                .any(|mb| mailbox_ids.contains(&mb.mailbox_id))\n        })\n    }\n\n    fn in_thread(&self, thread_id: u32) -> impl Iterator<Item = &MessageCache> {\n        self.emails\n            .items\n            .iter()\n            .filter(move |m| m.thread_id == thread_id)\n    }\n\n    fn with_keyword(&self, keyword: &Keyword) -> impl Iterator<Item = &MessageCache> {\n        let keyword_id = keyword_to_id(self, keyword);\n        self.emails\n            .items\n            .iter()\n            .filter(move |m| keyword_id.is_some_and(|id| m.keywords & (1 << id) != 0))\n    }\n\n    fn without_keyword(&self, keyword: &Keyword) -> impl Iterator<Item = &MessageCache> {\n        let keyword_id = keyword_to_id(self, keyword);\n        self.emails\n            .items\n            .iter()\n            .filter(move |m| keyword_id.is_none_or(|id| m.keywords & (1 << id) == 0))\n    }\n\n    fn in_mailbox_with_keyword(\n        &self,\n        mailbox_id: u32,\n        keyword: &Keyword,\n    ) -> impl Iterator<Item = &MessageCache> {\n        let keyword_id = keyword_to_id(self, keyword);\n        self.emails.items.iter().filter(move |m| {\n            m.mailboxes.iter().any(|m| m.mailbox_id == mailbox_id)\n                && keyword_id.is_some_and(|id| m.keywords & (1 << id) != 0)\n        })\n    }\n\n    fn in_mailbox_without_keyword(\n        &self,\n        mailbox_id: u32,\n        keyword: &Keyword,\n    ) -> impl Iterator<Item = &MessageCache> {\n        let keyword_id = keyword_to_id(self, keyword);\n        self.emails.items.iter().filter(move |m| {\n            m.mailboxes.iter().any(|m| m.mailbox_id == mailbox_id)\n                && keyword_id.is_none_or(|id| m.keywords & (1 << id) == 0)\n        })\n    }\n\n    fn shared_messages(\n        &self,\n        access_token: &AccessToken,\n        check_acls: impl Into<Bitmap<Acl>> + Sync + Send,\n    ) -> RoaringBitmap {\n        let check_acls = check_acls.into();\n        let mut shared_messages = RoaringBitmap::new();\n        for mailbox in &self.mailboxes.items {\n            if mailbox\n                .acls\n                .as_slice()\n                .effective_acl(access_token)\n                .contains_all(check_acls)\n            {\n                shared_messages.extend(\n                    self.in_mailbox(mailbox.document_id)\n                        .map(|item| item.document_id),\n                );\n            }\n        }\n        shared_messages\n    }\n\n    fn email_document_ids(&self) -> RoaringBitmap {\n        RoaringBitmap::from_iter(self.emails.index.keys())\n    }\n\n    fn email_by_id(&self, id: &u32) -> Option<&MessageCache> {\n        self.emails\n            .index\n            .get(id)\n            .and_then(|idx| self.emails.items.get(*idx as usize))\n    }\n\n    fn has_email_id(&self, id: &u32) -> bool {\n        self.emails.index.contains_key(id)\n    }\n\n    fn expand_keywords(&self, message: &MessageCache) -> impl Iterator<Item = Keyword> {\n        KeywordsIter(message.keywords).map(move |id| match Keyword::try_from_id(id) {\n            Ok(keyword) => keyword,\n            Err(id) => Keyword::Other(self.emails.keywords[id - OTHER].clone()),\n        })\n    }\n\n    fn has_keyword(&self, message: &MessageCache, keyword: &Keyword) -> bool {\n        keyword_to_id(self, keyword).is_some_and(|id| message.keywords & (1 << id) != 0)\n    }\n}\n\nfn email_insert(cache: &mut MessagesCacheBuilder, item: MessageCache) {\n    let id = item.document_id;\n    if let Some(idx) = cache.index.get(&id) {\n        cache.items[*idx as usize] = item;\n    } else {\n        cache.size += (std::mem::size_of::<MessageCache>()\n            + (std::mem::size_of::<u32>() * 2)\n            + (item.mailboxes.len() * std::mem::size_of::<MessageUidCache>()))\n            as u64;\n\n        let idx = cache.items.len() as u32;\n        cache.items.push(item);\n        cache.index.insert(id, idx);\n    }\n}\n\n#[inline]\nfn keyword_to_id(cache: &MessageStoreCache, keyword: &Keyword) -> Option<u32> {\n    match keyword.id() {\n        Ok(id) => Some(id),\n        Err(name) => cache\n            .emails\n            .keywords\n            .iter()\n            .position(|k| **k == *name)\n            .map(|idx| (OTHER + idx) as u32),\n    }\n}\n\n#[derive(Clone, Copy, Debug)]\nstruct KeywordsIter(u128);\n\nimpl Iterator for KeywordsIter {\n    type Item = usize;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        if self.0 != 0 {\n            let item = 127 - self.0.leading_zeros();\n            self.0 ^= 1 << item;\n            Some(item as usize)\n        } else {\n            None\n        }\n    }\n}\n"
  },
  {
    "path": "crates/email/src/cache/mailbox.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::mailbox::{ArchivedMailbox, Mailbox, manage::MailboxFnc};\nuse common::{\n    MailboxCache, MailboxesCache, MessageStoreCache, Server, auth::AccessToken,\n    sharing::EffectiveAcl,\n};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse store::{ahash::AHashMap, roaring::RoaringBitmap};\nuse trc::AddContext;\nuse types::{\n    acl::{Acl, AclGrant},\n    collection::Collection,\n    special_use::SpecialUse,\n};\nuse utils::{map::bitmap::Bitmap, topological::TopologicalSort};\n\nstruct MailboxesCacheBuilder {\n    pub change_id: u64,\n    pub index: AHashMap<u32, u32>,\n    pub items: Vec<MailboxCache>,\n    pub size: u64,\n}\n\npub(crate) async fn update_mailbox_cache(\n    server: &Server,\n    account_id: u32,\n    changed_ids: &AHashMap<u32, bool>,\n    store_cache: &MessageStoreCache,\n) -> trc::Result<MailboxesCache> {\n    let mut new_cache = MailboxesCacheBuilder {\n        items: Vec::with_capacity(store_cache.mailboxes.items.len()),\n        index: AHashMap::with_capacity(store_cache.mailboxes.items.len()),\n        size: 0,\n        change_id: 0,\n    };\n\n    for (document_id, is_update) in changed_ids {\n        if *is_update\n            && let Some(archive) = server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Mailbox,\n                    *document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n        {\n            insert_item(\n                &mut new_cache,\n                *document_id,\n                archive.unarchive::<Mailbox>()?,\n            );\n        }\n    }\n\n    for item in store_cache.mailboxes.items.iter() {\n        if !changed_ids.contains_key(&item.document_id) {\n            mailbox_insert(&mut new_cache, item.clone());\n        }\n    }\n\n    build_tree(&mut new_cache);\n\n    Ok(new_cache.build())\n}\n\npub(crate) async fn full_mailbox_cache_build(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<MailboxesCache> {\n    // Build cache\n    let mut cache = MailboxesCacheBuilder {\n        items: Default::default(),\n        index: Default::default(),\n        size: 0,\n        change_id: 0,\n    };\n\n    server\n        .archives(\n            account_id,\n            Collection::Mailbox,\n            &(),\n            |document_id, archive| {\n                insert_item(&mut cache, document_id, archive.unarchive::<Mailbox>()?);\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    if cache.items.is_empty() {\n        server\n            .create_system_folders(account_id)\n            .await\n            .caused_by(trc::location!())?;\n        server\n            .archives(\n                account_id,\n                Collection::Mailbox,\n                &(),\n                |document_id, archive| {\n                    insert_item(&mut cache, document_id, archive.unarchive::<Mailbox>()?);\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    build_tree(&mut cache);\n\n    Ok(cache.build())\n}\n\nfn insert_item(cache: &mut MailboxesCacheBuilder, document_id: u32, mailbox: &ArchivedMailbox) {\n    let parent_id = mailbox.parent_id.to_native();\n    let item = MailboxCache {\n        document_id,\n        name: mailbox.name.as_str().into(),\n        path: \"\".into(),\n        role: (&mailbox.role).into(),\n        parent_id: if parent_id > 0 {\n            parent_id - 1\n        } else {\n            u32::MAX\n        },\n        sort_order: mailbox\n            .sort_order\n            .as_ref()\n            .map(|s| s.to_native())\n            .unwrap_or(u32::MAX),\n        subscribers: mailbox.subscribers.iter().map(|s| s.to_native()).collect(),\n        uid_validity: mailbox.uid_validity.to_native(),\n        acls: mailbox\n            .acls\n            .iter()\n            .map(|acl| AclGrant {\n                account_id: acl.account_id.to_native(),\n                grants: Bitmap::from(&acl.grants),\n            })\n            .collect(),\n    };\n\n    mailbox_insert(cache, item);\n}\n\nfn build_tree(cache: &mut MailboxesCacheBuilder) {\n    cache.size = 0;\n    let mut topological_sort = TopologicalSort::with_capacity(cache.items.len());\n\n    for (idx, mailbox) in cache.items.iter_mut().enumerate() {\n        topological_sort.insert(\n            if mailbox.parent_id == u32::MAX {\n                0\n            } else {\n                mailbox.parent_id + 1\n            },\n            mailbox.document_id + 1,\n        );\n        mailbox.path = if matches!(mailbox.role, SpecialUse::Inbox) {\n            \"INBOX\".into()\n        } else if mailbox.is_root() && mailbox.name.as_str().eq_ignore_ascii_case(\"inbox\") {\n            format!(\"INBOX {}\", idx + 1)\n        } else {\n            mailbox.name.clone()\n        };\n\n        cache.size += item_size(mailbox);\n    }\n\n    for folder_id in topological_sort.into_iterator() {\n        if folder_id != 0 {\n            let folder_id = folder_id - 1;\n            if let Some((path, parent_path)) = by_id(cache, &folder_id)\n                .and_then(|folder| {\n                    folder\n                        .parent_id()\n                        .map(|parent_id| (&folder.path, parent_id))\n                })\n                .and_then(|(path, parent_id)| {\n                    by_id(cache, &parent_id).map(|folder| (path, &folder.path))\n                })\n            {\n                let mut new_path = String::with_capacity(parent_path.len() + path.len() + 1);\n                new_path.push_str(parent_path.as_str());\n                new_path.push('/');\n                new_path.push_str(path.as_str());\n                let folder = by_id_mut(cache, &folder_id).unwrap();\n                folder.path = new_path;\n            }\n        }\n    }\n}\n\nimpl MailboxesCacheBuilder {\n    fn build(mut self) -> MailboxesCache {\n        self.index.shrink_to_fit();\n        MailboxesCache {\n            change_id: self.change_id,\n            index: self.index,\n            items: self.items.into_boxed_slice(),\n            size: self.size,\n        }\n    }\n}\n\npub trait MailboxCacheAccess {\n    fn mailbox_by_id(&self, id: &u32) -> Option<&MailboxCache>;\n    fn mailbox_by_name(&self, name: &str) -> Option<&MailboxCache>;\n    fn mailbox_by_path(&self, name: &str) -> Option<&MailboxCache>;\n    fn mailbox_by_role(&self, role: &SpecialUse) -> Option<&MailboxCache>;\n    fn shared_mailboxes(\n        &self,\n        access_token: &AccessToken,\n        check_acls: impl Into<Bitmap<Acl>> + Sync + Send,\n    ) -> RoaringBitmap;\n    fn has_mailbox_id(&self, id: &u32) -> bool;\n}\n\nimpl MailboxCacheAccess for MessageStoreCache {\n    fn mailbox_by_name(&self, name: &str) -> Option<&MailboxCache> {\n        self.mailboxes\n            .items\n            .iter()\n            .find(|m| m.name.eq_ignore_ascii_case(name))\n    }\n\n    fn mailbox_by_path(&self, path: &str) -> Option<&MailboxCache> {\n        self.mailboxes\n            .items\n            .iter()\n            .find(|m| m.path.eq_ignore_ascii_case(path))\n    }\n\n    fn mailbox_by_role(&self, role: &SpecialUse) -> Option<&MailboxCache> {\n        self.mailboxes.items.iter().find(|m| &m.role == role)\n    }\n\n    fn shared_mailboxes(\n        &self,\n        access_token: &AccessToken,\n        check_acls: impl Into<Bitmap<Acl>> + Sync + Send,\n    ) -> RoaringBitmap {\n        let check_acls = check_acls.into();\n\n        RoaringBitmap::from_iter(\n            self.mailboxes\n                .items\n                .iter()\n                .filter(|m| {\n                    m.acls\n                        .as_slice()\n                        .effective_acl(access_token)\n                        .contains_all(check_acls)\n                })\n                .map(|m| m.document_id),\n        )\n    }\n\n    fn mailbox_by_id(&self, id: &u32) -> Option<&MailboxCache> {\n        self.mailboxes\n            .index\n            .get(id)\n            .and_then(|idx| self.mailboxes.items.get(*idx as usize))\n    }\n\n    fn has_mailbox_id(&self, id: &u32) -> bool {\n        self.mailboxes.index.contains_key(id)\n    }\n}\n\n#[inline(always)]\nfn by_id<'x>(cache: &'x MailboxesCacheBuilder, id: &u32) -> Option<&'x MailboxCache> {\n    cache\n        .index\n        .get(id)\n        .and_then(|idx| cache.items.get(*idx as usize))\n}\n\n#[inline(always)]\nfn by_id_mut<'x>(cache: &'x mut MailboxesCacheBuilder, id: &u32) -> Option<&'x mut MailboxCache> {\n    cache\n        .index\n        .get(id)\n        .and_then(|idx| cache.items.get_mut(*idx as usize))\n}\n\nfn mailbox_insert(cache: &mut MailboxesCacheBuilder, item: MailboxCache) {\n    let id = item.document_id;\n    if let Some(idx) = cache.index.get(&id) {\n        cache.items[*idx as usize] = item;\n    } else {\n        let idx = cache.items.len() as u32;\n        cache.items.push(item);\n        cache.index.insert(id, idx);\n    }\n}\n\n#[inline(always)]\nfn item_size(item: &MailboxCache) -> u64 {\n    (std::mem::size_of::<MailboxCache>()\n        + (if item.name.len() > std::mem::size_of::<String>() {\n            item.name.len()\n        } else {\n            0\n        })\n        + (if item.path.len() > std::mem::size_of::<String>() {\n            item.path.len()\n        } else {\n            0\n        })) as u64\n}\n"
  },
  {
    "path": "crates/email/src/cache/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{CacheSwap, MessageStoreCache, Server};\nuse email::{full_email_cache_build, update_email_cache};\nuse mailbox::{full_mailbox_cache_build, update_mailbox_cache};\nuse std::{collections::hash_map::Entry, sync::Arc, time::Instant};\nuse store::{\n    ahash::AHashMap,\n    query::log::{Change, Query},\n};\nuse tokio::sync::Semaphore;\nuse trc::{AddContext, StoreEvent};\nuse types::collection::SyncCollection;\n\npub mod email;\npub mod mailbox;\n\npub trait MessageCacheFetch: Sync + Send {\n    fn get_cached_messages(\n        &self,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Arc<MessageStoreCache>>> + Send;\n}\n\nimpl MessageCacheFetch for Server {\n    async fn get_cached_messages(&self, account_id: u32) -> trc::Result<Arc<MessageStoreCache>> {\n        let cache_ = match self\n            .inner\n            .cache\n            .messages\n            .get_value_or_guard_async(&account_id)\n            .await\n        {\n            Ok(cache) => cache,\n            Err(guard) => {\n                let start_time = Instant::now();\n                let cache = full_cache_build(self, account_id, Arc::new(Semaphore::new(1))).await?;\n\n                if guard.insert(CacheSwap::new(cache.clone())).is_err() {\n                    self.inner\n                        .cache\n                        .messages\n                        .insert(account_id, CacheSwap::new(cache.clone()));\n                }\n\n                trc::event!(\n                    Store(StoreEvent::CacheMiss),\n                    AccountId = account_id,\n                    Collection = SyncCollection::Email.as_str(),\n                    Total = vec![cache.emails.items.len(), cache.mailboxes.items.len()],\n                    ChangeId = cache.last_change_id,\n                    Elapsed = start_time.elapsed(),\n                );\n\n                return Ok(cache);\n            }\n        };\n\n        // Obtain current state\n        let cache = cache_.load_full();\n        let start_time = Instant::now();\n        let changes = self\n            .core\n            .storage\n            .data\n            .changes(\n                account_id,\n                SyncCollection::Email.into(),\n                Query::Since(cache.last_change_id),\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        // Regenerate cache if the change log has been truncated\n        if changes.is_truncated {\n            let cache = full_cache_build(self, account_id, cache.update_lock.clone()).await?;\n            cache_.update(cache.clone());\n\n            trc::event!(\n                Store(StoreEvent::CacheStale),\n                AccountId = account_id,\n                Collection = SyncCollection::Email.as_str(),\n                ChangeId = cache.last_change_id,\n                Total = vec![cache.emails.items.len(), cache.mailboxes.items.len()],\n                Elapsed = start_time.elapsed(),\n            );\n\n            return Ok(cache);\n        }\n\n        // Verify changes\n        if changes.changes.is_empty() {\n            trc::event!(\n                Store(StoreEvent::CacheHit),\n                AccountId = account_id,\n                Collection = SyncCollection::Email.as_str(),\n                ChangeId = cache.last_change_id,\n                Elapsed = start_time.elapsed(),\n            );\n\n            return Ok(cache);\n        }\n\n        // Lock for updates\n        let _permit = cache.update_lock.acquire().await;\n        let cache = cache_.0.load();\n        let mut cache = if cache.last_change_id >= changes.to_change_id {\n            trc::event!(\n                Store(StoreEvent::CacheHit),\n                AccountId = account_id,\n                Collection = SyncCollection::Email.as_str(),\n                ChangeId = cache.last_change_id,\n                Elapsed = start_time.elapsed(),\n            );\n\n            return Ok(cache.clone());\n        } else {\n            cache.as_ref().clone()\n        };\n\n        let mut changed_items: AHashMap<u32, bool> = AHashMap::with_capacity(changes.changes.len());\n        let mut changed_containers: AHashMap<u32, bool> =\n            AHashMap::with_capacity(changes.changes.len());\n        let mut has_container_property_changes = false;\n\n        for change in changes.changes {\n            match change {\n                Change::InsertItem(id) => match changed_items.entry(id as u32) {\n                    Entry::Occupied(mut entry) => {\n                        *entry.get_mut() = true;\n                    }\n                    Entry::Vacant(entry) => {\n                        entry.insert(true);\n                    }\n                },\n                Change::UpdateItem(id) => {\n                    changed_items.insert(id as u32, true);\n                }\n                Change::DeleteItem(id) => {\n                    match changed_items.entry(id as u32) {\n                        Entry::Occupied(mut entry) => {\n                            // Thread reassignment\n                            *entry.get_mut() = true;\n                        }\n                        Entry::Vacant(entry) => {\n                            entry.insert(false);\n                        }\n                    }\n                }\n                Change::InsertContainer(id) | Change::UpdateContainer(id) => {\n                    changed_containers.insert(id as u32, true);\n                }\n                Change::DeleteContainer(id) => {\n                    changed_containers.insert(id as u32, false);\n                }\n                Change::UpdateContainerProperty(_) => {\n                    has_container_property_changes = true;\n                }\n            }\n        }\n\n        if !changed_items.is_empty() {\n            let mut email_cache =\n                update_email_cache(self, account_id, &changed_items, &cache).await?;\n            email_cache.change_id = changes.item_change_id.unwrap_or(changes.to_change_id);\n            cache.emails = Arc::new(email_cache);\n        }\n\n        if !changed_containers.is_empty() {\n            let mut mailbox_cache =\n                update_mailbox_cache(self, account_id, &changed_containers, &cache).await?;\n            mailbox_cache.change_id = changes.container_change_id.unwrap_or(changes.to_change_id);\n            cache.mailboxes = Arc::new(mailbox_cache);\n        } else if has_container_property_changes {\n            let mut mailbox_cache = cache.mailboxes.as_ref().clone();\n            mailbox_cache.change_id = changes.container_change_id.unwrap_or(changes.to_change_id);\n            cache.mailboxes = Arc::new(mailbox_cache);\n        }\n        cache.size = cache.emails.size + cache.mailboxes.size;\n        cache.last_change_id = changes.to_change_id;\n\n        let cache = Arc::new(cache);\n        cache_.update(cache.clone());\n\n        trc::event!(\n            Store(StoreEvent::CacheUpdate),\n            AccountId = account_id,\n            Collection = SyncCollection::Email.as_str(),\n            ChangeId = cache.last_change_id,\n            Details = vec![changed_items.len(), changed_containers.len()],\n            Total = vec![cache.emails.items.len(), cache.mailboxes.items.len()],\n            Elapsed = start_time.elapsed(),\n        );\n\n        Ok(cache)\n    }\n}\n\nasync fn full_cache_build(\n    server: &Server,\n    account_id: u32,\n    update_lock: Arc<Semaphore>,\n) -> trc::Result<Arc<MessageStoreCache>> {\n    let last_change_id = server\n        .core\n        .storage\n        .data\n        .get_last_change_id(account_id, SyncCollection::Email.into())\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let mut emails = full_email_cache_build(server, account_id).await?;\n    let mut mailboxes = full_mailbox_cache_build(server, account_id).await?;\n    let size = emails.size + mailboxes.size;\n    emails.change_id = last_change_id;\n    mailboxes.change_id = last_change_id;\n\n    Ok(Arc::new(MessageStoreCache {\n        update_lock,\n        emails: Arc::new(emails),\n        mailboxes: Arc::new(mailboxes),\n        last_change_id,\n        size,\n    }))\n}\n"
  },
  {
    "path": "crates/email/src/identity/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ArchivedIdentity, Identity};\nuse common::storage::index::{IndexValue, IndexableAndSerializableObject, IndexableObject};\nuse types::collection::SyncCollection;\n\nimpl IndexableObject for Identity {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [IndexValue::LogItem {\n            sync_collection: SyncCollection::Identity,\n            prefix: None,\n        }]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedIdentity {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [IndexValue::LogItem {\n            sync_collection: SyncCollection::Identity,\n            prefix: None,\n        }]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for Identity {\n    fn is_versioned() -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/email/src/identity/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod index;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct Identity {\n    pub name: String,\n    pub email: String,\n    pub reply_to: Option<Vec<EmailAddress>>,\n    pub bcc: Option<Vec<EmailAddress>>,\n    pub text_signature: String,\n    pub html_signature: String,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct EmailAddress {\n    pub name: Option<String>,\n    pub email: String,\n}\n"
  },
  {
    "path": "crates/email/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod cache;\npub mod identity;\npub mod mailbox;\npub mod message;\npub mod push;\npub mod sieve;\npub mod submission;\n"
  },
  {
    "path": "crates/email/src/mailbox/destroy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::*;\nuse crate::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    message::metadata::MessageData,\n};\nuse common::{\n    Server, auth::AccessToken, sharing::EffectiveAcl, storage::index::ObjectIndexBuilder,\n};\nuse store::{\n    SerializeInfallible,\n    roaring::RoaringBitmap,\n    write::{BatchBuilder, SearchIndex, TaskEpoch, TaskQueueClass, ValueClass},\n};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, VanishedCollection},\n    field::MailboxField,\n};\n\npub trait MailboxDestroy: Sync + Send {\n    fn mailbox_destroy(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        access_token: &AccessToken,\n        remove_emails: bool,\n    ) -> impl Future<Output = trc::Result<Result<Option<u64>, MailboxDestroyError>>> + Send;\n}\n\npub enum MailboxDestroyError {\n    CannotDestroy,\n    Forbidden,\n    HasChildren,\n    HasEmails,\n    NotFound,\n    AssertionFailed,\n}\n\nimpl MailboxDestroy for Server {\n    async fn mailbox_destroy(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        access_token: &AccessToken,\n        remove_emails: bool,\n    ) -> trc::Result<Result<Option<u64>, MailboxDestroyError>> {\n        // Internal folders cannot be deleted\n        #[cfg(not(feature = \"test_mode\"))]\n        if [INBOX_ID, TRASH_ID, JUNK_ID].contains(&document_id) {\n            return Ok(Err(MailboxDestroyError::CannotDestroy));\n        }\n\n        // Verify that this mailbox does not have sub-mailboxes\n        let cache = self\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n        if cache\n            .mailboxes\n            .items\n            .iter()\n            .any(|item| item.parent_id == document_id)\n        {\n            return Ok(Err(MailboxDestroyError::HasChildren));\n        }\n\n        // Verify that the mailbox is empty\n        let mut batch = BatchBuilder::new();\n\n        batch.with_account_id(account_id);\n\n        let message_ids =\n            RoaringBitmap::from_iter(cache.in_mailbox(document_id).map(|m| m.document_id));\n\n        if !message_ids.is_empty() {\n            if remove_emails {\n                // If the message is in multiple mailboxes, untag it from the current mailbox,\n                // otherwise delete it.\n\n                self.archives(\n                    account_id,\n                    Collection::Email,\n                    &message_ids,\n                    |message_id, message_data_| {\n                        // Remove mailbox from list\n                        let prev_message_data = message_data_\n                            .to_unarchived::<MessageData>()\n                            .caused_by(trc::location!())?;\n                        if !prev_message_data\n                            .inner\n                            .mailboxes\n                            .iter()\n                            .any(|id| id.mailbox_id == document_id)\n                        {\n                            return Ok(true);\n                        }\n\n                        if prev_message_data.inner.mailboxes.len() == 1 {\n                            // Delete message\n                            for mailbox in prev_message_data.inner.mailboxes.iter() {\n                                batch.log_vanished_item(\n                                    VanishedCollection::Email,\n                                    (mailbox.mailbox_id.to_native(), mailbox.uid.to_native()),\n                                );\n                            }\n                            batch\n                                .with_collection(Collection::Email)\n                                .with_document(message_id)\n                                .custom(\n                                    ObjectIndexBuilder::<_, ()>::new()\n                                        .with_access_token(access_token)\n                                        .with_current(prev_message_data),\n                                )\n                                .caused_by(trc::location!())?\n                                .set(\n                                    ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                                        index: SearchIndex::Email,\n                                        due: TaskEpoch::now(),\n                                        is_insert: false,\n                                    }),\n                                    0u64.serialize(),\n                                )\n                                .commit_point();\n                        } else {\n                            let new_message_data = MessageData {\n                                mailboxes: prev_message_data\n                                    .inner\n                                    .mailboxes\n                                    .iter()\n                                    .filter(|m| m.mailbox_id != document_id)\n                                    .map(|m| m.to_native())\n                                    .collect(),\n                                keywords: prev_message_data\n                                    .inner\n                                    .keywords\n                                    .iter()\n                                    .map(|k| k.to_native())\n                                    .collect(),\n                                thread_id: prev_message_data.inner.thread_id.to_native(),\n                                size: prev_message_data.inner.size.to_native(),\n                            };\n\n                            // Untag message from mailbox\n                            batch\n                                .with_collection(Collection::Email)\n                                .with_document(message_id)\n                                .custom(\n                                    ObjectIndexBuilder::new()\n                                        .with_access_token(access_token)\n                                        .with_changes(new_message_data)\n                                        .with_current(prev_message_data),\n                                )\n                                .caused_by(trc::location!())?\n                                .commit_point();\n                        }\n\n                        Ok(true)\n                    },\n                )\n                .await\n                .caused_by(trc::location!())?;\n            } else {\n                return Ok(Err(MailboxDestroyError::HasEmails));\n            }\n        }\n\n        // Obtain mailbox\n        if let Some(mailbox_) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::Mailbox,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n        {\n            let mailbox = mailbox_\n                .to_unarchived::<Mailbox>()\n                .caused_by(trc::location!())?;\n            // Validate ACLs\n            if access_token.is_shared(account_id) {\n                let acl = mailbox.inner.acls.effective_acl(access_token);\n                if !acl.contains(Acl::Delete) || (remove_emails && !acl.contains(Acl::RemoveItems))\n                {\n                    return Ok(Err(MailboxDestroyError::Forbidden));\n                }\n            }\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Mailbox)\n                .with_document(document_id)\n                .clear(MailboxField::UidCounter)\n                .custom(ObjectIndexBuilder::<_, ()>::new().with_current(mailbox))\n                .caused_by(trc::location!())?;\n        } else {\n            return Ok(Err(MailboxDestroyError::NotFound));\n        };\n\n        if !batch.is_empty() {\n            match self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n            {\n                Ok(change_id) => {\n                    self.notify_task_queue();\n\n                    Ok(Ok(Some(change_id)))\n                }\n                Err(err) if err.is_assertion_failure() => {\n                    Ok(Err(MailboxDestroyError::AssertionFailed))\n                }\n                Err(err) => Err(err.caused_by(trc::location!())),\n            }\n        } else {\n            Ok(Ok(None))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/email/src/mailbox/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ArchivedMailbox, Mailbox};\nuse common::storage::index::{IndexValue, IndexableAndSerializableObject, IndexableObject};\nuse types::{acl::AclGrant, collection::SyncCollection};\n\nimpl IndexableObject for Mailbox {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::LogContainer {\n                sync_collection: SyncCollection::Email,\n            },\n            IndexValue::Acl {\n                value: (&self.acls).into(),\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedMailbox {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::LogContainer {\n                sync_collection: SyncCollection::Email,\n            },\n            IndexValue::Acl {\n                value: self\n                    .acls\n                    .iter()\n                    .map(AclGrant::from)\n                    .collect::<Vec<_>>()\n                    .into(),\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for Mailbox {\n    fn is_versioned() -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/email/src/mailbox/manage.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::*;\nuse crate::cache::MessageCacheFetch;\nuse common::{Server, storage::index::ObjectIndexBuilder};\nuse std::future::Future;\nuse store::write::BatchBuilder;\nuse trc::AddContext;\nuse types::collection::Collection;\n\npub trait MailboxFnc: Sync + Send {\n    fn create_system_folders(\n        &self,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n\n    fn mailbox_create_path(\n        &self,\n        account_id: u32,\n        path: &str,\n    ) -> impl Future<Output = trc::Result<Option<u32>>> + Send;\n}\n\nimpl MailboxFnc for Server {\n    async fn create_system_folders(&self, account_id: u32) -> trc::Result<()> {\n        #[cfg(feature = \"test_mode\")]\n        if account_id == 0 {\n            return Ok(());\n        }\n\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Mailbox);\n\n        // Create mailboxes\n        let mut last_document_id = ARCHIVE_ID;\n        for folder in &self.core.jmap.default_folders {\n            let document_id = match folder.special_use {\n                SpecialUse::Inbox => INBOX_ID,\n                SpecialUse::Trash => TRASH_ID,\n                SpecialUse::Junk => JUNK_ID,\n                SpecialUse::Drafts => DRAFTS_ID,\n                SpecialUse::Sent => SENT_ID,\n                SpecialUse::Archive => ARCHIVE_ID,\n                SpecialUse::None\n                | SpecialUse::Important\n                | SpecialUse::Memos\n                | SpecialUse::Scheduled\n                | SpecialUse::Snoozed => {\n                    last_document_id += 1;\n                    last_document_id\n                }\n                SpecialUse::Shared => unreachable!(),\n            };\n\n            let mut object = Mailbox::new(folder.name.clone()).with_role(folder.special_use);\n            if folder.subscribe {\n                object.add_subscriber(account_id);\n            }\n            batch\n                .with_document(document_id)\n                .custom(ObjectIndexBuilder::<(), _>::new().with_changes(object))\n                .caused_by(trc::location!())?;\n        }\n        self.store()\n            .assign_document_ids(account_id, Collection::Mailbox, (ARCHIVE_ID + 1) as u64)\n            .await\n            .caused_by(trc::location!())?;\n\n        self.core\n            .storage\n            .data\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(())\n    }\n\n    async fn mailbox_create_path(&self, account_id: u32, path: &str) -> trc::Result<Option<u32>> {\n        let cache = self\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        let mut next_parent_id = 0;\n        let mut create_paths = Vec::with_capacity(2);\n\n        let mut path = path.split('/').map(|v| v.trim());\n        let mut found_path = String::with_capacity(16);\n        {\n            while let Some(name) = path.next() {\n                if !found_path.is_empty() {\n                    found_path.push('/');\n                }\n\n                for ch in name.chars() {\n                    for ch in ch.to_lowercase() {\n                        found_path.push(ch);\n                    }\n                }\n\n                if let Some(item) = cache\n                    .mailboxes\n                    .items\n                    .iter()\n                    .find(|item| item.path.to_lowercase() == found_path)\n                {\n                    next_parent_id = item.document_id + 1;\n                } else {\n                    create_paths.push(name.to_string());\n                    create_paths.extend(path.map(|v| v.to_string()));\n                    break;\n                }\n            }\n        }\n\n        // Create missing folders\n        if !create_paths.is_empty() {\n            if create_paths\n                .iter()\n                .any(|name| name.len() > self.core.jmap.mailbox_name_max_len)\n            {\n                return Ok(None);\n            }\n\n            let mut next_document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::Mailbox, create_paths.len() as u64)\n                .await\n                .caused_by(trc::location!())?;\n            let mut batch = BatchBuilder::new();\n            for name in create_paths {\n                let document_id = next_document_id;\n                next_document_id -= 1;\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Mailbox)\n                    .with_document(document_id)\n                    .custom(\n                        ObjectIndexBuilder::<(), _>::new()\n                            .with_changes(Mailbox::new(name).with_parent_id(next_parent_id)),\n                    )\n                    .caused_by(trc::location!())?;\n                next_parent_id = document_id + 1;\n            }\n\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n        }\n\n        Ok(Some(next_parent_id - 1))\n    }\n}\n"
  },
  {
    "path": "crates/email/src/mailbox/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse types::{acl::AclGrant, special_use::SpecialUse};\n\npub mod destroy;\npub mod index;\npub mod manage;\n\npub const INBOX_ID: u32 = 0;\npub const TRASH_ID: u32 = 1;\npub const JUNK_ID: u32 = 2;\npub const DRAFTS_ID: u32 = 3;\npub const SENT_ID: u32 = 4;\npub const ARCHIVE_ID: u32 = 5;\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\n#[rkyv(derive(Debug))]\npub struct Mailbox {\n    pub name: String,\n    pub role: SpecialUse,\n    pub parent_id: u32,\n    pub sort_order: Option<u32>,\n    pub uid_validity: u32,\n    pub subscribers: Vec<u32>,\n    pub acls: Vec<AclGrant>,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, Copy)]\n#[rkyv(derive(Debug, Clone, Copy))]\npub struct UidMailbox {\n    pub mailbox_id: u32,\n    pub uid: u32,\n}\n\nimpl Mailbox {\n    pub fn new(name: impl Into<String>) -> Self {\n        Mailbox {\n            name: name.into(),\n            role: SpecialUse::None,\n            parent_id: 0,\n            sort_order: None,\n            uid_validity: rand::random::<u32>(),\n            subscribers: vec![],\n            acls: vec![],\n        }\n    }\n\n    pub fn with_role(mut self, role: SpecialUse) -> Self {\n        self.role = role;\n        self\n    }\n\n    pub fn with_parent_id(mut self, parent_id: u32) -> Self {\n        self.parent_id = parent_id;\n        self\n    }\n\n    pub fn with_sort_order(mut self, sort_order: u32) -> Self {\n        self.sort_order = Some(sort_order);\n        self\n    }\n\n    pub fn with_subscriber(mut self, subscriber: u32) -> Self {\n        self.subscribers.push(subscriber);\n        self\n    }\n\n    pub fn add_subscriber(&mut self, subscriber: u32) -> bool {\n        if !self.subscribers.contains(&subscriber) {\n            self.subscribers.push(subscriber);\n            true\n        } else {\n            false\n        }\n    }\n\n    pub fn remove_subscriber(&mut self, subscriber: u32) {\n        self.subscribers.retain(|&x| x != subscriber);\n    }\n\n    pub fn is_subscribed(&self, subscriber: u32) -> bool {\n        self.subscribers.contains(&subscriber)\n    }\n}\n\nimpl ArchivedMailbox {\n    pub fn is_subscribed(&self, subscriber: u32) -> bool {\n        self.subscribers.iter().any(|x| u32::from(x) == subscriber)\n    }\n}\n\nimpl PartialEq for UidMailbox {\n    fn eq(&self, other: &Self) -> bool {\n        self.mailbox_id == other.mailbox_id\n    }\n}\n\nimpl Eq for UidMailbox {}\n\nimpl UidMailbox {\n    pub fn new(mailbox_id: u32, uid: u32) -> Self {\n        UidMailbox { mailbox_id, uid }\n    }\n\n    pub fn new_unassigned(mailbox_id: u32) -> Self {\n        UidMailbox { mailbox_id, uid: 0 }\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/copy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ingest::{EmailIngest, IngestedEmail},\n    metadata::{MessageData, MessageMetadata},\n};\nuse crate::{\n    mailbox::UidMailbox,\n    message::{\n        index::extractors::VisitTextArchived,\n        ingest::{MergeThreadIds, ThreadInfo},\n        metadata::{\n            MESSAGE_HAS_ATTACHMENT, MESSAGE_RECEIVED_MASK, MetadataHeaderName, MetadataHeaderValue,\n        },\n    },\n};\nuse common::{Server, auth::ResourceToken, storage::index::ObjectIndexBuilder};\nuse mail_parser::parsers::fields::thread::thread_name;\nuse store::write::{\n    BatchBuilder, IndexPropertyClass, SearchIndex, TaskEpoch, TaskQueueClass, ValueClass,\n};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    blob::{BlobClass, BlobId},\n    collection::{Collection, SyncCollection},\n    field::EmailField,\n    keyword::Keyword,\n};\nuse utils::cheeky_hash::CheekyHash;\n\npub enum CopyMessageError {\n    NotFound,\n    OverQuota,\n}\n\npub trait EmailCopy: Sync + Send {\n    #[allow(clippy::too_many_arguments)]\n    fn copy_message(\n        &self,\n        from_account_id: u32,\n        from_message_id: u32,\n        resource_token: &ResourceToken,\n        mailboxes: Vec<u32>,\n        keywords: Vec<Keyword>,\n        received_at: Option<u64>,\n        session_id: u64,\n    ) -> impl Future<Output = trc::Result<Result<IngestedEmail, CopyMessageError>>> + Send;\n}\n\nimpl EmailCopy for Server {\n    #[allow(clippy::too_many_arguments)]\n    async fn copy_message(\n        &self,\n        from_account_id: u32,\n        from_message_id: u32,\n        resource_token: &ResourceToken,\n        mailboxes: Vec<u32>,\n        keywords: Vec<Keyword>,\n        received_at: Option<u64>,\n        session_id: u64,\n    ) -> trc::Result<Result<IngestedEmail, CopyMessageError>> {\n        // Obtain metadata\n        let account_id = resource_token.account_id;\n        let mut metadata = if let Some(metadata) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                from_account_id,\n                Collection::Email,\n                from_message_id,\n                EmailField::Metadata,\n            ))\n            .await?\n        {\n            metadata\n                .deserialize::<MessageMetadata>()\n                .caused_by(trc::location!())?\n        } else {\n            return Ok(Err(CopyMessageError::NotFound));\n        };\n\n        // Check quota\n        let size = metadata.root_part().offset_end;\n        match self.has_available_quota(resource_token, size as u64).await {\n            Ok(_) => (),\n            Err(err) => {\n                if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota))\n                    || err.matches(trc::EventType::Limit(trc::LimitEvent::TenantQuota))\n                {\n                    trc::error!(err.account_id(account_id).span_id(session_id));\n                    return Ok(Err(CopyMessageError::OverQuota));\n                } else {\n                    return Err(err);\n                }\n            }\n        }\n\n        // Set receivedAt\n        if let Some(received_at) = received_at {\n            metadata.rcvd_attach = (metadata.rcvd_attach & MESSAGE_HAS_ATTACHMENT)\n                | (received_at & MESSAGE_RECEIVED_MASK);\n        }\n\n        // Obtain threadId\n        let mut message_ids = Vec::new();\n        let mut subject = \"\";\n        for header in &metadata.contents[0].parts[0].headers {\n            match &header.name {\n                MetadataHeaderName::MessageId => {\n                    header.value.visit_text(|id| {\n                        if !id.is_empty() {\n                            message_ids.push(CheekyHash::new(id.as_bytes()));\n                        }\n                    });\n                }\n                MetadataHeaderName::InReplyTo\n                | MetadataHeaderName::References\n                | MetadataHeaderName::ResentMessageId => {\n                    header.value.visit_text(|id| {\n                        if !id.is_empty() {\n                            message_ids.push(CheekyHash::new(id.as_bytes()));\n                        }\n                    });\n                }\n                MetadataHeaderName::Subject if subject.is_empty() => {\n                    subject = thread_name(match &header.value {\n                        MetadataHeaderValue::Text(text) => text.as_ref(),\n                        MetadataHeaderValue::TextList(list) if !list.is_empty() => {\n                            list.first().unwrap().as_ref()\n                        }\n                        _ => \"\",\n                    });\n                }\n                _ => (),\n            }\n        }\n\n        // Obtain threadId\n        let thread_result = self\n            .find_thread_id(account_id, subject, &message_ids)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Assign id\n        let mut email = IngestedEmail {\n            size: size as usize,\n            ..Default::default()\n        };\n        let blob_hash = metadata.blob_hash.clone();\n\n        // Assign IMAP UIDs\n        let mut mailbox_ids = Vec::with_capacity(mailboxes.len());\n        email.imap_uids = Vec::with_capacity(mailboxes.len());\n        let mut ids = self\n            .assign_email_ids(account_id, mailboxes.iter().copied(), true)\n            .await\n            .caused_by(trc::location!())?;\n        let document_id = ids.next().unwrap();\n        for (uid, mailbox_id) in ids.zip(mailboxes.iter().copied()) {\n            mailbox_ids.push(UidMailbox::new(mailbox_id, uid));\n            email.imap_uids.push(uid);\n        }\n\n        // Prepare batch\n        let mut batch = BatchBuilder::new();\n        batch.with_account_id(account_id);\n\n        // Determine thread id\n        let thread_id = if let Some(thread_id) = thread_result.thread_id {\n            thread_id\n        } else {\n            batch\n                .with_collection(Collection::Thread)\n                .with_document(document_id)\n                .log_container_insert(SyncCollection::Thread);\n            document_id\n        };\n        batch\n            .with_collection(Collection::Email)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_tenant_id(resource_token.tenant.map(|t| t.id))\n                    .with_changes(MessageData {\n                        mailboxes: mailbox_ids.into_boxed_slice(),\n                        keywords: keywords.into_boxed_slice(),\n                        thread_id,\n                        size,\n                    }),\n            )\n            .caused_by(trc::location!())?\n            .set(\n                ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                    property: EmailField::Threading.into(),\n                    hash: thread_result.thread_hash,\n                }),\n                ThreadInfo::serialize(thread_id, &message_ids),\n            )\n            .set(\n                ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                    index: SearchIndex::Email,\n                    due: TaskEpoch::now(),\n                    is_insert: true,\n                }),\n                vec![],\n            );\n\n        // Merge threads if necessary\n        if let Some(merge_threads) = MergeThreadIds::new(thread_result).serialize() {\n            batch.set(\n                ValueClass::TaskQueue(TaskQueueClass::MergeThreads {\n                    due: TaskEpoch::now(),\n                }),\n                merge_threads,\n            );\n        }\n\n        metadata\n            .index(&mut batch, true)\n            .caused_by(trc::location!())?;\n\n        // Insert and obtain ids\n        let change_id = self\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?\n            .last_change_id(account_id)?;\n\n        // Request indexing\n        self.notify_task_queue();\n\n        // Update response\n        email.document_id = document_id;\n        email.thread_id = thread_id;\n        email.change_id = change_id;\n        email.blob_id = BlobId::new(\n            blob_hash,\n            BlobClass::Linked {\n                account_id,\n                collection: Collection::Email.into(),\n                document_id,\n            },\n        );\n\n        Ok(Ok(email))\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/crypto.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, collections::BTreeSet, fmt::Display, io::Cursor};\n\nuse aes::cipher::{BlockEncryptMut, KeyIvInit, block_padding::Pkcs7};\n\nuse mail_builder::{encoders::base64::base64_encode_mime, mime::make_boundary};\nuse mail_parser::{Message, MimeHeaders, PartType, decoders::base64::base64_decode};\nuse openpgp::{\n    parse::Parse,\n    serialize::stream,\n    types::{KeyFlags, SymmetricAlgorithm},\n};\nuse rand::{RngCore, SeedableRng, rngs::StdRng};\nuse rasn::types::{ObjectIdentifier, OctetString};\nuse rasn_cms::{\n    AlgorithmIdentifier, CONTENT_DATA, CONTENT_ENVELOPED_DATA, EncryptedContent,\n    EncryptedContentInfo, EncryptedKey, EnvelopedData, IssuerAndSerialNumber,\n    KeyTransRecipientInfo, RecipientIdentifier, RecipientInfo,\n    algorithms::{AES128_CBC, AES256_CBC, RSA},\n    pkcs7_compat::EncapsulatedContentInfo,\n};\nuse rsa::{Pkcs1v15Encrypt, RsaPublicKey, pkcs1::DecodeRsaPublicKey};\nuse sequoia_openpgp as openpgp;\nuse store::{Deserialize, write::Archive};\n\nconst P: openpgp::policy::StandardPolicy<'static> = openpgp::policy::StandardPolicy::new();\n\n#[derive(Debug)]\npub enum EncryptMessageError {\n    AlreadyEncrypted,\n    Error(String),\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    Copy,\n    serde::Serialize,\n    serde::Deserialize,\n)]\n#[rkyv(derive(Clone, Copy))]\npub enum Algorithm {\n    Aes128,\n    Aes256,\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    Copy,\n    PartialEq,\n    Eq,\n    serde::Serialize,\n    serde::Deserialize,\n)]\npub enum EncryptionMethod {\n    PGP,\n    SMIME,\n}\n\n#[derive(\n    Clone,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    serde::Serialize,\n    serde::Deserialize,\n)]\npub struct EncryptionParams {\n    pub certs: Box<[Box<[u8]>]>,\n    pub flags: u64,\n}\n\npub const ENCRYPT_TRAIN_SPAM_FILTER: u64 = 1;\npub const ENCRYPT_METHOD_SMIME: u64 = 1 << 1;\npub const ENCRYPT_METHOD_PGP: u64 = 1 << 2;\npub const ENCRYPT_ALGO_AES256: u64 = 1 << 3;\npub const ENCRYPT_ALGO_AES128: u64 = 1 << 4;\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    serde::Serialize,\n    serde::Deserialize,\n    Default,\n)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum EncryptionType {\n    PGP {\n        algo: Algorithm,\n        certs: String,\n        allow_spam_training: bool,\n    },\n    SMIME {\n        algo: Algorithm,\n        certs: String,\n        allow_spam_training: bool,\n    },\n    #[default]\n    Disabled,\n}\n\n#[allow(async_fn_in_trait)]\npub trait EncryptMessage {\n    async fn encrypt(\n        &self,\n        params: &ArchivedEncryptionParams,\n    ) -> Result<Vec<u8>, EncryptMessageError>;\n    fn is_encrypted(&self) -> bool;\n}\n\nimpl EncryptMessage for Message<'_> {\n    async fn encrypt(\n        &self,\n        params: &ArchivedEncryptionParams,\n    ) -> Result<Vec<u8>, EncryptMessageError> {\n        let root = self.root_part();\n        let raw_message = self.raw_message();\n        let mut outer_message = Vec::with_capacity((raw_message.len() as f64 * 1.5) as usize);\n        let mut inner_message = Vec::with_capacity(raw_message.len());\n\n        // Move MIME headers and body to inner message\n        for header in root.headers() {\n            (if header.name.is_mime_header() {\n                &mut inner_message\n            } else {\n                &mut outer_message\n            })\n            .extend_from_slice(\n                &raw_message[header.offset_field() as usize..header.offset_end() as usize],\n            );\n        }\n        inner_message.extend_from_slice(b\"\\r\\n\");\n        inner_message.extend_from_slice(&raw_message[root.raw_body_offset() as usize..]);\n\n        // Encrypt inner message\n        match params.method() {\n            EncryptionMethod::PGP => {\n                // Prepare encrypted message\n                let boundary = make_boundary(\"_\");\n                outer_message.extend_from_slice(\n                    concat!(\n                        \"Content-Type: multipart/encrypted;\\r\\n\\t\",\n                        \"protocol=\\\"application/pgp-encrypted\\\";\\r\\n\\t\",\n                        \"boundary=\\\"\"\n                    )\n                    .as_bytes(),\n                );\n                outer_message.extend_from_slice(boundary.as_bytes());\n                outer_message.extend_from_slice(\n                    concat!(\n                        \"\\\"\\r\\n\\r\\n\",\n                        \"OpenPGP/MIME message (Automatically encrypted by Stalwart)\\r\\n\\r\\n\",\n                        \"--\"\n                    )\n                    .as_bytes(),\n                );\n                outer_message.extend_from_slice(boundary.as_bytes());\n                outer_message.extend_from_slice(\n                    concat!(\n                        \"\\r\\nContent-Type: application/pgp-encrypted\\r\\n\\r\\n\",\n                        \"Version: 1\\r\\n\\r\\n--\"\n                    )\n                    .as_bytes(),\n                );\n                outer_message.extend_from_slice(boundary.as_bytes());\n                outer_message.extend_from_slice(\n                    concat!(\n                        \"\\r\\nContent-Type: application/octet-stream; name=\\\"encrypted.asc\\\"\\r\\n\",\n                        \"Content-Disposition: inline; filename=\\\"encrypted.asc\\\"\\r\\n\\r\\n\"\n                    )\n                    .as_bytes(),\n                );\n\n                let certs = params\n                    .certs\n                    .iter()\n                    .map(openpgp::Cert::from_bytes)\n                    .collect::<Result<Vec<_>, _>>()\n                    .map_err(|err| {\n                        EncryptMessageError::Error(format!(\n                            \"Failed to parse OpenPGP public key: {}\",\n                            err\n                        ))\n                    })?;\n\n                // Encrypt contents (TODO: use rayon)\n                let algo = params.algo();\n                let encrypted_contents = tokio::task::spawn_blocking(move || {\n                    // Parse public key\n                    let mut keys = Vec::with_capacity(certs.len());\n                    let policy = openpgp::policy::StandardPolicy::new();\n\n                    for cert in &certs {\n                        for key in cert\n                            .keys()\n                            .with_policy(&policy, None)\n                            .supported()\n                            .alive()\n                            .revoked(false)\n                            .key_flags(KeyFlags::empty().set_transport_encryption())\n                        {\n                            keys.push(key);\n                        }\n                    }\n\n                    // Compose a writer stack corresponding to the output format and\n                    // packet structure we want.\n                    let mut sink = Vec::with_capacity(inner_message.len());\n\n                    // Stream an OpenPGP message.\n                    let message = stream::Armorer::new(stream::Message::new(&mut sink))\n                        .build()\n                        .map_err(|err| {\n                            EncryptMessageError::Error(format!(\"Failed to create armorer: {}\", err))\n                        })?;\n                    let message = stream::Encryptor::for_recipients(message, keys)\n                        .symmetric_algo(match algo {\n                            Algorithm::Aes128 => SymmetricAlgorithm::AES128,\n                            Algorithm::Aes256 => SymmetricAlgorithm::AES256,\n                        })\n                        .build()\n                        .map_err(|err| {\n                            EncryptMessageError::Error(format!(\n                                \"Failed to build encryptor: {}\",\n                                err\n                            ))\n                        })?;\n                    let mut message =\n                        stream::LiteralWriter::new(message).build().map_err(|err| {\n                            EncryptMessageError::Error(format!(\n                                \"Failed to create literal writer: {}\",\n                                err\n                            ))\n                        })?;\n                    std::io::copy(&mut Cursor::new(inner_message), &mut message).map_err(\n                        |err| {\n                            EncryptMessageError::Error(format!(\n                                \"Failed to encrypt message: {}\",\n                                err\n                            ))\n                        },\n                    )?;\n                    message.finalize().map_err(|err| {\n                        EncryptMessageError::Error(format!(\"Failed to finalize message: {}\", err))\n                    })?;\n\n                    String::from_utf8(sink).map_err(|err| {\n                        EncryptMessageError::Error(format!(\n                            \"Failed to convert encrypted message to UTF-8: {}\",\n                            err\n                        ))\n                    })\n                })\n                .await\n                .map_err(|err| {\n                    EncryptMessageError::Error(format!(\"Failed to encrypt message: {}\", err))\n                })??;\n                outer_message.extend_from_slice(encrypted_contents.as_bytes());\n                outer_message.extend_from_slice(b\"\\r\\n--\");\n                outer_message.extend_from_slice(boundary.as_bytes());\n                outer_message.extend_from_slice(b\"--\\r\\n\");\n            }\n            EncryptionMethod::SMIME => {\n                // Generate random IV\n                let mut rng = StdRng::from_entropy();\n                let mut iv = vec![0u8; 16];\n                rng.fill_bytes(&mut iv);\n\n                // Generate random key\n                let mut key = vec![0u8; params.key_size()];\n                rng.fill_bytes(&mut key);\n\n                // Encrypt contents (TODO: use rayon)\n                let algo = params.algo();\n                let (encrypted_contents, key, iv) = tokio::task::spawn_blocking(move || {\n                    (algo.encrypt(&key, &iv, &inner_message), key, iv)\n                })\n                .await\n                .map_err(|err| {\n                    EncryptMessageError::Error(format!(\"Failed to encrypt message: {}\", err))\n                })?;\n\n                // Encrypt key using public keys\n                #[allow(clippy::mutable_key_type)]\n                let mut recipient_infos = BTreeSet::new();\n                for cert in params.certs.iter() {\n                    let cert =\n                        rasn::der::decode::<rasn_pkix::Certificate>(cert).map_err(|err| {\n                            EncryptMessageError::Error(format!(\n                                \"Failed to parse certificate: {}\",\n                                err\n                            ))\n                        })?;\n\n                    let public_key = RsaPublicKey::from_pkcs1_der(\n                        cert.tbs_certificate\n                            .subject_public_key_info\n                            .subject_public_key\n                            .as_raw_slice(),\n                    )\n                    .map_err(|err| {\n                        EncryptMessageError::Error(format!(\"Failed to parse public key: {}\", err))\n                    })?;\n                    let encrypted_key = public_key\n                        .encrypt(&mut rng, Pkcs1v15Encrypt, &key[..])\n                        .map_err(|err| {\n                            EncryptMessageError::Error(format!(\"Failed to encrypt key: {}\", err))\n                        })\n                        .unwrap();\n\n                    recipient_infos.insert(RecipientInfo::KeyTransRecipientInfo(\n                        KeyTransRecipientInfo {\n                            version: 0.into(),\n                            rid: RecipientIdentifier::IssuerAndSerialNumber(\n                                IssuerAndSerialNumber {\n                                    issuer: cert.tbs_certificate.issuer,\n                                    serial_number: cert.tbs_certificate.serial_number,\n                                },\n                            ),\n                            key_encryption_algorithm: AlgorithmIdentifier {\n                                algorithm: RSA.into(),\n                                parameters: Some(\n                                    rasn::der::encode(&())\n                                        .map_err(|err| {\n                                            EncryptMessageError::Error(format!(\n                                                \"Failed to encode RSA algorithm identifier: {}\",\n                                                err\n                                            ))\n                                        })?\n                                        .into(),\n                                ),\n                            },\n                            encrypted_key: EncryptedKey::from(encrypted_key),\n                        },\n                    ));\n                }\n\n                let pkcs7 = rasn::der::encode(&EncapsulatedContentInfo {\n                    content_type: CONTENT_ENVELOPED_DATA.into(),\n                    content: Some(\n                        rasn::der::encode(&EnvelopedData {\n                            version: 0.into(),\n                            originator_info: None,\n                            recipient_infos,\n                            encrypted_content_info: EncryptedContentInfo {\n                                content_type: CONTENT_DATA.into(),\n                                content_encryption_algorithm: AlgorithmIdentifier {\n                                    algorithm: params.to_algorithm_identifier(),\n                                    parameters: Some(\n                                        rasn::der::encode(&OctetString::from(iv))\n                                            .map_err(|err| {\n                                                EncryptMessageError::Error(format!(\n                                                    \"Failed to encode IV: {}\",\n                                                    err\n                                                ))\n                                            })?\n                                            .into(),\n                                    ),\n                                },\n                                encrypted_content: Some(EncryptedContent::from(encrypted_contents)),\n                            },\n                            unprotected_attrs: None,\n                        })\n                        .map_err(|err| {\n                            EncryptMessageError::Error(format!(\n                                \"Failed to encode EnvelopedData: {}\",\n                                err\n                            ))\n                        })?\n                        .into(),\n                    ),\n                })\n                .map_err(|err| {\n                    EncryptMessageError::Error(format!(\"Failed to encode ContentInfo: {}\", err))\n                })?;\n\n                // Generate message\n                outer_message.extend_from_slice(\n                    concat!(\n                        \"Content-Type: application/pkcs7-mime;\\r\\n\",\n                        \"\\tname=\\\"smime.p7m\\\";\\r\\n\",\n                        \"\\tsmime-type=enveloped-data\\r\\n\",\n                        \"Content-Disposition: attachment;\\r\\n\",\n                        \"\\tfilename=\\\"smime.p7m\\\"\\r\\n\",\n                        \"Content-Transfer-Encoding: base64\\r\\n\\r\\n\"\n                    )\n                    .as_bytes(),\n                );\n                base64_encode_mime(&pkcs7, &mut outer_message, false).map_err(|err| {\n                    EncryptMessageError::Error(format!(\"Failed to base64 encode PKCS7: {}\", err))\n                })?;\n            }\n        }\n\n        Ok(outer_message)\n    }\n\n    fn is_encrypted(&self) -> bool {\n        if self.content_type().is_some_and(|ct| {\n            let main_type = ct.c_type.as_ref();\n            let sub_type = ct\n                .c_subtype\n                .as_ref()\n                .map(|s| s.as_ref())\n                .unwrap_or_default();\n\n            (main_type.eq_ignore_ascii_case(\"application\")\n                && (sub_type.eq_ignore_ascii_case(\"pkcs7-mime\")\n                    || sub_type.eq_ignore_ascii_case(\"pkcs7-signature\")\n                    || (sub_type.eq_ignore_ascii_case(\"octet-stream\")\n                        && self.attachment_name().is_some_and(|name| {\n                            name.rsplit_once('.')\n                                .is_some_and(|(_, ext)| [\"p7m\", \"p7s\", \"p7c\", \"p7z\"].contains(&ext))\n                        }))))\n                || (main_type.eq_ignore_ascii_case(\"multipart\")\n                    && sub_type.eq_ignore_ascii_case(\"encrypted\"))\n        }) {\n            return true;\n        }\n\n        if self.parts.len() <= 2 {\n            let mut text_part = None;\n            let mut is_multipart = false;\n\n            for part in &self.parts {\n                match &part.body {\n                    PartType::Text(text) => {\n                        text_part = Some(text.as_ref());\n                    }\n                    PartType::Multipart(_) => {\n                        is_multipart = true;\n                    }\n                    _ => (),\n                }\n            }\n\n            match text_part {\n                Some(text) if self.parts.len() == 1 || is_multipart => {\n                    if text.trim_start().starts_with(\"-----BEGIN PGP MESSAGE-----\") {\n                        return true;\n                    }\n                }\n                _ => (),\n            }\n        }\n\n        false\n    }\n}\n\nimpl ArchivedEncryptionParams {\n    pub fn method(&self) -> EncryptionMethod {\n        if self.flags & ENCRYPT_METHOD_PGP != 0 {\n            EncryptionMethod::PGP\n        } else {\n            EncryptionMethod::SMIME\n        }\n    }\n\n    pub fn algo(&self) -> Algorithm {\n        if self.flags & ENCRYPT_ALGO_AES256 != 0 {\n            Algorithm::Aes256\n        } else {\n            Algorithm::Aes128\n        }\n    }\n\n    fn key_size(&self) -> usize {\n        if self.flags & ENCRYPT_ALGO_AES256 != 0 {\n            32\n        } else {\n            16\n        }\n    }\n\n    fn to_algorithm_identifier(&self) -> ObjectIdentifier {\n        if self.flags & ENCRYPT_ALGO_AES256 != 0 {\n            AES256_CBC.into()\n        } else {\n            AES128_CBC.into()\n        }\n    }\n\n    pub fn can_train_spam_filter(&self) -> bool {\n        self.flags & ENCRYPT_TRAIN_SPAM_FILTER != 0\n    }\n}\n\nimpl Algorithm {\n    fn encrypt(&self, key: &[u8], iv: &[u8], contents: &[u8]) -> Vec<u8> {\n        match self {\n            Algorithm::Aes128 => cbc::Encryptor::<aes::Aes128>::new(key.into(), iv.into())\n                .encrypt_padded_vec_mut::<Pkcs7>(contents),\n            Algorithm::Aes256 => cbc::Encryptor::<aes::Aes256>::new(key.into(), iv.into())\n                .encrypt_padded_vec_mut::<Pkcs7>(contents),\n        }\n    }\n}\n\n#[allow(clippy::type_complexity)]\npub fn try_parse_certs(\n    expected_method: EncryptionMethod,\n    cert: Vec<u8>,\n) -> Result<Box<[Box<[u8]>]>, Cow<'static, str>> {\n    // Check if it's a PEM file\n    let (flags, certs) = if let Some(result) = try_parse_pem(&cert)? {\n        (result.flags, result.certs)\n    } else if rasn::der::decode::<rasn_pkix::Certificate>(&cert[..]).is_ok() {\n        (\n            ENCRYPT_METHOD_SMIME,\n            Box::from_iter([cert.into_boxed_slice()]),\n        )\n    } else if let Ok(cert_) = openpgp::Cert::from_bytes(&cert[..]) {\n        if !has_pgp_keys(cert_) {\n            (\n                ENCRYPT_METHOD_PGP,\n                Box::from_iter([cert.into_boxed_slice()]),\n            )\n        } else {\n            return Err(\"Could not find any suitable keys in certificate\".into());\n        }\n    } else {\n        return Err(\"Could not find any valid certificates\".into());\n    };\n\n    if expected_method.flags() & flags != 0 {\n        Ok(certs)\n    } else {\n        Err(\"No valid certificates found for the selected encryption\".into())\n    }\n}\n\nfn has_pgp_keys(cert: openpgp::Cert) -> bool {\n    cert.keys()\n        .with_policy(&P, None)\n        .supported()\n        .alive()\n        .revoked(false)\n        .key_flags(KeyFlags::empty().set_transport_encryption())\n        .next()\n        .is_some()\n}\n\n#[allow(clippy::type_complexity)]\nfn try_parse_pem(bytes_: &[u8]) -> Result<Option<EncryptionParams>, Cow<'static, str>> {\n    if let Some(internal) = std::str::from_utf8(bytes_)\n        .ok()\n        .and_then(|cert| cert.strip_prefix(\"-----STALWART CERTIFICATE-----\"))\n    {\n        return base64_decode(internal.as_bytes())\n            .ok_or(Cow::from(\"Failed to decode base64\"))\n            .and_then(|bytes| {\n                Archive::deserialize_owned(bytes)\n                    .and_then(|arch| arch.deserialize::<EncryptionParams>())\n                    .map_err(|_| Cow::from(\"Failed to deserialize internal certificate\"))\n            })\n            .map(Some);\n    }\n\n    let mut bytes = bytes_.iter().enumerate();\n    let mut buf = vec![];\n    let mut method = None;\n    let mut certs: Vec<Box<[u8]>> = vec![];\n\n    loop {\n        // Find start of PEM block\n        let mut start_pos = 0;\n        for (pos, &ch) in bytes.by_ref() {\n            if ch.is_ascii_whitespace() {\n                continue;\n            } else if ch == b'-' {\n                start_pos = pos;\n                break;\n            } else {\n                return Ok(None);\n            }\n        }\n\n        // Find block type\n        for (_, &ch) in bytes.by_ref() {\n            match ch {\n                b'-' => (),\n                b'\\n' => break,\n                _ => {\n                    if ch.is_ascii() {\n                        buf.push(ch.to_ascii_uppercase());\n                    } else {\n                        return Ok(None);\n                    }\n                }\n            }\n        }\n        if buf.is_empty() {\n            break;\n        }\n\n        // Find type\n        let tag = std::str::from_utf8(&buf).unwrap();\n        if tag.contains(\"CERTIFICATE\") {\n            if method.is_some_and(|m| m == EncryptionMethod::PGP) {\n                return Err(\"Cannot mix OpenPGP and S/MIME certificates\".into());\n            } else {\n                method = Some(EncryptionMethod::SMIME);\n            }\n        } else if tag.contains(\"PGP\") {\n            if method.is_some_and(|m| m == EncryptionMethod::SMIME) {\n                return Err(\"Cannot mix OpenPGP and S/MIME certificates\".into());\n            } else {\n                method = Some(EncryptionMethod::PGP);\n            }\n        } else {\n            // Ignore block\n            let mut found_end = false;\n            for (_, &ch) in bytes.by_ref() {\n                if ch == b'-' {\n                    found_end = true;\n                } else if ch == b'\\n' && found_end {\n                    break;\n                }\n            }\n            buf.clear();\n            continue;\n        }\n\n        // Collect base64\n        buf.clear();\n        let mut found_end = false;\n        let mut end_pos = 0;\n        for (pos, &ch) in bytes.by_ref() {\n            match ch {\n                b'-' => {\n                    found_end = true;\n                }\n                b'\\n' => {\n                    if found_end {\n                        end_pos = pos;\n                        break;\n                    }\n                }\n                _ => {\n                    if !ch.is_ascii_whitespace() {\n                        buf.push(ch);\n                    }\n                }\n            }\n        }\n\n        // Decode base64\n        let cert = base64_decode(&buf)\n            .ok_or_else(|| Cow::from(\"Failed to decode base64 certificate.\"))?\n            .into_boxed_slice();\n        match method.unwrap() {\n            EncryptionMethod::PGP => match openpgp::Cert::from_bytes(bytes_) {\n                Ok(cert) => {\n                    if !has_pgp_keys(cert) {\n                        return Err(\"Could not find any suitable keys in OpenPGP public key\".into());\n                    }\n                    certs.push(\n                        bytes_\n                            .get(start_pos..end_pos + 1)\n                            .unwrap_or_default()\n                            .into(),\n                    );\n                }\n                Err(err) => {\n                    return Err(format!(\"Failed to decode OpenPGP public key: {err}\").into());\n                }\n            },\n            EncryptionMethod::SMIME => {\n                if let Err(err) = rasn::der::decode::<rasn_pkix::Certificate>(&cert) {\n                    return Err(format!(\"Failed to decode X509 certificate: {err}\").into());\n                }\n                certs.push(cert);\n            }\n        }\n        buf.clear();\n    }\n\n    Ok(method.map(|method| EncryptionParams {\n        flags: method.flags(),\n        certs: certs.into_boxed_slice(),\n    }))\n}\n\nimpl EncryptionMethod {\n    pub fn flags(&self) -> u64 {\n        match self {\n            EncryptionMethod::PGP => ENCRYPT_METHOD_PGP,\n            EncryptionMethod::SMIME => ENCRYPT_METHOD_SMIME,\n        }\n    }\n}\n\nimpl Algorithm {\n    pub fn flags(&self) -> u64 {\n        match self {\n            Algorithm::Aes128 => ENCRYPT_ALGO_AES128,\n            Algorithm::Aes256 => ENCRYPT_ALGO_AES256,\n        }\n    }\n}\n\nimpl Display for EncryptionMethod {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            EncryptionMethod::PGP => write!(f, \"OpenPGP\"),\n            EncryptionMethod::SMIME => write!(f, \"S/MIME\"),\n        }\n    }\n}\n\nimpl Display for Algorithm {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Algorithm::Aes128 => write!(f, \"AES-128\"),\n            Algorithm::Aes256 => write!(f, \"AES-256\"),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::metadata::MessageData;\nuse common::{KV_LOCK_PURGE_ACCOUNT, Server, storage::index::ObjectIndexBuilder};\nuse directory::backend::internal::manage::ManageDirectory;\nuse groupware::calendar::storage::ItipAutoExpunge;\nuse std::future::Future;\nuse store::rand::prelude::SliceRandom;\nuse store::write::key::DeserializeBigEndian;\nuse store::write::{IndexPropertyClass, SearchIndex, TaskEpoch, TaskQueueClass, now};\nuse store::{IterateParams, SerializeInfallible, U32_LEN, U64_LEN, ValueKey};\nuse store::{\n    roaring::RoaringBitmap,\n    write::{BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::collection::{Collection, VanishedCollection};\nuse types::field::{EmailField, EmailSubmissionField};\n\npub trait EmailDeletion: Sync + Send {\n    fn emails_delete(\n        &self,\n        account_id: u32,\n        tenant_id: Option<u32>,\n        batch: &mut BatchBuilder,\n        document_ids: RoaringBitmap,\n    ) -> impl Future<Output = trc::Result<RoaringBitmap>> + Send;\n\n    fn purge_accounts(&self, use_roles: bool) -> impl Future<Output = ()> + Send;\n\n    fn purge_account(&self, account_id: u32) -> impl Future<Output = ()> + Send;\n\n    fn purge_email_submissions(\n        &self,\n        account_id: u32,\n        hold_period: u64,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n\n    fn emails_auto_expunge(\n        &self,\n        account_id: u32,\n        hold_period: u64,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n}\n\nimpl EmailDeletion for Server {\n    async fn emails_delete(\n        &self,\n        account_id: u32,\n        tenant_id: Option<u32>,\n        batch: &mut BatchBuilder,\n        document_ids: RoaringBitmap,\n    ) -> trc::Result<RoaringBitmap> {\n        let mut deleted_ids = RoaringBitmap::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Email);\n        self.archives(\n            account_id,\n            Collection::Email,\n            &document_ids,\n            |document_id, data_| {\n                // Add changes to batch\n                let metadata = data_\n                    .to_unarchived::<MessageData>()\n                    .caused_by(trc::location!())?;\n                for mailbox in metadata.inner.mailboxes.iter() {\n                    batch.log_vanished_item(\n                        VanishedCollection::Email,\n                        (mailbox.mailbox_id.to_native(), mailbox.uid.to_native()),\n                    );\n                }\n                batch\n                    .with_document(document_id)\n                    .custom(\n                        ObjectIndexBuilder::<_, ()>::new()\n                            .with_tenant_id(tenant_id)\n                            .with_current(metadata),\n                    )\n                    .caused_by(trc::location!())?\n                    .set(\n                        ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                            index: SearchIndex::Email,\n                            due: TaskEpoch::now(),\n                            is_insert: false,\n                        }),\n                        0u64.serialize(),\n                    )\n                    .commit_point();\n\n                deleted_ids.insert(document_id);\n\n                Ok(true)\n            },\n        )\n        .await?;\n\n        let not_destroyed = if document_ids.len() == deleted_ids.len() {\n            RoaringBitmap::new()\n        } else {\n            deleted_ids ^= document_ids;\n            deleted_ids\n        };\n\n        Ok(not_destroyed)\n    }\n\n    async fn purge_accounts(&self, use_roles: bool) {\n        if let Ok(account_ids) = self.store().principal_ids(None, None).await {\n            let mut account_ids: Vec<u32> = account_ids\n                .into_iter()\n                .filter(|id| {\n                    !use_roles\n                        || self\n                            .core\n                            .network\n                            .roles\n                            .purge_accounts\n                            .is_enabled_for_integer(*id)\n                })\n                .collect();\n\n            // Shuffle account ids\n            account_ids.shuffle(&mut store::rand::rng());\n\n            for account_id in account_ids {\n                self.purge_account(account_id).await;\n            }\n        }\n    }\n\n    async fn purge_account(&self, account_id: u32) {\n        // Lock account\n        match self\n            .core\n            .storage\n            .lookup\n            .try_lock(KV_LOCK_PURGE_ACCOUNT, &account_id.to_be_bytes(), 3600)\n            .await\n        {\n            Ok(true) => (),\n            Ok(false) => {\n                trc::event!(Purge(trc::PurgeEvent::InProgress), AccountId = account_id,);\n                return;\n            }\n            Err(err) => {\n                trc::error!(\n                    err.details(\"Failed to lock account.\")\n                        .account_id(account_id)\n                );\n                return;\n            }\n        }\n\n        // Auto-expunge deleted and junk messages\n        if let Some(hold_period) = self.core.jmap.mail_autoexpunge_after\n            && let Err(err) = self.emails_auto_expunge(account_id, hold_period).await\n        {\n            trc::error!(\n                err.details(\"Failed to auto-expunge e-mail messages.\")\n                    .account_id(account_id)\n            );\n        }\n\n        // Auto-expunge iMIP messages\n        if let Some(hold_period) = self.core.groupware.itip_inbox_auto_expunge\n            && let Err(err) = self.itip_auto_expunge(account_id, hold_period).await\n        {\n            trc::error!(\n                err.details(\"Failed to auto-expunge iTIP messages.\")\n                    .account_id(account_id)\n            );\n        }\n\n        // Delete old e-mail submissions\n        if let Some(hold_period) = self.core.jmap.email_submission_autoexpunge_after\n            && let Err(err) = self.purge_email_submissions(account_id, hold_period).await\n        {\n            trc::error!(\n                err.details(\"Failed to auto-expunge e-mail submissions.\")\n                    .account_id(account_id)\n            );\n        }\n\n        // Purge changelogs\n        if let Err(err) = self\n            .delete_changes(\n                account_id,\n                self.core.jmap.changes_max_history,\n                self.core.jmap.share_notification_max_history,\n            )\n            .await\n        {\n            trc::error!(\n                err.details(\"Failed to purge changes.\")\n                    .account_id(account_id)\n            );\n        }\n\n        // Delete lock\n        if let Err(err) = self\n            .in_memory_store()\n            .remove_lock(KV_LOCK_PURGE_ACCOUNT, &account_id.to_be_bytes())\n            .await\n        {\n            trc::error!(err.details(\"Failed to delete lock.\").account_id(account_id));\n        }\n    }\n\n    async fn emails_auto_expunge(&self, account_id: u32, hold_period: u64) -> trc::Result<()> {\n        // Filter messages by received date\n        let mut destroy_ids = RoaringBitmap::new();\n        let cutoff = now().saturating_sub(hold_period);\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id: 0,\n                        class: ValueClass::Property(EmailField::DeletedAt.into()),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id: u32::MAX,\n                        class: ValueClass::Property(EmailField::DeletedAt.into()),\n                    },\n                )\n                .ascending(),\n                |key, value| {\n                    let deleted_at = value.deserialize_be_u64(0)?;\n                    if deleted_at <= cutoff {\n                        destroy_ids.insert(key.deserialize_be_u32(key.len() - U32_LEN)?);\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if destroy_ids.is_empty() {\n            return Ok(());\n        }\n\n        trc::event!(\n            Purge(trc::PurgeEvent::AutoExpunge),\n            Collection = Collection::Email.as_str(),\n            AccountId = account_id,\n            Total = destroy_ids.len(),\n        );\n\n        // Delete messages\n        let mut batch = BatchBuilder::new();\n        let tenant_id = self\n            .store()\n            .get_principal(account_id)\n            .await\n            .caused_by(trc::location!())?\n            .and_then(|p| p.tenant());\n        self.emails_delete(account_id, tenant_id, &mut batch, destroy_ids)\n            .await?;\n        self.commit_batch(batch).await?;\n        self.notify_task_queue();\n\n        Ok(())\n    }\n\n    async fn purge_email_submissions(&self, account_id: u32, hold_period: u64) -> trc::Result<()> {\n        // Filter messages by received date\n        let mut destroy_ids = Vec::new();\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::EmailSubmission.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: EmailSubmissionField::Metadata.into(),\n                            value: 0,\n                        }),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id: u32::MAX,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: EmailSubmissionField::Metadata.into(),\n                            value: now().saturating_sub(hold_period),\n                        }),\n                    },\n                )\n                .ascending()\n                .no_values(),\n                |key, _| {\n                    destroy_ids.push((\n                        key.deserialize_be_u32(key.len() - U32_LEN)?,\n                        key.deserialize_be_u64(key.len() - U32_LEN - U64_LEN)?,\n                    ));\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if destroy_ids.is_empty() {\n            return Ok(());\n        }\n\n        trc::event!(\n            Purge(trc::PurgeEvent::AutoExpunge),\n            Collection = Collection::EmailSubmission.as_str(),\n            AccountId = account_id,\n            Total = destroy_ids.len(),\n        );\n\n        // Delete messages\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::EmailSubmission);\n\n        for (document_id, send_at) in destroy_ids {\n            batch\n                .with_document(document_id)\n                .clear(EmailSubmissionField::Metadata)\n                .clear(ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                    property: EmailSubmissionField::Metadata.into(),\n                    value: send_at,\n                }))\n                .commit_point();\n        }\n\n        self.commit_batch(batch).await?;\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/delivery.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ingest::{EmailIngest, IngestEmail, IngestSource};\nuse crate::{mailbox::INBOX_ID, sieve::ingest::SieveScriptIngest};\nuse common::{\n    Server,\n    ipc::{EmailPush, PushNotification},\n};\nuse directory::Permission;\nuse mail_parser::MessageParser;\nuse std::{borrow::Cow, future::Future};\nuse store::ahash::AHashMap;\nuse types::blob_hash::BlobHash;\n\n#[derive(Debug)]\npub struct IngestMessage {\n    pub sender_address: String,\n    pub sender_authenticated: bool,\n    pub recipients: Vec<IngestRecipient>,\n    pub message_blob: BlobHash,\n    pub message_size: u64,\n    pub session_id: u64,\n}\n\n#[derive(Debug)]\npub struct IngestRecipient {\n    pub address: String,\n    pub is_spam: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum LocalDeliveryStatus {\n    Success,\n    TemporaryFailure {\n        reason: Cow<'static, str>,\n    },\n    PermanentFailure {\n        code: [u8; 3],\n        reason: Cow<'static, str>,\n    },\n}\n\npub struct LocalDeliveryResult {\n    pub status: Vec<LocalDeliveryStatus>,\n    pub autogenerated: Vec<AutogeneratedMessage>,\n}\n\npub struct AutogeneratedMessage {\n    pub sender_address: String,\n    pub recipients: Vec<String>,\n    pub message: Vec<u8>,\n}\n\npub trait MailDelivery: Sync + Send {\n    fn deliver_message(\n        &self,\n        message: IngestMessage,\n    ) -> impl Future<Output = LocalDeliveryResult> + Send;\n}\n\nimpl MailDelivery for Server {\n    async fn deliver_message(&self, message: IngestMessage) -> LocalDeliveryResult {\n        // Read message\n        let raw_message = match self\n            .core\n            .storage\n            .blob\n            .get_blob(message.message_blob.as_slice(), 0..usize::MAX)\n            .await\n        {\n            Ok(Some(raw_message)) => raw_message,\n            Ok(None) => {\n                trc::event!(\n                    MessageIngest(trc::MessageIngestEvent::Error),\n                    Reason = \"Blob not found.\",\n                    SpanId = message.session_id,\n                    CausedBy = trc::location!()\n                );\n\n                return LocalDeliveryResult {\n                    status: (0..message.recipients.len())\n                        .map(|_| LocalDeliveryStatus::TemporaryFailure {\n                            reason: \"Blob not found.\".into(),\n                        })\n                        .collect::<Vec<_>>(),\n                    autogenerated: vec![],\n                };\n            }\n            Err(err) => {\n                trc::error!(\n                    err.details(\"Failed to fetch message blob.\")\n                        .span_id(message.session_id)\n                        .caused_by(trc::location!())\n                );\n\n                return LocalDeliveryResult {\n                    status: (0..message.recipients.len())\n                        .map(|_| LocalDeliveryStatus::TemporaryFailure {\n                            reason: \"Temporary I/O error.\".into(),\n                        })\n                        .collect::<Vec<_>>(),\n                    autogenerated: vec![],\n                };\n            }\n        };\n\n        // Obtain the account IDs for each recipient\n        let mut account_ids: AHashMap<u32, usize> =\n            AHashMap::with_capacity(message.recipients.len());\n        let mut result = LocalDeliveryResult {\n            status: Vec::with_capacity(message.recipients.len()),\n            autogenerated: Vec::new(),\n        };\n\n        for rcpt in message.recipients {\n            let account_id = match self\n                .email_to_id(\n                    &self.core.storage.directory,\n                    &rcpt.address,\n                    message.session_id,\n                )\n                .await\n            {\n                Ok(Some(account_id)) => account_id,\n                Ok(None) => {\n                    // Something went wrong\n                    result.status.push(LocalDeliveryStatus::PermanentFailure {\n                        code: [5, 5, 0],\n                        reason: \"Mailbox not found.\".into(),\n                    });\n                    continue;\n                }\n                Err(err) => {\n                    trc::error!(\n                        err.details(\"Failed to lookup recipient.\")\n                            .ctx(trc::Key::To, rcpt.address.to_string())\n                            .span_id(message.session_id)\n                            .caused_by(trc::location!())\n                    );\n                    result.status.push(LocalDeliveryStatus::TemporaryFailure {\n                        reason: \"Address lookup failed.\".into(),\n                    });\n                    continue;\n                }\n            };\n            if let Some(status) = account_ids\n                .get(&account_id)\n                .and_then(|pos| result.status.get(*pos))\n            {\n                result.status.push(status.clone());\n                continue;\n            }\n\n            // Obtain access token\n            let status = match self.get_access_token(account_id).await.and_then(|token| {\n                token\n                    .assert_has_permission(Permission::EmailReceive)\n                    .map(|_| token)\n            }) {\n                Ok(access_token) => {\n                    // Check if there is an active sieve script\n                    match self.sieve_script_get_active(account_id).await {\n                        Ok(None) => {\n                            // Ingest message\n                            self.email_ingest(IngestEmail {\n                                raw_message: &raw_message,\n                                blob_hash: Some(&message.message_blob),\n                                message: MessageParser::new().parse(&raw_message),\n                                access_token: &access_token,\n                                mailbox_ids: vec![INBOX_ID],\n                                keywords: vec![],\n                                received_at: None,\n                                source: IngestSource::Smtp {\n                                    deliver_to: &rcpt.address,\n                                    is_sender_authenticated: message.sender_authenticated,\n                                    is_spam: rcpt.is_spam,\n                                },\n                                session_id: message.session_id,\n                            })\n                            .await\n                        }\n                        Ok(Some(active_script)) => {\n                            self.sieve_script_ingest(\n                                &access_token,\n                                &message.message_blob,\n                                &raw_message,\n                                &message.sender_address,\n                                message.sender_authenticated,\n                                &rcpt,\n                                message.session_id,\n                                active_script,\n                                &mut result.autogenerated,\n                            )\n                            .await\n                        }\n                        Err(err) => Err(err),\n                    }\n                }\n\n                Err(err) => Err(err),\n            };\n\n            let status = match status {\n                Ok(ingested_message) => {\n                    // Notify state change\n                    if ingested_message.change_id != u64::MAX {\n                        self.broadcast_push_notification(PushNotification::EmailPush(EmailPush {\n                            account_id,\n                            email_id: ingested_message.document_id,\n                            change_id: ingested_message.change_id,\n                        }))\n                        .await;\n                    }\n\n                    LocalDeliveryStatus::Success\n                }\n                Err(err) => {\n                    let status = match err.as_ref() {\n                        trc::EventType::Limit(trc::LimitEvent::Quota) => {\n                            LocalDeliveryStatus::TemporaryFailure {\n                                reason: \"Mailbox over quota.\".into(),\n                            }\n                        }\n                        trc::EventType::Limit(trc::LimitEvent::TenantQuota) => {\n                            LocalDeliveryStatus::TemporaryFailure {\n                                reason: \"Organization over quota.\".into(),\n                            }\n                        }\n                        trc::EventType::Security(trc::SecurityEvent::Unauthorized) => {\n                            LocalDeliveryStatus::PermanentFailure {\n                                code: [5, 5, 0],\n                                reason: \"This account is not authorized to receive email.\".into(),\n                            }\n                        }\n                        trc::EventType::MessageIngest(trc::MessageIngestEvent::Error) => {\n                            LocalDeliveryStatus::PermanentFailure {\n                                code: err\n                                    .value(trc::Key::Code)\n                                    .and_then(|v| v.to_uint())\n                                    .map(|n| {\n                                        [(n / 100) as u8, ((n % 100) / 10) as u8, (n % 10) as u8]\n                                    })\n                                    .unwrap_or([5, 5, 0]),\n                                reason: err\n                                    .value_as_str(trc::Key::Reason)\n                                    .unwrap_or_default()\n                                    .to_string()\n                                    .into(),\n                            }\n                        }\n                        _ => LocalDeliveryStatus::TemporaryFailure {\n                            reason: \"Transient server failure.\".into(),\n                        },\n                    };\n\n                    trc::error!(\n                        err.ctx(trc::Key::To, rcpt.address.to_string())\n                            .span_id(message.session_id)\n                    );\n\n                    status\n                }\n            };\n\n            // Cache response for UID to avoid duplicate deliveries\n            account_ids.insert(account_id, result.status.len());\n\n            result.status.push(status);\n        }\n\n        result\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/index/extractors.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::message::metadata::{\n    ArchivedMessageMetadataContents, ArchivedMessageMetadataPart, ArchivedMetadataHeaderValue,\n    MetadataHeaderName, MetadataHeaderValue,\n};\nuse mail_parser::{Addr, Address, Group, HeaderValue};\nuse nlp::language::Language;\nuse rkyv::option::ArchivedOption;\nuse std::borrow::Cow;\n\nimpl ArchivedMessageMetadataContents {\n    pub fn is_html_part(&self, part_id: u16) -> bool {\n        self.html_body.iter().any(|&id| id == part_id)\n    }\n\n    pub fn is_text_part(&self, part_id: u16) -> bool {\n        self.text_body.iter().any(|&id| id == part_id)\n    }\n}\n\nimpl ArchivedMessageMetadataPart {\n    pub fn language(&self) -> Option<Language> {\n        self.header_value(&MetadataHeaderName::ContentLanguage)\n            .and_then(|v| {\n                Language::from_iso_639(v.as_text()?)\n                    .unwrap_or(Language::Unknown)\n                    .into()\n            })\n    }\n}\n\n#[derive(Debug, PartialEq, Eq)]\npub enum AddressElement {\n    Name,\n    Address,\n    GroupName,\n}\n\npub trait VisitText {\n    fn visit_addresses(&self, visitor: impl FnMut(AddressElement, &str));\n    fn visit_text<'x>(&'x self, visitor: impl FnMut(&'x str));\n    fn into_visit_text(self, visitor: impl FnMut(String));\n}\n\nimpl VisitText for HeaderValue<'_> {\n    fn visit_addresses(&self, mut visitor: impl FnMut(AddressElement, &str)) {\n        match self {\n            HeaderValue::Address(Address::List(addr_list)) => {\n                for addr in addr_list {\n                    if let Some(name) = &addr.name {\n                        visitor(AddressElement::Name, name);\n                    }\n                    if let Some(addr) = &addr.address {\n                        visitor(AddressElement::Address, addr);\n                    }\n                }\n            }\n            HeaderValue::Address(Address::Group(groups)) => {\n                for group in groups {\n                    if let Some(name) = &group.name {\n                        visitor(AddressElement::GroupName, name);\n                    }\n\n                    for addr in &group.addresses {\n                        if let Some(name) = &addr.name {\n                            visitor(AddressElement::Name, name);\n                        }\n                        if let Some(addr) = &addr.address {\n                            visitor(AddressElement::Address, addr);\n                        }\n                    }\n                }\n            }\n            _ => (),\n        }\n    }\n\n    fn visit_text<'x>(&'x self, mut visitor: impl FnMut(&'x str)) {\n        match &self {\n            HeaderValue::Text(text) => {\n                visitor(text.as_ref());\n            }\n            HeaderValue::TextList(texts) => {\n                for text in texts {\n                    visitor(text.as_ref());\n                }\n            }\n            _ => (),\n        }\n    }\n\n    fn into_visit_text(self, mut visitor: impl FnMut(String)) {\n        match self {\n            HeaderValue::Text(text) => {\n                visitor(text.into_owned());\n            }\n            HeaderValue::TextList(texts) => {\n                for text in texts {\n                    visitor(text.into_owned());\n                }\n            }\n            _ => (),\n        }\n    }\n}\n\npub trait VisitTextArchived {\n    fn visit_addresses(&self, visitor: impl FnMut(AddressElement, &str));\n    fn visit_text(&self, visitor: impl FnMut(&str));\n}\n\nimpl VisitTextArchived for MetadataHeaderValue {\n    fn visit_addresses(&self, mut visitor: impl FnMut(AddressElement, &str)) {\n        match self {\n            MetadataHeaderValue::AddressList(addr_list) => {\n                for addr in addr_list.iter() {\n                    if let Some(name) = &addr.name {\n                        visitor(AddressElement::Name, name);\n                    }\n                    if let Some(addr) = &addr.address {\n                        visitor(AddressElement::Address, addr);\n                    }\n                }\n            }\n            MetadataHeaderValue::AddressGroup(groups) => {\n                for group in groups.iter() {\n                    if let Some(name) = &group.name {\n                        visitor(AddressElement::GroupName, name);\n                    }\n\n                    for addr in group.addresses.iter() {\n                        if let Some(name) = &addr.name {\n                            visitor(AddressElement::Name, name);\n                        }\n                        if let Some(addr) = &addr.address {\n                            visitor(AddressElement::Address, addr);\n                        }\n                    }\n                }\n            }\n            _ => (),\n        }\n    }\n\n    fn visit_text(&self, mut visitor: impl FnMut(&str)) {\n        match &self {\n            MetadataHeaderValue::Text(text) => {\n                visitor(text.as_ref());\n            }\n            MetadataHeaderValue::TextList(texts) => {\n                for text in texts.iter() {\n                    visitor(text.as_ref());\n                }\n            }\n            _ => (),\n        }\n    }\n}\n\nimpl VisitTextArchived for ArchivedMetadataHeaderValue {\n    fn visit_addresses(&self, mut visitor: impl FnMut(AddressElement, &str)) {\n        match self {\n            ArchivedMetadataHeaderValue::AddressList(addr_list) => {\n                for addr in addr_list.iter() {\n                    if let ArchivedOption::Some(name) = &addr.name {\n                        visitor(AddressElement::Name, name);\n                    }\n                    if let ArchivedOption::Some(addr) = &addr.address {\n                        visitor(AddressElement::Address, addr);\n                    }\n                }\n            }\n            ArchivedMetadataHeaderValue::AddressGroup(groups) => {\n                for group in groups.iter() {\n                    if let ArchivedOption::Some(name) = &group.name {\n                        visitor(AddressElement::GroupName, name);\n                    }\n\n                    for addr in group.addresses.iter() {\n                        if let ArchivedOption::Some(name) = &addr.name {\n                            visitor(AddressElement::Name, name);\n                        }\n                        if let ArchivedOption::Some(addr) = &addr.address {\n                            visitor(AddressElement::Address, addr);\n                        }\n                    }\n                }\n            }\n            _ => (),\n        }\n    }\n\n    fn visit_text(&self, mut visitor: impl FnMut(&str)) {\n        match &self {\n            ArchivedMetadataHeaderValue::Text(text) => {\n                visitor(text.as_ref());\n            }\n            ArchivedMetadataHeaderValue::TextList(texts) => {\n                for text in texts.iter() {\n                    visitor(text.as_ref());\n                }\n            }\n            _ => (),\n        }\n    }\n}\n\npub trait TrimTextValue {\n    fn trim_text(self, length: usize) -> Self;\n}\n\nimpl TrimTextValue for HeaderValue<'_> {\n    fn trim_text(self, length: usize) -> Self {\n        match self {\n            HeaderValue::Address(Address::List(v)) => {\n                HeaderValue::Address(Address::List(v.trim_text(length)))\n            }\n            HeaderValue::Address(Address::Group(v)) => {\n                HeaderValue::Address(Address::Group(v.trim_text(length)))\n            }\n            HeaderValue::Text(v) => HeaderValue::Text(v.trim_text(length)),\n            HeaderValue::TextList(v) => HeaderValue::TextList(v.trim_text(length)),\n            v => v,\n        }\n    }\n}\n\nimpl TrimTextValue for Addr<'_> {\n    fn trim_text(self, length: usize) -> Self {\n        Self {\n            name: self.name.map(|v| v.trim_text(length)),\n            address: self.address.map(|v| v.trim_text(length)),\n        }\n    }\n}\n\nimpl TrimTextValue for Group<'_> {\n    fn trim_text(self, length: usize) -> Self {\n        Self {\n            name: self.name.map(|v| v.trim_text(length)),\n            addresses: self.addresses.trim_text(length),\n        }\n    }\n}\n\nimpl TrimTextValue for &str {\n    fn trim_text(self, length: usize) -> Self {\n        if self.len() < length {\n            self\n        } else {\n            let mut index = 0;\n\n            for (i, _) in self.char_indices() {\n                if i > length {\n                    break;\n                }\n                index = i;\n            }\n\n            &self[..index]\n        }\n    }\n}\n\nimpl TrimTextValue for Cow<'_, str> {\n    fn trim_text(self, length: usize) -> Self {\n        if self.len() < length {\n            self\n        } else {\n            let mut result = String::with_capacity(length);\n            for (i, c) in self.char_indices() {\n                if i > length {\n                    break;\n                }\n                result.push(c);\n            }\n            result.into()\n        }\n    }\n}\n\nimpl<T: TrimTextValue> TrimTextValue for Vec<T> {\n    fn trim_text(self, length: usize) -> Self {\n        self.into_iter().map(|v| v.trim_text(length)).collect()\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/index/metadata.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::message::{\n    index::{IndexMessage, MAX_MESSAGE_PARTS, PREVIEW_LENGTH},\n    metadata::{\n        ArchivedMessageMetadata, ArchivedMessageMetadataPart, ArchivedMetadataHeaderName,\n        MESSAGE_HAS_ATTACHMENT, MESSAGE_RECEIVED_MASK, MessageData, MessageMetadata,\n        MessageMetadataPart, build_metadata_contents,\n    },\n};\nuse common::storage::index::ObjectIndexBuilder;\nuse mail_parser::{\n    PartType,\n    decoders::html::html_to_text,\n    parsers::{fields::thread::thread_name, preview::preview_text},\n};\nuse store::{\n    Serialize,\n    write::{Archiver, BatchBuilder, BlobLink, BlobOp, IndexPropertyClass, ValueClass},\n};\nuse trc::AddContext;\nuse types::{blob_hash::BlobHash, field::EmailField};\nuse utils::cheeky_hash::CheekyHash;\n\nimpl MessageMetadata {\n    #[inline(always)]\n    pub fn root_part(&self) -> &MessageMetadataPart {\n        &self.contents[0].parts[0]\n    }\n\n    pub fn index(self, batch: &mut BatchBuilder, set: bool) -> trc::Result<()> {\n        if set {\n            batch\n                .set(\n                    BlobOp::Link {\n                        hash: self.blob_hash.clone(),\n                        to: BlobLink::Document,\n                    },\n                    Vec::new(),\n                )\n                .set(EmailField::Metadata, Archiver::new(self).serialize()?);\n        } else {\n            batch\n                .clear(BlobOp::Link {\n                    hash: self.blob_hash.clone(),\n                    to: BlobLink::Document,\n                })\n                .clear(EmailField::Metadata);\n        }\n\n        Ok(())\n    }\n}\n\nimpl ArchivedMessageMetadata {\n    #[inline(always)]\n    pub fn root_part(&self) -> &ArchivedMessageMetadataPart {\n        &self.contents[0].parts[0]\n    }\n\n    pub fn unindex(&self, batch: &mut BatchBuilder) {\n        // Delete metadata\n        let thread_name = self\n            .contents\n            .first()\n            .and_then(|c| c.parts.first())\n            .and_then(|p| {\n                p.headers.iter().rev().find_map(|h| {\n                    if let ArchivedMetadataHeaderName::Subject = &h.name {\n                        h.value.as_text()\n                    } else {\n                        None\n                    }\n                })\n            })\n            .map(thread_name)\n            .unwrap_or_default();\n\n        batch\n            .clear(EmailField::Metadata)\n            .clear(ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                property: EmailField::Threading.into(),\n                hash: CheekyHash::new(if !thread_name.is_empty() {\n                    thread_name\n                } else {\n                    \"!\"\n                }),\n            }))\n            .clear(BlobOp::Link {\n                hash: BlobHash::from(&self.blob_hash),\n                to: BlobLink::Document,\n            });\n    }\n}\n\nimpl IndexMessage for BatchBuilder {\n    fn index_message<'x>(\n        &mut self,\n        tenant_id: Option<u32>,\n        mut message: mail_parser::Message<'x>,\n        extra_headers: Vec<u8>,\n        mut extra_headers_parsed: Vec<mail_parser::Header<'x>>,\n        blob_hash: BlobHash,\n        data: MessageData,\n        received_at: u64,\n    ) -> trc::Result<&mut Self> {\n        let mut has_attachments = false;\n        let mut preview = None;\n        let preview_part_id = message\n            .text_body\n            .first()\n            .or_else(|| message.html_body.first())\n            .copied()\n            .unwrap_or(u32::MAX);\n\n        for (part_id, part) in message.parts.iter().take(MAX_MESSAGE_PARTS).enumerate() {\n            let part_id = part_id as u32;\n            match &part.body {\n                mail_parser::PartType::Text(text) => {\n                    if part_id == preview_part_id {\n                        preview =\n                            preview_text(text.replace('\\r', \"\").into(), PREVIEW_LENGTH).into();\n                    }\n\n                    if !message.text_body.contains(&part_id)\n                        && !message.html_body.contains(&part_id)\n                    {\n                        has_attachments = true;\n                    }\n                }\n                mail_parser::PartType::Html(html) => {\n                    let text = html_to_text(html);\n                    if part_id == preview_part_id {\n                        preview =\n                            preview_text(text.replace('\\r', \"\").into(), PREVIEW_LENGTH).into();\n                    }\n\n                    if !message.text_body.contains(&part_id)\n                        && !message.html_body.contains(&part_id)\n                    {\n                        has_attachments = true;\n                    }\n                }\n                mail_parser::PartType::Binary(_) | mail_parser::PartType::Message(_)\n                    if !has_attachments =>\n                {\n                    has_attachments = true;\n                }\n                _ => {}\n            }\n        }\n\n        // Build raw headers\n        let root_part = message.root_part();\n        let mut raw_headers = Vec::with_capacity(\n            (root_part.offset_body - root_part.offset_header) as usize + extra_headers.len(),\n        );\n        raw_headers.extend_from_slice(&extra_headers);\n        raw_headers.extend_from_slice(\n            message\n                .raw_message\n                .as_ref()\n                .get(root_part.offset_header as usize..root_part.offset_body as usize)\n                .unwrap_or_default(),\n        );\n\n        // Add additional headers to message\n        let blob_body_offset = if !extra_headers.is_empty() {\n            // Add extra headers to root part\n            let offset_start = extra_headers.len() as u32;\n            let mut part_iter_stack = Vec::new();\n            let mut part_iter = message.parts.iter_mut();\n\n            loop {\n                if let Some(part) = part_iter.next() {\n                    // Increment header offsets\n                    for header in part.headers.iter_mut() {\n                        header.offset_field += offset_start;\n                        header.offset_start += offset_start;\n                        header.offset_end += offset_start;\n                    }\n\n                    // Adjust part offsets\n                    part.offset_body += offset_start;\n                    part.offset_end += offset_start;\n                    part.offset_header += offset_start;\n\n                    if let PartType::Message(sub_message) = &mut part.body\n                        && sub_message.root_part().offset_header != 0\n                    {\n                        part_iter_stack.push(part_iter);\n                        part_iter = sub_message.parts.iter_mut();\n                    }\n                } else if let Some(iter) = part_iter_stack.pop() {\n                    part_iter = iter;\n                } else {\n                    break;\n                }\n            }\n\n            // Add extra headers to root part\n            let root_part = &mut message.parts[0];\n            extra_headers_parsed.append(&mut root_part.headers);\n            root_part.offset_header = 0;\n            root_part.headers = extra_headers_parsed;\n            root_part.offset_body - offset_start\n        } else {\n            message.root_part().offset_body\n        };\n\n        // Build metadata\n        let metadata = MessageMetadata {\n            preview: preview.unwrap_or_default().into_owned().into_boxed_str(),\n            raw_headers: raw_headers.into_boxed_slice(),\n            contents: build_metadata_contents(message),\n            blob_hash,\n            blob_body_offset,\n            rcvd_attach: (if has_attachments {\n                MESSAGE_HAS_ATTACHMENT\n            } else {\n                0\n            }) | (received_at & MESSAGE_RECEIVED_MASK),\n        };\n\n        self.set(\n            BlobOp::Link {\n                hash: metadata.blob_hash.clone(),\n                to: BlobLink::Document,\n            },\n            Vec::new(),\n        )\n        .custom(\n            ObjectIndexBuilder::<(), _>::new()\n                .with_tenant_id(tenant_id)\n                .with_changes(data),\n        )\n        .caused_by(trc::location!())?\n        .set(\n            EmailField::Metadata,\n            Archiver::new(metadata)\n                .serialize()\n                .caused_by(trc::location!())?,\n        );\n\n        Ok(self)\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/index/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    mailbox::{JUNK_ID, TRASH_ID},\n    message::metadata::{ArchivedMessageData, MessageData},\n};\nuse common::storage::index::{IndexItem, IndexValue, IndexableObject};\nuse store::write::now;\nuse types::{blob_hash::BlobHash, collection::SyncCollection, field::EmailField};\n\npub mod extractors;\npub mod metadata;\npub mod search;\n\npub(super) const MAX_MESSAGE_PARTS: usize = 1000;\npub const PREVIEW_LENGTH: usize = 256;\n\nimpl IndexableObject for MessageData {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        let mut mailboxes = Vec::with_capacity(self.mailboxes.len());\n        let mut is_in_trash = false;\n\n        for mailbox in &self.mailboxes {\n            mailboxes.push(mailbox.mailbox_id);\n            is_in_trash |= mailbox.mailbox_id == TRASH_ID || mailbox.mailbox_id == JUNK_ID;\n        }\n\n        [\n            IndexValue::Property {\n                field: EmailField::DeletedAt.into(),\n                value: if is_in_trash {\n                    IndexItem::from(now())\n                } else {\n                    IndexItem::None\n                },\n            },\n            IndexValue::Quota { used: self.size },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::Email,\n                prefix: self.thread_id.into(),\n            },\n            IndexValue::LogContainerProperty {\n                sync_collection: SyncCollection::Thread,\n                ids: vec![self.thread_id],\n            },\n            IndexValue::LogContainerProperty {\n                sync_collection: SyncCollection::Email,\n                ids: mailboxes,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedMessageData {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        let mut mailboxes = Vec::with_capacity(self.mailboxes.len());\n        let mut is_in_trash = false;\n\n        for mailbox in self.mailboxes.iter() {\n            let mailbox_id = mailbox.mailbox_id.to_native();\n            mailboxes.push(mailbox_id);\n            is_in_trash |= mailbox_id == TRASH_ID || mailbox_id == JUNK_ID;\n        }\n\n        [\n            IndexValue::Property {\n                field: EmailField::DeletedAt.into(),\n                value: if is_in_trash {\n                    IndexItem::from(now())\n                } else {\n                    IndexItem::None\n                },\n            },\n            IndexValue::Quota {\n                used: self.size.to_native(),\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::Email,\n                prefix: self.thread_id.to_native().into(),\n            },\n            IndexValue::LogContainerProperty {\n                sync_collection: SyncCollection::Thread,\n                ids: vec![self.thread_id.to_native()],\n            },\n            IndexValue::LogContainerProperty {\n                sync_collection: SyncCollection::Email,\n                ids: mailboxes,\n            },\n        ]\n        .into_iter()\n    }\n}\n\npub(super) trait IndexMessage {\n    #[allow(clippy::too_many_arguments)]\n    fn index_message<'x>(\n        &mut self,\n        tenant_id: Option<u32>,\n        message: mail_parser::Message<'x>,\n        extra_headers: Vec<u8>,\n        extra_headers_parsed: Vec<mail_parser::Header<'x>>,\n        blob_hash: BlobHash,\n        data: MessageData,\n        received_at: u64,\n    ) -> trc::Result<&mut Self>;\n}\n"
  },
  {
    "path": "crates/email/src/message/index/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::message::{\n    index::{MAX_MESSAGE_PARTS, extractors::VisitTextArchived},\n    metadata::{\n        ArchivedMessageMetadata, ArchivedMetadataHeaderName, ArchivedMetadataHeaderValue,\n        ArchivedMetadataPartType, DecodedPartContent, MESSAGE_RECEIVED_MASK, MetadataHeaderName,\n    },\n};\nuse mail_parser::{DateTime, decoders::html::html_to_text, parsers::fields::thread::thread_name};\nuse nlp::{\n    language::{\n        Language,\n        detect::{LanguageDetector, MIN_LANGUAGE_SCORE},\n    },\n    tokenizers::word::WordTokenizer,\n};\nuse store::{\n    ahash::AHashSet,\n    backend::MAX_TOKEN_LENGTH,\n    search::{EmailSearchField, IndexDocument, SearchField},\n    write::SearchIndex,\n};\nuse utils::chained_bytes::ChainedBytes;\n\nimpl ArchivedMessageMetadata {\n    pub fn index_document(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        raw_message: &[u8],\n        index_fields: &AHashSet<SearchField>,\n        default_language: Language,\n    ) -> IndexDocument {\n        let mut detector = LanguageDetector::new();\n        let mut language = Language::Unknown;\n        let message_contents = &self.contents[0];\n        let mut document = IndexDocument::new(SearchIndex::Email)\n            .with_account_id(account_id)\n            .with_document_id(document_id);\n\n        let raw_message = ChainedBytes::new(self.raw_headers.as_ref()).with_last(\n            raw_message\n                .get(self.blob_body_offset.to_native() as usize..)\n                .unwrap_or_default(),\n        );\n\n        if index_fields.is_empty()\n            || index_fields.contains(&SearchField::Email(EmailSearchField::ReceivedAt))\n        {\n            document.index_unsigned(\n                SearchField::Email(EmailSearchField::ReceivedAt),\n                self.rcvd_attach.to_native() & MESSAGE_RECEIVED_MASK,\n            );\n        }\n        if index_fields.is_empty()\n            || index_fields.contains(&SearchField::Email(EmailSearchField::Size))\n        {\n            document.index_unsigned(\n                SearchField::Email(EmailSearchField::Size),\n                raw_message.len() as u32,\n            );\n        }\n\n        for (part_id, part) in message_contents\n            .parts\n            .iter()\n            .take(MAX_MESSAGE_PARTS)\n            .enumerate()\n        {\n            let part_language = part.language().unwrap_or(language);\n            if part_id == 0 {\n                language = part_language;\n\n                for header in part.headers.iter().rev() {\n                    match &header.name {\n                        ArchivedMetadataHeaderName::From => {\n                            if index_fields.is_empty()\n                                || index_fields\n                                    .contains(&SearchField::Email(EmailSearchField::From))\n                            {\n                                header.value.visit_addresses(|_, value| {\n                                    document.index_text(\n                                        SearchField::Email(EmailSearchField::From),\n                                        value,\n                                        Language::None,\n                                    );\n                                });\n                            }\n                        }\n                        ArchivedMetadataHeaderName::To => {\n                            if index_fields.is_empty()\n                                || index_fields.contains(&SearchField::Email(EmailSearchField::To))\n                            {\n                                header.value.visit_addresses(|_, value| {\n                                    document.index_text(\n                                        SearchField::Email(EmailSearchField::To),\n                                        value,\n                                        Language::None,\n                                    );\n                                });\n                            }\n                        }\n                        ArchivedMetadataHeaderName::Cc => {\n                            if index_fields.is_empty()\n                                || index_fields.contains(&SearchField::Email(EmailSearchField::Cc))\n                            {\n                                header.value.visit_addresses(|_, value| {\n                                    document.index_text(\n                                        SearchField::Email(EmailSearchField::Cc),\n                                        value,\n                                        Language::None,\n                                    );\n                                });\n                            }\n                        }\n                        ArchivedMetadataHeaderName::Bcc => {\n                            if index_fields.is_empty()\n                                || index_fields.contains(&SearchField::Email(EmailSearchField::Bcc))\n                            {\n                                header.value.visit_addresses(|_, value| {\n                                    document.index_text(\n                                        SearchField::Email(EmailSearchField::Bcc),\n                                        value,\n                                        Language::None,\n                                    );\n                                });\n                            }\n                        }\n                        ArchivedMetadataHeaderName::Subject => {\n                            if (index_fields.is_empty()\n                                || index_fields\n                                    .contains(&SearchField::Email(EmailSearchField::Subject)))\n                                && let Some(subject) = header.value.as_text()\n                            {\n                                let subject = thread_name(subject);\n\n                                if part_language.is_unknown() {\n                                    detector.detect(subject, MIN_LANGUAGE_SCORE);\n                                }\n\n                                document.index_text(\n                                    SearchField::Email(EmailSearchField::Subject),\n                                    subject,\n                                    part_language,\n                                );\n                            }\n                        }\n                        ArchivedMetadataHeaderName::Date => {\n                            if (index_fields.is_empty()\n                                || index_fields\n                                    .contains(&SearchField::Email(EmailSearchField::SentAt)))\n                                && let Some(date) = header.value.as_datetime()\n                            {\n                                document.index_integer(\n                                    SearchField::Email(EmailSearchField::SentAt),\n                                    DateTime::from(date).to_timestamp(),\n                                );\n                            }\n                        }\n                        _ => {\n                            #[cfg(not(feature = \"test_mode\"))]\n                            let index_headers = index_fields\n                                .contains(&SearchField::Email(EmailSearchField::Headers));\n\n                            #[cfg(feature = \"test_mode\")]\n                            let index_headers = true;\n\n                            if index_headers {\n                                let mut value = String::new();\n                                match &header.value {\n                                    ArchivedMetadataHeaderValue::AddressList(_)\n                                    | ArchivedMetadataHeaderValue::AddressGroup(_) => {\n                                        header.value.visit_addresses(|_, addr| {\n                                            if !value.is_empty() {\n                                                value.push(' ');\n                                            }\n                                            value.push_str(addr);\n                                        });\n                                    }\n                                    ArchivedMetadataHeaderValue::Text(_)\n                                    | ArchivedMetadataHeaderValue::TextList(_) => {\n                                        header.value.visit_text(|text| {\n                                            if !value.is_empty() {\n                                                value.push(' ');\n                                            }\n                                            value.push_str(text);\n                                        });\n                                    }\n                                    _ => {\n                                        if (matches!(\n                                            header.value,\n                                            ArchivedMetadataHeaderValue::ContentType(_)\n                                        ) || matches!(\n                                            header.name,\n                                            ArchivedMetadataHeaderName::Received\n                                        )) && let Some(header) =\n                                            raw_message.get(header.value_range())\n                                        {\n                                            let header = std::str::from_utf8(header.as_ref())\n                                                .unwrap_or_default();\n\n                                            for word in WordTokenizer::new(header, MAX_TOKEN_LENGTH)\n                                            {\n                                                if !value.is_empty() {\n                                                    value.push(' ');\n                                                }\n                                                value.push_str(word.word.as_ref());\n                                            }\n                                        }\n                                    }\n                                }\n\n                                document.insert_key_value(\n                                    EmailSearchField::Headers,\n                                    header.name.as_str(),\n                                    value,\n                                );\n                            }\n                        }\n                    }\n                }\n            }\n\n            let part_id = part_id as u16;\n            match &part.body {\n                ArchivedMetadataPartType::Text | ArchivedMetadataPartType::Html => {\n                    let text = match (part.decode_contents(&raw_message), &part.body) {\n                        (DecodedPartContent::Text(text), ArchivedMetadataPartType::Text) => text,\n                        (DecodedPartContent::Text(html), ArchivedMetadataPartType::Html) => {\n                            html_to_text(html.as_ref()).into()\n                        }\n                        _ => unreachable!(),\n                    };\n\n                    if message_contents.is_html_part(part_id)\n                        || message_contents.is_text_part(part_id)\n                    {\n                        if index_fields.is_empty()\n                            || index_fields.contains(&SearchField::Email(EmailSearchField::Body))\n                        {\n                            if part_language.is_unknown() {\n                                detector.detect(text.as_ref(), MIN_LANGUAGE_SCORE);\n                            }\n\n                            document.index_text(\n                                SearchField::Email(EmailSearchField::Body),\n                                text.as_ref(),\n                                part_language,\n                            );\n                        }\n                    } else if index_fields.is_empty()\n                        || index_fields.contains(&SearchField::Email(EmailSearchField::Attachment))\n                    {\n                        if part_language.is_unknown() {\n                            detector.detect(text.as_ref(), MIN_LANGUAGE_SCORE);\n                        }\n\n                        document.index_text(\n                            SearchField::Email(EmailSearchField::Attachment),\n                            text.as_ref(),\n                            part_language,\n                        );\n                    }\n                }\n                ArchivedMetadataPartType::Message(nested_message_id)\n                    if index_fields.is_empty()\n                        || index_fields\n                            .contains(&SearchField::Email(EmailSearchField::Attachment)) =>\n                {\n                    let nested_message = self.message_id(*nested_message_id);\n                    let nested_message_language = nested_message\n                        .root_part()\n                        .language()\n                        .unwrap_or(Language::Unknown);\n                    if let Some(ArchivedMetadataHeaderValue::Text(subject)) = nested_message\n                        .root_part()\n                        .header_value(&MetadataHeaderName::Subject)\n                    {\n                        if nested_message_language.is_unknown() {\n                            detector.detect(subject.as_ref(), MIN_LANGUAGE_SCORE);\n                        }\n\n                        document.index_text(\n                            SearchField::Email(EmailSearchField::Attachment),\n                            subject.as_ref(),\n                            nested_message_language,\n                        );\n                    }\n\n                    for sub_part in nested_message.parts.iter().take(MAX_MESSAGE_PARTS) {\n                        let language = sub_part.language().unwrap_or(nested_message_language);\n                        match &sub_part.body {\n                            ArchivedMetadataPartType::Text | ArchivedMetadataPartType::Html => {\n                                let text = match (\n                                    sub_part.decode_contents(&raw_message),\n                                    &sub_part.body,\n                                ) {\n                                    (\n                                        DecodedPartContent::Text(text),\n                                        ArchivedMetadataPartType::Text,\n                                    ) => text,\n                                    (\n                                        DecodedPartContent::Text(html),\n                                        ArchivedMetadataPartType::Html,\n                                    ) => html_to_text(html.as_ref()).into(),\n                                    _ => unreachable!(),\n                                };\n\n                                if language.is_unknown() {\n                                    detector.detect(text.as_ref(), MIN_LANGUAGE_SCORE);\n                                }\n\n                                document.index_text(\n                                    SearchField::Email(EmailSearchField::Attachment),\n                                    text.as_ref(),\n                                    language,\n                                );\n                            }\n                            _ => (),\n                        }\n                    }\n                }\n                _ => {}\n            }\n        }\n\n        #[cfg(not(feature = \"test_mode\"))]\n        document.set_unknown_language(\n            detector\n                .most_frequent_language()\n                .unwrap_or(default_language),\n        );\n\n        #[cfg(feature = \"test_mode\")]\n        document.set_unknown_language(default_language);\n\n        let has_attachment =\n            document.has_field(&(SearchField::Email(EmailSearchField::Attachment)));\n\n        document.index_bool(EmailSearchField::HasAttachment, has_attachment);\n        document\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/ingest.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::crypto::{EncryptMessage, EncryptMessageError};\nuse crate::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess, mailbox::MailboxCacheAccess},\n    mailbox::{INBOX_ID, JUNK_ID, SENT_ID, TRASH_ID, UidMailbox},\n    message::{\n        crypto::EncryptionParams,\n        index::{IndexMessage, extractors::VisitText},\n        metadata::{MessageData, MessageMetadata},\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse directory::Permission;\nuse groupware::{\n    calendar::itip::{ItipIngest, ItipIngestError},\n    scheduling::{ItipError, ItipMessages},\n};\nuse mail_parser::{\n    DateTime, Header, HeaderName, HeaderValue, Message, MessageParser, MimeHeaders, PartType,\n    parsers::fields::thread::thread_name,\n};\nuse std::{borrow::Cow, cmp::Ordering, fmt::Write, time::Instant};\nuse std::{future::Future, hash::Hasher};\nuse store::write::{AlignedBytes, Archive};\nuse store::{\n    IndexKeyPrefix, IterateParams, U32_LEN, ValueKey,\n    ahash::{AHashMap, AHashSet},\n    write::{\n        AssignedId, AssignedIds, BatchBuilder, BlobLink, BlobOp, IndexPropertyClass, SearchIndex,\n        TaskEpoch, TaskQueueClass, ValueClass, key::DeserializeBigEndian, now,\n    },\n};\nuse trc::{AddContext, MessageIngestEvent, SpamEvent};\nuse types::{\n    blob::{BlobClass, BlobId},\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection},\n    field::{ContactField, EmailField, MailboxField, PrincipalField},\n    keyword::Keyword,\n    special_use::SpecialUse,\n};\nuse utils::{cheeky_hash::CheekyHash, sanitize_email};\n\n#[derive(Default)]\npub struct IngestedEmail {\n    pub document_id: u32,\n    pub thread_id: u32,\n    pub change_id: u64,\n    pub blob_id: BlobId,\n    pub size: usize,\n    pub imap_uids: Vec<u32>,\n}\n\npub struct IngestEmail<'x> {\n    pub raw_message: &'x [u8],\n    pub blob_hash: Option<&'x BlobHash>,\n    pub message: Option<Message<'x>>,\n    pub access_token: &'x AccessToken,\n    pub mailbox_ids: Vec<u32>,\n    pub keywords: Vec<Keyword>,\n    pub received_at: Option<u64>,\n    pub source: IngestSource<'x>,\n    pub session_id: u64,\n}\n\n#[derive(Clone, Copy, PartialEq, Eq, Debug)]\npub enum IngestSource<'x> {\n    Smtp {\n        deliver_to: &'x str,\n        is_sender_authenticated: bool,\n        is_spam: bool,\n    },\n    Jmap {\n        train_classifier: bool,\n    },\n    Imap {\n        train_classifier: bool,\n    },\n    Restore,\n}\n\npub trait EmailIngest: Sync + Send {\n    fn email_ingest(\n        &self,\n        params: IngestEmail,\n    ) -> impl Future<Output = trc::Result<IngestedEmail>> + Send;\n    fn find_thread_id(\n        &self,\n        account_id: u32,\n        thread_name: &str,\n        message_ids: &[CheekyHash],\n    ) -> impl Future<Output = trc::Result<ThreadResult>> + Send;\n    fn assign_email_ids(\n        &self,\n        account_id: u32,\n        mailbox_ids: impl IntoIterator<Item = u32> + Sync + Send,\n        generate_email_id: bool,\n    ) -> impl Future<Output = trc::Result<impl Iterator<Item = u32> + 'static>> + Send;\n    fn add_account_spam_sample(\n        &self,\n        batch: &mut BatchBuilder,\n        account_id: u32,\n        document_id: u32,\n        is_spam: bool,\n        span_id: u64,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n    fn add_spam_sample(\n        &self,\n        batch: &mut BatchBuilder,\n        hash: BlobHash,\n        is_spam: bool,\n        hold_sample: bool,\n        span_id: u64,\n    );\n}\n\npub struct ThreadResult {\n    pub thread_id: Option<u32>,\n    pub thread_hash: CheekyHash,\n    pub merge_ids: Vec<u32>,\n    pub duplicate_ids: Vec<u32>,\n}\n\nimpl EmailIngest for Server {\n    #[allow(clippy::blocks_in_conditions)]\n    async fn email_ingest(&self, mut params: IngestEmail<'_>) -> trc::Result<IngestedEmail> {\n        // Check quota\n        let start_time = Instant::now();\n        let account_id = params.access_token.primary_id;\n        let tenant_id = params.access_token.tenant.map(|t| t.id);\n        let mut raw_message_len = params.raw_message.len() as u64;\n        let resource_token = params.access_token.as_resource_token();\n        self.has_available_quota(&resource_token, raw_message_len)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Parse message\n        let mut raw_message = Cow::from(params.raw_message);\n        let mut message = params.message.ok_or_else(|| {\n            trc::EventType::MessageIngest(trc::MessageIngestEvent::Error)\n                .ctx(trc::Key::Code, 550)\n                .ctx(trc::Key::Reason, \"Failed to parse e-mail message.\")\n        })?;\n\n        // Obtain message references and thread name\n        let mut message_id = None;\n        let mut message_ids = Vec::new();\n        let thread_result = {\n            let mut subject = \"\";\n            for header in message.root_part().headers().iter().rev() {\n                match &header.name {\n                    HeaderName::MessageId => header.value.visit_text(|id| {\n                        if !id.is_empty() {\n                            if message_id.is_none() {\n                                message_id = id.to_string().into();\n                            }\n                            message_ids.push(CheekyHash::new(id.as_bytes()));\n                        }\n                    }),\n                    HeaderName::InReplyTo\n                    | HeaderName::References\n                    | HeaderName::ResentMessageId => {\n                        header.value.visit_text(|id| {\n                            if !id.is_empty() {\n                                message_ids.push(CheekyHash::new(id.as_bytes()));\n                            }\n                        });\n                    }\n                    HeaderName::Subject if subject.is_empty() => {\n                        subject = thread_name(match &header.value {\n                            HeaderValue::Text(text) => text.as_ref(),\n                            HeaderValue::TextList(list) if !list.is_empty() => {\n                                list.first().unwrap().as_ref()\n                            }\n                            _ => \"\",\n                        });\n                    }\n                    _ => (),\n                }\n            }\n\n            message_ids.sort_unstable();\n            message_ids.dedup();\n\n            self.find_thread_id(account_id, subject, &message_ids)\n                .await?\n        };\n\n        // Skip duplicate messages for SMTP ingestion\n        if !thread_result.duplicate_ids.is_empty() && params.source.is_smtp() {\n            // Fetch cached messages\n            let cache = self\n                .get_cached_messages(account_id)\n                .await\n                .caused_by(trc::location!())?;\n\n            // Skip duplicate messages\n            let target_mailbox_id = params.mailbox_ids.first().copied().unwrap_or(INBOX_ID);\n            if !cache\n                .in_mailboxes(&[target_mailbox_id, JUNK_ID])\n                .any(|m| thread_result.duplicate_ids.contains(&m.document_id))\n            {\n                trc::event!(\n                    MessageIngest(MessageIngestEvent::Duplicate),\n                    SpanId = params.session_id,\n                    AccountId = account_id,\n                    MessageId = message_id,\n                );\n\n                return Ok(IngestedEmail {\n                    document_id: 0,\n                    thread_id: 0,\n                    change_id: u64::MAX,\n                    blob_id: BlobId::default(),\n                    imap_uids: Vec::new(),\n                    size: 0,\n                });\n            }\n        }\n\n        // Spam classification and training\n        let mut train_spam = None;\n        let mut extra_headers = String::new();\n        let mut extra_headers_parsed = Vec::new();\n        let mut itip_messages = Vec::new();\n        let is_spam = match params.source {\n            IngestSource::Smtp {\n                deliver_to,\n                is_sender_authenticated,\n                mut is_spam,\n            } => {\n                // Add delivered to header\n                if self.core.smtp.session.data.add_delivered_to {\n                    extra_headers = format!(\"Delivered-To: {deliver_to}\\r\\n\");\n                    extra_headers_parsed.push(Header {\n                        name: HeaderName::Other(\"Delivered-To\".into()),\n                        value: HeaderValue::Text(deliver_to.into()),\n                        offset_field: 0,\n                        offset_start: 13,\n                        offset_end: extra_headers.len() as u32,\n                    });\n                }\n\n                // Spam training on confirmed false positives\n                if self.core.spam.enabled {\n                    let mut overridden = None;\n                    // If the message is classified as spam, check whether the\n                    // sender address is present in the user's address book.\n                    if is_spam\n                        && self.core.spam.card_is_ham\n                        && let Some(sender) = message\n                            .from()\n                            .and_then(|s| s.first())\n                            .and_then(|s| s.address())\n                            .and_then(sanitize_email)\n                        && sender != deliver_to\n                        && is_sender_authenticated\n                        && self\n                            .document_exists(\n                                account_id,\n                                Collection::ContactCard,\n                                ContactField::Email,\n                                sender.as_bytes(),\n                            )\n                            .await\n                            .caused_by(trc::location!())?\n                    {\n                        is_spam = false;\n                        if self\n                            .core\n                            .spam\n                            .classifier\n                            .as_ref()\n                            .is_some_and(|c| c.auto_learn_card_is_ham)\n                        {\n                            train_spam = Some(false);\n                        }\n                        overridden = Some(\"card-exists\");\n                    }\n\n                    // Check if the message is a trusted reply to a previous message\n                    if is_spam\n                        && self.core.spam.trusted_reply\n                        && let Some(thread_id) = thread_result.thread_id\n                    {\n                        let cache = self\n                            .get_cached_messages(account_id)\n                            .await\n                            .caused_by(trc::location!())?;\n                        let sent_folder_id = cache\n                            .mailbox_by_role(&SpecialUse::Sent)\n                            .map(|m| m.document_id)\n                            .unwrap_or(SENT_ID);\n\n                        if cache\n                            .in_thread(thread_id)\n                            .any(|m| m.mailboxes.iter().any(|mb| mb.mailbox_id == sent_folder_id))\n                        {\n                            is_spam = false;\n                            if self\n                                .core\n                                .spam\n                                .classifier\n                                .as_ref()\n                                .is_some_and(|c| c.auto_learn_reply_ham)\n                            {\n                                train_spam = Some(false);\n                            }\n                            overridden = Some(\"trusted-reply\");\n                        }\n                    }\n\n                    // Add Spam-Status header\n                    const HEADER: &str = \"X-Spam-Status\";\n                    let offset_field = extra_headers.len();\n                    let offset_start = offset_field + HEADER.len() + 1;\n                    let result = if is_spam { \"Yes\" } else { \"No\" };\n                    if let Some(reason) = overridden {\n                        let _ = write!(\n                            &mut extra_headers,\n                            \"{HEADER}: {result}, reason={reason}\\r\\n\",\n                        );\n                    } else {\n                        let _ = write!(&mut extra_headers, \"{HEADER}: {result}\\r\\n\",);\n                    }\n\n                    extra_headers_parsed.push(Header {\n                        name: HeaderName::Other(HEADER.into()),\n                        value: HeaderValue::Text(\n                            extra_headers[offset_start + 1..extra_headers.len() - 2]\n                                .to_string()\n                                .into(),\n                        ),\n                        offset_field: offset_field as u32,\n                        offset_start: offset_start as u32,\n                        offset_end: extra_headers.len() as u32,\n                    });\n\n                    if is_spam && params.mailbox_ids == [INBOX_ID] {\n                        params.mailbox_ids[0] = JUNK_ID;\n                        params.keywords.push(Keyword::Junk);\n                    }\n                }\n\n                // iMIP processing\n                if self.core.groupware.itip_enabled\n                    && !is_spam\n                    && is_sender_authenticated\n                    && params\n                        .access_token\n                        .has_permission(Permission::CalendarSchedulingReceive)\n                {\n                    let mut sender = None;\n                    for part in &message.parts {\n                        if part.content_type().is_some_and(|ct| {\n                            ct.ctype().eq_ignore_ascii_case(\"text\")\n                                && ct\n                                    .subtype()\n                                    .is_some_and(|st| st.eq_ignore_ascii_case(\"calendar\"))\n                                && ct.has_attribute(\"method\")\n                        }) && let Some(itip_message) = part.text_contents()\n                        {\n                            if itip_message.len() < self.core.groupware.itip_inbound_max_ical_size {\n                                if let Some(sender) = sender.get_or_insert_with(|| {\n                                    message\n                                        .from()\n                                        .and_then(|s| s.first())\n                                        .and_then(|s| s.address())\n                                        .and_then(sanitize_email)\n                                }) {\n                                    match self\n                                        .itip_ingest(\n                                            params.access_token,\n                                            &resource_token,\n                                            sender,\n                                            deliver_to,\n                                            itip_message,\n                                        )\n                                        .await\n                                    {\n                                        Ok(message) => {\n                                            if let Some(message) = message {\n                                                itip_messages.push(message);\n                                            }\n                                            trc::event!(\n                                                Calendar(trc::CalendarEvent::ItipMessageReceived),\n                                                SpanId = params.session_id,\n                                                From = sender.to_string(),\n                                                AccountId = account_id,\n                                            );\n                                        }\n                                        Err(ItipIngestError::Message(itip_error)) => {\n                                            match itip_error {\n                                                ItipError::NothingToSend\n                                                | ItipError::OtherSchedulingAgent => (),\n                                                err => {\n                                                    trc::event!(\n                                                        Calendar(\n                                                            trc::CalendarEvent::ItipMessageError\n                                                        ),\n                                                        SpanId = params.session_id,\n                                                        From = sender.to_string(),\n                                                        AccountId = account_id,\n                                                        Details = err.to_string(),\n                                                    )\n                                                }\n                                            }\n                                        }\n                                        Err(ItipIngestError::Internal(err)) => {\n                                            trc::error!(err.caused_by(trc::location!()));\n                                        }\n                                    }\n                                }\n                            } else {\n                                trc::event!(\n                                    Calendar(trc::CalendarEvent::ItipMessageError),\n                                    SpanId = params.session_id,\n                                    From = message\n                                        .from()\n                                        .and_then(|a| a.first())\n                                        .and_then(|a| a.address())\n                                        .map(|a| a.to_string()),\n                                    AccountId = account_id,\n                                    Details = \"iMIP message too large\",\n                                    Limit = self.core.groupware.itip_inbound_max_ical_size,\n                                    Size = itip_message.len(),\n                                )\n                            }\n                        }\n                    }\n                }\n\n                is_spam\n            }\n            IngestSource::Jmap { train_classifier } | IngestSource::Imap { train_classifier } => {\n                // Determine spam training\n                if train_classifier && self.core.spam.enabled {\n                    if params.keywords.contains(&Keyword::Junk) {\n                        train_spam = Some(true);\n                    } else if params.keywords.contains(&Keyword::NotJunk) {\n                        if !params.mailbox_ids.contains(&TRASH_ID) {\n                            train_spam = Some(false);\n                        }\n                    } else if params.mailbox_ids[0] == JUNK_ID {\n                        train_spam = Some(true);\n                    } else if params.mailbox_ids[0] == INBOX_ID {\n                        train_spam = Some(false);\n                    }\n                }\n\n                // Set receivedAt if not present\n                if params.received_at.is_none() {\n                    params.received_at = message\n                        .root_part()\n                        .headers()\n                        .iter()\n                        .filter_map(|header| {\n                            if let (HeaderName::Received, HeaderValue::Received(received)) =\n                                (&header.name, &header.value)\n                            {\n                                received.date.map(|dt| dt.to_timestamp() as u64)\n                            } else {\n                                None\n                            }\n                        })\n                        .max();\n                }\n\n                false\n            }\n            _ => false,\n        };\n\n        // Encrypt message\n        let do_encrypt = match params.source {\n            IngestSource::Jmap { .. } | IngestSource::Imap { .. } => {\n                self.core.jmap.encrypt && self.core.jmap.encrypt_append\n            }\n            IngestSource::Smtp { .. } => self.core.jmap.encrypt,\n            IngestSource::Restore => false,\n        };\n        let is_encrypted = if do_encrypt\n            && !message.is_encrypted()\n            && let Some(encrypt_params_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                    account_id,\n                    Collection::Principal,\n                    0,\n                    PrincipalField::EncryptionKeys,\n                ))\n                .await\n                .caused_by(trc::location!())?\n        {\n            let encrypt_params = encrypt_params_\n                .unarchive::<EncryptionParams>()\n                .caused_by(trc::location!())?;\n            match message.encrypt(encrypt_params).await {\n                Ok(new_raw_message) => {\n                    raw_message = Cow::from(new_raw_message);\n                    raw_message_len = raw_message.len() as u64;\n                    message = MessageParser::default()\n                        .parse(raw_message.as_ref())\n                        .ok_or_else(|| {\n                            trc::EventType::MessageIngest(trc::MessageIngestEvent::Error)\n                                .ctx(trc::Key::Code, 550)\n                                .ctx(\n                                    trc::Key::Reason,\n                                    \"Failed to parse encrypted e-mail message.\",\n                                )\n                        })?;\n\n                    // Disable spam training if requested\n                    if !encrypt_params.can_train_spam_filter() {\n                        train_spam = None;\n                    }\n\n                    // Remove contents from parsed message\n                    for part in &mut message.parts {\n                        match &mut part.body {\n                            PartType::Text(txt) | PartType::Html(txt) => {\n                                *txt = Cow::from(\"\");\n                            }\n                            PartType::Binary(bin) | PartType::InlineBinary(bin) => {\n                                *bin = Cow::from(&[][..]);\n                            }\n                            PartType::Message(_) => {\n                                part.body = PartType::Binary(Cow::from(&[][..]));\n                            }\n                            PartType::Multipart(_) => (),\n                        }\n                    }\n\n                    true\n                }\n                Err(EncryptMessageError::Error(err)) => {\n                    trc::bail!(\n                        trc::StoreEvent::CryptoError\n                            .into_err()\n                            .caused_by(trc::location!())\n                            .reason(err)\n                    );\n                }\n                _ => unreachable!(),\n            }\n        } else {\n            false\n        };\n\n        // Store blob\n        let (blob_hash, blob_hold) = if !is_encrypted && let Some(blob_hash) = params.blob_hash {\n            (blob_hash.clone(), None)\n        } else {\n            self.put_temporary_blob(account_id, raw_message.as_ref(), 60)\n                .await\n                .map(|(hash, op)| (hash, Some(op)))\n                .caused_by(trc::location!())?\n        };\n\n        // Assign IMAP UIDs\n        let mut mailbox_ids = Vec::with_capacity(params.mailbox_ids.len());\n        let mut imap_uids = Vec::with_capacity(params.mailbox_ids.len());\n        let mut ids = self\n            .assign_email_ids(account_id, params.mailbox_ids.iter().copied(), true)\n            .await\n            .caused_by(trc::location!())?;\n        let document_id = ids.next().unwrap();\n        for (uid, mailbox_id) in ids.zip(params.mailbox_ids.iter().copied()) {\n            mailbox_ids.push(UidMailbox::new(mailbox_id, uid));\n            imap_uids.push(uid);\n        }\n\n        // Build write batch\n        let mut batch = BatchBuilder::new();\n        let mailbox_ids_event = mailbox_ids\n            .iter()\n            .map(|m| trc::Value::from(m.mailbox_id))\n            .collect::<Vec<_>>();\n        batch.with_account_id(account_id);\n\n        // Determine thread id\n        let thread_id = if let Some(thread_id) = thread_result.thread_id {\n            thread_id\n        } else {\n            batch\n                .with_collection(Collection::Thread)\n                .with_document(document_id)\n                .log_container_insert(SyncCollection::Thread);\n            document_id\n        };\n\n        let data = MessageData {\n            mailboxes: mailbox_ids.into_boxed_slice(),\n            keywords: params.keywords.into_boxed_slice(),\n            thread_id,\n            size: (message.raw_message.len() + extra_headers.len()) as u32,\n        };\n\n        batch\n            .with_collection(Collection::Email)\n            .with_document(document_id)\n            .index_message(\n                tenant_id,\n                message,\n                extra_headers.into_bytes(),\n                extra_headers_parsed,\n                blob_hash.clone(),\n                data,\n                params.received_at.unwrap_or_else(now),\n            )\n            .caused_by(trc::location!())?\n            .set(\n                ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                    property: EmailField::Threading.into(),\n                    hash: thread_result.thread_hash,\n                }),\n                ThreadInfo::serialize(thread_id, &message_ids),\n            )\n            .set(\n                ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                    index: SearchIndex::Email,\n                    due: TaskEpoch::now(),\n                    is_insert: true,\n                }),\n                vec![],\n            );\n\n        if let Some(blob_hold) = blob_hold {\n            batch.clear(blob_hold);\n        }\n\n        // Merge threads if necessary\n        if let Some(merge_threads) = MergeThreadIds::new(thread_result).serialize() {\n            batch.set(\n                ValueClass::TaskQueue(TaskQueueClass::MergeThreads {\n                    due: TaskEpoch::now(),\n                }),\n                merge_threads,\n            );\n        }\n\n        // Request spam training\n        if let Some(learn_spam) = train_spam {\n            self.add_spam_sample(\n                &mut batch,\n                params.blob_hash.unwrap_or(&blob_hash).clone(),\n                learn_spam,\n                !is_encrypted,\n                params.session_id,\n            );\n        }\n\n        // Add iTIP responses to batch\n        if !itip_messages.is_empty() {\n            ItipMessages::new(itip_messages)\n                .queue(&mut batch)\n                .caused_by(trc::location!())?;\n        }\n\n        // Insert and obtain ids\n        let change_id = self\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?\n            .last_change_id(account_id)?;\n\n        // Request FTS index\n        self.notify_task_queue();\n\n        trc::event!(\n            MessageIngest(match params.source {\n                IngestSource::Smtp { .. } =>\n                    if !is_spam {\n                        MessageIngestEvent::Ham\n                    } else {\n                        MessageIngestEvent::Spam\n                    },\n                IngestSource::Jmap { .. } | IngestSource::Restore => MessageIngestEvent::JmapAppend,\n                IngestSource::Imap { .. } => MessageIngestEvent::ImapAppend,\n            }),\n            SpanId = params.session_id,\n            AccountId = account_id,\n            DocumentId = document_id,\n            MailboxId = mailbox_ids_event,\n            BlobId = blob_hash.to_hex(),\n            ChangeId = change_id,\n            MessageId = message_id,\n            Size = raw_message_len,\n            Elapsed = start_time.elapsed(),\n        );\n\n        Ok(IngestedEmail {\n            document_id,\n            thread_id,\n            change_id,\n            blob_id: BlobId {\n                hash: blob_hash,\n                class: BlobClass::Linked {\n                    account_id,\n                    collection: Collection::Email.into(),\n                    document_id,\n                },\n                section: None,\n            },\n            size: raw_message_len as usize,\n            imap_uids,\n        })\n    }\n\n    async fn find_thread_id(\n        &self,\n        account_id: u32,\n        thread_name: &str,\n        message_ids: &[CheekyHash],\n    ) -> trc::Result<ThreadResult> {\n        let mut result = ThreadResult {\n            thread_id: None,\n            thread_hash: CheekyHash::new(if !thread_name.is_empty() {\n                thread_name\n            } else {\n                \"!\"\n            }),\n            merge_ids: vec![],\n            duplicate_ids: vec![],\n        };\n\n        if message_ids.is_empty() {\n            return Ok(result);\n        }\n\n        // Find thread ids\n        let key_len = IndexKeyPrefix::len() + result.thread_hash.len() + U32_LEN;\n        let document_id_pos = key_len - U32_LEN;\n        let mut thread_merge = ThreadMerge::new();\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                            property: EmailField::Threading.into(),\n                            hash: result.thread_hash,\n                        }),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id: u32::MAX,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                            property: EmailField::Threading.into(),\n                            hash: result.thread_hash,\n                        }),\n                    },\n                )\n                .ascending(),\n                |key, value| {\n                    if key.len() == key_len {\n                        // Find matching references\n                        let references = value.get(U32_LEN..).unwrap_or_default();\n\n                        if has_message_id(message_ids, references) {\n                            let document_id = key.deserialize_be_u32(document_id_pos)?;\n                            let thread_id = value.deserialize_be_u32(0)?;\n\n                            if message_ids.len() == 1\n                                || (message_ids.len() == references.len() / CheekyHash::HASH_SIZE\n                                    && references\n                                        .chunks_exact(CheekyHash::HASH_SIZE)\n                                        .zip(message_ids.iter())\n                                        .all(|(a, b)| a == b.as_raw_bytes()))\n                            {\n                                result.duplicate_ids.push(document_id);\n                            }\n\n                            thread_merge.add(thread_id, document_id);\n                        }\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        match thread_merge.num_thread_ids() {\n            0 => Ok(result),\n            1 => {\n                // Happy path, only one thread id\n                result.thread_id = thread_merge.thread_ids().next().copied();\n                Ok(result)\n            }\n            _ => {\n                // Multiple thread ids that this message belongs to, merge them\n                let thread_merge = thread_merge.merge();\n                result.merge_ids = thread_merge.merge_ids;\n                result.thread_id = Some(thread_merge.thread_id);\n                Ok(result)\n            }\n        }\n    }\n\n    async fn assign_email_ids(\n        &self,\n        account_id: u32,\n        mailbox_ids: impl IntoIterator<Item = u32> + Sync + Send,\n        generate_email_id: bool,\n    ) -> trc::Result<impl Iterator<Item = u32> + 'static> {\n        // Increment UID next\n        let mut batch = BatchBuilder::new();\n        batch.with_account_id(account_id);\n\n        let mut expected_ids = 0;\n        if generate_email_id {\n            batch\n                .with_collection(Collection::Email)\n                .add_and_get(ValueClass::DocumentId, 1);\n            expected_ids += 1;\n        }\n\n        batch.with_collection(Collection::Mailbox);\n\n        for mailbox_id in mailbox_ids {\n            batch\n                .with_document(mailbox_id)\n                .add_and_get(MailboxField::UidCounter, 1);\n            expected_ids += 1;\n        }\n\n        let ids = if expected_ids > 0 {\n            self.core.storage.data.write(batch.build_all()).await?\n        } else {\n            AssignedIds::default()\n        };\n        if ids.ids.len() == expected_ids {\n            Ok(ids.ids.into_iter().map(|id| match id {\n                AssignedId::Counter(id) => id as u32,\n                AssignedId::ChangeId(_) => unreachable!(),\n            }))\n        } else {\n            Err(trc::StoreEvent::UnexpectedError\n                .caused_by(trc::location!())\n                .ctx(trc::Key::Reason, \"No all document ids were generated\"))\n        }\n    }\n\n    async fn add_account_spam_sample(\n        &self,\n        batch: &mut BatchBuilder,\n        account_id: u32,\n        document_id: u32,\n        is_spam: bool,\n        span_id: u64,\n    ) -> trc::Result<()> {\n        if self.core.spam.classifier.is_some()\n            && let Some(archive) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                    account_id,\n                    Collection::Email,\n                    document_id,\n                    EmailField::Metadata,\n                ))\n                .await\n                .caused_by(trc::location!())?\n        {\n            let metadata = archive\n                .to_unarchived::<MessageMetadata>()\n                .caused_by(trc::location!())?;\n            self.add_spam_sample(\n                batch,\n                (&metadata.inner.blob_hash).into(),\n                is_spam,\n                true,\n                span_id,\n            );\n        }\n\n        Ok(())\n    }\n\n    fn add_spam_sample(\n        &self,\n        batch: &mut BatchBuilder,\n        hash: BlobHash,\n        is_spam: bool,\n        hold_sample: bool,\n        span_id: u64,\n    ) {\n        if let Some(config) = &self.core.spam.classifier {\n            let mut dt = DateTime::from_timestamp(now() as i64);\n            dt.hour = 0;\n            dt.minute = 0;\n            dt.second = 0;\n            let until = dt.to_timestamp() as u64 + config.hold_samples_for;\n\n            batch\n                .set(\n                    BlobOp::Link {\n                        hash: hash.clone(),\n                        to: BlobLink::Temporary { until },\n                    },\n                    vec![BlobLink::SPAM_SAMPLE_LINK],\n                )\n                .set(\n                    BlobOp::SpamSample { hash, until },\n                    vec![u8::from(is_spam), u8::from(hold_sample)],\n                );\n\n            trc::event!(\n                Spam(SpamEvent::TrainSampleAdded),\n                AccountId = batch.last_account_id(),\n                Details = if is_spam { \"spam\" } else { \"ham\" },\n                Expires = trc::Value::Timestamp(until),\n                SpanId = span_id,\n            );\n        }\n    }\n}\n\nfn has_message_id(a: &[CheekyHash], b: &[u8]) -> bool {\n    let mut i = 0;\n    let mut j = 0;\n\n    let a_len = a.len();\n    let b_len = b.len() / CheekyHash::HASH_SIZE;\n\n    while i < a_len && j < b_len {\n        match a[i]\n            .as_raw_bytes()\n            .as_slice()\n            .cmp(&b[j * CheekyHash::HASH_SIZE..(j + 1) * CheekyHash::HASH_SIZE])\n        {\n            std::cmp::Ordering::Equal => return true,\n            std::cmp::Ordering::Less => i += 1,\n            std::cmp::Ordering::Greater => j += 1,\n        }\n    }\n\n    false\n}\n\nimpl IngestSource<'_> {\n    pub fn is_smtp(&self) -> bool {\n        matches!(self, Self::Smtp { .. })\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct MergeThreadIds<T> {\n    pub thread_hash: CheekyHash,\n    pub merge_ids: T,\n}\n\nimpl MergeThreadIds<Vec<u32>> {\n    pub(crate) fn new(thread_result: ThreadResult) -> Self {\n        Self {\n            thread_hash: thread_result.thread_hash,\n            merge_ids: thread_result.merge_ids,\n        }\n    }\n\n    pub(crate) fn serialize(&self) -> Option<Vec<u8>> {\n        if !self.merge_ids.is_empty() {\n            let mut buf =\n                Vec::with_capacity(self.thread_hash.len() + self.merge_ids.len() * U32_LEN);\n            buf.extend_from_slice(self.thread_hash.as_bytes());\n            for id in &self.merge_ids {\n                buf.extend_from_slice(&id.to_be_bytes());\n            }\n            Some(buf)\n        } else {\n            None\n        }\n    }\n}\n\nimpl MergeThreadIds<AHashSet<u32>> {\n    pub fn deserialize(bytes: &[u8]) -> Option<Self> {\n        if !bytes.is_empty() {\n            let thread_hash = CheekyHash::deserialize(bytes)?;\n            let mut merge_ids =\n                AHashSet::with_capacity(((bytes.len() - thread_hash.len()) / U32_LEN) + 1);\n            let mut start_offset = thread_hash.len();\n\n            while let Some(id_bytes) = bytes.get(start_offset..start_offset + U32_LEN) {\n                merge_ids.insert(u32::from_be_bytes(id_bytes.try_into().ok()?));\n                start_offset += U32_LEN;\n            }\n\n            Some(Self {\n                thread_hash,\n                merge_ids,\n            })\n        } else {\n            None\n        }\n    }\n}\n\nimpl std::hash::Hash for MergeThreadIds<AHashSet<u32>> {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.thread_hash.hash(state);\n        self.merge_ids.len().hash(state);\n    }\n}\n\npub struct ThreadInfo;\n\nimpl ThreadInfo {\n    pub fn serialize(thread_id: u32, ref_ids: &[CheekyHash]) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(U32_LEN + 1 + ref_ids.len() * CheekyHash::HASH_SIZE);\n        buf.extend_from_slice(&thread_id.to_be_bytes());\n        for ref_id in ref_ids {\n            buf.extend_from_slice(ref_id.as_raw_bytes());\n        }\n        buf\n    }\n}\n\npub struct ThreadMerge {\n    entries: AHashMap<u32, Vec<u32>>,\n}\n\npub struct ThreadMergeResult {\n    pub thread_id: u32,\n    pub merge_ids: Vec<u32>,\n}\n\nimpl ThreadMerge {\n    #[allow(clippy::new_without_default)]\n    pub fn new() -> Self {\n        Self {\n            entries: AHashMap::with_capacity(8),\n        }\n    }\n\n    pub fn add(&mut self, thread_id: u32, document_id: u32) {\n        self.entries.entry(thread_id).or_default().push(document_id);\n    }\n\n    pub fn num_thread_ids(&self) -> usize {\n        self.entries.len()\n    }\n\n    pub fn thread_ids(&self) -> impl Iterator<Item = &u32> {\n        self.entries.keys()\n    }\n\n    pub fn thread_groups(&self) -> impl Iterator<Item = (&u32, &Vec<u32>)> {\n        self.entries.iter()\n    }\n\n    pub fn merge_thread_id(&self) -> u32 {\n        let mut max_thread_id = u32::MAX;\n        let mut max_count = 0;\n\n        for (thread_id, ids) in &self.entries {\n            match ids.len().cmp(&max_count) {\n                Ordering::Greater => {\n                    max_count = ids.len();\n                    max_thread_id = *thread_id;\n                }\n                Ordering::Equal => {\n                    if *thread_id < max_thread_id {\n                        max_thread_id = *thread_id;\n                    }\n                }\n                Ordering::Less => (),\n            }\n        }\n\n        max_thread_id\n    }\n\n    pub fn merge(self) -> ThreadMergeResult {\n        let mut max_thread_id = u32::MAX;\n        let mut max_count = 0;\n        let mut merge_ids = Vec::with_capacity(self.entries.len());\n\n        for (thread_id, ids) in self.entries {\n            match ids.len().cmp(&max_count) {\n                Ordering::Greater => {\n                    max_count = ids.len();\n                    max_thread_id = thread_id;\n                }\n                Ordering::Equal => {\n                    if thread_id < max_thread_id {\n                        max_thread_id = thread_id;\n                    }\n                }\n                Ordering::Less => (),\n            }\n            merge_ids.push(thread_id);\n        }\n\n        ThreadMergeResult {\n            thread_id: max_thread_id,\n            merge_ids,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/metadata.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::mailbox::{ArchivedUidMailbox, UidMailbox};\nuse common::storage::index::IndexableAndSerializableObject;\nuse mail_parser::{\n    Addr, Address, Attribute, ContentType, DateTime, Encoding, Group, HeaderName, HeaderValue,\n    PartType,\n    decoders::{\n        base64::base64_decode, charsets::map::charset_decoder,\n        quoted_printable::quoted_printable_decode,\n    },\n};\nuse rkyv::{boxed::ArchivedBox, rend::u16_le};\nuse std::{borrow::Cow, collections::VecDeque, ops::Range};\nuse types::{\n    blob_hash::BlobHash,\n    keyword::{ArchivedKeyword, Keyword},\n};\nuse utils::chained_bytes::ChainedBytes;\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct MessageData {\n    pub mailboxes: Box<[UidMailbox]>,\n    pub keywords: Box<[Keyword]>,\n    pub thread_id: u32,\n    pub size: u32,\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct MessageMetadata {\n    pub contents: Box<[MessageMetadataContents]>,\n    pub rcvd_attach: u64,\n    pub blob_hash: BlobHash,\n    pub blob_body_offset: u32,\n    pub preview: Box<str>,\n    pub raw_headers: Box<[u8]>,\n}\n\npub const MESSAGE_HAS_ATTACHMENT: u64 = 1 << 63;\npub const MESSAGE_RECEIVED_MASK: u64 = !MESSAGE_HAS_ATTACHMENT;\n\nimpl IndexableAndSerializableObject for MessageData {\n    fn is_versioned() -> bool {\n        true\n    }\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct MessageMetadataContents {\n    pub html_body: Box<[u16]>,\n    pub text_body: Box<[u16]>,\n    pub attachments: Box<[u16]>,\n    pub parts: Box<[MessageMetadataPart]>,\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct MessageMetadataPart {\n    pub headers: Box<[MetadataHeader]>,\n    pub body: MetadataPartType,\n    pub flags: u32,\n    pub offset_header: u32,\n    pub offset_body: u32,\n    pub offset_end: u32,\n}\n\npub const PART_ENCODING_BASE64: u32 = 1 << 31;\npub const PART_ENCODING_QP: u32 = 1 << 30;\npub const PART_ENCODING_PROBLEM: u32 = 1 << 29;\npub const PART_SIZE_MASK: u32 = !(PART_ENCODING_BASE64 | PART_ENCODING_QP | PART_ENCODING_PROBLEM);\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub struct MetadataHeader {\n    pub name: MetadataHeaderName,\n    pub value: MetadataHeaderValue,\n    pub base_offset: u32,\n    pub start: u16,\n    pub end: u16,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub enum MetadataHeaderName {\n    Other(Box<str>),\n    Subject,\n    From,\n    To,\n    Cc,\n    Date,\n    Bcc,\n    ReplyTo,\n    Sender,\n    Comments,\n    InReplyTo,\n    Keywords,\n    Received,\n    MessageId,\n    References,\n    ReturnPath,\n    MimeVersion,\n    ContentDescription,\n    ContentId,\n    ContentLanguage,\n    ContentLocation,\n    ContentTransferEncoding,\n    ContentType,\n    ContentDisposition,\n    ResentTo,\n    ResentFrom,\n    ResentBcc,\n    ResentCc,\n    ResentSender,\n    ResentDate,\n    ResentMessageId,\n    ListArchive,\n    ListHelp,\n    ListId,\n    ListOwner,\n    ListPost,\n    ListSubscribe,\n    ListUnsubscribe,\n    DkimSignature,\n    ArcAuthenticationResults,\n    ArcMessageSignature,\n    ArcSeal,\n\n    // Delivery/Routing\n    DeliveredTo,\n    XOriginalTo,\n    ReturnReceiptTo,\n    DispositionNotificationTo,\n    ErrorsTo,\n\n    // Authentication\n    AuthenticationResults,\n    ReceivedSpf,\n\n    // Spam/Virus\n    XSpamStatus,\n    XSpamScore,\n    XSpamFlag,\n    XSpamResult,\n\n    // Priority\n    Importance,\n    Priority,\n    XPriority,\n    XMSMailPriority,\n\n    // Client/Agent\n    XMailer,\n    UserAgent,\n    XMimeOLE,\n\n    // Network/Origin\n    XOriginatingIp,\n    XForwardedTo,\n    XForwardedFor,\n\n    // Auto-response\n    AutoSubmitted,\n    XAutoResponseSuppress,\n    Precedence,\n\n    // Organization/Threading\n    Organization,\n    ThreadIndex,\n    ThreadTopic,\n\n    // List (additional)\n    ListUnsubscribePost,\n    FeedbackId,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub enum MetadataHeaderValue {\n    AddressList(Box<[MetadataAddress]>),\n    AddressGroup(Box<[MetadataAddressGroup]>),\n    Text(Box<str>),\n    TextList(Box<[Box<str>]>),\n    DateTime(MetadataDateTime),\n    ContentType(MetadataContentType),\n    Empty,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub struct MetadataDateTime {\n    pub year: u16,\n    pub month: u8,\n    pub day: u8,\n    pub hour: u8,\n    pub minute: u8,\n    pub second: u8,\n    pub tz_hour: i8,\n    pub tz_minute: u8,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub struct MetadataAddress {\n    pub name: Option<Box<str>>,\n    pub address: Option<Box<str>>,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub struct MetadataAddressGroup {\n    pub name: Option<Box<str>>,\n    pub addresses: Box<[MetadataAddress]>,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub struct MetadataContentType {\n    pub c_type: Box<str>,\n    pub c_subtype: Option<Box<str>>,\n    pub attributes: Box<[MetadataAttribute]>,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\n#[rkyv(compare(PartialEq))]\npub struct MetadataAttribute {\n    pub name: Box<str>,\n    pub value: Box<str>,\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub enum MetadataPartType {\n    Text,\n    Html,\n    Binary,\n    InlineBinary,\n    Message(u16),\n    Multipart(Box<[u16]>),\n}\n\nimpl MessageMetadataContents {\n    pub fn root_part(&self) -> &MessageMetadataPart {\n        &self.parts[0]\n    }\n}\n\n#[derive(Debug)]\npub struct DecodedParts<'x> {\n    pub raw_messages: Vec<DecodedRawMessage<'x>>,\n    pub parts: Vec<DecodedPart<'x>>,\n}\n\n#[derive(Debug)]\npub enum DecodedRawMessage<'x> {\n    Borrowed(ChainedBytes<'x>),\n    Owned(Vec<u8>),\n}\n\n#[derive(Debug)]\npub struct DecodedPart<'x> {\n    pub message_id: usize,\n    pub part_offset: usize,\n    pub content: DecodedPartContent<'x>,\n}\n\n#[derive(Debug)]\npub enum DecodedPartContent<'x> {\n    Text(Cow<'x, str>),\n    Binary(Cow<'x, [u8]>),\n}\n\nimpl<'x> DecodedParts<'x> {\n    #[inline]\n    pub fn raw_message(&self, message_id: usize) -> Option<&DecodedRawMessage<'x>> {\n        self.raw_messages.get(message_id)\n    }\n\n    #[inline]\n    pub fn raw_message_section(\n        &'_ self,\n        message_id: usize,\n        range: Range<usize>,\n    ) -> Option<Cow<'_, [u8]>> {\n        self.raw_messages.get(message_id).and_then(|m| m.get(range))\n    }\n\n    #[inline]\n    pub fn part(&self, message_id: usize, part_offset: usize) -> Option<&DecodedPartContent<'x>> {\n        self.parts\n            .iter()\n            .find(|p| p.message_id == message_id && p.part_offset == part_offset)\n            .map(|p| &p.content)\n    }\n\n    #[inline]\n    pub fn text_part(&self, message_id: usize, part_offset: usize) -> Option<&str> {\n        self.part(message_id, part_offset).and_then(|p| match p {\n            DecodedPartContent::Text(text) => Some(text.as_ref()),\n            DecodedPartContent::Binary(_) => None,\n        })\n    }\n\n    #[inline]\n    pub fn binary_part(&self, message_id: usize, part_offset: usize) -> Option<&[u8]> {\n        self.part(message_id, part_offset).map(|p| match p {\n            DecodedPartContent::Text(part) => part.as_bytes(),\n            DecodedPartContent::Binary(binary) => binary.as_ref(),\n        })\n    }\n}\n\nimpl DecodedPartContent<'_> {\n    pub fn as_bytes(&self) -> &[u8] {\n        match self {\n            DecodedPartContent::Text(text) => text.as_bytes(),\n            DecodedPartContent::Binary(binary) => binary,\n        }\n    }\n\n    #[allow(clippy::len_without_is_empty)]\n    pub fn len(&self) -> usize {\n        match self {\n            DecodedPartContent::Text(text) => text.len(),\n            DecodedPartContent::Binary(binary) => binary.len(),\n        }\n    }\n\n    pub fn as_str(&self) -> &str {\n        match self {\n            DecodedPartContent::Text(text) => text,\n            DecodedPartContent::Binary(binary) => std::str::from_utf8(binary).unwrap_or_default(),\n        }\n    }\n}\n\nimpl<'x> DecodedRawMessage<'x> {\n    pub fn get(&'_ self, index: Range<usize>) -> Option<Cow<'_, [u8]>> {\n        match self {\n            DecodedRawMessage::Borrowed(bytes) => bytes.get(index),\n            DecodedRawMessage::Owned(vec) => vec.get(index).map(Cow::Borrowed),\n        }\n    }\n}\n\nimpl ArchivedMessageMetadata {\n    #[inline(always)]\n    pub fn message_id(&self, message_id: u16_le) -> &ArchivedMessageMetadataContents {\n        &self.contents[u16::from(message_id) as usize]\n    }\n\n    pub fn decode_contents<'x>(&self, raw: ChainedBytes<'x>) -> DecodedParts<'x> {\n        let mut result = DecodedParts {\n            raw_messages: Vec::with_capacity(self.contents.len()),\n            parts: Vec::new(),\n        };\n\n        for _ in 0..self.contents.len() {\n            result\n                .raw_messages\n                .push(DecodedRawMessage::Borrowed(raw.clone()));\n        }\n\n        for (message_id, contents) in self.contents.iter().enumerate() {\n            for part in contents.parts.iter() {\n                let part_offset = u32::from(part.offset_header) as usize;\n                match &part.body {\n                    ArchivedMetadataPartType::Text\n                    | ArchivedMetadataPartType::Html\n                    | ArchivedMetadataPartType::Binary\n                    | ArchivedMetadataPartType::InlineBinary => {\n                        match result.raw_messages.get(message_id).unwrap() {\n                            DecodedRawMessage::Borrowed(bytes) => {\n                                result.parts.push(DecodedPart {\n                                    message_id,\n                                    part_offset,\n                                    content: part.decode_contents(bytes),\n                                });\n                            }\n                            DecodedRawMessage::Owned(bytes) => {\n                                result.parts.push(DecodedPart {\n                                    message_id,\n                                    part_offset,\n                                    content: match part.decode_contents(&ChainedBytes::new(bytes)) {\n                                        DecodedPartContent::Text(text) => {\n                                            DecodedPartContent::Text(text.into_owned().into())\n                                        }\n                                        DecodedPartContent::Binary(binary) => {\n                                            DecodedPartContent::Binary(binary.into_owned().into())\n                                        }\n                                    },\n                                });\n                            }\n                        }\n                    }\n                    ArchivedMetadataPartType::Message(nested_message_id) => {\n                        let sub_contents =\n                            if (part.flags & (PART_ENCODING_BASE64 | PART_ENCODING_QP)) != 0 {\n                                match result.raw_messages.get(message_id).unwrap() {\n                                    DecodedRawMessage::Borrowed(bytes) => {\n                                        part.contents(bytes).into_owned()\n                                    }\n                                    DecodedRawMessage::Owned(bytes) => {\n                                        let bytes = ChainedBytes::new(bytes);\n                                        part.contents(&bytes).into_owned()\n                                    }\n                                }\n                            } else if let Some(DecodedRawMessage::Owned(bytes)) =\n                                result.raw_messages.get(message_id)\n                            {\n                                bytes.clone()\n                            } else {\n                                continue;\n                            };\n\n                        result.raw_messages[usize::from(*nested_message_id)] =\n                            DecodedRawMessage::Owned(sub_contents);\n                    }\n                    _ => {}\n                }\n            }\n        }\n\n        result\n    }\n}\n\nimpl ArchivedMessageMetadataPart {\n    pub fn contents<'x>(&self, raw_message: &ChainedBytes<'x>) -> Cow<'x, [u8]> {\n        let bytes = raw_message.get(self.body_to_end()).unwrap_or_default();\n\n        if (self.flags & PART_ENCODING_BASE64) != 0 {\n            base64_decode(bytes.as_ref()).unwrap_or_default().into()\n        } else if (self.flags & PART_ENCODING_QP) != 0 {\n            quoted_printable_decode(bytes.as_ref())\n                .unwrap_or_default()\n                .into()\n        } else {\n            bytes\n        }\n    }\n\n    #[inline(always)]\n    pub fn body_to_end(&self) -> Range<usize> {\n        (self.offset_body.to_native() as usize)..(self.offset_end.to_native() as usize)\n    }\n\n    #[inline(always)]\n    pub fn header_to_end(&self) -> Range<usize> {\n        self.offset_header.to_native() as usize..self.offset_end.to_native() as usize\n    }\n\n    #[inline(always)]\n    pub fn header_to_body(&self) -> Range<usize> {\n        self.offset_header.to_native() as usize..self.offset_body.to_native() as usize\n    }\n\n    pub fn decode_contents<'x>(&self, raw_message: &ChainedBytes<'x>) -> DecodedPartContent<'x> {\n        let bytes = self.contents(raw_message);\n\n        match self.body {\n            ArchivedMetadataPartType::Text | ArchivedMetadataPartType::Html => {\n                DecodedPartContent::Text(\n                    match (\n                        bytes,\n                        self.header_value(&MetadataHeaderName::ContentType)\n                            .and_then(|c| c.as_content_type())\n                            .and_then(|ct| {\n                                ct.attribute(\"charset\")\n                                    .and_then(|c| charset_decoder(c.as_bytes()))\n                            }),\n                    ) {\n                        (Cow::Owned(vec), Some(charset_decoder)) => charset_decoder(&vec).into(),\n                        (Cow::Owned(vec), None) => String::from_utf8(vec)\n                            .unwrap_or_else(|e| String::from_utf8_lossy(e.as_bytes()).into_owned())\n                            .into(),\n                        (Cow::Borrowed(bytes), Some(charset_decoder)) => {\n                            charset_decoder(bytes).into()\n                        }\n                        (Cow::Borrowed(bytes), None) => String::from_utf8_lossy(bytes),\n                    },\n                )\n            }\n            ArchivedMetadataPartType::Binary => DecodedPartContent::Binary(bytes),\n            ArchivedMetadataPartType::InlineBinary => DecodedPartContent::Binary(bytes),\n            ArchivedMetadataPartType::Message(_) | ArchivedMetadataPartType::Multipart(_) => {\n                unreachable!()\n            }\n        }\n    }\n}\n\npub fn build_metadata_contents(\n    message: mail_parser::Message<'_>,\n) -> Box<[MessageMetadataContents]> {\n    let mut messages = VecDeque::from([message]);\n    let mut message_id = 0;\n    let mut contents = Vec::new();\n\n    while let Some(message) = messages.pop_front() {\n        let mut parts = Vec::with_capacity(message.parts.len());\n\n        for part in message.parts {\n            let (size, body) = match part.body {\n                PartType::Text(contents) => (contents.len(), MetadataPartType::Text),\n                PartType::Html(contents) => (contents.len(), MetadataPartType::Html),\n                PartType::Binary(contents) => (contents.len(), MetadataPartType::Binary),\n                PartType::InlineBinary(contents) => {\n                    (contents.len(), MetadataPartType::InlineBinary)\n                }\n                PartType::Message(message) => {\n                    let message_len = message.root_part().raw_len();\n                    messages.push_back(message);\n                    message_id += 1;\n\n                    (message_len as usize, MetadataPartType::Message(message_id))\n                }\n                PartType::Multipart(parts) => (\n                    0,\n                    MetadataPartType::Multipart(parts.into_iter().map(|p| p as u16).collect()),\n                ),\n            };\n\n            let flags = match part.encoding {\n                Encoding::None => 0,\n                Encoding::QuotedPrintable => PART_ENCODING_QP,\n                Encoding::Base64 => PART_ENCODING_BASE64,\n            } | (if part.is_encoding_problem {\n                PART_ENCODING_PROBLEM\n            } else {\n                0\n            }) | (size as u32 & PART_SIZE_MASK);\n\n            parts.push(MessageMetadataPart {\n                headers: part\n                    .headers\n                    .into_iter()\n                    .map(|hdr| MetadataHeader {\n                        value: if matches!(\n                            &hdr.name,\n                            HeaderName::Subject\n                                | HeaderName::From\n                                | HeaderName::To\n                                | HeaderName::Cc\n                                | HeaderName::Date\n                                | HeaderName::Bcc\n                                | HeaderName::ReplyTo\n                                | HeaderName::Sender\n                                | HeaderName::Comments\n                                | HeaderName::InReplyTo\n                                | HeaderName::Keywords\n                                | HeaderName::MessageId\n                                | HeaderName::References\n                                | HeaderName::ResentMessageId\n                                | HeaderName::ContentDescription\n                                | HeaderName::ContentId\n                                | HeaderName::ContentLanguage\n                                | HeaderName::ContentLocation\n                                | HeaderName::ContentTransferEncoding\n                                | HeaderName::ContentType\n                                | HeaderName::ContentDisposition\n                                | HeaderName::ListId\n                        ) {\n                            hdr.value\n                        } else {\n                            HeaderValue::Empty\n                        }\n                        .into(),\n                        name: hdr.name.into(),\n                        base_offset: hdr.offset_field,\n                        start: (hdr.offset_start - hdr.offset_field) as u16,\n                        end: (hdr.offset_end - hdr.offset_field) as u16,\n                    })\n                    .collect(),\n                body,\n                flags,\n                offset_header: part.offset_header,\n                offset_body: part.offset_body,\n                offset_end: part.offset_end,\n            });\n        }\n        contents.push(MessageMetadataContents {\n            html_body: message.html_body.into_iter().map(|c| c as u16).collect(),\n            text_body: message.text_body.into_iter().map(|c| c as u16).collect(),\n            attachments: message.attachments.into_iter().map(|c| c as u16).collect(),\n            parts: parts.into_boxed_slice(),\n        });\n    }\n    contents.into_boxed_slice()\n}\n\nimpl ArchivedMessageMetadataPart {\n    pub fn is_message(&self) -> bool {\n        matches!(self.body, ArchivedMetadataPartType::Message(_))\n    }\n\n    pub fn sub_parts(&self) -> Option<&ArchivedBox<[u16_le]>> {\n        if let ArchivedMetadataPartType::Multipart(parts) = &self.body {\n            Some(parts)\n        } else {\n            None\n        }\n    }\n\n    pub fn raw_len(&self) -> usize {\n        (u32::from(self.offset_end)).saturating_sub(u32::from(self.offset_header)) as usize\n    }\n\n    pub fn header_values(\n        &self,\n        name: &MetadataHeaderName,\n    ) -> impl Iterator<Item = &ArchivedMetadataHeaderValue> + Sync + Send {\n        self.headers.iter().filter_map(move |header| {\n            if &header.name == name {\n                Some(&header.value)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn header_value(&self, name: &MetadataHeaderName) -> Option<&ArchivedMetadataHeaderValue> {\n        self.headers.iter().rev().find_map(move |header| {\n            if &header.name == name {\n                Some(&header.value)\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn subject(&self) -> Option<&str> {\n        self.header_value(&MetadataHeaderName::Subject)\n            .and_then(|header| header.as_text())\n    }\n\n    pub fn date(&self) -> Option<DateTime> {\n        self.header_value(&MetadataHeaderName::Date)\n            .and_then(|header| header.as_datetime())\n            .map(|dt| dt.into())\n    }\n\n    pub fn message_id(&self) -> Option<&str> {\n        self.header_value(&MetadataHeaderName::MessageId)\n            .and_then(|header| header.as_text())\n    }\n\n    pub fn in_reply_to(&self) -> &ArchivedMetadataHeaderValue {\n        self.header_value(&MetadataHeaderName::InReplyTo)\n            .unwrap_or(&ArchivedMetadataHeaderValue::Empty)\n    }\n\n    pub fn content_description(&self) -> Option<&str> {\n        self.header_value(&MetadataHeaderName::ContentDescription)\n            .and_then(|header| header.as_text())\n    }\n\n    pub fn content_disposition(&self) -> Option<&ArchivedMetadataContentType> {\n        self.header_value(&MetadataHeaderName::ContentDisposition)\n            .and_then(|header| header.as_content_type())\n    }\n\n    pub fn content_id(&self) -> Option<&str> {\n        self.header_value(&MetadataHeaderName::ContentId)\n            .and_then(|header| header.as_text())\n    }\n\n    pub fn content_transfer_encoding(&self) -> Option<&str> {\n        self.header_value(&MetadataHeaderName::ContentTransferEncoding)\n            .and_then(|header| header.as_text())\n    }\n\n    pub fn content_type(&self) -> Option<&ArchivedMetadataContentType> {\n        self.header_value(&MetadataHeaderName::ContentType)\n            .and_then(|header| header.as_content_type())\n    }\n\n    pub fn content_language(&self) -> &ArchivedMetadataHeaderValue {\n        self.header_value(&MetadataHeaderName::ContentLanguage)\n            .unwrap_or(&ArchivedMetadataHeaderValue::Empty)\n    }\n\n    pub fn content_location(&self) -> Option<&str> {\n        self.header_value(&MetadataHeaderName::ContentLocation)\n            .and_then(|header| header.as_text())\n    }\n\n    pub fn attachment_name(&self) -> Option<&str> {\n        self.content_disposition()\n            .and_then(|cd| cd.attribute(\"filename\"))\n            .or_else(|| self.content_type().and_then(|ct| ct.attribute(\"name\")))\n    }\n}\n\nimpl From<HeaderName<'_>> for MetadataHeaderName {\n    fn from(value: HeaderName<'_>) -> Self {\n        match value {\n            HeaderName::Subject => MetadataHeaderName::Subject,\n            HeaderName::From => MetadataHeaderName::From,\n            HeaderName::To => MetadataHeaderName::To,\n            HeaderName::Cc => MetadataHeaderName::Cc,\n            HeaderName::Date => MetadataHeaderName::Date,\n            HeaderName::Bcc => MetadataHeaderName::Bcc,\n            HeaderName::ReplyTo => MetadataHeaderName::ReplyTo,\n            HeaderName::Sender => MetadataHeaderName::Sender,\n            HeaderName::Comments => MetadataHeaderName::Comments,\n            HeaderName::InReplyTo => MetadataHeaderName::InReplyTo,\n            HeaderName::Keywords => MetadataHeaderName::Keywords,\n            HeaderName::Received => MetadataHeaderName::Received,\n            HeaderName::MessageId => MetadataHeaderName::MessageId,\n            HeaderName::References => MetadataHeaderName::References,\n            HeaderName::ReturnPath => MetadataHeaderName::ReturnPath,\n            HeaderName::MimeVersion => MetadataHeaderName::MimeVersion,\n            HeaderName::ContentDescription => MetadataHeaderName::ContentDescription,\n            HeaderName::ContentId => MetadataHeaderName::ContentId,\n            HeaderName::ContentLanguage => MetadataHeaderName::ContentLanguage,\n            HeaderName::ContentLocation => MetadataHeaderName::ContentLocation,\n            HeaderName::ContentTransferEncoding => MetadataHeaderName::ContentTransferEncoding,\n            HeaderName::ContentType => MetadataHeaderName::ContentType,\n            HeaderName::ContentDisposition => MetadataHeaderName::ContentDisposition,\n            HeaderName::ResentTo => MetadataHeaderName::ResentTo,\n            HeaderName::ResentFrom => MetadataHeaderName::ResentFrom,\n            HeaderName::ResentBcc => MetadataHeaderName::ResentBcc,\n            HeaderName::ResentCc => MetadataHeaderName::ResentCc,\n            HeaderName::ResentSender => MetadataHeaderName::ResentSender,\n            HeaderName::ResentDate => MetadataHeaderName::ResentDate,\n            HeaderName::ResentMessageId => MetadataHeaderName::ResentMessageId,\n            HeaderName::ListArchive => MetadataHeaderName::ListArchive,\n            HeaderName::ListHelp => MetadataHeaderName::ListHelp,\n            HeaderName::ListId => MetadataHeaderName::ListId,\n            HeaderName::ListOwner => MetadataHeaderName::ListOwner,\n            HeaderName::ListPost => MetadataHeaderName::ListPost,\n            HeaderName::ListSubscribe => MetadataHeaderName::ListSubscribe,\n            HeaderName::ListUnsubscribe => MetadataHeaderName::ListUnsubscribe,\n            HeaderName::DkimSignature => MetadataHeaderName::DkimSignature,\n            HeaderName::ArcAuthenticationResults => MetadataHeaderName::ArcAuthenticationResults,\n            HeaderName::ArcMessageSignature => MetadataHeaderName::ArcMessageSignature,\n            HeaderName::ArcSeal => MetadataHeaderName::ArcSeal,\n            HeaderName::Other(value) => {\n                let name = hashify::tiny_map_ignore_case!(value.as_bytes(),\n                    // Delivery/Routing\n                    \"Delivered-To\" => MetadataHeaderName::DeliveredTo,\n                    \"X-Original-To\" => MetadataHeaderName::XOriginalTo,\n                    \"Return-Receipt-To\" => MetadataHeaderName::ReturnReceiptTo,\n                    \"Disposition-Notification-To\" => MetadataHeaderName::DispositionNotificationTo,\n                    \"Errors-To\" => MetadataHeaderName::ErrorsTo,\n\n                    // Authentication\n                    \"Authentication-Results\" => MetadataHeaderName::AuthenticationResults,\n                    \"Received-SPF\" => MetadataHeaderName::ReceivedSpf,\n\n                    // Spam/Virus\n                    \"X-Spam-Status\" => MetadataHeaderName::XSpamStatus,\n                    \"X-Spam-Score\" => MetadataHeaderName::XSpamScore,\n                    \"X-Spam-Flag\" => MetadataHeaderName::XSpamFlag,\n                    \"X-Spam-Result\" => MetadataHeaderName::XSpamResult,\n\n                    // Priority\n                    \"Importance\" => MetadataHeaderName::Importance,\n                    \"Priority\" => MetadataHeaderName::Priority,\n                    \"X-Priority\" => MetadataHeaderName::XPriority,\n                    \"X-MSMail-Priority\" => MetadataHeaderName::XMSMailPriority,\n\n                    // Client/Agent\n                    \"X-Mailer\" => MetadataHeaderName::XMailer,\n                    \"User-Agent\" => MetadataHeaderName::UserAgent,\n                    \"X-MimeOLE\" => MetadataHeaderName::XMimeOLE,\n\n                    // Network/Origin\n                    \"X-Originating-IP\" => MetadataHeaderName::XOriginatingIp,\n                    \"X-Forwarded-To\" => MetadataHeaderName::XForwardedTo,\n                    \"X-Forwarded-For\" => MetadataHeaderName::XForwardedFor,\n\n                    // Auto-response\n                    \"Auto-Submitted\" => MetadataHeaderName::AutoSubmitted,\n                    \"X-Auto-Response-Suppress\" => MetadataHeaderName::XAutoResponseSuppress,\n                    \"Precedence\" => MetadataHeaderName::Precedence,\n\n                    // Organization/Threading\n                    \"Organization\" => MetadataHeaderName::Organization,\n                    \"Thread-Index\" => MetadataHeaderName::ThreadIndex,\n                    \"Thread-Topic\" => MetadataHeaderName::ThreadTopic,\n\n                    // List (additional)\n                    \"List-Unsubscribe-Post\" => MetadataHeaderName::ListUnsubscribePost,\n                    \"Feedback-ID\" => MetadataHeaderName::FeedbackId,\n                );\n                name.unwrap_or_else(|| {\n                    MetadataHeaderName::Other(value.into_owned().into_boxed_str())\n                })\n            }\n            other => MetadataHeaderName::Other(other.as_str().to_string().into_boxed_str()),\n        }\n    }\n}\n\nimpl From<HeaderValue<'_>> for MetadataHeaderValue {\n    fn from(value: HeaderValue<'_>) -> Self {\n        match value {\n            HeaderValue::Address(address) => match address {\n                Address::List(address) => MetadataHeaderValue::AddressList(\n                    address\n                        .into_iter()\n                        .map(|a| MetadataAddress {\n                            name: a.name.map(|a| a.into_owned().into_boxed_str()),\n                            address: a.address.map(|a| a.into_owned().into_boxed_str()),\n                        })\n                        .collect(),\n                ),\n                Address::Group(groups) => MetadataHeaderValue::AddressGroup(\n                    groups\n                        .into_iter()\n                        .map(|g| MetadataAddressGroup {\n                            name: g.name.map(|a| a.into_owned().into_boxed_str()),\n                            addresses: g\n                                .addresses\n                                .into_iter()\n                                .map(|a| MetadataAddress {\n                                    name: a.name.map(|a| a.into_owned().into_boxed_str()),\n                                    address: a.address.map(|a| a.into_owned().into_boxed_str()),\n                                })\n                                .collect(),\n                        })\n                        .collect(),\n                ),\n            },\n            HeaderValue::Text(text) => {\n                MetadataHeaderValue::Text(text.into_owned().into_boxed_str())\n            }\n            HeaderValue::TextList(texts) => MetadataHeaderValue::TextList(\n                texts\n                    .into_iter()\n                    .map(|v| v.into_owned().into_boxed_str())\n                    .collect(),\n            ),\n            HeaderValue::DateTime(dt) => MetadataHeaderValue::DateTime(MetadataDateTime {\n                year: dt.year,\n                month: dt.month,\n                day: dt.day,\n                hour: dt.hour,\n                minute: dt.minute,\n                second: dt.second,\n                tz_hour: (if dt.tz_before_gmt { -1 } else { 1 }) * dt.tz_hour as i8,\n                tz_minute: dt.tz_minute,\n            }),\n            HeaderValue::ContentType(ct) => MetadataHeaderValue::ContentType(MetadataContentType {\n                c_type: ct.c_type.into_owned().into_boxed_str(),\n                c_subtype: ct.c_subtype.map(|v| v.into_owned().into_boxed_str()),\n                attributes: ct\n                    .attributes\n                    .unwrap_or_default()\n                    .into_iter()\n                    .map(|a| MetadataAttribute {\n                        name: a.name.into_owned().into_boxed_str(),\n                        value: a.value.into_owned().into_boxed_str(),\n                    })\n                    .collect(),\n            }),\n            HeaderValue::Received(_) | HeaderValue::Empty => MetadataHeaderValue::Empty,\n        }\n    }\n}\n\nimpl From<&ArchivedMetadataDateTime> for DateTime {\n    fn from(dt: &ArchivedMetadataDateTime) -> Self {\n        DateTime {\n            year: dt.year.to_native(),\n            month: dt.month,\n            day: dt.day,\n            hour: dt.hour,\n            minute: dt.minute,\n            second: dt.second,\n            tz_before_gmt: dt.tz_hour < 0,\n            tz_hour: dt.tz_hour.unsigned_abs(),\n            tz_minute: dt.tz_minute,\n        }\n    }\n}\n\nimpl ArchivedMessageMetadataContents {\n    pub fn root_part(&self) -> &ArchivedMessageMetadataPart {\n        &self.parts[0]\n    }\n}\n\n#[derive(Default)]\npub struct MessageDataBuilder {\n    pub mailboxes: Vec<UidMailbox>,\n    pub keywords: Vec<Keyword>,\n    pub thread_id: u32,\n    pub size: u32,\n}\n\nimpl MessageDataBuilder {\n    pub fn set_keywords(&mut self, keywords: Vec<Keyword>) {\n        self.keywords = keywords;\n    }\n\n    pub fn add_keyword(&mut self, keyword: Keyword) -> bool {\n        if !self.keywords.contains(&keyword) {\n            self.keywords.push(keyword);\n            true\n        } else {\n            false\n        }\n    }\n\n    pub fn remove_keyword(&mut self, keyword: &Keyword) -> bool {\n        let prev_len = self.keywords.len();\n        self.keywords.retain(|k| k != keyword);\n        self.keywords.len() != prev_len\n    }\n\n    pub fn set_mailboxes(&mut self, mailboxes: Vec<UidMailbox>) {\n        self.mailboxes = mailboxes;\n    }\n\n    pub fn add_mailbox(&mut self, mailbox: UidMailbox) {\n        if !self.mailboxes.contains(&mailbox) {\n            self.mailboxes.push(mailbox);\n        }\n    }\n\n    pub fn remove_mailbox(&mut self, mailbox: u32) {\n        self.mailboxes.retain(|m| m.mailbox_id != mailbox);\n    }\n\n    pub fn has_keyword(&self, keyword: &Keyword) -> bool {\n        self.keywords.iter().any(|k| k == keyword)\n    }\n\n    pub fn has_keyword_changes(&self, prev_data: &ArchivedMessageData) -> bool {\n        self.keywords.len() != prev_data.keywords.len()\n            || !self\n                .keywords\n                .iter()\n                .all(|k| prev_data.keywords.iter().any(|pk| pk == k))\n    }\n\n    pub fn added_keywords(\n        &self,\n        prev_data: &ArchivedMessageData,\n    ) -> impl Iterator<Item = &Keyword> {\n        self.keywords\n            .iter()\n            .filter(|k| prev_data.keywords.iter().all(|pk| pk != *k))\n    }\n\n    pub fn removed_keywords<'x>(\n        &'x self,\n        prev_data: &'x ArchivedMessageData,\n    ) -> impl Iterator<Item = &'x ArchivedKeyword> {\n        prev_data\n            .keywords\n            .iter()\n            .filter(|k| self.keywords.iter().all(|pk| pk != *k))\n    }\n\n    pub fn added_mailboxes(\n        &self,\n        prev_data: &ArchivedMessageData,\n    ) -> impl Iterator<Item = &UidMailbox> {\n        self.mailboxes.iter().filter(|m| {\n            prev_data\n                .mailboxes\n                .iter()\n                .all(|pm| pm.mailbox_id != m.mailbox_id)\n        })\n    }\n\n    pub fn removed_mailboxes<'x>(\n        &'x self,\n        prev_data: &'x ArchivedMessageData,\n    ) -> impl Iterator<Item = &'x ArchivedUidMailbox> {\n        prev_data.mailboxes.iter().filter(|m| {\n            self.mailboxes\n                .iter()\n                .all(|pm| pm.mailbox_id != m.mailbox_id)\n        })\n    }\n\n    pub fn has_mailbox_changes(&self, prev_data: &ArchivedMessageData) -> bool {\n        self.mailboxes.len() != prev_data.mailboxes.len()\n            || !self.mailboxes.iter().all(|m| {\n                prev_data\n                    .mailboxes\n                    .iter()\n                    .any(|pm| pm.mailbox_id == m.mailbox_id)\n            })\n    }\n\n    pub fn seal(self) -> MessageData {\n        MessageData {\n            mailboxes: self.mailboxes.into_boxed_slice(),\n            keywords: self.keywords.into_boxed_slice(),\n            thread_id: self.thread_id,\n            size: self.size,\n        }\n    }\n}\n\nimpl MessageData {\n    pub fn has_mailbox_id(&self, mailbox_id: u32) -> bool {\n        self.mailboxes.iter().any(|m| m.mailbox_id == mailbox_id)\n    }\n}\n\nimpl ArchivedMessageData {\n    pub fn has_mailbox_id(&self, mailbox_id: u32) -> bool {\n        self.mailboxes.iter().any(|m| m.mailbox_id == mailbox_id)\n    }\n\n    pub fn message_uid(&self, mailbox_id: u32) -> Option<u32> {\n        self.mailboxes\n            .iter()\n            .find(|m| m.mailbox_id == mailbox_id)\n            .map(|m| m.uid.to_native())\n    }\n\n    pub fn to_builder(&self) -> MessageDataBuilder {\n        MessageDataBuilder {\n            mailboxes: self.mailboxes.iter().map(|m| m.to_native()).collect(),\n            keywords: self.keywords.iter().map(|k| k.to_native()).collect(),\n            thread_id: self.thread_id.to_native(),\n            size: self.size.to_native(),\n        }\n    }\n}\n\nimpl ArchivedMetadataContentType {\n    pub fn ctype(&self) -> &str {\n        &self.c_type\n    }\n\n    pub fn subtype(&self) -> Option<&str> {\n        self.c_subtype.as_ref().map(|s| s.as_ref())\n    }\n\n    pub fn attribute(&self, name: &str) -> Option<&str> {\n        self.attributes\n            .iter()\n            .find(|a| *a.name == *name)\n            .map(|a| a.value.as_ref())\n    }\n\n    /// Returns `true` when the provided attribute name is present\n    pub fn has_attribute(&self, name: &str) -> bool {\n        self.attributes.iter().any(|a| *a.name == *name)\n    }\n\n    pub fn is_attachment(&self) -> bool {\n        self.c_type.eq_ignore_ascii_case(\"attachment\")\n    }\n\n    pub fn is_inline(&self) -> bool {\n        self.c_type.eq_ignore_ascii_case(\"inline\")\n    }\n}\n\nimpl ArchivedMetadataHeaderValue {\n    pub fn is_empty(&self) -> bool {\n        self == &MetadataHeaderValue::Empty\n    }\n\n    pub fn as_text(&self) -> Option<&str> {\n        match self {\n            ArchivedMetadataHeaderValue::Text(s) => Some(s.as_ref()),\n            ArchivedMetadataHeaderValue::TextList(l) => l.last().map(|v| v.as_ref()),\n            _ => None,\n        }\n    }\n\n    pub fn as_text_list(&self) -> Option<&[ArchivedBox<str>]> {\n        match self {\n            ArchivedMetadataHeaderValue::Text(s) => Some(std::slice::from_ref(s)),\n            ArchivedMetadataHeaderValue::TextList(l) => Some(l.as_ref()),\n            _ => None,\n        }\n    }\n\n    pub fn as_content_type(&self) -> Option<&ArchivedMetadataContentType> {\n        match self {\n            ArchivedMetadataHeaderValue::ContentType(c) => Some(c),\n            _ => None,\n        }\n    }\n\n    pub fn as_datetime(&self) -> Option<&ArchivedMetadataDateTime> {\n        match self {\n            ArchivedMetadataHeaderValue::DateTime(d) => Some(d),\n            _ => None,\n        }\n    }\n\n    pub fn as_single_address(&self) -> Option<&ArchivedMetadataAddress> {\n        match self {\n            ArchivedMetadataHeaderValue::AddressList(list) => list.first(),\n            ArchivedMetadataHeaderValue::AddressGroup(groups) => {\n                groups.first().and_then(|g| g.addresses.first())\n            }\n            _ => None,\n        }\n    }\n}\n\nimpl ArchivedUidMailbox {\n    pub fn to_native(&self) -> UidMailbox {\n        UidMailbox {\n            mailbox_id: self.mailbox_id.to_native(),\n            uid: self.uid.to_native(),\n        }\n    }\n}\n\nimpl ArchivedMetadataHeader {\n    #[inline(always)]\n    pub fn value_range(&self) -> Range<usize> {\n        (self.base_offset.to_native() as usize + self.start.to_native() as usize)\n            ..(self.base_offset.to_native() as usize + self.end.to_native() as usize)\n    }\n\n    #[inline(always)]\n    pub fn name_value_range(&self) -> Range<usize> {\n        (self.base_offset.to_native() as usize)\n            ..(self.base_offset.to_native() as usize + self.end.to_native() as usize)\n    }\n}\n\nimpl ArchivedMetadataHeaderName {\n    pub fn is_mime_header(&self) -> bool {\n        matches!(\n            self,\n            ArchivedMetadataHeaderName::ContentDescription\n                | ArchivedMetadataHeaderName::ContentId\n                | ArchivedMetadataHeaderName::ContentLanguage\n                | ArchivedMetadataHeaderName::ContentLocation\n                | ArchivedMetadataHeaderName::ContentTransferEncoding\n                | ArchivedMetadataHeaderName::ContentType\n                | ArchivedMetadataHeaderName::ContentDisposition\n        )\n    }\n\n    pub fn as_str(&self) -> &str {\n        match self {\n            ArchivedMetadataHeaderName::Subject => \"Subject\",\n            ArchivedMetadataHeaderName::From => \"From\",\n            ArchivedMetadataHeaderName::To => \"To\",\n            ArchivedMetadataHeaderName::Cc => \"Cc\",\n            ArchivedMetadataHeaderName::Date => \"Date\",\n            ArchivedMetadataHeaderName::Bcc => \"Bcc\",\n            ArchivedMetadataHeaderName::ReplyTo => \"Reply-To\",\n            ArchivedMetadataHeaderName::Sender => \"Sender\",\n            ArchivedMetadataHeaderName::Comments => \"Comments\",\n            ArchivedMetadataHeaderName::InReplyTo => \"In-Reply-To\",\n            ArchivedMetadataHeaderName::Keywords => \"Keywords\",\n            ArchivedMetadataHeaderName::Received => \"Received\",\n            ArchivedMetadataHeaderName::MessageId => \"Message-ID\",\n            ArchivedMetadataHeaderName::References => \"References\",\n            ArchivedMetadataHeaderName::ReturnPath => \"Return-Path\",\n            ArchivedMetadataHeaderName::MimeVersion => \"MIME-Version\",\n            ArchivedMetadataHeaderName::ContentDescription => \"Content-Description\",\n            ArchivedMetadataHeaderName::ContentId => \"Content-ID\",\n            ArchivedMetadataHeaderName::ContentLanguage => \"Content-Language\",\n            ArchivedMetadataHeaderName::ContentLocation => \"Content-Location\",\n            ArchivedMetadataHeaderName::ContentTransferEncoding => \"Content-Transfer-Encoding\",\n            ArchivedMetadataHeaderName::ContentType => \"Content-Type\",\n            ArchivedMetadataHeaderName::ContentDisposition => \"Content-Disposition\",\n            ArchivedMetadataHeaderName::ResentTo => \"Resent-To\",\n            ArchivedMetadataHeaderName::ResentFrom => \"Resent-From\",\n            ArchivedMetadataHeaderName::ResentBcc => \"Resent-Bcc\",\n            ArchivedMetadataHeaderName::ResentCc => \"Resent-Cc\",\n            ArchivedMetadataHeaderName::ResentSender => \"Resent-Sender\",\n            ArchivedMetadataHeaderName::ResentDate => \"Resent-Date\",\n            ArchivedMetadataHeaderName::ResentMessageId => \"Resent-Message-ID\",\n            ArchivedMetadataHeaderName::ListArchive => \"List-Archive\",\n            ArchivedMetadataHeaderName::ListHelp => \"List-Help\",\n            ArchivedMetadataHeaderName::ListId => \"List-ID\",\n            ArchivedMetadataHeaderName::ListOwner => \"List-Owner\",\n            ArchivedMetadataHeaderName::ListPost => \"List-Post\",\n            ArchivedMetadataHeaderName::ListSubscribe => \"List-Subscribe\",\n            ArchivedMetadataHeaderName::ListUnsubscribe => \"List-Unsubscribe\",\n            ArchivedMetadataHeaderName::ArcAuthenticationResults => \"ARC-Authentication-Results\",\n            ArchivedMetadataHeaderName::ArcMessageSignature => \"ARC-Message-Signature\",\n            ArchivedMetadataHeaderName::ArcSeal => \"ARC-Seal\",\n            ArchivedMetadataHeaderName::DkimSignature => \"DKIM-Signature\",\n            ArchivedMetadataHeaderName::DeliveredTo => \"Delivered-To\",\n            ArchivedMetadataHeaderName::XOriginalTo => \"X-Original-To\",\n            ArchivedMetadataHeaderName::ReturnReceiptTo => \"Return-Receipt-To\",\n            ArchivedMetadataHeaderName::DispositionNotificationTo => \"Disposition-Notification-To\",\n            ArchivedMetadataHeaderName::ErrorsTo => \"Errors-To\",\n            ArchivedMetadataHeaderName::AuthenticationResults => \"Authentication-Results\",\n            ArchivedMetadataHeaderName::ReceivedSpf => \"Received-SPF\",\n            ArchivedMetadataHeaderName::XSpamStatus => \"X-Spam-Status\",\n            ArchivedMetadataHeaderName::XSpamScore => \"X-Spam-Score\",\n            ArchivedMetadataHeaderName::XSpamFlag => \"X-Spam-Flag\",\n            ArchivedMetadataHeaderName::XSpamResult => \"X-Spam-Result\",\n            ArchivedMetadataHeaderName::Importance => \"Importance\",\n            ArchivedMetadataHeaderName::Priority => \"Priority\",\n            ArchivedMetadataHeaderName::XPriority => \"X-Priority\",\n            ArchivedMetadataHeaderName::XMSMailPriority => \"X-MSMail-Priority\",\n            ArchivedMetadataHeaderName::XMailer => \"X-Mailer\",\n            ArchivedMetadataHeaderName::UserAgent => \"User-Agent\",\n            ArchivedMetadataHeaderName::XMimeOLE => \"X-MimeOLE\",\n            ArchivedMetadataHeaderName::XOriginatingIp => \"X-Originating-IP\",\n            ArchivedMetadataHeaderName::XForwardedTo => \"X-Forwarded-To\",\n            ArchivedMetadataHeaderName::XForwardedFor => \"X-Forwarded-For\",\n            ArchivedMetadataHeaderName::AutoSubmitted => \"Auto-Submitted\",\n            ArchivedMetadataHeaderName::XAutoResponseSuppress => \"X-Auto-Response-Suppress\",\n            ArchivedMetadataHeaderName::Precedence => \"Precedence\",\n            ArchivedMetadataHeaderName::Organization => \"Organization\",\n            ArchivedMetadataHeaderName::ThreadIndex => \"Thread-Index\",\n            ArchivedMetadataHeaderName::ThreadTopic => \"Thread-Topic\",\n            ArchivedMetadataHeaderName::ListUnsubscribePost => \"List-Unsubscribe-Post\",\n            ArchivedMetadataHeaderName::FeedbackId => \"Feedback-ID\",\n            ArchivedMetadataHeaderName::Other(name) => name.as_ref(),\n        }\n    }\n}\n\nimpl From<&ArchivedMetadataHeaderValue> for HeaderValue<'static> {\n    fn from(value: &ArchivedMetadataHeaderValue) -> Self {\n        match value {\n            ArchivedMetadataHeaderValue::AddressList(addr) => HeaderValue::Address(Address::List(\n                addr.as_ref().iter().map(Into::into).collect(),\n            )),\n            ArchivedMetadataHeaderValue::AddressGroup(addr) => HeaderValue::Address(\n                Address::Group(addr.as_ref().iter().map(Into::into).collect()),\n            ),\n            ArchivedMetadataHeaderValue::Text(text) => HeaderValue::Text(text.to_string().into()),\n            ArchivedMetadataHeaderValue::TextList(textlist) => HeaderValue::TextList(\n                textlist\n                    .as_ref()\n                    .iter()\n                    .map(|s| s.to_string().into())\n                    .collect(),\n            ),\n            ArchivedMetadataHeaderValue::DateTime(dt) => HeaderValue::DateTime(dt.into()),\n            ArchivedMetadataHeaderValue::ContentType(ct) => HeaderValue::ContentType(ct.into()),\n            ArchivedMetadataHeaderValue::Empty => HeaderValue::Empty,\n        }\n    }\n}\n\nimpl From<&ArchivedMetadataAddress> for Addr<'static> {\n    fn from(value: &ArchivedMetadataAddress) -> Self {\n        Addr {\n            name: value.name.as_ref().map(|n| n.to_string().into()),\n            address: value.address.as_ref().map(|a| a.to_string().into()),\n        }\n    }\n}\n\nimpl From<&ArchivedMetadataAddressGroup> for Group<'static> {\n    fn from(value: &ArchivedMetadataAddressGroup) -> Self {\n        Group {\n            name: value.name.as_ref().map(|n| n.to_string().into()),\n            addresses: value.addresses.as_ref().iter().map(Into::into).collect(),\n        }\n    }\n}\n\nimpl From<&ArchivedMetadataContentType> for ContentType<'static> {\n    fn from(value: &ArchivedMetadataContentType) -> Self {\n        ContentType {\n            c_type: value.ctype().to_string().into(),\n            c_subtype: value.subtype().map(|s| s.to_string().into()),\n            attributes: Some(\n                value\n                    .attributes\n                    .iter()\n                    .map(|a| Attribute {\n                        name: a.name.to_string().into(),\n                        value: a.value.to_string().into(),\n                    })\n                    .collect(),\n            ),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/email/src/message/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod copy;\npub mod crypto;\npub mod delete;\npub mod delivery;\npub mod index;\npub mod ingest;\npub mod metadata;\n"
  },
  {
    "path": "crates/email/src/push/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse types::type_state::DataType;\nuse utils::map::bitmap::Bitmap;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Default, Debug, Clone, PartialEq, Eq,\n)]\npub struct PushSubscription {\n    pub id: u32,\n    pub url: String,\n    pub device_client_id: String,\n    pub expires: u64,\n    pub verification_code: String,\n    pub verified: bool,\n    pub types: Bitmap<DataType>,\n    pub keys: Option<Keys>,\n    pub email_push: Vec<EmailPush>,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct Keys {\n    pub p256dh: Vec<u8>,\n    pub auth: Vec<u8>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Default, Debug, Clone, PartialEq, Eq,\n)]\npub struct PushSubscriptions {\n    pub subscriptions: Vec<PushSubscription>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Default, Debug, Clone, PartialEq, Eq,\n)]\npub struct EmailPush {\n    pub account_id: u32,\n    pub properties: u64,\n    pub filters: Vec<EmailPushFilter>,\n    pub flags: u16,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub enum EmailPushFilter {\n    Condition { field: u8, value: EmailPushValue },\n    And,\n    Or,\n    Not,\n    End,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub enum EmailPushValue {\n    Text(String),\n    Number(u64),\n    TextList(Vec<String>),\n    NumberList(Vec<u64>),\n}\n"
  },
  {
    "path": "crates/email/src/sieve/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::SieveScript;\nuse common::{Server, auth::AccessToken, storage::index::ObjectIndexBuilder};\nuse store::write::BatchBuilder;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::SieveField};\n\npub trait SieveScriptDelete: Sync + Send {\n    fn sieve_script_delete(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        access_token: &AccessToken,\n        batch: &mut BatchBuilder,\n    ) -> impl Future<Output = trc::Result<bool>> + Send;\n}\n\nimpl SieveScriptDelete for Server {\n    async fn sieve_script_delete(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        access_token: &AccessToken,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<bool> {\n        // Fetch record\n        if let Some(obj_) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::SieveScript,\n                document_id,\n            ))\n            .await?\n        {\n            // Delete record\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::SieveScript)\n                .with_document(document_id)\n                .clear(SieveField::Ids)\n                .custom(\n                    ObjectIndexBuilder::<_, ()>::new()\n                        .with_current(\n                            obj_.to_unarchived::<SieveScript>()\n                                .caused_by(trc::location!())?,\n                        )\n                        .with_access_token(access_token),\n                )\n                .caused_by(trc::location!())?\n                .commit_point();\n\n            Ok(true)\n        } else {\n            Ok(false)\n        }\n    }\n}\n"
  },
  {
    "path": "crates/email/src/sieve/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ArchivedSieveScript, SieveScript};\nuse common::storage::index::{IndexValue, IndexableAndSerializableObject, IndexableObject};\nuse types::{collection::SyncCollection, field::SieveField};\n\nimpl IndexableObject for SieveScript {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Index {\n                field: SieveField::Name.into(),\n                value: self.name.as_str().to_lowercase().into(),\n            },\n            IndexValue::Blob {\n                value: self.blob_hash.clone(),\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::SieveScript,\n                prefix: None,\n            },\n            IndexValue::Quota { used: self.size },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for SieveScript {\n    fn is_versioned() -> bool {\n        false\n    }\n}\n\nimpl IndexableObject for &ArchivedSieveScript {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Index {\n                field: SieveField::Name.into(),\n                value: self.name.to_lowercase().into(),\n            },\n            IndexValue::Blob {\n                value: (&self.blob_hash).into(),\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::SieveScript,\n                prefix: None,\n            },\n            IndexValue::Quota {\n                used: u32::from(self.size),\n            },\n        ]\n        .into_iter()\n    }\n}\n"
  },
  {
    "path": "crates/email/src/sieve/ingest.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ActiveScript, SeenIdHash, SieveScript};\nuse crate::{\n    cache::{MessageCacheFetch, mailbox::MailboxCacheAccess},\n    mailbox::{INBOX_ID, TRASH_ID, manage::MailboxFnc},\n    message::{\n        delivery::{AutogeneratedMessage, IngestRecipient},\n        ingest::{EmailIngest, IngestEmail, IngestSource, IngestedEmail},\n    },\n};\nuse common::{Server, auth::AccessToken, scripts::plugins::PluginContext};\nuse directory::QueryParams;\nuse mail_parser::MessageParser;\nuse sieve::{Envelope, Event, Input, Mailbox, Recipient, Sieve, SpamStatus};\nuse std::{borrow::Cow, sync::Arc};\nuse std::{future::Future, str::FromStr};\nuse store::{\n    Deserialize, Serialize, ValueKey,\n    ahash::AHashMap,\n    dispatch::lookup::KeyValue,\n    write::{\n        AlignedBytes, Archive, ArchiveVersion, Archiver, BatchBuilder, BlobLink, BlobOp, ValueClass,\n    },\n};\nuse trc::{AddContext, SieveEvent};\nuse types::{\n    blob_hash::BlobHash,\n    collection::Collection,\n    field::{PrincipalField, SieveField},\n    id::Id,\n    keyword::Keyword,\n    special_use::SpecialUse,\n};\nuse utils::config::utils::ParseValue;\n\nstruct SieveMessage<'x> {\n    pub raw_message: Cow<'x, [u8]>,\n    pub file_into: Vec<u32>,\n    pub did_file_into: bool,\n    pub flags: Vec<Keyword>,\n}\n\npub trait SieveScriptIngest: Sync + Send {\n    #[allow(clippy::too_many_arguments)]\n    fn sieve_script_ingest(\n        &self,\n        access_token: &AccessToken,\n        blob_hash: &BlobHash,\n        raw_message: &[u8],\n        envelope_from: &str,\n        envelope_from_authenticated: bool,\n        envelope_to: &IngestRecipient,\n        session_id: u64,\n        active_script: ActiveScript,\n        autogenerated: &mut Vec<AutogeneratedMessage>,\n    ) -> impl Future<Output = trc::Result<IngestedEmail>> + Send;\n\n    fn sieve_script_get_active_id(\n        &self,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Option<u32>>> + Send;\n\n    fn sieve_script_get_active(\n        &self,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Option<ActiveScript>>> + Send;\n\n    fn sieve_script_get_by_name(\n        &self,\n        account_id: u32,\n        name: &str,\n    ) -> impl Future<Output = trc::Result<Option<Sieve>>> + Send;\n\n    fn sieve_script_compile(\n        &self,\n        account_id: u32,\n        document_id: u32,\n    ) -> impl Future<Output = trc::Result<Option<CompiledScript>>> + Send;\n}\n\nimpl SieveScriptIngest for Server {\n    #[allow(clippy::blocks_in_conditions)]\n    async fn sieve_script_ingest(\n        &self,\n        access_token: &AccessToken,\n        blob_hash: &BlobHash,\n        raw_message: &[u8],\n        envelope_from: &str,\n        envelope_from_authenticated: bool,\n        envelope_to: &IngestRecipient,\n        session_id: u64,\n        active_script: ActiveScript,\n        autogenerated: &mut Vec<AutogeneratedMessage>,\n    ) -> trc::Result<IngestedEmail> {\n        // Parse message\n        let message = if let Some(message) = MessageParser::new().parse(raw_message) {\n            message\n        } else {\n            return Err(\n                trc::EventType::MessageIngest(trc::MessageIngestEvent::Error)\n                    .ctx(trc::Key::Code, 550)\n                    .ctx(trc::Key::Reason, \"Failed to parse e-mail message.\"),\n            );\n        };\n\n        // Obtain mailboxIds\n        let account_id = access_token.primary_id;\n        let mut cache = self\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Create Sieve instance\n        let mut instance = self.core.sieve.untrusted_runtime.filter_parsed(message);\n\n        // Set account name and email\n        let mail_from = self\n            .core\n            .storage\n            .directory\n            .query(QueryParams::id(account_id).with_return_member_of(false))\n            .await\n            .caused_by(trc::location!())?\n            .and_then(|p| {\n                instance.set_user_full_name(p.description().unwrap_or_else(|| p.name()));\n                p.into_primary_email()\n            });\n\n        // Set account address\n        let mail_from = mail_from.unwrap_or_else(|| envelope_to.address.as_str().into());\n        instance.set_user_address(&mail_from);\n\n        // Set envelope\n        instance.set_envelope(Envelope::From, envelope_from);\n        instance.set_envelope(Envelope::To, envelope_to.address.as_str());\n        instance.set_spam_status(if envelope_to.is_spam {\n            SpamStatus::Spam\n        } else {\n            SpamStatus::Ham\n        });\n\n        let mut input = Input::script(\n            active_script.script_name.to_string(),\n            active_script.script.clone(),\n        );\n\n        let mut do_discard = false;\n        let mut do_deliver = false;\n\n        let mut reject_reason = None;\n        let mut messages: Vec<SieveMessage> = vec![SieveMessage {\n            raw_message: raw_message.into(),\n            file_into: Vec::new(),\n            flags: Vec::new(),\n            did_file_into: false,\n        }];\n        let mut ingested_message = IngestedEmail {\n            document_id: 0,\n            thread_id: 0,\n            change_id: u64::MAX,\n            blob_id: Default::default(),\n            size: raw_message.len(),\n            imap_uids: Vec::new(),\n        };\n        let mut checked_ids: AHashMap<SeenIdHash, bool> = AHashMap::new();\n\n        while let Some(event) = instance.run(input) {\n            match event {\n                Ok(event) => match event {\n                    Event::IncludeScript { name, .. } => match &name {\n                        sieve::Script::Personal(name_) => {\n                            if let Ok(Some(script)) =\n                                self.sieve_script_get_by_name(account_id, name_).await\n                            {\n                                input = Input::script(name, script);\n                            } else {\n                                input = false.into();\n                            }\n                        }\n                        sieve::Script::Global(name_) => {\n                            if let Some(script) =\n                                self.get_untrusted_sieve_script(&name_.to_lowercase(), session_id)\n                            {\n                                input = Input::script(name, script.clone());\n                            } else {\n                                input = false.into();\n                            }\n                        }\n                    },\n                    Event::MailboxExists {\n                        mailboxes,\n                        special_use,\n                    } => {\n                        if !mailboxes.is_empty() {\n                            let mut special_use_ids = Vec::with_capacity(special_use.len());\n                            for role in special_use {\n                                special_use_ids.push(if role.eq_ignore_ascii_case(\"inbox\") {\n                                    INBOX_ID\n                                } else if role.eq_ignore_ascii_case(\"trash\") {\n                                    TRASH_ID\n                                } else {\n                                    let mut mailbox_id = u32::MAX;\n                                    if let Ok(role) = SpecialUse::parse_value(&role)\n                                        && let Some(m) = cache.mailbox_by_role(&role)\n                                    {\n                                        mailbox_id = m.document_id;\n                                    }\n\n                                    mailbox_id\n                                });\n                            }\n\n                            let mut result = true;\n                            for mailbox in mailboxes {\n                                match mailbox {\n                                    Mailbox::Name(name) => {\n                                        if !matches!(\n                                            cache.mailbox_by_path(&name),\n                                            Some(item) if special_use_ids.is_empty() ||\n                                            special_use_ids.contains(&item.document_id)\n                                        ) {\n                                            result = false;\n                                            break;\n                                        }\n                                    }\n                                    Mailbox::Id(id) => {\n                                        if !matches!(Id::from_str(&id), Ok(id) if\n                                                        cache.has_mailbox_id(&id.document_id()) &&\n                                                        (special_use_ids.is_empty() ||\n                                                        special_use_ids.contains(&id.document_id())))\n                                        {\n                                            result = false;\n                                            break;\n                                        }\n                                    }\n                                }\n                            }\n                            input = result.into();\n                        } else if !special_use.is_empty() {\n                            let mut result = true;\n\n                            for role in special_use {\n                                if !role.eq_ignore_ascii_case(\"inbox\")\n                                    && !role.eq_ignore_ascii_case(\"trash\")\n                                {\n                                    let role = SpecialUse::parse_value(&role);\n                                    if role.is_err()\n                                        || cache.mailbox_by_role(&role.unwrap()).is_none()\n                                    {\n                                        result = false;\n                                        break;\n                                    }\n                                }\n                            }\n                            input = result.into();\n                        } else {\n                            input = false.into();\n                        }\n                    }\n                    Event::DuplicateId { id, expiry, last } => {\n                        let id_hash = SeenIdHash::new(\n                            account_id,\n                            active_script.version.hash().unwrap_or_default(),\n                            &id,\n                        );\n                        if let Some(result) = checked_ids.get(&id_hash) {\n                            input = (*result).into();\n                        } else {\n                            let exists = self\n                                .in_memory_store()\n                                .key_get::<()>(id_hash.key())\n                                .await\n                                .caused_by(trc::location!())?\n                                .is_some();\n\n                            if !exists || last {\n                                self.in_memory_store()\n                                    .key_set(KeyValue::new(id_hash.key(), vec![]).expires(expiry))\n                                    .await\n                                    .caused_by(trc::location!())?;\n                            }\n\n                            checked_ids.insert(id_hash, exists);\n                            input = exists.into();\n                        }\n                    }\n                    Event::Discard => {\n                        do_discard = true;\n                        input = true.into();\n                    }\n                    Event::Reject { reason, .. } => {\n                        reject_reason = reason.into();\n                        do_discard = true;\n                        input = true.into();\n                    }\n                    Event::Keep { flags, message_id } => {\n                        if let Some(message) = messages.get_mut(message_id) {\n                            message.flags = flags.into_iter().map(Keyword::from).collect();\n                            if !message.file_into.contains(&INBOX_ID) {\n                                message.file_into.push(INBOX_ID);\n                            }\n                            do_deliver = true;\n                        } else {\n                            trc::event!(\n                                Sieve(SieveEvent::UnexpectedError),\n                                Details = \"Unknown message id.\",\n                                MessageId = message_id,\n                                SpanId = session_id\n                            );\n                        }\n                        input = true.into();\n                    }\n                    Event::FileInto {\n                        folder,\n                        flags,\n                        mailbox_id,\n                        special_use,\n                        create,\n                        message_id,\n                    } => {\n                        let mut target_id = u32::MAX;\n\n                        // Find mailbox by Id\n                        if let Some(mailbox_id) = mailbox_id.and_then(|m| Id::from_str(&m).ok()) {\n                            let mailbox_id = mailbox_id.document_id();\n                            if cache.has_mailbox_id(&mailbox_id) {\n                                target_id = mailbox_id;\n                            }\n                        }\n\n                        // Find mailbox by role\n                        if let Some(special_use) = special_use\n                            && target_id == u32::MAX\n                        {\n                            if special_use.eq_ignore_ascii_case(\"inbox\") {\n                                target_id = INBOX_ID;\n                            } else if special_use.eq_ignore_ascii_case(\"trash\") {\n                                target_id = TRASH_ID;\n                            } else if let Ok(role) = SpecialUse::parse_value(&special_use)\n                                && let Some(item) = cache.mailbox_by_role(&role)\n                            {\n                                target_id = item.document_id;\n                            }\n                        }\n\n                        // Find mailbox by name\n                        if target_id == u32::MAX {\n                            if !create {\n                                if let Some(m) = cache.mailbox_by_path(&folder) {\n                                    target_id = m.document_id;\n                                }\n                            } else if let Some(document_id) = self\n                                .mailbox_create_path(account_id, &folder)\n                                .await\n                                .caused_by(trc::location!())?\n                            {\n                                cache = self\n                                    .get_cached_messages(account_id)\n                                    .await\n                                    .caused_by(trc::location!())?;\n                                target_id = document_id;\n                            }\n                        }\n\n                        // Default to Inbox\n                        if target_id == u32::MAX {\n                            target_id = INBOX_ID;\n                        }\n\n                        if let Some(message) = messages.get_mut(message_id) {\n                            message.flags = flags.into_iter().map(Keyword::from).collect();\n                            if !message.file_into.contains(&target_id) {\n                                message.file_into.push(target_id);\n                            }\n                            message.did_file_into = true;\n                            do_deliver = true;\n                        } else {\n                            trc::event!(\n                                Sieve(SieveEvent::UnexpectedError),\n                                Details = \"Unknown message id.\",\n                                MessageId = message_id,\n                                SpanId = session_id\n                            );\n                        }\n                        input = true.into();\n                    }\n                    Event::SendMessage {\n                        recipient,\n                        message_id,\n                        ..\n                    } => {\n                        input = true.into();\n                        if let Some(message) = messages.get(message_id) {\n                            let recipients: Vec<String> = match recipient {\n                                Recipient::Address(rcpt) => vec![rcpt],\n                                Recipient::Group(rcpts) => rcpts,\n                                Recipient::List(_) => {\n                                    // Not yet implemented\n                                    continue;\n                                }\n                            };\n\n                            if message.raw_message.len() <= self.core.jmap.mail_max_size {\n                                trc::event!(\n                                    Sieve(SieveEvent::SendMessage),\n                                    From = mail_from.clone(),\n                                    To = recipients\n                                        .iter()\n                                        .map(|r| trc::Value::String(r.as_str().into()))\n                                        .collect::<Vec<_>>(),\n                                    Size = message.raw_message.len(),\n                                    SpanId = session_id\n                                );\n\n                                autogenerated.push(AutogeneratedMessage {\n                                    sender_address: mail_from.clone(),\n                                    recipients,\n                                    message: message.raw_message.to_vec(),\n                                });\n                            } else {\n                                trc::event!(\n                                    Sieve(SieveEvent::MessageTooLarge),\n                                    From = mail_from.clone(),\n                                    To = recipients\n                                        .iter()\n                                        .map(|r| trc::Value::String(r.as_str().into()))\n                                        .collect::<Vec<_>>(),\n                                    Size = message.raw_message.len(),\n                                    Limit = self.core.jmap.mail_max_size,\n                                    SpanId = session_id,\n                                );\n                            }\n                        } else {\n                            trc::event!(\n                                Sieve(SieveEvent::UnexpectedError),\n                                Details = \"Unknown message id.\",\n                                MessageId = message_id,\n                                SpanId = session_id\n                            );\n\n                            continue;\n                        }\n                    }\n                    Event::ListContains { .. }\n                    | Event::Notify { .. }\n                    | Event::SetEnvelope { .. } => {\n                        // Not allowed\n                        input = false.into();\n                    }\n                    Event::Function { id, arguments } => {\n                        input = self\n                            .core\n                            .run_plugin(\n                                id,\n                                PluginContext {\n                                    session_id,\n                                    server: self,\n                                    message: instance.message(),\n                                    modifications: &mut Vec::new(),\n                                    access_token: access_token.into(),\n                                    arguments,\n                                },\n                            )\n                            .await;\n                    }\n                    Event::CreatedMessage { message, .. } => {\n                        messages.push(SieveMessage {\n                            raw_message: message.into(),\n                            file_into: Vec::new(),\n                            flags: Vec::new(),\n                            did_file_into: false,\n                        });\n                        input = true.into();\n                    }\n                },\n\n                #[cfg(feature = \"test_mode\")]\n                Err(sieve::runtime::RuntimeError::ScriptErrorMessage(err)) => {\n                    panic!(\"Sieve test failed: {}\", err);\n                }\n\n                Err(err) => {\n                    trc::event!(\n                        Sieve(SieveEvent::RuntimeError),\n                        Reason = err.to_string(),\n                        SpanId = session_id\n                    );\n\n                    input = true.into();\n                }\n            }\n        }\n\n        // Fail-safe, no discard and no keep seen, assume that something went wrong and file anyway.\n        if !do_deliver && !do_discard {\n            messages[0].file_into.push(INBOX_ID);\n        }\n\n        // Deliver messages\n        let mut last_temp_error = None;\n        let mut has_delivered = false;\n        for (message_id, sieve_message) in messages.into_iter().enumerate() {\n            if !sieve_message.file_into.is_empty() {\n                // Parse message if needed\n                let (blob_hash, message) = if message_id == 0 && !instance.has_message_changed() {\n                    (blob_hash.into(), instance.take_message())\n                } else if let Some(message) =\n                    MessageParser::new().parse(sieve_message.raw_message.as_ref())\n                {\n                    (None, message)\n                } else {\n                    trc::event!(\n                        Sieve(SieveEvent::UnexpectedError),\n                        Details = \"Failed to parse Sieve generated message.\",\n                        SpanId = session_id\n                    );\n\n                    continue;\n                };\n\n                // Deliver message\n                match self\n                    .email_ingest(IngestEmail {\n                        raw_message: &sieve_message.raw_message,\n                        blob_hash,\n                        message: message.into(),\n                        access_token,\n                        mailbox_ids: sieve_message.file_into,\n                        keywords: sieve_message.flags,\n                        received_at: None,\n                        source: IngestSource::Smtp {\n                            deliver_to: envelope_to.address.as_str(),\n                            is_sender_authenticated: envelope_from_authenticated,\n                            is_spam: envelope_to.is_spam,\n                        },\n                        session_id,\n                    })\n                    .await\n                {\n                    Ok(ingested_message_) => {\n                        has_delivered = true;\n                        ingested_message = ingested_message_;\n                    }\n                    Err(err) => {\n                        last_temp_error = err.into();\n                    }\n                }\n            }\n        }\n\n        if let Some(reject_reason) = reject_reason {\n            Err(\n                trc::EventType::MessageIngest(trc::MessageIngestEvent::Error)\n                    .ctx(trc::Key::Code, 571)\n                    .ctx(trc::Key::Reason, reject_reason),\n            )\n        } else if has_delivered || last_temp_error.is_none() {\n            Ok(ingested_message)\n        } else {\n            // There were problems during delivery\n            #[allow(clippy::unnecessary_unwrap)]\n            Err(last_temp_error.unwrap())\n        }\n    }\n\n    async fn sieve_script_get_active_id(&self, account_id: u32) -> trc::Result<Option<u32>> {\n        self.store()\n            .get_value::<u32>(ValueKey {\n                account_id,\n                collection: Collection::Principal.into(),\n                document_id: 0,\n                class: ValueClass::Property(PrincipalField::ActiveScriptId.into()),\n            })\n            .await\n            .caused_by(trc::location!())\n    }\n\n    async fn sieve_script_get_active(&self, account_id: u32) -> trc::Result<Option<ActiveScript>> {\n        // Find the currently active script\n        if let Some(document_id) = self\n            .store()\n            .get_value::<u32>(ValueKey {\n                account_id,\n                collection: Collection::Principal.into(),\n                document_id: 0,\n                class: ValueClass::Property(PrincipalField::ActiveScriptId.into()),\n            })\n            .await\n            .caused_by(trc::location!())?\n        {\n            if let Some(script) = self.sieve_script_compile(account_id, document_id).await? {\n                Ok(Some(ActiveScript {\n                    document_id,\n                    script: Arc::new(script.script),\n                    script_name: script.name,\n                    version: script.version,\n                }))\n            } else {\n                Ok(None)\n            }\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn sieve_script_get_by_name(\n        &self,\n        account_id: u32,\n        name: &str,\n    ) -> trc::Result<Option<Sieve>> {\n        // Find the script by name\n        if let Some(document_id) = self\n            .document_ids_matching(\n                account_id,\n                Collection::SieveScript,\n                SieveField::Name,\n                name.as_bytes(),\n            )\n            .await\n            .caused_by(trc::location!())?\n            .min()\n        {\n            self.sieve_script_compile(account_id, document_id)\n                .await\n                .map(|script| script.map(|s| s.script))\n        } else {\n            Ok(None)\n        }\n    }\n\n    #[allow(clippy::blocks_in_conditions)]\n    async fn sieve_script_compile(\n        &self,\n        account_id: u32,\n        document_id: u32,\n    ) -> trc::Result<Option<CompiledScript>> {\n        // Obtain script object\n        let Some(script_object) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::SieveScript,\n                document_id,\n            ))\n            .await?\n        else {\n            return Ok(None);\n        };\n\n        // Obtain the sieve script length\n        let version = script_object.version;\n        let unarchived_script = script_object\n            .unarchive::<SieveScript>()\n            .caused_by(trc::location!())?;\n        let script_offset = u32::from(unarchived_script.size) as usize;\n\n        // Obtain the sieve script blob\n        let script_bytes = self\n            .core\n            .storage\n            .blob\n            .get_blob(unarchived_script.blob_hash.0.as_ref(), 0..usize::MAX)\n            .await\n            .caused_by(trc::location!())?\n            .ok_or_else(|| {\n                trc::StoreEvent::NotFound\n                    .into_err()\n                    .caused_by(trc::location!())\n                    .document_id(document_id)\n            })?;\n\n        // Obtain the precompiled script\n        if let Some(script) = script_bytes.get(script_offset..).and_then(|bytes| {\n            <Archive<AlignedBytes> as Deserialize>::deserialize(bytes)\n                .ok()?\n                .deserialize::<Sieve>()\n                .ok()\n        }) {\n            Ok(Some(CompiledScript {\n                script,\n                name: unarchived_script.name.as_str().into(),\n                version,\n            }))\n        } else {\n            // Deserialization failed, probably because the script compiler version changed\n            match self.core.sieve.untrusted_compiler.compile(\n                script_bytes.get(0..script_offset).ok_or_else(|| {\n                    trc::StoreEvent::NotFound\n                        .into_err()\n                        .caused_by(trc::location!())\n                        .document_id(document_id)\n                })?,\n            ) {\n                Ok(sieve) => {\n                    // Store updated compiled sieve script\n                    let sieve = Archiver::new(sieve).untrusted();\n                    let compiled_bytes = sieve.serialize().caused_by(trc::location!())?;\n                    let mut updated_sieve_bytes =\n                        Vec::with_capacity(script_offset + compiled_bytes.len());\n                    updated_sieve_bytes.extend_from_slice(&script_bytes[0..script_offset]);\n                    updated_sieve_bytes.extend_from_slice(&compiled_bytes);\n\n                    // Store updated blob\n                    let (new_blob_hash, new_blob_hold) = self\n                        .put_temporary_blob(account_id, &updated_sieve_bytes, 60)\n                        .await?;\n                    let mut new_script_object =\n                        rkyv::deserialize(unarchived_script).caused_by(trc::location!())?;\n                    let blob_hash =\n                        std::mem::replace(&mut new_script_object.blob_hash, new_blob_hash.clone());\n                    let new_archive = Archiver::new(new_script_object);\n\n                    // Update script object\n                    let mut batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(account_id)\n                        .with_collection(Collection::SieveScript)\n                        .with_document(document_id)\n                        .assert_value(SieveField::Archive, &script_object)\n                        .set(\n                            SieveField::Archive,\n                            new_archive.serialize().caused_by(trc::location!())?,\n                        )\n                        .clear(BlobOp::Link {\n                            hash: blob_hash,\n                            to: BlobLink::Document,\n                        })\n                        .set(\n                            BlobOp::Link {\n                                hash: new_blob_hash,\n                                to: BlobLink::Document,\n                            },\n                            Vec::new(),\n                        )\n                        .clear(new_blob_hold);\n                    self.store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n\n                    Ok(Some(CompiledScript {\n                        script: sieve.into_inner(),\n                        name: new_archive.into_inner().name,\n                        version,\n                    }))\n                }\n                Err(error) => Err(trc::StoreEvent::UnexpectedError\n                    .caused_by(trc::location!())\n                    .reason(error)\n                    .details(\"Failed to compile Sieve script\")),\n            }\n        }\n    }\n}\n\npub struct CompiledScript {\n    pub script: Sieve,\n    pub name: String,\n    pub version: ArchiveVersion,\n}\n"
  },
  {
    "path": "crates/email/src/sieve/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::KV_SIEVE_ID;\nuse sieve::Sieve;\nuse std::sync::Arc;\nuse store::{blake3, write::ArchiveVersion};\nuse types::blob_hash::BlobHash;\n\npub mod delete;\npub mod index;\npub mod ingest;\n\n#[derive(Debug, Clone)]\npub struct ActiveScript {\n    pub document_id: u32,\n    pub version: ArchiveVersion,\n    pub script_name: String,\n    pub script: Arc<Sieve>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct SieveScript {\n    pub name: String,\n    pub blob_hash: BlobHash,\n    pub size: u32,\n    pub vacation_response: Option<VacationResponse>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct VacationResponse {\n    pub from_date: Option<u64>,\n    pub to_date: Option<u64>,\n    pub subject: Option<String>,\n    pub text_body: Option<String>,\n    pub html_body: Option<String>,\n}\n\nimpl SieveScript {\n    pub fn new(name: impl Into<String>, blob_hash: BlobHash) -> Self {\n        SieveScript {\n            name: name.into(),\n            blob_hash,\n            vacation_response: None,\n            size: 0,\n        }\n    }\n\n    pub fn with_name(mut self, name: impl Into<String>) -> Self {\n        self.name = name.into();\n        self\n    }\n\n    pub fn with_blob_hash(mut self, blob_hash: BlobHash) -> Self {\n        self.blob_hash = blob_hash;\n        self\n    }\n\n    pub fn with_size(mut self, size: u32) -> Self {\n        self.size = size;\n        self\n    }\n\n    pub fn with_vacation_response(mut self, vacation_response: VacationResponse) -> Self {\n        self.vacation_response = Some(vacation_response);\n        self\n    }\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, Hash)]\n#[repr(transparent)]\npub struct SeenIdHash(pub [u8; 32]);\n\nimpl SeenIdHash {\n    pub fn new(account_id: u32, hash: u32, id: &str) -> Self {\n        let mut hasher = blake3::Hasher::new();\n        hasher.update(&account_id.to_be_bytes());\n        hasher.update(&hash.to_be_bytes());\n        hasher.update(id.as_bytes());\n        SeenIdHash(hasher.finalize().into())\n    }\n\n    pub fn key(&self) -> Vec<u8> {\n        let mut result = Vec::with_capacity(self.0.len() + 1);\n        result.push(KV_SIEVE_ID);\n        result.extend_from_slice(&self.0);\n        result\n    }\n}\n\nimpl AsRef<[u8]> for SeenIdHash {\n    fn as_ref(&self) -> &[u8] {\n        &self.0\n    }\n}\n"
  },
  {
    "path": "crates/email/src/submission/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ArchivedEmailSubmission, EmailSubmission};\nuse common::storage::index::{IndexValue, IndexableAndSerializableObject, IndexableObject};\nuse store::{\n    U32_LEN,\n    write::{IndexPropertyClass, ValueClass, key::KeySerializer},\n};\nuse types::{collection::SyncCollection, field::EmailSubmissionField};\n\nimpl IndexableObject for EmailSubmission {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Property {\n                field: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                    property: EmailSubmissionField::Metadata.into(),\n                    value: self.send_at,\n                }),\n                value: KeySerializer::new(U32_LEN * 3 + 1)\n                    .write(self.email_id)\n                    .write(self.thread_id)\n                    .write(self.identity_id)\n                    .write(self.undo_status.as_index())\n                    .finalize()\n                    .into(),\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::EmailSubmission,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedEmailSubmission {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Property {\n                field: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                    property: EmailSubmissionField::Metadata.into(),\n                    value: self.send_at.to_native(),\n                }),\n                value: KeySerializer::new(U32_LEN * 3 + 1)\n                    .write(self.email_id.to_native())\n                    .write(self.thread_id.to_native())\n                    .write(self.identity_id.to_native())\n                    .write(self.undo_status.as_index())\n                    .finalize()\n                    .into(),\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::EmailSubmission,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for EmailSubmission {\n    fn is_versioned() -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/email/src/submission/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse utils::map::vec_map::VecMap;\n\npub mod index;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct EmailSubmission {\n    pub email_id: u32,\n    pub thread_id: u32,\n    pub identity_id: u32,\n    pub send_at: u64,\n    pub queue_id: Option<u64>,\n    pub undo_status: UndoStatus,\n    pub envelope: Envelope,\n    pub delivery_status: VecMap<String, DeliveryStatus>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct Envelope {\n    pub mail_from: Address,\n    pub rcpt_to: Vec<Address>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct Address {\n    pub email: String,\n    pub parameters: Option<VecMap<String, Option<String>>>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct DeliveryStatus {\n    pub smtp_reply: String,\n    pub delivered: Delivered,\n    pub displayed: bool,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub enum Delivered {\n    Queued,\n    Yes,\n    No,\n    #[default]\n    Unknown,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub enum UndoStatus {\n    #[default]\n    Pending,\n    Final,\n    Canceled,\n}\n\nimpl UndoStatus {\n    pub fn parse(s: &str) -> Option<Self> {\n        hashify::tiny_map!(s.as_bytes(),\n            \"pending\" => UndoStatus::Pending,\n            \"final\" => UndoStatus::Final,\n            \"canceled\" => UndoStatus::Canceled,\n            \"cancelled\" => UndoStatus::Canceled,\n        )\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            UndoStatus::Pending => \"pending\",\n            UndoStatus::Final => \"final\",\n            UndoStatus::Canceled => \"canceled\",\n        }\n    }\n\n    pub fn as_index(&self) -> u8 {\n        match self {\n            UndoStatus::Pending => b'p',\n            UndoStatus::Final => b'f',\n            UndoStatus::Canceled => b'c',\n        }\n    }\n}\n\nimpl ArchivedUndoStatus {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            ArchivedUndoStatus::Pending => \"pending\",\n            ArchivedUndoStatus::Final => \"final\",\n            ArchivedUndoStatus::Canceled => \"canceled\",\n        }\n    }\n\n    pub fn as_index(&self) -> u8 {\n        match self {\n            ArchivedUndoStatus::Pending => b'p',\n            ArchivedUndoStatus::Final => b'f',\n            ArchivedUndoStatus::Canceled => b'c',\n        }\n    }\n}\n\nimpl From<&ArchivedDeliveryStatus> for DeliveryStatus {\n    fn from(value: &ArchivedDeliveryStatus) -> Self {\n        DeliveryStatus {\n            smtp_reply: value.smtp_reply.to_string(),\n            delivered: match value.delivered {\n                ArchivedDelivered::Queued => Delivered::Queued,\n                ArchivedDelivered::Yes => Delivered::Yes,\n                ArchivedDelivered::No => Delivered::No,\n                ArchivedDelivered::Unknown => Delivered::Unknown,\n            },\n            displayed: value.displayed,\n        }\n    }\n}\n\nimpl Delivered {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Delivered::Queued => \"queued\",\n            Delivered::Yes => \"yes\",\n            Delivered::No => \"no\",\n            Delivered::Unknown => \"unknown\",\n        }\n    }\n}\n\nimpl ArchivedDelivered {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            ArchivedDelivered::Queued => \"queued\",\n            ArchivedDelivered::Yes => \"yes\",\n            ArchivedDelivered::No => \"no\",\n            ArchivedDelivered::Unknown => \"unknown\",\n        }\n    }\n}\n"
  },
  {
    "path": "crates/groupware/Cargo.toml",
    "content": "[package]\nname = \"groupware\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path = \"../utils\" }\nstore = { path = \"../store\" }\ncommon = { path =  \"../common\" }\ntypes = { path = \"../types\" }\ntrc = { path = \"../trc\" }\nnlp = { path = \"../nlp\" }\ndirectory = { path =  \"../directory\" }\ncalcard = { version = \"0.3\", features = [\"rkyv\"] }\nhashify = \"0.2\"\ntokio = { version = \"1.47\", features = [\"net\", \"macros\"] }\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\npercent-encoding = \"2.3.1\"\ncompact_str = \"0.9.0\"\nahash = { version = \"0.8\" }\nchrono = \"0.4.40\"\n\n[features]\ntest_mode = []\nenterprise = []\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/groupware/src/cache/calcard.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::GroupwareCache;\nuse crate::{\n    DavResourceName, RFC_3986,\n    calendar::{\n        ArchivedCalendar, ArchivedCalendarEvent, Calendar, CalendarEvent, SCHEDULE_INBOX_ID,\n        SCHEDULE_OUTBOX_ID, storage::ItipAutoExpunge,\n    },\n    contact::{AddressBook, ArchivedAddressBook, ArchivedContactCard, ContactCard},\n};\nuse calcard::common::timezone::Tz;\nuse common::{\n    DavName, DavPath, DavResource, DavResourceMetadata, DavResources, Server,\n    TinyCalendarPreferences, auth::AccessToken,\n};\nuse directory::backend::internal::manage::ManageDirectory;\nuse std::sync::Arc;\nuse store::ahash::{AHashMap, AHashSet};\nuse tokio::sync::Semaphore;\nuse trc::AddContext;\nuse types::{\n    acl::AclGrant,\n    collection::{Collection, SyncCollection},\n};\nuse utils::map::bitmap::Bitmap;\n\npub(super) async fn build_calcard_resources(\n    server: &Server,\n    access_token: &AccessToken,\n    account_id: u32,\n    sync_collection: SyncCollection,\n    container_collection: Collection,\n    item_collection: Collection,\n    update_lock: Arc<Semaphore>,\n) -> trc::Result<DavResources> {\n    let is_calendar = matches!(sync_collection, SyncCollection::Calendar);\n    let name = server\n        .store()\n        .get_principal_name(account_id)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_else(|| format!(\"_{account_id}\"));\n    let mut cache = DavResources {\n        base_path: format!(\n            \"{}/{}/\",\n            if is_calendar {\n                DavResourceName::Cal\n            } else {\n                DavResourceName::Card\n            }\n            .base_path(),\n            percent_encoding::utf8_percent_encode(&name, RFC_3986),\n        ),\n        paths: AHashSet::with_capacity(16),\n        resources: Vec::with_capacity(16),\n        item_change_id: 0,\n        container_change_id: 0,\n        highest_change_id: 0,\n        size: std::mem::size_of::<DavResources>() as u64,\n        update_lock,\n    };\n\n    let mut is_first_check = true;\n    loop {\n        let last_change_id = server\n            .core\n            .storage\n            .data\n            .get_last_change_id(account_id, sync_collection.into())\n            .await\n            .caused_by(trc::location!())?\n            .unwrap_or_default();\n        cache.item_change_id = last_change_id;\n        cache.container_change_id = last_change_id;\n        cache.highest_change_id = last_change_id;\n\n        server\n            .archives(\n                account_id,\n                container_collection,\n                &(),\n                |document_id, archive| {\n                    let resource = if is_calendar {\n                        resource_from_calendar(archive.unarchive::<Calendar>()?, document_id)\n                    } else {\n                        resource_from_addressbook(archive.unarchive::<AddressBook>()?, document_id)\n                    };\n                    let path = DavPath {\n                        path: resource.container_name().unwrap().to_string(),\n                        parent_id: None,\n                        hierarchy_seq: 1,\n                        resource_idx: cache.resources.len(),\n                    };\n\n                    cache.size += (std::mem::size_of::<DavPath>()\n                        + std::mem::size_of::<DavResource>()\n                        + (path.path.len()) * 2) as u64;\n                    cache.paths.insert(path);\n                    cache.resources.push(resource);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if cache.paths.is_empty() {\n            if is_first_check {\n                if is_calendar {\n                    server\n                        .create_default_calendar(access_token, account_id, &name)\n                        .await?;\n                } else {\n                    server\n                        .create_default_addressbook(access_token, account_id, &name)\n                        .await?;\n                }\n                is_first_check = false;\n                continue;\n            } else {\n                return Ok(cache);\n            }\n        }\n\n        let parent_range = cache.resources.len();\n        server\n            .archives(account_id, item_collection, &(), |document_id, archive| {\n                let resource = if is_calendar {\n                    resource_from_event(archive.unarchive::<CalendarEvent>()?, document_id)\n                } else {\n                    resource_from_card(archive.unarchive::<ContactCard>()?, document_id)\n                };\n                let resource_idx = cache.resources.len();\n\n                for name in resource.child_names().unwrap_or_default().iter() {\n                    if let Some(parent) =\n                        cache.resources.get(..parent_range).and_then(|resources| {\n                            resources.iter().find(|r| r.document_id == name.parent_id)\n                        })\n                    {\n                        let path = DavPath {\n                            path: format!(\"{}/{}\", parent.container_name().unwrap(), name.name),\n                            parent_id: Some(name.parent_id),\n                            hierarchy_seq: 0,\n                            resource_idx,\n                        };\n\n                        cache.size += (std::mem::size_of::<DavPath>()\n                            + name.name.len()\n                            + path.path.len()) as u64;\n                        cache.paths.insert(path);\n                    }\n                }\n                cache.size += std::mem::size_of::<DavResource>() as u64;\n                cache.resources.push(resource);\n\n                Ok(true)\n            })\n            .await\n            .caused_by(trc::location!())?;\n\n        return Ok(cache);\n    }\n}\n\npub(super) async fn build_scheduling_resources(\n    server: &Server,\n    account_id: u32,\n    update_lock: Arc<Semaphore>,\n) -> trc::Result<DavResources> {\n    let last_change_id = server\n        .core\n        .storage\n        .data\n        .get_last_change_id(account_id, SyncCollection::CalendarEventNotification.into())\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n\n    let name = server\n        .store()\n        .get_principal_name(account_id)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_else(|| format!(\"_{account_id}\"));\n\n    let item_ids = server\n        .itip_ids(account_id)\n        .await\n        .caused_by(trc::location!())?;\n\n    let mut cache = DavResources {\n        base_path: format!(\n            \"{}/{}/\",\n            DavResourceName::Scheduling.base_path(),\n            percent_encoding::utf8_percent_encode(&name, RFC_3986),\n        ),\n        paths: AHashSet::with_capacity((2 + item_ids.len()) as usize),\n        resources: Vec::with_capacity((2 + item_ids.len()) as usize),\n        item_change_id: last_change_id,\n        container_change_id: last_change_id,\n        highest_change_id: last_change_id,\n        size: std::mem::size_of::<DavResources>() as u64,\n        update_lock,\n    };\n\n    for (document_id, is_container) in item_ids\n        .into_iter()\n        .map(|document_id| (document_id, false))\n        .chain([(SCHEDULE_INBOX_ID, true), (SCHEDULE_OUTBOX_ID, true)])\n    {\n        let path = path_from_scheduling(document_id, cache.resources.len(), is_container);\n        cache.size += (std::mem::size_of::<DavPath>() + (path.path.len() * 2)) as u64\n            + std::mem::size_of::<DavResource>() as u64;\n        cache.paths.insert(path);\n        cache\n            .resources\n            .push(resource_from_scheduling(document_id, is_container));\n    }\n\n    Ok(cache)\n}\n\npub(super) fn build_simple_hierarchy(cache: &mut DavResources) {\n    cache.paths = AHashSet::with_capacity(cache.resources.len());\n    let name_idx = cache\n        .resources\n        .iter()\n        .filter_map(|resource| {\n            resource\n                .container_name()\n                .map(|name| (resource.document_id, name))\n        })\n        .collect::<AHashMap<_, _>>();\n\n    for (resource_idx, resource) in cache.resources.iter().enumerate() {\n        match &resource.data {\n            DavResourceMetadata::Calendar { name, .. }\n            | DavResourceMetadata::AddressBook { name, .. } => {\n                let path = DavPath {\n                    path: name.to_string(),\n                    parent_id: None,\n                    hierarchy_seq: 1,\n                    resource_idx,\n                };\n                cache.size +=\n                    (std::mem::size_of::<DavPath>() + name.len() + path.path.len()) as u64;\n                cache.paths.insert(path);\n            }\n            DavResourceMetadata::CalendarEvent { names, .. }\n            | DavResourceMetadata::ContactCard { names } => {\n                for name in names {\n                    if let Some(parent_name) = name_idx.get(&name.parent_id) {\n                        let path = DavPath {\n                            path: format!(\"{parent_name}/{}\", name.name),\n                            parent_id: Some(name.parent_id),\n                            hierarchy_seq: 0,\n                            resource_idx,\n                        };\n                        cache.size += (std::mem::size_of::<DavPath>()\n                            + name.name.len()\n                            + path.path.len()) as u64;\n                        cache.paths.insert(path);\n                    }\n                }\n            }\n            _ => unreachable!(),\n        }\n        cache.size += std::mem::size_of::<DavResource>() as u64;\n    }\n}\n\npub(super) fn resource_from_calendar(calendar: &ArchivedCalendar, document_id: u32) -> DavResource {\n    DavResource {\n        document_id,\n        data: DavResourceMetadata::Calendar {\n            name: calendar.name.to_string(),\n            acls: calendar\n                .acls\n                .iter()\n                .map(|acl| AclGrant {\n                    account_id: acl.account_id.to_native(),\n                    grants: Bitmap::from(&acl.grants),\n                })\n                .collect(),\n            preferences: calendar\n                .preferences\n                .iter()\n                .map(|pref| TinyCalendarPreferences {\n                    account_id: pref.account_id.to_native(),\n                    flags: pref.flags.to_native(),\n                    tz: pref.time_zone.tz().unwrap_or(Tz::UTC),\n                })\n                .collect(),\n        },\n    }\n}\n\npub(super) fn resource_from_event(event: &ArchivedCalendarEvent, document_id: u32) -> DavResource {\n    let (start, duration) = event.data.event_range().unwrap_or_default();\n    DavResource {\n        document_id,\n        data: DavResourceMetadata::CalendarEvent {\n            names: event\n                .names\n                .iter()\n                .map(|name| DavName {\n                    name: name.name.to_string(),\n                    parent_id: name.parent_id.to_native(),\n                })\n                .collect(),\n            start,\n            duration,\n        },\n    }\n}\n\npub(super) fn resource_from_scheduling(document_id: u32, is_container: bool) -> DavResource {\n    DavResource {\n        document_id,\n        data: DavResourceMetadata::CalendarEventNotification {\n            names: if !is_container {\n                [DavName {\n                    name: format!(\"{document_id}.ics\"),\n                    parent_id: SCHEDULE_INBOX_ID,\n                }]\n                .into_iter()\n                .collect()\n            } else {\n                Default::default()\n            },\n        },\n    }\n}\n\npub(super) fn path_from_scheduling(\n    document_id: u32,\n    resource_idx: usize,\n    is_container: bool,\n) -> DavPath {\n    if is_container {\n        DavPath {\n            path: if document_id == SCHEDULE_INBOX_ID {\n                \"inbox\".to_string()\n            } else {\n                \"outbox\".to_string()\n            },\n            parent_id: None,\n            hierarchy_seq: 1,\n            resource_idx,\n        }\n    } else {\n        DavPath {\n            path: format!(\"inbox/{document_id}.ics\"),\n            parent_id: Some(SCHEDULE_INBOX_ID),\n            hierarchy_seq: 0,\n            resource_idx,\n        }\n    }\n}\n\npub(super) fn resource_from_addressbook(\n    book: &ArchivedAddressBook,\n    document_id: u32,\n) -> DavResource {\n    DavResource {\n        document_id,\n        data: DavResourceMetadata::AddressBook {\n            name: book.name.to_string(),\n            acls: book\n                .acls\n                .iter()\n                .map(|acl| AclGrant {\n                    account_id: acl.account_id.to_native(),\n                    grants: Bitmap::from(&acl.grants),\n                })\n                .collect(),\n        },\n    }\n}\n\npub(super) fn resource_from_card(card: &ArchivedContactCard, document_id: u32) -> DavResource {\n    DavResource {\n        document_id,\n        data: DavResourceMetadata::ContactCard {\n            names: card\n                .names\n                .iter()\n                .map(|name| DavName {\n                    name: name.name.to_string(),\n                    parent_id: name.parent_id.to_native(),\n                })\n                .collect(),\n        },\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/cache/file.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    DavResourceName, RFC_3986,\n    file::{ArchivedFileNode, FileNode},\n};\nuse common::{DavPath, DavResource, DavResourceMetadata, DavResources, Server};\nuse directory::backend::internal::manage::ManageDirectory;\nuse std::sync::Arc;\nuse store::ahash::{AHashMap, AHashSet};\nuse tokio::sync::Semaphore;\nuse trc::AddContext;\nuse types::{\n    acl::AclGrant,\n    collection::{Collection, SyncCollection},\n};\nuse utils::{map::bitmap::Bitmap, topological::TopologicalSort};\n\npub(super) async fn build_file_resources(\n    server: &Server,\n    account_id: u32,\n    update_lock: Arc<Semaphore>,\n) -> trc::Result<DavResources> {\n    let last_change_id = server\n        .core\n        .storage\n        .data\n        .get_last_change_id(account_id, SyncCollection::FileNode.into())\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let name = server\n        .store()\n        .get_principal_name(account_id)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_else(|| format!(\"_{account_id}\"));\n\n    let mut resources = Vec::with_capacity(16);\n    server\n        .archives(\n            account_id,\n            Collection::FileNode,\n            &(),\n            |document_id, archive| {\n                resources.push(resource_from_file(\n                    archive.unarchive::<FileNode>()?,\n                    document_id,\n                ));\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    let mut files = DavResources {\n        base_path: format!(\n            \"{}/{}/\",\n            DavResourceName::File.base_path(),\n            percent_encoding::utf8_percent_encode(&name, RFC_3986),\n        ),\n        size: std::mem::size_of::<DavResources>() as u64,\n        paths: AHashSet::with_capacity(resources.len()),\n        resources,\n        item_change_id: last_change_id,\n        container_change_id: last_change_id,\n        highest_change_id: last_change_id,\n        update_lock,\n    };\n\n    build_nested_hierarchy(&mut files);\n\n    Ok(files)\n}\n\npub(super) fn build_nested_hierarchy(resources: &mut DavResources) {\n    let mut topological_sort = TopologicalSort::with_capacity(resources.resources.len());\n    let mut names = AHashMap::with_capacity(resources.resources.len());\n\n    for (resource_idx, resource) in resources.resources.iter().enumerate() {\n        if let DavResourceMetadata::File { parent_id, .. } = resource.data {\n            topological_sort.insert(\n                parent_id.map(|id| id + 1).unwrap_or_default(),\n                resource.document_id + 1,\n            );\n            names.insert(\n                resource.document_id,\n                DavPath {\n                    path: resource.container_name().unwrap().to_string(),\n                    parent_id,\n                    hierarchy_seq: 0,\n                    resource_idx,\n                },\n            );\n        }\n    }\n\n    for (hierarchy_sequence, folder_id) in topological_sort.into_iterator().enumerate() {\n        if folder_id != 0 {\n            let folder_id = folder_id - 1;\n            if let Some((name, parent_name)) = names\n                .get(&folder_id)\n                .and_then(|folder| folder.parent_id.map(|parent_id| (&folder.path, parent_id)))\n                .and_then(|(name, parent_id)| {\n                    names.get(&parent_id).map(|folder| (name, &folder.path))\n                })\n            {\n                let name = format!(\"{parent_name}/{name}\");\n                let folder = names.get_mut(&folder_id).unwrap();\n                folder.path = name;\n                folder.hierarchy_seq = hierarchy_sequence as u32;\n            } else {\n                names.get_mut(&folder_id).unwrap().hierarchy_seq = hierarchy_sequence as u32;\n            }\n        }\n    }\n\n    resources.paths = names\n        .into_values()\n        .inspect(|v| {\n            resources.size += (std::mem::size_of::<DavPath>()\n                + std::mem::size_of::<u32>()\n                + std::mem::size_of::<usize>()\n                + std::mem::size_of::<DavResource>()\n                + v.path.len()) as u64;\n        })\n        .collect();\n}\n\npub(super) fn resource_from_file(node: &ArchivedFileNode, document_id: u32) -> DavResource {\n    let parent_id = node.parent_id.to_native();\n    DavResource {\n        document_id,\n        data: DavResourceMetadata::File {\n            name: node.name.as_str().to_string(),\n            size: node.file.as_ref().map(|f| f.size.to_native()),\n            parent_id: if parent_id > 0 {\n                Some(parent_id - 1)\n            } else {\n                None\n            },\n            acls: node\n                .acls\n                .iter()\n                .map(|acl| AclGrant {\n                    account_id: acl.account_id.to_native(),\n                    grants: Bitmap::from(&acl.grants),\n                })\n                .collect(),\n        },\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/cache/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    cache::calcard::{build_scheduling_resources, path_from_scheduling, resource_from_scheduling},\n    calendar::{Calendar, CalendarEvent, CalendarPreferences},\n    contact::{AddressBook, AddressBookPreferences, ContactCard},\n    file::FileNode,\n};\nuse ahash::AHashSet;\nuse calcard::{\n    build_calcard_resources, build_simple_hierarchy, resource_from_addressbook,\n    resource_from_calendar, resource_from_card, resource_from_event,\n};\nuse common::{CacheSwap, DavResource, DavResources, Server, auth::AccessToken};\nuse file::{build_file_resources, build_nested_hierarchy, resource_from_file};\nuse std::{sync::Arc, time::Instant};\nuse store::{\n    SerializeInfallible, ValueKey,\n    ahash::AHashMap,\n    query::log::{Change, Query},\n    write::{AlignedBytes, Archive, BatchBuilder, ValueClass},\n};\nuse tokio::sync::Semaphore;\nuse trc::{AddContext, StoreEvent};\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n};\n\npub mod calcard;\npub mod file;\n\npub trait GroupwareCache: Sync + Send {\n    fn fetch_dav_resources(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        collection: SyncCollection,\n    ) -> impl Future<Output = trc::Result<Arc<DavResources>>> + Send;\n\n    fn create_default_addressbook(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        account_name: &str,\n    ) -> impl Future<Output = trc::Result<Option<u32>>> + Send;\n\n    fn create_default_calendar(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        account_name: &str,\n    ) -> impl Future<Output = trc::Result<Option<u32>>> + Send;\n\n    fn get_or_create_default_calendar(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Option<u32>>> + Send;\n\n    fn cached_dav_resources(\n        &self,\n        account_id: u32,\n        collection: SyncCollection,\n    ) -> Option<Arc<DavResources>>;\n}\n\nimpl GroupwareCache for Server {\n    async fn fetch_dav_resources(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        collection: SyncCollection,\n    ) -> trc::Result<Arc<DavResources>> {\n        let cache_store = match collection {\n            SyncCollection::Calendar => &self.inner.cache.events,\n            SyncCollection::AddressBook => &self.inner.cache.contacts,\n            SyncCollection::FileNode => &self.inner.cache.files,\n            SyncCollection::CalendarEventNotification => &self.inner.cache.scheduling,\n            _ => unreachable!(),\n        };\n        let cache_ = match cache_store.get_value_or_guard_async(&account_id).await {\n            Ok(cache) => cache,\n            Err(guard) => {\n                let start_time = Instant::now();\n                let cache = full_cache_build(\n                    self,\n                    account_id,\n                    collection,\n                    Arc::new(Semaphore::new(1)),\n                    access_token,\n                )\n                .await?;\n\n                if guard.insert(CacheSwap::new(cache.clone())).is_err() {\n                    cache_store.insert(account_id, CacheSwap::new(cache.clone()));\n                }\n\n                trc::event!(\n                    Store(StoreEvent::CacheMiss),\n                    AccountId = account_id,\n                    Collection = collection.as_str(),\n                    Total = cache.resources.len(),\n                    ChangeId = cache.highest_change_id,\n                    Elapsed = start_time.elapsed(),\n                );\n\n                return Ok(cache);\n            }\n        };\n\n        // Obtain current state\n        let cache = cache_.load_full();\n        let start_time = Instant::now();\n        let changes = self\n            .core\n            .storage\n            .data\n            .changes(\n                account_id,\n                collection.into(),\n                Query::Since(cache.highest_change_id),\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        // Regenerate cache if the change log has been truncated\n        if changes.is_truncated {\n            let cache = full_cache_build(\n                self,\n                account_id,\n                collection,\n                cache.update_lock.clone(),\n                access_token,\n            )\n            .await?;\n            cache_.update(cache.clone());\n\n            trc::event!(\n                Store(StoreEvent::CacheStale),\n                AccountId = account_id,\n                Collection = collection.as_str(),\n                ChangeId = cache.highest_change_id,\n                Total = cache.resources.len(),\n                Elapsed = start_time.elapsed(),\n            );\n\n            return Ok(cache);\n        }\n\n        // Verify changes\n        if changes.changes.is_empty() {\n            trc::event!(\n                Store(StoreEvent::CacheHit),\n                AccountId = account_id,\n                Collection = collection.as_str(),\n                ChangeId = cache.highest_change_id,\n                Elapsed = start_time.elapsed(),\n            );\n\n            return Ok(cache);\n        }\n\n        // Lock for updates\n        let _permit = cache.update_lock.acquire().await;\n        let cache = cache_.load_full();\n        if cache.highest_change_id >= changes.to_change_id {\n            trc::event!(\n                Store(StoreEvent::CacheHit),\n                AccountId = account_id,\n                Collection = collection.as_str(),\n                ChangeId = cache.highest_change_id,\n                Elapsed = start_time.elapsed(),\n            );\n\n            return Ok(cache);\n        }\n\n        let num_changes = changes.changes.len();\n        let cache = if !matches!(collection, SyncCollection::CalendarEventNotification) {\n            let mut updated_resources = AHashMap::with_capacity(8);\n            let has_no_children = collection == SyncCollection::FileNode;\n\n            process_changes(\n                self,\n                account_id,\n                collection,\n                has_no_children,\n                &mut updated_resources,\n                changes.changes,\n            )\n            .await?;\n\n            let mut rebuild_hierarchy = false;\n            let mut resources = Vec::with_capacity(cache.resources.len());\n\n            for resource in &cache.resources {\n                let is_container = has_no_children || resource.is_container();\n                if let Some(updated_resource) =\n                    updated_resources.remove(&(is_container, resource.document_id))\n                {\n                    if let Some(updated_resource) = updated_resource {\n                        rebuild_hierarchy =\n                            rebuild_hierarchy || updated_resource.has_hierarchy_changes(resource);\n                        resources.push(updated_resource);\n                    } else {\n                        // Deleted resource\n                        rebuild_hierarchy = true;\n                    }\n                } else {\n                    resources.push(resource.clone());\n                }\n            }\n\n            // Add new resources\n            for resource in updated_resources.into_values().flatten() {\n                resources.push(resource);\n                rebuild_hierarchy = true;\n            }\n\n            if rebuild_hierarchy {\n                let mut cache = DavResources {\n                    base_path: cache.base_path.clone(),\n                    paths: Default::default(),\n                    resources,\n                    item_change_id: changes.item_change_id.unwrap_or(cache.item_change_id),\n                    container_change_id: changes\n                        .container_change_id\n                        .unwrap_or(cache.container_change_id),\n                    highest_change_id: changes.to_change_id,\n                    size: std::mem::size_of::<DavResources>() as u64,\n                    update_lock: cache.update_lock.clone(),\n                };\n\n                if matches!(collection, SyncCollection::FileNode) {\n                    build_nested_hierarchy(&mut cache);\n                } else {\n                    build_simple_hierarchy(&mut cache);\n                }\n                cache\n            } else {\n                DavResources {\n                    base_path: cache.base_path.clone(),\n                    paths: cache.paths.clone(),\n                    resources,\n                    item_change_id: changes.item_change_id.unwrap_or(cache.item_change_id),\n                    container_change_id: changes\n                        .container_change_id\n                        .unwrap_or(cache.container_change_id),\n                    highest_change_id: changes.to_change_id,\n                    size: cache.size,\n                    update_lock: cache.update_lock.clone(),\n                }\n            }\n        } else {\n            let mut delete_ids = AHashSet::with_capacity(changes.changes.len());\n            let mut resources = Vec::with_capacity(cache.resources.len());\n            let mut paths = AHashSet::with_capacity(cache.paths.len());\n\n            for change in changes.changes {\n                match change {\n                    Change::InsertItem(document_id) => {\n                        let document_id = document_id as u32;\n                        paths.insert(path_from_scheduling(document_id, resources.len(), false));\n                        resources.push(resource_from_scheduling(document_id, false));\n                    }\n                    Change::DeleteItem(document_id) => {\n                        delete_ids.insert(document_id as u32);\n                    }\n                    _ => {}\n                }\n            }\n\n            for resource in &cache.resources {\n                if !delete_ids.contains(&resource.document_id) {\n                    paths.insert(path_from_scheduling(\n                        resource.document_id,\n                        resources.len(),\n                        resource.is_container(),\n                    ));\n                    resources.push(resource.clone());\n                }\n            }\n\n            DavResources {\n                base_path: cache.base_path.clone(),\n                paths,\n                resources,\n                item_change_id: changes.item_change_id.unwrap_or(cache.item_change_id),\n                container_change_id: changes\n                    .container_change_id\n                    .unwrap_or(cache.container_change_id),\n                highest_change_id: changes.to_change_id,\n                size: cache.size,\n                update_lock: cache.update_lock.clone(),\n            }\n        };\n\n        let cache = Arc::new(cache);\n        cache_.update(cache.clone());\n\n        trc::event!(\n            Store(StoreEvent::CacheUpdate),\n            AccountId = account_id,\n            Collection = collection.as_str(),\n            ChangeId = cache.highest_change_id,\n            Details = num_changes,\n            Total = cache.resources.len(),\n            Elapsed = start_time.elapsed(),\n        );\n\n        Ok(cache)\n    }\n\n    async fn create_default_addressbook(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        account_name: &str,\n    ) -> trc::Result<Option<u32>> {\n        if let Some(name) = &self.core.groupware.default_addressbook_name {\n            let mut batch = BatchBuilder::new();\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::AddressBook, 1)\n                .await?;\n            AddressBook {\n                name: name.clone(),\n                preferences: vec![AddressBookPreferences {\n                    account_id,\n                    name: format!(\n                        \"{} ({})\",\n                        self.core\n                            .groupware\n                            .default_addressbook_display_name\n                            .as_ref()\n                            .unwrap_or(name),\n                        account_name\n                    ),\n                    ..Default::default()\n                }],\n                ..Default::default()\n            }\n            .insert(access_token, account_id, document_id, &mut batch)?;\n            self.commit_batch(batch).await?;\n            Ok(Some(document_id))\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn create_default_calendar(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n        account_name: &str,\n    ) -> trc::Result<Option<u32>> {\n        if let Some(name) = &self.core.groupware.default_calendar_name {\n            let mut batch = BatchBuilder::new();\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::Calendar, 1)\n                .await?;\n            Calendar {\n                name: name.clone(),\n                preferences: vec![CalendarPreferences {\n                    account_id,\n                    name: format!(\n                        \"{} ({})\",\n                        self.core\n                            .groupware\n                            .default_calendar_display_name\n                            .as_ref()\n                            .unwrap_or(name),\n                        account_name\n                    ),\n                    ..Default::default()\n                }],\n                ..Default::default()\n            }\n            .insert(access_token, account_id, document_id, &mut batch)?;\n\n            // Set default calendar\n            batch\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .set(PrincipalField::DefaultCalendarId, document_id.serialize());\n\n            self.commit_batch(batch).await?;\n            Ok(Some(document_id))\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn get_or_create_default_calendar(\n        &self,\n        access_token: &AccessToken,\n        account_id: u32,\n    ) -> trc::Result<Option<u32>> {\n        let default_calendar_id = self\n            .store()\n            .get_value::<u32>(ValueKey {\n                account_id,\n                collection: Collection::Principal.into(),\n                document_id: 0,\n                class: ValueClass::Property(PrincipalField::DefaultCalendarId.into()),\n            })\n            .await\n            .caused_by(trc::location!())?;\n        if default_calendar_id.is_some() {\n            Ok(default_calendar_id)\n        } else {\n            self.fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n                .await\n                .map(|c| c.document_ids(true).next())\n        }\n    }\n\n    fn cached_dav_resources(\n        &self,\n        account_id: u32,\n        collection: SyncCollection,\n    ) -> Option<Arc<DavResources>> {\n        (match collection {\n            SyncCollection::Calendar => &self.inner.cache.events,\n            SyncCollection::AddressBook => &self.inner.cache.contacts,\n            SyncCollection::FileNode => &self.inner.cache.files,\n            _ => unreachable!(),\n        })\n        .get(&account_id)\n        .map(|cache| cache.load_full())\n    }\n}\n\nasync fn process_changes(\n    server: &Server,\n    account_id: u32,\n    collection: SyncCollection,\n    has_no_children: bool,\n    updated_resources: &mut AHashMap<(bool, u32), Option<DavResource>>,\n    changes: Vec<Change>,\n) -> trc::Result<()> {\n    for change in changes {\n        match change {\n            Change::InsertItem(id) | Change::UpdateItem(id) => {\n                let document_id = id as u32;\n                if let Some(archive) = server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        collection.collection(false),\n                        document_id,\n                    ))\n                    .await\n                    .caused_by(trc::location!())?\n                {\n                    updated_resources.insert(\n                        (has_no_children, document_id),\n                        Some(resource_from_archive(\n                            archive,\n                            document_id,\n                            collection,\n                            false,\n                        )?),\n                    );\n                } else {\n                    updated_resources.insert((has_no_children, document_id), None);\n                }\n            }\n            Change::DeleteItem(id) => {\n                updated_resources.insert((has_no_children, id as u32), None);\n            }\n            Change::InsertContainer(id) | Change::UpdateContainer(id) => {\n                let document_id = id as u32;\n                if let Some(archive) = server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        collection.collection(true),\n                        document_id,\n                    ))\n                    .await\n                    .caused_by(trc::location!())?\n                {\n                    updated_resources.insert(\n                        (true, document_id),\n                        Some(resource_from_archive(\n                            archive,\n                            document_id,\n                            collection,\n                            true,\n                        )?),\n                    );\n                } else {\n                    updated_resources.insert((true, document_id), None);\n                }\n            }\n            Change::DeleteContainer(id) => {\n                updated_resources.insert((true, id as u32), None);\n            }\n            Change::UpdateContainerProperty(_) => (),\n        }\n    }\n    Ok(())\n}\n\nasync fn full_cache_build(\n    server: &Server,\n    account_id: u32,\n    collection: SyncCollection,\n    update_lock: Arc<Semaphore>,\n    access_token: &AccessToken,\n) -> trc::Result<Arc<DavResources>> {\n    match collection {\n        SyncCollection::Calendar => {\n            build_calcard_resources(\n                server,\n                access_token,\n                account_id,\n                SyncCollection::Calendar,\n                Collection::Calendar,\n                Collection::CalendarEvent,\n                update_lock,\n            )\n            .await\n        }\n        SyncCollection::AddressBook => {\n            build_calcard_resources(\n                server,\n                access_token,\n                account_id,\n                SyncCollection::AddressBook,\n                Collection::AddressBook,\n                Collection::ContactCard,\n                update_lock,\n            )\n            .await\n        }\n        SyncCollection::FileNode => build_file_resources(server, account_id, update_lock).await,\n        SyncCollection::CalendarEventNotification => {\n            build_scheduling_resources(server, account_id, update_lock).await\n        }\n        _ => unreachable!(),\n    }\n    .map(Arc::new)\n}\n\nfn resource_from_archive(\n    archive: Archive<AlignedBytes>,\n    document_id: u32,\n    collection: SyncCollection,\n    is_container: bool,\n) -> trc::Result<DavResource> {\n    Ok(match collection {\n        SyncCollection::Calendar => {\n            if is_container {\n                resource_from_calendar(\n                    archive\n                        .unarchive::<Calendar>()\n                        .caused_by(trc::location!())?,\n                    document_id,\n                )\n            } else {\n                resource_from_event(\n                    archive\n                        .unarchive::<CalendarEvent>()\n                        .caused_by(trc::location!())?,\n                    document_id,\n                )\n            }\n        }\n        SyncCollection::AddressBook => {\n            if is_container {\n                resource_from_addressbook(\n                    archive\n                        .unarchive::<AddressBook>()\n                        .caused_by(trc::location!())?,\n                    document_id,\n                )\n            } else {\n                resource_from_card(\n                    archive\n                        .unarchive::<ContactCard>()\n                        .caused_by(trc::location!())?,\n                    document_id,\n                )\n            }\n        }\n        SyncCollection::FileNode => resource_from_file(\n            archive\n                .unarchive::<FileNode>()\n                .caused_by(trc::location!())?,\n            document_id,\n        ),\n        _ => unreachable!(),\n    })\n}\n"
  },
  {
    "path": "crates/groupware/src/calendar/alarm.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Alarm, AlarmDelta, ArchivedAlarmDelta, ArchivedCalendarEventData};\nuse calcard::{\n    common::timezone::Tz,\n    icalendar::{\n        ICalendarComponent, ICalendarParameterName, ICalendarParameterValue, ICalendarProperty,\n        ICalendarRelated, ICalendarValue,\n    },\n};\nuse chrono::{DateTime, TimeZone};\nuse std::str::FromStr;\nuse store::write::bitpack::BitpackIterator;\nuse utils::codec::leb128::Leb128Reader;\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\npub struct CalendarAlarm {\n    pub alarm_id: u16,\n    pub event_id: u16,\n    pub alarm_time: i64,\n    pub typ: CalendarAlarmType,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\npub enum CalendarAlarmType {\n    Email {\n        event_start: i64,\n        event_start_tz: u16,\n        event_end: i64,\n        event_end_tz: u16,\n    },\n    Display {\n        recurrence_id: Option<i64>,\n    },\n}\n\nimpl ArchivedCalendarEventData {\n    pub fn next_alarm(&self, start_time: i64, default_tz: Tz) -> Option<CalendarAlarm> {\n        if self.alarms.is_empty() {\n            return None;\n        }\n\n        let base_offset = self.base_offset.to_native();\n        let mut next_alarm: Option<CalendarAlarm> = None;\n\n        'outer: for range in self.time_ranges.iter() {\n            let comp_id = range.id.to_native();\n            let Some(alarm) = self.alarms.iter().find(|a| a.parent_id == comp_id) else {\n                continue;\n            };\n\n            let instances = range.instances.as_ref();\n            let (offset_or_count, bytes_read) = instances.read_leb128::<u32>()?;\n\n            let duration = range.duration.to_native() as i64;\n            let mut start_tz = Tz::from_id(range.start_tz.to_native())?;\n            let mut end_tz = Tz::from_id(range.end_tz.to_native())?;\n\n            if start_tz.is_floating() && !default_tz.is_floating() {\n                start_tz = default_tz;\n            }\n            if end_tz.is_floating() && !default_tz.is_floating() {\n                end_tz = default_tz;\n            }\n\n            if instances.len() > bytes_read {\n                // Recurring event\n                let unpacker =\n                    BitpackIterator::from_bytes_and_offset(instances, bytes_read, offset_or_count);\n                for start_offset in unpacker {\n                    let start_date_naive = start_offset as i64 + base_offset;\n                    let end_date_naive = start_date_naive + duration;\n                    let start = start_tz\n                        .from_local_datetime(\n                            &DateTime::from_timestamp(start_date_naive, 0)?.naive_local(),\n                        )\n                        .single()?\n                        .timestamp();\n                    let end = end_tz\n                        .from_local_datetime(\n                            &DateTime::from_timestamp(end_date_naive, 0)?.naive_local(),\n                        )\n                        .single()?\n                        .timestamp();\n\n                    if let Some(alarm_time) = alarm.delta.to_timestamp(start, end, default_tz)\n                        && alarm_time > start_time\n                        && next_alarm\n                            .as_ref()\n                            .is_none_or(|next| alarm_time < next.alarm_time)\n                    {\n                        next_alarm = Some(CalendarAlarm {\n                            alarm_id: alarm.id.to_native(),\n                            event_id: alarm.parent_id.to_native(),\n                            alarm_time,\n                            typ: if alarm.is_email_alert {\n                                CalendarAlarmType::Email {\n                                    event_start: start_date_naive,\n                                    event_start_tz: start_tz.as_id(),\n                                    event_end: end_date_naive,\n                                    event_end_tz: end_tz.as_id(),\n                                }\n                            } else {\n                                let comp =\n                                    &self.event.components[alarm.parent_id.to_native() as usize];\n\n                                CalendarAlarmType::Display {\n                                    recurrence_id: if comp.is_recurrent_or_override() {\n                                        start_date_naive.into()\n                                    } else {\n                                        None\n                                    },\n                                }\n                            },\n                        });\n                        continue 'outer;\n                    }\n                }\n            } else {\n                // Single event\n                let start_date_naive = offset_or_count as i64 + base_offset;\n                let end_date_naive = start_date_naive + duration;\n                let start = start_tz\n                    .from_local_datetime(\n                        &DateTime::from_timestamp(start_date_naive, 0)?.naive_local(),\n                    )\n                    .single()?\n                    .timestamp();\n                let end = end_tz\n                    .from_local_datetime(\n                        &DateTime::from_timestamp(end_date_naive, 0)?.naive_local(),\n                    )\n                    .single()?\n                    .timestamp();\n\n                if let Some(alarm_time) = alarm.delta.to_timestamp(start, end, default_tz)\n                    && alarm_time > start_time\n                    && next_alarm\n                        .as_ref()\n                        .is_none_or(|next| alarm_time < next.alarm_time)\n                {\n                    next_alarm = Some(CalendarAlarm {\n                        alarm_id: alarm.id.to_native(),\n                        event_id: alarm.parent_id.to_native(),\n                        alarm_time,\n                        typ: if alarm.is_email_alert {\n                            CalendarAlarmType::Email {\n                                event_start: start_date_naive,\n                                event_start_tz: start_tz.as_id(),\n                                event_end: end_date_naive,\n                                event_end_tz: end_tz.as_id(),\n                            }\n                        } else {\n                            let comp = &self.event.components[alarm.parent_id.to_native() as usize];\n\n                            CalendarAlarmType::Display {\n                                recurrence_id: if comp.is_recurrent_or_override() {\n                                    start_date_naive.into()\n                                } else {\n                                    None\n                                },\n                            }\n                        },\n                    });\n                }\n            }\n        }\n\n        next_alarm\n    }\n}\n\npub trait ExpandAlarm {\n    fn expand_alarm(&self, id: u16, parent_id: u16) -> Option<Alarm>;\n}\n\nimpl ExpandAlarm for ICalendarComponent {\n    fn expand_alarm(&self, id: u16, parent_id: u16) -> Option<Alarm> {\n        let mut trigger = None;\n        let mut is_email_alert = false;\n\n        for entry in self.entries.iter() {\n            match &entry.name {\n                ICalendarProperty::Trigger => {\n                    let mut tz = None;\n                    let mut trigger_start = true;\n\n                    for param in entry.params.iter() {\n                        match (&param.name, &param.value) {\n                            (\n                                ICalendarParameterName::Related,\n                                ICalendarParameterValue::Related(related),\n                            ) => {\n                                trigger_start = matches!(related, ICalendarRelated::Start);\n                            }\n                            (\n                                ICalendarParameterName::Tzid,\n                                ICalendarParameterValue::Text(tz_id),\n                            ) => {\n                                tz = Tz::from_str(tz_id).ok();\n                            }\n                            _ => {}\n                        }\n                    }\n\n                    trigger = match entry.values.first()? {\n                        ICalendarValue::PartialDateTime(dt) => {\n                            let tz = tz.unwrap_or(Tz::Floating);\n\n                            dt.to_date_time_with_tz(tz).map(|dt| {\n                                let timestamp = dt.timestamp();\n                                if !dt.timezone().is_floating() {\n                                    AlarmDelta::FixedUtc(timestamp)\n                                } else {\n                                    AlarmDelta::FixedFloating(timestamp)\n                                }\n                            })\n                        }\n                        ICalendarValue::Duration(duration) => {\n                            if trigger_start {\n                                Some(AlarmDelta::Start(duration.as_seconds()))\n                            } else {\n                                Some(AlarmDelta::End(duration.as_seconds()))\n                            }\n                        }\n                        _ => None,\n                    };\n                }\n                ICalendarProperty::Action => {\n                    is_email_alert = is_email_alert\n                        || entry\n                            .values\n                            .first()\n                            .and_then(|v| v.as_text())\n                            .is_some_and(|v| v.eq_ignore_ascii_case(\"email\"));\n                }\n                ICalendarProperty::Summary | ICalendarProperty::Description => {\n                    is_email_alert = is_email_alert\n                        || entry\n                            .values\n                            .first()\n                            .and_then(|v| v.as_text())\n                            .is_some_and(|v| v.contains(\"@email\"));\n                }\n                _ => {}\n            }\n        }\n\n        trigger.map(|delta| Alarm {\n            id,\n            parent_id,\n            delta,\n            is_email_alert,\n        })\n    }\n}\n\nimpl AlarmDelta {\n    pub fn to_timestamp(&self, start: i64, end: i64, default_tz: Tz) -> Option<i64> {\n        match self {\n            AlarmDelta::Start(delta) => Some(start + delta),\n            AlarmDelta::End(delta) => Some(end + delta),\n            AlarmDelta::FixedUtc(timestamp) => Some(*timestamp),\n            AlarmDelta::FixedFloating(timestamp) => default_tz\n                .from_local_datetime(&DateTime::from_timestamp(*timestamp, 0)?.naive_local())\n                .single()\n                .map(|dt| dt.timestamp()),\n        }\n    }\n}\n\nimpl ArchivedAlarmDelta {\n    pub fn to_timestamp(&self, start: i64, end: i64, default_tz: Tz) -> Option<i64> {\n        match self {\n            ArchivedAlarmDelta::Start(delta) => Some(start + delta.to_native()),\n            ArchivedAlarmDelta::End(delta) => Some(end + delta.to_native()),\n            ArchivedAlarmDelta::FixedUtc(timestamp) => Some(timestamp.to_native()),\n            ArchivedAlarmDelta::FixedFloating(timestamp) => default_tz\n                .from_local_datetime(\n                    &DateTime::from_timestamp(timestamp.to_native(), 0)?.naive_local(),\n                )\n                .single()\n                .map(|dt| dt.timestamp()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/calendar/dates.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ArchivedCalendarEventData, ArchivedTimezone, CalendarEventData, Timezone,\n    alarm::{CalendarAlarm, ExpandAlarm},\n};\nuse crate::calendar::{ComponentTimeRange, alarm::CalendarAlarmType};\nuse calcard::{\n    common::timezone::Tz,\n    icalendar::{ICalendar, ICalendarComponentType, dates::TimeOrDelta},\n};\nuse compact_str::ToCompactString;\nuse store::{\n    ahash::AHashMap,\n    write::{key::KeySerializer, now},\n};\n\nimpl CalendarEventData {\n    pub fn new(\n        ical: ICalendar,\n        default_tz: Tz,\n        max_expansions: usize,\n        next_email_alarm: &mut Option<CalendarAlarm>,\n    ) -> Self {\n        let mut ranges = TimeRanges::default();\n        let now = now() as i64;\n\n        let expanded = ical.expand_dates(default_tz, max_expansions);\n        let mut groups: AHashMap<(u16, u16, u16, i32), Vec<i64>> = AHashMap::with_capacity(16);\n        let mut alarms = AHashMap::with_capacity(16);\n\n        for event in expanded.events {\n            let start_naive = event.start.naive_local();\n            let start_tz = event.start.timezone().as_id();\n            let start_timestamp_utc = event.start.timestamp();\n            let start_timestamp_naive = start_naive.and_utc().timestamp();\n            let (end_timestamp_utc, end_timestamp_naive, end_tz) = match event.end {\n                TimeOrDelta::Time(time) => {\n                    let end_naive = time.naive_local();\n                    let end_timestamp_utc = time.timestamp();\n                    let end_timestamp_naive = end_naive.and_utc().timestamp();\n                    (\n                        end_timestamp_utc,\n                        end_timestamp_naive,\n                        time.timezone().as_id(),\n                    )\n                }\n                TimeOrDelta::Delta(delta) => {\n                    let delta = delta.num_seconds();\n                    (\n                        start_timestamp_utc + delta,\n                        start_timestamp_naive + delta,\n                        start_tz,\n                    )\n                }\n            };\n\n            // Expand alarms\n            let mut min = std::cmp::min(start_timestamp_utc, end_timestamp_utc);\n            let mut max = std::cmp::max(start_timestamp_utc, end_timestamp_utc);\n            for alarm in alarms.entry(event.comp_id).or_insert_with(|| {\n                ical.component_by_id(event.comp_id)\n                    .map_or(&[][..], |c| c.component_ids.as_slice())\n                    .iter()\n                    .filter_map(|alarm_id| {\n                        ical.component_by_id(*alarm_id).and_then(|alarm| {\n                            if alarm.component_type == ICalendarComponentType::VAlarm {\n                                alarm.expand_alarm(*alarm_id as u16, event.comp_id as u16)\n                            } else {\n                                None\n                            }\n                        })\n                    })\n                    .collect::<Vec<_>>()\n            }) {\n                if let Some(alarm_time) =\n                    alarm\n                        .delta\n                        .to_timestamp(start_timestamp_utc, end_timestamp_utc, default_tz)\n                {\n                    if alarm_time < min {\n                        min = alarm_time;\n                    }\n                    if alarm_time > max {\n                        max = alarm_time;\n                    }\n                    if alarm_time > now\n                        && next_email_alarm\n                            .as_ref()\n                            .is_none_or(|next| alarm_time < next.alarm_time)\n                    {\n                        *next_email_alarm = Some(CalendarAlarm {\n                            alarm_id: alarm.id,\n                            event_id: alarm.parent_id,\n                            alarm_time,\n                            typ: if alarm.is_email_alert {\n                                CalendarAlarmType::Email {\n                                    event_start: start_timestamp_naive,\n                                    event_end: end_timestamp_naive,\n                                    event_start_tz: start_tz,\n                                    event_end_tz: end_tz,\n                                }\n                            } else {\n                                CalendarAlarmType::Display {\n                                    recurrence_id: if ical.components[alarm.parent_id as usize]\n                                        .is_recurrent_or_override()\n                                    {\n                                        start_timestamp_naive.into()\n                                    } else {\n                                        None\n                                    },\n                                }\n                            },\n                        });\n                    }\n                }\n            }\n\n            ranges.update_base_offset(start_timestamp_naive, end_timestamp_naive);\n            ranges.update_utc_min_max(min, max);\n            groups\n                .entry((\n                    start_tz,\n                    end_tz,\n                    event.comp_id as u16,\n                    (end_timestamp_naive - start_timestamp_naive) as i32,\n                ))\n                .or_default()\n                .push(start_timestamp_naive);\n        }\n\n        let mut events = Vec::with_capacity(groups.len());\n        for ((start_tz, end_tz, id, duration), mut instances) in groups {\n            let instances = if instances.len() > 1 {\n                instances.sort_unstable();\n                // Bitpack instances\n                let mut instance_offsets = Vec::with_capacity(instances.len());\n                for instance in instances {\n                    debug_assert!(instance >= ranges.base_offset);\n                    instance_offsets.push((instance - ranges.base_offset) as u32);\n                }\n\n                KeySerializer::new(instance_offsets.len() * std::mem::size_of::<u32>())\n                    .bitpack_sorted(&instance_offsets)\n                    .finalize()\n            } else {\n                KeySerializer::new(std::mem::size_of::<u32>())\n                    .write_leb128((instances.first().unwrap() - ranges.base_offset) as u32)\n                    .finalize()\n            };\n\n            events.push(ComponentTimeRange {\n                id,\n                start_tz,\n                end_tz,\n                duration,\n                instances: instances.into_boxed_slice(),\n            });\n        }\n\n        if !expanded.errors.is_empty() {\n            trc::event!(\n                Calendar(trc::CalendarEvent::RuleExpansionError),\n                Reason = expanded\n                    .errors\n                    .into_iter()\n                    .map(|e| e.error.to_compact_string())\n                    .collect::<Vec<_>>(),\n                Details = ical.to_string(),\n                Limit = max_expansions,\n            );\n        }\n\n        CalendarEventData {\n            event: ical,\n            time_ranges: events.into_boxed_slice(),\n            alarms: alarms\n                .into_values()\n                .flatten()\n                .collect::<Vec<_>>()\n                .into_boxed_slice(),\n            base_offset: ranges.base_offset,\n            base_time_utc: (ranges.min_time_utc - ranges.base_offset) as u32,\n            duration: (ranges.max_time_utc - ranges.min_time_utc) as u32,\n        }\n    }\n\n    pub fn event_range(&self) -> Option<(i64, u32)> {\n        if self.base_offset != 0 {\n            Some((self.base_offset + self.base_time_utc as i64, self.duration))\n        } else {\n            None\n        }\n    }\n}\n\n#[derive(Default, Debug)]\nstruct TimeRanges {\n    max_time_utc: i64,\n    min_time_utc: i64,\n    base_offset: i64,\n}\n\nimpl TimeRanges {\n    pub fn update_base_offset(&mut self, t1: i64, t2: i64) {\n        let offset = std::cmp::min(t1, t2);\n        if offset < self.base_offset || self.base_offset == 0 {\n            self.base_offset = offset;\n        }\n    }\n\n    pub fn update_utc_min_max(&mut self, min: i64, max: i64) {\n        if min < self.min_time_utc || self.min_time_utc == 0 {\n            self.min_time_utc = min;\n        }\n        if max > self.max_time_utc {\n            self.max_time_utc = max;\n        }\n        if min < self.base_offset || self.base_offset == 0 {\n            self.base_offset = min;\n        }\n    }\n}\n\nimpl ArchivedCalendarEventData {\n    pub fn event_range(&self) -> Option<(i64, u32)> {\n        if self.base_offset != 0 {\n            Some((\n                self.base_offset.to_native() + self.base_time_utc.to_native() as i64,\n                self.duration.to_native(),\n            ))\n        } else {\n            None\n        }\n    }\n\n    pub fn event_range_start(&self) -> i64 {\n        self.base_offset.to_native() + self.base_time_utc.to_native() as i64\n    }\n\n    pub fn event_range_end(&self) -> i64 {\n        self.base_offset.to_native()\n            + self.base_time_utc.to_native() as i64\n            + self.duration.to_native() as i64\n    }\n}\n\nimpl CalendarEventData {\n    pub fn event_range_start(&self) -> i64 {\n        self.base_offset + self.base_time_utc as i64\n    }\n\n    pub fn event_range_end(&self) -> i64 {\n        self.base_offset + self.base_time_utc as i64 + self.duration as i64\n    }\n}\n\nimpl Timezone {\n    pub fn tz(&self) -> Option<Tz> {\n        match self {\n            Timezone::IANA(iana) => Tz::from_id(*iana),\n            Timezone::Custom(icalendar) => icalendar\n                .timezones()\n                .filter_map(|t| t.timezone().map(|x| x.1))\n                .next(),\n            Timezone::Default => None,\n        }\n    }\n}\n\nimpl ArchivedTimezone {\n    pub fn tz(&self) -> Option<Tz> {\n        match self {\n            ArchivedTimezone::IANA(iana) => Tz::from_id(iana.to_native()),\n            ArchivedTimezone::Custom(icalendar) => icalendar\n                .timezones()\n                .filter_map(|t| t.timezone().map(|x| x.1))\n                .next(),\n            ArchivedTimezone::Default => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/calendar/expand.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ArchivedCalendarEventData;\nuse crate::calendar::CalendarEventData;\nuse ahash::AHashSet;\nuse calcard::common::timezone::Tz;\nuse chrono::{DateTime, TimeZone};\nuse store::write::bitpack::BitpackIterator;\nuse types::TimeRange;\nuse utils::codec::leb128::Leb128Reader;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct CalendarEventExpansion {\n    pub comp_id: u32,\n    pub expansion_id: u32,\n    pub start: i64,\n    pub end: i64,\n}\n\nimpl ArchivedCalendarEventData {\n    pub fn expand(&self, default_tz: Tz, limit: TimeRange) -> Option<Vec<CalendarEventExpansion>> {\n        let mut expansion = Vec::with_capacity(self.time_ranges.len());\n        let base_offset = self.base_offset.to_native();\n        let mut base_expansion_id = 0;\n\n        'outer: for range in self.time_ranges.iter() {\n            let instances = range.instances.as_ref();\n            let (offset_or_count, bytes_read) = instances.read_leb128::<u32>()?;\n\n            let comp_id = range.id.to_native() as u32;\n            let duration = range.duration.to_native() as i64;\n            let mut start_tz = Tz::from_id(range.start_tz.to_native())?;\n            let mut end_tz = Tz::from_id(range.end_tz.to_native())?;\n            let is_todo = self.event.components[comp_id as usize]\n                .component_type\n                .is_todo();\n\n            if start_tz.is_floating() && !default_tz.is_floating() {\n                start_tz = default_tz;\n            }\n            if end_tz.is_floating() && !default_tz.is_floating() {\n                end_tz = default_tz;\n            }\n\n            if instances.len() > bytes_read {\n                // Recurring event\n                let unpacker =\n                    BitpackIterator::from_bytes_and_offset(instances, bytes_read, offset_or_count);\n                let mut expansion_id = base_expansion_id;\n                base_expansion_id += offset_or_count;\n                for start_offset in unpacker {\n                    let start_date_naive = start_offset as i64 + base_offset;\n                    let end_date_naive = start_date_naive + duration;\n                    let start = start_tz\n                        .from_local_datetime(\n                            &DateTime::from_timestamp(start_date_naive, 0)?.naive_local(),\n                        )\n                        .single()?\n                        .timestamp();\n                    let end = end_tz\n                        .from_local_datetime(\n                            &DateTime::from_timestamp(end_date_naive, 0)?.naive_local(),\n                        )\n                        .single()?\n                        .timestamp();\n\n                    if limit.is_in_range(is_todo, start, end) {\n                        expansion.push(CalendarEventExpansion {\n                            comp_id,\n                            expansion_id,\n                            start,\n                            end,\n                        });\n                    } else if start > limit.end {\n                        continue 'outer;\n                    }\n\n                    expansion_id += 1;\n                }\n            } else {\n                // Single event\n                let start_date_naive = offset_or_count as i64 + base_offset;\n                let end_date_naive = start_date_naive + duration;\n                let start = start_tz\n                    .from_local_datetime(\n                        &DateTime::from_timestamp(start_date_naive, 0)?.naive_local(),\n                    )\n                    .single()?\n                    .timestamp();\n                let end = end_tz\n                    .from_local_datetime(\n                        &DateTime::from_timestamp(end_date_naive, 0)?.naive_local(),\n                    )\n                    .single()?\n                    .timestamp();\n\n                if limit.is_in_range(is_todo, start, end) {\n                    expansion.push(CalendarEventExpansion {\n                        comp_id,\n                        expansion_id: base_expansion_id,\n                        start,\n                        end,\n                    });\n                }\n\n                base_expansion_id += 1;\n            }\n        }\n\n        Some(expansion)\n    }\n}\n\nimpl CalendarEventData {\n    pub fn expand_from_ids(\n        &self,\n        expansion_ids: &mut AHashSet<u32>,\n        default_tz: Tz,\n    ) -> Option<Vec<CalendarEventExpansion>> {\n        let mut expansion = Vec::with_capacity(expansion_ids.len());\n        let base_offset = self.base_offset;\n        let mut base_expansion_id = 0;\n\n        'outer: for range in self.time_ranges.iter() {\n            let instances = range.instances.as_ref();\n            let (offset_or_count, bytes_read) = instances.read_leb128::<u32>()?;\n            let mut start_tz = Tz::from_id(range.start_tz)?;\n            let mut end_tz = Tz::from_id(range.end_tz)?;\n\n            if start_tz.is_floating() && !default_tz.is_floating() {\n                start_tz = default_tz;\n            }\n            if end_tz.is_floating() && !default_tz.is_floating() {\n                end_tz = default_tz;\n            }\n\n            if instances.len() > bytes_read {\n                let match_range = base_expansion_id..base_expansion_id + offset_or_count;\n                let mut match_count = expansion_ids\n                    .iter()\n                    .filter(|id| match_range.contains(id))\n                    .count();\n                let mut expansion_id = base_expansion_id;\n                base_expansion_id += offset_or_count;\n\n                if match_count > 0 {\n                    let unpacker = BitpackIterator::from_bytes_and_offset(\n                        instances,\n                        bytes_read,\n                        offset_or_count,\n                    );\n                    for start_offset in unpacker {\n                        if expansion_ids.remove(&expansion_id) {\n                            let start_date_naive = start_offset as i64 + base_offset;\n                            let end_date_naive = start_date_naive + range.duration as i64;\n                            let start = start_tz\n                                .from_local_datetime(\n                                    &DateTime::from_timestamp(start_date_naive, 0)?.naive_local(),\n                                )\n                                .single()?\n                                .timestamp();\n                            let end = end_tz\n                                .from_local_datetime(\n                                    &DateTime::from_timestamp(end_date_naive, 0)?.naive_local(),\n                                )\n                                .single()?\n                                .timestamp();\n\n                            expansion.push(CalendarEventExpansion {\n                                comp_id: range.id as u32,\n                                expansion_id,\n                                start,\n                                end,\n                            });\n\n                            match_count -= 1;\n                            if match_count == 0 {\n                                if expansion_ids.is_empty() {\n                                    break 'outer;\n                                } else {\n                                    continue 'outer;\n                                }\n                            }\n                        }\n                        expansion_id += 1;\n                    }\n                }\n            } else {\n                if expansion_ids.remove(&base_expansion_id) {\n                    // Single event\n                    let start_date_naive = offset_or_count as i64 + base_offset;\n                    let end_date_naive = start_date_naive + range.duration as i64;\n                    let start = start_tz\n                        .from_local_datetime(\n                            &DateTime::from_timestamp(start_date_naive, 0)?.naive_local(),\n                        )\n                        .single()?\n                        .timestamp();\n                    let end = end_tz\n                        .from_local_datetime(\n                            &DateTime::from_timestamp(end_date_naive, 0)?.naive_local(),\n                        )\n                        .single()?\n                        .timestamp();\n\n                    expansion.push(CalendarEventExpansion {\n                        comp_id: range.id as u32,\n                        expansion_id: base_expansion_id,\n                        start,\n                        end,\n                    });\n\n                    if expansion_ids.is_empty() {\n                        break 'outer;\n                    }\n                }\n\n                base_expansion_id += 1;\n            }\n        }\n\n        if !expansion_ids.is_empty() {\n            expansion.extend(\n                expansion_ids\n                    .drain()\n                    .map(|expansion_id| CalendarEventExpansion {\n                        comp_id: u32::MAX,\n                        expansion_id,\n                        start: i64::MAX,\n                        end: i64::MAX,\n                    }),\n            );\n        }\n\n        Some(expansion)\n    }\n\n    pub fn expand_single(&self, comp_id: u32, default_tz: Tz) -> Option<CalendarEventExpansion> {\n        let range = self.time_ranges.iter().find(|r| r.id as u32 == comp_id)?;\n        let instances = range.instances.as_ref();\n        let (offset_or_count, bytes_read) = instances.read_leb128::<u32>()?;\n        let mut start_tz = Tz::from_id(range.start_tz)?;\n        let mut end_tz = Tz::from_id(range.end_tz)?;\n\n        if start_tz.is_floating() && !default_tz.is_floating() {\n            start_tz = default_tz;\n        }\n        if end_tz.is_floating() && !default_tz.is_floating() {\n            end_tz = default_tz;\n        }\n        let start_offset = if instances.len() > bytes_read {\n            // Recurring event\n            let mut unpacker =\n                BitpackIterator::from_bytes_and_offset(instances, bytes_read, offset_or_count);\n            unpacker.next()?\n        } else {\n            // Single event\n            offset_or_count\n        };\n        let start_date_naive = start_offset as i64 + self.base_offset;\n        let end_date_naive = start_date_naive + range.duration as i64;\n        let start = start_tz\n            .from_local_datetime(&DateTime::from_timestamp(start_date_naive, 0)?.naive_local())\n            .single()?\n            .timestamp();\n        let end = end_tz\n            .from_local_datetime(&DateTime::from_timestamp(end_date_naive, 0)?.naive_local())\n            .single()?\n            .timestamp();\n\n        Some(CalendarEventExpansion {\n            comp_id,\n            expansion_id: u32::MAX,\n            start,\n            end,\n        })\n    }\n}\n\nimpl Default for CalendarEventExpansion {\n    fn default() -> Self {\n        Self {\n            comp_id: u32::MAX,\n            expansion_id: u32::MAX,\n            start: i64::MAX,\n            end: i64::MAX,\n        }\n    }\n}\n\nimpl CalendarEventExpansion {\n    pub fn is_valid(&self) -> bool {\n        self.comp_id != u32::MAX && self.start != i64::MAX && self.end != i64::MAX\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/calendar/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ArchivedCalendar, ArchivedCalendarEvent, ArchivedCalendarPreferences, ArchivedDefaultAlert,\n    ArchivedTimezone, Calendar, CalendarEvent, CalendarPreferences, DefaultAlert, Timezone,\n};\nuse crate::calendar::{\n    ArchivedCalendarEventNotification, ArchivedChangedBy, ArchivedEventPreferences,\n    CalendarEventNotification, ChangedBy, EventPreferences,\n};\nuse ahash::AHashSet;\nuse calcard::icalendar::{\n    ArchivedICalendarParameterValue, ArchivedICalendarProperty, ArchivedICalendarValue,\n    ICalendarParameterValue, ICalendarProperty, ICalendarValue,\n};\nuse common::storage::index::{IndexValue, IndexableAndSerializableObject, IndexableObject};\nuse nlp::language::{\n    Language,\n    detect::{LanguageDetector, MIN_LANGUAGE_SCORE},\n};\nuse store::{\n    U32_LEN,\n    search::{CalendarSearchField, IndexDocument, SearchField},\n    write::{IndexPropertyClass, SearchIndex, ValueClass},\n    xxhash_rust::xxh3,\n};\nuse types::{\n    acl::AclGrant,\n    collection::SyncCollection,\n    field::{CalendarEventField, CalendarNotificationField},\n};\n\nimpl IndexableObject for Calendar {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Acl {\n                value: (&self.acls).into(),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogContainer {\n                sync_collection: SyncCollection::Calendar,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedCalendar {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Acl {\n                value: self\n                    .acls\n                    .iter()\n                    .map(AclGrant::from)\n                    .collect::<Vec<_>>()\n                    .into(),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogContainer {\n                sync_collection: SyncCollection::Calendar,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for Calendar {\n    fn is_versioned() -> bool {\n        true\n    }\n}\n\nimpl IndexableObject for CalendarEvent {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::SearchIndex {\n                index: SearchIndex::Calendar,\n                hash: self\n                    .hashes()\n                    .chain([self.data.event_range_start() as u64])\n                    .fold(0, |acc, hash| acc ^ hash),\n            },\n            IndexValue::Index {\n                field: CalendarEventField::Uid.into(),\n                value: self.data.event.uids().next().into(),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::Calendar,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedCalendarEvent {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::SearchIndex {\n                index: SearchIndex::Calendar,\n                hash: self\n                    .hashes()\n                    .chain([self.data.event_range_start() as u64])\n                    .fold(0, |acc, hash| acc ^ hash),\n            },\n            IndexValue::Index {\n                field: CalendarEventField::Uid.into(),\n                value: self.data.event.uids().next().into(),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::Calendar,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for CalendarEvent {\n    fn is_versioned() -> bool {\n        true\n    }\n}\n\nimpl IndexableObject for CalendarEventNotification {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::Property {\n                field: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                    property: CalendarNotificationField::CreatedToId.into(),\n                    value: self.created as u64,\n                }),\n                value: self.event_id.unwrap_or(u32::MAX).into(),\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::CalendarEventNotification,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedCalendarEventNotification {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::Property {\n                field: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                    property: CalendarNotificationField::CreatedToId.into(),\n                    value: self.created.to_native() as u64,\n                }),\n                value: self\n                    .event_id\n                    .as_ref()\n                    .map(|v| v.to_native())\n                    .unwrap_or(u32::MAX)\n                    .into(),\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::CalendarEventNotification,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for CalendarEventNotification {\n    fn is_versioned() -> bool {\n        false\n    }\n}\n\nimpl Calendar {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.preferences.iter().map(|p| p.size()).sum::<usize>()\n            + self.name.len()\n            + std::mem::size_of::<Calendar>()\n    }\n}\n\nimpl ArchivedCalendar {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.preferences.iter().map(|p| p.size()).sum::<usize>()\n            + self.name.len()\n            + std::mem::size_of::<Calendar>()\n    }\n}\n\nimpl CalendarEvent {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.display_name.as_ref().map_or(0, |n| n.len())\n            + self.names.iter().map(|n| n.name.len()).sum::<usize>()\n            + self.preferences.iter().map(|p| p.size()).sum::<usize>()\n            + self.size as usize\n            + std::mem::size_of::<CalendarEvent>()\n    }\n}\n\nimpl ArchivedCalendarEvent {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.display_name.as_ref().map_or(0, |n| n.len())\n            + self.names.iter().map(|n| n.name.len()).sum::<usize>()\n            + self.preferences.iter().map(|p| p.size()).sum::<usize>()\n            + self.size.to_native() as usize\n            + std::mem::size_of::<CalendarEvent>()\n    }\n}\n\nimpl CalendarEventNotification {\n    pub fn size(&self) -> usize {\n        (match &self.changed_by {\n            ChangedBy::PrincipalId(_) => U32_LEN,\n            ChangedBy::CalendarAddress(v) => v.len(),\n        }) + std::mem::size_of::<CalendarEventNotification>()\n            + self.size as usize\n    }\n}\n\nimpl ArchivedCalendarEventNotification {\n    pub fn size(&self) -> usize {\n        (match &self.changed_by {\n            ArchivedChangedBy::PrincipalId(_) => U32_LEN,\n            ArchivedChangedBy::CalendarAddress(v) => v.len(),\n        }) + std::mem::size_of::<CalendarEventNotification>()\n            + self.size.to_native() as usize\n    }\n}\n\nimpl CalendarPreferences {\n    pub fn size(&self) -> usize {\n        self.name.len()\n            + self.default_alerts.iter().map(|a| a.size()).sum::<usize>()\n            + self.description.as_ref().map_or(0, |n| n.len())\n            + self.color.as_ref().map_or(0, |n| n.len())\n            + self.time_zone.size()\n            + std::mem::size_of::<CalendarPreferences>()\n    }\n}\n\nimpl ArchivedCalendarPreferences {\n    pub fn size(&self) -> usize {\n        self.name.len()\n            + self.default_alerts.iter().map(|a| a.size()).sum::<usize>()\n            + self.description.as_ref().map_or(0, |n| n.len())\n            + self.color.as_ref().map_or(0, |n| n.len())\n            + self.time_zone.size()\n            + std::mem::size_of::<CalendarPreferences>()\n    }\n}\n\nimpl EventPreferences {\n    pub fn size(&self) -> usize {\n        self.alerts.iter().map(|a| a.size()).sum::<usize>()\n            + self.properties.iter().map(|p| p.size()).sum::<usize>()\n            + std::mem::size_of::<EventPreferences>()\n    }\n}\n\nimpl ArchivedEventPreferences {\n    pub fn size(&self) -> usize {\n        self.alerts.iter().map(|a| a.size()).sum::<usize>()\n            + self.properties.iter().map(|p| p.size()).sum::<usize>()\n            + std::mem::size_of::<EventPreferences>()\n    }\n}\n\nimpl Timezone {\n    pub fn size(&self) -> usize {\n        match self {\n            Timezone::IANA(_) => 2,\n            Timezone::Custom(c) => c.size(),\n            Timezone::Default => 0,\n        }\n    }\n}\n\nimpl ArchivedTimezone {\n    pub fn size(&self) -> usize {\n        match self {\n            ArchivedTimezone::IANA(_) => 2,\n            ArchivedTimezone::Custom(c) => c.size(),\n            ArchivedTimezone::Default => 0,\n        }\n    }\n}\n\nimpl DefaultAlert {\n    pub fn size(&self) -> usize {\n        std::mem::size_of::<DefaultAlert>() + self.id.len()\n    }\n}\n\nimpl ArchivedDefaultAlert {\n    pub fn size(&self) -> usize {\n        std::mem::size_of::<DefaultAlert>() + self.id.len()\n    }\n}\n\nimpl CalendarEvent {\n    pub fn hashes(&self) -> impl Iterator<Item = u64> {\n        self.data\n            .event\n            .components\n            .iter()\n            .filter(|e| e.component_type.is_scheduling_object())\n            .flat_map(|e| {\n                e.entries.iter().filter(|e| {\n                    matches!(\n                        e.name,\n                        ICalendarProperty::Summary\n                            | ICalendarProperty::Location\n                            | ICalendarProperty::Description\n                            | ICalendarProperty::Categories\n                            | ICalendarProperty::Comment\n                            | ICalendarProperty::Attendee\n                            | ICalendarProperty::Organizer\n                            | ICalendarProperty::Uid\n                    )\n                })\n            })\n            .flat_map(|e| {\n                e.values\n                    .iter()\n                    .filter_map(|v| match v {\n                        ICalendarValue::Text(v) => Some(v.as_str()),\n                        ICalendarValue::Uri(uri) => uri.as_str(),\n                        _ => None,\n                    })\n                    .chain(e.params.iter().filter_map(|p| match &p.value {\n                        ICalendarParameterValue::Text(v) => Some(v.as_str()),\n                        ICalendarParameterValue::Uri(uri) => uri.as_str(),\n                        _ => None,\n                    }))\n            })\n            .map(|v| xxh3::xxh3_64(v.as_bytes()))\n    }\n}\n\nimpl ArchivedCalendarEvent {\n    pub fn hashes(&self) -> impl Iterator<Item = u64> {\n        self.data\n            .event\n            .components\n            .iter()\n            .filter(|e| e.component_type.is_scheduling_object())\n            .flat_map(|e| {\n                e.entries.iter().filter(|e| {\n                    matches!(\n                        e.name,\n                        ArchivedICalendarProperty::Summary\n                            | ArchivedICalendarProperty::Location\n                            | ArchivedICalendarProperty::Description\n                            | ArchivedICalendarProperty::Categories\n                            | ArchivedICalendarProperty::Comment\n                            | ArchivedICalendarProperty::Attendee\n                            | ArchivedICalendarProperty::Organizer\n                            | ArchivedICalendarProperty::Uid\n                    )\n                })\n            })\n            .flat_map(|e| {\n                e.values\n                    .iter()\n                    .filter_map(|v| match v {\n                        ArchivedICalendarValue::Text(v) => Some(v.as_str()),\n                        ArchivedICalendarValue::Uri(uri) => uri.as_str(),\n                        _ => None,\n                    })\n                    .chain(e.params.iter().filter_map(|p| match &p.value {\n                        ArchivedICalendarParameterValue::Text(v) => Some(v.as_str()),\n                        ArchivedICalendarParameterValue::Uri(uri) => uri.as_str(),\n                        _ => None,\n                    }))\n            })\n            .map(|v| xxh3::xxh3_64(v.as_bytes()))\n    }\n}\n\nimpl ArchivedCalendarEvent {\n    pub fn index_document(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        index_fields: &AHashSet<SearchField>,\n        default_language: Language,\n    ) -> IndexDocument {\n        let mut document = IndexDocument::new(SearchIndex::Calendar)\n            .with_account_id(account_id)\n            .with_document_id(document_id);\n\n        if index_fields.is_empty()\n            || index_fields.contains(&SearchField::Calendar(CalendarSearchField::Start))\n        {\n            document.index_integer(CalendarSearchField::Start, self.data.event_range_start());\n        }\n\n        let mut detector = LanguageDetector::new();\n        for component in self\n            .data\n            .event\n            .components\n            .iter()\n            .filter(|e| e.component_type.is_scheduling_object())\n        {\n            for entry in component.entries.iter() {\n                let (is_lang, is_keyword, field) = match entry.name {\n                    ArchivedICalendarProperty::Summary => (true, false, CalendarSearchField::Title),\n                    ArchivedICalendarProperty::Description => {\n                        (true, false, CalendarSearchField::Description)\n                    }\n                    ArchivedICalendarProperty::Location => {\n                        (false, false, CalendarSearchField::Location)\n                    }\n                    ArchivedICalendarProperty::Organizer => {\n                        (false, false, CalendarSearchField::Owner)\n                    }\n                    ArchivedICalendarProperty::Attendee => {\n                        (false, false, CalendarSearchField::Attendee)\n                    }\n                    ArchivedICalendarProperty::Uid => (false, true, CalendarSearchField::Uid),\n                    _ => continue,\n                };\n                let field = SearchField::Calendar(field);\n\n                if index_fields.is_empty() || index_fields.contains(&field) {\n                    for value in entry\n                        .values\n                        .iter()\n                        .filter_map(|v| match v {\n                            ArchivedICalendarValue::Text(v) => Some(v.as_str()),\n                            ArchivedICalendarValue::Uri(uri) => uri.as_str(),\n                            _ => None,\n                        })\n                        .chain(entry.params.iter().filter_map(|p| match &p.value {\n                            ArchivedICalendarParameterValue::Text(v) => Some(v.as_str()),\n                            ArchivedICalendarParameterValue::Uri(uri) => uri.as_str(),\n                            _ => None,\n                        }))\n                    {\n                        let value = value.strip_prefix(\"mailto:\").unwrap_or(value).trim();\n                        let lang = if is_lang {\n                            detector.detect(value, MIN_LANGUAGE_SCORE);\n                            Language::Unknown\n                        } else {\n                            Language::None\n                        };\n\n                        if !is_keyword {\n                            document.index_text(field.clone(), value, lang);\n                        } else {\n                            document.index_keyword(field.clone(), value);\n                        }\n                    }\n                }\n            }\n        }\n\n        document.set_unknown_language(\n            detector\n                .most_frequent_language()\n                .unwrap_or(default_language),\n        );\n\n        document\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/calendar/itip.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    RFC_3986,\n    cache::GroupwareCache,\n    calendar::{\n        CalendarEvent, CalendarEventData, CalendarEventNotification, ChangedBy,\n        EVENT_NOTIFICATION_IS_CHANGE,\n    },\n    scheduling::{\n        ItipError, ItipMessage,\n        inbound::{\n            MergeResult, itip_import_message, itip_merge_changes, itip_method, itip_process_message,\n        },\n        snapshot::itip_snapshot,\n    },\n};\nuse calcard::{\n    common::{IanaString, timezone::Tz},\n    icalendar::{\n        ICalendar, ICalendarComponentType, ICalendarEntry, ICalendarMethod, ICalendarParameter,\n        ICalendarParameterName, ICalendarParameterValue, ICalendarParticipationStatus,\n        ICalendarProperty, ICalendarValue,\n    },\n};\nuse common::{\n    DavName, Server,\n    auth::{AccessToken, ResourceToken, oauth::GrantType},\n    config::groupware::CalendarTemplateVariable,\n    i18n,\n};\nuse store::{\n    ValueKey, rand,\n    write::{AlignedBytes, Archive, BatchBuilder, now},\n};\nuse trc::AddContext;\nuse types::{\n    collection::Collection,\n    field::{CalendarEventField, ContactField},\n};\nuse utils::{template::Variables, url_params::UrlParams};\n\npub enum ItipIngestError {\n    Message(ItipError),\n    Internal(trc::Error),\n}\n\n#[derive(Default)]\npub struct ItipRsvpUrl(String);\n\npub trait ItipIngest: Sync + Send {\n    fn itip_ingest(\n        &self,\n        access_token: &AccessToken,\n        resource_token: &ResourceToken,\n        sender: &str,\n        recipient: &str,\n        itip_message: &str,\n    ) -> impl Future<Output = Result<Option<ItipMessage<ICalendar>>, ItipIngestError>> + Send;\n\n    fn http_rsvp_url(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        attendee: &str,\n    ) -> impl Future<Output = Option<ItipRsvpUrl>> + Send;\n\n    fn http_rsvp_handle(\n        &self,\n        query: &str,\n        language: &str,\n    ) -> impl Future<Output = trc::Result<String>> + Send;\n}\n\nimpl ItipIngest for Server {\n    async fn itip_ingest(\n        &self,\n        access_token: &AccessToken,\n        resource_token: &ResourceToken,\n        sender: &str,\n        recipient: &str,\n        itip_message: &str,\n    ) -> Result<Option<ItipMessage<ICalendar>>, ItipIngestError> {\n        // Parse and validate the iTIP message\n        let mut itip = ICalendar::parse(itip_message)\n            .map_err(|_| ItipIngestError::Message(ItipError::ICalendarParseError))\n            .and_then(|ical| {\n                if ical.components.len() > 1\n                    && ical.components[0].component_type == ICalendarComponentType::VCalendar\n                {\n                    Ok(ical)\n                } else {\n                    Err(ItipIngestError::Message(ItipError::ICalendarParseError))\n                }\n            })?;\n\n        // Microsoft Exchange does not include the organizer in REPLY, assume it is the recipient.\n        // This will be validated against the stored event anyway.\n        if itip.components[0]\n            .property(&ICalendarProperty::Method)\n            .and_then(|v| v.values.first())\n            .is_some_and(|v| {\n                matches!(\n                    v,\n                    ICalendarValue::Method(ICalendarMethod::Reply | ICalendarMethod::Request)\n                )\n            })\n        {\n            for comp in &mut itip.components {\n                if comp.component_type.is_scheduling_object() {\n                    let mut has_organizer = false;\n                    let mut has_attendee = false;\n\n                    for entry in &comp.entries {\n                        match entry.name {\n                            ICalendarProperty::Organizer => has_organizer = true,\n                            ICalendarProperty::Attendee => has_attendee = true,\n                            _ => {}\n                        }\n                    }\n\n                    if has_attendee && !has_organizer {\n                        comp.entries.push(ICalendarEntry {\n                            name: ICalendarProperty::Organizer,\n                            params: vec![],\n                            values: vec![ICalendarValue::Text(format!(\"mailto:{recipient}\"))],\n                        });\n                    }\n                }\n            }\n        }\n\n        let itip_snapshots = itip_snapshot(&itip, access_token.emails.as_slice(), false)?;\n        if !itip_snapshots.sender_is_organizer_or_attendee(sender) {\n            return Err(ItipIngestError::Message(\n                ItipError::SenderIsNotOrganizerNorAttendee,\n            ));\n        }\n\n        // Obtain changedBy\n        let changed_by = if let Some(id) = self.email_to_id(self.directory(), sender, 0).await? {\n            ChangedBy::PrincipalId(id)\n        } else {\n            ChangedBy::CalendarAddress(sender.into())\n        };\n\n        // Find event by UID\n        let account_id = access_token.primary_id;\n        let document_id = self\n            .document_ids_matching(\n                account_id,\n                Collection::CalendarEvent,\n                CalendarEventField::Uid,\n                itip_snapshots.uid.as_bytes(),\n            )\n            .await\n            .caused_by(trc::location!())?\n            .iter()\n            .next();\n\n        if let Some(document_id) = document_id {\n            if let Some(archive) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                let event_ = archive\n                    .to_unarchived::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n                let mut event = event_\n                    .deserialize::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n\n                // Process the iTIP message\n                let snapshots =\n                    itip_snapshot(&event.data.event, access_token.emails.as_slice(), false)?;\n                let is_organizer_update = !itip_snapshots.organizer.email.is_local;\n                match itip_process_message(\n                    &event.data.event,\n                    snapshots,\n                    &itip,\n                    itip_snapshots,\n                    sender.to_string(),\n                )? {\n                    MergeResult::Actions(changes) => {\n                        // Merge changes\n                        itip_merge_changes(&mut event.data.event, changes);\n\n                        // Calculate the new ical size\n                        event.size = event.data.event.to_string().len() as u32;\n                        if event.size > self.core.groupware.max_ical_size as u32 {\n                            return Err(ItipIngestError::Message(ItipError::EventTooLarge));\n                        }\n\n                        // Validate quota\n                        let extra_bytes = (event.size as u64)\n                            .saturating_sub(event_.inner.size.to_native() as u64);\n                        if extra_bytes > 0\n                            && self\n                                .has_available_quota(resource_token, extra_bytes)\n                                .await\n                                .is_err()\n                        {\n                            return Err(ItipIngestError::Message(ItipError::QuotaExceeded));\n                        }\n\n                        // Build event\n                        let now = now() as i64;\n                        let prev_email_alarm = event_.inner.data.next_alarm(now, Tz::Floating);\n                        let mut next_email_alarm = None;\n                        event.data = CalendarEventData::new(\n                            event.data.event,\n                            Tz::Floating,\n                            self.core.groupware.max_ical_instances,\n                            &mut next_email_alarm,\n                        );\n                        if is_organizer_update {\n                            if let Some(schedule_tag) = &mut event.schedule_tag {\n                                *schedule_tag += 1;\n                            } else {\n                                event.schedule_tag = Some(1);\n                            }\n                        }\n\n                        // Build event for schedule inbox\n                        let itip_document_id = self\n                            .store()\n                            .assign_document_ids(\n                                account_id,\n                                Collection::CalendarEventNotification,\n                                1,\n                            )\n                            .await\n                            .caused_by(trc::location!())?;\n                        let itip_message = CalendarEventNotification {\n                            event: itip,\n                            changed_by,\n                            event_id: Some(document_id),\n                            flags: EVENT_NOTIFICATION_IS_CHANGE,\n                            size: itip_message.len() as u32,\n                            ..Default::default()\n                        };\n\n                        // Prepare write batch\n                        let mut batch = BatchBuilder::new();\n                        event\n                            .update(access_token, event_, account_id, document_id, &mut batch)\n                            .caused_by(trc::location!())?;\n                        if prev_email_alarm != next_email_alarm {\n                            if let Some(prev_alarm) = prev_email_alarm {\n                                prev_alarm.delete_task(&mut batch);\n                            }\n                            if let Some(next_alarm) = next_email_alarm {\n                                next_alarm.write_task(&mut batch);\n                            }\n                        }\n                        itip_message\n                            .insert(access_token, account_id, itip_document_id, &mut batch)\n                            .caused_by(trc::location!())?;\n                        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n                        Ok(None)\n                    }\n                    MergeResult::Message(itip_message) => Ok(Some(itip_message)),\n                    MergeResult::None => Ok(None),\n                }\n            } else {\n                Err(ItipIngestError::Message(ItipError::EventNotFound))\n            }\n        } else {\n            // Verify that auto-adding invitations is allowed\n            if !self.core.groupware.itip_auto_add\n                && !matches!(changed_by, ChangedBy::PrincipalId(_))\n                && !self\n                    .document_exists(\n                        account_id,\n                        Collection::ContactCard,\n                        ContactField::Email,\n                        sender.as_bytes(),\n                    )\n                    .await\n                    .caused_by(trc::location!())?\n            {\n                return Err(ItipIngestError::Message(ItipError::AutoAddDisabled));\n            } else if itip_method(&itip)? != &ICalendarMethod::Request {\n                return Err(ItipIngestError::Message(ItipError::EventNotFound));\n            }\n\n            // Import the iTIP message\n            let mut ical = itip.clone();\n            itip_import_message(&mut ical)?;\n\n            // Validate quota\n            if self\n                .has_available_quota(resource_token, itip_message.len() as u64)\n                .await\n                .is_err()\n            {\n                return Err(ItipIngestError::Message(ItipError::QuotaExceeded));\n            }\n\n            // Obtain parent calendar\n            let Some(parent_id) = self\n                .get_or_create_default_calendar(access_token, account_id)\n                .await\n                .caused_by(trc::location!())?\n            else {\n                return Err(ItipIngestError::Message(ItipError::NoDefaultCalendar));\n            };\n\n            // Build event\n            let mut next_email_alarm = None;\n            let now = now() as i64;\n            let event = CalendarEvent {\n                names: vec![DavName {\n                    name: format!(\"{}_{}.ics\", now, rand::random::<u64>()),\n                    parent_id,\n                }],\n                data: CalendarEventData::new(\n                    ical,\n                    Tz::Floating,\n                    self.core.groupware.max_ical_instances,\n                    &mut next_email_alarm,\n                ),\n                size: itip_message.len() as u32,\n                schedule_tag: Some(1),\n                ..Default::default()\n            };\n\n            // Obtain document ids\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::CalendarEvent, 1)\n                .await\n                .caused_by(trc::location!())?;\n            let itip_document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::CalendarEventNotification, 1)\n                .await\n                .caused_by(trc::location!())?;\n            let itip_message = CalendarEventNotification {\n                event: itip,\n                event_id: Some(document_id),\n                changed_by,\n                size: itip_message.len() as u32,\n                ..Default::default()\n            };\n\n            // Prepare write batch\n            let mut batch = BatchBuilder::new();\n            event\n                .insert(\n                    access_token,\n                    account_id,\n                    document_id,\n                    next_email_alarm,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n            itip_message\n                .insert(access_token, account_id, itip_document_id, &mut batch)\n                .caused_by(trc::location!())?;\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n            Ok(None)\n        }\n    }\n\n    async fn http_rsvp_url(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        attendee: &str,\n    ) -> Option<ItipRsvpUrl> {\n        if let Some(base_url) = &self.core.groupware.itip_http_rsvp_url {\n            match self\n                .encode_access_token(\n                    GrantType::Rsvp,\n                    account_id,\n                    &format!(\"{attendee};{document_id}\"),\n                    self.core.groupware.itip_http_rsvp_expiration,\n                )\n                .await\n            {\n                Ok(access_token) => Some(ItipRsvpUrl(format!(\n                    \"{base_url}?i={}\",\n                    percent_encoding::percent_encode(access_token.as_bytes(), RFC_3986)\n                ))),\n                Err(err) => {\n                    trc::error!(err.caused_by(trc::location!()));\n                    None\n                }\n            }\n        } else {\n            None\n        }\n    }\n\n    async fn http_rsvp_handle(&self, query: &str, language: &str) -> trc::Result<String> {\n        let response = if let Some(rsvp) = decode_rsvp_response(self, query).await {\n            if let Some(archive) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    rsvp.account_id,\n                    Collection::CalendarEvent,\n                    rsvp.document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                let event = archive\n                    .to_unarchived::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n                let mut new_event = event\n                    .deserialize::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n                let mut did_change = false;\n                let mut summary = None;\n                let mut description = None;\n                let mut found_participant = false;\n\n                for component in &mut new_event.data.event.components {\n                    if component.component_type.is_scheduling_object() {\n                        'outer: for entry in &mut component.entries {\n                            if entry.name == ICalendarProperty::Attendee\n                                && entry\n                                    .calendar_address()\n                                    .is_some_and(|v| v.eq_ignore_ascii_case(&rsvp.attendee))\n                            {\n                                let mut add_partstat = true;\n                                for param in &mut entry.params {\n                                    if let (\n                                        ICalendarParameterName::Partstat,\n                                        ICalendarParameterValue::Partstat(partstat),\n                                    ) = (&param.name, &mut param.value)\n                                    {\n                                        if partstat != &rsvp.partstat {\n                                            *partstat = rsvp.partstat.clone();\n                                            add_partstat = false;\n                                        } else {\n                                            continue 'outer;\n                                        }\n                                    }\n                                }\n\n                                if add_partstat {\n                                    entry\n                                        .params\n                                        .push(ICalendarParameter::partstat(rsvp.partstat.clone()));\n                                }\n                                found_participant = true;\n                                did_change = true;\n                            } else if summary.is_none() && entry.name == ICalendarProperty::Summary\n                            {\n                                summary = entry\n                                    .values\n                                    .first()\n                                    .and_then(|v| v.as_text())\n                                    .map(|s| s.to_string());\n                            } else if description.is_none()\n                                && entry.name == ICalendarProperty::Description\n                            {\n                                description = entry\n                                    .values\n                                    .first()\n                                    .and_then(|v| v.as_text())\n                                    .map(|s| s.to_string());\n                            }\n                        }\n                    }\n                }\n\n                if did_change {\n                    // Prepare write batch\n                    let access_token = self\n                        .get_access_token(rsvp.account_id)\n                        .await\n                        .caused_by(trc::location!())?;\n                    let mut batch = BatchBuilder::new();\n                    new_event\n                        .update(\n                            &access_token,\n                            event,\n                            rsvp.account_id,\n                            rsvp.document_id,\n                            &mut batch,\n                        )\n                        .caused_by(trc::location!())?;\n\n                    self.commit_batch(batch).await.caused_by(trc::location!())?;\n                }\n\n                if found_participant {\n                    Response::Success {\n                        summary,\n                        description,\n                    }\n                } else {\n                    Response::NoLongerParticipant\n                }\n            } else {\n                Response::EventNotFound\n            }\n        } else {\n            Response::ParseError\n        };\n\n        Ok(render_response(self, response, language))\n    }\n}\n\nstruct RsvpResponse {\n    account_id: u32,\n    document_id: u32,\n    attendee: String,\n    partstat: ICalendarParticipationStatus,\n}\n\nasync fn decode_rsvp_response(server: &Server, query: &str) -> Option<RsvpResponse> {\n    let params = UrlParams::new(query.into());\n    let token = params.get(\"i\")?;\n    let method = params.get(\"m\").and_then(|m| {\n        hashify::tiny_map_ignore_case!(m.as_bytes(),\n            \"ACCEPTED\" => ICalendarParticipationStatus::Accepted,\n            \"DECLINED\" => ICalendarParticipationStatus::Declined,\n            \"TENTATIVE\" => ICalendarParticipationStatus::Tentative,\n            \"COMPLETED\" => ICalendarParticipationStatus::Completed,\n            \"IN-PROCESS\" => ICalendarParticipationStatus::InProcess,\n        )\n    })?;\n    let token = server\n        .validate_access_token(GrantType::Rsvp.into(), token)\n        .await\n        .ok()?;\n    let (attendee, document_id) =\n        token\n            .client_id\n            .rsplit_once(';')\n            .and_then(|(attendee, doc_id)| {\n                doc_id\n                    .parse::<u32>()\n                    .ok()\n                    .map(|doc_id| (attendee.to_string(), doc_id))\n            })?;\n\n    RsvpResponse {\n        account_id: token.account_id,\n        document_id,\n        attendee,\n        partstat: method,\n    }\n    .into()\n}\n\nenum Response {\n    Success {\n        summary: Option<String>,\n        description: Option<String>,\n    },\n    EventNotFound,\n    ParseError,\n    NoLongerParticipant,\n}\n\nfn render_response(server: &Server, response: Response, language: &str) -> String {\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    let template = server\n        .core\n        .enterprise\n        .as_ref()\n        .and_then(|e| e.template_scheduling_web.as_ref())\n        .unwrap_or(&server.core.groupware.itip_template);\n    // SPDX-SnippetEnd\n\n    #[cfg(not(feature = \"enterprise\"))]\n    let template = &server.core.groupware.itip_template;\n    let locale = i18n::locale_or_default(language);\n\n    let mut variables = Variables::new();\n\n    match response {\n        Response::Success {\n            summary,\n            description,\n        } => {\n            variables.insert_single(\n                CalendarTemplateVariable::PageTitle,\n                locale.calendar_rsvp_recorded.to_string(),\n            );\n            variables.insert_single(\n                CalendarTemplateVariable::Header,\n                locale.calendar_rsvp_recorded.to_string(),\n            );\n            variables.insert_block(\n                CalendarTemplateVariable::EventDetails,\n                [\n                    summary.map(|summary| {\n                        [\n                            (\n                                CalendarTemplateVariable::Key,\n                                locale.calendar_summary.to_string(),\n                            ),\n                            (CalendarTemplateVariable::Value, summary),\n                        ]\n                    }),\n                    description.map(|description| {\n                        [\n                            (\n                                CalendarTemplateVariable::Key,\n                                locale.calendar_description.to_string(),\n                            ),\n                            (CalendarTemplateVariable::Value, description),\n                        ]\n                    }),\n                ]\n                .into_iter()\n                .flatten(),\n            );\n\n            variables.insert_single(CalendarTemplateVariable::Color, \"info\".to_string());\n        }\n        Response::EventNotFound => {\n            variables.insert_single(\n                CalendarTemplateVariable::PageTitle,\n                locale.calendar_rsvp_failed.to_string(),\n            );\n            variables.insert_single(\n                CalendarTemplateVariable::Header,\n                locale.calendar_event_not_found.to_string(),\n            );\n            variables.insert_single(CalendarTemplateVariable::Color, \"danger\".to_string());\n        }\n        Response::ParseError => {\n            variables.insert_single(\n                CalendarTemplateVariable::PageTitle,\n                locale.calendar_rsvp_failed.to_string(),\n            );\n            variables.insert_single(\n                CalendarTemplateVariable::Header,\n                locale.calendar_invalid_rsvp.to_string(),\n            );\n            variables.insert_single(CalendarTemplateVariable::Color, \"danger\".to_string());\n        }\n        Response::NoLongerParticipant => {\n            variables.insert_single(\n                CalendarTemplateVariable::PageTitle,\n                locale.calendar_rsvp_failed.to_string(),\n            );\n            variables.insert_single(\n                CalendarTemplateVariable::Header,\n                locale.calendar_not_participant.to_string(),\n            );\n            variables.insert_single(CalendarTemplateVariable::Color, \"warning\".to_string());\n        }\n    }\n    variables.insert_single(CalendarTemplateVariable::LogoCid, \"/logo.svg\".to_string());\n\n    template.eval(&variables)\n}\n\nimpl ItipRsvpUrl {\n    pub fn url(&self, partstat: &ICalendarParticipationStatus) -> String {\n        format!(\"{}&m={}\", self.0, partstat.as_str())\n    }\n}\n\nimpl From<ItipError> for ItipIngestError {\n    fn from(err: ItipError) -> Self {\n        ItipIngestError::Message(err)\n    }\n}\n\nimpl From<trc::Error> for ItipIngestError {\n    fn from(err: trc::Error) -> Self {\n        ItipIngestError::Internal(err)\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/calendar/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod alarm;\npub mod dates;\npub mod expand;\npub mod index;\npub mod itip;\npub mod storage;\n\nuse calcard::icalendar::{\n    ICalendar, ICalendarComponent, ICalendarComponentType, ICalendarDuration, ICalendarEntry,\n};\nuse common::{DavName, auth::AccessToken};\nuse types::{acl::AclGrant, dead_property::DeadProperty};\nuse utils::map::bitmap::BitmapItem;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct Calendar {\n    pub name: String,\n    pub preferences: Vec<CalendarPreferences>,\n    pub acls: Vec<AclGrant>,\n    pub supported_components: u64,\n    pub dead_properties: DeadProperty,\n    pub created: i64,\n    pub modified: i64,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum SupportedComponent {\n    VCalendar,     // [RFC5545, Section 3.4]\n    VEvent,        // [RFC5545, Section 3.6.1]\n    VTodo,         // [RFC5545, Section 3.6.2]\n    VJournal,      // [RFC5545, Section 3.6.3]\n    VFreebusy,     // [RFC5545, Section 3.6.4]\n    VTimezone,     // [RFC5545, Section 3.6.5]\n    VAlarm,        // [RFC5545, Section 3.6.6]\n    Standard,      // [RFC5545, Section 3.6.5]\n    Daylight,      // [RFC5545, Section 3.6.5]\n    VAvailability, // [RFC7953, Section 3.1]\n    Available,     // [RFC7953, Section 3.1]\n    Participant,   // [RFC9073, Section 7.1]\n    VLocation,     // [RFC9073, Section 7.2] [RFC Errata 7381]\n    VResource,     // [RFC9073, Section 7.3]\n    VStatus,       // draft-ietf-calext-ical-tasks-14\n    Other,\n}\n\npub const CALENDAR_SUBSCRIBED: u16 = 1;\npub const CALENDAR_INVISIBLE: u16 = 1 << 1;\npub const CALENDAR_AVAILABILITY_NONE: u16 = 1 << 2;\npub const CALENDAR_AVAILABILITY_ATTENDING: u16 = 1 << 3;\npub const CALENDAR_AVAILABILITY_ALL: u16 = 1 << 4;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarPreferences {\n    pub account_id: u32,\n    pub name: String,\n    pub description: Option<String>,\n    pub sort_order: u32,\n    pub color: Option<String>,\n    pub flags: u16,\n    pub time_zone: Timezone,\n    pub default_alerts: Vec<DefaultAlert>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct DefaultAlert {\n    pub id: String,\n    pub offset: ICalendarDuration,\n    pub flags: u16,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct ParticipantIdentities {\n    pub identities: Vec<ParticipantIdentity>,\n    pub default_name: String,\n    pub default: u32,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct ParticipantIdentity {\n    pub id: u32,\n    pub name: Option<String>,\n    pub calendar_address: String,\n}\n\npub const ALERT_WITH_TIME: u16 = 1;\npub const ALERT_EMAIL: u16 = 1 << 1;\npub const ALERT_RELATIVE_TO_END: u16 = 1 << 2;\n\npub const SCHEDULE_INBOX_ID: u32 = u32::MAX - 1;\npub const SCHEDULE_OUTBOX_ID: u32 = u32::MAX - 2;\n\npub const EVENT_INVITE_SELF: u16 = 1;\npub const EVENT_INVITE_OTHERS: u16 = 1 << 1;\npub const EVENT_HIDE_ATTENDEES: u16 = 1 << 2;\npub const EVENT_DRAFT: u16 = 1 << 3;\n\npub const EVENT_NOTIFICATION_IS_DRAFT: u16 = 1;\npub const EVENT_NOTIFICATION_IS_CHANGE: u16 = 1 << 1;\n\npub const PREF_USE_DEFAULT_ALERTS: u16 = 1;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEvent {\n    pub names: Vec<DavName>,\n    pub display_name: Option<String>,\n    pub data: CalendarEventData,\n    pub preferences: Vec<EventPreferences>,\n    pub flags: u16,\n    pub dead_properties: DeadProperty,\n    pub size: u32,\n    pub created: i64,\n    pub modified: i64,\n    pub schedule_tag: Option<u32>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEventNotification {\n    pub event: ICalendar,\n    pub event_id: Option<u32>,\n    pub changed_by: ChangedBy,\n    pub flags: u16,\n    pub size: u32,\n    pub created: i64,\n    pub modified: i64,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub enum ChangedBy {\n    PrincipalId(u32),\n    CalendarAddress(String),\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEventData {\n    pub event: ICalendar,\n    pub time_ranges: Box<[ComponentTimeRange]>,\n    pub alarms: Box<[Alarm]>,\n    pub base_offset: i64,\n    pub base_time_utc: u32,\n    pub duration: u32,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\n#[rkyv(compare(PartialEq), derive(Debug))]\npub struct Alarm {\n    pub id: u16,\n    pub parent_id: u16,\n    pub delta: AlarmDelta,\n    pub is_email_alert: bool,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\n#[rkyv(compare(PartialEq), derive(Debug))]\npub enum AlarmDelta {\n    Start(i64),\n    End(i64),\n    FixedUtc(i64),\n    FixedFloating(i64),\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct ComponentTimeRange {\n    pub id: u16,\n    pub start_tz: u16,\n    pub end_tz: u16,\n    pub duration: i32,\n    pub instances: Box<[u8]>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct EventPreferences {\n    pub account_id: u32,\n    pub flags: u16,\n    pub properties: Vec<ICalendarEntry>,\n    pub alerts: Vec<ICalendarComponent>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub enum Timezone {\n    IANA(u16),\n    Custom(ICalendar),\n    #[default]\n    Default,\n}\n\nimpl Calendar {\n    pub fn preferences(&self, access_token: &AccessToken) -> &CalendarPreferences {\n        if self.preferences.len() == 1 {\n            &self.preferences[0]\n        } else {\n            let account_id = access_token.primary_id();\n            self.preferences\n                .iter()\n                .find(|p| p.account_id == account_id)\n                .or_else(|| self.preferences.first())\n                .unwrap()\n        }\n    }\n\n    pub fn preferences_mut(&mut self, access_token: &AccessToken) -> &mut CalendarPreferences {\n        let account_id = access_token.primary_id();\n        let idx = if let Some(idx) = self\n            .preferences\n            .iter()\n            .position(|p| p.account_id == account_id)\n        {\n            idx\n        } else {\n            let mut preferences = self.preferences[0].clone();\n            preferences.account_id = account_id;\n            self.preferences.push(preferences);\n            self.preferences.len() - 1\n        };\n\n        &mut self.preferences[idx]\n    }\n}\n\nimpl ArchivedCalendar {\n    pub fn default_alerts(\n        &self,\n        access_token: &AccessToken,\n        with_time: bool,\n    ) -> impl Iterator<Item = &ArchivedDefaultAlert> {\n        self.preferences(access_token)\n            .default_alerts\n            .iter()\n            .filter(move |a| (a.flags & ALERT_WITH_TIME != 0) == with_time)\n    }\n\n    pub fn preferences(&self, access_token: &AccessToken) -> &ArchivedCalendarPreferences {\n        if self.preferences.len() == 1 {\n            &self.preferences[0]\n        } else {\n            let account_id = access_token.primary_id();\n            self.preferences\n                .iter()\n                .find(|p| p.account_id == account_id)\n                .or_else(|| self.preferences.first())\n                .unwrap()\n        }\n    }\n}\n\nimpl CalendarEvent {\n    pub fn preferences(&self, access_token: &AccessToken) -> Option<&EventPreferences> {\n        self.preferences\n            .iter()\n            .find(|p| p.account_id == access_token.primary_id())\n    }\n\n    pub fn preferences_mut(&mut self, access_token: &AccessToken) -> &mut EventPreferences {\n        let account_id = access_token.primary_id();\n        let idx = if let Some(idx) = self\n            .preferences\n            .iter()\n            .position(|p| p.account_id == account_id)\n        {\n            idx\n        } else {\n            self.preferences.push(EventPreferences {\n                account_id,\n                flags: PREF_USE_DEFAULT_ALERTS,\n                properties: Vec::new(),\n                alerts: Vec::new(),\n            });\n            self.preferences.len() - 1\n        };\n\n        &mut self.preferences[idx]\n    }\n\n    pub fn added_calendar_ids(\n        &self,\n        prev_data: &ArchivedCalendarEvent,\n    ) -> impl Iterator<Item = u32> {\n        self.names\n            .iter()\n            .filter(|m| prev_data.names.iter().all(|pm| pm.parent_id != m.parent_id))\n            .map(|m| m.parent_id)\n    }\n\n    pub fn removed_calendar_ids(\n        &self,\n        prev_data: &ArchivedCalendarEvent,\n    ) -> impl Iterator<Item = u32> {\n        prev_data\n            .names\n            .iter()\n            .filter(|m| self.names.iter().all(|pm| pm.parent_id != m.parent_id))\n            .map(|m| m.parent_id.to_native())\n    }\n\n    pub fn unchanged_calendar_ids(\n        &self,\n        prev_data: &ArchivedCalendarEvent,\n    ) -> impl Iterator<Item = u32> {\n        self.names\n            .iter()\n            .filter(|m| prev_data.names.iter().any(|pm| pm.parent_id == m.parent_id))\n            .map(|m| m.parent_id)\n    }\n}\n\nimpl ArchivedCalendarEvent {\n    pub fn preferences(&self, access_token: &AccessToken) -> Option<&ArchivedEventPreferences> {\n        self.preferences\n            .iter()\n            .find(|p| p.account_id == access_token.primary_id())\n    }\n}\n\nimpl Default for ChangedBy {\n    fn default() -> Self {\n        ChangedBy::CalendarAddress(\"\".into())\n    }\n}\n\nimpl From<u64> for SupportedComponent {\n    fn from(value: u64) -> Self {\n        match value {\n            0 => SupportedComponent::VCalendar,\n            1 => SupportedComponent::VEvent,\n            2 => SupportedComponent::VTodo,\n            3 => SupportedComponent::VJournal,\n            4 => SupportedComponent::VFreebusy,\n            5 => SupportedComponent::VTimezone,\n            6 => SupportedComponent::VAlarm,\n            7 => SupportedComponent::Standard,\n            8 => SupportedComponent::Daylight,\n            9 => SupportedComponent::VAvailability,\n            10 => SupportedComponent::Available,\n            11 => SupportedComponent::Participant,\n            12 => SupportedComponent::VLocation,\n            13 => SupportedComponent::VResource,\n            14 => SupportedComponent::VStatus,\n            _ => SupportedComponent::Other,\n        }\n    }\n}\n\nimpl From<SupportedComponent> for u64 {\n    fn from(value: SupportedComponent) -> Self {\n        match value {\n            SupportedComponent::VCalendar => 0,\n            SupportedComponent::VEvent => 1,\n            SupportedComponent::VTodo => 2,\n            SupportedComponent::VJournal => 3,\n            SupportedComponent::VFreebusy => 4,\n            SupportedComponent::VTimezone => 5,\n            SupportedComponent::VAlarm => 6,\n            SupportedComponent::Standard => 7,\n            SupportedComponent::Daylight => 8,\n            SupportedComponent::VAvailability => 9,\n            SupportedComponent::Available => 10,\n            SupportedComponent::Participant => 11,\n            SupportedComponent::VLocation => 12,\n            SupportedComponent::VResource => 13,\n            SupportedComponent::VStatus => 14,\n            SupportedComponent::Other => 15,\n        }\n    }\n}\n\nimpl BitmapItem for SupportedComponent {\n    fn max() -> u64 {\n        u64::from(SupportedComponent::Other)\n    }\n\n    fn is_valid(&self) -> bool {\n        !matches!(self, SupportedComponent::Other)\n    }\n}\n\nimpl From<ICalendarComponentType> for SupportedComponent {\n    fn from(value: ICalendarComponentType) -> Self {\n        match value {\n            ICalendarComponentType::VCalendar => SupportedComponent::VCalendar,\n            ICalendarComponentType::VEvent => SupportedComponent::VEvent,\n            ICalendarComponentType::VTodo => SupportedComponent::VTodo,\n            ICalendarComponentType::VJournal => SupportedComponent::VJournal,\n            ICalendarComponentType::VFreebusy => SupportedComponent::VFreebusy,\n            ICalendarComponentType::VTimezone => SupportedComponent::VTimezone,\n            ICalendarComponentType::VAlarm => SupportedComponent::VAlarm,\n            ICalendarComponentType::Standard => SupportedComponent::Standard,\n            ICalendarComponentType::Daylight => SupportedComponent::Daylight,\n            ICalendarComponentType::VAvailability => SupportedComponent::VAvailability,\n            ICalendarComponentType::Available => SupportedComponent::Available,\n            ICalendarComponentType::Participant => SupportedComponent::Participant,\n            ICalendarComponentType::VLocation => SupportedComponent::VLocation,\n            ICalendarComponentType::VResource => SupportedComponent::VResource,\n            ICalendarComponentType::VStatus => SupportedComponent::VStatus,\n            _ => SupportedComponent::Other,\n        }\n    }\n}\n\nimpl From<SupportedComponent> for ICalendarComponentType {\n    fn from(value: SupportedComponent) -> Self {\n        match value {\n            SupportedComponent::VCalendar => ICalendarComponentType::VCalendar,\n            SupportedComponent::VEvent => ICalendarComponentType::VEvent,\n            SupportedComponent::VTodo => ICalendarComponentType::VTodo,\n            SupportedComponent::VJournal => ICalendarComponentType::VJournal,\n            SupportedComponent::VFreebusy => ICalendarComponentType::VFreebusy,\n            SupportedComponent::VTimezone => ICalendarComponentType::VTimezone,\n            SupportedComponent::VAlarm => ICalendarComponentType::VAlarm,\n            SupportedComponent::Standard => ICalendarComponentType::Standard,\n            SupportedComponent::Daylight => ICalendarComponentType::Daylight,\n            SupportedComponent::VAvailability => ICalendarComponentType::VAvailability,\n            SupportedComponent::Available => ICalendarComponentType::Available,\n            SupportedComponent::Participant => ICalendarComponentType::Participant,\n            SupportedComponent::VLocation => ICalendarComponentType::VLocation,\n            SupportedComponent::VResource => ICalendarComponentType::VResource,\n            SupportedComponent::VStatus => ICalendarComponentType::VStatus,\n            SupportedComponent::Other => ICalendarComponentType::Other(Default::default()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/calendar/storage.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ArchivedCalendar, ArchivedCalendarEvent, Calendar, CalendarEvent, CalendarPreferences,\n    alarm::CalendarAlarm,\n};\nuse crate::{\n    DavResourceName, DestroyArchive, RFC_3986,\n    calendar::{\n        ArchivedCalendarEventNotification, CalendarEventNotification, alarm::CalendarAlarmType,\n    },\n    scheduling::{ItipMessages, event_cancel::itip_cancel},\n};\nuse calcard::common::timezone::Tz;\nuse common::{Server, auth::AccessToken, storage::index::ObjectIndexBuilder};\nuse store::{\n    IterateParams, U16_LEN, U32_LEN, U64_LEN, ValueKey,\n    roaring::RoaringBitmap,\n    write::{\n        AlignedBytes, Archive, BatchBuilder, IndexPropertyClass, TaskEpoch, TaskQueueClass,\n        ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n        now,\n    },\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, VanishedCollection},\n    field::CalendarNotificationField,\n};\n\npub trait ItipAutoExpunge: Sync + Send {\n    fn itip_ids(&self, account_id: u32) -> impl Future<Output = trc::Result<RoaringBitmap>> + Send;\n\n    fn itip_auto_expunge(\n        &self,\n        account_id: u32,\n        hold_period: u64,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n}\n\nimpl ItipAutoExpunge for Server {\n    async fn itip_ids(&self, account_id: u32) -> trc::Result<RoaringBitmap> {\n        let mut document_ids = RoaringBitmap::new();\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::CalendarEventNotification.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: CalendarNotificationField::CreatedToId.into(),\n                            value: 0,\n                        }),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::CalendarEventNotification.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: CalendarNotificationField::CreatedToId.into(),\n                            value: u64::MAX,\n                        }),\n                    },\n                )\n                .no_values()\n                .ascending(),\n                |key, _| {\n                    document_ids.insert(key.deserialize_be_u32(key.len() - U32_LEN)?);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())\n            .map(|_| document_ids)\n    }\n\n    async fn itip_auto_expunge(&self, account_id: u32, hold_period: u64) -> trc::Result<()> {\n        let mut destroy_ids = RoaringBitmap::new();\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::CalendarEventNotification.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: CalendarNotificationField::CreatedToId.into(),\n                            value: 0,\n                        }),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::CalendarEventNotification.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: CalendarNotificationField::CreatedToId.into(),\n                            value: now().saturating_sub(hold_period),\n                        }),\n                    },\n                )\n                .no_values()\n                .ascending(),\n                |key, _| {\n                    destroy_ids.insert(key.deserialize_be_u32(key.len() - U32_LEN)?);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if destroy_ids.is_empty() {\n            return Ok(());\n        }\n\n        trc::event!(\n            Purge(trc::PurgeEvent::AutoExpunge),\n            AccountId = account_id,\n            Collection = Collection::CalendarEventNotification.as_str(),\n            Total = destroy_ids.len(),\n        );\n\n        // Tombstone messages\n        let mut batch = BatchBuilder::new();\n        let access_token = self\n            .get_access_token(account_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        for document_id in destroy_ids {\n            // Fetch event\n            if let Some(event_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEventNotification,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                let event = event_\n                    .to_unarchived::<CalendarEventNotification>()\n                    .caused_by(trc::location!())?;\n                DestroyArchive(event)\n                    .delete(&access_token, account_id, document_id, &mut batch)\n                    .caused_by(trc::location!())?;\n            }\n        }\n\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n        Ok(())\n    }\n}\n\nimpl CalendarEvent {\n    pub fn update<'x>(\n        self,\n        access_token: &AccessToken,\n        event: Archive<&ArchivedCalendarEvent>,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        let mut new_event = self;\n\n        // Build event\n        new_event.modified = now() as i64;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::CalendarEvent)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_current(event)\n                    .with_changes(new_event)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n\n    pub fn insert<'x>(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        next_alarm: Option<CalendarAlarm>,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build event\n        let mut event = self;\n        let now = now() as i64;\n        event.modified = now;\n        event.created = now;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::CalendarEvent)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_changes(event)\n                    .with_access_token(access_token),\n            )\n            .map(|batch| {\n                if let Some(next_alarm) = next_alarm {\n                    next_alarm.write_task(batch);\n                }\n\n                batch.commit_point()\n            })\n    }\n}\n\nimpl Calendar {\n    pub fn insert<'x>(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build address calendar\n        let mut calendar = self;\n        let now = now() as i64;\n        calendar.modified = now;\n        calendar.created = now;\n\n        if calendar.preferences.is_empty() {\n            calendar.preferences.push(CalendarPreferences {\n                account_id,\n                name: \"default\".to_string(),\n                ..Default::default()\n            });\n        }\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Calendar)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_changes(calendar)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n\n    pub fn update<'x>(\n        self,\n        access_token: &AccessToken,\n        calendar: Archive<&ArchivedCalendar>,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build address calendar\n        let mut new_calendar = self;\n        new_calendar.modified = now() as i64;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Calendar)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_current(calendar)\n                    .with_changes(new_calendar)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n}\n\nimpl CalendarEventNotification {\n    pub fn insert<'x>(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build event\n        let mut event = self;\n        let now = now() as i64;\n        event.modified = now;\n        event.created = now;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::CalendarEventNotification)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_changes(event)\n                    .with_access_token(access_token),\n            )\n            .map(|batch| batch.commit_point())\n    }\n}\n\nimpl DestroyArchive<Archive<&ArchivedCalendar>> {\n    #[allow(clippy::too_many_arguments)]\n    pub async fn delete_with_events(\n        self,\n        server: &Server,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        children_ids: Vec<u32>,\n        delete_path: Option<String>,\n        send_itip: bool,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        // Process deletions\n        let calendar_id = document_id;\n        for document_id in children_ids {\n            if let Some(event_) = server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await?\n            {\n                DestroyArchive(\n                    event_\n                        .to_unarchived::<CalendarEvent>()\n                        .caused_by(trc::location!())?,\n                )\n                .delete(\n                    access_token,\n                    account_id,\n                    document_id,\n                    calendar_id,\n                    None,\n                    send_itip,\n                    batch,\n                )?;\n            }\n        }\n\n        self.delete(access_token, account_id, document_id, delete_path, batch)\n    }\n\n    pub fn delete(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        delete_path: Option<String>,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        let calendar = self.0;\n        // Delete calendar\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Calendar)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<_, ()>::new()\n                    .with_access_token(access_token)\n                    .with_current(calendar),\n            )\n            .caused_by(trc::location!())?;\n        if let Some(delete_path) = delete_path {\n            batch.log_vanished_item(VanishedCollection::Calendar, delete_path);\n        }\n        batch.commit_point();\n\n        Ok(())\n    }\n}\n\nimpl DestroyArchive<Archive<&ArchivedCalendarEvent>> {\n    #[allow(clippy::too_many_arguments)]\n    pub fn delete(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        calendar_id: u32,\n        delete_path: Option<String>,\n        send_itip: bool,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        if let Some(delete_idx) = self\n            .0\n            .inner\n            .names\n            .iter()\n            .position(|name| name.parent_id == calendar_id)\n        {\n            if self.0.inner.names.len() > 1 {\n                // Unlink calendar id from event\n                let event = self.0;\n                let mut new_event = event\n                    .deserialize::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n                new_event.names.swap_remove(delete_idx);\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::CalendarEvent)\n                    .with_document(document_id)\n                    .custom(\n                        ObjectIndexBuilder::new()\n                            .with_access_token(access_token)\n                            .with_current(event)\n                            .with_changes(new_event),\n                    )\n                    .caused_by(trc::location!())?;\n            } else {\n                self.delete_all(access_token, account_id, document_id, send_itip, batch)?;\n            }\n\n            if let Some(delete_path) = delete_path {\n                batch.log_vanished_item(VanishedCollection::Calendar, delete_path);\n            }\n\n            batch.commit_point();\n        }\n\n        Ok(())\n    }\n\n    #[allow(clippy::too_many_arguments)]\n    pub fn delete_all(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        send_itip: bool,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        let event = self.0;\n        // Delete event\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::CalendarEvent)\n            .with_document(document_id);\n\n        // Remove next alarm if it exists\n        let now = now() as i64;\n        if let Some(next_alarm) = event.inner.data.next_alarm(now, Tz::Floating) {\n            next_alarm.delete_task(batch);\n        }\n\n        // Scheduling\n        if send_itip\n            && event.inner.schedule_tag.is_some()\n            && event.inner.data.event_range_end() > now\n        {\n            let event = event\n                .deserialize::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            if let Ok(messages) =\n                itip_cancel(&event.data.event, access_token.emails.as_slice(), true)\n            {\n                ItipMessages::new(vec![messages])\n                    .queue(batch)\n                    .caused_by(trc::location!())?;\n            }\n        }\n\n        batch\n            .custom(\n                ObjectIndexBuilder::<_, ()>::new()\n                    .with_access_token(access_token)\n                    .with_current(event),\n            )\n            .caused_by(trc::location!())?;\n\n        Ok(())\n    }\n}\n\nimpl DestroyArchive<Archive<&ArchivedCalendarEventNotification>> {\n    #[allow(clippy::too_many_arguments)]\n    pub fn delete(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        // Delete event\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::CalendarEventNotification)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<_, ()>::new()\n                    .with_access_token(access_token)\n                    .with_current(self.0),\n            )\n            .caused_by(trc::location!())?\n            .commit_point();\n\n        Ok(())\n    }\n}\n\nimpl CalendarAlarm {\n    pub fn write_task(&self, batch: &mut BatchBuilder) {\n        match &self.typ {\n            CalendarAlarmType::Email {\n                event_start,\n                event_start_tz,\n                event_end,\n                event_end_tz,\n            } => {\n                batch.set(\n                    ValueClass::TaskQueue(TaskQueueClass::SendAlarm {\n                        due: TaskEpoch::new(self.alarm_time as u64),\n                        event_id: self.event_id,\n                        alarm_id: self.alarm_id,\n                        is_email_alert: true,\n                    }),\n                    KeySerializer::new((U64_LEN * 2) + (U16_LEN * 2))\n                        .write(*event_start as u64)\n                        .write(*event_end as u64)\n                        .write(*event_start_tz)\n                        .write(*event_end_tz)\n                        .finalize(),\n                );\n            }\n            CalendarAlarmType::Display { recurrence_id } => {\n                batch.set(\n                    ValueClass::TaskQueue(TaskQueueClass::SendAlarm {\n                        due: TaskEpoch::new(self.alarm_time as u64),\n                        event_id: self.event_id,\n                        alarm_id: self.alarm_id,\n                        is_email_alert: false,\n                    }),\n                    KeySerializer::new(U64_LEN)\n                        .write(recurrence_id.unwrap_or_default() as u64)\n                        .finalize(),\n                );\n            }\n        }\n    }\n\n    pub fn delete_task(&self, batch: &mut BatchBuilder) {\n        batch.clear(ValueClass::TaskQueue(TaskQueueClass::SendAlarm {\n            due: TaskEpoch::new(self.alarm_time as u64),\n            event_id: self.event_id,\n            alarm_id: self.alarm_id,\n            is_email_alert: matches!(self.typ, CalendarAlarmType::Email { .. }),\n        }));\n    }\n}\n\nimpl ArchivedCalendarEvent {\n    pub async fn webcal_uri(\n        &self,\n        server: &Server,\n        access_token: &AccessToken,\n    ) -> trc::Result<String> {\n        for event_name in self.names.iter() {\n            if let Some(calendar_) = server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    access_token.primary_id,\n                    Collection::Calendar,\n                    event_name.parent_id.to_native(),\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                let calendar = calendar_\n                    .unarchive::<Calendar>()\n                    .caused_by(trc::location!())?;\n                return Ok(format!(\n                    \"webcal://{}{}/{}/{}/{}\",\n                    server.core.network.server_name,\n                    DavResourceName::Cal.base_path(),\n                    percent_encoding::utf8_percent_encode(&access_token.name, RFC_3986),\n                    calendar.name,\n                    event_name.name\n                ));\n            }\n        }\n\n        Err(trc::StoreEvent::UnexpectedError\n            .into_err()\n            .details(\"Event is not linked to any calendar\"))\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/contact/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{AddressBook, ArchivedAddressBook, ArchivedContactCard, ContactCard};\nuse ahash::AHashSet;\nuse calcard::{\n    common::IanaString,\n    vcard::{ArchivedVCardProperty, ArchivedVCardValue, VCardProperty},\n};\nuse common::storage::index::{IndexValue, IndexableAndSerializableObject, IndexableObject};\nuse nlp::language::{\n    Language,\n    detect::{LanguageDetector, MIN_LANGUAGE_SCORE},\n};\nuse store::{\n    search::{ContactSearchField, IndexDocument, SearchField},\n    write::{IndexPropertyClass, SearchIndex, ValueClass},\n    xxhash_rust::xxh3,\n};\nuse types::{acl::AclGrant, collection::SyncCollection, field::ContactField};\nuse utils::sanitize_email;\n\nimpl IndexableObject for AddressBook {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Acl {\n                value: (&self.acls).into(),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogContainer {\n                sync_collection: SyncCollection::AddressBook,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedAddressBook {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Acl {\n                value: self\n                    .acls\n                    .iter()\n                    .map(AclGrant::from)\n                    .collect::<Vec<_>>()\n                    .into(),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogContainer {\n                sync_collection: SyncCollection::AddressBook,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for AddressBook {\n    fn is_versioned() -> bool {\n        true\n    }\n}\n\nimpl IndexableObject for ContactCard {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Index {\n                field: ContactField::Uid.into(),\n                value: self.card.uid().into(),\n            },\n            IndexValue::Index {\n                field: ContactField::Email.into(),\n                value: self.emails().next().into(),\n            },\n            IndexValue::Property {\n                field: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                    property: ContactField::CreatedToUpdated.into(),\n                    value: self.created as u64,\n                }),\n                value: self.modified.into(),\n            },\n            IndexValue::SearchIndex {\n                index: SearchIndex::Contacts,\n                hash: self.hashes().fold(0, |acc, hash| acc ^ hash),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::AddressBook,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedContactCard {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        [\n            IndexValue::Index {\n                field: ContactField::Uid.into(),\n                value: self.card.uid().into(),\n            },\n            IndexValue::Index {\n                field: ContactField::Email.into(),\n                value: self.emails().next().into(),\n            },\n            IndexValue::Property {\n                field: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                    property: ContactField::CreatedToUpdated.into(),\n                    value: self.created.to_native() as u64,\n                }),\n                value: (self.modified.to_native() as u64).into(),\n            },\n            IndexValue::SearchIndex {\n                index: SearchIndex::Contacts,\n                hash: self.hashes().fold(0, |acc, hash| acc ^ hash),\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n            IndexValue::LogItem {\n                sync_collection: SyncCollection::AddressBook,\n                prefix: None,\n            },\n        ]\n        .into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for ContactCard {\n    fn is_versioned() -> bool {\n        true\n    }\n}\n\nimpl AddressBook {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self\n                .preferences\n                .iter()\n                .map(|p| p.name.len() + p.description.as_ref().map_or(0, |n| n.len()))\n                .sum::<usize>()\n            + self.name.len()\n            + std::mem::size_of::<AddressBook>()\n    }\n}\n\nimpl ArchivedAddressBook {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self\n                .preferences\n                .iter()\n                .map(|p| p.name.len() + p.description.as_ref().map_or(0, |n| n.len()))\n                .sum::<usize>()\n            + self.name.len()\n            + std::mem::size_of::<AddressBook>()\n    }\n}\n\nimpl ContactCard {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.display_name.as_ref().map_or(0, |n| n.len())\n            + self.names.iter().map(|n| n.name.len()).sum::<usize>()\n            + self.size as usize\n            + std::mem::size_of::<ContactCard>()\n    }\n\n    pub fn hashes(&self) -> impl Iterator<Item = u64> {\n        self.card\n            .entries\n            .iter()\n            .filter(|e| {\n                matches!(\n                    e.name,\n                    VCardProperty::Adr\n                        | VCardProperty::N\n                        | VCardProperty::Fn\n                        | VCardProperty::Title\n                        | VCardProperty::Org\n                        | VCardProperty::Note\n                        | VCardProperty::Nickname\n                        | VCardProperty::Email\n                        | VCardProperty::Kind\n                        | VCardProperty::Uid\n                        | VCardProperty::Member\n                        | VCardProperty::Impp\n                        | VCardProperty::Socialprofile\n                        | VCardProperty::Tel\n                )\n            })\n            .flat_map(|e| e.values.iter().filter_map(|v| v.as_text()))\n            .map(|v| xxh3::xxh3_64(v.as_bytes()))\n    }\n\n    pub fn emails(&self) -> impl Iterator<Item = String> {\n        self.card.properties(&VCardProperty::Email).flat_map(|e| {\n            e.values\n                .iter()\n                .filter_map(|v| v.as_text().and_then(sanitize_email))\n        })\n    }\n}\n\nimpl ArchivedContactCard {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.display_name.as_ref().map_or(0, |n| n.len())\n            + self.names.iter().map(|n| n.name.len()).sum::<usize>()\n            + self.size.to_native() as usize\n            + std::mem::size_of::<ContactCard>()\n    }\n\n    pub fn hashes(&self) -> impl Iterator<Item = u64> {\n        self.card\n            .entries\n            .iter()\n            .filter(|e| {\n                matches!(\n                    e.name,\n                    ArchivedVCardProperty::Adr\n                        | ArchivedVCardProperty::N\n                        | ArchivedVCardProperty::Fn\n                        | ArchivedVCardProperty::Title\n                        | ArchivedVCardProperty::Org\n                        | ArchivedVCardProperty::Note\n                        | ArchivedVCardProperty::Nickname\n                        | ArchivedVCardProperty::Email\n                        | ArchivedVCardProperty::Kind\n                        | ArchivedVCardProperty::Uid\n                        | ArchivedVCardProperty::Member\n                        | ArchivedVCardProperty::Impp\n                        | ArchivedVCardProperty::Socialprofile\n                        | ArchivedVCardProperty::Tel\n                )\n            })\n            .flat_map(|e| e.values.iter().filter_map(|v| v.as_text()))\n            .map(|v| xxh3::xxh3_64(v.as_bytes()))\n    }\n\n    pub fn emails(&self) -> impl Iterator<Item = String> {\n        self.card.properties(&VCardProperty::Email).flat_map(|e| {\n            e.values\n                .iter()\n                .filter_map(|v| v.as_text().and_then(sanitize_email))\n        })\n    }\n\n    pub fn index_document(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        index_fields: &AHashSet<SearchField>,\n        default_language: Language,\n    ) -> IndexDocument {\n        let mut document = IndexDocument::new(SearchIndex::Contacts)\n            .with_account_id(account_id)\n            .with_document_id(document_id);\n        let mut detector = LanguageDetector::new();\n\n        for entry in self.card.entries.iter() {\n            let (is_text, is_keyword, field) = match entry.name {\n                ArchivedVCardProperty::N => (false, false, ContactSearchField::Name),\n                ArchivedVCardProperty::Nickname => (false, false, ContactSearchField::Nickname),\n                ArchivedVCardProperty::Org => (false, false, ContactSearchField::Organization),\n                ArchivedVCardProperty::Email => (false, false, ContactSearchField::Email),\n                ArchivedVCardProperty::Tel => (false, false, ContactSearchField::Phone),\n                ArchivedVCardProperty::Impp | ArchivedVCardProperty::Socialprofile => {\n                    (false, false, ContactSearchField::OnlineService)\n                }\n                ArchivedVCardProperty::Adr => (false, false, ContactSearchField::Address),\n                ArchivedVCardProperty::Note => (true, false, ContactSearchField::Note),\n                ArchivedVCardProperty::Kind => (false, true, ContactSearchField::Kind),\n                ArchivedVCardProperty::Uid => (false, true, ContactSearchField::Uid),\n                ArchivedVCardProperty::Member => (false, false, ContactSearchField::Member),\n                _ => continue,\n            };\n            let field = SearchField::Contact(field);\n\n            if index_fields.is_empty() || index_fields.contains(&field) {\n                for value in entry.values.iter() {\n                    match value {\n                        ArchivedVCardValue::Text(v) => {\n                            if !is_keyword {\n                                let lang = if is_text {\n                                    detector.detect(v.as_str().trim(), MIN_LANGUAGE_SCORE);\n                                    Language::Unknown\n                                } else {\n                                    Language::None\n                                };\n\n                                document.index_text(field.clone(), v, lang);\n                            } else {\n                                document.index_keyword(field.clone(), v.as_str());\n                            }\n                        }\n                        ArchivedVCardValue::Kind(v) => {\n                            document.index_keyword(field.clone(), v.as_str());\n                        }\n                        ArchivedVCardValue::Component(v) => {\n                            for item in v.iter() {\n                                document.index_text(field.clone(), item.trim(), Language::None);\n                            }\n                        }\n                        _ => (),\n                    }\n                }\n\n                /*for param in entry.params.iter() {\n                    if let ArchivedVCardParameterValue::Text(value) = &param.value {\n                        let lang = if is_text {\n                            detector.detect(value.as_str(), MIN_LANGUAGE_SCORE);\n                            Language::Unknown\n                        } else {\n                            Language::None\n                        };\n                        document.index_text(field.clone(), value, lang);\n                    }\n                }*/\n            }\n        }\n\n        document.set_unknown_language(\n            detector\n                .most_frequent_language()\n                .unwrap_or(default_language),\n        );\n\n        document\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/contact/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod index;\npub mod storage;\n\nuse calcard::vcard::VCard;\nuse common::{DavName, auth::AccessToken};\nuse types::{acl::AclGrant, dead_property::DeadProperty};\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct AddressBook {\n    pub name: String,\n    pub preferences: Vec<AddressBookPreferences>,\n    pub subscribers: Vec<u32>,\n    pub dead_properties: DeadProperty,\n    pub acls: Vec<AclGrant>,\n    pub created: i64,\n    pub modified: i64,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct AddressBookPreferences {\n    pub account_id: u32,\n    pub name: String,\n    pub description: Option<String>,\n    pub sort_order: u32,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct ContactCard {\n    pub names: Vec<DavName>,\n    pub display_name: Option<String>,\n    pub card: VCard,\n    pub dead_properties: DeadProperty,\n    pub created: i64,\n    pub modified: i64,\n    pub size: u32,\n}\n\nimpl AddressBook {\n    pub fn preferences(&self, access_token: &AccessToken) -> &AddressBookPreferences {\n        if self.preferences.len() == 1 {\n            &self.preferences[0]\n        } else {\n            let account_id = access_token.primary_id();\n            self.preferences\n                .iter()\n                .find(|p| p.account_id == account_id)\n                .or_else(|| self.preferences.first())\n                .unwrap()\n        }\n    }\n\n    pub fn preferences_mut(&mut self, access_token: &AccessToken) -> &mut AddressBookPreferences {\n        let account_id = access_token.primary_id();\n        let idx = if let Some(idx) = self\n            .preferences\n            .iter()\n            .position(|p| p.account_id == account_id)\n        {\n            idx\n        } else {\n            let mut preferences = self.preferences[0].clone();\n            preferences.account_id = account_id;\n            self.preferences.push(preferences);\n            self.preferences.len() - 1\n        };\n\n        &mut self.preferences[idx]\n    }\n}\n\nimpl ArchivedAddressBook {\n    pub fn preferences(&self, access_token: &AccessToken) -> &ArchivedAddressBookPreferences {\n        if self.preferences.len() == 1 {\n            &self.preferences[0]\n        } else {\n            let account_id = access_token.primary_id();\n            self.preferences\n                .iter()\n                .find(|p| p.account_id == account_id)\n                .or_else(|| self.preferences.first())\n                .unwrap()\n        }\n    }\n}\n\nimpl ContactCard {\n    pub fn added_addressbook_ids(\n        &self,\n        prev_data: &ArchivedContactCard,\n    ) -> impl Iterator<Item = u32> {\n        self.names\n            .iter()\n            .filter(|m| prev_data.names.iter().all(|pm| pm.parent_id != m.parent_id))\n            .map(|m| m.parent_id)\n    }\n\n    pub fn removed_addressbook_ids(\n        &self,\n        prev_data: &ArchivedContactCard,\n    ) -> impl Iterator<Item = u32> {\n        prev_data\n            .names\n            .iter()\n            .filter(|m| self.names.iter().all(|pm| pm.parent_id != m.parent_id))\n            .map(|m| m.parent_id.to_native())\n    }\n\n    pub fn unchanged_addressbook_ids(\n        &self,\n        prev_data: &ArchivedContactCard,\n    ) -> impl Iterator<Item = u32> {\n        self.names\n            .iter()\n            .filter(|m| prev_data.names.iter().any(|pm| pm.parent_id == m.parent_id))\n            .map(|m| m.parent_id)\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/contact/storage.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{AddressBook, ArchivedAddressBook, ArchivedContactCard, ContactCard};\nuse crate::DestroyArchive;\nuse common::{Server, auth::AccessToken, storage::index::ObjectIndexBuilder};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder, now},\n};\nuse trc::AddContext;\nuse types::collection::{Collection, VanishedCollection};\n\nimpl ContactCard {\n    pub fn update<'x>(\n        self,\n        access_token: &AccessToken,\n        card: Archive<&ArchivedContactCard>,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        let mut new_card = self;\n\n        // Build card\n        new_card.modified = now() as i64;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::ContactCard)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_current(card)\n                    .with_changes(new_card)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n\n    pub fn insert<'x>(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build card\n        let mut card = self;\n        let now = now() as i64;\n        card.modified = now;\n        card.created = now;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::ContactCard)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_changes(card)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n}\n\nimpl AddressBook {\n    pub fn insert<'x>(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build address book\n        let mut book = self;\n        let now = now() as i64;\n        book.modified = now;\n        book.created = now;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::AddressBook)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_changes(book)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n\n    pub fn update<'x>(\n        self,\n        access_token: &AccessToken,\n        book: Archive<&ArchivedAddressBook>,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build address book\n        let mut new_book = self;\n        new_book.modified = now() as i64;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::AddressBook)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_current(book)\n                    .with_changes(new_book)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n}\n\nimpl DestroyArchive<Archive<&ArchivedAddressBook>> {\n    #[allow(clippy::too_many_arguments)]\n    pub async fn delete_with_cards(\n        self,\n        server: &Server,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        children_ids: Vec<u32>,\n        delete_path: Option<String>,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        // Process deletions\n        let addressbook_id = document_id;\n        for document_id in children_ids {\n            if let Some(card_) = server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::ContactCard,\n                    document_id,\n                ))\n                .await?\n            {\n                DestroyArchive(\n                    card_\n                        .to_unarchived::<ContactCard>()\n                        .caused_by(trc::location!())?,\n                )\n                .delete(\n                    access_token,\n                    account_id,\n                    document_id,\n                    addressbook_id,\n                    None,\n                    batch,\n                )?;\n            }\n        }\n\n        self.delete(access_token, account_id, document_id, delete_path, batch)\n    }\n\n    pub fn delete(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        delete_path: Option<String>,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        let book = self.0;\n        // Delete addressbook\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::AddressBook)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<_, ()>::new()\n                    .with_access_token(access_token)\n                    .with_current(book),\n            )\n            .caused_by(trc::location!())?;\n\n        if let Some(delete_path) = delete_path {\n            batch.log_vanished_item(VanishedCollection::AddressBook, delete_path);\n        }\n\n        batch.commit_point();\n\n        Ok(())\n    }\n}\n\nimpl DestroyArchive<Archive<&ArchivedContactCard>> {\n    pub fn delete(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        addressbook_id: u32,\n        delete_path: Option<String>,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        let card = self.0;\n        if let Some(delete_idx) = card\n            .inner\n            .names\n            .iter()\n            .position(|name| name.parent_id == addressbook_id)\n        {\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::ContactCard);\n\n            if card.inner.names.len() > 1 {\n                // Unlink addressbook id from card\n                let mut new_card = card\n                    .deserialize::<ContactCard>()\n                    .caused_by(trc::location!())?;\n                new_card.names.swap_remove(delete_idx);\n                batch\n                    .with_document(document_id)\n                    .custom(\n                        ObjectIndexBuilder::new()\n                            .with_access_token(access_token)\n                            .with_current(card)\n                            .with_changes(new_card),\n                    )\n                    .caused_by(trc::location!())?;\n            } else {\n                // Delete card\n                batch\n                    .with_document(document_id)\n                    .custom(\n                        ObjectIndexBuilder::<_, ()>::new()\n                            .with_access_token(access_token)\n                            .with_current(card),\n                    )\n                    .caused_by(trc::location!())?;\n            }\n\n            if let Some(delete_path) = delete_path {\n                batch.log_vanished_item(VanishedCollection::AddressBook, delete_path);\n            }\n\n            batch.commit_point();\n        }\n\n        Ok(())\n    }\n\n    pub fn delete_all(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::ContactCard)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<_, ()>::new()\n                    .with_access_token(access_token)\n                    .with_current(self.0),\n            )\n            .caused_by(trc::location!())\n            .map(|b| {\n                b.commit_point();\n            })\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/file/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ArchivedFileNode, FileNode};\nuse common::storage::index::{IndexValue, IndexableAndSerializableObject, IndexableObject};\nuse types::{acl::AclGrant, collection::SyncCollection};\n\nimpl IndexableObject for FileNode {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        let mut values = Vec::with_capacity(6);\n\n        values.extend([\n            IndexValue::Acl {\n                value: (&self.acls).into(),\n            },\n            IndexValue::LogItem {\n                prefix: None,\n                sync_collection: SyncCollection::FileNode,\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n        ]);\n\n        if let Some(file) = &self.file {\n            values.extend([IndexValue::Blob {\n                value: file.blob_hash.clone(),\n            }]);\n        }\n\n        values.into_iter()\n    }\n}\n\nimpl IndexableObject for &ArchivedFileNode {\n    fn index_values(&self) -> impl Iterator<Item = IndexValue<'_>> {\n        let mut values = Vec::with_capacity(6);\n\n        values.extend([\n            IndexValue::Acl {\n                value: self\n                    .acls\n                    .iter()\n                    .map(AclGrant::from)\n                    .collect::<Vec<_>>()\n                    .into(),\n            },\n            IndexValue::LogItem {\n                prefix: None,\n                sync_collection: SyncCollection::FileNode,\n            },\n            IndexValue::Quota {\n                used: self.size() as u32,\n            },\n        ]);\n\n        if let Some(file) = self.file.as_ref() {\n            values.extend([IndexValue::Blob {\n                value: (&file.blob_hash).into(),\n            }]);\n        }\n\n        values.into_iter()\n    }\n}\n\nimpl IndexableAndSerializableObject for FileNode {\n    fn is_versioned() -> bool {\n        true\n    }\n}\n\nimpl FileNode {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.display_name.as_ref().map_or(0, |n| n.len())\n            + self.name.len()\n            + self.file.as_ref().map_or(0, |f| f.size as usize)\n            + std::mem::size_of::<FileNode>()\n    }\n}\n\nimpl ArchivedFileNode {\n    pub fn size(&self) -> usize {\n        self.dead_properties.size()\n            + self.display_name.as_ref().map_or(0, |n| n.len())\n            + self.name.len()\n            + self\n                .file\n                .as_ref()\n                .map_or(0, |f| f.size.to_native() as usize)\n            + std::mem::size_of::<FileNode>()\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/file/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod index;\npub mod storage;\n\nuse types::{acl::AclGrant, blob_hash::BlobHash, dead_property::DeadProperty};\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct FileNode {\n    pub parent_id: u32,\n    pub name: String,\n    pub display_name: Option<String>,\n    pub file: Option<FileProperties>,\n    pub created: i64,\n    pub modified: i64,\n    pub dead_properties: DeadProperty,\n    pub acls: Vec<AclGrant>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct FileProperties {\n    pub blob_hash: BlobHash,\n    pub size: u32,\n    pub media_type: Option<String>,\n    pub executable: bool,\n}\n"
  },
  {
    "path": "crates/groupware/src/file/storage.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ArchivedFileNode, FileNode};\nuse crate::DestroyArchive;\nuse common::{Server, auth::AccessToken, storage::index::ObjectIndexBuilder};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder, now},\n};\nuse trc::AddContext;\nuse types::collection::{Collection, VanishedCollection};\n\nimpl FileNode {\n    pub fn insert<'x>(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build node\n        let mut node = self;\n        let now = now() as i64;\n        node.modified = now;\n        node.created = now;\n\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::FileNode)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<(), _>::new()\n                    .with_changes(node)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n    pub fn update<'x>(\n        self,\n        access_token: &AccessToken,\n        node: Archive<&ArchivedFileNode>,\n        account_id: u32,\n        document_id: u32,\n        batch: &'x mut BatchBuilder,\n    ) -> trc::Result<&'x mut BatchBuilder> {\n        // Build node\n        let mut new_node = self;\n        new_node.modified = now() as i64;\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::FileNode)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_current(node)\n                    .with_changes(new_node)\n                    .with_access_token(access_token),\n            )\n            .map(|b| b.commit_point())\n    }\n}\n\nimpl DestroyArchive<Archive<&ArchivedFileNode>> {\n    pub fn delete(\n        self,\n        access_token: &AccessToken,\n        account_id: u32,\n        document_id: u32,\n        batch: &mut BatchBuilder,\n        path: String,\n    ) -> trc::Result<()> {\n        // Prepare write batch\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::FileNode)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::<_, ()>::new()\n                    .with_current(self.0)\n                    .with_access_token(access_token),\n            )?\n            .log_vanished_item(VanishedCollection::FileNode, path)\n            .commit_point();\n        Ok(())\n    }\n}\n\nimpl DestroyArchive<Vec<u32>> {\n    pub async fn delete(\n        self,\n        server: &Server,\n        access_token: &AccessToken,\n        account_id: u32,\n        delete_path: Option<String>,\n    ) -> trc::Result<()> {\n        // Process deletions\n        let mut batch = BatchBuilder::new();\n        self.delete_batch(server, access_token, account_id, delete_path, &mut batch)\n            .await?;\n        // Write changes\n        if !batch.is_empty() {\n            server\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(())\n    }\n\n    pub async fn delete_batch(\n        self,\n        server: &Server,\n        access_token: &AccessToken,\n        account_id: u32,\n        delete_path: Option<String>,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        // Process deletions\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::FileNode);\n        for document_id in self.0 {\n            if let Some(node) = server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::FileNode,\n                    document_id,\n                ))\n                .await?\n            {\n                // Delete record\n                batch\n                    .with_document(document_id)\n                    .custom(\n                        ObjectIndexBuilder::<_, ()>::new()\n                            .with_access_token(access_token)\n                            .with_current(\n                                node.to_unarchived::<FileNode>()\n                                    .caused_by(trc::location!())?,\n                            ),\n                    )\n                    .caused_by(trc::location!())?\n                    .commit_point();\n            }\n        }\n\n        if !batch.is_empty()\n            && let Some(delete_path) = delete_path\n        {\n            batch.log_vanished_item(VanishedCollection::FileNode, delete_path);\n        }\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse calcard::common::timezone::Tz;\nuse common::DavResources;\nuse percent_encoding::{AsciiSet, CONTROLS};\nuse types::collection::{Collection, SyncCollection};\n\npub mod cache;\npub mod calendar;\npub mod contact;\npub mod file;\npub mod scheduling;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum DavResourceName {\n    Card,\n    Cal,\n    File,\n    Principal,\n    Scheduling,\n}\n\npub const RFC_3986: &AsciiSet = &CONTROLS\n    .add(b' ')\n    .add(b'!')\n    .add(b'\"')\n    .add(b'#')\n    .add(b'$')\n    .add(b'%')\n    .add(b'&')\n    .add(b'\\'')\n    .add(b'(')\n    .add(b')')\n    .add(b'*')\n    .add(b'+')\n    .add(b',')\n    .add(b'/')\n    .add(b':')\n    .add(b';')\n    .add(b'<')\n    .add(b'=')\n    .add(b'>')\n    .add(b'?')\n    .add(b'@')\n    .add(b'[')\n    .add(b'\\\\')\n    .add(b']')\n    .add(b'^')\n    .add(b'`')\n    .add(b'{')\n    .add(b'|')\n    .add(b'}');\n\npub struct DestroyArchive<T>(pub T);\n\nimpl DavResourceName {\n    pub fn parse(service: &str) -> Option<Self> {\n        hashify::tiny_map!(service.as_bytes(),\n            \"card\" => DavResourceName::Card,\n            \"cal\" => DavResourceName::Cal,\n            \"file\" => DavResourceName::File,\n            \"pal\" => DavResourceName::Principal,\n            \"itip\" => DavResourceName::Scheduling,\n        )\n    }\n\n    pub fn base_path(&self) -> &'static str {\n        match self {\n            DavResourceName::Card => \"/dav/card\",\n            DavResourceName::Cal => \"/dav/cal\",\n            DavResourceName::File => \"/dav/file\",\n            DavResourceName::Principal => \"/dav/pal\",\n            DavResourceName::Scheduling => \"/dav/itip\",\n        }\n    }\n\n    pub fn collection_path(&self) -> &'static str {\n        match self {\n            DavResourceName::Card => \"/dav/card/\",\n            DavResourceName::Cal => \"/dav/cal/\",\n            DavResourceName::File => \"/dav/file/\",\n            DavResourceName::Principal => \"/dav/pal/\",\n            DavResourceName::Scheduling => \"/dav/itip/\",\n        }\n    }\n\n    pub fn name(&self) -> &'static str {\n        match self {\n            DavResourceName::Card => \"CardDAV\",\n            DavResourceName::Cal => \"CalDAV\",\n            DavResourceName::File => \"WebDAV\",\n            DavResourceName::Principal => \"Principal\",\n            DavResourceName::Scheduling => \"Scheduling\",\n        }\n    }\n}\n\nimpl From<DavResourceName> for Collection {\n    fn from(value: DavResourceName) -> Self {\n        match value {\n            DavResourceName::Card => Collection::AddressBook,\n            DavResourceName::Cal => Collection::Calendar,\n            DavResourceName::File => Collection::FileNode,\n            DavResourceName::Principal => Collection::Principal,\n            DavResourceName::Scheduling => Collection::CalendarEventNotification,\n        }\n    }\n}\n\nimpl From<Collection> for DavResourceName {\n    fn from(value: Collection) -> Self {\n        match value {\n            Collection::AddressBook => DavResourceName::Card,\n            Collection::Calendar => DavResourceName::Cal,\n            Collection::FileNode => DavResourceName::File,\n            Collection::Principal => DavResourceName::Principal,\n            Collection::CalendarEventNotification => DavResourceName::Scheduling,\n            _ => unreachable!(),\n        }\n    }\n}\n\nimpl From<SyncCollection> for DavResourceName {\n    fn from(value: SyncCollection) -> Self {\n        match value {\n            SyncCollection::AddressBook => DavResourceName::Card,\n            SyncCollection::Calendar => DavResourceName::Cal,\n            SyncCollection::FileNode => DavResourceName::File,\n            SyncCollection::CalendarEventNotification => DavResourceName::Scheduling,\n            _ => unreachable!(),\n        }\n    }\n}\n\npub trait DavCalendarResource {\n    fn calendar_default_tz(&self, calendar_id: u32, account_id: u32) -> Option<Tz>;\n}\n\nimpl DavCalendarResource for DavResources {\n    fn calendar_default_tz(&self, calendar_id: u32, account_id: u32) -> Option<Tz> {\n        self.container_resource_by_id(calendar_id)\n            .and_then(|c| c.calendar_preferences(account_id))\n            .map(|p| p.tz)\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/attendee.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{\n    Email, InstanceId, ItipEntryValue, ItipError, ItipMessage, ItipSnapshot, ItipSnapshots,\n    ItipSummary,\n    itip::{\n        ItipExportAs, can_attendee_modify_property, itip_add_tz, itip_build_envelope,\n        itip_export_component,\n    },\n    organizer::organizer_request_full,\n};\nuse ahash::AHashSet;\nuse calcard::{\n    common::PartialDateTime,\n    icalendar::{\n        ICalendar, ICalendarComponent, ICalendarComponentType, ICalendarMethod, ICalendarParameter,\n        ICalendarParticipationStatus, ICalendarProperty, ICalendarValue,\n    },\n};\n\npub(crate) fn attendee_handle_update(\n    new_ical: &ICalendar,\n    old_itip: ItipSnapshots<'_>,\n    new_itip: ItipSnapshots<'_>,\n) -> Result<Vec<ItipMessage<ICalendar>>, ItipError> {\n    let dt_stamp = PartialDateTime::now();\n    let mut message = ICalendar {\n        components: Vec::with_capacity(2),\n    };\n    message\n        .components\n        .push(itip_build_envelope(ICalendarMethod::Reply));\n\n    let mut mail_from = None;\n    let mut email_rcpt = AHashSet::new();\n    let mut new_delegates = AHashSet::new();\n    let mut part_stat = &ICalendarParticipationStatus::NeedsAction;\n\n    for (instance_id, instance) in &new_itip.components {\n        if let Some(old_instance) = old_itip.components.get(instance_id) {\n            match (instance.local_attendee(), old_instance.local_attendee()) {\n                (Some(local_attendee), Some(old_local_attendee))\n                    if local_attendee.email == old_local_attendee.email =>\n                {\n                    // Check added fields\n                    let mut send_update = false;\n                    for new_entry in instance.entries.difference(&old_instance.entries) {\n                        match (new_entry.name, &new_entry.value) {\n                            (ICalendarProperty::Exdate, ItipEntryValue::DateTime(date))\n                                if instance_id == &InstanceId::Main =>\n                            {\n                                if let Some((mut cancel_comp, attendee_email)) = attendee_decline(\n                                    instance_id,\n                                    &old_itip,\n                                    old_instance,\n                                    &dt_stamp,\n                                    &mut email_rcpt,\n                                    false,\n                                ) {\n                                    // Add EXDATE as RECURRENCE-ID\n                                    cancel_comp\n                                        .entries\n                                        .push(date.to_entry(ICalendarProperty::RecurrenceId));\n                                    part_stat = &ICalendarParticipationStatus::Declined;\n\n                                    // Add cancel component\n                                    let comp_id = message.components.len() as u32;\n                                    message.components[0].component_ids.push(comp_id);\n                                    message.components.push(cancel_comp);\n                                    mail_from = Some(&attendee_email.email);\n                                }\n                            }\n                            _ => {\n                                // Changing these properties is not allowed\n                                if !can_attendee_modify_property(\n                                    &instance.comp.component_type,\n                                    new_entry.name,\n                                ) {\n                                    return Err(ItipError::CannotModifyProperty(\n                                        new_entry.name.clone(),\n                                    ));\n                                } else {\n                                    send_update = send_update\n                                        || (instance.comp.component_type\n                                            == ICalendarComponentType::VTodo\n                                            && matches!(\n                                                new_entry.name,\n                                                ICalendarProperty::Status\n                                                    | ICalendarProperty::PercentComplete\n                                                    | ICalendarProperty::Completed\n                                            ));\n                                }\n                            }\n                        }\n                    }\n\n                    // Send participation status update\n                    if local_attendee.is_server_scheduling\n                        && ((local_attendee.part_stat != old_local_attendee.part_stat)\n                            || local_attendee.force_send.is_some()\n                            || send_update)\n                    {\n                        // Build the attendee list\n                        if let Some(new_partstat) = local_attendee.part_stat {\n                            part_stat = new_partstat;\n                        }\n                        let mut attendee_entry_uids = vec![local_attendee.entry_id];\n                        let old_delegates = old_instance\n                            .external_attendees()\n                            .filter(|a| a.is_delegated_from(old_local_attendee))\n                            .map(|a| a.email.email.as_str())\n                            .collect::<AHashSet<_>>();\n                        for external_attendee in instance.external_attendees() {\n                            if external_attendee.is_delegated_from(local_attendee) {\n                                if external_attendee.send_invite_messages()\n                                    && !old_delegates\n                                        .contains(&external_attendee.email.email.as_str())\n                                {\n                                    new_delegates.insert(external_attendee.email.email.as_str());\n                                }\n                            } else if external_attendee.is_delegated_to(local_attendee) {\n                                if external_attendee.send_update_messages() {\n                                    email_rcpt.insert(external_attendee.email.email.as_str());\n                                }\n                            } else {\n                                continue;\n                            }\n                            attendee_entry_uids.push(external_attendee.entry_id);\n                        }\n\n                        let comp_id = message.components.len() as u32;\n                        message.components[0].component_ids.push(comp_id);\n                        message.components.push(itip_export_component(\n                            instance.comp,\n                            new_itip.uid,\n                            &dt_stamp,\n                            instance.sequence.unwrap_or_default(),\n                            ItipExportAs::Attendee(attendee_entry_uids),\n                        ));\n                        mail_from = Some(&local_attendee.email.email);\n                    }\n\n                    // Check removed fields\n                    for removed_entry in old_instance.entries.difference(&instance.entries) {\n                        if !can_attendee_modify_property(\n                            &instance.comp.component_type,\n                            removed_entry.name,\n                        ) {\n                            // Removing these properties is not allowed\n                            return Err(ItipError::CannotModifyProperty(\n                                removed_entry.name.clone(),\n                            ));\n                        }\n                    }\n                }\n                _ => {\n                    // Change in local attendee email is not allowed\n                    return Err(ItipError::CannotModifyAddress);\n                }\n            }\n        } else if let Some(local_attendee) = instance\n            .local_attendee()\n            .filter(|_| instance_id != &InstanceId::Main)\n        {\n            let mut attendee_entry_uids = vec![local_attendee.entry_id];\n            for external_attendee in instance.external_attendees() {\n                if external_attendee.is_delegated_from(local_attendee) {\n                    if external_attendee.send_invite_messages() {\n                        new_delegates.insert(external_attendee.email.email.as_str());\n                    }\n                } else if external_attendee.is_delegated_to(local_attendee) {\n                    if external_attendee.send_update_messages() {\n                        email_rcpt.insert(external_attendee.email.email.as_str());\n                    }\n                } else {\n                    continue;\n                }\n                attendee_entry_uids.push(external_attendee.entry_id);\n            }\n\n            // A new instance has been added\n            let comp_id = message.components.len() as u32;\n            message.components[0].component_ids.push(comp_id);\n            message.components.push(itip_export_component(\n                instance.comp,\n                new_itip.uid,\n                &dt_stamp,\n                instance.sequence.unwrap_or_default(),\n                ItipExportAs::Attendee(attendee_entry_uids),\n            ));\n            mail_from = Some(&local_attendee.email.email);\n        } else {\n            return Err(ItipError::CannotModifyInstance);\n        }\n    }\n\n    for (instance_id, old_instance) in &old_itip.components {\n        if !new_itip.components.contains_key(instance_id) {\n            if instance_id != &InstanceId::Main && old_instance.has_local_attendee() {\n                // Send cancel message for removed instances\n                if let Some((cancel_comp, attendee_email)) = attendee_decline(\n                    instance_id,\n                    &old_itip,\n                    old_instance,\n                    &dt_stamp,\n                    &mut email_rcpt,\n                    false,\n                ) {\n                    // Add cancel component\n                    let comp_id = message.components.len() as u32;\n                    message.components[0].component_ids.push(comp_id);\n                    message.components.push(cancel_comp);\n                    mail_from = Some(&attendee_email.email);\n                }\n            } else {\n                // Removing instances is not allowed\n                return Err(ItipError::CannotModifyInstance);\n            }\n        }\n    }\n\n    if let Some(from) = mail_from {\n        email_rcpt.insert(&new_itip.organizer.email.email);\n\n        // Add timezones if needed\n        itip_add_tz(&mut message, new_ical);\n\n        let mut responses = vec![ItipMessage {\n            from: from.to_string(),\n            from_organizer: false,\n            to: email_rcpt.into_iter().map(|e| e.to_string()).collect(),\n            summary: ItipSummary::Rsvp {\n                part_stat: part_stat.clone(),\n                current: new_itip\n                    .main_instance_or_default()\n                    .build_summary(Some(&new_itip.organizer), &[]),\n            },\n            message,\n        }];\n\n        // Invite new delegates\n        if !new_delegates.is_empty() {\n            let from = from.to_string();\n            let new_delegates = new_delegates\n                .into_iter()\n                .map(|e| e.to_string())\n                .collect::<Vec<_>>();\n            if let Ok(messages_) = organizer_request_full(new_ical, &new_itip, None, true) {\n                for mut message in messages_ {\n                    message.from = from.clone();\n                    message.to = new_delegates.clone();\n                    message.from_organizer = false;\n                    responses.push(message);\n                }\n            }\n        }\n\n        Ok(responses)\n    } else {\n        Err(ItipError::NothingToSend)\n    }\n}\n\npub(crate) fn attendee_decline<'x>(\n    instance_id: &'x InstanceId,\n    itip: &'x ItipSnapshots<'x>,\n    comp: &'x ItipSnapshot<'x>,\n    dt_stamp: &'x PartialDateTime,\n    email_rcpt: &mut AHashSet<&'x str>,\n    skip_needs_action: bool,\n) -> Option<(ICalendarComponent, &'x Email)> {\n    let component = comp.comp;\n    let mut cancel_comp = ICalendarComponent {\n        component_type: component.component_type.clone(),\n        entries: Vec::with_capacity(5),\n        component_ids: vec![],\n    };\n\n    let mut local_attendee = None;\n    let mut delegated_from = None;\n\n    for attendee in &comp.attendees {\n        if attendee.email.is_local {\n            if attendee.is_server_scheduling\n                && attendee.rsvp.is_none_or(|rsvp| rsvp)\n                && match attendee.part_stat {\n                    Some(\n                        ICalendarParticipationStatus::Declined\n                        | ICalendarParticipationStatus::Delegated,\n                    ) => attendee.force_send.is_some(),\n                    Some(ICalendarParticipationStatus::NeedsAction) => !skip_needs_action,\n                    _ => true,\n                }\n            {\n                local_attendee = Some(attendee);\n            }\n        } else if attendee.delegated_to.iter().any(|d| d.is_local) {\n            cancel_comp\n                .entries\n                .push(component.entries[attendee.entry_id as usize].clone());\n            delegated_from = Some(&attendee.email.email);\n        }\n    }\n\n    local_attendee.map(|local_attendee| {\n        cancel_comp.add_property(\n            ICalendarProperty::Organizer,\n            ICalendarValue::Text(itip.organizer.email.to_string()),\n        );\n        cancel_comp.add_property_with_params(\n            ICalendarProperty::Attendee,\n            [ICalendarParameter::partstat(\n                ICalendarParticipationStatus::Declined,\n            )],\n            ICalendarValue::Text(local_attendee.email.to_string()),\n        );\n        cancel_comp.add_uid(itip.uid);\n        cancel_comp.add_dtstamp(dt_stamp.clone());\n        cancel_comp.add_sequence(comp.sequence.unwrap_or_default());\n        cancel_comp.entries.extend(\n            component\n                .entries\n                .iter()\n                .filter(|e| {\n                    matches!(\n                        e.name,\n                        ICalendarProperty::Dtstart\n                            | ICalendarProperty::Dtend\n                            | ICalendarProperty::Duration\n                            | ICalendarProperty::Due\n                            | ICalendarProperty::Description\n                            | ICalendarProperty::Summary\n                    )\n                })\n                .cloned(),\n        );\n\n        if let InstanceId::Recurrence(recurrence_id) = instance_id {\n            cancel_comp\n                .entries\n                .push(component.entries[recurrence_id.entry_id as usize].clone());\n        }\n        if let Some(delegated_from) = delegated_from {\n            email_rcpt.insert(delegated_from);\n        }\n\n        (cancel_comp, &local_attendee.email)\n    })\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/event_cancel.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{\n    InstanceId, ItipError, ItipMessage, ItipSummary,\n    attendee::attendee_decline,\n    itip::{itip_add_tz, itip_build_envelope},\n    snapshot::itip_snapshot,\n};\nuse ahash::AHashSet;\nuse calcard::{\n    common::PartialDateTime,\n    icalendar::{\n        ICalendar, ICalendarComponent, ICalendarComponentType, ICalendarMethod,\n        ICalendarParticipationStatus, ICalendarProperty, ICalendarStatus, ICalendarValue,\n    },\n};\n\npub fn itip_cancel(\n    ical: &ICalendar,\n    account_emails: &[String],\n    is_deletion: bool,\n) -> Result<ItipMessage<ICalendar>, ItipError> {\n    // Prepare iTIP message\n    let itip = itip_snapshot(ical, account_emails, false)?;\n    let dt_stamp = PartialDateTime::now();\n    let mut message = ICalendar {\n        components: Vec::with_capacity(2),\n    };\n\n    if itip.organizer.email.is_local {\n        // Send cancel message\n        let mut comp = itip_build_envelope(ICalendarMethod::Cancel);\n        comp.component_ids.push(1);\n        message.components.push(comp);\n\n        // Fetch guest emails\n        let mut recipients = AHashSet::new();\n        let mut cancel_guests = AHashSet::new();\n        let mut component_type = &ICalendarComponentType::VEvent;\n        let mut sequence = 0;\n        for (instance_id, comp) in &itip.components {\n            component_type = &comp.comp.component_type;\n            for attendee in &comp.attendees {\n                if attendee.send_update_messages() {\n                    recipients.insert(attendee.email.email.clone());\n                }\n                cancel_guests.insert(&attendee.email);\n            }\n\n            // Increment sequence if needed\n            if instance_id == &InstanceId::Main {\n                sequence = comp.sequence.unwrap_or_default() + 1;\n            }\n        }\n\n        if !recipients.is_empty() && component_type != &ICalendarComponentType::VFreebusy {\n            let instance = itip.main_instance_or_default();\n            message.components.push(build_cancel_component(\n                instance.comp,\n                sequence,\n                dt_stamp,\n                &[],\n            ));\n\n            // Add timezones\n            itip_add_tz(&mut message, ical);\n\n            Ok(ItipMessage {\n                to: recipients.into_iter().collect(),\n                summary: ItipSummary::Cancel(instance.build_summary(None, &[])),\n                from: itip.organizer.email.email,\n                from_organizer: true,\n                message,\n            })\n        } else {\n            Err(ItipError::NothingToSend)\n        }\n    } else {\n        // Send decline message\n        message\n            .components\n            .push(itip_build_envelope(ICalendarMethod::Reply));\n\n        // Decline attendance for all instances that have local attendees\n        let mut mail_from = None;\n        let mut email_rcpt = AHashSet::new();\n        for (instance_id, comp) in &itip.components {\n            if let Some((cancel_comp, attendee_email)) = attendee_decline(\n                instance_id,\n                &itip,\n                comp,\n                &dt_stamp,\n                &mut email_rcpt,\n                is_deletion,\n            ) {\n                // Add cancel component\n                let comp_id = message.components.len() as u32;\n                message.components[0].component_ids.push(comp_id);\n                message.components.push(cancel_comp);\n                mail_from = Some(&attendee_email.email);\n            }\n        }\n\n        if let Some(from) = mail_from {\n            // Add timezone information if needed\n            itip_add_tz(&mut message, ical);\n\n            email_rcpt.insert(&itip.organizer.email.email);\n\n            Ok(ItipMessage {\n                from: from.to_string(),\n                from_organizer: false,\n                to: email_rcpt.into_iter().map(|e| e.to_string()).collect(),\n                summary: ItipSummary::Rsvp {\n                    part_stat: ICalendarParticipationStatus::Declined,\n                    current: itip.main_instance_or_default().build_summary(None, &[]),\n                },\n                message,\n            })\n        } else {\n            Err(ItipError::NothingToSend)\n        }\n    }\n}\n\npub(crate) fn build_cancel_component(\n    component: &ICalendarComponent,\n    sequence: i64,\n    dt_stamp: PartialDateTime,\n    attendees: &[&str],\n) -> ICalendarComponent {\n    let mut cancel_comp = ICalendarComponent {\n        component_type: component.component_type.clone(),\n        entries: Vec::with_capacity(7),\n        component_ids: vec![],\n    };\n    cancel_comp.add_property(\n        ICalendarProperty::Status,\n        ICalendarValue::Status(ICalendarStatus::Cancelled),\n    );\n    cancel_comp.add_dtstamp(dt_stamp);\n    cancel_comp.add_sequence(sequence);\n    cancel_comp.entries.extend(\n        component\n            .entries\n            .iter()\n            .filter(|e| match e.name {\n                ICalendarProperty::Organizer\n                | ICalendarProperty::Uid\n                | ICalendarProperty::Summary\n                | ICalendarProperty::Dtstart\n                | ICalendarProperty::Dtend\n                | ICalendarProperty::Duration\n                | ICalendarProperty::Due\n                | ICalendarProperty::RecurrenceId\n                | ICalendarProperty::Created\n                | ICalendarProperty::LastModified\n                | ICalendarProperty::Description\n                | ICalendarProperty::Location => true,\n                ICalendarProperty::Attendee => {\n                    attendees.is_empty()\n                        || e.values\n                            .first()\n                            .and_then(|v| v.as_text())\n                            .is_some_and(|email| {\n                                attendees.iter().any(|attendee| {\n                                    email\n                                        .strip_suffix(attendee)\n                                        .is_some_and(|v| v.ends_with(':') || v.is_empty())\n                                })\n                            })\n                }\n                _ => false,\n            })\n            .cloned(),\n    );\n\n    cancel_comp\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/event_create.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{\n    ItipError, ItipMessage, itip::itip_finalize, organizer::organizer_request_full,\n    snapshot::itip_snapshot,\n};\nuse calcard::icalendar::ICalendar;\n\npub fn itip_create(\n    ical: &mut ICalendar,\n    account_emails: &[String],\n) -> Result<Vec<ItipMessage<ICalendar>>, ItipError> {\n    let itip = itip_snapshot(ical, account_emails, false)?;\n    if !itip.organizer.is_server_scheduling {\n        Err(ItipError::OtherSchedulingAgent)\n    } else if !itip.organizer.email.is_local {\n        Err(ItipError::NotOrganizer)\n    } else {\n        let mut sequences = Vec::new();\n        organizer_request_full(ical, &itip, Some(&mut sequences), true).inspect(|_| {\n            itip_finalize(ical, &sequences);\n        })\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/event_update.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{\n    ItipError, ItipMessage, attendee::attendee_handle_update, event_cancel::itip_cancel,\n    itip::itip_finalize, organizer::organizer_handle_update, snapshot::itip_snapshot,\n};\nuse calcard::icalendar::ICalendar;\n\npub fn itip_update(\n    ical: &mut ICalendar,\n    old_ical: &ICalendar,\n    account_emails: &[String],\n) -> Result<Vec<ItipMessage<ICalendar>>, ItipError> {\n    let old_itip = itip_snapshot(old_ical, account_emails, false)?;\n    match itip_snapshot(ical, account_emails, false) {\n        Ok(new_itip) => {\n            let mut sequences = Vec::new();\n            if old_itip.organizer.email != new_itip.organizer.email {\n                // RFC 6638 does not support replacing the organizer\n                Err(ItipError::OrganizerMismatch)\n            } else if old_itip.organizer.email.is_local {\n                organizer_handle_update(old_ical, ical, old_itip, new_itip, &mut sequences)\n            } else {\n                attendee_handle_update(ical, old_itip, new_itip)\n            }\n            .inspect(|_| {\n                itip_finalize(ical, &sequences);\n            })\n        }\n        Err(err) => {\n            match &err {\n                ItipError::NoSchedulingInfo\n                | ItipError::NotOrganizer\n                | ItipError::NotOrganizerNorAttendee\n                | ItipError::OtherSchedulingAgent => {\n                    if old_itip.organizer.email.is_local {\n                        // RFC 6638 does not support replacing the organizer, so we cancel the event\n                        itip_cancel(old_ical, account_emails, false).map(|message| vec![message])\n                    } else {\n                        Err(ItipError::CannotModifyAddress)\n                    }\n                }\n                _ => Err(err),\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/inbound.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{\n    InstanceId, ItipError, ItipMessage, ItipSnapshots, organizer::organizer_request_full,\n};\nuse ahash::AHashSet;\nuse calcard::icalendar::{\n    ICalendar, ICalendarComponent, ICalendarComponentType, ICalendarEntry, ICalendarMethod,\n    ICalendarParameter, ICalendarParameterName, ICalendarProperty, ICalendarStatus, ICalendarValue,\n    Uri,\n};\n\n#[derive(Debug)]\npub enum MergeAction {\n    AddEntries {\n        component_id: u16,\n        entries: Vec<ICalendarEntry>,\n    },\n    RemoveEntries {\n        component_id: u16,\n        entries: AHashSet<ICalendarProperty>,\n    },\n    AddParameters {\n        component_id: u16,\n        entry_id: u16,\n        parameters: Vec<ICalendarParameter>,\n    },\n    RemoveParameters {\n        component_id: u16,\n        entry_id: u16,\n        parameters: Vec<ICalendarParameterName>,\n    },\n    AddComponent {\n        component: ICalendarComponent,\n    },\n    RemoveComponent {\n        component_id: u16,\n    },\n}\n\npub enum MergeResult {\n    Actions(Vec<MergeAction>),\n    Message(ItipMessage<ICalendar>),\n    None,\n}\n\npub fn itip_process_message(\n    ical: &ICalendar,\n    snapshots: ItipSnapshots<'_>,\n    itip: &ICalendar,\n    itip_snapshots: ItipSnapshots<'_>,\n    sender: String,\n) -> Result<MergeResult, ItipError> {\n    if snapshots.organizer.email != itip_snapshots.organizer.email {\n        return Err(ItipError::OrganizerMismatch);\n    }\n\n    let method = itip_method(itip)?;\n    let mut merge_actions = Vec::new();\n\n    if snapshots.organizer.email.is_local {\n        // Handle attendee updates\n        if snapshots.organizer.email.email == sender {\n            return Err(ItipError::OrganizerIsLocalAddress);\n        }\n        match method {\n            ICalendarMethod::Reply => {\n                handle_reply(&snapshots, &itip_snapshots, &sender, &mut merge_actions)?;\n            }\n            ICalendarMethod::Refresh => {\n                return organizer_request_full(ical, &snapshots, None, false).and_then(\n                    |messages| {\n                        messages\n                            .into_iter()\n                            .next()\n                            .map(|mut message| {\n                                message.to = vec![sender];\n                                MergeResult::Message(message)\n                            })\n                            .ok_or(ItipError::NothingToSend)\n                    },\n                );\n            }\n            _ => return Err(ItipError::UnsupportedMethod(method.clone())),\n        }\n    } else {\n        // Handle organizer and attendees updates\n        match method {\n            ICalendarMethod::Request => {\n                let mut is_full_update = false;\n                for (instance_id, itip_snapshot) in &itip_snapshots.components {\n                    is_full_update = is_full_update || instance_id == &InstanceId::Main;\n                    let itip_component = &itip.components[itip_snapshot.comp_id as usize];\n\n                    if let Some(snapshot) = snapshots.components.get(instance_id) {\n                        // Merge instances\n                        if itip_snapshot.sequence.unwrap_or_default()\n                            >= snapshot.sequence.unwrap_or_default()\n                        {\n                            let mut changed_entries = itip_snapshot\n                                .entries\n                                .symmetric_difference(&snapshot.entries)\n                                .map(|entry| entry.name.clone())\n                                .collect::<AHashSet<_>>();\n                            if itip_snapshot.attendees != snapshot.attendees {\n                                changed_entries.insert(ICalendarProperty::Attendee);\n                            }\n                            if itip_snapshot.dtstamp.is_some()\n                                && itip_snapshot.dtstamp != snapshot.dtstamp\n                            {\n                                changed_entries.insert(ICalendarProperty::Dtstamp);\n                            }\n                            changed_entries.insert(ICalendarProperty::Sequence);\n\n                            if !changed_entries.is_empty() {\n                                let entries = itip_component\n                                    .entries\n                                    .iter()\n                                    .filter(|entry| changed_entries.contains(&entry.name))\n                                    .cloned()\n                                    .collect();\n                                merge_actions.push(MergeAction::RemoveEntries {\n                                    component_id: snapshot.comp_id,\n                                    entries: changed_entries,\n                                });\n                                merge_actions.push(MergeAction::AddEntries {\n                                    component_id: snapshot.comp_id,\n                                    entries,\n                                });\n                            }\n                        } else {\n                            return Err(ItipError::OutOfSequence);\n                        }\n                    } else {\n                        // Add instance\n                        merge_actions.push(MergeAction::AddComponent {\n                            component: ICalendarComponent {\n                                component_type: itip_component.component_type.clone(),\n                                entries: itip_component\n                                    .entries\n                                    .iter()\n                                    .filter(|entry| {\n                                        !matches!(entry.name, ICalendarProperty::Other(_))\n                                    })\n                                    .cloned()\n                                    .collect(),\n                                component_ids: vec![],\n                            },\n                        });\n                    }\n                }\n\n                if is_full_update {\n                    for (instance_id, snapshot) in &snapshots.components {\n                        if !itip_snapshots.components.contains_key(instance_id) {\n                            // Remove instance\n                            merge_actions.push(MergeAction::RemoveComponent {\n                                component_id: snapshot.comp_id,\n                            });\n                        }\n                    }\n                }\n            }\n            ICalendarMethod::Add => {\n                for (instance_id, itip_snapshot) in &itip_snapshots.components {\n                    if !snapshots.components.contains_key(instance_id) {\n                        let itip_component = &itip.components[itip_snapshot.comp_id as usize];\n                        merge_actions.push(MergeAction::AddComponent {\n                            component: ICalendarComponent {\n                                component_type: itip_component.component_type.clone(),\n                                entries: itip_component\n                                    .entries\n                                    .iter()\n                                    .filter(|entry| {\n                                        !matches!(entry.name, ICalendarProperty::Other(_))\n                                    })\n                                    .cloned()\n                                    .collect(),\n                                component_ids: vec![],\n                            },\n                        });\n                    }\n                }\n            }\n            ICalendarMethod::Cancel => {\n                let mut cancel_all_instances = false;\n                for (instance_id, itip_snapshot) in &itip_snapshots.components {\n                    if let Some(snapshot) = snapshots.components.get(instance_id) {\n                        if itip_snapshot.sequence.unwrap_or_default()\n                            >= snapshot.sequence.unwrap_or_default()\n                        {\n                            // Cancel instance\n                            let itip_component = itip_snapshot.comp;\n                            merge_actions.push(MergeAction::RemoveEntries {\n                                component_id: snapshot.comp_id,\n                                entries: [\n                                    ICalendarProperty::Organizer,\n                                    ICalendarProperty::Attendee,\n                                    ICalendarProperty::Status,\n                                    ICalendarProperty::Sequence,\n                                ]\n                                .into_iter()\n                                .collect(),\n                            });\n                            merge_actions.push(MergeAction::AddEntries {\n                                component_id: snapshot.comp_id,\n                                entries: itip_component\n                                    .entries\n                                    .iter()\n                                    .filter(|entry| {\n                                        matches!(\n                                            entry.name,\n                                            ICalendarProperty::Organizer\n                                                | ICalendarProperty::Attendee\n                                        )\n                                    })\n                                    .cloned()\n                                    .chain([ICalendarEntry {\n                                        name: ICalendarProperty::Status,\n                                        params: vec![],\n                                        values: vec![ICalendarValue::Status(\n                                            ICalendarStatus::Cancelled,\n                                        )],\n                                    }])\n                                    .collect(),\n                            });\n                            cancel_all_instances =\n                                cancel_all_instances || instance_id == &InstanceId::Main;\n                        } else {\n                            return Err(ItipError::OutOfSequence);\n                        }\n                    } else {\n                        let itip_component = itip_snapshot.comp;\n                        merge_actions.push(MergeAction::AddComponent {\n                            component: ICalendarComponent {\n                                component_type: itip_component.component_type.clone(),\n                                entries: itip_component\n                                    .entries\n                                    .iter()\n                                    .filter(|entry| {\n                                        !matches!(\n                                            entry.name,\n                                            ICalendarProperty::Status | ICalendarProperty::Other(_)\n                                        )\n                                    })\n                                    .cloned()\n                                    .chain([ICalendarEntry {\n                                        name: ICalendarProperty::Status,\n                                        params: vec![],\n                                        values: vec![ICalendarValue::Status(\n                                            ICalendarStatus::Cancelled,\n                                        )],\n                                    }])\n                                    .collect(),\n                                component_ids: vec![],\n                            },\n                        });\n                    }\n                }\n\n                if cancel_all_instances {\n                    // Remove all instances\n                    let itip_main = itip_snapshots.components.get(&InstanceId::Main).unwrap();\n                    let itip_component = itip_main.comp;\n                    for (instance_id, snapshot) in &snapshots.components {\n                        if !itip_snapshots.components.contains_key(instance_id) {\n                            merge_actions.push(MergeAction::RemoveEntries {\n                                component_id: snapshot.comp_id,\n                                entries: [\n                                    ICalendarProperty::Organizer,\n                                    ICalendarProperty::Attendee,\n                                    ICalendarProperty::Status,\n                                ]\n                                .into_iter()\n                                .collect(),\n                            });\n                            merge_actions.push(MergeAction::AddEntries {\n                                component_id: snapshot.comp_id,\n                                entries: itip_component\n                                    .entries\n                                    .iter()\n                                    .filter(|entry| {\n                                        matches!(\n                                            entry.name,\n                                            ICalendarProperty::Organizer\n                                                | ICalendarProperty::Attendee\n                                        )\n                                    })\n                                    .cloned()\n                                    .chain([ICalendarEntry {\n                                        name: ICalendarProperty::Status,\n                                        params: vec![],\n                                        values: vec![ICalendarValue::Status(\n                                            ICalendarStatus::Cancelled,\n                                        )],\n                                    }])\n                                    .collect(),\n                            });\n                        }\n                    }\n                }\n            }\n            ICalendarMethod::Reply\n                if itip_snapshots.components.values().any(|snapshot| {\n                    snapshot.external_attendees().any(|a| {\n                        a.email.email == sender && a.delegated_from.iter().any(|a| a.is_local)\n                    })\n                }) =>\n            {\n                handle_reply(&snapshots, &itip_snapshots, &sender, &mut merge_actions)?;\n            }\n            _ => return Err(ItipError::UnsupportedMethod(method.clone())),\n        }\n    }\n\n    if !merge_actions.is_empty() {\n        Ok(MergeResult::Actions(merge_actions))\n    } else {\n        Ok(MergeResult::None)\n    }\n}\n\npub fn itip_import_message(ical: &mut ICalendar) -> Result<(), ItipError> {\n    let mut expect_object_type = None;\n    for comp in ical.components.iter_mut() {\n        if comp.component_type.is_scheduling_object() {\n            match expect_object_type {\n                Some(expected) if expected != &comp.component_type => {\n                    return Err(ItipError::MultipleObjectTypes);\n                }\n                None => {\n                    expect_object_type = Some(&comp.component_type);\n                }\n                _ => {}\n            }\n        } else if comp.component_type == ICalendarComponentType::VCalendar {\n            comp.entries\n                .retain(|entry| !matches!(entry.name, ICalendarProperty::Method));\n        }\n    }\n\n    Ok(())\n}\n\nfn handle_reply(\n    snapshots: &ItipSnapshots<'_>,\n    itip_snapshots: &ItipSnapshots<'_>,\n    sender: &str,\n    merge_actions: &mut Vec<MergeAction>,\n) -> Result<(), ItipError> {\n    for (instance_id, itip_snapshot) in &itip_snapshots.components {\n        if let Some(snapshot) = snapshots.components.get(instance_id) {\n            if let (Some(attendee), Some(updated_attendee)) = (\n                snapshot.attendee_by_email(sender),\n                itip_snapshot.attendee_by_email(sender),\n            ) {\n                let itip_component = itip_snapshot.comp;\n                let changed_part_stat = attendee.part_stat != updated_attendee.part_stat;\n                let changed_rsvp = attendee.rsvp != updated_attendee.rsvp;\n                let changed_delegated_to = attendee.delegated_to != updated_attendee.delegated_to;\n                let has_request_status = !itip_snapshot.request_status.is_empty();\n\n                if changed_part_stat || changed_rsvp || changed_delegated_to || has_request_status {\n                    // Update participant status\n                    let mut add_parameters = Vec::new();\n                    let mut remove_parameters = Vec::new();\n                    if changed_part_stat {\n                        remove_parameters.push(ICalendarParameterName::Partstat);\n                        if let Some(part_stat) = updated_attendee.part_stat {\n                            add_parameters.push(ICalendarParameter::partstat(part_stat.clone()));\n                        }\n                    }\n\n                    if changed_rsvp {\n                        remove_parameters.push(ICalendarParameterName::Rsvp);\n                        if let Some(rsvp) = updated_attendee.rsvp {\n                            add_parameters.push(ICalendarParameter::rsvp(rsvp));\n                        }\n                    }\n\n                    if changed_delegated_to {\n                        remove_parameters.push(ICalendarParameterName::DelegatedTo);\n                        if !updated_attendee.delegated_to.is_empty() {\n                            add_parameters.extend(updated_attendee.delegated_to.iter().map(\n                                |email| {\n                                    ICalendarParameter::delegated_to(Uri::Location(\n                                        email.to_string(),\n                                    ))\n                                },\n                            ));\n                        }\n                    }\n\n                    if has_request_status {\n                        remove_parameters.push(ICalendarParameterName::ScheduleStatus);\n                        add_parameters.push(ICalendarParameter::schedule_status(\n                            itip_snapshot.request_status.join(\",\"),\n                        ));\n                    }\n\n                    merge_actions.push(MergeAction::RemoveParameters {\n                        component_id: snapshot.comp_id,\n                        entry_id: attendee.entry_id,\n                        parameters: remove_parameters,\n                    });\n                    merge_actions.push(MergeAction::AddParameters {\n                        component_id: snapshot.comp_id,\n                        entry_id: attendee.entry_id,\n                        parameters: add_parameters,\n                    });\n\n                    // Add unknown delegated attendees\n                    for delegated_to in &updated_attendee.delegated_to {\n                        if let Some(itip_delegated) =\n                            itip_snapshot.attendee_by_email(&delegated_to.email)\n                        {\n                            if let Some(delegated) = snapshot.attendee_by_email(&delegated_to.email)\n                            {\n                                if delegated != itip_delegated {\n                                    merge_actions.push(MergeAction::RemoveParameters {\n                                        component_id: snapshot.comp_id,\n                                        entry_id: delegated.entry_id,\n                                        parameters: vec![\n                                            ICalendarParameterName::DelegatedTo,\n                                            ICalendarParameterName::DelegatedFrom,\n                                            ICalendarParameterName::Partstat,\n                                            ICalendarParameterName::Rsvp,\n                                            ICalendarParameterName::ScheduleStatus,\n                                            ICalendarParameterName::Role,\n                                        ],\n                                    });\n                                    merge_actions.push(MergeAction::AddParameters {\n                                        component_id: snapshot.comp_id,\n                                        entry_id: delegated.entry_id,\n                                        parameters: itip_component.entries\n                                            [itip_delegated.entry_id as usize]\n                                            .params\n                                            .iter()\n                                            .filter(|param| {\n                                                matches!(\n                                                    param.name,\n                                                    ICalendarParameterName::DelegatedTo\n                                                        | ICalendarParameterName::DelegatedFrom\n                                                        | ICalendarParameterName::Partstat\n                                                        | ICalendarParameterName::Rsvp\n                                                        | ICalendarParameterName::ScheduleStatus\n                                                        | ICalendarParameterName::Role\n                                                )\n                                            })\n                                            .cloned()\n                                            .collect(),\n                                    });\n                                }\n                            } else {\n                                merge_actions.push(MergeAction::AddEntries {\n                                    component_id: snapshot.comp_id,\n                                    entries: vec![\n                                        itip_component.entries[itip_delegated.entry_id as usize]\n                                            .clone(),\n                                    ],\n                                });\n                            }\n                        }\n                    }\n                }\n\n                // Add changed properties for VTODO\n                if snapshot.comp.component_type == ICalendarComponentType::VTodo {\n                    let mut remove_entries = AHashSet::new();\n                    let mut add_entries = Vec::new();\n\n                    for entry in itip_component.entries.iter() {\n                        if matches!(\n                            entry.name,\n                            ICalendarProperty::PercentComplete\n                                | ICalendarProperty::Status\n                                | ICalendarProperty::Completed\n                        ) {\n                            remove_entries.insert(entry.name.clone());\n                            add_entries.push(entry.clone());\n                        }\n                    }\n\n                    if !add_entries.is_empty() {\n                        merge_actions.push(MergeAction::RemoveEntries {\n                            component_id: snapshot.comp_id,\n                            entries: remove_entries,\n                        });\n                        merge_actions.push(MergeAction::AddEntries {\n                            component_id: snapshot.comp_id,\n                            entries: add_entries,\n                        });\n                    }\n                }\n            } else {\n                return Err(ItipError::SenderIsNotParticipant(sender.to_string()));\n            }\n        } else if itip_snapshot.attendee_by_email(sender).is_some() {\n            // Add component\n            let itip_component = itip_snapshot.comp;\n            let is_todo = itip_component.component_type == ICalendarComponentType::VTodo;\n            merge_actions.push(MergeAction::AddComponent {\n                component: ICalendarComponent {\n                    component_type: itip_component.component_type.clone(),\n                    entries: itip_component\n                        .entries\n                        .iter()\n                        .filter(|entry| {\n                            matches!(\n                                entry.name,\n                                ICalendarProperty::Organizer\n                                    | ICalendarProperty::Attendee\n                                    | ICalendarProperty::Uid\n                                    | ICalendarProperty::Dtstamp\n                                    | ICalendarProperty::Sequence\n                                    | ICalendarProperty::RecurrenceId\n                            ) || (is_todo\n                                && matches!(\n                                    entry.name,\n                                    ICalendarProperty::PercentComplete\n                                        | ICalendarProperty::Status\n                                        | ICalendarProperty::Completed\n                                ))\n                        })\n                        .cloned()\n                        .collect(),\n                    component_ids: vec![],\n                },\n            });\n        } else {\n            return Err(ItipError::SenderIsNotParticipant(sender.to_string()));\n        }\n    }\n\n    Ok(())\n}\n\npub fn itip_merge_changes(ical: &mut ICalendar, changes: Vec<MergeAction>) {\n    let mut remove_component_ids: Vec<u32> = Vec::new();\n    for action in changes {\n        match action {\n            MergeAction::AddEntries {\n                component_id,\n                entries,\n            } => {\n                let component = &mut ical.components[component_id as usize];\n                component.entries.extend(entries);\n            }\n            MergeAction::RemoveEntries {\n                component_id,\n                entries,\n            } => {\n                let component = &mut ical.components[component_id as usize];\n                component\n                    .entries\n                    .retain(|entry| !entries.contains(&entry.name));\n            }\n            MergeAction::AddParameters {\n                component_id,\n                entry_id,\n                parameters,\n            } => {\n                ical.components[component_id as usize].entries[entry_id as usize]\n                    .params\n                    .extend(parameters);\n            }\n            MergeAction::RemoveParameters {\n                component_id,\n                entry_id,\n                parameters,\n            } => {\n                ical.components[component_id as usize].entries[entry_id as usize]\n                    .params\n                    .retain(|param| !parameters.contains(&param.name));\n            }\n            MergeAction::AddComponent { component } => {\n                let comp_id = ical.components.len() as u32;\n                if let Some(root) = ical\n                    .components\n                    .get_mut(0)\n                    .filter(|c| c.component_type == ICalendarComponentType::VCalendar)\n                {\n                    root.component_ids.push(comp_id);\n                    ical.components.push(component);\n                }\n            }\n            MergeAction::RemoveComponent { component_id } => {\n                remove_component_ids.push(component_id as u32);\n            }\n        }\n    }\n\n    if !remove_component_ids.is_empty() {\n        ical.remove_component_ids(&remove_component_ids);\n    }\n}\n\npub fn itip_method(ical: &ICalendar) -> Result<&ICalendarMethod, ItipError> {\n    ical.components\n        .first()\n        .and_then(|comp| {\n            comp.entries.iter().find_map(|entry| {\n                if entry.name == ICalendarProperty::Method {\n                    entry.values.first().and_then(|value| {\n                        if let ICalendarValue::Method(method) = value {\n                            Some(method)\n                        } else {\n                            None\n                        }\n                    })\n                } else {\n                    None\n                }\n            })\n        })\n        .ok_or(ItipError::MissingMethod)\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/itip.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{ArchivedItipSummary, ItipMessage, ItipMessages};\nuse calcard::{\n    common::{IanaString, PartialDateTime},\n    icalendar::{\n        ICalendar, ICalendarComponent, ICalendarComponentType, ICalendarEntry, ICalendarMethod,\n        ICalendarParameter, ICalendarParameterName, ICalendarParameterValue,\n        ICalendarParticipationStatus, ICalendarProperty, ICalendarValue,\n    },\n};\nuse common::PROD_ID;\nuse store::{\n    Serialize,\n    write::{Archiver, BatchBuilder, TaskEpoch, TaskQueueClass, ValueClass},\n};\nuse trc::AddContext;\n\npub(crate) fn itip_build_envelope(method: ICalendarMethod) -> ICalendarComponent {\n    ICalendarComponent {\n        component_type: ICalendarComponentType::VCalendar,\n        entries: vec![\n            ICalendarEntry {\n                name: ICalendarProperty::Version,\n                params: vec![],\n                values: vec![ICalendarValue::Text(\"2.0\".to_string())],\n            },\n            ICalendarEntry {\n                name: ICalendarProperty::Prodid,\n                params: vec![],\n                values: vec![ICalendarValue::Text(PROD_ID.to_string())],\n            },\n            ICalendarEntry {\n                name: ICalendarProperty::Method,\n                params: vec![],\n                values: vec![ICalendarValue::Method(method)],\n            },\n        ],\n        component_ids: Default::default(),\n    }\n}\n\npub(crate) enum ItipExportAs<'x> {\n    Organizer(&'x ICalendarParticipationStatus),\n    Attendee(Vec<u16>),\n}\n\npub(crate) fn itip_export_component(\n    component: &ICalendarComponent,\n    uid: &str,\n    dt_stamp: &PartialDateTime,\n    sequence: i64,\n    export_as: ItipExportAs<'_>,\n) -> ICalendarComponent {\n    let is_todo = component.component_type == ICalendarComponentType::VTodo;\n    let mut comp = ICalendarComponent {\n        component_type: component.component_type.clone(),\n        entries: Vec::with_capacity(component.entries.len() + 1),\n        component_ids: Default::default(),\n    };\n\n    comp.add_dtstamp(dt_stamp.clone());\n    comp.add_sequence(sequence);\n    comp.add_uid(uid);\n\n    for (entry_id, entry) in component.entries.iter().enumerate() {\n        match (&entry.name, &export_as) {\n            (\n                ICalendarProperty::Organizer | ICalendarProperty::Attendee,\n                ItipExportAs::Organizer(partstat),\n            ) => {\n                let mut new_entry = ICalendarEntry {\n                    name: entry.name.clone(),\n                    params: Vec::with_capacity(entry.params.len()),\n                    values: entry.values.clone(),\n                };\n                let mut has_partstat = false;\n                let mut rsvp = true;\n\n                for entry in &entry.params {\n                    match &entry.name {\n                        ICalendarParameterName::ScheduleStatus\n                        | ICalendarParameterName::ScheduleAgent\n                        | ICalendarParameterName::ScheduleForceSend => {}\n                        _ => {\n                            match &entry.name {\n                                ICalendarParameterName::Rsvp => {\n                                    rsvp = !matches!(\n                                        entry.value,\n                                        ICalendarParameterValue::Bool(false)\n                                    );\n                                }\n                                ICalendarParameterName::Partstat => {\n                                    has_partstat = true;\n                                }\n                                _ => {}\n                            }\n\n                            new_entry.params.push(entry.clone())\n                        }\n                    }\n                }\n\n                if !has_partstat && rsvp && entry.name == ICalendarProperty::Attendee {\n                    new_entry\n                        .params\n                        .push(ICalendarParameter::partstat((*partstat).clone()));\n                }\n\n                comp.entries.push(new_entry);\n            }\n            (\n                ICalendarProperty::Organizer | ICalendarProperty::Attendee,\n                ItipExportAs::Attendee(attendee_entry_ids),\n            ) => {\n                if attendee_entry_ids.contains(&(entry_id as u16))\n                    || entry.name == ICalendarProperty::Organizer\n                {\n                    comp.entries.push(ICalendarEntry {\n                        name: entry.name.clone(),\n                        params: entry\n                            .params\n                            .iter()\n                            .filter(|param| {\n                                !matches!(\n                                    &param.name,\n                                    ICalendarParameterName::ScheduleStatus\n                                        | ICalendarParameterName::ScheduleAgent\n                                        | ICalendarParameterName::ScheduleForceSend\n                                )\n                            })\n                            .cloned()\n                            .collect(),\n                        values: entry.values.clone(),\n                    });\n                }\n            }\n            (\n                ICalendarProperty::RequestStatus\n                | ICalendarProperty::Dtstamp\n                | ICalendarProperty::Sequence\n                | ICalendarProperty::Uid,\n                _,\n            ) => {}\n            (_, ItipExportAs::Organizer(_))\n            | (\n                ICalendarProperty::RecurrenceId\n                | ICalendarProperty::Dtstart\n                | ICalendarProperty::Dtend\n                | ICalendarProperty::Duration\n                | ICalendarProperty::Due\n                | ICalendarProperty::Description\n                | ICalendarProperty::Summary,\n                _,\n            ) => {\n                comp.entries.push(entry.clone());\n            }\n            (\n                ICalendarProperty::Status\n                | ICalendarProperty::PercentComplete\n                | ICalendarProperty::Completed,\n                _,\n            ) if is_todo => {\n                comp.entries.push(entry.clone());\n            }\n            _ => {}\n        }\n    }\n\n    if matches!(export_as, ItipExportAs::Attendee(_)) {\n        comp.entries.push(ICalendarEntry {\n            name: ICalendarProperty::RequestStatus,\n            params: vec![],\n            values: vec![\n                ICalendarValue::Text(\"2.0\".to_string()),\n                ICalendarValue::Text(\"Success\".to_string()),\n            ],\n        });\n    }\n\n    comp\n}\n\npub(crate) fn itip_finalize(ical: &mut ICalendar, scheduling_object_ids: &[u16]) {\n    for comp in ical.components.iter_mut() {\n        if comp.component_type.is_scheduling_object() {\n            // Remove scheduling info from non-updated components\n            for entry in comp.entries.iter_mut() {\n                if matches!(\n                    entry.name,\n                    ICalendarProperty::Organizer | ICalendarProperty::Attendee\n                ) {\n                    entry.params.retain(|param| {\n                        !matches!(param.name, ICalendarParameterName::ScheduleForceSend)\n                    });\n                }\n            }\n        }\n    }\n\n    for comp_id in scheduling_object_ids {\n        let comp = &mut ical.components[*comp_id as usize];\n        let mut found_sequence = false;\n        for entry in &mut comp.entries {\n            if entry.name == ICalendarProperty::Sequence {\n                if let Some(ICalendarValue::Integer(seq)) = entry.values.first_mut() {\n                    *seq += 1;\n                } else {\n                    entry.values = vec![ICalendarValue::Integer(1)];\n                }\n                found_sequence = true;\n                break;\n            }\n        }\n\n        if !found_sequence {\n            comp.add_sequence(1);\n        }\n    }\n}\n\npub(crate) fn itip_add_tz(message: &mut ICalendar, ical: &ICalendar) {\n    let mut has_timezones = false;\n\n    if message.components.iter().any(|c| {\n        has_timezones = has_timezones || c.component_type == ICalendarComponentType::VTimezone;\n\n        !has_timezones\n            && c.entries.iter().any(|e| {\n                e.params\n                    .iter()\n                    .any(|p| matches!(p.name, ICalendarParameterName::Tzid))\n            })\n    }) && !has_timezones\n    {\n        message.copy_timezones(ical);\n    }\n}\n\n#[inline]\npub(crate) fn can_attendee_modify_property(\n    component_type: &ICalendarComponentType,\n    property: &ICalendarProperty,\n) -> bool {\n    match component_type {\n        ICalendarComponentType::VEvent | ICalendarComponentType::VJournal => {\n            matches!(\n                property,\n                ICalendarProperty::Exdate\n                    | ICalendarProperty::Summary\n                    | ICalendarProperty::Description\n                    | ICalendarProperty::Comment\n            )\n        }\n        ICalendarComponentType::VTodo => matches!(\n            property,\n            ICalendarProperty::Exdate\n                | ICalendarProperty::Summary\n                | ICalendarProperty::Description\n                | ICalendarProperty::Status\n                | ICalendarProperty::PercentComplete\n                | ICalendarProperty::Completed\n                | ICalendarProperty::Comment\n        ),\n        _ => false,\n    }\n}\n\nimpl ItipMessages {\n    pub fn new(messages: Vec<ItipMessage<ICalendar>>) -> Self {\n        ItipMessages {\n            messages: messages.into_iter().map(|m| m.into()).collect(),\n        }\n    }\n\n    pub fn queue(self, batch: &mut BatchBuilder) -> trc::Result<()> {\n        let due = TaskEpoch::now().with_random_sequence_id();\n        batch.set(\n            ValueClass::TaskQueue(TaskQueueClass::SendImip {\n                due,\n                is_payload: false,\n            }),\n            vec![],\n        );\n        batch.set(\n            ValueClass::TaskQueue(TaskQueueClass::SendImip {\n                due,\n                is_payload: true,\n            }),\n            Archiver::new(self)\n                .serialize()\n                .caused_by(trc::location!())?,\n        );\n\n        Ok(())\n    }\n}\n\nimpl From<ItipMessage<ICalendar>> for ItipMessage<String> {\n    fn from(message: ItipMessage<ICalendar>) -> Self {\n        ItipMessage {\n            from: message.from,\n            from_organizer: message.from_organizer,\n            to: message.to,\n            summary: message.summary,\n            message: message.message.to_string(),\n        }\n    }\n}\n\nimpl ArchivedItipSummary {\n    pub fn method(&self) -> &str {\n        match self {\n            ArchivedItipSummary::Invite(_) => ICalendarMethod::Request.as_str(),\n            ArchivedItipSummary::Update { method, .. } => method.as_str(),\n            ArchivedItipSummary::Cancel(_) => ICalendarMethod::Cancel.as_str(),\n            ArchivedItipSummary::Rsvp { .. } => ICalendarMethod::Reply.as_str(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::{AHashMap, AHashSet};\nuse calcard::{\n    common::{IanaString, PartialDateTime},\n    icalendar::{\n        ICalendarComponent, ICalendarDuration, ICalendarEntry, ICalendarMethod, ICalendarParameter,\n        ICalendarParticipationRole, ICalendarParticipationStatus, ICalendarPeriod,\n        ICalendarProperty, ICalendarRecurrenceRule, ICalendarScheduleForceSendValue,\n        ICalendarStatus, ICalendarUserTypes, ICalendarValue, Uri,\n    },\n};\nuse std::{fmt::Display, hash::Hash};\n\npub mod attendee;\npub mod event_cancel;\npub mod event_create;\npub mod event_update;\npub mod inbound;\npub mod itip;\npub mod organizer;\npub mod snapshot;\n\n#[derive(Debug)]\npub struct ItipSnapshots<'x> {\n    pub organizer: Organizer<'x>,\n    pub uid: &'x str,\n    pub components: AHashMap<InstanceId, ItipSnapshot<'x>>,\n}\n\n#[derive(Debug)]\npub struct ItipSnapshot<'x> {\n    pub comp_id: u16,\n    pub comp: &'x ICalendarComponent,\n    pub attendees: AHashSet<Attendee<'x>>,\n    pub dtstamp: Option<&'x PartialDateTime>,\n    pub entries: AHashSet<ItipEntry<'x>>,\n    pub sequence: Option<i64>,\n    pub request_status: Vec<&'x str>,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash)]\npub struct ItipEntry<'x> {\n    pub name: &'x ICalendarProperty,\n    pub value: ItipEntryValue<'x>,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash)]\npub enum ItipEntryValue<'x> {\n    DateTime(ItipDateTime<'x>),\n    Period(&'x ICalendarPeriod),\n    Duration(&'x ICalendarDuration),\n    Status(&'x ICalendarStatus),\n    RRule(&'x ICalendarRecurrenceRule),\n    Text(&'x str),\n    Integer(i64),\n}\n\n#[derive(Debug)]\npub struct ItipDateTime<'x> {\n    pub date: &'x PartialDateTime,\n    pub tz_id: Option<&'x str>,\n    pub tz_code: u16,\n    pub timestamp: i64,\n}\n\n#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum InstanceId {\n    Main,\n    Recurrence(RecurrenceId),\n}\n\n#[derive(Debug, PartialOrd, Ord)]\npub struct RecurrenceId {\n    pub entry_id: u16,\n    pub date: i64,\n    pub this_and_future: bool,\n}\n\n#[derive(Debug)]\npub struct Attendee<'x> {\n    pub entry_id: u16,\n    pub email: Email,\n    pub name: Option<&'x str>,\n    pub part_stat: Option<&'x ICalendarParticipationStatus>,\n    pub delegated_from: Vec<Email>,\n    pub delegated_to: Vec<Email>,\n    pub role: Option<&'x ICalendarParticipationRole>,\n    pub cu_type: Option<&'x ICalendarUserTypes>,\n    pub sent_by: Option<Email>,\n    pub rsvp: Option<bool>,\n    pub is_server_scheduling: bool,\n    pub force_send: Option<&'x ICalendarScheduleForceSendValue>,\n}\n\n#[derive(Debug)]\npub struct Organizer<'x> {\n    pub entry_id: u16,\n    pub email: Email,\n    pub name: Option<&'x str>,\n    pub is_server_scheduling: bool,\n    pub force_send: Option<&'x ICalendarScheduleForceSendValue>,\n}\n\n#[derive(Debug)]\npub struct Email {\n    pub email: String,\n    pub is_local: bool,\n}\n\n#[derive(Debug)]\npub enum ItipError {\n    NoSchedulingInfo,\n    OtherSchedulingAgent,\n    NotOrganizer,\n    NotOrganizerNorAttendee,\n    NothingToSend,\n    MissingUid,\n    MultipleUid,\n    MultipleOrganizer,\n    MultipleObjectTypes,\n    MultipleObjectInstances,\n    CannotModifyProperty(ICalendarProperty),\n    CannotModifyInstance,\n    CannotModifyAddress,\n    OrganizerMismatch,\n    MissingMethod,\n    InvalidComponentType,\n    OutOfSequence,\n    OrganizerIsLocalAddress,\n    SenderIsNotOrganizerNorAttendee,\n    SenderIsNotParticipant(String),\n    UnknownParticipant(String),\n    UnsupportedMethod(ICalendarMethod),\n    ICalendarParseError,\n    EventNotFound,\n    EventTooLarge,\n    QuotaExceeded,\n    NoDefaultCalendar,\n    AutoAddDisabled,\n}\n\n#[derive(Debug, rkyv::Archive, rkyv::Deserialize, rkyv::Serialize)]\npub struct ItipMessage<T> {\n    pub from: String,\n    pub from_organizer: bool,\n    pub to: Vec<String>,\n    pub summary: ItipSummary,\n    pub message: T,\n}\n\n#[derive(Debug, Clone, rkyv::Archive, rkyv::Deserialize, rkyv::Serialize)]\npub enum ItipSummary {\n    Invite(Vec<ItipField>),\n    Update {\n        method: ICalendarMethod,\n        current: Vec<ItipField>,\n        previous: Vec<ItipField>,\n    },\n    Cancel(Vec<ItipField>),\n    Rsvp {\n        part_stat: ICalendarParticipationStatus,\n        current: Vec<ItipField>,\n    },\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq, rkyv::Archive, rkyv::Deserialize, rkyv::Serialize)]\npub struct ItipField {\n    pub name: ICalendarProperty,\n    pub value: ItipValue,\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq, rkyv::Archive, rkyv::Deserialize, rkyv::Serialize)]\npub enum ItipValue {\n    Text(String),\n    Time(ItipTime),\n    Rrule(Box<ICalendarRecurrenceRule>),\n    Participants(Vec<ItipParticipant>),\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq, rkyv::Archive, rkyv::Deserialize, rkyv::Serialize)]\npub struct ItipTime {\n    pub start: i64,\n    pub tz_id: u16,\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq, rkyv::Archive, rkyv::Deserialize, rkyv::Serialize)]\npub struct ItipParticipant {\n    pub email: String,\n    pub name: Option<String>,\n    pub is_organizer: bool,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize)]\npub struct ItipMessages {\n    pub messages: Vec<ItipMessage<String>>,\n}\n\nimpl Attendee<'_> {\n    pub fn send_invite_messages(&self) -> bool {\n        !self.email.is_local\n            && self.is_server_scheduling\n            && self.rsvp.is_none_or(|rsvp| rsvp)\n            && (self.force_send.is_some()\n                || self.part_stat.is_none_or(|part_stat| {\n                    part_stat == &ICalendarParticipationStatus::NeedsAction\n                }))\n    }\n\n    pub fn send_update_messages(&self) -> bool {\n        !self.email.is_local\n            && self.is_server_scheduling\n            && self.rsvp.is_none_or(|rsvp| rsvp)\n            && (self.force_send.is_some()\n                || self\n                    .part_stat\n                    .is_none_or(|part_stat| part_stat != &ICalendarParticipationStatus::Declined))\n    }\n\n    pub fn is_delegated_from(&self, attendee: &Attendee<'_>) -> bool {\n        self.delegated_from\n            .iter()\n            .any(|d| d.email == attendee.email.email)\n    }\n\n    pub fn is_delegated_to(&self, attendee: &Attendee<'_>) -> bool {\n        self.delegated_to\n            .iter()\n            .any(|d| d.email == attendee.email.email)\n    }\n}\n\nimpl Email {\n    pub fn new(email: &str, local_addresses: &[String]) -> Option<Self> {\n        email.contains('@').then(|| {\n            let email = email.trim().trim_start_matches(\"mailto:\").to_lowercase();\n            let is_local = local_addresses.contains(&email);\n            Email { email, is_local }\n        })\n    }\n\n    pub fn from_uri(uri: &Uri, local_addresses: &[String]) -> Option<Self> {\n        if let Uri::Location(uri) = uri {\n            Email::new(uri.as_str(), local_addresses)\n        } else {\n            None\n        }\n    }\n}\n\nimpl PartialEq for Attendee<'_> {\n    fn eq(&self, other: &Self) -> bool {\n        self.email == other.email\n            && self.part_stat == other.part_stat\n            && self.delegated_from == other.delegated_from\n            && self.delegated_to == other.delegated_to\n            && self.role == other.role\n            && self.cu_type == other.cu_type\n            && self.sent_by == other.sent_by\n    }\n}\n\nimpl Eq for Attendee<'_> {}\n\nimpl Hash for Attendee<'_> {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.email.hash(state);\n        self.part_stat.hash(state);\n        self.delegated_from.hash(state);\n        self.delegated_to.hash(state);\n        self.role.hash(state);\n        self.cu_type.hash(state);\n        self.sent_by.hash(state);\n    }\n}\n\nimpl Display for Email {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"mailto:{}\", self.email)\n    }\n}\n\nimpl Hash for Email {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.email.hash(state);\n    }\n}\n\nimpl PartialEq for Email {\n    fn eq(&self, other: &Self) -> bool {\n        self.email == other.email\n    }\n}\n\nimpl Eq for Email {}\n\nimpl PartialEq for RecurrenceId {\n    fn eq(&self, other: &Self) -> bool {\n        self.date == other.date && self.this_and_future == other.this_and_future\n    }\n}\n\nimpl Eq for RecurrenceId {}\n\nimpl Hash for RecurrenceId {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.date.hash(state);\n        self.this_and_future.hash(state);\n    }\n}\n\nimpl PartialEq for ItipDateTime<'_> {\n    fn eq(&self, other: &Self) -> bool {\n        self.timestamp == other.timestamp\n    }\n}\nimpl Eq for ItipDateTime<'_> {}\n\nimpl Hash for ItipDateTime<'_> {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.timestamp.hash(state);\n    }\n}\n\nimpl ItipDateTime<'_> {\n    pub fn to_entry(&self, name: ICalendarProperty) -> ICalendarEntry {\n        ICalendarEntry {\n            name,\n            params: self\n                .tz_id\n                .map(|tz_id| vec![ICalendarParameter::tzid(tz_id.to_string())])\n                .unwrap_or_default(),\n            values: vec![ICalendarValue::PartialDateTime(Box::new(self.date.clone()))],\n        }\n    }\n}\n\nimpl ItipError {\n    pub fn is_jmap_error(&self) -> bool {\n        matches!(\n            self,\n            ItipError::MultipleOrganizer\n                | ItipError::OrganizerIsLocalAddress\n                | ItipError::SenderIsNotParticipant(_)\n                | ItipError::OrganizerMismatch\n                | ItipError::CannotModifyProperty(_)\n                | ItipError::CannotModifyInstance\n                | ItipError::CannotModifyAddress\n                //| ItipError::MissingUid\n                | ItipError::MultipleUid\n                | ItipError::MultipleObjectTypes\n                | ItipError::MultipleObjectInstances\n                | ItipError::MissingMethod\n                | ItipError::InvalidComponentType\n                | ItipError::OutOfSequence\n                | ItipError::UnknownParticipant(_)\n                | ItipError::UnsupportedMethod(_)\n        )\n    }\n}\n\nimpl Display for ItipError {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            ItipError::NoSchedulingInfo => write!(f, \"No scheduling information found\"),\n            ItipError::OtherSchedulingAgent => write!(f, \"Other scheduling agent\"),\n            ItipError::NotOrganizer => write!(f, \"Not the organizer of the event\"),\n            ItipError::NotOrganizerNorAttendee => write!(f, \"Not an organizer or attendee\"),\n            ItipError::NothingToSend => write!(f, \"No iTIP messages to send\"),\n            ItipError::MissingUid => write!(f, \"Missing UID in iCalendar object\"),\n            ItipError::MultipleUid => write!(f, \"Multiple UIDs found in iCalendar object\"),\n            ItipError::MultipleOrganizer => {\n                write!(f, \"Multiple organizers found in iCalendar object\")\n            }\n            ItipError::MultipleObjectTypes => {\n                write!(f, \"Multiple object types found in iCalendar object\")\n            }\n            ItipError::MultipleObjectInstances => {\n                write!(f, \"Multiple object instances found in iCalendar object\")\n            }\n            ItipError::CannotModifyProperty(prop) => {\n                write!(f, \"Cannot modify property {}\", prop.as_str())\n            }\n            ItipError::CannotModifyInstance => write!(f, \"Cannot modify instance of the event\"),\n            ItipError::CannotModifyAddress => write!(f, \"Cannot modify address of the event\"),\n            ItipError::OrganizerMismatch => write!(f, \"Organizer mismatch in iCalendar object\"),\n            ItipError::MissingMethod => write!(f, \"Missing method in the iTIP message\"),\n            ItipError::InvalidComponentType => {\n                write!(f, \"Invalid component type in iCalendar object\")\n            }\n            ItipError::OutOfSequence => write!(f, \"Old sequence number found\"),\n            ItipError::OrganizerIsLocalAddress => {\n                write!(\n                    f,\n                    \"Organizer matches one of the recipient's account addresses\"\n                )\n            }\n            ItipError::SenderIsNotParticipant(participant) => {\n                write!(f, \"Sender {participant:?} is not a participant\")\n            }\n            ItipError::SenderIsNotOrganizerNorAttendee => {\n                write!(f, \"Sender is neither organizer nor attendee\")\n            }\n            ItipError::UnknownParticipant(participant) => {\n                write!(f, \"Unknown participant: {}\", participant)\n            }\n            ItipError::UnsupportedMethod(method) => {\n                write!(f, \"Unsupported method: {}\", method.as_str())\n            }\n            ItipError::ICalendarParseError => write!(f, \"Failed to parse iCalendar object\"),\n            ItipError::EventNotFound => write!(f, \"Event found in index but not in database\"),\n            ItipError::EventTooLarge => write!(\n                f,\n                \"Applying the iTIP message would exceed the maximum event size\"\n            ),\n            ItipError::QuotaExceeded => write!(f, \"Quota exceeded\"),\n            ItipError::NoDefaultCalendar => write!(f, \"No default calendar found for the account\"),\n            ItipError::AutoAddDisabled => {\n                write!(f, \"Auto-adding events is disabled for this account\")\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/organizer.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{\n    InstanceId, ItipError, ItipMessage, ItipSnapshots, ItipSummary,\n    event_cancel::build_cancel_component,\n    itip::{ItipExportAs, itip_add_tz, itip_build_envelope, itip_export_component},\n};\nuse ahash::{AHashMap, AHashSet};\nuse calcard::{\n    common::PartialDateTime,\n    icalendar::{\n        ICalendar, ICalendarComponent, ICalendarComponentType, ICalendarMethod,\n        ICalendarParticipationStatus, ICalendarProperty, ICalendarStatus,\n    },\n};\nuse std::collections::hash_map::Entry;\n\npub(crate) fn organizer_handle_update(\n    old_ical: &ICalendar,\n    new_ical: &ICalendar,\n    old_itip: ItipSnapshots<'_>,\n    new_itip: ItipSnapshots<'_>,\n    increment_sequences: &mut Vec<u16>,\n) -> Result<Vec<ItipMessage<ICalendar>>, ItipError> {\n    let mut changed_instances: Vec<(&InstanceId, &str, &ICalendarMethod)> = Vec::new();\n    let mut increment_sequence = false;\n    let mut changed_properties = AHashSet::new();\n\n    for (instance_id, instance) in &new_itip.components {\n        if let Some(old_instance) = old_itip.components.get(instance_id) {\n            let changed_entries = instance.entries != old_instance.entries;\n            let changed_attendees = instance.attendees != old_instance.attendees;\n\n            if changed_entries || changed_attendees {\n                if changed_entries {\n                    for entry in instance.entries.symmetric_difference(&old_instance.entries) {\n                        increment_sequence = increment_sequence\n                            || matches!(\n                                entry.name,\n                                ICalendarProperty::Dtstart\n                                    | ICalendarProperty::Dtend\n                                    | ICalendarProperty::Duration\n                                    | ICalendarProperty::Due\n                                    | ICalendarProperty::Rrule\n                                    | ICalendarProperty::Rdate\n                                    | ICalendarProperty::Exdate\n                                    | ICalendarProperty::Status\n                                    | ICalendarProperty::Location\n                            );\n                        changed_properties.insert(entry.name);\n                    }\n                }\n\n                if changed_attendees {\n                    changed_instances.extend(\n                        old_instance\n                            .external_attendees()\n                            .filter(|attendee| attendee.send_update_messages())\n                            .map(|attendee| attendee.email.email.as_str())\n                            .collect::<AHashSet<_>>()\n                            .difference(\n                                &instance\n                                    .external_attendees()\n                                    .map(|attendee| attendee.email.email.as_str())\n                                    .collect::<AHashSet<_>>(),\n                            )\n                            .map(|attendee| (instance_id, *attendee, &ICalendarMethod::Cancel)),\n                    );\n                    changed_properties.insert(&ICalendarProperty::Attendee);\n                    increment_sequence = true;\n                }\n\n                changed_instances.extend(instance.attendees.iter().filter_map(|attendee| {\n                    if attendee.send_update_messages() {\n                        Some((\n                            instance_id,\n                            attendee.email.email.as_str(),\n                            &ICalendarMethod::Request,\n                        ))\n                    } else {\n                        None\n                    }\n                }));\n            }\n        } else if instance_id != &InstanceId::Main {\n            changed_properties.insert(&ICalendarProperty::Exdate);\n            let method = if matches!(instance.comp.status(), Some(ICalendarStatus::Cancelled)) {\n                &ICalendarMethod::Cancel\n            } else {\n                &ICalendarMethod::Add\n            };\n\n            changed_instances.extend(instance.attendees.iter().filter_map(|attendee| {\n                if attendee.send_invite_messages() {\n                    Some((instance_id, attendee.email.email.as_str(), method))\n                } else {\n                    None\n                }\n            }));\n\n            increment_sequence = true;\n        } else {\n            return Err(ItipError::CannotModifyInstance);\n        }\n    }\n\n    for (instance_id, old_instance) in &old_itip.components {\n        if !new_itip.components.contains_key(instance_id) {\n            if instance_id != &InstanceId::Main {\n                changed_instances.extend(old_instance.attendees.iter().filter_map(|attendee| {\n                    if attendee.send_update_messages() {\n                        Some((\n                            instance_id,\n                            attendee.email.email.as_str(),\n                            &ICalendarMethod::Cancel,\n                        ))\n                    } else {\n                        None\n                    }\n                }));\n                changed_properties.insert(&ICalendarProperty::Exdate);\n                increment_sequence = true;\n            } else {\n                return Err(ItipError::CannotModifyInstance);\n            }\n        }\n    }\n\n    if changed_instances.is_empty() {\n        return Err(ItipError::NothingToSend);\n    }\n\n    // Remove partial notifications for attendees that receive a full update for the main instance\n    // or, that will receive both add and remove messages\n    let mut send_full_update: AHashSet<&str> = AHashSet::new();\n    let mut send_partial_update: AHashMap<&str, AHashMap<&ICalendarMethod, Vec<&InstanceId>>> =\n        AHashMap::new();\n    for (instance_id, email, method) in &changed_instances {\n        if *instance_id == &InstanceId::Main && *method == &ICalendarMethod::Request {\n            send_full_update.insert(*email);\n            send_partial_update.remove(email);\n        } else if !send_full_update.contains(email) {\n            match send_partial_update.entry(email) {\n                Entry::Occupied(mut entry) => {\n                    let entry = entry.get_mut();\n                    let is_empty = entry.is_empty();\n                    match entry.entry(method) {\n                        Entry::Occupied(mut method_entry) => {\n                            method_entry.get_mut().push(*instance_id);\n                        }\n                        Entry::Vacant(method_entry) if is_empty => {\n                            method_entry.insert(vec![*instance_id]);\n                        }\n                        _ => {\n                            // Switch to full update for this participant\n                            send_full_update.insert(*email);\n                            send_partial_update.remove(email);\n                        }\n                    }\n                }\n                Entry::Vacant(entry) => {\n                    entry.insert(AHashMap::from_iter([(*method, vec![*instance_id])]));\n                }\n            }\n        }\n    }\n\n    // Build summary of changed properties\n    let new_summary = new_itip\n        .main_instance_or_default()\n        .build_summary(Some(&new_itip.organizer), &[]);\n    let old_summary = old_itip\n        .main_instance_or_default()\n        .build_summary(Some(&old_itip.organizer), &new_summary);\n\n    // Prepare full updates\n    let mut messages = Vec::new();\n    if !send_full_update.is_empty() {\n        match organizer_request_full(\n            new_ical,\n            &new_itip,\n            increment_sequence.then_some(increment_sequences),\n            false,\n        ) {\n            Ok(messages_) => {\n                for mut message in messages_ {\n                    message.summary = ItipSummary::Update {\n                        method: ICalendarMethod::Request,\n                        current: new_summary.clone(),\n                        previous: old_summary.clone(),\n                    };\n                    messages.push(message);\n                }\n            }\n            Err(err) => {\n                if send_partial_update.is_empty() {\n                    return Err(err);\n                }\n            }\n        }\n    }\n\n    // Prepare partial updates\n    if !send_partial_update.is_empty() {\n        // Group updates by email and method\n        let mut updates: AHashMap<(&ICalendarMethod, Vec<&InstanceId>), Vec<&str>> =\n            AHashMap::new();\n        for (email, partial_updates) in send_partial_update {\n            for (method, mut instances) in partial_updates {\n                instances.sort_unstable();\n                instances.dedup();\n                updates.entry((method, instances)).or_default().push(email);\n            }\n        }\n\n        let dt_stamp = PartialDateTime::now();\n        for ((method, instances), emails) in updates {\n            let (mut ical, mut itip, is_cancel) = if matches!(method, ICalendarMethod::Cancel) {\n                (old_ical, &old_itip, true)\n            } else {\n                (new_ical, &new_itip, false)\n            };\n\n            // Prepare iTIP message\n            let mut message = ICalendar {\n                components: Vec::with_capacity(instances.len() + 1),\n            };\n            message.components.push(itip_build_envelope(method.clone()));\n\n            let mut increment_sequences = Vec::new();\n\n            for instance_id in instances {\n                let comp = match itip.components.get(instance_id) {\n                    Some(comp) => comp,\n                    None => {\n                        // New component added with CANCELLED status\n                        ical = new_ical;\n                        itip = &new_itip;\n                        itip.components.get(instance_id).unwrap()\n                    }\n                };\n                // Prepare component for iTIP\n                let sequence = if increment_sequence {\n                    comp.sequence.unwrap_or_default() + 1\n                } else {\n                    comp.sequence.unwrap_or_default()\n                };\n                let orig_component = comp.comp;\n                let component = if !is_cancel {\n                    if increment_sequence {\n                        increment_sequences.push(comp.comp_id);\n                    }\n\n                    // Export component with updated sequence and participation status\n                    itip_export_component(\n                        orig_component,\n                        itip.uid,\n                        &dt_stamp,\n                        sequence,\n                        ItipExportAs::Organizer(&ICalendarParticipationStatus::NeedsAction),\n                    )\n                } else {\n                    build_cancel_component(orig_component, sequence, dt_stamp.clone(), &emails)\n                };\n\n                // Add component to message\n                let comp_id = message.components.len() as u32;\n                message.components.push(component);\n                message.components[0].component_ids.push(comp_id);\n            }\n\n            // Add timezones\n            itip_add_tz(&mut message, ical);\n\n            messages.push(ItipMessage {\n                from: itip.organizer.email.email.clone(),\n                from_organizer: true,\n                to: emails.into_iter().map(|e| e.to_string()).collect(),\n                summary: if method == &ICalendarMethod::Cancel {\n                    ItipSummary::Cancel(\n                        new_summary\n                            .iter()\n                            .chain(old_summary.iter())\n                            .map(|summary| (&summary.name, summary))\n                            .collect::<AHashMap<_, _>>()\n                            .into_values()\n                            .cloned()\n                            .collect(),\n                    )\n                } else {\n                    ItipSummary::Update {\n                        method: method.clone(),\n                        current: new_summary.clone(),\n                        previous: old_summary.clone(),\n                    }\n                },\n                message,\n            });\n        }\n    }\n\n    Ok(messages)\n}\n\npub(crate) fn organizer_request_full(\n    ical: &ICalendar,\n    itip: &ItipSnapshots<'_>,\n    mut increment_sequence: Option<&mut Vec<u16>>,\n    is_first_request: bool,\n) -> Result<Vec<ItipMessage<ICalendar>>, ItipError> {\n    // Prepare iTIP message\n    let dt_stamp = PartialDateTime::now();\n    let mut message = ICalendar {\n        components: vec![ICalendarComponent::default(); ical.components.len()],\n    };\n    message.components[0] = itip_build_envelope(ICalendarMethod::Request);\n\n    let mut recipients = AHashSet::new();\n    let mut copy_components = AHashSet::new();\n\n    for comp in itip.components.values() {\n        // Skip private components\n        if comp.attendees.is_empty() {\n            continue;\n        }\n\n        // Prepare component for iTIP\n        let sequence = if let Some(increment_sequence) = &mut increment_sequence {\n            increment_sequence.push(comp.comp_id);\n            comp.sequence.unwrap_or_default() + 1\n        } else {\n            comp.sequence.unwrap_or_default()\n        };\n        let orig_component = &ical.components[comp.comp_id as usize];\n        let mut component = itip_export_component(\n            orig_component,\n            itip.uid,\n            &dt_stamp,\n            sequence,\n            ItipExportAs::Organizer(&ICalendarParticipationStatus::NeedsAction),\n        );\n\n        // Add VALARM sub-components\n        if is_first_request {\n            for sub_comp_id in &orig_component.component_ids {\n                if matches!(\n                    ical.components[*sub_comp_id as usize].component_type,\n                    ICalendarComponentType::VAlarm\n                ) {\n                    copy_components.insert(*sub_comp_id);\n                    component.component_ids.push(*sub_comp_id);\n                }\n            }\n        }\n\n        // Add component to message\n        message.components[comp.comp_id as usize] = component;\n        message.components[0]\n            .component_ids\n            .push(comp.comp_id as u32);\n\n        // Add attendees\n        for attendee in &comp.attendees {\n            if (is_first_request && attendee.send_invite_messages())\n                || (!is_first_request && attendee.send_update_messages())\n            {\n                recipients.insert(&attendee.email.email);\n            }\n        }\n    }\n\n    // Copy timezones and alarms\n    for (comp_id, comp) in ical.components.iter().enumerate() {\n        if matches!(comp.component_type, ICalendarComponentType::VTimezone) {\n            copy_components.extend(comp.component_ids.iter().copied());\n            message.components[0].component_ids.push(comp_id as u32);\n        } else if !copy_components.contains(&(comp_id as u32)) {\n            continue;\n        }\n        message.components[comp_id] = comp.clone();\n    }\n    message.components[0].component_ids.sort_unstable();\n\n    if !recipients.is_empty() {\n        Ok(vec![ItipMessage {\n            from: itip.organizer.email.email.clone(),\n            from_organizer: true,\n            to: recipients.into_iter().map(|e| e.to_string()).collect(),\n            summary: ItipSummary::Invite(\n                itip.main_instance_or_default()\n                    .build_summary(Some(&itip.organizer), &[]),\n            ),\n            message,\n        }])\n    } else {\n        Err(ItipError::NothingToSend)\n    }\n}\n"
  },
  {
    "path": "crates/groupware/src/scheduling/snapshot.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::scheduling::{\n    Attendee, Email, InstanceId, ItipDateTime, ItipEntry, ItipEntryValue, ItipError, ItipField,\n    ItipParticipant, ItipSnapshot, ItipSnapshots, ItipTime, ItipValue, Organizer, RecurrenceId,\n};\nuse ahash::AHashMap;\nuse calcard::icalendar::{\n    ICalendar, ICalendarParameterName, ICalendarParameterValue, ICalendarProperty,\n    ICalendarScheduleAgentValue, ICalendarValue, Uri,\n};\n\npub fn itip_snapshot<'x, 'y>(\n    ical: &'x ICalendar,\n    account_emails: &'y [String],\n    force_add_client_scheduling: bool,\n) -> Result<ItipSnapshots<'x>, ItipError> {\n    if !ical.components.iter().any(|comp| {\n        comp.component_type.is_scheduling_object()\n            && comp\n                .entries\n                .iter()\n                .any(|e| matches!(e.name, ICalendarProperty::Organizer))\n    }) {\n        return Err(ItipError::NoSchedulingInfo);\n    }\n\n    let mut organizer: Option<Organizer<'x>> = None;\n    let mut uid: Option<&'x str> = None;\n    let mut components = AHashMap::new();\n    let mut expect_object_type = None;\n    let mut has_local_emails = false;\n    let mut tz_resolver = None;\n\n    for (comp_id, comp) in ical.components.iter().enumerate() {\n        if comp.component_type.is_scheduling_object() {\n            match expect_object_type {\n                Some(expected) if expected != &comp.component_type => {\n                    return Err(ItipError::MultipleObjectTypes);\n                }\n                None => {\n                    expect_object_type = Some(&comp.component_type);\n                }\n                _ => {}\n            }\n\n            let mut sched_comp = ItipSnapshot {\n                comp_id: comp_id as u16,\n                comp,\n                attendees: Default::default(),\n                dtstamp: Default::default(),\n                entries: Default::default(),\n                sequence: Default::default(),\n                request_status: Default::default(),\n            };\n            let mut instance_id = InstanceId::Main;\n\n            for (entry_id, entry) in comp.entries.iter().enumerate() {\n                match &entry.name {\n                    ICalendarProperty::Organizer => {\n                        if let Some(email) = entry\n                            .values\n                            .first()\n                            .and_then(|v| v.as_text())\n                            .and_then(|v| Email::new(v, account_emails))\n                        {\n                            let mut part = Organizer {\n                                entry_id: entry_id as u16,\n                                email,\n                                is_server_scheduling: true,\n                                name: None,\n                                force_send: None,\n                            };\n                            has_local_emails |= part.email.is_local;\n\n                            for param in &entry.params {\n                                match (&param.name, &param.value) {\n                                    (\n                                        ICalendarParameterName::ScheduleAgent,\n                                        ICalendarParameterValue::ScheduleAgent(\n                                            ICalendarScheduleAgentValue::Client\n                                            | ICalendarScheduleAgentValue::None,\n                                        ),\n                                    ) => {\n                                        part.is_server_scheduling = false;\n                                    }\n                                    (\n                                        ICalendarParameterName::ScheduleForceSend,\n                                        ICalendarParameterValue::ScheduleForceSend(force_send),\n                                    ) => {\n                                        part.force_send = Some(force_send);\n                                    }\n                                    (\n                                        ICalendarParameterName::Cn,\n                                        ICalendarParameterValue::Text(name),\n                                    ) => {\n                                        part.name = Some(name.as_str());\n                                    }\n                                    _ => {}\n                                }\n                            }\n\n                            if !part.is_server_scheduling && !force_add_client_scheduling {\n                                return Err(ItipError::OtherSchedulingAgent);\n                            }\n\n                            match organizer {\n                                Some(existing_organizer)\n                                    if existing_organizer.email.email != part.email.email =>\n                                {\n                                    return Err(ItipError::MultipleOrganizer);\n                                }\n                                None => {\n                                    organizer = Some(part);\n                                }\n                                _ => {}\n                            }\n                        }\n                    }\n                    ICalendarProperty::Attendee => {\n                        if let Some(email) = entry\n                            .values\n                            .first()\n                            .and_then(|v| v.as_text())\n                            .and_then(|v| Email::new(v, account_emails))\n                        {\n                            let mut part = Attendee {\n                                entry_id: entry_id as u16,\n                                email,\n                                name: None,\n                                rsvp: None,\n                                is_server_scheduling: true,\n                                force_send: None,\n                                part_stat: None,\n                                delegated_from: vec![],\n                                delegated_to: vec![],\n                                cu_type: None,\n                                role: None,\n                                sent_by: None,\n                            };\n\n                            for param in &entry.params {\n                                match (&param.name, &param.value) {\n                                    (\n                                        ICalendarParameterName::ScheduleAgent,\n                                        ICalendarParameterValue::ScheduleAgent(agent),\n                                    ) => {\n                                        part.is_server_scheduling =\n                                            agent == &ICalendarScheduleAgentValue::Server;\n                                    }\n                                    (\n                                        ICalendarParameterName::Rsvp,\n                                        ICalendarParameterValue::Bool(rsvp),\n                                    ) => {\n                                        part.rsvp = Some(*rsvp);\n                                    }\n                                    (\n                                        ICalendarParameterName::ScheduleForceSend,\n                                        ICalendarParameterValue::ScheduleForceSend(force_send),\n                                    ) => {\n                                        part.force_send = Some(force_send);\n                                    }\n                                    (\n                                        ICalendarParameterName::Partstat,\n                                        ICalendarParameterValue::Partstat(value),\n                                    ) => {\n                                        part.part_stat = Some(value);\n                                    }\n                                    (\n                                        ICalendarParameterName::Cutype,\n                                        ICalendarParameterValue::Cutype(value),\n                                    ) => {\n                                        part.cu_type = Some(value);\n                                    }\n                                    (\n                                        ICalendarParameterName::DelegatedFrom,\n                                        ICalendarParameterValue::Uri(uri),\n                                    ) => {\n                                        if let Some(uri) = Email::from_uri(uri, account_emails) {\n                                            part.delegated_from.push(uri);\n                                        }\n                                    }\n                                    (\n                                        ICalendarParameterName::DelegatedTo,\n                                        ICalendarParameterValue::Uri(uri),\n                                    ) => {\n                                        if let Some(uri) = Email::from_uri(uri, account_emails) {\n                                            part.delegated_to.push(uri);\n                                        }\n                                    }\n                                    (\n                                        ICalendarParameterName::Role,\n                                        ICalendarParameterValue::Role(value),\n                                    ) => {\n                                        part.role = Some(value);\n                                    }\n                                    (\n                                        ICalendarParameterName::SentBy,\n                                        ICalendarParameterValue::Uri(value),\n                                    ) => {\n                                        part.sent_by = Email::from_uri(value, account_emails);\n                                    }\n                                    (\n                                        ICalendarParameterName::Cn,\n                                        ICalendarParameterValue::Text(name),\n                                    ) => {\n                                        part.name = Some(name.as_str());\n                                    }\n                                    _ => {}\n                                }\n                            }\n\n                            has_local_emails |= part.email.is_local\n                                && (force_add_client_scheduling || part.is_server_scheduling);\n\n                            sched_comp.attendees.insert(part);\n                        }\n                    }\n                    ICalendarProperty::Uid => {\n                        if let Some(uid_) = entry\n                            .values\n                            .first()\n                            .and_then(|v| v.as_text())\n                            .map(|v| v.trim())\n                            .filter(|v| !v.is_empty())\n                        {\n                            match uid {\n                                Some(existing_uid) if existing_uid != uid_ => {\n                                    return Err(ItipError::MultipleUid);\n                                }\n                                None => {\n                                    uid = Some(uid_);\n                                }\n                                _ => {}\n                            }\n                        }\n                    }\n                    ICalendarProperty::Sequence => {\n                        if let Some(sequence) = entry.values.first().and_then(|v| v.as_integer()) {\n                            sched_comp.sequence = Some(sequence);\n                        }\n                    }\n                    ICalendarProperty::RecurrenceId => {\n                        if let Some(date) =\n                            entry.values.first().and_then(|v| v.as_partial_date_time())\n                        {\n                            let mut this_and_future = false;\n                            let mut tz_id = None;\n\n                            for param in &entry.params {\n                                match (&param.name, &param.value) {\n                                    (\n                                        ICalendarParameterName::Tzid,\n                                        ICalendarParameterValue::Text(id),\n                                    ) => {\n                                        tz_id = Some(id.as_str());\n                                    }\n                                    (ICalendarParameterName::Range, _) => {\n                                        this_and_future = true;\n                                    }\n                                    _ => (),\n                                }\n                            }\n\n                            instance_id = InstanceId::Recurrence(RecurrenceId {\n                                entry_id: entry_id as u16,\n                                date: date\n                                    .to_date_time_with_tz(\n                                        tz_resolver\n                                            .get_or_insert_with(|| ical.build_tz_resolver())\n                                            .resolve_or_default(tz_id),\n                                    )\n                                    .map(|dt| dt.timestamp())\n                                    .unwrap_or_else(|| date.to_timestamp().unwrap_or_default()),\n                                this_and_future,\n                            });\n                        }\n                    }\n                    ICalendarProperty::RequestStatus => {\n                        if let Some(value) = entry.values.first().and_then(|v| v.as_text()) {\n                            sched_comp.request_status.push(value);\n                        }\n                    }\n                    ICalendarProperty::Dtstamp => {\n                        sched_comp.dtstamp =\n                            entry.values.first().and_then(|v| v.as_partial_date_time());\n                    }\n                    ICalendarProperty::Dtstart\n                    | ICalendarProperty::Dtend\n                    | ICalendarProperty::Duration\n                    | ICalendarProperty::Due\n                    | ICalendarProperty::Rrule\n                    | ICalendarProperty::Rdate\n                    | ICalendarProperty::Exdate\n                    | ICalendarProperty::Status\n                    | ICalendarProperty::Location\n                    | ICalendarProperty::Summary\n                    | ICalendarProperty::Description\n                    | ICalendarProperty::Priority\n                    | ICalendarProperty::PercentComplete\n                    | ICalendarProperty::Completed => {\n                        let tz_id = entry.tz_id();\n                        for value in &entry.values {\n                            let value = match value {\n                                ICalendarValue::Uri(Uri::Location(v)) => {\n                                    ItipEntryValue::Text(v.as_str())\n                                }\n                                ICalendarValue::PartialDateTime(date) => {\n                                    let tz = tz_resolver\n                                        .get_or_insert_with(|| ical.build_tz_resolver())\n                                        .resolve_or_default(tz_id);\n                                    ItipEntryValue::DateTime(ItipDateTime {\n                                        date: date.as_ref(),\n                                        tz_id,\n                                        tz_code: tz.as_id(),\n                                        timestamp: date\n                                            .to_date_time_with_tz(tz)\n                                            .map(|dt| dt.timestamp())\n                                            .unwrap_or_else(|| {\n                                                date.to_timestamp().unwrap_or_default()\n                                            }),\n                                    })\n                                }\n                                ICalendarValue::Duration(v) => ItipEntryValue::Duration(v),\n                                ICalendarValue::RecurrenceRule(v) => ItipEntryValue::RRule(v),\n                                ICalendarValue::Period(v) => ItipEntryValue::Period(v),\n                                ICalendarValue::Integer(v) => ItipEntryValue::Integer(*v),\n                                ICalendarValue::Text(v) => ItipEntryValue::Text(v.as_str()),\n                                ICalendarValue::Status(v) => ItipEntryValue::Status(v),\n                                _ => continue,\n                            };\n                            sched_comp.entries.insert(ItipEntry {\n                                name: &entry.name,\n                                value,\n                            });\n                        }\n                    }\n                    _ => {}\n                }\n            }\n\n            if components.insert(instance_id, sched_comp).is_some() {\n                return Err(ItipError::MultipleObjectInstances);\n            }\n        }\n    }\n\n    if has_local_emails {\n        Ok(ItipSnapshots {\n            organizer: organizer.ok_or(ItipError::NoSchedulingInfo)?,\n            uid: uid.ok_or(ItipError::MissingUid)?,\n            components,\n        })\n    } else {\n        Err(ItipError::NotOrganizerNorAttendee)\n    }\n}\n\nimpl ItipSnapshots<'_> {\n    pub fn sender_is_organizer_or_attendee(&self, email: &str) -> bool {\n        self.organizer.email.email == email\n            || self.components.values().any(|snapshot| {\n                snapshot\n                    .attendees\n                    .iter()\n                    .any(|attendee| attendee.email.email == email)\n            })\n    }\n\n    pub fn main_instance(&self) -> Option<&ItipSnapshot<'_>> {\n        self.components.get(&InstanceId::Main)\n    }\n\n    pub fn main_instance_or_default(&self) -> &ItipSnapshot<'_> {\n        self.main_instance()\n            .unwrap_or_else(|| self.components.values().next().unwrap())\n    }\n}\n\nimpl ItipSnapshot<'_> {\n    pub fn has_local_attendee(&self) -> bool {\n        self.attendees\n            .iter()\n            .any(|attendee| attendee.email.is_local)\n    }\n\n    pub fn local_attendee(&self) -> Option<&Attendee<'_>> {\n        self.attendees\n            .iter()\n            .find(|attendee| attendee.email.is_local)\n    }\n\n    pub fn external_attendees(&self) -> impl Iterator<Item = &Attendee<'_>> + '_ {\n        self.attendees.iter().filter(|item| !item.email.is_local)\n    }\n\n    pub fn attendee_by_email(&self, email: &str) -> Option<&Attendee<'_>> {\n        self.attendees\n            .iter()\n            .find(|attendee| attendee.email.email == email)\n    }\n\n    pub fn build_summary(\n        &self,\n        include_guests: Option<&Organizer<'_>>,\n        skip_fields: &[ItipField],\n    ) -> Vec<ItipField> {\n        let mut fields = Vec::with_capacity(5);\n\n        for entry in &self.entries {\n            if matches!(\n                entry.name,\n                ICalendarProperty::Summary\n                    | ICalendarProperty::Description\n                    | ICalendarProperty::Dtstart\n                    | ICalendarProperty::Location\n                    | ICalendarProperty::Rrule\n            ) {\n                let value = match &entry.value {\n                    ItipEntryValue::DateTime(dt) => ItipValue::Time(ItipTime {\n                        start: dt.timestamp,\n                        tz_id: dt.tz_code,\n                    }),\n                    ItipEntryValue::RRule(rule) => ItipValue::Rrule(Box::new((*rule).clone())),\n                    ItipEntryValue::Text(value) => ItipValue::Text(value.to_string()),\n                    _ => continue,\n                };\n                let field = ItipField {\n                    name: entry.name.clone(),\n                    value,\n                };\n\n                if !skip_fields.contains(&field) {\n                    fields.push(field);\n                }\n            }\n        }\n\n        if let Some(organizer) = include_guests {\n            let mut attendees = Vec::with_capacity(self.attendees.len());\n            for attendee in &self.attendees {\n                if attendee.email.email != organizer.email.email {\n                    attendees.push(ItipParticipant {\n                        email: attendee.email.email.to_string(),\n                        name: attendee.name.map(|n| n.to_string()),\n                        is_organizer: false,\n                    });\n                }\n            }\n            attendees.push(ItipParticipant {\n                email: organizer.email.email.to_string(),\n                name: organizer.name.map(|n| n.to_string()),\n                is_organizer: true,\n            });\n            attendees.sort_by(|a, b| {\n                if a.is_organizer && !b.is_organizer {\n                    std::cmp::Ordering::Less\n                } else if !a.is_organizer && b.is_organizer {\n                    std::cmp::Ordering::Greater\n                } else if let (Some(a_name), Some(b_name)) = (a.name.as_deref(), b.name.as_deref())\n                {\n                    match a_name.cmp(b_name) {\n                        std::cmp::Ordering::Equal => a.email.cmp(&b.email),\n                        ord => ord,\n                    }\n                } else {\n                    a.email.cmp(&b.email)\n                }\n            });\n\n            let field = ItipField {\n                name: ICalendarProperty::Attendee,\n                value: ItipValue::Participants(attendees),\n            };\n\n            if !skip_fields.contains(&field) {\n                fields.push(field);\n            }\n        }\n\n        fields\n    }\n}\n"
  },
  {
    "path": "crates/http/Cargo.toml",
    "content": "[package]\nname = \"http\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nstore = { path = \"../store\" }\ncommon = { path =  \"../common\" }\nutils = { path =  \"../utils\" }\ntrc = { path = \"../trc\" }\nemail = { path = \"../email\" }\nsmtp = { path = \"../smtp\" }\njmap = { path = \"../jmap\" }\ndav = { path = \"../dav\" }\ngroupware = { path = \"../groupware\" }\nspam-filter = { path = \"../spam-filter\" }\nhttp_proto = { path = \"../http-proto\" }\njmap_proto = { path = \"../jmap-proto\" }\ntypes = { path = \"../types\" }\ndirectory = { path =  \"../directory\" }\nservices = { path =  \"../services\" }\nsmtp-proto = { version = \"0.2\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] } \nmail-builder = { version = \"0.4\" }\nmail-auth = { version = \"0.7.1\", features = [\"generate\"] }\nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\ntokio = { version = \"1.47\", features = [\"rt\"] }\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\nhyper-util = { version = \"0.1.1\", features = [\"tokio\"] }\nhttp-body-util = \"0.1.0\"\nasync-stream = \"0.3.5\"\nquick-xml = \"0.38\"\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\nx509-parser = \"0.18\"\nchrono = \"0.4\"\nbase64 = \"0.22\"\npkcs8 = { version = \"0.10.2\", features = [\"alloc\", \"std\"] }\nrsa = \"0.9.2\"\nsha1 = \"0.10\"\nsha2 = \"0.10\"\nrev_lines = \"0.3.0\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\nform-data = { version = \"0.6.0\", features = [\"sync\"], default-features = false }\nmime = \"0.3.17\"\ncompact_str = \"0.9.0\"\n\n[dev-dependencies]\n\n[features]\ntest_mode = []\nenterprise = []\n"
  },
  {
    "path": "crates/http/src/auth/authenticate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::auth::AccessToken;\nuse common::{HttpAuthCache, Server, auth::AuthRequest, listener::limiter::InFlight};\nuse http_proto::{HttpRequest, HttpSessionData};\nuse hyper::header;\nuse mail_parser::decoders::base64::base64_decode;\nuse mail_send::Credentials;\nuse std::future::Future;\nuse std::sync::Arc;\nuse std::time::{Duration, Instant};\n\npub trait Authenticator: Sync + Send {\n    fn authenticate_headers(\n        &self,\n        req: &HttpRequest,\n        session: &HttpSessionData,\n        allow_api_access: bool,\n    ) -> impl Future<Output = trc::Result<(Option<InFlight>, Arc<AccessToken>)>> + Send;\n}\n\nimpl Authenticator for Server {\n    async fn authenticate_headers(\n        &self,\n        req: &HttpRequest,\n        session: &HttpSessionData,\n        allow_api_access: bool,\n    ) -> trc::Result<(Option<InFlight>, Arc<AccessToken>)> {\n        if let Some((mechanism, token)) = req.authorization() {\n            // Check if the credentials are cached\n            if let Some(http_cache) = self.inner.cache.http_auth.get(token) {\n                // Make sure the revision is still valid\n                if http_cache.expires <= Instant::now() {\n                    let access_token = self.get_access_token(http_cache.account_id).await?;\n                    if access_token.revision == http_cache.revision {\n                        // Enforce authenticated rate limit\n                        return self\n                            .is_http_authenticated_request_allowed(&access_token)\n                            .await\n                            .map(|in_flight| (in_flight, access_token));\n                    }\n                }\n\n                // If the revision is not valid, remove the cached credentials\n                self.inner.cache.http_auth.remove(token);\n            }\n\n            let credentials = if mechanism.eq_ignore_ascii_case(\"basic\") {\n                // Decode the base64 encoded credentials\n                decode_plain_auth(token).ok_or_else(|| {\n                    trc::AuthEvent::Error\n                        .into_err()\n                        .details(\"Failed to decode Basic auth request.\")\n                        .id(token.to_string())\n                        .caused_by(trc::location!())\n                })?\n            } else if mechanism.eq_ignore_ascii_case(\"bearer\") {\n                // Enforce anonymous rate limit\n                self.is_http_anonymous_request_allowed(&session.remote_ip)\n                    .await?;\n\n                decode_bearer_token(token, allow_api_access).ok_or_else(|| {\n                    trc::AuthEvent::Error\n                        .into_err()\n                        .details(\"Failed to decode Bearer token.\")\n                        .id(token.to_string())\n                        .caused_by(trc::location!())\n                })?\n            } else {\n                // Enforce anonymous rate limit\n                self.is_http_anonymous_request_allowed(&session.remote_ip)\n                    .await?;\n\n                return Err(trc::AuthEvent::Error\n                    .into_err()\n                    .reason(\"Unsupported authentication mechanism.\")\n                    .details(token.to_string())\n                    .caused_by(trc::location!()));\n            };\n\n            // Authenticate\n            let access_token = self\n                .authenticate(\n                    &AuthRequest::from_credentials(\n                        credentials,\n                        session.session_id,\n                        session.remote_ip,\n                    )\n                    .with_api_access(allow_api_access),\n                )\n                .await?;\n\n            // Cache credentials\n            self.inner.cache.http_auth.insert(\n                token.to_string(),\n                HttpAuthCache {\n                    account_id: access_token.primary_id(),\n                    revision: access_token.revision,\n                    expires: Instant::now()\n                        + Duration::from_secs(self.core.oauth.oauth_expiry_token),\n                },\n            );\n\n            // Enforce authenticated rate limit\n            self.is_http_authenticated_request_allowed(&access_token)\n                .await\n                .map(|in_flight| (in_flight, access_token))\n        } else {\n            // Enforce anonymous rate limit\n            self.is_http_anonymous_request_allowed(&session.remote_ip)\n                .await?;\n\n            Err(trc::AuthEvent::Failed\n                .into_err()\n                .details(\"Missing Authorization header.\")\n                .caused_by(trc::location!()))\n        }\n    }\n}\n\npub trait HttpHeaders {\n    fn authorization(&self) -> Option<(&str, &str)>;\n    fn authorization_basic(&self) -> Option<&str>;\n}\n\nimpl HttpHeaders for HttpRequest {\n    fn authorization(&self) -> Option<(&str, &str)> {\n        self.headers()\n            .get(header::AUTHORIZATION)\n            .and_then(|h| h.to_str().ok())\n            .and_then(|h| h.split_once(' ').map(|(l, t)| (l, t.trim())))\n    }\n\n    fn authorization_basic(&self) -> Option<&str> {\n        self.authorization().and_then(|(l, t)| {\n            if l.eq_ignore_ascii_case(\"basic\") {\n                Some(t)\n            } else {\n                None\n            }\n        })\n    }\n}\n\nfn decode_plain_auth(token: &str) -> Option<Credentials<String>> {\n    base64_decode(token.as_bytes())\n        .and_then(|token| String::from_utf8(token).ok())\n        .and_then(|token| {\n            token\n                .split_once(':')\n                .map(|(login, secret)| Credentials::Plain {\n                    username: login.trim().to_lowercase(),\n                    secret: secret.to_string(),\n                })\n        })\n}\n\nfn decode_bearer_token(token: &str, allow_api_access: bool) -> Option<Credentials<String>> {\n    if allow_api_access && let Some(token) = token.strip_prefix(\"api_\").and_then(decode_plain_auth)\n    {\n        return Some(token);\n    }\n\n    Some(Credentials::OAuthBearer {\n        token: token.to_string(),\n    })\n}\n"
  },
  {
    "path": "crates/http/src/auth/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod authenticate;\npub mod oauth;\n"
  },
  {
    "path": "crates/http/src/auth/oauth/auth.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::auth::oauth::OAuthStatus;\nuse common::{\n    KV_OAUTH, Server,\n    auth::{\n        AccessToken,\n        oauth::{CLIENT_ID_MAX_LEN, DEVICE_CODE_LEN, USER_CODE_ALPHABET, USER_CODE_LEN},\n    },\n};\nuse http_proto::*;\nuse serde::Deserialize;\nuse serde_json::json;\nuse std::future::Future;\nuse std::sync::Arc;\nuse store::{\n    Serialize,\n    dispatch::lookup::KeyValue,\n    write::{Archive, Archiver},\n};\nuse store::{\n    rand::{\n        Rng,\n        distr::{Alphanumeric, StandardUniform},\n        rng,\n    },\n    write::AlignedBytes,\n};\nuse trc::AddContext;\n\nuse super::{DeviceAuthResponse, FormData, MAX_POST_LEN, OAuthCode, OAuthCodeRequest};\n\n#[derive(Debug, serde::Serialize, Deserialize)]\npub struct OAuthMetadata {\n    pub issuer: String,\n    pub token_endpoint: String,\n    pub authorization_endpoint: String,\n    pub device_authorization_endpoint: String,\n    pub registration_endpoint: String,\n    pub introspection_endpoint: String,\n    pub grant_types_supported: Vec<String>,\n    pub response_types_supported: Vec<String>,\n    pub scopes_supported: Vec<String>,\n}\n\npub trait OAuthApiHandler: Sync + Send {\n    fn handle_oauth_api_request(\n        &self,\n        access_token: Arc<AccessToken>,\n        body: Option<Vec<u8>>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_device_auth(\n        &self,\n        req: &mut HttpRequest,\n        session: HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_oauth_metadata(\n        &self,\n        req: HttpRequest,\n        session: HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl OAuthApiHandler for Server {\n    async fn handle_oauth_api_request(\n        &self,\n        access_token: Arc<AccessToken>,\n        body: Option<Vec<u8>>,\n    ) -> trc::Result<HttpResponse> {\n        let request =\n            serde_json::from_slice::<OAuthCodeRequest>(body.as_deref().unwrap_or_default())\n                .map_err(|err| {\n                    trc::EventType::Resource(trc::ResourceEvent::BadParameters).from_json_error(err)\n                })?;\n\n        let response = match request {\n            OAuthCodeRequest::Code {\n                client_id,\n                redirect_uri,\n                nonce,\n            } => {\n                // Validate clientId\n                if client_id.len() > CLIENT_ID_MAX_LEN {\n                    return Err(trc::ManageEvent::Error\n                        .into_err()\n                        .details(\"Client ID is invalid.\"));\n                } else if redirect_uri\n                    .as_ref()\n                    .is_some_and(|uri| uri.starts_with(\"http://\"))\n                {\n                    return Err(trc::ManageEvent::Error\n                        .into_err()\n                        .details(\"Redirect URI must be HTTPS.\"));\n                }\n\n                // Generate client code\n                let client_code = rng()\n                    .sample_iter(Alphanumeric)\n                    .take(DEVICE_CODE_LEN)\n                    .map(char::from)\n                    .collect::<String>();\n\n                // Serialize OAuth code\n                let value = Archiver::new(OAuthCode {\n                    status: OAuthStatus::Authorized,\n                    account_id: access_token.primary_id(),\n                    client_id,\n                    nonce,\n                    params: redirect_uri.unwrap_or_default(),\n                })\n                .untrusted()\n                .serialize()\n                .caused_by(trc::location!())?;\n\n                // Insert client code\n                self.core\n                    .storage\n                    .lookup\n                    .key_set(\n                        KeyValue::with_prefix(KV_OAUTH, client_code.as_bytes(), value)\n                            .expires(self.core.oauth.oauth_expiry_auth_code),\n                    )\n                    .await?;\n\n                #[cfg(not(feature = \"enterprise\"))]\n                let is_enterprise = false;\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                let is_enterprise = self.core.is_enterprise_edition();\n                // SPDX-SnippetEnd\n\n                json!({\n                    \"data\": {\n                        \"code\": client_code,\n                        \"permissions\": access_token.permissions(),\n                        \"version\": env!(\"CARGO_PKG_VERSION\"),\n                        \"isEnterprise\": is_enterprise,\n                    },\n                })\n            }\n            OAuthCodeRequest::Device { code } => {\n                let mut success = false;\n\n                // Obtain code\n                if let Some(auth_code_) = self\n                    .core\n                    .storage\n                    .lookup\n                    .key_get::<Archive<AlignedBytes>>(KeyValue::<()>::build_key(\n                        KV_OAUTH,\n                        code.as_bytes(),\n                    ))\n                    .await?\n                {\n                    let oauth = auth_code_\n                        .unarchive::<OAuthCode>()\n                        .caused_by(trc::location!())?;\n                    if oauth.status == OAuthStatus::Pending {\n                        let new_oauth_code = OAuthCode {\n                            status: OAuthStatus::Authorized,\n                            account_id: access_token.primary_id(),\n                            client_id: oauth.client_id.to_string(),\n                            nonce: oauth.nonce.as_ref().map(|s| s.to_string()),\n                            params: Default::default(),\n                        };\n                        success = true;\n\n                        // Delete issued user code\n                        self.core\n                            .storage\n                            .lookup\n                            .key_delete(KeyValue::<()>::build_key(KV_OAUTH, code.as_bytes()))\n                            .await?;\n\n                        // Update device code status\n                        self.core\n                            .storage\n                            .lookup\n                            .key_set(\n                                KeyValue::with_prefix(\n                                    KV_OAUTH,\n                                    oauth.params.as_bytes(),\n                                    Archiver::new(new_oauth_code)\n                                        .untrusted()\n                                        .serialize()\n                                        .caused_by(trc::location!())?,\n                                )\n                                .expires(self.core.oauth.oauth_expiry_auth_code),\n                            )\n                            .await?;\n                    }\n                }\n\n                json!({\n                    \"data\": success,\n                })\n            }\n        };\n\n        Ok(JsonResponse::new(response).no_cache().into_http_response())\n    }\n\n    async fn handle_device_auth(\n        &self,\n        req: &mut HttpRequest,\n        session: HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        // Parse form\n        let mut form_data = FormData::from_request(req, MAX_POST_LEN, session.session_id).await?;\n        let client_id = form_data\n            .remove(\"client_id\")\n            .filter(|client_id| client_id.len() <= CLIENT_ID_MAX_LEN)\n            .ok_or_else(|| {\n                trc::ResourceEvent::BadParameters\n                    .into_err()\n                    .details(\"Client ID is missing.\")\n            })?;\n        let nonce = form_data.remove(\"nonce\");\n\n        // Generate device code\n        let device_code = rng()\n            .sample_iter(Alphanumeric)\n            .take(DEVICE_CODE_LEN)\n            .map(char::from)\n            .collect::<String>();\n\n        // Generate user code\n        let mut user_code = String::with_capacity(USER_CODE_LEN + 1);\n        for (pos, ch) in rng()\n            .sample_iter(StandardUniform)\n            .take(USER_CODE_LEN)\n            .map(|v: u64| char::from(USER_CODE_ALPHABET[v as usize % USER_CODE_ALPHABET.len()]))\n            .enumerate()\n        {\n            if pos == USER_CODE_LEN / 2 {\n                user_code.push('-');\n            }\n            user_code.push(ch);\n        }\n\n        // Add OAuth status\n        let oauth_code = Archiver::new(OAuthCode {\n            status: OAuthStatus::Pending,\n            account_id: u32::MAX,\n            client_id,\n            nonce,\n            params: device_code.clone(),\n        })\n        .untrusted()\n        .serialize()\n        .caused_by(trc::location!())?;\n\n        // Insert device code\n        self.core\n            .storage\n            .lookup\n            .key_set(\n                KeyValue::with_prefix(KV_OAUTH, device_code.as_bytes(), oauth_code.clone())\n                    .expires(self.core.oauth.oauth_expiry_user_code),\n            )\n            .await?;\n\n        // Insert user code\n        self.core\n            .storage\n            .lookup\n            .key_set(\n                KeyValue::with_prefix(KV_OAUTH, user_code.as_bytes(), oauth_code)\n                    .expires(self.core.oauth.oauth_expiry_user_code),\n            )\n            .await?;\n\n        // Build response\n        let base_url = HttpContext::new(&session, req)\n            .resolve_response_url(self)\n            .await;\n        Ok(JsonResponse::new(DeviceAuthResponse {\n            verification_uri: format!(\"{base_url}/authorize\"),\n            verification_uri_complete: format!(\"{base_url}/authorize/?code={user_code}\"),\n            device_code,\n            user_code,\n            expires_in: self.core.oauth.oauth_expiry_user_code,\n            interval: 5,\n        })\n        .no_cache()\n        .into_http_response())\n    }\n\n    async fn handle_oauth_metadata(\n        &self,\n        req: HttpRequest,\n        session: HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        let base_url = HttpContext::new(&session, &req)\n            .resolve_response_url(self)\n            .await\n            .to_string();\n\n        Ok(JsonResponse::new(OAuthMetadata {\n            authorization_endpoint: format!(\"{base_url}/authorize/code\",),\n            token_endpoint: format!(\"{base_url}/auth/token\"),\n            device_authorization_endpoint: format!(\"{base_url}/auth/device\"),\n            introspection_endpoint: format!(\"{base_url}/auth/introspect\"),\n            registration_endpoint: format!(\"{base_url}/auth/register\"),\n            grant_types_supported: vec![\n                \"authorization_code\".to_string(),\n                \"implicit\".to_string(),\n                \"urn:ietf:params:oauth:grant-type:device_code\".to_string(),\n            ],\n            response_types_supported: vec![\n                \"code\".to_string(),\n                \"id_token\".to_string(),\n                \"code token\".to_string(),\n                \"id_token token\".to_string(),\n            ],\n            scopes_supported: vec![\n                \"openid\".to_string(),\n                \"offline_access\".to_string(),\n                \"urn:ietf:params:jmap:core\".to_string(),\n                \"urn:ietf:params:jmap:mail\".to_string(),\n                \"urn:ietf:params:jmap:submission\".to_string(),\n                \"urn:ietf:params:jmap:vacationresponse\".to_string(),\n            ],\n            issuer: base_url,\n        })\n        .into_http_response())\n    }\n}\n"
  },
  {
    "path": "crates/http/src/auth/oauth/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse http_proto::{HttpRequest, request::fetch_body};\nuse hyper::header::CONTENT_TYPE;\nuse serde::{Deserialize, Serialize};\nuse utils::map::vec_map::VecMap;\n\npub mod auth;\npub mod openid;\npub mod registration;\npub mod token;\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Copy,\n    Clone,\n    Debug,\n    Serialize,\n    Deserialize,\n    PartialEq,\n    Eq,\n)]\n#[rkyv(compare(PartialEq))]\npub enum OAuthStatus {\n    Authorized,\n    TokenIssued,\n    Pending,\n}\n\nconst MAX_POST_LEN: usize = 2048;\n\npub struct OAuth {\n    pub key: String,\n    pub expiry_user_code: u64,\n    pub expiry_auth_code: u64,\n    pub expiry_token: u64,\n    pub expiry_refresh_token: u64,\n    pub expiry_refresh_token_renew: u64,\n    pub max_auth_attempts: u32,\n    pub metadata: String,\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct OAuthCode {\n    pub status: OAuthStatus,\n    pub account_id: u32,\n    pub client_id: String,\n    pub nonce: Option<String>,\n    pub params: String,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct DeviceAuthGet {\n    code: Option<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct DeviceAuthPost {\n    code: Option<String>,\n    email: Option<String>,\n    password: Option<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct DeviceAuthRequest {\n    client_id: String,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct DeviceAuthResponse {\n    pub device_code: String,\n    pub user_code: String,\n    pub verification_uri: String,\n    pub verification_uri_complete: String,\n    pub expires_in: u64,\n    pub interval: u64,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct CodeAuthRequest {\n    response_type: String,\n    client_id: String,\n    redirect_uri: String,\n    scope: Option<String>,\n    state: Option<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct CodeAuthForm {\n    code: String,\n    email: Option<String>,\n    password: Option<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct TokenRequest {\n    pub grant_type: String,\n    pub code: Option<String>,\n    pub device_code: Option<String>,\n    pub client_id: Option<String>,\n    pub refresh_token: Option<String>,\n    pub redirect_uri: Option<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize, PartialEq, Eq)]\n#[serde(untagged)]\npub enum TokenResponse {\n    Granted(OAuthResponse),\n    Error { error: ErrorType },\n}\n\n#[derive(Debug, Serialize, Deserialize, PartialEq, Eq)]\npub struct OAuthResponse {\n    pub access_token: String,\n    pub token_type: String,\n    pub expires_in: u64,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub refresh_token: Option<String>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub scope: Option<String>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub id_token: Option<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize, PartialEq, Eq)]\npub enum ErrorType {\n    #[serde(rename = \"invalid_grant\")]\n    InvalidGrant,\n    #[serde(rename = \"invalid_client\")]\n    InvalidClient,\n    #[serde(rename = \"invalid_scope\")]\n    InvalidScope,\n    #[serde(rename = \"invalid_request\")]\n    InvalidRequest,\n    #[serde(rename = \"unauthorized_client\")]\n    UnauthorizedClient,\n    #[serde(rename = \"unsupported_grant_type\")]\n    UnsupportedGrantType,\n    #[serde(rename = \"authorization_pending\")]\n    AuthorizationPending,\n    #[serde(rename = \"slow_down\")]\n    SlowDown,\n    #[serde(rename = \"access_denied\")]\n    AccessDenied,\n    #[serde(rename = \"expired_token\")]\n    ExpiredToken,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum OAuthCodeRequest {\n    Code {\n        client_id: String,\n        redirect_uri: Option<String>,\n        #[serde(default)]\n        nonce: Option<String>,\n    },\n    Device {\n        code: String,\n    },\n}\n\nimpl TokenResponse {\n    pub fn error(error: ErrorType) -> Self {\n        TokenResponse::Error { error }\n    }\n\n    pub fn is_error(&self) -> bool {\n        matches!(self, TokenResponse::Error { .. })\n    }\n}\n\n#[derive(Debug)]\npub struct FormData {\n    fields: VecMap<String, String>,\n}\n\nimpl FormData {\n    pub async fn from_request(\n        req: &mut HttpRequest,\n        max_len: usize,\n        session_id: u64,\n    ) -> trc::Result<Self> {\n        match (\n            req.headers()\n                .get(CONTENT_TYPE)\n                .and_then(|h| h.to_str().ok())\n                .and_then(|val| val.parse::<mime::Mime>().ok()),\n            fetch_body(req, max_len, session_id).await,\n        ) {\n            (Some(content_type), Some(body)) => {\n                let mut fields = VecMap::new();\n                if let Some(boundary) = content_type.get_param(mime::BOUNDARY) {\n                    for mut field in\n                        form_data::FormData::new(&body[..], boundary.as_str()).flatten()\n                    {\n                        let value = String::from_utf8_lossy(&field.bytes().unwrap_or_default())\n                            .into_owned();\n                        fields.append(field.name, value);\n                    }\n                } else {\n                    for (key, value) in http_proto::form_urlencoded::parse(&body) {\n                        fields.append(key.into_owned(), value.into_owned());\n                    }\n                }\n                Ok(FormData { fields })\n            }\n            _ => Err(trc::ResourceEvent::BadParameters\n                .into_err()\n                .details(\"Invalid post request\")),\n        }\n    }\n\n    pub fn get(&self, key: &str) -> Option<&str> {\n        self.fields.get(key).map(|v| v.as_str())\n    }\n\n    pub fn remove(&mut self, key: &str) -> Option<String> {\n        self.fields.remove(key)\n    }\n\n    pub fn has_field(&self, key: &str) -> bool {\n        self.fields.get(key).is_some_and(|v| !v.is_empty())\n    }\n\n    pub fn fields(&self) -> impl Iterator<Item = (&String, &String)> {\n        self.fields.iter()\n    }\n}\n"
  },
  {
    "path": "crates/http/src/auth/oauth/openid.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::{\n    Server,\n    auth::{AccessToken, oauth::oidc::Userinfo},\n};\nuse serde::{Deserialize, Serialize};\n\nuse http_proto::*;\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct OpenIdMetadata {\n    pub issuer: String,\n    pub authorization_endpoint: String,\n    pub token_endpoint: String,\n    pub userinfo_endpoint: String,\n    pub jwks_uri: String,\n    pub registration_endpoint: String,\n    pub device_authorization_endpoint: String,\n    pub scopes_supported: Vec<String>,\n    pub response_types_supported: Vec<String>,\n    pub subject_types_supported: Vec<String>,\n    pub grant_types_supported: Vec<String>,\n    pub id_token_signing_alg_values_supported: Vec<String>,\n    pub claims_supported: Vec<String>,\n}\n\npub trait OpenIdHandler: Sync + Send {\n    fn handle_userinfo_request(\n        &self,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_oidc_metadata(\n        &self,\n        req: HttpRequest,\n        session: HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl OpenIdHandler for Server {\n    async fn handle_userinfo_request(\n        &self,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        Ok(JsonResponse::new(Userinfo {\n            sub: Some(access_token.primary_id.to_string()),\n            name: access_token.description.clone(),\n            preferred_username: Some(access_token.name.clone()),\n            email: access_token.emails.first().cloned(),\n            email_verified: !access_token.emails.is_empty(),\n            ..Default::default()\n        })\n        .no_cache()\n        .into_http_response())\n    }\n\n    async fn handle_oidc_metadata(\n        &self,\n        req: HttpRequest,\n        session: HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        let base_url = HttpContext::new(&session, &req)\n            .resolve_response_url(self)\n            .await;\n\n        Ok(JsonResponse::new(OpenIdMetadata {\n            authorization_endpoint: format!(\"{base_url}/authorize/code\",),\n            token_endpoint: format!(\"{base_url}/auth/token\"),\n            userinfo_endpoint: format!(\"{base_url}/auth/userinfo\"),\n            jwks_uri: format!(\"{base_url}/auth/jwks.json\"),\n            registration_endpoint: format!(\"{base_url}/auth/register\"),\n            device_authorization_endpoint: format!(\"{base_url}/auth/device\"),\n            response_types_supported: vec![\n                \"code\".into(),\n                \"id_token\".into(),\n                \"id_token token\".into(),\n            ],\n            grant_types_supported: vec![\n                \"authorization_code\".into(),\n                \"implicit\".into(),\n                \"urn:ietf:params:oauth:grant-type:device_code\".into(),\n            ],\n            scopes_supported: vec![\"openid\".into(), \"offline_access\".into()],\n            subject_types_supported: vec![\"public\".into()],\n            id_token_signing_alg_values_supported: vec![\n                \"RS256\".into(),\n                \"RS384\".into(),\n                \"RS512\".into(),\n                \"ES256\".into(),\n                \"ES384\".into(),\n                \"PS256\".into(),\n                \"PS384\".into(),\n                \"PS512\".into(),\n                \"HS256\".into(),\n                \"HS384\".into(),\n                \"HS512\".into(),\n            ],\n            claims_supported: vec![\n                \"sub\".into(),\n                \"name\".into(),\n                \"preferred_username\".into(),\n                \"email\".into(),\n                \"email_verified\".into(),\n            ],\n            issuer: base_url,\n        })\n        .into_http_response())\n    }\n}\n"
  },
  {
    "path": "crates/http/src/auth/oauth/registration.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::{\n    Server,\n    auth::oauth::registration::{ClientRegistrationRequest, ClientRegistrationResponse},\n};\n\nuse directory::{\n    Permission, QueryParams, Type,\n    backend::internal::{\n        PrincipalField, PrincipalSet, lookup::DirectoryStore, manage::ManageDirectory,\n    },\n};\nuse store::rand::{Rng, distr::Alphanumeric, rng};\nuse trc::{AddContext, AuthEvent};\n\nuse crate::auth::authenticate::Authenticator;\nuse http_proto::{request::fetch_body, *};\n\nuse super::ErrorType;\n\npub trait ClientRegistrationHandler: Sync + Send {\n    fn handle_oauth_registration_request(\n        &self,\n        req: &mut HttpRequest,\n        session: HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn validate_client_registration(\n        &self,\n        client_id: &str,\n        redirect_uri: Option<&str>,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Option<ErrorType>>> + Send;\n}\nimpl ClientRegistrationHandler for Server {\n    async fn handle_oauth_registration_request(\n        &self,\n        req: &mut HttpRequest,\n        session: HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        if !self.core.oauth.allow_anonymous_client_registration {\n            // Authenticate request\n            let (_, access_token) = self.authenticate_headers(req, &session, true).await?;\n\n            // Validate permissions\n            access_token.assert_has_permission(Permission::OauthClientRegistration)?;\n        } else {\n            self.is_http_anonymous_request_allowed(&session.remote_ip)\n                .await?;\n        }\n\n        // Parse request\n        let body = fetch_body(req, 20 * 1024, session.session_id).await;\n        let request = serde_json::from_slice::<ClientRegistrationRequest>(\n            body.as_deref().unwrap_or_default(),\n        )\n        .map_err(|err| {\n            trc::EventType::Resource(trc::ResourceEvent::BadParameters).from_json_error(err)\n        })?;\n\n        // Generate client ID\n        let client_id = rng()\n            .sample_iter(Alphanumeric)\n            .take(20)\n            .map(|ch| char::from(ch.to_ascii_lowercase()))\n            .collect::<String>();\n        self.store()\n            .create_principal(\n                PrincipalSet::new(u32::MAX, Type::OauthClient)\n                    .with_field(PrincipalField::Name, client_id.clone())\n                    .with_field(PrincipalField::Urls, request.redirect_uris.clone())\n                    .with_opt_field(PrincipalField::Description, request.client_name.clone())\n                    .with_field(PrincipalField::Emails, request.contacts.clone())\n                    .with_opt_field(PrincipalField::Picture, request.logo_uri.clone()),\n                None,\n                None,\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        trc::event!(\n            Auth(AuthEvent::ClientRegistration),\n            Id = client_id.to_string(),\n            RemoteIp = session.remote_ip\n        );\n\n        Ok(JsonResponse::new(ClientRegistrationResponse {\n            client_id,\n            request,\n            ..Default::default()\n        })\n        .no_cache()\n        .into_http_response())\n    }\n\n    async fn validate_client_registration(\n        &self,\n        client_id: &str,\n        redirect_uri: Option<&str>,\n        account_id: u32,\n    ) -> trc::Result<Option<ErrorType>> {\n        if !self.core.oauth.require_client_authentication {\n            return Ok(None);\n        }\n\n        // Fetch client registration\n        let found_registration = if let Some(client) = self\n            .store()\n            .query(QueryParams::name(client_id).with_return_member_of(false))\n            .await\n            .caused_by(trc::location!())?\n            .filter(|p| p.typ() == Type::OauthClient)\n        {\n            if let Some(redirect_uri) = redirect_uri {\n                if client.urls().any(|uri| uri == redirect_uri) {\n                    return Ok(None);\n                }\n            } else {\n                // Device flow does not require a redirect URI\n\n                return Ok(None);\n            }\n\n            true\n        } else {\n            false\n        };\n\n        // Check if the account is allowed to override client registration\n        if self\n            .get_access_token(account_id)\n            .await\n            .caused_by(trc::location!())?\n            .has_permission(Permission::OauthClientOverride)\n        {\n            return Ok(None);\n        }\n\n        Ok(Some(if found_registration {\n            ErrorType::InvalidClient\n        } else {\n            ErrorType::InvalidRequest\n        }))\n    }\n}\n"
  },
  {
    "path": "crates/http/src/auth/oauth/token.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ArchivedOAuthStatus, ErrorType, FormData, MAX_POST_LEN, OAuthCode, OAuthResponse, OAuthStatus,\n    TokenResponse, registration::ClientRegistrationHandler,\n};\nuse common::{\n    KV_OAUTH, Server,\n    auth::{\n        AccessToken,\n        oauth::{GrantType, oidc::StandardClaims},\n    },\n};\nuse http_proto::*;\nuse hyper::StatusCode;\nuse std::future::Future;\nuse store::{\n    dispatch::lookup::KeyValue,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\n\npub trait TokenHandler: Sync + Send {\n    fn handle_token_request(\n        &self,\n        req: &mut HttpRequest,\n        session: HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_token_introspect(\n        &self,\n        req: &mut HttpRequest,\n        access_token: &AccessToken,\n        session_id: u64,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn issue_token(\n        &self,\n        account_id: u32,\n        client_id: &str,\n        issuer: String,\n        nonce: Option<String>,\n        with_refresh_token: bool,\n        with_id_token: bool,\n    ) -> impl Future<Output = trc::Result<OAuthResponse>> + Send;\n}\n\nimpl TokenHandler for Server {\n    // Token endpoint\n    async fn handle_token_request(\n        &self,\n        req: &mut HttpRequest,\n        session: HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        // Parse form\n        let params = FormData::from_request(req, MAX_POST_LEN, session.session_id).await?;\n        let grant_type = params.get(\"grant_type\").unwrap_or_default();\n\n        let mut response = TokenResponse::error(ErrorType::InvalidGrant);\n\n        let issuer = HttpContext::new(&session, req)\n            .resolve_response_url(self)\n            .await;\n\n        if grant_type.eq_ignore_ascii_case(\"authorization_code\") {\n            response = if let (Some(code), Some(client_id), Some(redirect_uri)) = (\n                params.get(\"code\"),\n                params.get(\"client_id\"),\n                params.get(\"redirect_uri\"),\n            ) {\n                // Obtain code\n                match self\n                    .core\n                    .storage\n                    .lookup\n                    .key_get::<Archive<AlignedBytes>>(KeyValue::<()>::build_key(\n                        KV_OAUTH,\n                        code.as_bytes(),\n                    ))\n                    .await?\n                {\n                    Some(auth_code_) => {\n                        let oauth = auth_code_\n                            .unarchive::<OAuthCode>()\n                            .caused_by(trc::location!())?;\n                        if client_id != oauth.client_id || redirect_uri != oauth.params {\n                            TokenResponse::error(ErrorType::InvalidClient)\n                        } else if oauth.status == OAuthStatus::Authorized {\n                            // Validate client id\n                            if let Some(error) = self\n                                .validate_client_registration(\n                                    client_id,\n                                    redirect_uri.into(),\n                                    oauth.account_id.into(),\n                                )\n                                .await?\n                            {\n                                TokenResponse::error(error)\n                            } else {\n                                // Mark this token as issued\n                                self.core\n                                    .storage\n                                    .lookup\n                                    .key_delete(KeyValue::<()>::build_key(\n                                        KV_OAUTH,\n                                        code.as_bytes(),\n                                    ))\n                                    .await?;\n\n                                // Issue token\n                                self.issue_token(\n                                    oauth.account_id.into(),\n                                    &oauth.client_id,\n                                    issuer,\n                                    oauth.nonce.as_ref().map(|s| s.as_str().into()),\n                                    true,\n                                    true,\n                                )\n                                .await\n                                .map(TokenResponse::Granted)\n                                .map_err(|err| {\n                                    trc::AuthEvent::Error\n                                        .into_err()\n                                        .details(err)\n                                        .caused_by(trc::location!())\n                                })?\n                            }\n                        } else {\n                            TokenResponse::error(ErrorType::InvalidGrant)\n                        }\n                    }\n                    None => TokenResponse::error(ErrorType::AccessDenied),\n                }\n            } else {\n                TokenResponse::error(ErrorType::InvalidClient)\n            };\n        } else if grant_type.eq_ignore_ascii_case(\"urn:ietf:params:oauth:grant-type:device_code\") {\n            response = TokenResponse::error(ErrorType::ExpiredToken);\n\n            if let (Some(device_code), Some(client_id)) =\n                (params.get(\"device_code\"), params.get(\"client_id\"))\n            {\n                // Obtain code\n                if let Some(auth_code_) = self\n                    .core\n                    .storage\n                    .lookup\n                    .key_get::<Archive<AlignedBytes>>(KeyValue::<()>::build_key(\n                        KV_OAUTH,\n                        device_code.as_bytes(),\n                    ))\n                    .await?\n                {\n                    let oauth = auth_code_\n                        .unarchive::<OAuthCode>()\n                        .caused_by(trc::location!())?;\n                    response = if oauth.client_id != client_id {\n                        TokenResponse::error(ErrorType::InvalidClient)\n                    } else {\n                        match oauth.status {\n                            ArchivedOAuthStatus::Authorized => {\n                                if let Some(error) = self\n                                    .validate_client_registration(\n                                        client_id,\n                                        None,\n                                        oauth.account_id.into(),\n                                    )\n                                    .await?\n                                {\n                                    TokenResponse::error(error)\n                                } else {\n                                    // Mark this token as issued\n                                    self.core\n                                        .storage\n                                        .lookup\n                                        .key_delete(KeyValue::<()>::build_key(\n                                            KV_OAUTH,\n                                            device_code.as_bytes(),\n                                        ))\n                                        .await?;\n\n                                    // Issue token\n                                    self.issue_token(\n                                        oauth.account_id.into(),\n                                        &oauth.client_id,\n                                        issuer,\n                                        oauth.nonce.as_ref().map(|s| s.as_str().into()),\n                                        true,\n                                        true,\n                                    )\n                                    .await\n                                    .map(TokenResponse::Granted)\n                                    .map_err(|err| {\n                                        trc::AuthEvent::Error\n                                            .into_err()\n                                            .details(err)\n                                            .caused_by(trc::location!())\n                                    })?\n                                }\n                            }\n                            ArchivedOAuthStatus::Pending => {\n                                TokenResponse::error(ErrorType::AuthorizationPending)\n                            }\n                            ArchivedOAuthStatus::TokenIssued => {\n                                TokenResponse::error(ErrorType::ExpiredToken)\n                            }\n                        }\n                    };\n                }\n            }\n        } else if grant_type.eq_ignore_ascii_case(\"refresh_token\") {\n            if let Some(refresh_token) = params.get(\"refresh_token\") {\n                response = match self\n                    .validate_access_token(GrantType::RefreshToken.into(), refresh_token)\n                    .await\n                {\n                    Ok(token_info) => self\n                        .issue_token(\n                            token_info.account_id,\n                            &token_info.client_id,\n                            issuer,\n                            None,\n                            token_info.expires_in\n                                <= self.core.oauth.oauth_expiry_refresh_token_renew,\n                            false,\n                        )\n                        .await\n                        .map(TokenResponse::Granted)\n                        .map_err(|err| {\n                            trc::AuthEvent::Error\n                                .into_err()\n                                .details(err)\n                                .caused_by(trc::location!())\n                        })?,\n                    Err(err) => {\n                        trc::error!(\n                            err.caused_by(trc::location!())\n                                .details(\"Failed to validate refresh token\")\n                                .span_id(session.session_id)\n                        );\n                        TokenResponse::error(ErrorType::InvalidGrant)\n                    }\n                };\n            } else {\n                response = TokenResponse::error(ErrorType::InvalidRequest);\n            }\n        }\n\n        Ok(JsonResponse::with_status(\n            if response.is_error() {\n                StatusCode::BAD_REQUEST\n            } else {\n                StatusCode::OK\n            },\n            response,\n        )\n        .into_http_response())\n    }\n\n    async fn handle_token_introspect(\n        &self,\n        req: &mut HttpRequest,\n        access_token: &AccessToken,\n        session_id: u64,\n    ) -> trc::Result<HttpResponse> {\n        // Parse token\n        let token = FormData::from_request(req, 1024, session_id)\n            .await?\n            .remove(\"token\")\n            .ok_or_else(|| {\n                trc::ResourceEvent::BadParameters\n                    .into_err()\n                    .details(\"Client ID is missing.\")\n            })?;\n\n        self.introspect_access_token(&token, access_token)\n            .await\n            .map(|response| JsonResponse::new(response).no_cache().into_http_response())\n    }\n\n    async fn issue_token(\n        &self,\n        account_id: u32,\n        client_id: &str,\n        issuer: String,\n        nonce: Option<String>,\n        with_refresh_token: bool,\n        with_id_token: bool,\n    ) -> trc::Result<OAuthResponse> {\n        Ok(OAuthResponse {\n            access_token: self\n                .encode_access_token(\n                    GrantType::AccessToken,\n                    account_id,\n                    client_id,\n                    self.core.oauth.oauth_expiry_token,\n                )\n                .await?,\n            token_type: \"bearer\".to_string(),\n            expires_in: self.core.oauth.oauth_expiry_token,\n            refresh_token: if with_refresh_token {\n                self.encode_access_token(\n                    GrantType::RefreshToken,\n                    account_id,\n                    client_id,\n                    self.core.oauth.oauth_expiry_refresh_token,\n                )\n                .await?\n                .into()\n            } else {\n                None\n            },\n            id_token: if with_id_token {\n                // Obtain access token\n                let access_token = self\n                    .get_access_token(account_id)\n                    .await\n                    .caused_by(trc::location!())?;\n\n                match self.issue_id_token(\n                    account_id.to_string(),\n                    issuer,\n                    client_id,\n                    StandardClaims {\n                        nonce,\n                        preferred_username: access_token.name.clone().into(),\n                        email: access_token.emails.first().cloned(),\n                        description: access_token.description.clone(),\n                    },\n                ) {\n                    Ok(id_token) => Some(id_token),\n                    Err(err) => {\n                        trc::error!(err);\n                        None\n                    }\n                }\n            } else {\n                None\n            },\n            scope: None,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/http/src/autoconfig/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, manager::webadmin::Resource};\nuse directory::QueryParams;\nuse http_proto::*;\nuse quick_xml::Reader;\nuse quick_xml::events::Event;\nuse std::fmt::Write;\nuse std::future::Future;\nuse trc::AddContext;\nuse utils::url_params::UrlParams;\n\npub trait Autoconfig: Sync + Send {\n    fn handle_autoconfig_request(\n        &self,\n        req: &HttpRequest,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n    fn handle_autodiscover_request(\n        &self,\n        body: Option<Vec<u8>>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n    fn autoconfig_parameters<'x>(\n        &'x self,\n        emailaddress: &'x str,\n        fail_if_invalid: bool,\n    ) -> impl Future<Output = trc::Result<(String, String, &'x str)>> + Send;\n}\n\nimpl Autoconfig for Server {\n    async fn handle_autoconfig_request(&self, req: &HttpRequest) -> trc::Result<HttpResponse> {\n        // Obtain parameters\n        let params = UrlParams::new(req.uri().query());\n        let emailaddress = params\n            .get(\"emailaddress\")\n            .unwrap_or_default()\n            .to_lowercase();\n        let (account_name, server_name, domain) =\n            self.autoconfig_parameters(&emailaddress, false).await?;\n        let services = self.core.storage.config.get_services().await?;\n\n        // Build XML response\n        let mut config = String::with_capacity(1024);\n        config.push_str(\"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?>\\n\");\n        config.push_str(\"<clientConfig version=\\\"1.1\\\">\\n\");\n        let _ = writeln!(&mut config, \"\\t<emailProvider id=\\\"{domain}\\\">\");\n        let _ = writeln!(&mut config, \"\\t\\t<domain>{domain}</domain>\");\n        let _ = writeln!(&mut config, \"\\t\\t<displayName>{emailaddress}</displayName>\");\n        let _ = writeln!(\n            &mut config,\n            \"\\t\\t<displayShortName>{domain}</displayShortName>\"\n        );\n        for (protocol, port, is_tls) in services {\n            let tag = match protocol.as_str() {\n                \"imap\" | \"pop3\" => \"incomingServer\",\n                \"smtp\" if port != 25 => \"outgoingServer\",\n                _ => continue,\n            };\n            let _ = writeln!(&mut config, \"\\t\\t<{tag} type=\\\"{protocol}\\\">\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t<hostname>{server_name}</hostname>\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t<port>{port}</port>\");\n            let _ = writeln!(\n                &mut config,\n                \"\\t\\t\\t<socketType>{}</socketType>\",\n                if is_tls { \"SSL\" } else { \"STARTTLS\" }\n            );\n            let _ = writeln!(&mut config, \"\\t\\t\\t<username>{account_name}</username>\");\n            let _ = writeln!(\n                &mut config,\n                \"\\t\\t\\t<authentication>password-cleartext</authentication>\"\n            );\n            let _ = writeln!(&mut config, \"\\t\\t</{tag}>\");\n        }\n\n        config.push_str(\"\\t</emailProvider>\\n\");\n\n        for (tag, protocol, url) in [\n            (\"addressBook\", \"carddav\", \"card\"),\n            (\"calendar\", \"caldav\", \"cal\"),\n            (\"fileShare\", \"webdav\", \"file\"),\n        ] {\n            let _ = writeln!(&mut config, \"\\t<{tag} type=\\\"{protocol}\\\">\");\n            let _ = writeln!(&mut config, \"\\t\\t<username>{account_name}</username>\");\n            let _ = writeln!(\n                &mut config,\n                \"\\t\\t<authentication>http-basic</authentication>\"\n            );\n            let _ = writeln!(\n                &mut config,\n                \"\\t\\t<serverURL>https://{server_name}/dav/{url}</serverURL>\"\n            );\n            let _ = writeln!(&mut config, \"\\t</{tag}>\");\n        }\n\n        let _ = writeln!(\n            &mut config,\n            \"\\t<clientConfigUpdate url=\\\"https://autoconfig.{domain}/mail/config-v1.1.xml\\\"></clientConfigUpdate>\"\n        );\n        config.push_str(\"</clientConfig>\\n\");\n\n        Ok(\n            Resource::new(\"application/xml; charset=utf-8\", config.into_bytes())\n                .into_http_response(),\n        )\n    }\n\n    async fn handle_autodiscover_request(\n        &self,\n        body: Option<Vec<u8>>,\n    ) -> trc::Result<HttpResponse> {\n        // Obtain parameters\n        let emailaddress = parse_autodiscover_request(body.as_deref().unwrap_or_default())\n            .map_err(|err| {\n                trc::ResourceEvent::BadParameters\n                    .into_err()\n                    .details(\"Failed to parse autodiscover request\")\n                    .ctx(trc::Key::Reason, err)\n            })?;\n        let (account_name, server_name, _) =\n            self.autoconfig_parameters(&emailaddress, true).await?;\n        let services = self.core.storage.config.get_services().await?;\n\n        // Build XML response\n        let mut config = String::with_capacity(1024);\n        let _ = writeln!(&mut config, \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\"?>\");\n        let _ = writeln!(\n            &mut config,\n            \"<Autodiscover xmlns=\\\"http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006\\\">\"\n        );\n        let _ = writeln!(\n            &mut config,\n            \"\\t<Response xmlns=\\\"http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a\\\">\"\n        );\n        let _ = writeln!(&mut config, \"\\t\\t<User>\");\n        let _ = writeln!(\n            &mut config,\n            \"\\t\\t\\t<DisplayName>{emailaddress}</DisplayName>\"\n        );\n        let _ = writeln!(\n            &mut config,\n            \"\\t\\t\\t<AutoDiscoverSMTPAddress>{emailaddress}</AutoDiscoverSMTPAddress>\"\n        );\n        // DeploymentId is a required field of User but we are not a MS Exchange server so use a random value\n        let _ = writeln!(\n            &mut config,\n            \"\\t\\t\\t<DeploymentId>644560b8-a1ce-429c-8ace-23395843f701</DeploymentId>\"\n        );\n        let _ = writeln!(&mut config, \"\\t\\t</User>\");\n        let _ = writeln!(&mut config, \"\\t\\t<Account>\");\n        let _ = writeln!(&mut config, \"\\t\\t\\t<AccountType>email</AccountType>\");\n        let _ = writeln!(&mut config, \"\\t\\t\\t<Action>settings</Action>\");\n        for (protocol, port, is_tls) in services {\n            match protocol.as_str() {\n                \"imap\" | \"pop3\" => (),\n                \"smtp\" if port != 25 => (),\n                _ => continue,\n            }\n\n            let _ = writeln!(&mut config, \"\\t\\t\\t<Protocol>\");\n            let _ = writeln!(\n                &mut config,\n                \"\\t\\t\\t\\t<Type>{}</Type>\",\n                protocol.to_uppercase()\n            );\n            let _ = writeln!(&mut config, \"\\t\\t\\t\\t<Server>{server_name}</Server>\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t\\t<Port>{port}</Port>\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t\\t<LoginName>{account_name}</LoginName>\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t\\t<AuthRequired>on</AuthRequired>\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t\\t<DirectoryPort>0</DirectoryPort>\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t\\t<ReferralPort>0</ReferralPort>\");\n            let _ = writeln!(\n                &mut config,\n                \"\\t\\t\\t\\t<SSL>{}</SSL>\",\n                if is_tls { \"on\" } else { \"off\" }\n            );\n            if is_tls {\n                let _ = writeln!(&mut config, \"\\t\\t\\t\\t<Encryption>TLS</Encryption>\");\n            }\n            let _ = writeln!(&mut config, \"\\t\\t\\t\\t<SPA>off</SPA>\");\n            let _ = writeln!(&mut config, \"\\t\\t\\t</Protocol>\");\n        }\n\n        let _ = writeln!(&mut config, \"\\t\\t</Account>\");\n        let _ = writeln!(&mut config, \"\\t</Response>\");\n        let _ = writeln!(&mut config, \"</Autodiscover>\");\n\n        Ok(\n            Resource::new(\"application/xml; charset=utf-8\", config.into_bytes())\n                .into_http_response(),\n        )\n    }\n\n    async fn autoconfig_parameters<'x>(\n        &'x self,\n        emailaddress: &'x str,\n        fail_if_invalid: bool,\n    ) -> trc::Result<(String, String, &'x str)> {\n        // Return EMAILADDRESS\n        let Some((_, domain)) = emailaddress.rsplit_once('@') else {\n            return if !fail_if_invalid {\n                Ok((\n                    \"%EMAILADDRESS%\".to_string(),\n                    self.core.network.server_name.clone(),\n                    &self.core.network.report_domain,\n                ))\n            } else {\n                Err(trc::ResourceEvent::BadParameters\n                    .into_err()\n                    .details(\"Missing domain in email address\"))\n            };\n        };\n\n        // Find the account name by e-mail address\n        let mut account_name = emailaddress.into();\n        if let Some(id) = self\n            .core\n            .storage\n            .directory\n            .email_to_id(emailaddress)\n            .await\n            .caused_by(trc::location!())?\n            && let Ok(Some(principal)) = self\n                .core\n                .storage\n                .directory\n                .query(QueryParams::id(id).with_return_member_of(false))\n                .await\n            && principal\n                .primary_email()\n                .is_some_and(|email| email.eq_ignore_ascii_case(emailaddress))\n        {\n            account_name = principal.name;\n        }\n\n        Ok((account_name, self.core.network.server_name.clone(), domain))\n    }\n}\n\nfn parse_autodiscover_request(bytes: &[u8]) -> Result<String, String> {\n    if bytes.is_empty() {\n        return Err(\"Empty request body\".to_string());\n    }\n\n    let mut reader = Reader::from_reader(bytes);\n    reader.config_mut().trim_text(true);\n    let mut buf = Vec::with_capacity(128);\n\n    'outer: for tag_name in [\"Autodiscover\", \"Request\", \"EMailAddress\"] {\n        loop {\n            match reader.read_event_into(&mut buf) {\n                Ok(Event::Start(e)) => {\n                    let found_tag_name = e.name();\n                    if tag_name\n                        .as_bytes()\n                        .eq_ignore_ascii_case(found_tag_name.as_ref())\n                    {\n                        continue 'outer;\n                    } else if tag_name == \"EMailAddress\" {\n                        // Skip unsupported tags under Request, such as AcceptableResponseSchema\n                        let mut tag_count = 0;\n                        loop {\n                            match reader.read_event_into(&mut buf) {\n                                Ok(Event::End(_)) => {\n                                    if tag_count == 0 {\n                                        break;\n                                    } else {\n                                        tag_count -= 1;\n                                    }\n                                }\n                                Ok(Event::Start(_)) => {\n                                    tag_count += 1;\n                                }\n                                Ok(Event::Eof) => {\n                                    return Err(format!(\n                                        \"Expected value, found unexpected EOF at position {}.\",\n                                        reader.buffer_position()\n                                    ));\n                                }\n                                _ => (),\n                            }\n                        }\n                    } else {\n                        return Err(format!(\n                            \"Expected tag {}, found unexpected tag {} at position {}.\",\n                            tag_name,\n                            String::from_utf8_lossy(found_tag_name.as_ref()),\n                            reader.buffer_position()\n                        ));\n                    }\n                }\n                Ok(Event::Decl(_) | Event::Text(_)) => (),\n                Err(e) => {\n                    return Err(format!(\n                        \"Error at position {}: {:?}\",\n                        reader.buffer_position(),\n                        e\n                    ));\n                }\n                Ok(event) => {\n                    return Err(format!(\n                        \"Expected tag {}, found unexpected event {event:?} at position {}.\",\n                        tag_name,\n                        reader.buffer_position()\n                    ));\n                }\n            }\n        }\n    }\n\n    if let Ok(Event::Text(text)) = reader.read_event_into(&mut buf)\n        && let Ok(text) = text.xml_content()\n        && text.contains('@')\n    {\n        return Ok(text.trim().to_lowercase());\n    }\n\n    Err(format!(\n        \"Expected email address, found unexpected value at position {}.\",\n        reader.buffer_position()\n    ))\n}\n\n#[cfg(test)]\nmod tests {\n\n    #[test]\n    fn parse_autodiscover() {\n        let r = r#\"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n            <Autodiscover xmlns=\"http://schemas.microsoft.com/exchange/autodiscover/outlook/requestschema/2006\">\n                <Request>\n                        <EMailAddress>email@example.com</EMailAddress>\n                        <AcceptableResponseSchema>http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a</AcceptableResponseSchema>\n                </Request>\n            </Autodiscover>\"#;\n\n        assert_eq!(\n            super::parse_autodiscover_request(r.as_bytes()).unwrap(),\n            \"email@example.com\"\n        );\n    }\n}\n"
  },
  {
    "path": "crates/http/src/form/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::auth::oauth::FormData;\nuse chrono::Utc;\nuse common::{\n    KV_RATE_LIMIT_CONTACT, Server,\n    config::network::{ContactForm, FieldOrDefault},\n    ip_to_bytes, psl,\n};\nuse email::message::delivery::{IngestMessage, IngestRecipient, LocalDeliveryStatus, MailDelivery};\nuse http_proto::*;\nuse hyper::StatusCode;\nuse mail_auth::common::cache::NoCache;\nuse mail_builder::{\n    MessageBuilder,\n    headers::{\n        HeaderType,\n        address::{Address, EmailAddress},\n    },\n    mime::make_boundary,\n};\nuse serde_json::json;\nuse std::{borrow::Cow, fmt::Write, future::Future};\nuse store::write::BatchBuilder;\nuse trc::AddContext;\n\npub trait FormHandler: Sync + Send {\n    fn handle_contact_form(\n        &self,\n        session: &HttpSessionData,\n        form: &ContactForm,\n        form_data: FormData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl FormHandler for Server {\n    async fn handle_contact_form(\n        &self,\n        session: &HttpSessionData,\n        form: &ContactForm,\n        form_data: FormData,\n    ) -> trc::Result<HttpResponse> {\n        // Validate rate\n        if let Some(rate) = &form.rate\n            && !session.remote_ip.is_loopback()\n            && self\n                .core\n                .storage\n                .lookup\n                .is_rate_allowed(\n                    KV_RATE_LIMIT_CONTACT,\n                    &ip_to_bytes(&session.remote_ip),\n                    rate,\n                    false,\n                )\n                .await\n                .caused_by(trc::location!())?\n                .is_some()\n        {\n            return Err(trc::LimitEvent::TooManyRequests.into_err());\n        }\n\n        // Validate honeypot\n        if form\n            .field_honey_pot\n            .as_ref()\n            .is_some_and(|field| form_data.has_field(field))\n        {\n            return Err(trc::ResourceEvent::BadParameters\n                .into_err()\n                .details(\"Honey pot field present\"));\n        }\n\n        // Obtain fields\n        let from_email = form_data\n            .get_or_default(&form.from_email)\n            .trim()\n            .to_lowercase();\n        let from_subject = form_data.get_or_default(&form.from_subject).trim();\n        let from_name = form_data.get_or_default(&form.from_name).trim();\n\n        // Validate email\n        let mut failure = None;\n        let mut has_success = false;\n        if form.validate_domain && from_email != form.from_email.default {\n            if let Some(domain) = from_email.rsplit_once('@').and_then(|(local, domain)| {\n                if !local.is_empty()\n                    && domain.contains('.')\n                    && psl::domain(domain.as_bytes()).is_some_and(|d| d.suffix().typ().is_some())\n                {\n                    Some(domain)\n                } else {\n                    None\n                }\n            }) {\n                if self\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .mx_lookup(domain, None::<&NoCache<_, _>>)\n                    .await\n                    .is_err()\n                {\n                    failure = Some(format!(\"No MX records found for domain {domain:?}. Please enter a valid email address.\", ).into());\n                }\n            } else {\n                failure = Some(Cow::Borrowed(\"Please enter a valid email address.\"));\n            }\n        }\n\n        // Discard empty forms\n        if failure.is_none() && form_data.fields().all(|(_, value)| value.trim().is_empty()) {\n            failure = Some(Cow::Borrowed(\"Empty form\"));\n        }\n\n        if failure.is_none() {\n            // Build body\n            let mut body = String::with_capacity(1024);\n            for (field, value) in form_data.fields() {\n                if !value.is_empty() {\n                    body.push_str(field);\n                    body.push_str(\": \");\n                    body.push_str(value);\n                    body.push_str(\"\\r\\n\");\n                }\n            }\n            let _ = write!(\n                &mut body,\n                \"Date: {}\\r\\n\",\n                Utc::now().format(\"%a, %d %b %Y %T %z\")\n            );\n            let _ = write!(\n                &mut body,\n                \"IP: {}:{}\\r\\n\",\n                session.remote_ip, session.remote_port\n            );\n\n            // Build message\n            let message = MessageBuilder::new()\n                .from((from_name, from_email.as_str()))\n                .header(\n                    \"To\",\n                    HeaderType::Address(Address::List(\n                        form.rcpt_to\n                            .iter()\n                            .map(|rcpt| {\n                                Address::Address(EmailAddress {\n                                    name: None,\n                                    email: rcpt.into(),\n                                })\n                            })\n                            .collect(),\n                    )),\n                )\n                .header(\"Auto-Submitted\", HeaderType::Text(\"auto-generated\".into()))\n                .message_id(format!(\n                    \"<{}@{}.{}>\",\n                    make_boundary(\".\"),\n                    session.remote_ip,\n                    session.remote_port\n                ))\n                .subject(from_subject)\n                .text_body(body)\n                .write_to_vec()\n                .unwrap_or_default();\n\n            // Reserve and write blob\n            let (message_blob, blob_hold) = self\n                .put_temporary_blob(u32::MAX, &message, 60)\n                .await\n                .caused_by(trc::location!())?;\n\n            for result in self\n                .deliver_message(IngestMessage {\n                    sender_address: from_email,\n                    sender_authenticated: false,\n                    recipients: form\n                        .rcpt_to\n                        .iter()\n                        .map(|address| IngestRecipient {\n                            address: address.clone(),\n                            is_spam: false,\n                        })\n                        .collect(),\n                    message_blob,\n                    message_size: message.len() as u64,\n                    session_id: session.session_id,\n                })\n                .await\n                .status\n            {\n                match result {\n                    LocalDeliveryStatus::Success => {\n                        has_success = true;\n                    }\n                    LocalDeliveryStatus::TemporaryFailure { reason }\n                    | LocalDeliveryStatus::PermanentFailure { reason, .. } => {\n                        failure = Some(reason)\n                    }\n                }\n            }\n\n            // Remove blob hold\n            let mut batch = BatchBuilder::new();\n            batch.clear(blob_hold);\n            self.store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n\n            // Suppress errors if there is at least one success\n            if has_success {\n                failure = None;\n            }\n        }\n\n        Ok(JsonResponse::with_status(\n            if has_success {\n                StatusCode::OK\n            } else {\n                StatusCode::BAD_REQUEST\n            },\n            json!({\n                \"data\": {\n                    \"success\": has_success,\n                    \"details\": failure,\n                },\n            }),\n        )\n        .into_http_response())\n    }\n}\n\nimpl FormData {\n    pub fn get_or_default<'x>(&'x self, field: &'x FieldOrDefault) -> &'x str {\n        if let Some(field_name) = &field.field {\n            self.get(field_name)\n                .filter(|f| !f.is_empty())\n                .unwrap_or(field.default.as_str())\n        } else {\n            field.default.as_str()\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod auth;\npub mod autoconfig;\npub mod form;\npub mod management;\npub mod request;\n\nuse std::sync::Arc;\n\nuse common::Inner;\n\n#[derive(Clone)]\npub struct HttpSessionManager {\n    pub inner: Arc<Inner>,\n}\n\nimpl HttpSessionManager {\n    pub fn new(inner: Arc<Inner>) -> Self {\n        Self { inner }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/crypto.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse directory::backend::internal::manage;\nuse email::message::crypto::{\n    ENCRYPT_TRAIN_SPAM_FILTER, EncryptMessage, EncryptMessageError, EncryptionMethod,\n    EncryptionParams, EncryptionType, try_parse_certs,\n};\nuse http_proto::*;\nuse mail_builder::encoders::base64::base64_encode_mime;\nuse mail_parser::MessageParser;\nuse serde_json::json;\nuse std::{future::Future, sync::Arc};\nuse store::{\n    Deserialize, Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::PrincipalField};\n\npub trait CryptoHandler: Sync + Send {\n    fn handle_crypto_get(\n        &self,\n        access_token: Arc<AccessToken>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_crypto_post(\n        &self,\n        access_token: Arc<AccessToken>,\n        body: Option<Vec<u8>>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl CryptoHandler for Server {\n    async fn handle_crypto_get(&self, access_token: Arc<AccessToken>) -> trc::Result<HttpResponse> {\n        let ec = if let Some(params_) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                access_token.primary_id(),\n                Collection::Principal,\n                0,\n                PrincipalField::EncryptionKeys,\n            ))\n            .await?\n        {\n            let params = params_\n                .unarchive::<EncryptionParams>()\n                .caused_by(trc::location!())?;\n            let algo = params.algo();\n            let method = params.method();\n            let allow_spam_training = params.can_train_spam_filter();\n            let mut certs = Vec::new();\n            certs.extend_from_slice(b\"-----STALWART CERTIFICATE-----\\r\\n\");\n            let _ = base64_encode_mime(&params_.into_inner(), &mut certs, false);\n            certs.extend_from_slice(b\"\\r\\n\");\n            let certs = String::from_utf8(certs).unwrap_or_default();\n\n            match method {\n                EncryptionMethod::PGP => EncryptionType::PGP {\n                    algo,\n                    certs,\n                    allow_spam_training,\n                },\n                EncryptionMethod::SMIME => EncryptionType::SMIME {\n                    algo,\n                    certs,\n                    allow_spam_training,\n                },\n            }\n        } else {\n            EncryptionType::Disabled\n        };\n\n        Ok(JsonResponse::new(json!({\n            \"data\": ec,\n        }))\n        .into_http_response())\n    }\n\n    async fn handle_crypto_post(\n        &self,\n        access_token: Arc<AccessToken>,\n        body: Option<Vec<u8>>,\n    ) -> trc::Result<HttpResponse> {\n        let request = serde_json::from_slice::<EncryptionType>(body.as_deref().unwrap_or_default())\n            .map_err(|err| trc::ResourceEvent::BadParameters.into_err().reason(err))?;\n\n        let (method, algo, mut certs, allow_spam_training) = match request {\n            EncryptionType::PGP {\n                algo,\n                certs,\n                allow_spam_training,\n            } => (EncryptionMethod::PGP, algo, certs, allow_spam_training),\n            EncryptionType::SMIME {\n                algo,\n                certs,\n                allow_spam_training,\n            } => (EncryptionMethod::SMIME, algo, certs, allow_spam_training),\n            EncryptionType::Disabled => {\n                // Disable encryption at rest\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(access_token.primary_id())\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .clear(PrincipalField::EncryptionKeys);\n                self.core.storage.data.write(batch.build_all()).await?;\n                return Ok(JsonResponse::new(json!({\n                    \"data\": (),\n                }))\n                .into_http_response());\n            }\n        };\n        if !certs.ends_with(\"\\n\") {\n            certs.push('\\n');\n        }\n\n        // Make sure Encryption is enabled\n        if !self.core.jmap.encrypt {\n            return Err(manage::unsupported(\n                \"Encryption-at-rest has been disabled by the system administrator\",\n            ));\n        }\n\n        // Parse certificates\n        let certs = try_parse_certs(method, certs.into_bytes())\n            .map_err(|err| manage::error(err, None::<u32>))?;\n        let num_certs = certs.len();\n        let params = Archiver::new(EncryptionParams {\n            flags: method.flags()\n                | algo.flags()\n                | if allow_spam_training {\n                    ENCRYPT_TRAIN_SPAM_FILTER\n                } else {\n                    0\n                },\n            certs,\n        })\n        .serialize()\n        .caused_by(trc::location!())?;\n\n        // Try a test encryption\n        if let Err(EncryptMessageError::Error(message)) = MessageParser::new()\n            .parse(\"Subject: test\\r\\ntest\\r\\n\".as_bytes())\n            .unwrap()\n            .encrypt(\n                <Archive<AlignedBytes> as Deserialize>::deserialize(params.as_slice())?\n                    .unarchive::<EncryptionParams>()?,\n            )\n            .await\n        {\n            return Err(manage::error(message, None::<u32>));\n        }\n\n        // Save encryption params\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(access_token.primary_id())\n            .with_collection(Collection::Principal)\n            .with_document(0)\n            .set(PrincipalField::EncryptionKeys, params);\n        self.core.storage.data.write(batch.build_all()).await?;\n\n        Ok(JsonResponse::new(json!({\n            \"data\": num_certs,\n        }))\n        .into_http_response())\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/dkim.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::str::FromStr;\n\nuse common::{Server, auth::AccessToken, config::smtp::auth::simple_pem_parse};\nuse directory::{Permission, backend::internal::manage};\nuse hyper::Method;\nuse mail_auth::{\n    common::crypto::{Ed25519Key, RsaKey, Sha256},\n    dkim::generate::DkimKeyPair,\n};\nuse mail_builder::encoders::base64::base64_encode;\nuse mail_parser::DateTime;\nuse pkcs8::Document;\nuse rsa::pkcs1::DecodeRsaPublicKey;\nuse serde::{Deserialize, Serialize};\nuse serde_json::json;\nuse store::write::now;\n\nuse http_proto::{request::decode_path_element, *};\nuse std::future::Future;\n\n#[derive(Debug, Serialize, Deserialize, Copy, Clone, PartialEq, Eq)]\npub enum Algorithm {\n    Rsa,\n    Ed25519,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\nstruct DkimSignature {\n    id: Option<String>,\n    algorithm: Algorithm,\n    domain: String,\n    selector: Option<String>,\n}\n\npub trait DkimManagement: Sync + Send {\n    fn handle_manage_dkim(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_get_public_key(\n        &self,\n        path: Vec<&str>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_create_signature(\n        &self,\n        body: Option<Vec<u8>>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn create_dkim_key(\n        &self,\n        algo: Algorithm,\n        id: impl AsRef<str> + Send,\n        domain: impl Into<String> + Send,\n        selector: impl Into<String> + Send,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n}\n\nimpl DkimManagement for Server {\n    async fn handle_manage_dkim(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        match *req.method() {\n            Method::GET => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::DkimSignatureGet)?;\n\n                self.handle_get_public_key(path).await\n            }\n            Method::POST => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::DkimSignatureCreate)?;\n\n                self.handle_create_signature(body).await\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n\n    async fn handle_get_public_key(&self, path: Vec<&str>) -> trc::Result<HttpResponse> {\n        let signature_id = match path.get(1) {\n            Some(signature_id) => decode_path_element(signature_id),\n            None => {\n                return Err(trc::ResourceEvent::NotFound.into_err());\n            }\n        };\n\n        let (pk, algo) = match (\n            self.core\n                .storage\n                .config\n                .get(&format!(\"signature.{signature_id}.private-key\"))\n                .await,\n            self.core\n                .storage\n                .config\n                .get(&format!(\"signature.{signature_id}.algorithm\"))\n                .await\n                .map(|algo| algo.and_then(|algo| algo.parse::<Algorithm>().ok())),\n        ) {\n            (Ok(Some(pk)), Ok(Some(algorithm))) => (pk, algorithm),\n            (Err(err), _) | (_, Err(err)) => return Err(err.caused_by(trc::location!())),\n            _ => return Err(trc::ResourceEvent::NotFound.into_err()),\n        };\n\n        Ok(JsonResponse::new(json!({\n            \"data\": obtain_dkim_public_key(algo, &pk)?,\n        }))\n        .into_http_response())\n    }\n\n    async fn handle_create_signature(&self, body: Option<Vec<u8>>) -> trc::Result<HttpResponse> {\n        let request =\n            match serde_json::from_slice::<DkimSignature>(body.as_deref().unwrap_or_default()) {\n                Ok(request) => request,\n                Err(err) => {\n                    return Err(\n                        trc::EventType::Resource(trc::ResourceEvent::BadParameters).reason(err)\n                    );\n                }\n            };\n\n        let algo_str = match request.algorithm {\n            Algorithm::Rsa => \"rsa\",\n            Algorithm::Ed25519 => \"ed25519\",\n        };\n        let id = request\n            .id\n            .unwrap_or_else(|| format!(\"{algo_str}-{}\", request.domain));\n        let selector = request.selector.unwrap_or_else(|| {\n            let dt = DateTime::from_timestamp(now() as i64);\n            format!(\n                \"{:04}{:02}{}\",\n                dt.year,\n                dt.month,\n                if Algorithm::Rsa == request.algorithm {\n                    \"r\"\n                } else {\n                    \"e\"\n                }\n            )\n        });\n\n        // Make sure the signature does not exist already\n        if let Some(value) = self\n            .core\n            .storage\n            .config\n            .get(&format!(\"signature.{id}.private-key\"))\n            .await?\n        {\n            return Err(manage::err_exists(\n                format!(\"signature.{id}.private-key\"),\n                value,\n            ));\n        }\n\n        // Create signature\n        self.create_dkim_key(request.algorithm, id, request.domain, selector)\n            .await?;\n\n        Ok(JsonResponse::new(json!({\n            \"data\": (),\n        }))\n        .into_http_response())\n    }\n\n    async fn create_dkim_key(\n        &self,\n        algo: Algorithm,\n        id: impl AsRef<str>,\n        domain: impl Into<String>,\n        selector: impl Into<String>,\n    ) -> trc::Result<()> {\n        let id = id.as_ref();\n        let (algorithm, pk_type) = match algo {\n            Algorithm::Rsa => (\"rsa-sha256\", \"RSA PRIVATE KEY\"),\n            Algorithm::Ed25519 => (\"ed25519-sha256\", \"PRIVATE KEY\"),\n        };\n        let mut pk = format!(\"-----BEGIN {pk_type}-----\\n\").into_bytes();\n        let mut lf_count = 65;\n        for ch in base64_encode(\n            match algo {\n                Algorithm::Rsa => DkimKeyPair::generate_rsa(2048),\n                Algorithm::Ed25519 => DkimKeyPair::generate_ed25519(),\n            }\n            .map_err(|err| {\n                manage::error(\"Failed to generate key\", err.to_string().into())\n                    .caused_by(trc::location!())\n            })?\n            .private_key(),\n        )\n        .unwrap_or_default()\n        {\n            pk.push(ch);\n            lf_count -= 1;\n            if lf_count == 0 {\n                pk.push(b'\\n');\n                lf_count = 65;\n            }\n        }\n        if lf_count != 65 {\n            pk.push(b'\\n');\n        }\n        pk.extend_from_slice(format!(\"-----END {pk_type}-----\\n\").as_bytes());\n\n        self.core\n            .storage\n            .config\n            .set(\n                [\n                    (\n                        format!(\"signature.{id}.private-key\"),\n                        String::from_utf8(pk).unwrap(),\n                    ),\n                    (format!(\"signature.{id}.domain\"), domain.into()),\n                    (format!(\"signature.{id}.selector\"), selector.into()),\n                    (format!(\"signature.{id}.algorithm\"), algorithm.to_string()),\n                    (\n                        format!(\"signature.{id}.canonicalization\"),\n                        \"relaxed/relaxed\".to_string(),\n                    ),\n                    (format!(\"signature.{id}.headers.0\"), \"From\".to_string()),\n                    (format!(\"signature.{id}.headers.1\"), \"To\".to_string()),\n                    (format!(\"signature.{id}.headers.2\"), \"Date\".to_string()),\n                    (format!(\"signature.{id}.headers.3\"), \"Subject\".to_string()),\n                    (\n                        format!(\"signature.{id}.headers.4\"),\n                        \"Message-ID\".to_string(),\n                    ),\n                    (format!(\"signature.{id}.report\"), \"false\".to_string()),\n                ],\n                true,\n            )\n            .await\n    }\n}\n\npub fn obtain_dkim_public_key(algo: Algorithm, pk: &str) -> trc::Result<String> {\n    match simple_pem_parse(pk) {\n        Some(der) => match algo {\n            Algorithm::Rsa => match RsaKey::<Sha256>::from_der(&der).and_then(|key| {\n                Document::from_pkcs1_der(&key.public_key())\n                    .map_err(|err| mail_auth::Error::CryptoError(err.to_string()))\n            }) {\n                Ok(pk) => Ok(\n                    String::from_utf8(base64_encode(pk.as_bytes()).unwrap_or_default())\n                        .unwrap_or_default(),\n                ),\n                Err(err) => Err(manage::error(\n                    \"Failed to read RSA DER\",\n                    err.to_string().into(),\n                )),\n            },\n            Algorithm::Ed25519 => {\n                match Ed25519Key::from_pkcs8_maybe_unchecked_der(&der)\n                    .map_err(|err| mail_auth::Error::CryptoError(err.to_string()))\n                {\n                    Ok(pk) => Ok(String::from_utf8(\n                        base64_encode(&pk.public_key()).unwrap_or_default(),\n                    )\n                    .unwrap_or_default()),\n                    Err(err) => Err(manage::error(\"Crypto error\", err.to_string().into())),\n                }\n            }\n        },\n        None => Err(manage::error(\"Failed to decode private key\", None::<u32>)),\n    }\n}\n\nimpl FromStr for Algorithm {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        match s.split_once('-').map(|(algo, _)| algo) {\n            Some(\"rsa\") => Ok(Algorithm::Rsa),\n            Some(\"ed25519\") => Ok(Algorithm::Ed25519),\n            _ => Err(()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/dns.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse directory::{\n    Permission,\n    backend::internal::manage::{self},\n};\n\nuse hyper::Method;\nuse serde::{Deserialize, Serialize};\nuse serde_json::json;\nuse sha1::Digest;\nuse utils::config::Config;\nuse x509_parser::parse_x509_certificate;\n\nuse crate::management::dkim::{Algorithm, obtain_dkim_public_key};\nuse http_proto::{request::decode_path_element, *};\nuse std::future::Future;\n\n#[derive(Debug, Serialize, Deserialize)]\npub struct DnsRecord {\n    #[serde(rename = \"type\")]\n    typ: String,\n    name: String,\n    content: String,\n}\n\npub trait DnsManagement: Sync + Send {\n    fn handle_manage_dns(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn build_dns_records(\n        &self,\n        domain_name: &str,\n    ) -> impl Future<Output = trc::Result<Vec<DnsRecord>>> + Send;\n}\n\nimpl DnsManagement for Server {\n    async fn handle_manage_dns(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        match (\n            path.get(1).copied().unwrap_or_default(),\n            path.get(2),\n            req.method(),\n        ) {\n            (\"records\", Some(domain), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::DomainGet)?;\n\n                // Obtain DNS records\n                let domain = decode_path_element(domain);\n                Ok(JsonResponse::new(json!({\n                    \"data\": self.build_dns_records(domain.as_ref()).await?,\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n\n    async fn build_dns_records(&self, domain_name: &str) -> trc::Result<Vec<DnsRecord>> {\n        // Obtain server name\n        let server_name = &self.core.network.server_name;\n        let mut records = Vec::new();\n\n        // Obtain DKIM keys\n        let mut keys = Config::default();\n        let mut signature_ids = Vec::new();\n        let mut has_macros = false;\n        for (key, value) in self.core.storage.config.list(\"signature.\", true).await? {\n            match key.strip_suffix(\".domain\") {\n                Some(key_id) if value == domain_name => {\n                    signature_ids.push(key_id.to_string());\n                }\n                _ => (),\n            }\n            if !has_macros && value.contains(\"%{\") {\n                has_macros = true;\n            }\n            keys.keys.insert(key, value);\n        }\n\n        // Add MX and CNAME records\n        records.push(DnsRecord {\n            typ: \"MX\".to_string(),\n            name: format!(\"{domain_name}.\"),\n            content: format!(\"10 {server_name}.\"),\n        });\n        if server_name.strip_prefix(\"mail.\") != Some(domain_name) {\n            records.push(DnsRecord {\n                typ: \"CNAME\".to_string(),\n                name: format!(\"mail.{domain_name}.\"),\n                content: format!(\"{server_name}.\"),\n            });\n        }\n\n        // Process DKIM keys\n        if has_macros {\n            keys.resolve_macros(&[\"env\", \"file\", \"cfg\"]).await;\n            keys.log_errors();\n        }\n        for signature_id in signature_ids {\n            if let (Some(algo), Some(pk), Some(selector)) = (\n                keys.value(format!(\"{signature_id}.algorithm\"))\n                    .and_then(|algo| algo.parse::<Algorithm>().ok()),\n                keys.value(format!(\"{signature_id}.private-key\")),\n                keys.value(format!(\"{signature_id}.selector\")),\n            ) {\n                match obtain_dkim_public_key(algo, pk) {\n                    Ok(public) => {\n                        records.push(DnsRecord {\n                            typ: \"TXT\".to_string(),\n                            name: format!(\"{selector}._domainkey.{domain_name}.\",),\n                            content: match algo {\n                                Algorithm::Rsa => format!(\"v=DKIM1; k=rsa; h=sha256; p={public}\"),\n                                Algorithm::Ed25519 => {\n                                    format!(\"v=DKIM1; k=ed25519; h=sha256; p={public}\")\n                                }\n                            },\n                        });\n                    }\n                    Err(err) => {\n                        trc::error!(err);\n                    }\n                }\n            }\n        }\n\n        // Add SPF records\n        if server_name.ends_with(&format!(\".{domain_name}\")) || server_name == domain_name {\n            records.push(DnsRecord {\n                typ: \"TXT\".to_string(),\n                name: format!(\"{server_name}.\"),\n                content: \"v=spf1 a ra=postmaster -all\".to_string(),\n            });\n        }\n        records.push(DnsRecord {\n            typ: \"TXT\".to_string(),\n            name: format!(\"{domain_name}.\"),\n            content: \"v=spf1 mx ra=postmaster -all\".to_string(),\n        });\n\n        let mut has_https = false;\n        for (protocol, port, is_tls) in self\n            .core\n            .storage\n            .config\n            .get_services()\n            .await\n            .unwrap_or_default()\n        {\n            match (protocol.as_str(), port) {\n                (\"smtp\", port @ 26..=u16::MAX) => {\n                    records.push(DnsRecord {\n                        typ: \"SRV\".to_string(),\n                        name: format!(\n                            \"_submission{}._tcp.{domain_name}.\",\n                            if is_tls { \"s\" } else { \"\" }\n                        ),\n                        content: format!(\"0 1 {port} {server_name}.\"),\n                    });\n                }\n                (\"imap\" | \"pop3\", port @ 1..=u16::MAX) => {\n                    records.push(DnsRecord {\n                        typ: \"SRV\".to_string(),\n                        name: format!(\n                            \"_{protocol}{}._tcp.{domain_name}.\",\n                            if is_tls { \"s\" } else { \"\" }\n                        ),\n                        content: format!(\"0 1 {port} {server_name}.\"),\n                    });\n                }\n                (\"http\", _) if is_tls => {\n                    has_https = true;\n                    for service in [\"jmap\", \"caldavs\", \"carddavs\"] {\n                        records.push(DnsRecord {\n                            typ: \"SRV\".to_string(),\n                            name: format!(\"_{service}._tcp.{domain_name}.\",),\n                            content: format!(\"0 1 {port} {server_name}.\"),\n                        });\n                    }\n                }\n                _ => (),\n            }\n        }\n\n        if has_https {\n            // Add autoconfig and autodiscover records\n            records.push(DnsRecord {\n                typ: \"CNAME\".to_string(),\n                name: format!(\"autoconfig.{domain_name}.\"),\n                content: format!(\"{server_name}.\"),\n            });\n            records.push(DnsRecord {\n                typ: \"CNAME\".to_string(),\n                name: format!(\"autodiscover.{domain_name}.\"),\n                content: format!(\"{server_name}.\"),\n            });\n\n            // Add MTA-STS records\n            if let Some(policy) = self.build_mta_sts_policy() {\n                records.push(DnsRecord {\n                    typ: \"CNAME\".to_string(),\n                    name: format!(\"mta-sts.{domain_name}.\"),\n                    content: format!(\"{server_name}.\"),\n                });\n                records.push(DnsRecord {\n                    typ: \"TXT\".to_string(),\n                    name: format!(\"_mta-sts.{domain_name}.\"),\n                    content: format!(\"v=STSv1; id={}\", policy.id),\n                });\n            }\n        }\n\n        // Add DMARC record\n        records.push(DnsRecord {\n            typ: \"TXT\".to_string(),\n            name: format!(\"_dmarc.{domain_name}.\"),\n            content: format!(\"v=DMARC1; p=reject; rua=mailto:postmaster@{domain_name}; ruf=mailto:postmaster@{domain_name}\",),\n        });\n\n        // Add TLS reporting record\n        records.push(DnsRecord {\n            typ: \"TXT\".to_string(),\n            name: format!(\"_smtp._tls.{domain_name}.\"),\n            content: format!(\"v=TLSRPTv1; rua=mailto:postmaster@{domain_name}\",),\n        });\n\n        // Add TLSA records\n        for (name, key) in self.inner.data.tls_certificates.load().iter() {\n            if !name.ends_with(domain_name)\n                || name.starts_with(\"mta-sts.\")\n                || name.starts_with(\"autoconfig.\")\n                || name.starts_with(\"autodiscover.\")\n            {\n                continue;\n            }\n\n            for (cert_num, cert) in key.cert.iter().enumerate() {\n                let parsed_cert = match parse_x509_certificate(cert) {\n                    Ok((_, parsed_cert)) => parsed_cert,\n                    Err(err) => {\n                        trc::error!(manage::error(\n                            \"Failed to parse certificate\",\n                            err.to_string().into()\n                        ));\n                        continue;\n                    }\n                };\n\n                let name = if !name.starts_with('.') {\n                    format!(\"_25._tcp.{name}.\")\n                } else {\n                    format!(\"_25._tcp.mail.{name}.\")\n                };\n                let cu = if cert_num == 0 { 3 } else { 2 };\n\n                for (s, cert) in [cert, parsed_cert.subject_pki.raw].into_iter().enumerate() {\n                    for (m, hash) in [\n                        format!(\"{:x}\", sha2::Sha256::digest(cert)),\n                        format!(\"{:x}\", sha2::Sha512::digest(cert)),\n                    ]\n                    .into_iter()\n                    .enumerate()\n                    {\n                        records.push(DnsRecord {\n                            typ: \"TLSA\".to_string(),\n                            name: name.clone(),\n                            content: format!(\"{} {} {} {}\", cu, s, m + 1, hash),\n                        });\n                    }\n                }\n            }\n        }\n\n        Ok(records)\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/enterprise/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\npub mod telemetry;\npub mod undelete;\n"
  },
  {
    "path": "crates/http/src/management/enterprise/telemetry.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse std::{\n    fmt::Write,\n    time::{Duration, Instant},\n};\n\nuse common::{\n    Server,\n    auth::{AccessToken, oauth::GrantType},\n    telemetry::{\n        metrics::store::{Metric, MetricsStore},\n        tracers::store::TracingStore,\n    },\n};\nuse directory::{Permission, backend::internal::manage};\nuse http_body_util::{StreamBody, combinators::BoxBody};\nuse http_proto::*;\nuse hyper::{\n    Method, StatusCode,\n    body::{Bytes, Frame},\n};\nuse mail_parser::DateTime;\nuse serde_json::json;\nuse std::future::Future;\nuse store::{\n    ahash::{AHashMap, AHashSet},\n    search::{SearchComparator, SearchField, SearchFilter, SearchQuery, TracingSearchField},\n    write::{SearchIndex, now},\n};\nuse trc::{\n    Collector, DeliveryEvent, EventType, Key, MetricType, QueueEvent, Value,\n    ipc::{bitset::Bitset, subscriber::SubscriberBuilder},\n    serializers::json::JsonEventSerializer,\n};\nuse utils::{snowflake::SnowflakeIdGenerator, url_params::UrlParams};\n\nuse crate::management::Timestamp;\n\npub trait TelemetryApi: Sync + Send {\n    fn handle_telemetry_api_request(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl TelemetryApi for Server {\n    async fn handle_telemetry_api_request(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        let params = UrlParams::new(req.uri().query());\n        let account_id = access_token.primary_id();\n\n        match (\n            path.get(1).copied().unwrap_or_default(),\n            path.get(2).copied(),\n            req.method(),\n        ) {\n            (\"traces\", None, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::TracingList)?;\n\n                let page: usize = params.parse(\"page\").unwrap_or(0);\n                let limit: usize = params.parse(\"limit\").unwrap_or(0);\n                let mut tracing_query = Vec::new();\n                tracing_query.push(SearchFilter::And);\n                if let Some(typ) = params.parse::<EventType>(\"type\") {\n                    tracing_query.push(SearchFilter::eq(TracingSearchField::EventType, typ.code()));\n                }\n                if let Some(queue_id) = params.parse::<u64>(\"queue_id\") {\n                    tracing_query.push(SearchFilter::eq(TracingSearchField::QueueId, queue_id));\n                }\n                if let Some(query) = params.get(\"filter\") {\n                    let mut buf = String::with_capacity(query.len());\n                    let mut in_quote = false;\n                    for ch in query.chars() {\n                        if ch.is_ascii_whitespace() {\n                            if in_quote {\n                                buf.push(' ');\n                            } else if !buf.is_empty() {\n                                tracing_query.push(SearchFilter::has_keyword(\n                                    TracingSearchField::Keywords,\n                                    buf,\n                                ));\n                                buf = String::new();\n                            }\n                        } else if ch == '\"' {\n                            buf.push(ch);\n                            if in_quote {\n                                if !buf.is_empty() {\n                                    tracing_query.push(SearchFilter::has_keyword(\n                                        TracingSearchField::Keywords,\n                                        buf,\n                                    ));\n                                    buf = String::new();\n                                }\n                                in_quote = false;\n                            } else {\n                                in_quote = true;\n                            }\n                        } else {\n                            buf.push(ch);\n                        }\n                    }\n                    if !buf.is_empty() {\n                        tracing_query\n                            .push(SearchFilter::has_keyword(TracingSearchField::Keywords, buf));\n                    }\n                }\n                let values = params.get(\"values\").is_some();\n                if let Some(before) = params\n                    .parse::<Timestamp>(\"before\")\n                    .map(|t| t.into_inner())\n                    .and_then(SnowflakeIdGenerator::from_timestamp)\n                {\n                    tracing_query.push(SearchFilter::lt(SearchField::Id, before));\n                }\n                if let Some(after) = params\n                    .parse::<Timestamp>(\"after\")\n                    .map(|t| t.into_inner())\n                    .and_then(SnowflakeIdGenerator::from_timestamp)\n                {\n                    tracing_query.push(SearchFilter::gt(SearchField::Id, after));\n                }\n                if !tracing_query.iter().any(|f| {\n                    matches!(\n                        f,\n                        SearchFilter::Operator {\n                            field: SearchField::Tracing(\n                                TracingSearchField::Keywords | TracingSearchField::QueueId\n                            ) | SearchField::Id,\n                            ..\n                        }\n                    )\n                }) {\n                    tracing_query.push(SearchFilter::gt(\n                        SearchField::Id,\n                        SnowflakeIdGenerator::from_timestamp(now() - 86400).unwrap_or_default(),\n                    ));\n                }\n\n                tracing_query.push(SearchFilter::End);\n\n                let store = &self\n                    .core\n                    .enterprise\n                    .as_ref()\n                    .and_then(|e| e.trace_store.as_ref())\n                    .ok_or_else(|| manage::unsupported(\"No tracing store has been configured\"))?\n                    .store;\n\n                let span_ids = self\n                    .search_store()\n                    .query_global(\n                        SearchQuery::new(SearchIndex::Tracing)\n                            .with_filters(tracing_query)\n                            .with_comparator(SearchComparator::Field {\n                                field: SearchField::Id,\n                                ascending: false,\n                            }),\n                    )\n                    .await?;\n\n                let (total, span_ids) = if limit > 0 {\n                    let offset = page.saturating_sub(1) * limit;\n                    (\n                        span_ids.len(),\n                        span_ids.into_iter().skip(offset).take(limit).collect(),\n                    )\n                } else {\n                    (span_ids.len(), span_ids)\n                };\n\n                if values && !span_ids.is_empty() {\n                    let mut values = Vec::with_capacity(span_ids.len());\n\n                    for span_id in span_ids {\n                        for event in store.get_span(span_id).await? {\n                            if matches!(\n                                event.inner.typ,\n                                EventType::Delivery(DeliveryEvent::AttemptStart)\n                                    | EventType::Queue(\n                                        QueueEvent::QueueMessage\n                                            | QueueEvent::QueueMessageAuthenticated\n                                    )\n                            ) {\n                                values.push(event);\n                                break;\n                            }\n                        }\n                    }\n\n                    Ok(JsonResponse::new(json!({\n                            \"data\": {\n                                \"items\": JsonEventSerializer::new(values).with_spans(),\n                                \"total\": total,\n                            },\n                    }))\n                    .into_http_response())\n                } else {\n                    Ok(JsonResponse::new(json!({\n                            \"data\": {\n                                \"items\": span_ids,\n                                \"total\": total,\n                            },\n                    }))\n                    .into_http_response())\n                }\n            }\n            (\"traces\", Some(\"live\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::TracingLive)?;\n\n                let mut key_filters = AHashMap::new();\n                let mut filter = None;\n\n                for (key, value) in params.into_inner() {\n                    if key == \"filter\" {\n                        filter = value.into_owned().into();\n                    } else if let Some(key) = Key::try_parse(key.to_ascii_lowercase().as_str()) {\n                        key_filters.insert(key, value.into_owned());\n                    }\n                }\n\n                let (_, mut rx) = SubscriberBuilder::new(\"live-tracer\".to_string())\n                    .with_interests(Box::new(Bitset::all()))\n                    .with_lossy(false)\n                    .register();\n                let throttle = Duration::from_secs(1);\n                let ping_interval = Duration::from_secs(30);\n                let ping_payload = Bytes::from(format!(\n                    \"event: ping\\ndata: {{\\\"interval\\\": {}}}\\n\\n\",\n                    ping_interval.as_millis()\n                ));\n                let mut last_ping = Instant::now();\n                let mut events = Vec::new();\n                let mut active_span_ids = AHashSet::new();\n\n                Ok(HttpResponse::new(StatusCode::OK)\n                .with_content_type(\"text/event-stream\")\n                .with_cache_control(\"no-store\")\n                .with_stream_body(BoxBody::new(StreamBody::new(\n                    async_stream::stream! {\n                        let mut last_message = Instant::now() - throttle;\n                        let mut timeout = ping_interval;\n\n                        loop {\n                            match tokio::time::timeout(timeout, rx.recv()).await {\n                                Ok(Some(event_batch)) => {\n                                    for event in event_batch {\n                                        if (filter.is_none() && key_filters.is_empty())\n                                            || event\n                                                .span_id()\n                                                .is_some_and(|span_id| active_span_ids.contains(&span_id))\n                                        {\n                                            events.push(event);\n                                        } else {\n                                            let mut matched_keys = AHashSet::new();\n                                            for (key, value) in event\n                                                .keys\n                                                .iter()\n                                                .chain(event.inner.span.as_ref().map_or(([]).iter(), |s| s.keys.iter()))\n                                            {\n                                                if let Some(needle) = key_filters.get(key).or(filter.as_ref()) {\n                                                    let matches = match value {\n                                                        Value::String(haystack) => haystack.contains(needle),\n                                                        Value::Timestamp(haystack) => {\n                                                            DateTime::from_timestamp(*haystack as i64)\n                                                                .to_rfc3339()\n                                                                .contains(needle)\n                                                        }\n                                                        Value::Bool(true) => needle == \"true\",\n                                                        Value::Bool(false) => needle == \"false\",\n                                                        Value::Ipv4(haystack) => haystack.to_string().contains(needle),\n                                                        Value::Ipv6(haystack) => haystack.to_string().contains(needle),\n                                                        Value::Event(_) |\n                                                        Value::Array(_) |\n                                                        Value::UInt(_) |\n                                                        Value::Int(_) |\n                                                        Value::Float(_) |\n                                                        Value::Duration(_) |\n                                                        Value::Bytes(_) |\n                                                        Value::None => false,\n                                                    };\n\n                                                    if matches {\n                                                        matched_keys.insert(*key);\n                                                        if filter.is_some() || matched_keys.len() == key_filters.len() {\n                                                            if let Some(span_id) = event.span_id() {\n                                                                active_span_ids.insert(span_id);\n                                                            }\n                                                            events.push(event);\n                                                            break;\n                                                        }\n                                                    }\n                                                }\n                                            }\n                                        }\n                                    }\n                                }\n                                Ok(None) => {\n                                    break;\n                                }\n                                Err(_) => (),\n                            }\n\n                            timeout = if !events.is_empty() {\n                                let elapsed = last_message.elapsed();\n                                if elapsed >= throttle {\n                                    last_message = Instant::now();\n                                    yield Ok(Frame::data(Bytes::from(format!(\n                                        \"event: trace\\ndata: {}\\n\\n\",\n                                        serde_json::to_string(\n                                            &JsonEventSerializer::new(std::mem::take(&mut events))\n                                            .with_description()\n                                            .with_explanation()).unwrap_or_default()\n                                    ))));\n\n                                    ping_interval\n                                } else {\n                                    throttle - elapsed\n                                }\n                            } else {\n                                let elapsed = last_ping.elapsed();\n                                if elapsed >= ping_interval {\n                                    last_ping = Instant::now();\n                                    yield Ok(Frame::data(ping_payload.clone()));\n                                    ping_interval\n                                } else {\n                                    ping_interval - elapsed\n                                }\n                            };\n                        }\n                    },\n                ))))\n            }\n            (\"trace\", id, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::TracingGet)?;\n\n                let store = &self\n                    .core\n                    .enterprise\n                    .as_ref()\n                    .and_then(|e| e.trace_store.as_ref())\n                    .ok_or_else(|| manage::unsupported(\"No tracing store has been configured\"))?\n                    .store;\n\n                let mut events = Vec::new();\n                for span_id in id\n                    .or_else(|| params.get(\"id\"))\n                    .unwrap_or_default()\n                    .split(',')\n                {\n                    if let Ok(span_id) = span_id.parse::<u64>() {\n                        events.push(\n                            JsonEventSerializer::new(store.get_span(span_id).await?)\n                                .with_description()\n                                .with_explanation(),\n                        );\n                    } else {\n                        events.push(JsonEventSerializer::new(Vec::new()));\n                    }\n                }\n\n                if events.len() == 1 && id.is_some() {\n                    Ok(JsonResponse::new(json!({\n                            \"data\": events.into_iter().next().unwrap(),\n                    }))\n                    .into_http_response())\n                } else {\n                    Ok(JsonResponse::new(json!({\n                            \"data\": events,\n                    }))\n                    .into_http_response())\n                }\n            }\n            (\"live\", Some(\"tracing-token\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::TracingLive)?;\n\n                // Issue a live telemetry token valid for 60 seconds\n                Ok(JsonResponse::new(json!({\n                    \"data\": self.encode_access_token(GrantType::LiveTracing, account_id,  \"web\", 60).await?,\n            }))\n            .into_http_response())\n            }\n            (\"live\", Some(\"metrics-token\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MetricsLive)?;\n\n                // Issue a live telemetry token valid for 60 seconds\n                Ok(JsonResponse::new(json!({\n                    \"data\": self.encode_access_token(GrantType::LiveMetrics, account_id, \"web\", 60).await?,\n            }))\n            .into_http_response())\n            }\n            (\"metrics\", None, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MetricsList)?;\n\n                let before = params\n                    .parse::<Timestamp>(\"before\")\n                    .map(|t| t.into_inner())\n                    .unwrap_or(u64::MAX);\n                let after = params\n                    .parse::<Timestamp>(\"after\")\n                    .map(|t| t.into_inner())\n                    .unwrap_or(0);\n                let results = self\n                    .core\n                    .enterprise\n                    .as_ref()\n                    .and_then(|e| e.metrics_store.as_ref())\n                    .ok_or_else(|| {\n                        manage::error(\n                            \"No metrics store has been defined\",\n                            \"You need to configure a metrics store in order to use this feature.\"\n                                .into(),\n                        )\n                    })?\n                    .store\n                    .query_metrics(after, before)\n                    .await?;\n                let mut metrics = Vec::with_capacity(results.len());\n\n                for metric in results {\n                    metrics.push(match metric {\n                        Metric::Counter {\n                            id,\n                            timestamp,\n                            value,\n                        } => Metric::Counter {\n                            id: id.name(),\n                            timestamp: DateTime::from_timestamp(timestamp as i64).to_rfc3339(),\n                            value,\n                        },\n                        Metric::Histogram {\n                            id,\n                            timestamp,\n                            count,\n                            sum,\n                        } => Metric::Histogram {\n                            id: id.name(),\n                            timestamp: DateTime::from_timestamp(timestamp as i64).to_rfc3339(),\n                            count,\n                            sum,\n                        },\n                        Metric::Gauge {\n                            id,\n                            timestamp,\n                            value,\n                        } => Metric::Gauge {\n                            id: id.name(),\n                            timestamp: DateTime::from_timestamp(timestamp as i64).to_rfc3339(),\n                            value,\n                        },\n                    });\n                }\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": metrics,\n                }))\n                .into_http_response())\n            }\n            (\"metrics\", Some(\"live\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MetricsLive)?;\n\n                let interval = Duration::from_secs(\n                    params\n                        .parse::<u64>(\"interval\")\n                        .filter(|interval| *interval >= 1)\n                        .unwrap_or(30),\n                );\n                let mut event_types = AHashSet::new();\n                let mut metric_types = AHashSet::new();\n                for metric_name in params.get(\"metrics\").unwrap_or_default().split(',') {\n                    let metric_name = metric_name.trim();\n                    if !metric_name.is_empty() {\n                        if let Some(event_type) = EventType::try_parse(metric_name) {\n                            event_types.insert(event_type);\n                        } else if let Some(metric_type) = MetricType::try_parse(metric_name) {\n                            metric_types.insert(metric_type);\n                        }\n                    }\n                }\n\n                // Refresh expensive metrics\n                for metric_type in [\n                    MetricType::QueueCount,\n                    MetricType::UserCount,\n                    MetricType::DomainCount,\n                ] {\n                    if metric_types.contains(&metric_type) {\n                        let value = match metric_type {\n                            MetricType::QueueCount => self.total_queued_messages().await?,\n                            MetricType::UserCount => self.total_accounts().await?,\n                            MetricType::DomainCount => self.total_domains().await?,\n                            _ => unreachable!(),\n                        };\n                        Collector::update_gauge(metric_type, value);\n                    }\n                }\n\n                Ok(HttpResponse::new(StatusCode::OK)\n                .with_content_type(\"text/event-stream\")\n                .with_cache_control(\"no-store\")\n                .with_stream_body(BoxBody::new(StreamBody::new(\n                    async_stream::stream! {\n                        loop {\n                            let mut metrics = String::with_capacity(512);\n                            metrics.push_str(\"event: metrics\\ndata: [\");\n                            let mut is_first = true;\n\n                            for counter in Collector::collect_counters(true) {\n                                if event_types.is_empty() || event_types.contains(&counter.id()) {\n                                    if !is_first {\n                                        metrics.push(',');\n                                    } else {\n                                        is_first = false;\n                                    }\n                                    let _ = write!(\n                                        &mut metrics,\n                                        \"{{\\\"id\\\":\\\"{}\\\",\\\"type\\\":\\\"counter\\\",\\\"value\\\":{}}}\",\n                                        counter.id().name(),\n                                        counter.value()\n                                    );\n                                }\n                            }\n                            for gauge in Collector::collect_gauges(true) {\n                                if metric_types.is_empty() || metric_types.contains(&gauge.id()) {\n                                    if !is_first {\n                                        metrics.push(',');\n                                    } else {\n                                        is_first = false;\n                                    }\n                                    let _ = write!(\n                                        &mut metrics,\n                                        \"{{\\\"id\\\":\\\"{}\\\",\\\"type\\\":\\\"gauge\\\",\\\"value\\\":{}}}\",\n                                        gauge.id().name(),\n                                        gauge.get()\n                                    );\n                                }\n                            }\n                            for histogram in Collector::collect_histograms(true) {\n                                if metric_types.is_empty() || metric_types.contains(&histogram.id()) {\n                                    if !is_first {\n                                        metrics.push(',');\n                                    } else {\n                                        is_first = false;\n                                    }\n                                    let _ = write!(\n                                        &mut metrics,\n                                        \"{{\\\"id\\\":\\\"{}\\\",\\\"type\\\":\\\"histogram\\\",\\\"count\\\":{},\\\"sum\\\":{}}}\",\n                                        histogram.id().name(),\n                                        histogram.count(),\n                                        histogram.sum()\n                                    );\n                                }\n                            }\n                            metrics.push_str(\"]\\n\\n\");\n\n                            yield Ok(Frame::data(Bytes::from(metrics)));\n                            tokio::time::sleep(interval).await;\n                        }\n                    },\n                ))))\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/enterprise/undelete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};\nuse common::{Server, enterprise::undelete::DeletedItemType};\nuse directory::backend::internal::manage::ManageDirectory;\nuse email::{\n    mailbox::INBOX_ID,\n    message::ingest::{EmailIngest, IngestEmail, IngestSource},\n};\nuse http_proto::{request::decode_path_element, *};\nuse hyper::Method;\nuse mail_parser::{DateTime, MessageParser};\nuse serde_json::json;\nuse std::future::Future;\nuse std::str::FromStr;\nuse store::write::{BatchBuilder, BlobLink, BlobOp};\nuse trc::AddContext;\nuse types::{blob_hash::BlobHash, collection::Collection};\nuse utils::url_params::UrlParams;\n\n#[derive(serde::Deserialize, serde::Serialize, Debug)]\npub struct UndeleteRequest<H, C, T> {\n    pub hash: H,\n    pub collection: C,\n    #[serde(rename = \"restoreTime\")]\n    pub time: T,\n    #[serde(rename = \"cancelDeletion\")]\n    #[serde(default)]\n    pub cancel_deletion: Option<T>,\n}\n\n#[derive(serde::Serialize, serde::Deserialize, PartialEq, Eq, Debug)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum UndeleteResponse {\n    Success,\n    NotFound,\n    Error { reason: String },\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct DeletedBlobResponse {\n    pub hash: String,\n    pub size: u32,\n    #[serde(rename = \"deletedAt\")]\n    pub deleted_at: String,\n    #[serde(rename = \"expiresAt\")]\n    pub expires_at: String,\n    pub item: DeletedItemResponse,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum DeletedItemResponse {\n    Email {\n        from: Box<str>,\n        subject: Box<str>,\n        received_at: String,\n    },\n    FileNode {\n        name: Box<str>,\n    },\n    CalendarEvent {\n        title: Box<str>,\n        start_time: String,\n    },\n    ContactCard {\n        name: Box<str>,\n    },\n    SieveScript {\n        name: Box<str>,\n    },\n}\n\npub trait UndeleteApi: Sync + Send {\n    fn handle_undelete_api_request(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl UndeleteApi for Server {\n    async fn handle_undelete_api_request(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        session: &HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        match (path.get(2).copied(), req.method()) {\n            (Some(account_name), &Method::GET) => {\n                let account_name = decode_path_element(account_name);\n                let account_id = self\n                    .core\n                    .storage\n                    .data\n                    .get_principal_id(account_name.as_ref())\n                    .await?\n                    .ok_or_else(|| trc::ResourceEvent::NotFound.into_err())?;\n                let mut deleted = self.core.list_deleted(account_id).await?;\n\n                let params = UrlParams::new(req.uri().query());\n                let limit = params.parse::<usize>(\"limit\").unwrap_or_default();\n                let mut offset = params\n                    .parse::<usize>(\"page\")\n                    .unwrap_or_default()\n                    .saturating_sub(1)\n                    * limit;\n\n                // Sort ascending by deleted_at\n                let total = deleted.len();\n                deleted.sort_by(|a, b| a.item.deleted_at.cmp(&b.item.deleted_at));\n                let mut results = Vec::with_capacity(if limit > 0 { limit } else { total });\n\n                for blob in deleted {\n                    if offset == 0 {\n                        results.push(DeletedBlobResponse {\n                            hash: URL_SAFE_NO_PAD.encode(blob.hash.as_slice()),\n                            size: blob.item.size,\n                            deleted_at: DateTime::from_timestamp(blob.item.deleted_at as i64)\n                                .to_rfc3339(),\n                            expires_at: DateTime::from_timestamp(blob.expires_at as i64)\n                                .to_rfc3339(),\n                            item: match blob.item.typ {\n                                DeletedItemType::Email {\n                                    from,\n                                    subject,\n                                    received_at,\n                                } => DeletedItemResponse::Email {\n                                    from,\n                                    subject,\n                                    received_at: DateTime::from_timestamp(received_at as i64)\n                                        .to_rfc3339(),\n                                },\n                                DeletedItemType::FileNode { name } => {\n                                    DeletedItemResponse::FileNode { name }\n                                }\n                                DeletedItemType::CalendarEvent { title, start_time } => {\n                                    DeletedItemResponse::CalendarEvent {\n                                        title,\n                                        start_time: DateTime::from_timestamp(start_time as i64)\n                                            .to_rfc3339(),\n                                    }\n                                }\n                                DeletedItemType::ContactCard { name } => {\n                                    DeletedItemResponse::ContactCard { name }\n                                }\n                                DeletedItemType::SieveScript { name } => {\n                                    DeletedItemResponse::SieveScript { name }\n                                }\n                            },\n                        });\n                        if results.len() == limit {\n                            break;\n                        }\n                    } else {\n                        offset -= 1;\n                    }\n                }\n\n                Ok(JsonResponse::new(json!({\n                        \"data\":{\n                            \"items\": results,\n                            \"total\": total,\n                        },\n                }))\n                .into_http_response())\n            }\n            (Some(account_name), &Method::POST) => {\n                let account_name = decode_path_element(account_name);\n                let account_id = self\n                    .core\n                    .storage\n                    .data\n                    .get_principal_id(account_name.as_ref())\n                    .await?\n                    .ok_or_else(|| trc::ResourceEvent::NotFound.into_err())?;\n\n                let requests: Vec<UndeleteRequest<BlobHash, Collection, u64>> =\n                    match serde_json::from_slice::<\n                        Option<Vec<UndeleteRequest<String, String, String>>>,\n                    >(body.as_deref().unwrap_or_default())\n                    {\n                        Ok(Some(requests)) => requests\n                            .into_iter()\n                            .map(|request| {\n                                UndeleteRequest {\n                                    hash: BlobHash::try_from_hash_slice(\n                                        URL_SAFE_NO_PAD\n                                            .decode(request.hash.as_bytes())\n                                            .ok()?\n                                            .as_slice(),\n                                    )\n                                    .ok()?,\n                                    collection: Collection::from_str(request.collection.as_str())\n                                        .ok()?,\n                                    time: DateTime::parse_rfc3339(request.time.as_str())?\n                                        .to_timestamp()\n                                        as u64,\n                                    cancel_deletion: if let Some(cancel_deletion) =\n                                        request.cancel_deletion\n                                    {\n                                        (DateTime::parse_rfc3339(cancel_deletion.as_str())?\n                                            .to_timestamp()\n                                            as u64)\n                                            .into()\n                                    } else {\n                                        None\n                                    },\n                                }\n                                .into()\n                            })\n                            .collect::<Option<Vec<_>>>()\n                            .ok_or_else(|| trc::ResourceEvent::BadParameters.into_err())?,\n                        Ok(None) => {\n                            let deleted = self.core.list_deleted(account_id).await?;\n                            let mut results = Vec::with_capacity(deleted.len());\n                            for blob in deleted {\n                                results.push(UndeleteRequest {\n                                    hash: blob.hash,\n                                    collection: match blob.item.typ {\n                                        DeletedItemType::Email { .. } => Collection::Email,\n                                        DeletedItemType::FileNode { .. } => Collection::FileNode,\n                                        DeletedItemType::CalendarEvent { .. } => {\n                                            Collection::CalendarEvent\n                                        }\n                                        DeletedItemType::ContactCard { .. } => {\n                                            Collection::ContactCard\n                                        }\n                                        DeletedItemType::SieveScript { .. } => {\n                                            Collection::SieveScript\n                                        }\n                                    },\n                                    time: blob.item.deleted_at,\n                                    cancel_deletion: blob.expires_at.into(),\n                                });\n                            }\n                            results\n                        }\n                        Err(_) => {\n                            return Err(trc::ResourceEvent::BadParameters.into_err());\n                        }\n                    };\n\n                let access_token = self\n                    .get_access_token(account_id)\n                    .await\n                    .caused_by(trc::location!())?;\n                let mut results = Vec::with_capacity(requests.len());\n                let mut batch = BatchBuilder::new();\n                batch.with_account_id(account_id);\n                for request in requests {\n                    match request.collection {\n                        Collection::Email => {\n                            match self\n                                .blob_store()\n                                .get_blob(request.hash.as_slice(), 0..usize::MAX)\n                                .await?\n                            {\n                                Some(bytes) => {\n                                    match self\n                                        .email_ingest(IngestEmail {\n                                            raw_message: &bytes,\n                                            message: MessageParser::new().parse(&bytes),\n                                            blob_hash: Some(&request.hash),\n                                            access_token: access_token.as_ref(),\n                                            mailbox_ids: vec![INBOX_ID],\n                                            keywords: vec![],\n                                            received_at: request.time.into(),\n                                            source: IngestSource::Restore,\n                                            session_id: session.session_id,\n                                        })\n                                        .await\n                                    {\n                                        Ok(_) => {\n                                            results.push(UndeleteResponse::Success);\n                                            if let Some(cancel_deletion) = request.cancel_deletion {\n                                                batch\n                                                    .clear(BlobOp::Link {\n                                                        hash: request.hash.clone(),\n                                                        to: BlobLink::Temporary {\n                                                            until: cancel_deletion,\n                                                        },\n                                                    })\n                                                    .clear(BlobOp::Undelete {\n                                                        hash: request.hash,\n                                                        until: cancel_deletion,\n                                                    });\n                                            }\n                                        }\n                                        Err(mut err)\n                                            if err.matches(trc::EventType::MessageIngest(\n                                                trc::MessageIngestEvent::Error,\n                                            )) =>\n                                        {\n                                            results.push(UndeleteResponse::Error {\n                                                reason: err\n                                                    .take_value(trc::Key::Reason)\n                                                    .and_then(|v| v.into_string())\n                                                    .unwrap()\n                                                    .to_string(),\n                                            });\n                                        }\n                                        Err(err) => {\n                                            return Err(err.caused_by(trc::location!()));\n                                        }\n                                    }\n                                }\n                                None => {\n                                    results.push(UndeleteResponse::NotFound);\n                                }\n                            }\n                        }\n                        _ => {\n                            results.push(UndeleteResponse::Error {\n                                reason: \"Unsupported collection\".to_string(),\n                            });\n                        }\n                    }\n                }\n\n                // Commit batch\n                if !batch.is_empty() {\n                    self.core\n                        .storage\n                        .data\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": results,\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/log.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    fs::{self, File},\n    io,\n    path::Path,\n};\n\nuse chrono::DateTime;\nuse common::{Server, auth::AccessToken};\nuse directory::{Permission, backend::internal::manage};\nuse rev_lines::RevLines;\nuse serde::Serialize;\nuse serde_json::json;\nuse std::future::Future;\nuse tokio::sync::oneshot;\nuse utils::url_params::UrlParams;\n\nuse http_proto::*;\n\n#[derive(Serialize)]\nstruct LogEntry {\n    timestamp: String,\n    level: String,\n    event: String,\n    event_id: String,\n    details: String,\n}\n\npub trait LogManagement: Sync + Send {\n    fn handle_view_logs(\n        &self,\n        req: &HttpRequest,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl LogManagement for Server {\n    async fn handle_view_logs(\n        &self,\n        req: &HttpRequest,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        // Validate the access token\n        access_token.assert_has_permission(Permission::LogsView)?;\n\n        let path = self\n            .core\n            .metrics\n            .log_path\n            .clone()\n            .ok_or_else(|| manage::unsupported(\"Tracer log path not configured\"))?;\n\n        let params = UrlParams::new(req.uri().query());\n        let filter = params.get(\"filter\").unwrap_or_default().to_string();\n        let page: usize = params.parse(\"page\").unwrap_or(0);\n        let limit: usize = params.parse(\"limit\").unwrap_or(100);\n        let offset = page.saturating_sub(1) * limit;\n\n        // TODO: Use worker pool\n        let (tx, rx) = oneshot::channel();\n        tokio::task::spawn_blocking(move || {\n            let _ = tx.send(read_log_files(path, &filter, offset, limit));\n        });\n\n        let (total, items) = rx\n            .await\n            .map_err(|err| {\n                trc::EventType::Server(trc::ServerEvent::ThreadError)\n                    .reason(err)\n                    .caused_by(trc::location!())\n            })?\n            .map_err(|err| {\n                trc::ManageEvent::Error\n                    .reason(err)\n                    .details(\"Failed to read log files\")\n                    .caused_by(trc::location!())\n            })?;\n\n        Ok(JsonResponse::new(json!({\n            \"data\": {\n                \"items\": items,\n                \"total\": total,\n            },\n        }))\n        .into_http_response())\n    }\n}\n\nfn read_log_files(\n    path: impl AsRef<Path>,\n    filter: &str,\n    mut offset: usize,\n    limit: usize,\n) -> io::Result<(usize, Vec<LogEntry>)> {\n    let mut logs = fs::read_dir(path)?.collect::<Result<Vec<_>, _>>()?;\n    let mut total = 0;\n\n    // Sort the entries by file name in reverse order.\n    logs.sort_by_key(|b| std::cmp::Reverse(b.file_name()));\n\n    // Iterate and print the file names.\n    let mut entries = Vec::with_capacity(limit);\n    let mut logs = logs.into_iter();\n    while let Some(log) = logs.next() {\n        if log.file_type()?.is_file() {\n            let mut rev_lines = RevLines::new(File::open(log.path())?);\n\n            while let Some(line) = rev_lines.next() {\n                let line = line.map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))?;\n                if filter.is_empty() || line.contains(filter) {\n                    total += 1;\n                    if offset == 0 {\n                        if let Some(entry) = LogEntry::from_line(&line) {\n                            entries.push(entry);\n                            if entries.len() == limit {\n                                if rev_lines.next().is_some() || logs.next().is_some() {\n                                    total += limit;\n                                }\n\n                                return Ok((total, entries));\n                            }\n                        }\n                    } else {\n                        offset -= 1;\n                    }\n                }\n            }\n        }\n    }\n\n    Ok((total, entries))\n}\n\nimpl LogEntry {\n    fn from_line(line: &str) -> Option<Self> {\n        let (timestamp, rest) = line.split_once(' ')?;\n        let timestamp = DateTime::parse_from_rfc3339(timestamp).ok()?;\n        let (level, rest) = rest.trim().split_once(' ')?;\n        let (event, rest) = rest.trim().split_once(\" (\")?;\n        let (event_id, details) = rest.split_once(\")\")?;\n        Some(Self {\n            timestamp: timestamp.to_rfc3339_opts(chrono::SecondsFormat::Secs, true),\n            level: level.to_string(),\n            event: event.to_string(),\n            event_id: event_id.to_string(),\n            details: details.trim().to_string(),\n        })\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod crypto;\npub mod dkim;\npub mod dns;\npub mod log;\npub mod principal;\npub mod queue;\npub mod reload;\npub mod report;\npub mod settings;\npub mod spam;\npub mod stores;\npub mod troubleshoot;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\npub mod enterprise;\n\n#[cfg(feature = \"enterprise\")]\nuse enterprise::telemetry::TelemetryApi;\n// SPDX-SnippetEnd\n\nuse crate::auth::oauth::auth::OAuthApiHandler;\nuse common::{Server, auth::AccessToken};\nuse crypto::CryptoHandler;\nuse directory::{Permission, backend::internal::manage};\nuse dkim::DkimManagement;\nuse dns::DnsManagement;\nuse http_proto::{request::fetch_body, *};\nuse hyper::{Method, StatusCode, header};\nuse jmap::api::{ToJmapHttpResponse, ToRequestError};\nuse jmap_proto::error::request::RequestError;\nuse log::LogManagement;\nuse mail_parser::DateTime;\nuse principal::PrincipalManager;\nuse queue::QueueManagement;\nuse reload::ManageReload;\nuse report::ManageReports;\nuse serde::Serialize;\nuse settings::ManageSettings;\nuse spam::ManageSpamHandler;\nuse std::future::Future;\nuse std::{str::FromStr, sync::Arc};\nuse store::write::now;\nuse stores::ManageStore;\nuse troubleshoot::TroubleshootApi;\n\n#[derive(Serialize)]\n#[serde(tag = \"error\")]\n#[serde(rename_all = \"camelCase\")]\npub enum ManagementApiError<'x> {\n    FieldAlreadyExists {\n        field: &'x str,\n        value: &'x str,\n    },\n    FieldMissing {\n        field: &'x str,\n    },\n    NotFound {\n        item: &'x str,\n    },\n    Unsupported {\n        details: &'x str,\n    },\n    AssertFailed,\n    Other {\n        details: &'x str,\n        reason: Option<&'x str>,\n    },\n}\n\npub trait ManagementApi: Sync + Send {\n    fn handle_api_manage_request(\n        &self,\n        req: &mut HttpRequest,\n        access_token: Arc<AccessToken>,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl ManagementApi for Server {\n    #[allow(unused_variables)]\n    async fn handle_api_manage_request(\n        &self,\n        req: &mut HttpRequest,\n        access_token: Arc<AccessToken>,\n        session: &HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        let body = fetch_body(req, 1024 * 1024, session.session_id).await;\n        let path = req.uri().path().split('/').skip(2).collect::<Vec<_>>();\n\n        match path.first().copied().unwrap_or_default() {\n            \"queue\" => self.handle_manage_queue(req, path, &access_token).await,\n            \"settings\" => {\n                self.handle_manage_settings(req, path, body, &access_token)\n                    .await\n            }\n            \"reports\" => self.handle_manage_reports(req, path, &access_token).await,\n            \"principal\" => {\n                self.handle_manage_principal(req, path, body, &access_token)\n                    .await\n            }\n            \"dns\" => self.handle_manage_dns(req, path, &access_token).await,\n            \"store\" => {\n                self.handle_manage_store(req, path, body, session, &access_token)\n                    .await\n            }\n            \"reload\" => self.handle_manage_reload(req, path, &access_token).await,\n            \"dkim\" => {\n                self.handle_manage_dkim(req, path, body, &access_token)\n                    .await\n            }\n            \"update\" => self.handle_manage_update(req, path, &access_token).await,\n            \"logs\" if req.method() == Method::GET => {\n                self.handle_view_logs(req, &access_token).await\n            }\n            \"spam-filter\" => {\n                self.handle_manage_spam(req, path, body, session, &access_token)\n                    .await\n            }\n            \"restart\" if req.method() == Method::GET => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::Restart)?;\n\n                Err(manage::unsupported(\"Restart is not yet supported\"))\n            }\n            \"oauth\" => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::AuthenticateOauth)?;\n\n                self.handle_oauth_api_request(access_token, body).await\n            }\n            \"account\" => match (path.get(1).copied().unwrap_or_default(), req.method()) {\n                (\"crypto\", &Method::POST) => {\n                    // Validate the access token\n                    access_token.assert_has_permission(Permission::ManageEncryption)?;\n\n                    self.handle_crypto_post(access_token, body).await\n                }\n                (\"crypto\", &Method::GET) => {\n                    // Validate the access token\n                    access_token.assert_has_permission(Permission::ManageEncryption)?;\n\n                    self.handle_crypto_get(access_token).await\n                }\n                (\"auth\", &Method::GET) => {\n                    // Validate the access token\n                    access_token.assert_has_permission(Permission::ManagePasswords)?;\n\n                    self.handle_account_auth_get(access_token).await\n                }\n                (\"auth\", &Method::POST) => {\n                    // Validate the access token\n                    access_token.assert_has_permission(Permission::ManagePasswords)?;\n\n                    self.handle_account_auth_post(req, access_token, body).await\n                }\n                _ => Err(trc::ResourceEvent::NotFound.into_err()),\n            },\n            \"troubleshoot\" => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::Troubleshoot)?;\n\n                self.handle_troubleshoot_api_request(req, path, &access_token, body)\n                    .await\n            }\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            \"telemetry\" => {\n                // WARNING: TAMPERING WITH THIS FUNCTION IS STRICTLY PROHIBITED\n                // Any attempt to modify, bypass, or disable this license validation mechanism\n                // constitutes a severe violation of the Stalwart Enterprise License Agreement.\n                // Such actions may result in immediate termination of your license, legal action,\n                // and substantial financial penalties. Stalwart Labs LLC actively monitors for\n                // unauthorized modifications and will pursue all available legal remedies against\n                // violators to the fullest extent of the law, including but not limited to claims\n                // for copyright infringement, breach of contract, and fraud.\n\n                if self.core.is_enterprise_edition() {\n                    self.handle_telemetry_api_request(req, path, &access_token)\n                        .await\n                } else {\n                    Err(manage::enterprise())\n                }\n            }\n            // SPDX-SnippetEnd\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n\npub(super) struct FutureTimestamp(u64);\npub(super) struct Timestamp(u64);\n\nimpl FromStr for Timestamp {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        if let Some(dt) = DateTime::parse_rfc3339(s) {\n            Ok(Timestamp(dt.to_timestamp() as u64))\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl FromStr for FutureTimestamp {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        if let Some(dt) = DateTime::parse_rfc3339(s) {\n            let instant = dt.to_timestamp() as u64;\n            if instant >= now() {\n                return Ok(FutureTimestamp(instant));\n            }\n        }\n\n        Err(())\n    }\n}\n\nimpl FutureTimestamp {\n    pub fn into_inner(self) -> u64 {\n        self.0\n    }\n}\n\nimpl Timestamp {\n    pub fn into_inner(self) -> u64 {\n        self.0\n    }\n}\n\npub trait ToManageHttpResponse {\n    fn into_http_response(self) -> HttpResponse;\n}\n\nimpl ToManageHttpResponse for &trc::Error {\n    fn into_http_response(self) -> HttpResponse {\n        match self.as_ref() {\n            trc::EventType::Manage(cause) => {\n                match cause {\n                    trc::ManageEvent::MissingParameter => ManagementApiError::FieldMissing {\n                        field: self.value_as_str(trc::Key::Key).unwrap_or_default(),\n                    },\n                    trc::ManageEvent::AlreadyExists => ManagementApiError::FieldAlreadyExists {\n                        field: self.value_as_str(trc::Key::Key).unwrap_or_default(),\n                        value: self.value_as_str(trc::Key::Value).unwrap_or_default(),\n                    },\n                    trc::ManageEvent::NotFound => ManagementApiError::NotFound {\n                        item: self.value_as_str(trc::Key::Key).unwrap_or_default(),\n                    },\n                    trc::ManageEvent::NotSupported => ManagementApiError::Unsupported {\n                        details: self\n                            .value(trc::Key::Details)\n                            .or_else(|| self.value(trc::Key::Reason))\n                            .and_then(|v| v.as_str())\n                            .unwrap_or(\"Requested action is unsupported\"),\n                    },\n                    trc::ManageEvent::AssertFailed => ManagementApiError::AssertFailed,\n                    trc::ManageEvent::Error => ManagementApiError::Other {\n                        reason: self.value_as_str(trc::Key::Reason),\n                        details: self\n                            .value_as_str(trc::Key::Details)\n                            .unwrap_or(\"Unknown error\"),\n                    },\n                }\n            }\n            .into_http_response(),\n            trc::EventType::Auth(\n                trc::AuthEvent::Failed | trc::AuthEvent::Error | trc::AuthEvent::TokenExpired,\n            ) => HttpResponse::unauthorized(true),\n            _ => self.to_request_error().into_http_response(),\n        }\n    }\n}\n\npub trait UnauthorizedResponse {\n    fn unauthorized(include_realms: bool) -> Self;\n}\n\nimpl UnauthorizedResponse for HttpResponse {\n    fn unauthorized(include_realms: bool) -> Self {\n        (if include_realms {\n            HttpResponse::new(StatusCode::UNAUTHORIZED)\n                .with_header(header::WWW_AUTHENTICATE, \"Bearer realm=\\\"Stalwart Server\\\"\")\n                .with_header(header::WWW_AUTHENTICATE, \"Basic realm=\\\"Stalwart Server\\\"\")\n        } else {\n            HttpResponse::new(StatusCode::UNAUTHORIZED)\n        })\n        .with_content_type(\"application/problem+json\")\n        .with_text_body(serde_json::to_string(&RequestError::unauthorized()).unwrap_or_default())\n    }\n}\n\nimpl ManagementApiError<'_> {\n    fn into_http_response(self) -> HttpResponse {\n        JsonResponse::new(self).into_http_response()\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/principal.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::management::stores::destroy_account_data;\nuse common::{Server, auth::AccessToken};\nuse directory::{\n    DirectoryInner, Permission, PrincipalData, QueryBy, QueryParams, Type,\n    backend::internal::{\n        PrincipalAction, PrincipalField, PrincipalSet, PrincipalUpdate, PrincipalValue,\n        lookup::DirectoryStore,\n        manage::{\n            self, ChangedPrincipals, ManageDirectory, PrincipalList, UpdatePrincipal, not_found,\n        },\n    },\n};\nuse http_proto::{request::decode_path_element, *};\nuse hyper::{Method, header};\nuse serde_json::json;\nuse std::future::Future;\nuse std::sync::Arc;\nuse trc::AddContext;\nuse utils::url_params::UrlParams;\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum AccountAuthRequest {\n    SetPassword { password: String },\n    EnableOtpAuth { url: String },\n    DisableOtpAuth { url: Option<String> },\n    AddAppPassword { name: String, password: String },\n    RemoveAppPassword { name: Option<String> },\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct AccountAuthResponse {\n    #[serde(rename = \"otpEnabled\")]\n    pub otp_auth: bool,\n    #[serde(rename = \"appPasswords\")]\n    pub app_passwords: Vec<String>,\n}\n\npub trait PrincipalManager: Sync + Send {\n    fn handle_manage_principal(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_account_auth_get(\n        &self,\n        access_token: Arc<AccessToken>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_account_auth_post(\n        &self,\n        req: &HttpRequest,\n        access_token: Arc<AccessToken>,\n        body: Option<Vec<u8>>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn assert_supported_directory(&self, override_: bool) -> trc::Result<()>;\n}\n\nimpl PrincipalManager for Server {\n    async fn handle_manage_principal(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        match (path.get(1).copied(), req.method()) {\n            (None | Some(\"deploy\"), &Method::POST) => {\n                // Parse principal\n                let principal =\n                    serde_json::from_slice::<PrincipalSet>(body.as_deref().unwrap_or_default())\n                        .map_err(|err| {\n                            trc::EventType::Resource(trc::ResourceEvent::BadParameters)\n                                .from_json_error(err)\n                        })?;\n\n                // Validate the access token\n                access_token.assert_has_permission(match principal.typ() {\n                    Type::Individual => Permission::IndividualCreate,\n                    Type::Group => Permission::GroupCreate,\n                    Type::List => Permission::MailingListCreate,\n                    Type::Domain => Permission::DomainCreate,\n                    Type::Tenant => Permission::TenantCreate,\n                    Type::Role => Permission::RoleCreate,\n                    Type::ApiKey => Permission::ApiKeyCreate,\n                    Type::OauthClient => Permission::OauthClientCreate,\n                    Type::Resource | Type::Location | Type::Other => Permission::PrincipalCreate,\n                })?;\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n\n                #[cfg(feature = \"enterprise\")]\n                {\n                    if (matches!(principal.typ(), Type::Tenant)\n                        || principal.has_field(PrincipalField::Tenant))\n                        && !self.core.is_enterprise_edition()\n                    {\n                        return Err(manage::enterprise());\n                    }\n\n                    if matches!(principal.typ(), Type::Individual)\n                        && self.core.is_enterprise_edition()\n                        && !self.can_create_account().await?\n                    {\n                        return Err(manage::error(\n                            \"License account limit reached\",\n                            format!(\n                                \"Enterprise licensed account limit reached: {} accounts licensed.\",\n                                self.licensed_accounts()\n                            )\n                            .into(),\n                        ));\n                    }\n                }\n\n                // SPDX-SnippetEnd\n\n                // Make sure the current directory supports updates\n                if matches!(principal.typ(), Type::Individual) {\n                    self.assert_supported_directory(path.get(1).copied() == Some(\"deploy\"))?;\n                }\n\n                // Validate roles\n                let tenant_id = access_token.tenant.map(|t| t.id);\n                for name in principal\n                    .get_str_array(PrincipalField::Roles)\n                    .unwrap_or_default()\n                {\n                    if let Some(pinfo) = self\n                        .store()\n                        .get_principal_info(name)\n                        .await\n                        .caused_by(trc::location!())?\n                        .filter(|v| v.typ == Type::Role && v.has_tenant_access(tenant_id))\n                        .or_else(|| PrincipalField::Roles.map_internal_roles(name))\n                    {\n                        let role_permissions =\n                            self.get_role_permissions(pinfo.id).await?.finalize_as_ref();\n                        let mut allowed_permissions = role_permissions.clone();\n                        allowed_permissions.intersection(&access_token.permissions);\n                        if allowed_permissions != role_permissions {\n                            return Err(manage::error(\n                                \"Invalid role\",\n                                format!(\"Your account cannot grant the {name:?} role\").into(),\n                            ));\n                        }\n                    }\n                }\n\n                // Set default report domain if missing\n                let report_domain = if principal.typ() == Type::Domain\n                    && self\n                        .core\n                        .storage\n                        .config\n                        .get(\"report.domain\")\n                        .await\n                        .is_ok_and(|v| v.is_none())\n                {\n                    principal.name().to_lowercase().into()\n                } else {\n                    None\n                };\n\n                // Create principal\n                let result = self\n                    .core\n                    .storage\n                    .data\n                    .create_principal(principal, tenant_id, Some(&access_token.permissions))\n                    .await?;\n\n                // Set report domain\n                if let Some(report_domain) = report_domain\n                    && let Err(err) = self\n                        .core\n                        .storage\n                        .config\n                        .set([(\"report.domain\", report_domain)], true)\n                        .await\n                {\n                    trc::error!(err.details(\"Failed to set report domain\"));\n                }\n\n                // Increment revision\n                self.invalidate_principal_caches(result.changed_principals)\n                    .await;\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": result.id,\n                }))\n                .into_http_response())\n            }\n            (None, &Method::GET) => {\n                // List principal ids\n                let params = UrlParams::new(req.uri().query());\n                let filter = params.get(\"filter\");\n                let page: usize = params.parse(\"page\").unwrap_or(0);\n                let limit: usize = params.parse(\"limit\").unwrap_or(0);\n                let count = params.get(\"count\").is_some();\n\n                // Parse types\n                let mut types = Vec::new();\n                for typ in params\n                    .get(\"types\")\n                    .or_else(|| params.get(\"type\"))\n                    .unwrap_or_default()\n                    .split(',')\n                {\n                    if let Some(typ) = Type::parse(typ)\n                        && !types.contains(&typ)\n                    {\n                        types.push(typ);\n                    }\n                }\n\n                // Parse fields\n                let mut fields = Vec::new();\n                for field in params.get(\"fields\").unwrap_or_default().split(',') {\n                    if let Some(field) = PrincipalField::try_parse(field)\n                        && !fields.contains(&field)\n                    {\n                        fields.push(field);\n                    }\n                }\n\n                // Validate the access token\n                let validate_types = if !types.is_empty() {\n                    types.as_slice()\n                } else {\n                    &[\n                        Type::Individual,\n                        Type::Group,\n                        Type::List,\n                        Type::Domain,\n                        Type::Tenant,\n                        Type::Role,\n                        Type::Other,\n                        Type::ApiKey,\n                        Type::OauthClient,\n                    ]\n                };\n                for typ in validate_types {\n                    access_token.assert_has_permission(match typ {\n                        Type::Individual => Permission::IndividualList,\n                        Type::Group => Permission::GroupList,\n                        Type::List => Permission::MailingListList,\n                        Type::Domain => Permission::DomainList,\n                        Type::Tenant => Permission::TenantList,\n                        Type::Role => Permission::RoleList,\n                        Type::ApiKey => Permission::ApiKeyList,\n                        Type::OauthClient => Permission::OauthClientList,\n                        Type::Resource | Type::Location | Type::Other => Permission::PrincipalList,\n                    })?;\n                }\n\n                let mut tenant = access_token.tenant.map(|t| t.id);\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                {\n                    if self.core.is_enterprise_edition() {\n                        if tenant.is_none() {\n                            // Limit search to current tenant\n                            if let Some(tenant_name) = params.get(\"tenant\") {\n                                tenant = self\n                                    .core\n                                    .storage\n                                    .data\n                                    .get_principal_info(tenant_name)\n                                    .await?\n                                    .filter(|p| p.typ == Type::Tenant)\n                                    .map(|p| p.id);\n                            }\n                        }\n                    } else if types.contains(&Type::Tenant) {\n                        return Err(manage::enterprise());\n                    }\n                }\n                // SPDX-SnippetEnd\n\n                let principals = self\n                    .store()\n                    .list_principals(\n                        filter,\n                        tenant,\n                        &types,\n                        fields.len() != 1\n                            || fields.first().is_none_or(|v| v != &PrincipalField::Name),\n                        page,\n                        limit,\n                    )\n                    .await?;\n\n                let principals: PrincipalList<PrincipalSet> = if !count {\n                    let mut expanded = PrincipalList {\n                        items: Vec::with_capacity(principals.items.len()),\n                        total: principals.total,\n                    };\n\n                    for principal in principals.items {\n                        expanded\n                            .items\n                            .push(self.store().map_principal(principal, &fields).await?);\n                    }\n\n                    expanded\n                } else {\n                    PrincipalList {\n                        items: vec![],\n                        total: principals.total,\n                    }\n                };\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": principals,\n                }))\n                .into_http_response())\n            }\n            (None, &Method::DELETE) => {\n                // List principal ids\n                let params = UrlParams::new(req.uri().query());\n                let filter = params.get(\"filter\");\n                let typ = params.parse::<Type>(\"type\").ok_or_else(|| {\n                    trc::EventType::Resource(trc::ResourceEvent::BadParameters)\n                        .into_err()\n                        .details(\"Invalid type\")\n                })?;\n                if params.get(\"confirm\") != Some(\"true\") {\n                    return Err(trc::EventType::Resource(trc::ResourceEvent::BadParameters)\n                        .into_err()\n                        .details(\"Missing confirmation parameter\"));\n                }\n\n                // Validate the access token\n                access_token.assert_has_permission(match typ {\n                    Type::Individual => Permission::IndividualDelete,\n                    Type::Group => Permission::GroupDelete,\n                    Type::List => Permission::MailingListDelete,\n                    Type::Domain => Permission::DomainDelete,\n                    Type::Tenant => Permission::TenantDelete,\n                    Type::Role => Permission::RoleDelete,\n                    Type::ApiKey => Permission::ApiKeyDelete,\n                    Type::OauthClient => Permission::OauthClientDelete,\n                    Type::Resource | Type::Location | Type::Other => Permission::PrincipalDelete,\n                })?;\n\n                let mut tenant = access_token.tenant.map(|t| t.id);\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                {\n                    if self.core.is_enterprise_edition() {\n                        if tenant.is_none() {\n                            // Limit search to current tenant\n                            if let Some(tenant_name) = params.get(\"tenant\") {\n                                tenant = self\n                                    .core\n                                    .storage\n                                    .data\n                                    .get_principal_info(tenant_name)\n                                    .await?\n                                    .filter(|p| p.typ == Type::Tenant)\n                                    .map(|p| p.id);\n                            }\n                        }\n                    } else if typ == Type::Tenant {\n                        return Err(manage::enterprise());\n                    }\n                }\n                // SPDX-SnippetEnd\n\n                let principals = self\n                    .store()\n                    .list_principals(filter, tenant, &[typ], false, 0, 0)\n                    .await?;\n\n                let found = !principals.items.is_empty();\n                if found {\n                    let server = self.clone();\n                    tokio::spawn(async move {\n                        for principal in principals.items {\n                            // Delete account\n                            match server\n                                .store()\n                                .delete_principal(QueryBy::Id(principal.id()))\n                                .await\n                            {\n                                Ok(changed_principals) => {\n                                    // Increment revision\n                                    server.invalidate_principal_caches(changed_principals).await;\n                                }\n                                Err(err) => {\n                                    trc::error!(err.details(\"Failed to delete principal\"));\n                                    continue;\n                                }\n                            }\n\n                            if let Err(err) = destroy_account_data(\n                                &server,\n                                principal.id(),\n                                matches!(typ, Type::Individual | Type::Group),\n                            )\n                            .await\n                            {\n                                trc::error!(err.details(\"Failed to delete principal\"));\n                            }\n                        }\n                    });\n                }\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": found,\n                }))\n                .into_http_response())\n            }\n            (Some(name), method) => {\n                // Fetch, update or delete principal\n                let name = decode_path_element(name);\n                let (account_id, typ) = self\n                    .core\n                    .storage\n                    .data\n                    .get_principal_info(name.as_ref())\n                    .await?\n                    .filter(|p| p.has_tenant_access(access_token.tenant.map(|t| t.id)))\n                    .map(|p| (p.id, p.typ))\n                    .ok_or_else(|| not_found(name.to_string()))?;\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n\n                #[cfg(feature = \"enterprise\")]\n                {\n                    if matches!(typ, Type::Tenant) && !self.core.is_enterprise_edition() {\n                        return Err(manage::enterprise());\n                    }\n                }\n\n                // SPDX-SnippetEnd\n\n                match *method {\n                    Method::GET => {\n                        // Validate the access token\n                        access_token.assert_has_permission(match typ {\n                            Type::Individual => Permission::IndividualGet,\n                            Type::Group => Permission::GroupGet,\n                            Type::List => Permission::MailingListGet,\n                            Type::Domain => Permission::DomainGet,\n                            Type::Tenant => Permission::TenantGet,\n                            Type::Role => Permission::RoleGet,\n                            Type::ApiKey => Permission::ApiKeyGet,\n                            Type::OauthClient => Permission::OauthClientGet,\n                            Type::Resource | Type::Location | Type::Other => {\n                                Permission::PrincipalGet\n                            }\n                        })?;\n\n                        let principal = self\n                            .store()\n                            .query(QueryParams::id(account_id).with_return_member_of(true))\n                            .await?\n                            .ok_or_else(|| trc::ManageEvent::NotFound.into_err())?;\n\n                        // Map fields\n                        let principal = self\n                            .core\n                            .storage\n                            .data\n                            .map_principal(principal, &[])\n                            .await\n                            .caused_by(trc::location!())?;\n\n                        Ok(JsonResponse::new(json!({\n                                \"data\": principal,\n                        }))\n                        .into_http_response())\n                    }\n                    Method::DELETE => {\n                        // Validate the access token\n                        access_token.assert_has_permission(match typ {\n                            Type::Individual => Permission::IndividualDelete,\n                            Type::Group => Permission::GroupDelete,\n                            Type::List => Permission::MailingListDelete,\n                            Type::Domain => Permission::DomainDelete,\n                            Type::Tenant => Permission::TenantDelete,\n                            Type::Role => Permission::RoleDelete,\n                            Type::ApiKey => Permission::ApiKeyDelete,\n                            Type::OauthClient => Permission::OauthClientDelete,\n                            Type::Resource | Type::Location | Type::Other => {\n                                Permission::PrincipalDelete\n                            }\n                        })?;\n\n                        // Delete account\n                        let changed_principals = self\n                            .store()\n                            .delete_principal(QueryBy::Id(account_id))\n                            .await?;\n\n                        if let Err(err) = destroy_account_data(\n                            self,\n                            account_id,\n                            matches!(typ, Type::Individual | Type::Group),\n                        )\n                        .await\n                        {\n                            trc::error!(err.details(\"Failed to delete principal\"));\n                        }\n\n                        // Increment revision\n                        self.invalidate_principal_caches(changed_principals).await;\n\n                        Ok(JsonResponse::new(json!({\n                            \"data\": (),\n                        }))\n                        .into_http_response())\n                    }\n                    Method::PATCH => {\n                        // Validate the access token\n                        let permission_needed = match typ {\n                            Type::Individual => Permission::IndividualUpdate,\n                            Type::Group => Permission::GroupUpdate,\n                            Type::List => Permission::MailingListUpdate,\n                            Type::Domain => Permission::DomainUpdate,\n                            Type::Tenant => Permission::TenantUpdate,\n                            Type::Role => Permission::RoleUpdate,\n                            Type::ApiKey => Permission::ApiKeyUpdate,\n                            Type::OauthClient => Permission::OauthClientUpdate,\n                            Type::Resource | Type::Location | Type::Other => {\n                                Permission::PrincipalUpdate\n                            }\n                        };\n                        access_token.assert_has_permission(permission_needed)?;\n\n                        let changes = serde_json::from_slice::<Vec<PrincipalUpdate>>(\n                            body.as_deref().unwrap_or_default(),\n                        )\n                        .map_err(|err| {\n                            trc::EventType::Resource(trc::ResourceEvent::BadParameters)\n                                .from_json_error(err)\n                        })?;\n\n                        // Validate changes\n                        let mut invalidate_logo_cache = false;\n                        for change in &changes {\n                            match change.field {\n                                PrincipalField::Secrets\n                                | PrincipalField::Name\n                                | PrincipalField::Emails\n                                | PrincipalField::Quota\n                                | PrincipalField::UsedQuota\n                                | PrincipalField::Description\n                                | PrincipalField::Type\n                                | PrincipalField::MemberOf\n                                | PrincipalField::Members\n                                | PrincipalField::Lists\n                                | PrincipalField::Urls\n                                | PrincipalField::ExternalMembers\n                                | PrincipalField::Locale => (),\n                                PrincipalField::Picture => {\n                                    invalidate_logo_cache |=\n                                        matches!(typ, Type::Domain | Type::Tenant);\n                                }\n                                PrincipalField::Tenant => {\n                                    // Tenants are not allowed to change their tenantId\n                                    if access_token.tenant.is_some() {\n                                        trc::bail!(\n                                            trc::SecurityEvent::Unauthorized\n                                                .into_err()\n                                                .details(permission_needed.name())\n                                                .ctx(\n                                                    trc::Key::Reason,\n                                                    \"Tenants cannot change their tenantId\"\n                                                )\n                                        );\n                                    }\n                                }\n                                PrincipalField::Roles\n                                | PrincipalField::EnabledPermissions\n                                | PrincipalField::DisabledPermissions => {\n                                    if change.field == PrincipalField::Roles\n                                        && matches!(\n                                            change.action,\n                                            PrincipalAction::AddItem | PrincipalAction::Set\n                                        )\n                                    {\n                                        let roles = match &change.value {\n                                            PrincipalValue::String(v) => std::slice::from_ref(v),\n                                            PrincipalValue::StringList(vec) => vec,\n                                            PrincipalValue::Integer(_)\n                                            | PrincipalValue::IntegerList(_) => continue,\n                                        };\n\n                                        // Validate roles\n                                        let tenant_id = access_token.tenant.map(|t| t.id);\n                                        for name in roles {\n                                            if let Some(pinfo) = self\n                                                .store()\n                                                .get_principal_info(name)\n                                                .await\n                                                .caused_by(trc::location!())?\n                                                .filter(|v| {\n                                                    v.typ == Type::Role\n                                                        && v.has_tenant_access(tenant_id)\n                                                })\n                                                .or_else(|| {\n                                                    PrincipalField::Roles.map_internal_roles(name)\n                                                })\n                                            {\n                                                let role_permissions = self\n                                                    .get_role_permissions(pinfo.id)\n                                                    .await?\n                                                    .finalize_as_ref();\n                                                let mut allowed_permissions =\n                                                    role_permissions.clone();\n                                                allowed_permissions\n                                                    .intersection(&access_token.permissions);\n                                                if allowed_permissions != role_permissions {\n                                                    return Err(manage::error(\n                                                        \"Invalid role\",\n                                                        format!(\"Your account cannot grant the {name:?} role\").into(),\n                                                    ));\n                                                }\n                                            }\n                                        }\n                                    }\n                                }\n                            }\n                        }\n\n                        // Update principal\n                        let changed_principals = self\n                            .core\n                            .storage\n                            .data\n                            .update_principal(\n                                UpdatePrincipal::by_id(account_id)\n                                    .with_updates(changes)\n                                    .with_tenant(access_token.tenant.map(|t| t.id))\n                                    .with_allowed_permissions(&access_token.permissions),\n                            )\n                            .await?;\n\n                        // Increment revision\n                        self.invalidate_principal_caches(changed_principals).await;\n\n                        // Invalidate logo cache if needed\n                        if invalidate_logo_cache {\n                            self.inner.data.logos.lock().clear();\n                        }\n\n                        Ok(JsonResponse::new(json!({\n                            \"data\": (),\n                        }))\n                        .into_http_response())\n                    }\n                    _ => Err(trc::ResourceEvent::NotFound.into_err()),\n                }\n            }\n\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n\n    async fn handle_account_auth_get(\n        &self,\n        access_token: Arc<AccessToken>,\n    ) -> trc::Result<HttpResponse> {\n        let mut response = AccountAuthResponse {\n            otp_auth: false,\n            app_passwords: Vec::new(),\n        };\n\n        if access_token.primary_id() != u32::MAX {\n            let principal = self\n                .directory()\n                .query(QueryParams::id(access_token.primary_id()).with_return_member_of(false))\n                .await?\n                .ok_or_else(|| trc::ManageEvent::NotFound.into_err())?;\n\n            for data in &principal.data {\n                match data {\n                    PrincipalData::OtpAuth(_) => {\n                        response.otp_auth = true;\n                    }\n                    PrincipalData::AppPassword(secret) => {\n                        if let Some((app_name, _)) =\n                            secret.strip_prefix(\"$app$\").and_then(|s| s.split_once('$'))\n                        {\n                            response.app_passwords.push(app_name.into());\n                        }\n                    }\n                    _ => {}\n                }\n            }\n        }\n\n        Ok(JsonResponse::new(json!({\n            \"data\": response,\n        }))\n        .into_http_response())\n    }\n\n    async fn handle_account_auth_post(\n        &self,\n        req: &HttpRequest,\n        access_token: Arc<AccessToken>,\n        body: Option<Vec<u8>>,\n    ) -> trc::Result<HttpResponse> {\n        // Parse request\n        let requests =\n            serde_json::from_slice::<Vec<AccountAuthRequest>>(body.as_deref().unwrap_or_default())\n                .map_err(|err| {\n                    trc::EventType::Resource(trc::ResourceEvent::BadParameters).from_json_error(err)\n                })?;\n\n        if requests.is_empty() {\n            return Err(trc::EventType::Resource(trc::ResourceEvent::BadParameters)\n                .into_err()\n                .details(\"Empty request\"));\n        }\n\n        // Make sure the user authenticated using Basic auth\n        if requests.iter().any(|r| {\n            matches!(\n                r,\n                AccountAuthRequest::DisableOtpAuth { .. }\n                    | AccountAuthRequest::EnableOtpAuth { .. }\n                    | AccountAuthRequest::SetPassword { .. }\n            )\n        }) && req\n            .headers()\n            .get(header::AUTHORIZATION)\n            .and_then(|h| h.to_str().ok())\n            .is_none_or(|header| !header.to_lowercase().starts_with(\"basic \"))\n        {\n            return Err(manage::error(\n                \"Password changes only allowed using Basic auth\",\n                None::<u32>,\n            ));\n        }\n\n        // Handle Fallback admin password changes\n        if access_token.primary_id() == u32::MAX {\n            match requests.into_iter().next().unwrap() {\n                AccountAuthRequest::SetPassword { password } => {\n                    self.core\n                        .storage\n                        .config\n                        .set(\n                            [(\"authentication.fallback-admin.secret\", password.to_string())],\n                            true,\n                        )\n                        .await?;\n\n                    // Increment revision\n                    self.invalidate_principal_caches(ChangedPrincipals::from_change(\n                        access_token.primary_id(),\n                        Type::Individual,\n                        PrincipalField::Secrets,\n                    ))\n                    .await;\n\n                    return Ok(JsonResponse::new(json!({\n                        \"data\": (),\n                    }))\n                    .into_http_response());\n                }\n                _ => {\n                    return Err(manage::error(\n                        \"Fallback administrator accounts do not support 2FA or AppPasswords\",\n                        None::<u32>,\n                    ));\n                }\n            }\n        }\n\n        // Make sure the current directory supports updates\n        if requests.iter().any(|r| {\n            matches!(\n                r,\n                AccountAuthRequest::SetPassword { .. }\n                    | AccountAuthRequest::EnableOtpAuth { .. }\n                    | AccountAuthRequest::DisableOtpAuth { .. }\n            )\n        }) {\n            self.assert_supported_directory(false)?;\n        }\n\n        // Build actions\n        let mut actions = Vec::with_capacity(requests.len());\n        for request in requests {\n            let (action, secret) = match request {\n                AccountAuthRequest::SetPassword { password } => {\n                    actions.push(PrincipalUpdate {\n                        action: PrincipalAction::RemoveItem,\n                        field: PrincipalField::Secrets,\n                        value: PrincipalValue::String(String::new()),\n                    });\n\n                    (PrincipalAction::AddItem, password)\n                }\n                AccountAuthRequest::EnableOtpAuth { url } => (PrincipalAction::AddItem, url),\n                AccountAuthRequest::DisableOtpAuth { url } => (\n                    PrincipalAction::RemoveItem,\n                    url.unwrap_or_else(|| \"otpauth://\".into()),\n                ),\n                AccountAuthRequest::AddAppPassword { name, password } => {\n                    (PrincipalAction::AddItem, format!(\"$app${name}${password}\"))\n                }\n                AccountAuthRequest::RemoveAppPassword { name } => (\n                    PrincipalAction::RemoveItem,\n                    format!(\"$app${}\", name.unwrap_or_default()),\n                ),\n            };\n\n            actions.push(PrincipalUpdate {\n                action,\n                field: PrincipalField::Secrets,\n                value: PrincipalValue::String(secret),\n            });\n        }\n\n        // Update password\n        let changed_principals = self\n            .core\n            .storage\n            .data\n            .update_principal(\n                UpdatePrincipal::by_id(access_token.primary_id())\n                    .with_updates(actions)\n                    .with_tenant(access_token.tenant.map(|t| t.id)),\n            )\n            .await?;\n\n        // Increment revision\n        self.invalidate_principal_caches(changed_principals).await;\n\n        Ok(JsonResponse::new(json!({\n            \"data\": (),\n        }))\n        .into_http_response())\n    }\n\n    fn assert_supported_directory(&self, override_: bool) -> trc::Result<()> {\n        let class = match &self.core.storage.directory.store {\n            DirectoryInner::Internal(_) => return Ok(()),\n            DirectoryInner::Ldap(_) => \"LDAP\",\n            DirectoryInner::Sql(_) => \"SQL\",\n            DirectoryInner::Imap(_) => \"IMAP\",\n            DirectoryInner::Smtp(_) => \"SMTP\",\n            DirectoryInner::Memory(_) => \"In-Memory\",\n            DirectoryInner::OpenId(_) => \"OpenID\",\n        };\n\n        if !override_ {\n            Err(manage::unsupported(format!(\n                concat!(\n                    \"{} directory cannot be managed. \",\n                    \"Only internal directories support inserts \",\n                    \"and update operations.\"\n                ),\n                class\n            )))\n        } else {\n            Ok(())\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/queue.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::FutureTimestamp;\nuse base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};\nuse common::{\n    Server,\n    auth::AccessToken,\n    config::smtp::queue::{ArchivedQueueExpiry, QueueExpiry, QueueName},\n    ipc::QueueEvent,\n};\nuse directory::{Permission, Type, backend::internal::manage::ManageDirectory};\nuse http_proto::{request::decode_path_element, *};\nuse hyper::Method;\nuse mail_auth::{\n    dmarc::URI,\n    mta_sts::ReportUri,\n    report::{self, tlsrpt::TlsReport},\n};\nuse mail_parser::DateTime;\nuse serde::{Deserializer, Serializer};\nuse serde_json::json;\nuse smtp::{\n    queue::{\n        self, ArchivedMessage, ArchivedStatus, ErrorDetails, QueueId, Status, spool::SmtpSpool,\n    },\n    reporting::{dmarc::DmarcReporting, tls::TlsReporting},\n};\nuse std::{future::Future, sync::atomic::Ordering};\nuse store::{\n    Deserialize, IterateParams, ValueKey,\n    write::{\n        AlignedBytes, Archive, QueueClass, ReportEvent, ValueClass, key::DeserializeBigEndian, now,\n    },\n};\nuse trc::AddContext;\nuse utils::url_params::UrlParams;\n\n#[derive(Debug, serde::Serialize, serde::Deserialize, PartialEq, Eq)]\npub struct Message {\n    pub id: QueueId,\n\n    pub return_path: String,\n\n    pub recipients: Vec<Recipient>,\n\n    #[serde(deserialize_with = \"deserialize_datetime\")]\n    #[serde(serialize_with = \"serialize_datetime\")]\n    pub created: DateTime,\n\n    pub size: u64,\n\n    #[serde(skip_serializing_if = \"is_zero\")]\n    #[serde(default)]\n    pub priority: i16,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub env_id: Option<String>,\n\n    pub blob_hash: String,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize, PartialEq, Eq)]\npub struct Recipient {\n    pub address: String,\n    pub queue: String,\n    pub status: Status<String, String>,\n    pub retry_num: u32,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(deserialize_with = \"deserialize_maybe_datetime\")]\n    #[serde(serialize_with = \"serialize_maybe_datetime\")]\n    pub next_retry: Option<DateTime>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(deserialize_with = \"deserialize_maybe_datetime\")]\n    #[serde(serialize_with = \"serialize_maybe_datetime\")]\n    pub next_notify: Option<DateTime>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(deserialize_with = \"deserialize_maybe_datetime\")]\n    #[serde(serialize_with = \"serialize_maybe_datetime\")]\n    pub expires: Option<DateTime>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub orcpt: Option<String>,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum Report {\n    Tls {\n        id: String,\n        domain: String,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        #[serde(serialize_with = \"serialize_datetime\")]\n        range_from: DateTime,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        #[serde(serialize_with = \"serialize_datetime\")]\n        range_to: DateTime,\n        report: TlsReport,\n        rua: Vec<ReportUri>,\n    },\n    Dmarc {\n        id: String,\n        domain: String,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        #[serde(serialize_with = \"serialize_datetime\")]\n        range_from: DateTime,\n        #[serde(deserialize_with = \"deserialize_datetime\")]\n        #[serde(serialize_with = \"serialize_datetime\")]\n        range_to: DateTime,\n        report: report::Report,\n        rua: Vec<URI>,\n    },\n}\n\npub trait QueueManagement: Sync + Send {\n    fn handle_manage_queue(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl QueueManagement for Server {\n    async fn handle_manage_queue(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        let params = UrlParams::new(req.uri().query());\n        let mut tenant_domains: Option<Vec<String>> = None;\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Limit to tenant domains\n        #[cfg(feature = \"enterprise\")]\n        if self.core.is_enterprise_edition()\n            && let Some(tenant) = access_token.tenant\n        {\n            tenant_domains = self\n                .core\n                .storage\n                .data\n                .list_principals(None, tenant.id.into(), &[Type::Domain], false, 0, 0)\n                .await\n                .map(|principals| {\n                    principals\n                        .items\n                        .into_iter()\n                        .map(|p| p.name)\n                        .collect::<Vec<_>>()\n                })\n                .caused_by(trc::location!())?\n                .into();\n        }\n\n        // SPDX-SnippetEnd\n\n        match (\n            path.get(1).copied().unwrap_or_default(),\n            path.get(2).copied().map(decode_path_element),\n            req.method(),\n        ) {\n            (\"messages\", None, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueList)?;\n\n                let result = fetch_queued_messages(self, &params, &tenant_domains).await?;\n\n                let queue_status = self.inner.data.queue_status.load(Ordering::Relaxed);\n\n                Ok(if !result.values.is_empty() {\n                    JsonResponse::new(json!({\n                            \"data\":{\n                                \"items\": result.values,\n                                \"total\": result.total,\n                                \"status\": queue_status,\n                            },\n                    }))\n                } else {\n                    JsonResponse::new(json!({\n                            \"data\": {\n                                \"items\": result.ids,\n                                \"total\":  result.total,\n                                \"status\": queue_status,\n                            },\n                    }))\n                }\n                .into_http_response())\n            }\n            (\"messages\", Some(queue_id), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueGet)?;\n\n                let queue_id = queue_id.parse().unwrap_or_default();\n                if let Some(message_) = self.read_message_archive(queue_id).await? {\n                    let message = message_.unarchive::<queue::Message>()?;\n                    if message.is_tenant_domain(&tenant_domains) {\n                        return Ok(JsonResponse::new(json!({\n                                \"data\": Message::from_archive(queue_id, message),\n                        }))\n                        .into_http_response());\n                    }\n                }\n                Err(trc::ResourceEvent::NotFound.into_err())\n            }\n            (\"messages\", None, &Method::PATCH) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueUpdate)?;\n\n                let time = params\n                    .parse::<FutureTimestamp>(\"at\")\n                    .map(|t| t.into_inner())\n                    .unwrap_or_else(now);\n                let result = fetch_queued_messages(self, &params, &tenant_domains).await?;\n\n                let found = !result.ids.is_empty();\n                if found {\n                    let server = self.clone();\n                    tokio::spawn(async move {\n                        for id in result.ids {\n                            if let Some(mut message) =\n                                server.read_message(id, QueueName::default()).await\n                            {\n                                let mut has_changes = false;\n\n                                for recipient in &mut message.message.recipients {\n                                    if matches!(\n                                        recipient.status,\n                                        Status::Scheduled | Status::TemporaryFailure(_)\n                                    ) {\n                                        recipient.retry.due = time;\n                                        if recipient\n                                            .expiration_time(message.message.created)\n                                            .is_some_and(|expires| expires > time)\n                                        {\n                                            recipient.expires =\n                                                QueueExpiry::Attempts(recipient.retry.inner + 10);\n                                        }\n                                        has_changes = true;\n                                    }\n                                }\n\n                                if has_changes {\n                                    message.save_changes(&server, None).await;\n                                }\n                            }\n                        }\n\n                        let _ = server.inner.ipc.queue_tx.send(QueueEvent::Refresh).await;\n                    });\n                }\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": found,\n                }))\n                .into_http_response())\n            }\n            (\"messages\", Some(queue_id), &Method::PATCH) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueUpdate)?;\n\n                let time = params\n                    .parse::<FutureTimestamp>(\"at\")\n                    .map(|t| t.into_inner())\n                    .unwrap_or_else(now);\n                let item = params.get(\"filter\");\n\n                if let Some(mut message) = self\n                    .read_message(queue_id.parse().unwrap_or_default(), QueueName::default())\n                    .await\n                    .filter(|message| {\n                        tenant_domains\n                            .as_ref()\n                            .is_none_or(|domains| message.has_domain(domains))\n                    })\n                {\n                    let mut found = false;\n\n                    for recipient in &mut message.message.recipients {\n                        if matches!(\n                            recipient.status,\n                            Status::Scheduled | Status::TemporaryFailure(_)\n                        ) && item\n                            .as_ref()\n                            .is_none_or(|item| recipient.address().contains(item))\n                        {\n                            recipient.retry.due = time;\n                            if recipient\n                                .expiration_time(message.message.created)\n                                .is_some_and(|expires| expires > time)\n                            {\n                                recipient.expires =\n                                    QueueExpiry::Attempts(recipient.retry.inner + 10);\n                            }\n                            found = true;\n                        }\n                    }\n\n                    if found {\n                        message.save_changes(self, None).await;\n                        let _ = self.inner.ipc.queue_tx.send(QueueEvent::Refresh).await;\n                    }\n\n                    Ok(JsonResponse::new(json!({\n                            \"data\": found,\n                    }))\n                    .into_http_response())\n                } else {\n                    Err(trc::ResourceEvent::NotFound.into_err())\n                }\n            }\n            (\"messages\", None, &Method::DELETE) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueDelete)?;\n\n                let result = fetch_queued_messages(self, &params, &tenant_domains).await?;\n\n                let found = !result.ids.is_empty();\n                if found {\n                    let server = self.clone();\n                    tokio::spawn(async move {\n                        let is_active = server.inner.data.queue_status.load(Ordering::Relaxed);\n\n                        if is_active {\n                            let _ = server\n                                .inner\n                                .ipc\n                                .queue_tx\n                                .send(QueueEvent::Paused(true))\n                                .await;\n                        }\n\n                        for id in result.ids {\n                            if let Some(message) =\n                                server.read_message(id, QueueName::default()).await\n                            {\n                                message.remove(&server, None).await;\n                            }\n                        }\n\n                        if is_active {\n                            let _ = server\n                                .inner\n                                .ipc\n                                .queue_tx\n                                .send(QueueEvent::Paused(false))\n                                .await;\n                        }\n                    });\n                }\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": found,\n                }))\n                .into_http_response())\n            }\n            (\"messages\", Some(queue_id), &Method::DELETE) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueDelete)?;\n\n                if let Some(mut message) = self\n                    .read_message(queue_id.parse().unwrap_or_default(), QueueName::default())\n                    .await\n                    .filter(|message| {\n                        tenant_domains\n                            .as_ref()\n                            .is_none_or(|domains| message.has_domain(domains))\n                    })\n                {\n                    let mut found = false;\n                    if let Some(item) = params.get(\"filter\") {\n                        // Cancel delivery for all recipients that match\n                        for rcpt in &mut message.message.recipients {\n                            if rcpt.address().contains(item) {\n                                rcpt.status = Status::PermanentFailure(ErrorDetails {\n                                    entity: \"localhost\".into(),\n                                    details: queue::Error::Io(\"Delivery canceled.\".into()),\n                                });\n                                found = true;\n                            }\n                        }\n                        if found {\n                            // Delete message if there are no pending deliveries\n                            if message.message.recipients.iter().any(|recipient| {\n                                matches!(\n                                    recipient.status,\n                                    Status::TemporaryFailure(_) | Status::Scheduled\n                                )\n                            }) {\n                                message.save_changes(self, None).await;\n                            } else {\n                                message.remove(self, None).await;\n                            }\n                        }\n                    } else {\n                        message.remove(self, None).await;\n                        found = true;\n                    }\n\n                    Ok(JsonResponse::new(json!({\n                            \"data\": found,\n                    }))\n                    .into_http_response())\n                } else {\n                    Err(trc::ResourceEvent::NotFound.into_err())\n                }\n            }\n            (\"reports\", None, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::OutgoingReportList)?;\n\n                let result = fetch_queued_reports(self, &params, &tenant_domains).await?;\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": {\n                            \"items\": result.ids.into_iter().map(|id| id.queue_id()).collect::<Vec<_>>(),\n                            \"total\": result.total,\n                        },\n                }))\n                .into_http_response())\n            }\n            (\"reports\", Some(report_id), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::OutgoingReportGet)?;\n\n                let mut result = None;\n                if let Some(report_id) = parse_queued_report_id(report_id.as_ref()) {\n                    match report_id {\n                        QueueClass::DmarcReportHeader(event)\n                            if tenant_domains.as_ref().is_none_or(|domains| {\n                                domains.iter().any(|dd| dd == &event.domain)\n                            }) =>\n                        {\n                            let mut rua = Vec::new();\n                            if let Some(report) = self\n                                .generate_dmarc_aggregate_report(&event, &mut rua, None, 0)\n                                .await?\n                            {\n                                result = Report::dmarc(event, report, rua).into();\n                            }\n                        }\n                        QueueClass::TlsReportHeader(event)\n                            if tenant_domains.as_ref().is_none_or(|domains| {\n                                domains.iter().any(|dd| dd == &event.domain)\n                            }) =>\n                        {\n                            let mut rua = Vec::new();\n                            if let Some(report) = self\n                                .generate_tls_aggregate_report(\n                                    std::slice::from_ref(&event),\n                                    &mut rua,\n                                    None,\n                                    0,\n                                )\n                                .await?\n                            {\n                                result = Report::tls(event, report, rua).into();\n                            }\n                        }\n                        _ => (),\n                    }\n                }\n\n                if let Some(result) = result {\n                    Ok(JsonResponse::new(json!({\n                            \"data\": result,\n                    }))\n                    .into_http_response())\n                } else {\n                    Err(trc::ResourceEvent::NotFound.into_err())\n                }\n            }\n            (\"reports\", None, &Method::DELETE) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::OutgoingReportDelete)?;\n\n                let result = fetch_queued_reports(self, &params, &tenant_domains).await?;\n                let found = !result.ids.is_empty();\n                if found {\n                    let server = self.clone();\n                    tokio::spawn(async move {\n                        for id in result.ids {\n                            match id {\n                                QueueClass::DmarcReportHeader(event) => {\n                                    server.delete_dmarc_report(event).await;\n                                }\n                                QueueClass::TlsReportHeader(event) => {\n                                    server.delete_tls_report(vec![event]).await;\n                                }\n                                _ => (),\n                            }\n                        }\n                    });\n                }\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": found,\n                }))\n                .into_http_response())\n            }\n            (\"reports\", Some(report_id), &Method::DELETE) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::OutgoingReportDelete)?;\n\n                if let Some(report_id) = parse_queued_report_id(report_id.as_ref()) {\n                    let result = match report_id {\n                        QueueClass::DmarcReportHeader(event)\n                            if tenant_domains.as_ref().is_none_or(|domains| {\n                                domains.iter().any(|dd| dd == &event.domain)\n                            }) =>\n                        {\n                            self.delete_dmarc_report(event).await;\n                            true\n                        }\n                        QueueClass::TlsReportHeader(event)\n                            if tenant_domains.as_ref().is_none_or(|domains| {\n                                domains.iter().any(|dd| dd == &event.domain)\n                            }) =>\n                        {\n                            self.delete_tls_report(vec![event]).await;\n                            true\n                        }\n                        _ => false,\n                    };\n\n                    Ok(JsonResponse::new(json!({\n                            \"data\": result,\n                    }))\n                    .into_http_response())\n                } else {\n                    Err(trc::ResourceEvent::NotFound.into_err())\n                }\n            }\n            (\"status\", None, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueGet)?;\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": self.inner.data.queue_status.load(Ordering::Relaxed),\n                }))\n                .into_http_response())\n            }\n            (\"status\", Some(action), &Method::PATCH) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::MessageQueueUpdate)?;\n\n                let prev_status = self.inner.data.queue_status.load(Ordering::Relaxed);\n\n                let _ = self\n                    .inner\n                    .ipc\n                    .queue_tx\n                    .send(QueueEvent::Paused(action == \"stop\"))\n                    .await;\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": prev_status,\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n\nimpl Message {\n    fn from_archive(id: u64, message: &ArchivedMessage) -> Self {\n        let now = now();\n\n        Message {\n            id,\n            return_path: message.return_path.to_string(),\n            created: DateTime::from_timestamp(u64::from(message.created) as i64),\n            size: message.size.into(),\n            priority: message.priority.into(),\n            env_id: message.env_id.as_ref().map(|id| id.to_string()),\n            recipients: message\n                .recipients\n                .iter()\n                .map(|rcpt| Recipient {\n                    address: rcpt.address().to_string(),\n                    queue: rcpt.queue.to_string(),\n                    status: match &rcpt.status {\n                        ArchivedStatus::Scheduled => Status::Scheduled,\n                        ArchivedStatus::Completed(status) => {\n                            Status::Completed(status.response.to_string())\n                        }\n                        ArchivedStatus::TemporaryFailure(status) => {\n                            Status::TemporaryFailure(status.to_string())\n                        }\n                        ArchivedStatus::PermanentFailure(status) => {\n                            Status::PermanentFailure(status.to_string())\n                        }\n                    },\n                    retry_num: rcpt.retry.inner.into(),\n                    next_retry: Some(DateTime::from_timestamp(u64::from(rcpt.retry.due) as i64)),\n                    next_notify: if rcpt.notify.due > now {\n                        DateTime::from_timestamp(u64::from(rcpt.notify.due) as i64).into()\n                    } else {\n                        None\n                    },\n                    expires: if let ArchivedQueueExpiry::Ttl(time) = &rcpt.expires {\n                        DateTime::from_timestamp((u64::from(*time) + message.created) as i64).into()\n                    } else {\n                        None\n                    },\n                    orcpt: rcpt.orcpt.as_ref().map(|orcpt| orcpt.to_string()),\n                })\n                .collect(),\n\n            blob_hash: URL_SAFE_NO_PAD.encode::<&[u8]>(message.blob_hash.0.as_slice()),\n        }\n    }\n}\n\nstruct QueuedMessages {\n    ids: Vec<u64>,\n    values: Vec<Message>,\n    total: usize,\n}\n\nasync fn fetch_queued_messages(\n    server: &Server,\n    params: &UrlParams<'_>,\n    tenant_domains: &Option<Vec<String>>,\n) -> trc::Result<QueuedMessages> {\n    let queue = params.get(\"queue\").and_then(QueueName::new);\n    let text = params.get(\"text\");\n    let from = params.get(\"from\");\n    let to = params.get(\"to\");\n    let before = params\n        .parse::<FutureTimestamp>(\"before\")\n        .map(|t| t.into_inner());\n    let after = params\n        .parse::<FutureTimestamp>(\"after\")\n        .map(|t| t.into_inner());\n    let page = params.parse::<usize>(\"page\").unwrap_or_default();\n    let limit = params.parse::<usize>(\"limit\").unwrap_or_default();\n    let values = params.has_key(\"values\");\n\n    let range_start = params.parse::<u64>(\"range-start\").unwrap_or_default();\n    let range_end = params.parse::<u64>(\"range-end\").unwrap_or(u64::MAX);\n    let max_total = params.parse::<usize>(\"max-total\").unwrap_or_default();\n\n    let mut result = QueuedMessages {\n        ids: Vec::new(),\n        values: Vec::new(),\n        total: 0,\n    };\n    let from_key = ValueKey::from(ValueClass::Queue(QueueClass::Message(range_start)));\n    let to_key = ValueKey::from(ValueClass::Queue(QueueClass::Message(range_end)));\n    let has_filters = text.is_some()\n        || from.is_some()\n        || to.is_some()\n        || before.is_some()\n        || after.is_some()\n        || queue.is_some();\n    let mut offset = page.saturating_sub(1) * limit;\n    let mut total_returned = 0;\n\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let message_ = <Archive<AlignedBytes> as Deserialize>::deserialize(value)\n                    .add_context(|ctx| ctx.ctx(trc::Key::Key, key))?;\n                let message = message_\n                    .unarchive::<queue::Message>()\n                    .add_context(|ctx| ctx.ctx(trc::Key::Key, key))?;\n                let matches = tenant_domains\n                    .as_ref()\n                    .is_none_or(|domains| message.has_domain(domains))\n                    && (!has_filters\n                        || (text\n                            .as_ref()\n                            .map(|text| {\n                                message.return_path.contains(text)\n                                    || message\n                                        .recipients\n                                        .iter()\n                                        .any(|r| r.address().contains(text))\n                            })\n                            .unwrap_or_else(|| {\n                                from.as_ref()\n                                    .is_none_or(|from| message.return_path.contains(from))\n                                    && to.as_ref().is_none_or(|to| {\n                                        message.recipients.iter().any(|r| r.address().contains(to))\n                                    })\n                            })\n                            && before.as_ref().is_none_or(|before| {\n                                message\n                                    .next_delivery_event(queue)\n                                    .is_some_and(|next| next < *before)\n                            })\n                            && after.as_ref().is_none_or(|after| {\n                                message\n                                    .next_delivery_event(queue)\n                                    .is_some_and(|next| next > *after)\n                            })\n                            && queue\n                                .as_ref()\n                                .is_none_or(|q| message.recipients.iter().any(|r| &r.queue == q))));\n\n                if matches {\n                    if offset == 0 {\n                        if limit == 0 || total_returned < limit {\n                            let queue_id = key.deserialize_be_u64(0)?;\n                            if values {\n                                result.values.push(Message::from_archive(queue_id, message));\n                            } else {\n                                result.ids.push(queue_id);\n                            }\n                            total_returned += 1;\n                        }\n                    } else {\n                        offset -= 1;\n                    }\n\n                    result.total += 1;\n                }\n\n                Ok(max_total == 0 || result.total < max_total)\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n        .map(|_| result)\n}\n\nstruct QueuedReports {\n    ids: Vec<QueueClass>,\n    total: usize,\n}\n\nasync fn fetch_queued_reports(\n    server: &Server,\n    params: &UrlParams<'_>,\n    tenant_domains: &Option<Vec<String>>,\n) -> trc::Result<QueuedReports> {\n    let domain = params.get(\"domain\").map(|d| d.to_lowercase());\n    let type_ = params.get(\"type\").and_then(|t| match t {\n        \"dmarc\" => 0u8.into(),\n        \"tls\" => 1u8.into(),\n        _ => None,\n    });\n    let page: usize = params.parse(\"page\").unwrap_or_default();\n    let limit: usize = params.parse(\"limit\").unwrap_or_default();\n\n    let range_start = params.parse::<u64>(\"range-start\").unwrap_or_default();\n    let range_end = params.parse::<u64>(\"range-end\").unwrap_or(u64::MAX);\n    let max_total = params.parse::<usize>(\"max-total\").unwrap_or_default();\n\n    let mut result = QueuedReports {\n        ids: Vec::new(),\n        total: 0,\n    };\n    let from_key = ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportHeader(\n        ReportEvent {\n            due: range_start,\n            policy_hash: 0,\n            seq_id: 0,\n            domain: String::new(),\n        },\n    )));\n    let to_key = ValueKey::from(ValueClass::Queue(QueueClass::TlsReportHeader(\n        ReportEvent {\n            due: range_end,\n            policy_hash: 0,\n            seq_id: 0,\n            domain: String::new(),\n        },\n    )));\n    let mut offset = page.saturating_sub(1) * limit;\n    let mut total_returned = 0;\n\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending().no_values(),\n            |key, _| {\n                if type_.is_none_or(|t| t == *key.last().unwrap()) {\n                    let event = ReportEvent::deserialize(key)?;\n                    if tenant_domains\n                        .as_ref()\n                        .is_none_or(|domains| domains.iter().any(|dd| dd == &event.domain))\n                        && event.seq_id != 0\n                        && domain.as_ref().is_none_or(|d| event.domain.contains(d))\n                    {\n                        if offset == 0 {\n                            if limit == 0 || total_returned < limit {\n                                result.ids.push(if *key.last().unwrap() == 0 {\n                                    QueueClass::DmarcReportHeader(event)\n                                } else {\n                                    QueueClass::TlsReportHeader(event)\n                                });\n                                total_returned += 1;\n                            }\n                        } else {\n                            offset -= 1;\n                        }\n\n                        result.total += 1;\n                    }\n                }\n\n                Ok(max_total == 0 || result.total < max_total)\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n        .map(|_| result)\n}\n\nimpl Report {\n    fn dmarc(event: ReportEvent, report: report::Report, rua: Vec<URI>) -> Self {\n        Self::Dmarc {\n            domain: event.domain.clone(),\n            range_from: DateTime::from_timestamp(event.seq_id as i64),\n            range_to: DateTime::from_timestamp(event.due as i64),\n            id: QueueClass::DmarcReportHeader(event).queue_id(),\n            report,\n            rua,\n        }\n    }\n\n    fn tls(event: ReportEvent, report: TlsReport, rua: Vec<ReportUri>) -> Self {\n        Self::Tls {\n            domain: event.domain.clone(),\n            range_from: DateTime::from_timestamp(event.seq_id as i64),\n            range_to: DateTime::from_timestamp(event.due as i64),\n            id: QueueClass::TlsReportHeader(event).queue_id(),\n            report,\n            rua,\n        }\n    }\n}\n\ntrait GenerateQueueId {\n    fn queue_id(&self) -> String;\n}\n\nimpl GenerateQueueId for QueueClass {\n    fn queue_id(&self) -> String {\n        match self {\n            QueueClass::DmarcReportHeader(h) => {\n                format!(\"d!{}!{}!{}!{}\", h.domain, h.policy_hash, h.seq_id, h.due)\n            }\n            QueueClass::TlsReportHeader(h) => {\n                format!(\"t!{}!{}!{}!{}\", h.domain, h.policy_hash, h.seq_id, h.due)\n            }\n            _ => unreachable!(),\n        }\n    }\n}\n\nfn parse_queued_report_id(id: &str) -> Option<QueueClass> {\n    let mut parts = id.split('!');\n    let type_ = parts.next()?;\n    let event = ReportEvent {\n        domain: parts.next()?.to_string(),\n        policy_hash: parts.next().and_then(|p| p.parse::<u64>().ok())?,\n        seq_id: parts.next().and_then(|p| p.parse::<u64>().ok())?,\n        due: parts.next().and_then(|p| p.parse::<u64>().ok())?,\n    };\n    match type_ {\n        \"d\" => Some(QueueClass::DmarcReportHeader(event)),\n        \"t\" => Some(QueueClass::TlsReportHeader(event)),\n        _ => None,\n    }\n}\n\nfn serialize_maybe_datetime<S>(value: &Option<DateTime>, serializer: S) -> Result<S::Ok, S::Error>\nwhere\n    S: Serializer,\n{\n    match value {\n        Some(value) => serializer.serialize_some(&value.to_rfc3339()),\n        None => serializer.serialize_none(),\n    }\n}\n\nfn deserialize_maybe_datetime<'de, D>(deserializer: D) -> Result<Option<DateTime>, D::Error>\nwhere\n    D: Deserializer<'de>,\n{\n    if let Some(value) = <Option<&str> as serde::Deserialize>::deserialize(deserializer)? {\n        if let Some(value) = DateTime::parse_rfc3339(value) {\n            Ok(Some(value))\n        } else {\n            Err(serde::de::Error::custom(\n                \"Failed to parse RFC3339 timestamp\",\n            ))\n        }\n    } else {\n        Ok(None)\n    }\n}\n\nfn serialize_datetime<S>(value: &DateTime, serializer: S) -> Result<S::Ok, S::Error>\nwhere\n    S: Serializer,\n{\n    serializer.serialize_str(&value.to_rfc3339())\n}\n\nfn deserialize_datetime<'de, D>(deserializer: D) -> Result<DateTime, D::Error>\nwhere\n    D: Deserializer<'de>,\n{\n    use serde::Deserialize;\n\n    if let Some(value) = DateTime::parse_rfc3339(<&str>::deserialize(deserializer)?) {\n        Ok(value)\n    } else {\n        Err(serde::de::Error::custom(\n            \"Failed to parse RFC3339 timestamp\",\n        ))\n    }\n}\n\nfn is_zero(num: &i16) -> bool {\n    *num == 0\n}\n\ntrait IsTenantDomain {\n    fn is_tenant_domain(&self, tenant_domains: &Option<Vec<String>>) -> bool;\n}\nimpl IsTenantDomain for ArchivedMessage {\n    fn is_tenant_domain(&self, tenant_domains: &Option<Vec<String>>) -> bool {\n        tenant_domains\n            .as_ref()\n            .is_none_or(|domains| self.has_domain(domains))\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/reload.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Server,\n    auth::AccessToken,\n    ipc::{BroadcastEvent, HousekeeperEvent},\n};\nuse directory::Permission;\nuse hyper::Method;\nuse serde_json::json;\nuse std::future::Future;\nuse utils::url_params::UrlParams;\n\nuse http_proto::*;\n\npub trait ManageReload: Sync + Send {\n    fn handle_manage_reload(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn handle_manage_update(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl ManageReload for Server {\n    async fn handle_manage_reload(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        // Validate the access token\n        access_token.assert_has_permission(Permission::SettingsReload)?;\n\n        match (path.get(1).copied(), req.method()) {\n            (Some(\"lookup\"), &Method::GET) => {\n                let result = self.reload_lookups().await?;\n                // Update core\n                if let Some(core) = result.new_core {\n                    self.inner.shared_core.store(core.into());\n                }\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": result.config,\n                }))\n                .into_http_response())\n            }\n            (Some(\"certificate\"), &Method::GET) => Ok(JsonResponse::new(json!({\n                \"data\": self.reload_certificates().await?.config,\n            }))\n            .into_http_response()),\n            (Some(\"server.blocked-ip\"), &Method::GET) => {\n                let result = self.reload_blocked_ips().await?;\n\n                self.cluster_broadcast(BroadcastEvent::ReloadBlockedIps)\n                    .await;\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": result.config,\n                }))\n                .into_http_response())\n            }\n            (_, &Method::GET) => {\n                let result = self.reload().await?;\n                if !UrlParams::new(req.uri().query()).has_key(\"dry-run\") {\n                    if let Some(core) = result.new_core {\n                        // Update core\n                        self.inner.shared_core.store(core.into());\n\n                        self.cluster_broadcast(BroadcastEvent::ReloadSettings).await;\n                    }\n\n                    if let Some(tracers) = result.tracers {\n                        // Update tracers\n\n                        // SPDX-SnippetBegin\n                        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                        // SPDX-License-Identifier: LicenseRef-SEL\n                        #[cfg(feature = \"enterprise\")]\n                        tracers.update(self.inner.shared_core.load().is_enterprise_edition());\n                        // SPDX-SnippetEnd\n                        #[cfg(not(feature = \"enterprise\"))]\n                        tracers.update(false);\n                    }\n\n                    // Reload settings\n                    self.inner\n                        .ipc\n                        .housekeeper_tx\n                        .send(HousekeeperEvent::ReloadSettings)\n                        .await\n                        .map_err(|err| {\n                            trc::EventType::Server(trc::ServerEvent::ThreadError)\n                                .reason(err)\n                                .details(concat!(\n                                    \"Failed to send settings reload \",\n                                    \"event to housekeeper\"\n                                ))\n                                .caused_by(trc::location!())\n                        })?;\n                }\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": result.config,\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n\n    async fn handle_manage_update(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        match (path.get(1).copied(), req.method()) {\n            (Some(\"spam-filter\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SpamFilterUpdate)?;\n                let params = UrlParams::new(req.uri().query());\n\n                let overwrite = params.has_key(\"overwrite\");\n                let force = params.has_key(\"force\");\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": self\n                    .core\n                    .storage\n                    .config\n                    .update_spam_rules(force, overwrite)\n                    .await?\n                    .map(|v| v.to_string()),\n                }))\n                .into_http_response())\n            }\n            (Some(\"webadmin\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::WebadminUpdate)?;\n\n                self.inner\n                    .data\n                    .webadmin\n                    .update_and_unpack(&self.core)\n                    .await?;\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": (),\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/report.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse directory::{Permission, Type, backend::internal::manage::ManageDirectory};\nuse http_proto::{request::decode_path_element, *};\nuse hyper::Method;\nuse mail_auth::report::{\n    Feedback,\n    tlsrpt::{FailureDetails, Policy, TlsReport},\n};\nuse serde_json::json;\nuse smtp::reporting::analysis::IncomingReport;\nuse std::future::Future;\nuse store::{\n    Deserialize, IterateParams, Key, U64_LEN, ValueKey,\n    write::{\n        AlignedBytes, Archive, BatchBuilder, ReportClass, ValueClass, key::DeserializeBigEndian,\n    },\n};\nuse trc::AddContext;\nuse utils::url_params::UrlParams;\n\nenum ReportType {\n    Dmarc,\n    Tls,\n    Arf,\n}\n\npub trait ManageReports: Sync + Send {\n    fn handle_manage_reports(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl ManageReports for Server {\n    async fn handle_manage_reports(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        let mut tenant_domains: Option<Vec<String>> = None;\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // Limit to tenant domains\n        #[cfg(feature = \"enterprise\")]\n        if self.core.is_enterprise_edition()\n            && let Some(tenant) = access_token.tenant\n        {\n            tenant_domains = self\n                .core\n                .storage\n                .data\n                .list_principals(None, tenant.id.into(), &[Type::Domain], false, 0, 0)\n                .await\n                .map(|principals| {\n                    principals\n                        .items\n                        .into_iter()\n                        .map(|p| p.name)\n                        .collect::<Vec<_>>()\n                })\n                .caused_by(trc::location!())?\n                .into();\n        }\n\n        // SPDX-SnippetEnd\n\n        match (\n            path.get(1).copied().unwrap_or_default(),\n            path.get(2).copied().map(decode_path_element),\n            req.method(),\n        ) {\n            (class @ (\"dmarc\" | \"tls\" | \"arf\"), None, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::IncomingReportList)?;\n\n                let params = UrlParams::new(req.uri().query());\n\n                let IncomingReports { ids, total } =\n                    fetch_incoming_reports(self, class, &params, &tenant_domains).await?;\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": {\n                            \"items\": ids.into_iter().map(|(id, expires)| {\n                                format!(\"{id}_{expires}\")\n                            }).collect::<Vec<_>>(),\n                            \"total\": total,\n                        },\n                }))\n                .into_http_response())\n            }\n            (class @ (\"dmarc\" | \"tls\" | \"arf\"), Some(report_id), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::IncomingReportGet)?;\n\n                if let Some(report_id) = parse_incoming_report_id(class, report_id.as_ref()) {\n                    match &report_id {\n                        ReportClass::Tls { .. } => match fetch_report::<IncomingReport<TlsReport>>(\n                            self,\n                            ValueKey::from(ValueClass::Report(report_id)),\n                        )\n                        .await?\n                        {\n                            Some(report)\n                                if tenant_domains\n                                    .as_ref()\n                                    .is_none_or(|domains| report.has_domain(domains)) =>\n                            {\n                                Ok(JsonResponse::new(json!({\n                                        \"data\": report,\n                                }))\n                                .into_http_response())\n                            }\n                            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n                        },\n                        ReportClass::Dmarc { .. } => {\n                            match fetch_report::<IncomingReport<mail_auth::report::Report>>(\n                                self,\n                                ValueKey::from(ValueClass::Report(report_id)),\n                            )\n                            .await?\n                            {\n                                Some(report)\n                                    if tenant_domains\n                                        .as_ref()\n                                        .is_none_or(|domains| report.has_domain(domains)) =>\n                                {\n                                    Ok(JsonResponse::new(json!({\n                                            \"data\": report,\n                                    }))\n                                    .into_http_response())\n                                }\n                                _ => Err(trc::ResourceEvent::NotFound.into_err()),\n                            }\n                        }\n                        ReportClass::Arf { .. } => match fetch_report::<IncomingReport<Feedback>>(\n                            self,\n                            ValueKey::from(ValueClass::Report(report_id)),\n                        )\n                        .await?\n                        {\n                            Some(report)\n                                if tenant_domains\n                                    .as_ref()\n                                    .is_none_or(|domains| report.has_domain(domains)) =>\n                            {\n                                Ok(JsonResponse::new(json!({\n                                        \"data\": report,\n                                }))\n                                .into_http_response())\n                            }\n                            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n                        },\n                    }\n                } else {\n                    Err(trc::ResourceEvent::NotFound.into_err())\n                }\n            }\n            (class @ (\"dmarc\" | \"tls\" | \"arf\"), None, &Method::DELETE) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::IncomingReportDelete)?;\n\n                let params = UrlParams::new(req.uri().query());\n\n                let IncomingReports { ids, .. } =\n                    fetch_incoming_reports(self, class, &params, &tenant_domains).await?;\n\n                let found = !ids.is_empty();\n                if found {\n                    let class = match class {\n                        \"dmarc\" => ReportClass::Dmarc { id: 0, expires: 0 },\n                        \"tls\" => ReportClass::Tls { id: 0, expires: 0 },\n                        \"arf\" => ReportClass::Arf { id: 0, expires: 0 },\n                        _ => unreachable!(),\n                    };\n                    let server = self.clone();\n                    tokio::spawn(async move {\n                        let mut batch = BatchBuilder::new();\n\n                        for (id, expires) in ids {\n                            let report_id = match &class {\n                                ReportClass::Dmarc { .. } => ReportClass::Dmarc { id, expires },\n                                ReportClass::Tls { .. } => ReportClass::Tls { id, expires },\n                                ReportClass::Arf { .. } => ReportClass::Arf { id, expires },\n                            };\n\n                            batch.clear(ValueClass::Report(report_id));\n\n                            if batch.is_large_batch() {\n                                if let Err(err) =\n                                    server.core.storage.data.write(batch.build_all()).await\n                                {\n                                    trc::error!(err.caused_by(trc::location!()));\n                                }\n                                batch = BatchBuilder::new();\n                            }\n                        }\n\n                        if !batch.is_empty()\n                            && let Err(err) =\n                                server.core.storage.data.write(batch.build_all()).await\n                        {\n                            trc::error!(err.caused_by(trc::location!()));\n                        }\n                    });\n                }\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": found,\n                }))\n                .into_http_response())\n            }\n            (class @ (\"dmarc\" | \"tls\" | \"arf\"), Some(report_id), &Method::DELETE) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::IncomingReportDelete)?;\n\n                if let Some(report_id) = parse_incoming_report_id(class, report_id.as_ref()) {\n                    if let Some(domains) = &tenant_domains {\n                        let is_tenant_report = match &report_id {\n                            ReportClass::Tls { .. } => fetch_report::<IncomingReport<TlsReport>>(\n                                self,\n                                ValueKey::from(ValueClass::Report(report_id.clone())),\n                            )\n                            .await?\n                            .is_none_or(|report| report.has_domain(domains)),\n                            ReportClass::Dmarc { .. } => {\n                                fetch_report::<IncomingReport<mail_auth::report::Report>>(\n                                    self,\n                                    ValueKey::from(ValueClass::Report(report_id.clone())),\n                                )\n                                .await?\n                                .is_none_or(|report| report.has_domain(domains))\n                            }\n\n                            ReportClass::Arf { .. } => fetch_report::<IncomingReport<Feedback>>(\n                                self,\n                                ValueKey::from(ValueClass::Report(report_id.clone())),\n                            )\n                            .await?\n                            .is_none_or(|report| report.has_domain(domains)),\n                        };\n\n                        if !is_tenant_report {\n                            return Err(trc::ResourceEvent::NotFound.into_err());\n                        }\n                    }\n\n                    let mut batch = BatchBuilder::new();\n                    batch.clear(ValueClass::Report(report_id));\n                    self.core.storage.data.write(batch.build_all()).await?;\n\n                    Ok(JsonResponse::new(json!({\n                            \"data\": true,\n                    }))\n                    .into_http_response())\n                } else {\n                    Err(trc::ResourceEvent::NotFound.into_err())\n                }\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n\nasync fn fetch_report<T>(server: &Server, key: impl Key) -> trc::Result<Option<T>>\nwhere\n    T: rkyv::Archive\n        + for<'a> rkyv::Serialize<\n            rkyv::api::high::HighSerializer<\n                rkyv::util::AlignedVec,\n                rkyv::ser::allocator::ArenaHandle<'a>,\n                rkyv::rancor::Error,\n            >,\n        >,\n    T::Archived: for<'a> rkyv::bytecheck::CheckBytes<rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>>\n        + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n{\n    if let Some(tls) = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(key)\n        .await?\n    {\n        tls.deserialize::<T>().map(Some)\n    } else {\n        Ok(None)\n    }\n}\n\nstruct IncomingReports {\n    ids: Vec<(u64, u64)>,\n    total: usize,\n}\n\nasync fn fetch_incoming_reports(\n    server: &Server,\n    class: &str,\n    params: &UrlParams<'_>,\n    tenant_domains: &Option<Vec<String>>,\n) -> trc::Result<IncomingReports> {\n    let filter = params.get(\"text\");\n    let page: usize = params.parse::<usize>(\"page\").unwrap_or_default();\n    let limit: usize = params.parse::<usize>(\"limit\").unwrap_or_default();\n\n    let range_start = params.parse::<u64>(\"range-start\").unwrap_or_default();\n    let range_end = params.parse::<u64>(\"range-end\").unwrap_or(u64::MAX);\n    let max_total = params.parse::<usize>(\"max-total\").unwrap_or_default();\n\n    let (from_key, to_key, typ) = match class {\n        \"dmarc\" => (\n            ValueKey::from(ValueClass::Report(ReportClass::Dmarc {\n                id: range_start,\n                expires: 0,\n            })),\n            ValueKey::from(ValueClass::Report(ReportClass::Dmarc {\n                id: range_end,\n                expires: u64::MAX,\n            })),\n            ReportType::Dmarc,\n        ),\n        \"tls\" => (\n            ValueKey::from(ValueClass::Report(ReportClass::Tls {\n                id: range_start,\n                expires: 0,\n            })),\n            ValueKey::from(ValueClass::Report(ReportClass::Tls {\n                id: range_end,\n                expires: u64::MAX,\n            })),\n            ReportType::Tls,\n        ),\n        \"arf\" => (\n            ValueKey::from(ValueClass::Report(ReportClass::Arf {\n                id: range_start,\n                expires: 0,\n            })),\n            ValueKey::from(ValueClass::Report(ReportClass::Arf {\n                id: range_end,\n                expires: u64::MAX,\n            })),\n            ReportType::Arf,\n        ),\n        _ => unreachable!(),\n    };\n\n    let mut results = IncomingReports {\n        ids: Vec::new(),\n        total: 0,\n    };\n    let mut offset = page.saturating_sub(1) * limit;\n    let mut last_id = 0;\n    let has_filters = filter.is_some() || tenant_domains.is_some();\n\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(from_key, to_key)\n                .set_values(has_filters)\n                .descending(),\n            |key, value| {\n                // Skip chunked records\n                let id = key.deserialize_be_u64(U64_LEN + 1)?;\n                if id == last_id {\n                    return Ok(true);\n                }\n                last_id = id;\n\n                // TODO: Support filtering chunked records (over 10MB) on FDB\n                let matches = if has_filters {\n                    let archive = <Archive<AlignedBytes> as Deserialize>::deserialize(value)?;\n                    match typ {\n                        ReportType::Dmarc => {\n                            let report = archive\n                                .deserialize::<IncomingReport<mail_auth::report::Report>>()\n                                .caused_by(trc::location!())?;\n\n                            filter.is_none_or(|f| report.contains(f))\n                                && tenant_domains\n                                    .as_ref()\n                                    .is_none_or(|domains| report.has_domain(domains))\n                        }\n                        ReportType::Tls => {\n                            let report = archive\n                                .deserialize::<IncomingReport<TlsReport>>()\n                                .caused_by(trc::location!())?;\n\n                            filter.is_none_or(|f| report.contains(f))\n                                && tenant_domains\n                                    .as_ref()\n                                    .is_none_or(|domains| report.has_domain(domains))\n                        }\n                        ReportType::Arf => {\n                            let report = archive\n                                .deserialize::<IncomingReport<Feedback>>()\n                                .caused_by(trc::location!())?;\n\n                            filter.is_none_or(|f| report.contains(f))\n                                && tenant_domains\n                                    .as_ref()\n                                    .is_none_or(|domains| report.has_domain(domains))\n                        }\n                    }\n                } else {\n                    true\n                };\n\n                if matches {\n                    if offset == 0 {\n                        if limit == 0 || results.ids.len() < limit {\n                            results.ids.push((id, key.deserialize_be_u64(1)?));\n                        }\n                    } else {\n                        offset -= 1;\n                    }\n\n                    results.total += 1;\n                }\n\n                Ok(max_total == 0 || results.total < max_total)\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n        .map(|_| results)\n}\n\nfn parse_incoming_report_id(class: &str, id: &str) -> Option<ReportClass> {\n    let mut parts = id.split('_');\n    let id = parts.next()?.parse().ok()?;\n    let expires = parts.next()?.parse().ok()?;\n    match class {\n        \"dmarc\" => Some(ReportClass::Dmarc { id, expires }),\n        \"tls\" => Some(ReportClass::Tls { id, expires }),\n        \"arf\" => Some(ReportClass::Arf { id, expires }),\n        _ => None,\n    }\n}\n\nimpl From<&str> for ReportType {\n    fn from(s: &str) -> Self {\n        match s {\n            \"dmarc\" => Self::Dmarc,\n            \"tls\" => Self::Tls,\n            \"arf\" => Self::Arf,\n            _ => unreachable!(),\n        }\n    }\n}\n\ntrait Contains {\n    fn contains(&self, text: &str) -> bool;\n}\n\nimpl Contains for mail_auth::report::Report {\n    fn contains(&self, text: &str) -> bool {\n        self.domain().contains(text)\n            || self.org_name().to_lowercase().contains(text)\n            || self.report_id().contains(text)\n            || self\n                .extra_contact_info()\n                .is_some_and(|c| c.to_lowercase().contains(text))\n            || self.records().iter().any(|record| record.contains(text))\n    }\n}\n\nimpl Contains for mail_auth::report::Record {\n    fn contains(&self, filter: &str) -> bool {\n        self.envelope_from().contains(filter)\n            || self.header_from().contains(filter)\n            || self.envelope_to().is_some_and(|to| to.contains(filter))\n            || self.dkim_auth_result().iter().any(|dkim| {\n                dkim.domain().contains(filter)\n                    || dkim.selector().contains(filter)\n                    || dkim\n                        .human_result()\n                        .as_ref()\n                        .is_some_and(|r| r.contains(filter))\n            })\n            || self.spf_auth_result().iter().any(|spf| {\n                spf.domain().contains(filter)\n                    || spf.human_result().is_some_and(|r| r.contains(filter))\n            })\n            || self\n                .source_ip()\n                .is_some_and(|ip| ip.to_string().contains(filter))\n    }\n}\n\nimpl Contains for TlsReport {\n    fn contains(&self, text: &str) -> bool {\n        self.organization_name\n            .as_ref()\n            .is_some_and(|o| o.to_lowercase().contains(text))\n            || self\n                .contact_info\n                .as_ref()\n                .is_some_and(|c| c.to_lowercase().contains(text))\n            || self.report_id.contains(text)\n            || self.policies.iter().any(|p| p.contains(text))\n    }\n}\n\nimpl Contains for Policy {\n    fn contains(&self, filter: &str) -> bool {\n        self.policy.policy_domain.contains(filter)\n            || self\n                .policy\n                .policy_string\n                .iter()\n                .any(|s| s.to_lowercase().contains(filter))\n            || self\n                .policy\n                .mx_host\n                .iter()\n                .any(|s| s.to_lowercase().contains(filter))\n            || self.failure_details.iter().any(|f| f.contains(filter))\n    }\n}\n\nimpl Contains for FailureDetails {\n    fn contains(&self, filter: &str) -> bool {\n        self.sending_mta_ip\n            .is_some_and(|s| s.to_string().contains(filter))\n            || self\n                .receiving_ip\n                .is_some_and(|s| s.to_string().contains(filter))\n            || self\n                .receiving_mx_hostname\n                .as_ref()\n                .is_some_and(|s| s.contains(filter))\n            || self\n                .receiving_mx_helo\n                .as_ref()\n                .is_some_and(|s| s.contains(filter))\n            || self\n                .additional_information\n                .as_ref()\n                .is_some_and(|s| s.contains(filter))\n            || self\n                .failure_reason_code\n                .as_ref()\n                .is_some_and(|s| s.contains(filter))\n    }\n}\n\nimpl Contains for Feedback<'_> {\n    fn contains(&self, text: &str) -> bool {\n        // Check if any of the string fields contain the filter\n        self.authentication_results()\n            .iter()\n            .any(|s| s.contains(text))\n            || self\n                .original_envelope_id()\n                .is_some_and(|s| s.contains(text))\n            || self.original_mail_from().is_some_and(|s| s.contains(text))\n            || self.original_rcpt_to().is_some_and(|s| s.contains(text))\n            || self.reported_domain().iter().any(|s| s.contains(text))\n            || self.reported_uri().iter().any(|s| s.contains(text))\n            || self.reporting_mta().is_some_and(|s| s.contains(text))\n            || self.user_agent().is_some_and(|s| s.contains(text))\n            || self.dkim_adsp_dns().is_some_and(|s| s.contains(text))\n            || self\n                .dkim_canonicalized_body()\n                .is_some_and(|s| s.contains(text))\n            || self\n                .dkim_canonicalized_header()\n                .is_some_and(|s| s.contains(text))\n            || self.dkim_domain().is_some_and(|s| s.contains(text))\n            || self.dkim_identity().is_some_and(|s| s.contains(text))\n            || self.dkim_selector().is_some_and(|s| s.contains(text))\n            || self.dkim_selector_dns().is_some_and(|s| s.contains(text))\n            || self.spf_dns().is_some_and(|s| s.contains(text))\n            || self.message().is_some_and(|s| s.contains(text))\n            || self.headers().is_some_and(|s| s.contains(text))\n    }\n}\n\nimpl<T: Contains> Contains for IncomingReport<T> {\n    fn contains(&self, text: &str) -> bool {\n        self.from.to_lowercase().contains(text)\n            || self.to.iter().any(|to| to.to_lowercase().contains(text))\n            || self.subject.to_lowercase().contains(text)\n            || self.report.contains(text)\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/settings.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse directory::Permission;\nuse hyper::Method;\nuse serde_json::json;\nuse store::ahash::AHashMap;\nuse utils::{config::ConfigKey, map::vec_map::VecMap, url_params::UrlParams};\n\nuse http_proto::{request::decode_path_element, *};\nuse std::future::Future;\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum UpdateSettings {\n    Delete {\n        keys: Vec<String>,\n    },\n    Clear {\n        prefix: String,\n        #[serde(default)]\n        filter: Option<String>,\n    },\n    Insert {\n        prefix: Option<String>,\n        values: Vec<(String, String)>,\n        assert_empty: bool,\n    },\n}\n\npub trait ManageSettings: Sync + Send {\n    fn handle_manage_settings(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl ManageSettings for Server {\n    async fn handle_manage_settings(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        match (path.get(1).copied(), req.method()) {\n            (Some(\"group\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SettingsList)?;\n\n                // List settings\n                let params = UrlParams::new(req.uri().query());\n                let prefix = params\n                    .get(\"prefix\")\n                    .map(|p| {\n                        if !p.ends_with('.') {\n                            format!(\"{p}.\")\n                        } else {\n                            p.to_string()\n                        }\n                    })\n                    .unwrap_or_default();\n                let suffix = params\n                    .get(\"suffix\")\n                    .map(|s| {\n                        if !s.starts_with('.') {\n                            format!(\".{s}\")\n                        } else {\n                            s.to_string()\n                        }\n                    })\n                    .unwrap_or_default();\n                let field = params.get(\"field\");\n                let filter = params.get(\"filter\").unwrap_or_default().to_lowercase();\n                let limit: usize = params.parse(\"limit\").unwrap_or(0);\n                let mut offset =\n                    params.parse::<usize>(\"page\").unwrap_or(0).saturating_sub(1) * limit;\n                let has_filter = !filter.is_empty();\n\n                let settings = self.core.storage.config.list(&prefix, true).await?;\n                if !suffix.is_empty() && !settings.is_empty() {\n                    // Obtain record ids\n                    let mut total = 0;\n                    let mut ids = Vec::new();\n                    for key in settings.keys() {\n                        if let Some(id) = key.strip_suffix(&suffix)\n                            && !id.is_empty()\n                        {\n                            if !has_filter {\n                                if offset == 0 {\n                                    if limit == 0 || ids.len() < limit {\n                                        ids.push(id);\n                                    }\n                                } else {\n                                    offset -= 1;\n                                }\n                                total += 1;\n                            } else {\n                                ids.push(id);\n                            }\n                        }\n                    }\n\n                    // Group settings by record id\n                    let mut records = Vec::new();\n                    for id in ids {\n                        let mut record = AHashMap::new();\n                        let prefix = format!(\"{id}.\");\n                        record.insert(\"_id\".to_string(), id.to_string());\n                        for (k, v) in &settings {\n                            if let Some(k) = k.strip_prefix(&prefix) {\n                                if field.is_none_or(|field| field == k) {\n                                    record.insert(k.to_string(), v.to_string());\n                                }\n                            } else if record.len() > 1 {\n                                break;\n                            }\n                        }\n\n                        if has_filter {\n                            if record\n                                .iter()\n                                .any(|(_, v)| v.to_lowercase().contains(&filter))\n                            {\n                                if offset == 0 {\n                                    if limit == 0 || records.len() < limit {\n                                        records.push(record);\n                                    }\n                                } else {\n                                    offset -= 1;\n                                }\n                                total += 1;\n                            }\n                        } else {\n                            records.push(record);\n                        }\n                    }\n\n                    Ok(JsonResponse::new(json!({\n                        \"data\": {\n                            \"total\": total,\n                            \"items\": records,\n                        },\n                    }))\n                    .into_http_response())\n                } else {\n                    let mut total = 0;\n                    let mut items = Vec::new();\n\n                    for (k, v) in settings {\n                        if filter.is_empty()\n                            || k.to_lowercase().contains(&filter)\n                            || v.to_lowercase().contains(&filter)\n                        {\n                            if offset == 0 {\n                                if limit == 0 || items.len() < limit {\n                                    let k =\n                                        k.strip_prefix(&prefix).map(|k| k.to_string()).unwrap_or(k);\n                                    items.push(json!({\n                                        \"_id\": k,\n                                        \"_value\": v,\n                                    }));\n                                }\n                            } else {\n                                offset -= 1;\n                            }\n                            total += 1;\n                        }\n                    }\n\n                    Ok(JsonResponse::new(json!({\n                        \"data\": {\n                            \"total\": total,\n                            \"items\": items,\n                        },\n                    }))\n                    .into_http_response())\n                }\n            }\n            (Some(\"list\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SettingsList)?;\n\n                // List settings\n                let params = UrlParams::new(req.uri().query());\n                let prefix = params\n                    .get(\"prefix\")\n                    .map(|p| {\n                        if !p.ends_with('.') {\n                            format!(\"{p}.\")\n                        } else {\n                            p.to_string()\n                        }\n                    })\n                    .unwrap_or_default();\n                let limit: usize = params.parse(\"limit\").unwrap_or(0);\n                let offset = params.parse::<usize>(\"page\").unwrap_or(0).saturating_sub(1) * limit;\n\n                let settings = self.core.storage.config.list(&prefix, true).await?;\n                let total = settings.len();\n                let items = settings\n                    .into_iter()\n                    .skip(offset)\n                    .take(if limit == 0 { total } else { limit })\n                    .collect::<VecMap<_, _>>();\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": {\n                        \"total\": total,\n                        \"items\": items,\n                    },\n                }))\n                .into_http_response())\n            }\n            (Some(\"keys\"), &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SettingsList)?;\n\n                // Obtain keys\n                let params = UrlParams::new(req.uri().query());\n                let keys = params\n                    .get(\"keys\")\n                    .map(|s| s.split(',').collect::<Vec<_>>())\n                    .unwrap_or_default();\n                let prefixes = params\n                    .get(\"prefixes\")\n                    .map(|s| s.split(',').collect::<Vec<_>>())\n                    .unwrap_or_default();\n                let mut results = AHashMap::with_capacity(keys.len());\n\n                for key in keys {\n                    if let Some(value) = self.core.storage.config.get(key).await? {\n                        results.insert(key.to_string(), value);\n                    }\n                }\n                for prefix in prefixes {\n                    let prefix = if !prefix.ends_with('.') {\n                        format!(\"{prefix}.\")\n                    } else {\n                        prefix.to_string()\n                    };\n                    results.extend(self.core.storage.config.list(&prefix, false).await?);\n                }\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": results,\n                }))\n                .into_http_response())\n            }\n            (Some(prefix), &Method::DELETE) if !prefix.is_empty() => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SettingsDelete)?;\n\n                let prefix = decode_path_element(prefix);\n\n                self.core.storage.config.clear(prefix.as_ref()).await?;\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": (),\n                }))\n                .into_http_response())\n            }\n            (None, &Method::POST) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SettingsUpdate)?;\n\n                let changes = serde_json::from_slice::<Vec<UpdateSettings>>(\n                    body.as_deref().unwrap_or_default(),\n                )\n                .map_err(|err| {\n                    trc::EventType::Resource(trc::ResourceEvent::BadParameters).from_json_error(err)\n                })?;\n\n                for change in changes {\n                    match change {\n                        UpdateSettings::Delete { keys } => {\n                            for key in keys {\n                                self.core.storage.config.clear(key).await?;\n                            }\n                        }\n                        UpdateSettings::Clear { prefix, filter } => {\n                            if let Some(filter) = filter {\n                                for (key, value) in\n                                    self.core.storage.config.list(&prefix, false).await?\n                                {\n                                    if value.to_lowercase().contains(&filter)\n                                        || key.to_lowercase().contains(&filter)\n                                    {\n                                        self.core.storage.config.clear(key).await?;\n                                    }\n                                }\n                            } else {\n                                self.core.storage.config.clear_prefix(&prefix).await?;\n                            }\n                        }\n                        UpdateSettings::Insert {\n                            prefix,\n                            values,\n                            assert_empty,\n                        } => {\n                            if assert_empty {\n                                if let Some(prefix) = &prefix {\n                                    if !self\n                                        .core\n                                        .storage\n                                        .config\n                                        .list(&format!(\"{prefix}.\"), true)\n                                        .await?\n                                        .is_empty()\n                                    {\n                                        return Err(trc::ManageEvent::AssertFailed.into_err());\n                                    }\n                                } else if let Some((key, _)) = values.first()\n                                    && self.core.storage.config.get(key).await?.is_some()\n                                {\n                                    return Err(trc::ManageEvent::AssertFailed.into_err());\n                                }\n                            }\n\n                            self.core\n                                .storage\n                                .config\n                                .set(\n                                    values.into_iter().map(|(key, value)| ConfigKey {\n                                        key: if let Some(prefix) = &prefix {\n                                            format!(\"{prefix}.{key}\")\n                                        } else {\n                                            key\n                                        },\n                                        value,\n                                    }),\n                                    true,\n                                )\n                                .await?;\n                        }\n                    }\n                }\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": (),\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/spam.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Server,\n    auth::AccessToken,\n    config::spamfilter::SpamFilterAction,\n    manager::{SPAM_CLASSIFIER_KEY, SPAM_TRAINER_KEY},\n    psl,\n};\nuse directory::{\n    Permission,\n    backend::internal::manage::{self, ManageDirectory},\n};\nuse email::message::ingest::EmailIngest;\nuse http_proto::{request::decode_path_element, *};\nuse hyper::Method;\nuse mail_auth::{\n    AuthenticatedMessage, DmarcResult, dmarc::verify::DmarcParameters, spf::verify::SpfParameters,\n};\nuse mail_parser::MessageParser;\nuse serde::{Deserialize, Serialize};\nuse serde_json::json;\nuse spam_filter::{\n    SpamFilterInput,\n    analysis::{init::SpamFilterInit, score::SpamFilterAnalyzeScore},\n    modules::classifier::SpamClassifier,\n};\nuse std::future::Future;\nuse std::net::IpAddr;\nuse store::{ahash::AHashMap, write::BatchBuilder};\n\npub trait ManageSpamHandler: Sync + Send {\n    fn handle_manage_spam(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        session: &HttpSessionData,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct SpamClassifyRequest {\n    pub message: String,\n\n    // Session details\n    pub remote_ip: IpAddr,\n    #[serde(default)]\n    pub ehlo_domain: String,\n    #[serde(default)]\n    pub authenticated_as: Option<String>,\n\n    // TLS\n    #[serde(default)]\n    pub is_tls: bool,\n\n    // Envelope\n    pub env_from: String,\n    pub env_from_flags: u64,\n    pub env_rcpt_to: Vec<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct SpamClassifyResponse {\n    pub score: f32,\n    pub tags: AHashMap<String, SpamFilterDisposition<f32>>,\n    pub disposition: SpamFilterDisposition<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\n#[serde(tag = \"action\")]\npub enum SpamFilterDisposition<T> {\n    Allow { value: T },\n    Discard,\n    Reject,\n}\n\nimpl ManageSpamHandler for Server {\n    async fn handle_manage_spam(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        session: &HttpSessionData,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        match (path.get(1).copied(), path.get(2).copied(), req.method()) {\n            (Some(\"upload\"), Some(class @ (\"ham\" | \"spam\")), &Method::POST) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SpamFilterTrain)?;\n\n                let message =\n                    body.ok_or_else(|| manage::error(\"Failed to parse message.\", None::<u64>))?;\n                let account_id = if let Some(account) =\n                    path.get(3).copied().filter(|a| !a.is_empty())\n                {\n                    let principal = self\n                        .store()\n                        .get_principal_info(decode_path_element(account).as_ref())\n                        .await?\n                        .ok_or_else(|| manage::not_found(account.to_string()))?;\n                    if access_token.tenant.is_some() && principal.tenant != access_token.tenant_id()\n                    {\n                        return Err(manage::error(\n                            \"Account does not belong to this tenant.\",\n                            None::<u64>,\n                        ));\n                    }\n\n                    principal.id\n                } else if access_token.tenant.is_none() {\n                    u32::MAX\n                } else {\n                    return Err(manage::error(\n                        \"Account ID is required for tenants.\",\n                        None::<u64>,\n                    ));\n                };\n\n                // Write sample\n                let (blob_hash, blob_hold) =\n                    self.put_temporary_blob(account_id, &message, 60).await?;\n                let mut batch = BatchBuilder::new();\n                batch.with_account_id(account_id).clear(blob_hold);\n                self.add_spam_sample(\n                    &mut batch,\n                    blob_hash,\n                    class == \"spam\",\n                    true,\n                    session.session_id,\n                );\n                self.store().write(batch.build_all()).await?;\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": (),\n                }))\n                .into_http_response())\n            }\n            (Some(\"train\"), request, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SpamFilterTrain)?;\n\n                let result = match request {\n                    Some(\"start\") | Some(\"reset\") => {\n                        if !self.inner.ipc.train_task_controller.is_running() {\n                            let reset = matches!(request, Some(\"reset\"));\n                            let server = self.clone();\n                            tokio::spawn(async move {\n                                if let Err(err) = server.spam_train(reset).await {\n                                    trc::error!(err.caused_by(trc::location!()));\n                                }\n                            });\n\n                            true\n                        } else {\n                            false\n                        }\n                    }\n                    Some(\"stop\") => {\n                        if self.inner.ipc.train_task_controller.is_running() {\n                            self.inner.ipc.train_task_controller.stop();\n                            true\n                        } else {\n                            false\n                        }\n                    }\n                    Some(\"delete\") => {\n                        for key in [SPAM_CLASSIFIER_KEY, SPAM_TRAINER_KEY] {\n                            self.blob_store().delete_blob(key).await?;\n                        }\n                        true\n                    }\n                    Some(\"status\") => self.inner.ipc.train_task_controller.is_running(),\n                    _ => {\n                        return Err(trc::ResourceEvent::NotFound.into_err());\n                    }\n                };\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": result,\n                }))\n                .into_http_response())\n            }\n            (Some(\"classify\"), _, &Method::POST) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::SpamFilterTest)?;\n\n                // Parse request\n                let request = serde_json::from_slice::<SpamClassifyRequest>(\n                    body.as_deref().unwrap_or_default(),\n                )\n                .map_err(|err| {\n                    trc::EventType::Resource(trc::ResourceEvent::BadParameters).from_json_error(err)\n                })?;\n\n                // Built spam filter input\n                let message = MessageParser::new()\n                    .parse(request.message.as_bytes())\n                    .filter(|m| m.root_part().headers().iter().any(|h| !h.name.is_other()))\n                    .ok_or_else(|| manage::error(\"Failed to parse message.\", None::<u64>))?;\n\n                let remote_ip = request.remote_ip;\n                let ehlo_domain = request.ehlo_domain.to_lowercase();\n                let mail_from = request.env_from.to_lowercase();\n                let mail_from_domain = mail_from.rsplit_once('@').map(|(_, domain)| domain);\n                let local_host = &self.core.network.server_name;\n\n                let spf_ehlo_result =\n                    self.core\n                        .smtp\n                        .resolvers\n                        .dns\n                        .verify_spf(self.inner.cache.build_auth_parameters(\n                            SpfParameters::verify_ehlo(remote_ip, &ehlo_domain, local_host),\n                        ))\n                        .await;\n\n                let iprev_result = self\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .verify_iprev(self.inner.cache.build_auth_parameters(remote_ip))\n                    .await;\n\n                let spf_mail_from_result = if let Some(mail_from_domain) = mail_from_domain {\n                    self.core\n                        .smtp\n                        .resolvers\n                        .dns\n                        .check_host(self.inner.cache.build_auth_parameters(SpfParameters::new(\n                            remote_ip,\n                            mail_from_domain,\n                            &ehlo_domain,\n                            local_host,\n                            &mail_from,\n                        )))\n                        .await\n                } else {\n                    self.core\n                        .smtp\n                        .resolvers\n                        .dns\n                        .check_host(self.inner.cache.build_auth_parameters(SpfParameters::new(\n                            remote_ip,\n                            &ehlo_domain,\n                            &ehlo_domain,\n                            local_host,\n                            &format!(\"postmaster@{ehlo_domain}\"),\n                        )))\n                        .await\n                };\n\n                let auth_message = AuthenticatedMessage::from_parsed(&message, true);\n\n                let dkim_output = self\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .verify_dkim(self.inner.cache.build_auth_parameters(&auth_message))\n                    .await;\n\n                let arc_output = self\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .verify_arc(self.inner.cache.build_auth_parameters(&auth_message))\n                    .await;\n\n                let dmarc_output = self\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .verify_dmarc(self.inner.cache.build_auth_parameters(DmarcParameters {\n                        message: &auth_message,\n                        dkim_output: &dkim_output,\n                        rfc5321_mail_from_domain: mail_from_domain.unwrap_or(ehlo_domain.as_str()),\n                        spf_output: &spf_mail_from_result,\n                        domain_suffix_fn: |domain| psl::domain_str(domain).unwrap_or(domain),\n                    }))\n                    .await;\n                let dmarc_pass = matches!(dmarc_output.spf_result(), DmarcResult::Pass)\n                    || matches!(dmarc_output.dkim_result(), DmarcResult::Pass);\n                let dmarc_result = if dmarc_pass {\n                    DmarcResult::Pass\n                } else if dmarc_output.spf_result() != &DmarcResult::None {\n                    dmarc_output.spf_result().clone()\n                } else if dmarc_output.dkim_result() != &DmarcResult::None {\n                    dmarc_output.dkim_result().clone()\n                } else {\n                    DmarcResult::None\n                };\n                let dmarc_policy = dmarc_output.policy();\n\n                let asn_geo = self.lookup_asn_country(remote_ip).await;\n\n                let input = SpamFilterInput {\n                    message: &message,\n                    span_id: session.session_id,\n                    arc_result: Some(&arc_output),\n                    spf_ehlo_result: Some(&spf_ehlo_result),\n                    spf_mail_from_result: Some(&spf_mail_from_result),\n                    dkim_result: dkim_output.as_slice(),\n                    dmarc_result: Some(&dmarc_result),\n                    dmarc_policy: Some(&dmarc_policy),\n                    iprev_result: Some(&iprev_result),\n                    remote_ip: request.remote_ip,\n                    ehlo_domain: Some(ehlo_domain.as_str()),\n                    authenticated_as: request.authenticated_as.as_deref(),\n                    asn: asn_geo.asn.as_ref().map(|a| a.id),\n                    country: asn_geo.country.as_ref().map(|c| c.as_str()),\n                    is_tls: request.is_tls,\n                    env_from: &request.env_from,\n                    env_from_flags: request.env_from_flags,\n                    env_rcpt_to: request.env_rcpt_to.iter().map(String::as_str).collect(),\n                    is_test: true,\n                    is_train: false,\n                };\n\n                // Classify\n                let mut ctx = self.spam_filter_init(input);\n                let result = self.spam_filter_classify(&mut ctx).await;\n\n                // Build response\n                let mut response = SpamClassifyResponse {\n                    score: ctx.result.score,\n                    tags: AHashMap::with_capacity(ctx.result.tags.len()),\n                    disposition: match result {\n                        SpamFilterAction::Allow(value) => SpamFilterDisposition::Allow {\n                            value: value.headers,\n                        },\n                        SpamFilterAction::Discard => SpamFilterDisposition::Discard,\n                        SpamFilterAction::Reject => SpamFilterDisposition::Reject,\n                        SpamFilterAction::Disabled => SpamFilterDisposition::Allow {\n                            value: String::new(),\n                        },\n                    },\n                };\n                for tag in ctx.result.tags {\n                    let disposition = match self.core.spam.lists.scores.get(&tag) {\n                        Some(SpamFilterAction::Allow(score)) => {\n                            SpamFilterDisposition::Allow { value: *score }\n                        }\n                        Some(SpamFilterAction::Discard) => SpamFilterDisposition::Discard,\n                        Some(SpamFilterAction::Reject) => SpamFilterDisposition::Reject,\n                        Some(SpamFilterAction::Disabled) | None => {\n                            SpamFilterDisposition::Allow { value: 0.0 }\n                        }\n                    };\n                    response.tags.insert(tag, disposition);\n                }\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": response,\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/management/stores.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};\nuse common::{\n    auth::AccessToken,\n    ipc::{HousekeeperEvent, PurgeType},\n    manager::webadmin::Resource,\n    storage::index::ObjectIndexBuilder,\n    *,\n};\nuse directory::{\n    Permission,\n    backend::internal::manage::{self, ManageDirectory},\n};\nuse email::{\n    cache::MessageCacheFetch,\n    message::{\n        ingest::EmailIngest,\n        metadata::{MessageData, MessageMetadata},\n    },\n    sieve::SieveScript,\n};\nuse groupware::{\n    calendar::{Calendar, CalendarEvent, CalendarEventNotification},\n    contact::{AddressBook, ContactCard},\n    file::FileNode,\n};\nuse http_proto::{request::decode_path_element, *};\nuse hyper::Method;\nuse serde_json::json;\nuse services::task_manager::index::ReindexIndexTask;\nuse std::future::Future;\nuse store::{\n    Serialize, ValueKey, rand,\n    search::SearchQuery,\n    write::{\n        AlignedBytes, Archive, Archiver, BatchBuilder, BlobLink, BlobOp, DirectoryClass,\n        SearchIndex, ValueClass,\n    },\n};\nuse trc::AddContext;\nuse types::{\n    blob_hash::BlobHash,\n    collection::Collection,\n    field::{EmailField, Field, MailboxField},\n};\nuse utils::url_params::UrlParams;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\nuse super::enterprise::undelete::UndeleteApi;\n// SPDX-SnippetEnd\n\npub trait ManageStore: Sync + Send {\n    fn handle_manage_store(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        session: &HttpSessionData,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n\n    fn housekeeper_request(\n        &self,\n        event: HousekeeperEvent,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl ManageStore for Server {\n    async fn handle_manage_store(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        body: Option<Vec<u8>>,\n        session: &HttpSessionData,\n        access_token: &AccessToken,\n    ) -> trc::Result<HttpResponse> {\n        match (\n            path.get(1).copied(),\n            path.get(2).copied(),\n            path.get(3).copied(),\n            req.method(),\n        ) {\n            (Some(\"blobs\"), Some(blob_hash), _, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::BlobFetch)?;\n\n                let blob_hash = URL_SAFE_NO_PAD\n                    .decode(decode_path_element(blob_hash).as_bytes())\n                    .map_err(|err| {\n                        trc::EventType::Resource(trc::ResourceEvent::BadParameters)\n                            .from_base64_error(err)\n                    })?;\n                let contents = self\n                    .core\n                    .storage\n                    .blob\n                    .get_blob(&blob_hash, 0..usize::MAX)\n                    .await?\n                    .ok_or_else(|| trc::ManageEvent::NotFound.into_err())?;\n                let params = UrlParams::new(req.uri().query());\n                let offset = params.parse(\"offset\").unwrap_or(0);\n                let limit = params.parse(\"limit\").unwrap_or(usize::MAX);\n                let contents = if offset == 0 && limit == usize::MAX {\n                    contents\n                } else {\n                    contents\n                        .get(offset..std::cmp::min(offset + limit, contents.len()))\n                        .unwrap_or_default()\n                        .to_vec()\n                };\n\n                Ok(Resource::new(\"application/octet-stream\", contents).into_http_response())\n            }\n            (Some(\"purge\"), Some(\"blob\"), _, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::PurgeBlobStore)?;\n\n                self.housekeeper_request(HousekeeperEvent::Purge(PurgeType::Blobs {\n                    store: self.core.storage.data.clone(),\n                    blob_store: self.core.storage.blob.clone(),\n                }))\n                .await\n            }\n            (Some(\"purge\"), Some(\"data\"), id, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::PurgeDataStore)?;\n\n                let store = if let Some(id) = id.filter(|id| *id != \"default\") {\n                    if let Some(store) = self.core.storage.stores.get(id) {\n                        store.clone()\n                    } else {\n                        return Err(trc::ResourceEvent::NotFound.into_err());\n                    }\n                } else {\n                    self.core.storage.data.clone()\n                };\n\n                self.housekeeper_request(HousekeeperEvent::Purge(PurgeType::Data(store)))\n                    .await\n            }\n            (Some(\"purge\"), Some(\"in-memory\"), id, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::PurgeInMemoryStore)?;\n\n                let store = if let Some(id) = id.filter(|id| *id != \"default\") {\n                    if let Some(store) = self.core.storage.lookups.get(id) {\n                        store.clone()\n                    } else {\n                        return Err(trc::ResourceEvent::NotFound.into_err());\n                    }\n                } else {\n                    self.core.storage.lookup.clone()\n                };\n\n                let prefix = match path.get(4).copied() {\n                    Some(\"acme\") => vec![KV_ACME].into(),\n                    Some(\"oauth\") => vec![KV_OAUTH].into(),\n                    Some(\"rate-rcpt\") => vec![KV_RATE_LIMIT_RCPT].into(),\n                    Some(\"rate-scan\") => vec![KV_RATE_LIMIT_SCAN].into(),\n                    Some(\"rate-loiter\") => vec![KV_RATE_LIMIT_LOITER].into(),\n                    Some(\"rate-auth\") => vec![KV_RATE_LIMIT_AUTH].into(),\n                    Some(\"rate-smtp\") => vec![KV_RATE_LIMIT_SMTP].into(),\n                    Some(\"rate-contact\") => vec![KV_RATE_LIMIT_CONTACT].into(),\n                    Some(\"rate-http-authenticated\") => {\n                        vec![KV_RATE_LIMIT_HTTP_AUTHENTICATED].into()\n                    }\n                    Some(\"rate-http-anonymous\") => vec![KV_RATE_LIMIT_HTTP_ANONYMOUS].into(),\n                    Some(\"rate-imap\") => vec![KV_RATE_LIMIT_IMAP].into(),\n                    Some(\"greylist\") => vec![KV_GREYLIST].into(),\n                    Some(\"lock-purge-account\") => vec![KV_LOCK_PURGE_ACCOUNT].into(),\n                    Some(\"lock-queue-message\") => vec![KV_LOCK_QUEUE_MESSAGE].into(),\n                    Some(\"lock-queue-report\") => vec![KV_LOCK_QUEUE_REPORT].into(),\n                    Some(\"lock-email-task\") => vec![KV_LOCK_TASK].into(),\n                    Some(\"lock-housekeeper\") => vec![KV_LOCK_HOUSEKEEPER].into(),\n                    _ => None,\n                };\n\n                self.housekeeper_request(HousekeeperEvent::Purge(PurgeType::Lookup {\n                    store,\n                    prefix,\n                }))\n                .await\n            }\n            (Some(\"purge\"), Some(\"account\"), id, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::PurgeAccount)?;\n\n                let account_id = if let Some(id) = id {\n                    self.core\n                        .storage\n                        .data\n                        .get_principal_id(decode_path_element(id).as_ref())\n                        .await?\n                        .ok_or_else(|| trc::ManageEvent::NotFound.into_err())?\n                        .into()\n                } else {\n                    None\n                };\n\n                self.housekeeper_request(HousekeeperEvent::Purge(PurgeType::Account {\n                    account_id,\n                    use_roles: false,\n                }))\n                .await\n            }\n            (Some(\"reindex\"), Some(index), id, &Method::GET) => {\n                // Validate the access token\n                access_token.assert_has_permission(Permission::FtsReindex)?;\n\n                let account_id = if let Some(id) = id {\n                    self.core\n                        .storage\n                        .data\n                        .get_principal_id(decode_path_element(id).as_ref())\n                        .await?\n                        .ok_or_else(|| trc::ManageEvent::NotFound.into_err())?\n                        .into()\n                } else {\n                    None\n                };\n                let tenant_id = access_token.tenant.map(|t| t.id);\n                let index = SearchIndex::try_from_str(index).ok_or_else(|| {\n                    trc::ResourceEvent::BadParameters.reason(\"Invalid search index specified\")\n                })?;\n\n                let jmap = self.clone();\n                tokio::spawn(async move {\n                    if let Err(err) = jmap.reindex(index, account_id, tenant_id).await {\n                        trc::error!(err.details(\"Failed to reindex FTS\"));\n                    }\n                });\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": (),\n                }))\n                .into_http_response())\n            }\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            (Some(\"undelete\"), _, _, _) => {\n                // WARNING: TAMPERING WITH THIS FUNCTION IS STRICTLY PROHIBITED\n                // Any attempt to modify, bypass, or disable this license validation mechanism\n                // constitutes a severe violation of the Stalwart Enterprise License Agreement.\n                // Such actions may result in immediate termination of your license, legal action,\n                // and substantial financial penalties. Stalwart Labs LLC actively monitors for\n                // unauthorized modifications and will pursue all available legal remedies against\n                // violators to the fullest extent of the law, including but not limited to claims\n                // for copyright infringement, breach of contract, and fraud.\n\n                // Validate the access token\n                access_token.assert_has_permission(Permission::Undelete)?;\n\n                if self.core.is_enterprise_edition() {\n                    self.handle_undelete_api_request(req, path, body, session)\n                        .await\n                } else {\n                    Err(manage::enterprise())\n                }\n            }\n            // SPDX-SnippetEnd\n            (Some(\"uids\"), Some(account_id), None, &Method::DELETE) => {\n                let account_id = self\n                    .core\n                    .storage\n                    .data\n                    .get_principal_id(decode_path_element(account_id).as_ref())\n                    .await?\n                    .ok_or_else(|| trc::ManageEvent::NotFound.into_err())?;\n\n                let result = reset_imap_uids(self, account_id).await?;\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": result,\n                }))\n                .into_http_response())\n            }\n            (Some(\"quota\"), Some(account_id), None, method @ (&Method::GET | &Method::DELETE)) => {\n                let account_id = self\n                    .core\n                    .storage\n                    .data\n                    .get_principal_id(decode_path_element(account_id).as_ref())\n                    .await?\n                    .ok_or_else(|| trc::ManageEvent::NotFound.into_err())?;\n\n                if method == Method::DELETE {\n                    recalculate_quota(self, account_id).await?;\n                }\n\n                let result = self.get_used_quota(account_id).await?;\n\n                Ok(JsonResponse::new(json!({\n                    \"data\": result,\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n\n    async fn housekeeper_request(&self, event: HousekeeperEvent) -> trc::Result<HttpResponse> {\n        self.inner\n            .ipc\n            .housekeeper_tx\n            .send(event)\n            .await\n            .map_err(|err| {\n                trc::EventType::Server(trc::ServerEvent::ThreadError)\n                    .reason(err)\n                    .details(\"Failed to send housekeeper event\")\n            })?;\n\n        Ok(JsonResponse::new(json!({\n            \"data\": (),\n        }))\n        .into_http_response())\n    }\n}\n\npub async fn recalculate_quota(server: &Server, account_id: u32) -> trc::Result<()> {\n    let mut quota = 0;\n\n    for collection in [\n        Collection::Email,\n        Collection::Calendar,\n        Collection::CalendarEvent,\n        Collection::CalendarEventNotification,\n        Collection::AddressBook,\n        Collection::ContactCard,\n        Collection::FileNode,\n    ] {\n        server\n            .archives(account_id, collection, &(), |_, archive| {\n                match collection {\n                    Collection::Email => {\n                        quota += archive.unarchive::<MessageData>()?.size.to_native() as i64;\n                    }\n                    Collection::Calendar => {\n                        quota += archive.unarchive::<Calendar>()?.size() as i64;\n                    }\n                    Collection::CalendarEvent => {\n                        quota += archive.unarchive::<CalendarEvent>()?.size() as i64;\n                    }\n                    Collection::CalendarEventNotification => {\n                        quota += archive.unarchive::<CalendarEventNotification>()?.size() as i64;\n                    }\n                    Collection::AddressBook => {\n                        quota += archive.unarchive::<AddressBook>()?.size() as i64;\n                    }\n                    Collection::ContactCard => {\n                        quota += archive.unarchive::<ContactCard>()?.size() as i64;\n                    }\n                    Collection::FileNode => {\n                        quota += archive.unarchive::<FileNode>()?.size() as i64;\n                    }\n                    _ => {}\n                }\n                Ok(true)\n            })\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    let mut batch = BatchBuilder::new();\n    batch\n        .clear(DirectoryClass::UsedQuota(account_id))\n        .add(DirectoryClass::UsedQuota(account_id), quota);\n    server\n        .store()\n        .write(batch.build_all())\n        .await\n        .caused_by(trc::location!())\n        .map(|_| ())\n}\n\npub async fn destroy_account_blobs(server: &Server, account_id: u32) -> trc::Result<()> {\n    let mut delete_keys = Vec::new();\n    for (collection, field) in [\n        (Collection::Email, u8::from(EmailField::Metadata)),\n        (Collection::FileNode, u8::from(Field::ARCHIVE)),\n        (Collection::SieveScript, u8::from(Field::ARCHIVE)),\n    ] {\n        server\n            .all_archives(account_id, collection, field, |document_id, archive| {\n                match collection {\n                    Collection::Email => {\n                        let message = archive.unarchive::<MessageMetadata>()?;\n                        delete_keys.push((\n                            collection,\n                            document_id,\n                            BlobHash::from(&message.blob_hash),\n                        ));\n                    }\n                    Collection::FileNode => {\n                        if let Some(file) = archive.unarchive::<FileNode>()?.file.as_ref() {\n                            delete_keys.push((\n                                collection,\n                                document_id,\n                                BlobHash::from(&file.blob_hash),\n                            ));\n                        }\n                    }\n                    Collection::SieveScript => {\n                        let sieve = archive.unarchive::<SieveScript>()?;\n                        delete_keys.push((\n                            collection,\n                            document_id,\n                            BlobHash::from(&sieve.blob_hash),\n                        ));\n                    }\n                    _ => {}\n                }\n                Ok(())\n            })\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    let mut batch = BatchBuilder::new();\n    batch.with_account_id(account_id);\n\n    for (collection, document_id, hash) in delete_keys {\n        if batch.is_large_batch() {\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n            batch = BatchBuilder::new();\n            batch.with_account_id(account_id);\n        }\n        batch\n            .with_collection(collection)\n            .with_document(document_id)\n            .clear(ValueClass::Blob(BlobOp::Link {\n                hash,\n                to: BlobLink::Document,\n            }));\n    }\n\n    if !batch.is_empty() {\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    Ok(())\n}\n\npub async fn destroy_account_data(\n    server: &Server,\n    account_id: u32,\n    has_data: bool,\n) -> trc::Result<()> {\n    // Unlink all accounts's blobs\n    if has_data {\n        destroy_account_blobs(server, account_id).await?;\n    }\n\n    // Destroy account data\n    server\n        .store()\n        .danger_destroy_account(account_id)\n        .await\n        .caused_by(trc::location!())?;\n\n    if has_data {\n        // Remove search index\n        for index in [\n            SearchIndex::Email,\n            SearchIndex::Contacts,\n            SearchIndex::Calendar,\n        ] {\n            if let Err(err) = server\n                .core\n                .storage\n                .fts\n                .unindex(SearchQuery::new(index).with_account_id(account_id))\n                .await\n            {\n                trc::error!(err.details(\"Failed to delete FTS index\"));\n            }\n        }\n    }\n\n    Ok(())\n}\n\npub async fn reset_imap_uids(server: &Server, account_id: u32) -> trc::Result<(u32, u32)> {\n    let mut mailbox_count = 0;\n    let mut email_count = 0;\n\n    let cache = server\n        .get_cached_messages(account_id)\n        .await\n        .caused_by(trc::location!())?;\n\n    for &mailbox_id in cache.mailboxes.index.keys() {\n        let mailbox = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::Mailbox,\n                mailbox_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or_else(|| trc::ImapEvent::Error.into_err().caused_by(trc::location!()))?\n            .into_deserialized::<email::mailbox::Mailbox>()\n            .caused_by(trc::location!())?;\n        let mut new_mailbox = mailbox.inner.clone();\n        new_mailbox.uid_validity = rand::random::<u32>();\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Mailbox)\n            .with_document(mailbox_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_current(mailbox)\n                    .with_changes(new_mailbox),\n            )\n            .caused_by(trc::location!())?\n            .clear(MailboxField::UidCounter);\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n        mailbox_count += 1;\n    }\n\n    // Reset all UIDs\n    for message_id in cache.emails.items.iter().map(|i| i.document_id) {\n        let data = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::Email,\n                message_id,\n            ))\n            .await\n            .caused_by(trc::location!())?;\n        let data_ = if let Some(data) = data {\n            data\n        } else {\n            continue;\n        };\n        let data = data_\n            .to_unarchived::<MessageData>()\n            .caused_by(trc::location!())?;\n        let mut new_data = data\n            .deserialize::<MessageData>()\n            .caused_by(trc::location!())?;\n\n        let ids = server\n            .assign_email_ids(\n                account_id,\n                new_data.mailboxes.iter().map(|m| m.mailbox_id),\n                false,\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        for (uid_mailbox, uid) in new_data.mailboxes.iter_mut().zip(ids) {\n            uid_mailbox.uid = uid;\n        }\n\n        // Prepare write batch\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Email)\n            .with_document(message_id)\n            .assert_value(ValueClass::Property(EmailField::Archive.into()), &data)\n            .set(\n                EmailField::Archive,\n                Archiver::new(new_data)\n                    .serialize()\n                    .caused_by(trc::location!())?,\n            );\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n        email_count += 1;\n    }\n\n    Ok((mailbox_count, email_count))\n}\n"
  },
  {
    "path": "crates/http/src/management/troubleshoot.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    future::Future,\n    net::{IpAddr, SocketAddr},\n    time::{Duration, Instant},\n};\n\nuse common::{\n    Server,\n    auth::{AccessToken, oauth::GrantType},\n    config::smtp::{\n        queue::MxConfig,\n        resolver::{Policy, Tlsa},\n    },\n    psl,\n};\nuse directory::backend::internal::manage;\nuse http_body_util::{StreamBody, combinators::BoxBody};\nuse hyper::{\n    Method, StatusCode,\n    body::{Bytes, Frame},\n};\nuse mail_auth::{\n    AuthenticatedMessage, DkimResult, DmarcResult, IpLookupStrategy, IprevOutput, IprevResult,\n    SpfOutput, SpfResult,\n    dmarc::{self, verify::DmarcParameters},\n    mta_sts::TlsRpt,\n    spf::verify::SpfParameters,\n};\nuse serde::{Deserialize, Serialize};\nuse serde_json::json;\nuse smtp::outbound::{\n    client::{SmtpClient, StartTlsResult},\n    dane::{dnssec::TlsaLookup, verify::TlsaVerify},\n    lookup::{DnsLookup, ToNextHop},\n    mta_sts::{lookup::MtaStsLookup, verify::VerifyPolicy},\n};\nuse tokio::{io::AsyncWriteExt, sync::mpsc};\nuse utils::url_params::UrlParams;\n\nuse http_proto::{request::decode_path_element, *};\n\npub trait TroubleshootApi: Sync + Send {\n    fn handle_troubleshoot_api_request(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n        body: Option<Vec<u8>>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl TroubleshootApi for Server {\n    async fn handle_troubleshoot_api_request(\n        &self,\n        req: &HttpRequest,\n        path: Vec<&str>,\n        access_token: &AccessToken,\n        body: Option<Vec<u8>>,\n    ) -> trc::Result<HttpResponse> {\n        let params = UrlParams::new(req.uri().query());\n        let account_id = access_token.primary_id();\n\n        match (\n            path.get(1).copied().unwrap_or_default(),\n            path.get(2).copied(),\n            req.method(),\n        ) {\n            (\"token\", None, &Method::GET) => {\n                // Issue a live telemetry token valid for 60 seconds\n                Ok(JsonResponse::new(json!({\n                    \"data\": self.encode_access_token(GrantType::Troubleshoot, account_id,  \"web\", 60).await?,\n            }))\n            .into_http_response())\n            }\n            (\"delivery\", Some(target), &Method::GET) => {\n                let timeout = Duration::from_secs(\n                    params\n                        .parse::<u64>(\"timeout\")\n                        .filter(|interval| *interval >= 1)\n                        .unwrap_or(30),\n                );\n\n                let mut rx = spawn_delivery_troubleshoot(\n                    self.clone(),\n                    decode_path_element(target).to_lowercase(),\n                    timeout,\n                );\n\n                Ok(HttpResponse::new(StatusCode::OK)\n                    .with_content_type(\"text/event-stream\")\n                    .with_cache_control(\"no-store\")\n                    .with_stream_body(BoxBody::new(StreamBody::new(async_stream::stream! {\n                        while let Some(stage) = rx.recv().await {\n                            yield Ok(stage.to_frame());\n                        }\n                        yield Ok(DeliveryStage::Completed.to_frame());\n                    }))))\n            }\n            (\"dmarc\", None, &Method::POST) => {\n                let request = serde_json::from_slice::<DmarcTroubleshootRequest>(\n                    body.as_deref().unwrap_or_default(),\n                )\n                .map_err(|err| {\n                    trc::EventType::Resource(trc::ResourceEvent::BadParameters).from_json_error(err)\n                })?;\n                let response = dmarc_troubleshoot(self, request).await.ok_or_else(|| {\n                    manage::error(\n                        \"Invalid message body\",\n                        \"Failed to parse message body\".into(),\n                    )\n                })?;\n\n                Ok(JsonResponse::new(json!({\n                        \"data\": response,\n                }))\n                .into_http_response())\n            }\n            _ => Err(trc::ResourceEvent::NotFound.into_err()),\n        }\n    }\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\n#[serde(tag = \"type\")]\nenum DeliveryStage {\n    MxLookupStart {\n        domain: String,\n    },\n    MxLookupSuccess {\n        mxs: Vec<MX>,\n        elapsed: u64,\n    },\n    MxLookupError {\n        reason: String,\n        elapsed: u64,\n    },\n    MtaStsFetchStart,\n    MtaStsFetchSuccess {\n        policy: Policy,\n        elapsed: u64,\n    },\n    MtaStsFetchError {\n        reason: String,\n        elapsed: u64,\n    },\n    MtaStsNotFound {\n        elapsed: u64,\n    },\n    TlsRptLookupStart,\n    TlsRptLookupSuccess {\n        rua: Vec<ReportUri>,\n        elapsed: u64,\n    },\n    TlsRptLookupError {\n        reason: String,\n        elapsed: u64,\n    },\n    TlsRptNotFound {\n        elapsed: u64,\n    },\n    DeliveryAttemptStart {\n        hostname: String,\n    },\n    MtaStsVerifySuccess,\n    MtaStsVerifyError {\n        reason: String,\n    },\n    TlsaLookupStart,\n    TlsaLookupSuccess {\n        record: Tlsa,\n        elapsed: u64,\n    },\n    TlsaNotFound {\n        elapsed: u64,\n        reason: String,\n    },\n    TlsaLookupError {\n        elapsed: u64,\n        reason: String,\n    },\n    IpLookupStart,\n    IpLookupSuccess {\n        remote_ips: Vec<IpAddr>,\n        elapsed: u64,\n    },\n    IpLookupError {\n        reason: String,\n        elapsed: u64,\n    },\n    ConnectionStart {\n        remote_ip: IpAddr,\n    },\n    ConnectionSuccess {\n        elapsed: u64,\n    },\n    ConnectionError {\n        elapsed: u64,\n        reason: String,\n    },\n    ReadGreetingStart,\n    ReadGreetingSuccess {\n        elapsed: u64,\n    },\n    ReadGreetingError {\n        elapsed: u64,\n        reason: String,\n    },\n    EhloStart,\n    EhloSuccess {\n        elapsed: u64,\n    },\n    EhloError {\n        elapsed: u64,\n        reason: String,\n    },\n    StartTlsStart,\n    StartTlsSuccess {\n        elapsed: u64,\n    },\n    StartTlsError {\n        elapsed: u64,\n        reason: String,\n    },\n    DaneVerifySuccess,\n    DaneVerifyError {\n        reason: String,\n    },\n    MailFromStart,\n    MailFromSuccess {\n        elapsed: u64,\n    },\n    MailFromError {\n        reason: String,\n        elapsed: u64,\n    },\n    RcptToStart,\n    RcptToSuccess {\n        elapsed: u64,\n    },\n    RcptToError {\n        reason: String,\n        elapsed: u64,\n    },\n    QuitStart,\n    QuitCompleted {\n        elapsed: u64,\n    },\n    Completed,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\nstruct MX {\n    pub exchanges: Vec<String>,\n    pub preference: u16,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\n#[serde(tag = \"type\")]\npub enum ReportUri {\n    Mail { email: String },\n    Http { url: String },\n}\n\nimpl DeliveryStage {\n    fn to_frame(&self) -> Frame<Bytes> {\n        let payload = format!(\n            \"event: event\\ndata: [{}]\\n\\n\",\n            serde_json::to_string(self).unwrap_or_default()\n        );\n        Frame::data(Bytes::from(payload))\n    }\n}\n\ntrait ElapsedMs {\n    fn elapsed_ms(&self) -> u64;\n}\n\nimpl ElapsedMs for Instant {\n    fn elapsed_ms(&self) -> u64 {\n        self.elapsed().as_millis() as u64\n    }\n}\nfn spawn_delivery_troubleshoot(\n    server: Server,\n    domain_or_email: String,\n    timeout: Duration,\n) -> mpsc::Receiver<DeliveryStage> {\n    let (tx, rx) = mpsc::channel(10);\n\n    tokio::spawn(async move {\n        let _ = delivery_troubleshoot(tx, server, domain_or_email, timeout).await;\n    });\n\n    rx\n}\n\nasync fn delivery_troubleshoot(\n    tx: mpsc::Sender<DeliveryStage>,\n    server: Server,\n    domain_or_email: String,\n    timeout: Duration,\n) -> Result<(), mpsc::error::SendError<DeliveryStage>> {\n    let (domain, email) = if let Some((_, domain)) = domain_or_email.rsplit_once('@') {\n        (domain.to_string(), Some(domain_or_email))\n    } else {\n        (domain_or_email, None)\n    };\n\n    let local_host = &server.core.network.server_name;\n\n    tx.send(DeliveryStage::MxLookupStart {\n        domain: domain.to_string(),\n    })\n    .await?;\n\n    // Lookup MX\n    let now = Instant::now();\n    let mxs = match server\n        .core\n        .smtp\n        .resolvers\n        .dns\n        .mx_lookup(&domain, Some(&server.inner.cache.dns_mx))\n        .await\n    {\n        Ok(mxs) => mxs,\n        Err(err) => {\n            tx.send(DeliveryStage::MxLookupError {\n                reason: err.to_string(),\n                elapsed: now.elapsed_ms(),\n            })\n            .await?;\n\n            return Ok(());\n        }\n    };\n\n    // Obtain remote host list\n    let mx_config = MxConfig {\n        max_mx: mxs.len(),\n        max_multi_homed: 10,\n        ip_lookup_strategy: IpLookupStrategy::Ipv4thenIpv6,\n    };\n    let hosts = if let Some(hosts) = mxs.to_remote_hosts(&domain, &mx_config) {\n        tx.send(DeliveryStage::MxLookupSuccess {\n            mxs: mxs\n                .iter()\n                .map(|mx| MX {\n                    exchanges: mx.exchanges.clone(),\n                    preference: mx.preference,\n                })\n                .collect(),\n            elapsed: now.elapsed_ms(),\n        })\n        .await?;\n\n        hosts\n    } else {\n        tx.send(DeliveryStage::MxLookupError {\n            reason: \"Null MX record\".to_string(),\n            elapsed: now.elapsed_ms(),\n        })\n        .await?;\n\n        return Ok(());\n    };\n\n    // Fetch MTA-STS policy\n    let now = Instant::now();\n    tx.send(DeliveryStage::MtaStsFetchStart).await?;\n    let mta_sts_policy = match server.lookup_mta_sts_policy(&domain, timeout).await {\n        Ok(policy) => {\n            tx.send(DeliveryStage::MtaStsFetchSuccess {\n                policy: policy.as_ref().clone(),\n                elapsed: now.elapsed_ms(),\n            })\n            .await?;\n            Some(policy)\n        }\n        Err(err) => {\n            if matches!(\n                &err,\n                smtp::outbound::mta_sts::Error::Dns(mail_auth::Error::DnsRecordNotFound(_))\n            ) {\n                tx.send(DeliveryStage::MtaStsNotFound {\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n            } else {\n                tx.send(DeliveryStage::MtaStsFetchError {\n                    reason: err.to_string(),\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n            }\n            None\n        }\n    };\n\n    // Fetch TLS-RPT settings\n    let now = Instant::now();\n    tx.send(DeliveryStage::TlsRptLookupStart).await?;\n    match server\n        .core\n        .smtp\n        .resolvers\n        .dns\n        .txt_lookup::<TlsRpt>(\n            format!(\"_smtp._tls.{domain}.\"),\n            Some(&server.inner.cache.dns_txt),\n        )\n        .await\n    {\n        Ok(record) => {\n            tx.send(DeliveryStage::TlsRptLookupSuccess {\n                rua: record\n                    .rua\n                    .iter()\n                    .map(|r| match r {\n                        mail_auth::mta_sts::ReportUri::Mail(email) => ReportUri::Mail {\n                            email: email.clone(),\n                        },\n                        mail_auth::mta_sts::ReportUri::Http(url) => {\n                            ReportUri::Http { url: url.clone() }\n                        }\n                    })\n                    .collect(),\n                elapsed: now.elapsed_ms(),\n            })\n            .await?;\n        }\n        Err(err) => {\n            if matches!(&err, mail_auth::Error::DnsRecordNotFound(_)) {\n                tx.send(DeliveryStage::TlsRptNotFound {\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n            } else {\n                tx.send(DeliveryStage::TlsRptLookupError {\n                    reason: err.to_string(),\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n            }\n        }\n    }\n\n    // Try with each host\n    'outer: for host in hosts {\n        let hostname = host.hostname();\n\n        tx.send(DeliveryStage::DeliveryAttemptStart {\n            hostname: hostname.to_string(),\n        })\n        .await?;\n\n        // Verify MTA-STS policy\n        if let Some(mta_sts_policy) = &mta_sts_policy {\n            if mta_sts_policy.verify(hostname) {\n                tx.send(DeliveryStage::MtaStsVerifySuccess).await?;\n            } else {\n                tx.send(DeliveryStage::MtaStsVerifyError {\n                    reason: \"Not authorized by policy\".to_string(),\n                })\n                .await?;\n\n                continue;\n            }\n        }\n\n        // Fetch TLSA record\n        tx.send(DeliveryStage::TlsaLookupStart).await?;\n\n        let now = Instant::now();\n        let dane_policy = match server.tlsa_lookup(format!(\"_25._tcp.{hostname}.\")).await {\n            Ok(Some(tlsa)) if tlsa.has_end_entities => {\n                tx.send(DeliveryStage::TlsaLookupSuccess {\n                    record: tlsa.as_ref().clone(),\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n\n                Some(tlsa)\n            }\n            Ok(Some(_)) => {\n                tx.send(DeliveryStage::TlsaLookupError {\n                    elapsed: now.elapsed_ms(),\n                    reason: \"TLSA record does not have end entities\".to_string(),\n                })\n                .await?;\n\n                None\n            }\n            Ok(None) => {\n                tx.send(DeliveryStage::TlsaNotFound {\n                    elapsed: now.elapsed_ms(),\n                    reason: \"No TLSA DNSSEC records found\".to_string(),\n                })\n                .await?;\n\n                None\n            }\n            Err(err) => {\n                if matches!(&err, mail_auth::Error::DnsRecordNotFound(_)) {\n                    tx.send(DeliveryStage::TlsaNotFound {\n                        elapsed: now.elapsed_ms(),\n                        reason: \"No TLSA records found for MX\".to_string(),\n                    })\n                    .await?;\n                } else {\n                    tx.send(DeliveryStage::TlsaLookupError {\n                        elapsed: now.elapsed_ms(),\n                        reason: err.to_string(),\n                    })\n                    .await?;\n                }\n                None\n            }\n        };\n\n        tx.send(DeliveryStage::IpLookupStart).await?;\n\n        let now = Instant::now();\n        match server\n            .ip_lookup(\n                host.fqdn_hostname().as_ref(),\n                IpLookupStrategy::Ipv4thenIpv6,\n                usize::MAX,\n            )\n            .await\n        {\n            Ok(remote_ips) if !remote_ips.is_empty() => {\n                tx.send(DeliveryStage::IpLookupSuccess {\n                    remote_ips: remote_ips.clone(),\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n\n                for remote_ip in remote_ips {\n                    // Start connection\n                    tx.send(DeliveryStage::ConnectionStart { remote_ip })\n                        .await?;\n\n                    let now = Instant::now();\n                    match SmtpClient::connect(SocketAddr::new(remote_ip, 25), timeout, 0).await {\n                        Ok(mut client) => {\n                            tx.send(DeliveryStage::ConnectionSuccess {\n                                elapsed: now.elapsed_ms(),\n                            })\n                            .await?;\n\n                            // Read greeting\n                            tx.send(DeliveryStage::ReadGreetingStart).await?;\n\n                            let now = Instant::now();\n                            if let Err(status) = client.read_greeting(hostname).await {\n                                tx.send(DeliveryStage::ReadGreetingError {\n                                    elapsed: now.elapsed_ms(),\n                                    reason: status.to_string(),\n                                })\n                                .await?;\n\n                                continue;\n                            }\n                            tx.send(DeliveryStage::ReadGreetingSuccess {\n                                elapsed: now.elapsed_ms(),\n                            })\n                            .await?;\n\n                            // Say EHLO\n                            tx.send(DeliveryStage::EhloStart).await?;\n\n                            let now = Instant::now();\n                            let capabilities = match tokio::time::timeout(timeout, async {\n                                client\n                                    .stream\n                                    .write_all(format!(\"EHLO {local_host}\\r\\n\",).as_bytes())\n                                    .await?;\n                                client.stream.flush().await?;\n                                client.read_ehlo().await\n                            })\n                            .await\n                            {\n                                Ok(Ok(capabilities)) => {\n                                    tx.send(DeliveryStage::EhloSuccess {\n                                        elapsed: now.elapsed_ms(),\n                                    })\n                                    .await?;\n\n                                    capabilities\n                                }\n                                Ok(Err(err)) => {\n                                    tx.send(DeliveryStage::EhloError {\n                                        elapsed: now.elapsed_ms(),\n                                        reason: err.to_string(),\n                                    })\n                                    .await?;\n\n                                    continue;\n                                }\n                                Err(_) => {\n                                    tx.send(DeliveryStage::EhloError {\n                                        elapsed: now.elapsed_ms(),\n                                        reason: \"Timed out reading response\".to_string(),\n                                    })\n                                    .await?;\n\n                                    continue;\n                                }\n                            };\n\n                            // Start TLS\n                            tx.send(DeliveryStage::StartTlsStart).await?;\n\n                            let now = Instant::now();\n                            let mut client = match client\n                                .try_start_tls(\n                                    &server.inner.data.smtp_connectors.pki_verify,\n                                    hostname,\n                                    &capabilities,\n                                )\n                                .await\n                            {\n                                StartTlsResult::Success { smtp_client } => {\n                                    tx.send(DeliveryStage::StartTlsSuccess {\n                                        elapsed: now.elapsed_ms(),\n                                    })\n                                    .await?;\n\n                                    smtp_client\n                                }\n                                StartTlsResult::Error { error } => {\n                                    tx.send(DeliveryStage::StartTlsError {\n                                        elapsed: now.elapsed_ms(),\n                                        reason: error.to_string(),\n                                    })\n                                    .await?;\n\n                                    continue;\n                                }\n                                StartTlsResult::Unavailable { response, .. } => {\n                                    tx.send(DeliveryStage::StartTlsError {\n                                        elapsed: now.elapsed_ms(),\n                                        reason: response.map(|r| r.to_string()).unwrap_or_else(\n                                            || \"STARTTLS not advertised by host\".to_string(),\n                                        ),\n                                    })\n                                    .await?;\n\n                                    continue;\n                                }\n                            };\n\n                            // Verify DANE policy\n                            if let Some(dane_policy) = &dane_policy {\n                                if let Err(err) = dane_policy.verify(\n                                    0,\n                                    hostname,\n                                    client.tls_connection().peer_certificates(),\n                                ) {\n                                    tx.send(DeliveryStage::DaneVerifyError {\n                                        reason: err.to_string(),\n                                    })\n                                    .await?;\n                                } else {\n                                    tx.send(DeliveryStage::DaneVerifySuccess).await?;\n                                }\n                            }\n\n                            // Say EHLO again (some SMTP servers require this)\n                            tx.send(DeliveryStage::EhloStart).await?;\n\n                            let now = Instant::now();\n                            match tokio::time::timeout(timeout, async {\n                                client\n                                    .stream\n                                    .write_all(format!(\"EHLO {local_host}\\r\\n\",).as_bytes())\n                                    .await?;\n                                client.stream.flush().await?;\n                                client.read_ehlo().await\n                            })\n                            .await\n                            {\n                                Ok(Ok(_)) => {\n                                    tx.send(DeliveryStage::EhloSuccess {\n                                        elapsed: now.elapsed_ms(),\n                                    })\n                                    .await?;\n                                }\n                                Ok(Err(err)) => {\n                                    tx.send(DeliveryStage::EhloError {\n                                        elapsed: now.elapsed_ms(),\n                                        reason: err.to_string(),\n                                    })\n                                    .await?;\n\n                                    continue;\n                                }\n                                Err(_) => {\n                                    tx.send(DeliveryStage::EhloError {\n                                        elapsed: now.elapsed_ms(),\n                                        reason: \"Timed out reading response\".to_string(),\n                                    })\n                                    .await?;\n\n                                    continue;\n                                }\n                            }\n\n                            // Verify recipient\n                            let mut is_success = email.is_none();\n                            if let Some(email) = &email {\n                                // MAIL FROM\n                                tx.send(DeliveryStage::MailFromStart).await?;\n\n                                let now = Instant::now();\n\n                                match client.cmd(b\"MAIL FROM:<>\\r\\n\").await.and_then(|r| {\n                                    if r.is_positive_completion() {\n                                        Ok(r)\n                                    } else {\n                                        Err(mail_send::Error::UnexpectedReply(r))\n                                    }\n                                }) {\n                                    Ok(_) => {\n                                        tx.send(DeliveryStage::MailFromSuccess {\n                                            elapsed: now.elapsed_ms(),\n                                        })\n                                        .await?;\n\n                                        // RCPT TO\n                                        tx.send(DeliveryStage::RcptToStart).await?;\n\n                                        let now = Instant::now();\n                                        match client\n                                            .cmd(format!(\"RCPT TO:<{email}>\\r\\n\").as_bytes())\n                                            .await\n                                            .and_then(|r| {\n                                                if r.is_positive_completion() {\n                                                    Ok(r)\n                                                } else {\n                                                    Err(mail_send::Error::UnexpectedReply(r))\n                                                }\n                                            }) {\n                                            Ok(_) => {\n                                                is_success = true;\n                                                tx.send(DeliveryStage::RcptToSuccess {\n                                                    elapsed: now.elapsed_ms(),\n                                                })\n                                                .await?;\n                                            }\n                                            Err(err) => {\n                                                tx.send(DeliveryStage::RcptToError {\n                                                    reason: err.to_string(),\n                                                    elapsed: now.elapsed_ms(),\n                                                })\n                                                .await?;\n                                            }\n                                        }\n                                    }\n                                    Err(err) => {\n                                        tx.send(DeliveryStage::MailFromError {\n                                            reason: err.to_string(),\n                                            elapsed: now.elapsed_ms(),\n                                        })\n                                        .await?;\n                                    }\n                                }\n                            }\n\n                            // QUIT\n                            tx.send(DeliveryStage::QuitStart).await?;\n\n                            let now = Instant::now();\n                            client.quit().await;\n                            tx.send(DeliveryStage::QuitCompleted {\n                                elapsed: now.elapsed_ms(),\n                            })\n                            .await?;\n\n                            if is_success {\n                                break 'outer;\n                            }\n                        }\n                        Err(err) => {\n                            tx.send(DeliveryStage::ConnectionError {\n                                elapsed: now.elapsed_ms(),\n                                reason: err.to_string(),\n                            })\n                            .await?;\n                        }\n                    }\n                }\n            }\n            Ok(_) => {\n                tx.send(DeliveryStage::IpLookupError {\n                    reason: \"No IP addresses found for host\".to_string(),\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n            }\n            Err(err) => {\n                tx.send(DeliveryStage::IpLookupError {\n                    reason: err.to_string(),\n                    elapsed: now.elapsed_ms(),\n                })\n                .await?;\n            }\n        }\n    }\n\n    Ok(())\n}\n\n#[derive(Debug, Serialize, Deserialize)]\nstruct DmarcTroubleshootRequest {\n    #[serde(rename = \"remoteIp\")]\n    remote_ip: IpAddr,\n    #[serde(rename = \"ehloDomain\")]\n    ehlo_domain: String,\n    #[serde(rename = \"mailFrom\")]\n    mail_from: String,\n    body: Option<String>,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\nstruct DmarcTroubleshootResponse {\n    #[serde(rename = \"spfEhloDomain\")]\n    spf_ehlo_domain: String,\n    #[serde(rename = \"spfEhloResult\")]\n    spf_ehlo_result: AuthResult,\n    #[serde(rename = \"spfMailFromDomain\")]\n    spf_mail_from_domain: String,\n    #[serde(rename = \"spfMailFromResult\")]\n    spf_mail_from_result: AuthResult,\n    #[serde(rename = \"ipRevResult\")]\n    ip_rev_result: AuthResult,\n    #[serde(rename = \"ipRevPtr\")]\n    ip_rev_ptr: Vec<String>,\n    #[serde(rename = \"dkimResults\")]\n    dkim_results: Vec<AuthResult>,\n    #[serde(rename = \"dkimPass\")]\n    dkim_pass: bool,\n    #[serde(rename = \"arcResult\")]\n    arc_result: AuthResult,\n    #[serde(rename = \"dmarcResult\")]\n    dmarc_result: AuthResult,\n    #[serde(rename = \"dmarcPass\")]\n    dmarc_pass: bool,\n    #[serde(rename = \"dmarcPolicy\")]\n    dmarc_policy: DmarcPolicy,\n    elapsed: u64,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\n#[serde(tag = \"type\")]\npub enum AuthResult {\n    Pass,\n    Fail { details: Option<String> },\n    SoftFail { details: Option<String> },\n    TempError { details: Option<String> },\n    PermError { details: Option<String> },\n    Neutral { details: Option<String> },\n    None,\n}\n\n#[derive(Debug, Serialize, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub enum DmarcPolicy {\n    None,\n    Quarantine,\n    Reject,\n    Unspecified,\n}\n\nasync fn dmarc_troubleshoot(\n    server: &Server,\n    request: DmarcTroubleshootRequest,\n) -> Option<DmarcTroubleshootResponse> {\n    let remote_ip = request.remote_ip;\n    let ehlo_domain = request.ehlo_domain.to_lowercase();\n    let mail_from = request.mail_from.to_lowercase();\n    let mail_from_domain = mail_from.rsplit_once('@').map(|(_, domain)| domain);\n\n    let local_host = &server.core.network.server_name;\n\n    let now = Instant::now();\n    let ehlo_spf_output = server\n        .core\n        .smtp\n        .resolvers\n        .dns\n        .verify_spf(\n            server\n                .inner\n                .cache\n                .build_auth_parameters(SpfParameters::verify_ehlo(\n                    remote_ip,\n                    &ehlo_domain,\n                    local_host,\n                )),\n        )\n        .await;\n\n    let iprev = server\n        .core\n        .smtp\n        .resolvers\n        .dns\n        .verify_iprev(server.inner.cache.build_auth_parameters(remote_ip))\n        .await;\n    let mail_spf_output = if let Some(mail_from_domain) = mail_from_domain {\n        server\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .check_host(server.inner.cache.build_auth_parameters(SpfParameters::new(\n                remote_ip,\n                mail_from_domain,\n                &ehlo_domain,\n                local_host,\n                &mail_from,\n            )))\n            .await\n    } else {\n        server\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .check_host(server.inner.cache.build_auth_parameters(SpfParameters::new(\n                remote_ip,\n                &ehlo_domain,\n                &ehlo_domain,\n                local_host,\n                &format!(\"postmaster@{ehlo_domain}\"),\n            )))\n            .await\n    };\n\n    let body = request\n        .body\n        .unwrap_or_else(|| format!(\"From: {mail_from}\\r\\nSubject: test\\r\\n\\r\\ntest\"));\n    let auth_message = AuthenticatedMessage::parse_with_opts(body.as_bytes(), true)?;\n\n    let dkim_output = server\n        .core\n        .smtp\n        .resolvers\n        .dns\n        .verify_dkim(server.inner.cache.build_auth_parameters(&auth_message))\n        .await;\n    let dkim_pass = dkim_output\n        .iter()\n        .any(|d| matches!(d.result(), DkimResult::Pass));\n\n    let arc_output = server\n        .core\n        .smtp\n        .resolvers\n        .dns\n        .verify_arc(server.inner.cache.build_auth_parameters(&auth_message))\n        .await;\n\n    let dmarc_output = server\n        .core\n        .smtp\n        .resolvers\n        .dns\n        .verify_dmarc(server.inner.cache.build_auth_parameters(DmarcParameters {\n            message: &auth_message,\n            dkim_output: &dkim_output,\n            rfc5321_mail_from_domain: mail_from_domain.unwrap_or(ehlo_domain.as_str()),\n            spf_output: &mail_spf_output,\n            domain_suffix_fn: |domain| psl::domain_str(domain).unwrap_or(domain),\n        }))\n        .await;\n    let dmarc_pass = matches!(dmarc_output.spf_result(), DmarcResult::Pass)\n        || matches!(dmarc_output.dkim_result(), DmarcResult::Pass);\n    let dmarc_result = if dmarc_pass {\n        DmarcResult::Pass\n    } else if dmarc_output.spf_result() != &DmarcResult::None {\n        dmarc_output.spf_result().clone()\n    } else if dmarc_output.dkim_result() != &DmarcResult::None {\n        dmarc_output.dkim_result().clone()\n    } else {\n        DmarcResult::None\n    };\n\n    Some(DmarcTroubleshootResponse {\n        spf_ehlo_domain: ehlo_spf_output.domain().to_string(),\n        spf_ehlo_result: (&ehlo_spf_output).into(),\n        spf_mail_from_domain: mail_spf_output.domain().to_string(),\n        spf_mail_from_result: (&mail_spf_output).into(),\n        ip_rev_ptr: iprev\n            .ptr\n            .as_ref()\n            .map(|ptr| ptr.as_ref().clone())\n            .unwrap_or_default(),\n        ip_rev_result: (&iprev).into(),\n        dkim_pass,\n        dkim_results: dkim_output\n            .iter()\n            .map(|result| result.result().into())\n            .collect(),\n        arc_result: arc_output.result().into(),\n        dmarc_result: (&dmarc_result).into(),\n        dmarc_policy: (&dmarc_output.policy()).into(),\n        dmarc_pass,\n        elapsed: now.elapsed_ms(),\n    })\n}\n\nimpl From<&SpfOutput> for AuthResult {\n    fn from(value: &SpfOutput) -> Self {\n        match value.result() {\n            SpfResult::Pass => AuthResult::Pass,\n            SpfResult::Fail => AuthResult::Fail {\n                details: value.explanation().map(|e| e.to_string()),\n            },\n            SpfResult::SoftFail => AuthResult::SoftFail {\n                details: value.explanation().map(|e| e.to_string()),\n            },\n            SpfResult::Neutral => AuthResult::Neutral {\n                details: value.explanation().map(|e| e.to_string()),\n            },\n            SpfResult::TempError => AuthResult::TempError {\n                details: value.explanation().map(|e| e.to_string()),\n            },\n            SpfResult::PermError => AuthResult::PermError {\n                details: value.explanation().map(|e| e.to_string()),\n            },\n            SpfResult::None => AuthResult::None,\n        }\n    }\n}\n\nimpl From<AuthResult> for SpfOutput {\n    fn from(value: AuthResult) -> Self {\n        match value {\n            AuthResult::Pass => SpfOutput::new(String::new()).with_result(SpfResult::Pass),\n            AuthResult::Fail { .. } => SpfOutput::new(String::new()).with_result(SpfResult::Fail),\n            AuthResult::SoftFail { .. } => {\n                SpfOutput::new(String::new()).with_result(SpfResult::SoftFail)\n            }\n            AuthResult::Neutral { .. } => {\n                SpfOutput::new(String::new()).with_result(SpfResult::Neutral)\n            }\n            AuthResult::TempError { .. } => {\n                SpfOutput::new(String::new()).with_result(SpfResult::TempError)\n            }\n            AuthResult::PermError { .. } => {\n                SpfOutput::new(String::new()).with_result(SpfResult::PermError)\n            }\n            AuthResult::None => SpfOutput::new(String::new()).with_result(SpfResult::None),\n        }\n    }\n}\n\nimpl From<&IprevOutput> for AuthResult {\n    fn from(value: &IprevOutput) -> Self {\n        match &value.result {\n            IprevResult::Pass => AuthResult::Pass,\n            IprevResult::Fail(error) => AuthResult::Fail {\n                details: error.to_string().into(),\n            },\n            IprevResult::TempError(error) => AuthResult::TempError {\n                details: error.to_string().into(),\n            },\n            IprevResult::PermError(error) => AuthResult::PermError {\n                details: error.to_string().into(),\n            },\n            IprevResult::None => AuthResult::None,\n        }\n    }\n}\n\nimpl From<AuthResult> for IprevResult {\n    fn from(value: AuthResult) -> Self {\n        match value {\n            AuthResult::Pass => IprevResult::Pass,\n            AuthResult::Fail { details } => {\n                IprevResult::Fail(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::TempError { details } => {\n                IprevResult::TempError(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::PermError { details } => {\n                IprevResult::PermError(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::None => IprevResult::None,\n            _ => IprevResult::None,\n        }\n    }\n}\n\nimpl From<&DkimResult> for AuthResult {\n    fn from(value: &DkimResult) -> Self {\n        match value {\n            DkimResult::Pass => AuthResult::Pass,\n            DkimResult::Neutral(error) => AuthResult::Neutral {\n                details: error.to_string().into(),\n            },\n            DkimResult::Fail(error) => AuthResult::Fail {\n                details: error.to_string().into(),\n            },\n            DkimResult::PermError(error) => AuthResult::PermError {\n                details: error.to_string().into(),\n            },\n            DkimResult::TempError(error) => AuthResult::TempError {\n                details: error.to_string().into(),\n            },\n            DkimResult::None => AuthResult::None,\n        }\n    }\n}\n\nimpl From<AuthResult> for DkimResult {\n    fn from(value: AuthResult) -> Self {\n        match value {\n            AuthResult::Pass => DkimResult::Pass,\n            AuthResult::Neutral { details } => {\n                DkimResult::Neutral(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::Fail { details } => {\n                DkimResult::Fail(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::PermError { details } => {\n                DkimResult::PermError(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::TempError { details } => {\n                DkimResult::TempError(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            _ => DkimResult::None,\n        }\n    }\n}\n\nimpl From<&DmarcResult> for AuthResult {\n    fn from(value: &DmarcResult) -> Self {\n        match value {\n            DmarcResult::Pass => AuthResult::Pass,\n            DmarcResult::Fail(error) => AuthResult::Fail {\n                details: error.to_string().into(),\n            },\n            DmarcResult::TempError(error) => AuthResult::TempError {\n                details: error.to_string().into(),\n            },\n            DmarcResult::PermError(error) => AuthResult::PermError {\n                details: error.to_string().into(),\n            },\n            DmarcResult::None => AuthResult::None,\n        }\n    }\n}\n\nimpl From<AuthResult> for DmarcResult {\n    fn from(value: AuthResult) -> Self {\n        match value {\n            AuthResult::Pass => DmarcResult::Pass,\n            AuthResult::Fail { details } => {\n                DmarcResult::Fail(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::TempError { details } => {\n                DmarcResult::TempError(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::PermError { details } => {\n                DmarcResult::PermError(mail_auth::Error::Io(details.unwrap_or_default()))\n            }\n            AuthResult::None => DmarcResult::None,\n            _ => DmarcResult::None,\n        }\n    }\n}\n\nimpl From<&dmarc::Policy> for DmarcPolicy {\n    fn from(value: &dmarc::Policy) -> Self {\n        match value {\n            dmarc::Policy::None => DmarcPolicy::None,\n            dmarc::Policy::Quarantine => DmarcPolicy::Quarantine,\n            dmarc::Policy::Reject => DmarcPolicy::Reject,\n            dmarc::Policy::Unspecified => DmarcPolicy::Unspecified,\n        }\n    }\n}\n\nimpl From<DmarcPolicy> for dmarc::Policy {\n    fn from(value: DmarcPolicy) -> Self {\n        match value {\n            DmarcPolicy::None => dmarc::Policy::None,\n            DmarcPolicy::Quarantine => dmarc::Policy::Quarantine,\n            DmarcPolicy::Reject => dmarc::Policy::Reject,\n            DmarcPolicy::Unspecified => dmarc::Policy::Unspecified,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http/src/request.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    HttpSessionManager,\n    auth::{\n        authenticate::{Authenticator, HttpHeaders},\n        oauth::{\n            FormData, auth::OAuthApiHandler, openid::OpenIdHandler,\n            registration::ClientRegistrationHandler, token::TokenHandler,\n        },\n    },\n    autoconfig::Autoconfig,\n    form::FormHandler,\n    management::{\n        ManagementApi, ToManageHttpResponse, UnauthorizedResponse, troubleshoot::TroubleshootApi,\n    },\n};\nuse common::{\n    Inner, KV_ACME, Server,\n    auth::{AccessToken, oauth::GrantType},\n    core::BuildServer,\n    ipc::PushEvent,\n    listener::{SessionData, SessionManager, SessionStream},\n    manager::webadmin::Resource,\n};\nuse dav::{DavMethod, request::DavRequestHandler};\nuse directory::Permission;\nuse groupware::{DavResourceName, calendar::itip::ItipIngest};\nuse http_proto::{\n    DownloadResponse, HtmlResponse, HttpContext, HttpRequest, HttpResponse, HttpResponseBody,\n    HttpSessionData, JsonProblemResponse, ToHttpResponse, form_urlencoded, request::fetch_body,\n};\nuse hyper::{\n    Method, StatusCode, body,\n    header::{self, CONTENT_TYPE},\n    server::conn::http1,\n    service::service_fn,\n};\nuse hyper_util::rt::TokioIo;\nuse jmap::{\n    api::{\n        ToJmapHttpResponse, event_source::EventSourceHandler, request::RequestHandler,\n        session::SessionHandler,\n    },\n    blob::{download::BlobDownload, upload::BlobUpload},\n    websocket::upgrade::WebSocketUpgrade,\n};\nuse jmap_proto::request::{Request, capability::Session};\nuse std::{net::IpAddr, str::FromStr, sync::Arc};\nuse store::dispatch::lookup::KeyValue;\nuse trc::SecurityEvent;\nuse types::{blob::BlobId, id::Id};\nuse utils::url_params::UrlParams;\n\npub trait ParseHttp: Sync + Send {\n    fn parse_http_request(\n        &self,\n        req: HttpRequest,\n        session: HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl ParseHttp for Server {\n    async fn parse_http_request(\n        &self,\n        mut req: HttpRequest,\n        session: HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        let mut path = req.uri().path().split('/');\n        path.next();\n\n        // Validate endpoint access\n        let ctx = HttpContext::new(&session, &req);\n        match ctx.has_endpoint_access(self).await {\n            StatusCode::OK => (),\n            status => {\n                // Allow loopback address to avoid lockouts\n                if !session.remote_ip.is_loopback() {\n                    return Ok(JsonProblemResponse(status).into_http_response());\n                }\n            }\n        }\n\n        match path.next().unwrap_or_default() {\n            \"jmap\" => {\n                match (path.next().unwrap_or_default(), req.method()) {\n                    (\"\", &Method::POST) => {\n                        // Authenticate request\n                        let (_in_flight, access_token) =\n                            self.authenticate_headers(&req, &session, false).await?;\n\n                        let bytes = fetch_body(\n                            &mut req,\n                            if !access_token.has_permission(Permission::UnlimitedUploads) {\n                                self.core.jmap.upload_max_size\n                            } else {\n                                0\n                            },\n                            session.session_id,\n                        )\n                        .await\n                        .ok_or_else(|| trc::LimitEvent::SizeRequest.into_err())?;\n\n                        return Ok(self\n                            .handle_jmap_request(\n                                Request::parse(\n                                    &bytes,\n                                    self.core.jmap.request_max_calls,\n                                    self.core.jmap.request_max_size,\n                                )?,\n                                access_token,\n                                &session,\n                            )\n                            .await\n                            .into_http_response());\n                    }\n                    (\"download\", &Method::GET) => {\n                        // Authenticate request\n                        let (_in_flight, access_token) =\n                            self.authenticate_headers(&req, &session, false).await?;\n\n                        if let (Some(_), Some(blob_id), Some(name)) = (\n                            path.next().and_then(|p| Id::from_str(p).ok()),\n                            path.next().and_then(BlobId::from_base32),\n                            path.next(),\n                        ) {\n                            return match self.blob_download(&blob_id, &access_token).await? {\n                                Some(blob) => Ok(DownloadResponse {\n                                    filename: name.to_string(),\n                                    content_type: req\n                                        .uri()\n                                        .query()\n                                        .and_then(|q| {\n                                            form_urlencoded::parse(q.as_bytes())\n                                                .find(|(k, _)| k == \"accept\")\n                                                .map(|(_, v)| v.into_owned())\n                                        })\n                                        .unwrap_or(\"application/octet-stream\".to_string()),\n                                    blob,\n                                }\n                                .into_http_response()),\n                                None => Err(trc::ResourceEvent::NotFound.into_err()),\n                            };\n                        }\n                    }\n                    (\"upload\", &Method::POST) => {\n                        // Authenticate request\n                        let (_in_flight, access_token) =\n                            self.authenticate_headers(&req, &session, false).await?;\n\n                        if let Some(account_id) = path.next().and_then(|p| Id::from_str(p).ok()) {\n                            return match fetch_body(\n                                &mut req,\n                                if !access_token.has_permission(Permission::UnlimitedUploads) {\n                                    self.core.jmap.upload_max_size\n                                } else {\n                                    0\n                                },\n                                session.session_id,\n                            )\n                            .await\n                            {\n                                Some(bytes) => Ok(self\n                                    .blob_upload(\n                                        account_id,\n                                        req.headers()\n                                            .get(CONTENT_TYPE)\n                                            .and_then(|h| h.to_str().ok())\n                                            .unwrap_or(\"application/octet-stream\"),\n                                        &bytes,\n                                        access_token,\n                                    )\n                                    .await?\n                                    .into_http_response()),\n                                None => Err(trc::LimitEvent::SizeUpload.into_err()),\n                            };\n                        }\n                    }\n                    (\"eventsource\", &Method::GET) => {\n                        // Authenticate request\n                        let (_in_flight, access_token) =\n                            self.authenticate_headers(&req, &session, false).await?;\n\n                        return self.handle_event_source(req, access_token).await;\n                    }\n                    (\"ws\", &Method::GET) => {\n                        // Authenticate request\n                        let (_in_flight, access_token) =\n                            self.authenticate_headers(&req, &session, false).await?;\n\n                        return self\n                            .upgrade_websocket_connection(req, access_token, session)\n                            .await;\n                    }\n                    (\"session\", &Method::GET) => {\n                        return if req.headers().contains_key(header::AUTHORIZATION) {\n                            // Authenticate request\n                            let (_in_flight, access_token) =\n                                self.authenticate_headers(&req, &session, false).await?;\n\n                            self.handle_session_resource(\n                                ctx.resolve_response_url(self).await,\n                                access_token,\n                            )\n                            .await\n                            .map(|s| s.into_http_response())\n                        } else {\n                            Ok(Session::new(\n                                ctx.resolve_response_url(self).await,\n                                &self.core.jmap.capabilities,\n                            )\n                            .into_http_response())\n                        };\n                    }\n                    (_, &Method::OPTIONS) => {\n                        return Ok(JsonProblemResponse(StatusCode::NO_CONTENT).into_http_response());\n                    }\n                    _ => (),\n                }\n            }\n            \"dav\" => {\n                let response = match (\n                    path.next().and_then(DavResourceName::parse),\n                    DavMethod::parse(req.method()),\n                ) {\n                    (Some(_), Some(DavMethod::OPTIONS)) => HttpResponse::new(StatusCode::OK)\n                        .with_header(\n                            \"DAV\",\n                            concat!(\n                                \"1, 2, 3, access-control, extended-mkcol, calendar-access, \",\n                                \"calendar-auto-schedule, calendar-no-timezone, addressbook\"\n                            ),\n                        )\n                        .with_header(\n                            \"Allow\",\n                            concat!(\n                                \"OPTIONS, GET, HEAD, POST, PUT, DELETE, COPY, MOVE, MKCALENDAR, \",\n                                \"MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, REPORT, ACL\"\n                            ),\n                        ),\n                    (Some(resource), Some(method)) => {\n                        // Authenticate request\n                        let (_in_flight, access_token) =\n                            self.authenticate_headers(&req, &session, false).await?;\n\n                        self.handle_dav_request(req, access_token, &session, resource, method)\n                            .await\n                    }\n                    (_, None) => HttpResponse::new(StatusCode::METHOD_NOT_ALLOWED),\n                    (None, _) => HttpResponse::new(StatusCode::NOT_FOUND),\n                };\n\n                return Ok(response);\n            }\n            \".well-known\" => match (path.next().unwrap_or_default(), req.method()) {\n                (\"jmap\", &Method::GET) => {\n                    return Ok(HttpResponse::new(StatusCode::TEMPORARY_REDIRECT)\n                        .with_no_cache()\n                        .with_location(\"/jmap/session\"));\n                }\n                (\"caldav\", _) => {\n                    return Ok(HttpResponse::new(StatusCode::TEMPORARY_REDIRECT)\n                        .with_no_cache()\n                        .with_location(DavResourceName::Cal.base_path()));\n                }\n                (\"carddav\", _) => {\n                    return Ok(HttpResponse::new(StatusCode::TEMPORARY_REDIRECT)\n                        .with_no_cache()\n                        .with_location(DavResourceName::Card.base_path()));\n                }\n                (\"oauth-authorization-server\", &Method::GET) => {\n                    // Limit anonymous requests\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return self.handle_oauth_metadata(req, session).await;\n                }\n                (\"openid-configuration\", &Method::GET) => {\n                    // Limit anonymous requests\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return self.handle_oidc_metadata(req, session).await;\n                }\n                (\"acme-challenge\", &Method::GET) if self.has_acme_http_providers() => {\n                    if let Some(token) = path.next() {\n                        return match self\n                            .core\n                            .storage\n                            .lookup\n                            .key_get::<String>(KeyValue::<()>::build_key(KV_ACME, token))\n                            .await?\n                        {\n                            Some(proof) => Ok(Resource::new(\"text/plain\", proof.into_bytes())\n                                .into_http_response()),\n                            None => Err(trc::ResourceEvent::NotFound.into_err()),\n                        };\n                    }\n                }\n                (\"mta-sts.txt\", &Method::GET) => {\n                    // Limit anonymous requests\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return if let Some(policy) = self.build_mta_sts_policy() {\n                        Ok(Resource::new(\"text/plain\", policy.to_string().into_bytes())\n                            .into_http_response())\n                    } else {\n                        Err(trc::ResourceEvent::NotFound.into_err())\n                    };\n                }\n                (\"mail-v1.xml\", &Method::GET) => {\n                    // Limit anonymous requests\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return self.handle_autoconfig_request(&req).await;\n                }\n                (\"autoconfig\", &Method::GET) => {\n                    if path.next().unwrap_or_default() == \"mail\"\n                        && path.next().unwrap_or_default() == \"config-v1.1.xml\"\n                    {\n                        // Limit anonymous requests\n                        self.is_http_anonymous_request_allowed(&session.remote_ip)\n                            .await?;\n\n                        return self.handle_autoconfig_request(&req).await;\n                    }\n                }\n                (_, &Method::OPTIONS) => {\n                    return Ok(JsonProblemResponse(StatusCode::NO_CONTENT).into_http_response());\n                }\n                _ => (),\n            },\n            \"auth\" => match (path.next().unwrap_or_default(), req.method()) {\n                (\"device\", &Method::POST) => {\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return self.handle_device_auth(&mut req, session).await;\n                }\n                (\"token\", &Method::POST) => {\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return self.handle_token_request(&mut req, session).await;\n                }\n                (\"introspect\", &Method::POST) => {\n                    // Authenticate request\n                    let (_in_flight, access_token) =\n                        self.authenticate_headers(&req, &session, false).await?;\n\n                    return self\n                        .handle_token_introspect(&mut req, &access_token, session.session_id)\n                        .await;\n                }\n                (\"userinfo\", &Method::GET) => {\n                    // Authenticate request\n                    let (_in_flight, access_token) =\n                        self.authenticate_headers(&req, &session, false).await?;\n\n                    return self.handle_userinfo_request(&access_token).await;\n                }\n                (\"register\", &Method::POST) => {\n                    return self\n                        .handle_oauth_registration_request(&mut req, session)\n                        .await;\n                }\n                (\"jwks.json\", &Method::GET) => {\n                    // Limit anonymous requests\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return Ok(self.core.oauth.oidc_jwks.clone().into_http_response());\n                }\n                (_, &Method::OPTIONS) => {\n                    return Ok(JsonProblemResponse(StatusCode::NO_CONTENT).into_http_response());\n                }\n                _ => (),\n            },\n            \"api\" => {\n                // Allow CORS preflight requests\n                if req.method() == Method::OPTIONS {\n                    return Ok(JsonProblemResponse(StatusCode::NO_CONTENT).into_http_response());\n                }\n\n                // Authenticate user\n                match self.authenticate_headers(&req, &session, true).await {\n                    Ok((_, access_token)) => {\n                        return self\n                            .handle_api_manage_request(&mut req, access_token, &session)\n                            .await;\n                    }\n                    Err(err) => {\n                        if err.matches(trc::EventType::Auth(trc::AuthEvent::Failed)) {\n                            let params = UrlParams::new(req.uri().query());\n                            let path = req.uri().path().split('/').skip(2).collect::<Vec<_>>();\n\n                            let (grant_type, token) = match (\n                                path.first().copied(),\n                                path.get(1).copied(),\n                                params.get(\"token\"),\n                            ) {\n                                // SPDX-SnippetBegin\n                                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                                // SPDX-License-Identifier: LicenseRef-SEL\n                                #[cfg(feature = \"enterprise\")]\n                                (Some(\"telemetry\"), Some(\"traces\"), Some(token))\n                                    if self.core.is_enterprise_edition() =>\n                                {\n                                    (GrantType::LiveTracing, token)\n                                }\n                                #[cfg(feature = \"enterprise\")]\n                                (Some(\"telemetry\"), Some(\"metrics\"), Some(token))\n                                    if self.core.is_enterprise_edition() =>\n                                {\n                                    (GrantType::LiveMetrics, token)\n                                }\n                                // SPDX-SnippetEnd\n                                (Some(\"troubleshoot\"), _, Some(token)) => {\n                                    (GrantType::Troubleshoot, token)\n                                }\n                                _ => return Ok(HttpResponse::unauthorized(false)),\n                            };\n                            let token_info =\n                                self.validate_access_token(grant_type.into(), token).await?;\n\n                            return match grant_type {\n                                // SPDX-SnippetBegin\n                                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                                // SPDX-License-Identifier: LicenseRef-SEL\n                                #[cfg(feature = \"enterprise\")]\n                                GrantType::LiveTracing | GrantType::LiveMetrics => {\n                                    use crate::management::enterprise::telemetry::TelemetryApi;\n                                    self.handle_telemetry_api_request(\n                                        &req,\n                                        path,\n                                        &AccessToken::from_id(token_info.account_id)\n                                            .with_permission(Permission::MetricsLive)\n                                            .with_permission(Permission::TracingLive),\n                                    )\n                                    .await\n                                }\n                                // SPDX-SnippetEnd\n                                GrantType::Troubleshoot => {\n                                    self.handle_troubleshoot_api_request(\n                                        &req,\n                                        path,\n                                        &AccessToken::from_id(token_info.account_id)\n                                            .with_permission(Permission::Troubleshoot),\n                                        None,\n                                    )\n                                    .await\n                                }\n                                _ => unreachable!(),\n                            };\n                        }\n\n                        return Err(err);\n                    }\n                }\n            }\n            \"mail\" => {\n                if req.method() == Method::GET\n                    && path.next().unwrap_or_default() == \"config-v1.1.xml\"\n                {\n                    // Limit anonymous requests\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return self.handle_autoconfig_request(&req).await;\n                }\n            }\n            \"calendar\" => {\n                // Limit anonymous requests\n                self.is_http_anonymous_request_allowed(&session.remote_ip)\n                    .await?;\n\n                if self.core.groupware.itip_http_rsvp_url.is_some()\n                    && req.method() == Method::GET\n                    && path.next().unwrap_or_default() == \"rsvp\"\n                {\n                    return self\n                        .http_rsvp_handle(\n                            req.uri().query().unwrap_or_default(),\n                            req.headers()\n                                .get(header::ACCEPT_LANGUAGE)\n                                .and_then(|v| v.to_str().ok())\n                                .map(|lang| {\n                                    let lang = lang.split_once(',').map_or(lang, |(l, _)| l);\n                                    lang.split_once(';').map_or(lang, |(l, _)| l)\n                                })\n                                .unwrap_or(\"en\"),\n                        )\n                        .await\n                        .map(|response| {\n                            HtmlResponse::new(response)\n                                .into_http_response()\n                                .with_no_store()\n                        });\n                }\n            }\n            \"autodiscover\" | \"Autodiscover\" => {\n                if req.method() == Method::POST\n                    && path\n                        .next()\n                        .unwrap_or_default()\n                        .eq_ignore_ascii_case(\"autodiscover.xml\")\n                {\n                    // Limit anonymous requests\n                    self.is_http_anonymous_request_allowed(&session.remote_ip)\n                        .await?;\n\n                    return self\n                        .handle_autodiscover_request(\n                            fetch_body(&mut req, 8192, session.session_id).await,\n                        )\n                        .await;\n                }\n            }\n            \"robots.txt\" => {\n                // Limit anonymous requests\n                self.is_http_anonymous_request_allowed(&session.remote_ip)\n                    .await?;\n\n                return Ok(\n                    Resource::new(\"text/plain\", b\"User-agent: *\\nDisallow: /\\n\".to_vec())\n                        .into_http_response(),\n                );\n            }\n            \"healthz\" => {\n                // Limit anonymous requests\n                self.is_http_anonymous_request_allowed(&session.remote_ip)\n                    .await?;\n\n                match path.next().unwrap_or_default() {\n                    \"live\" => {\n                        return Ok(JsonProblemResponse(StatusCode::OK).into_http_response());\n                    }\n                    \"ready\" => {\n                        return Ok(JsonProblemResponse({\n                            if !self.core.storage.data.is_none() {\n                                StatusCode::OK\n                            } else {\n                                StatusCode::SERVICE_UNAVAILABLE\n                            }\n                        })\n                        .into_http_response());\n                    }\n                    _ => (),\n                }\n            }\n            \"metrics\" => match path.next().unwrap_or_default() {\n                \"prometheus\" => {\n                    if let Some(prometheus) = &self.core.metrics.prometheus {\n                        if let Some(auth) = &prometheus.auth\n                            && req\n                                .authorization_basic()\n                                .is_none_or(|secret| secret != auth)\n                        {\n                            return Err(trc::AuthEvent::Failed\n                                .into_err()\n                                .details(\"Invalid or missing credentials.\")\n                                .caused_by(trc::location!()));\n                        }\n\n                        return Ok(Resource::new(\n                            \"text/plain; version=0.0.4\",\n                            self.export_prometheus_metrics().await?.into_bytes(),\n                        )\n                        .into_http_response());\n                    }\n                }\n                \"otel\" => {\n                    // Reserved for future use\n                }\n                _ => (),\n            },\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            \"logo.svg\" if self.is_enterprise_edition() => {\n                match self\n                    .logo_resource(\n                        req.headers()\n                            .get(header::HOST)\n                            .and_then(|h| h.to_str().ok())\n                            .map(|h| h.rsplit_once(':').map_or(h, |(h, _)| h))\n                            .unwrap_or_default(),\n                    )\n                    .await\n                {\n                    Ok(Some(resource)) => {\n                        return Ok(resource.into_http_response());\n                    }\n                    Ok(None) => (),\n                    Err(err) => {\n                        trc::error!(err.span_id(session.session_id));\n                    }\n                }\n\n                let resource = self.inner.data.webadmin.get(\"logo.svg\").await?;\n\n                if !resource.is_empty() {\n                    return Ok(resource.into_http_response());\n                }\n            }\n            // SPDX-SnippetEnd\n            \"form\" => {\n                if let Some(form) = &self.core.network.contact_form {\n                    match *req.method() {\n                        Method::POST => {\n                            self.is_http_anonymous_request_allowed(&session.remote_ip)\n                                .await?;\n\n                            let form_data =\n                                FormData::from_request(&mut req, form.max_size, session.session_id)\n                                    .await?;\n\n                            return self.handle_contact_form(&session, form, form_data).await;\n                        }\n                        Method::OPTIONS => {\n                            return Ok(\n                                JsonProblemResponse(StatusCode::NO_CONTENT).into_http_response()\n                            );\n                        }\n                        _ => {}\n                    }\n                }\n            }\n            _ => {\n                let path = req.uri().path();\n                let resource = self\n                    .inner\n                    .data\n                    .webadmin\n                    .get(path.strip_prefix('/').unwrap_or(path))\n                    .await?;\n\n                if !resource.is_empty() {\n                    return Ok(resource.into_http_response());\n                }\n            }\n        }\n\n        // Block dangerous URLs\n        let path = req.uri().path();\n        if self.is_http_banned_path(path, session.remote_ip).await? {\n            trc::event!(\n                Security(SecurityEvent::ScanBan),\n                SpanId = session.session_id,\n                RemoteIp = session.remote_ip,\n                Path = path.to_string(),\n            );\n        }\n\n        Err(trc::ResourceEvent::NotFound.into_err())\n    }\n}\n\nasync fn handle_session<T: SessionStream>(inner: Arc<Inner>, session: SessionData<T>) {\n    let _in_flight = session.in_flight;\n    let is_tls = session.stream.is_tls();\n\n    if let Err(http_err) = http1::Builder::new()\n        .keep_alive(true)\n        .serve_connection(\n            TokioIo::new(session.stream),\n            service_fn(|req: hyper::Request<body::Incoming>| {\n                let instance = session.instance.clone();\n                let inner = inner.clone();\n\n                async move {\n                    let server = inner.build_server();\n\n                    // Obtain remote IP\n                    let remote_ip = if !server.core.jmap.http_use_forwarded {\n                        trc::event!(\n                            Http(trc::HttpEvent::RequestUrl),\n                            SpanId = session.session_id,\n                            Url = req.uri().to_string(),\n                        );\n\n                        session.remote_ip\n                    } else if let Some(forwarded_for) = req\n                        .headers()\n                        .get(header::FORWARDED)\n                        .and_then(|h| h.to_str().ok())\n                        .and_then(|h| {\n                            let h = h.to_ascii_lowercase();\n                            h.split_once(\"for=\").and_then(|(_, rest)| {\n                                let mut start_ip = usize::MAX;\n                                let mut end_ip = usize::MAX;\n\n                                for (pos, ch) in rest.char_indices() {\n                                    match ch {\n                                        '0'..='9' | 'a'..='f' | ':' | '.' => {\n                                            if start_ip == usize::MAX {\n                                                start_ip = pos;\n                                            }\n                                            end_ip = pos;\n                                        }\n                                        '\"' | '[' | ' ' if start_ip == usize::MAX => {}\n                                        _ => {\n                                            break;\n                                        }\n                                    }\n                                }\n\n                                rest.get(start_ip..=end_ip)\n                                    .and_then(|h| h.parse::<IpAddr>().ok())\n                            })\n                        })\n                        .or_else(|| {\n                            req.headers()\n                                .get(\"X-Forwarded-For\")\n                                .and_then(|h| h.to_str().ok())\n                                .map(|h| h.split_once(',').map_or(h, |(ip, _)| ip).trim())\n                                .and_then(|h| h.parse::<IpAddr>().ok())\n                        })\n                    {\n                        // Check if the forwarded IP has been blocked\n                        if server.is_ip_blocked(&forwarded_for) {\n                            trc::event!(\n                                Security(trc::SecurityEvent::IpBlocked),\n                                ListenerId = instance.id.clone(),\n                                RemoteIp = forwarded_for,\n                                SpanId = session.session_id,\n                            );\n\n                            return Ok::<_, hyper::Error>(\n                                JsonProblemResponse(StatusCode::FORBIDDEN)\n                                    .into_http_response()\n                                    .build(),\n                            );\n                        }\n\n                        trc::event!(\n                            Http(trc::HttpEvent::RequestUrl),\n                            SpanId = session.session_id,\n                            RemoteIp = forwarded_for,\n                            Url = req.uri().to_string(),\n                        );\n\n                        forwarded_for\n                    } else {\n                        trc::event!(\n                            Http(trc::HttpEvent::XForwardedMissing),\n                            SpanId = session.session_id,\n                        );\n                        session.remote_ip\n                    };\n\n                    // Parse HTTP request\n                    let response = match Box::pin(server.parse_http_request(\n                        req,\n                        HttpSessionData {\n                            instance,\n                            local_ip: session.local_ip,\n                            local_port: session.local_port,\n                            remote_ip,\n                            remote_port: session.remote_port,\n                            is_tls,\n                            session_id: session.session_id,\n                        },\n                    ))\n                    .await\n                    {\n                        Ok(response) => response,\n                        Err(err) => {\n                            let response = err.into_http_response();\n                            trc::error!(err.span_id(session.session_id));\n                            response\n                        }\n                    };\n\n                    trc::event!(\n                        Http(trc::HttpEvent::ResponseBody),\n                        SpanId = session.session_id,\n                        Contents = match response.body() {\n                            HttpResponseBody::Text(value) =>\n                                trc::Value::String(value.as_str().into()),\n                            HttpResponseBody::Binary(_) =>\n                                trc::Value::String(\"[binary data]\".into()),\n                            HttpResponseBody::Stream(_) => trc::Value::String(\"[stream]\".into()),\n                            _ => trc::Value::None,\n                        },\n                        Code = response.status().as_u16(),\n                        Size = response.size(),\n                    );\n\n                    // Build response\n                    let mut response = response.build();\n\n                    // Add custom headers\n                    if !server.core.jmap.http_headers.is_empty() {\n                        let headers = response.headers_mut();\n\n                        for (header, value) in &server.core.jmap.http_headers {\n                            headers.insert(header.clone(), value.clone());\n                        }\n                    }\n\n                    Ok::<_, hyper::Error>(response)\n                }\n            }),\n        )\n        .with_upgrades()\n        .await\n    {\n        if http_err.is_parse() {\n            let server = inner.build_server();\n            if !server.core.jmap.http_use_forwarded {\n                match server.is_scanner_fail2banned(session.remote_ip).await {\n                    Ok(true) => {\n                        trc::event!(\n                            Security(SecurityEvent::ScanBan),\n                            SpanId = session.session_id,\n                            RemoteIp = session.remote_ip,\n                            Reason = http_err.to_string(),\n                        );\n                        return;\n                    }\n                    Ok(false) => {}\n                    Err(err) => {\n                        trc::error!(\n                            err.span_id(session.session_id)\n                                .details(\"Failed to check for fail2ban\")\n                        );\n                    }\n                }\n            }\n        }\n\n        trc::event!(\n            Http(trc::HttpEvent::Error),\n            SpanId = session.session_id,\n            Reason = http_err.to_string(),\n        );\n    }\n}\n\nimpl SessionManager for HttpSessionManager {\n    fn handle<T: SessionStream>(self, session: SessionData<T>) -> impl Future<Output = ()> + Send {\n        handle_session(self.inner, session)\n    }\n\n    #[allow(clippy::manual_async_fn)]\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send {\n        async {\n            let _ = self.inner.ipc.push_tx.send(PushEvent::Stop).await;\n        }\n    }\n}\n"
  },
  {
    "path": "crates/http-proto/Cargo.toml",
    "content": "[package]\nname = \"http_proto\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\ncommon = { path = \"../common\" }\ntrc = { path = \"../trc\" }\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\nhyper-util = { version = \"0.1.1\", features = [\"tokio\"] }\nhttp-body-util = \"0.1.0\"\nform_urlencoded = \"1.1.0\"\npercent-encoding = \"2.3.1\"\ncompact_str = \"0.9.0\"\n\n[dev-dependencies]\n\n[features]\ntest_mode = []\nenterprise = []\n"
  },
  {
    "path": "crates/http-proto/src/context.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Server,\n    expr::{functions::ResolveVariable, *},\n};\nuse compact_str::{ToCompactString, format_compact};\nuse hyper::StatusCode;\n\nuse crate::{HttpContext, HttpRequest, HttpSessionData};\n\nimpl<'x> HttpContext<'x> {\n    pub fn new(session: &'x HttpSessionData, req: &'x HttpRequest) -> Self {\n        Self { session, req }\n    }\n\n    pub async fn resolve_response_url(&self, server: &Server) -> String {\n        server\n            .eval_if(\n                &server.core.network.http_response_url,\n                self,\n                self.session.session_id,\n            )\n            .await\n            .unwrap_or_else(|| {\n                format!(\n                    \"http{}://{}:{}\",\n                    if self.session.is_tls { \"s\" } else { \"\" },\n                    self.session.local_ip,\n                    self.session.local_port\n                )\n            })\n    }\n\n    pub async fn has_endpoint_access(&self, server: &Server) -> StatusCode {\n        server\n            .eval_if(\n                &server.core.network.http_allowed_endpoint,\n                self,\n                self.session.session_id,\n            )\n            .await\n            .unwrap_or(StatusCode::OK)\n    }\n}\n\nimpl ResolveVariable for HttpContext<'_> {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        match variable {\n            V_REMOTE_IP => self.session.remote_ip.to_compact_string().into(),\n            V_REMOTE_PORT => self.session.remote_port.into(),\n            V_LOCAL_IP => self.session.local_ip.to_compact_string().into(),\n            V_LOCAL_PORT => self.session.local_port.into(),\n            V_TLS => self.session.is_tls.into(),\n            V_PROTOCOL => if self.session.is_tls { \"https\" } else { \"http\" }.into(),\n            V_LISTENER => self.session.instance.id.as_str().into(),\n            V_URL => self.req.uri().to_compact_string().into(),\n            V_URL_PATH => self.req.uri().path().into(),\n            V_METHOD => self.req.method().as_str().into(),\n            V_HEADERS => self\n                .req\n                .headers()\n                .iter()\n                .map(|(h, v)| {\n                    Variable::String(\n                        format_compact!(\"{}: {}\", h.as_str(), v.to_str().unwrap_or_default())\n                            .into(),\n                    )\n                })\n                .collect::<Vec<_>>()\n                .into(),\n            _ => Variable::default(),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n"
  },
  {
    "path": "crates/http-proto/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod context;\npub mod request;\npub mod response;\n\npub use form_urlencoded;\n\nuse std::{net::IpAddr, sync::Arc};\n\nuse common::listener::ServerInstance;\nuse hyper::StatusCode;\n\npub type HttpRequest = hyper::Request<hyper::body::Incoming>;\n\npub struct JsonResponse<T: serde::Serialize> {\n    status: StatusCode,\n    inner: T,\n    no_cache: bool,\n}\n\npub struct HtmlResponse {\n    status: StatusCode,\n    body: String,\n}\n\npub enum HttpResponseBody {\n    Text(String),\n    Binary(Vec<u8>),\n    Stream(http_body_util::combinators::BoxBody<hyper::body::Bytes, hyper::Error>),\n    WebsocketUpgrade(String),\n    Empty,\n}\n\npub struct HttpResponse {\n    status: StatusCode,\n    builder: hyper::http::response::Builder,\n    body: HttpResponseBody,\n}\n\npub struct HttpContext<'x> {\n    pub session: &'x HttpSessionData,\n    pub req: &'x HttpRequest,\n}\n\npub struct HttpSessionData {\n    pub instance: Arc<ServerInstance>,\n    pub local_ip: IpAddr,\n    pub local_port: u16,\n    pub remote_ip: IpAddr,\n    pub remote_port: u16,\n    pub is_tls: bool,\n    pub session_id: u64,\n}\n\npub struct DownloadResponse {\n    pub filename: String,\n    pub content_type: String,\n    pub blob: Vec<u8>,\n}\n\npub struct JsonProblemResponse(pub StatusCode);\n\nimpl<T: serde::Serialize> JsonResponse<T> {\n    pub fn new(inner: T) -> Self {\n        JsonResponse {\n            inner,\n            status: StatusCode::OK,\n            no_cache: false,\n        }\n    }\n\n    pub fn with_status(status: StatusCode, inner: T) -> Self {\n        JsonResponse {\n            inner,\n            status,\n            no_cache: false,\n        }\n    }\n\n    pub fn no_cache(mut self) -> Self {\n        self.no_cache = true;\n        self\n    }\n}\n\nimpl HtmlResponse {\n    pub fn new(body: String) -> Self {\n        HtmlResponse {\n            body,\n            status: StatusCode::OK,\n        }\n    }\n\n    pub fn with_status(status: StatusCode, body: String) -> Self {\n        HtmlResponse { body, status }\n    }\n}\n\npub trait ToHttpResponse {\n    fn into_http_response(self) -> HttpResponse;\n}\n"
  },
  {
    "path": "crates/http-proto/src/request.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse compact_str::ToCompactString;\nuse http_body_util::BodyExt;\n\nuse crate::HttpRequest;\n\n#[inline]\npub fn decode_path_element(item: &str) -> Cow<'_, str> {\n    percent_encoding::percent_decode_str(item)\n        .decode_utf8()\n        .unwrap_or_else(|_| item.into())\n}\n\npub async fn fetch_body(\n    req: &mut HttpRequest,\n    max_size: usize,\n    session_id: u64,\n) -> Option<Vec<u8>> {\n    let mut bytes = Vec::with_capacity(1024);\n    while let Some(Ok(frame)) = req.frame().await {\n        if let Some(data) = frame.data_ref() {\n            if bytes.len() + data.len() <= max_size || max_size == 0 {\n                bytes.extend_from_slice(data);\n            } else {\n                trc::event!(\n                    Http(trc::HttpEvent::RequestBody),\n                    SpanId = session_id,\n                    Details = req\n                        .headers()\n                        .iter()\n                        .map(|(k, v)| trc::Value::Array(vec![\n                            k.as_str().to_compact_string().into(),\n                            v.to_str().unwrap_or_default().to_compact_string().into()\n                        ]))\n                        .collect::<Vec<_>>(),\n                    Contents = std::str::from_utf8(&bytes)\n                        .unwrap_or(\"[binary data]\")\n                        .to_string(),\n                    Size = bytes.len(),\n                    Limit = max_size,\n                );\n\n                return None;\n            }\n        }\n    }\n\n    trc::event!(\n        Http(trc::HttpEvent::RequestBody),\n        SpanId = session_id,\n        Details = req\n            .headers()\n            .iter()\n            .map(|(k, v)| trc::Value::Array(vec![\n                k.as_str().to_compact_string().into(),\n                v.to_str().unwrap_or_default().to_compact_string().into()\n            ]))\n            .collect::<Vec<_>>(),\n        Contents = std::str::from_utf8(&bytes)\n            .unwrap_or(\"[binary data]\")\n            .to_string(),\n        Size = bytes.len(),\n    );\n\n    bytes.into()\n}\n"
  },
  {
    "path": "crates/http-proto/src/response.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::manager::webadmin::Resource;\nuse http_body_util::{BodyExt, Full};\nuse hyper::{\n    StatusCode,\n    body::Bytes,\n    header::{self, HeaderName, HeaderValue},\n};\nuse serde_json::json;\n\nuse crate::{\n    DownloadResponse, HtmlResponse, HttpResponse, HttpResponseBody, JsonProblemResponse,\n    JsonResponse, ToHttpResponse,\n};\n\nimpl HttpResponse {\n    pub fn new(status: StatusCode) -> Self {\n        HttpResponse {\n            status,\n            builder: hyper::Response::builder().status(status),\n            body: HttpResponseBody::Empty,\n        }\n    }\n\n    pub fn with_content_type<V>(mut self, content_type: V) -> Self\n    where\n        V: TryInto<HeaderValue>,\n        <V as TryInto<HeaderValue>>::Error: Into<hyper::http::Error>,\n    {\n        self.builder = self.builder.header(header::CONTENT_TYPE, content_type);\n        self\n    }\n\n    pub fn with_status_code(mut self, status: StatusCode) -> Self {\n        self.status = status;\n        self.builder = self.builder.status(status);\n        self\n    }\n\n    pub fn with_content_length(mut self, content_length: usize) -> Self {\n        self.builder = self.builder.header(header::CONTENT_LENGTH, content_length);\n        self\n    }\n\n    pub fn with_etag(mut self, etag: String) -> Self {\n        self.builder = self.builder.header(header::ETAG, etag);\n        self\n    }\n\n    pub fn with_etag_opt(self, etag: Option<String>) -> Self {\n        if let Some(etag) = etag {\n            self.with_etag(etag)\n        } else {\n            self\n        }\n    }\n\n    pub fn with_schedule_tag_opt(mut self, tag: Option<u32>) -> Self {\n        if let Some(tag) = tag {\n            self.builder = self.builder.header(\"Schedule-Tag\", format!(\"\\\"{tag}\\\"\"));\n            self\n        } else {\n            self\n        }\n    }\n\n    pub fn with_last_modified(mut self, last_modified: String) -> Self {\n        self.builder = self.builder.header(header::LAST_MODIFIED, last_modified);\n        self\n    }\n\n    pub fn with_lock_token(mut self, token_uri: &str) -> Self {\n        self.builder = self.builder.header(\"Lock-Token\", format!(\"<{token_uri}>\"));\n        self\n    }\n\n    pub fn with_header<K, V>(mut self, name: K, value: V) -> Self\n    where\n        K: TryInto<HeaderName>,\n        <K as TryInto<HeaderName>>::Error: Into<hyper::http::Error>,\n        V: TryInto<HeaderValue>,\n        <V as TryInto<HeaderValue>>::Error: Into<hyper::http::Error>,\n    {\n        self.builder = self.builder.header(name, value);\n        self\n    }\n\n    pub fn with_xml_body(self, body: impl Into<String>) -> Self {\n        self.with_text_body(body)\n            .with_content_type(\"application/xml; charset=utf-8\")\n    }\n\n    pub fn with_text_body(mut self, body: impl Into<String>) -> Self {\n        let body = body.into();\n        let body_len = body.len();\n        self.body = HttpResponseBody::Text(body);\n        self.with_content_length(body_len)\n    }\n\n    pub fn with_binary_body(mut self, body: impl Into<Vec<u8>>) -> Self {\n        let body = body.into();\n        let body_len = body.len();\n        self.body = HttpResponseBody::Binary(body);\n        self.with_content_length(body_len)\n    }\n\n    pub fn with_stream_body(\n        mut self,\n        stream: http_body_util::combinators::BoxBody<hyper::body::Bytes, hyper::Error>,\n    ) -> Self {\n        self.body = HttpResponseBody::Stream(stream);\n        self\n    }\n\n    pub fn with_websocket_upgrade(mut self, derived_key: String) -> Self {\n        self.body = HttpResponseBody::WebsocketUpgrade(derived_key);\n        self\n    }\n\n    pub fn with_content_disposition<V>(mut self, content_disposition: V) -> Self\n    where\n        V: TryInto<HeaderValue>,\n        <V as TryInto<HeaderValue>>::Error: Into<hyper::http::Error>,\n    {\n        self.builder = self\n            .builder\n            .header(header::CONTENT_DISPOSITION, content_disposition);\n        self\n    }\n\n    pub fn with_cache_control<V>(mut self, cache_control: V) -> Self\n    where\n        V: TryInto<HeaderValue>,\n        <V as TryInto<HeaderValue>>::Error: Into<hyper::http::Error>,\n    {\n        self.builder = self.builder.header(header::CACHE_CONTROL, cache_control);\n        self\n    }\n\n    pub fn with_no_store(mut self) -> Self {\n        self.builder = self\n            .builder\n            .header(header::CACHE_CONTROL, \"no-store, no-cache, must-revalidate\");\n        self\n    }\n\n    pub fn with_no_cache(mut self) -> Self {\n        self.builder = self.builder.header(header::CACHE_CONTROL, \"no-cache\");\n        self\n    }\n\n    pub fn with_location<V>(mut self, location: V) -> Self\n    where\n        V: TryInto<HeaderValue>,\n        <V as TryInto<HeaderValue>>::Error: Into<hyper::http::Error>,\n    {\n        self.builder = self.builder.header(header::LOCATION, location);\n        self\n    }\n\n    pub fn size(&self) -> usize {\n        match &self.body {\n            HttpResponseBody::Text(value) => value.len(),\n            HttpResponseBody::Binary(value) => value.len(),\n            _ => 0,\n        }\n    }\n\n    pub fn build(\n        self,\n    ) -> hyper::Response<http_body_util::combinators::BoxBody<hyper::body::Bytes, hyper::Error>>\n    {\n        match self.body {\n            HttpResponseBody::Text(body) => self.builder.body(\n                Full::new(Bytes::from(body))\n                    .map_err(|never| match never {})\n                    .boxed(),\n            ),\n            HttpResponseBody::Binary(body) => self.builder.body(\n                Full::new(Bytes::from(body))\n                    .map_err(|never| match never {})\n                    .boxed(),\n            ),\n            HttpResponseBody::Empty => self.builder.header(header::CONTENT_LENGTH, 0).body(\n                Full::new(Bytes::new())\n                    .map_err(|never| match never {})\n                    .boxed(),\n            ),\n            HttpResponseBody::Stream(stream) => self.builder.body(stream),\n            HttpResponseBody::WebsocketUpgrade(derived_key) => self\n                .builder\n                .header(header::CONNECTION, \"upgrade\")\n                .header(header::UPGRADE, \"websocket\")\n                .header(\"Sec-WebSocket-Accept\", &derived_key)\n                .header(\"Sec-WebSocket-Protocol\", \"jmap\")\n                .body(\n                    Full::new(Bytes::from(\"Switching to WebSocket protocol\"))\n                        .map_err(|never| match never {})\n                        .boxed(),\n                ),\n        }\n        .unwrap()\n    }\n\n    pub fn body(&self) -> &HttpResponseBody {\n        &self.body\n    }\n\n    pub fn status(&self) -> StatusCode {\n        self.status\n    }\n\n    pub fn headers(&self) -> Option<&hyper::HeaderMap<HeaderValue>> {\n        self.builder.headers_ref()\n    }\n}\n\nimpl<T: serde::Serialize> ToHttpResponse for JsonResponse<T> {\n    fn into_http_response(self) -> HttpResponse {\n        let response = HttpResponse::new(self.status)\n            .with_content_type(\"application/json; charset=utf-8\")\n            .with_text_body(serde_json::to_string(&self.inner).unwrap_or_default());\n\n        if self.no_cache {\n            response.with_no_store()\n        } else {\n            response\n        }\n    }\n}\n\nimpl ToHttpResponse for DownloadResponse {\n    fn into_http_response(self) -> HttpResponse {\n        HttpResponse::new(StatusCode::OK)\n            .with_content_type(self.content_type)\n            .with_content_disposition(format!(\n                \"attachment; filename=\\\"{}\\\"\",\n                self.filename.replace('\\\"', \"\\\\\\\"\")\n            ))\n            .with_cache_control(\"private, immutable, max-age=31536000\")\n            .with_binary_body(self.blob)\n    }\n}\n\nimpl ToHttpResponse for Resource<Vec<u8>> {\n    fn into_http_response(self) -> HttpResponse {\n        HttpResponse::new(StatusCode::OK)\n            .with_content_type(self.content_type.as_ref())\n            .with_binary_body(self.contents)\n    }\n}\n\nimpl ToHttpResponse for HtmlResponse {\n    fn into_http_response(self) -> HttpResponse {\n        HttpResponse::new(self.status)\n            .with_content_type(\"text/html; charset=utf-8\")\n            .with_text_body(self.body)\n    }\n}\n\nimpl ToHttpResponse for JsonProblemResponse {\n    fn into_http_response(self) -> HttpResponse {\n        HttpResponse::new(self.0)\n            .with_content_type(\"application/problem+json\")\n            .with_text_body(\n                serde_json::to_string(&json!(\n                    {\n                        \"type\": \"about:blank\",\n                        \"title\": self.0.canonical_reason().unwrap_or_default(),\n                        \"status\": self.0.as_u16(),\n                        \"detail\": self.0.canonical_reason().unwrap_or_default(),\n                    }\n                ))\n                .unwrap_or_default(),\n            )\n    }\n}\n"
  },
  {
    "path": "crates/imap/Cargo.toml",
    "content": "[package]\nname = \"imap\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nimap_proto = { path = \"../imap-proto\" }\ntypes = { path = \"../types\" }\ndirectory = { path = \"../directory\" }\ntrc = { path = \"../trc\" }\nstore = { path = \"../store\" }\ncommon = { path = \"../common\" }\nemail = { path = \"../email\" }\nnlp = { path = \"../nlp\" }\nutils = { path = \"../utils\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pemfile = \"2.0\"\ntokio = { version = \"1.47\", features = [\"full\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\nparking_lot = \"0.12\"\nahash = { version = \"0.8\" }\nmd5 = \"0.8.0\"\nrand = \"0.9.0\"\nindexmap = \"2.7.1\"\ncompact_str = \"0.9.0\"\n\n[features]\ntest_mode = []\n"
  },
  {
    "path": "crates/imap/src/core/client.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{iter::Peekable, sync::Arc, vec::IntoIter};\n\nuse common::{\n    KV_RATE_LIMIT_IMAP,\n    listener::{SessionResult, SessionStream},\n};\nuse imap_proto::{\n    Command, ResponseType, StatusResponse,\n    receiver::{self, Request},\n};\nuse trc::SecurityEvent;\n\nuse super::{SelectedMailbox, Session, SessionData, State};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn ingest(&mut self, bytes: &[u8]) -> SessionResult {\n        trc::event!(\n            Imap(trc::ImapEvent::RawInput),\n            SpanId = self.session_id,\n            Size = bytes.len(),\n            Contents = trc::Value::from_maybe_string(bytes),\n        );\n\n        let mut bytes = bytes.iter();\n        let mut requests = Vec::with_capacity(2);\n        let mut needs_literal = None;\n\n        loop {\n            match self.receiver.parse(&mut bytes) {\n                Ok(request) => match self.is_allowed(request).await {\n                    Ok(request) => {\n                        requests.push(request);\n                    }\n                    Err(err) => {\n                        if !self.write_error(err).await {\n                            return SessionResult::Close;\n                        }\n                    }\n                },\n                Err(receiver::Error::NeedsMoreData) => {\n                    break;\n                }\n                Err(receiver::Error::NeedsLiteral { size }) => {\n                    needs_literal = size.into();\n                    break;\n                }\n                Err(receiver::Error::Error { response }) => {\n                    // Check for port scanners\n                    if matches!(\n                        (&self.state, response.key(trc::Key::Code)),\n                        (\n                            State::NotAuthenticated { .. },\n                            Some(trc::Value::String(v))\n                        ) if v == \"PARSE\"\n                    ) {\n                        match self.server.is_scanner_fail2banned(self.remote_addr).await {\n                            Ok(true) => {\n                                trc::event!(\n                                    Security(SecurityEvent::ScanBan),\n                                    SpanId = self.session_id,\n                                    RemoteIp = self.remote_addr,\n                                    Reason = \"Invalid IMAP command\",\n                                );\n\n                                return SessionResult::Close;\n                            }\n                            Ok(false) => {}\n                            Err(err) => {\n                                trc::error!(\n                                    err.span_id(self.session_id)\n                                        .details(\"Failed to check for fail2ban\")\n                                );\n                            }\n                        }\n                    }\n\n                    if !self.write_error(response).await {\n                        return SessionResult::Close;\n                    }\n                    break;\n                }\n            }\n        }\n\n        let mut requests = requests.into_iter().peekable();\n        while let Some(request) = requests.next() {\n            let result = match request.command {\n                Command::List | Command::Lsub => self\n                    .handle_list(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Select | Command::Examine => self\n                    .handle_select(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Create => self\n                    .handle_create(group_requests(&mut requests, vec![request]))\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Delete => self\n                    .handle_delete(group_requests(&mut requests, vec![request]))\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Rename => self\n                    .handle_rename(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Status => self\n                    .handle_status(group_requests(&mut requests, vec![request]))\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Append => self\n                    .handle_append(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Close => self\n                    .handle_close(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Unselect => self\n                    .handle_unselect(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Expunge(is_uid) => self\n                    .handle_expunge(request, is_uid)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Search(is_uid) => self\n                    .handle_search(request, false, is_uid)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Fetch(_) => self\n                    .handle_fetch(group_requests(&mut requests, vec![request]))\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Store(is_uid) => self\n                    .handle_store(request, is_uid)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Copy(is_uid) => self\n                    .handle_copy_move(request, false, is_uid)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Move(is_uid) => self\n                    .handle_copy_move(request, true, is_uid)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Sort(is_uid) => self\n                    .handle_search(request, true, is_uid)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Thread(is_uid) => self\n                    .handle_thread(request, is_uid)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Idle => self\n                    .handle_idle(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Subscribe => self\n                    .handle_subscribe(request, true)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Unsubscribe => self\n                    .handle_subscribe(request, false)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Namespace => self\n                    .handle_namespace(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Authenticate => self\n                    .handle_authenticate(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Login => self\n                    .handle_login(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Capability => self\n                    .handle_capability(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Enable => self\n                    .handle_enable(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::StartTls => self\n                    .write_bytes(\n                        StatusResponse::ok(\"Begin TLS negotiation now\")\n                            .with_tag(request.tag)\n                            .into_bytes(),\n                    )\n                    .await\n                    .map(|_| SessionResult::UpgradeTls),\n                Command::Noop => self\n                    .handle_noop(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Check => self\n                    .handle_noop(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Logout => self\n                    .handle_logout(request)\n                    .await\n                    .map(|_| SessionResult::Close),\n                Command::SetAcl => self\n                    .handle_set_acl(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::DeleteAcl => self\n                    .handle_set_acl(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::GetAcl => self\n                    .handle_get_acl(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::ListRights => self\n                    .handle_list_rights(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::MyRights => self\n                    .handle_my_rights(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::GetQuota => self\n                    .handle_get_quota(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::GetQuotaRoot => self\n                    .handle_get_quota_root(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Unauthenticate => self\n                    .handle_unauthenticate(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n                Command::Id => self\n                    .handle_id(request)\n                    .await\n                    .map(|_| SessionResult::Continue),\n            };\n\n            match result {\n                Ok(SessionResult::Continue) => (),\n                Ok(result) => return result,\n                Err(err) => {\n                    if !self.write_error(err).await {\n                        return SessionResult::Close;\n                    }\n                }\n            }\n        }\n\n        if let Some(needs_literal) = needs_literal\n            && let Err(err) = self\n                .write_bytes(format!(\"+ Ready for {} bytes.\\r\\n\", needs_literal).into_bytes())\n                .await\n        {\n            self.write_error(err).await;\n            return SessionResult::Close;\n        }\n\n        SessionResult::Continue\n    }\n}\n\npub fn group_requests(\n    requests: &mut Peekable<IntoIter<Request<Command>>>,\n    mut grouped_requests: Vec<Request<Command>>,\n) -> Vec<Request<Command>> {\n    let last_command = grouped_requests.last().unwrap().command;\n    loop {\n        match requests.peek() {\n            Some(request) if request.command == last_command => {\n                grouped_requests.push(requests.next().unwrap());\n            }\n            _ => break,\n        }\n    }\n    grouped_requests\n}\n\nimpl<T: SessionStream> Session<T> {\n    async fn is_allowed(&self, request: Request<Command>) -> trc::Result<Request<Command>> {\n        let state = &self.state;\n        // Rate limit request\n        if let State::Authenticated { data } | State::Selected { data, .. } = state\n            && let Some(rate) = &self.server.core.imap.rate_requests\n            && data\n                .server\n                .core\n                .storage\n                .lookup\n                .is_rate_allowed(\n                    KV_RATE_LIMIT_IMAP,\n                    &data.account_id.to_be_bytes(),\n                    rate,\n                    true,\n                )\n                .await?\n                .is_some()\n        {\n            return Err(trc::LimitEvent::TooManyRequests.into_err());\n        }\n\n        match &request.command {\n            Command::Capability | Command::Noop | Command::Logout | Command::Id => Ok(request),\n            Command::StartTls => {\n                if !self.is_tls {\n                    if self.instance.acceptor.is_tls() {\n                        Ok(request)\n                    } else {\n                        Err(trc::ImapEvent::Error\n                            .into_err()\n                            .details(\"TLS is not available.\")\n                            .id(request.tag))\n                    }\n                } else {\n                    Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Already in TLS mode.\")\n                        .id(request.tag))\n                }\n            }\n            Command::Authenticate => {\n                if let State::NotAuthenticated { .. } = state {\n                    Ok(request)\n                } else {\n                    Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Already authenticated.\")\n                        .id(request.tag))\n                }\n            }\n            Command::Login => {\n                if let State::NotAuthenticated { .. } = state {\n                    if self.is_tls || self.server.core.imap.allow_plain_auth {\n                        Ok(request)\n                    } else {\n                        Err(trc::ImapEvent::Error\n                            .into_err()\n                            .details(\"LOGIN is disabled on the clear-text port.\")\n                            .id(request.tag))\n                    }\n                } else {\n                    Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Already authenticated.\")\n                        .id(request.tag))\n                }\n            }\n            Command::Enable\n            | Command::Select\n            | Command::Examine\n            | Command::Create\n            | Command::Delete\n            | Command::Rename\n            | Command::Subscribe\n            | Command::Unsubscribe\n            | Command::List\n            | Command::Lsub\n            | Command::Namespace\n            | Command::Status\n            | Command::Append\n            | Command::Idle\n            | Command::SetAcl\n            | Command::DeleteAcl\n            | Command::GetAcl\n            | Command::ListRights\n            | Command::MyRights\n            | Command::Unauthenticate\n            | Command::GetQuota\n            | Command::GetQuotaRoot => {\n                if let State::Authenticated { .. } | State::Selected { .. } = state {\n                    Ok(request)\n                } else {\n                    Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Not authenticated.\")\n                        .id(request.tag))\n                }\n            }\n            Command::Close\n            | Command::Unselect\n            | Command::Expunge(_)\n            | Command::Search(_)\n            | Command::Fetch(_)\n            | Command::Store(_)\n            | Command::Copy(_)\n            | Command::Move(_)\n            | Command::Check\n            | Command::Sort(_)\n            | Command::Thread(_) => match state {\n                State::Selected { mailbox, .. } => {\n                    if mailbox.is_select\n                        || !matches!(\n                            request.command,\n                            Command::Store(_) | Command::Expunge(_) | Command::Move(_),\n                        )\n                    {\n                        Ok(request)\n                    } else {\n                        Err(trc::ImapEvent::Error\n                            .into_err()\n                            .details(\"Not permitted in EXAMINE state.\")\n                            .id(request.tag))\n                    }\n                }\n                State::Authenticated { .. } => Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"No mailbox is selected.\")\n                    .ctx(trc::Key::Type, ResponseType::Bad)\n                    .id(request.tag)),\n                State::NotAuthenticated { .. } => Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Not authenticated.\")\n                    .id(request.tag)),\n            },\n        }\n    }\n}\n\nimpl<T: SessionStream> State<T> {\n    pub fn auth_failures(&self) -> u32 {\n        match self {\n            State::NotAuthenticated { auth_failures, .. } => *auth_failures,\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn session_data(&self) -> Arc<SessionData<T>> {\n        match self {\n            State::Authenticated { data } => data.clone(),\n            State::Selected { data, .. } => data.clone(),\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn mailbox_state(&self) -> (Arc<SessionData<T>>, Arc<SelectedMailbox>) {\n        match self {\n            State::Selected { data, mailbox, .. } => (data.clone(), mailbox.clone()),\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn session_mailbox_state(&self) -> (Arc<SessionData<T>>, Option<Arc<SelectedMailbox>>) {\n        match self {\n            State::Authenticated { data } => (data.clone(), None),\n            State::Selected { data, mailbox, .. } => (data.clone(), mailbox.clone().into()),\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn select_data(&self) -> (Arc<SessionData<T>>, Arc<SelectedMailbox>) {\n        match self {\n            State::Selected { data, mailbox } => (data.clone(), mailbox.clone()),\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn spawn_task<F, R, P>(&self, params: P, fnc: F) -> trc::Result<()>\n    where\n        F: FnOnce(P, &super::SessionData<T>) -> R + Send + 'static,\n        P: Send + Sync + 'static,\n        R: std::future::Future<Output = trc::Result<()>> + Send + 'static,\n    {\n        let data = self.session_data();\n\n        tokio::spawn(async move {\n            if let Err(err) = fnc(params, &data).await {\n                let _ = data.write_error(err).await;\n            }\n        });\n\n        Ok(())\n    }\n\n    pub fn is_authenticated(&self) -> bool {\n        matches!(self, State::Authenticated { .. } | State::Selected { .. })\n    }\n\n    pub fn close_mailbox(&self) -> bool {\n        matches!(self, State::Selected { .. })\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/core/mailbox.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Account, MailboxId, MailboxSync, Session, SessionData};\nuse crate::core::Mailbox;\nuse ahash::AHashMap;\nuse common::{\n    auth::AccessToken,\n    listener::{SessionStream, limiter::InFlight},\n    sharing::EffectiveAcl,\n};\nuse directory::backend::internal::manage::ManageDirectory;\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess, mailbox::MailboxCacheAccess},\n    mailbox::INBOX_ID,\n};\nuse imap_proto::protocol::list::Attribute;\nuse parking_lot::Mutex;\nuse std::{\n    collections::BTreeMap,\n    sync::{Arc, atomic::Ordering},\n};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{acl::Acl, collection::Collection, id::Id, keyword::Keyword, special_use::SpecialUse};\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn new(\n        session: &Session<T>,\n        access_token: Arc<AccessToken>,\n        in_flight: Option<InFlight>,\n    ) -> trc::Result<Self> {\n        let mut session = SessionData {\n            stream_tx: session.stream_tx.clone(),\n            server: session.server.clone(),\n            account_id: access_token.primary_id(),\n            session_id: session.session_id,\n            mailboxes: Mutex::new(vec![]),\n            state: access_token.state().into(),\n            access_token,\n            in_flight,\n        };\n        let access_token = session.access_token.clone();\n\n        // Fetch mailboxes for the main account\n        let mut mailboxes = vec![\n            session\n                .fetch_account_mailboxes(session.account_id, None, &access_token, None)\n                .await\n                .caused_by(trc::location!())?\n                .unwrap(),\n        ];\n\n        // Fetch shared mailboxes\n        for &account_id in access_token.shared_accounts(Collection::Mailbox) {\n            let prefix: String = format!(\n                \"{}/{}\",\n                session.server.core.jmap.shared_folder,\n                session\n                    .server\n                    .store()\n                    .get_principal_name(account_id)\n                    .await\n                    .caused_by(trc::location!())?\n                    .unwrap_or_else(|| Id::from(account_id).to_string())\n            );\n            mailboxes.push(\n                session\n                    .fetch_account_mailboxes(account_id, prefix.into(), &access_token, None)\n                    .await\n                    .caused_by(trc::location!())?\n                    .unwrap(),\n            );\n        }\n\n        session.mailboxes = Mutex::new(mailboxes);\n\n        Ok(session)\n    }\n\n    async fn fetch_account_mailboxes(\n        &self,\n        account_id: u32,\n        mailbox_prefix: Option<String>,\n        access_token: &AccessToken,\n        current_state: Option<u64>,\n    ) -> trc::Result<Option<Account>> {\n        let cache = self\n            .server\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n        if current_state.is_some_and(|state| state == cache.last_change_id) {\n            return Ok(None);\n        }\n\n        let shared_mailbox_ids = if access_token.is_primary_id(account_id)\n            || access_token.member_of.contains(&account_id)\n        {\n            None\n        } else {\n            cache.shared_mailboxes(access_token, Acl::Read).into()\n        };\n\n        // Build special uses\n        let mut special_uses = AHashMap::new();\n        for mailbox in &cache.mailboxes.items {\n            if shared_mailbox_ids\n                .as_ref()\n                .is_none_or(|ids| ids.contains(mailbox.document_id))\n                && !matches!(mailbox.role, SpecialUse::None)\n            {\n                special_uses.insert(mailbox.role, mailbox.document_id);\n            }\n        }\n\n        // Build account\n        let mut account = Account {\n            account_id,\n            prefix: mailbox_prefix,\n            mailbox_names: BTreeMap::new(),\n            mailbox_state: AHashMap::with_capacity(cache.mailboxes.items.len()),\n            last_change_id: cache.last_change_id,\n        };\n\n        for mailbox in &cache.mailboxes.items {\n            if shared_mailbox_ids\n                .as_ref()\n                .is_some_and(|ids| !ids.contains(mailbox.document_id))\n            {\n                continue;\n            }\n\n            // Build mailbox path and map it to its effective id\n            let mailbox_name = if let Some(prefix) = &account.prefix {\n                let mut name = String::with_capacity(prefix.len() + mailbox.path.len() + 1);\n                name.push_str(prefix.as_str());\n                name.push('/');\n                name.push_str(mailbox.path.as_str());\n                name\n            } else {\n                mailbox.path.clone()\n            };\n            let effective_mailbox_id = self\n                .server\n                .core\n                .jmap\n                .default_folders\n                .iter()\n                .find(|f| f.name == mailbox_name || f.aliases.iter().any(|a| a == &mailbox_name))\n                .and_then(|f| special_uses.get(&f.special_use))\n                .copied()\n                .unwrap_or(mailbox.document_id);\n            account\n                .mailbox_names\n                .insert(mailbox_name, effective_mailbox_id);\n            account.mailbox_state.insert(\n                mailbox.document_id,\n                Mailbox {\n                    has_children: cache\n                        .mailboxes\n                        .items\n                        .iter()\n                        .any(|child| child.parent_id == mailbox.document_id),\n                    is_subscribed: mailbox.subscribers.contains(&access_token.primary_id()),\n                    special_use: match mailbox.role {\n                        SpecialUse::Trash => Some(Attribute::Trash),\n                        SpecialUse::Junk => Some(Attribute::Junk),\n                        SpecialUse::Drafts => Some(Attribute::Drafts),\n                        SpecialUse::Archive => Some(Attribute::Archive),\n                        SpecialUse::Sent => Some(Attribute::Sent),\n                        SpecialUse::Important => Some(Attribute::Important),\n                        SpecialUse::Memos => Some(Attribute::Memos),\n                        SpecialUse::Scheduled => Some(Attribute::Scheduled),\n                        SpecialUse::Snoozed => Some(Attribute::Snoozed),\n                        _ => None,\n                    },\n                    total_messages: cache.in_mailbox(mailbox.document_id).count() as u64,\n                    total_unseen: cache\n                        .in_mailbox_without_keyword(mailbox.document_id, &Keyword::Seen)\n                        .count() as u64,\n                    total_deleted: cache\n                        .in_mailbox_with_keyword(mailbox.document_id, &Keyword::Deleted)\n                        .count() as u64,\n                    uid_validity: mailbox.uid_validity as u64,\n                    uid_next: self\n                        .get_uid_next(&MailboxId {\n                            account_id,\n                            mailbox_id: mailbox.document_id,\n                        })\n                        .await\n                        .caused_by(trc::location!())? as u64,\n                    total_deleted_storage: None,\n                    size: None,\n                },\n            );\n        }\n\n        Ok(account.into())\n    }\n\n    pub async fn synchronize_mailboxes(\n        &self,\n        return_changes: bool,\n    ) -> trc::Result<Option<MailboxSync>> {\n        let mut changes = if return_changes {\n            MailboxSync::default().into()\n        } else {\n            None\n        };\n\n        // Obtain access token\n        let access_token = self\n            .server\n            .get_access_token(self.account_id)\n            .await\n            .caused_by(trc::location!())?;\n        let state = access_token.state();\n\n        // Shared mailboxes might have changed\n        let mut added_accounts = Vec::new();\n        if self.state.load(Ordering::Relaxed) != state {\n            // Remove unlinked shared accounts\n            let mut added_account_ids = Vec::new();\n            {\n                let mut mailboxes = self.mailboxes.lock();\n                let mut new_accounts = Vec::with_capacity(mailboxes.len());\n                let has_access_to = access_token\n                    .shared_accounts(Collection::Mailbox)\n                    .copied()\n                    .collect::<Vec<_>>();\n                for account in mailboxes.drain(..) {\n                    if access_token.is_primary_id(account.account_id)\n                        || has_access_to.contains(&account.account_id)\n                    {\n                        new_accounts.push(account);\n                    } else {\n                        // Add unshared mailboxes to deleted list\n                        if let Some(changes) = &mut changes {\n                            for (mailbox_name, _) in account.mailbox_names {\n                                changes.deleted.push(mailbox_name);\n                            }\n                        }\n                    }\n                }\n\n                // Add new shared account ids\n                for account_id in has_access_to {\n                    if !new_accounts\n                        .iter()\n                        .skip(1)\n                        .any(|m| m.account_id == account_id)\n                    {\n                        added_account_ids.push(account_id);\n                    }\n                }\n                *mailboxes = new_accounts;\n            }\n\n            // Fetch mailboxes for each new shared account\n            for account_id in added_account_ids {\n                let prefix: String = format!(\n                    \"{}/{}\",\n                    self.server.core.jmap.shared_folder,\n                    self.server\n                        .store()\n                        .get_principal_name(account_id)\n                        .await\n                        .caused_by(trc::location!())?\n                        .unwrap_or_else(|| Id::from(account_id).to_string())\n                );\n                added_accounts.push(\n                    self.fetch_account_mailboxes(account_id, prefix.into(), &access_token, None)\n                        .await?\n                        .unwrap(),\n                );\n            }\n\n            // Update state\n            self.state.store(state, Ordering::Relaxed);\n        }\n\n        // Fetch mailbox changes for all accounts\n        let mut changed_accounts = Vec::new();\n        let account_states = self\n            .mailboxes\n            .lock()\n            .iter()\n            .map(|m| (m.account_id, m.prefix.clone(), m.last_change_id))\n            .collect::<Vec<_>>();\n        for (account_id, prefix, last_state) in account_states {\n            if let Some(changed_account) = self\n                .fetch_account_mailboxes(account_id, prefix, &access_token, last_state.into())\n                .await\n                .caused_by(trc::location!())?\n            {\n                changed_accounts.push(changed_account);\n            }\n        }\n\n        // Update mailboxes\n        if !changed_accounts.is_empty() || !added_accounts.is_empty() {\n            let mut mailboxes = self.mailboxes.lock();\n\n            for changed_account in changed_accounts {\n                if let Some(pos) = mailboxes\n                    .iter()\n                    .position(|a| a.account_id == changed_account.account_id)\n                {\n                    // Add changes and deletions\n                    if let Some(changes) = &mut changes {\n                        let old_account = &mailboxes[pos];\n                        let new_account = &changed_account;\n\n                        // Add new mailboxes\n                        for (mailbox_name, mailbox_id) in new_account.mailbox_names.iter() {\n                            if let Some(old_mailbox) = old_account.mailbox_state.get(mailbox_id) {\n                                if let Some(mailbox) = new_account.mailbox_state.get(mailbox_id)\n                                    && (mailbox.total_messages != old_mailbox.total_messages\n                                        || mailbox.total_unseen != old_mailbox.total_unseen)\n                                {\n                                    changes.changed.push(mailbox_name.clone());\n                                }\n                            } else {\n                                changes.added.push(mailbox_name.clone());\n                            }\n                        }\n\n                        // Add deleted mailboxes\n                        for (mailbox_name, mailbox_id) in &old_account.mailbox_names {\n                            if !new_account.mailbox_state.contains_key(mailbox_id) {\n                                changes.deleted.push(mailbox_name.clone());\n                            }\n                        }\n                    }\n\n                    mailboxes[pos] = changed_account;\n                } else {\n                    // Add newly shared accounts\n                    if let Some(changes) = &mut changes {\n                        changes\n                            .added\n                            .extend(changed_account.mailbox_names.keys().cloned());\n                    }\n\n                    mailboxes.push(changed_account);\n                }\n            }\n\n            if !added_accounts.is_empty() {\n                // Add newly shared accounts\n                if let Some(changes) = &mut changes {\n                    for added_account in &added_accounts {\n                        changes\n                            .added\n                            .extend(added_account.mailbox_names.keys().cloned());\n                    }\n                }\n                mailboxes.extend(added_accounts);\n            }\n        }\n\n        Ok(changes)\n    }\n\n    pub fn get_mailbox_by_name(&self, mailbox_name: &str) -> Option<MailboxId> {\n        let is_inbox = mailbox_name.eq_ignore_ascii_case(\"inbox\");\n        for account in self.mailboxes.lock().iter() {\n            if account\n                .prefix\n                .as_ref()\n                .is_none_or(|p| mailbox_name.starts_with(p.as_str()))\n            {\n                for (mailbox_name_, mailbox_id_) in account.mailbox_names.iter() {\n                    if (!is_inbox && mailbox_name_ == mailbox_name)\n                        || (is_inbox && *mailbox_id_ == INBOX_ID)\n                    {\n                        return MailboxId {\n                            account_id: account.account_id,\n                            mailbox_id: *mailbox_id_,\n                        }\n                        .into();\n                    }\n                }\n            }\n        }\n        None\n    }\n\n    pub async fn check_mailbox_acl(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        item: Acl,\n    ) -> trc::Result<bool> {\n        let access_token = self.get_access_token().await?;\n        Ok(access_token.is_member(account_id)\n            || self\n                .server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Mailbox,\n                    document_id,\n                ))\n                .await\n                .and_then(|mailbox| {\n                    if let Some(mailbox) = mailbox {\n                        Ok(Some(\n                            mailbox\n                                .unarchive::<email::mailbox::Mailbox>()?\n                                .acls\n                                .effective_acl(&access_token)\n                                .contains(item),\n                        ))\n                    } else {\n                        Ok(None)\n                    }\n                })?\n                .ok_or_else(|| {\n                    trc::ImapEvent::Error\n                        .caused_by(trc::location!())\n                        .details(\"Mailbox no longer exists.\")\n                })?)\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/core/message.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ImapUidToId, Mailbox, MailboxId, MailboxState, NextMailboxState, SelectedMailbox, SessionData,\n};\nuse crate::core::ImapId;\nuse ahash::AHashMap;\nuse common::listener::SessionStream;\nuse email::cache::MessageCacheFetch;\nuse imap_proto::protocol::{Sequence, expunge, select::Exists};\nuse std::collections::BTreeMap;\nuse store::{ValueKey, write::ValueClass};\nuse trc::AddContext;\nuse types::{collection::Collection, field::MailboxField};\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn fetch_messages(\n        &self,\n        mailbox: &MailboxId,\n        current_state: Option<u64>,\n    ) -> trc::Result<Option<MailboxState>> {\n        let cached_messages = self\n            .server\n            .get_cached_messages(mailbox.account_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        if current_state.is_some_and(|state| state == cached_messages.emails.change_id) {\n            return Ok(None);\n        }\n\n        // Obtain UID next and assign UIDs\n        let uid_map = cached_messages\n            .emails\n            .items\n            .iter()\n            .filter_map(|item| {\n                item.mailboxes.iter().find_map(|m| {\n                    if m.mailbox_id == mailbox.mailbox_id {\n                        Some((m.uid, item.document_id))\n                    } else {\n                        None\n                    }\n                })\n            })\n            .collect::<BTreeMap<u32, u32>>();\n        let mut uid_max = 0;\n        let mut id_to_imap = AHashMap::with_capacity(uid_map.len());\n        let mut uid_to_id = AHashMap::with_capacity(uid_map.len());\n\n        for (seqnum, (uid, message_id)) in uid_map.into_iter().enumerate() {\n            if uid > uid_max {\n                uid_max = uid;\n            }\n            id_to_imap.insert(\n                message_id,\n                ImapId {\n                    uid,\n                    seqnum: seqnum as u32 + 1,\n                },\n            );\n            uid_to_id.insert(uid, message_id);\n        }\n\n        Ok(Some(MailboxState {\n            total_messages: id_to_imap.len(),\n            id_to_imap,\n            uid_to_id,\n            uid_max,\n            modseq: cached_messages.emails.change_id,\n            next_state: None,\n        }))\n    }\n\n    pub async fn synchronize_messages(&self, mailbox: &SelectedMailbox) -> trc::Result<u64> {\n        // Obtain current modseq\n        let mut current_modseq = mailbox.state.lock().modseq;\n        if let Some(new_state) = self\n            .fetch_messages(&mailbox.id, current_modseq.into())\n            .await?\n        {\n            // Synchronize messages\n            let mut current_state = mailbox.state.lock();\n            current_modseq = new_state.modseq;\n\n            // Add missing uids\n            let mut deletions = current_state\n                .next_state\n                .take()\n                .map(|state| state.deletions)\n                .unwrap_or_default();\n            let mut id_to_imap = AHashMap::with_capacity(current_state.id_to_imap.len());\n            for (id, imap_id) in std::mem::take(&mut current_state.id_to_imap) {\n                if !new_state.uid_to_id.contains_key(&imap_id.uid) {\n                    // Add to deletions\n                    deletions.push(imap_id);\n\n                    // Invalidate entries\n                    current_state.uid_to_id.remove(&imap_id.uid);\n                } else {\n                    id_to_imap.insert(id, imap_id);\n                }\n            }\n            current_state.id_to_imap = id_to_imap;\n\n            // Update state\n            current_state.modseq = new_state.modseq;\n            current_state.next_state = Some(Box::new(NextMailboxState {\n                next_state: new_state,\n                deletions,\n            }));\n        }\n\n        Ok(current_modseq)\n    }\n\n    pub async fn write_mailbox_changes(\n        &self,\n        mailbox: &SelectedMailbox,\n        is_qresync: bool,\n    ) -> trc::Result<u64> {\n        // Resync mailbox\n        let modseq = self.synchronize_messages(mailbox).await?;\n        let mut buf = Vec::new();\n        {\n            let mut current_state = mailbox.state.lock();\n            if let Some(next_state) = current_state.next_state.take() {\n                if !next_state.deletions.is_empty() {\n                    let mut ids = next_state\n                        .deletions\n                        .into_iter()\n                        .map(|id| if is_qresync { id.uid } else { id.seqnum })\n                        .collect::<Vec<u32>>();\n                    ids.sort_unstable();\n                    expunge::Response { is_qresync, ids }.serialize_to(&mut buf);\n                }\n                if !buf.is_empty()\n                    || next_state\n                        .next_state\n                        .uid_max\n                        .saturating_sub(current_state.uid_max)\n                        > 0\n                {\n                    Exists {\n                        total_messages: next_state.next_state.total_messages,\n                    }\n                    .serialize(&mut buf);\n                }\n                *current_state = next_state.next_state;\n            }\n        }\n        if !buf.is_empty() {\n            self.write_bytes(buf).await?;\n        }\n\n        Ok(modseq)\n    }\n\n    pub async fn get_uid_next(&self, mailbox: &MailboxId) -> trc::Result<u32> {\n        self.server\n            .core\n            .storage\n            .data\n            .get_counter(ValueKey {\n                account_id: mailbox.account_id,\n                collection: Collection::Mailbox.into(),\n                document_id: mailbox.mailbox_id,\n                class: ValueClass::Property(MailboxField::UidCounter.into()),\n            })\n            .await\n            .map(|v| (v + 1) as u32)\n    }\n\n    pub fn mailbox_state(&self, mailbox: &MailboxId) -> Option<Mailbox> {\n        self.mailboxes\n            .lock()\n            .iter()\n            .find(|m| m.account_id == mailbox.account_id)\n            .and_then(|m| m.mailbox_state.get(&mailbox.mailbox_id))\n            .cloned()\n    }\n}\n\nimpl SelectedMailbox {\n    pub async fn sequence_to_ids(\n        &self,\n        sequence: &Sequence,\n        is_uid: bool,\n    ) -> trc::Result<AHashMap<u32, ImapId>> {\n        if !sequence.is_saved_search() {\n            let mut ids = AHashMap::new();\n            let state = self.state.lock();\n\n            if is_uid {\n                let id_to_imap = state\n                    .next_state\n                    .as_ref()\n                    .map(|s| &s.next_state.id_to_imap)\n                    .unwrap_or(&state.id_to_imap);\n                if !state.id_to_imap.is_empty() {\n                    for (id, imap_id) in id_to_imap {\n                        if sequence.contains(imap_id.uid, state.uid_max) {\n                            ids.insert(*id, *imap_id);\n                        }\n                    }\n                }\n            } else if !state.id_to_imap.is_empty() {\n                for (id, imap_id) in &state.id_to_imap {\n                    if sequence.contains(imap_id.seqnum, state.total_messages as u32) {\n                        ids.insert(*id, *imap_id);\n                    }\n                }\n            }\n\n            Ok(ids)\n        } else {\n            let saved_ids = self.get_saved_search().await.ok_or_else(|| {\n                trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"No saved search found.\")\n            })?;\n            let mut ids = AHashMap::with_capacity(saved_ids.len());\n            let state = self.state.lock();\n\n            for imap_id in saved_ids.iter() {\n                if let Some(id) = state.uid_to_id.get(&imap_id.uid) {\n                    ids.insert(*id, *imap_id);\n                }\n            }\n\n            Ok(ids)\n        }\n    }\n\n    pub async fn sequence_expand_missing(&self, sequence: &Sequence, is_uid: bool) -> Vec<u32> {\n        let mut deleted_ids = Vec::new();\n        if !sequence.is_saved_search() {\n            let state = self.state.lock();\n            if is_uid {\n                for uid in sequence.expand(state.uid_max) {\n                    if !state.uid_to_id.contains_key(&uid) {\n                        deleted_ids.push(uid);\n                    }\n                }\n            } else {\n                for seqnum in sequence.expand(state.total_messages as u32) {\n                    if seqnum > state.total_messages as u32 {\n                        deleted_ids.push(seqnum);\n                    }\n                }\n            }\n        } else if let Some(saved_ids) = self.get_saved_search().await {\n            let state = self.state.lock();\n            for id in saved_ids.iter() {\n                if !state.uid_to_id.contains_key(&id.uid) {\n                    deleted_ids.push(if is_uid { id.uid } else { id.seqnum });\n                }\n            }\n        }\n        deleted_ids.sort_unstable();\n        deleted_ids\n    }\n\n    pub fn append_messages(&self, ids: Vec<ImapUidToId>, modseq: Option<u64>) {\n        let mut mailbox = self.state.lock();\n        if modseq.unwrap_or(0) > mailbox.modseq {\n            let mut uid_max = 0;\n            for id in ids {\n                mailbox.total_messages += 1;\n                let seqnum = mailbox.total_messages as u32;\n                mailbox.uid_to_id.insert(id.uid, id.uid);\n                mailbox.id_to_imap.insert(\n                    id.id,\n                    ImapId {\n                        uid: id.uid,\n                        seqnum,\n                    },\n                );\n                uid_max = id.uid;\n            }\n            mailbox.uid_max = uid_max;\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/core/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    collections::BTreeMap,\n    net::IpAddr,\n    sync::{Arc, atomic::AtomicU32},\n};\n\nuse ahash::AHashMap;\nuse common::{\n    Inner, Server,\n    auth::AccessToken,\n    listener::{ServerInstance, SessionStream, limiter::InFlight},\n};\n\nuse imap_proto::{\n    Command,\n    protocol::{ProtocolVersion, list::Attribute},\n    receiver::Receiver,\n};\nuse tokio::{\n    io::{ReadHalf, WriteHalf},\n    sync::watch,\n};\nuse trc::AddContext;\n\npub mod client;\npub mod mailbox;\npub mod message;\npub mod session;\n\n#[derive(Clone)]\npub struct ImapSessionManager {\n    pub inner: Arc<Inner>,\n}\n\nimpl ImapSessionManager {\n    pub fn new(inner: Arc<Inner>) -> Self {\n        Self { inner }\n    }\n}\n\npub struct Session<T: SessionStream> {\n    pub server: Server,\n    pub instance: Arc<ServerInstance>,\n    pub receiver: Receiver<Command>,\n    pub version: ProtocolVersion,\n    pub state: State<T>,\n    pub is_tls: bool,\n    pub is_condstore: bool,\n    pub is_qresync: bool,\n    pub is_utf8: bool,\n    pub stream_rx: ReadHalf<T>,\n    pub stream_tx: Arc<tokio::sync::Mutex<WriteHalf<T>>>,\n    pub in_flight: InFlight,\n    pub remote_addr: IpAddr,\n    pub session_id: u64,\n}\n\npub struct SessionData<T: SessionStream> {\n    pub account_id: u32,\n    pub access_token: Arc<AccessToken>,\n    pub server: Server,\n    pub session_id: u64,\n    pub mailboxes: parking_lot::Mutex<Vec<Account>>,\n    pub stream_tx: Arc<tokio::sync::Mutex<WriteHalf<T>>>,\n    pub state: AtomicU32,\n    pub in_flight: Option<InFlight>,\n}\n\npub struct SelectedMailbox {\n    pub id: MailboxId,\n    pub state: parking_lot::Mutex<MailboxState>,\n    pub saved_search: parking_lot::Mutex<SavedSearch>,\n    pub is_select: bool,\n    pub is_condstore: bool,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Clone, Copy)]\npub struct AccountId {\n    pub account_id: u32,\n    pub primary_id: u32,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Clone, Copy)]\npub struct MailboxId {\n    pub account_id: u32,\n    pub mailbox_id: u32,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct Account {\n    pub account_id: u32,\n    pub prefix: Option<String>,\n    pub mailbox_names: BTreeMap<String, u32>,\n    pub mailbox_state: AHashMap<u32, Mailbox>,\n    pub last_change_id: u64,\n}\n\n#[derive(Debug, Default, Clone)]\npub struct Mailbox {\n    pub has_children: bool,\n    pub is_subscribed: bool,\n    pub special_use: Option<Attribute>,\n    pub total_messages: u64,\n    pub total_unseen: u64,\n    pub total_deleted: u64,\n    pub total_deleted_storage: Option<u64>,\n    pub uid_validity: u64,\n    pub uid_next: u64,\n    pub size: Option<u64>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct MailboxState {\n    pub uid_max: u32,\n    pub id_to_imap: AHashMap<u32, ImapId>,\n    pub uid_to_id: AHashMap<u32, u32>,\n    pub total_messages: usize,\n    pub modseq: u64,\n    pub next_state: Option<Box<NextMailboxState>>,\n}\n\n#[derive(Debug, Clone)]\npub struct NextMailboxState {\n    pub next_state: MailboxState,\n    pub deletions: Vec<ImapId>,\n}\n\n#[derive(Debug, Clone, Copy, Default)]\npub struct ImapId {\n    pub uid: u32,\n    pub seqnum: u32,\n}\n\n#[derive(Debug, Default)]\npub struct MailboxSync {\n    pub added: Vec<String>,\n    pub changed: Vec<String>,\n    pub deleted: Vec<String>,\n}\n\npub enum SavedSearch {\n    InFlight {\n        rx: watch::Receiver<Arc<Vec<ImapId>>>,\n    },\n    Results {\n        items: Arc<Vec<ImapId>>,\n    },\n    None,\n}\n\n#[derive(Debug, Clone, Copy, Default)]\npub struct ImapUidToId {\n    pub uid: u32,\n    pub id: u32,\n}\n\npub enum State<T: SessionStream> {\n    NotAuthenticated {\n        auth_failures: u32,\n    },\n    Authenticated {\n        data: Arc<SessionData<T>>,\n    },\n    Selected {\n        data: Arc<SessionData<T>>,\n        mailbox: Arc<SelectedMailbox>,\n    },\n}\n\nimpl<T: SessionStream> State<T> {\n    pub fn try_replace_stream_tx<U: SessionStream>(\n        self,\n        new_stream: Arc<tokio::sync::Mutex<WriteHalf<U>>>,\n    ) -> Option<State<U>> {\n        match self {\n            State::NotAuthenticated { auth_failures } => {\n                State::NotAuthenticated { auth_failures }.into()\n            }\n            State::Authenticated { data } => {\n                Arc::try_unwrap(data).ok().map(|data| State::Authenticated {\n                    data: Arc::new(data.replace_stream_tx(new_stream)),\n                })\n            }\n            State::Selected { data, mailbox } => {\n                Arc::try_unwrap(data).ok().map(|data| State::Selected {\n                    data: Arc::new(data.replace_stream_tx(new_stream)),\n                    mailbox,\n                })\n            }\n        }\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn get_access_token(&self) -> trc::Result<Arc<AccessToken>> {\n        self.server\n            .get_access_token(self.account_id)\n            .await\n            .caused_by(trc::location!())\n    }\n\n    pub fn replace_stream_tx<U: SessionStream>(\n        self,\n        new_stream: Arc<tokio::sync::Mutex<WriteHalf<U>>>,\n    ) -> SessionData<U> {\n        SessionData {\n            account_id: self.account_id,\n            server: self.server,\n            session_id: self.session_id,\n            mailboxes: self.mailboxes,\n            stream_tx: new_stream,\n            state: self.state,\n            in_flight: self.in_flight,\n            access_token: self.access_token,\n        }\n    }\n}\n\nimpl MailboxState {\n    pub fn map_result_id(&self, document_id: u32, is_uid: bool) -> Option<(u32, ImapId)> {\n        if let Some(imap_id) = self.id_to_imap.get(&document_id) {\n            Some((if is_uid { imap_id.uid } else { imap_id.seqnum }, *imap_id))\n        } else if is_uid {\n            self.next_state.as_ref().and_then(|s| {\n                s.next_state\n                    .id_to_imap\n                    .get(&document_id)\n                    .map(|imap_id| (imap_id.uid, *imap_id))\n            })\n        } else {\n            None\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/core/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse common::{\n    core::BuildServer,\n    listener::{SessionData, SessionManager, SessionResult, SessionStream, stream::NullIo},\n};\nuse imap_proto::{\n    protocol::{ProtocolVersion, SerializeResponse},\n    receiver::Receiver,\n};\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio_rustls::server::TlsStream;\n\nuse crate::{GREETING_WITH_TLS, GREETING_WITHOUT_TLS};\n\nuse super::{ImapSessionManager, Session, State};\n\nimpl SessionManager for ImapSessionManager {\n    #[allow(clippy::manual_async_fn)]\n    fn handle<T: SessionStream>(\n        self,\n        session: SessionData<T>,\n    ) -> impl std::future::Future<Output = ()> + Send {\n        async move {\n            if let Ok(mut session) = Session::new(session, self).await\n                && session.handle_conn().await\n                && session.instance.acceptor.is_tls()\n                && let Ok(mut session) = session.into_tls().await\n            {\n                session.handle_conn().await;\n            }\n        }\n    }\n\n    #[allow(clippy::manual_async_fn)]\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send {\n        async {}\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_conn(&mut self) -> bool {\n        let mut buf = vec![0; 8192];\n        let mut shutdown_rx = self.instance.shutdown_rx.clone();\n\n        loop {\n            tokio::select! {\n                result = tokio::time::timeout(\n                    if !matches!(self.state, State::NotAuthenticated {..}) {\n                        self.server.core.imap.timeout_auth\n                    } else {\n                        self.server.core.imap.timeout_unauth\n                    },\n                    self.stream_rx.read(&mut buf)) => {\n                    match result {\n                        Ok(Ok(bytes_read)) => {\n                            if bytes_read > 0 {\n                                match self.ingest(&buf[..bytes_read]).await {\n                                    SessionResult::Continue => (),\n                                    SessionResult::UpgradeTls => {\n                                        return true;\n                                    }\n                                    SessionResult::Close => {\n                                        break;\n                                    }\n                                }\n                            } else {\n                                trc::event!(\n                                    Network(trc::NetworkEvent::Closed),\n                                    SpanId = self.session_id,\n                                    CausedBy = trc::location!()\n                                );\n                                break;\n                            }\n                        },\n                        Ok(Err(err)) => {\n                            trc::event!(\n                                Network(trc::NetworkEvent::ReadError),\n                                SpanId = self.session_id,\n                                Reason = err.to_string(),\n                                CausedBy = trc::location!()\n                            );\n                            break;\n                        },\n                        Err(_) => {\n                            trc::event!(\n                                Network(trc::NetworkEvent::Timeout),\n                                SpanId = self.session_id,\n                                CausedBy = trc::location!()\n                            );\n                            self.write_bytes(&b\"* BYE Connection timed out.\\r\\n\"[..]).await.ok();\n                            break;\n                        }\n                    }\n                },\n                _ = shutdown_rx.changed() => {\n                    trc::event!(\n                        Network(trc::NetworkEvent::Closed),\n                        SpanId = self.session_id,\n                        Reason = \"Server shutting down\",\n                        CausedBy = trc::location!()\n                    );\n                    self.write_bytes(&b\"* BYE Server shutting down.\\r\\n\"[..]).await.ok();\n                    break;\n                }\n            };\n        }\n\n        false\n    }\n\n    pub async fn new(\n        mut session: SessionData<T>,\n        manager: ImapSessionManager,\n    ) -> Result<Session<T>, ()> {\n        // Write greeting\n        let is_tls = session.stream.is_tls();\n        let greeting = if !is_tls && session.instance.acceptor.is_tls() {\n            &GREETING_WITH_TLS\n        } else {\n            &GREETING_WITHOUT_TLS\n        };\n\n        if let Err(err) = session.stream.write_all(greeting).await {\n            trc::event!(\n                Network(trc::NetworkEvent::WriteError),\n                Reason = err.to_string(),\n                SpanId = session.session_id,\n                Details = \"Failed to write to stream\"\n            );\n            return Err(());\n        }\n        let _ = session.stream.flush().await;\n\n        // Split stream into read and write halves\n        let (stream_rx, stream_tx) = tokio::io::split(session.stream);\n        let server = manager.inner.build_server();\n\n        Ok(Session {\n            receiver: Receiver::with_max_request_size(server.core.imap.max_request_size),\n            version: ProtocolVersion::Rev1,\n            state: State::NotAuthenticated { auth_failures: 0 },\n            is_tls,\n            is_condstore: false,\n            is_qresync: false,\n            is_utf8: false,\n            server,\n            instance: session.instance,\n            session_id: session.session_id,\n            in_flight: session.in_flight,\n            remote_addr: session.remote_ip,\n            stream_rx,\n            stream_tx: Arc::new(tokio::sync::Mutex::new(stream_tx)),\n        })\n    }\n\n    pub async fn into_tls(self) -> Result<Session<TlsStream<T>>, ()> {\n        // Drop references to write half from state\n        let state = if let Some(state) =\n            self.state\n                .try_replace_stream_tx(Arc::new(tokio::sync::Mutex::new(\n                    tokio::io::split(NullIo::default()).1,\n                ))) {\n            state\n        } else {\n            trc::event!(\n                Network(trc::NetworkEvent::SplitError),\n                SpanId = self.session_id,\n                Details = \"Failed to obtain write half state\"\n            );\n            return Err(());\n        };\n\n        // Take ownership of WriteHalf and unsplit it from ReadHalf\n        let stream = if let Ok(stream_tx) =\n            Arc::try_unwrap(self.stream_tx).map(|mutex| mutex.into_inner())\n        {\n            self.stream_rx.unsplit(stream_tx)\n        } else {\n            trc::event!(\n                Network(trc::NetworkEvent::SplitError),\n                SpanId = self.session_id,\n                Details = \"Failed to take ownership of write half\"\n            );\n\n            return Err(());\n        };\n\n        // Upgrade to TLS\n        let (stream_rx, stream_tx) =\n            tokio::io::split(self.instance.tls_accept(stream, self.session_id).await?);\n        let stream_tx = Arc::new(tokio::sync::Mutex::new(stream_tx));\n\n        Ok(Session {\n            server: self.server,\n            instance: self.instance,\n            receiver: self.receiver,\n            version: self.version,\n            state: state.try_replace_stream_tx(stream_tx.clone()).unwrap(),\n            is_tls: true,\n            is_condstore: self.is_condstore,\n            is_qresync: self.is_qresync,\n            is_utf8: self.is_utf8,\n            session_id: self.session_id,\n            in_flight: self.in_flight,\n            remote_addr: self.remote_addr,\n            stream_rx,\n            stream_tx,\n        })\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn write_bytes(&self, bytes: impl AsRef<[u8]>) -> trc::Result<()> {\n        let bytes = bytes.as_ref();\n\n        trc::event!(\n            Imap(trc::ImapEvent::RawOutput),\n            SpanId = self.session_id,\n            Size = bytes.len(),\n            Contents = trc::Value::from_maybe_string(bytes),\n        );\n\n        let mut stream = self.stream_tx.lock().await;\n        if let Err(err) = stream.write_all(bytes).await {\n            Err(trc::NetworkEvent::WriteError\n                .into_err()\n                .reason(err)\n                .details(\"Failed to write to stream\"))\n        } else {\n            let _ = stream.flush().await;\n            Ok(())\n        }\n    }\n\n    pub async fn write_error(&self, err: trc::Error) -> bool {\n        if err.should_write_err() {\n            let disconnect = err.must_disconnect();\n            let bytes = err.serialize();\n            trc::error!(err.span_id(self.session_id));\n\n            if let Err(err) = self.write_bytes(bytes).await {\n                trc::error!(err.span_id(self.session_id));\n                false\n            } else {\n                !disconnect\n            }\n        } else {\n            trc::error!(err);\n\n            false\n        }\n    }\n}\n\nimpl<T: SessionStream> super::SessionData<T> {\n    pub async fn write_bytes(&self, bytes: impl AsRef<[u8]>) -> trc::Result<()> {\n        let bytes = bytes.as_ref();\n\n        trc::event!(\n            Imap(trc::ImapEvent::RawOutput),\n            SpanId = self.session_id,\n            Size = bytes.len(),\n            Contents = trc::Value::from_maybe_string(bytes),\n        );\n\n        let mut stream = self.stream_tx.lock().await;\n        if let Err(err) = stream.write_all(bytes.as_ref()).await {\n            Err(trc::NetworkEvent::WriteError\n                .into_err()\n                .reason(err)\n                .details(\"Failed to write to stream\"))\n        } else {\n            let _ = stream.flush().await;\n            Ok(())\n        }\n    }\n\n    pub async fn write_error(&self, err: trc::Error) -> trc::Result<()> {\n        if err.should_write_err() {\n            let bytes = err.serialize();\n            trc::error!(err.span_id(self.session_id));\n            self.write_bytes(bytes).await\n        } else {\n            trc::error!(err.span_id(self.session_id));\n            Ok(())\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::LazyLock;\n\nuse imap_proto::{ResponseCode, StatusResponse, protocol::capability::Capability};\n\npub mod core;\npub mod op;\n\nstatic SERVER_GREETING: &str = \"Stalwart IMAP4rev2 at your service.\";\n\npub(crate) static GREETING_WITH_TLS: LazyLock<Vec<u8>> = LazyLock::new(|| {\n    StatusResponse::ok(SERVER_GREETING)\n        .with_code(ResponseCode::Capability {\n            capabilities: Capability::all_capabilities(false, true),\n        })\n        .into_bytes()\n});\n\npub(crate) static GREETING_WITHOUT_TLS: LazyLock<Vec<u8>> = LazyLock::new(|| {\n    StatusResponse::ok(SERVER_GREETING)\n        .with_code(ResponseCode::Capability {\n            capabilities: Capability::all_capabilities(false, false),\n        })\n        .into_bytes()\n});\n\npub struct ImapError;\n"
  },
  {
    "path": "crates/imap/src/op/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{MailboxId, Session, SessionData, State},\n    op::ImapContext,\n    spawn_op,\n};\nuse common::{\n    auth::AccessToken, listener::SessionStream, sharing::EffectiveAcl,\n    storage::index::ObjectIndexBuilder,\n};\nuse compact_str::ToCompactString;\nuse directory::{\n    Permission, QueryParams, Type,\n    backend::internal::{\n        PrincipalField,\n        manage::{ChangedPrincipals, ManageDirectory},\n    },\n};\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse,\n    protocol::acl::{\n        Arguments, GetAclResponse, ListRightsResponse, ModRightsOp, MyRightsResponse, Rights,\n    },\n    receiver::Request,\n};\nuse std::{sync::Arc, time::Instant};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    acl::{Acl, AclGrant},\n    collection::Collection,\n};\nuse utils::map::bitmap::Bitmap;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_get_acl(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapAclGet)?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_acl(self.is_utf8)?;\n        let is_utf8 = self.version.is_rev2() || self.is_utf8;\n        let data = self.state.session_data();\n\n        spawn_op!(data, {\n            let (mailbox_id, mailbox_, _) = data\n                .get_acl_mailbox(&arguments, true)\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n            let mut permissions = Vec::new();\n            let mailbox = mailbox_\n                .to_unarchived::<email::mailbox::Mailbox>()\n                .imap_ctx(&arguments.tag, trc::location!())?;\n\n            // Add the current user if they are the owner or a group member\n            if data.access_token.is_member(mailbox_id.account_id) {\n                permissions.push((\n                    data.access_token.name.clone(),\n                    vec![\n                        Rights::Read,\n                        Rights::Lookup,\n                        Rights::Insert,\n                        Rights::DeleteMessages,\n                        Rights::Expunge,\n                        Rights::Seen,\n                        Rights::Write,\n                        Rights::CreateMailbox,\n                        Rights::DeleteMailbox,\n                        Rights::Post,\n                        Rights::Administer,\n                    ],\n                ));\n            }\n\n            for item in mailbox.inner.acls.iter() {\n                if item.account_id == mailbox_id.account_id {\n                    // Skip the current user, as they are already added above\n                    continue;\n                }\n\n                if let Some(account_name) = data\n                    .server\n                    .store()\n                    .get_principal_name(item.account_id.into())\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?\n                {\n                    let mut rights = Vec::new();\n\n                    for acl in Bitmap::from(&item.grants) {\n                        match acl {\n                            Acl::Read => {\n                                rights.push(Rights::Lookup);\n                            }\n                            Acl::Modify => {\n                                rights.push(Rights::CreateMailbox);\n                            }\n                            Acl::Delete => {\n                                rights.push(Rights::DeleteMailbox);\n                            }\n                            Acl::ReadItems => {\n                                rights.push(Rights::Read);\n                            }\n                            Acl::AddItems => {\n                                rights.push(Rights::Insert);\n                            }\n                            Acl::ModifyItems => {\n                                rights.push(Rights::Write);\n                                rights.push(Rights::Seen);\n                            }\n                            Acl::RemoveItems => {\n                                rights.push(Rights::DeleteMessages);\n                                rights.push(Rights::Expunge);\n                            }\n                            Acl::CreateChild => {\n                                rights.push(Rights::CreateMailbox);\n                            }\n                            Acl::Share => {\n                                rights.push(Rights::Administer);\n                            }\n                            Acl::Submit => {\n                                rights.push(Rights::Post);\n                            }\n                            _ => (),\n                        }\n                    }\n\n                    permissions.push((account_name, rights));\n                }\n            }\n\n            trc::event!(\n                Imap(trc::ImapEvent::GetAcl),\n                SpanId = data.session_id,\n                MailboxName = arguments.mailbox_name.clone(),\n                AccountId = mailbox_id.account_id,\n                MailboxId = mailbox_id.mailbox_id,\n                Total = permissions.len(),\n                Elapsed = op_start.elapsed()\n            );\n\n            data.write_bytes(\n                StatusResponse::completed(Command::GetAcl)\n                    .with_tag(arguments.tag)\n                    .serialize(\n                        GetAclResponse {\n                            mailbox_name: arguments.mailbox_name.to_string(),\n                            permissions,\n                        }\n                        .into_bytes(is_utf8),\n                    ),\n            )\n            .await\n        })\n    }\n\n    pub async fn handle_my_rights(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapMyRights)?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_acl(self.is_utf8)?;\n        let data = self.state.session_data();\n        let is_utf8 = self.version.is_rev2() || self.is_utf8;\n\n        spawn_op!(data, {\n            let (mailbox_id, mailbox_, access_token) = data\n                .get_acl_mailbox(&arguments, false)\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n            let mailbox = mailbox_\n                .to_unarchived::<email::mailbox::Mailbox>()\n                .imap_ctx(&arguments.tag, trc::location!())?;\n            let rights = if access_token.is_shared(mailbox_id.account_id) {\n                let acl = mailbox.inner.acls.effective_acl(&access_token);\n                let mut rights = Vec::with_capacity(5);\n                if acl.contains(Acl::ReadItems) {\n                    rights.push(Rights::Read);\n                    rights.push(Rights::Lookup);\n                }\n                if acl.contains(Acl::AddItems) {\n                    rights.push(Rights::Insert);\n                }\n                if acl.contains(Acl::RemoveItems) {\n                    rights.push(Rights::DeleteMessages);\n                    rights.push(Rights::Expunge);\n                }\n                if acl.contains(Acl::ModifyItems) {\n                    rights.push(Rights::Seen);\n                    rights.push(Rights::Write);\n                }\n                if acl.contains(Acl::CreateChild) {\n                    rights.push(Rights::CreateMailbox);\n                }\n                if acl.contains(Acl::Delete) {\n                    rights.push(Rights::DeleteMailbox);\n                }\n                if acl.contains(Acl::Submit) {\n                    rights.push(Rights::Post);\n                }\n                rights\n            } else {\n                vec![\n                    Rights::Read,\n                    Rights::Lookup,\n                    Rights::Insert,\n                    Rights::DeleteMessages,\n                    Rights::Expunge,\n                    Rights::Seen,\n                    Rights::Write,\n                    Rights::CreateMailbox,\n                    Rights::DeleteMailbox,\n                    Rights::Post,\n                    Rights::Administer,\n                ]\n            };\n\n            trc::event!(\n                Imap(trc::ImapEvent::MyRights),\n                SpanId = data.session_id,\n                MailboxName = arguments.mailbox_name.clone(),\n                AccountId = mailbox_id.account_id,\n                MailboxId = mailbox_id.mailbox_id,\n                Details = rights\n                    .iter()\n                    .map(|r| trc::Value::String(r.to_compact_string()))\n                    .collect::<Vec<_>>(),\n                Elapsed = op_start.elapsed()\n            );\n\n            data.write_bytes(\n                StatusResponse::completed(Command::MyRights)\n                    .with_tag(arguments.tag)\n                    .serialize(\n                        MyRightsResponse {\n                            mailbox_name: arguments.mailbox_name.to_string(),\n                            rights,\n                        }\n                        .into_bytes(is_utf8),\n                    ),\n            )\n            .await\n        })\n    }\n\n    pub async fn handle_set_acl(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapAclSet)?;\n\n        let op_start = Instant::now();\n        let command = request.command;\n        let arguments = request.parse_acl(self.is_utf8)?;\n        let data = self.state.session_data();\n\n        spawn_op!(data, {\n            // Validate mailbox\n            let (mailbox_id, current_mailbox, _) = data\n                .get_acl_mailbox(&arguments, false)\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n            let current_mailbox = current_mailbox\n                .into_deserialized::<email::mailbox::Mailbox>()\n                .imap_ctx(&arguments.tag, trc::location!())?;\n\n            // Obtain principal id\n            let acl_account_id = data\n                .server\n                .core\n                .storage\n                .directory\n                .query(\n                    QueryParams::name(arguments.identifier.as_ref().unwrap())\n                        .with_return_member_of(false),\n                )\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?\n                .ok_or_else(|| {\n                    trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Account does not exist\")\n                        .id(arguments.tag.to_string())\n                        .caused_by(trc::location!())\n                })?\n                .id();\n\n            // Prepare changes\n            let mut mailbox = current_mailbox.inner.clone();\n            let (op, rights) = arguments\n                .mod_rights\n                .map(|mr| {\n                    (\n                        mr.op,\n                        Bitmap::from_iter(mr.rights.into_iter().map(Acl::from)),\n                    )\n                })\n                .unwrap_or_else(|| (ModRightsOp::Replace, Bitmap::new()));\n\n            if let Some(item) = mailbox\n                .acls\n                .iter_mut()\n                .find(|item| item.account_id == acl_account_id)\n            {\n                match op {\n                    ModRightsOp::Replace => {\n                        if !rights.is_empty() {\n                            item.grants = rights;\n                        } else {\n                            mailbox\n                                .acls\n                                .retain(|item| item.account_id != acl_account_id);\n                        }\n                    }\n                    ModRightsOp::Add => {\n                        item.grants.union(&rights);\n                    }\n                    ModRightsOp::Remove => {\n                        for right in rights {\n                            item.grants.remove(right);\n                        }\n                        if item.grants.is_empty() {\n                            mailbox\n                                .acls\n                                .retain(|item| item.account_id != acl_account_id);\n                        }\n                    }\n                }\n            } else if !rights.is_empty() {\n                match op {\n                    ModRightsOp::Add | ModRightsOp::Replace => {\n                        mailbox.acls.push(AclGrant {\n                            account_id: acl_account_id,\n                            grants: rights,\n                        });\n                    }\n                    ModRightsOp::Remove => (),\n                }\n            }\n\n            if mailbox.acls.len() > data.server.core.groupware.max_shares_per_item {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Maximum shares per item exceeded\")\n                    .caused_by(trc::location!()));\n            }\n\n            let grants = mailbox\n                .acls\n                .iter()\n                .map(|r| trc::Value::from(r.account_id))\n                .collect::<Vec<_>>();\n\n            // Write changes\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(mailbox_id.account_id)\n                .with_collection(Collection::Mailbox)\n                .with_document(mailbox_id.mailbox_id)\n                .custom(\n                    ObjectIndexBuilder::new()\n                        .with_changes(mailbox)\n                        .with_current(current_mailbox),\n                )\n                .imap_ctx(&arguments.tag, trc::location!())?;\n\n            if !batch.is_empty() {\n                data.server\n                    .commit_batch(batch)\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n            }\n\n            // Invalidate ACLs\n            data.server\n                .invalidate_principal_caches(ChangedPrincipals::from_change(\n                    acl_account_id,\n                    Type::Individual,\n                    PrincipalField::EnabledPermissions,\n                ))\n                .await;\n\n            trc::event!(\n                Imap(trc::ImapEvent::SetAcl),\n                SpanId = data.session_id,\n                MailboxName = arguments.mailbox_name.clone(),\n                AccountId = mailbox_id.account_id,\n                MailboxId = mailbox_id.mailbox_id,\n                Details = grants,\n                Elapsed = op_start.elapsed()\n            );\n\n            data.write_bytes(\n                StatusResponse::completed(command)\n                    .with_tag(arguments.tag)\n                    .into_bytes(),\n            )\n            .await\n        })\n    }\n\n    pub async fn handle_list_rights(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapListRights)?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_acl(self.is_utf8)?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::ListRights),\n            SpanId = self.session_id,\n            MailboxName = arguments.mailbox_name.clone(),\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_bytes(\n            StatusResponse::completed(Command::ListRights)\n                .with_tag(arguments.tag)\n                .serialize(\n                    ListRightsResponse {\n                        mailbox_name: arguments.mailbox_name,\n                        identifier: arguments.identifier.unwrap(),\n                        permissions: vec![\n                            vec![Rights::Read],\n                            vec![Rights::Lookup],\n                            vec![Rights::Write, Rights::Seen],\n                            vec![Rights::Insert],\n                            vec![Rights::Expunge, Rights::DeleteMessages],\n                            vec![Rights::CreateMailbox],\n                            vec![Rights::DeleteMailbox],\n                            vec![Rights::Post],\n                            vec![Rights::Administer],\n                        ],\n                    }\n                    .into_bytes(self.version.is_rev2() || self.is_utf8),\n                ),\n        )\n        .await\n    }\n\n    pub fn assert_has_permission(&self, permission: Permission) -> trc::Result<bool> {\n        match &self.state {\n            State::Authenticated { data } | State::Selected { data, .. } => {\n                data.access_token.assert_has_permission(permission)\n            }\n            State::NotAuthenticated { .. } => Ok(false),\n        }\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    async fn get_acl_mailbox(\n        &self,\n        arguments: &Arguments,\n        validate: bool,\n    ) -> trc::Result<(MailboxId, Archive<AlignedBytes>, Arc<AccessToken>)> {\n        if let Some(mailbox) = self.get_mailbox_by_name(&arguments.mailbox_name) {\n            if let Some(values) = self\n                .server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    mailbox.account_id,\n                    Collection::Mailbox,\n                    mailbox.mailbox_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                let access_token = self.get_access_token().await.caused_by(trc::location!())?;\n                if !validate\n                    || access_token.is_member(mailbox.account_id)\n                    || values\n                        .unarchive::<email::mailbox::Mailbox>()\n                        .caused_by(trc::location!())?\n                        .acls\n                        .effective_acl(&access_token)\n                        .contains(Acl::Share)\n                {\n                    Ok((mailbox, values, access_token))\n                } else {\n                    Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"You do not have enough permissions to perform this operation.\")\n                        .code(ResponseCode::NoPerm))\n                }\n            } else {\n                Err(trc::ImapEvent::Error\n                    .caused_by(trc::location!())\n                    .details(\"Mailbox does not exist.\"))\n            }\n        } else {\n            Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\"Mailbox does not exist.\"))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/append.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapContext, ToModSeq};\nuse crate::{\n    core::{ImapUidToId, MailboxId, SelectedMailbox, Session, SessionData},\n    spawn_op,\n};\nuse common::{ipc::PushNotification, listener::SessionStream};\nuse directory::Permission;\nuse email::message::ingest::{EmailIngest, IngestEmail, IngestSource};\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse,\n    protocol::{append::Arguments, select::HighestModSeq},\n    receiver::Request,\n};\nuse mail_parser::MessageParser;\nuse std::{sync::Arc, time::Instant};\nuse types::{\n    acl::Acl,\n    keyword::Keyword,\n    type_state::{DataType, StateChange},\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_append(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapAppend)?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_append(self.is_utf8)?;\n        let (data, selected_mailbox) = self.state.session_mailbox_state();\n\n        // Refresh mailboxes\n        data.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Obtain mailbox\n        let mailbox = if let Some(mailbox) = data.get_mailbox_by_name(&arguments.mailbox_name) {\n            mailbox\n        } else {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\"Mailbox does not exist.\")\n                .code(ResponseCode::TryCreate)\n                .id(arguments.tag));\n        };\n        let is_qresync = self.is_qresync;\n\n        spawn_op!(data, {\n            let response = data\n                .append_messages(arguments, selected_mailbox, mailbox, is_qresync, op_start)\n                .await?\n                .into_bytes();\n\n            data.write_bytes(response).await\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    async fn append_messages(\n        &self,\n        arguments: Arguments,\n        selected_mailbox: Option<Arc<SelectedMailbox>>,\n        mailbox: MailboxId,\n        is_qresync: bool,\n        op_start: Instant,\n    ) -> trc::Result<StatusResponse> {\n        // Verify ACLs\n        let account_id = mailbox.account_id;\n        let mailbox_id = mailbox.mailbox_id;\n        if !self\n            .check_mailbox_acl(account_id, mailbox_id, Acl::AddItems)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?\n        {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\n                    \"You do not have the required permissions to append messages to this mailbox.\",\n                )\n                .code(ResponseCode::NoPerm)\n                .id(arguments.tag));\n        }\n\n        // Obtain access token\n        let access_token = self\n            .server\n            .get_access_token(mailbox.account_id)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Append messages\n        let mut response = StatusResponse::completed(Command::Append);\n        let mut created_ids = Vec::with_capacity(arguments.messages.len());\n        let mut last_change_id = None;\n        for message in arguments.messages {\n            match self\n                .server\n                .email_ingest(IngestEmail {\n                    raw_message: &message.message,\n                    message: MessageParser::new().parse(&message.message),\n                    blob_hash: None,\n                    access_token: &access_token,\n                    mailbox_ids: vec![mailbox_id],\n                    keywords: message.flags.into_iter().map(Keyword::from).collect(),\n                    received_at: message.received_at.map(|d| d as u64),\n                    source: IngestSource::Imap {\n                        train_classifier: true,\n                    },\n                    session_id: self.session_id,\n                })\n                .await\n            {\n                Ok(email) => {\n                    created_ids.push(ImapUidToId {\n                        uid: email.imap_uids[0],\n                        id: email.document_id,\n                    });\n                    last_change_id = Some(email.change_id);\n                }\n                Err(err) => {\n                    return Err(\n                        if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota)) {\n                            err.details(\"Disk quota exceeded.\")\n                                .code(ResponseCode::OverQuota)\n                        } else if err.matches(trc::EventType::Limit(trc::LimitEvent::TenantQuota)) {\n                            err.details(\"Organization disk quota exceeded.\")\n                                .code(ResponseCode::OverQuota)\n                        } else {\n                            err\n                        }\n                        .id(arguments.tag),\n                    );\n                }\n            }\n        }\n\n        // Broadcast changes\n        if let Some(change_id) = last_change_id {\n            self.server\n                .broadcast_push_notification(PushNotification::StateChange(\n                    StateChange::new(account_id)\n                        .with_change_id(change_id)\n                        .with_change(DataType::Email)\n                        .with_change(DataType::Mailbox)\n                        .with_change(DataType::Thread),\n                ))\n                .await;\n        }\n\n        trc::event!(\n            Imap(trc::ImapEvent::Append),\n            SpanId = self.session_id,\n            MailboxName = arguments.mailbox_name.clone(),\n            AccountId = account_id,\n            MailboxId = mailbox_id,\n            DocumentId = created_ids\n                .iter()\n                .map(|r| trc::Value::from(r.id))\n                .collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        if !created_ids.is_empty() {\n            let uids = created_ids.iter().map(|id| id.uid).collect();\n            match selected_mailbox {\n                Some(selected_mailbox) if selected_mailbox.id == mailbox => {\n                    // Write updated modseq\n                    if is_qresync {\n                        self.write_bytes(\n                            HighestModSeq::new(last_change_id.unwrap_or_default().to_modseq())\n                                .into_bytes(),\n                        )\n                        .await?;\n                    }\n\n                    selected_mailbox.append_messages(created_ids, last_change_id);\n                }\n                _ => {}\n            };\n            let uid_validity = self\n                .mailbox_state(&mailbox)\n                .map(|m| m.uid_validity as u32)\n                .unwrap_or_default();\n\n            response = response.with_code(ResponseCode::AppendUid { uid_validity, uids });\n        }\n\n        Ok(response.with_tag(arguments.tag))\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/authenticate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    auth::{\n        AuthRequest,\n        sasl::{sasl_decode_challenge_oauth, sasl_decode_challenge_plain},\n    },\n    listener::{SessionStream, limiter::LimiterResult},\n};\n\nuse directory::Permission;\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse,\n    protocol::{authenticate::Mechanism, capability::Capability},\n    receiver::{self, Request},\n};\nuse mail_parser::decoders::base64::base64_decode;\nuse mail_send::Credentials;\nuse std::sync::Arc;\n\nuse crate::core::{Session, SessionData, State};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_authenticate(&mut self, request: Request<Command>) -> trc::Result<()> {\n        let mut args = request.parse_authenticate()?;\n\n        match args.mechanism {\n            Mechanism::Plain | Mechanism::OAuthBearer | Mechanism::XOauth2 => {\n                if !args.params.is_empty() {\n                    let challenge = base64_decode(args.params.pop().unwrap().as_bytes())\n                        .ok_or_else(|| {\n                            trc::AuthEvent::Error\n                                .into_err()\n                                .details(\"Failed to decode challenge.\")\n                                .id(args.tag.clone())\n                                .code(ResponseCode::Parse)\n                        })?;\n\n                    let credentials = if args.mechanism == Mechanism::Plain {\n                        sasl_decode_challenge_plain(&challenge)\n                    } else {\n                        sasl_decode_challenge_oauth(&challenge)\n                    }\n                    .ok_or_else(|| {\n                        trc::AuthEvent::Error\n                            .into_err()\n                            .details(\"Invalid SASL challenge.\")\n                            .id(args.tag.clone())\n                    })?;\n\n                    self.authenticate(credentials, args.tag).await\n                } else {\n                    self.receiver.request = receiver::Request {\n                        tag: args.tag,\n                        command: Command::Authenticate,\n                        tokens: vec![receiver::Token::Argument(args.mechanism.into_bytes())],\n                    };\n                    self.receiver.state = receiver::State::Argument { last_ch: b' ' };\n                    self.write_bytes(b\"+ \\r\\n\".to_vec()).await\n                }\n            }\n            _ => Err(trc::AuthEvent::Error\n                .into_err()\n                .details(\"Authentication mechanism not supported.\")\n                .id(args.tag)\n                .code(ResponseCode::Cannot)),\n        }\n    }\n\n    pub async fn authenticate(\n        &mut self,\n        credentials: Credentials<String>,\n        tag: String,\n    ) -> trc::Result<()> {\n        // Authenticate\n        let access_token = self\n            .server\n            .authenticate(&AuthRequest::from_credentials(\n                credentials,\n                self.session_id,\n                self.remote_addr,\n            ))\n            .await\n            .map_err(|err| {\n                if err.matches(trc::EventType::Auth(trc::AuthEvent::Failed)) {\n                    let auth_failures = self.state.auth_failures();\n                    if auth_failures < self.server.core.imap.max_auth_failures {\n                        self.state = State::NotAuthenticated {\n                            auth_failures: auth_failures + 1,\n                        };\n                    } else {\n                        return trc::AuthEvent::TooManyAttempts.into_err().caused_by(err);\n                    }\n                }\n\n                err.id(tag.clone())\n            })\n            .and_then(|token| {\n                token\n                    .assert_has_permission(Permission::ImapAuthenticate)\n                    .map(|_| token)\n            })?;\n\n        // Enforce concurrency limits\n        let in_flight = match access_token.is_imap_request_allowed() {\n            LimiterResult::Allowed(in_flight) => Some(in_flight),\n            LimiterResult::Forbidden => {\n                return Err(trc::LimitEvent::ConcurrentRequest\n                    .into_err()\n                    .id(tag.clone()));\n            }\n            LimiterResult::Disabled => None,\n        };\n\n        // Create session\n        self.state = State::Authenticated {\n            data: Arc::new(\n                SessionData::new(self, access_token, in_flight)\n                    .await\n                    .map_err(|err| err.id(tag.clone()))?,\n            ),\n        };\n        self.write_bytes(\n            StatusResponse::ok(\"Authentication successful\")\n                .with_code(ResponseCode::Capability {\n                    capabilities: Capability::all_capabilities(\n                        true,\n                        !self.is_tls && self.instance.acceptor.is_tls(),\n                    ),\n                })\n                .with_tag(tag)\n                .into_bytes(),\n        )\n        .await\n    }\n\n    pub async fn handle_unauthenticate(&mut self, request: Request<Command>) -> trc::Result<()> {\n        self.state = State::NotAuthenticated { auth_failures: 0 };\n\n        self.write_bytes(\n            StatusResponse::completed(Command::Unauthenticate)\n                .with_tag(request.tag)\n                .into_bytes(),\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/capability.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse crate::core::Session;\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::{\n    Command, StatusResponse,\n    protocol::{\n        ImapResponse,\n        capability::{Capability, Response},\n    },\n    receiver::Request,\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_capability(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapCapability)?;\n\n        let op_start = Instant::now();\n        trc::event!(\n            Imap(trc::ImapEvent::Capabilities),\n            SpanId = self.session_id,\n            Tls = self.is_tls,\n            Strict = !self.server.core.imap.allow_plain_auth,\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_bytes(\n            StatusResponse::completed(Command::Capability)\n                .with_tag(request.tag)\n                .serialize(\n                    Response {\n                        capabilities: Capability::all_capabilities(\n                            self.state.is_authenticated(),\n                            !self.is_tls && self.instance.acceptor.is_tls(),\n                        ),\n                    }\n                    .serialize(),\n                ),\n        )\n        .await\n    }\n\n    pub async fn handle_id(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapId)?;\n\n        let op_start = Instant::now();\n        trc::event!(\n            Imap(trc::ImapEvent::Id),\n            SpanId = self.session_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_bytes(\n            StatusResponse::completed(Command::Id)\n                .with_tag(request.tag)\n                .serialize(\n                    concat!(\n                        \"* ID (\\\"name\\\" \\\"Stalwart\\\" \\\"version\\\" \\\"1.0.0\\\" \\\"vendor\\\" \\\"Stalwart Labs LLC\\\" \",\n                        \"\\\"support-url\\\" \\\"https://stalw.art\\\")\\r\\n\"\n                    )\n                    .as_bytes()\n                    .to_vec(),\n                ),\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/close.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse crate::core::{Session, State};\nuse common::listener::SessionStream;\nuse imap_proto::{Command, StatusResponse, receiver::Request};\nuse trc::AddContext;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_close(&mut self, request: Request<Command>) -> trc::Result<()> {\n        let op_start = Instant::now();\n        let (data, mailbox) = self.state.select_data();\n\n        if mailbox.is_select {\n            data.expunge(mailbox.clone(), None, op_start)\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        trc::event!(\n            Imap(trc::ImapEvent::Close),\n            SpanId = self.session_id,\n            AccountId = mailbox.id.account_id,\n            MailboxId = mailbox.id.mailbox_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        self.state = State::Authenticated { data };\n        self.write_bytes(\n            StatusResponse::completed(Command::Close)\n                .with_tag(request.tag)\n                .into_bytes(),\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ImapContext;\nuse crate::{\n    core::{MailboxId, SelectedMailbox, Session, SessionData},\n    spawn_op,\n};\nuse common::{ipc::PushNotification, listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    mailbox::{JUNK_ID, TRASH_ID, UidMailbox},\n    message::{\n        copy::{CopyMessageError, EmailCopy},\n        ingest::EmailIngest,\n        metadata::MessageData,\n    },\n};\nuse imap_proto::{\n    Command, ResponseCode, ResponseType, StatusResponse, protocol::copy_move::Arguments,\n    receiver::Request,\n};\nuse std::{sync::Arc, time::Instant};\nuse store::{\n    ValueKey,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, VanishedCollection},\n    type_state::{DataType, StateChange},\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_copy_move(\n        &mut self,\n        request: Request<Command>,\n        is_move: bool,\n        is_uid: bool,\n    ) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(if is_move {\n            Permission::ImapMove\n        } else {\n            Permission::ImapCopy\n        })?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_copy_move(self.is_utf8)?;\n        let (data, src_mailbox) = self.state.mailbox_state();\n        let is_qresync = self.is_qresync;\n\n        spawn_op!(data, {\n            // Refresh mailboxes\n            data.synchronize_mailboxes(false)\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n\n            // Make sure the mailbox exists.\n            let dest_mailbox =\n                if let Some(mailbox) = data.get_mailbox_by_name(&arguments.mailbox_name) {\n                    mailbox\n                } else {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Destination mailbox does not exist.\")\n                        .code(ResponseCode::TryCreate)\n                        .id(arguments.tag));\n                };\n\n            // Check that the destination mailbox is not the same as the source mailbox.\n            if src_mailbox.id.account_id == dest_mailbox.account_id\n                && src_mailbox.id.mailbox_id == dest_mailbox.mailbox_id\n            {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Source and destination mailboxes are the same.\")\n                    .code(ResponseCode::Cannot)\n                    .id(arguments.tag));\n            }\n\n            data.copy_move(\n                arguments,\n                src_mailbox,\n                dest_mailbox,\n                is_move,\n                is_uid,\n                is_qresync,\n                op_start,\n            )\n            .await\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    #[allow(clippy::too_many_arguments)]\n    pub async fn copy_move(\n        &self,\n        arguments: Arguments,\n        src_mailbox: Arc<SelectedMailbox>,\n        dest_mailbox: MailboxId,\n        is_move: bool,\n        is_uid: bool,\n        is_qresync: bool,\n        op_start: Instant,\n    ) -> trc::Result<()> {\n        self.synchronize_messages(&src_mailbox)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Convert IMAP ids to JMAP ids.\n        let ids = src_mailbox\n            .sequence_to_ids(&arguments.sequence_set, is_uid)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        if ids.is_empty() {\n            trc::event!(\n                Imap(if is_move {\n                    trc::ImapEvent::Move\n                } else {\n                    trc::ImapEvent::Copy\n                }),\n                SpanId = self.session_id,\n                Source = src_mailbox.id.account_id,\n                Details = trc::Value::None,\n                Uid = trc::Value::None,\n                AccountId = dest_mailbox.account_id,\n                MailboxId = dest_mailbox.mailbox_id,\n                Elapsed = op_start.elapsed()\n            );\n\n            return self\n                .write_bytes(\n                    StatusResponse::ok(if is_move {\n                        \"No messages were moved.\"\n                    } else {\n                        \"No messages were copied.\"\n                    })\n                    .with_tag(arguments.tag)\n                    .into_bytes(),\n                )\n                .await;\n        }\n\n        // Verify that the user can delete messages from the source mailbox.\n        if is_move\n            && !self\n                .check_mailbox_acl(\n                    src_mailbox.id.account_id,\n                    src_mailbox.id.mailbox_id,\n                    Acl::RemoveItems,\n                )\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?\n        {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(concat!(\n                    \"You do not have the required permissions to \",\n                    \"remove messages from the source mailbox.\"\n                ))\n                .code(ResponseCode::NoPerm)\n                .id(arguments.tag));\n        }\n\n        // Verify that the user can append messages to the destination mailbox.\n        let dest_mailbox_id = dest_mailbox.mailbox_id;\n        if !self\n            .check_mailbox_acl(dest_mailbox.account_id, dest_mailbox_id, Acl::AddItems)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?\n        {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(concat!(\n                    \"You do not have the required permissions to \",\n                    \"add messages to the destination mailbox.\"\n                ))\n                .code(ResponseCode::NoPerm)\n                .id(arguments.tag));\n        }\n\n        let mut response = StatusResponse::completed(if is_move {\n            Command::Move(is_uid)\n        } else {\n            Command::Copy(is_uid)\n        });\n        let mut did_move = false;\n        let mut copied_ids = Vec::with_capacity(ids.len());\n        let access_token = self\n            .server\n            .get_access_token(dest_mailbox.account_id)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        if src_mailbox.id.account_id == dest_mailbox.account_id {\n            // Mailboxes are in the same account\n            let account_id = src_mailbox.id.account_id;\n            let dest_mailbox_id = UidMailbox::new_unassigned(dest_mailbox_id);\n            let mut batch = BatchBuilder::new();\n\n            for (id, imap_id) in ids {\n                // Obtain mailbox tags\n                let data_ = if let Some(result) = self\n                    .get_message_data(account_id, id)\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?\n                {\n                    result\n                } else {\n                    continue;\n                };\n\n                // Deserialize\n                let data = data_\n                    .to_unarchived::<MessageData>()\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n\n                // Make sure the message still belongs to this mailbox\n                if !data\n                    .inner\n                    .mailboxes\n                    .iter()\n                    .any(|mailbox| mailbox.mailbox_id == src_mailbox.id.mailbox_id)\n                {\n                    continue;\n                }\n\n                // If the message is already in the destination mailbox, skip it.\n                if let Some(mailbox) = data\n                    .inner\n                    .mailboxes\n                    .iter()\n                    .find(|mailbox| mailbox.mailbox_id == dest_mailbox_id.mailbox_id)\n                {\n                    copied_ids.push((imap_id.uid, mailbox.uid.to_native()));\n\n                    if is_move {\n                        let mut new_data = data.inner.to_builder();\n                        new_data.remove_mailbox(src_mailbox.id.mailbox_id);\n                        batch\n                            .with_account_id(account_id)\n                            .with_collection(Collection::Email)\n                            .with_document(id)\n                            .custom(\n                                ObjectIndexBuilder::new()\n                                    .with_current(data)\n                                    .with_changes(new_data.seal()),\n                            )\n                            .imap_ctx(&arguments.tag, trc::location!())?\n                            .log_vanished_item(\n                                VanishedCollection::Email,\n                                (src_mailbox.id.mailbox_id, imap_id.uid),\n                            )\n                            .commit_point();\n                        did_move = true;\n                    }\n\n                    continue;\n                }\n\n                // Prepare changes\n                let mut new_data = data.inner.to_builder();\n\n                // Add destination folder\n                new_data.add_mailbox(dest_mailbox_id);\n                if is_move {\n                    new_data.remove_mailbox(src_mailbox.id.mailbox_id);\n                }\n\n                // Assign IMAP UIDs\n                let ids = self\n                    .server\n                    .assign_email_ids(\n                        account_id,\n                        new_data\n                            .mailboxes\n                            .iter()\n                            .filter(|m| m.uid == 0)\n                            .map(|m| m.mailbox_id),\n                        false,\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n\n                for (uid_mailbox, uid) in new_data\n                    .mailboxes\n                    .iter_mut()\n                    .filter(|m| m.uid == 0)\n                    .zip(ids)\n                {\n                    copied_ids.push((imap_id.uid, uid));\n                    uid_mailbox.uid = uid;\n                }\n\n                // Prepare write batch\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Email)\n                    .with_document(id)\n                    .custom(\n                        ObjectIndexBuilder::new()\n                            .with_current(data)\n                            .with_changes(new_data.seal()),\n                    )\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n                if is_move {\n                    batch.log_vanished_item(\n                        VanishedCollection::Email,\n                        (src_mailbox.id.mailbox_id, imap_id.uid),\n                    );\n                }\n\n                // Add message to training queue\n                if dest_mailbox_id.mailbox_id == JUNK_ID {\n                    self.server\n                        .add_account_spam_sample(&mut batch, account_id, id, true, self.session_id)\n                        .await\n                        .imap_ctx(&arguments.tag, trc::location!())?;\n                } else if src_mailbox.id.mailbox_id == JUNK_ID\n                    && dest_mailbox_id.mailbox_id != TRASH_ID\n                {\n                    self.server\n                        .add_account_spam_sample(&mut batch, account_id, id, false, self.session_id)\n                        .await\n                        .imap_ctx(&arguments.tag, trc::location!())?;\n                }\n\n                batch.commit_point();\n\n                // Update changelog\n                if is_move {\n                    did_move = true;\n                }\n            }\n\n            // Write changes\n            self.server\n                .commit_batch(batch)\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n        } else {\n            // Obtain quota for target account\n            let src_account_id = src_mailbox.id.account_id;\n            let mut dest_change_id = None;\n            let dest_account_id = dest_mailbox.account_id;\n            let resource_token = access_token.as_resource_token();\n            let mut destroy_ids = RoaringBitmap::new();\n            let cache = self\n                .server\n                .get_cached_messages(src_account_id)\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n            for (id, imap_id) in ids {\n                match self\n                    .server\n                    .copy_message(\n                        src_account_id,\n                        id,\n                        &resource_token,\n                        vec![dest_mailbox_id],\n                        cache\n                            .email_by_id(&id)\n                            .map(|e| cache.expand_keywords(e).collect())\n                            .unwrap_or_default(),\n                        None,\n                        self.session_id,\n                    )\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?\n                {\n                    Ok(email) => {\n                        dest_change_id = email.change_id.into();\n                        if let Some(assigned_uid) = email.imap_uids.first() {\n                            debug_assert!(*assigned_uid > 0);\n                            copied_ids.push((imap_id.uid, *assigned_uid));\n                        }\n                    }\n                    Err(err) => {\n                        match err {\n                            CopyMessageError::OverQuota => {\n                                response.rtype = ResponseType::No;\n                                response.code = Some(ResponseCode::OverQuota);\n                                response.message = \"Mailbox quota exceeded\".into();\n                            }\n                            CopyMessageError::NotFound => (),\n                        }\n                        continue;\n                    }\n                };\n\n                if is_move {\n                    destroy_ids.insert(id);\n                }\n            }\n\n            // Untag or delete emails\n            if !destroy_ids.is_empty() {\n                let mut batch = BatchBuilder::new();\n                self.email_untag_or_delete(\n                    src_account_id,\n                    src_mailbox.id.mailbox_id,\n                    &destroy_ids,\n                    &mut batch,\n                )\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n\n                self.server\n                    .commit_batch(batch)\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n\n                did_move = true;\n            }\n\n            // Broadcast changes on destination account\n            if let Some(change_id) = dest_change_id {\n                self.server\n                    .broadcast_push_notification(PushNotification::StateChange(\n                        StateChange::new(dest_account_id)\n                            .with_change_id(change_id)\n                            .with_change(DataType::Email)\n                            .with_change(DataType::Thread)\n                            .with_change(DataType::Mailbox),\n                    ))\n                    .await;\n            }\n        }\n\n        // Map copied JMAP Ids to IMAP UIDs in the destination folder.\n        if copied_ids.is_empty() {\n            return if response.rtype != ResponseType::Ok {\n                Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(response.message)\n                    .ctx_opt(trc::Key::Code, response.code)\n                    .id(arguments.tag))\n            } else {\n                trc::event!(\n                    Imap(if is_move {\n                        trc::ImapEvent::Move\n                    } else {\n                        trc::ImapEvent::Copy\n                    }),\n                    SpanId = self.session_id,\n                    Source = src_mailbox.id.account_id,\n                    Details = trc::Value::None,\n                    Uid = trc::Value::None,\n                    AccountId = dest_mailbox.account_id,\n                    MailboxId = dest_mailbox.mailbox_id,\n                    Elapsed = op_start.elapsed()\n                );\n\n                self.write_bytes(\n                    StatusResponse::ok(if is_move {\n                        \"No messages were moved.\"\n                    } else {\n                        \"No messages were copied.\"\n                    })\n                    .with_tag(arguments.tag)\n                    .into_bytes(),\n                )\n                .await\n            };\n        }\n\n        // Prepare response\n        let uid_validity = self\n            .mailbox_state(&dest_mailbox)\n            .map(|m| m.uid_validity as u32)\n            .unwrap_or_default();\n\n        let mut src_uids = Vec::with_capacity(copied_ids.len());\n        let mut dest_uids = Vec::with_capacity(copied_ids.len());\n        for (src_uid, dest_uid) in copied_ids {\n            src_uids.push(src_uid);\n            dest_uids.push(dest_uid);\n        }\n        src_uids.sort_unstable();\n        dest_uids.sort_unstable();\n\n        trc::event!(\n            Imap(if is_move {\n                trc::ImapEvent::Move\n            } else {\n                trc::ImapEvent::Copy\n            }),\n            SpanId = self.session_id,\n            Source = src_mailbox.id.account_id,\n            Details = src_uids\n                .iter()\n                .map(|r| trc::Value::from(*r))\n                .collect::<Vec<_>>(),\n            AccountId = dest_mailbox.account_id,\n            MailboxId = dest_mailbox.mailbox_id,\n            Uid = dest_uids\n                .iter()\n                .map(|r| trc::Value::from(*r))\n                .collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        let response = if is_move {\n            self.write_bytes(\n                StatusResponse::ok(\"Copied UIDs\")\n                    .with_code(ResponseCode::CopyUid {\n                        uid_validity,\n                        src_uids,\n                        dest_uids,\n                    })\n                    .into_bytes(),\n            )\n            .await?;\n\n            if did_move {\n                // Resynchronize source mailbox on a successful move\n                self.write_mailbox_changes(&src_mailbox, is_qresync)\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n            }\n\n            response.with_tag(arguments.tag).into_bytes()\n        } else {\n            response\n                .with_tag(arguments.tag)\n                .with_code(ResponseCode::CopyUid {\n                    uid_validity,\n                    src_uids,\n                    dest_uids,\n                })\n                .into_bytes()\n        };\n\n        self.write_bytes(response).await\n    }\n\n    pub async fn get_message_data(\n        &self,\n        account_id: u32,\n        id: u32,\n    ) -> trc::Result<Option<Archive<AlignedBytes>>> {\n        if let Some(data) = self\n            .server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::Email,\n                id,\n            ))\n            .await?\n        {\n            Ok(Some(data))\n        } else {\n            trc::event!(\n                Store(trc::StoreEvent::NotFound),\n                AccountId = account_id,\n                Collection = Collection::Email,\n                MessageId = id,\n                SpanId = self.session_id,\n                Details = \"Message not found\"\n            );\n\n            Ok(None)\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/create.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{Session, SessionData},\n    op::ImapContext,\n    spawn_op,\n};\nuse common::{listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse email::cache::{MessageCacheFetch, mailbox::MailboxCacheAccess};\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse,\n    protocol::{create::Arguments, list::Attribute},\n    receiver::Request,\n};\nuse std::time::Instant;\nuse store::write::BatchBuilder;\nuse trc::AddContext;\nuse types::{acl::Acl, collection::Collection, id::Id, special_use::SpecialUse};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_create(&mut self, requests: Vec<Request<Command>>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapCreate)?;\n\n        let data = self.state.session_data();\n        let is_utf8 = self.is_utf8;\n\n        spawn_op!(data, {\n            for request in requests {\n                match request.parse_create(is_utf8) {\n                    Ok(argument) => match data.create_folder(argument).await {\n                        Ok(response) => {\n                            data.write_bytes(response.into_bytes()).await?;\n                        }\n                        Err(error) => {\n                            data.write_error(error).await?;\n                        }\n                    },\n                    Err(err) => data.write_error(err).await?,\n                }\n            }\n\n            Ok(())\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn create_folder(&self, arguments: Arguments) -> trc::Result<StatusResponse> {\n        let op_start = Instant::now();\n\n        // Refresh mailboxes\n        self.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Validate mailbox name\n        let params = self\n            .validate_mailbox_create(&arguments.mailbox_name, arguments.mailbox_role)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n        debug_assert!(!params.path.is_empty());\n\n        // Build batch\n        let mut parent_id = params.parent_mailbox_id.map(|id| id + 1).unwrap_or(0);\n        let mut create_ids = Vec::with_capacity(params.path.len());\n        let mut next_document_id = self\n            .server\n            .store()\n            .assign_document_ids(\n                params.account_id,\n                Collection::Mailbox,\n                params.path.len() as u64,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        let mut batch = BatchBuilder::new();\n        for (pos, &path_item) in params.path.iter().enumerate() {\n            let mut mailbox = email::mailbox::Mailbox::new(path_item).with_parent_id(parent_id);\n\n            if pos == params.path.len() - 1\n                && let Some(mailbox_role) = arguments.mailbox_role.map(attr_to_role)\n            {\n                mailbox.role = mailbox_role;\n            }\n            let mailbox_id = next_document_id;\n            next_document_id -= 1;\n            batch\n                .with_account_id(params.account_id)\n                .with_collection(Collection::Mailbox)\n                .with_document(mailbox_id)\n                .custom(ObjectIndexBuilder::<(), _>::new().with_changes(mailbox))\n                .imap_ctx(&arguments.tag, trc::location!())?\n                .commit_point();\n            parent_id = mailbox_id + 1;\n            create_ids.push(mailbox_id);\n        }\n\n        self.server\n            .commit_batch(batch)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::CreateMailbox),\n            SpanId = self.session_id,\n            MailboxName = arguments.mailbox_name.clone(),\n            AccountId = params.account_id,\n            MailboxId = create_ids\n                .iter()\n                .map(|&id| trc::Value::from(id))\n                .collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        // Build response\n        Ok(StatusResponse::ok(\"Mailbox created.\")\n            .with_code(ResponseCode::MailboxId {\n                mailbox_id: Id::from_parts(params.account_id, parent_id - 1).to_string(),\n            })\n            .with_tag(arguments.tag))\n    }\n\n    pub async fn validate_mailbox_create<'x>(\n        &self,\n        mailbox_name: &'x str,\n        mailbox_role: Option<Attribute>,\n    ) -> trc::Result<CreateParams<'x>> {\n        // Remove leading and trailing separators\n        let mut name = mailbox_name.trim();\n        if let Some(suffix) = name.strip_prefix('/') {\n            name = suffix.trim();\n        };\n        if let Some(prefix) = name.strip_suffix('/') {\n            name = prefix.trim();\n        }\n        if name.is_empty() {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(format!(\"Invalid folder name '{mailbox_name}'.\",)));\n        }\n\n        // Build path\n        let mut path = Vec::new();\n        if name.contains('/') {\n            // Locate parent mailbox\n            for path_item in name.split('/') {\n                let path_item = path_item.trim();\n                if path_item.is_empty() {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Invalid empty path item.\"));\n                } else if path_item.len() > self.server.core.jmap.mailbox_name_max_len {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Mailbox name is too long.\"));\n                }\n                path.push(path_item);\n            }\n\n            if path.len() > self.server.core.jmap.mailbox_max_depth {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Mailbox path is too deep.\"));\n            }\n        } else {\n            path.push(name);\n        }\n\n        // Validate special folders\n        let mut parent_mailbox_id = None;\n        let mut parent_mailbox_name = None;\n        let (account_id, path) = {\n            let mailboxes = self.mailboxes.lock();\n            let (account, full_path, prefix) =\n                if path.first() == Some(&self.server.core.jmap.shared_folder.as_str()) {\n                    // Shared Folders/<username>/<folder>\n                    if path.len() < 3 {\n                        return Err(trc::ImapEvent::Error\n                            .into_err()\n                            .details(\"Mailboxes under root shared folders are not allowed.\")\n                            .code(ResponseCode::Cannot));\n                    }\n\n                    // Build path\n                    let root = &mut path[2];\n                    if root.eq_ignore_ascii_case(\"INBOX\") {\n                        *root = \"INBOX\";\n                    }\n                    let full_path = path.join(\"/\");\n                    let prefix = Some(format!(\"{}/{}\", path[0], path[1]));\n\n                    // Locate account\n                    if let Some(account) = mailboxes\n                        .iter()\n                        .skip(1)\n                        .find(|account| account.prefix == prefix)\n                    {\n                        (account, full_path, prefix)\n                    } else {\n                        #[allow(clippy::unnecessary_literal_unwrap)]\n                        return Err(trc::ImapEvent::Error.into_err().details(format!(\n                            \"Shared account '{}' not found.\",\n                            prefix.unwrap_or_default()\n                        )));\n                    }\n                } else if let Some(account) = mailboxes.first() {\n                    let root = &mut path[0];\n                    if root.eq_ignore_ascii_case(\"INBOX\") {\n                        *root = \"INBOX\";\n                    }\n\n                    (account, path.join(\"/\"), None)\n                } else {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Internal server error.\")\n                        .caused_by(trc::location!())\n                        .code(ResponseCode::ContactAdmin));\n                };\n\n            // Locate parent mailbox\n            if account.mailbox_names.contains_key(&full_path) {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(format!(\"Mailbox '{}' already exists.\", full_path))\n                    .code(ResponseCode::AlreadyExists));\n            }\n\n            (\n                account.account_id,\n                if path.len() > 1 {\n                    let mut create_path = Vec::with_capacity(path.len());\n                    while !path.is_empty() {\n                        let mailbox_name: String = path.join(\"/\");\n                        if let Some(&mailbox_id) = account.mailbox_names.get(&mailbox_name) {\n                            parent_mailbox_id = mailbox_id.into();\n                            parent_mailbox_name = mailbox_name.into();\n                            break;\n                        } else if prefix\n                            .as_ref()\n                            .is_some_and(|prefix| prefix == &mailbox_name)\n                        {\n                            break;\n                        } else {\n                            create_path.push(path.pop().unwrap());\n                        }\n                    }\n                    create_path.reverse();\n                    create_path\n                } else {\n                    path\n                },\n            )\n        };\n\n        // Validate ACLs\n        if let Some(parent_mailbox_id) = parent_mailbox_id {\n            if !self\n                .check_mailbox_acl(account_id, parent_mailbox_id, Acl::CreateChild)\n                .await?\n            {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"You are not allowed to create sub mailboxes under this mailbox.\")\n                    .code(ResponseCode::NoPerm));\n            }\n        } else if self.account_id != account_id\n            && !self\n                .get_access_token()\n                .await\n                .caused_by(trc::location!())?\n                .is_member(account_id)\n        {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\"You are not allowed to create root folders under shared folders.\")\n                .code(ResponseCode::Cannot));\n        }\n\n        Ok(CreateParams {\n            account_id,\n            path,\n            parent_mailbox_id,\n            parent_mailbox_name,\n            special_use: if let Some(mailbox_role) = mailbox_role {\n                // Make sure role is unique\n                let special_use = attr_to_role(mailbox_role);\n                if self\n                    .server\n                    .get_cached_messages(account_id)\n                    .await\n                    .caused_by(trc::location!())?\n                    .mailbox_by_role(&special_use)\n                    .is_some()\n                {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(format!(\n                            \"A mailbox with role '{}' already exists.\",\n                            special_use.as_str().unwrap_or_default()\n                        ))\n                        .code(ResponseCode::UseAttr));\n                }\n                Some(mailbox_role)\n            } else {\n                None\n            },\n            is_rename: false,\n        })\n    }\n}\n\n#[derive(Debug)]\npub struct CreateParams<'x> {\n    pub account_id: u32,\n    pub path: Vec<&'x str>,\n    pub parent_mailbox_id: Option<u32>,\n    pub parent_mailbox_name: Option<String>,\n    pub special_use: Option<Attribute>,\n    pub is_rename: bool,\n}\n\n#[inline]\nfn attr_to_role(attr: Attribute) -> SpecialUse {\n    match attr {\n        Attribute::Archive => SpecialUse::Archive,\n        Attribute::Drafts => SpecialUse::Drafts,\n        Attribute::Junk => SpecialUse::Junk,\n        Attribute::Sent => SpecialUse::Sent,\n        Attribute::Trash => SpecialUse::Trash,\n        Attribute::Important => SpecialUse::Important,\n        Attribute::Memos => SpecialUse::Memos,\n        Attribute::Scheduled => SpecialUse::Scheduled,\n        Attribute::Snoozed => SpecialUse::Snoozed,\n        _ => SpecialUse::None,\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ImapContext;\nuse crate::{\n    core::{Session, SessionData},\n    spawn_op,\n};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::mailbox::destroy::{MailboxDestroy, MailboxDestroyError};\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse, protocol::delete::Arguments, receiver::Request,\n};\nuse std::time::Instant;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_delete(&mut self, requests: Vec<Request<Command>>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapDelete)?;\n\n        let data = self.state.session_data();\n        let is_utf8 = self.is_utf8;\n\n        spawn_op!(data, {\n            for request in requests {\n                match request.parse_delete(is_utf8) {\n                    Ok(argument) => match data.delete_folder(argument).await {\n                        Ok(response) => {\n                            data.write_bytes(response.into_bytes()).await?;\n                        }\n                        Err(error) => {\n                            data.write_error(error).await?;\n                        }\n                    },\n                    Err(response) => data.write_error(response).await?,\n                }\n            }\n\n            Ok(())\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn delete_folder(&self, arguments: Arguments) -> trc::Result<StatusResponse> {\n        let op_start = Instant::now();\n\n        // Refresh mailboxes\n        self.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Validate mailbox\n        let (account_id, mailbox_id) =\n            if let Some(mailbox) = self.get_mailbox_by_name(&arguments.mailbox_name) {\n                (mailbox.account_id, mailbox.mailbox_id)\n            } else {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Mailbox does not exist.\")\n                    .code(ResponseCode::TryCreate)\n                    .id(arguments.tag));\n            };\n\n        // Delete message\n        let access_token = self\n            .get_access_token()\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        if let Err(err) = self\n            .server\n            .mailbox_destroy(account_id, mailbox_id, &access_token, true)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?\n        {\n            let (code, message) = match err {\n                MailboxDestroyError::CannotDestroy => {\n                    (ResponseCode::NoPerm, \"You cannot delete system mailboxes\")\n                }\n                MailboxDestroyError::Forbidden => (\n                    ResponseCode::NoPerm,\n                    \"You do not have enough permissions to delete this mailbox\",\n                ),\n                MailboxDestroyError::HasChildren => {\n                    (ResponseCode::HasChildren, \"Mailbox has children\")\n                }\n                MailboxDestroyError::HasEmails => (ResponseCode::HasChildren, \"Mailbox has emails\"),\n                MailboxDestroyError::NotFound => (ResponseCode::NonExistent, \"Mailbox not found\"),\n                MailboxDestroyError::AssertionFailed => (\n                    ResponseCode::Cannot,\n                    \"Another process is accessing this mailbox\",\n                ),\n            };\n\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(message)\n                .code(code)\n                .id(arguments.tag));\n        }\n\n        // Update mailbox cache\n        for account in self.mailboxes.lock().iter_mut() {\n            if account.account_id == account_id {\n                account.mailbox_names.remove(&arguments.mailbox_name);\n                account.mailbox_state.remove(&mailbox_id);\n                break;\n            }\n        }\n\n        trc::event!(\n            Imap(trc::ImapEvent::DeleteMailbox),\n            SpanId = self.session_id,\n            MailboxName = arguments.mailbox_name,\n            AccountId = account_id,\n            MailboxId = mailbox_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::ok(\"Mailbox deleted.\").with_tag(arguments.tag))\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/enable.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse crate::core::Session;\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::{\n    Command, StatusResponse,\n    protocol::{ImapResponse, ProtocolVersion, capability::Capability, enable},\n    receiver::Request,\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_enable(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapEnable)?;\n\n        let op_start = Instant::now();\n\n        let arguments = request.parse_enable()?;\n        let mut response = enable::Response {\n            enabled: Vec::with_capacity(arguments.capabilities.len()),\n        };\n\n        for capability in arguments.capabilities {\n            match capability {\n                Capability::IMAP4rev2 => {\n                    self.version = ProtocolVersion::Rev2;\n                    self.is_utf8 = true;\n                }\n                Capability::IMAP4rev1 => {\n                    self.version = ProtocolVersion::Rev1;\n                }\n                Capability::CondStore => {\n                    self.is_condstore = true;\n                }\n                Capability::QResync => {\n                    self.is_qresync = true;\n                    self.is_condstore = true;\n                }\n                Capability::Utf8Accept => {\n                    self.is_utf8 = true;\n                }\n                _ => {\n                    continue;\n                }\n            }\n            response.enabled.push(capability);\n        }\n\n        trc::event!(\n            Imap(trc::ImapEvent::Enable),\n            SpanId = self.session_id,\n            Details = response\n                .enabled\n                .iter()\n                .map(|c| trc::Value::from(format!(\"{c:?}\")))\n                .collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_bytes(\n            StatusResponse::ok(\"ENABLE successful.\")\n                .with_tag(arguments.tag)\n                .serialize(response.serialize()),\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/expunge.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapContext, ToModSeq};\nuse crate::core::{ImapId, SavedSearch, SelectedMailbox, Session, SessionData};\nuse ahash::AHashMap;\nuse common::{listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    message::metadata::MessageData,\n};\nuse imap_proto::{\n    Command, ResponseCode, ResponseType, StatusResponse,\n    parser::parse_sequence_set,\n    receiver::{Request, Token},\n};\nuse std::{sync::Arc, time::Instant};\nuse store::{\n    SerializeInfallible,\n    roaring::RoaringBitmap,\n    write::{BatchBuilder, SearchIndex, TaskEpoch, TaskQueueClass, ValueClass},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, VanishedCollection},\n    keyword::Keyword,\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_expunge(\n        &mut self,\n        request: Request<Command>,\n        is_uid: bool,\n    ) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapExpunge)?;\n\n        let op_start = Instant::now();\n        let (data, mailbox) = self.state.select_data();\n\n        // Validate ACL\n        if !data\n            .check_mailbox_acl(\n                mailbox.id.account_id,\n                mailbox.id.mailbox_id,\n                Acl::RemoveItems,\n            )\n            .await\n            .imap_ctx(&request.tag, trc::location!())?\n        {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(concat!(\n                    \"You do not have the required permissions \",\n                    \"to remove messages from this mailbox.\"\n                ))\n                .code(ResponseCode::NoPerm)\n                .id(request.tag));\n        }\n\n        // Parse sequence to operate on\n        let sequence = match request.tokens.into_iter().next() {\n            Some(Token::Argument(value)) if is_uid => {\n                let sequence = parse_sequence_set(&value).map_err(|err| {\n                    trc::ImapEvent::Error\n                        .into_err()\n                        .details(err)\n                        .ctx(trc::Key::Type, ResponseType::Bad)\n                        .id(request.tag.clone())\n                })?;\n                Some(\n                    mailbox\n                        .sequence_to_ids(&sequence, true)\n                        .await\n                        .map_err(|err| err.id(request.tag.clone()))?,\n                )\n            }\n\n            _ => None,\n        };\n\n        // Expunge\n        data.expunge(mailbox.clone(), sequence, op_start)\n            .await\n            .imap_ctx(&request.tag, trc::location!())?;\n\n        // Clear saved searches\n        *mailbox.saved_search.lock() = SavedSearch::None;\n\n        // Synchronize messages\n        let modseq = data\n            .write_mailbox_changes(&mailbox, self.is_qresync)\n            .await\n            .imap_ctx(&request.tag, trc::location!())?;\n        let mut response =\n            StatusResponse::completed(Command::Expunge(is_uid)).with_tag(request.tag);\n\n        if self.is_condstore {\n            response = response.with_code(ResponseCode::HighestModseq {\n                modseq: modseq.to_modseq(),\n            });\n        }\n\n        self.write_bytes(response.into_bytes()).await\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn expunge(\n        &self,\n        mailbox: Arc<SelectedMailbox>,\n        sequence: Option<AHashMap<u32, ImapId>>,\n        op_start: Instant,\n    ) -> trc::Result<()> {\n        // Obtain message ids\n        let account_id = mailbox.id.account_id;\n        let mut deleted_ids = RoaringBitmap::from_iter(\n            self.server\n                .get_cached_messages(account_id)\n                .await\n                .caused_by(trc::location!())?\n                .in_mailbox_with_keyword(mailbox.id.mailbox_id, &Keyword::Deleted)\n                .map(|m| m.document_id),\n        );\n\n        // Filter by sequence\n        if let Some(sequence) = &sequence {\n            deleted_ids &= RoaringBitmap::from_iter(sequence.keys());\n        }\n\n        // Delete ids\n        let mut batch = BatchBuilder::new();\n        self.email_untag_or_delete(account_id, mailbox.id.mailbox_id, &deleted_ids, &mut batch)\n            .await\n            .caused_by(trc::location!())?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::Expunge),\n            SpanId = self.session_id,\n            AccountId = account_id,\n            MailboxId = mailbox.id.mailbox_id,\n            DocumentId = deleted_ids.iter().map(trc::Value::from).collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        // Write changes on source account\n        if !batch.is_empty() {\n            self.server\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?;\n            self.server.notify_task_queue();\n        }\n\n        Ok(())\n    }\n\n    pub async fn email_untag_or_delete(\n        &self,\n        account_id: u32,\n        mailbox_id: u32,\n        deleted_ids: &RoaringBitmap,\n        batch: &mut BatchBuilder,\n    ) -> trc::Result<()> {\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Email);\n\n        self.server\n            .archives(\n                account_id,\n                Collection::Email,\n                deleted_ids,\n                |document_id, data_| {\n                    let metadata = data_\n                        .to_unarchived::<MessageData>()\n                        .caused_by(trc::location!())?;\n\n                    if let Some(message_uid) = metadata.inner.message_uid(mailbox_id) {\n                        // Add vanished items\n                        batch.with_document(document_id);\n                        batch.log_vanished_item(\n                            VanishedCollection::Email,\n                            (mailbox_id, message_uid),\n                        );\n\n                        if metadata.inner.mailboxes.len() == 1 {\n                            // Delete message\n                            batch\n                                .custom(\n                                    ObjectIndexBuilder::<_, ()>::new()\n                                        .with_access_token(&self.access_token)\n                                        .with_current(metadata),\n                                )\n                                .caused_by(trc::location!())?\n                                .set(\n                                    ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                                        index: SearchIndex::Email,\n                                        due: TaskEpoch::now(),\n                                        is_insert: false,\n                                    }),\n                                    0u64.serialize(),\n                                )\n                                .commit_point();\n                        } else {\n                            // Untag message from this mailbox and remove Deleted flag\n                            let mut new_metadata = metadata.inner.to_builder();\n                            new_metadata.remove_mailbox(mailbox_id);\n                            new_metadata.remove_keyword(&Keyword::Deleted);\n\n                            // Write changes\n                            batch\n                                .custom(\n                                    ObjectIndexBuilder::new()\n                                        .with_current(metadata)\n                                        .with_changes(new_metadata.seal()),\n                                )\n                                .caused_by(trc::location!())?\n                                .commit_point();\n                        }\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/fetch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{FromModSeq, ImapContext};\nuse crate::{\n    core::{SelectedMailbox, Session, SessionData},\n    spawn_op,\n};\nuse ahash::AHashMap;\nuse common::{listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    message::metadata::{\n        ArchivedMessageMetadata, ArchivedMessageMetadataContents, ArchivedMetadataHeaderValue,\n        ArchivedMetadataPartType, DecodedParts, MESSAGE_RECEIVED_MASK, MessageData,\n        MessageMetadata, MetadataHeaderName, PART_ENCODING_PROBLEM,\n    },\n};\nuse imap_proto::{\n    Command, ResponseCode, ResponseType, StatusResponse,\n    parser::PushUnique,\n    protocol::{\n        Flag,\n        expunge::Vanished,\n        fetch::{\n            self, Arguments, Attribute, BodyContents, BodyPart, BodyPartExtension, BodyPartFields,\n            DataItem, Envelope, FetchItem, Section,\n        },\n    },\n    receiver::Request,\n};\nuse std::{borrow::Cow, sync::Arc, time::Instant};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse store::{\n    query::log::{Change, Query},\n    rkyv::rend::u16_le,\n    write::BatchBuilder,\n};\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection, VanishedCollection},\n    field::EmailField,\n    id::Id,\n    keyword::Keyword,\n};\nuse utils::chained_bytes::{ChainedBytes, SliceRange};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_fetch(&mut self, requests: Vec<Request<Command>>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapFetch)?;\n\n        let (data, mailbox) = self.state.select_data();\n        let is_qresync = self.is_qresync;\n\n        let mut ops = Vec::with_capacity(requests.len());\n\n        for request in requests {\n            let is_uid = matches!(request.command, Command::Fetch(true));\n            match request.parse_fetch() {\n                Ok(arguments) => {\n                    let enabled_condstore = if !self.is_condstore\n                        && arguments.changed_since.is_some()\n                        || arguments.attributes.contains(&Attribute::ModSeq)\n                    {\n                        self.is_condstore = true;\n                        true\n                    } else {\n                        false\n                    };\n\n                    ops.push(Ok((is_uid, enabled_condstore, arguments)));\n                }\n                Err(err) => {\n                    ops.push(Err(err));\n                }\n            }\n        }\n\n        spawn_op!(data, {\n            for op in ops {\n                match op {\n                    Ok((is_uid, enabled_condstore, arguments)) => {\n                        let response = data\n                            .fetch(\n                                arguments,\n                                mailbox.clone(),\n                                is_uid,\n                                is_qresync,\n                                enabled_condstore,\n                                Instant::now(),\n                            )\n                            .await?;\n\n                        data.write_bytes(response.into_bytes()).await?;\n                    }\n                    Err(err) => data.write_error(err).await?,\n                }\n            }\n\n            Ok(())\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    #[allow(clippy::too_many_arguments)]\n    pub async fn fetch(\n        &self,\n        mut arguments: Arguments,\n        mailbox: Arc<SelectedMailbox>,\n        is_uid: bool,\n        is_qresync: bool,\n        enabled_condstore: bool,\n        op_start: Instant,\n    ) -> trc::Result<StatusResponse> {\n        // Validate VANISHED parameter\n        if arguments.include_vanished {\n            if !is_qresync {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Enable QRESYNC first to use the VANISHED parameter.\")\n                    .ctx(trc::Key::Type, ResponseType::Bad)\n                    .id(arguments.tag));\n            } else if !is_uid {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"VANISHED parameter is only available for UID FETCH.\")\n                    .ctx(trc::Key::Type, ResponseType::Bad)\n                    .id(arguments.tag));\n            }\n        }\n\n        // Resync messages if needed\n        let account_id = mailbox.id.account_id;\n        let mut modseq = self\n            .synchronize_messages(&mailbox)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Convert IMAP ids to JMAP ids.\n        let mut ids = mailbox\n            .sequence_to_ids(&arguments.sequence_set, is_uid)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Convert state to modseq\n        if let Some(changed_since) = arguments.changed_since {\n            // Obtain changes since the modseq.\n            let changelog = self\n                .server\n                .store()\n                .changes(\n                    account_id,\n                    SyncCollection::Email.into(),\n                    Query::from_modseq(changed_since),\n                )\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n\n            // Process changes\n            let mut changed_ids = AHashMap::new();\n            let mut has_vanished = false;\n\n            for change in changelog.changes {\n                match change {\n                    Change::InsertItem(id) | Change::UpdateItem(id) => {\n                        let id = (id & u32::MAX as u64) as u32;\n                        if let Some(uid) = ids.get(&id) {\n                            changed_ids.insert(id, *uid);\n                        }\n                        if !has_vanished {\n                            has_vanished = matches!(change, Change::UpdateItem(_));\n                        }\n                    }\n                    Change::DeleteItem(_) => {\n                        has_vanished = true;\n                    }\n                    _ => (),\n                }\n            }\n\n            // Send vanished UIDs\n            if arguments.include_vanished && has_vanished {\n                // Add to vanished all known destroyed Ids\n                let vanished = self\n                    .server\n                    .store()\n                    .vanished::<(u32, u32)>(\n                        account_id,\n                        VanishedCollection::Email.into(),\n                        Query::from_modseq(changed_since),\n                    )\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?\n                    .into_iter()\n                    .filter_map(|(mailbox_id, uid)| {\n                        if mailbox.id.mailbox_id == mailbox_id {\n                            Some(uid)\n                        } else {\n                            None\n                        }\n                    })\n                    .collect::<Vec<_>>();\n\n                if !vanished.is_empty() {\n                    let mut buf = Vec::with_capacity(vanished.len() * 3);\n                    Vanished {\n                        earlier: true,\n                        ids: vanished,\n                    }\n                    .serialize(&mut buf);\n                    self.write_bytes(buf).await?;\n                }\n            }\n\n            // Filter out ids without changes\n            if changed_ids.is_empty() {\n                // Condstore was just enabled, return highest modseq.\n                if enabled_condstore {\n                    self.write_bytes(\n                        StatusResponse::ok(\"Highest Modseq\")\n                            .with_code(ResponseCode::highest_modseq(modseq))\n                            .into_bytes(),\n                    )\n                    .await?;\n                }\n\n                trc::event!(\n                    Imap(trc::ImapEvent::Fetch),\n                    SpanId = self.session_id,\n                    AccountId = account_id,\n                    MailboxId = mailbox.id.mailbox_id,\n                    Elapsed = op_start.elapsed()\n                );\n\n                return Ok(\n                    StatusResponse::completed(Command::Fetch(is_uid)).with_tag(arguments.tag)\n                );\n            }\n            ids = changed_ids;\n            arguments.attributes.push_unique(Attribute::ModSeq);\n        }\n\n        // Build properties list\n        let mut set_seen_flags = false;\n        let mut needs_blobs = false;\n\n        for attribute in &arguments.attributes {\n            match attribute {\n                Attribute::BodySection { sections, .. }\n                    if sections.first().is_some_and(|s| {\n                        matches!(s, Section::Header | Section::HeaderFields { .. })\n                    }) => {}\n                Attribute::Body | Attribute::BodyStructure | Attribute::BinarySize { .. } => {\n                    /*\n                        Note that this did not result in \\Seen being set, because\n                        RFC822.HEADER response data occurs as a result of a FETCH\n                        of RFC822.HEADER.  BODY[HEADER] response data occurs as a\n                        result of a FETCH of BODY[HEADER] (which sets \\Seen) or\n                        BODY.PEEK[HEADER] (which does not set \\Seen).\n                    */\n                    needs_blobs = true;\n                }\n                Attribute::BodySection { peek, .. } | Attribute::Binary { peek, .. } => {\n                    if mailbox.is_select && !*peek {\n                        set_seen_flags = true;\n                    }\n                    needs_blobs = true;\n                }\n                Attribute::Rfc822Text | Attribute::Rfc822 => {\n                    if mailbox.is_select {\n                        set_seen_flags = true;\n                    }\n                    needs_blobs = true;\n                }\n                _ => (),\n            }\n        }\n\n        if set_seen_flags\n            && !self\n                .check_mailbox_acl(\n                    mailbox.id.account_id,\n                    mailbox.id.mailbox_id,\n                    Acl::ModifyItems,\n                )\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?\n        {\n            set_seen_flags = false;\n        }\n\n        if is_uid {\n            if arguments.attributes.is_empty() {\n                arguments.attributes.push(Attribute::Flags);\n            } else if !arguments.attributes.contains(&Attribute::Uid) {\n                arguments.attributes.insert(0, Attribute::Uid);\n            }\n        }\n\n        // Process each message\n        let mut batch = BatchBuilder::new();\n        let mut ids = ids\n            .into_iter()\n            .map(|(id, imap_id)| (imap_id.seqnum, imap_id.uid, id))\n            .collect::<Vec<_>>();\n        ids.sort_unstable_by_key(|(seqnum, _, _)| *seqnum);\n        let fetched_ids = ids\n            .iter()\n            .map(|id| trc::Value::from(id.2))\n            .collect::<Vec<_>>();\n        let message_cache = self\n            .server\n            .get_cached_messages(account_id)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        for (seqnum, uid, id) in ids {\n            // Obtain attributes and keywords\n            let (metadata_, data) = if let (Some(email), Some(data)) = (\n                self.server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                        account_id,\n                        Collection::Email,\n                        id,\n                        EmailField::Metadata,\n                    ))\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?,\n                message_cache.email_by_id(&id),\n            ) {\n                (email, data)\n            } else {\n                trc::event!(\n                    Store(trc::StoreEvent::NotFound),\n                    AccountId = account_id,\n                    DocumentId = id,\n                    Collection = Collection::Email,\n                    Details = \"Message metadata not found.\",\n                    CausedBy = trc::location!(),\n                );\n                continue;\n            };\n            let metadata = metadata_\n                .unarchive::<MessageMetadata>()\n                .imap_ctx(&arguments.tag, trc::location!())?;\n            let raw_body;\n\n            // Fetch and parse blob\n            let mut raw_message = ChainedBytes::new(metadata.raw_headers.as_ref());\n            if needs_blobs {\n                // Retrieve raw message if needed\n                raw_body = self\n                    .server\n                    .blob_store()\n                    .get_blob(metadata.blob_hash.0.as_slice(), 0..usize::MAX)\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n\n                if let Some(raw_body) = &raw_body {\n                    raw_message.append(\n                        raw_body\n                            .get(metadata.blob_body_offset.to_native() as usize..)\n                            .unwrap_or_default(),\n                    );\n                } else {\n                    trc::event!(\n                        Store(trc::StoreEvent::NotFound),\n                        AccountId = account_id,\n                        DocumentId = id,\n                        Collection = Collection::Email,\n                        BlobId = metadata.blob_hash.0.as_slice(),\n                        Details = \"Blob not found.\",\n                        CausedBy = trc::location!(),\n                    );\n\n                    continue;\n                }\n            }\n\n            let message = &metadata.contents[0];\n            let decoded = metadata.decode_contents(raw_message.clone());\n\n            // Build response\n            let mut items = Vec::with_capacity(arguments.attributes.len());\n            let set_seen_flag = set_seen_flags && !message_cache.has_keyword(data, &Keyword::Seen);\n\n            for attribute in &arguments.attributes {\n                match attribute {\n                    Attribute::Envelope => {\n                        items.push(DataItem::Envelope {\n                            envelope: message.envelope(),\n                        });\n                    }\n                    Attribute::Flags => {\n                        let mut flags = message_cache\n                            .expand_keywords(data)\n                            .map(Flag::from)\n                            .collect::<Vec<_>>();\n                        if set_seen_flag {\n                            flags.push(Flag::Seen);\n                        }\n                        items.push(DataItem::Flags { flags });\n                    }\n                    Attribute::InternalDate => {\n                        items.push(DataItem::InternalDate {\n                            date: (metadata.rcvd_attach.to_native() & MESSAGE_RECEIVED_MASK) as i64,\n                        });\n                    }\n                    Attribute::Preview { .. } => {\n                        items.push(DataItem::Preview {\n                            contents: if !metadata.preview.is_empty() {\n                                Some(metadata.preview.as_bytes().into())\n                            } else {\n                                None\n                            },\n                        });\n                    }\n                    Attribute::Rfc822Size => {\n                        items.push(DataItem::Rfc822Size {\n                            size: data.size as usize,\n                        });\n                    }\n                    Attribute::Uid => {\n                        items.push(DataItem::Uid { uid });\n                    }\n                    Attribute::Rfc822 => {\n                        items.push(DataItem::Rfc822 {\n                            contents: raw_message.get_full_range(),\n                        });\n                    }\n                    Attribute::Rfc822Header => {\n                        let contents = raw_message.get_slice_range(\n                            0..u32::from(metadata.root_part().offset_body) as usize,\n                        );\n\n                        if contents != SliceRange::None {\n                            items.push(DataItem::Rfc822Header { contents });\n                        }\n                    }\n                    Attribute::Rfc822Text => {\n                        items.push(DataItem::Rfc822Text {\n                            contents: raw_message.get_full_range(),\n                        });\n                    }\n                    Attribute::Body => {\n                        items.push(DataItem::Body {\n                            part: metadata.body_structure(&decoded, false),\n                        });\n                    }\n                    Attribute::BodyStructure => {\n                        items.push(DataItem::BodyStructure {\n                            part: metadata.body_structure(&decoded, true),\n                        });\n                    }\n                    Attribute::BodySection {\n                        sections, partial, ..\n                    } => {\n                        if let Some(contents) = metadata.body_section(&decoded, sections, *partial)\n                        {\n                            items.push(DataItem::BodySection {\n                                sections: sections.to_vec(),\n                                origin_octet: partial.map(|(start, _)| start),\n                                contents,\n                            });\n                        }\n                    }\n\n                    Attribute::Binary {\n                        sections, partial, ..\n                    } => match metadata.binary(&decoded, sections, *partial) {\n                        Ok(Some(contents)) => {\n                            items.push(DataItem::Binary {\n                                sections: sections.to_vec(),\n                                offset: partial.map(|(start, _)| start),\n                                contents,\n                            });\n                        }\n                        Err(_) => {\n                            self.write_error(\n                                trc::ImapEvent::Error\n                                    .into_err()\n                                    .details(format!(\n                                        \"Failed to decode part {} of message {}.\",\n                                        sections\n                                            .iter()\n                                            .map(|s| s.to_string())\n                                            .collect::<Vec<_>>()\n                                            .join(\".\"),\n                                        if is_uid { uid } else { seqnum }\n                                    ))\n                                    .code(ResponseCode::UnknownCte),\n                            )\n                            .await?;\n                            continue;\n                        }\n                        _ => (),\n                    },\n                    Attribute::BinarySize { sections } => {\n                        if let Some(size) = metadata.binary_size(&decoded, sections) {\n                            items.push(DataItem::BinarySize {\n                                sections: sections.to_vec(),\n                                size,\n                            });\n                        }\n                    }\n                    Attribute::ModSeq => {\n                        items.push(DataItem::ModSeq {\n                            modseq: data.change_id + 1,\n                        });\n                    }\n                    Attribute::EmailId => {\n                        items.push(DataItem::EmailId {\n                            email_id: Id::from_parts(account_id, id).to_string(),\n                        });\n                    }\n                    Attribute::ThreadId => {\n                        items.push(DataItem::ThreadId {\n                            thread_id: Id::from_parts(account_id, data.thread_id).to_string(),\n                        });\n                    }\n                }\n            }\n\n            // Add flags to the response if the message was unseen\n            if set_seen_flag && !arguments.attributes.contains(&Attribute::Flags) {\n                let mut flags = message_cache\n                    .expand_keywords(data)\n                    .map(Flag::from)\n                    .collect::<Vec<_>>();\n                flags.push(Flag::Seen);\n                items.push(DataItem::Flags { flags });\n            }\n\n            // Serialize fetch item\n            let mut buf = Vec::with_capacity(128);\n            FetchItem { id: seqnum, items }.serialize(&mut buf);\n            self.write_bytes(buf).await?;\n\n            // Add to set flags\n            if set_seen_flag\n                && let Some(data_) = self\n                    .server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::Email,\n                        id,\n                    ))\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?\n            {\n                let data = data_\n                    .to_unarchived::<MessageData>()\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n                let mut new_data = data.inner.to_builder();\n                new_data.keywords.push(Keyword::Seen);\n\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Email)\n                    .with_document(id)\n                    .custom(\n                        ObjectIndexBuilder::new()\n                            .with_current(data)\n                            .with_changes(new_data.seal()),\n                    )\n                    .imap_ctx(&arguments.tag, trc::location!())?\n                    .commit_point();\n            }\n        }\n\n        // Set Seen ids\n        if !batch.is_empty() {\n            match self\n                .server\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .imap_ctx(&arguments.tag, trc::location!())\n            {\n                Ok(change_id) => {\n                    modseq = change_id;\n                }\n                Err(err) => {\n                    if !err.is_assertion_failure() {\n                        return Err(err.id(arguments.tag));\n                    }\n                }\n            }\n        }\n\n        trc::event!(\n            Imap(trc::ImapEvent::Fetch),\n            SpanId = self.session_id,\n            AccountId = account_id,\n            MailboxId = mailbox.id.mailbox_id,\n            DocumentId = fetched_ids,\n            Details = arguments\n                .attributes\n                .iter()\n                .map(|c| trc::Value::from(format!(\"{c:?}\")))\n                .collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        // Condstore was enabled with this command\n        if enabled_condstore {\n            self.write_bytes(\n                StatusResponse::ok(\"Highest Modseq\")\n                    .with_code(ResponseCode::highest_modseq(modseq))\n                    .into_bytes(),\n            )\n            .await?;\n        }\n\n        Ok(StatusResponse::completed(Command::Fetch(is_uid)).with_tag(arguments.tag))\n    }\n}\n\n#[allow(clippy::result_unit_err)]\npub trait AsImapDataItem {\n    fn body_structure(&'_ self, decoded: &DecodedParts<'_>, is_extended: bool) -> BodyPart<'_>;\n    fn body_section<'x>(\n        &self,\n        decoded: &'x DecodedParts<'x>,\n        sections: &[Section],\n        partial: Option<(u32, u32)>,\n    ) -> Option<Cow<'x, [u8]>>;\n    fn binary<'x>(\n        &self,\n        decoded: &'x DecodedParts<'x>,\n        sections: &[u32],\n        partial: Option<(u32, u32)>,\n    ) -> Result<Option<BodyContents<'x>>, ()>;\n    fn binary_size(&self, decoded: &DecodedParts<'_>, sections: &[u32]) -> Option<usize>;\n}\n\n#[allow(clippy::result_unit_err)]\npub trait AsImapDataItemPart {\n    fn as_body_part(\n        &'_ self,\n        decoded: &DecodedParts<'_>,\n        message_id: usize,\n        part_id: usize,\n        is_extended: bool,\n    ) -> BodyPart<'_>;\n\n    fn envelope(&'_ self) -> Envelope<'_>;\n}\n\nimpl AsImapDataItemPart for ArchivedMessageMetadataContents {\n    fn as_body_part(\n        &'_ self,\n        decoded: &DecodedParts<'_>,\n        message_id: usize,\n        part_id: usize,\n        is_extended: bool,\n    ) -> BodyPart<'_> {\n        let part = &self.parts[part_id];\n        let body = decoded.raw_message_section(message_id, part.body_to_end());\n        let (is_multipart, is_text) = match &part.body {\n            ArchivedMetadataPartType::Text | ArchivedMetadataPartType::Html => (false, true),\n            ArchivedMetadataPartType::Multipart(_) => (true, false),\n            _ => (false, false),\n        };\n        let content_type = part\n            .header_value(&MetadataHeaderName::ContentType)\n            .and_then(|ct| ct.as_content_type());\n\n        let mut body_md5 = None;\n        let mut extension = BodyPartExtension::default();\n        let mut fields = BodyPartFields::default();\n\n        if !is_multipart || is_extended {\n            fields.body_parameters = content_type\n                .as_ref()\n                .map(|ct| {\n                    ct.attributes\n                        .iter()\n                        .map(|k| (k.name.as_ref().into(), k.value.as_ref().into()))\n                        .collect::<Vec<_>>()\n                })\n                .filter(|p| !p.is_empty())\n        }\n\n        if !is_multipart {\n            fields.body_subtype = content_type\n                .as_ref()\n                .and_then(|ct| ct.c_subtype.as_ref().map(|cs| cs.as_ref().into()));\n\n            fields.body_id = part\n                .header_value(&MetadataHeaderName::ContentId)\n                .and_then(|id| id.as_text().map(|id| format!(\"<{}>\", id).into()));\n\n            fields.body_description = part\n                .header_value(&MetadataHeaderName::ContentDescription)\n                .and_then(|ct| ct.as_text().map(|ct| ct.into()));\n\n            fields.body_encoding = part\n                .header_value(&MetadataHeaderName::ContentTransferEncoding)\n                .and_then(|ct| ct.as_text().map(|ct| ct.into()));\n\n            fields.body_size_octets = body.as_ref().map(|b| b.len()).unwrap_or(0);\n\n            if is_text {\n                if fields.body_subtype.is_none() {\n                    fields.body_subtype = Some(\"plain\".into());\n                }\n                if fields.body_encoding.is_none() {\n                    fields.body_encoding = Some(\"7bit\".into());\n                }\n                if fields.body_parameters.is_none() {\n                    fields.body_parameters = Some(vec![(\"charset\".into(), \"us-ascii\".into())]);\n                }\n            }\n        }\n\n        if is_extended {\n            if !is_multipart {\n                body_md5 = body\n                    .as_ref()\n                    .map(|b| format!(\"{:x}\", md5::compute(b)).into());\n            }\n\n            extension.body_disposition = part\n                .header_value(&MetadataHeaderName::ContentDisposition)\n                .and_then(|cd| cd.as_content_type())\n                .map(|cd| {\n                    (\n                        cd.c_type.as_ref().into(),\n                        cd.attributes\n                            .iter()\n                            .map(|k| (k.name.as_ref().into(), k.value.as_ref().into()))\n                            .collect::<Vec<_>>(),\n                    )\n                });\n\n            extension.body_language = part\n                .header_value(&MetadataHeaderName::ContentLanguage)\n                .and_then(|hv| {\n                    hv.as_text_list()\n                        .map(|list| list.iter().map(|item| item.as_ref().into()).collect())\n                });\n\n            extension.body_location = part\n                .header_value(&MetadataHeaderName::ContentLocation)\n                .and_then(|ct| ct.as_text().map(|ct| ct.into()));\n        }\n\n        match &part.body {\n            ArchivedMetadataPartType::Multipart(parts) => BodyPart::Multipart {\n                body_parts: Vec::with_capacity(parts.len()),\n                body_subtype: content_type\n                    .as_ref()\n                    .and_then(|ct| ct.c_subtype.as_ref().map(|cs| cs.as_ref().into()))\n                    .unwrap_or_else(|| \"\".into()),\n                body_parameters: fields.body_parameters,\n                extension,\n            },\n            ArchivedMetadataPartType::Message(_) => BodyPart::Message {\n                fields,\n                envelope: None,\n                body: None,\n                body_size_lines: 0,\n                body_md5,\n                extension,\n            },\n            _ => {\n                if is_text {\n                    BodyPart::Text {\n                        fields,\n                        body_size_lines: body\n                            .as_ref()\n                            .map(|b| b.iter().filter(|&&ch| ch == b'\\n').count())\n                            .unwrap_or(0),\n                        body_md5,\n                        extension,\n                    }\n                } else {\n                    BodyPart::Basic {\n                        body_type: content_type\n                            .as_ref()\n                            .map(|ct| Cow::from(ct.c_type.as_ref())),\n                        fields,\n                        body_md5,\n                        extension,\n                    }\n                }\n            }\n        }\n    }\n\n    fn envelope(&'_ self) -> Envelope<'_> {\n        let headers = self.root_part();\n        Envelope {\n            date: headers.date(),\n            subject: headers.subject().map(|s| s.into()),\n            from: headers\n                .header_values(&MetadataHeaderName::From)\n                .flat_map(|a| a.as_imap_address())\n                .collect(),\n            sender: headers\n                .header_values(&MetadataHeaderName::Sender)\n                .flat_map(|a| a.as_imap_address())\n                .collect(),\n            reply_to: headers\n                .header_values(&MetadataHeaderName::ReplyTo)\n                .flat_map(|a| a.as_imap_address())\n                .collect(),\n            to: headers\n                .header_values(&MetadataHeaderName::To)\n                .flat_map(|a| a.as_imap_address())\n                .collect(),\n            cc: headers\n                .header_values(&MetadataHeaderName::Cc)\n                .flat_map(|a| a.as_imap_address())\n                .collect(),\n            bcc: headers\n                .header_values(&MetadataHeaderName::Bcc)\n                .flat_map(|a| a.as_imap_address())\n                .collect(),\n            in_reply_to: headers.in_reply_to().as_text_list().map(|list| {\n                let mut irt = String::with_capacity(list.len() * 10);\n                for (pos, l) in list.iter().enumerate() {\n                    if pos > 0 {\n                        irt.push(' ');\n                    }\n                    irt.push('<');\n                    irt.push_str(l.as_ref());\n                    irt.push('>');\n                }\n                irt.into()\n            }),\n            message_id: headers.message_id().map(|id| format!(\"<{}>\", id).into()),\n        }\n    }\n}\n\nimpl AsImapDataItem for ArchivedMessageMetadata {\n    fn body_structure(&'_ self, decoded: &DecodedParts<'_>, is_extended: bool) -> BodyPart<'_> {\n        let mut stack = Vec::new();\n        let base_part = [u16_le::from_native(0)];\n        let mut parts = base_part.as_slice().iter();\n        let mut message = &self.contents[0];\n        let mut root_part = None;\n        let mut message_id = 0;\n\n        loop {\n            while let Some(part_id) = parts.next() {\n                let part_id = u16::from(part_id) as usize;\n                let mut part = message.as_body_part(decoded, message_id, part_id, is_extended);\n\n                match &message.parts[part_id].body {\n                    ArchivedMetadataPartType::Message(nested_message_id) => {\n                        let nested_message = self.message_id(*nested_message_id);\n                        part.set_envelope(nested_message.envelope());\n                        if let Some(root_part) = root_part {\n                            if stack.len() == 10_000 {\n                                debug_assert!(false, \"Too much nesting in message metadata\");\n                                return root_part;\n                            }\n                            stack.push((root_part, parts, (message, message_id).into()));\n                        }\n                        root_part = part.into();\n                        parts = base_part.as_slice().iter();\n                        message = nested_message;\n                        message_id = u16::from(*nested_message_id) as usize;\n                        continue;\n                    }\n                    ArchivedMetadataPartType::Multipart(subparts) => {\n                        if let Some(root_part) = root_part {\n                            if stack.len() == 10_000 {\n                                debug_assert!(false, \"Too much nesting in message metadata\");\n                                return root_part;\n                            }\n                            stack.push((root_part, parts, None));\n                        }\n                        root_part = part.into();\n                        parts = subparts.iter();\n                        continue;\n                    }\n                    _ => (),\n                }\n                if let Some(root_part) = &mut root_part {\n                    root_part.add_part(part);\n                } else {\n                    return part;\n                }\n            }\n            if let Some((mut prev_root_part, prev_parts, prev_message)) = stack.pop() {\n                if let Some((prev_message, prev_message_id)) = prev_message {\n                    message = prev_message;\n                    message_id = prev_message_id;\n                }\n\n                prev_root_part.add_part(root_part.unwrap());\n                parts = prev_parts;\n                root_part = prev_root_part.into();\n            } else {\n                break;\n            }\n        }\n\n        root_part.unwrap()\n    }\n\n    fn body_section<'x>(\n        &self,\n        decoded: &'x DecodedParts<'x>,\n        sections: &[Section],\n        partial: Option<(u32, u32)>,\n    ) -> Option<Cow<'x, [u8]>> {\n        let mut part = self.root_part();\n        if sections.is_empty() {\n            return Some(get_cow_partial_bytes(\n                decoded.raw_message_section(0, part.header_to_end())?,\n                partial,\n            ));\n        }\n\n        let mut message = &self.contents[0];\n        let mut message_id = 0;\n        let mut sections_iter = sections.iter().enumerate().peekable();\n\n        while let Some((section_num, section)) = sections_iter.next() {\n            match section {\n                Section::Part { num } => {\n                    part = if let Some(sub_part_ids) = part.sub_parts() {\n                        sub_part_ids\n                            .as_ref()\n                            .get((*num).saturating_sub(1) as usize)\n                            .and_then(|pos| message.parts.as_ref().get(u16::from(*pos) as usize))\n                    } else if *num == 1 && (section_num == sections.len() - 1 || part.is_message())\n                    {\n                        Some(part)\n                    } else {\n                        None\n                    }?;\n\n                    if let ArchivedMetadataPartType::Message(nested_message_id) = &part.body\n                        && let Some((\n                            _,\n                            Section::Part { .. }\n                            | Section::Header\n                            | Section::HeaderFields { .. }\n                            | Section::Text,\n                        )) = sections_iter.peek()\n                    {\n                        message = self.message_id(*nested_message_id);\n                        part = message.root_part();\n                        message_id = u16::from(nested_message_id) as usize;\n                    }\n                }\n                Section::Header => {\n                    return Some(get_cow_partial_bytes(\n                        decoded.raw_message_section(message_id, part.header_to_body())?,\n                        partial,\n                    ));\n                }\n                Section::HeaderFields { not, fields } => {\n                    let mut headers = Vec::with_capacity(\n                        u32::from(part.offset_body).saturating_sub(u32::from(part.offset_header))\n                            as usize,\n                    );\n                    for header in part.headers.iter() {\n                        let header_name = header.name.as_str();\n                        if fields.iter().any(|f| header_name.eq_ignore_ascii_case(f)) != *not {\n                            headers.extend_from_slice(header_name.as_bytes());\n                            headers.push(b':');\n                            headers.extend_from_slice(\n                                &decoded\n                                    .raw_message_section(message_id, header.value_range())\n                                    .unwrap_or_default(),\n                            );\n                        }\n                    }\n\n                    headers.extend_from_slice(b\"\\r\\n\");\n\n                    return Some(if partial.is_none() {\n                        headers.into()\n                    } else {\n                        get_partial_bytes(&headers, partial).to_vec().into()\n                    });\n                }\n                Section::Text => {\n                    return Some(get_cow_partial_bytes(\n                        decoded.raw_message_section(message_id, part.body_to_end())?,\n                        partial,\n                    ));\n                }\n                Section::Mime => {\n                    let mut headers = Vec::with_capacity(\n                        u32::from(part.offset_body).saturating_sub(u32::from(part.offset_header))\n                            as usize,\n                    );\n                    for header in part.headers.iter() {\n                        if header.name.is_mime_header()\n                            || header.name.as_str().starts_with(\"Content-\")\n                        {\n                            headers.extend_from_slice(header.name.as_str().as_bytes());\n                            headers.extend_from_slice(b\":\");\n                            headers.extend_from_slice(\n                                &decoded\n                                    .raw_message_section(message_id, header.value_range())\n                                    .unwrap_or_default(),\n                            );\n                        }\n                    }\n                    headers.extend_from_slice(b\"\\r\\n\");\n                    return Some(if partial.is_none() {\n                        headers.into()\n                    } else {\n                        get_partial_bytes(&headers, partial).to_vec().into()\n                    });\n                }\n            }\n        }\n\n        // BODY[x] should return both headers and body, but most clients\n        // expect BODY[x] to return only the body, just like BOXY[x.TEXT] does.\n\n        Some(get_cow_partial_bytes(\n            decoded.raw_message_section(message_id, part.body_to_end())?,\n            partial,\n        ))\n    }\n\n    fn binary<'x>(\n        &self,\n        decoded: &'x DecodedParts<'x>,\n        sections: &[u32],\n        partial: Option<(u32, u32)>,\n    ) -> Result<Option<BodyContents<'x>>, ()> {\n        let mut message = &self.contents[0];\n        let mut message_id = 0;\n        let mut part = self.root_part();\n        let mut sections_iter = sections.iter().enumerate().peekable();\n\n        while let Some((section_num, num)) = sections_iter.next() {\n            part = if let Some(sub_part_ids) = part.sub_parts() {\n                if let Some(part) = sub_part_ids\n                    .as_ref()\n                    .get((*num).saturating_sub(1) as usize)\n                    .and_then(|pos| message.parts.as_ref().get(u16::from(*pos) as usize))\n                {\n                    part\n                } else {\n                    return Ok(None);\n                }\n            } else if *num == 1 && (section_num == sections.len() - 1 || part.is_message()) {\n                part\n            } else {\n                return Ok(None);\n            };\n\n            if let (ArchivedMetadataPartType::Message(nested_message), Some(_)) =\n                (&part.body, sections_iter.peek())\n            {\n                message = self.message_id(*nested_message);\n                part = message.root_part();\n                message_id = u16::from(nested_message) as usize;\n            }\n        }\n\n        if (part.flags & PART_ENCODING_PROBLEM) == 0 {\n            let part_offset = u32::from(part.offset_header) as usize;\n            Ok(match &part.body {\n                ArchivedMetadataPartType::Text | ArchivedMetadataPartType::Html => {\n                    BodyContents::Text(String::from_utf8_lossy(get_partial_bytes(\n                        decoded\n                            .binary_part(message_id, part_offset)\n                            .unwrap_or_default(),\n                        partial,\n                    )))\n                    .into()\n                }\n                ArchivedMetadataPartType::Binary | ArchivedMetadataPartType::InlineBinary => {\n                    BodyContents::Bytes(\n                        get_partial_bytes(\n                            decoded\n                                .binary_part(message_id, part_offset)\n                                .unwrap_or_default(),\n                            partial,\n                        )\n                        .into(),\n                    )\n                    .into()\n                }\n                ArchivedMetadataPartType::Message(message) => BodyContents::Bytes({\n                    {\n                        let part = self.message_id(*message).root_part();\n                        get_cow_partial_bytes(\n                            decoded\n                                .raw_message_section(message_id, part.header_to_end())\n                                .unwrap_or_default(),\n                            partial,\n                        )\n                    }\n                })\n                .into(),\n                ArchivedMetadataPartType::Multipart(_) => {\n                    BodyContents::Bytes(get_cow_partial_bytes(\n                        decoded\n                            .raw_message_section(message_id, part.header_to_end())\n                            .unwrap_or_default(),\n                        partial,\n                    ))\n                    .into()\n                }\n            })\n        } else {\n            Err(())\n        }\n    }\n\n    fn binary_size(&self, decoded: &DecodedParts<'_>, sections: &[u32]) -> Option<usize> {\n        let mut message = &self.contents[0];\n        let mut message_id = 0;\n        let mut part = self.root_part();\n        let mut sections_iter = sections.iter().enumerate().peekable();\n\n        while let Some((section_num, num)) = sections_iter.next() {\n            part = if let Some(sub_part_ids) = part.sub_parts() {\n                sub_part_ids\n                    .as_ref()\n                    .get((*num).saturating_sub(1) as usize)\n                    .and_then(|pos| message.parts.as_ref().get(u16::from(pos) as usize))\n            } else if *num == 1 && (section_num == sections.len() - 1 || part.is_message()) {\n                Some(part)\n            } else {\n                None\n            }?;\n\n            if let (ArchivedMetadataPartType::Message(nested_message), Some(_)) =\n                (&part.body, sections_iter.peek())\n            {\n                message = self.message_id(*nested_message);\n                message_id = u16::from(nested_message) as usize;\n                part = message.root_part();\n            }\n        }\n\n        match &part.body {\n            ArchivedMetadataPartType::Text\n            | ArchivedMetadataPartType::Html\n            | ArchivedMetadataPartType::Binary\n            | ArchivedMetadataPartType::InlineBinary => decoded\n                .part(message_id, u32::from(part.offset_header) as usize)\n                .map(|p| p.len())\n                .unwrap_or_default(),\n            ArchivedMetadataPartType::Message(message) => {\n                self.message_id(*message).root_part().raw_len()\n            }\n            ArchivedMetadataPartType::Multipart(_) => part.raw_len(),\n        }\n        .into()\n    }\n}\n\n#[inline(always)]\nfn get_partial_bytes(bytes: &[u8], partial: Option<(u32, u32)>) -> &[u8] {\n    if let Some((start, end)) = partial {\n        bytes\n            .get(start as usize..std::cmp::min((start + end) as usize, bytes.len()))\n            .unwrap_or_default()\n    } else {\n        bytes\n    }\n}\n\n#[inline(always)]\nfn get_cow_partial_bytes(bytes: Cow<'_, [u8]>, partial: Option<(u32, u32)>) -> Cow<'_, [u8]> {\n    if let Some((start, end)) = partial {\n        let range = start as usize..std::cmp::min((start + end) as usize, bytes.len());\n        match bytes {\n            Cow::Borrowed(bytes) => Cow::Borrowed(bytes.get(range).unwrap_or_default()),\n            Cow::Owned(bytes) => Cow::Owned(bytes.get(range).unwrap_or_default().to_vec()),\n        }\n    } else {\n        bytes\n    }\n}\n\ntrait AsImapAddress {\n    fn as_imap_address(&'_ self) -> Vec<fetch::Address<'_>>;\n}\n\nimpl AsImapAddress for ArchivedMetadataHeaderValue {\n    fn as_imap_address(&'_ self) -> Vec<fetch::Address<'_>> {\n        let mut addresses = Vec::new();\n\n        match self {\n            ArchivedMetadataHeaderValue::AddressList(list) => {\n                for addr in list.iter() {\n                    if let Some(email) = addr.address.as_ref() {\n                        addresses.push(fetch::Address::Single(fetch::EmailAddress {\n                            name: addr.name.as_ref().map(|n| n.as_ref().into()),\n                            address: email.as_ref().into(),\n                        }));\n                    }\n                }\n            }\n            ArchivedMetadataHeaderValue::AddressGroup(list) => {\n                for group in list.iter() {\n                    addresses.push(fetch::Address::Group(fetch::AddressGroup {\n                        name: group.name.as_ref().map(|n| n.as_ref().into()),\n                        addresses: group\n                            .addresses\n                            .iter()\n                            .filter_map(|addr| {\n                                fetch::EmailAddress {\n                                    name: addr.name.as_ref().map(|n| n.as_ref().into()),\n                                    address: addr.address.as_ref()?.as_ref().into(),\n                                }\n                                .into()\n                            })\n                            .collect(),\n                    }));\n                }\n            }\n            _ => (),\n        }\n\n        addresses\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/idle.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{SelectedMailbox, Session, SessionData, State},\n    op::ImapContext,\n};\nuse ahash::AHashSet;\nuse common::{ipc::PushNotification, listener::SessionStream};\nuse directory::Permission;\nuse imap_proto::{\n    Command, StatusResponse,\n    protocol::{\n        Sequence, fetch,\n        list::{Attribute, ListItem},\n        status::Status,\n    },\n    receiver::Request,\n};\nuse std::{sync::Arc, time::Instant};\nuse store::query::log::Query;\nuse tokio::io::AsyncReadExt;\nuse trc::AddContext;\nuse types::{collection::SyncCollection, type_state::DataType};\nuse utils::map::bitmap::Bitmap;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_idle(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapIdle)?;\n\n        let op_start = Instant::now();\n        let (data, mailbox, types) = match &self.state {\n            State::Authenticated { data, .. } => {\n                (data.clone(), None, Bitmap::from_iter([DataType::Mailbox]))\n            }\n            State::Selected { data, mailbox, .. } => (\n                data.clone(),\n                mailbox.clone().into(),\n                Bitmap::from_iter([DataType::Email, DataType::Mailbox, DataType::EmailDelivery]),\n            ),\n            _ => unreachable!(),\n        };\n        let is_rev2 = self.version.is_rev2();\n        let is_utf8 = self.is_utf8;\n        let is_qresync = self.is_qresync;\n\n        // Register with push manager\n        let mut push_rx = self\n            .server\n            .subscribe_push_manager(&data.access_token, types)\n            .await\n            .imap_ctx(&request.tag, trc::location!())?;\n\n        // Send continuation response\n        self.write_bytes(b\"+ Idling, send 'DONE' to stop.\\r\\n\".to_vec())\n            .await?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::IdleStart),\n            SpanId = self.session_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        let op_start = Instant::now();\n        let mut buf = vec![0; 4];\n        loop {\n            tokio::select! {\n                result = tokio::time::timeout(self.server.core.imap.timeout_idle, self.stream_rx.read_exact(&mut buf)) => {\n                    match result {\n                        Ok(Ok(bytes_read)) => {\n                            if bytes_read > 0 {\n                                if (buf[..bytes_read]).windows(4).any(|w| w == b\"DONE\") {\n                                    trc::event!(Imap(trc::ImapEvent::IdleStop), SpanId = self.session_id, Elapsed = op_start.elapsed());\n                                    return self.write_bytes(StatusResponse::completed(Command::Idle)\n                                                                    .with_tag(request.tag)\n                                                                    .into_bytes()).await;\n                                }\n                            } else {\n                                return Err(trc::NetworkEvent::Closed.into_err().details(\"IMAP connection closed by client.\").id(request.tag));\n                            }\n                        },\n                        Ok(Err(err)) => {\n                            return Err(trc::NetworkEvent::ReadError.into_err().reason(err).details(\"IMAP connection error.\").id(request.tag));\n                        },\n                        Err(_) => {\n                            self.write_bytes(&b\"* BYE IDLE timed out.\\r\\n\"[..]).await.ok();\n                            return Err(trc::NetworkEvent::Timeout.into_err().details(\"IMAP IDLE timed out.\").id(request.tag));\n                        }\n                    }\n                }\n                push_notification = push_rx.recv() => {\n                    if let Some(push_notification) = push_notification {\n                        let mut has_mailbox_changes = false;\n                        let mut has_email_changes = false;\n\n                        match push_notification {\n                            PushNotification::StateChange(state_change) => {\n                                for type_state in state_change.types {\n                                    match type_state {\n                                        DataType::Email | DataType::EmailDelivery => {\n                                            has_email_changes = true;\n                                        }\n                                        DataType::Mailbox => {\n                                            has_mailbox_changes = true;\n                                        }\n                                        _ => {}\n                                    }\n                                }\n                            },\n                            PushNotification::EmailPush(_) => {\n                                has_email_changes = true;\n                                has_mailbox_changes = true;\n                            },\n                            PushNotification::CalendarAlert(_) => (),\n                        }\n\n                        if has_mailbox_changes || has_email_changes {\n                            data.write_changes(&mailbox, has_mailbox_changes, has_email_changes, is_qresync, is_rev2, is_utf8).await?;\n                        }\n                    } else {\n                        self.write_bytes(&b\"* BYE Server shutting down.\\r\\n\"[..]).await.ok();\n                        return Err(trc::NetworkEvent::Closed.into_err().details(\"IDLE channel closed.\").id(request.tag));\n                    }\n                }\n            }\n        }\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn write_changes(\n        &self,\n        mailbox: &Option<Arc<SelectedMailbox>>,\n        check_mailboxes: bool,\n        check_emails: bool,\n        is_qresync: bool,\n        is_rev2: bool,\n        is_utf8: bool,\n    ) -> trc::Result<()> {\n        // Fetch all changed mailboxes\n        if check_mailboxes {\n            let changes = self\n                .synchronize_mailboxes(true)\n                .await\n                .caused_by(trc::location!())?\n                .unwrap();\n\n            let mut buf = Vec::with_capacity(64);\n\n            // List deleted mailboxes\n            for mailbox_name in changes.deleted {\n                ListItem {\n                    mailbox_name,\n                    attributes: vec![Attribute::NonExistent],\n                    tags: vec![],\n                }\n                .serialize(&mut buf, is_rev2, is_utf8, false);\n            }\n\n            // List added mailboxes\n            for mailbox_name in changes.added {\n                ListItem {\n                    mailbox_name,\n                    attributes: vec![],\n                    tags: vec![],\n                }\n                .serialize(&mut buf, is_rev2, is_utf8, false);\n            }\n            // Obtain status of changed mailboxes\n            for mailbox_name in changes.changed {\n                if let Ok(status) = self\n                    .status(\n                        mailbox_name,\n                        &[\n                            Status::Messages,\n                            Status::Unseen,\n                            Status::UidNext,\n                            Status::UidValidity,\n                        ],\n                    )\n                    .await\n                {\n                    status.serialize(&mut buf, is_utf8);\n                }\n            }\n\n            if !buf.is_empty() {\n                self.write_bytes(buf).await?;\n            }\n        }\n\n        // Fetch selected mailbox changes\n        if check_emails {\n            // Synchronize emails\n            if let Some(mailbox) = mailbox {\n                // Obtain changes since last sync\n                let modseq = mailbox.state.lock().modseq;\n                let new_state = self\n                    .write_mailbox_changes(mailbox, is_qresync)\n                    .await\n                    .caused_by(trc::location!())?;\n                if new_state == modseq {\n                    return Ok(());\n                }\n\n                // Obtain changed messages\n                let changelog = self\n                    .server\n                    .store()\n                    .changes(\n                        mailbox.id.account_id,\n                        SyncCollection::Email.into(),\n                        Query::Since(modseq),\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n                let changed_ids = {\n                    let state = mailbox.state.lock();\n                    changelog\n                        .changes\n                        .into_iter()\n                        .filter_map(|change| {\n                            change.try_unwrap_item_id().and_then(|item_id| {\n                                state\n                                    .id_to_imap\n                                    .get(&((item_id & u32::MAX as u64) as u32))\n                                    .map(|id| id.uid)\n                            })\n                        })\n                        .collect::<AHashSet<_>>()\n                };\n\n                if !changed_ids.is_empty() {\n                    let op_start = Instant::now();\n                    return self\n                        .fetch(\n                            fetch::Arguments {\n                                tag: \"\".into(),\n                                sequence_set: Sequence::List {\n                                    items: changed_ids\n                                        .into_iter()\n                                        .map(|uid| Sequence::Number { value: uid })\n                                        .collect(),\n                                },\n                                attributes: vec![fetch::Attribute::Flags, fetch::Attribute::Uid],\n                                changed_since: None,\n                                include_vanished: false,\n                            },\n                            mailbox.clone(),\n                            true,\n                            is_qresync,\n                            false,\n                            op_start,\n                        )\n                        .await\n                        .caused_by(trc::location!())\n                        .map(|_| ());\n                }\n            }\n        }\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/list.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse crate::{\n    core::{Session, SessionData},\n    spawn_op,\n};\nuse common::listener::SessionStream;\n\nuse directory::Permission;\nuse imap_proto::{\n    Command, StatusResponse,\n    protocol::{\n        ImapResponse, ProtocolVersion,\n        list::{\n            self, Arguments, Attribute, ChildInfo, ListItem, ReturnOption, SelectionOption, Tag,\n        },\n    },\n    receiver::Request,\n};\nuse trc::StoreEvent;\n\nuse super::ImapContext;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_list(&mut self, request: Request<Command>) -> trc::Result<()> {\n        let op_start = Instant::now();\n        let command = request.command;\n        let is_lsub = command == Command::Lsub;\n        let arguments = if !is_lsub {\n            // Validate access\n            self.assert_has_permission(Permission::ImapList)?;\n\n            request.parse_list(self.is_utf8)\n        } else {\n            // Validate access\n            self.assert_has_permission(Permission::ImapLsub)?;\n\n            request.parse_lsub(self.is_utf8)\n        }?;\n\n        if !arguments.is_separator_query() {\n            let data = self.state.session_data();\n            let version = self.version;\n            let is_utf8 = self.is_utf8;\n\n            spawn_op!(\n                data,\n                data.list(arguments, is_lsub, version, is_utf8, op_start)\n                    .await\n            )\n        } else {\n            self.write_bytes(\n                StatusResponse::completed(command)\n                    .with_tag(arguments.unwrap_tag())\n                    .serialize(\n                        list::Response {\n                            is_rev2: self.version.is_rev2(),\n                            is_utf8: self.is_utf8,\n                            is_lsub,\n                            list_items: vec![ListItem {\n                                mailbox_name: \"\".into(),\n                                attributes: vec![Attribute::NoSelect],\n                                tags: vec![],\n                            }],\n                            status_items: Vec::new(),\n                        }\n                        .serialize(),\n                    ),\n            )\n            .await\n        }\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn list(\n        &self,\n        arguments: Arguments,\n        is_lsub: bool,\n        version: ProtocolVersion,\n        is_utf8: bool,\n        op_start: Instant,\n    ) -> trc::Result<()> {\n        let (tag, reference_name, mut patterns, selection_options, return_options) = match arguments\n        {\n            Arguments::Basic {\n                tag,\n                reference_name,\n                mailbox_name,\n            } => (\n                tag,\n                reference_name,\n                vec![mailbox_name],\n                Vec::new(),\n                Vec::new(),\n            ),\n            Arguments::Extended {\n                tag,\n                reference_name,\n                mailbox_name,\n                selection_options,\n                return_options,\n            } => (\n                tag,\n                reference_name,\n                mailbox_name,\n                selection_options,\n                return_options,\n            ),\n        };\n\n        // Refresh mailboxes\n        self.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&tag, trc::location!())?;\n\n        // Process arguments\n        let mut filter_subscribed = false;\n        let mut filter_special_use = false;\n        let mut recursive_match = false;\n        let mut include_special_use = true;\n        let mut include_subscribed = false;\n        let mut include_children = false;\n        let mut include_status = None;\n        for selection_option in &selection_options {\n            match selection_option {\n                SelectionOption::Subscribed => {\n                    filter_subscribed = true;\n                    include_subscribed = true;\n                }\n                SelectionOption::Remote => (),\n                SelectionOption::SpecialUse => {\n                    filter_special_use = true;\n                    include_special_use = true;\n                }\n                SelectionOption::RecursiveMatch => {\n                    recursive_match = true;\n                }\n            }\n        }\n        for return_option in &return_options {\n            match return_option {\n                ReturnOption::Subscribed => {\n                    include_subscribed = true;\n                }\n                ReturnOption::Children => {\n                    include_children = true;\n                }\n                ReturnOption::Status(status) => {\n                    include_status = status.into();\n                }\n                ReturnOption::SpecialUse => {\n                    include_special_use = true;\n                }\n            }\n        }\n        if recursive_match && !filter_subscribed {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\"RECURSIVEMATCH requires the SUBSCRIBED selection option.\")\n                .id(tag));\n        }\n\n        // Append reference name\n        if !patterns.is_empty() && !reference_name.is_empty() {\n            patterns.iter_mut().for_each(|item| {\n                *item = format!(\"{}{}\", reference_name, item);\n            })\n        }\n\n        let mut list_items = Vec::with_capacity(10);\n\n        // Add mailboxes\n        let mut added_shared_folder = false;\n        for account in self.mailboxes.lock().iter() {\n            if let Some(prefix) = &account.prefix {\n                if !added_shared_folder {\n                    if !filter_subscribed\n                        && matches_pattern(&patterns, &self.server.core.jmap.shared_folder)\n                    {\n                        list_items.push(ListItem {\n                            mailbox_name: self.server.core.jmap.shared_folder.as_str().into(),\n                            attributes: if include_children {\n                                vec![Attribute::HasChildren, Attribute::NoSelect]\n                            } else {\n                                vec![Attribute::NoSelect]\n                            },\n                            tags: vec![],\n                        });\n                    }\n                    added_shared_folder = true;\n                }\n                if !filter_subscribed && matches_pattern(&patterns, prefix) {\n                    list_items.push(ListItem {\n                        mailbox_name: prefix.clone(),\n                        attributes: if include_children {\n                            vec![Attribute::HasChildren, Attribute::NoSelect]\n                        } else {\n                            vec![Attribute::NoSelect]\n                        },\n                        tags: vec![],\n                    });\n                }\n            }\n\n            for (mailbox_name, mailbox_id) in &account.mailbox_names {\n                if matches_pattern(&patterns, mailbox_name) {\n                    let mailbox = if let Some(mailbox) = account.mailbox_state.get(mailbox_id) {\n                        mailbox\n                    } else {\n                        trc::event!(\n                            Store(StoreEvent::UnexpectedError),\n                            Details = \"IMAP mailbox no longer present in account state\",\n                            Id = *mailbox_id,\n                            Details = account\n                                .mailbox_state\n                                .keys()\n                                .copied()\n                                .map(trc::Value::from)\n                                .collect::<Vec<_>>()\n                        );\n                        continue;\n                    };\n                    let mut has_recursive_match = false;\n                    if recursive_match {\n                        let prefix = format!(\"{}/\", mailbox_name);\n                        for (mailbox_name, mailbox_id) in &account.mailbox_names {\n                            if mailbox_name.starts_with(&prefix)\n                                && account.mailbox_state.get(mailbox_id).unwrap().is_subscribed\n                            {\n                                has_recursive_match = true;\n                                break;\n                            }\n                        }\n                    }\n                    if !filter_subscribed || mailbox.is_subscribed || has_recursive_match {\n                        let mut attributes = Vec::with_capacity(2);\n                        if include_children {\n                            attributes.push(if mailbox.has_children {\n                                Attribute::HasChildren\n                            } else {\n                                Attribute::HasNoChildren\n                            });\n                        }\n                        if include_subscribed && mailbox.is_subscribed {\n                            attributes.push(Attribute::Subscribed);\n                        }\n                        if include_special_use {\n                            if let Some(special_use) = &mailbox.special_use {\n                                attributes.push(*special_use);\n                            } else if filter_special_use {\n                                continue;\n                            }\n                        }\n                        list_items.push(ListItem {\n                            mailbox_name: mailbox_name.clone(),\n                            attributes,\n                            tags: if !has_recursive_match {\n                                vec![]\n                            } else {\n                                vec![Tag::ChildInfo(vec![ChildInfo::Subscribed])]\n                            },\n                        });\n                    }\n                }\n            }\n        }\n\n        // Add status response\n        let mut status_items = Vec::new();\n        if let Some(include_status) = include_status {\n            for list_item in &list_items {\n                match self\n                    .status(list_item.mailbox_name.clone(), include_status)\n                    .await\n                    .imap_ctx(&tag, trc::location!())\n                {\n                    Ok(status_item) => {\n                        status_items.push(status_item);\n                    }\n                    Err(err) => {\n                        self.write_error(err).await?;\n                    }\n                }\n            }\n        }\n\n        trc::event!(\n            Imap(if !is_lsub {\n                trc::ImapEvent::List\n            } else {\n                trc::ImapEvent::Lsub\n            }),\n            SpanId = self.session_id,\n            Details = list_items\n                .iter()\n                .map(|item| trc::Value::from(item.mailbox_name.clone()))\n                .collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        // Write response\n        self.write_bytes(\n            StatusResponse::completed(if !is_lsub {\n                Command::List\n            } else {\n                Command::Lsub\n            })\n            .with_tag(tag)\n            .serialize(\n                list::Response {\n                    is_rev2: version.is_rev2(),\n                    is_utf8,\n                    is_lsub,\n                    list_items,\n                    status_items,\n                }\n                .serialize(),\n            ),\n        )\n        .await\n    }\n}\n\n#[allow(clippy::while_let_on_iterator)]\npub fn matches_pattern(patterns: &[String], mailbox_name: &str) -> bool {\n    if patterns.is_empty() {\n        return true;\n    }\n\n    'outer: for pattern in patterns {\n        let mut pattern_bytes = pattern.as_bytes().iter().enumerate().peekable();\n        let mut mailbox_name = mailbox_name.as_bytes().iter().peekable();\n\n        'inner: while let Some((pos, &ch)) = pattern_bytes.next() {\n            if ch == b'%' || ch == b'*' {\n                let mut end_pos = pos;\n                while let Some(&(_, &next_ch)) = pattern_bytes.peek() {\n                    if next_ch == b'%' || next_ch == b'*' {\n                        break;\n                    } else {\n                        end_pos = pattern_bytes.next().unwrap().0;\n                    }\n                }\n                if end_pos > pos {\n                    let match_bytes = &pattern.as_bytes()[pos + 1..end_pos + 1];\n                    let mut match_count = 0;\n                    let pattern_eof = end_pos == pattern.len() - 1;\n\n                    loop {\n                        match mailbox_name.next() {\n                            Some(&ch) => {\n                                if match_bytes[match_count] == ch {\n                                    match_count += 1;\n                                    if match_count == match_bytes.len() {\n                                        if !pattern_eof {\n                                            continue 'inner;\n                                        } else if mailbox_name.peek().is_none() {\n                                            return true;\n                                        } else {\n                                            // Match needs to be at the end of the string,\n                                            // reset counter.\n                                            match_count = 0;\n                                        }\n                                    }\n                                } else if match_count > 0 {\n                                    match_count = 0;\n                                }\n                            }\n                            None => continue 'outer,\n                        }\n                    }\n                } else if ch == b'*' || !mailbox_name.any(|&ch| ch == b'/') {\n                    return true;\n                } else {\n                    continue 'outer;\n                }\n            } else {\n                match mailbox_name.next() {\n                    Some(&mch) if mch == ch => (),\n                    _ => continue 'outer,\n                }\n            }\n        }\n\n        if mailbox_name.next().is_none() {\n            return true;\n        }\n    }\n\n    false\n}\n"
  },
  {
    "path": "crates/imap/src/op/login.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap_proto::{Command, receiver::Request};\n\nuse crate::core::Session;\nuse common::listener::SessionStream;\nuse mail_send::Credentials;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_login(&mut self, request: Request<Command>) -> trc::Result<()> {\n        let arguments = request.parse_login()?;\n\n        self.authenticate(\n            Credentials::Plain {\n                username: arguments.username.to_string(),\n                secret: arguments.password.to_string(),\n            },\n            arguments.tag,\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/logout.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse crate::core::Session;\nuse common::listener::SessionStream;\nuse imap_proto::{Command, StatusResponse, receiver::Request};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_logout(&mut self, request: Request<Command>) -> trc::Result<()> {\n        let op_start = Instant::now();\n\n        let mut response =\n            StatusResponse::bye(\"Stalwart IMAP4rev2 bids you farewell.\".to_string()).into_bytes();\n\n        trc::event!(\n            Imap(trc::ImapEvent::Logout),\n            SpanId = self.session_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        response.extend(\n            StatusResponse::completed(Command::Logout)\n                .with_tag(request.tag)\n                .into_bytes(),\n        );\n        self.write_bytes(response).await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ::store::query::log::Query;\nuse imap_proto::ResponseCode;\n\npub mod acl;\npub mod append;\npub mod authenticate;\npub mod capability;\npub mod close;\npub mod copy_move;\npub mod create;\npub mod delete;\npub mod enable;\npub mod expunge;\npub mod fetch;\npub mod idle;\npub mod list;\npub mod login;\npub mod logout;\npub mod namespace;\npub mod noop;\npub mod quota;\npub mod rename;\npub mod search;\npub mod select;\npub mod status;\npub mod store;\npub mod subscribe;\npub mod thread;\n\ntrait FromModSeq {\n    fn from_modseq(modseq: u64) -> Self;\n}\n\ntrait ToModSeq {\n    fn to_modseq(&self) -> u64;\n}\n\nimpl FromModSeq for Query {\n    fn from_modseq(modseq: u64) -> Self {\n        if modseq > 0 {\n            Query::Since(modseq - 1)\n        } else {\n            Query::All\n        }\n    }\n}\n\nimpl ToModSeq for u64 {\n    fn to_modseq(&self) -> u64 {\n        if *self > 0 { *self + 1 } else { 0 }\n    }\n}\n\n#[macro_export]\nmacro_rules! spawn_op {\n    ($data:expr, $($code:tt)*) => {\n        {\n\n        tokio::spawn(async move {\n            let data = &($data);\n\n            if let Err(err) = (async {\n                $($code)*\n            })\n            .await\n            {\n                let _ = data.write_error(err).await;\n            }\n        });\n\n        Ok(())}\n    };\n}\npub trait ImapContext<T> {\n    fn imap_ctx(self, tag: &str, location: &'static str) -> trc::Result<T>;\n}\n\nimpl<T> ImapContext<T> for trc::Result<T> {\n    fn imap_ctx(self, tag: &str, location: &'static str) -> trc::Result<T> {\n        match self {\n            Ok(value) => Ok(value),\n            Err(err) => Err(\n                if !err.matches(trc::EventType::Imap(trc::ImapEvent::Error)) {\n                    err.ctx(trc::Key::Id, tag.to_string())\n                        .ctx(trc::Key::Details, \"Internal Server Error\")\n                        .ctx(trc::Key::Code, ResponseCode::ContactAdmin)\n                        .ctx(trc::Key::CausedBy, location)\n                } else {\n                    err.ctx(trc::Key::Id, tag.to_string())\n                },\n            ),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/namespace.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::Session;\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::{\n    Command, StatusResponse,\n    protocol::{ImapResponse, namespace::Response},\n    receiver::Request,\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_namespace(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapNamespace)?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::Namespace),\n            SpanId = self.session_id,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        self.write_bytes(\n            StatusResponse::completed(Command::Namespace)\n                .with_tag(request.tag)\n                .serialize(\n                    Response {\n                        shared_prefix: if self.state.session_data().mailboxes.lock().len() > 1 {\n                            Some(self.server.core.jmap.shared_folder.as_str().into())\n                        } else {\n                            None\n                        },\n                    }\n                    .serialize(),\n                ),\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/noop.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse crate::core::{Session, State};\nuse common::listener::SessionStream;\nuse imap_proto::{Command, StatusResponse, receiver::Request};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_noop(&mut self, request: Request<Command>) -> trc::Result<()> {\n        let op_start = Instant::now();\n\n        if let State::Selected { data, mailbox, .. } = &self.state {\n            data.write_changes(\n                &Some(mailbox.clone()),\n                false,\n                true,\n                self.is_qresync,\n                self.version.is_rev2(),\n                self.is_utf8,\n            )\n            .await?;\n        }\n\n        trc::event!(\n            Imap(trc::ImapEvent::Noop),\n            SpanId = self.session_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_bytes(\n            StatusResponse::completed(request.command)\n                .with_tag(request.tag)\n                .into_bytes(),\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/quota.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{Session, SessionData},\n    op::ImapContext,\n    spawn_op,\n};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse,\n    protocol::{\n        ImapResponse,\n        capability::QuotaResourceName,\n        quota::{Arguments, QuotaItem, QuotaResource, Response},\n    },\n    receiver::Request,\n};\nuse std::time::Instant;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_get_quota(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapStatus)?;\n\n        let data = self.state.session_data();\n\n        spawn_op!(data, {\n            match request.parse_get_quota() {\n                Ok(argument) => match data.get_quota(argument).await {\n                    Ok(response) => {\n                        data.write_bytes(response).await?;\n                    }\n                    Err(error) => {\n                        data.write_error(error).await?;\n                    }\n                },\n                Err(err) => data.write_error(err).await?,\n            }\n\n            Ok(())\n        })\n    }\n\n    pub async fn handle_get_quota_root(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapStatus)?;\n\n        let data = self.state.session_data();\n        let is_utf8 = self.is_utf8;\n\n        spawn_op!(data, {\n            match request.parse_get_quota_root(is_utf8) {\n                Ok(argument) => match data.get_quota_root(argument).await {\n                    Ok(response) => {\n                        data.write_bytes(response).await?;\n                    }\n                    Err(error) => {\n                        data.write_error(error).await?;\n                    }\n                },\n                Err(err) => data.write_error(err).await?,\n            }\n\n            Ok(())\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn get_quota(&self, arguments: Arguments) -> trc::Result<Vec<u8>> {\n        let op_start = Instant::now();\n\n        // Refresh mailboxes\n        self.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Validate quota root\n        let account_id: u32 = arguments\n            .name\n            .strip_prefix(\"#\")\n            .and_then(|id| id.parse().ok())\n            .filter(|id| self.access_token.is_member(*id))\n            .ok_or_else(|| {\n                trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Invalid quota root parameter.\")\n                    .id(arguments.tag.to_string())\n            })?;\n\n        // Obtain access token for mailbox\n        let access_token = self\n            .server\n            .get_access_token(account_id)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n        let used_quota = self\n            .server\n            .get_used_quota(account_id)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::GetQuota),\n            SpanId = self.session_id,\n            Id = arguments.name.clone(),\n            Details = vec![\n                trc::Value::from(used_quota),\n                trc::Value::from(access_token.quota)\n            ],\n            Elapsed = op_start.elapsed()\n        );\n\n        // Build response\n        let response = Response {\n            quota_root_items: vec![],\n            quota_items: vec![QuotaItem {\n                name: arguments.name,\n                resources: if access_token.quota > 0 {\n                    vec![QuotaResource {\n                        resource: QuotaResourceName::Storage,\n                        total: access_token.quota,\n                        used: used_quota as u64,\n                    }]\n                } else {\n                    vec![]\n                },\n            }],\n        };\n\n        Ok(StatusResponse::ok(\"GETQUOTA successful.\")\n            .with_tag(arguments.tag)\n            .serialize(response.serialize()))\n    }\n\n    pub async fn get_quota_root(&self, arguments: Arguments) -> trc::Result<Vec<u8>> {\n        let op_start = Instant::now();\n\n        // Refresh mailboxes\n        self.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Validate mailbox\n        let account_id = if let Some(mailbox) = self.get_mailbox_by_name(&arguments.name) {\n            mailbox.account_id\n        } else {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\"Mailbox does not exist.\")\n                .code(ResponseCode::TryCreate)\n                .id(arguments.tag));\n        };\n\n        // Obtain access token for mailbox\n        let access_token = self\n            .server\n            .get_access_token(account_id)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n        let used_quota = self\n            .server\n            .get_used_quota(account_id)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::GetQuota),\n            SpanId = self.session_id,\n            MailboxName = arguments.name.clone(),\n            Details = vec![\n                trc::Value::from(used_quota),\n                trc::Value::from(access_token.quota)\n            ],\n            Elapsed = op_start.elapsed()\n        );\n\n        // Build response\n        let response = Response {\n            quota_root_items: vec![arguments.name, format!(\"#{account_id}\")],\n            quota_items: vec![QuotaItem {\n                name: format!(\"#{account_id}\"),\n                resources: if access_token.quota > 0 {\n                    vec![QuotaResource {\n                        resource: QuotaResourceName::Storage,\n                        total: access_token.quota,\n                        used: used_quota as u64,\n                    }]\n                } else {\n                    vec![]\n                },\n            }],\n        };\n\n        Ok(StatusResponse::ok(\"GETQUOTAROOT successful.\")\n            .with_tag(arguments.tag)\n            .serialize(response.serialize()))\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/rename.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{Session, SessionData},\n    spawn_op,\n};\nuse common::{listener::SessionStream, sharing::EffectiveAcl, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse, protocol::rename::Arguments, receiver::Request,\n};\nuse std::time::Instant;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{acl::Acl, collection::Collection};\n\nuse super::ImapContext;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_rename(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapRename)?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_rename(self.is_utf8)?;\n        let data = self.state.session_data();\n\n        spawn_op!(data, {\n            let response = data.rename_folder(arguments, op_start).await?;\n            data.write_bytes(response.into_bytes()).await\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn rename_folder(\n        &self,\n        arguments: Arguments,\n        op_start: Instant,\n    ) -> trc::Result<StatusResponse> {\n        // Refresh mailboxes\n        self.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Validate mailbox name\n        let mut params = self\n            .validate_mailbox_create(&arguments.new_mailbox_name, None)\n            .await\n            .add_context(|err| err.id(arguments.tag.clone()))?;\n        params.is_rename = true;\n\n        // Validate source mailbox\n        let mailbox_id = {\n            let mut mailbox_id = None;\n            for account in self.mailboxes.lock().iter() {\n                if let Some(mailbox_id_) = account.mailbox_names.get(&arguments.mailbox_name) {\n                    if account.account_id == params.account_id {\n                        mailbox_id = (*mailbox_id_).into();\n                        break;\n                    } else {\n                        return Err(trc::ImapEvent::Error\n                            .into_err()\n                            .details(\"Cannot move mailboxes between accounts.\")\n                            .code(ResponseCode::Cannot)\n                            .id(arguments.tag));\n                    }\n                }\n            }\n            if let Some(mailbox_id) = mailbox_id {\n                mailbox_id\n            } else {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(format!(\"Mailbox '{}' not found.\", arguments.mailbox_name))\n                    .code(ResponseCode::NonExistent)\n                    .id(arguments.tag));\n            }\n        };\n\n        // Obtain mailbox\n        let mailbox_ = self\n            .server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                params.account_id,\n                Collection::Mailbox,\n                mailbox_id,\n            ))\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?\n            .ok_or_else(|| {\n                trc::ImapEvent::Error\n                    .into_err()\n                    .details(format!(\"Mailbox '{}' not found.\", arguments.mailbox_name))\n                    .caused_by(trc::location!())\n                    .code(ResponseCode::NonExistent)\n                    .id(arguments.tag.clone())\n            })?;\n        let mailbox = mailbox_\n            .to_unarchived::<email::mailbox::Mailbox>()\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Validate ACL\n        let access_token = self\n            .get_access_token()\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n        if access_token.is_shared(params.account_id)\n            && !mailbox\n                .inner\n                .acls\n                .effective_acl(&access_token)\n                .contains(Acl::Modify)\n        {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\"You are not allowed to rename this mailbox.\")\n                .code(ResponseCode::NoPerm)\n                .id(arguments.tag));\n        }\n\n        // Get new mailbox name from path\n        let new_mailbox_name = params.path.pop().unwrap();\n\n        // Build batch\n        let mut parent_id = params.parent_mailbox_id.map(|id| id + 1).unwrap_or(0);\n        let mut create_ids = Vec::with_capacity(params.path.len());\n        let mut next_document_id = self\n            .server\n            .store()\n            .assign_document_ids(\n                params.account_id,\n                Collection::Mailbox,\n                params.path.len() as u64,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        let mut batch = BatchBuilder::new();\n\n        for &path_item in params.path.iter() {\n            let mailbox_id = next_document_id;\n            next_document_id -= 1;\n\n            batch\n                .with_account_id(params.account_id)\n                .with_collection(Collection::Mailbox)\n                .with_document(mailbox_id)\n                .custom(ObjectIndexBuilder::<(), _>::new().with_changes(\n                    email::mailbox::Mailbox::new(path_item).with_parent_id(parent_id),\n                ))\n                .imap_ctx(&arguments.tag, trc::location!())?\n                .commit_point();\n\n            parent_id = mailbox_id + 1;\n            create_ids.push(mailbox_id);\n        }\n\n        let mut new_mailbox = mailbox\n            .deserialize::<email::mailbox::Mailbox>()\n            .caused_by(trc::location!())?;\n        new_mailbox.name = new_mailbox_name.into();\n        new_mailbox.parent_id = parent_id;\n        new_mailbox.uid_validity = rand::random::<u32>();\n        batch\n            .with_account_id(params.account_id)\n            .with_collection(Collection::Mailbox)\n            .with_document(mailbox_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_current(mailbox)\n                    .with_changes(new_mailbox),\n            )\n            .imap_ctx(&arguments.tag, trc::location!())?;\n        self.server\n            .commit_batch(batch)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        trc::event!(\n            Imap(trc::ImapEvent::RenameMailbox),\n            SpanId = self.session_id,\n            AccountId = params.account_id,\n            MailboxName = arguments.new_mailbox_name,\n            MailboxId = mailbox_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::completed(Command::Rename).with_tag(arguments.tag))\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{FromModSeq, ToModSeq};\nuse crate::{\n    core::{ImapId, SavedSearch, SelectedMailbox, Session, SessionData},\n    spawn_op,\n};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::cache::{MessageCacheFetch, email::MessageCacheAccess};\nuse imap_proto::{\n    Command, StatusResponse,\n    protocol::{\n        Sequence,\n        search::{self, Arguments, Comparator, Filter, Response, ResultOption},\n    },\n    receiver::Request,\n};\nuse mail_parser::HeaderName;\nuse nlp::language::Language;\nuse std::{str::FromStr, sync::Arc, time::Instant};\nuse store::{\n    query::log::Query,\n    roaring::RoaringBitmap,\n    search::{\n        EmailSearchField, SearchComparator, SearchFilter, SearchOperator, SearchQuery, SearchValue,\n    },\n    write::{SearchIndex, now},\n};\nuse tokio::sync::watch;\nuse trc::AddContext;\nuse types::{collection::SyncCollection, id::Id, keyword::Keyword};\nuse utils::map::vec_map::VecMap;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_search(\n        &mut self,\n        request: Request<Command>,\n        is_sort: bool,\n        is_uid: bool,\n    ) -> trc::Result<()> {\n        let op_start = Instant::now();\n        let mut arguments = if !is_sort {\n            // Validate access\n            self.assert_has_permission(Permission::ImapSearch)?;\n\n            request.parse_search(self.version)\n        } else {\n            // Validate access\n            self.assert_has_permission(Permission::ImapSort)?;\n\n            request.parse_sort()\n        }?;\n\n        let (data, mailbox) = self.state.mailbox_state();\n\n        // Create channel for results\n        let (results_tx, prev_saved_search) =\n            if arguments.result_options.contains(&ResultOption::Save) {\n                let prev_saved_search = Some(mailbox.get_saved_search().await);\n                let (tx, rx) = watch::channel(Arc::new(Vec::new()));\n                *mailbox.saved_search.lock() = SavedSearch::InFlight { rx };\n                (tx.into(), prev_saved_search)\n            } else {\n                (None, None)\n            };\n\n        spawn_op!(data, {\n            let tag = std::mem::take(&mut arguments.tag);\n            let bytes = match data\n                .search(\n                    arguments,\n                    mailbox.clone(),\n                    results_tx,\n                    prev_saved_search.clone(),\n                    is_uid,\n                    op_start,\n                )\n                .await\n            {\n                Ok(response) => {\n                    let response = response.serialize(&tag);\n                    StatusResponse::completed(if !is_sort {\n                        Command::Search(is_uid)\n                    } else {\n                        Command::Sort(is_uid)\n                    })\n                    .with_tag(tag)\n                    .serialize(response)\n                }\n                Err(err) => {\n                    if let Some(prev_saved_search) = prev_saved_search {\n                        *mailbox.saved_search.lock() = prev_saved_search\n                            .map_or(SavedSearch::None, |s| SavedSearch::Results { items: s });\n                    }\n                    return Err(err.id(tag));\n                }\n            };\n            data.write_bytes(bytes).await\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn search(\n        &self,\n        arguments: Arguments,\n        mailbox: Arc<SelectedMailbox>,\n        results_tx: Option<watch::Sender<Arc<Vec<ImapId>>>>,\n        prev_saved_search: Option<Option<Arc<Vec<ImapId>>>>,\n        is_uid: bool,\n        op_start: Instant,\n    ) -> trc::Result<search::Response> {\n        // Run query\n        let is_sort = arguments.sort.is_some();\n        let (result_set, include_highest_modseq) = self\n            .query(\n                arguments.filter,\n                arguments.sort.unwrap_or_default(),\n                &mailbox,\n                &prev_saved_search,\n            )\n            .await?;\n\n        // Obtain modseq\n        let highest_modseq = if include_highest_modseq {\n            self.synchronize_messages(&mailbox)\n                .await?\n                .to_modseq()\n                .into()\n        } else {\n            None\n        };\n\n        // Sort and map ids\n        let mut min: Option<(u32, ImapId)> = None;\n        let mut max: Option<(u32, ImapId)> = None;\n        let mut total = 0;\n        let results_len = result_set.len();\n        let mut saved_results = if results_tx.is_some() {\n            Some(Vec::with_capacity(results_len))\n        } else {\n            None\n        };\n        let mut imap_ids = Vec::with_capacity(results_len);\n        mailbox.map_search_results(\n            result_set.into_iter(),\n            is_uid,\n            arguments.result_options.contains(&ResultOption::Min),\n            arguments.result_options.contains(&ResultOption::Max),\n            &mut min,\n            &mut max,\n            &mut total,\n            &mut imap_ids,\n            &mut saved_results,\n        );\n        if !is_sort {\n            imap_ids.sort_unstable();\n        }\n\n        // Save results\n        if let (Some(results_tx), Some(saved_results)) = (results_tx, saved_results) {\n            let saved_results = Arc::new(saved_results);\n            *mailbox.saved_search.lock() = SavedSearch::Results {\n                items: saved_results.clone(),\n            };\n            results_tx.send(saved_results).ok();\n        }\n\n        trc::event!(\n            Imap(if !is_sort {\n                trc::ImapEvent::Search\n            } else {\n                trc::ImapEvent::Sort\n            }),\n            SpanId = self.session_id,\n            AccountId = mailbox.id.account_id,\n            MailboxId = mailbox.id.mailbox_id,\n            Total = total,\n            Elapsed = op_start.elapsed()\n        );\n\n        // Build response\n        Ok(Response {\n            is_uid,\n            min: min.map(|(id, _)| id),\n            max: max.map(|(id, _)| id),\n            count: if arguments.result_options.contains(&ResultOption::Count) {\n                Some(total)\n            } else {\n                None\n            },\n            ids: if arguments.result_options.is_empty()\n                || arguments.result_options.contains(&ResultOption::All)\n            {\n                imap_ids\n            } else {\n                vec![]\n            },\n            is_sort,\n            is_esearch: arguments.is_esearch,\n            highest_modseq,\n        })\n    }\n\n    pub async fn query(\n        &self,\n        imap_filter: Vec<Filter>,\n        imap_comparator: Vec<Comparator>,\n        mailbox: &SelectedMailbox,\n        prev_saved_search: &Option<Option<Arc<Vec<ImapId>>>>,\n    ) -> trc::Result<(Vec<u32>, bool)> {\n        // Obtain message ids\n        let mut filters = Vec::with_capacity(imap_filter.len() + 1);\n        let cache = self\n            .server\n            .get_cached_messages(mailbox.id.account_id)\n            .await\n            .caused_by(trc::location!())?;\n        let message_ids = RoaringBitmap::from_iter(\n            cache\n                .in_mailbox(mailbox.id.mailbox_id)\n                .map(|m| m.document_id),\n        );\n\n        // Convert query\n        let mut include_highest_modseq = false;\n        for filter in imap_filter {\n            match filter {\n                Filter::Sequence(sequence, uid_filter) => {\n                    let mut set = RoaringBitmap::new();\n                    if let (Sequence::SavedSearch, Some(prev_saved_search)) =\n                        (&sequence, &prev_saved_search)\n                    {\n                        if let Some(prev_saved_search) = prev_saved_search {\n                            let state = mailbox.state.lock();\n                            for imap_id in prev_saved_search.iter() {\n                                if let Some(id) = state.uid_to_id.get(&imap_id.uid) {\n                                    set.insert(*id);\n                                }\n                            }\n                        } else {\n                            return Err(trc::ImapEvent::Error\n                                .into_err()\n                                .details(\"No saved search found.\"));\n                        }\n                    } else {\n                        for id in mailbox.sequence_to_ids(&sequence, uid_filter).await?.keys() {\n                            set.insert(*id);\n                        }\n                    }\n                    filters.push(SearchFilter::is_in_set(set));\n                }\n                Filter::All => {\n                    filters.push(SearchFilter::is_in_set(message_ids.clone()));\n                }\n                Filter::Answered => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache\n                            .with_keyword(&Keyword::Answered)\n                            .map(|m| m.document_id),\n                    )));\n                }\n                Filter::Before(date) => {\n                    filters.push(SearchFilter::lt(EmailSearchField::ReceivedAt, date));\n                }\n                Filter::Deleted => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache.with_keyword(&Keyword::Deleted).map(|m| m.document_id),\n                    )));\n                }\n                Filter::Draft => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache.with_keyword(&Keyword::Draft).map(|m| m.document_id),\n                    )));\n                }\n                Filter::Flagged => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache.with_keyword(&Keyword::Flagged).map(|m| m.document_id),\n                    )));\n                }\n                Filter::Keyword(keyword) => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache\n                            .with_keyword(&Keyword::from(keyword))\n                            .map(|m| m.document_id),\n                    )));\n                }\n                Filter::Larger(size) => {\n                    filters.push(SearchFilter::gt(EmailSearchField::Size, size));\n                }\n                Filter::On(date) => {\n                    filters.push(SearchFilter::And);\n                    filters.push(SearchFilter::ge(EmailSearchField::ReceivedAt, date));\n                    filters.push(SearchFilter::lt(EmailSearchField::ReceivedAt, date + 86400));\n                    filters.push(SearchFilter::End);\n                }\n                Filter::Seen => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache.with_keyword(&Keyword::Seen).map(|m| m.document_id),\n                    )));\n                }\n                Filter::SentBefore(date) => {\n                    filters.push(SearchFilter::lt(EmailSearchField::SentAt, date));\n                }\n                Filter::SentOn(date) => {\n                    filters.push(SearchFilter::And);\n                    filters.push(SearchFilter::ge(EmailSearchField::SentAt, date));\n                    filters.push(SearchFilter::lt(EmailSearchField::SentAt, date + 86400));\n                    filters.push(SearchFilter::End);\n                }\n                Filter::SentSince(date) => {\n                    filters.push(SearchFilter::ge(EmailSearchField::SentAt, date));\n                }\n                Filter::Since(date) => {\n                    filters.push(SearchFilter::ge(EmailSearchField::ReceivedAt, date));\n                }\n                Filter::Smaller(size) => {\n                    filters.push(SearchFilter::lt(EmailSearchField::Size, size));\n                }\n                Filter::Unanswered => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache\n                            .without_keyword(&Keyword::Answered)\n                            .map(|m| m.document_id),\n                    )));\n                }\n                Filter::Undeleted => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache\n                            .without_keyword(&Keyword::Deleted)\n                            .map(|m| m.document_id),\n                    )));\n                }\n                Filter::Undraft => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache\n                            .without_keyword(&Keyword::Draft)\n                            .map(|m| m.document_id),\n                    )));\n                }\n                Filter::Unflagged => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache\n                            .without_keyword(&Keyword::Flagged)\n                            .map(|m| m.document_id),\n                    )));\n                }\n                Filter::Unkeyword(keyword) => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache\n                            .without_keyword(&Keyword::from(keyword))\n                            .map(|m| m.document_id),\n                    )));\n                }\n                Filter::Unseen => {\n                    filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                        cache.without_keyword(&Keyword::Seen).map(|m| m.document_id),\n                    )));\n                }\n                Filter::Recent => {\n                    //filters.push(SearchFilter::is_in_set(self.get_recent(&mailbox.id)));\n                }\n                Filter::New => {\n                    /*filters.push(SearchFilter::And);\n                    filters.push(SearchFilter::is_in_set(self.get_recent(&mailbox.id)));\n                    filters.push(SearchFilter::Not);\n                    filters.push(SearchFilter::is_in_bitmap(\n                        EmailSearchField::Keywords,\n                        Keyword::Seen,\n                    ));\n                    filters.push(SearchFilter::End);\n                    filters.push(SearchFilter::End);*/\n                }\n                Filter::Old => {\n                    /*filters.push(SearchFilter::Not);\n                    filters.push(SearchFilter::is_in_set(self.get_recent(&mailbox.id)));\n                    filters.push(SearchFilter::End);*/\n                }\n                Filter::Older(secs) => {\n                    filters.push(SearchFilter::le(\n                        EmailSearchField::ReceivedAt,\n                        now().saturating_sub(secs as u64),\n                    ));\n                }\n                Filter::Younger(secs) => {\n                    filters.push(SearchFilter::ge(\n                        EmailSearchField::ReceivedAt,\n                        now().saturating_sub(secs as u64),\n                    ));\n                }\n                Filter::ModSeq((modseq, _)) => {\n                    let mut set = RoaringBitmap::new();\n                    for id in self\n                        .server\n                        .store()\n                        .changes(\n                            mailbox.id.account_id,\n                            SyncCollection::Email.into(),\n                            Query::from_modseq(modseq),\n                        )\n                        .await?\n                        .changes\n                        .into_iter()\n                        .filter_map(|change| change.try_unwrap_item_id())\n                    {\n                        let id = (id & u32::MAX as u64) as u32;\n                        if message_ids.contains(id) {\n                            set.insert(id);\n                        }\n                    }\n                    filters.push(SearchFilter::is_in_set(set));\n                    include_highest_modseq = true;\n                }\n                Filter::EmailId(id) => {\n                    if let Ok(id) = Id::from_str(&id) {\n                        filters.push(SearchFilter::is_in_set(\n                            RoaringBitmap::from_sorted_iter([id.document_id()]).unwrap(),\n                        ));\n                    } else {\n                        return Err(trc::ImapEvent::Error\n                            .into_err()\n                            .details(format!(\"Failed to parse email id '{id}'.\",)));\n                    }\n                }\n                Filter::ThreadId(id) => {\n                    if let Ok(id) = Id::from_str(&id) {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.in_thread(id.document_id()).map(|m| m.document_id),\n                        )));\n                    } else {\n                        return Err(trc::ImapEvent::Error\n                            .into_err()\n                            .details(format!(\"Failed to parse thread id '{id}'.\",)));\n                    }\n                }\n                Filter::Bcc(text) => {\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::Bcc,\n                        text,\n                        Language::None,\n                    ));\n                }\n                Filter::Body(text) => {\n                    filters.push(SearchFilter::has_text_detect(\n                        EmailSearchField::Body,\n                        text,\n                        self.server.core.jmap.default_language,\n                    ));\n                }\n                Filter::Cc(text) => {\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::Cc,\n                        text,\n                        Language::None,\n                    ));\n                }\n                Filter::From(text) => {\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::From,\n                        text,\n                        Language::None,\n                    ));\n                }\n                Filter::Header(header, value) => {\n                    if let Some(header) = HeaderName::parse(header) {\n                        match header {\n                            HeaderName::Subject => {\n                                filters.push(SearchFilter::has_text_detect(\n                                    EmailSearchField::Subject,\n                                    value,\n                                    self.server.core.jmap.default_language,\n                                ));\n                            }\n                            header @ (HeaderName::From\n                            | HeaderName::To\n                            | HeaderName::Cc\n                            | HeaderName::Bcc) => {\n                                filters.push(SearchFilter::has_text(\n                                    match header {\n                                        HeaderName::From => EmailSearchField::From,\n                                        HeaderName::To => EmailSearchField::To,\n                                        HeaderName::Cc => EmailSearchField::Cc,\n                                        HeaderName::Bcc => EmailSearchField::Bcc,\n                                        _ => unreachable!(),\n                                    },\n                                    value,\n                                    Language::None,\n                                ));\n                            }\n                            header => {\n                                let op = if matches!(\n                                    header,\n                                    HeaderName::MessageId\n                                        | HeaderName::InReplyTo\n                                        | HeaderName::References\n                                        | HeaderName::ResentMessageId\n                                ) || value.is_empty()\n                                {\n                                    SearchOperator::Equal\n                                } else {\n                                    SearchOperator::Contains\n                                };\n\n                                filters.push(SearchFilter::cond(\n                                    EmailSearchField::Headers,\n                                    op,\n                                    SearchValue::KeyValues(\n                                        VecMap::with_capacity(1)\n                                            .with_append(header.as_str().to_lowercase(), value),\n                                    ),\n                                ));\n                            }\n                        }\n                    }\n                }\n                Filter::Subject(text) => {\n                    filters.push(SearchFilter::has_text_detect(\n                        EmailSearchField::Subject,\n                        text,\n                        self.server.core.jmap.default_language,\n                    ));\n                }\n                Filter::Text(text) => {\n                    let (text, language) =\n                        Language::detect(text, self.server.core.jmap.default_language);\n\n                    filters.push(SearchFilter::Or);\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::From,\n                        &text,\n                        Language::None,\n                    ));\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::To,\n                        &text,\n                        Language::None,\n                    ));\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::Cc,\n                        &text,\n                        Language::None,\n                    ));\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::Bcc,\n                        &text,\n                        Language::None,\n                    ));\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::Subject,\n                        &text,\n                        language,\n                    ));\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::Body,\n                        &text,\n                        language,\n                    ));\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::Attachment,\n                        text,\n                        language,\n                    ));\n                    filters.push(SearchFilter::End);\n                }\n                Filter::To(text) => {\n                    filters.push(SearchFilter::has_text(\n                        EmailSearchField::To,\n                        text,\n                        Language::None,\n                    ));\n                }\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::End => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        // Convert comparators\n        let mut comparators = Vec::with_capacity(imap_comparator.len());\n        for comparator in imap_comparator {\n            comparators.push(match comparator.sort {\n                search::Sort::Arrival => {\n                    SearchComparator::field(EmailSearchField::ReceivedAt, comparator.ascending)\n                }\n                search::Sort::Cc => {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"Sorting by CC is not supported.\"));\n                }\n                search::Sort::Date => {\n                    SearchComparator::field(EmailSearchField::SentAt, comparator.ascending)\n                }\n                search::Sort::From | search::Sort::DisplayFrom => {\n                    SearchComparator::field(EmailSearchField::From, comparator.ascending)\n                }\n                search::Sort::Size => {\n                    SearchComparator::field(EmailSearchField::Size, comparator.ascending)\n                }\n                search::Sort::Subject => {\n                    SearchComparator::field(EmailSearchField::Subject, comparator.ascending)\n                }\n                search::Sort::To | search::Sort::DisplayTo => {\n                    SearchComparator::field(EmailSearchField::To, comparator.ascending)\n                }\n            });\n        }\n\n        // Run query\n        self.server\n            .search_store()\n            .query_account(\n                SearchQuery::new(SearchIndex::Email)\n                    .with_filters(filters)\n                    .with_comparators(comparators)\n                    .with_account_id(mailbox.id.account_id)\n                    .with_mask(message_ids),\n            )\n            .await\n            .map(|res| (res, include_highest_modseq))\n            .caused_by(trc::location!())\n    }\n}\n\nimpl SelectedMailbox {\n    pub async fn get_saved_search(&self) -> Option<Arc<Vec<ImapId>>> {\n        let mut rx = match &*self.saved_search.lock() {\n            SavedSearch::InFlight { rx } => rx.clone(),\n            SavedSearch::Results { items } => {\n                return Some(items.clone());\n            }\n            SavedSearch::None => {\n                return None;\n            }\n        };\n        rx.changed().await.ok();\n        let v = rx.borrow();\n        Some(v.clone())\n    }\n\n    #[allow(clippy::too_many_arguments)]\n    pub fn map_search_results(\n        &self,\n        ids: impl Iterator<Item = u32>,\n        is_uid: bool,\n        find_min: bool,\n        find_max: bool,\n        min: &mut Option<(u32, ImapId)>,\n        max: &mut Option<(u32, ImapId)>,\n        total: &mut u32,\n        imap_ids: &mut Vec<u32>,\n        saved_results: &mut Option<Vec<ImapId>>,\n    ) {\n        let state = self.state.lock();\n        let find_min_or_max = find_min || find_max;\n        for document_id in ids {\n            if let Some((id, imap_id)) = state.map_result_id(document_id, is_uid) {\n                if find_min_or_max {\n                    if find_min {\n                        if let Some((prev_min, _)) = min {\n                            if id < *prev_min {\n                                *min = Some((id, imap_id));\n                            }\n                        } else {\n                            *min = Some((id, imap_id));\n                        }\n                    }\n                    if find_max {\n                        if let Some((prev_max, _)) = max {\n                            if id > *prev_max {\n                                *max = Some((id, imap_id));\n                            }\n                        } else {\n                            *max = Some((id, imap_id));\n                        }\n                    }\n                } else {\n                    imap_ids.push(id);\n                    if let Some(r) = saved_results.as_mut() {\n                        r.push(imap_id)\n                    }\n                }\n                *total += 1;\n            }\n        }\n        if find_min || find_max {\n            for (id, imap_id) in [min, max].into_iter().flatten() {\n                imap_ids.push(*id);\n                if let Some(r) = saved_results.as_mut() {\n                    r.push(*imap_id)\n                }\n            }\n        }\n    }\n}\n\nimpl SavedSearch {\n    pub async fn unwrap(&self) -> Option<Arc<Vec<ImapId>>> {\n        match self {\n            SavedSearch::InFlight { rx } => {\n                let mut rx = rx.clone();\n                rx.changed().await.ok();\n                let v = rx.borrow();\n                Some(v.clone())\n            }\n            SavedSearch::Results { items } => Some(items.clone()),\n            SavedSearch::None => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/select.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapContext, ToModSeq};\nuse crate::core::{SavedSearch, SelectedMailbox, Session, State};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse,\n    protocol::{\n        ImapResponse, Sequence, fetch,\n        list::ListItem,\n        select::{HighestModSeq, Response},\n    },\n    receiver::Request,\n};\nuse std::{sync::Arc, time::Instant};\nuse types::id::Id;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_select(&mut self, request: Request<Command>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(if request.command == Command::Select {\n            Permission::ImapSelect\n        } else {\n            Permission::ImapExamine\n        })?;\n\n        let op_start = Instant::now();\n        let is_select = request.command == Command::Select;\n        let command = request.command;\n        let arguments = request.parse_select(self.is_utf8)?;\n        let data = self.state.session_data();\n\n        // Refresh mailboxes\n        data.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        if let Some(mailbox) = data.get_mailbox_by_name(&arguments.mailbox_name) {\n            // Try obtaining the mailbox from the cache\n            let state = data\n                .fetch_messages(&mailbox, None)\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?\n                .unwrap();\n\n            // Synchronize messages\n            let closed_previous = self.state.close_mailbox();\n            let is_condstore = self.is_condstore || arguments.condstore;\n\n            // Build new state\n            let is_rev2 = self.version.is_rev2();\n            let is_utf8 = self.is_utf8;\n            let mailbox_state = data.mailbox_state(&mailbox).unwrap();\n            let total_messages = state.total_messages;\n            let highest_modseq = if is_condstore {\n                HighestModSeq::new(state.modseq.to_modseq()).into()\n            } else {\n                None\n            };\n            let mailbox = Arc::new(SelectedMailbox {\n                id: mailbox,\n                state: parking_lot::Mutex::new(state),\n                saved_search: parking_lot::Mutex::new(SavedSearch::None),\n                is_select,\n                is_condstore,\n            });\n\n            // Validate QRESYNC arguments\n            if let Some(qresync) = arguments.qresync {\n                if !self.is_qresync {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(\"QRESYNC is not enabled.\")\n                        .id(arguments.tag));\n                }\n                if qresync.uid_validity == mailbox_state.uid_validity as u32 {\n                    // Send flags for changed messages\n                    data.fetch(\n                        fetch::Arguments {\n                            tag: \"\".into(),\n                            sequence_set: qresync\n                                .known_uids\n                                .or_else(|| qresync.seq_match.map(|(_, s)| s))\n                                .unwrap_or(Sequence::Range {\n                                    start: 1.into(),\n                                    end: None,\n                                }),\n                            attributes: vec![fetch::Attribute::Flags],\n                            changed_since: qresync.modseq.into(),\n                            include_vanished: true,\n                        },\n                        mailbox.clone(),\n                        true,\n                        true,\n                        false,\n                        Instant::now(),\n                    )\n                    .await\n                    .imap_ctx(&arguments.tag, trc::location!())?;\n                }\n            }\n\n            trc::event!(\n                Imap(trc::ImapEvent::Select),\n                SpanId = self.session_id,\n                MailboxName = arguments.mailbox_name.clone(),\n                AccountId = mailbox.id.account_id,\n                MailboxId = mailbox.id.mailbox_id,\n                Total = total_messages,\n                UidNext = mailbox_state.uid_next,\n                UidValidity = mailbox_state.uid_validity,\n                Elapsed = op_start.elapsed()\n            );\n\n            // Build response\n            let response = Response {\n                mailbox: ListItem::new(arguments.mailbox_name),\n                total_messages,\n                recent_messages: 0,\n                unseen_seq: 0,\n                uid_validity: mailbox_state.uid_validity as u32,\n                uid_next: mailbox_state.uid_next as u32,\n                closed_previous,\n                is_rev2,\n                is_utf8,\n                highest_modseq,\n                mailbox_id: Id::from_parts(mailbox.id.account_id, mailbox.id.mailbox_id)\n                    .to_string(),\n            };\n\n            // Update state\n            self.state = State::Selected { data, mailbox };\n\n            self.write_bytes(\n                StatusResponse::completed(command)\n                    .with_tag(arguments.tag)\n                    .with_code(if is_select {\n                        ResponseCode::ReadWrite\n                    } else {\n                        ResponseCode::ReadOnly\n                    })\n                    .serialize(response.serialize()),\n            )\n            .await\n        } else {\n            Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\"Mailbox does not exist.\")\n                .code(ResponseCode::NonExistent)\n                .id(arguments.tag))\n        }\n    }\n\n    pub async fn handle_unselect(&mut self, request: Request<Command>) -> trc::Result<()> {\n        self.state.close_mailbox();\n        self.state = State::Authenticated {\n            data: self.state.session_data(),\n        };\n        self.write_bytes(\n            StatusResponse::completed(Command::Unselect)\n                .with_tag(request.tag)\n                .into_bytes(),\n        )\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/status.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ToModSeq;\nuse crate::{\n    core::{Mailbox, Session, SessionData},\n    op::ImapContext,\n    spawn_op,\n};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::cache::{MessageCacheFetch, email::MessageCacheAccess};\nuse imap_proto::{\n    Command, ResponseCode, StatusResponse,\n    parser::PushUnique,\n    protocol::status::{Status, StatusItem, StatusItemType},\n    receiver::Request,\n};\nuse std::time::Instant;\nuse trc::AddContext;\nuse types::{id::Id, keyword::Keyword};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_status(&mut self, requests: Vec<Request<Command>>) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapStatus)?;\n\n        let is_utf8 = self.is_utf8;\n        let data = self.state.session_data();\n\n        spawn_op!(data, {\n            let mut did_sync = false;\n\n            for request in requests.into_iter() {\n                match request.parse_status(is_utf8) {\n                    Ok(arguments) => {\n                        let op_start = Instant::now();\n                        if !did_sync {\n                            // Refresh mailboxes\n                            data.synchronize_mailboxes(false)\n                                .await\n                                .imap_ctx(&arguments.tag, trc::location!())?;\n                            did_sync = true;\n                        }\n\n                        // Fetch status\n                        let status = data\n                            .status(arguments.mailbox_name, &arguments.items)\n                            .await\n                            .imap_ctx(&arguments.tag, trc::location!())?;\n\n                        trc::event!(\n                            Imap(trc::ImapEvent::Status),\n                            SpanId = data.session_id,\n                            MailboxName = status.mailbox_name.clone(),\n                            Details = arguments\n                                .items\n                                .iter()\n                                .map(|c| trc::Value::from(format!(\"{c:?}\")))\n                                .collect::<Vec<_>>(),\n                            Elapsed = op_start.elapsed()\n                        );\n\n                        let mut buf = Vec::with_capacity(32);\n                        status.serialize(&mut buf, is_utf8);\n                        data.write_bytes(\n                            StatusResponse::completed(Command::Status)\n                                .with_tag(arguments.tag)\n                                .serialize(buf),\n                        )\n                        .await?;\n                    }\n                    Err(err) => data.write_error(err).await?,\n                }\n            }\n\n            Ok(())\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn status(&self, mailbox_name: String, items: &[Status]) -> trc::Result<StatusItem> {\n        // Get mailbox id\n        let mailbox = if let Some(mailbox) = self.get_mailbox_by_name(&mailbox_name) {\n            mailbox\n        } else {\n            // Some IMAP clients will try to get the status of a mailbox with the NoSelect flag\n            return if mailbox_name == self.server.core.jmap.shared_folder\n                || mailbox_name\n                    .split_once('/')\n                    .is_some_and(|(base_name, path)| {\n                        base_name == self.server.core.jmap.shared_folder && !path.contains('/')\n                    })\n            {\n                Ok(StatusItem {\n                    mailbox_name,\n                    items: items\n                        .iter()\n                        .map(|item| {\n                            (\n                                *item,\n                                match item {\n                                    Status::Messages\n                                    | Status::Size\n                                    | Status::Unseen\n                                    | Status::Recent\n                                    | Status::Deleted\n                                    | Status::HighestModSeq\n                                    | Status::DeletedStorage => StatusItemType::Number(0),\n                                    Status::UidNext | Status::UidValidity => {\n                                        StatusItemType::Number(1)\n                                    }\n                                    Status::MailboxId => StatusItemType::String(\"none\".into()),\n                                },\n                            )\n                        })\n                        .collect(),\n                })\n            } else {\n                Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Mailbox does not exist.\")\n                    .code(ResponseCode::NonExistent))\n            };\n        };\n\n        // Make sure all requested fields are up to date\n        let mut items_update = Vec::with_capacity(items.len());\n        let mut items_response = Vec::with_capacity(items.len());\n\n        for account in self.mailboxes.lock().iter_mut() {\n            if account.account_id == mailbox.account_id {\n                let mailbox_state =\n                    if let Some(mailbox_state) = account.mailbox_state.get(&mailbox.mailbox_id) {\n                        mailbox_state\n                    } else {\n                        continue;\n                    };\n                for item in items {\n                    match item {\n                        Status::Messages => {\n                            items_response.push((\n                                *item,\n                                StatusItemType::Number(mailbox_state.total_messages),\n                            ));\n                        }\n                        Status::UidNext => {\n                            items_response\n                                .push((*item, StatusItemType::Number(mailbox_state.uid_next)));\n                        }\n                        Status::UidValidity => {\n                            items_response\n                                .push((*item, StatusItemType::Number(mailbox_state.uid_validity)));\n                        }\n                        Status::Unseen => {\n                            items_response\n                                .push((*item, StatusItemType::Number(mailbox_state.total_unseen)));\n                        }\n                        Status::Deleted => {\n                            items_response\n                                .push((*item, StatusItemType::Number(mailbox_state.total_deleted)));\n                        }\n                        Status::DeletedStorage => {\n                            if let Some(value) = mailbox_state.total_deleted_storage {\n                                items_response.push((*item, StatusItemType::Number(value)));\n                            } else {\n                                items_update.push_unique(*item);\n                            }\n                        }\n                        Status::Size => {\n                            if let Some(value) = mailbox_state.size {\n                                items_response.push((*item, StatusItemType::Number(value)));\n                            } else {\n                                items_update.push_unique(*item);\n                            }\n                        }\n                        Status::HighestModSeq => {\n                            items_response.push((\n                                *item,\n                                StatusItemType::Number(account.last_change_id.to_modseq()),\n                            ));\n                        }\n                        Status::MailboxId => {\n                            items_response.push((\n                                *item,\n                                StatusItemType::String(\n                                    Id::from_parts(mailbox.account_id, mailbox.mailbox_id)\n                                        .to_string(),\n                                ),\n                            ));\n                        }\n                        Status::Recent => {\n                            items_response.push((*item, StatusItemType::Number(0)));\n                        }\n                    }\n                }\n                break;\n            }\n        }\n\n        if !items_update.is_empty() {\n            // Retrieve latest values\n            let mut values_update = Vec::with_capacity(items_update.len());\n\n            let cache = self\n                .server\n                .get_cached_messages(mailbox.account_id)\n                .await\n                .caused_by(trc::location!())?;\n\n            for item in items_update {\n                let result = match item {\n                    Status::DeletedStorage => cache\n                        .in_mailbox_with_keyword(mailbox.mailbox_id, &Keyword::Deleted)\n                        .map(|x| x.size)\n                        .sum::<u32>() as u64,\n                    Status::Size => cache\n                        .in_mailbox(mailbox.mailbox_id)\n                        .map(|x| x.size)\n                        .sum::<u32>() as u64,\n                    _ => {\n                        unreachable!()\n                    }\n                };\n\n                items_response.push((item, StatusItemType::Number(result)));\n                values_update.push((item, result));\n            }\n\n            // Update cache\n            for account in self.mailboxes.lock().iter_mut() {\n                if account.account_id == mailbox.account_id {\n                    let mailbox_state = account\n                        .mailbox_state\n                        .entry(mailbox.mailbox_id)\n                        .or_insert_with(Mailbox::default);\n\n                    for (item, value) in values_update {\n                        match item {\n                            Status::DeletedStorage => {\n                                mailbox_state.total_deleted_storage = value.into()\n                            }\n                            Status::Size => mailbox_state.size = value.into(),\n                            Status::Recent => {\n                                items_response\n                                    .iter_mut()\n                                    .find(|(i, _)| *i == Status::Recent)\n                                    .unwrap()\n                                    .1 = StatusItemType::Number(0);\n                            }\n                            _ => {\n                                unreachable!()\n                            }\n                        }\n                    }\n\n                    break;\n                }\n            }\n        }\n\n        // Generate response\n        Ok(StatusItem {\n            mailbox_name,\n            items: items_response,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/store.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{FromModSeq, ImapContext};\nuse crate::{\n    core::{SelectedMailbox, Session, SessionData},\n    spawn_op,\n};\nuse ahash::AHashSet;\nuse common::{listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse email::{\n    mailbox::TRASH_ID,\n    message::{ingest::EmailIngest, metadata::MessageData},\n};\nuse imap_proto::{\n    Command, ResponseCode, ResponseType, StatusResponse,\n    protocol::{\n        Flag, ImapResponse,\n        fetch::{DataItem, FetchItem},\n        store::{Arguments, Operation, Response},\n    },\n    receiver::Request,\n};\nuse std::{sync::Arc, time::Instant};\nuse store::{\n    ValueKey,\n    query::log::{Change, Query},\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    keyword::Keyword,\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_store(\n        &mut self,\n        request: Request<Command>,\n        is_uid: bool,\n    ) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapStore)?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_store()?;\n        let (data, mailbox) = self.state.select_data();\n        let is_condstore = self.is_condstore || mailbox.is_condstore;\n\n        spawn_op!(data, {\n            let response = data\n                .store(arguments, mailbox, is_uid, is_condstore, op_start)\n                .await?;\n\n            data.write_bytes(response).await\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn store(\n        &self,\n        arguments: Arguments,\n        mailbox: Arc<SelectedMailbox>,\n        is_uid: bool,\n        is_condstore: bool,\n        op_start: Instant,\n    ) -> trc::Result<Vec<u8>> {\n        // Resync messages if needed\n        let account_id = mailbox.id.account_id;\n        self.synchronize_messages(&mailbox)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n\n        // Convert IMAP ids to JMAP ids.\n        let mut ids = mailbox\n            .sequence_to_ids(&arguments.sequence_set, is_uid)\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?;\n        if ids.is_empty() {\n            return Ok(StatusResponse::completed(Command::Store(is_uid))\n                .with_tag(arguments.tag)\n                .into_bytes());\n        }\n\n        // Verify that the user can modify messages in this mailbox.\n        if !self\n            .check_mailbox_acl(\n                mailbox.id.account_id,\n                mailbox.id.mailbox_id,\n                Acl::ModifyItems,\n            )\n            .await\n            .imap_ctx(&arguments.tag, trc::location!())?\n        {\n            return Err(trc::ImapEvent::Error\n                .into_err()\n                .details(\n                    \"You do not have the required permissions to modify messages in this mailbox.\",\n                )\n                .id(arguments.tag)\n                .code(ResponseCode::NoPerm)\n                .caused_by(trc::location!()));\n        }\n\n        // Filter out unchanged since ids\n        let mut response_code = None;\n        let mut unchanged_failed = false;\n        if let Some(unchanged_since) = arguments.unchanged_since {\n            // Obtain changes since the modseq.\n            let changelog = self\n                .server\n                .store()\n                .changes(\n                    account_id,\n                    SyncCollection::Email.into(),\n                    Query::from_modseq(unchanged_since),\n                )\n                .await\n                .imap_ctx(&arguments.tag, trc::location!())?;\n\n            let mut modified = mailbox\n                .sequence_expand_missing(&arguments.sequence_set, is_uid)\n                .await;\n\n            // Add all IDs that changed in this mailbox\n            for (id, is_delete) in changelog.changes.into_iter().filter_map(|change| {\n                change.item_id().map(|id| {\n                    (\n                        (id & u32::MAX as u64) as u32,\n                        matches!(change, Change::DeleteItem(_)),\n                    )\n                })\n            }) {\n                if let Some(imap_id) = ids.remove(&id) {\n                    if is_uid {\n                        modified.push(imap_id.uid);\n                    } else {\n                        modified.push(imap_id.seqnum);\n                        if is_delete {\n                            unchanged_failed = true;\n                        }\n                    }\n                }\n            }\n\n            if !modified.is_empty() {\n                modified.sort_unstable();\n                response_code = ResponseCode::Modified { ids: modified }.into();\n            }\n        }\n\n        // Build response\n        let mut response = if !unchanged_failed {\n            StatusResponse::completed(Command::Store(is_uid))\n        } else {\n            StatusResponse::no(\"Some of the messages no longer exist.\")\n        }\n        .with_tag(arguments.tag);\n        if let Some(response_code) = response_code {\n            response = response.with_code(response_code)\n        }\n        if ids.is_empty() {\n            trc::event!(\n                Imap(trc::ImapEvent::Store),\n                SpanId = self.session_id,\n                AccountId = mailbox.id.account_id,\n                MailboxId = mailbox.id.mailbox_id,\n                Type = format!(\"{:?}\", arguments.operation),\n                Details = arguments\n                    .keywords\n                    .iter()\n                    .map(|c| trc::Value::from(format!(\"{c:?}\")))\n                    .collect::<Vec<_>>(),\n                Elapsed = op_start.elapsed()\n            );\n\n            return Ok(response.into_bytes());\n        }\n        let mut items = Response {\n            items: Vec::with_capacity(ids.len()),\n        };\n\n        // Process each change\n        let set_keywords = arguments\n            .keywords\n            .iter()\n            .map(|k| Keyword::from(k.clone()))\n            .collect::<Vec<_>>();\n        let mut changed_mailboxes = AHashSet::new();\n        let mut batch = BatchBuilder::new();\n\n        for (id, imap_id) in &ids {\n            // Obtain message data\n            let data_ = if let Some(data) = self\n                .server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Email,\n                    *id,\n                ))\n                .await\n                .imap_ctx(response.tag.as_ref().unwrap(), trc::location!())?\n            {\n                data\n            } else {\n                continue;\n            };\n\n            // Deserialize\n            let data = data_\n                .to_unarchived::<MessageData>()\n                .imap_ctx(response.tag.as_ref().unwrap(), trc::location!())?;\n            let mut new_data = data.inner.to_builder();\n\n            // Apply changes\n            let mut seen_changed = false;\n            match arguments.operation {\n                Operation::Set => {\n                    seen_changed = set_keywords.contains(&Keyword::Seen)\n                        != new_data.has_keyword(&Keyword::Seen);\n                    new_data.set_keywords(set_keywords.clone());\n                }\n                Operation::Add => {\n                    for keyword in &set_keywords {\n                        if new_data.add_keyword(keyword.clone()) && keyword == &Keyword::Seen {\n                            seen_changed = true;\n                        }\n                    }\n                }\n                Operation::Clear => {\n                    for keyword in &set_keywords {\n                        if new_data.remove_keyword(keyword) && keyword == &Keyword::Seen {\n                            seen_changed = true;\n                        }\n                    }\n                }\n            }\n\n            if !new_data.has_keyword_changes(data.inner) {\n                continue;\n            }\n\n            // Train spam filter\n            let mut train_spam = None;\n            for keyword in new_data.added_keywords(data.inner) {\n                if keyword == &Keyword::Junk {\n                    train_spam = Some(true);\n                    break;\n                } else if keyword == &Keyword::NotJunk && !data.inner.has_mailbox_id(TRASH_ID) {\n                    // Only train as ham if not in Trash (Apple likes to add NotJunk to trashed items, which would be spammy)\n                    train_spam = Some(false);\n                    break;\n                }\n            }\n            if train_spam.is_none() {\n                for keyword in new_data.removed_keywords(data.inner) {\n                    if keyword == &Keyword::Junk {\n                        if !data.inner.has_mailbox_id(TRASH_ID) {\n                            train_spam = Some(false);\n                        }\n                        break;\n                    }\n                }\n            }\n\n            // Convert keywords to flags\n            let flags = if !arguments.is_silent {\n                new_data\n                    .keywords\n                    .iter()\n                    .cloned()\n                    .map(Flag::from)\n                    .collect::<Vec<_>>()\n            } else {\n                vec![]\n            };\n\n            // Set all current mailboxes as changed if the Seen tag changed\n            if seen_changed {\n                for mailbox_id in new_data.mailboxes.iter() {\n                    changed_mailboxes.insert(mailbox_id.mailbox_id);\n                }\n            }\n\n            // Write changes\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Email)\n                .with_document(*id)\n                .custom(\n                    ObjectIndexBuilder::new()\n                        .with_current(data)\n                        .with_changes(new_data.seal()),\n                )\n                .imap_ctx(response.tag.as_ref().unwrap(), trc::location!())?;\n\n            // Add spam train task\n            if let Some(learn_spam) = train_spam {\n                self.server\n                    .add_account_spam_sample(\n                        &mut batch,\n                        account_id,\n                        *id,\n                        learn_spam,\n                        self.session_id,\n                    )\n                    .await\n                    .imap_ctx(response.tag.as_ref().unwrap(), trc::location!())?;\n            }\n\n            // Set commit point\n            batch.commit_point();\n\n            // Add item to response\n            if !arguments.is_silent {\n                let mut data_items = vec![DataItem::Flags { flags }];\n                if is_uid {\n                    data_items.push(DataItem::Uid { uid: imap_id.uid });\n                }\n                items.items.push(FetchItem {\n                    id: imap_id.seqnum,\n                    items: data_items,\n                });\n            } else if is_condstore {\n                items.items.push(FetchItem {\n                    id: imap_id.seqnum,\n                    items: if is_uid {\n                        vec![DataItem::Uid { uid: imap_id.uid }]\n                    } else {\n                        vec![]\n                    },\n                });\n            }\n        }\n\n        // Log mailbox changes\n        if !changed_mailboxes.is_empty() {\n            for parent_id in changed_mailboxes {\n                batch.log_container_property_change(SyncCollection::Email, parent_id);\n            }\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            match self\n                .server\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(mailbox.id.account_id))\n                .caused_by(trc::location!())\n            {\n                Ok(change_id) => {\n                    if is_condstore {\n                        let modseq = change_id + 1;\n                        for item in items.items.iter_mut() {\n                            item.items.push(DataItem::ModSeq { modseq });\n                        }\n                    }\n                }\n                Err(err) if err.is_assertion_failure() => {\n                    items.items.clear();\n                    response.rtype = ResponseType::No;\n                    response.message = \"Some messages were modified by another process.\".into();\n                }\n                Err(err) => {\n                    return Err(err.id(response.tag.unwrap()));\n                }\n            }\n        }\n\n        trc::event!(\n            Imap(trc::ImapEvent::Store),\n            SpanId = self.session_id,\n            AccountId = mailbox.id.account_id,\n            MailboxId = mailbox.id.mailbox_id,\n            DocumentId = ids\n                .iter()\n                .map(|id| trc::Value::from(*id.0))\n                .collect::<Vec<_>>(),\n            Type = format!(\"{:?}\", arguments.operation),\n            Details = arguments\n                .keywords\n                .iter()\n                .map(|c| trc::Value::from(format!(\"{c:?}\")))\n                .collect::<Vec<_>>(),\n            Elapsed = op_start.elapsed()\n        );\n\n        // Send response\n        Ok(response.serialize(items.serialize()))\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/subscribe.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ImapContext;\nuse crate::{\n    core::{Session, SessionData},\n    spawn_op,\n};\nuse common::{listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse imap_proto::{Command, ResponseCode, StatusResponse, receiver::Request};\nuse std::time::Instant;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse types::collection::Collection;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_subscribe(\n        &mut self,\n        request: Request<Command>,\n        is_subscribe: bool,\n    ) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapSubscribe)?;\n\n        let op_start = Instant::now();\n        let arguments = request.parse_subscribe(self.is_utf8)?;\n        let data = self.state.session_data();\n\n        spawn_op!(data, {\n            let response = data\n                .subscribe_folder(\n                    arguments.tag,\n                    arguments.mailbox_name,\n                    is_subscribe,\n                    op_start,\n                )\n                .await?;\n\n            data.write_bytes(response.into_bytes()).await\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn subscribe_folder(\n        &self,\n        tag: String,\n        mailbox_name: String,\n        subscribe: bool,\n        op_start: Instant,\n    ) -> trc::Result<StatusResponse> {\n        // Refresh mailboxes\n        self.synchronize_mailboxes(false)\n            .await\n            .imap_ctx(&tag, trc::location!())?;\n\n        // Validate mailbox\n        let (account_id, mailbox_id) = match self.get_mailbox_by_name(&mailbox_name) {\n            Some(mailbox) => (mailbox.account_id, mailbox.mailbox_id),\n            None => {\n                return Err(trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Mailbox does not exist.\")\n                    .code(ResponseCode::NonExistent)\n                    .id(tag)\n                    .caused_by(trc::location!()));\n            }\n        };\n\n        // Verify if mailbox is already subscribed/unsubscribed\n        for account in self.mailboxes.lock().iter_mut() {\n            if account.account_id == account_id {\n                if let Some(mailbox) = account.mailbox_state.get(&mailbox_id)\n                    && mailbox.is_subscribed == subscribe\n                {\n                    return Err(trc::ImapEvent::Error\n                        .into_err()\n                        .details(if subscribe {\n                            \"Mailbox is already subscribed.\"\n                        } else {\n                            \"Mailbox is already unsubscribed.\"\n                        })\n                        .id(tag));\n                }\n                break;\n            }\n        }\n\n        // Obtain mailbox\n        let mailbox_ = self\n            .server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::Mailbox,\n                mailbox_id,\n            ))\n            .await\n            .imap_ctx(&tag, trc::location!())?\n            .ok_or_else(|| {\n                trc::ImapEvent::Error\n                    .into_err()\n                    .details(\"Mailbox does not exist.\")\n                    .code(ResponseCode::NonExistent)\n                    .id(tag.clone())\n                    .caused_by(trc::location!())\n            })?;\n        let mailbox = mailbox_\n            .to_unarchived::<email::mailbox::Mailbox>()\n            .imap_ctx(&tag, trc::location!())?;\n\n        if (subscribe && !mailbox.inner.is_subscribed(self.account_id))\n            || (!subscribe && mailbox.inner.is_subscribed(self.account_id))\n        {\n            // Build batch\n            let mut new_mailbox = mailbox.deserialize().imap_ctx(&tag, trc::location!())?;\n            if subscribe {\n                new_mailbox.subscribers.push(self.account_id);\n            } else {\n                new_mailbox.remove_subscriber(self.account_id);\n            }\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Mailbox)\n                .with_document(mailbox_id)\n                .custom(\n                    ObjectIndexBuilder::new()\n                        .with_current(mailbox)\n                        .with_changes(new_mailbox),\n                )\n                .imap_ctx(&tag, trc::location!())?;\n            self.server\n                .commit_batch(batch)\n                .await\n                .imap_ctx(&tag, trc::location!())?;\n\n            // Update mailbox cache\n            for account in self.mailboxes.lock().iter_mut() {\n                if account.account_id == account_id {\n                    if let Some(mailbox) = account.mailbox_state.get_mut(&mailbox_id) {\n                        mailbox.is_subscribed = subscribe;\n                    }\n                    break;\n                }\n            }\n        }\n\n        trc::event!(\n            Imap(if subscribe {\n                trc::ImapEvent::Subscribe\n            } else {\n                trc::ImapEvent::Unsubscribe\n            }),\n            SpanId = self.session_id,\n            AccountId = account_id,\n            MailboxId = mailbox_id,\n            MailboxName = mailbox_name,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::ok(if subscribe {\n            \"Mailbox subscribed.\"\n        } else {\n            \"Mailbox unsubscribed.\"\n        })\n        .with_tag(tag))\n    }\n}\n"
  },
  {
    "path": "crates/imap/src/op/thread.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{SelectedMailbox, Session, SessionData},\n    spawn_op,\n};\nuse ahash::AHashMap;\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::cache::{MessageCacheFetch, email::MessageCacheAccess};\nuse imap_proto::{\n    Command, StatusResponse,\n    protocol::{\n        ImapResponse,\n        thread::{Arguments, Response},\n    },\n    receiver::Request,\n};\nuse std::{sync::Arc, time::Instant};\nuse trc::AddContext;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_thread(\n        &mut self,\n        request: Request<Command>,\n        is_uid: bool,\n    ) -> trc::Result<()> {\n        // Validate access\n        self.assert_has_permission(Permission::ImapThread)?;\n\n        let op_start = Instant::now();\n        let command = request.command;\n        let mut arguments = request.parse_thread()?;\n        let (data, mailbox) = self.state.mailbox_state();\n\n        spawn_op!(data, {\n            let tag = std::mem::take(&mut arguments.tag);\n\n            match data.thread(arguments, mailbox, is_uid, op_start).await {\n                Ok(response) => {\n                    data.write_bytes(\n                        StatusResponse::completed(command)\n                            .with_tag(tag)\n                            .serialize(response.serialize()),\n                    )\n                    .await\n                }\n                Err(err) => Err(err.id(tag)),\n            }\n        })\n    }\n}\n\nimpl<T: SessionStream> SessionData<T> {\n    pub async fn thread(\n        &self,\n        arguments: Arguments,\n        mailbox: Arc<SelectedMailbox>,\n        is_uid: bool,\n        op_start: Instant,\n    ) -> trc::Result<Response> {\n        // Run query\n        let (result_set, _) = self\n            .query(arguments.filter, vec![], &mailbox, &None)\n            .await?;\n\n        // Synchronize mailbox\n        if !result_set.is_empty() {\n            self.synchronize_messages(&mailbox)\n                .await\n                .caused_by(trc::location!())?;\n        } else {\n            return Ok(Response {\n                is_uid,\n                threads: vec![],\n            });\n        }\n\n        // Lock the cache\n        let cache = self\n            .server\n            .get_cached_messages(mailbox.id.account_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        // Group messages by thread\n        let mut threads: AHashMap<u32, Vec<u32>> = AHashMap::new();\n        let state = mailbox.state.lock();\n        for document_id in result_set {\n            if let Some(item) = cache.email_by_id(&document_id)\n                && let Some((imap_id, _)) = state.map_result_id(document_id, is_uid)\n            {\n                threads.entry(item.thread_id).or_default().push(imap_id);\n            }\n        }\n\n        let mut threads = threads\n            .into_iter()\n            .map(|(_, mut messages)| {\n                messages.sort_unstable();\n                messages\n            })\n            .collect::<Vec<_>>();\n        threads.sort_unstable();\n\n        trc::event!(\n            Imap(trc::ImapEvent::Thread),\n            SpanId = self.session_id,\n            AccountId = mailbox.id.account_id,\n            MailboxId = mailbox.id.mailbox_id,\n            Total = threads.len(),\n            Elapsed = op_start.elapsed()\n        );\n\n        // Build response\n        Ok(Response { is_uid, threads })\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/Cargo.toml",
    "content": "[package]\nname = \"imap_proto\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\ntypes = { path = \"../types\" }\nutils = { path = \"../utils\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] } \nahash = { version = \"0.8\" }\nchrono = { version = \"0.4\"}\ntrc = { path = \"../trc\" }\nhashify = { version = \"0.2\" }\ncompact_str = \"0.9.0\"\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/imap-proto/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse protocol::capability::Capability;\nuse std::borrow::Cow;\n\npub mod parser;\npub mod protocol;\npub mod receiver;\npub mod utf7;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]\npub enum Command {\n    // Client Commands - Any State\n    Capability,\n    #[default]\n    Noop,\n    Logout,\n\n    // Client Commands - Not Authenticated State\n    StartTls,\n    Authenticate,\n    Login,\n\n    // Client Commands - Authenticated State\n    Enable,\n    Select,\n    Examine,\n    Create,\n    Delete,\n    Rename,\n    Subscribe,\n    Unsubscribe,\n    List,\n    Namespace,\n    Status,\n    Append,\n    Idle,\n\n    // Client Commands - Selected State\n    Close,\n    Unselect,\n    Expunge(bool),\n    Search(bool),\n    Fetch(bool),\n    Store(bool),\n    Copy(bool),\n    Move(bool),\n\n    // IMAP4rev1\n    Lsub,\n    Check,\n\n    // RFC 5256\n    Sort(bool),\n    Thread(bool),\n\n    // RFC 4314\n    SetAcl,\n    DeleteAcl,\n    GetAcl,\n    ListRights,\n    MyRights,\n\n    // RFC 8437\n    Unauthenticate,\n\n    // RFC 2971\n    Id,\n\n    // RFC 9208\n    GetQuota,\n    GetQuotaRoot,\n}\n\nimpl Command {\n    pub fn is_uid(&self) -> bool {\n        matches!(\n            self,\n            Command::Fetch(true)\n                | Command::Search(true)\n                | Command::Copy(true)\n                | Command::Move(true)\n                | Command::Store(true)\n                | Command::Expunge(true)\n                | Command::Sort(true)\n                | Command::Thread(true)\n        )\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ResponseCode {\n    Alert,\n    AlreadyExists,\n    AppendUid {\n        uid_validity: u32,\n        uids: Vec<u32>,\n    },\n    AuthenticationFailed,\n    AuthorizationFailed,\n    BadCharset,\n    Cannot,\n    Capability {\n        capabilities: Vec<Capability>,\n    },\n    ClientBug,\n    Closed,\n    ContactAdmin,\n    CopyUid {\n        uid_validity: u32,\n        src_uids: Vec<u32>,\n        dest_uids: Vec<u32>,\n    },\n    Corruption,\n    Expired,\n    ExpungeIssued,\n    HasChildren,\n    InUse,\n    Limit,\n    NonExistent,\n    NoPerm,\n    OverQuota,\n    Parse,\n    PermanentFlags,\n    PrivacyRequired,\n    ReadOnly,\n    ReadWrite,\n    ServerBug,\n    TryCreate,\n    UidNext,\n    UidNotSticky,\n    UidValidity,\n    Unavailable,\n    UnknownCte,\n\n    // CONDSTORE\n    Modified {\n        ids: Vec<u32>,\n    },\n    HighestModseq {\n        modseq: u64,\n    },\n\n    // ObjectID\n    MailboxId {\n        mailbox_id: String,\n    },\n\n    // USEATTR\n    UseAttr,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct StatusResponse {\n    pub tag: Option<String>,\n    pub code: Option<ResponseCode>,\n    pub message: Cow<'static, str>,\n    pub rtype: ResponseType,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ResponseType {\n    Ok,\n    No,\n    Bad,\n    PreAuth,\n    Bye,\n}\n\nimpl ResponseCode {\n    pub fn highest_modseq(modseq: u64) -> Self {\n        ResponseCode::HighestModseq {\n            modseq: if modseq > 0 { modseq + 1 } else { 0 },\n        }\n    }\n}\n\nimpl StatusResponse {\n    pub fn bad(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            tag: None,\n            code: None,\n            message: message.into(),\n            rtype: ResponseType::Bad,\n        }\n    }\n\n    pub fn parse_error(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            tag: None,\n            code: ResponseCode::Parse.into(),\n            message: message.into(),\n            rtype: ResponseType::Bad,\n        }\n    }\n\n    pub fn database_failure() -> Self {\n        StatusResponse::no(\"Database failure.\").with_code(ResponseCode::ContactAdmin)\n    }\n\n    pub fn completed(command: Command) -> Self {\n        StatusResponse::ok(format!(\"{} completed\", command))\n    }\n\n    pub fn with_code(mut self, code: ResponseCode) -> Self {\n        self.code = Some(code);\n        self\n    }\n\n    pub fn with_tag(mut self, tag: impl Into<String>) -> Self {\n        self.tag = Some(tag.into());\n        self\n    }\n\n    pub fn no(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            tag: None,\n            code: None,\n            message: message.into(),\n            rtype: ResponseType::No,\n        }\n    }\n\n    pub fn ok(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            tag: None,\n            code: None,\n            message: message.into(),\n            rtype: ResponseType::Ok,\n        }\n    }\n\n    pub fn bye(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            tag: None,\n            code: None,\n            message: message.into(),\n            rtype: ResponseType::Bye,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::acl::{self, ModRights, ModRightsOp, Rights},\n    receiver::{Request, bad},\n    utf7::utf7_maybe_decode,\n};\n\nuse super::PushUnique;\n\n/*\n\n   setacl          = \"SETACL\" SP mailbox SP identifier\n                       SP mod-rights\n\n   deleteacl       = \"DELETEACL\" SP mailbox SP identifier\n\n   getacl          = \"GETACL\" SP mailbox\n\n   listrights      = \"LISTRIGHTS\" SP mailbox SP identifier\n\n   myrights        = \"MYRIGHTS\" SP mailbox\n\n*/\n\nimpl Request<Command> {\n    pub fn parse_acl(self, is_utf8: bool) -> trc::Result<acl::Arguments> {\n        let (has_identifier, has_mod_rights) = match self.command {\n            Command::SetAcl => (true, true),\n            Command::DeleteAcl | Command::ListRights => (true, false),\n            Command::GetAcl | Command::MyRights => (false, false),\n            _ => unreachable!(),\n        };\n        let mut tokens = self.tokens.into_iter();\n        let mailbox_name = utf7_maybe_decode(\n            tokens\n                .next()\n                .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing mailbox name.\"))?\n                .unwrap_string()\n                .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n            is_utf8,\n        );\n        let identifier = if has_identifier {\n            tokens\n                .next()\n                .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing identifier.\"))?\n                .unwrap_string()\n                .map_err(|v| bad(self.tag.to_compact_string(), v))?\n                .into()\n        } else {\n            None\n        };\n        let mod_rights = if has_mod_rights {\n            ModRights::parse(\n                &tokens\n                    .next()\n                    .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing rights.\"))?\n                    .unwrap_bytes(),\n            )\n            .map_err(|v| bad(self.tag.to_compact_string(), v))?\n            .into()\n        } else {\n            None\n        };\n\n        Ok(acl::Arguments {\n            tag: self.tag,\n            mailbox_name,\n            identifier,\n            mod_rights,\n        })\n    }\n}\n\nimpl ModRights {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        let mut op = ModRightsOp::Replace;\n        let mut rights = Vec::with_capacity(value.len());\n        for (pos, ch) in value.iter().enumerate() {\n            rights.push_unique(match ch {\n                b'l' => Rights::Lookup,\n                b'r' => Rights::Read,\n                b's' => Rights::Seen,\n                b'w' => Rights::Write,\n                b'i' => Rights::Insert,\n                b'p' => Rights::Post,\n                b'k' => Rights::CreateMailbox,\n                b'x' => Rights::DeleteMailbox,\n                b't' => Rights::DeleteMessages,\n                b'e' => Rights::Expunge,\n                b'a' => Rights::Administer,\n                // RFC2086\n                b'd' => Rights::DeleteMessages,\n                b'c' => Rights::CreateMailbox,\n                b'+' if pos == 0 => {\n                    op = ModRightsOp::Add;\n                    continue;\n                }\n                b'-' if pos == 0 => {\n                    op = ModRightsOp::Remove;\n                    continue;\n                }\n                _ => {\n                    return Err(\n                        format!(\"Invalid character {:?} in rights.\", char::from(*ch)).into(),\n                    );\n                }\n            })\n        }\n\n        if !rights.is_empty() {\n            Ok(ModRights { op, rights })\n        } else {\n            Err(\"At least one right has to be specified.\".into())\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::{\n        protocol::acl::{self, ModRights, ModRightsOp, Rights},\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_acl() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A003 Setacl INBOX/Drafts Byron lrswikda\\r\\n\",\n                acl::Arguments {\n                    tag: \"A003\".into(),\n                    mailbox_name: \"INBOX/Drafts\".into(),\n                    identifier: Some(\"Byron\".into()),\n                    mod_rights: ModRights {\n                        op: ModRightsOp::Replace,\n                        rights: vec![\n                            Rights::Lookup,\n                            Rights::Read,\n                            Rights::Seen,\n                            Rights::Write,\n                            Rights::Insert,\n                            Rights::CreateMailbox,\n                            Rights::DeleteMessages,\n                            Rights::Administer,\n                        ],\n                    }\n                    .into(),\n                },\n            ),\n            (\n                \"A002 SETACL INBOX/Drafts Chris +cda\\r\\n\",\n                acl::Arguments {\n                    tag: \"A002\".into(),\n                    mailbox_name: \"INBOX/Drafts\".into(),\n                    identifier: Some(\"Chris\".into()),\n                    mod_rights: ModRights {\n                        op: ModRightsOp::Add,\n                        rights: vec![\n                            Rights::CreateMailbox,\n                            Rights::DeleteMessages,\n                            Rights::Administer,\n                        ],\n                    }\n                    .into(),\n                },\n            ),\n            (\n                \"A036 SETACL INBOX/Drafts John -lrswicda\\r\\n\",\n                acl::Arguments {\n                    tag: \"A036\".into(),\n                    mailbox_name: \"INBOX/Drafts\".into(),\n                    identifier: Some(\"John\".into()),\n                    mod_rights: ModRights {\n                        op: ModRightsOp::Remove,\n                        rights: vec![\n                            Rights::Lookup,\n                            Rights::Read,\n                            Rights::Seen,\n                            Rights::Write,\n                            Rights::Insert,\n                            Rights::CreateMailbox,\n                            Rights::DeleteMessages,\n                            Rights::Administer,\n                        ],\n                    }\n                    .into(),\n                },\n            ),\n            (\n                \"A001 GETACL INBOX/Drafts\\r\\n\",\n                acl::Arguments {\n                    tag: \"A001\".into(),\n                    mailbox_name: \"INBOX/Drafts\".into(),\n                    identifier: None,\n                    mod_rights: None,\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_acl(false)\n                    .unwrap(),\n                arguments,\n                \"{:?}\",\n                command\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/append.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::{\n        Flag,\n        append::{self, Message},\n    },\n    receiver::{Request, Token, bad},\n    utf7::utf7_maybe_decode,\n};\n\nuse super::parse_datetime;\n\nenum State {\n    None,\n    Flags,\n    UTF8,\n    UTF8Data,\n}\n\nimpl Request<Command> {\n    pub fn parse_append(self, is_utf8: bool) -> trc::Result<append::Arguments> {\n        match self.tokens.len() {\n            0 | 1 => Err(self.into_error(\"Missing arguments.\")),\n            _ => {\n                // Obtain mailbox name\n                let mut tokens = self.tokens.into_iter().peekable();\n                let mailbox_name = utf7_maybe_decode(\n                    tokens\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    is_utf8,\n                );\n                let mut messages = Vec::new();\n\n                while tokens.peek().is_some() {\n                    // Parse flags\n                    let mut message = Message {\n                        message: vec![],\n                        flags: vec![],\n                        received_at: None,\n                    };\n                    let mut state = State::None;\n                    let mut seen_flags = false;\n\n                    while let Some(token) = tokens.next() {\n                        match token {\n                            Token::ParenthesisOpen => {\n                                state = match state {\n                                    State::None if !seen_flags => {\n                                        seen_flags = true;\n                                        State::Flags\n                                    }\n                                    State::UTF8 => State::UTF8Data,\n                                    _ => {\n                                        return Err(bad(\n                                            self.tag.to_compact_string(),\n                                            \"Invalid opening parenthesis found.\",\n                                        ));\n                                    }\n                                };\n                            }\n                            Token::ParenthesisClose => match state {\n                                State::None | State::UTF8 => {\n                                    return Err(bad(\n                                        self.tag.to_compact_string(),\n                                        \"Invalid closing parenthesis found.\",\n                                    ));\n                                }\n                                State::Flags => {\n                                    state = State::None;\n                                }\n                                State::UTF8Data => {\n                                    break;\n                                }\n                            },\n                            Token::Argument(value) => match state {\n                                State::None => {\n                                    if value.eq_ignore_ascii_case(b\"utf8\") {\n                                        state = State::UTF8;\n                                    } else if matches!(tokens.peek(), Some(Token::Argument(_)))\n                                        && value.len() <= 28\n                                        && !value.contains(&b'\\n')\n                                    {\n                                        if let Ok(date_time) = parse_datetime(&value) {\n                                            message.received_at = Some(date_time);\n                                        } else {\n                                            return Err(bad(\n                                                self.tag.to_compact_string(),\n                                                \"Failed to parse received time.\",\n                                            ));\n                                        }\n                                    } else {\n                                        message.message = value;\n                                        break;\n                                    }\n                                }\n                                State::Flags => {\n                                    message.flags.push(\n                                        Flag::parse_imap(value)\n                                            .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                    );\n                                }\n                                State::UTF8 => {\n                                    return Err(bad(\n                                        self.tag.to_compact_string(),\n                                        \"Expected parenthesis after UTF8.\",\n                                    ));\n                                }\n                                State::UTF8Data => {\n                                    if message.message.is_empty() {\n                                        message.message = value;\n                                    } else {\n                                        return Err(bad(\n                                            self.tag.to_compact_string(),\n                                            \"Invalid parameter after message literal.\",\n                                        ));\n                                    }\n                                }\n                            },\n                            _ => {\n                                return Err(bad(\n                                    self.tag.to_compact_string(),\n                                    \"Invalid arguments.\",\n                                ));\n                            }\n                        }\n                    }\n\n                    messages.push(message);\n                }\n\n                Ok(append::Arguments {\n                    tag: self.tag,\n                    mailbox_name,\n                    messages,\n                })\n            }\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::{\n        protocol::{\n            Flag,\n            append::{self, Message},\n        },\n        receiver::{Error, Receiver},\n    };\n\n    #[test]\n    fn parse_append() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A003 APPEND saved-messages (\\\\Seen) {1+}\\r\\na\\r\\n\",\n                append::Arguments {\n                    tag: \"A003\".into(),\n                    mailbox_name: \"saved-messages\".into(),\n                    messages: vec![Message {\n                        message: vec![b'a'],\n                        flags: vec![Flag::Seen],\n                        received_at: None,\n                    }],\n                },\n            ),\n            (\n                \"A003 APPEND \\\"hello world\\\" (\\\\Seen \\\\Draft $MDNSent) {1+}\\r\\na\\r\\n\",\n                append::Arguments {\n                    tag: \"A003\".into(),\n                    mailbox_name: \"hello world\".into(),\n                    messages: vec![Message {\n                        message: vec![b'a'],\n                        flags: vec![Flag::Seen, Flag::Draft, Flag::MDNSent],\n                        received_at: None,\n                    }],\n                },\n            ),\n            (\n                \"A003 APPEND \\\"hi\\\" ($Junk) \\\"7-Feb-1994 22:43:04 -0800\\\" {1+}\\r\\na\\r\\n\",\n                append::Arguments {\n                    tag: \"A003\".into(),\n                    mailbox_name: \"hi\".into(),\n                    messages: vec![Message {\n                        message: vec![b'a'],\n                        flags: vec![Flag::Junk],\n                        received_at: Some(760689784),\n                    }],\n                },\n            ),\n            (\n                \"A003 APPEND \\\"hi\\\" \\\"20-Nov-2022 23:59:59 +0300\\\" {1+}\\r\\na\\r\\n\",\n                append::Arguments {\n                    tag: \"A003\".into(),\n                    mailbox_name: \"hi\".into(),\n                    messages: vec![Message {\n                        message: vec![b'a'],\n                        flags: vec![],\n                        received_at: Some(1668977999),\n                    }],\n                },\n            ),\n            (\n                \"A003 APPEND \\\"hi\\\" \\\"20-Nov-2022 23:59:59 +0300\\\" ~{1+}\\r\\na\\r\\n\",\n                append::Arguments {\n                    tag: \"A003\".into(),\n                    mailbox_name: \"hi\".into(),\n                    messages: vec![Message {\n                        message: vec![b'a'],\n                        flags: vec![],\n                        received_at: Some(1668977999),\n                    }],\n                },\n            ),\n            (\n                \"42 APPEND \\\"Drafts\\\" (\\\\Draft) UTF8 (~{5+}\\r\\nhello)\\r\\n\",\n                append::Arguments {\n                    tag: \"42\".into(),\n                    mailbox_name: \"Drafts\".into(),\n                    messages: vec![Message {\n                        message: vec![b'h', b'e', b'l', b'l', b'o'],\n                        flags: vec![Flag::Draft],\n                        received_at: None,\n                    }],\n                },\n            ),\n            (\n                \"42 APPEND \\\"Drafts\\\" (\\\\Draft) \\\"20-Nov-2022 23:59:59 +0300\\\" UTF8 (~{5+}\\r\\nhello)\\r\\n\",\n                append::Arguments {\n                    tag: \"42\".into(),\n                    mailbox_name: \"Drafts\".into(),\n                    messages: vec![Message {\n                        message: vec![b'h', b'e', b'l', b'l', b'o'],\n                        flags: vec![Flag::Draft],\n                        received_at: Some(1668977999),\n                    }],\n                },\n            ),\n            (\n                \"A003 APPEND \\\"&A8g- \\\\\\\"&A9QD1APUA9gD3APcA-+\\\\\\\"\\\" (\\\\Seen) \\\"7-Feb-1994 22:43:04 -0800\\\" {1+}\\r\\na\\r\\n\",\n                append::Arguments {\n                    tag: \"A003\".into(),\n                    mailbox_name: \"ψ \\\"ϔϔϔϘϜϜ+\\\"\".into(),\n                    messages: vec![Message {\n                        message: vec![b'a'],\n                        flags: vec![Flag::Seen],\n                        received_at: Some(760689784),\n                    }],\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .expect(command)\n                    .parse_append(false)\n                    .expect(command),\n                arguments,\n                \"{:?}\",\n                command\n            );\n        }\n\n        // Multiappend\n        for line in [\n            \"A003 APPEND saved-messages (\\\\Seen) UTF8 ({329}\\r\\n\",\n            \"Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)\\r\\n\",\n            \"From: Fred Foobar <foobar@Blurdybloop.example.COM>\\r\\n\",\n            \"Subject: afternoon meeting\\r\\n\",\n            \"To: mooch@owatagu.example.net\\r\\n\",\n            \"Message-Id: <B27397-0100000@Blurdybloop.example.COM>\\r\\n\",\n            \"MIME-Version: 1.0\\r\\n\",\n            \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\",\n            \"\\r\\n\",\n            \"Hello Joe, do you think we can meet at 3:30 tomorrow?\\r\\n)\",\n            \" (\\\\Seen) \\\"7-Feb-1994 22:43:04 -0800\\\" {295}\\r\\n\",\n            \"Date: Mon, 7 Feb 1994 22:43:04 -0800 (PST)\\r\\n\",\n            \"From: Joe Mooch <mooch@OWaTaGu.example.net>\\r\\n\",\n            \"Subject: Re: afternoon meeting\\r\\n\",\n            \"To: foobar@blurdybloop.example.com\\r\\n\",\n            \"Message-Id: <a0434793874930@OWaTaGu.example.net>\\r\\n\",\n            \"MIME-Version: 1.0\\r\\n\",\n            \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\\r\\n\",\n            \"3:30 is fine with me.\\r\\n\\r\\n\",\n        ] {\n            match receiver.parse(&mut line.as_bytes().iter()) {\n                Ok(request) => {\n                    assert_eq!(\n                        request.parse_append(false).unwrap(),\n                        append::Arguments {\n                            tag: \"A003\".into(),\n                            mailbox_name: \"saved-messages\".into(),\n                            messages: vec![\n                                Message {\n                                    message: concat!(\n                                        \"Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)\\r\\n\",\n                                        \"From: Fred Foobar <foobar@Blurdybloop.example.COM>\\r\\n\",\n                                        \"Subject: afternoon meeting\\r\\n\",\n                                        \"To: mooch@owatagu.example.net\\r\\n\",\n                                        \"Message-Id: <B27397-0100000@Blurdybloop.example.COM>\\r\\n\",\n                                        \"MIME-Version: 1.0\\r\\n\",\n                                        \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\",\n                                        \"\\r\\n\",\n                                        \"Hello Joe, do you think we can meet at 3:30 tomorrow?\\r\\n\",\n                                    )\n                                    .as_bytes()\n                                    .to_vec(),\n                                    flags: vec![Flag::Seen],\n                                    received_at: None,\n                                },\n                                Message {\n                                    message: concat!(\n                                        \"Date: Mon, 7 Feb 1994 22:43:04 -0800 (PST)\\r\\n\",\n                                        \"From: Joe Mooch <mooch@OWaTaGu.example.net>\\r\\n\",\n                                        \"Subject: Re: afternoon meeting\\r\\n\",\n                                        \"To: foobar@blurdybloop.example.com\\r\\n\",\n                                        \"Message-Id: <a0434793874930@OWaTaGu.example.net>\\r\\n\",\n                                        \"MIME-Version: 1.0\\r\\n\",\n                                        \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\\r\\n\",\n                                        \"3:30 is fine with me.\\r\\n\",\n                                    )\n                                    .as_bytes()\n                                    .to_vec(),\n                                    flags: vec![Flag::Seen],\n                                    received_at: Some(760689784),\n                                }\n                            ],\n                        },\n                    );\n                }\n                Err(err) => match err {\n                    Error::NeedsMoreData | Error::NeedsLiteral { .. } => (),\n                    Error::Error { response } => panic!(\"{:?}\", response),\n                },\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/authenticate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::authenticate::{self, Mechanism},\n    receiver::{Request, bad},\n};\n\nimpl Request<Command> {\n    pub fn parse_authenticate(self) -> trc::Result<authenticate::Arguments> {\n        if !self.tokens.is_empty() {\n            let mut tokens = self.tokens.into_iter();\n            Ok(authenticate::Arguments {\n                mechanism: Mechanism::parse(&tokens.next().unwrap().unwrap_bytes())\n                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                params: tokens\n                    .filter_map(|token| token.unwrap_string().ok())\n                    .collect(),\n                tag: self.tag,\n            })\n        } else {\n            Err(self.into_error(\"Authentication mechanism missing.\"))\n        }\n    }\n}\n\nimpl Mechanism {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(value,\n            \"PLAIN\" => Self::Plain,\n            \"CRAM-MD5\" => Self::CramMd5,\n            \"DIGEST-MD5\" => Self::DigestMd5,\n            \"SCRAM-SHA-1\" => Self::ScramSha1,\n            \"SCRAM-SHA-256\" => Self::ScramSha256,\n            \"APOP\" => Self::Apop,\n            \"NTLM\" => Self::Ntlm,\n            \"GSSAPI\" => Self::Gssapi,\n            \"ANONYMOUS\" => Self::Anonymous,\n            \"EXTERNAL\" => Self::External,\n            \"OAUTHBEARER\" => Self::OAuthBearer,\n            \"XOAUTH2\" => Self::XOauth2,\n        )\n        .ok_or_else(|| {\n            format!(\n                \"Unsupported mechanism '{}'.\",\n                String::from_utf8_lossy(value)\n            )\n            .into()\n        })\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::authenticate::{self, Mechanism},\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_authenticate() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"a002 AUTHENTICATE \\\"EXTERNAL\\\" {16+}\\r\\nfred@example.com\\r\\n\",\n                authenticate::Arguments {\n                    tag: \"a002\".into(),\n                    mechanism: Mechanism::External,\n                    params: vec![\"fred@example.com\".into()],\n                },\n            ),\n            (\n                \"A01 AUTHENTICATE PLAIN\\r\\n\",\n                authenticate::Arguments {\n                    tag: \"A01\".into(),\n                    mechanism: Mechanism::Plain,\n                    params: vec![],\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_authenticate()\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::copy_move,\n    receiver::{Request, bad},\n    utf7::utf7_maybe_decode,\n};\n\nuse super::parse_sequence_set;\n\nimpl Request<Command> {\n    pub fn parse_copy_move(self, is_utf8: bool) -> trc::Result<copy_move::Arguments> {\n        if self.tokens.len() > 1 {\n            let mut tokens = self.tokens.into_iter();\n\n            Ok(copy_move::Arguments {\n                sequence_set: parse_sequence_set(\n                    &tokens\n                        .next()\n                        .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing sequence set.\"))?\n                        .unwrap_bytes(),\n                )\n                .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                mailbox_name: utf7_maybe_decode(\n                    tokens\n                        .next()\n                        .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing mailbox name.\"))?\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    is_utf8,\n                ),\n                tag: self.tag,\n            })\n        } else {\n            Err(self.into_error(\"Missing arguments.\"))\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::{Sequence, copy_move},\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_copy() {\n        let mut receiver = Receiver::new();\n\n        assert_eq!(\n            receiver\n                .parse(&mut \"A003 COPY 2:4 MEETING\\r\\n\".as_bytes().iter())\n                .unwrap()\n                .parse_copy_move(false)\n                .unwrap(),\n            copy_move::Arguments {\n                sequence_set: Sequence::Range {\n                    start: 2.into(),\n                    end: 4.into(),\n                },\n                mailbox_name: \"MEETING\".into(),\n                tag: \"A003\".into(),\n            }\n        );\n        assert_eq!(\n            receiver\n                .parse(&mut \"A003 COPY 2:4 \\\"You &- Me\\\"\\r\\n\".as_bytes().iter())\n                .unwrap()\n                .parse_copy_move(false)\n                .unwrap(),\n            copy_move::Arguments {\n                sequence_set: Sequence::Range {\n                    start: 2.into(),\n                    end: 4.into(),\n                },\n                mailbox_name: \"You & Me\".into(),\n                tag: \"A003\".into(),\n            }\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/create.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::{CompactString, ToCompactString, format_compact};\n\nuse crate::{\n    Command,\n    protocol::{create, list::Attribute},\n    receiver::{Request, Token, bad},\n    utf7::utf7_maybe_decode,\n};\n\nimpl Request<Command> {\n    pub fn parse_create(self, is_utf8: bool) -> trc::Result<create::Arguments> {\n        if !self.tokens.is_empty() {\n            let mut tokens = self.tokens.into_iter();\n            let mailbox_name = utf7_maybe_decode(\n                tokens\n                    .next()\n                    .unwrap()\n                    .unwrap_string()\n                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                is_utf8,\n            );\n            let mailbox_role = if let Some(Token::ParenthesisOpen) = tokens.next() {\n                match tokens.next() {\n                    Some(Token::Argument(param)) if param.eq_ignore_ascii_case(b\"USE\") => (),\n                    _ => {\n                        return Err(bad(\n                            CompactString::from_string_buffer(self.tag),\n                            \"Failed to parse, expected 'USE'.\",\n                        ));\n                    }\n                }\n                if tokens\n                    .next()\n                    .is_none_or(|token| !token.is_parenthesis_open())\n                {\n                    return Err(bad(\n                        CompactString::from_string_buffer(self.tag),\n                        \"Expected '(' after 'USE'.\",\n                    ));\n                }\n                match tokens.next() {\n                    Some(Token::Argument(value)) => {\n                        let r = hashify::tiny_map_ignore_case!(value.as_slice(),\n                            \"\\\\Archive\" => Some(Attribute::Archive),\n                            \"\\\\Drafts\" => Some(Attribute::Drafts),\n                            \"\\\\Junk\" => Some(Attribute::Junk),\n                            \"\\\\Sent\" => Some(Attribute::Sent),\n                            \"\\\\Trash\" => Some(Attribute::Trash),\n                            \"\\\\Important\" => Some(Attribute::Important),\n                            \"\\\\Memos\" => Some(Attribute::Memos),\n                            \"\\\\Scheduled\" => Some(Attribute::Scheduled),\n                            \"\\\\Snoozed\" => Some(Attribute::Snoozed),\n                            \"\\\\All\" => None,\n                        );\n\n                        match r {\n                            Some(Some(tag)) => Some(tag),\n                            Some(None) => {\n                                return Err(bad(\n                                    CompactString::from_string_buffer(self.tag),\n                                    \"A mailbox with the \\\"\\\\All\\\" attribute already exists.\",\n                                ));\n                            }\n                            None => {\n                                return Err(bad(\n                                    CompactString::from_string_buffer(self.tag),\n                                    format_compact!(\n                                        \"Special use attribute {:?} is not supported.\",\n                                        String::from_utf8_lossy(&value)\n                                    ),\n                                ));\n                            }\n                        }\n                    }\n                    _ => {\n                        return Err(bad(\n                            CompactString::from_string_buffer(self.tag),\n                            \"Invalid SPECIAL-USE attribute.\",\n                        ));\n                    }\n                }\n            } else {\n                None\n            };\n\n            Ok(create::Arguments {\n                mailbox_name,\n                mailbox_role,\n                tag: self.tag,\n            })\n        } else {\n            Err(self.into_error(\"Missing arguments.\"))\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::{\n        protocol::{create, list::Attribute},\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_create() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A142 CREATE 12345\\r\\n\",\n                create::Arguments {\n                    tag: \"A142\".into(),\n                    mailbox_name: \"12345\".into(),\n                    mailbox_role: None,\n                },\n            ),\n            (\n                \"A142 CREATE \\\"my funky mailbox\\\"\\r\\n\",\n                create::Arguments {\n                    tag: \"A142\".into(),\n                    mailbox_name: \"my funky mailbox\".into(),\n                    mailbox_role: None,\n                },\n            ),\n            (\n                \"t1 CREATE \\\"Important Messages\\\" (USE (\\\\Important))\\r\\n\",\n                create::Arguments {\n                    tag: \"t1\".into(),\n                    mailbox_name: \"Important Messages\".into(),\n                    mailbox_role: Some(Attribute::Important),\n                },\n            ),\n            (\n                \"A142 CREATE \\\"Test-ąęć-Test\\\"\\r\\n\",\n                create::Arguments {\n                    tag: \"A142\".into(),\n                    mailbox_name: \"Test-ąęć-Test\".into(),\n                    mailbox_role: None,\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_create(true)\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::delete,\n    receiver::{Request, bad},\n    utf7::utf7_maybe_decode,\n};\n\nimpl Request<Command> {\n    pub fn parse_delete(self, is_utf8: bool) -> trc::Result<delete::Arguments> {\n        match self.tokens.len() {\n            1 => Ok(delete::Arguments {\n                mailbox_name: utf7_maybe_decode(\n                    self.tokens\n                        .into_iter()\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    is_utf8,\n                ),\n                tag: self.tag,\n            }),\n            0 => Err(self.into_error(\"Missing mailbox name.\")),\n            _ => Err(self.into_error(\"Too many arguments.\")),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{protocol::delete, receiver::Receiver};\n\n    #[test]\n    fn parse_delete() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A142 DELETE INBOX\\r\\n\",\n                delete::Arguments {\n                    mailbox_name: \"INBOX\".into(),\n                    tag: \"A142\".into(),\n                },\n            ),\n            (\n                \"A142 DELETE \\\"my funky mailbox\\\"\\r\\n\",\n                delete::Arguments {\n                    mailbox_name: \"my funky mailbox\".into(),\n                    tag: \"A142\".into(),\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_delete(true)\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/enable.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::{capability::Capability, enable},\n    receiver::{Request, bad},\n};\n\nimpl Request<Command> {\n    pub fn parse_enable(self) -> trc::Result<enable::Arguments> {\n        let len = self.tokens.len();\n        if len > 0 {\n            let mut capabilities = Vec::with_capacity(len);\n            for capability in self.tokens {\n                capabilities.push(\n                    Capability::parse(&capability.unwrap_bytes())\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                );\n            }\n            Ok(enable::Arguments {\n                tag: self.tag,\n                capabilities,\n            })\n        } else {\n            Err(self.into_error(\"Missing arguments.\"))\n        }\n    }\n}\n\nimpl Capability {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(value,\n            \"IMAP4rev2\" => Self::IMAP4rev2,\n            \"STARTTLS\" => Self::StartTLS,\n            \"LOGINDISABLED\" => Self::LoginDisabled,\n            \"CONDSTORE\" => Self::CondStore,\n            \"QRESYNC\" => Self::QResync,\n            \"UTF8=ACCEPT\" => Self::Utf8Accept,\n        )\n        .ok_or_else(|| {\n            format!(\n                \"Unsupported capability '{}'.\",\n                String::from_utf8_lossy(value)\n            )\n            .into()\n        })\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::{capability::Capability, enable},\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_enable() {\n        let mut receiver = Receiver::new();\n\n        assert_eq!(\n            receiver\n                .parse(&mut \"t2 ENABLE IMAP4rev2 CONDSTORE\\r\\n\".as_bytes().iter())\n                .unwrap()\n                .parse_enable()\n                .unwrap(),\n            enable::Arguments {\n                tag: \"t2\".into(),\n                capabilities: vec![Capability::IMAP4rev2, Capability::CondStore],\n            }\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/fetch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\nuse std::iter::Peekable;\nuse std::vec::IntoIter;\n\nuse compact_str::{CompactString, ToCompactString, format_compact};\n\nuse crate::{\n    Command,\n    protocol::fetch::{self, Attribute, Section},\n    receiver::{Request, Token, bad},\n};\n\nuse super::{PushUnique, parse_number, parse_sequence_set};\n\nimpl Request<Command> {\n    #[allow(clippy::while_let_on_iterator)]\n    pub fn parse_fetch(self) -> trc::Result<fetch::Arguments> {\n        if self.tokens.len() < 2 {\n            return Err(self.into_error(\"Missing parameters.\"));\n        }\n\n        let mut tokens = self.tokens.into_iter().peekable();\n        let mut attributes = Vec::new();\n        let sequence_set = parse_sequence_set(\n            &tokens\n                .next()\n                .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing sequence set.\"))?\n                .unwrap_bytes(),\n        )\n        .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n\n        let mut in_parentheses = false;\n\n        while let Some(token) = tokens.next() {\n            match token {\n                Token::Argument(value) => {\n                    hashify::fnc_map_ignore_case!(value.as_slice(),\n                        \"ALL\" => {\n                            attributes = vec![\n                                Attribute::Flags,\n                                Attribute::InternalDate,\n                                Attribute::Rfc822Size,\n                                Attribute::Envelope,\n                            ];\n                            break;\n                        },\n                        \"FULL\" => {\n                            attributes = vec![\n                                Attribute::Flags,\n                                Attribute::InternalDate,\n                                Attribute::Rfc822Size,\n                                Attribute::Envelope,\n                                Attribute::Body,\n                            ];\n                            break;\n                        },\n                        \"FAST\" => {\n                            attributes = vec![\n                                Attribute::Flags,\n                                Attribute::InternalDate,\n                                Attribute::Rfc822Size,\n                            ];\n                            break;\n                        },\n                        \"ENVELOPE\" => {\n                            attributes.push_unique(Attribute::Envelope);\n                        },\n                        \"FLAGS\" => {\n                            attributes.push_unique(Attribute::Flags);\n                        },\n                        \"INTERNALDATE\" => {\n                            attributes.push_unique(Attribute::InternalDate);\n                        },\n                        \"BODYSTRUCTURE\" => {\n                            attributes.push_unique(Attribute::BodyStructure);\n                        },\n                        \"UID\" => {\n                            attributes.push_unique(Attribute::Uid);\n                        },\n                        \"RFC822\" => {\n                            attributes.push_unique(\n                                if tokens.peek().is_some_and(|token| token.is_dot()) {\n                                    tokens.next();\n                                    let rfc822 = tokens\n                                        .next()\n                                        .ok_or_else(|| {\n                                            bad(self.tag.to_compact_string(), \"Missing RFC822 parameter.\")\n                                        })?\n                                        .unwrap_bytes();\n                                    if rfc822.eq_ignore_ascii_case(b\"HEADER\") {\n                                        Attribute::Rfc822Header\n                                    } else if rfc822.eq_ignore_ascii_case(b\"SIZE\") {\n                                        Attribute::Rfc822Size\n                                    } else if rfc822.eq_ignore_ascii_case(b\"TEXT\") {\n                                        Attribute::Rfc822Text\n                                    } else {\n                                        return Err(bad(\n                                            CompactString::from_string_buffer(self.tag),\n                                            format_compact!(\n                                                \"Invalid RFC822 parameter {:?}.\",\n                                                String::from_utf8_lossy(&rfc822)\n                                            ),\n                                        ));\n                                    }\n                                } else {\n                                    Attribute::Rfc822\n                                },\n                            );\n                        },\n                        \"BODY\" => {\n                            let is_peek = match tokens.peek() {\n                                Some(Token::BracketOpen) => {\n                                    tokens.next();\n                                    false\n                                }\n                                Some(Token::Dot) => {\n                                    tokens.next();\n                                    if tokens\n                                        .next()\n                                        .is_none_or( |token| !token.eq_ignore_ascii_case(b\"PEEK\"))\n                                    {\n                                        return Err(bad(\n                                            self.tag.to_compact_string(),\n                                            \"Expected 'PEEK' after '.'.\",\n                                        ));\n                                    }\n                                    if tokens.next().is_none_or( |token| !token.is_bracket_open()) {\n                                        return Err(bad(\n                                            self.tag.to_compact_string(),\n                                            \"Expected '[' after 'BODY.PEEK'\",\n                                        ));\n                                    }\n                                    true\n                                }\n                                _ => {\n                                    attributes.push_unique(Attribute::Body);\n\n                                    if !in_parentheses {\n                                        break;\n                                    } else {\n                                        continue;\n                                    }\n                                }\n                            };\n\n                            // Parse section-spect\n                            let mut sections = Vec::new();\n                            while let Some(token) = tokens.next() {\n                                match token {\n                                    Token::BracketClose => break,\n                                    Token::Argument(value) => {\n                                        let section = if value.eq_ignore_ascii_case(b\"HEADER\") {\n                                            if let Some(Token::Dot) = tokens.peek() {\n                                                tokens.next();\n                                                if tokens.next().is_none_or( |token| {\n                                                    !token.eq_ignore_ascii_case(b\"FIELDS\")\n                                                }) {\n                                                    return Err(bad(\n                                                        CompactString::from_string_buffer(self.tag),\n                                                        \"Expected 'FIELDS' after 'HEADER.'.\",\n                                                    ));\n                                                }\n                                                let is_not = if let Some(Token::Dot) = tokens.peek() {\n                                                    tokens.next();\n                                                    if tokens.next().is_none_or( |token| {\n                                                        !token.eq_ignore_ascii_case(b\"NOT\")\n                                                    }) {\n                                                        return Err(bad(\n                                                            CompactString::from_string_buffer(self.tag),\n                                                            \"Expected 'NOT' after 'HEADER.FIELDS.'.\",\n                                                        ));\n                                                    }\n                                                    true\n                                                } else {\n                                                    false\n                                                };\n                                                if tokens\n                                                    .next()\n                                                    .is_none_or( |token| !token.is_parenthesis_open())\n                                                {\n                                                    return Err(bad(\n                                                        CompactString::from_string_buffer(self.tag),\n                                                        \"Expected '(' after 'HEADER.FIELDS'.\",\n                                                    ));\n                                                }\n                                                let mut fields = Vec::new();\n                                                while let Some(token) = tokens.next() {\n                                                    match token {\n                                                        Token::ParenthesisClose => break,\n                                                        Token::Argument(value) => {\n                                                            fields.push(String::from_utf8(value).map_err(\n                                                            |_| bad(self.tag.to_compact_string(),\"Invalid UTF-8 in header field name.\"),\n                                                        )?);\n                                                        }\n                                                        _ => {\n                                                            return Err(bad(\n                                                                CompactString::from_string_buffer(self.tag),\n                                                                \"Expected field name.\",\n                                                            ))\n                                                        }\n                                                    }\n                                                }\n                                                Section::HeaderFields {\n                                                    not: is_not,\n                                                    fields,\n                                                }\n                                            } else {\n                                                Section::Header\n                                            }\n                                        } else if value.eq_ignore_ascii_case(b\"TEXT\") {\n                                            Section::Text\n                                        } else if value.eq_ignore_ascii_case(b\"MIME\") {\n                                            Section::Mime\n                                        } else {\n                                            Section::Part {\n                                                num: parse_number::<u32>(&value)\n                                                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                            }\n                                        };\n                                        sections.push(section);\n                                    }\n                                    Token::Dot => (),\n                                    _ => {\n                                        return Err(bad(\n                                            CompactString::from_string_buffer(self.tag),\n                                            format_compact!(\n                                                \"Invalid token {:?} found in section-spect.\",\n                                                token\n                                            ),\n                                        ))\n                                    }\n                                }\n                            }\n\n                            attributes.push_unique(Attribute::BodySection {\n                                peek: is_peek,\n                                sections,\n                                partial: parse_partial(&mut tokens)\n                                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                            });\n                        },\n                        \"BINARY\" => {\n                            let (is_peek, is_size) = if let Some(Token::Dot) = tokens.peek() {\n                                tokens.next();\n                                let param = tokens\n                                    .next()\n                                    .ok_or({\n                                        bad(self.tag.to_compact_string(),\"Missing parameter after 'BINARY.'.\")\n                                    })?\n                                    .unwrap_bytes();\n                                if param.eq_ignore_ascii_case(b\"PEEK\") {\n                                    (true, false)\n                                } else if param.eq_ignore_ascii_case(b\"SIZE\") {\n                                    (false, true)\n                                } else {\n                                    return Err(bad(\n                                        CompactString::from_string_buffer(self.tag),\n                                        \"Expected 'PEEK' or 'SIZE' after 'BINARY.'.\",\n                                    ));\n                                }\n                            } else {\n                                (false, false)\n                            };\n\n                            // Parse section-part\n                            if tokens.next().is_none_or( |token| !token.is_bracket_open()) {\n                                return Err(bad(self.tag.to_compact_string(), \"Expected '[' after 'BINARY'.\"));\n                            }\n                            let mut sections = Vec::new();\n                            while let Some(token) = tokens.next() {\n                                match token {\n                                    Token::Argument(value) => {\n                                        sections.push(\n                                            parse_number::<u32>(&value)\n                                                .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                        );\n                                    }\n                                    Token::Dot => (),\n                                    Token::BracketClose => break,\n                                    _ => {\n                                        return Err(bad(\n                                            CompactString::from_string_buffer(self.tag),\n                                            format_compact!(\n                                                \"Expected part section integer, got {:?}.\",\n                                                token.to_string()\n                                            ),\n                                        ))\n                                    }\n                                }\n                            }\n                            attributes.push_unique(if !is_size {\n                                Attribute::Binary {\n                                    peek: is_peek,\n                                    sections,\n                                    partial: parse_partial(&mut tokens)\n                                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                }\n                            } else {\n                                Attribute::BinarySize { sections }\n                            });\n                        },\n                        \"PREVIEW\" => {\n                            attributes.push_unique(Attribute::Preview {\n                                lazy: if let Some(Token::ParenthesisOpen) = tokens.peek() {\n                                    tokens.next();\n                                    let mut is_lazy = false;\n                                    while let Some(token) = tokens.next() {\n                                        match token {\n                                            Token::ParenthesisClose => break,\n                                            Token::Argument(value) => {\n                                                if value.eq_ignore_ascii_case(b\"LAZY\") {\n                                                    is_lazy = true;\n                                                }\n                                            }\n                                            _ => (),\n                                        }\n                                    }\n                                    is_lazy\n                                } else {\n                                    false\n                                },\n                            });\n                        },\n                        \"MODSEQ\" => {\n                            attributes.push_unique(Attribute::ModSeq);\n                        },\n                        \"EMAILID\" => {\n                            attributes.push_unique(Attribute::EmailId);\n                        },\n                        \"THREADID\" => {\n                            attributes.push_unique(Attribute::ThreadId);\n                        },\n                        _ => {\n                            return Err(bad(\n                                CompactString::from_string_buffer(self.tag),\n                                format_compact!(\"Invalid attribute {:?}\", String::from_utf8_lossy(&value)),\n                            ));\n                        }\n                    );\n\n                    if !in_parentheses {\n                        break;\n                    }\n                }\n                Token::ParenthesisOpen => {\n                    if !in_parentheses {\n                        in_parentheses = true;\n                    } else {\n                        return Err(bad(\n                            self.tag.to_compact_string(),\n                            \"Unexpected parenthesis open.\",\n                        ));\n                    }\n                }\n                Token::ParenthesisClose => {\n                    if in_parentheses {\n                        break;\n                    } else {\n                        return Err(bad(\n                            self.tag.to_compact_string(),\n                            \"Unexpected parenthesis close.\",\n                        ));\n                    }\n                }\n                _ => {\n                    return Err(bad(\n                        CompactString::from_string_buffer(self.tag),\n                        format_compact!(\"Invalid fetch argument {:?}.\", token.to_string()),\n                    ));\n                }\n            }\n        }\n\n        // CONDSTORE parameters\n        let mut changed_since = None;\n        let mut include_vanished = false;\n        if let Some(Token::ParenthesisOpen) = tokens.peek() {\n            tokens.next();\n            while let Some(token) = tokens.next() {\n                match token {\n                    Token::Argument(param) if param.eq_ignore_ascii_case(b\"CHANGEDSINCE\") => {\n                        changed_since = parse_number::<u64>(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| {\n                                    bad(\n                                        self.tag.to_compact_string(),\n                                        \"Missing CHANGEDSINCE parameter.\",\n                                    )\n                                })?\n                                .unwrap_bytes(),\n                        )\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?\n                        .into();\n                    }\n                    Token::Argument(param) if param.eq_ignore_ascii_case(b\"VANISHED\") => {\n                        include_vanished = true;\n                    }\n                    Token::ParenthesisClose => {\n                        break;\n                    }\n                    _ => {\n                        return Err(bad(\n                            self.tag.to_compact_string(),\n                            format_compact!(\"Unsupported parameter '{}'.\", token),\n                        ));\n                    }\n                }\n            }\n        }\n\n        if !attributes.is_empty() {\n            Ok(fetch::Arguments {\n                tag: self.tag,\n                sequence_set,\n                attributes,\n                changed_since,\n                include_vanished,\n            })\n        } else {\n            Err(bad(\n                CompactString::from_string_buffer(self.tag),\n                \"No data items to fetch specified.\",\n            ))\n        }\n    }\n}\n\npub fn parse_partial(tokens: &mut Peekable<IntoIter<Token>>) -> super::Result<Option<(u32, u32)>> {\n    if tokens.peek().is_none_or(|token| !token.is_lt()) {\n        return Ok(None);\n    }\n    tokens.next();\n\n    let start = parse_number::<u32>(\n        &tokens\n            .next()\n            .ok_or_else(|| Cow::from(\"Missing partial start.\"))?\n            .unwrap_bytes(),\n    )?;\n\n    if tokens.next().is_none_or(|token| !token.is_dot()) {\n        return Err(\"Expected '.' after partial start.\".into());\n    }\n\n    let end = parse_number::<u32>(\n        &tokens\n            .next()\n            .ok_or_else(|| Cow::from(\"Missing partial end.\"))?\n            .unwrap_bytes(),\n    )?;\n\n    if end == 0 {\n        return Err(\"Invalid partial range.\".into());\n    }\n\n    if tokens.next().is_none_or(|token| !token.is_gt()) {\n        return Err(\"Expected '>' after range.\".into());\n    }\n\n    Ok(Some((start, end)))\n}\n\n/*\n\n   fetch           = \"FETCH\" SP sequence-set SP (\n                     \"ALL\" / \"FULL\" / \"FAST\" /\n                     fetch-att / \"(\" fetch-att *(SP fetch-att) \")\")\n\n   fetch-att       = \"ENVELOPE\" / \"FLAGS\" / \"INTERNALDATE\" /\n                     \"RFC822\" [\".HEADER\" / \".SIZE\" / \".TEXT\"] /\n                     \"BODY\" [\"STRUCTURE\"] / \"UID\" /\n                     \"BODY\" section [partial] /\n                     \"BODY.PEEK\" section [partial] /\n                     \"BINARY\" [\".PEEK\"] section-binary [partial] /\n                     \"BINARY.SIZE\" section-binary\n\n   partial         = \"<\" number64 \".\" nz-number64 \">\"\n                       ; Partial FETCH request. 0-based offset of\n                       ; the first octet, followed by the number of\n                       ; octets in the fragment.\n\n   section         = \"[\" [section-spec] \"]\"\n\n   section-binary  = \"[\" [section-part] \"]\"\n\n   section-msgtext = \"HEADER\" /\n                     \"HEADER.FIELDS\" [\".NOT\"] SP header-list /\n                     \"TEXT\"\n                       ; top-level or MESSAGE/RFC822 or\n                       ; MESSAGE/GLOBAL part\n\n   section-part    = nz-number *(\".\" nz-number)\n                       ; body part reference.\n                       ; Allows for accessing nested body parts.\n\n   section-spec    = section-msgtext / (section-part [\".\" section-text])\n\n   section-text    = section-msgtext / \"MIME\"\n                       ; text other than actual body part (headers,\n                       ; etc.)\n\n\n*/\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::{\n            Sequence,\n            fetch::{self, Attribute, Section},\n        },\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_fetch() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A654 FETCH 2:4 (FLAGS BODY[HEADER.FIELDS (DATE FROM)])\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A654\".into(),\n                    sequence_set: Sequence::range(2.into(), 4.into()),\n                    attributes: vec![\n                        Attribute::Flags,\n                        Attribute::BodySection {\n                            peek: false,\n                            sections: vec![Section::HeaderFields {\n                                not: false,\n                                fields: vec![\"DATE\".into(), \"FROM\".into()],\n                            }],\n                            partial: None,\n                        },\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 BODY[]\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![Attribute::BodySection {\n                        peek: false,\n                        sections: vec![],\n                        partial: None,\n                    }],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 (BODY[HEADER])\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![Attribute::BodySection {\n                        peek: false,\n                        sections: vec![Section::Header],\n                        partial: None,\n                    }],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 (BODY.PEEK[HEADER.FIELDS (X-MAILER)] PREVIEW(LAZY))\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::BodySection {\n                            peek: true,\n                            sections: vec![Section::HeaderFields {\n                                not: false,\n                                fields: vec![\"X-MAILER\".into()],\n                            }],\n                            partial: None,\n                        },\n                        Attribute::Preview { lazy: true },\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 (BODY[HEADER.FIELDS.NOT (FROM TO SUBJECT)])\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![Attribute::BodySection {\n                        peek: false,\n                        sections: vec![Section::HeaderFields {\n                            not: true,\n                            fields: vec![\"FROM\".into(), \"TO\".into(), \"SUBJECT\".into()],\n                        }],\n                        partial: None,\n                    }],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 (BODY[MIME] BODY[TEXT] PREVIEW)\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::BodySection {\n                            peek: false,\n                            sections: vec![Section::Mime],\n                            partial: None,\n                        },\n                        Attribute::BodySection {\n                            peek: false,\n                            sections: vec![Section::Text],\n                            partial: None,\n                        },\n                        Attribute::Preview { lazy: false },\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 (BODYSTRUCTURE ENVELOPE FLAGS INTERNALDATE UID)\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::BodyStructure,\n                        Attribute::Envelope,\n                        Attribute::Flags,\n                        Attribute::InternalDate,\n                        Attribute::Uid,\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 (RFC822 RFC822.HEADER RFC822.SIZE RFC822.TEXT)\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::Rfc822,\n                        Attribute::Rfc822Header,\n                        Attribute::Rfc822Size,\n                        Attribute::Rfc822Text,\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                concat!(\n                    \"A001 FETCH 1 (\",\n                    \"BODY[4.2.HEADER]<0.20> \",\n                    \"BODY.PEEK[3.2.2.2] \",\n                    \"BODY[4.2.TEXT]<4.100> \",\n                    \"BINARY[1.2.3] \",\n                    \"BINARY.PEEK[4] \",\n                    \"BINARY[6.5.4]<100.200> \",\n                    \"BINARY.PEEK[7]<9.88> \",\n                    \"BINARY.SIZE[9.1]\",\n                    \")\\r\\n\"\n                ),\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::BodySection {\n                            peek: false,\n                            sections: vec![\n                                Section::Part { num: 4 },\n                                Section::Part { num: 2 },\n                                Section::Header,\n                            ],\n                            partial: Some((0, 20)),\n                        },\n                        Attribute::BodySection {\n                            peek: true,\n                            sections: vec![\n                                Section::Part { num: 3 },\n                                Section::Part { num: 2 },\n                                Section::Part { num: 2 },\n                                Section::Part { num: 2 },\n                            ],\n                            partial: None,\n                        },\n                        Attribute::BodySection {\n                            peek: false,\n                            sections: vec![\n                                Section::Part { num: 4 },\n                                Section::Part { num: 2 },\n                                Section::Text,\n                            ],\n                            partial: Some((4, 100)),\n                        },\n                        Attribute::Binary {\n                            peek: false,\n                            sections: vec![1, 2, 3],\n                            partial: None,\n                        },\n                        Attribute::Binary {\n                            peek: true,\n                            sections: vec![4],\n                            partial: None,\n                        },\n                        Attribute::Binary {\n                            peek: false,\n                            sections: vec![6, 5, 4],\n                            partial: Some((100, 200)),\n                        },\n                        Attribute::Binary {\n                            peek: true,\n                            sections: vec![7],\n                            partial: Some((9, 88)),\n                        },\n                        Attribute::BinarySize {\n                            sections: vec![9, 1],\n                        },\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 ALL\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::Flags,\n                        Attribute::InternalDate,\n                        Attribute::Rfc822Size,\n                        Attribute::Envelope,\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 FULL\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::Flags,\n                        Attribute::InternalDate,\n                        Attribute::Rfc822Size,\n                        Attribute::Envelope,\n                        Attribute::Body,\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"A001 FETCH 1 FAST\\r\\n\",\n                fetch::Arguments {\n                    tag: \"A001\".into(),\n                    sequence_set: Sequence::number(1),\n                    attributes: vec![\n                        Attribute::Flags,\n                        Attribute::InternalDate,\n                        Attribute::Rfc822Size,\n                    ],\n                    changed_since: None,\n                    include_vanished: false,\n                },\n            ),\n            (\n                \"s100 UID FETCH 1:* (FLAGS MODSEQ) (CHANGEDSINCE 12345 VANISHED)\\r\\n\",\n                fetch::Arguments {\n                    tag: \"s100\".into(),\n                    sequence_set: Sequence::range(1.into(), None),\n                    attributes: vec![Attribute::Flags, Attribute::ModSeq],\n                    changed_since: 12345.into(),\n                    include_vanished: true,\n                },\n            ),\n            (\n                \"9 UID FETCH 1:* UID (VANISHED CHANGEDSINCE 1)\\r\\n\",\n                fetch::Arguments {\n                    tag: \"9\".into(),\n                    sequence_set: Sequence::range(1.into(), None),\n                    attributes: vec![Attribute::Uid],\n                    changed_since: 1.into(),\n                    include_vanished: true,\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_fetch()\n                    .expect(command),\n                arguments,\n                \"{}\",\n                command\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/list.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::{CompactString, ToCompactString};\n\nuse crate::{\n    Command,\n    protocol::{\n        list::{self, ReturnOption, SelectionOption},\n        status::Status,\n    },\n    receiver::{Request, Token, bad},\n    utf7::utf7_maybe_decode,\n};\n\nimpl Request<Command> {\n    #[allow(clippy::while_let_on_iterator)]\n    pub fn parse_list(self, is_utf8: bool) -> trc::Result<list::Arguments> {\n        match self.tokens.len() {\n            0 | 1 => Err(self.into_error(\"Missing arguments.\")),\n            2 => {\n                let mut tokens = self.tokens.into_iter();\n                Ok(list::Arguments::Basic {\n                    reference_name: tokens\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    mailbox_name: utf7_maybe_decode(\n                        tokens\n                            .next()\n                            .unwrap()\n                            .unwrap_string()\n                            .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                        is_utf8,\n                    ),\n                    tag: self.tag,\n                })\n            }\n            _ => {\n                let mut tokens = self.tokens.into_iter();\n                let mut selection_options = Vec::new();\n                let mut return_options = Vec::new();\n                let mut mailbox_name = Vec::new();\n\n                let reference_name = match tokens.next().unwrap() {\n                    Token::ParenthesisOpen => {\n                        while let Some(token) = tokens.next() {\n                            match token {\n                                Token::ParenthesisClose => break,\n                                Token::Argument(value) => {\n                                    selection_options.push(\n                                        SelectionOption::parse(&value)\n                                            .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                    );\n                                }\n                                _ => {\n                                    return Err(bad(\n                                        self.tag.to_compact_string(),\n                                        \"Invalid selection option argument.\",\n                                    ));\n                                }\n                            }\n                        }\n                        tokens\n                            .next()\n                            .ok_or_else(|| {\n                                bad(self.tag.to_compact_string(), \"Missing reference name.\")\n                            })?\n                            .unwrap_string()\n                            .map_err(|v| bad(self.tag.to_compact_string(), v))?\n                    }\n                    token => token\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                };\n\n                match tokens\n                    .next()\n                    .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing mailbox name.\"))?\n                {\n                    Token::ParenthesisOpen => {\n                        while let Some(token) = tokens.next() {\n                            match token {\n                                Token::ParenthesisClose => break,\n                                token => {\n                                    mailbox_name.push(\n                                        token\n                                            .unwrap_string()\n                                            .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                    );\n                                }\n                            }\n                        }\n                    }\n                    token => {\n                        mailbox_name.push(utf7_maybe_decode(\n                            token\n                                .unwrap_string()\n                                .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                            is_utf8,\n                        ));\n                    }\n                }\n\n                if tokens\n                    .next()\n                    .is_some_and(|token| token.eq_ignore_ascii_case(b\"return\"))\n                {\n                    if tokens\n                        .next()\n                        .is_none_or(|token| !token.is_parenthesis_open())\n                    {\n                        return Err(bad(\n                            self.tag.to_compact_string(),\n                            \"Invalid return option, expected parenthesis.\",\n                        ));\n                    }\n\n                    while let Some(token) = tokens.next() {\n                        match token {\n                            Token::ParenthesisClose => break,\n                            Token::Argument(value) => {\n                                let mut return_option = ReturnOption::parse(&value)\n                                    .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n                                if let ReturnOption::Status(status) = &mut return_option {\n                                    if tokens\n                                        .next()\n                                        .is_none_or(|token| !token.is_parenthesis_open())\n                                    {\n                                        return Err(bad(\n                                            CompactString::from_string_buffer(self.tag),\n                                            \"Invalid return option, expected parenthesis after STATUS.\",\n                                        ));\n                                    }\n                                    while let Some(token) = tokens.next() {\n                                        match token {\n                                            Token::ParenthesisClose => break,\n                                            Token::Argument(value) => {\n                                                status.push(Status::parse(&value).map_err(\n                                                    |v| bad(self.tag.to_compact_string(), v),\n                                                )?);\n                                            }\n                                            _ => {\n                                                return Err(bad(\n                                                    CompactString::from_string_buffer(self.tag),\n                                                    \"Invalid status return option argument.\",\n                                                ));\n                                            }\n                                        }\n                                    }\n                                }\n                                return_options.push(return_option);\n                            }\n                            _ => {\n                                return Err(bad(\n                                    self.tag.to_compact_string(),\n                                    \"Invalid return option argument.\",\n                                ));\n                            }\n                        }\n                    }\n                }\n\n                Ok(list::Arguments::Extended {\n                    tag: self.tag,\n                    reference_name,\n                    mailbox_name,\n                    selection_options,\n                    return_options,\n                })\n            }\n        }\n    }\n}\n\nimpl SelectionOption {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(value,\n            \"SUBSCRIBED\" => Self::Subscribed,\n            \"REMOTE\" => Self::Remote,\n            \"RECURSIVEMATCH\" => Self::RecursiveMatch,\n            \"SPECIAL-USE\" => Self::SpecialUse,\n        )\n        .ok_or_else(|| {\n            format!(\n                \"Unsupported selection option '{}'.\",\n                String::from_utf8_lossy(value)\n            )\n            .into()\n        })\n    }\n}\n\nimpl ReturnOption {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(value,\n            \"SUBSCRIBED\" => Self::Subscribed,\n            \"CHILDREN\" => Self::Children,\n            \"STATUS\" => Self::Status(Vec::with_capacity(2)),\n            \"SPECIAL-USE\" => Self::SpecialUse,\n        )\n        .ok_or_else(|| format!(\"Invalid return option {:?}\", String::from_utf8_lossy(value)).into())\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::{\n            list::{self, ReturnOption, SelectionOption},\n            status::Status,\n        },\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_list() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A682 LIST \\\"\\\" *\\r\\n\",\n                list::Arguments::Basic {\n                    tag: \"A682\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: \"*\".into(),\n                },\n            ),\n            (\n                \"A02 LIST (SUBSCRIBED) \\\"\\\" \\\"*\\\"\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A02\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"*\".into()],\n                    selection_options: vec![SelectionOption::Subscribed],\n                    return_options: vec![],\n                },\n            ),\n            (\n                \"A03 LIST () \\\"\\\" \\\"%\\\" RETURN (CHILDREN)\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A03\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"%\".into()],\n                    selection_options: vec![],\n                    return_options: vec![ReturnOption::Children],\n                },\n            ),\n            (\n                \"A04 LIST (REMOTE) \\\"\\\" \\\"%\\\" RETURN (CHILDREN)\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A04\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"%\".into()],\n                    selection_options: vec![SelectionOption::Remote],\n                    return_options: vec![ReturnOption::Children],\n                },\n            ),\n            (\n                \"A05 LIST (REMOTE SUBSCRIBED) \\\"\\\" \\\"*\\\"\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A05\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"*\".into()],\n                    selection_options: vec![SelectionOption::Remote, SelectionOption::Subscribed],\n                    return_options: vec![],\n                },\n            ),\n            (\n                \"A06 LIST (REMOTE) \\\"\\\" \\\"*\\\" RETURN (SUBSCRIBED)\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A06\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"*\".into()],\n                    selection_options: vec![SelectionOption::Remote],\n                    return_options: vec![ReturnOption::Subscribed],\n                },\n            ),\n            (\n                \"C04 LIST (SUBSCRIBED RECURSIVEMATCH) \\\"\\\" \\\"%\\\"\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"C04\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"%\".into()],\n                    selection_options: vec![\n                        SelectionOption::Subscribed,\n                        SelectionOption::RecursiveMatch,\n                    ],\n                    return_options: vec![],\n                },\n            ),\n            (\n                \"C04 LIST (SUBSCRIBED RECURSIVEMATCH) \\\"\\\" \\\"%\\\" RETURN (CHILDREN)\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"C04\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"%\".into()],\n                    selection_options: vec![\n                        SelectionOption::Subscribed,\n                        SelectionOption::RecursiveMatch,\n                    ],\n                    return_options: vec![ReturnOption::Children],\n                },\n            ),\n            (\n                \"a1 LIST \\\"\\\" (\\\"foo\\\")\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"a1\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"foo\".into()],\n                    selection_options: vec![],\n                    return_options: vec![],\n                },\n            ),\n            (\n                \"a3.1 LIST \\\"\\\" (% music/rock)\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"a3.1\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"%\".into(), \"music/rock\".into()],\n                    selection_options: vec![],\n                    return_options: vec![],\n                },\n            ),\n            (\n                \"BBB LIST \\\"\\\" (\\\"INBOX\\\" \\\"Drafts\\\" \\\"Sent/%\\\")\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"BBB\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"INBOX\".into(), \"Drafts\".into(), \"Sent/%\".into()],\n                    selection_options: vec![],\n                    return_options: vec![],\n                },\n            ),\n            (\n                \"A01 LIST \\\"\\\" % RETURN (STATUS (MESSAGES UNSEEN))\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A01\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"%\".into()],\n                    selection_options: vec![],\n                    return_options: vec![ReturnOption::Status(vec![\n                        Status::Messages,\n                        Status::Unseen,\n                    ])],\n                },\n            ),\n            (\n                concat!(\n                    \"A02 LIST (SUBSCRIBED RECURSIVEMATCH) \\\"\\\" \",\n                    \"% RETURN (CHILDREN STATUS (MESSAGES))\\r\\n\"\n                ),\n                list::Arguments::Extended {\n                    tag: \"A02\".into(),\n                    reference_name: \"\".into(),\n                    mailbox_name: vec![\"%\".into()],\n                    selection_options: vec![\n                        SelectionOption::Subscribed,\n                        SelectionOption::RecursiveMatch,\n                    ],\n                    return_options: vec![\n                        ReturnOption::Children,\n                        ReturnOption::Status(vec![Status::Messages]),\n                    ],\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_list(true)\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/login.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::login,\n    receiver::{Request, bad},\n};\n\nimpl Request<Command> {\n    pub fn parse_login(self) -> trc::Result<login::Arguments> {\n        match self.tokens.len() {\n            2 => {\n                let mut tokens = self.tokens.into_iter();\n                Ok(login::Arguments {\n                    username: tokens\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    password: tokens\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    tag: self.tag,\n                })\n            }\n            0 => Err(self.into_error(\"Missing arguments.\")),\n            _ => Err(self.into_error(\"Too many arguments.\")),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{protocol::login, receiver::Receiver};\n\n    #[test]\n    fn parse_login() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"a001 LOGIN SMITH SESAME\\r\\n\",\n                login::Arguments {\n                    tag: \"a001\".into(),\n                    username: \"SMITH\".into(),\n                    password: \"SESAME\".into(),\n                },\n            ),\n            (\n                \"A001 LOGIN {11+}\\r\\nFRED FOOBAR {7+}\\r\\nfat man\\r\\n\",\n                login::Arguments {\n                    tag: \"A001\".into(),\n                    username: \"FRED FOOBAR\".into(),\n                    password: \"fat man\".into(),\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_login()\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/lsub.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::list::{self, SelectionOption},\n    receiver::{Request, bad},\n    utf7::utf7_maybe_decode,\n};\n\nimpl Request<Command> {\n    pub fn parse_lsub(self, is_utf8: bool) -> trc::Result<list::Arguments> {\n        if self.tokens.len() > 1 {\n            let mut tokens = self.tokens.into_iter();\n\n            Ok(list::Arguments::Extended {\n                reference_name: tokens\n                    .next()\n                    .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing reference name.\"))?\n                    .unwrap_string()\n                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                mailbox_name: vec![utf7_maybe_decode(\n                    tokens\n                        .next()\n                        .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing mailbox name.\"))?\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    is_utf8,\n                )],\n                selection_options: vec![SelectionOption::Subscribed],\n                return_options: vec![],\n                tag: self.tag,\n            })\n        } else {\n            Err(self.into_error(\"Missing arguments.\"))\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::list::{self, SelectionOption},\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_lsub() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A002 LSUB \\\"#news.\\\" \\\"comp.mail.*\\\"\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A002\".into(),\n                    reference_name: \"#news.\".into(),\n                    mailbox_name: vec![\"comp.mail.*\".into()],\n                    selection_options: vec![SelectionOption::Subscribed],\n                    return_options: vec![],\n                },\n            ),\n            (\n                \"A002 LSUB \\\"#news.\\\" \\\"comp.%\\\"\\r\\n\",\n                list::Arguments::Extended {\n                    tag: \"A002\".into(),\n                    reference_name: \"#news.\".into(),\n                    mailbox_name: vec![\"comp.%\".into()],\n                    selection_options: vec![SelectionOption::Subscribed],\n                    return_options: vec![],\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_lsub(false)\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod append;\npub mod authenticate;\npub mod copy_move;\npub mod create;\npub mod delete;\npub mod enable;\npub mod fetch;\npub mod list;\npub mod login;\npub mod lsub;\npub mod quota;\npub mod rename;\npub mod search;\npub mod select;\npub mod sort;\npub mod status;\npub mod store;\npub mod subscribe;\npub mod thread;\n\nuse std::{borrow::Cow, str::FromStr};\n\nuse chrono::{DateTime, NaiveDate};\n\nuse crate::{\n    Command,\n    protocol::{Flag, Sequence},\n    receiver::CommandParser,\n};\n\npub type Result<T> = std::result::Result<T, Cow<'static, str>>;\n\nimpl CommandParser for Command {\n    fn parse(value: &[u8], uid: bool) -> Option<Self> {\n        hashify::tiny_map!(value,\n            \"CAPABILITY\" => Command::Capability,\n            \"NOOP\" => Command::Noop,\n            \"LOGOUT\" => Command::Logout,\n            \"STARTTLS\" => Command::StartTls,\n            \"AUTHENTICATE\" => Command::Authenticate,\n            \"LOGIN\" => Command::Login,\n            \"ENABLE\" => Command::Enable,\n            \"SELECT\" => Command::Select,\n            \"EXAMINE\" => Command::Examine,\n            \"CREATE\" => Command::Create,\n            \"DELETE\" => Command::Delete,\n            \"RENAME\" => Command::Rename,\n            \"SUBSCRIBE\" => Command::Subscribe,\n            \"UNSUBSCRIBE\" => Command::Unsubscribe,\n            \"LIST\" => Command::List,\n            \"NAMESPACE\" => Command::Namespace,\n            \"STATUS\" => Command::Status,\n            \"APPEND\" => Command::Append,\n            \"IDLE\" => Command::Idle,\n            \"CLOSE\" => Command::Close,\n            \"UNSELECT\" => Command::Unselect,\n            \"EXPUNGE\" => Command::Expunge(uid),\n            \"SEARCH\" => Command::Search(uid),\n            \"FETCH\" => Command::Fetch(uid),\n            \"STORE\" => Command::Store(uid),\n            \"COPY\" => Command::Copy(uid),\n            \"MOVE\" => Command::Move(uid),\n            \"SORT\" => Command::Sort(uid),\n            \"THREAD\" => Command::Thread(uid),\n            \"LSUB\" => Command::Lsub,\n            \"CHECK\" => Command::Check,\n            \"SETACL\" => Command::SetAcl,\n            \"DELETEACL\" => Command::DeleteAcl,\n            \"GETACL\" => Command::GetAcl,\n            \"LISTRIGHTS\" => Command::ListRights,\n            \"MYRIGHTS\" => Command::MyRights,\n            \"UNAUTHENTICATE\" => Command::Unauthenticate,\n            \"ID\" => Command::Id,\n            \"GETQUOTA\" => Command::GetQuota,\n            \"GETQUOTAROOT\" => Command::GetQuotaRoot,\n        )\n    }\n\n    #[inline(always)]\n    fn tokenize_brackets(&self) -> bool {\n        matches!(self, Command::Fetch(_))\n    }\n}\n\nimpl Flag {\n    pub fn parse_imap(value: Vec<u8>) -> Result<Self> {\n        if !value.is_empty() {\n            let flag = hashify::tiny_map_ignore_case!(value.as_slice(),\n                \"\\\\Seen\" => Flag::Seen,\n                \"\\\\Answered\" => Flag::Answered,\n                \"\\\\Flagged\" => Flag::Flagged,\n                \"\\\\Deleted\" => Flag::Deleted,\n                \"\\\\Draft\" => Flag::Draft,\n                \"\\\\Recent\" => Flag::Recent,\n                \"\\\\Important\" => Flag::Important,\n                \"$Forwarded\" => Flag::Forwarded,\n                \"$MDNSent\" => Flag::MDNSent,\n                \"$Junk\" => Flag::Junk,\n                \"$NotJunk\" => Flag::NotJunk,\n                \"$Phishing\" => Flag::Phishing,\n                \"$Important\" => Flag::Important,\n                \"$autosent\" => Flag::Autosent,\n                \"$canunsubscribe\" => Flag::CanUnsubscribe,\n                \"$followed\" => Flag::Followed,\n                \"$hasattachment\" => Flag::HasAttachment,\n                \"$hasmemo\" => Flag::HasMemo,\n                \"$hasnoattachment\" => Flag::HasNoAttachment,\n                \"$imported\" => Flag::Imported,\n                \"$istrusted\" => Flag::IsTrusted,\n                \"$MailFlagBit0\" => Flag::MailFlagBit0,\n                \"$MailFlagBit1\" => Flag::MailFlagBit1,\n                \"$MailFlagBit2\" => Flag::MailFlagBit2,\n                \"$maskedemail\" => Flag::MaskedEmail,\n                \"$memo\" => Flag::Memo,\n                \"$muted\" => Flag::Muted,\n                \"$new\" => Flag::New,\n                \"$notify\" => Flag::Notify,\n                \"$unsubscribed\" => Flag::Unsubscribed,\n            );\n\n            if let Some(flag) = flag {\n                Ok(flag)\n            } else {\n                String::from_utf8(value)\n                    .map_err(|_| Cow::from(\"Invalid UTF-8.\"))\n                    .map(|v| Flag::Keyword(v.into_boxed_str()))\n            }\n        } else {\n            Err(Cow::from(\"Null flags are not allowed.\"))\n        }\n    }\n\n    pub fn parse_jmap(value: String) -> Self {\n        if value.starts_with('$') {\n            hashify::tiny_map_ignore_case!(value.as_bytes(),\n                \"$seen\" => Flag::Seen,\n                \"$draft\" => Flag::Draft,\n                \"$flagged\" => Flag::Flagged,\n                \"$answered\" => Flag::Answered,\n                \"$recent\" => Flag::Recent,\n                \"$important\" => Flag::Important,\n                \"$phishing\" => Flag::Phishing,\n                \"$junk\" => Flag::Junk,\n                \"$notjunk\" => Flag::NotJunk,\n                \"$deleted\" => Flag::Deleted,\n                \"$forwarded\" => Flag::Forwarded,\n                \"$mdnsent\" => Flag::MDNSent,\n                \"$autosent\" => Flag::Autosent,\n                \"$canunsubscribe\" => Flag::CanUnsubscribe,\n                \"$followed\" => Flag::Followed,\n                \"$hasattachment\" => Flag::HasAttachment,\n                \"$hasmemo\" => Flag::HasMemo,\n                \"$hasnoattachment\" => Flag::HasNoAttachment,\n                \"$imported\" => Flag::Imported,\n                \"$istrusted\" => Flag::IsTrusted,\n                \"$MailFlagBit0\" => Flag::MailFlagBit0,\n                \"$MailFlagBit1\" => Flag::MailFlagBit1,\n                \"$MailFlagBit2\" => Flag::MailFlagBit2,\n                \"$maskedemail\" => Flag::MaskedEmail,\n                \"$memo\" => Flag::Memo,\n                \"$muted\" => Flag::Muted,\n                \"$new\" => Flag::New,\n                \"$notify\" => Flag::Notify,\n                \"$unsubscribed\" => Flag::Unsubscribed,\n            )\n            .unwrap_or_else(|| Flag::Keyword(value.into_boxed_str()))\n        } else {\n            let mut keyword = String::with_capacity(value.len());\n            for c in value.chars() {\n                if c.is_ascii_alphanumeric() {\n                    keyword.push(c);\n                } else {\n                    keyword.push('_');\n                }\n            }\n            Flag::Keyword(keyword.into_boxed_str())\n        }\n    }\n}\n\npub fn parse_datetime(value: &[u8]) -> Result<i64> {\n    std::str::from_utf8(value)\n        .map_err(|_| Cow::from(\"Expected date/time, found an invalid UTF-8 string.\"))\n        .and_then(|datetime| {\n            DateTime::parse_from_str(datetime.trim(), \"%d-%b-%Y %H:%M:%S %z\")\n                .map_err(|_| Cow::from(format!(\"Failed to parse date/time '{}'.\", datetime)))\n                .map(|dt| dt.timestamp())\n        })\n}\n\npub fn parse_date(value: &[u8]) -> Result<i64> {\n    std::str::from_utf8(value)\n        .map_err(|_| Cow::from(\"Expected date, found an invalid UTF-8 string.\"))\n        .and_then(|date| {\n            NaiveDate::parse_from_str(date.trim(), \"%d-%b-%Y\")\n                .map_err(|_| Cow::from(format!(\"Failed to parse date '{}'.\", date)))\n                .map(|dt| {\n                    dt.and_hms_opt(0, 0, 0)\n                        .unwrap_or_default()\n                        .and_utc()\n                        .timestamp()\n                })\n        })\n}\n\npub fn parse_number<T: FromStr>(value: &[u8]) -> Result<T> {\n    std::str::from_utf8(value)\n        .map_err(|_| Cow::from(\"Expected a number, found an invalid UTF-8 string.\"))\n        .and_then(|string| {\n            string\n                .parse::<T>()\n                .map_err(|_| Cow::from(format!(\"Expected a number, found {:?}.\", string)))\n        })\n}\n\npub fn parse_sequence_set(value: &[u8]) -> Result<Sequence> {\n    let mut sequence_set = Vec::new();\n\n    let mut range_start = None;\n    let mut token_start = None;\n\n    let mut is_wildcard = false;\n    let mut is_range = false;\n    let mut is_saved_search = false;\n\n    for (mut pos, ch) in value.iter().enumerate() {\n        let mut add_token = false;\n        match ch {\n            b',' => {\n                add_token = true;\n            }\n            b':' => {\n                if !is_range {\n                    if let Some(from_pos) = token_start {\n                        range_start =\n                            parse_number::<u32>(value.get(from_pos..pos).ok_or_else(|| {\n                                Cow::from(format!(\n                                    \"Invalid sequence set {:?}, parse error.\",\n                                    String::from_utf8_lossy(value)\n                                ))\n                            })?)?\n                            .into();\n                        token_start = None;\n                    } else if is_wildcard {\n                        is_wildcard = false;\n                    } else {\n                        return Err(Cow::from(format!(\n                            \"Invalid sequence set {:?}, number expected before ':'.\",\n                            String::from_utf8_lossy(value)\n                        )));\n                    }\n                    is_range = true;\n                } else {\n                    return Err(Cow::from(format!(\n                        \"Invalid sequence set {:?}, ':' appears multiple times.\",\n                        String::from_utf8_lossy(value)\n                    )));\n                }\n            }\n            b'*' => {\n                if !is_wildcard {\n                    if value.len() == 1 {\n                        return Ok(Sequence::Range {\n                            start: None,\n                            end: None,\n                        });\n                    } else if token_start.is_none() {\n                        is_wildcard = true;\n                    } else {\n                        return Err(Cow::from(format!(\n                            \"Invalid sequence set {:?}, invalid use of '*'.\",\n                            String::from_utf8_lossy(value)\n                        )));\n                    }\n                } else {\n                    return Err(Cow::from(format!(\n                        \"Invalid sequence set {:?}, '*' appears multiple times.\",\n                        String::from_utf8_lossy(value)\n                    )));\n                }\n            }\n            b'$' => {\n                if value.get(pos + 1).is_none_or(|&ch| ch == b',') {\n                    is_saved_search = true;\n                } else {\n                    return Err(Cow::from(format!(\n                        \"Invalid sequence set {:?}, unexpected token after '$'.\",\n                        String::from_utf8_lossy(value)\n                    )));\n                }\n            }\n            _ => {\n                if ch.is_ascii_digit() {\n                    if is_wildcard {\n                        return Err(Cow::from(format!(\n                            \"Invalid sequence set {:?}, invalid use of '*'.\",\n                            String::from_utf8_lossy(value)\n                        )));\n                    }\n                    if token_start.is_none() {\n                        token_start = pos.into();\n                    }\n                } else {\n                    return Err(Cow::from(format!(\n                        \"Invalid sequence set {:?}, found invalid character '{}' at position {}.\",\n                        String::from_utf8_lossy(value),\n                        ch,\n                        pos\n                    )));\n                }\n            }\n        }\n\n        if add_token || pos == value.len() - 1 {\n            if is_range {\n                sequence_set.push(Sequence::Range {\n                    start: range_start,\n                    end: if !is_wildcard {\n                        if !add_token {\n                            pos += 1;\n                        }\n                        parse_number::<u32>(\n                            value\n                                .get(\n                                    token_start.ok_or_else(|| {\n                                        Cow::from(format!(\n                                            \"Invalid sequence set {:?}, expected number.\",\n                                            String::from_utf8_lossy(value)\n                                        ))\n                                    })?..pos,\n                                )\n                                .ok_or_else(|| {\n                                    Cow::from(format!(\n                                        \"Invalid sequence set {:?}, parse error.\",\n                                        String::from_utf8_lossy(value)\n                                    ))\n                                })?,\n                        )?\n                        .into()\n                    } else {\n                        is_wildcard = false;\n                        None\n                    },\n                });\n                is_range = false;\n                range_start = None;\n            } else {\n                if !add_token {\n                    pos += 1;\n                }\n                if is_wildcard {\n                    sequence_set.push(Sequence::Range {\n                        start: None,\n                        end: None,\n                    });\n                    is_wildcard = false;\n                } else if is_saved_search {\n                    sequence_set.push(Sequence::SavedSearch);\n                    is_saved_search = false;\n                } else {\n                    sequence_set.push(Sequence::Number {\n                        value: parse_number(\n                            value\n                                .get(\n                                    token_start.ok_or_else(|| {\n                                        Cow::from(format!(\n                                            \"Invalid sequence set {:?}, expected number.\",\n                                            String::from_utf8_lossy(value)\n                                        ))\n                                    })?..pos,\n                                )\n                                .ok_or_else(|| {\n                                    Cow::from(format!(\n                                        \"Invalid sequence set {:?}, parse error.\",\n                                        String::from_utf8_lossy(value)\n                                    ))\n                                })?,\n                        )?,\n                    });\n                }\n            }\n            token_start = None;\n        }\n    }\n\n    match sequence_set.len() {\n        1 => Ok(sequence_set.pop().unwrap()),\n        0 => Err(Cow::from(\"Invalid empty sequence set.\")),\n        _ => Ok(Sequence::List {\n            items: sequence_set,\n        }),\n    }\n}\n\npub trait PushUnique<T> {\n    fn push_unique(&mut self, value: T);\n}\n\nimpl<T: PartialEq> PushUnique<T> for Vec<T> {\n    fn push_unique(&mut self, value: T) {\n        if !self.contains(&value) {\n            self.push(value);\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::Sequence;\n\n    #[test]\n    fn parse_sequence_set() {\n        for (sequence, expected_result) in [\n            (\"$\", Sequence::SavedSearch),\n            (\n                \"*\",\n                Sequence::Range {\n                    start: None,\n                    end: None,\n                },\n            ),\n            (\n                \"1,3000:3021\",\n                Sequence::List {\n                    items: vec![\n                        Sequence::Number { value: 1 },\n                        Sequence::Range {\n                            start: 3000.into(),\n                            end: 3021.into(),\n                        },\n                    ],\n                },\n            ),\n            (\n                \"2,4:7,9,12:*\",\n                Sequence::List {\n                    items: vec![\n                        Sequence::Number { value: 2 },\n                        Sequence::Range {\n                            start: 4.into(),\n                            end: 7.into(),\n                        },\n                        Sequence::Number { value: 9 },\n                        Sequence::Range {\n                            start: 12.into(),\n                            end: None,\n                        },\n                    ],\n                },\n            ),\n            (\n                \"*:4,5:7\",\n                Sequence::List {\n                    items: vec![\n                        Sequence::Range {\n                            start: None,\n                            end: 4.into(),\n                        },\n                        Sequence::Range {\n                            start: 5.into(),\n                            end: 7.into(),\n                        },\n                    ],\n                },\n            ),\n            (\n                \"2,4,5\",\n                Sequence::List {\n                    items: vec![\n                        Sequence::Number { value: 2 },\n                        Sequence::Number { value: 4 },\n                        Sequence::Number { value: 5 },\n                    ],\n                },\n            ),\n        ] {\n            assert_eq!(\n                super::parse_sequence_set(sequence.as_bytes()).unwrap(),\n                expected_result\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/quota.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::quota,\n    receiver::{Request, bad},\n    utf7::utf7_maybe_decode,\n};\n\nimpl Request<Command> {\n    pub fn parse_get_quota_root(self, is_utf8: bool) -> trc::Result<quota::Arguments> {\n        match self.tokens.len() {\n            1 => Ok(quota::Arguments {\n                name: utf7_maybe_decode(\n                    self.tokens\n                        .into_iter()\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    is_utf8,\n                ),\n                tag: self.tag,\n            }),\n            0 => Err(self.into_error(\"Missing mailbox name.\")),\n            _ => Err(self.into_error(\"Too many arguments.\")),\n        }\n    }\n\n    pub fn parse_get_quota(self) -> trc::Result<quota::Arguments> {\n        match self.tokens.len() {\n            1 => Ok(quota::Arguments {\n                name: self\n                    .tokens\n                    .into_iter()\n                    .next()\n                    .unwrap()\n                    .unwrap_string()\n                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                tag: self.tag,\n            }),\n            0 => Err(self.into_error(\"Missing quota root.\")),\n            _ => Err(self.into_error(\"Too many arguments.\")),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{protocol::quota, receiver::Receiver};\n\n    #[test]\n    fn parse_quota() {\n        let mut receiver = Receiver::new();\n\n        let (command, arguments) = (\n            \"A142 GETQUOTAROOT INBOX\\r\\n\",\n            quota::Arguments {\n                name: \"INBOX\".into(),\n                tag: \"A142\".into(),\n            },\n        );\n        assert_eq!(\n            receiver\n                .parse(&mut command.as_bytes().iter())\n                .unwrap()\n                .parse_get_quota_root(true)\n                .unwrap(),\n            arguments\n        );\n\n        let (command, arguments) = (\n            \"A142 GETQUOTA \\\"my funky mailbox\\\"\\r\\n\",\n            quota::Arguments {\n                name: \"my funky mailbox\".into(),\n                tag: \"A142\".into(),\n            },\n        );\n        assert_eq!(\n            receiver\n                .parse(&mut command.as_bytes().iter())\n                .unwrap()\n                .parse_get_quota()\n                .unwrap(),\n            arguments\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/rename.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::rename,\n    receiver::{Request, bad},\n    utf7::utf7_maybe_decode,\n};\n\nimpl Request<Command> {\n    pub fn parse_rename(self, is_utf8: bool) -> trc::Result<rename::Arguments> {\n        match self.tokens.len() {\n            2 => {\n                let mut tokens = self.tokens.into_iter();\n                Ok(rename::Arguments {\n                    mailbox_name: utf7_maybe_decode(\n                        tokens\n                            .next()\n                            .unwrap()\n                            .unwrap_string()\n                            .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                        is_utf8,\n                    ),\n                    new_mailbox_name: utf7_maybe_decode(\n                        tokens\n                            .next()\n                            .unwrap()\n                            .unwrap_string()\n                            .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                        is_utf8,\n                    ),\n                    tag: self.tag,\n                })\n            }\n            0 => Err(self.into_error(\"Missing argument.\")),\n            1 => Err(self.into_error(\"Missing new mailbox name.\")),\n            _ => Err(self.into_error(\"Too many arguments.\")),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{protocol::rename, receiver::Receiver};\n\n    #[test]\n    fn parse_rename() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A142 RENAME \\\"my funky mailbox\\\" Private\\r\\n\",\n                rename::Arguments {\n                    mailbox_name: \"my funky mailbox\".into(),\n                    new_mailbox_name: \"Private\".into(),\n                    tag: \"A142\".into(),\n                },\n            ),\n            (\n                \"A142 RENAME {1+}\\r\\na {1+}\\r\\nb\\r\\n\",\n                rename::Arguments {\n                    mailbox_name: \"a\".into(),\n                    new_mailbox_name: \"b\".into(),\n                    tag: \"A142\".into(),\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_rename(true)\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\nuse std::iter::Peekable;\nuse std::vec::IntoIter;\n\nuse compact_str::ToCompactString;\nuse mail_parser::decoders::charsets::DecoderFnc;\nuse mail_parser::decoders::charsets::map::charset_decoder;\n\nuse crate::Command;\nuse crate::protocol::search::{self, Filter};\nuse crate::protocol::search::{ModSeqEntry, ResultOption};\nuse crate::protocol::{Flag, ProtocolVersion};\nuse crate::receiver::{Request, Token, bad};\n\nuse super::{parse_date, parse_number, parse_sequence_set};\n\nimpl Request<Command> {\n    #[allow(clippy::while_let_on_iterator)]\n    pub fn parse_search(self, version: ProtocolVersion) -> trc::Result<search::Arguments> {\n        if self.tokens.is_empty() {\n            return Err(self.into_error(\"Missing search criteria.\"));\n        }\n\n        let mut tokens = self.tokens.into_iter().peekable();\n        let mut result_options = Vec::new();\n        let mut decoder = None;\n        let mut is_esearch = version.is_rev2();\n\n        loop {\n            match tokens.peek() {\n                Some(Token::Argument(value)) if value.eq_ignore_ascii_case(b\"return\") => {\n                    tokens.next();\n                    is_esearch = true;\n                    result_options = parse_result_options(&mut tokens)\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n                }\n                Some(Token::Argument(value)) if value.eq_ignore_ascii_case(b\"charset\") => {\n                    tokens.next();\n                    decoder = charset_decoder(\n                        &tokens\n                            .next()\n                            .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing charset.\"))?\n                            .unwrap_bytes(),\n                    );\n                }\n                _ => break,\n            }\n        }\n\n        let filter = parse_filters(&mut tokens, decoder)\n            .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n\n        match filter.len() {\n            0 => Err(bad(\n                self.tag.to_compact_string(),\n                \"No filters found in command.\",\n            )),\n            _ => Ok(search::Arguments {\n                tag: self.tag,\n                result_options,\n                filter,\n                sort: None,\n                is_esearch,\n            }),\n        }\n    }\n}\n\npub fn parse_result_options(\n    tokens: &mut Peekable<IntoIter<Token>>,\n) -> super::Result<Vec<ResultOption>> {\n    let mut result_options = Vec::new();\n    if tokens\n        .next()\n        .is_none_or(|token| !token.is_parenthesis_open())\n    {\n        return Err(Cow::from(\"Invalid result option, expected parenthesis.\"));\n    }\n\n    for token in tokens {\n        match token {\n            Token::ParenthesisClose => break,\n            Token::Argument(value) => {\n                result_options.push(ResultOption::parse(&value)?);\n            }\n            _ => return Err(Cow::from(\"Invalid result option argument.\")),\n        }\n    }\n\n    Ok(result_options)\n}\n\npub fn parse_filters(\n    tokens: &mut Peekable<IntoIter<Token>>,\n    decoder: Option<DecoderFnc>,\n) -> super::Result<Vec<Filter>> {\n    let mut filters = Vec::new();\n    let mut filters_len = 0;\n    let mut filters_stack = Vec::new();\n    let mut operator = Filter::And;\n\n    while let Some(token) = tokens.next() {\n        let mut found_parenthesis = false;\n        match token {\n            Token::Argument(value) => {\n                hashify::fnc_map_ignore_case!(value.as_slice(),\n                    \"ALL\" => {\n                        filters.push(Filter::All);\n\n                    },\n                    \"ANSWERED\" => {\n                        filters.push(Filter::Answered);\n\n                    },\n                    \"BCC\" => {\n                        filters.push(Filter::Bcc(decode_argument(tokens, decoder)?));\n\n                    },\n                    \"BEFORE\" => {\n                        filters.push(Filter::Before(parse_date(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected date\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n                    },\n                    \"BODY\" => {\n                        filters.push(Filter::Body(decode_argument(tokens, decoder)?));\n\n\n                    },\n                    \"CC\" => {\n                        filters.push(Filter::Cc(decode_argument(tokens, decoder)?));\n\n\n                    },\n                    \"DELETED\" => {\n                        filters.push(Filter::Deleted);\n\n\n                    },\n                    \"DRAFT\" => {\n                        filters.push(Filter::Draft);\n\n\n                    },\n                    \"FLAGGED\" => {\n                        filters.push(Filter::Flagged);\n\n\n                    },\n                    \"FROM\" => {\n                        filters.push(Filter::From(decode_argument(tokens, decoder)?));\n\n\n                    },\n                    \"HEADER\" => {\n                        filters.push(Filter::Header(\n                            decode_argument(tokens, decoder)?,\n                            decode_argument(tokens, decoder)?,\n                        ));\n\n\n                    },\n                    \"KEYWORD\" => {\n                        filters.push(Filter::Keyword(Flag::parse_imap(\n                            tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected keyword\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"LARGER\" => {\n                        filters.push(Filter::Larger(parse_number::<u32>(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected integer\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"ON\" => {\n                        filters.push(Filter::On(parse_date(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected date\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"SEEN\" => {\n                        filters.push(Filter::Seen);\n\n\n                    },\n                    \"SENTBEFORE\" => {\n                        filters.push(Filter::SentBefore(parse_date(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected date\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"SENTON\" => {\n                        filters.push(Filter::SentOn(parse_date(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected date\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"SENTSINCE\" => {\n                        filters.push(Filter::SentSince(parse_date(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected date\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"SINCE\" => {\n                        filters.push(Filter::Since(parse_date(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected date\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"SMALLER\" => {\n                        filters.push(Filter::Smaller(parse_number::<u32>(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected integer\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"SUBJECT\" => {\n                        filters.push(Filter::Subject(decode_argument(tokens, decoder)?));\n\n\n                    },\n                    \"TEXT\" => {\n                        filters.push(Filter::Text(decode_argument(tokens, decoder)?));\n\n\n                    },\n                    \"TO\" => {\n                        filters.push(Filter::To(decode_argument(tokens, decoder)?));\n\n\n                    },\n                    \"UID\" => {\n                        filters.push(Filter::Sequence(\n                            parse_sequence_set(\n                                &tokens\n                                    .next()\n                                    .ok_or_else(|| Cow::from(\"Missing sequence set.\"))?\n                                    .unwrap_bytes(),\n                            )?,\n                            true,\n                        ));\n\n\n                    },\n                    \"UNANSWERED\" => {\n                        filters.push(Filter::Unanswered);\n\n\n                    },\n                    \"UNDELETED\" => {\n                        filters.push(Filter::Undeleted);\n\n\n                    },\n                    \"UNDRAFT\" => {\n                        filters.push(Filter::Undraft);\n\n\n                    },\n                    \"UNFLAGGED\" => {\n                        filters.push(Filter::Unflagged);\n\n\n                    },\n                    \"UNKEYWORD\" => {\n                        filters.push(Filter::Unkeyword(Flag::parse_imap(\n                            tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected keyword\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"UNSEEN\" => {\n                        filters.push(Filter::Unseen);\n\n\n                    },\n                    \"OLDER\" => {\n                        filters.push(Filter::Older(parse_number::<u32>(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected integer\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"YOUNGER\" => {\n                        filters.push(Filter::Younger(parse_number::<u32>(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected integer\"))?\n                                .unwrap_bytes(),\n                        )?));\n\n\n                    },\n                    \"OLD\" => {\n                        filters.push(Filter::Old);\n\n                    },\n                    \"NEW\" => {\n                        filters.push(Filter::New);\n\n                    },\n                    \"RECENT\" => {\n                        filters.push(Filter::Recent);\n\n                    },\n                    \"MODSEQ\" => {\n                        let param = tokens\n                            .next()\n                            .ok_or_else(|| Cow::from(\"Missing MODSEQ parameters.\"))?\n                            .unwrap_bytes();\n                        if param.is_empty() || param.iter().any(|ch| !ch.is_ascii_digit()) {\n                            if param.len() <= 7 || !param.starts_with(b\"/flags/\") {\n                                return Err(format!(\n                                    \"Unsupported MODSEQ parameter '{}'.\",\n                                    String::from_utf8_lossy(&param)\n                                )\n                                .into());\n                            }\n                            let flag = Flag::parse_imap((param[7..]).to_vec())?;\n                            let mod_seq_entry = match tokens.next() {\n                                Some(Token::Argument(value)) if value.eq_ignore_ascii_case(b\"all\") => {\n                                    ModSeqEntry::All(flag)\n                                }\n                                Some(Token::Argument(value))\n                                    if value.eq_ignore_ascii_case(b\"shared\") =>\n                                {\n                                    ModSeqEntry::Shared(flag)\n                                }\n                                Some(Token::Argument(value)) if value.eq_ignore_ascii_case(b\"priv\") => {\n                                    ModSeqEntry::Private(flag)\n                                }\n                                Some(token) => {\n                                    return Err(\n                                        format!(\"Unsupported MODSEQ parameter '{}'.\", token).into()\n                                    );\n                                }\n                                None => {\n                                    return Err(\"Missing MODSEQ entry-type-req parameter.\".into());\n                                }\n                            };\n                            filters.push(Filter::ModSeq((\n                                parse_number::<u64>(\n                                    &tokens\n                                        .next()\n                                        .ok_or_else(|| {\n                                            Cow::from(\"Missing MODSEQ mod-sequence-valzer parameter.\")\n                                        })?\n                                        .unwrap_bytes(),\n                                )?,\n                                mod_seq_entry,\n                            )));\n                        } else {\n                            filters.push(Filter::ModSeq((\n                                parse_number::<u64>(&param)?,\n                                ModSeqEntry::None,\n                            )));\n                        }\n\n                    },\n                    \"EMAILID\" => {\n                        filters.push(Filter::EmailId(\n                            tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected an EMAILID value.\"))?\n                                .unwrap_string()?,\n                        ));\n\n                    },\n                    \"THREADID\" => {\n                        filters.push(Filter::ThreadId(\n                            tokens\n                                .next()\n                                .ok_or_else(|| Cow::from(\"Expected an THREADID value.\"))?\n                                .unwrap_string()?,\n                        ));\n\n                    },\n                    \"OR\" => {\n                        if filters_stack.len() > 10 {\n                            return Err(Cow::from(\"Too many nested filters\"));\n                        }\n\n                        filters_stack.push((filters, operator, filters_len));\n                        filters_len = 0;\n                        filters = Vec::with_capacity(2);\n                        operator = Filter::Or;\n                        continue;\n                    },\n                    \"NOT\" => {\n                        if filters_stack.len() > 10 {\n                            return Err(Cow::from(\"Too many nested filters\"));\n                        }\n\n                        filters_stack.push((filters, operator, filters_len));\n                        filters_len = 0;\n                        filters = Vec::with_capacity(1);\n                        operator = Filter::Not;\n                        continue;\n                    },\n                    _ => {\n                        filters.push(Filter::Sequence(parse_sequence_set(&value)?, false));\n                    }\n                );\n\n                filters_len += 1;\n            }\n            Token::ParenthesisOpen => {\n                if filters_stack.len() > 10 {\n                    return Err(Cow::from(\"Too many nested filters\"));\n                }\n\n                filters_stack.push((filters, operator, filters_len));\n                filters_len = 0;\n                filters = Vec::with_capacity(5);\n                operator = Filter::And;\n                continue;\n            }\n            Token::ParenthesisClose => {\n                if filters_stack.is_empty() {\n                    return Err(Cow::from(\"Unexpected parenthesis.\"));\n                }\n\n                found_parenthesis = true;\n            }\n            token => return Err(format!(\"Unexpected token {:?}.\", token.to_string()).into()),\n        }\n\n        if !filters_stack.is_empty()\n            && (found_parenthesis\n                || (operator == Filter::Or && filters_len == 2)\n                || (operator == Filter::Not && filters_len == 1))\n        {\n            while let Some((mut prev_filters, prev_operator, prev_filters_len)) =\n                filters_stack.pop()\n            {\n                if operator == Filter::And && (prev_operator != Filter::Or || filters_len == 1) {\n                    prev_filters.extend(filters);\n                    filters_len += prev_filters_len;\n                } else {\n                    prev_filters.push(operator);\n                    prev_filters.extend(filters);\n                    prev_filters.push(Filter::End);\n                    filters_len = prev_filters_len + 1;\n                }\n                operator = prev_operator;\n                filters = prev_filters;\n\n                if operator == Filter::And || (operator == Filter::Or && filters_len < 2) {\n                    break;\n                }\n            }\n        }\n    }\n    Ok(filters)\n}\n\npub fn decode_argument(\n    tokens: &mut Peekable<IntoIter<Token>>,\n    decoder: Option<DecoderFnc>,\n) -> super::Result<String> {\n    let argument = tokens\n        .next()\n        .ok_or_else(|| Cow::from(\"Expected string.\"))?\n        .unwrap_bytes();\n\n    if let Some(decoder) = decoder {\n        Ok(decoder(&argument))\n    } else {\n        Ok(String::from_utf8(argument).map_err(|_| Cow::from(\"Invalid UTF-8 argument.\"))?)\n    }\n}\n\nimpl ResultOption {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(\n            value,\n            \"min\" => Self::Min,\n            \"max\" => Self::Max,\n            \"all\" => Self::All,\n            \"count\" => Self::Count,\n            \"save\" => Self::Save,\n            \"context\" => Self::Context,\n        )\n        .ok_or_else(|| {\n            format!(\n                \"Invalid result option '{}'.\",\n                String::from_utf8_lossy(value)\n            )\n            .into()\n        })\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::{\n            Flag, ProtocolVersion, Sequence,\n            search::{self, Filter, ModSeqEntry, ResultOption},\n        },\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_search() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                b\"A282 SEARCH RETURN (MIN COUNT) FLAGGED SINCE 1-Feb-1994 NOT FROM \\\"Smith\\\"\\r\\n\"\n                    .to_vec(),\n                search::Arguments {\n                    tag: \"A282\".into(),\n                    result_options: vec![ResultOption::Min, ResultOption::Count],\n                    filter: vec![\n                        Filter::Flagged,\n                        Filter::Since(760060800),\n                        Filter::Not,\n                        Filter::From(\"Smith\".into()),\n                        Filter::End,\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                b\"A283 SEARCH RETURN () FLAGGED SINCE 1-Feb-1994 NOT FROM \\\"Smith\\\"\\r\\n\".to_vec(),\n                search::Arguments {\n                    tag: \"A283\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::Flagged,\n                        Filter::Since(760060800),\n                        Filter::Not,\n                        Filter::From(\"Smith\".into()),\n                        Filter::End,\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                b\"A301 SEARCH $ SMALLER 4096\\r\\n\".to_vec(),\n                search::Arguments {\n                    tag: \"A301\".into(),\n                    result_options: vec![],\n                    filter: vec![Filter::seq_saved_search(), Filter::Smaller(4096)],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                \"P283 SEARCH CHARSET UTF-8 (OR $ 1,3000:3021) TEXT {8+}\\r\\nмать\\r\\n\"\n                    .as_bytes()\n                    .to_vec(),\n                search::Arguments {\n                    tag: \"P283\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::Or,\n                        Filter::seq_saved_search(),\n                        Filter::Sequence(\n                            Sequence::List {\n                                items: vec![\n                                    Sequence::number(1),\n                                    Sequence::range(3000.into(), 3021.into()),\n                                ],\n                            },\n                            false,\n                        ),\n                        Filter::End,\n                        Filter::Text(\"мать\".into()),\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                b\"F282 SEARCH RETURN (SAVE) KEYWORD $Junk\\r\\n\".to_vec(),\n                search::Arguments {\n                    tag: \"F282\".into(),\n                    result_options: vec![ResultOption::Save],\n                    filter: vec![Filter::Keyword(Flag::Junk)],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                [\n                    b\"F282 SEARCH OR OR FROM hello@world.com TO \".to_vec(),\n                    b\"test@example.com OR BCC jane@foobar.com \".to_vec(),\n                    b\"CC john@doe.com\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                search::Arguments {\n                    tag: \"F282\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::Or,\n                        Filter::Or,\n                        Filter::From(\"hello@world.com\".into()),\n                        Filter::To(\"test@example.com\".into()),\n                        Filter::End,\n                        Filter::Or,\n                        Filter::Bcc(\"jane@foobar.com\".into()),\n                        Filter::Cc(\"john@doe.com\".into()),\n                        Filter::End,\n                        Filter::End,\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                [\n                    b\"abc SEARCH OR SMALLER 10000 OR \".to_vec(),\n                    b\"HEADER Subject \\\"ravioli festival\\\" \".to_vec(),\n                    b\"HEADER From \\\"dr. ravioli\\\"\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                search::Arguments {\n                    tag: \"abc\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::Or,\n                        Filter::Smaller(10000),\n                        Filter::Or,\n                        Filter::Header(\"Subject\".into(), \"ravioli festival\".into()),\n                        Filter::Header(\"From\".into(), \"dr. ravioli\".into()),\n                        Filter::End,\n                        Filter::End,\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                [\n                    b\"abc SEARCH (DELETED SEEN ANSWERED) \".to_vec(),\n                    b\"NOT (FROM john TO jane BCC bill) \".to_vec(),\n                    b\"(1,30:* UID 1,2,3,4 $)\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                search::Arguments {\n                    tag: \"abc\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::Deleted,\n                        Filter::Seen,\n                        Filter::Answered,\n                        Filter::Not,\n                        Filter::From(\"john\".into()),\n                        Filter::To(\"jane\".into()),\n                        Filter::Bcc(\"bill\".into()),\n                        Filter::End,\n                        Filter::Sequence(\n                            Sequence::List {\n                                items: vec![Sequence::number(1), Sequence::range(30.into(), None)],\n                            },\n                            false,\n                        ),\n                        Filter::Sequence(\n                            Sequence::List {\n                                items: vec![\n                                    Sequence::number(1),\n                                    Sequence::number(2),\n                                    Sequence::number(3),\n                                    Sequence::number(4),\n                                ],\n                            },\n                            true,\n                        ),\n                        Filter::seq_saved_search(),\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                [\n                    b\"abc SEARCH *:* UID *:100,100:* \".to_vec(),\n                    b\"(FLAGGED (DRAFT (DELETED (ANSWERED)))) \".to_vec(),\n                    b\"OR (SENTON 20-Nov-2022) (LARGER 8196)\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                search::Arguments {\n                    tag: \"abc\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::seq_range(None, None),\n                        Filter::Sequence(\n                            Sequence::List {\n                                items: vec![\n                                    Sequence::range(None, 100.into()),\n                                    Sequence::range(100.into(), None),\n                                ],\n                            },\n                            true,\n                        ),\n                        Filter::Flagged,\n                        Filter::Draft,\n                        Filter::Deleted,\n                        Filter::Answered,\n                        Filter::Or,\n                        Filter::SentOn(1668902400),\n                        Filter::Larger(8196),\n                        Filter::End,\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                [\n                    b\"abc SEARCH NOT (FROM john OR TO jane CC bill) \".to_vec(),\n                    b\"OR (UNDELETED ALL) ($ NOT FLAGGED) \".to_vec(),\n                    b\"(((KEYWORD \\\"tps report\\\")))\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                search::Arguments {\n                    tag: \"abc\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::Not,\n                        Filter::From(\"john\".into()),\n                        Filter::Or,\n                        Filter::To(\"jane\".into()),\n                        Filter::Cc(\"bill\".into()),\n                        Filter::End,\n                        Filter::End,\n                        Filter::Or,\n                        Filter::And,\n                        Filter::Undeleted,\n                        Filter::All,\n                        Filter::End,\n                        Filter::And,\n                        Filter::seq_saved_search(),\n                        Filter::Not,\n                        Filter::Flagged,\n                        Filter::End,\n                        Filter::End,\n                        Filter::End,\n                        Filter::Keyword(Flag::Keyword(\"tps report\".into())),\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                [\n                    b\"B283 SEARCH RETURN (SAVE MIN MAX) CHARSET KOI8-R TEXT \".to_vec(),\n                    b\"{11+}\\r\\n\\xf0\\xd2\\xc9\\xd7\\xc5\\xd4, \\xcd\\xc9\\xd2\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                search::Arguments {\n                    tag: \"B283\".into(),\n                    result_options: vec![ResultOption::Save, ResultOption::Min, ResultOption::Max],\n                    filter: vec![Filter::Text(\"Привет, мир\".into())],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                b\"B283 SEARCH CHARSET BIG5 FROM \\\"\\xa7A\\xa6n\\xa1A\\xa5@\\xac\\xc9\\\"\\r\\n\".to_vec(),\n                search::Arguments {\n                    tag: \"B283\".into(),\n                    result_options: vec![],\n                    filter: vec![Filter::From(\"你好，世界\".into())],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                b\"a SEARCH MODSEQ \\\"/flags/\\\\draft\\\" all 620162338\\r\\n\".to_vec(),\n                search::Arguments {\n                    tag: \"a\".into(),\n                    result_options: vec![],\n                    filter: vec![Filter::ModSeq((620162338, ModSeqEntry::All(Flag::Draft)))],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                b\"t SEARCH OR NOT MODSEQ 720162338 LARGER 50000\\r\\n\".to_vec(),\n                search::Arguments {\n                    tag: \"t\".into(),\n                    result_options: vec![],\n                    filter: vec![\n                        Filter::Or,\n                        Filter::Not,\n                        Filter::ModSeq((720162338, ModSeqEntry::None)),\n                        Filter::End,\n                        Filter::Larger(50000),\n                        Filter::End,\n                    ],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n            (\n                b\"5 UID SEARCH BEFORE 1-Dec-2023\\r\\n\".to_vec(),\n                search::Arguments {\n                    tag: \"5\".into(),\n                    result_options: vec![],\n                    filter: vec![Filter::Before(1701388800)],\n                    is_esearch: true,\n                    sort: None,\n                },\n            ),\n        ] {\n            let command_str = String::from_utf8_lossy(&command).into_owned();\n            assert_eq!(\n                receiver\n                    .parse(&mut command.iter())\n                    .unwrap()\n                    .parse_search(ProtocolVersion::Rev2)\n                    .expect(&command_str),\n                arguments,\n                \"{}\",\n                command_str\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/select.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::{CompactString, ToCompactString, format_compact};\n\nuse crate::{\n    Command,\n    protocol::select::{self, QResync},\n    receiver::{Request, Token, bad},\n    utf7::utf7_maybe_decode,\n};\n\nuse super::{parse_number, parse_sequence_set};\n\nimpl Request<Command> {\n    pub fn parse_select(self, is_utf8: bool) -> trc::Result<select::Arguments> {\n        if !self.tokens.is_empty() {\n            let mut tokens = self.tokens.into_iter().peekable();\n\n            // Mailbox name\n            let mailbox_name = utf7_maybe_decode(\n                tokens\n                    .next()\n                    .unwrap()\n                    .unwrap_string()\n                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                is_utf8,\n            );\n\n            // CONDSTORE parameters\n            let mut condstore = false;\n            let mut qresync = None;\n            match tokens.next() {\n                Some(Token::ParenthesisOpen) => {\n                    while let Some(token) = tokens.next() {\n                        match token {\n                            Token::Argument(param) if param.eq_ignore_ascii_case(b\"CONDSTORE\") => {\n                                condstore = true;\n                            }\n                            Token::Argument(param) if param.eq_ignore_ascii_case(b\"QRESYNC\") => {\n                                if tokens\n                                    .next()\n                                    .is_none_or(|token| !token.is_parenthesis_open())\n                                {\n                                    return Err(bad(\n                                        CompactString::from_string_buffer(self.tag),\n                                        \"Expected '(' after 'QRESYNC'.\",\n                                    ));\n                                }\n\n                                let uid_validity = parse_number::<u32>(\n                                    &tokens\n                                        .next()\n                                        .ok_or_else(|| {\n                                            bad(\n                                                self.tag.to_compact_string(),\n                                                \"Missing uidvalidity parameter for QRESYNC.\",\n                                            )\n                                        })?\n                                        .unwrap_bytes(),\n                                )\n                                .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n                                let modseq = parse_number::<u64>(\n                                    &tokens\n                                        .next()\n                                        .ok_or_else(|| {\n                                            bad(\n                                                self.tag.to_compact_string(),\n                                                \"Missing modseq parameter for QRESYNC.\",\n                                            )\n                                        })?\n                                        .unwrap_bytes(),\n                                )\n                                .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n\n                                let mut known_uids = None;\n                                let mut seq_match = None;\n                                let has_seq_match = match tokens.peek() {\n                                    Some(Token::Argument(value)) => {\n                                        known_uids = parse_sequence_set(value)\n                                            .map_err(|v| bad(self.tag.to_compact_string(), v))?\n                                            .into();\n                                        tokens.next();\n                                        if matches!(tokens.peek(), Some(Token::ParenthesisOpen)) {\n                                            tokens.next();\n                                            true\n                                        } else {\n                                            false\n                                        }\n                                    }\n                                    Some(Token::ParenthesisOpen) => {\n                                        tokens.next();\n                                        true\n                                    }\n                                    _ => false,\n                                };\n\n                                if has_seq_match {\n                                    seq_match = Some((\n                                        parse_sequence_set(\n                                            &tokens\n                                                .next()\n                                                .ok_or_else(|| {\n                                                    bad(\n                                            self.tag.to_compact_string(),\n                                            \"Missing known-sequence-set parameter for QRESYNC.\",\n                                        )\n                                                })?\n                                                .unwrap_bytes(),\n                                        )\n                                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                        parse_sequence_set(\n                                            &tokens\n                                                .next()\n                                                .ok_or_else(|| {\n                                                    bad(\n                                                self.tag.to_compact_string(),\n                                                \"Missing known-uid-set parameter for QRESYNC.\",\n                                            )\n                                                })?\n                                                .unwrap_bytes(),\n                                        )\n                                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                                    ));\n                                    if tokens\n                                        .next()\n                                        .is_none_or(|token| !token.is_parenthesis_close())\n                                    {\n                                        return Err(bad(\n                                            CompactString::from_string_buffer(self.tag),\n                                            \"Missing ')' for 'QRESYNC'.\",\n                                        ));\n                                    }\n                                }\n\n                                if tokens\n                                    .next()\n                                    .is_none_or(|token| !token.is_parenthesis_close())\n                                {\n                                    return Err(bad(\n                                        CompactString::from_string_buffer(self.tag),\n                                        \"Missing ')' for 'QRESYNC'.\",\n                                    ));\n                                }\n\n                                qresync = QResync {\n                                    uid_validity,\n                                    modseq,\n                                    known_uids,\n                                    seq_match,\n                                }\n                                .into();\n                            }\n                            Token::ParenthesisClose => {\n                                break;\n                            }\n                            _ => {\n                                return Err(bad(\n                                    CompactString::from_string_buffer(self.tag),\n                                    format_compact!(\"Unexpected value '{}'.\", token),\n                                ));\n                            }\n                        }\n                    }\n                }\n                Some(token) => {\n                    return Err(bad(\n                        CompactString::from_string_buffer(self.tag),\n                        format_compact!(\"Unexpected value '{}'.\", token),\n                    ));\n                }\n                None => (),\n            }\n\n            Ok(select::Arguments {\n                mailbox_name,\n                tag: self.tag,\n                condstore,\n                qresync,\n            })\n        } else {\n            Err(self.into_error(\"Missing mailbox name.\"))\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        protocol::{\n            Sequence,\n            select::{self, QResync},\n        },\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_select() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A142 SELECT INBOX\\r\\n\",\n                select::Arguments {\n                    mailbox_name: \"INBOX\".into(),\n                    tag: \"A142\".into(),\n                    condstore: false,\n                    qresync: None,\n                },\n            ),\n            (\n                \"A142 SELECT \\\"my funky mailbox\\\"\\r\\n\",\n                select::Arguments {\n                    mailbox_name: \"my funky mailbox\".into(),\n                    tag: \"A142\".into(),\n                    condstore: false,\n                    qresync: None,\n                },\n            ),\n            (\n                \"A142 SELECT INBOX (CONDSTORE)\\r\\n\",\n                select::Arguments {\n                    mailbox_name: \"INBOX\".into(),\n                    tag: \"A142\".into(),\n                    condstore: true,\n                    qresync: None,\n                },\n            ),\n            (\n                \"A142 SELECT INBOX (QRESYNC (3857529045 20010715194032001 1:198))\\r\\n\",\n                select::Arguments {\n                    mailbox_name: \"INBOX\".into(),\n                    tag: \"A142\".into(),\n                    condstore: false,\n                    qresync: QResync {\n                        uid_validity: 3857529045,\n                        modseq: 20010715194032001,\n                        known_uids: Some(Sequence::Range {\n                            start: Some(1),\n                            end: Some(198),\n                        }),\n                        seq_match: None,\n                    }\n                    .into(),\n                },\n            ),\n            (\n                concat!(\n                    \"A03 SELECT INBOX (QRESYNC (67890007 90060115194045000 \",\n                    \"41:211,214:541) CONDSTORE)\\r\\n\"\n                ),\n                select::Arguments {\n                    mailbox_name: \"INBOX\".into(),\n                    tag: \"A03\".into(),\n                    condstore: true,\n                    qresync: QResync {\n                        uid_validity: 67890007,\n                        modseq: 90060115194045000,\n                        known_uids: Some(Sequence::List {\n                            items: vec![\n                                Sequence::Range {\n                                    start: Some(41),\n                                    end: Some(211),\n                                },\n                                Sequence::Range {\n                                    start: Some(214),\n                                    end: Some(541),\n                                },\n                            ],\n                        }),\n                        seq_match: None,\n                    }\n                    .into(),\n                },\n            ),\n            (\n                concat!(\n                    \"B04 SELECT INBOX (QRESYNC (67890007 \",\n                    \"90060115194045000 1:29997 (5000,7500,9000,9990:9999 15000,\",\n                    \"22500,27000,29970,29973,29976,29979,29982,29985,29988,29991,\",\n                    \"29994,29997)))\\r\\n\"\n                ),\n                select::Arguments {\n                    mailbox_name: \"INBOX\".into(),\n                    tag: \"B04\".into(),\n                    condstore: false,\n                    qresync: QResync {\n                        uid_validity: 67890007,\n                        modseq: 90060115194045000,\n                        known_uids: Some(Sequence::Range {\n                            start: Some(1),\n                            end: Some(29997),\n                        }),\n                        seq_match: Some((\n                            Sequence::List {\n                                items: vec![\n                                    Sequence::Number { value: 5000 },\n                                    Sequence::Number { value: 7500 },\n                                    Sequence::Number { value: 9000 },\n                                    Sequence::Range {\n                                        start: Some(9990),\n                                        end: Some(9999),\n                                    },\n                                ],\n                            },\n                            Sequence::List {\n                                items: vec![\n                                    Sequence::Number { value: 15000 },\n                                    Sequence::Number { value: 22500 },\n                                    Sequence::Number { value: 27000 },\n                                    Sequence::Number { value: 29970 },\n                                    Sequence::Number { value: 29973 },\n                                    Sequence::Number { value: 29976 },\n                                    Sequence::Number { value: 29979 },\n                                    Sequence::Number { value: 29982 },\n                                    Sequence::Number { value: 29985 },\n                                    Sequence::Number { value: 29988 },\n                                    Sequence::Number { value: 29991 },\n                                    Sequence::Number { value: 29994 },\n                                    Sequence::Number { value: 29997 },\n                                ],\n                            },\n                        )),\n                    }\n                    .into(),\n                },\n            ),\n            (\n                \"A12 SELECT \\\"INBOX\\\" (QRESYNC (1693237464 16582))\\r\\n\",\n                select::Arguments {\n                    mailbox_name: \"INBOX\".into(),\n                    tag: \"A12\".into(),\n                    condstore: false,\n                    qresync: QResync {\n                        uid_validity: 1693237464,\n                        modseq: 16582,\n                        known_uids: None,\n                        seq_match: None,\n                    }\n                    .into(),\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap_or_else(|err| panic!(\n                        \"Failed to parse command '{}': {:?}\",\n                        command, err\n                    ))\n                    .parse_select(true)\n                    .unwrap_or_else(|err| panic!(\n                        \"Failed to parse command '{}': {:?}\",\n                        command, err\n                    )),\n                arguments,\n                \"Failed to parse {}\",\n                command\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/sort.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\nuse mail_parser::decoders::charsets::map::charset_decoder;\n\nuse crate::{\n    Command,\n    protocol::search::{Arguments, Comparator, Sort},\n    receiver::{Request, Token, bad},\n};\n\nuse super::search::{parse_filters, parse_result_options};\n\nimpl Request<Command> {\n    #[allow(clippy::while_let_on_iterator)]\n    pub fn parse_sort(self) -> trc::Result<Arguments> {\n        if self.tokens.is_empty() {\n            return Err(self.into_error(\"Missing sort criteria.\"));\n        }\n\n        let mut tokens = self.tokens.into_iter().peekable();\n        let mut sort = Vec::new();\n\n        let (result_options, is_esearch) = match tokens.peek() {\n            Some(Token::Argument(value)) if value.eq_ignore_ascii_case(b\"return\") => {\n                tokens.next();\n                (\n                    parse_result_options(&mut tokens)\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    true,\n                )\n            }\n            _ => (Vec::new(), false),\n        };\n\n        if tokens\n            .next()\n            .is_none_or(|token| !token.is_parenthesis_open())\n        {\n            return Err(bad(\n                self.tag.to_compact_string(),\n                \"Expected sort criteria between parentheses.\",\n            ));\n        }\n\n        let mut is_ascending = true;\n        while let Some(token) = tokens.next() {\n            match token {\n                Token::ParenthesisClose => break,\n                Token::Argument(value) => {\n                    if value.eq_ignore_ascii_case(b\"REVERSE\") {\n                        is_ascending = false;\n                    } else {\n                        sort.push(Comparator {\n                            sort: Sort::parse(&value)\n                                .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                            ascending: is_ascending,\n                        });\n                        is_ascending = true;\n                    }\n                }\n                _ => {\n                    return Err(bad(\n                        self.tag.to_compact_string(),\n                        \"Invalid result option argument.\",\n                    ));\n                }\n            }\n        }\n\n        if sort.is_empty() {\n            return Err(bad(self.tag.to_compact_string(), \"Missing sort criteria.\"));\n        }\n\n        let decoder = charset_decoder(\n            &tokens\n                .next()\n                .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing charset.\"))?\n                .unwrap_bytes(),\n        );\n\n        let filter = parse_filters(&mut tokens, decoder)\n            .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n        match filter.len() {\n            0 => Err(bad(\n                self.tag.to_compact_string(),\n                \"No filters found in command.\",\n            )),\n            _ => Ok(Arguments {\n                sort: sort.into(),\n                result_options,\n                filter,\n                is_esearch,\n                tag: self.tag,\n            }),\n        }\n    }\n}\n\nimpl Sort {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(value,\n            \"ARRIVAL\" => Self::Arrival,\n            \"CC\" => Self::Cc,\n            \"DATE\" => Self::Date,\n            \"FROM\" => Self::From,\n            \"SIZE\" => Self::Size,\n            \"SUBJECT\" => Self::Subject,\n            \"TO\" => Self::To,\n            \"DISPLAYFROM\" => Self::DisplayFrom,\n            \"DISPLAYTO\" => Self::DisplayTo,\n        )\n        .ok_or_else(|| format!(\"Invalid sort criteria {:?}\", String::from_utf8_lossy(value)).into())\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::{\n        protocol::{\n            Flag,\n            search::{Arguments, Comparator, Filter, ResultOption, Sort},\n        },\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_sort() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                b\"A282 SORT (SUBJECT) UTF-8 SINCE 1-Feb-1994\\r\\n\".to_vec(),\n                Arguments {\n                    sort: vec![Comparator {\n                        sort: Sort::Subject,\n                        ascending: true,\n                    }]\n                    .into(),\n                    filter: vec![Filter::Since(760060800)],\n                    result_options: Vec::new(),\n                    is_esearch: false,\n                    tag: \"A282\".into(),\n                },\n            ),\n            (\n                b\"A283 SORT (SUBJECT REVERSE DATE) UTF-8 ALL\\r\\n\".to_vec(),\n                Arguments {\n                    sort: vec![\n                        Comparator {\n                            sort: Sort::Subject,\n                            ascending: true,\n                        },\n                        Comparator {\n                            sort: Sort::Date,\n                            ascending: false,\n                        },\n                    ]\n                    .into(),\n                    filter: vec![Filter::All],\n                    result_options: Vec::new(),\n                    is_esearch: false,\n                    tag: \"A283\".into(),\n                },\n            ),\n            (\n                b\"A284 SORT (SUBJECT) US-ASCII TEXT \\\"not in mailbox\\\"\\r\\n\".to_vec(),\n                Arguments {\n                    sort: vec![Comparator {\n                        sort: Sort::Subject,\n                        ascending: true,\n                    }]\n                    .into(),\n                    filter: vec![Filter::Text(\"not in mailbox\".into())],\n                    result_options: Vec::new(),\n                    is_esearch: false,\n                    tag: \"A284\".into(),\n                },\n            ),\n            (\n                [\n                    b\"A284 SORT (REVERSE ARRIVAL FROM) iso-8859-6 SUBJECT \".to_vec(),\n                    b\"\\\"\\xe5\\xd1\\xcd\\xc8\\xc7 \\xc8\\xc7\\xe4\\xd9\\xc7\\xe4\\xe5\\\"\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                Arguments {\n                    sort: vec![\n                        Comparator {\n                            sort: Sort::Arrival,\n                            ascending: false,\n                        },\n                        Comparator {\n                            sort: Sort::From,\n                            ascending: true,\n                        },\n                    ]\n                    .into(),\n                    filter: vec![Filter::Subject(\"مرحبا بالعالم\".into())],\n                    result_options: Vec::new(),\n                    is_esearch: false,\n                    tag: \"A284\".into(),\n                },\n            ),\n            (\n                [\n                    b\"E01 UID SORT RETURN (COUNT) (REVERSE DATE) \".to_vec(),\n                    b\"UTF-8 UNDELETED UNKEYWORD $Junk\\r\\n\".to_vec(),\n                ]\n                .concat(),\n                Arguments {\n                    sort: vec![Comparator {\n                        sort: Sort::Date,\n                        ascending: false,\n                    }]\n                    .into(),\n                    filter: vec![Filter::Undeleted, Filter::Unkeyword(Flag::Junk)],\n                    result_options: vec![ResultOption::Count],\n                    is_esearch: true,\n                    tag: \"E01\".into(),\n                },\n            ),\n        ] {\n            let command_str = String::from_utf8_lossy(&command).into_owned();\n\n            assert_eq!(\n                receiver\n                    .parse(&mut command.iter())\n                    .unwrap()\n                    .parse_sort()\n                    .expect(&command_str),\n                arguments,\n                \"{}\",\n                command_str\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/status.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::{CompactString, ToCompactString};\n\nuse crate::Command;\nuse crate::protocol::status;\nuse crate::protocol::status::Status;\nuse crate::receiver::{Request, Token, bad};\nuse crate::utf7::utf7_maybe_decode;\n\nimpl Request<Command> {\n    pub fn parse_status(self, is_utf8: bool) -> trc::Result<status::Arguments> {\n        match self.tokens.len() {\n            0..=3 => Err(self.into_error(\"Missing arguments.\")),\n            len => {\n                let mut tokens = self.tokens.into_iter();\n                let mailbox_name = utf7_maybe_decode(\n                    tokens\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    is_utf8,\n                );\n                let mut items = Vec::with_capacity(len - 2);\n\n                if tokens\n                    .next()\n                    .is_none_or(|token| !token.is_parenthesis_open())\n                {\n                    return Err(bad(\n                        self.tag.to_compact_string(),\n                        \"Expected parenthesis after mailbox name.\",\n                    ));\n                }\n\n                #[allow(clippy::while_let_on_iterator)]\n                while let Some(token) = tokens.next() {\n                    match token {\n                        Token::ParenthesisClose => break,\n                        Token::Argument(value) => {\n                            items.push(\n                                Status::parse(&value)\n                                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                            );\n                        }\n                        _ => {\n                            return Err(bad(\n                                self.tag.to_compact_string(),\n                                \"Invalid status return option argument.\",\n                            ));\n                        }\n                    }\n                }\n\n                if !items.is_empty() {\n                    Ok(status::Arguments {\n                        tag: self.tag,\n                        mailbox_name,\n                        items,\n                    })\n                } else {\n                    Err(bad(\n                        CompactString::from_string_buffer(self.tag),\n                        \"At least one status item is required.\",\n                    ))\n                }\n            }\n        }\n    }\n}\n\nimpl Status {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(value,\n            \"MESSAGES\" => Self::Messages,\n            \"UIDNEXT\" => Self::UidNext,\n            \"UIDVALIDITY\" => Self::UidValidity,\n            \"UNSEEN\" => Self::Unseen,\n            \"DELETED\" => Self::Deleted,\n            \"SIZE\" => Self::Size,\n            \"HIGHESTMODSEQ\" => Self::HighestModSeq,\n            \"MAILBOXID\" => Self::MailboxId,\n            \"RECENT\" => Self::Recent,\n            \"DELETED-STORAGE\" => Self::DeletedStorage\n        )\n        .ok_or_else(|| {\n            format!(\n                \"Invalid status option '{}'.\",\n                String::from_utf8_lossy(value)\n            )\n            .into()\n        })\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{protocol::status, receiver::Receiver};\n\n    #[test]\n    fn parse_status() {\n        let mut receiver = Receiver::new();\n\n        assert_eq!(\n            receiver\n                .parse(\n                    &mut \"A042 STATUS blurdybloop (UIDNEXT MESSAGES)\\r\\n\"\n                        .as_bytes()\n                        .iter()\n                )\n                .unwrap()\n                .parse_status(true)\n                .unwrap(),\n            status::Arguments {\n                tag: \"A042\".into(),\n                mailbox_name: \"blurdybloop\".into(),\n                items: vec![status::Status::UidNext, status::Status::Messages],\n            }\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/store.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::{CompactString, ToCompactString, format_compact};\n\nuse crate::{\n    Command,\n    protocol::{\n        Flag,\n        store::{self, Operation},\n    },\n    receiver::{Request, Token, bad},\n};\n\nuse super::{parse_number, parse_sequence_set};\n\nimpl Request<Command> {\n    pub fn parse_store(self) -> trc::Result<store::Arguments> {\n        let mut tokens = self.tokens.into_iter().peekable();\n\n        // Sequence set\n        let sequence_set = parse_sequence_set(\n            &tokens\n                .next()\n                .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing sequence set.\"))?\n                .unwrap_bytes(),\n        )\n        .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n        let mut unchanged_since = None;\n\n        // CONDSTORE parameters\n        if let Some(Token::ParenthesisOpen) = tokens.peek() {\n            tokens.next();\n            while let Some(token) = tokens.next() {\n                match token {\n                    Token::Argument(param) if param.eq_ignore_ascii_case(b\"UNCHANGEDSINCE\") => {\n                        unchanged_since = parse_number::<u64>(\n                            &tokens\n                                .next()\n                                .ok_or_else(|| {\n                                    bad(\n                                        self.tag.to_compact_string(),\n                                        \"Missing UNCHANGEDSINCE parameter.\",\n                                    )\n                                })?\n                                .unwrap_bytes(),\n                        )\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?\n                        .into();\n                    }\n                    Token::ParenthesisClose => {\n                        break;\n                    }\n                    _ => {\n                        return Err(bad(\n                            self.tag.to_compact_string(),\n                            format_compact!(\"Unsupported parameter '{}'.\", token),\n                        ));\n                    }\n                }\n            }\n        }\n\n        // Operation\n        let operation = tokens\n            .next()\n            .ok_or_else(|| {\n                bad(\n                    self.tag.to_compact_string(),\n                    \"Missing message data item name.\",\n                )\n            })?\n            .unwrap_bytes();\n        let (is_silent, operation) = hashify::tiny_map_ignore_case!(operation.as_slice(),\n            \"FLAGS\" => (false, Operation::Set),\n            \"FLAGS.SILENT\" => (true, Operation::Set),\n            \"+FLAGS\" => (false, Operation::Add),\n            \"+FLAGS.SILENT\" => (true, Operation::Add),\n            \"-FLAGS\" => (false, Operation::Clear),\n            \"-FLAGS.SILENT\" => (true, Operation::Clear),\n        )\n        .ok_or_else(|| {\n            bad(\n                self.tag.to_compact_string(),\n                format_compact!(\n                    \"Unsupported message data item name: {:?}\",\n                    String::from_utf8_lossy(&operation)\n                ),\n            )\n        })?;\n\n        // Flags\n        let mut keywords = Vec::new();\n        match tokens\n            .next()\n            .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing flags to set.\"))?\n        {\n            Token::ParenthesisOpen => {\n                for token in tokens {\n                    match token {\n                        Token::Argument(flag) => {\n                            keywords.push(\n                                Flag::parse_imap(flag)\n                                    .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                            );\n                        }\n                        Token::ParenthesisClose => {\n                            break;\n                        }\n                        _ => {\n                            return Err(bad(self.tag.to_compact_string(), \"Unsupported flag.\"));\n                        }\n                    }\n                }\n            }\n            Token::Argument(flag) => {\n                keywords.push(\n                    Flag::parse_imap(flag).map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                );\n            }\n            _ => {\n                return Err(bad(\n                    CompactString::from_string_buffer(self.tag),\n                    \"Invalid flags parameter.\",\n                ));\n            }\n        }\n\n        if !keywords.is_empty() || operation == Operation::Set {\n            Ok(store::Arguments {\n                tag: self.tag,\n                sequence_set,\n                operation,\n                is_silent,\n                keywords,\n                unchanged_since,\n            })\n        } else {\n            Err(bad(self.tag.to_compact_string(), \"Missing flags to set.\"))\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::{\n        protocol::{\n            Flag, Sequence,\n            store::{self, Operation},\n        },\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_store() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A003 STORE 2:4 +FLAGS (\\\\Deleted)\\r\\n\",\n                store::Arguments {\n                    sequence_set: Sequence::Range {\n                        start: 2.into(),\n                        end: 4.into(),\n                    },\n                    is_silent: false,\n                    operation: Operation::Add,\n                    keywords: vec![Flag::Deleted],\n                    tag: \"A003\".into(),\n                    unchanged_since: None,\n                },\n            ),\n            (\n                \"A004 STORE *:100 -FLAGS.SILENT ($Phishing $Junk)\\r\\n\",\n                store::Arguments {\n                    sequence_set: Sequence::Range {\n                        start: None,\n                        end: 100.into(),\n                    },\n                    is_silent: true,\n                    operation: Operation::Clear,\n                    keywords: vec![Flag::Phishing, Flag::Junk],\n                    tag: \"A004\".into(),\n                    unchanged_since: None,\n                },\n            ),\n            (\n                \"d105 STORE 7,5,9 (UNCHANGEDSINCE 320162338) +FLAGS.SILENT \\\\Deleted\\r\\n\",\n                store::Arguments {\n                    sequence_set: Sequence::List {\n                        items: vec![\n                            Sequence::Number { value: 7 },\n                            Sequence::Number { value: 5 },\n                            Sequence::Number { value: 9 },\n                        ],\n                    },\n                    is_silent: true,\n                    operation: Operation::Add,\n                    keywords: vec![Flag::Deleted],\n                    tag: \"d105\".into(),\n                    unchanged_since: Some(320162338),\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_store()\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/subscribe.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\n\nuse crate::{\n    Command,\n    protocol::subscribe,\n    receiver::{Request, bad},\n    utf7::utf7_maybe_decode,\n};\n\nimpl Request<Command> {\n    pub fn parse_subscribe(self, is_utf8: bool) -> trc::Result<subscribe::Arguments> {\n        match self.tokens.len() {\n            1 => Ok(subscribe::Arguments {\n                mailbox_name: utf7_maybe_decode(\n                    self.tokens\n                        .into_iter()\n                        .next()\n                        .unwrap()\n                        .unwrap_string()\n                        .map_err(|v| bad(self.tag.to_compact_string(), v))?,\n                    is_utf8,\n                ),\n                tag: self.tag,\n            }),\n            0 => Err(self.into_error(\"Missing mailbox name.\")),\n            _ => Err(self.into_error(\"Too many arguments.\")),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{protocol::subscribe, receiver::Receiver};\n\n    #[test]\n    fn parse_subscribe() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                \"A142 SUBSCRIBE #news.comp.mail.mime\\r\\n\",\n                subscribe::Arguments {\n                    mailbox_name: \"#news.comp.mail.mime\".into(),\n                    tag: \"A142\".into(),\n                },\n            ),\n            (\n                \"A142 SUBSCRIBE \\\"#news.comp.mail.mime\\\"\\r\\n\",\n                subscribe::Arguments {\n                    mailbox_name: \"#news.comp.mail.mime\".into(),\n                    tag: \"A142\".into(),\n                },\n            ),\n        ] {\n            assert_eq!(\n                receiver\n                    .parse(&mut command.as_bytes().iter())\n                    .unwrap()\n                    .parse_subscribe(true)\n                    .unwrap(),\n                arguments\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/parser/thread.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::ToCompactString;\nuse mail_parser::decoders::charsets::map::charset_decoder;\n\nuse crate::{\n    Command,\n    protocol::thread::{self, Algorithm},\n    receiver::{Request, bad},\n};\n\nuse super::search::parse_filters;\n\nimpl Request<Command> {\n    #[allow(clippy::while_let_on_iterator)]\n    pub fn parse_thread(self) -> trc::Result<thread::Arguments> {\n        if self.tokens.is_empty() {\n            return Err(self.into_error(\"Missing thread criteria.\"));\n        }\n\n        let mut tokens = self.tokens.into_iter().peekable();\n        let algorithm = Algorithm::parse(\n            &tokens\n                .next()\n                .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing threading algorithm.\"))?\n                .unwrap_bytes(),\n        )\n        .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n\n        let decoder = charset_decoder(\n            &tokens\n                .next()\n                .ok_or_else(|| bad(self.tag.to_compact_string(), \"Missing charset.\"))?\n                .unwrap_bytes(),\n        );\n\n        let filter = parse_filters(&mut tokens, decoder)\n            .map_err(|v| bad(self.tag.to_compact_string(), v))?;\n        match filter.len() {\n            0 => Err(bad(\n                self.tag.to_compact_string(),\n                \"No filters found in command.\",\n            )),\n            _ => Ok(thread::Arguments {\n                algorithm,\n                filter,\n                tag: self.tag,\n            }),\n        }\n    }\n}\n\nimpl Algorithm {\n    pub fn parse(value: &[u8]) -> super::Result<Self> {\n        hashify::tiny_map_ignore_case!(value,\n            \"ORDEREDSUBJECT\" => Self::OrderedSubject,\n            \"REFERENCES\" => Self::References,\n        )\n        .ok_or_else(|| {\n            format!(\n                \"Invalid threading algorithm {:?}\",\n                String::from_utf8_lossy(value)\n            )\n            .into()\n        })\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::{\n        protocol::{\n            search::Filter,\n            thread::{self, Algorithm},\n        },\n        receiver::Receiver,\n    };\n\n    #[test]\n    fn parse_thread() {\n        let mut receiver = Receiver::new();\n\n        for (command, arguments) in [\n            (\n                b\"A283 THREAD ORDEREDSUBJECT UTF-8 SINCE 5-MAR-2000\\r\\n\".to_vec(),\n                thread::Arguments {\n                    algorithm: Algorithm::OrderedSubject,\n                    filter: vec![Filter::Since(952214400)],\n                    tag: \"A283\".into(),\n                },\n            ),\n            (\n                b\"A284 THREAD REFERENCES US-ASCII TEXT \\\"gewp\\\"\\r\\n\".to_vec(),\n                thread::Arguments {\n                    algorithm: Algorithm::References,\n                    filter: vec![Filter::Text(\"gewp\".into())],\n                    tag: \"A284\".into(),\n                },\n            ),\n        ] {\n            let command_str = String::from_utf8_lossy(&command).into_owned();\n\n            assert_eq!(\n                receiver\n                    .parse(&mut command.iter())\n                    .unwrap()\n                    .parse_thread()\n                    .expect(&command_str),\n                arguments,\n                \"{}\",\n                command_str\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n/*\n\n   l - lookup (mailbox is visible to LIST/LSUB commands, SUBSCRIBE\n       mailbox)\n   r - read (SELECT the mailbox, perform STATUS)\n   s - keep seen/unseen information across sessions (set or clear\n       \\SEEN flag via STORE, also set \\SEEN during APPEND/COPY/\n       FETCH BODY[...])\n   w - write (set or clear flags other than \\SEEN and \\DELETED via\n       STORE, also set them during APPEND/COPY)\n   i - insert (perform APPEND, COPY into mailbox)\n   p - post (send mail to submission address for mailbox,\n       not enforced by IMAP4 itself)\n   k - create mailboxes (CREATE new sub-mailboxes in any\n       implementation-defined hierarchy, parent mailbox for the new\n       mailbox name in RENAME)\n   x - delete mailbox (DELETE mailbox, old mailbox name in RENAME)\n   t - delete messages (set or clear \\DELETED flag via STORE, set\n       \\DELETED flag during APPEND/COPY)\n   e - perform EXPUNGE and expunge as a part of CLOSE\n   a - administer (perform SETACL/DELETEACL/GETACL/LISTRIGHTS)\n\n   // RFC2086\n   c - create (CREATE new sub-mailboxes in any implementation-defined\n       hierarchy)\n   d - delete (STORE DELETED flag, perform EXPUNGE)\n\n*/\n\nuse types::acl::Acl;\n\nuse super::quoted_string;\nuse crate::utf7::utf7_encode;\nuse std::fmt::Display;\n\n#[derive(Debug, PartialEq, Eq, Clone, Copy)]\npub enum Rights {\n    Lookup,\n    Read,\n    Seen,\n    Write,\n    Insert,\n    Post,\n    CreateMailbox,\n    DeleteMailbox,\n    DeleteMessages,\n    Expunge,\n    Administer,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone)]\npub struct ModRights {\n    pub op: ModRightsOp,\n    pub rights: Vec<Rights>,\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, Copy)]\npub enum ModRightsOp {\n    Add,\n    Remove,\n    Replace,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n    pub identifier: Option<String>,\n    pub mod_rights: Option<ModRights>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct GetAclResponse {\n    pub mailbox_name: String,\n    pub permissions: Vec<(String, Vec<Rights>)>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct ListRightsResponse {\n    pub mailbox_name: String,\n    pub identifier: String,\n    pub permissions: Vec<Vec<Rights>>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct MyRightsResponse {\n    pub mailbox_name: String,\n    pub rights: Vec<Rights>,\n}\n\nimpl GetAclResponse {\n    pub fn into_bytes(self, is_utf8: bool) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(self.mailbox_name.len() + 10 * self.permissions.len() * 5);\n        buf.extend_from_slice(b\"* ACL \");\n        if is_utf8 {\n            quoted_string(&mut buf, &self.mailbox_name);\n        } else {\n            quoted_string(&mut buf, &utf7_encode(&self.mailbox_name));\n        }\n        for (identifier, rights) in self.permissions {\n            buf.extend_from_slice(b\" \");\n            quoted_string(&mut buf, &identifier);\n            buf.extend_from_slice(b\" \");\n\n            for right in rights {\n                buf.push(right.to_char());\n            }\n        }\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\nimpl ListRightsResponse {\n    pub fn into_bytes(self, is_utf8: bool) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(\n            self.mailbox_name.len() + self.identifier.len() + 10 * self.permissions.len() * 5,\n        );\n        buf.extend_from_slice(b\"* LISTRIGHTS \");\n        if is_utf8 {\n            quoted_string(&mut buf, &self.mailbox_name);\n        } else {\n            quoted_string(&mut buf, &utf7_encode(&self.mailbox_name));\n        }\n        buf.extend_from_slice(b\" \");\n        quoted_string(&mut buf, &self.identifier);\n        for rights in self.permissions {\n            buf.extend_from_slice(b\" \");\n            for right in rights {\n                buf.push(right.to_char());\n            }\n        }\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\nimpl MyRightsResponse {\n    pub fn into_bytes(self, is_utf8: bool) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(self.mailbox_name.len() + 10 + self.rights.len());\n        buf.extend_from_slice(b\"* MYRIGHTS \");\n        if is_utf8 {\n            quoted_string(&mut buf, &self.mailbox_name);\n        } else {\n            quoted_string(&mut buf, &utf7_encode(&self.mailbox_name));\n        }\n        buf.extend_from_slice(b\" \");\n        for right in self.rights {\n            buf.push(right.to_char());\n        }\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\nimpl Rights {\n    pub fn to_char(&self) -> u8 {\n        match self {\n            Rights::Lookup => b'l',\n            Rights::Read => b'r',\n            Rights::Seen => b's',\n            Rights::Write => b'w',\n            Rights::Insert => b'i',\n            Rights::Post => b'p',\n            Rights::CreateMailbox => b'k',\n            Rights::DeleteMailbox => b'x',\n            Rights::DeleteMessages => b't',\n            Rights::Expunge => b'e',\n            Rights::Administer => b'a',\n        }\n    }\n}\n\nimpl Display for Rights {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Rights::Lookup => write!(f, \"l\"),\n            Rights::Read => write!(f, \"r\"),\n            Rights::Seen => write!(f, \"s\"),\n            Rights::Write => write!(f, \"w\"),\n            Rights::Insert => write!(f, \"i\"),\n            Rights::Post => write!(f, \"p\"),\n            Rights::CreateMailbox => write!(f, \"k\"),\n            Rights::DeleteMailbox => write!(f, \"x\"),\n            Rights::DeleteMessages => write!(f, \"t\"),\n            Rights::Expunge => write!(f, \"e\"),\n            Rights::Administer => write!(f, \"a\"),\n        }\n    }\n}\n\nimpl From<Rights> for Acl {\n    fn from(value: Rights) -> Self {\n        match value {\n            Rights::Lookup => Acl::Read,\n            Rights::Read => Acl::ReadItems,\n            Rights::Seen => Acl::ModifyItems,\n            Rights::Write => Acl::ModifyItems,\n            Rights::Insert => Acl::AddItems,\n            Rights::Post => Acl::Submit,\n            Rights::CreateMailbox => Acl::CreateChild,\n            Rights::DeleteMailbox => Acl::Delete,\n            Rights::DeleteMessages => Acl::RemoveItems,\n            Rights::Expunge => Acl::RemoveItems,\n            Rights::Administer => Acl::Share,\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::protocol::acl::{GetAclResponse, ListRightsResponse, MyRightsResponse, Rights};\n\n    #[test]\n    fn serialize_acl() {\n        assert_eq!(\n            String::from_utf8(\n                GetAclResponse {\n                    mailbox_name: \"INBOX\".into(),\n                    permissions: vec![\n                        (\n                            \"Fred\".into(),\n                            vec![\n                                Rights::Lookup,\n                                Rights::Read,\n                                Rights::Seen,\n                                Rights::Write,\n                                Rights::Insert,\n                                Rights::CreateMailbox,\n                                Rights::DeleteMessages,\n                                Rights::Administer,\n                            ]\n                        ),\n                        (\n                            \"David\".into(),\n                            vec![\n                                Rights::CreateMailbox,\n                                Rights::DeleteMessages,\n                                Rights::Administer,\n                            ]\n                        )\n                    ]\n                }\n                .into_bytes(true)\n            )\n            .unwrap(),\n            \"* ACL \\\"INBOX\\\" \\\"Fred\\\" lrswikta \\\"David\\\" kta\\r\\n\"\n        );\n\n        assert_eq!(\n            String::from_utf8(\n                ListRightsResponse {\n                    mailbox_name: \"Deleted Items\".into(),\n                    identifier: \"Fred\".into(),\n                    permissions: vec![\n                        vec![Rights::Lookup, Rights::Read],\n                        vec![Rights::Administer],\n                        vec![Rights::DeleteMailbox]\n                    ]\n                }\n                .into_bytes(true)\n            )\n            .unwrap(),\n            \"* LISTRIGHTS \\\"Deleted Items\\\" \\\"Fred\\\" lr a x\\r\\n\"\n        );\n\n        assert_eq!(\n            String::from_utf8(\n                MyRightsResponse {\n                    mailbox_name: \"Important\".into(),\n                    rights: vec![Rights::Lookup, Rights::Read, Rights::DeleteMailbox]\n                }\n                .into_bytes(true)\n            )\n            .unwrap(),\n            \"* MYRIGHTS \\\"Important\\\" lrx\\r\\n\"\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/append.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::Flag;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n    pub messages: Vec<Message>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Message {\n    pub message: Vec<u8>,\n    pub flags: Vec<Flag>,\n    pub received_at: Option<i64>,\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/authenticate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mechanism: Mechanism,\n    pub params: Vec<String>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Mechanism {\n    Plain,\n    CramMd5,\n    DigestMd5,\n    ScramSha1,\n    ScramSha256,\n    Apop,\n    Ntlm,\n    Gssapi,\n    Anonymous,\n    External,\n    OAuthBearer,\n    XOauth2,\n}\n\nimpl Mechanism {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(match self {\n            Mechanism::Plain => b\"PLAIN\",\n            Mechanism::CramMd5 => b\"CRAM-MD5\",\n            Mechanism::DigestMd5 => b\"DIGEST-MD5\",\n            Mechanism::ScramSha1 => b\"SCRAM-SHA-1\",\n            Mechanism::ScramSha256 => b\"SCRAM-SHA-256\",\n            Mechanism::Apop => b\"APOP\",\n            Mechanism::Ntlm => b\"NTLM\",\n            Mechanism::Gssapi => b\"GSSAPI\",\n            Mechanism::Anonymous => b\"ANONYMOUS\",\n            Mechanism::External => b\"EXTERNAL\",\n            Mechanism::OAuthBearer => b\"OAUTHBEARER\",\n            Mechanism::XOauth2 => b\"XOAUTH2\",\n        });\n    }\n\n    pub fn into_bytes(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(10);\n        self.serialize(&mut buf);\n        buf\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/capability.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapResponse, authenticate::Mechanism};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Response {\n    pub capabilities: Vec<Capability>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Capability {\n    IMAP4rev2,\n    IMAP4rev1,\n    StartTLS,\n    LoginDisabled,\n    Idle,\n    Namespace,\n    Id,\n    Children,\n    MultiAppend,\n    Binary,\n    Unselect,\n    ACL,\n    UIDPlus,\n    ESearch,\n    SASLIR, //SASL-IR\n    Within,\n    Enable,\n    SearchRes,\n    Sort,\n    Thread,       //THREAD=REFERENCES\n    ListExtended, //LIST-EXTENDED\n    ListStatus,   //LIST-STATUS\n    ESort,\n    SortDisplay,      //SORT=DISPLAY\n    SpecialUse,       //SPECIAL-USE\n    CreateSpecialUse, //CREATE-SPECIAL-USEE\n    Move,\n    CondStore,\n    QResync,\n    LiteralPlus, //LITERAL+\n    UnAuthenticate,\n    StatusSize, //STATUS=SIZE\n    ObjectId,\n    Preview,\n    Utf8Accept,\n    Auth(Mechanism),\n    Quota,\n    QuotaResource(QuotaResourceName),\n    QuotaSet,\n    JmapAccess,\n}\n\n/*\n\nSTORAGE \tThe physical space estimate, in units of 1024 octets, of the mailboxes governed by the quota root. \tDELETED-STORAGE STATUS request data item and response data item \tN/A \t[Alexey_Melnikov] \t[IESG] \t[RFC9208, Section 5.1]\nMESSAGE \tThe number of messages stored within the mailboxes governed by the quota root. \tDELETED STATUS request data item and response data item \tN/A \t[Alexey_Melnikov] \t[IESG] \t[RFC9208, Section 5.2]\nMAILBOX \tThe number of mailboxes governed by the quota root. \tN/A \tN/A \t[Alexey_Melnikov] \t[IESG] \t[RFC9208, Section 5.3]\nANNOTATION-STORAGE\n\n*/\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum QuotaResourceName {\n    Storage,\n    Message,\n    Mailbox,\n    AnnotationStorage,\n}\n\nimpl Capability {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(match self {\n            Capability::Auth(mechanism) => {\n                buf.extend_from_slice(b\"AUTH=\");\n                mechanism.serialize(buf);\n                return;\n            }\n            Capability::IMAP4rev2 => b\"IMAP4rev2\",\n            Capability::IMAP4rev1 => b\"IMAP4rev1\",\n            Capability::StartTLS => b\"STARTTLS\",\n            Capability::LoginDisabled => b\"LOGINDISABLED\",\n            Capability::CondStore => b\"CONDSTORE\",\n            Capability::QResync => b\"QRESYNC\",\n            Capability::LiteralPlus => b\"LITERAL+\",\n            Capability::UnAuthenticate => b\"UNAUTHENTICATE\",\n            Capability::StatusSize => b\"STATUS=SIZE\",\n            Capability::ObjectId => b\"OBJECTID\",\n            Capability::Preview => b\"PREVIEW\",\n            Capability::Idle => b\"IDLE\",\n            Capability::Namespace => b\"NAMESPACE\",\n            Capability::Id => b\"ID\",\n            Capability::Children => b\"CHILDREN\",\n            Capability::MultiAppend => b\"MULTIAPPEND\",\n            Capability::Binary => b\"BINARY\",\n            Capability::Unselect => b\"UNSELECT\",\n            Capability::ACL => b\"ACL\",\n            Capability::UIDPlus => b\"UIDPLUS\",\n            Capability::ESearch => b\"ESEARCH\",\n            Capability::SASLIR => b\"SASL-IR\",\n            Capability::Within => b\"WITHIN\",\n            Capability::Enable => b\"ENABLE\",\n            Capability::SearchRes => b\"SEARCHRES\",\n            Capability::Sort => b\"SORT\",\n            Capability::Thread => b\"THREAD=REFERENCES\",\n            Capability::ListExtended => b\"LIST-EXTENDED\",\n            Capability::ListStatus => b\"LIST-STATUS\",\n            Capability::ESort => b\"ESORT\",\n            Capability::SortDisplay => b\"SORT=DISPLAY\",\n            Capability::SpecialUse => b\"SPECIAL-USE\",\n            Capability::CreateSpecialUse => b\"CREATE-SPECIAL-USE\",\n            Capability::Move => b\"MOVE\",\n            Capability::Utf8Accept => b\"UTF8=ACCEPT\",\n            Capability::Quota => b\"QUOTA\",\n            Capability::QuotaResource(quota_resource) => {\n                buf.extend_from_slice(b\"QUOTA=RES-\");\n                buf.extend_from_slice(match quota_resource {\n                    QuotaResourceName::Storage => b\"STORAGE\",\n                    QuotaResourceName::Message => b\"MESSAGE\",\n                    QuotaResourceName::Mailbox => b\"MAILBOX\",\n                    QuotaResourceName::AnnotationStorage => b\"ANNOTATION-STORAGE\",\n                });\n                return;\n            }\n            Capability::QuotaSet => b\"QUOTA=SET\",\n            Capability::JmapAccess => b\"JMAPACCESS\",\n        });\n    }\n\n    pub fn all_capabilities(is_authenticated: bool, offer_tls: bool) -> Vec<Capability> {\n        let mut capabilities = vec![\n            Capability::IMAP4rev2,\n            Capability::IMAP4rev1,\n            Capability::Enable,\n            Capability::SASLIR,\n            Capability::LiteralPlus,\n            Capability::Id,\n            Capability::Utf8Accept,\n            Capability::JmapAccess,\n        ];\n\n        if is_authenticated {\n            capabilities.extend([\n                Capability::Idle,\n                Capability::Namespace,\n                Capability::Children,\n                Capability::MultiAppend,\n                Capability::Binary,\n                Capability::Unselect,\n                Capability::ACL,\n                Capability::UIDPlus,\n                Capability::ESearch,\n                Capability::Within,\n                Capability::SearchRes,\n                Capability::Sort,\n                Capability::Thread,\n                Capability::ListExtended,\n                Capability::ListStatus,\n                Capability::ESort,\n                Capability::SortDisplay,\n                Capability::SpecialUse,\n                Capability::CreateSpecialUse,\n                Capability::Move,\n                Capability::CondStore,\n                Capability::QResync,\n                Capability::UnAuthenticate,\n                Capability::StatusSize,\n                Capability::ObjectId,\n                Capability::Preview,\n                Capability::Quota,\n                Capability::QuotaResource(QuotaResourceName::Storage),\n            ]);\n        } else {\n            capabilities.extend([\n                Capability::Auth(Mechanism::Plain),\n                Capability::Auth(Mechanism::OAuthBearer),\n                Capability::Auth(Mechanism::XOauth2),\n            ]);\n        }\n        if offer_tls {\n            capabilities.push(Capability::StartTLS);\n        }\n\n        capabilities\n    }\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        buf.extend_from_slice(b\"* CAPABILITY\");\n        for capability in self.capabilities.iter() {\n            buf.push(b' ');\n            capability.serialize(&mut buf);\n        }\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::{\n        ImapResponse,\n        capability::{Capability, Response},\n    };\n\n    #[test]\n    fn serialize_capability() {\n        assert_eq!(\n            &Response {\n                capabilities: vec![\n                    Capability::IMAP4rev2,\n                    Capability::StartTLS,\n                    Capability::LoginDisabled\n                ],\n            }\n            .serialize(),\n            \"* CAPABILITY IMAP4rev2 STARTTLS LOGINDISABLED\\r\\n\".as_bytes()\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::Sequence;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub sequence_set: Sequence,\n    pub mailbox_name: String,\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/create.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::list::Attribute;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n    pub mailbox_role: Option<Attribute>,\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/enable.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapResponse, capability::Capability};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub capabilities: Vec<Capability>,\n}\n\npub struct Response {\n    pub enabled: Vec<Capability>,\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        if !self.enabled.is_empty() {\n            let mut buf = Vec::with_capacity(64);\n            buf.extend(b\"* ENABLED\");\n            for capability in self.enabled {\n                buf.push(b' ');\n                capability.serialize(&mut buf);\n            }\n            buf.push(b'\\r');\n            buf.push(b'\\n');\n            buf\n        } else {\n            Vec::new()\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/expunge.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapResponse, serialize_sequence};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Response {\n    pub is_qresync: bool,\n    pub ids: Vec<u32>,\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        self.serialize_to(&mut buf);\n        buf\n    }\n}\n\nimpl Response {\n    pub fn serialize_to(self, buf: &mut Vec<u8>) {\n        if !self.is_qresync {\n            for (num_deletions, id) in self.ids.into_iter().enumerate() {\n                buf.extend_from_slice(b\"* \");\n                buf.extend_from_slice(\n                    id.saturating_sub(num_deletions as u32)\n                        .to_string()\n                        .as_bytes(),\n                );\n                buf.extend_from_slice(b\" EXPUNGE\\r\\n\");\n            }\n        } else {\n            Vanished {\n                earlier: false,\n                ids: self.ids,\n            }\n            .serialize(buf);\n        }\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Vanished {\n    pub earlier: bool,\n    pub ids: Vec<u32>,\n}\n\nimpl Vanished {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        if self.earlier {\n            buf.extend_from_slice(b\"* VANISHED (EARLIER) \");\n        } else {\n            buf.extend_from_slice(b\"* VANISHED \");\n        }\n        serialize_sequence(buf, &self.ids);\n        buf.extend_from_slice(b\"\\r\\n\");\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::ImapResponse;\n\n    #[test]\n    fn serialize_expunge() {\n        assert_eq!(\n            String::from_utf8(\n                super::Response {\n                    is_qresync: false,\n                    ids: vec![3, 4, 5]\n                }\n                .serialize()\n            )\n            .unwrap(),\n            concat!(\"* 3 EXPUNGE\\r\\n\", \"* 3 EXPUNGE\\r\\n\", \"* 3 EXPUNGE\\r\\n\",)\n        );\n\n        assert_eq!(\n            String::from_utf8(\n                super::Response {\n                    is_qresync: false,\n                    ids: vec![3, 4, 7, 9, 11]\n                }\n                .serialize()\n            )\n            .unwrap(),\n            concat!(\n                \"* 3 EXPUNGE\\r\\n\",\n                \"* 3 EXPUNGE\\r\\n\",\n                \"* 5 EXPUNGE\\r\\n\",\n                \"* 6 EXPUNGE\\r\\n\",\n                \"* 7 EXPUNGE\\r\\n\",\n            )\n        );\n\n        assert_eq!(\n            String::from_utf8(\n                super::Response {\n                    is_qresync: true,\n                    ids: vec![3, 4, 5]\n                }\n                .serialize()\n            )\n            .unwrap(),\n            \"* VANISHED 3:5\\r\\n\"\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/fetch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse mail_parser::DateTime;\nuse utils::chained_bytes::SliceRange;\n\nuse crate::protocol::literal_string_slice;\n\nuse super::{\n    Flag, ImapResponse, Sequence, literal_string, quoted_or_literal_string,\n    quoted_or_literal_string_or_nil, quoted_rfc2822_or_nil, quoted_timestamp,\n};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub sequence_set: Sequence,\n    pub attributes: Vec<Attribute>,\n    pub changed_since: Option<u64>,\n    pub include_vanished: bool,\n}\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Response<'x> {\n    pub is_uid: bool,\n    pub items: Vec<FetchItem<'x>>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct FetchItem<'x> {\n    pub id: u32,\n    pub items: Vec<DataItem<'x>>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Attribute {\n    Envelope,\n    Flags,\n    InternalDate,\n    Rfc822,\n    Rfc822Size,\n    Rfc822Header,\n    Rfc822Text,\n    Body,\n    BodyStructure,\n    BodySection {\n        peek: bool,\n        sections: Vec<Section>,\n        partial: Option<(u32, u32)>,\n    },\n    Uid,\n    Binary {\n        peek: bool,\n        sections: Vec<u32>,\n        partial: Option<(u32, u32)>,\n    },\n    BinarySize {\n        sections: Vec<u32>,\n    },\n    Preview {\n        lazy: bool,\n    },\n    ModSeq,\n    EmailId,\n    ThreadId,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Section {\n    Part { num: u32 },\n    Header,\n    HeaderFields { not: bool, fields: Vec<String> },\n    Text,\n    Mime,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum DataItem<'x> {\n    Binary {\n        sections: Vec<u32>,\n        offset: Option<u32>,\n        contents: BodyContents<'x>,\n    },\n    BinarySize {\n        sections: Vec<u32>,\n        size: usize,\n    },\n    Body {\n        part: BodyPart<'x>,\n    },\n    BodyStructure {\n        part: BodyPart<'x>,\n    },\n    BodySection {\n        sections: Vec<Section>,\n        origin_octet: Option<u32>,\n        contents: Cow<'x, [u8]>,\n    },\n    Envelope {\n        envelope: Envelope<'x>,\n    },\n    Flags {\n        flags: Vec<Flag>,\n    },\n    InternalDate {\n        date: i64,\n    },\n    Uid {\n        uid: u32,\n    },\n    Rfc822 {\n        contents: SliceRange<'x>,\n    },\n    Rfc822Header {\n        contents: SliceRange<'x>,\n    },\n    Rfc822Size {\n        size: usize,\n    },\n    Rfc822Text {\n        contents: SliceRange<'x>,\n    },\n    Preview {\n        contents: Option<Cow<'x, [u8]>>,\n    },\n    ModSeq {\n        modseq: u64,\n    },\n    EmailId {\n        email_id: String,\n    },\n    ThreadId {\n        thread_id: String,\n    },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Address<'x> {\n    Single(EmailAddress<'x>),\n    Group(AddressGroup<'x>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct AddressGroup<'x> {\n    pub name: Option<Cow<'x, str>>,\n    pub addresses: Vec<EmailAddress<'x>>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct EmailAddress<'x> {\n    pub name: Option<Cow<'x, str>>,\n    pub address: Cow<'x, str>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum BodyContents<'x> {\n    Text(Cow<'x, str>),\n    Bytes(Cow<'x, [u8]>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\npub struct Envelope<'x> {\n    pub date: Option<DateTime>,\n    pub subject: Option<Cow<'x, str>>,\n    pub from: Vec<Address<'x>>,\n    pub sender: Vec<Address<'x>>,\n    pub reply_to: Vec<Address<'x>>,\n    pub to: Vec<Address<'x>>,\n    pub cc: Vec<Address<'x>>,\n    pub bcc: Vec<Address<'x>>,\n    pub in_reply_to: Option<Cow<'x, str>>,\n    pub message_id: Option<Cow<'x, str>>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\n#[allow(clippy::type_complexity)]\npub enum BodyPart<'x> {\n    Multipart {\n        body_parts: Vec<BodyPart<'x>>,\n        body_subtype: Cow<'x, str>,\n        // Extension data\n        body_parameters: Option<Vec<(Cow<'x, str>, Cow<'x, str>)>>,\n        extension: BodyPartExtension<'x>,\n    },\n    Basic {\n        body_type: Option<Cow<'x, str>>,\n        fields: BodyPartFields<'x>,\n        // Extension data\n        body_md5: Option<Cow<'x, str>>,\n        extension: BodyPartExtension<'x>,\n    },\n    Text {\n        fields: BodyPartFields<'x>,\n        body_size_lines: usize,\n        // Extension data\n        body_md5: Option<Cow<'x, str>>,\n        extension: BodyPartExtension<'x>,\n    },\n    Message {\n        fields: BodyPartFields<'x>,\n        envelope: Option<Box<Envelope<'x>>>,\n        body: Option<Box<BodyPart<'x>>>,\n        body_size_lines: usize,\n        // Extension data\n        body_md5: Option<Cow<'x, str>>,\n        extension: BodyPartExtension<'x>,\n    },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\npub struct BodyPartFields<'x> {\n    pub body_subtype: Option<Cow<'x, str>>,\n    pub body_parameters: Option<Vec<(Cow<'x, str>, Cow<'x, str>)>>,\n    pub body_id: Option<Cow<'x, str>>,\n    pub body_description: Option<Cow<'x, str>>,\n    pub body_encoding: Option<Cow<'x, str>>,\n    pub body_size_octets: usize,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\n#[allow(clippy::type_complexity)]\npub struct BodyPartExtension<'x> {\n    pub body_disposition: Option<(Cow<'x, str>, Vec<(Cow<'x, str>, Cow<'x, str>)>)>,\n    pub body_language: Option<Vec<Cow<'x, str>>>,\n    pub body_location: Option<Cow<'x, str>>,\n}\n\nimpl Address<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        match self {\n            Address::Single(addr) => addr.serialize(buf),\n            Address::Group(addr) => addr.serialize(buf),\n        }\n    }\n\n    pub fn into_owned<'y>(self) -> Address<'y> {\n        match self {\n            Address::Single(addr) => Address::Single(addr.into_owned()),\n            Address::Group(addr) => Address::Group(addr.into_owned()),\n        }\n    }\n}\n\nimpl EmailAddress<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.push(b'(');\n        if let Some(name) = &self.name {\n            quoted_or_literal_string(buf, name);\n        } else {\n            buf.extend_from_slice(b\"NIL\");\n        }\n\n        let addr = if let Some((route, addr)) = self.address.split_once(':') {\n            buf.push(b' ');\n            quoted_or_literal_string(buf, route);\n            buf.push(b' ');\n            addr\n        } else {\n            buf.extend_from_slice(b\" NIL \");\n            &self.address\n        };\n\n        if let Some((local, host)) = addr.rsplit_once('@') {\n            quoted_or_literal_string(buf, local);\n            buf.push(b' ');\n            quoted_or_literal_string(buf, host);\n        } else {\n            quoted_or_literal_string(buf, &self.address);\n            buf.extend_from_slice(b\" \\\"\\\"\");\n        }\n        buf.push(b')');\n    }\n\n    pub fn into_owned<'y>(self) -> EmailAddress<'y> {\n        EmailAddress {\n            name: self.name.map(|n| n.into_owned().into()),\n            address: self.address.into_owned().into(),\n        }\n    }\n}\n\nimpl AddressGroup<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(b\"(NIL NIL \");\n        if let Some(name) = &self.name {\n            quoted_or_literal_string(buf, name);\n        } else {\n            buf.extend_from_slice(b\"\\\"\\\"\");\n        }\n        buf.extend_from_slice(b\" NIL)\");\n        for addr in &self.addresses {\n            addr.serialize(buf);\n        }\n        buf.extend_from_slice(b\"(NIL NIL NIL NIL)\");\n    }\n\n    pub fn into_owned<'y>(self) -> AddressGroup<'y> {\n        AddressGroup {\n            name: self.name.map(|n| n.into_owned().into()),\n            addresses: self\n                .addresses\n                .into_iter()\n                .map(|addr| addr.into_owned())\n                .collect(),\n        }\n    }\n}\n\nimpl<'x> BodyPart<'x> {\n    pub fn serialize(&self, buf: &mut Vec<u8>, is_extended: bool) {\n        buf.push(b'(');\n        match self {\n            BodyPart::Multipart {\n                body_parts,\n                body_subtype,\n                body_parameters,\n                extension,\n            } => {\n                for part in body_parts.iter() {\n                    part.serialize(buf, is_extended);\n                }\n                buf.push(b' ');\n                quoted_or_literal_string(buf, body_subtype);\n                if is_extended {\n                    if let Some(body_parameters) = body_parameters {\n                        buf.extend_from_slice(b\" (\");\n                        for (pos, (key, value)) in body_parameters.iter().enumerate() {\n                            if pos > 0 {\n                                buf.push(b' ');\n                            }\n                            quoted_or_literal_string(buf, key);\n                            buf.push(b' ');\n                            quoted_or_literal_string(buf, value);\n                        }\n                        buf.push(b')');\n                    } else {\n                        buf.extend_from_slice(b\" NIL\");\n                    }\n                    buf.push(b' ');\n                    extension.serialize(buf);\n                }\n            }\n            BodyPart::Basic {\n                body_type,\n                fields,\n                body_md5,\n                extension,\n            } => {\n                quoted_or_literal_string_or_nil(buf, body_type.as_deref());\n                buf.push(b' ');\n                fields.serialize(buf);\n                if is_extended {\n                    buf.push(b' ');\n                    quoted_or_literal_string_or_nil(buf, body_md5.as_deref());\n                    buf.push(b' ');\n                    extension.serialize(buf);\n                }\n            }\n            BodyPart::Text {\n                fields,\n                body_size_lines,\n                body_md5,\n                extension,\n            } => {\n                buf.extend_from_slice(b\"\\\"text\\\" \");\n                fields.serialize(buf);\n                buf.push(b' ');\n                buf.extend_from_slice(body_size_lines.to_string().as_bytes());\n                if is_extended {\n                    buf.push(b' ');\n                    quoted_or_literal_string_or_nil(buf, body_md5.as_deref());\n                    buf.push(b' ');\n                    extension.serialize(buf);\n                }\n            }\n            BodyPart::Message {\n                fields,\n                envelope,\n                body,\n                body_size_lines,\n                body_md5,\n                extension,\n            } => {\n                buf.extend_from_slice(b\"\\\"message\\\" \");\n                fields.serialize(buf);\n                buf.push(b' ');\n                if let Some(envelope) = envelope {\n                    envelope.serialize(buf);\n                } else {\n                    buf.extend_from_slice(b\"NIL\");\n                }\n                buf.push(b' ');\n                if let Some(body) = body {\n                    body.serialize(buf, is_extended);\n                } else {\n                    buf.extend_from_slice(b\"NIL\");\n                }\n                buf.push(b' ');\n                buf.extend_from_slice(body_size_lines.to_string().as_bytes());\n                if is_extended {\n                    buf.push(b' ');\n                    quoted_or_literal_string_or_nil(buf, body_md5.as_deref());\n                    buf.push(b' ');\n                    extension.serialize(buf);\n                }\n            }\n        }\n        buf.push(b')');\n    }\n\n    pub fn add_part(&mut self, part: BodyPart<'x>) {\n        match self {\n            BodyPart::Multipart { body_parts, .. } => body_parts.push(part),\n            BodyPart::Message { body, .. } => *body = Box::new(part).into(),\n            _ => debug_assert!(false, \"Cannot add a part to a non-multipart body part\"),\n        }\n    }\n\n    pub fn set_envelope(&mut self, envelope_: Envelope<'x>) {\n        match self {\n            BodyPart::Message { envelope, .. } => *envelope = Some(Box::new(envelope_)),\n            _ => debug_assert!(false, \"Cannot set envelope on a non-message body part\"),\n        }\n    }\n\n    pub fn into_owned<'y>(self) -> BodyPart<'y> {\n        match self {\n            BodyPart::Multipart {\n                body_parts,\n                body_subtype,\n                body_parameters,\n                extension,\n            } => BodyPart::Multipart {\n                body_parts: body_parts.into_iter().map(|v| v.into_owned()).collect(),\n                body_subtype: body_subtype.into_owned().into(),\n                body_parameters: body_parameters.map(|b| {\n                    b.into_iter()\n                        .map(|(k, v)| (k.into_owned().into(), v.into_owned().into()))\n                        .collect::<Vec<_>>()\n                }),\n                extension: extension.into_owned(),\n            },\n            BodyPart::Basic {\n                body_type,\n                fields,\n                body_md5,\n                extension,\n            } => BodyPart::Basic {\n                body_type: body_type.map(|v| v.into_owned().into()),\n                fields: fields.into_owned(),\n                body_md5: body_md5.map(|v| v.into_owned().into()),\n                extension: extension.into_owned(),\n            },\n            BodyPart::Text {\n                fields,\n                body_size_lines,\n                body_md5,\n                extension,\n            } => BodyPart::Text {\n                fields: fields.into_owned(),\n                body_size_lines,\n                body_md5: body_md5.map(|v| v.into_owned().into()),\n                extension: extension.into_owned(),\n            },\n            BodyPart::Message {\n                fields,\n                envelope,\n                body,\n                body_size_lines,\n                body_md5,\n                extension,\n            } => BodyPart::Message {\n                fields: fields.into_owned(),\n                envelope: envelope.map(|v| Box::new(v.into_owned())),\n                body: body.map(|b| Box::new(b.into_owned())),\n                body_size_lines,\n                body_md5: body_md5.map(|v| v.into_owned().into()),\n                extension: extension.into_owned(),\n            },\n        }\n    }\n}\n\nimpl BodyPartFields<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        quoted_or_literal_string_or_nil(buf, self.body_subtype.as_deref());\n        if let Some(body_parameters) = &self.body_parameters {\n            buf.extend_from_slice(b\" (\");\n            for (pos, (key, value)) in body_parameters.iter().enumerate() {\n                if pos > 0 {\n                    buf.push(b' ');\n                }\n                quoted_or_literal_string(buf, key);\n                buf.push(b' ');\n                quoted_or_literal_string(buf, value);\n            }\n            buf.push(b')');\n        } else {\n            buf.extend_from_slice(b\" NIL\");\n        }\n        for item in [&self.body_id, &self.body_description, &self.body_encoding] {\n            buf.push(b' ');\n            quoted_or_literal_string_or_nil(buf, item.as_deref());\n        }\n        buf.push(b' ');\n        buf.extend_from_slice(self.body_size_octets.to_string().as_bytes());\n    }\n\n    pub fn into_owned<'y>(self) -> BodyPartFields<'y> {\n        BodyPartFields {\n            body_subtype: self.body_subtype.map(|v| v.into_owned().into()),\n            body_parameters: self.body_parameters.map(|b| {\n                b.into_iter()\n                    .map(|(k, v)| (k.into_owned().into(), v.into_owned().into()))\n                    .collect::<Vec<_>>()\n            }),\n            body_id: self.body_id.map(|v| v.into_owned().into()),\n            body_description: self.body_description.map(|v| v.into_owned().into()),\n            body_encoding: self.body_encoding.map(|v| v.into_owned().into()),\n            body_size_octets: self.body_size_octets,\n        }\n    }\n}\n\nimpl BodyPartExtension<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        if let Some((disposition, parameters)) = &self.body_disposition {\n            buf.push(b'(');\n            quoted_or_literal_string(buf, disposition);\n            if !parameters.is_empty() {\n                buf.extend_from_slice(b\" (\");\n                for (pos, (key, value)) in parameters.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b' ');\n                    }\n                    quoted_or_literal_string(buf, key);\n                    buf.push(b' ');\n                    quoted_or_literal_string(buf, value);\n                }\n                buf.extend_from_slice(b\"))\");\n            } else {\n                buf.extend_from_slice(b\" NIL)\");\n            }\n        } else {\n            buf.extend_from_slice(b\"NIL\");\n        }\n        if let Some(body_language) = &self.body_language {\n            match body_language.len() {\n                0 => buf.extend_from_slice(b\" NIL\"),\n                1 => {\n                    buf.push(b' ');\n                    quoted_or_literal_string(buf, body_language.last().unwrap());\n                }\n                _ => {\n                    buf.extend_from_slice(b\" (\");\n                    for (pos, lang) in body_language.iter().enumerate() {\n                        if pos > 0 {\n                            buf.push(b' ');\n                        }\n                        quoted_or_literal_string(buf, lang);\n                    }\n                    buf.push(b')');\n                }\n            }\n        } else {\n            buf.extend_from_slice(b\" NIL\");\n        }\n        buf.push(b' ');\n        quoted_or_literal_string_or_nil(buf, self.body_location.as_deref());\n    }\n\n    pub fn into_owned<'y>(self) -> BodyPartExtension<'y> {\n        BodyPartExtension {\n            body_disposition: self.body_disposition.map(|(a, b)| {\n                (\n                    a.into_owned().into(),\n                    b.into_iter()\n                        .map(|(k, v)| (k.into_owned().into(), v.into_owned().into()))\n                        .collect::<Vec<_>>(),\n                )\n            }),\n            body_language: self\n                .body_language\n                .map(|v| v.into_iter().map(|a| a.into_owned().into()).collect()),\n            body_location: self.body_location.map(|v| v.into_owned().into()),\n        }\n    }\n}\n\nimpl BodyContents<'_> {\n    pub fn into_owned<'y>(self) -> BodyContents<'y> {\n        match self {\n            BodyContents::Text(text) => BodyContents::Text(text.into_owned().into()),\n            BodyContents::Bytes(bytes) => BodyContents::Bytes(bytes.into_owned().into()),\n        }\n    }\n}\n\nimpl Section {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        match self {\n            Section::Part { num } => {\n                buf.extend_from_slice(num.to_string().as_bytes());\n            }\n            Section::Header => {\n                buf.extend_from_slice(b\"HEADER\");\n            }\n            Section::HeaderFields { not, fields } => {\n                if !not {\n                    buf.extend_from_slice(b\"HEADER.FIELDS \");\n                } else {\n                    buf.extend_from_slice(b\"HEADER.FIELDS.NOT \");\n                }\n                buf.push(b'(');\n                for (pos, field) in fields.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b' ');\n                    }\n                    buf.extend_from_slice(field.as_str().to_ascii_uppercase().as_bytes());\n                }\n                buf.push(b')');\n            }\n            Section::Text => {\n                buf.extend_from_slice(b\"TEXT\");\n            }\n            Section::Mime => {\n                buf.extend_from_slice(b\"MIME\");\n            }\n        };\n    }\n}\n\nstatic DUMMY_ADDRESS: [Address; 1] = [Address::Single(EmailAddress {\n    name: None,\n    address: Cow::Borrowed(\"unknown@localhost\"),\n})];\n\nimpl Envelope<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.push(b'(');\n        quoted_rfc2822_or_nil(buf, &self.date);\n        buf.push(b' ');\n        quoted_or_literal_string_or_nil(buf, self.subject.as_deref());\n\n        // Note: [RFC-2822] requires that all messages have a valid\n        // From header.  Therefore, the from, sender, and reply-to\n        // members in the envelope can not be NIL.\n\n        let from = if !self.from.is_empty() {\n            &self.from[..]\n        } else {\n            &DUMMY_ADDRESS[..]\n        };\n\n        self.serialize_addresses(buf, from);\n        self.serialize_addresses(\n            buf,\n            if !self.sender.is_empty() {\n                &self.sender\n            } else {\n                from\n            },\n        );\n        self.serialize_addresses(\n            buf,\n            if !self.reply_to.is_empty() {\n                &self.reply_to\n            } else {\n                from\n            },\n        );\n        self.serialize_addresses(buf, &self.to);\n        self.serialize_addresses(buf, &self.cc);\n        self.serialize_addresses(buf, &self.bcc);\n        for item in [&self.in_reply_to, &self.message_id] {\n            buf.push(b' ');\n            quoted_or_literal_string_or_nil(buf, item.as_deref());\n        }\n        buf.push(b')');\n    }\n\n    fn serialize_addresses(&self, buf: &mut Vec<u8>, addresses: &[Address]) {\n        buf.push(b' ');\n        if !addresses.is_empty() {\n            buf.push(b'(');\n            for address in addresses {\n                address.serialize(buf);\n            }\n            buf.push(b')');\n        } else {\n            buf.extend_from_slice(b\"NIL\");\n        }\n    }\n\n    pub fn into_owned<'y>(self) -> Envelope<'y> {\n        Envelope {\n            date: self.date,\n            subject: self.subject.map(|v| v.into_owned().into()),\n            from: self.from.into_iter().map(|v| v.into_owned()).collect(),\n            sender: self.sender.into_iter().map(|v| v.into_owned()).collect(),\n            reply_to: self.reply_to.into_iter().map(|v| v.into_owned()).collect(),\n            to: self.to.into_iter().map(|v| v.into_owned()).collect(),\n            cc: self.cc.into_iter().map(|v| v.into_owned()).collect(),\n            bcc: self.bcc.into_iter().map(|v| v.into_owned()).collect(),\n            in_reply_to: self.in_reply_to.map(|v| v.into_owned().into()),\n            message_id: self.message_id.map(|v| v.into_owned().into()),\n        }\n    }\n}\n\nimpl DataItem<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        match self {\n            DataItem::Binary {\n                sections,\n                offset,\n                contents,\n            } => {\n                buf.extend_from_slice(b\"BINARY[\");\n                for (pos, section) in sections.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b'.');\n                    }\n                    buf.extend_from_slice(section.to_string().as_bytes());\n                }\n                if let Some(offset) = offset {\n                    buf.extend_from_slice(b\"]<\");\n                    buf.extend_from_slice(offset.to_string().as_bytes());\n                    buf.extend_from_slice(b\"> \");\n                } else {\n                    buf.extend_from_slice(b\"] \");\n                }\n                match contents {\n                    BodyContents::Text(text) => {\n                        literal_string(buf, text.as_bytes());\n                    }\n                    BodyContents::Bytes(bytes) => {\n                        buf.extend_from_slice(b\"~{\");\n                        buf.extend_from_slice(bytes.len().to_string().as_bytes());\n                        buf.extend_from_slice(b\"}\\r\\n\");\n                        buf.extend_from_slice(bytes);\n                    }\n                }\n            }\n            DataItem::BinarySize { sections, size } => {\n                buf.extend_from_slice(b\"BINARY.SIZE[\");\n                for (pos, section) in sections.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b'.');\n                    }\n                    buf.extend_from_slice(section.to_string().as_bytes());\n                }\n                buf.extend_from_slice(b\"] \");\n                buf.extend_from_slice(size.to_string().as_bytes());\n            }\n            DataItem::Body { part } => {\n                buf.extend_from_slice(b\"BODY \");\n                part.serialize(buf, false);\n            }\n            DataItem::BodyStructure { part } => {\n                buf.extend_from_slice(b\"BODYSTRUCTURE \");\n                part.serialize(buf, true);\n            }\n            DataItem::BodySection {\n                sections,\n                origin_octet,\n                contents,\n            } => {\n                buf.extend_from_slice(b\"BODY[\");\n                for (pos, section) in sections.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b'.');\n                    }\n                    section.serialize(buf);\n                }\n                if let Some(origin_octet) = origin_octet {\n                    buf.extend_from_slice(b\"]<\");\n                    buf.extend_from_slice(origin_octet.to_string().as_bytes());\n                    buf.extend_from_slice(b\"> \");\n                } else {\n                    buf.extend_from_slice(b\"] \");\n                }\n                literal_string(buf, contents);\n            }\n            DataItem::Envelope { envelope } => {\n                buf.extend_from_slice(b\"ENVELOPE \");\n                envelope.serialize(buf);\n            }\n            DataItem::Flags { flags } => {\n                buf.extend_from_slice(b\"FLAGS (\");\n                for (pos, flag) in flags.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b' ');\n                    }\n                    flag.serialize(buf);\n                }\n                buf.push(b')');\n            }\n            DataItem::InternalDate { date } => {\n                buf.extend_from_slice(b\"INTERNALDATE \");\n                quoted_timestamp(buf, *date);\n            }\n            DataItem::Uid { uid } => {\n                buf.extend_from_slice(b\"UID \");\n                buf.extend_from_slice(uid.to_string().as_bytes());\n            }\n            DataItem::Rfc822 { contents } => {\n                buf.extend_from_slice(b\"RFC822 \");\n                literal_string_slice(buf, contents);\n            }\n            DataItem::Rfc822Header { contents } => {\n                buf.extend_from_slice(b\"RFC822.HEADER \");\n                literal_string_slice(buf, contents);\n            }\n            DataItem::Rfc822Size { size } => {\n                buf.extend_from_slice(b\"RFC822.SIZE \");\n                buf.extend_from_slice(size.to_string().as_bytes());\n            }\n            DataItem::Rfc822Text { contents } => {\n                buf.extend_from_slice(b\"RFC822.TEXT \");\n                literal_string_slice(buf, contents);\n            }\n            DataItem::Preview { contents } => {\n                buf.extend_from_slice(b\"PREVIEW \");\n                if let Some(contents) = contents {\n                    literal_string(buf, contents);\n                } else {\n                    buf.extend_from_slice(b\"NIL\");\n                }\n            }\n            DataItem::ModSeq { modseq } => {\n                buf.extend_from_slice(b\"MODSEQ (\");\n                buf.extend_from_slice(modseq.to_string().as_bytes());\n                buf.push(b')');\n            }\n            DataItem::EmailId { email_id } => {\n                buf.extend_from_slice(b\"EMAILID (\");\n                buf.extend_from_slice(email_id.as_bytes());\n                buf.push(b')');\n            }\n            DataItem::ThreadId { thread_id } => {\n                buf.extend_from_slice(b\"THREADID (\");\n                buf.extend_from_slice(thread_id.as_bytes());\n                buf.push(b')');\n            }\n        }\n    }\n}\n\nimpl FetchItem<'_> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(b\"* \");\n        buf.extend_from_slice(self.id.to_string().as_bytes());\n        buf.extend_from_slice(b\" FETCH (\");\n        for (pos, item) in self.items.iter().enumerate() {\n            if pos > 0 {\n                buf.push(b' ');\n            }\n            item.serialize(buf);\n        }\n        buf.extend_from_slice(b\")\\r\\n\");\n    }\n}\n\nimpl ImapResponse for Response<'_> {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(128);\n        for item in &self.items {\n            item.serialize(&mut buf);\n        }\n        buf\n    }\n}\n\n/*\n\n   body            = \"(\" (body-type-1part / body-type-mpart) \")\"\n\n   body-type-1part = (body-type-basic / body-type-msg / body-type-text)\n                     [SP body-ext-1part]\n\n   body-type-basic = media-basic SP body-fields\n                       ; MESSAGE subtype MUST NOT be \"RFC822\" or\n                       ; \"GLOBAL\"\n\n   body-type-mpart = 1*body SP media-subtype\n                     [SP body-ext-mpart]\n                       ; MULTIPART body part\n\n   body-type-msg   = media-message SP body-fields SP envelope\n                     SP body SP body-fld-lines\n\n   body-type-text  = media-text SP body-fields SP body-fld-lines\n\n   body-fields     = body-fld-param SP body-fld-id SP body-fld-desc SP\n                     body-fld-enc SP body-fld-octets\n\n   media-message   = DQUOTE \"MESSAGE\" DQUOTE SP\n                     DQUOTE (\"RFC822\" / \"GLOBAL\") DQUOTE\n                       ; Defined in [MIME-IMT]\n\n   media-basic     = ((DQUOTE (\"APPLICATION\" / \"AUDIO\" / \"IMAGE\" /\n                     \"FONT\" / \"MESSAGE\" / \"MODEL\" / \"VIDEO\" ) DQUOTE)\n                     / string)\n                     SP media-subtype\n\n   envelope        = \"(\" env-date SP env-subject SP env-from SP\n                     env-sender SP env-reply-to SP env-to SP env-cc SP\n                     env-bcc SP env-in-reply-to SP env-message-id \")\"\n\n   body-fld-lines  = number64\n\n*/\n\n#[cfg(test)]\nmod tests {\n\n    use mail_parser::DateTime;\n    use utils::chained_bytes::SliceRange;\n\n    use crate::protocol::{Flag, ImapResponse};\n\n    use super::{\n        Address, AddressGroup, BodyPart, BodyPartExtension, BodyPartFields, DataItem, EmailAddress,\n        Envelope, FetchItem, Response, Section,\n    };\n\n    #[test]\n    fn serialize_fetch_data_item() {\n        for (item, expected_response) in [\n            (\n                super::DataItem::Envelope {\n                    envelope: Envelope {\n                        date: DateTime::from_timestamp(837570205).into(),\n                        subject: Some(\"IMAP4rev2 WG mtg summary and minutes\".into()),\n                        from: vec![Address::Single(EmailAddress {\n                            name: Some(\"Terry Gray\".into()),\n                            address: \"gray@cac.washington.edu\".into(),\n                        })],\n                        sender: vec![Address::Single(EmailAddress {\n                            name: Some(\"Terry Gray\".into()),\n                            address: \"gray@cac.washington.edu\".into(),\n                        })],\n                        reply_to: vec![Address::Single(EmailAddress {\n                            name: Some(\"Terry Gray\".into()),\n                            address: \"gray@cac.washington.edu\".into(),\n                        })],\n                        to: vec![Address::Single(EmailAddress {\n                            name: None,\n                            address: \"imap@cac.washington.edu\".into(),\n                        })],\n                        cc: vec![\n                            Address::Single(EmailAddress {\n                                name: None,\n                                address: \"minutes@CNRI.Reston.VA.US\".into(),\n                            }),\n                            Address::Single(EmailAddress {\n                                name: Some(\"John Klensin\".into()),\n                                address: \"KLENSIN@MIT.EDU\".into(),\n                            }),\n                        ],\n                        bcc: vec![],\n                        in_reply_to: None,\n                        message_id: Some(\"<B27397-0100000@cac.washington.ed>\".into()),\n                    },\n                },\n                concat!(\n                    \"ENVELOPE (\\\"Wed, 17 Jul 1996 02:23:25 +0000\\\" \",\n                    \"\\\"IMAP4rev2 WG mtg summary and minutes\\\" \",\n                    \"((\\\"Terry Gray\\\" NIL \\\"gray\\\" \\\"cac.washington.edu\\\")) \",\n                    \"((\\\"Terry Gray\\\" NIL \\\"gray\\\" \\\"cac.washington.edu\\\")) \",\n                    \"((\\\"Terry Gray\\\" NIL \\\"gray\\\" \\\"cac.washington.edu\\\")) \",\n                    \"((NIL NIL \\\"imap\\\" \\\"cac.washington.edu\\\")) \",\n                    \"((NIL NIL \\\"minutes\\\" \\\"CNRI.Reston.VA.US\\\")\",\n                    \"(\\\"John Klensin\\\" NIL \\\"KLENSIN\\\" \\\"MIT.EDU\\\")) NIL NIL \",\n                    \"\\\"<B27397-0100000@cac.washington.ed>\\\")\"\n                ),\n            ),\n            (\n                super::DataItem::Envelope {\n                    envelope: Envelope {\n                        date: DateTime::from_timestamp(837570205).into(),\n                        subject: Some(\"Group test\".into()),\n                        from: vec![Address::Single(EmailAddress {\n                            name: Some(\"Bill Foobar\".into()),\n                            address: \"foobar@example.com\".into(),\n                        })],\n                        sender: vec![],\n                        reply_to: vec![],\n                        to: vec![Address::Group(AddressGroup {\n                            name: Some(\"Friends and Family\".into()),\n                            addresses: vec![\n                                EmailAddress {\n                                    name: Some(\"John Doe\".into()),\n                                    address: \"jdoe@example.com\".into(),\n                                },\n                                EmailAddress {\n                                    name: Some(\"Jane Smith\".into()),\n                                    address: \"jane.smith@example.com\".into(),\n                                },\n                            ],\n                        })],\n                        cc: vec![],\n                        bcc: vec![],\n                        in_reply_to: None,\n                        message_id: Some(\"<B27397-0100000@cac.washington.ed>\".into()),\n                    },\n                },\n                concat!(\n                    \"ENVELOPE (\\\"Wed, 17 Jul 1996 02:23:25 +0000\\\" \",\n                    \"\\\"Group test\\\" \",\n                    \"((\\\"Bill Foobar\\\" NIL \\\"foobar\\\" \\\"example.com\\\")) \",\n                    \"((\\\"Bill Foobar\\\" NIL \\\"foobar\\\" \\\"example.com\\\")) \",\n                    \"((\\\"Bill Foobar\\\" NIL \\\"foobar\\\" \\\"example.com\\\")) \",\n                    \"((NIL NIL \\\"Friends and Family\\\" NIL)\",\n                    \"(\\\"John Doe\\\" NIL \\\"jdoe\\\" \\\"example.com\\\")\",\n                    \"(\\\"Jane Smith\\\" NIL \\\"jane.smith\\\" \\\"example.com\\\")\",\n                    \"(NIL NIL NIL NIL)) \",\n                    \"NIL NIL NIL \\\"<B27397-0100000@cac.washington.ed>\\\")\"\n                ),\n            ),\n            (\n                super::DataItem::Body {\n                    part: BodyPart::Text {\n                        fields: BodyPartFields {\n                            body_subtype: Some(\"PLAIN\".into()),\n                            body_parameters: vec![(\"CHARSET\".into(), \"US-ASCII\".into())].into(),\n                            body_id: None,\n                            body_description: None,\n                            body_encoding: Some(\"7BIT\".into()),\n                            body_size_octets: 2279,\n                        },\n                        body_size_lines: 48,\n                        body_md5: None,\n                        extension: BodyPartExtension {\n                            body_disposition: None,\n                            body_language: None,\n                            body_location: None,\n                        },\n                    },\n                },\n                \"BODY (\\\"text\\\" \\\"PLAIN\\\" (\\\"CHARSET\\\" \\\"US-ASCII\\\") NIL NIL \\\"7BIT\\\" 2279 48)\",\n            ),\n            (\n                super::DataItem::Body {\n                    part: BodyPart::Message {\n                        fields: BodyPartFields {\n                            body_subtype: Some(\"RFC822\".into()),\n                            body_parameters: None,\n                            body_id: Some(\"<abc@123>\".into()),\n                            body_description: Some(\"An attached email\".into()),\n                            body_encoding: Some(\"quoted-printable\".into()),\n                            body_size_octets: 9323,\n                        },\n                        envelope: Box::new(Envelope {\n                            date: DateTime::from_timestamp(837570205).into(),\n                            subject: Some(\"Hello world!\".into()),\n                            from: vec![Address::Single(EmailAddress {\n                                name: Some(\"Terry Gray\".into()),\n                                address: \"gray@cac.washington.edu\".into(),\n                            })],\n                            sender: vec![Address::Single(EmailAddress {\n                                name: Some(\"Terry Gray\".into()),\n                                address: \"gray@cac.washington.edu\".into(),\n                            })],\n                            reply_to: vec![Address::Single(EmailAddress {\n                                name: Some(\"Terry Gray\".into()),\n                                address: \"gray@cac.washington.edu\".into(),\n                            })],\n                            to: vec![Address::Single(EmailAddress {\n                                name: None,\n                                address: \"imap@cac.washington.edu\".into(),\n                            })],\n                            cc: vec![],\n                            bcc: vec![],\n                            in_reply_to: None,\n                            message_id: Some(\"<4234324@domain.com>\".into()),\n                        })\n                        .into(),\n                        body: Box::new(BodyPart::Text {\n                            fields: BodyPartFields {\n                                body_subtype: Some(\"HTML\".into()),\n                                body_parameters: None,\n                                body_id: None,\n                                body_description: None,\n                                body_encoding: Some(\"8BIT\".into()),\n                                body_size_octets: 4234,\n                            },\n                            body_size_lines: 431,\n                            body_md5: None,\n                            extension: BodyPartExtension {\n                                body_disposition: None,\n                                body_language: None,\n                                body_location: None,\n                            },\n                        })\n                        .into(),\n                        body_size_lines: 908,\n                        body_md5: None,\n                        extension: BodyPartExtension {\n                            body_disposition: None,\n                            body_language: None,\n                            body_location: None,\n                        },\n                    },\n                },\n                concat!(\n                    \"BODY (\\\"message\\\" \\\"RFC822\\\" NIL \\\"<abc@123>\\\" \\\"An attached email\\\" \",\n                    \"\\\"quoted-printable\\\" 9323 (\\\"Wed, 17 Jul 1996 02:23:25 +0000\\\" \",\n                    \"\\\"Hello world!\\\" \",\n                    \"((\\\"Terry Gray\\\" NIL \\\"gray\\\" \\\"cac.washington.edu\\\")) \",\n                    \"((\\\"Terry Gray\\\" NIL \\\"gray\\\" \\\"cac.washington.edu\\\")) \",\n                    \"((\\\"Terry Gray\\\" NIL \\\"gray\\\" \\\"cac.washington.edu\\\")) \",\n                    \"((NIL NIL \\\"imap\\\" \\\"cac.washington.edu\\\")) NIL NIL NIL \",\n                    \"\\\"<4234324@domain.com>\\\") (\\\"text\\\" \\\"HTML\\\" NIL NIL NIL \",\n                    \"\\\"8BIT\\\" 4234 431) 908)\"\n                ),\n            ),\n            (\n                super::DataItem::Body {\n                    part: BodyPart::Multipart {\n                        body_parts: vec![\n                            BodyPart::Text {\n                                fields: BodyPartFields {\n                                    body_subtype: Some(\"PLAIN\".into()),\n                                    body_parameters: vec![(\"CHARSET\".into(), \"US-ASCII\".into())]\n                                        .into(),\n                                    body_id: None,\n                                    body_description: None,\n                                    body_encoding: Some(\"7BIT\".into()),\n                                    body_size_octets: 1152,\n                                },\n                                body_size_lines: 23,\n                                body_md5: None,\n                                extension: BodyPartExtension {\n                                    body_disposition: None,\n                                    body_language: None,\n                                    body_location: None,\n                                },\n                            },\n                            BodyPart::Text {\n                                fields: BodyPartFields {\n                                    body_subtype: Some(\"PLAIN\".into()),\n                                    body_parameters: vec![\n                                        (\"CHARSET\".into(), \"US-ASCII\".into()),\n                                        (\"NAME\".into(), \"cc.diff\".into()),\n                                    ]\n                                    .into(),\n                                    body_id: Some(\n                                        \"<960723163407.20117h@cac.washington.edu>\".into(),\n                                    ),\n                                    body_description: Some(\"Compiler diff\".into()),\n                                    body_encoding: Some(\"BASE64\".into()),\n                                    body_size_octets: 4554,\n                                },\n                                body_size_lines: 73,\n                                body_md5: None,\n                                extension: BodyPartExtension {\n                                    body_disposition: None,\n                                    body_language: None,\n                                    body_location: None,\n                                },\n                            },\n                        ],\n                        body_subtype: \"MIXED\".into(),\n                        body_parameters: None,\n                        extension: BodyPartExtension {\n                            body_disposition: None,\n                            body_language: None,\n                            body_location: None,\n                        },\n                    },\n                },\n                concat!(\n                    \"BODY ((\\\"text\\\" \\\"PLAIN\\\" (\\\"CHARSET\\\" \\\"US-ASCII\\\") \",\n                    \"NIL NIL \\\"7BIT\\\" 1152 23)\",\n                    \"(\\\"text\\\" \\\"PLAIN\\\" (\\\"CHARSET\\\" \\\"US-ASCII\\\" \\\"NAME\\\" \\\"cc.diff\\\") \",\n                    \"\\\"<960723163407.20117h@cac.washington.edu>\\\" \\\"Compiler diff\\\" \",\n                    \"\\\"BASE64\\\" 4554 73) \\\"MIXED\\\")\",\n                ),\n            ),\n            (\n                DataItem::BodyStructure {\n                    part: BodyPart::Multipart {\n                        body_parts: vec![\n                            BodyPart::Multipart {\n                                body_parts: vec![\n                                    BodyPart::Text {\n                                        fields: BodyPartFields {\n                                            body_subtype: Some(\"PLAIN\".into()),\n                                            body_parameters: vec![(\n                                                \"CHARSET\".into(),\n                                                \"UTF-8\".into(),\n                                            )]\n                                            .into(),\n                                            body_id: Some(\"<111@domain.com>\".into()),\n                                            body_description: Some(\"Text part\".into()),\n                                            body_encoding: Some(\"7BIT\".into()),\n                                            body_size_octets: 1152,\n                                        },\n                                        body_size_lines: 23,\n                                        body_md5: Some(\"8o3456\".into()),\n                                        extension: BodyPartExtension {\n                                            body_disposition: (\"inline\".into(), vec![]).into(),\n                                            body_language: vec![\"en-US\".into()].into(),\n                                            body_location: Some(\"right here\".into()),\n                                        },\n                                    },\n                                    BodyPart::Text {\n                                        fields: BodyPartFields {\n                                            body_subtype: Some(\"HTML\".into()),\n                                            body_parameters: vec![(\n                                                \"CHARSET\".into(),\n                                                \"UTF-8\".into(),\n                                            )]\n                                            .into(),\n                                            body_id: Some(\"<54535@domain.com>\".into()),\n                                            body_description: Some(\"HTML part\".into()),\n                                            body_encoding: Some(\"8BIT\".into()),\n                                            body_size_octets: 45345,\n                                        },\n                                        body_size_lines: 994,\n                                        body_md5: Some(\"53454\".into()),\n                                        extension: BodyPartExtension {\n                                            body_disposition: (\n                                                \"attachment\".into(),\n                                                vec![(\"filename\".into(), \"myfile.txt\".into())],\n                                            )\n                                                .into(),\n                                            body_language: vec![\"en-US\".into(), \"de-DE\".into()]\n                                                .into(),\n                                            body_location: Some(\"right there\".into()),\n                                        },\n                                    },\n                                ],\n                                body_subtype: \"ALTERNATIVE\".into(),\n                                body_parameters: vec![(\n                                    \"x-param\".into(),\n                                    \"a very special parameter\".into(),\n                                )]\n                                .into(),\n                                extension: BodyPartExtension {\n                                    body_disposition: None,\n                                    body_language: vec![\"en-US\".into()].into(),\n                                    body_location: Some(\"unknown\".into()),\n                                },\n                            },\n                            BodyPart::Basic {\n                                body_type: Some(\"APPLICATION\".into()),\n                                fields: BodyPartFields {\n                                    body_subtype: Some(\"MSWORD\".into()),\n                                    body_parameters: vec![(\n                                        \"NAME\".into(),\n                                        \"chimichangas.docx\".into(),\n                                    )]\n                                    .into(),\n                                    body_id: Some(\"<4444@chimi.changa>\".into()),\n                                    body_description: Some(\"Chimichangas recipe\".into()),\n                                    body_encoding: Some(\"base64\".into()),\n                                    body_size_octets: 84723,\n                                },\n                                body_md5: Some(\"1234\".into()),\n                                extension: BodyPartExtension {\n                                    body_disposition: (\n                                        \"attachment\".into(),\n                                        vec![(\"filename\".into(), \"chimichangas.docx\".into())],\n                                    )\n                                        .into(),\n                                    body_language: vec![\"en-MX\".into()].into(),\n                                    body_location: Some(\"secret location\".into()),\n                                },\n                            },\n                        ],\n                        body_subtype: \"MIXED\".into(),\n                        body_parameters: None,\n                        extension: BodyPartExtension {\n                            body_disposition: None,\n                            body_language: None,\n                            body_location: None,\n                        },\n                    },\n                },\n                concat!(\n                    \"BODYSTRUCTURE (((\\\"text\\\" \\\"PLAIN\\\" (\\\"CHARSET\\\" \\\"UTF-8\\\") \",\n                    \"\\\"<111@domain.com>\\\" \\\"Text part\\\" \\\"7BIT\\\" 1152 23 \\\"8o3456\\\" \",\n                    \"(\\\"inline\\\" NIL) \\\"en-US\\\" \\\"right here\\\")\",\n                    \"(\\\"text\\\" \\\"HTML\\\" (\\\"CHARSET\\\" \\\"UTF-8\\\") \",\n                    \"\\\"<54535@domain.com>\\\" \\\"HTML part\\\" \\\"8BIT\\\" 45345 994 \\\"53454\\\" \",\n                    \"(\\\"attachment\\\" (\\\"filename\\\" \\\"myfile.txt\\\")) \",\n                    \"(\\\"en-US\\\" \\\"de-DE\\\") \",\n                    \"\\\"right there\\\") \\\"ALTERNATIVE\\\" (\\\"x-param\\\" \",\n                    \"\\\"a very special parameter\\\") \",\n                    \"NIL \\\"en-US\\\" \\\"unknown\\\")\",\n                    \"(\\\"APPLICATION\\\" \\\"MSWORD\\\" (\\\"NAME\\\" \\\"chimichangas.docx\\\") \",\n                    \"\\\"<4444@chimi.changa>\\\" \\\"Chimichangas recipe\\\" \\\"base64\\\"\",\n                    \" 84723 \\\"1234\\\" \",\n                    \"(\\\"attachment\\\" (\\\"filename\\\" \\\"chimichangas.docx\\\")) \\\"en-MX\\\" \",\n                    \"\\\"secret location\\\") \\\"MIXED\\\" NIL NIL NIL NIL)\",\n                ),\n            ),\n            (\n                super::DataItem::Binary {\n                    sections: vec![1, 2, 3],\n                    offset: 10.into(),\n                    contents: super::BodyContents::Bytes(b\"hello\".to_vec().into()),\n                },\n                \"BINARY[1.2.3]<10> ~{5}\\r\\nhello\",\n            ),\n            (\n                super::DataItem::Binary {\n                    sections: vec![1, 2, 3],\n                    offset: None,\n                    contents: super::BodyContents::Text(\"hello\".into()),\n                },\n                \"BINARY[1.2.3] {5}\\r\\nhello\",\n            ),\n            (\n                super::DataItem::BodySection {\n                    sections: vec![\n                        Section::Part { num: 1 },\n                        Section::Part { num: 2 },\n                        Section::Mime,\n                    ],\n                    origin_octet: 11.into(),\n                    contents: b\"howdy\"[..].into(),\n                },\n                \"BODY[1.2.MIME]<11> {5}\\r\\nhowdy\",\n            ),\n            (\n                super::DataItem::BodySection {\n                    sections: vec![Section::HeaderFields {\n                        not: true,\n                        fields: vec![\"Subject\".into(), \"x-special\".into()],\n                    }],\n                    origin_octet: None,\n                    contents: b\"howdy\"[..].into(),\n                },\n                \"BODY[HEADER.FIELDS.NOT (SUBJECT X-SPECIAL)] {5}\\r\\nhowdy\",\n            ),\n            (\n                super::DataItem::BodySection {\n                    sections: vec![Section::HeaderFields {\n                        not: false,\n                        fields: vec![\"From\".into(), \"List-Archive\".into()],\n                    }],\n                    origin_octet: None,\n                    contents: b\"howdy\"[..].into(),\n                },\n                \"BODY[HEADER.FIELDS (FROM LIST-ARCHIVE)] {5}\\r\\nhowdy\",\n            ),\n            (\n                super::DataItem::Flags {\n                    flags: vec![Flag::Seen],\n                },\n                \"FLAGS (\\\\Seen)\",\n            ),\n            (\n                super::DataItem::InternalDate { date: 482374938 },\n                \"INTERNALDATE \\\"15-Apr-1985 01:02:18 +0000\\\"\",\n            ),\n        ] {\n            let mut buf = Vec::with_capacity(100);\n\n            item.serialize(&mut buf);\n\n            assert_eq!(String::from_utf8(buf).unwrap(), expected_response);\n        }\n    }\n\n    #[test]\n    fn serialize_fetch() {\n        assert_eq!(\n            String::from_utf8(\n                Response {\n                    is_uid: false,\n                    items: vec![FetchItem {\n                        id: 123,\n                        items: vec![\n                            super::DataItem::Flags {\n                                flags: vec![Flag::Deleted, Flag::Flagged],\n                            },\n                            super::DataItem::Uid { uid: 983 },\n                            super::DataItem::Rfc822Size { size: 443 },\n                            super::DataItem::Rfc822Text {\n                                contents: SliceRange::Single(&b\"hi\"[..]),\n                            },\n                            super::DataItem::Rfc822Header {\n                                contents: SliceRange::Single(&b\"header\"[..]),\n                            },\n                        ],\n                    }],\n                }\n                .serialize(),\n            )\n            .unwrap(),\n            concat!(\n                \"* 123 FETCH (FLAGS (\\\\Deleted \\\\Flagged) \",\n                \"UID 983 \",\n                \"RFC822.SIZE 443 \",\n                \"RFC822.TEXT {2}\\r\\nhi \",\n                \"RFC822.HEADER {6}\\r\\nheader)\\r\\n\",\n            )\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/list.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::utf7::utf7_encode;\n\nuse super::{\n    ImapResponse, quoted_string,\n    status::{Status, StatusItem},\n};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Arguments {\n    Basic {\n        tag: String,\n        reference_name: String,\n        mailbox_name: String,\n    },\n    Extended {\n        tag: String,\n        reference_name: String,\n        mailbox_name: Vec<String>,\n        selection_options: Vec<SelectionOption>,\n        return_options: Vec<ReturnOption>,\n    },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Response {\n    pub is_rev2: bool,\n    pub is_utf8: bool,\n    pub is_lsub: bool,\n    pub list_items: Vec<ListItem>,\n    pub status_items: Vec<StatusItem>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum SelectionOption {\n    Subscribed,\n    Remote,\n    RecursiveMatch,\n    SpecialUse,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ReturnOption {\n    Subscribed,\n    Children,\n    Status(Vec<Status>),\n    SpecialUse,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum Attribute {\n    NoInferiors,\n    NoSelect,\n    Marked,\n    Unmarked,\n    NonExistent,\n    HasChildren,\n    HasNoChildren,\n    Subscribed,\n    Remote,\n    All,\n    Archive,\n    Drafts,\n    Flagged,\n    Junk,\n    Sent,\n    Trash,\n    Important,\n    Memos,\n    Scheduled,\n    Snoozed,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ChildInfo {\n    Subscribed,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Tag {\n    ChildInfo(Vec<ChildInfo>),\n    OldName(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct ListItem {\n    pub mailbox_name: String,\n    pub attributes: Vec<Attribute>,\n    pub tags: Vec<Tag>,\n}\n\nimpl Arguments {\n    pub fn is_separator_query(&self) -> bool {\n        match self {\n            Arguments::Basic {\n                mailbox_name,\n                reference_name,\n                ..\n            } => mailbox_name.is_empty() && reference_name.is_empty(),\n            Arguments::Extended {\n                mailbox_name,\n                reference_name,\n                ..\n            } => mailbox_name.is_empty() && reference_name.is_empty(),\n        }\n    }\n\n    pub fn unwrap_tag(self) -> String {\n        match self {\n            Arguments::Basic { tag, .. } => tag,\n            Arguments::Extended { tag, .. } => tag,\n        }\n    }\n}\n\nimpl Attribute {\n    pub fn is_rev1(&self) -> bool {\n        matches!(\n            self,\n            Attribute::NoInferiors | Attribute::NoSelect | Attribute::Marked | Attribute::Unmarked\n        )\n    }\n\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(match self {\n            Attribute::NoInferiors => b\"\\\\NoInferiors\",\n            Attribute::NoSelect => b\"\\\\NoSelect\",\n            Attribute::Marked => b\"\\\\Marked\",\n            Attribute::Unmarked => b\"\\\\Unmarked\",\n            Attribute::NonExistent => b\"\\\\NonExistent\",\n            Attribute::HasChildren => b\"\\\\HasChildren\",\n            Attribute::HasNoChildren => b\"\\\\HasNoChildren\",\n            Attribute::Subscribed => b\"\\\\Subscribed\",\n            Attribute::Remote => b\"\\\\Remote\",\n            Attribute::All => b\"\\\\All\",\n            Attribute::Archive => b\"\\\\Archive\",\n            Attribute::Drafts => b\"\\\\Drafts\",\n            Attribute::Flagged => b\"\\\\Flagged\",\n            Attribute::Junk => b\"\\\\Junk\",\n            Attribute::Sent => b\"\\\\Sent\",\n            Attribute::Trash => b\"\\\\Trash\",\n            Attribute::Important => b\"\\\\Important\",\n            Attribute::Memos => b\"\\\\Memos\",\n            Attribute::Scheduled => b\"\\\\Scheduled\",\n            Attribute::Snoozed => b\"\\\\Snoozed\",\n        });\n    }\n}\n\nimpl TryFrom<&str> for Attribute {\n    type Error = ();\n\n    fn try_from(value: &str) -> Result<Self, Self::Error> {\n        hashify::tiny_map!(value.as_bytes(),\n            \"archive\" => Attribute::Archive,\n            \"drafts\" => Attribute::Drafts,\n            \"junk\" => Attribute::Junk,\n            \"sent\" => Attribute::Sent,\n            \"trash\" => Attribute::Trash,\n            \"important\" => Attribute::Important,\n            \"memos\" => Attribute::Memos,\n            \"scheduled\" => Attribute::Scheduled,\n            \"snoozed\" => Attribute::Snoozed,\n        )\n        .ok_or(())\n    }\n}\n\nimpl ChildInfo {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.push(b'\\\"');\n        buf.extend_from_slice(match self {\n            ChildInfo::Subscribed => b\"SUBSCRIBED\",\n        });\n        buf.push(b'\\\"');\n    }\n}\n\nimpl Tag {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        match self {\n            Tag::ChildInfo(child_info) => {\n                buf.extend_from_slice(b\"\\\"CHILDINFO\\\" (\");\n                for (pos, child_info) in child_info.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b' ');\n                    }\n                    child_info.serialize(buf);\n                }\n                buf.push(b')');\n            }\n            Tag::OldName(old_name) => {\n                buf.extend_from_slice(b\"\\\"OLDNAME\\\" (\");\n                quoted_string(buf, old_name);\n                buf.push(b')');\n            }\n        }\n    }\n}\n\nimpl ListItem {\n    pub fn new(name: impl Into<String>) -> Self {\n        ListItem {\n            mailbox_name: name.into(),\n            attributes: Vec::new(),\n            tags: Vec::new(),\n        }\n    }\n\n    pub fn serialize(&self, buf: &mut Vec<u8>, is_rev2: bool, is_utf8: bool, is_lsub: bool) {\n        let normalized_mailbox_name = utf7_encode(&self.mailbox_name);\n        if !is_lsub {\n            buf.extend_from_slice(b\"* LIST (\");\n        } else {\n            buf.extend_from_slice(b\"* LSUB (\");\n        }\n        for (pos, attr) in self.attributes.iter().enumerate() {\n            if pos > 0 {\n                buf.push(b' ');\n            }\n            attr.serialize(buf);\n        }\n        buf.extend_from_slice(b\") \\\"/\\\" \");\n        let mut extra_tags = Vec::new();\n\n        if normalized_mailbox_name != self.mailbox_name {\n            if is_rev2 || is_utf8 {\n                quoted_string(buf, &self.mailbox_name);\n                if is_rev2 {\n                    extra_tags.push(Tag::OldName(normalized_mailbox_name));\n                }\n            } else {\n                quoted_string(buf, &normalized_mailbox_name);\n            }\n        } else {\n            quoted_string(buf, &self.mailbox_name);\n        }\n\n        if !extra_tags.is_empty() || !self.tags.is_empty() {\n            buf.extend_from_slice(b\" (\");\n            for (pos, tag) in extra_tags.iter().chain(self.tags.iter()).enumerate() {\n                if pos > 0 {\n                    buf.push(b' ');\n                }\n                tag.serialize(buf);\n            }\n            buf.extend_from_slice(b\")\\r\\n\");\n        } else {\n            buf.extend_from_slice(b\"\\r\\n\");\n        }\n    }\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(100);\n\n        match (self.list_items.is_empty(), self.status_items.is_empty()) {\n            (false, false) => {\n                for (list_item, status_item) in self.list_items.iter().zip(self.status_items.iter())\n                {\n                    list_item.serialize(&mut buf, self.is_rev2, self.is_utf8, self.is_lsub);\n                    status_item.serialize(&mut buf, self.is_rev2);\n                }\n            }\n            (false, true) => {\n                for list_item in &self.list_items {\n                    list_item.serialize(&mut buf, self.is_rev2, self.is_utf8, self.is_lsub);\n                }\n            }\n            (true, false) => {\n                for status_item in &self.status_items {\n                    status_item.serialize(&mut buf, self.is_rev2);\n                }\n            }\n            _ => (),\n        }\n\n        buf\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use crate::protocol::{\n        ImapResponse,\n        status::{Status, StatusItem, StatusItemType},\n    };\n\n    use super::{Attribute, ChildInfo, ListItem, Tag};\n\n    #[test]\n    fn serialize_list_item() {\n        for (response, expected_v2, expected_v1) in [\n            (\n                super::ListItem {\n                    mailbox_name: \"\".into(),\n                    attributes: vec![],\n                    tags: vec![],\n                },\n                \"* LIST () \\\"/\\\" \\\"\\\"\\r\\n\",\n                \"* LIST () \\\"/\\\" \\\"\\\"\\r\\n\",\n            ),\n            (\n                super::ListItem {\n                    mailbox_name: \"中國書店\".into(),\n                    attributes: vec![Attribute::NoInferiors, Attribute::Drafts],\n                    tags: vec![],\n                },\n                concat!(\n                    \"* LIST (\\\\NoInferiors \\\\Drafts) \\\"/\\\" \\\"中國書店\\\" \",\n                    \"(\\\"OLDNAME\\\" (\\\"&Ti1XC2b4Xpc-\\\"))\\r\\n\"\n                ),\n                \"* LIST (\\\\NoInferiors \\\\Drafts) \\\"/\\\" \\\"&Ti1XC2b4Xpc-\\\"\\r\\n\",\n            ),\n            (\n                super::ListItem {\n                    mailbox_name: \"☺\".into(),\n                    attributes: vec![Attribute::Subscribed, Attribute::Remote],\n                    tags: vec![Tag::ChildInfo(vec![ChildInfo::Subscribed])],\n                },\n                concat!(\n                    \"* LIST (\\\\Subscribed \\\\Remote) \\\"/\\\" \\\"☺\\\" \",\n                    \"(\\\"OLDNAME\\\" (\\\"&Jjo-\\\") \\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\"))\\r\\n\"\n                ),\n                concat!(\n                    \"* LIST (\\\\Subscribed \\\\Remote) \\\"/\\\" \\\"&Jjo-\\\" \",\n                    \"(\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\"))\\r\\n\"\n                ),\n            ),\n            (\n                super::ListItem {\n                    mailbox_name: \"foo\".into(),\n                    attributes: vec![Attribute::HasNoChildren],\n                    tags: vec![Tag::ChildInfo(vec![ChildInfo::Subscribed])],\n                },\n                \"* LIST (\\\\HasNoChildren) \\\"/\\\" \\\"foo\\\" (\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\"))\\r\\n\",\n                \"* LIST (\\\\HasNoChildren) \\\"/\\\" \\\"foo\\\" (\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\"))\\r\\n\",\n            ),\n        ] {\n            let mut buf_1 = Vec::with_capacity(100);\n            let mut buf_2 = Vec::with_capacity(100);\n\n            response.serialize(&mut buf_1, false, false, false);\n            response.serialize(&mut buf_2, true, true, false);\n\n            let response_v1 = String::from_utf8(buf_1).unwrap();\n            let response_v2 = String::from_utf8(buf_2).unwrap();\n\n            assert_eq!(response_v2, expected_v2);\n            assert_eq!(response_v1, expected_v1);\n        }\n    }\n\n    #[test]\n    fn serialize_list() {\n        let mut response = super::Response {\n            list_items: vec![\n                ListItem {\n                    mailbox_name: \"INBOX\".into(),\n                    attributes: vec![Attribute::Subscribed],\n                    tags: vec![],\n                },\n                ListItem {\n                    mailbox_name: \"foo\".into(),\n                    attributes: vec![],\n                    tags: vec![Tag::ChildInfo(vec![ChildInfo::Subscribed])],\n                },\n            ],\n            status_items: vec![\n                StatusItem {\n                    mailbox_name: \"INBOX\".into(),\n                    items: vec![(Status::Messages, StatusItemType::Number(17))],\n                },\n                StatusItem {\n                    mailbox_name: \"foo\".into(),\n                    items: vec![\n                        (Status::Messages, StatusItemType::Number(30)),\n                        (Status::Unseen, StatusItemType::Number(29)),\n                    ],\n                },\n            ],\n            is_lsub: false,\n            is_rev2: true,\n            is_utf8: true,\n        };\n        let expected_v2 = concat!(\n            \"* LIST (\\\\Subscribed) \\\"/\\\" \\\"INBOX\\\"\\r\\n\",\n            \"* STATUS \\\"INBOX\\\" (MESSAGES 17)\\r\\n\",\n            \"* LIST () \\\"/\\\" \\\"foo\\\" (\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\"))\\r\\n\",\n            \"* STATUS \\\"foo\\\" (MESSAGES 30 UNSEEN 29)\\r\\n\",\n        );\n        let expected_v1 = concat!(\n            \"* LSUB (\\\\Subscribed) \\\"/\\\" \\\"INBOX\\\"\\r\\n\",\n            \"* LSUB () \\\"/\\\" \\\"foo\\\" (\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\"))\\r\\n\",\n        );\n\n        let response_v2 = String::from_utf8(response.clone().serialize()).unwrap();\n        response.is_rev2 = false;\n        response.is_utf8 = false;\n        response.is_lsub = true;\n        response.status_items.clear();\n        let response_v1 = String::from_utf8(response.serialize()).unwrap();\n\n        assert_eq!(response_v2, expected_v2);\n        assert_eq!(response_v1, expected_v1);\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/login.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub username: String,\n    pub password: String,\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{Command, ResponseCode, ResponseType, StatusResponse};\nuse ahash::AHashSet;\nuse chrono::{DateTime, Utc};\nuse compact_str::CompactString;\nuse std::{cmp::Ordering, fmt::Display};\nuse types::keyword::{ArchivedKeyword, Keyword};\nuse utils::chained_bytes::SliceRange;\n\npub mod acl;\npub mod append;\npub mod authenticate;\npub mod capability;\npub mod copy_move;\npub mod create;\npub mod delete;\npub mod enable;\npub mod expunge;\npub mod fetch;\npub mod list;\npub mod login;\npub mod namespace;\npub mod quota;\npub mod rename;\npub mod search;\npub mod select;\npub mod status;\npub mod store;\npub mod subscribe;\npub mod thread;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum ProtocolVersion {\n    Rev1,\n    Rev2,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Sequence {\n    Number {\n        value: u32,\n    },\n    Range {\n        start: Option<u32>,\n        end: Option<u32>,\n    },\n    SavedSearch,\n    List {\n        items: Vec<Sequence>,\n    },\n}\n\nimpl Sequence {\n    pub fn number(value: u32) -> Sequence {\n        Sequence::Number { value }\n    }\n\n    pub fn range(start: Option<u32>, end: Option<u32>) -> Sequence {\n        Sequence::Range { start, end }\n    }\n\n    pub fn contains(&self, value: u32, max_value: u32) -> bool {\n        match self {\n            Sequence::Number { value: number } => *number == value,\n            Sequence::Range { start, end } => match (start, end) {\n                (Some(start), Some(end)) => {\n                    value >= *start && value <= *end || value >= *end && value <= *start\n                }\n                (Some(range), None) | (None, Some(range)) => {\n                    value >= *range && value <= max_value || value >= max_value && value <= *range\n                }\n                (None, None) => value == max_value,\n            },\n            Sequence::List { items } => {\n                for item in items {\n                    if item.contains(value, max_value) {\n                        return true;\n                    }\n                }\n                false\n            }\n            Sequence::SavedSearch => false,\n        }\n    }\n\n    pub fn is_saved_search(&self) -> bool {\n        match self {\n            Sequence::SavedSearch => true,\n            Sequence::List { items } => items.iter().any(|s| s.is_saved_search()),\n            _ => false,\n        }\n    }\n\n    pub fn expand(&self, max_value: u32) -> AHashSet<u32> {\n        match self {\n            Sequence::Number { value } => AHashSet::from_iter([*value]),\n            Sequence::List { items } => {\n                let mut result = AHashSet::with_capacity(items.len());\n                for item in items {\n                    match item {\n                        Sequence::Number { value } => {\n                            result.insert(*value);\n                        }\n                        Sequence::Range { start, end } => {\n                            let start = start.unwrap_or(max_value);\n                            let end = end.unwrap_or(max_value);\n                            match start.cmp(&end) {\n                                Ordering::Equal => {\n                                    result.insert(start);\n                                }\n                                Ordering::Less => {\n                                    result.extend(start..=end);\n                                }\n                                Ordering::Greater => {\n                                    result.extend(end..=start);\n                                }\n                            }\n                        }\n                        _ => (),\n                    }\n                }\n                result\n            }\n            Sequence::Range { start, end } => {\n                let mut result = AHashSet::new();\n                let start = start.unwrap_or(max_value);\n                let end = end.unwrap_or(max_value);\n                match start.cmp(&end) {\n                    Ordering::Equal => {\n                        result.insert(start);\n                    }\n                    Ordering::Less => {\n                        result.extend(start..=end);\n                    }\n                    Ordering::Greater => {\n                        result.extend(end..=start);\n                    }\n                }\n                result\n            }\n            _ => AHashSet::new(),\n        }\n    }\n}\n\npub trait ImapResponse {\n    fn serialize(self) -> Vec<u8>;\n}\n\npub fn quoted_string(buf: &mut Vec<u8>, text: &str) {\n    buf.push(b'\"');\n    for &c in text.as_bytes() {\n        if c == b'\\\\' || c == b'\"' {\n            buf.push(b'\\\\');\n        }\n        buf.push(c);\n    }\n    buf.push(b'\"');\n}\n\npub fn quoted_or_literal_string(buf: &mut Vec<u8>, text: &str) {\n    if text\n        .as_bytes()\n        .iter()\n        .any(|ch| [b'\\\\', b'\"', b'\\r', b'\\n'].contains(ch))\n    {\n        literal_string(buf, text.as_bytes())\n    } else {\n        buf.push(b'\"');\n        buf.extend_from_slice(text.as_bytes());\n        buf.push(b'\"');\n    }\n}\npub fn quoted_or_literal_string_or_nil(buf: &mut Vec<u8>, text: Option<&str>) {\n    if let Some(text) = text {\n        quoted_or_literal_string(buf, text);\n    } else {\n        buf.extend_from_slice(b\"NIL\");\n    }\n}\n\npub fn quoted_string_or_nil(buf: &mut Vec<u8>, text: Option<&str>) {\n    if let Some(text) = text {\n        quoted_string(buf, text);\n    } else {\n        buf.extend_from_slice(b\"NIL\");\n    }\n}\n\npub fn literal_string(buf: &mut Vec<u8>, text: &[u8]) {\n    buf.push(b'{');\n    buf.extend_from_slice(text.len().to_string().as_bytes());\n    buf.extend_from_slice(b\"}\\r\\n\");\n    buf.extend_from_slice(text);\n}\n\npub fn literal_string_slice(buf: &mut Vec<u8>, text: &SliceRange<'_>) {\n    buf.push(b'{');\n    buf.extend_from_slice(text.len().to_string().as_bytes());\n    buf.extend_from_slice(b\"}\\r\\n\");\n    buf.extend(*text);\n}\n\npub fn quoted_timestamp(buf: &mut Vec<u8>, timestamp: i64) {\n    buf.push(b'\"');\n    buf.extend_from_slice(\n        DateTime::<Utc>::from_timestamp(timestamp, 0)\n            .unwrap_or_default()\n            .format(\"%d-%b-%Y %H:%M:%S %z\")\n            .to_string()\n            .as_bytes(),\n    );\n    buf.push(b'\"');\n}\n\npub fn quoted_rfc2822(buf: &mut Vec<u8>, timestamp: &mail_parser::DateTime) {\n    buf.push(b'\"');\n    buf.extend_from_slice(timestamp.to_rfc822().as_bytes());\n    buf.push(b'\"');\n}\n\npub fn quoted_rfc2822_or_nil(buf: &mut Vec<u8>, timestamp: &Option<mail_parser::DateTime>) {\n    if let Some(timestamp) = timestamp {\n        quoted_rfc2822(buf, timestamp);\n    } else {\n        buf.extend_from_slice(b\"NIL\");\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Flag {\n    Seen,\n    Draft,\n    Flagged,\n    Answered,\n    Recent,\n    Important,\n    Phishing,\n    Junk,\n    NotJunk,\n    Deleted,\n    Forwarded,\n    MDNSent,\n    Autosent,\n    CanUnsubscribe,\n    Followed,\n    HasAttachment,\n    HasMemo,\n    HasNoAttachment,\n    Imported,\n    IsTrusted,\n    MailFlagBit0,\n    MailFlagBit1,\n    MailFlagBit2,\n    MaskedEmail,\n    Memo,\n    Muted,\n    New,\n    Notify,\n    Unsubscribed,\n    Keyword(Box<str>),\n}\n\nimpl Flag {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(match self {\n            Flag::Seen => b\"\\\\Seen\",\n            Flag::Draft => b\"\\\\Draft\",\n            Flag::Flagged => b\"\\\\Flagged\",\n            Flag::Answered => b\"\\\\Answered\",\n            Flag::Recent => b\"\\\\Recent\",\n            Flag::Important => b\"\\\\Important\",\n            Flag::Phishing => b\"$Phishing\",\n            Flag::Junk => b\"$Junk\",\n            Flag::NotJunk => b\"$NotJunk\",\n            Flag::Deleted => b\"\\\\Deleted\",\n            Flag::Forwarded => b\"$Forwarded\",\n            Flag::MDNSent => b\"$MDNSent\",\n            Flag::Autosent => b\"$autosent\",\n            Flag::CanUnsubscribe => b\"$canunsubscribe\",\n            Flag::Followed => b\"$followed\",\n            Flag::HasAttachment => b\"$hasattachment\",\n            Flag::HasMemo => b\"$hasmemo\",\n            Flag::HasNoAttachment => b\"$hasnoattachment\",\n            Flag::Imported => b\"$imported\",\n            Flag::IsTrusted => b\"$istrusted\",\n            Flag::MailFlagBit0 => b\"$MailFlagBit0\",\n            Flag::MailFlagBit1 => b\"$MailFlagBit1\",\n            Flag::MailFlagBit2 => b\"$MailFlagBit2\",\n            Flag::MaskedEmail => b\"$maskedemail\",\n            Flag::Memo => b\"$memo\",\n            Flag::Muted => b\"$muted\",\n            Flag::New => b\"$new\",\n            Flag::Notify => b\"$notify\",\n            Flag::Unsubscribed => b\"$unsubscribed\",\n            Flag::Keyword(keyword) => keyword.as_bytes(),\n        });\n    }\n}\n\nimpl From<Keyword> for Flag {\n    fn from(value: Keyword) -> Self {\n        match value {\n            Keyword::Seen => Flag::Seen,\n            Keyword::Draft => Flag::Draft,\n            Keyword::Flagged => Flag::Flagged,\n            Keyword::Answered => Flag::Answered,\n            Keyword::Recent => Flag::Recent,\n            Keyword::Important => Flag::Important,\n            Keyword::Phishing => Flag::Phishing,\n            Keyword::Junk => Flag::Junk,\n            Keyword::NotJunk => Flag::NotJunk,\n            Keyword::Deleted => Flag::Deleted,\n            Keyword::Forwarded => Flag::Forwarded,\n            Keyword::MdnSent => Flag::MDNSent,\n            Keyword::Autosent => Flag::Autosent,\n            Keyword::CanUnsubscribe => Flag::CanUnsubscribe,\n            Keyword::Followed => Flag::Followed,\n            Keyword::HasAttachment => Flag::HasAttachment,\n            Keyword::HasMemo => Flag::HasMemo,\n            Keyword::HasNoAttachment => Flag::HasNoAttachment,\n            Keyword::Imported => Flag::Imported,\n            Keyword::IsTrusted => Flag::IsTrusted,\n            Keyword::MailFlagBit0 => Flag::MailFlagBit0,\n            Keyword::MailFlagBit1 => Flag::MailFlagBit1,\n            Keyword::MailFlagBit2 => Flag::MailFlagBit2,\n            Keyword::MaskedEmail => Flag::MaskedEmail,\n            Keyword::Memo => Flag::Memo,\n            Keyword::Muted => Flag::Muted,\n            Keyword::New => Flag::New,\n            Keyword::Notify => Flag::Notify,\n            Keyword::Unsubscribed => Flag::Unsubscribed,\n            Keyword::Other(value) => Flag::Keyword(value),\n        }\n    }\n}\n\nimpl From<&ArchivedKeyword> for Flag {\n    fn from(value: &ArchivedKeyword) -> Self {\n        match value {\n            ArchivedKeyword::Seen => Flag::Seen,\n            ArchivedKeyword::Draft => Flag::Draft,\n            ArchivedKeyword::Flagged => Flag::Flagged,\n            ArchivedKeyword::Answered => Flag::Answered,\n            ArchivedKeyword::Recent => Flag::Recent,\n            ArchivedKeyword::Important => Flag::Important,\n            ArchivedKeyword::Phishing => Flag::Phishing,\n            ArchivedKeyword::Junk => Flag::Junk,\n            ArchivedKeyword::NotJunk => Flag::NotJunk,\n            ArchivedKeyword::Deleted => Flag::Deleted,\n            ArchivedKeyword::Forwarded => Flag::Forwarded,\n            ArchivedKeyword::MdnSent => Flag::MDNSent,\n            ArchivedKeyword::Autosent => Flag::Autosent,\n            ArchivedKeyword::CanUnsubscribe => Flag::CanUnsubscribe,\n            ArchivedKeyword::Followed => Flag::Followed,\n            ArchivedKeyword::HasAttachment => Flag::HasAttachment,\n            ArchivedKeyword::HasMemo => Flag::HasMemo,\n            ArchivedKeyword::HasNoAttachment => Flag::HasNoAttachment,\n            ArchivedKeyword::Imported => Flag::Imported,\n            ArchivedKeyword::IsTrusted => Flag::IsTrusted,\n            ArchivedKeyword::MailFlagBit0 => Flag::MailFlagBit0,\n            ArchivedKeyword::MailFlagBit1 => Flag::MailFlagBit1,\n            ArchivedKeyword::MailFlagBit2 => Flag::MailFlagBit2,\n            ArchivedKeyword::MaskedEmail => Flag::MaskedEmail,\n            ArchivedKeyword::Memo => Flag::Memo,\n            ArchivedKeyword::Muted => Flag::Muted,\n            ArchivedKeyword::New => Flag::New,\n            ArchivedKeyword::Notify => Flag::Notify,\n            ArchivedKeyword::Unsubscribed => Flag::Unsubscribed,\n            ArchivedKeyword::Other(value) => Flag::Keyword(value.as_ref().into()),\n        }\n    }\n}\n\nimpl From<Flag> for Keyword {\n    fn from(value: Flag) -> Self {\n        match value {\n            Flag::Seen => Keyword::Seen,\n            Flag::Draft => Keyword::Draft,\n            Flag::Flagged => Keyword::Flagged,\n            Flag::Answered => Keyword::Answered,\n            Flag::Recent => Keyword::Recent,\n            Flag::Important => Keyword::Important,\n            Flag::Phishing => Keyword::Phishing,\n            Flag::Junk => Keyword::Junk,\n            Flag::NotJunk => Keyword::NotJunk,\n            Flag::Deleted => Keyword::Deleted,\n            Flag::Forwarded => Keyword::Forwarded,\n            Flag::MDNSent => Keyword::MdnSent,\n            Flag::Autosent => Keyword::Autosent,\n            Flag::CanUnsubscribe => Keyword::CanUnsubscribe,\n            Flag::Followed => Keyword::Followed,\n            Flag::HasAttachment => Keyword::HasAttachment,\n            Flag::HasMemo => Keyword::HasMemo,\n            Flag::HasNoAttachment => Keyword::HasNoAttachment,\n            Flag::Imported => Keyword::Imported,\n            Flag::IsTrusted => Keyword::IsTrusted,\n            Flag::MailFlagBit0 => Keyword::MailFlagBit0,\n            Flag::MailFlagBit1 => Keyword::MailFlagBit1,\n            Flag::MailFlagBit2 => Keyword::MailFlagBit2,\n            Flag::MaskedEmail => Keyword::MaskedEmail,\n            Flag::Memo => Keyword::Memo,\n            Flag::Muted => Keyword::Muted,\n            Flag::New => Keyword::New,\n            Flag::Notify => Keyword::Notify,\n            Flag::Unsubscribed => Keyword::Unsubscribed,\n            Flag::Keyword(value) => Keyword::from_boxed_other(value),\n        }\n    }\n}\n\nimpl ResponseCode {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(match self {\n            ResponseCode::Alert => b\"ALERT\",\n            ResponseCode::AlreadyExists => b\"ALREADYEXISTS\",\n            ResponseCode::AppendUid { uid_validity, uids } => {\n                buf.extend_from_slice(b\"APPENDUID \");\n                buf.extend_from_slice(uid_validity.to_string().as_bytes());\n                buf.push(b' ');\n                serialize_sequence(buf, uids);\n                return;\n            }\n            ResponseCode::AuthenticationFailed => b\"AUTHENTICATIONFAILED\",\n            ResponseCode::AuthorizationFailed => b\"AUTHORIZATIONFAILED\",\n            ResponseCode::BadCharset => b\"BADCHARSET\",\n            ResponseCode::Cannot => b\"CANNOT\",\n            ResponseCode::Capability { capabilities } => {\n                buf.extend_from_slice(b\"CAPABILITY\");\n                for capability in capabilities {\n                    buf.push(b' ');\n                    capability.serialize(buf);\n                }\n                return;\n            }\n            ResponseCode::ClientBug => b\"CLIENTBUG\",\n            ResponseCode::Closed => b\"CLOSED\",\n            ResponseCode::ContactAdmin => b\"CONTACTADMIN\",\n            ResponseCode::CopyUid {\n                uid_validity,\n                src_uids,\n                dest_uids,\n            } => {\n                buf.extend_from_slice(b\"COPYUID \");\n                buf.extend_from_slice(uid_validity.to_string().as_bytes());\n                buf.push(b' ');\n                serialize_sequence(buf, src_uids);\n                buf.push(b' ');\n                serialize_sequence(buf, dest_uids);\n                return;\n            }\n            ResponseCode::Corruption => b\"CORRUPTION\",\n            ResponseCode::Expired => b\"EXPIRED\",\n            ResponseCode::ExpungeIssued => b\"EXPUNGEISSUED\",\n            ResponseCode::HasChildren => b\"HASCHILDREN\",\n            ResponseCode::InUse => b\"INUSE\",\n            ResponseCode::Limit => b\"LIMIT\",\n            ResponseCode::NonExistent => b\"NONEXISTENT\",\n            ResponseCode::NoPerm => b\"NOPERM\",\n            ResponseCode::OverQuota => b\"OVERQUOTA\",\n            ResponseCode::Parse => b\"PARSE\",\n            ResponseCode::PermanentFlags => b\"PERMANENTFLAGS\",\n            ResponseCode::PrivacyRequired => b\"PRIVACYREQUIRED\",\n            ResponseCode::ReadOnly => b\"READ-ONLY\",\n            ResponseCode::ReadWrite => b\"READ-WRITE\",\n            ResponseCode::ServerBug => b\"SERVERBUG\",\n            ResponseCode::TryCreate => b\"TRYCREATE\",\n            ResponseCode::UidNext => b\"UIDNEXT\",\n            ResponseCode::UidNotSticky => b\"UIDNOTSTICKY\",\n            ResponseCode::UidValidity => b\"UIDVALIDITY\",\n            ResponseCode::Unavailable => b\"UNAVAILABLE\",\n            ResponseCode::UnknownCte => b\"UNKNOWN-CTE\",\n            ResponseCode::Modified { ids } => {\n                buf.extend_from_slice(b\"MODIFIED \");\n                serialize_sequence(buf, ids);\n                return;\n            }\n            ResponseCode::MailboxId { mailbox_id } => {\n                buf.extend_from_slice(b\"MAILBOXID (\");\n                buf.extend_from_slice(mailbox_id.as_bytes());\n                buf.push(b')');\n                return;\n            }\n            ResponseCode::HighestModseq { modseq } => {\n                buf.extend_from_slice(b\"HIGHESTMODSEQ \");\n                buf.extend_from_slice(modseq.to_string().as_bytes());\n                return;\n            }\n            ResponseCode::UseAttr => b\"USEATTR\",\n        });\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        // Only returns the name without arguments\n        match self {\n            ResponseCode::Alert => \"ALERT\",\n            ResponseCode::AlreadyExists => \"ALREADYEXISTS\",\n            ResponseCode::AppendUid { .. } => \"APPENDUID\",\n            ResponseCode::AuthenticationFailed => \"AUTHENTICATIONFAILED\",\n            ResponseCode::AuthorizationFailed => \"AUTHORIZATIONFAILED\",\n            ResponseCode::BadCharset => \"BADCHARSET\",\n            ResponseCode::Cannot => \"CANNOT\",\n            ResponseCode::Capability { .. } => \"CAPABILITY\",\n            ResponseCode::ClientBug => \"CLIENTBUG\",\n            ResponseCode::Closed => \"CLOSED\",\n            ResponseCode::ContactAdmin => \"CONTACTADMIN\",\n            ResponseCode::CopyUid { .. } => \"COPYUID\",\n            ResponseCode::Corruption => \"CORRUPTION\",\n            ResponseCode::Expired => \"EXPIRED\",\n            ResponseCode::ExpungeIssued => \"EXPUNGEISSUED\",\n            ResponseCode::HasChildren => \"HASCHILDREN\",\n            ResponseCode::InUse => \"INUSE\",\n            ResponseCode::Limit => \"LIMIT\",\n            ResponseCode::NonExistent => \"NONEXISTENT\",\n            ResponseCode::NoPerm => \"NOPERM\",\n            ResponseCode::OverQuota => \"OVERQUOTA\",\n            ResponseCode::Parse => \"PARSE\",\n            ResponseCode::PermanentFlags => \"PERMANENTFLAGS\",\n            ResponseCode::PrivacyRequired => \"PRIVACYREQUIRED\",\n            ResponseCode::ReadOnly => \"READ-ONLY\",\n            ResponseCode::ReadWrite => \"READ-WRITE\",\n            ResponseCode::ServerBug => \"SERVERBUG\",\n            ResponseCode::TryCreate => \"TRYCREATE\",\n            ResponseCode::UidNext => \"UIDNEXT\",\n            ResponseCode::UidNotSticky => \"UIDNOTSTICKY\",\n            ResponseCode::UidValidity => \"UIDVALIDITY\",\n            ResponseCode::Unavailable => \"UNAVAILABLE\",\n            ResponseCode::UnknownCte => \"UNKNOWN-CTE\",\n            ResponseCode::Modified { .. } => \"MODIFIED\",\n            ResponseCode::MailboxId { .. } => \"MAILBOXID\",\n            ResponseCode::HighestModseq { .. } => \"HIGHESTMODSEQ\",\n            ResponseCode::UseAttr => \"USEATTR\",\n        }\n    }\n}\n\nimpl ResponseType {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(self.as_str().as_bytes());\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            ResponseType::Ok => \"OK\",\n            ResponseType::No => \"NO\",\n            ResponseType::Bad => \"BAD\",\n            ResponseType::PreAuth => \"PREAUTH\",\n            ResponseType::Bye => \"BYE\",\n        }\n    }\n}\n\nimpl From<ResponseCode> for trc::Value {\n    fn from(value: ResponseCode) -> Self {\n        trc::Value::String(CompactString::const_new(value.as_str()))\n    }\n}\n\nimpl From<ResponseType> for trc::Value {\n    fn from(value: ResponseType) -> Self {\n        trc::Value::String(CompactString::const_new(value.as_str()))\n    }\n}\n\nimpl StatusResponse {\n    pub fn serialize(self, mut buf: Vec<u8>) -> Vec<u8> {\n        if let Some(tag) = &self.tag {\n            buf.extend_from_slice(tag.as_bytes());\n        } else {\n            buf.push(b'*');\n        }\n        buf.push(b' ');\n        self.rtype.serialize(&mut buf);\n        buf.push(b' ');\n        if let Some(code) = &self.code {\n            buf.push(b'[');\n            code.serialize(&mut buf);\n            buf.extend_from_slice(b\"] \");\n        }\n        buf.extend_from_slice(self.message.as_bytes());\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n\n    pub fn into_bytes(self) -> Vec<u8> {\n        self.serialize(Vec::with_capacity(16))\n    }\n}\n\npub trait SerializeResponse {\n    fn serialize(&self) -> Vec<u8>;\n}\n\nimpl SerializeResponse for trc::Error {\n    fn serialize(&self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(128);\n        if let Some(tag) = self.value_as_str(trc::Key::Id) {\n            buf.extend_from_slice(tag.as_bytes());\n        } else {\n            buf.push(b'*');\n        }\n        buf.push(b' ');\n        buf.extend_from_slice(self.value_as_str(trc::Key::Type).unwrap_or(\"NO\").as_bytes());\n        buf.push(b' ');\n        if let Some(code) = self\n            .value_as_str(trc::Key::Code)\n            .or_else(|| match self.as_ref() {\n                trc::EventType::Store(trc::StoreEvent::NotFound) => {\n                    Some(ResponseCode::NonExistent.as_str())\n                }\n                trc::EventType::Store(_) => Some(ResponseCode::ContactAdmin.as_str()),\n                trc::EventType::Limit(trc::LimitEvent::Quota) => {\n                    Some(ResponseCode::OverQuota.as_str())\n                }\n                trc::EventType::Limit(_) => Some(ResponseCode::Limit.as_str()),\n                trc::EventType::Auth(_) => Some(ResponseCode::AuthenticationFailed.as_str()),\n                trc::EventType::Security(_) => Some(ResponseCode::AuthorizationFailed.as_str()),\n                _ => None,\n            })\n        {\n            buf.push(b'[');\n            buf.extend_from_slice(code.as_bytes());\n            buf.extend_from_slice(b\"] \");\n        }\n        buf.extend_from_slice(\n            self.value_as_str(trc::Key::Details)\n                .unwrap_or_else(|| self.as_ref().message())\n                .as_bytes(),\n        );\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\nimpl ProtocolVersion {\n    #[inline(always)]\n    pub fn is_rev2(&self) -> bool {\n        matches!(self, ProtocolVersion::Rev2)\n    }\n\n    #[inline(always)]\n    pub fn is_rev1(&self) -> bool {\n        matches!(self, ProtocolVersion::Rev1)\n    }\n}\n\npub fn serialize_sequence(buf: &mut Vec<u8>, list: &[u32]) {\n    let mut ids = list.iter().peekable();\n    while let Some(&id) = ids.next() {\n        buf.extend_from_slice(id.to_string().as_bytes());\n        let mut range_id = id;\n        loop {\n            match ids.peek() {\n                Some(&&next_id) if next_id == range_id + 1 => {\n                    range_id += 1;\n                    ids.next();\n                }\n                next => {\n                    if range_id != id {\n                        buf.push(b':');\n                        buf.extend_from_slice(range_id.to_string().as_bytes());\n                    }\n                    if next.is_some() {\n                        buf.push(b',');\n                    }\n                    break;\n                }\n            }\n        }\n    }\n}\n\nimpl Display for Command {\n    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {\n        match self {\n            Command::Capability => write!(f, \"CAPABILITY\"),\n            Command::Noop => write!(f, \"NOOP\"),\n            Command::Logout => write!(f, \"LOGOUT\"),\n            Command::StartTls => write!(f, \"STARTTLS\"),\n            Command::Authenticate => write!(f, \"AUTHENTICATE\"),\n            Command::Login => write!(f, \"LOGIN\"),\n            Command::Enable => write!(f, \"ENABLE\"),\n            Command::Select => write!(f, \"SELECT\"),\n            Command::Examine => write!(f, \"EXAMINE\"),\n            Command::Create => write!(f, \"CREATE\"),\n            Command::Delete => write!(f, \"DELETE\"),\n            Command::Rename => write!(f, \"RENAME\"),\n            Command::Subscribe => write!(f, \"SUBSCRIBE\"),\n            Command::Unsubscribe => write!(f, \"UNSUBSCRIBE\"),\n            Command::List => write!(f, \"LIST\"),\n            Command::Namespace => write!(f, \"NAMESPACE\"),\n            Command::Status => write!(f, \"STATUS\"),\n            Command::Append => write!(f, \"APPEND\"),\n            Command::Idle => write!(f, \"IDLE\"),\n            Command::Close => write!(f, \"CLOSE\"),\n            Command::Unselect => write!(f, \"UNSELECT\"),\n            Command::Expunge(false) => write!(f, \"EXPUNGE\"),\n            Command::Search(false) => write!(f, \"SEARCH\"),\n            Command::Fetch(false) => write!(f, \"FETCH\"),\n            Command::Store(false) => write!(f, \"STORE\"),\n            Command::Copy(false) => write!(f, \"COPY\"),\n            Command::Move(false) => write!(f, \"MOVE\"),\n            Command::Sort(false) => write!(f, \"SORT\"),\n            Command::Thread(false) => write!(f, \"THREAD\"),\n            Command::Expunge(true) => write!(f, \"UID EXPUNGE\"),\n            Command::Search(true) => write!(f, \"UID SEARCH\"),\n            Command::Fetch(true) => write!(f, \"UID FETCH\"),\n            Command::Store(true) => write!(f, \"UID STORE\"),\n            Command::Copy(true) => write!(f, \"UID COPY\"),\n            Command::Move(true) => write!(f, \"UID MOVE\"),\n            Command::Sort(true) => write!(f, \"UID SORT\"),\n            Command::Thread(true) => write!(f, \"UID THREAD\"),\n            Command::Lsub => write!(f, \"LSUB\"),\n            Command::Check => write!(f, \"CHECK\"),\n            Command::SetAcl => write!(f, \"SETACL\"),\n            Command::DeleteAcl => write!(f, \"DELETEACL\"),\n            Command::GetAcl => write!(f, \"GETACL\"),\n            Command::ListRights => write!(f, \"LISTRIGHTS\"),\n            Command::MyRights => write!(f, \"MYRIGHTS\"),\n            Command::Unauthenticate => write!(f, \"UNAUTHENTICATE\"),\n            Command::Id => write!(f, \"ID\"),\n            Command::GetQuota => write!(f, \"GETQUOTA\"),\n            Command::GetQuotaRoot => write!(f, \"GETQUOTAROOT\"),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::parser::parse_sequence_set;\n\n    #[test]\n    fn sequence_set_contains() {\n        for (sequence, expected_result, max_value) in [\n            (\"1,5:10\", vec![1, 5, 6, 7, 8, 9, 10], 10),\n            (\"2,4:7,9,12:*\", vec![2, 4, 5, 6, 7, 9, 12, 13, 14, 15], 15),\n            (\"*:4,5:7\", vec![4, 5, 6, 7], 7),\n            (\"2,4,5\", vec![2, 4, 5], 5),\n        ] {\n            let sequence = parse_sequence_set(sequence.as_bytes()).unwrap();\n\n            assert_eq!(\n                (1..=15)\n                    .filter(|num| sequence.contains(*num, max_value))\n                    .collect::<Vec<_>>(),\n                expected_result\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/namespace.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapResponse, quoted_string};\n\npub struct Response {\n    pub shared_prefix: Option<String>,\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        if let Some(shared_prefix) = &self.shared_prefix {\n            buf.extend_from_slice(b\"* NAMESPACE ((\\\"\\\" \\\"/\\\")) ((\");\n            quoted_string(&mut buf, shared_prefix);\n            buf.extend_from_slice(b\" \\\"/\\\")) NIL\\r\\n\");\n        } else {\n            buf.extend_from_slice(b\"* NAMESPACE ((\\\"\\\" \\\"/\\\")) NIL NIL\\r\\n\");\n        }\n        buf\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/quota.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapResponse, capability::QuotaResourceName, quoted_string};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub name: String,\n}\n\npub struct QuotaItem {\n    pub name: String,\n    pub resources: Vec<QuotaResource>,\n}\n\npub struct QuotaResource {\n    pub resource: QuotaResourceName,\n    pub total: u64,\n    pub used: u64,\n}\n\npub struct Response {\n    pub quota_root_items: Vec<String>,\n    pub quota_items: Vec<QuotaItem>,\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        if !self.quota_root_items.is_empty() {\n            buf.extend_from_slice(b\"* QUOTAROOT\");\n            for item in &self.quota_root_items {\n                buf.push(b' ');\n                quoted_string(&mut buf, item);\n            }\n            buf.extend_from_slice(b\"\\r\\n\");\n        }\n\n        if !self.quota_items.is_empty() {\n            for item in &self.quota_items {\n                buf.extend_from_slice(b\"* QUOTA \");\n                quoted_string(&mut buf, &item.name);\n                buf.extend_from_slice(b\" (\");\n                for (pos, resource) in item.resources.iter().enumerate() {\n                    if pos > 0 {\n                        buf.push(b' ');\n                    }\n\n                    let mut total = resource.total;\n                    let mut used = resource.used;\n\n                    match resource.resource {\n                        QuotaResourceName::Storage => {\n                            total /= 1024;\n                            used /= 1024;\n\n                            buf.extend_from_slice(b\"STORAGE \")\n                        }\n                        QuotaResourceName::Message => buf.extend_from_slice(b\"MESSAGE \"),\n                        QuotaResourceName::Mailbox => buf.extend_from_slice(b\"MAILBOX \"),\n                        QuotaResourceName::AnnotationStorage => {\n                            buf.extend_from_slice(b\"ANNOTATION-STORAGE \")\n                        }\n                    }\n\n                    buf.extend_from_slice(format!(\"{used} {total}\").as_bytes());\n                }\n                buf.extend_from_slice(b\")\\r\\n\");\n            }\n        }\n\n        buf\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::{ImapResponse, capability::QuotaResourceName};\n\n    use super::{QuotaItem, QuotaResource};\n\n    #[test]\n    fn serialize_quota() {\n        for (response, expected) in [\n            (\n                super::Response {\n                    quota_root_items: vec![\"INBOX\".into(), \"#test\".into()],\n                    quota_items: vec![],\n                },\n                \"* QUOTAROOT \\\"INBOX\\\" \\\"#test\\\"\\r\\n\",\n            ),\n            (\n                super::Response {\n                    quota_root_items: vec![],\n                    quota_items: vec![QuotaItem {\n                        name: \"INBOX\".into(),\n                        resources: vec![QuotaResource {\n                            resource: QuotaResourceName::Storage,\n                            total: 1073741824,\n                            used: 1048576,\n                        }],\n                    }],\n                },\n                \"* QUOTA \\\"INBOX\\\" (STORAGE 1024 1048576)\\r\\n\",\n            ),\n            (\n                super::Response {\n                    quota_root_items: vec![\"my mailbox\".into(), \"\".into()],\n                    quota_items: vec![QuotaItem {\n                        name: \"INBOX\".into(),\n                        resources: vec![\n                            QuotaResource {\n                                resource: QuotaResourceName::Storage,\n                                total: 1073741824,\n                                used: 1048576,\n                            },\n                            QuotaResource {\n                                resource: QuotaResourceName::Message,\n                                total: 100,\n                                used: 2,\n                            },\n                        ],\n                    }],\n                },\n                concat!(\n                    \"* QUOTAROOT \\\"my mailbox\\\" \\\"\\\"\\r\\n\",\n                    \"* QUOTA \\\"INBOX\\\" (STORAGE 1024 1048576 MESSAGE 2 100)\\r\\n\"\n                ),\n            ),\n        ] {\n            assert_eq!(String::from_utf8(response.serialize()).unwrap(), expected);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/rename.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n    pub new_mailbox_name: String,\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Flag, Sequence, quoted_string, serialize_sequence};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub is_esearch: bool,\n    pub sort: Option<Vec<Comparator>>,\n    pub result_options: Vec<ResultOption>,\n    pub filter: Vec<Filter>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Sort {\n    Arrival,\n    Cc,\n    Date,\n    From,\n    DisplayFrom,\n    Size,\n    Subject,\n    To,\n    DisplayTo,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Comparator {\n    pub sort: Sort,\n    pub ascending: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Response {\n    pub is_uid: bool,\n    pub is_esearch: bool,\n    pub is_sort: bool,\n    pub ids: Vec<u32>,\n    pub min: Option<u32>,\n    pub max: Option<u32>,\n    pub count: Option<u32>,\n    pub highest_modseq: Option<u64>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ResultOption {\n    Min,\n    Max,\n    All,\n    Count,\n    Save,\n    Context,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Filter {\n    Sequence(Sequence, bool),\n    All,\n    Answered,\n    Bcc(String),\n    Before(i64),\n    Body(String),\n    Cc(String),\n    Deleted,\n    Draft,\n    Flagged,\n    From(String),\n    Header(String, String),\n    Keyword(Flag),\n    Larger(u32),\n    On(i64),\n    Seen,\n    SentBefore(i64),\n    SentOn(i64),\n    SentSince(i64),\n    Since(i64),\n    Smaller(u32),\n    Subject(String),\n    Text(String),\n    To(String),\n    Unanswered,\n    Undeleted,\n    Undraft,\n    Unflagged,\n    Unkeyword(Flag),\n    Unseen,\n\n    // Logical operators\n    And,\n    Or,\n    Not,\n    End,\n\n    // Imap4rev1\n    Recent,\n    New,\n    Old,\n\n    // RFC 5032 - WITHIN\n    Older(u32),\n    Younger(u32),\n\n    // RFC 4551 - CONDSTORE\n    ModSeq((u64, ModSeqEntry)),\n\n    // RFC 8474 - ObjectID\n    EmailId(String),\n    ThreadId(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ModSeqEntry {\n    Shared(Flag),\n    Private(Flag),\n    All(Flag),\n    None,\n}\n\nimpl Filter {\n    pub fn seq_saved_search() -> Filter {\n        Filter::Sequence(Sequence::SavedSearch, false)\n    }\n\n    pub fn seq_range(start: Option<u32>, end: Option<u32>) -> Filter {\n        Filter::Sequence(Sequence::Range { start, end }, false)\n    }\n}\n\nimpl Response {\n    pub fn serialize(self, tag: &str) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        if self.is_esearch {\n            buf.extend_from_slice(b\"* ESEARCH (TAG \");\n            quoted_string(&mut buf, tag);\n            buf.extend_from_slice(b\")\");\n            if self.is_uid {\n                buf.extend_from_slice(b\" UID\");\n            }\n            if let Some(count) = &self.count {\n                buf.extend_from_slice(b\" COUNT \");\n                buf.extend_from_slice(count.to_string().as_bytes());\n            }\n            if let Some(min) = &self.min {\n                buf.extend_from_slice(b\" MIN \");\n                buf.extend_from_slice(min.to_string().as_bytes());\n            }\n            if let Some(max) = &self.max {\n                buf.extend_from_slice(b\" MAX \");\n                buf.extend_from_slice(max.to_string().as_bytes());\n            }\n            if !self.ids.is_empty() {\n                buf.extend_from_slice(b\" ALL \");\n                serialize_sequence(&mut buf, &self.ids);\n            }\n            if let Some(highest_modseq) = self.highest_modseq {\n                buf.extend_from_slice(b\" MODSEQ \");\n                buf.extend_from_slice(highest_modseq.to_string().as_bytes());\n            }\n        } else {\n            if !self.is_sort {\n                buf.extend_from_slice(b\"* SEARCH\");\n            } else {\n                buf.extend_from_slice(b\"* SORT\");\n            }\n            if !self.ids.is_empty() {\n                for id in &self.ids {\n                    buf.push(b' ');\n                    buf.extend_from_slice(id.to_string().as_bytes());\n                }\n            }\n            if let Some(highest_modseq) = self.highest_modseq {\n                buf.extend_from_slice(b\" (MODSEQ \");\n                buf.extend_from_slice(highest_modseq.to_string().as_bytes());\n                buf.push(b')');\n            }\n        }\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    #[test]\n    fn serialize_search() {\n        for (mut response, tag, expected_v2, expected_v1) in [\n            (\n                super::Response {\n                    is_uid: false,\n                    is_esearch: true,\n                    is_sort: false,\n                    ids: vec![2, 10, 11],\n                    min: 2.into(),\n                    max: 11.into(),\n                    count: 3.into(),\n                    highest_modseq: None,\n                },\n                \"A283\",\n                \"* ESEARCH (TAG \\\"A283\\\") COUNT 3 MIN 2 MAX 11 ALL 2,10:11\\r\\n\",\n                \"* SEARCH 2 10 11\\r\\n\",\n            ),\n            (\n                super::Response {\n                    is_uid: false,\n                    is_esearch: true,\n                    is_sort: false,\n                    ids: vec![\n                        1, 2, 3, 5, 10, 11, 12, 13, 90, 92, 93, 94, 95, 96, 97, 98, 99,\n                    ],\n                    min: None,\n                    max: None,\n                    count: None,\n                    highest_modseq: None,\n                },\n                \"A283\",\n                \"* ESEARCH (TAG \\\"A283\\\") ALL 1:3,5,10:13,90,92:99\\r\\n\",\n                \"* SEARCH 1 2 3 5 10 11 12 13 90 92 93 94 95 96 97 98 99\\r\\n\",\n            ),\n            (\n                super::Response {\n                    is_uid: false,\n                    is_esearch: true,\n                    is_sort: false,\n                    ids: vec![],\n                    min: None,\n                    max: None,\n                    count: None,\n                    highest_modseq: None,\n                },\n                \"A283\",\n                \"* ESEARCH (TAG \\\"A283\\\")\\r\\n\",\n                \"* SEARCH\\r\\n\",\n            ),\n            (\n                super::Response {\n                    is_uid: false,\n                    is_esearch: true,\n                    is_sort: false,\n                    ids: vec![10, 11, 12, 13, 21],\n                    min: None,\n                    max: None,\n                    count: None,\n                    highest_modseq: 12345.into(),\n                },\n                \"A283\",\n                \"* ESEARCH (TAG \\\"A283\\\") ALL 10:13,21 MODSEQ 12345\\r\\n\",\n                \"* SEARCH 10 11 12 13 21 (MODSEQ 12345)\\r\\n\",\n            ),\n        ] {\n            let response_v2 = String::from_utf8(response.clone().serialize(tag)).unwrap();\n            response.is_esearch = false;\n            let response_v1 = String::from_utf8(response.serialize(tag)).unwrap();\n\n            assert_eq!(response_v2, expected_v2);\n            assert_eq!(response_v1, expected_v1);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/select.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{ResponseCode, StatusResponse};\n\nuse super::{ImapResponse, Sequence, list::ListItem};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n    pub condstore: bool,\n    pub qresync: Option<QResync>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct QResync {\n    pub uid_validity: u32,\n    pub modseq: u64,\n    pub known_uids: Option<Sequence>,\n    pub seq_match: Option<(Sequence, Sequence)>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct HighestModSeq(u64);\n\n#[derive(Debug, Clone)]\npub struct Response {\n    pub mailbox: ListItem,\n    pub total_messages: usize,\n    pub recent_messages: usize,\n    pub unseen_seq: u32,\n    pub uid_validity: u32,\n    pub uid_next: u32,\n    pub is_rev2: bool,\n    pub is_utf8: bool,\n    pub closed_previous: bool,\n    pub highest_modseq: Option<HighestModSeq>,\n    pub mailbox_id: String,\n}\n\n#[derive(Debug, Clone)]\npub struct Exists {\n    pub total_messages: usize,\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(100);\n        if self.closed_previous {\n            buf = StatusResponse::ok(\"Closed previous mailbox\")\n                .with_code(ResponseCode::Closed)\n                .serialize(buf);\n        }\n        buf.extend_from_slice(b\"* \");\n        buf.extend_from_slice(self.total_messages.to_string().as_bytes());\n        if !self.is_rev2 && self.recent_messages > 0 {\n            buf.extend_from_slice(\n                b\" EXISTS\\r\\n* FLAGS (\\\\Answered \\\\Flagged \\\\Deleted \\\\Seen \\\\Draft \\\\Recent)\\r\\n\",\n            );\n        } else {\n            buf.extend_from_slice(\n                b\" EXISTS\\r\\n* FLAGS (\\\\Answered \\\\Flagged \\\\Deleted \\\\Seen \\\\Draft)\\r\\n\",\n            );\n        }\n        if self.is_rev2 {\n            self.mailbox\n                .serialize(&mut buf, self.is_rev2, self.is_utf8, false);\n        } else {\n            buf.extend_from_slice(b\"* \");\n            buf.extend_from_slice(self.recent_messages.to_string().as_bytes());\n            buf.extend_from_slice(b\" RECENT\\r\\n\");\n            if self.unseen_seq > 0 {\n                buf.extend_from_slice(b\"* OK [UNSEEN \");\n                buf.extend_from_slice(self.unseen_seq.to_string().as_bytes());\n                buf.extend_from_slice(b\"] Unseen messages\\r\\n\");\n            }\n        }\n        buf.extend_from_slice(\n            b\"* OK [PERMANENTFLAGS (\\\\Deleted \\\\Seen \\\\Answered \\\\Flagged \\\\Draft \\\\*)] All allowed\\r\\n\",\n        );\n        buf.extend_from_slice(b\"* OK [UIDVALIDITY \");\n        buf.extend_from_slice(self.uid_validity.to_string().as_bytes());\n        buf.extend_from_slice(b\"] UIDs valid\\r\\n* OK [UIDNEXT \");\n        buf.extend_from_slice(self.uid_next.to_string().as_bytes());\n        buf.extend_from_slice(b\"] Next predicted UID\\r\\n\");\n        if let Some(highest_modseq) = self.highest_modseq {\n            highest_modseq.serialize(&mut buf);\n        }\n        buf.extend_from_slice(b\"* OK [MAILBOXID (\");\n        buf.extend_from_slice(self.mailbox_id.as_bytes());\n        buf.extend_from_slice(b\")] Unique Mailbox ID\\r\\n\");\n        buf\n    }\n}\n\nimpl HighestModSeq {\n    pub fn new(modseq: u64) -> Self {\n        Self(modseq)\n    }\n\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(b\"* OK [HIGHESTMODSEQ \");\n        buf.extend_from_slice(self.0.to_string().as_bytes());\n        buf.extend_from_slice(b\"] Highest Modseq\\r\\n\");\n    }\n\n    pub fn into_bytes(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(40);\n        self.serialize(&mut buf);\n        buf\n    }\n}\n\nimpl Exists {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(b\"* \");\n        buf.extend_from_slice(self.total_messages.to_string().as_bytes());\n        buf.extend_from_slice(b\" EXISTS\\r\\n\");\n    }\n\n    pub fn into_bytes(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(15);\n        self.serialize(&mut buf);\n        buf\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::{ImapResponse, list::ListItem};\n\n    use super::HighestModSeq;\n\n    #[test]\n    fn serialize_select() {\n        for (mut response, _tag, expected_v2, expected_v1) in [\n            (\n                super::Response {\n                    mailbox: ListItem::new(\"INBOX\"),\n                    total_messages: 172,\n                    recent_messages: 5,\n                    unseen_seq: 3,\n                    uid_validity: 3857529045,\n                    uid_next: 4392,\n                    closed_previous: false,\n                    is_rev2: true,\n                    is_utf8: true,\n                    highest_modseq: HighestModSeq::new(100).into(),\n                    mailbox_id: \"abc\".into(),\n                },\n                \"A142\",\n                concat!(\n                    \"* 172 EXISTS\\r\\n\",\n                    \"* FLAGS (\\\\Answered \\\\Flagged \\\\Deleted \\\\Seen \\\\Draft)\\r\\n\",\n                    \"* LIST () \\\"/\\\" \\\"INBOX\\\"\\r\\n\",\n                    \"* OK [PERMANENTFLAGS (\\\\Deleted \\\\Seen \\\\Answered \\\\Flagged \\\\Draft \\\\*)] All allowed\\r\\n\",\n                    \"* OK [UIDVALIDITY 3857529045] UIDs valid\\r\\n\",\n                    \"* OK [UIDNEXT 4392] Next predicted UID\\r\\n\",\n                    \"* OK [HIGHESTMODSEQ 100] Highest Modseq\\r\\n\",\n                    \"* OK [MAILBOXID (abc)] Unique Mailbox ID\\r\\n\"\n                ),\n                concat!(\n                    \"* 172 EXISTS\\r\\n\",\n                    \"* FLAGS (\\\\Answered \\\\Flagged \\\\Deleted \\\\Seen \\\\Draft \\\\Recent)\\r\\n\",\n                    \"* 5 RECENT\\r\\n\",\n                    \"* OK [UNSEEN 3] Unseen messages\\r\\n\",\n                    \"* OK [PERMANENTFLAGS (\\\\Deleted \\\\Seen \\\\Answered \\\\Flagged \\\\Draft \\\\*)] All allowed\\r\\n\",\n                    \"* OK [UIDVALIDITY 3857529045] UIDs valid\\r\\n\",\n                    \"* OK [UIDNEXT 4392] Next predicted UID\\r\\n\",\n                    \"* OK [HIGHESTMODSEQ 100] Highest Modseq\\r\\n\",\n                    \"* OK [MAILBOXID (abc)] Unique Mailbox ID\\r\\n\"\n                ),\n            ),\n            (\n                super::Response {\n                    mailbox: ListItem::new(\"~peter/mail/台北/日本語\"),\n                    total_messages: 172,\n                    recent_messages: 5,\n                    unseen_seq: 3,\n                    uid_validity: 3857529045,\n                    uid_next: 4392,\n                    closed_previous: true,\n                    is_rev2: true,\n                    is_utf8: true,\n                    highest_modseq: None,\n                    mailbox_id: \"abc\".into(),\n                },\n                \"A142\",\n                concat!(\n                    \"* OK [CLOSED] Closed previous mailbox\\r\\n\",\n                    \"* 172 EXISTS\\r\\n\",\n                    \"* FLAGS (\\\\Answered \\\\Flagged \\\\Deleted \\\\Seen \\\\Draft)\\r\\n\",\n                    \"* LIST () \\\"/\\\" \\\"~peter/mail/台北/日本語\\\" (\\\"OLDNAME\\\" \",\n                    \"(\\\"~peter/mail/&U,BTFw-/&ZeVnLIqe-\\\"))\\r\\n\",\n                    \"* OK [PERMANENTFLAGS (\\\\Deleted \\\\Seen \\\\Answered \\\\Flagged \\\\Draft \\\\*)] All allowed\\r\\n\",\n                    \"* OK [UIDVALIDITY 3857529045] UIDs valid\\r\\n\",\n                    \"* OK [UIDNEXT 4392] Next predicted UID\\r\\n\",\n                    \"* OK [MAILBOXID (abc)] Unique Mailbox ID\\r\\n\"\n                ),\n                concat!(\n                    \"* OK [CLOSED] Closed previous mailbox\\r\\n\",\n                    \"* 172 EXISTS\\r\\n\",\n                    \"* FLAGS (\\\\Answered \\\\Flagged \\\\Deleted \\\\Seen \\\\Draft \\\\Recent)\\r\\n\",\n                    \"* 5 RECENT\\r\\n\",\n                    \"* OK [UNSEEN 3] Unseen messages\\r\\n\",\n                    \"* OK [PERMANENTFLAGS (\\\\Deleted \\\\Seen \\\\Answered \\\\Flagged \\\\Draft \\\\*)] All allowed\\r\\n\",\n                    \"* OK [UIDVALIDITY 3857529045] UIDs valid\\r\\n\",\n                    \"* OK [UIDNEXT 4392] Next predicted UID\\r\\n\",\n                    \"* OK [MAILBOXID (abc)] Unique Mailbox ID\\r\\n\"\n                ),\n            ),\n        ] {\n            let response_v2 = String::from_utf8(response.clone().serialize()).unwrap();\n            response.is_rev2 = false;\n            let response_v1 = String::from_utf8(response.serialize()).unwrap();\n\n            assert_eq!(response_v2, expected_v2);\n            assert_eq!(response_v1, expected_v1);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/status.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::utf7::utf7_encode;\n\nuse super::quoted_string;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n    pub items: Vec<Status>,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum Status {\n    Messages,\n    UidNext,\n    UidValidity,\n    Unseen,\n    Deleted,\n    Size,\n    Recent,\n    HighestModSeq,\n    MailboxId,\n    DeletedStorage,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct StatusItem {\n    pub mailbox_name: String,\n    pub items: Vec<(Status, StatusItemType)>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum StatusItemType {\n    Number(u64),\n    String(String),\n}\n\nimpl StatusItem {\n    pub fn serialize(&self, buf: &mut Vec<u8>, is_utf8: bool) {\n        buf.extend_from_slice(b\"* STATUS \");\n        if is_utf8 {\n            quoted_string(buf, &self.mailbox_name);\n        } else {\n            quoted_string(buf, &utf7_encode(&self.mailbox_name));\n        }\n        buf.extend_from_slice(b\" (\");\n        for (pos, (status_item, value)) in self.items.iter().enumerate() {\n            if pos > 0 {\n                buf.push(b' ');\n            }\n\n            buf.extend_from_slice(match status_item {\n                Status::Messages => b\"MESSAGES \",\n                Status::UidNext => b\"UIDNEXT \",\n                Status::UidValidity => b\"UIDVALIDITY \",\n                Status::Unseen => b\"UNSEEN \",\n                Status::Deleted => b\"DELETED \",\n                Status::Size => b\"SIZE \",\n                Status::HighestModSeq => b\"HIGHESTMODSEQ \",\n                Status::MailboxId => b\"MAILBOXID \",\n                Status::Recent => b\"RECENT \",\n                Status::DeletedStorage => b\"DELETED-STORAGE \",\n            });\n\n            match value {\n                StatusItemType::Number(num) => {\n                    buf.extend_from_slice(num.to_string().as_bytes());\n                }\n                StatusItemType::String(str) => {\n                    buf.push(b'(');\n                    buf.extend_from_slice(str.as_bytes());\n                    buf.push(b')');\n                }\n            }\n        }\n        buf.extend_from_slice(b\")\\r\\n\");\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::status::{Status, StatusItem, StatusItemType};\n\n    #[test]\n    fn serialize_status() {\n        let mut buf = Vec::new();\n        StatusItem {\n            mailbox_name: \"blurdybloop\".into(),\n            items: vec![\n                (Status::Messages, StatusItemType::Number(231)),\n                (Status::UidNext, StatusItemType::Number(44292)),\n                (Status::MailboxId, StatusItemType::String(\"abc-123\".into())),\n            ],\n        }\n        .serialize(&mut buf, true);\n\n        assert_eq!(\n            String::from_utf8(buf).unwrap(),\n            \"* STATUS \\\"blurdybloop\\\" (MESSAGES 231 UIDNEXT 44292 MAILBOXID (abc-123))\\r\\n\"\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/store.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Flag, ImapResponse, Sequence, fetch::FetchItem};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub sequence_set: Sequence,\n    pub operation: Operation,\n    pub is_silent: bool,\n    pub keywords: Vec<Flag>,\n    pub unchanged_since: Option<u64>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Operation {\n    Set,\n    Add,\n    Clear,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Response<'x> {\n    pub items: Vec<FetchItem<'x>>,\n}\n\nimpl ImapResponse for Response<'_> {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        for item in &self.items {\n            item.serialize(&mut buf);\n        }\n        buf\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/subscribe.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub mailbox_name: String,\n}\n"
  },
  {
    "path": "crates/imap-proto/src/protocol/thread.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ImapResponse, search::Filter};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Arguments {\n    pub tag: String,\n    pub filter: Vec<Filter>,\n    pub algorithm: Algorithm,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Algorithm {\n    OrderedSubject,\n    References,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Response {\n    pub is_uid: bool,\n    pub threads: Vec<Vec<u32>>,\n}\n\nimpl ImapResponse for Response {\n    fn serialize(self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        buf.extend_from_slice(b\"* THREAD \");\n        for thread in &self.threads {\n            buf.push(b'(');\n            for (pos, id) in thread.iter().enumerate() {\n                if pos > 0 {\n                    buf.push(b' ');\n                }\n                buf.extend_from_slice(id.to_string().as_bytes());\n            }\n            buf.push(b')');\n        }\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::ImapResponse;\n\n    #[test]\n    fn serialize_thread() {\n        assert_eq!(\n            String::from_utf8(\n                super::Response {\n                    is_uid: true,\n                    threads: vec![vec![2, 10, 11], vec![49], vec![1, 3]],\n                }\n                .serialize()\n            )\n            .unwrap(),\n            \"* THREAD (2 10 11)(49)(1 3)\\r\\n\"\n        );\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/receiver.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ResponseCode, ResponseType};\nuse compact_str::{CompactString, format_compact};\nuse std::fmt::Display;\n\n#[derive(Debug, Clone)]\npub enum Error {\n    NeedsMoreData,\n    NeedsLiteral { size: u32 },\n    Error { response: trc::Error },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Request<T: CommandParser> {\n    pub tag: String,\n    pub command: T,\n    pub tokens: Vec<Token>,\n}\n\npub trait CommandParser: Sized + Default {\n    fn parse(bytes: &[u8], is_uid: bool) -> Option<Self>;\n    fn tokenize_brackets(&self) -> bool;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Token {\n    Argument(Vec<u8>),\n    ParenthesisOpen,  // (\n    ParenthesisClose, // )\n    BracketOpen,      // [\n    BracketClose,     // ]\n    Lt,               // <\n    Gt,               // >\n    Dot,              // .\n    Nil,              // NIL\n}\n\nimpl<T: CommandParser> Default for Request<T> {\n    fn default() -> Self {\n        Self {\n            tag: String::new(),\n            command: T::default(),\n            tokens: Vec::new(),\n        }\n    }\n}\n\n#[derive(Debug, Copy, Clone, PartialEq, Eq)]\npub enum State {\n    Start,\n    Tag,\n    Command { is_uid: bool },\n    Argument { last_ch: u8 },\n    ArgumentQuoted { escaped: bool },\n    Literal { non_sync: bool },\n    LiteralSeek { size: u32, non_sync: bool },\n    LiteralData { remaining: u32 },\n}\n\npub struct Receiver<T: CommandParser> {\n    buf: ArgumentBuffer,\n    pub request: Request<T>,\n    pub state: State,\n    pub max_request_size: usize,\n    pub current_request_size: usize,\n    pub start_state: State,\n}\n\nconst ARG_MAX_LEN: usize = 4096;\n\nstruct ArgumentBuffer {\n    buf: Vec<u8>,\n}\n\nimpl<T: CommandParser> Receiver<T> {\n    pub fn new() -> Self {\n        Receiver {\n            max_request_size: 25 * 1024 * 1024, // 25MB\n            ..Default::default()\n        }\n    }\n\n    pub fn with_start_state(mut self, state: State) -> Self {\n        self.state = state;\n        self.start_state = state;\n        self\n    }\n\n    pub fn with_max_request_size(max_request_size: usize) -> Self {\n        Receiver {\n            max_request_size,\n            ..Default::default()\n        }\n    }\n\n    pub fn error_reset(&mut self, message: impl Into<trc::Value>) -> Error {\n        let request = std::mem::take(&mut self.request);\n        let err = Error::err(\n            if !request.tag.is_empty() {\n                request.tag.into()\n            } else {\n                None\n            },\n            message,\n        );\n        self.buf = ArgumentBuffer::default();\n        self.state = self.start_state;\n        self.current_request_size = 0;\n        err\n    }\n\n    fn push_argument(&mut self, in_quote: bool) -> Result<(), Error> {\n        if !self.buf.is_empty() {\n            self.current_request_size += self.buf.len();\n            if self.current_request_size > self.max_request_size {\n                return Err(self.error_reset(format_compact!(\n                    \"Request exceeds maximum limit of {} bytes.\",\n                    self.max_request_size\n                )));\n            }\n            self.request.tokens.push(Token::Argument(self.buf.take()));\n        } else if in_quote {\n            self.request.tokens.push(Token::Nil);\n        }\n        Ok(())\n    }\n\n    fn push_token(&mut self, token: Token) -> Result<(), Error> {\n        self.current_request_size += 1;\n        if self.current_request_size > self.max_request_size {\n            return Err(self.error_reset(format_compact!(\n                \"Request exceeds maximum limit of {} bytes.\",\n                self.max_request_size\n            )));\n        }\n        self.request.tokens.push(token);\n        Ok(())\n    }\n\n    pub fn parse(&mut self, bytes: &mut std::slice::Iter<'_, u8>) -> Result<Request<T>, Error> {\n        #[allow(clippy::while_let_on_iterator)]\n        while let Some(&ch) = bytes.next() {\n            match self.state {\n                State::Start => {\n                    if !ch.is_ascii_whitespace() {\n                        // SAFETY: This called just once\n                        self.buf.push_unchecked(ch);\n                        self.state = State::Tag;\n                    }\n                }\n                State::Tag => match ch {\n                    b' ' => {\n                        if !self.buf.is_empty() {\n                            self.request.tag =\n                                String::from_utf8(self.buf.take()).map_err(|_| {\n                                    self.error_reset(\"Tag is not a valid UTF-8 string.\")\n                                })?;\n                            self.state = State::Command { is_uid: false };\n                        }\n                    }\n                    b'\\t' | b'\\r' => {}\n                    b'\\n' => {\n                        return Err(self.error_reset(format_compact!(\n                            \"Missing command after tag {:?}, found CRLF instead.\",\n                            self.buf.as_str()\n                        )));\n                    }\n                    _ => {\n                        self.buf.push_checked(ch, 128).map_err(|_| {\n                            self.error_reset(\"Tag exceeds maximum length of 128 characters.\")\n                        })?;\n                    }\n                },\n                State::Command { is_uid } => {\n                    if ch.is_ascii_alphanumeric() {\n                        self.buf\n                            .push_checked(ch.to_ascii_uppercase(), 15)\n                            .map_err(|_| {\n                                self.error_reset(\"Command exceeds maximum length of 15 characters.\")\n                            })?;\n                    } else if ch.is_ascii_whitespace() {\n                        if !self.buf.is_empty() {\n                            if !self.buf.as_ref().eq_ignore_ascii_case(b\"UID\") {\n                                self.request.command = T::parse(self.buf.as_ref(), is_uid)\n                                    .ok_or_else(|| {\n                                        let err = format_compact!(\n                                            \"Unrecognized command '{}'.\",\n                                            String::from_utf8_lossy(self.buf.as_ref())\n                                        );\n                                        self.error_reset(err)\n                                    })?;\n                                self.buf.clear();\n                                if ch != b'\\n' {\n                                    self.state = State::Argument { last_ch: b' ' };\n                                } else {\n                                    self.state = self.start_state;\n                                    self.current_request_size = 0;\n                                    return Ok(std::mem::take(&mut self.request));\n                                }\n                            } else {\n                                self.buf.clear();\n                                self.state = State::Command { is_uid: true };\n                            }\n                        }\n                    } else {\n                        return Err(self.error_reset(format_compact!(\n                            \"Invalid character {:?} in command name.\",\n                            ch as char\n                        )));\n                    }\n                }\n                State::Argument { last_ch } => match ch {\n                    b'\\\"' if last_ch.is_ascii_whitespace() => {\n                        self.push_argument(false)?;\n                        self.state = State::ArgumentQuoted { escaped: false };\n                    }\n                    b'{' if last_ch.is_ascii_whitespace()\n                        || (last_ch == b'~' && self.buf.len() == 1) =>\n                    {\n                        if last_ch != b'~' {\n                            self.push_argument(false)?;\n                        } else {\n                            self.buf.clear();\n                        }\n                        self.state = State::Literal { non_sync: false };\n                    }\n                    b'(' => {\n                        self.push_argument(false)?;\n                        self.push_token(Token::ParenthesisOpen)?;\n                    }\n                    b')' => {\n                        self.push_argument(false)?;\n                        self.push_token(Token::ParenthesisClose)?;\n                    }\n                    b'[' if self.request.command.tokenize_brackets() => {\n                        self.push_argument(false)?;\n                        self.push_token(Token::BracketOpen)?;\n                    }\n                    b']' if self.request.command.tokenize_brackets() => {\n                        self.push_argument(false)?;\n                        self.push_token(Token::BracketClose)?;\n                    }\n                    b'<' if self.request.command.tokenize_brackets() => {\n                        self.push_argument(false)?;\n                        self.push_token(Token::Lt)?;\n                    }\n                    b'>' if self.request.command.tokenize_brackets() => {\n                        self.push_argument(false)?;\n                        self.push_token(Token::Gt)?;\n                    }\n                    b'.' if self.request.command.tokenize_brackets() => {\n                        self.push_argument(false)?;\n                        self.push_token(Token::Dot)?;\n                    }\n                    b'\\n' => {\n                        self.push_argument(false)?;\n                        self.state = self.start_state;\n                        self.current_request_size = 0;\n                        return Ok(std::mem::take(&mut self.request));\n                    }\n                    _ if ch.is_ascii_whitespace() => {\n                        self.push_argument(false)?;\n                        self.state = State::Argument { last_ch: ch };\n                    }\n                    _ => {\n                        self.buf.push_checked(ch, ARG_MAX_LEN).map_err(|_| {\n                            self.error_reset(\"Argument exceeds maximum length of 4096 bytes.\")\n                        })?;\n                        self.state = State::Argument { last_ch: ch };\n                    }\n                },\n                State::ArgumentQuoted { escaped } => match ch {\n                    b'\\\"' => {\n                        if !escaped {\n                            self.push_argument(true)?;\n                            self.state = State::Argument { last_ch: b' ' };\n                        } else {\n                            self.buf\n                                .push_checked(ch, ARG_MAX_LEN)\n                                .map_err(|_| self.error_reset(\"Quoted argument too long.\"))?;\n                            self.state = State::ArgumentQuoted { escaped: false };\n                        }\n                    }\n                    b'\\\\' => {\n                        if escaped {\n                            self.buf\n                                .push_checked(ch, ARG_MAX_LEN)\n                                .map_err(|_| self.error_reset(\"Quoted argument too long.\"))?;\n                        }\n                        self.state = State::ArgumentQuoted { escaped: !escaped };\n                    }\n                    b'\\n' => {\n                        return Err(self.error_reset(\"Unterminated quoted argument.\"));\n                    }\n                    _ => {\n                        if escaped {\n                            // SAFETY: We check the size below\n                            self.buf.push_unchecked(b'\\\\');\n                        }\n                        self.buf\n                            .push_checked(ch, ARG_MAX_LEN)\n                            .map_err(|_| self.error_reset(\"Quoted argument too long.\"))?;\n                        self.state = State::ArgumentQuoted { escaped: false };\n                    }\n                },\n                State::Literal { non_sync } => {\n                    match ch {\n                        b'}' => {\n                            if !self.buf.is_empty() {\n                                let size = self.buf.as_str().parse::<u32>().map_err(|_| {\n                                    self.error_reset(\"Literal size is not a valid number.\")\n                                })?;\n                                if self.current_request_size + size as usize > self.max_request_size\n                                {\n                                    return Err(self.error_reset(format_compact!(\n                                        \"Literal exceeds the maximum request size of {} bytes.\",\n                                        self.max_request_size\n                                    )));\n                                }\n                                self.state = State::LiteralSeek { size, non_sync };\n                                self.buf.resize_buffer(size as usize);\n                                self.buf.clear();\n                            } else {\n                                return Err(self.error_reset(\"Invalid empty literal.\"));\n                            }\n                        }\n                        b'+' => {\n                            if !self.buf.is_empty() {\n                                self.state = State::Literal { non_sync: true };\n                            } else {\n                                return Err(self.error_reset(\"Invalid non-sync literal.\"));\n                            }\n                        }\n                        _ if ch.is_ascii_digit() => {\n                            if !non_sync {\n                                self.buf.push_checked(ch, 15).map_err(|_| {\n                                    self.error_reset(\"Literal size exceeds maximum of 15 digits.\")\n                                })?;\n                            } else {\n                                // Digit found after non-sync '+' flag\n                                return Err(self.error_reset(\"Invalid literal.\"));\n                            }\n                        }\n                        _ => {\n                            return Err(self.error_reset(format_compact!(\n                                \"Invalid character {:?} in literal.\",\n                                ch as char\n                            )));\n                        }\n                    }\n                }\n                State::LiteralSeek { size, non_sync } => {\n                    if ch == b'\\n' {\n                        if size > 0 {\n                            self.state = State::LiteralData { remaining: size };\n                        } else {\n                            self.state = State::Argument { last_ch: b' ' };\n                            self.push_token(Token::Nil)?;\n                        }\n                        if !non_sync {\n                            return Err(Error::NeedsLiteral { size });\n                        }\n                    } else if !ch.is_ascii_whitespace() {\n                        return Err(\n                            self.error_reset(\"Expected CRLF after literal, found an invalid char.\")\n                        );\n                    }\n                }\n                State::LiteralData { remaining } => {\n                    // SAFETY: We checked the size before entering this state\n                    self.buf.push_unchecked(ch);\n\n                    if remaining > 1 {\n                        self.state = State::LiteralData {\n                            remaining: remaining - 1,\n                        };\n                    } else {\n                        self.push_argument(false)?;\n                        self.state = State::Argument { last_ch: b' ' };\n                    }\n                }\n            }\n        }\n\n        Err(Error::NeedsMoreData)\n    }\n}\n\nimpl ArgumentBuffer {\n    pub fn new() -> Self {\n        ArgumentBuffer {\n            buf: Vec::with_capacity(10),\n        }\n    }\n\n    pub fn resize_buffer(&mut self, size: usize) {\n        if self.buf.capacity() < size {\n            self.buf.reserve(size - self.buf.capacity());\n        }\n    }\n\n    #[inline(always)]\n    pub fn push_checked(&mut self, byte: u8, limit: usize) -> Result<(), ()> {\n        if self.buf.len() < limit {\n            self.buf.push(byte);\n            Ok(())\n        } else {\n            Err(())\n        }\n    }\n\n    #[inline(always)]\n    pub fn push_unchecked(&mut self, byte: u8) {\n        self.buf.push(byte);\n    }\n\n    pub fn take(&mut self) -> Vec<u8> {\n        let buf = self.buf.clone();\n        self.buf.clear();\n        buf\n    }\n\n    #[inline(always)]\n    pub fn len(&self) -> usize {\n        self.buf.len()\n    }\n\n    #[inline(always)]\n    pub fn is_empty(&self) -> bool {\n        self.buf.is_empty()\n    }\n\n    #[inline(always)]\n    pub fn clear(&mut self) {\n        self.buf.clear();\n    }\n\n    #[inline(always)]\n    pub fn as_str(&self) -> &str {\n        std::str::from_utf8(&self.buf).unwrap_or_default()\n    }\n}\n\nimpl Token {\n    pub fn unwrap_string(self) -> crate::parser::Result<String> {\n        match self {\n            Token::Argument(value) => {\n                String::from_utf8(value).map_err(|_| \"Invalid UTF-8 in argument.\".into())\n            }\n            other => Ok(other.to_string()),\n        }\n    }\n\n    pub fn unwrap_bytes(self) -> Vec<u8> {\n        match self {\n            Token::Argument(value) => value,\n            other => other.as_bytes().to_vec(),\n        }\n    }\n\n    pub fn eq_ignore_ascii_case(&self, bytes: &[u8]) -> bool {\n        match self {\n            Token::Argument(argument) => argument.eq_ignore_ascii_case(bytes),\n            Token::ParenthesisOpen => bytes.eq(b\"(\"),\n            Token::ParenthesisClose => bytes.eq(b\")\"),\n            Token::BracketOpen => bytes.eq(b\"[\"),\n            Token::BracketClose => bytes.eq(b\"]\"),\n            Token::Gt => bytes.eq(b\">\"),\n            Token::Lt => bytes.eq(b\"<\"),\n            Token::Dot => bytes.eq(b\".\"),\n            Token::Nil => bytes.is_empty(),\n        }\n    }\n\n    pub fn is_parenthesis_open(&self) -> bool {\n        matches!(self, Token::ParenthesisOpen)\n    }\n\n    pub fn is_parenthesis_close(&self) -> bool {\n        matches!(self, Token::ParenthesisClose)\n    }\n\n    pub fn is_bracket_open(&self) -> bool {\n        matches!(self, Token::BracketOpen)\n    }\n\n    pub fn is_bracket_close(&self) -> bool {\n        matches!(self, Token::BracketClose)\n    }\n\n    pub fn is_dot(&self) -> bool {\n        matches!(self, Token::Dot)\n    }\n\n    pub fn is_lt(&self) -> bool {\n        matches!(self, Token::Lt)\n    }\n\n    pub fn is_gt(&self) -> bool {\n        matches!(self, Token::Gt)\n    }\n}\n\nimpl AsRef<[u8]> for ArgumentBuffer {\n    fn as_ref(&self) -> &[u8] {\n        &self.buf\n    }\n}\n\nimpl Default for ArgumentBuffer {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n\nimpl Display for Token {\n    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {\n        f.write_str(&String::from_utf8_lossy(self.as_bytes()))\n    }\n}\n\nimpl Token {\n    pub fn as_bytes(&self) -> &[u8] {\n        match self {\n            Token::Argument(value) => value,\n            Token::ParenthesisOpen => b\"(\",\n            Token::ParenthesisClose => b\")\",\n            Token::BracketOpen => b\"[\",\n            Token::BracketClose => b\"]\",\n            Token::Gt => b\">\",\n            Token::Lt => b\"<\",\n            Token::Dot => b\".\",\n            Token::Nil => b\"\",\n        }\n    }\n}\n\nimpl Error {\n    pub fn err(tag: Option<impl Into<CompactString>>, message: impl Into<trc::Value>) -> Self {\n        Error::Error {\n            response: trc::ImapEvent::Error\n                .ctx(trc::Key::Details, message)\n                .ctx_opt(trc::Key::Id, tag.map(Into::into))\n                .ctx(trc::Key::Type, ResponseType::Bad)\n                .code(ResponseCode::Parse),\n        }\n    }\n}\n\nimpl<T: CommandParser> Default for Receiver<T> {\n    fn default() -> Self {\n        Self {\n            buf: Default::default(),\n            request: Default::default(),\n            state: State::Start,\n            start_state: State::Start,\n            max_request_size: 25 * 1024 * 1024,\n            current_request_size: 0,\n        }\n    }\n}\n\nimpl<T: CommandParser> Request<T> {\n    pub fn into_error(self, message: impl Into<trc::Value>) -> trc::Error {\n        trc::ImapEvent::Error\n            .ctx(trc::Key::Details, message)\n            .ctx(trc::Key::Id, CompactString::from_string_buffer(self.tag))\n    }\n\n    pub fn into_parse_error(self, message: impl Into<trc::Value>) -> trc::Error {\n        trc::ImapEvent::Error\n            .ctx(trc::Key::Details, message)\n            .ctx(trc::Key::Id, CompactString::from_string_buffer(self.tag))\n            .ctx(trc::Key::Code, ResponseCode::Parse)\n            .ctx(trc::Key::Type, ResponseType::Bad)\n    }\n}\n\npub(crate) fn bad(tag: impl Into<trc::Value>, message: impl Into<trc::Value>) -> trc::Error {\n    trc::ImapEvent::Error\n        .ctx(trc::Key::Details, message)\n        .ctx(trc::Key::Id, tag)\n        .ctx(trc::Key::Type, ResponseType::Bad)\n}\n\n/*\n\nastring         = 1*ASTRING-CHAR / string\n\nstring          = quoted / literal\n\nliteral         = \"{\" number64 [\"+\"] \"}\" CRLF *CHAR8\n\nquoted          = DQUOTE *QUOTED-CHAR DQUOTE\n\nASTRING-CHAR   = ATOM-CHAR / resp-specials\n\natom            = 1*ATOM-CHAR\n\nATOM-CHAR       = <any CHAR except atom-specials>\n\natom-specials   = \"(\" / \")\" / \"{\" / SP / CTL / list-wildcards /\n                  quoted-specials / resp-specials\n\nresp-specials   = \"]\"\n\nlist-wildcards  = \"%\" / \"*\"\n\nquoted-specials = DQUOTE / \"\\\"\n\nDQUOTE         =  %x22 ; \" (Double Quote)\n\n*/\n\n#[cfg(test)]\nmod tests {\n\n    use crate::Command;\n\n    use super::{Error, Receiver, Request, Token};\n\n    #[test]\n    fn receiver_parse_ok() {\n        let mut receiver = Receiver::new();\n\n        for (frames, expected_requests) in [\n            (\n                vec![\"abcd CAPABILITY\\r\\n\"],\n                vec![Request {\n                    tag: \"abcd\".into(),\n                    command: Command::Capability,\n                    tokens: vec![],\n                }],\n            ),\n            (\n                vec![\"A023 LO\", \"GOUT\\r\\n\"],\n                vec![Request {\n                    tag: \"A023\".into(),\n                    command: Command::Logout,\n                    tokens: vec![],\n                }],\n            ),\n            (\n                vec![\"  A001 AUTHENTICATE GSSAPI  \\r\\n\"],\n                vec![Request {\n                    tag: \"A001\".into(),\n                    command: Command::Authenticate,\n                    tokens: vec![Token::Argument(b\"GSSAPI\".to_vec())],\n                }],\n            ),\n            (\n                vec![\"A03   AUTHENTICATE \", \"PLAIN dGVzdAB0ZXN\", \"0AHRlc3Q=\\r\\n\"],\n                vec![Request {\n                    tag: \"A03\".into(),\n                    command: Command::Authenticate,\n                    tokens: vec![\n                        Token::Argument(b\"PLAIN\".to_vec()),\n                        Token::Argument(b\"dGVzdAB0ZXN0AHRlc3Q=\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"A003 CREATE owatagusiam/\\r\\n\"],\n                vec![Request {\n                    tag: \"A003\".into(),\n                    command: Command::Create,\n                    tokens: vec![Token::Argument(b\"owatagusiam/\".to_vec())],\n                }],\n            ),\n            (\n                vec![\"A682 LIST \\\"\\\" *\\r\\n\"],\n                vec![Request {\n                    tag: \"A682\".into(),\n                    command: Command::List,\n                    tokens: vec![Token::Nil, Token::Argument(b\"*\".to_vec())],\n                }],\n            ),\n            (\n                vec![\"A03 LIST () \\\"\\\" \\\"%\\\" RETURN (CHILDREN)\\r\\n\"],\n                vec![Request {\n                    tag: \"A03\".into(),\n                    command: Command::List,\n                    tokens: vec![\n                        Token::ParenthesisOpen,\n                        Token::ParenthesisClose,\n                        Token::Nil,\n                        Token::Argument(b\"%\".to_vec()),\n                        Token::Argument(b\"RETURN\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"CHILDREN\".to_vec()),\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\"A05 LIST (REMOTE SUBSCRIBED) \\\"\\\" \\\"*\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"A05\".into(),\n                    command: Command::List,\n                    tokens: vec![\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"REMOTE\".to_vec()),\n                        Token::Argument(b\"SUBSCRIBED\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Nil,\n                        Token::Argument(b\"*\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"a1 list \\\"\\\" (\\\"foo\\\")\\r\\n\"],\n                vec![Request {\n                    tag: \"a1\".into(),\n                    command: Command::List,\n                    tokens: vec![\n                        Token::Nil,\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"foo\".to_vec()),\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\"a3.1 LIST \\\"\\\" (% music/rock)\\r\\n\"],\n                vec![Request {\n                    tag: \"a3.1\".into(),\n                    command: Command::List,\n                    tokens: vec![\n                        Token::Nil,\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"%\".to_vec()),\n                        Token::Argument(b\"music/rock\".to_vec()),\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\"A01 LIST \\\"\\\" % RETURN (STATUS (MESSAGES UNSEEN))\\r\\n\"],\n                vec![Request {\n                    tag: \"A01\".into(),\n                    command: Command::List,\n                    tokens: vec![\n                        Token::Nil,\n                        Token::Argument(b\"%\".to_vec()),\n                        Token::Argument(b\"RETURN\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"STATUS\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"MESSAGES\".to_vec()),\n                        Token::Argument(b\"UNSEEN\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\" A01 LiSt \\\"\\\"  % RETURN ( STATUS ( MESSAGES UNSEEN ) ) \\r\\n\"],\n                vec![Request {\n                    tag: \"A01\".into(),\n                    command: Command::List,\n                    tokens: vec![\n                        Token::Nil,\n                        Token::Argument(b\"%\".to_vec()),\n                        Token::Argument(b\"RETURN\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"STATUS\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"MESSAGES\".to_vec()),\n                        Token::Argument(b\"UNSEEN\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\"A02 LIST (SUBSCRIBED RECURSIVEMATCH) \\\"\\\" % RETURN (STATUS (MESSAGES))\\r\\n\"],\n                vec![Request {\n                    tag: \"A02\".into(),\n                    command: Command::List,\n                    tokens: vec![\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"SUBSCRIBED\".to_vec()),\n                        Token::Argument(b\"RECURSIVEMATCH\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Nil,\n                        Token::Argument(b\"%\".to_vec()),\n                        Token::Argument(b\"RETURN\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"STATUS\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"MESSAGES\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\"A002 CREATE \\\"INBOX.Sent Mail\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"A002\".into(),\n                    command: Command::Create,\n                    tokens: vec![Token::Argument(b\"INBOX.Sent Mail\".to_vec())],\n                }],\n            ),\n            (\n                vec![\"A002 CREATE \\\"Maibox \\\\\\\"quo\\\\\\\\ted\\\\\\\" \\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"A002\".into(),\n                    command: Command::Create,\n                    tokens: vec![Token::Argument(b\"Maibox \\\"quo\\\\ted\\\" \".to_vec())],\n                }],\n            ),\n            (\n                vec![\"A004 COPY 2:4 meeting\\r\\n\"],\n                vec![Request {\n                    tag: \"A004\".into(),\n                    command: Command::Copy(false),\n                    tokens: vec![\n                        Token::Argument(b\"2:4\".to_vec()),\n                        Token::Argument(b\"meeting\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\n                    \"A282 SEARCH RETURN (MIN COU\",\n                    \"NT) FLAGGED SINCE 1-Feb-1994 \",\n                    \"NOT FROM \\\"Smith\\\"\\r\\n\",\n                ],\n                vec![Request {\n                    tag: \"A282\".into(),\n                    command: Command::Search(false),\n                    tokens: vec![\n                        Token::Argument(b\"RETURN\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"MIN\".to_vec()),\n                        Token::Argument(b\"COUNT\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Argument(b\"FLAGGED\".to_vec()),\n                        Token::Argument(b\"SINCE\".to_vec()),\n                        Token::Argument(b\"1-Feb-1994\".to_vec()),\n                        Token::Argument(b\"NOT\".to_vec()),\n                        Token::Argument(b\"FROM\".to_vec()),\n                        Token::Argument(b\"Smith\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"F284 UID STORE $ +FLAGS.Silent (\\\\Deleted)\\r\\n\"],\n                vec![Request {\n                    tag: \"F284\".into(),\n                    command: Command::Store(true),\n                    tokens: vec![\n                        Token::Argument(b\"$\".to_vec()),\n                        Token::Argument(b\"+FLAGS.Silent\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"\\\\Deleted\".to_vec()),\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\"A654 FETCH 2:4 (FLAGS BODY[HEADER.FIELDS (DATE FROM)])\\r\\n\"],\n                vec![Request {\n                    tag: \"A654\".into(),\n                    command: Command::Fetch(false),\n                    tokens: vec![\n                        Token::Argument(b\"2:4\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"FLAGS\".to_vec()),\n                        Token::Argument(b\"BODY\".to_vec()),\n                        Token::BracketOpen,\n                        Token::Argument(b\"HEADER\".to_vec()),\n                        Token::Dot,\n                        Token::Argument(b\"FIELDS\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"DATE\".to_vec()),\n                        Token::Argument(b\"FROM\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::BracketClose,\n                        Token::ParenthesisClose,\n                    ],\n                }],\n            ),\n            (\n                vec![\n                    \"B283 UID SEARCH RETURN (SAVE) CHARSET \",\n                    \"KOI8-R (OR $ 1,3000:3021) TEXT \\\"hello world\\\"\\r\\n\",\n                ],\n                vec![Request {\n                    tag: \"B283\".into(),\n                    command: Command::Search(true),\n                    tokens: vec![\n                        Token::Argument(b\"RETURN\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"SAVE\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Argument(b\"CHARSET\".to_vec()),\n                        Token::Argument(b\"KOI8-R\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"OR\".to_vec()),\n                        Token::Argument(b\"$\".to_vec()),\n                        Token::Argument(b\"1,3000:3021\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Argument(b\"TEXT\".to_vec()),\n                        Token::Argument(b\"hello world\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\n                    \"P283 SEARCH CHARSET UTF-8 (OR $ 1,3000:3021) \",\n                    \"TEXT {8+}\\r\\nмать\\r\\n\",\n                ],\n                vec![Request {\n                    tag: \"P283\".into(),\n                    command: Command::Search(false),\n                    tokens: vec![\n                        Token::Argument(b\"CHARSET\".to_vec()),\n                        Token::Argument(b\"UTF-8\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"OR\".to_vec()),\n                        Token::Argument(b\"$\".to_vec()),\n                        Token::Argument(b\"1,3000:3021\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Argument(b\"TEXT\".to_vec()),\n                        Token::Argument(\"мать\".to_string().into_bytes()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"A001 LOGIN {11}\\r\\n\", \"FRED FOOBAR {7}\\r\\n\", \"fat man\\r\\n\"],\n                vec![Request {\n                    tag: \"A001\".into(),\n                    command: Command::Login,\n                    tokens: vec![\n                        Token::Argument(b\"FRED FOOBAR\".to_vec()),\n                        Token::Argument(b\"fat man\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"TAG3 CREATE \\\"Test-ąęć-Test\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"TAG3\".into(),\n                    command: Command::Create,\n                    tokens: vec![Token::Argument(\"Test-ąęć-Test\".as_bytes().to_vec())],\n                }],\n            ),\n            (\n                vec![\"abc LOGIN {0}\\r\\n\", \"\\r\\n\"],\n                vec![Request {\n                    tag: \"abc\".into(),\n                    command: Command::Login,\n                    tokens: vec![Token::Nil],\n                }],\n            ),\n            (\n                vec![\"abc LOGIN {0+}\\r\\n\\r\\n\"],\n                vec![Request {\n                    tag: \"abc\".into(),\n                    command: Command::Login,\n                    tokens: vec![Token::Nil],\n                }],\n            ),\n            (\n                vec![\n                    \"A003 APPEND saved-messages (\\\\Seen) {297+}\\r\\n\",\n                    \"Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)\\r\\n\",\n                    \"From: Fred Foobar <foobar@example.com>\\r\\n\",\n                    \"Subject: afternoon meeting\\r\\n\",\n                    \"To: mooch@example.com\\r\\n\",\n                    \"Message-Id: <B27397-0100000@example.com>\\r\\n\",\n                    \"MIME-Version: 1.0\\r\\n\",\n                    \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\",\n                    \"\\r\\n\",\n                    \"Hello Joe, do you think we can meet at 3:30 tomorrow?\\r\\n\\r\\n\",\n                ],\n                vec![Request {\n                    tag: \"A003\".into(),\n                    command: Command::Append,\n                    tokens: vec![\n                        Token::Argument(b\"saved-messages\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"\\\\Seen\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Argument(\n                            concat!(\n                                \"Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)\\r\\n\",\n                                \"From: Fred Foobar <foobar@example.com>\\r\\n\",\n                                \"Subject: afternoon meeting\\r\\n\",\n                                \"To: mooch@example.com\\r\\n\",\n                                \"Message-Id: <B27397-0100000@example.com>\\r\\n\",\n                                \"MIME-Version: 1.0\\r\\n\",\n                                \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\",\n                                \"\\r\\n\",\n                                \"Hello Joe, do you think we can meet at 3:30 tomorrow?\\r\\n\"\n                            )\n                            .as_bytes()\n                            .to_vec(),\n                        ),\n                    ],\n                }],\n            ),\n            (\n                vec![\n                    \"A003 APPEND saved-messages (\\\\Seen) {326}\\r\\n\",\n                    \"Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)\\r\\n\",\n                    \"From: Fred Foobar <foobar@Blurdybloop.example>\\r\\n\",\n                    \"Subject: afternoon meeting\\r\\n\",\n                    \"To: mooch@owatagu.siam.edu.example\\r\\n\",\n                    \"Message-Id: <B27397-0100000@Blurdybloop.example>\\r\\n\",\n                    \"MIME-Version: 1.0\\r\\n\",\n                    \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\",\n                    \"\\r\\n\",\n                    \"Hello Joe, do you think we can meet at 3:30 tomorrow?\\r\\n\\r\\n\",\n                ],\n                vec![Request {\n                    tag: \"A003\".into(),\n                    command: Command::Append,\n                    tokens: vec![\n                        Token::Argument(b\"saved-messages\".to_vec()),\n                        Token::ParenthesisOpen,\n                        Token::Argument(b\"\\\\Seen\".to_vec()),\n                        Token::ParenthesisClose,\n                        Token::Argument(\n                            concat!(\n                                \"Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)\\r\\n\",\n                                \"From: Fred Foobar <foobar@Blurdybloop.example>\\r\\n\",\n                                \"Subject: afternoon meeting\\r\\n\",\n                                \"To: mooch@owatagu.siam.edu.example\\r\\n\",\n                                \"Message-Id: <B27397-0100000@Blurdybloop.example>\\r\\n\",\n                                \"MIME-Version: 1.0\\r\\n\",\n                                \"Content-Type: TEXT/PLAIN; CHARSET=US-ASCII\\r\\n\",\n                                \"\\r\\n\",\n                                \"Hello Joe, do you think we can meet at 3:30 tomorrow?\\r\\n\",\n                            )\n                            .as_bytes()\n                            .to_vec(),\n                        ),\n                    ],\n                }],\n            ),\n            (\n                vec![\"001 NOOP\\r\\n002 CAPABILITY\\r\\nabc LOGIN hello world\\r\\n\"],\n                vec![\n                    Request {\n                        tag: \"001\".into(),\n                        command: Command::Noop,\n                        tokens: vec![],\n                    },\n                    Request {\n                        tag: \"002\".into(),\n                        command: Command::Capability,\n                        tokens: vec![],\n                    },\n                    Request {\n                        tag: \"abc\".into(),\n                        command: Command::Login,\n                        tokens: vec![\n                            Token::Argument(b\"hello\".to_vec()),\n                            Token::Argument(b\"world\".to_vec()),\n                        ],\n                    },\n                ],\n            ),\n        ] {\n            let mut requests = Vec::new();\n            for frame in &frames {\n                let mut bytes = frame.as_bytes().iter();\n                loop {\n                    match receiver.parse(&mut bytes) {\n                        Ok(request) => requests.push(request),\n                        Err(Error::NeedsMoreData | Error::NeedsLiteral { .. }) => break,\n                        Err(err) => panic!(\"{:?} for frames {:#?}\", err, frames),\n                    }\n                }\n            }\n            assert_eq!(requests, expected_requests, \"{:#?}\", frames);\n        }\n    }\n\n    #[test]\n    fn receiver_parse_invalid() {\n        let mut receiver = Receiver::<Command>::new();\n        for invalid in [\n            //\"\\r\\n\",\n            //\"  \\r \\n\",\n            \"a001\\r\\n\",\n            \"a001 unknown\\r\\n\",\n            \"a001 login {abc}\\r\\n\",\n            \"a001 login {+30}\\r\\n\",\n            \"a001 login {30} junk\\r\\n\",\n        ] {\n            match receiver.parse(&mut invalid.as_bytes().iter()) {\n                Err(Error::Error { .. }) => {}\n                result => panic!(\"Expecter error, got: {:?}\", result),\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/imap-proto/src/utf7.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n// Ported from https://github.com/jstedfast/MailKit/blob/master/MailKit/Net/Imap/ImapEncoding.cs\n// Author: Jeffrey Stedfast <jestedfa@microsoft.com>\n\nstatic UTF_7_RANK: &[u8] = &[\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 62, 63, 255, 255, 255, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 255,\n    255, 255, 255, 255, 255, 255, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,\n    19, 20, 21, 22, 23, 24, 25, 255, 255, 255, 255, 255, 255, 26, 27, 28, 29, 30, 31, 32, 33, 34,\n    35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 255, 255, 255, 255, 255,\n];\n\nstatic UTF_7_MAP: &[u8] = b\"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+,\";\n\npub fn utf7_decode(text: &str) -> Option<String> {\n    let mut bytes: Vec<u16> = Vec::with_capacity(text.len());\n    let mut bits = 0;\n    let mut v: u32 = 0;\n    let mut shifted = false;\n    let mut text = text.chars().peekable();\n\n    while let Some(ch) = text.next() {\n        if shifted {\n            if ch == '-' {\n                shifted = false;\n                bits = 0;\n                v = 0;\n            } else if ch as usize > 127 {\n                return None;\n            } else {\n                let rank = *UTF_7_RANK.get(ch as usize)?;\n\n                if rank == 0xff {\n                    return None;\n                }\n\n                v = (v << 6) | rank as u32;\n                bits += 6;\n\n                if bits >= 16 {\n                    bytes.push(((v >> (bits - 16)) & 0xffff) as u16);\n                    bits -= 16;\n                }\n            }\n        } else if ch == '&' {\n            match text.peek() {\n                Some('-') => {\n                    bytes.push(b'&' as u16);\n                    text.next();\n                }\n                Some(_) => {\n                    shifted = true;\n                }\n                None => {\n                    bytes.push(ch as u16);\n                }\n            }\n        } else {\n            bytes.push(ch as u16);\n        }\n    }\n\n    String::from_utf16(&bytes).ok()\n}\n\npub fn utf7_encode(text: &str) -> String {\n    let mut result = String::with_capacity(text.len());\n    let mut shifted = false;\n    let mut bits = 0;\n    let mut u: u32 = 0;\n\n    for ch in text.encode_utf16() {\n        if (0x20..0x7f).contains(&ch) {\n            if shifted {\n                if bits > 0 {\n                    result.push(char::from(UTF_7_MAP[((u << (6 - bits)) & 0x3f) as usize]));\n                }\n                result.push('-');\n                shifted = false;\n                bits = 0;\n            }\n\n            if ch == 0x26 {\n                result.push_str(\"&-\");\n            } else {\n                result.push((ch as u8) as char);\n            }\n        } else {\n            if !shifted {\n                result.push('&');\n                shifted = true;\n            }\n\n            u = (u << 16) | ch as u32;\n            bits += 16;\n\n            while bits >= 6 {\n                result.push(char::from(UTF_7_MAP[((u >> (bits - 6)) & 0x3f) as usize]));\n                bits -= 6;\n            }\n        }\n    }\n\n    if shifted {\n        if bits > 0 {\n            result.push(char::from(UTF_7_MAP[((u << (6 - bits)) & 0x3f) as usize]));\n        }\n        result.push('-');\n    }\n\n    result\n}\n\n#[inline(always)]\npub fn utf7_maybe_decode(text: String, is_utf8: bool) -> String {\n    if is_utf8 {\n        text\n    } else {\n        utf7_decode(&text).unwrap_or(text)\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    #[test]\n    fn utf7_decode() {\n        for (input, expected_result) in [\n            (\"~peter/mail/&U,BTFw-/&ZeVnLIqe-\", \"~peter/mail/台北/日本語\"),\n            (\"&U,BTF2XlZyyKng-\", \"台北日本語\"),\n            (\"Hello, World&ACE-\", \"Hello, World!\"),\n            (\"Hi Mom -&Jjo--!\", \"Hi Mom -☺-!\"),\n            (\"&ZeVnLIqe-\", \"日本語\"),\n            (\"Item 3 is &AKM-1.\", \"Item 3 is £1.\"),\n            (\"Plus minus &- -&- &--\", \"Plus minus & -& &-\"),\n            (\n                \"&APw-ber ihre mi&AN8-liche Lage&ADs- &ACI-wir\",\n                \"über ihre mißliche Lage; \\\"wir\",\n            ),\n            (\n                concat!(\n                    \"&ACI-The sayings of Confucius,&ACI- James R. Ware, trans.  &U,BTFw-:\\n\",\n                    \"&ZYeB9FH6ckh5Pg-, 1980.\\n\",\n                    \"&Vttm+E6UfZM-, &W4tRQ066bOg-, &UxdOrA-:  &Ti1XC2b4Xpc-, 1990.\"\n                ),\n                concat!(\n                    \"\\\"The sayings of Confucius,\\\" James R. Ware, trans.  台北:\\n\",\n                    \"文致出版社, 1980.\\n\",\n                    \"四書五經, 宋元人注, 北京:  中國書店, 1990.\"\n                ),\n            ),\n            (\"Test-ąęć-Test\", \"Test-ąęć-Test\"),\n            (r#\"&A8g- \"&A9QD1APUA9gD3APcA-+\"\"#, \"ψ \\\"ϔϔϔϘϜϜ+\\\"\"),\n        ] {\n            assert_eq!(\n                super::utf7_decode(input).expect(input),\n                expected_result,\n                \"while decoding {:?}\",\n                input\n            );\n        }\n    }\n\n    #[test]\n    fn utf7_encode() {\n        for (expected_result, input) in [\n            (\"~peter/mail/&U,BTFw-/&ZeVnLIqe-\", \"~peter/mail/台北/日本語\"),\n            (\"&U,BTF2XlZyyKng-\", \"台北日本語\"),\n            (\"Hi Mom -&Jjo--!\", \"Hi Mom -☺-!\"),\n            (\"&ZeVnLIqe-\", \"日本語\"),\n            (\"Item 3 is &AKM-1.\", \"Item 3 is £1.\"),\n            (\"Plus minus &- -&- &--\", \"Plus minus & -& &-\"),\n            (\"&VMhUyNg93gQ-\", \"哈哈😄\"),\n        ] {\n            assert_eq!(\n                super::utf7_encode(input),\n                expected_result,\n                \"while encoding {:?}\",\n                expected_result\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/Cargo.toml",
    "content": "[package]\nname = \"jmap\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nstore = { path = \"../store\" }\nnlp = { path = \"../nlp\" }\nhttp_proto = { path = \"../http-proto\" }\njmap_proto = { path = \"../jmap-proto\" }\ntypes = { path = \"../types\" }\nsmtp = { path =  \"../smtp\" }\nutils = { path =  \"../utils\" }\ncommon = { path =  \"../common\" }\nservices = { path =  \"../services\" }\ndirectory = { path =  \"../directory\" }\ntrc = { path = \"../trc\" }\nspam-filter = { path = \"../spam-filter\" }\nemail = { path = \"../email\" }\ngroupware = { path = \"../groupware\" }\ncalcard = { version = \"0.3\" }\nsmtp-proto = { version = \"0.2\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] } \nmail-builder = { version = \"0.4\" }\nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nmail-auth = { version = \"0.7.1\", features = [\"generate\"] }\nsieve-rs = { version = \"0.7\", features = [\"rkyv\"] } \njmap-tools = { version = \"0.1\", features = [\"rkyv\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\nhyper-util = { version = \"0.1.1\", features = [\"tokio\"] }\nhttp-body-util = \"0.1.0\"\ntokio = { version = \"1.47\", features = [\"rt\"] }\nfutures-util = \"0.3.28\"\nasync-stream = \"0.3.5\"\nbase64 = \"0.22\"\np256 = { version = \"0.13\", features = [\"ecdh\"] }\nhkdf = \"0.12.3\"\nsha1 = \"0.10\"\nsha2 = \"0.10\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\"]}\ntokio-tungstenite = \"0.28\"\ntungstenite = \"0.28\"\nchrono = \"0.4\"\nrand = \"0.9.0\"\npkcs8 = { version = \"0.10.2\", features = [\"alloc\", \"std\"] }\nlz4_flex = { version = \"0.12\", default-features = false }\naes-gcm = \"0.10.1\"\naes-gcm-siv = \"0.11.1\"\nrsa = \"0.9.2\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\nhashify = \"0.2\"\n\n[features]\ntest_mode = []\nenterprise = []\n"
  },
  {
    "path": "crates/jmap/src/addressbook/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::acl::JmapRights, changes::state::JmapCacheState};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse groupware::{cache::GroupwareCache, contact::AddressBook};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::addressbook::{self, AddressBookProperty, AddressBookValue},\n};\nuse jmap_tools::{Map, Value};\nuse store::{ValueKey, roaring::RoaringBitmap, write::{AlignedBytes, Archive, ValueClass}};\nuse trc::AddContext;\nuse types::{\n    acl::{Acl, AclGrant},\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n};\n\npub trait AddressBookGet: Sync + Send {\n    fn address_book_get(\n        &self,\n        request: GetRequest<addressbook::AddressBook>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<addressbook::AddressBook>>> + Send;\n}\n\nimpl AddressBookGet for Server {\n    async fn address_book_get(\n        &self,\n        mut request: GetRequest<addressbook::AddressBook>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<addressbook::AddressBook>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::Description,\n            AddressBookProperty::SortOrder,\n            AddressBookProperty::IsDefault,\n            AddressBookProperty::IsSubscribed,\n            AddressBookProperty::MyRights,\n        ]);\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await?;\n        let address_book_ids = if access_token.is_member(account_id) {\n            cache.document_ids(true).collect::<RoaringBitmap>()\n        } else {\n            cache.shared_containers(access_token, [Acl::Read, Acl::ReadItems], true)\n        };\n        let default_address_book_id = self\n            .store()\n            .get_value::<u32>(ValueKey {\n                account_id,\n                collection: Collection::Principal.into(),\n                document_id: 0,\n                class: ValueClass::Property(PrincipalField::DefaultAddressBookId.into()),\n            })\n            .await\n            .caused_by(trc::location!())?\n            .or_else(|| {\n                if address_book_ids.len() == 1 {\n                    address_book_ids.iter().next()\n                } else {\n                    None\n                }\n            });\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            address_book_ids\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: cache.get_state(true).into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the address_book object\n            let document_id = id.document_id();\n            if !address_book_ids.contains(document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n            let _address_book = if let Some(address_book) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::AddressBook,\n                    document_id,\n                ))\n                .await?\n            {\n                address_book\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n            let address_book = _address_book\n                .unarchive::<AddressBook>()\n                .caused_by(trc::location!())?;\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                match property {\n                    AddressBookProperty::Id => {\n                        result.insert_unchecked(AddressBookProperty::Id, AddressBookValue::Id(id));\n                    }\n                    AddressBookProperty::Name => {\n                        result.insert_unchecked(\n                            AddressBookProperty::Name,\n                            address_book.preferences(access_token).name.to_string(),\n                        );\n                    }\n                    AddressBookProperty::Description => {\n                        result.insert_unchecked(\n                            AddressBookProperty::Description,\n                            address_book\n                                .preferences(access_token)\n                                .description\n                                .as_ref()\n                                .map(|v| v.to_string()),\n                        );\n                    }\n                    AddressBookProperty::SortOrder => {\n                        result.insert_unchecked(\n                            AddressBookProperty::SortOrder,\n                            address_book\n                                .preferences(access_token)\n                                .sort_order\n                                .to_native(),\n                        );\n                    }\n                    AddressBookProperty::IsDefault => {\n                        result.insert_unchecked(\n                            AddressBookProperty::IsDefault,\n                            default_address_book_id == Some(document_id),\n                        );\n                    }\n                    AddressBookProperty::IsSubscribed => {\n                        result.insert_unchecked(\n                            AddressBookProperty::IsSubscribed,\n                            address_book\n                                .subscribers\n                                .iter()\n                                .any(|account_id| *account_id == access_token.primary_id()),\n                        );\n                    }\n                    AddressBookProperty::ShareWith => {\n                        result.insert_unchecked(\n                            AddressBookProperty::ShareWith,\n                            JmapRights::share_with::<addressbook::AddressBook>(\n                                account_id,\n                                access_token,\n                                &address_book\n                                    .acls\n                                    .iter()\n                                    .map(AclGrant::from)\n                                    .collect::<Vec<_>>(),\n                            ),\n                        );\n                    }\n                    AddressBookProperty::MyRights => {\n                        result.insert_unchecked(\n                            AddressBookProperty::MyRights,\n                            if access_token.is_shared(account_id) {\n                                JmapRights::rights::<addressbook::AddressBook>(\n                                    address_book.acls.effective_acl(access_token),\n                                )\n                            } else {\n                                JmapRights::all_rights::<addressbook::AddressBook>()\n                            },\n                        );\n                    }\n                    property => {\n                        result.insert_unchecked(property.clone(), Value::Null);\n                    }\n                }\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/addressbook/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/addressbook/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::api::acl::{JmapAcl, JmapRights};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse groupware::{\n    DestroyArchive,\n    cache::GroupwareCache,\n    contact::{AddressBook, AddressBookPreferences, ContactCard},\n};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::set::{SetRequest, SetResponse},\n    object::addressbook::{self, AddressBookProperty, AddressBookValue},\n    request::{IntoValid, reference::MaybeIdReference},\n    types::state::State,\n};\nuse jmap_tools::{JsonPointerItem, Key, Value};\nuse rand::{Rng, distr::Alphanumeric};\nuse store::{\n    SerializeInfallible, ValueKey,\n    ahash::AHashSet,\n    write::{AlignedBytes, Archive, BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n};\n\npub trait AddressBookSet: Sync + Send {\n    fn address_book_set(\n        &self,\n        request: SetRequest<'_, addressbook::AddressBook>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<SetResponse<addressbook::AddressBook>>> + Send;\n}\n\nimpl AddressBookSet for Server {\n    async fn address_book_set(\n        &self,\n        mut request: SetRequest<'_, addressbook::AddressBook>,\n        access_token: &AccessToken,\n        _session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<addressbook::AddressBook>> {\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n        let is_shared = access_token.is_shared(account_id);\n        let mut set_default = None;\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        'create: for (id, object) in request.unwrap_create() {\n            if is_shared {\n                response.not_created.append(\n                    id,\n                    SetError::forbidden()\n                        .with_description(\"Cannot create address books in a shared account.\"),\n                );\n                continue 'create;\n            }\n\n            let mut address_book = AddressBook {\n                name: rand::rng()\n                    .sample_iter(Alphanumeric)\n                    .take(10)\n                    .map(char::from)\n                    .collect::<String>(),\n                preferences: vec![AddressBookPreferences {\n                    account_id,\n                    name: \"Address Book\".to_string(),\n                    ..Default::default()\n                }],\n                ..Default::default()\n            };\n\n            // Process changes\n            if let Err(err) = update_address_book(object, &mut address_book, access_token) {\n                response.not_created.append(id, err);\n                continue 'create;\n            }\n\n            // Validate ACLs\n            if !address_book.acls.is_empty() {\n                if let Err(err) = self.acl_validate(&address_book.acls).await {\n                    response.not_created.append(id, err.into());\n                    continue 'create;\n                }\n\n                self.refresh_acls(&address_book.acls, None).await;\n            }\n\n            // Insert record\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::AddressBook, 1)\n                .await\n                .caused_by(trc::location!())?;\n            address_book\n                .insert(access_token, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?;\n\n            if let Some(MaybeIdReference::Reference(id_ref)) =\n                &request.arguments.on_success_set_is_default\n                && id_ref == &id\n            {\n                set_default = Some(document_id);\n            }\n\n            response.created(id, document_id);\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain address book\n            let document_id = id.document_id();\n            let address_book_ = if let Some(address_book_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::AddressBook,\n                    document_id,\n                ))\n                .await?\n            {\n                address_book_\n            } else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n            let address_book = address_book_\n                .to_unarchived::<AddressBook>()\n                .caused_by(trc::location!())?;\n            let mut new_address_book = address_book\n                .deserialize::<AddressBook>()\n                .caused_by(trc::location!())?;\n\n            // Apply changes\n            let has_acl_changes =\n                match update_address_book(object, &mut new_address_book, access_token) {\n                    Ok(has_acl_changes_) => has_acl_changes_,\n                    Err(err) => {\n                        response.not_updated.append(id, err);\n                        continue 'update;\n                    }\n                };\n\n            // Validate ACL\n            if is_shared {\n                let acl = address_book.inner.acls.effective_acl(access_token);\n                if !acl.contains(Acl::Modify) || (has_acl_changes && !acl.contains(Acl::Share)) {\n                    response.not_updated.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to modify this address book.\"),\n                    );\n                    continue 'update;\n                }\n            }\n            if has_acl_changes {\n                if let Err(err) = self.acl_validate(&new_address_book.acls).await {\n                    response.not_updated.append(id, err.into());\n                    continue 'update;\n                }\n                self.refresh_archived_acls(\n                    &new_address_book.acls,\n                    address_book.inner.acls.as_slice(),\n                )\n                .await;\n            }\n\n            // Update record\n            new_address_book\n                .update(\n                    access_token,\n                    address_book,\n                    account_id,\n                    document_id,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        let mut reset_default_address_book = false;\n        if !will_destroy.is_empty() {\n            let mut destroy_children = AHashSet::new();\n            let mut destroy_parents = AHashSet::new();\n            let default_address_book_id = self\n                .store()\n                .get_value::<u32>(ValueKey {\n                    account_id,\n                    collection: Collection::Principal.into(),\n                    document_id: 0,\n                    class: ValueClass::Property(PrincipalField::DefaultAddressBookId.into()),\n                })\n                .await\n                .caused_by(trc::location!())?;\n\n            let on_destroy_remove_contents = request\n                .arguments\n                .on_destroy_remove_contents\n                .unwrap_or(false);\n\n            for id in will_destroy {\n                let document_id = id.document_id();\n\n                if !cache.has_container_id(&document_id) {\n                    response.not_destroyed.append(id, SetError::not_found());\n                    continue;\n                };\n\n                let Some(address_book_) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::AddressBook,\n                        document_id,\n                    ))\n                    .await\n                    .caused_by(trc::location!())?\n                else {\n                    response.not_destroyed.append(id, SetError::not_found());\n                    continue;\n                };\n\n                let address_book = address_book_\n                    .to_unarchived::<AddressBook>()\n                    .caused_by(trc::location!())?;\n\n                // Validate ACLs\n                if is_shared\n                    && !address_book\n                        .inner\n                        .acls\n                        .effective_acl(access_token)\n                        .contains_all([Acl::Delete, Acl::RemoveItems].into_iter())\n                {\n                    response.not_destroyed.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to delete this address book.\"),\n                    );\n                    continue;\n                }\n\n                // Obtain children ids\n                let children_ids = cache.children_ids(document_id).collect::<Vec<_>>();\n                if !children_ids.is_empty() && !on_destroy_remove_contents {\n                    response\n                        .not_destroyed\n                        .append(id, SetError::address_book_has_contents());\n                    continue;\n                }\n                destroy_children.extend(children_ids.iter().copied());\n                destroy_parents.insert(document_id);\n\n                // Delete record\n                DestroyArchive(address_book)\n                    .delete(access_token, account_id, document_id, None, &mut batch)\n                    .caused_by(trc::location!())?;\n\n                if default_address_book_id == Some(document_id) {\n                    reset_default_address_book = true;\n                }\n\n                response.destroyed.push(id);\n            }\n\n            // Delete children\n            if !destroy_children.is_empty() {\n                for document_id in destroy_children {\n                    if let Some(card_) = self\n                        .store()\n                        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                            account_id,\n                            Collection::ContactCard,\n                            document_id,\n                        ))\n                        .await?\n                    {\n                        let card = card_\n                            .to_unarchived::<ContactCard>()\n                            .caused_by(trc::location!())?;\n\n                        if card\n                            .inner\n                            .names\n                            .iter()\n                            .all(|n| destroy_parents.contains(&n.parent_id.to_native()))\n                        {\n                            // Card only belongs to address books being deleted, delete it\n                            DestroyArchive(card).delete_all(\n                                access_token,\n                                account_id,\n                                document_id,\n                                &mut batch,\n                            )?;\n                        } else {\n                            // Unlink addressbook id from card\n                            let mut new_card = card\n                                .deserialize::<ContactCard>()\n                                .caused_by(trc::location!())?;\n                            new_card\n                                .names\n                                .retain(|n| !destroy_parents.contains(&n.parent_id));\n                            new_card.update(\n                                access_token,\n                                card,\n                                account_id,\n                                document_id,\n                                &mut batch,\n                            )?;\n                        }\n                    }\n                }\n            }\n        }\n\n        // Set default address book\n        if let Some(MaybeIdReference::Id(id)) = &request.arguments.on_success_set_is_default {\n            set_default = Some(id.document_id());\n        }\n        if let Some(default_address_book_id) = set_default {\n            if response.not_created.is_empty()\n                && response.not_updated.is_empty()\n                && response.not_destroyed.is_empty()\n            {\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .set(\n                        PrincipalField::DefaultAddressBookId,\n                        default_address_book_id.serialize(),\n                    );\n            }\n        } else if reset_default_address_book {\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .clear(PrincipalField::DefaultAddressBookId);\n        }\n\n        // Write changes\n        if !batch.is_empty()\n            && let Ok(change_id) = self\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?\n                .last_change_id(account_id)\n        {\n            self.notify_task_queue();\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n}\n\nfn update_address_book(\n    updates: Value<'_, AddressBookProperty, AddressBookValue>,\n    address_book: &mut AddressBook,\n    access_token: &AccessToken,\n) -> Result<bool, SetError<AddressBookProperty>> {\n    let mut has_acl_changes = false;\n\n    for (property, value) in updates.into_expanded_object() {\n        let Key::Property(property) = property else {\n            return Err(SetError::invalid_properties()\n                .with_property(property.to_owned())\n                .with_description(\"Invalid property.\"));\n        };\n\n        match (property, value) {\n            (AddressBookProperty::Name, Value::Str(value)) if (1..=255).contains(&value.len()) => {\n                address_book.preferences_mut(access_token).name = value.into_owned();\n            }\n            (AddressBookProperty::Description, Value::Str(value)) if value.len() < 255 => {\n                address_book.preferences_mut(access_token).description = value.into_owned().into();\n            }\n            (AddressBookProperty::Description, Value::Null) => {\n                address_book.preferences_mut(access_token).description = None;\n            }\n            (AddressBookProperty::SortOrder, Value::Number(value)) => {\n                address_book.preferences_mut(access_token).sort_order = value.cast_to_u64() as u32;\n            }\n            (AddressBookProperty::IsSubscribed, Value::Bool(subscribe)) => {\n                let account_id = access_token.primary_id();\n                if subscribe {\n                    if !address_book.subscribers.contains(&account_id) {\n                        address_book.subscribers.push(account_id);\n                    }\n                } else {\n                    address_book.subscribers.retain(|id| *id != account_id);\n                }\n            }\n            (AddressBookProperty::ShareWith, value) => {\n                address_book.acls = JmapRights::acl_set::<addressbook::AddressBook>(value)?;\n                has_acl_changes = true;\n            }\n            (AddressBookProperty::Pointer(pointer), value)\n                if matches!(\n                    pointer.first(),\n                    Some(JsonPointerItem::Key(Key::Property(\n                        AddressBookProperty::ShareWith\n                    )))\n                ) =>\n            {\n                let mut pointer = pointer.iter();\n                pointer.next();\n\n                address_book.acls = JmapRights::acl_patch::<addressbook::AddressBook>(\n                    std::mem::take(&mut address_book.acls),\n                    pointer,\n                    value,\n                )?;\n                has_acl_changes = true;\n            }\n            (property, _) => {\n                return Err(SetError::invalid_properties()\n                    .with_property(property.clone())\n                    .with_description(\"Field could not be set.\"));\n            }\n        }\n    }\n\n    // Validate name\n    if address_book.preferences(access_token).name.is_empty() {\n        return Err(SetError::invalid_properties()\n            .with_property(AddressBookProperty::Name)\n            .with_description(\"Missing name.\"));\n    }\n\n    Ok(has_acl_changes)\n}\n"
  },
  {
    "path": "crates/jmap/src/api/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse directory::backend::internal::manage::ManageDirectory;\nuse jmap_proto::{\n    error::set::SetError,\n    object::{JmapRight, JmapSharedObject},\n};\nuse jmap_tools::{JsonPointerIter, Key, Map, Property, Value};\nuse types::{\n    acl::{Acl, AclGrant},\n    id::Id,\n};\nuse utils::map::bitmap::Bitmap;\n\npub struct JmapRights;\n\nimpl JmapRights {\n    pub fn acl_set<T: JmapSharedObject>(\n        value: Value<'_, T::Property, T::Element>,\n    ) -> Result<Vec<AclGrant>, SetError<T::Property>>\n    where\n        Id: TryFrom<T::Property>,\n        T::Right: TryFrom<T::Property>,\n    {\n        let mut grants = Vec::new();\n\n        for (key, value) in value.into_expanded_object() {\n            let account_id = key\n                .try_into_property()\n                .and_then(|p| Id::try_from(p).ok())\n                .ok_or_else(|| {\n                    SetError::invalid_properties()\n                        .with_property(T::SHARE_WITH_PROPERTY)\n                        .with_description(\"Invalid account id.\")\n                })?\n                .document_id();\n\n            if !grants\n                .iter()\n                .any(|item: &AclGrant| item.account_id == account_id)\n            {\n                let acls = Self::map_acls::<T>(value)?;\n                if !acls.is_empty() {\n                    grants.push(AclGrant {\n                        account_id,\n                        grants: acls,\n                    });\n                }\n            }\n        }\n\n        Ok(grants)\n    }\n\n    pub fn acl_patch<T: JmapSharedObject>(\n        mut grants: Vec<AclGrant>,\n        mut path: JsonPointerIter<'_, T::Property>,\n        value: Value<'_, T::Property, T::Element>,\n    ) -> Result<Vec<AclGrant>, SetError<T::Property>>\n    where\n        Id: TryFrom<T::Property>,\n        T::Right: TryFrom<T::Property>,\n    {\n        let account_id = path\n            .next()\n            .and_then(|item| item.as_property_key())\n            .cloned()\n            .and_then(|p| Id::try_from(p).ok())\n            .ok_or_else(|| {\n                SetError::invalid_properties()\n                    .with_property(T::SHARE_WITH_PROPERTY)\n                    .with_description(\"Invalid account id.\")\n            })?\n            .document_id();\n\n        if let Some(right) = path.next() {\n            let is_set = match value {\n                Value::Bool(is_set) => is_set,\n                Value::Null => false,\n                _ => {\n                    return Err(SetError::invalid_properties()\n                        .with_property(T::SHARE_WITH_PROPERTY)\n                        .with_description(\"Invalid ACL value.\"));\n                }\n            };\n\n            let acl = right\n                .as_property_key()\n                .cloned()\n                .and_then(|p| T::Right::try_from(p).ok())\n                .ok_or_else(|| {\n                    SetError::invalid_properties()\n                        .with_property(T::SHARE_WITH_PROPERTY)\n                        .with_description(format!(\n                            \"Invalid permission {:?}.\",\n                            right.to_cow().unwrap_or_default()\n                        ))\n                })?\n                .to_acl()\n                .iter()\n                .copied();\n\n            if let Some(acl_item) = grants.iter_mut().find(|item| item.account_id == account_id) {\n                if is_set {\n                    acl_item.grants.insert_many(acl);\n                } else {\n                    acl_item.grants.insert_many(acl);\n                    if acl_item.grants.is_empty() {\n                        grants.retain(|item| item.account_id != account_id);\n                    }\n                }\n            } else if is_set {\n                grants.push(AclGrant {\n                    account_id,\n                    grants: Bitmap::from_iter(acl),\n                });\n            }\n        } else {\n            let acls = Self::map_acls::<T>(value)?;\n            if !acls.is_empty() {\n                if let Some(acl_item) = grants.iter_mut().find(|item| item.account_id == account_id)\n                {\n                    acl_item.grants = acls;\n                } else {\n                    grants.push(AclGrant {\n                        account_id,\n                        grants: acls,\n                    });\n                }\n            } else {\n                grants.retain(|item| item.account_id != account_id);\n            }\n        }\n\n        Ok(grants)\n    }\n\n    fn map_acls<T: JmapSharedObject>(\n        value: Value<'_, T::Property, T::Element>,\n    ) -> Result<Bitmap<Acl>, SetError<T::Property>>\n    where\n        Id: TryFrom<T::Property>,\n        T::Right: TryFrom<T::Property>,\n    {\n        let mut acls = Bitmap::new();\n\n        for key in value.into_expanded_boolean_set() {\n            acls.insert_many(\n                key.as_property()\n                    .and_then(|p| T::Right::try_from(p.clone()).ok())\n                    .ok_or_else(|| {\n                        SetError::invalid_properties()\n                            .with_property(T::SHARE_WITH_PROPERTY)\n                            .with_description(format!(\"Invalid permission {:?}.\", key.to_string()))\n                    })?\n                    .to_acl()\n                    .iter()\n                    .copied(),\n            );\n        }\n\n        Ok(acls)\n    }\n\n    pub fn all_rights<T: JmapSharedObject>() -> Value<'static, T::Property, T::Element> {\n        let rights = T::Right::all_rights();\n        let mut obj = Map::with_capacity(rights.len());\n\n        for right in rights {\n            obj.insert_unchecked(Key::Property((*right).into()), Value::Bool(true));\n        }\n\n        Value::Object(obj)\n    }\n\n    pub fn rights<T: JmapSharedObject>(\n        acls: Bitmap<Acl>,\n    ) -> Value<'static, T::Property, T::Element> {\n        let mut obj = Map::with_capacity(3);\n\n        for right in T::Right::all_rights() {\n            obj.insert_unchecked(\n                Key::Property((*right).into()),\n                Value::Bool(right.to_acl().iter().all(|acl| acls.contains(*acl))),\n            );\n        }\n\n        Value::Object(obj)\n    }\n\n    pub fn share_with<T: JmapSharedObject>(\n        account_id: u32,\n        access_token: &AccessToken,\n        grants: &[AclGrant],\n    ) -> Value<'static, T::Property, T::Element>\n    where\n        T::Property: From<Id>,\n    {\n        if access_token.is_member(account_id)\n            || grants.effective_acl(access_token).contains(Acl::Share)\n        {\n            let mut share_with = Map::with_capacity(grants.len());\n            for grant in grants {\n                share_with.insert_unchecked(\n                    Key::Property(Id::from(grant.account_id).into()),\n                    Self::rights::<T>(grant.grants),\n                );\n            }\n\n            Value::Object(share_with)\n        } else {\n            Value::Null\n        }\n    }\n}\n\npub trait JmapAcl {\n    fn acl_validate(\n        &self,\n        grants: &[AclGrant],\n    ) -> impl Future<Output = Result<(), ShareValidationError>> + Send;\n}\n\npub enum ShareValidationError {\n    MaxSharesExceeded(usize),\n    InvalidAccountId(Id),\n}\n\nimpl JmapAcl for Server {\n    async fn acl_validate(&self, grants: &[AclGrant]) -> Result<(), ShareValidationError> {\n        if grants.len() > self.core.groupware.max_shares_per_item {\n            return Err(ShareValidationError::MaxSharesExceeded(\n                self.core.groupware.max_shares_per_item,\n            ));\n        }\n\n        let principal_ids = self\n            .store()\n            .principal_ids(None, None)\n            .await\n            .unwrap_or_default();\n\n        for grant in grants {\n            if !principal_ids.contains(grant.account_id) {\n                return Err(ShareValidationError::InvalidAccountId(Id::from(\n                    grant.account_id,\n                )));\n            }\n        }\n\n        Ok(())\n    }\n}\n\nimpl<T: Property> From<ShareValidationError> for SetError<T> {\n    fn from(err: ShareValidationError) -> Self {\n        match err {\n            ShareValidationError::MaxSharesExceeded(max) => SetError::invalid_properties()\n                .with_description(format!(\n                    \"Maximum number of shares per item exceeded (max: {max})\"\n                )),\n            ShareValidationError::InvalidAccountId(id) => SetError::invalid_properties()\n                .with_description(format!(\"Account id {id} is invalid.\")),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/api/auth.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::auth::AccessToken;\nuse directory::Permission;\nuse jmap_proto::request::{\n    CopyRequestMethod, GetRequestMethod, ParseRequestMethod, QueryChangesRequestMethod,\n    QueryRequestMethod, RequestMethod, SetRequestMethod, method::MethodObject,\n};\nuse types::{collection::Collection, id::Id};\n\npub trait JmapAuthorization {\n    fn assert_is_member(&self, account_id: Id) -> trc::Result<&Self>;\n    fn assert_has_jmap_permission(\n        &self,\n        request: &RequestMethod,\n        object: MethodObject,\n    ) -> trc::Result<()>;\n    fn assert_has_access(&self, to_account_id: Id, to_collection: Collection)\n    -> trc::Result<&Self>;\n}\n\nimpl JmapAuthorization for AccessToken {\n    fn assert_is_member(&self, account_id: Id) -> trc::Result<&Self> {\n        if self.is_member(account_id.document_id()) {\n            Ok(self)\n        } else {\n            Err(trc::JmapEvent::Forbidden\n                .into_err()\n                .details(format!(\"You are not an owner of account {}\", account_id)))\n        }\n    }\n\n    fn assert_has_access(\n        &self,\n        to_account_id: Id,\n        to_collection: Collection,\n    ) -> trc::Result<&Self> {\n        if self.has_access(to_account_id.document_id(), to_collection) {\n            Ok(self)\n        } else {\n            Err(trc::JmapEvent::Forbidden.into_err().details(format!(\n                \"You do not have access to account {}\",\n                to_account_id\n            )))\n        }\n    }\n\n    fn assert_has_jmap_permission(\n        &self,\n        request: &RequestMethod,\n        object: MethodObject,\n    ) -> trc::Result<()> {\n        let permission = match request {\n            RequestMethod::Get(m) => match &m {\n                GetRequestMethod::Email(_) => Permission::JmapEmailGet,\n                GetRequestMethod::Mailbox(_) => Permission::JmapMailboxGet,\n                GetRequestMethod::Thread(_) => Permission::JmapThreadGet,\n                GetRequestMethod::Identity(_) => Permission::JmapIdentityGet,\n                GetRequestMethod::EmailSubmission(_) => Permission::JmapEmailSubmissionGet,\n                GetRequestMethod::PushSubscription(_) => Permission::JmapPushSubscriptionGet,\n                GetRequestMethod::Sieve(_) => Permission::JmapSieveScriptGet,\n                GetRequestMethod::VacationResponse(_) => Permission::JmapVacationResponseGet,\n                GetRequestMethod::Principal(_) => Permission::JmapPrincipalGet,\n                GetRequestMethod::Quota(_) => Permission::JmapQuotaGet,\n                GetRequestMethod::Blob(_) => Permission::JmapBlobGet,\n                GetRequestMethod::AddressBook(_) => Permission::JmapAddressBookGet,\n                GetRequestMethod::ContactCard(_) => Permission::JmapContactCardGet,\n                GetRequestMethod::FileNode(_) => Permission::JmapFileNodeGet,\n                GetRequestMethod::PrincipalAvailability(_) => {\n                    Permission::JmapPrincipalGetAvailability\n                }\n                GetRequestMethod::Calendar(_) => Permission::JmapCalendarGet,\n                GetRequestMethod::CalendarEvent(_) => Permission::JmapCalendarEventGet,\n                GetRequestMethod::CalendarEventNotification(_) => {\n                    Permission::JmapCalendarEventNotificationGet\n                }\n                GetRequestMethod::ParticipantIdentity(_) => Permission::JmapParticipantIdentityGet,\n                GetRequestMethod::ShareNotification(_) => Permission::JmapShareNotificationGet,\n            },\n            RequestMethod::Set(m) => match &m {\n                SetRequestMethod::Email(_) => Permission::JmapEmailSet,\n                SetRequestMethod::Mailbox(_) => Permission::JmapMailboxSet,\n                SetRequestMethod::Identity(_) => Permission::JmapIdentitySet,\n                SetRequestMethod::EmailSubmission(_) => Permission::JmapEmailSubmissionSet,\n                SetRequestMethod::PushSubscription(_) => Permission::JmapPushSubscriptionSet,\n                SetRequestMethod::Sieve(_) => Permission::JmapSieveScriptSet,\n                SetRequestMethod::VacationResponse(_) => Permission::JmapVacationResponseSet,\n                SetRequestMethod::AddressBook(_) => Permission::JmapAddressBookSet,\n                SetRequestMethod::ContactCard(_) => Permission::JmapContactCardSet,\n                SetRequestMethod::FileNode(_) => Permission::JmapFileNodeSet,\n                SetRequestMethod::ShareNotification(_) => Permission::JmapShareNotificationSet,\n                SetRequestMethod::Calendar(_) => Permission::JmapCalendarSet,\n                SetRequestMethod::CalendarEvent(_) => Permission::JmapCalendarEventSet,\n                SetRequestMethod::CalendarEventNotification(_) => {\n                    Permission::JmapCalendarEventNotificationSet\n                }\n                SetRequestMethod::ParticipantIdentity(_) => Permission::JmapParticipantIdentitySet,\n            },\n            RequestMethod::Changes(_) => match object {\n                MethodObject::Email => Permission::JmapEmailChanges,\n                MethodObject::Mailbox => Permission::JmapMailboxChanges,\n                MethodObject::Thread => Permission::JmapThreadChanges,\n                MethodObject::Identity => Permission::JmapIdentityChanges,\n                MethodObject::EmailSubmission => Permission::JmapEmailSubmissionChanges,\n                MethodObject::Quota => Permission::JmapQuotaChanges,\n                MethodObject::ContactCard => Permission::JmapContactCardChanges,\n                MethodObject::FileNode => Permission::JmapFileNodeChanges,\n                MethodObject::Calendar => Permission::JmapCalendarChanges,\n                MethodObject::CalendarEvent => Permission::JmapCalendarEventChanges,\n                MethodObject::CalendarEventNotification => {\n                    Permission::JmapCalendarEventNotificationChanges\n                }\n                MethodObject::ParticipantIdentity => Permission::JmapParticipantIdentityChanges,\n                MethodObject::ShareNotification => Permission::JmapShareNotificationChanges,\n                MethodObject::Principal => Permission::JmapPrincipalChanges,\n                MethodObject::Core\n                | MethodObject::Blob\n                | MethodObject::PushSubscription\n                | MethodObject::SearchSnippet\n                | MethodObject::VacationResponse\n                | MethodObject::SieveScript\n                | MethodObject::AddressBook => Permission::JmapEmailChanges,\n            },\n            RequestMethod::Copy(m) => match &m {\n                CopyRequestMethod::Email(_) => Permission::JmapEmailCopy,\n                CopyRequestMethod::Blob(_) => Permission::JmapBlobCopy,\n                CopyRequestMethod::ContactCard(_) => Permission::JmapContactCardCopy,\n                CopyRequestMethod::CalendarEvent(_) => Permission::JmapCalendarEventCopy,\n            },\n            RequestMethod::ImportEmail(_) => Permission::JmapEmailImport,\n            RequestMethod::Parse(m) => match &m {\n                ParseRequestMethod::Email(_) => Permission::JmapEmailParse,\n                ParseRequestMethod::ContactCard(_) => Permission::JmapContactCardParse,\n                ParseRequestMethod::CalendarEvent(_) => Permission::JmapCalendarEventParse,\n            },\n            RequestMethod::QueryChanges(m) => match m {\n                QueryChangesRequestMethod::Email(_) => Permission::JmapEmailQueryChanges,\n                QueryChangesRequestMethod::Mailbox(_) => Permission::JmapMailboxQueryChanges,\n                QueryChangesRequestMethod::EmailSubmission(_) => {\n                    Permission::JmapEmailSubmissionQueryChanges\n                }\n                QueryChangesRequestMethod::Sieve(_) => Permission::JmapSieveScriptQueryChanges,\n                QueryChangesRequestMethod::Principal(_) => Permission::JmapPrincipalQueryChanges,\n                QueryChangesRequestMethod::Quota(_) => Permission::JmapQuotaQueryChanges,\n                QueryChangesRequestMethod::ContactCard(_) => {\n                    Permission::JmapContactCardQueryChanges\n                }\n                QueryChangesRequestMethod::FileNode(_) => Permission::JmapFileNodeQueryChanges,\n                QueryChangesRequestMethod::CalendarEvent(_) => {\n                    Permission::JmapCalendarEventQueryChanges\n                }\n                QueryChangesRequestMethod::CalendarEventNotification(_) => {\n                    Permission::JmapCalendarEventNotificationQueryChanges\n                }\n                QueryChangesRequestMethod::ShareNotification(_) => {\n                    Permission::JmapShareNotificationQueryChanges\n                }\n            },\n            RequestMethod::Query(m) => match m {\n                QueryRequestMethod::Email(_) => Permission::JmapEmailQuery,\n                QueryRequestMethod::Mailbox(_) => Permission::JmapMailboxQuery,\n                QueryRequestMethod::EmailSubmission(_) => Permission::JmapEmailSubmissionQuery,\n                QueryRequestMethod::Sieve(_) => Permission::JmapSieveScriptQuery,\n                QueryRequestMethod::Principal(_) => Permission::JmapPrincipalQuery,\n                QueryRequestMethod::Quota(_) => Permission::JmapQuotaQuery,\n                QueryRequestMethod::ContactCard(_) => Permission::JmapContactCardQuery,\n                QueryRequestMethod::FileNode(_) => Permission::JmapFileNodeQuery,\n                QueryRequestMethod::CalendarEvent(_) => Permission::JmapCalendarEventQuery,\n                QueryRequestMethod::CalendarEventNotification(_) => {\n                    Permission::JmapCalendarEventNotificationQuery\n                }\n                QueryRequestMethod::ShareNotification(_) => Permission::JmapShareNotificationQuery,\n            },\n            RequestMethod::SearchSnippet(_) => Permission::JmapSearchSnippet,\n            RequestMethod::ValidateScript(_) => Permission::JmapSieveScriptValidate,\n            RequestMethod::LookupBlob(_) => Permission::JmapBlobLookup,\n            RequestMethod::UploadBlob(_) => Permission::JmapBlobUpload,\n            RequestMethod::Echo(_) => Permission::JmapEcho,\n            RequestMethod::Error(_) => return Ok(()),\n        };\n\n        if self.has_permission(permission) {\n            Ok(())\n        } else {\n            Err(trc::JmapEvent::Forbidden\n                .into_err()\n                .details(\"You are not authorized to perform this action\"))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/api/event_source.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::api::IntoPushObject;\nuse common::{LONG_1D_SLUMBER, Server, auth::AccessToken, ipc::PushNotification};\nuse http_body_util::{StreamBody, combinators::BoxBody};\nuse http_proto::*;\nuse hyper::{\n    StatusCode,\n    body::{Bytes, Frame},\n};\nuse jmap_proto::{response::status::PushObject, types::state::State};\nuse std::{future::Future, str::FromStr};\nuse std::{\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse types::{id::Id, type_state::DataType};\nuse utils::map::{bitmap::Bitmap, vec_map::VecMap};\n\nstruct Ping {\n    interval: Duration,\n    last_ping: Instant,\n    payload: Bytes,\n}\n\npub trait EventSourceHandler: Sync + Send {\n    fn handle_event_source(\n        &self,\n        req: HttpRequest,\n        access_token: Arc<AccessToken>,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl EventSourceHandler for Server {\n    async fn handle_event_source(\n        &self,\n        req: HttpRequest,\n        access_token: Arc<AccessToken>,\n    ) -> trc::Result<HttpResponse> {\n        // Parse query\n        let mut ping = 0;\n        let mut types = Bitmap::default();\n        let mut close_after_state = false;\n\n        for (key, value) in\n            http_proto::form_urlencoded::parse(req.uri().query().unwrap_or_default().as_bytes())\n        {\n            hashify::fnc_map!(key.as_bytes(),\n                \"types\" => {\n                    for type_state in value.split(',') {\n                        if type_state == \"*\" {\n                            types = Bitmap::all();\n                            break;\n                        } else if let Ok(type_state) = DataType::from_str(type_state) {\n                            types.insert(type_state);\n                        } else {\n                            return Err(trc::ResourceEvent::BadParameters.into_err());\n                        }\n                    }\n                },\n                \"closeafter\" => match value.as_ref() {\n                    \"state\" => {\n                        close_after_state = true;\n                    }\n                    \"no\" => {}\n                    _ => return Err(trc::ResourceEvent::BadParameters.into_err()),\n                },\n                \"ping\" => match value.parse::<u32>() {\n                    Ok(value) => {\n                        ping = value;\n                    }\n                    Err(_) => return Err(trc::ResourceEvent::BadParameters.into_err()),\n                },\n                _ => {}\n            );\n        }\n\n        let mut ping = if ping > 0 {\n            #[cfg(not(feature = \"test_mode\"))]\n            let interval = std::cmp::max(ping, 30) * 1000;\n            #[cfg(feature = \"test_mode\")]\n            let interval = ping * 1000;\n\n            Ping {\n                interval: Duration::from_millis(interval as u64),\n                last_ping: Instant::now() - Duration::from_millis(interval as u64),\n                payload: Bytes::from(format!(\n                    \"event: ping\\ndata: {{\\\"interval\\\": {}}}\\n\\n\",\n                    interval\n                )),\n            }\n            .into()\n        } else {\n            None\n        };\n\n        // Register with push manager\n        let mut push_rx = self.subscribe_push_manager(&access_token, types).await?;\n        let mut changed: VecMap<Id, VecMap<DataType, State>> = VecMap::new();\n        let throttle = self.core.jmap.event_source_throttle;\n\n        Ok(HttpResponse::new(StatusCode::OK)\n            .with_content_type(\"text/event-stream\")\n            .with_cache_control(\"no-store\")\n            .with_stream_body(BoxBody::new(StreamBody::new(async_stream::stream! {\n                let mut last_message = Instant::now() - throttle;\n                let mut timeout =\n                    ping.as_ref().map(|p| p.interval).unwrap_or(LONG_1D_SLUMBER);\n\n                loop {\n                    match tokio::time::timeout(timeout, push_rx.recv()).await {\n                        Ok(Some(notification)) => {\n                            match notification {\n                                PushNotification::StateChange(state_change) => {\n                                    for type_state in state_change.types {\n                                        changed\n                                            .get_mut_or_insert(state_change.account_id.into())\n                                            .set(type_state, (state_change.change_id).into());\n                                    }\n                                }\n                                PushNotification::CalendarAlert(calendar_alert) => {\n                                    yield Ok(Frame::data(Bytes::from(format!(\n                                        \"event: calendarAlert\\ndata: {}\\n\\n\",\n                                        serde_json::to_string(&calendar_alert.into_push_object()).unwrap()\n                                    ))));\n                                }\n                                PushNotification::EmailPush(email_push) => {\n                                    let state_change = email_push.to_state_change();\n                                    for type_state in state_change.types {\n                                        changed\n                                            .get_mut_or_insert(state_change.account_id.into())\n                                            .set(type_state, state_change.change_id.into());\n                                    }\n                                }\n                            }\n                        }\n                        Ok(None) => {\n                            break;\n                        }\n                        Err(_) => (),\n                    }\n\n                    timeout = if !changed.is_empty() {\n                        let elapsed = last_message.elapsed();\n                        if elapsed >= throttle {\n                            last_message = Instant::now();\n                            let response =\n                                PushObject::StateChange { changed: std::mem::take(&mut changed) };\n\n                            yield Ok(Frame::data(Bytes::from(format!(\n                                \"event: state\\ndata: {}\\n\\n\",\n                                serde_json::to_string(&response).unwrap()\n                            ))));\n\n                            if close_after_state {\n                                break;\n                            }\n\n                            ping.as_ref().map(|p| p.interval).unwrap_or(LONG_1D_SLUMBER)\n                        } else {\n                            throttle - elapsed\n                        }\n                    } else if let Some(ping) = &mut ping {\n                        let elapsed = ping.last_ping.elapsed();\n                        if elapsed >= ping.interval {\n                            ping.last_ping = Instant::now();\n                            yield Ok(Frame::data(ping.payload.clone()));\n                            ping.interval\n                        } else {\n                            ping.interval - elapsed\n                        }\n                    } else {\n                        LONG_1D_SLUMBER\n                    };\n                }\n            }))))\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/api/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::blob::UploadResponse;\nuse calcard::jscalendar::JSCalendarDateTime;\nuse common::ipc::{CalendarAlert, PushNotification};\nuse http_proto::{HttpResponse, JsonResponse, ToHttpResponse};\nuse hyper::StatusCode;\nuse jmap_proto::{\n    error::request::{RequestError, RequestLimitError},\n    request::capability::Session,\n    response::{Response, status::PushObject},\n    types::state::State,\n};\nuse types::{id::Id, type_state::DataType};\nuse utils::map::vec_map::VecMap;\n\npub mod acl;\npub mod auth;\npub mod event_source;\npub mod query;\npub mod request;\npub mod session;\n\nimpl ToHttpResponse for UploadResponse {\n    fn into_http_response(self) -> HttpResponse {\n        JsonResponse::new(self).into_http_response()\n    }\n}\n\npub trait ToJmapHttpResponse {\n    fn into_http_response(self) -> HttpResponse;\n}\n\nimpl ToJmapHttpResponse for Response<'_> {\n    fn into_http_response(self) -> HttpResponse {\n        JsonResponse::new(self).into_http_response()\n    }\n}\n\nimpl ToJmapHttpResponse for Session {\n    fn into_http_response(self) -> HttpResponse {\n        JsonResponse::new(self).into_http_response()\n    }\n}\n\nimpl ToJmapHttpResponse for RequestError<'_> {\n    fn into_http_response(self) -> HttpResponse {\n        HttpResponse::new(StatusCode::from_u16(self.status).unwrap_or(StatusCode::BAD_REQUEST))\n            .with_content_type(\"application/problem+json\")\n            .with_text_body(serde_json::to_string(&self).unwrap_or_default())\n    }\n}\n\npub trait ToRequestError {\n    fn to_request_error(&self) -> RequestError<'_>;\n}\n\nimpl ToRequestError for trc::Error {\n    fn to_request_error(&self) -> RequestError<'_> {\n        let details_or_reason = self\n            .value(trc::Key::Details)\n            .or_else(|| self.value(trc::Key::Reason))\n            .and_then(|v| v.as_str());\n        let details = details_or_reason.unwrap_or_else(|| self.as_ref().message());\n\n        match self.as_ref() {\n            trc::EventType::Jmap(cause) => match cause {\n                trc::JmapEvent::UnknownCapability => RequestError::unknown_capability(details),\n                trc::JmapEvent::NotJson => RequestError::not_json(details),\n                trc::JmapEvent::NotRequest => RequestError::not_request(details),\n                _ => RequestError::invalid_parameters(),\n            },\n            trc::EventType::Limit(cause) => match cause {\n                trc::LimitEvent::SizeRequest => RequestError::limit(RequestLimitError::SizeRequest),\n                trc::LimitEvent::SizeUpload => RequestError::limit(RequestLimitError::SizeUpload),\n                trc::LimitEvent::CallsIn => RequestError::limit(RequestLimitError::CallsIn),\n                trc::LimitEvent::ConcurrentRequest | trc::LimitEvent::ConcurrentConnection => {\n                    RequestError::limit(RequestLimitError::ConcurrentRequest)\n                }\n                trc::LimitEvent::ConcurrentUpload => {\n                    RequestError::limit(RequestLimitError::ConcurrentUpload)\n                }\n                trc::LimitEvent::Quota => RequestError::over_quota(),\n                trc::LimitEvent::TenantQuota => RequestError::tenant_over_quota(),\n                trc::LimitEvent::BlobQuota => RequestError::over_blob_quota(\n                    self.value(trc::Key::Total)\n                        .and_then(|v| v.to_uint())\n                        .unwrap_or_default() as usize,\n                    self.value(trc::Key::Size)\n                        .and_then(|v| v.to_uint())\n                        .unwrap_or_default() as usize,\n                ),\n                trc::LimitEvent::TooManyRequests => RequestError::too_many_requests(),\n            },\n            trc::EventType::Auth(cause) => match cause {\n                trc::AuthEvent::MissingTotp => {\n                    RequestError::blank(402, \"TOTP code required\", cause.message())\n                }\n                trc::AuthEvent::TooManyAttempts => RequestError::too_many_auth_attempts(),\n                _ => RequestError::unauthorized(),\n            },\n            trc::EventType::Security(cause) => match cause {\n                trc::SecurityEvent::AuthenticationBan\n                | trc::SecurityEvent::ScanBan\n                | trc::SecurityEvent::AbuseBan\n                | trc::SecurityEvent::LoiterBan\n                | trc::SecurityEvent::IpBlocked => RequestError::too_many_auth_attempts(),\n                trc::SecurityEvent::Unauthorized => RequestError::forbidden(),\n            },\n            trc::EventType::Resource(cause) => match cause {\n                trc::ResourceEvent::NotFound => RequestError::not_found(),\n                trc::ResourceEvent::BadParameters => RequestError::blank(\n                    StatusCode::BAD_REQUEST.as_u16(),\n                    \"Invalid parameters\",\n                    details_or_reason.unwrap_or(\"One or multiple parameters could not be parsed.\"),\n                ),\n                trc::ResourceEvent::Error => RequestError::internal_server_error(),\n                _ => RequestError::internal_server_error(),\n            },\n            _ => RequestError::internal_server_error(),\n        }\n    }\n}\n\npub(crate) trait IntoPushObject {\n    fn into_push_object(self) -> PushObject;\n}\n\nimpl IntoPushObject for Vec<PushNotification> {\n    fn into_push_object(self) -> PushObject {\n        let mut changed: VecMap<Id, VecMap<DataType, State>> = VecMap::new();\n        let mut objects = Vec::with_capacity(self.len());\n        for notification in self {\n            match notification {\n                PushNotification::StateChange(state_change) => {\n                    for type_state in state_change.types {\n                        changed\n                            .get_mut_or_insert(state_change.account_id.into())\n                            .set(type_state, (state_change.change_id).into());\n                    }\n                }\n                PushNotification::CalendarAlert(calendar_alert) => {\n                    objects.push(calendar_alert.into_push_object());\n                }\n                PushNotification::EmailPush(email_push) => {\n                    let state_change = email_push.to_state_change();\n                    for type_state in state_change.types {\n                        changed\n                            .get_mut_or_insert(state_change.account_id.into())\n                            .set(type_state, state_change.change_id.into());\n                    }\n                }\n            }\n        }\n\n        if !objects.is_empty() {\n            if changed.is_empty() {\n                objects.push(PushObject::StateChange { changed });\n            }\n            if objects.len() > 1 {\n                PushObject::Group { entries: objects }\n            } else {\n                objects.into_iter().next().unwrap()\n            }\n        } else {\n            PushObject::StateChange { changed }\n        }\n    }\n}\n\nimpl IntoPushObject for CalendarAlert {\n    fn into_push_object(self) -> PushObject {\n        PushObject::CalendarAlert {\n            account_id: self.account_id.into(),\n            calendar_event_id: self.event_id.into(),\n            uid: self.uid,\n            recurrence_id: self\n                .recurrence_id\n                .map(|timestamp| JSCalendarDateTime::new(timestamp, true).to_rfc3339()),\n            alert_id: self.alert_id,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/api/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_proto::{\n    method::query::{QueryRequest, QueryResponse},\n    object::JmapObject,\n    types::state::State,\n};\nuse types::id::Id;\n\npub struct QueryResponseBuilder {\n    requested_position: i32,\n    position: i32,\n    pub limit: usize,\n    anchor: u32,\n    anchor_offset: i32,\n    has_anchor: bool,\n    anchor_found: bool,\n\n    pub response: QueryResponse,\n}\n\nimpl QueryResponseBuilder {\n    pub fn new<T: JmapObject + Sync + Send>(\n        total_results: usize,\n        max_results: usize,\n        query_state: State,\n        request: &QueryRequest<T>,\n    ) -> Self {\n        let (limit_total, limit) = if let Some(limit) = request.limit {\n            if limit > 0 {\n                let limit = std::cmp::min(limit, max_results);\n                (std::cmp::min(limit, total_results), limit)\n            } else {\n                (0, 0)\n            }\n        } else {\n            (std::cmp::min(max_results, total_results), max_results)\n        };\n\n        let (has_anchor, anchor) = request\n            .anchor\n            .map(|anchor| (true, anchor.document_id()))\n            .unwrap_or((false, 0));\n\n        QueryResponseBuilder {\n            requested_position: request.position.unwrap_or(0),\n            position: request.position.unwrap_or(0),\n            limit: limit_total,\n            anchor,\n            anchor_offset: request.anchor_offset.unwrap_or(0),\n            has_anchor,\n            anchor_found: false,\n            response: QueryResponse {\n                account_id: request.account_id,\n                query_state,\n                can_calculate_changes: true,\n                position: 0,\n                ids: vec![],\n                total: if request.calculate_total.unwrap_or(false) {\n                    Some(total_results)\n                } else {\n                    None\n                },\n                limit: if total_results > limit {\n                    Some(limit)\n                } else {\n                    None\n                },\n            },\n        }\n    }\n\n    #[inline(always)]\n    pub fn add(&mut self, prefix_id: u32, document_id: u32) -> bool {\n        self.add_id(Id::from_parts(prefix_id, document_id))\n    }\n\n    pub fn add_id(&mut self, id: Id) -> bool {\n        let document_id = id.document_id();\n\n        // Pagination\n        if !self.has_anchor {\n            if self.position >= 0 {\n                if self.position > 0 {\n                    self.position -= 1;\n                } else {\n                    self.response.ids.push(id);\n                    if self.response.ids.len() == self.limit {\n                        return false;\n                    }\n                }\n            } else {\n                self.response.ids.push(id);\n            }\n        } else if self.anchor_offset >= 0 {\n            if !self.anchor_found {\n                if document_id != self.anchor {\n                    return true;\n                }\n                self.anchor_found = true;\n            }\n\n            if self.anchor_offset > 0 {\n                self.anchor_offset -= 1;\n            } else {\n                self.response.ids.push(id);\n                if self.response.ids.len() == self.limit {\n                    return false;\n                }\n            }\n        } else {\n            self.anchor_found = document_id == self.anchor;\n            self.response.ids.push(id);\n\n            if self.anchor_found {\n                self.position = self.anchor_offset;\n                return false;\n            }\n        }\n\n        true\n    }\n\n    pub fn is_full(&self) -> bool {\n        self.response.ids.len() == self.limit\n    }\n\n    pub fn build(mut self) -> trc::Result<QueryResponse> {\n        if !self.has_anchor || self.anchor_found {\n            if !self.has_anchor && self.requested_position >= 0 {\n                self.response.position = if self.position == 0 {\n                    self.requested_position\n                } else {\n                    0\n                };\n            } else if self.position >= 0 {\n                self.response.position = self.position;\n            } else {\n                let position = self.position.unsigned_abs() as usize;\n                let start_offset = if position < self.response.ids.len() {\n                    self.response.ids.len() - position\n                } else {\n                    0\n                };\n                self.response.position = start_offset as i32;\n                let end_offset = if self.limit > 0 {\n                    std::cmp::min(start_offset + self.limit, self.response.ids.len())\n                } else {\n                    self.response.ids.len()\n                };\n\n                self.response.ids = self.response.ids[start_offset..end_offset].to_vec()\n            }\n\n            Ok(self.response)\n        } else {\n            Err(trc::JmapEvent::AnchorNotFound.into_err())\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/api/request.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    addressbook::{get::AddressBookGet, set::AddressBookSet},\n    api::auth::JmapAuthorization,\n    blob::{copy::BlobCopy, get::BlobOperations, upload::BlobUpload},\n    calendar::{get::CalendarGet, set::CalendarSet},\n    calendar_event::{\n        copy::JmapCalendarEventCopy, get::CalendarEventGet, parse::CalendarEventParse,\n        query::CalendarEventQuery, set::CalendarEventSet,\n    },\n    calendar_event_notification::{\n        get::CalendarEventNotificationGet, query::CalendarEventNotificationQuery,\n        set::CalendarEventNotificationSet,\n    },\n    changes::{get::ChangesLookup, query::QueryChanges},\n    contact::{\n        copy::JmapContactCardCopy, get::ContactCardGet, parse::ContactCardParse,\n        query::ContactCardQuery, set::ContactCardSet,\n    },\n    email::{\n        copy::JmapEmailCopy, get::EmailGet, import::EmailImport, parse::EmailParse,\n        query::EmailQuery, set::EmailSet, snippet::EmailSearchSnippet,\n    },\n    file::{get::FileNodeGet, query::FileNodeQuery, set::FileNodeSet},\n    identity::{get::IdentityGet, set::IdentitySet},\n    mailbox::{get::MailboxGet, query::MailboxQuery, set::MailboxSet},\n    participant_identity::{get::ParticipantIdentityGet, set::ParticipantIdentitySet},\n    principal::{availability::PrincipalGetAvailability, get::PrincipalGet, query::PrincipalQuery},\n    push::{get::PushSubscriptionFetch, set::PushSubscriptionSet},\n    quota::{get::QuotaGet, query::QuotaQuery},\n    share_notification::{\n        get::ShareNotificationGet, query::ShareNotificationQuery, set::ShareNotificationSet,\n    },\n    sieve::{\n        get::SieveScriptGet, query::SieveScriptQuery, set::SieveScriptSet,\n        validate::SieveScriptValidate,\n    },\n    submission::{get::EmailSubmissionGet, query::EmailSubmissionQuery, set::EmailSubmissionSet},\n    thread::get::ThreadGet,\n    vacation::{get::VacationResponseGet, set::VacationResponseSet},\n};\nuse common::{Server, auth::AccessToken};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    request::{\n        Call, CopyRequestMethod, GetRequestMethod, ParseRequestMethod, QueryRequestMethod, Request,\n        RequestMethod, SetRequestMethod, method::MethodName,\n    },\n    response::{Response, ResponseMethod, SetResponseMethod},\n};\nuse std::future::Future;\nuse std::{sync::Arc, time::Instant};\nuse trc::JmapEvent;\nuse types::{collection::Collection, id::Id};\n\npub trait RequestHandler: Sync + Send {\n    fn handle_jmap_request<'x>(\n        &self,\n        request: Request<'x>,\n        access_token: Arc<AccessToken>,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = Response<'x>> + Send;\n\n    fn handle_method_call<'x>(\n        &self,\n        method: RequestMethod<'x>,\n        method_name: MethodName,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<ResponseMethod<'x>>> + Send;\n}\n\nimpl RequestHandler for Server {\n    #![allow(clippy::large_futures)]\n    async fn handle_jmap_request<'x>(\n        &self,\n        request: Request<'x>,\n        access_token: Arc<AccessToken>,\n        session: &HttpSessionData,\n    ) -> Response<'x> {\n        let add_created_ids = request.created_ids.is_some();\n        let mut response = Response::new(\n            access_token.state(),\n            request.created_ids.unwrap_or_default(),\n            request.method_calls.len(),\n        );\n\n        for mut call in request.method_calls {\n            // Resolve result and id references\n            if let Err(error) = response.resolve_references(&mut call.method) {\n                let method_error = error.clone();\n\n                trc::error!(error.span_id(session.session_id));\n\n                response.push_response(call.id, MethodName::error(), method_error);\n                continue;\n            }\n\n            loop {\n                let mut next_call = None;\n\n                // Add response\n                let method_name = call.name.as_str();\n                match self\n                    .handle_method_call(\n                        call.method,\n                        call.name,\n                        &access_token,\n                        &mut next_call,\n                        session,\n                    )\n                    .await\n                {\n                    Ok(mut method_response) => {\n                        match &mut method_response {\n                            ResponseMethod::Set(set_response) => {\n                                // Add created ids\n                                match set_response {\n                                    SetResponseMethod::Email(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::Mailbox(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::Identity(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::EmailSubmission(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::PushSubscription(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::Sieve(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::VacationResponse(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::AddressBook(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::ContactCard(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::FileNode(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::ShareNotification(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::Calendar(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::CalendarEvent(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::ParticipantIdentity(set_response) => {\n                                        set_response.update_created_ids(&mut response);\n                                    }\n                                    SetResponseMethod::CalendarEventNotification(_) => {}\n                                }\n                            }\n                            ResponseMethod::ImportEmail(import_response) => {\n                                // Add created ids\n                                import_response.update_created_ids(&mut response);\n                            }\n                            ResponseMethod::UploadBlob(upload_response) => {\n                                // Add created blobIds\n                                upload_response.update_created_ids(&mut response);\n                            }\n                            _ => {}\n                        }\n\n                        response.push_response(call.id, call.name, method_response);\n                    }\n                    Err(error) => {\n                        let method_error = error.clone();\n\n                        trc::error!(\n                            error\n                                .span_id(session.session_id)\n                                .ctx_unique(trc::Key::AccountId, access_token.primary_id())\n                                .caused_by(method_name)\n                        );\n\n                        response.push_error(call.id, method_error);\n                    }\n                }\n\n                // Process next call\n                if let Some(next_call) = next_call {\n                    call = next_call;\n                    call.id\n                        .clone_from(&response.method_responses.last().unwrap().id);\n                } else {\n                    break;\n                }\n            }\n        }\n\n        if !add_created_ids {\n            response.created_ids.clear();\n        }\n\n        response\n    }\n\n    async fn handle_method_call<'x>(\n        &self,\n        method: RequestMethod<'x>,\n        method_name: MethodName,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        session: &HttpSessionData,\n    ) -> trc::Result<ResponseMethod<'x>> {\n        let op_start = Instant::now();\n\n        // Check permissions\n        access_token.assert_has_jmap_permission(&method, method_name.obj)?;\n\n        // Handle method\n        let response = match method {\n            RequestMethod::Get(req) => match req {\n                GetRequestMethod::Email(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Email)?;\n\n                    self.email_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::Mailbox(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Mailbox)?;\n\n                    self.mailbox_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::Thread(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Email)?;\n\n                    self.thread_get(req).await?.into()\n                }\n                GetRequestMethod::Identity(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.identity_get(req).await?.into()\n                }\n                GetRequestMethod::EmailSubmission(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.email_submission_get(req).await?.into()\n                }\n                GetRequestMethod::PushSubscription(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    self.push_subscription_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::Sieve(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.sieve_script_get(req).await?.into()\n                }\n                GetRequestMethod::VacationResponse(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.vacation_response_get(req).await?.into()\n                }\n                GetRequestMethod::Principal(req) => {\n                    self.principal_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::Quota(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.quota_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::Blob(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.blob_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::AddressBook(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::AddressBook)?;\n\n                    self.address_book_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::ContactCard(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::ContactCard)?;\n\n                    self.contact_card_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::FileNode(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::FileNode)?;\n\n                    self.file_node_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::PrincipalAvailability(req) => self\n                    .principal_get_availability(req, access_token)\n                    .await?\n                    .into(),\n                GetRequestMethod::Calendar(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Calendar)?;\n\n                    self.calendar_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::CalendarEvent(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::CalendarEvent)?;\n\n                    self.calendar_event_get(req, access_token).await?.into()\n                }\n                GetRequestMethod::CalendarEventNotification(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.calendar_event_notification_get(req, access_token)\n                        .await?\n                        .into()\n                }\n                GetRequestMethod::ParticipantIdentity(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.participant_identity_get(req).await?.into()\n                }\n                GetRequestMethod::ShareNotification(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.share_notification_get(req).await?.into()\n                }\n            },\n            RequestMethod::Query(req) => match req {\n                QueryRequestMethod::Email(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Email)?;\n\n                    self.email_query(req, access_token).await?.into()\n                }\n                QueryRequestMethod::Mailbox(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Mailbox)?;\n\n                    self.mailbox_query(req, access_token).await?.into()\n                }\n                QueryRequestMethod::EmailSubmission(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.email_submission_query(req).await?.into()\n                }\n                QueryRequestMethod::Sieve(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.sieve_script_query(req).await?.into()\n                }\n                QueryRequestMethod::Principal(req) => self\n                    .principal_query(req, access_token, session)\n                    .await?\n                    .into(),\n                QueryRequestMethod::Quota(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.quota_query(req, access_token).await?.into()\n                }\n                QueryRequestMethod::ContactCard(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::ContactCard)?;\n\n                    self.contact_card_query(req, access_token).await?.into()\n                }\n                QueryRequestMethod::FileNode(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::FileNode)?;\n\n                    self.file_node_query(req, access_token).await?.into()\n                }\n                QueryRequestMethod::CalendarEvent(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::CalendarEvent)?;\n\n                    self.calendar_event_query(req, access_token).await?.into()\n                }\n                QueryRequestMethod::CalendarEventNotification(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.calendar_event_notification_query(req, access_token)\n                        .await?\n                        .into()\n                }\n                QueryRequestMethod::ShareNotification(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.share_notification_query(req).await?.into()\n                }\n            },\n            RequestMethod::Set(req) => match req {\n                SetRequestMethod::Email(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Email)?;\n\n                    self.email_set(req, access_token, session).await?.into()\n                }\n                SetRequestMethod::Mailbox(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Mailbox)?;\n\n                    self.mailbox_set(req, access_token).await?.into()\n                }\n                SetRequestMethod::Identity(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.identity_set(req, access_token).await?.into()\n                }\n                SetRequestMethod::EmailSubmission(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.email_submission_set(req, &session.instance, next_call)\n                        .await?\n                        .into()\n                }\n                SetRequestMethod::PushSubscription(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    self.push_subscription_set(req, access_token).await?.into()\n                }\n                SetRequestMethod::Sieve(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.sieve_script_set(req, access_token, session)\n                        .await?\n                        .into()\n                }\n                SetRequestMethod::VacationResponse(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.vacation_response_set(req, access_token).await?.into()\n                }\n                SetRequestMethod::AddressBook(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::AddressBook)?;\n\n                    self.address_book_set(req, access_token, session)\n                        .await?\n                        .into()\n                }\n                SetRequestMethod::ContactCard(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::ContactCard)?;\n\n                    self.contact_card_set(req, access_token, session)\n                        .await?\n                        .into()\n                }\n                SetRequestMethod::FileNode(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::FileNode)?;\n\n                    self.file_node_set(req, access_token, session).await?.into()\n                }\n                SetRequestMethod::ShareNotification(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.share_notification_set(req).await?.into()\n                }\n                SetRequestMethod::Calendar(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Calendar)?;\n\n                    self.calendar_set(req, access_token, session).await?.into()\n                }\n                SetRequestMethod::CalendarEvent(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::CalendarEvent)?;\n\n                    self.calendar_event_set(req, access_token, session)\n                        .await?\n                        .into()\n                }\n                SetRequestMethod::CalendarEventNotification(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.calendar_event_notification_set(req, access_token, session)\n                        .await?\n                        .into()\n                }\n                SetRequestMethod::ParticipantIdentity(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.participant_identity_set(req, access_token)\n                        .await?\n                        .into()\n                }\n            },\n            RequestMethod::Changes(mut req) => {\n                set_account_id_if_missing(&mut req.account_id, access_token);\n\n                self.changes(req, method_name.obj, access_token)\n                    .await?\n                    .into_method_response()\n            }\n            RequestMethod::Copy(req) => match req {\n                CopyRequestMethod::Email(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    set_account_id_if_missing(&mut req.from_account_id, access_token);\n\n                    access_token\n                        .assert_has_access(req.account_id, Collection::Email)?\n                        .assert_has_access(req.from_account_id, Collection::Email)?;\n\n                    self.email_copy(req, access_token, next_call, session)\n                        .await?\n                        .into()\n                }\n                CopyRequestMethod::Blob(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_is_member(req.account_id)?;\n\n                    self.blob_copy(req, access_token).await?.into()\n                }\n                CopyRequestMethod::ContactCard(mut req) => {\n                    set_account_id_if_missing(&mut req.from_account_id, access_token);\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n\n                    access_token\n                        .assert_has_access(req.account_id, Collection::ContactCard)?\n                        .assert_has_access(req.from_account_id, Collection::ContactCard)?;\n\n                    self.contact_card_copy(req, access_token, next_call, session)\n                        .await?\n                        .into()\n                }\n                CopyRequestMethod::CalendarEvent(mut req) => {\n                    set_account_id_if_missing(&mut req.from_account_id, access_token);\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n\n                    access_token\n                        .assert_has_access(req.account_id, Collection::CalendarEvent)?\n                        .assert_has_access(req.from_account_id, Collection::CalendarEvent)?;\n\n                    self.calendar_event_copy(req, access_token, next_call, session)\n                        .await?\n                        .into()\n                }\n            },\n            RequestMethod::ImportEmail(mut req) => {\n                set_account_id_if_missing(&mut req.account_id, access_token);\n                access_token.assert_has_access(req.account_id, Collection::Email)?;\n\n                self.email_import(req, access_token, session).await?.into()\n            }\n            RequestMethod::Parse(req) => match req {\n                ParseRequestMethod::Email(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::Email)?;\n\n                    self.email_parse(req, access_token).await?.into()\n                }\n                ParseRequestMethod::ContactCard(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::ContactCard)?;\n\n                    self.contact_card_parse(req, access_token).await?.into()\n                }\n                ParseRequestMethod::CalendarEvent(mut req) => {\n                    set_account_id_if_missing(&mut req.account_id, access_token);\n                    access_token.assert_has_access(req.account_id, Collection::CalendarEvent)?;\n\n                    self.calendar_event_parse(req, access_token).await?.into()\n                }\n            },\n            RequestMethod::QueryChanges(req) => self.query_changes(req, access_token).await?.into(),\n            RequestMethod::SearchSnippet(mut req) => {\n                set_account_id_if_missing(&mut req.account_id, access_token);\n                access_token.assert_has_access(req.account_id, Collection::Email)?;\n\n                self.email_search_snippet(req, access_token).await?.into()\n            }\n            RequestMethod::ValidateScript(mut req) => {\n                set_account_id_if_missing(&mut req.account_id, access_token);\n                access_token.assert_is_member(req.account_id)?;\n\n                self.sieve_script_validate(req, access_token).await?.into()\n            }\n            RequestMethod::LookupBlob(mut req) => {\n                set_account_id_if_missing(&mut req.account_id, access_token);\n                access_token.assert_is_member(req.account_id)?;\n\n                self.blob_lookup(req).await?.into()\n            }\n            RequestMethod::UploadBlob(mut req) => {\n                set_account_id_if_missing(&mut req.account_id, access_token);\n                access_token.assert_is_member(req.account_id)?;\n\n                self.blob_upload_many(req, access_token).await?.into()\n            }\n            RequestMethod::Echo(req) => req.into(),\n            RequestMethod::Error(error) => return Err(error),\n        };\n\n        trc::event!(\n            Jmap(JmapEvent::MethodCall),\n            Id = method_name.as_str(),\n            SpanId = session.session_id,\n            AccountId = access_token.primary_id(),\n            Elapsed = op_start.elapsed(),\n        );\n\n        Ok(response)\n    }\n}\n\n#[inline]\npub(crate) fn set_account_id_if_missing(account_id: &mut Id, access_token: &AccessToken) {\n    if !account_id.is_valid() {\n        *account_id = Id::from(access_token.primary_id());\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/api/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse directory::Permission;\nuse jmap_proto::request::capability::{\n    Account, Capabilities, Capability, EmptyCapabilities, Session,\n};\nuse std::future::Future;\nuse std::sync::Arc;\nuse types::id::Id;\nuse utils::map::vec_map::VecMap;\n\npub trait SessionHandler: Sync + Send {\n    fn handle_session_resource(\n        &self,\n        base_url: String,\n        access_token: Arc<AccessToken>,\n    ) -> impl Future<Output = trc::Result<Session>> + Send;\n}\n\nimpl SessionHandler for Server {\n    async fn handle_session_resource(\n        &self,\n        base_url: String,\n        access_token: Arc<AccessToken>,\n    ) -> trc::Result<Session> {\n        let mut session = Session::new(base_url, &self.core.jmap.capabilities);\n        session.set_state(access_token.state());\n        let account_capabilities = &self.core.jmap.capabilities.account;\n\n        // Set primary account\n        session.username = access_token.name.to_string();\n        let account_id = Id::from(access_token.primary_id());\n        let mut account = Account {\n            name: access_token.name.to_string(),\n            is_personal: true,\n            is_read_only: false,\n            account_capabilities: VecMap::with_capacity(account_capabilities.len()),\n        };\n        for capability in access_token.account_capabilities() {\n            session.primary_accounts.append(capability, account_id);\n            account.account_capabilities.append(\n                capability,\n                account_capabilities\n                    .get(&capability)\n                    .map(|v| v.to_account_capabilities(account_id.into(), true))\n                    .unwrap_or_else(|| Capabilities::Empty(EmptyCapabilities::default())),\n            );\n        }\n        session.accounts.append(account_id, account);\n\n        // Add secondary accounts\n        for &account_id in access_token.secondary_ids() {\n            let is_owner = access_token.is_member(account_id);\n            let access_token = match self.get_access_token(account_id).await {\n                Ok(token) => token,\n                Err(err) => {\n                    if err.matches(trc::EventType::Auth(trc::AuthEvent::Error)) {\n                        continue;\n                    } else {\n                        return Err(err.caused_by(trc::location!()));\n                    }\n                }\n            };\n\n            let account_id = Id::from(account_id);\n            let mut account = Account {\n                name: access_token.name.to_string(),\n                is_personal: false,\n                is_read_only: false,\n                account_capabilities: VecMap::with_capacity(account_capabilities.len()),\n            };\n            for capability in access_token.account_capabilities() {\n                account.account_capabilities.append(\n                    capability,\n                    account_capabilities\n                        .get(&capability)\n                        .map(|v| v.to_account_capabilities(account_id.into(), is_owner))\n                        .unwrap_or_else(|| Capabilities::Empty(EmptyCapabilities::default())),\n                );\n            }\n            session.accounts.append(account_id, account);\n        }\n\n        Ok(session)\n    }\n}\n\ntrait AccountCapabilities {\n    fn account_capabilities(&self) -> impl Iterator<Item = Capability>;\n}\n\nimpl AccountCapabilities for AccessToken {\n    fn account_capabilities(&self) -> impl Iterator<Item = Capability> {\n        Capability::all_capabilities()\n            .iter()\n            .filter(move |capability| {\n                let permission = match capability {\n                    Capability::Mail => Permission::JmapEmailGet,\n                    Capability::Submission => Permission::JmapEmailSubmissionSet,\n                    Capability::VacationResponse => Permission::JmapVacationResponseGet,\n                    Capability::Contacts => Permission::JmapContactCardGet,\n                    Capability::ContactsParse => Permission::JmapContactCardParse,\n                    Capability::Calendars => Permission::JmapCalendarEventGet,\n                    Capability::CalendarsParse => Permission::JmapCalendarEventParse,\n                    Capability::Sieve => Permission::JmapSieveScriptGet,\n                    Capability::Blob => Permission::JmapBlobGet,\n                    Capability::Quota => Permission::JmapQuotaGet,\n                    Capability::FileNode => Permission::JmapFileNodeGet,\n                    Capability::WebSocket\n                    | Capability::Principals\n                    | Capability::PrincipalsAvailability => return true,\n                    Capability::Core | Capability::PrincipalsOwner => return false,\n                };\n                self.has_permission(permission)\n            })\n            .copied()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/blob/copy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::download::BlobDownload;\nuse common::{Server, auth::AccessToken};\nuse directory::Permission;\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::copy::{CopyBlobRequest, CopyBlobResponse},\n    request::IntoValid,\n};\nuse std::future::Future;\nuse store::write::{BatchBuilder, BlobLink, BlobOp, now};\nuse trc::AddContext;\nuse types::blob::{BlobClass, BlobId};\nuse utils::map::vec_map::VecMap;\n\npub trait BlobCopy: Sync + Send {\n    fn blob_copy(\n        &self,\n        request: CopyBlobRequest,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<CopyBlobResponse>> + Send;\n}\n\nimpl BlobCopy for Server {\n    async fn blob_copy(\n        &self,\n        request: CopyBlobRequest,\n        access_token: &AccessToken,\n    ) -> trc::Result<CopyBlobResponse> {\n        let mut response = CopyBlobResponse {\n            from_account_id: request.from_account_id,\n            account_id: request.account_id,\n            copied: VecMap::with_capacity(request.blob_ids.len()),\n            not_copied: VecMap::new(),\n        };\n        let account_id = request.account_id.document_id();\n\n        for blob_id in request.blob_ids.into_valid() {\n            if self.has_access_blob(&blob_id, access_token).await? {\n                // Enforce quota\n                let used = self\n                    .core\n                    .storage\n                    .data\n                    .blob_quota(account_id)\n                    .await\n                    .caused_by(trc::location!())?;\n\n                if ((self.core.jmap.upload_tmp_quota_size > 0\n                    && used.bytes >= self.core.jmap.upload_tmp_quota_size)\n                    || (self.core.jmap.upload_tmp_quota_amount > 0\n                        && used.count + 1 > self.core.jmap.upload_tmp_quota_amount))\n                    && !access_token.has_permission(Permission::UnlimitedUploads)\n                {\n                    response.not_copied.append(\n                        blob_id,\n                        SetError::over_quota().with_description(format!(\n                            \"You have exceeded the blob quota of {} files or {} bytes.\",\n                            self.core.jmap.upload_tmp_quota_amount,\n                            self.core.jmap.upload_tmp_quota_size\n                        )),\n                    );\n                    continue;\n                }\n\n                let mut batch = BatchBuilder::new();\n                let until = now() + self.core.jmap.upload_tmp_ttl;\n                batch.with_account_id(account_id).set(\n                    BlobOp::Link {\n                        hash: blob_id.hash.clone(),\n                        to: BlobLink::Temporary { until },\n                    },\n                    vec![],\n                );\n                self.store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n\n                let dest_blob_id = BlobId {\n                    hash: blob_id.hash.clone(),\n                    class: BlobClass::Reserved {\n                        account_id,\n                        expires: until,\n                    },\n                    section: blob_id.section.clone(),\n                };\n\n                response.copied.append(blob_id, dest_blob_id);\n            } else {\n                response.not_copied.append(\n                    blob_id,\n                    SetError::new(SetErrorType::BlobNotFound).with_description(\n                        \"blobId does not exist or not enough permissions to access it.\",\n                    ),\n                );\n            }\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/blob/download.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse email::cache::MessageCacheFetch;\nuse email::cache::email::MessageCacheAccess;\nuse email::message::metadata::MessageMetadata;\nuse groupware::cache::GroupwareCache;\nuse store::ValueKey;\nuse store::write::{AlignedBytes, Archive};\nuse std::future::Future;\nuse trc::AddContext;\nuse types::acl::Acl;\nuse types::blob::{BlobClass, BlobId};\nuse types::collection::{Collection, SyncCollection};\nuse types::field::EmailField;\nuse utils::chained_bytes::ChainedBytes;\n\npub trait BlobDownload: Sync + Send {\n    fn blob_download(\n        &self,\n        blob_id: &BlobId,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<Option<Vec<u8>>>> + Send;\n\n    fn has_access_blob(\n        &self,\n        blob_id: &BlobId,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<bool>> + Send;\n}\n\nimpl BlobDownload for Server {\n    #[allow(clippy::blocks_in_conditions)]\n    async fn blob_download(\n        &self,\n        blob_id: &BlobId,\n        access_token: &AccessToken,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        if self.has_access_blob(blob_id, access_token).await? {\n            if let Some(section) = &blob_id.section {\n                self.get_blob_section(&blob_id.hash, section)\n                    .await\n                    .caused_by(trc::location!())\n            } else {\n                let blob = self\n                    .blob_store()\n                    .get_blob(blob_id.hash.as_slice(), 0..usize::MAX)\n                    .await\n                    .caused_by(trc::location!());\n                match (&blob_id.class, blob) {\n                    (\n                        BlobClass::Linked {\n                            account_id,\n                            collection,\n                            document_id,\n                        },\n                        Ok(Some(data)),\n                    ) if *collection == Collection::Email as u8 => {\n                        let Some(archive) = self\n                            .store()\n                            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                                *account_id,\n                                Collection::Email,\n                                *document_id,\n                                EmailField::Metadata,\n                            ))\n                            .await\n                            .caused_by(trc::location!())?\n                        else {\n                            return Ok(Some(data));\n                        };\n                        let metadata = archive\n                            .to_unarchived::<MessageMetadata>()\n                            .caused_by(trc::location!())?;\n                        let body_offset = metadata.inner.blob_body_offset.to_native();\n                        if metadata.inner.root_part().offset_body.to_native() != body_offset {\n                            let raw_message = ChainedBytes::new(\n                                metadata.inner.raw_headers.as_ref(),\n                            )\n                            .with_last(data.get(body_offset as usize..).unwrap_or_default());\n                            Ok(Some(raw_message.to_bytes()))\n                        } else {\n                            Ok(Some(data))\n                        }\n                    }\n                    (_, blob) => blob,\n                }\n            }\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn has_access_blob(\n        &self,\n        blob_id: &BlobId,\n        access_token: &AccessToken,\n    ) -> trc::Result<bool> {\n        Ok(self\n            .store()\n            .blob_has_access(&blob_id.hash, &blob_id.class)\n            .await\n            .caused_by(trc::location!())?\n            && match &blob_id.class {\n                BlobClass::Linked {\n                    account_id,\n                    collection,\n                    document_id,\n                } => {\n                    if access_token.is_member(*account_id) {\n                        true\n                    } else {\n                        match Collection::from(*collection) {\n                            Collection::Email => self\n                                .get_cached_messages(*account_id)\n                                .await\n                                .caused_by(trc::location!())?\n                                .shared_messages(access_token, Acl::ReadItems)\n                                .contains(*document_id),\n                            collection @ (Collection::FileNode\n                            | Collection::ContactCard\n                            | Collection::CalendarEvent) => self\n                                .fetch_dav_resources(\n                                    access_token,\n                                    *account_id,\n                                    SyncCollection::from(collection),\n                                )\n                                .await\n                                .caused_by(trc::location!())?\n                                .shared_items(access_token, [Acl::ReadItems], true)\n                                .contains(*document_id),\n                            _ => false,\n                        }\n                    }\n                }\n                BlobClass::Reserved { account_id, .. } => access_token.is_member(*account_id),\n            })\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/blob/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::download::BlobDownload;\nuse common::{Server, auth::AccessToken};\nuse email::message::metadata::MessageData;\nuse jmap_proto::{\n    method::{\n        get::{GetRequest, GetResponse},\n        lookup::{BlobInfo, BlobLookupRequest, BlobLookupResponse},\n    },\n    object::blob::{Blob, BlobProperty, BlobValue, DataProperty, DigestProperty},\n    request::{IntoValid, MaybeInvalid},\n};\nuse jmap_tools::{Map, Value};\nuse mail_builder::encoders::base64::base64_encode;\nuse sha1::{Digest, Sha1};\nuse sha2::{Sha256, Sha512};\nuse store::{ValueKey, write::{AlignedBytes, Archive}};\nuse std::future::Future;\nuse trc::AddContext;\nuse types::{blob::BlobClass, collection::Collection, id::Id, type_state::DataType};\nuse utils::map::vec_map::VecMap;\n\npub trait BlobOperations: Sync + Send {\n    fn blob_get(\n        &self,\n        request: GetRequest<Blob>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<Blob>>> + Send;\n\n    fn blob_lookup(\n        &self,\n        request: BlobLookupRequest,\n    ) -> impl Future<Output = trc::Result<BlobLookupResponse>> + Send;\n}\n\nimpl BlobOperations for Server {\n    async fn blob_get(\n        &self,\n        mut request: GetRequest<Blob>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<Blob>> {\n        let ids = request\n            .unwrap_ids(self.core.jmap.get_max_objects)?\n            .unwrap_or_default();\n        let properties = request.unwrap_properties(&[\n            BlobProperty::Id,\n            BlobProperty::Data(DataProperty::Default),\n            BlobProperty::Size,\n        ]);\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: None,\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        let range_from = request.arguments.offset.unwrap_or(0);\n        let range_to = request\n            .arguments\n            .length\n            .map(|length| range_from.saturating_add(length))\n            .unwrap_or(usize::MAX);\n\n        for blob_id in ids {\n            if let Some(bytes) = self.blob_download(&blob_id, access_token).await? {\n                let mut blob = Map::with_capacity(properties.len());\n                let bytes_range = if range_from == 0 && range_to == usize::MAX {\n                    &bytes[..]\n                } else {\n                    let range_to = if range_to != usize::MAX && range_to > bytes.len() {\n                        blob.insert_unchecked(BlobProperty::IsTruncated, true);\n                        bytes.len()\n                    } else {\n                        range_to\n                    };\n                    bytes.get(range_from..range_to).unwrap_or_default()\n                };\n\n                for property in &properties {\n                    let mut property = property.clone();\n                    let value: Value<'static, BlobProperty, BlobValue> = match &property {\n                        BlobProperty::Id => Value::Element(BlobValue::BlobId(blob_id.clone())),\n                        BlobProperty::Size => bytes.len().into(),\n                        BlobProperty::Digest(digest) => match digest {\n                            DigestProperty::Sha => {\n                                let mut hasher = Sha1::new();\n                                hasher.update(bytes_range);\n                                String::from_utf8(\n                                    base64_encode(&hasher.finalize()[..]).unwrap_or_default(),\n                                )\n                                .unwrap()\n                            }\n                            DigestProperty::Sha256 => {\n                                let mut hasher = Sha256::new();\n                                hasher.update(bytes_range);\n                                String::from_utf8(\n                                    base64_encode(&hasher.finalize()[..]).unwrap_or_default(),\n                                )\n                                .unwrap()\n                            }\n                            DigestProperty::Sha512 => {\n                                let mut hasher = Sha512::new();\n                                hasher.update(bytes_range);\n                                String::from_utf8(\n                                    base64_encode(&hasher.finalize()[..]).unwrap_or_default(),\n                                )\n                                .unwrap()\n                            }\n                        }\n                        .into(),\n                        BlobProperty::Data(data) => match data {\n                            DataProperty::AsText => match std::str::from_utf8(bytes_range) {\n                                Ok(text) => text.to_string().into(),\n                                Err(_) => {\n                                    blob.insert_unchecked(BlobProperty::IsEncodingProblem, true);\n                                    Value::Null\n                                }\n                            },\n                            DataProperty::AsBase64 => {\n                                String::from_utf8(base64_encode(bytes_range).unwrap_or_default())\n                                    .unwrap()\n                                    .into()\n                            }\n                            DataProperty::Default => match std::str::from_utf8(bytes_range) {\n                                Ok(text) => {\n                                    property = BlobProperty::Data(DataProperty::AsText);\n                                    text.to_string().into()\n                                }\n                                Err(_) => {\n                                    property = BlobProperty::Data(DataProperty::AsBase64);\n                                    blob.insert_unchecked(BlobProperty::IsEncodingProblem, true);\n                                    String::from_utf8(\n                                        base64_encode(bytes_range).unwrap_or_default(),\n                                    )\n                                    .unwrap()\n                                    .into()\n                                }\n                            },\n                        },\n                        _ => Value::Null,\n                    };\n                    blob.insert_unchecked(property, value);\n                }\n\n                // Add result to response\n                response.list.push(blob.into());\n            } else {\n                response.not_found.push(blob_id);\n            }\n        }\n\n        Ok(response)\n    }\n\n    async fn blob_lookup(&self, request: BlobLookupRequest) -> trc::Result<BlobLookupResponse> {\n        let mut include_email = false;\n        let mut include_mailbox = false;\n        let mut include_thread = false;\n\n        let type_names = request\n            .type_names\n            .into_iter()\n            .map(|tn| match tn {\n                MaybeInvalid::Value(value) => {\n                    match &value {\n                        DataType::Email => {\n                            include_email = true;\n                        }\n                        DataType::Mailbox => {\n                            include_mailbox = true;\n                        }\n                        DataType::Thread => {\n                            include_thread = true;\n                        }\n                        _ => (),\n                    }\n\n                    Ok(value)\n                }\n                MaybeInvalid::Invalid(_) => Err(trc::JmapEvent::UnknownDataType.into_err()),\n            })\n            .collect::<Result<Vec<_>, _>>()?;\n        let req_account_id = request.account_id.document_id();\n        let mut response = BlobLookupResponse {\n            account_id: request.account_id,\n            list: Vec::with_capacity(request.ids.len()),\n            not_found: vec![],\n        };\n\n        for id in request.ids.into_valid() {\n            let mut matched_ids = VecMap::new();\n\n            match &id.class {\n                BlobClass::Linked {\n                    account_id,\n                    collection,\n                    document_id,\n                } if *account_id == req_account_id => {\n                    let collection = Collection::from(*collection);\n                    if collection == Collection::Email {\n                        if let Some(data_) = self\n                            .store()\n                            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                                req_account_id,\n                                Collection::Email,\n                                *document_id,\n                            ))\n                            .await?\n                        {\n                            let data = data_\n                                .unarchive::<MessageData>()\n                                .caused_by(trc::location!())?;\n                            if include_email {\n                                matched_ids.append(\n                                    DataType::Email,\n                                    vec![Id::from_parts(u32::from(data.thread_id), *document_id)],\n                                );\n                            }\n                            if include_thread {\n                                matched_ids.append(\n                                    DataType::Thread,\n                                    vec![Id::from(u32::from(data.thread_id))],\n                                );\n                            }\n                            if include_mailbox {\n                                matched_ids.append(\n                                    DataType::Mailbox,\n                                    data.mailboxes\n                                        .iter()\n                                        .map(|m| {\n                                            debug_assert!(m.uid != 0);\n                                            Id::from(u32::from(m.mailbox_id))\n                                        })\n                                        .collect::<Vec<_>>(),\n                                );\n                            }\n                        }\n                    } else {\n                        match DataType::try_from(collection) {\n                            Ok(data_type) if type_names.contains(&data_type) => {\n                                matched_ids.append(data_type, vec![Id::from(*document_id)]);\n                            }\n                            _ => (),\n                        }\n                    }\n                }\n                BlobClass::Reserved { account_id, .. } if *account_id == req_account_id => {}\n                _ => {\n                    response.not_found.push(id);\n                    continue;\n                }\n            }\n\n            response.list.push(BlobInfo { id, matched_ids });\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/blob/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse types::{blob::BlobId, id::Id};\n\npub mod copy;\npub mod download;\npub mod get;\npub mod upload;\n\n#[derive(Debug, serde::Serialize)]\npub struct UploadResponse {\n    #[serde(rename(serialize = \"accountId\"))]\n    account_id: Id,\n    #[serde(rename(serialize = \"blobId\"))]\n    blob_id: BlobId,\n    #[serde(rename(serialize = \"type\"))]\n    c_type: String,\n    size: usize,\n}\n"
  },
  {
    "path": "crates/jmap/src/blob/upload.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse super::{UploadResponse, download::BlobDownload};\nuse common::{Server, auth::AccessToken};\nuse directory::Permission;\nuse jmap_proto::{\n    error::set::SetError,\n    method::upload::{\n        BlobUploadRequest, BlobUploadResponse, BlobUploadResponseObject, DataSourceObject,\n    },\n    request::reference::MaybeIdReference,\n};\nuse std::future::Future;\nuse trc::AddContext;\nuse types::id::Id;\n\n#[cfg(feature = \"test_mode\")]\npub static DISABLE_UPLOAD_QUOTA: std::sync::atomic::AtomicBool =\n    std::sync::atomic::AtomicBool::new(true);\n\npub trait BlobUpload: Sync + Send {\n    fn blob_upload_many(\n        &self,\n        request: BlobUploadRequest,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<BlobUploadResponse>> + Send;\n\n    fn blob_upload(\n        &self,\n        account_id: Id,\n        content_type: &str,\n        data: &[u8],\n        access_token: Arc<AccessToken>,\n    ) -> impl Future<Output = trc::Result<UploadResponse>> + Send;\n}\n\nimpl BlobUpload for Server {\n    async fn blob_upload_many(\n        &self,\n        request: BlobUploadRequest,\n        access_token: &AccessToken,\n    ) -> trc::Result<BlobUploadResponse> {\n        let mut response = BlobUploadResponse {\n            account_id: request.account_id,\n            created: Default::default(),\n            not_created: Default::default(),\n        };\n        let account_id = request.account_id.document_id();\n\n        if request.create.len() > self.core.jmap.set_max_objects {\n            return Err(trc::JmapEvent::RequestTooLarge.into_err());\n        }\n\n        'outer: for (create_id, upload_object) in request.create {\n            let mut data = Vec::new();\n\n            for data_source in upload_object.data {\n                let bytes = match data_source {\n                    DataSourceObject::Id { id, length, offset } => {\n                        let id = match id {\n                            MaybeIdReference::Id(id) => id,\n                            MaybeIdReference::Reference(reference) => {\n                                if let Some(obj) = response.created.get(&reference) {\n                                    obj.id.clone()\n                                } else {\n                                    response.not_created.append(\n                                        create_id,\n                                        SetError::not_found().with_description(format!(\n                                            \"Id reference {reference:?} not found.\"\n                                        )),\n                                    );\n                                    continue 'outer;\n                                }\n                            }\n                            MaybeIdReference::Invalid(id) => {\n                                response.not_created.append(\n                                    create_id,\n                                    SetError::invalid_properties()\n                                        .with_description(format!(\"Invalid blobId {id}.\")),\n                                );\n                                continue 'outer;\n                            }\n                        };\n\n                        if !self.has_access_blob(&id, access_token).await? {\n                            response.not_created.append(\n                                create_id,\n                                SetError::forbidden().with_description(format!(\n                                    \"You do not have access to blobId {id}.\"\n                                )),\n                            );\n                            continue 'outer;\n                        }\n\n                        let offset = offset.unwrap_or(0);\n                        let length = length\n                            .map(|length| length.saturating_add(offset))\n                            .unwrap_or(usize::MAX);\n                        let bytes = if let Some(section) = &id.section {\n                            self.get_blob_section(&id.hash, section)\n                                .await?\n                                .map(|bytes| {\n                                    if offset == 0 && length == usize::MAX {\n                                        bytes\n                                    } else {\n                                        bytes\n                                            .get(offset..std::cmp::min(length, bytes.len()))\n                                            .unwrap_or_default()\n                                            .to_vec()\n                                    }\n                                })\n                        } else {\n                            self.blob_store()\n                                .get_blob(id.hash.as_slice(), offset..length)\n                                .await?\n                        };\n                        if let Some(bytes) = bytes {\n                            bytes\n                        } else {\n                            response.not_created.append(\n                                create_id,\n                                SetError::blob_not_found()\n                                    .with_description(format!(\"BlobId {id} not found.\")),\n                            );\n                            continue 'outer;\n                        }\n                    }\n                    DataSourceObject::Value(bytes) => bytes,\n                    DataSourceObject::Null => {\n                        response.not_created.append(\n                            create_id,\n                            SetError::invalid_properties()\n                                .with_description(\"Invalid DataSourceObject.\"),\n                        );\n                        continue 'outer;\n                    }\n                };\n\n                if bytes.len() + data.len() < self.core.jmap.upload_max_size {\n                    data.extend(bytes);\n                } else {\n                    response.not_created.append(\n                        create_id,\n                        SetError::too_large().with_description(format!(\n                            \"Upload size exceeds maximum of {} bytes.\",\n                            self.core.jmap.upload_max_size\n                        )),\n                    );\n                    continue 'outer;\n                }\n            }\n\n            if data.is_empty() {\n                response.not_created.append(\n                    create_id,\n                    SetError::invalid_properties()\n                        .with_description(\"Must specify at least one valid DataSourceObject.\"),\n                );\n                continue 'outer;\n            }\n\n            // Enforce quota\n            let used = self\n                .core\n                .storage\n                .data\n                .blob_quota(account_id)\n                .await\n                .caused_by(trc::location!())?;\n\n            if ((self.core.jmap.upload_tmp_quota_size > 0\n                && used.bytes + data.len() > self.core.jmap.upload_tmp_quota_size)\n                || (self.core.jmap.upload_tmp_quota_amount > 0\n                    && used.count + 1 > self.core.jmap.upload_tmp_quota_amount))\n                && !access_token.has_permission(Permission::UnlimitedUploads)\n            {\n                response.not_created.append(\n                    create_id,\n                    SetError::over_quota().with_description(format!(\n                        \"You have exceeded the blob upload quota of {} files or {} bytes.\",\n                        self.core.jmap.upload_tmp_quota_amount,\n                        self.core.jmap.upload_tmp_quota_size\n                    )),\n                );\n                continue 'outer;\n            }\n\n            // Write blob\n            response.created.insert(\n                create_id,\n                BlobUploadResponseObject {\n                    id: self.put_jmap_blob(account_id, &data).await?,\n                    type_: upload_object.type_,\n                    size: data.len(),\n                },\n            );\n        }\n\n        Ok(response)\n    }\n\n    async fn blob_upload(\n        &self,\n        account_id: Id,\n        content_type: &str,\n        data: &[u8],\n        access_token: Arc<AccessToken>,\n    ) -> trc::Result<UploadResponse> {\n        // Limit concurrent uploads\n        let _in_flight = self\n            .is_upload_allowed(&access_token)\n            .caused_by(trc::location!())?;\n\n        #[cfg(feature = \"test_mode\")]\n        {\n            // Used for concurrent upload tests\n            if data == b\"sleep\" {\n                tokio::time::sleep(std::time::Duration::from_secs(1)).await;\n            }\n        }\n\n        // Enforce quota\n        let used = self\n            .core\n            .storage\n            .data\n            .blob_quota(account_id.document_id())\n            .await\n            .caused_by(trc::location!())?;\n\n        if ((self.core.jmap.upload_tmp_quota_size > 0\n            && used.bytes + data.len() > self.core.jmap.upload_tmp_quota_size)\n            || (self.core.jmap.upload_tmp_quota_amount > 0\n                && used.count + 1 > self.core.jmap.upload_tmp_quota_amount))\n            && !access_token.has_permission(Permission::UnlimitedUploads)\n        {\n            let err = Err(trc::LimitEvent::BlobQuota\n                .into_err()\n                .ctx(trc::Key::Size, self.core.jmap.upload_tmp_quota_size)\n                .ctx(trc::Key::Total, self.core.jmap.upload_tmp_quota_amount));\n\n            #[cfg(feature = \"test_mode\")]\n            if !DISABLE_UPLOAD_QUOTA.load(std::sync::atomic::Ordering::Relaxed) {\n                return err;\n            }\n\n            #[cfg(not(feature = \"test_mode\"))]\n            return err;\n        }\n\n        Ok(UploadResponse {\n            account_id,\n            blob_id: self\n                .put_jmap_blob(account_id.document_id(), data)\n                .await\n                .caused_by(trc::location!())?,\n            c_type: content_type.to_string(),\n            size: data.len(),\n        })\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::acl::JmapRights, calendar::Availability, changes::state::JmapCacheState};\nuse calcard::jscalendar::{JSCalendarAlertAction, JSCalendarRelativeTo, JSCalendarType};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{\n        ALERT_EMAIL, ALERT_RELATIVE_TO_END, ArchivedDefaultAlert, CALENDAR_INVISIBLE,\n        CALENDAR_SUBSCRIBED, Calendar,\n    },\n};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::calendar::{self, CalendarProperty, CalendarValue, IncludeInAvailability},\n};\nuse jmap_tools::{Key, Map, Value};\nuse store::{ValueKey, roaring::RoaringBitmap, write::{AlignedBytes, Archive, ValueClass}};\nuse trc::AddContext;\nuse types::{\n    acl::{Acl, AclGrant},\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n};\n\npub trait CalendarGet: Sync + Send {\n    fn calendar_get(\n        &self,\n        request: GetRequest<calendar::Calendar>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<calendar::Calendar>>> + Send;\n}\n\nimpl CalendarGet for Server {\n    async fn calendar_get(\n        &self,\n        mut request: GetRequest<calendar::Calendar>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<calendar::Calendar>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::Description,\n            CalendarProperty::Color,\n            CalendarProperty::TimeZone,\n            CalendarProperty::SortOrder,\n            CalendarProperty::IsDefault,\n            CalendarProperty::IsSubscribed,\n            CalendarProperty::MyRights,\n        ]);\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await?;\n        let is_owner = access_token.is_member(account_id);\n        let calendar_ids = if is_owner {\n            cache.document_ids(true).collect::<RoaringBitmap>()\n        } else {\n            cache.shared_containers(access_token, [Acl::Read, Acl::ReadItems], true)\n        };\n        let default_calendar_id = self\n            .store()\n            .get_value::<u32>(ValueKey {\n                account_id,\n                collection: Collection::Principal.into(),\n                document_id: 0,\n                class: ValueClass::Property(PrincipalField::DefaultCalendarId.into()),\n            })\n            .await\n            .caused_by(trc::location!())?\n            .or_else(|| {\n                if calendar_ids.len() == 1 {\n                    calendar_ids.iter().next()\n                } else {\n                    None\n                }\n            });\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            calendar_ids\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: cache.get_state(true).into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the calendar object\n            let document_id = id.document_id();\n            if !calendar_ids.contains(document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n            let _calendar = if let Some(calendar) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Calendar,\n                    document_id,\n                ))\n                .await?\n            {\n                calendar\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n            let calendar = _calendar\n                .unarchive::<Calendar>()\n                .caused_by(trc::location!())?;\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                match property {\n                    CalendarProperty::Id => {\n                        result.insert_unchecked(CalendarProperty::Id, CalendarValue::Id(id));\n                    }\n                    CalendarProperty::Name => {\n                        result.insert_unchecked(\n                            CalendarProperty::Name,\n                            calendar.preferences(access_token).name.to_string(),\n                        );\n                    }\n                    CalendarProperty::Description => {\n                        result.insert_unchecked(\n                            CalendarProperty::Description,\n                            calendar\n                                .preferences(access_token)\n                                .description\n                                .as_ref()\n                                .map(|v| v.to_string()),\n                        );\n                    }\n                    CalendarProperty::SortOrder => {\n                        result.insert_unchecked(\n                            CalendarProperty::SortOrder,\n                            calendar.preferences(access_token).sort_order.to_native(),\n                        );\n                    }\n                    CalendarProperty::IsDefault => {\n                        result.insert_unchecked(\n                            CalendarProperty::IsDefault,\n                            default_calendar_id == Some(document_id),\n                        );\n                    }\n                    CalendarProperty::IsSubscribed => {\n                        result.insert_unchecked(\n                            CalendarProperty::IsSubscribed,\n                            Value::Bool(\n                                calendar.preferences(access_token).flags & CALENDAR_SUBSCRIBED != 0,\n                            ),\n                        );\n                    }\n                    CalendarProperty::Color => {\n                        result.insert_unchecked(\n                            CalendarProperty::Color,\n                            calendar\n                                .preferences(access_token)\n                                .color\n                                .as_ref()\n                                .map(|c| c.to_string()),\n                        );\n                    }\n                    CalendarProperty::IsVisible => {\n                        result.insert_unchecked(\n                            CalendarProperty::IsVisible,\n                            Value::Bool(\n                                calendar.preferences(access_token).flags & CALENDAR_INVISIBLE == 0,\n                            ),\n                        );\n                    }\n                    CalendarProperty::IncludeInAvailability => {\n                        result.insert_unchecked(\n                            CalendarProperty::IncludeInAvailability,\n                            Value::Element(CalendarValue::IncludeInAvailability(\n                                IncludeInAvailability::from_flags(\n                                    calendar.preferences(access_token).flags.to_native(),\n                                )\n                                .unwrap_or(if is_owner {\n                                    IncludeInAvailability::All\n                                } else {\n                                    IncludeInAvailability::None\n                                }),\n                            )),\n                        );\n                    }\n                    CalendarProperty::DefaultAlertsWithTime => {\n                        result.insert_unchecked(\n                            CalendarProperty::DefaultAlertsWithTime,\n                            Value::Object(Map::from_iter(\n                                calendar\n                                    .default_alerts(access_token, true)\n                                    .map(default_alarm_to_value),\n                            )),\n                        );\n                    }\n                    CalendarProperty::DefaultAlertsWithoutTime => {\n                        result.insert_unchecked(\n                            CalendarProperty::DefaultAlertsWithoutTime,\n                            Value::Object(Map::from_iter(\n                                calendar\n                                    .default_alerts(access_token, false)\n                                    .map(default_alarm_to_value),\n                            )),\n                        );\n                    }\n                    CalendarProperty::TimeZone => {\n                        result.insert_unchecked(\n                            CalendarProperty::TimeZone,\n                            calendar\n                                .preferences(access_token)\n                                .time_zone\n                                .tz()\n                                .map(|tz| Value::Element(CalendarValue::Timezone(tz)))\n                                .unwrap_or(Value::Null),\n                        );\n                    }\n                    CalendarProperty::ShareWith => {\n                        result.insert_unchecked(\n                            CalendarProperty::ShareWith,\n                            JmapRights::share_with::<calendar::Calendar>(\n                                account_id,\n                                access_token,\n                                &calendar.acls.iter().map(AclGrant::from).collect::<Vec<_>>(),\n                            ),\n                        );\n                    }\n                    CalendarProperty::MyRights => {\n                        result.insert_unchecked(\n                            CalendarProperty::MyRights,\n                            if access_token.is_shared(account_id) {\n                                JmapRights::rights::<calendar::Calendar>(\n                                    calendar.acls.effective_acl(access_token),\n                                )\n                            } else {\n                                JmapRights::all_rights::<calendar::Calendar>()\n                            },\n                        );\n                    }\n                    property => {\n                        result.insert_unchecked(property.clone(), Value::Null);\n                    }\n                }\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n\nfn default_alarm_to_value(\n    alarm: &ArchivedDefaultAlert,\n) -> (\n    Key<'static, CalendarProperty>,\n    Value<'static, CalendarProperty, CalendarValue>,\n) {\n    (\n        Key::Owned(alarm.id.to_string()),\n        Value::Object(Map::from(vec![\n            (\n                Key::Property(CalendarProperty::Type),\n                Value::Element(CalendarValue::Type(JSCalendarType::Alert)),\n            ),\n            (\n                Key::Property(CalendarProperty::Action),\n                Value::Element(CalendarValue::Action(if alarm.flags & ALERT_EMAIL != 0 {\n                    JSCalendarAlertAction::Email\n                } else {\n                    JSCalendarAlertAction::Display\n                })),\n            ),\n            (\n                Key::Property(CalendarProperty::Trigger),\n                Value::Object(Map::from(vec![\n                    (\n                        Key::Property(CalendarProperty::Type),\n                        Value::Element(CalendarValue::Type(JSCalendarType::OffsetTrigger)),\n                    ),\n                    (\n                        Key::Property(CalendarProperty::Offset),\n                        Value::Element(CalendarValue::Duration(alarm.offset.to_native())),\n                    ),\n                    (\n                        Key::Property(CalendarProperty::RelativeTo),\n                        Value::Element(CalendarValue::RelativeTo(\n                            if alarm.flags & ALERT_RELATIVE_TO_END != 0 {\n                                JSCalendarRelativeTo::End\n                            } else {\n                                JSCalendarRelativeTo::Start\n                            },\n                        )),\n                    ),\n                ])),\n            ),\n        ])),\n    )\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse groupware::calendar::{\n    CALENDAR_AVAILABILITY_ALL, CALENDAR_AVAILABILITY_ATTENDING, CALENDAR_AVAILABILITY_NONE,\n};\nuse jmap_proto::object::calendar::IncludeInAvailability;\n\npub mod get;\npub mod set;\n\npub(crate) trait Availability: Sized {\n    fn from_flags(flags: u16) -> Option<Self>;\n}\n\nimpl Availability for IncludeInAvailability {\n    fn from_flags(flags: u16) -> Option<Self> {\n        if flags & CALENDAR_AVAILABILITY_ALL != 0 {\n            Some(IncludeInAvailability::All)\n        } else if flags & CALENDAR_AVAILABILITY_ATTENDING != 0 {\n            Some(IncludeInAvailability::Attending)\n        } else if flags & CALENDAR_AVAILABILITY_NONE != 0 {\n            Some(IncludeInAvailability::None)\n        } else {\n            None\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::api::acl::{JmapAcl, JmapRights};\nuse calcard::jscalendar::{JSCalendarAlertAction, JSCalendarRelativeTo, JSCalendarType};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse groupware::{\n    DestroyArchive,\n    cache::GroupwareCache,\n    calendar::{\n        ALERT_EMAIL, ALERT_RELATIVE_TO_END, ALERT_WITH_TIME, CALENDAR_AVAILABILITY_ALL,\n        CALENDAR_AVAILABILITY_ATTENDING, CALENDAR_AVAILABILITY_NONE, CALENDAR_INVISIBLE,\n        CALENDAR_SUBSCRIBED, Calendar, CalendarEvent, CalendarPreferences, DefaultAlert, Timezone,\n    },\n};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::set::{SetRequest, SetResponse},\n    object::calendar::{self, CalendarProperty, CalendarValue, IncludeInAvailability},\n    request::{IntoValid, reference::MaybeIdReference},\n    types::state::State,\n};\nuse jmap_tools::{JsonPointerItem, Key, Map, Value};\nuse rand::{Rng, distr::Alphanumeric};\nuse store::{\n    SerializeInfallible, ValueKey,\n    ahash::AHashSet,\n    write::{AlignedBytes, Archive, BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n};\n\npub trait CalendarSet: Sync + Send {\n    fn calendar_set(\n        &self,\n        request: SetRequest<'_, calendar::Calendar>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<SetResponse<calendar::Calendar>>> + Send;\n}\n\nimpl CalendarSet for Server {\n    async fn calendar_set(\n        &self,\n        mut request: SetRequest<'_, calendar::Calendar>,\n        access_token: &AccessToken,\n        _session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<calendar::Calendar>> {\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n        let is_shared = access_token.is_shared(account_id);\n        let mut set_default = None;\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        'create: for (id, object) in request.unwrap_create() {\n            if is_shared {\n                response.not_created.append(\n                    id,\n                    SetError::forbidden()\n                        .with_description(\"Cannot create calendars in a shared account.\"),\n                );\n                continue 'create;\n            }\n\n            let mut calendar = Calendar {\n                name: rand::rng()\n                    .sample_iter(Alphanumeric)\n                    .take(10)\n                    .map(char::from)\n                    .collect::<String>(),\n                preferences: vec![CalendarPreferences {\n                    account_id,\n                    name: \"\".to_string(),\n                    ..Default::default()\n                }],\n                ..Default::default()\n            };\n\n            // Process changes\n            if let Err(err) = update_calendar(object, &mut calendar, access_token) {\n                response.not_created.append(id, err);\n                continue 'create;\n            }\n\n            // Validate ACLs\n            if !calendar.acls.is_empty() {\n                if let Err(err) = self.acl_validate(&calendar.acls).await {\n                    response.not_created.append(id, err.into());\n                    continue 'create;\n                }\n\n                self.refresh_acls(&calendar.acls, None).await;\n            }\n\n            // Insert record\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::Calendar, 1)\n                .await\n                .caused_by(trc::location!())?;\n            calendar\n                .insert(access_token, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?;\n\n            if let Some(MaybeIdReference::Reference(id_ref)) =\n                &request.arguments.on_success_set_is_default\n                && id_ref == &id\n            {\n                set_default = Some(document_id);\n            }\n\n            response.created(id, document_id);\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain calendar\n            let document_id = id.document_id();\n            let calendar_ = if let Some(calendar_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Calendar,\n                    document_id,\n                ))\n                .await?\n            {\n                calendar_\n            } else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n            let calendar = calendar_\n                .to_unarchived::<Calendar>()\n                .caused_by(trc::location!())?;\n            let mut new_calendar = calendar\n                .deserialize::<Calendar>()\n                .caused_by(trc::location!())?;\n\n            // Apply changes\n            let has_acl_changes = match update_calendar(object, &mut new_calendar, access_token) {\n                Ok(has_acl_changes_) => has_acl_changes_,\n                Err(err) => {\n                    response.not_updated.append(id, err);\n                    continue 'update;\n                }\n            };\n\n            // Validate ACL\n            if is_shared {\n                let acl = calendar.inner.acls.effective_acl(access_token);\n                if !acl.contains(Acl::Modify) || (has_acl_changes && !acl.contains(Acl::Share)) {\n                    response.not_updated.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to modify this calendar.\"),\n                    );\n                    continue 'update;\n                }\n            }\n            if has_acl_changes {\n                if let Err(err) = self.acl_validate(&new_calendar.acls).await {\n                    response.not_updated.append(id, err.into());\n                    continue 'update;\n                }\n                self.refresh_archived_acls(&new_calendar.acls, calendar.inner.acls.as_slice())\n                    .await;\n            }\n\n            // Update record\n            new_calendar\n                .update(access_token, calendar, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?;\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        let mut reset_default_calendar = false;\n        if !will_destroy.is_empty() {\n            let mut destroy_children = AHashSet::new();\n            let mut destroy_parents = AHashSet::new();\n            let default_calendar_id = self\n                .store()\n                .get_value::<u32>(ValueKey {\n                    account_id,\n                    collection: Collection::Principal.into(),\n                    document_id: 0,\n                    class: ValueClass::Property(PrincipalField::DefaultCalendarId.into()),\n                })\n                .await\n                .caused_by(trc::location!())?;\n            let on_destroy_remove_events =\n                request.arguments.on_destroy_remove_events.unwrap_or(false);\n            for id in will_destroy {\n                let document_id = id.document_id();\n\n                if !cache.has_container_id(&document_id) {\n                    response.not_destroyed.append(id, SetError::not_found());\n                    continue;\n                };\n\n                let Some(calendar_) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::Calendar,\n                        document_id,\n                    ))\n                    .await\n                    .caused_by(trc::location!())?\n                else {\n                    response.not_destroyed.append(id, SetError::not_found());\n                    continue;\n                };\n\n                let calendar = calendar_\n                    .to_unarchived::<Calendar>()\n                    .caused_by(trc::location!())?;\n\n                // Validate ACLs\n                if is_shared\n                    && !calendar\n                        .inner\n                        .acls\n                        .effective_acl(access_token)\n                        .contains_all([Acl::Delete, Acl::RemoveItems].into_iter())\n                {\n                    response.not_destroyed.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to delete this calendar.\"),\n                    );\n                    continue;\n                }\n\n                // Obtain children ids\n                let children_ids = cache.children_ids(document_id).collect::<Vec<_>>();\n                if !children_ids.is_empty() && !on_destroy_remove_events {\n                    response\n                        .not_destroyed\n                        .append(id, SetError::calendar_has_event());\n                    continue;\n                }\n                destroy_children.extend(children_ids.iter().copied());\n                destroy_parents.insert(document_id);\n\n                // Delete record\n                DestroyArchive(calendar)\n                    .delete(access_token, account_id, document_id, None, &mut batch)\n                    .caused_by(trc::location!())?;\n\n                if default_calendar_id == Some(document_id) {\n                    reset_default_calendar = true;\n                }\n\n                response.destroyed.push(id);\n            }\n\n            // Delete children\n            if !destroy_children.is_empty() {\n                for document_id in destroy_children {\n                    if let Some(event_) = self\n                        .store()\n                        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                            account_id,\n                            Collection::CalendarEvent,\n                            document_id,\n                        ))\n                        .await?\n                    {\n                        let event = event_\n                            .to_unarchived::<CalendarEvent>()\n                            .caused_by(trc::location!())?;\n\n                        if event\n                            .inner\n                            .names\n                            .iter()\n                            .all(|n| destroy_parents.contains(&n.parent_id.to_native()))\n                        {\n                            // Event only belongs to calendars being deleted, delete it\n                            DestroyArchive(event).delete_all(\n                                access_token,\n                                account_id,\n                                document_id,\n                                false,\n                                &mut batch,\n                            )?;\n                        } else {\n                            // Unlink calendar id from event\n                            let mut new_event = event\n                                .deserialize::<CalendarEvent>()\n                                .caused_by(trc::location!())?;\n                            new_event\n                                .names\n                                .retain(|n| !destroy_parents.contains(&n.parent_id));\n                            new_event.update(\n                                access_token,\n                                event,\n                                account_id,\n                                document_id,\n                                &mut batch,\n                            )?;\n                        }\n                    }\n                }\n            }\n        }\n\n        // Set default calendar\n        if let Some(MaybeIdReference::Id(id)) = &request.arguments.on_success_set_is_default {\n            set_default = Some(id.document_id());\n        }\n        if let Some(default_calendar_id) = set_default {\n            if response.not_created.is_empty()\n                && response.not_updated.is_empty()\n                && response.not_destroyed.is_empty()\n            {\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .set(\n                        PrincipalField::DefaultCalendarId,\n                        default_calendar_id.serialize(),\n                    );\n            }\n        } else if reset_default_calendar {\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .clear(PrincipalField::DefaultCalendarId);\n        }\n\n        // Write changes\n        if !batch.is_empty()\n            && let Ok(change_id) = self\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?\n                .last_change_id(account_id)\n        {\n            self.notify_task_queue();\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n}\n\nfn update_calendar(\n    updates: Value<'_, CalendarProperty, CalendarValue>,\n    calendar: &mut Calendar,\n    access_token: &AccessToken,\n) -> Result<bool, SetError<CalendarProperty>> {\n    let mut has_acl_changes = false;\n\n    for (property, value) in updates.into_expanded_object() {\n        let Key::Property(property) = property else {\n            return Err(SetError::invalid_properties()\n                .with_property(property.to_owned())\n                .with_description(\"Invalid property.\"));\n        };\n\n        match (property, value) {\n            (CalendarProperty::Name, Value::Str(value)) if (1..=255).contains(&value.len()) => {\n                calendar.preferences_mut(access_token).name = value.into_owned();\n            }\n            (CalendarProperty::Description, Value::Str(value)) if value.len() < 255 => {\n                calendar.preferences_mut(access_token).description = value.into_owned().into();\n            }\n            (CalendarProperty::Description, Value::Null) => {\n                calendar.preferences_mut(access_token).description = None;\n            }\n            (CalendarProperty::Color, Value::Str(value)) if value.len() < 16 => {\n                calendar.preferences_mut(access_token).color = value.into_owned().into();\n            }\n            (CalendarProperty::Color, Value::Null) => {\n                calendar.preferences_mut(access_token).color = None;\n            }\n            (CalendarProperty::TimeZone, Value::Element(CalendarValue::Timezone(tz))) => {\n                calendar.preferences_mut(access_token).time_zone = Timezone::IANA(tz.as_id());\n            }\n            (CalendarProperty::TimeZone, Value::Null) => {\n                calendar.preferences_mut(access_token).time_zone = Timezone::Default;\n            }\n            (CalendarProperty::SortOrder, Value::Number(value)) => {\n                calendar.preferences_mut(access_token).sort_order = value.cast_to_u64() as u32;\n            }\n            (CalendarProperty::IsSubscribed, Value::Bool(subscribe)) => {\n                if subscribe {\n                    calendar.preferences_mut(access_token).flags |= CALENDAR_SUBSCRIBED;\n                } else {\n                    calendar.preferences_mut(access_token).flags &= !CALENDAR_SUBSCRIBED;\n                }\n            }\n            (CalendarProperty::IsVisible, Value::Bool(visible)) => {\n                if visible {\n                    calendar.preferences_mut(access_token).flags &= !CALENDAR_INVISIBLE;\n                } else {\n                    calendar.preferences_mut(access_token).flags |= CALENDAR_INVISIBLE;\n                }\n            }\n            (\n                CalendarProperty::IncludeInAvailability,\n                Value::Element(CalendarValue::IncludeInAvailability(availability)),\n            ) => {\n                let flags = &mut calendar.preferences_mut(access_token).flags;\n\n                match availability {\n                    IncludeInAvailability::All => {\n                        *flags &= !(CALENDAR_AVAILABILITY_NONE | CALENDAR_AVAILABILITY_ATTENDING);\n                        *flags |= CALENDAR_AVAILABILITY_ALL;\n                    }\n                    IncludeInAvailability::Attending => {\n                        *flags &= !(CALENDAR_AVAILABILITY_NONE | CALENDAR_AVAILABILITY_ALL);\n                        *flags |= CALENDAR_AVAILABILITY_ATTENDING;\n                    }\n                    IncludeInAvailability::None => {\n                        *flags &= !(CALENDAR_AVAILABILITY_ATTENDING | CALENDAR_AVAILABILITY_ALL);\n                        *flags |= CALENDAR_AVAILABILITY_NONE;\n                    }\n                }\n            }\n            (\n                property @ (CalendarProperty::DefaultAlertsWithTime\n                | CalendarProperty::DefaultAlertsWithoutTime),\n                Value::Object(value),\n            ) => {\n                let with_time = matches!(property, CalendarProperty::DefaultAlertsWithTime);\n                let alerts = &mut calendar.preferences_mut(access_token).default_alerts;\n\n                alerts.retain(|alert| (alert.flags & ALERT_WITH_TIME != 0) != with_time);\n\n                for (key, value) in value.into_vec() {\n                    if let Value::Object(value) = value {\n                        alerts.push(value_to_default_alert(\n                            key.to_string().into_owned(),\n                            value,\n                            with_time,\n                        )?);\n                    }\n                }\n            }\n            (CalendarProperty::ShareWith, value) => {\n                calendar.acls = JmapRights::acl_set::<calendar::Calendar>(value)?;\n                has_acl_changes = true;\n            }\n            (CalendarProperty::Pointer(pointer), value) => {\n                let mut ptr_iter = pointer.iter();\n\n                match ptr_iter.next() {\n                    Some(JsonPointerItem::Key(Key::Property(CalendarProperty::ShareWith))) => {\n                        calendar.acls = JmapRights::acl_patch::<calendar::Calendar>(\n                            std::mem::take(&mut calendar.acls),\n                            ptr_iter,\n                            value,\n                        )?;\n                        has_acl_changes = true;\n                    }\n                    Some(JsonPointerItem::Key(Key::Property(\n                        property @ (CalendarProperty::DefaultAlertsWithTime\n                        | CalendarProperty::DefaultAlertsWithoutTime),\n                    ))) => match (ptr_iter.next(), ptr_iter.next()) {\n                        (\n                            Some(key @ (JsonPointerItem::Key(_) | JsonPointerItem::Number(_))),\n                            None,\n                        ) => {\n                            let id = match key {\n                                JsonPointerItem::Key(key) => key.to_string().into_owned(),\n                                JsonPointerItem::Number(n) => n.to_string(),\n                                _ => unreachable!(),\n                            };\n                            let with_time =\n                                matches!(property, CalendarProperty::DefaultAlertsWithTime);\n                            let alerts = &mut calendar.preferences_mut(access_token).default_alerts;\n                            alerts.retain(|alert| {\n                                (alert.flags & ALERT_WITH_TIME != 0) != with_time || alert.id != id\n                            });\n\n                            if let Value::Object(value) = value {\n                                alerts.push(value_to_default_alert(id, value, with_time)?);\n                            }\n                        }\n                        _ => {\n                            return Err(SetError::invalid_properties()\n                                .with_property(CalendarProperty::Pointer(pointer))\n                                .with_description(\"Field could not be patched.\"));\n                        }\n                    },\n                    _ => {\n                        return Err(SetError::invalid_properties()\n                            .with_property(CalendarProperty::Pointer(pointer))\n                            .with_description(\"Field could not be patched.\"));\n                    }\n                }\n            }\n            (property, _) => {\n                return Err(SetError::invalid_properties()\n                    .with_property(property)\n                    .with_description(\"Field could not be set.\"));\n            }\n        }\n    }\n\n    // Validate name\n    if calendar.preferences(access_token).name.is_empty() {\n        return Err(SetError::invalid_properties()\n            .with_property(CalendarProperty::Name)\n            .with_description(\"Missing name.\"));\n    }\n\n    Ok(has_acl_changes)\n}\n\nfn value_to_default_alert(\n    id: String,\n    value: Map<'_, CalendarProperty, CalendarValue>,\n    with_time: bool,\n) -> Result<DefaultAlert, SetError<CalendarProperty>> {\n    let mut alert = DefaultAlert {\n        id,\n        ..Default::default()\n    };\n    let mut has_offset = false;\n\n    for (key, value) in value.into_vec() {\n        let Key::Property(key) = key else {\n            continue;\n        };\n\n        match (key, value) {\n            (CalendarProperty::Type, Value::Element(CalendarValue::Type(value))) => {\n                if value != JSCalendarType::Alert {\n                    return Err(SetError::invalid_properties()\n                        .with_property(CalendarProperty::Trigger)\n                        .with_description(\"Invalid alert object type.\"));\n                }\n            }\n            (\n                CalendarProperty::Action,\n                Value::Element(CalendarValue::Action(JSCalendarAlertAction::Email)),\n            ) => {\n                alert.flags |= ALERT_EMAIL;\n            }\n            (CalendarProperty::Trigger, Value::Object(value)) => {\n                for (key, value) in value.into_vec() {\n                    let Key::Property(key) = key else {\n                        continue;\n                    };\n\n                    match (key, value) {\n                        (\n                            CalendarProperty::RelativeTo,\n                            Value::Element(CalendarValue::RelativeTo(JSCalendarRelativeTo::End)),\n                        ) => {\n                            alert.flags |= ALERT_RELATIVE_TO_END;\n                        }\n                        (\n                            CalendarProperty::Offset,\n                            Value::Element(CalendarValue::Duration(value)),\n                        ) => {\n                            alert.offset = value;\n                            has_offset = true;\n                        }\n                        (CalendarProperty::Offset, Value::Element(CalendarValue::Type(value))) => {\n                            if value != JSCalendarType::OffsetTrigger {\n                                return Err(SetError::invalid_properties()\n                                    .with_property(CalendarProperty::Trigger)\n                                    .with_description(\"Invalid alert trigger type.\"));\n                            }\n                        }\n                        _ => {}\n                    }\n                }\n            }\n            _ => {}\n        }\n    }\n\n    if has_offset {\n        if with_time {\n            alert.flags |= ALERT_WITH_TIME;\n        }\n\n        Ok(alert)\n    } else {\n        Err(SetError::invalid_properties()\n            .with_property(CalendarProperty::Trigger)\n            .with_description(\"Missing alert offset.\"))\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event/copy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    calendar_event::{CalendarSyntheticId, set::CalendarEventSet},\n    changes::state::JmapCacheState,\n};\nuse calcard::jscalendar::JSCalendarProperty;\nuse common::{Server, auth::AccessToken};\nuse groupware::{cache::GroupwareCache, calendar::CalendarEvent};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::{\n        copy::{CopyRequest, CopyResponse},\n        set::SetRequest,\n    },\n    object::calendar_event,\n    request::{\n        Call, IntoValid, MaybeInvalid, RequestMethod, SetRequestMethod,\n        method::{MethodFunction, MethodName, MethodObject},\n        reference::MaybeResultReference,\n    },\n    types::state::State,\n};\nuse store::{ValueKey, roaring::RoaringBitmap, write::{AlignedBytes, Archive, BatchBuilder}};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\nuse utils::map::vec_map::VecMap;\n\npub trait JmapCalendarEventCopy: Sync + Send {\n    fn calendar_event_copy<'x>(\n        &self,\n        request: CopyRequest<'x, calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<CopyResponse<calendar_event::CalendarEvent>>> + Send;\n}\n\nimpl JmapCalendarEventCopy for Server {\n    async fn calendar_event_copy<'x>(\n        &self,\n        request: CopyRequest<'x, calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        _session: &HttpSessionData,\n    ) -> trc::Result<CopyResponse<calendar_event::CalendarEvent>> {\n        let account_id = request.account_id.document_id();\n        let from_account_id = request.from_account_id.document_id();\n\n        if account_id == from_account_id {\n            return Err(trc::JmapEvent::InvalidArguments\n                .into_err()\n                .details(\"From accountId is equal to fromAccountId\"));\n        }\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let old_state = cache.assert_state(false, &request.if_in_state)?;\n        let mut response = CopyResponse {\n            from_account_id: request.from_account_id,\n            account_id: request.account_id,\n            new_state: old_state.clone(),\n            old_state,\n            created: VecMap::with_capacity(request.create.len()),\n            not_created: VecMap::new(),\n        };\n\n        let from_cache = self\n            .fetch_dav_resources(access_token, from_account_id, SyncCollection::Calendar)\n            .await\n            .caused_by(trc::location!())?;\n        let from_calendar_event_ids = if access_token.is_member(from_account_id) {\n            from_cache.document_ids(false).collect::<RoaringBitmap>()\n        } else {\n            from_cache.shared_items(access_token, [Acl::ReadItems], true)\n        };\n\n        let can_add_calendars = if access_token.is_shared(account_id) {\n            cache\n                .shared_containers(access_token, [Acl::AddItems], true)\n                .into()\n        } else {\n            None\n        };\n        let on_success_delete = request.on_success_destroy_original.unwrap_or(false);\n        let mut destroy_ids = Vec::new();\n\n        // Obtain quota\n        let mut batch = BatchBuilder::new();\n\n        'create: for (id, create) in request.create.into_valid() {\n            let from_calendar_event_id = id.document_id();\n            if !from_calendar_event_ids.contains(from_calendar_event_id) {\n                response.not_created.append(\n                    id,\n                    SetError::not_found().with_description(format!(\n                        \"Item {} not found in account {}.\",\n                        id, response.from_account_id\n                    )),\n                );\n                continue;\n            }\n            if id.is_synthetic() {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(JSCalendarProperty::Id)\n                        .with_description(format!(\n                            \"Item {} is a synthetic id and cannot be copied.\",\n                            id\n                        )),\n                );\n                continue;\n            }\n\n            let Some(_calendar_event) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    from_account_id,\n                    Collection::CalendarEvent,\n                    from_calendar_event_id,\n                ))\n                .await?\n            else {\n                response.not_created.append(\n                    id,\n                    SetError::not_found().with_description(format!(\n                        \"Item {} not found in account {}.\",\n                        id, response.from_account_id\n                    )),\n                );\n                continue;\n            };\n\n            let calendar_event = _calendar_event\n                .deserialize::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            match self\n                .create_calendar_event(\n                    &cache,\n                    &mut batch,\n                    access_token,\n                    account_id,\n                    false,\n                    &can_add_calendars,\n                    calendar_event.data.event.into_jscalendar(),\n                    create,\n                )\n                .await?\n            {\n                Ok(document_id) => {\n                    response.created(id, document_id);\n\n                    // Add to destroy list\n                    if on_success_delete {\n                        destroy_ids.push(MaybeInvalid::Value(id));\n                    }\n                }\n                Err(err) => {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                }\n            }\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n            self.notify_task_queue();\n\n            response.new_state = State::Exact(change_id);\n        }\n\n        // Destroy ids\n        if on_success_delete && !destroy_ids.is_empty() {\n            *next_call = Call {\n                id: String::new(),\n                name: MethodName::new(MethodObject::CalendarEvent, MethodFunction::Set),\n                method: RequestMethod::Set(SetRequestMethod::CalendarEvent(SetRequest {\n                    account_id: request.from_account_id,\n                    if_in_state: request.destroy_from_if_in_state,\n                    create: None,\n                    update: None,\n                    destroy: MaybeResultReference::Value(destroy_ids).into(),\n                    arguments: Default::default(),\n                })),\n            }\n            .into();\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{calendar_event::CalendarSyntheticId, changes::state::JmapCacheState};\nuse calcard::{\n    common::{PartialDateTime, timezone::Tz},\n    icalendar::{\n        ICalendar, ICalendarComponent, ICalendarComponentType, ICalendarEntry, ICalendarParameter,\n        ICalendarParameterName, ICalendarParameterValue, ICalendarParticipationRole,\n        ICalendarProperty, ICalendarValue,\n    },\n    jscalendar::{\n        JSCalendarDateTime, JSCalendarProperty, JSCalendarValue, import::ConversionOptions,\n    },\n};\nuse chrono::DateTime;\nuse common::{Server, auth::AccessToken};\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{\n        CalendarEvent, EVENT_DRAFT, EVENT_HIDE_ATTENDEES, EVENT_INVITE_OTHERS, EVENT_INVITE_SELF,\n        PREF_USE_DEFAULT_ALERTS, expand::CalendarEventExpansion,\n    },\n};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::{JmapObjectId, calendar_event},\n    request::{IntoValid, reference::MaybeResultReference},\n};\nuse jmap_tools::{Key, Map, Value};\nuse std::{str::FromStr, sync::Arc};\nuse store::{\n    ValueKey,\n    ahash::{AHashMap, AHashSet},\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    blob::BlobId,\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub trait CalendarEventGet: Sync + Send {\n    fn calendar_event_get(\n        &self,\n        request: GetRequest<calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<calendar_event::CalendarEvent>>> + Send;\n}\n\nimpl CalendarEventGet for Server {\n    async fn calendar_event_get(\n        &self,\n        mut request: GetRequest<calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<calendar_event::CalendarEvent>> {\n        let return_all_properties = request\n            .properties\n            .as_ref()\n            .is_none_or(|v| matches!(v, MaybeResultReference::Value(v) if v.is_empty()));\n        let properties = request.unwrap_properties(&[]);\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await?;\n        let calendar_event_ids = if access_token.is_member(account_id) {\n            cache.document_ids(false).collect::<RoaringBitmap>()\n        } else {\n            cache.shared_items(access_token, [Acl::ReadItems], true)\n        };\n        let (mut ids, has_synthetic_ids) = if let Some(rr) = request.ids.take() {\n            let rr = rr.unwrap();\n            if rr.len() > self.core.jmap.get_max_objects {\n                return Err(trc::JmapEvent::RequestTooLarge.into_err());\n            }\n            let mut ids = Vec::with_capacity(rr.len());\n            let mut has_synthetic_ids = false;\n\n            for id in rr.into_valid() {\n                has_synthetic_ids |= id.is_synthetic();\n                ids.push(id);\n            }\n\n            (ids, has_synthetic_ids)\n        } else {\n            (\n                calendar_event_ids\n                    .iter()\n                    .take(self.core.jmap.get_max_objects)\n                    .map(Into::into)\n                    .collect::<Vec<_>>(),\n                false,\n            )\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: cache.get_state(false).into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n        let mut return_converted_props = !return_all_properties;\n        let mut return_is_origin = false;\n        let mut return_utc_dates = false;\n\n        let (jmap_properties, jscal_properties) = if !return_all_properties {\n            let mut jmap_properties = Vec::with_capacity(4);\n            let mut jscal_properties = Vec::with_capacity(properties.len());\n\n            for property in properties {\n                match property {\n                    JSCalendarProperty::Id\n                    | JSCalendarProperty::BaseEventId\n                    | JSCalendarProperty::CalendarIds\n                    | JSCalendarProperty::IsDraft\n                    | JSCalendarProperty::UseDefaultAlerts\n                    | JSCalendarProperty::MayInviteSelf\n                    | JSCalendarProperty::MayInviteOthers\n                    | JSCalendarProperty::HideAttendees => {\n                        jmap_properties.push(property);\n                    }\n                    JSCalendarProperty::UtcStart | JSCalendarProperty::UtcEnd => {\n                        return_utc_dates = true;\n                        jmap_properties.push(property);\n                    }\n                    JSCalendarProperty::IsOrigin => {\n                        return_is_origin = true;\n                    }\n                    _ => {\n                        if matches!(property, JSCalendarProperty::ICalendar) {\n                            return_converted_props = true;\n                        }\n\n                        jscal_properties.push(property);\n                    }\n                }\n            }\n            (jmap_properties, jscal_properties)\n        } else {\n            return_is_origin = true;\n            (\n                vec![\n                    JSCalendarProperty::Id,\n                    JSCalendarProperty::CalendarIds,\n                    JSCalendarProperty::IsDraft,\n                    JSCalendarProperty::IsOrigin,\n                ],\n                vec![],\n            )\n        };\n        let return_is_origin = if return_is_origin {\n            if access_token.primary_id() == account_id {\n                OriginAddresses::Ref(access_token)\n            } else {\n                OriginAddresses::Owned(self.get_access_token(account_id).await?)\n            }\n        } else {\n            OriginAddresses::None\n        };\n\n        // Sort by baseId\n        let mut original_order: Option<AHashMap<Id, usize>> = None;\n        if has_synthetic_ids {\n            original_order = Some(ids.iter().enumerate().map(|(i, id)| (*id, i)).collect());\n            ids.sort_unstable_by_key(|id| id.document_id());\n        }\n        let mut ids = ids.into_iter().peekable();\n\n        // Process arguments\n        let override_range = if request.arguments.recurrence_overrides_after.is_some()\n            || request.arguments.recurrence_overrides_before.is_some()\n        {\n            let after = request\n                .arguments\n                .recurrence_overrides_after\n                .map(|v| v.timestamp)\n                .unwrap_or(i64::MIN);\n            let before = request\n                .arguments\n                .recurrence_overrides_before\n                .map(|v| v.timestamp)\n                .unwrap_or(i64::MAX);\n            if after < before {\n                Some(after..before)\n            } else {\n                None\n            }\n        } else {\n            None\n        };\n        let default_tz = request.arguments.time_zone.unwrap_or(Tz::UTC);\n        let reduce_participants = request.arguments.reduce_participants.unwrap_or(false);\n\n        'outer: while let Some(id) = ids.next() {\n            // Obtain the calendar_event object\n            let document_id = id.document_id();\n            if !calendar_event_ids.contains(document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n\n            let Some(_calendar_event) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await?\n            else {\n                response.not_found.push(id);\n                continue;\n            };\n            let mut calendar_event = _calendar_event\n                .deserialize::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            // Extract expansion ids from synthetic ids\n            let mut expansion_ids = AHashSet::new();\n            let mut include_base_event = false;\n            if let Some(expansion_id) = id.expansion_id() {\n                expansion_ids.insert(expansion_id);\n            } else {\n                include_base_event = true;\n            }\n            while let Some(next_id) = ids.peek() {\n                if next_id.document_id() == document_id {\n                    if let Some(expansion_id) = next_id.expansion_id() {\n                        expansion_ids.insert(expansion_id);\n                    } else {\n                        include_base_event = true;\n                    }\n                    ids.next();\n                } else {\n                    break;\n                }\n            }\n\n            // Reduce participants\n            if reduce_participants {\n                for component in &mut calendar_event.data.event.components {\n                    if component.component_type.is_scheduling_object() {\n                        component.entries.retain(|entry| match &entry.name {\n                            ICalendarProperty::Attendee => {\n                                entry.parameters(&ICalendarParameterName::Role).any(|role| {\n                                    matches!(\n                                        role,\n                                        ICalendarParameterValue::Role(\n                                            ICalendarParticipationRole::Owner,\n                                        ),\n                                    )\n                                }) || entry.calendar_address().is_some_and(|addr| {\n                                    access_token\n                                        .emails\n                                        .iter()\n                                        .any(|a| a.eq_ignore_ascii_case(addr))\n                                })\n                            }\n                            _ => true,\n                        });\n                    }\n                }\n            }\n\n            // Expand synthetic ids\n            let mut results = Vec::with_capacity(expansion_ids.len() + 1);\n            if !expansion_ids.is_empty() {\n                let ical = &calendar_event.data.event;\n                if let Some(expansions) = calendar_event\n                    .data\n                    .expand_from_ids(&mut expansion_ids, default_tz)\n                {\n                    for expansion in expansions {\n                        if !expansion.is_valid() {\n                            response.not_found.push(<Id as CalendarSyntheticId>::new(\n                                expansion.expansion_id,\n                                document_id,\n                            ));\n                            continue 'outer;\n                        }\n                        let component = &ical.components[expansion.comp_id as usize];\n                        let is_recurrent = component.is_recurrent();\n                        let is_recurrent_or_override =\n                            is_recurrent || component.is_recurrence_override();\n                        let mut has_duration = false;\n                        let component_ids = &component.component_ids;\n                        let mut tz = None;\n                        let mut component = ICalendarComponent {\n                            component_type: component.component_type.clone(),\n                            component_ids: Vec::new(),\n                            entries: component\n                                .entries\n                                .iter()\n                                .filter(|entry| match &entry.name {\n                                    ICalendarProperty::Dtstart\n                                    | ICalendarProperty::Dtend\n                                    | ICalendarProperty::Exdate\n                                    | ICalendarProperty::Exrule\n                                    | ICalendarProperty::Rdate\n                                    | ICalendarProperty::Rrule\n                                    | ICalendarProperty::RecurrenceId => {\n                                        if let Some(new_tz) = entry\n                                            .tz_id()\n                                            .and_then(|id| Tz::from_str(id).ok())\n                                            .filter(|tz| *tz != Tz::UTC)\n                                        {\n                                            tz = Some(new_tz);\n                                        }\n                                        false\n                                    }\n                                    ICalendarProperty::Due\n                                    | ICalendarProperty::Completed\n                                    | ICalendarProperty::Created => is_recurrent,\n                                    ICalendarProperty::Duration => {\n                                        has_duration = true;\n                                        true\n                                    }\n                                    _ => true,\n                                })\n                                .cloned()\n                                .collect::<Vec<_>>(),\n                        };\n\n                        let tz = tz.unwrap_or(default_tz);\n                        let tz_name = tz.name().unwrap_or_default().to_string();\n\n                        let start_timestamp = DateTime::from_timestamp(expansion.start, 0)\n                            .map(|dt| dt.with_timezone(&tz))\n                            .map(|dt| dt.naive_local())\n                            .map(|dt| dt.and_utc().timestamp())\n                            .unwrap_or(expansion.start);\n\n                        let end_timestamp = DateTime::from_timestamp(expansion.end, 0)\n                            .map(|dt| dt.with_timezone(&tz))\n                            .map(|dt| dt.naive_local())\n                            .map(|dt| dt.and_utc().timestamp())\n                            .unwrap_or(expansion.end);\n\n                        component.entries.push(ICalendarEntry {\n                            name: ICalendarProperty::Dtstart,\n                            params: vec![ICalendarParameter::tzid(tz_name.clone())],\n                            values: vec![ICalendarValue::PartialDateTime(Box::new(\n                                PartialDateTime::from_naive_timestamp(start_timestamp),\n                            ))],\n                        });\n\n                        if is_recurrent_or_override {\n                            component.entries.push(ICalendarEntry {\n                                name: ICalendarProperty::RecurrenceId,\n                                params: vec![ICalendarParameter::tzid(tz_name.clone())],\n                                values: vec![ICalendarValue::PartialDateTime(Box::new(\n                                    PartialDateTime::from_naive_timestamp(start_timestamp),\n                                ))],\n                            });\n                        }\n\n                        if !has_duration {\n                            component.entries.push(ICalendarEntry {\n                                name: ICalendarProperty::Dtend,\n                                params: vec![ICalendarParameter::tzid(tz_name)],\n                                values: vec![ICalendarValue::PartialDateTime(Box::new(\n                                    PartialDateTime::from_naive_timestamp(end_timestamp),\n                                ))],\n                            });\n                        }\n\n                        let mut expanded_ical = ICalendar {\n                            components: vec![\n                                ICalendarComponent {\n                                    component_type: ICalendarComponentType::VCalendar,\n                                    entries: vec![],\n                                    component_ids: vec![1],\n                                },\n                                component,\n                            ],\n                        };\n\n                        if !component_ids.is_empty() {\n                            for component_id in component_ids {\n                                let mut sub_component =\n                                    ical.components[*component_id as usize].clone();\n                                sub_component.component_ids.clear();\n                                let component_id = expanded_ical.components.len() as u32;\n                                expanded_ical.components.push(sub_component);\n                                expanded_ical.components[1].component_ids.push(component_id);\n                            }\n                        }\n\n                        results.push((\n                            <Id as CalendarSyntheticId>::new(expansion.expansion_id, document_id),\n                            expanded_ical,\n                            expansion,\n                        ));\n                    }\n                } else {\n                    response\n                        .not_found\n                        .extend(expansion_ids.into_iter().map(|expansion_id| {\n                            <Id as CalendarSyntheticId>::new(expansion_id, document_id)\n                        }));\n                    continue;\n                }\n            }\n\n            if include_base_event {\n                let mut event = std::mem::take(&mut calendar_event.data.event);\n\n                // Obtain UTC start/end if requested\n                let expansion = if return_utc_dates\n                    && let Some(expansion) = event\n                        .components\n                        .iter()\n                        .position(|c| {\n                            c.component_type.is_scheduling_object() && !c.is_recurrence_override()\n                        })\n                        .and_then(|comp_id| {\n                            calendar_event\n                                .data\n                                .expand_single(comp_id as u32, default_tz)\n                        }) {\n                    expansion\n                } else {\n                    CalendarEventExpansion::default()\n                };\n\n                // Remove recurrence ids\n                if let Some(range) = &override_range {\n                    let remove_ids = event\n                        .components\n                        .iter()\n                        .enumerate()\n                        .filter_map(|(comp_id, c)| {\n                            if c.is_recurrence_override()\n                                && let Some(timestamp) = c\n                                    .property(&ICalendarProperty::RecurrenceId)\n                                    .and_then(|p| p.values.first())\n                                    .and_then(|v| v.as_partial_date_time())\n                                    .and_then(|v| v.to_date_time())\n                                    .and_then(|v| v.to_date_time_with_tz(default_tz))\n                                    .map(|v| v.timestamp())\n                                && !range.contains(&timestamp)\n                            {\n                                Some(comp_id as u32)\n                            } else {\n                                None\n                            }\n                        })\n                        .collect::<AHashSet<_>>();\n                    if !remove_ids.is_empty() {\n                        for component in &mut event.components {\n                            component\n                                .component_ids\n                                .retain(|id| !remove_ids.contains(id));\n                        }\n                    }\n                }\n\n                results.push((Id::from(document_id), event, expansion));\n            }\n\n            for (id, ical, expansion) in results {\n                let is_origin = return_is_origin.addresses().is_some_and(|addresses| {\n                    ical.components\n                        .iter()\n                        .find(|c| c.component_type.is_scheduling_object())\n                        .and_then(|c| c.property(&ICalendarProperty::Organizer))\n                        .and_then(|v| v.calendar_address())\n                        .is_none_or(|v| addresses.iter().any(|a| a.eq_ignore_ascii_case(v)))\n                });\n\n                let jscal = ical\n                    .into_jscalendar_with_opt::<Id, BlobId>(\n                        ConversionOptions::default()\n                            .include_ical_components(return_converted_props)\n                            .return_first(true),\n                    )\n                    .into_inner();\n                let mut result = if return_all_properties {\n                    jscal.into_object().unwrap()\n                } else {\n                    Map::from_iter(jscal.into_expanded_object().filter(|(k, _)| {\n                        k.as_property()\n                            .is_some_and(|p| jscal_properties.contains(p))\n                    }))\n                };\n\n                for property in &jmap_properties {\n                    match property {\n                        JSCalendarProperty::Id => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::Id,\n                                Value::Element(JSCalendarValue::Id(id)),\n                            );\n                        }\n                        JSCalendarProperty::BaseEventId => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::BaseEventId,\n                                Value::Element(JSCalendarValue::Id(id.document_id().into())),\n                            );\n                        }\n                        JSCalendarProperty::CalendarIds => {\n                            let mut obj = Map::with_capacity(calendar_event.names.len());\n                            for id in calendar_event.names.iter() {\n                                obj.insert_unchecked(\n                                    JSCalendarProperty::IdValue(Id::from(id.parent_id)),\n                                    true,\n                                );\n                            }\n                            result.insert_unchecked(\n                                JSCalendarProperty::CalendarIds,\n                                Value::Object(obj),\n                            );\n                        }\n                        JSCalendarProperty::IsDraft => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::IsDraft,\n                                Value::Bool(calendar_event.flags & EVENT_DRAFT != 0),\n                            );\n                        }\n                        JSCalendarProperty::IsOrigin => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::IsOrigin,\n                                Value::Bool(is_origin),\n                            );\n                        }\n                        JSCalendarProperty::MayInviteSelf => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::MayInviteSelf,\n                                Value::Bool(calendar_event.flags & EVENT_INVITE_SELF != 0),\n                            );\n                        }\n                        JSCalendarProperty::MayInviteOthers => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::MayInviteOthers,\n                                Value::Bool(calendar_event.flags & EVENT_INVITE_OTHERS != 0),\n                            );\n                        }\n                        JSCalendarProperty::HideAttendees => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::HideAttendees,\n                                Value::Bool(calendar_event.flags & EVENT_HIDE_ATTENDEES != 0),\n                            );\n                        }\n\n                        JSCalendarProperty::UtcStart => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::UtcStart,\n                                Value::Element(JSCalendarValue::DateTime(JSCalendarDateTime::new(\n                                    expansion.start,\n                                    false,\n                                ))),\n                            );\n                        }\n                        JSCalendarProperty::UtcEnd => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::UtcEnd,\n                                Value::Element(JSCalendarValue::DateTime(JSCalendarDateTime::new(\n                                    expansion.end,\n                                    false,\n                                ))),\n                            );\n                        }\n                        JSCalendarProperty::UseDefaultAlerts => {\n                            result.insert_unchecked(\n                                JSCalendarProperty::UseDefaultAlerts,\n                                Value::Bool(\n                                    calendar_event\n                                        .preferences(access_token)\n                                        .is_none_or(|v| v.flags & PREF_USE_DEFAULT_ALERTS != 0),\n                                ),\n                            );\n                        }\n\n                        _ => {}\n                    }\n                }\n\n                response.list.push(result.into());\n            }\n        }\n\n        // Restore original order\n        if let Some(original_order) = original_order {\n            response.list.sort_by_key(|obj| {\n                obj.as_object()\n                    .unwrap()\n                    .get(&Key::Property(JSCalendarProperty::<Id>::Id))\n                    .and_then(|v| v.as_element())\n                    .and_then(|v: &JSCalendarValue<Id, BlobId>| v.as_id())\n                    .and_then(|id| original_order.get(&id))\n                    .cloned()\n                    .unwrap_or(usize::MAX)\n            });\n        }\n\n        Ok(response)\n    }\n}\n\nenum OriginAddresses<'x> {\n    Owned(Arc<AccessToken>),\n    Ref(&'x AccessToken),\n    None,\n}\n\nimpl<'x> OriginAddresses<'x> {\n    fn addresses(&self) -> Option<&[String]> {\n        match self {\n            OriginAddresses::Owned(t) if !t.emails.is_empty() => Some(&t.emails),\n            OriginAddresses::Ref(t) if !t.emails.is_empty() => Some(&t.emails),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse calcard::jscalendar::JSCalendarProperty;\nuse common::Server;\nuse jmap_proto::error::set::SetError;\nuse trc::AddContext;\nuse types::{collection::Collection, field::CalendarEventField, id::Id};\n\npub mod copy;\npub mod get;\npub mod parse;\npub mod query;\npub mod set;\n\n/*\n\nTODO: Not yet implemented:\n\n- CalendarEvent\n    - Per-user properties (However, the database schema is ready to support this)\n    - mayInviteSelf, mayInviteOthers and hideAttendees (stored but not enforced)\n\n- CalendarEvent/set\n   - synthetic id update and removal\n\n- Principal/getAvailability\n  - If there are overlapping BusyPeriod time ranges with different \"busyStatus\" properties\n    the server MUST choose the value in the following order: confirmed > unavailable > tentative.\n  - Return event properties\n\n*/\n\npub trait CalendarSyntheticId {\n    fn new(expansion_id: u32, document_id: u32) -> Self;\n\n    fn is_synthetic(&self) -> bool;\n\n    fn expansion_id(&self) -> Option<u32>;\n}\n\nimpl CalendarSyntheticId for Id {\n    fn new(expansion_id: u32, document_id: u32) -> Id {\n        Id::from_parts(expansion_id + 1, document_id)\n    }\n\n    fn expansion_id(&self) -> Option<u32> {\n        let prefix = self.prefix_id();\n        if prefix > 0 { Some(prefix - 1) } else { None }\n    }\n\n    fn is_synthetic(&self) -> bool {\n        self.prefix_id() > 0\n    }\n}\n\npub(super) async fn assert_is_unique_uid(\n    server: &Server,\n    account_id: u32,\n    uid: Option<&str>,\n) -> trc::Result<Result<(), SetError<JSCalendarProperty<Id>>>> {\n    if let Some(uid) = uid\n        && server\n            .document_exists(\n                account_id,\n                Collection::CalendarEvent,\n                CalendarEventField::Uid,\n                uid.as_bytes(),\n            )\n            .await\n            .caused_by(trc::location!())?\n    {\n        Ok(Err(SetError::invalid_properties()\n            .with_property(JSCalendarProperty::Uid)\n            .with_description(format!(\n                \"An event with UID {uid} already exists.\",\n            ))))\n    } else {\n        Ok(Ok(()))\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event/parse.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::blob::download::BlobDownload;\nuse calcard::{\n    icalendar::ICalendar,\n    jscalendar::{JSCalendarProperty, import::ConversionOptions},\n};\nuse common::{Server, auth::AccessToken};\nuse jmap_proto::{\n    method::parse::{ParseRequest, ParseResponse},\n    object::calendar_event::CalendarEvent,\n    request::IntoValid,\n};\nuse jmap_tools::{Key, Value};\nuse types::{blob::BlobId, id::Id};\nuse utils::map::vec_map::VecMap;\n\npub trait CalendarEventParse: Sync + Send {\n    fn calendar_event_parse(\n        &self,\n        request: ParseRequest<CalendarEvent>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<ParseResponse<CalendarEvent>>> + Send;\n}\n\nimpl CalendarEventParse for Server {\n    async fn calendar_event_parse(\n        &self,\n        request: ParseRequest<CalendarEvent>,\n        access_token: &AccessToken,\n    ) -> trc::Result<ParseResponse<CalendarEvent>> {\n        if request.blob_ids.len() > self.core.jmap.calendar_parse_max_items {\n            return Err(trc::JmapEvent::RequestTooLarge.into_err());\n        }\n        let return_all_properties = request.properties.is_none();\n        let properties = request\n            .properties\n            .map(|v| v.into_valid().collect::<Vec<_>>())\n            .unwrap_or_default();\n\n        let mut response = ParseResponse {\n            account_id: request.account_id,\n            parsed: VecMap::with_capacity(request.blob_ids.len()),\n            not_parsable: vec![],\n            not_found: vec![],\n        };\n\n        for blob_id in request.blob_ids.into_valid() {\n            // Fetch raw message to parse\n            let raw_vcard = match self.blob_download(&blob_id, access_token).await? {\n                Some(raw_vcard) => raw_vcard,\n                None => {\n                    response.not_found.push(blob_id);\n                    continue;\n                }\n            };\n            let Ok(vcard) = ICalendar::parse(std::str::from_utf8(&raw_vcard).unwrap_or_default())\n            else {\n                response.not_parsable.push(blob_id);\n                continue;\n            };\n            let mut js_calendar_entries = vcard\n                .into_jscalendar_with_opt::<Id, BlobId>(ConversionOptions::default())\n                .into_inner()\n                .into_object()\n                .unwrap()\n                .remove(&Key::Property(JSCalendarProperty::Entries))\n                .unwrap()\n                .into_array()\n                .unwrap();\n\n            if !return_all_properties {\n                for entry in &mut js_calendar_entries {\n                    entry\n                        .as_object_mut()\n                        .unwrap()\n                        .as_mut_vec()\n                        .retain(|(k, _)| k.as_property().is_some_and(|k| properties.contains(k)));\n                }\n            }\n\n            response\n                .parsed\n                .append(blob_id, Value::Array(js_calendar_entries));\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::JmapCacheState};\nuse calcard::{common::timezone::Tz, jscalendar::JSCalendarDateTime};\nuse chrono::offset::TimeZone;\nuse common::{Server, auth::AccessToken};\nuse groupware::{cache::GroupwareCache, calendar::CalendarEvent};\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::calendar_event::{self, CalendarEventComparator, CalendarEventFilter},\n    request::MaybeInvalid,\n};\nuse nlp::language::Language;\nuse std::{cmp::Ordering, sync::Arc};\nuse store::{\n    ValueKey, roaring::RoaringBitmap, search::{CalendarSearchField, SearchComparator, SearchFilter, SearchQuery}, write::{AlignedBytes, Archive, SearchIndex}\n};\nuse trc::AddContext;\nuse types::{\n    TimeRange,\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\n\npub trait CalendarEventQuery: Sync + Send {\n    fn calendar_event_query(\n        &self,\n        request: QueryRequest<calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl CalendarEventQuery for Server {\n    async fn calendar_event_query(\n        &self,\n        mut request: QueryRequest<calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let mut filters = Vec::with_capacity(request.filter.len());\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await?;\n        let default_tz = request.arguments.time_zone.unwrap_or(Tz::UTC);\n        let mut filter: Option<TimeRange> = None;\n\n        // Extract from/to arguments\n        for cond in &request.filter {\n            if let Filter::Property(CalendarEventFilter::After(date)) = cond {\n                if let Some(after) = local_timestamp(date, default_tz) {\n                    filter.get_or_insert_default().start = after;\n                }\n            } else if let Filter::Property(CalendarEventFilter::Before(date)) = cond\n                && let Some(before) = local_timestamp(date, default_tz)\n            {\n                filter.get_or_insert_default().end = before;\n            }\n        }\n\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    CalendarEventFilter::InCalendar(MaybeInvalid::Value(id)) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.children_ids(id.document_id()),\n                        )))\n                    }\n                    CalendarEventFilter::Uid(uid) => {\n                        filters.push(SearchFilter::eq(CalendarSearchField::Uid, uid));\n                    }\n                    CalendarEventFilter::Text(value) => {\n                        let (text, language) =\n                            Language::detect(value, self.core.jmap.default_language);\n                        filters.push(SearchFilter::Or);\n                        filters.push(SearchFilter::has_text(\n                            CalendarSearchField::Title,\n                            text.clone(),\n                            language,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            CalendarSearchField::Description,\n                            text.clone(),\n                            language,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            CalendarSearchField::Location,\n                            text.clone(),\n                            language,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            CalendarSearchField::Owner,\n                            text.clone(),\n                            language,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            CalendarSearchField::Attendee,\n                            text,\n                            language,\n                        ));\n                        filters.push(SearchFilter::End);\n                    }\n                    CalendarEventFilter::Title(title) => {\n                        filters.push(SearchFilter::has_text_detect(\n                            CalendarSearchField::Title,\n                            title,\n                            self.core.jmap.default_language,\n                        ));\n                    }\n                    CalendarEventFilter::Description(description) => {\n                        filters.push(SearchFilter::has_text_detect(\n                            CalendarSearchField::Description,\n                            description,\n                            self.core.jmap.default_language,\n                        ));\n                    }\n                    CalendarEventFilter::Location(location) => {\n                        filters.push(SearchFilter::has_text_detect(\n                            CalendarSearchField::Location,\n                            location,\n                            self.core.jmap.default_language,\n                        ));\n                    }\n                    CalendarEventFilter::Owner(owner) => {\n                        filters.push(SearchFilter::has_text(\n                            CalendarSearchField::Owner,\n                            owner,\n                            Language::None,\n                        ));\n                    }\n                    CalendarEventFilter::Attendee(attendee) => {\n                        filters.push(SearchFilter::has_text(\n                            CalendarSearchField::Attendee,\n                            attendee,\n                            Language::None,\n                        ));\n                    }\n                    CalendarEventFilter::After(after) => {\n                        /*\n                            The end of the event, or any recurrence of the event, in the time zone given\n                            as the \"timeZone\" argument, must be after this date to match the condition.\n                        */\n                        if let Some(after) = local_timestamp(&after, default_tz) {\n                            filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                                cache.resources.iter().filter_map(|r| {\n                                    r.event_time_range()\n                                        .and_then(|(_, end)| (after < end).then_some(r.document_id))\n                                }),\n                            )));\n                        }\n                    }\n                    CalendarEventFilter::Before(before) => {\n                        /*\n                            The start of the event, or any recurrence of the event, in the time zone given\n                            as the \"timeZone\" argument, must be before this date to match the condition.\n                        */\n\n                        if let Some(before) = local_timestamp(&before, default_tz) {\n                            filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                                cache.resources.iter().filter_map(|r| {\n                                    r.event_time_range().and_then(|(start, _)| {\n                                        (before > start).then_some(r.document_id)\n                                    })\n                                }),\n                            )));\n                        }\n                    }\n                    unsupported => {\n                        return Err(trc::JmapEvent::UnsupportedFilter\n                            .into_err()\n                            .details(unsupported.into_string()));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        let expand_recurrences = request.arguments.expand_recurrences.unwrap_or(false);\n        let comparators = if !expand_recurrences {\n            request\n                .sort\n                .take()\n                .unwrap_or_default()\n                .into_iter()\n                .map(|comparator| match comparator.property {\n                    CalendarEventComparator::Start | CalendarEventComparator::RecurrenceId => {\n                        Ok(SearchComparator::field(\n                            CalendarSearchField::Start,\n                            comparator.is_ascending,\n                        ))\n                    }\n                    CalendarEventComparator::Uid => Ok(SearchComparator::field(\n                        CalendarSearchField::Uid,\n                        comparator.is_ascending,\n                    )),\n                    CalendarEventComparator::Created | CalendarEventComparator::Updated => {\n                        Err(trc::JmapEvent::UnsupportedSort\n                            .into_err()\n                            .details(comparator.property.into_string().into_owned()))\n                    }\n                    CalendarEventComparator::_T(other) => Err(trc::JmapEvent::UnsupportedSort\n                        .into_err()\n                        .details(other.to_string())),\n                })\n                .collect::<Result<Vec<_>, _>>()?\n        } else {\n            vec![]\n        };\n\n        let results = self\n            .search_store()\n            .query_account(\n                SearchQuery::new(SearchIndex::Calendar)\n                    .with_filters(filters)\n                    .with_comparators(comparators)\n                    .with_account_id(account_id)\n                    .with_mask(if access_token.is_shared(account_id) {\n                        cache.shared_items(access_token, [Acl::ReadItems], true)\n                    } else {\n                        cache.document_ids(false).collect()\n                    }),\n            )\n            .await?;\n\n        // Extract comparators\n        let comparators = request\n            .sort\n            .as_deref()\n            .filter(|s| !s.is_empty())\n            .unwrap_or_default();\n\n        if expand_recurrences && !results.is_empty() {\n            let Some(time_range) = filter.filter(|f| f.start != i64::MIN && f.end != i64::MAX)\n            else {\n                return Err(trc::JmapEvent::InvalidArguments.into_err().details(\n                    \"Both 'after' and 'before' filters are required when expanding recurrences\",\n                ));\n            };\n            let max_instances = self.core.groupware.max_ical_instances;\n            let mut expanded_results = Vec::with_capacity(results.len() as usize);\n            let has_uid_comparator = comparators\n                .iter()\n                .any(|c| matches!(c.property, CalendarEventComparator::Uid));\n\n            for document_id in results {\n                let Some(_calendar_event) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::CalendarEvent,\n                        document_id,\n                    ))\n                    .await?\n                else {\n                    continue;\n                };\n                let calendar_event = _calendar_event\n                    .unarchive::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n\n                // Expand recurrences\n                let uid = if has_uid_comparator {\n                    Arc::new(\n                        calendar_event\n                            .data\n                            .event\n                            .uids()\n                            .next()\n                            .unwrap_or_default()\n                            .to_string(),\n                    )\n                } else {\n                    Arc::new(String::new())\n                };\n                for expansion in calendar_event\n                    .data\n                    .expand(default_tz, time_range)\n                    .unwrap_or_default()\n                {\n                    if expanded_results.len() < max_instances {\n                        expanded_results.push(SearchResult {\n                            created: calendar_event.created.to_native().to_be_bytes(),\n                            updated: calendar_event.modified.to_native().to_be_bytes(),\n                            start: expansion.start.to_be_bytes(),\n                            uid: uid.clone(),\n                            document_id,\n                            expansion_id: expansion.expansion_id.into(),\n                        });\n                    } else {\n                        return Err(trc::JmapEvent::InvalidArguments.into_err().details(\n                            \"The number of expanded recurrences exceeds the server limit\",\n                        ));\n                    }\n                }\n            }\n\n            let mut response = QueryResponseBuilder::new(\n                expanded_results.len(),\n                self.core.jmap.query_max_results,\n                cache.get_state(false),\n                &request,\n            );\n            // Sort results\n            if !expanded_results.is_empty() {\n                expanded_results.sort_by(|a, b| {\n                    for comparator in comparators {\n                        let ordering = if comparator.is_ascending {\n                            a.get_property(&comparator.property)\n                                .cmp(b.get_property(&comparator.property))\n                        } else {\n                            b.get_property(&comparator.property)\n                                .cmp(a.get_property(&comparator.property))\n                        };\n\n                        if ordering != Ordering::Equal {\n                            return ordering;\n                        }\n                    }\n                    Ordering::Equal\n                });\n\n                // Add results\n                for result in expanded_results {\n                    if !response.add(result.expansion_id.unwrap() + 1, result.document_id) {\n                        break;\n                    }\n                }\n            }\n            response.build()\n        } else {\n            let mut response = QueryResponseBuilder::new(\n                results.len(),\n                self.core.jmap.query_max_results,\n                cache.get_state(false),\n                &request,\n            );\n            for document_id in results {\n                if !response.add(0, document_id) {\n                    break;\n                }\n            }\n            response.build()\n        }\n    }\n}\n\nfn local_timestamp(dt: &JSCalendarDateTime, tz: Tz) -> Option<i64> {\n    tz.from_local_datetime(&dt.to_naive_date_time()?)\n        .single()\n        .map(|dt| dt.timestamp())\n}\n\n#[derive(Debug)]\nstruct SearchResult {\n    expansion_id: Option<u32>,\n    document_id: u32,\n    start: [u8; std::mem::size_of::<i64>()],\n    created: [u8; std::mem::size_of::<i64>()],\n    updated: [u8; std::mem::size_of::<i64>()],\n    uid: Arc<String>,\n}\n\nimpl SearchResult {\n    fn get_property(&self, comparator: &CalendarEventComparator) -> &[u8] {\n        match comparator {\n            CalendarEventComparator::Uid => self.uid.as_bytes(),\n            CalendarEventComparator::Start | CalendarEventComparator::RecurrenceId => {\n                self.start.as_ref()\n            }\n            CalendarEventComparator::Created => self.created.as_ref(),\n            CalendarEventComparator::Updated => self.updated.as_ref(),\n            CalendarEventComparator::_T(_) => &[],\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::calendar_event::{CalendarSyntheticId, assert_is_unique_uid};\nuse calcard::{\n    common::timezone::Tz,\n    icalendar::{\n        ICalendarAction, ICalendarComponent, ICalendarComponentType, ICalendarDuration,\n        ICalendarEntry, ICalendarParameter, ICalendarParameterValue, ICalendarProperty,\n        ICalendarRelated, ICalendarValue,\n    },\n    jscalendar::{JSCalendar, JSCalendarDateTime, JSCalendarProperty, JSCalendarValue},\n};\nuse chrono::DateTime;\nuse common::{DavName, DavResources, Server, auth::AccessToken};\nuse directory::Permission;\nuse groupware::{\n    DestroyArchive,\n    cache::GroupwareCache,\n    calendar::{\n        ALERT_EMAIL, ALERT_RELATIVE_TO_END, ArchivedDefaultAlert, Calendar, CalendarEvent,\n        CalendarEventData, EVENT_DRAFT, EVENT_HIDE_ATTENDEES, EVENT_INVITE_OTHERS,\n        EVENT_INVITE_SELF,\n    },\n    scheduling::{ItipMessages, event_create::itip_create, event_update::itip_update},\n};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::set::{SetRequest, SetResponse},\n    object::calendar_event,\n    request::IntoValid,\n    types::state::State,\n};\nuse jmap_tools::{JsonPointerHandler, JsonPointerItem, Key, Map, Value};\nuse std::{borrow::Cow, str::FromStr};\nuse store::{\n    ValueKey,\n    ahash::AHashSet,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive, BatchBuilder, now, serialize::rkyv_deserialize},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    blob::BlobId,\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub trait CalendarEventSet: Sync + Send {\n    fn calendar_event_set(\n        &self,\n        request: SetRequest<'_, calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<SetResponse<calendar_event::CalendarEvent>>> + Send;\n\n    #[allow(clippy::too_many_arguments)]\n    fn create_calendar_event(\n        &self,\n        cache: &DavResources,\n        batch: &mut BatchBuilder,\n        access_token: &AccessToken,\n        account_id: u32,\n        send_scheduling_messages: bool,\n        can_add_calendars: &Option<RoaringBitmap>,\n        js_calendar_event: JSCalendar<'_, Id, BlobId>,\n        updates: Value<'_, JSCalendarProperty<Id>, JSCalendarValue<Id, BlobId>>,\n    ) -> impl Future<Output = trc::Result<Result<u32, SetError<JSCalendarProperty<Id>>>>>;\n}\n\nimpl CalendarEventSet for Server {\n    async fn calendar_event_set(\n        &self,\n        mut request: SetRequest<'_, calendar_event::CalendarEvent>,\n        access_token: &AccessToken,\n        _session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<calendar_event::CalendarEvent>> {\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n            .await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Obtain calendarIds\n        let (can_add_calendars, can_delete_calendars, can_modify_calendars) =\n            if access_token.is_shared(account_id) {\n                (\n                    cache\n                        .shared_containers(access_token, [Acl::AddItems], true)\n                        .into(),\n                    cache\n                        .shared_containers(access_token, [Acl::RemoveItems], true)\n                        .into(),\n                    cache\n                        .shared_containers(access_token, [Acl::ModifyItems], true)\n                        .into(),\n                )\n            } else {\n                (None, None, None)\n            };\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        let send_scheduling_messages = request.arguments.send_scheduling_messages.unwrap_or(false);\n        'create: for (id, object) in request.unwrap_create() {\n            match self\n                .create_calendar_event(\n                    &cache,\n                    &mut batch,\n                    access_token,\n                    account_id,\n                    send_scheduling_messages,\n                    &can_add_calendars,\n                    JSCalendar::default(),\n                    object,\n                )\n                .await?\n            {\n                Ok(document_id) => {\n                    response.created(id, document_id);\n                }\n                Err(err) => {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                }\n            }\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            } else if id.is_synthetic() {\n                response.not_updated.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(JSCalendarProperty::Id)\n                        .with_description(\"Updating synthetic ids is not yet supported.\"),\n                );\n                continue 'update;\n            }\n\n            // Obtain calendar_event card\n            let document_id = id.document_id();\n            let calendar_event_ = if let Some(calendar_event_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await?\n            {\n                calendar_event_\n            } else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n            let calendar_event = calendar_event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n            let mut new_calendar_event = calendar_event\n                .deserialize::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n            let mut js_calendar_group =\n                std::mem::take(&mut new_calendar_event.data.event).into_jscalendar::<Id, BlobId>();\n\n            // Process changes\n            if let Err(err) = update_calendar_event(\n                access_token,\n                object,\n                &mut new_calendar_event,\n                &mut js_calendar_group,\n            ) {\n                response.not_updated.append(id, err);\n                continue 'update;\n            }\n\n            // Convert JSCalendar to iCalendar\n            let Some(ical) = js_calendar_group.into_icalendar() else {\n                response.not_updated.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_description(\"Failed to convert calendar event to iCalendar.\"),\n                );\n                continue 'update;\n            };\n            new_calendar_event.data.event = ical;\n\n            // Validate UID\n            match (\n                new_calendar_event.data.event.uids().next(),\n                calendar_event.inner.data.event.uids().next(),\n            ) {\n                (Some(old_uid), Some(new_uid)) if old_uid == new_uid => {}\n                (None, None) | (None, Some(_)) => {}\n                _ => {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(JSCalendarProperty::Uid)\n                            .with_description(\"You cannot change the UID of a calendar event.\"),\n                    );\n                    continue 'update;\n                }\n            }\n\n            // Validate new calendarIds\n            for calendar_id in new_calendar_event.added_calendar_ids(calendar_event.inner) {\n                if !cache.has_container_id(&calendar_id) {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(JSCalendarProperty::CalendarIds)\n                            .with_description(format!(\n                                \"calendarId {} does not exist.\",\n                                Id::from(calendar_id)\n                            )),\n                    );\n                    continue 'update;\n                } else if can_add_calendars\n                    .as_ref()\n                    .is_some_and(|ids| !ids.contains(calendar_id))\n                {\n                    response.not_updated.append(\n                        id,\n                        SetError::forbidden().with_description(format!(\n                            \"You are not allowed to add calendar events to calendar {}.\",\n                            Id::from(calendar_id)\n                        )),\n                    );\n                    continue 'update;\n                }\n            }\n\n            // Validate deleted calendarIds\n            if let Some(can_delete_calendars) = &can_delete_calendars {\n                for calendar_id in new_calendar_event.removed_calendar_ids(calendar_event.inner) {\n                    if !can_delete_calendars.contains(calendar_id) {\n                        response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(format!(\n                                \"You are not allowed to remove calendar events from calendar {}.\",\n                                Id::from(calendar_id)\n                            )),\n                        );\n                        continue 'update;\n                    }\n                }\n            }\n\n            // Validate changed calendarIds\n            if let Some(can_modify_calendars) = &can_modify_calendars {\n                for calendar_id in new_calendar_event.unchanged_calendar_ids(calendar_event.inner) {\n                    if !can_modify_calendars.contains(calendar_id) {\n                        response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(format!(\n                                \"You are not allowed to modify calendar {}.\",\n                                Id::from(calendar_id)\n                            )),\n                        );\n                        continue 'update;\n                    }\n                }\n            }\n\n            // Check size and quota\n            new_calendar_event.size = new_calendar_event.data.event.size() as u32;\n            if new_calendar_event.size as usize > self.core.groupware.max_ical_size {\n                response.not_updated.append(\n                    id,\n                    SetError::invalid_properties().with_description(format!(\n                        \"Event size {} exceeds the maximum allowed size of {} bytes.\",\n                        new_calendar_event.size, self.core.groupware.max_ical_size\n                    )),\n                );\n                continue 'update;\n            }\n\n            // Obtain previous alarm\n            let now = now() as i64;\n            let prev_email_alarm = calendar_event.inner.data.next_alarm(now, Tz::Floating);\n\n            // Build event\n            let mut next_email_alarm = None;\n            new_calendar_event.data = CalendarEventData::new(\n                new_calendar_event.data.event,\n                Tz::Floating,\n                self.core.groupware.max_ical_instances,\n                &mut next_email_alarm,\n            );\n\n            // Scheduling\n            let mut itip_messages = None;\n            if send_scheduling_messages\n                && self.core.groupware.itip_enabled\n                && !access_token.emails.is_empty()\n                && access_token.has_permission(Permission::CalendarSchedulingSend)\n                && new_calendar_event.data.event_range_end() > now\n            {\n                let result = if new_calendar_event.schedule_tag.is_some() {\n                    let old_ical = rkyv_deserialize(&calendar_event.inner.data.event)\n                        .caused_by(trc::location!())?;\n\n                    itip_update(\n                        &mut new_calendar_event.data.event,\n                        &old_ical,\n                        access_token.emails.as_slice(),\n                    )\n                } else {\n                    itip_create(\n                        &mut new_calendar_event.data.event,\n                        access_token.emails.as_slice(),\n                    )\n                };\n\n                match result {\n                    Ok(messages) => {\n                        let mut is_organizer = false;\n                        if messages\n                            .iter()\n                            .map(|r| {\n                                is_organizer = r.from_organizer;\n                                r.to.len()\n                            })\n                            .sum::<usize>()\n                            < self.core.groupware.itip_outbound_max_recipients\n                        {\n                            // Only update schedule tag if the user is the organizer\n                            if is_organizer {\n                                if let Some(schedule_tag) = &mut new_calendar_event.schedule_tag {\n                                    *schedule_tag += 1;\n                                } else {\n                                    new_calendar_event.schedule_tag = Some(1);\n                                }\n                            }\n\n                            itip_messages = Some(ItipMessages::new(messages));\n                        } else {\n                            response.not_updated.append(\n                                id,\n                                SetError::invalid_properties()\n                                    .with_property(JSCalendarProperty::Participants)\n                                    .with_description(concat!(\n                                        \"The number of scheduling message recipients \",\n                                        \"exceeds the maximum allowed.\"\n                                    )),\n                            );\n                            continue 'update;\n                        }\n                    }\n                    Err(err) => {\n                        if err.is_jmap_error() {\n                            response.not_updated.append(\n                                id,\n                                SetError::invalid_properties()\n                                    .with_property(JSCalendarProperty::Participants)\n                                    .with_description(err.to_string()),\n                            );\n                            continue 'update;\n                        }\n\n                        // Event changed, but there are no iTIP messages to send\n                        if let Some(schedule_tag) = &mut new_calendar_event.schedule_tag {\n                            *schedule_tag += 1;\n                        }\n                    }\n                }\n            }\n\n            // Validate quota\n            let extra_bytes = (new_calendar_event.size as u64)\n                .saturating_sub(u32::from(calendar_event.inner.size) as u64);\n            if extra_bytes > 0 {\n                match self\n                    .has_available_quota(\n                        &self.get_resource_token(access_token, account_id).await?,\n                        extra_bytes,\n                    )\n                    .await\n                {\n                    Ok(_) => {}\n                    Err(err) if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota)) => {\n                        response.not_updated.append(id, SetError::over_quota());\n                        continue 'update;\n                    }\n                    Err(err) => return Err(err.caused_by(trc::location!())),\n                }\n            }\n\n            // Update record\n            new_calendar_event\n                .update(\n                    access_token,\n                    calendar_event,\n                    account_id,\n                    document_id,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n            if prev_email_alarm != next_email_alarm {\n                if let Some(prev_alarm) = prev_email_alarm {\n                    prev_alarm.delete_task(&mut batch);\n                }\n                if let Some(next_alarm) = next_email_alarm {\n                    next_alarm.write_task(&mut batch);\n                }\n            }\n            if let Some(itip_messages) = itip_messages {\n                itip_messages\n                    .queue(&mut batch)\n                    .caused_by(trc::location!())?;\n            }\n\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        'destroy: for id in will_destroy {\n            let document_id = id.document_id();\n\n            if !cache.has_item_id(&document_id) {\n                response.not_destroyed.append(id, SetError::not_found());\n                continue;\n            } else if id.is_synthetic() {\n                response.not_destroyed.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(JSCalendarProperty::Id)\n                        .with_description(\"Deleting synthetic ids is not yet supported.\"),\n                );\n                continue;\n            }\n\n            let Some(calendar_event_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            else {\n                response.not_destroyed.append(id, SetError::not_found());\n                continue;\n            };\n\n            let calendar_event = calendar_event_\n                .to_unarchived::<CalendarEvent>()\n                .caused_by(trc::location!())?;\n\n            // Validate ACLs\n            if let Some(can_delete_calendars) = &can_delete_calendars {\n                for name in calendar_event.inner.names.iter() {\n                    let parent_id = name.parent_id.to_native();\n                    if !can_delete_calendars.contains(parent_id) {\n                        response.not_destroyed.append(\n                            id,\n                            SetError::forbidden().with_description(format!(\n                                \"You are not allowed to remove events from calendar {}.\",\n                                Id::from(parent_id)\n                            )),\n                        );\n                        continue 'destroy;\n                    }\n                }\n            }\n\n            // Delete event\n            DestroyArchive(calendar_event)\n                .delete_all(\n                    access_token,\n                    account_id,\n                    document_id,\n                    send_scheduling_messages,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n\n            response.destroyed.push(id);\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n            self.notify_task_queue();\n\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n\n    async fn create_calendar_event(\n        &self,\n        cache: &DavResources,\n        batch: &mut BatchBuilder,\n        access_token: &AccessToken,\n        account_id: u32,\n        send_scheduling_messages: bool,\n        can_add_calendars: &Option<RoaringBitmap>,\n        mut js_calendar_group: JSCalendar<'_, Id, BlobId>,\n        updates: Value<'_, JSCalendarProperty<Id>, JSCalendarValue<Id, BlobId>>,\n    ) -> trc::Result<Result<u32, SetError<JSCalendarProperty<Id>>>> {\n        // Process changes\n        let mut event = CalendarEvent::default();\n        let use_default_alerts = match update_calendar_event(\n            access_token,\n            updates,\n            &mut event,\n            &mut js_calendar_group,\n        ) {\n            Ok(use_default_alerts) => use_default_alerts,\n            Err(err) => {\n                return Ok(Err(err));\n            }\n        };\n\n        // Convert JSCalendar to iCalendar\n        let Some(mut ical) = js_calendar_group.into_icalendar() else {\n            return Ok(Err(SetError::invalid_properties().with_description(\n                \"Failed to convert calendar event to iCalendar.\",\n            )));\n        };\n\n        // Verify that the calendar ids valid\n        let default_alert_comp_id = ical.components.len();\n        for name in &event.names {\n            if !cache.has_container_id(&name.parent_id) {\n                return Ok(Err(SetError::invalid_properties()\n                    .with_property(JSCalendarProperty::CalendarIds)\n                    .with_description(format!(\n                        \"calendarId {} does not exist.\",\n                        Id::from(name.parent_id)\n                    ))));\n            } else if can_add_calendars\n                .as_ref()\n                .is_some_and(|ids| !ids.contains(name.parent_id))\n            {\n                return Ok(Err(SetError::forbidden().with_description(format!(\n                    \"You are not allowed to add calendar events to calendar {}.\",\n                    Id::from(name.parent_id)\n                ))));\n            } else if let Some(show_without_time) = use_default_alerts\n                && let Some(_calendar) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::Calendar,\n                        name.parent_id,\n                    ))\n                    .await?\n            {\n                ical.components.extend(\n                    _calendar\n                        .unarchive::<Calendar>()\n                        .caused_by(trc::location!())?\n                        .default_alerts(access_token, !show_without_time)\n                        .map(default_alert_to_ical),\n                );\n            }\n        }\n\n        // Add default alarms\n        if ical.components.len() > default_alert_comp_id {\n            let component_ids = default_alert_comp_id as u32..ical.components.len() as u32;\n            for component in &mut ical.components {\n                if component.component_type.is_event_or_todo()\n                    && !component.is_recurrence_override()\n                {\n                    component.component_ids.extend(component_ids.clone());\n                }\n            }\n        }\n\n        // Validate UID\n        if let Err(err) = assert_is_unique_uid(self, account_id, ical.uids().next()).await? {\n            return Ok(Err(err));\n        }\n\n        // Check size and quota\n        let size = ical.size();\n        if size > self.core.groupware.max_ical_size {\n            return Ok(Err(SetError::invalid_properties().with_description(\n                format!(\n                    \"Event size {} exceeds the maximum allowed size of {} bytes.\",\n                    size, self.core.groupware.max_ical_size\n                ),\n            )));\n        }\n\n        // Build event\n        let mut next_email_alarm = None;\n        event.data = CalendarEventData::new(\n            ical,\n            Tz::Floating,\n            self.core.groupware.max_ical_instances,\n            &mut next_email_alarm,\n        );\n        event.size = size as u32;\n\n        // Scheduling\n        let mut itip_messages = None;\n        if send_scheduling_messages\n            && self.core.groupware.itip_enabled\n            && !access_token.emails.is_empty()\n            && access_token.has_permission(Permission::CalendarSchedulingSend)\n            && event.data.event_range_end() > now() as i64\n        {\n            match itip_create(&mut event.data.event, access_token.emails.as_slice()) {\n                Ok(messages) => {\n                    if messages.iter().map(|r| r.to.len()).sum::<usize>()\n                        < self.core.groupware.itip_outbound_max_recipients\n                    {\n                        event.schedule_tag = Some(1);\n                        itip_messages = Some(ItipMessages::new(messages));\n                    } else {\n                        return Ok(Err(SetError::invalid_properties()\n                            .with_property(JSCalendarProperty::Participants)\n                            .with_description(concat!(\n                                \"The number of scheduling message recipients \",\n                                \"exceeds the maximum allowed.\"\n                            ))));\n                    }\n                }\n                Err(err) => {\n                    if err.is_jmap_error() {\n                        return Ok(Err(SetError::invalid_properties()\n                            .with_property(JSCalendarProperty::Participants)\n                            .with_description(err.to_string())));\n                    }\n                }\n            }\n        }\n\n        // Validate quota\n        match self\n            .has_available_quota(\n                &self.get_resource_token(access_token, account_id).await?,\n                size as u64,\n            )\n            .await\n        {\n            Ok(_) => {}\n            Err(err) if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota)) => {\n                return Ok(Err(SetError::over_quota()));\n            }\n            Err(err) => return Err(err.caused_by(trc::location!())),\n        }\n\n        // Insert record\n        let document_id = self\n            .store()\n            .assign_document_ids(account_id, Collection::CalendarEvent, 1)\n            .await\n            .caused_by(trc::location!())?;\n        event\n            .insert(\n                access_token,\n                account_id,\n                document_id,\n                next_email_alarm,\n                batch,\n            )\n            .caused_by(trc::location!())?;\n\n        if let Some(itip_messages) = itip_messages {\n            itip_messages.queue(batch).caused_by(trc::location!())?;\n        }\n\n        Ok(Ok(document_id))\n    }\n}\n\nfn update_calendar_event<'x>(\n    _access_token: &AccessToken,\n    updates: Value<'x, JSCalendarProperty<Id>, JSCalendarValue<Id, BlobId>>,\n    event: &mut CalendarEvent,\n    js_calendar_group: &mut JSCalendar<'x, Id, BlobId>,\n) -> Result<Option<bool>, SetError<JSCalendarProperty<Id>>> {\n    // Extract event\n    let js_calendar_events = js_calendar_group\n        .0\n        .as_object_mut()\n        .unwrap()\n        .get_mut(&Key::Property(JSCalendarProperty::Entries))\n        .unwrap()\n        .as_array_mut()\n        .unwrap();\n\n    let js_calendar_event = if let Some(js_calendar_event) = js_calendar_events.first_mut() {\n        js_calendar_event\n    } else {\n        js_calendar_events.push(Value::Object(Map::new()));\n        js_calendar_events.first_mut().unwrap()\n    };\n\n    let mut utc_start = None;\n    let mut utc_end = None;\n    let mut use_default_alerts = false;\n    let mut show_without_time = false;\n    let mut entries = js_calendar_event.as_object_mut().unwrap();\n\n    for (property, value) in updates.into_expanded_object() {\n        let Key::Property(property) = property else {\n            return Err(SetError::invalid_properties()\n                .with_property(property.to_owned())\n                .with_description(\"Invalid property.\"));\n        };\n\n        match (property, value) {\n            (JSCalendarProperty::IsDraft, Value::Bool(set)) => {\n                if set {\n                    event.flags |= EVENT_DRAFT;\n                } else {\n                    event.flags &= !EVENT_DRAFT;\n                }\n            }\n            (JSCalendarProperty::MayInviteSelf, Value::Bool(set)) => {\n                if set {\n                    event.flags |= EVENT_INVITE_SELF;\n                } else {\n                    event.flags &= !EVENT_INVITE_SELF;\n                }\n            }\n            (JSCalendarProperty::MayInviteOthers, Value::Bool(set)) => {\n                if set {\n                    event.flags |= EVENT_INVITE_OTHERS;\n                } else {\n                    event.flags &= !EVENT_INVITE_OTHERS;\n                }\n            }\n            (JSCalendarProperty::HideAttendees, Value::Bool(set)) => {\n                if set {\n                    event.flags |= EVENT_HIDE_ATTENDEES;\n                } else {\n                    event.flags &= !EVENT_HIDE_ATTENDEES;\n                }\n            }\n            (JSCalendarProperty::UseDefaultAlerts, Value::Bool(set)) => {\n                use_default_alerts = set;\n            }\n            (JSCalendarProperty::UtcStart, Value::Element(JSCalendarValue::DateTime(start))) => {\n                utc_start = Some(start.timestamp);\n            }\n            (JSCalendarProperty::UtcEnd, Value::Element(JSCalendarValue::DateTime(end))) => {\n                utc_end = Some(end.timestamp);\n            }\n            (JSCalendarProperty::CalendarIds, value) => {\n                patch_parent_ids(&mut event.names, None, value)?;\n            }\n            (JSCalendarProperty::Pointer(pointer), value) => {\n                if matches!(\n                    pointer.first(),\n                    Some(JsonPointerItem::Key(Key::Property(\n                        JSCalendarProperty::CalendarIds\n                    )))\n                ) {\n                    let mut pointer = pointer.iter();\n                    pointer.next();\n                    patch_parent_ids(&mut event.names, pointer.next(), value)?;\n                } else if !js_calendar_event.patch_jptr(pointer.iter(), value) {\n                    return Err(SetError::invalid_properties()\n                        .with_property(JSCalendarProperty::Pointer(pointer))\n                        .with_description(\"Patch operation failed.\"));\n                }\n                entries = js_calendar_event.as_object_mut().unwrap();\n            }\n            (\n                property @ (JSCalendarProperty::Id\n                | JSCalendarProperty::BaseEventId\n                | JSCalendarProperty::IsOrigin\n                | JSCalendarProperty::Method),\n                _,\n            ) => {\n                return Err(SetError::invalid_properties()\n                    .with_property(property)\n                    .with_description(\"This property is immutable.\"));\n            }\n            (\n                property @ (JSCalendarProperty::IsDraft\n                | JSCalendarProperty::MayInviteSelf\n                | JSCalendarProperty::MayInviteOthers\n                | JSCalendarProperty::HideAttendees\n                | JSCalendarProperty::UseDefaultAlerts\n                | JSCalendarProperty::UtcStart\n                | JSCalendarProperty::UtcEnd),\n                _,\n            ) => {\n                return Err(SetError::invalid_properties()\n                    .with_property(property)\n                    .with_description(\"Invalid value.\"));\n            }\n            (\n                property @ (JSCalendarProperty::Locations | JSCalendarProperty::Participants),\n                Value::Object(values),\n            ) => {\n                for (_, value) in values.iter() {\n                    if let Some(values) = value\n                        .as_object_and_get(&Key::Property(JSCalendarProperty::Links))\n                        .and_then(|v| v.as_object())\n                    {\n                        for (_, value) in values.iter() {\n                            if value.as_object().is_some_and(|v| {\n                                v.keys()\n                                    .any(|k| matches!(k, Key::Property(JSCalendarProperty::BlobId)))\n                            }) {\n                                return Err(SetError::invalid_properties()\n                                    .with_property(property)\n                                    .with_description(\"blobIds in links is not supported.\"));\n                            }\n                        }\n                    }\n                }\n                entries.insert(property, Value::Object(values));\n            }\n            (property, value) => {\n                if let (JSCalendarProperty::ShowWithoutTime, Value::Bool(set)) = (&property, &value)\n                {\n                    show_without_time = *set;\n                }\n\n                entries.insert(property, value);\n            }\n        }\n    }\n\n    // Validate UTC start/end\n    if let (Some(mut start), Some(mut end)) = (utc_start, utc_end) {\n        if start >= end {\n            return Err(SetError::invalid_properties()\n                .with_properties([JSCalendarProperty::UtcStart, JSCalendarProperty::UtcEnd])\n                .with_description(\"utcStart must be before utcEnd.\"));\n        }\n\n        if let Some(timezone) = entries\n            .get(&Key::Property(JSCalendarProperty::TimeZone))\n            .and_then(|v| v.as_str())\n            .and_then(|tz| Tz::from_str(tz.as_ref()).ok())\n        {\n            if let Some(dt_start) =\n                DateTime::from_timestamp(start, 0).map(|dt| dt.with_timezone(&timezone))\n            {\n                start = dt_start.naive_local().and_utc().timestamp();\n            }\n            if let Some(dt_end) =\n                DateTime::from_timestamp(end, 0).map(|dt| dt.with_timezone(&timezone))\n            {\n                end = dt_end.naive_local().and_utc().timestamp();\n            }\n        } else {\n            entries.insert(\n                Key::Property(JSCalendarProperty::TimeZone),\n                Value::Str(Cow::Borrowed(\"Etc/UTC\")),\n            );\n        }\n\n        entries.insert(\n            Key::Property(JSCalendarProperty::Start),\n            Value::Element(JSCalendarValue::DateTime(JSCalendarDateTime::new(\n                start, true,\n            ))),\n        );\n        entries.insert(\n            Key::Property(JSCalendarProperty::Duration),\n            Value::Element(JSCalendarValue::Duration(ICalendarDuration::from_seconds(\n                end - start,\n            ))),\n        );\n    } else if utc_start.is_some() || utc_end.is_some() {\n        return Err(SetError::invalid_properties()\n            .with_properties([JSCalendarProperty::UtcStart, JSCalendarProperty::UtcEnd])\n            .with_description(\"Both utcStart and utcEnd must be provided.\"));\n    }\n\n    // Make sure the calendar_event belongs to at least one calendar\n    if event.names.is_empty() {\n        return Err(SetError::invalid_properties()\n            .with_property(JSCalendarProperty::CalendarIds)\n            .with_description(\"Event has to belong to at least one calendar.\"));\n    }\n\n    Ok(use_default_alerts.then_some(show_without_time))\n}\n\nfn patch_parent_ids(\n    current: &mut Vec<DavName>,\n    patch: Option<&JsonPointerItem<JSCalendarProperty<Id>>>,\n    update: Value<'_, JSCalendarProperty<Id>, JSCalendarValue<Id, BlobId>>,\n) -> Result<(), SetError<JSCalendarProperty<Id>>> {\n    match (patch, update) {\n        (\n            Some(JsonPointerItem::Key(Key::Property(JSCalendarProperty::IdValue(id)))),\n            Value::Bool(false) | Value::Null,\n        ) => {\n            let id = id.document_id();\n            current.retain(|name| name.parent_id != id);\n            Ok(())\n        }\n        (\n            Some(JsonPointerItem::Key(Key::Property(JSCalendarProperty::IdValue(id)))),\n            Value::Bool(true),\n        ) => {\n            let id = id.document_id();\n            if !current.iter().any(|name| name.parent_id == id) {\n                current.push(DavName::new_with_rand_name(id));\n            }\n            Ok(())\n        }\n        (None, Value::Object(object)) => {\n            let mut new_ids = object\n                .into_expanded_boolean_set()\n                .filter_map(|id| {\n                    if let Key::Property(JSCalendarProperty::IdValue(id)) = id {\n                        Some(id.document_id())\n                    } else {\n                        None\n                    }\n                })\n                .collect::<AHashSet<_>>();\n\n            current.retain(|name| new_ids.remove(&name.parent_id));\n\n            for id in new_ids {\n                current.push(DavName::new_with_rand_name(id));\n            }\n\n            Ok(())\n        }\n        _ => Err(SetError::invalid_properties()\n            .with_property(JSCalendarProperty::CalendarIds)\n            .with_description(\"Invalid patch operation for calendarIds.\")),\n    }\n}\n\nfn default_alert_to_ical(alert: &ArchivedDefaultAlert) -> ICalendarComponent {\n    let flags = alert.flags.to_native();\n    ICalendarComponent {\n        component_type: ICalendarComponentType::VAlarm,\n        entries: vec![\n            ICalendarEntry::new(ICalendarProperty::Action).with_value(\n                if flags & ALERT_EMAIL != 0 {\n                    ICalendarValue::Action(ICalendarAction::Email)\n                } else {\n                    ICalendarValue::Action(ICalendarAction::Display)\n                },\n            ),\n            ICalendarEntry::new(ICalendarProperty::Trigger)\n                .with_param_opt((flags & ALERT_RELATIVE_TO_END != 0).then_some(\n                    ICalendarParameter::related(ICalendarParameterValue::Related(\n                        ICalendarRelated::End,\n                    )),\n                ))\n                .with_value(ICalendarValue::Duration(alert.offset.to_native())),\n        ],\n        component_ids: vec![],\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event_notification/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::changes::state::JmapCacheState;\nuse calcard::{\n    icalendar::{ArchivedICalendarProperty, ICalendar},\n    jscalendar::import::ConversionOptions,\n};\nuse common::{Server, auth::AccessToken};\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{\n        ArchivedChangedBy, CalendarEventNotification, EVENT_NOTIFICATION_IS_CHANGE,\n        EVENT_NOTIFICATION_IS_DRAFT,\n    },\n};\nuse jmap_proto::{\n    method::get::GetRequest,\n    object::calendar_event_notification::{\n        self, CalendarEventNotificationGetResponse, CalendarEventNotificationObject,\n        CalendarEventNotificationProperty, CalendarEventNotificationType, PersonObject,\n    },\n    types::date::UTCDate,\n};\nuse store::{ValueKey, write::{AlignedBytes, Archive, serialize::rkyv_deserialize}};\nuse trc::AddContext;\nuse types::{\n    blob::BlobId,\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub trait CalendarEventNotificationGet: Sync + Send {\n    fn calendar_event_notification_get(\n        &self,\n        request: GetRequest<calendar_event_notification::CalendarEventNotification>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<CalendarEventNotificationGetResponse>> + Send;\n}\n\nimpl CalendarEventNotificationGet for Server {\n    async fn calendar_event_notification_get(\n        &self,\n        mut request: GetRequest<calendar_event_notification::CalendarEventNotification>,\n        access_token: &AccessToken,\n    ) -> trc::Result<CalendarEventNotificationGetResponse> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            CalendarEventNotificationProperty::Id,\n            CalendarEventNotificationProperty::Created,\n            CalendarEventNotificationProperty::Type,\n            CalendarEventNotificationProperty::ChangedBy,\n        ]);\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(\n                access_token,\n                account_id,\n                SyncCollection::CalendarEventNotification,\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            cache\n                .document_ids(false)\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = CalendarEventNotificationGetResponse {\n            account_id: request.account_id.into(),\n            state: cache.get_state(false).into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the event object\n            let document_id = id.document_id();\n            let _event = if let Some(event) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEventNotification,\n                    document_id,\n                ))\n                .await?\n            {\n                event\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n            let event = _event\n                .unarchive::<CalendarEventNotification>()\n                .caused_by(trc::location!())?;\n            let mut result = CalendarEventNotificationObject {\n                id,\n                ..Default::default()\n            };\n            for property in &properties {\n                match property {\n                    CalendarEventNotificationProperty::Id => {}\n                    CalendarEventNotificationProperty::Created => {\n                        result.created = Some(UTCDate::from_timestamp(event.created.to_native()));\n                    }\n                    CalendarEventNotificationProperty::CalendarEventId => {\n                        result.calendar_event_id =\n                            event.event_id.as_ref().map(|id| id.to_native().into());\n                    }\n                    CalendarEventNotificationProperty::ChangedBy => {\n                        let mut changed_by = PersonObject::default();\n\n                        match &event.changed_by {\n                            ArchivedChangedBy::PrincipalId(id) => {\n                                if let Ok(token) = self.get_access_token(id.to_native()).await {\n                                    changed_by.name = token.description.clone().unwrap_or_default();\n                                    changed_by.email = token.emails.first().cloned();\n                                }\n                                changed_by.principal_id = Some(id.to_native().into());\n                            }\n                            ArchivedChangedBy::CalendarAddress(email) => {\n                                changed_by.email = Some(email.to_string());\n                                changed_by.calendar_address = Some(format!(\"mailto:{email}\"));\n                            }\n                        }\n\n                        result.changed_by = Some(changed_by);\n                    }\n                    CalendarEventNotificationProperty::Comment => {\n                        result.comment = event\n                            .event\n                            .components\n                            .iter()\n                            .filter(|c| c.component_type.is_scheduling_object())\n                            .flat_map(|c| c.entries.iter())\n                            .find(|e| matches!(e.name, ArchivedICalendarProperty::Comment))\n                            .and_then(|e| e.values.first().and_then(|v| v.as_text()))\n                            .map(|v| v.to_string());\n                    }\n                    CalendarEventNotificationProperty::Type => {\n                        result.notification_type =\n                            Some(if event.flags & EVENT_NOTIFICATION_IS_CHANGE != 0 {\n                                CalendarEventNotificationType::Updated\n                            } else if !event.event.components.is_empty() {\n                                CalendarEventNotificationType::Created\n                            } else {\n                                CalendarEventNotificationType::Destroyed\n                            });\n                    }\n                    CalendarEventNotificationProperty::IsDraft => {\n                        result.is_draft = Some(event.flags & EVENT_NOTIFICATION_IS_DRAFT != 0);\n                    }\n                    CalendarEventNotificationProperty::Event => {\n                        if event.flags & EVENT_NOTIFICATION_IS_CHANGE == 0 && result.event.is_none()\n                        {\n                            let js_event = rkyv_deserialize::<_, ICalendar>(&event.event)\n                                .caused_by(trc::location!())?\n                                .into_jscalendar_with_opt::<Id, BlobId>(\n                                    ConversionOptions::default()\n                                        .include_ical_components(false)\n                                        .return_first(true),\n                                );\n                            result.event = js_event.into();\n                        }\n                    }\n                    CalendarEventNotificationProperty::EventPatch => {\n                        if event.flags & EVENT_NOTIFICATION_IS_CHANGE != 0\n                            && result.event_patch.is_none()\n                        {\n                            let js_event = rkyv_deserialize::<_, ICalendar>(&event.event)\n                                .caused_by(trc::location!())?\n                                .into_jscalendar_with_opt::<Id, BlobId>(\n                                    ConversionOptions::default()\n                                        .include_ical_components(false)\n                                        .return_first(true),\n                                );\n                            result.event_patch = js_event.into();\n                        }\n                    }\n                }\n            }\n            response.list.push(result);\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event_notification/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/calendar_event_notification/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::JmapCacheState};\nuse common::{Server, auth::AccessToken};\nuse groupware::cache::GroupwareCache;\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::calendar_event_notification::{\n        CalendarEventNotification, CalendarEventNotificationComparator,\n        CalendarEventNotificationFilter,\n    },\n    request::IntoValid,\n};\nuse store::{\n    IterateParams, U32_LEN, U64_LEN, ValueKey,\n    ahash::AHashSet,\n    roaring::RoaringBitmap,\n    search::{SearchFilter, SearchQuery},\n    write::{IndexPropertyClass, SearchIndex, ValueClass, key::DeserializeBigEndian},\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::CalendarNotificationField,\n};\n\npub trait CalendarEventNotificationQuery: Sync + Send {\n    fn calendar_event_notification_query(\n        &self,\n        request: QueryRequest<CalendarEventNotification>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nstruct Notification {\n    document_id: u32,\n    created: u64,\n    event_id: u32,\n}\n\nimpl CalendarEventNotificationQuery for Server {\n    async fn calendar_event_notification_query(\n        &self,\n        mut request: QueryRequest<CalendarEventNotification>,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let mut filters = Vec::with_capacity(request.filter.len());\n        let cache = self\n            .fetch_dav_resources(\n                access_token,\n                account_id,\n                SyncCollection::CalendarEventNotification,\n            )\n            .await?;\n        let mut notifications = Vec::with_capacity(16);\n        let mut document_ids = RoaringBitmap::new();\n\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::CalendarEventNotification.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: CalendarNotificationField::CreatedToId.into(),\n                            value: 0,\n                        }),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::CalendarEventNotification.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: CalendarNotificationField::CreatedToId.into(),\n                            value: u64::MAX,\n                        }),\n                    },\n                )\n                .ascending(),\n                |key, value| {\n                    let document_id = key.deserialize_be_u32(key.len() - U32_LEN)?;\n                    notifications.push(Notification {\n                        document_id,\n                        created: key.deserialize_be_u64(key.len() - U32_LEN - U64_LEN)?,\n                        event_id: value.deserialize_be_u32(0)?,\n                    });\n                    document_ids.insert(document_id);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    CalendarEventNotificationFilter::Before(before) => {\n                        let before = before.timestamp() as u64;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            notifications\n                                .iter()\n                                .filter_map(|n| (n.created < before).then_some(n.document_id)),\n                        )))\n                    }\n                    CalendarEventNotificationFilter::After(after) => {\n                        let after = after.timestamp() as u64;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            notifications\n                                .iter()\n                                .filter_map(|n| (n.created > after).then_some(n.document_id)),\n                        )))\n                    }\n                    CalendarEventNotificationFilter::CalendarEventIds(ids) => {\n                        let ids = ids\n                            .into_valid()\n                            .map(|id| id.document_id())\n                            .collect::<AHashSet<_>>();\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            notifications\n                                .iter()\n                                .filter_map(|n| ids.contains(&n.event_id).then_some(n.document_id)),\n                        )))\n                    }\n                    unsupported => {\n                        return Err(trc::JmapEvent::UnsupportedFilter\n                            .into_err()\n                            .details(unsupported.into_string()));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        // Parse sort criteria\n        let mut is_ascending = true;\n        for comparator in request.sort.take().unwrap_or_default() {\n            match comparator.property {\n                CalendarEventNotificationComparator::Created => {\n                    is_ascending = comparator.is_ascending;\n                }\n                CalendarEventNotificationComparator::_T(unsupported) => {\n                    return Err(trc::JmapEvent::UnsupportedSort\n                        .into_err()\n                        .details(unsupported));\n                }\n            };\n        }\n        if !is_ascending {\n            notifications.reverse();\n        }\n\n        let results = SearchQuery::new(SearchIndex::InMemory)\n            .with_filters(filters)\n            .with_mask(document_ids)\n            .filter()\n            .into_bitmap();\n\n        let mut response = QueryResponseBuilder::new(\n            results.len() as usize,\n            self.core.jmap.query_max_results,\n            cache.get_state(false),\n            &request,\n        );\n\n        if !results.is_empty() {\n            let results = results.into_iter().collect::<AHashSet<_>>();\n            for notification in notifications {\n                if results.contains(&notification.document_id)\n                    && !response.add(0, notification.document_id)\n                {\n                    break;\n                }\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/calendar_event_notification/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse groupware::{DestroyArchive, cache::GroupwareCache, calendar::CalendarEventNotification};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::set::{SetRequest, SetResponse},\n    object::calendar_event_notification,\n    request::IntoValid,\n    types::state::State,\n};\nuse store::{ValueKey, write::{AlignedBytes, Archive, BatchBuilder}};\nuse trc::AddContext;\nuse types::collection::{Collection, SyncCollection};\n\npub trait CalendarEventNotificationSet: Sync + Send {\n    fn calendar_event_notification_set(\n        &self,\n        request: SetRequest<'_, calendar_event_notification::CalendarEventNotification>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<\n        Output = trc::Result<SetResponse<calendar_event_notification::CalendarEventNotification>>,\n    > + Send;\n}\n\nimpl CalendarEventNotificationSet for Server {\n    async fn calendar_event_notification_set(\n        &self,\n        mut request: SetRequest<'_, calendar_event_notification::CalendarEventNotification>,\n        access_token: &AccessToken,\n        _session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<calendar_event_notification::CalendarEventNotification>> {\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(\n                access_token,\n                account_id,\n                SyncCollection::CalendarEventNotification,\n            )\n            .await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n\n        let mut batch = BatchBuilder::new();\n        for (id, _) in request.unwrap_create() {\n            response.not_created.append(\n                id,\n                SetError::forbidden().with_description(\"Cannot create event notifications.\"),\n            );\n        }\n\n        // Process updates\n        for (id, _) in request.unwrap_update().into_valid() {\n            response.not_updated.append(\n                id,\n                SetError::forbidden().with_description(\"Cannot update event notifications.\"),\n            );\n        }\n\n        // Process deletions\n        for id in request.unwrap_destroy().into_valid() {\n            let document_id = id.document_id();\n\n            if !cache.has_item_id(&document_id) {\n                response.not_destroyed.append(id, SetError::not_found());\n                continue;\n            };\n\n            let _event = if let Some(event) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEventNotification,\n                    document_id,\n                ))\n                .await?\n            {\n                event\n            } else {\n                response.not_destroyed.append(id, SetError::not_found());\n                continue;\n            };\n            let event = _event\n                .to_unarchived::<CalendarEventNotification>()\n                .caused_by(trc::location!())?;\n\n            DestroyArchive(event)\n                .delete(access_token, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?;\n\n            response.destroyed.push(id);\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/changes/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::auth::JmapAuthorization, changes::state::JmapCacheState};\nuse common::{Server, auth::AccessToken};\nuse email::cache::MessageCacheFetch;\nuse groupware::cache::GroupwareCache;\nuse jmap_proto::{\n    method::changes::{ChangesRequest, ChangesResponse},\n    object::{JmapObject, NullObject, mailbox::MailboxProperty},\n    request::method::MethodObject,\n    response::{ChangesResponseMethod, ResponseMethod},\n    types::state::State,\n};\nuse std::future::Future;\nuse store::query::log::{Change, Query};\nuse trc::AddContext;\nuse types::collection::{Collection, SyncCollection};\n\npub trait ChangesLookup: Sync + Send {\n    fn changes(\n        &self,\n        request: ChangesRequest,\n        object: MethodObject,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<IntermediateChangesResponse>> + Send;\n}\n\npub struct IntermediateChangesResponse {\n    pub response: ChangesResponse<NullObject>,\n    pub object: MethodObject,\n    pub only_container_changes: bool,\n}\n\nimpl ChangesLookup for Server {\n    async fn changes(\n        &self,\n        request: ChangesRequest,\n        object: MethodObject,\n        access_token: &AccessToken,\n    ) -> trc::Result<IntermediateChangesResponse> {\n        // Map collection and validate ACLs\n        let (collection, is_container) = match object {\n            MethodObject::Email => {\n                access_token.assert_has_access(request.account_id, Collection::Email)?;\n                (SyncCollection::Email, false)\n            }\n            MethodObject::Mailbox => {\n                access_token.assert_has_access(request.account_id, Collection::Mailbox)?;\n\n                (SyncCollection::Email, true)\n            }\n            MethodObject::Thread => {\n                access_token.assert_has_access(request.account_id, Collection::Email)?;\n\n                (SyncCollection::Thread, true)\n            }\n            MethodObject::Identity => {\n                access_token.assert_is_member(request.account_id)?;\n\n                (SyncCollection::Identity, false)\n            }\n            MethodObject::EmailSubmission => {\n                access_token.assert_is_member(request.account_id)?;\n\n                (SyncCollection::EmailSubmission, false)\n            }\n            MethodObject::AddressBook => {\n                access_token.assert_has_access(request.account_id, Collection::AddressBook)?;\n\n                (SyncCollection::AddressBook, true)\n            }\n            MethodObject::ContactCard => {\n                access_token.assert_has_access(request.account_id, Collection::ContactCard)?;\n\n                (SyncCollection::AddressBook, false)\n            }\n            MethodObject::FileNode => {\n                access_token.assert_has_access(request.account_id, Collection::FileNode)?;\n\n                (SyncCollection::FileNode, false)\n            }\n            MethodObject::Calendar => {\n                access_token.assert_has_access(request.account_id, Collection::Calendar)?;\n\n                (SyncCollection::Calendar, true)\n            }\n            MethodObject::CalendarEvent => {\n                access_token.assert_has_access(request.account_id, Collection::CalendarEvent)?;\n\n                (SyncCollection::Calendar, false)\n            }\n            MethodObject::CalendarEventNotification => {\n                access_token.assert_is_member(request.account_id)?;\n\n                (SyncCollection::CalendarEventNotification, false)\n            }\n            MethodObject::ShareNotification => {\n                access_token.assert_is_member(request.account_id)?;\n\n                (SyncCollection::ShareNotification, false)\n            }\n            _ => {\n                return Err(trc::JmapEvent::CannotCalculateChanges.into_err());\n            }\n        };\n        let max_changes = std::cmp::min(\n            request\n                .max_changes\n                .filter(|n| *n != 0)\n                .unwrap_or(usize::MAX),\n            self.core.jmap.changes_max_results.unwrap_or(usize::MAX),\n        );\n        let mut response: ChangesResponse<NullObject> = ChangesResponse {\n            account_id: request.account_id,\n            old_state: request.since_state.clone(),\n            new_state: State::Initial,\n            has_more_changes: false,\n            created: vec![],\n            updated: vec![],\n            destroyed: vec![],\n            updated_properties: None,\n        };\n        let account_id = request.account_id.document_id();\n\n        let (items_sent, changelog) = match &request.since_state {\n            State::Initial => {\n                let changelog = self\n                    .store()\n                    .changes(account_id, collection.into(), Query::All)\n                    .await?;\n                if changelog.changes.is_empty() && changelog.from_change_id == 0 {\n                    return Ok(IntermediateChangesResponse {\n                        response,\n                        object,\n                        only_container_changes: false,\n                    });\n                }\n\n                (0, changelog)\n            }\n            State::Exact(change_id) => {\n                let last_state = match collection {\n                    SyncCollection::Calendar | SyncCollection::AddressBook => self\n                        .fetch_dav_resources(access_token, account_id, collection)\n                        .await\n                        .caused_by(trc::location!())?\n                        .get_state(is_container)\n                        .into(),\n                    SyncCollection::Email => self\n                        .get_cached_messages(account_id)\n                        .await?\n                        .get_state(is_container)\n                        .into(),\n                    _ => None,\n                };\n\n                if let Some(last_state) = last_state {\n                    response.new_state = last_state;\n\n                    if response.new_state == State::Exact(*change_id) {\n                        return Ok(IntermediateChangesResponse {\n                            response,\n                            object,\n                            only_container_changes: false,\n                        });\n                    }\n                }\n\n                (\n                    0,\n                    self.store()\n                        .changes(account_id, collection.into(), Query::Since(*change_id))\n                        .await?,\n                )\n            }\n            State::Intermediate(intermediate_state) => {\n                let changelog = self\n                    .store()\n                    .changes(\n                        account_id,\n                        collection.into(),\n                        Query::RangeInclusive(intermediate_state.from_id, intermediate_state.to_id),\n                    )\n                    .await?;\n                if (is_container\n                    && intermediate_state.items_sent >= changelog.total_container_changes())\n                    || (!is_container\n                        && intermediate_state.items_sent >= changelog.total_item_changes())\n                {\n                    (\n                        0,\n                        self.store()\n                            .changes(\n                                account_id,\n                                collection.into(),\n                                Query::Since(intermediate_state.to_id),\n                            )\n                            .await?,\n                    )\n                } else {\n                    (intermediate_state.items_sent, changelog)\n                }\n            }\n        };\n\n        if (changelog.is_truncated || changelog.from_change_id == 0)\n            && request.since_state != State::Initial\n        {\n            return Err(trc::JmapEvent::CannotCalculateChanges.into_err().details(\n                if changelog.is_truncated {\n                    \"Change log is truncated\"\n                } else {\n                    \"Since state is invalid\"\n                },\n            ));\n        }\n\n        let mut changes = changelog\n            .changes\n            .into_iter()\n            .filter(|change| {\n                (is_container && change.is_container_change())\n                    || (!is_container && change.is_item_change())\n            })\n            .skip(items_sent)\n            .peekable();\n\n        let mut items_changed = false;\n        for change in (&mut changes).take(max_changes) {\n            match change {\n                Change::InsertContainer(item) | Change::InsertItem(item) => {\n                    response.created.push(item.into());\n                }\n                Change::UpdateContainer(item) | Change::UpdateItem(item) => {\n                    response.updated.push(item.into());\n                    items_changed = true;\n                }\n                Change::DeleteContainer(item) | Change::DeleteItem(item) => {\n                    response.destroyed.push(item.into());\n                }\n                Change::UpdateContainerProperty(item) => {\n                    response.updated.push(item.into());\n                }\n            };\n        }\n\n        let change_id = (if is_container {\n            changelog.container_change_id\n        } else {\n            changelog.item_change_id\n        })\n        .unwrap_or(changelog.to_change_id);\n\n        response.has_more_changes = changes.peek().is_some();\n        if response.has_more_changes {\n            response.new_state = State::new_intermediate(\n                changelog.from_change_id,\n                change_id,\n                items_sent + max_changes,\n            );\n        } else if response.new_state == State::Initial {\n            response.new_state = State::new_exact(change_id)\n        }\n\n        Ok(IntermediateChangesResponse {\n            only_container_changes: is_container && !response.updated.is_empty() && !items_changed,\n            response,\n            object,\n        })\n    }\n}\n\nimpl IntermediateChangesResponse {\n    pub fn into_method_response(self) -> ResponseMethod<'static> {\n        ResponseMethod::Changes(match self.object {\n            MethodObject::Email => ChangesResponseMethod::Email(transmute_response(self.response)),\n            MethodObject::Mailbox => {\n                let mut response = transmute_response(self.response);\n                if self.only_container_changes {\n                    response.updated_properties = vec![\n                        MailboxProperty::TotalEmails.into(),\n                        MailboxProperty::UnreadEmails.into(),\n                        MailboxProperty::TotalThreads.into(),\n                        MailboxProperty::UnreadThreads.into(),\n                    ]\n                    .into();\n                }\n                ChangesResponseMethod::Mailbox(response)\n            }\n            MethodObject::Thread => {\n                ChangesResponseMethod::Thread(transmute_response(self.response))\n            }\n            MethodObject::Identity => {\n                ChangesResponseMethod::Identity(transmute_response(self.response))\n            }\n            MethodObject::EmailSubmission => {\n                ChangesResponseMethod::EmailSubmission(transmute_response(self.response))\n            }\n            MethodObject::AddressBook => {\n                ChangesResponseMethod::AddressBook(transmute_response(self.response))\n            }\n            MethodObject::ContactCard => {\n                ChangesResponseMethod::ContactCard(transmute_response(self.response))\n            }\n            MethodObject::FileNode => {\n                ChangesResponseMethod::FileNode(transmute_response(self.response))\n            }\n            MethodObject::Calendar => {\n                ChangesResponseMethod::Calendar(transmute_response(self.response))\n            }\n            MethodObject::CalendarEvent => {\n                ChangesResponseMethod::CalendarEvent(transmute_response(self.response))\n            }\n            MethodObject::CalendarEventNotification => {\n                ChangesResponseMethod::CalendarEventNotification(transmute_response(self.response))\n            }\n            MethodObject::ShareNotification => {\n                ChangesResponseMethod::ShareNotification(transmute_response(self.response))\n            }\n            MethodObject::ParticipantIdentity\n            | MethodObject::Core\n            | MethodObject::Blob\n            | MethodObject::PushSubscription\n            | MethodObject::SearchSnippet\n            | MethodObject::VacationResponse\n            | MethodObject::SieveScript\n            | MethodObject::Principal\n            | MethodObject::Quota => unreachable!(),\n        })\n    }\n}\n\nfn transmute_response<T: JmapObject>(response: ChangesResponse<NullObject>) -> ChangesResponse<T> {\n    ChangesResponse {\n        account_id: response.account_id,\n        old_state: response.old_state,\n        new_state: response.new_state,\n        has_more_changes: response.has_more_changes,\n        created: response.created,\n        updated: response.updated,\n        destroyed: response.destroyed,\n        updated_properties: None,\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/changes/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\npub mod state;\n"
  },
  {
    "path": "crates/jmap/src/changes/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::get::ChangesLookup;\nuse crate::{\n    api::request::set_account_id_if_missing, calendar_event::query::CalendarEventQuery,\n    calendar_event_notification::query::CalendarEventNotificationQuery,\n    contact::query::ContactCardQuery, email::query::EmailQuery, file::query::FileNodeQuery,\n    mailbox::query::MailboxQuery, share_notification::query::ShareNotificationQuery,\n    sieve::query::SieveScriptQuery, submission::query::EmailSubmissionQuery,\n};\nuse common::{Server, auth::AccessToken};\nuse jmap_proto::{\n    method::{\n        changes::{ChangesRequest, ChangesResponse},\n        query_changes::{AddedItem, QueryChangesRequest, QueryChangesResponse},\n    },\n    object::{JmapObject, NullObject},\n    request::{QueryChangesRequestMethod, method::MethodObject},\n};\nuse std::future::Future;\n\npub trait QueryChanges: Sync + Send {\n    fn query_changes(\n        &self,\n        request: QueryChangesRequestMethod,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryChangesResponse>> + Send;\n}\n\nimpl QueryChanges for Server {\n    async fn query_changes(\n        &self,\n        request: QueryChangesRequestMethod,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryChangesResponse> {\n        let mut response;\n        let mut is_mutable = true;\n        let results;\n        let changes;\n        let has_changes;\n        let up_to_id;\n\n        match request {\n            QueryChangesRequestMethod::Email(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::Email,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                is_mutable = request.filter.iter().any(|f| !f.is_immutable())\n                    || request\n                        .sort\n                        .as_ref()\n                        .is_some_and(|sort| sort.iter().any(|s| !s.is_immutable()));\n\n                results = self.email_query(request.into(), access_token).await?;\n            }\n            QueryChangesRequestMethod::Mailbox(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::Mailbox,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self.mailbox_query(request.into(), access_token).await?;\n            }\n            QueryChangesRequestMethod::EmailSubmission(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::EmailSubmission,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self.email_submission_query(request.into()).await?;\n            }\n            QueryChangesRequestMethod::Sieve(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::SieveScript,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self.sieve_script_query(request.into()).await?;\n            }\n            QueryChangesRequestMethod::ContactCard(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::ContactCard,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self\n                    .contact_card_query(request.into(), access_token)\n                    .await?;\n            }\n            QueryChangesRequestMethod::FileNode(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::FileNode,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self.file_node_query(request.into(), access_token).await?;\n            }\n            QueryChangesRequestMethod::CalendarEvent(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::CalendarEvent,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self\n                    .calendar_event_query(request.into(), access_token)\n                    .await?;\n            }\n            QueryChangesRequestMethod::CalendarEventNotification(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::CalendarEventNotification,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self\n                    .calendar_event_notification_query(request.into(), access_token)\n                    .await?;\n            }\n            QueryChangesRequestMethod::ShareNotification(mut request) => {\n                // Query changes\n                set_account_id_if_missing(&mut request.account_id, access_token);\n                changes = self\n                    .changes(\n                        build_changes_request(&request),\n                        MethodObject::ShareNotification,\n                        access_token,\n                    )\n                    .await?\n                    .response;\n                let calculate_total = request.calculate_total.unwrap_or(false);\n                has_changes = changes.has_changes();\n                response = build_query_changes_response(&request, &changes);\n\n                if !has_changes && !calculate_total {\n                    return Ok(response);\n                }\n\n                up_to_id = request.up_to_id;\n                results = self.share_notification_query(request.into()).await?;\n            }\n            QueryChangesRequestMethod::Principal(_) => {\n                return Err(trc::JmapEvent::CannotCalculateChanges.into_err());\n            }\n            QueryChangesRequestMethod::Quota(_) => {\n                return Err(trc::JmapEvent::CannotCalculateChanges.into_err());\n            }\n        }\n\n        if has_changes {\n            if is_mutable {\n                for (index, id) in results.ids.into_iter().enumerate() {\n                    if changes.created.contains(&id) || changes.updated.contains(&id) {\n                        response.added.push(AddedItem::new(id, index));\n                    }\n                }\n\n                response.removed = changes.updated;\n            } else {\n                for (index, id) in results.ids.into_iter().enumerate() {\n                    if changes.created.contains(&id) {\n                        response.added.push(AddedItem::new(id, index));\n                    }\n                    if matches!(up_to_id, Some(up_to_id) if up_to_id == id) {\n                        break;\n                    }\n                }\n            }\n\n            if !changes.destroyed.is_empty() {\n                response.removed.extend(changes.destroyed);\n            }\n        }\n        response.total = results.total;\n\n        Ok(response)\n    }\n}\n\nfn build_changes_request<T: JmapObject>(req: &QueryChangesRequest<T>) -> ChangesRequest {\n    ChangesRequest {\n        account_id: req.account_id,\n        since_state: req.since_query_state.clone(),\n        max_changes: req.max_changes,\n    }\n}\n\nfn build_query_changes_response<T: JmapObject>(\n    req: &QueryChangesRequest<T>,\n    changes: &ChangesResponse<NullObject>,\n) -> QueryChangesResponse {\n    QueryChangesResponse {\n        account_id: req.account_id,\n        old_query_state: changes.old_state.clone(),\n        new_query_state: changes.new_state.clone(),\n        total: None,\n        removed: vec![],\n        added: vec![],\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/changes/state.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{DavResources, MessageStoreCache, Server};\nuse jmap_proto::types::state::State;\nuse std::future::Future;\nuse trc::AddContext;\nuse types::collection::SyncCollection;\n\npub trait StateManager: Sync + Send {\n    fn get_state(\n        &self,\n        account_id: u32,\n        collection: SyncCollection,\n    ) -> impl Future<Output = trc::Result<State>> + Send;\n\n    fn assert_state(\n        &self,\n        account_id: u32,\n        collection: SyncCollection,\n        if_in_state: &Option<State>,\n    ) -> impl Future<Output = trc::Result<State>> + Send;\n}\n\npub trait JmapCacheState: Sync + Send {\n    fn get_state(&self, is_container: bool) -> State;\n\n    fn assert_state(&self, is_container: bool, if_in_state: &Option<State>) -> trc::Result<State> {\n        let old_state: State = self.get_state(is_container);\n        if let Some(if_in_state) = if_in_state\n            && &old_state != if_in_state\n        {\n            return Err(trc::JmapEvent::StateMismatch.into_err());\n        }\n        Ok(old_state)\n    }\n}\n\nimpl StateManager for Server {\n    async fn get_state(&self, account_id: u32, collection: SyncCollection) -> trc::Result<State> {\n        self.core\n            .storage\n            .data\n            .get_last_change_id(account_id, collection.into())\n            .await\n            .caused_by(trc::location!())\n            .map(State::from)\n    }\n\n    async fn assert_state(\n        &self,\n        account_id: u32,\n        collection: SyncCollection,\n        if_in_state: &Option<State>,\n    ) -> trc::Result<State> {\n        let old_state: State = self.get_state(account_id, collection).await?;\n        if let Some(if_in_state) = if_in_state\n            && &old_state != if_in_state\n        {\n            return Err(trc::JmapEvent::StateMismatch.into_err());\n        }\n\n        Ok(old_state)\n    }\n}\n\nimpl JmapCacheState for MessageStoreCache {\n    fn get_state(&self, is_container: bool) -> State {\n        if is_container {\n            State::from(self.mailboxes.change_id)\n        } else {\n            State::from(self.emails.change_id)\n        }\n    }\n}\n\nimpl JmapCacheState for DavResources {\n    fn get_state(&self, is_container: bool) -> State {\n        if is_container {\n            State::from(self.container_change_id)\n        } else {\n            State::from(self.item_change_id)\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/contact/copy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{changes::state::JmapCacheState, contact::set::ContactCardSet};\nuse common::{Server, auth::AccessToken};\nuse groupware::{cache::GroupwareCache, contact::ContactCard};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::{\n        copy::{CopyRequest, CopyResponse},\n        set::SetRequest,\n    },\n    object::contact,\n    request::{\n        Call, IntoValid, MaybeInvalid, RequestMethod, SetRequestMethod,\n        method::{MethodFunction, MethodName, MethodObject},\n        reference::MaybeResultReference,\n    },\n    types::state::State,\n};\nuse store::{ValueKey, roaring::RoaringBitmap, write::{AlignedBytes, Archive, BatchBuilder}};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n};\nuse utils::map::vec_map::VecMap;\n\npub trait JmapContactCardCopy: Sync + Send {\n    fn contact_card_copy<'x>(\n        &self,\n        request: CopyRequest<'x, contact::ContactCard>,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<CopyResponse<contact::ContactCard>>> + Send;\n}\n\nimpl JmapContactCardCopy for Server {\n    async fn contact_card_copy<'x>(\n        &self,\n        request: CopyRequest<'x, contact::ContactCard>,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        _session: &HttpSessionData,\n    ) -> trc::Result<CopyResponse<contact::ContactCard>> {\n        let account_id = request.account_id.document_id();\n        let from_account_id = request.from_account_id.document_id();\n\n        if account_id == from_account_id {\n            return Err(trc::JmapEvent::InvalidArguments\n                .into_err()\n                .details(\"From accountId is equal to fromAccountId\"));\n        }\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n        let old_state = cache.assert_state(false, &request.if_in_state)?;\n        let mut response = CopyResponse {\n            from_account_id: request.from_account_id,\n            account_id: request.account_id,\n            new_state: old_state.clone(),\n            old_state,\n            created: VecMap::with_capacity(request.create.len()),\n            not_created: VecMap::new(),\n        };\n\n        let from_cache = self\n            .fetch_dav_resources(access_token, from_account_id, SyncCollection::AddressBook)\n            .await\n            .caused_by(trc::location!())?;\n        let from_contact_ids = if access_token.is_member(from_account_id) {\n            from_cache.document_ids(false).collect::<RoaringBitmap>()\n        } else {\n            from_cache.shared_items(access_token, [Acl::ReadItems], true)\n        };\n\n        let can_add_address_books = if access_token.is_shared(account_id) {\n            cache\n                .shared_containers(access_token, [Acl::AddItems], true)\n                .into()\n        } else {\n            None\n        };\n        let on_success_delete = request.on_success_destroy_original.unwrap_or(false);\n        let mut destroy_ids = Vec::new();\n\n        // Obtain quota\n        let mut batch = BatchBuilder::new();\n\n        'create: for (id, create) in request.create.into_valid() {\n            let from_contact_id = id.document_id();\n            if !from_contact_ids.contains(from_contact_id) {\n                response.not_created.append(\n                    id,\n                    SetError::not_found().with_description(format!(\n                        \"Item {} not found in account {}.\",\n                        id, response.from_account_id\n                    )),\n                );\n                continue;\n            }\n\n            let Some(_contact) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    from_account_id,\n                    Collection::ContactCard,\n                    from_contact_id,\n                ))\n                .await?\n            else {\n                response.not_created.append(\n                    id,\n                    SetError::not_found().with_description(format!(\n                        \"Item {} not found in account {}.\",\n                        id, response.from_account_id\n                    )),\n                );\n                continue;\n            };\n\n            let contact = _contact\n                .deserialize::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            match self\n                .create_contact_card(\n                    &cache,\n                    &mut batch,\n                    access_token,\n                    account_id,\n                    &can_add_address_books,\n                    contact.card.into_jscontact(),\n                    create,\n                )\n                .await?\n            {\n                Ok(document_id) => {\n                    response.created(id, document_id);\n\n                    // Add to destroy list\n                    if on_success_delete {\n                        destroy_ids.push(MaybeInvalid::Value(id));\n                    }\n                }\n                Err(err) => {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                }\n            }\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n\n            response.new_state = State::Exact(change_id);\n        }\n\n        // Destroy ids\n        if on_success_delete && !destroy_ids.is_empty() {\n            *next_call = Call {\n                id: String::new(),\n                name: MethodName::new(MethodObject::ContactCard, MethodFunction::Set),\n                method: RequestMethod::Set(SetRequestMethod::ContactCard(SetRequest {\n                    account_id: request.from_account_id,\n                    if_in_state: request.destroy_from_if_in_state,\n                    create: None,\n                    update: None,\n                    destroy: MaybeResultReference::Value(destroy_ids).into(),\n                    arguments: Default::default(),\n                })),\n            }\n            .into();\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/contact/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::changes::state::JmapCacheState;\nuse calcard::jscontact::{JSContactProperty, JSContactValue, import::ConversionOptions};\nuse common::{Server, auth::AccessToken};\nuse groupware::{cache::GroupwareCache, contact::ContactCard};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::contact,\n    request::reference::MaybeResultReference,\n};\nuse jmap_tools::{Map, Value};\nuse store::{ValueKey, roaring::RoaringBitmap, write::{AlignedBytes, Archive}};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    blob::BlobId,\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub trait ContactCardGet: Sync + Send {\n    fn contact_card_get(\n        &self,\n        request: GetRequest<contact::ContactCard>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<contact::ContactCard>>> + Send;\n}\n\nimpl ContactCardGet for Server {\n    async fn contact_card_get(\n        &self,\n        mut request: GetRequest<contact::ContactCard>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<contact::ContactCard>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let return_all_properties = request\n            .properties\n            .as_ref()\n            .is_none_or(|v| matches!(v, MaybeResultReference::Value(v) if v.is_empty()));\n        let properties =\n            request.unwrap_properties(&[JSContactProperty::Id, JSContactProperty::AddressBookIds]);\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await?;\n        let contact_ids = if access_token.is_member(account_id) {\n            cache.document_ids(false).collect::<RoaringBitmap>()\n        } else {\n            cache.shared_items(access_token, [Acl::ReadItems], true)\n        };\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            contact_ids\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: cache.get_state(false).into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n        let mut return_id = return_all_properties;\n        let mut return_address_book_ids = return_all_properties;\n        let mut return_converted_props = !return_all_properties;\n\n        if !return_all_properties {\n            for property in &properties {\n                match property {\n                    JSContactProperty::Id => {\n                        return_id = true;\n                    }\n                    JSContactProperty::AddressBookIds => {\n                        return_address_book_ids = true;\n                    }\n                    JSContactProperty::VCard => {\n                        return_converted_props = true;\n                    }\n                    _ => {}\n                }\n            }\n        }\n\n        for id in ids {\n            // Obtain the contact object\n            let document_id = id.document_id();\n            if !contact_ids.contains(document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n\n            let _contact = if let Some(contact) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::ContactCard,\n                    document_id,\n                ))\n                .await?\n            {\n                contact\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n\n            let contact = _contact\n                .deserialize::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            let jscontact = contact\n                .card\n                .into_jscontact_with_options::<Id, BlobId>(\n                    ConversionOptions::default().include_vcard_parameters(return_converted_props),\n                )\n                .into_inner();\n            let mut result = if return_all_properties {\n                jscontact.into_object().unwrap()\n            } else {\n                Map::from_iter(\n                    jscontact\n                        .into_expanded_object()\n                        .filter(|(k, _)| k.as_property().is_some_and(|p| properties.contains(p))),\n                )\n            };\n\n            if return_id {\n                result.insert_unchecked(\n                    JSContactProperty::Id,\n                    Value::Element(JSContactValue::Id(id)),\n                );\n            }\n\n            if return_address_book_ids {\n                let mut obj = Map::with_capacity(contact.names.len());\n                for id in contact.names.iter() {\n                    obj.insert_unchecked(JSContactProperty::IdValue(Id::from(id.parent_id)), true);\n                }\n                result.insert_unchecked(JSContactProperty::AddressBookIds, Value::Object(obj));\n            }\n\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/contact/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse calcard::jscontact::JSContactProperty;\nuse common::{DavName, DavResources, Server};\nuse jmap_proto::error::set::SetError;\nuse trc::AddContext;\nuse types::{collection::Collection, field::ContactField, id::Id};\n\npub mod copy;\npub mod get;\npub mod parse;\npub mod query;\npub mod set;\n\npub(super) async fn assert_is_unique_uid(\n    server: &Server,\n    resources: &DavResources,\n    account_id: u32,\n    addressbook_ids: &[DavName],\n    uid: Option<&str>,\n) -> trc::Result<Result<(), SetError<JSContactProperty<Id>>>> {\n    if let Some(uid) = uid {\n        let hits = server\n            .document_ids_matching(\n                account_id,\n                Collection::ContactCard,\n                ContactField::Uid,\n                uid.as_bytes(),\n            )\n            .await\n            .caused_by(trc::location!())?;\n        if !hits.is_empty() {\n            for document_id in resources\n                .paths\n                .iter()\n                .filter(move |item| {\n                    item.parent_id\n                        .is_some_and(|id| addressbook_ids.iter().any(|ab| ab.parent_id == id))\n                })\n                .map(|path| resources.resources[path.resource_idx].document_id)\n            {\n                if hits.contains(document_id) {\n                    return Ok(Err(SetError::invalid_properties()\n                        .with_property(JSContactProperty::Uid)\n                        .with_description(format!(\n                            \"Contact with UID {uid} already exists with id {}.\",\n                            Id::from(document_id)\n                        ))));\n                }\n            }\n        }\n    }\n\n    Ok(Ok(()))\n}\n"
  },
  {
    "path": "crates/jmap/src/contact/parse.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::blob::download::BlobDownload;\nuse calcard::vcard::VCard;\nuse common::{Server, auth::AccessToken};\nuse jmap_proto::{\n    method::parse::{ParseRequest, ParseResponse},\n    object::contact::ContactCard,\n    request::IntoValid,\n};\nuse types::{blob::BlobId, id::Id};\nuse utils::map::vec_map::VecMap;\n\npub trait ContactCardParse: Sync + Send {\n    fn contact_card_parse(\n        &self,\n        request: ParseRequest<ContactCard>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<ParseResponse<ContactCard>>> + Send;\n}\n\nimpl ContactCardParse for Server {\n    async fn contact_card_parse(\n        &self,\n        request: ParseRequest<ContactCard>,\n        access_token: &AccessToken,\n    ) -> trc::Result<ParseResponse<ContactCard>> {\n        if request.blob_ids.len() > self.core.jmap.contact_parse_max_items {\n            return Err(trc::JmapEvent::RequestTooLarge.into_err());\n        }\n        let return_all_properties = request.properties.is_none();\n        let properties = request\n            .properties\n            .map(|v| v.into_valid().collect::<Vec<_>>())\n            .unwrap_or_default();\n\n        let mut response = ParseResponse {\n            account_id: request.account_id,\n            parsed: VecMap::with_capacity(request.blob_ids.len()),\n            not_parsable: vec![],\n            not_found: vec![],\n        };\n\n        for blob_id in request.blob_ids.into_valid() {\n            // Fetch raw message to parse\n            let raw_vcard = match self.blob_download(&blob_id, access_token).await? {\n                Some(raw_vcard) => raw_vcard,\n                None => {\n                    response.not_found.push(blob_id);\n                    continue;\n                }\n            };\n            let Ok(vcard) = VCard::parse(std::str::from_utf8(&raw_vcard).unwrap_or_default())\n            else {\n                response.not_parsable.push(blob_id);\n                continue;\n            };\n            let mut js_contact = vcard.into_jscontact::<Id, BlobId>();\n\n            if !return_all_properties {\n                js_contact\n                    .0\n                    .as_object_mut()\n                    .unwrap()\n                    .as_mut_vec()\n                    .retain(|(k, _)| k.as_property().is_some_and(|k| properties.contains(k)));\n            }\n\n            response.parsed.append(blob_id, js_contact.into_inner());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/contact/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::JmapCacheState};\nuse common::{Server, auth::AccessToken};\nuse groupware::cache::GroupwareCache;\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::contact::{ContactCard, ContactCardComparator, ContactCardFilter},\n    request::MaybeInvalid,\n};\nuse store::{\n    IterateParams, U32_LEN, U64_LEN, ValueKey,\n    roaring::RoaringBitmap,\n    search::{ContactSearchField, SearchComparator, SearchFilter, SearchQuery},\n    write::{IndexPropertyClass, SearchIndex, ValueClass, key::DeserializeBigEndian},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    field::ContactField,\n};\nuse utils::sanitize_email;\n\npub trait ContactCardQuery: Sync + Send {\n    fn contact_card_query(\n        &self,\n        request: QueryRequest<ContactCard>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\n#[derive(Clone)]\nstruct CreatedUpdated {\n    document_id: u32,\n    created: u64,\n    updated: u64,\n}\n\nimpl ContactCardQuery for Server {\n    async fn contact_card_query(\n        &self,\n        mut request: QueryRequest<ContactCard>,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let mut filters = Vec::with_capacity(request.filter.len());\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await?;\n        let mut created_to_updated = Vec::new();\n\n        if request.filter.iter().any(|cond| {\n            matches!(\n                cond,\n                Filter::Property(\n                    ContactCardFilter::CreatedBefore(_)\n                        | ContactCardFilter::CreatedAfter(_)\n                        | ContactCardFilter::UpdatedBefore(_)\n                        | ContactCardFilter::UpdatedAfter(_)\n                )\n            )\n        }) || request.sort.as_ref().is_some_and(|v| {\n            v.iter().any(|sort| {\n                matches!(\n                    sort.property,\n                    ContactCardComparator::Created | ContactCardComparator::Updated\n                )\n            })\n        }) {\n            self.store()\n                .iterate(\n                    IterateParams::new(\n                        ValueKey {\n                            account_id,\n                            collection: Collection::ContactCard.into(),\n                            document_id: 0,\n                            class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                                property: ContactField::CreatedToUpdated.into(),\n                                value: 0,\n                            }),\n                        },\n                        ValueKey {\n                            account_id,\n                            collection: Collection::ContactCard.into(),\n                            document_id: 0,\n                            class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                                property: ContactField::CreatedToUpdated.into(),\n                                value: u64::MAX,\n                            }),\n                        },\n                    )\n                    .ascending(),\n                    |key, value| {\n                        created_to_updated.push(CreatedUpdated {\n                            document_id: key.deserialize_be_u32(key.len() - U32_LEN)?,\n                            created: key.deserialize_be_u64(key.len() - U32_LEN - U64_LEN)?,\n                            updated: value.deserialize_be_u64(0)?,\n                        });\n\n                        Ok(true)\n                    },\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    ContactCardFilter::InAddressBook(MaybeInvalid::Value(id)) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.children_ids(id.document_id()),\n                        )))\n                    }\n                    ContactCardFilter::Name(value)\n                    | ContactCardFilter::NameGiven(value)\n                    | ContactCardFilter::NameSurname(value)\n                    | ContactCardFilter::NameSurname2(value) => {\n                        filters.push(SearchFilter::has_keyword(ContactSearchField::Name, value));\n                    }\n                    ContactCardFilter::Nickname(value) => {\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Nickname,\n                            value,\n                        ));\n                    }\n                    ContactCardFilter::Organization(value) => {\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Organization,\n                            value,\n                        ));\n                    }\n                    ContactCardFilter::Phone(value) => {\n                        filters.push(SearchFilter::has_keyword(ContactSearchField::Phone, value));\n                    }\n                    ContactCardFilter::OnlineService(value) => {\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::OnlineService,\n                            value,\n                        ));\n                    }\n                    ContactCardFilter::Address(value) => {\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Address,\n                            value,\n                        ));\n                    }\n                    ContactCardFilter::Note(value) => {\n                        filters.push(SearchFilter::has_text_detect(\n                            ContactSearchField::Note,\n                            value,\n                            self.core.jmap.default_language,\n                        ));\n                    }\n                    ContactCardFilter::HasMember(value) => {\n                        filters.push(SearchFilter::has_keyword(ContactSearchField::Member, value));\n                    }\n                    ContactCardFilter::Kind(value) => {\n                        filters.push(SearchFilter::eq(ContactSearchField::Kind, value));\n                    }\n                    ContactCardFilter::Uid(value) => {\n                        filters.push(SearchFilter::eq(ContactSearchField::Uid, value))\n                    }\n                    ContactCardFilter::Email(email) => filters.push(SearchFilter::has_keyword(\n                        ContactSearchField::Email,\n                        sanitize_email(&email).unwrap_or(email),\n                    )),\n                    ContactCardFilter::Text(value) => {\n                        filters.push(SearchFilter::Or);\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Name,\n                            value.clone(),\n                        ));\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Nickname,\n                            value.clone(),\n                        ));\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Organization,\n                            value.clone(),\n                        ));\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Email,\n                            value.clone(),\n                        ));\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Phone,\n                            value.clone(),\n                        ));\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::OnlineService,\n                            value.clone(),\n                        ));\n                        filters.push(SearchFilter::has_keyword(\n                            ContactSearchField::Address,\n                            value.clone(),\n                        ));\n                        filters.push(SearchFilter::has_text_detect(\n                            ContactSearchField::Note,\n                            value,\n                            self.core.jmap.default_language,\n                        ));\n                        filters.push(SearchFilter::End);\n                    }\n                    ContactCardFilter::CreatedBefore(before) => {\n                        let before = before.timestamp() as u64;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            created_to_updated\n                                .iter()\n                                .filter_map(|cu| (cu.created < before).then_some(cu.document_id)),\n                        )));\n                    }\n                    ContactCardFilter::CreatedAfter(after) => {\n                        let after = after.timestamp() as u64;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            created_to_updated\n                                .iter()\n                                .filter_map(|cu| (cu.created > after).then_some(cu.document_id)),\n                        )));\n                    }\n                    ContactCardFilter::UpdatedBefore(before) => {\n                        let before = before.timestamp() as u64;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            created_to_updated\n                                .iter()\n                                .filter_map(|cu| (cu.updated < before).then_some(cu.document_id)),\n                        )));\n                    }\n                    ContactCardFilter::UpdatedAfter(after) => {\n                        let after = after.timestamp() as u64;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            created_to_updated\n                                .iter()\n                                .filter_map(|cu| (cu.updated > after).then_some(cu.document_id)),\n                        )));\n                    }\n                    unsupported => {\n                        return Err(trc::JmapEvent::UnsupportedFilter\n                            .into_err()\n                            .details(unsupported.into_string()));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        let comparators = request\n            .sort\n            .take()\n            .unwrap_or_default()\n            .into_iter()\n            .map(|comparator| match comparator.property {\n                ContactCardComparator::Created => Ok(SearchComparator::sorted_set(\n                    created_to_updated\n                        .iter()\n                        .enumerate()\n                        .map(|(idx, u)| (u.document_id, idx as u32))\n                        .collect(),\n                    comparator.is_ascending,\n                )),\n                ContactCardComparator::Updated => {\n                    let mut updated = created_to_updated.clone();\n                    updated.sort_by(|a, b| a.updated.cmp(&b.updated));\n                    Ok(SearchComparator::sorted_set(\n                        updated\n                            .iter()\n                            .enumerate()\n                            .map(|(idx, u)| (u.document_id, idx as u32))\n                            .collect(),\n                        comparator.is_ascending,\n                    ))\n                }\n                other => Err(trc::JmapEvent::UnsupportedSort\n                    .into_err()\n                    .details(other.into_string())),\n            })\n            .collect::<Result<Vec<_>, _>>()?;\n\n        let results = self\n            .search_store()\n            .query_account(\n                SearchQuery::new(SearchIndex::Contacts)\n                    .with_filters(filters)\n                    .with_comparators(comparators)\n                    .with_account_id(account_id)\n                    .with_mask(if access_token.is_shared(account_id) {\n                        cache.shared_items(access_token, [Acl::ReadItems], true)\n                    } else {\n                        cache.document_ids(false).collect()\n                    }),\n            )\n            .await?;\n\n        let mut response = QueryResponseBuilder::new(\n            results.len(),\n            self.core.jmap.query_max_results,\n            cache.get_state(false),\n            &request,\n        );\n\n        for document_id in results {\n            if !response.add(0, document_id) {\n                break;\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/contact/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::contact::assert_is_unique_uid;\nuse calcard::jscontact::{JSContact, JSContactProperty, JSContactValue};\nuse common::{DavName, DavResources, Server, auth::AccessToken};\nuse groupware::{DestroyArchive, cache::GroupwareCache, contact::ContactCard};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::set::{SetRequest, SetResponse},\n    object::contact,\n    request::IntoValid,\n    types::state::State,\n};\nuse jmap_tools::{JsonPointerHandler, JsonPointerItem, Key, Value};\nuse store::{\n    ValueKey,\n    ahash::AHashSet,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    blob::BlobId,\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub trait ContactCardSet: Sync + Send {\n    fn contact_card_set(\n        &self,\n        request: SetRequest<'_, contact::ContactCard>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<SetResponse<contact::ContactCard>>> + Send;\n\n    #[allow(clippy::too_many_arguments)]\n    fn create_contact_card(\n        &self,\n        cache: &DavResources,\n        batch: &mut BatchBuilder,\n        access_token: &AccessToken,\n        account_id: u32,\n        can_add_address_books: &Option<RoaringBitmap>,\n        js_contact: JSContact<'_, Id, BlobId>,\n        updates: Value<'_, JSContactProperty<Id>, JSContactValue<Id, BlobId>>,\n    ) -> impl Future<Output = trc::Result<Result<u32, SetError<JSContactProperty<Id>>>>>;\n}\n\nimpl ContactCardSet for Server {\n    async fn contact_card_set(\n        &self,\n        mut request: SetRequest<'_, contact::ContactCard>,\n        access_token: &AccessToken,\n        _session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<contact::ContactCard>> {\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::AddressBook)\n            .await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Obtain addressBookIds\n        let (can_add_address_books, can_delete_address_books, can_modify_address_books) =\n            if access_token.is_shared(account_id) {\n                (\n                    cache\n                        .shared_containers(access_token, [Acl::AddItems], true)\n                        .into(),\n                    cache\n                        .shared_containers(access_token, [Acl::RemoveItems], true)\n                        .into(),\n                    cache\n                        .shared_containers(access_token, [Acl::ModifyItems], true)\n                        .into(),\n                )\n            } else {\n                (None, None, None)\n            };\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        'create: for (id, object) in request.unwrap_create() {\n            match self\n                .create_contact_card(\n                    &cache,\n                    &mut batch,\n                    access_token,\n                    account_id,\n                    &can_add_address_books,\n                    JSContact::default(),\n                    object,\n                )\n                .await?\n            {\n                Ok(document_id) => {\n                    response.created(id, document_id);\n                }\n                Err(err) => {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                }\n            }\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain contact card\n            let document_id = id.document_id();\n            let contact_card_ = if let Some(contact_card_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::ContactCard,\n                    document_id,\n                ))\n                .await?\n            {\n                contact_card_\n            } else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n            let contact_card = contact_card_\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n            let mut new_contact_card = contact_card\n                .deserialize::<ContactCard>()\n                .caused_by(trc::location!())?;\n            let mut js_contact = new_contact_card.card.into_jscontact();\n\n            // Process changes\n            if let Err(err) =\n                update_contact_card(object, &mut new_contact_card.names, &mut js_contact)\n            {\n                response.not_updated.append(id, err);\n                continue 'update;\n            }\n\n            // Convert JSContact to vCard\n            if let Some(vcard) = js_contact.into_vcard() {\n                new_contact_card.size = vcard.size() as u32;\n                new_contact_card.card = vcard;\n            } else {\n                response.not_updated.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_description(\"Failed to convert contact to vCard.\"),\n                );\n                continue 'update;\n            }\n\n            // Validate UID\n            match (new_contact_card.card.uid(), contact_card.inner.card.uid()) {\n                (Some(old_uid), Some(new_uid)) if old_uid == new_uid => {}\n                (None, None) | (None, Some(_)) => {}\n                _ => {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(JSContactProperty::Uid)\n                            .with_description(\"You cannot change the UID of a contact.\"),\n                    );\n                    continue 'update;\n                }\n            }\n\n            // Validate new addressBookIds\n            for addressbook_id in new_contact_card.added_addressbook_ids(contact_card.inner) {\n                if !cache.has_container_id(&addressbook_id) {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(JSContactProperty::AddressBookIds)\n                            .with_description(format!(\n                                \"addressBookId {} does not exist.\",\n                                Id::from(addressbook_id)\n                            )),\n                    );\n                    continue 'update;\n                } else if can_add_address_books\n                    .as_ref()\n                    .is_some_and(|ids| !ids.contains(addressbook_id))\n                {\n                    response.not_updated.append(\n                        id,\n                        SetError::forbidden().with_description(format!(\n                            \"You are not allowed to add contacts to address book {}.\",\n                            Id::from(addressbook_id)\n                        )),\n                    );\n                    continue 'update;\n                }\n            }\n\n            // Validate deleted addressBookIds\n            if let Some(can_delete_address_books) = &can_delete_address_books {\n                for addressbook_id in new_contact_card.removed_addressbook_ids(contact_card.inner) {\n                    if !can_delete_address_books.contains(addressbook_id) {\n                        response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(format!(\n                                \"You are not allowed to remove contacts from address book {}.\",\n                                Id::from(addressbook_id)\n                            )),\n                        );\n                        continue 'update;\n                    }\n                }\n            }\n\n            // Validate changed addressBookIds\n            if let Some(can_modify_address_books) = &can_modify_address_books {\n                for addressbook_id in new_contact_card.unchanged_addressbook_ids(contact_card.inner)\n                {\n                    if !can_modify_address_books.contains(addressbook_id) {\n                        response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(format!(\n                                \"You are not allowed to modify address book {}.\",\n                                Id::from(addressbook_id)\n                            )),\n                        );\n                        continue 'update;\n                    }\n                }\n            }\n\n            // Check size and quota\n            if new_contact_card.size as usize > self.core.groupware.max_vcard_size {\n                response.not_updated.append(\n                    id,\n                    SetError::invalid_properties().with_description(format!(\n                        \"Contact size {} exceeds the maximum allowed size of {} bytes.\",\n                        new_contact_card.size, self.core.groupware.max_vcard_size\n                    )),\n                );\n                continue 'update;\n            }\n            let extra_bytes = (new_contact_card.size as u64)\n                .saturating_sub(u32::from(contact_card.inner.size) as u64);\n            if extra_bytes > 0 {\n                match self\n                    .has_available_quota(\n                        &self.get_resource_token(access_token, account_id).await?,\n                        extra_bytes,\n                    )\n                    .await\n                {\n                    Ok(_) => {}\n                    Err(err) if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota)) => {\n                        response.not_updated.append(id, SetError::over_quota());\n                        continue 'update;\n                    }\n                    Err(err) => return Err(err.caused_by(trc::location!())),\n                }\n            }\n\n            // Update record\n            new_contact_card\n                .update(\n                    access_token,\n                    contact_card,\n                    account_id,\n                    document_id,\n                    &mut batch,\n                )\n                .caused_by(trc::location!())?;\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        'destroy: for id in will_destroy {\n            let document_id = id.document_id();\n\n            if !cache.has_item_id(&document_id) {\n                response.not_destroyed.append(id, SetError::not_found());\n                continue;\n            };\n\n            let Some(contact_card_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::ContactCard,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            else {\n                response.not_destroyed.append(id, SetError::not_found());\n                continue;\n            };\n\n            let contact_card = contact_card_\n                .to_unarchived::<ContactCard>()\n                .caused_by(trc::location!())?;\n\n            // Validate ACLs\n            if let Some(can_delete_address_books) = &can_delete_address_books {\n                for name in contact_card.inner.names.iter() {\n                    let parent_id = name.parent_id.to_native();\n                    if !can_delete_address_books.contains(parent_id) {\n                        response.not_destroyed.append(\n                            id,\n                            SetError::forbidden().with_description(format!(\n                                \"You are not allowed to remove contacts from address book {}.\",\n                                Id::from(parent_id)\n                            )),\n                        );\n                        continue 'destroy;\n                    }\n                }\n            }\n\n            // Delete record\n            DestroyArchive(contact_card)\n                .delete_all(access_token, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?;\n\n            response.destroyed.push(id);\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n\n            self.notify_task_queue();\n\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n\n    async fn create_contact_card(\n        &self,\n        cache: &DavResources,\n        batch: &mut BatchBuilder,\n        access_token: &AccessToken,\n        account_id: u32,\n        can_add_address_books: &Option<RoaringBitmap>,\n        mut js_contact: JSContact<'_, Id, BlobId>,\n        updates: Value<'_, JSContactProperty<Id>, JSContactValue<Id, BlobId>>,\n    ) -> trc::Result<Result<u32, SetError<JSContactProperty<Id>>>> {\n        // Process changes\n        let mut names = Vec::new();\n        if let Err(err) = update_contact_card(updates, &mut names, &mut js_contact) {\n            return Ok(Err(err));\n        }\n\n        // Verify that the address book ids valid\n        for name in &names {\n            if !cache.has_container_id(&name.parent_id) {\n                return Ok(Err(SetError::invalid_properties()\n                    .with_property(JSContactProperty::AddressBookIds)\n                    .with_description(format!(\n                        \"addressBookId {} does not exist.\",\n                        Id::from(name.parent_id)\n                    ))));\n            } else if can_add_address_books\n                .as_ref()\n                .is_some_and(|ids| !ids.contains(name.parent_id))\n            {\n                return Ok(Err(SetError::forbidden().with_description(format!(\n                    \"You are not allowed to add contacts to address book {}.\",\n                    Id::from(name.parent_id)\n                ))));\n            }\n        }\n\n        // Convert JSContact to vCard\n        let Some(card) = js_contact.into_vcard() else {\n            return Ok(Err(SetError::invalid_properties()\n                .with_description(\"Failed to convert contact to vCard.\")));\n        };\n\n        // Validate UID\n        if let Err(err) = assert_is_unique_uid(self, cache, account_id, &names, card.uid()).await? {\n            return Ok(Err(err));\n        }\n\n        // Check size and quota\n        let size = card.size();\n        if size > self.core.groupware.max_vcard_size {\n            return Ok(Err(SetError::invalid_properties().with_description(\n                format!(\n                    \"Contact size {} exceeds the maximum allowed size of {} bytes.\",\n                    size, self.core.groupware.max_vcard_size\n                ),\n            )));\n        }\n        match self\n            .has_available_quota(\n                &self.get_resource_token(access_token, account_id).await?,\n                size as u64,\n            )\n            .await\n        {\n            Ok(_) => {}\n            Err(err) if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota)) => {\n                return Ok(Err(SetError::over_quota()));\n            }\n            Err(err) => return Err(err.caused_by(trc::location!())),\n        }\n\n        // Insert record\n        let document_id = self\n            .store()\n            .assign_document_ids(account_id, Collection::ContactCard, 1)\n            .await\n            .caused_by(trc::location!())?;\n        ContactCard {\n            names,\n            size: size as u32,\n            card,\n            ..Default::default()\n        }\n        .insert(access_token, account_id, document_id, batch)\n        .caused_by(trc::location!())\n        .map(|_| Ok(document_id))\n    }\n}\n\nfn update_contact_card<'x>(\n    updates: Value<'x, JSContactProperty<Id>, JSContactValue<Id, BlobId>>,\n    addressbooks: &mut Vec<DavName>,\n    js_contact: &mut JSContact<'x, Id, BlobId>,\n) -> Result<(), SetError<JSContactProperty<Id>>> {\n    let mut entries = js_contact.0.as_object_mut().unwrap();\n\n    for (property, value) in updates.into_expanded_object() {\n        let Key::Property(property) = property else {\n            return Err(SetError::invalid_properties()\n                .with_property(property.to_owned())\n                .with_description(\"Invalid property.\"));\n        };\n\n        match (property, value) {\n            (JSContactProperty::AddressBookIds, value) => {\n                patch_parent_ids(addressbooks, None, value)?;\n            }\n            (JSContactProperty::Pointer(pointer), value) => {\n                if matches!(\n                    pointer.first(),\n                    Some(JsonPointerItem::Key(Key::Property(\n                        JSContactProperty::AddressBookIds\n                    )))\n                ) {\n                    let mut pointer = pointer.iter();\n                    pointer.next();\n                    patch_parent_ids(addressbooks, pointer.next(), value)?;\n                } else if !js_contact.0.patch_jptr(pointer.iter(), value) {\n                    return Err(SetError::invalid_properties()\n                        .with_property(JSContactProperty::Pointer(pointer))\n                        .with_description(\"Patch operation failed.\"));\n                }\n                entries = js_contact.0.as_object_mut().unwrap();\n            }\n            (JSContactProperty::Media, Value::Object(media)) => {\n                for (_, value) in media.iter() {\n                    if value.as_object().is_some_and(|v| {\n                        v.keys()\n                            .any(|k| matches!(k, Key::Property(JSContactProperty::BlobId)))\n                    }) {\n                        return Err(SetError::invalid_properties()\n                            .with_property(JSContactProperty::Media)\n                            .with_description(\"blobIds in media is not supported.\"));\n                    }\n                }\n                entries.insert(JSContactProperty::Media, Value::Object(media));\n            }\n            (property, value) => {\n                entries.insert(property, value);\n            }\n        }\n    }\n\n    // Make sure the contact belongs to at least one address book\n    if addressbooks.is_empty() {\n        return Err(SetError::invalid_properties()\n            .with_property(JSContactProperty::AddressBookIds)\n            .with_description(\"Contact has to belong to at least one address book.\"));\n    }\n\n    Ok(())\n}\n\nfn patch_parent_ids(\n    current: &mut Vec<DavName>,\n    patch: Option<&JsonPointerItem<JSContactProperty<Id>>>,\n    update: Value<'_, JSContactProperty<Id>, JSContactValue<Id, BlobId>>,\n) -> Result<(), SetError<JSContactProperty<Id>>> {\n    match (patch, update) {\n        (\n            Some(JsonPointerItem::Key(Key::Property(JSContactProperty::IdValue(id)))),\n            Value::Bool(false) | Value::Null,\n        ) => {\n            let id = id.document_id();\n            current.retain(|name| name.parent_id != id);\n            Ok(())\n        }\n        (\n            Some(JsonPointerItem::Key(Key::Property(JSContactProperty::IdValue(id)))),\n            Value::Bool(true),\n        ) => {\n            let id = id.document_id();\n            if !current.iter().any(|name| name.parent_id == id) {\n                current.push(DavName::new_with_rand_name(id));\n            }\n            Ok(())\n        }\n        (None, Value::Object(object)) => {\n            let mut new_ids = object\n                .into_expanded_boolean_set()\n                .filter_map(|id| {\n                    if let Key::Property(JSContactProperty::IdValue(id)) = id {\n                        Some(id.document_id())\n                    } else {\n                        None\n                    }\n                })\n                .collect::<AHashSet<_>>();\n\n            current.retain(|name| new_ids.remove(&name.parent_id));\n\n            for id in new_ids {\n                current.push(DavName::new_with_rand_name(id));\n            }\n\n            Ok(())\n        }\n        _ => Err(SetError::invalid_properties()\n            .with_property(JSContactProperty::AddressBookIds)\n            .with_description(\"Invalid patch operation for addressBookIds.\")),\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/email/body.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse email::message::metadata::{\n    ArchivedMessageMetadataContents, ArchivedMetadataHeaderValue, ArchivedMetadataPartType,\n    PART_ENCODING_BASE64, PART_ENCODING_QP, PART_SIZE_MASK,\n};\nuse jmap_proto::object::email::{EmailProperty, EmailValue};\nuse jmap_tools::{Map, Value};\nuse mail_parser::{HeaderValue, MessagePart, MimeHeaders, PartType};\nuse types::blob::BlobId;\nuse utils::chained_bytes::ChainedBytes;\n\nuse super::headers::HeaderToValue;\n\npub trait ToBodyPart {\n    fn to_body_part(\n        &self,\n        part_id: u32,\n        properties: &[EmailProperty],\n        raw_message: &ChainedBytes<'_>,\n        blob_id: &BlobId,\n        blob_body_offset: isize,\n    ) -> Value<'static, EmailProperty, EmailValue>;\n}\n\nimpl ToBodyPart for Vec<MessagePart<'_>> {\n    fn to_body_part(\n        &self,\n        part_id: u32,\n        properties: &[EmailProperty],\n        raw_message: &ChainedBytes<'_>,\n        blob_id: &BlobId,\n        blob_body_offset: isize,\n    ) -> Value<'static, EmailProperty, EmailValue> {\n        let mut parts = vec![part_id].into_iter();\n        let mut parts_stack = Vec::new();\n        let mut subparts = Vec::with_capacity(1);\n\n        loop {\n            if let Some((part_id, part)) = parts\n                .next()\n                .map(|part_id| (part_id, &self[part_id as usize]))\n            {\n                let mut values = Map::with_capacity(properties.len());\n                let multipart = if let PartType::Multipart(parts) = &part.body {\n                    parts.into()\n                } else {\n                    None\n                };\n\n                for property in properties {\n                    let value = match property {\n                        EmailProperty::PartId if multipart.is_none() => part_id.to_string().into(),\n                        EmailProperty::BlobId if multipart.is_none() => {\n                            let base_offset = blob_id.start_offset() as isize + blob_body_offset;\n                            BlobId::new_section(\n                                blob_id.hash.clone(),\n                                blob_id.class.clone(),\n                                (part.offset_body as isize + base_offset) as usize,\n                                (part.offset_end as isize + base_offset) as usize,\n                                part.encoding as u8,\n                            )\n                            .into()\n                        }\n                        EmailProperty::Size if multipart.is_none() => match &part.body {\n                            PartType::Text(text) | PartType::Html(text) => text.len(),\n                            PartType::Binary(bin) | PartType::InlineBinary(bin) => bin.len(),\n                            PartType::Message(message) => message.root_part().raw_len() as usize,\n                            PartType::Multipart(_) => 0,\n                        }\n                        .into(),\n                        EmailProperty::Name => part.attachment_name().map(|v| v.to_string()).into(),\n                        EmailProperty::Type => part\n                            .content_type()\n                            .map(|ct| {\n                                ct.subtype()\n                                    .map(|st| format!(\"{}/{}\", ct.ctype(), st))\n                                    .unwrap_or_else(|| ct.ctype().to_string())\n                            })\n                            .or_else(|| match &part.body {\n                                PartType::Text(_) => Some(\"text/plain\".to_string()),\n                                PartType::Html(_) => Some(\"text/html\".to_string()),\n                                PartType::Message(_) => Some(\"message/rfc822\".to_string()),\n                                _ => None,\n                            })\n                            .into(),\n                        EmailProperty::Charset => part\n                            .content_type()\n                            .and_then(|ct| ct.attribute(\"charset\"))\n                            .or(match &part.body {\n                                PartType::Text(_) | PartType::Html(_) => Some(\"us-ascii\"),\n                                _ => None,\n                            })\n                            .map(|v| v.to_string())\n                            .into(),\n                        EmailProperty::Disposition => part\n                            .content_disposition()\n                            .map(|cd| cd.ctype())\n                            .map(|v| v.to_string())\n                            .into(),\n                        EmailProperty::Cid => part.content_id().map(|v| v.to_string()).into(),\n                        EmailProperty::Language => match part.content_language() {\n                            HeaderValue::Text(text) => vec![text.to_string()].into(),\n                            HeaderValue::TextList(list) => list\n                                .iter()\n                                .map(|text| text.to_string().into())\n                                .collect::<Vec<Value<'static, EmailProperty, EmailValue>>>()\n                                .into(),\n                            _ => Value::Null,\n                        },\n                        EmailProperty::Location => {\n                            part.content_location().map(|v| v.to_string()).into()\n                        }\n                        EmailProperty::Header(_) => {\n                            part.headers.header_to_value(property, raw_message)\n                        }\n                        EmailProperty::Headers => part.headers.headers_to_value(raw_message),\n                        EmailProperty::SubParts => continue,\n                        _ => Value::Null,\n                    };\n                    values.insert_unchecked(property.clone(), value);\n                }\n\n                subparts.push(values);\n\n                if let Some(multipart) = multipart {\n                    if parts_stack.len() == 10_000 {\n                        debug_assert!(false, \"Too much nesting in message metadata\");\n                        return Value::Null;\n                    }\n                    let multipart = multipart.clone();\n                    parts_stack.push((\n                        parts,\n                        std::mem::replace(&mut subparts, Vec::with_capacity(multipart.len())),\n                    ));\n                    parts = multipart.into_iter();\n                }\n            } else if let Some((prev_parts, mut prev_subparts)) = parts_stack.pop() {\n                prev_subparts\n                    .last_mut()\n                    .unwrap()\n                    .insert_unchecked(EmailProperty::SubParts, subparts);\n                parts = prev_parts;\n                subparts = prev_subparts;\n            } else {\n                return subparts.pop().map(Into::into).unwrap_or_default();\n            }\n        }\n    }\n}\n\nimpl ToBodyPart for ArchivedMessageMetadataContents {\n    fn to_body_part(\n        &self,\n        part_id: u32,\n        properties: &[EmailProperty],\n        raw_message: &ChainedBytes<'_>,\n        blob_id: &BlobId,\n        blob_body_offset: isize,\n    ) -> Value<'static, EmailProperty, EmailValue> {\n        let mut parts = vec![part_id].into_iter();\n        let mut parts_stack = Vec::new();\n        let mut subparts = Vec::with_capacity(1);\n\n        loop {\n            if let Some((part_id, part)) = parts\n                .next()\n                .map(|part_id| (part_id, &self.parts[part_id as usize]))\n            {\n                let mut values = Map::with_capacity(properties.len());\n                let multipart = if let ArchivedMetadataPartType::Multipart(parts) = &part.body {\n                    parts.into()\n                } else {\n                    None\n                };\n\n                for property in properties {\n                    let value = match property {\n                        EmailProperty::PartId if multipart.is_none() => part_id.to_string().into(),\n                        EmailProperty::BlobId if multipart.is_none() => {\n                            let base_offset = blob_id.start_offset() as isize + blob_body_offset;\n                            let flags = part.flags.to_native();\n                            let encoding = if flags & PART_ENCODING_BASE64 != 0 {\n                                2\n                            } else if flags & PART_ENCODING_QP != 0 {\n                                1\n                            } else {\n                                0\n                            };\n                            BlobId::new_section(\n                                blob_id.hash.clone(),\n                                blob_id.class.clone(),\n                                (u32::from(part.offset_body) as isize + base_offset) as usize,\n                                (u32::from(part.offset_end) as isize + base_offset) as usize,\n                                encoding,\n                            )\n                            .into()\n                        }\n                        EmailProperty::Size if multipart.is_none() => {\n                            (part.flags.to_native() & PART_SIZE_MASK).into()\n                        }\n                        EmailProperty::Name => part.attachment_name().map(|v| v.to_string()).into(),\n                        EmailProperty::Type => part\n                            .content_type()\n                            .map(|ct| {\n                                ct.subtype()\n                                    .map(|st| format!(\"{}/{}\", ct.ctype(), st))\n                                    .unwrap_or_else(|| ct.ctype().to_string())\n                            })\n                            .or_else(|| match &part.body {\n                                ArchivedMetadataPartType::Text => Some(\"text/plain\".to_string()),\n                                ArchivedMetadataPartType::Html => Some(\"text/html\".to_string()),\n                                ArchivedMetadataPartType::Message(_) => {\n                                    Some(\"message/rfc822\".to_string())\n                                }\n                                _ => None,\n                            })\n                            .into(),\n                        EmailProperty::Charset => {\n                            part.content_type()\n                                .and_then(|ct| ct.attribute(\"charset\"))\n                                .or(match &part.body {\n                                    ArchivedMetadataPartType::Text\n                                    | ArchivedMetadataPartType::Html => Some(\"us-ascii\"),\n                                    _ => None,\n                                })\n                                .map(|v| v.to_string())\n                                .into()\n                        }\n                        EmailProperty::Disposition => part\n                            .content_disposition()\n                            .map(|cd| cd.ctype())\n                            .map(|v| v.to_string())\n                            .into(),\n                        EmailProperty::Cid => part.content_id().map(|v| v.to_string()).into(),\n                        EmailProperty::Language => match part.content_language() {\n                            ArchivedMetadataHeaderValue::Text(text) => {\n                                vec![text.to_string()].into()\n                            }\n                            ArchivedMetadataHeaderValue::TextList(list) => list\n                                .iter()\n                                .map(|text| text.to_string().into())\n                                .collect::<Vec<Value<'static, EmailProperty, EmailValue>>>()\n                                .into(),\n                            _ => Value::Null,\n                        },\n                        EmailProperty::Location => {\n                            part.content_location().map(|v| v.to_string()).into()\n                        }\n                        EmailProperty::Header(_) => part.header_to_value(property, raw_message),\n                        EmailProperty::Headers => part.headers_to_value(raw_message),\n                        EmailProperty::SubParts => continue,\n                        _ => Value::Null,\n                    };\n                    values.insert_unchecked(property.clone(), value);\n                }\n\n                subparts.push(values);\n\n                if let Some(multipart) = multipart {\n                    if parts_stack.len() == 10_000 {\n                        debug_assert!(false, \"Too much nesting in message metadata\");\n                        return Value::Null;\n                    }\n                    let multipart = multipart\n                        .iter()\n                        .map(|id| u16::from(id) as u32)\n                        .collect::<Vec<_>>();\n                    parts_stack.push((\n                        parts,\n                        std::mem::replace(&mut subparts, Vec::with_capacity(multipart.len())),\n                    ));\n                    parts = multipart.into_iter();\n                }\n            } else if let Some((prev_parts, mut prev_subparts)) = parts_stack.pop() {\n                prev_subparts\n                    .last_mut()\n                    .unwrap()\n                    .insert_unchecked(EmailProperty::SubParts, subparts);\n                parts = prev_parts;\n                subparts = prev_subparts;\n            } else {\n                return subparts.pop().map(Into::into).unwrap_or_default();\n            }\n        }\n    }\n}\n\npub(super) trait TruncateBody {\n    fn truncate(&self, max_len: usize) -> (bool, String);\n}\n\nimpl TruncateBody for PartType<'_> {\n    fn truncate(&self, max_len: usize) -> (bool, String) {\n        match self {\n            PartType::Text(text) => truncate_plain(text, max_len),\n            PartType::Html(html) => truncate_html(html, max_len),\n            PartType::Binary(bytes) | PartType::InlineBinary(bytes) => {\n                PartType::Text(String::from_utf8_lossy(bytes)).truncate(max_len)\n            }\n            _ => (false, \"\".into()),\n        }\n    }\n}\n\npub(crate) fn truncate_plain(text: &str, mut max_len: usize) -> (bool, String) {\n    if max_len != 0 && text.len() > max_len {\n        let add_dots = max_len > 6;\n        if add_dots {\n            max_len -= 3;\n        }\n        let mut result = String::with_capacity(max_len);\n        for ch in text.chars() {\n            if ch != '\\r' {\n                if ch.len_utf8() + result.len() > max_len {\n                    break;\n                }\n                result.push(ch);\n            }\n        }\n        if add_dots {\n            result.push_str(\"...\");\n        }\n        (true, result)\n    } else {\n        (false, text.replace('\\r', \"\"))\n    }\n}\n\npub(crate) fn truncate_html(html: &str, mut max_len: usize) -> (bool, String) {\n    if max_len != 0 && html.len() > max_len {\n        let add_dots = max_len > 6;\n        if add_dots {\n            max_len -= 3;\n        }\n\n        let mut result = String::with_capacity(max_len);\n        let mut in_tag = false;\n        let mut in_comment = false;\n        let mut last_tag_end_pos = 0;\n        let mut cr_count = 0;\n        for (pos, ch) in html.char_indices() {\n            let mut set_last_tag = 0;\n            match ch {\n                '<' if !in_tag => {\n                    in_tag = true;\n                    if let Some(\"!--\") = html.get(pos + 1..pos + 4) {\n                        in_comment = true;\n                    }\n                    set_last_tag = pos;\n                }\n                '>' if in_tag => {\n                    if in_comment {\n                        if let Some(\"--\") = html.get(pos - 2..pos) {\n                            in_comment = false;\n                            in_tag = false;\n                            set_last_tag = pos + 1;\n                        }\n                    } else {\n                        in_tag = false;\n                        set_last_tag = pos + 1;\n                    }\n                }\n                '\\r' => {\n                    cr_count += 1;\n                    continue;\n                }\n                _ => (),\n            }\n            if ch.len_utf8() + pos - cr_count > max_len {\n                result.push_str(\n                    &html[0..if (in_tag || set_last_tag > 0) && last_tag_end_pos > 0 {\n                        last_tag_end_pos\n                    } else {\n                        pos\n                    }]\n                        .replace('\\r', \"\"),\n                );\n                if add_dots {\n                    result.push_str(\"...\");\n                }\n                break;\n            } else if set_last_tag > 0 {\n                last_tag_end_pos = set_last_tag;\n            }\n        }\n        (true, result)\n    } else {\n        (false, html.replace('\\r', \"\"))\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/email/copy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    changes::state::JmapCacheState,\n    email::{PatchResult, handle_email_patch, ingested_into_object},\n};\nuse common::{Server, auth::AccessToken};\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess, mailbox::MailboxCacheAccess},\n    message::copy::{CopyMessageError, EmailCopy},\n};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::{\n        copy::{CopyRequest, CopyResponse},\n        set::SetRequest,\n    },\n    object::email::{Email, EmailProperty, EmailValue},\n    request::{\n        Call, IntoValid, MaybeInvalid, RequestMethod, SetRequestMethod,\n        method::{MethodFunction, MethodName, MethodObject},\n        reference::MaybeResultReference,\n    },\n};\nuse jmap_tools::{Key, Value};\nuse std::future::Future;\nuse trc::AddContext;\nuse types::acl::Acl;\nuse utils::map::vec_map::VecMap;\n\npub trait JmapEmailCopy: Sync + Send {\n    fn email_copy<'x>(\n        &self,\n        request: CopyRequest<'x, Email>,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<CopyResponse<Email>>> + Send;\n}\n\nimpl JmapEmailCopy for Server {\n    async fn email_copy<'x>(\n        &self,\n        request: CopyRequest<'x, Email>,\n        access_token: &AccessToken,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n        session: &HttpSessionData,\n    ) -> trc::Result<CopyResponse<Email>> {\n        let account_id = request.account_id.document_id();\n        let from_account_id = request.from_account_id.document_id();\n\n        if account_id == from_account_id {\n            return Err(trc::JmapEvent::InvalidArguments\n                .into_err()\n                .details(\"From accountId is equal to fromAccountId\"));\n        }\n        let cache = self.get_cached_messages(account_id).await?;\n        let old_state = cache.assert_state(false, &request.if_in_state)?;\n        let mut response = CopyResponse {\n            from_account_id: request.from_account_id,\n            account_id: request.account_id,\n            new_state: old_state.clone(),\n            old_state,\n            created: VecMap::with_capacity(request.create.len()),\n            not_created: VecMap::new(),\n        };\n\n        let from_cache = self\n            .get_cached_messages(from_account_id)\n            .await\n            .caused_by(trc::location!())?;\n        let from_message_ids = if access_token.is_member(from_account_id) {\n            from_cache.email_document_ids()\n        } else {\n            from_cache.shared_messages(access_token, Acl::ReadItems)\n        };\n\n        let can_add_mailbox_ids = if access_token.is_shared(account_id) {\n            cache.shared_mailboxes(access_token, Acl::AddItems).into()\n        } else {\n            None\n        };\n        let on_success_delete = request.on_success_destroy_original.unwrap_or(false);\n        let mut destroy_ids = Vec::new();\n\n        // Obtain quota\n        let resource_token = self.get_resource_token(access_token, account_id).await?;\n\n        'create: for (id, create) in request.create.into_valid() {\n            let from_message_id = id.document_id();\n            if !from_message_ids.contains(from_message_id) {\n                response.not_created.append(\n                    id,\n                    SetError::not_found().with_description(format!(\n                        \"Item {} not found in account {}.\",\n                        id, response.from_account_id\n                    )),\n                );\n                continue;\n            }\n\n            let mut mailboxes = Vec::new();\n            let mut keywords = Vec::new();\n            let mut received_at = None;\n\n            for (property, value) in create.into_expanded_object() {\n                match (property, value) {\n                    (Key::Property(EmailProperty::MailboxIds), Value::Object(ids)) => {\n                        mailboxes = ids\n                            .into_expanded_boolean_set()\n                            .filter_map(|id| {\n                                id.try_into_property()?.try_into_id()?.document_id().into()\n                            })\n                            .collect();\n                    }\n                    (Key::Property(EmailProperty::Keywords), Value::Object(keywords_)) => {\n                        keywords = keywords_\n                            .into_expanded_boolean_set()\n                            .filter_map(|id| id.try_into_property()?.try_into_keyword())\n                            .collect();\n                    }\n                    (Key::Property(EmailProperty::Pointer(pointer)), value) => {\n                        match handle_email_patch(&pointer, value) {\n                            PatchResult::SetKeyword(keyword) => {\n                                if !keywords.contains(keyword) {\n                                    keywords.push(keyword.clone());\n                                }\n                            }\n                            PatchResult::RemoveKeyword(keyword) => {\n                                keywords.retain(|k| k != keyword);\n                            }\n                            PatchResult::AddMailbox(id) => {\n                                if !mailboxes.contains(&id) {\n                                    mailboxes.push(id);\n                                }\n                            }\n                            PatchResult::RemoveMailbox(id) => {\n                                mailboxes.retain(|mid| mid != &id);\n                            }\n                            PatchResult::Invalid(set_error) => {\n                                response.not_created.append(id, set_error);\n                                continue 'create;\n                            }\n                        }\n                    }\n                    (\n                        Key::Property(EmailProperty::ReceivedAt),\n                        Value::Element(EmailValue::Date(value)),\n                    ) => {\n                        received_at = value.into();\n                    }\n                    (property, _) => {\n                        response.not_created.append(\n                            id,\n                            SetError::invalid_properties()\n                                .with_property(property.into_owned())\n                                .with_description(\"Invalid property or value.\".to_string()),\n                        );\n                        continue 'create;\n                    }\n                }\n            }\n\n            // Make sure message belongs to at least one mailbox\n            if mailboxes.is_empty() {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(EmailProperty::MailboxIds)\n                        .with_description(\"Message has to belong to at least one mailbox.\"),\n                );\n                continue 'create;\n            }\n\n            // Verify that the mailboxIds are valid\n            for mailbox_id in &mailboxes {\n                if !cache.has_mailbox_id(mailbox_id) {\n                    response.not_created.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(EmailProperty::MailboxIds)\n                            .with_description(format!(\"mailboxId {mailbox_id} does not exist.\")),\n                    );\n                    continue 'create;\n                } else if matches!(&can_add_mailbox_ids, Some(ids) if !ids.contains(*mailbox_id)) {\n                    response.not_created.append(\n                        id,\n                        SetError::forbidden().with_description(format!(\n                            \"You are not allowed to add messages to mailbox {mailbox_id}.\"\n                        )),\n                    );\n                    continue 'create;\n                }\n            }\n\n            // Add response\n            match self\n                .copy_message(\n                    from_account_id,\n                    from_message_id,\n                    &resource_token,\n                    mailboxes,\n                    keywords,\n                    received_at.map(|dt| dt.timestamp() as u64),\n                    session.session_id,\n                )\n                .await?\n            {\n                Ok(email) => {\n                    response\n                        .created\n                        .append(id, ingested_into_object(email).into());\n                }\n                Err(err) => {\n                    response.not_created.append(\n                        id,\n                        match err {\n                            CopyMessageError::NotFound => SetError::not_found()\n                                .with_description(\"Message not found in account.\"),\n                            CopyMessageError::OverQuota => SetError::over_quota(),\n                        },\n                    );\n                }\n            }\n\n            // Add to destroy list\n            if on_success_delete {\n                destroy_ids.push(MaybeInvalid::Value(id));\n            }\n        }\n\n        // Update state\n        if !response.created.is_empty() {\n            response.new_state = self.get_cached_messages(account_id).await?.get_state(false);\n        }\n\n        // Destroy ids\n        if on_success_delete && !destroy_ids.is_empty() {\n            *next_call = Call {\n                id: String::new(),\n                name: MethodName::new(MethodObject::Email, MethodFunction::Set),\n                method: RequestMethod::Set(SetRequestMethod::Email(SetRequest {\n                    account_id: request.from_account_id,\n                    if_in_state: request.destroy_from_if_in_state,\n                    create: None,\n                    update: None,\n                    destroy: MaybeResultReference::Value(destroy_ids).into(),\n                    arguments: Default::default(),\n                })),\n            }\n            .into();\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/email/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    body::{ToBodyPart, truncate_html, truncate_plain},\n    headers::IntoForm,\n};\nuse crate::{changes::state::JmapCacheState, email::headers::HeaderToValue};\nuse common::{Server, auth::AccessToken};\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    message::metadata::{\n        ArchivedMetadataPartType, MESSAGE_HAS_ATTACHMENT, MESSAGE_RECEIVED_MASK, MessageMetadata,\n        MetadataHeaderName, PART_ENCODING_PROBLEM,\n    },\n};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::email::{Email, EmailProperty, EmailValue, HeaderForm},\n    request::IntoValid,\n    types::date::UTCDate,\n};\nuse jmap_tools::{Key, Map, Value};\nuse mail_parser::HeaderValue;\nuse std::future::Future;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::{AddContext, StoreEvent};\nuse types::{\n    acl::Acl,\n    blob::{BlobClass, BlobId},\n    blob_hash::BlobHash,\n    collection::Collection,\n    field::EmailField,\n    id::Id,\n};\nuse utils::chained_bytes::ChainedBytes;\n\npub trait EmailGet: Sync + Send {\n    fn email_get(\n        &self,\n        request: GetRequest<Email>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<Email>>> + Send;\n}\n\nimpl EmailGet for Server {\n    async fn email_get(\n        &self,\n        mut request: GetRequest<Email>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<Email>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            EmailProperty::Id,\n            EmailProperty::BlobId,\n            EmailProperty::ThreadId,\n            EmailProperty::MailboxIds,\n            EmailProperty::Keywords,\n            EmailProperty::Size,\n            EmailProperty::ReceivedAt,\n            EmailProperty::MessageId,\n            EmailProperty::InReplyTo,\n            EmailProperty::References,\n            EmailProperty::Sender,\n            EmailProperty::From,\n            EmailProperty::To,\n            EmailProperty::Cc,\n            EmailProperty::Bcc,\n            EmailProperty::ReplyTo,\n            EmailProperty::Subject,\n            EmailProperty::SentAt,\n            EmailProperty::HasAttachment,\n            EmailProperty::Preview,\n            EmailProperty::BodyValues,\n            EmailProperty::TextBody,\n            EmailProperty::HtmlBody,\n            EmailProperty::Attachments,\n        ]);\n        let body_properties = request\n            .arguments\n            .body_properties\n            .map(|v| v.into_valid().collect())\n            .unwrap_or_else(|| {\n                vec![\n                    EmailProperty::PartId,\n                    EmailProperty::BlobId,\n                    EmailProperty::Size,\n                    EmailProperty::Name,\n                    EmailProperty::Type,\n                    EmailProperty::Charset,\n                    EmailProperty::Disposition,\n                    EmailProperty::Cid,\n                    EmailProperty::Language,\n                    EmailProperty::Location,\n                ]\n            });\n        let fetch_text_body_values = request.arguments.fetch_text_body_values.unwrap_or(false);\n        let fetch_html_body_values = request.arguments.fetch_html_body_values.unwrap_or(false);\n        let fetch_all_body_values = request.arguments.fetch_all_body_values.unwrap_or(false);\n        let max_body_value_bytes = request.arguments.max_body_value_bytes.unwrap_or(0);\n\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n        let message_ids = if access_token.is_member(account_id) {\n            cache.email_document_ids()\n        } else {\n            cache.shared_messages(access_token, Acl::ReadItems)\n        };\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            cache\n                .emails\n                .items\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(|item| Id::from_parts(item.thread_id, item.document_id))\n                .collect()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: cache.get_state(false).into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        // Check if we need to fetch the raw headers or body\n        let mut needs_body = false;\n        for property in &properties {\n            if matches!(\n                property,\n                EmailProperty::BodyValues\n                    | EmailProperty::TextBody\n                    | EmailProperty::HtmlBody\n                    | EmailProperty::Attachments\n                    | EmailProperty::BodyStructure\n            ) {\n                needs_body = true;\n                break;\n            }\n        }\n\n        for id in ids {\n            // Obtain the email object\n            if !message_ids.contains(id.document_id()) {\n                response.not_found.push(id);\n                continue;\n            }\n            let metadata_ = match self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                    account_id,\n                    Collection::Email,\n                    id.document_id(),\n                    EmailField::Metadata,\n                ))\n                .await?\n            {\n                Some(metadata) => metadata,\n                None => {\n                    response.not_found.push(id);\n                    continue;\n                }\n            };\n            let metadata = metadata_\n                .unarchive::<MessageMetadata>()\n                .caused_by(trc::location!())?;\n\n            // Obtain message data\n            let data = match cache.email_by_id(&id.document_id()) {\n                Some(data) => data,\n                None => {\n                    response.not_found.push(id);\n                    continue;\n                }\n            };\n\n            // Retrieve raw message if needed\n            let blob_hash = BlobHash::from(&metadata.blob_hash);\n            let raw_body;\n            let mut raw_message = ChainedBytes::new(metadata.raw_headers.as_ref());\n            if needs_body {\n                raw_body = self\n                    .blob_store()\n                    .get_blob(blob_hash.as_slice(), 0..usize::MAX)\n                    .await?;\n\n                if let Some(raw_body) = &raw_body {\n                    raw_message.append(\n                        raw_body\n                            .get(metadata.blob_body_offset.to_native() as usize..)\n                            .unwrap_or_default(),\n                    );\n                } else {\n                    trc::event!(\n                        Store(StoreEvent::NotFound),\n                        AccountId = account_id,\n                        DocumentId = id.document_id(),\n                        Collection = Collection::Email,\n                        BlobId = blob_hash.to_hex(),\n                        Details = \"Blob not found.\",\n                        CausedBy = trc::location!(),\n                    );\n\n                    response.not_found.push(id);\n                    continue;\n                }\n            }\n            let blob_id = BlobId {\n                hash: blob_hash,\n                class: BlobClass::Linked {\n                    account_id,\n                    collection: Collection::Email.into(),\n                    document_id: id.document_id(),\n                },\n                section: None,\n            };\n\n            // Prepare response\n            let mut email: Map<'_, EmailProperty, EmailValue> =\n                Map::with_capacity(properties.len());\n            let contents = &metadata.contents[0];\n            let root_part = &contents.parts[0];\n            let blob_body_offset = metadata.blob_body_offset.to_native() as isize\n                - root_part.offset_body.to_native() as isize;\n            for property in &properties {\n                match property {\n                    EmailProperty::Id => {\n                        email.insert_unchecked(EmailProperty::Id, Id::from(*id));\n                    }\n                    EmailProperty::ThreadId => {\n                        email.insert_unchecked(EmailProperty::ThreadId, Id::from(id.prefix_id()));\n                    }\n                    EmailProperty::BlobId => {\n                        email.insert_unchecked(EmailProperty::BlobId, blob_id.clone());\n                    }\n                    EmailProperty::MailboxIds => {\n                        let mut obj = Map::with_capacity(data.mailboxes.len());\n                        for id in data.mailboxes.iter() {\n                            debug_assert!(id.uid != 0);\n                            obj.insert_unchecked(\n                                EmailProperty::IdValue(Id::from(id.mailbox_id)),\n                                true,\n                            );\n                        }\n\n                        email.insert_unchecked(property.clone(), Value::Object(obj));\n                    }\n                    EmailProperty::Keywords => {\n                        let mut obj = Map::with_capacity(2);\n                        for keyword in cache.expand_keywords(data) {\n                            obj.insert_unchecked(EmailProperty::Keyword(keyword), true);\n                        }\n                        email.insert_unchecked(property.clone(), Value::Object(obj));\n                    }\n                    EmailProperty::Size => {\n                        email.insert_unchecked(EmailProperty::Size, data.size);\n                    }\n                    EmailProperty::ReceivedAt => {\n                        email.insert_unchecked(\n                            EmailProperty::ReceivedAt,\n                            EmailValue::Date(UTCDate::from_timestamp(\n                                (metadata.rcvd_attach.to_native() & MESSAGE_RECEIVED_MASK) as i64,\n                            )),\n                        );\n                    }\n                    EmailProperty::Preview => {\n                        if !metadata.preview.is_empty() {\n                            email.insert_unchecked(\n                                EmailProperty::Preview,\n                                metadata.preview.to_string(),\n                            );\n                        }\n                    }\n                    EmailProperty::HasAttachment => {\n                        email.insert_unchecked(\n                            EmailProperty::HasAttachment,\n                            (metadata.rcvd_attach.to_native() & MESSAGE_HAS_ATTACHMENT) != 0,\n                        );\n                    }\n                    EmailProperty::Subject => {\n                        email.insert_unchecked(\n                            EmailProperty::Subject,\n                            root_part\n                                .header_value(&MetadataHeaderName::Subject)\n                                .map(|value| HeaderValue::from(value).into_form(&HeaderForm::Text))\n                                .unwrap_or_default(),\n                        );\n                    }\n                    EmailProperty::SentAt => {\n                        email.insert_unchecked(\n                            EmailProperty::SentAt,\n                            root_part\n                                .header_value(&MetadataHeaderName::Date)\n                                .map(|value| HeaderValue::from(value).into_form(&HeaderForm::Date))\n                                .unwrap_or_default(),\n                        );\n                    }\n                    EmailProperty::MessageId\n                    | EmailProperty::InReplyTo\n                    | EmailProperty::References => {\n                        email.insert_unchecked(\n                            property.clone(),\n                            root_part\n                                .header_value(&match property {\n                                    EmailProperty::MessageId => MetadataHeaderName::MessageId,\n                                    EmailProperty::InReplyTo => MetadataHeaderName::InReplyTo,\n                                    EmailProperty::References => MetadataHeaderName::References,\n                                    _ => unreachable!(),\n                                })\n                                .map(|value| {\n                                    HeaderValue::from(value).into_form(&HeaderForm::MessageIds)\n                                })\n                                .unwrap_or_default(),\n                        );\n                    }\n\n                    EmailProperty::Sender\n                    | EmailProperty::From\n                    | EmailProperty::To\n                    | EmailProperty::Cc\n                    | EmailProperty::Bcc\n                    | EmailProperty::ReplyTo => {\n                        email.insert_unchecked(\n                            property.clone(),\n                            root_part\n                                .header_value(&match property {\n                                    EmailProperty::Sender => MetadataHeaderName::Sender,\n                                    EmailProperty::From => MetadataHeaderName::From,\n                                    EmailProperty::To => MetadataHeaderName::To,\n                                    EmailProperty::Cc => MetadataHeaderName::Cc,\n                                    EmailProperty::Bcc => MetadataHeaderName::Bcc,\n                                    EmailProperty::ReplyTo => MetadataHeaderName::ReplyTo,\n                                    _ => unreachable!(),\n                                })\n                                .map(|value| {\n                                    HeaderValue::from(value).into_form(&HeaderForm::Addresses)\n                                })\n                                .unwrap_or_default(),\n                        );\n                    }\n                    EmailProperty::Header(_) => {\n                        email.insert_unchecked(\n                            property.clone(),\n                            root_part.header_to_value(property, &raw_message),\n                        );\n                    }\n                    EmailProperty::Headers => {\n                        email.insert_unchecked(\n                            EmailProperty::Headers,\n                            root_part.headers_to_value(&raw_message),\n                        );\n                    }\n                    EmailProperty::TextBody\n                    | EmailProperty::HtmlBody\n                    | EmailProperty::Attachments => {\n                        let list = match property {\n                            EmailProperty::TextBody => &contents.text_body,\n                            EmailProperty::HtmlBody => &contents.html_body,\n                            EmailProperty::Attachments => &contents.attachments,\n                            _ => unreachable!(),\n                        }\n                        .iter();\n                        email.insert_unchecked(\n                            property.clone(),\n                            list.map(|part_id| {\n                                contents.to_body_part(\n                                    u16::from(part_id) as u32,\n                                    &body_properties,\n                                    &raw_message,\n                                    &blob_id,\n                                    blob_body_offset,\n                                )\n                            })\n                            .collect::<Vec<_>>(),\n                        );\n                    }\n                    EmailProperty::BodyStructure => {\n                        email.insert_unchecked(\n                            EmailProperty::BodyStructure,\n                            contents.to_body_part(\n                                0,\n                                &body_properties,\n                                &raw_message,\n                                &blob_id,\n                                blob_body_offset,\n                            ),\n                        );\n                    }\n                    EmailProperty::BodyValues => {\n                        let mut body_values = Map::with_capacity(contents.parts.len());\n                        for (part_id, part) in contents.parts.iter().enumerate() {\n                            if ((contents.is_html_part(part_id as u16)\n                                && (fetch_all_body_values || fetch_html_body_values))\n                                || (contents.is_text_part(part_id as u16)\n                                    && (fetch_all_body_values || fetch_text_body_values)))\n                                && matches!(\n                                    part.body,\n                                    ArchivedMetadataPartType::Text | ArchivedMetadataPartType::Html\n                                )\n                            {\n                                let contents = part.decode_contents(&raw_message);\n\n                                let (is_truncated, value) = match &part.body {\n                                    ArchivedMetadataPartType::Text => {\n                                        truncate_plain(contents.as_str(), max_body_value_bytes)\n                                    }\n                                    ArchivedMetadataPartType::Html => {\n                                        truncate_html(contents.as_str(), max_body_value_bytes)\n                                    }\n                                    _ => unreachable!(),\n                                };\n\n                                body_values.insert_unchecked(\n                                    Key::Owned(part_id.to_string()),\n                                    Map::with_capacity(3)\n                                        .with_key_value(\n                                            EmailProperty::IsEncodingProblem,\n                                            (part.flags & PART_ENCODING_PROBLEM) != 0,\n                                        )\n                                        .with_key_value(EmailProperty::IsTruncated, is_truncated)\n                                        .with_key_value(EmailProperty::Value, value),\n                                );\n                            }\n                        }\n                        email.insert_unchecked(EmailProperty::BodyValues, body_values);\n                    }\n\n                    _ => {\n                        return Err(trc::JmapEvent::InvalidArguments\n                            .into_err()\n                            .details(format!(\"Invalid property {property:?}\")));\n                    }\n                }\n            }\n            response.list.push(email.into());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/email/headers.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse email::message::metadata::{ArchivedMessageMetadataPart, ArchivedMetadataHeaderValue};\nuse jmap_proto::{\n    object::email::{EmailProperty, EmailValue, HeaderForm, HeaderProperty},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Key, Map, Value};\nuse mail_builder::{\n    MessageBuilder,\n    headers::{\n        address::{Address, EmailAddress, GroupedAddresses},\n        date::Date,\n        message_id::MessageId,\n        raw::Raw,\n        text::Text,\n        url::URL,\n    },\n};\nuse mail_parser::{Addr, DateTime, Group, Header, HeaderName, HeaderValue, parsers::MessageStream};\nuse utils::chained_bytes::ChainedBytes;\n\npub trait IntoForm {\n    fn into_form(self, form: &HeaderForm) -> Value<'static, EmailProperty, EmailValue>;\n}\n\npub trait HeaderToValue {\n    fn header_to_value(\n        &self,\n        property: &EmailProperty,\n        raw_message: &ChainedBytes<'_>,\n    ) -> Value<'static, EmailProperty, EmailValue>;\n    fn headers_to_value(\n        &self,\n        raw_message: &ChainedBytes<'_>,\n    ) -> Value<'static, EmailProperty, EmailValue>;\n}\n\npub trait ValueToHeader<'x> {\n    fn try_into_grouped_addresses(self) -> Option<GroupedAddresses<'x>>;\n    fn try_into_address_list(self) -> Option<Vec<Address<'x>>>;\n    fn try_into_address(self) -> Option<EmailAddress<'x>>;\n}\n\npub trait BuildHeader<'x>: Sized {\n    fn build_header(\n        self,\n        header: HeaderProperty,\n        value: Value<'x, EmailProperty, EmailValue>,\n    ) -> Result<Self, HeaderProperty>;\n}\n\nimpl HeaderToValue for Vec<Header<'_>> {\n    fn header_to_value(\n        &self,\n        property: &EmailProperty,\n        raw_message: &ChainedBytes<'_>,\n    ) -> Value<'static, EmailProperty, EmailValue> {\n        let (header_name, form, all) = match property {\n            EmailProperty::Header(header) => (\n                HeaderName::parse(header.header.as_str())\n                    .unwrap_or_else(|| HeaderName::Other(header.header.as_str().into())),\n                header.form,\n                header.all,\n            ),\n            EmailProperty::Sender => (HeaderName::Sender, HeaderForm::Addresses, false),\n            EmailProperty::From => (HeaderName::From, HeaderForm::Addresses, false),\n            EmailProperty::To => (HeaderName::To, HeaderForm::Addresses, false),\n            EmailProperty::Cc => (HeaderName::Cc, HeaderForm::Addresses, false),\n            EmailProperty::Bcc => (HeaderName::Bcc, HeaderForm::Addresses, false),\n            EmailProperty::ReplyTo => (HeaderName::ReplyTo, HeaderForm::Addresses, false),\n            EmailProperty::Subject => (HeaderName::Subject, HeaderForm::Text, false),\n            EmailProperty::MessageId => (HeaderName::MessageId, HeaderForm::MessageIds, false),\n            EmailProperty::InReplyTo => (HeaderName::InReplyTo, HeaderForm::MessageIds, false),\n            EmailProperty::References => (HeaderName::References, HeaderForm::MessageIds, false),\n            EmailProperty::SentAt => (HeaderName::Date, HeaderForm::Date, false),\n            _ => return Value::Null,\n        };\n\n        let is_raw = matches!(form, HeaderForm::Raw) || matches!(header_name, HeaderName::Other(_));\n        let mut headers = Vec::new();\n        let header_name = header_name.as_str();\n        for header in self.iter().rev() {\n            if header.name.as_str().eq_ignore_ascii_case(header_name) {\n                let raw_header;\n                let header_value = if is_raw || matches!(header.value, HeaderValue::Empty) {\n                    raw_header =\n                        raw_message.get(header.offset_start as usize..header.offset_end as usize);\n\n                    if let Some(bytes) = &raw_header {\n                        let bytes = bytes.as_ref();\n                        match form {\n                            HeaderForm::Raw => {\n                                HeaderValue::Text(String::from_utf8_lossy(bytes.trim_end()))\n                            }\n                            HeaderForm::Text => MessageStream::new(bytes).parse_unstructured(),\n                            HeaderForm::Addresses\n                            | HeaderForm::GroupedAddresses\n                            | HeaderForm::URLs => MessageStream::new(bytes).parse_address(),\n                            HeaderForm::MessageIds => MessageStream::new(bytes).parse_id(),\n                            HeaderForm::Date => MessageStream::new(bytes).parse_date(),\n                        }\n                    } else {\n                        HeaderValue::Empty\n                    }\n                } else {\n                    header.value.clone()\n                };\n                headers.push(header_value.into_form(&form));\n                if !all {\n                    break;\n                }\n            }\n        }\n\n        if !all {\n            headers.pop().unwrap_or_default()\n        } else {\n            if headers.len() > 1 {\n                headers.reverse();\n            }\n            Value::Array(headers)\n        }\n    }\n\n    fn headers_to_value(\n        &self,\n        raw_message: &ChainedBytes<'_>,\n    ) -> Value<'static, EmailProperty, EmailValue> {\n        let mut headers = Vec::with_capacity(self.len());\n        for header in self.iter() {\n            headers.push(Value::Object(\n                Map::with_capacity(2)\n                    .with_key_value(EmailProperty::Name, header.name().to_string())\n                    .with_key_value(\n                        EmailProperty::Value,\n                        String::from_utf8_lossy(\n                            raw_message\n                                .get(header.offset_start as usize..header.offset_end as usize)\n                                .unwrap_or_default()\n                                .as_ref()\n                                .trim_end(),\n                        )\n                        .into_owned(),\n                    ),\n            ));\n        }\n        headers.into()\n    }\n}\n\nimpl IntoForm for HeaderValue<'_> {\n    fn into_form(self, form: &HeaderForm) -> Value<'static, EmailProperty, EmailValue> {\n        match (self, form) {\n            (HeaderValue::Text(text), HeaderForm::Raw | HeaderForm::Text) => {\n                text.into_owned().into()\n            }\n            (HeaderValue::TextList(texts), HeaderForm::Raw | HeaderForm::Text) => {\n                texts.join(\", \").into()\n            }\n            (HeaderValue::Text(text), HeaderForm::MessageIds) => {\n                Value::Array(vec![text.into_owned().into()])\n            }\n            (HeaderValue::TextList(texts), HeaderForm::MessageIds) => {\n                Value::Array(texts.into_iter().map(|t| t.into_owned().into()).collect())\n            }\n            (HeaderValue::DateTime(datetime), HeaderForm::Date) => from_mail_datetime(datetime),\n            (HeaderValue::Address(mail_parser::Address::List(addrlist)), HeaderForm::URLs) => {\n                Value::Array(\n                    addrlist\n                        .into_iter()\n                        .filter_map(|addr| match addr {\n                            Addr {\n                                address: Some(addr),\n                                ..\n                            } if addr.contains(':') => Some(addr.into_owned().into()),\n                            _ => None,\n                        })\n                        .collect(),\n                )\n            }\n            (HeaderValue::Address(mail_parser::Address::List(addrlist)), HeaderForm::Addresses) => {\n                from_mail_addrlist(addrlist)\n            }\n            (\n                HeaderValue::Address(mail_parser::Address::Group(grouplist)),\n                HeaderForm::Addresses,\n            ) => Value::Array(\n                grouplist\n                    .into_iter()\n                    .flat_map(|group| group.addresses.into_iter().map(from_mail_addr))\n                    .collect(),\n            ),\n            (\n                HeaderValue::Address(mail_parser::Address::List(addrlist)),\n                HeaderForm::GroupedAddresses,\n            ) => Value::Array(vec![\n                Map::with_capacity(2)\n                    .with_key_value(EmailProperty::Name, Value::Null)\n                    .with_key_value(EmailProperty::Addresses, from_mail_addrlist(addrlist))\n                    .into(),\n            ]),\n            (\n                HeaderValue::Address(mail_parser::Address::Group(grouplist)),\n                HeaderForm::GroupedAddresses,\n            ) => Value::Array(\n                grouplist\n                    .into_iter()\n                    .map(from_mail_group)\n                    .collect::<Vec<Value<'static, EmailProperty, EmailValue>>>(),\n            ),\n\n            _ => Value::Null,\n        }\n    }\n}\n\nimpl<'x> ValueToHeader<'x> for Value<'x, EmailProperty, EmailValue> {\n    fn try_into_grouped_addresses(self) -> Option<GroupedAddresses<'x>> {\n        let mut obj = self.into_object()?;\n        Some(GroupedAddresses {\n            name: obj\n                .remove(&Key::Property(EmailProperty::Name))\n                .and_then(|n| n.into_string()),\n            addresses: obj\n                .remove(&Key::Property(EmailProperty::Addresses))?\n                .try_into_address_list()?,\n        })\n    }\n\n    fn try_into_address_list(self) -> Option<Vec<Address<'x>>> {\n        let list = self.into_array()?;\n        let mut addresses = Vec::with_capacity(list.len());\n        for value in list {\n            addresses.push(Address::Address(value.try_into_address()?));\n        }\n        Some(addresses)\n    }\n\n    fn try_into_address(self) -> Option<EmailAddress<'x>> {\n        let mut obj = self.into_object()?;\n        Some(EmailAddress {\n            name: obj\n                .remove(&Key::Property(EmailProperty::Name))\n                .and_then(|n| n.into_string()),\n            email: obj\n                .remove(&Key::Property(EmailProperty::Email))?\n                .into_string()?,\n        })\n    }\n}\n\nimpl<'x> BuildHeader<'x> for MessageBuilder<'x> {\n    fn build_header(\n        self,\n        header: HeaderProperty,\n        value: Value<'x, EmailProperty, EmailValue>,\n    ) -> Result<Self, HeaderProperty> {\n        Ok(match (&header.form, header.all, value) {\n            (HeaderForm::Raw, false, Value::Str(value)) => {\n                self.header(header.header, Raw::from(value))\n            }\n            (HeaderForm::Raw, true, Value::Array(value)) => self.headers(\n                header.header,\n                value\n                    .into_iter()\n                    .filter_map(|v| Raw::from(v.into_string()?).into()),\n            ),\n            (HeaderForm::Date, false, Value::Element(EmailValue::Date(value))) => {\n                self.header(header.header, Date::new(value.timestamp()))\n            }\n            (HeaderForm::Date, true, Value::Array(value)) => self.headers(\n                header.header,\n                value\n                    .into_iter()\n                    .filter_map(|v| Date::new(unwrap_date(v)?.timestamp()).into()),\n            ),\n            (HeaderForm::Text, false, Value::Str(value)) => {\n                self.header(header.header, Text::from(value))\n            }\n            (HeaderForm::Text, true, Value::Array(value)) => self.headers(\n                header.header,\n                value\n                    .into_iter()\n                    .filter_map(|v| Text::from(v.into_string()?).into()),\n            ),\n            (HeaderForm::URLs, false, Value::Array(value)) => self.header(\n                header.header,\n                URL {\n                    url: value\n                        .into_iter()\n                        .filter_map(|v| v.into_string()?.into())\n                        .collect(),\n                },\n            ),\n            (HeaderForm::URLs, true, Value::Array(value)) => self.headers(\n                header.header,\n                value.into_iter().filter_map(|value| {\n                    URL {\n                        url: value\n                            .into_array()?\n                            .into_iter()\n                            .filter_map(|v| v.into_string()?.into())\n                            .collect(),\n                    }\n                    .into()\n                }),\n            ),\n            (HeaderForm::MessageIds, false, Value::Array(value)) => self.header(\n                header.header,\n                MessageId {\n                    id: value\n                        .into_iter()\n                        .filter_map(|v| v.into_string()?.into())\n                        .collect(),\n                },\n            ),\n            (HeaderForm::MessageIds, true, Value::Array(value)) => self.headers(\n                header.header,\n                value.into_iter().filter_map(|value| {\n                    MessageId {\n                        id: value\n                            .into_array()?\n                            .into_iter()\n                            .filter_map(|v| v.into_string()?.into())\n                            .collect(),\n                    }\n                    .into()\n                }),\n            ),\n            (HeaderForm::Addresses, false, Value::Array(value)) => self.header(\n                header.header,\n                Address::new_list(\n                    value\n                        .into_iter()\n                        .filter_map(|v| Address::Address(v.try_into_address()?).into())\n                        .collect(),\n                ),\n            ),\n            (HeaderForm::Addresses, true, Value::Array(value)) => self.headers(\n                header.header,\n                value\n                    .into_iter()\n                    .filter_map(|v| Address::new_list(v.try_into_address_list()?).into()),\n            ),\n            (HeaderForm::GroupedAddresses, false, Value::Array(value)) => self.header(\n                header.header,\n                Address::new_list(\n                    value\n                        .into_iter()\n                        .filter_map(|v| Address::Group(v.try_into_grouped_addresses()?).into())\n                        .collect(),\n                ),\n            ),\n            (HeaderForm::GroupedAddresses, true, Value::Array(value)) => self.headers(\n                header.header,\n                value.into_iter().filter_map(|v| {\n                    Address::new_list(\n                        v.into_array()?\n                            .into_iter()\n                            .filter_map(|v| Address::Group(v.try_into_grouped_addresses()?).into())\n                            .collect::<Vec<_>>(),\n                    )\n                    .into()\n                }),\n            ),\n            _ => {\n                return Err(header);\n            }\n        })\n    }\n}\n\nimpl HeaderToValue for ArchivedMessageMetadataPart {\n    fn header_to_value(\n        &self,\n        property: &EmailProperty,\n        raw_message: &ChainedBytes<'_>,\n    ) -> Value<'static, EmailProperty, EmailValue> {\n        let (header_name, form, all) = match property {\n            EmailProperty::Header(header) => (\n                HeaderName::parse(header.header.as_str())\n                    .unwrap_or_else(|| HeaderName::Other(header.header.as_str().into())),\n                header.form,\n                header.all,\n            ),\n            EmailProperty::Sender => (HeaderName::Sender, HeaderForm::Addresses, false),\n            EmailProperty::From => (HeaderName::From, HeaderForm::Addresses, false),\n            EmailProperty::To => (HeaderName::To, HeaderForm::Addresses, false),\n            EmailProperty::Cc => (HeaderName::Cc, HeaderForm::Addresses, false),\n            EmailProperty::Bcc => (HeaderName::Bcc, HeaderForm::Addresses, false),\n            EmailProperty::ReplyTo => (HeaderName::ReplyTo, HeaderForm::Addresses, false),\n            EmailProperty::Subject => (HeaderName::Subject, HeaderForm::Text, false),\n            EmailProperty::MessageId => (HeaderName::MessageId, HeaderForm::MessageIds, false),\n            EmailProperty::InReplyTo => (HeaderName::InReplyTo, HeaderForm::MessageIds, false),\n            EmailProperty::References => (HeaderName::References, HeaderForm::MessageIds, false),\n            EmailProperty::SentAt => (HeaderName::Date, HeaderForm::Date, false),\n            _ => return Value::Null,\n        };\n\n        let is_raw = matches!(form, HeaderForm::Raw) || matches!(header_name, HeaderName::Other(_));\n        let mut headers = Vec::new();\n        let header_name = header_name.as_str();\n        for header in self.headers.iter().rev() {\n            if header.name.as_str().eq_ignore_ascii_case(header_name) {\n                let raw_header;\n                let header_value =\n                    if is_raw || matches!(header.value, ArchivedMetadataHeaderValue::Empty) {\n                        raw_header = raw_message.get(header.value_range());\n\n                        if let Some(bytes) = &raw_header {\n                            let bytes = bytes.as_ref();\n                            match form {\n                                HeaderForm::Raw => {\n                                    HeaderValue::Text(String::from_utf8_lossy(bytes.trim_end()))\n                                }\n                                HeaderForm::Text => MessageStream::new(bytes).parse_unstructured(),\n                                HeaderForm::Addresses\n                                | HeaderForm::GroupedAddresses\n                                | HeaderForm::URLs => MessageStream::new(bytes).parse_address(),\n                                HeaderForm::MessageIds => MessageStream::new(bytes).parse_id(),\n                                HeaderForm::Date => MessageStream::new(bytes).parse_date(),\n                            }\n                        } else {\n                            HeaderValue::Empty\n                        }\n                    } else {\n                        HeaderValue::from(&header.value)\n                    };\n                headers.push(header_value.into_form(&form));\n                if !all {\n                    break;\n                }\n            }\n        }\n\n        if !all {\n            headers.pop().unwrap_or_default()\n        } else {\n            if headers.len() > 1 {\n                headers.reverse();\n            }\n            Value::Array(headers)\n        }\n    }\n\n    fn headers_to_value(\n        &self,\n        raw_message: &ChainedBytes<'_>,\n    ) -> Value<'static, EmailProperty, EmailValue> {\n        let mut headers = Vec::with_capacity(self.headers.len());\n        for header in self.headers.iter() {\n            headers.push(Value::Object(\n                Map::with_capacity(2)\n                    .with_key_value(EmailProperty::Name, header.name.as_str().to_string())\n                    .with_key_value(\n                        EmailProperty::Value,\n                        String::from_utf8_lossy(\n                            raw_message\n                                .get(header.value_range())\n                                .unwrap_or_default()\n                                .as_ref()\n                                .trim_end(),\n                        )\n                        .into_owned(),\n                    ),\n            ));\n        }\n        headers.into()\n    }\n}\n\ntrait ByteTrim {\n    fn trim_end(&self) -> Self;\n}\n\nimpl ByteTrim for &[u8] {\n    fn trim_end(&self) -> Self {\n        let mut end = self.len();\n        while end > 0 && self[end - 1].is_ascii_whitespace() {\n            end -= 1;\n        }\n        &self[..end]\n    }\n}\n\n#[inline]\npub(crate) fn unwrap_date(value: Value<'_, EmailProperty, EmailValue>) -> Option<UTCDate> {\n    match value {\n        Value::Element(EmailValue::Date(date)) => Some(date),\n        _ => None,\n    }\n}\n\nfn from_mail_datetime(date: DateTime) -> Value<'static, EmailProperty, EmailValue> {\n    Value::Element(EmailValue::Date(UTCDate {\n        year: date.year,\n        month: date.month,\n        day: date.day,\n        hour: date.hour,\n        minute: date.minute,\n        second: date.second,\n        tz_before_gmt: date.tz_before_gmt,\n        tz_hour: date.tz_hour,\n        tz_minute: date.tz_minute,\n    }))\n}\n\nfn from_mail_addr(value: Addr<'_>) -> Value<'static, EmailProperty, EmailValue> {\n    Value::Object(\n        Map::with_capacity(2)\n            .with_key_value(EmailProperty::Name, value.name.map(|v| v.into_owned()))\n            .with_key_value(\n                EmailProperty::Email,\n                value.address.unwrap_or_default().into_owned(),\n            ),\n    )\n}\n\nfn from_mail_group(group: Group<'_>) -> Value<'static, EmailProperty, EmailValue> {\n    Value::Object(\n        Map::with_capacity(2)\n            .with_key_value(EmailProperty::Name, group.name.map(|v| v.into_owned()))\n            .with_key_value(\n                EmailProperty::Addresses,\n                from_mail_addrlist(group.addresses),\n            ),\n    )\n}\n\nfn from_mail_addrlist(addrlist: Vec<Addr<'_>>) -> Value<'static, EmailProperty, EmailValue> {\n    Value::Array(\n        addrlist\n            .into_iter()\n            .map(from_mail_addr)\n            .collect::<Vec<Value<'static, EmailProperty, EmailValue>>>(),\n    )\n}\n"
  },
  {
    "path": "crates/jmap/src/email/import.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    blob::download::BlobDownload, changes::state::JmapCacheState, email::ingested_into_object,\n};\nuse common::{Server, auth::AccessToken};\nuse email::{\n    cache::{MessageCacheFetch, mailbox::MailboxCacheAccess},\n    mailbox::JUNK_ID,\n    message::ingest::{EmailIngest, IngestEmail, IngestSource},\n};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::import::{ImportEmailRequest, ImportEmailResponse},\n    object::email::EmailProperty,\n    request::MaybeInvalid,\n    types::state::State,\n};\nuse mail_parser::MessageParser;\nuse std::future::Future;\nuse types::{acl::Acl, id::Id, keyword::Keyword};\nuse utils::map::vec_map::VecMap;\n\npub trait EmailImport: Sync + Send {\n    fn email_import(\n        &self,\n        request: ImportEmailRequest,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<ImportEmailResponse>> + Send;\n}\n\nimpl EmailImport for Server {\n    async fn email_import(\n        &self,\n        request: ImportEmailRequest,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> trc::Result<ImportEmailResponse> {\n        // Validate state\n        let account_id = request.account_id.document_id();\n        let cache = self.get_cached_messages(account_id).await?;\n        let old_state: State = cache.assert_state(false, &request.if_in_state)?;\n        let can_add_mailbox_ids = if access_token.is_shared(account_id) {\n            cache.shared_mailboxes(access_token, Acl::AddItems).into()\n        } else {\n            None\n        };\n\n        // Obtain import access token\n        let import_access_token = if account_id != access_token.primary_id() {\n            #[cfg(feature = \"test_mode\")]\n            {\n                std::sync::Arc::new(AccessToken::from_id(account_id)).into()\n            }\n\n            #[cfg(not(feature = \"test_mode\"))]\n            {\n                use trc::AddContext;\n                self.get_access_token(account_id)\n                    .await\n                    .caused_by(trc::location!())?\n                    .into()\n            }\n        } else {\n            None\n        };\n\n        let mut response = ImportEmailResponse {\n            account_id: request.account_id,\n            new_state: old_state.clone(),\n            old_state: old_state.into(),\n            created: VecMap::with_capacity(request.emails.len()),\n            not_created: VecMap::new(),\n        };\n\n        'outer: for (id, email) in request.emails {\n            // Validate mailboxIds\n            let mailbox_ids = email\n                .mailbox_ids\n                .unwrap()\n                .into_iter()\n                .filter_map(|m| m.try_unwrap().map(|m| m.document_id()))\n                .collect::<Vec<_>>();\n            if mailbox_ids.is_empty() {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(EmailProperty::MailboxIds)\n                        .with_description(\"Message must belong to at least one mailbox.\"),\n                );\n                continue;\n            }\n            for mailbox_id in &mailbox_ids {\n                if !cache.has_mailbox_id(mailbox_id) {\n                    response.not_created.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(EmailProperty::MailboxIds)\n                            .with_description(format!(\n                                \"Mailbox {} does not exist.\",\n                                Id::from(*mailbox_id)\n                            )),\n                    );\n                    continue 'outer;\n                } else if matches!(&can_add_mailbox_ids, Some(ids) if !ids.contains(*mailbox_id)) {\n                    response.not_created.append(\n                        id,\n                        SetError::forbidden().with_description(format!(\n                            \"You are not allowed to add messages to mailbox {}.\",\n                            Id::from(*mailbox_id)\n                        )),\n                    );\n                    continue 'outer;\n                }\n            }\n\n            let MaybeInvalid::Value(blob_id) = email.blob_id else {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(EmailProperty::BlobId)\n                        .with_description(\"Invalid blob id.\"),\n                );\n                continue;\n            };\n\n            // Fetch raw message to import\n            let raw_message = match self.blob_download(&blob_id, access_token).await? {\n                Some(raw_message) => raw_message,\n                None => {\n                    response.not_created.append(\n                        id,\n                        SetError::new(SetErrorType::BlobNotFound)\n                            .with_description(format!(\"BlobId {} not found.\", blob_id)),\n                    );\n                    continue;\n                }\n            };\n\n            // Import message\n            match self\n                .email_ingest(IngestEmail {\n                    raw_message: &raw_message,\n                    message: MessageParser::new().parse(&raw_message),\n                    blob_hash: Some(&blob_id.hash),\n                    access_token: import_access_token.as_deref().unwrap_or(access_token),\n                    source: IngestSource::Jmap {\n                        train_classifier: email\n                            .keywords\n                            .iter()\n                            .any(|k| matches!(k, Keyword::Junk | Keyword::NotJunk))\n                            || mailbox_ids.contains(&JUNK_ID),\n                    },\n                    mailbox_ids,\n                    keywords: email.keywords,\n                    received_at: email.received_at.map(|r| r.into()),\n                    session_id: session.session_id,\n                })\n                .await\n            {\n                Ok(email) => {\n                    response\n                        .created\n                        .append(id, ingested_into_object(email).into());\n                }\n                Err(mut err) => match err.as_ref() {\n                    trc::EventType::Limit(trc::LimitEvent::Quota) => {\n                        response.not_created.append(\n                            id,\n                            SetError::new(SetErrorType::OverQuota)\n                                .with_description(\"You have exceeded your disk quota.\"),\n                        );\n                    }\n                    trc::EventType::MessageIngest(trc::MessageIngestEvent::Error) => {\n                        response.not_created.append(\n                            id,\n                            SetError::new(SetErrorType::InvalidEmail).with_description(\n                                err.take_value(trc::Key::Reason)\n                                    .and_then(|v| v.into_string())\n                                    .unwrap(),\n                            ),\n                        );\n                    }\n                    _ => {\n                        return Err(err);\n                    }\n                },\n            }\n        }\n\n        // Update state\n        if !response.created.is_empty() {\n            response.new_state = self.get_cached_messages(account_id).await?.get_state(false);\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/email/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse email::message::ingest::IngestedEmail;\nuse jmap_proto::{\n    error::set::SetError,\n    object::email::{EmailProperty, EmailValue},\n};\nuse jmap_tools::{JsonPointer, JsonPointerItem, Key, Map, Value};\nuse types::{id::Id, keyword::Keyword};\n\npub mod body;\npub mod copy;\npub mod get;\npub mod headers;\npub mod import;\npub mod parse;\npub mod query;\npub mod set;\npub mod snippet;\n\nfn ingested_into_object(email: IngestedEmail) -> Map<'static, EmailProperty, EmailValue> {\n    Map::with_capacity(3)\n        .with_key_value(\n            EmailProperty::Id,\n            Id::from_parts(email.thread_id, email.document_id),\n        )\n        .with_key_value(EmailProperty::ThreadId, Id::from(email.thread_id))\n        .with_key_value(EmailProperty::BlobId, email.blob_id)\n        .with_key_value(EmailProperty::Size, email.size)\n}\n\npub(crate) enum PatchResult<'x> {\n    SetKeyword(&'x Keyword),\n    RemoveKeyword(&'x Keyword),\n    AddMailbox(u32),\n    RemoveMailbox(u32),\n    Invalid(SetError<EmailProperty>),\n}\n\npub(crate) fn handle_email_patch<'x>(\n    pointer: &'x JsonPointer<EmailProperty>,\n    value: Value<'_, EmailProperty, EmailValue>,\n) -> PatchResult<'x> {\n    let mut pointer_iter = pointer.iter();\n\n    match (pointer_iter.next(), pointer_iter.next()) {\n        (\n            Some(JsonPointerItem::Key(Key::Property(EmailProperty::Keywords))),\n            Some(JsonPointerItem::Key(Key::Property(EmailProperty::Keyword(keyword)))),\n        ) => match value {\n            Value::Bool(true) => return PatchResult::SetKeyword(keyword),\n            Value::Bool(false) | Value::Null => return PatchResult::RemoveKeyword(keyword),\n            _ => (),\n        },\n        (\n            Some(JsonPointerItem::Key(Key::Property(EmailProperty::MailboxIds))),\n            Some(JsonPointerItem::Key(Key::Property(EmailProperty::IdValue(id)))),\n        ) => match value {\n            Value::Bool(true) => return PatchResult::AddMailbox(id.document_id()),\n            Value::Bool(false) | Value::Null => {\n                return PatchResult::RemoveMailbox(id.document_id());\n            }\n            _ => (),\n        },\n        _ => (),\n    }\n\n    PatchResult::Invalid(\n        SetError::invalid_properties()\n            .with_property(EmailProperty::Pointer(pointer.clone()))\n            .with_description(\"Invalid patch value\".to_string()),\n    )\n}\n"
  },
  {
    "path": "crates/jmap/src/email/parse.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    body::{ToBodyPart, TruncateBody},\n    headers::HeaderToValue,\n};\nuse crate::blob::download::BlobDownload;\nuse common::{Server, auth::AccessToken};\nuse email::message::index::PREVIEW_LENGTH;\nuse jmap_proto::{\n    method::parse::{ParseRequest, ParseResponse},\n    object::email::{Email, EmailProperty},\n    request::IntoValid,\n};\nuse jmap_tools::{Key, Map, Value};\nuse mail_parser::{\n    MessageParser, PartType, decoders::html::html_to_text, parsers::preview::preview_text,\n};\nuse std::future::Future;\nuse utils::{chained_bytes::ChainedBytes, map::vec_map::VecMap};\n\npub trait EmailParse: Sync + Send {\n    fn email_parse(\n        &self,\n        request: ParseRequest<Email>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<ParseResponse<Email>>> + Send;\n}\n\nimpl EmailParse for Server {\n    async fn email_parse(\n        &self,\n        request: ParseRequest<Email>,\n        access_token: &AccessToken,\n    ) -> trc::Result<ParseResponse<Email>> {\n        if request.blob_ids.len() > self.core.jmap.mail_parse_max_items {\n            return Err(trc::JmapEvent::RequestTooLarge.into_err());\n        }\n        let properties = request\n            .properties\n            .map(|v| v.into_valid().collect())\n            .unwrap_or_else(|| {\n                vec![\n                    EmailProperty::BlobId,\n                    EmailProperty::Size,\n                    EmailProperty::ReceivedAt,\n                    EmailProperty::MessageId,\n                    EmailProperty::InReplyTo,\n                    EmailProperty::References,\n                    EmailProperty::Sender,\n                    EmailProperty::From,\n                    EmailProperty::To,\n                    EmailProperty::Cc,\n                    EmailProperty::Bcc,\n                    EmailProperty::ReplyTo,\n                    EmailProperty::Subject,\n                    EmailProperty::SentAt,\n                    EmailProperty::HasAttachment,\n                    EmailProperty::Preview,\n                    EmailProperty::BodyValues,\n                    EmailProperty::TextBody,\n                    EmailProperty::HtmlBody,\n                    EmailProperty::Attachments,\n                ]\n            });\n        let body_properties = request\n            .arguments\n            .body_properties\n            .map(|v| v.into_valid().collect())\n            .unwrap_or_else(|| {\n                vec![\n                    EmailProperty::PartId,\n                    EmailProperty::BlobId,\n                    EmailProperty::Size,\n                    EmailProperty::Name,\n                    EmailProperty::Type,\n                    EmailProperty::Charset,\n                    EmailProperty::Disposition,\n                    EmailProperty::Cid,\n                    EmailProperty::Language,\n                    EmailProperty::Location,\n                ]\n            });\n        let fetch_text_body_values = request.arguments.fetch_text_body_values.unwrap_or(false);\n        let fetch_html_body_values = request.arguments.fetch_html_body_values.unwrap_or(false);\n        let fetch_all_body_values = request.arguments.fetch_all_body_values.unwrap_or(false);\n        let max_body_value_bytes = request.arguments.max_body_value_bytes.unwrap_or(0);\n\n        let mut response = ParseResponse {\n            account_id: request.account_id,\n            parsed: VecMap::with_capacity(request.blob_ids.len()),\n            not_parsable: vec![],\n            not_found: vec![],\n        };\n\n        for blob_id in request.blob_ids.into_valid() {\n            // Fetch raw message to parse\n            let raw_message = match self.blob_download(&blob_id, access_token).await? {\n                Some(raw_message) => raw_message,\n                None => {\n                    response.not_found.push(blob_id);\n                    continue;\n                }\n            };\n            let message = if let Some(message) = MessageParser::new().parse(&raw_message) {\n                message\n            } else {\n                response.not_parsable.push(blob_id);\n                continue;\n            };\n            let raw_message = ChainedBytes::new(&raw_message);\n\n            // Prepare response\n            let mut email = Map::with_capacity(properties.len());\n            for property in &properties {\n                match property {\n                    EmailProperty::BlobId => {\n                        email.insert_unchecked(EmailProperty::BlobId, blob_id.clone());\n                    }\n\n                    EmailProperty::Size => {\n                        email.insert_unchecked(\n                            EmailProperty::Size,\n                            Value::Number(raw_message.len().into()),\n                        );\n                    }\n                    EmailProperty::HasAttachment => {\n                        email.insert_unchecked(\n                            EmailProperty::HasAttachment,\n                            Value::Bool(message.parts.iter().enumerate().any(|(part_id, part)| {\n                                let part_id = part_id as u32;\n                                match &part.body {\n                                    PartType::Html(_) | PartType::Text(_) => {\n                                        !message.text_body.contains(&part_id)\n                                            && !message.html_body.contains(&part_id)\n                                    }\n                                    PartType::Binary(_) | PartType::Message(_) => true,\n                                    _ => false,\n                                }\n                            })),\n                        );\n                    }\n                    EmailProperty::Preview => {\n                        email.insert_unchecked(\n                            EmailProperty::Preview,\n                            match message\n                                .text_body\n                                .first()\n                                .or_else(|| message.html_body.first())\n                                .and_then(|idx| message.parts.get(*idx as usize))\n                                .map(|part| &part.body)\n                            {\n                                Some(PartType::Text(text)) => {\n                                    preview_text(text.replace('\\r', \"\").into(), PREVIEW_LENGTH)\n                                        .into()\n                                }\n                                Some(PartType::Html(html)) => preview_text(\n                                    html_to_text(html).replace('\\r', \"\").into(),\n                                    PREVIEW_LENGTH,\n                                )\n                                .into(),\n                                _ => Value::Null,\n                            },\n                        );\n                    }\n                    EmailProperty::MessageId\n                    | EmailProperty::InReplyTo\n                    | EmailProperty::References\n                    | EmailProperty::Sender\n                    | EmailProperty::From\n                    | EmailProperty::To\n                    | EmailProperty::Cc\n                    | EmailProperty::Bcc\n                    | EmailProperty::ReplyTo\n                    | EmailProperty::Subject\n                    | EmailProperty::SentAt\n                    | EmailProperty::Header(_) => {\n                        email.insert_unchecked(\n                            property.clone(),\n                            message.parts[0]\n                                .headers\n                                .header_to_value(property, &raw_message),\n                        );\n                    }\n                    EmailProperty::Headers => {\n                        email.insert_unchecked(\n                            EmailProperty::Headers,\n                            message.parts[0].headers.headers_to_value(&raw_message),\n                        );\n                    }\n                    EmailProperty::TextBody\n                    | EmailProperty::HtmlBody\n                    | EmailProperty::Attachments => {\n                        let list = match property {\n                            EmailProperty::TextBody => &message.text_body,\n                            EmailProperty::HtmlBody => &message.html_body,\n                            EmailProperty::Attachments => &message.attachments,\n                            _ => unreachable!(),\n                        }\n                        .iter();\n                        email.insert_unchecked(\n                            property.clone(),\n                            list.map(|part_id| {\n                                message.parts.to_body_part(\n                                    *part_id,\n                                    &body_properties,\n                                    &raw_message,\n                                    &blob_id,\n                                    0,\n                                )\n                            })\n                            .collect::<Vec<_>>(),\n                        );\n                    }\n                    EmailProperty::BodyStructure => {\n                        email.insert_unchecked(\n                            EmailProperty::BodyStructure,\n                            message.parts.to_body_part(\n                                0,\n                                &body_properties,\n                                &raw_message,\n                                &blob_id,\n                                0,\n                            ),\n                        );\n                    }\n                    EmailProperty::BodyValues => {\n                        let mut body_values = Map::with_capacity(message.parts.len());\n                        for (part_id, part) in message.parts.iter().enumerate() {\n                            let part_id = part_id as u32;\n                            if ((message.html_body.contains(&part_id)\n                                && (fetch_all_body_values || fetch_html_body_values))\n                                || (message.text_body.contains(&part_id)\n                                    && (fetch_all_body_values || fetch_text_body_values)))\n                                && part.is_text()\n                            {\n                                let (is_truncated, value) =\n                                    part.body.truncate(max_body_value_bytes);\n                                body_values.insert_unchecked(\n                                    Key::Owned(part_id.to_string()),\n                                    Map::with_capacity(3)\n                                        .with_key_value(\n                                            EmailProperty::IsEncodingProblem,\n                                            part.is_encoding_problem,\n                                        )\n                                        .with_key_value(EmailProperty::IsTruncated, is_truncated)\n                                        .with_key_value(EmailProperty::Value, value),\n                                );\n                            }\n                        }\n                        email.insert_unchecked(EmailProperty::BodyValues, body_values);\n                    }\n                    EmailProperty::Id\n                    | EmailProperty::ThreadId\n                    | EmailProperty::Keywords\n                    | EmailProperty::MailboxIds\n                    | EmailProperty::ReceivedAt => {\n                        email.insert_unchecked(property.clone(), Value::Null);\n                    }\n\n                    _ => {\n                        return Err(trc::JmapEvent::InvalidArguments\n                            .into_err()\n                            .details(format!(\"Invalid property {property:?}\")));\n                    }\n                }\n            }\n            response.parsed.append(blob_id, email.into());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/email/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::JmapCacheState};\nuse common::{MessageStoreCache, Server, auth::AccessToken};\nuse email::cache::{MessageCacheFetch, email::MessageCacheAccess};\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::email::{Email, EmailComparator, EmailFilter},\n};\nuse mail_parser::HeaderName;\nuse nlp::language::Language;\nuse std::future::Future;\nuse store::{\n    ahash::{AHashMap, AHashSet},\n    roaring::RoaringBitmap,\n    search::{\n        EmailSearchField, SearchComparator, SearchFilter, SearchOperator, SearchQuery, SearchValue,\n    },\n    write::SearchIndex,\n};\nuse trc::AddContext;\nuse types::{acl::Acl, keyword::Keyword};\nuse utils::map::vec_map::VecMap;\n\npub trait EmailQuery: Sync + Send {\n    fn email_query(\n        &self,\n        request: QueryRequest<Email>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl EmailQuery for Server {\n    async fn email_query(\n        &self,\n        mut request: QueryRequest<Email>,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let mut filters = Vec::with_capacity(request.filter.len());\n        let cached_messages = self\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        for filter in std::mem::take(&mut request.filter) {\n            match filter {\n                Filter::Property(cond) => match cond {\n                    EmailFilter::Text(text) => {\n                        let (text, language) =\n                            Language::detect(text, self.core.jmap.default_language);\n\n                        filters.push(SearchFilter::Or);\n                        filters.push(SearchFilter::has_text(\n                            EmailSearchField::From,\n                            &text,\n                            Language::None,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            EmailSearchField::To,\n                            &text,\n                            Language::None,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            EmailSearchField::Cc,\n                            &text,\n                            Language::None,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            EmailSearchField::Bcc,\n                            &text,\n                            Language::None,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            EmailSearchField::Subject,\n                            &text,\n                            language,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            EmailSearchField::Body,\n                            &text,\n                            language,\n                        ));\n                        filters.push(SearchFilter::has_text(\n                            EmailSearchField::Attachment,\n                            text,\n                            language,\n                        ));\n                        filters.push(SearchFilter::End);\n                    }\n                    EmailFilter::From(text) => filters.push(SearchFilter::has_text(\n                        EmailSearchField::From,\n                        text,\n                        Language::None,\n                    )),\n                    EmailFilter::To(text) => filters.push(SearchFilter::has_text(\n                        EmailSearchField::To,\n                        text,\n                        Language::None,\n                    )),\n                    EmailFilter::Cc(text) => filters.push(SearchFilter::has_text(\n                        EmailSearchField::Cc,\n                        text,\n                        Language::None,\n                    )),\n                    EmailFilter::Bcc(text) => filters.push(SearchFilter::has_text(\n                        EmailSearchField::Bcc,\n                        text,\n                        Language::None,\n                    )),\n                    EmailFilter::Subject(text) => filters.push(SearchFilter::has_text_detect(\n                        EmailSearchField::Subject,\n                        text,\n                        self.core.jmap.default_language,\n                    )),\n                    EmailFilter::Body(text) => filters.push(SearchFilter::has_text_detect(\n                        EmailSearchField::Body,\n                        text,\n                        self.core.jmap.default_language,\n                    )),\n                    EmailFilter::Header(header) => {\n                        let mut header = header.into_iter();\n                        let header_name = header.next().ok_or_else(|| {\n                            trc::JmapEvent::InvalidArguments\n                                .into_err()\n                                .details(\"Header name is missing.\".to_string())\n                        })?;\n\n                        if let Some(header_name) = HeaderName::parse(header_name) {\n                            let value = header.next();\n                            let op = if matches!(\n                                header_name,\n                                HeaderName::MessageId\n                                    | HeaderName::InReplyTo\n                                    | HeaderName::References\n                                    | HeaderName::ResentMessageId\n                            ) || value.is_none()\n                            {\n                                SearchOperator::Equal\n                            } else {\n                                SearchOperator::Contains\n                            };\n\n                            filters.push(SearchFilter::cond(\n                                EmailSearchField::Headers,\n                                op,\n                                SearchValue::KeyValues(VecMap::with_capacity(1).with_append(\n                                    header_name.as_str().to_lowercase(),\n                                    value.unwrap_or_default(),\n                                )),\n                            ));\n                        }\n                    }\n                    EmailFilter::InMailbox(mailbox) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cached_messages\n                                .in_mailbox(mailbox.document_id())\n                                .map(|item| item.document_id),\n                        )))\n                    }\n                    EmailFilter::InMailboxOtherThan(mailboxes) => {\n                        let mailboxes = mailboxes\n                            .into_iter()\n                            .map(|m| m.document_id())\n                            .collect::<AHashSet<_>>();\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cached_messages.emails.items.iter().filter_map(|item| {\n                                if item\n                                    .mailboxes\n                                    .iter()\n                                    .any(|mb| mailboxes.contains(&mb.mailbox_id))\n                                {\n                                    None\n                                } else {\n                                    Some(item.document_id)\n                                }\n                            }),\n                        )));\n                    }\n                    EmailFilter::Before(date) => filters.push(SearchFilter::lt(\n                        EmailSearchField::ReceivedAt,\n                        date.timestamp(),\n                    )),\n                    EmailFilter::After(date) => filters.push(SearchFilter::gt(\n                        EmailSearchField::ReceivedAt,\n                        date.timestamp(),\n                    )),\n                    EmailFilter::MinSize(size) => {\n                        filters.push(SearchFilter::ge(EmailSearchField::Size, size))\n                    }\n                    EmailFilter::MaxSize(size) => {\n                        filters.push(SearchFilter::lt(EmailSearchField::Size, size))\n                    }\n                    EmailFilter::AllInThreadHaveKeyword(keyword) => filters.push(\n                        SearchFilter::is_in_set(thread_keywords(&cached_messages, keyword, true)),\n                    ),\n                    EmailFilter::SomeInThreadHaveKeyword(keyword) => filters.push(\n                        SearchFilter::is_in_set(thread_keywords(&cached_messages, keyword, false)),\n                    ),\n                    EmailFilter::NoneInThreadHaveKeyword(keyword) => {\n                        filters.push(SearchFilter::Not);\n                        filters.push(SearchFilter::is_in_set(thread_keywords(\n                            &cached_messages,\n                            keyword,\n                            false,\n                        )));\n                        filters.push(SearchFilter::End);\n                    }\n                    EmailFilter::HasKeyword(keyword) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cached_messages\n                                .with_keyword(&keyword)\n                                .map(|item| item.document_id),\n                        )));\n                    }\n                    EmailFilter::NotKeyword(keyword) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cached_messages\n                                .without_keyword(&keyword)\n                                .map(|item| item.document_id),\n                        )));\n                    }\n                    EmailFilter::HasAttachment(has_attach) => {\n                        filters.push(SearchFilter::eq(\n                            EmailSearchField::HasAttachment,\n                            has_attach,\n                        ));\n                    }\n\n                    // Non-standard\n                    EmailFilter::Id(ids) => {\n                        let mut set = RoaringBitmap::new();\n                        for id in ids {\n                            set.insert(id.document_id());\n                        }\n                        filters.push(SearchFilter::is_in_set(set));\n                    }\n                    EmailFilter::SentBefore(date) => {\n                        filters.push(SearchFilter::lt(EmailSearchField::SentAt, date.timestamp()))\n                    }\n                    EmailFilter::SentAfter(date) => {\n                        filters.push(SearchFilter::gt(EmailSearchField::SentAt, date.timestamp()))\n                    }\n                    EmailFilter::InThread(id) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cached_messages\n                                .in_thread(id.document_id())\n                                .map(|item| item.document_id),\n                        )))\n                    }\n                    other => {\n                        return Err(trc::JmapEvent::UnsupportedFilter\n                            .into_err()\n                            .details(other.to_string()));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        // Parse sort criteria\n        let mut comparators = Vec::with_capacity(request.sort.as_ref().map_or(1, |s| s.len()));\n        for comparator in request\n            .sort\n            .take()\n            .filter(|s| !s.is_empty())\n            .unwrap_or_default()\n        {\n            comparators.push(match comparator.property {\n                EmailComparator::ReceivedAt => {\n                    SearchComparator::field(EmailSearchField::ReceivedAt, comparator.is_ascending)\n                }\n                EmailComparator::Size => {\n                    SearchComparator::field(EmailSearchField::Size, comparator.is_ascending)\n                }\n                EmailComparator::From => {\n                    SearchComparator::field(EmailSearchField::From, comparator.is_ascending)\n                }\n                EmailComparator::To => {\n                    SearchComparator::field(EmailSearchField::To, comparator.is_ascending)\n                }\n                EmailComparator::Subject => {\n                    SearchComparator::field(EmailSearchField::Subject, comparator.is_ascending)\n                }\n                EmailComparator::SentAt => {\n                    SearchComparator::field(EmailSearchField::SentAt, comparator.is_ascending)\n                }\n                EmailComparator::HasKeyword(keyword) => SearchComparator::set(\n                    RoaringBitmap::from_iter(\n                        cached_messages\n                            .with_keyword(&keyword)\n                            .map(|item| item.document_id),\n                    ),\n                    comparator.is_ascending,\n                ),\n                EmailComparator::AllInThreadHaveKeyword(keyword) => SearchComparator::set(\n                    thread_keywords(&cached_messages, keyword, true),\n                    comparator.is_ascending,\n                ),\n                EmailComparator::SomeInThreadHaveKeyword(keyword) => SearchComparator::set(\n                    thread_keywords(&cached_messages, keyword, false),\n                    comparator.is_ascending,\n                ),\n                // Non-standard\n                EmailComparator::Cc => {\n                    SearchComparator::field(EmailSearchField::Cc, comparator.is_ascending)\n                }\n\n                other => {\n                    return Err(trc::JmapEvent::UnsupportedSort\n                        .into_err()\n                        .details(other.to_string()));\n                }\n            });\n        }\n\n        let results = self\n            .search_store()\n            .query_account(\n                SearchQuery::new(SearchIndex::Email)\n                    .with_filters(filters)\n                    .with_comparators(comparators)\n                    .with_account_id(account_id)\n                    .with_mask(if access_token.is_shared(account_id) {\n                        cached_messages.shared_messages(access_token, Acl::ReadItems)\n                    } else {\n                        cached_messages\n                            .emails\n                            .items\n                            .iter()\n                            .map(|item| item.document_id)\n                            .collect()\n                    }),\n            )\n            .await?;\n\n        let mut response = QueryResponseBuilder::new(\n            results.len(),\n            self.core.jmap.query_max_results,\n            cached_messages.get_state(false),\n            &request,\n        );\n\n        if !results.is_empty() {\n            let collapse_threads = request.arguments.collapse_threads.unwrap_or(false);\n            let mut seen_thread_ids = AHashSet::new();\n\n            for document_id in results {\n                let Some(thread_id) = cached_messages\n                    .email_by_id(&document_id)\n                    .map(|email| email.thread_id)\n                else {\n                    continue;\n                };\n                if collapse_threads && !seen_thread_ids.insert(thread_id) {\n                    continue;\n                }\n\n                if !response.add(thread_id, document_id) {\n                    break;\n                }\n            }\n        }\n\n        response.build()\n    }\n}\n\nfn thread_keywords(cache: &MessageStoreCache, keyword: Keyword, match_all: bool) -> RoaringBitmap {\n    let keyword_doc_ids =\n        RoaringBitmap::from_iter(cache.with_keyword(&keyword).map(|item| item.document_id));\n    if keyword_doc_ids.is_empty() {\n        return keyword_doc_ids;\n    }\n    let mut not_matched_ids = RoaringBitmap::new();\n    let mut matched_ids = RoaringBitmap::new();\n\n    let mut thread_map: AHashMap<u32, RoaringBitmap> = AHashMap::new();\n\n    for item in &cache.emails.items {\n        thread_map\n            .entry(item.thread_id)\n            .or_default()\n            .insert(item.document_id);\n    }\n\n    for item in &cache.emails.items {\n        let keyword_doc_id = item.document_id;\n        if !keyword_doc_ids.contains(keyword_doc_id)\n            || matched_ids.contains(keyword_doc_id)\n            || not_matched_ids.contains(keyword_doc_id)\n        {\n            continue;\n        }\n\n        if let Some(thread_doc_ids) = thread_map.get(&item.thread_id) {\n            let mut thread_tag_intersection = thread_doc_ids.clone();\n            thread_tag_intersection &= &keyword_doc_ids;\n\n            if (match_all && &thread_tag_intersection == thread_doc_ids)\n                || (!match_all && !thread_tag_intersection.is_empty())\n            {\n                matched_ids |= thread_doc_ids;\n            } else if !thread_tag_intersection.is_empty() {\n                not_matched_ids |= &thread_tag_intersection;\n            }\n        }\n    }\n\n    matched_ids\n}\n"
  },
  {
    "path": "crates/jmap/src/email/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::headers::{BuildHeader, ValueToHeader};\nuse crate::{\n    blob::download::BlobDownload,\n    changes::state::JmapCacheState,\n    email::{PatchResult, handle_email_patch, ingested_into_object},\n};\nuse common::{\n    Server, auth::AccessToken, ipc::PushNotification, storage::index::ObjectIndexBuilder,\n};\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess, mailbox::MailboxCacheAccess},\n    mailbox::{JUNK_ID, TRASH_ID, UidMailbox},\n    message::{\n        delete::EmailDeletion,\n        ingest::{EmailIngest, IngestEmail, IngestSource},\n        metadata::MessageData,\n    },\n};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::email::{Email, EmailProperty, EmailValue},\n    references::resolve::ResolveCreatedReference,\n    request::IntoValid,\n    types::state::State,\n};\nuse jmap_tools::{Key, Value};\nuse mail_builder::{\n    MessageBuilder,\n    headers::{\n        HeaderType, address::Address, content_type::ContentType, date::Date, message_id::MessageId,\n        raw::Raw, text::Text,\n    },\n    mime::{BodyPart, MimePart},\n};\nuse mail_parser::MessageParser;\nuse std::future::Future;\nuse std::{borrow::Cow, collections::HashMap};\nuse store::{\n    ValueKey,\n    ahash::AHashMap,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection, VanishedCollection},\n    id::Id,\n    keyword::{ArchivedKeyword, Keyword},\n    type_state::{DataType, StateChange},\n};\n\npub trait EmailSet: Sync + Send {\n    fn email_set(\n        &self,\n        request: SetRequest<'_, Email>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<SetResponse<Email>>> + Send;\n}\n\nimpl EmailSet for Server {\n    async fn email_set(\n        &self,\n        mut request: SetRequest<'_, Email>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<Email>> {\n        // Prepare response\n        let account_id = request.account_id.document_id();\n        let cache = self.get_cached_messages(account_id).await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?\n            .with_state(cache.assert_state(false, &request.if_in_state)?);\n\n        // Obtain mailboxIds\n        let (can_add_mailbox_ids, can_delete_mailbox_ids, can_modify_mailbox_ids) =\n            if access_token.is_shared(account_id) {\n                (\n                    cache.shared_mailboxes(access_token, Acl::AddItems).into(),\n                    cache\n                        .shared_mailboxes(access_token, Acl::RemoveItems)\n                        .into(),\n                    cache\n                        .shared_mailboxes(access_token, Acl::ModifyItems)\n                        .into(),\n                )\n            } else {\n                (None, None, None)\n            };\n\n        // Obtain import access token\n        let import_access_token = if account_id != access_token.primary_id() {\n            #[cfg(feature = \"test_mode\")]\n            {\n                std::sync::Arc::new(AccessToken::from_id(account_id)).into()\n            }\n\n            #[cfg(not(feature = \"test_mode\"))]\n            {\n                self.get_access_token(account_id)\n                    .await\n                    .caused_by(trc::location!())?\n                    .into()\n            }\n        } else {\n            None\n        };\n\n        let mut last_change_id = None;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Process creates\n        'create: for (id, object) in request.unwrap_create() {\n            let Value::Object(mut object) = object else {\n                continue;\n            };\n\n            let has_body_structure =\n                object.contains_key(&Key::Property(EmailProperty::BodyStructure));\n            let mut builder = MessageBuilder::new();\n            let mut mailboxes = Vec::new();\n            let mut keywords = Vec::new();\n            let mut received_at = None;\n\n            // Parse body values\n            let body_values = object\n                .remove(&Key::Property(EmailProperty::BodyValues))\n                .and_then(|obj| obj.into_object())\n                .and_then(|obj| {\n                    let mut values = HashMap::with_capacity(obj.len());\n                    for (key, value) in obj.into_vec() {\n                        let id = key.into_string();\n                        if let Value::Object(mut bv) = value {\n                            values.insert(\n                                id,\n                                bv.remove(&Key::Property(EmailProperty::Value))?\n                                    .into_string()?,\n                            );\n                        } else {\n                            return None;\n                        }\n                    }\n                    Some(values)\n                });\n            let mut size_attachments = 0;\n\n            // Parse properties\n            for (property, mut value) in object.into_vec() {\n                if let Err(err) = response.resolve_self_references(&mut value) {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                };\n                let Key::Property(property) = property else {\n                    response.invalid_property_create(id, property.into_owned());\n                    continue 'create;\n                };\n\n                match (property, value) {\n                    (EmailProperty::MailboxIds, Value::Object(ids)) => {\n                        mailboxes = ids\n                            .into_expanded_boolean_set()\n                            .filter_map(|id| {\n                                id.try_into_property()?.try_into_id()?.document_id().into()\n                            })\n                            .collect();\n                    }\n                    (EmailProperty::Keywords, Value::Object(keywords_)) => {\n                        keywords = keywords_\n                            .into_expanded_boolean_set()\n                            .filter_map(|id| id.try_into_property()?.try_into_keyword())\n                            .collect();\n                    }\n                    (EmailProperty::Pointer(pointer), value) => {\n                        match handle_email_patch(&pointer, value) {\n                            PatchResult::SetKeyword(keyword) => {\n                                if !keywords.contains(keyword) {\n                                    keywords.push(keyword.clone());\n                                }\n                            }\n                            PatchResult::RemoveKeyword(keyword) => {\n                                keywords.retain(|k| k != keyword);\n                            }\n                            PatchResult::AddMailbox(id) => {\n                                if !mailboxes.contains(&id) {\n                                    mailboxes.push(id);\n                                }\n                            }\n                            PatchResult::RemoveMailbox(id) => {\n                                mailboxes.retain(|mid| mid != &id);\n                            }\n                            PatchResult::Invalid(set_error) => {\n                                response.not_created.append(id, set_error);\n                                continue 'create;\n                            }\n                        }\n                    }\n                    (\n                        header @ (EmailProperty::MessageId\n                        | EmailProperty::InReplyTo\n                        | EmailProperty::References),\n                        Value::Array(values),\n                    ) => {\n                        builder = builder.header(\n                            header.as_rfc_header(),\n                            MessageId {\n                                id: values\n                                    .into_iter()\n                                    .filter_map(|value| value.into_string())\n                                    .collect(),\n                            },\n                        );\n                    }\n\n                    (\n                        header @ (EmailProperty::Sender\n                        | EmailProperty::From\n                        | EmailProperty::To\n                        | EmailProperty::Cc\n                        | EmailProperty::Bcc\n                        | EmailProperty::ReplyTo),\n                        value,\n                    ) => {\n                        if let Some(addresses) = value.try_into_address_list() {\n                            builder =\n                                builder.header(header.as_rfc_header(), Address::List(addresses));\n                        } else {\n                            response.invalid_property_create(id, header);\n                            continue 'create;\n                        }\n                    }\n                    (EmailProperty::Subject, Value::Str(value)) => {\n                        builder = builder.subject(value);\n                    }\n\n                    (EmailProperty::ReceivedAt, Value::Element(EmailValue::Date(value))) => {\n                        received_at = (value.timestamp() as u64).into();\n                    }\n\n                    (EmailProperty::SentAt, Value::Element(EmailValue::Date(value))) => {\n                        builder = builder.date(Date::new(value.timestamp()));\n                    }\n\n                    (\n                        property @ (EmailProperty::TextBody\n                        | EmailProperty::HtmlBody\n                        | EmailProperty::Attachments\n                        | EmailProperty::BodyStructure),\n                        value,\n                    ) => {\n                        // Validate request\n                        let (values, expected_content_type) = match property {\n                            EmailProperty::BodyStructure => (vec![value], None),\n                            EmailProperty::TextBody | EmailProperty::HtmlBody\n                                if !has_body_structure =>\n                            {\n                                let values = value.into_array().unwrap_or_default();\n                                if values.len() <= 1 {\n                                    (\n                                        values,\n                                        Some(match property {\n                                            EmailProperty::TextBody => \"text/plain\",\n                                            EmailProperty::HtmlBody => \"text/html\",\n                                            _ => unreachable!(),\n                                        }),\n                                    )\n                                } else {\n                                    response.not_created.append(\n                                        id,\n                                        SetError::invalid_properties()\n                                            .with_property(property)\n                                            .with_description(\"Only one part is allowed.\"),\n                                    );\n                                    continue 'create;\n                                }\n                            }\n                            EmailProperty::Attachments if !has_body_structure => {\n                                (value.into_array().unwrap_or_default(), None)\n                            }\n                            _ => {\n                                response.not_created.append(\n                                    id,\n                                    SetError::invalid_properties()\n                                        .with_properties([property, EmailProperty::BodyStructure])\n                                        .with_description(\n                                            \"Cannot set both properties on a same request.\",\n                                        ),\n                                );\n                                continue 'create;\n                            }\n                        };\n\n                        // Iterate parts\n                        let mut values_stack = Vec::new();\n                        let mut values = values.into_iter();\n                        let mut parts = Vec::new();\n                        loop {\n                            while let Some(value) = values.next() {\n                                let mut blob_id = None;\n                                let mut part_id = None;\n                                let mut content_type = None;\n                                let mut content_disposition = None;\n                                let mut name = None;\n                                let mut charset = None;\n                                let mut subparts = None;\n                                let mut has_size = false;\n                                let mut headers: Vec<(Cow<str>, HeaderType)> = Vec::new();\n\n                                if let Some(obj) = value.into_object() {\n                                    for (body_property, value) in obj.into_vec() {\n                                        let Key::Property(body_property) = body_property else {\n                                            continue;\n                                        };\n\n                                        match (body_property, value) {\n                                            (EmailProperty::Type, Value::Str(value)) => {\n                                                content_type = value.into_owned().into();\n                                            }\n                                            (EmailProperty::PartId, Value::Str(value)) => {\n                                                part_id = value.into_owned().into();\n                                            }\n                                            (\n                                                EmailProperty::BlobId,\n                                                Value::Element(EmailValue::BlobId(value)),\n                                            ) => {\n                                                blob_id = value.into();\n                                            }\n                                            (EmailProperty::Disposition, Value::Str(value)) => {\n                                                content_disposition = value.into_owned().into();\n                                            }\n                                            (EmailProperty::Name, Value::Str(value)) => {\n                                                name = value.into_owned().into();\n                                            }\n                                            (EmailProperty::Charset, Value::Str(value)) => {\n                                                charset = value.into_owned().into();\n                                            }\n                                            (EmailProperty::Language, Value::Array(values)) => {\n                                                headers.push((\n                                                    \"Content-Language\".into(),\n                                                    Text::new(\n                                                        values\n                                                            .into_iter()\n                                                            .filter_map(|v| v.into_string())\n                                                            .fold(\n                                                                String::with_capacity(64),\n                                                                |mut h, v| {\n                                                                    if !h.is_empty() {\n                                                                        h.push_str(\", \");\n                                                                    }\n                                                                    h.push_str(&v);\n                                                                    h\n                                                                },\n                                                            ),\n                                                    )\n                                                    .into(),\n                                                ));\n                                            }\n                                            (EmailProperty::Cid, Value::Str(value)) => {\n                                                headers.push((\n                                                    \"Content-ID\".into(),\n                                                    MessageId::new(value).into(),\n                                                ));\n                                            }\n                                            (EmailProperty::Location, Value::Str(value)) => {\n                                                headers.push((\n                                                    \"Content-Location\".into(),\n                                                    Text::new(value).into(),\n                                                ));\n                                            }\n                                            (EmailProperty::Header(header), Value::Str(value))\n                                                if !header.header.eq_ignore_ascii_case(\n                                                    \"content-transfer-encoding\",\n                                                ) =>\n                                            {\n                                                headers.push((\n                                                    header.header.into(),\n                                                    Raw::from(value).into(),\n                                                ));\n                                            }\n                                            (\n                                                EmailProperty::Header(header),\n                                                Value::Array(values),\n                                            ) if !header.header.eq_ignore_ascii_case(\n                                                \"content-transfer-encoding\",\n                                            ) =>\n                                            {\n                                                for value in values {\n                                                    if let Some(value) = value.into_string() {\n                                                        headers.push((\n                                                            header.header.clone().into(),\n                                                            Raw::from(value).into(),\n                                                        ));\n                                                    }\n                                                }\n                                            }\n                                            (EmailProperty::Headers, _) => {\n                                                response.not_created.append(\n                                                    id,\n                                                    SetError::invalid_properties()\n                                                        .with_property((\n                                                            property,\n                                                            EmailProperty::Headers,\n                                                        ))\n                                                        .with_description(\n                                                            \"Headers have to be set individually.\",\n                                                        ),\n                                                );\n                                                continue 'create;\n                                            }\n                                            (EmailProperty::Size, _) => {\n                                                has_size = true;\n                                            }\n                                            (EmailProperty::SubParts, Value::Array(values)) => {\n                                                subparts = values.into();\n                                            }\n                                            (body_property, value) if value != Value::Null => {\n                                                response.not_created.append(\n                                                    id,\n                                                    SetError::invalid_properties()\n                                                        .with_property((property, body_property))\n                                                        .with_description(\"Cannot set property.\"),\n                                                );\n                                                continue 'create;\n                                            }\n                                            _ => {}\n                                        }\n                                    }\n                                }\n\n                                // Validate content-type\n                                let content_type =\n                                    content_type.unwrap_or_else(|| \"text/plain\".to_string());\n                                let is_multipart = content_type.starts_with(\"multipart/\");\n                                if is_multipart {\n                                    if !matches!(property, EmailProperty::BodyStructure) {\n                                        response.not_created.append(\n                                            id,\n                                            SetError::invalid_properties()\n                                                .with_property((property, EmailProperty::Type))\n                                                .with_description(\"Multiparts can only be set with bodyStructure.\"),\n                                        );\n                                        continue 'create;\n                                    }\n                                } else if expected_content_type\n                                    .as_ref()\n                                    .is_some_and(|v| v != &content_type)\n                                {\n                                    response.not_created.append(\n                                        id,\n                                        SetError::invalid_properties()\n                                            .with_property((property, EmailProperty::Type))\n                                            .with_description(format!(\n                                                \"Expected one body part of type \\\"{}\\\"\",\n                                                expected_content_type.unwrap()\n                                            )),\n                                    );\n                                    continue 'create;\n                                }\n\n                                // Validate partId/blobId\n                                match (blob_id.is_some(), part_id.is_some()) {\n                                    (true, true) if !is_multipart => {\n                                        response.not_created.append(\n                                        id,\n                                        SetError::invalid_properties()\n                                            .with_properties([(property.clone(), EmailProperty::BlobId), (property, EmailProperty::PartId)])\n                                            .with_description(\n                                                \"Cannot specify both \\\"partId\\\" and \\\"blobId\\\".\",\n                                            ),\n                                    );\n                                        continue 'create;\n                                    }\n                                    (false, false) if !is_multipart => {\n                                        response.not_created.append(\n                                        id,\n                                        SetError::invalid_properties()\n                                            .with_description(\"Expected a \\\"partId\\\" or \\\"blobId\\\" field in body part.\"),\n                                    );\n                                        continue 'create;\n                                    }\n                                    (false, true) if !is_multipart && has_size => {\n                                        response.not_created.append(\n                                        id,\n                                        SetError::invalid_properties()\n                                            .with_property((property, EmailProperty::Size))\n                                            .with_description(\n                                                \"Cannot specify \\\"size\\\" when providing a \\\"partId\\\".\",\n                                            ),\n                                    );\n                                        continue 'create;\n                                    }\n                                    (true, _) | (_, true) if is_multipart => {\n                                        response.not_created.append(\n                                        id,\n                                        SetError::invalid_properties()\n                                            .with_properties([(property.clone(), EmailProperty::BlobId), (property, EmailProperty::PartId)])\n                                            .with_description(\n                                                \"Cannot specify \\\"partId\\\" or \\\"blobId\\\" in multipart body parts.\",\n                                            ),\n                                    );\n                                        continue 'create;\n                                    }\n                                    _ => (),\n                                }\n\n                                // Set Content-Type and Content-Disposition\n                                let mut content_type = ContentType::new(content_type);\n                                if !is_multipart {\n                                    if let Some(charset) = charset {\n                                        if part_id.is_none() {\n                                            content_type\n                                                .attributes\n                                                .push((\"charset\".into(), charset.into()));\n                                        } else {\n                                            response.not_created.append(\n                                            id,\n                                            SetError::invalid_properties()\n                                                .with_property((property, EmailProperty::Charset))\n                                                .with_description(\n                                                    \"Cannot specify a character set when providing a \\\"partId\\\".\",\n                                                ),\n                                        );\n                                            continue 'create;\n                                        }\n                                    } else if part_id.is_some() {\n                                        content_type\n                                            .attributes\n                                            .push((\"charset\".into(), \"utf-8\".into()));\n                                    }\n                                    match (content_disposition, name) {\n                                        (Some(disposition), Some(filename)) => {\n                                            headers.push((\n                                                \"Content-Disposition\".into(),\n                                                ContentType::new(disposition)\n                                                    .attribute(\"filename\", filename)\n                                                    .into(),\n                                            ));\n                                        }\n                                        (Some(disposition), None) => {\n                                            headers.push((\n                                                \"Content-Disposition\".into(),\n                                                ContentType::new(disposition).into(),\n                                            ));\n                                        }\n                                        (None, Some(filename)) => {\n                                            content_type\n                                                .attributes\n                                                .push((\"name\".into(), filename.into()));\n                                        }\n                                        (None, None) => (),\n                                    };\n                                }\n                                headers.push((\"Content-Type\".into(), content_type.into()));\n\n                                // In test, sort headers to avoid randomness\n                                #[cfg(feature = \"test_mode\")]\n                                {\n                                    headers.sort_unstable_by(|a, b| match a.0.cmp(&b.0) {\n                                        std::cmp::Ordering::Equal => a.1.cmp(&b.1),\n                                        ord => ord,\n                                    });\n                                }\n                                // Retrieve contents\n                                parts.push(MimePart {\n                                    headers,\n                                    contents: if !is_multipart {\n                                        if let Some(blob_id) = blob_id {\n                                            match self.blob_download(&blob_id, access_token).await? {\n                                                Some(contents) => {\n                                                    BodyPart::Binary(contents.into())\n                                                }\n                                                None => {\n                                                    response.not_created.append(\n                                                    id,\n                                                    SetError::new(SetErrorType::BlobNotFound).with_description(\n                                                        format!(\"blobId {blob_id} does not exist on this server.\")\n                                                    ),\n                                                );\n                                                    continue 'create;\n                                                }\n                                            }\n                                        } else if let Some(part_id) = part_id {\n                                            if let Some(contents) =\n                                                body_values.as_ref().and_then(|bv| bv.get(&part_id))\n                                            {\n                                                BodyPart::Text(contents.as_ref().into())\n                                            } else {\n                                                response.not_created.append(\n                                                    id,\n                                                    SetError::invalid_properties()\n                                                        .with_property((property, EmailProperty::PartId))\n                                                        .with_description(format!(\n                                                        \"Missing body value for partId {part_id:?}\"\n                                                    )),\n                                                );\n                                                continue 'create;\n                                            }\n                                        } else {\n                                            unreachable!()\n                                        }\n                                    } else {\n                                        BodyPart::Multipart(vec![])\n                                    },\n                                });\n\n                                // Check attachment sizes\n                                if !is_multipart {\n                                    size_attachments += parts.last().unwrap().size();\n                                    if self.core.jmap.mail_attachments_max_size > 0\n                                        && size_attachments\n                                            > self.core.jmap.mail_attachments_max_size\n                                    {\n                                        response.not_created.append(\n                                            id,\n                                            SetError::invalid_properties()\n                                                .with_property(property)\n                                                .with_description(format!(\n                                                    \"Message exceeds maximum size of {} bytes.\",\n                                                    self.core.jmap.mail_attachments_max_size\n                                                )),\n                                        );\n                                        continue 'create;\n                                    }\n                                } else if let Some(subparts) = subparts {\n                                    values_stack.push((values, parts));\n                                    parts = Vec::with_capacity(subparts.len());\n                                    values = subparts.into_iter();\n                                    continue;\n                                }\n                            }\n\n                            if let Some((prev_values, mut prev_parts)) = values_stack.pop() {\n                                values = prev_values;\n                                prev_parts.last_mut().unwrap().contents =\n                                    BodyPart::Multipart(parts);\n                                parts = prev_parts;\n                            } else {\n                                break;\n                            }\n                        }\n\n                        match property {\n                            EmailProperty::TextBody => {\n                                builder.text_body = parts.pop();\n                            }\n                            EmailProperty::HtmlBody => {\n                                builder.html_body = parts.pop();\n                            }\n                            EmailProperty::Attachments => {\n                                builder.attachments = parts.into();\n                            }\n                            _ => {\n                                builder.body = parts.pop();\n                            }\n                        }\n                    }\n\n                    (EmailProperty::Header(header), value) => {\n                        match builder.build_header(header, value) {\n                            Ok(builder_) => {\n                                builder = builder_;\n                            }\n                            Err(header) => {\n                                response.invalid_property_create(id, EmailProperty::Header(header));\n                                continue 'create;\n                            }\n                        }\n                    }\n\n                    (_, Value::Null) => (),\n\n                    (property, _) => {\n                        response.invalid_property_create(id, property);\n                        continue 'create;\n                    }\n                }\n            }\n\n            // Make sure message belongs to at least one mailbox\n            if mailboxes.is_empty() {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(EmailProperty::MailboxIds)\n                        .with_description(\"Message has to belong to at least one mailbox.\"),\n                );\n                continue 'create;\n            }\n\n            // Verify that the mailboxIds are valid\n            for mailbox_id in &mailboxes {\n                if !cache.has_mailbox_id(mailbox_id) {\n                    response.not_created.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(EmailProperty::MailboxIds)\n                            .with_description(format!(\n                                \"mailboxId {} does not exist.\",\n                                Id::from(*mailbox_id)\n                            )),\n                    );\n                    continue 'create;\n                } else if can_add_mailbox_ids\n                    .as_ref()\n                    .is_some_and(|ids| !ids.contains(*mailbox_id))\n                {\n                    response.not_created.append(\n                        id,\n                        SetError::forbidden().with_description(format!(\n                            \"You are not allowed to add messages to mailbox {}.\",\n                            Id::from(*mailbox_id)\n                        )),\n                    );\n                    continue 'create;\n                }\n            }\n\n            // Make sure the message is not empty\n            if builder.headers.is_empty()\n                && builder.body.is_none()\n                && builder.html_body.is_none()\n                && builder.text_body.is_none()\n                && builder.attachments.is_none()\n            {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_description(\"Message has to have at least one header or body part.\"),\n                );\n                continue 'create;\n            }\n\n            // In test, sort headers to avoid randomness\n            #[cfg(feature = \"test_mode\")]\n            {\n                builder\n                    .headers\n                    .sort_unstable_by(|a, b| match a.0.cmp(&b.0) {\n                        std::cmp::Ordering::Equal => a.1.cmp(&b.1),\n                        ord => ord,\n                    });\n            }\n\n            // Build message\n            let mut raw_message = Vec::with_capacity((4 * size_attachments / 3) + 1024);\n            builder.write_to(&mut raw_message).unwrap_or_default();\n\n            // Ingest message\n            match self\n                .email_ingest(IngestEmail {\n                    raw_message: &raw_message,\n                    message: MessageParser::new().parse(&raw_message),\n                    blob_hash: None,\n                    access_token: import_access_token.as_deref().unwrap_or(access_token),\n                    mailbox_ids: mailboxes,\n                    keywords,\n                    received_at,\n                    source: IngestSource::Jmap {\n                        train_classifier: true,\n                    },\n                    session_id: session.session_id,\n                })\n                .await\n            {\n                Ok(message) => {\n                    last_change_id = message.change_id.into();\n                    response\n                        .created\n                        .insert(id, ingested_into_object(message).into());\n                }\n                Err(err) if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota)) => {\n                    response.not_created.append(\n                        id,\n                        SetError::new(SetErrorType::OverQuota)\n                            .with_description(\"You have exceeded your disk quota.\"),\n                    );\n                }\n                Err(err) => return Err(err),\n            }\n        }\n\n        // Process updates\n        let mut batch = BatchBuilder::new();\n        let mut changed_mailboxes: AHashMap<u32, Vec<u32>> = AHashMap::new();\n        let mut will_update = Vec::with_capacity(request.update.as_ref().map_or(0, |u| u.len()));\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain message data\n            let document_id = id.document_id();\n            let data_ = match self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Email,\n                    document_id,\n                ))\n                .await?\n            {\n                Some(data) => data,\n                None => {\n                    response.not_updated.append(id, SetError::not_found());\n                    continue 'update;\n                }\n            };\n            let data = data_\n                .to_unarchived::<MessageData>()\n                .caused_by(trc::location!())?;\n            let mut new_data = data.inner.to_builder();\n\n            for (property, mut value) in object.into_expanded_object() {\n                if let Err(err) = response.resolve_self_references(&mut value) {\n                    response.not_updated.append(id, err);\n                    continue 'update;\n                };\n\n                match (property, value) {\n                    (Key::Property(EmailProperty::MailboxIds), Value::Object(ids)) => {\n                        new_data.set_mailboxes(\n                            ids.into_expanded_boolean_set()\n                                .filter_map(|id| {\n                                    UidMailbox::new_unassigned(\n                                        id.try_into_property()?.try_into_id()?.document_id(),\n                                    )\n                                    .into()\n                                })\n                                .collect(),\n                        );\n                    }\n                    (Key::Property(EmailProperty::Keywords), Value::Object(keywords_)) => {\n                        new_data.set_keywords(\n                            keywords_\n                                .into_expanded_boolean_set()\n                                .filter_map(|keyword| {\n                                    keyword.try_into_property()?.try_into_keyword()\n                                })\n                                .collect(),\n                        );\n                    }\n                    (Key::Property(EmailProperty::Pointer(pointer)), value) => {\n                        match handle_email_patch(&pointer, value) {\n                            PatchResult::SetKeyword(keyword) => {\n                                new_data.add_keyword(keyword.clone());\n                            }\n                            PatchResult::RemoveKeyword(keyword) => {\n                                new_data.remove_keyword(keyword);\n                            }\n                            PatchResult::AddMailbox(id) => {\n                                new_data.add_mailbox(UidMailbox::new_unassigned(id));\n                            }\n                            PatchResult::RemoveMailbox(id) => {\n                                new_data.remove_mailbox(id);\n                            }\n                            PatchResult::Invalid(set_error) => {\n                                response.not_updated.append(id, set_error);\n                                continue 'update;\n                            }\n                        }\n                    }\n                    (property, _) => {\n                        response.invalid_property_update(id, property.into_owned());\n                        continue 'update;\n                    }\n                }\n            }\n\n            let has_keyword_changes = new_data.has_keyword_changes(data.inner);\n            let has_mailbox_changes = new_data.has_mailbox_changes(data.inner);\n            if !has_keyword_changes && !has_mailbox_changes {\n                response.updated.append(id, None);\n                continue 'update;\n            }\n\n            // Process keywords\n            let mut train_spam = None;\n            if has_keyword_changes {\n                // Verify permissions on shared accounts\n                if can_modify_mailbox_ids.as_ref().is_some_and(|ids| {\n                    !new_data\n                        .mailboxes\n                        .iter()\n                        .any(|mb| ids.contains(mb.mailbox_id))\n                }) {\n                    response.not_updated.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to modify keywords.\"),\n                    );\n                    continue 'update;\n                }\n\n                // Process keyword changes\n                let mut changed_seen = false;\n                for keyword in new_data.added_keywords(data.inner) {\n                    match keyword {\n                        Keyword::Seen => {\n                            changed_seen = true;\n                        }\n                        Keyword::Junk => {\n                            train_spam = Some(true);\n                        }\n                        Keyword::NotJunk => {\n                            train_spam = Some(false);\n                        }\n                        _ => {}\n                    }\n                }\n                for keyword in new_data.removed_keywords(data.inner) {\n                    match keyword {\n                        ArchivedKeyword::Seen => {\n                            changed_seen = true;\n                        }\n                        ArchivedKeyword::Junk if train_spam.is_none() => {\n                            train_spam = Some(false);\n                        }\n                        _ => {}\n                    }\n                }\n\n                // Set all current mailboxes as changed if the Seen tag changed\n                if changed_seen {\n                    for mailbox_id in new_data.mailboxes.iter() {\n                        changed_mailboxes.insert(mailbox_id.mailbox_id, Vec::new());\n                    }\n                }\n            }\n\n            // Process mailboxes\n            if has_mailbox_changes {\n                // Make sure the message is at least in one mailbox\n                if new_data.mailboxes.is_empty() {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(EmailProperty::MailboxIds)\n                            .with_description(\"Message has to belong to at least one mailbox.\"),\n                    );\n                    continue 'update;\n                }\n\n                // Make sure all new mailboxIds are valid\n                for mailbox_id in new_data.added_mailboxes(data.inner) {\n                    if cache.has_mailbox_id(&mailbox_id.mailbox_id) {\n                        // Verify permissions on shared accounts\n                        if can_add_mailbox_ids\n                            .as_ref()\n                            .is_none_or(|ids| ids.contains(mailbox_id.mailbox_id))\n                        {\n                            if mailbox_id.mailbox_id == JUNK_ID {\n                                train_spam = Some(true);\n                            }\n\n                            changed_mailboxes.insert(mailbox_id.mailbox_id, Vec::new());\n                        } else {\n                            response.not_updated.append(\n                                id,\n                                SetError::forbidden().with_description(format!(\n                                    \"You are not allowed to add messages to mailbox {}.\",\n                                    Id::from(mailbox_id.mailbox_id)\n                                )),\n                            );\n                            continue 'update;\n                        }\n                    } else {\n                        response.not_updated.append(\n                            id,\n                            SetError::invalid_properties()\n                                .with_property(EmailProperty::MailboxIds)\n                                .with_description(format!(\n                                    \"mailboxId {} does not exist.\",\n                                    Id::from(mailbox_id.mailbox_id)\n                                )),\n                        );\n                        continue 'update;\n                    }\n                }\n\n                // Add all removed mailboxes to change list\n                for mailbox_id in new_data.removed_mailboxes(data.inner) {\n                    // Verify permissions on shared accounts\n                    if can_delete_mailbox_ids\n                        .as_ref()\n                        .is_none_or(|ids| ids.contains(u32::from(mailbox_id.mailbox_id)))\n                    {\n                        if mailbox_id.mailbox_id == JUNK_ID\n                            && !new_data\n                                .mailboxes\n                                .iter()\n                                .any(|mb| mb.mailbox_id == TRASH_ID)\n                        {\n                            train_spam = Some(false);\n                        }\n\n                        changed_mailboxes\n                            .entry(mailbox_id.mailbox_id.to_native())\n                            .or_default()\n                            .push(mailbox_id.uid.to_native());\n                    } else {\n                        response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(format!(\n                                \"You are not allowed to delete messages from mailbox {}.\",\n                                mailbox_id.mailbox_id\n                            )),\n                        );\n                        continue 'update;\n                    }\n                }\n\n                // Obtain IMAP UIDs for added mailboxes\n                let ids = self\n                    .assign_email_ids(\n                        account_id,\n                        new_data\n                            .mailboxes\n                            .iter()\n                            .filter(|m| m.uid == 0)\n                            .map(|m| m.mailbox_id),\n                        false,\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n                for (uid_mailbox, uid) in new_data\n                    .mailboxes\n                    .iter_mut()\n                    .filter(|m| m.uid == 0)\n                    .zip(ids)\n                {\n                    uid_mailbox.uid = uid;\n                }\n            }\n\n            // Write changes\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Email)\n                .with_document(document_id)\n                .custom(\n                    ObjectIndexBuilder::new()\n                        .with_current(data)\n                        .with_changes(new_data.seal()),\n                )\n                .caused_by(trc::location!())?;\n\n            if let Some(train_spam) = train_spam {\n                self.add_account_spam_sample(\n                    &mut batch,\n                    account_id,\n                    document_id,\n                    train_spam,\n                    session.session_id,\n                )\n                .await\n                .caused_by(trc::location!())?;\n            }\n\n            batch.commit_point();\n            will_update.push(id);\n        }\n\n        if !batch.is_empty() {\n            // Log mailbox changes\n            for (parent_id, deleted_uids) in changed_mailboxes {\n                batch.log_container_property_change(SyncCollection::Email, parent_id);\n                for deleted_uid in deleted_uids {\n                    batch.log_vanished_item(VanishedCollection::Email, (parent_id, deleted_uid));\n                }\n            }\n\n            match self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n            {\n                Ok(change_id) => {\n                    last_change_id = change_id.into();\n\n                    // Add to updated list\n                    for id in will_update {\n                        response.updated.append(id, None);\n                    }\n                }\n                Err(err) if err.is_assertion_failure() => {\n                    for id in will_update {\n                        response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(\n                                \"Another process modified this message, please try again.\",\n                            ),\n                        );\n                    }\n                }\n                Err(err) => {\n                    return Err(err.caused_by(trc::location!()));\n                }\n            }\n        }\n\n        // Process deletions\n        if !will_destroy.is_empty() {\n            let email_ids = cache.email_document_ids();\n            let can_destroy_message_ids = if access_token.is_shared(account_id) {\n                cache.shared_messages(access_token, Acl::RemoveItems).into()\n            } else {\n                None\n            };\n            let mut destroy_ids = RoaringBitmap::new();\n            for destroy_id in will_destroy {\n                let document_id = destroy_id.document_id();\n\n                if email_ids.contains(document_id) {\n                    if !matches!(&can_destroy_message_ids, Some(ids) if !ids.contains(document_id))\n                    {\n                        destroy_ids.insert(document_id);\n                        response.destroyed.push(destroy_id);\n                    } else {\n                        response.not_destroyed.append(\n                            destroy_id,\n                            SetError::forbidden()\n                                .with_description(\"You are not allowed to delete this message.\"),\n                        );\n                    }\n                } else {\n                    response\n                        .not_destroyed\n                        .append(destroy_id, SetError::not_found());\n                }\n            }\n\n            if !destroy_ids.is_empty() {\n                // Batch delete messages\n                let mut batch = BatchBuilder::new();\n                let not_destroyed = self\n                    .emails_delete(\n                        account_id,\n                        access_token.tenant_id(),\n                        &mut batch,\n                        destroy_ids,\n                    )\n                    .await?;\n                if !batch.is_empty() {\n                    last_change_id = self\n                        .commit_batch(batch)\n                        .await\n                        .and_then(|ids| ids.last_change_id(account_id))\n                        .caused_by(trc::location!())?\n                        .into();\n                    self.notify_task_queue();\n                }\n\n                // Mark messages that were not found as not destroyed (this should not occur in practice)\n                if !not_destroyed.is_empty() {\n                    let mut destroyed = Vec::with_capacity(response.destroyed.len());\n\n                    for destroy_id in response.destroyed {\n                        if not_destroyed.contains(destroy_id.document_id()) {\n                            response\n                                .not_destroyed\n                                .append(destroy_id, SetError::not_found());\n                        } else {\n                            destroyed.push(destroy_id);\n                        }\n                    }\n\n                    response.destroyed = destroyed;\n                }\n            }\n        }\n\n        // Update state\n        if let Some(change_id) = last_change_id {\n            if response.updated.is_empty() && response.destroyed.is_empty() {\n                // Message ingest does not broadcast state changes\n                self.broadcast_push_notification(PushNotification::StateChange(\n                    StateChange::new(account_id)\n                        .with_change_id(change_id)\n                        .with_change(DataType::Email)\n                        .with_change(DataType::Mailbox)\n                        .with_change(DataType::Thread),\n                ))\n                .await;\n            }\n\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/email/snippet.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    message::metadata::{\n        ArchivedMetadataPartType, DecodedPartContent, MessageMetadata, MetadataHeaderName,\n    },\n};\nuse jmap_proto::{\n    method::{\n        query::Filter,\n        search_snippet::{GetSearchSnippetRequest, GetSearchSnippetResponse, SearchSnippet},\n    },\n    object::email::EmailFilter,\n    request::IntoValid,\n};\nuse mail_parser::decoders::html::html_to_text;\nuse nlp::language::{Language, search_snippet::generate_snippet, stemmer::Stemmer};\nuse std::future::Future;\nuse store::{\n    ValueKey,\n    backend::MAX_TOKEN_LENGTH,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{acl::Acl, collection::Collection, field::EmailField};\nuse utils::chained_bytes::ChainedBytes;\n\npub trait EmailSearchSnippet: Sync + Send {\n    fn email_search_snippet(\n        &self,\n        request: GetSearchSnippetRequest,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetSearchSnippetResponse>> + Send;\n}\n\nimpl EmailSearchSnippet for Server {\n    async fn email_search_snippet(\n        &self,\n        request: GetSearchSnippetRequest,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetSearchSnippetResponse> {\n        let mut filter_stack = vec![];\n        let mut include_term = true;\n        let mut terms = vec![];\n        let mut is_exact = false;\n        let mut language = self.core.jmap.default_language;\n\n        for cond in request.filter {\n            match cond {\n                Filter::Property(cond) => {\n                    if let EmailFilter::Text(text)\n                    | EmailFilter::Subject(text)\n                    | EmailFilter::Body(text) = cond\n                        && include_term\n                    {\n                        let (text, language_) =\n                            Language::detect(text, self.core.jmap.default_language);\n                        language = language_;\n                        if (text.starts_with('\"') && text.ends_with('\"'))\n                            || (text.starts_with('\\'') && text.ends_with('\\''))\n                        {\n                            for token in language.tokenize_text(&text, MAX_TOKEN_LENGTH) {\n                                terms.push(token.word.into_owned());\n                            }\n                            is_exact = true;\n                        } else {\n                            for token in Stemmer::new(&text, language, MAX_TOKEN_LENGTH) {\n                                terms.push(token.word.into_owned());\n                                if let Some(stemmed_word) = token.stemmed_word {\n                                    terms.push(stemmed_word.into_owned());\n                                }\n                            }\n                        }\n                    }\n                }\n                Filter::And | Filter::Or => {\n                    filter_stack.push(cond);\n                }\n                Filter::Not => {\n                    filter_stack.push(cond);\n                    include_term = !include_term;\n                }\n                Filter::Close => {\n                    if matches!(filter_stack.pop(), Some(Filter::Not)) {\n                        include_term = !include_term;\n                    }\n                }\n            }\n        }\n        let account_id = request.account_id.document_id();\n        let cached_messages = self\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n        let document_ids = if access_token.is_member(account_id) {\n            cached_messages.email_document_ids()\n        } else {\n            cached_messages.shared_messages(access_token, Acl::ReadItems)\n        };\n\n        let email_ids = request.email_ids.unwrap();\n        let mut response = GetSearchSnippetResponse {\n            account_id: request.account_id,\n            list: Vec::with_capacity(email_ids.len()),\n            not_found: vec![],\n        };\n\n        if email_ids.len() > self.core.jmap.snippet_max_results {\n            return Err(trc::JmapEvent::RequestTooLarge.into_err());\n        }\n\n        for email_id in email_ids.into_valid() {\n            let document_id = email_id.document_id();\n            let mut snippet = SearchSnippet {\n                email_id,\n                subject: None,\n                preview: None,\n            };\n            if !document_ids.contains(document_id) {\n                response.not_found.push(email_id);\n                continue;\n            } else if terms.is_empty() {\n                response.list.push(snippet);\n                continue;\n            }\n            let metadata_ = match self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                    account_id,\n                    Collection::Email,\n                    document_id,\n                    EmailField::Metadata,\n                ))\n                .await?\n            {\n                Some(metadata) => metadata,\n                None => {\n                    response.not_found.push(email_id);\n                    continue;\n                }\n            };\n            let metadata = metadata_\n                .unarchive::<MessageMetadata>()\n                .caused_by(trc::location!())?;\n\n            // Add subject snippet\n            let contents = &metadata.contents[0];\n            if let Some(subject) = contents\n                .root_part()\n                .header_value(&MetadataHeaderName::Subject)\n                .and_then(|v| v.as_text())\n                .and_then(|v| generate_snippet(v, &terms, language, is_exact))\n            {\n                snippet.subject = subject.into();\n            }\n\n            // Download message\n            let raw_body = if let Some(raw_body) = self\n                .blob_store()\n                .get_blob(metadata.blob_hash.0.as_slice(), 0..usize::MAX)\n                .await?\n            {\n                raw_body\n            } else {\n                trc::event!(\n                    Store(trc::StoreEvent::NotFound),\n                    AccountId = account_id,\n                    DocumentId = email_id.document_id(),\n                    Collection = Collection::Email,\n                    BlobId = metadata.blob_hash.0.as_slice(),\n                    Details = \"Blob not found.\",\n                    CausedBy = trc::location!(),\n                );\n\n                response.not_found.push(email_id);\n                continue;\n            };\n            let raw_message = ChainedBytes::new(metadata.raw_headers.as_ref()).with_last(\n                raw_body\n                    .get(metadata.blob_body_offset.to_native() as usize..)\n                    .unwrap_or_default(),\n            );\n\n            // Find a matching part\n            'outer: for part in contents.parts.iter() {\n                match &part.body {\n                    ArchivedMetadataPartType::Text => {\n                        let text = match part.decode_contents(&raw_message) {\n                            DecodedPartContent::Text(text) => text,\n                            _ => unreachable!(),\n                        };\n\n                        if let Some(body) = generate_snippet(&text, &terms, language, is_exact) {\n                            snippet.preview = body.into();\n                            break;\n                        }\n                    }\n                    ArchivedMetadataPartType::Html => {\n                        let text = match part.decode_contents(&raw_message) {\n                            DecodedPartContent::Text(html) => html_to_text(&html),\n                            _ => unreachable!(),\n                        };\n\n                        if let Some(body) = generate_snippet(&text, &terms, language, is_exact) {\n                            snippet.preview = body.into();\n                            break;\n                        }\n                    }\n                    ArchivedMetadataPartType::Message(message) => {\n                        for part in metadata.contents[u16::from(message) as usize].parts.iter() {\n                            if let ArchivedMetadataPartType::Text | ArchivedMetadataPartType::Html =\n                                part.body\n                            {\n                                let text = match (part.decode_contents(&raw_message), &part.body) {\n                                    (\n                                        DecodedPartContent::Text(text),\n                                        ArchivedMetadataPartType::Text,\n                                    ) => text,\n                                    (\n                                        DecodedPartContent::Text(html),\n                                        ArchivedMetadataPartType::Html,\n                                    ) => html_to_text(&html).into(),\n                                    _ => unreachable!(),\n                                };\n\n                                if let Some(body) =\n                                    generate_snippet(&text, &terms, language, is_exact)\n                                {\n                                    snippet.preview = body.into();\n                                    break 'outer;\n                                }\n                            }\n                        }\n                    }\n                    _ => (),\n                }\n            }\n            //}\n\n            response.list.push(snippet);\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/file/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::acl::JmapRights, changes::state::JmapCacheState};\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse groupware::{cache::GroupwareCache, file::FileNode};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::file_node::{self, FileNodeProperty, FileNodeValue},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Map, Value};\nuse store::{ValueKey, roaring::RoaringBitmap, write::{AlignedBytes, Archive, now}};\nuse trc::AddContext;\nuse types::{\n    acl::{Acl, AclGrant},\n    blob::{BlobClass, BlobId},\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection},\n};\n\npub trait FileNodeGet: Sync + Send {\n    fn file_node_get(\n        &self,\n        request: GetRequest<file_node::FileNode>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<file_node::FileNode>>> + Send;\n}\n\nimpl FileNodeGet for Server {\n    async fn file_node_get(\n        &self,\n        mut request: GetRequest<file_node::FileNode>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<file_node::FileNode>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            FileNodeProperty::Id,\n            FileNodeProperty::Name,\n            FileNodeProperty::ParentId,\n            FileNodeProperty::Size,\n        ]);\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await?;\n        let file_node_ids = if access_token.is_member(account_id) {\n            cache\n                .resources\n                .iter()\n                .map(|r| r.document_id)\n                .collect::<RoaringBitmap>()\n        } else {\n            cache.shared_containers(access_token, [Acl::Read, Acl::ReadItems], true)\n        };\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            file_node_ids\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: cache.get_state(true).into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the file_node object\n            let document_id = id.document_id();\n            if !file_node_ids.contains(document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n            let _file_node = if let Some(file_node) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::FileNode,\n                    document_id,\n                ))\n                .await?\n            {\n                file_node\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n            let file_node = _file_node\n                .unarchive::<FileNode>()\n                .caused_by(trc::location!())?;\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                match property {\n                    FileNodeProperty::Id => {\n                        result.insert_unchecked(FileNodeProperty::Id, FileNodeValue::Id(id));\n                    }\n                    FileNodeProperty::Name => {\n                        result.insert_unchecked(FileNodeProperty::Name, file_node.name.to_string());\n                    }\n                    FileNodeProperty::ShareWith => {\n                        result.insert_unchecked(\n                            FileNodeProperty::ShareWith,\n                            JmapRights::share_with::<file_node::FileNode>(\n                                account_id,\n                                access_token,\n                                &file_node\n                                    .acls\n                                    .iter()\n                                    .map(AclGrant::from)\n                                    .collect::<Vec<_>>(),\n                            ),\n                        );\n                    }\n                    FileNodeProperty::MyRights => {\n                        result.insert_unchecked(\n                            FileNodeProperty::MyRights,\n                            if access_token.is_shared(account_id) {\n                                JmapRights::rights::<file_node::FileNode>(\n                                    file_node.acls.effective_acl(access_token),\n                                )\n                            } else {\n                                JmapRights::all_rights::<file_node::FileNode>()\n                            },\n                        );\n                    }\n                    FileNodeProperty::ParentId => {\n                        let parent_id = file_node.parent_id.to_native();\n\n                        result.insert_unchecked(\n                            FileNodeProperty::ParentId,\n                            if parent_id > 0 {\n                                Value::Element(FileNodeValue::Id((parent_id - 1).into()))\n                            } else {\n                                Value::Null\n                            },\n                        );\n                    }\n                    FileNodeProperty::BlobId => {\n                        result.insert_unchecked(\n                            FileNodeProperty::BlobId,\n                            if let Some(file) = file_node.file.as_ref() {\n                                Value::Element(FileNodeValue::BlobId(BlobId::new(\n                                    BlobHash::from(&file.blob_hash),\n                                    BlobClass::Linked {\n                                        account_id,\n                                        collection: Collection::FileNode.into(),\n                                        document_id: id.document_id(),\n                                    },\n                                )))\n                            } else {\n                                Value::Null\n                            },\n                        );\n                    }\n                    FileNodeProperty::Size => {\n                        result.insert_unchecked(\n                            FileNodeProperty::Size,\n                            if let Some(file) = file_node.file.as_ref() {\n                                Value::Number(file.size.to_native().into())\n                            } else {\n                                Value::Null\n                            },\n                        );\n                    }\n                    FileNodeProperty::Type => {\n                        result.insert_unchecked(\n                            FileNodeProperty::Type,\n                            if let Some(file) =\n                                file_node.file.as_ref().and_then(|f| f.media_type.as_ref())\n                            {\n                                Value::Str(file.to_string().into())\n                            } else {\n                                Value::Null\n                            },\n                        );\n                    }\n                    FileNodeProperty::Executable => {\n                        result.insert_unchecked(\n                            FileNodeProperty::Executable,\n                            if let Some(file) = file_node.file.as_ref() {\n                                Value::Bool(file.executable)\n                            } else {\n                                Value::Null\n                            },\n                        );\n                    }\n                    FileNodeProperty::Created => {\n                        result.insert_unchecked(\n                            FileNodeProperty::Created,\n                            Value::Element(FileNodeValue::Date(UTCDate::from_timestamp(\n                                file_node.created.to_native(),\n                            ))),\n                        );\n                    }\n                    FileNodeProperty::Modified => {\n                        result.insert_unchecked(\n                            FileNodeProperty::Modified,\n                            Value::Element(FileNodeValue::Date(UTCDate::from_timestamp(\n                                file_node.modified.to_native(),\n                            ))),\n                        );\n                    }\n                    FileNodeProperty::Accessed => {\n                        result.insert_unchecked(\n                            FileNodeProperty::Accessed,\n                            Value::Element(FileNodeValue::Date(UTCDate::from_timestamp(\n                                now() as i64\n                            ))),\n                        );\n                    }\n                    FileNodeProperty::IsSubscribed => {\n                        result.insert_unchecked(FileNodeProperty::IsSubscribed, Value::Bool(true));\n                    }\n                    property => {\n                        result.insert_unchecked(property.clone(), Value::Null);\n                    }\n                }\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/file/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/file/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::JmapCacheState};\nuse common::{Server, auth::AccessToken};\nuse groupware::cache::GroupwareCache;\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::file_node::{FileNode, FileNodeFilter},\n    request::MaybeInvalid,\n};\nuse store::{\n    roaring::RoaringBitmap,\n    search::{SearchFilter, SearchQuery},\n    write::SearchIndex,\n};\nuse types::{acl::Acl, collection::SyncCollection};\n\npub trait FileNodeQuery: Sync + Send {\n    fn file_node_query(\n        &self,\n        request: QueryRequest<FileNode>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl FileNodeQuery for Server {\n    async fn file_node_query(\n        &self,\n        mut request: QueryRequest<FileNode>,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let mut filters = Vec::with_capacity(request.filter.len());\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await?;\n\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    FileNodeFilter::AncestorId(MaybeInvalid::Value(id)) => {\n                        if let Some(resource) =\n                            cache.container_resource_path_by_id(id.document_id())\n                        {\n                            filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                                cache.subtree(resource.path()).map(|r| r.document_id()),\n                            )))\n                        } else {\n                            filters.push(SearchFilter::is_in_set(RoaringBitmap::new()));\n                        }\n                    }\n                    FileNodeFilter::ParentId(MaybeInvalid::Value(id)) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.children_ids(id.document_id()),\n                        )));\n                    }\n                    FileNodeFilter::HasParentId(has_parent_id) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.resources.iter().filter_map(|r| {\n                                if has_parent_id == r.parent_id().is_some() {\n                                    Some(r.document_id)\n                                } else {\n                                    None\n                                }\n                            }),\n                        )));\n                    }\n                    FileNodeFilter::Name(name) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.resources.iter().filter_map(|r| {\n                                if r.container_name().is_some_and(|n| n == name) {\n                                    Some(r.document_id)\n                                } else {\n                                    None\n                                }\n                            }),\n                        )));\n                    }\n                    FileNodeFilter::NameMatch(name) => {\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.resources.iter().filter_map(|r| {\n                                if r.container_name().is_some_and(|n| name.matches(n)) {\n                                    Some(r.document_id)\n                                } else {\n                                    None\n                                }\n                            }),\n                        )));\n                    }\n                    FileNodeFilter::MinSize(size) => {\n                        let size = size as u32;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.resources.iter().filter_map(|r| {\n                                if r.size().is_some_and(|s| s >= size) {\n                                    Some(r.document_id)\n                                } else {\n                                    None\n                                }\n                            }),\n                        )));\n                    }\n                    FileNodeFilter::MaxSize(size) => {\n                        let size = size as u32;\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            cache.resources.iter().filter_map(|r| {\n                                if r.size().is_some_and(|s| s <= size) {\n                                    Some(r.document_id)\n                                } else {\n                                    None\n                                }\n                            }),\n                        )));\n                    }\n                    unsupported => {\n                        return Err(trc::JmapEvent::UnsupportedFilter\n                            .into_err()\n                            .details(unsupported.into_string()));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        if request.sort.as_ref().is_some_and(|s| !s.is_empty()) {\n            return Err(trc::JmapEvent::UnsupportedSort\n                .into_err()\n                .details(\"Sorting is not supported on FileNode\"));\n        }\n\n        let results = SearchQuery::new(SearchIndex::InMemory)\n            .with_filters(filters)\n            .with_mask(if access_token.is_shared(account_id) {\n                cache.shared_containers(access_token, [Acl::ReadItems], true)\n            } else {\n                cache.document_ids(false).collect()\n            })\n            .filter()\n            .into_bitmap();\n\n        let mut response = QueryResponseBuilder::new(\n            results.len() as usize,\n            self.core.jmap.query_max_results,\n            cache.get_state(false),\n            &request,\n        );\n\n        for document_id in results {\n            if !response.add(0, document_id) {\n                break;\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/file/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    api::acl::{JmapAcl, JmapRights},\n    blob::download::BlobDownload,\n};\nuse common::{DavResourceMetadata, DavResources, Server, auth::AccessToken, sharing::EffectiveAcl};\nuse groupware::{DestroyArchive, cache::GroupwareCache, file::FileNode};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::SetError,\n    method::set::{SetRequest, SetResponse},\n    object::file_node::{self, FileNodeProperty, FileNodeValue},\n    references::resolve::ResolveCreatedReference,\n    request::IntoValid,\n    types::state::State,\n};\nuse jmap_tools::{JsonPointerItem, Key, Value};\nuse store::{\n    ValueKey,\n    ahash::{AHashMap, AHashSet},\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    acl::{Acl, AclGrant},\n    blob::BlobId,\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub trait FileNodeSet: Sync + Send {\n    fn file_node_set(\n        &self,\n        request: SetRequest<'_, file_node::FileNode>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<SetResponse<file_node::FileNode>>> + Send;\n}\n\nimpl FileNodeSet for Server {\n    async fn file_node_set(\n        &self,\n        mut request: SetRequest<'_, file_node::FileNode>,\n        access_token: &AccessToken,\n        _session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<file_node::FileNode>> {\n        let account_id = request.account_id.document_id();\n        let cache = self\n            .fetch_dav_resources(access_token, account_id, SyncCollection::FileNode)\n            .await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n        let is_shared = access_token.is_shared(account_id);\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        let mut created_folders = AHashMap::new();\n        'create: for (id, object) in request.unwrap_create() {\n            let mut file_node = FileNode::default();\n\n            // Process changes\n            let has_acl_changes = match update_file_node(object, &mut file_node, &mut response) {\n                Ok(result) => {\n                    if let Some(blob_id) = result.blob_id {\n                        let file_details = file_node.file.get_or_insert_default();\n                        if !self.has_access_blob(&blob_id, access_token).await? {\n                            response.not_created.append(\n                                id,\n                                SetError::forbidden().with_description(format!(\n                                    \"You do not have access to blobId {blob_id}.\"\n                                )),\n                            );\n                            continue;\n                        } else if let Some(blob_contents) = self\n                            .blob_store()\n                            .get_blob(blob_id.hash.as_slice(), 0..usize::MAX)\n                            .await?\n                        {\n                            file_details.size = blob_contents.len() as u32;\n                        } else {\n                            response.not_created.append(\n                                id,\n                                SetError::invalid_properties()\n                                    .with_property(FileNodeProperty::BlobId)\n                                    .with_description(\"Blob could not be found.\"),\n                            );\n                            continue 'create;\n                        }\n\n                        file_details.blob_hash = blob_id.hash;\n                    }\n\n                    // Validate blob hash\n                    if file_node\n                        .file\n                        .as_ref()\n                        .is_some_and(|f| f.blob_hash.is_empty())\n                    {\n                        response.not_created.append(\n                            id,\n                            SetError::invalid_properties()\n                                .with_property(FileNodeProperty::BlobId)\n                                .with_description(\"Missing blob id.\"),\n                        );\n                        continue 'create;\n                    }\n\n                    result.has_acl_changes\n                }\n                Err(err) => {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                }\n            };\n\n            // Validate hierarchy\n            if let Err(err) =\n                validate_file_node_hierarchy(None, &file_node, is_shared, &cache, &created_folders)\n            {\n                response.not_created.append(id, err);\n                continue 'create;\n            }\n\n            // Inherit ACLs from parent\n            if file_node.parent_id > 0 {\n                let parent_id = file_node.parent_id - 1;\n                let parent_acls = created_folders.get(&parent_id).cloned().or_else(|| {\n                    cache\n                        .container_resource_by_id(parent_id)\n                        .and_then(|r| r.acls())\n                        .map(|a| a.to_vec())\n                });\n                if !has_acl_changes {\n                    if let Some(parent_acls) = parent_acls {\n                        file_node.acls = parent_acls;\n                    }\n                } else if is_shared\n                    && parent_acls\n                        .is_none_or(|acls| !acls.effective_acl(access_token).contains(Acl::Share))\n                {\n                    response.not_created.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to share this file node.\"),\n                    );\n                    continue 'create;\n                }\n            }\n\n            // Validate ACLs\n            if !file_node.acls.is_empty() {\n                if let Err(err) = self.acl_validate(&file_node.acls).await {\n                    response.not_created.append(id, err.into());\n                    continue 'create;\n                }\n\n                self.refresh_acls(&file_node.acls, None).await;\n            }\n\n            // Insert record\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::FileNode, 1)\n                .await\n                .caused_by(trc::location!())?;\n            if file_node.file.is_none() {\n                created_folders.insert(document_id, file_node.acls.clone());\n            }\n            file_node\n                .insert(access_token, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?;\n            response.created(id, document_id);\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain file node\n            let document_id = id.document_id();\n            let file_node_ = if let Some(file_node_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::FileNode,\n                    document_id,\n                ))\n                .await?\n            {\n                file_node_\n            } else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n            let file_node = file_node_\n                .to_unarchived::<FileNode>()\n                .caused_by(trc::location!())?;\n            let mut new_file_node = file_node\n                .deserialize::<FileNode>()\n                .caused_by(trc::location!())?;\n\n            // Apply changes\n            let has_acl_changes = match update_file_node(object, &mut new_file_node, &mut response)\n            {\n                Ok(result) => {\n                    if let Some(blob_id) = result.blob_id {\n                        let file_details = new_file_node.file.get_or_insert_default();\n                        if !self.has_access_blob(&blob_id, access_token).await? {\n                            response.not_updated.append(\n                                id,\n                                SetError::forbidden().with_description(format!(\n                                    \"You do not have access to blobId {blob_id}.\"\n                                )),\n                            );\n                            continue;\n                        } else if let Some(blob_contents) = self\n                            .blob_store()\n                            .get_blob(blob_id.hash.as_slice(), 0..usize::MAX)\n                            .await?\n                        {\n                            file_details.size = blob_contents.len() as u32;\n                        } else {\n                            response.not_updated.append(\n                                id,\n                                SetError::invalid_properties()\n                                    .with_property(FileNodeProperty::BlobId)\n                                    .with_description(\"Blob could not be found.\"),\n                            );\n                            continue 'update;\n                        }\n\n                        file_details.blob_hash = blob_id.hash;\n                    }\n\n                    result.has_acl_changes\n                }\n                Err(err) => {\n                    response.not_updated.append(id, err);\n                    continue 'update;\n                }\n            };\n\n            // Validate hierarchy\n            if let Err(err) = validate_file_node_hierarchy(\n                Some(document_id),\n                &new_file_node,\n                is_shared,\n                &cache,\n                &created_folders,\n            ) {\n                response.not_updated.append(id, err);\n                continue 'update;\n            }\n\n            // Validate ACL\n            if is_shared {\n                let acl = file_node.inner.acls.effective_acl(access_token);\n                if !acl.contains(Acl::Modify) || (has_acl_changes && !acl.contains(Acl::Share)) {\n                    response.not_updated.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to modify this file node.\"),\n                    );\n                    continue 'update;\n                }\n            }\n            if has_acl_changes {\n                if let Err(err) = self.acl_validate(&new_file_node.acls).await {\n                    response.not_updated.append(id, err.into());\n                    continue 'update;\n                }\n                self.refresh_acls(\n                    &new_file_node.acls,\n                    Some(\n                        file_node\n                            .inner\n                            .acls\n                            .iter()\n                            .map(AclGrant::from)\n                            .collect::<Vec<_>>()\n                            .as_slice(),\n                    ),\n                )\n                .await;\n            }\n\n            // Update record\n            new_file_node\n                .update(access_token, file_node, account_id, document_id, &mut batch)\n                .caused_by(trc::location!())?;\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        let on_destroy_remove_children = request\n            .arguments\n            .on_destroy_remove_children\n            .unwrap_or(false);\n        let mut destroy_ids = AHashSet::with_capacity(will_destroy.len());\n        'destroy: for id in will_destroy {\n            let document_id = id.document_id();\n\n            let Some(file_node) = cache.any_resource_path_by_id(document_id) else {\n                response.not_destroyed.append(id, SetError::not_found());\n                continue 'destroy;\n            };\n\n            // Find ids to delete\n            let mut ids = cache.subtree(file_node.path()).collect::<Vec<_>>();\n            if ids.is_empty() {\n                debug_assert!(false, \"Resource found in cache but not in subtree\");\n                continue 'destroy;\n            }\n\n            // Sort ids descending from the deepest to the root\n            ids.sort_unstable_by_key(|b| std::cmp::Reverse(b.hierarchy_seq()));\n            let mut sorted_ids = Vec::with_capacity(ids.len());\n            sorted_ids.extend(ids.into_iter().map(|a| a.document_id()));\n\n            // Validate not already deleted\n            for child_id in &sorted_ids {\n                if !destroy_ids.insert(*child_id) {\n                    response.not_destroyed.append(\n                        id,\n                        SetError::will_destroy().with_description(\n                            \"File node or one of its children is already marked for deletion.\",\n                        ),\n                    );\n                    continue 'destroy;\n                }\n            }\n\n            // Validate ACLs\n            if !access_token.is_member(account_id) {\n                let permissions = cache.shared_containers(access_token, [Acl::Delete], false);\n                if permissions.len() < sorted_ids.len() as u64\n                    || !sorted_ids.iter().all(|id| permissions.contains(*id))\n                {\n                    response.not_destroyed.append(\n                        id,\n                        SetError::forbidden()\n                            .with_description(\"You are not allowed to delete this file node.\"),\n                    );\n                    continue 'destroy;\n                }\n            }\n\n            // Obtain children ids\n            if sorted_ids.len() > 1 && !on_destroy_remove_children {\n                response\n                    .not_destroyed\n                    .append(id, SetError::node_has_children());\n                continue 'destroy;\n            }\n\n            // Delete record\n            response\n                .destroyed\n                .extend(sorted_ids.iter().copied().map(Id::from));\n\n            DestroyArchive(sorted_ids)\n                .delete_batch(\n                    self,\n                    access_token,\n                    account_id,\n                    cache.format_resource(file_node).into(),\n                    &mut batch,\n                )\n                .await?;\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n}\n\nstruct UpdateResult {\n    has_acl_changes: bool,\n    blob_id: Option<BlobId>,\n}\n\nfn update_file_node(\n    updates: Value<'_, FileNodeProperty, FileNodeValue>,\n    file_node: &mut FileNode,\n    response: &mut SetResponse<file_node::FileNode>,\n) -> Result<UpdateResult, SetError<FileNodeProperty>> {\n    let mut has_acl_changes = false;\n    let mut blob_id = None;\n\n    for (property, mut value) in updates.into_expanded_object() {\n        let Key::Property(property) = property else {\n            return Err(SetError::invalid_properties()\n                .with_property(property.to_owned())\n                .with_description(\"Invalid property.\"));\n        };\n\n        response.resolve_self_references(&mut value)?;\n\n        match (property, value) {\n            (FileNodeProperty::Name, Value::Str(value))\n                if (1..=255).contains(&value.len())\n                    && !value.contains('/')\n                    && ![\".\", \"..\"].contains(&value.as_ref()) =>\n            {\n                file_node.name = value.into_owned();\n            }\n            (FileNodeProperty::ParentId, Value::Element(FileNodeValue::Id(value))) => {\n                file_node.parent_id = value.document_id() + 1;\n            }\n            (FileNodeProperty::ParentId, Value::Null) => {\n                file_node.parent_id = 0;\n            }\n            (FileNodeProperty::BlobId, Value::Element(FileNodeValue::BlobId(value))) => {\n                if file_node\n                    .file\n                    .as_ref()\n                    .is_none_or(|f| f.blob_hash != value.hash)\n                {\n                    blob_id = Some(value);\n                }\n            }\n            (FileNodeProperty::BlobId, Value::Null) => {}\n            (FileNodeProperty::Size, Value::Number(value)) => {\n                file_node.file.get_or_insert_default().size = value.cast_to_u64() as u32;\n            }\n            (FileNodeProperty::Type, Value::Str(value)) if (1..=30).contains(&value.len()) => {\n                file_node.file.get_or_insert_default().media_type = value.into_owned().into();\n            }\n            (FileNodeProperty::Type, Value::Null) => {\n                file_node.file.get_or_insert_default().media_type = None;\n            }\n            (FileNodeProperty::Executable, Value::Bool(value)) => {\n                file_node.file.get_or_insert_default().executable = value;\n            }\n            (FileNodeProperty::Executable, Value::Null) => {\n                file_node.file.get_or_insert_default().executable = false;\n            }\n            (FileNodeProperty::Created, Value::Element(FileNodeValue::Date(value))) => {\n                file_node.created = value.timestamp();\n            }\n            (FileNodeProperty::Modified, Value::Element(FileNodeValue::Date(value))) => {\n                file_node.modified = value.timestamp();\n            }\n            (FileNodeProperty::ShareWith, value) => {\n                file_node.acls = JmapRights::acl_set::<file_node::FileNode>(value)?;\n                has_acl_changes = true;\n            }\n            (FileNodeProperty::Pointer(pointer), value)\n                if matches!(\n                    pointer.first(),\n                    Some(JsonPointerItem::Key(Key::Property(\n                        FileNodeProperty::ShareWith\n                    )))\n                ) =>\n            {\n                let mut pointer = pointer.iter();\n                pointer.next();\n\n                file_node.acls = JmapRights::acl_patch::<file_node::FileNode>(\n                    std::mem::take(&mut file_node.acls),\n                    pointer,\n                    value,\n                )?;\n                has_acl_changes = true;\n            }\n            (property, _) => {\n                return Err(SetError::invalid_properties()\n                    .with_property(property.clone())\n                    .with_description(\"Field could not be set.\"));\n            }\n        }\n    }\n\n    // Validate name\n    if file_node.name.is_empty() {\n        return Err(SetError::invalid_properties()\n            .with_property(FileNodeProperty::Name)\n            .with_description(\"Missing name.\"));\n    }\n\n    Ok(UpdateResult {\n        has_acl_changes,\n        blob_id,\n    })\n}\n\nfn validate_file_node_hierarchy(\n    document_id: Option<u32>,\n    node: &FileNode,\n    is_shared: bool,\n    cache: &DavResources,\n    created_folders: &AHashMap<u32, Vec<AclGrant>>,\n) -> Result<(), SetError<FileNodeProperty>> {\n    let node_parent_id = if node.parent_id == 0 {\n        if is_shared && document_id.is_none() {\n            return Err(SetError::invalid_properties()\n                .with_property(FileNodeProperty::ParentId)\n                .with_description(\"Cannot create top-level folder in a shared account.\"));\n        }\n        None\n    } else {\n        let parent_id = node.parent_id - 1;\n\n        if let Some(document_id) = document_id {\n            if document_id == parent_id {\n                return Err(SetError::invalid_properties()\n                    .with_property(FileNodeProperty::ParentId)\n                    .with_description(\"A file node cannot be its own parent.\"));\n            }\n\n            // Validate circular references\n            if let Some(file) = cache.container_resource_path_by_id(document_id)\n                && cache\n                    .subtree(file.path())\n                    .any(|r| r.document_id() == parent_id)\n            {\n                return Err(SetError::invalid_properties()\n                    .with_property(FileNodeProperty::ParentId)\n                    .with_description(\"Circular reference in parent ids.\"));\n            }\n        }\n\n        // Make sure the parent is a container\n        if !created_folders.contains_key(&parent_id)\n            && cache.container_resource_by_id(parent_id).is_none()\n        {\n            return Err(SetError::invalid_properties()\n                .with_property(FileNodeProperty::ParentId)\n                .with_description(\"Parent ID does not exist or is not a folder.\"));\n        }\n\n        Some(parent_id)\n    };\n\n    // Validate name uniqueness\n    for resource in &cache.resources {\n        if let DavResourceMetadata::File {\n            name, parent_id, ..\n        } = &resource.data\n            && document_id.is_none_or(|id| id != resource.document_id)\n            && node_parent_id == *parent_id\n            && node.name == *name\n        {\n            return Err(SetError::invalid_properties()\n                .with_property(FileNodeProperty::Name)\n                .with_description(\"A node with the same name already exists in this folder.\"));\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/jmap/src/identity/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::changes::state::StateManager;\nuse common::{Server, storage::index::ObjectIndexBuilder};\nuse directory::{PrincipalData, QueryParams};\nuse email::identity::{ArchivedEmailAddress, Identity};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::identity::{self, IdentityProperty, IdentityValue},\n};\nuse jmap_tools::{Map, Value};\nuse std::future::Future;\nuse store::{\n    ValueKey, rkyv::{option::ArchivedOption, vec::ArchivedVec}, roaring::RoaringBitmap, write::{AlignedBytes, Archive, BatchBuilder}\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::IdentityField,\n};\nuse utils::sanitize_email;\n\npub trait IdentityGet: Sync + Send {\n    fn identity_get(\n        &self,\n        request: GetRequest<identity::Identity>,\n    ) -> impl Future<Output = trc::Result<GetResponse<identity::Identity>>> + Send;\n\n    fn identity_get_or_create(\n        &self,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<RoaringBitmap>> + Send;\n}\n\nimpl IdentityGet for Server {\n    async fn identity_get(\n        &self,\n        mut request: GetRequest<identity::Identity>,\n    ) -> trc::Result<GetResponse<identity::Identity>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            IdentityProperty::Id,\n            IdentityProperty::Name,\n            IdentityProperty::Email,\n            IdentityProperty::ReplyTo,\n            IdentityProperty::Bcc,\n            IdentityProperty::TextSignature,\n            IdentityProperty::HtmlSignature,\n            IdentityProperty::MayDelete,\n        ]);\n        let account_id = request.account_id.document_id();\n        let identity_ids = self.identity_get_or_create(account_id).await?;\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            identity_ids\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: self\n                .get_state(account_id, SyncCollection::Identity)\n                .await?\n                .into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the identity object\n            let document_id = id.document_id();\n            if !identity_ids.contains(document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n            let _identity = if let Some(identity) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Identity,\n                    document_id,\n                ))\n                .await?\n            {\n                identity\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n            let identity = _identity\n                .unarchive::<Identity>()\n                .caused_by(trc::location!())?;\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                match property {\n                    IdentityProperty::Id => {\n                        result.insert_unchecked(IdentityProperty::Id, IdentityValue::Id(id));\n                    }\n                    IdentityProperty::MayDelete => {\n                        result.insert_unchecked(IdentityProperty::MayDelete, Value::Bool(true));\n                    }\n                    IdentityProperty::Name => {\n                        result.insert_unchecked(IdentityProperty::Name, identity.name.to_string());\n                    }\n                    IdentityProperty::Email => {\n                        result\n                            .insert_unchecked(IdentityProperty::Email, identity.email.to_string());\n                    }\n                    IdentityProperty::TextSignature => {\n                        result.insert_unchecked(\n                            IdentityProperty::TextSignature,\n                            identity.text_signature.to_string(),\n                        );\n                    }\n                    IdentityProperty::HtmlSignature => {\n                        result.insert_unchecked(\n                            IdentityProperty::HtmlSignature,\n                            identity.html_signature.to_string(),\n                        );\n                    }\n                    IdentityProperty::Bcc => {\n                        result\n                            .insert_unchecked(IdentityProperty::Bcc, email_to_value(&identity.bcc));\n                    }\n                    IdentityProperty::ReplyTo => {\n                        result.insert_unchecked(\n                            IdentityProperty::ReplyTo,\n                            email_to_value(&identity.reply_to),\n                        );\n                    }\n                    property => {\n                        result.insert_unchecked(property.clone(), Value::Null);\n                    }\n                }\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n\n    async fn identity_get_or_create(&self, account_id: u32) -> trc::Result<RoaringBitmap> {\n        let mut identity_ids = self\n            .document_ids(account_id, Collection::Identity, IdentityField::DocumentId)\n            .await?;\n        if !identity_ids.is_empty() {\n            return Ok(identity_ids);\n        }\n\n        // Obtain principal\n        let principal = if let Some(principal) = self\n            .core\n            .storage\n            .directory\n            .query(QueryParams::id(account_id).with_return_member_of(false))\n            .await\n            .caused_by(trc::location!())?\n        {\n            principal\n        } else {\n            return Ok(identity_ids);\n        };\n\n        let mut emails = Vec::new();\n        let mut description = None;\n        for data in principal.data {\n            match data {\n                PrincipalData::PrimaryEmail(v) | PrincipalData::EmailAlias(v) => emails.push(v),\n                PrincipalData::Description(v) => description = Some(v),\n                _ => {}\n            }\n        }\n\n        let num_emails = emails.len();\n        if num_emails == 0 {\n            return Ok(identity_ids);\n        }\n\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Identity);\n\n        // Create identities\n        let name = description.unwrap_or(principal.name);\n        let mut next_document_id = self\n            .store()\n            .assign_document_ids(account_id, Collection::Identity, num_emails as u64)\n            .await\n            .caused_by(trc::location!())?;\n        for email in &emails {\n            let email = sanitize_email(email).unwrap_or_default();\n            if email.is_empty() || email.starts_with('@') {\n                continue;\n            }\n            let name = if name.is_empty() {\n                email.clone()\n            } else {\n                name.clone()\n            };\n            let document_id = next_document_id;\n            next_document_id -= 1;\n            batch\n                .with_document(document_id)\n                .tag(IdentityField::DocumentId)\n                .custom(ObjectIndexBuilder::<(), _>::new().with_changes(Identity {\n                    name,\n                    email,\n                    ..Default::default()\n                }))\n                .caused_by(trc::location!())?;\n            identity_ids.insert(document_id);\n        }\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n        Ok(identity_ids)\n    }\n}\n\nfn email_to_value(\n    email: &ArchivedOption<ArchivedVec<ArchivedEmailAddress>>,\n) -> Value<'static, IdentityProperty, IdentityValue> {\n    if let ArchivedOption::Some(email) = email {\n        Value::Array(\n            email\n                .iter()\n                .map(|email| {\n                    Value::Object(\n                        Map::with_capacity(2)\n                            .with_key_value(IdentityProperty::Name, &email.name)\n                            .with_key_value(IdentityProperty::Email, &email.email),\n                    )\n                })\n                .collect(),\n        )\n    } else {\n        Value::Null\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/identity/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/identity/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken, storage::index::ObjectIndexBuilder};\nuse directory::QueryParams;\nuse email::identity::{EmailAddress, Identity};\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::identity::{self, IdentityProperty, IdentityValue},\n    references::resolve::ResolveCreatedReference,\n    request::IntoValid,\n    types::state::State,\n};\nuse jmap_tools::{Key, Value};\nuse std::future::Future;\nuse store::{ValueKey, write::{AlignedBytes, Archive, BatchBuilder}};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::{Field, IdentityField},\n};\nuse utils::sanitize_email;\n\npub trait IdentitySet: Sync + Send {\n    fn identity_set(\n        &self,\n        request: SetRequest<'_, identity::Identity>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<SetResponse<identity::Identity>>> + Send;\n}\n\nimpl IdentitySet for Server {\n    async fn identity_set(\n        &self,\n        mut request: SetRequest<'_, identity::Identity>,\n        access_token: &AccessToken,\n    ) -> trc::Result<SetResponse<identity::Identity>> {\n        let account_id = request.account_id.document_id();\n        let identity_ids = self\n            .document_ids(account_id, Collection::Identity, IdentityField::DocumentId)\n            .await?;\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        'create: for (id, object) in request.unwrap_create() {\n            let mut identity = Identity::default();\n\n            for (property, mut value) in object.into_expanded_object() {\n                if let Err(err) = response\n                    .resolve_self_references(&mut value)\n                    .and_then(|_| validate_identity_value(&property, value, &mut identity, true))\n                {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                }\n            }\n\n            // Validate email address\n            if !identity.email.is_empty() {\n                if self\n                    .directory()\n                    .query(QueryParams::id(account_id).with_return_member_of(false))\n                    .await?\n                    .is_none_or(|p| !p.email_addresses().any(|e| e == identity.email))\n                {\n                    response.not_created.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(IdentityProperty::Email)\n                            .with_description(\n                                \"E-mail address not configured for this account.\".to_string(),\n                            ),\n                    );\n                    continue 'create;\n                }\n            } else {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(IdentityProperty::Email)\n                        .with_description(\"Missing e-mail address.\"),\n                );\n                continue 'create;\n            }\n\n            // Validate quota\n            if identity_ids.len() >= access_token.object_quota(Collection::Identity) as u64 {\n                response.not_created.append(\n                    id,\n                    SetError::new(SetErrorType::OverQuota).with_description(concat!(\n                        \"There are too many identities, \",\n                        \"please delete some before adding a new one.\"\n                    )),\n                );\n                continue 'create;\n            }\n\n            // Insert record\n            let document_id = self\n                .store()\n                .assign_document_ids(account_id, Collection::Identity, 1)\n                .await\n                .caused_by(trc::location!())?;\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Identity)\n                .with_document(document_id)\n                .tag(IdentityField::DocumentId)\n                .custom(ObjectIndexBuilder::<(), _>::new().with_changes(identity))\n                .caused_by(trc::location!())?\n                .commit_point();\n            response.created(id, document_id);\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain identity\n            let document_id = id.document_id();\n            let identity_ = if let Some(identity_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Identity,\n                    document_id,\n                ))\n                .await?\n            {\n                identity_\n            } else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n            let identity = identity_\n                .to_unarchived::<Identity>()\n                .caused_by(trc::location!())?;\n            let mut new_identity = identity\n                .deserialize::<Identity>()\n                .caused_by(trc::location!())?;\n\n            for (property, mut value) in object.into_expanded_object() {\n                if let Err(err) = response.resolve_self_references(&mut value).and_then(|_| {\n                    validate_identity_value(&property, value, &mut new_identity, false)\n                }) {\n                    response.not_updated.append(id, err);\n                    continue 'update;\n                }\n            }\n\n            // Update record\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Identity)\n                .with_document(document_id)\n                .custom(\n                    ObjectIndexBuilder::new()\n                        .with_current(identity)\n                        .with_changes(new_identity),\n                )\n                .caused_by(trc::location!())?\n                .commit_point();\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        for id in will_destroy {\n            let document_id = id.document_id();\n            if identity_ids.contains(document_id) {\n                // Update record\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Identity)\n                    .with_document(document_id)\n                    .untag(IdentityField::DocumentId)\n                    .clear(Field::ARCHIVE)\n                    .log_item_delete(SyncCollection::Identity, None)\n                    .commit_point();\n                response.destroyed.push(id);\n            } else {\n                response.not_destroyed.append(id, SetError::not_found());\n            }\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(response)\n    }\n}\n\nfn validate_identity_value(\n    property: &Key<'_, IdentityProperty>,\n    value: Value<'_, IdentityProperty, IdentityValue>,\n    identity: &mut Identity,\n    is_create: bool,\n) -> Result<(), SetError<IdentityProperty>> {\n    let Key::Property(property) = property else {\n        return Err(SetError::invalid_properties()\n            .with_property(property.to_owned())\n            .with_description(\"Invalid property.\"));\n    };\n\n    match (property, value) {\n        (IdentityProperty::Name, Value::Str(value)) if value.len() < 255 => {\n            identity.name = value.into_owned();\n        }\n        (IdentityProperty::Email, Value::Str(value)) if is_create && value.len() < 255 => {\n            identity.email = sanitize_email(&value).ok_or_else(|| {\n                SetError::invalid_properties()\n                    .with_property(IdentityProperty::Email)\n                    .with_description(\"Invalid e-mail address.\")\n            })?;\n        }\n        (IdentityProperty::TextSignature, Value::Str(value)) if value.len() < 2048 => {\n            identity.text_signature = value.into_owned();\n        }\n        (IdentityProperty::HtmlSignature, Value::Str(value)) if value.len() < 2048 => {\n            identity.html_signature = value.into_owned();\n        }\n        (IdentityProperty::ReplyTo | IdentityProperty::Bcc, Value::Array(value)) => {\n            let mut addresses = Vec::with_capacity(value.len());\n            for addr in value {\n                let mut address = EmailAddress {\n                    name: None,\n                    email: \"\".into(),\n                };\n                let mut is_valid = false;\n                if let Value::Object(obj) = addr {\n                    for (key, value) in obj.into_vec() {\n                        match (key, value) {\n                            (Key::Property(IdentityProperty::Email), Value::Str(value))\n                                if value.len() < 255 =>\n                            {\n                                is_valid = true;\n                                address.email = value.into_owned();\n                            }\n                            (Key::Property(IdentityProperty::Name), Value::Str(value))\n                                if value.len() < 255 =>\n                            {\n                                address.name = Some(value.into_owned());\n                            }\n                            (Key::Property(IdentityProperty::Name), Value::Null) => (),\n                            _ => {\n                                is_valid = false;\n                                break;\n                            }\n                        }\n                    }\n                }\n\n                if is_valid && !address.email.is_empty() {\n                    addresses.push(address);\n                } else {\n                    return Err(SetError::invalid_properties()\n                        .with_property(property.clone())\n                        .with_description(\"Invalid e-mail address object.\"));\n                }\n            }\n\n            match property {\n                IdentityProperty::ReplyTo => {\n                    identity.reply_to = Some(addresses);\n                }\n                IdentityProperty::Bcc => {\n                    identity.bcc = Some(addresses);\n                }\n                _ => unreachable!(),\n            }\n        }\n        (IdentityProperty::Name, Value::Null) => {\n            identity.name.clear();\n        }\n        (IdentityProperty::TextSignature, Value::Null) => {\n            identity.text_signature.clear();\n        }\n        (IdentityProperty::HtmlSignature, Value::Null) => {\n            identity.html_signature.clear();\n        }\n        (IdentityProperty::ReplyTo, Value::Null) => identity.reply_to = None,\n        (IdentityProperty::Bcc, Value::Null) => identity.bcc = None,\n        (property, _) => {\n            return Err(SetError::invalid_properties()\n                .with_property(property.clone())\n                .with_description(\"Field could not be set.\"));\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/jmap/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#![warn(clippy::large_futures)]\n\npub mod addressbook;\npub mod api;\npub mod blob;\npub mod calendar;\npub mod calendar_event;\npub mod calendar_event_notification;\npub mod changes;\npub mod contact;\npub mod email;\npub mod file;\npub mod identity;\npub mod mailbox;\npub mod participant_identity;\npub mod principal;\npub mod push;\npub mod quota;\npub mod share_notification;\npub mod sieve;\npub mod submission;\npub mod thread;\npub mod vacation;\npub mod websocket;\n"
  },
  {
    "path": "crates/jmap/src/mailbox/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken, sharing::EffectiveAcl};\nuse email::cache::{MessageCacheFetch, email::MessageCacheAccess, mailbox::MailboxCacheAccess};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::mailbox::{Mailbox, MailboxProperty, MailboxValue},\n};\nuse jmap_tools::{Map, Value};\nuse std::future::Future;\nuse store::ahash::AHashSet;\nuse types::{acl::Acl, keyword::Keyword, special_use::SpecialUse};\n\nuse crate::api::acl::JmapRights;\n\npub trait MailboxGet: Sync + Send {\n    fn mailbox_get(\n        &self,\n        request: GetRequest<Mailbox>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<Mailbox>>> + Send;\n}\n\nimpl MailboxGet for Server {\n    async fn mailbox_get(\n        &self,\n        mut request: GetRequest<Mailbox>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<Mailbox>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            MailboxProperty::Id,\n            MailboxProperty::Name,\n            MailboxProperty::ParentId,\n            MailboxProperty::Role,\n            MailboxProperty::SortOrder,\n            MailboxProperty::IsSubscribed,\n            MailboxProperty::TotalEmails,\n            MailboxProperty::UnreadEmails,\n            MailboxProperty::TotalThreads,\n            MailboxProperty::UnreadThreads,\n            MailboxProperty::MyRights,\n        ]);\n        let account_id = request.account_id.document_id();\n        let cache = self.get_cached_messages(account_id).await?;\n        let shared_ids = if access_token.is_shared(account_id) {\n            cache.shared_mailboxes(access_token, Acl::Read).into()\n        } else {\n            None\n        };\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            cache\n                .mailboxes\n                .index\n                .keys()\n                .filter(|id| shared_ids.as_ref().is_none_or(|ids| ids.contains(**id)))\n                .copied()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: Some(cache.mailboxes.change_id.into()),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the mailbox object\n            let document_id = id.document_id();\n            let cached_mailbox = if let Some(mailbox) =\n                cache.mailbox_by_id(&document_id).filter(|_| {\n                    shared_ids\n                        .as_ref()\n                        .is_none_or(|ids| ids.contains(document_id))\n                }) {\n                mailbox\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n\n            let mut mailbox = Map::with_capacity(properties.len());\n\n            for property in &properties {\n                let value = match property {\n                    MailboxProperty::Id => Value::Element(MailboxValue::Id(id)),\n                    MailboxProperty::Name => Value::Str(cached_mailbox.name.to_string().into()),\n                    MailboxProperty::Role => match cached_mailbox.role {\n                        SpecialUse::None => Value::Null,\n                        role => Value::Element(MailboxValue::Role(role)),\n                    },\n                    MailboxProperty::SortOrder => {\n                        Value::Number(cached_mailbox.sort_order().unwrap_or_default().into())\n                    }\n                    MailboxProperty::ParentId => {\n                        if let Some(parent_id) = cached_mailbox.parent_id() {\n                            Value::Element(MailboxValue::Id(parent_id.into()))\n                        } else {\n                            Value::Null\n                        }\n                    }\n                    MailboxProperty::TotalEmails => {\n                        Value::Number(cache.in_mailbox(document_id).count().into())\n                    }\n                    MailboxProperty::UnreadEmails => Value::Number(\n                        cache\n                            .in_mailbox_without_keyword(document_id, &Keyword::Seen)\n                            .count()\n                            .into(),\n                    ),\n                    MailboxProperty::TotalThreads => Value::Number(\n                        cache\n                            .in_mailbox(document_id)\n                            .map(|m| m.thread_id)\n                            .collect::<AHashSet<_>>()\n                            .len()\n                            .into(),\n                    ),\n                    MailboxProperty::UnreadThreads => Value::Number(\n                        cache\n                            .in_mailbox_without_keyword(document_id, &Keyword::Seen)\n                            .map(|m| m.thread_id)\n                            .collect::<AHashSet<_>>()\n                            .len()\n                            .into(),\n                    ),\n                    MailboxProperty::MyRights => {\n                        if access_token.is_shared(account_id) {\n                            JmapRights::rights::<Mailbox>(\n                                cached_mailbox.acls.as_slice().effective_acl(access_token),\n                            )\n                        } else {\n                            JmapRights::all_rights::<Mailbox>()\n                        }\n                    }\n                    MailboxProperty::IsSubscribed => Value::Bool(\n                        cached_mailbox\n                            .subscribers\n                            .contains(&access_token.primary_id()),\n                    ),\n                    MailboxProperty::ShareWith => JmapRights::share_with::<Mailbox>(\n                        account_id,\n                        access_token,\n                        &cached_mailbox.acls,\n                    ),\n                    _ => Value::Null,\n                };\n\n                mailbox.insert_unchecked(property.clone(), value);\n            }\n\n            // Add result to response\n            response.list.push(mailbox.into());\n        }\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/mailbox/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/mailbox/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::JmapCacheState};\nuse common::{Server, auth::AccessToken};\nuse email::cache::{MessageCacheFetch, mailbox::MailboxCacheAccess};\nuse jmap_proto::{\n    method::query::{Comparator, Filter, QueryRequest, QueryResponse},\n    object::mailbox::{Mailbox, MailboxComparator, MailboxFilter},\n};\nuse std::{collections::BTreeMap, future::Future};\nuse store::{\n    ahash::AHashMap,\n    roaring::RoaringBitmap,\n    search::{SearchComparator, SearchFilter, SearchQuery},\n    write::SearchIndex,\n};\nuse types::{acl::Acl, special_use::SpecialUse};\n\npub trait MailboxQuery: Sync + Send {\n    fn mailbox_query(\n        &self,\n        request: QueryRequest<Mailbox>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl MailboxQuery for Server {\n    async fn mailbox_query(\n        &self,\n        mut request: QueryRequest<Mailbox>,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let sort_as_tree = request.arguments.sort_as_tree.unwrap_or(false);\n        let filter_as_tree = request.arguments.filter_as_tree.unwrap_or(false);\n        let mut filters = Vec::with_capacity(request.filter.len());\n        let mailboxes = self.get_cached_messages(account_id).await?;\n\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => {\n                    match cond {\n                        MailboxFilter::ParentId(parent_id) => {\n                            let parent_id = parent_id\n                                .and_then(|id| id.try_unwrap().map(|id| id.document_id()))\n                                .unwrap_or(u32::MAX);\n                            filters.push(SearchFilter::is_in_set(\n                                mailboxes\n                                    .mailboxes\n                                    .items\n                                    .iter()\n                                    .filter(|mailbox| mailbox.parent_id == parent_id)\n                                    .map(|m| m.document_id)\n                                    .collect::<RoaringBitmap>(),\n                            ));\n                        }\n                        MailboxFilter::Name(name) => {\n                            #[cfg(feature = \"test_mode\")]\n                            {\n                                // Used for concurrent requests tests\n                                if name == \"__sleep\" {\n                                    tokio::time::sleep(std::time::Duration::from_secs(1)).await;\n                                }\n                            }\n                            let name = name.to_lowercase();\n                            filters.push(SearchFilter::is_in_set(\n                                mailboxes\n                                    .mailboxes\n                                    .items\n                                    .iter()\n                                    .filter(|mailbox| mailbox.name.to_lowercase().contains(&name))\n                                    .map(|m| m.document_id)\n                                    .collect::<RoaringBitmap>(),\n                            ));\n                        }\n                        MailboxFilter::Role(role) => {\n                            if let Some(role) = role {\n                                filters.push(SearchFilter::is_in_set(\n                                    mailboxes\n                                        .mailboxes\n                                        .items\n                                        .iter()\n                                        .filter(|mailbox| mailbox.role == role)\n                                        .map(|m| m.document_id)\n                                        .collect::<RoaringBitmap>(),\n                                ));\n                            } else {\n                                filters.push(SearchFilter::is_in_set(\n                                    mailboxes\n                                        .mailboxes\n                                        .items\n                                        .iter()\n                                        .filter(|mailbox| matches!(mailbox.role, SpecialUse::None))\n                                        .map(|m| m.document_id)\n                                        .collect::<RoaringBitmap>(),\n                                ));\n                            }\n                        }\n                        MailboxFilter::HasAnyRole(has_role) => {\n                            filters.push(SearchFilter::is_in_set(\n                                mailboxes\n                                    .mailboxes\n                                    .items\n                                    .iter()\n                                    .filter(|mailbox| {\n                                        matches!(mailbox.role, SpecialUse::None) != has_role\n                                    })\n                                    .map(|m| m.document_id)\n                                    .collect::<RoaringBitmap>(),\n                            ));\n                        }\n                        MailboxFilter::IsSubscribed(is_subscribed) => {\n                            filters.push(SearchFilter::is_in_set(\n                                mailboxes\n                                    .mailboxes\n                                    .items\n                                    .iter()\n                                    .filter(|mailbox| {\n                                        mailbox.subscribers.contains(&access_token.primary_id)\n                                            == is_subscribed\n                                    })\n                                    .map(|m| m.document_id)\n                                    .collect::<RoaringBitmap>(),\n                            ));\n                        }\n                        MailboxFilter::_T(other) => {\n                            return Err(trc::JmapEvent::UnsupportedFilter\n                                .into_err()\n                                .details(other));\n                        }\n                    }\n                }\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        let mut comparators = Vec::with_capacity(request.sort.as_ref().map_or(1, |s| s.len()));\n\n        // Sort as tree\n        if sort_as_tree {\n            let sorted_set = mailboxes\n                .mailboxes\n                .items\n                .iter()\n                .map(|mailbox| (mailbox.path.as_str(), mailbox.document_id))\n                .collect::<BTreeMap<_, _>>();\n            comparators.push(SearchComparator::sorted_set(\n                sorted_set\n                    .into_iter()\n                    .enumerate()\n                    .map(|(i, (_, v))| (v, i as u32))\n                    .collect(),\n                true,\n            ));\n        }\n\n        // Parse sort criteria\n        for comparator in request\n            .sort\n            .take()\n            .filter(|s| !s.is_empty())\n            .unwrap_or_else(|| vec![Comparator::ascending(MailboxComparator::ParentId)])\n        {\n            comparators.push(match comparator.property {\n                MailboxComparator::Name => {\n                    let sorted_set = mailboxes\n                        .mailboxes\n                        .items\n                        .iter()\n                        .map(|mailbox| (mailbox.name.as_str(), mailbox.document_id))\n                        .collect::<BTreeMap<_, _>>();\n\n                    SearchComparator::sorted_set(\n                        sorted_set\n                            .into_iter()\n                            .enumerate()\n                            .map(|(i, (_, v))| (v, i as u32))\n                            .collect(),\n                        comparator.is_ascending,\n                    )\n                }\n                MailboxComparator::SortOrder => {\n                    let sorted_set = mailboxes\n                        .mailboxes\n                        .items\n                        .iter()\n                        .map(|mailbox| (mailbox.document_id, mailbox.sort_order))\n                        .collect::<AHashMap<_, _>>();\n\n                    SearchComparator::sorted_set(sorted_set, comparator.is_ascending)\n                }\n                MailboxComparator::ParentId => {\n                    let sorted_set = mailboxes\n                        .mailboxes\n                        .items\n                        .iter()\n                        .map(|mailbox| {\n                            (\n                                mailbox.document_id,\n                                mailbox.parent_id().map(|id| id + 1).unwrap_or_default(),\n                            )\n                        })\n                        .collect::<AHashMap<_, _>>();\n\n                    SearchComparator::sorted_set(sorted_set, comparator.is_ascending)\n                }\n\n                MailboxComparator::_T(other) => {\n                    return Err(trc::JmapEvent::UnsupportedSort.into_err().details(other));\n                }\n            });\n        }\n\n        let mut results = SearchQuery::new(SearchIndex::InMemory)\n            .with_filters(filters)\n            .with_comparators(comparators)\n            .with_mask(if access_token.is_shared(account_id) {\n                mailboxes.shared_mailboxes(access_token, Acl::Read)\n            } else {\n                mailboxes\n                    .mailboxes\n                    .items\n                    .iter()\n                    .map(|m| m.document_id)\n                    .collect()\n            })\n            .filter();\n\n        // Filter as tree\n        if filter_as_tree {\n            let mut new_results = RoaringBitmap::new();\n\n            for document_id in results.results() {\n                let mut check_id = document_id;\n                for _ in 0..self.core.jmap.mailbox_max_depth {\n                    if let Some(mailbox) = mailboxes.mailbox_by_id(&check_id) {\n                        if let Some(parent_id) = mailbox.parent_id() {\n                            if results.results().contains(parent_id) {\n                                check_id = parent_id;\n                            } else {\n                                break;\n                            }\n                        } else {\n                            new_results.insert(document_id);\n                        }\n                    }\n                }\n            }\n\n            results.update_results(new_results);\n        }\n\n        let mut response = QueryResponseBuilder::new(\n            results.results().len() as usize,\n            self.core.jmap.query_max_results,\n            mailboxes.get_state(true),\n            &request,\n        );\n\n        for document_id in results.into_sorted() {\n            if !response.add(0, document_id) {\n                break;\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/mailbox/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    api::acl::{JmapAcl, JmapRights},\n    changes::state::JmapCacheState,\n};\nuse common::{\n    Server, auth::AccessToken, sharing::EffectiveAcl, storage::index::ObjectIndexBuilder,\n};\n#[allow(unused_imports)]\nuse email::mailbox::{INBOX_ID, JUNK_ID, TRASH_ID, UidMailbox};\nuse email::{\n    cache::{MessageCacheFetch, mailbox::MailboxCacheAccess},\n    mailbox::{\n        Mailbox,\n        destroy::{MailboxDestroy, MailboxDestroyError},\n    },\n};\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::mailbox::{self, MailboxProperty, MailboxValue},\n    references::resolve::ResolveCreatedReference,\n    request::IntoValid,\n    types::state::State,\n};\nuse jmap_tools::{JsonPointerItem, Key, Map, Value};\nuse std::future::Future;\nuse store::{\n    ValueKey,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive, BatchBuilder, assert::AssertValue},\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl, collection::Collection, field::MailboxField, id::Id, special_use::SpecialUse,\n};\n\npub struct SetContext<'x> {\n    account_id: u32,\n    access_token: &'x AccessToken,\n    is_shared: bool,\n    response: SetResponse<mailbox::Mailbox>,\n    mailbox_ids: RoaringBitmap,\n    will_destroy: Vec<Id>,\n}\n\npub trait MailboxSet: Sync + Send {\n    fn mailbox_set(\n        &self,\n        request: SetRequest<'_, mailbox::Mailbox>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<SetResponse<mailbox::Mailbox>>> + Send;\n\n    fn mailbox_set_item(\n        &self,\n        changes_: Map<'_, MailboxProperty, MailboxValue>,\n        update: Option<(u32, Archive<Mailbox>)>,\n        ctx: &SetContext,\n    ) -> impl Future<\n        Output = trc::Result<\n            Result<ObjectIndexBuilder<Mailbox, Mailbox>, SetError<MailboxProperty>>,\n        >,\n    > + Send;\n}\n\nimpl MailboxSet for Server {\n    #[allow(clippy::blocks_in_conditions)]\n    async fn mailbox_set(\n        &self,\n        mut request: SetRequest<'_, mailbox::Mailbox>,\n        access_token: &AccessToken,\n    ) -> trc::Result<SetResponse<mailbox::Mailbox>> {\n        // Prepare response\n        let account_id = request.account_id.document_id();\n        let on_destroy_remove_emails = request.arguments.on_destroy_remove_emails.unwrap_or(false);\n        let cache = self.get_cached_messages(account_id).await?;\n        let mut ctx = SetContext {\n            account_id,\n            is_shared: access_token.is_shared(account_id),\n            access_token,\n            response: SetResponse::from_request(&request, self.core.jmap.set_max_objects)?\n                .with_state(cache.assert_state(true, &request.if_in_state)?),\n            mailbox_ids: RoaringBitmap::from_iter(cache.mailboxes.index.keys()),\n            will_destroy: request.unwrap_destroy().into_valid().collect(),\n        };\n        let mut change_id = None;\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        'create: for (id, object) in request.unwrap_create() {\n            let Some(object) = object.into_object() else {\n                continue;\n            };\n\n            // Validate quota\n            if ctx.mailbox_ids.len() >= access_token.object_quota(Collection::Mailbox) as u64 {\n                ctx.response.not_created.append(\n                    id,\n                    SetError::new(SetErrorType::OverQuota).with_description(concat!(\n                        \"There are too many mailboxes, \",\n                        \"please delete some before adding a new one.\"\n                    )),\n                );\n                continue 'create;\n            }\n\n            match self.mailbox_set_item(object, None, &ctx).await? {\n                Ok(builder) => {\n                    batch\n                        .with_account_id(account_id)\n                        .with_collection(Collection::Mailbox);\n\n                    let parent_id = builder.changes().unwrap().parent_id;\n                    if parent_id > 0 {\n                        batch\n                            .with_document(parent_id - 1)\n                            .assert_value(MailboxField::Archive, AssertValue::Some);\n                    }\n\n                    let document_id = self\n                        .store()\n                        .assign_document_ids(account_id, Collection::Mailbox, 1)\n                        .await\n                        .caused_by(trc::location!())?;\n\n                    batch\n                        .with_document(document_id)\n                        .custom(builder)\n                        .caused_by(trc::location!())?\n                        .commit_point();\n\n                    ctx.mailbox_ids.insert(document_id);\n                    ctx.response.created(id, document_id);\n                }\n                Err(err) => {\n                    ctx.response.not_created.append(id, err);\n                    continue 'create;\n                }\n            }\n        }\n\n        if !batch.is_empty() {\n            change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?\n                .into();\n        }\n\n        // Process updates\n        let mut will_update = Vec::with_capacity(request.update.as_ref().map_or(0, |u| u.len()));\n        let mut batch = BatchBuilder::new();\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if ctx.will_destroy.contains(&id) {\n                ctx.response\n                    .not_updated\n                    .append(id, SetError::will_destroy());\n                continue 'update;\n            }\n            let Some(object) = object.into_object() else {\n                continue 'update;\n            };\n\n            // Obtain mailbox\n            let document_id = id.document_id();\n            if let Some(mailbox) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::Mailbox,\n                    document_id,\n                ))\n                .await?\n            {\n                // Validate ACL\n                let mailbox = mailbox\n                    .into_deserialized::<email::mailbox::Mailbox>()\n                    .caused_by(trc::location!())?;\n                if ctx.is_shared {\n                    let acl = mailbox.inner.acls.effective_acl(access_token);\n                    if !acl.contains(Acl::Modify) {\n                        ctx.response.not_updated.append(\n                            id,\n                            SetError::forbidden()\n                                .with_description(\"You are not allowed to modify this mailbox.\"),\n                        );\n                        continue 'update;\n                    } else if object.contains_key(&Key::Property(MailboxProperty::ShareWith))\n                        && !acl.contains(Acl::Share)\n                    {\n                        ctx.response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(\n                                \"You are not allowed to change the permissions of this mailbox.\",\n                            ),\n                        );\n                        continue 'update;\n                    }\n                }\n\n                match self\n                    .mailbox_set_item(object, (document_id, mailbox).into(), &ctx)\n                    .await?\n                {\n                    Ok(builder) => {\n                        batch\n                            .with_account_id(account_id)\n                            .with_collection(Collection::Mailbox);\n\n                        let parent_id = builder.changes().unwrap().parent_id;\n                        if parent_id > 0 {\n                            batch\n                                .with_document(parent_id - 1)\n                                .assert_value(MailboxField::Archive, AssertValue::Some);\n                        }\n\n                        batch\n                            .with_document(document_id)\n                            .custom(builder)\n                            .caused_by(trc::location!())?\n                            .commit_point();\n                        will_update.push(id);\n                    }\n                    Err(err) => {\n                        ctx.response.not_updated.append(id, err);\n                        continue 'update;\n                    }\n                }\n            } else {\n                ctx.response.not_updated.append(id, SetError::not_found());\n            }\n        }\n\n        if !batch.is_empty() {\n            match self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n            {\n                Ok(change_id_) => {\n                    change_id = Some(change_id_);\n                    for id in will_update {\n                        ctx.response.updated.append(id, None);\n                    }\n                }\n                Err(err) if err.is_assertion_failure() => {\n                    for id in will_update {\n                        ctx.response.not_updated.append(\n                            id,\n                            SetError::forbidden().with_description(\n                                \"Another process modified this mailbox, please try again.\",\n                            ),\n                        );\n                    }\n                }\n                Err(err) => {\n                    return Err(err.caused_by(trc::location!()));\n                }\n            }\n        }\n\n        // Process deletions\n        for id in ctx.will_destroy {\n            match self\n                .mailbox_destroy(\n                    account_id,\n                    id.document_id(),\n                    ctx.access_token,\n                    on_destroy_remove_emails,\n                )\n                .await?\n            {\n                Ok(change_id_) => {\n                    if change_id_.is_some() {\n                        change_id = change_id_;\n                    }\n                    ctx.response.destroyed.push(id);\n                }\n                Err(err) => {\n                    ctx.response.not_destroyed.append(\n                        id,\n                        match err {\n                            MailboxDestroyError::CannotDestroy => SetError::forbidden()\n                                .with_description(\n                                    \"You are not allowed to delete Inbox, Junk or Trash folders.\",\n                                ),\n                            MailboxDestroyError::Forbidden => SetError::forbidden()\n                                .with_description(\"You are not allowed to delete this mailbox.\"),\n                            MailboxDestroyError::HasChildren => {\n                                SetError::new(SetErrorType::MailboxHasChild)\n                                    .with_description(\"Mailbox has at least one children.\")\n                            }\n                            MailboxDestroyError::HasEmails => {\n                                SetError::new(SetErrorType::MailboxHasEmail)\n                                    .with_description(\"Mailbox is not empty.\")\n                            }\n                            MailboxDestroyError::NotFound => SetError::not_found(),\n                            MailboxDestroyError::AssertionFailed => SetError::forbidden()\n                                .with_description(concat!(\n                                    \"Another process modified a message in this mailbox \",\n                                    \"while deleting it, please try again.\"\n                                )),\n                        },\n                    );\n                }\n            }\n        }\n\n        // Write changes\n        if let Some(change_id) = change_id {\n            ctx.response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(ctx.response)\n    }\n\n    #[allow(clippy::blocks_in_conditions)]\n    async fn mailbox_set_item(\n        &self,\n        changes_: Map<'_, MailboxProperty, MailboxValue>,\n        update: Option<(u32, Archive<Mailbox>)>,\n        ctx: &SetContext<'_>,\n    ) -> trc::Result<Result<ObjectIndexBuilder<Mailbox, Mailbox>, SetError<MailboxProperty>>> {\n        // Parse properties\n        let mut changes = update\n            .as_ref()\n            .map(|(_, obj)| obj.inner.clone())\n            .unwrap_or_else(|| Mailbox::new(String::new()));\n        let mut has_acl_changes = false;\n        for (property, mut value) in changes_.into_vec() {\n            if let Err(err) = ctx.response.resolve_self_references(&mut value) {\n                return Ok(Err(err));\n            };\n            match (&property, value) {\n                (Key::Property(MailboxProperty::Name), Value::Str(value)) => {\n                    let value = value.trim();\n                    if !value.is_empty() && value.len() < self.core.jmap.mailbox_name_max_len {\n                        changes.name = value.into();\n                    } else {\n                        return Ok(Err(SetError::invalid_properties()\n                            .with_property(MailboxProperty::Name)\n                            .with_description(\n                                if !value.is_empty() {\n                                    \"Mailbox name is too long.\"\n                                } else {\n                                    \"Mailbox name cannot be empty.\"\n                                }\n                                .to_string(),\n                            )));\n                    }\n                }\n                (\n                    Key::Property(MailboxProperty::ParentId),\n                    Value::Element(MailboxValue::Id(value)),\n                ) => {\n                    let parent_id = value.document_id();\n                    if ctx.will_destroy.contains(&value) {\n                        return Ok(Err(SetError::will_destroy()\n                            .with_description(\"Parent ID will be destroyed.\")));\n                    } else if !ctx.mailbox_ids.contains(parent_id) {\n                        return Ok(Err(SetError::invalid_properties()\n                            .with_description(\"Parent ID does not exist.\")));\n                    }\n                    changes.parent_id = parent_id + 1;\n                }\n                (Key::Property(MailboxProperty::ParentId), Value::Null) => {\n                    changes.parent_id = 0;\n                }\n                (Key::Property(MailboxProperty::IsSubscribed), Value::Bool(subscribe)) => {\n                    let account_id = ctx.access_token.primary_id();\n                    if subscribe {\n                        if !changes.subscribers.contains(&account_id) {\n                            changes.subscribers.push(account_id);\n                        }\n                    } else {\n                        changes.subscribers.retain(|id| *id != account_id);\n                    }\n                }\n                (\n                    Key::Property(MailboxProperty::Role),\n                    Value::Element(MailboxValue::Role(role)),\n                ) => {\n                    changes.role = role;\n                }\n                (Key::Property(MailboxProperty::Role), Value::Null) => {\n                    changes.role = SpecialUse::None;\n                }\n                (Key::Property(MailboxProperty::SortOrder), Value::Number(value)) => {\n                    changes.sort_order = Some(value.cast_to_u64() as u32);\n                }\n                (Key::Property(MailboxProperty::ShareWith), value) => {\n                    match JmapRights::acl_set::<mailbox::Mailbox>(value) {\n                        Ok(acls) => {\n                            has_acl_changes = true;\n                            changes.acls = acls;\n                            continue;\n                        }\n                        Err(err) => {\n                            return Ok(Err(err));\n                        }\n                    }\n                }\n                (Key::Property(MailboxProperty::Pointer(pointer)), value)\n                    if matches!(\n                        pointer.first(),\n                        Some(JsonPointerItem::Key(Key::Property(\n                            MailboxProperty::ShareWith\n                        )))\n                    ) =>\n                {\n                    let mut pointer = pointer.iter();\n                    pointer.next();\n\n                    match JmapRights::acl_patch::<mailbox::Mailbox>(changes.acls, pointer, value) {\n                        Ok(acls) => {\n                            has_acl_changes = true;\n                            changes.acls = acls;\n                            continue;\n                        }\n                        Err(err) => {\n                            return Ok(Err(err));\n                        }\n                    }\n                }\n\n                _ => {\n                    return Ok(Err(SetError::invalid_properties()\n                        .with_property(property.into_owned())\n                        .with_description(\"Invalid property or value.\".to_string())));\n                }\n            }\n        }\n\n        // Validate depth and circular parent-child relationship\n        if update\n            .as_ref()\n            .is_none_or(|(_, m)| m.inner.parent_id != changes.parent_id)\n        {\n            let mut mailbox_parent_id = changes.parent_id;\n            let current_mailbox_id = update\n                .as_ref()\n                .map_or(u32::MAX, |(mailbox_id, _)| *mailbox_id + 1);\n            let mut success = false;\n            for depth in 0..self.core.jmap.mailbox_max_depth {\n                if mailbox_parent_id == current_mailbox_id {\n                    return Ok(Err(SetError::invalid_properties()\n                        .with_property(MailboxProperty::ParentId)\n                        .with_description(\"Mailbox cannot be a parent of itself.\")));\n                } else if mailbox_parent_id == 0 {\n                    if depth == 0 && ctx.is_shared {\n                        return Ok(Err(SetError::forbidden()\n                            .with_description(\"You are not allowed to create root folders.\")));\n                    }\n                    success = true;\n                    break;\n                }\n                let parent_document_id = mailbox_parent_id - 1;\n\n                if let Some(mailbox_) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        ctx.account_id,\n                        Collection::Mailbox,\n                        parent_document_id,\n                    ))\n                    .await?\n                {\n                    let mailbox = mailbox_\n                        .unarchive::<email::mailbox::Mailbox>()\n                        .caused_by(trc::location!())?;\n                    if depth == 0\n                        && ctx.is_shared\n                        && !mailbox\n                            .acls\n                            .effective_acl(ctx.access_token)\n                            .contains(Acl::CreateChild)\n                    {\n                        return Ok(Err(SetError::forbidden().with_description(\n                            \"You are not allowed to create sub mailboxes under this mailbox.\",\n                        )));\n                    }\n\n                    mailbox_parent_id = mailbox.parent_id.into();\n                } else if ctx.mailbox_ids.contains(parent_document_id) {\n                    // Parent mailbox is probably created within the same request\n                    success = true;\n                    break;\n                } else {\n                    return Ok(Err(SetError::invalid_properties()\n                        .with_property(MailboxProperty::ParentId)\n                        .with_description(\"Mailbox parent does not exist.\")));\n                }\n            }\n\n            if !success {\n                return Ok(Err(SetError::invalid_properties()\n                    .with_property(MailboxProperty::ParentId)\n                    .with_description(\n                        \"Mailbox parent-child relationship is too deep.\",\n                    )));\n            }\n        }\n\n        let cached_mailboxes = self.get_cached_messages(ctx.account_id).await?;\n\n        // Verify that the mailbox role is unique.\n        if update\n            .as_ref()\n            .is_none_or(|(_, m)| m.inner.role != changes.role)\n        {\n            if !matches!(changes.role, SpecialUse::None)\n                && cached_mailboxes.mailbox_by_role(&changes.role).is_some()\n            {\n                return Ok(Err(SetError::invalid_properties()\n                    .with_property(MailboxProperty::Role)\n                    .with_description(format!(\n                        \"A mailbox with role '{}' already exists.\",\n                        changes.role.as_str().unwrap_or_default()\n                    ))));\n            }\n\n            // Role of internal folders cannot be modified\n            if update.as_ref().is_some_and(|(document_id, _)| {\n                *document_id == INBOX_ID || *document_id == TRASH_ID || *document_id == JUNK_ID\n            }) {\n                return Ok(Err(SetError::invalid_properties()\n                    .with_property(MailboxProperty::Role)\n                    .with_description(\n                        \"You are not allowed to change the role of Inbox, Junk or Trash folders.\",\n                    )));\n            }\n        }\n\n        // Verify that the mailbox name is unique.\n        if !changes.name.is_empty() {\n            // Obtain parent mailbox id\n            let lower_name = changes.name.to_lowercase();\n            if update\n                .as_ref()\n                .is_none_or(|(_, m)| m.inner.name != changes.name)\n                && cached_mailboxes.mailboxes.items.iter().any(|m| {\n                    m.name.to_lowercase() == lower_name\n                        && m.parent_id().map_or(0, |id| id + 1) == changes.parent_id\n                })\n            {\n                return Ok(Err(SetError::invalid_properties()\n                    .with_property(MailboxProperty::Name)\n                    .with_description(format!(\n                        \"A mailbox with name '{}' already exists.\",\n                        changes.name\n                    ))));\n            }\n        } else {\n            return Ok(Err(SetError::invalid_properties()\n                .with_property(MailboxProperty::Name)\n                .with_description(\"Mailbox name cannot be empty.\")));\n        }\n\n        // Refresh ACLs\n        let current = update.map(|(_, current)| current);\n        if has_acl_changes {\n            if !changes.acls.is_empty()\n                && let Err(err) = self.acl_validate(&changes.acls).await\n            {\n                return Ok(Err(err.into()));\n            }\n\n            self.refresh_acls(\n                &changes.acls,\n                current.as_ref().map(|m| m.inner.acls.as_slice()),\n            )\n            .await;\n        }\n\n        // Validate\n        Ok(Ok(ObjectIndexBuilder::new()\n            .with_changes(changes)\n            .with_current_opt(current)))\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/participant_identity/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse directory::{PrincipalData, QueryParams};\nuse groupware::calendar::{ParticipantIdentities, ParticipantIdentity};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::participant_identity::{self, ParticipantIdentityProperty, ParticipantIdentityValue},\n};\nuse jmap_tools::{Map, Value};\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::PrincipalField, id::Id};\n\npub trait ParticipantIdentityGet: Sync + Send {\n    fn participant_identity_get(\n        &self,\n        request: GetRequest<participant_identity::ParticipantIdentity>,\n    ) -> impl Future<Output = trc::Result<GetResponse<participant_identity::ParticipantIdentity>>> + Send;\n\n    fn participant_identity_get_or_create(\n        &self,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Option<Archive<AlignedBytes>>>> + Send;\n}\n\nimpl ParticipantIdentityGet for Server {\n    async fn participant_identity_get(\n        &self,\n        mut request: GetRequest<participant_identity::ParticipantIdentity>,\n    ) -> trc::Result<GetResponse<participant_identity::ParticipantIdentity>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            ParticipantIdentityProperty::Id,\n            ParticipantIdentityProperty::Name,\n            ParticipantIdentityProperty::CalendarAddress,\n            ParticipantIdentityProperty::IsDefault,\n        ]);\n        let account_id = request.account_id.document_id();\n        let identities = self.participant_identity_get_or_create(account_id).await?;\n\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: None,\n            list: Vec::new(),\n            not_found: vec![],\n        };\n\n        let Some(identities) = identities else {\n            response.not_found = ids.unwrap_or_default();\n            return Ok(response);\n        };\n\n        let identities = identities\n            .unarchive::<ParticipantIdentities>()\n            .caused_by(trc::location!())?;\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            identities\n                .identities\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(|i| Id::from(i.id.to_native()))\n                .collect::<Vec<_>>()\n        };\n\n        for id in ids {\n            // Obtain the identity object\n            let document_id = id.document_id();\n            let Some(identity) = identities.identities.iter().find(|i| i.id == document_id) else {\n                response.not_found.push(id);\n                continue;\n            };\n\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                let value = match &property {\n                    ParticipantIdentityProperty::Id => {\n                        Value::Element(ParticipantIdentityValue::Id(id))\n                    }\n                    ParticipantIdentityProperty::Name => Value::Str(\n                        identity\n                            .name\n                            .as_ref()\n                            .map(|n| n.as_str())\n                            .unwrap_or(identities.default_name.as_str())\n                            .to_string()\n                            .into(),\n                    ),\n                    ParticipantIdentityProperty::CalendarAddress => {\n                        Value::Str(identity.calendar_address.to_string().into())\n                    }\n                    ParticipantIdentityProperty::IsDefault => {\n                        Value::Bool(identities.default == document_id)\n                    }\n                };\n                result.insert_unchecked(property.clone(), value);\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n\n    async fn participant_identity_get_or_create(\n        &self,\n        account_id: u32,\n    ) -> trc::Result<Option<Archive<AlignedBytes>>> {\n        if let Some(identities) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                account_id,\n                Collection::Principal,\n                0,\n                PrincipalField::ParticipantIdentities,\n            ))\n            .await?\n        {\n            return Ok(Some(identities));\n        }\n\n        // Obtain principal\n        let principal = if let Some(principal) = self\n            .core\n            .storage\n            .directory\n            .query(QueryParams::id(account_id).with_return_member_of(false))\n            .await\n            .caused_by(trc::location!())?\n        {\n            principal\n        } else {\n            return Ok(None);\n        };\n        let mut emails = Vec::new();\n        let mut description = None;\n        for data in principal.data {\n            match data {\n                PrincipalData::PrimaryEmail(v) | PrincipalData::EmailAlias(v) => emails.push(v),\n                PrincipalData::Description(v) => description = Some(v),\n                _ => {}\n            }\n        }\n        let num_emails = emails.len();\n        if num_emails == 0 {\n            return Ok(None);\n        }\n\n        // Build identities\n        let identities = ParticipantIdentities {\n            identities: emails\n                .iter()\n                .enumerate()\n                .map(|(id, email)| ParticipantIdentity {\n                    id: id as u32,\n                    name: None,\n                    calendar_address: format!(\"mailto:{email}\"),\n                })\n                .collect(),\n            default: 0,\n            default_name: description.unwrap_or(principal.name),\n        };\n\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Principal)\n            .with_document(0)\n            .set(\n                PrincipalField::ParticipantIdentities,\n                Archiver::new(identities)\n                    .serialize()\n                    .caused_by(trc::location!())?,\n            );\n\n        self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n        self.store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                account_id,\n                Collection::Principal,\n                0,\n                PrincipalField::ParticipantIdentities,\n            ))\n            .await\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/participant_identity/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/participant_identity/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::participant_identity::get::ParticipantIdentityGet;\nuse common::{Server, auth::AccessToken};\nuse directory::QueryParams;\nuse groupware::calendar::{ParticipantIdentities, ParticipantIdentity};\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::participant_identity::{self, ParticipantIdentityProperty, ParticipantIdentityValue},\n    request::{IntoValid, reference::MaybeIdReference},\n};\nuse jmap_tools::{Key, Value};\nuse store::{\n    Serialize,\n    ahash::AHashSet,\n    write::{Archiver, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::PrincipalField};\nuse utils::sanitize_email;\n\npub trait ParticipantIdentitySet: Sync + Send {\n    fn participant_identity_set(\n        &self,\n        request: SetRequest<'_, participant_identity::ParticipantIdentity>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<SetResponse<participant_identity::ParticipantIdentity>>> + Send;\n}\n\nimpl ParticipantIdentitySet for Server {\n    async fn participant_identity_set(\n        &self,\n        mut request: SetRequest<'_, participant_identity::ParticipantIdentity>,\n        access_token: &AccessToken,\n    ) -> trc::Result<SetResponse<participant_identity::ParticipantIdentity>> {\n        let account_id = request.account_id.document_id();\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n        let (identity_archive, mut identities) =\n            match self.participant_identity_get_or_create(account_id).await? {\n                Some(archive) => {\n                    let identities = archive\n                        .deserialize::<ParticipantIdentities>()\n                        .caused_by(trc::location!())?;\n\n                    (Some(archive), identities)\n                }\n                None => (None, ParticipantIdentities::default()),\n            };\n\n        // Obtain allowed emails\n        let allowed_emails = self\n            .directory()\n            .query(QueryParams::id(account_id).with_return_member_of(false))\n            .await?\n            .map(|p| p.into_email_addresses().collect::<AHashSet<_>>())\n            .unwrap_or_default();\n\n        // Process creates\n        let mut has_changes = false;\n        'create: for (id, object) in request.unwrap_create() {\n            let mut identity = ParticipantIdentity::default();\n\n            if let Err(err) = validate_identity_value(object, &mut identity, &allowed_emails) {\n                response.not_created.append(id, err);\n                continue 'create;\n            }\n\n            if identities\n                .identities\n                .iter()\n                .any(|i| i.calendar_address == identity.calendar_address)\n            {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_property(ParticipantIdentityProperty::CalendarAddress)\n                        .with_description(\"Calendar address already in use.\".to_string()),\n                );\n                continue 'create;\n            }\n\n            // Validate quota\n            if identities.identities.len()\n                >= access_token.object_quota(Collection::Identity) as usize\n            {\n                response.not_created.append(\n                    id,\n                    SetError::new(SetErrorType::OverQuota).with_description(concat!(\n                        \"There are too many identities, \",\n                        \"please delete some before adding a new one.\"\n                    )),\n                );\n                continue 'create;\n            }\n\n            let document_id = identities\n                .identities\n                .iter()\n                .map(|i| i.id)\n                .max()\n                .unwrap_or_default()\n                + 1;\n            identity.id = document_id;\n            identities.identities.push(identity);\n\n            if let Some(MaybeIdReference::Reference(id_ref)) =\n                &request.arguments.on_success_set_is_default\n                && id_ref == &id\n            {\n                identities.default = document_id;\n            }\n\n            has_changes = true;\n            response.created(id, document_id);\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            let Some(identity) = identities\n                .identities\n                .iter_mut()\n                .find(|i| i.id == id.document_id())\n            else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n\n            if let Err(err) = validate_identity_value(object, identity, &allowed_emails) {\n                response.not_updated.append(id, err);\n                continue 'update;\n            }\n\n            has_changes = true;\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        for id in &will_destroy {\n            let document_id = id.document_id();\n            if identities.identities.iter().any(|i| i.id == document_id) {\n                response.destroyed.push(*id);\n            } else {\n                response.not_destroyed.append(*id, SetError::not_found());\n            }\n        }\n        if !response.destroyed.is_empty() {\n            has_changes = true;\n            identities\n                .identities\n                .retain(|i| !response.destroyed.iter().any(|id| id.document_id() == i.id));\n        }\n\n        if let Some(MaybeIdReference::Id(id)) = request.arguments.on_success_set_is_default {\n            let id = id.document_id();\n            if identities.identities.iter().any(|i| i.id == id) {\n                identities.default = id;\n                has_changes = true;\n            }\n        }\n\n        // Write changes\n        if has_changes {\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0);\n            if let Some(archive) = identity_archive {\n                batch.assert_value(PrincipalField::ParticipantIdentities, archive);\n            }\n            batch.set(\n                PrincipalField::ParticipantIdentities,\n                Archiver::new(identities)\n                    .serialize()\n                    .caused_by(trc::location!())?,\n            );\n\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n        }\n\n        Ok(response)\n    }\n}\n\nfn validate_identity_value(\n    update: Value<'_, ParticipantIdentityProperty, ParticipantIdentityValue>,\n    identity: &mut ParticipantIdentity,\n    allowed_emails: &AHashSet<String>,\n) -> Result<(), SetError<ParticipantIdentityProperty>> {\n    for (property, value) in update.into_expanded_object() {\n        let Key::Property(property) = property else {\n            return Err(SetError::invalid_properties()\n                .with_property(property.to_owned())\n                .with_description(\"Invalid property.\"));\n        };\n\n        match (property, value) {\n            (ParticipantIdentityProperty::Name, Value::Str(value)) if value.len() < 255 => {\n                identity.name = value.into_owned().into();\n            }\n            (ParticipantIdentityProperty::CalendarAddress, Value::Str(value)) => {\n                if identity.calendar_address != value {\n                    let email = if let Some(email) = value.strip_prefix(\"mailto:\") {\n                        sanitize_email(email)\n                    } else {\n                        sanitize_email(&value)\n                    };\n\n                    if let Some(email) = email {\n                        if allowed_emails.iter().any(|e| e == &email) {\n                            identity.calendar_address = format!(\"mailto:{email}\");\n                        } else {\n                            return Err(SetError::invalid_properties()\n                                .with_property(ParticipantIdentityProperty::CalendarAddress)\n                                .with_description(\n                                    \"Calendar address not configured for this account.\".to_string(),\n                                ));\n                        }\n                    } else {\n                        return Err(SetError::invalid_properties()\n                            .with_property(ParticipantIdentityProperty::CalendarAddress)\n                            .with_description(\"Invalid or missing calendar address.\".to_string()));\n                    }\n                }\n            }\n            (property, _) => {\n                return Err(SetError::invalid_properties()\n                    .with_property(property.clone())\n                    .with_description(\"Field could not be set.\"));\n            }\n        }\n    }\n\n    // Validate email address\n    if !identity.calendar_address.is_empty() {\n        Ok(())\n    } else {\n        Err(SetError::invalid_properties()\n            .with_property(ParticipantIdentityProperty::CalendarAddress)\n            .with_description(\"Missing calendar address.\"))\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/principal/availability.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{calendar::Availability, calendar_event::CalendarSyntheticId};\nuse calcard::{\n    common::timezone::Tz,\n    icalendar::{\n        ArchivedICalendarClassification, ArchivedICalendarParameterValue,\n        ArchivedICalendarParticipationStatus, ArchivedICalendarProperty, ArchivedICalendarStatus,\n        ArchivedICalendarTransparency, ArchivedICalendarValue, ICalendarParameterName,\n    },\n    jscalendar::{JSCalendar, JSCalendarProperty, JSCalendarValue},\n};\nuse common::{Server, TinyCalendarPreferences, auth::AccessToken};\nuse directory::Permission;\nuse groupware::{\n    cache::GroupwareCache,\n    calendar::{CALENDAR_SUBSCRIBED, CalendarEvent},\n};\nuse jmap_proto::{\n    method::availability::{\n        BusyPeriod, BusyStatus, GetAvailabilityRequest, GetAvailabilityResponse,\n    },\n    object::calendar::IncludeInAvailability,\n    request::IntoValid,\n    types::date::UTCDate,\n};\nuse jmap_tools::{Key, Map, Value};\nuse std::{collections::hash_map::Entry, future::Future};\nuse store::{ValueKey, ahash::AHashMap, write::{AlignedBytes, Archive}};\nuse trc::AddContext;\nuse types::{\n    TimeRange,\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\nuse utils::sanitize_email;\n\npub trait PrincipalGetAvailability: Sync + Send {\n    fn principal_get_availability(\n        &self,\n        request: GetAvailabilityRequest,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetAvailabilityResponse>> + Send;\n}\n\nimpl PrincipalGetAvailability for Server {\n    async fn principal_get_availability(\n        &self,\n        request: GetAvailabilityRequest,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetAvailabilityResponse> {\n        if !self.core.groupware.allow_directory_query\n            && !access_token.has_permission(Permission::IndividualList)\n        {\n            return Err(trc::JmapEvent::Forbidden\n                .into_err()\n                .details(\"The administrator has disabled directory queries.\".to_string()));\n        }\n\n        // Process parameters\n        if !request.id.is_valid() {\n            return Err(trc::JmapEvent::InvalidArguments\n                .into_err()\n                .details(\"Missing principal id\"));\n        }\n        let properties = request\n            .event_properties\n            .map(|props| props.into_valid().collect::<Vec<_>>())\n            .unwrap_or_default();\n        if properties\n            .iter()\n            .any(|p| !matches!(p, JSCalendarProperty::Id | JSCalendarProperty::BaseEventId))\n        {\n            return Err(trc::JmapEvent::InvalidArguments\n                .into_err()\n                .details(\"Only 'id' and 'baseEventId' properties are supported in results\"));\n        }\n        let return_event_details = !properties.is_empty();\n        let max_instances = self.core.groupware.max_ical_instances;\n        let filter = TimeRange {\n            start: request.utc_start.timestamp(),\n            end: request.utc_end.timestamp(),\n        };\n        let principal_id = request.id.document_id();\n        let principal = self\n            .get_access_token(principal_id)\n            .await\n            .caused_by(trc::location!())?;\n        let mut periods = Vec::new();\n\n        for account_id in principal.all_ids_by_collection(Collection::Calendar) {\n            let resources = self\n                .fetch_dav_resources(access_token, account_id, SyncCollection::Calendar)\n                .await\n                .caused_by(trc::location!())?;\n\n            // Obtain shared ids\n            let is_account_owner = principal_id == account_id;\n            let shared_ids = if !access_token.is_member(account_id) {\n                // Condition: The user has the \"mayReadFreeBusy\" permission for the calendar.\n                let shared_ids = resources.shared_items(\n                    access_token,\n                    [Acl::ReadItems, Acl::SchedulingReadFreeBusy],\n                    true,\n                );\n                if shared_ids.is_empty() {\n                    continue;\n                }\n\n                shared_ids.into()\n            } else {\n                None\n            };\n\n            // Condition: The event finishes after the \"utcStart\" argument and starts before the \"utcEnd\" argument.\n            let mut preferences_cache: AHashMap<u32, Option<&TinyCalendarPreferences>> =\n                AHashMap::default();\n            'next_event: for resource in resources.resources.iter().filter(|r| {\n                r.event_time_range().is_some_and(|(start, end)| {\n                    shared_ids\n                        .as_ref()\n                        .is_none_or(|ids| ids.contains(r.document_id))\n                        && filter.is_in_range(false, start, end)\n                })\n            }) {\n                // Obtain calendar settings\n                let mut include_in_availability = None;\n                let mut default_tz = Tz::UTC;\n                let mut is_subscribed = is_account_owner;\n                for calendar_id in resource\n                    .child_names()\n                    .unwrap_or_default()\n                    .iter()\n                    .map(|n| n.parent_id)\n                {\n                    match preferences_cache.entry(calendar_id) {\n                        Entry::Occupied(e) => {\n                            if let Some(prefs) = e.get() {\n                                default_tz = prefs.tz;\n                                is_subscribed |= prefs.flags & CALENDAR_SUBSCRIBED != 0;\n                                include_in_availability =\n                                    IncludeInAvailability::from_flags(prefs.flags);\n                            }\n                        }\n                        Entry::Vacant(e) => {\n                            if let Some(prefs) = resources\n                                .container_resource_by_id(calendar_id)\n                                .and_then(|r| r.calendar_preferences(principal_id))\n                            {\n                                default_tz = prefs.tz;\n                                is_subscribed |= prefs.flags & CALENDAR_SUBSCRIBED != 0;\n                                include_in_availability =\n                                    IncludeInAvailability::from_flags(prefs.flags);\n                                e.insert(Some(prefs));\n                            } else {\n                                e.insert(None);\n                            }\n                        }\n                    }\n                }\n                let include_in_availability = include_in_availability.unwrap_or({\n                    if is_account_owner {\n                        IncludeInAvailability::All\n                    } else {\n                        IncludeInAvailability::None\n                    }\n                });\n\n                if !is_subscribed || include_in_availability == IncludeInAvailability::None {\n                    continue 'next_event;\n                }\n\n                // Fetch event\n                let document_id = resource.document_id;\n                let Some(archive) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::CalendarEvent,\n                        document_id,\n                    ))\n                    .await\n                    .caused_by(trc::location!())?\n                else {\n                    continue;\n                };\n                let event = archive\n                    .unarchive::<CalendarEvent>()\n                    .caused_by(trc::location!())?;\n\n                // Find the component ids that match the criteria\n                let mut matching_component_ids = AHashMap::new();\n                'next_component: for (component_id, component) in\n                    event.data.event.components.iter().enumerate()\n                {\n                    if !component.component_type.is_event_or_todo() {\n                        continue 'next_component;\n                    }\n\n                    let mut is_cancelled = false;\n                    let mut is_main_event = true;\n                    let mut busy_status = None;\n\n                    for entry in component.entries.iter() {\n                        match (&entry.name, entry.values.first()) {\n                            (\n                                ArchivedICalendarProperty::Status,\n                                Some(ArchivedICalendarValue::Status(\n                                    ArchivedICalendarStatus::Cancelled,\n                                )),\n                            ) => {\n                                // The \"status\" property of the event is not \"cancelled\".\n                                is_cancelled = true;\n                            }\n                            (ArchivedICalendarProperty::RecurrenceId, _) => {\n                                is_main_event = false;\n                            }\n                            (\n                                ArchivedICalendarProperty::Class,\n                                Some(ArchivedICalendarValue::Classification(\n                                    ArchivedICalendarClassification::Confidential,\n                                )),\n                            ) => {\n                                // Condition: The event's \"privacy\" property is not \"secret\".\n                                continue 'next_component;\n                            }\n                            (\n                                ArchivedICalendarProperty::Transp,\n                                Some(ArchivedICalendarValue::Transparency(\n                                    ArchivedICalendarTransparency::Transparent,\n                                )),\n                            ) => {\n                                // Condition: The \"freeBusyStatus\" property of the event is \"busy\" (or omitted, as this is the default).\n                                continue 'next_component;\n                            }\n                            (ArchivedICalendarProperty::Attendee, Some(value))\n                                if include_in_availability == IncludeInAvailability::Attending =>\n                            {\n                                if let Some(attendee) = value.as_text().and_then(|attendee| {\n                                    sanitize_email(\n                                        attendee.strip_prefix(\"mailto:\").unwrap_or(attendee),\n                                    )\n                                }) {\n                                    // Condition: the Principal is a participant of the event, and has a \"participationStatus\" of \"accepted\" or \"tentative\".\n                                    if principal.emails.contains(&attendee) {\n                                        busy_status = Some(\n                                            entry\n                                                .parameters(&ICalendarParameterName::Partstat)\n                                                .next()\n                                                .map(|v| {\n                                                    match v {\n                                                ArchivedICalendarParameterValue::Partstat(\n                                                    ArchivedICalendarParticipationStatus::Accepted,\n                                                ) => BusyStatus::Confirmed,\n                                                ArchivedICalendarParameterValue::Partstat(\n                                                    ArchivedICalendarParticipationStatus::Tentative,\n                                                ) => BusyStatus::Tentative,\n                                                ArchivedICalendarParameterValue::Partstat(\n                                                    ArchivedICalendarParticipationStatus::Declined,\n                                                ) => {\n                                                    is_cancelled = true;\n                                                    BusyStatus::Unavailable\n                                                }\n                                                _ => BusyStatus::Unavailable,\n                                            }\n                                                })\n                                                .unwrap_or(BusyStatus::Unavailable),\n                                        );\n                                    }\n                                }\n                            }\n                            _ => (),\n                        }\n                    }\n\n                    if is_cancelled {\n                        if is_main_event {\n                            continue 'next_event;\n                        } else {\n                            continue 'next_component;\n                        }\n                    }\n\n                    let busy_status = if let Some(busy_status) = busy_status {\n                        busy_status\n                    } else if include_in_availability == IncludeInAvailability::All {\n                        BusyStatus::Confirmed\n                    } else {\n                        continue 'next_component;\n                    };\n\n                    matching_component_ids.insert(component_id as u32, busy_status);\n                }\n\n                if matching_component_ids.is_empty() {\n                    // No events matched the criteria\n                    continue 'next_event;\n                }\n\n                for expansion in event.data.expand(default_tz, filter).unwrap_or_default() {\n                    let Some(busy_status) = matching_component_ids.get(&expansion.comp_id) else {\n                        continue;\n                    };\n                    if periods.len() < max_instances {\n                        periods.push(FreeBusyResult {\n                            utc_start: expansion.start,\n                            utc_end: expansion.end,\n                            busy_status: *busy_status,\n                            expansion_id: expansion.comp_id,\n                            document_id,\n                        });\n                    } else {\n                        return Err(trc::JmapEvent::RequestTooLarge\n                            .into_err()\n                            .details(\"The number of expanded instances exceeds the server limit\"));\n                    }\n                }\n            }\n        }\n\n        let mut result = GetAvailabilityResponse {\n            list: Vec::with_capacity(periods.len()),\n        };\n\n        if periods.is_empty() {\n            return Ok(result);\n        }\n\n        // Sort by busy status and start time\n        periods.sort_unstable_by(|a, b| {\n            a.busy_status\n                .cmp(&b.busy_status)\n                .then_with(|| a.utc_start.cmp(&b.utc_start))\n        });\n\n        if return_event_details {\n            for period in periods {\n                result.list.push(period.into());\n            }\n        } else {\n            // Merge intervals with same busy status\n            let mut start_time = periods[0].utc_start;\n            let mut end_time = periods[0].utc_end;\n            let mut current_status = periods[0].busy_status;\n\n            for curr in periods.iter().skip(1) {\n                if curr.utc_start <= end_time && curr.busy_status == current_status {\n                    end_time = end_time.max(curr.utc_end);\n                } else {\n                    result.list.push(BusyPeriod {\n                        utc_start: UTCDate::from_timestamp(start_time),\n                        utc_end: UTCDate::from_timestamp(end_time),\n                        busy_status: Some(current_status),\n                        event: None,\n                    });\n                    start_time = curr.utc_start;\n                    end_time = curr.utc_end;\n                    current_status = curr.busy_status;\n                }\n            }\n\n            result.list.push(BusyPeriod {\n                utc_start: UTCDate::from_timestamp(start_time),\n                utc_end: UTCDate::from_timestamp(end_time),\n                busy_status: Some(current_status),\n                event: None,\n            });\n        }\n\n        Ok(result)\n    }\n}\n\nstruct FreeBusyResult {\n    utc_start: i64,\n    utc_end: i64,\n    busy_status: BusyStatus,\n    expansion_id: u32,\n    document_id: u32,\n}\n\nimpl From<FreeBusyResult> for BusyPeriod {\n    fn from(value: FreeBusyResult) -> Self {\n        BusyPeriod {\n            utc_start: UTCDate::from_timestamp(value.utc_start),\n            utc_end: UTCDate::from_timestamp(value.utc_end),\n            busy_status: Some(value.busy_status),\n            event: JSCalendar(Value::Object(Map::from(vec![\n                (\n                    Key::Property(JSCalendarProperty::Id),\n                    Value::Element(JSCalendarValue::Id(<Id as CalendarSyntheticId>::new(\n                        value.expansion_id,\n                        value.document_id,\n                    ))),\n                ),\n                (\n                    Key::Property(JSCalendarProperty::BaseEventId),\n                    Value::Element(JSCalendarValue::Id(Id::from(value.document_id))),\n                ),\n            ])))\n            .into(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/principal/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse directory::{Permission, QueryParams, Type, backend::internal::manage::ManageDirectory};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::principal::{Principal, PrincipalProperty, PrincipalType, PrincipalValue},\n    request::capability::Capability,\n    types::state::State,\n};\nuse jmap_tools::{Key, Map, Value};\nuse std::future::Future;\nuse store::roaring::RoaringBitmap;\nuse trc::AddContext;\n\npub trait PrincipalGet: Sync + Send {\n    fn principal_get(\n        &self,\n        request: GetRequest<Principal>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<Principal>>> + Send;\n}\n\nimpl PrincipalGet for Server {\n    async fn principal_get(\n        &self,\n        mut request: GetRequest<Principal>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<Principal>> {\n        if !self.core.groupware.allow_directory_query\n            && !access_token.has_permission(Permission::IndividualList)\n        {\n            return Err(trc::JmapEvent::Forbidden\n                .into_err()\n                .details(\"The administrator has disabled directory queries.\".to_string()));\n        }\n\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            PrincipalProperty::Id,\n            PrincipalProperty::Type,\n            PrincipalProperty::Name,\n            PrincipalProperty::Description,\n            PrincipalProperty::Email,\n        ]);\n\n        // Return all principals\n        let principal_ids = self\n            .store()\n            .list_principals(\n                None,\n                access_token.tenant_id(),\n                &[\n                    Type::Individual,\n                    Type::Group,\n                    Type::Resource,\n                    Type::Location,\n                ],\n                false,\n                0,\n                0,\n            )\n            .await\n            .caused_by(trc::location!())?\n            .items\n            .into_iter()\n            .map(|p| p.id())\n            .collect::<RoaringBitmap>();\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            principal_ids\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: State::Initial.into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the principal\n            let document_id = id.document_id();\n            let principal = if principal_ids.contains(document_id)\n                && let Some(principal) = self\n                    .core\n                    .storage\n                    .directory\n                    .query(QueryParams::id(document_id).with_return_member_of(false))\n                    .await?\n            {\n                principal\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                let value = match property {\n                    PrincipalProperty::Id => Value::Element(PrincipalValue::Id(id)),\n                    PrincipalProperty::Type => {\n                        Value::Element(PrincipalValue::Type(match principal.typ() {\n                            Type::Individual => PrincipalType::Individual,\n                            Type::Group => PrincipalType::Group,\n                            Type::Resource => PrincipalType::Resource,\n                            Type::Location => PrincipalType::Location,\n                            _ => PrincipalType::Other,\n                        }))\n                    }\n                    PrincipalProperty::Name => Value::Str(principal.name().to_string().into()),\n                    PrincipalProperty::Description => principal\n                        .description()\n                        .map(|v| Value::Str(v.to_string().into()))\n                        .unwrap_or(Value::Null),\n                    PrincipalProperty::Email => principal\n                        .primary_email()\n                        .map(|email| Value::Str(email.to_string().into()))\n                        .unwrap_or(Value::Null),\n                    PrincipalProperty::Accounts => Value::Object(Map::from(vec![(\n                        Key::Property(PrincipalProperty::IdValue(id)),\n                        Value::Object(Map::from_iter(\n                            [\n                                Capability::Mail,\n                                Capability::Contacts,\n                                Capability::Calendars,\n                                Capability::FileNode,\n                                Capability::Principals,\n                            ]\n                            .iter()\n                            .map(|cap| {\n                                (\n                                    Key::Property(PrincipalProperty::Capability(*cap)),\n                                    Value::Object(Map::new()),\n                                )\n                            })\n                            .chain([\n                                (\n                                    Key::Property(PrincipalProperty::Capability(\n                                        Capability::PrincipalsOwner,\n                                    )),\n                                    Value::Object(Map::from(vec![\n                                        (\n                                            Key::Borrowed(\"accountIdForPrincipal\"),\n                                            Value::Element(PrincipalValue::Id(id)),\n                                        ),\n                                        (\n                                            Key::Borrowed(\"principalId\"),\n                                            Value::Element(PrincipalValue::Id(id)),\n                                        ),\n                                    ])),\n                                ),\n                                (\n                                    Key::Property(PrincipalProperty::Capability(\n                                        Capability::Calendars,\n                                    )),\n                                    Value::Object(Map::from(vec![\n                                        (\n                                            Key::Borrowed(\"accountId\"),\n                                            Value::Element(PrincipalValue::Id(id)),\n                                        ),\n                                        (Key::Borrowed(\"mayGetAvailability\"), Value::Bool(true)),\n                                        (Key::Borrowed(\"mayShareWith\"), Value::Bool(true)),\n                                        (\n                                            Key::Borrowed(\"calendarAddress\"),\n                                            Value::Str(\n                                                principal\n                                                    .primary_email()\n                                                    .map(|email| format!(\"mailto:{}\", email))\n                                                    .unwrap_or_default()\n                                                    .into(),\n                                            ),\n                                        ),\n                                    ])),\n                                ),\n                            ]),\n                        )),\n                    )])),\n                    PrincipalProperty::Capabilities => Value::Object(Map::from_iter(\n                        [\n                            Capability::Mail,\n                            Capability::Contacts,\n                            Capability::Calendars,\n                            Capability::FileNode,\n                            Capability::Principals,\n                        ]\n                        .iter()\n                        .map(|cap| {\n                            (\n                                Key::Property(PrincipalProperty::Capability(*cap)),\n                                Value::Object(Map::new()),\n                            )\n                        }),\n                    )),\n                    _ => Value::Null,\n                };\n\n                result.insert_unchecked(property.clone(), value);\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/principal/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod availability;\npub mod get;\npub mod query;\n"
  },
  {
    "path": "crates/jmap/src/principal/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::api::query::QueryResponseBuilder;\nuse common::{Server, auth::AccessToken};\nuse directory::{Permission, QueryParams, Type, backend::internal::manage::ManageDirectory};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::principal::{Principal, PrincipalFilter, PrincipalType},\n    types::state::State,\n};\nuse std::future::Future;\nuse store::{\n    roaring::RoaringBitmap,\n    search::{SearchFilter, SearchQuery},\n    write::SearchIndex,\n};\nuse trc::AddContext;\n\npub trait PrincipalQuery: Sync + Send {\n    fn principal_query(\n        &self,\n        request: QueryRequest<Principal>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl PrincipalQuery for Server {\n    async fn principal_query(\n        &self,\n        mut request: QueryRequest<Principal>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> trc::Result<QueryResponse> {\n        if !self.core.groupware.allow_directory_query\n            && !access_token.has_permission(Permission::IndividualList)\n        {\n            return Err(trc::JmapEvent::Forbidden\n                .into_err()\n                .details(\"The administrator has disabled directory queries.\".to_string()));\n        }\n\n        let principal_ids = self\n            .store()\n            .list_principals(\n                None,\n                access_token.tenant_id(),\n                &[\n                    Type::Individual,\n                    Type::Group,\n                    Type::Resource,\n                    Type::Location,\n                ],\n                false,\n                0,\n                0,\n            )\n            .await\n            .caused_by(trc::location!())?\n            .items\n            .into_iter()\n            .map(|p| p.id())\n            .collect::<RoaringBitmap>();\n\n        let mut filters = Vec::with_capacity(request.filter.len());\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    PrincipalFilter::Name(name) => {\n                        if let Some(principal) = self\n                            .core\n                            .storage\n                            .directory\n                            .query(QueryParams::name(name.as_str()).with_return_member_of(false))\n                            .await?\n                        {\n                            filters.push(SearchFilter::is_in_set(\n                                RoaringBitmap::from_sorted_iter([principal.id()]).unwrap(),\n                            ));\n                        }\n                    }\n                    PrincipalFilter::Email(email) => {\n                        if let Some(id) = self\n                            .email_to_id(self.directory(), &email, session.session_id)\n                            .await?\n                        {\n                            filters.push(SearchFilter::is_in_set(\n                                RoaringBitmap::from_sorted_iter([id]).unwrap(),\n                            ));\n                        }\n                    }\n                    PrincipalFilter::AccountIds(ids) => {\n                        filters.push(SearchFilter::is_in_set(\n                            ids.into_iter()\n                                .filter_map(|id| {\n                                    let id = id.document_id();\n                                    if principal_ids.contains(id) {\n                                        Some(id)\n                                    } else {\n                                        None\n                                    }\n                                })\n                                .collect::<RoaringBitmap>(),\n                        ));\n                    }\n                    PrincipalFilter::Text(text) => {\n                        filters.push(SearchFilter::is_in_set(\n                            self.store()\n                                .list_principals(\n                                    Some(text.as_str()),\n                                    access_token.tenant.map(|t| t.id),\n                                    &[],\n                                    false,\n                                    0,\n                                    0,\n                                )\n                                .await?\n                                .items\n                                .into_iter()\n                                .map(|p| p.id())\n                                .collect::<RoaringBitmap>(),\n                        ));\n                    }\n                    PrincipalFilter::Type(principal_type) => {\n                        let typ = match principal_type {\n                            PrincipalType::Individual => Type::Individual,\n                            PrincipalType::Group => Type::Group,\n                            PrincipalType::Resource => Type::Resource,\n                            PrincipalType::Location => Type::Location,\n                            PrincipalType::Other => Type::Other,\n                        };\n\n                        filters.push(SearchFilter::is_in_set(\n                            self.store()\n                                .list_principals(\n                                    None,\n                                    access_token.tenant.map(|t| t.id),\n                                    &[typ],\n                                    false,\n                                    0,\n                                    0,\n                                )\n                                .await?\n                                .items\n                                .into_iter()\n                                .map(|p| p.id())\n                                .collect::<RoaringBitmap>(),\n                        ));\n                    }\n                    other => {\n                        return Err(trc::JmapEvent::UnsupportedFilter\n                            .into_err()\n                            .details(other.to_string()));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        let results = SearchQuery::new(SearchIndex::InMemory)\n            .with_filters(filters)\n            .with_mask(principal_ids)\n            .filter()\n            .into_bitmap();\n\n        let mut response = QueryResponseBuilder::new(\n            results.len() as usize,\n            self.core.jmap.query_max_results,\n            State::Initial,\n            &request,\n        );\n\n        for document_id in results {\n            if !response.add(0, document_id) {\n                break;\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/push/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken, ipc::PushEvent};\nuse email::push::PushSubscriptions;\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::push_subscription::{self, PushSubscriptionProperty, PushSubscriptionValue},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Map, Value};\nuse std::future::Future;\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, now},\n};\nuse trc::{AddContext, ServerEvent};\nuse types::{collection::Collection, field::PrincipalField, id::Id};\nuse utils::map::bitmap::Bitmap;\n\npub trait PushSubscriptionFetch: Sync + Send {\n    fn push_subscription_get(\n        &self,\n        request: GetRequest<push_subscription::PushSubscription>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<push_subscription::PushSubscription>>> + Send;\n}\n\nimpl PushSubscriptionFetch for Server {\n    async fn push_subscription_get(\n        &self,\n        mut request: GetRequest<push_subscription::PushSubscription>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<push_subscription::PushSubscription>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            PushSubscriptionProperty::Id,\n            PushSubscriptionProperty::DeviceClientId,\n            PushSubscriptionProperty::VerificationCode,\n            PushSubscriptionProperty::Expires,\n            PushSubscriptionProperty::Types,\n        ]);\n\n        let account_id = access_token.primary_id();\n\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: None,\n            list: Vec::new(),\n            not_found: vec![],\n        };\n\n        let Some(subscriptions_) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                account_id,\n                Collection::Principal,\n                0,\n                PrincipalField::PushSubscriptions,\n            ))\n            .await?\n        else {\n            for id in ids.unwrap_or_default() {\n                response.not_found.push(id);\n            }\n            return Ok(response);\n        };\n        let subscriptions = subscriptions_\n            .to_unarchived::<PushSubscriptions>()\n            .caused_by(trc::location!())?;\n\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            subscriptions\n                .inner\n                .subscriptions\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(|s| Id::from(s.id.to_native()))\n                .collect::<Vec<_>>()\n        };\n\n        for id in ids {\n            // Obtain the push subscription object\n            let document_id = id.document_id();\n            let Some(push) = subscriptions\n                .inner\n                .subscriptions\n                .iter()\n                .find(|p| p.id.to_native() == document_id)\n            else {\n                response.not_found.push(id);\n                continue;\n            };\n\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                match property {\n                    PushSubscriptionProperty::Id => {\n                        result.insert_unchecked(PushSubscriptionProperty::Id, id);\n                    }\n                    PushSubscriptionProperty::Url | PushSubscriptionProperty::Keys => {\n                        return Err(trc::JmapEvent::Forbidden.into_err().details(\n                            \"The 'url' and 'keys' properties are not readable\".to_string(),\n                        ));\n                    }\n                    PushSubscriptionProperty::DeviceClientId => {\n                        result.insert_unchecked(\n                            PushSubscriptionProperty::DeviceClientId,\n                            &push.device_client_id,\n                        );\n                    }\n                    PushSubscriptionProperty::Types => {\n                        let mut types = Vec::new();\n                        for typ in Bitmap::from(&push.types).into_iter() {\n                            types.push(Value::Element(PushSubscriptionValue::Types(typ)));\n                        }\n                        result\n                            .insert_unchecked(PushSubscriptionProperty::Types, Value::Array(types));\n                    }\n                    PushSubscriptionProperty::Expires => {\n                        if push.expires > 0 {\n                            result.insert_unchecked(\n                                PushSubscriptionProperty::Expires,\n                                Value::Element(PushSubscriptionValue::Date(\n                                    UTCDate::from_timestamp(u64::from(push.expires) as i64),\n                                )),\n                            );\n                        } else {\n                            result.insert_unchecked(PushSubscriptionProperty::Expires, Value::Null);\n                        }\n                    }\n                    property => {\n                        result.insert_unchecked(property.clone(), Value::Null);\n                    }\n                }\n            }\n            response.list.push(result.into());\n        }\n\n        // Purge old subscriptions\n        let current_time = now();\n        if subscriptions\n            .inner\n            .subscriptions\n            .iter()\n            .any(|s| s.expires.to_native() < current_time)\n        {\n            let mut updated_subscriptions = subscriptions.deserialize::<PushSubscriptions>()?;\n            updated_subscriptions\n                .subscriptions\n                .retain(|s| s.expires >= current_time);\n            let mut batch = BatchBuilder::new();\n\n            if updated_subscriptions.subscriptions.is_empty() {\n                batch\n                    .with_account_id(u32::MAX)\n                    .with_collection(Collection::Principal)\n                    .with_account_id(account_id)\n                    .tag(PrincipalField::PushSubscriptions);\n            }\n\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .assert_value(PrincipalField::PushSubscriptions, subscriptions);\n\n            if !updated_subscriptions.subscriptions.is_empty() {\n                batch.set(\n                    PrincipalField::PushSubscriptions,\n                    Archiver::new(updated_subscriptions)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n            } else {\n                batch.clear(PrincipalField::PushSubscriptions);\n            }\n\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n            // Update push servers\n            if self\n                .inner\n                .ipc\n                .push_tx\n                .clone()\n                .send(PushEvent::PushServerUpdate {\n                    account_id,\n                    broadcast: true,\n                })\n                .await\n                .is_err()\n            {\n                trc::event!(\n                    Server(ServerEvent::ThreadError),\n                    Details = \"Error sending push updates.\",\n                    CausedBy = trc::location!()\n                );\n            }\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/push/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/push/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse base64::{Engine, engine::general_purpose};\nuse common::{Server, auth::AccessToken, ipc::PushEvent};\nuse email::push::{Keys, PushSubscription, PushSubscriptions};\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::push_subscription::{self, PushSubscriptionProperty, PushSubscriptionValue},\n    references::resolve::ResolveCreatedReference,\n    request::IntoValid,\n    types::date::UTCDate,\n};\nuse jmap_tools::{Key, Map, Value};\nuse rand::distr::Alphanumeric;\nuse std::future::Future;\nuse store::{\n    Serialize, ValueKey,\n    rand::{Rng, rng},\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, now},\n};\nuse trc::{AddContext, ServerEvent};\nuse types::{collection::Collection, field::PrincipalField};\nuse utils::map::bitmap::Bitmap;\n\nconst EXPIRES_MAX: i64 = 7 * 24 * 3600; // 7 days\nconst VERIFICATION_CODE_LEN: usize = 32;\n\npub trait PushSubscriptionSet: Sync + Send {\n    fn push_subscription_set(\n        &self,\n        request: SetRequest<'_, push_subscription::PushSubscription>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<SetResponse<push_subscription::PushSubscription>>> + Send;\n}\n\nimpl PushSubscriptionSet for Server {\n    async fn push_subscription_set(\n        &self,\n        mut request: SetRequest<'_, push_subscription::PushSubscription>,\n        access_token: &AccessToken,\n    ) -> trc::Result<SetResponse<push_subscription::PushSubscription>> {\n        // Load existing push subscriptions\n        let account_id = access_token.primary_id();\n        let subscriptions_archive = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                account_id,\n                Collection::Principal,\n                0,\n                PrincipalField::PushSubscriptions,\n            ))\n            .await?;\n        let mut subscriptions = if let Some(subscriptions) = &subscriptions_archive {\n            subscriptions\n                .deserialize::<PushSubscriptions>()\n                .caused_by(trc::location!())?\n        } else {\n            PushSubscriptions::default()\n        };\n\n        let num_subscriptions = subscriptions.subscriptions.len();\n        let mut max_id = 0;\n        let current_time = now();\n        subscriptions.subscriptions.retain(|s| {\n            max_id = max_id.max(s.id);\n\n            s.expires > current_time\n        });\n        let mut has_changes = num_subscriptions != subscriptions.subscriptions.len();\n\n        // Prepare response\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Process creates\n        'create: for (id, object) in request.unwrap_create() {\n            let mut push = PushSubscription::default();\n\n            if subscriptions.subscriptions.len()\n                >= access_token.object_quota(Collection::PushSubscription) as usize\n            {\n                response.not_created.append(id, SetError::new(SetErrorType::OverQuota).with_description(\n                    \"There are too many subscriptions, please delete some before adding a new one.\",\n                ));\n                continue 'create;\n            }\n\n            for (property, mut value) in object.into_expanded_object() {\n                if let Err(err) = response\n                    .resolve_self_references(&mut value)\n                    .and_then(|_| validate_push_value(&property, value, &mut push, true))\n                {\n                    response.not_created.append(id, err);\n                    continue 'create;\n                }\n            }\n\n            if push.device_client_id.is_empty() || push.url.is_empty() {\n                response.not_created.append(\n                    id,\n                    SetError::invalid_properties()\n                        .with_properties([\n                            PushSubscriptionProperty::DeviceClientId,\n                            PushSubscriptionProperty::Url,\n                        ])\n                        .with_description(\"Missing required properties\"),\n                );\n                continue 'create;\n            }\n\n            // Add expiry time if missing\n            if push.expires == 0 {\n                push.expires = now() + EXPIRES_MAX as u64;\n            }\n            let expires = UTCDate::from_timestamp(push.expires as i64);\n\n            // Generate random verification code\n            push.verification_code = rng()\n                .sample_iter(Alphanumeric)\n                .take(VERIFICATION_CODE_LEN)\n                .map(char::from)\n                .collect::<String>();\n\n            // Set id\n            max_id += 1;\n            let document_id = max_id;\n            push.id = document_id;\n\n            // Insert record\n            subscriptions.subscriptions.push(push);\n            response.created.insert(\n                id,\n                Map::with_capacity(1)\n                    .with_key_value(\n                        PushSubscriptionProperty::Id,\n                        PushSubscriptionValue::Id(document_id.into()),\n                    )\n                    .with_key_value(PushSubscriptionProperty::Keys, Value::Null)\n                    .with_key_value(\n                        PushSubscriptionProperty::Expires,\n                        PushSubscriptionValue::Date(expires),\n                    )\n                    .into(),\n            );\n            has_changes = true;\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain push subscription\n            let document_id = id.document_id();\n            let Some(push) = subscriptions\n                .subscriptions\n                .iter_mut()\n                .find(|p| p.id == document_id)\n            else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n\n            for (property, mut value) in object.into_expanded_object() {\n                if let Err(err) = response\n                    .resolve_self_references(&mut value)\n                    .and_then(|_| validate_push_value(&property, value, push, false))\n                {\n                    response.not_updated.append(id, err);\n                    continue 'update;\n                }\n            }\n\n            has_changes = true;\n            response.updated.append(id, None);\n        }\n\n        // Process deletions\n        for id in will_destroy {\n            let document_id = id.document_id();\n            if let Some(idx) = subscriptions\n                .subscriptions\n                .iter()\n                .position(|p| p.id == document_id)\n            {\n                subscriptions.subscriptions.swap_remove(idx);\n                has_changes = true;\n                response.destroyed.push(id);\n            } else {\n                response.not_destroyed.append(id, SetError::not_found());\n            }\n        }\n\n        // Update push subscriptions\n        if has_changes {\n            // Save changes\n            let mut batch = BatchBuilder::new();\n\n            if subscriptions_archive.is_none() {\n                batch\n                    .with_account_id(u32::MAX)\n                    .with_collection(Collection::Principal)\n                    .with_document(account_id)\n                    .tag(PrincipalField::PushSubscriptions);\n            } else if subscriptions.subscriptions.is_empty() {\n                batch\n                    .with_account_id(u32::MAX)\n                    .with_collection(Collection::Principal)\n                    .with_document(account_id)\n                    .untag(PrincipalField::PushSubscriptions);\n            }\n\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0);\n\n            if let Some(subscriptions_archive) = subscriptions_archive {\n                batch.assert_value(PrincipalField::PushSubscriptions, subscriptions_archive);\n            }\n\n            if !subscriptions.subscriptions.is_empty() {\n                batch.set(\n                    PrincipalField::PushSubscriptions,\n                    Archiver::new(subscriptions)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n            } else {\n                batch.clear(PrincipalField::PushSubscriptions);\n            }\n\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n\n            // Notify push manager\n            if self\n                .inner\n                .ipc\n                .push_tx\n                .clone()\n                .send(PushEvent::PushServerUpdate {\n                    account_id,\n                    broadcast: true,\n                })\n                .await\n                .is_err()\n            {\n                trc::event!(\n                    Server(ServerEvent::ThreadError),\n                    Details = \"Error sending push updates.\",\n                    CausedBy = trc::location!()\n                );\n            }\n        }\n\n        Ok(response)\n    }\n}\n\nfn validate_push_value(\n    property: &Key<PushSubscriptionProperty>,\n    value: Value<'_, PushSubscriptionProperty, PushSubscriptionValue>,\n    push: &mut PushSubscription,\n    is_create: bool,\n) -> Result<(), SetError<PushSubscriptionProperty>> {\n    let Key::Property(property) = property else {\n        return Err(SetError::invalid_properties()\n            .with_property(property.to_owned())\n            .with_description(\"Invalid property.\"));\n    };\n\n    match (property, value) {\n        (PushSubscriptionProperty::DeviceClientId, Value::Str(value))\n            if is_create && value.len() < 255 =>\n        {\n            push.device_client_id = value.into_owned();\n        }\n        (PushSubscriptionProperty::Url, Value::Str(value))\n            if is_create && value.len() < 512 && value.starts_with(\"https://\") =>\n        {\n            push.url = value.into_owned();\n        }\n        (PushSubscriptionProperty::Keys, Value::Object(value)) if is_create && value.len() == 2 => {\n            if let (Some(auth), Some(p256dh)) = (\n                value\n                    .get(&Key::Property(PushSubscriptionProperty::Auth))\n                    .and_then(|v| v.as_str())\n                    .and_then(|v| general_purpose::URL_SAFE.decode(v.as_ref()).ok()),\n                value\n                    .get(&Key::Property(PushSubscriptionProperty::P256dh))\n                    .and_then(|v| v.as_str())\n                    .and_then(|v| general_purpose::URL_SAFE.decode(v.as_ref()).ok()),\n            ) {\n                push.keys = Some(Keys { auth, p256dh });\n            } else {\n                return Err(SetError::invalid_properties()\n                    .with_property(property.clone())\n                    .with_description(\"Failed to decode keys.\"));\n            }\n        }\n        (PushSubscriptionProperty::Expires, Value::Element(PushSubscriptionValue::Date(value))) => {\n            let current_time = now() as i64;\n            let expires = value.timestamp();\n            push.expires = if expires > current_time && (expires - current_time) > EXPIRES_MAX {\n                current_time + EXPIRES_MAX\n            } else {\n                expires\n            } as u64;\n        }\n        (PushSubscriptionProperty::Expires, Value::Null) => {\n            push.expires = now() + EXPIRES_MAX as u64;\n        }\n        (PushSubscriptionProperty::Types, Value::Array(value)) => {\n            push.types.clear();\n\n            for item in value {\n                if let Value::Element(PushSubscriptionValue::Types(dt)) = item {\n                    push.types.insert(dt);\n                } else {\n                    return Err(SetError::invalid_properties()\n                        .with_property(property.clone())\n                        .with_description(\"Invalid data type.\"));\n                }\n            }\n        }\n        (PushSubscriptionProperty::VerificationCode, Value::Str(value)) if !is_create => {\n            if push.verification_code == value {\n                push.verified = true;\n            } else {\n                return Err(SetError::invalid_properties()\n                    .with_property(property.clone())\n                    .with_description(\"Verification code does not match.\".to_string()));\n            }\n        }\n        (PushSubscriptionProperty::Keys, Value::Null) => {\n            push.keys = None;\n        }\n        (PushSubscriptionProperty::Types, Value::Null) => {\n            push.types = Bitmap::all();\n        }\n        (PushSubscriptionProperty::VerificationCode, Value::Null) => {}\n        (property, _) => {\n            return Err(SetError::invalid_properties()\n                .with_property(property.clone())\n                .with_description(\"Field could not be set.\"));\n        }\n    }\n\n    if is_create && push.types.is_empty() {\n        push.types = Bitmap::all();\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/jmap/src/quota/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::quota::{Quota, QuotaProperty, QuotaValue},\n    types::state::State,\n};\nuse jmap_tools::{Map, Value};\nuse std::{future::Future, sync::Arc};\nuse trc::AddContext;\nuse types::{id::Id, type_state::DataType};\n\npub trait QuotaGet: Sync + Send {\n    fn quota_get(\n        &self,\n        request: GetRequest<Quota>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<GetResponse<Quota>>> + Send;\n}\n\nimpl QuotaGet for Server {\n    async fn quota_get(\n        &self,\n        mut request: GetRequest<Quota>,\n        access_token: &AccessToken,\n    ) -> trc::Result<GetResponse<Quota>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            QuotaProperty::Id,\n            QuotaProperty::ResourceType,\n            QuotaProperty::Used,\n            QuotaProperty::WarnLimit,\n            QuotaProperty::SoftLimit,\n            QuotaProperty::HardLimit,\n            QuotaProperty::Scope,\n            QuotaProperty::Name,\n            QuotaProperty::Description,\n            QuotaProperty::Types,\n        ]);\n        let account_id = request.account_id.document_id();\n        let quota_ids = if access_token.quota > 0 {\n            vec![0u32]\n        } else {\n            vec![]\n        };\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            quota_ids.iter().map(|id| Id::from(*id)).collect()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: State::Initial.into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        let access_token = if account_id == access_token.primary_id() {\n            AccessTokenRef::Borrowed(access_token)\n        } else {\n            AccessTokenRef::Owned(\n                self.get_access_token(account_id)\n                    .await\n                    .caused_by(trc::location!())?,\n            )\n        };\n\n        for id in ids {\n            // Obtain the sieve script object\n            let document_id = id.document_id();\n            if !quota_ids.contains(&document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                let value = match property {\n                    QuotaProperty::Id => Value::Element(id.into()),\n                    QuotaProperty::ResourceType => \"octets\".to_string().into(),\n                    QuotaProperty::Used => (self.get_used_quota(account_id).await? as u64).into(),\n                    QuotaProperty::HardLimit => access_token.as_ref().quota.into(),\n                    QuotaProperty::Scope => \"account\".to_string().into(),\n                    QuotaProperty::Name => access_token.as_ref().name.to_string().into(),\n                    QuotaProperty::Description => access_token\n                        .as_ref()\n                        .description\n                        .as_ref()\n                        .map(|s| s.to_string())\n                        .into(),\n                    QuotaProperty::Types => vec![\n                        Value::Element(QuotaValue::Types(DataType::Email)),\n                        Value::Element(QuotaValue::Types(DataType::SieveScript)),\n                        Value::Element(QuotaValue::Types(DataType::FileNode)),\n                        Value::Element(QuotaValue::Types(DataType::CalendarEvent)),\n                        Value::Element(QuotaValue::Types(DataType::ContactCard)),\n                    ]\n                    .into(),\n\n                    _ => Value::Null,\n                };\n                result.insert_unchecked(property.clone(), value);\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n\nenum AccessTokenRef<'x> {\n    Owned(Arc<AccessToken>),\n    Borrowed(&'x AccessToken),\n}\n\nimpl AccessTokenRef<'_> {\n    fn as_ref(&self) -> &AccessToken {\n        match self {\n            AccessTokenRef::Owned(token) => token,\n            AccessTokenRef::Borrowed(token) => token,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/quota/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\n"
  },
  {
    "path": "crates/jmap/src/quota/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken};\nuse jmap_proto::{\n    method::query::{QueryRequest, QueryResponse},\n    object::quota::Quota,\n    types::state::State,\n};\nuse std::future::Future;\nuse types::id::Id;\n\npub trait QuotaQuery: Sync + Send {\n    fn quota_query(\n        &self,\n        request: QueryRequest<Quota>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl QuotaQuery for Server {\n    async fn quota_query(\n        &self,\n        request: QueryRequest<Quota>,\n        access_token: &AccessToken,\n    ) -> trc::Result<QueryResponse> {\n        Ok(QueryResponse {\n            account_id: request.account_id,\n            query_state: State::Initial,\n            can_calculate_changes: false,\n            position: 0,\n            ids: if access_token.quota > 0 {\n                vec![Id::new(0)]\n            } else {\n                vec![]\n            },\n            total: Some(1),\n            limit: None,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/share_notification/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, auth::AccessToken, sharing::notification::ShareNotification};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::{\n        JmapRight,\n        addressbook::AddressBookRight,\n        calendar::CalendarRight,\n        file_node::FileNodeRight,\n        mailbox::MailboxRight,\n        share_notification::{self, ShareNotificationProperty, ShareNotificationValue},\n    },\n    request::IntoValid,\n    types::{date::UTCDate, state::State},\n};\nuse jmap_tools::{Key, Map, Value};\nuse std::{sync::Arc, time::Duration};\nuse store::{\n    Deserialize, IterateParams, LogKey, U64_LEN,\n    ahash::{AHashMap, AHashSet},\n    write::key::DeserializeBigEndian,\n};\nuse trc::AddContext;\nuse types::{\n    acl::Acl,\n    collection::{Collection, SyncCollection},\n    id::Id,\n    type_state::DataType,\n};\nuse utils::{map::bitmap::Bitmap, snowflake::SnowflakeIdGenerator};\n\npub trait ShareNotificationGet: Sync + Send {\n    fn share_notification_get(\n        &self,\n        request: GetRequest<share_notification::ShareNotification>,\n    ) -> impl Future<Output = trc::Result<GetResponse<share_notification::ShareNotification>>> + Send;\n}\n\nimpl ShareNotificationGet for Server {\n    async fn share_notification_get(\n        &self,\n        mut request: GetRequest<share_notification::ShareNotification>,\n    ) -> trc::Result<GetResponse<share_notification::ShareNotification>> {\n        let properties = request.unwrap_properties(&[\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::Name,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::Created,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ]);\n\n        let account_id = request.account_id.document_id();\n        let mut min_id = u64::MAX;\n        let mut max_id = 0u64;\n\n        let mut token_cache: AHashMap<u32, Arc<AccessToken>> = AHashMap::new();\n\n        let mut ids = if let Some(ids) = request.ids.take() {\n            let ids = ids.unwrap();\n            if ids.len() <= self.core.jmap.get_max_objects {\n                ids.into_valid()\n                    .map(|id| {\n                        let id_num = *id.as_ref();\n                        if id_num < min_id {\n                            min_id = id_num;\n                        }\n                        if id_num > max_id {\n                            max_id = id_num;\n                        }\n                        id_num\n                    })\n                    .collect::<AHashSet<_>>()\n            } else {\n                return Err(trc::JmapEvent::RequestTooLarge.into_err());\n            }\n        } else {\n            AHashSet::new()\n        };\n        let has_ids = !ids.is_empty();\n\n        if min_id == u64::MAX {\n            min_id = SnowflakeIdGenerator::from_duration(\n                self.core\n                    .jmap\n                    .share_notification_max_history\n                    .unwrap_or(Duration::from_secs(30 * 86400)),\n            )\n            .unwrap_or_default();\n        }\n\n        if max_id == 0 {\n            max_id = u64::MAX;\n        }\n\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: None,\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n        let mut notifications = Vec::new();\n\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    LogKey {\n                        account_id,\n                        collection: SyncCollection::ShareNotification.into(),\n                        change_id: min_id,\n                    },\n                    LogKey {\n                        account_id,\n                        collection: SyncCollection::ShareNotification.into(),\n                        change_id: max_id.saturating_add(1),\n                    },\n                )\n                .descending(),\n                |key, value| {\n                    let change_id = key.deserialize_be_u64(key.len() - U64_LEN)?;\n                    if response.state.is_none() {\n                        response.state = Some(State::Exact(change_id));\n                    }\n\n                    if !has_ids || ids.remove(&change_id) {\n                        notifications.push((\n                            change_id,\n                            ShareNotification::deserialize(value).caused_by(trc::location!())?,\n                        ));\n                    }\n\n                    Ok((!has_ids || !ids.is_empty())\n                        && notifications.len() < self.core.jmap.get_max_objects)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        for (change_id, notification) in notifications {\n            let changed_by_token = if let Some(token) = token_cache.get(&notification.changed_by) {\n                token.clone()\n            } else {\n                let token = if let Ok(token) = self.get_access_token(notification.changed_by).await\n                {\n                    token\n                } else {\n                    Arc::new(AccessToken::from_id(notification.changed_by))\n                };\n\n                token_cache.insert(notification.changed_by, token.clone());\n                token\n            };\n\n            response.list.push(build_share_notification(\n                change_id,\n                notification,\n                &changed_by_token,\n                &properties,\n            ));\n        }\n\n        if response.state.is_none() {\n            response.state = Some(State::Initial);\n        }\n\n        response\n            .not_found\n            .extend(ids.into_iter().map(Id::from).collect::<Vec<_>>());\n\n        Ok(response)\n    }\n}\n\nfn build_share_notification(\n    id: u64,\n    mut notification: ShareNotification,\n    changed_by: &AccessToken,\n    properties: &[ShareNotificationProperty],\n) -> Value<'static, ShareNotificationProperty, ShareNotificationValue> {\n    let mut result = Map::with_capacity(properties.len());\n    for property in properties {\n        let value = match property {\n            ShareNotificationProperty::Id => Value::Element(ShareNotificationValue::Id(id.into())),\n            ShareNotificationProperty::Created => Value::Element(ShareNotificationValue::Date(\n                UTCDate::from_timestamp(SnowflakeIdGenerator::to_timestamp(id) as i64),\n            )),\n            ShareNotificationProperty::ChangedBy => Value::Object(Map::from(vec![\n                (\n                    Key::Property(ShareNotificationProperty::ChangedByPrincipalId),\n                    Value::Element(ShareNotificationValue::Id(notification.changed_by.into())),\n                ),\n                (\n                    Key::Property(ShareNotificationProperty::ChangedByName),\n                    Value::Str(\n                        changed_by\n                            .description\n                            .as_deref()\n                            .unwrap_or(changed_by.name.as_str())\n                            .to_string()\n                            .into(),\n                    ),\n                ),\n                (\n                    Key::Property(ShareNotificationProperty::ChangedByEmail),\n                    changed_by\n                        .emails\n                        .first()\n                        .map_or(Value::Null, |email| Value::Str(email.to_string().into())),\n                ),\n            ])),\n            ShareNotificationProperty::ObjectType => DataType::try_from(notification.object_type)\n                .ok()\n                .map(|typ| Value::Element(ShareNotificationValue::ObjectType(typ)))\n                .unwrap_or(Value::Null),\n            ShareNotificationProperty::ObjectAccountId => Value::Element(\n                ShareNotificationValue::Id(notification.object_account_id.into()),\n            ),\n            ShareNotificationProperty::ObjectId => {\n                Value::Element(ShareNotificationValue::Id(notification.object_id.into()))\n            }\n            ShareNotificationProperty::OldRights => {\n                map_rights(notification.object_type, notification.old_rights)\n            }\n            ShareNotificationProperty::NewRights => {\n                map_rights(notification.object_type, notification.new_rights)\n            }\n            ShareNotificationProperty::Name => {\n                Value::Str(std::mem::take(&mut notification.name).into())\n            }\n            _ => Value::Null,\n        };\n\n        result.insert_unchecked(property.clone(), value);\n    }\n\n    Value::Object(result)\n}\n\nfn map_rights(\n    object_type: Collection,\n    rights: Bitmap<Acl>,\n) -> Value<'static, ShareNotificationProperty, ShareNotificationValue> {\n    let mut obj = Map::with_capacity(3);\n\n    match object_type {\n        Collection::Calendar | Collection::CalendarEvent => {\n            for right in CalendarRight::all_rights() {\n                obj.insert_unchecked(\n                    Key::Borrowed(right.as_str()),\n                    Value::Bool(right.to_acl().iter().all(|acl| rights.contains(*acl))),\n                );\n            }\n        }\n        Collection::AddressBook | Collection::ContactCard => {\n            for right in AddressBookRight::all_rights() {\n                obj.insert_unchecked(\n                    Key::Borrowed(right.as_str()),\n                    Value::Bool(right.to_acl().iter().all(|acl| rights.contains(*acl))),\n                );\n            }\n        }\n        Collection::FileNode => {\n            for right in FileNodeRight::all_rights() {\n                obj.insert_unchecked(\n                    Key::Borrowed(right.as_str()),\n                    Value::Bool(right.to_acl().iter().all(|acl| rights.contains(*acl))),\n                );\n            }\n        }\n        Collection::Mailbox | Collection::Email => {\n            for right in MailboxRight::all_rights() {\n                obj.insert_unchecked(\n                    Key::Borrowed(right.as_str()),\n                    Value::Bool(right.to_acl().iter().all(|acl| rights.contains(*acl))),\n                );\n            }\n        }\n        _ => {}\n    }\n\n    Value::Object(obj)\n}\n"
  },
  {
    "path": "crates/jmap/src/share_notification/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/share_notification/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::api::query::QueryResponseBuilder;\nuse common::{Server, sharing::notification::ShareNotification};\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::share_notification::{self, ShareNotificationFilter},\n    types::state::State,\n};\nuse std::time::Duration;\nuse store::{Deserialize, IterateParams, LogKey, U64_LEN, write::key::DeserializeBigEndian};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\nuse utils::snowflake::SnowflakeIdGenerator;\n\npub trait ShareNotificationQuery: Sync + Send {\n    fn share_notification_query(\n        &self,\n        request: QueryRequest<share_notification::ShareNotification>,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl ShareNotificationQuery for Server {\n    async fn share_notification_query(\n        &self,\n        mut request: QueryRequest<share_notification::ShareNotification>,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let mut from_change_id = SnowflakeIdGenerator::from_duration(\n            self.core\n                .jmap\n                .share_notification_max_history\n                .unwrap_or(Duration::from_secs(30 * 86400)),\n        )\n        .unwrap_or_default();\n        let mut to_change_id = u64::MAX;\n        let mut collection = None;\n        let mut object_type = None;\n\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    ShareNotificationFilter::After(utcdate) => {\n                        from_change_id =\n                            SnowflakeIdGenerator::from_timestamp(utcdate.timestamp() as u64)\n                                .unwrap_or(0);\n                    }\n                    ShareNotificationFilter::Before(utcdate) => {\n                        to_change_id =\n                            SnowflakeIdGenerator::from_timestamp(utcdate.timestamp() as u64)\n                                .unwrap_or(u64::MAX);\n                    }\n                    ShareNotificationFilter::ObjectType(typ) => {\n                        collection = Collection::try_from(typ).ok();\n                    }\n                    ShareNotificationFilter::ObjectAccountId(id) => {\n                        object_type = Some(id.document_id());\n                    }\n                    ShareNotificationFilter::_T(other) => {\n                        return Err(trc::JmapEvent::UnsupportedFilter.into_err().details(other));\n                    }\n                },\n                Filter::And | Filter::Or | Filter::Not | Filter::Close => {\n                    return Err(trc::JmapEvent::UnsupportedFilter\n                        .into_err()\n                        .details(\"Logical operators are not supported\"));\n                }\n            }\n        }\n\n        let mut results = Vec::new();\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    LogKey {\n                        account_id,\n                        collection: SyncCollection::ShareNotification.into(),\n                        change_id: from_change_id,\n                    },\n                    LogKey {\n                        account_id,\n                        collection: SyncCollection::ShareNotification.into(),\n                        change_id: to_change_id,\n                    },\n                )\n                .descending(),\n                |key, value| {\n                    let change_id = key.deserialize_be_u64(key.len() - U64_LEN)?;\n\n                    if collection.is_some() || object_type.is_some() {\n                        let notification =\n                            ShareNotification::deserialize(value).caused_by(trc::location!())?;\n                        if collection.is_some_and(|c| c != notification.object_type)\n                            || object_type.is_some_and(|o| o != notification.object_account_id)\n                        {\n                            return Ok(true);\n                        }\n                    }\n\n                    results.push(Id::from(change_id));\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        let mut response = QueryResponseBuilder::new(\n            results.len(),\n            self.core.jmap.query_max_results,\n            State::Initial,\n            &request,\n        );\n\n        for id in results {\n            if !response.add_id(id) {\n                break;\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/share_notification/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse jmap_proto::{\n    error::set::SetError,\n    method::set::{SetRequest, SetResponse},\n    object::share_notification::ShareNotification,\n    request::IntoValid,\n};\nuse store::write::{BatchBuilder, ValueClass};\nuse trc::AddContext;\n\npub trait ShareNotificationSet: Sync + Send {\n    fn share_notification_set(\n        &self,\n        request: SetRequest<'_, ShareNotification>,\n    ) -> impl Future<Output = trc::Result<SetResponse<ShareNotification>>> + Send;\n}\n\nimpl ShareNotificationSet for Server {\n    async fn share_notification_set(\n        &self,\n        mut request: SetRequest<'_, ShareNotification>,\n    ) -> trc::Result<SetResponse<ShareNotification>> {\n        let account_id = request.account_id.document_id();\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n\n        for (id, _) in request.unwrap_create() {\n            response.not_created.append(\n                id,\n                SetError::forbidden().with_description(\"Cannot create share notifications.\"),\n            );\n        }\n\n        // Process updates\n        for (id, _) in request.unwrap_update().into_valid() {\n            response.not_updated.append(\n                id,\n                SetError::forbidden().with_description(\"Cannot update share notifications.\"),\n            );\n        }\n\n        // Process deletions\n        let mut batch = BatchBuilder::new();\n        batch.with_account_id(account_id);\n        for id in request.unwrap_destroy().into_valid() {\n            batch.clear(ValueClass::ShareNotification {\n                notification_id: id.id(),\n                notify_account_id: account_id,\n            });\n            response.destroyed.push(id);\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            self.commit_batch(batch).await.caused_by(trc::location!())?;\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/sieve/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::changes::state::StateManager;\nuse common::Server;\nuse email::sieve::{SieveScript, ingest::SieveScriptIngest};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::sieve::{Sieve, SieveProperty, SieveValue},\n};\nuse jmap_tools::{Map, Value};\nuse store::{ValueKey, write::{AlignedBytes, Archive}};\nuse std::future::Future;\nuse trc::AddContext;\nuse types::{\n    blob::{BlobClass, BlobId, BlobSection},\n    collection::{Collection, SyncCollection},\n    field::SieveField,\n};\n\npub trait SieveScriptGet: Sync + Send {\n    fn sieve_script_get(\n        &self,\n        request: GetRequest<Sieve>,\n    ) -> impl Future<Output = trc::Result<GetResponse<Sieve>>> + Send;\n}\n\nimpl SieveScriptGet for Server {\n    async fn sieve_script_get(\n        &self,\n        mut request: GetRequest<Sieve>,\n    ) -> trc::Result<GetResponse<Sieve>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            SieveProperty::Id,\n            SieveProperty::Name,\n            SieveProperty::BlobId,\n            SieveProperty::IsActive,\n        ]);\n        let account_id = request.account_id.document_id();\n        let script_ids = self\n            .document_ids(account_id, Collection::SieveScript, SieveField::Name)\n            .await?;\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            script_ids\n                .iter()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect::<Vec<_>>()\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: self\n                .get_state(account_id, SyncCollection::SieveScript)\n                .await?\n                .into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n        let active_script_id = self.sieve_script_get_active_id(account_id).await?;\n\n        for id in ids {\n            // Obtain the sieve script object\n            let document_id = id.document_id();\n            if !script_ids.contains(document_id) {\n                response.not_found.push(id);\n                continue;\n            }\n            let sieve_ = if let Some(sieve) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::SieveScript,\n                    document_id,\n                ))\n                .await?\n            {\n                sieve\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n            let sieve = sieve_\n                .unarchive::<SieveScript>()\n                .caused_by(trc::location!())?;\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                match property {\n                    SieveProperty::Id => {\n                        result.insert_unchecked(SieveProperty::Id, id);\n                    }\n                    SieveProperty::Name => {\n                        result.insert_unchecked(SieveProperty::Name, &sieve.name);\n                    }\n                    SieveProperty::IsActive => {\n                        result.insert_unchecked(\n                            SieveProperty::IsActive,\n                            active_script_id == Some(document_id),\n                        );\n                    }\n                    SieveProperty::BlobId => {\n                        let blob_id = BlobId {\n                            hash: (&sieve.blob_hash).into(),\n                            class: BlobClass::Linked {\n                                account_id,\n                                collection: Collection::SieveScript.into(),\n                                document_id,\n                            },\n                            section: BlobSection {\n                                size: u32::from(sieve.size) as usize,\n                                ..Default::default()\n                            }\n                            .into(),\n                        };\n\n                        result.insert_unchecked(\n                            SieveProperty::BlobId,\n                            Value::Element(SieveValue::BlobId(blob_id)),\n                        );\n                    }\n                }\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/sieve/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\npub mod set;\npub mod validate;\n"
  },
  {
    "path": "crates/jmap/src/sieve/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::StateManager};\nuse common::Server;\nuse email::sieve::ingest::SieveScriptIngest;\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::sieve::{Sieve, SieveComparator, SieveFilter},\n};\nuse std::future::Future;\nuse store::{\n    IndexKeyPrefix, IterateParams, U32_LEN,\n    roaring::RoaringBitmap,\n    search::{SearchFilter, SearchQuery},\n    write::{SearchIndex, key::DeserializeBigEndian},\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::SieveField,\n};\n\npub trait SieveScriptQuery: Sync + Send {\n    fn sieve_script_query(\n        &self,\n        request: QueryRequest<Sieve>,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nimpl SieveScriptQuery for Server {\n    async fn sieve_script_query(\n        &self,\n        mut request: QueryRequest<Sieve>,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n        let mut filters = Vec::with_capacity(request.filter.len());\n        let active_script_id = if request\n            .filter\n            .iter()\n            .any(|f| matches!(f, Filter::Property(SieveFilter::IsActive(_))))\n            || request.sort.as_ref().is_some_and(|s| {\n                s.iter()\n                    .any(|c| matches!(c.property, SieveComparator::IsActive))\n            }) {\n            self.sieve_script_get_active_id(account_id).await?\n        } else {\n            None\n        };\n\n        let mut document_ids = RoaringBitmap::new();\n        let mut names = Vec::new();\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    IndexKeyPrefix {\n                        account_id,\n                        collection: Collection::SieveScript.into(),\n                        field: SieveField::Name.into(),\n                    },\n                    IndexKeyPrefix {\n                        account_id,\n                        collection: Collection::SieveScript.into(),\n                        field: u8::from(SieveField::Name) + 1,\n                    },\n                )\n                .no_values(),\n                |key, _| {\n                    let document_id = key.deserialize_be_u32(key.len() - U32_LEN)?;\n\n                    names.push((\n                        document_id,\n                        key.get(IndexKeyPrefix::len()..key.len() - U32_LEN)\n                            .and_then(|v| std::str::from_utf8(v).ok())\n                            .unwrap_or_default()\n                            .to_string(),\n                    ));\n\n                    document_ids.insert(document_id);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    SieveFilter::Name(name) => {\n                        let name = name.to_lowercase();\n\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            names\n                                .iter()\n                                .filter_map(|(id, n)| (n.contains(&name)).then_some(*id))\n                                .collect::<Vec<_>>(),\n                        )));\n                    }\n                    SieveFilter::IsActive(is_active) => {\n                        if is_active {\n                            if let Some(active_script_id) = active_script_id {\n                                filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter([\n                                    active_script_id,\n                                ])));\n                            } else {\n                                // No active script, so no results\n                                filters.push(SearchFilter::is_in_set(RoaringBitmap::new()));\n                            }\n                        } else {\n                            let mut inactive_set = document_ids.clone();\n                            if let Some(active_script_id) = active_script_id {\n                                inactive_set.remove(active_script_id);\n                            }\n                            filters.push(SearchFilter::is_in_set(inactive_set));\n                        }\n                    }\n                    SieveFilter::_T(other) => {\n                        return Err(trc::JmapEvent::UnsupportedFilter.into_err().details(other));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        // Parse sort criteria\n        let mut sort_by_active = None;\n        for comparator in request\n            .sort\n            .take()\n            .filter(|s| !s.is_empty())\n            .unwrap_or_default()\n        {\n            match comparator.property {\n                SieveComparator::Name => {\n                    if !comparator.is_ascending {\n                        names.reverse();\n                    }\n                }\n                SieveComparator::IsActive => {\n                    sort_by_active = Some(comparator.is_ascending);\n                }\n                SieveComparator::_T(other) => {\n                    return Err(trc::JmapEvent::UnsupportedSort.into_err().details(other));\n                }\n            };\n        }\n\n        let mut results = SearchQuery::new(SearchIndex::InMemory)\n            .with_filters(filters)\n            .with_mask(document_ids)\n            .filter()\n            .into_bitmap();\n\n        let mut response = QueryResponseBuilder::new(\n            results.len() as usize,\n            self.core.jmap.query_max_results,\n            self.get_state(account_id, SyncCollection::SieveScript)\n                .await?,\n            &request,\n        );\n\n        if !results.is_empty() {\n            if matches!(sort_by_active, Some(true))\n                && results.remove(active_script_id.unwrap_or_default())\n                && !response.add(0, active_script_id.unwrap())\n            {\n                return response.build();\n            }\n\n            let mut last_id = None;\n            for (document_id, _) in names {\n                if results.contains(document_id) {\n                    if sort_by_active.is_some() && Some(document_id) == active_script_id {\n                        last_id = Some(document_id);\n                    } else if !response.add(0, document_id) {\n                        return response.build();\n                    }\n                }\n            }\n\n            if let Some(active_id) = last_id {\n                response.add(0, active_id);\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/sieve/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{blob::download::BlobDownload, changes::state::StateManager};\nuse common::{\n    Server,\n    auth::{AccessToken, ResourceToken},\n    storage::index::ObjectIndexBuilder,\n};\nuse email::sieve::{\n    ArchivedSieveScript, SieveScript, delete::SieveScriptDelete, ingest::SieveScriptIngest,\n};\nuse http_proto::HttpSessionData;\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::sieve::{Sieve, SieveProperty, SieveValue},\n    references::resolve::ResolveCreatedReference,\n    request::{IntoValid, reference::MaybeIdReference},\n    types::state::State,\n};\nuse jmap_tools::{Key, Map, Value};\nuse rand::distr::Alphanumeric;\nuse sieve::compiler::ErrorType;\nuse std::future::Future;\nuse store::{\n    Serialize, SerializeInfallible, ValueKey, rand::{Rng, rng}, write::{AlignedBytes, Archive, Archiver, BatchBuilder}\n};\nuse trc::AddContext;\nuse types::{\n    blob::{BlobClass, BlobId, BlobSection},\n    collection::{Collection, SyncCollection},\n    field::{PrincipalField, SieveField},\n    id::Id,\n};\n\npub struct SetContext<'x> {\n    resource_token: ResourceToken,\n    access_token: &'x AccessToken,\n    response: SetResponse<Sieve>,\n}\n\npub trait SieveScriptSet: Sync + Send {\n    fn sieve_script_set(\n        &self,\n        request: SetRequest<'_, Sieve>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> impl Future<Output = trc::Result<SetResponse<Sieve>>> + Send;\n\n    #[allow(clippy::type_complexity)]\n    fn sieve_set_item<'x>(\n        &self,\n        changes_: Value<'_, SieveProperty, SieveValue>,\n        update: Option<(u32, Archive<&'x ArchivedSieveScript>)>,\n        ctx: &SetContext,\n        session_id: u64,\n    ) -> impl Future<\n        Output = trc::Result<\n            Result<\n                (\n                    ObjectIndexBuilder<&'x ArchivedSieveScript, SieveScript>,\n                    Option<Vec<u8>>,\n                ),\n                SetError<SieveProperty>,\n            >,\n        >,\n    > + Send;\n}\n\nimpl SieveScriptSet for Server {\n    async fn sieve_script_set(\n        &self,\n        mut request: SetRequest<'_, Sieve>,\n        access_token: &AccessToken,\n        session: &HttpSessionData,\n    ) -> trc::Result<SetResponse<Sieve>> {\n        let account_id = request.account_id.document_id();\n        let sieve_ids = self\n            .document_ids(account_id, Collection::SieveScript, SieveField::Name)\n            .await?;\n        let mut ctx = SetContext {\n            resource_token: self.get_resource_token(access_token, account_id).await?,\n            access_token,\n            response: SetResponse::from_request(&request, self.core.jmap.set_max_objects)?\n                .with_state(\n                    self.assert_state(\n                        account_id,\n                        SyncCollection::SieveScript,\n                        &request.if_in_state,\n                    )\n                    .await?,\n                ),\n        };\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Validate active script id\n        if let Some(MaybeIdReference::Id(id)) = &request.arguments.on_success_activate_script\n            && !sieve_ids.contains(id.document_id())\n        {\n            request.arguments.on_success_activate_script = None;\n        }\n\n        // Process creates\n        let mut batch = BatchBuilder::new();\n        for (id, object) in request.unwrap_create() {\n            if sieve_ids.len() < access_token.object_quota(Collection::SieveScript) as u64 {\n                match self\n                    .sieve_set_item(object, None, &ctx, session.session_id)\n                    .await?\n                {\n                    Ok((mut builder, Some(blob))) => {\n                        // Store blob\n                        let sieve = &mut builder.changes_mut().unwrap();\n                        let (blob_hash, blob_hold) =\n                            self.put_temporary_blob(account_id, &blob, 60).await?;\n                        sieve.blob_hash = blob_hash;\n                        let blob_size = sieve.size as usize;\n                        let blob_hash = sieve.blob_hash.clone();\n\n                        // Write record\n                        let document_id = self\n                            .store()\n                            .assign_document_ids(account_id, Collection::SieveScript, 1)\n                            .await\n                            .caused_by(trc::location!())?;\n                        batch\n                            .with_account_id(account_id)\n                            .with_collection(Collection::SieveScript)\n                            .with_document(document_id)\n                            .custom(builder.with_access_token(ctx.access_token))\n                            .caused_by(trc::location!())?\n                            .clear(blob_hold)\n                            .commit_point();\n\n                        let mut result = Map::with_capacity(1)\n                            .with_key_value(SieveProperty::Id, SieveValue::Id(document_id.into()))\n                            .with_key_value(\n                                SieveProperty::BlobId,\n                                SieveValue::BlobId(BlobId {\n                                    hash: blob_hash,\n                                    class: BlobClass::Linked {\n                                        account_id,\n                                        collection: Collection::SieveScript.into(),\n                                        document_id,\n                                    },\n                                    section: BlobSection {\n                                        size: blob_size,\n                                        ..Default::default()\n                                    }\n                                    .into(),\n                                }),\n                            );\n\n                        // Update active script if needed\n                        if let Some(MaybeIdReference::Reference(id_ref)) =\n                            &request.arguments.on_success_activate_script\n                            && id_ref == &id\n                        {\n                            request.arguments.on_success_activate_script =\n                                Some(MaybeIdReference::Id(Id::from(document_id)));\n                            result.insert_unchecked(SieveProperty::IsActive, true);\n                        }\n\n                        // Add result with updated blobId\n                        ctx.response.created.insert(id, result.into());\n                    }\n                    Err(err) => {\n                        ctx.response.not_created.append(id, err);\n                    }\n                    _ => unreachable!(),\n                }\n            } else {\n                ctx.response.not_created.append(\n                    id,\n                    SetError::new(SetErrorType::OverQuota).with_description(concat!(\n                        \"There are too many sieve scripts, \",\n                        \"please delete some before adding a new one.\"\n                    )),\n                );\n            }\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                ctx.response\n                    .not_updated\n                    .append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain sieve script\n            let document_id = id.document_id();\n            if let Some(sieve_) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::SieveScript,\n                    document_id,\n                ))\n                .await?\n            {\n                let sieve = sieve_\n                    .to_unarchived::<SieveScript>()\n                    .caused_by(trc::location!())?;\n\n                match self\n                    .sieve_set_item(\n                        object,\n                        (document_id, sieve).into(),\n                        &ctx,\n                        session.session_id,\n                    )\n                    .await?\n                {\n                    Ok((mut builder, blob)) => {\n                        // Prepare write batch\n                        batch\n                            .with_account_id(account_id)\n                            .with_collection(Collection::SieveScript)\n                            .with_document(document_id);\n\n                        let blob_id = if let Some(blob) = blob {\n                            // Store blob\n                            let sieve = &mut builder.changes_mut().unwrap();\n                            let (blob_hash, blob_hold) =\n                                self.put_temporary_blob(account_id, &blob, 60).await?;\n                            sieve.blob_hash = blob_hash;\n                            batch.clear(blob_hold);\n\n                            BlobId {\n                                hash: sieve.blob_hash.clone(),\n                                class: BlobClass::Linked {\n                                    account_id,\n                                    collection: Collection::SieveScript.into(),\n                                    document_id,\n                                },\n                                section: BlobSection {\n                                    size: sieve.size as usize,\n                                    ..Default::default()\n                                }\n                                .into(),\n                            }\n                            .into()\n                        } else {\n                            None\n                        };\n\n                        // Write record\n                        batch\n                            .custom(builder.with_access_token(ctx.access_token))\n                            .caused_by(trc::location!())?\n                            .commit_point();\n\n                        // Update blobId property if needed\n                        let mut result = Map::with_capacity(1);\n                        if let Some(blob_id) = blob_id {\n                            result.insert_unchecked(\n                                SieveProperty::BlobId,\n                                SieveValue::BlobId(blob_id),\n                            );\n                        }\n\n                        // Add active script property if needed\n                        if let Some(MaybeIdReference::Id(id)) =\n                            &request.arguments.on_success_activate_script\n                            && document_id == id.document_id()\n                        {\n                            result.insert_unchecked(SieveProperty::IsActive, true);\n                        }\n\n                        // Add result\n                        ctx.response.updated.append(\n                            id,\n                            if !result.is_empty() {\n                                Value::Object(result).into()\n                            } else {\n                                None\n                            },\n                        );\n                    }\n                    Err(err) => {\n                        ctx.response.not_updated.append(id, err);\n                        continue 'update;\n                    }\n                }\n            } else {\n                ctx.response.not_updated.append(id, SetError::not_found());\n            }\n        }\n\n        // Process deletions\n        let active_script_id = self.sieve_script_get_active_id(account_id).await?;\n        for id in will_destroy {\n            let document_id = id.document_id();\n            if sieve_ids.contains(document_id) {\n                if active_script_id != Some(document_id) {\n                    if self\n                        .sieve_script_delete(account_id, document_id, ctx.access_token, &mut batch)\n                        .await?\n                    {\n                        ctx.response.destroyed.push(id);\n                    } else {\n                        ctx.response.not_destroyed.append(id, SetError::not_found());\n                    }\n                } else {\n                    ctx.response.not_destroyed.append(\n                        id,\n                        SetError::new(SetErrorType::ScriptIsActive)\n                            .with_description(\"Deactivate Sieve script before deletion.\"),\n                    );\n                }\n            } else {\n                ctx.response.not_destroyed.append(id, SetError::not_found());\n            }\n        }\n\n        // Activate / deactivate scripts\n        let on_success_deactivate_script = request\n            .arguments\n            .on_success_deactivate_script\n            .unwrap_or(false);\n        if ctx.response.not_created.is_empty()\n            && ctx.response.not_updated.is_empty()\n            && ctx.response.not_destroyed.is_empty()\n            && (request.arguments.on_success_activate_script.is_some()\n                || on_success_deactivate_script)\n        {\n            if let Some(MaybeIdReference::Id(id)) = request.arguments.on_success_activate_script {\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .set(PrincipalField::ActiveScriptId, id.document_id().serialize());\n            } else if on_success_deactivate_script {\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .clear(PrincipalField::ActiveScriptId);\n            }\n        }\n\n        // Write changes\n        if !batch.is_empty()\n            && let Ok(change_id) = self\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?\n                .last_change_id(account_id)\n        {\n            ctx.response.new_state = State::Exact(change_id).into();\n        }\n\n        Ok(ctx.response)\n    }\n\n    #[allow(clippy::blocks_in_conditions)]\n    async fn sieve_set_item<'x>(\n        &self,\n        changes_: Value<'_, SieveProperty, SieveValue>,\n        update: Option<(u32, Archive<&'x ArchivedSieveScript>)>,\n        ctx: &SetContext<'_>,\n        session_id: u64,\n    ) -> trc::Result<\n        Result<\n            (\n                ObjectIndexBuilder<&'x ArchivedSieveScript, SieveScript>,\n                Option<Vec<u8>>,\n            ),\n            SetError<SieveProperty>,\n        >,\n    > {\n        // Vacation script cannot be modified\n        if update\n            .as_ref()\n            .is_some_and(|(_, obj)| obj.inner.name.eq_ignore_ascii_case(\"vacation\"))\n        {\n            return Ok(Err(SetError::forbidden().with_description(concat!(\n                \"The 'vacation' script cannot be modified, \",\n                \"use VacationResponse/set instead.\"\n            ))));\n        }\n\n        // Parse properties\n        let mut changes = update\n            .as_ref()\n            .map(|(_, obj)| obj.deserialize().unwrap_or_default())\n            .unwrap_or_default();\n        let mut blob_id = None;\n        for (property, mut value) in changes_.into_expanded_object() {\n            if let Err(err) = ctx.response.resolve_self_references(&mut value) {\n                return Ok(Err(err));\n            };\n            match (&property, value) {\n                (Key::Property(SieveProperty::Name), Value::Str(value)) => {\n                    if value.len() > self.core.jmap.sieve_max_script_name {\n                        return Ok(Err(SetError::invalid_properties()\n                            .with_property(property.into_owned())\n                            .with_description(\"Script name is too long.\")));\n                    } else if value.eq_ignore_ascii_case(\"vacation\") {\n                        return Ok(Err(SetError::forbidden()\n                            .with_property(property.into_owned())\n                            .with_description(\n                                \"The 'vacation' name is reserved, please use a different name.\",\n                            )));\n                    } else if update\n                        .as_ref()\n                        .is_none_or(|(_, obj)| obj.inner.name != value.as_ref())\n                        && let Some(id) = self\n                            .document_ids_matching(\n                                ctx.resource_token.account_id,\n                                Collection::SieveScript,\n                                SieveField::Name,\n                                value.as_bytes(),\n                            )\n                            .await?\n                            .min()\n                    {\n                        return Ok(Err(SetError::already_exists()\n                            .with_existing_id(id.into())\n                            .with_description(format!(\n                                \"A sieve script with name '{}' already exists.\",\n                                value\n                            ))));\n                    }\n\n                    changes.name = value.into_owned();\n                }\n                (\n                    Key::Property(SieveProperty::BlobId),\n                    Value::Element(SieveValue::BlobId(value)),\n                ) => {\n                    blob_id = value.into();\n                    continue;\n                }\n                (Key::Property(SieveProperty::Name), Value::Null) => {\n                    continue;\n                }\n                _ => {\n                    return Ok(Err(SetError::invalid_properties()\n                        .with_property(property.into_owned())\n                        .with_description(\"Invalid property or value.\".to_string())));\n                }\n            }\n        }\n\n        if update.is_none() {\n            // Add name if missing\n            if changes.name.is_empty() {\n                changes.name = rng()\n                    .sample_iter(Alphanumeric)\n                    .take(15)\n                    .map(char::from)\n                    .collect::<String>();\n            }\n        }\n\n        let blob_update = if let Some(blob_id) = blob_id {\n            if update.as_ref().is_none_or( |(document_id, _)| {\n                !matches!(blob_id.class, BlobClass::Linked { account_id, collection, document_id: d } if account_id == ctx.resource_token.account_id && collection == u8::from(Collection::SieveScript) && *document_id == d)\n            }) {\n                // Check access\n                if let Some(mut bytes) = self.blob_download(&blob_id, ctx.access_token).await? {\n                    // Check quota\n                    match self\n                        .has_available_quota(&ctx.resource_token, bytes.len() as u64)\n                        .await\n                    {\n                        Ok(_) => (),\n                        Err(err) => {\n                            if err.matches(trc::EventType::Limit(trc::LimitEvent::Quota))\n                                || err.matches(trc::EventType::Limit(trc::LimitEvent::TenantQuota))\n                            {\n                                trc::error!(err.account_id(ctx.resource_token.account_id).span_id(session_id));\n                                return Ok(Err(SetError::over_quota()));\n                            } else {\n                                return Err(err);\n                            }\n                        }\n                    }\n\n                    // Compile script\n                    match self.core.sieve.untrusted_compiler.compile(&bytes) {\n                        Ok(script) => {\n                            changes.size = bytes.len() as u32;\n                            bytes.extend(Archiver::new(script).untrusted().serialize().caused_by(trc::location!())?);\n                            bytes.into()\n                        }\n                        Err(err) => {\n                            return Ok(Err(SetError::new(\n                                if let ErrorType::ScriptTooLong = &err.error_type() {\n                                    SetErrorType::TooLarge\n                                } else {\n                                    SetErrorType::InvalidScript\n                                },\n                            )\n                            .with_description(err.to_string())));\n                        }\n                    }\n                } else {\n                    return Ok(Err(SetError::new(SetErrorType::BlobNotFound)\n                        .with_property(SieveProperty::BlobId)\n                        .with_description(\"Blob does not exist.\")));\n                }\n            } else {\n                None\n            }\n        } else if update.is_none() {\n            return Ok(Err(SetError::invalid_properties()\n                .with_property(SieveProperty::BlobId)\n                .with_description(\"Missing blobId.\")));\n        } else {\n            None\n        };\n\n        // Validate\n        Ok(Ok((\n            ObjectIndexBuilder::new()\n                .with_changes(changes)\n                .with_current_opt(update.map(|(_, current)| current)),\n            blob_update,\n        )))\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/sieve/validate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::blob::download::BlobDownload;\nuse common::{Server, auth::AccessToken};\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::validate::{ValidateSieveScriptRequest, ValidateSieveScriptResponse},\n    request::MaybeInvalid,\n};\nuse std::future::Future;\n\npub trait SieveScriptValidate: Sync + Send {\n    fn sieve_script_validate(\n        &self,\n        request: ValidateSieveScriptRequest,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<ValidateSieveScriptResponse>> + Send;\n}\n\nimpl SieveScriptValidate for Server {\n    async fn sieve_script_validate(\n        &self,\n        request: ValidateSieveScriptRequest,\n        access_token: &AccessToken,\n    ) -> trc::Result<ValidateSieveScriptResponse> {\n        Ok(ValidateSieveScriptResponse {\n            account_id: request.account_id,\n            error: match request.blob_id {\n                MaybeInvalid::Value(blob_id) => {\n                    match self\n                        .blob_download(&blob_id, access_token)\n                        .await?\n                        .map(|bytes| self.core.sieve.untrusted_compiler.compile(&bytes))\n                    {\n                        Some(Ok(_)) => None,\n                        Some(Err(err)) => SetError::new(SetErrorType::InvalidScript)\n                            .with_description(err.to_string())\n                            .into(),\n                        None => SetError::new(SetErrorType::BlobNotFound).into(),\n                    }\n                }\n                MaybeInvalid::Invalid(_) => SetError::new(SetErrorType::BlobNotFound).into(),\n            },\n        })\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/submission/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::changes::state::StateManager;\nuse common::Server;\nuse email::submission::{\n    ArchivedAddress, ArchivedEnvelope, ArchivedUndoStatus, Delivered, DeliveryStatus,\n    EmailSubmission,\n};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::email_submission::{self, Displayed, EmailSubmissionProperty, EmailSubmissionValue},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Key, Map, Value};\nuse smtp::queue::{ArchivedError, ArchivedErrorDetails, ArchivedStatus, Message, spool::SmtpSpool};\nuse smtp_proto::ArchivedResponse;\nuse std::future::Future;\nuse store::{\n    IterateParams, U32_LEN, ValueKey,\n    rkyv::option::ArchivedOption,\n    write::{\n        AlignedBytes, Archive, IndexPropertyClass, ValueClass, key::DeserializeBigEndian, now,\n    },\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::EmailSubmissionField,\n    id::Id,\n};\nuse utils::map::vec_map::VecMap;\n\npub trait EmailSubmissionGet: Sync + Send {\n    fn email_submission_get(\n        &self,\n        request: GetRequest<email_submission::EmailSubmission>,\n    ) -> impl Future<Output = trc::Result<GetResponse<email_submission::EmailSubmission>>> + Send;\n}\n\nimpl EmailSubmissionGet for Server {\n    async fn email_submission_get(\n        &self,\n        mut request: GetRequest<email_submission::EmailSubmission>,\n    ) -> trc::Result<GetResponse<email_submission::EmailSubmission>> {\n        let ids = request.unwrap_ids(self.core.jmap.get_max_objects)?;\n        let properties = request.unwrap_properties(&[\n            EmailSubmissionProperty::Id,\n            EmailSubmissionProperty::EmailId,\n            EmailSubmissionProperty::IdentityId,\n            EmailSubmissionProperty::ThreadId,\n            EmailSubmissionProperty::Envelope,\n            EmailSubmissionProperty::SendAt,\n            EmailSubmissionProperty::UndoStatus,\n            EmailSubmissionProperty::DeliveryStatus,\n            EmailSubmissionProperty::DsnBlobIds,\n            EmailSubmissionProperty::MdnBlobIds,\n        ]);\n        let account_id = request.account_id.document_id();\n        let ids = if let Some(ids) = ids {\n            ids\n        } else {\n            let mut ids = Vec::with_capacity(16);\n\n            self.store()\n                .iterate(\n                    IterateParams::new(\n                        ValueKey {\n                            account_id,\n                            collection: Collection::CalendarEventNotification.into(),\n                            document_id: 0,\n                            class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                                property: EmailSubmissionField::Metadata.into(),\n                                value: now() - (3 * 86400),\n                            }),\n                        },\n                        ValueKey {\n                            account_id,\n                            collection: Collection::CalendarEventNotification.into(),\n                            document_id: 0,\n                            class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                                property: EmailSubmissionField::Metadata.into(),\n                                value: u64::MAX,\n                            }),\n                        },\n                    )\n                    .ascending()\n                    .no_values(),\n                    |key, _| {\n                        ids.push(Id::from(key.deserialize_be_u32(key.len() - U32_LEN)?));\n\n                        Ok(ids.len() < self.core.jmap.get_max_objects)\n                    },\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n            ids\n        };\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: self\n                .get_state(account_id, SyncCollection::EmailSubmission)\n                .await?\n                .into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        for id in ids {\n            // Obtain the email_submission object\n            let document_id = id.document_id();\n            let submission_ = if let Some(submission) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::EmailSubmission,\n                    document_id,\n                ))\n                .await?\n            {\n                submission\n            } else {\n                response.not_found.push(id);\n                continue;\n            };\n            let submission = submission_\n                .unarchive::<EmailSubmission>()\n                .caused_by(trc::location!())?;\n\n            // Obtain queueId\n            let mut delivery_status = submission\n                .delivery_status\n                .iter()\n                .map(|(k, v)| (k.to_string(), DeliveryStatus::from(v)))\n                .collect::<VecMap<_, _>>();\n            let mut is_pending = false;\n            if let Some(queue_id) = submission.queue_id.as_ref().map(u64::from)\n                && let Some(queued_message_) = self\n                    .read_message_archive(queue_id)\n                    .await\n                    .caused_by(trc::location!())?\n            {\n                let queued_message = queued_message_\n                    .unarchive::<Message>()\n                    .caused_by(trc::location!())?;\n                for rcpt in queued_message.recipients.iter() {\n                    *delivery_status.get_mut_or_insert(rcpt.address().to_string()) =\n                        DeliveryStatus {\n                            smtp_reply: match &rcpt.status {\n                                ArchivedStatus::Completed(reply) => {\n                                    format_archived_response(&reply.response)\n                                }\n                                ArchivedStatus::TemporaryFailure(reply)\n                                | ArchivedStatus::PermanentFailure(reply) => {\n                                    format_archived_error_details(reply)\n                                }\n                                ArchivedStatus::Scheduled => \"250 2.1.5 Queued\".to_string(),\n                            },\n                            delivered: match &rcpt.status {\n                                ArchivedStatus::Scheduled | ArchivedStatus::TemporaryFailure(_) => {\n                                    Delivered::Queued\n                                }\n                                ArchivedStatus::Completed(_) => Delivered::Yes,\n                                ArchivedStatus::PermanentFailure(_) => Delivered::No,\n                            },\n                            displayed: false,\n                        };\n                }\n                is_pending = true;\n            }\n\n            let mut result = Map::with_capacity(properties.len());\n            for property in &properties {\n                let value = match property {\n                    EmailSubmissionProperty::Id => Value::Element(id.into()),\n                    EmailSubmissionProperty::DeliveryStatus => {\n                        let mut status = Map::with_capacity(delivery_status.len());\n\n                        for (rcpt, delivery_status) in std::mem::take(&mut delivery_status) {\n                            status.insert_unchecked(\n                                Key::Owned(rcpt),\n                                Map::with_capacity(3)\n                                    .with_key_value(\n                                        EmailSubmissionProperty::Delivered,\n                                        EmailSubmissionValue::Delivered(\n                                            match delivery_status.delivered {\n                                                Delivered::Queued => {\n                                                    email_submission::Delivered::Queued\n                                                }\n                                                Delivered::Yes => email_submission::Delivered::Yes,\n                                                Delivered::No => email_submission::Delivered::No,\n                                                Delivered::Unknown => {\n                                                    email_submission::Delivered::Unknown\n                                                }\n                                            },\n                                        ),\n                                    )\n                                    .with_key_value(\n                                        EmailSubmissionProperty::SmtpReply,\n                                        delivery_status.smtp_reply,\n                                    )\n                                    .with_key_value(\n                                        EmailSubmissionProperty::Displayed,\n                                        Value::Element(EmailSubmissionValue::Displayed(\n                                            Displayed::Unknown,\n                                        )),\n                                    ),\n                            );\n                        }\n\n                        Value::Object(status)\n                    }\n                    EmailSubmissionProperty::UndoStatus => {\n                        Value::Element(EmailSubmissionValue::UndoStatus(if is_pending {\n                            email_submission::UndoStatus::Pending\n                        } else {\n                            match submission.undo_status {\n                                ArchivedUndoStatus::Pending => {\n                                    email_submission::UndoStatus::Pending\n                                }\n                                ArchivedUndoStatus::Final => email_submission::UndoStatus::Final,\n                                ArchivedUndoStatus::Canceled => {\n                                    email_submission::UndoStatus::Canceled\n                                }\n                            }\n                        }))\n                    }\n                    EmailSubmissionProperty::EmailId => Value::Element(\n                        Id::from_parts(\n                            u32::from(submission.thread_id),\n                            u32::from(submission.email_id),\n                        )\n                        .into(),\n                    ),\n                    EmailSubmissionProperty::IdentityId => {\n                        Value::Element(Id::from(u32::from(submission.identity_id)).into())\n                    }\n                    EmailSubmissionProperty::ThreadId => {\n                        Value::Element(Id::from(u32::from(submission.thread_id)).into())\n                    }\n                    EmailSubmissionProperty::Envelope => build_envelope(&submission.envelope),\n                    EmailSubmissionProperty::SendAt => Value::Element(EmailSubmissionValue::Date(\n                        UTCDate::from_timestamp(u64::from(submission.send_at) as i64),\n                    )),\n                    EmailSubmissionProperty::MdnBlobIds | EmailSubmissionProperty::DsnBlobIds => {\n                        Value::Array(vec![])\n                    }\n                    _ => Value::Null,\n                };\n\n                result.insert_unchecked(property.clone(), value);\n            }\n            response.list.push(result.into());\n        }\n\n        Ok(response)\n    }\n}\n\nfn build_envelope(\n    envelope: &ArchivedEnvelope,\n) -> Value<'static, EmailSubmissionProperty, EmailSubmissionValue> {\n    Map::with_capacity(2)\n        .with_key_value(\n            EmailSubmissionProperty::MailFrom,\n            build_address(&envelope.mail_from),\n        )\n        .with_key_value(\n            EmailSubmissionProperty::RcptTo,\n            Value::Array(envelope.rcpt_to.iter().map(build_address).collect()),\n        )\n        .into()\n}\n\nfn build_address(\n    envelope: &ArchivedAddress,\n) -> Value<'static, EmailSubmissionProperty, EmailSubmissionValue> {\n    Map::with_capacity(2)\n        .with_key_value(\n            EmailSubmissionProperty::Email,\n            Value::Str(envelope.email.to_string().into()),\n        )\n        .with_key_value(\n            EmailSubmissionProperty::Parameters,\n            if let ArchivedOption::Some(params) = &envelope.parameters {\n                Value::Object(Map::from_iter(\n                    params\n                        .iter()\n                        .map(|(k, v)| (Key::Owned(k.to_string()), v.into())),\n                ))\n            } else {\n                Value::Null\n            },\n        )\n        .into()\n}\n\nfn format_archived_response(response: &ArchivedResponse<Box<str>>) -> String {\n    format!(\n        \"Code: {}, Enhanced code: {}.{}.{}, Message: {}\",\n        response.code,\n        response.esc[0],\n        response.esc[1],\n        response.esc[2],\n        response.message.replace('\\n', \" \"),\n    )\n}\n\nfn format_archived_error_details(response: &ArchivedErrorDetails) -> String {\n    match &response.details {\n        ArchivedError::UnexpectedResponse(response) => format_archived_response(&response.response),\n        ArchivedError::DnsError(details)\n        | ArchivedError::Io(details)\n        | ArchivedError::ConnectionError(details)\n        | ArchivedError::TlsError(details)\n        | ArchivedError::DaneError(details)\n        | ArchivedError::MtaStsError(details) => details.to_string(),\n        ArchivedError::RateLimited => \"Rate limited\".to_string(),\n        ArchivedError::ConcurrencyLimited => \"Concurrency limited\".to_string(),\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/submission/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod query;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/submission/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{api::query::QueryResponseBuilder, changes::state::StateManager};\nuse common::Server;\nuse email::submission::UndoStatus;\nuse jmap_proto::{\n    method::query::{Filter, QueryRequest, QueryResponse},\n    object::email_submission::{self, EmailSubmissionComparator, EmailSubmissionFilter},\n    request::IntoValid,\n};\nuse std::future::Future;\nuse store::{\n    IterateParams, U32_LEN, U64_LEN, ValueKey,\n    ahash::AHashSet,\n    roaring::RoaringBitmap,\n    search::{SearchFilter, SearchQuery},\n    write::{IndexPropertyClass, SearchIndex, ValueClass, key::DeserializeBigEndian, now},\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::EmailSubmissionField,\n};\n\npub trait EmailSubmissionQuery: Sync + Send {\n    fn email_submission_query(\n        &self,\n        request: QueryRequest<email_submission::EmailSubmission>,\n    ) -> impl Future<Output = trc::Result<QueryResponse>> + Send;\n}\n\nstruct Submission {\n    document_id: u32,\n    send_at: u64,\n    email_id: u32,\n    thread_id: u32,\n    identity_id: u32,\n    undo_status: u8,\n}\n\nimpl EmailSubmissionQuery for Server {\n    async fn email_submission_query(\n        &self,\n        mut request: QueryRequest<email_submission::EmailSubmission>,\n    ) -> trc::Result<QueryResponse> {\n        let account_id = request.account_id.document_id();\n\n        let mut submissions = Vec::with_capacity(16);\n        let mut document_ids = RoaringBitmap::new();\n\n        self.store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::EmailSubmission.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: EmailSubmissionField::Metadata.into(),\n                            value: now() - (3 * 86400),\n                        }),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::EmailSubmission.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: EmailSubmissionField::Metadata.into(),\n                            value: u64::MAX,\n                        }),\n                    },\n                )\n                .ascending(),\n                |key, value| {\n                    let document_id = key.deserialize_be_u32(key.len() - U32_LEN)?;\n\n                    submissions.push(Submission {\n                        document_id,\n                        send_at: key.deserialize_be_u64(key.len() - U32_LEN - U64_LEN)?,\n                        email_id: value.deserialize_be_u32(0)?,\n                        thread_id: value.deserialize_be_u32(U32_LEN)?,\n                        identity_id: value.deserialize_be_u32(U32_LEN + U32_LEN)?,\n                        undo_status: value.last().copied().unwrap(),\n                    });\n\n                    document_ids.insert(document_id);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        let mut filters = Vec::with_capacity(request.filter.len());\n        for cond in std::mem::take(&mut request.filter) {\n            match cond {\n                Filter::Property(cond) => match cond {\n                    EmailSubmissionFilter::IdentityIds(ids) => {\n                        let ids = ids\n                            .into_valid()\n                            .map(|id| id.document_id())\n                            .collect::<AHashSet<_>>();\n\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            submissions\n                                .iter()\n                                .filter(|s| ids.contains(&s.identity_id))\n                                .map(|s| s.document_id),\n                        )));\n                    }\n                    EmailSubmissionFilter::EmailIds(ids) => {\n                        let ids = ids\n                            .into_valid()\n                            .map(|id| id.document_id())\n                            .collect::<AHashSet<_>>();\n\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            submissions\n                                .iter()\n                                .filter(|s| ids.contains(&s.email_id))\n                                .map(|s| s.document_id),\n                        )));\n                    }\n                    EmailSubmissionFilter::ThreadIds(ids) => {\n                        let ids = ids\n                            .into_valid()\n                            .map(|id| id.document_id())\n                            .collect::<AHashSet<_>>();\n\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            submissions\n                                .iter()\n                                .filter(|s| ids.contains(&s.thread_id))\n                                .map(|s| s.document_id),\n                        )));\n                    }\n                    EmailSubmissionFilter::UndoStatus(undo_status) => {\n                        let undo_status = match undo_status {\n                            email_submission::UndoStatus::Pending => UndoStatus::Pending,\n                            email_submission::UndoStatus::Final => UndoStatus::Final,\n                            email_submission::UndoStatus::Canceled => UndoStatus::Canceled,\n                        }\n                        .as_index();\n\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            submissions\n                                .iter()\n                                .filter(|s| s.undo_status == undo_status)\n                                .map(|s| s.document_id),\n                        )));\n                    }\n                    EmailSubmissionFilter::Before(before) => {\n                        let before = before.timestamp() as u64;\n\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            submissions\n                                .iter()\n                                .filter(|s| s.send_at < before)\n                                .map(|s| s.document_id),\n                        )));\n                    }\n                    EmailSubmissionFilter::After(after) => {\n                        let after = after.timestamp() as u64;\n\n                        filters.push(SearchFilter::is_in_set(RoaringBitmap::from_iter(\n                            submissions\n                                .iter()\n                                .filter(|s| s.send_at > after)\n                                .map(|s| s.document_id),\n                        )));\n                    }\n\n                    EmailSubmissionFilter::_T(other) => {\n                        return Err(trc::JmapEvent::UnsupportedFilter.into_err().details(other));\n                    }\n                },\n                Filter::And => {\n                    filters.push(SearchFilter::And);\n                }\n                Filter::Or => {\n                    filters.push(SearchFilter::Or);\n                }\n                Filter::Not => {\n                    filters.push(SearchFilter::Not);\n                }\n                Filter::Close => {\n                    filters.push(SearchFilter::End);\n                }\n            }\n        }\n\n        let results = SearchQuery::new(SearchIndex::InMemory)\n            .with_filters(filters)\n            .with_mask(document_ids)\n            .filter()\n            .into_bitmap();\n\n        let mut response = QueryResponseBuilder::new(\n            results.len() as usize,\n            self.core.jmap.query_max_results,\n            self.get_state(account_id, SyncCollection::EmailSubmission)\n                .await?,\n            &request,\n        );\n\n        if !results.is_empty() {\n            if let Some(comparator) = request.sort.take().unwrap_or_default().into_iter().next() {\n                match comparator.property {\n                    EmailSubmissionComparator::EmailId => {\n                        if comparator.is_ascending {\n                            submissions.sort_by_key(|s| s.email_id);\n                        } else {\n                            submissions.sort_by_key(|s| u32::MAX - s.email_id);\n                        }\n                    }\n                    EmailSubmissionComparator::ThreadId => {\n                        if comparator.is_ascending {\n                            submissions.sort_by_key(|s| s.thread_id);\n                        } else {\n                            submissions.sort_by_key(|s| u32::MAX - s.thread_id);\n                        }\n                    }\n                    EmailSubmissionComparator::SentAt => {\n                        if !comparator.is_ascending {\n                            submissions.reverse();\n                        }\n                    }\n                    EmailSubmissionComparator::_T(other) => {\n                        return Err(trc::JmapEvent::UnsupportedSort.into_err().details(other));\n                    }\n                }\n            }\n\n            for submission in submissions {\n                if results.contains(submission.document_id)\n                    && !response.add(0, submission.document_id)\n                {\n                    break;\n                }\n            }\n        }\n\n        response.build()\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/submission/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Server,\n    config::smtp::queue::QueueName,\n    listener::{ServerInstance, stream::NullIo},\n    storage::index::ObjectIndexBuilder,\n};\nuse email::{\n    identity::Identity,\n    message::metadata::{ArchivedMetadataHeaderName, ArchivedMetadataHeaderValue, MessageMetadata},\n    submission::{Address, Delivered, DeliveryStatus, EmailSubmission, UndoStatus},\n};\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::email_submission::{self, EmailSubmissionProperty, EmailSubmissionValue},\n    references::resolve::ResolveCreatedReference,\n    request::{\n        Call, IntoValid, MaybeInvalid, RequestMethod, SetRequestMethod,\n        method::{MethodFunction, MethodName, MethodObject},\n        reference::{MaybeIdReference, MaybeResultReference},\n    },\n    types::state::State,\n};\nuse jmap_tools::{Key, Value};\nuse smtp::{\n    core::{Session, SessionData},\n    queue::spool::SmtpSpool,\n};\nuse smtp_proto::{MailFrom, RcptTo, request::parser::Rfc5321Parser};\nuse std::{borrow::Cow, future::Future};\nuse std::{collections::HashMap, sync::Arc, time::Duration};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder, now},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::EmailField, id::Id};\nuse utils::{map::vec_map::VecMap, sanitize_email};\n\npub trait EmailSubmissionSet: Sync + Send {\n    fn email_submission_set<'x>(\n        &self,\n        request: SetRequest<'x, email_submission::EmailSubmission>,\n        instance: &Arc<ServerInstance>,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n    ) -> impl Future<Output = trc::Result<SetResponse<email_submission::EmailSubmission>>> + Send;\n\n    fn send_message(\n        &self,\n        account_id: u32,\n        response: &SetResponse<email_submission::EmailSubmission>,\n        instance: &Arc<ServerInstance>,\n        object: Value<'_, EmailSubmissionProperty, EmailSubmissionValue>,\n    ) -> impl Future<\n        Output = trc::Result<Result<EmailSubmission, SetError<EmailSubmissionProperty>>>,\n    > + Send;\n}\n\nimpl EmailSubmissionSet for Server {\n    async fn email_submission_set<'x>(\n        &self,\n        mut request: SetRequest<'x, email_submission::EmailSubmission>,\n        instance: &Arc<ServerInstance>,\n        next_call: &mut Option<Call<RequestMethod<'x>>>,\n    ) -> trc::Result<SetResponse<email_submission::EmailSubmission>> {\n        let account_id = request.account_id.document_id();\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?;\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Process creates\n        let mut success_email_ids = HashMap::new();\n        let mut batch = BatchBuilder::new();\n        for (id, object) in request.unwrap_create() {\n            match self\n                .send_message(account_id, &response, instance, object)\n                .await?\n            {\n                Ok(submission) => {\n                    // Add id mapping\n                    success_email_ids.insert(\n                        id.clone(),\n                        Id::from_parts(submission.thread_id, submission.email_id),\n                    );\n\n                    // Insert record\n                    let document_id = self\n                        .store()\n                        .assign_document_ids(account_id, Collection::EmailSubmission, 1)\n                        .await\n                        .caused_by(trc::location!())?;\n                    batch\n                        .with_account_id(account_id)\n                        .with_collection(Collection::EmailSubmission)\n                        .with_document(document_id)\n                        .custom(ObjectIndexBuilder::<(), _>::new().with_changes(submission))\n                        .caused_by(trc::location!())?\n                        .commit_point();\n                    response.created(id, document_id);\n                }\n                Err(err) => {\n                    response.not_created.append(id, err);\n                }\n            }\n        }\n\n        // Process updates\n        'update: for (id, object) in request.unwrap_update().into_valid() {\n            // Make sure id won't be destroyed\n            if will_destroy.contains(&id) {\n                response.not_updated.append(id, SetError::will_destroy());\n                continue 'update;\n            }\n\n            // Obtain submission\n            let document_id = id.document_id();\n            let submission = if let Some(submission) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::EmailSubmission,\n                    document_id,\n                ))\n                .await?\n            {\n                submission\n                    .into_deserialized::<EmailSubmission>()\n                    .caused_by(trc::location!())?\n            } else {\n                response.not_updated.append(id, SetError::not_found());\n                continue 'update;\n            };\n\n            let mut queue_id = u64::MAX;\n            let mut undo_status = None;\n\n            for (property, mut value) in object.into_expanded_object() {\n                if let Err(err) = response.resolve_self_references(&mut value) {\n                    response.not_updated.append(id, err);\n                    continue 'update;\n                };\n\n                if let (\n                    Key::Property(EmailSubmissionProperty::UndoStatus),\n                    Value::Element(EmailSubmissionValue::UndoStatus(undo_status_)),\n                    Some(queue_id_),\n                ) = (&property, value, submission.inner.queue_id)\n                {\n                    undo_status = undo_status_.into();\n                    queue_id = queue_id_;\n                } else {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(property.into_owned())\n                            .with_description(\"Field could not be set.\"),\n                    );\n                    continue 'update;\n                }\n            }\n\n            match undo_status {\n                Some(email_submission::UndoStatus::Canceled) => {\n                    if let Some(queue_message) =\n                        self.read_message(queue_id, QueueName::default()).await\n                    {\n                        // Delete message from queue\n                        queue_message.remove(self, None).await;\n\n                        // Update record\n                        let mut new_submission = submission.inner.clone();\n                        new_submission.undo_status = UndoStatus::Canceled;\n                        batch\n                            .with_account_id(account_id)\n                            .with_collection(Collection::EmailSubmission)\n                            .with_document(document_id)\n                            .custom(\n                                ObjectIndexBuilder::new()\n                                    .with_current(submission)\n                                    .with_changes(new_submission),\n                            )\n                            .caused_by(trc::location!())?\n                            .commit_point();\n                        response.updated.append(id, None);\n                    } else {\n                        response.not_updated.append(\n                            id,\n                            SetError::new(SetErrorType::CannotUnsend).with_description(\n                                \"The requested message is no longer in the queue.\",\n                            ),\n                        );\n                    }\n                }\n                Some(_) => {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_property(EmailSubmissionProperty::UndoStatus)\n                            .with_description(\"Email submissions can only be cancelled.\"),\n                    );\n                }\n                None => {\n                    response.not_updated.append(\n                        id,\n                        SetError::invalid_properties()\n                            .with_description(\"No properties to set were found.\"),\n                    );\n                }\n            }\n        }\n\n        // Process deletions\n        for id in will_destroy {\n            let document_id = id.document_id();\n            if let Some(submission) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::EmailSubmission,\n                    document_id,\n                ))\n                .await?\n            {\n                // Update record\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::EmailSubmission)\n                    .with_document(document_id)\n                    .custom(\n                        ObjectIndexBuilder::<_, ()>::new().with_current(\n                            submission\n                                .to_unarchived::<EmailSubmission>()\n                                .caused_by(trc::location!())?,\n                        ),\n                    )\n                    .caused_by(trc::location!())?\n                    .commit_point();\n                response.destroyed.push(id);\n            } else {\n                response.not_destroyed.append(id, SetError::not_found());\n            }\n        }\n\n        // Write changes\n        if !batch.is_empty() {\n            let change_id = self\n                .commit_batch(batch)\n                .await\n                .and_then(|ids| ids.last_change_id(account_id))\n                .caused_by(trc::location!())?;\n            response.new_state = State::Exact(change_id).into();\n        }\n\n        // On success\n        if (request\n            .arguments\n            .on_success_destroy_email\n            .as_ref()\n            .is_some_and(|p| !p.is_empty())\n            || request\n                .arguments\n                .on_success_update_email\n                .as_ref()\n                .is_some_and(|p| !p.is_empty()))\n            && response.has_changes()\n        {\n            *next_call = Call {\n                id: String::new(),\n                name: MethodName::new(MethodObject::Email, MethodFunction::Set),\n                method: RequestMethod::Set(SetRequestMethod::Email(SetRequest {\n                    account_id: request.account_id,\n                    if_in_state: None,\n                    create: None,\n                    update: request.arguments.on_success_update_email.map(|update| {\n                        update\n                            .into_iter()\n                            .filter_map(|(id, value)| {\n                                (\n                                    match id {\n                                        MaybeIdReference::Id(id) => MaybeInvalid::Value(id),\n                                        MaybeIdReference::Reference(id_ref) => {\n                                            MaybeInvalid::Value(*(success_email_ids.get(&id_ref)?))\n                                        }\n                                        MaybeIdReference::Invalid(id) => MaybeInvalid::Invalid(id),\n                                    },\n                                    value,\n                                )\n                                    .into()\n                            })\n                            .collect()\n                    }),\n                    destroy: request.arguments.on_success_destroy_email.map(|ids| {\n                        MaybeResultReference::Value(\n                            ids.into_iter()\n                                .filter_map(|id| match id {\n                                    MaybeIdReference::Id(id) => Some(id),\n                                    MaybeIdReference::Reference(id_ref) => {\n                                        success_email_ids.get(&id_ref).copied()\n                                    }\n                                    MaybeIdReference::Invalid(_) => None,\n                                })\n                                .map(MaybeInvalid::Value)\n                                .collect(),\n                        )\n                    }),\n                    arguments: Default::default(),\n                })),\n            }\n            .into();\n        }\n\n        Ok(response)\n    }\n\n    async fn send_message(\n        &self,\n        account_id: u32,\n        response: &SetResponse<email_submission::EmailSubmission>,\n        instance: &Arc<ServerInstance>,\n        object: Value<'_, EmailSubmissionProperty, EmailSubmissionValue>,\n    ) -> trc::Result<Result<EmailSubmission, SetError<EmailSubmissionProperty>>> {\n        let mut submission = EmailSubmission {\n            email_id: u32::MAX,\n            identity_id: u32::MAX,\n            thread_id: u32::MAX,\n            ..Default::default()\n        };\n        let mut mail_from: Option<MailFrom<Cow<'_, str>>> = None;\n        let mut rcpt_to: Vec<RcptTo<Cow<'_, str>>> = Vec::new();\n\n        for (property, mut value) in object.into_expanded_object() {\n            if let Err(err) = response.resolve_self_references(&mut value) {\n                return Ok(Err(err));\n            };\n\n            match (&property, value) {\n                (\n                    Key::Property(EmailSubmissionProperty::EmailId),\n                    Value::Element(EmailSubmissionValue::Id(value)),\n                ) => {\n                    submission.email_id = value.document_id();\n                    submission.thread_id = value.prefix_id();\n                }\n                (\n                    Key::Property(EmailSubmissionProperty::IdentityId),\n                    Value::Element(EmailSubmissionValue::Id(value)),\n                ) => {\n                    submission.identity_id = value.document_id();\n                }\n                (Key::Property(EmailSubmissionProperty::Envelope), Value::Object(value)) => {\n                    for (property, value) in value.into_vec() {\n                        match (&property, value) {\n                            (Key::Property(EmailSubmissionProperty::MailFrom), value) => {\n                                match parse_envelope_address(value) {\n                                    Ok((addr, params, smtp_params)) => {\n                                        match Rfc5321Parser::new(\n                                            &mut smtp_params\n                                                .as_ref()\n                                                .map_or(&b\"\\n\"[..], |p| p.as_bytes())\n                                                .iter(),\n                                        )\n                                        .mail_from_parameters(addr.into())\n                                        {\n                                            Ok(addr) => {\n                                                submission.envelope.mail_from = Address {\n                                                    email: addr.address.as_ref().to_string(),\n                                                    parameters: params,\n                                                };\n                                                mail_from = from_into_static(addr).into();\n                                            }\n                                            Err(err) => {\n                                                return Ok(Err(SetError::invalid_properties()\n                                                .with_property(EmailSubmissionProperty::Envelope)\n                                                .with_description(format!(\n                                                    \"Failed to parse mailFrom parameters: {err}.\"\n                                                ))));\n                                            }\n                                        }\n                                    }\n                                    Err(err) => {\n                                        return Ok(Err(err));\n                                    }\n                                }\n                            }\n                            (\n                                Key::Property(EmailSubmissionProperty::RcptTo),\n                                Value::Array(value),\n                            ) => {\n                                for addr in value {\n                                    match parse_envelope_address(addr) {\n                                        Ok((addr, params, smtp_params)) => {\n                                            match Rfc5321Parser::new(\n                                                &mut smtp_params\n                                                    .as_ref()\n                                                    .map_or(&b\"\\n\"[..], |p| p.as_bytes())\n                                                    .iter(),\n                                            )\n                                            .rcpt_to_parameters(addr.into())\n                                            {\n                                                Ok(addr) => {\n                                                    if !rcpt_to\n                                                        .iter()\n                                                        .any(|rcpt| rcpt.address == addr.address)\n                                                    {\n                                                        submission.envelope.rcpt_to.push(Address {\n                                                            email: addr\n                                                                .address\n                                                                .as_ref()\n                                                                .to_string(),\n                                                            parameters: params,\n                                                        });\n                                                        rcpt_to.push(rcpt_into_static(addr));\n                                                    }\n                                                }\n                                                Err(err) => {\n                                                    return Ok(Err(SetError::invalid_properties()\n                                                        .with_property(EmailSubmissionProperty::Envelope)\n                                                        .with_description(format!(\n                                                        \"Failed to parse rcptTo parameters: {err}.\"\n                                                    ))));\n                                                }\n                                            }\n                                        }\n                                        Err(err) => {\n                                            return Ok(Err(err));\n                                        }\n                                    }\n                                }\n                            }\n                            _ => {\n                                return Ok(Err(SetError::invalid_properties()\n                                    .with_property(EmailSubmissionProperty::Envelope)\n                                    .with_description(\"Invalid object property.\")));\n                            }\n                        }\n                    }\n                }\n                (Key::Property(EmailSubmissionProperty::Envelope), Value::Null) => {\n                    continue;\n                }\n                (Key::Property(EmailSubmissionProperty::UndoStatus), Value::Element(_)) => {\n                    continue;\n                }\n                _ => {\n                    return Ok(Err(SetError::invalid_properties()\n                        .with_property(property.into_owned())\n                        .with_description(\"Field could not be set.\")));\n                }\n            }\n        }\n\n        // Make sure we have all required fields.\n        if submission.email_id == u32::MAX || submission.identity_id == u32::MAX {\n            return Ok(Err(SetError::invalid_properties()\n                .with_properties([\n                    EmailSubmissionProperty::EmailId,\n                    EmailSubmissionProperty::IdentityId,\n                ])\n                .with_description(\n                    \"emailId and identityId properties are required.\",\n                )));\n        }\n\n        // Fetch identity's mailFrom\n        let identity_mail_from = if let Some(identity) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::Identity,\n                submission.identity_id,\n            ))\n            .await?\n        {\n            identity\n                .unarchive::<Identity>()\n                .caused_by(trc::location!())?\n                .email\n                .to_string()\n        } else {\n            return Ok(Err(SetError::invalid_properties()\n                .with_property(EmailSubmissionProperty::IdentityId)\n                .with_description(\"Identity not found.\")));\n        };\n\n        // Make sure the envelope address matches the identity email address\n        let mail_from = if let Some(mail_from) = mail_from {\n            if !mail_from.address.eq_ignore_ascii_case(&identity_mail_from) {\n                return Ok(Err(SetError::new(SetErrorType::ForbiddenFrom)\n                    .with_description(\n                        \"Envelope mailFrom does not match identity email address.\",\n                    )));\n            }\n            mail_from\n        } else {\n            submission.envelope.mail_from = Address {\n                email: identity_mail_from.clone(),\n                parameters: None,\n            };\n            MailFrom {\n                address: Cow::Owned(identity_mail_from),\n                ..Default::default()\n            }\n        };\n\n        // Obtain message metadata\n        let metadata_ = if let Some(metadata) = self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                account_id,\n                Collection::Email,\n                submission.email_id,\n                EmailField::Metadata,\n            ))\n            .await?\n        {\n            metadata\n        } else {\n            return Ok(Err(SetError::invalid_properties()\n                .with_property(EmailSubmissionProperty::EmailId)\n                .with_description(\"Email not found.\")));\n        };\n        let metadata = metadata_\n            .unarchive::<MessageMetadata>()\n            .caused_by(trc::location!())?;\n\n        // Add recipients to envelope if missing\n        let mut bcc_header = None;\n        if rcpt_to.is_empty() {\n            for header in metadata.contents[0].parts[0].headers.iter() {\n                if matches!(\n                    header.name,\n                    ArchivedMetadataHeaderName::To\n                        | ArchivedMetadataHeaderName::Cc\n                        | ArchivedMetadataHeaderName::Bcc\n                ) {\n                    if matches!(header.name, ArchivedMetadataHeaderName::Bcc) {\n                        bcc_header = Some(header);\n                    }\n                    match &header.value {\n                        ArchivedMetadataHeaderValue::AddressList(addr) => {\n                            for address in addr.iter() {\n                                if let Some(address) = address\n                                    .address\n                                    .as_ref()\n                                    .map(|v| v.as_ref())\n                                    .and_then(sanitize_email)\n                                    && !rcpt_to.iter().any(|rcpt| rcpt.address == address)\n                                {\n                                    submission.envelope.rcpt_to.push(Address {\n                                        email: address.to_string(),\n                                        parameters: None,\n                                    });\n                                    rcpt_to.push(RcptTo {\n                                        address: Cow::Owned(address),\n                                        ..Default::default()\n                                    });\n                                }\n                            }\n                        }\n                        ArchivedMetadataHeaderValue::AddressGroup(groups) => {\n                            for group in groups.iter() {\n                                for address in group.addresses.iter() {\n                                    if let Some(address) = address\n                                        .address\n                                        .as_ref()\n                                        .map(|v| v.as_ref())\n                                        .and_then(sanitize_email)\n                                        && !rcpt_to.iter().any(|rcpt| rcpt.address == address)\n                                    {\n                                        submission.envelope.rcpt_to.push(Address {\n                                            email: address.to_string(),\n                                            parameters: None,\n                                        });\n                                        rcpt_to.push(RcptTo {\n                                            address: Cow::Owned(address),\n                                            ..Default::default()\n                                        });\n                                    }\n                                }\n                            }\n                        }\n                        _ => {}\n                    }\n                }\n            }\n\n            if rcpt_to.is_empty() {\n                return Ok(Err(SetError::new(SetErrorType::NoRecipients)\n                    .with_description(\"No recipients found in email.\")));\n            }\n        } else {\n            bcc_header = metadata.contents[0].parts[0]\n                .headers\n                .iter()\n                .find(|header| matches!(header.name, ArchivedMetadataHeaderName::Bcc));\n        }\n\n        // Update sendAt\n        submission.send_at = if mail_from.hold_until > 0 {\n            mail_from.hold_until\n        } else if mail_from.hold_for > 0 {\n            mail_from.hold_for + now()\n        } else {\n            now()\n        };\n\n        // Obtain raw message\n        let mut message = if let Some(message) = self\n            .blob_store()\n            .get_blob(metadata.blob_hash.0.as_slice(), 0..usize::MAX)\n            .await?\n        {\n            if message.len() > self.core.jmap.mail_max_size {\n                return Ok(Err(SetError::new(SetErrorType::InvalidEmail)\n                    .with_description(format!(\n                        \"Message exceeds maximum size of {} bytes.\",\n                        self.core.jmap.mail_max_size\n                    ))));\n            }\n\n            message\n        } else {\n            return Ok(Err(SetError::invalid_properties()\n                .with_property(EmailSubmissionProperty::EmailId)\n                .with_description(\"Blob for email not found.\")));\n        };\n\n        // Remove BCC header if present\n        if let Some(bcc_header) = bcc_header {\n            let mut new_message = Vec::with_capacity(message.len());\n            let range = bcc_header.name_value_range();\n            new_message.extend_from_slice(&message[..range.start]);\n            new_message.extend_from_slice(&message[range.end..]);\n            message = new_message;\n        }\n\n        // Begin local SMTP session\n        let mut session = Session::<NullIo>::local(\n            self.clone(),\n            instance.clone(),\n            SessionData::local(\n                self.get_access_token(account_id)\n                    .await\n                    .caused_by(trc::location!())?,\n                None,\n                vec![],\n                vec![],\n                0,\n            ),\n        );\n\n        // Spawn SMTP session to avoid overflowing the stack\n        let handle = tokio::spawn(async move {\n            // MAIL FROM\n            let _ = session.handle_mail_from(mail_from).await;\n            if let Some(error) = session.has_failed() {\n                return Err(SetError::new(SetErrorType::ForbiddenMailFrom)\n                    .with_description(format!(\"Server rejected MAIL-FROM: {}\", error.trim())));\n            }\n\n            // RCPT TO\n            let mut responses = Vec::new();\n            let mut has_success = false;\n            session.params.rcpt_errors_wait = Duration::from_secs(0);\n            for rcpt in rcpt_to {\n                let addr = rcpt.address.clone();\n                let _ = session.handle_rcpt_to(rcpt).await;\n                let response = session.has_failed();\n                if response.is_none() {\n                    has_success = true;\n                }\n                responses.push((addr, response));\n            }\n\n            // DATA\n            if has_success {\n                session.data.message = message;\n                let response = session.queue_message().await;\n                if let smtp::core::State::Accepted(queue_id) = session.state {\n                    Ok((true, responses, Some(queue_id)))\n                } else {\n                    Err(\n                        SetError::new(SetErrorType::ForbiddenToSend).with_description(format!(\n                            \"Server rejected DATA: {}\",\n                            std::str::from_utf8(&response).unwrap().trim()\n                        )),\n                    )\n                }\n            } else {\n                Ok((false, responses, None))\n            }\n        });\n\n        match handle.await {\n            Ok(Ok((has_success, responses, queue_id))) => {\n                // Set queue ID\n                if let Some(queue_id) = queue_id {\n                    submission.queue_id = Some(queue_id);\n                }\n\n                // Set responses\n                submission.undo_status = if has_success {\n                    UndoStatus::Final\n                } else {\n                    UndoStatus::Pending\n                };\n                submission.delivery_status = responses\n                    .into_iter()\n                    .map(|(addr, response)| {\n                        (\n                            addr.to_string(),\n                            DeliveryStatus {\n                                delivered: if response.is_none() {\n                                    Delivered::Unknown\n                                } else {\n                                    Delivered::No\n                                },\n                                smtp_reply: response\n                                    .map(|s| s.to_string())\n                                    .unwrap_or_else(|| \"250 2.1.5 Queued\".to_string()),\n                                displayed: false,\n                            },\n                        )\n                    })\n                    .collect();\n\n                Ok(Ok(submission))\n            }\n            Ok(Err(err)) => Ok(Err(err)),\n            Err(err) => Err(trc::EventType::Server(trc::ServerEvent::ThreadError)\n                .reason(err)\n                .caused_by(trc::location!())\n                .details(\"Join Error\")),\n        }\n    }\n}\n\n#[allow(clippy::type_complexity)]\nfn parse_envelope_address(\n    envelope: Value<'_, EmailSubmissionProperty, EmailSubmissionValue>,\n) -> Result<\n    (\n        String,\n        Option<VecMap<String, Option<String>>>,\n        Option<String>,\n    ),\n    SetError<EmailSubmissionProperty>,\n> {\n    if let Value::Object(mut envelope) = envelope {\n        if let Some(Value::Str(addr)) =\n            envelope.remove(&Key::Property(EmailSubmissionProperty::Email))\n        {\n            if let Some(addr) = sanitize_email(&addr) {\n                if let Some(Value::Object(params)) =\n                    envelope.remove(&Key::Property(EmailSubmissionProperty::Parameters))\n                {\n                    let mut params_text = String::new();\n                    let mut params_list = VecMap::with_capacity(params.len());\n\n                    for (k, v) in params.into_vec() {\n                        let k = k.into_string();\n                        if !k.is_empty() {\n                            if !params_text.is_empty() {\n                                params_text.push(' ');\n                            }\n                            params_text.push_str(&k);\n                            if let Value::Str(v) = v {\n                                params_text.push('=');\n                                params_text.push_str(&v);\n                                params_list.append(k, Some(v.into_owned()));\n                            } else {\n                                params_list.append(k, None);\n                            }\n                        }\n                    }\n                    params_text.push('\\n');\n\n                    Ok((addr.to_string(), Some(params_list), Some(params_text)))\n                } else {\n                    Ok((addr.to_string(), None, None))\n                }\n            } else {\n                Err(SetError::invalid_properties()\n                    .with_property(EmailSubmissionProperty::Envelope)\n                    .with_description(format!(\"Invalid e-mail address {addr:?}.\")))\n            }\n        } else {\n            Err(SetError::invalid_properties()\n                .with_property(EmailSubmissionProperty::Envelope)\n                .with_description(\"Missing e-mail address field.\"))\n        }\n    } else {\n        Err(SetError::invalid_properties()\n            .with_property(EmailSubmissionProperty::Envelope)\n            .with_description(\"Invalid envelope object.\"))\n    }\n}\n\nfn from_into_static(from: MailFrom<Cow<'_, str>>) -> MailFrom<Cow<'static, str>> {\n    MailFrom {\n        address: from.address.into_owned().into(),\n        flags: from.flags,\n        size: from.size,\n        trans_id: from.trans_id.map(Cow::into_owned).map(Cow::Owned),\n        by: from.by,\n        env_id: from.env_id.map(Cow::into_owned).map(Cow::Owned),\n        solicit: from.solicit.map(Cow::into_owned).map(Cow::Owned),\n        mtrk: from\n            .mtrk\n            .map(smtp_proto::Mtrk::into_owned)\n            .map(|v| smtp_proto::Mtrk {\n                certifier: Cow::Owned(v.certifier),\n                timeout: v.timeout,\n            }),\n        auth: from.auth.map(Cow::into_owned).map(Cow::Owned),\n        hold_for: from.hold_for,\n        hold_until: from.hold_until,\n        mt_priority: from.mt_priority,\n    }\n}\n\nfn rcpt_into_static(rcpt: RcptTo<Cow<'_, str>>) -> RcptTo<Cow<'static, str>> {\n    RcptTo {\n        address: rcpt.address.into_owned().into(),\n        orcpt: rcpt.orcpt.map(Cow::into_owned).map(Cow::Owned),\n        rrvs: rcpt.rrvs,\n        flags: rcpt.flags,\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/thread/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::changes::state::StateManager;\nuse common::Server;\nuse email::cache::MessageCacheFetch;\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::thread::{Thread, ThreadProperty, ThreadValue},\n    request::MaybeInvalid,\n};\nuse jmap_tools::Map;\nuse std::future::Future;\nuse store::{\n    ahash::AHashMap,\n    roaring::RoaringBitmap,\n    search::{EmailSearchField, SearchComparator, SearchField, SearchQuery},\n    write::SearchIndex,\n};\nuse trc::AddContext;\nuse types::{collection::SyncCollection, id::Id};\n\npub trait ThreadGet: Sync + Send {\n    fn thread_get(\n        &self,\n        request: GetRequest<Thread>,\n    ) -> impl Future<Output = trc::Result<GetResponse<Thread>>> + Send;\n}\n\nimpl ThreadGet for Server {\n    async fn thread_get(\n        &self,\n        mut request: GetRequest<Thread>,\n    ) -> trc::Result<GetResponse<Thread>> {\n        let account_id = request.account_id.document_id();\n        let mut thread_map: AHashMap<u32, RoaringBitmap> = AHashMap::with_capacity(32);\n        let mut all_ids = RoaringBitmap::new();\n        for item in &self\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?\n            .emails\n            .items\n        {\n            thread_map\n                .entry(item.thread_id)\n                .or_default()\n                .insert(item.document_id);\n            all_ids.insert(item.document_id);\n        }\n\n        let ids = if let Some(ids) = request.unwrap_ids(self.core.jmap.get_max_objects)? {\n            ids\n        } else {\n            thread_map\n                .keys()\n                .copied()\n                .take(self.core.jmap.get_max_objects)\n                .map(Into::into)\n                .collect()\n        };\n        let add_email_ids = request.properties.is_none_or(|p| {\n            p.unwrap()\n                .contains(&MaybeInvalid::Value(ThreadProperty::EmailIds))\n        });\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: self\n                .get_state(account_id, SyncCollection::Thread)\n                .await?\n                .into(),\n            list: Vec::with_capacity(ids.len()),\n            not_found: vec![],\n        };\n\n        let ordered_ids = if add_email_ids && !all_ids.is_empty() {\n            Some(\n                self.search_store()\n                    .query_account(\n                        SearchQuery::new(SearchIndex::Email)\n                            .with_account_id(account_id)\n                            .with_mask(all_ids)\n                            .with_comparator(SearchComparator::Field {\n                                field: SearchField::Email(EmailSearchField::ReceivedAt),\n                                ascending: true,\n                            }),\n                    )\n                    .await?,\n            )\n        } else {\n            None\n        };\n\n        for id in ids {\n            let thread_id = id.document_id();\n            if let Some(mut document_ids) = thread_map.remove(&thread_id) {\n                let mut thread: Map<'_, ThreadProperty, ThreadValue> =\n                    Map::with_capacity(2).with_key_value(ThreadProperty::Id, id);\n                if let Some(ordered_ids) = &ordered_ids {\n                    let mut ids = Vec::with_capacity(document_ids.len() as usize);\n                    for &id in ordered_ids.iter() {\n                        if document_ids.remove(id) {\n                            ids.push(Id::from_parts(thread_id, id));\n                        }\n                    }\n                    for id in document_ids.iter() {\n                        ids.push(Id::from_parts(thread_id, id));\n                    }\n\n                    thread.insert_unchecked(ThreadProperty::EmailIds, ids);\n                }\n                response.list.push(thread.into());\n            } else {\n                response.not_found.push(id);\n            }\n        }\n\n        Ok(response)\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/thread/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\n"
  },
  {
    "path": "crates/jmap/src/vacation/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::changes::state::StateManager;\nuse common::Server;\nuse email::sieve::{SieveScript, ingest::SieveScriptIngest};\nuse jmap_proto::{\n    method::get::{GetRequest, GetResponse},\n    object::vacation_response::{\n        VacationResponse, VacationResponseProperty, VacationResponseValue,\n    },\n    request::reference::MaybeResultReference,\n    types::date::UTCDate,\n};\nuse jmap_tools::{Map, Value};\nuse std::future::Future;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::SieveField,\n    id::Id,\n};\n\npub trait VacationResponseGet: Sync + Send {\n    fn vacation_response_get(\n        &self,\n        request: GetRequest<VacationResponse>,\n    ) -> impl Future<Output = trc::Result<GetResponse<VacationResponse>>> + Send;\n\n    fn get_vacation_sieve_script_id(\n        &self,\n        account_id: u32,\n    ) -> impl Future<Output = trc::Result<Option<u32>>> + Send;\n}\n\nimpl VacationResponseGet for Server {\n    async fn vacation_response_get(\n        &self,\n        mut request: GetRequest<VacationResponse>,\n    ) -> trc::Result<GetResponse<VacationResponse>> {\n        let account_id = request.account_id.document_id();\n        let properties = request.unwrap_properties(&[\n            VacationResponseProperty::Id,\n            VacationResponseProperty::IsEnabled,\n            VacationResponseProperty::FromDate,\n            VacationResponseProperty::ToDate,\n            VacationResponseProperty::Subject,\n            VacationResponseProperty::TextBody,\n            VacationResponseProperty::HtmlBody,\n        ]);\n        let mut response = GetResponse {\n            account_id: request.account_id.into(),\n            state: self\n                .get_state(account_id, SyncCollection::SieveScript)\n                .await?\n                .into(),\n            list: Vec::with_capacity(1),\n            not_found: vec![],\n        };\n\n        let do_get = if let Some(MaybeResultReference::Value(ids)) = request.ids {\n            let mut do_get = false;\n            for id in ids {\n                match id.try_unwrap() {\n                    Some(id) if id.is_singleton() => {\n                        do_get = true;\n                    }\n                    Some(id) => {\n                        response.not_found.push(id);\n                    }\n                    _ => {}\n                }\n            }\n            do_get\n        } else {\n            true\n        };\n        if do_get {\n            if let Some(document_id) = self.get_vacation_sieve_script_id(account_id).await? {\n                if let Some(sieve_) = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::SieveScript,\n                        document_id,\n                    ))\n                    .await?\n                {\n                    let active_script_id = self.sieve_script_get_active_id(account_id).await?;\n                    let sieve = sieve_\n                        .unarchive::<SieveScript>()\n                        .caused_by(trc::location!())?;\n                    let vacation = sieve.vacation_response.as_ref();\n                    let mut result = Map::with_capacity(properties.len());\n                    for property in &properties {\n                        match property {\n                            VacationResponseProperty::Id => {\n                                result.insert_unchecked(\n                                    VacationResponseProperty::Id,\n                                    Id::singleton(),\n                                );\n                            }\n                            VacationResponseProperty::IsEnabled => {\n                                result.insert_unchecked(\n                                    VacationResponseProperty::IsEnabled,\n                                    active_script_id == Some(document_id),\n                                );\n                            }\n                            VacationResponseProperty::FromDate => {\n                                result.insert_unchecked(\n                                    VacationResponseProperty::FromDate,\n                                    vacation.and_then(|r| {\n                                        r.from_date\n                                            .as_ref()\n                                            .map(u64::from)\n                                            .map(UTCDate::from)\n                                            .map(|v| Value::Element(VacationResponseValue::Date(v)))\n                                    }),\n                                );\n                            }\n                            VacationResponseProperty::ToDate => {\n                                result.insert_unchecked(\n                                    VacationResponseProperty::ToDate,\n                                    vacation.and_then(|r| {\n                                        r.to_date\n                                            .as_ref()\n                                            .map(u64::from)\n                                            .map(UTCDate::from)\n                                            .map(|v| Value::Element(VacationResponseValue::Date(v)))\n                                    }),\n                                );\n                            }\n                            VacationResponseProperty::Subject => {\n                                result.insert_unchecked(\n                                    VacationResponseProperty::Subject,\n                                    vacation.and_then(|r| r.subject.as_ref()),\n                                );\n                            }\n                            VacationResponseProperty::TextBody => {\n                                result.insert_unchecked(\n                                    VacationResponseProperty::TextBody,\n                                    vacation.and_then(|r| r.text_body.as_ref()),\n                                );\n                            }\n                            VacationResponseProperty::HtmlBody => {\n                                result.insert_unchecked(\n                                    VacationResponseProperty::HtmlBody,\n                                    vacation.and_then(|r| r.html_body.as_ref()),\n                                );\n                            }\n                        }\n                    }\n                    response.list.push(result.into());\n                } else {\n                    response.not_found.push(Id::singleton());\n                }\n            } else {\n                response.not_found.push(Id::singleton());\n            }\n        }\n\n        Ok(response)\n    }\n\n    async fn get_vacation_sieve_script_id(&self, account_id: u32) -> trc::Result<Option<u32>> {\n        self.document_ids_matching(\n            account_id,\n            Collection::SieveScript,\n            SieveField::Name,\n            \"vacation\".as_bytes(),\n        )\n        .await\n        .map(|r| r.min())\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/vacation/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod get;\npub mod set;\n"
  },
  {
    "path": "crates/jmap/src/vacation/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::get::VacationResponseGet;\nuse crate::changes::state::StateManager;\nuse common::{Server, auth::AccessToken, storage::index::ObjectIndexBuilder};\nuse email::sieve::{\n    SieveScript, VacationResponse, delete::SieveScriptDelete, ingest::SieveScriptIngest,\n};\nuse jmap_proto::{\n    error::set::{SetError, SetErrorType},\n    method::set::{SetRequest, SetResponse},\n    object::vacation_response::{self, VacationResponseProperty, VacationResponseValue},\n    references::resolve::ResolveCreatedReference,\n    request::IntoValid,\n    types::date::UTCDate,\n};\nuse jmap_tools::{Key, Map, Value};\nuse mail_builder::MessageBuilder;\nuse mail_parser::decoders::html::html_to_text;\nuse std::borrow::Cow;\nuse std::future::Future;\nuse store::{\n    Serialize, SerializeInfallible, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::PrincipalField,\n    id::Id,\n};\n\npub trait VacationResponseSet: Sync + Send {\n    fn vacation_response_set(\n        &self,\n        request: SetRequest<'_, vacation_response::VacationResponse>,\n        access_token: &AccessToken,\n    ) -> impl Future<Output = trc::Result<SetResponse<vacation_response::VacationResponse>>> + Send;\n\n    fn build_script(&self, obj: &mut SieveScript) -> trc::Result<Vec<u8>>;\n}\n\nimpl VacationResponseSet for Server {\n    async fn vacation_response_set(\n        &self,\n        mut request: SetRequest<'_, vacation_response::VacationResponse>,\n        access_token: &AccessToken,\n    ) -> trc::Result<SetResponse<vacation_response::VacationResponse>> {\n        let account_id = request.account_id.document_id();\n        let mut response = SetResponse::from_request(&request, self.core.jmap.set_max_objects)?\n            .with_state(\n                self.assert_state(\n                    account_id,\n                    SyncCollection::SieveScript,\n                    &request.if_in_state,\n                )\n                .await?,\n            );\n        let will_destroy = request.unwrap_destroy().into_valid().collect::<Vec<_>>();\n\n        // Process set or update requests\n        let mut create_id = None;\n        let mut changes = None;\n        match (request.create, request.update) {\n            (Some(create), Some(update)) if !create.is_empty() && !update.is_empty() => {\n                return Err(trc::JmapEvent::InvalidArguments\n                    .into_err()\n                    .details(\"Creating and updating on the same request is not allowed.\"));\n            }\n            (Some(create), _) if !create.is_empty() => {\n                for (id, obj) in create {\n                    if will_destroy.contains(&Id::singleton()) {\n                        response.not_created.append(\n                            id,\n                            SetError::new(SetErrorType::WillDestroy)\n                                .with_description(\"ID will be destroyed.\"),\n                        );\n                    } else if create_id.is_some() {\n                        response.not_created.append(\n                            id,\n                            SetError::forbidden()\n                                .with_description(\"Only one object can be created.\"),\n                        );\n                    } else {\n                        create_id = Some(id);\n                        changes = Some(obj);\n                    }\n                }\n            }\n            (_, Some(update)) if !update.is_empty() => {\n                for (id, obj) in update.into_valid() {\n                    if id.is_singleton() {\n                        if !will_destroy.contains(&id) {\n                            changes = Some(obj);\n                        } else {\n                            response.not_updated.append(\n                                id,\n                                SetError::new(SetErrorType::WillDestroy)\n                                    .with_description(\"ID will be destroyed.\"),\n                            );\n                        }\n                    } else {\n                        response.not_updated.append(\n                            id,\n                            SetError::new(SetErrorType::NotFound).with_description(\"ID not found.\"),\n                        );\n                    }\n                }\n            }\n            _ => {\n                if will_destroy.is_empty() {\n                    return Ok(response);\n                }\n            }\n        }\n\n        // Prepare write batch\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::SieveScript);\n\n        // Process changes\n        let active_script_id = self.sieve_script_get_active_id(account_id).await?;\n        if let Some(changes) = changes {\n            // Obtain current script\n            let document_id = self.get_vacation_sieve_script_id(account_id).await?;\n\n            let (mut sieve, prev_sieve) = if let Some(document_id) = document_id {\n                let prev_sieve = self\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account_id,\n                        Collection::SieveScript,\n                        document_id,\n                    ))\n                    .await?\n                    .ok_or_else(|| {\n                        trc::StoreEvent::NotFound\n                            .into_err()\n                            .caused_by(trc::location!())\n                    })?\n                    .into_deserialized::<SieveScript>()\n                    .caused_by(trc::location!())?;\n                let mut sieve = prev_sieve.inner.clone();\n                if sieve.vacation_response.is_none() {\n                    sieve.vacation_response = VacationResponse::default().into();\n                }\n\n                (sieve, Some(prev_sieve))\n            } else {\n                (\n                    SieveScript {\n                        name: \"vacation\".into(),\n                        blob_hash: Default::default(),\n                        size: 0,\n                        vacation_response: VacationResponse::default().into(),\n                    },\n                    None,\n                )\n            };\n\n            // Parse properties\n            let mut is_active = false;\n            let mut build_script = create_id.is_some();\n            let vacation = sieve.vacation_response.as_mut().unwrap();\n\n            for (property, mut value) in changes.into_expanded_object() {\n                if let Err(err) = response.resolve_self_references(&mut value) {\n                    return Ok(set_error(response, create_id, err));\n                };\n\n                match (&property, value) {\n                    (Key::Property(VacationResponseProperty::Subject), Value::Str(value))\n                        if value.len() < 512 =>\n                    {\n                        build_script = true;\n                        vacation.subject = Some(value.into_owned());\n                    }\n                    (Key::Property(VacationResponseProperty::HtmlBody), Value::Str(value))\n                        if value.len() < 2048 =>\n                    {\n                        build_script = true;\n                        vacation.html_body = Some(value.into_owned());\n                    }\n                    (Key::Property(VacationResponseProperty::TextBody), Value::Str(value))\n                        if value.len() < 2048 =>\n                    {\n                        build_script = true;\n                        vacation.text_body = Some(value.into_owned());\n                    }\n                    (\n                        Key::Property(VacationResponseProperty::FromDate),\n                        Value::Element(VacationResponseValue::Date(date)),\n                    ) => {\n                        vacation.from_date = Some(date.timestamp() as u64);\n                        build_script = true;\n                    }\n                    (\n                        Key::Property(VacationResponseProperty::ToDate),\n                        Value::Element(VacationResponseValue::Date(date)),\n                    ) => {\n                        vacation.to_date = Some(date.timestamp() as u64);\n                        build_script = true;\n                    }\n                    (Key::Property(VacationResponseProperty::IsEnabled), Value::Bool(value)) => {\n                        is_active = value;\n                    }\n                    (Key::Property(VacationResponseProperty::IsEnabled), Value::Null) => {\n                        is_active = false;\n                    }\n                    (\n                        Key::Property(\n                            VacationResponseProperty::Subject\n                            | VacationResponseProperty::HtmlBody\n                            | VacationResponseProperty::TextBody\n                            | VacationResponseProperty::ToDate\n                            | VacationResponseProperty::FromDate,\n                        ),\n                        Value::Null,\n                    ) => {\n                        if create_id.is_none() {\n                            build_script = true;\n                            match property {\n                                Key::Property(VacationResponseProperty::Subject) => {\n                                    vacation.subject = None;\n                                }\n                                Key::Property(VacationResponseProperty::HtmlBody) => {\n                                    vacation.html_body = None;\n                                }\n                                Key::Property(VacationResponseProperty::TextBody) => {\n                                    vacation.text_body = None;\n                                }\n                                Key::Property(VacationResponseProperty::FromDate) => {\n                                    vacation.from_date = None;\n                                }\n                                Key::Property(VacationResponseProperty::ToDate) => {\n                                    vacation.to_date = None;\n                                }\n                                _ => unreachable!(),\n                            }\n                        }\n                    }\n                    _ => {\n                        return Ok(set_error(\n                            response,\n                            create_id,\n                            SetError::invalid_properties()\n                                .with_property(property.into_owned())\n                                .with_description(\"Field could not be set.\"),\n                        ));\n                    }\n                }\n            }\n\n            let mut obj = ObjectIndexBuilder::new()\n                .with_current_opt(prev_sieve)\n                .with_changes(sieve)\n                .with_access_token(access_token);\n\n            // Update id\n            let document_id = if let Some(document_id) = document_id {\n                batch.with_document(document_id);\n                document_id\n            } else {\n                let document_id = self\n                    .store()\n                    .assign_document_ids(account_id, Collection::SieveScript, 1)\n                    .await\n                    .caused_by(trc::location!())?;\n                batch.with_document(document_id);\n                document_id\n            };\n\n            // Create sieve script only if there are changes\n            if build_script {\n                // Upload new blob\n                let (blob_hash, blob_hold) = self\n                    .put_temporary_blob(\n                        account_id,\n                        &self.build_script(obj.changes_mut().unwrap())?,\n                        60,\n                    )\n                    .await?;\n                obj.changes_mut().unwrap().blob_hash = blob_hash;\n                batch.clear(blob_hold);\n            };\n            batch.custom(obj).caused_by(trc::location!())?;\n\n            // Deactivate other sieve scripts\n            let was_active = active_script_id == Some(document_id);\n            if is_active {\n                if !was_active {\n                    batch\n                        .with_collection(Collection::Principal)\n                        .with_document(0)\n                        .set(PrincipalField::ActiveScriptId, document_id.serialize());\n                }\n            } else if was_active {\n                batch\n                    .with_collection(Collection::Principal)\n                    .with_document(0)\n                    .clear(PrincipalField::ActiveScriptId);\n            }\n\n            // Write changes\n            if !batch.is_empty() {\n                response.new_state = Some(\n                    self.commit_batch(batch)\n                        .await\n                        .and_then(|ids| ids.last_change_id(account_id))\n                        .caused_by(trc::location!())?\n                        .into(),\n                );\n            }\n\n            // Add result\n            if let Some(create_id) = create_id {\n                response.created.insert(\n                    create_id,\n                    Map::with_capacity(1)\n                        .with_key_value(VacationResponseProperty::Id, Id::singleton())\n                        .into(),\n                );\n            } else {\n                response.updated.append(Id::singleton(), None);\n            }\n        } else if !will_destroy.is_empty() {\n            for id in will_destroy {\n                if id.is_singleton()\n                    && let Some(document_id) = self.get_vacation_sieve_script_id(account_id).await?\n                {\n                    self.sieve_script_delete(account_id, document_id, access_token, &mut batch)\n                        .await?;\n                    if active_script_id == Some(document_id) {\n                        batch\n                            .with_collection(Collection::Principal)\n                            .with_document(0)\n                            .clear(PrincipalField::ActiveScriptId);\n                    }\n\n                    response.destroyed.push(id);\n                    break;\n                }\n\n                response.not_destroyed.append(id, SetError::not_found());\n            }\n\n            // Write changes\n            if !batch.is_empty() {\n                response.new_state = Some(\n                    self.commit_batch(batch)\n                        .await\n                        .and_then(|ids| ids.last_change_id(account_id))\n                        .caused_by(trc::location!())?\n                        .into(),\n                );\n            }\n        }\n\n        Ok(response)\n    }\n\n    fn build_script(&self, obj: &mut SieveScript) -> trc::Result<Vec<u8>> {\n        // Build Sieve script\n        let mut script = Vec::with_capacity(1024);\n        script.extend_from_slice(b\"require [\\\"vacation\\\", \\\"relational\\\", \\\"date\\\"];\\r\\n\\r\\n\");\n        let mut num_blocks = 0;\n\n        // Add start date\n        if let Some(value) = obj.vacation_response.as_ref().and_then(|v| v.from_date) {\n            script.extend_from_slice(b\"if currentdate :value \\\"ge\\\" \\\"iso8601\\\" \\\"\");\n            script.extend_from_slice(UTCDate::from(value).to_string().as_bytes());\n            script.extend_from_slice(b\"\\\" {\\r\\n\");\n            num_blocks += 1;\n        }\n\n        // Add end date\n        if let Some(value) = obj.vacation_response.as_ref().and_then(|v| v.to_date) {\n            script.extend_from_slice(b\"if currentdate :value \\\"le\\\" \\\"iso8601\\\" \\\"\");\n            script.extend_from_slice(UTCDate::from(value).to_string().as_bytes());\n            script.extend_from_slice(b\"\\\" {\\r\\n\");\n            num_blocks += 1;\n        }\n\n        script.extend_from_slice(b\"vacation :mime \");\n        if let Some(value) = obj\n            .vacation_response\n            .as_ref()\n            .and_then(|v| v.subject.as_ref())\n        {\n            script.extend_from_slice(b\":subject \\\"\");\n            for &ch in value.as_bytes().iter() {\n                match ch {\n                    b'\\\\' | b'\\\"' => {\n                        script.push(b'\\\\');\n                    }\n                    b'\\r' | b'\\n' => {\n                        continue;\n                    }\n                    _ => (),\n                }\n                script.push(ch);\n            }\n            script.extend_from_slice(b\"\\\" \");\n        }\n\n        let mut text_body = if let Some(value) = obj\n            .vacation_response\n            .as_ref()\n            .and_then(|v| v.text_body.as_ref())\n        {\n            Cow::from(value.as_str()).into()\n        } else {\n            None\n        };\n        let html_body = if let Some(value) = obj\n            .vacation_response\n            .as_ref()\n            .and_then(|v| v.html_body.as_ref())\n        {\n            Cow::from(value.as_str()).into()\n        } else {\n            None\n        };\n        match (&html_body, &text_body) {\n            (Some(html_body), None) => {\n                text_body = Cow::from(html_to_text(html_body.as_ref())).into();\n            }\n            (None, None) => {\n                text_body = Cow::from(\"I am away.\").into();\n            }\n            _ => (),\n        }\n\n        let mut builder = MessageBuilder::new();\n        let mut body_len = 0;\n        if let Some(html_body) = html_body {\n            body_len = html_body.len();\n            builder = builder.html_body(html_body);\n        }\n        if let Some(text_body) = text_body {\n            body_len += text_body.len();\n            builder = builder.text_body(text_body);\n        }\n        let mut message_body = Vec::with_capacity(body_len + 128);\n        builder.write_body(&mut message_body).ok();\n\n        script.push(b'\\\"');\n        for ch in message_body {\n            if [b'\\\\', b'\\\"'].contains(&ch) {\n                script.push(b'\\\\');\n            }\n            script.push(ch);\n        }\n        script.extend_from_slice(b\"\\\";\\r\\n\");\n\n        // Close blocks\n        for _ in 0..num_blocks {\n            script.extend_from_slice(b\"}\\r\\n\");\n        }\n\n        match self.core.sieve.untrusted_compiler.compile(&script) {\n            Ok(compiled_script) => {\n                // Update blob length\n                obj.size = script.len() as u32;\n\n                // Serialize script\n                script.extend(\n                    Archiver::new(compiled_script)\n                        .untrusted()\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n\n                Ok(script)\n            }\n            Err(err) => Err(trc::StoreEvent::UnexpectedError\n                .caused_by(trc::location!())\n                .reason(err)\n                .details(\"Vacation Sieve Script failed to compile.\")),\n        }\n    }\n}\n\nfn set_error(\n    mut response: SetResponse<vacation_response::VacationResponse>,\n    id: Option<String>,\n    err: SetError<VacationResponseProperty>,\n) -> SetResponse<vacation_response::VacationResponse> {\n    if let Some(id) = id {\n        response.not_created.append(id, err);\n    } else {\n        response.not_updated.append(Id::singleton(), err);\n    }\n    response\n}\n"
  },
  {
    "path": "crates/jmap/src/websocket/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod stream;\npub mod upgrade;\n"
  },
  {
    "path": "crates/jmap/src/websocket/stream.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::api::{IntoPushObject, ToRequestError, request::RequestHandler};\nuse common::{Server, auth::AccessToken, ipc::PushNotification};\nuse futures_util::{SinkExt, StreamExt};\nuse http_proto::HttpSessionData;\nuse hyper::upgrade::Upgraded;\nuse hyper_util::rt::TokioIo;\nuse jmap_proto::{\n    error::request::RequestError,\n    request::websocket::{\n        WebSocketMessage, WebSocketPushObject, WebSocketRequestError, WebSocketResponse,\n    },\n};\nuse std::future::Future;\nuse std::{sync::Arc, time::Instant};\nuse tokio_tungstenite::WebSocketStream;\nuse trc::JmapEvent;\nuse tungstenite::Message;\nuse types::type_state::{DataType, StateChange};\nuse utils::map::bitmap::Bitmap;\n\npub trait WebSocketHandler: Sync + Send {\n    fn handle_websocket_stream(\n        &self,\n        stream: WebSocketStream<TokioIo<Upgraded>>,\n        access_token: Arc<AccessToken>,\n        session: HttpSessionData,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl WebSocketHandler for Server {\n    #![allow(clippy::large_futures)]\n    async fn handle_websocket_stream(\n        &self,\n        mut stream: WebSocketStream<TokioIo<Upgraded>>,\n        access_token: Arc<AccessToken>,\n        session: HttpSessionData,\n    ) {\n        trc::event!(\n            Jmap(JmapEvent::WebsocketStart),\n            SpanId = session.session_id,\n            AccountId = access_token.primary_id(),\n        );\n\n        // Set timeouts\n        let throttle = self.core.jmap.web_socket_throttle;\n        let timeout = self.core.jmap.web_socket_timeout;\n        let heartbeat = self.core.jmap.web_socket_heartbeat;\n        let mut last_request = Instant::now();\n        let mut last_changes_sent = Instant::now() - throttle;\n        let mut last_heartbeat = Instant::now() - heartbeat;\n        let mut next_event = heartbeat;\n\n        // Register with push manager\n        let mut push_rx = match self\n            .subscribe_push_manager(&access_token, Bitmap::all())\n            .await\n        {\n            Ok(push_rx) => push_rx,\n            Err(err) => {\n                trc::error!(\n                    err.details(\"Failed to subscribe to push manager\")\n                        .span_id(session.session_id)\n                );\n\n                let _ = stream\n                    .send(Message::Text(\n                        WebSocketRequestError::from(RequestError::internal_server_error())\n                            .to_json()\n                            .into(),\n                    ))\n                    .await;\n                return;\n            }\n        };\n\n        let mut notifications = Vec::new();\n        let mut change_types: Bitmap<DataType> = Bitmap::new();\n\n        loop {\n            tokio::select! {\n                event = tokio::time::timeout(next_event, stream.next()) => {\n                    match event {\n                        Ok(Some(Ok(event))) => {\n                            match event {\n                                Message::Text(text) => {\n                                    let response = match WebSocketMessage::parse(\n                                        text.as_bytes(),\n                                        self.core.jmap.request_max_calls,\n                                        self.core.jmap.request_max_size,\n                                    ) {\n                                        Ok(WebSocketMessage::Request(request)) => {\n                                            let response = self\n                                                .handle_jmap_request(\n                                                    request.request,\n                                                    access_token.clone(),\n                                                    &session,\n                                                )\n                                                .await;\n                                            WebSocketResponse::from_response(response, request.id)\n                                            .to_json()\n                                        }\n                                        Ok(WebSocketMessage::PushEnable(push_enable)) => {\n                                            change_types = if !push_enable.data_types.is_empty() {\n                                                push_enable.data_types.into()\n                                            } else {\n                                                Bitmap::all()\n                                            };\n                                            continue;\n                                        }\n                                        Ok(WebSocketMessage::PushDisable) => {\n                                            change_types = Bitmap::new();\n                                            continue;\n                                        }\n                                        Err(err) => {\n                                            let response = WebSocketRequestError::from(err.to_request_error()).to_json();\n                                            trc::error!(err.details(\"Failed to parse WebSocket message\").span_id(session.session_id));\n                                            response\n                                        },\n                                    };\n                                    if let Err(err) = stream.send(Message::Text(response.into())).await {\n                                        trc::event!(Jmap(JmapEvent::WebsocketError),\n                                                    Details = \"Failed to send text message\",\n                                                    SpanId = session.session_id,\n                                                    Reason = err.to_string()\n                                        );\n                                    }\n                                }\n                                Message::Ping(bytes) => {\n                                    if let Err(err) = stream.send(Message::Pong(bytes)).await {\n                                        trc::event!(Jmap(JmapEvent::WebsocketError),\n                                                    Details = \"Failed to send pong message\",\n                                                    SpanId = session.session_id,\n                                                    Reason = err.to_string()\n                                        );\n                                    }\n                                }\n                                Message::Close(frame) => {\n                                    let _ = stream.close(frame).await;\n                                    break;\n                                }\n                                _ => (),\n                            }\n\n                            last_request = Instant::now();\n                            last_heartbeat = Instant::now();\n                        }\n                        Ok(Some(Err(err))) => {\n                            trc::event!(Jmap(JmapEvent::WebsocketError),\n                                                    Details = \"Websocket error\",\n                                                    SpanId = session.session_id,\n                                                    Reason = err.to_string()\n                                        );\n                            break;\n                        }\n                        Ok(None) => break,\n                        Err(_) => {\n                            // Verify timeout\n                            if last_request.elapsed() > timeout {\n                                trc::event!(\n                                    Jmap(JmapEvent::WebsocketStop),\n                                    SpanId = session.session_id,\n                                    Reason = \"Idle client\"\n                                );\n\n                                break;\n                            }\n                        }\n                    }\n                }\n                push_notification = push_rx.recv() => {\n                    if let Some(push_notification) = push_notification {\n                        match push_notification {\n                            PushNotification::StateChange(state_change) => {\n                                let mut types = state_change.types;\n                                types.intersection(&change_types);\n\n                                if !types.is_empty() {\n                                    notifications.push(PushNotification::StateChange(\n                                        StateChange {\n                                            account_id: state_change.account_id,\n                                            types,\n                                            change_id: state_change.change_id,\n                                        }\n                                    ));\n                                }\n                            },\n                            PushNotification::EmailPush(email_push) => {\n                                let state_change = email_push.to_state_change();\n                                let mut types = state_change.types;\n                                types.intersection(&change_types);\n\n                                if !types.is_empty() {\n                                    notifications.push(PushNotification::StateChange(\n                                        StateChange {\n                                            account_id: state_change.account_id,\n                                            types,\n                                            change_id: state_change.change_id,\n                                        }\n                                    ));\n                                }\n                            },\n                            PushNotification::CalendarAlert(calendar_alert) => {\n                                if change_types.contains(DataType::CalendarAlert) {\n                                    notifications.push(PushNotification::CalendarAlert(\n                                        calendar_alert\n                                    ));\n                                }\n                            },\n                        }\n\n                    } else {\n                        trc::event!(\n                            Jmap(JmapEvent::WebsocketStop),\n                            SpanId = session.session_id,\n                            Reason = \"State manager channel closed\"\n                        );\n\n                        break;\n                    }\n                }\n            }\n\n            if !notifications.is_empty() {\n                // Send any queued changes\n                let elapsed = last_changes_sent.elapsed();\n                if elapsed >= throttle {\n                    let payload = WebSocketPushObject {\n                        push: std::mem::take(&mut notifications).into_push_object(),\n                        push_state: None,\n                    };\n                    if let Err(err) = stream.send(Message::Text(payload.to_json().into())).await {\n                        trc::event!(\n                            Jmap(JmapEvent::WebsocketError),\n                            Details = \"Failed to send state change message.\",\n                            SpanId = session.session_id,\n                            Reason = err.to_string()\n                        );\n                    }\n                    last_changes_sent = Instant::now();\n                    last_heartbeat = Instant::now();\n                    next_event = heartbeat;\n                } else {\n                    next_event = throttle - elapsed;\n                }\n            } else if last_heartbeat.elapsed() > heartbeat {\n                if let Err(err) = stream.send(Message::Ping(Vec::<u8>::new().into())).await {\n                    trc::event!(\n                        Jmap(JmapEvent::WebsocketError),\n                        Details = \"Failed to send ping message.\",\n                        SpanId = session.session_id,\n                        Reason = err.to_string()\n                    );\n                    break;\n                }\n                last_heartbeat = Instant::now();\n                next_event = heartbeat;\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap/src/websocket/upgrade.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse common::{Server, auth::AccessToken};\nuse hyper::StatusCode;\nuse hyper_util::rt::TokioIo;\nuse tokio_tungstenite::WebSocketStream;\nuse trc::JmapEvent;\nuse tungstenite::{handshake::derive_accept_key, protocol::Role};\n\nuse http_proto::*;\nuse std::future::Future;\n\nuse super::stream::WebSocketHandler;\n\npub trait WebSocketUpgrade: Sync + Send {\n    fn upgrade_websocket_connection(\n        &self,\n        req: HttpRequest,\n        access_token: Arc<AccessToken>,\n        session: HttpSessionData,\n    ) -> impl Future<Output = trc::Result<HttpResponse>> + Send;\n}\n\nimpl WebSocketUpgrade for Server {\n    async fn upgrade_websocket_connection(\n        &self,\n        req: HttpRequest,\n        access_token: Arc<AccessToken>,\n        session: HttpSessionData,\n    ) -> trc::Result<HttpResponse> {\n        let headers = req.headers();\n        if headers\n            .get(hyper::header::CONNECTION)\n            .and_then(|h| h.to_str().ok())\n            != Some(\"Upgrade\")\n            || headers\n                .get(hyper::header::UPGRADE)\n                .and_then(|h| h.to_str().ok())\n                != Some(\"websocket\")\n        {\n            return Err(trc::ResourceEvent::BadParameters\n                .into_err()\n                .details(\"WebSocket upgrade failed\")\n                .ctx(\n                    trc::Key::Reason,\n                    \"Missing or Invalid Connection or Upgrade headers.\",\n                ));\n        }\n        let derived_key = match (\n            headers\n                .get(\"Sec-WebSocket-Key\")\n                .and_then(|h| h.to_str().ok()),\n            headers\n                .get(\"Sec-WebSocket-Version\")\n                .and_then(|h| h.to_str().ok()),\n        ) {\n            (Some(key), Some(\"13\")) => derive_accept_key(key.as_bytes()),\n            _ => {\n                return Err(trc::ResourceEvent::BadParameters\n                    .into_err()\n                    .details(\"WebSocket upgrade failed\")\n                    .ctx(\n                        trc::Key::Reason,\n                        \"Missing or Invalid Sec-WebSocket-Key headers.\",\n                    ));\n            }\n        };\n\n        // Spawn WebSocket connection\n        let jmap = self.clone();\n        tokio::spawn(async move {\n            // Upgrade connection\n            let session_id = session.session_id;\n            match hyper::upgrade::on(req).await {\n                Ok(upgraded) => {\n                    Box::pin(\n                        jmap.handle_websocket_stream(\n                            WebSocketStream::from_raw_socket(\n                                TokioIo::new(upgraded),\n                                Role::Server,\n                                None,\n                            )\n                            .await,\n                            access_token,\n                            session,\n                        ),\n                    )\n                    .await;\n                }\n                Err(err) => {\n                    trc::event!(\n                        Jmap(JmapEvent::WebsocketError),\n                        Details = \"Websocket upgrade failed\",\n                        SpanId = session_id,\n                        Reason = err.to_string()\n                    );\n                }\n            }\n        });\n\n        Ok(HttpResponse::new(StatusCode::SWITCHING_PROTOCOLS).with_websocket_upgrade(derived_key))\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/Cargo.toml",
    "content": "[package]\nname = \"jmap_proto\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path = \"../utils\" }\ntypes = { path = \"../types\" }\ntrc = { path = \"../trc\" }\njmap-tools = { version = \"0.1\" }\ncalcard = { version = \"0.3\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] } \nserde = { version = \"1.0\", features = [\"derive\"]}\nahash = { version = \"0.8.2\", features = [\"serde\"] }\nserde_json = { version = \"1.0\", features = [\"raw_value\"] }\nhashify = \"0.2\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = { version = \"0.9.0\", features = [\"rkyv\", \"serde\"] }\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/jmap-proto/src/error/method.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse serde::Serialize;\nuse serde::ser::SerializeMap;\nuse std::fmt::Display;\n\n#[derive(Debug)]\npub enum MethodError {\n    InvalidArguments(String),\n    RequestTooLarge,\n    StateMismatch,\n    AnchorNotFound,\n    UnsupportedFilter(String),\n    UnsupportedSort(String),\n    ServerFail(String),\n    UnknownMethod(String),\n    ServerUnavailable,\n    ServerPartialFail,\n    InvalidResultReference(String),\n    Forbidden(String),\n    AccountNotFound,\n    AccountNotSupportedByMethod,\n    AccountReadOnly,\n    NotFound,\n    CannotCalculateChanges,\n    UnknownDataType,\n}\n\n#[derive(Debug)]\npub struct MethodErrorWrapper(trc::Error);\n\nimpl From<trc::Error> for MethodErrorWrapper {\n    fn from(value: trc::Error) -> Self {\n        MethodErrorWrapper(value)\n    }\n}\n\nimpl Display for MethodError {\n    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {\n        match self {\n            MethodError::InvalidArguments(err) => write!(f, \"Invalid arguments: {}\", err),\n            MethodError::RequestTooLarge => write!(f, \"Request too large\"),\n            MethodError::StateMismatch => write!(f, \"State mismatch\"),\n            MethodError::AnchorNotFound => write!(f, \"Anchor not found\"),\n            MethodError::UnsupportedFilter(err) => write!(f, \"Unsupported filter: {}\", err),\n            MethodError::UnsupportedSort(err) => write!(f, \"Unsupported sort: {}\", err),\n            MethodError::ServerFail(err) => write!(f, \"Server error: {}\", err),\n            MethodError::UnknownMethod(err) => write!(f, \"Unknown method: {}\", err),\n            MethodError::ServerUnavailable => write!(f, \"Server unavailable\"),\n            MethodError::ServerPartialFail => write!(f, \"Server partial fail\"),\n            MethodError::InvalidResultReference(err) => {\n                write!(f, \"Invalid result reference: {}\", err)\n            }\n            MethodError::Forbidden(err) => write!(f, \"Forbidden: {}\", err),\n            MethodError::AccountNotFound => write!(f, \"Account not found\"),\n            MethodError::AccountNotSupportedByMethod => {\n                write!(f, \"Account not supported by method\")\n            }\n            MethodError::AccountReadOnly => write!(f, \"Account read only\"),\n            MethodError::NotFound => write!(f, \"Not found\"),\n            MethodError::UnknownDataType => write!(f, \"Unknown data type\"),\n            MethodError::CannotCalculateChanges => write!(f, \"Cannot calculate changes\"),\n        }\n    }\n}\n\nimpl Serialize for MethodErrorWrapper {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        let mut map = serializer.serialize_map(2.into())?;\n\n        let description = self.0.value(trc::Key::Details).and_then(|v| v.as_str());\n\n        let (error_type, description) = match self.0.as_ref() {\n            trc::EventType::Jmap(cause) => match cause {\n                trc::JmapEvent::InvalidArguments => {\n                    (\"invalidArguments\", description.unwrap_or_default())\n                }\n                trc::JmapEvent::RequestTooLarge => (\n                    \"requestTooLarge\",\n                    concat!(\n                        \"The number of ids requested by the client exceeds the maximum number \",\n                        \"the server is willing to process in a single method call.\"\n                    ),\n                ),\n                trc::JmapEvent::StateMismatch => (\n                    \"stateMismatch\",\n                    concat!(\n                        \"An \\\"ifInState\\\" argument was supplied, but \",\n                        \"it does not match the current state.\"\n                    ),\n                ),\n                trc::JmapEvent::AnchorNotFound => (\n                    \"anchorNotFound\",\n                    concat!(\n                        \"An anchor argument was supplied, but it \",\n                        \"cannot be found in the results of the query.\"\n                    ),\n                ),\n                trc::JmapEvent::UnsupportedFilter => {\n                    (\"unsupportedFilter\", description.unwrap_or_default())\n                }\n                trc::JmapEvent::UnsupportedSort => {\n                    (\"unsupportedSort\", description.unwrap_or_default())\n                }\n                trc::JmapEvent::NotFound => (\"serverPartialFail\", {\n                    concat!(\n                        \"One or more items are no longer available on the \",\n                        \"server, please try again.\"\n                    )\n                }),\n                trc::JmapEvent::UnknownMethod => (\"unknownMethod\", description.unwrap_or_default()),\n                trc::JmapEvent::InvalidResultReference => {\n                    (\"invalidResultReference\", description.unwrap_or_default())\n                }\n                trc::JmapEvent::Forbidden => (\"forbidden\", description.unwrap_or_default()),\n                trc::JmapEvent::AccountNotFound => (\n                    \"accountNotFound\",\n                    \"The accountId does not correspond to a valid account\",\n                ),\n                trc::JmapEvent::AccountNotSupportedByMethod => (\n                    \"accountNotSupportedByMethod\",\n                    concat!(\n                        \"The accountId given corresponds to a valid account, \",\n                        \"but the account does not support this method or data type.\"\n                    ),\n                ),\n                trc::JmapEvent::AccountReadOnly => (\n                    \"accountReadOnly\",\n                    \"This method modifies state, but the account is read-only.\",\n                ),\n                trc::JmapEvent::UnknownDataType => (\n                    \"unknownDataType\",\n                    concat!(\n                        \"The server does not recognise this data type, \",\n                        \"or the capability to enable it is not present \",\n                        \"in the current Request Object.\"\n                    ),\n                ),\n                trc::JmapEvent::CannotCalculateChanges => (\n                    \"cannotCalculateChanges\",\n                    concat!(\n                        \"The server cannot calculate the changes \",\n                        \"between the old and new states.\"\n                    ),\n                ),\n                trc::JmapEvent::UnknownCapability\n                | trc::JmapEvent::NotJson\n                | trc::JmapEvent::NotRequest => (\n                    \"serverUnavailable\",\n                    concat!(\n                        \"This server is temporarily unavailable. \",\n                        \"Attempting this same operation later may succeed.\"\n                    ),\n                ),\n                _ => (\n                    \"serverUnavailable\",\n                    \"This server is temporarily unavailable.\",\n                ),\n            },\n            _ => (\n                \"serverUnavailable\",\n                concat!(\n                    \"This server is temporarily unavailable. \",\n                    \"Attempting this same operation later may succeed.\"\n                ),\n            ),\n        };\n\n        map.serialize_entry(\"type\", error_type)?;\n        if !description.is_empty() {\n            map.serialize_entry(\"description\", description)?;\n        }\n        map.end()\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/error/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod method;\npub mod request;\npub mod set;\n"
  },
  {
    "path": "crates/jmap-proto/src/error/request.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, fmt::Display};\n\n#[derive(Debug, Clone, Copy, serde::Serialize, serde::Deserialize)]\npub enum RequestLimitError {\n    #[serde(rename = \"maxSizeRequest\")]\n    SizeRequest,\n    #[serde(rename = \"maxSizeUpload\")]\n    SizeUpload,\n    #[serde(rename = \"maxCallsInRequest\")]\n    CallsIn,\n    #[serde(rename = \"maxConcurrentRequests\")]\n    ConcurrentRequest,\n    #[serde(rename = \"maxConcurrentUpload\")]\n    ConcurrentUpload,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub enum RequestErrorType {\n    #[serde(rename = \"urn:ietf:params:jmap:error:unknownCapability\")]\n    UnknownCapability,\n    #[serde(rename = \"urn:ietf:params:jmap:error:notJSON\")]\n    NotJSON,\n    #[serde(rename = \"urn:ietf:params:jmap:error:notRequest\")]\n    NotRequest,\n    #[serde(rename = \"urn:ietf:params:jmap:error:limit\")]\n    Limit,\n    #[serde(rename = \"about:blank\")]\n    Other,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct RequestError<'x> {\n    #[serde(rename = \"type\")]\n    pub p_type: RequestErrorType,\n    pub status: u16,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub title: Option<Cow<'x, str>>,\n    pub detail: Cow<'x, str>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub limit: Option<RequestLimitError>,\n}\n\nimpl<'x> RequestError<'x> {\n    pub fn blank(\n        status: u16,\n        title: impl Into<Cow<'x, str>>,\n        detail: impl Into<Cow<'x, str>>,\n    ) -> Self {\n        RequestError {\n            p_type: RequestErrorType::Other,\n            status,\n            title: Some(title.into()),\n            detail: detail.into(),\n            limit: None,\n        }\n    }\n\n    pub fn internal_server_error() -> Self {\n        RequestError::blank(\n            500,\n            \"Internal Server Error\",\n            concat!(\n                \"There was a problem while processing your request. \",\n                \"Please contact the system administrator if this problem persists.\"\n            ),\n        )\n    }\n\n    pub fn unavailable() -> Self {\n        RequestError::blank(\n            503,\n            \"Temporarily Unavailable\",\n            concat!(\n                \"There was a temporary problem while processing your request. \",\n                \"Please try again in a few moments.\"\n            ),\n        )\n    }\n\n    pub fn invalid_parameters() -> Self {\n        RequestError::blank(\n            400,\n            \"Invalid Parameters\",\n            \"One or multiple parameters could not be parsed.\",\n        )\n    }\n\n    pub fn forbidden() -> Self {\n        RequestError::blank(\n            403,\n            \"Forbidden\",\n            \"You do not have enough permissions to access this resource.\",\n        )\n    }\n\n    pub fn over_blob_quota(max_files: usize, max_bytes: usize) -> Self {\n        RequestError::blank(\n            403,\n            \"Quota exceeded\",\n            format!(\n                \"You have exceeded the blob upload quota of {} files or {} bytes.\",\n                max_files, max_bytes\n            ),\n        )\n    }\n\n    pub fn over_quota() -> Self {\n        RequestError::blank(\n            403,\n            \"Quota exceeded\",\n            \"You have exceeded your account quota.\",\n        )\n    }\n\n    pub fn tenant_over_quota() -> Self {\n        RequestError::blank(\n            403,\n            \"Tenant quota exceeded\",\n            \"Your organization has exceeded its quota.\",\n        )\n    }\n\n    pub fn too_many_requests() -> Self {\n        RequestError::blank(\n            429,\n            \"Too Many Requests\",\n            \"Your request has been rate limited. Please try again in a few seconds.\",\n        )\n    }\n\n    pub fn too_many_auth_attempts() -> Self {\n        RequestError::blank(\n            429,\n            \"Too Many Authentication Attempts\",\n            \"Your request has been rate limited. Please try again in a few minutes.\",\n        )\n    }\n\n    pub fn limit(limit_type: RequestLimitError) -> Self {\n        RequestError {\n            p_type: RequestErrorType::Limit,\n            status: 400,\n            title: None,\n            detail: match limit_type {\n                RequestLimitError::SizeRequest => concat!(\n                    \"The request is larger than the server \",\n                    \"is willing to process.\"\n                ),\n                RequestLimitError::SizeUpload => concat!(\n                    \"The uploaded file is larger than the server \",\n                    \"is willing to process.\"\n                ),\n                RequestLimitError::CallsIn => concat!(\n                    \"The request exceeds the maximum number \",\n                    \"of calls in a single request.\"\n                ),\n                RequestLimitError::ConcurrentRequest => concat!(\n                    \"The request exceeds the maximum number \",\n                    \"of concurrent requests.\"\n                ),\n                RequestLimitError::ConcurrentUpload => concat!(\n                    \"The request exceeds the maximum number \",\n                    \"of concurrent uploads.\"\n                ),\n            }\n            .into(),\n            limit: Some(limit_type),\n        }\n    }\n\n    pub fn not_found() -> Self {\n        RequestError::blank(\n            404,\n            \"Not Found\",\n            \"The requested resource does not exist on this server.\",\n        )\n    }\n\n    pub fn unauthorized() -> Self {\n        RequestError::blank(401, \"Unauthorized\", \"You have to authenticate first.\")\n    }\n\n    pub fn unknown_capability(capability: &'_ str) -> RequestError<'_> {\n        RequestError {\n            p_type: RequestErrorType::UnknownCapability,\n            limit: None,\n            title: None,\n            status: 400,\n            detail: format!(\n                concat!(\n                    \"The Request object used capability \",\n                    \"'{}', which is not supported\",\n                    \"by this server.\"\n                ),\n                capability\n            )\n            .into(),\n        }\n    }\n\n    pub fn not_json(detail: &'_ str) -> RequestError<'_> {\n        RequestError {\n            p_type: RequestErrorType::NotJSON,\n            limit: None,\n            title: None,\n            status: 400,\n            detail: format!(\"Failed to parse JSON: {detail}\").into(),\n        }\n    }\n\n    pub fn not_request(detail: impl Into<Cow<'x, str>>) -> RequestError<'x> {\n        RequestError {\n            p_type: RequestErrorType::NotRequest,\n            limit: None,\n            title: None,\n            status: 400,\n            detail: detail.into(),\n        }\n    }\n}\n\nimpl Display for RequestError<'_> {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(&self.detail)\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/error/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_tools::{Key, Property};\nuse std::borrow::Cow;\nuse types::id::Id;\n\n#[derive(Debug, Clone, serde::Serialize)]\n#[serde(bound(serialize = \"InvalidProperty<P>: serde::Serialize\"))]\npub struct SetError<P: Property> {\n    #[serde(rename = \"type\")]\n    pub type_: SetErrorType,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub description: Option<Cow<'static, str>>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub properties: Option<Vec<InvalidProperty<P>>>,\n\n    #[serde(rename = \"existingId\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    existing_id: Option<Id>,\n}\n\n#[derive(Debug, Clone)]\npub enum InvalidProperty<T: Property> {\n    Property(Key<'static, T>),\n    Path(Vec<Key<'static, T>>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]\npub enum SetErrorType {\n    #[serde(rename = \"forbidden\")]\n    Forbidden,\n    #[serde(rename = \"overQuota\")]\n    OverQuota,\n    #[serde(rename = \"tooLarge\")]\n    TooLarge,\n    #[serde(rename = \"rateLimit\")]\n    RateLimit,\n    #[serde(rename = \"notFound\")]\n    NotFound,\n    #[serde(rename = \"invalidPatch\")]\n    InvalidPatch,\n    #[serde(rename = \"willDestroy\")]\n    WillDestroy,\n    #[serde(rename = \"invalidProperties\")]\n    InvalidProperties,\n    #[serde(rename = \"singleton\")]\n    Singleton,\n    #[serde(rename = \"mailboxHasChild\")]\n    MailboxHasChild,\n    #[serde(rename = \"mailboxHasEmail\")]\n    MailboxHasEmail,\n    #[serde(rename = \"blobNotFound\")]\n    BlobNotFound,\n    #[serde(rename = \"tooManyKeywords\")]\n    TooManyKeywords,\n    #[serde(rename = \"tooManyMailboxes\")]\n    TooManyMailboxes,\n    #[serde(rename = \"forbiddenFrom\")]\n    ForbiddenFrom,\n    #[serde(rename = \"invalidEmail\")]\n    InvalidEmail,\n    #[serde(rename = \"tooManyRecipients\")]\n    TooManyRecipients,\n    #[serde(rename = \"noRecipients\")]\n    NoRecipients,\n    #[serde(rename = \"invalidRecipients\")]\n    InvalidRecipients,\n    #[serde(rename = \"forbiddenMailFrom\")]\n    ForbiddenMailFrom,\n    #[serde(rename = \"forbiddenToSend\")]\n    ForbiddenToSend,\n    #[serde(rename = \"cannotUnsend\")]\n    CannotUnsend,\n    #[serde(rename = \"alreadyExists\")]\n    AlreadyExists,\n    #[serde(rename = \"invalidScript\")]\n    InvalidScript,\n    #[serde(rename = \"scriptIsActive\")]\n    ScriptIsActive,\n    #[serde(rename = \"addressBookHasContents\")]\n    AddressBookHasContents,\n    #[serde(rename = \"nodeHasChildren\")]\n    NodeHasChildren,\n    #[serde(rename = \"calendarHasEvent\")]\n    CalendarHasEvent,\n}\n\nimpl SetErrorType {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            SetErrorType::Forbidden => \"forbidden\",\n            SetErrorType::OverQuota => \"overQuota\",\n            SetErrorType::TooLarge => \"tooLarge\",\n            SetErrorType::RateLimit => \"rateLimit\",\n            SetErrorType::NotFound => \"notFound\",\n            SetErrorType::InvalidPatch => \"invalidPatch\",\n            SetErrorType::WillDestroy => \"willDestroy\",\n            SetErrorType::InvalidProperties => \"invalidProperties\",\n            SetErrorType::Singleton => \"singleton\",\n            SetErrorType::BlobNotFound => \"blobNotFound\",\n            SetErrorType::MailboxHasChild => \"mailboxHasChild\",\n            SetErrorType::MailboxHasEmail => \"mailboxHasEmail\",\n            SetErrorType::TooManyKeywords => \"tooManyKeywords\",\n            SetErrorType::TooManyMailboxes => \"tooManyMailboxes\",\n            SetErrorType::ForbiddenFrom => \"forbiddenFrom\",\n            SetErrorType::InvalidEmail => \"invalidEmail\",\n            SetErrorType::TooManyRecipients => \"tooManyRecipients\",\n            SetErrorType::NoRecipients => \"noRecipients\",\n            SetErrorType::InvalidRecipients => \"invalidRecipients\",\n            SetErrorType::ForbiddenMailFrom => \"forbiddenMailFrom\",\n            SetErrorType::ForbiddenToSend => \"forbiddenToSend\",\n            SetErrorType::CannotUnsend => \"cannotUnsend\",\n            SetErrorType::AlreadyExists => \"alreadyExists\",\n            SetErrorType::InvalidScript => \"invalidScript\",\n            SetErrorType::ScriptIsActive => \"scriptIsActive\",\n            SetErrorType::AddressBookHasContents => \"addressBookHasContents\",\n            SetErrorType::NodeHasChildren => \"nodeHasChildren\",\n            SetErrorType::CalendarHasEvent => \"calendarHasEvent\",\n        }\n    }\n}\n\nimpl<T: Property> SetError<T> {\n    pub fn new(type_: SetErrorType) -> Self {\n        SetError {\n            type_,\n            description: None,\n            properties: None,\n            existing_id: None,\n        }\n    }\n\n    pub fn with_description(mut self, description: impl Into<Cow<'static, str>>) -> Self {\n        self.description = description.into().into();\n        self\n    }\n\n    pub fn with_property(mut self, property: impl Into<InvalidProperty<T>>) -> Self {\n        self.properties = vec![property.into()].into();\n        self\n    }\n\n    pub fn with_properties(\n        mut self,\n        properties: impl IntoIterator<Item = impl Into<InvalidProperty<T>>>,\n    ) -> Self {\n        self.properties = properties\n            .into_iter()\n            .map(Into::into)\n            .collect::<Vec<_>>()\n            .into();\n        self\n    }\n\n    pub fn with_existing_id(mut self, id: Id) -> Self {\n        self.existing_id = id.into();\n        self\n    }\n\n    pub fn invalid_properties() -> Self {\n        Self::new(SetErrorType::InvalidProperties)\n    }\n\n    pub fn forbidden() -> Self {\n        Self::new(SetErrorType::Forbidden)\n    }\n\n    pub fn not_found() -> Self {\n        Self::new(SetErrorType::NotFound)\n    }\n\n    pub fn blob_not_found() -> Self {\n        Self::new(SetErrorType::BlobNotFound)\n    }\n\n    pub fn over_quota() -> Self {\n        Self::new(SetErrorType::OverQuota).with_description(\"Account quota exceeded.\")\n    }\n\n    pub fn already_exists() -> Self {\n        Self::new(SetErrorType::AlreadyExists)\n    }\n\n    pub fn too_large() -> Self {\n        Self::new(SetErrorType::TooLarge)\n    }\n\n    pub fn will_destroy() -> Self {\n        Self::new(SetErrorType::WillDestroy).with_description(\"ID will be destroyed.\")\n    }\n\n    pub fn address_book_has_contents() -> Self {\n        Self::new(SetErrorType::AddressBookHasContents)\n            .with_description(\"Address book is not empty.\")\n    }\n\n    pub fn node_has_children() -> Self {\n        Self::new(SetErrorType::NodeHasChildren).with_description(\"Cannot delete non-empty folder.\")\n    }\n\n    pub fn calendar_has_event() -> Self {\n        Self::new(SetErrorType::CalendarHasEvent).with_description(\"Calendar is not empty.\")\n    }\n}\n\nimpl<T: Property> From<T> for InvalidProperty<T> {\n    fn from(property: T) -> Self {\n        InvalidProperty::Property(Key::Property(property))\n    }\n}\n\nimpl<T: Property> From<(T, T)> for InvalidProperty<T> {\n    fn from((a, b): (T, T)) -> Self {\n        InvalidProperty::Path(vec![Key::Property(a), Key::Property(b)])\n    }\n}\n\nimpl<T: Property> From<Key<'static, T>> for InvalidProperty<T> {\n    fn from(property: Key<'static, T>) -> Self {\n        InvalidProperty::Property(property)\n    }\n}\n\nimpl<T: Property> serde::Serialize for InvalidProperty<T> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        match self {\n            InvalidProperty::Property(p) => p.serialize(serializer),\n            InvalidProperty::Path(p) => {\n                use std::fmt::Write;\n                let mut path = String::with_capacity(64);\n                for (i, p) in p.iter().enumerate() {\n                    if i > 0 {\n                        path.push('/');\n                    }\n                    let _ = write!(path, \"{}\", p.to_string());\n                }\n                path.serialize(serializer)\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod error;\npub mod method;\npub mod object;\npub mod references;\npub mod request;\npub mod response;\npub mod types;\n"
  },
  {
    "path": "crates/jmap-proto/src/method/availability.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    request::{\n        MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n    },\n    types::date::UTCDate,\n};\nuse calcard::jscalendar::{JSCalendar, JSCalendarProperty};\nuse serde::{Deserialize, Deserializer, Serialize};\nuse types::{blob::BlobId, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct GetAvailabilityRequest {\n    pub account_id: Id,\n    pub id: Id,\n    pub utc_start: UTCDate,\n    pub utc_end: UTCDate,\n    pub show_details: bool,\n    pub event_properties: Option<Vec<MaybeInvalid<JSCalendarProperty<Id>>>>,\n}\n\n#[derive(Debug, Serialize, Clone)]\n#[serde(rename_all = \"camelCase\")]\npub struct GetAvailabilityResponse {\n    pub list: Vec<BusyPeriod>,\n}\n\n#[derive(Debug, Serialize, Clone)]\n#[serde(rename_all = \"camelCase\")]\npub struct BusyPeriod {\n    pub utc_start: UTCDate,\n    pub utc_end: UTCDate,\n    pub busy_status: Option<BusyStatus>,\n    pub event: Option<JSCalendar<'static, Id, BlobId>>,\n}\n\n#[derive(Debug, Serialize, Clone, Copy, PartialOrd, Ord, PartialEq, Eq)]\n#[serde(rename_all = \"lowercase\")]\npub enum BusyStatus {\n    Confirmed,\n    Tentative,\n    Unavailable,\n}\n\nimpl<'de> DeserializeArguments<'de> for GetAvailabilityRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"utcStart\" => {\n                self.utc_start = map.next_value()?;\n            },\n            b\"utcEnd\" => {\n                self.utc_end = map.next_value()?;\n            },\n            b\"id\" => {\n                self.id = map.next_value()?;\n            },\n            b\"showDetails\" => {\n                self.show_details = map.next_value()?;\n            },\n            b\"eventProperties\" => {\n                self.event_properties = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> Deserialize<'de> for GetAvailabilityRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/changes.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    method::PropertyWrapper,\n    object::JmapObject,\n    request::deserialize::{DeserializeArguments, deserialize_request},\n    types::state::State,\n};\nuse serde::{Deserialize, Deserializer};\nuse types::id::Id;\n\n#[derive(Debug, Clone, Default)]\npub struct ChangesRequest {\n    pub account_id: Id,\n    pub since_state: State,\n    pub max_changes: Option<usize>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct ChangesResponse<T: JmapObject> {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"oldState\")]\n    pub old_state: State,\n\n    #[serde(rename = \"newState\")]\n    pub new_state: State,\n\n    #[serde(rename = \"hasMoreChanges\")]\n    pub has_more_changes: bool,\n\n    pub created: Vec<Id>,\n\n    pub updated: Vec<Id>,\n\n    pub destroyed: Vec<Id>,\n\n    #[serde(rename = \"updatedProperties\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub updated_properties: Option<Vec<PropertyWrapper<T::Property>>>,\n}\n\nimpl<'de> DeserializeArguments<'de> for ChangesRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"sinceState\" => {\n                self.since_state = map.next_value()?;\n            },\n            b\"maxChanges\" => {\n                self.max_changes = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> Deserialize<'de> for ChangesRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<T: JmapObject> ChangesResponse<T> {\n    pub fn has_changes(&self) -> bool {\n        !self.created.is_empty() || !self.updated.is_empty() || !self.destroyed.is_empty()\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/copy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    error::set::SetError,\n    object::{JmapObject, blob::BlobProperty},\n    request::{\n        MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n        reference::MaybeIdReference,\n    },\n    types::state::State,\n};\nuse jmap_tools::{Key, Map, Value};\nuse serde::{Deserialize, Deserializer, Serialize};\nuse types::{blob::BlobId, id::Id};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone)]\npub struct CopyRequest<'x, T: JmapObject> {\n    pub from_account_id: Id,\n    pub if_from_in_state: Option<State>,\n    pub account_id: Id,\n    pub if_in_state: Option<State>,\n    pub create: VecMap<MaybeIdReference<Id>, Value<'x, T::Property, T::Element>>,\n    pub on_success_destroy_original: Option<bool>,\n    pub destroy_from_if_in_state: Option<State>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct CopyResponse<T: JmapObject> {\n    #[serde(rename = \"fromAccountId\")]\n    pub from_account_id: Id,\n\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"oldState\")]\n    pub old_state: State,\n\n    #[serde(rename = \"newState\")]\n    pub new_state: State,\n\n    #[serde(rename = \"created\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub created: VecMap<Id, Value<'static, T::Property, T::Element>>,\n\n    #[serde(rename = \"notCreated\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub not_created: VecMap<Id, SetError<T::Property>>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct CopyBlobRequest {\n    pub from_account_id: Id,\n    pub account_id: Id,\n    pub blob_ids: Vec<MaybeInvalid<BlobId>>,\n}\n\n#[derive(Debug, Clone, Serialize)]\npub struct CopyBlobResponse {\n    #[serde(rename = \"fromAccountId\")]\n    pub from_account_id: Id,\n\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"copied\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub copied: VecMap<BlobId, BlobId>,\n\n    #[serde(rename = \"notCopied\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub not_copied: VecMap<BlobId, SetError<BlobProperty>>,\n}\n\nimpl<'de, T: JmapObject> DeserializeArguments<'de> for CopyRequest<'de, T> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"ifInState\" => {\n                self.if_in_state = map.next_value()?;\n            },\n            b\"fromAccountId\" => {\n                self.from_account_id = map.next_value()?;\n            },\n            b\"ifFromInState\" => {\n                self.if_from_in_state = map.next_value()?;\n            },\n            b\"create\" => {\n                self.create = map.next_value()?;\n            },\n            b\"onSuccessDestroyOriginal\" => {\n                self.on_success_destroy_original = map.next_value()?;\n            },\n            b\"destroyFromIfInState\" => {\n                self.destroy_from_if_in_state = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for CopyBlobRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"fromAccountId\" => {\n                self.from_account_id = map.next_value()?;\n            },\n            b\"blobIds\" => {\n                self.blob_ids = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de, T: JmapObject> Deserialize<'de> for CopyRequest<'de, T> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<'de> Deserialize<'de> for CopyBlobRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<'de, T: JmapObject> Default for CopyRequest<'de, T> {\n    fn default() -> Self {\n        CopyRequest {\n            from_account_id: Id::default(),\n            if_from_in_state: None,\n            account_id: Id::default(),\n            if_in_state: None,\n            create: VecMap::new(),\n            on_success_destroy_original: None,\n            destroy_from_if_in_state: None,\n        }\n    }\n}\n\nimpl<T: JmapObject> CopyResponse<T> {\n    pub fn created(&mut self, id: Id, document_id: impl Into<T::Id>) {\n        let document_id = document_id.into();\n        self.created.append(\n            id,\n            Value::Object(Map::from(vec![(\n                Key::Property(T::ID_PROPERTY),\n                Value::Element(document_id.into()),\n            )])),\n        );\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::JmapObject,\n    request::{\n        IntoValid, MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n        reference::{MaybeIdReference, MaybeResultReference, ResultReference},\n    },\n    types::state::State,\n};\nuse jmap_tools::Value;\nuse serde::{Deserialize, Deserializer};\nuse types::id::Id;\n\n#[derive(Debug, Clone)]\npub struct GetRequest<T: JmapObject> {\n    pub account_id: Id,\n    pub ids: Option<MaybeResultReference<Vec<MaybeIdReference<T::Id>>>>,\n    pub properties: Option<MaybeResultReference<Vec<MaybeInvalid<T::Property>>>>,\n    pub arguments: T::GetArguments,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct GetResponse<T: JmapObject> {\n    #[serde(rename = \"accountId\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub account_id: Option<Id>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub state: Option<State>,\n\n    pub list: Vec<Value<'static, T::Property, T::Element>>,\n\n    #[serde(rename = \"notFound\")]\n    pub not_found: Vec<T::Id>,\n}\n\nimpl<'de, T: JmapObject> DeserializeArguments<'de> for GetRequest<T> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"ids\" => {\n                self.ids = map.next_value::<Option<Vec<MaybeIdReference<T::Id>>>>()?.map(MaybeResultReference::Value);\n            },\n            b\"properties\" => {\n                self.properties = map.next_value::<Option<Vec<MaybeInvalid<T::Property>>>>()?.map(MaybeResultReference::Value);\n            },\n            b\"#ids\" => {\n                self.ids = Some(MaybeResultReference::Reference(map.next_value::<ResultReference>()?));\n            },\n            b\"#properties\" => {\n                self.properties = Some(MaybeResultReference::Reference(map.next_value::<ResultReference>()?));\n            },\n            _ => {\n                self.arguments.deserialize_argument(key, map)?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de, T: JmapObject> Deserialize<'de> for GetRequest<T> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<T: JmapObject> Default for GetRequest<T> {\n    fn default() -> Self {\n        Self {\n            account_id: Id::default(),\n            ids: None,\n            properties: None,\n            arguments: T::GetArguments::default(),\n        }\n    }\n}\n\nimpl<T: JmapObject> GetRequest<T> {\n    pub fn unwrap_properties(&mut self, default: &[T::Property]) -> Vec<T::Property> {\n        if let Some(properties_) = self.properties.take().map(|p| p.unwrap()) {\n            let mut properties = Vec::with_capacity(properties_.len());\n            let id_prop = T::ID_PROPERTY;\n            let mut has_id = false;\n\n            for prop in properties_ {\n                if let MaybeInvalid::Value(p) = prop {\n                    if p == id_prop {\n                        has_id = true;\n                    }\n                    properties.push(p);\n                }\n            }\n\n            if !has_id {\n                properties.push(id_prop);\n            }\n\n            properties\n        } else {\n            default.to_vec()\n        }\n    }\n\n    pub fn unwrap_ids(&mut self, max_objects_in_get: usize) -> trc::Result<Option<Vec<T::Id>>> {\n        if let Some(ids) = self.ids.take() {\n            let ids = ids.unwrap();\n            if ids.len() <= max_objects_in_get {\n                Ok(Some(ids.into_valid().collect::<Vec<_>>()))\n            } else {\n                Err(trc::JmapEvent::RequestTooLarge.into_err())\n            }\n        } else {\n            Ok(None)\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/import.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    error::set::SetError,\n    method::JmapDict,\n    object::{\n        AnyId,\n        email::{EmailProperty, EmailValue},\n    },\n    request::{\n        MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n        reference::{MaybeIdReference, MaybeResultReference, ResultReference},\n    },\n    response::Response,\n    types::{date::UTCDate, state::State},\n};\nuse jmap_tools::{Key, Value};\nuse serde::{Deserialize, Deserializer};\nuse types::{blob::BlobId, id::Id, keyword::Keyword};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone, Default)]\npub struct ImportEmailRequest {\n    pub account_id: Id,\n    pub if_in_state: Option<State>,\n    pub emails: VecMap<String, ImportEmail>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct ImportEmail {\n    pub blob_id: MaybeInvalid<BlobId>,\n    pub mailbox_ids: MaybeResultReference<Vec<MaybeIdReference<Id>>>,\n    pub keywords: Vec<Keyword>,\n    pub received_at: Option<UTCDate>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct ImportEmailResponse {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"oldState\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub old_state: Option<State>,\n\n    #[serde(rename = \"newState\")]\n    pub new_state: State,\n\n    #[serde(rename = \"created\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub created: VecMap<String, Value<'static, EmailProperty, EmailValue>>,\n\n    #[serde(rename = \"notCreated\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub not_created: VecMap<String, SetError<EmailProperty>>,\n}\n\nimpl<'de> DeserializeArguments<'de> for ImportEmailRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"ifInState\" => {\n                self.if_in_state = map.next_value()?;\n            },\n            b\"emails\" => {\n                self.emails = map.next_value()?;\n            }\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for ImportEmail {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"blobId\" => {\n                self.blob_id = map.next_value()?;\n            },\n            b\"keywords\" => {\n                self.keywords = map.next_value::<JmapDict<Keyword>>()?.0;\n            },\n            b\"receivedAt\" => {\n                self.received_at = map.next_value()?;\n            },\n            b\"mailboxIds\" => {\n                self.mailbox_ids = MaybeResultReference::Value(map.next_value::<JmapDict<MaybeIdReference<Id>>>()?.0);\n            },\n            b\"#mailboxIds\" => {\n                self.mailbox_ids = MaybeResultReference::Reference(map.next_value::<ResultReference>()?);\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> Deserialize<'de> for ImportEmail {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<'de> Deserialize<'de> for ImportEmailRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl ImportEmailResponse {\n    pub fn update_created_ids(&self, response: &mut Response) {\n        for (user_id, obj) in &self.created {\n            if let Value::Object(obj) = obj\n                && let Some(Value::Element(EmailValue::Id(id))) =\n                    obj.get(&Key::Property(EmailProperty::Id))\n            {\n                response.created_ids.insert(user_id.clone(), AnyId::Id(*id));\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::request::{\n    MaybeInvalid,\n    deserialize::{DeserializeArguments, deserialize_request},\n};\nuse serde::{Deserialize, Deserializer};\nuse types::{blob::BlobId, id::Id, type_state::DataType};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone, Default)]\npub struct BlobLookupRequest {\n    pub account_id: Id,\n    pub type_names: Vec<MaybeInvalid<DataType>>,\n    pub ids: Vec<MaybeInvalid<BlobId>>,\n}\n\n#[derive(Debug, Clone, Default, serde::Serialize)]\npub struct BlobLookupResponse {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"list\")]\n    pub list: Vec<BlobInfo>,\n\n    #[serde(rename = \"notFound\")]\n    pub not_found: Vec<BlobId>,\n}\n\n#[derive(Debug, Clone, Default, serde::Serialize)]\npub struct BlobInfo {\n    pub id: BlobId,\n    #[serde(rename = \"matchedIds\")]\n    pub matched_ids: VecMap<DataType, Vec<Id>>,\n}\n\nimpl<'de> DeserializeArguments<'de> for BlobLookupRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"typeNames\" => {\n                self.type_names = map.next_value()?;\n            },\n            b\"ids\" => {\n                self.ids = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> Deserialize<'de> for BlobLookupRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashMap;\nuse jmap_tools::Property;\nuse serde::{\n    Deserialize, Deserializer, Serialize, Serializer,\n    de::{self, MapAccess, Visitor},\n};\nuse std::{borrow::Cow, fmt, str::FromStr};\n\npub mod availability;\npub mod changes;\npub mod copy;\npub mod get;\npub mod import;\npub mod lookup;\npub mod parse;\npub mod query;\npub mod query_changes;\npub mod search_snippet;\npub mod set;\npub mod upload;\npub mod validate;\n\n#[inline(always)]\nfn ahash_is_empty<K, V>(map: &AHashMap<K, V>) -> bool {\n    map.is_empty()\n}\n\n#[derive(Debug, Clone)]\n#[repr(transparent)]\npub struct PropertyWrapper<T: Property>(pub T);\n\nimpl<T: Property> From<T> for PropertyWrapper<T> {\n    fn from(value: T) -> Self {\n        Self(value)\n    }\n}\n\nimpl<T: Property> Serialize for PropertyWrapper<T> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        serializer.serialize_str(self.0.to_cow().as_ref())\n    }\n}\n\npub(crate) struct JmapDict<T: FromStr>(pub Vec<T>);\n\nstruct JmapDictVisitor<'de, T: FromStr> {\n    marker: std::marker::PhantomData<&'de T>,\n}\n\nimpl<'de, T: FromStr> Visitor<'de> for JmapDictVisitor<'de, T> {\n    type Value = JmapDict<T>;\n\n    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n        formatter.write_str(\"a map\")\n    }\n\n    fn visit_map<M>(self, mut access: M) -> Result<Self::Value, M::Error>\n    where\n        M: MapAccess<'de>,\n    {\n        let mut vec = Vec::with_capacity(3);\n\n        while let Some(key) = access.next_key::<Cow<'de, str>>()? {\n            let key = T::from_str(&key).map_err(|_| de::Error::custom(\"invalid dictionary key\"))?;\n            if access.next_value::<Option<bool>>()?.unwrap_or(false) {\n                vec.push(key);\n            }\n        }\n\n        Ok(JmapDict(vec))\n    }\n}\n\nimpl<'de, T: FromStr + 'static> Deserialize<'de> for JmapDict<T> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserializer.deserialize_map(JmapDictVisitor {\n            marker: std::marker::PhantomData,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/parse.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::JmapObject,\n    request::{\n        MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n        reference::MaybeIdReference,\n    },\n};\nuse jmap_tools::Value;\nuse serde::{Deserialize, Deserializer};\nuse types::{blob::BlobId, id::Id};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone)]\npub struct ParseRequest<T: JmapObject> {\n    pub account_id: Id,\n    pub blob_ids: Vec<MaybeIdReference<BlobId>>,\n    pub properties: Option<Vec<MaybeInvalid<T::Property>>>,\n    pub arguments: T::ParseArguments,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct ParseResponse<T: JmapObject> {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"parsed\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub parsed: VecMap<BlobId, Value<'static, T::Property, T::Element>>,\n\n    #[serde(rename = \"notParsable\")]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub not_parsable: Vec<BlobId>,\n\n    #[serde(rename = \"notFound\")]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub not_found: Vec<BlobId>,\n}\n\nimpl<'de, T: JmapObject> DeserializeArguments<'de> for ParseRequest<T> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"blobIds\" => {\n                self.blob_ids = map.next_value()?;\n            },\n            b\"properties\" => {\n                self.properties = map.next_value()?;\n            },\n            _ => {\n                self.arguments.deserialize_argument(key, map)?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de, T: JmapObject> Deserialize<'de> for ParseRequest<T> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<T: JmapObject> Default for ParseRequest<T> {\n    fn default() -> Self {\n        Self {\n            account_id: Id::default(),\n            blob_ids: Vec::default(),\n            properties: None,\n            arguments: T::ParseArguments::default(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::JmapObject,\n    request::deserialize::{DeserializeArguments, deserialize_request},\n    types::state::State,\n};\nuse serde::{\n    Deserialize, Deserializer,\n    de::{self, DeserializeSeed, MapAccess, SeqAccess, Visitor},\n};\nuse std::{\n    borrow::Cow,\n    fmt::{self},\n};\nuse types::id::Id;\n\n#[derive(Debug, Clone)]\npub struct QueryRequest<T: JmapObject> {\n    pub account_id: Id,\n    pub filter: Vec<Filter<T::Filter>>,\n    pub sort: Option<Vec<Comparator<T::Comparator>>>,\n    pub position: Option<i32>,\n    pub anchor: Option<Id>,\n    pub anchor_offset: Option<i32>,\n    pub limit: Option<usize>,\n    pub calculate_total: Option<bool>,\n    pub arguments: T::QueryArguments,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct QueryResponse {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"queryState\")]\n    pub query_state: State,\n\n    #[serde(rename = \"canCalculateChanges\")]\n    pub can_calculate_changes: bool,\n\n    #[serde(rename = \"position\")]\n    pub position: i32,\n\n    #[serde(rename = \"ids\")]\n    pub ids: Vec<Id>,\n\n    #[serde(rename = \"total\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub total: Option<usize>,\n\n    #[serde(rename = \"limit\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub limit: Option<usize>,\n}\n\n#[derive(Clone, Debug)]\npub enum Filter<T>\nwhere\n    T: for<'de> DeserializeArguments<'de> + Default,\n{\n    Property(T),\n    And,\n    Or,\n    Not,\n    Close,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Comparator<T>\nwhere\n    T: for<'de> DeserializeArguments<'de> + Default,\n{\n    pub is_ascending: bool,\n    pub collation: Option<String>,\n    pub property: T,\n}\n\nimpl<'de, T: JmapObject> DeserializeArguments<'de> for QueryRequest<T> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"filter\" => {\n                self.filter = map.next_value::<FilterWrapper<T::Filter>>()?.0;\n            },\n            b\"sort\" => {\n                self.sort = map.next_value()?;\n            },\n            b\"calculateTotal\" => {\n                self.calculate_total = map.next_value()?;\n            },\n            b\"position\" => {\n                self.position = map.next_value()?;\n            },\n            b\"anchor\" => {\n                self.anchor = map.next_value()?;\n            },\n            b\"anchorOffset\" => {\n                self.anchor_offset = map.next_value()?;\n            },\n            b\"limit\" => {\n                self.limit = map.next_value()?;\n            },\n            _ => {\n                self.arguments.deserialize_argument(key, map)?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de, T: JmapObject> Deserialize<'de> for QueryRequest<T> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<T: JmapObject> Default for QueryRequest<T> {\n    fn default() -> Self {\n        Self {\n            account_id: Id::default(),\n            filter: vec![],\n            sort: None,\n            position: None,\n            anchor: None,\n            anchor_offset: None,\n            limit: None,\n            calculate_total: None,\n            arguments: T::QueryArguments::default(),\n        }\n    }\n}\n\nstruct FilterMapCollector<'x, T: 'x>(&'x mut Vec<Filter<T>>)\nwhere\n    T: for<'de> DeserializeArguments<'de> + Default;\n\nstruct FilterListCollector<'x, T: 'x>(&'x mut Vec<Filter<T>>)\nwhere\n    T: for<'de> DeserializeArguments<'de> + Default;\n\npub(super) struct FilterWrapper<T>(pub Vec<Filter<T>>)\nwhere\n    T: for<'de> DeserializeArguments<'de> + Default;\n\nimpl<'de, T> Deserialize<'de> for FilterWrapper<T>\nwhere\n    T: for<'de2> DeserializeArguments<'de2> + Default,\n{\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        let mut items = Vec::new();\n        FilterMapCollector(&mut items)\n            .deserialize(deserializer)\n            .map(|_| FilterWrapper(items))\n    }\n}\n\nimpl<'de, 'x, T> DeserializeSeed<'de> for FilterMapCollector<'x, T>\nwhere\n    T: for<'de2> DeserializeArguments<'de2> + Default,\n{\n    type Value = ();\n\n    fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        struct FilterVisitor<'x, T: 'x>(&'x mut Vec<Filter<T>>)\n        where\n            T: for<'de2> DeserializeArguments<'de2> + Default;\n\n        impl<'de, 'x, T> Visitor<'de> for FilterVisitor<'x, T>\n        where\n            T: for<'de2> DeserializeArguments<'de2> + Default,\n        {\n            type Value = ();\n\n            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n                write!(formatter, \"a filter object\")\n            }\n\n            fn visit_map<V>(self, mut map: V) -> Result<(), V::Error>\n            where\n                V: MapAccess<'de>,\n            {\n                let mut filter = None;\n                let mut has_multiple_filters = false;\n                let mut has_conditions = None;\n                let mut op = None;\n\n                while let Some(key) = map.next_key::<Cow<str>>()? {\n                    match key.len() {\n                        8 if key == \"operator\" => {\n                            let op_ = hashify::tiny_map!(\n                                map.next_value::<&str>()?.as_bytes(),\n                                \"AND\" => Filter::And,\n                                \"OR\" => Filter::Or,\n                                \"NOT\" => Filter::Not,\n                            )\n                            .ok_or_else(|| {\n                                de::Error::custom(format!(\"Unknown filter operator: {}\", key))\n                            })?;\n\n                            if let Some(pos) = has_conditions {\n                                self.0[pos] = op_;\n                            } else {\n                                op = Some(op_);\n                            }\n                        }\n                        10 if key == \"conditions\" => {\n                            has_conditions = Some(self.0.len());\n                            self.0.push(op.take().unwrap_or(Filter::And));\n                            map.next_value_seed(FilterListCollector(self.0))?;\n                            self.0.push(Filter::Close);\n                        }\n                        _ => {\n                            if let Some(filter) = filter {\n                                if !has_multiple_filters {\n                                    self.0.push(Filter::And);\n                                    has_multiple_filters = true;\n                                }\n                                self.0.push(Filter::Property(filter));\n                            }\n                            let mut new_filter = T::default();\n                            new_filter.deserialize_argument(&key, &mut map)?;\n                            filter = Some(new_filter);\n                        }\n                    }\n                }\n\n                if let Some(filter) = filter {\n                    if has_conditions.is_some() {\n                        return Err(de::Error::custom(\n                            \"Cannot mix conditions with property filters\",\n                        ));\n                    }\n\n                    self.0.push(Filter::Property(filter));\n                    if has_multiple_filters {\n                        self.0.push(Filter::Close);\n                    }\n                }\n\n                Ok(())\n            }\n        }\n\n        deserializer.deserialize_map(FilterVisitor(self.0))\n    }\n}\n\nimpl<'de, 'x, T> DeserializeSeed<'de> for FilterListCollector<'x, T>\nwhere\n    T: for<'de2> DeserializeArguments<'de2> + Default,\n{\n    type Value = ();\n\n    fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        struct FilterVisitor<'x, T: 'x>(&'x mut Vec<Filter<T>>)\n        where\n            T: for<'de2> DeserializeArguments<'de2> + Default;\n\n        impl<'de, 'x, T> Visitor<'de> for FilterVisitor<'x, T>\n        where\n            T: for<'de2> DeserializeArguments<'de2> + Default,\n        {\n            type Value = ();\n\n            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n                write!(formatter, \"a filter list\")\n            }\n\n            fn visit_seq<A>(self, mut seq: A) -> Result<(), A::Error>\n            where\n                A: SeqAccess<'de>,\n            {\n                while let Some(()) = seq.next_element_seed(FilterMapCollector(self.0))? {}\n                Ok(())\n            }\n        }\n\n        deserializer.deserialize_seq(FilterVisitor(self.0))\n    }\n}\n\nimpl<'de, T> DeserializeArguments<'de> for Comparator<T>\nwhere\n    T: for<'de2> DeserializeArguments<'de2> + Default,\n{\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"isAscending\" => {\n                self.is_ascending = map.next_value()?;\n            },\n            b\"collation\" => {\n                self.collation = map.next_value()?;\n            },\n            _ => {\n                self.property.deserialize_argument(key, map)?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de, T> Deserialize<'de> for Comparator<T>\nwhere\n    T: for<'de2> DeserializeArguments<'de2> + Default,\n{\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<T> Comparator<T>\nwhere\n    T: for<'de> DeserializeArguments<'de> + Default,\n{\n    pub fn descending(property: T) -> Self {\n        Self {\n            property,\n            is_ascending: false,\n            collation: None,\n        }\n    }\n\n    pub fn ascending(property: T) -> Self {\n        Self {\n            property,\n            is_ascending: true,\n            collation: None,\n        }\n    }\n}\n\nimpl<T> Default for Comparator<T>\nwhere\n    T: for<'de> DeserializeArguments<'de> + Default,\n{\n    fn default() -> Self {\n        Self {\n            is_ascending: true,\n            collation: None,\n            property: T::default(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/query_changes.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    method::query::{Comparator, Filter, FilterWrapper, QueryRequest},\n    object::JmapObject,\n    request::deserialize::{DeserializeArguments, deserialize_request},\n    types::state::State,\n};\nuse serde::{Deserialize, Deserializer};\nuse types::id::Id;\n\n#[derive(Debug, Clone)]\npub struct QueryChangesRequest<T: JmapObject> {\n    pub account_id: Id,\n    pub filter: Vec<Filter<T::Filter>>,\n    pub sort: Option<Vec<Comparator<T::Comparator>>>,\n    pub since_query_state: State,\n    pub max_changes: Option<usize>,\n    pub up_to_id: Option<Id>,\n    pub calculate_total: Option<bool>,\n    pub arguments: T::QueryArguments,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct QueryChangesResponse {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"oldQueryState\")]\n    pub old_query_state: State,\n\n    #[serde(rename = \"newQueryState\")]\n    pub new_query_state: State,\n\n    #[serde(rename = \"total\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub total: Option<usize>,\n\n    #[serde(rename = \"removed\")]\n    pub removed: Vec<Id>,\n\n    #[serde(rename = \"added\")]\n    pub added: Vec<AddedItem>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct AddedItem {\n    pub id: Id,\n    pub index: usize,\n}\n\nimpl AddedItem {\n    pub fn new(id: Id, index: usize) -> Self {\n        Self { id, index }\n    }\n}\n\nimpl<'de, T: JmapObject> DeserializeArguments<'de> for QueryChangesRequest<T> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"filter\" => {\n                self.filter = map.next_value::<FilterWrapper<T::Filter>>()?.0;\n            },\n            b\"sort\" => {\n                self.sort = map.next_value()?;\n            },\n            b\"sinceQueryState\" => {\n                self.since_query_state = map.next_value()?;\n            },\n            b\"maxChanges\" => {\n                self.max_changes = map.next_value()?;\n            },\n            b\"upToId\" => {\n                self.up_to_id = map.next_value()?;\n            },\n            b\"calculateTotal\" => {\n                self.calculate_total = map.next_value()?;\n            },\n            _ => {\n                self.arguments.deserialize_argument(key, map)?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de, T: JmapObject> Deserialize<'de> for QueryChangesRequest<T> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<T: JmapObject> Default for QueryChangesRequest<T> {\n    fn default() -> Self {\n        Self {\n            account_id: Id::default(),\n            filter: Vec::new(),\n            sort: None,\n            since_query_state: State::default(),\n            max_changes: None,\n            up_to_id: None,\n            calculate_total: None,\n            arguments: T::QueryArguments::default(),\n        }\n    }\n}\n\nimpl<T: JmapObject> From<QueryChangesRequest<T>> for QueryRequest<T> {\n    fn from(request: QueryChangesRequest<T>) -> Self {\n        QueryRequest {\n            account_id: request.account_id,\n            filter: request.filter,\n            sort: request.sort,\n            position: None,\n            anchor: None,\n            anchor_offset: None,\n            limit: None,\n            calculate_total: request.calculate_total,\n            arguments: request.arguments,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/search_snippet.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::query::Filter;\nuse crate::{\n    method::query::FilterWrapper,\n    object::email::EmailFilter,\n    request::{\n        MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n        reference::{MaybeResultReference, ResultReference},\n    },\n};\nuse serde::{Deserialize, Deserializer};\nuse types::id::Id;\n\n#[derive(Debug, Clone)]\npub struct GetSearchSnippetRequest {\n    pub account_id: Id,\n    pub filter: Vec<Filter<EmailFilter>>,\n    pub email_ids: MaybeResultReference<Vec<MaybeInvalid<Id>>>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct GetSearchSnippetResponse {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"list\")]\n    pub list: Vec<SearchSnippet>,\n\n    #[serde(rename = \"notFound\")]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub not_found: Vec<Id>,\n}\n\n#[derive(serde::Serialize, Clone, Debug)]\npub struct SearchSnippet {\n    #[serde(rename = \"emailId\")]\n    pub email_id: Id,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub subject: Option<String>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub preview: Option<String>,\n}\n\nimpl<'de> DeserializeArguments<'de> for GetSearchSnippetRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"filter\" => {\n                self.filter = map.next_value::<FilterWrapper<EmailFilter>>()?.0;\n            },\n            b\"emailIds\" => {\n                self.email_ids = MaybeResultReference::Value(map.next_value::<Vec<MaybeInvalid<Id>>>()?);\n            },\n            b\"#emailIds\" => {\n                self.email_ids = MaybeResultReference::Reference(map.next_value::<ResultReference>()?);\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> Deserialize<'de> for GetSearchSnippetRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl Default for GetSearchSnippetRequest {\n    fn default() -> Self {\n        Self {\n            account_id: Id::default(),\n            filter: Vec::new(),\n            email_ids: MaybeResultReference::Value(Vec::new()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ahash_is_empty;\nuse crate::{\n    error::set::{InvalidProperty, SetError},\n    object::{JmapObject, JmapObjectId},\n    request::{\n        MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n        reference::{MaybeResultReference, ResultReference},\n    },\n    response::Response,\n    types::state::State,\n};\nuse ahash::AHashMap;\nuse jmap_tools::{Key, Map, Value};\nuse serde::{Deserialize, Deserializer};\nuse types::id::Id;\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone)]\n#[allow(clippy::type_complexity)]\npub struct SetRequest<'x, T: JmapObject> {\n    pub account_id: Id,\n    pub if_in_state: Option<State>,\n    pub create: Option<VecMap<String, Value<'x, T::Property, T::Element>>>,\n    pub update: Option<VecMap<MaybeInvalid<Id>, Value<'x, T::Property, T::Element>>>,\n    pub destroy: Option<MaybeResultReference<Vec<MaybeInvalid<Id>>>>,\n    pub arguments: T::SetArguments<'x>,\n}\n\n#[derive(Debug, Clone, Default, serde::Serialize)]\n#[allow(clippy::type_complexity)]\npub struct SetResponse<T: JmapObject> {\n    #[serde(rename = \"accountId\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub account_id: Option<Id>,\n\n    #[serde(rename = \"oldState\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub old_state: Option<State>,\n\n    #[serde(rename = \"newState\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub new_state: Option<State>,\n\n    #[serde(rename = \"created\")]\n    #[serde(skip_serializing_if = \"ahash_is_empty\")]\n    pub created: AHashMap<String, Value<'static, T::Property, T::Element>>,\n\n    #[serde(rename = \"updated\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub updated: VecMap<Id, Option<Value<'static, T::Property, T::Element>>>,\n\n    #[serde(rename = \"destroyed\")]\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    pub destroyed: Vec<Id>,\n\n    #[serde(rename = \"notCreated\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub not_created: VecMap<String, SetError<T::Property>>,\n\n    #[serde(rename = \"notUpdated\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub not_updated: VecMap<Id, SetError<T::Property>>,\n\n    #[serde(rename = \"notDestroyed\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub not_destroyed: VecMap<Id, SetError<T::Property>>,\n}\n\nimpl<'de, T: JmapObject> DeserializeArguments<'de> for SetRequest<'de, T> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"ifInState\" => {\n                self.if_in_state = map.next_value()?;\n            },\n            b\"create\" => {\n                self.create = map.next_value()?;\n            },\n            b\"update\" => {\n                self.update = map.next_value()?;\n            },\n            b\"destroy\" => {\n                self.destroy = map.next_value::<Option<Vec<MaybeInvalid<Id>>>>()?.map(MaybeResultReference::Value);\n            },\n            b\"#destroy\" => {\n                self.destroy = Some(MaybeResultReference::Reference(map.next_value::<ResultReference>()?));\n            }\n            _ => {\n                self.arguments.deserialize_argument(key, map)?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de, T: JmapObject> Deserialize<'de> for SetRequest<'de, T> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<'x, T: JmapObject> Default for SetRequest<'x, T> {\n    fn default() -> Self {\n        Self {\n            account_id: Id::default(),\n            if_in_state: None,\n            create: None,\n            update: None,\n            destroy: None,\n            arguments: T::SetArguments::default(),\n        }\n    }\n}\n\nimpl<'x, T: JmapObject> SetRequest<'x, T> {\n    pub fn validate(&self, max_objects_in_set: usize) -> trc::Result<()> {\n        if self.create.as_ref().map_or(0, |objs| objs.len())\n            + self.update.as_ref().map_or(0, |objs| objs.len())\n            + self.destroy.as_ref().map_or(0, |objs| {\n                if let MaybeResultReference::Value(ids) = objs {\n                    ids.len()\n                } else {\n                    0\n                }\n            })\n            > max_objects_in_set\n        {\n            Err(trc::JmapEvent::RequestTooLarge.into_err())\n        } else {\n            Ok(())\n        }\n    }\n\n    pub fn has_updates(&self) -> bool {\n        self.update.as_ref().is_some_and(|objs| !objs.is_empty())\n    }\n\n    pub fn has_creates(&self) -> bool {\n        self.create.as_ref().is_some_and(|objs| !objs.is_empty())\n    }\n\n    pub fn unwrap_create(&mut self) -> VecMap<String, Value<'x, T::Property, T::Element>> {\n        self.create.take().unwrap_or_default()\n    }\n\n    pub fn unwrap_update(\n        &mut self,\n    ) -> VecMap<MaybeInvalid<Id>, Value<'x, T::Property, T::Element>> {\n        self.update.take().unwrap_or_default()\n    }\n\n    pub fn unwrap_destroy(&mut self) -> Vec<MaybeInvalid<Id>> {\n        self.destroy\n            .take()\n            .map(|ids| ids.unwrap())\n            .unwrap_or_default()\n    }\n}\n\nimpl<T: JmapObject> SetResponse<T> {\n    pub fn from_request(request: &SetRequest<T>, max_objects: usize) -> trc::Result<Self> {\n        let n_create = request.create.as_ref().map_or(0, |objs| objs.len());\n        let n_update = request.update.as_ref().map_or(0, |objs| objs.len());\n        let n_destroy = request.destroy.as_ref().map_or(0, |objs| {\n            if let MaybeResultReference::Value(ids) = objs {\n                ids.len()\n            } else {\n                0\n            }\n        });\n        if n_create + n_update + n_destroy <= max_objects {\n            Ok(SetResponse {\n                account_id: if request.account_id.is_valid() {\n                    request.account_id.into()\n                } else {\n                    None\n                },\n                new_state: None,\n                old_state: None,\n                created: AHashMap::with_capacity(n_create),\n                updated: VecMap::with_capacity(n_update),\n                destroyed: Vec::with_capacity(n_destroy),\n                not_created: VecMap::new(),\n                not_updated: VecMap::new(),\n                not_destroyed: VecMap::new(),\n            })\n        } else {\n            Err(trc::JmapEvent::RequestTooLarge.into_err())\n        }\n    }\n\n    pub fn with_state(mut self, state: State) -> Self {\n        self.old_state = Some(state.clone());\n        self.new_state = Some(state);\n        self\n    }\n\n    pub fn created(&mut self, id: String, document_id: impl Into<T::Id>) {\n        self.created.insert(\n            id,\n            Value::Object(Map::from(vec![(\n                Key::Property(T::ID_PROPERTY),\n                Value::Element(document_id.into().into()),\n            )])),\n        );\n    }\n\n    pub fn invalid_property_create(\n        &mut self,\n        id: String,\n        property: impl Into<InvalidProperty<T::Property>>,\n    ) {\n        self.not_created.append(\n            id,\n            SetError::invalid_properties()\n                .with_property(property)\n                .with_description(\"Invalid property or value.\".to_string()),\n        );\n    }\n\n    pub fn invalid_property_update(\n        &mut self,\n        id: Id,\n        property: impl Into<InvalidProperty<T::Property>>,\n    ) {\n        self.not_updated.append(\n            id,\n            SetError::invalid_properties()\n                .with_property(property)\n                .with_description(\"Invalid property or value.\".to_string()),\n        );\n    }\n\n    pub fn update_created_ids(&self, response: &mut Response) {\n        for (user_id, obj) in &self.created {\n            if let Value::Object(obj) = obj\n                && let Some(Value::Element(id)) = obj.get(&Key::Property(T::ID_PROPERTY))\n                && let Some(id) = id.as_any_id()\n            {\n                response.created_ids.insert(user_id.clone(), id);\n            }\n        }\n    }\n\n    pub fn get_object_by_id(\n        &mut self,\n        id: Id,\n    ) -> Option<&mut Value<'static, T::Property, T::Element>> {\n        if let Some(obj) = self.updated.get_mut(&id) {\n            if let Some(obj) = obj {\n                return Some(obj);\n            } else {\n                *obj = Some(Value::Object(Map::with_capacity(1)));\n                return obj.as_mut().unwrap().into();\n            }\n        }\n\n        (&mut self.created)\n            .into_iter()\n            .map(|(_, obj)| obj)\n            .find(|obj| {\n                obj.as_object_and_get(&Key::Property(T::ID_PROPERTY))\n                    .and_then(|v| v.as_element())\n                    .and_then(|v| v.as_id())\n                    .is_some_and(|oid| oid == id)\n            })\n    }\n\n    pub fn has_changes(&self) -> bool {\n        !self.created.is_empty() || !self.updated.is_empty() || !self.destroyed.is_empty()\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/upload.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse super::ahash_is_empty;\nuse crate::{\n    error::set::SetError,\n    object::{AnyId, blob::BlobProperty},\n    request::{\n        deserialize::{DeserializeArguments, deserialize_request},\n        reference::MaybeIdReference,\n    },\n    response::Response,\n};\nuse ahash::AHashMap;\nuse mail_parser::decoders::base64::base64_decode;\nuse serde::{Deserialize, Deserializer};\nuse types::{blob::BlobId, id::Id};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone, Default)]\npub struct BlobUploadRequest {\n    pub account_id: Id,\n    pub create: VecMap<String, UploadObject>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct UploadObject {\n    pub type_: Option<String>,\n    pub data: Vec<DataSourceObject>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\npub enum DataSourceObject {\n    Id {\n        id: MaybeIdReference<BlobId>,\n        length: Option<usize>,\n        offset: Option<usize>,\n    },\n    Value(Vec<u8>),\n    #[default]\n    Null,\n}\n\n#[derive(Debug, Clone, Default, serde::Serialize)]\npub struct BlobUploadResponse {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n\n    #[serde(rename = \"created\")]\n    #[serde(skip_serializing_if = \"ahash_is_empty\")]\n    pub created: AHashMap<String, BlobUploadResponseObject>,\n\n    #[serde(rename = \"notCreated\")]\n    #[serde(skip_serializing_if = \"VecMap::is_empty\")]\n    pub not_created: VecMap<String, SetError<BlobProperty>>,\n}\n\n#[derive(Debug, Clone, Default, serde::Serialize)]\npub struct BlobUploadResponseObject {\n    pub id: BlobId,\n    #[serde(rename = \"type\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub type_: Option<String>,\n    pub size: usize,\n}\n\nimpl<'de> DeserializeArguments<'de> for BlobUploadRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"create\" => {\n                self.create = map.next_value()?;\n            }\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for UploadObject {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"type\" => {\n                self.type_ = map.next_value()?;\n            },\n            b\"data\" => {\n                self.data = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for DataSourceObject {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"data:asText\" => {\n                *self = DataSourceObject::Value(map.next_value::<String>().map(|v| v.into_bytes())?);\n            },\n            b\"data:asBase64\" => {\n                *self = DataSourceObject::Value(base64_decode(map.next_value::<Cow<'_, str>>()?.as_bytes()).ok_or_else(|| serde::de::Error::custom(\"Failed to decode base64 data\"))?);\n            },\n            b\"blobId\" => {\n                match self {\n                    DataSourceObject::Id { id, .. } => {\n                        *id = map.next_value()?;\n                    },\n                    _ => {\n                        *self = DataSourceObject::Id {\n                            id: map.next_value()?,\n                            length: None,\n                            offset: None,\n                        };\n                    }\n                }\n            },\n            b\"offset\" => {\n                match self {\n                    DataSourceObject::Id { offset, .. } => {\n                        *offset = map.next_value()?;\n                    },\n                    _ => {\n                        *self = DataSourceObject::Id {\n                            id: MaybeIdReference::Invalid(\"\".into()),\n                            length: None,\n                            offset: map.next_value()?,\n                        };\n                    }\n                }\n            },\n            b\"length\" => {\n                match self {\n                    DataSourceObject::Id { length, .. } => {\n                        *length = map.next_value()?;\n                    },\n                    _ => {\n                        *self = DataSourceObject::Id {\n                            id: MaybeIdReference::Invalid(\"\".into()),\n                            length: map.next_value()?,\n                            offset: None,\n                        };\n                    }\n                }\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl BlobUploadResponse {\n    pub fn update_created_ids(&self, response: &mut Response) {\n        for (user_id, obj) in &self.created {\n            response\n                .created_ids\n                .insert(user_id.clone(), AnyId::BlobId(obj.id.clone()));\n        }\n    }\n}\n\nimpl<'de> Deserialize<'de> for DataSourceObject {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<'de> Deserialize<'de> for UploadObject {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<'de> Deserialize<'de> for BlobUploadRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/method/validate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    error::set::SetError,\n    object::sieve::SieveProperty,\n    request::{\n        MaybeInvalid,\n        deserialize::{DeserializeArguments, deserialize_request},\n    },\n};\nuse serde::{Deserialize, Deserializer, Serialize};\nuse types::{blob::BlobId, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct ValidateSieveScriptRequest {\n    pub account_id: Id,\n    pub blob_id: MaybeInvalid<BlobId>,\n}\n\n#[derive(Debug, Serialize)]\npub struct ValidateSieveScriptResponse {\n    #[serde(rename = \"accountId\")]\n    pub account_id: Id,\n    pub error: Option<SetError<SieveProperty>>,\n}\n\nimpl<'de> DeserializeArguments<'de> for ValidateSieveScriptRequest {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountId\" => {\n                self.account_id = map.next_value()?;\n            },\n            b\"blobId\" => {\n                self.blob_id = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> Deserialize<'de> for ValidateSieveScriptRequest {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/addressbook.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{\n        AnyId, JmapObject, JmapObjectId, JmapRight, JmapSharedObject, MaybeReference, parse_ref,\n    },\n    request::{deserialize::DeserializeArguments, reference::MaybeIdReference},\n};\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::{acl::Acl, id::Id, special_use::SpecialUse};\n\n#[derive(Debug, Clone, Default)]\npub struct AddressBook;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum AddressBookProperty {\n    Id,\n    Name,\n    Description,\n    SortOrder,\n    IsDefault,\n    IsSubscribed,\n    ShareWith,\n    MyRights,\n\n    // Other\n    IdValue(Id),\n    Rights(AddressBookRight),\n    Pointer(JsonPointer<AddressBookProperty>),\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum AddressBookRight {\n    MayRead,\n    MayWrite,\n    MayShare,\n    MayDelete,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum AddressBookValue {\n    Id(Id),\n    IdReference(String),\n    Role(SpecialUse),\n}\n\nimpl Property for AddressBookProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        let allow_patch = key.is_none();\n        if let Some(Key::Property(key)) = key {\n            match key.patch_or_prop() {\n                AddressBookProperty::ShareWith => {\n                    Id::from_str(value).ok().map(AddressBookProperty::IdValue)\n                }\n                _ => AddressBookProperty::parse(value, allow_patch),\n            }\n        } else {\n            AddressBookProperty::parse(value, allow_patch)\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            AddressBookProperty::Id => \"id\",\n            AddressBookProperty::Name => \"name\",\n            AddressBookProperty::Description => \"description\",\n            AddressBookProperty::SortOrder => \"sortOrder\",\n            AddressBookProperty::IsDefault => \"isDefault\",\n            AddressBookProperty::IsSubscribed => \"isSubscribed\",\n            AddressBookProperty::ShareWith => \"shareWith\",\n            AddressBookProperty::MyRights => \"myRights\",\n            AddressBookProperty::Rights(addressbook_right) => addressbook_right.as_str(),\n            AddressBookProperty::Pointer(json_pointer) => return json_pointer.to_string().into(),\n            AddressBookProperty::IdValue(id) => return id.to_string().into(),\n        }\n        .into()\n    }\n}\n\nimpl AddressBookRight {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            AddressBookRight::MayRead => \"mayRead\",\n            AddressBookRight::MayWrite => \"mayWrite\",\n            AddressBookRight::MayShare => \"mayShare\",\n            AddressBookRight::MayDelete => \"mayDelete\",\n        }\n    }\n}\n\nimpl Element for AddressBookValue {\n    type Property = AddressBookProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                AddressBookProperty::Id => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(AddressBookValue::Id(v)),\n                    MaybeReference::Reference(v) => Some(AddressBookValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            AddressBookValue::Id(id) => id.to_string().into(),\n            AddressBookValue::IdReference(r) => format!(\"#{r}\").into(),\n            AddressBookValue::Role(special_use) => special_use.as_str().unwrap_or_default().into(),\n        }\n    }\n}\n\nimpl AddressBookProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => AddressBookProperty::Id,\n            b\"name\" => AddressBookProperty::Name,\n            b\"description\" => AddressBookProperty::Description,\n            b\"sortOrder\" => AddressBookProperty::SortOrder,\n            b\"isDefault\" => AddressBookProperty::IsDefault,\n            b\"isSubscribed\" => AddressBookProperty::IsSubscribed,\n            b\"shareWith\" => AddressBookProperty::ShareWith,\n            b\"myRights\" => AddressBookProperty::MyRights,\n            b\"mayRead\" => AddressBookProperty::Rights(AddressBookRight::MayRead),\n            b\"mayWrite\" => AddressBookProperty::Rights(AddressBookRight::MayWrite),\n            b\"mayShare\" => AddressBookProperty::Rights(AddressBookRight::MayShare),\n            b\"mayDelete\" => AddressBookProperty::Rights(AddressBookRight::MayDelete)\n        )\n        .or_else(|| {\n            if allow_patch && value.contains('/') {\n                AddressBookProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &AddressBookProperty {\n        if let AddressBookProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct AddressBookSetArguments {\n    pub on_destroy_remove_contents: Option<bool>,\n    pub on_success_set_is_default: Option<MaybeIdReference<Id>>,\n}\n\nimpl<'de> DeserializeArguments<'de> for AddressBookSetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"onDestroyRemoveContents\" => {\n                self.on_destroy_remove_contents = map.next_value()?;\n            },\n            b\"onSuccessSetIsDefault\" => {\n                self.on_success_set_is_default = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl FromStr for AddressBookProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        AddressBookProperty::parse(s, false).ok_or(())\n    }\n}\n\nimpl JmapObject for AddressBook {\n    type Property = AddressBookProperty;\n\n    type Element = AddressBookValue;\n\n    type Id = Id;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = AddressBookSetArguments;\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = AddressBookProperty::Id;\n}\n\nimpl JmapSharedObject for AddressBook {\n    type Right = AddressBookRight;\n\n    const SHARE_WITH_PROPERTY: Self::Property = AddressBookProperty::ShareWith;\n}\n\nimpl From<Id> for AddressBookProperty {\n    fn from(id: Id) -> Self {\n        AddressBookProperty::IdValue(id)\n    }\n}\n\nimpl TryFrom<AddressBookProperty> for Id {\n    type Error = ();\n\n    fn try_from(value: AddressBookProperty) -> Result<Self, Self::Error> {\n        if let AddressBookProperty::IdValue(id) = value {\n            Ok(id)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl TryFrom<AddressBookProperty> for AddressBookRight {\n    type Error = ();\n\n    fn try_from(value: AddressBookProperty) -> Result<Self, Self::Error> {\n        if let AddressBookProperty::Rights(right) = value {\n            Ok(right)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl From<Id> for AddressBookValue {\n    fn from(id: Id) -> Self {\n        AddressBookValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for AddressBookValue {\n    fn as_id(&self) -> Option<Id> {\n        if let AddressBookValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let AddressBookValue::Id(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let AddressBookValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(new_id) = new_id {\n            *self = AddressBookValue::Id(new_id);\n            return true;\n        }\n        false\n    }\n}\n\nimpl JmapRight for AddressBookRight {\n    fn to_acl(&self) -> &'static [Acl] {\n        match self {\n            AddressBookRight::MayDelete => &[Acl::Delete, Acl::RemoveItems],\n            AddressBookRight::MayShare => &[Acl::Share],\n            AddressBookRight::MayRead => &[Acl::Read, Acl::ReadItems],\n            AddressBookRight::MayWrite => &[Acl::Modify, Acl::AddItems, Acl::ModifyItems],\n        }\n    }\n\n    fn all_rights() -> &'static [Self] {\n        &[\n            AddressBookRight::MayRead,\n            AddressBookRight::MayWrite,\n            AddressBookRight::MayDelete,\n            AddressBookRight::MayShare,\n        ]\n    }\n}\n\nimpl From<AddressBookRight> for AddressBookProperty {\n    fn from(right: AddressBookRight) -> Self {\n        AddressBookProperty::Rights(right)\n    }\n}\n\nimpl JmapObjectId for AddressBookProperty {\n    fn as_id(&self) -> Option<Id> {\n        if let AddressBookProperty::IdValue(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let AddressBookProperty::IdValue(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(new_id) = new_id {\n            *self = AddressBookProperty::IdValue(new_id);\n            return true;\n        }\n        false\n    }\n}\n\nimpl std::fmt::Display for AddressBookProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"{}\", self.to_cow())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId, MaybeReference, parse_ref},\n    request::deserialize::DeserializeArguments,\n};\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::{blob::BlobId, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct Blob;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum BlobProperty {\n    Id,\n    BlobId,\n    Type,\n    Size,\n    Digest(DigestProperty),\n    Data(DataProperty),\n    IsEncodingProblem,\n    IsTruncated,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum DigestProperty {\n    Sha,\n    Sha256,\n    Sha512,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum DataProperty {\n    AsText,\n    AsBase64,\n    Default,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum BlobValue {\n    BlobId(BlobId),\n    IdReference(String),\n}\n\nimpl Property for BlobProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        BlobProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            BlobProperty::BlobId => \"blobId\",\n            BlobProperty::Id => \"id\",\n            BlobProperty::Size => \"size\",\n            BlobProperty::Type => \"type\",\n            BlobProperty::IsEncodingProblem => \"isEncodingProblem\",\n            BlobProperty::IsTruncated => \"isTruncated\",\n            BlobProperty::Data(data) => match data {\n                DataProperty::AsText => \"data:asText\",\n                DataProperty::AsBase64 => \"data:asBase64\",\n                DataProperty::Default => \"data\",\n            },\n            BlobProperty::Digest(digest) => match digest {\n                DigestProperty::Sha => \"digest:sha\",\n                DigestProperty::Sha256 => \"digest:sha-256\",\n                DigestProperty::Sha512 => \"digest:sha-512\",\n            },\n        }\n        .into()\n    }\n}\n\nimpl Element for BlobValue {\n    type Property = BlobProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                BlobProperty::BlobId => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(BlobValue::BlobId(v)),\n                    MaybeReference::Reference(v) => Some(BlobValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            BlobValue::BlobId(blob_id) => blob_id.to_string().into(),\n            BlobValue::IdReference(r) => format!(\"#{r}\").into(),\n        }\n    }\n}\n\nimpl BlobProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"blobId\" => BlobProperty::BlobId,\n            b\"id\" => BlobProperty::Id,\n            b\"size\" => BlobProperty::Size,\n            b\"type\" => BlobProperty::Type,\n            b\"isEncodingProblem\" => BlobProperty::IsEncodingProblem,\n            b\"isTruncated\" => BlobProperty::IsTruncated,\n            b\"data:asText\" => BlobProperty::Data(DataProperty::AsText),\n            b\"data:asBase64\" => BlobProperty::Data(DataProperty::AsBase64),\n            b\"data\" => BlobProperty::Data(DataProperty::Default),\n            b\"digest:sha\" => BlobProperty::Digest(DigestProperty::Sha),\n            b\"digest:sha-256\" => BlobProperty::Digest(DigestProperty::Sha256),\n            b\"digest:sha-512\" => BlobProperty::Digest(DigestProperty::Sha512),\n        )\n    }\n}\n\nimpl FromStr for BlobProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        BlobProperty::parse(s).ok_or(())\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct BlobGetArguments {\n    pub offset: Option<usize>,\n    pub length: Option<usize>,\n}\n\nimpl<'de> DeserializeArguments<'de> for BlobGetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n             b\"offset\" => {\n                self.offset = map.next_value()?;\n            },\n            b\"length\" => {\n                self.length = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl JmapObject for Blob {\n    type Property = BlobProperty;\n\n    type Element = BlobValue;\n\n    type Id = BlobId;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = BlobGetArguments;\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = BlobProperty::Id;\n}\n\nimpl From<BlobId> for BlobValue {\n    fn from(id: BlobId) -> Self {\n        BlobValue::BlobId(id)\n    }\n}\n\nimpl JmapObjectId for BlobValue {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            BlobValue::BlobId(id) => Some(AnyId::BlobId(id.clone())),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let BlobValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::BlobId(id) = new_id {\n            *self = BlobValue::BlobId(id);\n            return true;\n        }\n        false\n    }\n}\n\nimpl JmapObjectId for BlobProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/calendar.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{\n        AnyId, JmapObject, JmapObjectId, JmapRight, JmapSharedObject, MaybeReference, parse_ref,\n    },\n    request::{deserialize::DeserializeArguments, reference::MaybeIdReference},\n    types::date::UTCDate,\n};\nuse calcard::{\n    common::{IanaParse, timezone::Tz},\n    icalendar::ICalendarDuration,\n    jscalendar::{JSCalendarAlertAction, JSCalendarRelativeTo, JSCalendarType},\n};\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem, Key, Property};\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\nuse types::{acl::Acl, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct Calendar;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum CalendarProperty {\n    Id,\n    Name,\n    Description,\n    Color,\n    SortOrder,\n    IsSubscribed,\n    IsVisible,\n    IsDefault,\n    IncludeInAvailability,\n    DefaultAlertsWithTime,\n    DefaultAlertsWithoutTime,\n    TimeZone,\n    ShareWith,\n    MyRights,\n\n    // Alert object properties\n    When,\n    Trigger,\n    Offset,\n    RelativeTo,\n    Action,\n    Type,\n\n    // Other\n    IdValue(Id),\n    Rights(CalendarRight),\n    Pointer(JsonPointer<CalendarProperty>),\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum CalendarRight {\n    MayReadFreeBusy,\n    MayReadItems,\n    MayWriteAll,\n    MayWriteOwn,\n    MayUpdatePrivate,\n    MayRSVP,\n    MayShare,\n    MayDelete,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum CalendarValue {\n    Id(Id),\n    IdReference(String),\n    IncludeInAvailability(IncludeInAvailability),\n    Date(UTCDate),\n    Timezone(Tz),\n    Action(JSCalendarAlertAction),\n    RelativeTo(JSCalendarRelativeTo),\n    Type(JSCalendarType),\n    Duration(ICalendarDuration),\n}\n\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum IncludeInAvailability {\n    #[default]\n    All,\n    Attending,\n    None,\n}\n\nimpl Property for CalendarProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        let allow_patch = key.is_none();\n        if let Some(Key::Property(key)) = key {\n            match key.patch_or_prop() {\n                CalendarProperty::ShareWith => {\n                    Id::from_str(value).ok().map(CalendarProperty::IdValue)\n                }\n                _ => CalendarProperty::parse(value, allow_patch),\n            }\n        } else {\n            CalendarProperty::parse(value, allow_patch)\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            CalendarProperty::Id => \"id\",\n            CalendarProperty::Name => \"name\",\n            CalendarProperty::Description => \"description\",\n            CalendarProperty::Color => \"color\",\n            CalendarProperty::SortOrder => \"sortOrder\",\n            CalendarProperty::IsSubscribed => \"isSubscribed\",\n            CalendarProperty::IsVisible => \"isVisible\",\n            CalendarProperty::IsDefault => \"isDefault\",\n            CalendarProperty::IncludeInAvailability => \"includeInAvailability\",\n            CalendarProperty::DefaultAlertsWithTime => \"defaultAlertsWithTime\",\n            CalendarProperty::DefaultAlertsWithoutTime => \"defaultAlertsWithoutTime\",\n            CalendarProperty::TimeZone => \"timeZone\",\n            CalendarProperty::ShareWith => \"shareWith\",\n            CalendarProperty::MyRights => \"myRights\",\n            CalendarProperty::When => \"when\",\n            CalendarProperty::Trigger => \"trigger\",\n            CalendarProperty::Offset => \"offset\",\n            CalendarProperty::RelativeTo => \"relativeTo\",\n            CalendarProperty::Action => \"action\",\n            CalendarProperty::Type => \"@type\",\n            CalendarProperty::Rights(calendar_right) => calendar_right.as_str(),\n            CalendarProperty::Pointer(json_pointer) => return json_pointer.to_string().into(),\n            CalendarProperty::IdValue(id) => return id.to_string().into(),\n        }\n        .into()\n    }\n}\n\nimpl CalendarRight {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            CalendarRight::MayReadFreeBusy => \"mayReadFreeBusy\",\n            CalendarRight::MayReadItems => \"mayReadItems\",\n            CalendarRight::MayWriteAll => \"mayWriteAll\",\n            CalendarRight::MayWriteOwn => \"mayWriteOwn\",\n            CalendarRight::MayUpdatePrivate => \"mayUpdatePrivate\",\n            CalendarRight::MayRSVP => \"mayRSVP\",\n            CalendarRight::MayShare => \"mayShare\",\n            CalendarRight::MayDelete => \"mayDelete\",\n        }\n    }\n}\n\nimpl IncludeInAvailability {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"all\" => IncludeInAvailability::All,\n            b\"attending\" => IncludeInAvailability::Attending,\n            b\"none\" => IncludeInAvailability::None,\n        )\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            IncludeInAvailability::All => \"all\",\n            IncludeInAvailability::Attending => \"attending\",\n            IncludeInAvailability::None => \"none\",\n        }\n    }\n}\n\nimpl Element for CalendarValue {\n    type Property = CalendarProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                CalendarProperty::Id => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(CalendarValue::Id(v)),\n                    MaybeReference::Reference(v) => Some(CalendarValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                CalendarProperty::TimeZone => Tz::from_str(value).ok().map(CalendarValue::Timezone),\n                CalendarProperty::IncludeInAvailability => {\n                    IncludeInAvailability::parse(value).map(CalendarValue::IncludeInAvailability)\n                }\n                CalendarProperty::Action => JSCalendarAlertAction::from_str(value)\n                    .ok()\n                    .map(CalendarValue::Action),\n                CalendarProperty::RelativeTo => JSCalendarRelativeTo::from_str(value)\n                    .ok()\n                    .map(CalendarValue::RelativeTo),\n                CalendarProperty::When => UTCDate::from_str(value).ok().map(CalendarValue::Date),\n                CalendarProperty::Offset => {\n                    ICalendarDuration::parse(value.as_bytes()).map(CalendarValue::Duration)\n                }\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            CalendarValue::Id(id) => id.to_string().into(),\n            CalendarValue::IdReference(r) => format!(\"#{r}\").into(),\n            CalendarValue::IncludeInAvailability(include) => include.as_str().into(),\n            CalendarValue::Date(date) => date.to_string().into(),\n            CalendarValue::Action(action) => action.as_str().into(),\n            CalendarValue::RelativeTo(relative) => relative.as_str().into(),\n            CalendarValue::Type(typ) => typ.as_str().into(),\n            CalendarValue::Duration(dur) => dur.to_string().into(),\n            CalendarValue::Timezone(tz) => tz.name().unwrap_or_default(),\n        }\n    }\n}\n\nimpl CalendarProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => CalendarProperty::Id,\n            b\"name\" => CalendarProperty::Name,\n            b\"description\" => CalendarProperty::Description,\n            b\"color\" => CalendarProperty::Color,\n            b\"sortOrder\" => CalendarProperty::SortOrder,\n            b\"isSubscribed\" => CalendarProperty::IsSubscribed,\n            b\"isVisible\" => CalendarProperty::IsVisible,\n            b\"isDefault\" => CalendarProperty::IsDefault,\n            b\"includeInAvailability\" => CalendarProperty::IncludeInAvailability,\n            b\"defaultAlertsWithTime\" => CalendarProperty::DefaultAlertsWithTime,\n            b\"defaultAlertsWithoutTime\" => CalendarProperty::DefaultAlertsWithoutTime,\n            b\"timeZone\" => CalendarProperty::TimeZone,\n            b\"shareWith\" => CalendarProperty::ShareWith,\n            b\"myRights\" => CalendarProperty::MyRights,\n            b\"mayReadFreeBusy\" => CalendarProperty::Rights(CalendarRight::MayReadFreeBusy),\n            b\"mayReadItems\" => CalendarProperty::Rights(CalendarRight::MayReadItems),\n            b\"mayWriteAll\" => CalendarProperty::Rights(CalendarRight::MayWriteAll),\n            b\"mayWriteOwn\" => CalendarProperty::Rights(CalendarRight::MayWriteOwn),\n            b\"mayUpdatePrivate\" => CalendarProperty::Rights(CalendarRight::MayUpdatePrivate),\n            b\"mayRSVP\" => CalendarProperty::Rights(CalendarRight::MayRSVP),\n            b\"mayShare\" => CalendarProperty::Rights(CalendarRight::MayShare),\n            b\"mayDelete\" => CalendarProperty::Rights(CalendarRight::MayDelete),\n            b\"@type\" => CalendarProperty::Type,\n            b\"when\" => CalendarProperty::When,\n            b\"trigger\" => CalendarProperty::Trigger,\n            b\"offset\" => CalendarProperty::Offset,\n            b\"relativeTo\" => CalendarProperty::RelativeTo,\n            b\"action\" => CalendarProperty::Action,\n        )\n        .or_else(|| {\n            if allow_patch && value.contains('/') {\n                CalendarProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &CalendarProperty {\n        if let CalendarProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct CalendarSetArguments {\n    pub on_destroy_remove_events: Option<bool>,\n    pub on_success_set_is_default: Option<MaybeIdReference<Id>>,\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarSetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"onDestroyRemoveEvents\" => {\n                self.on_destroy_remove_events = map.next_value()?;\n            },\n            b\"onSuccessSetIsDefault\" => {\n                self.on_success_set_is_default = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl FromStr for CalendarProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        CalendarProperty::parse(s, false).ok_or(())\n    }\n}\n\nimpl JmapObject for Calendar {\n    type Property = CalendarProperty;\n\n    type Element = CalendarValue;\n\n    type Id = Id;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = CalendarSetArguments;\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = CalendarProperty::Id;\n}\n\nimpl JmapSharedObject for Calendar {\n    type Right = CalendarRight;\n\n    const SHARE_WITH_PROPERTY: Self::Property = CalendarProperty::ShareWith;\n}\n\nimpl From<Id> for CalendarProperty {\n    fn from(id: Id) -> Self {\n        CalendarProperty::IdValue(id)\n    }\n}\n\nimpl TryFrom<CalendarProperty> for Id {\n    type Error = ();\n\n    fn try_from(value: CalendarProperty) -> Result<Self, Self::Error> {\n        if let CalendarProperty::IdValue(id) = value {\n            Ok(id)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl TryFrom<CalendarProperty> for CalendarRight {\n    type Error = ();\n\n    fn try_from(value: CalendarProperty) -> Result<Self, Self::Error> {\n        if let CalendarProperty::Rights(right) = value {\n            Ok(right)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl From<Id> for CalendarValue {\n    fn from(id: Id) -> Self {\n        CalendarValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for CalendarValue {\n    fn as_id(&self) -> Option<Id> {\n        if let CalendarValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let CalendarValue::Id(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let CalendarValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(new_id) = new_id {\n            *self = CalendarValue::Id(new_id);\n            return true;\n        }\n        false\n    }\n}\n\nimpl JmapRight for CalendarRight {\n    fn to_acl(&self) -> &'static [Acl] {\n        match self {\n            CalendarRight::MayReadFreeBusy => &[Acl::SchedulingReadFreeBusy],\n            CalendarRight::MayReadItems => &[Acl::Read, Acl::ReadItems],\n            CalendarRight::MayWriteAll => &[\n                Acl::Modify,\n                Acl::AddItems,\n                Acl::ModifyItems,\n                Acl::RemoveItems,\n            ],\n            CalendarRight::MayWriteOwn => &[Acl::ModifyItemsOwn],\n            CalendarRight::MayUpdatePrivate => &[Acl::ModifyPrivateProperties],\n            CalendarRight::MayRSVP => &[Acl::ModifyRSVP],\n            CalendarRight::MayShare => &[Acl::Share],\n            CalendarRight::MayDelete => &[Acl::Delete, Acl::RemoveItems],\n        }\n    }\n\n    fn all_rights() -> &'static [Self] {\n        &[\n            CalendarRight::MayReadFreeBusy,\n            CalendarRight::MayReadItems,\n            CalendarRight::MayWriteAll,\n            CalendarRight::MayWriteOwn,\n            CalendarRight::MayUpdatePrivate,\n            CalendarRight::MayRSVP,\n            CalendarRight::MayShare,\n            CalendarRight::MayDelete,\n        ]\n    }\n}\n\nimpl From<CalendarRight> for CalendarProperty {\n    fn from(right: CalendarRight) -> Self {\n        CalendarProperty::Rights(right)\n    }\n}\n\nimpl JmapObjectId for CalendarProperty {\n    fn as_id(&self) -> Option<Id> {\n        if let CalendarProperty::IdValue(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let CalendarProperty::IdValue(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(new_id) = new_id {\n            *self = CalendarProperty::IdValue(new_id);\n            return true;\n        }\n        false\n    }\n}\n\nimpl Display for CalendarProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"{}\", self.to_cow())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/calendar_event.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    request::{MaybeInvalid, deserialize::DeserializeArguments},\n};\nuse calcard::{\n    common::timezone::Tz,\n    jscalendar::{JSCalendarDateTime, JSCalendarProperty, JSCalendarValue},\n};\nuse jmap_tools::{JsonPointerItem, Key};\nuse mail_parser::DateTime;\nuse std::{borrow::Cow, str::FromStr};\nuse types::{blob::BlobId, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct CalendarEvent;\n\nimpl JmapObject for CalendarEvent {\n    type Property = JSCalendarProperty<Id>;\n\n    type Element = JSCalendarValue<Id, BlobId>;\n\n    type Id = Id;\n\n    type Filter = CalendarEventFilter;\n\n    type Comparator = CalendarEventComparator;\n\n    type GetArguments = CalendarEventGetArguments;\n\n    type SetArguments<'de> = CalendarEventSetArguments;\n\n    type QueryArguments = CalendarEventQueryArguments;\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = JSCalendarProperty::Id;\n}\n\nimpl JmapObjectId for JSCalendarValue<Id, BlobId> {\n    fn as_id(&self) -> Option<Id> {\n        if let JSCalendarValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            JSCalendarValue::Id(id) => Some(AnyId::Id(*id)),\n            JSCalendarValue::BlobId(blob_id) => Some(AnyId::BlobId(blob_id.clone())),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        match self {\n            JSCalendarValue::IdReference(r) => Some(r),\n            _ => None,\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = JSCalendarValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum CalendarEventFilter {\n    InCalendar(MaybeInvalid<Id>),\n    After(JSCalendarDateTime),\n    Before(JSCalendarDateTime),\n    Text(String),\n    Title(String),\n    Description(String),\n    Location(String),\n    Owner(String),\n    Attendee(String),\n    Uid(String),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum CalendarEventComparator {\n    Start,\n    Uid,\n    RecurrenceId,\n    Created,\n    Updated,\n    _T(String),\n}\n\n#[derive(Debug, Clone, Default)]\npub struct CalendarEventGetArguments {\n    pub recurrence_overrides_before: Option<JSCalendarDateTime>,\n    pub recurrence_overrides_after: Option<JSCalendarDateTime>,\n    pub reduce_participants: Option<bool>,\n    pub time_zone: Option<Tz>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct CalendarEventSetArguments {\n    pub send_scheduling_messages: Option<bool>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct CalendarEventQueryArguments {\n    pub expand_recurrences: Option<bool>,\n    pub time_zone: Option<Tz>,\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarEventFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"inCalendar\" => {\n                *self = CalendarEventFilter::InCalendar(map.next_value()?);\n            },\n            b\"after\" => {\n                *self = CalendarEventFilter::After(map.next_value::<LocalTime>()?.0);\n            },\n            b\"before\" => {\n                *self = CalendarEventFilter::Before(map.next_value::<LocalTime>()?.0);\n            },\n            b\"text\" => {\n                *self = CalendarEventFilter::Text(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"title\" => {\n                *self = CalendarEventFilter::Title(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"description\" => {\n                *self = CalendarEventFilter::Description(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"location\" => {\n                *self = CalendarEventFilter::Location(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"owner\" => {\n                *self = CalendarEventFilter::Owner(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"attendee\" => {\n                *self = CalendarEventFilter::Attendee(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"uid\" => {\n                *self = CalendarEventFilter::Uid(map.next_value()?);\n            },\n            _ => {\n                *self = CalendarEventFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarEventComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"start\" => {\n                    *self = CalendarEventComparator::Start;\n                },\n                b\"uid\" => {\n                    *self = CalendarEventComparator::Uid;\n                },\n                b\"recurrenceId\" => {\n                    *self = CalendarEventComparator::RecurrenceId;\n                },\n                b\"created\" => {\n                    *self = CalendarEventComparator::Created;\n                },\n                b\"updated\" => {\n                    *self = CalendarEventComparator::Updated;\n                },\n                _ => {\n                    *self = CalendarEventComparator::_T(value.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarEventGetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"recurrenceOverridesBefore\" => {\n                self.recurrence_overrides_before = map.next_value::<Option<LocalTime>>()?.map(|lt| lt.0)\n            },\n            b\"recurrenceOverridesAfter\" => {\n                self.recurrence_overrides_after = map.next_value::<Option<LocalTime>>()?.map(|lt| lt.0);\n            },\n            b\"reduceParticipants\" => {\n                self.reduce_participants = map.next_value()?;\n            },\n            b\"timeZone\" => {\n                self.time_zone = map.next_value::<Option<&str>>()?.and_then(|s| Tz::from_str(s).ok());\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarEventSetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"sendSchedulingMessages\" => {\n                self.send_scheduling_messages = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarEventQueryArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"expandRecurrences\" => {\n                self.expand_recurrences = map.next_value()?;\n            },\n            b\"timeZone\" => {\n                self.time_zone = map.next_value::<Option<&str>>()?.and_then(|s| Tz::from_str(s).ok());\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n        Ok(())\n    }\n}\n\nimpl CalendarEventFilter {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            CalendarEventFilter::InCalendar(_) => \"inCalendar\",\n            CalendarEventFilter::After(_) => \"after\",\n            CalendarEventFilter::Before(_) => \"before\",\n            CalendarEventFilter::Text(_) => \"text\",\n            CalendarEventFilter::Title(_) => \"title\",\n            CalendarEventFilter::Description(_) => \"description\",\n            CalendarEventFilter::Location(_) => \"location\",\n            CalendarEventFilter::Owner(_) => \"owner\",\n            CalendarEventFilter::Attendee(_) => \"attendee\",\n            CalendarEventFilter::Uid(_) => \"uid\",\n            CalendarEventFilter::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl CalendarEventComparator {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            CalendarEventComparator::Start => \"start\",\n            CalendarEventComparator::Uid => \"uid\",\n            CalendarEventComparator::RecurrenceId => \"recurrenceId\",\n            CalendarEventComparator::Created => \"created\",\n            CalendarEventComparator::Updated => \"updated\",\n            CalendarEventComparator::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl Default for CalendarEventFilter {\n    fn default() -> Self {\n        CalendarEventFilter::_T(String::new())\n    }\n}\n\nimpl Default for CalendarEventComparator {\n    fn default() -> Self {\n        CalendarEventComparator::_T(String::new())\n    }\n}\n\nstruct LocalTime(JSCalendarDateTime);\n\nimpl<'de> serde::Deserialize<'de> for LocalTime {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        let value = <&str>::deserialize(deserializer)?;\n\n        if let Some(dt) = DateTime::parse_rfc3339(value) {\n            Ok(LocalTime(JSCalendarDateTime {\n                timestamp: dt.to_timestamp_local(),\n                is_local: true,\n            }))\n        } else {\n            Err(serde::de::Error::custom(format!(\n                \"Invalid datetime: {}\",\n                value\n            )))\n        }\n    }\n}\n\nimpl JmapObjectId for JSCalendarProperty<Id> {\n    fn as_id(&self) -> Option<Id> {\n        if let JSCalendarProperty::IdValue(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let JSCalendarProperty::IdValue(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        match self {\n            JSCalendarProperty::IdReference(r) => Some(r),\n            JSCalendarProperty::Pointer(value) => {\n                let value = value.as_slice();\n                match (value.first(), value.get(1)) {\n                    (\n                        Some(JsonPointerItem::Key(Key::Property(JSCalendarProperty::CalendarIds))),\n                        Some(JsonPointerItem::Key(Key::Property(JSCalendarProperty::IdReference(\n                            r,\n                        )))),\n                    ) => Some(r),\n                    _ => None,\n                }\n            }\n            _ => None,\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            if let JSCalendarProperty::Pointer(value) = self {\n                let value = value.as_mut_slice();\n                if let Some(value) = value.get_mut(1) {\n                    *value = JsonPointerItem::Key(Key::Property(JSCalendarProperty::IdValue(id)));\n                    return true;\n                }\n            } else {\n                *self = JSCalendarProperty::IdValue(id);\n                return true;\n            }\n        }\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/calendar_event_notification.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    request::{MaybeInvalid, deserialize::DeserializeArguments},\n    types::{date::UTCDate, state::State},\n};\nuse calcard::jscalendar::JSCalendar;\nuse jmap_tools::{Element, Key, Property};\nuse serde::Serialize;\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\nuse types::{blob::BlobId, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct CalendarEventNotification;\n\n#[derive(Debug, Serialize, Clone, Default)]\n#[serde(rename_all = \"camelCase\")]\npub struct CalendarEventNotificationObject {\n    pub id: Id,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub created: Option<UTCDate>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub changed_by: Option<PersonObject>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub comment: Option<String>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    #[serde(rename = \"type\")]\n    pub notification_type: Option<CalendarEventNotificationType>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub calendar_event_id: Option<Id>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub is_draft: Option<bool>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub event: Option<JSCalendar<'static, Id, BlobId>>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub event_patch: Option<JSCalendar<'static, Id, BlobId>>,\n}\n\n#[derive(Debug, Serialize, Clone, Default)]\n#[serde(rename_all = \"camelCase\")]\npub struct PersonObject {\n    pub name: String,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub email: Option<String>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub principal_id: Option<Id>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub calendar_address: Option<String>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct CalendarEventNotificationGetResponse {\n    #[serde(rename = \"accountId\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub account_id: Option<Id>,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub state: Option<State>,\n\n    pub list: Vec<CalendarEventNotificationObject>,\n\n    #[serde(rename = \"notFound\")]\n    pub not_found: Vec<Id>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum CalendarEventNotificationProperty {\n    Id,\n    Created,\n    ChangedBy,\n    Comment,\n    Type,\n    CalendarEventId,\n    IsDraft,\n    Event,\n    EventPatch,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum CalendarEventNotificationValue {\n    Id(Id),\n    Date(UTCDate),\n    Type(CalendarEventNotificationType),\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum CalendarEventNotificationType {\n    Created,\n    Updated,\n    Destroyed,\n}\n\nimpl Property for CalendarEventNotificationProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        CalendarEventNotificationProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            CalendarEventNotificationProperty::Id => \"id\",\n            CalendarEventNotificationProperty::Created => \"created\",\n            CalendarEventNotificationProperty::ChangedBy => \"changedBy\",\n            CalendarEventNotificationProperty::Comment => \"comment\",\n            CalendarEventNotificationProperty::Type => \"type\",\n            CalendarEventNotificationProperty::CalendarEventId => \"calendarEventId\",\n            CalendarEventNotificationProperty::IsDraft => \"isDraft\",\n            CalendarEventNotificationProperty::Event => \"event\",\n            CalendarEventNotificationProperty::EventPatch => \"eventPatch\",\n        }\n        .into()\n    }\n}\n\nimpl Element for CalendarEventNotificationValue {\n    type Property = CalendarEventNotificationProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                CalendarEventNotificationProperty::Id\n                | CalendarEventNotificationProperty::CalendarEventId => Id::from_str(value)\n                    .ok()\n                    .map(CalendarEventNotificationValue::Id),\n                CalendarEventNotificationProperty::Created => UTCDate::from_str(value)\n                    .ok()\n                    .map(CalendarEventNotificationValue::Date),\n                CalendarEventNotificationProperty::Type => {\n                    CalendarEventNotificationType::parse(value)\n                        .map(CalendarEventNotificationValue::Type)\n                }\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            CalendarEventNotificationValue::Id(id) => id.to_string().into(),\n            CalendarEventNotificationValue::Date(date) => date.to_string().into(),\n            CalendarEventNotificationValue::Type(t) => t.as_str().into(),\n        }\n    }\n}\n\nimpl CalendarEventNotificationType {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"created\" => CalendarEventNotificationType::Created,\n            b\"updated\" => CalendarEventNotificationType::Updated,\n            b\"destroyed\" => CalendarEventNotificationType::Destroyed,\n        )\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            CalendarEventNotificationType::Created => \"created\",\n            CalendarEventNotificationType::Updated => \"updated\",\n            CalendarEventNotificationType::Destroyed => \"destroyed\",\n        }\n    }\n}\n\nimpl CalendarEventNotificationProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => CalendarEventNotificationProperty::Id,\n            b\"created\" => CalendarEventNotificationProperty::Created,\n            b\"changedBy\" => CalendarEventNotificationProperty::ChangedBy,\n            b\"comment\" => CalendarEventNotificationProperty::Comment,\n            b\"type\" => CalendarEventNotificationProperty::Type,\n            b\"calendarEventId\" => CalendarEventNotificationProperty::CalendarEventId,\n            b\"isDraft\" => CalendarEventNotificationProperty::IsDraft,\n            b\"event\" => CalendarEventNotificationProperty::Event,\n            b\"eventPatch\" => CalendarEventNotificationProperty::EventPatch\n        )\n    }\n}\n\nimpl FromStr for CalendarEventNotificationProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        CalendarEventNotificationProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for CalendarEventNotification {\n    type Property = CalendarEventNotificationProperty;\n\n    type Element = CalendarEventNotificationValue;\n\n    type Id = Id;\n\n    type Filter = CalendarEventNotificationFilter;\n\n    type Comparator = CalendarEventNotificationComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = CalendarEventNotificationProperty::Id;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum CalendarEventNotificationFilter {\n    After(UTCDate),\n    Before(UTCDate),\n    Type(CalendarEventNotificationType),\n    CalendarEventIds(Vec<MaybeInvalid<Id>>),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum CalendarEventNotificationComparator {\n    Created,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarEventNotificationFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"after\" => {\n                *self = CalendarEventNotificationFilter::After(map.next_value()?);\n            },\n            b\"before\" => {\n                *self = CalendarEventNotificationFilter::Before(map.next_value()?);\n            },\n            b\"type\" => {\n                *self = CalendarEventNotificationFilter::Type(map.next_value()?);\n            },\n            b\"calendarEventIds\" => {\n                *self = CalendarEventNotificationFilter::CalendarEventIds(map.next_value()?);\n            },\n            _ => {\n                *self = CalendarEventNotificationFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for CalendarEventNotificationComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"created\" => {\n                    *self = CalendarEventNotificationComparator::Created;\n                },\n                _ => {\n                    *self = CalendarEventNotificationComparator::_T(value.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n        Ok(())\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for CalendarEventNotificationType {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        CalendarEventNotificationType::parse(<&str>::deserialize(deserializer)?)\n            .ok_or_else(|| serde::de::Error::custom(\"invalid CalendarEventNotificationType\"))\n    }\n}\n\nimpl CalendarEventNotificationFilter {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            CalendarEventNotificationFilter::After(_) => \"after\",\n            CalendarEventNotificationFilter::Before(_) => \"before\",\n            CalendarEventNotificationFilter::Type(_) => \"type\",\n            CalendarEventNotificationFilter::CalendarEventIds(_) => \"calendarEventIds\",\n            CalendarEventNotificationFilter::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl CalendarEventNotificationComparator {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            CalendarEventNotificationComparator::Created => \"created\",\n            CalendarEventNotificationComparator::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl Default for CalendarEventNotificationFilter {\n    fn default() -> Self {\n        CalendarEventNotificationFilter::_T(String::new())\n    }\n}\n\nimpl Default for CalendarEventNotificationComparator {\n    fn default() -> Self {\n        CalendarEventNotificationComparator::_T(String::new())\n    }\n}\n\nimpl TryFrom<CalendarEventNotificationProperty> for Id {\n    type Error = ();\n\n    fn try_from(_: CalendarEventNotificationProperty) -> Result<Self, Self::Error> {\n        Err(())\n    }\n}\n\nimpl From<Id> for CalendarEventNotificationValue {\n    fn from(id: Id) -> Self {\n        CalendarEventNotificationValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for CalendarEventNotificationValue {\n    fn as_id(&self) -> Option<Id> {\n        if let CalendarEventNotificationValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let CalendarEventNotificationValue::Id(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n\nimpl JmapObjectId for CalendarEventNotificationProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n\nimpl serde::Serialize for CalendarEventNotificationType {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.as_str())\n    }\n}\n\nimpl Display for CalendarEventNotificationProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"{}\", self.to_cow())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/contact.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    request::{MaybeInvalid, deserialize::DeserializeArguments},\n    types::date::UTCDate,\n};\nuse calcard::jscontact::{JSContactProperty, JSContactValue};\nuse jmap_tools::{JsonPointerItem, Key};\nuse std::borrow::Cow;\nuse types::{blob::BlobId, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct ContactCard;\n\nimpl JmapObject for ContactCard {\n    type Property = JSContactProperty<Id>;\n\n    type Element = JSContactValue<Id, BlobId>;\n\n    type Id = Id;\n\n    type Filter = ContactCardFilter;\n\n    type Comparator = ContactCardComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = JSContactProperty::Id;\n}\n\nimpl JmapObjectId for JSContactValue<Id, BlobId> {\n    fn as_id(&self) -> Option<Id> {\n        if let JSContactValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            JSContactValue::Id(id) => Some(AnyId::Id(*id)),\n            JSContactValue::BlobId(id) => Some(AnyId::BlobId(id.clone())),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        match self {\n            JSContactValue::IdReference(r) => Some(r),\n            _ => None,\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        match new_id {\n            AnyId::Id(id) => {\n                *self = JSContactValue::Id(id);\n            }\n            AnyId::BlobId(id) => {\n                *self = JSContactValue::BlobId(id);\n            }\n        }\n\n        true\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ContactCardFilter {\n    InAddressBook(MaybeInvalid<Id>),\n    Uid(String),\n    HasMember(String),\n    Kind(String),\n    CreatedBefore(UTCDate),\n    CreatedAfter(UTCDate),\n    UpdatedBefore(UTCDate),\n    UpdatedAfter(UTCDate),\n    Text(String),\n    Name(String),\n    NameGiven(String),\n    NameSurname(String),\n    NameSurname2(String),\n    Nickname(String),\n    Organization(String),\n    Email(String),\n    Phone(String),\n    OnlineService(String),\n    Address(String),\n    Note(String),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ContactCardComparator {\n    Created,\n    Updated,\n    NameGiven,\n    NameSurname,\n    NameSurname2,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for ContactCardFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"inAddressBook\" => {\n                *self = ContactCardFilter::InAddressBook(map.next_value()?);\n            },\n            b\"uid\" => {\n                *self = ContactCardFilter::Uid(map.next_value()?);\n            },\n            b\"hasMember\" => {\n                *self = ContactCardFilter::HasMember(map.next_value()?);\n            },\n            b\"kind\" => {\n                *self = ContactCardFilter::Kind(map.next_value()?);\n            },\n            b\"createdBefore\" => {\n                *self = ContactCardFilter::CreatedBefore(map.next_value()?);\n            },\n            b\"createdAfter\" => {\n                *self = ContactCardFilter::CreatedAfter(map.next_value()?);\n            },\n            b\"updatedBefore\" => {\n                *self = ContactCardFilter::UpdatedBefore(map.next_value()?);\n            },\n            b\"updatedAfter\" => {\n                *self = ContactCardFilter::UpdatedAfter(map.next_value()?);\n            },\n            b\"text\" => {\n                *self = ContactCardFilter::Text(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"name\" => {\n                *self = ContactCardFilter::Name(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"name/given\" => {\n                *self = ContactCardFilter::NameGiven(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"name/surname\" => {\n                *self = ContactCardFilter::NameSurname(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"name/surname2\" => {\n                *self = ContactCardFilter::NameSurname2(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"nickname\" => {\n                *self = ContactCardFilter::Nickname(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"organization\" => {\n                *self = ContactCardFilter::Organization(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"email\" => {\n                *self = ContactCardFilter::Email(map.next_value()?);\n            },\n            b\"phone\" => {\n                *self = ContactCardFilter::Phone(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"onlineService\" => {\n                *self = ContactCardFilter::OnlineService(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"address\" => {\n                *self = ContactCardFilter::Address(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            b\"note\" => {\n                *self = ContactCardFilter::Note(map.next_value::<Cow<str>>()?.to_lowercase());\n            },\n            _ => {\n                *self = ContactCardFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for ContactCardComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"created\" => {\n                    *self = ContactCardComparator::Created;\n                },\n                b\"updated\" => {\n                    *self = ContactCardComparator::Updated;\n                },\n                b\"name/given\" => {\n                    *self = ContactCardComparator::NameGiven;\n                },\n                b\"name/surname\" => {\n                    *self = ContactCardComparator::NameSurname;\n                },\n                b\"name/surname2\" => {\n                    *self = ContactCardComparator::NameSurname2;\n                },\n                _ => {\n                    *self = ContactCardComparator::_T(value.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n        Ok(())\n    }\n}\n\nimpl ContactCardFilter {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            ContactCardFilter::InAddressBook(_) => \"inAddressBook\",\n            ContactCardFilter::Uid(_) => \"uid\",\n            ContactCardFilter::HasMember(_) => \"hasMember\",\n            ContactCardFilter::Kind(_) => \"kind\",\n            ContactCardFilter::CreatedBefore(_) => \"createdBefore\",\n            ContactCardFilter::CreatedAfter(_) => \"createdAfter\",\n            ContactCardFilter::UpdatedBefore(_) => \"updatedBefore\",\n            ContactCardFilter::UpdatedAfter(_) => \"updatedAfter\",\n            ContactCardFilter::Text(_) => \"text\",\n            ContactCardFilter::Name(_) => \"name\",\n            ContactCardFilter::NameGiven(_) => \"name/given\",\n            ContactCardFilter::NameSurname(_) => \"name/surname\",\n            ContactCardFilter::NameSurname2(_) => \"name/surname2\",\n            ContactCardFilter::Nickname(_) => \"nickname\",\n            ContactCardFilter::Organization(_) => \"organization\",\n            ContactCardFilter::Email(_) => \"email\",\n            ContactCardFilter::Phone(_) => \"phone\",\n            ContactCardFilter::OnlineService(_) => \"onlineService\",\n            ContactCardFilter::Address(_) => \"address\",\n            ContactCardFilter::Note(_) => \"note\",\n            ContactCardFilter::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl ContactCardComparator {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            ContactCardComparator::Created => \"created\",\n            ContactCardComparator::Updated => \"updated\",\n            ContactCardComparator::NameGiven => \"name/given\",\n            ContactCardComparator::NameSurname => \"name/surname\",\n            ContactCardComparator::NameSurname2 => \"name/surname2\",\n            ContactCardComparator::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl Default for ContactCardFilter {\n    fn default() -> Self {\n        ContactCardFilter::_T(String::new())\n    }\n}\n\nimpl Default for ContactCardComparator {\n    fn default() -> Self {\n        ContactCardComparator::_T(String::new())\n    }\n}\n\nimpl JmapObjectId for JSContactProperty<Id> {\n    fn as_id(&self) -> Option<Id> {\n        if let JSContactProperty::IdValue(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let JSContactProperty::IdValue(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        match self {\n            JSContactProperty::IdReference(r) => Some(r),\n            JSContactProperty::Pointer(value) => {\n                let value = value.as_slice();\n                match (value.first(), value.get(1)) {\n                    (\n                        Some(JsonPointerItem::Key(Key::Property(\n                            JSContactProperty::AddressBookIds,\n                        ))),\n                        Some(JsonPointerItem::Key(Key::Property(JSContactProperty::IdReference(\n                            r,\n                        )))),\n                    ) => Some(r),\n                    _ => None,\n                }\n            }\n            _ => None,\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            if let JSContactProperty::Pointer(value) = self {\n                let value = value.as_mut_slice();\n                if let Some(value) = value.get_mut(1) {\n                    *value = JsonPointerItem::Key(Key::Property(JSContactProperty::IdValue(id)));\n                    return true;\n                }\n            } else {\n                *self = JSContactProperty::IdValue(id);\n                return true;\n            }\n        }\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/email.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    method::query::{Comparator, Filter},\n    object::{AnyId, JmapObject, JmapObjectId, MaybeReference, parse_ref},\n    request::{MaybeInvalid, deserialize::DeserializeArguments},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem, Key, Property};\nuse mail_parser::HeaderName;\nuse serde::Serialize;\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\nuse types::{blob::BlobId, id::Id, keyword::Keyword};\n\n#[derive(Debug, Clone, Default)]\npub struct Email;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum EmailProperty {\n    // Metadata\n    Id,\n    BlobId,\n    ThreadId,\n    MailboxIds,\n    Keywords,\n    Size,\n    ReceivedAt,\n\n    // Address\n    Name,\n    Email,\n\n    // GroupedAddresses\n    Addresses,\n\n    // Header Fields Properties\n    Value,\n    Header(HeaderProperty),\n\n    // Convenience properties\n    MessageId,\n    InReplyTo,\n    References,\n    Sender,\n    From,\n    To,\n    Cc,\n    Bcc,\n    ReplyTo,\n    Subject,\n    SentAt,\n\n    // Body Parts\n    TextBody,\n    HtmlBody,\n    Attachments,\n    PartId,\n    Headers,\n    Type,\n    Charset,\n    Disposition,\n    Cid,\n    Language,\n    Location,\n    SubParts,\n    BodyStructure,\n    BodyValues,\n    IsEncodingProblem,\n    IsTruncated,\n    HasAttachment,\n    Preview,\n\n    // Other\n    Keyword(Keyword),\n    IdValue(Id),\n    IdReference(String),\n    Pointer(JsonPointer<EmailProperty>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub struct HeaderProperty {\n    pub form: HeaderForm,\n    pub header: String,\n    pub all: bool,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum HeaderForm {\n    Raw,\n    Text,\n    Addresses,\n    GroupedAddresses,\n    MessageIds,\n    Date,\n    URLs,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum EmailValue {\n    Id(Id),\n    Date(UTCDate),\n    BlobId(BlobId),\n    IdReference(String),\n}\n\nimpl Property for EmailProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        let allow_patch = key.is_none();\n        if let Some(Key::Property(key)) = key {\n            match key.patch_or_prop() {\n                EmailProperty::Keywords => EmailProperty::Keyword(Keyword::parse(value)).into(),\n                EmailProperty::MailboxIds => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(EmailProperty::IdValue(v)),\n                    MaybeReference::Reference(v) => Some(EmailProperty::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                _ => EmailProperty::parse(value, allow_patch),\n            }\n        } else {\n            EmailProperty::parse(value, allow_patch)\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            EmailProperty::Attachments => \"attachments\",\n            EmailProperty::Bcc => \"bcc\",\n            EmailProperty::BlobId => \"blobId\",\n            EmailProperty::BodyStructure => \"bodyStructure\",\n            EmailProperty::BodyValues => \"bodyValues\",\n            EmailProperty::Cc => \"cc\",\n            EmailProperty::Charset => \"charset\",\n            EmailProperty::Cid => \"cid\",\n            EmailProperty::Disposition => \"disposition\",\n            EmailProperty::Email => \"email\",\n            EmailProperty::From => \"from\",\n            EmailProperty::HasAttachment => \"hasAttachment\",\n            EmailProperty::Headers => \"headers\",\n            EmailProperty::HtmlBody => \"htmlBody\",\n            EmailProperty::Id => \"id\",\n            EmailProperty::InReplyTo => \"inReplyTo\",\n            EmailProperty::Keywords => \"keywords\",\n            EmailProperty::Language => \"language\",\n            EmailProperty::Location => \"location\",\n            EmailProperty::MailboxIds => \"mailboxIds\",\n            EmailProperty::MessageId => \"messageId\",\n            EmailProperty::Name => \"name\",\n            EmailProperty::PartId => \"partId\",\n            EmailProperty::Preview => \"preview\",\n            EmailProperty::ReceivedAt => \"receivedAt\",\n            EmailProperty::References => \"references\",\n            EmailProperty::ReplyTo => \"replyTo\",\n            EmailProperty::Sender => \"sender\",\n            EmailProperty::SentAt => \"sentAt\",\n            EmailProperty::Size => \"size\",\n            EmailProperty::Subject => \"subject\",\n            EmailProperty::SubParts => \"subParts\",\n            EmailProperty::TextBody => \"textBody\",\n            EmailProperty::ThreadId => \"threadId\",\n            EmailProperty::To => \"to\",\n            EmailProperty::Type => \"type\",\n            EmailProperty::Addresses => \"addresses\",\n            EmailProperty::Value => \"value\",\n            EmailProperty::IsEncodingProblem => \"isEncodingProblem\",\n            EmailProperty::IsTruncated => \"isTruncated\",\n            EmailProperty::Header(header) => return header.to_string().into(),\n            EmailProperty::Keyword(keyword) => return keyword.to_string().into(),\n            EmailProperty::IdValue(id) => return id.to_string().into(),\n            EmailProperty::Pointer(json_pointer) => return json_pointer.to_string().into(),\n            EmailProperty::IdReference(r) => return format!(\"#{r}\").into(),\n        }\n        .into()\n    }\n}\n\nimpl Element for EmailValue {\n    type Property = EmailProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                EmailProperty::Id | EmailProperty::ThreadId | EmailProperty::MailboxIds => {\n                    match parse_ref(value) {\n                        MaybeReference::Value(v) => Some(EmailValue::Id(v)),\n                        MaybeReference::Reference(v) => Some(EmailValue::IdReference(v)),\n                        MaybeReference::ParseError => None,\n                    }\n                }\n                EmailProperty::BlobId => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(EmailValue::BlobId(v)),\n                    MaybeReference::Reference(v) => Some(EmailValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                EmailProperty::Header(HeaderProperty {\n                    form: HeaderForm::Date,\n                    ..\n                })\n                | EmailProperty::ReceivedAt\n                | EmailProperty::SentAt => UTCDate::from_str(value).ok().map(EmailValue::Date),\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            EmailValue::Id(id) => id.to_string().into(),\n            EmailValue::Date(utcdate) => utcdate.to_string().into(),\n            EmailValue::BlobId(blob_id) => blob_id.to_string().into(),\n            EmailValue::IdReference(r) => format!(\"#{r}\").into(),\n        }\n    }\n}\n\nimpl EmailProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n                \"id\" => EmailProperty::Id,\n                \"blobId\" => EmailProperty::BlobId,\n                \"threadId\" => EmailProperty::ThreadId,\n                \"mailboxIds\" => EmailProperty::MailboxIds,\n                \"keywords\" => EmailProperty::Keywords,\n                \"size\" => EmailProperty::Size,\n                \"receivedAt\" => EmailProperty::ReceivedAt,\n                \"name\" => EmailProperty::Name,\n                \"email\" => EmailProperty::Email,\n                \"addresses\" => EmailProperty::Addresses,\n                \"value\" => EmailProperty::Value,\n                \"messageId\" => EmailProperty::MessageId,\n                \"inReplyTo\" => EmailProperty::InReplyTo,\n                \"references\" => EmailProperty::References,\n                \"sender\" => EmailProperty::Sender,\n                \"from\" => EmailProperty::From,\n                \"to\" => EmailProperty::To,\n                \"cc\" => EmailProperty::Cc,\n                \"bcc\" => EmailProperty::Bcc,\n                \"replyTo\" => EmailProperty::ReplyTo,\n                \"subject\" => EmailProperty::Subject,\n                \"sentAt\" => EmailProperty::SentAt,\n                \"textBody\" => EmailProperty::TextBody,\n                \"htmlBody\" => EmailProperty::HtmlBody,\n                \"attachments\" => EmailProperty::Attachments,\n                \"partId\" => EmailProperty::PartId,\n                \"headers\" => EmailProperty::Headers,\n                \"type\" => EmailProperty::Type,\n                \"charset\" => EmailProperty::Charset,\n                \"disposition\" => EmailProperty::Disposition,\n                \"cid\" => EmailProperty::Cid,\n                \"language\" => EmailProperty::Language,\n                \"location\" => EmailProperty::Location,\n                \"subParts\" => EmailProperty::SubParts,\n                \"bodyStructure\" => EmailProperty::BodyStructure,\n                \"bodyValues\" => EmailProperty::BodyValues,\n                \"isEncodingProblem\" => EmailProperty::IsEncodingProblem,\n                \"isTruncated\" => EmailProperty::IsTruncated,\n                \"hasAttachment\" => EmailProperty::HasAttachment,\n                \"preview\" => EmailProperty::Preview\n        )\n        .or_else(|| {\n            if let Some(header) = value.strip_prefix(\"header:\") {\n                HeaderProperty::parse(header).map(EmailProperty::Header)\n            } else if allow_patch && value.contains('/') {\n                EmailProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &EmailProperty {\n        if let EmailProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n\n    pub fn as_rfc_header(&self) -> HeaderName<'static> {\n        match self {\n            EmailProperty::MessageId => HeaderName::MessageId,\n            EmailProperty::InReplyTo => HeaderName::InReplyTo,\n            EmailProperty::References => HeaderName::References,\n            EmailProperty::Sender => HeaderName::Sender,\n            EmailProperty::From => HeaderName::From,\n            EmailProperty::To => HeaderName::To,\n            EmailProperty::Cc => HeaderName::Cc,\n            EmailProperty::Bcc => HeaderName::Bcc,\n            EmailProperty::ReplyTo => HeaderName::ReplyTo,\n            EmailProperty::Subject => HeaderName::Subject,\n            EmailProperty::SentAt => HeaderName::Date,\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn try_into_id(self) -> Option<Id> {\n        match self {\n            EmailProperty::IdValue(id) => Some(id),\n            _ => None,\n        }\n    }\n\n    pub fn try_into_keyword(self) -> Option<Keyword> {\n        match self {\n            EmailProperty::Keyword(keyword) => Some(keyword),\n            _ => None,\n        }\n    }\n}\n\nimpl HeaderProperty {\n    fn parse(value: &str) -> Option<Self> {\n        let mut result = HeaderProperty {\n            form: HeaderForm::Raw,\n            header: String::new(),\n            all: false,\n        };\n\n        for (pos, value) in value.split(':').enumerate() {\n            match pos {\n                0 => {\n                    result.header = value.to_string();\n                }\n                1 => {\n                    hashify::fnc_map!(value.as_bytes(),\n                        b\"asText\" => { result.form = HeaderForm::Text;},\n                        b\"asAddresses\" => { result.form = HeaderForm::Addresses;},\n                        b\"asGroupedAddresses\" => { result.form = HeaderForm::GroupedAddresses;},\n                        b\"asMessageIds\" => { result.form = HeaderForm::MessageIds;},\n                        b\"asDate\" => { result.form = HeaderForm::Date;},\n                        b\"asURLs\" => { result.form = HeaderForm::URLs;},\n                        b\"asRaw\"  => { result.form = HeaderForm::Raw; },\n                        b\"all\"  => { result.all = true; },\n                        _ => {\n                            return None;\n                        }\n                    );\n                }\n                2 if value == \"all\" && !result.all => {\n                    result.all = true;\n                }\n                _ => return None,\n            }\n        }\n\n        if !result.header.is_empty() {\n            Some(result)\n        } else {\n            None\n        }\n    }\n}\n\nimpl Display for HeaderProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {\n        write!(f, \"header:{}\", self.header)?;\n        self.form.fmt(f)?;\n        if self.all { write!(f, \":all\") } else { Ok(()) }\n    }\n}\n\nimpl Display for HeaderForm {\n    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {\n        match self {\n            HeaderForm::Raw => Ok(()),\n            HeaderForm::Text => write!(f, \":asText\"),\n            HeaderForm::Addresses => write!(f, \":asAddresses\"),\n            HeaderForm::GroupedAddresses => write!(f, \":asGroupedAddresses\"),\n            HeaderForm::MessageIds => write!(f, \":asMessageIds\"),\n            HeaderForm::Date => write!(f, \":asDate\"),\n            HeaderForm::URLs => write!(f, \":asURLs\"),\n        }\n    }\n}\n\nimpl FromStr for EmailProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        EmailProperty::parse(s, false).ok_or(())\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct EmailGetArguments {\n    pub body_properties: Option<Vec<MaybeInvalid<EmailProperty>>>,\n    pub fetch_text_body_values: Option<bool>,\n    pub fetch_html_body_values: Option<bool>,\n    pub fetch_all_body_values: Option<bool>,\n    pub max_body_value_bytes: Option<usize>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct EmailQueryArguments {\n    pub collapse_threads: Option<bool>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct EmailParseArguments {\n    pub body_properties: Option<Vec<MaybeInvalid<EmailProperty>>>,\n    pub fetch_text_body_values: Option<bool>,\n    pub fetch_html_body_values: Option<bool>,\n    pub fetch_all_body_values: Option<bool>,\n    pub max_body_value_bytes: Option<usize>,\n}\n\nimpl<'de> DeserializeArguments<'de> for EmailGetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"bodyProperties\" => {\n                self.body_properties = map.next_value()?;\n            },\n            b\"fetchTextBodyValues\" => {\n                self.fetch_text_body_values = map.next_value()?;\n            },\n            b\"fetchHTMLBodyValues\" => {\n                self.fetch_html_body_values = map.next_value()?;\n            },\n            b\"fetchAllBodyValues\" => {\n                self.fetch_all_body_values = map.next_value()?;\n            },\n            b\"maxBodyValueBytes\" => {\n                self.max_body_value_bytes = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for EmailQueryArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"collapseThreads\" {\n            self.collapse_threads = map.next_value()?;\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for EmailParseArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"bodyProperties\" => {\n                self.body_properties = map.next_value()?;\n            },\n            b\"fetchTextBodyValues\" => {\n                self.fetch_text_body_values = map.next_value()?;\n            },\n            b\"fetchHTMLBodyValues\" => {\n                self.fetch_html_body_values = map.next_value()?;\n            },\n            b\"fetchAllBodyValues\" => {\n                self.fetch_all_body_values = map.next_value()?;\n            },\n            b\"maxBodyValueBytes\" => {\n                self.max_body_value_bytes = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl JmapObject for Email {\n    type Property = EmailProperty;\n\n    type Element = EmailValue;\n\n    type Id = Id;\n\n    type Filter = EmailFilter;\n\n    type Comparator = EmailComparator;\n\n    type GetArguments = EmailGetArguments;\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = EmailQueryArguments;\n\n    type CopyArguments = ();\n\n    type ParseArguments = EmailParseArguments;\n\n    const ID_PROPERTY: Self::Property = EmailProperty::Id;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum EmailFilter {\n    InMailbox(Id),\n    InMailboxOtherThan(Vec<Id>),\n    Before(UTCDate),\n    After(UTCDate),\n    MinSize(u32),\n    MaxSize(u32),\n    AllInThreadHaveKeyword(Keyword),\n    SomeInThreadHaveKeyword(Keyword),\n    NoneInThreadHaveKeyword(Keyword),\n    HasKeyword(Keyword),\n    NotKeyword(Keyword),\n    HasAttachment(bool),\n    From(String),\n    To(String),\n    Cc(String),\n    Bcc(String),\n    Subject(String),\n    Body(String),\n    Header(Vec<String>),\n    Text(String),\n    SentBefore(UTCDate),\n    SentAfter(UTCDate),\n    InThread(Id),\n    Id(Vec<Id>),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum EmailComparator {\n    ReceivedAt,\n    Size,\n    From,\n    To,\n    Subject,\n    Cc,\n    SentAt,\n    ThreadId,\n    HasKeyword(Keyword),\n    AllInThreadHaveKeyword(Keyword),\n    SomeInThreadHaveKeyword(Keyword),\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for EmailFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"inMailbox\" => {\n                *self = EmailFilter::InMailbox(map.next_value()?);\n            },\n            b\"inMailboxOtherThan\" => {\n                *self = EmailFilter::InMailboxOtherThan(map.next_value()?);\n            },\n            b\"before\" => {\n                *self = EmailFilter::Before(map.next_value()?);\n            },\n            b\"after\" => {\n                *self = EmailFilter::After(map.next_value()?);\n            },\n            b\"minSize\" => {\n                *self = EmailFilter::MinSize(map.next_value()?);\n            },\n            b\"maxSize\" => {\n                *self = EmailFilter::MaxSize(map.next_value()?);\n            },\n            b\"allInThreadHaveKeyword\" => {\n                *self = EmailFilter::AllInThreadHaveKeyword(map.next_value()?);\n            },\n            b\"someInThreadHaveKeyword\" => {\n                *self = EmailFilter::SomeInThreadHaveKeyword(map.next_value()?);\n            },\n            b\"noneInThreadHaveKeyword\" => {\n                *self = EmailFilter::NoneInThreadHaveKeyword(map.next_value()?);\n            },\n            b\"hasKeyword\" => {\n                *self = EmailFilter::HasKeyword(map.next_value()?);\n            },\n            b\"notKeyword\" => {\n                *self = EmailFilter::NotKeyword(map.next_value()?);\n            },\n            b\"hasAttachment\" => {\n                *self = EmailFilter::HasAttachment(map.next_value()?);\n            },\n            b\"from\" => {\n                *self = EmailFilter::From(map.next_value()?);\n            },\n            b\"to\" => {\n                *self = EmailFilter::To(map.next_value()?);\n            },\n            b\"cc\" => {\n                *self = EmailFilter::Cc(map.next_value()?);\n            },\n            b\"bcc\" => {\n                *self = EmailFilter::Bcc(map.next_value()?);\n            },\n            b\"subject\" => {\n                *self = EmailFilter::Subject(map.next_value()?);\n            },\n            b\"body\" => {\n                *self = EmailFilter::Body(map.next_value()?);\n            },\n            b\"header\" => {\n                *self = EmailFilter::Header(map.next_value()?);\n            },\n            b\"text\" => {\n                *self = EmailFilter::Text(map.next_value()?);\n            },\n            b\"sentBefore\" => {\n                *self = EmailFilter::SentBefore(map.next_value()?);\n            },\n            b\"sentAfter\" => {\n                *self = EmailFilter::SentAfter(map.next_value()?);\n            },\n            b\"inThread\" => {\n                *self = EmailFilter::InThread(map.next_value()?);\n            },\n            b\"id\" => {\n                *self = EmailFilter::Id(map.next_value()?);\n            },\n            _ => {\n                *self = EmailFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for EmailComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"receivedAt\" => {\n                    *self = EmailComparator::ReceivedAt;\n                },\n                b\"size\" => {\n                    *self = EmailComparator::Size;\n                },\n                b\"from\" => {\n                    *self = EmailComparator::From;\n                },\n                b\"to\" => {\n                    *self = EmailComparator::To;\n                },\n                b\"cc\" => {\n                    *self = EmailComparator::Cc;\n                },\n                b\"subject\" => {\n                    *self = EmailComparator::Subject;\n                },\n                b\"sentAt\" => {\n                    *self = EmailComparator::SentAt;\n                },\n                b\"threadId\" => {\n                    *self = EmailComparator::ThreadId;\n                },\n                b\"hasKeyword\" => {\n                    *self = EmailComparator::HasKeyword(self.take_keyword());\n                },\n                b\"allInThreadHaveKeyword\" => {\n                    *self = EmailComparator::AllInThreadHaveKeyword(self.take_keyword());\n                },\n                b\"someInThreadHaveKeyword\" => {\n                    *self = EmailComparator::SomeInThreadHaveKeyword(self.take_keyword());\n                },\n                _ => {\n                    *self = EmailComparator::_T(key.to_string());\n                }\n            );\n        } else if key == \"keyword\" {\n            let keyword: Keyword = map.next_value()?;\n            match self {\n                EmailComparator::HasKeyword(_) => *self = EmailComparator::HasKeyword(keyword),\n                EmailComparator::AllInThreadHaveKeyword(_) => {\n                    *self = EmailComparator::AllInThreadHaveKeyword(keyword)\n                }\n                EmailComparator::SomeInThreadHaveKeyword(_) => {\n                    *self = EmailComparator::SomeInThreadHaveKeyword(keyword)\n                }\n                _ => {\n                    *self = EmailComparator::HasKeyword(keyword);\n                }\n            }\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Default for EmailFilter {\n    fn default() -> Self {\n        EmailFilter::_T(\"\".to_string())\n    }\n}\n\nimpl Default for EmailComparator {\n    fn default() -> Self {\n        EmailComparator::_T(\"\".to_string())\n    }\n}\n\nimpl EmailComparator {\n    fn take_keyword(&mut self) -> Keyword {\n        match self {\n            EmailComparator::HasKeyword(k) => {\n                std::mem::replace(k, Keyword::Other(Default::default()))\n            }\n            EmailComparator::AllInThreadHaveKeyword(k) => {\n                std::mem::replace(k, Keyword::Other(Default::default()))\n            }\n            EmailComparator::SomeInThreadHaveKeyword(k) => {\n                std::mem::replace(k, Keyword::Other(Default::default()))\n            }\n            _ => Keyword::Other(Default::default()),\n        }\n    }\n}\n\nimpl Display for EmailFilter {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(match self {\n            EmailFilter::InMailbox(_) => \"inMailbox\",\n            EmailFilter::InMailboxOtherThan(_) => \"inMailboxOtherThan\",\n            EmailFilter::Before(_) => \"before\",\n            EmailFilter::After(_) => \"after\",\n            EmailFilter::MinSize(_) => \"minSize\",\n            EmailFilter::MaxSize(_) => \"maxSize\",\n            EmailFilter::AllInThreadHaveKeyword(_) => \"allInThreadHaveKeyword\",\n            EmailFilter::SomeInThreadHaveKeyword(_) => \"someInThreadHaveKeyword\",\n            EmailFilter::NoneInThreadHaveKeyword(_) => \"noneInThreadHaveKeyword\",\n            EmailFilter::HasKeyword(_) => \"hasKeyword\",\n            EmailFilter::NotKeyword(_) => \"notKeyword\",\n            EmailFilter::HasAttachment(_) => \"hasAttachment\",\n            EmailFilter::From(_) => \"from\",\n            EmailFilter::To(_) => \"to\",\n            EmailFilter::Cc(_) => \"cc\",\n            EmailFilter::Bcc(_) => \"bcc\",\n            EmailFilter::Subject(_) => \"subject\",\n            EmailFilter::Body(_) => \"body\",\n            EmailFilter::Header(_) => \"header\",\n            EmailFilter::Text(_) => \"text\",\n            EmailFilter::SentBefore(_) => \"sentBefore\",\n            EmailFilter::SentAfter(_) => \"sentAfter\",\n            EmailFilter::InThread(_) => \"inThread\",\n            EmailFilter::Id(_) => \"id\",\n            EmailFilter::_T(v) => v.as_str(),\n        })\n    }\n}\n\nimpl Display for EmailComparator {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(self.as_str())\n    }\n}\n\nimpl EmailComparator {\n    pub fn as_str(&self) -> &str {\n        match self {\n            EmailComparator::ReceivedAt => \"receivedAt\",\n            EmailComparator::Size => \"size\",\n            EmailComparator::From => \"from\",\n            EmailComparator::To => \"to\",\n            EmailComparator::Subject => \"subject\",\n            EmailComparator::Cc => \"cc\",\n            EmailComparator::SentAt => \"sentAt\",\n            EmailComparator::ThreadId => \"threadId\",\n            EmailComparator::HasKeyword(_) => \"hasKeyword\",\n            EmailComparator::AllInThreadHaveKeyword(_) => \"allInThreadHaveKeyword\",\n            EmailComparator::SomeInThreadHaveKeyword(_) => \"someInThreadHaveKeyword\",\n            EmailComparator::_T(v) => v.as_str(),\n        }\n    }\n}\n\nimpl Serialize for EmailComparator {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.as_str())\n    }\n}\n\nimpl Filter<EmailFilter> {\n    pub fn is_immutable(&self) -> bool {\n        match self {\n            Filter::Property(f) => f.is_immutable(),\n            Filter::And | Filter::Or | Filter::Not | Filter::Close => true,\n        }\n    }\n}\n\nimpl EmailFilter {\n    pub fn is_immutable(&self) -> bool {\n        matches!(\n            self,\n            EmailFilter::Before(_)\n                | EmailFilter::After(_)\n                | EmailFilter::MinSize(_)\n                | EmailFilter::MaxSize(_)\n                | EmailFilter::HasAttachment(_)\n                | EmailFilter::From(_)\n                | EmailFilter::To(_)\n                | EmailFilter::Cc(_)\n                | EmailFilter::Bcc(_)\n                | EmailFilter::Subject(_)\n                | EmailFilter::Body(_)\n                | EmailFilter::Header(_)\n                | EmailFilter::Text(_)\n                | EmailFilter::Id(_)\n                | EmailFilter::SentBefore(_)\n                | EmailFilter::SentAfter(_)\n        )\n    }\n}\n\nimpl Comparator<EmailComparator> {\n    pub fn is_immutable(&self) -> bool {\n        self.property.is_immutable()\n    }\n}\n\nimpl EmailComparator {\n    pub fn is_immutable(&self) -> bool {\n        matches!(\n            self,\n            EmailComparator::ReceivedAt\n                | EmailComparator::Size\n                | EmailComparator::From\n                | EmailComparator::To\n                | EmailComparator::Subject\n                | EmailComparator::Cc\n                | EmailComparator::SentAt\n        )\n    }\n}\n\nimpl JmapObjectId for EmailValue {\n    fn as_id(&self) -> Option<Id> {\n        if let EmailValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            EmailValue::Id(id) => Some(AnyId::Id(*id)),\n            EmailValue::BlobId(id) => Some(AnyId::BlobId(id.clone())),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let EmailValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        match new_id {\n            AnyId::Id(id) => {\n                *self = EmailValue::Id(id);\n            }\n            AnyId::BlobId(id) => {\n                *self = EmailValue::BlobId(id);\n            }\n        }\n        true\n    }\n}\n\nimpl From<Id> for EmailValue {\n    fn from(id: Id) -> Self {\n        EmailValue::Id(id)\n    }\n}\n\nimpl From<BlobId> for EmailValue {\n    fn from(id: BlobId) -> Self {\n        EmailValue::BlobId(id)\n    }\n}\n\nimpl From<UTCDate> for EmailValue {\n    fn from(date: UTCDate) -> Self {\n        EmailValue::Date(date)\n    }\n}\n\nimpl JmapObjectId for EmailProperty {\n    fn as_id(&self) -> Option<Id> {\n        if let EmailProperty::IdValue(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let EmailProperty::IdValue(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        match self {\n            EmailProperty::IdReference(r) => Some(r),\n            EmailProperty::Pointer(value) => {\n                let value = value.as_slice();\n                match (value.first(), value.get(1)) {\n                    (\n                        Some(JsonPointerItem::Key(Key::Property(EmailProperty::MailboxIds))),\n                        Some(JsonPointerItem::Key(Key::Property(EmailProperty::IdReference(r)))),\n                    ) => Some(r),\n                    _ => None,\n                }\n            }\n            _ => None,\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            if let EmailProperty::Pointer(value) = self {\n                let value = value.as_mut_slice();\n                if let Some(value) = value.get_mut(1) {\n                    *value = JsonPointerItem::Key(Key::Property(EmailProperty::IdValue(id)));\n                    return true;\n                }\n            } else {\n                *self = EmailProperty::IdValue(id);\n                return true;\n            }\n        }\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/email_submission.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{\n        AnyId, JmapObject, JmapObjectId, MaybeReference,\n        email::{EmailProperty, EmailValue},\n        parse_ref,\n    },\n    request::{MaybeInvalid, deserialize::DeserializeArguments, reference::MaybeIdReference},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem, Key, Property, Value};\nuse std::{borrow::Cow, str::FromStr};\nuse types::{blob::BlobId, id::Id};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone, Default)]\npub struct EmailSubmission;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum EmailSubmissionProperty {\n    Id,\n    IdentityId,\n    ThreadId,\n    EmailId,\n    Envelope,\n    MailFrom,\n    RcptTo,\n    Email,\n    Parameters,\n    SendAt,\n    UndoStatus,\n    DeliveryStatus,\n    SmtpReply,\n    Delivered,\n    Displayed,\n    DsnBlobIds,\n    MdnBlobIds,\n\n    Pointer(JsonPointer<EmailSubmissionProperty>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum EmailSubmissionValue {\n    Id(Id),\n    Date(UTCDate),\n    BlobId(BlobId),\n    UndoStatus(UndoStatus),\n    Delivered(Delivered),\n    Displayed(Displayed),\n    IdReference(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum UndoStatus {\n    Pending,\n    Final,\n    Canceled,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum Delivered {\n    Queued,\n    Yes,\n    No,\n    Unknown,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum Displayed {\n    Yes,\n    Unknown,\n}\n\nimpl Property for EmailSubmissionProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        EmailSubmissionProperty::parse(value, key.is_none())\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            EmailSubmissionProperty::DeliveryStatus => \"deliveryStatus\",\n            EmailSubmissionProperty::DsnBlobIds => \"dsnBlobIds\",\n            EmailSubmissionProperty::Email => \"email\",\n            EmailSubmissionProperty::Envelope => \"envelope\",\n            EmailSubmissionProperty::Id => \"id\",\n            EmailSubmissionProperty::IdentityId => \"identityId\",\n            EmailSubmissionProperty::MdnBlobIds => \"mdnBlobIds\",\n            EmailSubmissionProperty::SendAt => \"sendAt\",\n            EmailSubmissionProperty::ThreadId => \"threadId\",\n            EmailSubmissionProperty::UndoStatus => \"undoStatus\",\n            EmailSubmissionProperty::Parameters => \"parameters\",\n            EmailSubmissionProperty::SmtpReply => \"smtpReply\",\n            EmailSubmissionProperty::Delivered => \"delivered\",\n            EmailSubmissionProperty::Displayed => \"displayed\",\n            EmailSubmissionProperty::MailFrom => \"mailFrom\",\n            EmailSubmissionProperty::RcptTo => \"rcptTo\",\n            EmailSubmissionProperty::EmailId => \"emailId\",\n            EmailSubmissionProperty::Pointer(json_pointer) => {\n                return json_pointer.to_string().into();\n            }\n        }\n        .into()\n    }\n}\n\nimpl Element for EmailSubmissionValue {\n    type Property = EmailSubmissionProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                EmailSubmissionProperty::Id\n                | EmailSubmissionProperty::ThreadId\n                | EmailSubmissionProperty::IdentityId\n                | EmailSubmissionProperty::EmailId => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(EmailSubmissionValue::Id(v)),\n                    MaybeReference::Reference(v) => Some(EmailSubmissionValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                EmailSubmissionProperty::MdnBlobIds | EmailSubmissionProperty::DsnBlobIds => {\n                    match parse_ref(value) {\n                        MaybeReference::Value(v) => Some(EmailSubmissionValue::BlobId(v)),\n                        MaybeReference::Reference(v) => Some(EmailSubmissionValue::IdReference(v)),\n                        MaybeReference::ParseError => None,\n                    }\n                }\n                EmailSubmissionProperty::SendAt => UTCDate::from_str(value)\n                    .ok()\n                    .map(EmailSubmissionValue::Date),\n                EmailSubmissionProperty::UndoStatus => {\n                    UndoStatus::parse(value).map(EmailSubmissionValue::UndoStatus)\n                }\n                EmailSubmissionProperty::Delivered => {\n                    Delivered::parse(value).map(EmailSubmissionValue::Delivered)\n                }\n                EmailSubmissionProperty::Displayed => {\n                    Displayed::parse(value).map(EmailSubmissionValue::Displayed)\n                }\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            EmailSubmissionValue::Id(id) => id.to_string().into(),\n            EmailSubmissionValue::Date(utcdate) => utcdate.to_string().into(),\n            EmailSubmissionValue::BlobId(blob_id) => blob_id.to_string().into(),\n            EmailSubmissionValue::IdReference(r) => format!(\"#{r}\").into(),\n            EmailSubmissionValue::UndoStatus(undo_status) => undo_status.as_str().into(),\n            EmailSubmissionValue::Delivered(delivered) => delivered.as_str().into(),\n            EmailSubmissionValue::Displayed(displayed) => displayed.as_str().into(),\n        }\n    }\n}\n\nimpl EmailSubmissionProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            \"id\" => EmailSubmissionProperty::Id,\n            \"identityId\" => EmailSubmissionProperty::IdentityId,\n            \"threadId\" => EmailSubmissionProperty::ThreadId,\n            \"emailId\" => EmailSubmissionProperty::EmailId,\n            \"envelope\" => EmailSubmissionProperty::Envelope,\n            \"mailFrom\" => EmailSubmissionProperty::MailFrom,\n            \"rcptTo\" => EmailSubmissionProperty::RcptTo,\n            \"email\" => EmailSubmissionProperty::Email,\n            \"parameters\" => EmailSubmissionProperty::Parameters,\n            \"sendAt\" => EmailSubmissionProperty::SendAt,\n            \"undoStatus\" => EmailSubmissionProperty::UndoStatus,\n            \"deliveryStatus\" => EmailSubmissionProperty::DeliveryStatus,\n            \"smtpReply\" => EmailSubmissionProperty::SmtpReply,\n            \"delivered\" => EmailSubmissionProperty::Delivered,\n            \"displayed\" => EmailSubmissionProperty::Displayed,\n            \"dsnBlobIds\" => EmailSubmissionProperty::DsnBlobIds,\n            \"mdnBlobIds\" => EmailSubmissionProperty::MdnBlobIds,\n        )\n        .or_else(|| {\n            if allow_patch && value.contains('/') {\n                EmailSubmissionProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &EmailSubmissionProperty {\n        if let EmailSubmissionProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n}\n\nimpl UndoStatus {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"pending\" => UndoStatus::Pending,\n            b\"final\" => UndoStatus::Final,\n            b\"canceled\" => UndoStatus::Canceled,\n        )\n    }\n\n    fn as_str(&self) -> &'static str {\n        match self {\n            UndoStatus::Pending => \"pending\",\n            UndoStatus::Final => \"final\",\n            UndoStatus::Canceled => \"canceled\",\n        }\n    }\n}\n\nimpl Delivered {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"queued\" => Delivered::Queued,\n            b\"yes\" => Delivered::Yes,\n            b\"no\" => Delivered::No,\n            b\"unknown\" => Delivered::Unknown,\n        )\n    }\n\n    fn as_str(&self) -> &'static str {\n        match self {\n            Delivered::Queued => \"queued\",\n            Delivered::Yes => \"yes\",\n            Delivered::No => \"no\",\n            Delivered::Unknown => \"unknown\",\n        }\n    }\n}\n\nimpl Displayed {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"yes\" => Displayed::Yes,\n            b\"unknown\" => Displayed::Unknown,\n        )\n    }\n\n    fn as_str(&self) -> &'static str {\n        match self {\n            Displayed::Yes => \"yes\",\n            Displayed::Unknown => \"unknown\",\n        }\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct EmailSubmissionSetArguments<'x> {\n    pub on_success_update_email:\n        Option<VecMap<MaybeIdReference<Id>, Value<'x, EmailProperty, EmailValue>>>,\n    pub on_success_destroy_email: Option<Vec<MaybeIdReference<Id>>>,\n}\n\nimpl<'x> DeserializeArguments<'x> for EmailSubmissionSetArguments<'x> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'x>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"onSuccessUpdateEmail\" => {\n                self.on_success_update_email = map.next_value()?;\n            },\n            b\"onSuccessDestroyEmail\" => {\n                self.on_success_destroy_email = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl FromStr for EmailSubmissionProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        EmailSubmissionProperty::parse(s, false).ok_or(())\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for UndoStatus {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        UndoStatus::parse(<&str>::deserialize(deserializer)?)\n            .ok_or_else(|| serde::de::Error::custom(\"invalid JMAP UndoStatus\"))\n    }\n}\n\nimpl JmapObject for EmailSubmission {\n    type Property = EmailSubmissionProperty;\n\n    type Element = EmailSubmissionValue;\n\n    type Id = Id;\n\n    type Filter = EmailSubmissionFilter;\n\n    type Comparator = EmailSubmissionComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = EmailSubmissionSetArguments<'de>;\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = EmailSubmissionProperty::Id;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum EmailSubmissionFilter {\n    IdentityIds(Vec<MaybeInvalid<Id>>),\n    EmailIds(Vec<MaybeInvalid<Id>>),\n    ThreadIds(Vec<MaybeInvalid<Id>>),\n    Before(UTCDate),\n    After(UTCDate),\n    UndoStatus(UndoStatus),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum EmailSubmissionComparator {\n    EmailId,\n    ThreadId,\n    SentAt,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for EmailSubmissionFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"identityIds\" => {\n                *self = EmailSubmissionFilter::IdentityIds(map.next_value()?);\n            },\n            b\"emailIds\" => {\n                *self = EmailSubmissionFilter::EmailIds(map.next_value()?);\n            },\n            b\"threadIds\" => {\n                *self = EmailSubmissionFilter::ThreadIds(map.next_value()?);\n            },\n            b\"before\" => {\n                *self = EmailSubmissionFilter::Before(map.next_value()?);\n            },\n            b\"after\" => {\n                *self = EmailSubmissionFilter::After(map.next_value()?);\n            },\n            b\"undoStatus\" => {\n                *self = EmailSubmissionFilter::UndoStatus(map.next_value()?);\n            },\n            _ => {\n                *self = EmailSubmissionFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for EmailSubmissionComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n\n                b\"emailId\" => {\n                    *self = EmailSubmissionComparator::EmailId;\n                },\n                b\"threadId\" => {\n                    *self = EmailSubmissionComparator::ThreadId;\n                },\n                b\"sentAt\" => {\n                    *self = EmailSubmissionComparator::SentAt;\n                },\n                _ => {\n                    *self = EmailSubmissionComparator::_T(key.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Default for EmailSubmissionFilter {\n    fn default() -> Self {\n        EmailSubmissionFilter::_T(\"\".to_string())\n    }\n}\n\nimpl Default for EmailSubmissionComparator {\n    fn default() -> Self {\n        EmailSubmissionComparator::_T(\"\".to_string())\n    }\n}\n\nimpl From<Id> for EmailSubmissionValue {\n    fn from(id: Id) -> Self {\n        EmailSubmissionValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for EmailSubmissionValue {\n    fn as_id(&self) -> Option<Id> {\n        match self {\n            EmailSubmissionValue::Id(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            EmailSubmissionValue::Id(id) => Some(AnyId::Id(*id)),\n            EmailSubmissionValue::BlobId(blob_id) => Some(AnyId::BlobId(blob_id.clone())),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let EmailSubmissionValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        match new_id {\n            AnyId::Id(id) => {\n                *self = EmailSubmissionValue::Id(id);\n            }\n            AnyId::BlobId(blob_id) => {\n                *self = EmailSubmissionValue::BlobId(blob_id);\n            }\n        }\n        true\n    }\n}\n\nimpl JmapObjectId for EmailSubmissionProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/file_node.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{\n        AnyId, JmapObject, JmapObjectId, JmapRight, JmapSharedObject, MaybeReference, parse_ref,\n    },\n    request::{MaybeInvalid, deserialize::DeserializeArguments},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem, Key, Property};\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\nuse types::{acl::Acl, blob::BlobId, id::Id};\nuse utils::glob::GlobPattern;\n\n#[derive(Debug, Clone, Default)]\npub struct FileNode;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum FileNodeProperty {\n    Id,\n    ParentId,\n    BlobId,\n    Size,\n    Name,\n    Type,\n    Created,\n    Modified,\n    Accessed,\n    Executable,\n    MyRights,\n    ShareWith,\n    IsSubscribed,\n\n    // Other\n    IdValue(Id),\n    Rights(FileNodeRight),\n    Pointer(JsonPointer<FileNodeProperty>),\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum FileNodeRight {\n    MayRead,\n    MayWrite,\n    MayShare,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum FileNodeValue {\n    Id(Id),\n    Date(UTCDate),\n    BlobId(BlobId),\n    IdReference(String),\n}\n\nimpl Property for FileNodeProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        let allow_patch = key.is_none();\n        if let Some(Key::Property(key)) = key {\n            match key.patch_or_prop() {\n                FileNodeProperty::ShareWith => {\n                    Id::from_str(value).ok().map(FileNodeProperty::IdValue)\n                }\n                _ => FileNodeProperty::parse(value, allow_patch),\n            }\n        } else {\n            FileNodeProperty::parse(value, allow_patch)\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            FileNodeProperty::Id => \"id\",\n            FileNodeProperty::ParentId => \"parentId\",\n            FileNodeProperty::BlobId => \"blobId\",\n            FileNodeProperty::Size => \"size\",\n            FileNodeProperty::Name => \"name\",\n            FileNodeProperty::Type => \"type\",\n            FileNodeProperty::Created => \"created\",\n            FileNodeProperty::Modified => \"modified\",\n            FileNodeProperty::Accessed => \"accessed\",\n            FileNodeProperty::Executable => \"executable\",\n            FileNodeProperty::MyRights => \"myRights\",\n            FileNodeProperty::ShareWith => \"shareWith\",\n            FileNodeProperty::IsSubscribed => \"isSubscribed\",\n            FileNodeProperty::Rights(file_right) => file_right.as_str(),\n            FileNodeProperty::Pointer(json_pointer) => return json_pointer.to_string().into(),\n            FileNodeProperty::IdValue(id) => return id.to_string().into(),\n        }\n        .into()\n    }\n}\n\nimpl FileNodeRight {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            FileNodeRight::MayRead => \"mayRead\",\n            FileNodeRight::MayWrite => \"mayWrite\",\n            FileNodeRight::MayShare => \"mayShare\",\n        }\n    }\n}\n\nimpl Element for FileNodeValue {\n    type Property = FileNodeProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                FileNodeProperty::Id | FileNodeProperty::ParentId => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(FileNodeValue::Id(v)),\n                    MaybeReference::Reference(v) => Some(FileNodeValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                FileNodeProperty::BlobId => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(FileNodeValue::BlobId(v)),\n                    MaybeReference::Reference(v) => Some(FileNodeValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                FileNodeProperty::Created\n                | FileNodeProperty::Modified\n                | FileNodeProperty::Accessed => {\n                    UTCDate::from_str(value).ok().map(FileNodeValue::Date)\n                }\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            FileNodeValue::Id(id) => id.to_string().into(),\n            FileNodeValue::Date(utcdate) => utcdate.to_string().into(),\n            FileNodeValue::BlobId(blob_id) => blob_id.to_string().into(),\n            FileNodeValue::IdReference(r) => format!(\"#{r}\").into(),\n        }\n    }\n}\n\nimpl FileNodeProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => FileNodeProperty::Id,\n            b\"parentId\" => FileNodeProperty::ParentId,\n            b\"blobId\" => FileNodeProperty::BlobId,\n            b\"size\" => FileNodeProperty::Size,\n            b\"name\" => FileNodeProperty::Name,\n            b\"type\" => FileNodeProperty::Type,\n            b\"created\" => FileNodeProperty::Created,\n            b\"modified\" => FileNodeProperty::Modified,\n            b\"accessed\" => FileNodeProperty::Accessed,\n            b\"executable\" => FileNodeProperty::Executable,\n            b\"myRights\" => FileNodeProperty::MyRights,\n            b\"shareWith\" => FileNodeProperty::ShareWith,\n            b\"isSubscribed\" => FileNodeProperty::IsSubscribed,\n            b\"mayRead\" => FileNodeProperty::Rights(FileNodeRight::MayRead),\n            b\"mayWrite\" => FileNodeProperty::Rights(FileNodeRight::MayWrite),\n            b\"mayShare\" => FileNodeProperty::Rights(FileNodeRight::MayShare),\n        )\n        .or_else(|| {\n            if allow_patch && value.contains('/') {\n                FileNodeProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &FileNodeProperty {\n        if let FileNodeProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct FileNodeSetArguments {\n    pub on_destroy_remove_children: Option<bool>,\n}\n\nimpl<'x> DeserializeArguments<'x> for FileNodeSetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'x>,\n    {\n        if key == \"onDestroyRemoveChildren\" {\n            self.on_destroy_remove_children = map.next_value()?;\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct FileNodeQueryArguments {\n    pub depth: Option<u32>,\n}\n\nimpl<'x> DeserializeArguments<'x> for FileNodeQueryArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'x>,\n    {\n        if key == \"depth\" {\n            self.depth = map.next_value()?;\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl FromStr for FileNodeProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        FileNodeProperty::parse(s, false).ok_or(())\n    }\n}\n\nimpl JmapObject for FileNode {\n    type Property = FileNodeProperty;\n\n    type Element = FileNodeValue;\n\n    type Id = Id;\n\n    type Filter = FileNodeFilter;\n\n    type Comparator = FileNodeComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = FileNodeSetArguments;\n\n    type QueryArguments = FileNodeQueryArguments;\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = FileNodeProperty::Id;\n}\n\nimpl JmapSharedObject for FileNode {\n    type Right = FileNodeRight;\n\n    const SHARE_WITH_PROPERTY: Self::Property = FileNodeProperty::ShareWith;\n}\n\nimpl From<Id> for FileNodeProperty {\n    fn from(id: Id) -> Self {\n        FileNodeProperty::IdValue(id)\n    }\n}\n\nimpl JmapRight for FileNodeRight {\n    fn to_acl(&self) -> &'static [Acl] {\n        match self {\n            FileNodeRight::MayShare => &[Acl::Share],\n            FileNodeRight::MayRead => &[Acl::Read, Acl::ReadItems],\n            FileNodeRight::MayWrite => &[\n                Acl::Modify,\n                Acl::AddItems,\n                Acl::ModifyItems,\n                Acl::Delete,\n                Acl::RemoveItems,\n            ],\n        }\n    }\n\n    fn all_rights() -> &'static [Self] {\n        &[\n            FileNodeRight::MayRead,\n            FileNodeRight::MayWrite,\n            FileNodeRight::MayShare,\n        ]\n    }\n}\n\nimpl From<FileNodeRight> for FileNodeProperty {\n    fn from(right: FileNodeRight) -> Self {\n        FileNodeProperty::Rights(right)\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum FileNodeFilter {\n    HasParentId(bool),\n    ParentId(MaybeInvalid<Id>),\n    AncestorId(MaybeInvalid<Id>),\n    HasType(bool),\n    BlobId(MaybeInvalid<BlobId>),\n    IsExecutable(bool),\n    CreatedBefore(UTCDate),\n    CreatedAfter(UTCDate),\n    ModifiedBefore(UTCDate),\n    ModifiedAfter(UTCDate),\n    AccessedBefore(UTCDate),\n    AccessedAfter(UTCDate),\n    MinSize(u64),\n    MaxSize(u64),\n    Name(String),\n    NameMatch(GlobPattern),\n    Type(String),\n    TypeMatch(GlobPattern),\n    Text(String),\n    Body(String),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum FileNodeComparator {\n    Name,\n    Size,\n    Created,\n    Modified,\n    Type,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for FileNodeFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"hasParentId\" => {\n                *self = FileNodeFilter::HasParentId(map.next_value()?);\n            },\n            b\"parentId\" => {\n                *self = FileNodeFilter::ParentId(map.next_value()?);\n            },\n            b\"ancestorId\" => {\n                *self = FileNodeFilter::AncestorId(map.next_value()?);\n            },\n            b\"hasType\" => {\n                *self = FileNodeFilter::HasType(map.next_value()?);\n            },\n            b\"blobId\" => {\n                *self = FileNodeFilter::BlobId(map.next_value()?);\n            },\n            b\"isExecutable\" => {\n                *self = FileNodeFilter::IsExecutable(map.next_value()?);\n            },\n            b\"createdBefore\" => {\n                *self = FileNodeFilter::CreatedBefore(map.next_value()?);\n            },\n            b\"createdAfter\" => {\n                *self = FileNodeFilter::CreatedAfter(map.next_value()?);\n            },\n            b\"modifiedBefore\" => {\n                *self = FileNodeFilter::ModifiedBefore(map.next_value()?);\n            },\n            b\"modifiedAfter\" => {\n                *self = FileNodeFilter::ModifiedAfter(map.next_value()?);\n            },\n            b\"accessedBefore\" => {\n                *self = FileNodeFilter::AccessedBefore(map.next_value()?);\n            },\n            b\"accessedAfter\" => {\n                *self = FileNodeFilter::AccessedAfter(map.next_value()?);\n            },\n            b\"minSize\" => {\n                *self = FileNodeFilter::MinSize(map.next_value()?);\n            },\n            b\"maxSize\" => {\n                *self = FileNodeFilter::MaxSize(map.next_value()?);\n            },\n            b\"name\" => {\n                *self = FileNodeFilter::Name(map.next_value()?);\n            },\n            b\"nameMatch\" => {\n                *self = FileNodeFilter::NameMatch(map.next_value()?);\n            },\n            b\"type\" => {\n                *self = FileNodeFilter::Type(map.next_value()?);\n            },\n            b\"typeMatch\" => {\n                *self = FileNodeFilter::TypeMatch(map.next_value()?);\n            },\n            b\"body\" => {\n                *self = FileNodeFilter::Body(map.next_value()?);\n            },\n            b\"text\" => {\n                *self = FileNodeFilter::Text(map.next_value()?);\n            },\n            _ => {\n                *self = FileNodeFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for FileNodeComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"name\" => {\n                    *self = FileNodeComparator::Name;\n                },\n                b\"size\" => {\n                    *self = FileNodeComparator::Size;\n                },\n                b\"created\" => {\n                    *self = FileNodeComparator::Created;\n                },\n                b\"modified\" => {\n                    *self = FileNodeComparator::Modified;\n                },\n                b\"type\" => {\n                    *self = FileNodeComparator::Type;\n                },\n                _ => {\n                    *self = FileNodeComparator::_T(key.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Default for FileNodeFilter {\n    fn default() -> Self {\n        FileNodeFilter::_T(\"\".to_string())\n    }\n}\n\nimpl Default for FileNodeComparator {\n    fn default() -> Self {\n        FileNodeComparator::_T(\"\".to_string())\n    }\n}\n\nimpl From<Id> for FileNodeValue {\n    fn from(id: Id) -> Self {\n        FileNodeValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for FileNodeValue {\n    fn as_id(&self) -> Option<Id> {\n        match self {\n            FileNodeValue::Id(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            FileNodeValue::Id(id) => Some(AnyId::Id(*id)),\n            FileNodeValue::BlobId(blob_id) => Some(AnyId::BlobId(blob_id.clone())),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let FileNodeValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        match new_id {\n            AnyId::Id(id) => {\n                *self = FileNodeValue::Id(id);\n            }\n            AnyId::BlobId(blob_id) => {\n                *self = FileNodeValue::BlobId(blob_id);\n            }\n        }\n        true\n    }\n}\n\nimpl FileNodeFilter {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            FileNodeFilter::HasParentId(_) => \"hasParentId\",\n            FileNodeFilter::ParentId(_) => \"parentId\",\n            FileNodeFilter::AncestorId(_) => \"ancestorId\",\n            FileNodeFilter::HasType(_) => \"hasType\",\n            FileNodeFilter::BlobId(_) => \"blobId\",\n            FileNodeFilter::IsExecutable(_) => \"isExecutable\",\n            FileNodeFilter::CreatedBefore(_) => \"createdBefore\",\n            FileNodeFilter::CreatedAfter(_) => \"createdAfter\",\n            FileNodeFilter::ModifiedBefore(_) => \"modifiedBefore\",\n            FileNodeFilter::ModifiedAfter(_) => \"modifiedAfter\",\n            FileNodeFilter::AccessedBefore(_) => \"accessedBefore\",\n            FileNodeFilter::AccessedAfter(_) => \"accessedAfter\",\n            FileNodeFilter::MinSize(_) => \"minSize\",\n            FileNodeFilter::MaxSize(_) => \"maxSize\",\n            FileNodeFilter::Name(_) => \"name\",\n            FileNodeFilter::NameMatch(_) => \"nameMatch\",\n            FileNodeFilter::Type(_) => \"type\",\n            FileNodeFilter::TypeMatch(_) => \"typeMatch\",\n            FileNodeFilter::Text(_) => \"text\",\n            FileNodeFilter::Body(_) => \"body\",\n            FileNodeFilter::_T(s) => return s.into(),\n        }\n        .into()\n    }\n}\n\nimpl FileNodeComparator {\n    pub fn as_str(&self) -> &str {\n        match self {\n            FileNodeComparator::Name => \"name\",\n            FileNodeComparator::Size => \"size\",\n            FileNodeComparator::Created => \"created\",\n            FileNodeComparator::Modified => \"modified\",\n            FileNodeComparator::Type => \"type\",\n            FileNodeComparator::_T(s) => s.as_ref(),\n        }\n    }\n\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            FileNodeComparator::Name => \"name\",\n            FileNodeComparator::Size => \"size\",\n            FileNodeComparator::Created => \"created\",\n            FileNodeComparator::Modified => \"modified\",\n            FileNodeComparator::Type => \"type\",\n            FileNodeComparator::_T(s) => return s.into(),\n        }\n        .into()\n    }\n}\n\nimpl serde::Serialize for FileNodeComparator {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.as_str())\n    }\n}\n\nimpl TryFrom<FileNodeProperty> for Id {\n    type Error = ();\n\n    fn try_from(value: FileNodeProperty) -> Result<Self, Self::Error> {\n        if let FileNodeProperty::IdValue(id) = value {\n            Ok(id)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl TryFrom<FileNodeProperty> for FileNodeRight {\n    type Error = ();\n\n    fn try_from(value: FileNodeProperty) -> Result<Self, Self::Error> {\n        if let FileNodeProperty::Rights(right) = value {\n            Ok(right)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl JmapObjectId for FileNodeProperty {\n    fn as_id(&self) -> Option<Id> {\n        if let FileNodeProperty::IdValue(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let FileNodeProperty::IdValue(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = FileNodeProperty::IdValue(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl Display for FileNodeProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"{}\", self.to_cow())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/identity.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::object::{AnyId, JmapObject, JmapObjectId};\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::id::Id;\n\n#[derive(Debug, Clone, Default)]\npub struct Identity;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum IdentityProperty {\n    Id,\n    Name,\n    Email,\n    ReplyTo,\n    Bcc,\n    TextSignature,\n    HtmlSignature,\n    MayDelete,\n\n    // Other\n    Pointer(JsonPointer<IdentityProperty>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum IdentityValue {\n    Id(Id),\n}\n\nimpl Property for IdentityProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        IdentityProperty::parse(value, key.is_none())\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            IdentityProperty::Bcc => \"bcc\",\n            IdentityProperty::Email => \"email\",\n            IdentityProperty::HtmlSignature => \"htmlSignature\",\n            IdentityProperty::Id => \"id\",\n            IdentityProperty::MayDelete => \"mayDelete\",\n            IdentityProperty::Name => \"name\",\n            IdentityProperty::ReplyTo => \"replyTo\",\n            IdentityProperty::TextSignature => \"textSignature\",\n            IdentityProperty::Pointer(json_pointer) => return json_pointer.to_string().into(),\n        }\n        .into()\n    }\n}\n\nimpl Element for IdentityValue {\n    type Property = IdentityProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                IdentityProperty::Id => Id::from_str(value).ok().map(IdentityValue::Id),\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            IdentityValue::Id(id) => id.to_string().into(),\n        }\n    }\n}\n\nimpl IdentityProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => IdentityProperty::Id,\n            b\"name\" => IdentityProperty::Name,\n            b\"email\" => IdentityProperty::Email,\n            b\"replyTo\" => IdentityProperty::ReplyTo,\n            b\"bcc\" => IdentityProperty::Bcc,\n            b\"textSignature\" => IdentityProperty::TextSignature,\n            b\"htmlSignature\" => IdentityProperty::HtmlSignature,\n            b\"mayDelete\" => IdentityProperty::MayDelete,\n        )\n        .or_else(|| {\n            if allow_patch && value.contains('/') {\n                IdentityProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &IdentityProperty {\n        if let IdentityProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n}\n\nimpl FromStr for IdentityProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        IdentityProperty::parse(s, false).ok_or(())\n    }\n}\n\nimpl JmapObject for Identity {\n    type Property = IdentityProperty;\n\n    type Element = IdentityValue;\n\n    type Id = Id;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = IdentityProperty::Id;\n}\n\nimpl From<Id> for IdentityValue {\n    fn from(id: Id) -> Self {\n        IdentityValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for IdentityValue {\n    fn as_id(&self) -> Option<Id> {\n        match self {\n            IdentityValue::Id(id) => Some(*id),\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            IdentityValue::Id(id) => Some(AnyId::Id(*id)),\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = IdentityValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl JmapObjectId for IdentityProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/mailbox.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{\n        AnyId, JmapObject, JmapObjectId, JmapRight, JmapSharedObject, MaybeReference, parse_ref,\n    },\n    request::{deserialize::DeserializeArguments, reference::MaybeIdReference},\n};\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::{acl::Acl, id::Id, special_use::SpecialUse};\n\n#[derive(Debug, Clone, Default)]\npub struct Mailbox;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum MailboxProperty {\n    Id,\n    Name,\n    ParentId,\n    Role,\n    SortOrder,\n    TotalEmails,\n    UnreadEmails,\n    TotalThreads,\n    UnreadThreads,\n    ShareWith,\n    MyRights,\n    IsSubscribed,\n\n    // Other\n    IdValue(Id),\n    Rights(MailboxRight),\n    Pointer(JsonPointer<MailboxProperty>),\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum MailboxRight {\n    MayReadItems,\n    MayAddItems,\n    MayRemoveItems,\n    MaySetSeen,\n    MaySetKeywords,\n    MayCreateChild,\n    MayRename,\n    MaySubmit,\n    MayDelete,\n    MayShare,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum MailboxValue {\n    Id(Id),\n    IdReference(String),\n    Role(SpecialUse),\n}\n\nimpl Property for MailboxProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        let allow_patch = key.is_none();\n        if let Some(Key::Property(key)) = key {\n            match key.patch_or_prop() {\n                MailboxProperty::ShareWith => {\n                    Id::from_str(value).ok().map(MailboxProperty::IdValue)\n                }\n                _ => MailboxProperty::parse(value, allow_patch),\n            }\n        } else {\n            MailboxProperty::parse(value, allow_patch)\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            MailboxProperty::Id => \"id\",\n            MailboxProperty::IsSubscribed => \"isSubscribed\",\n            MailboxProperty::MyRights => \"myRights\",\n            MailboxProperty::Name => \"name\",\n            MailboxProperty::ParentId => \"parentId\",\n            MailboxProperty::Role => \"role\",\n            MailboxProperty::SortOrder => \"sortOrder\",\n            MailboxProperty::TotalEmails => \"totalEmails\",\n            MailboxProperty::TotalThreads => \"totalThreads\",\n            MailboxProperty::UnreadEmails => \"unreadEmails\",\n            MailboxProperty::UnreadThreads => \"unreadThreads\",\n            MailboxProperty::ShareWith => \"shareWith\",\n            MailboxProperty::Rights(mailbox_right) => mailbox_right.as_str(),\n            MailboxProperty::Pointer(json_pointer) => return json_pointer.to_string().into(),\n            MailboxProperty::IdValue(id) => return id.to_string().into(),\n        }\n        .into()\n    }\n}\n\nimpl MailboxRight {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            MailboxRight::MayReadItems => \"mayReadItems\",\n            MailboxRight::MayAddItems => \"mayAddItems\",\n            MailboxRight::MayRemoveItems => \"mayRemoveItems\",\n            MailboxRight::MaySetSeen => \"maySetSeen\",\n            MailboxRight::MaySetKeywords => \"maySetKeywords\",\n            MailboxRight::MayCreateChild => \"mayCreateChild\",\n            MailboxRight::MayRename => \"mayRename\",\n            MailboxRight::MaySubmit => \"maySubmit\",\n            MailboxRight::MayDelete => \"mayDelete\",\n            MailboxRight::MayShare => \"mayShare\",\n        }\n    }\n}\n\nimpl Element for MailboxValue {\n    type Property = MailboxProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                MailboxProperty::Id | MailboxProperty::ParentId => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(MailboxValue::Id(v)),\n                    MaybeReference::Reference(v) => Some(MailboxValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                MailboxProperty::Role => SpecialUse::parse(value).map(MailboxValue::Role),\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            MailboxValue::Id(id) => id.to_string().into(),\n            MailboxValue::IdReference(r) => format!(\"#{r}\").into(),\n            MailboxValue::Role(special_use) => special_use.as_str().unwrap_or_default().into(),\n        }\n    }\n}\n\nimpl MailboxProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => MailboxProperty::Id,\n            b\"name\" => MailboxProperty::Name,\n            b\"parentId\" => MailboxProperty::ParentId,\n            b\"role\" => MailboxProperty::Role,\n            b\"sortOrder\" => MailboxProperty::SortOrder,\n            b\"totalEmails\" => MailboxProperty::TotalEmails,\n            b\"unreadEmails\" => MailboxProperty::UnreadEmails,\n            b\"totalThreads\" => MailboxProperty::TotalThreads,\n            b\"unreadThreads\" => MailboxProperty::UnreadThreads,\n            b\"shareWith\" => MailboxProperty::ShareWith,\n            b\"myRights\" => MailboxProperty::MyRights,\n            b\"mayReadItems\" => MailboxProperty::Rights(MailboxRight::MayReadItems),\n            b\"mayAddItems\" => MailboxProperty::Rights(MailboxRight::MayAddItems),\n            b\"mayRemoveItems\" => MailboxProperty::Rights(MailboxRight::MayRemoveItems),\n            b\"maySetSeen\" => MailboxProperty::Rights(MailboxRight::MaySetSeen),\n            b\"maySetKeywords\" => MailboxProperty::Rights(MailboxRight::MaySetKeywords),\n            b\"mayCreateChild\" => MailboxProperty::Rights(MailboxRight::MayCreateChild),\n            b\"mayRename\" => MailboxProperty::Rights(MailboxRight::MayRename),\n            b\"maySubmit\" => MailboxProperty::Rights(MailboxRight::MaySubmit),\n            b\"mayDelete\" => MailboxProperty::Rights(MailboxRight::MayDelete),\n            b\"mayShare\" => MailboxProperty::Rights(MailboxRight::MayShare),\n            b\"isSubscribed\" => MailboxProperty::IsSubscribed,\n        )\n        .or_else(|| {\n            if allow_patch && value.contains('/') {\n                MailboxProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &MailboxProperty {\n        if let MailboxProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct MailboxSetArguments {\n    pub on_destroy_remove_emails: Option<bool>,\n}\n\n#[derive(Debug, Clone, Default)]\npub struct MailboxQueryArguments {\n    pub sort_as_tree: Option<bool>,\n    pub filter_as_tree: Option<bool>,\n}\n\nimpl<'de> DeserializeArguments<'de> for MailboxSetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"onDestroyRemoveEmails\" {\n            self.on_destroy_remove_emails = map.next_value()?;\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for MailboxQueryArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"sortAsTree\" => {\n                self.sort_as_tree = map.next_value()?;\n            },\n            b\"filterAsTree\" => {\n                self.filter_as_tree = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl FromStr for MailboxProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        MailboxProperty::parse(s, false).ok_or(())\n    }\n}\n\nimpl JmapObject for Mailbox {\n    type Property = MailboxProperty;\n\n    type Element = MailboxValue;\n\n    type Id = Id;\n\n    type Filter = MailboxFilter;\n\n    type Comparator = MailboxComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = MailboxSetArguments;\n\n    type QueryArguments = MailboxQueryArguments;\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = MailboxProperty::Id;\n}\n\nimpl JmapSharedObject for Mailbox {\n    type Right = MailboxRight;\n\n    const SHARE_WITH_PROPERTY: Self::Property = MailboxProperty::ShareWith;\n}\n\nimpl From<Id> for MailboxProperty {\n    fn from(id: Id) -> Self {\n        MailboxProperty::IdValue(id)\n    }\n}\n\nimpl TryFrom<MailboxProperty> for Id {\n    type Error = ();\n\n    fn try_from(value: MailboxProperty) -> Result<Self, Self::Error> {\n        if let MailboxProperty::IdValue(id) = value {\n            Ok(id)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl TryFrom<MailboxProperty> for MailboxRight {\n    type Error = ();\n\n    fn try_from(value: MailboxProperty) -> Result<Self, Self::Error> {\n        if let MailboxProperty::Rights(right) = value {\n            Ok(right)\n        } else {\n            Err(())\n        }\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum MailboxFilter {\n    Name(String),\n    ParentId(Option<MaybeIdReference<Id>>),\n    Role(Option<SpecialUse>),\n    HasAnyRole(bool),\n    IsSubscribed(bool),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum MailboxComparator {\n    SortOrder,\n    Name,\n    ParentId,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for MailboxFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"name\" => {\n                *self = MailboxFilter::Name(map.next_value()?);\n            },\n            b\"parentId\" => {\n                *self = MailboxFilter::ParentId(map.next_value()?);\n            },\n            b\"role\" => {\n                *self = MailboxFilter::Role(map.next_value::<Option<RoleWrapper>>()?.map(|r| r.0));\n            },\n            b\"hasAnyRole\" => {\n                *self = MailboxFilter::HasAnyRole(map.next_value()?);\n            },\n            b\"isSubscribed\" => {\n                *self = MailboxFilter::IsSubscribed(map.next_value()?);\n            },\n            _ => {\n                *self = MailboxFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for MailboxComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"sortOrder\" => {\n                    *self = MailboxComparator::SortOrder;\n                },\n                b\"name\" => {\n                    *self = MailboxComparator::Name;\n                },\n                b\"parentId\" => {\n                    *self = MailboxComparator::ParentId;\n                },\n                _ => {\n                    *self = MailboxComparator::_T(key.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Default for MailboxFilter {\n    fn default() -> Self {\n        MailboxFilter::_T(\"\".to_string())\n    }\n}\n\nimpl Default for MailboxComparator {\n    fn default() -> Self {\n        MailboxComparator::_T(\"\".to_string())\n    }\n}\n\nstruct RoleWrapper(SpecialUse);\n\nimpl<'de> serde::Deserialize<'de> for RoleWrapper {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        SpecialUse::parse(<&str>::deserialize(deserializer)?)\n            .map(RoleWrapper)\n            .ok_or_else(|| serde::de::Error::custom(\"invalid JMAP role\"))\n    }\n}\n\nimpl From<Id> for MailboxValue {\n    fn from(id: Id) -> Self {\n        MailboxValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for MailboxValue {\n    fn as_id(&self) -> Option<Id> {\n        if let MailboxValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let MailboxValue::Id(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let MailboxValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = MailboxValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl JmapRight for MailboxRight {\n    fn to_acl(&self) -> &'static [Acl] {\n        match self {\n            MailboxRight::MayReadItems => &[Acl::Read, Acl::ReadItems],\n            MailboxRight::MayAddItems => &[Acl::AddItems],\n            MailboxRight::MayRemoveItems => &[Acl::RemoveItems],\n            MailboxRight::MaySetSeen => &[Acl::ModifyItems],\n            MailboxRight::MaySetKeywords => &[Acl::ModifyItems],\n            MailboxRight::MayCreateChild => &[Acl::CreateChild],\n            MailboxRight::MayRename => &[Acl::Modify],\n            MailboxRight::MaySubmit => &[Acl::Submit],\n            MailboxRight::MayDelete => &[Acl::Delete],\n            MailboxRight::MayShare => &[Acl::Share],\n        }\n    }\n\n    fn all_rights() -> &'static [Self] {\n        &[\n            MailboxRight::MayReadItems,\n            MailboxRight::MayAddItems,\n            MailboxRight::MayRemoveItems,\n            MailboxRight::MaySetSeen,\n            MailboxRight::MaySetKeywords,\n            MailboxRight::MayCreateChild,\n            MailboxRight::MayRename,\n            MailboxRight::MaySubmit,\n            MailboxRight::MayDelete,\n            MailboxRight::MayShare,\n        ]\n    }\n}\n\nimpl From<MailboxRight> for MailboxProperty {\n    fn from(right: MailboxRight) -> Self {\n        MailboxProperty::Rights(right)\n    }\n}\n\nimpl JmapObjectId for MailboxProperty {\n    fn as_id(&self) -> Option<Id> {\n        if let MailboxProperty::IdValue(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let MailboxProperty::IdValue(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = MailboxProperty::IdValue(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::request::deserialize::DeserializeArguments;\nuse jmap_tools::{Element, Null, Property};\nuse serde::Serialize;\nuse std::{fmt::Debug, str::FromStr};\nuse types::{acl::Acl, blob::BlobId, id::Id};\n\npub mod addressbook;\npub mod blob;\npub mod calendar;\npub mod calendar_event;\npub mod calendar_event_notification;\npub mod contact;\npub mod email;\npub mod email_submission;\npub mod file_node;\npub mod identity;\npub mod mailbox;\npub mod participant_identity;\npub mod principal;\npub mod push_subscription;\npub mod quota;\npub mod search_snippet;\npub mod share_notification;\npub mod sieve;\npub mod thread;\npub mod vacation_response;\n\npub trait JmapObject: std::fmt::Debug {\n    type Property: Property + JmapObjectId + FromStr + Debug + Sync + Send;\n    type Element: Element<Property = Self::Property> + JmapObjectId + Debug + Sync + Send;\n    type Id: FromStr + TryFrom<AnyId> + Into<Self::Element> + Serialize + Debug + Sync + Send;\n\n    type Filter: Default + for<'de> DeserializeArguments<'de> + Debug + Sync + Send;\n    type Comparator: Default + for<'de> DeserializeArguments<'de> + Debug + Sync + Send;\n\n    type GetArguments: Default + for<'de> DeserializeArguments<'de> + Debug + Sync + Send;\n    type SetArguments<'de>: Default + DeserializeArguments<'de> + Debug + Sync + Send;\n    type QueryArguments: Default + for<'de> DeserializeArguments<'de> + Debug + Sync + Send;\n    type CopyArguments: Default + for<'de> DeserializeArguments<'de> + Debug + Sync + Send;\n    type ParseArguments: Default + for<'de> DeserializeArguments<'de> + Debug + Sync + Send;\n\n    const ID_PROPERTY: Self::Property;\n}\n\npub trait JmapSharedObject: JmapObject {\n    type Right: JmapRight + Into<Self::Property> + Debug + Clone + Copy + Sync + Send;\n\n    const SHARE_WITH_PROPERTY: Self::Property;\n}\n\npub trait JmapRight: Clone + Copy + Sized + 'static {\n    fn all_rights() -> &'static [Self];\n    fn to_acl(&self) -> &'static [Acl];\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Serialize)]\n#[serde(untagged)]\npub enum AnyId {\n    Id(Id),\n    BlobId(BlobId),\n}\n\npub trait JmapObjectId {\n    fn as_id(&self) -> Option<Id>;\n    fn as_any_id(&self) -> Option<AnyId>;\n    fn as_id_ref(&self) -> Option<&str>;\n    fn try_set_id(&mut self, new_id: AnyId) -> bool;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\nenum MaybeReference<T: FromStr> {\n    Value(T),\n    Reference(String),\n    ParseError,\n}\n\nfn parse_ref<T: FromStr>(value: &str) -> MaybeReference<T> {\n    if let Some(reference) = value.strip_prefix('#') {\n        MaybeReference::Reference(reference.to_string())\n    } else {\n        T::from_str(value)\n            .map(MaybeReference::Value)\n            .unwrap_or(MaybeReference::ParseError)\n    }\n}\n\nimpl From<Id> for AnyId {\n    fn from(value: Id) -> Self {\n        AnyId::Id(value)\n    }\n}\n\nimpl From<BlobId> for AnyId {\n    fn from(value: BlobId) -> Self {\n        AnyId::BlobId(value)\n    }\n}\n\nimpl TryFrom<AnyId> for Id {\n    type Error = ();\n\n    fn try_from(value: AnyId) -> Result<Self, Self::Error> {\n        if let AnyId::Id(id) = value {\n            Ok(id)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl TryFrom<AnyId> for BlobId {\n    type Error = ();\n\n    fn try_from(value: AnyId) -> Result<Self, Self::Error> {\n        if let AnyId::BlobId(id) = value {\n            Ok(id)\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for AnyId {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        let value = <&str>::deserialize(deserializer)?;\n        if let Some(blob_id) = BlobId::from_base32(value) {\n            Ok(AnyId::BlobId(blob_id))\n        } else if let Ok(id) = Id::from_str(value) {\n            Ok(AnyId::Id(id))\n        } else {\n            Err(serde::de::Error::custom(format!(\n                \"Invalid AnyId: {}\",\n                value\n            )))\n        }\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize)]\npub struct NullObject;\n\nimpl JmapObject for NullObject {\n    type Property = Null;\n    type Element = Null;\n    type Id = Null;\n\n    type Filter = ();\n    type Comparator = ();\n\n    type GetArguments = ();\n    type SetArguments<'de> = ();\n    type QueryArguments = ();\n    type CopyArguments = ();\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = Null;\n}\n\nimpl JmapRight for Null {\n    fn all_rights() -> &'static [Self] {\n        unreachable!()\n    }\n\n    fn to_acl(&self) -> &'static [Acl] {\n        unreachable!()\n    }\n}\n\nimpl FromStr for NullObject {\n    type Err = ();\n\n    fn from_str(_: &str) -> Result<Self, Self::Err> {\n        unreachable!()\n    }\n}\n\nimpl JmapObjectId for Null {\n    fn as_id(&self) -> Option<Id> {\n        unreachable!()\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        unreachable!()\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        unreachable!()\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        unreachable!()\n    }\n}\n\nimpl TryFrom<AnyId> for Null {\n    type Error = ();\n\n    fn try_from(_: AnyId) -> Result<Self, Self::Error> {\n        unreachable!()\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/participant_identity.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    request::{deserialize::DeserializeArguments, reference::MaybeIdReference},\n};\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\nuse types::id::Id;\n\n#[derive(Debug, Clone, Default)]\npub struct ParticipantIdentity;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum ParticipantIdentityProperty {\n    Id,\n    Name,\n    CalendarAddress,\n    IsDefault,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum ParticipantIdentityValue {\n    Id(Id),\n}\n\nimpl Property for ParticipantIdentityProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        ParticipantIdentityProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            ParticipantIdentityProperty::Id => \"id\",\n            ParticipantIdentityProperty::Name => \"name\",\n            ParticipantIdentityProperty::CalendarAddress => \"calendarAddress\",\n            ParticipantIdentityProperty::IsDefault => \"isDefault\",\n        }\n        .into()\n    }\n}\n\nimpl Element for ParticipantIdentityValue {\n    type Property = ParticipantIdentityProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                ParticipantIdentityProperty::Id => {\n                    Id::from_str(value).ok().map(ParticipantIdentityValue::Id)\n                }\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            ParticipantIdentityValue::Id(id) => id.to_string().into(),\n        }\n    }\n}\n\nimpl ParticipantIdentityProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => ParticipantIdentityProperty::Id,\n            b\"name\" => ParticipantIdentityProperty::Name,\n            b\"calendarAddress\" => ParticipantIdentityProperty::CalendarAddress,\n            b\"isDefault\" => ParticipantIdentityProperty::IsDefault\n        )\n    }\n\n    fn as_str(&self) -> &'static str {\n        match self {\n            ParticipantIdentityProperty::Id => \"id\",\n            ParticipantIdentityProperty::Name => \"name\",\n            ParticipantIdentityProperty::CalendarAddress => \"calendarAddress\",\n            ParticipantIdentityProperty::IsDefault => \"isDefault\",\n        }\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct ParticipantIdentitySetArguments {\n    pub on_success_set_is_default: Option<MaybeIdReference<Id>>,\n}\n\nimpl<'de> DeserializeArguments<'de> for ParticipantIdentitySetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"onSuccessSetIsDefault\" => {\n                self.on_success_set_is_default = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl FromStr for ParticipantIdentityProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        ParticipantIdentityProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for ParticipantIdentity {\n    type Property = ParticipantIdentityProperty;\n\n    type Element = ParticipantIdentityValue;\n\n    type Id = Id;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ParticipantIdentitySetArguments;\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = ParticipantIdentityProperty::Id;\n}\n\nimpl TryFrom<ParticipantIdentityProperty> for Id {\n    type Error = ();\n\n    fn try_from(_: ParticipantIdentityProperty) -> Result<Self, Self::Error> {\n        Err(())\n    }\n}\n\nimpl From<Id> for ParticipantIdentityValue {\n    fn from(id: Id) -> Self {\n        ParticipantIdentityValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for ParticipantIdentityValue {\n    fn as_id(&self) -> Option<Id> {\n        let ParticipantIdentityValue::Id(id) = self;\n        Some(*id)\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        let ParticipantIdentityValue::Id(id) = self;\n        Some(AnyId::Id(*id))\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(new_id) = new_id {\n            *self = ParticipantIdentityValue::Id(new_id);\n            return true;\n        }\n        false\n    }\n}\n\nimpl JmapObjectId for ParticipantIdentityProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n\nimpl Display for ParticipantIdentityProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        self.as_str().fmt(f)\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/principal.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\nuse types::id::Id;\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    request::{capability::Capability, deserialize::DeserializeArguments},\n};\n\n#[derive(Debug, Clone, Default)]\npub struct Principal;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum PrincipalProperty {\n    Id,\n    Type,\n    Name,\n    Description,\n    Email,\n    Timezone,\n    Capabilities,\n    Accounts,\n    IdValue(Id),\n    Capability(Capability),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum PrincipalValue {\n    Id(Id),\n    Type(PrincipalType),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum PrincipalType {\n    Individual,\n    Group,\n    Resource,\n    Location,\n    Other,\n}\n\nimpl Property for PrincipalProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        PrincipalProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            PrincipalProperty::Capabilities => \"capabilities\",\n            PrincipalProperty::Description => \"description\",\n            PrincipalProperty::Email => \"email\",\n            PrincipalProperty::Id => \"id\",\n            PrincipalProperty::Name => \"name\",\n            PrincipalProperty::Timezone => \"timezone\",\n            PrincipalProperty::Type => \"type\",\n            PrincipalProperty::Accounts => \"accounts\",\n            PrincipalProperty::Capability(cap) => cap.as_str(),\n            PrincipalProperty::IdValue(id) => return id.to_string().into(),\n        }\n        .into()\n    }\n}\n\nimpl Element for PrincipalValue {\n    type Property = PrincipalProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                PrincipalProperty::Id => Id::from_str(value).ok().map(PrincipalValue::Id),\n                PrincipalProperty::Type => PrincipalType::parse(value).map(PrincipalValue::Type),\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            PrincipalValue::Id(id) => id.to_string().into(),\n            PrincipalValue::Type(t) => t.as_str().into(),\n        }\n    }\n}\n\nimpl PrincipalProperty {\n    pub fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => PrincipalProperty::Id,\n            b\"type\" => PrincipalProperty::Type,\n            b\"name\" => PrincipalProperty::Name,\n            b\"description\" => PrincipalProperty::Description,\n            b\"email\" => PrincipalProperty::Email,\n            b\"timeZone\" => PrincipalProperty::Timezone,\n            b\"capabilities\" => PrincipalProperty::Capabilities,\n            b\"accounts\" => PrincipalProperty::Accounts,\n        )\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            PrincipalProperty::Id => \"id\",\n            PrincipalProperty::Type => \"type\",\n            PrincipalProperty::Name => \"name\",\n            PrincipalProperty::Description => \"description\",\n            PrincipalProperty::Email => \"email\",\n            PrincipalProperty::Timezone => \"timeZone\",\n            PrincipalProperty::Capabilities => \"capabilities\",\n            PrincipalProperty::Accounts => \"accounts\",\n            PrincipalProperty::Capability(cap) => cap.as_str(),\n            PrincipalProperty::IdValue(_) => \"\",\n        }\n    }\n}\n\nimpl PrincipalType {\n    pub fn parse(s: &str) -> Option<Self> {\n        hashify::tiny_map!(s.as_bytes(),\n            b\"individual\" => PrincipalType::Individual,\n            b\"group\" => PrincipalType::Group,\n            b\"resource\" => PrincipalType::Resource,\n            b\"location\" => PrincipalType::Location,\n            b\"other\" => PrincipalType::Other,\n        )\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            PrincipalType::Individual => \"individual\",\n            PrincipalType::Group => \"group\",\n            PrincipalType::Resource => \"resource\",\n            PrincipalType::Location => \"location\",\n            PrincipalType::Other => \"other\",\n        }\n    }\n}\n\nimpl FromStr for PrincipalProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        PrincipalProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for Principal {\n    type Property = PrincipalProperty;\n\n    type Element = PrincipalValue;\n\n    type Id = Id;\n\n    type Filter = PrincipalFilter;\n\n    type Comparator = PrincipalComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = PrincipalProperty::Id;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum PrincipalFilter {\n    AccountIds(Vec<Id>),\n    Email(String),\n    Name(String),\n    Text(String),\n    Type(PrincipalType),\n    Timezone(String),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum PrincipalComparator {\n    Name,\n    Email,\n    Type,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for PrincipalFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"accountIds\" => {\n                *self = PrincipalFilter::AccountIds(map.next_value()?);\n            },\n            b\"email\" => {\n                *self = PrincipalFilter::Email(map.next_value()?);\n            },\n            b\"name\" => {\n                *self = PrincipalFilter::Name(map.next_value()?);\n            },\n            b\"text\" => {\n                *self = PrincipalFilter::Text(map.next_value()?);\n            },\n            b\"type\" => {\n                *self = PrincipalFilter::Type(map.next_value()?);\n            },\n            b\"timeZone\" => {\n                *self = PrincipalFilter::Timezone(map.next_value()?);\n            },\n            _ => {\n                *self = PrincipalFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for PrincipalComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"name\" => {\n                    *self = PrincipalComparator::Name;\n                },\n                b\"email\" => {\n                    *self = PrincipalComparator::Email;\n                },\n                b\"type\" => {\n                    *self = PrincipalComparator::Type;\n                },\n                _ => {\n                    *self = PrincipalComparator::_T(key.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Default for PrincipalFilter {\n    fn default() -> Self {\n        PrincipalFilter::_T(\"\".to_string())\n    }\n}\n\nimpl Default for PrincipalComparator {\n    fn default() -> Self {\n        PrincipalComparator::_T(\"\".to_string())\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for PrincipalType {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        PrincipalType::parse(<&str>::deserialize(deserializer)?)\n            .ok_or_else(|| serde::de::Error::custom(\"invalid JMAP PrincipalType\"))\n    }\n}\n\nimpl From<Id> for PrincipalValue {\n    fn from(id: Id) -> Self {\n        PrincipalValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for PrincipalValue {\n    fn as_id(&self) -> Option<Id> {\n        if let PrincipalValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let PrincipalValue::Id(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = PrincipalValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl Display for PrincipalFilter {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(match self {\n            PrincipalFilter::AccountIds(_) => \"accountIds\",\n            PrincipalFilter::Email(_) => \"email\",\n            PrincipalFilter::Name(_) => \"name\",\n            PrincipalFilter::Text(_) => \"text\",\n            PrincipalFilter::Type(_) => \"type\",\n            PrincipalFilter::Timezone(_) => \"timezone\",\n            PrincipalFilter::_T(other) => other,\n        })\n    }\n}\n\nimpl JmapObjectId for PrincipalProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n\nimpl Display for PrincipalProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(self.as_str())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/push_subscription.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::object::{AnyId, JmapObject, JmapObjectId};\nuse crate::types::date::UTCDate;\nuse jmap_tools::{Element, JsonPointer, JsonPointerItem};\nuse jmap_tools::{Key, Property};\nuse std::borrow::Cow;\nuse std::str::FromStr;\nuse types::{id::Id, type_state::DataType};\n\n#[derive(Debug, Clone, Default)]\npub struct PushSubscription;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum PushSubscriptionProperty {\n    Id,\n    DeviceClientId,\n    Url,\n    Keys,\n    P256dh,\n    Auth,\n    VerificationCode,\n    Expires,\n    Types,\n\n    // Other\n    Pointer(JsonPointer<PushSubscriptionProperty>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum PushSubscriptionValue {\n    Id(Id),\n    Date(UTCDate),\n    Types(DataType),\n}\n\nimpl Property for PushSubscriptionProperty {\n    fn try_parse(key: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        PushSubscriptionProperty::parse(value, key.is_none())\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            PushSubscriptionProperty::DeviceClientId => \"deviceClientId\",\n            PushSubscriptionProperty::Expires => \"expires\",\n            PushSubscriptionProperty::Id => \"id\",\n            PushSubscriptionProperty::Keys => \"keys\",\n            PushSubscriptionProperty::Types => \"types\",\n            PushSubscriptionProperty::Url => \"url\",\n            PushSubscriptionProperty::VerificationCode => \"verificationCode\",\n            PushSubscriptionProperty::P256dh => \"p256dh\",\n            PushSubscriptionProperty::Auth => \"auth\",\n            PushSubscriptionProperty::Pointer(json_pointer) => {\n                return json_pointer.to_string().into();\n            }\n        }\n        .into()\n    }\n}\n\nimpl PushSubscriptionProperty {\n    fn parse(value: &str, allow_patch: bool) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => PushSubscriptionProperty::Id,\n            b\"deviceClientId\" => PushSubscriptionProperty::DeviceClientId,\n            b\"url\" => PushSubscriptionProperty::Url,\n            b\"keys\" => PushSubscriptionProperty::Keys,\n            b\"p256dh\" => PushSubscriptionProperty::P256dh,\n            b\"auth\" => PushSubscriptionProperty::Auth,\n            b\"verificationCode\" => PushSubscriptionProperty::VerificationCode,\n            b\"expires\" => PushSubscriptionProperty::Expires,\n            b\"types\" => PushSubscriptionProperty::Types,\n        )\n        .or_else(|| {\n            if allow_patch && value.contains('/') {\n                PushSubscriptionProperty::Pointer(JsonPointer::parse(value)).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    fn patch_or_prop(&self) -> &PushSubscriptionProperty {\n        if let PushSubscriptionProperty::Pointer(ptr) = self\n            && let Some(JsonPointerItem::Key(Key::Property(prop))) = ptr.last()\n        {\n            prop\n        } else {\n            self\n        }\n    }\n}\n\nimpl Element for PushSubscriptionValue {\n    type Property = PushSubscriptionProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop.patch_or_prop() {\n                PushSubscriptionProperty::Id => {\n                    Id::from_str(value).ok().map(PushSubscriptionValue::Id)\n                }\n                PushSubscriptionProperty::Types => {\n                    DataType::parse(value).map(PushSubscriptionValue::Types)\n                }\n                PushSubscriptionProperty::Expires => UTCDate::from_str(value)\n                    .ok()\n                    .map(PushSubscriptionValue::Date),\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            PushSubscriptionValue::Id(id) => id.to_string().into(),\n            PushSubscriptionValue::Date(utcdate) => utcdate.to_string().into(),\n            PushSubscriptionValue::Types(data_type) => data_type.as_str().into(),\n        }\n    }\n}\n\nimpl FromStr for PushSubscriptionProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        PushSubscriptionProperty::parse(s, false).ok_or(())\n    }\n}\n\nimpl JmapObject for PushSubscription {\n    type Property = PushSubscriptionProperty;\n\n    type Element = PushSubscriptionValue;\n\n    type Id = Id;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = PushSubscriptionProperty::Id;\n}\n\nimpl From<Id> for PushSubscriptionValue {\n    fn from(id: Id) -> Self {\n        PushSubscriptionValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for PushSubscriptionValue {\n    fn as_id(&self) -> Option<Id> {\n        match self {\n            PushSubscriptionValue::Id(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            PushSubscriptionValue::Id(id) => Some(AnyId::Id(*id)),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = PushSubscriptionValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl JmapObjectId for PushSubscriptionProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/quota.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    request::deserialize::DeserializeArguments,\n};\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::{id::Id, type_state::DataType};\n\n#[derive(Debug, Clone, Default)]\npub struct Quota;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum QuotaProperty {\n    Id,\n    ResourceType,\n    Used,\n    Name,\n    Scope,\n    Types,\n    HardLimit,\n    WarnLimit,\n    SoftLimit,\n    Description,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum QuotaValue {\n    Id(Id),\n    Types(DataType),\n}\n\nimpl Property for QuotaProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        QuotaProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            QuotaProperty::Description => \"description\",\n            QuotaProperty::Id => \"id\",\n            QuotaProperty::Name => \"name\",\n            QuotaProperty::Types => \"types\",\n            QuotaProperty::ResourceType => \"resourceType\",\n            QuotaProperty::Used => \"used\",\n            QuotaProperty::HardLimit => \"hardLimit\",\n            QuotaProperty::Scope => \"scope\",\n            QuotaProperty::WarnLimit => \"warnLimit\",\n            QuotaProperty::SoftLimit => \"softLimit\",\n        }\n        .into()\n    }\n}\n\nimpl QuotaProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => QuotaProperty::Id,\n            b\"resourceType\" => QuotaProperty::ResourceType,\n            b\"used\" => QuotaProperty::Used,\n            b\"name\" => QuotaProperty::Name,\n            b\"scope\" => QuotaProperty::Scope,\n            b\"types\" => QuotaProperty::Types,\n            b\"hardLimit\" => QuotaProperty::HardLimit,\n            b\"warnLimit\" => QuotaProperty::WarnLimit,\n            b\"softLimit\" => QuotaProperty::SoftLimit,\n            b\"description\" => QuotaProperty::Description,\n        )\n    }\n}\n\nimpl Element for QuotaValue {\n    type Property = QuotaProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                QuotaProperty::Id => Id::from_str(value).ok().map(QuotaValue::Id),\n                QuotaProperty::Types => DataType::parse(value).map(QuotaValue::Types),\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            QuotaValue::Id(id) => id.to_string().into(),\n            QuotaValue::Types(data_type) => data_type.as_str().into(),\n        }\n    }\n}\n\nimpl FromStr for QuotaProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        QuotaProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for Quota {\n    type Property = QuotaProperty;\n\n    type Element = QuotaValue;\n\n    type Id = Id;\n\n    type Filter = QuotaFilter;\n\n    type Comparator = QuotaComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = QuotaProperty::Id;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum QuotaFilter {\n    Name(String),\n    Type(String),\n    Scope(String),\n    ResourceType(String),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum QuotaComparator {\n    Name,\n    Type,\n    Used,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for QuotaFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"name\" => {\n                *self = QuotaFilter::Name(map.next_value()?);\n            },\n            b\"type\" => {\n                *self = QuotaFilter::Type(map.next_value()?);\n            },\n            b\"scope\" => {\n                *self = QuotaFilter::Scope(map.next_value()?);\n            },\n            b\"resourceType\" => {\n                *self = QuotaFilter::ResourceType(map.next_value()?);\n            },\n            _ => {\n                *self = QuotaFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for QuotaComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"name\" => {\n                    *self = QuotaComparator::Name;\n                },\n                b\"type\" => {\n                    *self = QuotaComparator::Type;\n                },\n                b\"used\" => {\n                    *self = QuotaComparator::Used;\n                },\n                _ => {\n                    *self = QuotaComparator::_T(key.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Default for QuotaFilter {\n    fn default() -> Self {\n        QuotaFilter::_T(\"\".to_string())\n    }\n}\n\nimpl Default for QuotaComparator {\n    fn default() -> Self {\n        QuotaComparator::_T(\"\".to_string())\n    }\n}\n\nimpl From<Id> for QuotaValue {\n    fn from(id: Id) -> Self {\n        QuotaValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for QuotaValue {\n    fn as_id(&self) -> Option<Id> {\n        if let QuotaValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        self.as_id().map(AnyId::Id)\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = QuotaValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl JmapObjectId for QuotaProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/search_snippet.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::id::Id;\n\n#[derive(Debug, Clone, Default)]\npub struct SearchSnippet;\n\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum SearchSnippetProperty {\n    EmailId,\n    Subject,\n    Preview,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum SearchSnippetValue {\n    Id(Id),\n}\n\nimpl Property for SearchSnippetProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        SearchSnippetProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            SearchSnippetProperty::Preview => \"preview\",\n            SearchSnippetProperty::Subject => \"subject\",\n            SearchSnippetProperty::EmailId => \"emailId\",\n        }\n        .into()\n    }\n}\n\nimpl Element for SearchSnippetValue {\n    type Property = SearchSnippetProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                SearchSnippetProperty::EmailId => {\n                    Id::from_str(value).ok().map(SearchSnippetValue::Id)\n                }\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            SearchSnippetValue::Id(id) => id.to_string().into(),\n        }\n    }\n}\n\nimpl SearchSnippetProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"emailId\" => SearchSnippetProperty::EmailId,\n            b\"subject\" => SearchSnippetProperty::Subject,\n            b\"preview\" => SearchSnippetProperty::Preview,\n        )\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/share_notification.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    request::deserialize::DeserializeArguments,\n    types::date::UTCDate,\n};\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\nuse types::{id::Id, type_state::DataType};\n\n#[derive(Debug, Clone, Default)]\npub struct ShareNotification;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum ShareNotificationProperty {\n    Id,\n    Created,\n    ChangedBy,\n    ChangedByName,\n    ChangedByEmail,\n    ChangedByPrincipalId,\n    ObjectType,\n    ObjectAccountId,\n    ObjectId,\n    OldRights,\n    NewRights,\n    Name,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum ShareNotificationValue {\n    Id(Id),\n    Date(UTCDate),\n    ObjectType(DataType),\n}\n\nimpl Property for ShareNotificationProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        ShareNotificationProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            ShareNotificationProperty::Id => \"id\",\n            ShareNotificationProperty::Created => \"created\",\n            ShareNotificationProperty::ChangedBy => \"changedBy\",\n            ShareNotificationProperty::ChangedByName => \"name\",\n            ShareNotificationProperty::ChangedByEmail => \"email\",\n            ShareNotificationProperty::ChangedByPrincipalId => \"principalId\",\n            ShareNotificationProperty::ObjectType => \"objectType\",\n            ShareNotificationProperty::ObjectAccountId => \"objectAccountId\",\n            ShareNotificationProperty::ObjectId => \"objectId\",\n            ShareNotificationProperty::OldRights => \"oldRights\",\n            ShareNotificationProperty::NewRights => \"newRights\",\n            ShareNotificationProperty::Name => \"name\",\n        }\n        .into()\n    }\n}\n\nimpl Element for ShareNotificationValue {\n    type Property = ShareNotificationProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                ShareNotificationProperty::Id\n                | ShareNotificationProperty::ChangedByPrincipalId\n                | ShareNotificationProperty::ObjectAccountId\n                | ShareNotificationProperty::ObjectId => {\n                    Id::from_str(value).ok().map(ShareNotificationValue::Id)\n                }\n                ShareNotificationProperty::Created => UTCDate::from_str(value)\n                    .ok()\n                    .map(ShareNotificationValue::Date),\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            ShareNotificationValue::Id(id) => id.to_string().into(),\n            ShareNotificationValue::Date(date) => date.to_string().into(),\n            ShareNotificationValue::ObjectType(ty) => ty.as_str().into(),\n        }\n    }\n}\n\nimpl ShareNotificationProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => ShareNotificationProperty::Id,\n            b\"created\" => ShareNotificationProperty::Created,\n            b\"changedBy\" => ShareNotificationProperty::ChangedBy,\n            b\"name\" => ShareNotificationProperty::ChangedByName,\n            b\"email\" => ShareNotificationProperty::ChangedByEmail,\n            b\"principalId\" => ShareNotificationProperty::ChangedByPrincipalId,\n            b\"objectType\" => ShareNotificationProperty::ObjectType,\n            b\"objectAccountId\" => ShareNotificationProperty::ObjectAccountId,\n            b\"objectId\" => ShareNotificationProperty::ObjectId,\n            b\"oldRights\" => ShareNotificationProperty::OldRights,\n            b\"newRights\" => ShareNotificationProperty::NewRights\n        )\n    }\n}\n\nimpl FromStr for ShareNotificationProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        ShareNotificationProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for ShareNotification {\n    type Property = ShareNotificationProperty;\n\n    type Element = ShareNotificationValue;\n\n    type Id = Id;\n\n    type Filter = ShareNotificationFilter;\n\n    type Comparator = ShareNotificationComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = ShareNotificationProperty::Id;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ShareNotificationFilter {\n    After(UTCDate),\n    Before(UTCDate),\n    ObjectType(DataType),\n    ObjectAccountId(Id),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ShareNotificationComparator {\n    Created,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for ShareNotificationFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"after\" => {\n                *self = ShareNotificationFilter::After(map.next_value()?);\n            },\n            b\"before\" => {\n                *self = ShareNotificationFilter::Before(map.next_value()?);\n            },\n            b\"objectType\" => {\n                *self = ShareNotificationFilter::ObjectType(map.next_value()?);\n            },\n            b\"objectAccountId\" => {\n                *self = ShareNotificationFilter::ObjectAccountId(map.next_value()?);\n            },\n            _ => {\n                *self = ShareNotificationFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for ShareNotificationComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"created\" => {\n                    *self = ShareNotificationComparator::Created;\n                },\n                _ => {\n                    *self = ShareNotificationComparator::_T(value.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n        Ok(())\n    }\n}\n\nimpl ShareNotificationFilter {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            ShareNotificationFilter::After(_) => \"after\",\n            ShareNotificationFilter::Before(_) => \"before\",\n            ShareNotificationFilter::ObjectType(_) => \"objectType\",\n            ShareNotificationFilter::ObjectAccountId(_) => \"objectAccountId\",\n            ShareNotificationFilter::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl ShareNotificationComparator {\n    pub fn into_string(self) -> Cow<'static, str> {\n        match self {\n            ShareNotificationComparator::Created => \"created\",\n            ShareNotificationComparator::_T(s) => return Cow::Owned(s),\n        }\n        .into()\n    }\n}\n\nimpl Default for ShareNotificationFilter {\n    fn default() -> Self {\n        ShareNotificationFilter::_T(String::new())\n    }\n}\n\nimpl Default for ShareNotificationComparator {\n    fn default() -> Self {\n        ShareNotificationComparator::_T(String::new())\n    }\n}\n\nimpl TryFrom<ShareNotificationProperty> for Id {\n    type Error = ();\n\n    fn try_from(_: ShareNotificationProperty) -> Result<Self, Self::Error> {\n        Err(())\n    }\n}\n\nimpl From<Id> for ShareNotificationValue {\n    fn from(id: Id) -> Self {\n        ShareNotificationValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for ShareNotificationValue {\n    fn as_id(&self) -> Option<Id> {\n        if let ShareNotificationValue::Id(id) = self {\n            Some(*id)\n        } else {\n            None\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        if let ShareNotificationValue::Id(id) = self {\n            Some(AnyId::Id(*id))\n        } else {\n            None\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n\nimpl JmapObjectId for ShareNotificationProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n\nimpl Display for ShareNotificationProperty {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"{}\", self.to_cow())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/sieve.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, DeserializeArguments, JmapObject, JmapObjectId, MaybeReference, parse_ref},\n    request::reference::MaybeIdReference,\n};\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::{blob::BlobId, id::Id};\n\n#[derive(Debug, Clone, Default)]\npub struct Sieve;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum SieveProperty {\n    Id,\n    Name,\n    BlobId,\n    IsActive,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum SieveValue {\n    Id(Id),\n    BlobId(BlobId),\n    IdReference(String),\n}\n\nimpl Property for SieveProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        SieveProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            SieveProperty::BlobId => \"blobId\",\n            SieveProperty::Id => \"id\",\n            SieveProperty::Name => \"name\",\n            SieveProperty::IsActive => \"isActive\",\n        }\n        .into()\n    }\n}\n\nimpl Element for SieveValue {\n    type Property = SieveProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                SieveProperty::Id => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(SieveValue::Id(v)),\n                    MaybeReference::Reference(v) => Some(SieveValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                SieveProperty::BlobId => match parse_ref(value) {\n                    MaybeReference::Value(v) => Some(SieveValue::BlobId(v)),\n                    MaybeReference::Reference(v) => Some(SieveValue::IdReference(v)),\n                    MaybeReference::ParseError => None,\n                },\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            SieveValue::Id(id) => id.to_string().into(),\n            SieveValue::BlobId(blob_id) => blob_id.to_string().into(),\n            SieveValue::IdReference(r) => format!(\"#{r}\").into(),\n        }\n    }\n}\n\nimpl SieveProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => SieveProperty::Id,\n            b\"name\" => SieveProperty::Name,\n            b\"blobId\" => SieveProperty::BlobId,\n            b\"isActive\" => SieveProperty::IsActive,\n        )\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct SieveSetArguments {\n    pub on_success_activate_script: Option<MaybeIdReference<Id>>,\n    pub on_success_deactivate_script: Option<bool>,\n}\n\nimpl<'de> DeserializeArguments<'de> for SieveSetArguments {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"onSuccessActivateScript\" => {\n                self.on_success_activate_script = map.next_value()?;\n            },\n            b\"onSuccessDeactivateScript\" => {\n                self.on_success_deactivate_script = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl FromStr for SieveProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        SieveProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for Sieve {\n    type Property = SieveProperty;\n\n    type Element = SieveValue;\n\n    type Id = Id;\n\n    type Filter = SieveFilter;\n\n    type Comparator = SieveComparator;\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = SieveSetArguments;\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = SieveProperty::Id;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum SieveFilter {\n    Name(String),\n    IsActive(bool),\n    _T(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum SieveComparator {\n    Name,\n    IsActive,\n    _T(String),\n}\n\nimpl<'de> DeserializeArguments<'de> for SieveFilter {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"name\" => {\n                *self = SieveFilter::Name(map.next_value()?);\n            },\n            b\"isActive\" => {\n                *self = SieveFilter::IsActive(map.next_value()?);\n            },\n            _ => {\n                *self = SieveFilter::_T(key.to_string());\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for SieveComparator {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        if key == \"property\" {\n            let value = map.next_value::<Cow<str>>()?;\n            hashify::fnc_map!(value.as_bytes(),\n                b\"name\" => {\n                    *self = SieveComparator::Name;\n                },\n                b\"isActive\" => {\n                    *self = SieveComparator::IsActive;\n                },\n                _ => {\n                    *self = SieveComparator::_T(key.to_string());\n                }\n            );\n        } else {\n            let _ = map.next_value::<serde::de::IgnoredAny>()?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl Default for SieveFilter {\n    fn default() -> Self {\n        SieveFilter::_T(\"\".to_string())\n    }\n}\n\nimpl Default for SieveComparator {\n    fn default() -> Self {\n        SieveComparator::_T(\"\".to_string())\n    }\n}\n\nimpl From<Id> for SieveValue {\n    fn from(id: Id) -> Self {\n        SieveValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for SieveValue {\n    fn as_id(&self) -> Option<Id> {\n        match self {\n            SieveValue::Id(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            SieveValue::Id(id) => Some(AnyId::Id(*id)),\n            SieveValue::BlobId(id) => Some(AnyId::BlobId(id.clone())),\n            SieveValue::IdReference(_) => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        if let SieveValue::IdReference(r) = self {\n            Some(r)\n        } else {\n            None\n        }\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        match new_id {\n            AnyId::Id(id) => {\n                *self = SieveValue::Id(id);\n            }\n            AnyId::BlobId(id) => {\n                *self = SieveValue::BlobId(id);\n            }\n        }\n        true\n    }\n}\n\nimpl JmapObjectId for SieveProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/thread.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::id::Id;\n\nuse crate::object::{AnyId, JmapObject, JmapObjectId};\n\n#[derive(Debug, Clone, Default)]\npub struct Thread;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum ThreadProperty {\n    Id,\n    EmailIds,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum ThreadValue {\n    Id(Id),\n}\n\nimpl Property for ThreadProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        ThreadProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            ThreadProperty::Id => \"id\",\n            ThreadProperty::EmailIds => \"emailIds\",\n        }\n        .into()\n    }\n}\n\nimpl Element for ThreadValue {\n    type Property = ThreadProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(_) = key {\n            Id::from_str(value).ok().map(ThreadValue::Id)\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            ThreadValue::Id(id) => id.to_string().into(),\n        }\n    }\n}\n\nimpl ThreadProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => ThreadProperty::Id,\n            b\"emailIds\" => ThreadProperty::EmailIds,\n        )\n    }\n}\n\nimpl FromStr for ThreadProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        ThreadProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for Thread {\n    type Property = ThreadProperty;\n\n    type Element = ThreadValue;\n\n    type Id = Id;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = ThreadProperty::Id;\n}\n\nimpl From<Id> for ThreadValue {\n    fn from(id: Id) -> Self {\n        ThreadValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for ThreadValue {\n    fn as_id(&self) -> Option<Id> {\n        match self {\n            ThreadValue::Id(id) => Some(*id),\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        self.as_id().map(AnyId::Id)\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = ThreadValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl JmapObjectId for ThreadProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/object/vacation_response.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObject, JmapObjectId},\n    types::date::UTCDate,\n};\nuse jmap_tools::{Element, Key, Property};\nuse std::{borrow::Cow, str::FromStr};\nuse types::id::Id;\n\n#[derive(Debug, Clone, Default)]\npub struct VacationResponse;\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum VacationResponseProperty {\n    Id,\n    IsEnabled,\n    FromDate,\n    ToDate,\n    Subject,\n    TextBody,\n    HtmlBody,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum VacationResponseValue {\n    Id(Id),\n    Date(UTCDate),\n}\n\nimpl Property for VacationResponseProperty {\n    fn try_parse(_: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {\n        VacationResponseProperty::parse(value)\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            VacationResponseProperty::HtmlBody => \"htmlBody\",\n            VacationResponseProperty::Id => \"id\",\n            VacationResponseProperty::TextBody => \"textBody\",\n            VacationResponseProperty::FromDate => \"fromDate\",\n            VacationResponseProperty::IsEnabled => \"isEnabled\",\n            VacationResponseProperty::ToDate => \"toDate\",\n            VacationResponseProperty::Subject => \"subject\",\n        }\n        .into()\n    }\n}\n\nimpl Element for VacationResponseValue {\n    type Property = VacationResponseProperty;\n\n    fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {\n        if let Key::Property(prop) = key {\n            match prop {\n                VacationResponseProperty::Id => {\n                    Id::from_str(value).ok().map(VacationResponseValue::Id)\n                }\n                VacationResponseProperty::FromDate | VacationResponseProperty::ToDate => {\n                    UTCDate::from_str(value)\n                        .ok()\n                        .map(VacationResponseValue::Date)\n                }\n                _ => None,\n            }\n        } else {\n            None\n        }\n    }\n\n    fn to_cow(&self) -> Cow<'static, str> {\n        match self {\n            VacationResponseValue::Id(id) => id.to_string().into(),\n            VacationResponseValue::Date(utcdate) => utcdate.to_string().into(),\n        }\n    }\n}\n\nimpl VacationResponseProperty {\n    fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"id\" => VacationResponseProperty::Id,\n            b\"isEnabled\" => VacationResponseProperty::IsEnabled,\n            b\"fromDate\" => VacationResponseProperty::FromDate,\n            b\"toDate\" => VacationResponseProperty::ToDate,\n            b\"textBody\" => VacationResponseProperty::TextBody,\n            b\"htmlBody\" => VacationResponseProperty::HtmlBody,\n            b\"subject\" => VacationResponseProperty::Subject,\n        )\n    }\n}\n\nimpl FromStr for VacationResponseProperty {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        VacationResponseProperty::parse(s).ok_or(())\n    }\n}\n\nimpl JmapObject for VacationResponse {\n    type Property = VacationResponseProperty;\n\n    type Element = VacationResponseValue;\n\n    type Id = Id;\n\n    type Filter = ();\n\n    type Comparator = ();\n\n    type GetArguments = ();\n\n    type SetArguments<'de> = ();\n\n    type QueryArguments = ();\n\n    type CopyArguments = ();\n\n    type ParseArguments = ();\n\n    const ID_PROPERTY: Self::Property = VacationResponseProperty::Id;\n}\n\nimpl From<Id> for VacationResponseValue {\n    fn from(id: Id) -> Self {\n        VacationResponseValue::Id(id)\n    }\n}\n\nimpl JmapObjectId for VacationResponseValue {\n    fn as_id(&self) -> Option<Id> {\n        match self {\n            VacationResponseValue::Id(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        match self {\n            VacationResponseValue::Id(id) => Some(AnyId::Id(*id)),\n            _ => None,\n        }\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, new_id: AnyId) -> bool {\n        if let AnyId::Id(id) = new_id {\n            *self = VacationResponseValue::Id(id);\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl JmapObjectId for VacationResponseProperty {\n    fn as_id(&self) -> Option<Id> {\n        None\n    }\n\n    fn as_any_id(&self) -> Option<AnyId> {\n        None\n    }\n\n    fn as_id_ref(&self) -> Option<&str> {\n        None\n    }\n\n    fn try_set_id(&mut self, _: AnyId) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/references/eval.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    object::{AnyId, JmapObjectId},\n    references::{\n        Graph,\n        jsptr::{EvalResults, ResponsePtr},\n    },\n    request::reference::ResultReference,\n    response::{ChangesResponseMethod, GetResponseMethod, Response, ResponseMethod},\n};\nuse compact_str::format_compact;\nuse jmap_tools::{Element, Key, Property, Value};\nuse types::{blob::BlobId, id::Id};\n\nimpl Response<'_> {\n    pub(crate) fn eval_result_references(&self, rr: &ResultReference) -> trc::Result<EvalResults> {\n        let mut results = EvalResults::default();\n\n        for response in &self.method_responses {\n            if response.id == rr.result_of && response.name == rr.name {\n                let path = rr.path.iter();\n                let success = match &response.method {\n                    ResponseMethod::Get(response) => match response {\n                        GetResponseMethod::Email(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Mailbox(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Thread(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Identity(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::EmailSubmission(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::PushSubscription(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Sieve(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::VacationResponse(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Principal(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Quota(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Blob(response) => response.eval_jptr(path, &mut results),\n                        GetResponseMethod::AddressBook(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::ContactCard(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::FileNode(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::Calendar(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::CalendarEvent(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::CalendarEventNotification(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::ParticipantIdentity(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::ShareNotification(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        GetResponseMethod::PrincipalAvailability(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                    },\n                    ResponseMethod::Changes(response) => match response {\n                        ChangesResponseMethod::Email(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::Mailbox(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::Thread(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::Identity(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::EmailSubmission(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::Quota(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::AddressBook(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::ContactCard(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::FileNode(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::Calendar(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::CalendarEvent(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::CalendarEventNotification(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                        ChangesResponseMethod::ShareNotification(response) => {\n                            response.eval_jptr(path, &mut results)\n                        }\n                    },\n                    ResponseMethod::Query(response) => response.eval_jptr(path, &mut results),\n                    ResponseMethod::QueryChanges(response) => {\n                        response.eval_jptr(path, &mut results)\n                    }\n                    _ => false,\n                };\n\n                if success {\n                    return Ok(results);\n                }\n            }\n        }\n\n        Err(trc::JmapEvent::InvalidResultReference\n            .into_err()\n            .details(format_compact!(\n                \"Result reference to {}#{} not found.\",\n                rr.result_of,\n                rr.name\n            )))\n    }\n\n    pub(crate) fn eval_id_reference(&self, ir: &str) -> trc::Result<Id> {\n        if let Some(AnyId::Id(id)) = self.created_ids.get(ir) {\n            Ok(*id)\n        } else {\n            Err(trc::JmapEvent::InvalidResultReference\n                .into_err()\n                .details(format_compact!(\"Id reference {ir:?} not found.\")))\n        }\n    }\n\n    pub(crate) fn eval_blob_id_reference(&self, ir: &str) -> trc::Result<BlobId> {\n        if let Some(AnyId::BlobId(id)) = self.created_ids.get(ir) {\n            Ok(id.clone())\n        } else {\n            Err(trc::JmapEvent::InvalidResultReference\n                .into_err()\n                .details(format_compact!(\"blobId reference {ir:?} not found.\")))\n        }\n    }\n}\n\npub(crate) trait EvalObjectReferences {\n    fn eval_object_references(\n        &mut self,\n        response: &Response<'_>,\n        graph: &mut Graph<'_>,\n        depth: usize,\n    ) -> trc::Result<()>;\n}\n\nimpl<'x, P, E> EvalObjectReferences for Value<'x, P, E>\nwhere\n    P: Property + JmapObjectId,\n    E: Element<Property = P> + JmapObjectId,\n{\n    fn eval_object_references(\n        &mut self,\n        response: &Response<'_>,\n        graph: &mut Graph<'_>,\n        depth: usize,\n    ) -> trc::Result<()> {\n        let Value::Object(obj) = self else {\n            return Ok(());\n        };\n\n        for (key, value) in obj.as_mut_vec() {\n            // Resolve patch with references (e.g. mailboxIds/#idRef)\n            if depth == 0\n                && let Key::Property(property) = key\n                && let Some(id_ref) = property.as_id_ref()\n            {\n                if let Some(id) = response.created_ids.get(id_ref) {\n                    if !property.try_set_id(id.clone()) {\n                        return Err(trc::JmapEvent::InvalidResultReference\n                            .into_err()\n                            .details(\"Id reference points to invalid type.\"));\n                    }\n                } else {\n                    return Err(trc::JmapEvent::InvalidResultReference\n                        .into_err()\n                        .details(format_compact!(\"Id reference {id_ref:?} not found.\")));\n                }\n            }\n\n            match value {\n                Value::Element(element) => {\n                    if let Some(id_ref) = element.as_id_ref() {\n                        if let Some(id) = response.created_ids.get(id_ref) {\n                            if !element.try_set_id(id.clone()) {\n                                return Err(trc::JmapEvent::InvalidResultReference\n                                    .into_err()\n                                    .details(\"Id reference points to invalid type.\"));\n                            }\n                        } else if let Graph::Some { child_id, graph } = graph {\n                            graph\n                                .entry(child_id.to_string())\n                                .or_insert_with(Vec::new)\n                                .push(id_ref.to_string());\n                        } else {\n                            return Err(trc::JmapEvent::InvalidResultReference\n                                .into_err()\n                                .details(format_compact!(\"Id reference {id_ref:?} not found.\")));\n                        }\n                    }\n                }\n                Value::Array(items) if depth == 0 => {\n                    // Resolve references in arrays (e.g. emailIds: [#idRef1, #idRef2])\n                    for item in items {\n                        item.eval_object_references(response, graph, depth + 1)?;\n                    }\n                }\n                Value::Object(items) if depth == 0 => {\n                    // Resolve references in JMAP sets (e.g. mailboxIds: { \"#idRef1\": true, \"#idRef2\": true })\n                    for (key, _) in items.as_mut_vec() {\n                        if let Key::Property(property) = key\n                            && let Some(id_ref) = property.as_id_ref()\n                        {\n                            if let Some(id) = response.created_ids.get(id_ref) {\n                                if !property.try_set_id(id.clone()) {\n                                    return Err(trc::JmapEvent::InvalidResultReference\n                                        .into_err()\n                                        .details(\"Id reference points to invalid type.\"));\n                                }\n                            } else {\n                                return Err(trc::JmapEvent::InvalidResultReference\n                                    .into_err()\n                                    .details(format_compact!(\n                                        \"Id reference {id_ref:?} not found.\"\n                                    )));\n                            }\n                        }\n                    }\n                }\n                _ => {}\n            }\n        }\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/references/jsptr.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    method::{\n        PropertyWrapper,\n        availability::{BusyPeriod, GetAvailabilityResponse},\n        changes::ChangesResponse,\n        get::GetResponse,\n        query::QueryResponse,\n        query_changes::{AddedItem, QueryChangesResponse},\n    },\n    object::{\n        AnyId, JmapObject, JmapObjectId,\n        calendar_event_notification::{\n            CalendarEventNotificationGetResponse, CalendarEventNotificationObject,\n        },\n    },\n    request::reference::ResultReference,\n};\nuse compact_str::format_compact;\nuse jmap_tools::{Element, JsonPointerItem, JsonPointerIter, Key, Null, Property, Value};\nuse std::{borrow::Cow, str::FromStr};\nuse types::{blob::BlobId, id::Id};\n\npub(crate) trait ResponsePtr {\n    fn eval_jptr(&self, pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool;\n}\n\n#[derive(Debug, Default)]\n#[repr(transparent)]\npub(crate) struct EvalResults(Vec<EvalResult>);\n\n#[derive(Debug)]\npub(crate) enum EvalResult {\n    Id(AnyId),\n    Property(Cow<'static, str>),\n}\n\nimpl<T> ResponsePtr for Vec<T>\nwhere\n    T: ResponsePtr,\n{\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next() {\n            Some(JsonPointerItem::Number(n)) => {\n                if let Some(v) = self.get(*n as usize) {\n                    v.eval_jptr(pointer, results);\n                }\n            }\n            Some(JsonPointerItem::Wildcard | JsonPointerItem::Root) | None => {\n                for v in self {\n                    v.eval_jptr(pointer.clone(), results);\n                }\n            }\n            _ => (),\n        }\n\n        true\n    }\n}\n\nimpl<'ctx, P, E> ResponsePtr for Value<'ctx, P, E>\nwhere\n    P: Property,\n    E: Element<Property = P> + JmapObjectId,\n{\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next() {\n            Some(JsonPointerItem::Key(key)) => {\n                if let Some(key) = key.as_string_key()\n                    && let Value::Object(map) = self\n                    && let Some(v) = map.get(&Key::Borrowed(key))\n                {\n                    v.eval_jptr(pointer, results);\n                }\n            }\n            Some(JsonPointerItem::Number(n)) => match self {\n                Value::Array(values) => {\n                    if let Some(v) = values.get(*n as usize) {\n                        v.eval_jptr(pointer, results);\n                    }\n                }\n                Value::Object(map) => {\n                    let n = Key::Owned(n.to_string());\n                    if let Some(v) = map.get(&n) {\n                        v.eval_jptr(pointer, results);\n                    }\n                }\n                _ => {}\n            },\n            Some(JsonPointerItem::Wildcard) => match self {\n                Value::Array(values) => {\n                    for v in values {\n                        v.eval_jptr(pointer.clone(), results);\n                    }\n                }\n                Value::Object(map) => {\n                    for v in map.values() {\n                        v.eval_jptr(pointer.clone(), results);\n                    }\n                }\n                _ => {}\n            },\n            Some(JsonPointerItem::Root) | None => match self {\n                Value::Element(e) => {\n                    if let Some(id) = e.as_any_id() {\n                        results.0.push(EvalResult::Id(id));\n                    }\n                }\n                Value::Array(list) => {\n                    for item in list {\n                        if let Value::Element(e) = item\n                            && let Some(id) = e.as_any_id()\n                        {\n                            results.0.push(EvalResult::Id(id));\n                        }\n                    }\n                }\n                _ => (),\n            },\n        }\n\n        true\n    }\n}\n\nimpl ResponsePtr for Id {\n    fn eval_jptr(&self, _pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        results.0.push(EvalResult::Id(AnyId::Id(*self)));\n        true\n    }\n}\n\nimpl ResponsePtr for BlobId {\n    fn eval_jptr(&self, _pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        results.0.push(EvalResult::Id(AnyId::BlobId(self.clone())));\n        true\n    }\n}\n\nimpl<T: Property> ResponsePtr for PropertyWrapper<T> {\n    fn eval_jptr(&self, _: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        results.0.push(EvalResult::Property(self.0.to_cow()));\n        true\n    }\n}\n\nimpl<T: JmapObject> ResponsePtr for GetResponse<T> {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"list\") => {\n                self.list.eval_jptr(pointer, results);\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl<T: JmapObject> ResponsePtr for ChangesResponse<T> {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        if let Some(property) = pointer.next().and_then(|item| item.as_string_key()) {\n            hashify::fnc_map!(property.as_bytes(),\n                \"created\" => {\n                    self.created.eval_jptr(pointer, results);\n                },\n                \"updated\" => {\n                    self.updated.eval_jptr(pointer, results);\n                },\n                \"updatedProperties\" => {\n                    if let Some(props) = &self.updated_properties {\n                        props.eval_jptr(pointer, results);\n                    }\n                },\n                _ => {\n                    return false;\n                }\n            );\n\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl ResponsePtr for QueryResponse {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"ids\") => {\n                self.ids.eval_jptr(pointer, results);\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl ResponsePtr for QueryChangesResponse {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"added\") => {\n                self.added.eval_jptr(pointer, results);\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl ResponsePtr for AddedItem {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"id\") => {\n                results.0.push(EvalResult::Id(AnyId::Id(self.id)));\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl ResponsePtr for CalendarEventNotificationGetResponse {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"list\") => {\n                self.list.eval_jptr(pointer, results);\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl ResponsePtr for CalendarEventNotificationObject {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"id\") => {\n                results.0.push(EvalResult::Id(AnyId::Id(self.id)));\n                true\n            }\n            Some(\"calendarEventId\") => {\n                if let Some(id) = &self.calendar_event_id {\n                    results.0.push(EvalResult::Id(AnyId::Id(*id)));\n                }\n                true\n            }\n            Some(\"event\") => {\n                if let Some(event) = &self.event {\n                    event.0.eval_jptr(pointer, results);\n                }\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl ResponsePtr for GetAvailabilityResponse {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"list\") => {\n                self.list.eval_jptr(pointer, results);\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl ResponsePtr for BusyPeriod {\n    fn eval_jptr(&self, mut pointer: JsonPointerIter<'_, Null>, results: &mut EvalResults) -> bool {\n        match pointer.next().and_then(|item| item.as_string_key()) {\n            Some(\"event\") => {\n                if let Some(event) = &self.event {\n                    event.0.eval_jptr(pointer, results);\n                }\n                true\n            }\n            _ => false,\n        }\n    }\n}\n\nimpl EvalResults {\n    pub fn into_ids<T: TryFrom<AnyId>>(\n        self,\n        rr: &ResultReference,\n    ) -> impl Iterator<Item = trc::Result<T>> {\n        self.0.into_iter().map(move |id| {\n            if let EvalResult::Id(any_id) = id {\n                T::try_from(any_id).map_err(|_| {\n                    trc::JmapEvent::InvalidResultReference\n                        .into_err()\n                        .details(format_compact!(\n                            \"Failed to evaluate {rr} result reference: Invalid Id type.\"\n                        ))\n                })\n            } else {\n                Err(trc::JmapEvent::InvalidResultReference\n                    .into_err()\n                    .details(format_compact!(\n                        \"Failed to evaluate {rr} result reference: Invalid Id type.\"\n                    )))\n            }\n        })\n    }\n\n    pub fn into_properties<T: Property + FromStr>(\n        self,\n        rr: &ResultReference,\n    ) -> impl Iterator<Item = trc::Result<T>> {\n        self.0.into_iter().map(move |prop| {\n            if let EvalResult::Property(prop) = prop {\n                T::from_str(&prop).map_err(|_| {\n                    trc::JmapEvent::InvalidResultReference\n                        .into_err()\n                        .details(format_compact!(\n                            \"Failed to evaluate {rr} result reference: Invalid property.\"\n                        ))\n                })\n            } else {\n                Err(trc::JmapEvent::InvalidResultReference\n                    .into_err()\n                    .details(format_compact!(\n                        \"Failed to evaluate {rr} result reference: Invalid property.\"\n                    )))\n            }\n        })\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/references/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse compact_str::format_compact;\nuse std::collections::HashMap;\nuse utils::map::vec_map::VecMap;\n\npub mod eval;\npub mod jsptr;\npub mod resolve;\n\npub(crate) enum Graph<'x> {\n    Some {\n        child_id: &'x str,\n        graph: &'x mut HashMap<String, Vec<String>>,\n    },\n    None,\n}\n\nfn topological_sort<T>(\n    create: &mut VecMap<String, T>,\n    graph: HashMap<String, Vec<String>>,\n) -> trc::Result<VecMap<String, T>> {\n    // Make sure all references exist\n    for (from_id, to_ids) in graph.iter() {\n        for to_id in to_ids {\n            if !create.contains_key(to_id) {\n                return Err(trc::JmapEvent::InvalidResultReference.into_err().details(\n                    format_compact!(\n                        \"Invalid reference to non-existing object {to_id:?} from {from_id:?}\"\n                    ),\n                ));\n            }\n        }\n    }\n\n    let mut sorted_create = VecMap::with_capacity(create.len());\n    let mut it_stack = Vec::new();\n    let keys = graph.keys().cloned().collect::<Vec<_>>();\n    let mut it = keys.iter();\n\n    'main: loop {\n        while let Some(from_id) = it.next() {\n            if let Some(to_ids) = graph.get(from_id) {\n                it_stack.push((it, from_id));\n                if it_stack.len() > 1000 {\n                    return Err(trc::JmapEvent::InvalidArguments\n                        .into_err()\n                        .details(\"Cyclical references are not allowed.\"));\n                }\n                it = to_ids.iter();\n                continue;\n            } else if let Some((id, value)) = create.remove_entry(from_id) {\n                sorted_create.append(id, value);\n                if create.is_empty() {\n                    break 'main;\n                }\n            }\n        }\n\n        if let Some((prev_it, from_id)) = it_stack.pop() {\n            it = prev_it;\n            if let Some((id, value)) = create.remove_entry(from_id) {\n                sorted_create.append(id, value);\n                if create.is_empty() {\n                    break 'main;\n                }\n            }\n        } else {\n            break;\n        }\n    }\n\n    // Add remaining items\n    if !create.is_empty() {\n        for (id, value) in std::mem::take(create) {\n            sorted_create.append(id, value);\n        }\n    }\n    Ok(sorted_create)\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{\n        method::{changes::ChangesResponse, get::GetResponse, query::QueryResponse},\n        object::{\n            email::{EmailProperty, EmailValue},\n            mailbox::{MailboxProperty, MailboxValue},\n            thread::{ThreadProperty, ThreadValue},\n        },\n        request::{\n            Call, GetRequestMethod, Request, RequestMethod, SetRequestMethod,\n            reference::{MaybeIdReference, MaybeResultReference},\n        },\n        response::{ChangesResponseMethod, GetResponseMethod, Response, ResponseMethod},\n    };\n    use jmap_tools::{Key, Map, Value};\n    use std::collections::HashMap;\n    use types::id::Id;\n\n    #[test]\n    fn eval_value_references() {\n        let request = Request::parse(\n            br##\"{\n                    \"using\":[\"urn:ietf:params:jmap:mail\"],\n                    \"methodCalls\": [[ \"Email/query\", {\n                            \"accountId\": \"a\",\n                            \"filter\": { \"inMailbox\": \"a\" },\n                            \"sort\": [{ \"property\": \"receivedAt\", \"isAscending\": false }],\n                            \"collapseThreads\": true,\n                            \"position\": 0,\n                            \"limit\": 10,\n                            \"calculateTotal\": true\n                        }, \"t0\" ],\n                        [ \"Email/get\", {\n                            \"accountId\": \"a\",\n                            \"#ids\": {\n                            \"resultOf\": \"t0\",\n                            \"name\": \"Email/query\",\n                            \"path\": \"/ids\"\n                            },\n                            \"properties\": [ \"threadId\" ]\n                        }, \"t1\" ],\n                        [ \"Thread/get\", {\n                            \"accountId\": \"a\",\n                            \"#ids\": {\n                            \"resultOf\": \"t1\",\n                            \"name\": \"Email/get\",\n                            \"path\": \"/list/*/threadId\"\n                            }\n                        }, \"t2\" ],\n                        [ \"Email/get\", {\n                            \"accountId\": \"a\",\n                            \"#ids\": {\n                            \"resultOf\": \"t2\",\n                            \"name\": \"Thread/get\",\n                            \"path\": \"/list/*/emailIds\"\n                            },\n                            \"properties\": [ \"from\", \"receivedAt\", \"subject\" ]\n                        }, \"t3\" ]]\n                    }\"##,\n            100,\n            1024 * 1024,\n        )\n        .unwrap();\n\n        let mut response = Response::new(\n            1234,\n            request.created_ids.unwrap_or_default(),\n            request.method_calls.len(),\n        );\n\n        assert_eq!(request.method_calls.len(), 4);\n\n        for (test_num, mut call) in request.method_calls.into_iter().enumerate() {\n            match test_num {\n                0 => {\n                    response.method_responses.push(Call {\n                        id: call.id,\n                        name: call.name,\n                        method: ResponseMethod::Query(QueryResponse {\n                            account_id: Id::new(1),\n                            query_state: Default::default(),\n                            can_calculate_changes: Default::default(),\n                            position: Default::default(),\n                            ids: vec![Id::new(4), Id::new(5)],\n                            total: Default::default(),\n                            limit: Default::default(),\n                        }),\n                    });\n                }\n                1 => {\n                    response.resolve_references(&mut call.method).unwrap();\n                    match call.method {\n                        RequestMethod::Get(GetRequestMethod::Email(req)) => {\n                            assert_eq!(\n                                req.ids,\n                                Some(MaybeResultReference::Value(vec![\n                                    MaybeIdReference::Id(Id::new(4)),\n                                    MaybeIdReference::Id(Id::new(5))\n                                ]))\n                            );\n                        }\n                        _ => panic!(\"Expected Email Get Request\"),\n                    }\n                    response.method_responses.push(Call {\n                        id: call.id,\n                        name: call.name,\n                        method: ResponseMethod::Get(GetResponseMethod::Email(GetResponse {\n                            account_id: Id::new(1).into(),\n                            state: Default::default(),\n                            list: vec![\n                                Value::Object(Map::from(vec![(\n                                    Key::Property(EmailProperty::ThreadId),\n                                    Value::Element(EmailValue::Id(Id::new(9))),\n                                )])),\n                                Value::Object(Map::from(vec![(\n                                    Key::Property(EmailProperty::ThreadId),\n                                    Value::Element(EmailValue::Id(Id::new(10))),\n                                )])),\n                            ],\n                            not_found: Default::default(),\n                        })),\n                    });\n                }\n                2 => {\n                    response.resolve_references(&mut call.method).unwrap();\n                    match call.method {\n                        RequestMethod::Get(GetRequestMethod::Thread(req)) => {\n                            assert_eq!(\n                                req.ids,\n                                Some(MaybeResultReference::Value(vec![\n                                    MaybeIdReference::Id(Id::new(9)),\n                                    MaybeIdReference::Id(Id::new(10))\n                                ]))\n                            );\n                        }\n                        _ => panic!(\"Expected Thread Get Request\"),\n                    }\n                    response.method_responses.push(Call {\n                        id: call.id,\n                        name: call.name,\n                        method: ResponseMethod::Get(GetResponseMethod::Thread(GetResponse {\n                            account_id: Id::new(1).into(),\n                            state: Default::default(),\n                            list: vec![\n                                Value::Object(Map::from(vec![(\n                                    Key::Property(ThreadProperty::EmailIds),\n                                    Value::Array(vec![\n                                        Value::Element(ThreadValue::Id(Id::new(100))),\n                                        Value::Element(ThreadValue::Id(Id::new(101))),\n                                    ]),\n                                )])),\n                                Value::Object(Map::from(vec![(\n                                    Key::Property(ThreadProperty::EmailIds),\n                                    Value::Array(vec![\n                                        Value::Element(ThreadValue::Id(Id::new(102))),\n                                        Value::Element(ThreadValue::Id(Id::new(103))),\n                                    ]),\n                                )])),\n                            ],\n                            not_found: Default::default(),\n                        })),\n                    });\n                }\n                3 => {\n                    response.resolve_references(&mut call.method).unwrap();\n                    match call.method {\n                        RequestMethod::Get(GetRequestMethod::Email(req)) => {\n                            assert_eq!(\n                                req.ids,\n                                Some(MaybeResultReference::Value(vec![\n                                    MaybeIdReference::Id(Id::new(100)),\n                                    MaybeIdReference::Id(Id::new(101)),\n                                    MaybeIdReference::Id(Id::new(102)),\n                                    MaybeIdReference::Id(Id::new(103)),\n                                ]))\n                            );\n                        }\n                        _ => panic!(\"Expected Mailbox Get Request\"),\n                    }\n                }\n                _ => panic!(\"Unexpected invocation {}\", test_num),\n            }\n        }\n    }\n\n    #[test]\n    fn eval_property_references() {\n        let request = Request::parse(\n            br##\"{\n                    \"using\":[\"urn:ietf:params:jmap:mail\"],\n                    \"methodCalls\": [\n                    [\"Mailbox/changes\",{\n                    \"accountId\":\"s\",\n                    \"sinceState\":\"srxqk071myhgkyay\"\n                    },\"0\"],\n                    [\"Mailbox/get\",{\n                    \"accountId\":\"s\",\n                    \"#ids\":{\"name\":\"Mailbox/changes\",\"path\":\"/created\",\"resultOf\":\"0\"}\n                    },\"1\"],\n                    [\"Mailbox/get\",{\n                    \"accountId\":\"s\",\n                    \"#ids\":{\"name\":\"Mailbox/changes\",\"path\":\"/updated\",\"resultOf\":\"0\"},\n                    \"#properties\":{\"name\":\"Mailbox/changes\",\"path\":\"/updatedProperties\",\"resultOf\":\"0\"}\n                    },\"2\"]\n                    ]\n                    }\"##,\n            100,\n            1024 * 1024,\n        )\n        .unwrap();\n\n        let mut response = Response::new(\n            1234,\n            request.created_ids.unwrap_or_default(),\n            request.method_calls.len(),\n        );\n\n        assert_eq!(request.method_calls.len(), 3);\n\n        for (test_num, mut call) in request.method_calls.into_iter().enumerate() {\n            match test_num {\n                0 => {\n                    response.method_responses.push(Call {\n                        id: call.id,\n                        name: call.name,\n                        method: ResponseMethod::Changes(ChangesResponseMethod::Mailbox(\n                            ChangesResponse {\n                                account_id: Id::new(1),\n                                old_state: Default::default(),\n                                new_state: Default::default(),\n                                has_more_changes: Default::default(),\n                                created: Default::default(),\n                                updated: vec![Id::new(2), Id::new(3)],\n                                destroyed: Default::default(),\n                                updated_properties: Some(vec![\n                                    MailboxProperty::Name.into(),\n                                    MailboxProperty::ParentId.into(),\n                                ]),\n                            },\n                        )),\n                    });\n                }\n                1 => {\n                    response.resolve_references(&mut call.method).unwrap();\n                    match call.method {\n                        RequestMethod::Get(GetRequestMethod::Mailbox(req)) => {\n                            assert_eq!(req.ids, Some(MaybeResultReference::Value(vec![])));\n                        }\n                        _ => panic!(\"Expected Mailbox Get Request\"),\n                    }\n                }\n                2 => {\n                    response.resolve_references(&mut call.method).unwrap();\n                    match call.method {\n                        RequestMethod::Get(GetRequestMethod::Mailbox(req)) => {\n                            assert_eq!(\n                                req.ids,\n                                Some(MaybeResultReference::Value(vec![\n                                    MaybeIdReference::Id(Id::new(2)),\n                                    MaybeIdReference::Id(Id::new(3))\n                                ]))\n                            );\n                        }\n                        _ => panic!(\"Expected Mailbox Get Request\"),\n                    }\n                }\n                _ => panic!(\"Unexpected invocation {}\", test_num),\n            }\n        }\n    }\n\n    #[test]\n    fn eval_create_references() {\n        let request = Request::parse(\n            br##\"{\n                    \"using\": [\n                        \"urn:ietf:params:jmap:core\",\n                        \"urn:ietf:params:jmap:mail\"\n                    ],\n                    \"methodCalls\": [\n                        [\n                            \"Mailbox/set\",\n                            {\n                                \"accountId\": \"b\",\n                                \"create\": {\n                                    \"a\": {\n                                        \"name\": \"Folder a\",\n                                        \"parentId\": \"#b\"\n                                    },\n                                    \"b\": {\n                                        \"name\": \"Folder b\",\n                                        \"parentId\": \"#c\"\n                                    },\n                                    \"c\": {\n                                        \"name\": \"Folder c\",\n                                        \"parentId\": \"#d\"\n                                    },\n                                    \"d\": {\n                                        \"name\": \"Folder d\",\n                                        \"parentId\": \"#e\"\n                                    },\n                                    \"e\": {\n                                        \"name\": \"Folder e\",\n                                        \"parentId\": \"#f\"\n                                    },\n                                    \"f\": {\n                                        \"name\": \"Folder f\",\n                                        \"parentId\": \"#g\"\n                                    },\n                                    \"g\": {\n                                        \"name\": \"Folder g\",\n                                        \"parentId\": null\n                                    }\n                                }\n                            },\n                            \"fulltree\"\n                        ],\n                        [\n                            \"Mailbox/set\",\n                            {\n                                \"accountId\": \"b\",\n                                \"create\": {\n                                    \"a1\": {\n                                        \"name\": \"Folder a1\",\n                                        \"parentId\": null\n                                    },\n                                    \"b2\": {\n                                        \"name\": \"Folder b2\",\n                                        \"parentId\": \"#a1\"\n                                    },\n                                    \"c3\": {\n                                        \"name\": \"Folder c3\",\n                                        \"parentId\": \"#a1\"\n                                    },\n                                    \"d4\": {\n                                        \"name\": \"Folder d4\",\n                                        \"parentId\": \"#b2\"\n                                    },\n                                    \"e5\": {\n                                        \"name\": \"Folder e5\",\n                                        \"parentId\": \"#b2\"\n                                    },\n                                    \"f6\": {\n                                        \"name\": \"Folder f6\",\n                                        \"parentId\": \"#d4\"\n                                    },\n                                    \"g7\": {\n                                        \"name\": \"Folder g7\",\n                                        \"parentId\": \"#e5\"\n                                    }\n                                }\n                            },\n                            \"fulltree2\"\n                        ],\n                        [\n                            \"Mailbox/set\",\n                            {\n                                \"accountId\": \"b\",\n                                \"create\": {\n                                    \"z\": {\n                                        \"name\": \"Folder Z\",\n                                        \"parentId\": \"#x\"\n                                    },\n                                    \"y\": {\n                                        \"name\": null\n                                    },\n                                    \"x\": {\n                                        \"name\": \"Folder X\"\n                                    }\n                                }\n                            },\n                            \"xyz\"\n                        ],\n                        [\n                            \"Mailbox/set\",\n                            {\n                                \"accountId\": \"b\",\n                                \"create\": {\n                                    \"a\": {\n                                        \"name\": \"Folder a\",\n                                        \"parentId\": \"#b\"\n                                    },\n                                    \"b\": {\n                                        \"name\": \"Folder b\",\n                                        \"parentId\": \"#c\"\n                                    },\n                                    \"c\": {\n                                        \"name\": \"Folder c\",\n                                        \"parentId\": \"#d\"\n                                    },\n                                    \"d\": {\n                                        \"name\": \"Folder d\",\n                                        \"parentId\": \"#a\"\n                                    }\n                                }\n                            },\n                            \"circular\"\n                        ]\n                    ]\n                }\"##,\n            100,\n            1024 * 1024,\n        )\n        .unwrap();\n\n        let response = Response::new(\n            1234,\n            request.created_ids.unwrap_or_default(),\n            request.method_calls.len(),\n        );\n\n        for (test_num, mut call) in request.method_calls.into_iter().enumerate() {\n            match response.resolve_references(&mut call.method) {\n                Ok(_) => assert!(\n                    (0..3).contains(&test_num),\n                    \"Unexpected invocation {}\",\n                    test_num\n                ),\n                Err(err) => {\n                    assert_eq!(test_num, 3);\n                    assert!(\n                        err.matches(trc::EventType::Jmap(trc::JmapEvent::InvalidArguments)),\n                        \"{:?}\",\n                        err\n                    );\n                    continue;\n                }\n            }\n\n            if let RequestMethod::Set(SetRequestMethod::Mailbox(request)) = call.method {\n                if test_num == 0 {\n                    assert_eq!(\n                        request\n                            .create\n                            .unwrap()\n                            .into_iter()\n                            .map(|b| b.0)\n                            .collect::<Vec<_>>(),\n                        [\"g\", \"f\", \"e\", \"d\", \"c\", \"b\", \"a\"]\n                            .iter()\n                            .map(|i| i.to_string())\n                            .collect::<Vec<_>>()\n                    );\n                } else if test_num == 1 {\n                    let mut pending_ids = vec![\"a1\", \"b2\", \"d4\", \"e5\", \"f6\", \"c3\", \"g7\"];\n\n                    for (id, _) in request.create.as_ref().unwrap() {\n                        match id.as_str() {\n                            \"a1\" => (),\n                            \"b2\" | \"c3\" => assert!(!pending_ids.contains(&\"a1\")),\n                            \"d4\" | \"e5\" => assert!(!pending_ids.contains(&\"b2\")),\n                            \"f6\" => assert!(!pending_ids.contains(&\"d4\")),\n                            \"g7\" => assert!(!pending_ids.contains(&\"e5\")),\n                            _ => panic!(\"Unexpected ID\"),\n                        }\n                        pending_ids.retain(|i| i != id);\n                    }\n\n                    if !pending_ids.is_empty() {\n                        panic!(\n                            \"Unexpected order: {:?}\",\n                            request\n                                .create\n                                .as_ref()\n                                .unwrap()\n                                .iter()\n                                .map(|b| b.0.to_string())\n                                .collect::<Vec<_>>()\n                        );\n                    }\n                } else if test_num == 2 {\n                    assert_eq!(\n                        request\n                            .create\n                            .unwrap()\n                            .into_iter()\n                            .map(|b| b.0)\n                            .collect::<Vec<_>>(),\n                        [\"x\", \"z\", \"y\"]\n                            .iter()\n                            .map(|i| i.to_string())\n                            .collect::<Vec<_>>()\n                    );\n                }\n            } else {\n                panic!(\"Expected Set Mailbox Request\");\n            }\n        }\n\n        let request = Request::parse(\n            br##\"{\n                \"using\": [\n                    \"urn:ietf:params:jmap:core\",\n                    \"urn:ietf:params:jmap:mail\"\n                ],\n                \"methodCalls\": [\n                    [\n                        \"Mailbox/set\",\n                        {\n                            \"accountId\": \"b\",\n                            \"create\": {\n                                \"a\": {\n                                    \"name\": \"a\",\n                                    \"parentId\": \"#x\"\n                                },\n                                \"b\": {\n                                    \"name\": \"b\",\n                                    \"parentId\": \"#y\"\n                                },\n                                \"c\": {\n                                    \"name\": \"c\",\n                                    \"parentId\": \"#z\"\n                                }\n                            }\n                        },\n                        \"ref1\"\n                    ],\n                    [\n                        \"Mailbox/set\",\n                        {\n                            \"accountId\": \"b\",\n                            \"create\": {\n                                \"a1\": {\n                                    \"name\": \"a1\",\n                                    \"parentId\": \"#a\"\n                                },\n                                \"b2\": {\n                                    \"name\": \"b2\",\n                                    \"parentId\": \"#b\"\n                                },\n                                \"c3\": {\n                                    \"name\": \"c3\",\n                                    \"parentId\": \"#c\"\n                                }\n                            }\n                        },\n                        \"red2\"\n                    ]\n                ],\n                \"createdIds\": {\n                    \"x\": \"b\",\n                    \"y\": \"c\",\n                    \"z\": \"d\"\n                }\n            }\"##,\n            1024,\n            1024 * 1024,\n        )\n        .unwrap();\n\n        let mut response = Response::new(\n            1234,\n            request.created_ids.unwrap_or_default(),\n            request.method_calls.len(),\n        );\n\n        let mut invocations = request.method_calls.into_iter();\n        let mut call = invocations.next().unwrap();\n        response.resolve_references(&mut call.method).unwrap();\n\n        if let RequestMethod::Set(SetRequestMethod::Mailbox(request)) = call.method {\n            let create = request\n                .create\n                .as_ref()\n                .unwrap()\n                .iter()\n                .map(|(p, v)| {\n                    (\n                        p.as_str(),\n                        v.as_object()\n                            .unwrap()\n                            .get(&Key::Property(MailboxProperty::ParentId))\n                            .unwrap(),\n                    )\n                })\n                .collect::<HashMap<_, _>>();\n            assert_eq!(\n                *create.get(\"a\").unwrap(),\n                &Value::Element(MailboxValue::Id(Id::new(1)))\n            );\n            assert_eq!(\n                *create.get(\"b\").unwrap(),\n                &Value::Element(MailboxValue::Id(Id::new(2)))\n            );\n            assert_eq!(\n                *create.get(\"c\").unwrap(),\n                &Value::Element(MailboxValue::Id(Id::new(3)))\n            );\n        } else {\n            panic!(\"Expected Mailbox Set Request\");\n        }\n\n        response\n            .created_ids\n            .insert(\"a\".to_string(), Id::new(5).into());\n        response\n            .created_ids\n            .insert(\"b\".to_string(), Id::new(6).into());\n        response\n            .created_ids\n            .insert(\"c\".to_string(), Id::new(7).into());\n\n        let mut call = invocations.next().unwrap();\n        response.resolve_references(&mut call.method).unwrap();\n\n        if let RequestMethod::Set(SetRequestMethod::Mailbox(request)) = call.method {\n            let create = request\n                .create\n                .as_ref()\n                .unwrap()\n                .iter()\n                .map(|(p, v)| {\n                    (\n                        p.as_str(),\n                        v.as_object()\n                            .unwrap()\n                            .get(&Key::Property(MailboxProperty::ParentId))\n                            .unwrap(),\n                    )\n                })\n                .collect::<HashMap<_, _>>();\n            assert_eq!(\n                *create.get(\"a1\").unwrap(),\n                &Value::Element(MailboxValue::Id(Id::new(5)))\n            );\n            assert_eq!(\n                *create.get(\"b2\").unwrap(),\n                &Value::Element(MailboxValue::Id(Id::new(6)))\n            );\n            assert_eq!(\n                *create.get(\"c3\").unwrap(),\n                &Value::Element(MailboxValue::Id(Id::new(7)))\n            );\n        } else {\n            panic!(\"Expected Mailbox Set Request\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/references/resolve.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    error::set::SetError,\n    method::{\n        copy::CopyRequest,\n        get::GetRequest,\n        import::ImportEmailRequest,\n        parse::ParseRequest,\n        search_snippet::GetSearchSnippetRequest,\n        set::{SetRequest, SetResponse},\n        upload::{BlobUploadRequest, DataSourceObject},\n    },\n    object::{AnyId, JmapObject, JmapObjectId},\n    references::{Graph, eval::EvalObjectReferences, topological_sort},\n    request::{\n        CopyRequestMethod, GetRequestMethod, MaybeInvalid, ParseRequestMethod, RequestMethod,\n        SetRequestMethod,\n        reference::{MaybeIdReference, MaybeResultReference},\n    },\n    response::Response,\n};\nuse compact_str::format_compact;\nuse jmap_tools::{Element, Key, Property, Value};\nuse std::collections::HashMap;\nuse types::id::Id;\n\nimpl Response<'_> {\n    pub fn resolve_references(&self, request: &mut RequestMethod) -> trc::Result<()> {\n        match request {\n            RequestMethod::Get(request) => match request {\n                GetRequestMethod::Email(request) => request.resolve_references(self)?,\n                GetRequestMethod::Mailbox(request) => request.resolve_references(self)?,\n                GetRequestMethod::Thread(request) => request.resolve_references(self)?,\n                GetRequestMethod::Identity(request) => request.resolve_references(self)?,\n                GetRequestMethod::EmailSubmission(request) => request.resolve_references(self)?,\n                GetRequestMethod::PushSubscription(request) => request.resolve_references(self)?,\n                GetRequestMethod::Sieve(request) => request.resolve_references(self)?,\n                GetRequestMethod::VacationResponse(request) => request.resolve_references(self)?,\n                GetRequestMethod::Principal(request) => request.resolve_references(self)?,\n                GetRequestMethod::Quota(request) => request.resolve_references(self)?,\n                GetRequestMethod::Blob(request) => request.resolve_references(self)?,\n                GetRequestMethod::AddressBook(request) => request.resolve_references(self)?,\n                GetRequestMethod::ContactCard(request) => request.resolve_references(self)?,\n                GetRequestMethod::FileNode(request) => request.resolve_references(self)?,\n                GetRequestMethod::ShareNotification(request) => request.resolve_references(self)?,\n                GetRequestMethod::Calendar(request) => request.resolve_references(self)?,\n                GetRequestMethod::CalendarEvent(request) => request.resolve_references(self)?,\n                GetRequestMethod::CalendarEventNotification(request) => {\n                    request.resolve_references(self)?\n                }\n                GetRequestMethod::ParticipantIdentity(request) => {\n                    request.resolve_references(self)?\n                }\n                GetRequestMethod::PrincipalAvailability(_) => (),\n            },\n            RequestMethod::Set(request) => match request {\n                SetRequestMethod::Email(request) => request.resolve_references(self)?,\n                SetRequestMethod::Mailbox(request) => request.resolve_references(self)?,\n                SetRequestMethod::Identity(request) => request.resolve_references(self)?,\n                SetRequestMethod::EmailSubmission(request) => request.resolve_references(self)?,\n                SetRequestMethod::PushSubscription(request) => request.resolve_references(self)?,\n                SetRequestMethod::Sieve(request) => request.resolve_references(self)?,\n                SetRequestMethod::VacationResponse(request) => request.resolve_references(self)?,\n                SetRequestMethod::AddressBook(request) => request.resolve_references(self)?,\n                SetRequestMethod::ContactCard(request) => request.resolve_references(self)?,\n                SetRequestMethod::FileNode(request) => request.resolve_references(self)?,\n                SetRequestMethod::ShareNotification(request) => request.resolve_references(self)?,\n                SetRequestMethod::Calendar(request) => request.resolve_references(self)?,\n                SetRequestMethod::CalendarEvent(request) => request.resolve_references(self)?,\n                SetRequestMethod::CalendarEventNotification(request) => {\n                    request.resolve_references(self)?\n                }\n                SetRequestMethod::ParticipantIdentity(request) => {\n                    request.resolve_references(self)?\n                }\n            },\n            RequestMethod::Copy(request) => match request {\n                CopyRequestMethod::Email(request) => request.resolve_references(self)?,\n                CopyRequestMethod::CalendarEvent(request) => request.resolve_references(self)?,\n                CopyRequestMethod::ContactCard(request) => request.resolve_references(self)?,\n                CopyRequestMethod::Blob(_) => (),\n            },\n            RequestMethod::ImportEmail(request) => request.resolve_references(self)?,\n            RequestMethod::SearchSnippet(request) => request.resolve_references(self)?,\n            RequestMethod::UploadBlob(request) => request.resolve_references(self)?,\n            RequestMethod::Parse(request) => match request {\n                ParseRequestMethod::Email(request) => request.resolve_references(self)?,\n                ParseRequestMethod::ContactCard(request) => request.resolve_references(self)?,\n                ParseRequestMethod::CalendarEvent(request) => request.resolve_references(self)?,\n            },\n            _ => {}\n        }\n\n        Ok(())\n    }\n}\n\npub trait ResolveCreatedReference<P, E>\nwhere\n    P: Property,\n    E: Element<Property = P> + JmapObjectId,\n{\n    fn get_created_id(&self, id_ref: &str) -> Option<AnyId>;\n\n    fn resolve_self_references(&self, value: &mut Value<'_, P, E>) -> Result<(), SetError<P>> {\n        match value {\n            Value::Element(element) => {\n                if let Some(id_ref) = element.as_id_ref() {\n                    if let Some(id) = self.get_created_id(id_ref) {\n                        if !element.try_set_id(id) {\n                            return Err(SetError::invalid_properties()\n                                .with_description(\"Id reference points to invalid type.\"));\n                        }\n                    } else {\n                        return Err(SetError::not_found()\n                            .with_description(format!(\"Id reference {id_ref:?} not found.\")));\n                    }\n                }\n            }\n            Value::Array(items) => {\n                for item in items {\n                    self.resolve_self_references(item)?;\n                }\n            }\n            _ => {}\n        }\n\n        Ok(())\n    }\n}\n\npub(crate) trait ResolveReference {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()>;\n}\n\nimpl<T: JmapObject> ResolveReference for GetRequest<T> {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {\n        // Resolve id references\n        match &mut self.ids {\n            Some(MaybeResultReference::Reference(reference)) => {\n                self.ids = Some(MaybeResultReference::Value(\n                    response\n                        .eval_result_references(reference)?\n                        .into_ids::<T::Id>(reference)\n                        .map(|f| f.map(MaybeIdReference::Id))\n                        .collect::<Result<_, _>>()?,\n                ));\n            }\n            Some(MaybeResultReference::Value(ids)) => {\n                for id in ids {\n                    if let MaybeIdReference::Reference(reference) = id {\n                        if let Some(resolved_id) = response\n                            .created_ids\n                            .get(reference)\n                            .cloned()\n                            .and_then(|v| T::Id::try_from(v).ok())\n                        {\n                            *id = MaybeIdReference::Id(resolved_id);\n                        } else {\n                            return Err(trc::JmapEvent::InvalidResultReference.into_err().details(\n                                format_compact!(\n                                    \"Id reference {reference:?} does not exist or is invalid.\"\n                                ),\n                            ));\n                        }\n                    }\n                }\n            }\n            _ => (),\n        }\n\n        // Resolve properties references\n        if let Some(MaybeResultReference::Reference(reference)) = &self.properties {\n            self.properties = Some(MaybeResultReference::Value(\n                response\n                    .eval_result_references(reference)?\n                    .into_properties::<T::Property>(reference)\n                    .map(|f| f.map(MaybeInvalid::Value))\n                    .collect::<Result<_, _>>()?,\n            ));\n        }\n\n        Ok(())\n    }\n}\n\nimpl<'x, T: JmapObject> ResolveReference for SetRequest<'x, T> {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {\n        // Resolve create references\n        if let Some(create) = &mut self.create {\n            let mut graph = HashMap::with_capacity(create.len());\n            for (id, obj) in create.iter_mut() {\n                obj.eval_object_references(\n                    response,\n                    &mut Graph::Some {\n                        child_id: &*id,\n                        graph: &mut graph,\n                    },\n                    0,\n                )?;\n            }\n\n            // Perform topological sort\n            if !graph.is_empty() {\n                self.create = topological_sort(create, graph)?.into();\n            }\n        }\n\n        // Resolve update references\n        if let Some(update) = &mut self.update {\n            for obj in update.values_mut() {\n                obj.eval_object_references(response, &mut Graph::None, 0)?;\n            }\n        }\n\n        // Resolve destroy references\n        if let Some(MaybeResultReference::Reference(reference)) = &self.destroy {\n            self.destroy = Some(MaybeResultReference::Value(\n                response\n                    .eval_result_references(reference)?\n                    .into_ids::<Id>(reference)\n                    .map(|f| f.map(MaybeInvalid::Value))\n                    .collect::<Result<_, _>>()?,\n            ));\n        }\n\n        Ok(())\n    }\n}\n\nimpl<'x, T: JmapObject> ResolveReference for CopyRequest<'x, T> {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {\n        // Resolve create references\n        for (id, obj) in self.create.iter_mut() {\n            obj.eval_object_references(response, &mut Graph::None, 0)?;\n\n            if let MaybeIdReference::Reference(ir) = id {\n                *id = MaybeIdReference::Id(response.eval_id_reference(ir)?);\n            }\n        }\n\n        Ok(())\n    }\n}\n\nimpl<T: JmapObject> ResolveReference for ParseRequest<T> {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {\n        // Resolve blobId references\n        for id in self.blob_ids.iter_mut() {\n            if let MaybeIdReference::Reference(ir) = id {\n                *id = MaybeIdReference::Id(response.eval_blob_id_reference(ir)?);\n            }\n        }\n\n        Ok(())\n    }\n}\n\nimpl ResolveReference for ImportEmailRequest {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {\n        // Resolve email mailbox references\n        for email in self.emails.values_mut() {\n            match &mut email.mailbox_ids {\n                MaybeResultReference::Reference(reference) => {\n                    email.mailbox_ids = MaybeResultReference::Value(\n                        response\n                            .eval_result_references(reference)?\n                            .into_ids::<Id>(reference)\n                            .map(|f| f.map(MaybeIdReference::Id))\n                            .collect::<Result<_, _>>()?,\n                    );\n                }\n                MaybeResultReference::Value(values) => {\n                    for value in values {\n                        if let MaybeIdReference::Reference(ir) = value {\n                            *value = MaybeIdReference::Id(response.eval_id_reference(ir)?);\n                        }\n                    }\n                }\n            }\n        }\n\n        Ok(())\n    }\n}\n\nimpl ResolveReference for GetSearchSnippetRequest {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {\n        // Resolve emailIds references\n        if let MaybeResultReference::Reference(reference) = &self.email_ids {\n            self.email_ids = MaybeResultReference::Value(\n                response\n                    .eval_result_references(reference)?\n                    .into_ids::<Id>(reference)\n                    .map(|f| f.map(MaybeInvalid::Value))\n                    .collect::<Result<_, _>>()?,\n            );\n        }\n\n        Ok(())\n    }\n}\n\nimpl ResolveReference for BlobUploadRequest {\n    fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {\n        let mut graph = HashMap::with_capacity(self.create.len());\n        for (create_id, object) in self.create.iter_mut() {\n            for data in &mut object.data {\n                if let DataSourceObject::Id { id, .. } = data\n                    && let MaybeIdReference::Reference(parent_id) = id\n                {\n                    match response.created_ids.get(parent_id) {\n                        Some(AnyId::BlobId(blob_id)) => {\n                            *id = MaybeIdReference::Id(blob_id.clone());\n                        }\n                        Some(_) => {\n                            return Err(trc::JmapEvent::InvalidResultReference.into_err().details(\n                                format_compact!(\n                                    \"Id reference {parent_id:?} points to invalid type.\"\n                                ),\n                            ));\n                        }\n                        None => {\n                            graph\n                                .entry(create_id.to_string())\n                                .or_insert_with(Vec::new)\n                                .push(parent_id.to_string());\n                        }\n                    }\n                }\n            }\n        }\n\n        // Perform topological sort\n        if !graph.is_empty() {\n            self.create = topological_sort(&mut self.create, graph)?;\n        }\n\n        Ok(())\n    }\n}\n\nimpl<T> ResolveCreatedReference<T::Property, T::Element> for SetResponse<T>\nwhere\n    T: JmapObject,\n{\n    fn get_created_id(&self, id_ref: &str) -> Option<AnyId> {\n        self.created\n            .get(id_ref)\n            .and_then(|v| v.as_object())\n            .and_then(|v| v.get(&Key::Property(T::ID_PROPERTY)))\n            .and_then(|v| v.as_element())\n            .and_then(|v| v.as_any_id())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/request/capability.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt;\n\nuse crate::{\n    object::{email::EmailComparator, file_node::FileNodeComparator},\n    response::serialize::serialize_hex,\n    types::date::UTCDate,\n};\nuse ahash::AHashMap;\nuse serde::{Deserialize, Deserializer};\nuse types::{id::Id, type_state::DataType};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct Session {\n    #[serde(rename(serialize = \"capabilities\"))]\n    pub capabilities: VecMap<Capability, Capabilities>,\n    #[serde(rename(serialize = \"accounts\"))]\n    pub accounts: VecMap<Id, Account>,\n    #[serde(rename(serialize = \"primaryAccounts\"))]\n    pub primary_accounts: VecMap<Capability, Id>,\n    #[serde(rename(serialize = \"username\"))]\n    pub username: String,\n    #[serde(rename(serialize = \"apiUrl\"))]\n    pub api_url: String,\n    #[serde(rename(serialize = \"downloadUrl\"))]\n    pub download_url: String,\n    #[serde(rename(serialize = \"uploadUrl\"))]\n    pub upload_url: String,\n    #[serde(rename(serialize = \"eventSourceUrl\"))]\n    pub event_source_url: String,\n    #[serde(rename(serialize = \"state\"))]\n    #[serde(serialize_with = \"serialize_hex\")]\n    pub state: u32,\n    #[serde(skip)]\n    pub base_url: String,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct Account {\n    #[serde(rename(serialize = \"name\"))]\n    pub name: String,\n    #[serde(rename(serialize = \"isPersonal\"))]\n    pub is_personal: bool,\n    #[serde(rename(serialize = \"isReadOnly\"))]\n    pub is_read_only: bool,\n    #[serde(rename(serialize = \"accountCapabilities\"))]\n    pub account_capabilities: VecMap<Capability, Capabilities>,\n}\n\n#[derive(Debug, Clone, Copy, serde::Serialize, Hash, PartialEq, Eq, PartialOrd, Ord)]\npub enum Capability {\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:core\"))]\n    Core = 1 << 0,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:mail\"))]\n    Mail = 1 << 1,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:submission\"))]\n    Submission = 1 << 2,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:vacationresponse\"))]\n    VacationResponse = 1 << 3,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:contacts\"))]\n    Contacts = 1 << 4,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:contacts:parse\"))]\n    ContactsParse = 1 << 5,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:calendars\"))]\n    Calendars = 1 << 6,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:calendars:parse\"))]\n    CalendarsParse = 1 << 7,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:websocket\"))]\n    WebSocket = 1 << 8,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:sieve\"))]\n    Sieve = 1 << 9,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:blob\"))]\n    Blob = 1 << 10,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:quota\"))]\n    Quota = 1 << 11,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:principals\"))]\n    Principals = 1 << 12,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:principals:owner\"))]\n    PrincipalsOwner = 1 << 13,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:principals:availability\"))]\n    PrincipalsAvailability = 1 << 14,\n    #[serde(rename(serialize = \"urn:ietf:params:jmap:filenode\"))]\n    FileNode = 1 << 15,\n}\n\n#[derive(Debug, Clone, Copy, Default)]\n#[repr(transparent)]\npub struct CapabilityIds(pub u32);\n\n#[derive(Debug, Clone, serde::Serialize)]\n#[serde(untagged)]\n#[allow(dead_code)]\npub enum Capabilities {\n    Core(CoreCapabilities),\n    Mail(MailCapabilities),\n    Submission(SubmissionCapabilities),\n    WebSocket(WebSocketCapabilities),\n    SieveAccount(SieveAccountCapabilities),\n    SieveSession(SieveSessionCapabilities),\n    Blob(BlobCapabilities),\n    Contacts(ContactsCapabilities),\n    Principals(PrincipalCapabilities),\n    PrincipalsAvailability(PrincipalAvailabilityCapabilities),\n    Calendar(CalendarCapabilities),\n    FileNode(FileNodeCapabilities),\n    Empty(EmptyCapabilities),\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct CoreCapabilities {\n    #[serde(rename(serialize = \"maxSizeUpload\"))]\n    pub max_size_upload: usize,\n    #[serde(rename(serialize = \"maxConcurrentUpload\"))]\n    pub max_concurrent_upload: usize,\n    #[serde(rename(serialize = \"maxSizeRequest\"))]\n    pub max_size_request: usize,\n    #[serde(rename(serialize = \"maxConcurrentRequests\"))]\n    pub max_concurrent_requests: usize,\n    #[serde(rename(serialize = \"maxCallsInRequest\"))]\n    pub max_calls_in_request: usize,\n    #[serde(rename(serialize = \"maxObjectsInGet\"))]\n    pub max_objects_in_get: usize,\n    #[serde(rename(serialize = \"maxObjectsInSet\"))]\n    pub max_objects_in_set: usize,\n    #[serde(rename(serialize = \"collationAlgorithms\"))]\n    pub collation_algorithms: Vec<String>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct WebSocketCapabilities {\n    #[serde(rename(serialize = \"url\"))]\n    pub url: String,\n    #[serde(rename(serialize = \"supportsPush\"))]\n    pub supports_push: bool,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct SieveSessionCapabilities {\n    #[serde(rename(serialize = \"implementation\"))]\n    pub implementation: &'static str,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct SieveAccountCapabilities {\n    #[serde(rename(serialize = \"maxSizeScriptName\"))]\n    pub max_script_name: usize,\n    #[serde(rename(serialize = \"maxSizeScript\"))]\n    pub max_script_size: usize,\n    #[serde(rename(serialize = \"maxNumberScripts\"))]\n    pub max_scripts: usize,\n    #[serde(rename(serialize = \"maxNumberRedirects\"))]\n    pub max_redirects: usize,\n    #[serde(rename(serialize = \"sieveExtensions\"))]\n    pub extensions: Vec<String>,\n    #[serde(rename(serialize = \"notificationMethods\"))]\n    pub notification_methods: Option<Vec<String>>,\n    #[serde(rename(serialize = \"externalLists\"))]\n    pub ext_lists: Option<Vec<String>>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct MailCapabilities {\n    #[serde(rename(serialize = \"maxMailboxesPerEmail\"))]\n    pub max_mailboxes_per_email: Option<usize>,\n    #[serde(rename(serialize = \"maxMailboxDepth\"))]\n    pub max_mailbox_depth: usize,\n    #[serde(rename(serialize = \"maxSizeMailboxName\"))]\n    pub max_size_mailbox_name: usize,\n    #[serde(rename(serialize = \"maxSizeAttachmentsPerEmail\"))]\n    pub max_size_attachments_per_email: usize,\n    #[serde(rename(serialize = \"emailQuerySortOptions\"))]\n    pub email_query_sort_options: Vec<EmailComparator>,\n    #[serde(rename(serialize = \"mayCreateTopLevelMailbox\"))]\n    pub may_create_top_level_mailbox: bool,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct SubmissionCapabilities {\n    #[serde(rename(serialize = \"maxDelayedSend\"))]\n    pub max_delayed_send: usize,\n    #[serde(rename(serialize = \"submissionExtensions\"))]\n    pub submission_extensions: VecMap<String, Vec<String>>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct BlobCapabilities {\n    #[serde(rename(serialize = \"maxSizeBlobSet\"))]\n    pub max_size_blob_set: usize,\n    #[serde(rename(serialize = \"maxDataSources\"))]\n    pub max_data_sources: usize,\n    #[serde(rename(serialize = \"supportedTypeNames\"))]\n    pub supported_type_names: Vec<DataType>,\n    #[serde(rename(serialize = \"supportedDigestAlgorithms\"))]\n    pub supported_digest_algorithms: Vec<&'static str>,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct CalendarCapabilities {\n    #[serde(rename(serialize = \"maxCalendarsPerEvent\"))]\n    pub max_calendars_per_event: Option<usize>,\n    #[serde(rename(serialize = \"minDateTime\"))]\n    pub min_date_time: UTCDate,\n    #[serde(rename(serialize = \"maxDateTime\"))]\n    pub max_date_time: UTCDate,\n    #[serde(rename(serialize = \"maxExpandedQueryDuration\"))]\n    pub max_expanded_query_duration: String,\n    #[serde(rename(serialize = \"maxParticipantsPerEvent\"))]\n    pub max_participants_per_event: Option<usize>,\n    #[serde(rename(serialize = \"mayCreateCalendar\"))]\n    pub may_create_calendar: bool,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct ContactsCapabilities {\n    #[serde(rename(serialize = \"maxAddressBooksPerCard\"))]\n    pub max_address_books_per_card: Option<usize>,\n    #[serde(rename(serialize = \"mayCreateAddressBook\"))]\n    pub may_create_address_book: bool,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct PrincipalAvailabilityCapabilities {\n    #[serde(rename(serialize = \"maxAvailabilityDuration\"))]\n    pub max_availability_duration: String,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct PrincipalCapabilities {\n    #[serde(rename(serialize = \"currentUserPrincipalId\"))]\n    pub current_user_principal_id: Option<Id>,\n}\n\n/*#[derive(Debug, Clone, serde::Serialize)]\npub struct PrincipalOwnerCapabilities {\n    #[serde(rename(serialize = \"accountIdForPrincipal\"))]\n    pub account_id_for_principal: Id,\n\n    #[serde(rename(serialize = \"principalId\"))]\n    pub principal_id: Id,\n}\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct PrincipalCalendarCapabilities {\n    #[serde(rename(serialize = \"accountIdForPrincipal\"))]\n    pub account_id_for_principal: Option<Id>,\n    #[serde(rename(serialize = \"mayGetAvailability\"))]\n    pub may_get_availability: bool,\n    #[serde(rename(serialize = \"mayShareWith\"))]\n    pub may_share_with: bool,\n    #[serde(rename(serialize = \"calendarAddress\"))]\n    pub calendar_address: String,\n}*/\n\n#[derive(Debug, Clone, serde::Serialize)]\npub struct FileNodeCapabilities {\n    #[serde(rename(serialize = \"maxFileNodeDepth\"))]\n    pub max_file_node_depth: Option<usize>,\n    #[serde(rename(serialize = \"maxSizeFileNodeName\"))]\n    pub max_size_file_node_name: usize,\n    #[serde(rename(serialize = \"fileNodeQuerySortOptions\"))]\n    pub file_node_query_sort_options: Vec<FileNodeComparator>,\n    #[serde(rename(serialize = \"mayCreateTopLevelFileNode\"))]\n    pub may_create_top_level_file_node: bool,\n}\n\n#[derive(Debug, Clone, Default, serde::Serialize)]\npub struct EmptyCapabilities {}\n\n#[derive(Default, Clone)]\npub struct BaseCapabilities {\n    pub session: VecMap<Capability, Capabilities>,\n    pub account: AHashMap<Capability, Capabilities>,\n}\n\nimpl Capability {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Capability::Core => \"urn:ietf:params:jmap:core\",\n            Capability::Mail => \"urn:ietf:params:jmap:mail\",\n            Capability::Submission => \"urn:ietf:params:jmap:submission\",\n            Capability::VacationResponse => \"urn:ietf:params:jmap:vacationresponse\",\n            Capability::Contacts => \"urn:ietf:params:jmap:contacts\",\n            Capability::ContactsParse => \"urn:ietf:params:jmap:contacts:parse\",\n            Capability::Calendars => \"urn:ietf:params:jmap:calendars\",\n            Capability::CalendarsParse => \"urn:ietf:params:jmap:calendars:parse\",\n            Capability::WebSocket => \"urn:ietf:params:jmap:websocket\",\n            Capability::Sieve => \"urn:ietf:params:jmap:sieve\",\n            Capability::Blob => \"urn:ietf:params:jmap:blob\",\n            Capability::Quota => \"urn:ietf:params:jmap:quota\",\n            Capability::Principals => \"urn:ietf:params:jmap:principals\",\n            Capability::PrincipalsOwner => \"urn:ietf:params:jmap:principals:owner\",\n            Capability::PrincipalsAvailability => \"urn:ietf:params:jmap:principals:availability\",\n            Capability::FileNode => \"urn:ietf:params:jmap:filenode\",\n        }\n    }\n\n    pub fn all_capabilities() -> &'static [Capability] {\n        &[\n            Capability::Core,\n            Capability::Mail,\n            Capability::Submission,\n            Capability::VacationResponse,\n            Capability::Contacts,\n            Capability::ContactsParse,\n            Capability::Calendars,\n            Capability::CalendarsParse,\n            Capability::WebSocket,\n            Capability::Sieve,\n            Capability::Blob,\n            Capability::Quota,\n            Capability::Principals,\n            Capability::PrincipalsAvailability,\n            Capability::FileNode,\n        ]\n    }\n}\n\nimpl Session {\n    pub fn new(base_url: impl Into<String>, base_capabilities: &BaseCapabilities) -> Session {\n        let base_url = base_url.into();\n        let mut capabilities = base_capabilities.session.clone();\n        capabilities.append(\n            Capability::WebSocket,\n            Capabilities::WebSocket(WebSocketCapabilities::new(&base_url)),\n        );\n\n        Session {\n            capabilities,\n            accounts: VecMap::new(),\n            primary_accounts: VecMap::new(),\n            username: \"\".to_string(),\n            api_url: format!(\"{}/jmap/\", base_url),\n            download_url: format!(\n                \"{}/jmap/download/{{accountId}}/{{blobId}}/{{name}}?accept={{type}}\",\n                base_url\n            ),\n            upload_url: format!(\"{}/jmap/upload/{{accountId}}/\", base_url),\n            event_source_url: format!(\n                \"{}/jmap/eventsource/?types={{types}}&closeafter={{closeafter}}&ping={{ping}}\",\n                base_url\n            ),\n            base_url,\n            state: 0,\n        }\n    }\n\n    pub fn set_state(&mut self, state: u32) {\n        self.state = state;\n    }\n\n    pub fn api_url(&self) -> &str {\n        &self.api_url\n    }\n\n    pub fn base_url(&self) -> &str {\n        &self.base_url\n    }\n}\n\nimpl Default for SieveSessionCapabilities {\n    fn default() -> Self {\n        Self {\n            implementation: \"Stalwart v1.0.0\",\n        }\n    }\n}\n\nimpl WebSocketCapabilities {\n    pub fn new(base_url: &str) -> Self {\n        WebSocketCapabilities {\n            url: format!(\n                \"ws{}/jmap/ws\",\n                base_url.strip_prefix(\"http\").unwrap_or_default()\n            ),\n            supports_push: true,\n        }\n    }\n}\n\nimpl Capabilities {\n    pub fn to_account_capabilities(\n        &self,\n        current_user_principal_id: Option<Id>,\n        may_create: bool,\n    ) -> Capabilities {\n        match self {\n            Capabilities::Contacts(contacts_capabilities) => {\n                Capabilities::Contacts(ContactsCapabilities {\n                    may_create_address_book: may_create,\n                    ..contacts_capabilities.clone()\n                })\n            }\n            Capabilities::Principals(_) => Capabilities::Principals(PrincipalCapabilities {\n                current_user_principal_id,\n            }),\n            Capabilities::Calendar(calendar_capabilities) => {\n                Capabilities::Calendar(CalendarCapabilities {\n                    may_create_calendar: may_create,\n                    ..calendar_capabilities.clone()\n                })\n            }\n            Capabilities::FileNode(file_node_capabilities) => {\n                Capabilities::FileNode(FileNodeCapabilities {\n                    may_create_top_level_file_node: may_create,\n                    ..file_node_capabilities.clone()\n                })\n            }\n            _ => self.clone(),\n        }\n    }\n}\n\nimpl Capability {\n    pub fn parse(s: &str) -> Option<Self> {\n        hashify::tiny_map!(s.as_bytes(),\n            \"urn:ietf:params:jmap:core\" => Capability::Core,\n            \"urn:ietf:params:jmap:mail\" => Capability::Mail,\n            \"urn:ietf:params:jmap:submission\" => Capability::Submission,\n            \"urn:ietf:params:jmap:vacationresponse\" => Capability::VacationResponse,\n            \"urn:ietf:params:jmap:contacts\" => Capability::Contacts,\n            \"urn:ietf:params:jmap:calendars\" => Capability::Calendars,\n            \"urn:ietf:params:jmap:websocket\" => Capability::WebSocket,\n            \"urn:ietf:params:jmap:sieve\" => Capability::Sieve,\n            \"urn:ietf:params:jmap:blob\" => Capability::Blob,\n            \"urn:ietf:params:jmap:quota\" => Capability::Quota,\n            \"urn:ietf:params:jmap:principals\" => Capability::Principals,\n            \"urn:ietf:params:jmap:principals:owner\" => Capability::PrincipalsOwner,\n            \"urn:ietf:params:jmap:filenode\" => Capability::FileNode,\n            \"urn:ietf:params:jmap:principals:availability\" => Capability::PrincipalsAvailability,\n            \"urn:ietf:params:jmap:contacts:parse\" => Capability::ContactsParse,\n            \"urn:ietf:params:jmap:calendars:parse\" => Capability::CalendarsParse,\n        )\n    }\n}\n\nimpl<'de> Deserialize<'de> for CapabilityIds {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        struct CapabilityIdsVisitor;\n\n        impl<'de> serde::de::Visitor<'de> for CapabilityIdsVisitor {\n            type Value = CapabilityIds;\n\n            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n                formatter.write_str(\"an array of capability strings\")\n            }\n\n            fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>\n            where\n                A: serde::de::SeqAccess<'de>,\n            {\n                let mut capability_flags = 0u32;\n\n                while let Some(capability_str) = seq.next_element::<&str>()? {\n                    let capability = Capability::parse(capability_str).ok_or_else(|| {\n                        serde::de::Error::custom(format!(\"Unknown capability: {capability_str:?}\"))\n                    })?;\n\n                    capability_flags |= capability as u32;\n                }\n\n                Ok(CapabilityIds(capability_flags))\n            }\n        }\n\n        deserializer.deserialize_seq(CapabilityIdsVisitor)\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/request/deserialize.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fmt, marker::PhantomData};\n\nuse serde::{\n    Deserializer,\n    de::{self, MapAccess, Visitor},\n};\n\npub trait DeserializeArguments<'de> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: MapAccess<'de>;\n}\n\nimpl<'de> DeserializeArguments<'de> for () {\n    fn deserialize_argument<A>(&mut self, _key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: MapAccess<'de>,\n    {\n        let _: de::IgnoredAny = map.next_value()?;\n        Ok(())\n    }\n}\n\npub(crate) fn deserialize_request<'de, T, D>(deserializer: D) -> Result<T, D::Error>\nwhere\n    T: DeserializeArguments<'de> + Default,\n    D: Deserializer<'de>,\n{\n    struct DirectArgumentsVisitor<T> {\n        _phantom: PhantomData<T>,\n    }\n\n    impl<T> DirectArgumentsVisitor<T> {\n        fn new() -> Self {\n            Self {\n                _phantom: PhantomData,\n            }\n        }\n    }\n\n    impl<'de, T> Visitor<'de> for DirectArgumentsVisitor<T>\n    where\n        T: DeserializeArguments<'de> + Default,\n    {\n        type Value = T;\n\n        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n            formatter.write_str(\"a JMAP request object\")\n        }\n\n        fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>\n        where\n            A: MapAccess<'de>,\n        {\n            let mut target = T::default();\n\n            while let Some(key) = map.next_key::<&str>()? {\n                target\n                    .deserialize_argument(key, &mut map)\n                    .map_err(de::Error::custom)?;\n            }\n\n            Ok(target)\n        }\n    }\n\n    deserializer.deserialize_map(DirectArgumentsVisitor::<T>::new())\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/request/method.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub struct MethodName {\n    pub obj: MethodObject,\n    pub fnc: MethodFunction,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum MethodObject {\n    Email,\n    Mailbox,\n    Core,\n    Blob,\n    PushSubscription,\n    Thread,\n    SearchSnippet,\n    Identity,\n    EmailSubmission,\n    VacationResponse,\n    SieveScript,\n    Principal,\n    Quota,\n    Calendar,\n    CalendarEvent,\n    CalendarEventNotification,\n    AddressBook,\n    ContactCard,\n    FileNode,\n    ParticipantIdentity,\n    ShareNotification,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum MethodFunction {\n    Get,\n    Set,\n    Changes,\n    Query,\n    QueryChanges,\n    Copy,\n    Import,\n    Parse,\n    Validate,\n    Lookup,\n    Upload,\n    Echo,\n    GetAvailability,\n}\n\nimpl Display for MethodName {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(self.as_str())\n    }\n}\n\nimpl MethodName {\n    pub fn new(obj: MethodObject, fnc: MethodFunction) -> Self {\n        Self { obj, fnc }\n    }\n\n    pub fn error() -> Self {\n        Self {\n            obj: MethodObject::Thread,\n            fnc: MethodFunction::Echo,\n        }\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match (self.fnc, self.obj) {\n            (MethodFunction::Get, MethodObject::PushSubscription) => \"PushSubscription/get\",\n            (MethodFunction::Set, MethodObject::PushSubscription) => \"PushSubscription/set\",\n\n            (MethodFunction::Get, MethodObject::Mailbox) => \"Mailbox/get\",\n            (MethodFunction::Changes, MethodObject::Mailbox) => \"Mailbox/changes\",\n            (MethodFunction::Query, MethodObject::Mailbox) => \"Mailbox/query\",\n            (MethodFunction::QueryChanges, MethodObject::Mailbox) => \"Mailbox/queryChanges\",\n            (MethodFunction::Set, MethodObject::Mailbox) => \"Mailbox/set\",\n\n            (MethodFunction::Get, MethodObject::Thread) => \"Thread/get\",\n            (MethodFunction::Changes, MethodObject::Thread) => \"Thread/changes\",\n\n            (MethodFunction::Get, MethodObject::Email) => \"Email/get\",\n            (MethodFunction::Changes, MethodObject::Email) => \"Email/changes\",\n            (MethodFunction::Query, MethodObject::Email) => \"Email/query\",\n            (MethodFunction::QueryChanges, MethodObject::Email) => \"Email/queryChanges\",\n            (MethodFunction::Set, MethodObject::Email) => \"Email/set\",\n            (MethodFunction::Copy, MethodObject::Email) => \"Email/copy\",\n            (MethodFunction::Import, MethodObject::Email) => \"Email/import\",\n            (MethodFunction::Parse, MethodObject::Email) => \"Email/parse\",\n\n            (MethodFunction::Get, MethodObject::SearchSnippet) => \"SearchSnippet/get\",\n\n            (MethodFunction::Get, MethodObject::Identity) => \"Identity/get\",\n            (MethodFunction::Changes, MethodObject::Identity) => \"Identity/changes\",\n            (MethodFunction::Set, MethodObject::Identity) => \"Identity/set\",\n\n            (MethodFunction::Get, MethodObject::EmailSubmission) => \"EmailSubmission/get\",\n            (MethodFunction::Changes, MethodObject::EmailSubmission) => \"EmailSubmission/changes\",\n            (MethodFunction::Query, MethodObject::EmailSubmission) => \"EmailSubmission/query\",\n            (MethodFunction::QueryChanges, MethodObject::EmailSubmission) => {\n                \"EmailSubmission/queryChanges\"\n            }\n            (MethodFunction::Set, MethodObject::EmailSubmission) => \"EmailSubmission/set\",\n\n            (MethodFunction::Get, MethodObject::VacationResponse) => \"VacationResponse/get\",\n            (MethodFunction::Set, MethodObject::VacationResponse) => \"VacationResponse/set\",\n\n            (MethodFunction::Get, MethodObject::SieveScript) => \"SieveScript/get\",\n            (MethodFunction::Set, MethodObject::SieveScript) => \"SieveScript/set\",\n            (MethodFunction::Query, MethodObject::SieveScript) => \"SieveScript/query\",\n            (MethodFunction::Validate, MethodObject::SieveScript) => \"SieveScript/validate\",\n\n            (MethodFunction::Get, MethodObject::Principal) => \"Principal/get\",\n            (MethodFunction::Set, MethodObject::Principal) => \"Principal/set\",\n            (MethodFunction::Query, MethodObject::Principal) => \"Principal/query\",\n            (MethodFunction::Changes, MethodObject::Principal) => \"Principal/changes\",\n            (MethodFunction::QueryChanges, MethodObject::Principal) => \"Principal/queryChanges\",\n            (MethodFunction::GetAvailability, MethodObject::Principal) => \"Principal/getAvailability\",\n\n            (MethodFunction::Get, MethodObject::Quota) => \"Quota/get\",\n            (MethodFunction::Changes, MethodObject::Quota) => \"Quota/changes\",\n            (MethodFunction::Query, MethodObject::Quota) => \"Quota/query\",\n            (MethodFunction::QueryChanges, MethodObject::Quota) => \"Quota/queryChanges\",\n\n            (MethodFunction::Get, MethodObject::Blob) => \"Blob/get\",\n            (MethodFunction::Copy, MethodObject::Blob) => \"Blob/copy\",\n            (MethodFunction::Lookup, MethodObject::Blob) => \"Blob/lookup\",\n            (MethodFunction::Upload, MethodObject::Blob) => \"Blob/upload\",\n\n            (MethodFunction::Get, MethodObject::AddressBook) => \"AddressBook/get\",\n            (MethodFunction::Changes, MethodObject::AddressBook) => \"AddressBook/changes\",\n            (MethodFunction::Set, MethodObject::AddressBook) => \"AddressBook/set\",\n\n            (MethodFunction::Get, MethodObject::ContactCard) => \"ContactCard/get\",\n            (MethodFunction::Changes, MethodObject::ContactCard) => \"ContactCard/changes\",\n            (MethodFunction::Query, MethodObject::ContactCard) => \"ContactCard/query\",\n            (MethodFunction::QueryChanges, MethodObject::ContactCard) => \"ContactCard/queryChanges\",\n            (MethodFunction::Set, MethodObject::ContactCard) => \"ContactCard/set\",\n            (MethodFunction::Copy, MethodObject::ContactCard) => \"ContactCard/copy\",\n            (MethodFunction::Parse, MethodObject::ContactCard) => \"ContactCard/parse\",\n\n            (MethodFunction::Get, MethodObject::FileNode) => \"FileNode/get\",\n            (MethodFunction::Changes, MethodObject::FileNode) => \"FileNode/changes\",\n            (MethodFunction::Query, MethodObject::FileNode) => \"FileNode/query\",\n            (MethodFunction::QueryChanges, MethodObject::FileNode) => \"FileNode/queryChanges\",\n            (MethodFunction::Set, MethodObject::FileNode) => \"FileNode/set\",\n\n            (MethodFunction::Get, MethodObject::ShareNotification) => \"ShareNotification/get\",\n            (MethodFunction::Changes, MethodObject::ShareNotification) => \"ShareNotification/changes\",\n            (MethodFunction::Query, MethodObject::ShareNotification) => \"ShareNotification/query\",\n            (MethodFunction::QueryChanges, MethodObject::ShareNotification) => \"ShareNotification/queryChanges\",\n            (MethodFunction::Set, MethodObject::ShareNotification) => \"ShareNotification/set\",\n\n            (MethodFunction::Get, MethodObject::Calendar) => \"Calendar/get\",\n            (MethodFunction::Changes, MethodObject::Calendar) => \"Calendar/changes\",\n            (MethodFunction::Set, MethodObject::Calendar) => \"Calendar/set\",\n\n            (MethodFunction::Get, MethodObject::CalendarEvent) => \"CalendarEvent/get\",\n            (MethodFunction::Changes, MethodObject::CalendarEvent) => \"CalendarEvent/changes\",\n            (MethodFunction::Query, MethodObject::CalendarEvent) => \"CalendarEvent/query\",\n            (MethodFunction::QueryChanges, MethodObject::CalendarEvent) => \"CalendarEvent/queryChanges\",\n            (MethodFunction::Set, MethodObject::CalendarEvent) => \"CalendarEvent/set\",\n            (MethodFunction::Copy, MethodObject::CalendarEvent) => \"CalendarEvent/copy\",\n            (MethodFunction::Parse, MethodObject::CalendarEvent) => \"CalendarEvent/parse\",\n\n            (MethodFunction::Get, MethodObject::CalendarEventNotification) => \"CalendarEventNotification/get\",\n            (MethodFunction::Changes, MethodObject::CalendarEventNotification) => \"CalendarEventNotification/changes\",\n            (MethodFunction::Query, MethodObject::CalendarEventNotification) => \"CalendarEventNotification/query\",\n            (MethodFunction::QueryChanges, MethodObject::CalendarEventNotification) => \"CalendarEventNotification/queryChanges\",\n            (MethodFunction::Set, MethodObject::CalendarEventNotification) => \"CalendarEventNotification/set\",\n\n            (MethodFunction::Get, MethodObject::ParticipantIdentity) => \"ParticipantIdentity/get\",\n            (MethodFunction::Changes, MethodObject::ParticipantIdentity) => \"ParticipantIdentity/changes\",\n            (MethodFunction::Set, MethodObject::ParticipantIdentity) => \"ParticipantIdentity/set\",\n\n            (MethodFunction::Echo, MethodObject::Core) => \"Core/echo\",\n            _ => \"error\",\n        }\n    }\n\n    pub fn parse(s: &str) -> Option<Self> {\n       hashify::tiny_map!(s.as_bytes(), \n            \"PushSubscription/get\" => (MethodObject::PushSubscription, MethodFunction::Get),\n            \"PushSubscription/set\" => (MethodObject::PushSubscription, MethodFunction::Set),\n\n            \"Mailbox/get\" => (MethodObject::Mailbox, MethodFunction::Get),\n            \"Mailbox/changes\" => (MethodObject::Mailbox, MethodFunction::Changes),\n            \"Mailbox/query\" => (MethodObject::Mailbox, MethodFunction::Query),\n            \"Mailbox/queryChanges\" => (MethodObject::Mailbox, MethodFunction::QueryChanges),\n            \"Mailbox/set\" => (MethodObject::Mailbox, MethodFunction::Set),\n\n            \"Thread/get\" => (MethodObject::Thread, MethodFunction::Get),\n            \"Thread/changes\" => (MethodObject::Thread, MethodFunction::Changes),\n\n            \"Email/get\" => (MethodObject::Email, MethodFunction::Get),\n            \"Email/changes\" => (MethodObject::Email, MethodFunction::Changes),\n            \"Email/query\" => (MethodObject::Email, MethodFunction::Query),\n            \"Email/queryChanges\" => (MethodObject::Email, MethodFunction::QueryChanges),\n            \"Email/set\" => (MethodObject::Email, MethodFunction::Set),\n            \"Email/copy\" => (MethodObject::Email, MethodFunction::Copy),\n            \"Email/import\" => (MethodObject::Email, MethodFunction::Import),\n            \"Email/parse\" => (MethodObject::Email, MethodFunction::Parse),\n\n            \"SearchSnippet/get\" => (MethodObject::SearchSnippet, MethodFunction::Get),\n\n            \"Identity/get\" => (MethodObject::Identity, MethodFunction::Get),\n            \"Identity/changes\" => (MethodObject::Identity, MethodFunction::Changes),\n            \"Identity/set\" => (MethodObject::Identity, MethodFunction::Set),\n\n            \"EmailSubmission/get\" => (MethodObject::EmailSubmission, MethodFunction::Get),\n            \"EmailSubmission/changes\" => (MethodObject::EmailSubmission, MethodFunction::Changes),\n            \"EmailSubmission/query\" => (MethodObject::EmailSubmission, MethodFunction::Query),\n            \"EmailSubmission/queryChanges\" => (MethodObject::EmailSubmission, MethodFunction::QueryChanges),\n            \"EmailSubmission/set\" => (MethodObject::EmailSubmission, MethodFunction::Set),\n\n            \"VacationResponse/get\" => (MethodObject::VacationResponse, MethodFunction::Get),\n            \"VacationResponse/set\" => (MethodObject::VacationResponse, MethodFunction::Set),\n\n            \"SieveScript/get\" => (MethodObject::SieveScript, MethodFunction::Get),\n            \"SieveScript/set\" => (MethodObject::SieveScript, MethodFunction::Set),\n            \"SieveScript/query\" => (MethodObject::SieveScript, MethodFunction::Query),\n            \"SieveScript/validate\" => (MethodObject::SieveScript, MethodFunction::Validate),\n\n            \"Principal/get\" => (MethodObject::Principal, MethodFunction::Get),\n            \"Principal/set\" => (MethodObject::Principal, MethodFunction::Set),\n            \"Principal/query\" => (MethodObject::Principal, MethodFunction::Query),\n            \"Principal/changes\" => (MethodObject::Principal, MethodFunction::Changes),\n            \"Principal/queryChanges\" => (MethodObject::Principal, MethodFunction::QueryChanges),\n            \"Principal/getAvailability\" => (MethodObject::Principal, MethodFunction::GetAvailability),\n\n            \"Quota/get\" => (MethodObject::Quota, MethodFunction::Get),\n            \"Quota/changes\" => (MethodObject::Quota, MethodFunction::Changes),\n            \"Quota/query\" => (MethodObject::Quota, MethodFunction::Query),\n            \"Quota/queryChanges\" => (MethodObject::Quota, MethodFunction::QueryChanges),\n\n            \"Blob/get\" => (MethodObject::Blob, MethodFunction::Get),\n            \"Blob/copy\" => (MethodObject::Blob, MethodFunction::Copy),\n            \"Blob/lookup\" => (MethodObject::Blob, MethodFunction::Lookup),\n            \"Blob/upload\" => (MethodObject::Blob, MethodFunction::Upload),\n\n            \"AddressBook/get\" => (MethodObject::AddressBook, MethodFunction::Get),\n            \"AddressBook/changes\" => (MethodObject::AddressBook, MethodFunction::Changes),\n            \"AddressBook/set\" => (MethodObject::AddressBook, MethodFunction::Set),\n\n            \"ContactCard/get\" => (MethodObject::ContactCard, MethodFunction::Get),\n            \"ContactCard/changes\" => (MethodObject::ContactCard, MethodFunction::Changes),\n            \"ContactCard/query\" => (MethodObject::ContactCard, MethodFunction::Query),\n            \"ContactCard/queryChanges\" => (MethodObject::ContactCard, MethodFunction::QueryChanges),\n            \"ContactCard/set\" => (MethodObject::ContactCard, MethodFunction::Set),\n            \"ContactCard/copy\" => (MethodObject::ContactCard, MethodFunction::Copy),\n            \"ContactCard/parse\" => (MethodObject::ContactCard, MethodFunction::Parse),\n\n            \"FileNode/get\" => (MethodObject::FileNode, MethodFunction::Get),\n            \"FileNode/changes\" => (MethodObject::FileNode, MethodFunction::Changes),\n            \"FileNode/query\" => (MethodObject::FileNode, MethodFunction::Query),\n            \"FileNode/queryChanges\" => (MethodObject::FileNode, MethodFunction::QueryChanges),\n            \"FileNode/set\" => (MethodObject::FileNode, MethodFunction::Set),\n\n            \"ShareNotification/get\" => (MethodObject::ShareNotification, MethodFunction::Get),\n            \"ShareNotification/changes\" => (MethodObject::ShareNotification, MethodFunction::Changes),\n            \"ShareNotification/set\" => (MethodObject::ShareNotification, MethodFunction::Set),\n            \"ShareNotification/query\" => (MethodObject::ShareNotification, MethodFunction::Query),\n            \"ShareNotification/queryChanges\" => (MethodObject::ShareNotification, MethodFunction::QueryChanges),\n\n            \"Calendar/get\" => (MethodObject::Calendar, MethodFunction::Get),\n            \"Calendar/changes\" => (MethodObject::Calendar, MethodFunction::Changes),\n            \"Calendar/set\" => (MethodObject::Calendar, MethodFunction::Set),\n\n            \"CalendarEvent/get\" => (MethodObject::CalendarEvent, MethodFunction::Get),\n            \"CalendarEvent/changes\" => (MethodObject::CalendarEvent, MethodFunction::Changes),\n            \"CalendarEvent/query\" => (MethodObject::CalendarEvent, MethodFunction::Query),\n            \"CalendarEvent/queryChanges\" => (MethodObject::CalendarEvent, MethodFunction::QueryChanges),\n            \"CalendarEvent/set\" => (MethodObject::CalendarEvent, MethodFunction::Set),\n            \"CalendarEvent/copy\" => (MethodObject::CalendarEvent, MethodFunction::Copy),\n            \"CalendarEvent/parse\" => (MethodObject::CalendarEvent, MethodFunction::Parse),\n\n            \"CalendarEventNotification/get\" => (MethodObject::CalendarEventNotification, MethodFunction::Get),\n            \"CalendarEventNotification/changes\" => (MethodObject::CalendarEventNotification, MethodFunction::Changes),\n            \"CalendarEventNotification/set\" => (MethodObject::CalendarEventNotification, MethodFunction::Set),\n            \"CalendarEventNotification/query\" => (MethodObject::CalendarEventNotification, MethodFunction::Query),\n            \"CalendarEventNotification/queryChanges\" => (MethodObject::CalendarEventNotification, MethodFunction::QueryChanges),\n\n            \"ParticipantIdentity/get\" => (MethodObject::ParticipantIdentity, MethodFunction::Get),\n            \"ParticipantIdentity/changes\" => (MethodObject::ParticipantIdentity, MethodFunction::Changes),\n            \"ParticipantIdentity/set\" => (MethodObject::ParticipantIdentity, MethodFunction::Set),\n\n            \"Core/echo\" => (MethodObject::Core, MethodFunction::Echo),\n\n        ).map(|(obj, fnc)| MethodName { obj, fnc })\n    }\n\n}\n\nimpl Display for MethodObject {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(match self {\n            MethodObject::Blob => \"Blob\",\n            MethodObject::EmailSubmission => \"EmailSubmission\",\n            MethodObject::SearchSnippet => \"SearchSnippet\",\n            MethodObject::Identity => \"Identity\",\n            MethodObject::VacationResponse => \"VacationResponse\",\n            MethodObject::PushSubscription => \"PushSubscription\",\n            MethodObject::SieveScript => \"SieveScript\",\n            MethodObject::Principal => \"Principal\",\n            MethodObject::Core => \"Core\",\n            MethodObject::Mailbox => \"Mailbox\",\n            MethodObject::Thread => \"Thread\",\n            MethodObject::Email => \"Email\",\n            MethodObject::Quota => \"Quota\",\n            MethodObject::AddressBook => \"AddressBook\",\n            MethodObject::ContactCard => \"ContactCard\",\n            MethodObject::FileNode => \"FileNode\",\n            MethodObject::ParticipantIdentity => \"ParticipantIdentity\",\n            MethodObject::Calendar => \"Calendar\",\n            MethodObject::CalendarEvent => \"CalendarEvent\",\n            MethodObject::CalendarEventNotification => \"CalendarEventNotification\",\n            MethodObject::ShareNotification => \"ShareNotification\",\n        })\n    }\n}\n\n\nimpl<'de> serde::Deserialize<'de> for MethodName {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        let value = <&str>::deserialize(deserializer)?;\n\n        MethodName::parse(value).ok_or_else(|| {\n            serde::de::Error::custom(format!(\"Invalid method name: {:?}\", value))\n        })\n    }\n}\n\nimpl serde::Serialize for MethodName {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.as_str())\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/request/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod capability;\npub mod deserialize;\npub mod method;\npub mod parser;\npub mod reference;\npub mod websocket;\n\nuse self::method::MethodName;\nuse crate::{\n    method::{\n        availability::GetAvailabilityRequest,\n        changes::ChangesRequest,\n        copy::{CopyBlobRequest, CopyRequest},\n        get::GetRequest,\n        import::ImportEmailRequest,\n        lookup::BlobLookupRequest,\n        parse::ParseRequest,\n        query::QueryRequest,\n        query_changes::QueryChangesRequest,\n        search_snippet::GetSearchSnippetRequest,\n        set::SetRequest,\n        upload::BlobUploadRequest,\n        validate::ValidateSieveScriptRequest,\n    },\n    object::{\n        AnyId, addressbook::AddressBook, blob::Blob, calendar::Calendar,\n        calendar_event::CalendarEvent, calendar_event_notification::CalendarEventNotification,\n        contact::ContactCard, email::Email, email_submission::EmailSubmission, file_node::FileNode,\n        identity::Identity, mailbox::Mailbox, participant_identity::ParticipantIdentity,\n        principal::Principal, push_subscription::PushSubscription, quota::Quota,\n        share_notification::ShareNotification, sieve::Sieve, thread::Thread,\n        vacation_response::VacationResponse,\n    },\n    request::{capability::CapabilityIds, reference::MaybeIdReference},\n};\nuse jmap_tools::{Null, Value};\nuse std::{collections::HashMap, fmt::Debug, str::FromStr};\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug)]\npub struct Request<'x> {\n    pub using: CapabilityIds,\n    pub method_calls: Vec<Call<RequestMethod<'x>>>,\n    pub created_ids: Option<HashMap<String, AnyId>>,\n}\n\n#[derive(Debug)]\npub struct Call<T> {\n    pub id: String,\n    pub name: MethodName,\n    pub method: T,\n}\n\n#[derive(Debug)]\npub enum RequestMethod<'x> {\n    Get(GetRequestMethod),\n    Set(SetRequestMethod<'x>),\n    Changes(ChangesRequest),\n    Copy(CopyRequestMethod<'x>),\n    ImportEmail(ImportEmailRequest),\n    Parse(ParseRequestMethod),\n    Query(QueryRequestMethod),\n    QueryChanges(QueryChangesRequestMethod),\n    SearchSnippet(GetSearchSnippetRequest),\n    ValidateScript(ValidateSieveScriptRequest),\n    LookupBlob(BlobLookupRequest),\n    UploadBlob(BlobUploadRequest),\n    Echo(Value<'x, Null, Null>),\n    Error(trc::Error),\n}\n\n#[derive(Debug)]\npub enum GetRequestMethod {\n    Email(GetRequest<Email>),\n    Mailbox(GetRequest<Mailbox>),\n    Thread(GetRequest<Thread>),\n    Identity(GetRequest<Identity>),\n    EmailSubmission(GetRequest<EmailSubmission>),\n    PushSubscription(GetRequest<PushSubscription>),\n    Sieve(GetRequest<Sieve>),\n    VacationResponse(GetRequest<VacationResponse>),\n    Principal(GetRequest<Principal>),\n    PrincipalAvailability(GetAvailabilityRequest),\n    Quota(GetRequest<Quota>),\n    Blob(GetRequest<Blob>),\n    AddressBook(GetRequest<AddressBook>),\n    ContactCard(GetRequest<ContactCard>),\n    FileNode(GetRequest<FileNode>),\n    Calendar(GetRequest<Calendar>),\n    CalendarEvent(GetRequest<CalendarEvent>),\n    CalendarEventNotification(GetRequest<CalendarEventNotification>),\n    ParticipantIdentity(GetRequest<ParticipantIdentity>),\n    ShareNotification(GetRequest<ShareNotification>),\n}\n\n#[derive(Debug)]\npub enum SetRequestMethod<'x> {\n    Email(SetRequest<'x, Email>),\n    Mailbox(SetRequest<'x, Mailbox>),\n    Identity(SetRequest<'x, Identity>),\n    EmailSubmission(SetRequest<'x, EmailSubmission>),\n    PushSubscription(SetRequest<'x, PushSubscription>),\n    Sieve(SetRequest<'x, Sieve>),\n    VacationResponse(SetRequest<'x, VacationResponse>),\n    AddressBook(SetRequest<'x, AddressBook>),\n    ContactCard(SetRequest<'x, ContactCard>),\n    FileNode(SetRequest<'x, FileNode>),\n    ShareNotification(SetRequest<'x, ShareNotification>),\n    Calendar(SetRequest<'x, Calendar>),\n    CalendarEvent(SetRequest<'x, CalendarEvent>),\n    CalendarEventNotification(SetRequest<'x, CalendarEventNotification>),\n    ParticipantIdentity(SetRequest<'x, ParticipantIdentity>),\n}\n\n#[derive(Debug)]\npub enum CopyRequestMethod<'x> {\n    Email(CopyRequest<'x, Email>),\n    ContactCard(CopyRequest<'x, ContactCard>),\n    CalendarEvent(CopyRequest<'x, CalendarEvent>),\n    Blob(CopyBlobRequest),\n}\n\n#[derive(Debug)]\npub enum QueryRequestMethod {\n    Email(QueryRequest<Email>),\n    Mailbox(QueryRequest<Mailbox>),\n    EmailSubmission(QueryRequest<EmailSubmission>),\n    Sieve(QueryRequest<Sieve>),\n    Principal(QueryRequest<Principal>),\n    Quota(QueryRequest<Quota>),\n    ContactCard(QueryRequest<ContactCard>),\n    FileNode(QueryRequest<FileNode>),\n    CalendarEvent(QueryRequest<CalendarEvent>),\n    CalendarEventNotification(QueryRequest<CalendarEventNotification>),\n    ShareNotification(QueryRequest<ShareNotification>),\n}\n\n#[derive(Debug)]\npub enum QueryChangesRequestMethod {\n    Email(QueryChangesRequest<Email>),\n    Mailbox(QueryChangesRequest<Mailbox>),\n    EmailSubmission(QueryChangesRequest<EmailSubmission>),\n    Sieve(QueryChangesRequest<Sieve>),\n    Principal(QueryChangesRequest<Principal>),\n    Quota(QueryChangesRequest<Quota>),\n    ContactCard(QueryChangesRequest<ContactCard>),\n    FileNode(QueryChangesRequest<FileNode>),\n    CalendarEvent(QueryChangesRequest<CalendarEvent>),\n    CalendarEventNotification(QueryChangesRequest<CalendarEventNotification>),\n    ShareNotification(QueryChangesRequest<ShareNotification>),\n}\n\n#[derive(Debug)]\npub enum ParseRequestMethod {\n    Email(ParseRequest<Email>),\n    ContactCard(ParseRequest<ContactCard>),\n    CalendarEvent(ParseRequest<CalendarEvent>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum MaybeInvalid<V: FromStr> {\n    Value(V),\n    Invalid(String),\n}\n\nimpl<'de, V: FromStr> serde::Deserialize<'de> for MaybeInvalid<V> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        let value = <&str>::deserialize(deserializer)?;\n\n        if let Ok(id) = V::from_str(value) {\n            Ok(MaybeInvalid::Value(id))\n        } else {\n            Ok(MaybeInvalid::Invalid(value.to_string()))\n        }\n    }\n}\n\nimpl<V: FromStr> Default for MaybeInvalid<V> {\n    fn default() -> Self {\n        MaybeInvalid::Invalid(\"\".to_string())\n    }\n}\n\n#[allow(clippy::derivable_impls)]\nimpl Default for Request<'_> {\n    fn default() -> Self {\n        Request {\n            using: CapabilityIds::default(),\n            method_calls: Vec::new(),\n            created_ids: None,\n        }\n    }\n}\n\nimpl<T> MaybeInvalid<T>\nwhere\n    T: FromStr,\n{\n    pub fn try_unwrap(self) -> Option<T> {\n        match self {\n            MaybeInvalid::Value(id) => Some(id),\n            MaybeInvalid::Invalid(_) => None,\n        }\n    }\n}\n\npub trait IntoValid {\n    type Item;\n\n    fn into_valid(self) -> impl Iterator<Item = Self::Item>;\n}\n\nimpl<T: FromStr> IntoValid for Vec<MaybeInvalid<T>> {\n    type Item = T;\n\n    fn into_valid(self) -> impl Iterator<Item = Self::Item> {\n        self.into_iter().filter_map(|v| v.try_unwrap())\n    }\n}\n\nimpl<T: FromStr> IntoValid for Vec<MaybeIdReference<T>> {\n    type Item = T;\n\n    fn into_valid(self) -> impl Iterator<Item = Self::Item> {\n        self.into_iter().filter_map(|v| v.try_unwrap())\n    }\n}\n\nimpl<T: FromStr + Eq, V> IntoValid for VecMap<MaybeInvalid<T>, V> {\n    type Item = (T, V);\n\n    fn into_valid(self) -> impl Iterator<Item = Self::Item> {\n        self.into_iter()\n            .filter_map(|(k, v)| k.try_unwrap().map(|k| (k, v)))\n    }\n}\n\nimpl<T: FromStr + Eq, V> IntoValid for VecMap<MaybeIdReference<T>, V> {\n    type Item = (T, V);\n\n    fn into_valid(self) -> impl Iterator<Item = Self::Item> {\n        self.into_iter()\n            .filter_map(|(k, v)| k.try_unwrap().map(|k| (k, v)))\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/request/parser.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    Call, Request, RequestMethod,\n    method::{MethodFunction, MethodName, MethodObject},\n};\nuse crate::request::{\n    CopyRequestMethod, GetRequestMethod, ParseRequestMethod, QueryChangesRequestMethod,\n    QueryRequestMethod, SetRequestMethod,\n    deserialize::{DeserializeArguments, deserialize_request},\n};\nuse serde::{\n    Deserialize, Deserializer,\n    de::{self, SeqAccess, Visitor},\n};\nuse std::fmt::{self, Display};\n\nimpl<'x> Request<'x> {\n    pub fn parse(json: &'x [u8], max_calls: usize, max_size: usize) -> trc::Result<Self> {\n        if json.len() <= max_size {\n            match serde_json::from_slice::<Request>(json) {\n                Ok(request) => {\n                    if request.method_calls.len() <= max_calls {\n                        Ok(request)\n                    } else {\n                        Err(trc::LimitEvent::CallsIn.into_err())\n                    }\n                }\n                Err(err) => Err(trc::JmapEvent::NotRequest\n                    .into_err()\n                    .details(err.to_string())),\n            }\n        } else {\n            Err(trc::LimitEvent::SizeRequest.into_err())\n        }\n    }\n}\n\nimpl<'de> DeserializeArguments<'de> for Request<'de> {\n    fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        hashify::fnc_map!(key.as_bytes(),\n            b\"using\" => {\n                self.using = map.next_value()?;\n            },\n            b\"methodCalls\" => {\n                self.method_calls = map.next_value()?;\n            },\n            b\"createdIds\" => {\n                self.created_ids = map.next_value()?;\n            },\n            _ => {\n                let _ = map.next_value::<serde::de::IgnoredAny>()?;\n            }\n        );\n\n        Ok(())\n    }\n}\n\nstruct CallVisitor;\n\nimpl<'de> Visitor<'de> for CallVisitor {\n    type Value = Call<RequestMethod<'de>>;\n\n    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n        formatter.write_str(\"an array with 3 elements\")\n    }\n\n    fn visit_seq<V>(self, mut seq: V) -> Result<Call<RequestMethod<'de>>, V::Error>\n    where\n        V: SeqAccess<'de>,\n    {\n        let method_name = seq\n            .next_element::<&str>()?\n            .ok_or_else(|| de::Error::invalid_length(0, &self))?;\n        let name = match MethodName::parse(method_name) {\n            Some(name) => name,\n            None => {\n                // Ignore the rest of the call\n                let _ = seq\n                    .next_element::<serde::de::IgnoredAny>()?\n                    .ok_or_else(|| de::Error::invalid_length(1, &self))?;\n                let id = seq\n                    .next_element::<String>()?\n                    .ok_or_else(|| de::Error::invalid_length(2, &self))?;\n\n                return Ok(Call {\n                    id,\n                    method: RequestMethod::Error(\n                        trc::JmapEvent::UnknownMethod\n                            .into_err()\n                            .details(method_name.to_string()),\n                    ),\n                    name: MethodName::error(),\n                });\n            }\n        };\n\n        let method = match (&name.fnc, &name.obj) {\n            (MethodFunction::Get, MethodObject::Email) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Email(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::Mailbox) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Mailbox(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::Thread) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Thread(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::Identity) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Identity(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::EmailSubmission) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::EmailSubmission(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::PushSubscription) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::PushSubscription(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::VacationResponse) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::VacationResponse(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::SieveScript) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Sieve(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::Principal) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Principal(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::Quota) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Quota(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::Blob) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Blob(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::Calendar) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Calendar(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::CalendarEvent) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::CalendarEvent(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::CalendarEventNotification) => {\n                match seq.next_element() {\n                    Ok(Some(value)) => {\n                        RequestMethod::Get(GetRequestMethod::CalendarEventNotification(value))\n                    }\n                    Err(err) => RequestMethod::invalid(err),\n                    Ok(None) => {\n                        return Err(de::Error::invalid_length(1, &self));\n                    }\n                }\n            }\n            (MethodFunction::Get, MethodObject::ParticipantIdentity) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::ParticipantIdentity(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::AddressBook) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::AddressBook(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::ContactCard) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::ContactCard(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::FileNode) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::FileNode(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::ShareNotification) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::ShareNotification(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Get, MethodObject::SearchSnippet) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::SearchSnippet(value),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::Email) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::Email(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::Mailbox) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::Mailbox(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::Identity) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::Identity(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::EmailSubmission) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::EmailSubmission(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::PushSubscription) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::PushSubscription(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::VacationResponse) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::VacationResponse(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::SieveScript) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::Sieve(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::Calendar) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::Calendar(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::CalendarEvent) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::CalendarEvent(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::CalendarEventNotification) => {\n                match seq.next_element() {\n                    Ok(Some(value)) => {\n                        RequestMethod::Set(SetRequestMethod::CalendarEventNotification(value))\n                    }\n                    Err(err) => RequestMethod::invalid(err),\n                    Ok(None) => {\n                        return Err(de::Error::invalid_length(1, &self));\n                    }\n                }\n            }\n            (MethodFunction::Set, MethodObject::ParticipantIdentity) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::ParticipantIdentity(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::AddressBook) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::AddressBook(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::ContactCard) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::ContactCard(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::FileNode) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::FileNode(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Set, MethodObject::ShareNotification) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::ShareNotification(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::Email) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::Email(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::Mailbox) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::Mailbox(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::EmailSubmission) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::EmailSubmission(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::SieveScript) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::Sieve(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::Principal) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::Principal(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::Quota) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::Quota(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::CalendarEvent) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::CalendarEvent(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::CalendarEventNotification) => {\n                match seq.next_element() {\n                    Ok(Some(value)) => {\n                        RequestMethod::Query(QueryRequestMethod::CalendarEventNotification(value))\n                    }\n                    Err(err) => RequestMethod::invalid(err),\n                    Ok(None) => {\n                        return Err(de::Error::invalid_length(1, &self));\n                    }\n                }\n            }\n            (MethodFunction::Query, MethodObject::ContactCard) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::ContactCard(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::FileNode) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::FileNode(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Query, MethodObject::ShareNotification) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::Query(QueryRequestMethod::ShareNotification(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::Email) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::Email(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::Mailbox) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::Mailbox(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::EmailSubmission) => {\n                match seq.next_element() {\n                    Ok(Some(value)) => RequestMethod::QueryChanges(\n                        QueryChangesRequestMethod::EmailSubmission(value),\n                    ),\n                    Err(err) => RequestMethod::invalid(err),\n                    Ok(None) => {\n                        return Err(de::Error::invalid_length(1, &self));\n                    }\n                }\n            }\n            (MethodFunction::QueryChanges, MethodObject::SieveScript) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::Sieve(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::Principal) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::Principal(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::Quota) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::Quota(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::CalendarEvent) => match seq.next_element()\n            {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::CalendarEvent(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::CalendarEventNotification) => {\n                match seq.next_element() {\n                    Ok(Some(value)) => RequestMethod::QueryChanges(\n                        QueryChangesRequestMethod::CalendarEventNotification(value),\n                    ),\n                    Err(err) => RequestMethod::invalid(err),\n                    Ok(None) => {\n                        return Err(de::Error::invalid_length(1, &self));\n                    }\n                }\n            }\n            (MethodFunction::QueryChanges, MethodObject::ContactCard) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::ContactCard(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::FileNode) => match seq.next_element() {\n                Ok(Some(value)) => {\n                    RequestMethod::QueryChanges(QueryChangesRequestMethod::FileNode(value))\n                }\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::QueryChanges, MethodObject::ShareNotification) => {\n                match seq.next_element() {\n                    Ok(Some(value)) => RequestMethod::QueryChanges(\n                        QueryChangesRequestMethod::ShareNotification(value),\n                    ),\n                    Err(err) => RequestMethod::invalid(err),\n                    Ok(None) => {\n                        return Err(de::Error::invalid_length(1, &self));\n                    }\n                }\n            }\n            (MethodFunction::Changes, _) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Changes(value),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Copy, MethodObject::Email) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Copy(CopyRequestMethod::Email(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Copy, MethodObject::Blob) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Copy(CopyRequestMethod::Blob(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Copy, MethodObject::CalendarEvent) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Copy(CopyRequestMethod::CalendarEvent(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Copy, MethodObject::ContactCard) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Copy(CopyRequestMethod::ContactCard(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Lookup, MethodObject::Blob) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::LookupBlob(value),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Upload, MethodObject::Blob) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::UploadBlob(value),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Import, MethodObject::Email) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::ImportEmail(value),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Parse, MethodObject::Email) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Parse(ParseRequestMethod::Email(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Parse, MethodObject::CalendarEvent) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Parse(ParseRequestMethod::CalendarEvent(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Parse, MethodObject::ContactCard) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Parse(ParseRequestMethod::ContactCard(value)),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::GetAvailability, MethodObject::Principal) => {\n                match seq.next_element() {\n                    Ok(Some(value)) => {\n                        RequestMethod::Get(GetRequestMethod::PrincipalAvailability(value))\n                    }\n                    Err(err) => RequestMethod::invalid(err),\n                    Ok(None) => {\n                        return Err(de::Error::invalid_length(1, &self));\n                    }\n                }\n            }\n            (MethodFunction::Validate, MethodObject::SieveScript) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::ValidateScript(value),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            (MethodFunction::Echo, MethodObject::Core) => match seq.next_element() {\n                Ok(Some(value)) => RequestMethod::Echo(value),\n                Err(err) => RequestMethod::invalid(err),\n                Ok(None) => {\n                    return Err(de::Error::invalid_length(1, &self));\n                }\n            },\n            _ => {\n                return Err(de::Error::custom(format!(\n                    \"Invalid method function/object combination: {}\",\n                    method_name\n                )));\n            }\n        };\n\n        let id = seq\n            .next_element::<String>()?\n            .ok_or_else(|| de::Error::invalid_length(2, &self))?;\n\n        Ok(Call { id, method, name })\n    }\n}\n\nimpl RequestMethod<'_> {\n    fn invalid(err: impl Display) -> Self {\n        RequestMethod::Error(\n            trc::JmapEvent::InvalidArguments\n                .into_err()\n                .details(err.to_string()),\n        )\n    }\n}\n\nimpl<'de> Deserialize<'de> for Request<'de> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserialize_request(deserializer)\n    }\n}\n\nimpl<'de> Deserialize<'de> for Call<RequestMethod<'de>> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserializer.deserialize_seq(CallVisitor)\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::request::Request;\n\n    const TEST: &str = r#\"\n    {\n        \"using\": [ \"urn:ietf:params:jmap:core\", \"urn:ietf:params:jmap:mail\" ],\n        \"methodCalls\": [\n          [ \"method1\", {\n            \"arg1\": \"arg1data\",\n            \"arg2\": \"arg2data\"\n          }, \"c1\" ],\n          [ \"Core/echo\", {\n            \"hello\": true,\n            \"high\": 5\n          }, \"c2\" ],\n          [ \"method3\", {\"hello\": [{\"a\": {\"b\": true}}]}, \"c3\" ]\n        ],\n        \"createdIds\": {\n            \"c1\": \"m1\",\n            \"c2\": \"m2\"\n        }\n      }\n    \"#;\n\n    const TEST1: &str = r#\"\n    {\n    \"using\": [\n        \"urn:ietf:params:jmap:core\",\n        \"urn:ietf:params:jmap:mail\"\n    ],\n    \"methodCalls\": [\n        [\n        \"Email/query\",\n        {\n            \"accountId\": \"0\",\n            \"filter\": { \"conditions\": [ { \"hasKeyword\": \"music\", \"maxSize\": 455 }, { \"hasKeyword\": \"video\" }, { \"operator\": \"AND\", \"conditions\": [ { \"subject\": \"test\" }, { \"minSize\": 100 } ] } ], \"operator\": \"OR\" },\n            \"sort\": [\n            {\n                \"property\": \"subject\",\n                \"isAscending\": true\n            },\n            {\n                \"property\": \"allInThreadHaveKeyword\",\n                \"isAscending\": false,\n                \"keyword\": \"$seen\"\n            },\n            {\n                \"keyword\": \"$junk\",\n                \"property\": \"someInThreadHaveKeyword\",\n                \"collation\": \"i;octet\",\n                \"isAscending\": false\n            }\n            ],\n            \"position\": 0,\n            \"limit\": 10\n        },\n        \"c1\"\n        ]\n    ],\n    \"createdIds\": {}\n    }\n    \"#;\n\n    const TEST2: &str = r##\"\n    {\n        \"using\": [\n          \"urn:ietf:params:jmap:submission\",\n          \"urn:ietf:params:jmap:mail\",\n          \"urn:ietf:params:jmap:core\"\n        ],\n        \"methodCalls\": [\n          [\n            \"Email/set\",\n            {\n              \"accountId\": \"c\",\n              \"create\": {\n                \"c37ee58b-e224-4799-88e6-1d7484e3b782\": {\n                  \"mailboxIds\": {\n                    \"9\": true\n                  },\n                  \"subject\": \"test\",\n                  \"from\": [\n                    {\n                      \"name\": \"Foo\",\n                      \"email\": \"foo@bar.com\"\n                    }\n                  ],\n                  \"to\": [\n                    {\n                      \"name\": null,\n                      \"email\": \"bar@foo.com\"\n                    }\n                  ],\n                  \"cc\": [],\n                  \"bcc\": [],\n                  \"replyTo\": [\n                    {\n                      \"name\": null,\n                      \"email\": \"foo@bar.com\"\n                    }\n                  ],\n                  \"htmlBody\": [\n                    {\n                      \"partId\": \"c37ee58b-e224-4799-88e6-1d7484e3b782\",\n                      \"type\": \"text/html\"\n                    }\n                  ],\n                  \"bodyValues\": {\n                    \"c37ee58b-e224-4799-88e6-1d7484e3b782\": {\n                      \"value\": \"<p>test email<br></p>\",\n                      \"isEncodingProblem\": false,\n                      \"isTruncated\": false\n                    }\n                  },\n                  \"header:User-Agent:asText\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/113.0\"\n                }\n              }\n            },\n            \"c0\"\n          ],\n          [\n            \"EmailSubmission/set\",\n            {\n              \"accountId\": \"c\",\n              \"create\": {\n                \"c37ee58b-e224-4799-88e6-1d7484e3b782\": {\n                  \"identityId\": \"a\",\n                  \"emailId\": \"#c37ee58b-e224-4799-88e6-1d7484e3b782\",\n                  \"envelope\": {\n                    \"mailFrom\": {\n                      \"email\": \"foo@bar.com\"\n                    },\n                    \"rcptTo\": [\n                      {\n                        \"email\": \"bar@foo.com\"\n                      }\n                    ]\n                  }\n                }\n              },\n              \"onSuccessUpdateEmail\": {\n                \"#c37ee58b-e224-4799-88e6-1d7484e3b782\": {\n                  \"mailboxIds/d\": true,\n                  \"mailboxIds/9\": null,\n                  \"keywords/$seen\": true,\n                  \"keywords/$draft\": null\n                }\n              }\n            },\n            \"c1\"\n          ]\n        ]\n      }\n    \"##;\n\n    #[test]\n    fn parse_request() {\n        println!(\"{:#?}\", Request::parse(TEST.as_bytes(), 10, 10240));\n        println!(\"{:#?}\", Request::parse(TEST1.as_bytes(), 10, 10240));\n        println!(\"{:#?}\", Request::parse(TEST2.as_bytes(), 10, 10240));\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/request/reference.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::method::MethodName;\nuse jmap_tools::{JsonPointer, Null};\nuse std::{borrow::Cow, fmt::Display, str::FromStr};\n\n#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]\npub struct ResultReference {\n    #[serde(rename = \"resultOf\")]\n    pub result_of: String,\n    pub name: MethodName,\n    pub path: JsonPointer<Null>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub enum MaybeIdReference<V: FromStr> {\n    Id(V),\n    Reference(String),\n    Invalid(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum MaybeResultReference<V> {\n    Value(V),\n    Reference(ResultReference),\n}\n\nimpl Display for ResultReference {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(\n            f,\n            \"{{ resultOf: {}, name: {}, path: {} }}\",\n            self.result_of, self.name, self.path\n        )\n    }\n}\n\nimpl<V: FromStr + Display> Display for MaybeIdReference<V> {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            MaybeIdReference::Id(id) => write!(f, \"{}\", id),\n            MaybeIdReference::Reference(str) => write!(f, \"#{}\", str),\n            MaybeIdReference::Invalid(str) => write!(f, \"{}\", str),\n        }\n    }\n}\n\nimpl<'de, V: FromStr> serde::Deserialize<'de> for MaybeIdReference<V> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        let value = <Cow<'de, str>>::deserialize(deserializer)?;\n\n        if let Some(reference) = value.strip_prefix('#') {\n            if reference.is_empty() {\n                return Ok(MaybeIdReference::Invalid(value.into_owned()));\n            }\n            Ok(MaybeIdReference::Reference(reference.to_string()))\n        } else if let Ok(id) = V::from_str(value.as_ref()) {\n            Ok(MaybeIdReference::Id(id))\n        } else {\n            Ok(MaybeIdReference::Invalid(value.into_owned()))\n        }\n    }\n}\n\nimpl<V: FromStr> FromStr for MaybeIdReference<V> {\n    type Err = V::Err;\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        if let Some(reference) = s.strip_prefix('#') {\n            if reference.is_empty() {\n                return Ok(MaybeIdReference::Invalid(s.to_string()));\n            }\n            Ok(MaybeIdReference::Reference(reference.to_string()))\n        } else if let Ok(id) = V::from_str(s) {\n            Ok(MaybeIdReference::Id(id))\n        } else {\n            Ok(MaybeIdReference::Invalid(s.to_string()))\n        }\n    }\n}\n\nimpl<V: Display + FromStr> serde::Serialize for MaybeIdReference<V> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        match self {\n            MaybeIdReference::Id(id) => serializer.serialize_str(&id.to_string()),\n            MaybeIdReference::Reference(str) => serializer.serialize_str(&format!(\"#{}\", str)),\n            MaybeIdReference::Invalid(str) => serializer.serialize_str(str),\n        }\n    }\n}\n\nimpl<V: Default> Default for MaybeResultReference<V> {\n    fn default() -> Self {\n        MaybeResultReference::Value(V::default())\n    }\n}\n\nimpl<T: Default> MaybeResultReference<T> {\n    pub fn unwrap(self) -> T {\n        match self {\n            MaybeResultReference::Value(v) => v,\n            MaybeResultReference::Reference(_) => T::default(),\n        }\n    }\n}\n\nimpl<T: FromStr> MaybeIdReference<T> {\n    pub fn try_unwrap(self) -> Option<T> {\n        match self {\n            MaybeIdReference::Id(id) => Some(id),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/request/websocket.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::Request;\nuse crate::{\n    error::request::{RequestError, RequestErrorType, RequestLimitError},\n    object::AnyId,\n    request::{Call, deserialize::DeserializeArguments},\n    response::{Response, ResponseMethod, serialize::serialize_hex, status::PushObject},\n};\nuse serde::{\n    Deserialize, Deserializer,\n    de::{self, MapAccess, Visitor},\n};\nuse std::{borrow::Cow, collections::HashMap, fmt};\nuse types::type_state::DataType;\n\n#[derive(Debug)]\npub struct WebSocketRequest<'x> {\n    pub id: Option<String>,\n    pub request: Request<'x>,\n}\n\n#[derive(Debug, serde::Serialize)]\npub struct WebSocketResponse<'x> {\n    #[serde(rename = \"@type\")]\n    _type: WebSocketResponseType,\n\n    #[serde(rename = \"methodResponses\")]\n    method_responses: Vec<Call<ResponseMethod<'x>>>,\n\n    #[serde(rename = \"sessionState\")]\n    #[serde(serialize_with = \"serialize_hex\")]\n    session_state: u32,\n\n    #[serde(rename(deserialize = \"createdIds\"))]\n    #[serde(skip_serializing_if = \"HashMap::is_empty\")]\n    created_ids: HashMap<String, AnyId>,\n\n    #[serde(rename = \"requestId\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    request_id: Option<String>,\n}\n\n#[derive(Debug, PartialEq, Eq, serde::Serialize)]\npub enum WebSocketResponseType {\n    Response,\n}\n\n#[derive(Debug, Default, PartialEq, Eq)]\npub struct WebSocketPushEnable {\n    pub data_types: Vec<DataType>,\n    pub push_state: Option<String>,\n}\n\n#[derive(Debug)]\npub enum WebSocketMessage<'x> {\n    Request(WebSocketRequest<'x>),\n    PushEnable(WebSocketPushEnable),\n    PushDisable,\n}\n\n#[derive(serde::Serialize, Debug)]\npub struct WebSocketPushObject {\n    #[serde(flatten)]\n    pub push: PushObject,\n\n    #[serde(rename = \"pushState\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub push_state: Option<String>,\n}\n\n#[derive(Debug, serde::Serialize)]\npub struct WebSocketRequestError<'x> {\n    #[serde(rename = \"@type\")]\n    pub type_: WebSocketRequestErrorType,\n\n    #[serde(rename = \"type\")]\n    p_type: RequestErrorType,\n\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    limit: Option<RequestLimitError>,\n    status: u16,\n    detail: Cow<'x, str>,\n\n    #[serde(rename = \"requestId\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub request_id: Option<String>,\n}\n\n#[derive(serde::Serialize, Debug)]\npub enum WebSocketRequestErrorType {\n    RequestError,\n}\n\nenum MessageType {\n    Request,\n    PushEnable,\n    PushDisable,\n    None,\n}\n\nimpl<'x> WebSocketMessage<'x> {\n    pub fn parse(json: &'x [u8], max_calls: usize, max_size: usize) -> trc::Result<Self> {\n        if json.len() <= max_size {\n            match serde_json::from_slice::<Self>(json) {\n                Ok(WebSocketMessage::Request(req))\n                    if req.request.method_calls.len() > max_calls =>\n                {\n                    Err(trc::LimitEvent::CallsIn.into_err())\n                }\n                Ok(msg) => Ok(msg),\n                Err(err) => Err(trc::JmapEvent::NotRequest\n                    .into_err()\n                    .details(format!(\"Invalid WebSocket JMAP request {err}\"))),\n            }\n        } else {\n            Err(trc::LimitEvent::SizeRequest.into_err())\n        }\n    }\n}\n\nimpl<'de: 'x, 'x: 'de> Deserialize<'de> for WebSocketMessage<'x> {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: Deserializer<'de>,\n    {\n        deserializer.deserialize_map(WebSocketMessageVisitor)\n    }\n}\n\nstruct WebSocketMessageVisitor;\n\nimpl<'de> Visitor<'de> for WebSocketMessageVisitor {\n    type Value = WebSocketMessage<'de>;\n\n    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n        formatter.write_str(\"a WebSocketMessage as a map\")\n    }\n\n    fn visit_map<V>(self, mut map: V) -> Result<WebSocketMessage<'de>, V::Error>\n    where\n        V: MapAccess<'de>,\n    {\n        let mut message_type = MessageType::None;\n        let mut request = WebSocketRequest {\n            id: None,\n            request: Request::default(),\n        };\n        let mut push_enable = WebSocketPushEnable::default();\n\n        let mut found_request_keys = false;\n        let mut found_push_keys = false;\n\n        while let Some(key) = map.next_key::<&str>()? {\n            hashify::fnc_map!(key.as_bytes(),\n                b\"@type\" => {\n                    message_type = MessageType::parse(map.next_value()?);\n                },\n                b\"dataTypes\" => {\n                    push_enable.data_types = map.next_value::<Option<Vec<DataType>>>()?.unwrap_or_default();\n                    found_push_keys = true;\n                },\n                b\"pushState\" => {\n                    push_enable.push_state = map.next_value()?;\n                    found_push_keys = true;\n                },\n                b\"id\" => {\n                    request.id = map.next_value()?;\n                },\n                _ => {\n                    request.request.deserialize_argument(key, &mut map)?;\n                    found_request_keys = true;\n                }\n            );\n        }\n\n        match message_type {\n            MessageType::Request if found_request_keys => Ok(WebSocketMessage::Request(request)),\n            MessageType::PushEnable if found_push_keys => {\n                Ok(WebSocketMessage::PushEnable(push_enable))\n            }\n            MessageType::PushDisable if !found_request_keys && !found_push_keys => {\n                Ok(WebSocketMessage::PushDisable)\n            }\n            _ => Err(de::Error::custom(\"Invalid WebSocket JMAP request\")),\n        }\n    }\n}\n\nimpl MessageType {\n    fn parse(s: &str) -> Self {\n        hashify::tiny_map!(s.as_bytes(),\n            b\"Request\" => MessageType::Request,\n            b\"WebSocketPushEnable\" => MessageType::PushEnable,\n            b\"WebSocketPushDisable\" => MessageType::PushDisable,\n        )\n        .unwrap_or(MessageType::None)\n    }\n}\n\nimpl<'x> WebSocketRequestError<'x> {\n    pub fn from_error(error: RequestError<'x>, request_id: Option<String>) -> Self {\n        Self {\n            type_: WebSocketRequestErrorType::RequestError,\n            p_type: error.p_type,\n            limit: error.limit,\n            status: error.status,\n            detail: error.detail,\n            request_id,\n        }\n    }\n\n    pub fn to_json(&self) -> String {\n        serde_json::to_string(self).unwrap()\n    }\n}\n\nimpl<'x> From<RequestError<'x>> for WebSocketRequestError<'x> {\n    fn from(value: RequestError<'x>) -> Self {\n        Self::from_error(value, None)\n    }\n}\n\nimpl<'x> WebSocketResponse<'x> {\n    pub fn from_response(response: Response<'x>, request_id: Option<String>) -> Self {\n        Self {\n            _type: WebSocketResponseType::Response,\n            method_responses: response.method_responses,\n            session_state: response.session_state,\n            created_ids: response.created_ids,\n            request_id,\n        }\n    }\n\n    pub fn to_json(&self) -> String {\n        serde_json::to_string(self).unwrap()\n    }\n}\n\nimpl WebSocketPushObject {\n    pub fn to_json(&self) -> String {\n        serde_json::to_string(self).unwrap()\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/response/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod serialize;\npub mod status;\n\nuse self::serialize::serialize_hex;\nuse crate::{\n    error::method::MethodErrorWrapper,\n    method::{\n        availability::GetAvailabilityResponse,\n        changes::ChangesResponse,\n        copy::{CopyBlobResponse, CopyResponse},\n        get::GetResponse,\n        import::ImportEmailResponse,\n        lookup::BlobLookupResponse,\n        parse::ParseResponse,\n        query::QueryResponse,\n        query_changes::QueryChangesResponse,\n        search_snippet::GetSearchSnippetResponse,\n        set::SetResponse,\n        upload::BlobUploadResponse,\n        validate::ValidateSieveScriptResponse,\n    },\n    object::{\n        AnyId,\n        addressbook::AddressBook,\n        blob::Blob,\n        calendar::Calendar,\n        calendar_event::CalendarEvent,\n        calendar_event_notification::{\n            CalendarEventNotification, CalendarEventNotificationGetResponse,\n        },\n        contact::ContactCard,\n        email::Email,\n        email_submission::EmailSubmission,\n        file_node::FileNode,\n        identity::Identity,\n        mailbox::Mailbox,\n        participant_identity::ParticipantIdentity,\n        principal::Principal,\n        push_subscription::PushSubscription,\n        quota::Quota,\n        share_notification::ShareNotification,\n        sieve::Sieve,\n        thread::Thread,\n        vacation_response::VacationResponse,\n    },\n    request::{Call, method::MethodName},\n};\nuse jmap_tools::{Null, Value};\nuse std::collections::HashMap;\n\n#[derive(Debug, serde::Serialize)]\n#[serde(untagged)]\npub enum ResponseMethod<'x> {\n    Get(GetResponseMethod),\n    Set(SetResponseMethod),\n    Changes(ChangesResponseMethod),\n    Copy(CopyResponseMethod),\n    ImportEmail(ImportEmailResponse),\n    Parse(ParseResponseMethod),\n    QueryChanges(QueryChangesResponse),\n    Query(QueryResponse),\n    SearchSnippet(GetSearchSnippetResponse),\n    ValidateScript(ValidateSieveScriptResponse),\n    LookupBlob(BlobLookupResponse),\n    UploadBlob(BlobUploadResponse),\n    Echo(Value<'x, Null, Null>),\n    Error(MethodErrorWrapper),\n}\n\n#[derive(Debug, serde::Serialize)]\n#[serde(untagged)]\npub enum GetResponseMethod {\n    Email(GetResponse<Email>),\n    Mailbox(GetResponse<Mailbox>),\n    Thread(GetResponse<Thread>),\n    Identity(GetResponse<Identity>),\n    EmailSubmission(GetResponse<EmailSubmission>),\n    PushSubscription(GetResponse<PushSubscription>),\n    Sieve(GetResponse<Sieve>),\n    VacationResponse(GetResponse<VacationResponse>),\n    Principal(GetResponse<Principal>),\n    PrincipalAvailability(GetAvailabilityResponse),\n    Quota(GetResponse<Quota>),\n    Blob(GetResponse<Blob>),\n    AddressBook(GetResponse<AddressBook>),\n    ContactCard(GetResponse<ContactCard>),\n    FileNode(GetResponse<FileNode>),\n    Calendar(GetResponse<Calendar>),\n    CalendarEvent(GetResponse<CalendarEvent>),\n    CalendarEventNotification(CalendarEventNotificationGetResponse),\n    ParticipantIdentity(GetResponse<ParticipantIdentity>),\n    ShareNotification(GetResponse<ShareNotification>),\n}\n\n#[derive(Debug, serde::Serialize)]\n#[serde(untagged)]\npub enum SetResponseMethod {\n    Email(SetResponse<Email>),\n    Mailbox(SetResponse<Mailbox>),\n    Identity(SetResponse<Identity>),\n    EmailSubmission(SetResponse<EmailSubmission>),\n    PushSubscription(SetResponse<PushSubscription>),\n    Sieve(SetResponse<Sieve>),\n    VacationResponse(SetResponse<VacationResponse>),\n    AddressBook(SetResponse<AddressBook>),\n    ContactCard(SetResponse<ContactCard>),\n    FileNode(SetResponse<FileNode>),\n    ShareNotification(SetResponse<ShareNotification>),\n    Calendar(SetResponse<Calendar>),\n    CalendarEvent(SetResponse<CalendarEvent>),\n    CalendarEventNotification(SetResponse<CalendarEventNotification>),\n    ParticipantIdentity(SetResponse<ParticipantIdentity>),\n}\n\n#[derive(Debug, serde::Serialize)]\n#[serde(untagged)]\npub enum ChangesResponseMethod {\n    Email(ChangesResponse<Email>),\n    Mailbox(ChangesResponse<Mailbox>),\n    Thread(ChangesResponse<Thread>),\n    Identity(ChangesResponse<Identity>),\n    EmailSubmission(ChangesResponse<EmailSubmission>),\n    Quota(ChangesResponse<Quota>),\n    AddressBook(ChangesResponse<AddressBook>),\n    ContactCard(ChangesResponse<ContactCard>),\n    FileNode(ChangesResponse<FileNode>),\n    Calendar(ChangesResponse<Calendar>),\n    CalendarEvent(ChangesResponse<CalendarEvent>),\n    CalendarEventNotification(ChangesResponse<CalendarEventNotification>),\n    ShareNotification(ChangesResponse<ShareNotification>),\n}\n\n#[derive(Debug, serde::Serialize)]\n#[serde(untagged)]\npub enum CopyResponseMethod {\n    Email(CopyResponse<Email>),\n    ContactCard(CopyResponse<ContactCard>),\n    CalendarEvent(CopyResponse<CalendarEvent>),\n    Blob(CopyBlobResponse),\n}\n\n#[derive(Debug, serde::Serialize)]\n#[serde(untagged)]\npub enum ParseResponseMethod {\n    Email(ParseResponse<Email>),\n    ContactCard(ParseResponse<ContactCard>),\n    CalendarEvent(ParseResponse<CalendarEvent>),\n}\n\n#[derive(Debug, serde::Serialize)]\npub struct Response<'x> {\n    #[serde(rename = \"methodResponses\")]\n    pub method_responses: Vec<Call<ResponseMethod<'x>>>,\n\n    #[serde(rename = \"sessionState\")]\n    #[serde(serialize_with = \"serialize_hex\")]\n    pub session_state: u32,\n\n    #[serde(rename = \"createdIds\")]\n    #[serde(skip_serializing_if = \"HashMap::is_empty\")]\n    pub created_ids: HashMap<String, AnyId>,\n}\n\nimpl<'x> Response<'x> {\n    pub fn new(session_state: u32, created_ids: HashMap<String, AnyId>, capacity: usize) -> Self {\n        Response {\n            session_state,\n            created_ids,\n            method_responses: Vec::with_capacity(capacity),\n        }\n    }\n\n    pub fn push_response(\n        &mut self,\n        id: String,\n        name: MethodName,\n        method: impl Into<ResponseMethod<'x>>,\n    ) {\n        self.method_responses.push(Call {\n            id,\n            method: method.into(),\n            name,\n        });\n    }\n\n    pub fn push_error(&mut self, id: String, err: impl Into<MethodErrorWrapper>) {\n        self.method_responses.push(Call {\n            id,\n            method: ResponseMethod::Error(err.into()),\n            name: MethodName::error(),\n        });\n    }\n\n    pub fn push_created_id(&mut self, create_id: String, id: impl Into<AnyId>) {\n        self.created_ids.insert(create_id, id.into());\n    }\n}\n\nimpl From<trc::Error> for ResponseMethod<'_> {\n    fn from(error: trc::Error) -> Self {\n        ResponseMethod::Error(error.into())\n    }\n}\n\nimpl<'x, T: Into<ResponseMethod<'x>>> From<trc::Result<T>> for ResponseMethod<'x> {\n    fn from(result: trc::Result<T>) -> Self {\n        match result {\n            Ok(value) => value.into(),\n            Err(error) => error.into(),\n        }\n    }\n}\n\nimpl<'x> From<GetResponse<Email>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Email>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Email(value))\n    }\n}\n\nimpl<'x> From<GetResponse<Mailbox>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Mailbox>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Mailbox(value))\n    }\n}\n\nimpl<'x> From<GetResponse<Thread>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Thread>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Thread(value))\n    }\n}\n\nimpl<'x> From<GetResponse<Identity>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Identity>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Identity(value))\n    }\n}\n\nimpl<'x> From<GetResponse<EmailSubmission>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<EmailSubmission>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::EmailSubmission(value))\n    }\n}\n\nimpl<'x> From<GetResponse<PushSubscription>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<PushSubscription>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::PushSubscription(value))\n    }\n}\n\nimpl<'x> From<GetResponse<Sieve>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Sieve>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Sieve(value))\n    }\n}\n\nimpl<'x> From<GetResponse<VacationResponse>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<VacationResponse>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::VacationResponse(value))\n    }\n}\n\nimpl<'x> From<GetResponse<Principal>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Principal>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Principal(value))\n    }\n}\n\nimpl<'x> From<GetResponse<Quota>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Quota>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Quota(value))\n    }\n}\n\nimpl<'x> From<GetResponse<Blob>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<Blob>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Blob(value))\n    }\n}\n\nimpl<'x> From<GetResponse<ContactCard>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<ContactCard>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::ContactCard(value))\n    }\n}\n\nimpl<'x> From<GetResponse<AddressBook>> for ResponseMethod<'x> {\n    fn from(value: GetResponse<AddressBook>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::AddressBook(value))\n    }\n}\n\nimpl<'x> From<SetResponse<Email>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<Email>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::Email(value))\n    }\n}\n\nimpl<'x> From<SetResponse<Mailbox>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<Mailbox>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::Mailbox(value))\n    }\n}\n\nimpl<'x> From<SetResponse<Identity>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<Identity>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::Identity(value))\n    }\n}\n\nimpl<'x> From<SetResponse<EmailSubmission>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<EmailSubmission>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::EmailSubmission(value))\n    }\n}\n\nimpl<'x> From<SetResponse<PushSubscription>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<PushSubscription>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::PushSubscription(value))\n    }\n}\n\nimpl<'x> From<SetResponse<Sieve>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<Sieve>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::Sieve(value))\n    }\n}\n\nimpl<'x> From<SetResponse<VacationResponse>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<VacationResponse>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::VacationResponse(value))\n    }\n}\n\nimpl<'x> From<SetResponse<AddressBook>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<AddressBook>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::AddressBook(value))\n    }\n}\n\nimpl<'x> From<SetResponse<ContactCard>> for ResponseMethod<'x> {\n    fn from(value: SetResponse<ContactCard>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::ContactCard(value))\n    }\n}\n\nimpl<'x> From<ChangesResponse<Email>> for ResponseMethod<'x> {\n    fn from(value: ChangesResponse<Email>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::Email(value))\n    }\n}\n\nimpl<'x> From<ChangesResponse<Mailbox>> for ResponseMethod<'x> {\n    fn from(value: ChangesResponse<Mailbox>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::Mailbox(value))\n    }\n}\n\nimpl<'x> From<ChangesResponse<Thread>> for ResponseMethod<'x> {\n    fn from(value: ChangesResponse<Thread>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::Thread(value))\n    }\n}\n\nimpl<'x> From<ChangesResponse<Identity>> for ResponseMethod<'x> {\n    fn from(value: ChangesResponse<Identity>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::Identity(value))\n    }\n}\n\nimpl<'x> From<ChangesResponse<EmailSubmission>> for ResponseMethod<'x> {\n    fn from(value: ChangesResponse<EmailSubmission>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::EmailSubmission(value))\n    }\n}\n\nimpl<'x> From<ChangesResponse<Quota>> for ResponseMethod<'x> {\n    fn from(value: ChangesResponse<Quota>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::Quota(value))\n    }\n}\n\nimpl<'x> From<ChangesResponse<AddressBook>> for ResponseMethod<'x> {\n    fn from(value: ChangesResponse<AddressBook>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::AddressBook(value))\n    }\n}\n\nimpl<'x> From<CopyResponse<Email>> for ResponseMethod<'x> {\n    fn from(value: CopyResponse<Email>) -> Self {\n        ResponseMethod::Copy(CopyResponseMethod::Email(value))\n    }\n}\n\nimpl<'x> From<CopyBlobResponse> for ResponseMethod<'x> {\n    fn from(value: CopyBlobResponse) -> Self {\n        ResponseMethod::Copy(CopyResponseMethod::Blob(value))\n    }\n}\n\nimpl<'x> From<CopyResponse<ContactCard>> for ResponseMethod<'x> {\n    fn from(value: CopyResponse<ContactCard>) -> Self {\n        ResponseMethod::Copy(CopyResponseMethod::ContactCard(value))\n    }\n}\n\nimpl<'x> From<ImportEmailResponse> for ResponseMethod<'x> {\n    fn from(value: ImportEmailResponse) -> Self {\n        ResponseMethod::ImportEmail(value)\n    }\n}\n\nimpl<'x> From<ParseResponse<Email>> for ResponseMethod<'x> {\n    fn from(value: ParseResponse<Email>) -> Self {\n        ResponseMethod::Parse(ParseResponseMethod::Email(value))\n    }\n}\n\nimpl<'x> From<ParseResponse<ContactCard>> for ResponseMethod<'x> {\n    fn from(value: ParseResponse<ContactCard>) -> Self {\n        ResponseMethod::Parse(ParseResponseMethod::ContactCard(value))\n    }\n}\n\nimpl<'x> From<QueryChangesResponse> for ResponseMethod<'x> {\n    fn from(value: QueryChangesResponse) -> Self {\n        ResponseMethod::QueryChanges(value)\n    }\n}\n\nimpl<'x> From<QueryResponse> for ResponseMethod<'x> {\n    fn from(value: QueryResponse) -> Self {\n        ResponseMethod::Query(value)\n    }\n}\n\nimpl<'x> From<GetSearchSnippetResponse> for ResponseMethod<'x> {\n    fn from(value: GetSearchSnippetResponse) -> Self {\n        ResponseMethod::SearchSnippet(value)\n    }\n}\n\nimpl<'x> From<ValidateSieveScriptResponse> for ResponseMethod<'x> {\n    fn from(value: ValidateSieveScriptResponse) -> Self {\n        ResponseMethod::ValidateScript(value)\n    }\n}\n\nimpl<'x> From<BlobLookupResponse> for ResponseMethod<'x> {\n    fn from(value: BlobLookupResponse) -> Self {\n        ResponseMethod::LookupBlob(value)\n    }\n}\n\nimpl<'x> From<BlobUploadResponse> for ResponseMethod<'x> {\n    fn from(value: BlobUploadResponse) -> Self {\n        ResponseMethod::UploadBlob(value)\n    }\n}\n\nimpl<'x> From<Value<'x, Null, Null>> for ResponseMethod<'x> {\n    fn from(value: Value<'x, Null, Null>) -> Self {\n        ResponseMethod::Echo(value)\n    }\n}\n\nimpl<'x> From<MethodErrorWrapper> for ResponseMethod<'x> {\n    fn from(value: MethodErrorWrapper) -> Self {\n        ResponseMethod::Error(value)\n    }\n}\n\nimpl From<GetResponse<FileNode>> for ResponseMethod<'_> {\n    fn from(response: GetResponse<FileNode>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::FileNode(response))\n    }\n}\n\nimpl From<SetResponse<FileNode>> for ResponseMethod<'_> {\n    fn from(response: SetResponse<FileNode>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::FileNode(response))\n    }\n}\n\nimpl From<ChangesResponse<FileNode>> for ResponseMethod<'_> {\n    fn from(response: ChangesResponse<FileNode>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::FileNode(response))\n    }\n}\n\nimpl From<GetAvailabilityResponse> for ResponseMethod<'_> {\n    fn from(response: GetAvailabilityResponse) -> Self {\n        ResponseMethod::Get(GetResponseMethod::PrincipalAvailability(response))\n    }\n}\n\nimpl From<GetResponse<Calendar>> for ResponseMethod<'_> {\n    fn from(response: GetResponse<Calendar>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::Calendar(response))\n    }\n}\n\nimpl From<SetResponse<Calendar>> for ResponseMethod<'_> {\n    fn from(response: SetResponse<Calendar>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::Calendar(response))\n    }\n}\n\nimpl From<ChangesResponse<CalendarEvent>> for ResponseMethod<'_> {\n    fn from(response: ChangesResponse<CalendarEvent>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::CalendarEvent(response))\n    }\n}\n\nimpl From<ChangesResponse<CalendarEventNotification>> for ResponseMethod<'_> {\n    fn from(response: ChangesResponse<CalendarEventNotification>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::CalendarEventNotification(response))\n    }\n}\n\nimpl From<SetResponse<CalendarEvent>> for ResponseMethod<'_> {\n    fn from(response: SetResponse<CalendarEvent>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::CalendarEvent(response))\n    }\n}\n\nimpl From<SetResponse<ParticipantIdentity>> for ResponseMethod<'_> {\n    fn from(response: SetResponse<ParticipantIdentity>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::ParticipantIdentity(response))\n    }\n}\n\nimpl From<GetResponse<ParticipantIdentity>> for ResponseMethod<'_> {\n    fn from(response: GetResponse<ParticipantIdentity>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::ParticipantIdentity(response))\n    }\n}\n\nimpl From<ChangesResponse<ShareNotification>> for ResponseMethod<'_> {\n    fn from(response: ChangesResponse<ShareNotification>) -> Self {\n        ResponseMethod::Changes(ChangesResponseMethod::ShareNotification(response))\n    }\n}\n\nimpl From<SetResponse<ShareNotification>> for ResponseMethod<'_> {\n    fn from(response: SetResponse<ShareNotification>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::ShareNotification(response))\n    }\n}\n\nimpl From<GetResponse<ShareNotification>> for ResponseMethod<'_> {\n    fn from(response: GetResponse<ShareNotification>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::ShareNotification(response))\n    }\n}\n\nimpl From<GetResponse<CalendarEvent>> for ResponseMethod<'_> {\n    fn from(response: GetResponse<CalendarEvent>) -> Self {\n        ResponseMethod::Get(GetResponseMethod::CalendarEvent(response))\n    }\n}\n\nimpl From<ParseResponse<CalendarEvent>> for ResponseMethod<'_> {\n    fn from(value: ParseResponse<CalendarEvent>) -> Self {\n        ResponseMethod::Parse(ParseResponseMethod::CalendarEvent(value))\n    }\n}\n\nimpl From<CopyResponse<CalendarEvent>> for ResponseMethod<'_> {\n    fn from(value: CopyResponse<CalendarEvent>) -> Self {\n        ResponseMethod::Copy(CopyResponseMethod::CalendarEvent(value))\n    }\n}\n\nimpl From<CalendarEventNotificationGetResponse> for ResponseMethod<'_> {\n    fn from(value: CalendarEventNotificationGetResponse) -> Self {\n        ResponseMethod::Get(GetResponseMethod::CalendarEventNotification(value))\n    }\n}\n\nimpl From<SetResponse<CalendarEventNotification>> for ResponseMethod<'_> {\n    fn from(value: SetResponse<CalendarEventNotification>) -> Self {\n        ResponseMethod::Set(SetResponseMethod::CalendarEventNotification(value))\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/response/serialize.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ResponseMethod;\nuse crate::request::Call;\nuse serde::{Serialize, ser::SerializeSeq};\n\nimpl Serialize for Call<ResponseMethod<'_>> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        let mut seq = serializer.serialize_seq(3.into())?;\n        seq.serialize_element(&self.name.to_string())?;\n        seq.serialize_element(&self.method)?;\n        seq.serialize_element(&self.id)?;\n        seq.end()\n    }\n}\n\npub fn serialize_hex<S>(value: &u32, serializer: S) -> Result<S::Ok, S::Error>\nwhere\n    S: serde::Serializer,\n{\n    format!(\"{:x}\", value).serialize(serializer)\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/response/status.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::types::state::State;\nuse types::{id::Id, type_state::DataType};\nuse utils::map::vec_map::VecMap;\n\n#[derive(serde::Serialize, serde::Deserialize, Debug)]\n#[serde(tag = \"@type\")]\npub enum PushObject {\n    StateChange {\n        changed: VecMap<Id, VecMap<DataType, State>>,\n    },\n    EmailPush {\n        #[serde(rename = \"accountId\")]\n        account_id: Id,\n        email: EmailPushObject,\n    },\n    CalendarAlert {\n        #[serde(rename = \"accountId\")]\n        account_id: Id,\n        #[serde(rename = \"calendarEventId\")]\n        calendar_event_id: Id,\n        uid: String,\n        #[serde(rename = \"recurrenceId\")]\n        recurrence_id: Option<String>,\n        #[serde(rename = \"alertId\")]\n        alert_id: String,\n    },\n    Group {\n        entries: Vec<PushObject>,\n    },\n}\n\n#[derive(serde::Serialize, serde::Deserialize, Debug)]\npub struct EmailPushObject {\n    pub subject: String,\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/types/date.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fmt::Display, str::FromStr};\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]\npub struct UTCDate {\n    pub year: u16,\n    pub month: u8,\n    pub day: u8,\n    pub hour: u8,\n    pub minute: u8,\n    pub second: u8,\n    pub tz_before_gmt: bool,\n    pub tz_hour: u8,\n    pub tz_minute: u8,\n}\n\nimpl FromStr for UTCDate {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        // 2004 - 06 - 28 T 23 : 43 : 45 . 000 Z\n        // 1969 - 02 - 13 T 23 : 32 : 00 - 03 : 30\n        //   0     1    2    3    4    5    6    7\n\n        let mut pos = 0;\n        let mut parts = [0u32; 8];\n        let mut parts_sizes = [\n            4u32, // Year (0)\n            2u32, // Month (1)\n            2u32, // Day (2)\n            2u32, // Hour (3)\n            2u32, // Minute (4)\n            2u32, // Second (5)\n            2u32, // TZ Hour (6)\n            2u32, // TZ Minute (7)\n        ];\n        let mut skip_digits = false;\n        let mut is_plus = true;\n\n        for ch in s.as_bytes() {\n            match ch {\n                b'0'..=b'9' => {\n                    if !skip_digits {\n                        if parts_sizes[pos] > 0 {\n                            parts_sizes[pos] -= 1;\n                            parts[pos] += (ch - b'0') as u32 * u32::pow(10, parts_sizes[pos]);\n                        } else {\n                            break;\n                        }\n                    }\n                }\n                b'-' => {\n                    if pos <= 1 {\n                        pos += 1;\n                    } else if pos == 5 {\n                        pos += 1;\n                        is_plus = false;\n                        skip_digits = false;\n                    } else {\n                        break;\n                    }\n                }\n                b'T' => {\n                    if pos == 2 {\n                        pos += 1;\n                    } else {\n                        break;\n                    }\n                }\n                b':' => {\n                    if [3, 4, 6].contains(&pos) {\n                        pos += 1;\n                    } else {\n                        break;\n                    }\n                }\n                b'+' => {\n                    if pos == 5 {\n                        pos += 1;\n                        skip_digits = false;\n                    } else {\n                        break;\n                    }\n                }\n                b'.' => {\n                    if pos == 5 {\n                        skip_digits = true;\n                    } else {\n                        break;\n                    }\n                }\n                b'Z' | b'z' => (),\n                _ => {\n                    break;\n                }\n            }\n        }\n\n        if pos >= 5 {\n            Ok(UTCDate {\n                year: parts[0] as u16,\n                month: parts[1] as u8,\n                day: parts[2] as u8,\n                hour: parts[3] as u8,\n                minute: parts[4] as u8,\n                second: parts[5] as u8,\n                tz_hour: parts[6] as u8,\n                tz_minute: parts[7] as u8,\n                tz_before_gmt: !is_plus,\n            })\n        } else {\n            Err(())\n        }\n    }\n}\n\nimpl UTCDate {\n    pub fn from_timestamp(timestamp: i64) -> Self {\n        // Ported from http://howardhinnant.github.io/date_algorithms.html#civil_from_days\n        let (z, seconds) = ((timestamp / 86400) + 719468, timestamp % 86400);\n        let era: i64 = (if z >= 0 { z } else { z - 146096 }) / 146097;\n        let doe: u64 = (z - era * 146097) as u64; // [0, 146096]\n        let yoe: u64 = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365; // [0, 399]\n        let y: i64 = (yoe as i64) + era * 400;\n        let doy: u64 = doe - (365 * yoe + yoe / 4 - yoe / 100); // [0, 365]\n        let mp = (5 * doy + 2) / 153; // [0, 11]\n        let d: u64 = doy - (153 * mp + 2) / 5 + 1; // [1, 31]\n        let m: u64 = if mp < 10 { mp + 3 } else { mp - 9 }; // [1, 12]\n        let (h, mn, s) = (seconds / 3600, (seconds / 60) % 60, seconds % 60);\n\n        UTCDate {\n            year: (y + i64::from(m <= 2)) as u16,\n            month: m as u8,\n            day: d as u8,\n            hour: h as u8,\n            minute: mn as u8,\n            second: s as u8,\n            tz_before_gmt: false,\n            tz_hour: 0,\n            tz_minute: 0,\n        }\n    }\n\n    pub fn is_valid(&self) -> bool {\n        (0..=23).contains(&self.tz_hour)\n            && (1970..=3000).contains(&self.year)\n            && (0..=59).contains(&self.tz_minute)\n            && (1..=12).contains(&self.month)\n            && (1..=31).contains(&self.day)\n            && (0..=23).contains(&self.hour)\n            && (0..=59).contains(&self.minute)\n            && (0..=59).contains(&self.second)\n    }\n\n    pub fn timestamp(&self) -> i64 {\n        // Ported from https://github.com/protocolbuffers/upb/blob/22182e6e/upb/json_decode.c#L982-L992\n        let month = self.month as u32;\n        let year_base = 4800; /* Before min year, multiple of 400. */\n        let m_adj = month.wrapping_sub(3); /* March-based month. */\n        let carry = i64::from(m_adj > month);\n        let adjust = if carry > 0 { 12 } else { 0 };\n        let y_adj = self.year as i64 + year_base - carry;\n        let month_days = ((m_adj.wrapping_add(adjust)) * 62719 + 769) / 2048;\n        let leap_days = y_adj / 4 - y_adj / 100 + y_adj / 400;\n        (y_adj * 365 + leap_days + month_days as i64 + (self.day as i64 - 1) - 2472632) * 86400\n            + self.hour as i64 * 3600\n            + self.minute as i64 * 60\n            + self.second as i64\n            + ((self.tz_hour as i64 * 3600 + self.tz_minute as i64 * 60)\n                * if self.tz_before_gmt { 1 } else { -1 })\n    }\n}\n\nimpl Display for UTCDate {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        if self.tz_hour != 0 || self.tz_minute != 0 {\n            write!(\n                f,\n                \"{:04}-{:02}-{:02}T{:02}:{:02}:{:02}{}{:02}:{:02}\",\n                self.year,\n                self.month,\n                self.day,\n                self.hour,\n                self.minute,\n                self.second,\n                if self.tz_before_gmt && (self.tz_hour > 0 || self.tz_minute > 0) {\n                    \"-\"\n                } else {\n                    \"+\"\n                },\n                self.tz_hour,\n                self.tz_minute,\n            )\n        } else {\n            write!(\n                f,\n                \"{:04}-{:02}-{:02}T{:02}:{:02}:{:02}Z\",\n                self.year, self.month, self.day, self.hour, self.minute, self.second,\n            )\n        }\n    }\n}\n\nimpl serde::Serialize for UTCDate {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.to_string().as_str())\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for UTCDate {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        UTCDate::from_str(<&str>::deserialize(deserializer)?)\n            .map_err(|_| serde::de::Error::custom(\"invalid JMAP UTCDate\"))\n    }\n}\n\nimpl From<UTCDate> for u64 {\n    fn from(value: UTCDate) -> Self {\n        value.timestamp() as u64\n    }\n}\n\nimpl From<u64> for UTCDate {\n    fn from(value: u64) -> Self {\n        UTCDate::from_timestamp(value as i64)\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::types::date::UTCDate;\n    use std::str::FromStr;\n\n    #[test]\n    fn parse_jmap_date() {\n        for (input, expected_result) in [\n            (\"1997-11-21T09:55:06-06:00\", \"1997-11-21T09:55:06-06:00\"),\n            (\"1997-11-21T09:55:06+00:00\", \"1997-11-21T09:55:06Z\"),\n            (\"2021-01-01T09:55:06+02:00\", \"2021-01-01T09:55:06+02:00\"),\n            (\"2004-06-28T23:43:45.000Z\", \"2004-06-28T23:43:45Z\"),\n            (\"1997-11-21T09:55:06.123+00:00\", \"1997-11-21T09:55:06Z\"),\n            (\n                \"2021-01-01T09:55:06.4567+02:00\",\n                \"2021-01-01T09:55:06+02:00\",\n            ),\n        ] {\n            let date = UTCDate::from_str(input).unwrap();\n            assert_eq!(date.to_string(), expected_result);\n\n            let timestamp = date.timestamp();\n            assert_eq!(UTCDate::from_timestamp(timestamp).timestamp(), timestamp);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/jmap-proto/src/types/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod date;\npub mod state;\n"
  },
  {
    "path": "crates/jmap-proto/src/types/state.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse types::ChangeId;\nuse utils::codec::{\n    base32_custom::{Base32Reader, Base32Writer},\n    leb128::{Leb128Iterator, Leb128Writer},\n};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct JMAPIntermediateState {\n    pub from_id: ChangeId,\n    pub to_id: ChangeId,\n    pub items_sent: usize,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\npub enum State {\n    #[default]\n    Initial,\n    Exact(ChangeId),\n    Intermediate(JMAPIntermediateState),\n}\n\nimpl From<ChangeId> for State {\n    fn from(change_id: ChangeId) -> Self {\n        State::Exact(change_id)\n    }\n}\n\nimpl From<Option<ChangeId>> for State {\n    fn from(change_id: Option<ChangeId>) -> Self {\n        match change_id {\n            Some(change_id) => State::Exact(change_id),\n            None => State::Initial,\n        }\n    }\n}\n\nimpl State {\n    pub fn parse(value: &str) -> Option<Self> {\n        let mut it = value.as_bytes().iter();\n\n        match it.next()? {\n            b'n' => Some(State::Initial),\n            b's' => {\n                let mut reader = Base32Reader::from_iter(it);\n                reader.next_leb128::<ChangeId>().map(State::Exact)\n            }\n            b'r' => {\n                let mut it = Base32Reader::from_iter(it);\n\n                if let (Some(from_id), Some(to_id), Some(items_sent)) = (\n                    it.next_leb128::<ChangeId>(),\n                    it.next_leb128::<ChangeId>(),\n                    it.next_leb128::<usize>(),\n                ) {\n                    if items_sent > 0 {\n                        Some(State::Intermediate(JMAPIntermediateState {\n                            from_id,\n                            to_id: from_id.saturating_add(to_id),\n                            items_sent,\n                        }))\n                    } else {\n                        None\n                    }\n                } else {\n                    None\n                }\n            }\n            _ => None,\n        }\n    }\n\n    pub fn new_initial() -> Self {\n        State::Initial\n    }\n\n    pub fn new_exact(id: ChangeId) -> Self {\n        State::Exact(id)\n    }\n\n    pub fn new_intermediate(from_id: ChangeId, to_id: ChangeId, items_sent: usize) -> Self {\n        State::Intermediate(JMAPIntermediateState {\n            from_id,\n            to_id,\n            items_sent,\n        })\n    }\n\n    pub fn get_change_id(&self) -> ChangeId {\n        match self {\n            State::Exact(id) => *id,\n            State::Intermediate(intermediate) => intermediate.to_id,\n            State::Initial => ChangeId::MAX,\n        }\n    }\n}\n\nimpl serde::Serialize for State {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.to_string().as_str())\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for State {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        State::parse(<&str>::deserialize(deserializer)?)\n            .ok_or_else(|| serde::de::Error::custom(\"invalid JMAP State\"))\n    }\n}\n\nimpl std::fmt::Display for State {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let mut writer = Base32Writer::with_capacity(10);\n\n        match self {\n            State::Initial => {\n                writer.push_char('n');\n            }\n            State::Exact(id) => {\n                writer.push_char('s');\n                writer.write_leb128(*id).unwrap();\n            }\n            State::Intermediate(intermediate) => {\n                writer.push_char('r');\n                writer.write_leb128(intermediate.from_id).unwrap();\n                writer\n                    .write_leb128(intermediate.to_id - intermediate.from_id)\n                    .unwrap();\n                writer.write_leb128(intermediate.items_sent).unwrap();\n            }\n        }\n\n        f.write_str(&writer.finalize())\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::State;\n    use types::ChangeId;\n\n    #[test]\n    fn test_state_id() {\n        for id in [\n            State::new_initial(),\n            State::new_exact(0),\n            State::new_exact(12345678),\n            State::new_exact(ChangeId::MAX),\n            State::new_intermediate(0, 0, 1),\n            State::new_intermediate(1024, 2048, 100),\n            State::new_intermediate(12345678, 87654321, 1),\n            State::new_intermediate(0, 0, 12345678),\n            State::new_intermediate(0, 87654321, 12345678),\n            State::new_intermediate(12345678, 87654321, 1),\n            State::new_intermediate(12345678, 87654321, 12345678),\n            State::new_intermediate(ChangeId::MAX, ChangeId::MAX, ChangeId::MAX as usize),\n        ] {\n            assert_eq!(State::parse(&id.to_string()).unwrap(), id);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/main/Cargo.toml",
    "content": "[package]\nname = \"stalwart\"\ndescription = \"Stalwart Mail and Collaboration Server\"\nauthors = [ \"Stalwart Labs LLC <hello@stalw.art>\"]\nrepository = \"https://github.com/stalwartlabs/stalwart\"\nhomepage = \"https://stalw.art\"\nkeywords = [\"imap\", \"jmap\", \"smtp\", \"email\", \"mail\", \"webdav\", \"server\"]\ncategories = [\"email\"]\nlicense = \"AGPL-3.0-only OR LicenseRef-SEL\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[[bin]]\nname = \"stalwart\"\npath = \"src/main.rs\"\n\n[dependencies]\nstore = { path = \"../store\" }\njmap = { path = \"../jmap\" }\ntypes = { path = \"../types\" }\nsmtp = { path = \"../smtp\" }\nimap = { path = \"../imap\" }\npop3 = { path = \"../pop3\" }\nspam-filter = { path = \"../spam-filter\" }\nmanagesieve = { path = \"../managesieve\" }\ncommon = { path = \"../common\" }\nemail = { path = \"../email\" }\ndirectory = { path = \"../directory\" }\nhttp = { path = \"../http\" }\ndav = { path = \"../dav\" }\ngroupware = { path = \"../groupware\" }\nservices = { path = \"../services\" }\ntrc = { path = \"../trc\" }\nutils = { path = \"../utils\" }\nmigration = { path = \"../migration\" }\ntokio = { version = \"1.47\", features = [\"full\"] }\n\n[target.'cfg(not(target_env = \"msvc\"))'.dependencies]\njemallocator = \"0.5.0\"\n\n[features]\n#default = [\"sqlite\", \"postgres\", \"mysql\", \"rocks\", \"s3\", \"redis\", \"azure\", \"nats\", \"enterprise\"]\ndefault = [\"rocks\", \"enterprise\"]\nsqlite = [\"store/sqlite\"]\nfoundationdb = [\"store/foundation\", \"common/foundation\"]\npostgres = [\"store/postgres\"]\nmysql = [\"store/mysql\"]\nrocks = [\"store/rocks\"]\ns3 = [\"store/s3\"]\nredis = [\"store/redis\"]\nnats = [\"store/nats\"]\nazure = [\"store/azure\"]\nzenoh = [\"store/zenoh\"]\nkafka = [\"store/kafka\"]\nenterprise = [ \"jmap/enterprise\", \n               \"smtp/enterprise\", \n               \"common/enterprise\", \n               \"store/enterprise\", \n               \"managesieve/enterprise\", \n               \"directory/enterprise\", \n               \"email/enterprise\",\n               \"spam-filter/enterprise\",\n               \"http/enterprise\",\n               \"dav/enterprise\",\n               \"groupware/enterprise\",\n               \"trc/enterprise\",\n               \"services/enterprise\",\n               \"migration/enterprise\" ]\n"
  },
  {
    "path": "crates/main/src/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#![warn(clippy::large_futures)]\n#![warn(clippy::cast_possible_truncation)]\n#![warn(clippy::cast_possible_wrap)]\n#![warn(clippy::cast_sign_loss)]\n\nuse common::{config::server::ServerProtocol, core::BuildServer, manager::boot::BootManager};\nuse http::HttpSessionManager;\nuse imap::core::ImapSessionManager;\nuse managesieve::core::ManageSieveSessionManager;\nuse pop3::Pop3SessionManager;\nuse services::{StartServices, broadcast::subscriber::spawn_broadcast_subscriber};\nuse smtp::{StartQueueManager, core::SmtpSessionManager};\nuse std::time::Duration;\nuse trc::Collector;\nuse utils::wait_for_shutdown;\n\n#[cfg(not(target_env = \"msvc\"))]\nuse jemallocator::Jemalloc;\n\n#[cfg(not(target_env = \"msvc\"))]\n#[global_allocator]\nstatic GLOBAL: Jemalloc = Jemalloc;\n\n#[tokio::main]\nasync fn main() -> std::io::Result<()> {\n    // Load config and apply macros\n    let mut init = Box::pin(BootManager::init()).await;\n\n    // Migrate database\n    if let Err(err) = migration::try_migrate(&init.inner.build_server()).await {\n        trc::event!(\n            Server(trc::ServerEvent::StartupError),\n            Details = \"Failed to migrate database, aborting startup.\",\n            Reason = err,\n        );\n        return Ok(());\n    }\n\n    // Init services\n    init.start_services().await;\n    init.start_queue_manager();\n\n    // Log configuration errors\n    init.config.log_errors();\n    init.config.log_warnings();\n\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    // Log licensing information\n    #[cfg(feature = \"enterprise\")]\n    init.inner.build_server().log_license_details();\n    // SPDX-SnippetEnd\n\n    // Spawn servers\n    let (shutdown_tx, shutdown_rx) = init.servers.spawn(|server, acceptor, shutdown_rx| {\n        match &server.protocol {\n            ServerProtocol::Smtp | ServerProtocol::Lmtp => server.spawn(\n                SmtpSessionManager::new(init.inner.clone()),\n                init.inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Http => server.spawn(\n                HttpSessionManager::new(init.inner.clone()),\n                init.inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Imap => server.spawn(\n                ImapSessionManager::new(init.inner.clone()),\n                init.inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Pop3 => server.spawn(\n                Pop3SessionManager::new(init.inner.clone()),\n                init.inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::ManageSieve => server.spawn(\n                ManageSieveSessionManager::new(init.inner.clone()),\n                init.inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n        };\n    });\n\n    // Start broadcast subscriber\n    spawn_broadcast_subscriber(init.inner, shutdown_rx);\n\n    // Wait for shutdown signal\n    wait_for_shutdown().await;\n\n    // Shutdown collector\n    Collector::shutdown();\n\n    // Stop services\n    let _ = shutdown_tx.send(true);\n\n    // Wait for services to finish\n    tokio::time::sleep(Duration::from_secs(1)).await;\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/managesieve/Cargo.toml",
    "content": "[package]\nname = \"managesieve\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nimap_proto = { path = \"../imap-proto\" }\nimap = { path = \"../imap\" }\ntypes = { path = \"../types\" }\njmap_proto = { path = \"../jmap-proto\" }\ndirectory = { path = \"../directory\" }\ncommon = { path = \"../common\" }\nstore = { path = \"../store\" }\nutils = { path = \"../utils\" }\nemail = { path = \"../email\" }\ntrc = { path = \"../trc\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nsieve-rs = { version = \"0.7\", features = [\"rkyv\"] } \nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pemfile = \"2.0\"\ntokio = { version = \"1.47\", features = [\"full\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\nparking_lot = \"0.12\"\nahash = { version = \"0.8\" }\nmd5 = \"0.8.0\"\ncompact_str = \"0.9.0\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\n\n[features]\ntest_mode = []\nenterprise = []\n"
  },
  {
    "path": "crates/managesieve/src/core/client.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Command, ResponseCode, SerializeResponse, Session, State};\nuse common::{\n    KV_RATE_LIMIT_IMAP,\n    listener::{SessionResult, SessionStream},\n};\nuse imap_proto::receiver::{self, Request};\nuse tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt};\nuse trc::{AddContext, SecurityEvent};\nuse types::{collection::Collection, field::SieveField};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn ingest(&mut self, bytes: &[u8]) -> SessionResult {\n        let mut bytes = bytes.iter();\n        let mut requests = Vec::with_capacity(2);\n        let mut needs_literal = None;\n\n        loop {\n            match self.receiver.parse(&mut bytes) {\n                Ok(request) => match self.validate_request(request).await {\n                    Ok(request) => {\n                        requests.push(request);\n                    }\n                    Err(err) => {\n                        let mut disconnect = err.must_disconnect();\n\n                        if let Err(err) = self.write_error(err).await {\n                            trc::error!(err.span_id(self.session_id));\n                            disconnect = true;\n                        }\n\n                        if disconnect {\n                            return SessionResult::Close;\n                        }\n                    }\n                },\n                Err(receiver::Error::NeedsMoreData) => {\n                    break;\n                }\n                Err(receiver::Error::NeedsLiteral { size }) => {\n                    needs_literal = size.into();\n                    break;\n                }\n                Err(receiver::Error::Error { response }) => {\n                    // Check for port scanners\n                    if matches!(\n                        (&self.state, response.key(trc::Key::Code)),\n                        (\n                            State::NotAuthenticated { .. },\n                            Some(trc::Value::String(v))\n                        ) if v == \"PARSE\"\n                    ) {\n                        match self.server.is_scanner_fail2banned(self.remote_addr).await {\n                            Ok(true) => {\n                                trc::event!(\n                                    Security(SecurityEvent::ScanBan),\n                                    SpanId = self.session_id,\n                                    RemoteIp = self.remote_addr,\n                                    Reason = \"Invalid ManageSieve command\",\n                                );\n\n                                return SessionResult::Close;\n                            }\n                            Ok(false) => {}\n                            Err(err) => {\n                                trc::error!(\n                                    err.span_id(self.session_id)\n                                        .details(\"Failed to check for fail2ban\")\n                                );\n                            }\n                        }\n                    }\n\n                    if let Err(err) = self.write_error(response).await {\n                        trc::error!(err.span_id(self.session_id));\n                        return SessionResult::Close;\n                    }\n                    break;\n                }\n            }\n        }\n\n        for request in requests {\n            let command = request.command;\n            match match command {\n                Command::ListScripts => self.handle_listscripts().await,\n                Command::PutScript => self.handle_putscript(request).await,\n                Command::SetActive => self.handle_setactive(request).await,\n                Command::GetScript => self.handle_getscript(request).await,\n                Command::DeleteScript => self.handle_deletescript(request).await,\n                Command::RenameScript => self.handle_renamescript(request).await,\n                Command::CheckScript => self.handle_checkscript(request).await,\n                Command::HaveSpace => self.handle_havespace(request).await,\n                Command::Capability => self.handle_capability(\"\").await,\n                Command::Authenticate => self.handle_authenticate(request).await,\n                Command::StartTls => self.handle_start_tls().await,\n                Command::Logout => self.handle_logout().await,\n                Command::Noop => self.handle_noop(request).await,\n                Command::Unauthenticate => self.handle_unauthenticate().await,\n            } {\n                Ok(response) => {\n                    if let Err(err) = self.write(&response).await {\n                        trc::error!(err.span_id(self.session_id));\n                        return SessionResult::Close;\n                    }\n\n                    match command {\n                        Command::Logout => return SessionResult::Close,\n                        Command::StartTls => return SessionResult::UpgradeTls,\n                        _ => (),\n                    }\n                }\n                Err(err) => {\n                    let mut disconnect = err.must_disconnect();\n\n                    if let Err(err) = self.write_error(err).await {\n                        trc::error!(err.span_id(self.session_id));\n                        disconnect = true;\n                    }\n\n                    if disconnect {\n                        return SessionResult::Close;\n                    }\n                }\n            }\n        }\n\n        if let Some(needs_literal) = needs_literal\n            && let Err(err) = self\n                .write(format!(\"OK Ready for {} bytes.\\r\\n\", needs_literal).as_bytes())\n                .await\n        {\n            trc::error!(err.span_id(self.session_id));\n            return SessionResult::Close;\n        }\n\n        SessionResult::Continue\n    }\n\n    async fn validate_request(&self, command: Request<Command>) -> trc::Result<Request<Command>> {\n        match &command.command {\n            Command::Capability | Command::Logout | Command::Noop => Ok(command),\n            Command::Authenticate => {\n                if let State::NotAuthenticated { .. } = &self.state {\n                    if self.stream.is_tls() || self.server.core.imap.allow_plain_auth {\n                        Ok(command)\n                    } else {\n                        Err(trc::ManageSieveEvent::Error\n                            .into_err()\n                            .code(ResponseCode::EncryptNeeded)\n                            .details(\"Cannot authenticate over plain-text.\"))\n                    }\n                } else {\n                    Err(trc::ManageSieveEvent::Error\n                        .into_err()\n                        .details(\"Already authenticated.\"))\n                }\n            }\n            Command::StartTls => {\n                if !self.stream.is_tls() {\n                    Ok(command)\n                } else {\n                    Err(trc::ManageSieveEvent::Error\n                        .into_err()\n                        .details(\"Already in TLS mode.\"))\n                }\n            }\n            Command::HaveSpace\n            | Command::PutScript\n            | Command::ListScripts\n            | Command::SetActive\n            | Command::GetScript\n            | Command::DeleteScript\n            | Command::RenameScript\n            | Command::CheckScript\n            | Command::Unauthenticate => {\n                if let State::Authenticated { access_token, .. } = &self.state {\n                    if let Some(rate) = &self.server.core.imap.rate_requests {\n                        if self\n                            .server\n                            .core\n                            .storage\n                            .lookup\n                            .is_rate_allowed(\n                                KV_RATE_LIMIT_IMAP,\n                                &access_token.primary_id().to_be_bytes(),\n                                rate,\n                                true,\n                            )\n                            .await\n                            .caused_by(trc::location!())?\n                            .is_none()\n                        {\n                            Ok(command)\n                        } else {\n                            Err(trc::LimitEvent::TooManyRequests\n                                .into_err()\n                                .code(ResponseCode::TryLater))\n                        }\n                    } else {\n                        Ok(command)\n                    }\n                } else {\n                    Err(trc::ManageSieveEvent::Error\n                        .into_err()\n                        .details(\"Not authenticated.\"))\n                }\n            }\n        }\n    }\n}\n\nimpl<T: AsyncWrite + AsyncRead + Unpin> Session<T> {\n    #[inline(always)]\n    pub async fn write(&mut self, bytes: &[u8]) -> trc::Result<()> {\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::RawOutput),\n            SpanId = self.session_id,\n            Size = bytes.len(),\n            Contents = trc::Value::from_maybe_string(bytes),\n        );\n\n        self.stream.write_all(bytes).await.map_err(|err| {\n            trc::NetworkEvent::WriteError\n                .into_err()\n                .reason(err)\n                .caused_by(trc::location!())\n        })?;\n        self.stream.flush().await.map_err(|err| {\n            trc::NetworkEvent::FlushError\n                .into_err()\n                .reason(err)\n                .caused_by(trc::location!())\n        })?;\n\n        Ok(())\n    }\n\n    pub async fn write_error(&mut self, error: trc::Error) -> trc::Result<()> {\n        let bytes = error.serialize();\n        trc::error!(error.span_id(self.session_id));\n        self.write(&bytes).await\n    }\n\n    #[inline(always)]\n    pub async fn read(&mut self, bytes: &mut [u8]) -> trc::Result<usize> {\n        let len = self.stream.read(bytes).await.map_err(|err| {\n            trc::NetworkEvent::ReadError\n                .into_err()\n                .reason(err)\n                .caused_by(trc::location!())\n        })?;\n\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::RawInput),\n            SpanId = self.session_id,\n            Size = len,\n            Contents = trc::Value::from_maybe_string(bytes.get(0..len).unwrap_or_default()),\n        );\n\n        Ok(len)\n    }\n}\n\nimpl<T: AsyncWrite + AsyncRead> Session<T> {\n    pub async fn get_script_id(&self, account_id: u32, name: &str) -> trc::Result<u32> {\n        self.server\n            .document_ids_matching(\n                account_id,\n                Collection::SieveScript,\n                SieveField::Name,\n                name.to_lowercase().as_bytes(),\n            )\n            .await\n            .caused_by(trc::location!())\n            .and_then(|results| {\n                results.min().ok_or_else(|| {\n                    trc::ManageSieveEvent::Error\n                        .into_err()\n                        .code(ResponseCode::NonExistent)\n                        .reason(\"There is no script by that name\")\n                })\n            })\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/core/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod client;\npub mod session;\n\nuse std::{borrow::Cow, net::IpAddr, sync::Arc};\n\nuse common::{\n    Inner, Server,\n    auth::AccessToken,\n    listener::{ServerInstance, limiter::InFlight},\n};\n\nuse compact_str::CompactString;\nuse imap_proto::receiver::{CommandParser, Receiver};\nuse tokio::io::{AsyncRead, AsyncWrite};\n\npub struct Session<T: AsyncRead + AsyncWrite> {\n    pub server: Server,\n    pub instance: Arc<ServerInstance>,\n    pub receiver: Receiver<Command>,\n    pub state: State,\n    pub remote_addr: IpAddr,\n    pub stream: T,\n    pub session_id: u64,\n    pub in_flight: InFlight,\n}\n\npub enum State {\n    NotAuthenticated {\n        auth_failures: u32,\n    },\n    Authenticated {\n        access_token: Arc<AccessToken>,\n        in_flight: Option<InFlight>,\n    },\n}\n\nimpl State {\n    pub fn access_token(&self) -> &AccessToken {\n        match self {\n            State::Authenticated { access_token, .. } => access_token,\n            State::NotAuthenticated { .. } => unreachable!(\"Not authenticated\"),\n        }\n    }\n}\n\n#[derive(Clone)]\npub struct ManageSieveSessionManager {\n    pub inner: Arc<Inner>,\n}\n\nimpl ManageSieveSessionManager {\n    pub fn new(inner: Arc<Inner>) -> Self {\n        Self { inner }\n    }\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]\npub enum Command {\n    Authenticate,\n    StartTls,\n    Logout,\n    Capability,\n    HaveSpace,\n    PutScript,\n    ListScripts,\n    SetActive,\n    GetScript,\n    DeleteScript,\n    RenameScript,\n    CheckScript,\n    #[default]\n    Noop,\n    Unauthenticate,\n}\n\nimpl CommandParser for Command {\n    fn parse(value: &[u8], _is_uid: bool) -> Option<Self> {\n        match value {\n            b\"AUTHENTICATE\" => Some(Command::Authenticate),\n            b\"STARTTLS\" => Some(Command::StartTls),\n            b\"LOGOUT\" => Some(Command::Logout),\n            b\"CAPABILITY\" => Some(Command::Capability),\n            b\"HAVESPACE\" => Some(Command::HaveSpace),\n            b\"PUTSCRIPT\" => Some(Command::PutScript),\n            b\"LISTSCRIPTS\" => Some(Command::ListScripts),\n            b\"SETACTIVE\" => Some(Command::SetActive),\n            b\"GETSCRIPT\" => Some(Command::GetScript),\n            b\"DELETESCRIPT\" => Some(Command::DeleteScript),\n            b\"RENAMESCRIPT\" => Some(Command::RenameScript),\n            b\"CHECKSCRIPT\" => Some(Command::CheckScript),\n            b\"NOOP\" => Some(Command::Noop),\n            b\"UNAUTHENTICATE\" => Some(Command::Unauthenticate),\n            _ => None,\n        }\n    }\n\n    fn tokenize_brackets(&self) -> bool {\n        false\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct StatusResponse {\n    pub code: Option<ResponseCode>,\n    pub message: Cow<'static, str>,\n    pub rtype: ResponseType,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ResponseType {\n    Ok,\n    No,\n    Bye,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ResponseCode {\n    AuthTooWeak,\n    EncryptNeeded,\n    Quota,\n    QuotaMaxScripts,\n    QuotaMaxSize,\n    Referral,\n    Sasl,\n    TransitionNeeded,\n    TryLater,\n    Active,\n    NonExistent,\n    AlreadyExists,\n    Tag(String),\n    Warnings,\n}\n\nimpl ResponseCode {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(match self {\n            ResponseCode::AuthTooWeak => b\"AUTH-TOO-WEAK\",\n            ResponseCode::EncryptNeeded => b\"ENCRYPT-NEEDED\",\n            ResponseCode::Quota => b\"QUOTA\",\n            ResponseCode::QuotaMaxScripts => b\"QUOTA/MAXSCRIPTS\",\n            ResponseCode::QuotaMaxSize => b\"QUOTA/MAXSIZE\",\n            ResponseCode::Referral => b\"REFERRAL\",\n            ResponseCode::Sasl => b\"SASL\",\n            ResponseCode::TransitionNeeded => b\"TRANSITION-NEEDED\",\n            ResponseCode::TryLater => b\"TRYLATER\",\n            ResponseCode::Active => b\"ACTIVE\",\n            ResponseCode::NonExistent => b\"NONEXISTENT\",\n            ResponseCode::AlreadyExists => b\"ALREADYEXISTS\",\n            ResponseCode::Tag(tag) => {\n                buf.extend_from_slice(b\"TAG {\");\n                buf.extend_from_slice(tag.len().to_string().as_bytes());\n                buf.extend_from_slice(b\"}\\r\\n\");\n                buf.extend_from_slice(tag.as_bytes());\n                return;\n            }\n            ResponseCode::Warnings => b\"WARNINGS\",\n        });\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            ResponseCode::AuthTooWeak => \"AUTH-TOO-WEAK\",\n            ResponseCode::EncryptNeeded => \"ENCRYPT-NEEDED\",\n            ResponseCode::Quota => \"QUOTA\",\n            ResponseCode::QuotaMaxScripts => \"QUOTA/MAXSCRIPTS\",\n            ResponseCode::QuotaMaxSize => \"QUOTA/MAXSIZE\",\n            ResponseCode::Referral => \"REFERRAL\",\n            ResponseCode::Sasl => \"SASL\",\n            ResponseCode::TransitionNeeded => \"TRANSITION-NEEDED\",\n            ResponseCode::TryLater => \"TRYLATER\",\n            ResponseCode::Active => \"ACTIVE\",\n            ResponseCode::NonExistent => \"NONEXISTENT\",\n            ResponseCode::AlreadyExists => \"ALREADYEXISTS\",\n            ResponseCode::Tag(_) => \"TAG\",\n            ResponseCode::Warnings => \"WARNINGS\",\n        }\n    }\n}\n\nimpl ResponseType {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(self.as_str().as_bytes());\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            ResponseType::Ok => \"OK\",\n            ResponseType::No => \"NO\",\n            ResponseType::Bye => \"BYE\",\n        }\n    }\n}\n\nimpl StatusResponse {\n    pub fn serialize(self, mut buf: Vec<u8>) -> Vec<u8> {\n        self.rtype.serialize(&mut buf);\n        if let Some(code) = &self.code {\n            buf.extend_from_slice(b\" (\");\n            code.serialize(&mut buf);\n            buf.push(b')');\n        }\n        if !self.message.is_empty() {\n            buf.extend_from_slice(b\" \\\"\");\n            for ch in self.message.as_bytes() {\n                if [b'\\\"', b'\\\\'].contains(ch) {\n                    buf.push(b'\\\\');\n                }\n                buf.push(*ch);\n            }\n            buf.push(b'\\\"');\n        }\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n\n    pub fn into_bytes(self) -> Vec<u8> {\n        self.serialize(Vec::with_capacity(16))\n    }\n\n    pub fn with_code(mut self, code: ResponseCode) -> Self {\n        self.code = Some(code);\n        self\n    }\n\n    pub fn no(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            code: None,\n            message: message.into(),\n            rtype: ResponseType::No,\n        }\n    }\n\n    pub fn ok(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            code: None,\n            message: message.into(),\n            rtype: ResponseType::Ok,\n        }\n    }\n\n    pub fn bye(message: impl Into<Cow<'static, str>>) -> Self {\n        StatusResponse {\n            code: None,\n            message: message.into(),\n            rtype: ResponseType::Bye,\n        }\n    }\n\n    pub fn database_failure() -> Self {\n        StatusResponse {\n            code: Some(ResponseCode::TryLater),\n            message: Cow::Borrowed(\"Database failure\"),\n            rtype: ResponseType::No,\n        }\n    }\n}\n\npub trait SerializeResponse {\n    fn serialize(&self) -> Vec<u8>;\n}\n\nimpl SerializeResponse for trc::Error {\n    fn serialize(&self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(64);\n        buf.extend_from_slice(self.value_as_str(trc::Key::Type).unwrap_or(\"NO\").as_bytes());\n        if let Some(code) = self\n            .value_as_str(trc::Key::Code)\n            .or_else(|| match self.as_ref() {\n                trc::EventType::Store(trc::StoreEvent::NotFound) => {\n                    Some(ResponseCode::NonExistent.as_str())\n                }\n                trc::EventType::Store(_) => Some(ResponseCode::TryLater.as_str()),\n                trc::EventType::Limit(trc::LimitEvent::Quota) => Some(ResponseCode::Quota.as_str()),\n                trc::EventType::Limit(_) => Some(ResponseCode::TryLater.as_str()),\n                _ => None,\n            })\n        {\n            buf.extend_from_slice(b\" (\");\n            buf.extend_from_slice(code.as_bytes());\n            buf.push(b')');\n        }\n        let message = self\n            .value_as_str(trc::Key::Details)\n            .unwrap_or_else(|| self.as_ref().message());\n        buf.extend_from_slice(b\" \\\"\");\n        for ch in message.as_bytes() {\n            if [b'\\\"', b'\\\\'].contains(ch) {\n                buf.push(b'\\\\');\n            }\n            buf.push(*ch);\n        }\n        buf.push(b'\\\"');\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\nimpl From<ResponseCode> for trc::Value {\n    fn from(value: ResponseCode) -> Self {\n        trc::Value::String(CompactString::const_new(value.as_str()))\n    }\n}\n\nimpl From<ResponseType> for trc::Value {\n    fn from(value: ResponseType) -> Self {\n        trc::Value::String(CompactString::const_new(value.as_str()))\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/core/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    core::BuildServer,\n    listener::{SessionData, SessionManager, SessionResult, SessionStream},\n};\nuse imap_proto::receiver::{self, Receiver};\nuse tokio_rustls::server::TlsStream;\n\nuse crate::SERVER_GREETING;\n\nuse super::{ManageSieveSessionManager, Session, State};\n\nimpl SessionManager for ManageSieveSessionManager {\n    #[allow(clippy::manual_async_fn)]\n    fn handle<T: SessionStream>(\n        self,\n        session: SessionData<T>,\n    ) -> impl std::future::Future<Output = ()> + Send {\n        async move {\n            // Create session\n            let server = self.inner.build_server();\n            let mut session = Session {\n                receiver: Receiver::with_max_request_size(server.core.imap.max_request_size)\n                    .with_start_state(receiver::State::Command { is_uid: false }),\n                server,\n                instance: session.instance,\n                state: State::NotAuthenticated { auth_failures: 0 },\n                session_id: session.session_id,\n                stream: session.stream,\n                in_flight: session.in_flight,\n                remote_addr: session.remote_ip,\n            };\n\n            if session\n                .write(&session.handle_capability(SERVER_GREETING).await.unwrap())\n                .await\n                .is_ok()\n                && session.handle_conn().await\n                && session.instance.acceptor.is_tls()\n                && let Ok(mut session) = session.into_tls().await\n            {\n                let _ = session\n                    .write(&session.handle_capability(SERVER_GREETING).await.unwrap())\n                    .await;\n                session.handle_conn().await;\n            }\n        }\n    }\n\n    #[allow(clippy::manual_async_fn)]\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send {\n        async {}\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_conn(&mut self) -> bool {\n        let mut buf = vec![0; 8192];\n        let mut shutdown_rx = self.instance.shutdown_rx.clone();\n\n        loop {\n            tokio::select! {\n                result = tokio::time::timeout(\n                    if !matches!(self.state, State::NotAuthenticated {..}) {\n                        self.server.core.imap.timeout_auth\n                    } else {\n                        self.server.core.imap.timeout_unauth\n                    },\n                    self.read(&mut buf)) => {\n                        match result {\n                            Ok(Ok(bytes_read)) => {\n                                if bytes_read > 0 {\n                                    match self.ingest(&buf[..bytes_read]).await {\n                                        SessionResult::Continue => (),\n                                        SessionResult::UpgradeTls => {\n                                            return true;\n                                        }\n                                        SessionResult::Close => {\n                                            break;\n                                        }\n                                    }\n                                } else {\n                                    trc::event!(\n                                        Network(trc::NetworkEvent::Closed),\n                                        SpanId = self.session_id,\n                                        CausedBy = trc::location!()\n                                    );\n                                    break;\n                                }\n                            }\n                            Ok(Err(err)) => {\n                                trc::event!(\n                                    Network(trc::NetworkEvent::ReadError),\n                                    SpanId = self.session_id,\n                                    Reason = err,\n                                    CausedBy = trc::location!()\n                                );\n                                break;\n                            }\n                            Err(_) => {\n                                trc::event!(\n                                    Network(trc::NetworkEvent::Timeout),\n                                    SpanId = self.session_id,\n                                    CausedBy = trc::location!()\n                                );\n                                self\n                                    .write(b\"BYE \\\"Connection timed out.\\\"\\r\\n\")\n                                    .await\n                                    .ok();\n                                break;\n                            }\n                        }\n                },\n                _ = shutdown_rx.changed() => {\n                    trc::event!(\n                        Network(trc::NetworkEvent::Closed),\n                        SpanId = self.session_id,\n                        Reason = \"Server shutting down\",\n                        CausedBy = trc::location!()\n                    );\n                    self.write(b\"BYE \\\"Server shutting down.\\\"\\r\\n\").await.ok();\n                    break;\n                }\n            };\n        }\n\n        false\n    }\n\n    pub async fn into_tls(self) -> Result<Session<TlsStream<T>>, ()> {\n        Ok(Session {\n            stream: self\n                .instance\n                .tls_accept(self.stream, self.session_id)\n                .await?,\n            state: self.state,\n            instance: self.instance,\n            in_flight: self.in_flight,\n            session_id: self.session_id,\n            server: self.server,\n            receiver: self.receiver,\n            remote_addr: self.remote_addr,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod core;\npub mod op;\n\nstatic SERVER_GREETING: &str = \"Stalwart ManageSieve at your service.\";\n\n#[cfg(test)]\nmod tests {\n    use imap_proto::receiver::{Error, Receiver, Request, State, Token};\n\n    use crate::core::Command;\n\n    #[test]\n    fn receiver_parse_managesieve() {\n        let mut receiver = Receiver::new().with_start_state(State::Command { is_uid: false });\n\n        for (frames, expected_requests) in [\n            (\n                vec![\"Authenticate \\\"DIGEST-MD5\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::Authenticate,\n                    tokens: vec![Token::Argument(b\"DIGEST-MD5\".to_vec())],\n                }],\n            ),\n            (\n                vec![\n                    \"  AUTHENTICATE  \\\"GSSAPI\\\"  {56+}\\r\\n\",\n                    \"cnNwYXV0aD1lYTQwZjYwMzM1YzQyN2I1NTI3Yjg0ZGJhYmNkZmZmZA==\\r\\n\",\n                ],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::Authenticate,\n                    tokens: vec![\n                        Token::Argument(b\"GSSAPI\".to_vec()),\n                        Token::Argument(\n                            b\"cnNwYXV0aD1lYTQwZjYwMzM1YzQyN2I1NTI3Yjg0ZGJhYmNkZmZmZA==\".to_vec(),\n                        ),\n                    ],\n                }],\n            ),\n            (\n                vec![\"Authenticate \\\"PLAIN\\\" \\\"QJIrweAPyo6Q1T9xu\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::Authenticate,\n                    tokens: vec![\n                        Token::Argument(b\"PLAIN\".to_vec()),\n                        Token::Argument(b\"QJIrweAPyo6Q1T9xu\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"StartTls\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::StartTls,\n                    tokens: vec![],\n                }],\n            ),\n            (\n                vec![\"HAVESPACE \\\"myscript\\\" 999999\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::HaveSpace,\n                    tokens: vec![\n                        Token::Argument(b\"myscript\".to_vec()),\n                        Token::Argument(b\"999999\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\n                    \"Putscript \\\"foo\\\" {31+}\\r\\n\",\n                    \"#comment\\r\\n\",\n                    \"InvalidSieveCommand\\r\\n\\r\\n\",\n                ],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::PutScript,\n                    tokens: vec![\n                        Token::Argument(b\"foo\".to_vec()),\n                        Token::Argument(b\"#comment\\r\\nInvalidSieveCommand\\r\\n\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"Listscripts\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::ListScripts,\n                    tokens: vec![],\n                }],\n            ),\n            (\n                vec![\"Setactive \\\"baz\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::SetActive,\n                    tokens: vec![Token::Argument(b\"baz\".to_vec())],\n                }],\n            ),\n            (\n                vec![\"Renamescript \\\"foo\\\" \\\"bar\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::RenameScript,\n                    tokens: vec![\n                        Token::Argument(b\"foo\".to_vec()),\n                        Token::Argument(b\"bar\".to_vec()),\n                    ],\n                }],\n            ),\n            (\n                vec![\"NOOP \\\"STARTTLS-SYNC-42\\\"\\r\\n\"],\n                vec![Request {\n                    tag: \"\".into(),\n                    command: Command::Noop,\n                    tokens: vec![Token::Argument(b\"STARTTLS-SYNC-42\".to_vec())],\n                }],\n            ),\n        ] {\n            let mut requests = Vec::new();\n            for frame in &frames {\n                let mut bytes = frame.as_bytes().iter();\n                loop {\n                    match receiver.parse(&mut bytes) {\n                        Ok(request) => requests.push(request),\n                        Err(Error::NeedsMoreData | Error::NeedsLiteral { .. }) => break,\n                        Err(err) => panic!(\"{:?} for frames {:#?}\", err, frames),\n                    }\n                }\n            }\n            assert_eq!(requests, expected_requests, \"{:#?}\", frames);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/authenticate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    auth::{\n        AuthRequest,\n        sasl::{sasl_decode_challenge_oauth, sasl_decode_challenge_plain},\n    },\n    listener::{SessionStream, limiter::LimiterResult},\n};\n\nuse directory::Permission;\nuse imap_proto::{\n    protocol::authenticate::Mechanism,\n    receiver::{self, Request},\n};\nuse mail_parser::decoders::base64::base64_decode;\n\nuse crate::core::{Command, Session, State, StatusResponse};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_authenticate(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        if request.tokens.is_empty() {\n            return Err(trc::AuthEvent::Error\n                .into_err()\n                .details(\"Authentication mechanism missing.\"));\n        }\n\n        let mut tokens = request.tokens.into_iter();\n        let mechanism = Mechanism::parse(&tokens.next().unwrap().unwrap_bytes())\n            .map_err(|err| trc::AuthEvent::Error.into_err().details(err))?;\n        let mut params: Vec<String> = tokens\n            .filter_map(|token| token.unwrap_string().ok())\n            .collect();\n\n        let credentials = match mechanism {\n            Mechanism::Plain | Mechanism::OAuthBearer | Mechanism::XOauth2 => {\n                if !params.is_empty() {\n                    base64_decode(params.pop().unwrap().as_bytes())\n                        .and_then(|challenge| {\n                            if mechanism == Mechanism::Plain {\n                                sasl_decode_challenge_plain(&challenge)\n                            } else {\n                                sasl_decode_challenge_oauth(&challenge)\n                            }\n                        })\n                        .ok_or_else(|| {\n                            trc::AuthEvent::Error\n                                .into_err()\n                                .details(\"Failed to decode challenge.\")\n                        })?\n                } else {\n                    self.receiver.request = receiver::Request {\n                        tag: \"\".into(),\n                        command: Command::Authenticate,\n                        tokens: vec![receiver::Token::Argument(mechanism.into_bytes())],\n                    };\n                    self.receiver.state = receiver::State::Argument { last_ch: b' ' };\n                    return Ok(b\"{0}\\r\\n\".to_vec());\n                }\n            }\n            _ => {\n                return Err(trc::AuthEvent::Error\n                    .into_err()\n                    .details(\"Authentication mechanism not supported.\"));\n            }\n        };\n\n        // Authenticate\n        let access_token = self\n            .server\n            .authenticate(&AuthRequest::from_credentials(\n                credentials,\n                self.session_id,\n                self.remote_addr,\n            ))\n            .await\n            .map_err(|err| {\n                if err.matches(trc::EventType::Auth(trc::AuthEvent::Failed)) {\n                    match &self.state {\n                        State::NotAuthenticated { auth_failures }\n                            if *auth_failures < self.server.core.imap.max_auth_failures =>\n                        {\n                            self.state = State::NotAuthenticated {\n                                auth_failures: auth_failures + 1,\n                            };\n                        }\n                        _ => {\n                            return trc::AuthEvent::TooManyAttempts.into_err().caused_by(err);\n                        }\n                    }\n                }\n\n                err\n            })\n            .and_then(|token| {\n                token\n                    .assert_has_permission(Permission::SieveAuthenticate)\n                    .map(|_| token)\n            })?;\n\n        // Enforce concurrency limits\n        let in_flight = match access_token.is_imap_request_allowed() {\n            LimiterResult::Allowed(in_flight) => Some(in_flight),\n            LimiterResult::Forbidden => {\n                return Err(trc::LimitEvent::ConcurrentRequest.into_err());\n            }\n            LimiterResult::Disabled => None,\n        };\n\n        // Create session\n        self.state = State::Authenticated {\n            access_token,\n            in_flight,\n        };\n\n        Ok(StatusResponse::ok(\"Authentication successful\").into_bytes())\n    }\n\n    pub async fn handle_unauthenticate(&mut self) -> trc::Result<Vec<u8>> {\n        self.state = State::NotAuthenticated { auth_failures: 0 };\n\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::Unauthenticate),\n            SpanId = self.session_id,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        Ok(StatusResponse::ok(\"Unauthenticate successful.\").into_bytes())\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/capability.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::{Session, StatusResponse};\nuse common::listener::SessionStream;\nuse jmap_proto::request::capability::Capabilities;\nuse std::time::Instant;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_capability(&self, message: &'static str) -> trc::Result<Vec<u8>> {\n        let op_start = Instant::now();\n\n        let mut response = Vec::with_capacity(128);\n        response.extend_from_slice(b\"\\\"IMPLEMENTATION\\\" \\\"Stalwart ManageSieve\\\"\\r\\n\");\n        response.extend_from_slice(b\"\\\"VERSION\\\" \\\"1.0\\\"\\r\\n\");\n        if !self.stream.is_tls() {\n            response.extend_from_slice(b\"\\\"STARTTLS\\\"\\r\\n\");\n        }\n        if self.stream.is_tls() || self.server.core.imap.allow_plain_auth {\n            response.extend_from_slice(b\"\\\"SASL\\\" \\\"PLAIN OAUTHBEARER XOAUTH2\\\"\\r\\n\");\n        } else {\n            response.extend_from_slice(b\"\\\"SASL\\\" \\\"OAUTHBEARER XOAUTH2\\\"\\r\\n\");\n        };\n        if let Some(sieve) =\n            self.server\n                .core\n                .jmap\n                .capabilities\n                .account\n                .iter()\n                .find_map(|(_, item)| {\n                    if let Capabilities::SieveAccount(sieve) = item {\n                        Some(sieve)\n                    } else {\n                        None\n                    }\n                })\n        {\n            response.extend_from_slice(b\"\\\"SIEVE\\\" \\\"\");\n            response.extend_from_slice(sieve.extensions.join(\" \").as_bytes());\n            response.extend_from_slice(b\"\\\"\\r\\n\");\n            if let Some(notification_methods) = &sieve.notification_methods {\n                response.extend_from_slice(b\"\\\"NOTIFY\\\" \\\"\");\n                response.extend_from_slice(notification_methods.join(\" \").as_bytes());\n                response.extend_from_slice(b\"\\\"\\r\\n\");\n            }\n            if sieve.max_redirects > 0 {\n                response.extend_from_slice(b\"\\\"MAXREDIRECTS\\\" \\\"\");\n                response.extend_from_slice(sieve.max_redirects.to_string().as_bytes());\n                response.extend_from_slice(b\"\\\"\\r\\n\");\n            }\n        } else {\n            response.extend_from_slice(b\"\\\"SIEVE\\\" \\\"\\\"\\r\\n\");\n        }\n\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::Capabilities),\n            SpanId = self.session_id,\n            Tls = self.stream.is_tls(),\n            Strict = !self.server.core.imap.allow_plain_auth,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::ok(message).serialize(response))\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/checkscript.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::receiver::Request;\n\nuse crate::core::{Command, Session, StatusResponse};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_checkscript(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SieveCheckScript)?;\n\n        let op_start = Instant::now();\n\n        if request.tokens.is_empty() {\n            return Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(\"Expected script as a parameter.\"));\n        }\n\n        let script = request.tokens.into_iter().next().unwrap().unwrap_bytes();\n        self.server\n            .core\n            .sieve\n            .untrusted_compiler\n            .compile(&script)\n            .map(|_| {\n                trc::event!(\n                    ManageSieve(trc::ManageSieveEvent::CheckScript),\n                    SpanId = self.session_id,\n                    Size = script.len(),\n                    Elapsed = op_start.elapsed()\n                );\n\n                StatusResponse::ok(\"Script is valid.\").into_bytes()\n            })\n            .map_err(|err| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(err.to_string())\n            })\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/deletescript.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::{Command, ResponseCode, Session, StatusResponse};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::sieve::{delete::SieveScriptDelete, ingest::SieveScriptIngest};\nuse imap_proto::receiver::Request;\nuse std::time::Instant;\nuse store::write::BatchBuilder;\nuse trc::AddContext;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_deletescript(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SieveDeleteScript)?;\n\n        let op_start = Instant::now();\n\n        let name = request\n            .tokens\n            .into_iter()\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected script name as a parameter.\")\n            })?;\n\n        let access_token = self.state.access_token();\n        let account_id = access_token.primary_id();\n        let document_id = self.get_script_id(account_id, &name).await?;\n        let mut batch = BatchBuilder::new();\n\n        let active_script_id = self.server.sieve_script_get_active_id(account_id).await?;\n        if active_script_id != Some(document_id) {\n            if self\n                .server\n                .sieve_script_delete(account_id, document_id, access_token, &mut batch)\n                .await\n                .caused_by(trc::location!())?\n            {\n                if !batch.is_empty() {\n                    self.server\n                        .commit_batch(batch)\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n\n                trc::event!(\n                    ManageSieve(trc::ManageSieveEvent::DeleteScript),\n                    SpanId = self.session_id,\n                    Id = name,\n                    DocumentId = document_id,\n                    Elapsed = op_start.elapsed()\n                );\n\n                Ok(StatusResponse::ok(\"Deleted.\").into_bytes())\n            } else {\n                Err(trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Script not found\"))\n            }\n        } else {\n            Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(\"You may not delete an active script\")\n                .code(ResponseCode::Active))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/getscript.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::{Command, ResponseCode, Session, StatusResponse};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::sieve::SieveScript;\nuse imap_proto::receiver::Request;\nuse std::time::Instant;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{blob::BlobSection, blob_hash::BlobHash, collection::Collection};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_getscript(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SieveGetScript)?;\n\n        let op_start = Instant::now();\n        let name = request\n            .tokens\n            .into_iter()\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected script name as a parameter.\")\n            })?;\n        let account_id = self.state.access_token().primary_id();\n        let document_id = self.get_script_id(account_id, &name).await?;\n        let sieve_ = self\n            .server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::SieveScript,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Script not found\")\n                    .code(ResponseCode::NonExistent)\n            })?;\n        let sieve = sieve_\n            .unarchive::<SieveScript>()\n            .caused_by(trc::location!())?;\n        let blob_size = u32::from(sieve.size) as usize;\n        let script = self\n            .server\n            .get_blob_section(\n                &BlobHash::from(&sieve.blob_hash),\n                &BlobSection {\n                    size: blob_size,\n                    ..Default::default()\n                },\n            )\n            .await\n            .caused_by(trc::location!())?\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Script blob not found\")\n                    .code(ResponseCode::NonExistent)\n            })?;\n        debug_assert_eq!(script.len(), blob_size);\n\n        let mut response = Vec::with_capacity(script.len() + 32);\n        response.push(b'{');\n        response.extend_from_slice(blob_size.to_string().as_bytes());\n        response.extend_from_slice(b\"}\\r\\n\");\n        response.extend(script);\n        response.extend_from_slice(b\"\\r\\n\");\n\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::GetScript),\n            SpanId = self.session_id,\n            Id = name,\n            DocumentId = document_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::ok(\"\").serialize(response))\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/havespace.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::receiver::Request;\nuse trc::AddContext;\n\nuse crate::core::{Command, ResponseCode, Session, StatusResponse};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_havespace(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SieveHaveSpace)?;\n\n        let op_start = Instant::now();\n        let mut tokens = request.tokens.into_iter();\n        let name = tokens\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected script name as a parameter.\")\n            })?;\n        let size: usize = tokens\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected script size as a parameter.\")\n            })?\n            .parse::<usize>()\n            .map_err(|_| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Invalid size parameter.\")\n            })?;\n\n        // Validate name\n        let access_token = self.state.access_token();\n        let account_id = access_token.primary_id();\n        self.validate_name(account_id, &name).await?;\n\n        // Validate quota\n        if access_token.quota == 0\n            || size as i64\n                + self\n                    .server\n                    .get_used_quota(account_id)\n                    .await\n                    .caused_by(trc::location!())?\n                <= access_token.quota as i64\n        {\n            trc::event!(\n                ManageSieve(trc::ManageSieveEvent::HaveSpace),\n                SpanId = self.session_id,\n                Size = size,\n                Elapsed = op_start.elapsed()\n            );\n\n            Ok(StatusResponse::ok(\"\").into_bytes())\n        } else {\n            Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(\"Quota exceeded.\")\n                .code(ResponseCode::QuotaMaxSize))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/listscripts.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::{Session, StatusResponse};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::sieve::{SieveScript, ingest::SieveScriptIngest};\nuse std::time::Instant;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::SieveField};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_listscripts(&mut self) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SieveListScripts)?;\n\n        let op_start = Instant::now();\n        let account_id = self.state.access_token().primary_id();\n        let document_ids = self\n            .server\n            .document_ids(account_id, Collection::SieveScript, SieveField::Name)\n            .await\n            .caused_by(trc::location!())?;\n\n        if document_ids.is_empty() {\n            return Ok(StatusResponse::ok(\"\").into_bytes());\n        }\n\n        let mut response = Vec::with_capacity(128);\n        let count = document_ids.len();\n        let active_script_id = self.server.sieve_script_get_active_id(account_id).await?;\n\n        for document_id in document_ids {\n            if let Some(script_) = self\n                .server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::SieveScript,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                let script = script_\n                    .unarchive::<SieveScript>()\n                    .caused_by(trc::location!())?;\n                response.push(b'\\\"');\n                for ch in script.name.as_bytes() {\n                    if [b'\\\\', b'\\\"'].contains(ch) {\n                        response.push(b'\\\\');\n                    }\n                    response.push(*ch);\n                }\n                if active_script_id == Some(document_id) {\n                    response.extend_from_slice(b\"\\\" ACTIVE\\r\\n\");\n                } else {\n                    response.extend_from_slice(b\"\\\"\\r\\n\");\n                }\n            }\n        }\n\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::ListScripts),\n            SpanId = self.session_id,\n            Total = count,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::ok(\"\").serialize(response))\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/logout.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse tokio::io::{AsyncRead, AsyncWrite};\n\nuse crate::core::{Session, StatusResponse};\n\nimpl<T: AsyncRead + AsyncWrite> Session<T> {\n    pub async fn handle_logout(&mut self) -> trc::Result<Vec<u8>> {\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::Logout),\n            SpanId = self.session_id,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        Ok(StatusResponse::ok(\"Stalwart ManageSieve bids you farewell.\").into_bytes())\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::{Session, State, StatusResponse};\nuse common::listener::SessionStream;\nuse directory::Permission;\n\npub mod authenticate;\npub mod capability;\npub mod checkscript;\npub mod deletescript;\npub mod getscript;\npub mod havespace;\npub mod listscripts;\npub mod logout;\npub mod noop;\npub mod putscript;\npub mod renamescript;\npub mod setactive;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_start_tls(&self) -> trc::Result<Vec<u8>> {\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::StartTls),\n            SpanId = self.session_id,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        Ok(StatusResponse::ok(\"Begin TLS negotiation now\").into_bytes())\n    }\n\n    pub fn assert_has_permission(&self, permission: Permission) -> trc::Result<bool> {\n        match &self.state {\n            State::Authenticated { access_token, .. } => {\n                access_token.assert_has_permission(permission)\n            }\n            State::NotAuthenticated { .. } => Ok(false),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/noop.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap_proto::receiver::Request;\nuse tokio::io::{AsyncRead, AsyncWrite};\n\nuse crate::core::{Command, ResponseCode, Session, StatusResponse};\n\nimpl<T: AsyncRead + AsyncWrite> Session<T> {\n    pub async fn handle_noop(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::Noop),\n            SpanId = self.session_id,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        Ok(if let Some(tag) = request\n            .tokens\n            .into_iter()\n            .next()\n            .and_then(|t| t.unwrap_string().ok())\n        {\n            StatusResponse::ok(\"Done\").with_code(ResponseCode::Tag(tag))\n        } else {\n            StatusResponse::ok(\"Done\")\n        }\n        .into_bytes())\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/putscript.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::{Command, ResponseCode, Session, StatusResponse};\nuse common::{listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse email::sieve::SieveScript;\nuse imap_proto::receiver::Request;\nuse sieve::compiler::ErrorType;\nuse std::time::Instant;\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::SieveField};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_putscript(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SievePutScript)?;\n\n        let op_start = Instant::now();\n        let mut tokens = request.tokens.into_iter();\n        let name = tokens\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected script name as a parameter.\")\n            })?\n            .trim()\n            .to_string();\n        let mut script_bytes = tokens\n            .next()\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected script as a parameter.\")\n            })?\n            .unwrap_bytes();\n        let script_size = script_bytes.len() as i64;\n\n        // Check quota\n        let access_token = self.state.access_token();\n        let account_id = access_token.primary_id();\n        self.server\n            .has_available_quota(&access_token.as_resource_token(), script_bytes.len() as u64)\n            .await\n            .caused_by(trc::location!())?;\n\n        if self\n            .server\n            .document_ids(account_id, Collection::SieveScript, SieveField::Name)\n            .await\n            .caused_by(trc::location!())?\n            .len()\n            > access_token.object_quota(Collection::SieveScript) as u64\n        {\n            return Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(\"Too many scripts.\")\n                .code(ResponseCode::QuotaMaxScripts));\n        }\n\n        // Compile script\n        match self\n            .server\n            .core\n            .sieve\n            .untrusted_compiler\n            .compile(&script_bytes)\n        {\n            Ok(compiled_script) => {\n                script_bytes.extend(\n                    Archiver::new(compiled_script)\n                        .untrusted()\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n            }\n            Err(err) => {\n                return Err(if let ErrorType::ScriptTooLong = &err.error_type() {\n                    trc::ManageSieveEvent::Error\n                        .into_err()\n                        .details(err.to_string())\n                        .code(ResponseCode::QuotaMaxSize)\n                } else {\n                    trc::ManageSieveEvent::Error\n                        .into_err()\n                        .details(err.to_string())\n                });\n            }\n        }\n\n        // Validate name\n        if let Some(document_id) = self.validate_name(account_id, &name).await? {\n            // Obtain script values\n            let script_ = self\n                .server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::SieveScript,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| {\n                    trc::ManageSieveEvent::Error\n                        .into_err()\n                        .details(\"Script not found\")\n                        .code(ResponseCode::NonExistent)\n                })?;\n            let script = script_\n                .to_unarchived::<SieveScript>()\n                .caused_by(trc::location!())?;\n\n            // Write script blob\n            let (blob_hash, blob_hold) = self\n                .server\n                .put_temporary_blob(account_id, &script_bytes, 60)\n                .await?;\n\n            // Write record\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::SieveScript)\n                .with_document(document_id)\n                .custom(\n                    ObjectIndexBuilder::new()\n                        .with_changes(\n                            script\n                                .deserialize()\n                                .caused_by(trc::location!())?\n                                .with_size(script_size as u32)\n                                .with_blob_hash(blob_hash.clone()),\n                        )\n                        .with_current(script)\n                        .with_access_token(access_token),\n                )\n                .caused_by(trc::location!())?\n                .clear(blob_hold);\n\n            self.server\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?;\n\n            trc::event!(\n                ManageSieve(trc::ManageSieveEvent::UpdateScript),\n                SpanId = self.session_id,\n                Id = name.to_string(),\n                DocumentId = document_id,\n                Size = script_size,\n                Elapsed = op_start.elapsed(),\n            );\n        } else {\n            // Write script blob\n            let (blob_hash, blob_hold) = self\n                .server\n                .put_temporary_blob(account_id, &script_bytes, 60)\n                .await?;\n\n            // Write record\n            let mut batch = BatchBuilder::new();\n            let document_id = self\n                .server\n                .store()\n                .assign_document_ids(account_id, Collection::SieveScript, 1)\n                .await\n                .caused_by(trc::location!())?;\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::SieveScript)\n                .with_document(document_id)\n                .custom(\n                    ObjectIndexBuilder::<(), _>::new()\n                        .with_changes(\n                            SieveScript::new(name.clone(), blob_hash.clone())\n                                .with_size(script_size as u32),\n                        )\n                        .with_access_token(access_token),\n                )\n                .caused_by(trc::location!())?\n                .clear(blob_hold);\n\n            self.server\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?;\n\n            trc::event!(\n                ManageSieve(trc::ManageSieveEvent::CreateScript),\n                SpanId = self.session_id,\n                Id = name,\n                DocumentId = document_id,\n                Elapsed = op_start.elapsed()\n            );\n        }\n\n        Ok(StatusResponse::ok(\"Success.\").into_bytes())\n    }\n\n    pub async fn validate_name(&self, account_id: u32, name: &str) -> trc::Result<Option<u32>> {\n        if name.is_empty() {\n            Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(\"Script name cannot be empty.\"))\n        } else if name.len() > self.server.core.jmap.sieve_max_script_name {\n            Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(\"Script name is too long.\"))\n        } else if name.eq_ignore_ascii_case(\"vacation\") {\n            Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(\"The 'vacation' name is reserved, please use a different name.\"))\n        } else {\n            Ok(self\n                .server\n                .document_ids_matching(\n                    account_id,\n                    Collection::SieveScript,\n                    SieveField::Name,\n                    name.to_lowercase().as_bytes(),\n                )\n                .await\n                .caused_by(trc::location!())?\n                .min())\n        }\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/renamescript.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::{Command, ResponseCode, Session, StatusResponse};\nuse common::{listener::SessionStream, storage::index::ObjectIndexBuilder};\nuse directory::Permission;\nuse email::sieve::SieveScript;\nuse imap_proto::receiver::Request;\nuse std::time::Instant;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::collection::Collection;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_renamescript(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SieveRenameScript)?;\n\n        let op_start = Instant::now();\n        let mut tokens = request.tokens.into_iter();\n        let name = tokens\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected old script name as a parameter.\")\n            })?\n            .trim()\n            .to_string();\n        let new_name = tokens\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected new script name as a parameter.\")\n            })?\n            .trim()\n            .to_string();\n\n        // Validate name\n        if name == new_name {\n            return Ok(StatusResponse::ok(\"Old and new script names are the same.\").into_bytes());\n        }\n        let account_id = self.state.access_token().primary_id();\n        let document_id = self.get_script_id(account_id, &name).await?;\n        if self.validate_name(account_id, &new_name).await?.is_some() {\n            return Err(trc::ManageSieveEvent::Error\n                .into_err()\n                .details(format!(\"A sieve script with name '{name}' already exists.\",))\n                .code(ResponseCode::AlreadyExists));\n        }\n\n        // Obtain script values\n        let script = self\n            .server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::SieveScript,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Script not found\")\n                    .code(ResponseCode::NonExistent)\n            })?\n            .into_deserialized::<SieveScript>()\n            .caused_by(trc::location!())?;\n\n        // Write record\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::SieveScript)\n            .with_document(document_id)\n            .custom(\n                ObjectIndexBuilder::new()\n                    .with_changes(script.inner.clone().with_name(new_name.clone()))\n                    .with_current(script),\n            )\n            .caused_by(trc::location!())?;\n        if !batch.is_empty() {\n            self.server\n                .commit_batch(batch)\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::RenameScript),\n            SpanId = self.session_id,\n            Id = new_name,\n            DocumentId = document_id,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::ok(\"Success.\").into_bytes())\n    }\n}\n"
  },
  {
    "path": "crates/managesieve/src/op/setactive.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse imap_proto::receiver::Request;\nuse store::{SerializeInfallible, write::BatchBuilder};\nuse trc::AddContext;\nuse types::{collection::Collection, field::PrincipalField};\n\nuse crate::core::{Command, Session, StatusResponse};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_setactive(&mut self, request: Request<Command>) -> trc::Result<Vec<u8>> {\n        // Validate access\n        self.assert_has_permission(Permission::SieveSetActive)?;\n\n        let op_start = Instant::now();\n        let name = request\n            .tokens\n            .into_iter()\n            .next()\n            .and_then(|s| s.unwrap_string().ok())\n            .ok_or_else(|| {\n                trc::ManageSieveEvent::Error\n                    .into_err()\n                    .details(\"Expected script name as a parameter.\")\n            })?;\n\n        // De/activate script\n        let account_id = self.state.access_token().primary_id();\n        let mut batch = BatchBuilder::new();\n        if !name.is_empty() {\n            let document_id = self.get_script_id(account_id, &name).await?;\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .set(PrincipalField::ActiveScriptId, document_id.serialize());\n        } else {\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .clear(PrincipalField::ActiveScriptId);\n        }\n        self.server\n            .commit_batch(batch)\n            .await\n            .caused_by(trc::location!())?;\n\n        trc::event!(\n            ManageSieve(trc::ManageSieveEvent::SetActive),\n            SpanId = self.session_id,\n            Id = name,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(StatusResponse::ok(\"Success\").into_bytes())\n    }\n}\n"
  },
  {
    "path": "crates/migration/Cargo.toml",
    "content": "[package]\nname = \"migration\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path = \"../utils\" }\nnlp = { path = \"../nlp\" }\nstore = { path = \"../store\" }\ntrc = { path = \"../trc\" }\ntypes = { path = \"../types\" }\ncommon = { path =  \"../common\" }\nemail = { path =  \"../email\" }\ndirectory = { path =  \"../directory\" }\nsmtp = { path =  \"../smtp\" }\ngroupware = { path =  \"../groupware\" }\ndav-proto = { path =  \"../dav-proto\" }\nproc_macros = { path =  \"../utils/proc-macros\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-auth = { version = \"0.7.1\", features = [\"rkyv\"] }\nsmtp-proto = { version = \"0.2\", features = [\"rkyv\", \"serde\"] }\nsieve-rs = { version = \"0.7\", features = [\"rkyv\"] } \ncalcard_latest = { package = \"calcard\", version = \"0.3\", features = [\"rkyv\"] }\ncalcard_v01 = { package = \"calcard\", version = \"0.1\", features = [\"rkyv\"] }\ntokio = { version = \"1.47\", features = [\"net\", \"macros\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\nbincode = \"1.3.3\" \nlz4_flex = { version = \"0.12\", default-features = false } \nbase64 = \"0.22\"\nfutures = \"0.3\"\nnum_cpus = \"1.13.1\"\n\n[features]\ntest_mode = []\nenterprise = []\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/migration/src/addressbook_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse groupware::contact::{AddressBook, AddressBookPreferences};\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, serialize::rkyv_deserialize},\n};\nuse trc::AddContext;\nuse types::{acl::AclGrant, collection::Collection, dead_property::DeadProperty, field::Field};\n\nuse crate::get_document_ids;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct AddressBookV2 {\n    pub name: String,\n    pub display_name: Option<String>,\n    pub description: Option<String>,\n    pub sort_order: u32,\n    pub is_default: bool,\n    pub subscribers: Vec<u32>,\n    pub dead_properties: DeadProperty,\n    pub acls: Vec<AclGrant>,\n    pub created: i64,\n    pub modified: i64,\n}\n\npub(crate) async fn migrate_addressbook_v013(server: &Server, account_id: u32) -> trc::Result<u64> {\n    let document_ids = get_document_ids(server, account_id, Collection::AddressBook)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    if document_ids.is_empty() {\n        return Ok(0);\n    }\n    let mut num_migrated = 0;\n\n    for document_id in document_ids.iter() {\n        let Some(archive) = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::AddressBook,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n        else {\n            continue;\n        };\n\n        match archive.unarchive_untrusted::<AddressBookV2>() {\n            Ok(book) => {\n                let book = rkyv_deserialize::<_, AddressBookV2>(book).unwrap();\n                let new_book = AddressBook {\n                    name: book.name,\n                    preferences: vec![AddressBookPreferences {\n                        account_id,\n                        name: book\n                            .display_name\n                            .unwrap_or_else(|| \"Address Book\".to_string()),\n                        description: book.description,\n                        sort_order: book.sort_order,\n                    }],\n                    subscribers: book.subscribers,\n                    dead_properties: book.dead_properties,\n                    acls: book.acls,\n                    created: book.created,\n                    modified: book.modified,\n                };\n\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::AddressBook)\n                    .with_document(document_id)\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(new_book)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                num_migrated += 1;\n            }\n            Err(err) => {\n                if let Err(err_) = archive.unarchive_untrusted::<AddressBook>() {\n                    trc::error!(err_.caused_by(trc::location!()));\n                    return Err(err.caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    Ok(num_migrated)\n}\n"
  },
  {
    "path": "crates/migration/src/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse store::{\n    IterateParams, SUBSPACE_BLOB_LINK, Serialize, SerializeInfallible, U32_LEN, U64_LEN, ValueKey,\n    write::{\n        AnyClass, Archiver, BatchBuilder, BlobLink, BlobOp, ValueClass, key::DeserializeBigEndian,\n        now,\n    },\n};\nuse trc::AddContext;\nuse types::blob_hash::{BLOB_HASH_LEN, BlobHash};\n\nconst SUBSPACE_BLOB_RESERVE: u8 = b'j';\n\npub(crate) async fn migrate_blobs_v014(server: &Server) -> trc::Result<()> {\n    let mut num_blobs = 0;\n    for byte in 0..=u8::MAX {\n        // Validate linked blobs\n        let mut from_hash = BlobHash::default();\n        let mut to_hash = BlobHash::new_max();\n        from_hash.0[0] = byte;\n        to_hash.0[0] = byte;\n        let from_key = ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Blob(BlobOp::Commit { hash: from_hash }),\n        };\n        let to_key = ValueKey {\n            account_id: u32::MAX,\n            collection: u8::MAX,\n            document_id: u32::MAX,\n            class: ValueClass::Blob(BlobOp::Link {\n                hash: to_hash,\n                to: BlobLink::Document,\n            }),\n        };\n\n        let mut keys = Vec::new();\n        server\n            .store()\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending().no_values(),\n                |key, value| {\n                    if key.len() == BLOB_HASH_LEN + U64_LEN + 1 {\n                        let hash =\n                            BlobHash::try_from_hash_slice(key.get(0..BLOB_HASH_LEN).ok_or_else(\n                                || trc::Error::corrupted_key(key, value.into(), trc::location!()),\n                            )?)\n                            .unwrap();\n                        let account_id = key.deserialize_be_u32(BLOB_HASH_LEN)?;\n                        let document_id = key.deserialize_be_u32(BLOB_HASH_LEN + U32_LEN + 1)?;\n                        let collection = key[BLOB_HASH_LEN + U32_LEN];\n\n                        if account_id == u32::MAX && document_id == u32::MAX && collection == 0 {\n                            keys.push((key.to_vec(), BlobOp::Commit { hash }));\n                        } else if collection == u8::MAX {\n                            keys.push((\n                                key.to_vec(),\n                                BlobOp::Link {\n                                    hash,\n                                    to: BlobLink::Id {\n                                        id: ((account_id as u64) << 32) | document_id as u64,\n                                    },\n                                },\n                            ));\n                        }\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        let mut batch = BatchBuilder::new();\n        num_blobs += keys.len();\n        for (key, op) in keys {\n            batch\n                .clear(ValueClass::Any(AnyClass {\n                    subspace: SUBSPACE_BLOB_LINK,\n                    key,\n                }))\n                .set(op, vec![]);\n\n            if batch.is_large_batch() {\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                batch = BatchBuilder::new();\n            }\n        }\n        if !batch.is_empty() {\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n    }\n\n    trc::event!(\n        Server(trc::ServerEvent::Startup),\n        Details = format!(\"Migrated {num_blobs} blob links\")\n    );\n\n    enum OldType {\n        Quota { size: u32 },\n        Undelete { deleted_at: u64, size: u32 },\n        Temp,\n        None,\n    }\n\n    struct OldBlobEntry {\n        account_id: u32,\n        until: u64,\n        hash: BlobHash,\n        blob_type: OldType,\n        old_key: Vec<u8>,\n    }\n\n    let mut entries = Vec::new();\n    let now = now();\n    server\n        .store()\n        .iterate(\n            IterateParams::new(\n                ValueKey::from(ValueClass::Any(AnyClass {\n                    subspace: SUBSPACE_BLOB_RESERVE,\n                    key: vec![0u8],\n                })),\n                ValueKey::from(ValueClass::Any(AnyClass {\n                    subspace: SUBSPACE_BLOB_RESERVE,\n                    key: vec![u8::MAX; 32],\n                })),\n            )\n            .ascending(),\n            |key, value| {\n                if key.len() == BLOB_HASH_LEN + U64_LEN + U32_LEN {\n                    let account_id = key.deserialize_be_u32(0)?;\n                    let hash = BlobHash::try_from_hash_slice(\n                        key.get(U32_LEN..BLOB_HASH_LEN + U32_LEN).ok_or_else(|| {\n                            trc::Error::corrupted_key(key, value.into(), trc::location!())\n                        })?,\n                    )\n                    .unwrap();\n                    let until = key.deserialize_be_u64(BLOB_HASH_LEN + U32_LEN)?;\n\n                    let blob_type = if until > now {\n                        if value.len() == U32_LEN {\n                            let size = value.deserialize_be_u32(0)?;\n                            if size != 0 {\n                                OldType::Quota { size }\n                            } else {\n                                OldType::Temp\n                            }\n                        } else if value.len() == U64_LEN + U32_LEN + 1 {\n                            let size = value.deserialize_be_u32(0)?;\n                            let deleted_at = value.deserialize_be_u64(U32_LEN)?;\n                            OldType::Undelete { deleted_at, size }\n                        } else {\n                            OldType::Temp\n                        }\n                    } else {\n                        OldType::None\n                    };\n\n                    entries.push(OldBlobEntry {\n                        account_id,\n                        until,\n                        hash,\n                        blob_type,\n                        old_key: key.to_vec(),\n                    });\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    let mut batch = BatchBuilder::new();\n    let num_entries = entries.len();\n    for entry in entries {\n        batch\n            .clear(ValueClass::Any(AnyClass {\n                subspace: SUBSPACE_BLOB_RESERVE,\n                key: entry.old_key,\n            }))\n            .with_account_id(entry.account_id);\n\n        match entry.blob_type {\n            OldType::Quota { size } => {\n                batch\n                    .set(\n                        BlobOp::Link {\n                            hash: entry.hash.clone(),\n                            to: BlobLink::Temporary { until: entry.until },\n                        },\n                        vec![BlobLink::QUOTA_LINK],\n                    )\n                    .set(\n                        BlobOp::Quota {\n                            hash: entry.hash,\n                            until: entry.until,\n                        },\n                        size.serialize(),\n                    );\n            }\n            OldType::Undelete { deleted_at, size } => {\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n\n                #[cfg(feature = \"enterprise\")]\n                {\n                    batch\n                        .set(\n                            BlobOp::Link {\n                                hash: entry.hash.clone(),\n                                to: BlobLink::Temporary { until: entry.until },\n                            },\n                            vec![BlobLink::UNDELETE_LINK],\n                        )\n                        .set(\n                            BlobOp::Undelete {\n                                hash: entry.hash,\n                                until: entry.until,\n                            },\n                            Archiver::new(common::enterprise::undelete::DeletedItem {\n                                typ: common::enterprise::undelete::DeletedItemType::Email {\n                                    from: \"unknown\".into(),\n                                    subject: \"unknown\".into(),\n                                    received_at: deleted_at,\n                                },\n                                size,\n                                deleted_at,\n                            })\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                        );\n                }\n\n                // SPDX-SnippetEnd\n            }\n            OldType::Temp => {\n                batch.set(\n                    BlobOp::Link {\n                        hash: entry.hash,\n                        to: BlobLink::Temporary { until: entry.until },\n                    },\n                    vec![],\n                );\n            }\n            OldType::None => (),\n        }\n\n        if batch.is_large_batch() {\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n            batch = BatchBuilder::new();\n        }\n    }\n\n    trc::event!(\n        Server(trc::ServerEvent::Startup),\n        Details = format!(\"Migrated {num_entries} temporary blob links\")\n    );\n\n    if !batch.is_empty() {\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/migration/src/calendar_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse groupware::calendar::{Calendar, CalendarPreferences, Timezone};\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, serialize::rkyv_deserialize},\n};\nuse trc::AddContext;\nuse types::{acl::AclGrant, collection::Collection, dead_property::DeadProperty, field::Field};\n\nuse crate::{event_v2::migrate_icalendar_v02, get_document_ids};\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarV2 {\n    pub name: String,\n    pub preferences: Vec<CalendarPreferencesV2>,\n    pub default_alerts: Vec<DefaultAlertV2>,\n    pub acls: Vec<AclGrant>,\n    pub dead_properties: DeadProperty,\n    pub created: i64,\n    pub modified: i64,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarPreferencesV2 {\n    pub account_id: u32,\n    pub name: String,\n    pub description: Option<String>,\n    pub sort_order: u32,\n    pub color: Option<String>,\n    pub flags: u16,\n    pub time_zone: TimezoneV2,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub enum TimezoneV2 {\n    IANA(u16),\n    Custom(calcard_v01::icalendar::ICalendar),\n    #[default]\n    Default,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct DefaultAlertV2 {\n    pub account_id: u32,\n    pub id: String,\n    pub alert: calcard_v01::icalendar::ICalendar,\n    pub with_time: bool,\n}\n\npub(crate) async fn migrate_calendar_v013(server: &Server, account_id: u32) -> trc::Result<u64> {\n    let document_ids = get_document_ids(server, account_id, Collection::Calendar)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    if document_ids.is_empty() {\n        return Ok(0);\n    }\n    let mut num_migrated = 0;\n\n    for document_id in document_ids.iter() {\n        let Some(archive) = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::Calendar,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n        else {\n            continue;\n        };\n\n        match archive.unarchive_untrusted::<CalendarV2>() {\n            Ok(calendar) => {\n                let calendar = rkyv_deserialize::<_, CalendarV2>(calendar).unwrap();\n                let new_calendar = Calendar {\n                    name: calendar.name,\n                    preferences: calendar\n                        .preferences\n                        .into_iter()\n                        .map(|pref| CalendarPreferences {\n                            account_id: pref.account_id,\n                            name: pref.name,\n                            description: pref.description,\n                            sort_order: pref.sort_order,\n                            color: pref.color,\n                            flags: 0,\n                            time_zone: match pref.time_zone {\n                                TimezoneV2::IANA(tzid) => Timezone::IANA(tzid),\n                                TimezoneV2::Custom(tz) => {\n                                    Timezone::Custom(migrate_icalendar_v02(tz))\n                                }\n                                TimezoneV2::Default => Timezone::Default,\n                            },\n                            default_alerts: Vec::new(),\n                        })\n                        .collect(),\n                    acls: calendar.acls,\n                    supported_components: 0,\n                    dead_properties: calendar.dead_properties,\n                    created: calendar.created,\n                    modified: calendar.modified,\n                };\n\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Calendar)\n                    .with_document(document_id)\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(new_calendar)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                num_migrated += 1;\n            }\n            Err(err) => {\n                if let Err(err_) = archive.unarchive_untrusted::<Calendar>() {\n                    trc::error!(err_.caused_by(trc::location!()));\n                    return Err(err.caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    Ok(num_migrated)\n}\n"
  },
  {
    "path": "crates/migration/src/changelog.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse store::{\n    SUBSPACE_LOGS, U64_LEN,\n    write::{AnyKey, key::KeySerializer},\n};\nuse trc::AddContext;\n\npub(crate) async fn reset_changelog(server: &Server) -> trc::Result<()> {\n    // Delete changes\n    server\n        .store()\n        .delete_range(\n            AnyKey {\n                subspace: SUBSPACE_LOGS,\n                key: KeySerializer::new(U64_LEN).write(0u8).finalize(),\n            },\n            AnyKey {\n                subspace: SUBSPACE_LOGS,\n                key: KeySerializer::new(U64_LEN)\n                    .write(&[u8::MAX; 16][..])\n                    .finalize(),\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n}\n"
  },
  {
    "path": "crates/migration/src/contact_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{DavName, Server};\nuse groupware::contact::ContactCard;\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, serialize::rkyv_deserialize},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, dead_property::DeadProperty, field::Field};\n\nuse crate::get_document_ids;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct ContactCardV2 {\n    pub names: Vec<DavName>,\n    pub display_name: Option<String>,\n    pub card: calcard_v01::vcard::VCard,\n    pub dead_properties: DeadProperty,\n    pub created: i64,\n    pub modified: i64,\n    pub size: u32,\n}\n\npub(crate) async fn migrate_contacts_v013(server: &Server, account_id: u32) -> trc::Result<u64> {\n    let document_ids = get_document_ids(server, account_id, Collection::ContactCard)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n\n    let mut num_migrated = 0;\n\n    for document_id in document_ids.iter() {\n        let Some(archive) = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::ContactCard,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n        else {\n            continue;\n        };\n\n        match archive.unarchive_untrusted::<ContactCardV2>() {\n            Ok(contact) => {\n                let contact = rkyv_deserialize::<_, ContactCardV2>(contact).unwrap();\n                let new_contact = ContactCard {\n                    names: contact.names,\n                    display_name: contact.display_name,\n                    dead_properties: contact.dead_properties,\n                    size: contact.size,\n                    created: contact.created,\n                    modified: contact.modified,\n                    card: calcard_latest::vcard::VCard::parse(contact.card.to_string())\n                        .unwrap_or_default(),\n                };\n\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::ContactCard)\n                    .with_document(document_id)\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(new_contact)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                num_migrated += 1;\n            }\n            Err(err) => {\n                if let Err(err_) = archive.unarchive_untrusted::<ContactCard>() {\n                    trc::error!(err_.caused_by(trc::location!()));\n                    return Err(err.caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    Ok(num_migrated)\n}\n"
  },
  {
    "path": "crates/migration/src/email_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{LegacyBincode, get_properties};\nuse crate::{email_v2::LegacyKeyword, get_bitmap, get_document_ids, v014::SUBSPACE_BITMAP_TAG};\nuse common::Server;\nuse email::{\n    mailbox::*,\n    message::{\n        index::extractors::VisitTextArchived,\n        ingest::ThreadInfo,\n        metadata::{\n            MESSAGE_HAS_ATTACHMENT, MESSAGE_RECEIVED_MASK, MessageDataBuilder, MessageMetadata,\n            MessageMetadataContents, MessageMetadataPart, MetadataHeader, MetadataHeaderName,\n            MetadataHeaderValue, MetadataPartType, PART_ENCODING_BASE64, PART_ENCODING_PROBLEM,\n            PART_ENCODING_QP, PART_SIZE_MASK,\n        },\n    },\n};\nuse mail_parser::{\n    Address, Attribute, ContentType, DateTime, Encoding, HeaderName, HeaderValue, Received,\n    parsers::fields::thread::thread_name,\n};\nuse std::{borrow::Cow, collections::VecDeque};\nuse store::{\n    Deserialize, SUBSPACE_INDEXES, SUBSPACE_PROPERTY, Serialize, SerializeInfallible, U32_LEN,\n    U64_LEN, ValueKey,\n    ahash::AHashMap,\n    write::{\n        AlignedBytes, AnyKey, Archive, Archiver, BatchBuilder, IndexPropertyClass, ValueClass,\n        key::KeySerializer,\n    },\n};\nuse trc::AddContext;\nuse types::{\n    blob_hash::BlobHash,\n    collection::Collection,\n    field::{EmailField, Field},\n    keyword::*,\n};\nuse utils::{cheeky_hash::CheekyHash, codec::leb128::Leb128Iterator};\n\nconst FIELD_KEYWORDS: u8 = 4;\nconst FIELD_THREAD_ID: u8 = 33;\nconst FIELD_CID: u8 = 76;\npub(crate) const FIELD_MAILBOX_IDS: u8 = 7;\n\nconst BM_MARKER: u8 = 1 << 7;\n\npub(crate) async fn migrate_emails_v011(server: &Server, account_id: u32) -> trc::Result<u64> {\n    // Obtain email ids\n    let mut document_ids = get_document_ids(server, account_id, Collection::Email)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_emails = document_ids.len();\n    if num_emails == 0 {\n        return Ok(0);\n    }\n    let tombstoned_ids = get_bitmap(\n        server,\n        AnyKey {\n            subspace: SUBSPACE_BITMAP_TAG,\n            key: KeySerializer::new(U64_LEN + U32_LEN + 1)\n                .write(account_id)\n                .write(u8::from(Collection::Email))\n                .write(FIELD_MAILBOX_IDS)\n                .write_leb128(u32::MAX - 1)\n                .finalize(),\n        },\n        AnyKey {\n            subspace: SUBSPACE_BITMAP_TAG,\n            key: KeySerializer::new(U64_LEN + U32_LEN + 1)\n                .write(account_id)\n                .write(u8::from(Collection::Email))\n                .write(FIELD_MAILBOX_IDS)\n                .write_leb128(u32::MAX - 1)\n                .finalize(),\n        },\n    )\n    .await\n    .caused_by(trc::location!())?\n    .unwrap_or_default();\n\n    let mut message_data: AHashMap<u32, MessageDataBuilder> =\n        AHashMap::with_capacity(num_emails as usize);\n    let mut did_migrate = false;\n\n    // Obtain mailboxes\n    for (message_id, uid_mailbox) in get_properties::<Mailboxes, _>(\n        server,\n        account_id,\n        Collection::Email,\n        &(),\n        FIELD_MAILBOX_IDS,\n    )\n    .await\n    .caused_by(trc::location!())?\n    {\n        message_data.entry(message_id).or_default().mailboxes = uid_mailbox.0;\n    }\n\n    // Obtain keywords\n    for (message_id, keywords) in\n        get_properties::<Keywords, _>(server, account_id, Collection::Email, &(), FIELD_KEYWORDS)\n            .await\n            .caused_by(trc::location!())?\n    {\n        message_data.entry(message_id).or_default().keywords =\n            keywords.0.into_iter().map(Into::into).collect();\n    }\n\n    // Obtain threadIds\n    for (message_id, thread_id) in\n        get_properties::<u32, _>(server, account_id, Collection::Email, &(), FIELD_THREAD_ID)\n            .await\n            .caused_by(trc::location!())?\n    {\n        message_data.entry(message_id).or_default().thread_id = thread_id;\n    }\n\n    // Write message data\n    for (message_id, mut data) in message_data {\n        if !tombstoned_ids.contains(message_id) {\n            let (size, metadata) = match server\n                .store()\n                .get_value::<LegacyBincode<LegacyMessageMetadata>>(ValueKey {\n                    account_id,\n                    collection: Collection::Email.into(),\n                    document_id: message_id,\n                    class: ValueClass::Property(EmailField::Metadata.into()),\n                })\n                .await\n            {\n                Ok(Some(legacy_metadata)) => (\n                    legacy_metadata.inner.size as u32,\n                    MessageMetadata::from_legacy(legacy_metadata.inner),\n                ),\n                Ok(None) => {\n                    continue;\n                }\n                Err(err) => {\n                    match server\n                        .store()\n                        .get_value::<Archive<AlignedBytes>>(ValueKey {\n                            account_id,\n                            collection: Collection::Email.into(),\n                            document_id: message_id,\n                            class: ValueClass::Property(EmailField::Metadata.into()),\n                        })\n                        .await\n                    {\n                        Ok(Some(archive)) => {\n                            let metadata: MessageMetadata = archive\n                                .deserialize_untrusted()\n                                .caused_by(trc::location!())?;\n                            (metadata.root_part().offset_end, metadata)\n                        }\n                        _ => {\n                            return Err(err\n                                .account_id(account_id)\n                                .document_id(message_id)\n                                .caused_by(trc::location!()));\n                        }\n                    }\n                }\n            };\n\n            did_migrate = true;\n            document_ids.insert(message_id);\n\n            let mut message_ids = Vec::new();\n            let mut subject = \"\";\n            for header in &metadata.contents[0].parts[0].headers {\n                match &header.name {\n                    MetadataHeaderName::MessageId => {\n                        header.value.visit_text(|id| {\n                            if !id.is_empty() {\n                                message_ids.push(CheekyHash::new(id.as_bytes()));\n                            }\n                        });\n                    }\n                    MetadataHeaderName::InReplyTo\n                    | MetadataHeaderName::References\n                    | MetadataHeaderName::ResentMessageId => {\n                        header.value.visit_text(|id| {\n                            if !id.is_empty() {\n                                message_ids.push(CheekyHash::new(id.as_bytes()));\n                            }\n                        });\n                    }\n                    MetadataHeaderName::Subject if subject.is_empty() => {\n                        subject = thread_name(match &header.value {\n                            MetadataHeaderValue::Text(text) => text.as_ref(),\n                            MetadataHeaderValue::TextList(list) if !list.is_empty() => {\n                                list.first().unwrap().as_ref()\n                            }\n                            _ => \"\",\n                        });\n                    }\n                    _ => (),\n                }\n            }\n\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Email)\n                .with_document(message_id);\n\n            if data\n                .mailboxes\n                .iter()\n                .any(|mailbox| mailbox.mailbox_id == TRASH_ID || mailbox.mailbox_id == JUNK_ID)\n            {\n                batch.set(\n                    ValueClass::Property(EmailField::DeletedAt.into()),\n                    (metadata.rcvd_attach & MESSAGE_RECEIVED_MASK).serialize(),\n                );\n            }\n            data.size = size;\n            batch\n                .set(\n                    ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                        property: EmailField::Threading.into(),\n                        hash: CheekyHash::new(if !subject.is_empty() { subject } else { \"!\" }),\n                    }),\n                    ThreadInfo::serialize(data.thread_id, &message_ids),\n                )\n                .set(\n                    Field::ARCHIVE,\n                    Archiver::new(data.seal())\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                )\n                .set(\n                    EmailField::Metadata,\n                    Archiver::new(metadata)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n    }\n\n    // Delete keyword bitmaps\n    for field in [FIELD_KEYWORDS, FIELD_KEYWORDS | BM_MARKER] {\n        server\n            .store()\n            .delete_range(\n                AnyKey {\n                    subspace: SUBSPACE_BITMAP_TAG,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::Email))\n                        .write(field)\n                        .finalize(),\n                },\n                AnyKey {\n                    subspace: SUBSPACE_BITMAP_TAG,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::Email))\n                        .write(field)\n                        .write(&[u8::MAX; 8][..])\n                        .finalize(),\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    // Delete messageId index, now in References\n    const MESSAGE_ID_FIELD: u8 = 11;\n    server\n        .store()\n        .delete_range(\n            AnyKey {\n                subspace: SUBSPACE_INDEXES,\n                key: KeySerializer::new(U64_LEN)\n                    .write(account_id)\n                    .write(u8::from(Collection::Email))\n                    .write(MESSAGE_ID_FIELD)\n                    .finalize(),\n            },\n            AnyKey {\n                subspace: SUBSPACE_INDEXES,\n                key: KeySerializer::new(U64_LEN)\n                    .write(account_id)\n                    .write(u8::from(Collection::Email))\n                    .write(MESSAGE_ID_FIELD)\n                    .write(&[u8::MAX; 8][..])\n                    .finalize(),\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    // Delete values\n    for property in [\n        FIELD_MAILBOX_IDS,\n        FIELD_KEYWORDS,\n        FIELD_THREAD_ID,\n        FIELD_CID,\n    ] {\n        server\n            .store()\n            .delete_range(\n                AnyKey {\n                    subspace: SUBSPACE_PROPERTY,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::Email))\n                        .write(property)\n                        .finalize(),\n                },\n                AnyKey {\n                    subspace: SUBSPACE_PROPERTY,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::Email))\n                        .write(property)\n                        .write(&[u8::MAX; 8][..])\n                        .finalize(),\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    // Increment document id counter\n    if did_migrate {\n        server\n            .store()\n            .assign_document_ids(\n                account_id,\n                Collection::Email,\n                document_ids.max().map(|id| id as u64).unwrap_or(num_emails) + 1,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        Ok(num_emails)\n    } else {\n        Ok(0)\n    }\n}\n\npub trait FromLegacy {\n    fn from_legacy(legacy: LegacyMessageMetadata<'_>) -> Self;\n}\n\nimpl FromLegacy for MessageMetadata {\n    fn from_legacy(legacy: LegacyMessageMetadata<'_>) -> Self {\n        let mut contents = Vec::new();\n        let mut messages = VecDeque::from([legacy.contents]);\n        let mut message_id = 0;\n\n        while let Some(message) = messages.pop_front() {\n            let mut parts = Vec::new();\n\n            for part in message.parts {\n                let body = match part.body {\n                    LegacyMetadataPartType::Text => MetadataPartType::Text,\n                    LegacyMetadataPartType::Html => MetadataPartType::Html,\n                    LegacyMetadataPartType::Binary => MetadataPartType::Binary,\n                    LegacyMetadataPartType::InlineBinary => MetadataPartType::InlineBinary,\n                    LegacyMetadataPartType::Message(message) => {\n                        messages.push_back(message);\n                        message_id += 1;\n                        MetadataPartType::Message(message_id)\n                    }\n                    LegacyMetadataPartType::Multipart(parts) => {\n                        MetadataPartType::Multipart(parts.into_iter().map(|p| p as u16).collect())\n                    }\n                };\n\n                let flags = match part.encoding {\n                    Encoding::None => 0,\n                    Encoding::QuotedPrintable => PART_ENCODING_QP,\n                    Encoding::Base64 => PART_ENCODING_BASE64,\n                } | (if part.is_encoding_problem {\n                    PART_ENCODING_PROBLEM\n                } else {\n                    0\n                }) | (part.size as u32 & PART_SIZE_MASK);\n\n                parts.push(MessageMetadataPart {\n                    headers: part\n                        .headers\n                        .into_iter()\n                        .map(|hdr| MetadataHeader {\n                            value: if matches!(\n                                &hdr.name,\n                                HeaderName::Subject\n                                    | HeaderName::From\n                                    | HeaderName::To\n                                    | HeaderName::Cc\n                                    | HeaderName::Date\n                                    | HeaderName::Bcc\n                                    | HeaderName::ReplyTo\n                                    | HeaderName::Sender\n                                    | HeaderName::Comments\n                                    | HeaderName::InReplyTo\n                                    | HeaderName::Keywords\n                                    | HeaderName::MessageId\n                                    | HeaderName::References\n                                    | HeaderName::ResentMessageId\n                                    | HeaderName::ContentDescription\n                                    | HeaderName::ContentId\n                                    | HeaderName::ContentLanguage\n                                    | HeaderName::ContentLocation\n                                    | HeaderName::ContentTransferEncoding\n                                    | HeaderName::ContentType\n                                    | HeaderName::ContentDisposition\n                                    | HeaderName::ListId\n                            ) {\n                                HeaderValue::from(hdr.value)\n                            } else {\n                                HeaderValue::Empty\n                            }\n                            .into(),\n                            name: hdr.name.into(),\n                            base_offset: hdr.offset_field as u32,\n                            start: (hdr.offset_start - hdr.offset_field) as u16,\n                            end: (hdr.offset_end - hdr.offset_field) as u16,\n                        })\n                        .collect(),\n                    flags,\n                    body,\n                    offset_header: part.offset_header as u32,\n                    offset_body: part.offset_body as u32,\n                    offset_end: part.offset_end as u32,\n                });\n            }\n\n            contents.push(MessageMetadataContents {\n                html_body: message.html_body.into_iter().map(|c| c as u16).collect(),\n                text_body: message.text_body.into_iter().map(|c| c as u16).collect(),\n                attachments: message.attachments.into_iter().map(|c| c as u16).collect(),\n                parts: parts.into_boxed_slice(),\n            });\n        }\n\n        MessageMetadata {\n            blob_body_offset: contents.first().unwrap().root_part().offset_body,\n            contents: contents.into_boxed_slice(),\n            blob_hash: legacy.blob_hash,\n            preview: legacy.preview.into_boxed_str(),\n            raw_headers: legacy.raw_headers.into_boxed_slice(),\n            rcvd_attach: (if legacy.has_attachments {\n                MESSAGE_HAS_ATTACHMENT\n            } else {\n                0\n            }) | (legacy.received_at & MESSAGE_RECEIVED_MASK),\n        }\n    }\n}\n\npub struct Mailboxes(Vec<UidMailbox>);\npub struct Keywords(Vec<LegacyKeyword>);\n\nimpl Deserialize for Mailboxes {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        let mut bytes = bytes.iter();\n        let len: usize = bytes\n            .next_leb128()\n            .ok_or_else(|| trc::StoreEvent::DataCorruption.caused_by(trc::location!()))?;\n        let mut list = Vec::with_capacity(len);\n        for _ in 0..len {\n            list.push(UidMailbox {\n                mailbox_id: bytes\n                    .next_leb128()\n                    .ok_or_else(|| trc::StoreEvent::DataCorruption.caused_by(trc::location!()))?,\n                uid: bytes\n                    .next_leb128()\n                    .ok_or_else(|| trc::StoreEvent::DataCorruption.caused_by(trc::location!()))?,\n            });\n        }\n        Ok(Mailboxes(list))\n    }\n}\n\nimpl Deserialize for Keywords {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        let mut bytes = bytes.iter();\n        let len: usize = bytes\n            .next_leb128()\n            .ok_or_else(|| trc::StoreEvent::DataCorruption.caused_by(trc::location!()))?;\n        let mut list = Vec::with_capacity(len);\n        for _ in 0..len {\n            list.push(\n                deserialize_keyword(&mut bytes)\n                    .ok_or_else(|| trc::StoreEvent::DataCorruption.caused_by(trc::location!()))?,\n            );\n        }\n        Ok(Keywords(list))\n    }\n}\n\nfn deserialize_keyword(bytes: &mut std::slice::Iter<'_, u8>) -> Option<LegacyKeyword> {\n    match bytes.next_leb128::<usize>()? {\n        SEEN => Some(LegacyKeyword::Seen),\n        DRAFT => Some(LegacyKeyword::Draft),\n        FLAGGED => Some(LegacyKeyword::Flagged),\n        ANSWERED => Some(LegacyKeyword::Answered),\n        RECENT => Some(LegacyKeyword::Recent),\n        IMPORTANT => Some(LegacyKeyword::Important),\n        PHISHING => Some(LegacyKeyword::Phishing),\n        JUNK => Some(LegacyKeyword::Junk),\n        NOTJUNK => Some(LegacyKeyword::NotJunk),\n        DELETED => Some(LegacyKeyword::Deleted),\n        FORWARDED => Some(LegacyKeyword::Forwarded),\n        MDN_SENT => Some(LegacyKeyword::MdnSent),\n        other => {\n            let len = other - 12;\n            let mut keyword = Vec::with_capacity(len);\n            for _ in 0..len {\n                keyword.push(*bytes.next()?);\n            }\n            Some(LegacyKeyword::Other(String::from_utf8(keyword).ok()?))\n        }\n    }\n}\n\npub type LegacyMessagePartId = usize;\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct LegacyMessageMetadata<'x> {\n    pub contents: LegacyMessageMetadataContents<'x>,\n    pub blob_hash: BlobHash,\n    pub size: usize,\n    pub received_at: u64,\n    pub preview: String,\n    pub has_attachments: bool,\n    pub raw_headers: Vec<u8>,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct LegacyMessageMetadataContents<'x> {\n    pub html_body: Vec<LegacyMessagePartId>,\n    pub text_body: Vec<LegacyMessagePartId>,\n    pub attachments: Vec<LegacyMessagePartId>,\n    pub parts: Vec<LegacyMessageMetadataPart<'x>>,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct LegacyMessageMetadataPart<'x> {\n    pub headers: Vec<LegacyHeader<'x>>,\n    pub is_encoding_problem: bool,\n    pub body: LegacyMetadataPartType<'x>,\n    pub encoding: Encoding,\n    pub size: usize,\n    pub offset_header: usize,\n    pub offset_body: usize,\n    pub offset_end: usize,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct LegacyHeader<'x> {\n    pub name: HeaderName<'x>,\n    pub value: LegacyHeaderValue<'x>,\n    pub offset_field: usize,\n    pub offset_start: usize,\n    pub offset_end: usize,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize, Default)]\npub enum LegacyHeaderValue<'x> {\n    /// Address list or group\n    Address(Address<'x>),\n\n    /// String\n    Text(Cow<'x, str>),\n\n    /// List of strings\n    TextList(Vec<Cow<'x, str>>),\n\n    /// Datetime\n    DateTime(DateTime),\n\n    /// Content-Type or Content-Disposition header\n    ContentType(LegacyContentType<'x>),\n\n    /// Received header\n    Received(Box<Received<'x>>),\n\n    #[default]\n    Empty,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub struct LegacyContentType<'x> {\n    pub c_type: Cow<'x, str>,\n    pub c_subtype: Option<Cow<'x, str>>,\n    pub attributes: Option<Vec<(Cow<'x, str>, Cow<'x, str>)>>,\n}\n\n#[derive(Debug, serde::Serialize, serde::Deserialize)]\npub enum LegacyMetadataPartType<'x> {\n    Text,\n    Html,\n    Binary,\n    InlineBinary,\n    Message(LegacyMessageMetadataContents<'x>),\n    Multipart(Vec<LegacyMessagePartId>),\n}\n\nimpl From<LegacyHeaderValue<'_>> for HeaderValue<'static> {\n    fn from(value: LegacyHeaderValue<'_>) -> Self {\n        match value {\n            LegacyHeaderValue::Address(address) => HeaderValue::Address(address.into_owned()),\n            LegacyHeaderValue::Text(cow) => HeaderValue::Text(cow.into_owned().into()),\n            LegacyHeaderValue::TextList(cows) => HeaderValue::TextList(\n                cows.into_iter()\n                    .map(|cow| cow.into_owned().into())\n                    .collect(),\n            ),\n            LegacyHeaderValue::DateTime(date_time) => HeaderValue::DateTime(date_time),\n            LegacyHeaderValue::ContentType(legacy_content_type) => {\n                HeaderValue::ContentType(ContentType {\n                    c_type: legacy_content_type.c_type.into_owned().into(),\n                    c_subtype: legacy_content_type.c_subtype.map(|s| s.into_owned().into()),\n                    attributes: legacy_content_type.attributes.map(|attrs| {\n                        attrs\n                            .into_iter()\n                            .map(|(k, v)| Attribute {\n                                name: k.into_owned().into(),\n                                value: v.into_owned().into(),\n                            })\n                            .collect()\n                    }),\n                })\n            }\n            LegacyHeaderValue::Received(received) => {\n                HeaderValue::Received(Box::new(received.into_owned()))\n            }\n            LegacyHeaderValue::Empty => HeaderValue::Empty,\n        }\n    }\n}\n\n/*pub(crate) fn encode_message_id(message_id: &str) -> Vec<u8> {\n    let mut msg_id = Vec::with_capacity(message_id.len() + 1);\n    msg_id.extend_from_slice(message_id.as_bytes());\n    msg_id.push(0);\n    msg_id\n}*/\n"
  },
  {
    "path": "crates/migration/src/email_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{email_v1::FIELD_MAILBOX_IDS, get_bitmap, v014::SUBSPACE_BITMAP_TAG};\nuse common::Server;\nuse email::{\n    mailbox::{JUNK_ID, TRASH_ID, UidMailbox},\n    message::{\n        index::extractors::VisitTextArchived,\n        ingest::ThreadInfo,\n        metadata::{\n            MESSAGE_HAS_ATTACHMENT, MESSAGE_RECEIVED_MASK, MessageData, MessageMetadata,\n            MessageMetadataContents, MessageMetadataPart, MetadataHeader, MetadataHeaderName,\n            MetadataHeaderValue, MetadataPartType, PART_ENCODING_BASE64, PART_ENCODING_PROBLEM,\n            PART_ENCODING_QP, PART_SIZE_MASK,\n        },\n    },\n};\nuse mail_parser::{Encoding, Header, parsers::fields::thread::thread_name};\nuse store::{\n    Serialize, SerializeInfallible, U32_LEN, U64_LEN, ValueKey,\n    rand::{self, seq::SliceRandom},\n    write::{\n        AlignedBytes, AnyKey, Archive, Archiver, BatchBuilder, IndexPropertyClass, ValueClass,\n        key::KeySerializer,\n    },\n};\nuse trc::AddContext;\nuse types::{blob_hash::BlobHash, collection::Collection, field::EmailField, keyword::*};\nuse utils::cheeky_hash::CheekyHash;\n\npub(crate) async fn migrate_emails_v014(server: &Server, account_id: u32) -> trc::Result<u64> {\n    let tombstoned_ids = get_bitmap(\n        server,\n        AnyKey {\n            subspace: SUBSPACE_BITMAP_TAG,\n            key: KeySerializer::new(U64_LEN + U32_LEN + 1)\n                .write(account_id)\n                .write(u8::from(Collection::Email))\n                .write(FIELD_MAILBOX_IDS)\n                .write_leb128(u32::MAX - 1)\n                .finalize(),\n        },\n        AnyKey {\n            subspace: SUBSPACE_BITMAP_TAG,\n            key: KeySerializer::new(U64_LEN + U32_LEN + 1)\n                .write(account_id)\n                .write(u8::from(Collection::Email))\n                .write(FIELD_MAILBOX_IDS)\n                .write_leb128(u32::MAX - 1)\n                .finalize(),\n        },\n    )\n    .await\n    .caused_by(trc::location!())?\n    .unwrap_or_default();\n\n    let mut migrate = Vec::new();\n\n    server\n        .archives(\n            account_id,\n            Collection::Email,\n            &(),\n            |document_id, archive| {\n                match archive.deserialize_untrusted::<LegacyMessageData>() {\n                    Ok(metadata) => {\n                        migrate.push((document_id, metadata));\n                    }\n                    Err(err) => {\n                        if archive.deserialize_untrusted::<MessageData>().is_err() {\n                            return Err(err\n                                .account_id(account_id)\n                                .document_id(document_id)\n                                .caused_by(trc::location!()));\n                        }\n                    }\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    migrate.shuffle(&mut rand::rng());\n\n    let num_emails = migrate.len();\n    for (document_id, legacy_data) in migrate {\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Email)\n            .with_document(document_id);\n\n        if !tombstoned_ids.contains(document_id) {\n            let (size, metadata) = match server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                    account_id,\n                    Collection::Email,\n                    document_id,\n                    EmailField::Metadata,\n                ))\n                .await?\n            {\n                Some(metadata) => match metadata.deserialize_untrusted::<LegacyMessageMetadata>() {\n                    Ok(legacy) => (legacy.size, MessageMetadata::from(legacy)),\n                    Err(err) => match metadata.deserialize_untrusted::<MessageMetadata>() {\n                        Ok(metadata) => (metadata.root_part().offset_end, metadata),\n                        Err(_) => {\n                            return Err(err\n                                .account_id(account_id)\n                                .document_id(document_id)\n                                .caused_by(trc::location!()));\n                        }\n                    },\n                },\n                None => {\n                    batch.clear(EmailField::Archive).clear(EmailField::Metadata);\n                    continue;\n                }\n            };\n            let data = MessageData {\n                mailboxes: legacy_data.mailboxes.into_boxed_slice(),\n                keywords: legacy_data.keywords.into_iter().map(Into::into).collect(),\n                thread_id: legacy_data.thread_id,\n                size,\n            };\n            let mut message_ids = Vec::new();\n            let mut subject = \"\";\n            for header in &metadata.contents[0].parts[0].headers {\n                match &header.name {\n                    MetadataHeaderName::MessageId => {\n                        header.value.visit_text(|id| {\n                            if !id.is_empty() {\n                                message_ids.push(CheekyHash::new(id.as_bytes()));\n                            }\n                        });\n                    }\n                    MetadataHeaderName::InReplyTo\n                    | MetadataHeaderName::References\n                    | MetadataHeaderName::ResentMessageId => {\n                        header.value.visit_text(|id| {\n                            if !id.is_empty() {\n                                message_ids.push(CheekyHash::new(id.as_bytes()));\n                            }\n                        });\n                    }\n                    MetadataHeaderName::Subject if subject.is_empty() => {\n                        subject = thread_name(match &header.value {\n                            MetadataHeaderValue::Text(text) => text.as_ref(),\n                            MetadataHeaderValue::TextList(list) if !list.is_empty() => {\n                                list.first().unwrap().as_ref()\n                            }\n                            _ => \"\",\n                        });\n                    }\n                    _ => (),\n                }\n            }\n\n            if data\n                .mailboxes\n                .iter()\n                .any(|mailbox| mailbox.mailbox_id == TRASH_ID || mailbox.mailbox_id == JUNK_ID)\n            {\n                batch.set(\n                    ValueClass::Property(EmailField::DeletedAt.into()),\n                    (metadata.rcvd_attach & MESSAGE_RECEIVED_MASK).serialize(),\n                );\n            }\n\n            batch\n                .set(\n                    ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                        property: EmailField::Threading.into(),\n                        hash: CheekyHash::new(if !subject.is_empty() { subject } else { \"!\" }),\n                    }),\n                    ThreadInfo::serialize(data.thread_id, &message_ids),\n                )\n                .set(\n                    EmailField::Archive,\n                    Archiver::new(data)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                )\n                .set(\n                    EmailField::Metadata,\n                    Archiver::new(metadata)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n        } else {\n            batch.clear(EmailField::Archive).clear(EmailField::Metadata);\n        }\n\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    Ok(num_emails as u64)\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug, Default)]\npub struct LegacyMessageData {\n    pub mailboxes: Vec<UidMailbox>,\n    pub keywords: Vec<LegacyKeyword>,\n    pub thread_id: u32,\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct LegacyMessageMetadata<'x> {\n    pub contents: Vec<LegacyMessageMetadataContents<'x>>,\n    pub blob_hash: BlobHash,\n    pub size: u32,\n    pub received_at: u64,\n    pub preview: String,\n    pub has_attachments: bool,\n    pub raw_headers: Vec<u8>,\n}\n\nimpl<'x> From<LegacyMessageMetadata<'x>> for MessageMetadata {\n    fn from(legacy: LegacyMessageMetadata<'x>) -> Self {\n        MessageMetadata {\n            blob_body_offset: legacy\n                .contents\n                .first()\n                .unwrap()\n                .parts\n                .first()\n                .unwrap()\n                .offset_body,\n            contents: legacy.contents.into_iter().map(Into::into).collect(),\n            blob_hash: legacy.blob_hash,\n            preview: legacy.preview.into_boxed_str(),\n            raw_headers: legacy.raw_headers.into_boxed_slice(),\n            rcvd_attach: (if legacy.has_attachments {\n                MESSAGE_HAS_ATTACHMENT\n            } else {\n                0\n            }) | (legacy.received_at & MESSAGE_RECEIVED_MASK),\n        }\n    }\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct LegacyMessageMetadataContents<'x> {\n    pub html_body: Vec<u16>,\n    pub text_body: Vec<u16>,\n    pub attachments: Vec<u16>,\n    pub parts: Vec<LegacyMessageMetadataPart<'x>>,\n}\n\nimpl<'x> From<LegacyMessageMetadataContents<'x>> for MessageMetadataContents {\n    fn from(contents: LegacyMessageMetadataContents) -> Self {\n        MessageMetadataContents {\n            html_body: contents.html_body.into_boxed_slice(),\n            text_body: contents.text_body.into_boxed_slice(),\n            attachments: contents.attachments.into_boxed_slice(),\n            parts: contents.parts.into_iter().map(Into::into).collect(),\n        }\n    }\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub struct LegacyMessageMetadataPart<'x> {\n    pub headers: Vec<Header<'x>>,\n    pub is_encoding_problem: bool,\n    pub body: LegacyMetadataPartType,\n    pub encoding: Encoding,\n    pub size: u32,\n    pub offset_header: u32,\n    pub offset_body: u32,\n    pub offset_end: u32,\n}\n\nimpl<'x> From<LegacyMessageMetadataPart<'x>> for MessageMetadataPart {\n    fn from(part: LegacyMessageMetadataPart<'x>) -> Self {\n        let flags = match part.encoding {\n            Encoding::None => 0,\n            Encoding::QuotedPrintable => PART_ENCODING_QP,\n            Encoding::Base64 => PART_ENCODING_BASE64,\n        } | (if part.is_encoding_problem {\n            PART_ENCODING_PROBLEM\n        } else {\n            0\n        }) | (part.size & PART_SIZE_MASK);\n\n        MessageMetadataPart {\n            headers: part\n                .headers\n                .into_iter()\n                .map(|hdr| MetadataHeader {\n                    value: hdr.value.into(),\n                    name: hdr.name.into(),\n                    base_offset: hdr.offset_field,\n                    start: (hdr.offset_start - hdr.offset_field) as u16,\n                    end: (hdr.offset_end - hdr.offset_field) as u16,\n                })\n                .collect(),\n            flags,\n            body: part.body.into(),\n            offset_header: part.offset_header,\n            offset_body: part.offset_body,\n            offset_end: part.offset_end,\n        }\n    }\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug)]\npub enum LegacyMetadataPartType {\n    Text,\n    Html,\n    Binary,\n    InlineBinary,\n    Message(u16),\n    Multipart(Vec<u16>),\n}\n\nimpl From<LegacyMetadataPartType> for MetadataPartType {\n    fn from(value: LegacyMetadataPartType) -> Self {\n        match value {\n            LegacyMetadataPartType::Text => MetadataPartType::Text,\n            LegacyMetadataPartType::Html => MetadataPartType::Html,\n            LegacyMetadataPartType::Binary => MetadataPartType::Binary,\n            LegacyMetadataPartType::InlineBinary => MetadataPartType::InlineBinary,\n            LegacyMetadataPartType::Message(id) => MetadataPartType::Message(id),\n            LegacyMetadataPartType::Multipart(children) => {\n                MetadataPartType::Multipart(children.into_boxed_slice())\n            }\n        }\n    }\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    Hash,\n    Default,\n    PartialOrd,\n    Ord,\n    serde::Serialize,\n)]\n#[serde(untagged)]\n#[rkyv(derive(PartialEq), compare(PartialEq))]\npub enum LegacyKeyword {\n    #[serde(rename(serialize = \"$seen\"))]\n    Seen,\n    #[serde(rename(serialize = \"$draft\"))]\n    Draft,\n    #[serde(rename(serialize = \"$flagged\"))]\n    Flagged,\n    #[serde(rename(serialize = \"$answered\"))]\n    Answered,\n    #[default]\n    #[serde(rename(serialize = \"$recent\"))]\n    Recent,\n    #[serde(rename(serialize = \"$important\"))]\n    Important,\n    #[serde(rename(serialize = \"$phishing\"))]\n    Phishing,\n    #[serde(rename(serialize = \"$junk\"))]\n    Junk,\n    #[serde(rename(serialize = \"$notjunk\"))]\n    NotJunk,\n    #[serde(rename(serialize = \"$deleted\"))]\n    Deleted,\n    #[serde(rename(serialize = \"$forwarded\"))]\n    Forwarded,\n    #[serde(rename(serialize = \"$mdnsent\"))]\n    MdnSent,\n    Other(String),\n}\n\nimpl From<LegacyKeyword> for Keyword {\n    fn from(kw: LegacyKeyword) -> Self {\n        match kw {\n            LegacyKeyword::Seen => Keyword::Seen,\n            LegacyKeyword::Draft => Keyword::Draft,\n            LegacyKeyword::Flagged => Keyword::Flagged,\n            LegacyKeyword::Answered => Keyword::Answered,\n            LegacyKeyword::Recent => Keyword::Recent,\n            LegacyKeyword::Important => Keyword::Important,\n            LegacyKeyword::Phishing => Keyword::Phishing,\n            LegacyKeyword::Junk => Keyword::Junk,\n            LegacyKeyword::NotJunk => Keyword::NotJunk,\n            LegacyKeyword::Deleted => Keyword::Deleted,\n            LegacyKeyword::Forwarded => Keyword::Forwarded,\n            LegacyKeyword::MdnSent => Keyword::MdnSent,\n            LegacyKeyword::Other(s) => Keyword::Other(s.into_boxed_str()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/encryption_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse email::message::crypto::EncryptionParams;\nuse store::{\n    Deserialize, Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::PrincipalField};\n\nuse crate::encryption_v2::LegacyEncryptionParams;\n\npub(crate) async fn migrate_encryption_params_v011(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<u64> {\n    match server\n        .store()\n        .get_value::<VeryOldLegacyEncryptionParams>(ValueKey {\n            account_id,\n            collection: Collection::Principal.into(),\n            document_id: 0,\n            class: ValueClass::from(PrincipalField::EncryptionKeys),\n        })\n        .await\n    {\n        Ok(Some(legacy)) => {\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .set(\n                    PrincipalField::EncryptionKeys,\n                    Archiver::new(EncryptionParams::from(legacy.0))\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n            return Ok(1);\n        }\n        Ok(None) => (),\n        Err(err) => {\n            if server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey {\n                    account_id,\n                    collection: Collection::Principal.into(),\n                    document_id: 0,\n                    class: ValueClass::from(PrincipalField::EncryptionKeys),\n                })\n                .await\n                .is_err()\n            {\n                return Err(err.account_id(account_id).caused_by(trc::location!()));\n            }\n        }\n    }\n    Ok(0)\n}\n\nstruct VeryOldLegacyEncryptionParams(LegacyEncryptionParams);\n\nimpl Deserialize for VeryOldLegacyEncryptionParams {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        let version = *bytes\n            .first()\n            .ok_or_else(|| trc::StoreEvent::DataCorruption.caused_by(trc::location!()))?;\n        match version {\n            1 if bytes.len() > 1 => bincode::deserialize(&bytes[1..])\n                .map(VeryOldLegacyEncryptionParams)\n                .map_err(|err| {\n                    trc::EventType::Store(trc::StoreEvent::DeserializeError)\n                        .reason(err)\n                        .caused_by(trc::location!())\n                }),\n\n            _ => Err(trc::StoreEvent::DeserializeError\n                .into_err()\n                .caused_by(trc::location!())\n                .ctx(trc::Key::Value, version as u64)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/encryption_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse email::message::crypto::{\n    ENCRYPT_ALGO_AES128, ENCRYPT_ALGO_AES256, ENCRYPT_METHOD_PGP, ENCRYPT_METHOD_SMIME,\n    EncryptionParams,\n};\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::PrincipalField};\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    Copy,\n    PartialEq,\n    Eq,\n    serde::Serialize,\n    serde::Deserialize,\n)]\npub enum EncryptionMethod {\n    PGP,\n    SMIME,\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    Copy,\n    serde::Serialize,\n    serde::Deserialize,\n)]\n#[rkyv(derive(Clone, Copy))]\npub enum Algorithm {\n    Aes128,\n    Aes256,\n}\n\n#[derive(\n    Clone,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    serde::Serialize,\n    serde::Deserialize,\n)]\npub struct LegacyEncryptionParams {\n    pub method: EncryptionMethod,\n    pub algo: Algorithm,\n    pub certs: Vec<Vec<u8>>,\n}\n\npub(crate) async fn migrate_encryption_params_v014(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<u64> {\n    let Some(params) = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey {\n            account_id,\n            collection: Collection::Principal.into(),\n            document_id: 0,\n            class: ValueClass::from(PrincipalField::EncryptionKeys),\n        })\n        .await\n        .caused_by(trc::location!())?\n    else {\n        return Ok(0);\n    };\n\n    match params.deserialize_untrusted::<LegacyEncryptionParams>() {\n        Ok(legacy) => {\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .set(\n                    PrincipalField::EncryptionKeys,\n                    Archiver::new(EncryptionParams::from(legacy))\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n            Ok(1)\n        }\n        Err(err) => {\n            if params.deserialize_untrusted::<EncryptionParams>().is_err() {\n                return Err(err.account_id(account_id).caused_by(trc::location!()));\n            }\n            Ok(0)\n        }\n    }\n}\n\nimpl From<LegacyEncryptionParams> for EncryptionParams {\n    fn from(legacy: LegacyEncryptionParams) -> Self {\n        EncryptionParams {\n            flags: match legacy.method {\n                EncryptionMethod::PGP => ENCRYPT_METHOD_PGP,\n                EncryptionMethod::SMIME => ENCRYPT_METHOD_SMIME,\n            } | match legacy.algo {\n                Algorithm::Aes128 => ENCRYPT_ALGO_AES128,\n                Algorithm::Aes256 => ENCRYPT_ALGO_AES256,\n            },\n            certs: legacy\n                .certs\n                .into_iter()\n                .map(|c| c.into_boxed_slice())\n                .collect(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/event_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{DavName, Server};\nuse groupware::calendar::{AlarmDelta, CalendarEvent, CalendarEventData, ComponentTimeRange};\nuse store::{\n    Serialize, ValueKey,\n    rand::{self, seq::SliceRandom},\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, serialize::rkyv_deserialize},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, dead_property::DeadProperty, field::Field};\n\nuse crate::{event_v2::migrate_icalendar_v02, get_document_ids};\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEventV1 {\n    pub names: Vec<DavName>,\n    pub display_name: Option<String>,\n    pub data: CalendarEventDataV1,\n    pub user_properties: Vec<UserProperties>,\n    pub flags: u16,\n    pub dead_properties: DeadProperty,\n    pub size: u32,\n    pub created: i64,\n    pub modified: i64,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct UserProperties {\n    pub account_id: u32,\n    pub properties: calcard_v01::icalendar::ICalendar,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEventDataV1 {\n    pub event: calcard_v01::icalendar::ICalendar,\n    pub time_ranges: Box<[ComponentTimeRange]>,\n    pub alarms: Box<[AlarmV1]>,\n    pub base_offset: i64,\n    pub base_time_utc: u32,\n    pub duration: u32,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(compare(PartialEq), derive(Debug))]\npub struct AlarmV1 {\n    pub comp_id: u16,\n    pub alarms: Box<[AlarmDelta]>,\n}\n\npub(crate) async fn migrate_calendar_events_v012(server: &Server) -> trc::Result<()> {\n    // Obtain email ids\n    let account_ids = get_document_ids(server, u32::MAX, Collection::Principal)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_accounts = account_ids.len();\n    if num_accounts == 0 {\n        return Ok(());\n    }\n\n    let mut account_ids = account_ids.into_iter().collect::<Vec<_>>();\n\n    account_ids.shuffle(&mut rand::rng());\n\n    for account_id in account_ids {\n        let document_ids = get_document_ids(server, account_id, Collection::CalendarEvent)\n            .await\n            .caused_by(trc::location!())?\n            .unwrap_or_default();\n        if document_ids.is_empty() {\n            continue;\n        }\n        let mut num_migrated = 0;\n\n        for document_id in document_ids.iter() {\n            let Some(archive) = server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                    account_id,\n                    Collection::CalendarEvent,\n                    document_id,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            else {\n                continue;\n            };\n\n            match archive.unarchive_untrusted::<CalendarEventV1>() {\n                Ok(event) => {\n                    let event = rkyv_deserialize::<_, CalendarEventV1>(event).unwrap();\n                    let mut next_email_alarm = None;\n                    let new_event = CalendarEvent {\n                        names: event.names,\n                        display_name: event.display_name,\n                        data: CalendarEventData::new(\n                            migrate_icalendar_v02(event.data.event),\n                            calcard_latest::common::timezone::Tz::Floating,\n                            server.core.groupware.max_ical_instances,\n                            &mut next_email_alarm,\n                        ),\n                        preferences: Default::default(),\n                        flags: event.flags,\n                        dead_properties: event.dead_properties,\n                        size: event.size,\n                        created: event.created,\n                        modified: event.modified,\n                        schedule_tag: None,\n                    };\n                    let mut batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(account_id)\n                        .with_collection(Collection::CalendarEvent)\n                        .with_document(document_id)\n                        .set(\n                            Field::ARCHIVE,\n                            Archiver::new(new_event)\n                                .serialize()\n                                .caused_by(trc::location!())?,\n                        );\n                    if let Some(next_email_alarm) = next_email_alarm {\n                        next_email_alarm.write_task(&mut batch);\n                    }\n                    server\n                        .store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                    num_migrated += 1;\n                }\n                Err(err) => {\n                    if let Err(err_) = archive.unarchive_untrusted::<CalendarEvent>() {\n                        trc::error!(err_.caused_by(trc::location!()));\n                        return Err(err.caused_by(trc::location!()));\n                    }\n                }\n            }\n        }\n\n        if num_migrated > 0 {\n            trc::event!(\n                Server(trc::ServerEvent::Startup),\n                Details =\n                    format!(\"Migrated {num_migrated} Calendar Events for account {account_id}\")\n            );\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/migration/src/event_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{DavName, Server};\nuse groupware::calendar::{\n    Alarm, CalendarEvent, CalendarEventData, CalendarEventNotification, ComponentTimeRange,\n};\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, serialize::rkyv_deserialize},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, dead_property::DeadProperty, field::Field};\n\nuse crate::get_document_ids;\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEventV2 {\n    pub names: Vec<DavName>,\n    pub display_name: Option<String>,\n    pub data: CalendarEventDataV2,\n    pub user_properties: Vec<UserPropertiesV2>,\n    pub flags: u16,\n    pub dead_properties: DeadProperty,\n    pub size: u32,\n    pub created: i64,\n    pub modified: i64,\n    pub schedule_tag: Option<u32>,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct UserPropertiesV2 {\n    pub account_id: u32,\n    pub properties: calcard_v01::icalendar::ICalendar,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEventDataV2 {\n    pub event: calcard_v01::icalendar::ICalendar,\n    pub time_ranges: Box<[ComponentTimeRange]>,\n    pub alarms: Box<[Alarm]>,\n    pub base_offset: i64,\n    pub base_time_utc: u32,\n    pub duration: u32,\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\npub struct CalendarEventNotificationV2 {\n    pub itip: calcard_v01::icalendar::ICalendar,\n    pub event_id: Option<u32>,\n    pub flags: u16,\n    pub size: u32,\n    pub created: i64,\n    pub modified: i64,\n}\n\npub(crate) async fn migrate_calendar_events_v013(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<u64> {\n    let document_ids = get_document_ids(server, account_id, Collection::CalendarEvent)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n\n    let mut num_migrated = 0;\n\n    for document_id in document_ids.iter() {\n        let Some(archive) = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::CalendarEvent,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n        else {\n            continue;\n        };\n\n        match archive.unarchive_untrusted::<CalendarEventV2>() {\n            Ok(event) => {\n                let event = rkyv_deserialize::<_, CalendarEventV2>(event).unwrap();\n                let new_event = CalendarEvent {\n                    names: event.names,\n                    display_name: event.display_name,\n                    data: CalendarEventData {\n                        event: migrate_icalendar_v02(event.data.event),\n                        time_ranges: event.data.time_ranges,\n                        alarms: event.data.alarms,\n                        base_offset: event.data.base_offset,\n                        base_time_utc: event.data.base_time_utc,\n                        duration: event.data.duration,\n                    },\n                    preferences: Default::default(),\n                    flags: event.flags,\n                    dead_properties: event.dead_properties,\n                    size: event.size,\n                    created: event.created,\n                    modified: event.modified,\n                    schedule_tag: None,\n                };\n\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::CalendarEvent)\n                    .with_document(document_id)\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(new_event)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                num_migrated += 1;\n            }\n            Err(err) => {\n                if let Err(err_) = archive.unarchive_untrusted::<CalendarEvent>() {\n                    trc::error!(err_.caused_by(trc::location!()));\n                    return Err(err.caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    Ok(num_migrated)\n}\n\npub(crate) async fn migrate_calendar_scheduling_v013(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<u64> {\n    let document_ids = get_document_ids(server, account_id, Collection::CalendarEventNotification)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n\n    let mut num_migrated = 0;\n\n    for document_id in document_ids.iter() {\n        let Some(archive) = server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::CalendarEventNotification,\n                document_id,\n            ))\n            .await\n            .caused_by(trc::location!())?\n        else {\n            continue;\n        };\n\n        match archive.unarchive_untrusted::<CalendarEventNotificationV2>() {\n            Ok(event) => {\n                let event = rkyv_deserialize::<_, CalendarEventNotificationV2>(event).unwrap();\n                let new_event = CalendarEventNotification {\n                    event: migrate_icalendar_v02(event.itip),\n                    event_id: event.event_id,\n                    changed_by: Default::default(),\n                    flags: 0,\n                    size: event.size,\n                    created: event.created,\n                    modified: event.modified,\n                };\n\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::CalendarEventNotification)\n                    .with_document(document_id)\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(new_event)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                num_migrated += 1;\n            }\n            Err(err) => {\n                if let Err(err_) = archive.unarchive_untrusted::<CalendarEventNotification>() {\n                    trc::error!(err_.caused_by(trc::location!()));\n                    return Err(err.caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    Ok(num_migrated)\n}\n\npub(crate) fn migrate_icalendar_v02(\n    ical: calcard_v01::icalendar::ICalendar,\n) -> calcard_latest::icalendar::ICalendar {\n    calcard_latest::icalendar::ICalendar::parse(ical.to_string()).unwrap_or_default()\n}\n"
  },
  {
    "path": "crates/migration/src/identity_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::object::Object;\nuse crate::{\n    get_document_ids,\n    object::{FromLegacy, Property, Value},\n};\nuse common::Server;\nuse email::identity::{EmailAddress, Identity};\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::Field};\n\npub(crate) async fn migrate_identities_v011(server: &Server, account_id: u32) -> trc::Result<u64> {\n    // Obtain identity ids\n    let identity_ids = get_document_ids(server, account_id, Collection::Identity)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_identities = identity_ids.len();\n    if num_identities == 0 {\n        return Ok(0);\n    }\n    let mut did_migrate = false;\n\n    for identity_id in &identity_ids {\n        match server\n            .store()\n            .get_value::<Object<Value>>(ValueKey {\n                account_id,\n                collection: Collection::Identity.into(),\n                document_id: identity_id,\n                class: ValueClass::Property(Field::ARCHIVE.into()),\n            })\n            .await\n        {\n            Ok(Some(legacy)) => {\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Identity)\n                    .with_document(identity_id)\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(Identity::from_legacy(legacy))\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n\n                did_migrate = true;\n\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            Ok(None) => (),\n            Err(err) => {\n                if server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey {\n                        account_id,\n                        collection: Collection::Identity.into(),\n                        document_id: identity_id,\n                        class: ValueClass::Property(Field::ARCHIVE.into()),\n                    })\n                    .await\n                    .is_err()\n                {\n                    return Err(err\n                        .account_id(account_id)\n                        .document_id(identity_id)\n                        .caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    // Increment document id counter\n    if did_migrate {\n        server\n            .store()\n            .assign_document_ids(\n                account_id,\n                Collection::Identity,\n                identity_ids\n                    .max()\n                    .map(|id| id as u64)\n                    .unwrap_or(num_identities)\n                    + 1,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        Ok(num_identities)\n    } else {\n        Ok(0)\n    }\n}\n\nimpl FromLegacy for Identity {\n    fn from_legacy(legacy: Object<Value>) -> Self {\n        Identity {\n            name: legacy\n                .get(&Property::Name)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n            email: legacy\n                .get(&Property::Email)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n            reply_to: convert_email_addresses(legacy.get(&Property::ReplyTo)),\n            bcc: convert_email_addresses(legacy.get(&Property::Bcc)),\n            text_signature: legacy\n                .get(&Property::TextSignature)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n            html_signature: legacy\n                .get(&Property::HtmlSignature)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n        }\n    }\n}\n\nfn convert_email_addresses(value: &Value) -> Option<Vec<EmailAddress>> {\n    if let Value::List(value) = value {\n        let mut addrs = Vec::with_capacity(value.len());\n        for addr in value {\n            if let Value::Object(obj) = addr {\n                let mut addr = EmailAddress {\n                    name: None,\n                    email: String::new(),\n                };\n                for (key, value) in &obj.properties {\n                    match (key, value) {\n                        (Property::Email, Value::Text(value)) => {\n                            addr.email = value.to_string();\n                        }\n                        (Property::Name, Value::Text(value)) => {\n                            addr.name = Some(value.to_string());\n                        }\n                        _ => {\n                            break;\n                        }\n                    }\n                }\n                if !addr.email.is_empty() {\n                    addrs.push(addr);\n                }\n            }\n        }\n        if !addrs.is_empty() { Some(addrs) } else { None }\n    } else {\n        None\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    blob::migrate_blobs_v014,\n    queue_v1::{migrate_queue_v011, migrate_queue_v012},\n    queue_v2::migrate_queue_v014,\n    v011::migrate_v0_11,\n    v012::migrate_v0_12,\n    v013::migrate_v0_13,\n    v014::{SUBSPACE_BITMAP_ID, migrate_principal_v0_14, migrate_v0_14},\n};\nuse common::{DATABASE_SCHEMA_VERSION, Server, manager::boot::DEFAULT_SETTINGS};\nuse std::time::Duration;\nuse store::{\n    Deserialize, IterateParams, SUBSPACE_PROPERTY, SUBSPACE_QUEUE_MESSAGE, SUBSPACE_REPORT_IN,\n    SUBSPACE_REPORT_OUT, SUBSPACE_SETTINGS, SerializeInfallible, U32_LEN, Value, ValueKey,\n    dispatch::DocumentSet,\n    roaring::RoaringBitmap,\n    write::{\n        AnyClass, AnyKey, BatchBuilder, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n    },\n};\nuse trc::AddContext;\nuse types::collection::Collection;\n\npub mod addressbook_v2;\npub mod blob;\npub mod calendar_v2;\npub mod changelog;\npub mod contact_v2;\npub mod email_v1;\npub mod email_v2;\npub mod encryption_v1;\npub mod encryption_v2;\npub mod event_v1;\npub mod event_v2;\npub mod identity_v1;\npub mod mailbox;\npub mod object;\npub mod principal_v1;\npub mod principal_v2;\npub mod push_v1;\npub mod push_v2;\npub mod queue_v1;\npub mod queue_v2;\npub mod report;\npub mod sieve_v1;\npub mod sieve_v2;\npub mod submission;\npub mod tasks_v1;\npub mod tasks_v2;\npub mod threads;\npub mod v011;\npub mod v012;\npub mod v013;\npub mod v014;\n\nconst LOCK_WAIT_TIME_ACCOUNT: u64 = 3 * 60;\nconst LOCK_WAIT_TIME_CORE: u64 = 5 * 60;\nconst LOCK_RETRY_TIME: Duration = Duration::from_secs(30);\n\npub async fn try_migrate(server: &Server) -> trc::Result<()> {\n    for var in [\n        \"FORCE_MIGRATE_QUEUE\",\n        \"FORCE_MIGRATE_BLOBS\",\n        \"FORCE_MIGRATE_ACCOUNT\",\n        \"FORCE_MIGRATE\",\n    ] {\n        let Some(version) = std::env::var(var).ok().and_then(|s| s.parse::<u32>().ok()) else {\n            continue;\n        };\n        match var {\n            \"FORCE_MIGRATE_QUEUE\" => match version {\n                1 => {\n                    migrate_queue_v011(server)\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n                2 => {\n                    migrate_queue_v012(server)\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n                4 => {\n                    migrate_queue_v014(server)\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n                _ => {\n                    panic!(\"Unknown migration queue version: {version}\");\n                }\n            },\n            \"FORCE_MIGRATE_BLOBS\" => {\n                migrate_blobs_v014(server)\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            \"FORCE_MIGRATE\" => match version {\n                1 => {\n                    migrate_v0_12(server, true)\n                        .await\n                        .caused_by(trc::location!())?;\n                    migrate_v0_13(server).await.caused_by(trc::location!())?;\n                    migrate_v0_14(server).await.caused_by(trc::location!())?;\n                }\n                2 => {\n                    migrate_v0_12(server, false)\n                        .await\n                        .caused_by(trc::location!())?;\n                    migrate_v0_13(server).await.caused_by(trc::location!())?;\n                    migrate_v0_14(server).await.caused_by(trc::location!())?;\n                }\n                3 => {\n                    migrate_v0_13(server).await.caused_by(trc::location!())?;\n                    migrate_v0_14(server).await.caused_by(trc::location!())?;\n                }\n                4 => {\n                    migrate_v0_14(server).await.caused_by(trc::location!())?;\n                }\n                _ => {\n                    panic!(\"Unknown migration version: {version}\");\n                }\n            },\n            \"FORCE_MIGRATE_ACCOUNT\" => {\n                migrate_principal_v0_14(server, version)\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            _ => unreachable!(),\n        }\n\n        return Ok(());\n    }\n\n    let add_v013_config = match server\n        .store()\n        .get_value::<u32>(AnyKey {\n            subspace: SUBSPACE_PROPERTY,\n            key: vec![0u8],\n        })\n        .await\n        .caused_by(trc::location!())?\n    {\n        Some(DATABASE_SCHEMA_VERSION) => {\n            return Ok(());\n        }\n        Some(1) => {\n            migrate_v0_12(server, true)\n                .await\n                .caused_by(trc::location!())?;\n            migrate_v0_13(server).await.caused_by(trc::location!())?;\n            migrate_v0_14(server).await.caused_by(trc::location!())?;\n            true\n        }\n        Some(2) => {\n            migrate_v0_12(server, false)\n                .await\n                .caused_by(trc::location!())?;\n            migrate_v0_13(server).await.caused_by(trc::location!())?;\n            migrate_v0_14(server).await.caused_by(trc::location!())?;\n            true\n        }\n        Some(3) => {\n            migrate_v0_13(server).await.caused_by(trc::location!())?;\n            migrate_v0_14(server).await.caused_by(trc::location!())?;\n            false\n        }\n        Some(4) => {\n            migrate_v0_14(server).await.caused_by(trc::location!())?;\n            false\n        }\n        Some(version) => {\n            panic!(\n                \"Unknown database schema version, expected {} or below, found {}\",\n                DATABASE_SCHEMA_VERSION, version\n            );\n        }\n        _ => {\n            if !is_new_install(server).await.caused_by(trc::location!())? {\n                migrate_v0_11(server).await.caused_by(trc::location!())?;\n                true\n            } else {\n                false\n            }\n        }\n    };\n\n    let mut batch = BatchBuilder::new();\n    batch.set(\n        ValueClass::Any(AnyClass {\n            subspace: SUBSPACE_PROPERTY,\n            key: vec![0u8],\n        }),\n        DATABASE_SCHEMA_VERSION.serialize(),\n    );\n\n    if add_v013_config {\n        for (key, value) in DEFAULT_SETTINGS {\n            if key\n                .strip_prefix(\"queue.\")\n                .is_some_and(|s| !s.starts_with(\"limiter.\") && !s.starts_with(\"quota.\"))\n            {\n                batch.set(\n                    ValueClass::Any(AnyClass {\n                        subspace: SUBSPACE_SETTINGS,\n                        key: key.as_bytes().to_vec(),\n                    }),\n                    value.as_bytes().to_vec(),\n                );\n            }\n        }\n    }\n\n    server\n        .store()\n        .write(batch.build_all())\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(())\n}\n\nasync fn is_new_install(server: &Server) -> trc::Result<bool> {\n    for subspace in [\n        SUBSPACE_QUEUE_MESSAGE,\n        SUBSPACE_REPORT_IN,\n        SUBSPACE_REPORT_OUT,\n        SUBSPACE_PROPERTY,\n    ] {\n        let mut has_data = false;\n\n        server\n            .store()\n            .iterate(\n                IterateParams::new(\n                    AnyKey {\n                        subspace,\n                        key: vec![0u8],\n                    },\n                    AnyKey {\n                        subspace,\n                        key: vec![u8::MAX; 16],\n                    },\n                )\n                .no_values(),\n                |_, _| {\n                    has_data = true;\n\n                    Ok(false)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if has_data {\n            return Ok(false);\n        }\n    }\n\n    Ok(true)\n}\n\nasync fn get_properties<U, I>(\n    server: &Server,\n    account_id: u32,\n    collection: Collection,\n    iterate: &I,\n    property: u8,\n) -> trc::Result<Vec<(u32, U)>>\nwhere\n    I: DocumentSet + Send + Sync,\n    U: Deserialize + 'static,\n{\n    let collection: u8 = collection.into();\n    let expected_results = iterate.len();\n    let mut results = Vec::with_capacity(expected_results);\n\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(\n                ValueKey {\n                    account_id,\n                    collection,\n                    document_id: iterate.min(),\n                    class: ValueClass::Property(property),\n                },\n                ValueKey {\n                    account_id,\n                    collection,\n                    document_id: iterate.max(),\n                    class: ValueClass::Property(property),\n                },\n            ),\n            |key, value| {\n                let document_id = key.deserialize_be_u32(key.len() - U32_LEN)?;\n                if iterate.contains(document_id) {\n                    results.push((document_id, U::deserialize(value)?));\n                    Ok(expected_results == 0 || results.len() < expected_results)\n                } else {\n                    Ok(true)\n                }\n            },\n        )\n        .await\n        .add_context(|err| {\n            err.caused_by(trc::location!())\n                .account_id(account_id)\n                .collection(collection)\n                .id(property.to_string())\n        })\n        .map(|_| results)\n}\n\npub async fn get_document_ids(\n    server: &Server,\n    account_id: u32,\n    collection: Collection,\n) -> trc::Result<Option<RoaringBitmap>> {\n    let collection: u8 = collection.into();\n    get_bitmap(\n        server,\n        AnyKey {\n            subspace: SUBSPACE_BITMAP_ID,\n            key: KeySerializer::new(U32_LEN + 1)\n                .write(account_id)\n                .write(collection)\n                .write(0u32)\n                .finalize(),\n        },\n        AnyKey {\n            subspace: SUBSPACE_BITMAP_ID,\n            key: KeySerializer::new(U32_LEN + 1)\n                .write(account_id)\n                .write(collection)\n                .write(u32::MAX)\n                .finalize(),\n        },\n    )\n    .await\n}\n\npub async fn get_bitmap(\n    server: &Server,\n    from_key: AnyKey<Vec<u8>>,\n    to_key: AnyKey<Vec<u8>>,\n) -> trc::Result<Option<RoaringBitmap>> {\n    let mut results = RoaringBitmap::new();\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(from_key, to_key).no_values(),\n            |key, _| {\n                results.insert(key.deserialize_be_u32(key.len() - U32_LEN)?);\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n        .map(|_| {\n            if !results.is_empty() {\n                Some(results)\n            } else {\n                None\n            }\n        })\n}\n\npub struct LegacyBincode<T: serde::de::DeserializeOwned> {\n    pub inner: T,\n}\n\nimpl<T: serde::de::DeserializeOwned> LegacyBincode<T> {\n    pub fn new(inner: T) -> Self {\n        Self { inner }\n    }\n}\n\nimpl<T: serde::de::DeserializeOwned> From<Value<'static>> for LegacyBincode<T> {\n    fn from(_: Value<'static>) -> Self {\n        unreachable!(\"From Value called on LegacyBincode<T>\")\n    }\n}\n\nimpl<T: serde::de::DeserializeOwned + Sized + Sync + Send> Deserialize for LegacyBincode<T> {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        lz4_flex::decompress_size_prepended(bytes)\n            .map_err(|err| {\n                trc::StoreEvent::DecompressError\n                    .ctx(trc::Key::Value, bytes)\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n            .and_then(|result| {\n                bincode::deserialize(&result).map_err(|err| {\n                    trc::StoreEvent::DataCorruption\n                        .ctx(trc::Key::Value, bytes)\n                        .caused_by(trc::location!())\n                        .reason(err)\n                })\n            })\n            .map(|inner| Self { inner })\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/mailbox.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::object::Object;\nuse crate::{\n    get_document_ids,\n    object::{FromLegacy, Property, Value},\n    v014::{SUBSPACE_BITMAP_TAG, SUBSPACE_BITMAP_TEXT},\n};\nuse common::Server;\nuse email::mailbox::Mailbox;\nuse store::{\n    SUBSPACE_INDEXES, Serialize, U64_LEN, ValueKey, rand,\n    write::{\n        AlignedBytes, AnyKey, Archive, Archiver, BatchBuilder, ValueClass, key::KeySerializer,\n    },\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::Field, special_use::SpecialUse};\nuse utils::config::utils::ParseValue;\n\npub(crate) async fn migrate_mailboxes(server: &Server, account_id: u32) -> trc::Result<u64> {\n    // Obtain email ids\n    let mailbox_ids = get_document_ids(server, account_id, Collection::Mailbox)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_mailboxes = mailbox_ids.len();\n    if num_mailboxes == 0 {\n        return Ok(0);\n    }\n    let mut did_migrate = false;\n\n    for mailbox_id in &mailbox_ids {\n        match server\n            .store()\n            .get_value::<Object<Value>>(ValueKey {\n                account_id,\n                collection: Collection::Mailbox.into(),\n                document_id: mailbox_id,\n                class: ValueClass::Property(Field::ARCHIVE.into()),\n            })\n            .await\n        {\n            Ok(Some(legacy)) => {\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::Mailbox)\n                    .with_document(mailbox_id)\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(Mailbox::from_legacy(legacy))\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                did_migrate = true;\n\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            Ok(None) => (),\n            Err(err) => {\n                if server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey {\n                        account_id,\n                        collection: Collection::Mailbox.into(),\n                        document_id: mailbox_id,\n                        class: ValueClass::Property(Field::ARCHIVE.into()),\n                    })\n                    .await\n                    .is_err()\n                {\n                    return Err(err\n                        .account_id(account_id)\n                        .document_id(mailbox_id)\n                        .caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    // Delete indexes\n    for subspace in [SUBSPACE_INDEXES, SUBSPACE_BITMAP_TAG, SUBSPACE_BITMAP_TEXT] {\n        server\n            .store()\n            .delete_range(\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::Mailbox))\n                        .finalize(),\n                },\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::Mailbox))\n                        .write(&[u8::MAX; 16][..])\n                        .finalize(),\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    // Increment document id counter\n    if did_migrate {\n        server\n            .store()\n            .assign_document_ids(\n                account_id,\n                Collection::Mailbox,\n                mailbox_ids\n                    .max()\n                    .map(|id| id as u64)\n                    .unwrap_or(num_mailboxes)\n                    + 1,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        Ok(num_mailboxes)\n    } else {\n        Ok(0)\n    }\n}\n\nimpl FromLegacy for Mailbox {\n    fn from_legacy(legacy: Object<Value>) -> Self {\n        Mailbox {\n            name: legacy\n                .get(&Property::Name)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n            role: legacy\n                .get(&Property::Role)\n                .as_string()\n                .and_then(|r| SpecialUse::parse_value(r).ok())\n                .unwrap_or(SpecialUse::None),\n            parent_id: legacy\n                .get(&Property::ParentId)\n                .as_uint()\n                .unwrap_or_default() as u32,\n            sort_order: legacy.get(&Property::SortOrder).as_uint().map(|s| s as u32),\n            uid_validity: rand::random(),\n            subscribers: legacy\n                .get(&Property::IsSubscribed)\n                .as_list()\n                .map(|s| s.as_slice())\n                .unwrap_or_default()\n                .iter()\n                .filter_map(|s| s.as_uint())\n                .map(|s| s as u32)\n                .collect(),\n            acls: legacy\n                .get(&Property::Acl)\n                .as_acl()\n                .cloned()\n                .unwrap_or_default(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/object.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::slice::Iter;\nuse store::{Deserialize, U64_LEN};\nuse types::{acl::AclGrant, blob::BlobId, id::Id, keyword::*};\nuse utils::{\n    codec::leb128::Leb128Iterator,\n    map::{bitmap::Bitmap, vec_map::VecMap},\n};\n\n#[derive(Debug, Clone, Default, PartialEq, Eq)]\npub struct Object<T> {\n    pub properties: VecMap<Property, T>,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Clone)]\npub enum Property {\n    Acl,\n    Aliases,\n    Attachments,\n    Bcc,\n    BlobId,\n    BodyStructure,\n    BodyValues,\n    Capabilities,\n    Cc,\n    Charset,\n    Cid,\n    DeliveryStatus,\n    Description,\n    DeviceClientId,\n    Disposition,\n    DsnBlobIds,\n    Email,\n    EmailId,\n    EmailIds,\n    Envelope,\n    Expires,\n    From,\n    FromDate,\n    HasAttachment,\n    Headers,\n    HtmlBody,\n    HtmlSignature,\n    Id,\n    IdentityId,\n    InReplyTo,\n    IsActive,\n    IsEnabled,\n    IsSubscribed,\n    Keys,\n    Keywords,\n    Language,\n    Location,\n    MailboxIds,\n    MayDelete,\n    MdnBlobIds,\n    Members,\n    MessageId,\n    MyRights,\n    Name,\n    ParentId,\n    PartId,\n    Picture,\n    Preview,\n    Quota,\n    ReceivedAt,\n    References,\n    ReplyTo,\n    Role,\n    Secret,\n    SendAt,\n    Sender,\n    SentAt,\n    Size,\n    SortOrder,\n    Subject,\n    SubParts,\n    TextBody,\n    TextSignature,\n    ThreadId,\n    Timezone,\n    To,\n    ToDate,\n    TotalEmails,\n    TotalThreads,\n    Type,\n    Types,\n    UndoStatus,\n    UnreadEmails,\n    UnreadThreads,\n    Url,\n    VerificationCode,\n    Addresses,\n    P256dh,\n    Auth,\n    Value,\n    SmtpReply,\n    Delivered,\n    Displayed,\n    MailFrom,\n    RcptTo,\n    Parameters,\n    IsEncodingProblem,\n    IsTruncated,\n    MayReadItems,\n    MayAddItems,\n    MayRemoveItems,\n    MaySetSeen,\n    MaySetKeywords,\n    MayCreateChild,\n    MayRename,\n    MaySubmit,\n    ResourceType,\n    Used,\n    HardLimit,\n    WarnLimit,\n    SoftLimit,\n    Scope,\n    _T(String),\n}\n\nimpl Object<Value> {\n    pub fn with_capacity(capacity: usize) -> Self {\n        Self {\n            properties: VecMap::with_capacity(capacity),\n        }\n    }\n\n    pub fn set(&mut self, property: Property, value: impl Into<Value>) -> bool {\n        self.properties.set(property, value.into())\n    }\n\n    pub fn append(&mut self, property: Property, value: impl Into<Value>) {\n        self.properties.append(property, value.into());\n    }\n\n    pub fn with_property(mut self, property: Property, value: impl Into<Value>) -> Self {\n        self.properties.append(property, value.into());\n        self\n    }\n\n    pub fn remove(&mut self, property: &Property) -> Value {\n        self.properties.remove(property).unwrap_or(Value::Null)\n    }\n\n    pub fn get(&self, property: &Property) -> &Value {\n        self.properties.get(property).unwrap_or(&Value::Null)\n    }\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\npub enum Value {\n    Text(String),\n    UnsignedInt(u64),\n    Bool(bool),\n    Id(Id),\n    Date(UTCDate),\n    BlobId(BlobId),\n    Keyword(Keyword),\n    List(Vec<Value>),\n    Object(Object<Value>),\n    Acl(Vec<AclGrant>),\n    Blob(Vec<u8>),\n    #[default]\n    Null,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, Hash)]\npub struct UTCDate {\n    pub year: u16,\n    pub month: u8,\n    pub day: u8,\n    pub hour: u8,\n    pub minute: u8,\n    pub second: u8,\n    pub tz_before_gmt: bool,\n    pub tz_hour: u8,\n    pub tz_minute: u8,\n}\n\nconst TEXT: u8 = 0;\nconst UNSIGNED_INT: u8 = 1;\nconst BOOL_TRUE: u8 = 2;\nconst BOOL_FALSE: u8 = 3;\nconst ID: u8 = 4;\nconst DATE: u8 = 5;\nconst BLOB_ID: u8 = 6;\nconst BLOB: u8 = 7;\nconst KEYWORD: u8 = 8;\nconst LIST: u8 = 9;\nconst OBJECT: u8 = 10;\nconst ACL: u8 = 11;\nconst NULL: u8 = 12;\n\npub trait DeserializeFrom: Sized {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Self>;\n}\n\nimpl Deserialize for Object<Value> {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Object::deserialize_from(&mut bytes.iter()).ok_or_else(|| {\n            trc::StoreEvent::DataCorruption\n                .caused_by(trc::location!())\n                .ctx(trc::Key::Value, bytes)\n        })\n    }\n}\n\nimpl DeserializeFrom for AclGrant {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Self> {\n        let account_id = bytes.next_leb128()?;\n        let mut grants = [0u8; U64_LEN];\n        for byte in grants.iter_mut() {\n            *byte = *bytes.next()?;\n        }\n\n        Some(Self {\n            account_id,\n            grants: Bitmap::from(u64::from_be_bytes(grants)),\n        })\n    }\n}\n\nimpl DeserializeFrom for Object<Value> {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Object<Value>> {\n        let len = bytes.next_leb128()?;\n        let mut properties = VecMap::with_capacity(len);\n        for _ in 0..len {\n            let key = Property::deserialize_from(bytes)?;\n            let value = Value::deserialize_from(bytes)?;\n            properties.append(key, value);\n        }\n        Some(Object { properties })\n    }\n}\n\nimpl DeserializeFrom for Value {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Self> {\n        match *bytes.next()? {\n            TEXT => Some(Value::Text(String::deserialize_from(bytes)?)),\n            UNSIGNED_INT => Some(Value::UnsignedInt(bytes.next_leb128()?)),\n            BOOL_TRUE => Some(Value::Bool(true)),\n            BOOL_FALSE => Some(Value::Bool(false)),\n            ID => Some(Value::Id(Id::new(bytes.next_leb128()?))),\n            DATE => Some(Value::Date(UTCDate::from_timestamp(\n                bytes.next_leb128::<u64>()? as i64,\n            ))),\n            BLOB_ID => Some(Value::BlobId(BlobId::deserialize_from(bytes)?)),\n            KEYWORD => Some(Value::Keyword(Keyword::deserialize_from(bytes)?)),\n            LIST => {\n                let len = bytes.next_leb128()?;\n                let mut items = Vec::with_capacity(len);\n                for _ in 0..len {\n                    items.push(Value::deserialize_from(bytes)?);\n                }\n                Some(Value::List(items))\n            }\n            OBJECT => Some(Value::Object(Object::deserialize_from(bytes)?)),\n            BLOB => Some(Value::Blob(Vec::deserialize_from(bytes)?)),\n            ACL => {\n                let len = bytes.next_leb128()?;\n                let mut items = Vec::with_capacity(len);\n                for _ in 0..len {\n                    items.push(AclGrant::deserialize_from(bytes)?);\n                }\n                Some(Value::Acl(items))\n            }\n            NULL => Some(Value::Null),\n            _ => None,\n        }\n    }\n}\n\nimpl DeserializeFrom for u32 {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Self> {\n        bytes.next_leb128()\n    }\n}\n\nimpl DeserializeFrom for u64 {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Self> {\n        bytes.next_leb128()\n    }\n}\n\nimpl DeserializeFrom for String {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Self> {\n        <Vec<u8>>::deserialize_from(bytes).and_then(|s| String::from_utf8(s).ok())\n    }\n}\n\nimpl DeserializeFrom for Vec<u8> {\n    fn deserialize_from(bytes: &mut Iter<'_, u8>) -> Option<Self> {\n        let len: usize = bytes.next_leb128()?;\n        let mut buf = Vec::with_capacity(len);\n        for _ in 0..len {\n            buf.push(*bytes.next()?);\n        }\n        buf.into()\n    }\n}\n\nimpl DeserializeFrom for BlobId {\n    fn deserialize_from(bytes: &mut std::slice::Iter<'_, u8>) -> Option<Self> {\n        BlobId::from_iter(bytes)\n    }\n}\n\nimpl DeserializeFrom for Keyword {\n    fn deserialize_from(bytes: &mut std::slice::Iter<'_, u8>) -> Option<Self> {\n        match bytes.next_leb128::<usize>()? {\n            SEEN => Some(Keyword::Seen),\n            DRAFT => Some(Keyword::Draft),\n            FLAGGED => Some(Keyword::Flagged),\n            ANSWERED => Some(Keyword::Answered),\n            RECENT => Some(Keyword::Recent),\n            IMPORTANT => Some(Keyword::Important),\n            PHISHING => Some(Keyword::Phishing),\n            JUNK => Some(Keyword::Junk),\n            NOTJUNK => Some(Keyword::NotJunk),\n            DELETED => Some(Keyword::Deleted),\n            FORWARDED => Some(Keyword::Forwarded),\n            MDN_SENT => Some(Keyword::MdnSent),\n            other => {\n                let len = other - 12;\n                let mut keyword = Vec::with_capacity(len);\n                for _ in 0..len {\n                    keyword.push(*bytes.next()?);\n                }\n                Some(Keyword::Other(\n                    String::from_utf8(keyword).ok()?.into_boxed_str(),\n                ))\n            }\n        }\n    }\n}\n\nimpl DeserializeFrom for Property {\n    fn deserialize_from(bytes: &mut std::slice::Iter<'_, u8>) -> Option<Self> {\n        match *bytes.next()? {\n            0 => Some(Property::IsActive),\n            1 => Some(Property::IsEnabled),\n            2 => Some(Property::IsSubscribed),\n            3 => Some(Property::Keys),\n            4 => Some(Property::Keywords),\n            5 => Some(Property::Language),\n            6 => Some(Property::Location),\n            7 => Some(Property::MailboxIds),\n            8 => Some(Property::MayDelete),\n            9 => Some(Property::MdnBlobIds),\n            10 => Some(Property::Members),\n            11 => Some(Property::MessageId),\n            12 => Some(Property::MyRights),\n            13 => Some(Property::Name),\n            14 => Some(Property::ParentId),\n            15 => Some(Property::PartId),\n            16 => Some(Property::Picture),\n            17 => Some(Property::Preview),\n            18 => Some(Property::Quota),\n            19 => Some(Property::ReceivedAt),\n            20 => Some(Property::References),\n            21 => Some(Property::ReplyTo),\n            22 => Some(Property::Role),\n            23 => Some(Property::Secret),\n            24 => Some(Property::SendAt),\n            25 => Some(Property::Sender),\n            26 => Some(Property::SentAt),\n            27 => Some(Property::Size),\n            28 => Some(Property::SortOrder),\n            29 => Some(Property::Subject),\n            30 => Some(Property::SubParts),\n            31 => Some(Property::TextBody),\n            32 => Some(Property::TextSignature),\n            33 => Some(Property::ThreadId),\n            34 => Some(Property::Timezone),\n            35 => Some(Property::To),\n            36 => Some(Property::ToDate),\n            37 => Some(Property::TotalEmails),\n            38 => Some(Property::TotalThreads),\n            39 => Some(Property::Type),\n            40 => Some(Property::Types),\n            41 => Some(Property::UndoStatus),\n            42 => Some(Property::UnreadEmails),\n            43 => Some(Property::UnreadThreads),\n            44 => Some(Property::Url),\n            45 => Some(Property::VerificationCode),\n            46 => Some(Property::Parameters),\n            47 => Some(Property::Addresses),\n            48 => Some(Property::P256dh),\n            49 => Some(Property::Auth),\n            50 => Some(Property::Value),\n            51 => Some(Property::SmtpReply),\n            52 => Some(Property::Delivered),\n            53 => Some(Property::Displayed),\n            54 => Some(Property::MailFrom),\n            55 => Some(Property::RcptTo),\n            56 => Some(Property::IsEncodingProblem),\n            57 => Some(Property::IsTruncated),\n            58 => Some(Property::MayReadItems),\n            59 => Some(Property::MayAddItems),\n            60 => Some(Property::MayRemoveItems),\n            61 => Some(Property::MaySetSeen),\n            62 => Some(Property::MaySetKeywords),\n            63 => Some(Property::MayCreateChild),\n            64 => Some(Property::MayRename),\n            65 => Some(Property::MaySubmit),\n            66 => Some(Property::Acl),\n            67 => Some(Property::Aliases),\n            68 => Some(Property::Attachments),\n            69 => Some(Property::Bcc),\n            70 => Some(Property::BlobId),\n            71 => Some(Property::BodyStructure),\n            72 => Some(Property::BodyValues),\n            73 => Some(Property::Capabilities),\n            74 => Some(Property::Cc),\n            75 => Some(Property::Charset),\n            76 => Some(Property::Cid),\n            77 => Some(Property::DeliveryStatus),\n            78 => Some(Property::Description),\n            79 => Some(Property::DeviceClientId),\n            80 => Some(Property::Disposition),\n            81 => Some(Property::DsnBlobIds),\n            82 => Some(Property::Email),\n            83 => Some(Property::EmailId),\n            84 => Some(Property::EmailIds),\n            85 => Some(Property::Envelope),\n            86 => Some(Property::Expires),\n            87 => Some(Property::From),\n            88 => Some(Property::FromDate),\n            89 => Some(Property::HasAttachment),\n            91 => Some(Property::Headers),\n            92 => Some(Property::HtmlBody),\n            93 => Some(Property::HtmlSignature),\n            94 => Some(Property::Id),\n            95 => Some(Property::IdentityId),\n            96 => Some(Property::InReplyTo),\n            97 => String::deserialize_from(bytes).map(Property::_T),\n            98 => Some(Property::ResourceType),\n            99 => Some(Property::Used),\n            100 => Some(Property::HardLimit),\n            101 => Some(Property::WarnLimit),\n            102 => Some(Property::SoftLimit),\n            103 => Some(Property::Scope),\n            _ => None,\n        }\n    }\n}\n\npub trait FromLegacy {\n    fn from_legacy(legacy: Object<Value>) -> Self;\n}\n\npub trait TryFromLegacy: Sized {\n    fn try_from_legacy(legacy: Object<Value>) -> Option<Self>;\n}\n\nimpl Value {\n    pub fn try_unwrap_id(self) -> Option<Id> {\n        match self {\n            Value::Id(id) => id.into(),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_bool(self) -> Option<bool> {\n        match self {\n            Value::Bool(b) => b.into(),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_keyword(self) -> Option<Keyword> {\n        match self {\n            Value::Keyword(k) => k.into(),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_string(self) -> Option<String> {\n        match self {\n            Value::Text(s) => Some(s),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_object(self) -> Option<Object<Value>> {\n        match self {\n            Value::Object(o) => Some(o),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_list(self) -> Option<Vec<Value>> {\n        match self {\n            Value::List(l) => Some(l),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_date(self) -> Option<UTCDate> {\n        match self {\n            Value::Date(d) => Some(d),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_blob_id(self) -> Option<BlobId> {\n        match self {\n            Value::BlobId(b) => Some(b),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_uint(self) -> Option<u64> {\n        match self {\n            Value::UnsignedInt(u) => Some(u),\n            _ => None,\n        }\n    }\n\n    pub fn as_string(&self) -> Option<&str> {\n        match self {\n            Value::Text(s) => Some(s),\n            _ => None,\n        }\n    }\n\n    pub fn as_id(&self) -> Option<&Id> {\n        match self {\n            Value::Id(id) => Some(id),\n            _ => None,\n        }\n    }\n\n    pub fn as_blob_id(&self) -> Option<&BlobId> {\n        match self {\n            Value::BlobId(id) => Some(id),\n            _ => None,\n        }\n    }\n\n    pub fn as_list(&self) -> Option<&Vec<Value>> {\n        match self {\n            Value::List(l) => Some(l),\n            _ => None,\n        }\n    }\n\n    pub fn as_acl(&self) -> Option<&Vec<AclGrant>> {\n        match self {\n            Value::Acl(l) => Some(l),\n            _ => None,\n        }\n    }\n\n    pub fn as_uint(&self) -> Option<u64> {\n        match self {\n            Value::UnsignedInt(u) => Some(*u),\n            Value::Id(id) => Some(*id.as_ref()),\n            _ => None,\n        }\n    }\n\n    pub fn as_bool(&self) -> Option<bool> {\n        match self {\n            Value::Bool(b) => Some(*b),\n            _ => None,\n        }\n    }\n\n    pub fn as_date(&self) -> Option<&UTCDate> {\n        match self {\n            Value::Date(d) => Some(d),\n            _ => None,\n        }\n    }\n\n    pub fn as_obj(&self) -> Option<&Object<Value>> {\n        match self {\n            Value::Object(o) => Some(o),\n            _ => None,\n        }\n    }\n\n    pub fn as_obj_mut(&mut self) -> Option<&mut Object<Value>> {\n        match self {\n            Value::Object(o) => Some(o),\n            _ => None,\n        }\n    }\n\n    pub fn try_cast_uint(&self) -> Option<u64> {\n        match self {\n            Value::UnsignedInt(u) => Some(*u),\n            Value::Id(id) => Some(id.id()),\n            Value::Bool(b) => Some(*b as u64),\n            _ => None,\n        }\n    }\n}\n\nimpl UTCDate {\n    pub fn from_timestamp(timestamp: i64) -> Self {\n        // Ported from http://howardhinnant.github.io/date_algorithms.html#civil_from_days\n        let (z, seconds) = ((timestamp / 86400) + 719468, timestamp % 86400);\n        let era: i64 = (if z >= 0 { z } else { z - 146096 }) / 146097;\n        let doe: u64 = (z - era * 146097) as u64; // [0, 146096]\n        let yoe: u64 = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365; // [0, 399]\n        let y: i64 = (yoe as i64) + era * 400;\n        let doy: u64 = doe - (365 * yoe + yoe / 4 - yoe / 100); // [0, 365]\n        let mp = (5 * doy + 2) / 153; // [0, 11]\n        let d: u64 = doy - (153 * mp + 2) / 5 + 1; // [1, 31]\n        let m: u64 = if mp < 10 { mp + 3 } else { mp - 9 }; // [1, 12]\n        let (h, mn, s) = (seconds / 3600, (seconds / 60) % 60, seconds % 60);\n\n        UTCDate {\n            year: (y + i64::from(m <= 2)) as u16,\n            month: m as u8,\n            day: d as u8,\n            hour: h as u8,\n            minute: mn as u8,\n            second: s as u8,\n            tz_before_gmt: false,\n            tz_hour: 0,\n            tz_minute: 0,\n        }\n    }\n\n    pub fn timestamp(&self) -> i64 {\n        // Ported from https://github.com/protocolbuffers/upb/blob/22182e6e/upb/json_decode.c#L982-L992\n        let month = self.month as u32;\n        let year_base = 4800; /* Before min year, multiple of 400. */\n        let m_adj = month.wrapping_sub(3); /* March-based month. */\n        let carry = i64::from(m_adj > month);\n        let adjust = if carry > 0 { 12 } else { 0 };\n        let y_adj = self.year as i64 + year_base - carry;\n        let month_days = ((m_adj.wrapping_add(adjust)) * 62719 + 769) / 2048;\n        let leap_days = y_adj / 4 - y_adj / 100 + y_adj / 400;\n        (y_adj * 365 + leap_days + month_days as i64 + (self.day as i64 - 1) - 2472632) * 86400\n            + self.hour as i64 * 3600\n            + self.minute as i64 * 60\n            + self.second as i64\n            + ((self.tz_hour as i64 * 3600 + self.tz_minute as i64 * 60)\n                * if self.tz_before_gmt { 1 } else { -1 })\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/principal_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    email_v1::migrate_emails_v011, encryption_v1::migrate_encryption_params_v011, get_document_ids,\n    identity_v1::migrate_identities_v011, mailbox::migrate_mailboxes,\n    push_v1::migrate_push_subscriptions_v011, sieve_v1::migrate_sieve_v011,\n    submission::migrate_email_submissions, threads::migrate_threads,\n};\nuse common::Server;\nuse directory::{\n    Permission, Principal, PrincipalData, ROLE_ADMIN, ROLE_USER, Type,\n    backend::internal::{PrincipalField, PrincipalSet, SpecialSecrets},\n};\nuse nlp::tokenizers::word::WordTokenizer;\nuse std::{slice::Iter, time::Instant};\nuse store::{\n    Deserialize, Serialize, ValueKey,\n    ahash::{AHashMap, AHashSet},\n    backend::MAX_TOKEN_LENGTH,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, DirectoryClass, ValueClass},\n};\nuse trc::AddContext;\nuse types::collection::Collection;\nuse utils::codec::leb128::Leb128Iterator;\n\npub(crate) async fn migrate_principals_v0_11(server: &Server) -> trc::Result<RoaringBitmap> {\n    // Obtain email ids\n    let principal_ids = get_document_ids(server, u32::MAX, Collection::Principal)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_principals = principal_ids.len();\n    if num_principals == 0 {\n        return Ok(principal_ids);\n    }\n    let mut num_migrated = 0;\n\n    for principal_id in principal_ids.iter() {\n        match server\n            .store()\n            .get_value::<LegacyPrincipal>(ValueKey {\n                account_id: u32::MAX,\n                collection: Collection::Principal.into(),\n                document_id: principal_id,\n                class: ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n            })\n            .await\n        {\n            Ok(Some(legacy)) => {\n                let mut principal = Principal::from_legacy(legacy);\n                principal.sort();\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(u32::MAX)\n                    .with_collection(Collection::Principal)\n                    .with_document(principal_id);\n\n                build_search_index(&mut batch, principal_id, &principal);\n\n                batch.set(\n                    ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n                    Archiver::new(principal)\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n                num_migrated += 1;\n\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            Ok(None) => (),\n            Err(err) => {\n                if server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey {\n                        account_id: u32::MAX,\n                        collection: Collection::Principal.into(),\n                        document_id: principal_id,\n                        class: ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n                    })\n                    .await\n                    .is_err()\n                {\n                    return Err(err.account_id(principal_id).caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    // Increment document id counter\n    if num_migrated > 0 {\n        server\n            .store()\n            .assign_document_ids(\n                u32::MAX,\n                Collection::Principal,\n                principal_ids\n                    .max()\n                    .map(|id| id as u64)\n                    .unwrap_or(num_principals)\n                    + 1,\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details = format!(\"Migrated {num_migrated} principals\",)\n        );\n    }\n\n    Ok(principal_ids)\n}\n\npub(crate) async fn migrate_principal_v0_11(server: &Server, account_id: u32) -> trc::Result<()> {\n    let start_time = Instant::now();\n    let num_emails = migrate_emails_v011(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_mailboxes = migrate_mailboxes(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_params = migrate_encryption_params_v011(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_subscriptions = migrate_push_subscriptions_v011(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_sieve = migrate_sieve_v011(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_submissions = migrate_email_submissions(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_threads = migrate_threads(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_identities = migrate_identities_v011(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n\n    if num_emails > 0\n        || num_mailboxes > 0\n        || num_params > 0\n        || num_subscriptions > 0\n        || num_sieve > 0\n        || num_submissions > 0\n        || num_threads > 0\n        || num_identities > 0\n    {\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details = format!(\n                \"Migrated accountId {account_id} with {num_emails} emails, {num_mailboxes} mailboxes, {num_params} encryption params, {num_submissions} email submissions, {num_sieve} sieve scripts, {num_subscriptions} push subscriptions, {num_threads} threads, and {num_identities} identities\"\n            ),\n            Elapsed = start_time.elapsed()\n        );\n    }\n\n    Ok(())\n}\n\ntrait FromLegacy {\n    fn from_legacy(legacy: LegacyPrincipal) -> Self;\n}\n\nimpl FromLegacy for Principal {\n    fn from_legacy(legacy: LegacyPrincipal) -> Self {\n        let mut legacy = legacy.0;\n        let mut principal = Principal {\n            id: legacy.id,\n            typ: legacy.typ,\n            name: legacy.name().to_string(),\n            data: Default::default(),\n        };\n\n        // Map fields\n        let mut has_secret = false;\n        for secret in legacy\n            .take_str_array(PrincipalField::Secrets)\n            .unwrap_or_default()\n        {\n            if secret.is_otp_secret() {\n                principal.data.push(PrincipalData::OtpAuth(secret));\n            } else if secret.is_app_secret() {\n                principal.data.push(PrincipalData::AppPassword(secret));\n            } else if !has_secret {\n                principal.data.push(PrincipalData::Password(secret));\n                has_secret = true;\n            }\n        }\n        for (idx, email) in legacy\n            .take_str_array(PrincipalField::Emails)\n            .unwrap_or_default()\n            .into_iter()\n            .enumerate()\n        {\n            if idx == 0 {\n                principal\n                    .data\n                    .push(PrincipalData::PrimaryEmail(email.clone()));\n            } else {\n                principal\n                    .data\n                    .push(PrincipalData::EmailAlias(email.clone()));\n            }\n        }\n        if let Some(picture) = legacy.take_str(PrincipalField::Picture) {\n            principal.data.push(PrincipalData::Picture(picture));\n        }\n        for url in legacy\n            .take_str_array(PrincipalField::Urls)\n            .unwrap_or_default()\n        {\n            principal.data.push(PrincipalData::Url(url));\n        }\n        for member in legacy\n            .take_str_array(PrincipalField::ExternalMembers)\n            .unwrap_or_default()\n        {\n            principal.data.push(PrincipalData::ExternalMember(member));\n        }\n\n        if let Some(quotas) = legacy.take_int_array(PrincipalField::Quota) {\n            for (idx, quota) in quotas.into_iter().take(Type::MAX_ID + 2).enumerate() {\n                if quota != 0 {\n                    if idx != 0 {\n                        principal.data.push(PrincipalData::DirectoryQuota {\n                            quota: quota as u32,\n                            typ: Type::from_u8((idx - 1) as u8),\n                        });\n                    } else {\n                        principal.data.push(PrincipalData::DiskQuota(quota));\n                    }\n                }\n            }\n        }\n\n        // Map permissions\n        let mut permissions = AHashMap::new();\n        for field in [\n            PrincipalField::EnabledPermissions,\n            PrincipalField::DisabledPermissions,\n        ] {\n            let is_disabled = field == PrincipalField::DisabledPermissions;\n            if let Some(ids) = legacy.take_int_array(field) {\n                for id in ids {\n                    if Permission::from_id(id as u32).is_some() {\n                        permissions.insert(id as u32, is_disabled);\n                    }\n                }\n            }\n        }\n        if !permissions.is_empty() {\n            for (k, v) in permissions {\n                principal.data.push(PrincipalData::Permission {\n                    permission_id: k,\n                    grant: !v,\n                });\n            }\n        }\n\n        principal\n    }\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\npub struct LegacyPrincipal(PrincipalSet);\n\nimpl Deserialize for LegacyPrincipal {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        deserialize(bytes).ok_or_else(|| {\n            trc::StoreEvent::DataCorruption\n                .caused_by(trc::location!())\n                .ctx(trc::Key::Value, bytes)\n        })\n    }\n}\n\nconst INT_MARKER: u8 = 1 << 7;\n\nfn deserialize(bytes: &[u8]) -> Option<LegacyPrincipal> {\n    let mut bytes = bytes.iter();\n\n    match *bytes.next()? {\n        1 => {\n            // Version 1 (legacy)\n            let id = bytes.next_leb128()?;\n            let type_id = *bytes.next()?;\n\n            let mut principal = PrincipalSet {\n                id,\n                typ: Type::from_u8(type_id),\n                ..Default::default()\n            };\n\n            principal.set(PrincipalField::Quota, bytes.next_leb128::<u64>()?);\n            principal.set(PrincipalField::Name, deserialize_string(&mut bytes)?);\n            if let Some(description) = deserialize_string(&mut bytes).filter(|s| !s.is_empty()) {\n                principal.set(PrincipalField::Description, description);\n            }\n            for key in [PrincipalField::Secrets, PrincipalField::Emails] {\n                for _ in 0..bytes.next_leb128::<usize>()? {\n                    principal.append_str(key, deserialize_string(&mut bytes)?);\n                }\n            }\n\n            LegacyPrincipal(principal.with_field(\n                PrincipalField::Roles,\n                if type_id != 4 { ROLE_USER } else { ROLE_ADMIN },\n            ))\n            .into()\n        }\n        2 => {\n            // Version 2\n            let typ = Type::from_u8(*bytes.next()?);\n            let num_fields = bytes.next_leb128::<usize>()?;\n\n            let mut principal = PrincipalSet {\n                id: u32::MAX,\n                typ,\n                fields: AHashMap::with_capacity(num_fields),\n            };\n\n            for _ in 0..num_fields {\n                let id = *bytes.next()?;\n                let num_values = bytes.next_leb128::<usize>()?;\n\n                if (id & INT_MARKER) == 0 {\n                    let field = PrincipalField::from_id(id)?;\n                    if num_values == 1 {\n                        principal.set(field, deserialize_string(&mut bytes)?);\n                    } else {\n                        let mut values = Vec::with_capacity(num_values);\n                        for _ in 0..num_values {\n                            values.push(deserialize_string(&mut bytes)?);\n                        }\n                        principal.set(field, values);\n                    }\n                } else {\n                    let field = PrincipalField::from_id(id & !INT_MARKER)?;\n                    if num_values == 1 {\n                        principal.set(field, bytes.next_leb128::<u64>()?);\n                    } else {\n                        let mut values = Vec::with_capacity(num_values);\n                        for _ in 0..num_values {\n                            values.push(bytes.next_leb128::<u64>()?);\n                        }\n                        principal.set(field, values);\n                    }\n                }\n            }\n\n            LegacyPrincipal(principal).into()\n        }\n        _ => None,\n    }\n}\n\nfn deserialize_string(bytes: &mut Iter<'_, u8>) -> Option<String> {\n    let len = bytes.next_leb128()?;\n    let mut string = Vec::with_capacity(len);\n    for _ in 0..len {\n        string.push(*bytes.next()?);\n    }\n    String::from_utf8(string).ok()\n}\n\npub(crate) fn build_search_index(batch: &mut BatchBuilder, principal_id: u32, new: &Principal) {\n    let mut new_words = AHashSet::new();\n\n    for word in [Some(new.name.as_str()), new.description()]\n        .into_iter()\n        .chain(new.email_addresses().map(Some))\n        .flatten()\n    {\n        new_words.extend(WordTokenizer::new(word, MAX_TOKEN_LENGTH).map(|t| t.word));\n    }\n\n    for word in new_words {\n        batch.set(\n            DirectoryClass::Index {\n                word: word.as_bytes().to_vec(),\n                principal_id,\n            },\n            vec![],\n        );\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/principal_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    addressbook_v2::migrate_addressbook_v013,\n    calendar_v2::migrate_calendar_v013,\n    contact_v2::migrate_contacts_v013,\n    event_v2::{migrate_calendar_events_v013, migrate_calendar_scheduling_v013},\n    get_document_ids,\n    push_v2::migrate_push_subscriptions_v013,\n    sieve_v2::migrate_sieve_v013,\n};\nuse common::Server;\nuse directory::{Principal, PrincipalData, Type, backend::internal::SpecialSecrets};\nuse proc_macros::EnumMethods;\nuse std::time::Instant;\nuse store::{\n    Serialize, ValueKey,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, DirectoryClass, ValueClass},\n};\nuse trc::AddContext;\nuse types::collection::Collection;\n\npub(crate) async fn migrate_principals_v0_13(server: &Server) -> trc::Result<RoaringBitmap> {\n    // Obtain email ids\n    let principal_ids = get_document_ids(server, u32::MAX, Collection::Principal)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_principals = principal_ids.len();\n    if num_principals == 0 {\n        return Ok(principal_ids);\n    }\n    let mut num_migrated = 0;\n\n    for principal_id in principal_ids.iter() {\n        match server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey {\n                account_id: u32::MAX,\n                collection: Collection::Principal.into(),\n                document_id: principal_id,\n                class: ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n            })\n            .await\n        {\n            Ok(Some(legacy)) => match legacy.deserialize_untrusted::<PrincipalV2>() {\n                Ok(old_principal) => {\n                    let mut principal = Principal {\n                        id: principal_id,\n                        typ: old_principal.typ,\n                        name: old_principal.name,\n                        data: Vec::new(),\n                    };\n\n                    let mut has_secret = false;\n                    for secret in old_principal.secrets {\n                        if secret.is_otp_secret() {\n                            principal.data.push(PrincipalData::OtpAuth(secret));\n                        } else if secret.is_app_secret() {\n                            principal.data.push(PrincipalData::AppPassword(secret));\n                        } else if !has_secret {\n                            principal.data.push(PrincipalData::Password(secret));\n                            has_secret = true;\n                        }\n                    }\n\n                    for (idx, email) in old_principal.emails.into_iter().enumerate() {\n                        if idx == 0 {\n                            principal.data.push(PrincipalData::PrimaryEmail(email));\n                        } else {\n                            principal.data.push(PrincipalData::EmailAlias(email));\n                        }\n                    }\n\n                    if let Some(description) = old_principal.description {\n                        principal.data.push(PrincipalData::Description(description));\n                    }\n\n                    if let Some(quota) = old_principal.quota\n                        && quota > 0\n                    {\n                        principal.data.push(PrincipalData::DiskQuota(quota));\n                    }\n\n                    if let Some(tenant) = old_principal.tenant {\n                        principal.data.push(PrincipalData::Tenant(tenant));\n                    }\n\n                    for item in old_principal.data {\n                        match item {\n                            PrincipalDataV2::MemberOf(items) => {\n                                for item in items {\n                                    principal.data.push(PrincipalData::MemberOf(item));\n                                }\n                            }\n                            PrincipalDataV2::Roles(items) => {\n                                for item in items {\n                                    principal.data.push(PrincipalData::Role(item));\n                                }\n                            }\n                            PrincipalDataV2::Lists(items) => {\n                                for item in items {\n                                    principal.data.push(PrincipalData::List(item));\n                                }\n                            }\n                            PrincipalDataV2::Permissions(items) => {\n                                for item in items {\n                                    principal.data.push(PrincipalData::Permission {\n                                        permission_id: item.permission.id(),\n                                        grant: item.grant,\n                                    });\n                                }\n                            }\n                            PrincipalDataV2::Picture(item) => {\n                                principal.data.push(PrincipalData::Picture(item));\n                            }\n                            PrincipalDataV2::ExternalMembers(items) => {\n                                for item in items {\n                                    principal.data.push(PrincipalData::ExternalMember(item));\n                                }\n                            }\n                            PrincipalDataV2::Urls(items) => {\n                                for item in items {\n                                    principal.data.push(PrincipalData::Url(item));\n                                }\n                            }\n                            PrincipalDataV2::PrincipalQuota(items) => {\n                                for item in items {\n                                    principal.data.push(PrincipalData::DirectoryQuota {\n                                        quota: item.quota as u32,\n                                        typ: item.typ,\n                                    });\n                                }\n                            }\n                            PrincipalDataV2::Locale(item) => {\n                                principal.data.push(PrincipalData::Locale(item));\n                            }\n                        }\n                    }\n\n                    principal.sort();\n\n                    let mut batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(u32::MAX)\n                        .with_collection(Collection::Principal)\n                        .with_document(principal_id);\n\n                    batch.set(\n                        ValueClass::Directory(DirectoryClass::Principal(principal_id)),\n                        Archiver::new(principal)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                    num_migrated += 1;\n\n                    server\n                        .store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n                Err(_) => {\n                    if let Err(err) = legacy.deserialize_untrusted::<Principal>() {\n                        return Err(err.account_id(principal_id).caused_by(trc::location!()));\n                    }\n                }\n            },\n            Ok(None) => (),\n            Err(err) => {\n                return Err(err.account_id(principal_id).caused_by(trc::location!()));\n            }\n        }\n    }\n\n    if num_migrated > 0 {\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details = format!(\"Migrated {num_migrated} principals\",)\n        );\n    }\n\n    Ok(principal_ids)\n}\n\npub(crate) async fn migrate_principal_v0_13(server: &Server, account_id: u32) -> trc::Result<()> {\n    let start_time = Instant::now();\n    let num_push = migrate_push_subscriptions_v013(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_sieve = migrate_sieve_v013(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_calendars = migrate_calendar_v013(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_events = migrate_calendar_events_v013(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_event_scheduling = migrate_calendar_scheduling_v013(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_books = migrate_addressbook_v013(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n    let num_contacts = migrate_contacts_v013(server, account_id)\n        .await\n        .caused_by(trc::location!())?;\n\n    if num_sieve > 0\n        || num_books > 0\n        || num_contacts > 0\n        || num_calendars > 0\n        || num_events > 0\n        || num_push > 0\n        || num_event_scheduling > 0\n    {\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details = format!(\n                \"Migrated accountId {account_id} with {num_sieve} sieve scripts, {num_push} push subscriptions, {num_calendars} calendars, {num_events} calendar events, {num_event_scheduling} event scheduling, {num_books} address books and {num_contacts} contacts\"\n            ),\n            Elapsed = start_time.elapsed()\n        );\n    }\n\n    Ok(())\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct PrincipalV2 {\n    pub id: u32,\n    pub typ: Type,\n    pub name: String,\n    pub description: Option<String>,\n    pub secrets: Vec<String>,\n    pub emails: Vec<String>,\n    pub quota: Option<u64>,\n    pub tenant: Option<u32>,\n    pub data: Vec<PrincipalDataV2>,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub enum PrincipalDataV2 {\n    MemberOf(Vec<u32>),\n    Roles(Vec<u32>),\n    Lists(Vec<u32>),\n    Permissions(Vec<PermissionGrantV2>),\n    Picture(String),\n    ExternalMembers(Vec<String>),\n    Urls(Vec<String>),\n    PrincipalQuota(Vec<PrincipalQuotaV2>),\n    Locale(String),\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct PrincipalQuotaV2 {\n    pub quota: u64,\n    pub typ: Type,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct PermissionGrantV2 {\n    pub permission: PermissionV2,\n    pub grant: bool,\n}\n\n#[derive(\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    Debug,\n    Clone,\n    Copy,\n    PartialEq,\n    Eq,\n    Hash,\n    serde::Serialize,\n    serde::Deserialize,\n    EnumMethods,\n)]\n#[serde(rename_all = \"kebab-case\")]\npub enum PermissionV2 {\n    // WARNING: add new ids at the end (TODO: use static ids)\n\n    // Admin\n    Impersonate,\n    UnlimitedRequests,\n    UnlimitedUploads,\n    DeleteSystemFolders,\n    MessageQueueList,\n    MessageQueueGet,\n    MessageQueueUpdate,\n    MessageQueueDelete,\n    OutgoingReportList,\n    OutgoingReportGet,\n    OutgoingReportDelete,\n    IncomingReportList,\n    IncomingReportGet,\n    IncomingReportDelete,\n    SettingsList,\n    SettingsUpdate,\n    SettingsDelete,\n    SettingsReload,\n    IndividualList,\n    IndividualGet,\n    IndividualUpdate,\n    IndividualDelete,\n    IndividualCreate,\n    GroupList,\n    GroupGet,\n    GroupUpdate,\n    GroupDelete,\n    GroupCreate,\n    DomainList,\n    DomainGet,\n    DomainCreate,\n    DomainUpdate,\n    DomainDelete,\n    TenantList,\n    TenantGet,\n    TenantCreate,\n    TenantUpdate,\n    TenantDelete,\n    MailingListList,\n    MailingListGet,\n    MailingListCreate,\n    MailingListUpdate,\n    MailingListDelete,\n    RoleList,\n    RoleGet,\n    RoleCreate,\n    RoleUpdate,\n    RoleDelete,\n    PrincipalList,\n    PrincipalGet,\n    PrincipalCreate,\n    PrincipalUpdate,\n    PrincipalDelete,\n    BlobFetch,\n    PurgeBlobStore,\n    PurgeDataStore,\n    PurgeInMemoryStore,\n    PurgeAccount,\n    FtsReindex,\n    Undelete,\n    DkimSignatureCreate,\n    DkimSignatureGet,\n    SpamFilterUpdate,\n    WebadminUpdate,\n    LogsView,\n    SpamFilterTrain,\n    Restart,\n    TracingList,\n    TracingGet,\n    TracingLive,\n    MetricsList,\n    MetricsLive,\n\n    // Generic\n    Authenticate,\n    AuthenticateOauth,\n    EmailSend,\n    EmailReceive,\n\n    // Account Management\n    ManageEncryption,\n    ManagePasswords,\n\n    // JMAP\n    JmapEmailGet,\n    JmapMailboxGet,\n    JmapThreadGet,\n    JmapIdentityGet,\n    JmapEmailSubmissionGet,\n    JmapPushSubscriptionGet,\n    JmapSieveScriptGet,\n    JmapVacationResponseGet,\n    JmapPrincipalGet,\n    JmapQuotaGet,\n    JmapBlobGet,\n    JmapEmailSet,\n    JmapMailboxSet,\n    JmapIdentitySet,\n    JmapEmailSubmissionSet,\n    JmapPushSubscriptionSet,\n    JmapSieveScriptSet,\n    JmapVacationResponseSet,\n    JmapEmailChanges,\n    JmapMailboxChanges,\n    JmapThreadChanges,\n    JmapIdentityChanges,\n    JmapEmailSubmissionChanges,\n    JmapQuotaChanges,\n    JmapEmailCopy,\n    JmapBlobCopy,\n    JmapEmailImport,\n    JmapEmailParse,\n    JmapEmailQueryChanges,\n    JmapMailboxQueryChanges,\n    JmapEmailSubmissionQueryChanges,\n    JmapSieveScriptQueryChanges,\n    JmapPrincipalQueryChanges,\n    JmapQuotaQueryChanges,\n    JmapEmailQuery,\n    JmapMailboxQuery,\n    JmapEmailSubmissionQuery,\n    JmapSieveScriptQuery,\n    JmapPrincipalQuery,\n    JmapQuotaQuery,\n    JmapSearchSnippet,\n    JmapSieveScriptValidate,\n    JmapBlobLookup,\n    JmapBlobUpload,\n    JmapEcho,\n\n    // IMAP\n    ImapAuthenticate,\n    ImapAclGet,\n    ImapAclSet,\n    ImapMyRights,\n    ImapListRights,\n    ImapAppend,\n    ImapCapability,\n    ImapId,\n    ImapCopy,\n    ImapMove,\n    ImapCreate,\n    ImapDelete,\n    ImapEnable,\n    ImapExpunge,\n    ImapFetch,\n    ImapIdle,\n    ImapList,\n    ImapLsub,\n    ImapNamespace,\n    ImapRename,\n    ImapSearch,\n    ImapSort,\n    ImapSelect,\n    ImapExamine,\n    ImapStatus,\n    ImapStore,\n    ImapSubscribe,\n    ImapThread,\n\n    // POP3\n    Pop3Authenticate,\n    Pop3List,\n    Pop3Uidl,\n    Pop3Stat,\n    Pop3Retr,\n    Pop3Dele,\n\n    // ManageSieve\n    SieveAuthenticate,\n    SieveListScripts,\n    SieveSetActive,\n    SieveGetScript,\n    SievePutScript,\n    SieveDeleteScript,\n    SieveRenameScript,\n    SieveCheckScript,\n    SieveHaveSpace,\n\n    // API keys\n    ApiKeyList,\n    ApiKeyGet,\n    ApiKeyCreate,\n    ApiKeyUpdate,\n    ApiKeyDelete,\n\n    // OAuth clients\n    OauthClientList,\n    OauthClientGet,\n    OauthClientCreate,\n    OauthClientUpdate,\n    OauthClientDelete,\n\n    // OAuth client registration\n    OauthClientRegistration,\n    OauthClientOverride,\n\n    AiModelInteract,\n    Troubleshoot,\n    SpamFilterClassify,\n\n    // WebDAV permissions\n    DavSyncCollection,\n    DavExpandProperty,\n\n    DavPrincipalAcl,\n    DavPrincipalList,\n    DavPrincipalMatch,\n    DavPrincipalSearch,\n    DavPrincipalSearchPropSet,\n\n    DavFilePropFind,\n    DavFilePropPatch,\n    DavFileGet,\n    DavFileMkCol,\n    DavFileDelete,\n    DavFilePut,\n    DavFileCopy,\n    DavFileMove,\n    DavFileLock,\n    DavFileAcl,\n\n    DavCardPropFind,\n    DavCardPropPatch,\n    DavCardGet,\n    DavCardMkCol,\n    DavCardDelete,\n    DavCardPut,\n    DavCardCopy,\n    DavCardMove,\n    DavCardLock,\n    DavCardAcl,\n    DavCardQuery,\n    DavCardMultiGet,\n\n    DavCalPropFind,\n    DavCalPropPatch,\n    DavCalGet,\n    DavCalMkCol,\n    DavCalDelete,\n    DavCalPut,\n    DavCalCopy,\n    DavCalMove,\n    DavCalLock,\n    DavCalAcl,\n    DavCalQuery,\n    DavCalMultiGet,\n    DavCalFreeBusyQuery,\n\n    CalendarAlarms,\n    CalendarSchedulingSend,\n    CalendarSchedulingReceive,\n    // WARNING: add new ids at the end (TODO: use static ids)\n}\n"
  },
  {
    "path": "crates/migration/src/push_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::object::Object;\nuse crate::{\n    get_document_ids,\n    object::{FromLegacy, Property, Value},\n};\nuse base64::{Engine, engine::general_purpose};\nuse common::Server;\nuse email::push::{Keys, PushSubscription, PushSubscriptions};\nuse store::{\n    Serialize, ValueKey,\n    write::{Archiver, BatchBuilder, ValueClass},\n};\nuse trc::AddContext;\nuse types::{\n    collection::Collection,\n    field::{Field, PrincipalField},\n    type_state::DataType,\n};\n\npub(crate) async fn migrate_push_subscriptions_v011(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<u64> {\n    // Obtain email ids\n    let push_subscription_ids = get_document_ids(server, account_id, Collection::PushSubscription)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_push_subscriptions = push_subscription_ids.len();\n    if num_push_subscriptions == 0 {\n        return Ok(0);\n    }\n    let mut subscriptions = Vec::with_capacity(num_push_subscriptions as usize);\n\n    for push_subscription_id in &push_subscription_ids {\n        match server\n            .store()\n            .get_value::<Object<Value>>(ValueKey {\n                account_id,\n                collection: Collection::PushSubscription.into(),\n                document_id: push_subscription_id,\n                class: ValueClass::Property(Field::ARCHIVE.into()),\n            })\n            .await\n        {\n            Ok(Some(legacy)) => {\n                let mut subscription = PushSubscription::from_legacy(legacy);\n                subscription.id = push_subscription_id;\n                subscriptions.push(subscription);\n            }\n            Ok(None) => (),\n            Err(err) => {\n                return Err(err\n                    .account_id(account_id)\n                    .document_id(push_subscription_id)\n                    .caused_by(trc::location!()));\n            }\n        }\n    }\n\n    if !subscriptions.is_empty() {\n        // Save changes\n        let num_push_subscriptions = subscriptions.len() as u64;\n        let mut batch = BatchBuilder::new();\n\n        batch\n            .with_account_id(u32::MAX)\n            .with_collection(Collection::Principal)\n            .with_document(account_id)\n            .tag(PrincipalField::PushSubscriptions)\n            .with_account_id(account_id)\n            .with_collection(Collection::PushSubscription);\n\n        for subscription in &subscriptions {\n            batch.with_document(subscription.id).clear(Field::ARCHIVE);\n        }\n\n        batch\n            .with_collection(Collection::Principal)\n            .with_document(0)\n            .set(\n                PrincipalField::PushSubscriptions,\n                Archiver::new(PushSubscriptions { subscriptions })\n                    .serialize()\n                    .caused_by(trc::location!())?,\n            );\n\n        server\n            .commit_batch(batch)\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(num_push_subscriptions)\n    } else {\n        Ok(0)\n    }\n}\n\nimpl FromLegacy for PushSubscription {\n    fn from_legacy(legacy: Object<Value>) -> Self {\n        let (verification_code, verified) = legacy\n            .get(&Property::VerificationCode)\n            .as_string()\n            .map(|c| (c.to_string(), true))\n            .or_else(|| {\n                legacy\n                    .get(&Property::Value)\n                    .as_string()\n                    .map(|c| (c.to_string(), false))\n            })\n            .unwrap_or_default();\n\n        PushSubscription {\n            id: 0,\n            url: legacy\n                .get(&Property::Url)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n            device_client_id: legacy\n                .get(&Property::DeviceClientId)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n            expires: legacy\n                .get(&Property::Expires)\n                .as_date()\n                .map(|s| s.timestamp() as u64)\n                .unwrap_or_default(),\n            verification_code,\n            verified,\n            types: legacy\n                .get(&Property::Types)\n                .as_list()\n                .map(|l| l.as_slice())\n                .unwrap_or_default()\n                .iter()\n                .filter_map(|v| v.as_string().and_then(DataType::parse))\n                .collect(),\n            keys: convert_keys(legacy.get(&Property::Keys)),\n            email_push: vec![],\n        }\n    }\n}\n\nfn convert_keys(value: &Value) -> Option<Keys> {\n    let mut addr = Keys {\n        p256dh: Default::default(),\n        auth: Default::default(),\n    };\n    if let Value::Object(obj) = value {\n        for (key, value) in &obj.properties {\n            match (key, value) {\n                (Property::Auth, Value::Text(value)) => {\n                    addr.auth = general_purpose::URL_SAFE.decode(value).unwrap_or_default();\n                }\n                (Property::P256dh, Value::Text(value)) => {\n                    addr.p256dh = general_purpose::URL_SAFE.decode(value).unwrap_or_default();\n                }\n                _ => {}\n            }\n        }\n    }\n    if !addr.p256dh.is_empty() && !addr.auth.is_empty() {\n        Some(addr)\n    } else {\n        None\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/push_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse email::push::{Keys, PushSubscription, PushSubscriptions};\nuse store::{\n    Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, now},\n};\nuse trc::AddContext;\nuse types::{\n    collection::Collection,\n    field::{Field, PrincipalField},\n    type_state::DataType,\n};\nuse utils::map::bitmap::Bitmap;\n\nuse crate::get_document_ids;\n\npub(crate) async fn migrate_push_subscriptions_v013(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<u64> {\n    // Obtain email ids\n    let push_ids = get_document_ids(server, account_id, Collection::PushSubscription)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_pushes = push_ids.len();\n    if num_pushes == 0 {\n        return Ok(0);\n    }\n    let mut subscriptions = Vec::with_capacity(num_pushes as usize);\n\n    for push_id in &push_ids {\n        match server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::PushSubscription,\n                push_id,\n            ))\n            .await\n        {\n            Ok(Some(legacy)) => match legacy.deserialize_untrusted::<PushSubscriptionV2>() {\n                Ok(old_push) => {\n                    subscriptions.push(PushSubscription {\n                        id: push_id,\n                        url: old_push.url,\n                        device_client_id: old_push.device_client_id,\n                        expires: old_push.expires,\n                        verification_code: old_push.verification_code,\n                        verified: old_push.verified,\n                        types: old_push.types,\n                        keys: old_push.keys,\n                        email_push: Vec::new(),\n                    });\n                }\n                Err(err) => {\n                    return Err(err.account_id(push_id).caused_by(trc::location!()));\n                }\n            },\n            Ok(None) => (),\n            Err(err) => {\n                return Err(err.account_id(push_id).caused_by(trc::location!()));\n            }\n        }\n    }\n\n    if !subscriptions.is_empty() {\n        // Save changes\n        let num_push_subscriptions = subscriptions.len() as u64;\n        let now = now();\n        let mut batch = BatchBuilder::new();\n\n        // Delete archived and document ids\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::PushSubscription);\n        for subscription in &subscriptions {\n            batch.with_document(subscription.id).clear(Field::ARCHIVE);\n        }\n\n        subscriptions.retain(|s| s.verified && s.expires > now);\n\n        if !subscriptions.is_empty() {\n            batch\n                .with_account_id(u32::MAX)\n                .with_collection(Collection::Principal)\n                .with_document(account_id)\n                .tag(PrincipalField::PushSubscriptions)\n                .with_account_id(account_id)\n                .with_collection(Collection::Principal)\n                .with_document(0)\n                .set(\n                    PrincipalField::PushSubscriptions,\n                    Archiver::new(PushSubscriptions { subscriptions })\n                        .serialize()\n                        .caused_by(trc::location!())?,\n                );\n        }\n\n        server\n            .commit_batch(batch)\n            .await\n            .caused_by(trc::location!())?;\n\n        Ok(num_push_subscriptions)\n    } else {\n        Ok(0)\n    }\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Default, Debug, Clone, PartialEq, Eq,\n)]\npub struct PushSubscriptionV2 {\n    pub url: String,\n    pub device_client_id: String,\n    pub expires: u64,\n    pub verification_code: String,\n    pub verified: bool,\n    pub types: Bitmap<DataType>,\n    pub keys: Option<Keys>,\n}\n"
  },
  {
    "path": "crates/migration/src/queue_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    LegacyBincode,\n    queue_v2::{LegacyHostResponse, LegacyQuotaKey},\n};\nuse common::{\n    Server,\n    config::smtp::queue::{DEFAULT_QUEUE_NAME, QueueExpiry, QueueName},\n};\nuse smtp::queue::{\n    Error, ErrorDetails, HostResponse, Message, QueueId, Recipient, Schedule, Status,\n    UnexpectedResponse,\n};\nuse smtp_proto::Response;\nuse std::net::{IpAddr, Ipv4Addr};\nuse store::{\n    IterateParams, SUBSPACE_QUEUE_EVENT, Serialize, U64_LEN, ValueKey,\n    ahash::AHashMap,\n    write::{\n        AlignedBytes, AnyClass, Archive, Archiver, BatchBuilder, QueueClass, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n        now,\n    },\n};\nuse trc::AddContext;\nuse types::blob_hash::BlobHash;\n\npub(crate) async fn migrate_queue_v011(server: &Server) -> trc::Result<()> {\n    let mut count = 0;\n    let now = now();\n\n    for (queue_id, due) in get_queue_events(server).await? {\n        match server\n            .store()\n            .get_value::<LegacyBincode<MessageV011>>(ValueKey::from(ValueClass::Queue(\n                QueueClass::Message(queue_id),\n            )))\n            .await\n        {\n            Ok(Some(bincoded)) => {\n                let mut batch = BatchBuilder::new();\n                let message = Message::from(bincoded.inner);\n                if let Some(due) = due {\n                    batch.clear(ValueClass::Any(AnyClass {\n                        subspace: SUBSPACE_QUEUE_EVENT,\n                        key: KeySerializer::new(16).write(due).write(queue_id).finalize(),\n                    }));\n                }\n                batch\n                    .set(\n                        ValueClass::Queue(QueueClass::MessageEvent(store::write::QueueEvent {\n                            due: due.unwrap_or(now),\n                            queue_id,\n                            queue_name: DEFAULT_QUEUE_NAME.into_inner(),\n                        })),\n                        vec![],\n                    )\n                    .set(\n                        ValueClass::Queue(QueueClass::Message(queue_id)),\n                        Archiver::new(message)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                count += 1;\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            Ok(None) => {\n                if let Some(due) = due {\n                    let mut batch = BatchBuilder::new();\n                    batch.clear(ValueClass::Any(AnyClass {\n                        subspace: SUBSPACE_QUEUE_EVENT,\n                        key: KeySerializer::new(16).write(due).write(queue_id).finalize(),\n                    }));\n                    server\n                        .store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n            }\n            Err(err) => {\n                if server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Queue(\n                        QueueClass::Message(queue_id),\n                    )))\n                    .await\n                    .is_err()\n                {\n                    return Err(err\n                        .ctx(trc::Key::QueueId, queue_id)\n                        .caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    if count > 0 {\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details = format!(\"Migrated {count} queued messages\",)\n        );\n    }\n\n    Ok(())\n}\n\npub(crate) async fn migrate_queue_v012(server: &Server) -> trc::Result<()> {\n    let mut count = 0;\n    let now = now();\n\n    for (queue_id, due) in get_queue_events(server).await? {\n        match server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Queue(\n                QueueClass::Message(queue_id),\n            )))\n            .await\n            .and_then(|archive| {\n                if let Some(archive) = archive {\n                    archive.deserialize_untrusted::<MessageV012>().map(Some)\n                } else {\n                    Ok(None)\n                }\n            }) {\n            Ok(Some(archive)) => {\n                let message = Message::from(archive);\n                let mut batch = BatchBuilder::new();\n                if let Some(due) = due {\n                    batch.clear(ValueClass::Any(AnyClass {\n                        subspace: SUBSPACE_QUEUE_EVENT,\n                        key: KeySerializer::new(16).write(due).write(queue_id).finalize(),\n                    }));\n                }\n                batch\n                    .set(\n                        ValueClass::Queue(QueueClass::MessageEvent(store::write::QueueEvent {\n                            due: due.unwrap_or(now),\n                            queue_id,\n                            queue_name: DEFAULT_QUEUE_NAME.into_inner(),\n                        })),\n                        vec![],\n                    )\n                    .set(\n                        ValueClass::Queue(QueueClass::Message(queue_id)),\n                        Archiver::new(message)\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                    );\n                count += 1;\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            Ok(None) => {\n                if let Some(due) = due {\n                    let mut batch = BatchBuilder::new();\n                    batch.clear(ValueClass::Any(AnyClass {\n                        subspace: SUBSPACE_QUEUE_EVENT,\n                        key: KeySerializer::new(16).write(due).write(queue_id).finalize(),\n                    }));\n                    server\n                        .store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n            }\n            Err(err) => {\n                if server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Queue(\n                        QueueClass::Message(queue_id),\n                    )))\n                    .await\n                    .and_then(|archive| {\n                        if let Some(archive) = archive {\n                            archive.deserialize_untrusted::<Message>().map(Some)\n                        } else {\n                            Ok(None)\n                        }\n                    })\n                    .is_err()\n                {\n                    return Err(err\n                        .ctx(trc::Key::QueueId, queue_id)\n                        .caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    if count > 0 {\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details = format!(\"Migrated {count} queued messages\",)\n        );\n    }\n\n    Ok(())\n}\n\nasync fn get_queue_events(server: &Server) -> trc::Result<AHashMap<u64, Option<u64>>> {\n    let from_key = ValueKey::from(ValueClass::Queue(QueueClass::MessageEvent(\n        store::write::QueueEvent {\n            due: 0,\n            queue_id: 0,\n            queue_name: [0; 8],\n        },\n    )));\n    let to_key = ValueKey::from(ValueClass::Queue(QueueClass::MessageEvent(\n        store::write::QueueEvent {\n            due: u64::MAX,\n            queue_id: u64::MAX,\n            queue_name: [u8::MAX; 8],\n        },\n    )));\n\n    let mut queue_ids: AHashMap<u64, Option<u64>> = AHashMap::new();\n    server\n        .store()\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending().no_values(),\n            |key, _| {\n                queue_ids.insert(\n                    key.deserialize_be_u64(U64_LEN)?,\n                    Some(key.deserialize_be_u64(0)?),\n                );\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    let from_key = ValueKey::from(ValueClass::Queue(QueueClass::Message(0)));\n    let to_key = ValueKey::from(ValueClass::Queue(QueueClass::Message(u64::MAX)));\n    server\n        .store()\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending().no_values(),\n            |key, _| {\n                let queue_id = key.deserialize_be_u64(0)?;\n\n                if !queue_ids.contains_key(&queue_id) {\n                    queue_ids.insert(queue_id, None);\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(queue_ids)\n}\n\nimpl<SIZE, IDX> From<LegacyMessage<SIZE, IDX>> for Message\nwhere\n    SIZE: AsU64,\n    IDX: AsU64,\n{\n    fn from(message: LegacyMessage<SIZE, IDX>) -> Self {\n        let domains = message.domains;\n        Message {\n            created: message.created,\n            blob_hash: message.blob_hash,\n            return_path: message.return_path_lcase.into_boxed_str(),\n            recipients: message\n                .recipients\n                .into_iter()\n                .map(|r| {\n                    let domain = &domains[r.domain_idx.as_u64() as usize];\n                    let mut rcpt = Recipient::new(r.address);\n                    rcpt.status = match r.status {\n                        Status::Scheduled => match &domain.status {\n                            Status::Scheduled | Status::Completed(_) => Status::Scheduled,\n                            Status::TemporaryFailure(err) => {\n                                Status::TemporaryFailure(migrate_legacy_error(&domain.domain, err))\n                            }\n                            Status::PermanentFailure(err) => {\n                                Status::PermanentFailure(migrate_legacy_error(&domain.domain, err))\n                            }\n                        },\n                        Status::Completed(details) => Status::Completed(HostResponse {\n                            hostname: details.hostname.into_boxed_str(),\n                            response: Response {\n                                code: details.response.code,\n                                esc: details.response.esc,\n                                message: details.response.message.into_boxed_str(),\n                            },\n                        }),\n                        Status::TemporaryFailure(err) => {\n                            Status::TemporaryFailure(migrate_host_response(err))\n                        }\n                        Status::PermanentFailure(err) => {\n                            Status::PermanentFailure(migrate_host_response(err))\n                        }\n                    };\n                    rcpt.flags = r.flags;\n                    rcpt.orcpt = r.orcpt.map(|o| o.into_boxed_str());\n                    rcpt.retry = domain.retry.clone();\n                    rcpt.notify = domain.notify.clone();\n                    rcpt.queue = QueueName::default();\n                    rcpt.expires = QueueExpiry::Ttl(domain.expires.saturating_sub(now()));\n                    rcpt\n                })\n                .collect(),\n            flags: message.flags,\n            env_id: message.env_id.map(|e| e.into_boxed_str()),\n            priority: message.priority,\n            size: message.size.as_u64(),\n            quota_keys: message.quota_keys.into_iter().map(Into::into).collect(),\n            received_from_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),\n            received_via_port: 0,\n        }\n    }\n}\n\ntrait AsU64 {\n    fn as_u64(&self) -> u64;\n}\nimpl AsU64 for usize {\n    fn as_u64(&self) -> u64 {\n        *self as u64\n    }\n}\nimpl AsU64 for u32 {\n    fn as_u64(&self) -> u64 {\n        *self as u64\n    }\n}\nimpl AsU64 for u64 {\n    fn as_u64(&self) -> u64 {\n        *self\n    }\n}\n\nfn migrate_legacy_error(domain: &str, err: &LegacyError) -> ErrorDetails {\n    match err {\n        LegacyError::DnsError(err) => ErrorDetails {\n            entity: domain.into(),\n            details: Error::DnsError(err.as_str().into()),\n        },\n        LegacyError::UnexpectedResponse(err) => ErrorDetails {\n            entity: err.hostname.entity.as_str().into(),\n            details: Error::UnexpectedResponse(UnexpectedResponse {\n                command: err.hostname.details.as_str().into(),\n                response: Response {\n                    code: err.response.code,\n                    esc: err.response.esc,\n                    message: err.response.message.as_str().into(),\n                },\n            }),\n        },\n        LegacyError::ConnectionError(err) => ErrorDetails {\n            entity: err.entity.as_str().into(),\n            details: Error::ConnectionError(err.details.as_str().into()),\n        },\n        LegacyError::TlsError(err) => ErrorDetails {\n            entity: err.entity.as_str().into(),\n            details: Error::TlsError(err.details.as_str().into()),\n        },\n        LegacyError::DaneError(err) => ErrorDetails {\n            entity: err.entity.as_str().into(),\n            details: Error::DaneError(err.details.as_str().into()),\n        },\n        LegacyError::MtaStsError(err) => ErrorDetails {\n            entity: domain.into(),\n            details: Error::MtaStsError(err.as_str().into()),\n        },\n        LegacyError::RateLimited => ErrorDetails {\n            entity: domain.into(),\n            details: Error::RateLimited,\n        },\n        LegacyError::ConcurrencyLimited => ErrorDetails {\n            entity: domain.into(),\n            details: Error::ConcurrencyLimited,\n        },\n        LegacyError::Io(err) => ErrorDetails {\n            entity: domain.into(),\n            details: Error::Io(err.as_str().into()),\n        },\n    }\n}\n\nfn migrate_host_response(response: LegacyHostResponse<LegacyErrorDetails>) -> ErrorDetails {\n    ErrorDetails {\n        entity: response.hostname.entity.into_boxed_str(),\n        details: Error::UnexpectedResponse(UnexpectedResponse {\n            command: response.hostname.details.into_boxed_str(),\n            response: Response {\n                code: response.response.code,\n                esc: response.response.esc,\n                message: response.response.message.into_boxed_str(),\n            },\n        }),\n    }\n}\n\npub type MessageV011 = LegacyMessage<usize, usize>;\npub type MessageV012 = LegacyMessage<u64, u32>;\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct LegacyMessage<SIZE, IDX> {\n    pub queue_id: QueueId,\n    pub created: u64,\n    pub blob_hash: BlobHash,\n\n    pub return_path: String,\n    pub return_path_lcase: String,\n    pub return_path_domain: String,\n    pub recipients: Vec<LegacyRecipient<IDX>>,\n    pub domains: Vec<LegacyDomain>,\n\n    pub flags: u64,\n    pub env_id: Option<String>,\n    pub priority: i16,\n\n    pub size: SIZE,\n    pub quota_keys: Vec<LegacyQuotaKey>,\n\n    #[serde(skip)]\n    #[rkyv(with = rkyv::with::Skip)]\n    pub span_id: u64,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct LegacyRecipient<IDX> {\n    pub domain_idx: IDX,\n    pub address: String,\n    pub address_lcase: String,\n    pub status: Status<LegacyHostResponse<String>, LegacyHostResponse<LegacyErrorDetails>>,\n    pub flags: u64,\n    pub orcpt: Option<String>,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct LegacyDomain {\n    pub domain: String,\n    pub retry: Schedule<u32>,\n    pub notify: Schedule<u32>,\n    pub expires: u64,\n    pub status: Status<(), LegacyError>,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub enum LegacyError {\n    DnsError(String),\n    UnexpectedResponse(LegacyHostResponse<LegacyErrorDetails>),\n    ConnectionError(LegacyErrorDetails),\n    TlsError(LegacyErrorDetails),\n    DaneError(LegacyErrorDetails),\n    MtaStsError(String),\n    RateLimited,\n    ConcurrencyLimited,\n    Io(String),\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct LegacyErrorDetails {\n    pub entity: String,\n    pub details: String,\n}\n"
  },
  {
    "path": "crates/migration/src/queue_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Server,\n    config::smtp::queue::{QueueExpiry, QueueName},\n};\nuse smtp::queue::{\n    Error, ErrorDetails, HostResponse, Message, QuotaKey, Recipient, Schedule, Status,\n    UnexpectedResponse,\n};\nuse smtp_proto::Response;\nuse std::net::IpAddr;\nuse store::{\n    Deserialize, IterateParams, Serialize, ValueKey,\n    write::{\n        AlignedBytes, Archive, Archiver, BatchBuilder, QueueClass, ValueClass,\n        key::DeserializeBigEndian,\n    },\n};\nuse trc::AddContext;\nuse types::blob_hash::BlobHash;\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug, Clone, PartialEq, Eq)]\npub struct LegacyMessage {\n    pub created: u64,\n    pub blob_hash: BlobHash,\n\n    pub return_path: String,\n    pub recipients: Vec<LegacyRecipient>,\n\n    pub received_from_ip: IpAddr,\n    pub received_via_port: u16,\n\n    pub flags: u64,\n    pub env_id: Option<String>,\n    pub priority: i16,\n\n    pub size: u64,\n    pub quota_keys: Vec<LegacyQuotaKey>,\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    serde::Deserialize,\n)]\npub struct LegacyRecipient {\n    pub address: String,\n\n    pub retry: Schedule<u32>,\n    pub notify: Schedule<u32>,\n    pub expires: QueueExpiry,\n\n    pub queue: QueueName,\n    pub status: Status<LegacyHostResponse<String>, LegacyErrorDetails>,\n    pub flags: u64,\n    pub orcpt: Option<String>,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct LegacyHostResponse<T> {\n    pub hostname: T,\n    pub response: Response<String>,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct LegacyUnexpectedResponse {\n    pub command: String,\n    pub response: Response<String>,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Default,\n    serde::Deserialize,\n)]\npub struct LegacyErrorDetails {\n    pub entity: String,\n    pub details: LegacyError,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n    Default,\n)]\npub enum LegacyError {\n    DnsError(String),\n    UnexpectedResponse(LegacyUnexpectedResponse),\n    ConnectionError(String),\n    TlsError(String),\n    DaneError(String),\n    MtaStsError(String),\n    RateLimited,\n    #[default]\n    ConcurrencyLimited,\n    Io(String),\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    serde::Deserialize,\n)]\npub enum LegacyQuotaKey {\n    Size { key: Vec<u8>, id: u64 },\n    Count { key: Vec<u8>, id: u64 },\n}\n\npub(crate) async fn migrate_queue_v014(server: &Server) -> trc::Result<()> {\n    let mut messages = Vec::new();\n    server\n        .store()\n        .iterate(\n            IterateParams::new(\n                ValueKey::from(ValueClass::Queue(QueueClass::Message(0))),\n                ValueKey::from(ValueClass::Queue(QueueClass::Message(u64::MAX))),\n            ),\n            |key, value| {\n                let archive = <Archive<AlignedBytes> as Deserialize>::deserialize(value)\n                    .caused_by(trc::location!())?;\n                match archive.deserialize_untrusted::<LegacyMessage>() {\n                    Ok(message) => {\n                        messages.push((key.deserialize_be_u64(0)?, Message::from(message)));\n                    }\n                    Err(err) => {\n                        if archive.deserialize_untrusted::<Message>().is_err() {\n                            return Err(err.caused_by(trc::location!()));\n                        }\n                    }\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    let mut batch = BatchBuilder::new();\n    let count = messages.len();\n    for (queue_id, message) in messages {\n        batch.set(\n            ValueClass::Queue(QueueClass::Message(queue_id)),\n            Archiver::new(message)\n                .serialize()\n                .caused_by(trc::location!())?,\n        );\n\n        if batch.is_large_batch() {\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n            batch = BatchBuilder::new();\n        }\n    }\n\n    if !batch.is_empty() {\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    trc::event!(\n        Server(trc::ServerEvent::Startup),\n        Details = format!(\"Migrated {count} queued messages\",)\n    );\n\n    Ok(())\n}\n\nimpl From<LegacyMessage> for Message {\n    fn from(legacy: LegacyMessage) -> Self {\n        Message {\n            created: legacy.created,\n            blob_hash: legacy.blob_hash,\n\n            return_path: legacy.return_path.into_boxed_str(),\n            recipients: legacy.recipients.into_iter().map(|r| r.into()).collect(),\n\n            received_from_ip: legacy.received_from_ip,\n            received_via_port: legacy.received_via_port,\n\n            flags: legacy.flags,\n            env_id: legacy.env_id.map(|s| s.into_boxed_str()),\n            priority: legacy.priority,\n\n            size: legacy.size,\n            quota_keys: legacy.quota_keys.into_iter().map(|qk| qk.into()).collect(),\n        }\n    }\n}\n\nimpl From<LegacyRecipient> for Recipient {\n    fn from(legacy: LegacyRecipient) -> Self {\n        Recipient {\n            address: legacy.address.into_boxed_str(),\n            retry: legacy.retry,\n            notify: legacy.notify,\n            expires: legacy.expires,\n            queue: legacy.queue,\n            status: match legacy.status {\n                Status::Scheduled => Status::Scheduled,\n                Status::Completed(status) => Status::Completed(status.into()),\n                Status::TemporaryFailure(status) => Status::TemporaryFailure(status.into()),\n                Status::PermanentFailure(status) => Status::PermanentFailure(status.into()),\n            },\n            flags: legacy.flags,\n            orcpt: legacy.orcpt.map(|s| s.into_boxed_str()),\n        }\n    }\n}\n\nimpl From<LegacyErrorDetails> for ErrorDetails {\n    fn from(legacy: LegacyErrorDetails) -> Self {\n        ErrorDetails {\n            entity: legacy.entity.into_boxed_str(),\n            details: legacy.details.into(),\n        }\n    }\n}\n\nimpl From<LegacyQuotaKey> for QuotaKey {\n    fn from(legacy: LegacyQuotaKey) -> Self {\n        match legacy {\n            LegacyQuotaKey::Size { key, id } => QuotaKey::Size {\n                key: key.into(),\n                id,\n            },\n            LegacyQuotaKey::Count { key, id } => QuotaKey::Count {\n                key: key.into(),\n                id,\n            },\n        }\n    }\n}\n\nimpl From<LegacyError> for Error {\n    fn from(legacy: LegacyError) -> Self {\n        match legacy {\n            LegacyError::DnsError(s) => Error::DnsError(s.into_boxed_str()),\n            LegacyError::UnexpectedResponse(ur) => Error::UnexpectedResponse(ur.into()),\n            LegacyError::ConnectionError(s) => Error::ConnectionError(s.into_boxed_str()),\n            LegacyError::TlsError(s) => Error::TlsError(s.into_boxed_str()),\n            LegacyError::DaneError(s) => Error::DaneError(s.into_boxed_str()),\n            LegacyError::MtaStsError(s) => Error::MtaStsError(s.into_boxed_str()),\n            LegacyError::RateLimited => Error::RateLimited,\n            LegacyError::ConcurrencyLimited => Error::ConcurrencyLimited,\n            LegacyError::Io(s) => Error::Io(s.into_boxed_str()),\n        }\n    }\n}\n\nimpl From<LegacyUnexpectedResponse> for UnexpectedResponse {\n    fn from(legacy: LegacyUnexpectedResponse) -> Self {\n        UnexpectedResponse {\n            command: legacy.command.into_boxed_str(),\n            response: Response {\n                code: legacy.response.code,\n                esc: legacy.response.esc,\n                message: legacy.response.message.into_boxed_str(),\n            },\n        }\n    }\n}\n\nimpl From<LegacyHostResponse<String>> for HostResponse<Box<str>> {\n    fn from(legacy: LegacyHostResponse<String>) -> Self {\n        HostResponse {\n            hostname: legacy.hostname.into_boxed_str(),\n            response: Response {\n                code: legacy.response.code,\n                esc: legacy.response.esc,\n                message: legacy.response.message.into_boxed_str(),\n            },\n        }\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/report.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::LegacyBincode;\nuse common::Server;\nuse mail_auth::report::{Feedback, Report, tlsrpt::TlsReport};\nuse smtp::reporting::analysis::IncomingReport;\nuse store::{\n    IterateParams, SUBSPACE_REPORT_OUT, Serialize, U64_LEN, ValueKey,\n    ahash::AHashSet,\n    write::{\n        AlignedBytes, AnyKey, Archive, Archiver, BatchBuilder, ReportClass, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n    },\n};\nuse trc::AddContext;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\nenum ReportType {\n    Dmarc,\n    Tls,\n    Arf,\n}\n\npub(crate) async fn migrate_reports(server: &Server) -> trc::Result<()> {\n    let mut num_dmarc = 0;\n    let mut num_tls = 0;\n    let mut num_arf = 0;\n\n    for report in [ReportType::Dmarc, ReportType::Tls, ReportType::Arf] {\n        let (from_key, to_key) = match report {\n            ReportType::Dmarc => (\n                ValueKey::from(ValueClass::Report(ReportClass::Dmarc { id: 0, expires: 0 })),\n                ValueKey::from(ValueClass::Report(ReportClass::Dmarc {\n                    id: u64::MAX,\n                    expires: u64::MAX,\n                })),\n            ),\n            ReportType::Tls => (\n                ValueKey::from(ValueClass::Report(ReportClass::Tls { id: 0, expires: 0 })),\n                ValueKey::from(ValueClass::Report(ReportClass::Tls {\n                    id: u64::MAX,\n                    expires: u64::MAX,\n                })),\n            ),\n            ReportType::Arf => (\n                ValueKey::from(ValueClass::Report(ReportClass::Arf { id: 0, expires: 0 })),\n                ValueKey::from(ValueClass::Report(ReportClass::Arf {\n                    id: u64::MAX,\n                    expires: u64::MAX,\n                })),\n            ),\n        };\n\n        let mut results = AHashSet::new();\n\n        server\n            .core\n            .storage\n            .data\n            .iterate(\n                IterateParams::new(from_key, to_key).no_values(),\n                |key, _| {\n                    results.insert((\n                        report,\n                        key.deserialize_be_u64(U64_LEN + 1)?,\n                        key.deserialize_be_u64(1)?,\n                    ));\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        for (report, id, expires) in results {\n            match report {\n                ReportType::Dmarc => {\n                    match server\n                        .store()\n                        .get_value::<LegacyBincode<IncomingReport<Report>>>(ValueKey::from(\n                            ValueClass::Report(ReportClass::Dmarc { id, expires }),\n                        ))\n                        .await\n                    {\n                        Ok(Some(bincoded)) => {\n                            let mut batch = BatchBuilder::new();\n                            batch.set(\n                                ValueClass::Report(ReportClass::Dmarc { id, expires }),\n                                Archiver::new(bincoded.inner)\n                                    .serialize()\n                                    .caused_by(trc::location!())?,\n                            );\n                            num_dmarc += 1;\n                            server\n                                .store()\n                                .write(batch.build_all())\n                                .await\n                                .caused_by(trc::location!())?;\n                        }\n                        Ok(None) => (),\n                        Err(err) => {\n                            if server\n                                .store()\n                                .get_value::<Archive<AlignedBytes>>(ValueKey::from(\n                                    ValueClass::Report(ReportClass::Dmarc { id, expires }),\n                                ))\n                                .await\n                                .is_err()\n                            {\n                                return Err(err.ctx(trc::Key::Id, id).caused_by(trc::location!()));\n                            }\n                        }\n                    }\n                }\n                ReportType::Tls => {\n                    match server\n                        .store()\n                        .get_value::<LegacyBincode<IncomingReport<TlsReport>>>(ValueKey::from(\n                            ValueClass::Report(ReportClass::Tls { id, expires }),\n                        ))\n                        .await\n                    {\n                        Ok(Some(bincoded)) => {\n                            let mut batch = BatchBuilder::new();\n                            batch.set(\n                                ValueClass::Report(ReportClass::Tls { id, expires }),\n                                Archiver::new(bincoded.inner)\n                                    .serialize()\n                                    .caused_by(trc::location!())?,\n                            );\n                            num_tls += 1;\n                            server\n                                .store()\n                                .write(batch.build_all())\n                                .await\n                                .caused_by(trc::location!())?;\n                        }\n                        Ok(None) => (),\n                        Err(err) => {\n                            if server\n                                .store()\n                                .get_value::<Archive<AlignedBytes>>(ValueKey::from(\n                                    ValueClass::Report(ReportClass::Tls { id, expires }),\n                                ))\n                                .await\n                                .is_err()\n                            {\n                                return Err(err.ctx(trc::Key::Id, id).caused_by(trc::location!()));\n                            }\n                        }\n                    }\n                }\n                ReportType::Arf => {\n                    match server\n                        .store()\n                        .get_value::<LegacyBincode<IncomingReport<Feedback>>>(ValueKey::from(\n                            ValueClass::Report(ReportClass::Arf { id, expires }),\n                        ))\n                        .await\n                    {\n                        Ok(Some(bincoded)) => {\n                            let mut batch = BatchBuilder::new();\n                            batch.set(\n                                ValueClass::Report(ReportClass::Arf { id, expires }),\n                                Archiver::new(bincoded.inner)\n                                    .serialize()\n                                    .caused_by(trc::location!())?,\n                            );\n                            num_arf += 1;\n                            server\n                                .store()\n                                .write(batch.build_all())\n                                .await\n                                .caused_by(trc::location!())?;\n                        }\n                        Ok(None) => (),\n                        Err(err) => {\n                            if server\n                                .store()\n                                .get_value::<Archive<AlignedBytes>>(ValueKey::from(\n                                    ValueClass::Report(ReportClass::Arf { id, expires }),\n                                ))\n                                .await\n                                .is_err()\n                            {\n                                return Err(err.ctx(trc::Key::Id, id).caused_by(trc::location!()));\n                            }\n                        }\n                    }\n                }\n            }\n        }\n    }\n\n    // Delete outgoing reports\n    server\n        .store()\n        .delete_range(\n            AnyKey {\n                subspace: SUBSPACE_REPORT_OUT,\n                key: KeySerializer::new(U64_LEN).write(0u8).finalize(),\n            },\n            AnyKey {\n                subspace: SUBSPACE_REPORT_OUT,\n                key: KeySerializer::new(U64_LEN)\n                    .write(&[u8::MAX; 16][..])\n                    .finalize(),\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    if num_dmarc > 0 || num_tls > 0 || num_arf > 0 {\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details =\n                format!(\"Migrated {num_dmarc} DMARC, {num_tls} TLS, and {num_arf} ARF reports\")\n        );\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/migration/src/sieve_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::object::Object;\nuse crate::{\n    get_document_ids,\n    object::{Property, TryFromLegacy, Value},\n    v014::SUBSPACE_BITMAP_TEXT,\n};\nuse common::Server;\nuse email::sieve::{SieveScript, VacationResponse};\nuse store::{\n    SUBSPACE_INDEXES, SUBSPACE_PROPERTY, Serialize, SerializeInfallible, U64_LEN, ValueKey,\n    write::{\n        AlignedBytes, AnyKey, Archive, Archiver, BatchBuilder, ValueClass, key::KeySerializer,\n    },\n};\nuse trc::{AddContext, StoreEvent};\nuse types::{\n    collection::Collection,\n    field::{Field, PrincipalField, SieveField},\n};\n\npub(crate) async fn migrate_sieve_v011(server: &Server, account_id: u32) -> trc::Result<u64> {\n    // Obtain email ids\n    let script_ids = get_document_ids(server, account_id, Collection::SieveScript)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_scripts = script_ids.len();\n    if num_scripts == 0 {\n        return Ok(0);\n    }\n    let mut did_migrate = false;\n\n    // Delete indexes\n    for subspace in [SUBSPACE_INDEXES, SUBSPACE_BITMAP_TEXT] {\n        server\n            .store()\n            .delete_range(\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::SieveScript))\n                        .finalize(),\n                },\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::SieveScript))\n                        .write(&[u8::MAX; 16][..])\n                        .finalize(),\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    for script_id in &script_ids {\n        match server\n            .store()\n            .get_value::<Object<Value>>(ValueKey {\n                account_id,\n                collection: Collection::SieveScript.into(),\n                document_id: script_id,\n                class: ValueClass::Property(Field::ARCHIVE.into()),\n            })\n            .await\n        {\n            Ok(Some(legacy)) => {\n                let is_active = legacy\n                    .get(&Property::IsActive)\n                    .as_bool()\n                    .unwrap_or_default();\n\n                if let Some(script) = SieveScript::try_from_legacy(legacy) {\n                    let mut batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(account_id)\n                        .with_collection(Collection::SieveScript)\n                        .with_document(script_id)\n                        .index(SieveField::Name, script.name.to_lowercase())\n                        .set(\n                            Field::ARCHIVE,\n                            Archiver::new(script)\n                                .serialize()\n                                .caused_by(trc::location!())?,\n                        );\n\n                    if is_active {\n                        batch\n                            .with_collection(Collection::Principal)\n                            .with_document(0)\n                            .set(PrincipalField::ActiveScriptId, script_id.serialize());\n                    }\n\n                    did_migrate = true;\n\n                    server\n                        .store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                } else {\n                    trc::event!(\n                        Store(StoreEvent::DataCorruption),\n                        Details = \"Failed to migrate SieveScript\",\n                        AccountId = account_id,\n                    )\n                }\n            }\n            Ok(None) => (),\n            Err(err) => {\n                if server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey {\n                        account_id,\n                        collection: Collection::SieveScript.into(),\n                        document_id: script_id,\n                        class: ValueClass::Property(Field::ARCHIVE.into()),\n                    })\n                    .await\n                    .is_err()\n                {\n                    return Err(err\n                        .account_id(account_id)\n                        .document_id(script_id)\n                        .caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    // Delete emailIds property\n    server\n        .store()\n        .delete_range(\n            AnyKey {\n                subspace: SUBSPACE_PROPERTY,\n                key: KeySerializer::new(U64_LEN)\n                    .write(account_id)\n                    .write(u8::from(Collection::SieveScript))\n                    .write(u8::from(SieveField::Ids))\n                    .finalize(),\n            },\n            AnyKey {\n                subspace: SUBSPACE_PROPERTY,\n                key: KeySerializer::new(U64_LEN)\n                    .write(account_id)\n                    .write(u8::from(Collection::SieveScript))\n                    .write(u8::from(SieveField::Ids))\n                    .write(&[u8::MAX; 8][..])\n                    .finalize(),\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    // Increment document id counter\n    if did_migrate {\n        server\n            .store()\n            .assign_document_ids(\n                account_id,\n                Collection::SieveScript,\n                script_ids.max().map(|id| id as u64).unwrap_or(num_scripts) + 1,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        Ok(num_scripts)\n    } else {\n        Ok(0)\n    }\n}\n\nimpl TryFromLegacy for SieveScript {\n    fn try_from_legacy(legacy: Object<Value>) -> Option<Self> {\n        let blob_id = legacy.get(&Property::BlobId).as_blob_id()?;\n        Some(SieveScript {\n            name: legacy\n                .get(&Property::Name)\n                .as_string()\n                .unwrap_or_default()\n                .to_string(),\n            blob_hash: blob_id.hash.clone(),\n            size: blob_id.section.as_ref()?.size as u32,\n            vacation_response: VacationResponse::try_from_legacy(legacy),\n        })\n    }\n}\n\nimpl TryFromLegacy for VacationResponse {\n    fn try_from_legacy(legacy: Object<Value>) -> Option<Self> {\n        let vacation = VacationResponse {\n            from_date: legacy\n                .get(&Property::FromDate)\n                .as_date()\n                .map(|s| s.timestamp() as u64),\n            to_date: legacy\n                .get(&Property::ToDate)\n                .as_date()\n                .map(|s| s.timestamp() as u64),\n            subject: legacy\n                .get(&Property::Name)\n                .as_string()\n                .map(|s| s.to_string()),\n            text_body: legacy\n                .get(&Property::TextBody)\n                .as_string()\n                .map(|s| s.to_string()),\n            html_body: legacy\n                .get(&Property::HtmlBody)\n                .as_string()\n                .map(|s| s.to_string()),\n        };\n\n        if vacation.from_date.is_some()\n            || vacation.to_date.is_some()\n            || vacation.subject.is_some()\n            || vacation.text_body.is_some()\n            || vacation.html_body.is_some()\n        {\n            Some(vacation)\n        } else {\n            None\n        }\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/sieve_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse email::sieve::{SieveScript, VacationResponse};\nuse store::{\n    Serialize, SerializeInfallible, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder},\n};\nuse trc::AddContext;\nuse types::{\n    blob_hash::BlobHash,\n    collection::Collection,\n    field::{Field, PrincipalField},\n};\n\nuse crate::get_document_ids;\n\npub(crate) async fn migrate_sieve_v013(server: &Server, account_id: u32) -> trc::Result<u64> {\n    // Obtain email ids\n    let script_ids = get_document_ids(server, account_id, Collection::SieveScript)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_scripts = script_ids.len();\n    if num_scripts == 0 {\n        return Ok(0);\n    }\n    let mut num_migrated = 0;\n\n    for script_id in &script_ids {\n        match server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                account_id,\n                Collection::SieveScript,\n                script_id,\n            ))\n            .await\n        {\n            Ok(Some(legacy)) => match legacy.deserialize_untrusted::<SieveScriptV2>() {\n                Ok(old_sieve) => {\n                    let script = SieveScript {\n                        name: old_sieve.name,\n                        blob_hash: old_sieve.blob_hash,\n                        size: old_sieve.size,\n                        vacation_response: old_sieve.vacation_response,\n                    };\n\n                    let mut batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(account_id)\n                        .with_collection(Collection::SieveScript)\n                        .with_document(script_id)\n                        .unindex(Field::new(0u8), vec![u8::from(old_sieve.is_active)])\n                        .set(\n                            Field::ARCHIVE,\n                            Archiver::new(script)\n                                .serialize()\n                                .caused_by(trc::location!())?,\n                        );\n\n                    if old_sieve.is_active {\n                        batch\n                            .with_account_id(account_id)\n                            .with_collection(Collection::Principal)\n                            .with_document(0)\n                            .set(PrincipalField::ActiveScriptId, script_id.serialize());\n                    }\n                    num_migrated += 1;\n\n                    server\n                        .store()\n                        .write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n                Err(_) => {\n                    if let Err(err) = legacy.deserialize_untrusted::<SieveScript>() {\n                        return Err(err.account_id(script_id).caused_by(trc::location!()));\n                    }\n                }\n            },\n            Ok(None) => (),\n            Err(err) => {\n                return Err(err.account_id(script_id).caused_by(trc::location!()));\n            }\n        }\n    }\n\n    Ok(num_migrated)\n}\n\n#[derive(\n    rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default, Clone, PartialEq, Eq,\n)]\n#[rkyv(derive(Debug))]\npub struct SieveScriptV2 {\n    pub name: String,\n    pub is_active: bool,\n    pub blob_hash: BlobHash,\n    pub size: u32,\n    pub vacation_response: Option<VacationResponse>,\n}\n"
  },
  {
    "path": "crates/migration/src/submission.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::object::Object;\nuse crate::{\n    get_document_ids,\n    object::{FromLegacy, Property, Value},\n    v014::{SUBSPACE_BITMAP_TAG, SUBSPACE_BITMAP_TEXT},\n};\nuse common::Server;\nuse email::submission::{\n    Address, Delivered, DeliveryStatus, EmailSubmission, Envelope, UndoStatus,\n};\nuse store::{\n    SUBSPACE_INDEXES, Serialize, U32_LEN, U64_LEN, ValueKey,\n    write::{\n        AlignedBytes, AnyKey, Archive, Archiver, BatchBuilder, IndexPropertyClass, ValueClass,\n        key::KeySerializer,\n    },\n};\nuse trc::AddContext;\nuse types::{\n    collection::Collection,\n    field::{EmailSubmissionField, Field},\n};\nuse utils::map::vec_map::VecMap;\n\npub(crate) async fn migrate_email_submissions(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<u64> {\n    // Obtain email ids\n    let email_submission_ids = get_document_ids(server, account_id, Collection::EmailSubmission)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_email_submissions = email_submission_ids.len();\n    if num_email_submissions == 0 {\n        return Ok(0);\n    }\n    let mut did_migrate = false;\n\n    // Delete indexes\n    for subspace in [SUBSPACE_INDEXES, SUBSPACE_BITMAP_TAG, SUBSPACE_BITMAP_TEXT] {\n        server\n            .store()\n            .delete_range(\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::EmailSubmission))\n                        .finalize(),\n                },\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U64_LEN)\n                        .write(account_id)\n                        .write(u8::from(Collection::EmailSubmission))\n                        .write(&[u8::MAX; 16][..])\n                        .finalize(),\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    for email_submission_id in &email_submission_ids {\n        match server\n            .store()\n            .get_value::<Object<Value>>(ValueKey {\n                account_id,\n                collection: Collection::EmailSubmission.into(),\n                document_id: email_submission_id,\n                class: ValueClass::Property(Field::ARCHIVE.into()),\n            })\n            .await\n        {\n            Ok(Some(legacy)) => {\n                let es = EmailSubmission::from_legacy(legacy);\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_account_id(account_id)\n                    .with_collection(Collection::EmailSubmission)\n                    .with_document(email_submission_id)\n                    .set(\n                        ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                            property: EmailSubmissionField::Metadata.into(),\n                            value: es.send_at,\n                        }),\n                        KeySerializer::new(U32_LEN * 3 + 1)\n                            .write(es.email_id)\n                            .write(es.thread_id)\n                            .write(es.identity_id)\n                            .write(es.undo_status.as_index())\n                            .finalize(),\n                    )\n                    .set(\n                        Field::ARCHIVE,\n                        Archiver::new(es).serialize().caused_by(trc::location!())?,\n                    );\n                did_migrate = true;\n\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            Ok(None) => (),\n            Err(err) => {\n                if server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey {\n                        account_id,\n                        collection: Collection::EmailSubmission.into(),\n                        document_id: email_submission_id,\n                        class: ValueClass::Property(Field::ARCHIVE.into()),\n                    })\n                    .await\n                    .is_err()\n                {\n                    return Err(err\n                        .account_id(account_id)\n                        .document_id(email_submission_id)\n                        .caused_by(trc::location!()));\n                }\n            }\n        }\n    }\n\n    // Increment document id counter\n    if did_migrate {\n        server\n            .store()\n            .assign_document_ids(\n                account_id,\n                Collection::EmailSubmission,\n                email_submission_ids\n                    .max()\n                    .map(|id| id as u64)\n                    .unwrap_or(num_email_submissions)\n                    + 1,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        Ok(num_email_submissions)\n    } else {\n        Ok(0)\n    }\n}\n\nimpl FromLegacy for EmailSubmission {\n    fn from_legacy(legacy: Object<Value>) -> Self {\n        EmailSubmission {\n            email_id: legacy.get(&Property::EmailId).as_uint().unwrap_or_default() as u32,\n            thread_id: legacy\n                .get(&Property::ThreadId)\n                .as_uint()\n                .unwrap_or_default() as u32,\n            identity_id: legacy\n                .get(&Property::IdentityId)\n                .as_uint()\n                .unwrap_or_default() as u32,\n            send_at: legacy\n                .get(&Property::SentAt)\n                .as_date()\n                .map(|s| s.timestamp() as u64)\n                .unwrap_or_default(),\n            queue_id: legacy.get(&Property::MessageId).as_uint(),\n            undo_status: legacy\n                .get(&Property::UndoStatus)\n                .as_string()\n                .and_then(UndoStatus::parse)\n                .unwrap_or(UndoStatus::Final),\n            envelope: convert_envelope(legacy.get(&Property::Envelope)),\n            delivery_status: convert_delivery_status(legacy.get(&Property::DeliveryStatus)),\n        }\n    }\n}\n\nfn convert_delivery_status(value: &Value) -> VecMap<String, DeliveryStatus> {\n    let mut status = VecMap::new();\n    if let Value::List(list) = value {\n        for value in list {\n            if let Value::Object(obj) = value {\n                for (k, v) in obj.properties.iter() {\n                    if let (Property::_T(k), Value::Object(v)) = (k, v) {\n                        let mut delivery_status = DeliveryStatus {\n                            smtp_reply: String::new(),\n                            delivered: Delivered::Unknown,\n                            displayed: false,\n                        };\n\n                        for (property, value) in &v.properties {\n                            match (property, value) {\n                                (Property::Delivered, Value::Text(v)) => match v.as_str() {\n                                    \"queued\" => delivery_status.delivered = Delivered::Queued,\n                                    \"yes\" => delivery_status.delivered = Delivered::Yes,\n                                    \"unknown\" => delivery_status.delivered = Delivered::Unknown,\n                                    \"no\" => delivery_status.delivered = Delivered::No,\n                                    _ => {}\n                                },\n                                (Property::SmtpReply, Value::Text(v)) => {\n                                    delivery_status.smtp_reply = v.to_string();\n                                }\n\n                                _ => {}\n                            }\n                        }\n\n                        status.append(k.to_string(), delivery_status);\n                    }\n                }\n            }\n        }\n    }\n    status\n}\n\nfn convert_envelope(value: &Value) -> Envelope {\n    let mut envelope = Envelope {\n        mail_from: Default::default(),\n        rcpt_to: vec![],\n    };\n\n    if let Value::Object(obj) = value {\n        for (property, value) in &obj.properties {\n            match (property, value) {\n                (Property::MailFrom, _) => {\n                    envelope.mail_from = convert_envelope_address(value).unwrap_or_default();\n                }\n                (Property::RcptTo, Value::List(value)) => {\n                    for addr in value {\n                        if let Some(addr) = convert_envelope_address(addr) {\n                            envelope.rcpt_to.push(addr);\n                        }\n                    }\n                }\n                _ => {}\n            }\n        }\n    }\n\n    envelope\n}\n\nfn convert_envelope_address(envelope: &Value) -> Option<Address> {\n    if let Value::Object(envelope) = envelope\n        && let (Value::Text(email), Value::Object(params)) = (\n            envelope.get(&Property::Email),\n            envelope.get(&Property::Parameters),\n        )\n    {\n        let mut addr = Address {\n            email: email.to_string(),\n            parameters: None,\n        };\n        for (k, v) in params.properties.iter() {\n            if let Property::_T(k) = &k\n                && !k.is_empty()\n            {\n                let k = k.to_string();\n                let v = v.as_string().map(|s| s.to_string());\n\n                addr.parameters.get_or_insert_default().append(k, v);\n            }\n        }\n        return Some(addr);\n    }\n\n    None\n}\n"
  },
  {
    "path": "crates/migration/src/tasks_v1.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse store::{\n    IterateParams, SUBSPACE_TASK_QUEUE, U64_LEN, ValueKey,\n    write::{\n        AnyClass, BatchBuilder, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n        now,\n    },\n};\nuse trc::AddContext;\n\npub(crate) async fn migrate_tasks_v011(server: &Server) -> trc::Result<()> {\n    let from_key = ValueKey::<ValueClass> {\n        account_id: 0,\n        collection: 0,\n        document_id: 0,\n        class: ValueClass::Any(AnyClass {\n            subspace: SUBSPACE_TASK_QUEUE,\n            key: KeySerializer::new(U64_LEN).write(0u64).finalize(),\n        }),\n    };\n    let to_key = ValueKey::<ValueClass> {\n        account_id: u32::MAX,\n        collection: u8::MAX,\n        document_id: u32::MAX,\n        class: ValueClass::Any(AnyClass {\n            subspace: SUBSPACE_TASK_QUEUE,\n            key: KeySerializer::new(U64_LEN).write(u64::MAX).finalize(),\n        }),\n    };\n\n    let now = now();\n    let mut migrate_tasks = Vec::new();\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let due = key.deserialize_be_u64(0)?;\n\n                if due > now {\n                    migrate_tasks.push((key.to_vec(), value.to_vec()));\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    if !migrate_tasks.is_empty() {\n        let num_migrated = migrate_tasks.len();\n        let mut batch = BatchBuilder::new();\n        for (key, value) in migrate_tasks {\n            let mut new_key = key.clone();\n            new_key[0..8].copy_from_slice(&now.to_be_bytes());\n\n            batch\n                .clear(ValueClass::Any(AnyClass {\n                    subspace: SUBSPACE_TASK_QUEUE,\n                    key,\n                }))\n                .set(\n                    ValueClass::Any(AnyClass {\n                        subspace: SUBSPACE_TASK_QUEUE,\n                        key: new_key,\n                    }),\n                    value,\n                );\n        }\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n\n        trc::event!(\n            Server(trc::ServerEvent::Startup),\n            Details = format!(\"Migrated {num_migrated} tasks\")\n        );\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/migration/src/tasks_v2.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse store::{\n    IterateParams, SUBSPACE_TASK_QUEUE, U32_LEN, U64_LEN, ValueKey,\n    write::{\n        AnyClass, BatchBuilder, TaskEpoch, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n    },\n};\nuse trc::AddContext;\n\npub(crate) async fn migrate_tasks_v014(server: &Server) -> trc::Result<()> {\n    let from_key = ValueKey::<ValueClass> {\n        account_id: 0,\n        collection: 0,\n        document_id: 0,\n        class: ValueClass::Any(AnyClass {\n            subspace: SUBSPACE_TASK_QUEUE,\n            key: KeySerializer::new(U64_LEN).write(0u64).finalize(),\n        }),\n    };\n    let to_key = ValueKey::<ValueClass> {\n        account_id: u32::MAX,\n        collection: u8::MAX,\n        document_id: u32::MAX,\n        class: ValueClass::Any(AnyClass {\n            subspace: SUBSPACE_TASK_QUEUE,\n            key: KeySerializer::new(U64_LEN).write(u64::MAX).finalize(),\n        }),\n    };\n\n    let mut delete_tasks = Vec::new();\n    let mut insert_tasks = Vec::new();\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                match key.get(U64_LEN + U32_LEN) {\n                    Some(0..=2) => {\n                        delete_tasks.push(key.to_vec());\n                    }\n                    None => {\n                        return Err(trc::Error::corrupted_key(key, None, trc::location!()));\n                    }\n                    _ => {\n                        let due = key.deserialize_be_u64(0)?;\n                        let maybe_epoch = TaskEpoch::from_inner(due);\n                        if maybe_epoch.attempt() != 0 {\n                            delete_tasks.push(key.to_vec());\n                            let epoch = TaskEpoch::new(due).inner();\n                            let mut new_key = Vec::with_capacity(key.len());\n                            new_key.extend_from_slice(&epoch.to_be_bytes());\n                            new_key.extend_from_slice(&key[U64_LEN..]);\n                            insert_tasks.push((new_key, value.to_vec()));\n                        }\n                    }\n                };\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    let num_migrated = delete_tasks.len() + insert_tasks.len();\n    if num_migrated != 0 {\n        let mut batch = BatchBuilder::new();\n        let mut batch_len = 0;\n        for (key, value) in insert_tasks {\n            batch_len += key.len() + value.len();\n            batch.set(\n                ValueClass::Any(AnyClass {\n                    subspace: SUBSPACE_TASK_QUEUE,\n                    key,\n                }),\n                value,\n            );\n            if batch_len > 4 * 1024 * 1024 {\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                batch = BatchBuilder::new();\n                batch_len = 0;\n            }\n        }\n\n        for key in delete_tasks {\n            batch_len += key.len();\n            batch.clear(ValueClass::Any(AnyClass {\n                subspace: SUBSPACE_TASK_QUEUE,\n                key,\n            }));\n            if batch_len > 4 * 1024 * 1024 {\n                server\n                    .store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                batch = BatchBuilder::new();\n                batch_len = 0;\n            }\n        }\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    trc::event!(\n        Server(trc::ServerEvent::Startup),\n        Details = format!(\"Migrated {num_migrated} tasks\")\n    );\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/migration/src/threads.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\nuse store::{\n    U64_LEN,\n    write::{AnyKey, key::KeySerializer},\n};\nuse trc::AddContext;\nuse types::collection::Collection;\n\nuse crate::{get_document_ids, v014::SUBSPACE_BITMAP_ID};\n\npub(crate) async fn migrate_threads(server: &Server, account_id: u32) -> trc::Result<u64> {\n    // Obtain email ids\n    let thread_ids = get_document_ids(server, account_id, Collection::Thread)\n        .await\n        .caused_by(trc::location!())?\n        .unwrap_or_default();\n    let num_threads = thread_ids.len();\n    if num_threads == 0 {\n        return Ok(0);\n    }\n\n    // Delete threads\n    server\n        .store()\n        .delete_range(\n            AnyKey {\n                subspace: SUBSPACE_BITMAP_ID,\n                key: KeySerializer::new(U64_LEN)\n                    .write(account_id)\n                    .write(u8::from(Collection::Thread))\n                    .finalize(),\n            },\n            AnyKey {\n                subspace: SUBSPACE_BITMAP_ID,\n                key: KeySerializer::new(U64_LEN)\n                    .write(account_id)\n                    .write(u8::from(Collection::Thread))\n                    .write(&[u8::MAX; 16][..])\n                    .finalize(),\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    // Increment document id counter\n    server\n        .store()\n        .assign_document_ids(\n            account_id,\n            Collection::Thread,\n            thread_ids.max().map(|id| id as u64).unwrap_or(num_threads) + 1,\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(num_threads)\n}\n"
  },
  {
    "path": "crates/migration/src/v011.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    LOCK_RETRY_TIME, LOCK_WAIT_TIME_ACCOUNT, LOCK_WAIT_TIME_CORE,\n    changelog::reset_changelog,\n    get_document_ids,\n    principal_v1::{migrate_principal_v0_11, migrate_principals_v0_11},\n    queue_v1::migrate_queue_v011,\n    report::migrate_reports,\n};\nuse common::{KV_LOCK_HOUSEKEEPER, Server};\nuse store::{\n    dispatch::lookup::KeyValue,\n    rand::{self, seq::SliceRandom},\n};\nuse trc::AddContext;\nuse types::collection::Collection;\n\npub(crate) async fn migrate_v0_11(server: &Server) -> trc::Result<()> {\n    let force_lock = std::env::var(\"FORCE_LOCK\").is_ok();\n    let in_memory = server.in_memory_store();\n    let principal_ids;\n\n    loop {\n        if force_lock\n            || in_memory\n                .try_lock(\n                    KV_LOCK_HOUSEKEEPER,\n                    b\"migrate_core_lock\",\n                    LOCK_WAIT_TIME_CORE,\n                )\n                .await\n                .caused_by(trc::location!())?\n        {\n            if in_memory\n                .key_get::<()>(KeyValue::<()>::build_key(\n                    KV_LOCK_HOUSEKEEPER,\n                    b\"migrate_core_done\",\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .is_none()\n            {\n                migrate_queue_v011(server)\n                    .await\n                    .caused_by(trc::location!())?;\n                migrate_reports(server).await.caused_by(trc::location!())?;\n                reset_changelog(server).await.caused_by(trc::location!())?;\n                principal_ids = migrate_principals_v0_11(server)\n                    .await\n                    .caused_by(trc::location!())?;\n\n                in_memory\n                    .key_set(\n                        KeyValue::new(\n                            KeyValue::<()>::build_key(KV_LOCK_HOUSEKEEPER, b\"migrate_core_done\"),\n                            b\"1\".to_vec(),\n                        )\n                        .expires(86400),\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n            } else {\n                principal_ids = get_document_ids(server, u32::MAX, Collection::Principal)\n                    .await\n                    .caused_by(trc::location!())?\n                    .unwrap_or_default();\n\n                trc::event!(\n                    Server(trc::ServerEvent::Startup),\n                    Details = format!(\"Migration completed by another node.\",)\n                );\n            }\n\n            in_memory\n                .remove_lock(KV_LOCK_HOUSEKEEPER, b\"migrate_core_lock\")\n                .await\n                .caused_by(trc::location!())?;\n            break;\n        } else {\n            trc::event!(\n                Server(trc::ServerEvent::Startup),\n                Details = format!(\"Migration lock busy, waiting 30 seconds.\",)\n            );\n\n            tokio::time::sleep(LOCK_RETRY_TIME).await;\n        }\n    }\n\n    if !principal_ids.is_empty() {\n        let mut principal_ids = principal_ids.into_iter().collect::<Vec<_>>();\n        principal_ids.shuffle(&mut rand::rng());\n\n        loop {\n            let mut skipped_principal_ids = Vec::new();\n            let mut num_migrated = 0;\n\n            for principal_id in principal_ids {\n                let lock_key = format!(\"migrate_{principal_id}_lock\");\n                let done_key = format!(\"migrate_{principal_id}_done\");\n\n                if force_lock\n                    || in_memory\n                        .try_lock(\n                            KV_LOCK_HOUSEKEEPER,\n                            lock_key.as_bytes(),\n                            LOCK_WAIT_TIME_ACCOUNT,\n                        )\n                        .await\n                        .caused_by(trc::location!())?\n                {\n                    if in_memory\n                        .key_get::<()>(KeyValue::<()>::build_key(\n                            KV_LOCK_HOUSEKEEPER,\n                            done_key.as_bytes(),\n                        ))\n                        .await\n                        .caused_by(trc::location!())?\n                        .is_none()\n                    {\n                        migrate_principal_v0_11(server, principal_id)\n                            .await\n                            .caused_by(trc::location!())?;\n\n                        num_migrated += 1;\n\n                        in_memory\n                            .key_set(\n                                KeyValue::new(\n                                    KeyValue::<()>::build_key(\n                                        KV_LOCK_HOUSEKEEPER,\n                                        done_key.as_bytes(),\n                                    ),\n                                    b\"1\".to_vec(),\n                                )\n                                .expires(86400),\n                            )\n                            .await\n                            .caused_by(trc::location!())?;\n                    }\n\n                    in_memory\n                        .remove_lock(KV_LOCK_HOUSEKEEPER, lock_key.as_bytes())\n                        .await\n                        .caused_by(trc::location!())?;\n                } else {\n                    skipped_principal_ids.push(principal_id);\n                }\n            }\n\n            if !skipped_principal_ids.is_empty() {\n                trc::event!(\n                    Server(trc::ServerEvent::Startup),\n                    Details = format!(\n                        \"Migrated {num_migrated} accounts and {} are locked by another node, waiting 30 seconds.\",\n                        skipped_principal_ids.len()\n                    )\n                );\n                tokio::time::sleep(LOCK_RETRY_TIME).await;\n                principal_ids = skipped_principal_ids;\n            } else {\n                trc::event!(\n                    Server(trc::ServerEvent::Startup),\n                    Details = format!(\"Account migration completed.\",)\n                );\n                break;\n            }\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/migration/src/v012.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    LOCK_RETRY_TIME, LOCK_WAIT_TIME_CORE, event_v1::migrate_calendar_events_v012,\n    queue_v1::migrate_queue_v012, tasks_v1::migrate_tasks_v011,\n};\nuse common::{KV_LOCK_HOUSEKEEPER, Server};\nuse trc::AddContext;\n\npub(crate) async fn migrate_v0_12(server: &Server, migrate_tasks: bool) -> trc::Result<()> {\n    let force_lock = std::env::var(\"FORCE_LOCK\").is_ok();\n    let in_memory = server.in_memory_store();\n\n    loop {\n        if force_lock\n            || in_memory\n                .try_lock(\n                    KV_LOCK_HOUSEKEEPER,\n                    b\"migrate_core_lock\",\n                    LOCK_WAIT_TIME_CORE,\n                )\n                .await\n                .caused_by(trc::location!())?\n        {\n            migrate_queue_v012(server)\n                .await\n                .caused_by(trc::location!())?;\n\n            if migrate_tasks {\n                migrate_tasks_v011(server)\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n\n            in_memory\n                .remove_lock(KV_LOCK_HOUSEKEEPER, b\"migrate_core_lock\")\n                .await\n                .caused_by(trc::location!())?;\n            break;\n        } else {\n            trc::event!(\n                Server(trc::ServerEvent::Startup),\n                Details = format!(\"Migration lock busy, waiting 30 seconds.\",)\n            );\n\n            tokio::time::sleep(LOCK_RETRY_TIME).await;\n        }\n    }\n\n    if migrate_tasks {\n        migrate_calendar_events_v012(server)\n            .await\n            .caused_by(trc::location!())\n    } else {\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/migration/src/v013.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    LOCK_RETRY_TIME, LOCK_WAIT_TIME_ACCOUNT, LOCK_WAIT_TIME_CORE, get_document_ids,\n    principal_v2::{migrate_principal_v0_13, migrate_principals_v0_13},\n};\nuse common::{KV_LOCK_HOUSEKEEPER, Server};\nuse store::{\n    dispatch::lookup::KeyValue,\n    rand::{self, seq::SliceRandom},\n};\nuse trc::AddContext;\nuse types::collection::Collection;\n\npub(crate) async fn migrate_v0_13(server: &Server) -> trc::Result<()> {\n    let force_lock = std::env::var(\"FORCE_LOCK\").is_ok();\n    let in_memory = server.in_memory_store();\n    let principal_ids;\n\n    loop {\n        if force_lock\n            || in_memory\n                .try_lock(\n                    KV_LOCK_HOUSEKEEPER,\n                    b\"migrate_core_lock\",\n                    LOCK_WAIT_TIME_CORE,\n                )\n                .await\n                .caused_by(trc::location!())?\n        {\n            if in_memory\n                .key_get::<()>(KeyValue::<()>::build_key(\n                    KV_LOCK_HOUSEKEEPER,\n                    b\"migrate_core_done\",\n                ))\n                .await\n                .caused_by(trc::location!())?\n                .is_none()\n            {\n                principal_ids = migrate_principals_v0_13(server)\n                    .await\n                    .caused_by(trc::location!())?;\n\n                in_memory\n                    .key_set(\n                        KeyValue::new(\n                            KeyValue::<()>::build_key(KV_LOCK_HOUSEKEEPER, b\"migrate_core_done\"),\n                            b\"1\".to_vec(),\n                        )\n                        .expires(86400),\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n            } else {\n                principal_ids = get_document_ids(server, u32::MAX, Collection::Principal)\n                    .await\n                    .caused_by(trc::location!())?\n                    .unwrap_or_default();\n\n                trc::event!(\n                    Server(trc::ServerEvent::Startup),\n                    Details = format!(\"Migration completed by another node.\",)\n                );\n            }\n\n            in_memory\n                .remove_lock(KV_LOCK_HOUSEKEEPER, b\"migrate_core_lock\")\n                .await\n                .caused_by(trc::location!())?;\n            break;\n        } else {\n            trc::event!(\n                Server(trc::ServerEvent::Startup),\n                Details = format!(\"Migration lock busy, waiting 30 seconds.\",)\n            );\n\n            tokio::time::sleep(LOCK_RETRY_TIME).await;\n        }\n    }\n\n    if !principal_ids.is_empty() {\n        let mut principal_ids = principal_ids.into_iter().collect::<Vec<_>>();\n        principal_ids.shuffle(&mut rand::rng());\n\n        loop {\n            let mut skipped_principal_ids = Vec::new();\n            let mut num_migrated = 0;\n\n            for principal_id in principal_ids {\n                let lock_key = format!(\"migrate_{principal_id}_lock\");\n                let done_key = format!(\"migrate_{principal_id}_done\");\n\n                if force_lock\n                    || in_memory\n                        .try_lock(\n                            KV_LOCK_HOUSEKEEPER,\n                            lock_key.as_bytes(),\n                            LOCK_WAIT_TIME_ACCOUNT,\n                        )\n                        .await\n                        .caused_by(trc::location!())?\n                {\n                    if in_memory\n                        .key_get::<()>(KeyValue::<()>::build_key(\n                            KV_LOCK_HOUSEKEEPER,\n                            done_key.as_bytes(),\n                        ))\n                        .await\n                        .caused_by(trc::location!())?\n                        .is_none()\n                    {\n                        migrate_principal_v0_13(server, principal_id)\n                            .await\n                            .caused_by(trc::location!())?;\n\n                        num_migrated += 1;\n\n                        in_memory\n                            .key_set(\n                                KeyValue::new(\n                                    KeyValue::<()>::build_key(\n                                        KV_LOCK_HOUSEKEEPER,\n                                        done_key.as_bytes(),\n                                    ),\n                                    b\"1\".to_vec(),\n                                )\n                                .expires(86400),\n                            )\n                            .await\n                            .caused_by(trc::location!())?;\n                    }\n\n                    in_memory\n                        .remove_lock(KV_LOCK_HOUSEKEEPER, lock_key.as_bytes())\n                        .await\n                        .caused_by(trc::location!())?;\n                } else {\n                    skipped_principal_ids.push(principal_id);\n                }\n            }\n\n            if !skipped_principal_ids.is_empty() {\n                trc::event!(\n                    Server(trc::ServerEvent::Startup),\n                    Details = format!(\n                        \"Migrated {num_migrated} accounts and {} are locked by another node, waiting 30 seconds.\",\n                        skipped_principal_ids.len()\n                    )\n                );\n                tokio::time::sleep(LOCK_RETRY_TIME).await;\n                principal_ids = skipped_principal_ids;\n            } else {\n                trc::event!(\n                    Server(trc::ServerEvent::Startup),\n                    Details = format!(\"Account migration completed.\",)\n                );\n                break;\n            }\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/migration/src/v014.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    blob::migrate_blobs_v014, email_v2::migrate_emails_v014,\n    encryption_v2::migrate_encryption_params_v014, queue_v2::migrate_queue_v014,\n    tasks_v2::migrate_tasks_v014,\n};\nuse common::Server;\nuse directory::backend::internal::manage::ManageDirectory;\nuse email::submission::EmailSubmission;\nuse groupware::{calendar::CalendarEventNotification, contact::ContactCard};\nuse std::sync::Arc;\nuse store::{\n    SUBSPACE_INDEXES, SerializeInfallible, U32_LEN, U64_LEN,\n    rand::{self, seq::SliceRandom},\n    write::{\n        AnyKey, BatchBuilder, IndexPropertyClass, Operation, ValueClass, ValueOp,\n        key::KeySerializer,\n    },\n};\nuse tokio::sync::Semaphore;\nuse trc::AddContext;\nuse types::{\n    collection::Collection,\n    field::{CalendarNotificationField, ContactField, EmailSubmissionField, IdentityField},\n};\n\npub const SUBSPACE_BITMAP_ID: u8 = b'b';\npub const SUBSPACE_BITMAP_TAG: u8 = b'c';\npub const SUBSPACE_BITMAP_TEXT: u8 = b'v';\npub const SUBSPACE_FTS_INDEX: u8 = b'g';\npub const SUBSPACE_TELEMETRY_INDEX: u8 = b'w';\n\npub async fn migrate_v0_14(server: &Server) -> trc::Result<()> {\n    // Migrate global data\n    let mut tasks = Vec::new();\n    let _server = server.clone();\n    tasks.push(tokio::spawn(\n        async move { migrate_queue_v014(&_server).await },\n    ));\n    let _server = server.clone();\n    tasks.push(tokio::spawn(\n        async move { migrate_blobs_v014(&_server).await },\n    ));\n    let _server = server.clone();\n    tasks.push(tokio::spawn(\n        async move { migrate_tasks_v014(&_server).await },\n    ));\n    futures::future::join_all(tasks)\n        .await\n        .into_iter()\n        .collect::<Result<trc::Result<()>, _>>()\n        .map_err(|err| {\n            trc::EventType::Server(trc::ServerEvent::ThreadError)\n                .reason(err)\n                .caused_by(trc::location!())\n                .details(\"Join Error\")\n        })??;\n\n    // Migrate account data\n    let mut principal_ids = server\n        .store()\n        .principal_ids(None, None)\n        .await\n        .unwrap_or_default()\n        .into_iter()\n        .collect::<Vec<_>>();\n    principal_ids.shuffle(&mut rand::rng());\n    let semaphore = Arc::new(Semaphore::new(\n        std::env::var(\"NUM_THREADS\")\n            .ok()\n            .and_then(|s| s.parse::<usize>().ok())\n            .unwrap_or_else(|| num_cpus::get().min(2) * 2),\n    ));\n    let mut tasks = Vec::with_capacity(principal_ids.len());\n    let num_principals = principal_ids.len();\n    for principal_id in principal_ids {\n        let permit = semaphore.clone().acquire_owned().await.unwrap();\n        let _server = server.clone();\n        tasks.push(tokio::spawn(async move {\n            let result = migrate_principal_v0_14(&_server, principal_id).await;\n            drop(permit);\n            result\n        }));\n    }\n    futures::future::join_all(tasks)\n        .await\n        .into_iter()\n        .collect::<Result<trc::Result<()>, _>>()\n        .map_err(|err| {\n            trc::EventType::Server(trc::ServerEvent::ThreadError)\n                .reason(err)\n                .caused_by(trc::location!())\n                .details(\"Join Error\")\n        })??;\n\n    trc::event!(\n        Server(trc::ServerEvent::Startup),\n        Details = format!(\"Migrated {num_principals} accounts\")\n    );\n\n    // Delete old subspaces\n    for subspace in [\n        SUBSPACE_BITMAP_ID,\n        SUBSPACE_BITMAP_TAG,\n        SUBSPACE_BITMAP_TEXT,\n        SUBSPACE_FTS_INDEX,\n        SUBSPACE_TELEMETRY_INDEX,\n    ] {\n        server\n            .store()\n            .delete_range(\n                AnyKey {\n                    subspace,\n                    key: vec![0u8],\n                },\n                AnyKey {\n                    subspace,\n                    key: vec![u8::MAX; 32],\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    trc::event!(\n        Server(trc::ServerEvent::Startup),\n        Details = format!(\"Migration to v0.15 completed\")\n    );\n\n    Ok(())\n}\n\npub(crate) async fn migrate_principal_v0_14(server: &Server, account_id: u32) -> trc::Result<()> {\n    let emails = migrate_emails_v014(server, account_id).await?;\n    let params = migrate_encryption_params_v014(server, account_id).await?;\n    let (num_contacts, num_calendars, num_email_submissions, num_identities) =\n        migrate_indexes(server, account_id).await?;\n\n    trc::event!(\n        Server(trc::ServerEvent::Startup),\n        Details = format!(\n            \"Migrated account {account_id}: {emails} emails, {params} encryption params, {num_contacts} contacts, {num_calendars} calendars, {num_email_submissions} submissions, and {num_identities} identities\"\n        )\n    );\n\n    Ok(())\n}\n\npub(crate) async fn migrate_indexes(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<(usize, usize, usize, usize)> {\n    /*\n\n           EmailSubmissionField::UndoStatus => 41,\n           EmailSubmissionField::EmailId => 83,\n           EmailSubmissionField::ThreadId => 33,\n           EmailSubmissionField::IdentityId => 95,\n           EmailSubmissionField::SendAt => 24,\n\n    */\n\n    /*\n\n           ContactField::Created => 2,\n           ContactField::Updated => 3,\n           ContactField::Text => 4,\n    */\n\n    /*\n\n           CalendarField::Text => 1,\n           CalendarField::Created => 2,\n           CalendarField::Updated => 3,\n           CalendarField::Start => 4,\n           CalendarField::EventId => 5,\n    */\n\n    /*\n\n            EmailField::From => 87,\n            EmailField::To => 35,\n            EmailField::Cc => 74,\n            EmailField::Bcc => 69,\n            EmailField::Subject => 29,\n            EmailField::Size => 27,\n            EmailField::References => 20,\n            EmailField::MailboxIds => 7,\n            EmailField::ReceivedAt => 19,\n            EmailField::SentAt => 26,\n            EmailField::HasAttachment => 89,\n\n    */\n\n    for (collection, fields) in [\n        (\n            Collection::Email,\n            &[87u8, 35, 74, 69, 29, 27, 20, 7, 19, 26, 89][..],\n        ),\n        (Collection::EmailSubmission, &[41, 83, 33, 95, 24][..]),\n        (Collection::ContactCard, &[1, 2, 3, 4][..]),\n        (Collection::CalendarEvent, &[1, 2, 3, 4][..]),\n        (Collection::CalendarEventNotification, &[2, 5][..]),\n    ] {\n        for index in fields {\n            server\n                .store()\n                .delete_range(\n                    AnyKey {\n                        subspace: SUBSPACE_INDEXES,\n                        key: KeySerializer::new(U64_LEN * 3)\n                            .write(account_id)\n                            .write(u8::from(collection))\n                            .write(*index)\n                            .finalize(),\n                    },\n                    AnyKey {\n                        subspace: SUBSPACE_INDEXES,\n                        key: KeySerializer::new(U64_LEN * 4)\n                            .write(account_id)\n                            .write(u8::from(collection))\n                            .write(*index)\n                            .write(&[u8::MAX; 8][..])\n                            .finalize(),\n                    },\n                )\n                .await\n                .caused_by(trc::location!())?;\n        }\n    }\n\n    let mut indexes = Vec::new();\n    let mut num_contacts = 0;\n    let mut num_calendars = 0;\n    let mut num_email_submissions = 0;\n    let mut num_identities = 0;\n    for collection in [\n        Collection::ContactCard,\n        Collection::CalendarEventNotification,\n        Collection::EmailSubmission,\n        Collection::Identity,\n    ] {\n        server\n            .archives(account_id, collection, &(), |document_id, archive| {\n                match collection {\n                    Collection::ContactCard => {\n                        let data = archive\n                            .unarchive_untrusted::<ContactCard>()\n                            .caused_by(trc::location!())?;\n\n                        if let Some(email) = data.emails().next() {\n                            indexes.push((\n                                collection,\n                                document_id,\n                                Operation::Index {\n                                    field: ContactField::Email.into(),\n                                    key: email.into_bytes(),\n                                    set: true,\n                                },\n                            ));\n                        }\n                        num_contacts += 1;\n                        indexes.push((\n                            collection,\n                            document_id,\n                            Operation::Value {\n                                class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                                    property: ContactField::CreatedToUpdated.into(),\n                                    value: data.created.to_native() as u64,\n                                }),\n                                op: ValueOp::Set((data.modified.to_native() as u64).serialize()),\n                            },\n                        ));\n                    }\n                    Collection::CalendarEventNotification => {\n                        let data = archive\n                            .unarchive_untrusted::<CalendarEventNotification>()\n                            .caused_by(trc::location!())?;\n                        num_calendars += 1;\n                        indexes.push((\n                            collection,\n                            document_id,\n                            Operation::Value {\n                                class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                                    property: CalendarNotificationField::CreatedToId.into(),\n                                    value: data.created.to_native() as u64,\n                                }),\n                                op: ValueOp::Set(\n                                    data.event_id\n                                        .as_ref()\n                                        .map(|v| v.to_native())\n                                        .unwrap_or(u32::MAX)\n                                        .serialize(),\n                                ),\n                            },\n                        ));\n                    }\n                    Collection::EmailSubmission => {\n                        let data = archive\n                            .unarchive_untrusted::<EmailSubmission>()\n                            .caused_by(trc::location!())?;\n                        num_email_submissions += 1;\n                        indexes.push((\n                            collection,\n                            document_id,\n                            Operation::Value {\n                                class: ValueClass::IndexProperty(IndexPropertyClass::Integer {\n                                    property: EmailSubmissionField::Metadata.into(),\n                                    value: data.send_at.to_native(),\n                                }),\n                                op: ValueOp::Set(\n                                    KeySerializer::new(U32_LEN * 3 + 1)\n                                        .write(data.email_id.to_native())\n                                        .write(data.thread_id.to_native())\n                                        .write(data.identity_id.to_native())\n                                        .write(data.undo_status.as_index())\n                                        .finalize(),\n                                ),\n                            },\n                        ));\n                    }\n                    Collection::Identity => {\n                        num_identities += 1;\n                        indexes.push((\n                            collection,\n                            document_id,\n                            Operation::Index {\n                                field: IdentityField::DocumentId.into(),\n                                key: vec![],\n                                set: true,\n                            },\n                        ));\n                    }\n                    _ => unreachable!(),\n                }\n\n                Ok(true)\n            })\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    let mut batch = BatchBuilder::new();\n    for (collection, document_id, op) in indexes {\n        batch\n            .with_account_id(account_id)\n            .with_collection(collection)\n            .with_document(document_id)\n            .any_op(op);\n        if batch.is_large_batch() || batch.len() == 255 {\n            server\n                .store()\n                .write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n            batch = BatchBuilder::new();\n        }\n    }\n\n    if !batch.is_empty() {\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n    }\n\n    Ok((\n        num_contacts,\n        num_calendars,\n        num_email_submissions,\n        num_identities,\n    ))\n}\n"
  },
  {
    "path": "crates/nlp/Cargo.toml",
    "content": "[package]\nname = \"nlp\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path =  \"../utils\" }\nxxhash-rust = { version = \"0.8.5\", features = [\"xxh3\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nnohash = \"0.2.0\"\nahash = { version = \"0.8.3\", features = [\"serde\"] }\nwhatlang = \"0.18\" # Language detection\nrust-stemmers = \"1.2\" # Stemmers\njieba-rs = \"0.8\" # Chinese stemmer\nlru-cache = \"0.1.2\"\nparking_lot = \"0.12.1\"\npsl = \"2\"\nmaplit = \"1.0.2\"\nhashify = \"0.2.1\"\nrand = \"0.9.2\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\n\n[features]\ntest_mode = []\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\nbincode = \"1.3.3\"\n"
  },
  {
    "path": "crates/nlp/src/classifier/adam.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::classifier::{Optimizer, model::FhClassifier};\n\npub struct Adam {\n    parameters: Vec<f32>,\n    bias: f32,\n    learning_rate: f32,\n    beta1: f32,\n    beta2: f32,\n    epsilon: f32,\n    t: f32,\n    m0: Vec<f32>,\n    v0: Vec<f32>,\n    m_bias: f32,\n    v_bias: f32,\n\n    // Step info\n    bias2_sqrt: f32,\n    alpha_t: f32,\n}\n\nimpl Adam {\n    pub fn new(n_parameters: usize, learning_rate: f32) -> Self {\n        Adam {\n            parameters: vec![0.0; n_parameters],\n            learning_rate,\n            beta1: 0.9,\n            beta2: 0.999,\n            epsilon: 1e-8,\n            t: 0.0,\n            m0: vec![0.0; n_parameters],\n            v0: vec![0.0; n_parameters],\n            m_bias: 0.0,\n            v_bias: 0.0,\n            bias: 0.0,\n            bias2_sqrt: 0.0,\n            alpha_t: 0.0,\n        }\n    }\n\n    pub fn with_hyperparams(mut self, beta1: f32, beta2: f32, epsilon: f32) -> Self {\n        self.beta1 = beta1;\n        self.beta2 = beta2;\n        self.epsilon = epsilon;\n        self\n    }\n\n    pub fn with_initial_weights(self, value: f32) -> Self {\n        Adam {\n            parameters: vec![value; self.parameters.len()],\n            ..self\n        }\n    }\n}\n\nimpl Optimizer for Adam {\n    #[inline(always)]\n    fn step(&mut self) {\n        self.t += 1.0;\n        let bias1 = 1.0 - self.beta1.powf(self.t);\n        self.bias2_sqrt = (1.0 - self.beta2.powf(self.t)).sqrt();\n        self.alpha_t = self.learning_rate / bias1;\n    }\n\n    #[inline(always)]\n    fn update_param(&mut self, i: usize, g: f32) {\n        self.m0[i] = self.beta1 * self.m0[i] + (1.0 - self.beta1) * g;\n        self.v0[i] = self.beta2 * self.v0[i] + (1.0 - self.beta2) * g * g;\n        self.parameters[i] -=\n            self.alpha_t * self.m0[i] / (self.v0[i].sqrt() / self.bias2_sqrt + self.epsilon);\n    }\n\n    #[inline(always)]\n    fn update_bias(&mut self, g: f32) {\n        self.m_bias = self.beta1 * self.m_bias + (1.0 - self.beta1) * g;\n        self.v_bias = self.beta2 * self.v_bias + (1.0 - self.beta2) * g * g;\n        self.bias -=\n            self.alpha_t * self.m_bias / (self.v_bias.sqrt() / self.bias2_sqrt + self.epsilon);\n    }\n\n    #[inline(always)]\n    fn get_param(&self, idx: usize) -> f32 {\n        self.parameters[idx]\n    }\n\n    #[inline(always)]\n    fn get_bias(&self) -> f32 {\n        self.bias\n    }\n\n    #[inline(always)]\n    fn get_param_mut(&mut self, idx: usize) -> &mut f32 {\n        &mut self.parameters[idx]\n    }\n\n    fn build_classifier(&self) -> FhClassifier {\n        FhClassifier {\n            parameters: self.parameters.clone(),\n            bias: self.bias,\n        }\n    }\n\n    fn num_parameters(&self) -> usize {\n        self.parameters.len()\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/classifier/feature.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::collections::HashMap;\nuse xxhash_rust::xxh3::xxh3_64_with_seed;\n\n#[derive(Debug)]\npub struct Sample<T> {\n    pub features: Vec<T>,\n    pub class: f32,\n}\n\npub struct FhFeatureBuilder {\n    pub(super) weight_mask: u64,\n}\n\n#[derive(Debug)]\npub struct FhFeature {\n    pub idx: usize,\n    pub weight: f32,\n}\n\n#[derive(Debug)]\npub struct CcfhFeature {\n    pub idx_w1: usize,\n    pub idx_w2: usize,\n    pub idx_i: usize,\n    pub weight: f32,\n}\n\npub struct CcfhFeatureBuilder {\n    pub(super) weight_mask: u64,\n    pub(super) indicator_mask: u64,\n}\n\npub trait FeatureWeight {\n    fn idx(&self) -> usize;\n    fn weight(&self) -> f32;\n    fn weight_mut(&mut self) -> &mut f32;\n}\n\npub trait UnprocessedFeature {\n    fn prefix(&self) -> u16;\n    fn value(&self) -> &[u8];\n}\n\nimpl FeatureWeight for FhFeature {\n    fn weight(&self) -> f32 {\n        self.weight\n    }\n\n    fn weight_mut(&mut self) -> &mut f32 {\n        &mut self.weight\n    }\n\n    fn idx(&self) -> usize {\n        self.idx\n    }\n}\n\nimpl FeatureWeight for CcfhFeature {\n    fn weight(&self) -> f32 {\n        self.weight\n    }\n\n    fn weight_mut(&mut self) -> &mut f32 {\n        &mut self.weight\n    }\n\n    fn idx(&self) -> usize {\n        self.idx_w1\n    }\n}\n\nimpl FeatureBuilder for FhFeatureBuilder {\n    type Feature = FhFeature;\n\n    fn build_feature(&self, bytes: &[u8], weight: f32) -> FhFeature {\n        let hash1 = xxh3_64_with_seed(bytes, 0);\n        let sign = if hash1 & (1 << 63) == 0 { 1.0 } else { -1.0 };\n\n        FhFeature {\n            idx: (hash1 & self.weight_mask) as usize,\n            weight: sign * weight,\n        }\n    }\n}\n\nimpl FeatureBuilder for CcfhFeatureBuilder {\n    type Feature = CcfhFeature;\n\n    fn build_feature(&self, bytes: &[u8], weight: f32) -> CcfhFeature {\n        let hash1 = xxh3_64_with_seed(bytes, 0);\n        let hash2 = xxh3_64_with_seed(bytes, 0x9E3779B97F4A7C15);\n        let hash3 = xxh3_64_with_seed(bytes, 0x517CC1B727220A95);\n        let sign = if hash3 & (1 << 63) == 0 { 1.0 } else { -1.0 };\n\n        CcfhFeature {\n            idx_w1: (hash1 & self.weight_mask) as usize,\n            idx_w2: (hash2 & self.weight_mask) as usize,\n            idx_i: (hash3 & self.indicator_mask) as usize,\n            weight: sign * weight,\n        }\n    }\n}\n\npub trait FeatureBuilder {\n    // Feature type associated type\n    type Feature: FeatureWeight;\n\n    fn build_feature(&self, bytes: &[u8], weight: f32) -> Self::Feature;\n\n    fn scale<I: UnprocessedFeature>(&self, features: &mut HashMap<I, f32>) {\n        // Log frequency scaling\n        for x in features.values_mut() {\n            *x = x.ln_1p();\n        }\n    }\n\n    fn build<I: UnprocessedFeature>(\n        &self,\n        features_in: &HashMap<I, f32>,\n        account_id: Option<u32>,\n        l2_normalize: bool,\n    ) -> Vec<Self::Feature> {\n        let mut features_out = Vec::with_capacity(features_in.len());\n        let mut buf = Vec::with_capacity(2 + 4 + 63);\n        for (feature, count) in features_in {\n            buf.extend_from_slice(&feature.prefix().to_be_bytes());\n            buf.extend_from_slice(feature.value());\n            features_out.push(self.build_feature(&buf, *count));\n\n            if let Some(account_id) = account_id {\n                buf.extend_from_slice(&account_id.to_be_bytes());\n                features_out.push(self.build_feature(&buf, *count));\n            }\n\n            buf.clear();\n        }\n\n        // L2 normalization\n        if l2_normalize {\n            let sum_of_squares = features_out\n                .iter()\n                .map(|f| f.weight() as f64 * f.weight() as f64)\n                .sum::<f64>();\n            if sum_of_squares > 0.0 {\n                let norm = sum_of_squares.sqrt() as f32;\n                for feature in &mut features_out {\n                    *feature.weight_mut() /= norm;\n                }\n            }\n        }\n\n        features_out\n    }\n}\n\nimpl<T> Sample<T> {\n    pub fn new(features: Vec<T>, class: bool) -> Self {\n        Self {\n            features,\n            class: if class { 1.0 } else { 0.0 },\n        }\n    }\n}\n\nimpl<T> AsRef<Sample<T>> for Sample<T> {\n    fn as_ref(&self) -> &Sample<T> {\n        self\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/classifier/ftrl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::classifier::{Optimizer, model::FhClassifier};\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug)]\npub struct Ftrl {\n    alpha: f64,\n    beta: f64,\n    l1_ratio: f64,\n    l2_ratio: f64,\n    zn: Vec<Zn>,\n    zn_bias: Zn,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Clone, Copy, Debug, Default)]\npub struct Zn {\n    z: f32,\n    n: f64,\n}\n\nimpl Ftrl {\n    pub fn new(n_features: usize) -> Self {\n        Ftrl {\n            alpha: 2.0,\n            beta: 1.0,\n            l1_ratio: 0.001,\n            l2_ratio: 0.0001,\n            zn: vec![Zn::default(); n_features],\n            zn_bias: Zn::default(),\n        }\n    }\n\n    pub fn with_hyperparams(mut self, alpha: f64, beta: f64, l1_ratio: f64, l2_ratio: f64) -> Self {\n        self.alpha = alpha;\n        self.beta = beta;\n        self.l1_ratio = l1_ratio;\n        self.l2_ratio = l2_ratio;\n        self\n    }\n\n    pub fn set_hyperparams(&mut self, alpha: f64, beta: f64, l1_ratio: f64, l2_ratio: f64) {\n        self.alpha = alpha;\n        self.beta = beta;\n        self.l1_ratio = l1_ratio;\n        self.l2_ratio = l2_ratio;\n    }\n\n    pub fn with_initial_weights(self, value: f32) -> Self {\n        Ftrl {\n            zn: vec![Zn { z: value, n: 0.0 }; self.zn.len()],\n            ..self\n        }\n    }\n}\n\nimpl Optimizer for Ftrl {\n    #[inline(always)]\n    fn update_param(&mut self, idx: usize, grad: f32) {\n        let zn = &mut self.zn[idx];\n        let current_w = if zn.z.abs() as f64 <= self.l1_ratio {\n            0.0\n        } else {\n            -(zn.z - zn.z.signum() * self.l1_ratio as f32)\n                / (self.l2_ratio + (self.beta + zn.n.sqrt()) / self.alpha) as f32\n        };\n        let grad = grad as f64;\n        let grad_sq = grad * grad;\n        let sigma = ((zn.n + grad_sq).sqrt() - zn.n.sqrt()) / self.alpha;\n        zn.z += (grad - sigma * current_w as f64) as f32;\n        zn.n += grad_sq;\n    }\n\n    #[inline(always)]\n    fn update_bias(&mut self, grad: f32) {\n        let current_bias = -self.zn_bias.z\n            / ((self.zn_bias.n.sqrt() + self.beta) / self.alpha + self.l2_ratio) as f32;\n        let grad = grad as f64;\n        let grad_sq = grad * grad;\n        let sigma = ((self.zn_bias.n + grad_sq).sqrt() - self.zn_bias.n.sqrt()) / self.alpha;\n        self.zn_bias.z += (grad - sigma * current_bias as f64) as f32;\n        self.zn_bias.n += grad_sq;\n    }\n\n    #[inline(always)]\n    fn get_param(&self, idx: usize) -> f32 {\n        let zn = self.zn[idx];\n        if zn.z.abs() as f64 <= self.l1_ratio {\n            0.0\n        } else {\n            -(zn.z - zn.z.signum() * self.l1_ratio as f32)\n                / (self.l2_ratio + (self.beta + zn.n.sqrt()) / self.alpha) as f32\n        }\n    }\n\n    #[inline(always)]\n    fn get_bias(&self) -> f32 {\n        -self.zn_bias.z / ((self.zn_bias.n.sqrt() + self.beta) / self.alpha + self.l2_ratio) as f32\n    }\n\n    fn step(&mut self) {}\n\n    #[inline(always)]\n    fn get_param_mut(&mut self, idx: usize) -> &mut f32 {\n        &mut self.zn[idx].z\n    }\n\n    fn build_classifier(&self) -> FhClassifier {\n        FhClassifier {\n            parameters: self\n                .zn\n                .iter()\n                .map(|zn| {\n                    if zn.z.abs() as f64 <= self.l1_ratio {\n                        0.0\n                    } else {\n                        -(zn.z - zn.z.signum() * self.l1_ratio as f32)\n                            / (self.l2_ratio + (self.beta + zn.n.sqrt()) / self.alpha) as f32\n                    }\n                })\n                .collect(),\n            bias: self.get_bias(),\n        }\n    }\n\n    fn num_parameters(&self) -> usize {\n        self.zn.len()\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/classifier/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::classifier::model::FhClassifier;\n\npub mod adam;\npub mod feature;\npub mod ftrl;\npub mod model;\npub mod reservoir;\npub mod sgd;\npub mod train;\n\nconst MAX_DLOSS: f32 = 1e4;\n\npub trait Optimizer {\n    fn step(&mut self);\n    fn update_param(&mut self, i: usize, g: f32);\n    fn update_bias(&mut self, g: f32);\n    fn get_param(&self, idx: usize) -> f32;\n    fn get_param_mut(&mut self, idx: usize) -> &mut f32;\n    fn get_bias(&self) -> f32;\n    fn build_classifier(&self) -> FhClassifier;\n    fn num_parameters(&self) -> usize;\n}\n\n#[inline(always)]\nfn sigmoid(z: f32) -> f32 {\n    let z = z.clamp(-35.0, 35.0);\n    if z >= 0.0 {\n        1.0 / (1.0 + (-z).exp())\n    } else {\n        let exp_z = z.exp();\n        exp_z / (1.0 + exp_z)\n    }\n}\n\n#[inline(always)]\nfn gradient(y: f32, p: f32) -> f32 {\n    if p > -16.0 {\n        let exp_tmp = (-p).exp();\n        ((1.0 - y) - y * exp_tmp) / (1.0 + exp_tmp)\n    } else {\n        p.exp() - y\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/classifier/model.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::classifier::{\n    feature::{CcfhFeature, CcfhFeatureBuilder, FhFeature, FhFeatureBuilder},\n    sigmoid,\n};\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]\npub struct FhClassifier {\n    pub(crate) parameters: Vec<f32>,\n    pub(crate) bias: f32,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]\npub struct CcfhClassifier {\n    pub(crate) parameters: Vec<f32>,\n    pub(crate) indicators: Vec<f32>,\n    pub(crate) bias: f32,\n}\n\nimpl FhClassifier {\n    pub fn predict_proba_sample(&self, features: &[FhFeature]) -> f32 {\n        let mut z: f32 = 0.0;\n\n        for f in features {\n            z += self.parameters[f.idx] * f.weight;\n        }\n\n        sigmoid(z + self.bias)\n    }\n\n    pub fn predict(&self, features: &[FhFeature]) -> f32 {\n        if self.predict_proba_sample(features) > 0.7 {\n            1.0\n        } else {\n            0.0\n        }\n    }\n\n    pub fn predict_batch<I>(&self, test: I) -> Vec<f32>\n    where\n        I: IntoIterator,\n        I::Item: AsRef<Vec<FhFeature>>,\n    {\n        test.into_iter()\n            .map(|features| self.predict(features.as_ref()))\n            .collect()\n    }\n\n    pub fn feature_builder(&self) -> FhFeatureBuilder {\n        FhFeatureBuilder {\n            weight_mask: (self.parameters.len() - 1) as u64,\n        }\n    }\n\n    pub fn parameters(&self) -> &[f32] {\n        &self.parameters\n    }\n\n    pub fn bias(&self) -> f32 {\n        self.bias\n    }\n}\n\nimpl CcfhClassifier {\n    pub fn predict_proba_sample(&self, features: &[CcfhFeature]) -> f32 {\n        let mut z: f32 = 0.0;\n        for f in features {\n            let q = self.indicators[f.idx_i];\n            let v1 = self.parameters[f.idx_w1];\n            let v2 = self.parameters[f.idx_w2];\n            z += (q * v1 + (1.0 - q) * v2) * f.weight;\n        }\n        sigmoid(z + self.bias)\n    }\n\n    pub fn predict(&self, features: &[CcfhFeature]) -> f32 {\n        if self.predict_proba_sample(features) >= 0.5 {\n            1.0\n        } else {\n            0.0\n        }\n    }\n\n    pub fn predict_batch<I>(&self, test: I) -> Vec<f32>\n    where\n        I: IntoIterator,\n        I::Item: AsRef<Vec<CcfhFeature>>,\n    {\n        test.into_iter()\n            .map(|features| self.predict(features.as_ref()))\n            .collect()\n    }\n\n    pub fn feature_builder(&self) -> CcfhFeatureBuilder {\n        CcfhFeatureBuilder {\n            weight_mask: (self.parameters.len() - 1) as u64,\n            indicator_mask: (self.indicators.len() - 1) as u64,\n        }\n    }\n\n    pub fn is_active(&self) -> bool {\n        !self.parameters.is_empty()\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/classifier/reservoir.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse rand::{Rng, seq::IndexedRandom};\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug)]\npub struct SampleReservoir<T> {\n    pub spam: SampleReservoirClass<T>,\n    pub ham: SampleReservoirClass<T>,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug)]\npub struct SampleReservoirClass<T> {\n    pub buffer: Vec<T>,\n    pub total_seen: u64,\n}\n\nimpl<T: Clone + Eq> SampleReservoir<T> {\n    pub fn update_reservoir(&mut self, item: &T, is_spam: bool, capacity: usize) {\n        let class = if is_spam {\n            &mut self.spam\n        } else {\n            &mut self.ham\n        };\n\n        class.total_seen += 1;\n\n        if class.buffer.len() < capacity {\n            class.buffer.push(item.clone());\n        } else if let Some(buf) = class\n            .buffer\n            .get_mut(rand::rng().random_range(0..class.total_seen as usize))\n        {\n            *buf = item.clone();\n        }\n    }\n\n    pub fn update_counts(&mut self, is_spam: bool) {\n        let class = if is_spam {\n            &mut self.spam\n        } else {\n            &mut self.ham\n        };\n\n        class.total_seen += 1;\n    }\n\n    pub fn replay_samples(\n        &mut self,\n        count_needed: usize,\n        is_spam: bool,\n    ) -> impl Iterator<Item = &T> {\n        (if is_spam {\n            &mut self.spam\n        } else {\n            &mut self.ham\n        })\n        .buffer\n        .choose_multiple(&mut rand::rng(), count_needed)\n    }\n\n    pub fn remove_sample(&mut self, item: &T, is_spam: bool) {\n        let class = if is_spam {\n            &mut self.spam\n        } else {\n            &mut self.ham\n        };\n\n        if let Some(pos) = class.buffer.iter().position(|x| x == item) {\n            class.buffer.swap_remove(pos);\n        }\n    }\n}\n\nimpl<T> Default for SampleReservoir<T> {\n    fn default() -> Self {\n        SampleReservoir {\n            spam: SampleReservoirClass {\n                buffer: Vec::new(),\n                total_seen: 0,\n            },\n            ham: SampleReservoirClass {\n                buffer: Vec::new(),\n                total_seen: 0,\n            },\n        }\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/classifier/sgd.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::classifier::{Optimizer, gradient, model::FhClassifier};\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]\npub struct Sgd {\n    parameters: Vec<f32>,\n    bias: f32,\n    alpha: f64,\n    l1_ratio: f64,\n    l2_ratio: f64,\n    t: f64,\n    w_scale: f32,\n    optimal_init: f64,\n    eta: f32,\n    u: f32,\n    q: Vec<f32>,\n}\n\nimpl Sgd {\n    pub fn new(n_features: usize, alpha: f64, l1_ratio: f64, l2_ratio: f64) -> Self {\n        let typw = (1.0 / alpha.sqrt()).sqrt();\n        let initial_eta0 = typw / 1.0_f64.max(gradient(1.0, -typw as f32) as f64);\n        let optimal_init = 1.0 / (initial_eta0 * alpha);\n\n        Sgd {\n            parameters: vec![0.0; n_features],\n            bias: 0.0,\n            alpha,\n            l1_ratio,\n            l2_ratio,\n            t: 0.0,\n            w_scale: 1.0,\n            optimal_init,\n            eta: initial_eta0 as f32,\n            u: 0.0,\n            q: vec![0.0; n_features],\n        }\n    }\n\n    pub fn with_initial_parameters(self, value: f32) -> Self {\n        Sgd {\n            parameters: vec![value; self.parameters.len()],\n            ..self\n        }\n    }\n\n    fn maybe_rescale(&mut self) {\n        if !(1e-6..=1e6).contains(&self.w_scale) {\n            for w in &mut self.parameters {\n                *w *= self.w_scale;\n            }\n            self.w_scale = 1.0;\n        }\n    }\n\n    #[inline(always)]\n    fn apply_l1_penalty(&mut self) {\n        if self.l1_ratio > 0.0 {\n            for (z, q) in self.parameters.iter_mut().zip(self.q.iter_mut()) {\n                let z_orig = *z;\n                let scaled_z = *z * self.w_scale;\n                if scaled_z > 0.0 {\n                    *z = (*z - (self.u + *q) / self.w_scale).max(0.0);\n                } else if scaled_z < 0.0 {\n                    *z = (*z + (self.u - *q) / self.w_scale).min(0.0);\n                }\n                *q += self.w_scale * (z_orig - *z);\n            }\n        }\n    }\n}\n\nimpl Optimizer for Sgd {\n    fn step(&mut self) {\n        self.t += 1.0;\n        self.eta = (1.0 / ((self.alpha) * (self.optimal_init + self.t - 1.0))) as f32;\n        self.w_scale *= 1.0 - ((1.0 - self.l1_ratio) as f32 * self.eta * self.l2_ratio as f32);\n        self.u += self.eta * self.l1_ratio as f32 * self.alpha as f32;\n    }\n\n    fn update_param(&mut self, i: usize, g: f32) {\n        self.parameters[i] += (-self.eta * g) / self.w_scale;\n    }\n\n    fn update_bias(&mut self, g: f32) {\n        self.bias += -self.eta * g;\n        self.maybe_rescale();\n        self.apply_l1_penalty();\n    }\n\n    #[inline(always)]\n    fn get_param(&self, idx: usize) -> f32 {\n        self.parameters[idx] * self.w_scale\n    }\n\n    #[inline(always)]\n    fn get_bias(&self) -> f32 {\n        self.bias\n    }\n\n    #[inline(always)]\n    fn get_param_mut(&mut self, idx: usize) -> &mut f32 {\n        &mut self.parameters[idx]\n    }\n\n    fn build_classifier(&self) -> FhClassifier {\n        FhClassifier {\n            parameters: self.parameters.iter().map(|w| w * self.w_scale).collect(),\n            bias: self.bias,\n        }\n    }\n\n    fn num_parameters(&self) -> usize {\n        self.parameters.len()\n    }\n}\n\n#[cfg(test)]\npub mod tests {\n    use crate::classifier::{\n        Optimizer,\n        adam::Adam,\n        feature::{\n            CcfhFeature, CcfhFeatureBuilder, FeatureBuilder, FhFeature, FhFeatureBuilder, Sample,\n            UnprocessedFeature,\n        },\n        ftrl::Ftrl,\n        train::{CcfhTrainer, FhTrainer},\n    };\n    use rand::{SeedableRng, rngs::StdRng, seq::SliceRandom};\n    use std::{\n        collections::HashMap,\n        fs::File,\n        io::{BufRead, BufReader},\n        time::Instant,\n    };\n\n    #[ignore]\n    #[test]\n    fn text_classifier() {\n        let reader = BufReader::new(\n            File::open(\"/Users/me/code/playground/phishing_email.csv\")\n                .expect(\"Could not open file\"),\n        );\n        let mut samples = Vec::with_capacity(1024);\n\n        let time = Instant::now();\n\n        for line in reader.lines().skip(1) {\n            let line = line.unwrap();\n            let (text, class) = line.trim().rsplit_once(',').unwrap();\n            //let (class, text) = line.trim().split_once(',').unwrap();\n            let text = text.trim_start_matches('\"').trim_end_matches('\"');\n\n            samples.push((text.to_string(), class == \"1\"));\n        }\n\n        println!(\"Loaded {} samples in {:?}\", samples.len(), time.elapsed());\n\n        samples.shuffle(&mut StdRng::seed_from_u64(42));\n\n        let (train_samples, test_samples) = train_test_split(&samples, 0.2);\n\n        println!(\n            \"Training samples: {}, Testing samples: {}\",\n            train_samples.len(),\n            test_samples.len()\n        );\n\n        const FH_SIZE: usize = 16;\n        const CCFH_SIZE: usize = FH_SIZE - 2;\n        let mut rng = StdRng::seed_from_u64(42);\n\n        let fh_builder = FhFeatureBuilder {\n            weight_mask: (1 << FH_SIZE) - 1,\n        };\n        let mut fh_train_samples = build_fh_samples(train_samples.as_slice(), &fh_builder);\n        fh_train_samples.shuffle(&mut rng);\n        let fh_test_samples = build_fh_samples(test_samples.as_slice(), &fh_builder);\n        let ccfh_builder = CcfhFeatureBuilder {\n            weight_mask: (1 << FH_SIZE) - 1,\n            indicator_mask: (1 << CCFH_SIZE) - 1,\n        };\n        let mut ccfh_train_samples = build_ccfh_samples(train_samples.as_slice(), &ccfh_builder);\n        ccfh_train_samples.shuffle(&mut rng);\n        let ccfh_test_samples = build_ccfh_samples(test_samples.as_slice(), &ccfh_builder);\n\n        fh_model_stats(\n            \"FTRL\",\n            FhTrainer::new(Ftrl::new(1 << FH_SIZE)),\n            &fh_train_samples,\n            &fh_test_samples,\n        );\n\n        ccfh_model_stats(\n            \"FTRL + FTRL\",\n            CcfhTrainer::new(\n                Ftrl::new(1 << FH_SIZE),\n                Ftrl::new(1 << CCFH_SIZE).with_initial_weights(0.5),\n            ),\n            &ccfh_train_samples,\n            &ccfh_test_samples,\n        );\n\n        fh_model_stats(\n            \"Adam\",\n            FhTrainer::new(Adam::new(1 << FH_SIZE, 0.01)),\n            &fh_train_samples,\n            &fh_test_samples,\n        );\n\n        ccfh_model_stats(\n            \"Adam + Adam\",\n            CcfhTrainer::new(\n                Adam::new(1 << FH_SIZE, 0.01),\n                Adam::new(1 << CCFH_SIZE, 0.01).with_initial_weights(0.5),\n            ),\n            &ccfh_train_samples,\n            &ccfh_test_samples,\n        );\n\n        /*fh_model_stats(\n            \"SGD\",\n            FhTrainer::new(Sgd::new(1 << FH_SIZE, 0.0001, 0.0, 0.0001)),\n            &fh_train_samples,\n            &fh_test_samples,\n        );\n\n        ccfh_model_stats(\n            \"FTRL + SGD\",\n            CcfhTrainer::new(\n                Ftrl::new(1 << FH_SIZE),\n                Sgd::new(1 << CCFH_SIZE, 0.0001, 0.0, 0.0001).with_initial_parameters(0.5),\n            ),\n            &ccfh_train_samples,\n            &ccfh_test_samples,\n        );*/\n    }\n\n    fn fh_model_stats(\n        name: &str,\n        mut model: FhTrainer<impl Optimizer>,\n        train_samples: &[Sample<FhFeature>],\n        test_samples: &[Sample<FhFeature>],\n    ) {\n        print!(\"⏳ Training {}... \", name);\n        let time = Instant::now();\n        let mut batch = Vec::new();\n        for sample in train_samples {\n            batch.push(sample);\n            if batch.len() == 128 {\n                model.fit(&mut batch, 5);\n                batch.clear();\n            }\n        }\n        if !batch.is_empty() {\n            model.fit(&mut batch, 5);\n        }\n        println!(\" trained in {:?}\", time.elapsed());\n        let y_pred = model\n            .build_classifier()\n            .predict_batch(test_samples.iter().map(|s| &s.features));\n        let y_train: Vec<f32> = test_samples.iter().map(|s| s.class).collect();\n        println!(\"Accuracy: {:.4}\", accuracy_score(&y_train, &y_pred));\n        println!(\"Precision: {:.4}\", precision_score(&y_train, &y_pred, 1.0));\n        println!(\"Recall: {:.4}\", recall_score(&y_train, &y_pred, 1.0));\n        println!(\"F1 Score: {:.4}\", f1_score(&y_train, &y_pred, 1.0));\n    }\n\n    fn ccfh_model_stats(\n        name: &str,\n        mut model: CcfhTrainer<impl Optimizer, impl Optimizer>,\n        train_samples: &[Sample<CcfhFeature>],\n        test_samples: &[Sample<CcfhFeature>],\n    ) {\n        print!(\"⏳ Training {}... \", name);\n        let time = Instant::now();\n        let mut batch = Vec::new();\n        for sample in train_samples {\n            batch.push(sample);\n            if batch.len() == 128 {\n                model.fit(&mut batch, 5);\n                batch.clear();\n            }\n        }\n        if !batch.is_empty() {\n            model.fit(&mut batch, 5);\n        }\n        println!(\" trained in {:?}\", time.elapsed());\n        let y_pred = model\n            .build_classifier()\n            .predict_batch(test_samples.iter().map(|s| &s.features));\n        let y_train: Vec<f32> = test_samples.iter().map(|s| s.class).collect();\n        println!(\"Accuracy: {:.4}\", accuracy_score(&y_train, &y_pred));\n        println!(\"Precision: {:.4}\", precision_score(&y_train, &y_pred, 1.0));\n        println!(\"Recall: {:.4}\", recall_score(&y_train, &y_pred, 1.0));\n        println!(\"F1 Score: {:.4}\", f1_score(&y_train, &y_pred, 1.0));\n    }\n\n    fn accuracy_score(y_true: &[f32], y_pred: &[f32]) -> f32 {\n        y_true\n            .iter()\n            .zip(y_pred.iter())\n            .filter(|(true_val, pred_val)| **true_val == **pred_val)\n            .count() as f32\n            / y_true.len() as f32\n    }\n\n    fn precision_score(y_true: &[f32], y_pred: &[f32], positive_class: f32) -> f32 {\n        let true_positives = y_true\n            .iter()\n            .zip(y_pred.iter())\n            .filter(|(true_val, pred_val)| {\n                **pred_val == positive_class && **true_val == positive_class\n            })\n            .count() as f32;\n\n        let predicted_positives = y_pred\n            .iter()\n            .filter(|pred_val| **pred_val == positive_class)\n            .count() as f32;\n\n        if predicted_positives == 0.0 {\n            0.0\n        } else {\n            true_positives / predicted_positives\n        }\n    }\n\n    fn recall_score(y_true: &[f32], y_pred: &[f32], positive_class: f32) -> f32 {\n        let true_positives = y_true\n            .iter()\n            .zip(y_pred.iter())\n            .filter(|(true_val, pred_val)| {\n                **pred_val == positive_class && **true_val == positive_class\n            })\n            .count() as f32;\n\n        let actual_positives = y_true\n            .iter()\n            .filter(|true_val| **true_val == positive_class)\n            .count() as f32;\n\n        if actual_positives == 0.0 {\n            0.0\n        } else {\n            true_positives / actual_positives\n        }\n    }\n\n    fn f1_score(y_true: &[f32], y_pred: &[f32], positive_class: f32) -> f32 {\n        let precision = precision_score(y_true, y_pred, positive_class);\n        let recall = recall_score(y_true, y_pred, positive_class);\n\n        if precision + recall == 0.0 {\n            0.0\n        } else {\n            2.0 * (precision * recall) / (precision + recall)\n        }\n    }\n\n    #[allow(clippy::type_complexity)]\n    pub fn train_test_split(\n        data: &[(String, bool)],\n        test_size: f32,\n    ) -> (Vec<(&String, bool)>, Vec<(&String, bool)>) {\n        let mut class_0: Vec<(&String, bool)> = Vec::new();\n        let mut class_1: Vec<(&String, bool)> = Vec::new();\n\n        for (sample, class) in data {\n            if !*class {\n                class_0.push((sample, *class));\n            } else {\n                class_1.push((sample, *class));\n            }\n        }\n\n        let test_count_0 = (class_0.len() as f32 * test_size).round() as usize;\n        let test_count_1 = (class_1.len() as f32 * test_size).round() as usize;\n\n        let (test_0, train_0) = class_0.split_at(test_count_0);\n        let (test_1, train_1) = class_1.split_at(test_count_1);\n\n        let mut train = Vec::new();\n        let mut test = Vec::new();\n\n        train.extend_from_slice(train_0);\n        train.extend_from_slice(train_1);\n        test.extend_from_slice(test_0);\n        test.extend_from_slice(test_1);\n\n        (train, test)\n    }\n\n    pub fn build_fh_samples(\n        data: &[(&String, bool)],\n        builder: &FhFeatureBuilder,\n    ) -> Vec<Sample<FhFeature>> {\n        let mut samples = Vec::with_capacity(data.len());\n\n        for (text, class) in data {\n            let mut sample: HashMap<String, f32> = HashMap::new();\n            for word in text.split_whitespace() {\n                *sample.entry(word.to_string()).or_default() += 1.0;\n            }\n            builder.scale(&mut sample);\n            samples.push(Sample {\n                features: builder.build(&sample, 12345.into(), true),\n                class: if *class { 1.0 } else { 0.0 },\n            });\n        }\n\n        samples\n    }\n\n    pub fn build_ccfh_samples(\n        data: &[(&String, bool)],\n        builder: &CcfhFeatureBuilder,\n    ) -> Vec<Sample<CcfhFeature>> {\n        let mut samples = Vec::with_capacity(data.len());\n\n        for (text, class) in data {\n            let mut sample: HashMap<String, f32> = HashMap::new();\n            for word in text.split_whitespace() {\n                *sample.entry(word.to_string()).or_default() += 1.0;\n            }\n            builder.scale(&mut sample);\n            samples.push(Sample {\n                features: builder.build(&sample, 12345.into(), true),\n                class: if *class { 1.0 } else { 0.0 },\n            });\n        }\n\n        samples\n    }\n\n    impl UnprocessedFeature for String {\n        fn prefix(&self) -> u16 {\n            0\n        }\n\n        fn value(&self) -> &[u8] {\n            self.as_bytes()\n        }\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/classifier/train.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::classifier::{\n    MAX_DLOSS, Optimizer,\n    feature::{CcfhFeature, CcfhFeatureBuilder, FhFeature, FhFeatureBuilder, Sample},\n    gradient,\n    model::{CcfhClassifier, FhClassifier},\n};\nuse rand::{SeedableRng, rngs::StdRng, seq::SliceRandom};\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]\npub struct FhTrainer<T: Optimizer> {\n    pub optimizer: T,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]\npub struct CcfhTrainer<W: Optimizer, I: Optimizer> {\n    pub w_optimizer: W,\n    pub i_optimizer: I,\n}\n\nimpl<T: Optimizer> FhTrainer<T> {\n    pub fn new(optimizer: T) -> Self {\n        FhTrainer { optimizer }\n    }\n\n    pub fn fit(&mut self, samples: &mut [impl AsRef<Sample<FhFeature>>], num_epochs: usize) {\n        for _ in 0..num_epochs {\n            samples.shuffle(&mut StdRng::seed_from_u64(42));\n\n            for sample in samples.iter() {\n                let sample = sample.as_ref();\n                let mut dot: f32 = 0.0;\n                for f in &sample.features {\n                    dot += self.optimizer.get_param(f.idx) * f.weight;\n                }\n                let p = dot + self.optimizer.get_bias();\n                let dloss = gradient(sample.class, p).clamp(-MAX_DLOSS, MAX_DLOSS);\n\n                self.optimizer.step();\n\n                for f in &sample.features {\n                    self.optimizer.update_param(f.idx, dloss * f.weight);\n                }\n\n                self.optimizer.update_bias(dloss);\n            }\n        }\n    }\n\n    pub fn feature_builder(&self) -> FhFeatureBuilder {\n        FhFeatureBuilder {\n            weight_mask: (self.optimizer.num_parameters() - 1) as u64,\n        }\n    }\n\n    pub fn build_classifier(&self) -> FhClassifier {\n        self.optimizer.build_classifier()\n    }\n\n    pub fn optimizer(&self) -> &T {\n        &self.optimizer\n    }\n\n    pub fn optimizer_mut(&mut self) -> &mut T {\n        &mut self.optimizer\n    }\n}\n\nimpl<W: Optimizer, I: Optimizer> CcfhTrainer<W, I> {\n    pub fn new(w_optimizer: W, i_optimizer: I) -> Self {\n        CcfhTrainer {\n            w_optimizer,\n            i_optimizer,\n        }\n    }\n\n    pub fn fit(&mut self, samples: &mut [impl AsRef<Sample<CcfhFeature>>], num_epochs: usize) {\n        for _ in 0..num_epochs {\n            samples.shuffle(&mut StdRng::seed_from_u64(42));\n\n            for sample in samples.iter() {\n                let sample = sample.as_ref();\n                let mut dot: f32 = 0.0;\n                for f in &sample.features {\n                    let q = self.i_optimizer.get_param(f.idx_i);\n                    let v1 = self.w_optimizer.get_param(f.idx_w1);\n                    let v2 = self.w_optimizer.get_param(f.idx_w2);\n                    dot += (q * v1 + (1.0 - q) * v2) * f.weight;\n                }\n                let p = dot + self.w_optimizer.get_bias();\n                let dloss = gradient(sample.class, p).clamp(-MAX_DLOSS, MAX_DLOSS);\n\n                self.w_optimizer.step();\n                self.i_optimizer.step();\n\n                for f in &sample.features {\n                    let q = self.i_optimizer.get_param(f.idx_i);\n                    let v1 = self.w_optimizer.get_param(f.idx_w1);\n                    let v2 = self.w_optimizer.get_param(f.idx_w2);\n\n                    // Update weights\n                    let d_v1 = f.weight * q;\n                    let d_v2 = f.weight * (1.0 - q);\n                    self.w_optimizer.update_param(f.idx_w1, dloss * d_v1);\n                    self.w_optimizer.update_param(f.idx_w2, dloss * d_v2);\n\n                    // Update indicator\n                    let d_q = (v1 - v2) * f.weight;\n                    self.i_optimizer.update_param(f.idx_i, dloss * d_q);\n                    let fi = self.i_optimizer.get_param_mut(f.idx_i);\n                    *fi = fi.clamp(0.0, 1.0);\n                }\n\n                self.w_optimizer.update_bias(dloss);\n            }\n        }\n    }\n\n    pub fn feature_builder(&self) -> CcfhFeatureBuilder {\n        CcfhFeatureBuilder {\n            weight_mask: (self.w_optimizer.num_parameters() - 1) as u64,\n            indicator_mask: (self.i_optimizer.num_parameters() - 1) as u64,\n        }\n    }\n\n    pub fn build_classifier(&self) -> CcfhClassifier {\n        let w_classifier = self.w_optimizer.build_classifier();\n        let i_classifier = self.i_optimizer.build_classifier();\n\n        CcfhClassifier {\n            parameters: w_classifier.parameters,\n            indicators: i_classifier.parameters,\n            bias: w_classifier.bias,\n        }\n    }\n\n    pub fn w_optimizer(&self) -> &W {\n        &self.w_optimizer\n    }\n\n    pub fn w_optimizer_mut(&mut self) -> &mut W {\n        &mut self.w_optimizer\n    }\n\n    pub fn i_optimizer(&self) -> &I {\n        &self.i_optimizer\n    }\n\n    pub fn i_optimizer_mut(&mut self) -> &mut I {\n        &mut self.i_optimizer\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/language/detect.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::Language;\nuse ahash::AHashMap;\nuse whatlang::{Lang, detect};\n\npub const MIN_LANGUAGE_SCORE: f64 = 0.6;\n\n#[derive(Debug)]\nstruct WeightedAverage {\n    weight: usize,\n    occurrences: usize,\n    confidence: f64,\n}\n\n#[derive(Debug)]\npub struct LanguageDetector {\n    lang_detected: AHashMap<Language, WeightedAverage>,\n}\n\nimpl Default for LanguageDetector {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n\nimpl LanguageDetector {\n    pub fn new() -> LanguageDetector {\n        LanguageDetector {\n            lang_detected: AHashMap::default(),\n        }\n    }\n\n    pub fn detect(&mut self, text: &str, min_score: f64) -> Language {\n        if let Some((language, confidence)) = LanguageDetector::detect_single(text) {\n            let w = self\n                .lang_detected\n                .entry(language)\n                .or_insert_with(|| WeightedAverage {\n                    weight: 0,\n                    confidence: 0.0,\n                    occurrences: 0,\n                });\n            w.occurrences += 1;\n            w.weight += text.len();\n            w.confidence += confidence * text.len() as f64;\n            if confidence < min_score {\n                Language::Unknown\n            } else {\n                language\n            }\n        } else {\n            Language::Unknown\n        }\n    }\n\n    pub fn most_frequent_language(&self) -> Option<Language> {\n        self.lang_detected\n            .iter()\n            .filter(|(l, _)| !matches!(l, Language::None))\n            .max_by(|(_, a), (_, b)| {\n                ((a.confidence / a.weight as f64) * a.occurrences as f64)\n                    .partial_cmp(&((b.confidence / b.weight as f64) * b.occurrences as f64))\n                    .unwrap_or(std::cmp::Ordering::Less)\n            })\n            .map(|(l, _)| *l)\n    }\n\n    pub fn detect_single(text: &str) -> Option<(Language, f64)> {\n        detect(text).map(|info| {\n            (\n                match info.lang() {\n                    Lang::Epo => Language::Esperanto,\n                    Lang::Eng => Language::English,\n                    Lang::Rus => Language::Russian,\n                    Lang::Cmn => Language::Mandarin,\n                    Lang::Spa => Language::Spanish,\n                    Lang::Por => Language::Portuguese,\n                    Lang::Ita => Language::Italian,\n                    Lang::Ben => Language::Bengali,\n                    Lang::Fra => Language::French,\n                    Lang::Deu => Language::German,\n                    Lang::Ukr => Language::Ukrainian,\n                    Lang::Kat => Language::Georgian,\n                    Lang::Ara => Language::Arabic,\n                    Lang::Hin => Language::Hindi,\n                    Lang::Jpn => Language::Japanese,\n                    Lang::Heb => Language::Hebrew,\n                    Lang::Yid => Language::Yiddish,\n                    Lang::Pol => Language::Polish,\n                    Lang::Amh => Language::Amharic,\n                    Lang::Jav => Language::Javanese,\n                    Lang::Kor => Language::Korean,\n                    Lang::Nob => Language::Bokmal,\n                    Lang::Dan => Language::Danish,\n                    Lang::Swe => Language::Swedish,\n                    Lang::Fin => Language::Finnish,\n                    Lang::Tur => Language::Turkish,\n                    Lang::Nld => Language::Dutch,\n                    Lang::Hun => Language::Hungarian,\n                    Lang::Ces => Language::Czech,\n                    Lang::Ell => Language::Greek,\n                    Lang::Bul => Language::Bulgarian,\n                    Lang::Bel => Language::Belarusian,\n                    Lang::Mar => Language::Marathi,\n                    Lang::Kan => Language::Kannada,\n                    Lang::Ron => Language::Romanian,\n                    Lang::Slv => Language::Slovene,\n                    Lang::Hrv => Language::Croatian,\n                    Lang::Srp => Language::Serbian,\n                    Lang::Mkd => Language::Macedonian,\n                    Lang::Lit => Language::Lithuanian,\n                    Lang::Lav => Language::Latvian,\n                    Lang::Est => Language::Estonian,\n                    Lang::Tam => Language::Tamil,\n                    Lang::Vie => Language::Vietnamese,\n                    Lang::Urd => Language::Urdu,\n                    Lang::Tha => Language::Thai,\n                    Lang::Guj => Language::Gujarati,\n                    Lang::Uzb => Language::Uzbek,\n                    Lang::Pan => Language::Punjabi,\n                    Lang::Aze => Language::Azerbaijani,\n                    Lang::Ind => Language::Indonesian,\n                    Lang::Tel => Language::Telugu,\n                    Lang::Pes => Language::Persian,\n                    Lang::Mal => Language::Malayalam,\n                    Lang::Ori => Language::Oriya,\n                    Lang::Mya => Language::Burmese,\n                    Lang::Nep => Language::Nepali,\n                    Lang::Sin => Language::Sinhalese,\n                    Lang::Khm => Language::Khmer,\n                    Lang::Tuk => Language::Turkmen,\n                    Lang::Aka => Language::Akan,\n                    Lang::Zul => Language::Zulu,\n                    Lang::Sna => Language::Shona,\n                    Lang::Afr => Language::Afrikaans,\n                    Lang::Lat => Language::Latin,\n                    Lang::Slk => Language::Slovak,\n                    Lang::Cat => Language::Catalan,\n                    Lang::Tgl => Language::Tagalog,\n                    Lang::Hye => Language::Armenian,\n                    _ => Language::Unknown,\n                },\n                info.confidence(),\n            )\n        })\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn detect_languages() {\n        let inputs = [\n            (\n                \"The quick brown fox jumps over the lazy dog\",\n                Language::English,\n            ),\n            (\n                \"Jovencillo emponzoñado de whisky: ¡qué figurota exhibe!\",\n                Language::Spanish,\n            ),\n            (\n                \"Ma la volpe col suo balzo ha raggiunto il quieto Fido\",\n                Language::Italian,\n            ),\n            (\n                \"Jaz em prisão bota que vexa dez cegonhas felizes\",\n                Language::Portuguese,\n            ),\n            (\n                \"Zwölf Boxkämpfer jagten Victor quer über den großen Sylter Deich\",\n                Language::German,\n            ),\n            (\"עטלף אבק נס דרך מזגן שהתפוצץ כי חם\", Language::Hebrew),\n            (\n                \"Съешь ещё этих мягких французских булок, да выпей же чаю\",\n                Language::Russian,\n            ),\n            (\n                \"Чуєш їх, доцю, га? Кумедна ж ти, прощайся без ґольфів!\",\n                Language::Ukrainian,\n            ),\n            (\n                \"Љубазни фењерџија чађавог лица хоће да ми покаже штос\",\n                Language::Serbian,\n            ),\n            (\n                \"Pijamalı hasta yağız şoföre çabucak güvendi\",\n                Language::Turkish,\n            ),\n            (\"己所不欲,勿施于人。\", Language::Mandarin),\n            (\"井の中の蛙大海を知らず\", Language::Japanese),\n            (\"시작이 반이다\", Language::Korean),\n        ];\n\n        let mut detector = LanguageDetector::new();\n\n        for input in inputs.iter() {\n            assert_eq!(detector.detect(input.0, 0.0), input.1);\n        }\n    }\n\n    #[test]\n    fn weighted_language() {\n        let mut detector = LanguageDetector::new();\n        for lang in [\n            (Language::Spanish, 0.5, 70),\n            (Language::Japanese, 0.2, 100),\n            (Language::Japanese, 0.3, 100),\n            (Language::Japanese, 0.4, 200),\n            (Language::English, 0.7, 50),\n        ]\n        .iter()\n        {\n            let w = detector\n                .lang_detected\n                .entry(lang.0)\n                .or_insert_with(|| WeightedAverage {\n                    weight: 0,\n                    confidence: 0.0,\n                    occurrences: 0,\n                });\n            w.occurrences += 1;\n            w.weight += lang.2;\n            w.confidence += lang.1 * lang.2 as f64;\n        }\n        assert_eq!(detector.most_frequent_language(), Some(Language::Japanese));\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/language/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod detect;\npub mod search_snippet;\npub mod stemmer;\npub mod stopwords;\n\nuse self::detect::LanguageDetector;\nuse crate::tokenizers::{\n    Token, chinese::ChineseTokenizer, japanese::JapaneseTokenizer, space::SpaceTokenizer,\n    word::WordTokenizer,\n};\nuse std::borrow::Cow;\nuse utils::config::utils::ParseValue;\n\npub type LanguageTokenizer<'x> = Box<dyn Iterator<Item = Token<Cow<'x, str>>> + 'x + Sync + Send>;\n\nimpl Language {\n    pub fn tokenize_text<'x>(\n        &self,\n        text: &'x str,\n        max_token_length: usize,\n    ) -> LanguageTokenizer<'x> {\n        match self {\n            Language::Japanese => Box::new(\n                JapaneseTokenizer::new(WordTokenizer::new(text, usize::MAX))\n                    .filter(move |t| t.word.len() <= max_token_length),\n            ),\n            Language::Mandarin => Box::new(\n                ChineseTokenizer::new(WordTokenizer::new(text, usize::MAX))\n                    .filter(move |t| t.word.len() <= max_token_length),\n            ),\n            Language::None => {\n                Box::new(\n                    SpaceTokenizer::new(text, max_token_length).map(|word| Token {\n                        word: word.into(),\n                        from: 0,\n                        to: 0,\n                    }),\n                )\n            }\n            _ => Box::new(WordTokenizer::new(text, max_token_length)),\n        }\n    }\n}\n\n#[derive(\n    Debug, PartialEq, Clone, Copy, Hash, Eq, serde::Serialize, serde::Deserialize, Default,\n)]\npub enum Language {\n    Esperanto = 0,\n    #[default]\n    English = 1,\n    Russian = 2,\n    Mandarin = 3,\n    Spanish = 4,\n    Portuguese = 5,\n    Italian = 6,\n    Bengali = 7,\n    French = 8,\n    German = 9,\n    Ukrainian = 10,\n    Georgian = 11,\n    Arabic = 12,\n    Hindi = 13,\n    Japanese = 14,\n    Hebrew = 15,\n    Yiddish = 16,\n    Polish = 17,\n    Amharic = 18,\n    Javanese = 19,\n    Korean = 20,\n    Bokmal = 21,\n    Danish = 22,\n    Swedish = 23,\n    Finnish = 24,\n    Turkish = 25,\n    Dutch = 26,\n    Hungarian = 27,\n    Czech = 28,\n    Greek = 29,\n    Bulgarian = 30,\n    Belarusian = 31,\n    Marathi = 32,\n    Kannada = 33,\n    Romanian = 34,\n    Slovene = 35,\n    Croatian = 36,\n    Serbian = 37,\n    Macedonian = 38,\n    Lithuanian = 39,\n    Latvian = 40,\n    Estonian = 41,\n    Tamil = 42,\n    Vietnamese = 43,\n    Urdu = 44,\n    Thai = 45,\n    Gujarati = 46,\n    Uzbek = 47,\n    Punjabi = 48,\n    Azerbaijani = 49,\n    Indonesian = 50,\n    Telugu = 51,\n    Persian = 52,\n    Malayalam = 53,\n    Oriya = 54,\n    Burmese = 55,\n    Nepali = 56,\n    Sinhalese = 57,\n    Khmer = 58,\n    Turkmen = 59,\n    Akan = 60,\n    Zulu = 61,\n    Shona = 62,\n    Afrikaans = 63,\n    Latin = 64,\n    Slovak = 65,\n    Catalan = 66,\n    Tagalog = 67,\n    Armenian = 68,\n    Unknown = 69,\n    None = 70,\n}\n\nimpl Language {\n    pub fn is_unknown(&self) -> bool {\n        matches!(self, Language::Unknown)\n    }\n\n    pub fn from_iso_639(code: &str) -> Option<Self> {\n        hashify::map!(\n            code.split_once('-').map(|c| c.0).unwrap_or(code).as_bytes(),\n            Language,\n            \"en\" => Language::English,\n            \"es\" => Language::Spanish,\n            \"pt\" => Language::Portuguese,\n            \"it\" => Language::Italian,\n            \"fr\" => Language::French,\n            \"de\" => Language::German,\n            \"da\" => Language::Danish,\n            \"ru\" => Language::Russian,\n            \"zh\" => Language::Mandarin,\n            \"ja\" => Language::Japanese,\n            \"ar\" => Language::Arabic,\n            \"hi\" => Language::Hindi,\n            \"ko\" => Language::Korean,\n            \"bn\" => Language::Bengali,\n            \"he\" => Language::Hebrew,\n            \"ur\" => Language::Urdu,\n            \"fa\" => Language::Persian,\n            \"ml\" => Language::Malayalam,\n            \"or\" => Language::Oriya,\n            \"my\" => Language::Burmese,\n            \"ne\" => Language::Nepali,\n            \"si\" => Language::Sinhalese,\n            \"km\" => Language::Khmer,\n            \"tk\" => Language::Turkmen,\n            \"am\" => Language::Amharic,\n            \"az\" => Language::Azerbaijani,\n            \"id\" => Language::Indonesian,\n            \"te\" => Language::Telugu,\n            \"ta\" => Language::Tamil,\n            \"vi\" => Language::Vietnamese,\n            \"gu\" => Language::Gujarati,\n            \"pa\" => Language::Punjabi,\n            \"uz\" => Language::Uzbek,\n            \"hy\" => Language::Armenian,\n            \"ka\" => Language::Georgian,\n            \"la\" => Language::Latin,\n            \"sl\" => Language::Slovene,\n            \"hr\" => Language::Croatian,\n            \"sr\" => Language::Serbian,\n            \"mk\" => Language::Macedonian,\n            \"lt\" => Language::Lithuanian,\n            \"lv\" => Language::Latvian,\n            \"et\" => Language::Estonian,\n            \"tl\" => Language::Tagalog,\n            \"af\" => Language::Afrikaans,\n            \"zu\" => Language::Zulu,\n            \"sn\" => Language::Shona,\n            \"ak\" => Language::Akan,\n            \"ca\" => Language::Catalan,\n            \"el\" => Language::Greek,\n            \"sv\" => Language::Swedish,\n            \"pl\" => Language::Polish\n        )\n        .copied()\n    }\n}\n\nimpl Language {\n    pub fn detect(text: String, default: Language) -> (String, Language) {\n        if let Some((l, t)) = text\n            .split_once(':')\n            .and_then(|(l, t)| (Language::from_iso_639(l)?, t).into())\n        {\n            (t.to_string(), l)\n        } else {\n            let l = LanguageDetector::detect_single(&text)\n                .and_then(|(l, c)| if c > 0.3 { Some(l) } else { None })\n                .unwrap_or(default);\n            (text, l)\n        }\n    }\n}\n\nimpl ParseValue for Language {\n    fn parse_value(value: &str) -> utils::config::Result<Self> {\n        Language::from_iso_639(value).ok_or_else(|| format!(\"Invalid language code: {}\", value))\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/language/search_snippet.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::Language;\n\nfn escape_char(c: char, string: &mut String) {\n    match c {\n        '&' => string.push_str(\"&amp;\"),\n        '<' => string.push_str(\"&lt;\"),\n        '>' => string.push_str(\"&gt;\"),\n        '\"' => string.push_str(\"&quot;\"),\n        '\\n' | '\\r' => string.push(' '),\n        _ => string.push(c),\n    }\n}\n\nfn escape_char_len(c: char) -> usize {\n    match c {\n        '&' => \"&amp;\".len(),\n        '<' => \"&lt;\".len(),\n        '>' => \"&gt;\".len(),\n        '\"' => \"&quot;\".len(),\n        '\\r' | '\\n' => 1,\n        _ => c.len_utf8(),\n    }\n}\n\npub struct Term {\n    offset: usize,\n    len: usize,\n}\n\npub fn generate_snippet(\n    text: &str,\n    needles: &[impl AsRef<str>],\n    language: Language,\n    is_exact: bool,\n) -> Option<String> {\n    let mut terms = Vec::new();\n    if is_exact {\n        let tokens = language.tokenize_text(text, 200).collect::<Vec<_>>();\n        for tokens in tokens.windows(needles.len()) {\n            if needles\n                .iter()\n                .zip(tokens)\n                .all(|(needle, token)| needle.as_ref() == token.word.as_ref())\n            {\n                for token in tokens {\n                    terms.push(Term {\n                        offset: token.from,\n                        len: token.to - token.from,\n                    });\n                }\n            }\n        }\n    } else {\n        for token in language.tokenize_text(text, 200) {\n            if needles.iter().any(|needle| {\n                let needle = needle.as_ref();\n                needle == token.word.as_ref() || needle.len() > 2 && token.word.contains(needle)\n            }) {\n                terms.push(Term {\n                    offset: token.from,\n                    len: token.to - token.from,\n                });\n            }\n        }\n    }\n    if terms.is_empty() {\n        return None;\n    }\n\n    let mut snippet = String::with_capacity(text.len());\n    let start_offset = terms.first()?.offset;\n\n    if start_offset > 0 {\n        let mut word_count = 0;\n        let mut from_offset = 0;\n        let mut last_is_space = false;\n\n        if text.len() > 240 {\n            for (pos, char) in text.get(0..start_offset)?.char_indices().rev() {\n                // Add up to 2 words or 40 characters of context\n                if char.is_whitespace() {\n                    if !last_is_space {\n                        word_count += 1;\n                        if word_count == 3 {\n                            break;\n                        }\n                        last_is_space = true;\n                    }\n                } else {\n                    last_is_space = false;\n                }\n                from_offset = pos;\n                if start_offset - from_offset >= 40 {\n                    break;\n                }\n            }\n        }\n\n        last_is_space = false;\n        for char in text.get(from_offset..start_offset)?.chars() {\n            if !char.is_whitespace() {\n                last_is_space = false;\n            } else {\n                if last_is_space {\n                    continue;\n                }\n                last_is_space = true;\n            }\n            escape_char(char, &mut snippet);\n        }\n    }\n\n    let mut terms = terms.iter().peekable();\n\n    'outer: while let Some(term) = terms.next() {\n        if snippet.len() + (\"<mark>\".len() * 2) + term.len + 1 > 255 {\n            break;\n        }\n\n        snippet.push_str(\"<mark>\");\n        snippet.push_str(text.get(term.offset..term.offset + term.len)?);\n        snippet.push_str(\"</mark>\");\n\n        let next_offset = if let Some(next_term) = terms.peek() {\n            next_term.offset\n        } else {\n            text.len()\n        };\n\n        let mut last_is_space = false;\n        for char in text.get(term.offset + term.len..next_offset)?.chars() {\n            if !char.is_whitespace() {\n                last_is_space = false;\n            } else {\n                if last_is_space {\n                    continue;\n                }\n                last_is_space = true;\n            }\n\n            if snippet.len() + escape_char_len(char) <= 255 {\n                escape_char(char, &mut snippet);\n            } else {\n                break 'outer;\n            }\n        }\n    }\n\n    Some(snippet)\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::language::{Language, search_snippet::generate_snippet};\n\n    #[test]\n    fn search_snippets() {\n        let inputs = [\n            (\n                vec![\n                    \"Help a friend from Abidjan Côte d'Ivoire\",\n                    concat!(\n                        \"When my mother died when she was given birth to me, my father took me so \",\n                        \"special because I am motherless. Before the death of my late father on 22nd June \",\n                        \"2013 in a private hospital here in Abidjan Côte d'Ivoire. He secretly called me on his \",\n                        \"bedside and told me that he has a sum of $7.5M (Seven Million five Hundred \",\n                        \"Thousand Dollars) left in a suspense account in a local bank here in Abidjan Côte \",\n                        \"d'Ivoire, that he used my name as his only daughter for the next of kin in deposit of \",\n                        \"the fund. \",\n                        \"I am 24year old. Dear I am honorably seeking your assistance in the following ways. \",\n                        \"1) To provide any bank account where this money would be transferred into. \",\n                        \"2) To serve as the guardian of this fund. \",\n                        \"3) To make arrangement for me to come over to your country to further my \",\n                        \"education and to secure a residential permit for me in your country. \",\n                        \"Moreover, I am willing to offer you 30 percent of the total sum as compensation for \",\n                        \"your effort input after the successful transfer of this fund to your nominated \",\n                        \"account overseas.\"\n                    ),\n                ],\n                vec![\n                    (\n                        vec![\"côte\"],\n                        vec![\n                            \"Help a friend from Abidjan <mark>Côte</mark> d'Ivoire\",\n                            concat!(\n                                \"in Abidjan <mark>Côte</mark> d'Ivoire. He secretly called me on his bedside \",\n                                \"and told me that he has a sum of $7.5M (Seven Million five Hundred Thousand \",\n                                \"Dollars) left in a suspense account in a local bank here in Abidjan \",\n                                \"<mark>Côte</mark> d'Ivoire, that \"\n                            ),\n                        ],\n                    ),\n                    (\n                        vec![\"your\", \"country\"],\n                        vec![concat!(\n                            \"honorably seeking <mark>your</mark> assistance in the following ways. \",\n                            \"1) To provide any bank account where this money would be transferred into. 2) \",\n                            \"To serve as the guardian of this fund. 3) To make arrangement for me to come \",\n                            \"over to <mark>your</mark> \"\n                        )],\n                    ),\n                    (\n                        vec![\"overseas\"],\n                        vec![\"nominated account <mark>overseas</mark>.\"],\n                    ),\n                ],\n            ),\n            (\n                vec![\n                    \"孫子兵法\",\n                    concat!(\n                        \"<\\\"孫子兵法：\\\">\",\n                        \"孫子曰：兵者，國之大事，死生之地，存亡之道，不可不察也。\",\n                        \"孫子曰：凡用兵之法，馳車千駟，革車千乘，帶甲十萬；千里饋糧，則內外之費賓客之用，膠漆之材，\",\n                        \"車甲之奉，日費千金，然後十萬之師舉矣。\",\n                        \"孫子曰：凡用兵之法，全國為上，破國次之；全旅為上，破旅次之；全卒為上，破卒次之；全伍為上，破伍次之。\",\n                        \"是故百戰百勝，非善之善者也；不戰而屈人之兵，善之善者也。\",\n                        \"孫子曰：昔之善戰者，先為不可勝，以待敵之可勝，不可勝在己，可勝在敵。故善戰者，能為不可勝，不能使敵必可勝。\",\n                        \"故曰：勝可知，而不可為。\",\n                        \"兵者，詭道也。故能而示之不能，用而示之不用，近而示之遠，遠而示之近。利而誘之，亂而取之，實而備之，強而避之，\",\n                        \"怒而撓之，卑而驕之，佚而勞之，親而離之。攻其無備，出其不意，此兵家之勝，不可先傳也。\",\n                        \"夫未戰而廟算勝者，得算多也；未戰而廟算不勝者，得算少也；多算勝，少算不勝，而況於無算乎？吾以此觀之，勝負見矣。\",\n                        \"孫子曰：凡治眾如治寡，分數是也。鬥眾如鬥寡，形名是也。三軍之眾，可使必受敵而無敗者，奇正是也。兵之所加，\",\n                        \"如以碬投卵者，虛實是也。\",\n                    ),\n                ],\n                vec![\n                    (\n                        vec![\"孫子兵法\"],\n                        vec![\n                            \"<mark>孫子兵法</mark>\",\n                            concat!(\n                                \"&lt;&quot;<mark>孫子兵法</mark>：&quot;&gt;孫子曰：兵者，國之大事，死生之地，存亡之道，\",\n                                \"不可不察也。孫子曰：凡用兵之法，馳車千駟，革車千乘，帶甲十萬；千里饋糧，則內外之費賓客之用，膠\"\n                            ),\n                        ],\n                    ),\n                    (\n                        vec![\"孫子曰\"],\n                        vec![concat!(\n                            \"&lt;&quot;孫子兵法：&quot;&gt;<mark>孫子曰</mark>：兵者，國之大事，死生之地，存亡之道，\",\n                            \"不可不察也。<mark>孫子曰</mark>：凡用兵之法，馳車千駟，革車千乘，帶甲十萬；千里饋糧，則內外之費賓\",\n                        )],\n                    ),\n                ],\n            ),\n        ];\n\n        for (parts, tests) in inputs {\n            for (needles, snippets) in tests {\n                let mut results = Vec::new();\n\n                for part in &parts {\n                    if let Some(matched) =\n                        generate_snippet(part, &needles, Language::English, false)\n                    {\n                        results.push(matched);\n                    }\n                }\n\n                assert_eq!(snippets, results);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/language/stemmer.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse rust_stemmers::Algorithm;\n\nuse super::{Language, LanguageTokenizer};\n\n#[derive(Debug, PartialEq, Eq)]\npub struct StemmedToken<'x> {\n    pub word: Cow<'x, str>,\n    pub stemmed_word: Option<Cow<'x, str>>,\n    pub from: usize, // Word offset in the text part\n    pub to: usize,   // Word length\n}\n\npub struct Stemmer<'x> {\n    stemmer: Option<rust_stemmers::Stemmer>,\n    tokenizer: LanguageTokenizer<'x>,\n}\n\nimpl<'x> Stemmer<'x> {\n    pub fn new(text: &'x str, language: Language, max_token_length: usize) -> Stemmer<'x> {\n        Stemmer {\n            tokenizer: language.tokenize_text(text, max_token_length),\n            stemmer: STEMMER_MAP[language as usize].map(rust_stemmers::Stemmer::create),\n        }\n    }\n}\n\nimpl<'x> Iterator for Stemmer<'x> {\n    type Item = StemmedToken<'x>;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        let token = self.tokenizer.next()?;\n        Some(StemmedToken {\n            stemmed_word: self.stemmer.as_ref().and_then(|stemmer| {\n                match stemmer.stem(&token.word) {\n                    Cow::Owned(text) if text.len() != token.word.len() || text != token.word => {\n                        Some(text.into())\n                    }\n                    _ => None,\n                }\n            }),\n            word: token.word,\n            from: token.from,\n            to: token.to,\n        })\n    }\n}\n\npub static STEMMER_MAP: &[Option<Algorithm>] = &[\n    None,                        // Esperanto = 0,\n    Some(Algorithm::English),    // English = 1,\n    Some(Algorithm::Russian),    // Russian = 2,\n    None,                        // Mandarin = 3,\n    Some(Algorithm::Spanish),    // Spanish = 4,\n    Some(Algorithm::Portuguese), // Portuguese = 5,\n    Some(Algorithm::Italian),    // Italian = 6,\n    None,                        // Bengali = 7,\n    Some(Algorithm::French),     // French = 8,\n    Some(Algorithm::German),     // German = 9,\n    None,                        // Ukrainian = 10,\n    None,                        // Georgian = 11,\n    Some(Algorithm::Arabic),     // Arabic = 12,\n    None,                        // Hindi = 13,\n    None,                        // Japanese = 14,\n    None,                        // Hebrew = 15,\n    None,                        // Yiddish = 16,\n    None,                        // Polish = 17,\n    None,                        // Amharic = 18,\n    None,                        // Javanese = 19,\n    None,                        // Korean = 20,\n    Some(Algorithm::Norwegian),  // Bokmal = 21,\n    Some(Algorithm::Danish),     // Danish = 22,\n    Some(Algorithm::Swedish),    // Swedish = 23,\n    Some(Algorithm::Finnish),    // Finnish = 24,\n    Some(Algorithm::Turkish),    // Turkish = 25,\n    Some(Algorithm::Dutch),      // Dutch = 26,\n    Some(Algorithm::Hungarian),  // Hungarian = 27,\n    None,                        // Czech = 28,\n    Some(Algorithm::Greek),      // Greek = 29,\n    None,                        // Bulgarian = 30,\n    None,                        // Belarusian = 31,\n    None,                        // Marathi = 32,\n    None,                        // Kannada = 33,\n    Some(Algorithm::Romanian),   // Romanian = 34,\n    None,                        // Slovene = 35,\n    None,                        // Croatian = 36,\n    None,                        // Serbian = 37,\n    None,                        // Macedonian = 38,\n    None,                        // Lithuanian = 39,\n    None,                        // Latvian = 40,\n    None,                        // Estonian = 41,\n    Some(Algorithm::Tamil),      // Tamil = 42,\n    None,                        // Vietnamese = 43,\n    None,                        // Urdu = 44,\n    None,                        // Thai = 45,\n    None,                        // Gujarati = 46,\n    None,                        // Uzbek = 47,\n    None,                        // Punjabi = 48,\n    None,                        // Azerbaijani = 49,\n    None,                        // Indonesian = 50,\n    None,                        // Telugu = 51,\n    None,                        // Persian = 52,\n    None,                        // Malayalam = 53,\n    None,                        // Oriya = 54,\n    None,                        // Burmese = 55,\n    None,                        // Nepali = 56,\n    None,                        // Sinhalese = 57,\n    None,                        // Khmer = 58,\n    None,                        // Turkmen = 59,\n    None,                        // Akan = 60,\n    None,                        // Zulu = 61,\n    None,                        // Shona = 62,\n    None,                        // Afrikaans = 63,\n    None,                        // Latin = 64,\n    None,                        // Slovak = 65,\n    None,                        // Catalan = 66,\n    None,                        // Tagalog = 67,\n    None,                        // Armenian = 68,\n    None,                        // Unknown = 69,\n    None,                        // None = 70,\n];\n\n#[cfg(test)]\nmod tests {\n\n    use super::*;\n\n    #[test]\n    fn stemmer() {\n        let inputs = [\n            (\n                \"love loving lovingly loved lovely\",\n                Language::English,\n                \"love\",\n            ),\n            (\"querer queremos quer\", Language::Spanish, \"quer\"),\n        ];\n\n        for (input, language, result) in inputs {\n            for token in Stemmer::new(input, language, 40) {\n                assert_eq!(token.stemmed_word.unwrap_or(token.word), result);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/language/stopwords.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub type StopwordFnc = fn(&str) -> bool;\n\npub static STOP_WORDS: &[Option<StopwordFnc>] = &[\n    None,              // Esperanto = 0,\n    Some(english),     // English = 1,\n    Some(russian),     // Russian = 2,\n    None,              // Mandarin = 3,\n    Some(spanish),     // Spanish = 4,\n    Some(portuguese),  // Portuguese = 5,\n    Some(italian),     // Italian = 6,\n    None,              // Bengali = 7,\n    Some(french),      // French = 8,\n    Some(german),      // German = 9,\n    None,              // Ukrainian = 10,\n    None,              // Georgian = 11,\n    Some(arabic),      // Arabic = 12,\n    None,              // Hindi = 13,\n    None,              // Japanese = 14,\n    None,              // Hebrew = 15,\n    None,              // Yiddish = 16,\n    None,              // Polish = 17,\n    None,              // Amharic = 18,\n    None,              // Javanese = 19,\n    None,              // Korean = 20,\n    Some(norwegian),   // Bokmal = 21,\n    Some(danish),      // Danish = 22,\n    Some(swedish),     // Swedish = 23,\n    Some(finnish),     // Finnish = 24,\n    Some(turkish),     // Turkish = 25,\n    Some(dutch),       // Dutch = 26,\n    Some(hungarian),   // Hungarian = 27,\n    None,              // Czech = 28,\n    Some(greek),       // Greek = 29,\n    None,              // Bulgarian = 30,\n    None,              // Belarusian = 31,\n    None,              // Marathi = 32,\n    None,              // Kannada = 33,\n    Some(romanian),    // Romanian = 34,\n    None,              // Slovene = 35,\n    None,              // Croatian = 36,\n    None,              // Serbian = 37,\n    None,              // Macedonian = 38,\n    None,              // Lithuanian = 39,\n    None,              // Latvian = 40,\n    None,              // Estonian = 41,\n    None,              // Tamil = 42,\n    None,              // Vietnamese = 43,\n    None,              // Urdu = 44,\n    None,              // Thai = 45,\n    None,              // Gujarati = 46,\n    None,              // Uzbek = 47,\n    None,              // Punjabi = 48,\n    Some(azarbaijani), // Azerbaijani = 49,\n    None,              // Indonesian = 50,\n    None,              // Telugu = 51,\n    None,              // Persian = 52,\n    None,              // Malayalam = 53,\n    None,              // Oriya = 54,\n    None,              // Burmese = 55,\n    Some(nepali),      // Nepali = 56,\n    None,              // Sinhalese = 57,\n    None,              // Khmer = 58,\n    None,              // Turkmen = 59,\n    None,              // Akan = 60,\n    None,              // Zulu = 61,\n    None,              // Shona = 62,\n    None,              // Afrikaans = 63,\n    None,              // Latin = 64,\n    None,              // Slovak = 65,\n    None,              // Catalan = 66,\n    None,              // Tagalog = 67,\n    None,              // Armenian = 68,\n    None,              // Unknown = 69,\n    None,              // None = 70,\n];\n\nfn arabic(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"آه\",\n        \"آي\",\n        \"أف\",\n        \"أم\",\n        \"أن\",\n        \"أو\",\n        \"أي\",\n        \"إذ\",\n        \"إن\",\n        \"إي\",\n        \"بخ\",\n        \"بس\",\n        \"بك\",\n        \"بل\",\n        \"به\",\n        \"بي\",\n        \"ته\",\n        \"تي\",\n        \"ثم\",\n        \"ذا\",\n        \"ذه\",\n        \"ذو\",\n        \"ذي\",\n        \"عل\",\n        \"عن\",\n        \"في\",\n        \"قد\",\n        \"كل\",\n        \"كم\",\n        \"كي\",\n        \"لا\",\n        \"لك\",\n        \"لم\",\n        \"لن\",\n        \"له\",\n        \"لو\",\n        \"لي\",\n        \"ما\",\n        \"مذ\",\n        \"مع\",\n        \"من\",\n        \"مه\",\n        \"ها\",\n        \"هل\",\n        \"هم\",\n        \"هن\",\n        \"هو\",\n        \"هي\",\n        \"يا\",\n        \"آها\",\n        \"أقل\",\n        \"ألا\",\n        \"أما\",\n        \"أنا\",\n        \"أنت\",\n        \"أنى\",\n        \"أوه\",\n        \"أين\",\n        \"إذا\",\n        \"إذن\",\n        \"إلا\",\n        \"إلى\",\n        \"إما\",\n        \"إنا\",\n        \"إنه\",\n        \"إيه\",\n        \"بعد\",\n        \"بعض\",\n        \"بكم\",\n        \"بكن\",\n        \"بلى\",\n        \"بما\",\n        \"بمن\",\n        \"بنا\",\n        \"بها\",\n        \"بهم\",\n        \"بهن\",\n        \"بيد\",\n        \"بين\",\n        \"تلك\",\n        \"تين\",\n        \"ثمة\",\n        \"حتى\",\n        \"حيث\",\n        \"حين\",\n        \"خلا\",\n        \"دون\",\n        \"ذات\",\n        \"ذاك\",\n        \"ذان\",\n        \"ذلك\",\n        \"ذوا\",\n        \"ذين\",\n        \"ريث\",\n        \"سوف\",\n        \"سوى\",\n        \"عدا\",\n        \"عسى\",\n        \"على\",\n        \"عما\",\n        \"عند\",\n        \"غير\",\n        \"فإن\",\n        \"فلا\",\n        \"فمن\",\n        \"فيم\",\n        \"فيه\",\n        \"كأن\",\n        \"كأي\",\n        \"كذا\",\n        \"كلا\",\n        \"كما\",\n        \"كيت\",\n        \"كيف\",\n        \"لئن\",\n        \"لدى\",\n        \"لست\",\n        \"لسن\",\n        \"لعل\",\n        \"لكم\",\n        \"لكن\",\n        \"لكي\",\n        \"لما\",\n        \"لنا\",\n        \"لها\",\n        \"لهم\",\n        \"لهن\",\n        \"ليت\",\n        \"ليس\",\n        \"متى\",\n        \"مما\",\n        \"ممن\",\n        \"منذ\",\n        \"منه\",\n        \"نحن\",\n        \"نحو\",\n        \"نعم\",\n        \"هاك\",\n        \"هذا\",\n        \"هذه\",\n        \"هذي\",\n        \"هلا\",\n        \"هما\",\n        \"هنا\",\n        \"هيا\",\n        \"هيت\",\n        \"وإذ\",\n        \"وإن\",\n        \"ولا\",\n        \"ولو\",\n        \"وما\",\n        \"ومن\",\n        \"وهو\",\n        \"أكثر\",\n        \"أنتم\",\n        \"أنتن\",\n        \"أيها\",\n        \"إذما\",\n        \"إليك\",\n        \"إنما\",\n        \"التي\",\n        \"الذي\",\n        \"بكما\",\n        \"بهما\",\n        \"تلكم\",\n        \"تينك\",\n        \"حاشا\",\n        \"حبذا\",\n        \"ذانك\",\n        \"ذلكم\",\n        \"ذلكن\",\n        \"ذينك\",\n        \"شتان\",\n        \"عليك\",\n        \"عليه\",\n        \"فإذا\",\n        \"فيما\",\n        \"فيها\",\n        \"كأين\",\n        \"كذلك\",\n        \"كلتا\",\n        \"كلما\",\n        \"لستم\",\n        \"لستن\",\n        \"لسنا\",\n        \"لكما\",\n        \"لهما\",\n        \"لولا\",\n        \"لوما\",\n        \"ليسا\",\n        \"ليست\",\n        \"ماذا\",\n        \"منها\",\n        \"مهما\",\n        \"هاته\",\n        \"هاتي\",\n        \"هذان\",\n        \"هذين\",\n        \"هكذا\",\n        \"هناك\",\n        \"وإذا\",\n        \"ولكن\",\n        \"أنتما\",\n        \"أولئك\",\n        \"أولاء\",\n        \"أينما\",\n        \"إليكم\",\n        \"إليكن\",\n        \"الذين\",\n        \"بماذا\",\n        \"تلكما\",\n        \"حيثما\",\n        \"ذلكما\",\n        \"ذواتا\",\n        \"ذواتي\",\n        \"كأنما\",\n        \"كيفما\",\n        \"لستما\",\n        \"لكنما\",\n        \"لكيلا\",\n        \"ليستا\",\n        \"ليسوا\",\n        \"هؤلاء\",\n        \"هاتان\",\n        \"هاتين\",\n        \"هاهنا\",\n        \"هنالك\",\n        \"هيهات\",\n        \"والذي\",\n        \"إليكما\",\n        \"اللائي\",\n        \"اللاتي\",\n        \"اللتان\",\n        \"اللتيا\",\n        \"اللتين\",\n        \"اللذان\",\n        \"اللذين\",\n        \"كلاهما\",\n        \"كليكما\",\n        \"كليهما\",\n        \"لاسيما\",\n        \"والذين\",\n        \"اللواتي\",\n    )\n}\n\nfn azarbaijani(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"a\",\n        \"ad\",\n        \"altmış\",\n        \"altı\",\n        \"amma\",\n        \"arasında\",\n        \"artıq\",\n        \"ay\",\n        \"az\",\n        \"bax\",\n        \"belə\",\n        \"beş\",\n        \"bilər\",\n        \"bir\",\n        \"biraz\",\n        \"biri\",\n        \"birşey\",\n        \"biz\",\n        \"bizim\",\n        \"bizlər\",\n        \"bu\",\n        \"buna\",\n        \"bundan\",\n        \"bunların\",\n        \"bunu\",\n        \"bunun\",\n        \"buradan\",\n        \"bütün\",\n        \"bəli\",\n        \"bəlkə\",\n        \"bəy\",\n        \"bəzi\",\n        \"bəzən\",\n        \"ci\",\n        \"çox\",\n        \"cu\",\n        \"cü\",\n        \"çünki\",\n        \"cı\",\n        \"da\",\n        \"daha\",\n        \"dedi\",\n        \"deyil\",\n        \"dir\",\n        \"doqquz\",\n        \"doqsan\",\n        \"dörd\",\n        \"düz\",\n        \"də\",\n        \"dək\",\n        \"dən\",\n        \"dəqiqə\",\n        \"edir\",\n        \"edən\",\n        \"elə\",\n        \"et\",\n        \"etdi\",\n        \"etmə\",\n        \"etmək\",\n        \"faiz\",\n        \"gilə\",\n        \"görə\",\n        \"ha\",\n        \"haqqında\",\n        \"harada\",\n        \"heç\",\n        \"hə\",\n        \"həm\",\n        \"həmin\",\n        \"həmişə\",\n        \"hər\",\n        \"idi\",\n        \"iki\",\n        \"il\",\n        \"ildə\",\n        \"ilk\",\n        \"ilə\",\n        \"in\",\n        \"indi\",\n        \"istifadə\",\n        \"isə\",\n        \"iyirmi\",\n        \"ki\",\n        \"kim\",\n        \"kimi\",\n        \"kimə\",\n        \"lakin\",\n        \"lap\",\n        \"mirşey\",\n        \"məhz\",\n        \"mən\",\n        \"mənə\",\n        \"niyə\",\n        \"nə\",\n        \"nəhayət\",\n        \"o\",\n        \"obirisi\",\n        \"of\",\n        \"olan\",\n        \"olar\",\n        \"olaraq\",\n        \"oldu\",\n        \"olduğu\",\n        \"olmadı\",\n        \"olmaz\",\n        \"olmuşdur\",\n        \"olsun\",\n        \"olur\",\n        \"on\",\n        \"ona\",\n        \"ondan\",\n        \"onlar\",\n        \"onlardan\",\n        \"onların\",\n        \"onsuzda\",\n        \"onu\",\n        \"onun\",\n        \"oradan\",\n        \"otuz\",\n        \"öz\",\n        \"özü\",\n        \"qarşı\",\n        \"qədər\",\n        \"qırx\",\n        \"saat\",\n        \"sadəcə\",\n        \"saniyə\",\n        \"siz\",\n        \"sizin\",\n        \"sizlər\",\n        \"sonra\",\n        \"səhv\",\n        \"səkkiz\",\n        \"səksən\",\n        \"sən\",\n        \"sənin\",\n        \"sənə\",\n        \"təəssüf\",\n        \"ü\",\n        \"üç\",\n        \"üçün\",\n        \"var\",\n        \"və\",\n        \"xan\",\n        \"xanım\",\n        \"xeyr\",\n        \"ya\",\n        \"yalnız\",\n        \"yaxşı\",\n        \"yeddi\",\n        \"yenə\",\n        \"yetmiş\",\n        \"yox\",\n        \"yoxdur\",\n        \"yoxsa\",\n        \"yüz\",\n        \"yəni\",\n        \"zaman\",\n        \"ı\",\n        \"ə\",\n        \"əgər\",\n        \"əlbəttə\",\n        \"əlli\",\n        \"ən\",\n        \"əslində\",\n    )\n}\n\nfn danish(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"ad\",\n        \"af\",\n        \"alle\",\n        \"alt\",\n        \"anden\",\n        \"at\",\n        \"blev\",\n        \"blive\",\n        \"bliver\",\n        \"da\",\n        \"de\",\n        \"dem\",\n        \"den\",\n        \"denne\",\n        \"der\",\n        \"deres\",\n        \"det\",\n        \"dette\",\n        \"dig\",\n        \"din\",\n        \"disse\",\n        \"dog\",\n        \"du\",\n        \"efter\",\n        \"eller\",\n        \"en\",\n        \"end\",\n        \"er\",\n        \"et\",\n        \"for\",\n        \"fra\",\n        \"ham\",\n        \"han\",\n        \"hans\",\n        \"har\",\n        \"havde\",\n        \"have\",\n        \"hende\",\n        \"hendes\",\n        \"her\",\n        \"hos\",\n        \"hun\",\n        \"hvad\",\n        \"hvis\",\n        \"hvor\",\n        \"i\",\n        \"ikke\",\n        \"ind\",\n        \"jeg\",\n        \"jer\",\n        \"jo\",\n        \"kunne\",\n        \"man\",\n        \"mange\",\n        \"med\",\n        \"meget\",\n        \"men\",\n        \"mig\",\n        \"min\",\n        \"mine\",\n        \"mit\",\n        \"mod\",\n        \"når\",\n        \"ned\",\n        \"noget\",\n        \"nogle\",\n        \"nu\",\n        \"og\",\n        \"også\",\n        \"om\",\n        \"op\",\n        \"os\",\n        \"over\",\n        \"på\",\n        \"sådan\",\n        \"selv\",\n        \"sig\",\n        \"sin\",\n        \"sine\",\n        \"sit\",\n        \"skal\",\n        \"skulle\",\n        \"som\",\n        \"thi\",\n        \"til\",\n        \"ud\",\n        \"under\",\n        \"var\",\n        \"være\",\n        \"været\",\n        \"vi\",\n        \"vil\",\n        \"ville\",\n        \"vor\",\n    )\n}\n\nfn dutch(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"aan\",\n        \"al\",\n        \"alles\",\n        \"als\",\n        \"altijd\",\n        \"andere\",\n        \"ben\",\n        \"bij\",\n        \"daar\",\n        \"dan\",\n        \"dat\",\n        \"de\",\n        \"der\",\n        \"deze\",\n        \"die\",\n        \"dit\",\n        \"doch\",\n        \"doen\",\n        \"door\",\n        \"dus\",\n        \"een\",\n        \"eens\",\n        \"en\",\n        \"er\",\n        \"ge\",\n        \"geen\",\n        \"geweest\",\n        \"haar\",\n        \"had\",\n        \"heb\",\n        \"hebben\",\n        \"heeft\",\n        \"hem\",\n        \"het\",\n        \"hier\",\n        \"hij\",\n        \"hoe\",\n        \"hun\",\n        \"iemand\",\n        \"iets\",\n        \"ik\",\n        \"in\",\n        \"is\",\n        \"ja\",\n        \"je\",\n        \"kan\",\n        \"kon\",\n        \"kunnen\",\n        \"maar\",\n        \"me\",\n        \"meer\",\n        \"men\",\n        \"met\",\n        \"mij\",\n        \"mijn\",\n        \"moet\",\n        \"na\",\n        \"naar\",\n        \"niet\",\n        \"niets\",\n        \"nog\",\n        \"nu\",\n        \"of\",\n        \"om\",\n        \"omdat\",\n        \"onder\",\n        \"ons\",\n        \"ook\",\n        \"op\",\n        \"over\",\n        \"reeds\",\n        \"te\",\n        \"tegen\",\n        \"toch\",\n        \"toen\",\n        \"tot\",\n        \"u\",\n        \"uit\",\n        \"uw\",\n        \"van\",\n        \"veel\",\n        \"voor\",\n        \"want\",\n        \"waren\",\n        \"was\",\n        \"wat\",\n        \"werd\",\n        \"wezen\",\n        \"wie\",\n        \"wil\",\n        \"worden\",\n        \"wordt\",\n        \"zal\",\n        \"ze\",\n        \"zelf\",\n        \"zich\",\n        \"zij\",\n        \"zijn\",\n        \"zo\",\n        \"zonder\",\n        \"zou\",\n    )\n}\n\nfn english(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"a\",\n        \"about\",\n        \"above\",\n        \"after\",\n        \"again\",\n        \"against\",\n        \"ain\",\n        \"all\",\n        \"am\",\n        \"an\",\n        \"and\",\n        \"any\",\n        \"are\",\n        \"aren\",\n        \"aren't\",\n        \"as\",\n        \"at\",\n        \"be\",\n        \"because\",\n        \"been\",\n        \"before\",\n        \"being\",\n        \"below\",\n        \"between\",\n        \"both\",\n        \"but\",\n        \"by\",\n        \"can\",\n        \"couldn\",\n        \"couldn't\",\n        \"d\",\n        \"did\",\n        \"didn\",\n        \"didn't\",\n        \"do\",\n        \"does\",\n        \"doesn\",\n        \"doesn't\",\n        \"doing\",\n        \"don\",\n        \"don't\",\n        \"down\",\n        \"during\",\n        \"each\",\n        \"few\",\n        \"for\",\n        \"from\",\n        \"further\",\n        \"had\",\n        \"hadn\",\n        \"hadn't\",\n        \"has\",\n        \"hasn\",\n        \"hasn't\",\n        \"have\",\n        \"haven\",\n        \"haven't\",\n        \"having\",\n        \"he\",\n        \"her\",\n        \"here\",\n        \"hers\",\n        \"herself\",\n        \"him\",\n        \"himself\",\n        \"his\",\n        \"how\",\n        \"i\",\n        \"if\",\n        \"in\",\n        \"into\",\n        \"is\",\n        \"isn\",\n        \"isn't\",\n        \"it\",\n        \"it's\",\n        \"its\",\n        \"itself\",\n        \"just\",\n        \"ll\",\n        \"m\",\n        \"ma\",\n        \"me\",\n        \"mightn\",\n        \"mightn't\",\n        \"more\",\n        \"most\",\n        \"mustn\",\n        \"mustn't\",\n        \"my\",\n        \"myself\",\n        \"needn\",\n        \"needn't\",\n        \"no\",\n        \"nor\",\n        \"not\",\n        \"now\",\n        \"o\",\n        \"of\",\n        \"off\",\n        \"on\",\n        \"once\",\n        \"only\",\n        \"or\",\n        \"other\",\n        \"our\",\n        \"ours\",\n        \"ourselves\",\n        \"out\",\n        \"over\",\n        \"own\",\n        \"re\",\n        \"s\",\n        \"same\",\n        \"shan\",\n        \"shan't\",\n        \"she\",\n        \"she's\",\n        \"should\",\n        \"should've\",\n        \"shouldn\",\n        \"shouldn't\",\n        \"so\",\n        \"some\",\n        \"such\",\n        \"t\",\n        \"than\",\n        \"that\",\n        \"that'll\",\n        \"the\",\n        \"their\",\n        \"theirs\",\n        \"them\",\n        \"themselves\",\n        \"then\",\n        \"there\",\n        \"these\",\n        \"they\",\n        \"this\",\n        \"those\",\n        \"through\",\n        \"to\",\n        \"too\",\n        \"under\",\n        \"until\",\n        \"up\",\n        \"ve\",\n        \"very\",\n        \"was\",\n        \"wasn\",\n        \"wasn't\",\n        \"we\",\n        \"were\",\n        \"weren\",\n        \"weren't\",\n        \"what\",\n        \"when\",\n        \"where\",\n        \"which\",\n        \"while\",\n        \"who\",\n        \"whom\",\n        \"why\",\n        \"will\",\n        \"with\",\n        \"won\",\n        \"won't\",\n        \"wouldn\",\n        \"wouldn't\",\n        \"y\",\n        \"you\",\n        \"you'd\",\n        \"you'll\",\n        \"you're\",\n        \"you've\",\n        \"your\",\n        \"yours\",\n        \"yourself\",\n        \"yourselves\",\n    )\n}\n\nfn finnish(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"ei\",\n        \"eivät\",\n        \"emme\",\n        \"en\",\n        \"et\",\n        \"että\",\n        \"ette\",\n        \"hän\",\n        \"häneen\",\n        \"hänellä\",\n        \"hänelle\",\n        \"häneltä\",\n        \"hänen\",\n        \"hänessä\",\n        \"hänestä\",\n        \"hänet\",\n        \"häntä\",\n        \"he\",\n        \"heidän\",\n        \"heidät\",\n        \"heihin\",\n        \"heillä\",\n        \"heille\",\n        \"heiltä\",\n        \"heissä\",\n        \"heistä\",\n        \"heitä\",\n        \"itse\",\n        \"ja\",\n        \"johon\",\n        \"joiden\",\n        \"joihin\",\n        \"joiksi\",\n        \"joilla\",\n        \"joille\",\n        \"joilta\",\n        \"joina\",\n        \"joissa\",\n        \"joista\",\n        \"joita\",\n        \"joka\",\n        \"joksi\",\n        \"jolla\",\n        \"jolle\",\n        \"jolta\",\n        \"jona\",\n        \"jonka\",\n        \"jos\",\n        \"jossa\",\n        \"josta\",\n        \"jota\",\n        \"jotka\",\n        \"kanssa\",\n        \"keiden\",\n        \"keihin\",\n        \"keiksi\",\n        \"keillä\",\n        \"keille\",\n        \"keiltä\",\n        \"keinä\",\n        \"keissä\",\n        \"keistä\",\n        \"keitä\",\n        \"keneen\",\n        \"keneksi\",\n        \"kenellä\",\n        \"kenelle\",\n        \"keneltä\",\n        \"kenen\",\n        \"kenenä\",\n        \"kenessä\",\n        \"kenestä\",\n        \"kenet\",\n        \"ketä\",\n        \"ketkä\",\n        \"koska\",\n        \"kuin\",\n        \"kuka\",\n        \"kun\",\n        \"me\",\n        \"meidän\",\n        \"meidät\",\n        \"meihin\",\n        \"meillä\",\n        \"meille\",\n        \"meiltä\",\n        \"meissä\",\n        \"meistä\",\n        \"meitä\",\n        \"mihin\",\n        \"mikä\",\n        \"miksi\",\n        \"millä\",\n        \"mille\",\n        \"miltä\",\n        \"minä\",\n        \"minkä\",\n        \"minua\",\n        \"minulla\",\n        \"minulle\",\n        \"minulta\",\n        \"minun\",\n        \"minussa\",\n        \"minusta\",\n        \"minut\",\n        \"minuun\",\n        \"missä\",\n        \"mistä\",\n        \"mitä\",\n        \"mitkä\",\n        \"mukaan\",\n        \"mutta\",\n        \"näiden\",\n        \"näihin\",\n        \"näiksi\",\n        \"näillä\",\n        \"näille\",\n        \"näiltä\",\n        \"näinä\",\n        \"näissä\",\n        \"näistä\",\n        \"näitä\",\n        \"nämä\",\n        \"ne\",\n        \"niiden\",\n        \"niihin\",\n        \"niiksi\",\n        \"niillä\",\n        \"niille\",\n        \"niiltä\",\n        \"niin\",\n        \"niinä\",\n        \"niissä\",\n        \"niistä\",\n        \"niitä\",\n        \"noiden\",\n        \"noihin\",\n        \"noiksi\",\n        \"noilla\",\n        \"noille\",\n        \"noilta\",\n        \"noin\",\n        \"noina\",\n        \"noissa\",\n        \"noista\",\n        \"noita\",\n        \"nuo\",\n        \"nyt\",\n        \"ole\",\n        \"olemme\",\n        \"olen\",\n        \"olet\",\n        \"olette\",\n        \"oli\",\n        \"olimme\",\n        \"olin\",\n        \"olisi\",\n        \"olisimme\",\n        \"olisin\",\n        \"olisit\",\n        \"olisitte\",\n        \"olisivat\",\n        \"olit\",\n        \"olitte\",\n        \"olivat\",\n        \"olla\",\n        \"olleet\",\n        \"ollut\",\n        \"on\",\n        \"ovat\",\n        \"poikki\",\n        \"se\",\n        \"sekä\",\n        \"sen\",\n        \"siihen\",\n        \"siinä\",\n        \"siitä\",\n        \"siksi\",\n        \"sillä\",\n        \"sille\",\n        \"siltä\",\n        \"sinä\",\n        \"sinua\",\n        \"sinulla\",\n        \"sinulle\",\n        \"sinulta\",\n        \"sinun\",\n        \"sinussa\",\n        \"sinusta\",\n        \"sinut\",\n        \"sinuun\",\n        \"sitä\",\n        \"tähän\",\n        \"tai\",\n        \"täksi\",\n        \"tallä\",\n        \"tälle\",\n        \"tältä\",\n        \"tämä\",\n        \"tämän\",\n        \"tänä\",\n        \"tässä\",\n        \"tästä\",\n        \"tätä\",\n        \"te\",\n        \"teidän\",\n        \"teidät\",\n        \"teihin\",\n        \"teillä\",\n        \"teille\",\n        \"teiltä\",\n        \"teissä\",\n        \"teistä\",\n        \"teitä\",\n        \"tuo\",\n        \"tuohon\",\n        \"tuoksi\",\n        \"tuolla\",\n        \"tuolle\",\n        \"tuolta\",\n        \"tuon\",\n        \"tuona\",\n        \"tuossa\",\n        \"tuosta\",\n        \"tuotä\",\n        \"vaan\",\n        \"vai\",\n        \"vaikka\",\n        \"yli\",\n    )\n}\n\nfn french(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"à\",\n        \"ai\",\n        \"aie\",\n        \"aient\",\n        \"aies\",\n        \"ait\",\n        \"as\",\n        \"au\",\n        \"aura\",\n        \"aurai\",\n        \"auraient\",\n        \"aurais\",\n        \"aurait\",\n        \"auras\",\n        \"aurez\",\n        \"auriez\",\n        \"aurions\",\n        \"aurons\",\n        \"auront\",\n        \"aux\",\n        \"avaient\",\n        \"avais\",\n        \"avait\",\n        \"avec\",\n        \"avez\",\n        \"aviez\",\n        \"avions\",\n        \"avons\",\n        \"ayant\",\n        \"ayante\",\n        \"ayantes\",\n        \"ayants\",\n        \"ayez\",\n        \"ayons\",\n        \"c\",\n        \"ce\",\n        \"ces\",\n        \"d\",\n        \"dans\",\n        \"de\",\n        \"des\",\n        \"du\",\n        \"elle\",\n        \"en\",\n        \"es\",\n        \"est\",\n        \"et\",\n        \"étaient\",\n        \"étais\",\n        \"était\",\n        \"étant\",\n        \"étante\",\n        \"étantes\",\n        \"étants\",\n        \"été\",\n        \"étée\",\n        \"étées\",\n        \"étés\",\n        \"êtes\",\n        \"étiez\",\n        \"étions\",\n        \"eu\",\n        \"eue\",\n        \"eues\",\n        \"eûmes\",\n        \"eurent\",\n        \"eus\",\n        \"eusse\",\n        \"eussent\",\n        \"eusses\",\n        \"eussiez\",\n        \"eussions\",\n        \"eut\",\n        \"eût\",\n        \"eûtes\",\n        \"eux\",\n        \"fûmes\",\n        \"furent\",\n        \"fus\",\n        \"fusse\",\n        \"fussent\",\n        \"fusses\",\n        \"fussiez\",\n        \"fussions\",\n        \"fut\",\n        \"fût\",\n        \"fûtes\",\n        \"il\",\n        \"j\",\n        \"je\",\n        \"l\",\n        \"la\",\n        \"le\",\n        \"leur\",\n        \"lui\",\n        \"m\",\n        \"ma\",\n        \"mais\",\n        \"me\",\n        \"même\",\n        \"mes\",\n        \"moi\",\n        \"mon\",\n        \"n\",\n        \"ne\",\n        \"nos\",\n        \"notre\",\n        \"nous\",\n        \"on\",\n        \"ont\",\n        \"ou\",\n        \"par\",\n        \"pas\",\n        \"pour\",\n        \"qu\",\n        \"que\",\n        \"qui\",\n        \"s\",\n        \"sa\",\n        \"se\",\n        \"sera\",\n        \"serai\",\n        \"seraient\",\n        \"serais\",\n        \"serait\",\n        \"seras\",\n        \"serez\",\n        \"seriez\",\n        \"serions\",\n        \"serons\",\n        \"seront\",\n        \"ses\",\n        \"soient\",\n        \"sois\",\n        \"soit\",\n        \"sommes\",\n        \"son\",\n        \"sont\",\n        \"soyez\",\n        \"soyons\",\n        \"suis\",\n        \"sur\",\n        \"t\",\n        \"ta\",\n        \"te\",\n        \"tes\",\n        \"toi\",\n        \"ton\",\n        \"tu\",\n        \"un\",\n        \"une\",\n        \"vos\",\n        \"votre\",\n        \"vous\",\n        \"y\",\n    )\n}\n\nfn german(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"aber\",\n        \"alle\",\n        \"allem\",\n        \"allen\",\n        \"aller\",\n        \"alles\",\n        \"als\",\n        \"also\",\n        \"am\",\n        \"an\",\n        \"ander\",\n        \"andere\",\n        \"anderem\",\n        \"anderen\",\n        \"anderer\",\n        \"anderes\",\n        \"anderm\",\n        \"andern\",\n        \"anderr\",\n        \"anders\",\n        \"auch\",\n        \"auf\",\n        \"aus\",\n        \"bei\",\n        \"bin\",\n        \"bis\",\n        \"bist\",\n        \"da\",\n        \"damit\",\n        \"dann\",\n        \"das\",\n        \"dasselbe\",\n        \"dazu\",\n        \"daß\",\n        \"dein\",\n        \"deine\",\n        \"deinem\",\n        \"deinen\",\n        \"deiner\",\n        \"deines\",\n        \"dem\",\n        \"demselben\",\n        \"den\",\n        \"denn\",\n        \"denselben\",\n        \"der\",\n        \"derer\",\n        \"derselbe\",\n        \"derselben\",\n        \"des\",\n        \"desselben\",\n        \"dessen\",\n        \"dich\",\n        \"die\",\n        \"dies\",\n        \"diese\",\n        \"dieselbe\",\n        \"dieselben\",\n        \"diesem\",\n        \"diesen\",\n        \"dieser\",\n        \"dieses\",\n        \"dir\",\n        \"doch\",\n        \"dort\",\n        \"du\",\n        \"durch\",\n        \"ein\",\n        \"eine\",\n        \"einem\",\n        \"einen\",\n        \"einer\",\n        \"eines\",\n        \"einig\",\n        \"einige\",\n        \"einigem\",\n        \"einigen\",\n        \"einiger\",\n        \"einiges\",\n        \"einmal\",\n        \"er\",\n        \"es\",\n        \"etwas\",\n        \"euch\",\n        \"euer\",\n        \"eure\",\n        \"eurem\",\n        \"euren\",\n        \"eurer\",\n        \"eures\",\n        \"für\",\n        \"gegen\",\n        \"gewesen\",\n        \"hab\",\n        \"habe\",\n        \"haben\",\n        \"hat\",\n        \"hatte\",\n        \"hatten\",\n        \"hier\",\n        \"hin\",\n        \"hinter\",\n        \"ich\",\n        \"ihm\",\n        \"ihn\",\n        \"ihnen\",\n        \"ihr\",\n        \"ihre\",\n        \"ihrem\",\n        \"ihren\",\n        \"ihrer\",\n        \"ihres\",\n        \"im\",\n        \"in\",\n        \"indem\",\n        \"ins\",\n        \"ist\",\n        \"jede\",\n        \"jedem\",\n        \"jeden\",\n        \"jeder\",\n        \"jedes\",\n        \"jene\",\n        \"jenem\",\n        \"jenen\",\n        \"jener\",\n        \"jenes\",\n        \"jetzt\",\n        \"kann\",\n        \"kein\",\n        \"keine\",\n        \"keinem\",\n        \"keinen\",\n        \"keiner\",\n        \"keines\",\n        \"können\",\n        \"könnte\",\n        \"machen\",\n        \"man\",\n        \"manche\",\n        \"manchem\",\n        \"manchen\",\n        \"mancher\",\n        \"manches\",\n        \"mein\",\n        \"meine\",\n        \"meinem\",\n        \"meinen\",\n        \"meiner\",\n        \"meines\",\n        \"mich\",\n        \"mir\",\n        \"mit\",\n        \"muss\",\n        \"musste\",\n        \"nach\",\n        \"nicht\",\n        \"nichts\",\n        \"noch\",\n        \"nun\",\n        \"nur\",\n        \"ob\",\n        \"oder\",\n        \"ohne\",\n        \"sehr\",\n        \"sein\",\n        \"seine\",\n        \"seinem\",\n        \"seinen\",\n        \"seiner\",\n        \"seines\",\n        \"selbst\",\n        \"sich\",\n        \"sie\",\n        \"sind\",\n        \"so\",\n        \"solche\",\n        \"solchem\",\n        \"solchen\",\n        \"solcher\",\n        \"solches\",\n        \"soll\",\n        \"sollte\",\n        \"sondern\",\n        \"sonst\",\n        \"über\",\n        \"um\",\n        \"und\",\n        \"uns\",\n        \"unser\",\n        \"unsere\",\n        \"unserem\",\n        \"unseren\",\n        \"unseres\",\n        \"unter\",\n        \"viel\",\n        \"vom\",\n        \"von\",\n        \"vor\",\n        \"während\",\n        \"war\",\n        \"waren\",\n        \"warst\",\n        \"was\",\n        \"weg\",\n        \"weil\",\n        \"weiter\",\n        \"welche\",\n        \"welchem\",\n        \"welchen\",\n        \"welcher\",\n        \"welches\",\n        \"wenn\",\n        \"werde\",\n        \"werden\",\n        \"wie\",\n        \"wieder\",\n        \"will\",\n        \"wir\",\n        \"wird\",\n        \"wirst\",\n        \"wo\",\n        \"wollen\",\n        \"wollte\",\n        \"würde\",\n        \"würden\",\n        \"zu\",\n        \"zum\",\n        \"zur\",\n        \"zwar\",\n        \"zwischen\",\n    )\n}\n\nfn greek(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"η\",\n        \"κ\",\n        \"ο\",\n        \"ἃ\",\n        \"ἡ\",\n        \"ἢ\",\n        \"ἣ\",\n        \"ἤ\",\n        \"ἥ\",\n        \"ὁ\",\n        \"ὃ\",\n        \"ὅ\",\n        \"ὦ\",\n        \"ᾧ\",\n        \"δ'\",\n        \"αν\",\n        \"αἱ\",\n        \"αἳ\",\n        \"αἵ\",\n        \"αὖ\",\n        \"γα\",\n        \"γε\",\n        \"δέ\",\n        \"δή\",\n        \"δε\",\n        \"δὲ\",\n        \"δὴ\",\n        \"δ’\",\n        \"επ\",\n        \"εἰ\",\n        \"εἴ\",\n        \"θα\",\n        \"κι\",\n        \"μή\",\n        \"μα\",\n        \"με\",\n        \"μη\",\n        \"μὴ\",\n        \"να\",\n        \"οι\",\n        \"οἱ\",\n        \"οἳ\",\n        \"οὐ\",\n        \"οὗ\",\n        \"σε\",\n        \"σύ\",\n        \"σὺ\",\n        \"τά\",\n        \"τί\",\n        \"τα\",\n        \"τε\",\n        \"τι\",\n        \"το\",\n        \"τό\",\n        \"τὰ\",\n        \"τὸ\",\n        \"τῇ\",\n        \"τῷ\",\n        \"ωσ\",\n        \"ἀπ\",\n        \"ἀφ\",\n        \"ἂν\",\n        \"ἄν\",\n        \"ἐκ\",\n        \"ἐν\",\n        \"ἐξ\",\n        \"ἐφ\",\n        \"ἧς\",\n        \"ὃν\",\n        \"ὃς\",\n        \"ὅς\",\n        \"ὅσ\",\n        \"ὑπ\",\n        \"ὡς\",\n        \"ὡσ\",\n        \"ὥς\",\n        \"δι'\",\n        \"γα^\",\n        \"απο\",\n        \"γάρ\",\n        \"για\",\n        \"γὰρ\",\n        \"δαί\",\n        \"δαὶ\",\n        \"δεν\",\n        \"διά\",\n        \"διὰ\",\n        \"εαν\",\n        \"ενω\",\n        \"επι\",\n        \"εἰς\",\n        \"εἰσ\",\n        \"καί\",\n        \"καθ\",\n        \"και\",\n        \"κατ\",\n        \"καὶ\",\n        \"κἀν\",\n        \"κἂν\",\n        \"μέν\",\n        \"μεθ\",\n        \"μετ\",\n        \"μην\",\n        \"μἐν\",\n        \"μὲν\",\n        \"μὴν\",\n        \"οσο\",\n        \"οτι\",\n        \"οἷς\",\n        \"οὐδ\",\n        \"οὐκ\",\n        \"οὐχ\",\n        \"οὓς\",\n        \"οὖν\",\n        \"παρ\",\n        \"που\",\n        \"ποῦ\",\n        \"προ\",\n        \"πρὸ\",\n        \"πως\",\n        \"πωσ\",\n        \"στη\",\n        \"στο\",\n        \"σόσ\",\n        \"σύν\",\n        \"σὸς\",\n        \"σὺν\",\n        \"τήν\",\n        \"τίς\",\n        \"τίσ\",\n        \"την\",\n        \"τησ\",\n        \"τις\",\n        \"τισ\",\n        \"τοί\",\n        \"τοι\",\n        \"τον\",\n        \"του\",\n        \"τοῦ\",\n        \"των\",\n        \"τόν\",\n        \"τὰς\",\n        \"τὴν\",\n        \"τὸν\",\n        \"τῆς\",\n        \"τῆσ\",\n        \"τῶν\",\n        \"ἀπό\",\n        \"ἀπὸ\",\n        \"ἄρα\",\n        \"ἅμα\",\n        \"ἐάν\",\n        \"ἐγώ\",\n        \"ἐγὼ\",\n        \"ἐπί\",\n        \"ἐπὶ\",\n        \"ἐὰν\",\n        \"ἔτι\",\n        \"ἵνα\",\n        \"ὅδε\",\n        \"ὅτε\",\n        \"ὅτι\",\n        \"ὑπό\",\n        \"ὑπὸ\",\n        \"ἀλλ'\",\n        \"αλλα\",\n        \"αντι\",\n        \"αυτα\",\n        \"αυτη\",\n        \"αυτο\",\n        \"γοῦν\",\n        \"δαίσ\",\n        \"δαὶς\",\n        \"εἰμί\",\n        \"εἰμὶ\",\n        \"εἴμι\",\n        \"εἴτε\",\n        \"ισωσ\",\n        \"κατά\",\n        \"κατα\",\n        \"κατὰ\",\n        \"μήτε\",\n        \"μετά\",\n        \"μετα\",\n        \"μετὰ\",\n        \"ομωσ\",\n        \"οπωσ\",\n        \"οὐδέ\",\n        \"οὐδὲ\",\n        \"οὐχὶ\",\n        \"οὔτε\",\n        \"οὕτω\",\n        \"παρά\",\n        \"παρα\",\n        \"παρὰ\",\n        \"περί\",\n        \"περὶ\",\n        \"ποια\",\n        \"ποιο\",\n        \"ποτε\",\n        \"προσ\",\n        \"πρόσ\",\n        \"πρὸς\",\n        \"στην\",\n        \"στον\",\n        \"ταῖς\",\n        \"τινα\",\n        \"τοτε\",\n        \"τούσ\",\n        \"τοὺς\",\n        \"τοῖς\",\n        \"τότε\",\n        \"ἀλλά\",\n        \"ἀλλὰ\",\n        \"ἀλλ’\",\n        \"ἐμόσ\",\n        \"ἐμὸς\",\n        \"ἐπεὶ\",\n        \"ἐστι\",\n        \"ὅθεν\",\n        \"ὅπερ\",\n        \"ὑμόσ\",\n        \"ὑπέρ\",\n        \"ὑπὲρ\",\n        \"ὥστε\",\n        \"αυτεσ\",\n        \"αυτοι\",\n        \"αυτοσ\",\n        \"αυτων\",\n        \"αὐτόσ\",\n        \"αὐτὸς\",\n        \"ειμαι\",\n        \"ειναι\",\n        \"εισαι\",\n        \"ειστε\",\n        \"οὐδὲν\",\n        \"οὕτως\",\n        \"οὕτωσ\",\n        \"οὗτος\",\n        \"οὗτοσ\",\n        \"ποιεσ\",\n        \"ποιοι\",\n        \"ποιοσ\",\n        \"ποιων\",\n        \"ἄλλος\",\n        \"ἄλλοσ\",\n        \"ὅστις\",\n        \"ὅστισ\",\n        \"αυτουσ\",\n        \"εκεινα\",\n        \"εκεινη\",\n        \"εκεινο\",\n        \"καίτοι\",\n        \"οὐδείσ\",\n        \"οὐδεὶς\",\n        \"ποιουσ\",\n        \"ἑαυτοῦ\",\n        \"ειμαστε\",\n        \"εκεινεσ\",\n        \"εκεινοι\",\n        \"εκεινοσ\",\n        \"εκεινων\",\n        \"εκεινουσ\",\n        \"τοιοῦτος\",\n        \"τοιοῦτοσ\",\n    )\n}\n\nfn hungarian(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"a\",\n        \"abban\",\n        \"ahhoz\",\n        \"ahogy\",\n        \"ahol\",\n        \"aki\",\n        \"akik\",\n        \"akkor\",\n        \"alatt\",\n        \"által\",\n        \"általában\",\n        \"amely\",\n        \"amelyek\",\n        \"amelyekben\",\n        \"amelyeket\",\n        \"amelyet\",\n        \"amelynek\",\n        \"ami\",\n        \"amíg\",\n        \"amikor\",\n        \"amit\",\n        \"amolyan\",\n        \"annak\",\n        \"arra\",\n        \"arról\",\n        \"át\",\n        \"az\",\n        \"azért\",\n        \"azok\",\n        \"azon\",\n        \"azonban\",\n        \"azt\",\n        \"aztán\",\n        \"azután\",\n        \"azzal\",\n        \"bár\",\n        \"be\",\n        \"belül\",\n        \"benne\",\n        \"cikk\",\n        \"cikkek\",\n        \"cikkeket\",\n        \"csak\",\n        \"de\",\n        \"e\",\n        \"ebben\",\n        \"eddig\",\n        \"egész\",\n        \"egy\",\n        \"egyéb\",\n        \"egyes\",\n        \"egyetlen\",\n        \"egyik\",\n        \"egyre\",\n        \"ehhez\",\n        \"ekkor\",\n        \"el\",\n        \"elég\",\n        \"ellen\",\n        \"elõ\",\n        \"elõször\",\n        \"elõtt\",\n        \"elsõ\",\n        \"emilyen\",\n        \"én\",\n        \"ennek\",\n        \"éppen\",\n        \"erre\",\n        \"és\",\n        \"ez\",\n        \"ezek\",\n        \"ezen\",\n        \"ezért\",\n        \"ezt\",\n        \"ezzel\",\n        \"fel\",\n        \"felé\",\n        \"hanem\",\n        \"hiszen\",\n        \"hogy\",\n        \"hogyan\",\n        \"igen\",\n        \"így\",\n        \"ill\",\n        \"ill.\",\n        \"illetve\",\n        \"ilyen\",\n        \"ilyenkor\",\n        \"ismét\",\n        \"ison\",\n        \"itt\",\n        \"jó\",\n        \"jobban\",\n        \"jól\",\n        \"kell\",\n        \"kellett\",\n        \"keressünk\",\n        \"keresztül\",\n        \"ki\",\n        \"kívül\",\n        \"között\",\n        \"közül\",\n        \"legalább\",\n        \"legyen\",\n        \"lehet\",\n        \"lehetett\",\n        \"lenne\",\n        \"lenni\",\n        \"lesz\",\n        \"lett\",\n        \"maga\",\n        \"magát\",\n        \"majd\",\n        \"már\",\n        \"más\",\n        \"másik\",\n        \"meg\",\n        \"még\",\n        \"mellett\",\n        \"mely\",\n        \"melyek\",\n        \"mert\",\n        \"mi\",\n        \"miért\",\n        \"míg\",\n        \"mikor\",\n        \"milyen\",\n        \"minden\",\n        \"mindenki\",\n        \"mindent\",\n        \"mindig\",\n        \"mint\",\n        \"mintha\",\n        \"mit\",\n        \"mivel\",\n        \"most\",\n        \"nagy\",\n        \"nagyobb\",\n        \"nagyon\",\n        \"ne\",\n        \"néha\",\n        \"néhány\",\n        \"nekem\",\n        \"neki\",\n        \"nélkül\",\n        \"nem\",\n        \"nincs\",\n        \"õ\",\n        \"õk\",\n        \"õket\",\n        \"olyan\",\n        \"össze\",\n        \"ott\",\n        \"pedig\",\n        \"persze\",\n        \"rá\",\n        \"s\",\n        \"saját\",\n        \"sem\",\n        \"semmi\",\n        \"sok\",\n        \"sokat\",\n        \"sokkal\",\n        \"számára\",\n        \"szemben\",\n        \"szerint\",\n        \"szinte\",\n        \"talán\",\n        \"tehát\",\n        \"teljes\",\n        \"több\",\n        \"tovább\",\n        \"továbbá\",\n        \"úgy\",\n        \"ugyanis\",\n        \"új\",\n        \"újabb\",\n        \"újra\",\n        \"után\",\n        \"utána\",\n        \"utolsó\",\n        \"vagy\",\n        \"vagyis\",\n        \"vagyok\",\n        \"valaki\",\n        \"valami\",\n        \"valamint\",\n        \"való\",\n        \"van\",\n        \"vannak\",\n        \"vele\",\n        \"vissza\",\n        \"viszont\",\n        \"volna\",\n        \"volt\",\n        \"voltak\",\n        \"voltam\",\n        \"voltunk\",\n    )\n}\n\nfn italian(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"a\",\n        \"abbia\",\n        \"abbiamo\",\n        \"abbiano\",\n        \"abbiate\",\n        \"ad\",\n        \"agl\",\n        \"agli\",\n        \"ai\",\n        \"al\",\n        \"all\",\n        \"alla\",\n        \"alle\",\n        \"allo\",\n        \"anche\",\n        \"avemmo\",\n        \"avendo\",\n        \"avesse\",\n        \"avessero\",\n        \"avessi\",\n        \"avessimo\",\n        \"aveste\",\n        \"avesti\",\n        \"avete\",\n        \"aveva\",\n        \"avevamo\",\n        \"avevano\",\n        \"avevate\",\n        \"avevi\",\n        \"avevo\",\n        \"avrà\",\n        \"avrai\",\n        \"avranno\",\n        \"avrebbe\",\n        \"avrebbero\",\n        \"avrei\",\n        \"avremmo\",\n        \"avremo\",\n        \"avreste\",\n        \"avresti\",\n        \"avrete\",\n        \"avrò\",\n        \"avuta\",\n        \"avute\",\n        \"avuti\",\n        \"avuto\",\n        \"c\",\n        \"che\",\n        \"chi\",\n        \"ci\",\n        \"coi\",\n        \"col\",\n        \"come\",\n        \"con\",\n        \"contro\",\n        \"cui\",\n        \"da\",\n        \"dagl\",\n        \"dagli\",\n        \"dai\",\n        \"dal\",\n        \"dall\",\n        \"dalla\",\n        \"dalle\",\n        \"dallo\",\n        \"degl\",\n        \"degli\",\n        \"dei\",\n        \"del\",\n        \"dell\",\n        \"della\",\n        \"delle\",\n        \"dello\",\n        \"di\",\n        \"dov\",\n        \"dove\",\n        \"e\",\n        \"è\",\n        \"ebbe\",\n        \"ebbero\",\n        \"ebbi\",\n        \"ed\",\n        \"era\",\n        \"erano\",\n        \"eravamo\",\n        \"eravate\",\n        \"eri\",\n        \"ero\",\n        \"essendo\",\n        \"faccia\",\n        \"facciamo\",\n        \"facciano\",\n        \"facciate\",\n        \"faccio\",\n        \"facemmo\",\n        \"facendo\",\n        \"facesse\",\n        \"facessero\",\n        \"facessi\",\n        \"facessimo\",\n        \"faceste\",\n        \"facesti\",\n        \"faceva\",\n        \"facevamo\",\n        \"facevano\",\n        \"facevate\",\n        \"facevi\",\n        \"facevo\",\n        \"fai\",\n        \"fanno\",\n        \"farà\",\n        \"farai\",\n        \"faranno\",\n        \"farebbe\",\n        \"farebbero\",\n        \"farei\",\n        \"faremmo\",\n        \"faremo\",\n        \"fareste\",\n        \"faresti\",\n        \"farete\",\n        \"farò\",\n        \"fece\",\n        \"fecero\",\n        \"feci\",\n        \"fosse\",\n        \"fossero\",\n        \"fossi\",\n        \"fossimo\",\n        \"foste\",\n        \"fosti\",\n        \"fu\",\n        \"fui\",\n        \"fummo\",\n        \"furono\",\n        \"gli\",\n        \"ha\",\n        \"hai\",\n        \"hanno\",\n        \"ho\",\n        \"i\",\n        \"il\",\n        \"in\",\n        \"io\",\n        \"l\",\n        \"la\",\n        \"le\",\n        \"lei\",\n        \"li\",\n        \"lo\",\n        \"loro\",\n        \"lui\",\n        \"ma\",\n        \"mi\",\n        \"mia\",\n        \"mie\",\n        \"miei\",\n        \"mio\",\n        \"ne\",\n        \"negl\",\n        \"negli\",\n        \"nei\",\n        \"nel\",\n        \"nell\",\n        \"nella\",\n        \"nelle\",\n        \"nello\",\n        \"noi\",\n        \"non\",\n        \"nostra\",\n        \"nostre\",\n        \"nostri\",\n        \"nostro\",\n        \"o\",\n        \"per\",\n        \"perché\",\n        \"più\",\n        \"quale\",\n        \"quanta\",\n        \"quante\",\n        \"quanti\",\n        \"quanto\",\n        \"quella\",\n        \"quelle\",\n        \"quelli\",\n        \"quello\",\n        \"questa\",\n        \"queste\",\n        \"questi\",\n        \"questo\",\n        \"sarà\",\n        \"sarai\",\n        \"saranno\",\n        \"sarebbe\",\n        \"sarebbero\",\n        \"sarei\",\n        \"saremmo\",\n        \"saremo\",\n        \"sareste\",\n        \"saresti\",\n        \"sarete\",\n        \"sarò\",\n        \"se\",\n        \"sei\",\n        \"si\",\n        \"sia\",\n        \"siamo\",\n        \"siano\",\n        \"siate\",\n        \"siete\",\n        \"sono\",\n        \"sta\",\n        \"stai\",\n        \"stando\",\n        \"stanno\",\n        \"starà\",\n        \"starai\",\n        \"staranno\",\n        \"starebbe\",\n        \"starebbero\",\n        \"starei\",\n        \"staremmo\",\n        \"staremo\",\n        \"stareste\",\n        \"staresti\",\n        \"starete\",\n        \"starò\",\n        \"stava\",\n        \"stavamo\",\n        \"stavano\",\n        \"stavate\",\n        \"stavi\",\n        \"stavo\",\n        \"stemmo\",\n        \"stesse\",\n        \"stessero\",\n        \"stessi\",\n        \"stessimo\",\n        \"steste\",\n        \"stesti\",\n        \"stette\",\n        \"stettero\",\n        \"stetti\",\n        \"stia\",\n        \"stiamo\",\n        \"stiano\",\n        \"stiate\",\n        \"sto\",\n        \"su\",\n        \"sua\",\n        \"sue\",\n        \"sugl\",\n        \"sugli\",\n        \"sui\",\n        \"sul\",\n        \"sull\",\n        \"sulla\",\n        \"sulle\",\n        \"sullo\",\n        \"suo\",\n        \"suoi\",\n        \"ti\",\n        \"tra\",\n        \"tu\",\n        \"tua\",\n        \"tue\",\n        \"tuo\",\n        \"tuoi\",\n        \"tutti\",\n        \"tutto\",\n        \"un\",\n        \"una\",\n        \"uno\",\n        \"vi\",\n        \"voi\",\n        \"vostra\",\n        \"vostre\",\n        \"vostri\",\n        \"vostro\",\n    )\n}\n\nfn norwegian(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"å\",\n        \"alle\",\n        \"at\",\n        \"av\",\n        \"både\",\n        \"båe\",\n        \"bare\",\n        \"begge\",\n        \"ble\",\n        \"blei\",\n        \"bli\",\n        \"blir\",\n        \"blitt\",\n        \"da\",\n        \"då\",\n        \"de\",\n        \"deg\",\n        \"dei\",\n        \"deim\",\n        \"deira\",\n        \"deires\",\n        \"dem\",\n        \"den\",\n        \"denne\",\n        \"der\",\n        \"dere\",\n        \"deres\",\n        \"det\",\n        \"dette\",\n        \"di\",\n        \"din\",\n        \"disse\",\n        \"ditt\",\n        \"du\",\n        \"dykk\",\n        \"dykkar\",\n        \"eg\",\n        \"ein\",\n        \"eit\",\n        \"eitt\",\n        \"eller\",\n        \"elles\",\n        \"en\",\n        \"enn\",\n        \"er\",\n        \"et\",\n        \"ett\",\n        \"etter\",\n        \"for\",\n        \"før\",\n        \"fordi\",\n        \"fra\",\n        \"ha\",\n        \"hadde\",\n        \"han\",\n        \"hans\",\n        \"har\",\n        \"hennar\",\n        \"henne\",\n        \"hennes\",\n        \"her\",\n        \"hjå\",\n        \"ho\",\n        \"hoe\",\n        \"honom\",\n        \"hoss\",\n        \"hossen\",\n        \"hun\",\n        \"hva\",\n        \"hvem\",\n        \"hver\",\n        \"hvilke\",\n        \"hvilken\",\n        \"hvis\",\n        \"hvor\",\n        \"hvordan\",\n        \"hvorfor\",\n        \"i\",\n        \"ikke\",\n        \"ikkje\",\n        \"ingen\",\n        \"ingi\",\n        \"inkje\",\n        \"inn\",\n        \"inni\",\n        \"ja\",\n        \"jeg\",\n        \"kan\",\n        \"kom\",\n        \"korleis\",\n        \"korso\",\n        \"kun\",\n        \"kunne\",\n        \"kva\",\n        \"kvar\",\n        \"kvarhelst\",\n        \"kven\",\n        \"kvi\",\n        \"kvifor\",\n        \"man\",\n        \"mange\",\n        \"me\",\n        \"med\",\n        \"medan\",\n        \"meg\",\n        \"meget\",\n        \"mellom\",\n        \"men\",\n        \"mi\",\n        \"min\",\n        \"mine\",\n        \"mitt\",\n        \"mot\",\n        \"mykje\",\n        \"nå\",\n        \"når\",\n        \"ned\",\n        \"no\",\n        \"noe\",\n        \"noen\",\n        \"noka\",\n        \"noko\",\n        \"nokon\",\n        \"nokor\",\n        \"nokre\",\n        \"og\",\n        \"også\",\n        \"om\",\n        \"opp\",\n        \"oss\",\n        \"over\",\n        \"på\",\n        \"så\",\n        \"samme\",\n        \"sånn\",\n        \"seg\",\n        \"selv\",\n        \"si\",\n        \"sia\",\n        \"sidan\",\n        \"siden\",\n        \"sin\",\n        \"sine\",\n        \"sitt\",\n        \"sjøl\",\n        \"skal\",\n        \"skulle\",\n        \"slik\",\n        \"so\",\n        \"som\",\n        \"somme\",\n        \"somt\",\n        \"til\",\n        \"um\",\n        \"upp\",\n        \"ut\",\n        \"uten\",\n        \"var\",\n        \"vår\",\n        \"være\",\n        \"vart\",\n        \"vært\",\n        \"varte\",\n        \"ved\",\n        \"vere\",\n        \"verte\",\n        \"vi\",\n        \"vil\",\n        \"ville\",\n        \"vore\",\n        \"vors\",\n        \"vort\",\n    )\n}\n\nfn nepali(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"छ\",\n        \"त\",\n        \"न\",\n        \"म\",\n        \"र\",\n        \"अब\",\n        \"आए\",\n        \"उप\",\n        \"एक\",\n        \"ओठ\",\n        \"औं\",\n        \"का\",\n        \"कि\",\n        \"के\",\n        \"को\",\n        \"गए\",\n        \"छु\",\n        \"छू\",\n        \"जब\",\n        \"जे\",\n        \"जो\",\n        \"तर\",\n        \"तल\",\n        \"ती\",\n        \"नि\",\n        \"नै\",\n        \"नौ\",\n        \"भए\",\n        \"भन\",\n        \"भर\",\n        \"मा\",\n        \"यस\",\n        \"या\",\n        \"यी\",\n        \"यो\",\n        \"ले\",\n        \"सो\",\n        \"हो\",\n        \"कम से कम\",\n        \"अझै\",\n        \"अरु\",\n        \"अलग\",\n        \"आदि\",\n        \"आफू\",\n        \"आयो\",\n        \"कतै\",\n        \"कसै\",\n        \"किन\",\n        \"गयौ\",\n        \"गरि\",\n        \"गरी\",\n        \"गैर\",\n        \"चार\",\n        \"छन्\",\n        \"छैन\",\n        \"छौं\",\n        \"जान\",\n        \"जुन\",\n        \"ठीक\",\n        \"तथा\",\n        \"तिर\",\n        \"तीन\",\n        \"थिए\",\n        \"दिए\",\n        \"दुई\",\n        \"पछि\",\n        \"पटक\",\n        \"पनि\",\n        \"बने\",\n        \"बरु\",\n        \"बीच\",\n        \"भने\",\n        \"भन्\",\n        \"यति\",\n        \"यदि\",\n        \"यसो\",\n        \"रही\",\n        \"रूप\",\n        \"लाई\",\n        \"संग\",\n        \"सधै\",\n        \"सबै\",\n        \"समय\",\n        \"सही\",\n        \"सात\",\n        \"साथ\",\n        \"हरे\",\n        \"हुन\",\n        \"अन्य\",\n        \"आजको\",\n        \"आत्म\",\n        \"उनको\",\n        \"उनले\",\n        \"एउटै\",\n        \"एकदम\",\n        \"कसरी\",\n        \"कुनै\",\n        \"कुरा\",\n        \"केही\",\n        \"कोही\",\n        \"गरेर\",\n        \"गरौं\",\n        \"गर्छ\",\n        \"गर्न\",\n        \"चाले\",\n        \"जबकि\",\n        \"जसको\",\n        \"जसमा\",\n        \"जसले\",\n        \"जहाँ\",\n        \"तपाई\",\n        \"तिनी\",\n        \"तिमी\",\n        \"त्यो\",\n        \"थिएन\",\n        \"थियो\",\n        \"देखि\",\n        \"देखे\",\n        \"धेरै\",\n        \"नत्र\",\n        \"नयाँ\",\n        \"पर्छ\",\n        \"पाँच\",\n        \"प्लस\",\n        \"फेरी\",\n        \"बारे\",\n        \"भएको\",\n        \"मलाई\",\n        \"माथि\",\n        \"मेरो\",\n        \"यसको\",\n        \"यसरी\",\n        \"यहाँ\",\n        \"राखे\",\n        \"लगभग\",\n        \"लागि\",\n        \"शायद\",\n        \"संगै\",\n        \"सक्छ\",\n        \"सम्म\",\n        \"साथै\",\n        \"सायद\",\n        \"सारा\",\n        \"सोही\",\n        \"हरेक\",\n        \"हुने\",\n        \"हुन्\",\n        \"अक्सर\",\n        \"अगाडी\",\n        \"अर्को\",\n        \"आफ्नै\",\n        \"आफ्नो\",\n        \"कसैले\",\n        \"कृपया\",\n        \"गरेका\",\n        \"गरेको\",\n        \"गर्छु\",\n        \"गर्दै\",\n        \"गर्नु\",\n        \"गर्ने\",\n        \"चाहिए\",\n        \"जसबाट\",\n        \"जसलाई\",\n        \"जस्तै\",\n        \"जस्तो\",\n        \"जाहिर\",\n        \"तापनी\",\n        \"देखेर\",\n        \"नजिकै\",\n        \"निम्न\",\n        \"पक्का\",\n        \"पक्कै\",\n        \"पहिले\",\n        \"पहिलो\",\n        \"पूर्व\",\n        \"प्रति\",\n        \"बाहिर\",\n        \"बाहेक\",\n        \"बिशेष\",\n        \"बीचमा\",\n        \"भन्छु\",\n        \"भन्दा\",\n        \"भन्ने\",\n        \"भित्र\",\n        \"मात्र\",\n        \"मुख्य\",\n        \"यसपछि\",\n        \"यस्तो\",\n        \"रहेका\",\n        \"रहेको\",\n        \"राख्छ\",\n        \"सट्टा\",\n        \"सम्भव\",\n        \"हुन्छ\",\n        \"अनुसार\",\n        \"अन्यथा\",\n        \"अरुलाई\",\n        \"अर्थात\",\n        \"आफूलाई\",\n        \"उदाहरण\",\n        \"उहालाई\",\n        \"किनभने\",\n        \"क्रमशः\",\n        \"जताततै\",\n        \"तत्काल\",\n        \"तपाईको\",\n        \"तेस्रो\",\n        \"त्यहाँ\",\n        \"त्सपछि\",\n        \"त्सैले\",\n        \"देखियो\",\n        \"देखेको\",\n        \"दोस्रो\",\n        \"निम्ति\",\n        \"पाँचौं\",\n        \"प्रतेक\",\n        \"भन्छन्\",\n        \"भित्री\",\n        \"यथोचित\",\n        \"यद्यपि\",\n        \"राम्रो\",\n        \"वरीपरी\",\n        \"सबैलाई\",\n        \"स्पष्ट\",\n        \"अन्यत्र\",\n        \"अर्थात्\",\n        \"कहाँबाट\",\n        \"चाहन्छु\",\n        \"तदनुसार\",\n        \"तिनीहरू\",\n        \"देखिन्छ\",\n        \"पछिल्लो\",\n        \"पर्थ्यो\",\n        \"पहिल्यै\",\n        \"बिरुद्ध\",\n        \"यसबाहेक\",\n        \"साँच्चै\",\n        \"अन्तर्गत\",\n        \"तुरुन्तै\",\n        \"तेस्कारण\",\n        \"दिनुभएको\",\n        \"पर्याप्त\",\n        \"भन्नुभयो\",\n        \"यहाँसम्म\",\n        \"वास्तवमा\",\n        \"गर्नुपर्छ\",\n        \"जस्तोसुकै\",\n        \"तिनीहरुको\",\n        \"दिनुहुन्छ\",\n        \"निर्दिष्ट\",\n        \"कहिलेकाहीं\",\n        \"चाहनुहुन्छ\",\n        \"तिनिहरुलाई\",\n        \"निम्नानुसार\",\n    )\n}\n\nfn portuguese(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"a\",\n        \"à\",\n        \"ao\",\n        \"aos\",\n        \"aquela\",\n        \"aquelas\",\n        \"aquele\",\n        \"aqueles\",\n        \"aquilo\",\n        \"as\",\n        \"às\",\n        \"até\",\n        \"com\",\n        \"como\",\n        \"da\",\n        \"das\",\n        \"de\",\n        \"dela\",\n        \"delas\",\n        \"dele\",\n        \"deles\",\n        \"depois\",\n        \"do\",\n        \"dos\",\n        \"e\",\n        \"ela\",\n        \"elas\",\n        \"ele\",\n        \"eles\",\n        \"em\",\n        \"entre\",\n        \"era\",\n        \"eram\",\n        \"éramos\",\n        \"essa\",\n        \"essas\",\n        \"esse\",\n        \"esses\",\n        \"esta\",\n        \"está\",\n        \"estamos\",\n        \"estão\",\n        \"estas\",\n        \"estava\",\n        \"estavam\",\n        \"estávamos\",\n        \"este\",\n        \"esteja\",\n        \"estejam\",\n        \"estejamos\",\n        \"estes\",\n        \"esteve\",\n        \"estive\",\n        \"estivemos\",\n        \"estiver\",\n        \"estivera\",\n        \"estiveram\",\n        \"estivéramos\",\n        \"estiverem\",\n        \"estivermos\",\n        \"estivesse\",\n        \"estivessem\",\n        \"estivéssemos\",\n        \"estou\",\n        \"eu\",\n        \"foi\",\n        \"fomos\",\n        \"for\",\n        \"fora\",\n        \"foram\",\n        \"fôramos\",\n        \"forem\",\n        \"formos\",\n        \"fosse\",\n        \"fossem\",\n        \"fôssemos\",\n        \"fui\",\n        \"há\",\n        \"haja\",\n        \"hajam\",\n        \"hajamos\",\n        \"hão\",\n        \"havemos\",\n        \"hei\",\n        \"houve\",\n        \"houvemos\",\n        \"houver\",\n        \"houvera\",\n        \"houverá\",\n        \"houveram\",\n        \"houvéramos\",\n        \"houverão\",\n        \"houverei\",\n        \"houverem\",\n        \"houveremos\",\n        \"houveria\",\n        \"houveriam\",\n        \"houveríamos\",\n        \"houvermos\",\n        \"houvesse\",\n        \"houvessem\",\n        \"houvéssemos\",\n        \"isso\",\n        \"isto\",\n        \"já\",\n        \"lhe\",\n        \"lhes\",\n        \"mais\",\n        \"mas\",\n        \"me\",\n        \"mesmo\",\n        \"meu\",\n        \"meus\",\n        \"minha\",\n        \"minhas\",\n        \"muito\",\n        \"na\",\n        \"não\",\n        \"nas\",\n        \"nem\",\n        \"no\",\n        \"nos\",\n        \"nós\",\n        \"nossa\",\n        \"nossas\",\n        \"nosso\",\n        \"nossos\",\n        \"num\",\n        \"numa\",\n        \"o\",\n        \"os\",\n        \"ou\",\n        \"para\",\n        \"pela\",\n        \"pelas\",\n        \"pelo\",\n        \"pelos\",\n        \"por\",\n        \"qual\",\n        \"quando\",\n        \"que\",\n        \"quem\",\n        \"são\",\n        \"se\",\n        \"seja\",\n        \"sejam\",\n        \"sejamos\",\n        \"sem\",\n        \"será\",\n        \"serão\",\n        \"serei\",\n        \"seremos\",\n        \"seria\",\n        \"seriam\",\n        \"seríamos\",\n        \"seu\",\n        \"seus\",\n        \"só\",\n        \"somos\",\n        \"sou\",\n        \"sua\",\n        \"suas\",\n        \"também\",\n        \"te\",\n        \"tem\",\n        \"tém\",\n        \"temos\",\n        \"tenha\",\n        \"tenham\",\n        \"tenhamos\",\n        \"tenho\",\n        \"terá\",\n        \"terão\",\n        \"terei\",\n        \"teremos\",\n        \"teria\",\n        \"teriam\",\n        \"teríamos\",\n        \"teu\",\n        \"teus\",\n        \"teve\",\n        \"tinha\",\n        \"tinham\",\n        \"tínhamos\",\n        \"tive\",\n        \"tivemos\",\n        \"tiver\",\n        \"tivera\",\n        \"tiveram\",\n        \"tivéramos\",\n        \"tiverem\",\n        \"tivermos\",\n        \"tivesse\",\n        \"tivessem\",\n        \"tivéssemos\",\n        \"tu\",\n        \"tua\",\n        \"tuas\",\n        \"um\",\n        \"uma\",\n        \"você\",\n        \"vocês\",\n        \"vos\",\n    )\n}\n\nfn romanian(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"a\",\n        \"abia\",\n        \"acea\",\n        \"aceasta\",\n        \"această\",\n        \"aceea\",\n        \"aceeasi\",\n        \"acei\",\n        \"aceia\",\n        \"acel\",\n        \"acela\",\n        \"acelasi\",\n        \"acele\",\n        \"acelea\",\n        \"acest\",\n        \"acesta\",\n        \"aceste\",\n        \"acestea\",\n        \"acestei\",\n        \"acestia\",\n        \"acestui\",\n        \"aceşti\",\n        \"aceştia\",\n        \"adica\",\n        \"ai\",\n        \"aia\",\n        \"aibă\",\n        \"aici\",\n        \"al\",\n        \"ala\",\n        \"ale\",\n        \"alea\",\n        \"alt\",\n        \"alta\",\n        \"altceva\",\n        \"altcineva\",\n        \"alte\",\n        \"altfel\",\n        \"alti\",\n        \"altii\",\n        \"altul\",\n        \"am\",\n        \"anume\",\n        \"apoi\",\n        \"ar\",\n        \"are\",\n        \"as\",\n        \"asa\",\n        \"asta\",\n        \"astea\",\n        \"astfel\",\n        \"asupra\",\n        \"atare\",\n        \"atat\",\n        \"atata\",\n        \"atatea\",\n        \"atatia\",\n        \"ati\",\n        \"atit\",\n        \"atita\",\n        \"atitea\",\n        \"atitia\",\n        \"atunci\",\n        \"au\",\n        \"avea\",\n        \"avem\",\n        \"aveţi\",\n        \"avut\",\n        \"aş\",\n        \"aţi\",\n        \"ba\",\n        \"ca\",\n        \"cam\",\n        \"cand\",\n        \"care\",\n        \"careia\",\n        \"carora\",\n        \"caruia\",\n        \"cat\",\n        \"cât\",\n        \"câte\",\n        \"catre\",\n        \"câtva\",\n        \"câţi\",\n        \"ce\",\n        \"cea\",\n        \"ceea\",\n        \"cei\",\n        \"ceilalti\",\n        \"cel\",\n        \"cele\",\n        \"celor\",\n        \"ceva\",\n        \"chiar\",\n        \"ci\",\n        \"cind\",\n        \"cînd\",\n        \"cine\",\n        \"cineva\",\n        \"cit\",\n        \"cît\",\n        \"cita\",\n        \"cite\",\n        \"cîte\",\n        \"citeva\",\n        \"citi\",\n        \"citiva\",\n        \"cîtva\",\n        \"cîţi\",\n        \"cu\",\n        \"cui\",\n        \"cum\",\n        \"cumva\",\n        \"că\",\n        \"căci\",\n        \"cărei\",\n        \"căror\",\n        \"cărui\",\n        \"către\",\n        \"da\",\n        \"daca\",\n        \"dacă\",\n        \"dar\",\n        \"dat\",\n        \"dată\",\n        \"dau\",\n        \"de\",\n        \"deasupra\",\n        \"deci\",\n        \"decit\",\n        \"deja\",\n        \"desi\",\n        \"despre\",\n        \"deşi\",\n        \"din\",\n        \"dintr\",\n        \"dintr-\",\n        \"dintre\",\n        \"doar\",\n        \"doi\",\n        \"doilea\",\n        \"două\",\n        \"drept\",\n        \"dupa\",\n        \"după\",\n        \"dă\",\n        \"e\",\n        \"ea\",\n        \"ei\",\n        \"el\",\n        \"ele\",\n        \"era\",\n        \"eram\",\n        \"este\",\n        \"eu\",\n        \"eşti\",\n        \"face\",\n        \"fara\",\n        \"fata\",\n        \"fel\",\n        \"fi\",\n        \"fie\",\n        \"fiecare\",\n        \"fii\",\n        \"fim\",\n        \"fiu\",\n        \"fiţi\",\n        \"foarte\",\n        \"fost\",\n        \"fără\",\n        \"i\",\n        \"ia\",\n        \"iar\",\n        \"ii\",\n        \"îi\",\n        \"il\",\n        \"îl\",\n        \"imi\",\n        \"îmi\",\n        \"in\",\n        \"în\",\n        \"inainte\",\n        \"inapoi\",\n        \"inca\",\n        \"incit\",\n        \"insa\",\n        \"intr\",\n        \"intre\",\n        \"isi\",\n        \"iti\",\n        \"îţi\",\n        \"la\",\n        \"lângă\",\n        \"le\",\n        \"li\",\n        \"lîngă\",\n        \"lor\",\n        \"lui\",\n        \"m\",\n        \"ma\",\n        \"mai\",\n        \"mâine\",\n        \"mea\",\n        \"mei\",\n        \"mele\",\n        \"mereu\",\n        \"meu\",\n        \"mi\",\n        \"mie\",\n        \"mîine\",\n        \"mine\",\n        \"mod\",\n        \"mult\",\n        \"multa\",\n        \"multe\",\n        \"multi\",\n        \"multă\",\n        \"mulţi\",\n        \"mă\",\n        \"ne\",\n        \"ni\",\n        \"nici\",\n        \"nimeni\",\n        \"nimic\",\n        \"niste\",\n        \"nişte\",\n        \"noastre\",\n        \"noastră\",\n        \"noi\",\n        \"nostri\",\n        \"nostru\",\n        \"nou\",\n        \"noua\",\n        \"nouă\",\n        \"noştri\",\n        \"nu\",\n        \"numai\",\n        \"o\",\n        \"or\",\n        \"ori\",\n        \"oricând\",\n        \"oricare\",\n        \"oricât\",\n        \"orice\",\n        \"oricînd\",\n        \"oricine\",\n        \"oricît\",\n        \"oricum\",\n        \"oriunde\",\n        \"pai\",\n        \"până\",\n        \"parca\",\n        \"patra\",\n        \"patru\",\n        \"pe\",\n        \"pentru\",\n        \"peste\",\n        \"pic\",\n        \"pina\",\n        \"pînă\",\n        \"poate\",\n        \"pot\",\n        \"prea\",\n        \"prima\",\n        \"primul\",\n        \"prin\",\n        \"printr-\",\n        \"putini\",\n        \"puţin\",\n        \"puţina\",\n        \"puţină\",\n        \"sa\",\n        \"sa-mi\",\n        \"sa-ti\",\n        \"sai\",\n        \"sale\",\n        \"sau\",\n        \"se\",\n        \"si\",\n        \"sint\",\n        \"sintem\",\n        \"spate\",\n        \"spre\",\n        \"sub\",\n        \"sunt\",\n        \"suntem\",\n        \"sunteţi\",\n        \"sus\",\n        \"să\",\n        \"săi\",\n        \"său\",\n        \"t\",\n        \"ta\",\n        \"tale\",\n        \"te\",\n        \"ti\",\n        \"tine\",\n        \"toata\",\n        \"toate\",\n        \"toată\",\n        \"tocmai\",\n        \"tot\",\n        \"toti\",\n        \"totul\",\n        \"totusi\",\n        \"totuşi\",\n        \"toţi\",\n        \"trei\",\n        \"treia\",\n        \"treilea\",\n        \"tu\",\n        \"tuturor\",\n        \"tăi\",\n        \"tău\",\n        \"u\",\n        \"ul\",\n        \"ului\",\n        \"un\",\n        \"una\",\n        \"unde\",\n        \"undeva\",\n        \"unei\",\n        \"uneia\",\n        \"unele\",\n        \"uneori\",\n        \"unii\",\n        \"unor\",\n        \"unora\",\n        \"unu\",\n        \"unui\",\n        \"unuia\",\n        \"unul\",\n        \"v\",\n        \"va\",\n        \"vi\",\n        \"voastre\",\n        \"voastră\",\n        \"voi\",\n        \"vom\",\n        \"vor\",\n        \"vostru\",\n        \"vouă\",\n        \"voştri\",\n        \"vreo\",\n        \"vreun\",\n        \"vă\",\n        \"zi\",\n        \"zice\",\n        \"şi\",\n        \"ţi\",\n        \"ţie\",\n        \"ăla\",\n        \"ălea\",\n        \"ăsta\",\n        \"ăstea\",\n        \"ăştia\",\n    )\n}\n\nfn russian(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"а\",\n        \"в\",\n        \"ж\",\n        \"и\",\n        \"к\",\n        \"о\",\n        \"с\",\n        \"у\",\n        \"я\",\n        \"бы\",\n        \"во\",\n        \"вы\",\n        \"да\",\n        \"до\",\n        \"ее\",\n        \"ей\",\n        \"же\",\n        \"за\",\n        \"из\",\n        \"им\",\n        \"их\",\n        \"ли\",\n        \"мы\",\n        \"на\",\n        \"не\",\n        \"ни\",\n        \"но\",\n        \"ну\",\n        \"об\",\n        \"он\",\n        \"от\",\n        \"по\",\n        \"со\",\n        \"то\",\n        \"ты\",\n        \"уж\",\n        \"без\",\n        \"был\",\n        \"вам\",\n        \"вас\",\n        \"вот\",\n        \"все\",\n        \"всю\",\n        \"где\",\n        \"два\",\n        \"для\",\n        \"его\",\n        \"ему\",\n        \"еще\",\n        \"или\",\n        \"как\",\n        \"кто\",\n        \"мне\",\n        \"мой\",\n        \"моя\",\n        \"над\",\n        \"нас\",\n        \"нее\",\n        \"ней\",\n        \"нет\",\n        \"ним\",\n        \"них\",\n        \"она\",\n        \"они\",\n        \"под\",\n        \"при\",\n        \"про\",\n        \"раз\",\n        \"сам\",\n        \"так\",\n        \"там\",\n        \"тем\",\n        \"том\",\n        \"тот\",\n        \"три\",\n        \"тут\",\n        \"уже\",\n        \"чем\",\n        \"что\",\n        \"эти\",\n        \"эту\",\n        \"была\",\n        \"были\",\n        \"было\",\n        \"быть\",\n        \"ведь\",\n        \"всех\",\n        \"даже\",\n        \"если\",\n        \"есть\",\n        \"куда\",\n        \"меня\",\n        \"надо\",\n        \"него\",\n        \"один\",\n        \"свою\",\n        \"себе\",\n        \"себя\",\n        \"тебя\",\n        \"того\",\n        \"тоже\",\n        \"хоть\",\n        \"чего\",\n        \"чтоб\",\n        \"чуть\",\n        \"этой\",\n        \"этом\",\n        \"этот\",\n        \"более\",\n        \"будет\",\n        \"будто\",\n        \"вдруг\",\n        \"всего\",\n        \"зачем\",\n        \"здесь\",\n        \"какая\",\n        \"какой\",\n        \"когда\",\n        \"лучше\",\n        \"между\",\n        \"много\",\n        \"может\",\n        \"можно\",\n        \"опять\",\n        \"перед\",\n        \"после\",\n        \"потом\",\n        \"почти\",\n        \"разве\",\n        \"такой\",\n        \"тогда\",\n        \"через\",\n        \"чтобы\",\n        \"этого\",\n        \"больше\",\n        \"всегда\",\n        \"другой\",\n        \"иногда\",\n        \"нельзя\",\n        \"нибудь\",\n        \"ничего\",\n        \"потому\",\n        \"сейчас\",\n        \"совсем\",\n        \"теперь\",\n        \"только\",\n        \"хорошо\",\n        \"впрочем\",\n        \"конечно\",\n        \"наконец\",\n        \"никогда\",\n    )\n}\n\nfn spanish(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"a\",\n        \"al\",\n        \"algo\",\n        \"algunas\",\n        \"algunos\",\n        \"ante\",\n        \"antes\",\n        \"como\",\n        \"con\",\n        \"contra\",\n        \"cual\",\n        \"cuando\",\n        \"de\",\n        \"del\",\n        \"desde\",\n        \"donde\",\n        \"durante\",\n        \"e\",\n        \"el\",\n        \"él\",\n        \"ella\",\n        \"ellas\",\n        \"ellos\",\n        \"en\",\n        \"entre\",\n        \"era\",\n        \"erais\",\n        \"éramos\",\n        \"eran\",\n        \"eras\",\n        \"eres\",\n        \"es\",\n        \"esa\",\n        \"esas\",\n        \"ese\",\n        \"eso\",\n        \"esos\",\n        \"esta\",\n        \"está\",\n        \"estaba\",\n        \"estabais\",\n        \"estábamos\",\n        \"estaban\",\n        \"estabas\",\n        \"estad\",\n        \"estada\",\n        \"estadas\",\n        \"estado\",\n        \"estados\",\n        \"estáis\",\n        \"estamos\",\n        \"están\",\n        \"estando\",\n        \"estar\",\n        \"estará\",\n        \"estarán\",\n        \"estarás\",\n        \"estaré\",\n        \"estaréis\",\n        \"estaremos\",\n        \"estaría\",\n        \"estaríais\",\n        \"estaríamos\",\n        \"estarían\",\n        \"estarías\",\n        \"estas\",\n        \"estás\",\n        \"este\",\n        \"esté\",\n        \"estéis\",\n        \"estemos\",\n        \"estén\",\n        \"estés\",\n        \"esto\",\n        \"estos\",\n        \"estoy\",\n        \"estuve\",\n        \"estuviera\",\n        \"estuvierais\",\n        \"estuviéramos\",\n        \"estuvieran\",\n        \"estuvieras\",\n        \"estuvieron\",\n        \"estuviese\",\n        \"estuvieseis\",\n        \"estuviésemos\",\n        \"estuviesen\",\n        \"estuvieses\",\n        \"estuvimos\",\n        \"estuviste\",\n        \"estuvisteis\",\n        \"estuvo\",\n        \"fue\",\n        \"fuera\",\n        \"fuerais\",\n        \"fuéramos\",\n        \"fueran\",\n        \"fueras\",\n        \"fueron\",\n        \"fuese\",\n        \"fueseis\",\n        \"fuésemos\",\n        \"fuesen\",\n        \"fueses\",\n        \"fui\",\n        \"fuimos\",\n        \"fuiste\",\n        \"fuisteis\",\n        \"ha\",\n        \"habéis\",\n        \"había\",\n        \"habíais\",\n        \"habíamos\",\n        \"habían\",\n        \"habías\",\n        \"habida\",\n        \"habidas\",\n        \"habido\",\n        \"habidos\",\n        \"habiendo\",\n        \"habrá\",\n        \"habrán\",\n        \"habrás\",\n        \"habré\",\n        \"habréis\",\n        \"habremos\",\n        \"habría\",\n        \"habríais\",\n        \"habríamos\",\n        \"habrían\",\n        \"habrías\",\n        \"han\",\n        \"has\",\n        \"hasta\",\n        \"hay\",\n        \"haya\",\n        \"hayáis\",\n        \"hayamos\",\n        \"hayan\",\n        \"hayas\",\n        \"he\",\n        \"hemos\",\n        \"hube\",\n        \"hubiera\",\n        \"hubierais\",\n        \"hubiéramos\",\n        \"hubieran\",\n        \"hubieras\",\n        \"hubieron\",\n        \"hubiese\",\n        \"hubieseis\",\n        \"hubiésemos\",\n        \"hubiesen\",\n        \"hubieses\",\n        \"hubimos\",\n        \"hubiste\",\n        \"hubisteis\",\n        \"hubo\",\n        \"la\",\n        \"las\",\n        \"le\",\n        \"les\",\n        \"lo\",\n        \"los\",\n        \"más\",\n        \"me\",\n        \"mi\",\n        \"mí\",\n        \"mía\",\n        \"mías\",\n        \"mío\",\n        \"míos\",\n        \"mis\",\n        \"mucho\",\n        \"muchos\",\n        \"muy\",\n        \"nada\",\n        \"ni\",\n        \"no\",\n        \"nos\",\n        \"nosotras\",\n        \"nosotros\",\n        \"nuestra\",\n        \"nuestras\",\n        \"nuestro\",\n        \"nuestros\",\n        \"o\",\n        \"os\",\n        \"otra\",\n        \"otras\",\n        \"otro\",\n        \"otros\",\n        \"para\",\n        \"pero\",\n        \"poco\",\n        \"por\",\n        \"porque\",\n        \"que\",\n        \"qué\",\n        \"quien\",\n        \"quienes\",\n        \"se\",\n        \"sea\",\n        \"seáis\",\n        \"seamos\",\n        \"sean\",\n        \"seas\",\n        \"sentid\",\n        \"sentida\",\n        \"sentidas\",\n        \"sentido\",\n        \"sentidos\",\n        \"será\",\n        \"serán\",\n        \"serás\",\n        \"seré\",\n        \"seréis\",\n        \"seremos\",\n        \"sería\",\n        \"seríais\",\n        \"seríamos\",\n        \"serían\",\n        \"serías\",\n        \"sí\",\n        \"siente\",\n        \"sin\",\n        \"sintiendo\",\n        \"sobre\",\n        \"sois\",\n        \"somos\",\n        \"son\",\n        \"soy\",\n        \"su\",\n        \"sus\",\n        \"suya\",\n        \"suyas\",\n        \"suyo\",\n        \"suyos\",\n        \"también\",\n        \"tanto\",\n        \"te\",\n        \"tendrá\",\n        \"tendrán\",\n        \"tendrás\",\n        \"tendré\",\n        \"tendréis\",\n        \"tendremos\",\n        \"tendría\",\n        \"tendríais\",\n        \"tendríamos\",\n        \"tendrían\",\n        \"tendrías\",\n        \"tened\",\n        \"tenéis\",\n        \"tenemos\",\n        \"tenga\",\n        \"tengáis\",\n        \"tengamos\",\n        \"tengan\",\n        \"tengas\",\n        \"tengo\",\n        \"tenía\",\n        \"teníais\",\n        \"teníamos\",\n        \"tenían\",\n        \"tenías\",\n        \"tenida\",\n        \"tenidas\",\n        \"tenido\",\n        \"tenidos\",\n        \"teniendo\",\n        \"ti\",\n        \"tiene\",\n        \"tienen\",\n        \"tienes\",\n        \"todo\",\n        \"todos\",\n        \"tu\",\n        \"tú\",\n        \"tus\",\n        \"tuve\",\n        \"tuviera\",\n        \"tuvierais\",\n        \"tuviéramos\",\n        \"tuvieran\",\n        \"tuvieras\",\n        \"tuvieron\",\n        \"tuviese\",\n        \"tuvieseis\",\n        \"tuviésemos\",\n        \"tuviesen\",\n        \"tuvieses\",\n        \"tuvimos\",\n        \"tuviste\",\n        \"tuvisteis\",\n        \"tuvo\",\n        \"tuya\",\n        \"tuyas\",\n        \"tuyo\",\n        \"tuyos\",\n        \"un\",\n        \"una\",\n        \"uno\",\n        \"unos\",\n        \"vosostras\",\n        \"vosostros\",\n        \"vuestra\",\n        \"vuestras\",\n        \"vuestro\",\n        \"vuestros\",\n        \"y\",\n        \"ya\",\n        \"yo\",\n    )\n}\n\nfn swedish(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"alla\",\n        \"allt\",\n        \"än\",\n        \"är\",\n        \"åt\",\n        \"att\",\n        \"av\",\n        \"blev\",\n        \"bli\",\n        \"blir\",\n        \"blivit\",\n        \"då\",\n        \"där\",\n        \"de\",\n        \"dem\",\n        \"den\",\n        \"denna\",\n        \"deras\",\n        \"dess\",\n        \"dessa\",\n        \"det\",\n        \"detta\",\n        \"dig\",\n        \"din\",\n        \"dina\",\n        \"ditt\",\n        \"du\",\n        \"efter\",\n        \"ej\",\n        \"eller\",\n        \"en\",\n        \"er\",\n        \"era\",\n        \"ert\",\n        \"ett\",\n        \"för\",\n        \"från\",\n        \"ha\",\n        \"hade\",\n        \"han\",\n        \"hans\",\n        \"har\",\n        \"här\",\n        \"henne\",\n        \"hennes\",\n        \"hon\",\n        \"honom\",\n        \"hur\",\n        \"i\",\n        \"icke\",\n        \"ingen\",\n        \"inom\",\n        \"inte\",\n        \"jag\",\n        \"ju\",\n        \"kan\",\n        \"kunde\",\n        \"man\",\n        \"med\",\n        \"mellan\",\n        \"men\",\n        \"mig\",\n        \"min\",\n        \"mina\",\n        \"mitt\",\n        \"mot\",\n        \"mycket\",\n        \"någon\",\n        \"något\",\n        \"några\",\n        \"när\",\n        \"ni\",\n        \"nu\",\n        \"och\",\n        \"om\",\n        \"oss\",\n        \"över\",\n        \"på\",\n        \"så\",\n        \"sådan\",\n        \"sådana\",\n        \"sådant\",\n        \"samma\",\n        \"sedan\",\n        \"sig\",\n        \"sin\",\n        \"sina\",\n        \"sitta\",\n        \"själv\",\n        \"skulle\",\n        \"som\",\n        \"till\",\n        \"under\",\n        \"upp\",\n        \"ut\",\n        \"utan\",\n        \"vad\",\n        \"var\",\n        \"vår\",\n        \"vara\",\n        \"våra\",\n        \"varför\",\n        \"varit\",\n        \"varje\",\n        \"vars\",\n        \"vart\",\n        \"vårt\",\n        \"vem\",\n        \"vi\",\n        \"vid\",\n        \"vilka\",\n        \"vilkas\",\n        \"vilken\",\n        \"vilket\",\n    )\n}\n\nfn turkish(input: &str) -> bool {\n    hashify::set!(\n        input.as_bytes(),\n        \"acaba\",\n        \"ama\",\n        \"aslında\",\n        \"az\",\n        \"bazı\",\n        \"belki\",\n        \"biri\",\n        \"birkaç\",\n        \"birşey\",\n        \"biz\",\n        \"bu\",\n        \"çok\",\n        \"çünkü\",\n        \"da\",\n        \"daha\",\n        \"de\",\n        \"defa\",\n        \"diye\",\n        \"en\",\n        \"eğer\",\n        \"gibi\",\n        \"hem\",\n        \"hep\",\n        \"hepsi\",\n        \"her\",\n        \"hiç\",\n        \"için\",\n        \"ile\",\n        \"ise\",\n        \"kez\",\n        \"ki\",\n        \"kim\",\n        \"mu\",\n        \"mü\",\n        \"mı\",\n        \"nasıl\",\n        \"ne\",\n        \"neden\",\n        \"nerde\",\n        \"nerede\",\n        \"nereye\",\n        \"niçin\",\n        \"niye\",\n        \"o\",\n        \"sanki\",\n        \"siz\",\n        \"tüm\",\n        \"ve\",\n        \"veya\",\n        \"ya\",\n        \"yani\",\n        \"şey\",\n        \"şu\",\n    )\n}\n\n/*\nNot yet available for auto-detection\n\nstatic KAZAKH: Set<&'static str> = phf_set! {\n    \"\",\n    \"е\",\n    \"о\",\n    \"я\",\n    \"ә\",\n    \"ай\",\n    \"ал\",\n    \"ау\",\n    \"ах\",\n    \"ей\",\n    \"еш\",\n    \"ие\",\n    \"кә\",\n    \"ой\",\n    \"ол\",\n    \"ох\",\n    \"па\",\n    \"уа\",\n    \"эй\",\n    \"эх\",\n    \"әй\",\n    \"өз\",\n    \"өй\",\n    \"ана\",\n    \"арс\",\n    \"аһа\",\n    \"бар\",\n    \"беу\",\n    \"біз\",\n    \"бұл\",\n    \"жоқ\",\n    \"кәһ\",\n    \"мен\",\n    \"моһ\",\n    \"осы\",\n    \"оһо\",\n    \"пай\",\n    \"сен\",\n    \"сол\",\n    \"соң\",\n    \"сіз\",\n    \"тек\",\n    \"тәк\",\n    \"уай\",\n    \"уау\",\n    \"ура\",\n    \"шек\",\n    \"ырс\",\n    \"ырқ\",\n    \"ыңқ\",\n    \"ірк\",\n    \"қап\",\n    \"құр\",\n    \"үйт\",\n    \"әні\",\n    \"өзі\",\n    \"арс-ұрс\",\n    \"пай-пай\",\n    \"паһ-паһ\",\n    \"қош-қош\",\n    \"анау\",\n    \"барқ\",\n    \"бері\",\n    \"бойы\",\n    \"болп\",\n    \"борт\",\n    \"былп\",\n    \"бүйт\",\n    \"бәрі\",\n    \"гүрс\",\n    \"гөрі\",\n    \"дүрс\",\n    \"дүңк\",\n    \"емес\",\n    \"жалп\",\n    \"желп\",\n    \"жуық\",\n    \"кірт\",\n    \"күрт\",\n    \"күңк\",\n    \"кәне\",\n    \"кәні\",\n    \"маңқ\",\n    \"морт\",\n    \"мына\",\n    \"мышы\",\n    \"мыңқ\",\n    \"міне\",\n    \"одан\",\n    \"олар\",\n    \"онда\",\n    \"оның\",\n    \"оған\",\n    \"пфша\",\n    \"пырс\",\n    \"пішә\",\n    \"сарт\",\n    \"саңқ\",\n    \"сона\",\n    \"сыңқ\",\n    \"тарс\",\n    \"таяу\",\n    \"тағы\",\n    \"таңқ\",\n    \"тырс\",\n    \"тыңқ\",\n    \"түге\",\n    \"шаңқ\",\n    \"шырт\",\n    \"шіңк\",\n    \"шәйт\",\n    \"ғана\",\n    \"қана\",\n    \"қолп\",\n    \"қорс\",\n    \"қоса\",\n    \"қыңқ\",\n    \"үшін\",\n    \"әйда\",\n    \"әрне\",\n    \"өзге\",\n    \"өзім\",\n    \"өзің\",\n    \"жалт-жалт\",\n    \"жалт-жұлт\",\n    \"сарт-сұрт\",\n    \"тарс-тұрс\",\n    \"шаңқ-шаңқ\",\n    \"шаңқ-шұңқ\",\n    \"қалт-қалт\",\n    \"қалт-құлт\",\n    \"қаңқ-қаңқ\",\n    \"қаңқ-құңқ\",\n    \"барша\",\n    \"бетер\",\n    \"бізге\",\n    \"бірақ\",\n    \"бірге\",\n    \"біреу\",\n    \"бүкіл\",\n    \"бұрын\",\n    \"дейін\",\n    \"ешбір\",\n    \"ешкім\",\n    \"кейін\",\n    \"күллі\",\n    \"күшім\",\n    \"маған\",\n    \"менде\",\n    \"менен\",\n    \"менің\",\n    \"мынау\",\n    \"пішту\",\n    \"сайын\",\n    \"салым\",\n    \"саған\",\n    \"сенде\",\n    \"сенен\",\n    \"сенің\",\n    \"солай\",\n    \"сонау\",\n    \"сорап\",\n    \"сізге\",\n    \"таман\",\n    \"тарта\",\n    \"түгел\",\n    \"шақты\",\n    \"шейін\",\n    \"ғұрлы\",\n    \"қарай\",\n    \"қатар\",\n    \"құрау\",\n    \"әрбір\",\n    \"әрине\",\n    \"әркім\",\n    \"әттең\",\n    \"әукім\",\n    \"өзіме\",\n    \"өзіне\",\n    \"сенен\tонан\",\n    \"арбаң-арбаң\",\n    \"батыр-бұтыр\",\n    \"далаң-далаң\",\n    \"митың-митың\",\n    \"салаң-сұлаң\",\n    \"құрау-құрау\",\n    \"ыржың-тыржың\",\n    \"алайда\",\n    \"алатау\",\n    \"алақай\",\n    \"арнайы\",\n    \"арқылы\",\n    \"барлық\",\n    \"бізбен\",\n    \"бізден\",\n    \"біздер\",\n    \"біздің\",\n    \"бұндай\",\n    \"дәнеңе\",\n    \"ештеме\",\n    \"кейбір\",\n    \"кәнеки\",\n    \"мұндай\",\n    \"оларға\",\n    \"онымен\",\n    \"осылай\",\n    \"осынау\",\n    \"себебі\",\n    \"сияқты\",\n    \"сондай\",\n    \"сізбен\",\n    \"сізден\",\n    \"сіздер\",\n    \"сіздің\",\n    \"тағыда\",\n    \"туралы\",\n    \"шамалы\",\n    \"шіркін\",\n    \"ғұрлым\",\n    \"қаралы\",\n    \"әлдене\",\n    \"өзінің\",\n    \"бүгжең-бүгжең\",\n    \"тарбаң-тарбаң\",\n    \"қайқаң-құйқаң\",\n    \"қаңғыр-күңгір\",\n    \"бойымен\",\n    \"бірдеме\",\n    \"бірнеше\",\n    \"ешқайсы\",\n    \"ешқашан\",\n    \"менімен\",\n    \"олардан\",\n    \"олардың\",\n    \"олармен\",\n    \"осындай\",\n    \"сенімен\",\n    \"сонымен\",\n    \"япырмай\",\n    \"әйтпесе\",\n    \"әлдекім\",\n    \"әншейін\",\n    \"әрқайсы\",\n    \"әрқалай\",\n    \"өзімнің\",\n    \"өйткені\",\n    \"әттеген-ай\",\n    \"арсалаң-арсалаң\",\n    \"ербелең-ербелең\",\n    \"қызараң-қызараң\",\n    \"айтпақшы\",\n    \"біздерге\",\n    \"дегенмен\",\n    \"ешқандай\",\n    \"кейбіреу\",\n    \"масқарай\",\n    \"мәссаған\",\n    \"ойпырмай\",\n    \"сіздерге\",\n    \"қайсыбір\",\n    \"әлденеше\",\n    \"алдақашан\",\n    \"біздерден\",\n    \"біздердің\",\n    \"біздермен\",\n    \"бәрекелді\",\n    \"сондықтан\",\n    \"сіздерден\",\n    \"сіздердің\",\n    \"сіздермен\",\n    \"әйткенмен\",\n    \"әлдеқалай\",\n    \"әлдеқашан\",\n    \"әттегенай\",\n    \"әлдеқайдан\",\n    \"астапыралла\",\n    \"жаракімалла\",\n};\n*/\n"
  },
  {
    "path": "crates/nlp/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod classifier;\npub mod language;\npub mod tokenizers;\n"
  },
  {
    "path": "crates/nlp/src/tokenizers/chinese.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, sync::LazyLock, vec::IntoIter};\n\nuse jieba_rs::Jieba;\n\nuse super::{InnerToken, Token};\n\npub(crate) static JIEBA: LazyLock<Jieba> = LazyLock::new(Jieba::new);\n\npub struct ChineseTokenizer<'x, T, I>\nwhere\n    T: Iterator<Item = Token<I>>,\n    I: InnerToken<'x>,\n{\n    tokenizer: T,\n    tokens: IntoIter<Token<I>>,\n    phantom: std::marker::PhantomData<&'x str>,\n}\n\nimpl<'x, T, I> ChineseTokenizer<'x, T, I>\nwhere\n    T: Iterator<Item = Token<I>>,\n    I: InnerToken<'x>,\n{\n    pub fn new(tokenizer: T) -> Self {\n        ChineseTokenizer {\n            tokenizer,\n            tokens: Vec::new().into_iter(),\n            phantom: std::marker::PhantomData,\n        }\n    }\n}\n\nimpl<'x, T, I> Iterator for ChineseTokenizer<'x, T, I>\nwhere\n    T: Iterator<Item = Token<I>>,\n    I: InnerToken<'x>,\n{\n    type Item = Token<I>;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        loop {\n            if let Some(token) = self.tokens.next() {\n                return Some(token);\n            } else {\n                let token = self.tokenizer.next()?;\n                if token.word.is_alphabetic_8bit() {\n                    let mut token_to = token.from;\n                    match token.word.unwrap_alphabetic() {\n                        Cow::Borrowed(word) => {\n                            self.tokens = JIEBA\n                                .cut(word, false)\n                                .into_iter()\n                                .map(|word| {\n                                    let token_from = token_to;\n                                    token_to += word.len();\n                                    Token {\n                                        word: I::new_alphabetic(word),\n                                        from: token_from,\n                                        to: token_to,\n                                    }\n                                })\n                                .collect::<Vec<_>>()\n                                .into_iter();\n                        }\n                        Cow::Owned(word) => {\n                            self.tokens = JIEBA\n                                .cut(&word, false)\n                                .into_iter()\n                                .map(|word| {\n                                    let token_from = token_to;\n                                    token_to += word.len();\n                                    Token {\n                                        word: I::new_alphabetic(word.to_string()),\n                                        from: token_from,\n                                        to: token_to,\n                                    }\n                                })\n                                .collect::<Vec<_>>()\n                                .into_iter();\n                        }\n                    }\n                } else {\n                    return token.into();\n                }\n            }\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::tokenizers::{Token, chinese::ChineseTokenizer, word::WordTokenizer};\n\n    #[test]\n    fn chinese_tokenizer() {\n        assert_eq!(\n            ChineseTokenizer::new(WordTokenizer::new(\n                \"孫子曰：兵者，國之大事，死生之地，存亡之道，不可不察也。\",\n                40\n            ),)\n            .collect::<Vec<_>>(),\n            vec![\n                Token {\n                    word: \"孫\".into(),\n                    from: 0,\n                    to: 3\n                },\n                Token {\n                    word: \"子\".into(),\n                    from: 3,\n                    to: 6\n                },\n                Token {\n                    word: \"曰\".into(),\n                    from: 6,\n                    to: 9\n                },\n                Token {\n                    word: \"兵\".into(),\n                    from: 12,\n                    to: 15\n                },\n                Token {\n                    word: \"者\".into(),\n                    from: 15,\n                    to: 18\n                },\n                Token {\n                    word: \"國\".into(),\n                    from: 21,\n                    to: 24\n                },\n                Token {\n                    word: \"之\".into(),\n                    from: 24,\n                    to: 27\n                },\n                Token {\n                    word: \"大事\".into(),\n                    from: 27,\n                    to: 33\n                },\n                Token {\n                    word: \"死\".into(),\n                    from: 36,\n                    to: 39\n                },\n                Token {\n                    word: \"生\".into(),\n                    from: 39,\n                    to: 42\n                },\n                Token {\n                    word: \"之\".into(),\n                    from: 42,\n                    to: 45\n                },\n                Token {\n                    word: \"地\".into(),\n                    from: 45,\n                    to: 48\n                },\n                Token {\n                    word: \"存亡\".into(),\n                    from: 51,\n                    to: 57\n                },\n                Token {\n                    word: \"之\".into(),\n                    from: 57,\n                    to: 60\n                },\n                Token {\n                    word: \"道\".into(),\n                    from: 60,\n                    to: 63\n                },\n                Token {\n                    word: \"不可不\".into(),\n                    from: 66,\n                    to: 75\n                },\n                Token {\n                    word: \"察\".into(),\n                    from: 75,\n                    to: 78\n                },\n                Token {\n                    word: \"也\".into(),\n                    from: 78,\n                    to: 81\n                }\n            ]\n        );\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/tokenizers/japanese.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{InnerToken, Token};\nuse maplit::hashmap;\nuse std::collections::HashMap;\nuse std::vec::IntoIter;\nuse std::{hash::Hash, sync::LazyLock};\n\npub struct JapaneseTokenizer<'x, T, I>\nwhere\n    T: Iterator<Item = Token<I>>,\n    I: InnerToken<'x>,\n{\n    tokenizer: T,\n    tokens: IntoIter<Token<I>>,\n    phantom: std::marker::PhantomData<&'x str>,\n}\n\nimpl<'x, T, I> JapaneseTokenizer<'x, T, I>\nwhere\n    T: Iterator<Item = Token<I>>,\n    I: InnerToken<'x>,\n{\n    pub fn new(tokenizer: T) -> Self {\n        JapaneseTokenizer {\n            tokenizer,\n            tokens: Vec::new().into_iter(),\n            phantom: std::marker::PhantomData,\n        }\n    }\n}\n\nimpl<'x, T, I> Iterator for JapaneseTokenizer<'x, T, I>\nwhere\n    T: Iterator<Item = Token<I>>,\n    I: InnerToken<'x>,\n{\n    type Item = Token<I>;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        loop {\n            if let Some(token) = self.tokens.next() {\n                return Some(token);\n            } else {\n                let token = self.tokenizer.next()?;\n                if token.word.is_alphabetic_8bit() {\n                    let mut token_to = token.from;\n                    self.tokens = tokenize(token.word.unwrap_alphabetic().as_ref())\n                        .into_iter()\n                        .map(|word| {\n                            let token_from = token_to;\n                            token_to += word.len();\n                            Token {\n                                word: I::new_alphabetic(word.to_string()),\n                                from: token_from,\n                                to: token_to,\n                            }\n                        })\n                        .collect::<Vec<_>>()\n                        .into_iter();\n                } else {\n                    return token.into();\n                }\n            }\n        }\n    }\n}\n\n// Ported from https://github.com/woxtu/rust-tinysegmenter, MIT license\n\nconst BIAS: i32 = -332;\n\nfn get_score<T: Eq + Hash>(d: &HashMap<T, i32>, s: &T) -> i32 {\n    d.get(s).cloned().unwrap_or(0)\n}\n\nfn get_ctype(c: char) -> char {\n    match c as u32 {\n        0x4E00 | 0x4E8C | 0x4E09 | 0x56DB | 0x4E94 | 0x516D | 0x4E03 | 0x516B | 0x4E5D | 0x5341 => {\n            'M'\n        }\n        0x767E | 0x5343 | 0x4E07 | 0x5104 | 0x5146 => 'M',\n        0x4E00..=0x9FA0 | 0x3005 | 0x3006 | 0x30F5 | 0x30F6 => 'H',\n        0x3041..=0x3093 => 'I',\n        0x30A1..=0x30F4 | 0x30FC | 0xFF71..=0xFF9D | 0xFF9E | 0xFF70 => 'K',\n        0x61..=0x7A | 0x41..=0x5A | 0xFF41..=0xFF5A | 0xFF21..=0xFF3A => 'A',\n        0x30..=0x3a | 0xFF10..=0xFF19 => 'N',\n        _ => 'O',\n    }\n}\n\npub fn tokenize(s: &str) -> Vec<String> {\n    if s.is_empty() {\n        return Vec::new();\n    }\n\n    let mut result = Vec::with_capacity(s.chars().count());\n\n    let segments = [B3, B2, B1]\n        .into_iter()\n        .chain(s.chars())\n        .chain([E1, E2, E3])\n        .collect::<Vec<_>>();\n\n    let ctypes = ['O'; 3]\n        .into_iter()\n        .chain(s.chars().map(get_ctype))\n        .chain(['O'; 3])\n        .collect::<Vec<_>>();\n\n    let mut word = segments[3].to_string();\n    let mut p = vec!['U'; 3];\n\n    for index in 4..segments.len() - 3 {\n        let mut score = BIAS;\n        let w = &segments[index - 3..index + 3];\n        let c = &ctypes[index - 3..index + 3];\n\n        score += get_score(&*UP1, &p[0]);\n        score += get_score(&*UP2, &p[1]);\n        score += get_score(&*UP3, &p[2]);\n        score += get_score(&*BP1, &(p[0], p[1]));\n        score += get_score(&*BP2, &(p[1], p[2]));\n        score += get_score(&*UW1, &w[0]);\n        score += get_score(&*UW2, &w[1]);\n        score += get_score(&*UW3, &w[2]);\n        score += get_score(&*UW4, &w[3]);\n        score += get_score(&*UW5, &w[4]);\n        score += get_score(&*UW6, &w[5]);\n        score += get_score(&*BW1, &(w[1], w[2]));\n        score += get_score(&*BW2, &(w[2], w[3]));\n        score += get_score(&*BW3, &(w[3], w[4]));\n        score += get_score(&*TW1, &(w[0], w[1], w[2]));\n        score += get_score(&*TW2, &(w[1], w[2], w[3]));\n        score += get_score(&*TW3, &(w[2], w[3], w[4]));\n        score += get_score(&*TW4, &(w[3], w[4], w[5]));\n        score += get_score(&*UC1, &c[0]);\n        score += get_score(&*UC2, &c[1]);\n        score += get_score(&*UC3, &c[2]);\n        score += get_score(&*UC4, &c[3]);\n        score += get_score(&*UC5, &c[4]);\n        score += get_score(&*UC6, &c[5]);\n        score += get_score(&*BC1, &(c[1], c[2]));\n        score += get_score(&*BC2, &(c[2], c[3]));\n        score += get_score(&*BC3, &(c[3], c[4]));\n        score += get_score(&*TC1, &(c[0], c[1], c[2]));\n        score += get_score(&*TC2, &(c[1], c[2], c[3]));\n        score += get_score(&*TC3, &(c[2], c[3], c[4]));\n        score += get_score(&*TC4, &(c[3], c[4], c[5]));\n        score += get_score(&*UQ1, &(p[0], c[0]));\n        score += get_score(&*UQ2, &(p[1], c[1]));\n        score += get_score(&*UQ3, &(p[2], c[2]));\n        score += get_score(&*BQ1, &(p[1], c[1], c[2]));\n        score += get_score(&*BQ2, &(p[1], c[2], c[3]));\n        score += get_score(&*BQ3, &(p[2], c[1], c[2]));\n        score += get_score(&*BQ4, &(p[2], c[2], c[3]));\n        score += get_score(&*TQ1, &(p[1], c[0], c[1], c[2]));\n        score += get_score(&*TQ2, &(p[1], c[1], c[2], c[3]));\n        score += get_score(&*TQ3, &(p[2], c[0], c[1], c[2]));\n        score += get_score(&*TQ4, &(p[2], c[1], c[2], c[3]));\n\n        p.remove(0);\n        p.push(if score < 0 { 'O' } else { 'B' });\n\n        if 0 < score {\n            result.push(word.clone());\n            word.clear();\n        }\n        word.push(segments[index]);\n    }\n\n    result.push(word.clone());\n    result\n}\n\nconst B1: char = '\\u{F0000}';\nconst B2: char = '\\u{F0001}';\nconst B3: char = '\\u{F0002}';\nconst E1: char = '\\u{F0003}';\nconst E2: char = '\\u{F0004}';\nconst E3: char = '\\u{F0005}';\n\nstatic BC1: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('H', 'H') => 6, ('I', 'I') => 2461, ('K', 'H') => 406, ('O', 'H') => -1378, }\n});\nstatic BC2: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('A', 'A') => -3267, ('A', 'I') => 2744, ('A', 'N') => -878, ('H', 'H') => -4070, ('H', 'M') => -1711, ('H', 'N') => 4012, ('H', 'O') => 3761, ('I', 'A') => 1327, ('I', 'H') => -1184, ('I', 'I') => -1332, ('I', 'K') => 1721, ('I', 'O') => 5492, ('K', 'I') => 3831, ('K', 'K') => -8741, ('M', 'H') => -3132, ('M', 'K') => 3334, ('O', 'O') => -2920, }\n});\nstatic BC3: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('H', 'H') => 996, ('H', 'I') => 626, ('H', 'K') => -721, ('H', 'N') => -1307, ('H', 'O') => -836, ('I', 'H') => -301, ('K', 'K') => 2762, ('M', 'K') => 1079, ('M', 'M') => 4034, ('O', 'A') => -1652, ('O', 'H') => 266, }\n});\nstatic BP1: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'B') => 295, ('O', 'B') => 304, ('O', 'O') => -125, ('U', 'B') => 352, }\n});\nstatic BP2: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'O') => 60, ('O', 'O') => -1762, }\n});\nstatic BQ1: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H', 'H') => 1150, ('B', 'H', 'M') => 1521, ('B', 'I', 'I') => -1158, ('B', 'I', 'M') => 886, ('B', 'M', 'H') => 1208, ('B', 'N', 'H') => 449, ('B', 'O', 'H') => -91, ('B', 'O', 'O') => -2597, ('O', 'H', 'I') => 451, ('O', 'I', 'H') => -296, ('O', 'K', 'A') => 1851, ('O', 'K', 'H') => -1020, ('O', 'K', 'K') => 904, ('O', 'O', 'O') => 2965, }\n});\nstatic BQ2: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H', 'H') => 118, ('B', 'H', 'I') => -1159, ('B', 'H', 'M') => 466, ('B', 'I', 'H') => -919, ('B', 'K', 'K') => -1720, ('B', 'K', 'O') => 864, ('O', 'H', 'H') => -1139, ('O', 'H', 'M') => -181, ('O', 'I', 'H') => 153, ('U', 'H', 'I') => -1146, }\n});\nstatic BQ3: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H', 'H') => -792, ('B', 'H', 'I') => 2664, ('B', 'I', 'I') => -299, ('B', 'K', 'I') => 419, ('B', 'M', 'H') => 937, ('B', 'M', 'M') => 8335, ('B', 'N', 'N') => 998, ('B', 'O', 'H') => 775, ('O', 'H', 'H') => 2174, ('O', 'H', 'M') => 439, ('O', 'I', 'I') => 280, ('O', 'K', 'H') => 1798, ('O', 'K', 'I') => -793, ('O', 'K', 'O') => -2242, ('O', 'M', 'H') => -2402, ('O', 'O', 'O') => 11699, }\n});\nstatic BQ4: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H', 'H') => -3895, ('B', 'I', 'H') => 3761, ('B', 'I', 'I') => -4654, ('B', 'I', 'K') => 1348, ('B', 'K', 'K') => -1806, ('B', 'M', 'I') => -3385, ('B', 'O', 'O') => -12396, ('O', 'A', 'H') => 926, ('O', 'H', 'H') => 266, ('O', 'H', 'K') => -2036, ('O', 'N', 'N') => -973, }\n});\nstatic BW1: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { (',', 'と') => 660, (',', '同') => 727, (B1, 'あ') => 1404, (B1, '同') => 542, ('、', 'と') => 660, ('、', '同') => 727, ('」', 'と') => 1682, ('あ', 'っ') => 1505, ('い', 'う') => 1743, ('い', 'っ') => -2055, ('い', 'る') => 672, ('う', 'し') => -4817, ('う', 'ん') => 665, ('か', 'ら') => 3472, ('が', 'ら') => 600, ('こ', 'う') => -790, ('こ', 'と') => 2083, ('こ', 'ん') => -1262, ('さ', 'ら') => -4143, ('さ', 'ん') => 4573, ('し', 'た') => 2641, ('し', 'て') => 1104, ('す', 'で') => -3399, ('そ', 'こ') => 1977, ('そ', 'れ') => -871, ('た', 'ち') => 1122, ('た', 'め') => 601, ('っ', 'た') => 3463, ('つ', 'い') => -802, ('て', 'い') => 805, ('て', 'き') => 1249, ('で', 'き') => 1127, ('で', 'す') => 3445, ('で', 'は') => 844, ('と', 'い') => -4915, ('と', 'み') => 1922, ('ど', 'こ') => 3887, ('な', 'い') => 5713, ('な', 'っ') => 3015, ('な', 'ど') => 7379, ('な', 'ん') => -1113, ('に', 'し') => 2468, ('に', 'は') => 1498, ('に', 'も') => 1671, ('に', '対') => -912, ('の', '一') => -501, ('の', '中') => 741, ('ま', 'せ') => 2448, ('ま', 'で') => 1711, ('ま', 'ま') => 2600, ('ま', 'る') => -2155, ('や', 'む') => -1947, ('よ', 'っ') => -2565, ('れ', 'た') => 2369, ('れ', 'で') => -913, ('を', 'し') => 1860, ('を', '見') => 731, ('亡', 'く') => -1886, ('京', '都') => 2558, ('取', 'り') => -2784, ('大', 'き') => -2604, ('大', '阪') => 1497, ('平', '方') => -2314, ('引', 'き') => -1336, ('日', '本') => -195, ('本', '当') => -2423, ('毎', '日') => -2113, ('目', '指') => -724, ('｣', 'と') => 1682, }\n});\nstatic BW2: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('.', '.') => -11822, ('1', '1') => -669, ('―', '―') => -5730, ('−', '−') => -13175, ('い', 'う') => -1609, ('う', 'か') => 2490, ('か', 'し') => -1350, ('か', 'も') => -602, ('か', 'ら') => -7194, ('か', 'れ') => 4612, ('が', 'い') => 853, ('が', 'ら') => -3198, ('き', 'た') => 1941, ('く', 'な') => -1597, ('こ', 'と') => -8392, ('こ', 'の') => -4193, ('さ', 'せ') => 4533, ('さ', 'れ') => 13168, ('さ', 'ん') => -3977, ('し', 'い') => -1819, ('し', 'か') => -545, ('し', 'た') => 5078, ('し', 'て') => 972, ('し', 'な') => 939, ('そ', 'の') => -3744, ('た', 'い') => -1253, ('た', 'た') => -662, ('た', 'だ') => -3857, ('た', 'ち') => -786, ('た', 'と') => 1224, ('た', 'は') => -939, ('っ', 'た') => 4589, ('っ', 'て') => 1647, ('っ', 'と') => -2094, ('て', 'い') => 6144, ('て', 'き') => 3640, ('て', 'く') => 2551, ('て', 'は') => -3110, ('て', 'も') => -3065, ('で', 'い') => 2666, ('で', 'き') => -1528, ('で', 'し') => -3828, ('で', 'す') => -4761, ('で', 'も') => -4203, ('と', 'い') => 1890, ('と', 'こ') => -1746, ('と', 'と') => -2279, ('と', 'の') => 720, ('と', 'み') => 5168, ('と', 'も') => -3941, ('な', 'い') => -2488, ('な', 'が') => -1313, ('な', 'ど') => -6509, ('な', 'の') => 2614, ('な', 'ん') => 3099, ('に', 'お') => -1615, ('に', 'し') => 2748, ('に', 'な') => 2454, ('に', 'よ') => -7236, ('に', '対') => -14943, ('に', '従') => -4688, ('に', '関') => -11388, ('の', 'か') => 2093, ('の', 'で') => -7059, ('の', 'に') => -6041, ('の', 'の') => -6125, ('は', 'い') => 1073, ('は', 'が') => -1033, ('は', 'ず') => -2532, ('ば', 'れ') => 1813, ('ま', 'し') => -1316, ('ま', 'で') => -6621, ('ま', 'れ') => 5409, ('め', 'て') => -3153, ('も', 'い') => 2230, ('も', 'の') => -10713, ('ら', 'か') => -944, ('ら', 'し') => -1611, ('ら', 'に') => -1897, ('り', 'し') => 651, ('り', 'ま') => 1620, ('れ', 'た') => 4270, ('れ', 'て') => 849, ('れ', 'ば') => 4114, ('ろ', 'う') => 6067, ('わ', 'れ') => 7901, ('を', '通') => -11877, ('ん', 'だ') => 728, ('ん', 'な') => -4115, ('一', '人') => 602, ('一', '方') => -1375, ('一', '日') => 970, ('一', '部') => -1051, ('上', 'が') => -4479, ('会', '社') => -1116, ('出', 'て') => 2163, ('分', 'の') => -7758, ('同', '党') => 970, ('同', '日') => -913, ('大', '阪') => -2471, ('委', '員') => -1250, ('少', 'な') => -1050, ('年', '度') => -8669, ('年', '間') => -1626, ('府', '県') => -2363, ('手', '権') => -1982, ('新', '聞') => -4066, ('日', '新') => -722, ('日', '本') => -7068, ('日', '米') => 3372, ('曜', '日') => -601, ('朝', '鮮') => -2355, ('本', '人') => -2697, ('東', '京') => -1543, ('然', 'と') => -1384, ('社', '会') => -1276, ('立', 'て') => -990, ('第', 'に') => -1612, ('米', '国') => -4268, ('１', '１') => -669, ('ｸ', 'ﾞ') => 1319,}\n});\nstatic BW3: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('あ', 'た') => -2194, ('あ', 'り') => 719, ('あ', 'る') => 3846, ('い', '.') => -1185, ('い', '。') => -1185, ('い', 'い') => 5308, ('い', 'え') => 2079, ('い', 'く') => 3029, ('い', 'た') => 2056, ('い', 'っ') => 1883, ('い', 'る') => 5600, ('い', 'わ') => 1527, ('う', 'ち') => 1117, ('う', 'と') => 4798, ('え', 'と') => 1454, ('か', '.') => 2857, ('か', '。') => 2857, ('か', 'け') => -743, ('か', 'っ') => -4098, ('か', 'に') => -669, ('か', 'ら') => 6520, ('か', 'り') => -2670, ('が', ',') => 1816, ('が', '、') => 1816, ('が', 'き') => -4855, ('が', 'け') => -1127, ('が', 'っ') => -913, ('が', 'ら') => -4977, ('が', 'り') => -2064, ('き', 'た') => 1645, ('け', 'ど') => 1374, ('こ', 'と') => 7397, ('こ', 'の') => 1542, ('こ', 'ろ') => -2757, ('さ', 'い') => -714, ('さ', 'を') => 976, ('し', ',') => 1557, ('し', '、') => 1557, ('し', 'い') => -3714, ('し', 'た') => 3562, ('し', 'て') => 1449, ('し', 'な') => 2608, ('し', 'ま') => 1200, ('す', '.') => -1310, ('す', '。') => -1310, ('す', 'る') => 6521, ('ず', ',') => 3426, ('ず', '、') => 3426, ('ず', 'に') => 841, ('そ', 'う') => 428, ('た', '.') => 8875, ('た', '。') => 8875, ('た', 'い') => -594, ('た', 'の') => 812, ('た', 'り') => -1183, ('た', 'る') => -853, ('だ', '.') => 4098, ('だ', '。') => 4098, ('だ', 'っ') => 1004, ('っ', 'た') => -4748, ('っ', 'て') => 300, ('て', 'い') => 6240, ('て', 'お') => 855, ('て', 'も') => 302, ('で', 'す') => 1437, ('で', 'に') => -1482, ('で', 'は') => 2295, ('と', 'う') => -1387, ('と', 'し') => 2266, ('と', 'の') => 541, ('と', 'も') => -3543, ('ど', 'う') => 4664, ('な', 'い') => 1796, ('な', 'く') => -903, ('な', 'ど') => 2135, ('に', ',') => -1021, ('に', '、') => -1021, ('に', 'し') => 1771, ('に', 'な') => 1906, ('に', 'は') => 2644, ('の', ',') => -724, ('の', '、') => -724, ('の', '子') => -1000, ('は', ',') => 1337, ('は', '、') => 1337, ('べ', 'き') => 2181, ('ま', 'し') => 1113, ('ま', 'す') => 6943, ('ま', 'っ') => -1549, ('ま', 'で') => 6154, ('ま', 'れ') => -793, ('ら', 'し') => 1479, ('ら', 'れ') => 6820, ('る', 'る') => 3818, ('れ', ',') => 854, ('れ', '、') => 854, ('れ', 'た') => 1850, ('れ', 'て') => 1375, ('れ', 'ば') => -3246, ('れ', 'る') => 1091, ('わ', 'れ') => -605, ('ん', 'だ') => 606, ('ん', 'で') => 798, ('カ', '月') => 990, ('会', '議') => 860, ('入', 'り') => 1232, ('大', '会') => 2217, ('始', 'め') => 1681, ('市', ' ') => 965, ('新', '聞') => -5055, ('日', ',') => 974, ('日', '、') => 974, ('社', '会') => 2024, ('ｶ', '月') => 990, }\n});\n\nstatic TC1: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('A', 'A', 'A') => 1093, ('H', 'H', 'H') => 1029, ('H', 'H', 'M') => 580, ('H', 'I', 'I') => 998, ('H', 'O', 'H') => -390, ('H', 'O', 'M') => -331, ('I', 'H', 'I') => 1169, ('I', 'O', 'H') => -142, ('I', 'O', 'I') => -1015, ('I', 'O', 'M') => 467, ('M', 'M', 'H') => 187, ('O', 'O', 'I') => -1832, }\n});\nstatic TC2: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('H', 'H', 'O') => 2088, ('H', 'I', 'I') => -1023, ('H', 'M', 'M') => -1154, ('I', 'H', 'I') => -1965, ('K', 'K', 'H') => 703, ('O', 'I', 'I') => -2649, }\n});\nstatic TC3: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('A', 'A', 'A') => -294, ('H', 'H', 'H') => 346, ('H', 'H', 'I') => -341, ('H', 'I', 'I') => -1088, ('H', 'I', 'K') => 731, ('H', 'O', 'H') => -1486, ('I', 'H', 'H') => 128, ('I', 'H', 'I') => -3041, ('I', 'H', 'O') => -1935, ('I', 'I', 'H') => -825, ('I', 'I', 'M') => -1035, ('I', 'O', 'I') => -542, ('K', 'H', 'H') => -1216, ('K', 'K', 'A') => 491, ('K', 'K', 'H') => -1217, ('K', 'O', 'K') => -1009, ('M', 'H', 'H') => -2694, ('M', 'H', 'M') => -457, ('M', 'H', 'O') => 123, ('M', 'M', 'H') => -471, ('N', 'N', 'H') => -1689, ('N', 'N', 'O') => 662, ('O', 'H', 'O') => -3393, }\n});\nstatic TC4: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('H', 'H', 'H') => -203, ('H', 'H', 'I') => 1344, ('H', 'H', 'K') => 365, ('H', 'H', 'M') => -122, ('H', 'H', 'N') => 182, ('H', 'H', 'O') => 669, ('H', 'I', 'H') => 804, ('H', 'I', 'I') => 679, ('H', 'O', 'H') => 446, ('I', 'H', 'H') => 695, ('I', 'H', 'O') => -2324, ('I', 'I', 'H') => 321, ('I', 'I', 'I') => 1497, ('I', 'I', 'O') => 656, ('I', 'O', 'O') => 54, ('K', 'A', 'K') => 4845, ('K', 'K', 'A') => 3386, ('K', 'K', 'K') => 3065, ('M', 'H', 'H') => -405, ('M', 'H', 'I') => 201, ('M', 'M', 'H') => -241, ('M', 'M', 'M') => 661, ('M', 'O', 'M') => 841, }\n});\nstatic TQ1: LazyLock<HashMap<(char, char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H', 'H', 'H') => -227, ('B', 'H', 'H', 'I') => 316, ('B', 'H', 'I', 'H') => -132, ('B', 'I', 'H', 'H') => 60, ('B', 'I', 'I', 'I') => 1595, ('B', 'N', 'H', 'H') => -744, ('B', 'O', 'H', 'H') => 225, ('B', 'O', 'O', 'O') => -908, ('O', 'A', 'K', 'K') => 482, ('O', 'H', 'H', 'H') => 281, ('O', 'H', 'I', 'H') => 249, ('O', 'I', 'H', 'I') => 200, ('O', 'I', 'I', 'H') => -68, }\n});\nstatic TQ2: LazyLock<HashMap<(char, char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'I', 'H', 'H') => -1401, ('B', 'I', 'I', 'I') => -1033, ('B', 'K', 'A', 'K') => -543, ('B', 'O', 'O', 'O') => -5591, }\n});\nstatic TQ3: LazyLock<HashMap<(char, char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H', 'H', 'H') => 478, ('B', 'H', 'H', 'M') => -1073, ('B', 'H', 'I', 'H') => 222, ('B', 'H', 'I', 'I') => -504, ('B', 'I', 'I', 'H') => -116, ('B', 'I', 'I', 'I') => -105, ('B', 'M', 'H', 'I') => -863, ('B', 'M', 'H', 'M') => -464, ('B', 'O', 'M', 'H') => 620, ('O', 'H', 'H', 'H') => 346, ('O', 'H', 'H', 'I') => 1729, ('O', 'H', 'I', 'I') => 997, ('O', 'H', 'M', 'H') => 481, ('O', 'I', 'H', 'H') => 623, ('O', 'I', 'I', 'H') => 1344, ('O', 'K', 'A', 'K') => 2792, ('O', 'K', 'H', 'H') => 587, ('O', 'K', 'K', 'A') => 679, ('O', 'O', 'H', 'H') => 110, ('O', 'O', 'I', 'I') => -685, }\n});\nstatic TQ4: LazyLock<HashMap<(char, char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H', 'H', 'H') => -721, ('B', 'H', 'H', 'M') => -3604, ('B', 'H', 'I', 'I') => -966, ('B', 'I', 'I', 'H') => -607, ('B', 'I', 'I', 'I') => -2181, ('O', 'A', 'A', 'A') => -2763, ('O', 'A', 'K', 'K') => 180, ('O', 'H', 'H', 'H') => -294, ('O', 'H', 'H', 'I') => 2446, ('O', 'H', 'H', 'O') => 480, ('O', 'H', 'I', 'H') => -1573, ('O', 'I', 'H', 'H') => 1935, ('O', 'I', 'H', 'I') => -493, ('O', 'I', 'I', 'H') => 626, ('O', 'I', 'I', 'I') => -4007, ('O', 'K', 'A', 'K') => -8156, }\n});\nstatic TW1: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('に', 'つ', 'い') => -4681, ('東', '京', '都') => 2026, }\n});\nstatic TW2: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('あ', 'る', '程') => -2049, ('い', 'っ', 'た') => -1256, ('こ', 'ろ', 'が') => -2434, ('し', 'ょ', 'う') => 3873, ('そ', 'の', '後') => -4430, ('だ', 'っ', 'て') => -1049, ('て', 'い', 'た') => 1833, ('と', 'し', 'て') => -4657, ('と', 'も', 'に') => -4517, ('も', 'の', 'で') => 1882, ('一', '気', 'に') => -792, ('初', 'め', 'て') => -1512, ('同', '時', 'に') => -8097, ('大', 'き', 'な') => -1255, ('対', 'し', 'て') => -2721, ('社', '会', '党') => -3216, }\n});\nstatic TW3: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('い', 'た', 'だ') => -1734, ('し', 'て', 'い') => 1314, ('と', 'し', 'て') => -4314, ('に', 'つ', 'い') => -5483, ('に', 'と', 'っ') => -5989, ('に', '当', 'た') => -6247, ('の', 'で', ',') => -727, ('の', 'で', '、') => -727, ('の', 'も', 'の') => -600, ('れ', 'か', 'ら') => -3752, ('十', '二', '月') => -2287, }\n});\nstatic TW4: LazyLock<HashMap<(char, char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('い', 'う', '.') => 8576, ('い', 'う', '。') => 8576, ('か', 'ら', 'な') => -2348, ('し', 'て', 'い') => 2958, ('た', 'が', ',') => 1516, ('た', 'が', '、') => 1516, ('て', 'い', 'る') => 1538, ('と', 'い', 'う') => 1349, ('ま', 'し', 'た') => 5543, ('ま', 'せ', 'ん') => 1097, ('よ', 'う', 'と') => -4258, ('よ', 'る', 'と') => 5865, }\n});\n\nstatic UC1: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'A' => 484, 'K' => 93, 'M' => 645, 'O' => -505, }\n});\nstatic UC2: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'A' => 819, 'H' => 1059, 'I' => 409, 'M' => 3987, 'N' => 5775, 'O' => 646, }\n});\nstatic UC3: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'A' => -1370, 'I' => 2311, }\n});\nstatic UC4: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'A' => -2643, 'H' => 1809, 'I' => -1032, 'K' => -3450, 'M' => 3565, 'N' => 3876, 'O' => 6646, }\n});\nstatic UC5: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'H' => 313, 'I' => -1238, 'K' => -799, 'M' => 539, 'O' => -831, }\n});\nstatic UC6: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'H' => -506, 'I' => -253, 'K' => 87, 'M' => 247, 'O' => -387, }\n});\nstatic UP1: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'O' => -214, }\n});\nstatic UP2: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'B' => 69, 'O' => 935, }\n});\nstatic UP3: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { 'B' => 189, }\n});\nstatic UQ1: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H') => 21, ('B', 'I') => -12, ('B', 'K') => -99, ('B', 'N') => 142, ('B', 'O') => -56, ('O', 'H') => -95, ('O', 'I') => 477, ('O', 'K') => 410, ('O', 'O') => -2422, }\n});\nstatic UQ2: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'H') => 216, ('B', 'I') => 113, ('O', 'K') => 1759, }\n});\nstatic UQ3: LazyLock<HashMap<(char, char), i32>> = LazyLock::new(|| {\n    hashmap! { ('B', 'A') => -479, ('B', 'H') => 42, ('B', 'I') => 1913, ('B', 'K') => -7198, ('B', 'M') => 3160, ('B', 'N') => 6427, ('B', 'O') => 14761, ('O', 'I') => -827, ('O', 'N') => -3212, }\n});\nstatic UW1: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { ',' => 156, '、' => 156, '「' => -463, 'あ' => -941, 'う' => -127, 'が' => -553, 'き' => 121, 'こ' => 505, 'で' => -201, 'と' => -547, 'ど' => -123, 'に' => -789, 'の' => -185, 'は' => -847, 'も' => -466, 'や' => -470, 'よ' => 182, 'ら' => -292, 'り' => 208, 'れ' => 169, 'を' => -446, 'ん' => -137, '・' => -135, '主' => -402, '京' => -268, '区' => -912, '午' => 871, '国' => -460, '大' => 561, '委' => 729, '市' => -411, '日' => -141, '理' => 361, '生' => -408, '県' => -386, '都' => -718, '｢' => -463, '･' => -135, }\n});\nstatic UW2: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { ',' => -829, '、' => -829, '〇' => 892, '「' => -645, '」' => 3145, 'あ' => -538, 'い' => 505, 'う' => 134, 'お' => -502, 'か' => 1454, 'が' => -856, 'く' => -412, 'こ' => 1141, 'さ' => 878, 'ざ' => 540, 'し' => 1529, 'す' => -675, 'せ' => 300, 'そ' => -1011, 'た' => 188, 'だ' => 1837, 'つ' => -949, 'て' => -291, 'で' => -268, 'と' => -981, 'ど' => 1273, 'な' => 1063, 'に' => -1764, 'の' => 130, 'は' => -409, 'ひ' => -1273, 'べ' => 1261, 'ま' => 600, 'も' => -1263, 'や' => -402, 'よ' => 1639, 'り' => -579, 'る' => -694, 'れ' => 571, 'を' => -2516, 'ん' => 2095, 'ア' => -587, 'カ' => 306, 'キ' => 568, 'ッ' => 831, '三' => -758, '不' => -2150, '世' => -302, '中' => -968, '主' => -861, '事' => 492, '人' => -123, '会' => 978, '保' => 362, '入' => 548, '初' => -3025, '副' => -1566, '北' => -3414, '区' => -422, '大' => -1769, '天' => -865, '太' => -483, '子' => -1519, '学' => 760, '実' => 1023, '小' => -2009, '市' => -813, '年' => -1060, '強' => 1067, '手' => -1519, '揺' => -1033, '政' => 1522, '文' => -1355, '新' => -1682, '日' => -1815, '明' => -1462, '最' => -630, '朝' => -1843, '本' => -1650, '東' => -931, '果' => -665, '次' => -2378, '民' => -180, '気' => -1740, '理' => 752, '発' => 529, '目' => -1584, '相' => -242, '県' => -1165, '立' => -763, '第' => 810, '米' => 509, '自' => -1353, '行' => 838, '西' => -744, '見' => -3874, '調' => 1010, '議' => 1198, '込' => 3041, '開' => 1758, '間' => -1257, '｢' => -645, '｣' => 3145, 'ｯ' => 831, 'ｱ' => -587, 'ｶ' => 306, 'ｷ' => 568, }\n});\nstatic UW3: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { ',' => 4889, '1' => -800, '−' => -1723, '、' => 4889, '々' => -2311, '〇' => 5827, '」' => 2670, '〓' => -3573, 'あ' => -2696, 'い' => 1006, 'う' => 2342, 'え' => 1983, 'お' => -4864, 'か' => -1163, 'が' => 3271, 'く' => 1004, 'け' => 388, 'げ' => 401, 'こ' => -3552, 'ご' => -3116, 'さ' => -1058, 'し' => -395, 'す' => 584, 'せ' => 3685, 'そ' => -5228, 'た' => 842, 'ち' => -521, 'っ' => -1444, 'つ' => -1081, 'て' => 6167, 'で' => 2318, 'と' => 1691, 'ど' => -899, 'な' => -2788, 'に' => 2745, 'の' => 4056, 'は' => 4555, 'ひ' => -2171, 'ふ' => -1798, 'へ' => 1199, 'ほ' => -5516, 'ま' => -4384, 'み' => -120, 'め' => 1205, 'も' => 2323, 'や' => -788, 'よ' => -202, 'ら' => 727, 'り' => 649, 'る' => 5905, 'れ' => 2773, 'わ' => -1207, 'を' => 6620, 'ん' => -518, 'ア' => 551, 'グ' => 1319, 'ス' => 874, 'ッ' => -1350, 'ト' => 521, 'ム' => 1109, 'ル' => 1591, 'ロ' => 2201, 'ン' => 278, '・' => -3794, '一' => -1619, '下' => -1759, '世' => -2087, '両' => 3815, '中' => 653, '主' => -758, '予' => -1193, '二' => 974, '人' => 2742, '今' => 792, '他' => 1889, '以' => -1368, '低' => 811, '何' => 4265, '作' => -361, '保' => -2439, '元' => 4858, '党' => 3593, '全' => 1574, '公' => -3030, '六' => 755, '共' => -1880, '円' => 5807, '再' => 3095, '分' => 457, '初' => 2475, '別' => 1129, '前' => 2286, '副' => 4437, '力' => 365, '動' => -949, '務' => -1872, '化' => 1327, '北' => -1038, '区' => 4646, '千' => -2309, '午' => -783, '協' => -1006, '口' => 483, '右' => 1233, '各' => 3588, '合' => -241, '同' => 3906, '和' => -837, '員' => 4513, '国' => 642, '型' => 1389, '場' => 1219, '外' => -241, '妻' => 2016, '学' => -1356, '安' => -423, '実' => -1008, '家' => 1078, '小' => -513, '少' => -3102, '州' => 1155, '市' => 3197, '平' => -1804, '年' => 2416, '広' => -1030, '府' => 1605, '度' => 1452, '建' => -2352, '当' => -3885, '得' => 1905, '思' => -1291, '性' => 1822, '戸' => -488, '指' => -3973, '政' => -2013, '教' => -1479, '数' => 3222, '文' => -1489, '新' => 1764, '日' => 2099, '旧' => 5792, '昨' => -661, '時' => -1248, '曜' => -951, '最' => -937, '月' => 4125, '期' => 360, '李' => 3094, '村' => 364, '東' => -805, '核' => 5156, '森' => 2438, '業' => 484, '氏' => 2613, '民' => -1694, '決' => -1073, '法' => 1868, '海' => -495, '無' => 979, '物' => 461, '特' => -3850, '生' => -273, '用' => 914, '町' => 1215, '的' => 7313, '直' => -1835, '省' => 792, '県' => 6293, '知' => -1528, '私' => 4231, '税' => 401, '立' => -960, '第' => 1201, '米' => 7767, '系' => 3066, '約' => 3663, '級' => 1384, '統' => -4229, '総' => 1163, '線' => 1255, '者' => 6457, '能' => 725, '自' => -2869, '英' => 785, '見' => 1044, '調' => -562, '財' => -733, '費' => 1777, '車' => 1835, '軍' => 1375, '込' => -1504, '通' => -1136, '選' => -681, '郎' => 1026, '郡' => 4404, '部' => 1200, '金' => 2163, '長' => 421, '開' => -1432, '間' => 1302, '関' => -1282, '雨' => 2009, '電' => -1045, '非' => 2066, '駅' => 1620, '１' => -800, '｣' => 2670, '･' => -3794, 'ｯ' => -1350, 'ｱ' => 551, 'ｽ' => 874, 'ﾄ' => 521, 'ﾑ' => 1109, 'ﾙ' => 1591, 'ﾛ' => 2201, 'ﾝ' => 278, }\n});\nstatic UW4: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { ',' => 3930, '.' => 3508, '―' => -4841, '、' => 3930, '。' => 3508, '〇' => 4999, '「' => 1895, '」' => 3798, '〓' => -5156, 'あ' => 4752, 'い' => -3435, 'う' => -640, 'え' => -2514, 'お' => 2405, 'か' => 530, 'が' => 6006, 'き' => -4482, 'ぎ' => -3821, 'く' => -3788, 'け' => -4376, 'げ' => -4734, 'こ' => 2255, 'ご' => 1979, 'さ' => 2864, 'し' => -843, 'じ' => -2506, 'す' => -731, 'ず' => 1251, 'せ' => 181, 'そ' => 4091, 'た' => 5034, 'だ' => 5408, 'ち' => -3654, 'っ' => -5882, 'つ' => -1659, 'て' => 3994, 'で' => 7410, 'と' => 4547, 'な' => 5433, 'に' => 6499, 'ぬ' => 1853, 'ね' => 1413, 'の' => 7396, 'は' => 8578, 'ば' => 1940, 'ひ' => 4249, 'び' => -4134, 'ふ' => 1345, 'へ' => 6665, 'べ' => -744, 'ほ' => 1464, 'ま' => 1051, 'み' => -2082, 'む' => -882, 'め' => -5046, 'も' => 4169, 'ゃ' => -2666, 'や' => 2795, 'ょ' => -1544, 'よ' => 3351, 'ら' => -2922, 'り' => -9726, 'る' => -14896, 'れ' => -2613, 'ろ' => -4570, 'わ' => -1783, 'を' => 13150, 'ん' => -2352, 'カ' => 2145, 'コ' => 1789, 'セ' => 1287, 'ッ' => -724, 'ト' => -403, 'メ' => -1635, 'ラ' => -881, 'リ' => -541, 'ル' => -856, 'ン' => -3637, '・' => -4371, 'ー' => -11870, '一' => -2069, '中' => 2210, '予' => 782, '事' => -190, '井' => -1768, '人' => 1036, '以' => 544, '会' => 950, '体' => -1286, '作' => 530, '側' => 4292, '先' => 601, '党' => -2006, '共' => -1212, '内' => 584, '円' => 788, '初' => 1347, '前' => 1623, '副' => 3879, '力' => -302, '動' => -740, '務' => -2715, '化' => 776, '区' => 4517, '協' => 1013, '参' => 1555, '合' => -1834, '和' => -681, '員' => -910, '器' => -851, '回' => 1500, '国' => -619, '園' => -1200, '地' => 866, '場' => -1410, '塁' => -2094, '士' => -1413, '多' => 1067, '大' => 571, '子' => -4802, '学' => -1397, '定' => -1057, '寺' => -809, '小' => 1910, '屋' => -1328, '山' => -1500, '島' => -2056, '川' => -2667, '市' => 2771, '年' => 374, '庁' => -4556, '後' => 456, '性' => 553, '感' => 916, '所' => -1566, '支' => 856, '改' => 787, '政' => 2182, '教' => 704, '文' => 522, '方' => -856, '日' => 1798, '時' => 1829, '最' => 845, '月' => -9066, '木' => -485, '来' => -442, '校' => -360, '業' => -1043, '氏' => 5388, '民' => -2716, '気' => -910, '沢' => -939, '済' => -543, '物' => -735, '率' => 672, '球' => -1267, '生' => -1286, '産' => -1101, '田' => -2900, '町' => 1826, '的' => 2586, '目' => 922, '省' => -3485, '県' => 2997, '空' => -867, '立' => -2112, '第' => 788, '米' => 2937, '系' => 786, '約' => 2171, '経' => 1146, '統' => -1169, '総' => 940, '線' => -994, '署' => 749, '者' => 2145, '能' => -730, '般' => -852, '行' => -792, '規' => 792, '警' => -1184, '議' => -244, '谷' => -1000, '賞' => 730, '車' => -1481, '軍' => 1158, '輪' => -1433, '込' => -3370, '近' => 929, '道' => -1291, '選' => 2596, '郎' => -4866, '都' => 1192, '野' => -1100, '銀' => -2213, '長' => 357, '間' => -2344, '院' => -2297, '際' => -2604, '電' => -878, '領' => -1659, '題' => -792, '館' => -1984, '首' => 1749, '高' => 2120, '｢' => 1895, '｣' => 3798, '･' => -4371, 'ｯ' => -724, 'ｰ' => -11870, 'ｶ' => 2145, 'ｺ' => 1789, 'ｾ' => 1287, 'ﾄ' => -403, 'ﾒ' => -1635, 'ﾗ' => -881, 'ﾘ' => -541, 'ﾙ' => -856, 'ﾝ' => -3637, }\n});\nstatic UW5: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { ',' => 465, '.' => -299, '1' => -514, E2 => -32768, ']' => -2762, '、' => 465, '。' => -299, '「' => 363, 'あ' => 1655, 'い' => 331, 'う' => -503, 'え' => 1199, 'お' => 527, 'か' => 647, 'が' => -421, 'き' => 1624, 'ぎ' => 1971, 'く' => 312, 'げ' => -983, 'さ' => -1537, 'し' => -1371, 'す' => -852, 'だ' => -1186, 'ち' => 1093, 'っ' => 52, 'つ' => 921, 'て' => -18, 'で' => -850, 'と' => -127, 'ど' => 1682, 'な' => -787, 'に' => -1224, 'の' => -635, 'は' => -578, 'べ' => 1001, 'み' => 502, 'め' => 865, 'ゃ' => 3350, 'ょ' => 854, 'り' => -208, 'る' => 429, 'れ' => 504, 'わ' => 419, 'を' => -1264, 'ん' => 327, 'イ' => 241, 'ル' => 451, 'ン' => -343, '中' => -871, '京' => 722, '会' => -1153, '党' => -654, '務' => 3519, '区' => -901, '告' => 848, '員' => 2104, '大' => -1296, '学' => -548, '定' => 1785, '嵐' => -1304, '市' => -2991, '席' => 921, '年' => 1763, '思' => 872, '所' => -814, '挙' => 1618, '新' => -1682, '日' => 218, '月' => -4353, '査' => 932, '格' => 1356, '機' => -1508, '氏' => -1347, '田' => 240, '町' => -3912, '的' => -3149, '相' => 1319, '省' => -1052, '県' => -4003, '研' => -997, '社' => -278, '空' => -813, '統' => 1955, '者' => -2233, '表' => 663, '語' => -1073, '議' => 1219, '選' => -1018, '郎' => -368, '長' => 786, '間' => 1191, '題' => 2368, '館' => -689, '１' => -514, '｢' => 363, 'ｲ' => 241, 'ﾙ' => 451, 'ﾝ' => -343, }\n});\nstatic UW6: LazyLock<HashMap<char, i32>> = LazyLock::new(|| {\n    hashmap! { ',' => 227, '.' => 808, '1' => -270, E1 => 306, '、' => 227, '。' => 808, 'あ' => -307, 'う' => 189, 'か' => 241, 'が' => -73, 'く' => -121, 'こ' => -200, 'じ' => 1782, 'す' => 383, 'た' => -428, 'っ' => 573, 'て' => -1014, 'で' => 101, 'と' => -105, 'な' => -253, 'に' => -149, 'の' => -417, 'は' => -236, 'も' => -206, 'り' => 187, 'る' => -135, 'を' => 195, 'ル' => -673, 'ン' => -496, '一' => -277, '中' => 201, '件' => -800, '会' => 624, '前' => 302, '区' => 1792, '員' => -1212, '委' => 798, '学' => -960, '市' => 887, '広' => -695, '後' => 535, '業' => -697, '相' => 753, '社' => -507, '福' => 974, '空' => -822, '者' => 1811, '連' => 463, '郎' => 1082, '１' => -270, 'ﾙ' => -673, 'ﾝ' => -496, }\n});\n\n#[cfg(test)]\nmod tests {\n    use crate::tokenizers::{Token, japanese::JapaneseTokenizer, word::WordTokenizer};\n\n    #[test]\n    fn japanese_tokenizer() {\n        assert_eq!(\n            JapaneseTokenizer::new(WordTokenizer::new(\n                \"お先に失礼します あなたの名前は何ですか 123 abc-872\",\n                40\n            ))\n            .collect::<Vec<_>>(),\n            vec![\n                Token {\n                    word: \"お先\".into(),\n                    from: 0,\n                    to: 6\n                },\n                Token {\n                    word: \"に\".into(),\n                    from: 6,\n                    to: 9\n                },\n                Token {\n                    word: \"失礼\".into(),\n                    from: 9,\n                    to: 15\n                },\n                Token {\n                    word: \"し\".into(),\n                    from: 15,\n                    to: 18\n                },\n                Token {\n                    word: \"ます\".into(),\n                    from: 18,\n                    to: 24\n                },\n                Token {\n                    word: \"あなた\".into(),\n                    from: 25,\n                    to: 34\n                },\n                Token {\n                    word: \"の\".into(),\n                    from: 34,\n                    to: 37\n                },\n                Token {\n                    word: \"名前\".into(),\n                    from: 37,\n                    to: 43\n                },\n                Token {\n                    word: \"は\".into(),\n                    from: 43,\n                    to: 46\n                },\n                Token {\n                    word: \"何\".into(),\n                    from: 46,\n                    to: 49\n                },\n                Token {\n                    word: \"です\".into(),\n                    from: 49,\n                    to: 55\n                },\n                Token {\n                    word: \"か\".into(),\n                    from: 55,\n                    to: 58\n                },\n                Token {\n                    word: \"123\".into(),\n                    from: 59,\n                    to: 62\n                },\n                Token {\n                    word: \"abc\".into(),\n                    from: 63,\n                    to: 66\n                },\n                Token {\n                    word: \"872\".into(),\n                    from: 67,\n                    to: 70\n                }\n            ]\n        );\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/tokenizers/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod chinese;\npub mod japanese;\npub mod space;\npub mod stream;\npub mod types;\npub mod word;\n\nuse std::borrow::Cow;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Token<T> {\n    pub word: T,\n    pub from: usize,\n    pub to: usize,\n}\n\npub trait InnerToken<'x>: Sized {\n    fn new_alphabetic(value: impl Into<Cow<'x, str>>) -> Self;\n    fn unwrap_alphabetic(self) -> Cow<'x, str>;\n    fn is_alphabetic(&self) -> bool;\n    fn is_alphabetic_8bit(&self) -> bool;\n}\n\nimpl<'x> InnerToken<'x> for Cow<'x, str> {\n    fn new_alphabetic(value: impl Into<Cow<'x, str>>) -> Self {\n        value.into()\n    }\n\n    fn is_alphabetic(&self) -> bool {\n        true\n    }\n\n    fn is_alphabetic_8bit(&self) -> bool {\n        !self.is_ascii()\n    }\n\n    fn unwrap_alphabetic(self) -> Cow<'x, str> {\n        self\n    }\n}\n\nimpl<T> Token<T> {\n    pub fn new(offset: usize, len: usize, word: T) -> Token<T> {\n        debug_assert!(offset <= u32::MAX as usize);\n        debug_assert!(len <= u8::MAX as usize);\n        Token {\n            from: offset,\n            to: offset + len,\n            word,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/tokenizers/space.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::str::Chars;\n\npub struct SpaceTokenizer<'x> {\n    iterator: Chars<'x>,\n    token: String,\n    max_token_length: usize,\n}\n\nimpl SpaceTokenizer<'_> {\n    pub fn new(text: &'_ str, max_token_length: usize) -> SpaceTokenizer<'_> {\n        SpaceTokenizer {\n            iterator: text.chars(),\n            token: String::new(),\n            max_token_length,\n        }\n    }\n}\n\nimpl Iterator for SpaceTokenizer<'_> {\n    type Item = String;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        for ch in self.iterator.by_ref() {\n            if ch.is_alphanumeric() {\n                if ch.is_uppercase() {\n                    for ch in ch.to_lowercase() {\n                        self.token.push(ch);\n                    }\n                } else {\n                    self.token.push(ch);\n                }\n            } else if !self.token.is_empty() {\n                if self.token.len() < self.max_token_length {\n                    return Some(std::mem::take(&mut self.token));\n                } else {\n                    self.token.clear();\n                }\n            }\n        }\n\n        if !self.token.is_empty() {\n            if self.token.len() < self.max_token_length {\n                return Some(std::mem::take(&mut self.token));\n            } else {\n                self.token.clear();\n            }\n        }\n\n        None\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/tokenizers/stream.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    language::{\n        Language,\n        detect::{LanguageDetector, MIN_LANGUAGE_SCORE},\n        stemmer::STEMMER_MAP,\n        stopwords::{STOP_WORDS, StopwordFnc},\n    },\n    tokenizers::{chinese::JIEBA, japanese},\n};\nuse std::borrow::Cow;\n\npub struct WordStemTokenizer {\n    stemmer: Stemmer,\n    stop_words: Option<StopwordFnc>,\n}\n\nenum Stemmer {\n    IndoEuropean(rust_stemmers::Stemmer),\n    Mandarin,\n    Japanese,\n    None,\n}\n\nimpl WordStemTokenizer {\n    pub fn new(text: &str) -> Self {\n        // Detect language\n        let (mut language, score) =\n            LanguageDetector::detect_single(text).unwrap_or((Language::English, 1.0));\n        if score < MIN_LANGUAGE_SCORE {\n            language = Language::English;\n        }\n\n        Self {\n            stemmer: match language {\n                Language::Mandarin => Stemmer::Mandarin,\n                Language::Japanese => Stemmer::Japanese,\n                _ => STEMMER_MAP[language as usize]\n                    .map(|algo| Stemmer::IndoEuropean(rust_stemmers::Stemmer::create(algo)))\n                    .unwrap_or(Stemmer::None),\n            },\n            stop_words: STOP_WORDS[language as usize],\n        }\n    }\n\n    pub fn tokenize<'x>(&self, word: &'x str, mut cb: impl FnMut(Cow<'x, str>)) {\n        if self.stop_words.is_some_and(|sw| sw(word)) {\n            return;\n        }\n        match &self.stemmer {\n            Stemmer::IndoEuropean(stemmer) => {\n                cb(stemmer.stem(word));\n            }\n            Stemmer::Mandarin => {\n                for word in JIEBA.cut(word, false) {\n                    cb(Cow::from(word));\n                }\n            }\n            Stemmer::Japanese => {\n                for word in japanese::tokenize(word) {\n                    cb(Cow::from(word));\n                }\n            }\n            Stemmer::None => {\n                cb(Cow::from(word));\n            }\n        }\n    }\n}\n\n#[cfg(test)]\npub mod tests {\n    use crate::tokenizers::{\n        stream::WordStemTokenizer,\n        types::{TokenType, TypesTokenizer},\n    };\n\n    #[test]\n    fn stream_tokenizer() {\n        let inputs = [\n            (\n                \"The quick brown fox jumps over the lazy dog\",\n                vec![\"quick\", \"brown\", \"fox\", \"jump\", \"lazi\", \"dog\"],\n            ),\n            (\n                \"Jovencillo emponzoñado de whisky: ¡qué figurota exhibe!\",\n                vec![\"jovencill\", \"emponzoñ\", \"whisky\", \"figurot\", \"exhib\"],\n            ),\n            (\n                \"Ma la volpe col suo balzo ha raggiunto il quieto Fido\",\n                vec![\"volp\", \"balz\", \"raggiunt\", \"quiet\", \"fid\"],\n            ),\n            (\n                \"Jaz em prisão bota que vexa dez cegonhas felizes\",\n                vec![\"jaz\", \"prisã\", \"bot\", \"vex\", \"dez\", \"cegonh\", \"feliz\"],\n            ),\n            (\n                \"Zwölf Boxkämpfer jagten Victor quer über den großen Sylter Deich\",\n                vec![\n                    \"zwolf\", \"boxkampf\", \"jagt\", \"victor\", \"quer\", \"gross\", \"sylt\", \"deich\",\n                ],\n            ),\n            (\n                \"עטלף אבק נס דרך מזגן שהתפוצץ כי חם\",\n                vec![\"עטלף\", \"אבק\", \"נס\", \"דרך\", \"מזגן\", \"שהתפוצץ\", \"כי\", \"חם\"],\n            ),\n            (\n                \"Съешь ещё этих мягких французских булок, да выпей же чаю\",\n                vec![\n                    \"съеш\",\n                    \"ещё\",\n                    \"эт\",\n                    \"мягк\",\n                    \"французск\",\n                    \"булок\",\n                    \"вып\",\n                    \"ча\",\n                ],\n            ),\n            (\n                \"Чуєш їх, доцю, га? Кумедна ж ти, прощайся без ґольфів!\",\n                vec![\n                    \"чуєш\",\n                    \"їх\",\n                    \"доцю\",\n                    \"га\",\n                    \"кумедна\",\n                    \"ж\",\n                    \"ти\",\n                    \"прощайся\",\n                    \"без\",\n                    \"ґольфів\",\n                ],\n            ),\n            (\n                \"Љубазни фењерџија чађавог лица хоће да ми покаже штос\",\n                vec![\n                    \"љубазни\",\n                    \"фењерџија\",\n                    \"чађавог\",\n                    \"лица\",\n                    \"хоће\",\n                    \"да\",\n                    \"ми\",\n                    \"покаже\",\n                    \"штос\",\n                ],\n            ),\n            (\n                \"Pijamalı hasta yağız şoföre çabucak güvendi\",\n                vec![\"pijamalı\", \"hasta\", \"yağız\", \"şoför\", \"çabucak\", \"güvendi\"],\n            ),\n            (\"己所不欲,勿施于人。\", vec![\"己所不欲\", \"勿施于人\"]),\n            (\n                \"井の中の蛙大海を知らず\",\n                vec![\"井\", \"の\", \"中\", \"の\", \"蛙大\", \"海\", \"を\", \"知ら\", \"ず\"],\n            ),\n            (\"시작이 반이다\", vec![\"시작이\", \"반이다\"]),\n        ];\n\n        for (input, expect) in inputs.iter() {\n            let tokenizer = WordStemTokenizer::new(input);\n            let mut result = Vec::new();\n            for token in TypesTokenizer::new(&input.to_lowercase()) {\n                if let TokenType::Alphabetic(word) = token.word {\n                    tokenizer.tokenize(word, |t| {\n                        result.push(t.into_owned());\n                    });\n                }\n            }\n\n            assert_eq!(&result, expect,);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/tokenizers/types.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::str::CharIndices;\n\nuse super::Token;\n\n#[derive(Debug)]\npub struct TypesTokenizer<'x> {\n    text: &'x str,\n    iter: CharIndices<'x>,\n    tokens: Vec<Token<TokenType<&'x str, &'x str, &'x str, &'x str>>>,\n    peek_pos: usize,\n    last_ch_is_space: bool,\n    last_token_is_dot: bool,\n    eof: bool,\n    tokenize_urls: bool,\n    tokenize_urls_without_scheme: bool,\n    tokenize_emails: bool,\n    tokenize_numbers: bool,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum TokenType<T, E, U, I> {\n    Alphabetic(T),\n    Alphanumeric(T),\n    Integer(T),\n    Other(char),\n    Punctuation(char),\n    Space,\n\n    // Detected types\n    Url(U),\n    UrlNoScheme(U),\n    UrlNoHost(T),\n    IpAddr(I),\n    Email(E),\n    Float(T),\n}\n\nimpl Copy for Token<TokenType<&'_ str, &'_ str, &'_ str, &'_ str>> {}\n\nimpl<'x> Iterator for TypesTokenizer<'x> {\n    type Item = Token<TokenType<&'x str, &'x str, &'x str, &'x str>>;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        let token = self.peek()?;\n        let last_is_dot = self.last_token_is_dot;\n        self.last_token_is_dot = matches!(token.word, TokenType::Punctuation('.'));\n\n        // Try parsing URL with scheme\n        if self.tokenize_urls\n            && matches!(\n            token.word,\n            TokenType::Alphabetic(t) | TokenType::Alphanumeric(t)\n            if t.len() <= 8 && t.is_ascii())\n            && self.try_skip_url_scheme()\n        {\n            if let Some(url) = self.try_parse_url(token.into()) {\n                self.peek_advance();\n                return Some(url);\n            } else {\n                self.peek_rewind();\n            }\n        }\n\n        // Try parsing email\n        if self.tokenize_emails && token.word.is_email_atom() {\n            self.peek_rewind();\n            if let Some(email) = self.try_parse_email() {\n                self.peek_advance();\n                return Some(email);\n            }\n            self.peek_rewind();\n        }\n\n        // Try parsing URL without scheme\n        if self.tokenize_urls_without_scheme && token.word.is_domain_atom(true) {\n            self.peek_rewind();\n            if let Some(url) = self.try_parse_url(None) {\n                self.peek_advance();\n                return Some(url);\n            }\n            self.peek_rewind();\n        }\n\n        // Try parsing currencies and floating point numbers\n        if self.tokenize_numbers\n            && !last_is_dot\n            && let Some(num) = self.try_parse_number()\n        {\n            self.peek_advance();\n            return Some(num);\n        }\n\n        self.peek_rewind();\n        self.next_()\n    }\n}\n\nimpl<'x> TypesTokenizer<'x> {\n    pub fn new(text: &'x str) -> Self {\n        Self {\n            text,\n            iter: text.char_indices(),\n            tokens: Vec::new(),\n            eof: false,\n            peek_pos: 0,\n            last_ch_is_space: false,\n            last_token_is_dot: false,\n            tokenize_urls: true,\n            tokenize_urls_without_scheme: true,\n            tokenize_emails: true,\n            tokenize_numbers: true,\n        }\n    }\n\n    pub fn tokenize_urls(mut self, tokenize: bool) -> Self {\n        self.tokenize_urls = tokenize;\n        self\n    }\n\n    pub fn tokenize_urls_without_scheme(mut self, tokenize: bool) -> Self {\n        self.tokenize_urls_without_scheme = tokenize;\n        self\n    }\n\n    pub fn tokenize_emails(mut self, tokenize: bool) -> Self {\n        self.tokenize_emails = tokenize;\n        self\n    }\n\n    pub fn tokenize_numbers(mut self, tokenize: bool) -> Self {\n        self.tokenize_numbers = tokenize;\n        self\n    }\n\n    fn consume(&mut self) -> bool {\n        let mut has_alpha = false;\n        let mut has_number = false;\n\n        let mut start_pos = usize::MAX;\n        let mut end_pos = usize::MAX;\n\n        let mut stop_char = None;\n\n        for (pos, ch) in self.iter.by_ref() {\n            if ch.is_alphabetic() {\n                has_alpha = true;\n            } else if ch.is_ascii_digit() {\n                has_number = true;\n            } else {\n                let last_was_space = self.last_ch_is_space;\n                self.last_ch_is_space = ch.is_whitespace();\n                stop_char = Token {\n                    word: if self.last_ch_is_space {\n                        if last_was_space {\n                            continue;\n                        } else {\n                            TokenType::Space\n                        }\n                    } else if ch.is_ascii() {\n                        TokenType::Punctuation(ch)\n                    } else {\n                        TokenType::Other(ch)\n                    },\n                    from: pos,\n                    to: pos + ch.len_utf8(),\n                }\n                .into();\n                break;\n            }\n            self.last_ch_is_space = false;\n\n            if start_pos == usize::MAX {\n                start_pos = pos;\n            }\n            end_pos = pos + ch.len_utf8();\n        }\n\n        if start_pos != usize::MAX {\n            let text = &self.text[start_pos..end_pos];\n\n            self.tokens.push(Token {\n                word: if has_alpha && has_number {\n                    TokenType::Alphanumeric(text)\n                } else if has_alpha {\n                    TokenType::Alphabetic(text)\n                } else {\n                    TokenType::Integer(text)\n                },\n                from: start_pos,\n                to: end_pos,\n            });\n            if let Some(stop_char) = stop_char {\n                self.tokens.push(stop_char);\n            }\n            true\n        } else if let Some(stop_char) = stop_char {\n            self.tokens.push(stop_char);\n            true\n        } else {\n            self.eof = true;\n            false\n        }\n    }\n\n    fn next_(&mut self) -> Option<Token<TokenType<&'x str, &'x str, &'x str, &'x str>>> {\n        if self.tokens.is_empty() && !self.eof {\n            self.consume();\n        }\n        if !self.tokens.is_empty() {\n            Some(self.tokens.remove(0))\n        } else {\n            None\n        }\n    }\n\n    fn peek(&mut self) -> Option<Token<TokenType<&'x str, &'x str, &'x str, &'x str>>> {\n        while self.tokens.len() <= self.peek_pos && !self.eof {\n            self.consume();\n        }\n        self.tokens.get(self.peek_pos).map(|t| {\n            self.peek_pos += 1;\n            *t\n        })\n    }\n\n    fn peek_advance(&mut self) {\n        if self.peek_pos > 0 {\n            self.tokens.drain(..self.peek_pos);\n            self.peek_pos = 0;\n        }\n    }\n\n    #[inline(always)]\n    fn peek_rewind(&mut self) {\n        self.peek_pos = 0;\n    }\n\n    fn try_parse_url(\n        &mut self,\n        scheme_token: Option<Token<TokenType<&'x str, &'x str, &'x str, &'x str>>>,\n    ) -> Option<Token<TokenType<&'x str, &'x str, &'x str, &'x str>>> {\n        let (has_scheme, allow_blank_host) = scheme_token.as_ref().map_or((false, false), |t| {\n            (\n                true,\n                matches!(t.word, TokenType::Alphabetic(s) if s.eq_ignore_ascii_case(\"file\")),\n            )\n        });\n        if has_scheme {\n            let restore_pos = self.peek_pos;\n            let mut has_user_info = false;\n            while let Some(token) = self.peek() {\n                match token.word {\n                    TokenType::Punctuation('@') => {\n                        has_user_info = true;\n                        break;\n                    }\n                    TokenType::Alphabetic(_)\n                    | TokenType::Alphanumeric(_)\n                    | TokenType::Integer(_)\n                    | TokenType::Punctuation(\n                        '-' | '.' | '_' | '~' | '!' | '$' | '&' | '\\'' | '(' | ')' | '*' | '+'\n                        | ',' | ';' | '=' | ':',\n                    ) => (),\n                    _ => break,\n                }\n            }\n\n            if !has_user_info {\n                self.peek_pos = restore_pos;\n            }\n        }\n\n        // Try parsing hostname\n        let mut is_valid_host = true;\n        let (host_start_pos, mut end_pos, is_ip) = if has_scheme {\n            let mut start_pos = usize::MAX;\n            let mut end_pos = usize::MAX;\n            let mut restore_pos = self.peek_pos;\n\n            let mut text_count = 0;\n            let mut int_count = 0;\n            let mut dot_count = 0;\n            let mut is_ipv6 = false;\n\n            let mut last_label_is_tld = false;\n\n            while let Some(token) = self.peek() {\n                match token.word {\n                    TokenType::Alphabetic(text) | TokenType::Alphanumeric(text) => {\n                        last_label_is_tld = text.len() >= 2\n                            && psl::Psl::find(\n                                &psl::List,\n                                [text.to_ascii_lowercase().as_bytes()].into_iter(),\n                            )\n                            .typ\n                            .is_some();\n                        text_count += 1;\n                    }\n                    TokenType::Integer(text) => {\n                        if text.len() <= 3 {\n                            int_count += 1;\n                        }\n                    }\n                    TokenType::Punctuation('.') => {\n                        dot_count += 1;\n                        continue;\n                    }\n                    TokenType::Punctuation('[') if start_pos == usize::MAX => {\n                        let (_, to) = self.try_parse_ipv6(token.from)?;\n                        start_pos = token.from;\n                        end_pos = to;\n                        restore_pos = self.peek_pos;\n                        is_ipv6 = true;\n                        break;\n                    }\n                    TokenType::Punctuation(\n                        '-' | '_' | '~' | '!' | '$' | '&' | '\\'' | '(' | ')' | '*' | '+' | ','\n                        | ';' | '=' | ':' | '%',\n                    ) => {\n                        continue;\n                    }\n                    TokenType::Punctuation('/') if allow_blank_host => {\n                        // Allow file://../ urls\n                        end_pos = token.from;\n                        restore_pos = self.peek_pos - 1;\n                        break;\n                    }\n                    _ => break,\n                }\n\n                if start_pos == usize::MAX {\n                    start_pos = token.from;\n                }\n                end_pos = token.to;\n                restore_pos = self.peek_pos;\n            }\n\n            self.peek_pos = restore_pos;\n            let is_ip = is_ipv6 || (int_count == 4 && dot_count == 3 && text_count == 0);\n            if end_pos != usize::MAX {\n                is_valid_host =\n                    (last_label_is_tld && dot_count >= 1 && (text_count + int_count) >= 2) || is_ip;\n                (start_pos, end_pos, is_ip)\n            } else {\n                return None;\n            }\n        } else {\n            // Strict hostname parsing\n            self.try_parse_hostname()?\n        };\n\n        // Try parsing port\n        let start_pos = scheme_token.map(|t| t.from).unwrap_or(host_start_pos);\n        let mut restore_pos = self.peek_pos;\n        let mut has_port = false;\n        let mut last_is_colon = false;\n        let mut found_query_start = false;\n        while let Some(token) = self.peek() {\n            match token.word {\n                TokenType::Punctuation(':') if !last_is_colon && !has_port => {\n                    last_is_colon = true;\n                }\n                TokenType::Integer(_) if last_is_colon => {\n                    has_port = true;\n                    last_is_colon = false;\n                    restore_pos = self.peek_pos;\n                    end_pos = token.to;\n                }\n                TokenType::Punctuation('/' | '?') if !last_is_colon => {\n                    found_query_start = true;\n                    end_pos = token.to;\n                    break;\n                }\n                _ => {\n                    self.peek_pos = restore_pos;\n                    break;\n                }\n            }\n        }\n\n        // Try parsing query\n        if found_query_start {\n            restore_pos = self.peek_pos;\n            let mut p_count = 0;\n            let mut b_count = 0;\n            let mut c_count = 0;\n            let mut seen_quote = false;\n            while let Some(token) = self.peek() {\n                match token.word {\n                    TokenType::Alphabetic(_)\n                    | TokenType::Alphanumeric(_)\n                    | TokenType::Integer(_)\n                    | TokenType::Other(_) => {}\n                    TokenType::Punctuation('(') => {\n                        p_count += 1;\n                        continue;\n                    }\n                    TokenType::Punctuation('[') => {\n                        b_count += 1;\n                        continue;\n                    }\n                    TokenType::Punctuation('{') => {\n                        c_count += 1;\n                        continue;\n                    }\n                    TokenType::Punctuation(')') if p_count > 0 => {\n                        p_count -= 1;\n                    }\n                    TokenType::Punctuation(']') if b_count > 0 => {\n                        b_count -= 1;\n                    }\n                    TokenType::Punctuation('}') if c_count > 0 => {\n                        c_count -= 1;\n                    }\n                    TokenType::Punctuation('\\'') => {\n                        if !seen_quote {\n                            seen_quote = true;\n                            continue;\n                        } else {\n                            seen_quote = false;\n                        }\n                    }\n                    TokenType::Punctuation('/') => {}\n                    TokenType::Punctuation(\n                        '-' | '_' | '~' | '!' | '$' | '&' | '*' | '+' | ',' | ';' | '=' | ':' | '%'\n                        | '?' | '.' | '@',\n                    ) => {\n                        continue;\n                    }\n                    _ => break,\n                }\n                end_pos = token.to;\n                restore_pos = self.peek_pos;\n            }\n            self.peek_pos = restore_pos;\n        }\n\n        let word = &self.text[start_pos..end_pos];\n        Token {\n            word: if has_scheme {\n                if is_valid_host {\n                    TokenType::Url(word)\n                } else {\n                    TokenType::UrlNoHost(word)\n                }\n            } else if is_ip && !found_query_start {\n                TokenType::IpAddr(word)\n            } else {\n                TokenType::UrlNoScheme(word)\n            },\n            from: start_pos,\n            to: end_pos,\n        }\n        .into()\n    }\n\n    fn try_parse_email(&mut self) -> Option<Token<TokenType<&'x str, &'x str, &'x str, &'x str>>> {\n        // Start token is a valid local part atom\n        let start_token = self.peek()?;\n        let mut last_is_dot = false;\n\n        // Find local part\n        loop {\n            let token = self.peek()?;\n            if token.to - start_token.from > 255 {\n                return None;\n            }\n            match token.word {\n                word if word.is_email_atom() => {\n                    last_is_dot = false;\n                }\n                TokenType::Punctuation('@') if !last_is_dot => {\n                    break;\n                }\n                TokenType::Punctuation('.') if !last_is_dot => {\n                    last_is_dot = true;\n                }\n                _ => {\n                    return None;\n                }\n            }\n        }\n\n        // Obtain domain part\n        let (_, end_pos, _) = self.try_parse_hostname()?;\n\n        Token {\n            word: TokenType::Email(&self.text[start_token.from..end_pos]),\n            from: start_token.from,\n            to: end_pos,\n        }\n        .into()\n    }\n\n    fn try_parse_hostname(&mut self) -> Option<(usize, usize, bool)> {\n        let mut last_ch = u8::MAX;\n        let mut has_int = false;\n        let mut has_alpha = false;\n        let mut last_label_is_tld = false;\n\n        let mut dot_count = 0;\n        let mut start_pos = usize::MAX;\n        let mut end_pos = usize::MAX;\n        let mut restore_pos = self.peek_pos;\n\n        while let Some(token) = self.peek() {\n            match token.word {\n                TokenType::Punctuation('.') if last_ch == 0 && start_pos != usize::MAX => {\n                    last_ch = b'.';\n                    dot_count += 1;\n                    continue;\n                }\n                TokenType::Punctuation('-') if last_ch == 0 || last_ch == b'-' => {\n                    last_ch = b'-';\n                    continue;\n                }\n                TokenType::Punctuation('[') if start_pos == usize::MAX => {\n                    return self\n                        .try_parse_ipv6(token.from)\n                        .map(|(from, to)| (from, to, true));\n                }\n                TokenType::Alphabetic(text) | TokenType::Alphanumeric(text) if text.len() <= 63 => {\n                    last_label_is_tld = text.len() >= 2\n                        && psl::Psl::find(\n                            &psl::List,\n                            [text.to_ascii_lowercase().as_bytes()].into_iter(),\n                        )\n                        .typ\n                        .is_some();\n                    has_alpha = true;\n                    last_ch = 0;\n                }\n                TokenType::Other(_) => {\n                    has_alpha = true;\n                    last_label_is_tld = false;\n                    last_ch = 0;\n                }\n                TokenType::Integer(text) => {\n                    if text.len() <= 3 {\n                        has_int = true;\n                    }\n                    last_label_is_tld = false;\n                    last_ch = 0;\n                }\n                _ => {\n                    break;\n                }\n            }\n\n            if start_pos == usize::MAX {\n                start_pos = token.from;\n            }\n            end_pos = token.to;\n            restore_pos = self.peek_pos;\n\n            if end_pos - start_pos > 255 {\n                return None;\n            }\n        }\n        self.peek_pos = restore_pos;\n\n        if last_ch == b'.' {\n            dot_count -= 1;\n        }\n\n        let is_ipv4 = has_int && !has_alpha && dot_count == 3;\n        if end_pos != usize::MAX && dot_count >= 1 && (last_label_is_tld || is_ipv4) {\n            (start_pos, end_pos, is_ipv4).into()\n        } else {\n            None\n        }\n    }\n\n    fn try_parse_ipv6(&mut self, start_pos: usize) -> Option<(usize, usize)> {\n        let mut found_colon = false;\n        let mut last_ch = u8::MAX;\n\n        while let Some(token) = self.peek() {\n            match token.word {\n                TokenType::Integer(_) | TokenType::Alphanumeric(_) => {\n                    last_ch = 0;\n                }\n                TokenType::Punctuation(':') if last_ch != b'.' => {\n                    found_colon = true;\n                    last_ch = b':';\n                }\n                TokenType::Punctuation('.') if last_ch == 0 => {\n                    last_ch = b'.';\n                }\n                TokenType::Punctuation(']') if found_colon && last_ch == 0 => {\n                    return (start_pos, token.to).into();\n                }\n                _ => return None,\n            }\n        }\n\n        None\n    }\n\n    fn try_parse_number(&mut self) -> Option<Token<TokenType<&'x str, &'x str, &'x str, &'x str>>> {\n        self.peek_rewind();\n        let mut start_pos = usize::MAX;\n        let mut end_pos = usize::MAX;\n        let mut restore_pos = self.peek_pos;\n\n        let mut seen_integer = 0;\n        let mut seen_dot = false;\n\n        while let Some(token) = self.peek() {\n            match token.word {\n                TokenType::Punctuation('-') if start_pos == usize::MAX => {}\n                TokenType::Integer(_) if seen_integer == 0 || seen_dot => {\n                    seen_integer += 1;\n                }\n                TokenType::Punctuation('.') if seen_integer != 0 => {\n                    if !seen_dot {\n                        seen_dot = true;\n                        continue;\n                    } else {\n                        // Avoid parsing num.num.num as floats\n                        return None;\n                    }\n                }\n                _ => break,\n            }\n\n            if start_pos == usize::MAX {\n                start_pos = token.from;\n            }\n            end_pos = token.to;\n            restore_pos = self.peek_pos;\n        }\n\n        self.peek_pos = restore_pos;\n\n        if seen_integer > 0 {\n            let text = &self.text[start_pos..end_pos];\n\n            Token {\n                word: if seen_integer == 2 {\n                    TokenType::Float(text)\n                } else {\n                    TokenType::Integer(text)\n                },\n                from: start_pos,\n                to: end_pos,\n            }\n            .into()\n        } else {\n            None\n        }\n    }\n\n    fn try_skip_url_scheme(&mut self) -> bool {\n        enum State {\n            None,\n            PlusAlpha,\n            Colon,\n            Slash1,\n            Slash2,\n        }\n        let mut state = State::None;\n\n        while let Some(token) = self.peek() {\n            state = match (token.word, state) {\n                (TokenType::Punctuation(':'), State::None | State::Colon) => State::Slash1,\n                (TokenType::Punctuation('/'), State::Slash1) => State::Slash2,\n                (TokenType::Punctuation('/'), State::Slash2) => return true,\n                (TokenType::Punctuation('+'), State::None) => State::PlusAlpha,\n                (TokenType::Alphabetic(t) | TokenType::Alphanumeric(t), State::PlusAlpha)\n                    if t.is_ascii() =>\n                {\n                    State::Colon\n                }\n                _ => break,\n            };\n        }\n        self.peek_rewind();\n        false\n    }\n}\n\nimpl<T, E, U, I> TokenType<T, E, U, I> {\n    fn is_email_atom(&self) -> bool {\n        matches!(\n            self,\n            TokenType::Alphabetic(_)\n                | TokenType::Integer(_)\n                | TokenType::Alphanumeric(_)\n                | TokenType::Other(_)\n                | TokenType::Punctuation(\n                    '!' | '#'\n                        | '$'\n                        | '%'\n                        | '&'\n                        | '\\''\n                        | '*'\n                        | '+'\n                        | '-'\n                        | '/'\n                        | '='\n                        | '?'\n                        | '^'\n                        | '_'\n                        | '`'\n                        | '{'\n                        | '|'\n                        | '}'\n                        | '~',\n                )\n        )\n    }\n\n    fn is_domain_atom(&self, is_start: bool) -> bool {\n        matches!(\n            self,\n            TokenType::Alphabetic(_)\n                | TokenType::Integer(_)\n                | TokenType::Alphanumeric(_)\n                | TokenType::Other(_)\n        ) || (!is_start && matches!(self, TokenType::Punctuation('-')))\n    }\n}\n\n#[cfg(test)]\nmod test {\n\n    use super::{TokenType, TypesTokenizer};\n\n    #[test]\n    fn type_tokenizer() {\n        // Credits: test suite from linkify crate\n        for (text, expected) in [\n            (\"\", vec![]),\n            (\"foo\", vec![TokenType::Alphabetic(\"foo\")]),\n            (\":\", vec![TokenType::Punctuation(':')]),\n            (\n                \"://\",\n                vec![\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                ],\n            ),\n            (\n                \":::\",\n                vec![\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation(':'),\n                ],\n            ),\n            (\n                \"://foo\",\n                vec![\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"foo\"),\n                ],\n            ),\n            (\n                \"1://foo\",\n                vec![\n                    TokenType::Integer(\"1\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"foo\"),\n                ],\n            ),\n            (\n                \"123://foo\",\n                vec![\n                    TokenType::Integer(\"123\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"foo\"),\n                ],\n            ),\n            (\n                \"+://foo\",\n                vec![\n                    TokenType::Punctuation('+'),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"foo\"),\n                ],\n            ),\n            (\n                \"-://foo\",\n                vec![\n                    TokenType::Punctuation('-'),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"foo\"),\n                ],\n            ),\n            (\n                \".://foo\",\n                vec![\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"foo\"),\n                ],\n            ),\n            (\"1abc://foo\", vec![TokenType::UrlNoHost(\"1abc://foo\")]),\n            (\"a://foo\", vec![TokenType::UrlNoHost(\"a://foo\")]),\n            (\"a123://foo\", vec![TokenType::UrlNoHost(\"a123://foo\")]),\n            (\"a123b://foo\", vec![TokenType::UrlNoHost(\"a123b://foo\")]),\n            (\"a+b://foo\", vec![TokenType::UrlNoHost(\"a+b://foo\")]),\n            (\n                \"a-b://foo\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('-'),\n                    TokenType::UrlNoHost(\"b://foo\"),\n                ],\n            ),\n            (\n                \"a.b://foo\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::UrlNoHost(\"b://foo\"),\n                ],\n            ),\n            (\"ABC://foo\", vec![TokenType::UrlNoHost(\"ABC://foo\")]),\n            (\n                \".http://example.org/\",\n                vec![\n                    TokenType::Punctuation('.'),\n                    TokenType::Url(\"http://example.org/\"),\n                ],\n            ),\n            (\n                \"1.http://example.org/\",\n                vec![\n                    TokenType::Integer(\"1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Url(\"http://example.org/\"),\n                ],\n            ),\n            (\n                \"ab://\",\n                vec![\n                    TokenType::Alphabetic(\"ab\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                ],\n            ),\n            (\n                \"file://\",\n                vec![\n                    TokenType::Alphabetic(\"file\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                ],\n            ),\n            (\n                \"file:// \",\n                vec![\n                    TokenType::Alphabetic(\"file\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Space,\n                ],\n            ),\n            (\n                \"\\\"file://\\\"\",\n                vec![\n                    TokenType::Punctuation('\"'),\n                    TokenType::Alphabetic(\"file\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('\"'),\n                ],\n            ),\n            (\n                \"\\\"file://...\\\", \",\n                vec![\n                    TokenType::Punctuation('\"'),\n                    TokenType::Alphabetic(\"file\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation(','),\n                    TokenType::Space,\n                ],\n            ),\n            (\n                \"file://somefile\",\n                vec![TokenType::UrlNoHost(\"file://somefile\")],\n            ),\n            (\n                \"file://../relative\",\n                vec![TokenType::UrlNoHost(\"file://../relative\")],\n            ),\n            (\n                \"http://a.\",\n                vec![\n                    TokenType::UrlNoHost(\"http://a\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\"http://127.0.0.1\", vec![TokenType::Url(\"http://127.0.0.1\")]),\n            (\n                \"http://127.0.0.1/\",\n                vec![TokenType::Url(\"http://127.0.0.1/\")],\n            ),\n            (\"ab://c\", vec![TokenType::UrlNoHost(\"ab://c\")]),\n            (\n                \"http://example.org/\",\n                vec![TokenType::Url(\"http://example.org/\")],\n            ),\n            (\n                \"http://example.org/123\",\n                vec![TokenType::Url(\"http://example.org/123\")],\n            ),\n            (\n                \"http://example.org/?foo=test&bar=123\",\n                vec![TokenType::Url(\"http://example.org/?foo=test&bar=123\")],\n            ),\n            (\n                \"http://example.org/?foo=%20\",\n                vec![TokenType::Url(\"http://example.org/?foo=%20\")],\n            ),\n            (\n                \"http://example.org/%3C\",\n                vec![TokenType::Url(\"http://example.org/%3C\")],\n            ),\n            (\"example.org/\", vec![TokenType::UrlNoScheme(\"example.org/\")]),\n            (\n                \"example.org/123\",\n                vec![TokenType::UrlNoScheme(\"example.org/123\")],\n            ),\n            (\n                \"example.org/?foo=test&bar=123\",\n                vec![TokenType::UrlNoScheme(\"example.org/?foo=test&bar=123\")],\n            ),\n            (\n                \"example.org/?foo=%20\",\n                vec![TokenType::UrlNoScheme(\"example.org/?foo=%20\")],\n            ),\n            (\n                \"example.org/%3C\",\n                vec![TokenType::UrlNoScheme(\"example.org/%3C\")],\n            ),\n            (\n                \"foo http://example.org/\",\n                vec![\n                    TokenType::Alphabetic(\"foo\"),\n                    TokenType::Space,\n                    TokenType::Url(\"http://example.org/\"),\n                ],\n            ),\n            (\n                \"http://example.org/ bar\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"http://example.org/\\tbar\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"http://example.org/\\nbar\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"http://example.org/\\u{b}bar\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"http://example.org/\\u{c}bar\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"http://example.org/\\rbar\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"foo example.org/\",\n                vec![\n                    TokenType::Alphabetic(\"foo\"),\n                    TokenType::Space,\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                ],\n            ),\n            (\n                \"example.org/ bar\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"example.org/\\tbar\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"example.org/\\nbar\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"example.org/\\u{b}bar\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"example.org/\\u{c}bar\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"example.org/\\rbar\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"http://example.org/<\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('<'),\n                ],\n            ),\n            (\n                \"http://example.org/>\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org/<>\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org/\\0\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\\0'),\n                ],\n            ),\n            (\n                \"http://example.org/\\u{e}\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\\u{e}'),\n                ],\n            ),\n            (\n                \"http://example.org/\\u{7f}\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\\u{7f}'),\n                ],\n            ),\n            (\n                \"http://example.org/\\u{9f}\",\n                vec![TokenType::Url(\"http://example.org/\\u{9f}\")],\n            ),\n            (\n                \"http://example.org/foo|bar\",\n                vec![\n                    TokenType::Url(\"http://example.org/foo\"),\n                    TokenType::Punctuation('|'),\n                    TokenType::Alphabetic(\"bar\"),\n                ],\n            ),\n            (\n                \"example.org/<\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('<'),\n                ],\n            ),\n            (\n                \"example.org/>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org/<>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org/\\0\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('\\0'),\n                ],\n            ),\n            (\n                \"example.org/\\u{e}\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('\\u{e}'),\n                ],\n            ),\n            (\n                \"example.org/\\u{7f}\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('\\u{7f}'),\n                ],\n            ),\n            (\n                \"example.org/\\u{9f}\",\n                vec![TokenType::UrlNoScheme(\"example.org/\\u{9f}\")],\n            ),\n            (\n                \"http://example.org/.\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"http://example.org/..\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"http://example.org/,\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(','),\n                ],\n            ),\n            (\n                \"http://example.org/:\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(':'),\n                ],\n            ),\n            (\n                \"http://example.org/?\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('?'),\n                ],\n            ),\n            (\n                \"http://example.org/!\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('!'),\n                ],\n            ),\n            (\n                \"http://example.org/;\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(';'),\n                ],\n            ),\n            (\n                \"example.org/.\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"example.org/..\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"example.org/,\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(','),\n                ],\n            ),\n            (\n                \"example.org/:\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(':'),\n                ],\n            ),\n            (\n                \"example.org/?\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('?'),\n                ],\n            ),\n            (\n                \"example.org/!\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('!'),\n                ],\n            ),\n            (\n                \"example.org/;\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(';'),\n                ],\n            ),\n            (\n                \"http://example.org/a(b)\",\n                vec![TokenType::Url(\"http://example.org/a(b)\")],\n            ),\n            (\n                \"http://example.org/a[b]\",\n                vec![TokenType::Url(\"http://example.org/a[b]\")],\n            ),\n            (\n                \"http://example.org/a{b}\",\n                vec![TokenType::Url(\"http://example.org/a{b}\")],\n            ),\n            (\n                \"http://example.org/a'b'\",\n                vec![TokenType::Url(\"http://example.org/a'b'\")],\n            ),\n            (\n                \"(http://example.org/)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"[http://example.org/]\",\n                vec![\n                    TokenType::Punctuation('['),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(']'),\n                ],\n            ),\n            (\n                \"{http://example.org/}\",\n                vec![\n                    TokenType::Punctuation('{'),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('}'),\n                ],\n            ),\n            (\n                \"\\\"http://example.org/\\\"\",\n                vec![\n                    TokenType::Punctuation('\"'),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\"'),\n                ],\n            ),\n            (\n                \"'http://example.org/'\",\n                vec![\n                    TokenType::Punctuation('\\''),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"example.org/a(b)\",\n                vec![TokenType::UrlNoScheme(\"example.org/a(b)\")],\n            ),\n            (\n                \"example.org/a[b]\",\n                vec![TokenType::UrlNoScheme(\"example.org/a[b]\")],\n            ),\n            (\n                \"example.org/a{b}\",\n                vec![TokenType::UrlNoScheme(\"example.org/a{b}\")],\n            ),\n            (\n                \"example.org/a'b'\",\n                vec![TokenType::UrlNoScheme(\"example.org/a'b'\")],\n            ),\n            (\n                \"(example.org/)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"[example.org/]\",\n                vec![\n                    TokenType::Punctuation('['),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(']'),\n                ],\n            ),\n            (\n                \"{example.org/}\",\n                vec![\n                    TokenType::Punctuation('{'),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('}'),\n                ],\n            ),\n            (\n                \"\\\"example.org/\\\"\",\n                vec![\n                    TokenType::Punctuation('\"'),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('\"'),\n                ],\n            ),\n            (\n                \"'example.org/'\",\n                vec![\n                    TokenType::Punctuation('\\''),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"((http://example.org/))\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"((http://example.org/a(b)))\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://example.org/a(b)\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"[(http://example.org/)]\",\n                vec![\n                    TokenType::Punctuation('['),\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(']'),\n                ],\n            ),\n            (\n                \"(http://example.org/).\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"(http://example.org/.)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"http://example.org/>\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org/(\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('('),\n                ],\n            ),\n            (\n                \"http://example.org/(.\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"http://example.org/]()\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation(']'),\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"((example.org/))\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"((example.org/a(b)))\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"example.org/a(b)\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"[(example.org/)]\",\n                vec![\n                    TokenType::Punctuation('['),\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(']'),\n                ],\n            ),\n            (\n                \"(example.org/).\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"(example.org/.)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"example.org/>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org/(\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('('),\n                ],\n            ),\n            (\n                \"example.org/(.\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"example.org/]()\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation(']'),\n                    TokenType::Punctuation('('),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"'https://example.org'\",\n                vec![\n                    TokenType::Punctuation('\\''),\n                    TokenType::Url(\"https://example.org\"),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"\\\"https://example.org\\\"\",\n                vec![\n                    TokenType::Punctuation('\"'),\n                    TokenType::Url(\"https://example.org\"),\n                    TokenType::Punctuation('\"'),\n                ],\n            ),\n            (\n                \"''https://example.org''\",\n                vec![\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Url(\"https://example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"'https://example.org''\",\n                vec![\n                    TokenType::Punctuation('\\''),\n                    TokenType::Url(\"https://example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"'https://example.org\",\n                vec![\n                    TokenType::Punctuation('\\''),\n                    TokenType::Url(\"https://example.org\"),\n                ],\n            ),\n            (\n                \"http://example.org/'_(foo)\",\n                vec![TokenType::Url(\"http://example.org/'_(foo)\")],\n            ),\n            (\n                \"http://example.org/'_(foo)'\",\n                vec![TokenType::Url(\"http://example.org/'_(foo)'\")],\n            ),\n            (\n                \"http://example.org/''\",\n                vec![TokenType::Url(\"http://example.org/''\")],\n            ),\n            (\n                \"http://example.org/'''\",\n                vec![\n                    TokenType::Url(\"http://example.org/''\"),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"http://example.org/'.\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"http://example.org/'a\",\n                vec![TokenType::Url(\"http://example.org/'a\")],\n            ),\n            (\n                \"http://example.org/it's\",\n                vec![TokenType::Url(\"http://example.org/it's\")],\n            ),\n            (\n                \"example.org/'_(foo)\",\n                vec![TokenType::UrlNoScheme(\"example.org/'_(foo)\")],\n            ),\n            (\n                \"example.org/'_(foo)'\",\n                vec![TokenType::UrlNoScheme(\"example.org/'_(foo)'\")],\n            ),\n            (\n                \"example.org/''\",\n                vec![TokenType::UrlNoScheme(\"example.org/''\")],\n            ),\n            (\n                \"example.org/'''\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/''\"),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"example.org/'.\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"example.org/'a\",\n                vec![TokenType::UrlNoScheme(\"example.org/'a\")],\n            ),\n            (\n                \"example.org/it's\",\n                vec![TokenType::UrlNoScheme(\"example.org/it's\")],\n            ),\n            (\n                \"http://example.org/\\\"a\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Alphabetic(\"a\"),\n                ],\n            ),\n            (\n                \"http://example.org/\\\"a\\\"\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('\"'),\n                ],\n            ),\n            (\n                \"http://example.org/`a\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('`'),\n                    TokenType::Alphabetic(\"a\"),\n                ],\n            ),\n            (\n                \"http://example.org/`a`\",\n                vec![\n                    TokenType::Url(\"http://example.org/\"),\n                    TokenType::Punctuation('`'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('`'),\n                ],\n            ),\n            (\n                \"https://example.org*\",\n                vec![\n                    TokenType::Url(\"https://example.org\"),\n                    TokenType::Punctuation('*'),\n                ],\n            ),\n            (\n                \"https://example.org/*\",\n                vec![\n                    TokenType::Url(\"https://example.org/\"),\n                    TokenType::Punctuation('*'),\n                ],\n            ),\n            (\n                \"https://example.org/**\",\n                vec![\n                    TokenType::Url(\"https://example.org/\"),\n                    TokenType::Punctuation('*'),\n                    TokenType::Punctuation('*'),\n                ],\n            ),\n            (\n                \"https://example.org/*/a\",\n                vec![TokenType::Url(\"https://example.org/*/a\")],\n            ),\n            (\n                \"example.org/`a\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('`'),\n                    TokenType::Alphabetic(\"a\"),\n                ],\n            ),\n            (\n                \"example.org/`a`\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org/\"),\n                    TokenType::Punctuation('`'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('`'),\n                ],\n            ),\n            (\n                \"http://example.org\\\">\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org'>\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org\\\"/>\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org'/>\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org<p>\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"p\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org</p>\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"p\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org\\\">\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org'>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org\\\"/>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org'/>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org<p>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"p\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org</p>\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"p\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"http://example.org\\\");\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(';'),\n                ],\n            ),\n            (\n                \"http://example.org');\",\n                vec![\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(';'),\n                ],\n            ),\n            (\n                \"<img src=\\\"http://example.org/test.svg\\\">\",\n                vec![\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"img\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"src\"),\n                    TokenType::Punctuation('='),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Url(\"http://example.org/test.svg\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"<div><a href=\\\"http://example.org\\\"></a></div>\",\n                vec![\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"div\"),\n                    TokenType::Punctuation('>'),\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"href\"),\n                    TokenType::Punctuation('='),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation('>'),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('>'),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"div\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"<div><a href=\\\"http://example.org\\\"\\n        ></a></div>\",\n                vec![\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"div\"),\n                    TokenType::Punctuation('>'),\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"href\"),\n                    TokenType::Punctuation('='),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Url(\"http://example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Space,\n                    TokenType::Punctuation('>'),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('>'),\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"div\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"<div>\\n       <img\\n         src=\\\"http://example.org/test3.jpg\\\" />\\n     </div>\",\n                vec![\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"div\"),\n                    TokenType::Punctuation('>'),\n                    TokenType::Space,\n                    TokenType::Punctuation('<'),\n                    TokenType::Alphabetic(\"img\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"src\"),\n                    TokenType::Punctuation('='),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Url(\"http://example.org/test3.jpg\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Space,\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('>'),\n                    TokenType::Space,\n                    TokenType::Punctuation('<'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Alphabetic(\"div\"),\n                    TokenType::Punctuation('>'),\n                ],\n            ),\n            (\n                \"example.org\\\");\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('\"'),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(';'),\n                ],\n            ),\n            (\n                \"example.org');\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.org\"),\n                    TokenType::Punctuation('\\''),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation(';'),\n                ],\n            ),\n            (\n                \"http://example.org/\",\n                vec![TokenType::Url(\"http://example.org/\")],\n            ),\n            (\n                \"http://example.org/a/\",\n                vec![TokenType::Url(\"http://example.org/a/\")],\n            ),\n            (\n                \"http://example.org//\",\n                vec![TokenType::Url(\"http://example.org//\")],\n            ),\n            (\"example.org/\", vec![TokenType::UrlNoScheme(\"example.org/\")]),\n            (\n                \"example.org/a/\",\n                vec![TokenType::UrlNoScheme(\"example.org/a/\")],\n            ),\n            (\n                \"example.org//\",\n                vec![TokenType::UrlNoScheme(\"example.org//\")],\n            ),\n            (\n                \"http://one.org/ http://two.org/\",\n                vec![\n                    TokenType::Url(\"http://one.org/\"),\n                    TokenType::Space,\n                    TokenType::Url(\"http://two.org/\"),\n                ],\n            ),\n            (\n                \"http://one.org/ : http://two.org/\",\n                vec![\n                    TokenType::Url(\"http://one.org/\"),\n                    TokenType::Space,\n                    TokenType::Punctuation(':'),\n                    TokenType::Space,\n                    TokenType::Url(\"http://two.org/\"),\n                ],\n            ),\n            (\n                \"(http://one.org/)(http://two.org/)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://one.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://two.org/\"),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"one.org/ two.org/\",\n                vec![\n                    TokenType::UrlNoScheme(\"one.org/\"),\n                    TokenType::Space,\n                    TokenType::UrlNoScheme(\"two.org/\"),\n                ],\n            ),\n            (\n                \"one.org/ : two.org/\",\n                vec![\n                    TokenType::UrlNoScheme(\"one.org/\"),\n                    TokenType::Space,\n                    TokenType::Punctuation(':'),\n                    TokenType::Space,\n                    TokenType::UrlNoScheme(\"two.org/\"),\n                ],\n            ),\n            (\n                \"(one.org/)(two.org/)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"one.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"two.org/\"),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"http://one.org/ two.org/\",\n                vec![\n                    TokenType::Url(\"http://one.org/\"),\n                    TokenType::Space,\n                    TokenType::UrlNoScheme(\"two.org/\"),\n                ],\n            ),\n            (\n                \"one.org/ : http://two.org/\",\n                vec![\n                    TokenType::UrlNoScheme(\"one.org/\"),\n                    TokenType::Space,\n                    TokenType::Punctuation(':'),\n                    TokenType::Space,\n                    TokenType::Url(\"http://two.org/\"),\n                ],\n            ),\n            (\n                \"(http://one.org/)(two.org/)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Url(\"http://one.org/\"),\n                    TokenType::Punctuation(')'),\n                    TokenType::Punctuation('('),\n                    TokenType::UrlNoScheme(\"two.org/\"),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"http://üñîçøðé.com\",\n                vec![TokenType::Url(\"http://üñîçøðé.com\")],\n            ),\n            (\n                \"http://üñîçøðé.com/ä\",\n                vec![TokenType::Url(\"http://üñîçøðé.com/ä\")],\n            ),\n            (\n                \"http://example.org/¡\",\n                vec![TokenType::Url(\"http://example.org/¡\")],\n            ),\n            (\n                \"http://example.org/¢\",\n                vec![TokenType::Url(\"http://example.org/¢\")],\n            ),\n            (\n                \"http://example.org/😀\",\n                vec![TokenType::Url(\"http://example.org/😀\")],\n            ),\n            (\n                \"http://example.org/¢/\",\n                vec![TokenType::Url(\"http://example.org/¢/\")],\n            ),\n            (\n                \"http://xn--c1h.example.com/\",\n                vec![TokenType::Url(\"http://xn--c1h.example.com/\")],\n            ),\n            (\"üñîçøðé.com\", vec![TokenType::UrlNoScheme(\"üñîçøðé.com\")]),\n            (\n                \"üñîçøðé.com/ä\",\n                vec![TokenType::UrlNoScheme(\"üñîçøðé.com/ä\")],\n            ),\n            (\n                \"example.org/¡\",\n                vec![TokenType::UrlNoScheme(\"example.org/¡\")],\n            ),\n            (\n                \"example.org/¢\",\n                vec![TokenType::UrlNoScheme(\"example.org/¢\")],\n            ),\n            (\n                \"example.org/😀\",\n                vec![TokenType::UrlNoScheme(\"example.org/😀\")],\n            ),\n            (\n                \"example.org/¢/\",\n                vec![TokenType::UrlNoScheme(\"example.org/¢/\")],\n            ),\n            (\n                \"xn--c1h.example.com/\",\n                vec![TokenType::UrlNoScheme(\"xn--c1h.example.com/\")],\n            ),\n            (\n                \"example.\",\n                vec![\n                    TokenType::Alphabetic(\"example\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"example./\",\n                vec![\n                    TokenType::Alphabetic(\"example\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('/'),\n                ],\n            ),\n            (\n                \"foo.com.\",\n                vec![\n                    TokenType::UrlNoScheme(\"foo.com\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"example.c\",\n                vec![\n                    TokenType::Alphabetic(\"example\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"c\"),\n                ],\n            ),\n            (\"example.co\", vec![TokenType::UrlNoScheme(\"example.co\")]),\n            (\"example.com\", vec![TokenType::UrlNoScheme(\"example.com\")]),\n            (\"e.com\", vec![TokenType::UrlNoScheme(\"e.com\")]),\n            (\n                \"exampl.e.c\",\n                vec![\n                    TokenType::Alphabetic(\"exampl\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"e\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"c\"),\n                ],\n            ),\n            (\"exampl.e.co\", vec![TokenType::UrlNoScheme(\"exampl.e.co\")]),\n            (\n                \"e.xample.c\",\n                vec![\n                    TokenType::Alphabetic(\"e\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"xample\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"c\"),\n                ],\n            ),\n            (\"e.xample.co\", vec![TokenType::UrlNoScheme(\"e.xample.co\")]),\n            (\n                \"v1.1.1\",\n                vec![\n                    TokenType::Alphanumeric(\"v1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"1\"),\n                ],\n            ),\n            (\n                \"foo.bar@example.org\",\n                vec![TokenType::Email(\"foo.bar@example.org\")],\n            ),\n            (\n                \"example.com@example.com\",\n                vec![TokenType::Email(\"example.com@example.com\")],\n            ),\n            (\n                \"Look, no scheme: example.org/foo email@foo.com\",\n                vec![\n                    TokenType::Alphabetic(\"Look\"),\n                    TokenType::Punctuation(','),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"no\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"scheme\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Space,\n                    TokenType::UrlNoScheme(\"example.org/foo\"),\n                    TokenType::Space,\n                    TokenType::Email(\"email@foo.com\"),\n                ],\n            ),\n            (\n                \"Web:\\nwww.foobar.co\\nE-Mail:\\n      bar@foobar.co (bla bla bla)\",\n                vec![\n                    TokenType::Alphabetic(\"Web\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Space,\n                    TokenType::UrlNoScheme(\"www.foobar.co\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"E\"),\n                    TokenType::Punctuation('-'),\n                    TokenType::Alphabetic(\"Mail\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Space,\n                    TokenType::Email(\"bar@foobar.co\"),\n                    TokenType::Space,\n                    TokenType::Punctuation('('),\n                    TokenType::Alphabetic(\"bla\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bla\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"bla\"),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"upi://pay?pa=XXXXXXX&pn=XXXXX\",\n                vec![TokenType::UrlNoHost(\"upi://pay?pa=XXXXXXX&pn=XXXXX\")],\n            ),\n            (\n                \"https://example.org?pa=XXXXXXX&pn=XXXXX\",\n                vec![TokenType::Url(\"https://example.org?pa=XXXXXXX&pn=XXXXX\")],\n            ),\n            (\n                \"website https://domain.com\",\n                vec![\n                    TokenType::Alphabetic(\"website\"),\n                    TokenType::Space,\n                    TokenType::Url(\"https://domain.com\"),\n                ],\n            ),\n            (\"a12.b-c.com\", vec![TokenType::UrlNoScheme(\"a12.b-c.com\")]),\n            (\n                \"v1.2.3\",\n                vec![\n                    TokenType::Alphanumeric(\"v1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"2\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"3\"),\n                ],\n            ),\n            (\n                \"https://12-7.0.0.1/\",\n                vec![TokenType::UrlNoHost(\"https://12-7.0.0.1/\")],\n            ),\n            (\n                \"https://user:pass@example.com/\",\n                vec![TokenType::Url(\"https://user:pass@example.com/\")],\n            ),\n            (\n                \"https://user:-.!$@example.com/\",\n                vec![TokenType::Url(\"https://user:-.!$@example.com/\")],\n            ),\n            (\n                \"https://user:!$&'()*+,;=@example.com/\",\n                vec![TokenType::Url(\"https://user:!$&'()*+,;=@example.com/\")],\n            ),\n            (\n                \"https://user:pass@ex@mple.com/\",\n                vec![\n                    TokenType::UrlNoHost(\"https://user:pass@ex\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::UrlNoScheme(\"mple.com/\"),\n                ],\n            ),\n            (\n                \"https://localhost:8080!\",\n                vec![\n                    TokenType::UrlNoHost(\"https://localhost:8080\"),\n                    TokenType::Punctuation('!'),\n                ],\n            ),\n            (\n                \"https://localhost:8080/\",\n                vec![TokenType::UrlNoHost(\"https://localhost:8080/\")],\n            ),\n            (\n                \"https://user:pass@example.com:8080/hi\",\n                vec![TokenType::Url(\"https://user:pass@example.com:8080/hi\")],\n            ),\n            (\n                \"https://127.0.0.1/\",\n                vec![TokenType::Url(\"https://127.0.0.1/\")],\n            ),\n            (\"1.0.0.0\", vec![TokenType::IpAddr(\"1.0.0.0\")]),\n            (\n                \"1.0.0.0/foo/bar\",\n                vec![TokenType::UrlNoScheme(\"1.0.0.0/foo/bar\")],\n            ),\n            (\"1.0 \", vec![TokenType::Float(\"1.0\"), TokenType::Space]),\n            (\n                \"1.0.0\",\n                vec![\n                    TokenType::Integer(\"1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"0\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"0\"),\n                ],\n            ),\n            (\n                \"1.0.0.0.0\",\n                vec![\n                    TokenType::Integer(\"1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::IpAddr(\"0.0.0.0\"),\n                ],\n            ),\n            (\n                \"1.0.0.\",\n                vec![\n                    TokenType::Integer(\"1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"0\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"0\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"https://example.com.:8080/test\",\n                vec![TokenType::Url(\"https://example.com.:8080/test\")],\n            ),\n            (\n                \"https://example.org'\",\n                vec![\n                    TokenType::Url(\"https://example.org\"),\n                    TokenType::Punctuation('\\''),\n                ],\n            ),\n            (\n                \"https://example.org'a@example.com\",\n                vec![TokenType::Url(\"https://example.org'a@example.com\")],\n            ),\n            (\n                \"https://a.com'https://b.com\",\n                vec![\n                    TokenType::UrlNoHost(\"https://a.com'https\"),\n                    TokenType::Punctuation(':'),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('/'),\n                    TokenType::UrlNoScheme(\"b.com\"),\n                ],\n            ),\n            (\n                \"https://example.com...\",\n                vec![\n                    TokenType::Url(\"https://example.com\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"www.example..com\",\n                vec![\n                    TokenType::Alphabetic(\"www\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"example\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"com\"),\n                ],\n            ),\n            (\n                \"https://.www.example.com\",\n                vec![TokenType::Url(\"https://.www.example.com\")],\n            ),\n            (\n                \"-a.com\",\n                vec![TokenType::Punctuation('-'), TokenType::UrlNoScheme(\"a.com\")],\n            ),\n            (\"https://a.-b.com\", vec![TokenType::Url(\"https://a.-b.com\")]),\n            (\n                \"a-.com\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('-'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"com\"),\n                ],\n            ),\n            (\n                \"a.b-.com\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"b\"),\n                    TokenType::Punctuation('-'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"com\"),\n                ],\n            ),\n            (\"https://a.b-.com\", vec![TokenType::Url(\"https://a.b-.com\")]),\n            (\n                \"https://example.com-/\",\n                vec![\n                    TokenType::Url(\"https://example.com\"),\n                    TokenType::Punctuation('-'),\n                    TokenType::Punctuation('/'),\n                ],\n            ),\n            (\n                \"https://example.org-\",\n                vec![\n                    TokenType::Url(\"https://example.org\"),\n                    TokenType::Punctuation('-'),\n                ],\n            ),\n            (\n                \"example.com@about\",\n                vec![\n                    TokenType::UrlNoScheme(\"example.com\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphabetic(\"about\"),\n                ],\n            ),\n            (\n                \"example.com/@about\",\n                vec![TokenType::UrlNoScheme(\"example.com/@about\")],\n            ),\n            (\n                \"https://example.com/@about\",\n                vec![TokenType::Url(\"https://example.com/@about\")],\n            ),\n            (\n                \"info@v1.1.1\",\n                vec![\n                    TokenType::Alphabetic(\"info\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphanumeric(\"v1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"1\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Integer(\"1\"),\n                ],\n            ),\n            (\"file:///\", vec![TokenType::UrlNoHost(\"file:///\")]),\n            (\n                \"file:///home/foo\",\n                vec![TokenType::UrlNoHost(\"file:///home/foo\")],\n            ),\n            (\n                \"file://localhost/home/foo\",\n                vec![TokenType::UrlNoHost(\"file://localhost/home/foo\")],\n            ),\n            (\n                \"facetime://+19995551234\",\n                vec![TokenType::UrlNoHost(\"facetime://+19995551234\")],\n            ),\n            (\n                \"test://123'456!!!\",\n                vec![\n                    TokenType::UrlNoHost(\"test://123'456\"),\n                    TokenType::Punctuation('!'),\n                    TokenType::Punctuation('!'),\n                    TokenType::Punctuation('!'),\n                ],\n            ),\n            (\n                \"test://123'456...\",\n                vec![\n                    TokenType::UrlNoHost(\"test://123'456\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"test://123'456!!!/\",\n                vec![\n                    TokenType::UrlNoHost(\"test://123'456\"),\n                    TokenType::Punctuation('!'),\n                    TokenType::Punctuation('!'),\n                    TokenType::Punctuation('!'),\n                    TokenType::Punctuation('/'),\n                ],\n            ),\n            (\n                \"test://123'456.../\",\n                vec![\n                    TokenType::UrlNoHost(\"test://123'456\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('/'),\n                ],\n            ),\n            (\n                \"1abc://example.com\",\n                vec![TokenType::Url(\"1abc://example.com\")],\n            ),\n            (\n                \"¡¢example.com\",\n                vec![TokenType::UrlNoScheme(\"¡¢example.com\")],\n            ),\n            (\"foo\", vec![TokenType::Alphabetic(\"foo\")]),\n            (\"@\", vec![TokenType::Punctuation('@')]),\n            (\n                \"a@\",\n                vec![TokenType::Alphabetic(\"a\"), TokenType::Punctuation('@')],\n            ),\n            (\n                \"@a\",\n                vec![TokenType::Punctuation('@'), TokenType::Alphabetic(\"a\")],\n            ),\n            (\n                \"@@@\",\n                vec![\n                    TokenType::Punctuation('@'),\n                    TokenType::Punctuation('@'),\n                    TokenType::Punctuation('@'),\n                ],\n            ),\n            (\"foo@example.com\", vec![TokenType::Email(\"foo@example.com\")]),\n            (\n                \"foo.bar@example.com\",\n                vec![TokenType::Email(\"foo.bar@example.com\")],\n            ),\n            (\n                \"#!$%&'*+-/=?^_`{}|~@example.org\",\n                vec![TokenType::Email(\"#!$%&'*+-/=?^_`{}|~@example.org\")],\n            ),\n            (\n                \"foo a@b.com\",\n                vec![\n                    TokenType::Alphabetic(\"foo\"),\n                    TokenType::Space,\n                    TokenType::Email(\"a@b.com\"),\n                ],\n            ),\n            (\n                \"a@b.com foo\",\n                vec![\n                    TokenType::Email(\"a@b.com\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"foo\"),\n                ],\n            ),\n            (\n                \"\\na@b.com\",\n                vec![TokenType::Space, TokenType::Email(\"a@b.com\")],\n            ),\n            (\n                \"a@b.com\\n\",\n                vec![TokenType::Email(\"a@b.com\"), TokenType::Space],\n            ),\n            (\n                \"(a@example.com)\",\n                vec![\n                    TokenType::Punctuation('('),\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation(')'),\n                ],\n            ),\n            (\n                \"\\\"a@example.com\\\"\",\n                vec![\n                    TokenType::Punctuation('\"'),\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation('\"'),\n                ],\n            ),\n            (\n                \"\\\"a@example.com\\\"\",\n                vec![\n                    TokenType::Punctuation('\"'),\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation('\"'),\n                ],\n            ),\n            (\n                \",a@example.com,\",\n                vec![\n                    TokenType::Punctuation(','),\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation(','),\n                ],\n            ),\n            (\n                \":a@example.com:\",\n                vec![\n                    TokenType::Punctuation(':'),\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation(':'),\n                ],\n            ),\n            (\n                \";a@example.com;\",\n                vec![\n                    TokenType::Punctuation(';'),\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation(';'),\n                ],\n            ),\n            (\n                \".@example.com\",\n                vec![\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('@'),\n                    TokenType::UrlNoScheme(\"example.com\"),\n                ],\n            ),\n            (\n                \"foo.@example.com\",\n                vec![\n                    TokenType::Alphabetic(\"foo\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('@'),\n                    TokenType::UrlNoScheme(\"example.com\"),\n                ],\n            ),\n            (\n                \".foo@example.com\",\n                vec![\n                    TokenType::Punctuation('.'),\n                    TokenType::Email(\"foo@example.com\"),\n                ],\n            ),\n            (\n                \".foo@example.com\",\n                vec![\n                    TokenType::Punctuation('.'),\n                    TokenType::Email(\"foo@example.com\"),\n                ],\n            ),\n            (\n                \"a..b@example.com\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Punctuation('.'),\n                    TokenType::Email(\"b@example.com\"),\n                ],\n            ),\n            (\n                \"a@example.com.\",\n                vec![\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"a@b\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphabetic(\"b\"),\n                ],\n            ),\n            (\n                \"a@b.\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphabetic(\"b\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"a@b.com.\",\n                vec![TokenType::Email(\"a@b.com\"), TokenType::Punctuation('.')],\n            ),\n            (\n                \"a@example.com-\",\n                vec![\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Punctuation('-'),\n                ],\n            ),\n            (\"a@foo-bar.com\", vec![TokenType::Email(\"a@foo-bar.com\")]),\n            (\n                \"a@-foo.com\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Punctuation('-'),\n                    TokenType::UrlNoScheme(\"foo.com\"),\n                ],\n            ),\n            (\n                \"a@b-.\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphabetic(\"b\"),\n                    TokenType::Punctuation('-'),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"a@b\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphabetic(\"b\"),\n                ],\n            ),\n            (\n                \"a@b.\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphabetic(\"b\"),\n                    TokenType::Punctuation('.'),\n                ],\n            ),\n            (\n                \"a@example.com b@example.com\",\n                vec![\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Space,\n                    TokenType::Email(\"b@example.com\"),\n                ],\n            ),\n            (\n                \"a@example.com @ b@example.com\",\n                vec![\n                    TokenType::Email(\"a@example.com\"),\n                    TokenType::Space,\n                    TokenType::Punctuation('@'),\n                    TokenType::Space,\n                    TokenType::Email(\"b@example.com\"),\n                ],\n            ),\n            (\n                \"a@xy.com;b@xy.com,c@xy.com\",\n                vec![\n                    TokenType::Email(\"a@xy.com\"),\n                    TokenType::Punctuation(';'),\n                    TokenType::Email(\"b@xy.com\"),\n                    TokenType::Punctuation(','),\n                    TokenType::Email(\"c@xy.com\"),\n                ],\n            ),\n            (\n                \"üñîçøðé@example.com\",\n                vec![TokenType::Email(\"üñîçøðé@example.com\")],\n            ),\n            (\n                \"üñîçøðé@üñîçøðé.com\",\n                vec![TokenType::Email(\"üñîçøðé@üñîçøðé.com\")],\n            ),\n            (\"www@example.com\", vec![TokenType::Email(\"www@example.com\")]),\n            (\n                \"a@a.xyϸ\",\n                vec![\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('@'),\n                    TokenType::Alphabetic(\"a\"),\n                    TokenType::Punctuation('.'),\n                    TokenType::Alphabetic(\"xyϸ\"),\n                ],\n            ),\n            (\n                \"100 -100 100.00 -100.00 $100 $100.00\",\n                vec![\n                    TokenType::Integer(\"100\"),\n                    TokenType::Space,\n                    TokenType::Integer(\"-100\"),\n                    TokenType::Space,\n                    TokenType::Float(\"100.00\"),\n                    TokenType::Space,\n                    TokenType::Float(\"-100.00\"),\n                    TokenType::Space,\n                    TokenType::Punctuation('$'),\n                    TokenType::Integer(\"100\"),\n                    TokenType::Space,\n                    TokenType::Punctuation('$'),\n                    TokenType::Float(\"100.00\"),\n                ],\n            ),\n            (\n                \" - 100 100 . 00\",\n                vec![\n                    TokenType::Space,\n                    TokenType::Punctuation('-'),\n                    TokenType::Space,\n                    TokenType::Integer(\"100\"),\n                    TokenType::Space,\n                    TokenType::Integer(\"100\"),\n                    TokenType::Space,\n                    TokenType::Punctuation('.'),\n                    TokenType::Space,\n                    TokenType::Integer(\"00\"),\n                ],\n            ),\n            (\n                \"send $100.00 to user@domain.com or visit domain.com/pay-me!\",\n                vec![\n                    TokenType::Alphabetic(\"send\"),\n                    TokenType::Space,\n                    TokenType::Punctuation('$'),\n                    TokenType::Float(\"100.00\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"to\"),\n                    TokenType::Space,\n                    TokenType::Email(\"user@domain.com\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"or\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"visit\"),\n                    TokenType::Space,\n                    TokenType::UrlNoScheme(\"domain.com/pay-me\"),\n                    TokenType::Punctuation('!'),\n                ],\n            ),\n            (\n                \"vＥⓡ𝔂 𝔽𝕌Ňℕｙ ţ乇𝕏𝓣 wWiIiIIttHh l133t5p3/-\\\\|<\",\n                vec![\n                    TokenType::Alphabetic(\"vＥⓡ𝔂\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"𝔽𝕌Ňℕｙ\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"ţ乇𝕏𝓣\"),\n                    TokenType::Space,\n                    TokenType::Alphabetic(\"wWiIiIIttHh\"),\n                    TokenType::Space,\n                    TokenType::Alphanumeric(\"l133t5p3\"),\n                    TokenType::Punctuation('/'),\n                    TokenType::Punctuation('-'),\n                    TokenType::Punctuation('\\\\'),\n                    TokenType::Punctuation('|'),\n                    TokenType::Punctuation('<'),\n                ],\n            ),\n        ] {\n            let result = TypesTokenizer::new(text)\n                .map(|t| t.word)\n                .collect::<Vec<_>>();\n\n            assert_eq!(result, expected, \"text: {:?}\", text);\n\n            /*print!(\"({text:?}, \");\n            print!(\"vec![\");\n            for (pos, item) in result.into_iter().enumerate() {\n                if pos > 0 {\n                    print!(\", \");\n                }\n                print!(\"TokenType::{:?}\", item);\n            }\n            println!(\"]),\");*/\n        }\n    }\n}\n"
  },
  {
    "path": "crates/nlp/src/tokenizers/word.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, str::CharIndices};\n\nuse super::Token;\n\npub struct WordTokenizer<'x> {\n    max_token_length: usize,\n    text: &'x str,\n    iterator: CharIndices<'x>,\n}\n\nimpl WordTokenizer<'_> {\n    pub fn new(text: &'_ str, max_token_length: usize) -> WordTokenizer<'_> {\n        WordTokenizer {\n            max_token_length,\n            text,\n            iterator: text.char_indices(),\n        }\n    }\n}\n\n/// Parses indo-european text into lowercase tokens.\nimpl<'x> Iterator for WordTokenizer<'x> {\n    type Item = Token<Cow<'x, str>>;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        while let Some((token_start, ch)) = self.iterator.next() {\n            if ch.is_alphanumeric() {\n                let mut is_uppercase = ch.is_uppercase();\n                let token_end = (&mut self.iterator)\n                    .filter_map(|(pos, ch)| {\n                        if ch.is_alphanumeric() {\n                            if !is_uppercase && ch.is_uppercase() {\n                                is_uppercase = true;\n                            }\n                            None\n                        } else {\n                            pos.into()\n                        }\n                    })\n                    .next()\n                    .unwrap_or(self.text.len());\n\n                let token_len = token_end - token_start;\n                if token_end > token_start && token_len <= self.max_token_length {\n                    return Token::new(\n                        token_start,\n                        token_len,\n                        if is_uppercase {\n                            self.text[token_start..token_end].to_lowercase().into()\n                        } else {\n                            self.text[token_start..token_end].into()\n                        },\n                    )\n                    .into();\n                }\n            }\n        }\n        None\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn indo_european_tokenizer() {\n        let inputs = [\n            (\n                \"The quick brown fox jumps over the lazy dog\",\n                vec![\n                    Token::new(0, 3, \"the\".into()),\n                    Token::new(4, 5, \"quick\".into()),\n                    Token::new(10, 5, \"brown\".into()),\n                    Token::new(16, 3, \"fox\".into()),\n                    Token::new(20, 5, \"jumps\".into()),\n                    Token::new(26, 4, \"over\".into()),\n                    Token::new(31, 3, \"the\".into()),\n                    Token::new(35, 4, \"lazy\".into()),\n                    Token::new(40, 3, \"dog\".into()),\n                ],\n            ),\n            (\n                \"Jovencillo EMPONZOÑADO de whisky: ¡qué figurota exhibe!\",\n                vec![\n                    Token::new(0, 10, \"jovencillo\".into()),\n                    Token::new(11, 12, \"emponzoñado\".into()),\n                    Token::new(24, 2, \"de\".into()),\n                    Token::new(27, 6, \"whisky\".into()),\n                    Token::new(37, 4, \"qué\".into()),\n                    Token::new(42, 8, \"figurota\".into()),\n                    Token::new(51, 6, \"exhibe\".into()),\n                ],\n            ),\n            (\n                \"ZWÖLF Boxkämpfer jagten Victor quer über den großen Sylter Deich\",\n                vec![\n                    Token::new(0, 6, \"zwölf\".into()),\n                    Token::new(7, 11, \"boxkämpfer\".into()),\n                    Token::new(19, 6, \"jagten\".into()),\n                    Token::new(26, 6, \"victor\".into()),\n                    Token::new(33, 4, \"quer\".into()),\n                    Token::new(38, 5, \"über\".into()),\n                    Token::new(44, 3, \"den\".into()),\n                    Token::new(48, 7, \"großen\".into()),\n                    Token::new(56, 6, \"sylter\".into()),\n                    Token::new(63, 5, \"deich\".into()),\n                ],\n            ),\n            (\n                \"Съешь ещё этих мягких французских булок, да выпей же чаю\",\n                vec![\n                    Token::new(0, 10, \"съешь\".into()),\n                    Token::new(11, 6, \"ещё\".into()),\n                    Token::new(18, 8, \"этих\".into()),\n                    Token::new(27, 12, \"мягких\".into()),\n                    Token::new(40, 22, \"французских\".into()),\n                    Token::new(63, 10, \"булок\".into()),\n                    Token::new(75, 4, \"да\".into()),\n                    Token::new(80, 10, \"выпей\".into()),\n                    Token::new(91, 4, \"же\".into()),\n                    Token::new(96, 6, \"чаю\".into()),\n                ],\n            ),\n            (\n                \"Pijamalı hasta yağız şoföre çabucak güvendi\",\n                vec![\n                    Token::new(0, 9, \"pijamalı\".into()),\n                    Token::new(10, 5, \"hasta\".into()),\n                    Token::new(16, 7, \"yağız\".into()),\n                    Token::new(24, 8, \"şoföre\".into()),\n                    Token::new(33, 8, \"çabucak\".into()),\n                    Token::new(42, 8, \"güvendi\".into()),\n                ],\n            ),\n        ];\n\n        for (input, tokens) in inputs.iter() {\n            for (pos, token) in WordTokenizer::new(input, 40).enumerate() {\n                assert_eq!(token, tokens[pos]);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/pop3/Cargo.toml",
    "content": "[package]\nname = \"pop3\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nstore = { path = \"../store\" }\ncommon = { path = \"../common\" }\ndirectory = { path = \"../directory\" }\nimap = { path = \"../imap\" }\nutils = { path = \"../utils\" }\ntrc = { path = \"../trc\" }\ntypes = { path = \"../types\" }\nemail = { path = \"../email\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\ntokio = { version = \"1.47\", features = [\"full\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\n\n[features]\ntest_mode = []\n"
  },
  {
    "path": "crates/pop3/src/client.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    KV_RATE_LIMIT_IMAP,\n    listener::{SessionResult, SessionStream},\n};\nuse mail_send::Credentials;\nuse trc::{AddContext, SecurityEvent};\n\nuse crate::{\n    Session, State,\n    protocol::{Command, Mechanism, request::Error},\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn ingest(&mut self, bytes: &[u8]) -> SessionResult {\n        trc::event!(\n            Pop3(trc::Pop3Event::RawInput),\n            SpanId = self.session_id,\n            Size = bytes.len(),\n            Contents = trc::Value::from_maybe_string(bytes),\n        );\n\n        let mut bytes = bytes.iter();\n        let mut requests = Vec::with_capacity(2);\n\n        loop {\n            match self.receiver.parse(&mut bytes) {\n                Ok(request) => {\n                    // Group delete requests when possible\n                    match (request, requests.last_mut()) {\n                        (Command::Dele { msg }, Some(Ok(Command::DeleMany { msgs }))) => {\n                            msgs.push(msg);\n                        }\n                        (Command::Dele { msg }, Some(Ok(Command::Dele { msg: other_msg }))) => {\n                            let request = Ok(Command::DeleMany {\n                                msgs: vec![*other_msg, msg],\n                            });\n                            requests.pop();\n                            requests.push(request);\n                        }\n                        (request, _) => {\n                            requests.push(Ok(request));\n                        }\n                    }\n                }\n                Err(Error::NeedsMoreData) => {\n                    break;\n                }\n                Err(Error::Parse(err)) => {\n                    // Check for port scanners\n                    if matches!(&self.state, State::NotAuthenticated { .. },) {\n                        match self.server.is_scanner_fail2banned(self.remote_addr).await {\n                            Ok(true) => {\n                                trc::event!(\n                                    Security(SecurityEvent::ScanBan),\n                                    SpanId = self.session_id,\n                                    RemoteIp = self.remote_addr,\n                                    Reason = \"Invalid POP3 command\",\n                                );\n\n                                return SessionResult::Close;\n                            }\n                            Ok(false) => {}\n                            Err(err) => {\n                                trc::error!(\n                                    err.span_id(self.session_id)\n                                        .details(\"Failed to check for fail2ban\")\n                                );\n                            }\n                        }\n                    }\n                    requests.push(Err(trc::Pop3Event::Error.into_err().details(err)));\n                }\n            }\n        }\n\n        for request in requests {\n            let result = match request {\n                Ok(command) => match self.validate_request(command).await {\n                    Ok(command) => match command {\n                        Command::User { name } => {\n                            if let State::NotAuthenticated { username, .. } = &mut self.state {\n                                let response = format!(\"{name} is a valid mailbox\");\n                                *username = Some(name);\n                                self.write_ok(response)\n                                    .await\n                                    .map(|_| SessionResult::Continue)\n                            } else {\n                                unreachable!();\n                            }\n                        }\n                        Command::Pass { string } => {\n                            let username =\n                                if let State::NotAuthenticated { username, .. } = &mut self.state {\n                                    username.take().unwrap()\n                                } else {\n                                    unreachable!()\n                                };\n                            self.handle_auth(Credentials::Plain {\n                                username,\n                                secret: string,\n                            })\n                            .await\n                            .map(|_| SessionResult::Continue)\n                        }\n                        Command::Quit => self.handle_quit().await.map(|_| SessionResult::Close),\n                        Command::Stat => self.handle_stat().await.map(|_| SessionResult::Continue),\n                        Command::List { msg } => {\n                            self.handle_list(msg).await.map(|_| SessionResult::Continue)\n                        }\n                        Command::Retr { msg } => self\n                            .handle_fetch(msg, None)\n                            .await\n                            .map(|_| SessionResult::Continue),\n                        Command::Dele { msg } => self\n                            .handle_dele(vec![msg])\n                            .await\n                            .map(|_| SessionResult::Continue),\n                        Command::DeleMany { msgs } => self\n                            .handle_dele(msgs)\n                            .await\n                            .map(|_| SessionResult::Continue),\n                        Command::Top { msg, n } => self\n                            .handle_fetch(msg, n.into())\n                            .await\n                            .map(|_| SessionResult::Continue),\n                        Command::Uidl { msg } => {\n                            self.handle_uidl(msg).await.map(|_| SessionResult::Continue)\n                        }\n                        Command::Noop => {\n                            trc::event!(\n                                Pop3(trc::Pop3Event::Noop),\n                                SpanId = self.session_id,\n                                Elapsed = trc::Value::Duration(0)\n                            );\n\n                            self.write_ok(\"NOOP\").await.map(|_| SessionResult::Continue)\n                        }\n                        Command::Rset => self.handle_rset().await.map(|_| SessionResult::Continue),\n                        Command::Capa => self.handle_capa().await.map(|_| SessionResult::Continue),\n                        Command::Stls => {\n                            self.handle_stls().await.map(|_| SessionResult::UpgradeTls)\n                        }\n                        Command::Utf8 => self.handle_utf8().await.map(|_| SessionResult::Continue),\n                        Command::Auth { mechanism, params } => self\n                            .handle_sasl(mechanism, params)\n                            .await\n                            .map(|_| SessionResult::Continue),\n                        Command::Apop { .. } => Err(trc::Pop3Event::Error\n                            .into_err()\n                            .details(\"APOP not supported.\")),\n                    },\n                    Err(err) => Err(err),\n                },\n                Err(err) => Err(err),\n            };\n\n            match result {\n                Ok(SessionResult::Continue) => (),\n                Ok(result) => return result,\n                Err(err) => {\n                    if !self.write_err(err).await {\n                        return SessionResult::Close;\n                    }\n                }\n            }\n        }\n\n        SessionResult::Continue\n    }\n\n    async fn validate_request(\n        &self,\n        command: Command<String, Mechanism>,\n    ) -> trc::Result<Command<String, Mechanism>> {\n        match &command {\n            Command::Capa | Command::Quit | Command::Noop => Ok(command),\n            Command::Auth {\n                mechanism: Mechanism::Plain,\n                ..\n            }\n            | Command::User { .. }\n            | Command::Pass { .. }\n            | Command::Apop { .. } => {\n                if let State::NotAuthenticated { username, .. } = &self.state {\n                    if self.stream.is_tls() || self.server.core.imap.allow_plain_auth {\n                        if !matches!(command, Command::Pass { .. }) || username.is_some() {\n                            Ok(command)\n                        } else {\n                            Err(trc::Pop3Event::Error\n                                .into_err()\n                                .details(\"Username was not provided.\"))\n                        }\n                    } else {\n                        Err(trc::Pop3Event::Error\n                            .into_err()\n                            .details(\"Cannot authenticate over plain-text.\"))\n                    }\n                } else {\n                    Err(trc::Pop3Event::Error\n                        .into_err()\n                        .details(\"Already authenticated.\"))\n                }\n            }\n            Command::Auth { .. } => {\n                if let State::NotAuthenticated { .. } = &self.state {\n                    Ok(command)\n                } else {\n                    Err(trc::Pop3Event::Error\n                        .into_err()\n                        .details(\"Already authenticated.\"))\n                }\n            }\n            Command::Stls => {\n                if !self.stream.is_tls() {\n                    Ok(command)\n                } else {\n                    Err(trc::Pop3Event::Error\n                        .into_err()\n                        .details(\"Already in TLS mode.\"))\n                }\n            }\n\n            Command::List { .. }\n            | Command::Retr { .. }\n            | Command::Dele { .. }\n            | Command::DeleMany { .. }\n            | Command::Top { .. }\n            | Command::Uidl { .. }\n            | Command::Utf8\n            | Command::Stat\n            | Command::Rset => {\n                if let State::Authenticated { mailbox, .. } = &self.state {\n                    if let Some(rate) = &self.server.core.imap.rate_requests {\n                        if self\n                            .server\n                            .core\n                            .storage\n                            .lookup\n                            .is_rate_allowed(\n                                KV_RATE_LIMIT_IMAP,\n                                &mailbox.account_id.to_be_bytes(),\n                                rate,\n                                true,\n                            )\n                            .await\n                            .caused_by(trc::location!())?\n                            .is_none()\n                        {\n                            Ok(command)\n                        } else {\n                            Err(trc::LimitEvent::TooManyRequests.into_err())\n                        }\n                    } else {\n                        Ok(command)\n                    }\n                } else {\n                    Err(trc::Pop3Event::Error\n                        .into_err()\n                        .details(\"Not authenticated.\"))\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{net::IpAddr, sync::Arc};\n\nuse common::{\n    Inner, Server,\n    auth::AccessToken,\n    listener::{ServerInstance, SessionStream, limiter::InFlight},\n};\nuse mailbox::Mailbox;\nuse protocol::request::Parser;\n\npub mod client;\npub mod mailbox;\npub mod op;\npub mod protocol;\npub mod session;\n\nstatic SERVER_GREETING: &str = \"+OK Stalwart POP3 at your service.\\r\\n\";\n\n#[derive(Clone)]\npub struct Pop3SessionManager {\n    pub inner: Arc<Inner>,\n}\n\nimpl Pop3SessionManager {\n    pub fn new(inner: Arc<Inner>) -> Self {\n        Self { inner }\n    }\n}\n\npub struct Session<T: SessionStream> {\n    pub server: Server,\n    pub instance: Arc<ServerInstance>,\n    pub receiver: Parser,\n    pub state: State,\n    pub stream: T,\n    pub in_flight: InFlight,\n    pub remote_addr: IpAddr,\n    pub session_id: u64,\n}\n\npub enum State {\n    NotAuthenticated {\n        auth_failures: u32,\n        username: Option<String>,\n    },\n    Authenticated {\n        mailbox: Mailbox,\n        in_flight: Option<InFlight>,\n        access_token: Arc<AccessToken>,\n    },\n}\n\nimpl State {\n    pub fn mailbox(&self) -> &Mailbox {\n        match self {\n            State::Authenticated { mailbox, .. } => mailbox,\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn mailbox_mut(&mut self) -> &mut Mailbox {\n        match self {\n            State::Authenticated { mailbox, .. } => mailbox,\n            _ => unreachable!(),\n        }\n    }\n\n    pub fn access_token(&self) -> &Arc<AccessToken> {\n        match self {\n            State::Authenticated { access_token, .. } => access_token,\n            _ => unreachable!(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/mailbox.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::Session;\nuse common::listener::SessionStream;\nuse email::{\n    cache::{MessageCacheFetch, mailbox::MailboxCacheAccess},\n    mailbox::INBOX_ID,\n};\nuse std::collections::BTreeMap;\nuse trc::AddContext;\nuse types::special_use::SpecialUse;\n\n#[derive(Default)]\npub struct Mailbox {\n    pub messages: Vec<Message>,\n    pub account_id: u32,\n    pub uid_validity: u32,\n    pub total: u32,\n    pub size: u32,\n}\n\npub struct Message {\n    pub id: u32,\n    pub uid: u32,\n    pub size: u32,\n    pub deleted: bool,\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn fetch_mailbox(&self, account_id: u32) -> trc::Result<Mailbox> {\n        // Obtain UID validity\n        let cache = self\n            .server\n            .get_cached_messages(account_id)\n            .await\n            .caused_by(trc::location!())?;\n\n        if cache.emails.items.is_empty() {\n            return Ok(Mailbox::default());\n        }\n\n        let uid_validity = cache\n            .mailbox_by_role(&SpecialUse::Inbox)\n            .map(|x| x.uid_validity)\n            .unwrap_or_default();\n\n        // Sort by UID\n        let message_map = cache\n            .emails\n            .items\n            .iter()\n            .filter_map(|message| {\n                message\n                    .mailboxes\n                    .iter()\n                    .find(|m| m.mailbox_id == INBOX_ID)\n                    .map(|m| (m.uid, (message.document_id, message.size)))\n            })\n            .collect::<BTreeMap<u32, (u32, u32)>>();\n\n        // Create mailbox\n        let mut mailbox = Mailbox {\n            messages: Vec::with_capacity(message_map.len()),\n            uid_validity,\n            account_id,\n            ..Default::default()\n        };\n        for (uid, (id, size)) in message_map {\n            mailbox.messages.push(Message {\n                id,\n                uid,\n                size,\n                deleted: false,\n            });\n            mailbox.total += 1;\n            mailbox.size += size;\n        }\n\n        Ok(mailbox)\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/op/authenticate.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    auth::{\n        AuthRequest,\n        sasl::{sasl_decode_challenge_oauth, sasl_decode_challenge_plain},\n    },\n    listener::{SessionStream, limiter::LimiterResult},\n};\nuse directory::Permission;\nuse mail_parser::decoders::base64::base64_decode;\nuse mail_send::Credentials;\n\nuse crate::{\n    Session, State,\n    protocol::{Command, Mechanism, request},\n};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_sasl(\n        &mut self,\n        mechanism: Mechanism,\n        mut params: Vec<String>,\n    ) -> trc::Result<()> {\n        match mechanism {\n            Mechanism::Plain | Mechanism::OAuthBearer | Mechanism::XOauth2 => {\n                if !params.is_empty() {\n                    let credentials = base64_decode(params.pop().unwrap().as_bytes())\n                        .and_then(|challenge| {\n                            if mechanism == Mechanism::Plain {\n                                sasl_decode_challenge_plain(&challenge)\n                            } else {\n                                sasl_decode_challenge_oauth(&challenge)\n                            }\n                        })\n                        .ok_or_else(|| {\n                            trc::AuthEvent::Error\n                                .into_err()\n                                .details(\"Invalid SASL challenge\")\n                        })?;\n\n                    self.handle_auth(credentials).await\n                } else {\n                    // TODO: This hack is temporary until the SASL library is developed\n                    self.receiver.state = request::State::Argument {\n                        request: Command::Auth {\n                            mechanism: mechanism.as_str().as_bytes().to_vec(),\n                            params: vec![],\n                        },\n                        num: 1,\n                        last_is_space: true,\n                    };\n\n                    self.write_bytes(\"+\\r\\n\").await\n                }\n            }\n            _ => Err(trc::AuthEvent::Error\n                .into_err()\n                .details(\"Authentication mechanism not supported.\")),\n        }\n    }\n\n    pub async fn handle_auth(&mut self, credentials: Credentials<String>) -> trc::Result<()> {\n        // Authenticate\n        let access_token = self\n            .server\n            .authenticate(&AuthRequest::from_credentials(\n                credentials,\n                self.session_id,\n                self.remote_addr,\n            ))\n            .await\n            .map_err(|err| {\n                if err.matches(trc::EventType::Auth(trc::AuthEvent::Failed)) {\n                    match &self.state {\n                        State::NotAuthenticated {\n                            auth_failures,\n                            username,\n                        } if *auth_failures < self.server.core.imap.max_auth_failures => {\n                            self.state = State::NotAuthenticated {\n                                auth_failures: auth_failures + 1,\n                                username: username.clone(),\n                            };\n                        }\n                        _ => {\n                            return trc::AuthEvent::TooManyAttempts.into_err().caused_by(err);\n                        }\n                    }\n                }\n\n                err\n            })\n            .and_then(|token| {\n                token\n                    .assert_has_permission(Permission::Pop3Authenticate)\n                    .map(|_| token)\n            })?;\n\n        // Enforce concurrency limits\n        let in_flight = match access_token.is_imap_request_allowed() {\n            LimiterResult::Allowed(in_flight) => Some(in_flight),\n            LimiterResult::Forbidden => {\n                return Err(trc::LimitEvent::ConcurrentRequest.into_err());\n            }\n            LimiterResult::Disabled => None,\n        };\n\n        // Fetch mailbox\n        let mailbox = self.fetch_mailbox(access_token.primary_id()).await?;\n\n        // Create session\n        self.state = State::Authenticated {\n            in_flight,\n            mailbox,\n            access_token,\n        };\n        self.write_ok(\"Authentication successful\").await\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/op/delete.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::message::delete::EmailDeletion;\nuse store::{roaring::RoaringBitmap, write::BatchBuilder};\nuse trc::AddContext;\n\nuse crate::{Session, State, protocol::response::Response};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_dele(&mut self, msgs: Vec<u32>) -> trc::Result<()> {\n        // Validate access\n        self.state\n            .access_token()\n            .assert_has_permission(Permission::Pop3Dele)?;\n\n        let op_start = Instant::now();\n        let mailbox = self.state.mailbox_mut();\n        let mut response = Vec::new();\n\n        for msg in &msgs {\n            if let Some(message) = mailbox.messages.get_mut(msg.saturating_sub(1) as usize) {\n                if !message.deleted {\n                    response.extend_from_slice(format!(\"+OK message {msg} deleted\\r\\n\").as_bytes());\n                    message.deleted = true;\n                } else {\n                    response.extend_from_slice(\n                        format!(\"-ERR message {msg} already deleted\\r\\n\").as_bytes(),\n                    );\n                }\n            } else {\n                response.extend_from_slice(\"-ERR no such message\\r\\n\".as_bytes());\n            }\n        }\n\n        trc::event!(\n            Pop3(trc::Pop3Event::Delete),\n            SpanId = self.session_id,\n            Total = msgs.len(),\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_bytes(response).await\n    }\n\n    pub async fn handle_rset(&mut self) -> trc::Result<()> {\n        let op_start = Instant::now();\n        let mut count = 0;\n        let mailbox = self.state.mailbox_mut();\n        for message in &mut mailbox.messages {\n            if message.deleted {\n                count += 1;\n                message.deleted = false;\n            }\n        }\n\n        trc::event!(\n            Pop3(trc::Pop3Event::Reset),\n            SpanId = self.session_id,\n            Total = count as u64,\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_ok(format!(\"{count} messages undeleted\")).await\n    }\n\n    pub async fn handle_quit(&mut self) -> trc::Result<()> {\n        let op_start = Instant::now();\n        let mut deleted_docs = Vec::new();\n\n        if let State::Authenticated { mailbox, .. } = &self.state {\n            let mut deleted = RoaringBitmap::new();\n            for message in &mailbox.messages {\n                if message.deleted {\n                    deleted.insert(message.id);\n                    deleted_docs.push(trc::Value::from(message.id));\n                }\n            }\n\n            if !deleted.is_empty() {\n                let num_deleted = deleted.len();\n                let mut batch = BatchBuilder::new();\n                let not_deleted = self\n                    .server\n                    .emails_delete(\n                        mailbox.account_id,\n                        self.state.access_token().tenant_id(),\n                        &mut batch,\n                        deleted,\n                    )\n                    .await\n                    .caused_by(trc::location!())?;\n\n                if !batch.is_empty() {\n                    self.server\n                        .commit_batch(batch)\n                        .await\n                        .caused_by(trc::location!())?;\n                    self.server.notify_task_queue();\n                }\n                if not_deleted.is_empty() {\n                    self.write_ok(format!(\n                        \"Stalwart POP3 bids you farewell ({num_deleted} messages deleted).\"\n                    ))\n                    .await?;\n                } else {\n                    self.write_bytes(\n                        Response::Err::<u32>(\"Some messages could not be deleted\".into())\n                            .serialize(),\n                    )\n                    .await?;\n                }\n            } else {\n                self.write_ok(\"Stalwart POP3 bids you farewell (no messages deleted).\")\n                    .await?;\n            }\n        } else {\n            self.write_ok(\"Stalwart POP3 bids you farewell.\").await?;\n        }\n\n        trc::event!(\n            Pop3(trc::Pop3Event::Quit),\n            SpanId = self.session_id,\n            DocumentId = deleted_docs,\n            Elapsed = op_start.elapsed()\n        );\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/op/fetch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{Session, protocol::response::Response};\nuse common::listener::SessionStream;\nuse directory::Permission;\nuse email::message::metadata::MessageMetadata;\nuse std::time::Instant;\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive},\n};\nuse trc::AddContext;\nuse types::{collection::Collection, field::EmailField};\nuse utils::chained_bytes::ChainedBytes;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_fetch(&mut self, msg: u32, lines: Option<u32>) -> trc::Result<()> {\n        // Validate access\n        self.state\n            .access_token()\n            .assert_has_permission(Permission::Pop3Retr)?;\n\n        let op_start = Instant::now();\n        let mailbox = self.state.mailbox();\n        if let Some(message) = mailbox.messages.get(msg.saturating_sub(1) as usize) {\n            if let Some(metadata_) = self\n                .server\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                    mailbox.account_id,\n                    Collection::Email,\n                    message.id,\n                    EmailField::Metadata,\n                ))\n                .await\n                .caused_by(trc::location!())?\n            {\n                let metadata = metadata_\n                    .unarchive::<MessageMetadata>()\n                    .caused_by(trc::location!())?;\n                if let Some(bytes) = self\n                    .server\n                    .blob_store()\n                    .get_blob(metadata.blob_hash.0.as_slice(), 0..usize::MAX)\n                    .await\n                    .caused_by(trc::location!())?\n                {\n                    trc::event!(\n                        Pop3(trc::Pop3Event::Fetch),\n                        SpanId = self.session_id,\n                        DocumentId = message.id,\n                        Elapsed = op_start.elapsed()\n                    );\n\n                    let bytes = ChainedBytes::new(metadata.raw_headers.as_ref())\n                        .with_last(\n                            bytes\n                                .get(metadata.blob_body_offset.to_native() as usize..)\n                                .unwrap_or_default(),\n                        )\n                        .get_full_range();\n\n                    self.write_bytes(\n                        Response::Message::<u32> {\n                            bytes,\n                            lines: lines.unwrap_or(0),\n                        }\n                        .serialize(),\n                    )\n                    .await\n                } else {\n                    Err(trc::Pop3Event::Error\n                        .into_err()\n                        .details(\"Failed to fetch message. Perhaps another session deleted it?\")\n                        .caused_by(trc::location!()))\n                }\n            } else {\n                Err(trc::Pop3Event::Error\n                    .into_err()\n                    .details(\"Failed to fetch message. Perhaps another session deleted it?\")\n                    .caused_by(trc::location!()))\n            }\n        } else {\n            Err(trc::Pop3Event::Error.into_err().details(\"No such message.\"))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/op/list.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse common::listener::SessionStream;\nuse directory::Permission;\n\nuse crate::{Session, protocol::response::Response};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_list(&mut self, msg: Option<u32>) -> trc::Result<()> {\n        // Validate access\n        self.state\n            .access_token()\n            .assert_has_permission(Permission::Pop3List)?;\n\n        let op_start = Instant::now();\n        let mailbox = self.state.mailbox();\n        if let Some(msg) = msg {\n            if let Some(message) = mailbox.messages.get(msg.saturating_sub(1) as usize) {\n                trc::event!(\n                    Pop3(trc::Pop3Event::ListMessage),\n                    SpanId = self.session_id,\n                    DocumentId = message.id,\n                    Size = message.size,\n                    Elapsed = op_start.elapsed()\n                );\n\n                self.write_ok(format!(\"{} {}\", msg, message.size)).await\n            } else {\n                Err(trc::Pop3Event::Error\n                    .into_err()\n                    .details(\"No such message.\")\n                    .caused_by(trc::location!()))\n            }\n        } else {\n            trc::event!(\n                Pop3(trc::Pop3Event::List),\n                SpanId = self.session_id,\n                Total = mailbox.messages.len(),\n                Elapsed = op_start.elapsed()\n            );\n\n            self.write_bytes(\n                Response::List(mailbox.messages.iter().map(|m| m.size).collect::<Vec<_>>())\n                    .serialize(),\n            )\n            .await\n        }\n    }\n\n    pub async fn handle_uidl(&mut self, msg: Option<u32>) -> trc::Result<()> {\n        // Validate access\n        self.state\n            .access_token()\n            .assert_has_permission(Permission::Pop3Uidl)?;\n\n        let op_start = Instant::now();\n        let mailbox = self.state.mailbox();\n        if let Some(msg) = msg {\n            if let Some(message) = mailbox.messages.get(msg.saturating_sub(1) as usize) {\n                trc::event!(\n                    Pop3(trc::Pop3Event::UidlMessage),\n                    SpanId = self.session_id,\n                    DocumentId = message.id,\n                    Uid = message.uid,\n                    UidValidity = mailbox.uid_validity,\n                    Elapsed = op_start.elapsed()\n                );\n\n                self.write_ok(format!(\"{} {}{}\", msg, mailbox.uid_validity, message.uid))\n                    .await\n            } else {\n                Err(trc::Pop3Event::Error\n                    .into_err()\n                    .details(\"No such message.\")\n                    .caused_by(trc::location!()))\n            }\n        } else {\n            trc::event!(\n                Pop3(trc::Pop3Event::Uidl),\n                SpanId = self.session_id,\n                Total = mailbox.messages.len(),\n                Elapsed = op_start.elapsed()\n            );\n\n            self.write_bytes(\n                Response::List(\n                    mailbox\n                        .messages\n                        .iter()\n                        .map(|m| format!(\"{}{}\", mailbox.uid_validity, m.uid))\n                        .collect::<Vec<_>>(),\n                )\n                .serialize(),\n            )\n            .await\n        }\n    }\n\n    pub async fn handle_stat(&mut self) -> trc::Result<()> {\n        // Validate access\n        self.state\n            .access_token()\n            .assert_has_permission(Permission::Pop3Stat)?;\n\n        let op_start = Instant::now();\n        let mailbox = self.state.mailbox();\n\n        trc::event!(\n            Pop3(trc::Pop3Event::Stat),\n            SpanId = self.session_id,\n            Total = mailbox.total,\n            Size = mailbox.size,\n            Elapsed = op_start.elapsed()\n        );\n\n        self.write_ok(format!(\"{} {}\", mailbox.total, mailbox.size))\n            .await\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/op/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::listener::SessionStream;\n\nuse crate::{\n    Session,\n    protocol::{Mechanism, response::Response},\n};\n\npub mod authenticate;\npub mod delete;\npub mod fetch;\npub mod list;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_capa(&mut self) -> trc::Result<()> {\n        let mechanisms = if self.stream.is_tls() || self.server.core.imap.allow_plain_auth {\n            vec![Mechanism::Plain, Mechanism::OAuthBearer, Mechanism::XOauth2]\n        } else {\n            vec![Mechanism::OAuthBearer, Mechanism::XOauth2]\n        };\n\n        trc::event!(\n            Pop3(trc::Pop3Event::Capabilities),\n            SpanId = self.session_id,\n            Tls = self.stream.is_tls(),\n            Strict = !self.server.core.imap.allow_plain_auth,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        self.write_bytes(\n            Response::Capability::<u32> {\n                mechanisms,\n                stls: !self.stream.is_tls(),\n            }\n            .serialize(),\n        )\n        .await\n    }\n\n    pub async fn handle_stls(&mut self) -> trc::Result<()> {\n        trc::event!(\n            Pop3(trc::Pop3Event::StartTls),\n            SpanId = self.session_id,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        self.write_ok(\"Begin TLS negotiation now\").await\n    }\n\n    pub async fn handle_utf8(&mut self) -> trc::Result<()> {\n        trc::event!(\n            Pop3(trc::Pop3Event::Utf8),\n            SpanId = self.session_id,\n            Elapsed = trc::Value::Duration(0)\n        );\n\n        self.write_ok(\"UTF8 enabled\").await\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/protocol/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod request;\npub mod response;\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\npub enum Command<T, M> {\n    // Authorization state\n    User {\n        name: T,\n    },\n    Pass {\n        string: T,\n    },\n    Apop {\n        name: T,\n        digest: T,\n    },\n    Quit,\n\n    // Transaction state\n    Stat,\n    List {\n        msg: Option<u32>,\n    },\n    Retr {\n        msg: u32,\n    },\n    Dele {\n        msg: u32,\n    },\n    DeleMany {\n        msgs: Vec<u32>,\n    },\n    #[default]\n    Noop,\n    Rset,\n    Top {\n        msg: u32,\n        n: u32,\n    },\n    Uidl {\n        msg: Option<u32>,\n    },\n\n    // Extensions\n    Capa,\n    Stls,\n    Utf8,\n    Auth {\n        mechanism: M,\n        params: Vec<T>,\n    },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Mechanism {\n    Plain,\n    CramMd5,\n    DigestMd5,\n    ScramSha1,\n    ScramSha256,\n    Apop,\n    Ntlm,\n    Gssapi,\n    Anonymous,\n    External,\n    OAuthBearer,\n    XOauth2,\n}\n"
  },
  {
    "path": "crates/pop3/src/protocol/request.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse super::{Command, Mechanism};\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Error {\n    NeedsMoreData,\n    Parse(Cow<'static, str>),\n}\n\n#[derive(Default, Debug)]\npub enum State {\n    #[default]\n    Init,\n    Command {\n        buf: [u8; 4],\n        len: usize,\n    },\n    Argument {\n        request: Command<Vec<u8>, Vec<u8>>,\n        num: usize,\n        last_is_space: bool,\n    },\n    Error {\n        reason: Cow<'static, str>,\n    },\n}\n\n#[derive(Default)]\npub struct Parser {\n    pub state: State,\n}\n\nconst MAX_ARG_LEN: usize = 256;\n\nimpl Parser {\n    pub fn parse(\n        &mut self,\n        bytes: &mut std::slice::Iter<'_, u8>,\n    ) -> Result<Command<String, Mechanism>, Error> {\n        for &byte in bytes {\n            match &mut self.state {\n                State::Init => match byte {\n                    b' ' | b'\\t' | b'\\r' | b'\\n' => {}\n                    b'a'..=b'z' => {\n                        self.state = State::Command {\n                            buf: [byte, 0, 0, 0],\n                            len: 1,\n                        };\n                    }\n                    b'A'..=b'Z' => {\n                        self.state = State::Command {\n                            buf: [byte | 0x20, 0, 0, 0],\n                            len: 1,\n                        };\n                    }\n                    _ => {\n                        self.state = State::Error {\n                            reason: \"Invalid command\".into(),\n                        };\n                    }\n                },\n                State::Command { buf, len } => match byte {\n                    b'a'..=b'z' | b'8' if *len < 4 => {\n                        buf[*len] = byte;\n                        *len += 1;\n                    }\n                    b'A'..=b'Z' if *len < 4 => {\n                        buf[*len] = byte | 0x20;\n                        *len += 1;\n                    }\n                    b' ' | b'\\t' if *len == 4 || *len == 3 => match Command::parse(buf) {\n                        Ok(request) => {\n                            self.state = State::Argument {\n                                request,\n                                num: 0,\n                                last_is_space: true,\n                            };\n                        }\n                        Err(err) => {\n                            self.state = State::Error { reason: err };\n                        }\n                    },\n                    b'\\r' => {}\n                    b'\\n' if *len == 4 || *len == 3 => match Command::parse(buf) {\n                        Ok(request) => {\n                            self.state = State::Init;\n                            return request.finalize(0);\n                        }\n                        Err(err) => {\n                            self.state = State::Init;\n                            return Err(Error::Parse(err));\n                        }\n                    },\n                    _ => {\n                        self.state = State::Error {\n                            reason: \"Invalid command\".into(),\n                        };\n                    }\n                },\n                State::Argument {\n                    request,\n                    num,\n                    last_is_space,\n                } => match byte {\n                    b' ' | b'\\t' => {\n                        *last_is_space = true;\n                    }\n                    b'\\r' => {}\n                    b'\\n' => {\n                        let request = std::mem::take(request).finalize(*num);\n                        self.state = State::Init;\n                        return request;\n                    }\n                    _ => {\n                        if *last_is_space {\n                            *num += 1;\n                        }\n\n                        match request.update_argument(*num, byte) {\n                            Ok(_) => {\n                                *last_is_space = false;\n                            }\n                            Err(err) => {\n                                self.state = State::Error { reason: err };\n                            }\n                        }\n                    }\n                },\n                State::Error { reason } => {\n                    if byte == b'\\n' {\n                        let reason = std::mem::take(reason);\n                        self.state = State::Init;\n                        return Err(Error::Parse(reason));\n                    }\n                }\n            }\n        }\n\n        Err(Error::NeedsMoreData)\n    }\n}\n\nimpl Command<Vec<u8>, Vec<u8>> {\n    pub fn parse(bytes: &[u8; 4]) -> Result<Self, Cow<'static, str>> {\n        match (bytes[0], bytes[1], bytes[2], bytes[3]) {\n            (b'u', b's', b'e', b'r') => Ok(Self::User { name: Vec::new() }),\n            (b'u', b'i', b'd', b'l') => Ok(Self::Uidl { msg: None }),\n            (b'u', b't', b'f', b'8') => Ok(Self::Utf8),\n            (b'p', b'a', b's', b's') => Ok(Self::Pass { string: Vec::new() }),\n            (b'a', b'p', b'o', b'p') => Ok(Self::Apop {\n                name: Vec::new(),\n                digest: Vec::new(),\n            }),\n            (b'a', b'u', b't', b'h') => Ok(Self::Auth {\n                mechanism: Vec::new(),\n                params: Vec::new(),\n            }),\n            (b'q', b'u', b'i', b't') => Ok(Self::Quit),\n            (b'l', b'i', b's', b't') => Ok(Self::List { msg: None }),\n            (b'r', b'e', b't', b'r') => Ok(Self::Retr { msg: 0 }),\n            (b'r', b's', b'e', b't') => Ok(Self::Rset),\n            (b'd', b'e', b'l', b'e') => Ok(Self::Dele { msg: 0 }),\n            (b'n', b'o', b'o', b'p') => Ok(Self::Noop),\n            (b't', b'o', b'p', 0) => Ok(Self::Top { msg: 0, n: 0 }),\n            (b'c', b'a', b'p', b'a') => Ok(Self::Capa),\n            (b's', b't', b'l', b's') => Ok(Self::Stls),\n            (b's', b't', b'a', b't') => Ok(Self::Stat),\n            _ => Err(\"Invalid command\".into()),\n        }\n    }\n\n    pub fn update_argument(&mut self, arg_num: usize, byte: u8) -> Result<(), Cow<'static, str>> {\n        match self {\n            Command::User { name } if arg_num == 1 && name.len() < MAX_ARG_LEN => {\n                name.push(byte);\n                Ok(())\n            }\n            Command::Pass { string } if arg_num == 1 && string.len() < MAX_ARG_LEN => {\n                string.push(byte);\n                Ok(())\n            }\n            Command::Apop { name, digest }\n                if arg_num <= 2 && name.len() < MAX_ARG_LEN && digest.len() < MAX_ARG_LEN =>\n            {\n                if arg_num == 1 {\n                    name.push(byte);\n                } else {\n                    digest.push(byte);\n                }\n                Ok(())\n            }\n            Command::List { msg } if arg_num == 1 => add_digit(msg.get_or_insert(0), byte),\n            Command::Retr { msg } if arg_num == 1 => add_digit(msg, byte),\n            Command::Dele { msg } if arg_num == 1 => add_digit(msg, byte),\n            Command::Top { msg, n } if arg_num <= 2 => {\n                if arg_num == 1 {\n                    add_digit(msg, byte)\n                } else {\n                    add_digit(n, byte)\n                }\n            }\n            Command::Uidl { msg } if arg_num == 1 => add_digit(msg.get_or_insert(0), byte),\n            Command::Auth { mechanism, params }\n                if arg_num <= 4\n                    && mechanism.len() < 64\n                    && params.iter().map(|p| p.len()).sum::<usize>() < (MAX_ARG_LEN * 4) =>\n            {\n                if arg_num == 1 {\n                    mechanism.push(byte);\n                } else {\n                    if params.len() < arg_num - 1 {\n                        params.push(Vec::new());\n                    }\n                    params.last_mut().unwrap().push(byte);\n                }\n                Ok(())\n            }\n            _ => Err(\"Too many arguments\".into()),\n        }\n    }\n\n    pub fn finalize(self, num_args: usize) -> Result<Command<String, Mechanism>, Error> {\n        match self {\n            Command::User { name } if num_args == 1 => {\n                into_string(name).map(|name| Command::User { name })\n            }\n            Command::Pass { string } if num_args == 1 => {\n                into_string(string).map(|string| Command::Pass { string })\n            }\n            Command::Apop { name, digest } if num_args == 2 => {\n                let name = into_string(name)?;\n                let digest = into_string(digest)?;\n                Ok(Command::Apop { name, digest })\n            }\n            Command::Quit => Ok(Command::Quit),\n            Command::Stat => Ok(Command::Stat),\n            Command::List { msg } => Ok(Command::List { msg }),\n            Command::Retr { msg } if num_args == 1 => Ok(Command::Retr { msg }),\n            Command::Dele { msg } if num_args == 1 => Ok(Command::Dele { msg }),\n            Command::Noop => Ok(Command::Noop),\n            Command::Rset => Ok(Command::Rset),\n            Command::Top { msg, n } if num_args == 2 => Ok(Command::Top { msg, n }),\n            Command::Uidl { msg } => Ok(Command::Uidl { msg }),\n            Command::Capa => Ok(Command::Capa),\n            Command::Stls => Ok(Command::Stls),\n            Command::Utf8 => Ok(Command::Utf8),\n            Command::Auth { mechanism, params } if num_args >= 1 => {\n                let mechanism = Mechanism::parse(&mechanism)?;\n                let params = params\n                    .into_iter()\n                    .map(into_string)\n                    .collect::<Result<_, _>>()?;\n\n                Ok(Command::Auth { mechanism, params })\n            }\n            _ => Err(Error::Parse(\"Missing arguments\".into())),\n        }\n    }\n}\n\n#[inline(always)]\nfn into_string(bytes: Vec<u8>) -> Result<String, Error> {\n    String::from_utf8(bytes).map_err(|_| Error::Parse(\"Invalid UTF-8\".into()))\n}\n\n#[inline(always)]\nfn add_digit(num: &mut u32, byte: u8) -> Result<(), Cow<'static, str>> {\n    if byte.is_ascii_digit() {\n        *num = num\n            .checked_mul(10)\n            .and_then(|n| n.checked_add((byte - b'0') as u32))\n            .ok_or(\"Numeric argument out of range\")?;\n        Ok(())\n    } else {\n        Err(\"Invalid digit\".into())\n    }\n}\n\nimpl Mechanism {\n    pub fn parse(value: &[u8]) -> Result<Self, Error> {\n        if value.eq_ignore_ascii_case(b\"PLAIN\") {\n            Ok(Self::Plain)\n        } else if value.eq_ignore_ascii_case(b\"CRAM-MD5\") {\n            Ok(Self::CramMd5)\n        } else if value.eq_ignore_ascii_case(b\"DIGEST-MD5\") {\n            Ok(Self::DigestMd5)\n        } else if value.eq_ignore_ascii_case(b\"SCRAM-SHA-1\") {\n            Ok(Self::ScramSha1)\n        } else if value.eq_ignore_ascii_case(b\"SCRAM-SHA-256\") {\n            Ok(Self::ScramSha256)\n        } else if value.eq_ignore_ascii_case(b\"APOP\") {\n            Ok(Self::Apop)\n        } else if value.eq_ignore_ascii_case(b\"NTLM\") {\n            Ok(Self::Ntlm)\n        } else if value.eq_ignore_ascii_case(b\"GSSAPI\") {\n            Ok(Self::Gssapi)\n        } else if value.eq_ignore_ascii_case(b\"ANONYMOUS\") {\n            Ok(Self::Anonymous)\n        } else if value.eq_ignore_ascii_case(b\"EXTERNAL\") {\n            Ok(Self::External)\n        } else if value.eq_ignore_ascii_case(b\"OAUTHBEARER\") {\n            Ok(Self::OAuthBearer)\n        } else if value.eq_ignore_ascii_case(b\"XOAUTH2\") {\n            Ok(Self::XOauth2)\n        } else {\n            Err(Error::Parse(\n                format!(\n                    \"Unsupported mechanism '{}'.\",\n                    String::from_utf8_lossy(value)\n                )\n                .into(),\n            ))\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::protocol::{Command, Mechanism, request::Error};\n\n    use super::Parser;\n\n    #[test]\n    fn parse_command() {\n        let mut parser = Parser::default();\n        let mut chunked = String::new();\n        let mut chunked_expected = Vec::new();\n\n        for (cmd, request) in [\n            (\"QuiT\", Command::Quit),\n            (\" \\r\\n NOOP \", Command::Noop),\n            (\"STAT \", Command::Stat),\n            (\"LIST \", Command::List { msg: None }),\n            (\" list  100  \", Command::List { msg: 100.into() }),\n            (\"retr 55\", Command::Retr { msg: 55 }),\n            (\"DELE 99\", Command::Dele { msg: 99 }),\n            (\" rset \", Command::Rset),\n            (\"top 8000 1234\", Command::Top { msg: 8000, n: 1234 }),\n            (\"uidl\", Command::Uidl { msg: None }),\n            (\"uidl 000099999\", Command::Uidl { msg: 99999.into() }),\n            (\n                \"USER test\",\n                Command::User {\n                    name: \"test\".to_string(),\n                },\n            ),\n            (\n                \"PASS secret\",\n                Command::Pass {\n                    string: \"secret\".to_string(),\n                },\n            ),\n            (\n                \"APOP mrose c4c9334bac560ecc979e58001b3e22fb\",\n                Command::Apop {\n                    name: \"mrose\".to_string(),\n                    digest: \"c4c9334bac560ecc979e58001b3e22fb\".to_string(),\n                },\n            ),\n            (\"utf8\", Command::Utf8),\n            (\"capa\", Command::Capa),\n            (\n                \"AUTH GSSAPI\",\n                Command::Auth {\n                    mechanism: Mechanism::Gssapi,\n                    params: vec![],\n                },\n            ),\n            (\n                \"AUTH PLAIN dGVzdAB0ZXN0AHRlc3Q=\",\n                Command::Auth {\n                    mechanism: Mechanism::Plain,\n                    params: vec![\"dGVzdAB0ZXN0AHRlc3Q=\".to_string()],\n                },\n            ),\n        ] {\n            assert_eq!(\n                parser.parse(&mut cmd.as_bytes().iter()),\n                Err(Error::NeedsMoreData)\n            );\n            assert_eq!(\n                parser.parse(&mut b\"\\r\\n\".iter()),\n                Ok(request.clone()),\n                \"{:?}\",\n                cmd\n            );\n            chunked.push_str(cmd);\n            chunked.push_str(\"\\r\\n\");\n            chunked_expected.push(request);\n        }\n\n        for chunk_size in [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] {\n            let mut parser = Parser::default();\n            let mut requests = Vec::new();\n\n            for chunk in chunked.as_bytes().chunks(chunk_size) {\n                let mut chunk = chunk.iter();\n                loop {\n                    match parser.parse(&mut chunk) {\n                        Ok(request) => {\n                            requests.push(request);\n                        }\n                        Err(Error::NeedsMoreData) => break,\n                        Err(err) => {\n                            panic!(\"Unexpected error on chunk size {chunk_size}: {err:?}\");\n                        }\n                    }\n                }\n            }\n\n            assert_eq!(requests, chunked_expected, \"Chunk size: {}\", chunk_size);\n        }\n\n        for cmd in [\n            \"user\",\n            \"pass\",\n            \"user a b\",\n            \"pass c d\",\n            \"apop\",\n            \"apop a\",\n            \"apop a b c\",\n            \"quit 1\",\n            \"stat 1\",\n            \"list 1 2\",\n            \"retr\",\n            \"retr 1 2\",\n            \"dele\",\n            \"dele 1 2\",\n            \"noop 1\",\n            \"rset 1\",\n            \"top\",\n            \"top 1 2 3\",\n            \"uidl 1 2 3\",\n            \"capa 1\",\n            \"stls 1\",\n            \"utf8 1\",\n            \"auth\",\n            \"auth unknown\",\n        ] {\n            assert_eq!(\n                parser.parse(&mut cmd.as_bytes().iter()),\n                Err(Error::NeedsMoreData)\n            );\n            let result = parser.parse(&mut b\"\\r\\n\".iter());\n            assert!(result.is_err(), \"{:?}\", result);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/protocol/response.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::Mechanism;\nuse std::{borrow::Cow, fmt::Display};\nuse utils::chained_bytes::SliceRange;\n\npub enum Response<'x, T> {\n    Ok(Cow<'static, str>),\n    Err(Cow<'static, str>),\n    List(Vec<T>),\n    Message {\n        bytes: SliceRange<'x>,\n        lines: u32,\n    },\n    Capability {\n        mechanisms: Vec<Mechanism>,\n        stls: bool,\n    },\n}\n\nimpl<'x, T: Display> Response<'x, T> {\n    pub fn serialize(&self) -> Vec<u8> {\n        match self {\n            Response::Ok(message) => {\n                let mut buf = Vec::with_capacity(message.len() + 6);\n                buf.extend_from_slice(b\"+OK \");\n                buf.extend_from_slice(message.as_bytes());\n                buf.extend_from_slice(b\"\\r\\n\");\n                buf\n            }\n            Response::Err(message) => {\n                let mut buf = Vec::with_capacity(message.len() + 6);\n                buf.extend_from_slice(b\"-ERR \");\n                buf.extend_from_slice(message.as_bytes());\n                buf.extend_from_slice(b\"\\r\\n\");\n                buf\n            }\n            Response::List(octets) => {\n                let mut buf = Vec::with_capacity(octets.len() * 8 + 10);\n                buf.extend_from_slice(format!(\"+OK {} messages\\r\\n\", octets.len()).as_bytes());\n                for (num, octet) in octets.iter().enumerate() {\n                    buf.extend_from_slice((num + 1).to_string().as_bytes());\n                    buf.extend_from_slice(b\" \");\n                    buf.extend_from_slice(octet.to_string().as_bytes());\n                    buf.extend_from_slice(b\"\\r\\n\");\n                }\n                buf.extend_from_slice(b\".\\r\\n\");\n                buf\n            }\n            Response::Message { bytes, lines } => {\n                let mut buf = Vec::with_capacity(bytes.len() + 10);\n                buf.extend_from_slice(b\"+OK \");\n                buf.extend_from_slice(bytes.len().to_string().as_bytes());\n                buf.extend_from_slice(b\" octets\\r\\n\");\n\n                let mut line_count = 0;\n                let mut last_byte = 0;\n\n                // Transparency procedure\n                for &byte in bytes.into_iter() {\n                    // POP3 requires that lines end with CRLF, do this check to ensure that\n                    if byte == b'\\n' && last_byte != b'\\r' {\n                        buf.push(b'\\r');\n                    }\n\n                    if byte == b'.' && last_byte == b'\\n' {\n                        buf.push(b'.');\n                    }\n                    buf.push(byte);\n                    last_byte = byte;\n\n                    if *lines > 0 && byte == b'\\n' {\n                        line_count += 1;\n                        if line_count == *lines {\n                            break;\n                        }\n                    }\n                }\n\n                if last_byte != b'\\n' {\n                    buf.extend_from_slice(b\"\\r\\n\");\n                }\n\n                buf.extend_from_slice(b\".\\r\\n\");\n                buf\n            }\n            Response::Capability { mechanisms, stls } => {\n                let mut buf = Vec::with_capacity(256);\n                buf.extend_from_slice(b\"+OK Capability list follows\\r\\n\");\n                if !mechanisms.is_empty() {\n                    if mechanisms.contains(&Mechanism::Plain) {\n                        buf.extend_from_slice(b\"USER\\r\\n\");\n                    }\n                    buf.extend_from_slice(b\"SASL\");\n                    for mechanism in mechanisms {\n                        buf.extend_from_slice(b\" \");\n                        buf.extend_from_slice(mechanism.as_str().as_bytes());\n                    }\n                    buf.extend_from_slice(b\"\\r\\n\");\n                }\n\n                if *stls {\n                    buf.extend_from_slice(b\"STLS\\r\\n\");\n                }\n\n                for capa in [\n                    \"TOP\",\n                    \"RESP-CODES\",\n                    \"PIPELINING\",\n                    \"EXPIRE NEVER\",\n                    \"UIDL\",\n                    \"UTF8\",\n                    \"IMPLEMENTATION Stalwart Server\",\n                ] {\n                    buf.extend_from_slice(capa.as_bytes());\n                    buf.extend_from_slice(b\"\\r\\n\");\n                }\n\n                buf.extend_from_slice(b\".\\r\\n\");\n                buf\n            }\n        }\n    }\n}\n\nimpl Mechanism {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Mechanism::Plain => \"PLAIN\",\n            Mechanism::CramMd5 => \"CRAM-MD5\",\n            Mechanism::DigestMd5 => \"DIGEST-MD5\",\n            Mechanism::ScramSha1 => \"SCRAM-SHA-1\",\n            Mechanism::ScramSha256 => \"SCRAM-SHA-256\",\n            Mechanism::Apop => \"APOP\",\n            Mechanism::Ntlm => \"NTLM\",\n            Mechanism::Gssapi => \"GSSAPI\",\n            Mechanism::Anonymous => \"ANONYMOUS\",\n            Mechanism::External => \"EXTERNAL\",\n            Mechanism::OAuthBearer => \"OAUTHBEARER\",\n            Mechanism::XOauth2 => \"XOAUTH2\",\n        }\n    }\n}\n\npub trait SerializeResponse {\n    fn serialize(&self) -> Vec<u8>;\n}\n\nimpl SerializeResponse for trc::Error {\n    fn serialize(&self) -> Vec<u8> {\n        let message = self\n            .value_as_str(trc::Key::Details)\n            .unwrap_or_else(|| self.as_ref().message());\n        let mut buf = Vec::with_capacity(message.len() + 6);\n        buf.extend_from_slice(b\"-ERR \");\n        buf.extend_from_slice(message.as_bytes());\n        buf.extend_from_slice(b\"\\r\\n\");\n        buf\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::Response;\n    use crate::protocol::Mechanism;\n    use utils::chained_bytes::SliceRange;\n\n    #[test]\n    fn serialize_response() {\n        for (cmd, expected) in [\n            (\n                Response::Ok(\"message 1 deleted\".into()),\n                \"+OK message 1 deleted\\r\\n\",\n            ),\n            (\n                Response::Err(\"permission denied\".into()),\n                \"-ERR permission denied\\r\\n\",\n            ),\n            (\n                Response::List(vec![100, 200, 300]),\n                \"+OK 3 messages\\r\\n1 100\\r\\n2 200\\r\\n3 300\\r\\n.\\r\\n\",\n            ),\n            (\n                Response::Capability {\n                    mechanisms: vec![Mechanism::Plain, Mechanism::CramMd5],\n                    stls: true,\n                },\n                concat!(\n                    \"+OK Capability list follows\\r\\n\",\n                    \"USER\\r\\n\",\n                    \"SASL PLAIN CRAM-MD5\\r\\n\",\n                    \"STLS\\r\\n\",\n                    \"TOP\\r\\n\",\n                    \"RESP-CODES\\r\\n\",\n                    \"PIPELINING\\r\\n\",\n                    \"EXPIRE NEVER\\r\\n\",\n                    \"UIDL\\r\\n\",\n                    \"UTF8\\r\\n\",\n                    \"IMPLEMENTATION Stalwart Server\\r\\n.\\r\\n\"\n                ),\n            ),\n            (\n                Response::Message {\n                    bytes: SliceRange::Split(b\"Subject: test\\r\\n\\r\\n.\\r\\n\", b\"test.\\r\\n.test\\r\\na\"),\n                    lines: 0,\n                },\n                \"+OK 35 octets\\r\\nSubject: test\\r\\n\\r\\n..\\r\\ntest.\\r\\n..test\\r\\na\\r\\n.\\r\\n\",\n            ),\n        ] {\n            assert_eq!(expected, String::from_utf8(cmd.serialize()).unwrap());\n        }\n    }\n}\n"
  },
  {
    "path": "crates/pop3/src/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse common::{\n    core::BuildServer,\n    listener::{SessionData, SessionManager, SessionResult, SessionStream},\n};\nuse tokio_rustls::server::TlsStream;\n\nuse crate::{\n    Pop3SessionManager, SERVER_GREETING, Session, State,\n    protocol::{\n        request::Parser,\n        response::{Response, SerializeResponse},\n    },\n};\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\n\nimpl SessionManager for Pop3SessionManager {\n    #[allow(clippy::manual_async_fn)]\n    fn handle<T: SessionStream>(\n        self,\n        session: SessionData<T>,\n    ) -> impl std::future::Future<Output = ()> + Send {\n        async move {\n            let mut session = Session {\n                server: self.inner.build_server(),\n                instance: session.instance,\n                receiver: Parser::default(),\n                state: State::NotAuthenticated {\n                    auth_failures: 0,\n                    username: None,\n                },\n                stream: session.stream,\n                in_flight: session.in_flight,\n                remote_addr: session.remote_ip,\n                session_id: session.session_id,\n            };\n\n            if session\n                .write_bytes(SERVER_GREETING.as_bytes())\n                .await\n                .is_ok()\n                && session.handle_conn().await\n                && session.instance.acceptor.is_tls()\n                && let Ok(mut session) = session.into_tls().await\n            {\n                session.handle_conn().await;\n            }\n        }\n    }\n\n    #[allow(clippy::manual_async_fn)]\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send {\n        async {}\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_conn(&mut self) -> bool {\n        let mut buf = vec![0; 8192];\n        let mut shutdown_rx = self.instance.shutdown_rx.clone();\n\n        loop {\n            tokio::select! {\n                result = tokio::time::timeout(\n                    if !matches!(self.state, State::NotAuthenticated {..}) {\n                        self.server.core.imap.timeout_auth\n                    } else {\n                        self.server.core.imap.timeout_unauth\n                    },\n                    self.stream.read(&mut buf)) => {\n                    match result {\n                        Ok(Ok(bytes_read)) => {\n                            if bytes_read > 0 {\n                                match self.ingest(&buf[..bytes_read]).await {\n                                    SessionResult::Continue => (),\n                                    SessionResult::UpgradeTls => {\n                                        return true;\n                                    }\n                                    SessionResult::Close => {\n                                        break;\n                                    }\n                                }\n                            } else {\n                                trc::event!(\n                                    Network(trc::NetworkEvent::Closed),\n                                    SpanId = self.session_id,\n                                    CausedBy = trc::location!()\n                                );\n                                break;\n                            }\n                        },\n                        Ok(Err(err)) => {\n                            trc::event!(\n                                Network(trc::NetworkEvent::ReadError),\n                                SpanId = self.session_id,\n                                Reason = err.to_string()    ,\n                                CausedBy = trc::location!()\n                            );\n                            break;\n                        },\n                        Err(_) => {\n                            trc::event!(\n                                Network(trc::NetworkEvent::Timeout),\n                                SpanId = self.session_id,\n                                CausedBy = trc::location!()\n                            );\n\n                            self.write_bytes(&b\"-ERR Connection timed out.\\r\\n\"[..]).await.ok();\n                            break;\n                        }\n                    }\n                },\n                _ = shutdown_rx.changed() => {\n                    trc::event!(\n                        Network(trc::NetworkEvent::Closed),\n                        SpanId = self.session_id,\n                        Reason = \"Server shutting down\",\n                        CausedBy = trc::location!()\n                    );\n\n                    self.write_bytes(&b\"* BYE Server shutting down.\\r\\n\"[..]).await.ok();\n                    break;\n                }\n            };\n        }\n\n        false\n    }\n\n    pub async fn into_tls(self) -> Result<Session<TlsStream<T>>, ()> {\n        Ok(Session {\n            stream: self\n                .instance\n                .tls_accept(self.stream, self.session_id)\n                .await?,\n            server: self.server,\n            instance: self.instance,\n            receiver: self.receiver,\n            state: self.state,\n            session_id: self.session_id,\n            in_flight: self.in_flight,\n            remote_addr: self.remote_addr,\n        })\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn write_bytes(&mut self, bytes: impl AsRef<[u8]>) -> trc::Result<()> {\n        let bytes = bytes.as_ref();\n\n        trc::event!(\n            Pop3(trc::Pop3Event::RawOutput),\n            SpanId = self.session_id,\n            Size = bytes.len(),\n            Contents = trc::Value::from_maybe_string(bytes),\n        );\n\n        self.stream.write_all(bytes.as_ref()).await.map_err(|err| {\n            trc::NetworkEvent::WriteError\n                .into_err()\n                .reason(err)\n                .caused_by(trc::location!())\n        })?;\n        self.stream.flush().await.map_err(|err| {\n            trc::NetworkEvent::WriteError\n                .into_err()\n                .reason(err)\n                .caused_by(trc::location!())\n        })\n    }\n\n    pub async fn write_ok(&mut self, message: impl Into<Cow<'static, str>>) -> trc::Result<()> {\n        self.write_bytes(Response::Ok::<u32>(message.into()).serialize())\n            .await\n    }\n\n    pub async fn write_err(&mut self, err: trc::Error) -> bool {\n        let disconnect = err.must_disconnect();\n        let response = err.serialize();\n        let write_err = err.should_write_err();\n\n        trc::error!(err.span_id(self.session_id));\n\n        if write_err && let Err(err) = self.write_bytes(response).await {\n            trc::error!(err.span_id(self.session_id));\n            return false;\n        }\n\n        !disconnect\n    }\n}\n"
  },
  {
    "path": "crates/services/Cargo.toml",
    "content": "[package]\nname = \"services\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nstore = { path = \"../store\" }\ncommon = { path =  \"../common\" }\nutils = { path =  \"../utils\" }\ntrc = { path = \"../trc\" }\nemail = { path = \"../email\" }\nsmtp = { path = \"../smtp\" }\ngroupware = { path = \"../groupware\" }\nspam-filter = { path = \"../spam-filter\" }\ntypes = { path = \"../types\" }\njmap_proto = { path = \"../jmap-proto\" }\ndirectory = { path =  \"../directory\" }\nsmtp-proto = { version = \"0.2\", features = [\"rkyv\", \"serde\"] }\ntokio = { version = \"1.47\", features = [\"rt\"] }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] }\nmail-builder = { version = \"0.4\" } \ncalcard = { version = \"0.3\", features = [\"rkyv\"] }\nchrono = { version = \"0.4\", features = [\"unstable-locales\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\nmemory-stats = \"1.2.0\"\naes-gcm = \"0.10.1\"\naes-gcm-siv = \"0.11.1\"\nrsa = \"0.9.2\"\np256 = { version = \"0.13\", features = [\"ecdh\"] }\nhkdf = \"0.12.3\"\nsha2 = \"0.10\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\"]}\nbase64 = \"0.22\"\ncompact_str = \"0.9.0\"\n\n[dev-dependencies]\n\n[features]\ntest_mode = []\nenterprise = []\n"
  },
  {
    "path": "crates/services/src/broadcast/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::ipc::{BroadcastEvent, CalendarAlert, EmailPush, PushNotification};\nuse std::{borrow::Borrow, io::Write};\nuse types::type_state::StateChange;\nuse utils::{\n    codec::leb128::{Leb128Iterator, Leb128Writer},\n    map::bitmap::Bitmap,\n};\n\npub mod publisher;\npub mod subscriber;\n\n#[derive(Debug)]\npub(crate) struct BroadcastBatch<T> {\n    messages: T,\n}\n\nconst MAX_BATCH_SIZE: usize = 100;\npub(crate) const BROADCAST_TOPIC: &str = \"stwt.agora\";\n\nimpl BroadcastBatch<Vec<BroadcastEvent>> {\n    pub fn init() -> Self {\n        Self {\n            messages: Vec::with_capacity(MAX_BATCH_SIZE),\n        }\n    }\n\n    pub fn insert(&mut self, message: BroadcastEvent) -> bool {\n        self.messages.push(message);\n        self.messages.len() < MAX_BATCH_SIZE\n    }\n\n    pub fn serialize(&self, node_id: u16) -> Vec<u8> {\n        let mut serialized =\n            Vec::with_capacity((self.messages.len() * 10) + std::mem::size_of::<u16>());\n        let _ = serialized.write_leb128(node_id);\n        for message in &self.messages {\n            match message {\n                BroadcastEvent::PushNotification(notification) => match notification {\n                    PushNotification::StateChange(state_change) => {\n                        serialized.push(0u8);\n                        let _ = serialized.write_leb128(state_change.change_id);\n                        let _ = serialized.write_leb128(*state_change.types.as_ref());\n                        let _ = serialized.write_leb128(state_change.account_id);\n                    }\n                    PushNotification::CalendarAlert(calendar_alert) => {\n                        serialized.push(1u8);\n                        let _ = serialized.write_leb128(calendar_alert.account_id);\n                        let _ = serialized.write_leb128(calendar_alert.event_id);\n                        let _ = serialized\n                            .write_leb128(calendar_alert.recurrence_id.unwrap_or_default() as u64);\n                        let _ = serialized.write_leb128(calendar_alert.uid.len());\n                        let _ = serialized.write(calendar_alert.uid.as_bytes());\n                        let _ = serialized.write_leb128(calendar_alert.alert_id.len());\n                        let _ = serialized.write(calendar_alert.alert_id.as_bytes());\n                    }\n                    PushNotification::EmailPush(email_push) => {\n                        serialized.push(2u8);\n                        let _ = serialized.write_leb128(email_push.account_id);\n                        let _ = serialized.write_leb128(email_push.email_id);\n                        let _ = serialized.write_leb128(email_push.change_id);\n                    }\n                },\n                BroadcastEvent::InvalidateAccessTokens(items) => {\n                    serialized.push(3u8);\n                    let _ = serialized.write_leb128(items.len());\n                    for item in items {\n                        let _ = serialized.write_leb128(*item);\n                    }\n                }\n                BroadcastEvent::InvalidateGroupwareCache(items) => {\n                    serialized.push(4u8);\n                    let _ = serialized.write_leb128(items.len());\n                    for item in items {\n                        let _ = serialized.write_leb128(*item);\n                    }\n                }\n                BroadcastEvent::ReloadSettings => {\n                    serialized.push(5u8);\n                }\n                BroadcastEvent::ReloadBlockedIps => {\n                    serialized.push(6u8);\n                }\n                BroadcastEvent::ReloadPushServers(account_id) => {\n                    serialized.push(7u8);\n                    let _ = serialized.write_leb128(*account_id);\n                }\n                BroadcastEvent::ReloadSpamFilter => {\n                    serialized.push(8u8);\n                }\n            }\n        }\n        serialized\n    }\n\n    pub fn clear(&mut self) {\n        self.messages.clear();\n    }\n}\n\nimpl<T, I> BroadcastBatch<T>\nwhere\n    T: Iterator<Item = I> + Leb128Iterator<I>,\n    I: Borrow<u8>,\n{\n    pub fn node_id(&mut self) -> Option<u16> {\n        self.messages.next_leb128::<u16>()\n    }\n\n    pub fn next_event(&mut self) -> Result<Option<BroadcastEvent>, ()> {\n        if let Some(id) = self.messages.next() {\n            match id.borrow() {\n                0 => Ok(Some(BroadcastEvent::PushNotification(\n                    PushNotification::StateChange(StateChange {\n                        change_id: self.messages.next_leb128().ok_or(())?,\n                        types: Bitmap::from(self.messages.next_leb128::<u64>().ok_or(())?),\n                        account_id: self.messages.next_leb128().ok_or(())?,\n                    }),\n                ))),\n\n                1 => {\n                    let account_id = self.messages.next_leb128().ok_or(())?;\n                    let event_id = self.messages.next_leb128().ok_or(())?;\n                    let recurrence_id = self.messages.next_leb128::<u64>().ok_or(())? as i64;\n                    let uid_len = self.messages.next_leb128::<usize>().ok_or(())?;\n                    let mut uid_bytes = vec![0u8; uid_len];\n                    for byte in uid_bytes.iter_mut() {\n                        *byte = self.messages.next().ok_or(())?.borrow().to_owned();\n                    }\n                    let uid = String::from_utf8(uid_bytes).map_err(|_| ())?;\n                    let alert_id_len = self.messages.next_leb128::<usize>().ok_or(())?;\n                    let mut alert_id_bytes = vec![0u8; alert_id_len];\n                    for byte in alert_id_bytes.iter_mut() {\n                        *byte = self.messages.next().ok_or(())?.borrow().to_owned();\n                    }\n                    let alert_id = String::from_utf8(alert_id_bytes).map_err(|_| ())?;\n                    Ok(Some(BroadcastEvent::PushNotification(\n                        PushNotification::CalendarAlert(CalendarAlert {\n                            account_id,\n                            event_id,\n                            recurrence_id: if recurrence_id == 0 {\n                                None\n                            } else {\n                                Some(recurrence_id)\n                            },\n                            uid,\n                            alert_id,\n                        }),\n                    )))\n                }\n\n                2 => Ok(Some(BroadcastEvent::PushNotification(\n                    PushNotification::EmailPush(EmailPush {\n                        account_id: self.messages.next_leb128().ok_or(())?,\n                        email_id: self.messages.next_leb128().ok_or(())?,\n                        change_id: self.messages.next_leb128().ok_or(())?,\n                    }),\n                ))),\n\n                3 => {\n                    let count = self.messages.next_leb128::<usize>().ok_or(())?;\n                    let mut items = Vec::with_capacity(count);\n                    for _ in 0..count {\n                        items.push(self.messages.next_leb128().ok_or(())?);\n                    }\n                    Ok(Some(BroadcastEvent::InvalidateAccessTokens(items)))\n                }\n\n                4 => {\n                    let count = self.messages.next_leb128::<usize>().ok_or(())?;\n                    let mut items = Vec::with_capacity(count);\n                    for _ in 0..count {\n                        items.push(self.messages.next_leb128().ok_or(())?);\n                    }\n                    Ok(Some(BroadcastEvent::InvalidateGroupwareCache(items)))\n                }\n\n                5 => Ok(Some(BroadcastEvent::ReloadSettings)),\n\n                6 => Ok(Some(BroadcastEvent::ReloadBlockedIps)),\n\n                7 => {\n                    let account_id = self.messages.next_leb128().ok_or(())?;\n                    Ok(Some(BroadcastEvent::ReloadPushServers(account_id)))\n                }\n\n                8 => Ok(Some(BroadcastEvent::ReloadSpamFilter)),\n\n                _ => Err(()),\n            }\n        } else {\n            Ok(None)\n        }\n    }\n}\n\nimpl<T> BroadcastBatch<T> {\n    pub fn new(messages: T) -> Self {\n        Self { messages }\n    }\n}\n"
  },
  {
    "path": "crates/services/src/broadcast/publisher.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse common::{Inner, ipc::BroadcastEvent};\nuse tokio::sync::mpsc;\nuse trc::ClusterEvent;\n\nuse super::{BROADCAST_TOPIC, BroadcastBatch};\n\npub fn spawn_broadcast_publisher(inner: Arc<Inner>, mut event_rx: mpsc::Receiver<BroadcastEvent>) {\n    let (pubsub, this_node_id) = {\n        let _core = inner.shared_core.load();\n        let pubsub = inner.shared_core.load().storage.pubsub.clone();\n        if pubsub.is_none() {\n            return;\n        }\n        (pubsub, _core.network.node_id as u16)\n    };\n\n    tokio::spawn(async move {\n        let mut batch = BroadcastBatch::init();\n\n        trc::event!(Cluster(ClusterEvent::PublisherStart));\n\n        while let Some(event) = event_rx.recv().await {\n            batch.insert(event);\n\n            while let Ok(event) = event_rx.try_recv() {\n                if !batch.insert(event) {\n                    break;\n                }\n            }\n\n            match pubsub\n                .publish(BROADCAST_TOPIC, batch.serialize(this_node_id))\n                .await\n            {\n                Ok(_) => {\n                    batch.clear();\n                }\n                Err(err) => {\n                    batch.clear();\n                    trc::event!(Cluster(ClusterEvent::PublisherError), CausedBy = err);\n                }\n            }\n        }\n\n        trc::event!(Cluster(ClusterEvent::PublisherStop));\n    });\n}\n"
  },
  {
    "path": "crates/services/src/broadcast/subscriber.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::broadcast::{BROADCAST_TOPIC, BroadcastBatch};\nuse common::{\n    Inner,\n    core::BuildServer,\n    ipc::{BroadcastEvent, HousekeeperEvent, PushEvent, PushNotification},\n};\nuse compact_str::CompactString;\nuse std::{sync::Arc, time::Duration};\nuse tokio::sync::watch;\nuse trc::{ClusterEvent, ServerEvent};\n\npub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Receiver<bool>) {\n    let this_node_id = {\n        let _core = inner.shared_core.load();\n        if _core.storage.pubsub.is_none() {\n            return;\n        }\n        _core.network.node_id as u16\n    };\n\n    tokio::spawn(async move {\n        let mut retry_count = 0;\n\n        trc::event!(Cluster(ClusterEvent::SubscriberStart));\n\n        loop {\n            let pubsub = inner.shared_core.load().storage.pubsub.clone();\n            if pubsub.is_none() {\n                trc::event!(\n                    Cluster(ClusterEvent::SubscriberError),\n                    Details = \"PubSub is no longer configured\"\n                );\n                break;\n            }\n\n            let mut stream = match pubsub.subscribe(BROADCAST_TOPIC).await {\n                Ok(stream) => {\n                    retry_count = 0;\n                    stream\n                }\n                Err(err) => {\n                    trc::event!(\n                        Cluster(ClusterEvent::SubscriberError),\n                        CausedBy = err,\n                        Details = \"Failed to subscribe to channel\"\n                    );\n\n                    match tokio::time::timeout(\n                        Duration::from_secs(1 << retry_count.max(6)),\n                        shutdown_rx.changed(),\n                    )\n                    .await\n                    {\n                        Ok(_) => {\n                            break;\n                        }\n                        Err(_) => {\n                            retry_count += 1;\n                            continue;\n                        }\n                    }\n                }\n            };\n\n            tokio::select! {\n                message = stream.next() => {\n                    match message {\n                        Some(message) => {\n                            let mut batch = BroadcastBatch::new(message.payload().iter());\n                            let node_id = match batch.node_id() {\n                                Some(node_id) => {\n                                    if node_id != this_node_id {\n                                        node_id\n                                    } else {\n                                        trc::event!(\n                                            Cluster(ClusterEvent::MessageSkipped),\n                                            Details = message.payload()\n                                        );\n                                        continue;\n                                    }\n                                }\n                                None => {\n                                    trc::event!(\n                                        Cluster(ClusterEvent::MessageInvalid),\n                                        Details = message.payload()\n                                    );\n                                    continue;\n                                }\n                            };\n\n                            loop {\n                                match batch.next_event() {\n                                    Ok(Some(event)) => {\n                                        trc::event!(\n                                            Cluster(ClusterEvent::MessageReceived),\n                                            From = node_id,\n                                            To = this_node_id,\n                                            Details = log_event(&event),\n                                        );\n                                        match event {\n                                            BroadcastEvent::PushNotification(notification) => {\n                                                if inner\n                                                    .ipc\n                                                    .push_tx\n                                                    .send(PushEvent::Publish {\n                                                        notification,\n                                                        broadcast: false,\n                                                    })\n                                                    .await\n                                                    .is_err()\n                                                {\n                                                    trc::event!(\n                                                        Server(ServerEvent::ThreadError),\n                                                        Details = \"Error sending push notification.\",\n                                                        CausedBy = trc::location!()\n                                                    );\n                                                }\n                                            }\n                                            BroadcastEvent::ReloadPushServers(account_id) => {\n                                                if inner\n                                                    .ipc\n                                                    .push_tx\n                                                    .send(PushEvent::PushServerUpdate { account_id, broadcast: false })\n                                                    .await\n                                                    .is_err()\n                                                {\n                                                    trc::event!(\n                                                        Server(ServerEvent::ThreadError),\n                                                        Details = \"Error sending reload request.\",\n                                                        CausedBy = trc::location!()\n                                                    );\n                                                }\n                                            }\n                                            BroadcastEvent::InvalidateAccessTokens(ids) => {\n                                                for id in &ids {\n                                                    inner.cache.permissions.remove(id);\n                                                    inner.cache.access_tokens.remove(id);\n                                                }\n                                            }\n                                            BroadcastEvent::InvalidateGroupwareCache(ids) => {\n                                                for id in &ids {\n                                                    inner.cache.files.remove(id);\n                                                    inner.cache.contacts.remove(id);\n                                                    inner.cache.events.remove(id);\n                                                    inner.cache.scheduling.remove(id);\n                                                }\n                                            }\n                                            BroadcastEvent::ReloadSettings => {\n                                                match inner.build_server().reload().await {\n                                                    Ok(result) => {\n                                                        if let Some(new_core) = result.new_core {\n                                                            // Update core\n                                                            inner.shared_core.store(new_core.into());\n\n                                                            if inner\n                                                                .ipc\n                                                                .housekeeper_tx\n                                                                .send(HousekeeperEvent::ReloadSettings)\n                                                                .await\n                                                                .is_err()\n                                                            {\n                                                                trc::event!(\n                                                                    Server(trc::ServerEvent::ThreadError),\n                                                                    Details = \"Failed to send setting reload event to housekeeper\",\n                                                                    CausedBy = trc::location!(),\n                                                                );\n                                                            }\n                                                        }\n                                                    }\n                                                    Err(err) => {\n                                                        trc::error!(\n                                                            err.details(\"Failed to reload settings\")\n                                                                .caused_by(trc::location!())\n                                                        );\n                                                    }\n                                                }\n                                            }\n                                            BroadcastEvent::ReloadBlockedIps => {\n                                                if let Err(err) = inner.build_server().reload_blocked_ips().await {\n                                                    trc::error!(\n                                                        err.details(\"Failed to reload settings\")\n                                                            .caused_by(trc::location!())\n                                                    );\n                                                }\n                                            }\n                                            BroadcastEvent::ReloadSpamFilter => {\n                                                if let Err(err) = inner.build_server().spam_model_reload().await {\n                                                    trc::error!(\n                                                        err.details(\"Failed to reload spam filter model\")\n                                                            .caused_by(trc::location!())\n                                                    );\n                                                }\n                                            }\n                                        }\n                                    }\n                                    Ok(None) => break,\n                                    Err(_) => {\n                                        trc::event!(\n                                            Cluster(ClusterEvent::MessageInvalid),\n                                            Details = message.payload()\n                                        );\n                                        break;\n                                    }\n                                }\n                            }\n                        }\n                        None => {\n                            trc::event!(\n                                Cluster(ClusterEvent::SubscriberDisconnected),\n                            );\n                        }\n                    }\n                },\n                _ = shutdown_rx.changed() => {\n                    break;\n                }\n            };\n        }\n\n        trc::event!(Cluster(ClusterEvent::SubscriberStop));\n    });\n}\n\nfn log_event(event: &BroadcastEvent) -> trc::Value {\n    match event {\n        BroadcastEvent::PushNotification(notification) => match notification {\n            PushNotification::StateChange(state_change) => trc::Value::Array(vec![\n                \"StateChange\".into(),\n                state_change.account_id.into(),\n                state_change.change_id.into(),\n                (*state_change.types.as_ref()).into(),\n            ]),\n            PushNotification::CalendarAlert(calendar_alert) => trc::Value::Array(vec![\n                \"CalendarAlert\".into(),\n                calendar_alert.account_id.into(),\n                calendar_alert.event_id.into(),\n                calendar_alert.recurrence_id.into(),\n                calendar_alert.uid.clone().into(),\n                calendar_alert.alert_id.clone().into(),\n            ]),\n            PushNotification::EmailPush(email_push) => trc::Value::Array(vec![\n                \"EmailPush\".into(),\n                email_push.account_id.into(),\n                email_push.email_id.into(),\n                email_push.change_id.into(),\n            ]),\n        },\n        BroadcastEvent::ReloadSettings => CompactString::const_new(\"ReloadSettings\").into(),\n        BroadcastEvent::ReloadBlockedIps => CompactString::const_new(\"ReloadBlockedIps\").into(),\n        BroadcastEvent::InvalidateAccessTokens(items) => {\n            let mut array = Vec::with_capacity(items.len() + 1);\n            array.push(\"InvalidateAccessTokens\".into());\n            for item in items {\n                array.push((*item).into());\n            }\n            trc::Value::Array(array)\n        }\n        BroadcastEvent::InvalidateGroupwareCache(items) => {\n            let mut array = Vec::with_capacity(items.len() + 1);\n            array.push(\"InvalidateGroupwareCache\".into());\n            for item in items {\n                array.push((*item).into());\n            }\n            trc::Value::Array(array)\n        }\n        BroadcastEvent::ReloadPushServers(account_id) => {\n            trc::Value::Array(vec![\"ReloadPushServers\".into(), (*account_id).into()])\n        }\n        BroadcastEvent::ReloadSpamFilter => CompactString::const_new(\"ReloadSpamFilter\").into(),\n    }\n}\n"
  },
  {
    "path": "crates/services/src/housekeeper/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Inner, KV_LOCK_HOUSEKEEPER, LONG_1D_SLUMBER, Server,\n    config::{spamfilter, telemetry::OtelMetrics},\n    core::BuildServer,\n    ipc::{BroadcastEvent, HousekeeperEvent, PurgeType},\n};\nuse email::message::delete::EmailDeletion;\nuse smtp::reporting::SmtpReporting;\nuse spam_filter::modules::classifier::SpamClassifier;\nuse std::{\n    collections::BinaryHeap,\n    future::Future,\n    sync::Arc,\n    time::{Duration, Instant, SystemTime},\n};\nuse store::{PurgeStore, write::now};\nuse tokio::sync::mpsc;\nuse trc::{Collector, MetricType, PurgeEvent};\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\nuse common::telemetry::{\n    metrics::store::{MetricsStore, SharedMetricHistory},\n    tracers::store::TracingStore,\n};\n// SPDX-SnippetEnd\n\n#[derive(PartialEq, Eq)]\nstruct Action {\n    due: Instant,\n    event: ActionClass,\n}\n\n#[derive(PartialEq, Eq, Debug)]\nenum ActionClass {\n    Account,\n    Store(usize),\n    Acme(String),\n    OtelMetrics,\n    CalculateMetrics,\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    InternalMetrics,\n    #[cfg(feature = \"enterprise\")]\n    AlertMetrics,\n    #[cfg(feature = \"enterprise\")]\n    RenewLicense,\n    // SPDX-SnippetEnd\n    TrainSpamClassifier,\n}\n\n#[derive(Default)]\nstruct Queue {\n    heap: BinaryHeap<Action>,\n}\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\nconst METRIC_ALERTS_INTERVAL: Duration = Duration::from_secs(5 * 60);\n// SPDX-SnippetEnd\n\npub fn spawn_housekeeper(inner: Arc<Inner>, mut rx: mpsc::Receiver<HousekeeperEvent>) {\n    tokio::spawn(async move {\n        trc::event!(Housekeeper(trc::HousekeeperEvent::Start));\n        let start_time = SystemTime::now();\n\n        // Add all events to queue\n        let mut queue = Queue::default();\n        {\n            let server = inner.build_server();\n            let roles = &server.core.network.roles;\n\n            // Account purge\n            if roles.purge_accounts.is_enabled_or_sharded() {\n                queue.schedule(\n                    Instant::now() + server.core.jmap.account_purge_frequency.time_to_next(),\n                    ActionClass::Account,\n                );\n            }\n\n            // Store purges\n            if roles.purge_stores.is_enabled_or_sharded() {\n                for (idx, schedule) in server.core.storage.purge_schedules.iter().enumerate() {\n                    queue.schedule(\n                        Instant::now() + schedule.cron.time_to_next(),\n                        ActionClass::Store(idx),\n                    );\n                }\n            }\n\n            // Spam classifier training\n            if roles.spam_training.is_enabled_or_sharded()\n                && let Some(train_frequency) = server\n                    .core\n                    .spam\n                    .classifier\n                    .as_ref()\n                    .and_then(|c| c.train_frequency)\n            {\n                let next_train = match server.inner.data.spam_classifier.load().as_ref() {\n                    spamfilter::SpamClassifier::FhClassifier {\n                        last_trained_at, ..\n                    }\n                    | spamfilter::SpamClassifier::CcfhClassifier {\n                        last_trained_at, ..\n                    } => now().saturating_sub(*last_trained_at).min(train_frequency),\n                    spamfilter::SpamClassifier::Disabled => train_frequency,\n                };\n\n                queue.schedule(\n                    Instant::now() + Duration::from_secs(next_train),\n                    ActionClass::TrainSpamClassifier,\n                );\n            }\n\n            // OTEL Push Metrics\n            if roles.push_metrics.is_enabled_or_sharded()\n                && let Some(otel) = &server.core.metrics.otel\n            {\n                OtelMetrics::enable_errors();\n                queue.schedule(Instant::now() + otel.interval, ActionClass::OtelMetrics);\n            }\n\n            // Calculate expensive metrics\n            queue.schedule(Instant::now(), ActionClass::CalculateMetrics);\n\n            // Add all ACME renewals to heap\n            for provider in server.core.acme.providers.values() {\n                if roles.renew_acme.is_enabled_for_hash(&provider.id) {\n                    match server.init_acme(provider).await {\n                        Ok(renew_at) => {\n                            queue.schedule(\n                                Instant::now() + renew_at,\n                                ActionClass::Acme(provider.id.clone()),\n                            );\n                        }\n                        Err(err) => {\n                            trc::error!(\n                                err.details(\"Failed to initialize ACME certificate manager.\")\n                            );\n                        }\n                    };\n                }\n            }\n\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n\n            // Enterprise Edition license management\n            #[cfg(feature = \"enterprise\")]\n            if let Some(enterprise) = &server.core.enterprise {\n                queue.schedule(\n                    Instant::now() + enterprise.license.renew_in(),\n                    ActionClass::RenewLicense,\n                );\n\n                if let Some(metrics_store) = enterprise.metrics_store.as_ref() {\n                    queue.schedule(\n                        Instant::now() + metrics_store.interval.time_to_next(),\n                        ActionClass::InternalMetrics,\n                    );\n                }\n\n                if !enterprise.metrics_alerts.is_empty() {\n                    queue.schedule(\n                        Instant::now() + METRIC_ALERTS_INTERVAL,\n                        ActionClass::AlertMetrics,\n                    );\n                }\n            }\n\n            // SPDX-SnippetEnd\n        }\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n        // Metrics history\n        #[cfg(feature = \"enterprise\")]\n        let metrics_history = SharedMetricHistory::default();\n        // SPDX-SnippetEnd\n\n        let mut next_metric_update = Instant::now();\n\n        loop {\n            match tokio::time::timeout(queue.wake_up_time(), rx.recv()).await {\n                Ok(Some(event)) => {\n                    match event {\n                        HousekeeperEvent::ReloadSettings => {\n                            let server = inner.build_server();\n\n                            // Reload OTEL push metrics\n                            match &server.core.metrics.otel {\n                                Some(otel) if !queue.has_action(&ActionClass::OtelMetrics) => {\n                                    OtelMetrics::enable_errors();\n\n                                    queue.schedule(\n                                        Instant::now() + otel.interval,\n                                        ActionClass::OtelMetrics,\n                                    );\n                                }\n                                _ => {}\n                            }\n\n                            // SPDX-SnippetBegin\n                            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                            // SPDX-License-Identifier: LicenseRef-SEL\n                            #[cfg(feature = \"enterprise\")]\n                            if let Some(enterprise) = &server.core.enterprise {\n                                if !queue.has_action(&ActionClass::RenewLicense) {\n                                    queue.schedule(\n                                        Instant::now() + enterprise.license.renew_in(),\n                                        ActionClass::RenewLicense,\n                                    );\n                                }\n\n                                if let Some(metrics_store) = enterprise.metrics_store.as_ref()\n                                    && !queue.has_action(&ActionClass::InternalMetrics)\n                                {\n                                    queue.schedule(\n                                        Instant::now() + metrics_store.interval.time_to_next(),\n                                        ActionClass::InternalMetrics,\n                                    );\n                                }\n\n                                if !enterprise.metrics_alerts.is_empty()\n                                    && !queue.has_action(&ActionClass::AlertMetrics)\n                                {\n                                    queue.schedule(Instant::now(), ActionClass::AlertMetrics);\n                                }\n                            }\n                            // SPDX-SnippetEnd\n\n                            // Reload queue settings\n                            server\n                                .inner\n                                .ipc\n                                .queue_tx\n                                .send(common::ipc::QueueEvent::ReloadSettings)\n                                .await\n                                .ok();\n\n                            // Reload ACME certificates\n                            tokio::spawn(async move {\n                                for provider in server.core.acme.providers.values() {\n                                    match server.init_acme(provider).await {\n                                        Ok(renew_at) => {\n                                            server\n                                                .inner\n                                                .ipc\n                                                .housekeeper_tx\n                                                .send(HousekeeperEvent::AcmeReschedule {\n                                                    provider_id: provider.id.clone(),\n                                                    renew_at: Instant::now() + renew_at,\n                                                })\n                                                .await\n                                                .ok();\n                                        }\n                                        Err(err) => {\n                                            trc::error!(err.details(\n                                                \"Failed to reload ACME certificate manager.\"\n                                            ));\n                                        }\n                                    };\n                                }\n                            });\n                        }\n                        HousekeeperEvent::AcmeReschedule {\n                            provider_id,\n                            renew_at,\n                        } => {\n                            let action = ActionClass::Acme(provider_id);\n                            queue.remove_action(&action);\n                            queue.schedule(renew_at, action);\n                        }\n                        HousekeeperEvent::Purge(purge) => {\n                            let server = inner.build_server();\n                            tokio::spawn(async move {\n                                server.purge(purge, 0).await;\n                            });\n                        }\n                        HousekeeperEvent::Exit => {\n                            trc::event!(\n                                Housekeeper(trc::HousekeeperEvent::Stop),\n                                Reason = \"Shutdown\"\n                            );\n\n                            return;\n                        }\n                    }\n                }\n                Ok(None) => {\n                    trc::event!(\n                        Housekeeper(trc::HousekeeperEvent::Stop),\n                        Reason = \"Channel closed\"\n                    );\n                    return;\n                }\n                Err(_) => {\n                    let server = inner.build_server();\n                    while let Some(event) = queue.pop() {\n                        match event.event {\n                            ActionClass::Acme(provider_id) => {\n                                trc::event!(Housekeeper(trc::HousekeeperEvent::Run), Type = \"acme\");\n\n                                let server = server.clone();\n                                tokio::spawn(async move {\n                                    if let Some(provider) =\n                                        server.core.acme.providers.get(&provider_id)\n                                    {\n                                        trc::event!(\n                                            Acme(trc::AcmeEvent::OrderStart),\n                                            Hostname = provider.domains.as_slice()\n                                        );\n\n                                        let renew_at = match server.renew(provider).await {\n                                            Ok(renew_at) => {\n                                                trc::event!(\n                                                    Acme(trc::AcmeEvent::OrderCompleted),\n                                                    Domain = provider.domains.as_slice(),\n                                                    Expires = trc::Value::Timestamp(\n                                                        now() + renew_at.as_secs()\n                                                    )\n                                                );\n\n                                                renew_at\n                                            }\n                                            Err(err) => {\n                                                trc::error!(\n                                                    err.details(\"Failed to renew certificates.\")\n                                                );\n\n                                                Duration::from_secs(3600)\n                                            }\n                                        };\n\n                                        server\n                                            .cluster_broadcast(BroadcastEvent::ReloadSettings)\n                                            .await;\n\n                                        server\n                                            .inner\n                                            .ipc\n                                            .housekeeper_tx\n                                            .send(HousekeeperEvent::AcmeReschedule {\n                                                provider_id: provider_id.clone(),\n                                                renew_at: Instant::now() + renew_at,\n                                            })\n                                            .await\n                                            .ok();\n                                    }\n                                });\n                            }\n                            ActionClass::Account => {\n                                trc::event!(\n                                    Housekeeper(trc::HousekeeperEvent::Run),\n                                    Type = \"purge_account\"\n                                );\n\n                                let server = server.clone();\n                                queue.schedule(\n                                    Instant::now()\n                                        + server.core.jmap.account_purge_frequency.time_to_next(),\n                                    ActionClass::Account,\n                                );\n                                tokio::spawn(async move {\n                                    server\n                                        .purge(\n                                            PurgeType::Account {\n                                                account_id: None,\n                                                use_roles: true,\n                                            },\n                                            0,\n                                        )\n                                        .await;\n                                });\n                            }\n                            ActionClass::Store(idx) => {\n                                if let Some(schedule) =\n                                    server.core.storage.purge_schedules.get(idx).cloned()\n                                {\n                                    trc::event!(\n                                        Housekeeper(trc::HousekeeperEvent::Run),\n                                        Type = \"purge_store\",\n                                        Id = idx\n                                    );\n\n                                    queue.schedule(\n                                        Instant::now() + schedule.cron.time_to_next(),\n                                        ActionClass::Store(idx),\n                                    );\n\n                                    let server = server.clone();\n                                    tokio::spawn(async move {\n                                        server\n                                            .purge(\n                                                match schedule.store {\n                                                    PurgeStore::Data(store) => {\n                                                        PurgeType::Data(store)\n                                                    }\n                                                    PurgeStore::Blobs { store, blob_store } => {\n                                                        PurgeType::Blobs { store, blob_store }\n                                                    }\n                                                    PurgeStore::Lookup(in_memory_store) => {\n                                                        PurgeType::Lookup {\n                                                            store: in_memory_store,\n                                                            prefix: None,\n                                                        }\n                                                    }\n                                                },\n                                                idx as u32,\n                                            )\n                                            .await;\n                                    });\n                                }\n                            }\n                            ActionClass::OtelMetrics => {\n                                if let Some(otel) = &server.core.metrics.otel {\n                                    trc::event!(\n                                        Housekeeper(trc::HousekeeperEvent::Run),\n                                        Type = \"metrics_report\"\n                                    );\n\n                                    queue.schedule(\n                                        Instant::now() + otel.interval,\n                                        ActionClass::OtelMetrics,\n                                    );\n\n                                    let otel = otel.clone();\n\n                                    // SPDX-SnippetBegin\n                                    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                                    // SPDX-License-Identifier: LicenseRef-SEL\n                                    #[cfg(feature = \"enterprise\")]\n                                    let is_enterprise = server.is_enterprise_edition();\n                                    // SPDX-SnippetEnd\n\n                                    #[cfg(not(feature = \"enterprise\"))]\n                                    let is_enterprise = false;\n\n                                    tokio::spawn(async move {\n                                        otel.push_metrics(is_enterprise, start_time).await;\n                                    });\n                                }\n                            }\n                            ActionClass::CalculateMetrics => {\n                                trc::event!(\n                                    Housekeeper(trc::HousekeeperEvent::Run),\n                                    Type = \"metrics_calculate\"\n                                );\n\n                                // Calculate expensive metrics every 5 minutes\n                                queue.schedule(\n                                    Instant::now() + Duration::from_secs(5 * 60),\n                                    ActionClass::CalculateMetrics,\n                                );\n\n                                let update_other_metrics = if Instant::now() >= next_metric_update {\n                                    next_metric_update =\n                                        Instant::now() + Duration::from_secs(86400);\n                                    true\n                                } else {\n                                    false\n                                };\n\n                                let server = server.clone();\n                                tokio::spawn(async move {\n                                    if server\n                                        .core\n                                        .network\n                                        .roles\n                                        .calculate_metrics\n                                        .is_enabled_or_sharded()\n                                    {\n                                        // SPDX-SnippetBegin\n                                        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                                        // SPDX-License-Identifier: LicenseRef-SEL\n                                        #[cfg(feature = \"enterprise\")]\n                                        if server.is_enterprise_edition() {\n                                            // Obtain queue size\n                                            match server.total_queued_messages().await {\n                                                Ok(total) => {\n                                                    Collector::update_gauge(\n                                                        MetricType::QueueCount,\n                                                        total,\n                                                    );\n                                                }\n                                                Err(err) => {\n                                                    trc::error!(\n                                                        err.details(\"Failed to obtain queue size\")\n                                                    );\n                                                }\n                                            }\n                                        }\n                                        // SPDX-SnippetEnd\n\n                                        if update_other_metrics {\n                                            match server.total_accounts().await {\n                                                Ok(total) => {\n                                                    Collector::update_gauge(\n                                                        MetricType::UserCount,\n                                                        total,\n                                                    );\n                                                }\n                                                Err(err) => {\n                                                    trc::error!(\n                                                        err.details(\n                                                            \"Failed to obtain account count\"\n                                                        )\n                                                    );\n                                                }\n                                            }\n\n                                            match server.total_domains().await {\n                                                Ok(total) => {\n                                                    Collector::update_gauge(\n                                                        MetricType::DomainCount,\n                                                        total,\n                                                    );\n                                                }\n                                                Err(err) => {\n                                                    trc::error!(\n                                                        err.details(\n                                                            \"Failed to obtain domain count\"\n                                                        )\n                                                    );\n                                                }\n                                            }\n                                        }\n                                    }\n\n                                    match tokio::task::spawn_blocking(memory_stats::memory_stats)\n                                        .await\n                                    {\n                                        Ok(Some(stats)) => {\n                                            Collector::update_gauge(\n                                                MetricType::ServerMemory,\n                                                stats.physical_mem as u64,\n                                            );\n                                        }\n                                        Ok(None) => {}\n                                        Err(err) => {\n                                            trc::error!(\n                                                trc::EventType::Server(\n                                                    trc::ServerEvent::ThreadError,\n                                                )\n                                                .reason(err)\n                                                .caused_by(trc::location!())\n                                                .details(\"Join Error\")\n                                            );\n                                        }\n                                    }\n                                });\n                            }\n                            ActionClass::TrainSpamClassifier => {\n                                if server\n                                    .core\n                                    .network\n                                    .roles\n                                    .spam_training\n                                    .is_enabled_or_sharded()\n                                    && let Some(train_frequency) = server\n                                        .core\n                                        .spam\n                                        .classifier\n                                        .as_ref()\n                                        .and_then(|c| c.train_frequency)\n                                {\n                                    trc::event!(\n                                        Housekeeper(trc::HousekeeperEvent::Run),\n                                        Type = \"spam_classifier_train\"\n                                    );\n\n                                    // Schedule next training\n                                    queue.schedule(\n                                        Instant::now() + Duration::from_secs(train_frequency),\n                                        ActionClass::TrainSpamClassifier,\n                                    );\n\n                                    let server = server.clone();\n                                    tokio::spawn(async move {\n                                        if let Err(err) = server.spam_train(false).await {\n                                            trc::error!(\n                                                err.details(\"Failed to train spam classifier\")\n                                            );\n                                        }\n                                    });\n                                }\n                            }\n\n                            // SPDX-SnippetBegin\n                            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                            // SPDX-License-Identifier: LicenseRef-SEL\n                            #[cfg(feature = \"enterprise\")]\n                            ActionClass::InternalMetrics => {\n                                if let Some(metrics_store) = &server\n                                    .core\n                                    .enterprise\n                                    .as_ref()\n                                    .and_then(|e| e.metrics_store.as_ref())\n                                {\n                                    trc::event!(\n                                        Housekeeper(trc::HousekeeperEvent::Run),\n                                        Type = \"metrics_internal\"\n                                    );\n\n                                    queue.schedule(\n                                        Instant::now() + metrics_store.interval.time_to_next(),\n                                        ActionClass::InternalMetrics,\n                                    );\n\n                                    let metrics_store = metrics_store.store.clone();\n                                    let metrics_history = metrics_history.clone();\n                                    let core = server.core.clone();\n                                    tokio::spawn(async move {\n                                        if let Err(err) = metrics_store\n                                            .write_metrics(core, now(), metrics_history)\n                                            .await\n                                        {\n                                            trc::error!(err.details(\"Failed to write metrics\"));\n                                        }\n                                    });\n                                }\n                            }\n\n                            #[cfg(feature = \"enterprise\")]\n                            ActionClass::AlertMetrics => {\n                                trc::event!(\n                                    Housekeeper(trc::HousekeeperEvent::Run),\n                                    Type = \"metrics_alert\"\n                                );\n\n                                let server = server.clone();\n\n                                tokio::spawn(async move {\n                                    if let Some(messages) = server.process_alerts().await {\n                                        for message in messages {\n                                            server\n                                                .send_autogenerated(\n                                                    message.from,\n                                                    message.to.into_iter(),\n                                                    message.body,\n                                                    None,\n                                                    0,\n                                                )\n                                                .await;\n                                        }\n                                    }\n                                });\n                            }\n\n                            #[cfg(feature = \"enterprise\")]\n                            ActionClass::RenewLicense => {\n                                trc::event!(\n                                    Housekeeper(trc::HousekeeperEvent::Run),\n                                    Type = \"renew_license\"\n                                );\n\n                                match server.reload().await {\n                                    Ok(result) => {\n                                        if let Some(new_core) = result.new_core {\n                                            if let Some(enterprise) = &new_core.enterprise {\n                                                let renew_in =\n                                                    if enterprise.license.is_near_expiration() {\n                                                        // Something went wrong during renewal, try again in 1 day or 1 hour,\n                                                        // depending on the time left on the license\n                                                        if enterprise.license.expires_in()\n                                                            < Duration::from_secs(86400)\n                                                        {\n                                                            Duration::from_secs(3600)\n                                                        } else {\n                                                            Duration::from_secs(86400)\n                                                        }\n                                                    } else {\n                                                        enterprise.license.renew_in()\n                                                    };\n\n                                                queue.schedule(\n                                                    Instant::now() + renew_in,\n                                                    ActionClass::RenewLicense,\n                                                );\n                                            }\n\n                                            // Update core\n                                            server.inner.shared_core.store(new_core.into());\n\n                                            server\n                                                .cluster_broadcast(BroadcastEvent::ReloadSettings)\n                                                .await;\n                                        }\n                                    }\n                                    Err(err) => {\n                                        trc::error!(err.details(\"Failed to reload configuration.\"));\n                                    }\n                                }\n                            } // SPDX-SnippetEnd\n                        }\n                    }\n                }\n            }\n        }\n    });\n}\n\npub trait Purge: Sync + Send {\n    fn purge(&self, purge: PurgeType, store_idx: u32) -> impl Future<Output = ()> + Send;\n}\n\nimpl Purge for Server {\n    async fn purge(&self, purge: PurgeType, store_idx: u32) {\n        // Lock task\n        let (lock_type, lock_name) = match &purge {\n            PurgeType::Data(_) => (\n                \"data\",\n                [0u8]\n                    .into_iter()\n                    .chain(store_idx.to_be_bytes().into_iter())\n                    .collect::<Vec<_>>()\n                    .into(),\n            ),\n            PurgeType::Blobs { .. } => (\n                \"blob\",\n                [1u8]\n                    .into_iter()\n                    .chain(store_idx.to_be_bytes().into_iter())\n                    .collect::<Vec<_>>()\n                    .into(),\n            ),\n            PurgeType::Lookup { prefix: None, .. } => (\n                \"in-memory\",\n                [2u8]\n                    .into_iter()\n                    .chain(store_idx.to_be_bytes().into_iter())\n                    .collect::<Vec<_>>()\n                    .into(),\n            ),\n            PurgeType::Lookup { .. } => (\"in-memory-prefix\", None),\n            PurgeType::Account { .. } => (\"account\", None),\n        };\n        if let Some(lock_name) = &lock_name {\n            match self\n                .core\n                .storage\n                .lookup\n                .try_lock(KV_LOCK_HOUSEKEEPER, lock_name, 3600)\n                .await\n            {\n                Ok(true) => (),\n                Ok(false) => {\n                    trc::event!(Purge(PurgeEvent::InProgress), Details = lock_type);\n                    return;\n                }\n                Err(err) => {\n                    trc::error!(err.details(\"Failed to lock task.\").details(lock_type));\n                    return;\n                }\n            }\n        }\n\n        trc::event!(Purge(PurgeEvent::Started), Type = lock_type, Id = store_idx);\n        let time = Instant::now();\n\n        match purge {\n            PurgeType::Data(store) => {\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                let trace_retention = self\n                    .core\n                    .enterprise\n                    .as_ref()\n                    .and_then(|e| e.trace_store.as_ref())\n                    .and_then(|t| t.retention);\n                #[cfg(feature = \"enterprise\")]\n                let metrics_retention = self\n                    .core\n                    .enterprise\n                    .as_ref()\n                    .and_then(|e| e.metrics_store.as_ref())\n                    .and_then(|m| m.retention);\n                // SPDX-SnippetEnd\n\n                if let Err(err) = store.purge_store().await {\n                    trc::error!(err.details(\"Failed to purge data store\"));\n                }\n\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                if let Some(trace_retention) = trace_retention\n                    && let Some(trace_store) = self\n                        .core\n                        .enterprise\n                        .as_ref()\n                        .and_then(|e| e.trace_store.as_ref())\n                    && let Err(err) = trace_store\n                        .store\n                        .purge_spans(trace_retention, self.search_store().into())\n                        .await\n                {\n                    trc::error!(err.details(\"Failed to purge tracing spans\"));\n                }\n\n                #[cfg(feature = \"enterprise\")]\n                if let Some(metrics_retention) = metrics_retention\n                    && let Some(metrics_store) = self\n                        .core\n                        .enterprise\n                        .as_ref()\n                        .and_then(|e| e.metrics_store.as_ref())\n                    && let Err(err) = metrics_store.store.purge_metrics(metrics_retention).await\n                {\n                    trc::error!(err.details(\"Failed to purge metrics\"));\n                }\n                // SPDX-SnippetEnd\n            }\n            PurgeType::Blobs { store, blob_store } => {\n                if let Err(err) = store.purge_blobs(blob_store).await {\n                    trc::error!(err.details(\"Failed to purge blob store\"));\n                }\n            }\n            PurgeType::Lookup { store, prefix } => {\n                if let Some(prefix) = prefix {\n                    if let Err(err) = store.key_delete_prefix(&prefix).await {\n                        trc::error!(\n                            err.details(\"Failed to delete key prefix\")\n                                .ctx(trc::Key::Key, prefix)\n                        );\n                    }\n                } else if let Err(err) = store.purge_in_memory_store().await {\n                    trc::error!(err.details(\"Failed to purge in-memory store\"));\n                }\n            }\n            PurgeType::Account {\n                account_id,\n                use_roles,\n            } => {\n                if let Some(account_id) = account_id {\n                    self.purge_account(account_id).await;\n                } else {\n                    self.purge_accounts(use_roles).await;\n                }\n            }\n        }\n\n        trc::event!(\n            Purge(PurgeEvent::Finished),\n            Type = lock_type,\n            Id = store_idx,\n            Elapsed = time.elapsed()\n        );\n\n        // Remove lock\n        if let Some(lock_name) = &lock_name\n            && let Err(err) = self\n                .in_memory_store()\n                .remove_lock(KV_LOCK_HOUSEKEEPER, lock_name)\n                .await\n        {\n            trc::error!(\n                err.details(\"Failed to delete task lock.\")\n                    .details(lock_type)\n            );\n        }\n    }\n}\n\nimpl Queue {\n    pub fn schedule(&mut self, due: Instant, event: ActionClass) {\n        trc::event!(\n            Housekeeper(trc::HousekeeperEvent::Schedule),\n            Due = trc::Value::Timestamp(\n                now() + due.saturating_duration_since(Instant::now()).as_secs()\n            ),\n            Id = format!(\"{:?}\", event)\n        );\n\n        self.heap.push(Action { due, event });\n    }\n\n    pub fn remove_action(&mut self, event: &ActionClass) {\n        self.heap.retain(|e| &e.event != event);\n    }\n\n    pub fn wake_up_time(&self) -> Duration {\n        self.heap\n            .peek()\n            .map(|e| e.due.saturating_duration_since(Instant::now()))\n            .unwrap_or(LONG_1D_SLUMBER)\n    }\n\n    pub fn pop(&mut self) -> Option<Action> {\n        if self.heap.peek()?.due <= Instant::now() {\n            self.heap.pop()\n        } else {\n            None\n        }\n    }\n\n    pub fn has_action(&self, event: &ActionClass) -> bool {\n        self.heap.iter().any(|e| &e.event == event)\n    }\n}\n\nimpl Ord for Action {\n    fn cmp(&self, other: &Self) -> std::cmp::Ordering {\n        self.due.cmp(&other.due).reverse()\n    }\n}\n\nimpl PartialOrd for Action {\n    fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {\n        Some(self.cmp(other))\n    }\n}\n"
  },
  {
    "path": "crates/services/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse broadcast::publisher::spawn_broadcast_publisher;\nuse common::{\n    Inner,\n    manager::boot::{BootManager, IpcReceivers},\n};\nuse housekeeper::spawn_housekeeper;\nuse state_manager::manager::spawn_push_router;\nuse std::sync::Arc;\nuse task_manager::spawn_task_manager;\n\npub mod broadcast;\npub mod housekeeper;\npub mod state_manager;\npub mod task_manager;\n\npub trait StartServices: Sync + Send {\n    fn start_services(&mut self) -> impl Future<Output = ()> + Send;\n}\n\npub trait SpawnServices {\n    fn spawn_services(&mut self, inner: Arc<Inner>);\n}\n\nimpl StartServices for BootManager {\n    async fn start_services(&mut self) {\n        // Unpack webadmin\n        if let Err(err) = self\n            .inner\n            .data\n            .webadmin\n            .unpack(&self.inner.shared_core.load().storage.blob)\n            .await\n        {\n            trc::event!(\n                Resource(trc::ResourceEvent::Error),\n                Reason = err,\n                Details = \"Failed to unpack webadmin bundle\"\n            );\n        }\n\n        self.ipc_rxs.spawn_services(self.inner.clone());\n    }\n}\n\nimpl SpawnServices for IpcReceivers {\n    fn spawn_services(&mut self, inner: Arc<Inner>) {\n        // Spawn push manager\n        spawn_push_router(inner.clone(), self.push_rx.take().unwrap());\n\n        // Spawn housekeeper\n        spawn_housekeeper(inner.clone(), self.housekeeper_rx.take().unwrap());\n\n        // Spawn broadcast publisher\n        if let Some(event_rx) = self.broadcast_rx.take() {\n            // Spawn broadcast publisher\n            spawn_broadcast_publisher(inner.clone(), event_rx);\n        }\n\n        // Spawn task manager\n        spawn_task_manager(inner);\n    }\n}\n"
  },
  {
    "path": "crates/services/src/state_manager/ece.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\nuse aes_gcm::{Aes128Gcm, Nonce, aead::Aead};\nuse hkdf::Hkdf;\nuse p256::{\n    PublicKey,\n    ecdh::EphemeralSecret,\n    elliptic_curve::{rand_core::OsRng, sec1::ToEncodedPoint},\n};\nuse sha2::Sha256;\nuse store::rand::Rng;\n\n/*\n\n From https://github.com/mozilla/rust-ece (MPL-2.0 license)\n Adapted to use 'aes-gcm' and 'p256' crates instead of 'openssl'.\n\n*/\n\nconst ECE_WEBPUSH_AES128GCM_IKM_INFO_PREFIX: &str = \"WebPush: info\\0\";\nconst ECE_WEBPUSH_AES128GCM_IKM_INFO_LENGTH: usize = 144;\nconst ECE_WEBPUSH_IKM_LENGTH: usize = 32;\nconst ECE_WEBPUSH_PUBLIC_KEY_LENGTH: usize = 65;\nconst ECE_WEBPUSH_DEFAULT_RS: u32 = 4096;\nconst ECE_WEBPUSH_DEFAULT_PADDING_BLOCK_SIZE: usize = 128;\n\nconst ECE_AES128GCM_PAD_SIZE: usize = 1;\nconst ECE_AES128GCM_KEY_INFO: &str = \"Content-Encoding: aes128gcm\\0\";\nconst ECE_AES128GCM_NONCE_INFO: &str = \"Content-Encoding: nonce\\0\";\nconst ECE_AES128GCM_HEADER_LENGTH: usize = 21;\nconst ECE_AES_KEY_LENGTH: usize = 16;\n\nconst ECE_NONCE_LENGTH: usize = 12;\nconst ECE_TAG_LENGTH: usize = 16;\n\npub fn ece_encrypt(\n    p256dh: &[u8],\n    client_auth_secret: &[u8],\n    mut data: &[u8],\n) -> Result<Vec<u8>, String> {\n    let salt = store::rand::rng().random::<[u8; 16]>();\n    let server_secret = EphemeralSecret::random(&mut OsRng);\n    let server_public_key = server_secret.public_key();\n    let server_public_key_bytes = server_public_key.to_encoded_point(false);\n\n    let client_public_key = PublicKey::from_sec1_bytes(p256dh).map_err(|e| e.to_string())?;\n    let shared_secret = server_secret.diffie_hellman(&client_public_key);\n\n    let ikm_info = generate_info(p256dh, server_public_key_bytes.as_bytes());\n    let ikm = hkdf_sha256(\n        client_auth_secret,\n        &shared_secret.raw_secret_bytes()[..],\n        &ikm_info,\n        ECE_WEBPUSH_IKM_LENGTH,\n    )?;\n    let key = hkdf_sha256(\n        &salt,\n        &ikm,\n        ECE_AES128GCM_KEY_INFO.as_bytes(),\n        ECE_AES_KEY_LENGTH,\n    )?;\n    let nonce = hkdf_sha256(\n        &salt,\n        &ikm,\n        ECE_AES128GCM_NONCE_INFO.as_bytes(),\n        ECE_NONCE_LENGTH,\n    )?;\n\n    // Calculate pad length\n    let mut pad_length = ECE_WEBPUSH_DEFAULT_PADDING_BLOCK_SIZE\n        - (data.len() % ECE_WEBPUSH_DEFAULT_PADDING_BLOCK_SIZE);\n    if pad_length < ECE_AES128GCM_PAD_SIZE {\n        pad_length += ECE_WEBPUSH_DEFAULT_PADDING_BLOCK_SIZE;\n    }\n\n    // Split into records\n    let rs = ECE_WEBPUSH_DEFAULT_RS as usize - ECE_TAG_LENGTH;\n    let mut min_num_records = data.len() / (rs - 1);\n    if !data.len().is_multiple_of(rs - 1) {\n        min_num_records += 1;\n    }\n    let mut pad_length = std::cmp::max(pad_length, min_num_records);\n    let total_size = data.len() + pad_length;\n    let mut num_records = total_size / rs;\n    let size_of_final_record = total_size % rs;\n    if size_of_final_record > 0 {\n        num_records += 1;\n    }\n    let data_per_record = data.len() / num_records;\n    let mut extra_data = data.len() % num_records;\n    if size_of_final_record > 0 && data_per_record > size_of_final_record - 1 {\n        extra_data += data_per_record - (size_of_final_record - 1)\n    }\n    let mut sequence_number = 0;\n    let mut plain_text =\n        Vec::with_capacity(data_per_record + ECE_WEBPUSH_DEFAULT_PADDING_BLOCK_SIZE);\n\n    // Write header\n    let key_id = server_public_key_bytes.as_bytes();\n    debug_assert_eq!(key_id.len(), ECE_WEBPUSH_PUBLIC_KEY_LENGTH);\n    let mut output = Vec::with_capacity(\n        ECE_AES128GCM_HEADER_LENGTH + key_id.len() + total_size + num_records * ECE_TAG_LENGTH,\n    );\n    output.extend_from_slice(&salt);\n    output.extend_from_slice(&ECE_WEBPUSH_DEFAULT_RS.to_be_bytes());\n    output.push(key_id.len() as u8);\n    output.extend_from_slice(key_id);\n\n    loop {\n        let records_remaining = num_records - sequence_number;\n        if records_remaining == 0 {\n            break;\n        }\n        let mut data_share = data_per_record;\n        if data_share > data.len() {\n            data_share = data.len();\n        } else if extra_data > 0 {\n            let mut extra_share = extra_data / (records_remaining - 1);\n            if !extra_data.is_multiple_of(records_remaining - 1) {\n                extra_share += 1;\n            }\n            data_share += extra_share;\n            extra_data -= extra_share;\n        }\n\n        let cur_data = &data[0..data_share];\n        data = &data[data_share..];\n        let padding = std::cmp::min(pad_length, rs - data_share);\n        pad_length -= padding;\n        let cur_sequence_number = sequence_number;\n        sequence_number += 1;\n\n        let padded_plaintext_len = cur_data.len() + padding;\n\n        plain_text.extend_from_slice(cur_data);\n        plain_text.push(if sequence_number == num_records { 2 } else { 1 });\n        plain_text.resize(padded_plaintext_len, 0);\n\n        output.extend_from_slice(&aes_gcm_128_encrypt(\n            &key,\n            &generate_iv(&nonce, cur_sequence_number),\n            &plain_text,\n        )?);\n        plain_text.clear();\n    }\n\n    Ok(output)\n}\n\nfn hkdf_sha256(salt: &[u8], secret: &[u8], info: &[u8], len: usize) -> Result<Vec<u8>, String> {\n    let (_, hk) = Hkdf::<Sha256>::extract(Some(salt), secret);\n    let mut okm = vec![0u8; len];\n    hk.expand(info, &mut okm).map_err(|e| e.to_string())?;\n    Ok(okm)\n}\n\n// TODO: Remove allow deprecated when aes-gcm 0.10 is updated\n#[allow(deprecated)]\nfn aes_gcm_128_encrypt(key: &[u8], nonce: &[u8], data: &[u8]) -> Result<Vec<u8>, String> {\n    <Aes128Gcm as aes_gcm::KeyInit>::new(\n        &sha2::digest::generic_array::GenericArray::clone_from_slice(key),\n    )\n    .encrypt(Nonce::from_slice(nonce), data)\n    .map_err(|e| e.to_string())\n}\n\nfn generate_info(\n    client_public_key: &[u8],\n    server_public_key: &[u8],\n) -> [u8; ECE_WEBPUSH_AES128GCM_IKM_INFO_LENGTH] {\n    let mut info = [0u8; ECE_WEBPUSH_AES128GCM_IKM_INFO_LENGTH];\n    let prefix = ECE_WEBPUSH_AES128GCM_IKM_INFO_PREFIX.as_bytes();\n    let mut offset = prefix.len();\n    info[0..offset].copy_from_slice(prefix);\n    info[offset..offset + ECE_WEBPUSH_PUBLIC_KEY_LENGTH].copy_from_slice(client_public_key);\n    offset += ECE_WEBPUSH_PUBLIC_KEY_LENGTH;\n    info[offset..].copy_from_slice(server_public_key);\n    info\n}\n\npub fn generate_iv(nonce: &[u8], counter: usize) -> [u8; ECE_NONCE_LENGTH] {\n    let mut iv = [0u8; ECE_NONCE_LENGTH];\n    let offset = ECE_NONCE_LENGTH - 8;\n    iv[0..offset].copy_from_slice(&nonce[0..offset]);\n    let mask = u64::from_be_bytes((&nonce[offset..]).try_into().unwrap());\n    iv[offset..].copy_from_slice(&(mask ^ (counter as u64)).to_be_bytes());\n    iv\n}\n"
  },
  {
    "path": "crates/services/src/state_manager/http.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Event, ece::ece_encrypt};\nuse crate::state_manager::PushRegistration;\nuse base64::Engine;\nuse calcard::jscalendar::JSCalendarDateTime;\nuse common::ipc::PushNotification;\nuse email::push::PushSubscription;\nuse jmap_proto::{\n    response::status::{EmailPushObject, PushObject},\n    types::state::State,\n};\nuse reqwest::header::{CONTENT_ENCODING, CONTENT_TYPE};\nuse std::time::{Duration, Instant};\nuse tokio::sync::mpsc;\nuse trc::PushSubscriptionEvent;\nuse types::{id::Id, type_state::DataType};\nuse utils::map::vec_map::VecMap;\n\nimpl PushRegistration {\n    pub fn send(&mut self, id: Id, push_tx: mpsc::Sender<Event>, push_timeout: Duration) {\n        let server = self.server.clone();\n        let notifications = std::mem::take(&mut self.notifications);\n\n        self.in_flight = true;\n        self.last_request = Instant::now();\n\n        tokio::spawn(async move {\n            let mut changed: VecMap<Id, VecMap<DataType, State>> = VecMap::new();\n            let mut objects = Vec::with_capacity(notifications.len());\n            for notification in &notifications {\n                match notification {\n                    PushNotification::StateChange(state_change) => {\n                        for type_state in state_change.types {\n                            changed\n                                .get_mut_or_insert(state_change.account_id.into())\n                                .set(type_state, (state_change.change_id).into());\n                        }\n                    }\n                    PushNotification::CalendarAlert(calendar_alert) => {\n                        objects.push(PushObject::CalendarAlert {\n                            account_id: calendar_alert.account_id.into(),\n                            calendar_event_id: calendar_alert.event_id.into(),\n                            uid: calendar_alert.uid.clone(),\n                            recurrence_id: calendar_alert.recurrence_id.map(|timestamp| {\n                                JSCalendarDateTime::new(timestamp, true).to_rfc3339()\n                            }),\n                            alert_id: calendar_alert.alert_id.clone(),\n                        });\n                    }\n                    PushNotification::EmailPush(email_push) => {\n                        objects.push(PushObject::EmailPush {\n                            account_id: email_push.account_id.into(),\n                            email: EmailPushObject {\n                                subject: Default::default(),\n                            },\n                        });\n                    }\n                }\n            }\n\n            let response = if !objects.is_empty() {\n                if changed.is_empty() {\n                    objects.push(PushObject::StateChange { changed });\n                }\n                if objects.len() > 1 {\n                    PushObject::Group { entries: objects }\n                } else {\n                    objects.into_iter().next().unwrap()\n                }\n            } else {\n                PushObject::StateChange { changed }\n            };\n\n            push_tx\n                .send(\n                    if http_request(\n                        &server,\n                        serde_json::to_string(&response).unwrap(),\n                        push_timeout,\n                    )\n                    .await\n                    {\n                        Event::DeliverySuccess { id }\n                    } else {\n                        Event::DeliveryFailure { id, notifications }\n                    },\n                )\n                .await\n                .ok();\n        });\n    }\n}\n\npub(crate) async fn http_request(\n    details: &PushSubscription,\n    mut body: String,\n    push_timeout: Duration,\n) -> bool {\n    let client_builder = reqwest::Client::builder().timeout(push_timeout);\n\n    #[cfg(feature = \"test_mode\")]\n    let client_builder = client_builder.danger_accept_invalid_certs(true);\n\n    let mut client = client_builder\n        .build()\n        .unwrap_or_default()\n        .post(details.url.as_str())\n        .header(CONTENT_TYPE, \"application/json\")\n        .header(\"TTL\", \"86400\");\n\n    if let Some(keys) = &details.keys {\n        match ece_encrypt(&keys.p256dh, &keys.auth, body.as_bytes())\n            .map(|b| base64::engine::general_purpose::URL_SAFE.encode(b))\n        {\n            Ok(body_) => {\n                body = body_;\n                client = client.header(CONTENT_ENCODING, \"aes128gcm\");\n            }\n            Err(err) => {\n                // Do not reattempt if encryption fails.\n\n                trc::event!(\n                    PushSubscription(PushSubscriptionEvent::Error),\n                    Details = \"Failed to encrypt push subscription\",\n                    Url = details.url.to_string(),\n                    Reason = err\n                );\n                return true;\n            }\n        }\n    }\n\n    match client.body(body).send().await {\n        Ok(response) => {\n            if response.status().is_success() {\n                trc::event!(\n                    PushSubscription(PushSubscriptionEvent::Success),\n                    Url = details.url.to_string()\n                );\n\n                true\n            } else {\n                trc::event!(\n                    PushSubscription(PushSubscriptionEvent::Error),\n                    Details = \"HTTP POST failed\",\n                    Url = details.url.to_string(),\n                    Code = response.status().as_u16(),\n                );\n\n                false\n            }\n        }\n        Err(err) => {\n            trc::event!(\n                PushSubscription(PushSubscriptionEvent::Error),\n                Details = \"HTTP POST failed\",\n                Url = details.url.to_string(),\n                Reason = err.to_string()\n            );\n\n            false\n        }\n    }\n}\n"
  },
  {
    "path": "crates/services/src/state_manager/manager.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Event, PURGE_EVERY, SEND_TIMEOUT, push::spawn_push_manager};\nuse crate::state_manager::IpcSubscriber;\nuse common::{\n    Inner,\n    ipc::{BroadcastEvent, PushEvent},\n};\nuse std::{sync::Arc, time::Instant};\nuse store::ahash::AHashMap;\nuse tokio::sync::mpsc;\nuse trc::ServerEvent;\n\n#[derive(Default)]\nstruct Subscriber {\n    ipc: Vec<IpcSubscriber>,\n    is_push: bool,\n}\n\n#[allow(clippy::unwrap_or_default)]\npub fn spawn_push_router(inner: Arc<Inner>, mut change_rx: mpsc::Receiver<PushEvent>) {\n    let push_tx = spawn_push_manager(inner.clone());\n\n    tokio::spawn(async move {\n        let mut subscribers: AHashMap<u32, Subscriber> = AHashMap::default();\n        let mut last_purge = Instant::now();\n\n        while let Some(event) = change_rx.recv().await {\n            let mut purge_needed = last_purge.elapsed() >= PURGE_EVERY;\n\n            match event {\n                PushEvent::Stop => {\n                    if push_tx.send(Event::Reset).await.is_err() {\n                        trc::event!(\n                            Server(ServerEvent::ThreadError),\n                            Details = \"Error sending push reset.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n                    break;\n                }\n\n                PushEvent::Subscribe {\n                    account_ids,\n                    types,\n                    tx,\n                } => {\n                    for account_id in account_ids {\n                        subscribers\n                            .entry(account_id)\n                            .or_default()\n                            .ipc\n                            .push(IpcSubscriber {\n                                types,\n                                tx: tx.clone(),\n                            });\n                    }\n                }\n\n                PushEvent::PushServerRegister { activate, expired } => {\n                    for account_id in activate {\n                        subscribers.entry(account_id).or_default().is_push = true;\n                    }\n\n                    for account_id in expired {\n                        let mut remove_account = false;\n                        if let Some(subscriber_list) = subscribers.get_mut(&account_id) {\n                            subscriber_list.is_push = false;\n                            remove_account = subscriber_list.ipc.is_empty();\n                        }\n                        if remove_account {\n                            subscribers.remove(&account_id);\n                        }\n                    }\n                }\n\n                PushEvent::Publish {\n                    notification,\n                    broadcast,\n                } => {\n                    // Publish event to cluster\n                    if broadcast\n                        && let Some(broadcast_tx) = &inner.ipc.broadcast_tx.clone()\n                        && broadcast_tx\n                            .send(BroadcastEvent::PushNotification(notification.clone()))\n                            .await\n                            .is_err()\n                    {\n                        trc::event!(\n                            Server(trc::ServerEvent::ThreadError),\n                            Details = \"Error sending broadcast event.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n\n                    let account_id = notification.account_id();\n                    if let Some(subscribers) = subscribers.get(&account_id) {\n                        for subscriber in &subscribers.ipc {\n                            if let Some(notification) = notification.filter_types(&subscriber.types)\n                            {\n                                if subscriber.is_valid() {\n                                    let subscriber_tx = subscriber.tx.clone();\n\n                                    tokio::spawn(async move {\n                                        // Timeout after 500ms in case there is a blocked client\n                                        if subscriber_tx\n                                            .send_timeout(notification, SEND_TIMEOUT)\n                                            .await\n                                            .is_err()\n                                        {\n                                            trc::event!(\n                                                Server(ServerEvent::ThreadError),\n                                                Details =\n                                                    \"Error sending state change to subscriber.\",\n                                                CausedBy = trc::location!()\n                                            );\n                                        }\n                                    });\n                                } else {\n                                    purge_needed = true;\n                                }\n                            }\n                        }\n\n                        if subscribers.is_push\n                            && push_tx.send(Event::Push { notification }).await.is_err()\n                        {\n                            trc::event!(\n                                Server(ServerEvent::ThreadError),\n                                Details = \"Error sending push updates.\",\n                                CausedBy = trc::location!()\n                            );\n                        }\n                    }\n                }\n\n                PushEvent::PushServerUpdate {\n                    account_id,\n                    broadcast,\n                } => {\n                    // Publish event to cluster\n                    if broadcast\n                        && let Some(broadcast_tx) = &inner.ipc.broadcast_tx.clone()\n                        && broadcast_tx\n                            .send(BroadcastEvent::ReloadPushServers(account_id))\n                            .await\n                            .is_err()\n                    {\n                        trc::event!(\n                            Server(trc::ServerEvent::ThreadError),\n                            Details = \"Error sending broadcast event.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n\n                    // Notify push manager\n                    if push_tx.send(Event::Update { account_id }).await.is_err() {\n                        trc::event!(\n                            Server(ServerEvent::ThreadError),\n                            Details = \"Error sending push updates.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n                }\n            }\n\n            if purge_needed {\n                let mut remove_account_ids = Vec::new();\n\n                for (account_id, subscribers) in &mut subscribers {\n                    subscribers.ipc.retain(|subscriber| subscriber.is_valid());\n\n                    if subscribers.ipc.is_empty() && !subscribers.is_push {\n                        remove_account_ids.push(*account_id);\n                    }\n                }\n\n                for remove_account_id in remove_account_ids {\n                    subscribers.remove(&remove_account_id);\n                }\n\n                last_purge = Instant::now();\n            }\n        }\n    });\n}\n"
  },
  {
    "path": "crates/services/src/state_manager/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod ece;\npub mod http;\npub mod manager;\npub mod push;\n\nuse common::ipc::PushNotification;\nuse email::push::PushSubscription;\nuse std::{\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse tokio::sync::mpsc;\nuse types::{id::Id, type_state::DataType};\nuse utils::map::bitmap::Bitmap;\n\nconst PURGE_EVERY: Duration = Duration::from_secs(3600);\nconst SEND_TIMEOUT: Duration = Duration::from_millis(500);\n\n#[derive(Debug)]\nstruct IpcSubscriber {\n    types: Bitmap<DataType>,\n    tx: mpsc::Sender<PushNotification>,\n}\n\n#[derive(Debug)]\npub struct PushRegistration {\n    server: Arc<PushSubscription>,\n    member_account_ids: Vec<u32>,\n    num_attempts: u32,\n    last_request: Instant,\n    notifications: Vec<PushNotification>,\n    in_flight: bool,\n}\n\n#[derive(Debug)]\npub enum Event {\n    Push {\n        notification: PushNotification,\n    },\n    Update {\n        account_id: u32,\n    },\n    DeliverySuccess {\n        id: Id,\n    },\n    DeliveryFailure {\n        id: Id,\n        notifications: Vec<PushNotification>,\n    },\n    Reset,\n}\n\nimpl IpcSubscriber {\n    fn is_valid(&self) -> bool {\n        !self.tx.is_closed()\n    }\n}\n"
  },
  {
    "path": "crates/services/src/state_manager/push.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Event, http::http_request};\nuse crate::state_manager::PushRegistration;\nuse common::{\n    IPC_CHANNEL_BUFFER, Inner, LONG_1Y_SLUMBER, Server,\n    core::BuildServer,\n    ipc::{PushEvent, PushNotification},\n};\nuse email::push::PushSubscriptions;\nuse std::{\n    collections::hash_map::Entry,\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse store::{\n    ValueKey,\n    ahash::{AHashMap, AHashSet},\n    write::{AlignedBytes, Archive, now},\n};\nuse tokio::sync::mpsc;\nuse trc::{AddContext, PushSubscriptionEvent, ServerEvent};\nuse types::{collection::Collection, field::PrincipalField, id::Id};\n\npub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {\n    let (push_tx_, mut push_rx) = mpsc::channel::<Event>(IPC_CHANNEL_BUFFER);\n    let push_tx = push_tx_.clone();\n\n    tokio::spawn(async move {});\n\n    tokio::spawn(async move {\n        let mut push_servers: AHashMap<Id, PushRegistration> = AHashMap::default();\n        let mut account_push_ids: AHashMap<u32, AHashSet<Id>> = AHashMap::default();\n        let mut last_verify: AHashMap<u32, Instant> = AHashMap::default();\n        let mut last_retry = Instant::now();\n        let mut retry_timeout = LONG_1Y_SLUMBER;\n        let mut retry_ids = AHashSet::default();\n\n        // Load active subscriptions on startup\n        {\n            let server = inner.build_server();\n\n            match server\n                .document_ids(\n                    u32::MAX,\n                    Collection::Principal,\n                    PrincipalField::PushSubscriptions,\n                )\n                .await\n            {\n                Ok(account_ids) => {\n                    for account_id in account_ids {\n                        if server\n                            .core\n                            .network\n                            .roles\n                            .push_notifications\n                            .is_enabled_for_integer(account_id)\n                        {\n                            // Load push subscriptions for account\n                            let (subscriptions, member_account_ids) =\n                                match load_push_subscriptions(&server, account_id).await {\n                                    Ok(subscriptions) => subscriptions,\n                                    Err(err) => {\n                                        trc::error!(err.caused_by(trc::location!()));\n                                        continue;\n                                    }\n                                };\n                            let current_time = now();\n                            for subscription in subscriptions\n                                .subscriptions\n                                .into_iter()\n                                .filter(|s| s.verified && s.expires > current_time)\n                            {\n                                let id = Id::from_parts(subscription.id, account_id);\n                                let subscription = Arc::new(subscription);\n\n                                for account_id in &member_account_ids {\n                                    account_push_ids.entry(*account_id).or_default().insert(id);\n                                }\n                                push_servers.insert(\n                                    id,\n                                    PushRegistration {\n                                        member_account_ids: member_account_ids.clone(),\n                                        num_attempts: 0,\n                                        last_request: Instant::now()\n                                            - (server.core.jmap.push_throttle\n                                                + Duration::from_millis(1)),\n                                        notifications: Vec::new(),\n                                        server: subscription.clone(),\n                                        in_flight: false,\n                                    },\n                                );\n                            }\n                        }\n                    }\n                }\n                Err(err) => {\n                    trc::error!(err.caused_by(trc::location!()));\n                }\n            }\n\n            // Subscribe to push events\n            if !account_push_ids.is_empty()\n                && server\n                    .inner\n                    .ipc\n                    .push_tx\n                    .clone()\n                    .send(PushEvent::PushServerRegister {\n                        activate: account_push_ids.keys().copied().collect(),\n                        expired: vec![],\n                    })\n                    .await\n                    .is_err()\n            {\n                trc::event!(\n                    Server(ServerEvent::ThreadError),\n                    Details = \"Error sending state change.\",\n                    CausedBy = trc::location!()\n                );\n            }\n        }\n\n        loop {\n            // Wait for the next event or timeout\n            let event_or_timeout = tokio::time::timeout(retry_timeout, push_rx.recv()).await;\n\n            // Load settings\n            let server = inner.build_server();\n            let push_attempt_interval = server.core.jmap.push_attempt_interval;\n            let push_attempts_max = server.core.jmap.push_attempts_max;\n            let push_retry_interval = server.core.jmap.push_retry_interval;\n            let push_timeout = server.core.jmap.push_timeout;\n            let push_verify_timeout = server.core.jmap.push_verify_timeout;\n            let push_throttle = server.core.jmap.push_throttle;\n\n            match event_or_timeout {\n                Ok(Some(event)) => match event {\n                    Event::Update { account_id } => {\n                        if !server\n                            .core\n                            .network\n                            .roles\n                            .push_notifications\n                            .is_enabled_for_integer(account_id)\n                        {\n                            continue;\n                        }\n\n                        // Load push subscriptions for account\n                        let (subscriptions, member_account_ids) =\n                            match load_push_subscriptions(&server, account_id).await {\n                                Ok(subscriptions) => subscriptions,\n                                Err(err) => {\n                                    trc::error!(err.caused_by(trc::location!()));\n                                    continue;\n                                }\n                            };\n                        let old_account_push_ids = account_push_ids\n                            .remove(&account_id)\n                            .filter(|v| !v.is_empty());\n\n                        // Process subscriptions\n                        let current_time = now();\n                        for subscription in subscriptions\n                            .subscriptions\n                            .into_iter()\n                            .filter(|s| s.expires > current_time)\n                        {\n                            let id = Id::from_parts(subscription.id, account_id);\n                            let subscription = Arc::new(subscription);\n\n                            if subscription.verified {\n                                for account_id in &member_account_ids {\n                                    account_push_ids.entry(*account_id).or_default().insert(id);\n                                }\n\n                                match push_servers.entry(id) {\n                                    Entry::Occupied(mut entry) => {\n                                        // Update existing subscription\n                                        let entry = entry.get_mut();\n                                        entry.server = subscription.clone();\n                                        entry.member_account_ids = member_account_ids.clone();\n                                    }\n                                    Entry::Vacant(entry) => {\n                                        entry.insert(PushRegistration {\n                                            member_account_ids: member_account_ids.clone(),\n                                            num_attempts: 0,\n                                            last_request: Instant::now()\n                                                - (push_throttle + Duration::from_millis(1)),\n                                            notifications: Vec::new(),\n                                            server: subscription.clone(),\n                                            in_flight: false,\n                                        });\n                                    }\n                                }\n                            } else {\n                                let current_time = Instant::now();\n\n                                #[cfg(feature = \"test_mode\")]\n                                if subscription.url.contains(\"skip_checks\") {\n                                    last_verify.insert(\n                                        account_id,\n                                        current_time\n                                            - (push_verify_timeout + Duration::from_millis(1)),\n                                    );\n                                }\n\n                                if last_verify\n                                    .get(&account_id)\n                                    .map(|last_verify| {\n                                        current_time - *last_verify > push_verify_timeout\n                                    })\n                                    .unwrap_or(true)\n                                {\n                                    tokio::spawn(async move {\n                                        http_request(\n                                            &subscription,\n                                            format!(\n                                                concat!(\n                                                    \"{{\\\"@type\\\":\\\"PushVerification\\\",\",\n                                                    \"\\\"pushSubscriptionId\\\":\\\"{}\\\",\",\n                                                    \"\\\"verificationCode\\\":\\\"{}\\\"}}\"\n                                                ),\n                                                Id::from(subscription.id),\n                                                subscription.verification_code\n                                            ),\n                                            push_timeout,\n                                        )\n                                        .await;\n                                    });\n\n                                    last_verify.insert(account_id, current_time);\n                                } else {\n                                    trc::event!(\n                                        PushSubscription(PushSubscriptionEvent::Error),\n                                        Details = \"Failed to verify push subscription\",\n                                        Url = subscription.url.clone(),\n                                        AccountId = account_id,\n                                        Reason = \"Too many requests\"\n                                    );\n\n                                    continue;\n                                }\n                            }\n                        }\n\n                        // Update subscriptions\n                        let mut remove_push_ids = AHashSet::new();\n                        let mut active_account_ids = Vec::new();\n                        let mut inactive_account_ids = Vec::new();\n                        match (old_account_push_ids, account_push_ids.get(&account_id)) {\n                            (Some(old), Some(current)) if &old != current => {\n                                for id in old.difference(current) {\n                                    remove_push_ids.insert(*id);\n                                }\n                                active_account_ids = member_account_ids;\n                            }\n                            (Some(old), None) => {\n                                remove_push_ids = old;\n                            }\n                            (None, Some(_)) => {\n                                active_account_ids = member_account_ids;\n                            }\n                            _ => {}\n                        }\n\n                        // Update push server registrations\n                        if !remove_push_ids.is_empty() {\n                            for id in remove_push_ids {\n                                if let Some(subscription) = push_servers.remove(&id) {\n                                    for account_id in &subscription.member_account_ids {\n                                        if let Some(ids) = account_push_ids.get_mut(account_id) {\n                                            ids.remove(&id);\n                                            if ids.is_empty() {\n                                                account_push_ids.remove(account_id);\n                                                inactive_account_ids.push(*account_id);\n                                            }\n                                        }\n                                    }\n                                }\n                            }\n                        }\n                        if (!active_account_ids.is_empty() || !inactive_account_ids.is_empty())\n                            && server\n                                .inner\n                                .ipc\n                                .push_tx\n                                .clone()\n                                .send(PushEvent::PushServerRegister {\n                                    activate: active_account_ids,\n                                    expired: inactive_account_ids,\n                                })\n                                .await\n                                .is_err()\n                        {\n                            trc::event!(\n                                Server(ServerEvent::ThreadError),\n                                Details = \"Error sending state change.\",\n                                CausedBy = trc::location!()\n                            );\n                        }\n                    }\n                    Event::Push { notification } => {\n                        let account_id = notification.account_id();\n                        if let Some(ids) = account_push_ids.get_mut(&account_id) {\n                            let current_time = now();\n                            let mut remove_ids = Vec::new();\n\n                            for id in ids.iter() {\n                                if let Some(subscription) = push_servers.get_mut(id) {\n                                    if subscription.server.expires > current_time {\n                                        if let Some(mut notification) =\n                                            notification.filter_types(&subscription.server.types)\n                                        {\n                                            // Build email push notification\n                                            if let PushNotification::EmailPush(email_push) =\n                                                &notification\n                                            {\n                                                if let Some(_email_push) = subscription\n                                                    .server\n                                                    .email_push\n                                                    .iter()\n                                                    .find(|ep| ep.account_id == account_id)\n                                                {\n                                                    // TODO: Apply filters once RFC is finalized\n                                                } else {\n                                                    notification = PushNotification::StateChange(\n                                                        email_push.to_state_change(),\n                                                    );\n                                                }\n                                            }\n\n                                            subscription.notifications.push(notification);\n                                            let last_request = subscription.last_request.elapsed();\n\n                                            if !subscription.in_flight\n                                                && ((subscription.num_attempts == 0\n                                                    && last_request > push_throttle)\n                                                    || ((1..push_attempts_max)\n                                                        .contains(&subscription.num_attempts)\n                                                        && last_request > push_attempt_interval))\n                                            {\n                                                subscription.send(\n                                                    *id,\n                                                    push_tx.clone(),\n                                                    push_timeout,\n                                                );\n                                                retry_ids.remove(id);\n                                            } else {\n                                                retry_ids.insert(*id);\n                                            }\n                                        }\n                                    } else {\n                                        push_servers.remove(id);\n                                    }\n                                } else {\n                                    remove_ids.push(*id);\n                                }\n                            }\n\n                            if !remove_ids.is_empty() {\n                                for remove_id in remove_ids {\n                                    ids.remove(&remove_id);\n                                }\n                                if ids.is_empty() {\n                                    account_push_ids.remove(&account_id);\n                                    if server\n                                        .inner\n                                        .ipc\n                                        .push_tx\n                                        .clone()\n                                        .send(PushEvent::PushServerRegister {\n                                            activate: vec![],\n                                            expired: vec![account_id],\n                                        })\n                                        .await\n                                        .is_err()\n                                    {\n                                        trc::event!(\n                                            Server(ServerEvent::ThreadError),\n                                            Details = \"Error sending state change.\",\n                                            CausedBy = trc::location!()\n                                        );\n                                    }\n                                }\n                            }\n                        }\n                    }\n                    Event::Reset => {\n                        push_servers.clear();\n                        account_push_ids.clear();\n                    }\n                    Event::DeliverySuccess { id } => {\n                        if let Some(subscription) = push_servers.get_mut(&id) {\n                            subscription.num_attempts = 0;\n                            subscription.in_flight = false;\n                            retry_ids.remove(&id);\n                        }\n                    }\n                    Event::DeliveryFailure { id, notifications } => {\n                        if let Some(subscription) = push_servers.get_mut(&id) {\n                            subscription.last_request = Instant::now();\n                            subscription.num_attempts += 1;\n                            subscription.notifications.extend(notifications);\n                            subscription.in_flight = false;\n                            retry_ids.insert(id);\n                        }\n                    }\n                },\n                Ok(None) => {\n                    break;\n                }\n                Err(_) => (),\n            }\n\n            retry_timeout = if !retry_ids.is_empty() {\n                let last_retry_elapsed = last_retry.elapsed();\n\n                if last_retry_elapsed >= push_retry_interval {\n                    let mut remove_ids = Vec::with_capacity(retry_ids.len());\n\n                    for retry_id in &retry_ids {\n                        if let Some(subscription) = push_servers.get_mut(retry_id) {\n                            let last_request = subscription.last_request.elapsed();\n\n                            if !subscription.in_flight\n                                && ((subscription.num_attempts == 0\n                                    && last_request >= push_throttle)\n                                    || (subscription.num_attempts > 0\n                                        && last_request >= push_attempt_interval))\n                            {\n                                if subscription.num_attempts < push_attempts_max {\n                                    subscription.send(*retry_id, push_tx.clone(), push_timeout);\n                                } else {\n                                    trc::event!(\n                                        PushSubscription(PushSubscriptionEvent::Error),\n                                        Details = \"Failed to deliver push subscription\",\n                                        Url = subscription.server.url.clone(),\n                                        Reason = \"Too many failed attempts\"\n                                    );\n\n                                    subscription.notifications.clear();\n                                    subscription.num_attempts = 0;\n                                }\n                                remove_ids.push(*retry_id);\n                            }\n                        } else {\n                            remove_ids.push(*retry_id);\n                        }\n                    }\n\n                    if remove_ids.len() < retry_ids.len() {\n                        for remove_id in remove_ids {\n                            retry_ids.remove(&remove_id);\n                        }\n                        last_retry = Instant::now();\n                        push_retry_interval\n                    } else {\n                        retry_ids.clear();\n                        LONG_1Y_SLUMBER\n                    }\n                } else {\n                    push_retry_interval - last_retry_elapsed\n                }\n            } else {\n                LONG_1Y_SLUMBER\n            };\n        }\n    });\n\n    push_tx_\n}\n\nasync fn load_push_subscriptions(\n    server: &Server,\n    account_id: u32,\n) -> trc::Result<(PushSubscriptions, Vec<u32>)> {\n    let member_of = server\n        .get_access_token(account_id)\n        .await\n        .caused_by(trc::location!())?\n        .member_ids()\n        .collect::<Vec<_>>();\n\n    if let Some(push_subscriptions) = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n            account_id,\n            Collection::Principal,\n            0,\n            PrincipalField::PushSubscriptions,\n        ))\n        .await?\n    {\n        push_subscriptions\n            .deserialize::<PushSubscriptions>()\n            .map(|push_subscriptions| (push_subscriptions, member_of))\n            .caused_by(trc::location!())\n    } else {\n        Ok((PushSubscriptions::default(), member_of))\n    }\n}\n"
  },
  {
    "path": "crates/services/src/task_manager/alarm.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse calcard::{\n    common::timezone::Tz,\n    icalendar::{ArchivedICalendarParameterName, ArchivedICalendarProperty, ICalendarProperty},\n};\nuse chrono::{DateTime, Locale};\nuse common::{\n    DEFAULT_LOGO_BASE64, Server,\n    auth::AccessToken,\n    config::groupware::CalendarTemplateVariable,\n    i18n,\n    ipc::{CalendarAlert, PushNotification},\n    listener::{ServerInstance, stream::NullIo},\n};\nuse directory::Permission;\nuse groupware::calendar::{\n    ArchivedCalendarEvent, CalendarEvent,\n    alarm::{CalendarAlarm, CalendarAlarmType},\n};\nuse mail_builder::{\n    MessageBuilder,\n    headers::{HeaderType, content_type::ContentType},\n    mime::{BodyPart, MimePart},\n};\nuse mail_parser::decoders::html::html_to_text;\nuse smtp::core::{Session, SessionData};\nuse smtp_proto::{MailFrom, RcptTo};\nuse std::{str::FromStr, sync::Arc, time::Duration};\nuse store::{\n    ValueKey,\n    write::{AlignedBytes, Archive, BatchBuilder, now},\n};\nuse trc::{AddContext, TaskQueueEvent};\nuse types::collection::Collection;\nuse utils::{sanitize_email, template::Variables};\n\npub trait SendAlarmTask: Sync + Send {\n    fn send_alarm(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        alarm: &CalendarAlarm,\n        server_instance: Arc<ServerInstance>,\n    ) -> impl Future<Output = bool> + Send;\n}\n\nimpl SendAlarmTask for Server {\n    async fn send_alarm(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        alarm: &CalendarAlarm,\n        server_instance: Arc<ServerInstance>,\n    ) -> bool {\n        match &alarm.typ {\n            CalendarAlarmType::Display { .. } => {\n                match send_display_alarm(self, account_id, document_id, alarm).await {\n                    Ok(result) => result,\n                    Err(err) => {\n                        trc::error!(\n                            err.account_id(account_id)\n                                .document_id(document_id)\n                                .caused_by(trc::location!())\n                                .details(\"Failed to process e-mail alarm\")\n                        );\n                        false\n                    }\n                }\n            }\n            CalendarAlarmType::Email { .. } => {\n                match send_email_alarm(self, account_id, document_id, alarm, server_instance).await\n                {\n                    Ok(result) => result,\n                    Err(err) => {\n                        trc::error!(\n                            err.account_id(account_id)\n                                .document_id(document_id)\n                                .caused_by(trc::location!())\n                                .details(\"Failed to process e-mail alarm\")\n                        );\n                        false\n                    }\n                }\n            }\n        }\n    }\n}\n\nasync fn send_email_alarm(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n    alarm: &CalendarAlarm,\n    server_instance: Arc<ServerInstance>,\n) -> trc::Result<bool> {\n    // Obtain access token\n    let access_token = server\n        .get_access_token(account_id)\n        .await\n        .caused_by(trc::location!())?;\n\n    if !access_token.has_permission(Permission::CalendarAlarms) {\n        trc::event!(\n            Calendar(trc::CalendarEvent::AlarmSkipped),\n            Reason = \"Account does not have permission to send calendar alarms\",\n            AccountId = account_id,\n            DocumentId = document_id,\n        );\n        return Ok(true);\n    } else if access_token.emails.is_empty() {\n        trc::event!(\n            Calendar(trc::CalendarEvent::AlarmFailed),\n            Reason = \"Account does not have any email addresses\",\n            AccountId = account_id,\n            DocumentId = document_id,\n        );\n        return Ok(true);\n    }\n\n    // Fetch event\n    let Some(event_) = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::CalendarEvent,\n            document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n    else {\n        trc::event!(\n            TaskQueue(TaskQueueEvent::MetadataNotFound),\n            Details = \"Calendar Event metadata not found\",\n            AccountId = account_id,\n            DocumentId = document_id,\n        );\n\n        return Ok(true);\n    };\n\n    // Unarchive event\n    let event = event_\n        .unarchive::<CalendarEvent>()\n        .caused_by(trc::location!())?;\n\n    // Build message body\n    let account_main_email = access_token.emails.first().unwrap();\n    let account_main_domain = account_main_email.rsplit('@').next().unwrap_or(\"localhost\");\n    let logo_cid = format!(\"logo.{}@{account_main_domain}\", now());\n    let Some(tpl) = build_template(\n        server,\n        &access_token,\n        account_id,\n        document_id,\n        alarm,\n        event,\n        &logo_cid,\n    )\n    .await?\n    else {\n        return Ok(true);\n    };\n    let txt_body = html_to_text(&tpl.body);\n\n    // Obtain logo image\n    let logo = match server.logo_resource(account_main_domain).await {\n        Ok(logo) => logo,\n        Err(err) => {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to fetch logo image\")\n            );\n            None\n        }\n    };\n    let logo = if let Some(logo) = &logo {\n        MimePart::new(\n            ContentType::new(logo.content_type.as_ref()),\n            BodyPart::Binary(logo.contents.as_slice().into()),\n        )\n    } else {\n        MimePart::new(\n            ContentType::new(\"image/png\"),\n            BodyPart::Binary(DEFAULT_LOGO_BASE64.as_bytes().into()),\n        )\n        .transfer_encoding(\"base64\")\n    }\n    .inline()\n    .cid(&logo_cid);\n\n    // Build message\n    let mail_from = if let Some(from_email) = &server.core.groupware.alarms_from_email {\n        from_email.to_string()\n    } else {\n        format!(\"calendar-notification@{account_main_domain}\")\n    };\n    let message = MessageBuilder::new()\n        .from((\n            server.core.groupware.alarms_from_name.as_str(),\n            mail_from.as_str(),\n        ))\n        .header(\"To\", HeaderType::Text(tpl.to.as_str().into()))\n        .header(\"Auto-Submitted\", HeaderType::Text(\"auto-generated\".into()))\n        .header(\"Reply-To\", HeaderType::Text(account_main_email.into()))\n        .subject(tpl.subject)\n        .body(MimePart::new(\n            ContentType::new(\"multipart/related\"),\n            BodyPart::Multipart(vec![\n                MimePart::new(\n                    ContentType::new(\"multipart/alternative\"),\n                    BodyPart::Multipart(vec![\n                        MimePart::new(\n                            ContentType::new(\"text/plain\"),\n                            BodyPart::Text(txt_body.into()),\n                        ),\n                        MimePart::new(\n                            ContentType::new(\"text/html\"),\n                            BodyPart::Text(tpl.body.into()),\n                        ),\n                    ]),\n                ),\n                logo,\n            ]),\n        ))\n        .write_to_vec()\n        .unwrap_or_default();\n\n    // Send message\n    let server_ = server.clone();\n    let mail_from = account_main_email.to_string();\n    let to = tpl.to;\n    let result = tokio::spawn(async move {\n        let mut session = Session::<NullIo>::local(\n            server_,\n            server_instance,\n            SessionData::local(access_token, None, vec![], vec![], 0),\n        );\n\n        // MAIL FROM\n        let _ = session\n            .handle_mail_from(MailFrom {\n                address: mail_from.into(),\n                ..Default::default()\n            })\n            .await;\n        if let Some(error) = session.has_failed() {\n            return Err(format!(\"Server rejected MAIL-FROM: {}\", error.trim()));\n        }\n\n        // RCPT TO\n        session.params.rcpt_errors_wait = Duration::from_secs(0);\n        let _ = session\n            .handle_rcpt_to(RcptTo {\n                address: to.into(),\n                ..Default::default()\n            })\n            .await;\n        if let Some(error) = session.has_failed() {\n            return Err(format!(\"Server rejected RCPT-TO: {}\", error.trim()));\n        }\n\n        // DATA\n        session.data.message = message;\n        let response = session.queue_message().await;\n        if let smtp::core::State::Accepted(queue_id) = session.state {\n            Ok(queue_id)\n        } else {\n            Err(format!(\n                \"Server rejected DATA: {}\",\n                std::str::from_utf8(&response).unwrap().trim()\n            ))\n        }\n    })\n    .await;\n\n    match result {\n        Ok(Ok(queue_id)) => {\n            trc::event!(\n                Calendar(trc::CalendarEvent::AlarmSent),\n                AccountId = account_id,\n                DocumentId = document_id,\n                QueueId = queue_id,\n            );\n        }\n        Ok(Err(err)) => {\n            trc::event!(\n                Calendar(trc::CalendarEvent::AlarmFailed),\n                AccountId = account_id,\n                DocumentId = document_id,\n                Reason = err,\n            );\n        }\n        Err(_) => {\n            trc::event!(\n                Server(trc::ServerEvent::ThreadError),\n                Details = \"Join Error\",\n                AccountId = account_id,\n                DocumentId = document_id,\n                CausedBy = trc::location!(),\n            );\n            return Ok(false);\n        }\n    }\n\n    write_next_alarm(server, account_id, document_id, event).await\n}\n\nasync fn send_display_alarm(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n    alarm: &CalendarAlarm,\n) -> trc::Result<bool> {\n    // Fetch event\n    let Some(event_) = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::CalendarEvent,\n            document_id,\n        ))\n        .await\n        .caused_by(trc::location!())?\n    else {\n        trc::event!(\n            TaskQueue(TaskQueueEvent::MetadataNotFound),\n            Details = \"Calendar Event metadata not found\",\n            AccountId = account_id,\n            DocumentId = document_id,\n        );\n\n        return Ok(true);\n    };\n\n    // Unarchive event\n    let event = event_\n        .unarchive::<CalendarEvent>()\n        .caused_by(trc::location!())?;\n\n    let recurrence_id = match &alarm.typ {\n        CalendarAlarmType::Display { recurrence_id } => *recurrence_id,\n        _ => None,\n    };\n\n    let ical = &event.data.event;\n    server\n        .broadcast_push_notification(PushNotification::CalendarAlert(CalendarAlert {\n            account_id,\n            event_id: document_id,\n            recurrence_id,\n            uid: ical.uids().next().unwrap_or_default().to_string(),\n            alert_id: ical\n                .components\n                .get(alarm.alarm_id as usize)\n                .and_then(|c| c.property(&ICalendarProperty::Jsid))\n                .and_then(|v| v.values.first())\n                .and_then(|v| v.as_text())\n                .map(|v| v.to_string())\n                .unwrap_or_else(|| {\n                    format!(\n                        \"k{}\",\n                        ical.components\n                            .get(alarm.event_id as usize)\n                            .and_then(|c| c\n                                .component_ids\n                                .iter()\n                                .position(|id| id.to_native() == alarm.alarm_id as u32))\n                            .unwrap_or_default()\n                            + 1\n                    )\n                }),\n        }))\n        .await;\n\n    write_next_alarm(server, account_id, document_id, event).await\n}\n\nasync fn write_next_alarm(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n    event: &ArchivedCalendarEvent,\n) -> trc::Result<bool> {\n    // Find next alarm time and write to task queue\n    let now = now() as i64;\n    if let Some(next_alarm) =\n        event\n            .data\n            .next_alarm(now, Default::default())\n            .and_then(|next_alarm| {\n                // Verify minimum interval\n                let max_next_alarm = now + server.core.groupware.alarms_minimum_interval;\n                if next_alarm.alarm_time < max_next_alarm {\n                    trc::event!(\n                        Calendar(trc::CalendarEvent::AlarmSkipped),\n                        Reason = \"Next alarm skipped due to minimum interval\",\n                        Details = next_alarm.alarm_time - now,\n                        AccountId = account_id,\n                        DocumentId = document_id,\n                    );\n                    event.data.next_alarm(max_next_alarm, Default::default())\n                } else {\n                    Some(next_alarm)\n                }\n            })\n    {\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::CalendarEvent)\n            .with_document(document_id);\n        next_alarm.write_task(&mut batch);\n        server\n            .store()\n            .write(batch.build_all())\n            .await\n            .caused_by(trc::location!())?;\n        server.notify_task_queue();\n    }\n\n    Ok(true)\n}\n\nstruct Details {\n    to: String,\n    subject: String,\n    body: String,\n}\n\nasync fn build_template(\n    server: &Server,\n    access_token: &AccessToken,\n    account_id: u32,\n    document_id: u32,\n    alarm: &CalendarAlarm,\n    event: &ArchivedCalendarEvent,\n    logo_cid: &str,\n) -> trc::Result<Option<Details>> {\n    let (Some(event_component), Some(alarm_component)) = (\n        event.data.event.components.get(alarm.event_id as usize),\n        event.data.event.components.get(alarm.alarm_id as usize),\n    ) else {\n        trc::event!(\n            TaskQueue(TaskQueueEvent::MetadataNotFound),\n            Details = \"Calendar Alarm component not found\",\n            AccountId = account_id,\n            DocumentId = document_id,\n        );\n        return Ok(None);\n    };\n\n    // Build webcal URI\n    let webcal_uri = match event.webcal_uri(server, access_token).await {\n        Ok(uri) => uri,\n        Err(err) => {\n            trc::error!(\n                err.account_id(account_id)\n                    .document_id(document_id)\n                    .caused_by(trc::location!())\n                    .details(\"Failed to generate webcal URI\")\n            );\n            String::from(\"#\")\n        }\n    };\n\n    // Obtain alarm details\n    let mut summary = None;\n    let mut description = None;\n    let mut rcpt_to = None;\n    let mut location = None;\n    let mut organizer = None;\n    let mut guests = vec![];\n\n    for entry in alarm_component.entries.iter() {\n        match &entry.name {\n            ArchivedICalendarProperty::Summary => {\n                summary = entry.values.first().and_then(|v| v.as_text());\n            }\n            ArchivedICalendarProperty::Description => {\n                description = entry.values.first().and_then(|v| v.as_text());\n            }\n            ArchivedICalendarProperty::Attendee => {\n                rcpt_to = entry\n                    .values\n                    .first()\n                    .and_then(|v| v.as_text())\n                    .map(|v| v.strip_prefix(\"mailto:\").unwrap_or(v))\n                    .and_then(sanitize_email);\n            }\n            _ => {}\n        }\n    }\n\n    for entry in event_component.entries.iter() {\n        match &entry.name {\n            ArchivedICalendarProperty::Summary if summary.is_none() => {\n                summary = entry.values.first().and_then(|v| v.as_text());\n            }\n            ArchivedICalendarProperty::Description if description.is_none() => {\n                description = entry.values.first().and_then(|v| v.as_text());\n            }\n            ArchivedICalendarProperty::Location => {\n                location = entry.values.first().and_then(|v| v.as_text());\n            }\n            ArchivedICalendarProperty::Organizer | ArchivedICalendarProperty::Attendee => {\n                let email = entry\n                    .values\n                    .first()\n                    .and_then(|v| v.as_text())\n                    .map(|v| v.strip_prefix(\"mailto:\").unwrap_or(v));\n                let name = entry.params.iter().find_map(|param| {\n                    if let ArchivedICalendarParameterName::Cn = param.name {\n                        param.value.as_text()\n                    } else {\n                        None\n                    }\n                });\n\n                if email.is_some() || name.is_some() {\n                    if matches!(entry.name, ArchivedICalendarProperty::Organizer) {\n                        organizer = Some((email, name));\n                    } else {\n                        guests.push((email, name));\n                    }\n                }\n            }\n            _ => {}\n        }\n    }\n\n    // Validate recipient\n    let rcpt_to = if let Some(rcpt_to) = rcpt_to {\n        if server.core.groupware.alarms_allow_external_recipients\n            || access_token.emails.iter().any(|email| email == &rcpt_to)\n        {\n            rcpt_to\n        } else {\n            trc::event!(\n                Calendar(trc::CalendarEvent::AlarmRecipientOverride),\n                Reason = \"External recipient not allowed for calendar alarms\",\n                Details = rcpt_to,\n                AccountId = account_id,\n                DocumentId = document_id,\n            );\n\n            access_token.emails.first().unwrap().to_string()\n        }\n    } else {\n        access_token.emails.first().unwrap().to_string()\n    };\n\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    let template = server\n        .core\n        .enterprise\n        .as_ref()\n        .and_then(|e| e.template_calendar_alarm.as_ref())\n        .unwrap_or(&server.core.groupware.alarms_template);\n    // SPDX-SnippetEnd\n\n    #[cfg(not(feature = \"enterprise\"))]\n    let template = &server.core.groupware.alarms_template;\n    let locale = i18n::locale_or_default(access_token.locale.as_deref().unwrap_or(\"en\"));\n    let chrono_locale = access_token\n        .locale\n        .as_deref()\n        .and_then(|locale| Locale::from_str(locale).ok())\n        .unwrap_or(Locale::en_US);\n    let (event_start, event_start_tz, event_end, event_end_tz) = match alarm.typ {\n        CalendarAlarmType::Email {\n            event_start,\n            event_start_tz,\n            event_end,\n            event_end_tz,\n        } => (event_start, event_start_tz, event_end, event_end_tz),\n        CalendarAlarmType::Display { .. } => unreachable!(),\n    };\n\n    let start = format!(\n        \"{} ({})\",\n        DateTime::from_timestamp(event_start, 0)\n            .unwrap_or_default()\n            .format_localized(locale.calendar_date_template, chrono_locale),\n        Tz::from_id(event_start_tz)\n            .unwrap_or(Tz::UTC)\n            .name()\n            .unwrap_or_default()\n    );\n    let end = format!(\n        \"{} ({})\",\n        DateTime::from_timestamp(event_end, 0)\n            .unwrap_or_default()\n            .format_localized(locale.calendar_date_template, chrono_locale),\n        Tz::from_id(event_end_tz)\n            .unwrap_or(Tz::UTC)\n            .name()\n            .unwrap_or_default()\n    );\n    let subject = format!(\n        \"{}: {} @ {}\",\n        locale.calendar_alarm_subject_prefix,\n        summary.or(description).unwrap_or(\"No Subject\"),\n        start\n    );\n    let organizer = organizer\n        .map(|(email, name)| match (email, name) {\n            (Some(email), Some(name)) => format!(\"{} <{}>\", name, email),\n            (Some(email), None) => email.to_string(),\n            (None, Some(name)) => name.to_string(),\n            _ => unreachable!(),\n        })\n        .unwrap_or_else(|| access_token.name.clone());\n    let mut variables = Variables::new();\n    variables.insert_single(CalendarTemplateVariable::PageTitle, subject.as_str());\n    variables.insert_single(\n        CalendarTemplateVariable::Header,\n        locale.calendar_alarm_header,\n    );\n    variables.insert_single(\n        CalendarTemplateVariable::Footer,\n        locale.calendar_alarm_footer,\n    );\n    variables.insert_single(\n        CalendarTemplateVariable::ActionName,\n        locale.calendar_alarm_open,\n    );\n    variables.insert_single(CalendarTemplateVariable::ActionUrl, webcal_uri.as_str());\n    variables.insert_single(\n        CalendarTemplateVariable::AttendeesTitle,\n        locale.calendar_attendees,\n    );\n    variables.insert_single(\n        CalendarTemplateVariable::EventTitle,\n        summary.unwrap_or_default(),\n    );\n    variables.insert_single(CalendarTemplateVariable::LogoCid, logo_cid);\n    if let Some(description) = description {\n        variables.insert_single(CalendarTemplateVariable::EventDescription, description);\n    }\n    variables.insert_block(\n        CalendarTemplateVariable::EventDetails,\n        [\n            Some([\n                (CalendarTemplateVariable::Key, locale.calendar_start),\n                (CalendarTemplateVariable::Value, start.as_str()),\n            ]),\n            Some([\n                (CalendarTemplateVariable::Key, locale.calendar_end),\n                (CalendarTemplateVariable::Value, end.as_str()),\n            ]),\n            location.map(|location| {\n                [\n                    (CalendarTemplateVariable::Key, locale.calendar_location),\n                    (CalendarTemplateVariable::Value, location),\n                ]\n            }),\n            Some([\n                (CalendarTemplateVariable::Key, locale.calendar_organizer),\n                (CalendarTemplateVariable::Value, organizer.as_str()),\n            ]),\n        ]\n        .into_iter()\n        .flatten(),\n    );\n    if !guests.is_empty() {\n        variables.insert_block(\n            CalendarTemplateVariable::Attendees,\n            guests.into_iter().map(|(email, name)| {\n                [\n                    (CalendarTemplateVariable::Key, name.unwrap_or_default()),\n                    (CalendarTemplateVariable::Value, email.unwrap_or_default()),\n                ]\n            }),\n        );\n    }\n    Ok(Some(Details {\n        to: rcpt_to,\n        body: template.eval(&variables),\n        subject,\n    }))\n}\n"
  },
  {
    "path": "crates/services/src/task_manager/imip.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse calcard::{\n    common::timezone::Tz,\n    icalendar::{\n        ArchivedICalendarDay, ArchivedICalendarFrequency, ArchivedICalendarMonth,\n        ArchivedICalendarParticipationStatus, ArchivedICalendarRecurrenceRule,\n        ArchivedICalendarWeekday, ICalendarParticipationStatus, ICalendarProperty,\n    },\n};\nuse chrono::{DateTime, Locale};\nuse common::{\n    DEFAULT_LOGO_BASE64, Server,\n    auth::AccessToken,\n    config::groupware::CalendarTemplateVariable,\n    i18n,\n    listener::{ServerInstance, stream::NullIo},\n};\nuse groupware::{\n    calendar::itip::ItipIngest,\n    scheduling::{ArchivedItipSummary, ArchivedItipValue, ItipMessages},\n};\nuse mail_builder::{\n    MessageBuilder,\n    headers::{HeaderType, content_type::ContentType},\n    mime::{BodyPart, MimePart},\n};\nuse mail_parser::decoders::html::html_to_text;\nuse smtp::core::{Session, SessionData};\nuse smtp_proto::{MailFrom, RcptTo};\nuse std::{str::FromStr, sync::Arc, time::Duration};\nuse store::{\n    ValueKey,\n    ahash::AHashMap,\n    rkyv::rend::{i16_le, i32_le},\n    write::{AlignedBytes, Archive, TaskEpoch, TaskQueueClass, ValueClass, now},\n};\nuse trc::AddContext;\nuse utils::template::{Variable, Variables};\n\npub trait SendImipTask: Sync + Send {\n    fn send_imip(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        due: TaskEpoch,\n        server_instance: Arc<ServerInstance>,\n    ) -> impl Future<Output = bool> + Send;\n}\n\nimpl SendImipTask for Server {\n    async fn send_imip(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        due: TaskEpoch,\n        server_instance: Arc<ServerInstance>,\n    ) -> bool {\n        match send_imip(self, account_id, document_id, due, server_instance).await {\n            Ok(result) => result,\n            Err(err) => {\n                trc::error!(\n                    err.account_id(account_id)\n                        .document_id(document_id)\n                        .caused_by(trc::location!())\n                        .details(\"Failed to process alarm\")\n                );\n                false\n            }\n        }\n    }\n}\n\nasync fn send_imip(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n    due: TaskEpoch,\n    server_instance: Arc<ServerInstance>,\n) -> trc::Result<bool> {\n    // Obtain access token\n    let access_token = server\n        .get_access_token(account_id)\n        .await\n        .caused_by(trc::location!())?;\n\n    // Obtain iMIP payload\n    let Some(archive) = server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey {\n            account_id,\n            collection: 0,\n            document_id,\n            class: ValueClass::TaskQueue(TaskQueueClass::SendImip {\n                due,\n                is_payload: true,\n            }),\n        })\n        .await\n        .caused_by(trc::location!())?\n    else {\n        trc::event!(\n            Calendar(trc::CalendarEvent::ItipMessageError),\n            AccountId = account_id,\n            DocumentId = document_id,\n            Reason = \"Missing iMIP payload\",\n        );\n        return Ok(true);\n    };\n\n    let imip = archive\n        .unarchive::<ItipMessages>()\n        .caused_by(trc::location!())?;\n\n    let sender_domain = imip\n        .messages\n        .first()\n        .and_then(|msg| msg.from.rsplit('@').next())\n        .unwrap_or(\"localhost\");\n\n    // Obtain logo image\n    let logo = match server.logo_resource(sender_domain).await {\n        Ok(logo) => logo,\n        Err(err) => {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to fetch logo image\")\n            );\n            None\n        }\n    };\n    let logo_cid = format!(\"logo.{}@{sender_domain}\", now());\n    let logo = if let Some(logo) = &logo {\n        MimePart::new(\n            ContentType::new(logo.content_type.as_ref()),\n            BodyPart::Binary(logo.contents.as_slice().into()),\n        )\n    } else {\n        MimePart::new(\n            ContentType::new(\"image/png\"),\n            BodyPart::Binary(DEFAULT_LOGO_BASE64.as_bytes().into()),\n        )\n        .transfer_encoding(\"base64\")\n    }\n    .inline()\n    .cid(&logo_cid);\n\n    for itip_message in imip.messages.iter() {\n        for recipient in itip_message.to.iter() {\n            // Build template\n            let tpl = build_itip_template(\n                server,\n                &access_token,\n                account_id,\n                document_id,\n                itip_message.from.as_str(),\n                recipient.as_str(),\n                &itip_message.summary,\n                &logo_cid,\n            )\n            .await;\n            let txt_body = html_to_text(&tpl.body);\n\n            // Build message\n            let message = MessageBuilder::new()\n                .from((\n                    access_token\n                        .description\n                        .as_deref()\n                        .unwrap_or(access_token.name.as_str()),\n                    itip_message.from.as_str(),\n                ))\n                .to(recipient.as_str())\n                .header(\"Auto-Submitted\", HeaderType::Text(\"auto-generated\".into()))\n                .header(\n                    \"Reply-To\",\n                    HeaderType::Text(itip_message.from.as_str().into()),\n                )\n                .subject(&tpl.subject)\n                .body(MimePart::new(\n                    ContentType::new(\"multipart/mixed\"),\n                    BodyPart::Multipart(vec![\n                        MimePart::new(\n                            ContentType::new(\"multipart/related\"),\n                            BodyPart::Multipart(vec![\n                                MimePart::new(\n                                    ContentType::new(\"multipart/alternative\"),\n                                    BodyPart::Multipart(vec![\n                                        MimePart::new(\n                                            ContentType::new(\"text/plain\"),\n                                            BodyPart::Text(txt_body.into()),\n                                        ),\n                                        MimePart::new(\n                                            ContentType::new(\"text/html\"),\n                                            BodyPart::Text(tpl.body.as_str().into()),\n                                        ),\n                                    ]),\n                                ),\n                                logo.clone(),\n                            ]),\n                        ),\n                        MimePart::new(\n                            ContentType::new(\"text/calendar\")\n                                .attribute(\"method\", itip_message.summary.method())\n                                .attribute(\"charset\", \"utf-8\"),\n                            BodyPart::Text(itip_message.message.as_str().into()),\n                        )\n                        .attachment(\"event.ics\"),\n                    ]),\n                ))\n                .write_to_vec()\n                .unwrap_or_default();\n\n            // Send message\n            let server_ = server.clone();\n            let server_instance = server_instance.clone();\n            let access_token = access_token.clone();\n            let from = itip_message.from.to_string();\n            let to = recipient.to_string();\n            tokio::spawn(async move {\n                let mut session = Session::<NullIo>::local(\n                    server_,\n                    server_instance,\n                    SessionData::local(access_token, None, vec![], vec![], 0),\n                );\n\n                // MAIL FROM\n                let _ = session\n                    .handle_mail_from(MailFrom {\n                        address: from.as_str().into(),\n                        ..Default::default()\n                    })\n                    .await;\n                if let Some(error) = session.has_failed() {\n                    trc::event!(\n                        Calendar(trc::CalendarEvent::ItipMessageError),\n                        AccountId = account_id,\n                        DocumentId = document_id,\n                        From = from,\n                        To = to,\n                        Reason = format!(\"Server rejected MAIL-FROM: {}\", error.trim()),\n                    );\n                    return;\n                }\n\n                // RCPT TO\n                session.params.rcpt_errors_wait = Duration::from_secs(0);\n                let _ = session\n                    .handle_rcpt_to(RcptTo {\n                        address: to.as_str().into(),\n                        ..Default::default()\n                    })\n                    .await;\n                if let Some(error) = session.has_failed() {\n                    trc::event!(\n                        Calendar(trc::CalendarEvent::ItipMessageError),\n                        AccountId = account_id,\n                        DocumentId = document_id,\n                        From = from,\n                        To = to,\n                        Reason = format!(\"Server rejected RCPT-TO: {}\", error.trim()),\n                    );\n                    return;\n                }\n\n                // DATA\n                session.data.message = message;\n                let response = session.queue_message().await;\n                if let smtp::core::State::Accepted(queue_id) = session.state {\n                    trc::event!(\n                        Calendar(trc::CalendarEvent::ItipMessageSent),\n                        From = from,\n                        To = to,\n                        AccountId = account_id,\n                        DocumentId = document_id,\n                        QueueId = queue_id,\n                    );\n                } else {\n                    trc::event!(\n                        Calendar(trc::CalendarEvent::ItipMessageError),\n                        From = from,\n                        To = to,\n                        AccountId = account_id,\n                        DocumentId = document_id,\n                        Reason = format!(\n                            \"Server rejected DATA: {}\",\n                            std::str::from_utf8(&response).unwrap().trim()\n                        ),\n                    );\n                }\n            })\n            .await\n            .map_err(|_| {\n                trc::Error::new(trc::EventType::Server(trc::ServerEvent::ThreadError))\n                    .caused_by(trc::location!())\n            })?;\n        }\n    }\n\n    Ok(true)\n}\n\npub struct Details {\n    pub subject: String,\n    pub body: String,\n}\n\n#[allow(clippy::too_many_arguments)]\npub async fn build_itip_template(\n    server: &Server,\n    access_token: &AccessToken,\n    account_id: u32,\n    document_id: u32,\n    from: &str,\n    to: &str,\n    summary: &ArchivedItipSummary,\n    logo_cid: &str,\n) -> Details {\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    let template = server\n        .core\n        .enterprise\n        .as_ref()\n        .and_then(|e| e.template_scheduling_email.as_ref())\n        .unwrap_or(&server.core.groupware.itip_template);\n    // SPDX-SnippetEnd\n    #[cfg(not(feature = \"enterprise\"))]\n    let template = &server.core.groupware.itip_template;\n    let locale = i18n::locale_or_default(access_token.locale.as_deref().unwrap_or(\"en\"));\n    let chrono_locale = access_token\n        .locale\n        .as_deref()\n        .and_then(|locale| Locale::from_str(locale).ok())\n        .unwrap_or(Locale::en_US);\n\n    let mut variables = Variables::new();\n    let mut subject;\n    let (fields, old_fields) = match summary {\n        ArchivedItipSummary::Invite(fields) => {\n            subject = format!(\"{}: \", locale.calendar_invitation);\n\n            (fields, None)\n        }\n        ArchivedItipSummary::Update {\n            current, previous, ..\n        } => {\n            subject = format!(\"{}: \", locale.calendar_updated_invitation);\n            variables.insert_single(\n                CalendarTemplateVariable::Header,\n                locale.calendar_event_updated.to_string(),\n            );\n            variables.insert_single(CalendarTemplateVariable::Color, \"info\".to_string());\n            (current, Some(previous))\n        }\n        ArchivedItipSummary::Cancel(fields) => {\n            subject = format!(\"{}: \", locale.calendar_cancelled);\n            variables.insert_single(\n                CalendarTemplateVariable::Header,\n                locale.calendar_event_cancelled.to_string(),\n            );\n            variables.insert_single(CalendarTemplateVariable::Color, \"danger\".to_string());\n            (fields, None)\n        }\n        ArchivedItipSummary::Rsvp { part_stat, current } => {\n            let (color, value) = match part_stat {\n                ArchivedICalendarParticipationStatus::Accepted => {\n                    subject = format!(\"{}: \", locale.calendar_accepted);\n\n                    (\n                        \"info\",\n                        locale.calendar_participant_accepted.replace(\"$name\", from),\n                    )\n                }\n                ArchivedICalendarParticipationStatus::Declined => {\n                    subject = format!(\"{}: \", locale.calendar_declined);\n                    (\n                        \"danger\",\n                        locale.calendar_participant_declined.replace(\"$name\", from),\n                    )\n                }\n                ArchivedICalendarParticipationStatus::Tentative => {\n                    subject = format!(\"{}: \", locale.calendar_tentative);\n                    (\n                        \"warning\",\n                        locale.calendar_participant_tentative.replace(\"$name\", from),\n                    )\n                }\n                ArchivedICalendarParticipationStatus::Delegated => {\n                    subject = format!(\"{}: \", locale.calendar_delegated);\n                    (\n                        \"warning\",\n                        locale.calendar_participant_delegated.replace(\"$name\", from),\n                    )\n                }\n                _ => {\n                    subject = format!(\"{}: \", locale.calendar_reply);\n                    (\n                        \"info\",\n                        locale.calendar_participant_reply.replace(\"$name\", from),\n                    )\n                }\n            };\n\n            variables.insert_single(CalendarTemplateVariable::Header, value);\n            variables.insert_single(CalendarTemplateVariable::Color, color.to_string());\n\n            (current, None)\n        }\n    };\n\n    let mut has_rrule = false;\n    let mut details = Vec::with_capacity(4);\n    for field in [\n        ICalendarProperty::Summary,\n        ICalendarProperty::Description,\n        ICalendarProperty::Rrule,\n        ICalendarProperty::Dtstart,\n        ICalendarProperty::Location,\n    ] {\n        let Some(entry) = fields.iter().find(|e| e.name == field) else {\n            continue;\n        };\n        let field_name = match &field {\n            ICalendarProperty::Summary => locale.calendar_summary,\n            ICalendarProperty::Description => locale.calendar_description,\n            ICalendarProperty::Rrule => {\n                has_rrule = true;\n                locale.calendar_when\n            }\n            ICalendarProperty::Dtstart if !has_rrule => locale.calendar_when,\n            ICalendarProperty::Location => locale.calendar_location,\n            _ => continue,\n        };\n        let value = format_field(\n            &entry.value,\n            locale.calendar_date_template_long,\n            chrono_locale,\n        );\n\n        match &field {\n            ICalendarProperty::Summary => {\n                subject.push_str(&value);\n            }\n            ICalendarProperty::Dtstart | ICalendarProperty::Rrule => {\n                subject.push_str(\" @ \");\n                subject.push_str(&value);\n            }\n            _ => (),\n        }\n\n        let mut fields = AHashMap::with_capacity(3);\n        fields.insert(CalendarTemplateVariable::Key, field_name.to_string());\n        fields.insert(CalendarTemplateVariable::Value, value);\n        if let Some(old_entry) =\n            old_fields.and_then(|fields| fields.iter().find(|e| e.name == field))\n        {\n            fields.insert(\n                CalendarTemplateVariable::Changed,\n                locale.calendar_changed.to_string(),\n            );\n            fields.insert(\n                CalendarTemplateVariable::OldValue,\n                format_field(\n                    &old_entry.value,\n                    locale.calendar_date_template,\n                    chrono_locale,\n                ),\n            );\n        }\n        details.push(fields);\n    }\n    variables.items.insert(\n        CalendarTemplateVariable::EventDetails,\n        Variable::Block(details),\n    );\n    variables.insert_single(CalendarTemplateVariable::PageTitle, subject.clone());\n    variables.insert_single(CalendarTemplateVariable::LogoCid, format!(\"cid:{logo_cid}\"));\n\n    if let Some(guests) = fields\n        .iter()\n        .find(|e| e.name == ICalendarProperty::Attendee)\n        && let ArchivedItipValue::Participants(guests) = &guests.value\n    {\n        variables.insert_single(\n            CalendarTemplateVariable::AttendeesTitle,\n            locale.calendar_attendees.to_string(),\n        );\n        variables.insert_block(\n            CalendarTemplateVariable::Attendees,\n            guests.iter().map(|guest| {\n                [\n                    (\n                        CalendarTemplateVariable::Key,\n                        if guest.is_organizer {\n                            if let Some(name) = guest.name.as_ref() {\n                                format!(\"{name} - {}\", locale.calendar_organizer)\n                            } else {\n                                locale.calendar_organizer.to_string()\n                            }\n                        } else {\n                            guest\n                                .name\n                                .as_ref()\n                                .map(|n| n.as_str())\n                                .unwrap_or_default()\n                                .to_string()\n                        },\n                    ),\n                    (CalendarTemplateVariable::Value, guest.email.to_string()),\n                ]\n            }),\n        );\n    }\n\n    // Add RSVP buttons\n    if matches!(\n        summary,\n        ArchivedItipSummary::Invite(_) | ArchivedItipSummary::Update { .. }\n    ) && let Some(rsvp_url) = server.http_rsvp_url(account_id, document_id, to).await\n    {\n        variables.insert_single(\n            CalendarTemplateVariable::Rsvp,\n            locale.calendar_reply_as.replace(\"$name\", to),\n        );\n        variables.insert_block(\n            CalendarTemplateVariable::Actions,\n            [\n                (\n                    ICalendarParticipationStatus::Accepted,\n                    locale.calendar_yes.to_string(),\n                    \"info\",\n                ),\n                (\n                    ICalendarParticipationStatus::Declined,\n                    locale.calendar_no.to_string(),\n                    \"danger\",\n                ),\n                (\n                    ICalendarParticipationStatus::Tentative,\n                    locale.calendar_maybe.to_string(),\n                    \"warning\",\n                ),\n            ]\n            .into_iter()\n            .map(|(status, title, color)| {\n                [\n                    (CalendarTemplateVariable::ActionName, title.to_string()),\n                    (CalendarTemplateVariable::ActionUrl, rsvp_url.url(&status)),\n                    (CalendarTemplateVariable::Color, color.to_string()),\n                ]\n            }),\n        );\n    }\n\n    // Add footer\n    variables.insert_block(\n        CalendarTemplateVariable::Footer,\n        [\n            [(\n                CalendarTemplateVariable::Key,\n                locale.calendar_imip_footer_1.to_string(),\n            )],\n            [(\n                CalendarTemplateVariable::Key,\n                locale.calendar_imip_footer_2.to_string(),\n            )],\n        ]\n        .into_iter(),\n    );\n\n    Details {\n        subject,\n        body: template.eval(&variables),\n    }\n}\n\nfn format_field(value: &ArchivedItipValue, template: &str, chrono_locale: Locale) -> String {\n    match value {\n        ArchivedItipValue::Text(text) => text.to_string(),\n        ArchivedItipValue::Time(time) => {\n            use chrono::TimeZone;\n            let tz = Tz::from_id(time.tz_id.to_native()).unwrap_or(Tz::UTC);\n            format!(\n                \"{} ({})\",\n                tz.from_utc_datetime(\n                    &DateTime::from_timestamp(time.start.to_native(), 0)\n                        .unwrap_or_default()\n                        .naive_local()\n                )\n                .format_localized(template, chrono_locale),\n                tz.name().unwrap_or_default()\n            )\n        }\n        ArchivedItipValue::Rrule(rrule) => RecurrenceFormatter.format(rrule),\n        ArchivedItipValue::Participants(_) => String::new(), // Handled separately\n    }\n}\n\n#[derive(Default)]\npub struct RecurrenceFormatter;\n\nimpl RecurrenceFormatter {\n    pub fn format(&self, rule: &ArchivedICalendarRecurrenceRule) -> String {\n        let mut parts = Vec::new();\n\n        // Format frequency and interval\n        let freq_part = self.format_frequency(\n            &rule.freq,\n            rule.interval.as_ref().map(|i| i.to_native()).unwrap_or(1),\n        );\n        parts.push(freq_part);\n\n        // Format day constraints\n        if !rule.byday.is_empty() {\n            parts.push(self.format_by_day(&rule.byday));\n        }\n\n        // Format time constraints\n        if !rule.byhour.is_empty() || !rule.byminute.is_empty() {\n            parts.push(self.format_time_constraints(&rule.byhour, &rule.byminute));\n        }\n\n        // Format month day constraints\n        if !rule.bymonthday.is_empty() {\n            parts.push(self.format_month_days(&rule.bymonthday));\n        }\n\n        // Format month constraints\n        if !rule.bymonth.is_empty() {\n            parts.push(self.format_months(&rule.bymonth));\n        }\n\n        // Format year day constraints\n        if !rule.byyearday.is_empty() {\n            parts.push(self.format_year_days(&rule.byyearday));\n        }\n\n        // Format week number constraints\n        if !rule.byweekno.is_empty() {\n            parts.push(self.format_week_numbers(&rule.byweekno));\n        }\n\n        // Format set position constraints\n        if !rule.bysetpos.is_empty() {\n            parts.push(self.format_set_positions(&rule.bysetpos));\n        }\n\n        // Format termination (until/count)\n        /*if let Some(until) = &rule.until {\n            parts.push(format!(\"until {}\", self.format_datetime(until)));\n        } else*/\n        if let Some(count) = rule.count.as_ref() {\n            let times = if *count == 1 { \"time\" } else { \"times\" };\n            parts.push(format!(\"for {} {}\", count, times));\n        }\n\n        parts.join(\" \")\n    }\n\n    fn format_frequency(&self, freq: &ArchivedICalendarFrequency, interval: u16) -> String {\n        let (singular, plural) = match freq {\n            ArchivedICalendarFrequency::Daily => (\"day\", \"days\"),\n            ArchivedICalendarFrequency::Weekly => (\"week\", \"weeks\"),\n            ArchivedICalendarFrequency::Monthly => (\"month\", \"months\"),\n            ArchivedICalendarFrequency::Yearly => (\"year\", \"years\"),\n            ArchivedICalendarFrequency::Hourly => (\"hour\", \"hours\"),\n            ArchivedICalendarFrequency::Minutely => (\"minute\", \"minutes\"),\n            ArchivedICalendarFrequency::Secondly => (\"second\", \"seconds\"),\n        };\n\n        if interval == 1 {\n            format!(\"Every {}\", singular)\n        } else {\n            format!(\"Every {} {}\", interval, plural)\n        }\n    }\n\n    fn format_by_day(&self, days: &[ArchivedICalendarDay]) -> String {\n        let day_names: Vec<String> = days.iter().map(|day| self.format_day(day)).collect();\n\n        format!(\"on {}\", self.format_list(&day_names))\n    }\n\n    fn format_day(&self, day: &ArchivedICalendarDay) -> String {\n        let day_name = match day.weekday {\n            ArchivedICalendarWeekday::Monday => \"Monday\",\n            ArchivedICalendarWeekday::Tuesday => \"Tuesday\",\n            ArchivedICalendarWeekday::Wednesday => \"Wednesday\",\n            ArchivedICalendarWeekday::Thursday => \"Thursday\",\n            ArchivedICalendarWeekday::Friday => \"Friday\",\n            ArchivedICalendarWeekday::Saturday => \"Saturday\",\n            ArchivedICalendarWeekday::Sunday => \"Sunday\",\n        };\n\n        if let Some(occurrence) = day.ordwk.as_ref().map(|o| o.to_native()) {\n            if occurrence > 0 {\n                format!(\"the {} {}\", self.ordinal(occurrence as u32), day_name)\n            } else {\n                format!(\n                    \"the {} {} from the end\",\n                    self.ordinal((-occurrence) as u32),\n                    day_name\n                )\n            }\n        } else {\n            day_name.to_string()\n        }\n    }\n\n    fn format_time_constraints(&self, hours: &[u8], minutes: &[u8]) -> String {\n        let mut time_parts = Vec::new();\n\n        if !hours.is_empty() && !minutes.is_empty() {\n            // Combine hours and minutes\n            for &hour in hours {\n                for &minute in minutes {\n                    time_parts.push(format!(\"{}:{:02}\", self.format_hour(hour), minute));\n                }\n            }\n        } else if !hours.is_empty() {\n            for &hour in hours {\n                time_parts.push(self.format_hour(hour));\n            }\n        } else if !minutes.is_empty() {\n            for &minute in minutes {\n                time_parts.push(format!(\":{:02}\", minute));\n            }\n        }\n\n        if !time_parts.is_empty() {\n            format!(\"at {}\", self.format_list(&time_parts))\n        } else {\n            String::new()\n        }\n    }\n\n    fn format_hour(&self, hour: u8) -> String {\n        match hour {\n            0 => \"12:00 AM\".to_string(),\n            1..=11 => format!(\"{}:00 AM\", hour),\n            12 => \"12:00 PM\".to_string(),\n            13..=23 => format!(\"{}:00 PM\", hour - 12),\n            _ => format!(\"{:02}:00\", hour),\n        }\n    }\n\n    fn format_month_days(&self, days: &[i8]) -> String {\n        let day_strings: Vec<String> = days\n            .iter()\n            .map(|&day| {\n                if day > 0 {\n                    self.ordinal(day as u32)\n                } else {\n                    format!(\"{} from the end\", self.ordinal((-day) as u32))\n                }\n            })\n            .collect();\n\n        format!(\"on the {}\", self.format_list(&day_strings))\n    }\n\n    fn format_months(&self, months: &[ArchivedICalendarMonth]) -> String {\n        let month_names: Vec<String> = months\n            .iter()\n            .map(|month| self.month_name(month.month()))\n            .collect();\n\n        format!(\"in {}\", self.format_list(&month_names))\n    }\n\n    fn format_year_days(&self, days: &[i16_le]) -> String {\n        let day_strings: Vec<String> = days\n            .iter()\n            .map(|&day| {\n                if day > 0 {\n                    format!(\"day {} of the year\", day)\n                } else {\n                    format!(\"day {} from the end of the year\", -day)\n                }\n            })\n            .collect();\n\n        format!(\"on {}\", self.format_list(&day_strings))\n    }\n\n    fn format_week_numbers(&self, weeks: &[i8]) -> String {\n        let week_strings: Vec<String> = weeks\n            .iter()\n            .map(|&week| {\n                if week > 0 {\n                    format!(\"week {}\", week)\n                } else {\n                    format!(\"week {} from the end\", -week)\n                }\n            })\n            .collect();\n\n        format!(\"in {}\", self.format_list(&week_strings))\n    }\n\n    fn format_set_positions(&self, positions: &[i32_le]) -> String {\n        let pos_strings: Vec<String> = positions\n            .iter()\n            .map(|&pos| {\n                if pos > 0 {\n                    self.ordinal(pos.to_native() as u32)\n                } else {\n                    format!(\"{} from the end\", self.ordinal((-pos) as u32))\n                }\n            })\n            .collect();\n\n        format!(\n            \"limited to the {} occurrence\",\n            self.format_list(&pos_strings)\n        )\n    }\n\n    fn format_list(&self, items: &[String]) -> String {\n        match items.len() {\n            0 => String::new(),\n            1 => items[0].clone(),\n            2 => format!(\"{} and {}\", items[0], items[1]),\n            _ => {\n                let rest = &items[..items.len() - 1];\n                format!(\"{}, and {}\", rest.join(\", \"), items.last().unwrap())\n            }\n        }\n    }\n\n    fn ordinal(&self, n: u32) -> String {\n        let suffix = match n % 100 {\n            11..=13 => \"th\",\n            _ => match n % 10 {\n                1 => \"st\",\n                2 => \"nd\",\n                3 => \"rd\",\n                _ => \"th\",\n            },\n        };\n        format!(\"{}{}\", n, suffix)\n    }\n\n    fn month_name(&self, month: u8) -> String {\n        match month {\n            1 => \"January\",\n            2 => \"February\",\n            3 => \"March\",\n            4 => \"April\",\n            5 => \"May\",\n            6 => \"June\",\n            7 => \"July\",\n            8 => \"August\",\n            9 => \"September\",\n            10 => \"October\",\n            11 => \"November\",\n            12 => \"December\",\n            _ => \"Unknown\",\n        }\n        .to_string()\n    }\n\n    /*fn format_datetime(&self, dt: &PartialDateTime) -> String {\n        format!(\"{:?}\", dt)\n    }*/\n}\n"
  },
  {
    "path": "crates/services/src/task_manager/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::task_manager::{IndexAction, Task};\nuse common::{Server, auth::AccessToken};\nuse directory::{Type, backend::internal::manage::ManageDirectory};\nuse email::{cache::MessageCacheFetch, message::metadata::MessageMetadata};\nuse groupware::{cache::GroupwareCache, calendar::CalendarEvent, contact::ContactCard};\nuse std::cmp::Ordering;\nuse store::{\n    IterateParams, SerializeInfallible, ValueKey,\n    ahash::AHashMap,\n    roaring::RoaringBitmap,\n    search::{IndexDocument, SearchField, SearchFilter, SearchQuery},\n    write::{\n        AlignedBytes, Archive, BatchBuilder, SearchIndex, TaskEpoch, TaskQueueClass,\n        TelemetryClass, ValueClass, key::DeserializeBigEndian,\n    },\n};\nuse trc::{AddContext, TaskQueueEvent};\nuse types::{\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection},\n    field::EmailField,\n};\n\npub(crate) trait SearchIndexTask: Sync + Send {\n    fn index(\n        &self,\n        tasks: &[Task<IndexAction>],\n    ) -> impl Future<Output = Vec<IndexTaskResult>> + Send;\n}\n\npub trait ReindexIndexTask: Sync + Send {\n    fn reindex(\n        &self,\n        index: SearchIndex,\n        account_id: Option<u32>,\n        tenant_id: Option<u32>,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n}\n\nconst NUM_INDEXES: usize = 5;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\nenum TaskType {\n    Insert,\n    Delete,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\nenum TaskStatus {\n    Success,\n    Failed,\n    Ignored,\n}\n\n#[derive(Debug)]\npub(crate) struct IndexTaskResult {\n    index: SearchIndex,\n    task_type: TaskType,\n    status: TaskStatus,\n}\n\nimpl SearchIndexTask for Server {\n    async fn index(&self, tasks: &[Task<IndexAction>]) -> Vec<IndexTaskResult> {\n        let mut results: Vec<IndexTaskResult> = Vec::with_capacity(tasks.len());\n        let mut batch = BatchBuilder::new();\n        let mut document_insertions = Vec::new();\n        let mut document_deletions: [AHashMap<u32, Vec<u32>>; NUM_INDEXES] =\n            std::array::from_fn(|_| AHashMap::new());\n\n        for task in tasks {\n            if task.action.is_insert {\n                let document = match task.action.index {\n                    SearchIndex::Email => {\n                        build_email_document(self, task.account_id, task.document_id).await\n                    }\n                    SearchIndex::Calendar => {\n                        build_calendar_document(self, task.account_id, task.document_id).await\n                    }\n                    SearchIndex::Contacts => {\n                        build_contact_document(self, task.account_id, task.document_id).await\n                    }\n                    SearchIndex::File => {\n                        // File indexing not implemented yet\n                        continue;\n                    }\n                    SearchIndex::Tracing => {\n                        build_tracing_span_document(self, task.account_id, task.document_id).await\n                    }\n                    SearchIndex::InMemory => unreachable!(),\n                };\n\n                let result = match document {\n                    Ok(Some(doc)) if !doc.is_empty() => {\n                        document_insertions.push(doc);\n                        TaskStatus::Success\n                    }\n                    Err(err) => {\n                        trc::error!(\n                            err.account_id(task.account_id)\n                                .document_id(task.document_id)\n                                .caused_by(trc::location!())\n                                .ctx(trc::Key::Collection, task.action.index.name())\n                                .details(\"Failed to build document for indexing\")\n                        );\n                        TaskStatus::Failed\n                    }\n                    _ => {\n                        trc::event!(\n                            TaskQueue(TaskQueueEvent::TaskIgnored),\n                            Collection = task.action.index.name(),\n                            Reason = \"Nothing to index\",\n                            AccountId = task.account_id,\n                            DocumentId = task.document_id,\n                        );\n                        TaskStatus::Ignored\n                    }\n                };\n\n                results.push(IndexTaskResult {\n                    task_type: TaskType::Insert,\n                    index: task.action.index,\n                    status: result,\n                });\n            } else {\n                let idx = match task.action.index {\n                    SearchIndex::Email => {\n                        if let Err(err) = delete_email_metadata(\n                            self,\n                            &mut batch,\n                            task.account_id,\n                            task.document_id,\n                        )\n                        .await\n                        {\n                            trc::error!(\n                                err.account_id(task.account_id)\n                                    .document_id(task.document_id)\n                                    .caused_by(trc::location!())\n                                    .details(\"Failed to delete email metadata from index\")\n                            );\n                            results.push(IndexTaskResult {\n                                task_type: TaskType::Delete,\n                                index: task.action.index,\n                                status: TaskStatus::Failed,\n                            });\n                            continue;\n                        }\n                        0\n                    }\n                    SearchIndex::Calendar => 1,\n                    SearchIndex::Contacts => 2,\n                    SearchIndex::File => 3,\n                    SearchIndex::Tracing | SearchIndex::InMemory => unreachable!(),\n                };\n\n                document_deletions[idx]\n                    .entry(task.account_id)\n                    .or_default()\n                    .push(task.document_id);\n\n                results.push(IndexTaskResult {\n                    task_type: TaskType::Delete,\n                    index: task.action.index,\n                    status: TaskStatus::Success,\n                });\n            }\n        }\n\n        // Commit deletion batch to data store\n        if !batch.is_empty()\n            && let Err(err) = self.store().write(batch.build_all()).await\n        {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to commit index deletions to data store\")\n            );\n            for r in results.iter_mut() {\n                if r.task_type == TaskType::Delete\n                    && r.status == TaskStatus::Success\n                    && r.index == SearchIndex::Email\n                {\n                    r.status = TaskStatus::Failed;\n                }\n            }\n            return results;\n        }\n\n        // Index documents\n        if !document_insertions.is_empty()\n            && let Err(err) = self.search_store().index(document_insertions).await\n        {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to index documents\")\n            );\n            for r in results.iter_mut() {\n                if r.task_type == TaskType::Insert && r.status == TaskStatus::Success {\n                    r.status = TaskStatus::Failed;\n                }\n            }\n            return results;\n        }\n\n        // Delete documents\n        for (accounts, index) in document_deletions.into_iter().zip([\n            SearchIndex::Email,\n            SearchIndex::Calendar,\n            SearchIndex::Contacts,\n        ]) {\n            let multi_account = match accounts.len().cmp(&1) {\n                Ordering::Greater => true,\n                Ordering::Equal => false,\n                Ordering::Less => continue,\n            };\n\n            let mut query = SearchQuery::new(index);\n            if multi_account {\n                query.add_filter(SearchFilter::Or);\n            }\n\n            for (account_id, document_ids) in accounts {\n                let multi_document = document_ids.len() > 1;\n                query\n                    .add_filter(SearchFilter::And)\n                    .add_filter(SearchFilter::eq(SearchField::AccountId, account_id));\n\n                if multi_document {\n                    query.add_filter(SearchFilter::Or);\n                }\n\n                for document_id in document_ids {\n                    query.add_filter(SearchFilter::eq(SearchField::DocumentId, document_id));\n                }\n\n                if multi_document {\n                    query.add_filter(SearchFilter::End);\n                }\n                query.add_filter(SearchFilter::End);\n            }\n\n            if multi_account {\n                query.add_filter(SearchFilter::End);\n            }\n\n            if let Err(err) = self.search_store().unindex(query).await {\n                trc::error!(\n                    err.caused_by(trc::location!())\n                        .details(\"Failed to delete documents from index\")\n                        .ctx(trc::Key::Collection, index.name())\n                );\n                for r in results.iter_mut() {\n                    if r.task_type == TaskType::Delete && r.status == TaskStatus::Success {\n                        r.status = TaskStatus::Failed;\n                    }\n                }\n                return results;\n            }\n        }\n\n        results\n    }\n}\n\nimpl ReindexIndexTask for Server {\n    async fn reindex(\n        &self,\n        index: SearchIndex,\n        account_id: Option<u32>,\n        tenant_id: Option<u32>,\n    ) -> trc::Result<()> {\n        let accounts = if let Some(account_id) = account_id {\n            RoaringBitmap::from_sorted_iter([account_id]).unwrap()\n        } else {\n            let mut accounts = RoaringBitmap::new();\n            for principal in self\n                .core\n                .storage\n                .data\n                .list_principals(\n                    None,\n                    tenant_id,\n                    &[Type::Individual, Type::Group],\n                    false,\n                    0,\n                    0,\n                )\n                .await\n                .caused_by(trc::location!())?\n                .items\n            {\n                accounts.insert(principal.id());\n            }\n            accounts\n        };\n        let due = TaskEpoch::now();\n\n        match index {\n            SearchIndex::Email => {\n                for account_id in accounts {\n                    let mut batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(account_id)\n                        .with_collection(Collection::Email);\n\n                    for document_id in self\n                        .get_cached_messages(account_id)\n                        .await\n                        .caused_by(trc::location!())?\n                        .emails\n                        .items\n                        .iter()\n                        .map(|v| v.document_id)\n                    {\n                        batch.with_document(document_id).set(\n                            ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                                due,\n                                index: SearchIndex::Email,\n                                is_insert: true,\n                            }),\n                            0u64.serialize(),\n                        );\n\n                        if batch.len() >= 2000 {\n                            self.core.storage.data.write(batch.build_all()).await?;\n                            batch = BatchBuilder::new();\n                            batch\n                                .with_account_id(account_id)\n                                .with_collection(Collection::Email);\n                        }\n                    }\n\n                    if !batch.is_empty() {\n                        self.core.storage.data.write(batch.build_all()).await?;\n                    }\n                }\n            }\n            SearchIndex::Calendar | SearchIndex::Contacts => {\n                for account_id in accounts {\n                    let cache = self\n                        .fetch_dav_resources(\n                            &AccessToken::from_id(account_id).with_tenant_id(tenant_id),\n                            account_id,\n                            if index == SearchIndex::Calendar {\n                                SyncCollection::Calendar\n                            } else {\n                                SyncCollection::AddressBook\n                            },\n                        )\n                        .await\n                        .caused_by(trc::location!())?;\n                    let mut batch = BatchBuilder::new();\n                    batch.with_account_id(account_id);\n\n                    for document_id in cache.document_ids(false) {\n                        batch.with_document(document_id).set(\n                            ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                                due,\n                                index,\n                                is_insert: true,\n                            }),\n                            0u64.serialize(),\n                        );\n\n                        if batch.len() >= 2000 {\n                            self.core.storage.data.write(batch.build_all()).await?;\n                            batch = BatchBuilder::new();\n                            batch.with_account_id(account_id);\n                        }\n                    }\n\n                    if !batch.is_empty() {\n                        self.core.storage.data.write(batch.build_all()).await?;\n                    }\n                }\n            }\n            SearchIndex::Tracing => {\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n\n                #[cfg(feature = \"enterprise\")]\n                if let Some(store) = self\n                    .core\n                    .enterprise\n                    .as_ref()\n                    .and_then(|e| e.trace_store.as_ref())\n                {\n                    let mut spans = Vec::new();\n                    store\n                        .store\n                        .iterate(\n                            IterateParams::new(\n                                ValueKey::from(ValueClass::Telemetry(TelemetryClass::Span {\n                                    span_id: 0,\n                                })),\n                                ValueKey::from(ValueClass::Telemetry(TelemetryClass::Span {\n                                    span_id: u64::MAX,\n                                })),\n                            )\n                            .no_values(),\n                            |key, _| {\n                                spans.push(key.deserialize_be_u64(0)?);\n                                Ok(true)\n                            },\n                        )\n                        .await\n                        .caused_by(trc::location!())?;\n\n                    let mut batch = BatchBuilder::new();\n                    for span_id in spans {\n                        batch\n                            .with_account_id((span_id >> 32) as u32) // TODO: This is hacky, improve\n                            .with_document(span_id as u32)\n                            .set(\n                                ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                                    due: TaskEpoch::now(),\n                                    index: SearchIndex::Tracing,\n                                    is_insert: true,\n                                }),\n                                vec![],\n                            );\n                        if batch.len() >= 2000 {\n                            self.core.storage.data.write(batch.build_all()).await?;\n                            batch = BatchBuilder::new();\n                        }\n                    }\n\n                    if !batch.is_empty() {\n                        self.core.storage.data.write(batch.build_all()).await?;\n                    }\n                }\n\n                // SPDX-SnippetEnd\n            }\n            SearchIndex::File | SearchIndex::InMemory => (),\n        }\n\n        // Request indexing\n        self.notify_task_queue();\n\n        Ok(())\n    }\n}\n\nasync fn build_email_document(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n) -> trc::Result<Option<IndexDocument>> {\n    let Some(index_fields) = server.core.jmap.index_fields.get(&SearchIndex::Email) else {\n        return Ok(None);\n    };\n\n    match server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n            account_id,\n            Collection::Email,\n            document_id,\n            EmailField::Metadata,\n        ))\n        .await?\n    {\n        Some(metadata_) => {\n            let metadata = metadata_\n                .unarchive::<MessageMetadata>()\n                .caused_by(trc::location!())?;\n\n            let raw_message = server\n                .blob_store()\n                .get_blob(metadata.blob_hash.0.as_slice(), 0..usize::MAX)\n                .await\n                .caused_by(trc::location!())?\n                .ok_or_else(|| {\n                    trc::StoreEvent::NotFound\n                        .into_err()\n                        .details(\"Blob not found\")\n                })?;\n\n            Ok(Some(metadata.index_document(\n                account_id,\n                document_id,\n                &raw_message,\n                index_fields,\n                server.core.jmap.default_language,\n            )))\n        }\n        None => Ok(None),\n    }\n}\n\nasync fn build_calendar_document(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n) -> trc::Result<Option<IndexDocument>> {\n    let Some(index_fields) = server.core.jmap.index_fields.get(&SearchIndex::Calendar) else {\n        return Ok(None);\n    };\n\n    match server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::CalendarEvent,\n            document_id,\n        ))\n        .await?\n    {\n        Some(metadata_) => Ok(Some(\n            metadata_\n                .unarchive::<CalendarEvent>()\n                .caused_by(trc::location!())?\n                .index_document(\n                    account_id,\n                    document_id,\n                    index_fields,\n                    server.core.jmap.default_language,\n                ),\n        )),\n        None => Ok(None),\n    }\n}\n\nasync fn build_contact_document(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n) -> trc::Result<Option<IndexDocument>> {\n    let Some(index_fields) = server.core.jmap.index_fields.get(&SearchIndex::Contacts) else {\n        return Ok(None);\n    };\n\n    match server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n            account_id,\n            Collection::ContactCard,\n            document_id,\n        ))\n        .await?\n    {\n        Some(metadata_) => Ok(Some(\n            metadata_\n                .unarchive::<ContactCard>()\n                .caused_by(trc::location!())?\n                .index_document(\n                    account_id,\n                    document_id,\n                    index_fields,\n                    server.core.jmap.default_language,\n                ),\n        )),\n        None => Ok(None),\n    }\n}\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n\n#[cfg(feature = \"enterprise\")]\nasync fn build_tracing_span_document(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n) -> trc::Result<Option<IndexDocument>> {\n    use common::telemetry::tracers::store::{TracingStore, build_span_document};\n\n    let Some(index_fields) = server.core.jmap.index_fields.get(&SearchIndex::Tracing) else {\n        return Ok(None);\n    };\n    let Some(store) = server\n        .core\n        .enterprise\n        .as_ref()\n        .and_then(|e| e.trace_store.as_ref())\n    else {\n        return Ok(None);\n    };\n\n    let span_id = ((account_id as u64) << 32) | document_id as u64;\n    let span = store.store.get_span(span_id).await?;\n\n    if !span.is_empty() {\n        Ok(Some(build_span_document(span_id, span, index_fields)))\n    } else {\n        Ok(None)\n    }\n}\n\n// SPDX-SnippetEnd\n\n#[cfg(not(feature = \"enterprise\"))]\nasync fn build_tracing_span_document(\n    _: &Server,\n    _: u32,\n    _: u32,\n) -> trc::Result<Option<IndexDocument>> {\n    Ok(None)\n}\n\nasync fn delete_email_metadata(\n    server: &Server,\n    batch: &mut BatchBuilder,\n    account_id: u32,\n    document_id: u32,\n) -> trc::Result<()> {\n    match server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n            account_id,\n            Collection::Email,\n            document_id,\n            EmailField::Metadata,\n        ))\n        .await?\n    {\n        Some(metadata_) => {\n            batch\n                .with_account_id(account_id)\n                .with_collection(Collection::Email)\n                .with_document(document_id);\n            let metadata = metadata_\n                .unarchive::<MessageMetadata>()\n                .caused_by(trc::location!())?;\n            metadata.unindex(batch);\n\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n\n            // Hold blob for undeletion\n            #[cfg(feature = \"enterprise\")]\n            {\n                use common::enterprise::undelete::DeletedItemType;\n                use email::message::metadata::ArchivedMetadataHeaderName;\n\n                if let Some(undelete) = server\n                    .core\n                    .enterprise\n                    .as_ref()\n                    .and_then(|e| e.undelete.as_ref())\n                {\n                    use common::enterprise::undelete::DeletedItem;\n                    use email::message::metadata::MESSAGE_RECEIVED_MASK;\n                    use store::{\n                        Serialize,\n                        write::{Archiver, BlobLink, BlobOp, now},\n                    };\n\n                    let root_part = metadata.root_part();\n                    let from: Option<Box<str>> = root_part.headers.iter().find_map(|h| {\n                        if let ArchivedMetadataHeaderName::From = &h.name {\n                            h.value.as_single_address().and_then(|addr| {\n                                match (addr.address.as_ref(), addr.name.as_ref()) {\n                                    (Some(address), Some(name)) => {\n                                        Some(format!(\"{} <{}>\", name, address).into_boxed_str())\n                                    }\n                                    (Some(address), None) => Some(address.as_ref().into()),\n                                    (None, Some(name)) => Some(name.as_ref().into()),\n                                    (None, None) => None,\n                                }\n                            })\n                        } else {\n                            None\n                        }\n                    });\n                    let subject: Option<Box<str>> = root_part.headers.iter().rev().find_map(|h| {\n                        if let ArchivedMetadataHeaderName::Subject = &h.name {\n                            h.value.as_text().map(Into::into)\n                        } else {\n                            None\n                        }\n                    });\n                    let now = now();\n                    let until = now + undelete.retention.as_secs();\n                    let blob_hash = BlobHash::from(&metadata.blob_hash);\n                    batch\n                        .set(\n                            BlobOp::Link {\n                                hash: blob_hash.clone(),\n                                to: BlobLink::Temporary { until },\n                            },\n                            vec![BlobLink::UNDELETE_LINK],\n                        )\n                        .set(\n                            BlobOp::Undelete {\n                                hash: blob_hash,\n                                until,\n                            },\n                            Archiver::new(DeletedItem {\n                                typ: DeletedItemType::Email {\n                                    from: from.unwrap_or_default(),\n                                    subject: subject.unwrap_or_default(),\n                                    received_at: metadata.rcvd_attach.to_native()\n                                        & MESSAGE_RECEIVED_MASK,\n                                },\n                                size: root_part.offset_end.to_native(),\n                                deleted_at: now,\n                            })\n                            .serialize()\n                            .caused_by(trc::location!())?,\n                        );\n                }\n            }\n\n            // SPDX-SnippetEnd\n        }\n        None => {\n            trc::event!(\n                TaskQueue(TaskQueueEvent::MetadataNotFound),\n                Details = \"E-mail metadata not found\",\n                AccountId = account_id,\n                DocumentId = document_id,\n            );\n        }\n    }\n\n    Ok(())\n}\n\nimpl IndexTaskResult {\n    pub fn is_done(&self) -> bool {\n        self.status != TaskStatus::Failed\n    }\n}\n"
  },
  {
    "path": "crates/services/src/task_manager/lock.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::task_manager::*;\n\npub(crate) trait TaskLockManager: Sync + Send {\n    fn try_lock_task(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        lock_key: Vec<u8>,\n        lock_expiry: u64,\n    ) -> impl Future<Output = bool> + Send;\n    fn remove_index_lock(&self, lock_key: Vec<u8>) -> impl Future<Output = ()> + Send;\n}\n\nimpl TaskLockManager for Server {\n    async fn try_lock_task(\n        &self,\n        account_id: u32,\n        document_id: u32,\n        lock_key: Vec<u8>,\n        lock_expiry: u64,\n    ) -> bool {\n        match self\n            .in_memory_store()\n            .try_lock(KV_LOCK_TASK, &lock_key, lock_expiry)\n            .await\n        {\n            Ok(result) => {\n                if !result {\n                    trc::event!(\n                        TaskQueue(TaskQueueEvent::TaskLocked),\n                        AccountId = account_id,\n                        DocumentId = document_id,\n                        Expires = trc::Value::Timestamp(now() + lock_expiry),\n                    );\n                }\n                result\n            }\n            Err(err) => {\n                trc::error!(\n                    err.account_id(account_id)\n                        .document_id(document_id)\n                        .details(\"Failed to lock task\")\n                );\n\n                false\n            }\n        }\n    }\n\n    async fn remove_index_lock(&self, lock_key: Vec<u8>) {\n        if let Err(err) = self\n            .in_memory_store()\n            .remove_lock(KV_LOCK_TASK, &lock_key)\n            .await\n        {\n            trc::error!(\n                err.details(\"Failed to unlock task\")\n                    .ctx(trc::Key::Key, lock_key)\n                    .caused_by(trc::location!())\n            );\n        }\n    }\n}\n\npub(crate) trait TaskLock {\n    fn account_id(&self) -> u32;\n    fn document_id(&self) -> u32;\n    fn lock_key(&self) -> Vec<u8>;\n    fn lock_expiry(&self) -> u64;\n    fn value_classes(&self) -> impl Iterator<Item = ValueClass>;\n}\n\nimpl TaskLock for Task<IndexAction> {\n    fn account_id(&self) -> u32 {\n        self.account_id\n    }\n\n    fn document_id(&self) -> u32 {\n        self.document_id\n    }\n\n    fn lock_key(&self) -> Vec<u8> {\n        KeySerializer::new((U32_LEN * 2) + U64_LEN + 2)\n            .write(0u8)\n            .write(self.due.inner())\n            .write_leb128(self.account_id)\n            .write_leb128(self.document_id)\n            .write(self.action.index.to_u8())\n            .finalize()\n    }\n\n    fn lock_expiry(&self) -> u64 {\n        INDEX_EXPIRY\n    }\n\n    fn value_classes(&self) -> impl Iterator<Item = ValueClass> {\n        std::iter::once(ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n            due: self.due,\n            index: self.action.index,\n            is_insert: self.action.is_insert,\n        }))\n    }\n}\n\nimpl TaskLock for Task<CalendarAlarm> {\n    fn account_id(&self) -> u32 {\n        self.account_id\n    }\n\n    fn document_id(&self) -> u32 {\n        self.document_id\n    }\n\n    fn lock_key(&self) -> Vec<u8> {\n        KeySerializer::new((U32_LEN * 2) + U64_LEN + 1)\n            .write(2u8)\n            .write(self.due.inner())\n            .write_leb128(self.account_id)\n            .write_leb128(self.document_id)\n            .finalize()\n    }\n\n    fn lock_expiry(&self) -> u64 {\n        ALARM_EXPIRY\n    }\n\n    fn value_classes(&self) -> impl Iterator<Item = ValueClass> {\n        std::iter::once(ValueClass::TaskQueue(TaskQueueClass::SendAlarm {\n            event_id: self.action.event_id,\n            alarm_id: self.action.alarm_id,\n            due: self.due,\n            is_email_alert: matches!(self.action.typ, CalendarAlarmType::Email { .. }),\n        }))\n    }\n}\n\nimpl TaskLock for Task<ImipAction> {\n    fn account_id(&self) -> u32 {\n        self.account_id\n    }\n\n    fn document_id(&self) -> u32 {\n        self.document_id\n    }\n\n    fn lock_key(&self) -> Vec<u8> {\n        KeySerializer::new((U32_LEN * 2) + U64_LEN + 1)\n            .write(3u8)\n            .write(self.due.inner())\n            .write_leb128(self.account_id)\n            .write_leb128(self.document_id)\n            .finalize()\n    }\n\n    fn lock_expiry(&self) -> u64 {\n        ALARM_EXPIRY\n    }\n\n    fn value_classes(&self) -> impl Iterator<Item = ValueClass> {\n        [\n            ValueClass::TaskQueue(TaskQueueClass::SendImip {\n                due: self.due,\n                is_payload: false,\n            }),\n            ValueClass::TaskQueue(TaskQueueClass::SendImip {\n                due: self.due,\n                is_payload: true,\n            }),\n        ]\n        .into_iter()\n    }\n}\n\nimpl TaskLock for Task<MergeThreadIds<AHashSet<u32>>> {\n    fn account_id(&self) -> u32 {\n        self.account_id\n    }\n\n    fn document_id(&self) -> u32 {\n        self.document_id\n    }\n\n    fn lock_key(&self) -> Vec<u8> {\n        KeySerializer::new((U32_LEN * 2) + U64_LEN + 1)\n            .write(4u8)\n            .write(self.due.inner())\n            .write_leb128(self.account_id)\n            .write_leb128(self.document_id)\n            .finalize()\n    }\n\n    fn lock_expiry(&self) -> u64 {\n        ALARM_EXPIRY\n    }\n\n    fn value_classes(&self) -> impl Iterator<Item = ValueClass> {\n        std::iter::once(ValueClass::TaskQueue(TaskQueueClass::MergeThreads {\n            due: self.due,\n        }))\n    }\n}\n\nimpl Task<TaskAction> {\n    pub(crate) fn lock_expiry(&self) -> u64 {\n        match &self.action {\n            TaskAction::UpdateIndex(_) => INDEX_EXPIRY,\n            TaskAction::SendAlarm(_) => ALARM_EXPIRY,\n            _ => ALARM_EXPIRY,\n        }\n    }\n\n    pub fn deserialize(key: &[u8], value: &[u8]) -> trc::Result<Self> {\n        let document_id = key.deserialize_be_u32(U64_LEN + U32_LEN + 1)?;\n\n        Ok(Task {\n            due: TaskEpoch::from_inner(key.deserialize_be_u64(0)?),\n            account_id: key.deserialize_be_u32(U64_LEN)?,\n            document_id,\n            action: match key.get(U64_LEN + U32_LEN) {\n                Some(v @ (7 | 8)) => TaskAction::UpdateIndex(IndexAction {\n                    index: key\n                        .last()\n                        .copied()\n                        .and_then(SearchIndex::try_from_u8)\n                        .ok_or_else(|| trc::Error::corrupted_key(key, None, trc::location!()))?,\n                    is_insert: *v == 7,\n                }),\n                Some(3) => TaskAction::SendAlarm(CalendarAlarm {\n                    event_id: key.deserialize_be_u16(U64_LEN + U32_LEN + U32_LEN + 1)?,\n                    alarm_id: key.deserialize_be_u16(U64_LEN + U32_LEN + U32_LEN + U16_LEN + 1)?,\n                    alarm_time: 0,\n                    typ: CalendarAlarmType::Email {\n                        event_start: value.deserialize_be_u64(0)? as i64,\n                        event_end: value.deserialize_be_u64(U64_LEN)? as i64,\n                        event_start_tz: value.deserialize_be_u16(U64_LEN * 2)?,\n                        event_end_tz: value.deserialize_be_u16((U64_LEN * 2) + U16_LEN)?,\n                    },\n                }),\n                Some(6) => {\n                    let recurrence_id = value.deserialize_be_u64(0)? as i64;\n\n                    TaskAction::SendAlarm(CalendarAlarm {\n                        event_id: key.deserialize_be_u16(U64_LEN + U32_LEN + U32_LEN + 1)?,\n                        alarm_id: key\n                            .deserialize_be_u16(U64_LEN + U32_LEN + U32_LEN + U16_LEN + 1)?,\n                        alarm_time: 0,\n                        typ: CalendarAlarmType::Display {\n                            recurrence_id: if recurrence_id != 0 {\n                                Some(recurrence_id)\n                            } else {\n                                None\n                            },\n                        },\n                    })\n                }\n                Some(4) => TaskAction::SendImip,\n                Some(9) => {\n                    TaskAction::MergeThreads(MergeThreadIds::deserialize(value).ok_or_else(\n                        || trc::Error::corrupted_key(key, value.into(), trc::location!()),\n                    )?)\n                }\n                _ => return Err(trc::Error::corrupted_key(key, None, trc::location!())),\n            },\n        })\n    }\n}\n"
  },
  {
    "path": "crates/services/src/task_manager/merge_threads.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{Server, storage::index::ObjectIndexBuilder};\nuse email::message::{\n    ingest::{MergeThreadIds, ThreadMerge},\n    metadata::MessageData,\n};\nuse std::time::Duration;\nuse store::{\n    IndexKeyPrefix, IterateParams, U32_LEN, ValueKey,\n    ahash::{AHashMap, AHashSet},\n    rand::Rng,\n    write::{\n        AlignedBytes, Archive, BatchBuilder, IndexPropertyClass, ValueClass,\n        key::DeserializeBigEndian,\n    },\n};\nuse trc::AddContext;\nuse types::{\n    collection::{Collection, SyncCollection},\n    field::EmailField,\n};\n\nconst MAX_RETRIES: usize = 5;\n\npub trait MergeThreadsTask: Sync + Send {\n    fn merge_threads(\n        &self,\n        account_id: u32,\n        threads: &MergeThreadIds<AHashSet<u32>>,\n    ) -> impl Future<Output = bool> + Send;\n}\n\nimpl MergeThreadsTask for Server {\n    async fn merge_threads(\n        &self,\n        account_id: u32,\n        threads: &MergeThreadIds<AHashSet<u32>>,\n    ) -> bool {\n        match merge_threads(self, account_id, threads).await {\n            Ok(_) => true,\n            Err(err) => {\n                trc::error!(\n                    err.account_id(account_id)\n                        .details(\"Failed to merge threads\")\n                );\n                false\n            }\n        }\n    }\n}\n\nasync fn merge_threads(\n    server: &Server,\n    account_id: u32,\n    merge_threads: &MergeThreadIds<AHashSet<u32>>,\n) -> trc::Result<()> {\n    let key_len = IndexKeyPrefix::len() + merge_threads.thread_hash.len() + U32_LEN;\n    let document_id_pos = key_len - U32_LEN;\n    let mut thread_merge = ThreadMerge::new();\n    let mut thread_index = AHashMap::new();\n    let mut try_count = 0;\n\n    'retry: loop {\n        // Find thread ids\n        server\n            .store()\n            .iterate(\n                IterateParams::new(\n                    ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id: 0,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                            property: EmailField::Threading.into(),\n                            hash: merge_threads.thread_hash,\n                        }),\n                    },\n                    ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id: u32::MAX,\n                        class: ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                            property: EmailField::Threading.into(),\n                            hash: merge_threads.thread_hash,\n                        }),\n                    },\n                )\n                .ascending(),\n                |key, value| {\n                    if key.len() == key_len {\n                        let thread_id = value.deserialize_be_u32(0)?;\n                        if merge_threads.merge_ids.contains(&thread_id) {\n                            let document_id = key.deserialize_be_u32(document_id_pos)?;\n\n                            thread_merge.add(thread_id, document_id);\n                            thread_index.insert(document_id, value.to_vec());\n                        }\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if thread_merge.num_thread_ids() < 2 {\n            // Another process merged the threads already?\n            return Ok(());\n        }\n        let thread_id = thread_merge.merge_thread_id();\n\n        // Delete all but the most common threadId\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Thread);\n\n        for &delete_thread_id in thread_merge.thread_ids() {\n            if delete_thread_id != thread_id {\n                batch\n                    .with_document(delete_thread_id)\n                    .log_container_delete(SyncCollection::Thread);\n            }\n        }\n\n        // Move messages to the new threadId\n        batch.with_collection(Collection::Email);\n\n        for (&group_thread_id, document_ids) in thread_merge.thread_groups() {\n            if thread_id != group_thread_id {\n                for &document_id in document_ids {\n                    if let Some(data_) = server\n                        .store()\n                        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                            account_id,\n                            Collection::Email,\n                            document_id,\n                        ))\n                        .await\n                        .caused_by(trc::location!())?\n                    {\n                        let data = data_\n                            .to_unarchived::<MessageData>()\n                            .caused_by(trc::location!())?;\n                        if data.inner.thread_id != group_thread_id {\n                            try_count += 1;\n                            continue 'retry;\n                        }\n\n                        // Update thread id\n                        let mut new_data = data\n                            .deserialize::<MessageData>()\n                            .caused_by(trc::location!())?;\n                        new_data.thread_id = thread_id;\n                        batch\n                            .with_document(document_id)\n                            .custom(\n                                ObjectIndexBuilder::new()\n                                    .with_current(data)\n                                    .with_changes(new_data),\n                            )\n                            .caused_by(trc::location!())?;\n\n                        // Update thread index property\n                        let mut thread_index = thread_index.remove(&document_id).unwrap();\n                        thread_index[0..U32_LEN].copy_from_slice(&thread_id.to_be_bytes());\n                        batch.set(\n                            ValueClass::IndexProperty(IndexPropertyClass::Hash {\n                                property: EmailField::Threading.into(),\n                                hash: merge_threads.thread_hash,\n                            }),\n                            thread_index,\n                        );\n                    }\n                }\n            }\n        }\n\n        match server.commit_batch(batch).await {\n            Ok(_) => return Ok(()),\n            Err(err) if err.is_assertion_failure() && try_count < MAX_RETRIES => {\n                let backoff = store::rand::rng().random_range(50..=300);\n                tokio::time::sleep(Duration::from_millis(backoff)).await;\n                try_count += 1;\n            }\n            Err(err) => {\n                return Err(err.caused_by(trc::location!()));\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/services/src/task_manager/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::task_manager::imip::SendImipTask;\nuse crate::task_manager::index::SearchIndexTask;\nuse crate::task_manager::lock::{TaskLock, TaskLockManager};\nuse crate::task_manager::merge_threads::MergeThreadsTask;\nuse alarm::SendAlarmTask;\nuse common::IPC_CHANNEL_BUFFER;\nuse common::config::server::ServerProtocol;\nuse common::listener::limiter::ConcurrencyLimiter;\nuse common::listener::{ServerInstance, TcpAcceptor};\nuse common::{Inner, KV_LOCK_TASK, Server, core::BuildServer};\nuse email::message::ingest::MergeThreadIds;\nuse groupware::calendar::alarm::{CalendarAlarm, CalendarAlarmType};\nuse std::collections::hash_map::Entry;\nuse std::future::Future;\nuse std::time::Duration;\nuse std::{sync::Arc, time::Instant};\nuse store::ahash::AHashSet;\nuse store::rand;\nuse store::rand::seq::SliceRandom;\nuse store::write::{SearchIndex, TaskEpoch};\nuse store::{\n    IterateParams, U16_LEN, U32_LEN, U64_LEN, ValueKey,\n    ahash::AHashMap,\n    write::{\n        BatchBuilder, TaskQueueClass, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n        now,\n    },\n};\nuse tokio::sync::{mpsc, watch};\nuse trc::TaskQueueEvent;\nuse utils::snowflake::SnowflakeIdGenerator;\n\npub mod alarm;\npub mod imip;\npub mod index;\npub mod lock;\npub mod merge_threads;\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq)]\npub struct Task<T> {\n    pub account_id: u32,\n    pub document_id: u32,\n    pub due: TaskEpoch,\n    pub action: T,\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq)]\npub enum TaskAction {\n    UpdateIndex(IndexAction),\n    SendAlarm(CalendarAlarm),\n    SendImip,\n    MergeThreads(MergeThreadIds<AHashSet<u32>>),\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq)]\npub struct IndexAction {\n    pub index: SearchIndex,\n    pub is_insert: bool,\n}\n\n#[derive(Debug, Clone, Hash, PartialEq, Eq)]\npub(crate) struct ImipAction;\n\nconst INDEX_EXPIRY: u64 = 60 * 5; // 5 minutes\nconst ALARM_EXPIRY: u64 = 60 * 2; // 2 minutes\nconst QUEUE_REFRESH_INTERVAL: u64 = 60 * 5; // 5 minutes\n\npub(crate) struct TaskManagerIpc {\n    tx_fts: mpsc::Sender<Task<IndexAction>>,\n    tx_alarm: mpsc::Sender<Task<CalendarAlarm>>,\n    tx_imip: mpsc::Sender<Task<ImipAction>>,\n    tx_threads: mpsc::Sender<Task<MergeThreadIds<AHashSet<u32>>>>,\n    locked: AHashMap<Vec<u8>, Locked>,\n    revision: u64,\n}\n\nstruct Locked {\n    expires: Instant,\n    revision: u64,\n}\n\npub fn spawn_task_manager(inner: Arc<Inner>) {\n    // Create mpsc channels for the different task types\n    let (tx_index_1, mut rx_index_1) = mpsc::channel::<Task<IndexAction>>(IPC_CHANNEL_BUFFER);\n    let (tx_index_2, mut rx_index_2) = mpsc::channel::<Task<CalendarAlarm>>(IPC_CHANNEL_BUFFER);\n    let (tx_index_3, mut rx_index_3) = mpsc::channel::<Task<ImipAction>>(IPC_CHANNEL_BUFFER);\n    let (tx_index_4, mut rx_index_4) =\n        mpsc::channel::<Task<MergeThreadIds<AHashSet<u32>>>>(IPC_CHANNEL_BUFFER);\n\n    // Create dummy server instance for alarms\n    let server_instance = Arc::new(ServerInstance {\n        id: \"_local\".to_string(),\n        protocol: ServerProtocol::Smtp,\n        acceptor: TcpAcceptor::Plain,\n        limiter: ConcurrencyLimiter::new(100),\n        shutdown_rx: watch::channel(false).1,\n        proxy_networks: vec![],\n        span_id_gen: Arc::new(SnowflakeIdGenerator::new()),\n    });\n\n    // Indexing worker\n    {\n        let inner = inner.clone();\n        tokio::spawn(async move {\n            while let Some(task) = rx_index_1.recv().await {\n                let server = inner.build_server();\n                let batch_size = server.core.jmap.index_batch_size;\n                let mut batch = Vec::with_capacity(batch_size);\n                batch.push(task);\n\n                while batch.len() < batch_size {\n                    match rx_index_1.try_recv() {\n                        Ok(task) => batch.push(task),\n                        Err(_) => break,\n                    }\n                }\n\n                if batch.len() > 1 {\n                    batch.shuffle(&mut rand::rng());\n                }\n\n                // Lock tasks\n                let mut locked_batch = Vec::with_capacity(batch.len());\n                for task in batch {\n                    if server\n                        .try_lock_task(\n                            task.account_id,\n                            task.document_id,\n                            task.lock_key(),\n                            task.lock_expiry(),\n                        )\n                        .await\n                    {\n                        locked_batch.push(task);\n                    }\n                }\n\n                // Dispatch\n                if !locked_batch.is_empty() {\n                    let success = server.index(&locked_batch).await;\n\n                    if success.iter().all(|t| t.is_done()) {\n                        delete_tasks(&server, &locked_batch).await;\n                    } else {\n                        trc::event!(\n                            TaskQueue(TaskQueueEvent::TaskFailed),\n                            Total = locked_batch.len(),\n                            Details = \"Indexing task failed\",\n                        );\n\n                        // Remove successful entries from queue\n                        let mut to_delete = Vec::with_capacity(locked_batch.len());\n                        for (task, result) in locked_batch.into_iter().zip(success.into_iter()) {\n                            if result.is_done() {\n                                to_delete.push(task);\n                            }\n                        }\n                        if !to_delete.is_empty() {\n                            delete_tasks(&server, &to_delete).await;\n                        }\n                    }\n                }\n            }\n        });\n    }\n\n    // Send alarm worker\n    {\n        let inner = inner.clone();\n        let server_instance = server_instance.clone();\n        tokio::spawn(async move {\n            while let Some(task) = rx_index_2.recv().await {\n                let server = inner.build_server();\n\n                // Lock task\n                if server.core.groupware.alarms_enabled\n                    && server\n                        .try_lock_task(\n                            task.account_id,\n                            task.document_id,\n                            task.lock_key(),\n                            task.lock_expiry(),\n                        )\n                        .await\n                {\n                    let success = server\n                        .send_alarm(\n                            task.account_id,\n                            task.document_id,\n                            &task.action,\n                            server_instance.clone(),\n                        )\n                        .await;\n\n                    // Remove entry from queue\n                    if success {\n                        delete_tasks(&server, &[task]).await;\n                    } else {\n                        trc::event!(\n                            TaskQueue(TaskQueueEvent::TaskFailed),\n                            AccountId = task.account_id,\n                            DocumentId = task.document_id,\n                            Details = \"Sending alarm task failed\",\n                        );\n                    }\n                }\n            }\n        });\n    }\n\n    // Send iMIP worker\n    {\n        let inner = inner.clone();\n        let server_instance = server_instance.clone();\n        tokio::spawn(async move {\n            while let Some(task) = rx_index_3.recv().await {\n                let server = inner.build_server();\n\n                // Lock task\n                if server.core.groupware.itip_enabled\n                    && server\n                        .try_lock_task(\n                            task.account_id,\n                            task.document_id,\n                            task.lock_key(),\n                            task.lock_expiry(),\n                        )\n                        .await\n                {\n                    let success = server\n                        .send_imip(\n                            task.account_id,\n                            task.document_id,\n                            task.due,\n                            server_instance.clone(),\n                        )\n                        .await;\n\n                    // Remove entry from queue\n                    if success {\n                        delete_tasks(&server, &[task]).await;\n                    } else {\n                        trc::event!(\n                            TaskQueue(TaskQueueEvent::TaskFailed),\n                            AccountId = task.account_id,\n                            DocumentId = task.document_id,\n                            Details = \"Sending iMIP task failed\",\n                        );\n                    }\n                }\n            }\n        });\n    }\n\n    // Merge threads worker\n    {\n        let inner = inner.clone();\n        tokio::spawn(async move {\n            while let Some(task) = rx_index_4.recv().await {\n                let server = inner.build_server();\n\n                // Lock task\n                if server\n                    .try_lock_task(\n                        task.account_id,\n                        task.document_id,\n                        task.lock_key(),\n                        task.lock_expiry(),\n                    )\n                    .await\n                {\n                    let success = server.merge_threads(task.account_id, &task.action).await;\n\n                    // Remove entry from queue\n                    if success {\n                        delete_tasks(&server, &[task]).await;\n                    } else {\n                        trc::event!(\n                            TaskQueue(TaskQueueEvent::TaskFailed),\n                            AccountId = task.account_id,\n                            DocumentId = task.document_id,\n                            Details = \"Merging threads task failed\",\n                        );\n                    }\n                }\n            }\n        });\n    }\n\n    tokio::spawn(async move {\n        let mut ipc = TaskManagerIpc {\n            tx_fts: tx_index_1,\n            tx_alarm: tx_index_2,\n            tx_imip: tx_index_3,\n            tx_threads: tx_index_4,\n            locked: Default::default(),\n            revision: 0,\n        };\n        let rx = inner.ipc.task_tx.clone();\n        loop {\n            // Index any queued tasks\n            let sleep_for = inner.build_server().process_tasks(&mut ipc).await;\n\n            // Wait for a signal or sleep until the next task is due\n            let _ = tokio::time::timeout(sleep_for, rx.notified()).await;\n        }\n    });\n}\n\npub(crate) trait TaskQueueManager: Sync + Send {\n    fn process_tasks(&self, ipc: &mut TaskManagerIpc) -> impl Future<Output = Duration> + Send;\n}\n\nimpl TaskQueueManager for Server {\n    async fn process_tasks(&self, ipc: &mut TaskManagerIpc) -> Duration {\n        let now_timestamp = now();\n        let from_key = ValueKey::<ValueClass> {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                due: TaskEpoch::from_inner(0),\n                index: SearchIndex::Email,\n                is_insert: true,\n            }),\n        };\n        let to_key = ValueKey::<ValueClass> {\n            account_id: u32::MAX,\n            collection: u8::MAX,\n            document_id: u32::MAX,\n            class: ValueClass::TaskQueue(TaskQueueClass::UpdateIndex {\n                due: TaskEpoch::new(now_timestamp + QUEUE_REFRESH_INTERVAL)\n                    .with_attempt(u16::MAX)\n                    .with_sequence_id(u16::MAX),\n                index: SearchIndex::Email,\n                is_insert: true,\n            }),\n        };\n\n        // Retrieve tasks pending to be processed\n        let mut tasks = Vec::new();\n        let now = Instant::now();\n        let mut next_event = None;\n        ipc.revision += 1;\n        let _ = self\n            .store()\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending(),\n                |key, value| {\n                    let task = Task::deserialize(key, value)?;\n\n                    let task_due = task.due.due();\n                    if task_due <= now_timestamp {\n                        match ipc.locked.entry(key.to_vec()) {\n                            Entry::Occupied(mut entry) => {\n                                let locked = entry.get_mut();\n                                if locked.expires <= now {\n                                    locked.expires = Instant::now()\n                                        + std::time::Duration::from_secs(task.lock_expiry() + 1);\n                                    tasks.push(task);\n                                }\n                                locked.revision = ipc.revision;\n                            }\n                            Entry::Vacant(entry) => {\n                                entry.insert(Locked {\n                                    expires: Instant::now()\n                                        + std::time::Duration::from_secs(task.lock_expiry() + 1),\n                                    revision: ipc.revision,\n                                });\n                                tasks.push(task);\n                            }\n                        }\n\n                        Ok(true)\n                    } else {\n                        next_event = Some(task_due);\n                        Ok(false)\n                    }\n                },\n            )\n            .await\n            .map_err(|err| {\n                trc::error!(\n                    err.caused_by(trc::location!())\n                        .details(\"Failed to iterate over task queue.\")\n                );\n            });\n\n        if !tasks.is_empty() || !ipc.locked.is_empty() {\n            trc::event!(\n                TaskQueue(TaskQueueEvent::TaskAcquired),\n                Total = tasks.len(),\n                Details = ipc.locked.len(),\n            );\n        }\n\n        // Shuffle tasks\n        if tasks.len() > 1 {\n            tasks.shuffle(&mut rand::rng());\n        }\n\n        // Dispatch tasks\n        let roles = &self.core.network.roles;\n        for event in tasks {\n            match event.action {\n                TaskAction::UpdateIndex(index)\n                    if roles.fts_indexing.is_enabled_for_hash(&event) =>\n                {\n                    if ipc\n                        .tx_fts\n                        .send(Task {\n                            account_id: event.account_id,\n                            document_id: event.document_id,\n                            due: event.due,\n                            action: index,\n                        })\n                        .await\n                        .is_err()\n                    {\n                        trc::event!(\n                            Server(trc::ServerEvent::ThreadError),\n                            Details = \"Error sending task.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n                }\n                TaskAction::SendAlarm(alarm)\n                    if roles.calendar_alerts.is_enabled_for_hash(&event) =>\n                {\n                    if ipc\n                        .tx_alarm\n                        .send(Task {\n                            account_id: event.account_id,\n                            document_id: event.document_id,\n                            due: event.due,\n                            action: alarm,\n                        })\n                        .await\n                        .is_err()\n                    {\n                        trc::event!(\n                            Server(trc::ServerEvent::ThreadError),\n                            Details = \"Error sending task.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n                }\n                TaskAction::SendImip if roles.imip_processing.is_enabled_for_hash(&event) => {\n                    if ipc\n                        .tx_imip\n                        .send(Task {\n                            account_id: event.account_id,\n                            document_id: event.document_id,\n                            due: event.due,\n                            action: ImipAction,\n                        })\n                        .await\n                        .is_err()\n                    {\n                        trc::event!(\n                            Server(trc::ServerEvent::ThreadError),\n                            Details = \"Error sending task.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n                }\n                TaskAction::MergeThreads(info)\n                    if roles.merge_threads.is_enabled_for_hash(&event) =>\n                {\n                    if ipc\n                        .tx_threads\n                        .send(Task {\n                            account_id: event.account_id,\n                            document_id: event.document_id,\n                            due: event.due,\n                            action: info,\n                        })\n                        .await\n                        .is_err()\n                    {\n                        trc::event!(\n                            Server(trc::ServerEvent::ThreadError),\n                            Details = \"Error sending task.\",\n                            CausedBy = trc::location!()\n                        );\n                    }\n                }\n                _ => {\n                    trc::event!(\n                        TaskQueue(TaskQueueEvent::TaskIgnored),\n                        Details = event.action.name(),\n                        AccountId = event.account_id,\n                        DocumentId = event.document_id,\n                    );\n\n                    continue;\n                }\n            }\n        }\n\n        // Delete expired locks\n        let now = Instant::now();\n        ipc.locked\n            .retain(|_, locked| locked.expires > now && locked.revision == ipc.revision);\n        Duration::from_secs(next_event.map_or(QUEUE_REFRESH_INTERVAL, |timestamp| {\n            timestamp.saturating_sub(store::write::now())\n        }))\n    }\n}\n\nasync fn delete_tasks<T: TaskLock>(server: &Server, tasks: &[T]) {\n    let mut batch = BatchBuilder::new();\n\n    for task in tasks {\n        batch\n            .with_account_id(task.account_id())\n            .with_document(task.document_id());\n\n        for value in task.value_classes() {\n            batch.clear(value);\n        }\n    }\n\n    if let Err(err) = server.store().write(batch.build_all()).await {\n        trc::error!(err.details(\"Failed to remove task(s) from queue.\"));\n    }\n\n    for task in tasks {\n        server.remove_index_lock(task.lock_key()).await;\n    }\n}\n\nimpl TaskAction {\n    pub fn name(&self) -> &'static str {\n        match self {\n            TaskAction::UpdateIndex(_) => \"UpdateIndex\",\n            TaskAction::SendAlarm(_) => \"SendAlarm\",\n            TaskAction::SendImip => \"SendImip\",\n            TaskAction::MergeThreads(_) => \"MergeThreads\",\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/Cargo.toml",
    "content": "[package]\nname = \"smtp\"\ndescription = \"Stalwart SMTP Server\"\nauthors = [ \"Stalwart Labs LLC <hello@stalw.art>\"]\nrepository = \"https://github.com/stalwartlabs/smtp-server\"\nhomepage = \"https://stalw.art/smtp\"\nkeywords = [\"smtp\", \"email\", \"mail\", \"server\"]\ncategories = [\"email\"]\nlicense = \"AGPL-3.0-only OR LicenseRef-SEL\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nstore = { path =  \"../store\" }\ntypes = { path =  \"../types\" }\nutils = { path =  \"../utils\" }\nnlp = { path =  \"../nlp\" }\ndirectory = { path =  \"../directory\" }\ncommon = { path =  \"../common\" }\nemail = { path =  \"../email\" }\nspam-filter = { path =  \"../spam-filter\" }\ntrc = { path = \"../trc\" }\nmail-auth = { version = \"0.7.1\", features = [\"rkyv\"] }\nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-builder = { version = \"0.4\" } \nsmtp-proto = { version = \"0.2\", features = [\"rkyv\", \"serde\"] }\nsieve-rs = { version = \"0.7\", features = [\"rkyv\"] } \nahash = { version = \"0.8\" }\nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pemfile = \"2.0\"\nrustls-pki-types = { version = \"1\" }\ntokio = { version = \"1.47\", features = [\"full\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\nwebpki-roots = { version = \"1.0\"}\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\nhyper-util = { version = \"0.1.1\", features = [\"tokio\"] }\nhttp-body-util = \"0.1.0\"\nform_urlencoded = \"1.1.0\"\nsha1 = \"0.10\"\nsha2 = \"0.10.6\"\nmd5 = \"0.8.0\"\nrayon = \"1.5\"\nparking_lot = \"0.12\"\nregex = \"1.7.0\"\nblake3 = \"1.3\"\nlru-cache = \"0.1.2\"\nrand = \"0.9.0\"\nx509-parser = \"0.18\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\"] }\nserde = { version = \"1.0\", features = [\"derive\", \"rc\"] }\nserde_json = \"1.0\"\nnum_cpus = \"1.15.0\"\nchrono = \"0.4\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\n\n[features]\ntest_mode = []\nenterprise = []\n\n#[[bench]]\n#name = \"hash\"\n#harness = false\n"
  },
  {
    "path": "crates/smtp/src/core/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{inbound::auth::SaslToken, queue::QueueId};\nuse common::{\n    Inner, Server,\n    auth::AccessToken,\n    config::smtp::auth::VerifyStrategy,\n    listener::{ServerInstance, asn::AsnGeoLookupResult},\n};\nuse directory::Directory;\nuse mail_auth::{IprevOutput, SpfOutput};\nuse smtp_proto::request::receiver::{\n    BdatReceiver, DataReceiver, DummyDataReceiver, DummyLineReceiver, LineReceiver, RequestReceiver,\n};\nuse std::{\n    hash::Hash,\n    net::IpAddr,\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse tokio::io::{AsyncRead, AsyncWrite};\nuse utils::DomainPart;\n\npub mod params;\npub mod throttle;\n\n#[derive(Clone)]\npub struct SmtpSessionManager {\n    pub inner: Arc<Inner>,\n}\n\nimpl SmtpSessionManager {\n    pub fn new(inner: Arc<Inner>) -> Self {\n        Self { inner }\n    }\n}\n\npub enum State {\n    Request(RequestReceiver),\n    Bdat(BdatReceiver),\n    Data(DataReceiver),\n    Sasl(LineReceiver<SaslToken>),\n    DataTooLarge(DummyDataReceiver),\n    RequestTooLarge(DummyLineReceiver),\n    Accepted(QueueId),\n    None,\n}\n\npub struct Session<T: AsyncWrite + AsyncRead> {\n    pub hostname: String,\n    pub state: State,\n    pub instance: Arc<ServerInstance>,\n    pub server: Server,\n    pub stream: T,\n    pub data: SessionData,\n    pub params: SessionParameters,\n}\n\npub struct SessionData {\n    pub session_id: u64,\n    pub local_ip: IpAddr,\n    pub local_ip_str: String,\n    pub local_port: u16,\n    pub remote_ip: IpAddr,\n    pub remote_ip_str: String,\n    pub remote_port: u16,\n    pub asn_geo_data: AsnGeoLookupResult,\n    pub helo_domain: String,\n\n    pub mail_from: Option<SessionAddress>,\n    pub rcpt_to: Vec<SessionAddress>,\n    pub rcpt_errors: usize,\n    pub rcpt_oks: usize,\n    pub message: Vec<u8>,\n\n    pub authenticated_as: Option<Arc<AccessToken>>,\n    pub auth_errors: usize,\n\n    pub priority: i16,\n    pub delivery_by: i64,\n    pub future_release: u64,\n\n    pub valid_until: Instant,\n    pub bytes_left: usize,\n    pub messages_sent: usize,\n\n    pub iprev: Option<IprevOutput>,\n    pub spf_ehlo: Option<SpfOutput>,\n    pub spf_mail_from: Option<SpfOutput>,\n    pub dnsbl_error: Option<Vec<u8>>,\n}\n\n#[derive(Clone, Debug)]\npub struct SessionAddress {\n    pub address: String,\n    pub address_lcase: String,\n    pub domain: String,\n    pub flags: u64,\n    pub dsn_info: Option<String>,\n}\n\n#[derive(Debug, Default)]\npub struct SessionParameters {\n    // Global parameters\n    pub timeout: Duration,\n\n    // Ehlo parameters\n    pub ehlo_require: bool,\n    pub ehlo_reject_non_fqdn: bool,\n\n    // Auth parameters\n    pub auth_directory: Option<Arc<Directory>>,\n    pub auth_require: bool,\n    pub auth_errors_max: usize,\n    pub auth_errors_wait: Duration,\n\n    // Rcpt parameters\n    pub rcpt_errors_max: usize,\n    pub rcpt_errors_wait: Duration,\n    pub rcpt_max: usize,\n    pub rcpt_dsn: bool,\n    pub can_expn: bool,\n    pub can_vrfy: bool,\n    pub max_message_size: usize,\n\n    // Mail authentication parameters\n    pub iprev: VerifyStrategy,\n    pub spf_ehlo: VerifyStrategy,\n    pub spf_mail_from: VerifyStrategy,\n}\n\nimpl SessionData {\n    pub fn new(\n        local_ip: IpAddr,\n        local_port: u16,\n        remote_ip: IpAddr,\n        remote_port: u16,\n        asn_geo_data: AsnGeoLookupResult,\n        session_id: u64,\n    ) -> Self {\n        SessionData {\n            session_id,\n            local_ip,\n            local_port,\n            remote_ip,\n            local_ip_str: local_ip.to_string(),\n            remote_ip_str: remote_ip.to_string(),\n            remote_port,\n            asn_geo_data,\n            helo_domain: String::new(),\n            mail_from: None,\n            rcpt_to: Vec::new(),\n            authenticated_as: None,\n            priority: 0,\n            valid_until: Instant::now(),\n            rcpt_errors: 0,\n            rcpt_oks: 0,\n            message: Vec::with_capacity(0),\n            auth_errors: 0,\n            messages_sent: 0,\n            bytes_left: 0,\n            delivery_by: 0,\n            future_release: 0,\n            iprev: None,\n            spf_ehlo: None,\n            spf_mail_from: None,\n            dnsbl_error: None,\n        }\n    }\n}\n\nimpl Default for State {\n    fn default() -> Self {\n        State::Request(RequestReceiver::default())\n    }\n}\n\nimpl PartialEq for SessionAddress {\n    fn eq(&self, other: &Self) -> bool {\n        self.address_lcase == other.address_lcase\n    }\n}\n\nimpl Eq for SessionAddress {}\n\nimpl Hash for SessionAddress {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.address_lcase.hash(state);\n    }\n}\n\nimpl Ord for SessionAddress {\n    fn cmp(&self, other: &Self) -> std::cmp::Ordering {\n        match self.domain.cmp(&other.domain) {\n            std::cmp::Ordering::Equal => self.address_lcase.cmp(&other.address_lcase),\n            order => order,\n        }\n    }\n}\n\nimpl PartialOrd for SessionAddress {\n    fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {\n        Some(self.cmp(other))\n    }\n}\n\nimpl Session<common::listener::stream::NullIo> {\n    pub fn local(\n        server: Server,\n        instance: std::sync::Arc<ServerInstance>,\n        data: SessionData,\n    ) -> Self {\n        Session {\n            hostname: \"localhost\".into(),\n            state: State::None,\n            instance,\n            server,\n            stream: common::listener::stream::NullIo::default(),\n            data,\n            params: SessionParameters {\n                timeout: Default::default(),\n                ehlo_require: Default::default(),\n                ehlo_reject_non_fqdn: Default::default(),\n                auth_directory: Default::default(),\n                auth_require: Default::default(),\n                auth_errors_max: Default::default(),\n                auth_errors_wait: Default::default(),\n                rcpt_errors_max: Default::default(),\n                rcpt_errors_wait: Default::default(),\n                rcpt_max: Default::default(),\n                rcpt_dsn: Default::default(),\n                max_message_size: Default::default(),\n                iprev: VerifyStrategy::Disable,\n                spf_ehlo: VerifyStrategy::Disable,\n                spf_mail_from: VerifyStrategy::Disable,\n                can_expn: false,\n                can_vrfy: false,\n            },\n        }\n    }\n\n    pub fn has_failed(&mut self) -> Option<String> {\n        if self.stream.tx_buf.first().is_none_or(|&c| c == b'2') {\n            self.stream.tx_buf.clear();\n            None\n        } else {\n            let response = std::str::from_utf8(&self.stream.tx_buf)\n                .unwrap()\n                .trim()\n                .into();\n            self.stream.tx_buf.clear();\n            Some(response)\n        }\n    }\n}\n\nimpl SessionData {\n    pub fn local(\n        authenticated_as: Arc<AccessToken>,\n        mail_from: Option<SessionAddress>,\n        rcpt_to: Vec<SessionAddress>,\n        message: Vec<u8>,\n        session_id: u64,\n    ) -> Self {\n        SessionData {\n            local_ip: IpAddr::V4(std::net::Ipv4Addr::new(127, 0, 0, 1)),\n            remote_ip: IpAddr::V4(std::net::Ipv4Addr::new(127, 0, 0, 1)),\n            local_ip_str: \"127.0.0.1\".into(),\n            remote_ip_str: \"127.0.0.1\".into(),\n            remote_port: 0,\n            local_port: 0,\n            session_id,\n            asn_geo_data: AsnGeoLookupResult::default(),\n            helo_domain: \"localhost\".into(),\n            mail_from,\n            rcpt_to,\n            rcpt_errors: 0,\n            rcpt_oks: 0,\n            message,\n            authenticated_as: Some(authenticated_as),\n            auth_errors: 0,\n            priority: 0,\n            delivery_by: 0,\n            future_release: 0,\n            valid_until: Instant::now(),\n            bytes_left: 0,\n            messages_sent: 0,\n            iprev: None,\n            spf_ehlo: None,\n            spf_mail_from: None,\n            dnsbl_error: None,\n        }\n    }\n}\n\nimpl Default for SessionData {\n    fn default() -> Self {\n        Self::local(Arc::new(AccessToken::from_id(0)), None, vec![], vec![], 0)\n    }\n}\n\nimpl SessionAddress {\n    pub fn new(address: String) -> Self {\n        let address_lcase = address.to_lowercase();\n        SessionAddress {\n            domain: address_lcase.domain_part().into(),\n            address_lcase,\n            address,\n            flags: 0,\n            dsn_info: None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/core/params.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse common::{config::smtp::auth::VerifyStrategy, listener::SessionStream};\n\nuse super::Session;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn eval_session_params(&mut self) {\n        let c = &self.server.core.smtp.session;\n        self.data.bytes_left = self\n            .server\n            .eval_if(&c.transfer_limit, self, self.data.session_id)\n            .await\n            .unwrap_or(250 * 1024 * 1024);\n        self.data.valid_until += self\n            .server\n            .eval_if(&c.duration, self, self.data.session_id)\n            .await\n            .unwrap_or_else(|| Duration::from_secs(15 * 60));\n\n        self.params.timeout = self\n            .server\n            .eval_if(&c.timeout, self, self.data.session_id)\n            .await\n            .unwrap_or_else(|| Duration::from_secs(5 * 60));\n        self.params.spf_ehlo = self\n            .server\n            .eval_if(\n                &self.server.core.smtp.mail_auth.spf.verify_ehlo,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .unwrap_or(VerifyStrategy::Relaxed);\n        self.params.spf_mail_from = self\n            .server\n            .eval_if(\n                &self.server.core.smtp.mail_auth.spf.verify_mail_from,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .unwrap_or(VerifyStrategy::Relaxed);\n        self.params.iprev = self\n            .server\n            .eval_if(\n                &self.server.core.smtp.mail_auth.iprev.verify,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .unwrap_or(VerifyStrategy::Relaxed);\n\n        // Ehlo parameters\n        let ec = &self.server.core.smtp.session.ehlo;\n        self.params.ehlo_require = self\n            .server\n            .eval_if(&ec.require, self, self.data.session_id)\n            .await\n            .unwrap_or(true);\n        self.params.ehlo_reject_non_fqdn = self\n            .server\n            .eval_if(&ec.reject_non_fqdn, self, self.data.session_id)\n            .await\n            .unwrap_or(true);\n\n        // Auth parameters\n        let ac = &self.server.core.smtp.session.auth;\n        self.params.auth_directory = self\n            .server\n            .eval_if::<String, _>(&ac.directory, self, self.data.session_id)\n            .await\n            .and_then(|name| self.server.get_directory(&name))\n            .cloned();\n        self.params.auth_require = self\n            .server\n            .eval_if(&ac.require, self, self.data.session_id)\n            .await\n            .unwrap_or(false);\n        self.params.auth_errors_max = self\n            .server\n            .eval_if(&ac.errors_max, self, self.data.session_id)\n            .await\n            .unwrap_or(3);\n        self.params.auth_errors_wait = self\n            .server\n            .eval_if(&ac.errors_wait, self, self.data.session_id)\n            .await\n            .unwrap_or_else(|| Duration::from_secs(30));\n\n        // VRFY/EXPN parameters\n        let ec = &self.server.core.smtp.session.extensions;\n        self.params.can_expn = self\n            .server\n            .eval_if(&ec.expn, self, self.data.session_id)\n            .await\n            .unwrap_or(false);\n        self.params.can_vrfy = self\n            .server\n            .eval_if(&ec.vrfy, self, self.data.session_id)\n            .await\n            .unwrap_or(false);\n    }\n\n    pub async fn eval_post_auth_params(&mut self) {\n        // Refresh VRFY/EXPN parameters\n        let ec = &self.server.core.smtp.session.extensions;\n        self.params.can_expn = self\n            .server\n            .eval_if(&ec.expn, self, self.data.session_id)\n            .await\n            .unwrap_or(false);\n        self.params.can_vrfy = self\n            .server\n            .eval_if(&ec.vrfy, self, self.data.session_id)\n            .await\n            .unwrap_or(false);\n    }\n\n    pub async fn eval_rcpt_params(&mut self) {\n        let rc = &self.server.core.smtp.session.rcpt;\n        self.params.rcpt_errors_max = self\n            .server\n            .eval_if(&rc.errors_max, self, self.data.session_id)\n            .await\n            .unwrap_or(10);\n        self.params.rcpt_errors_wait = self\n            .server\n            .eval_if(&rc.errors_wait, self, self.data.session_id)\n            .await\n            .unwrap_or_else(|| Duration::from_secs(30));\n        self.params.rcpt_max = self\n            .server\n            .eval_if(&rc.max_recipients, self, self.data.session_id)\n            .await\n            .unwrap_or(100);\n        self.params.rcpt_dsn = self\n            .server\n            .eval_if(\n                &self.server.core.smtp.session.extensions.dsn,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .unwrap_or(true);\n\n        self.params.max_message_size = self\n            .server\n            .eval_if(\n                &self.server.core.smtp.session.data.max_message_size,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .unwrap_or(25 * 1024 * 1024);\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/core/throttle.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    KV_RATE_LIMIT_SMTP, ThrottleKey,\n    config::smtp::*,\n    expr::{functions::ResolveVariable, *},\n    listener::SessionStream,\n};\nuse queue::QueueQuota;\nuse trc::SmtpEvent;\nuse utils::config::Rate;\n\nuse super::Session;\n\npub trait NewKey: Sized {\n    fn new_key(&self, e: &impl ResolveVariable, context: &str) -> ThrottleKey;\n}\n\nimpl NewKey for QueueQuota {\n    fn new_key(&self, e: &impl ResolveVariable, _: &str) -> ThrottleKey {\n        let mut hasher = blake3::Hasher::new();\n\n        if (self.keys & THROTTLE_RCPT) != 0 {\n            hasher.update(e.resolve_variable(V_RECIPIENT).to_string().as_bytes());\n        }\n        if (self.keys & THROTTLE_RCPT_DOMAIN) != 0 {\n            hasher.update(\n                e.resolve_variable(V_RECIPIENT_DOMAIN)\n                    .to_string()\n                    .as_bytes(),\n            );\n        }\n        if (self.keys & THROTTLE_SENDER) != 0 {\n            let sender = e.resolve_variable(V_SENDER).into_string();\n            hasher.update(\n                if !sender.is_empty() {\n                    sender.as_ref()\n                } else {\n                    \"<>\"\n                }\n                .as_bytes(),\n            );\n        }\n        if (self.keys & THROTTLE_SENDER_DOMAIN) != 0 {\n            let sender_domain = e.resolve_variable(V_SENDER_DOMAIN).into_string();\n            hasher.update(\n                if !sender_domain.is_empty() {\n                    sender_domain.as_ref()\n                } else {\n                    \"<>\"\n                }\n                .as_bytes(),\n            );\n        }\n\n        if let Some(messages) = &self.messages {\n            hasher.update(&messages.to_ne_bytes()[..]);\n        }\n\n        if let Some(size) = &self.size {\n            hasher.update(&size.to_ne_bytes()[..]);\n        }\n\n        ThrottleKey {\n            hash: hasher.finalize().into(),\n        }\n    }\n}\n\nimpl NewKey for QueueRateLimiter {\n    fn new_key(&self, e: &impl ResolveVariable, context: &str) -> ThrottleKey {\n        let mut hasher = blake3::Hasher::new();\n\n        if (self.keys & THROTTLE_RCPT) != 0 {\n            hasher.update(e.resolve_variable(V_RECIPIENT).to_string().as_bytes());\n        }\n        if (self.keys & THROTTLE_RCPT_DOMAIN) != 0 {\n            hasher.update(\n                e.resolve_variable(V_RECIPIENT_DOMAIN)\n                    .to_string()\n                    .as_bytes(),\n            );\n        }\n        if (self.keys & THROTTLE_SENDER) != 0 {\n            let sender = e.resolve_variable(V_SENDER).into_string();\n            hasher.update(\n                if !sender.is_empty() {\n                    sender.as_ref()\n                } else {\n                    \"<>\"\n                }\n                .as_bytes(),\n            );\n        }\n        if (self.keys & THROTTLE_SENDER_DOMAIN) != 0 {\n            let sender_domain = e.resolve_variable(V_SENDER_DOMAIN).into_string();\n            hasher.update(\n                if !sender_domain.is_empty() {\n                    sender_domain.as_ref()\n                } else {\n                    \"<>\"\n                }\n                .as_bytes(),\n            );\n        }\n        if (self.keys & THROTTLE_HELO_DOMAIN) != 0 {\n            hasher.update(e.resolve_variable(V_HELO_DOMAIN).to_string().as_bytes());\n        }\n        if (self.keys & THROTTLE_AUTH_AS) != 0 {\n            hasher.update(\n                e.resolve_variable(V_AUTHENTICATED_AS)\n                    .to_string()\n                    .as_bytes(),\n            );\n        }\n        if (self.keys & THROTTLE_LISTENER) != 0 {\n            hasher.update(e.resolve_variable(V_LISTENER).to_string().as_bytes());\n        }\n        if (self.keys & THROTTLE_MX) != 0 {\n            hasher.update(e.resolve_variable(V_MX).to_string().as_bytes());\n        }\n        if (self.keys & THROTTLE_REMOTE_IP) != 0 {\n            hasher.update(e.resolve_variable(V_REMOTE_IP).to_string().as_bytes());\n        }\n        if (self.keys & THROTTLE_LOCAL_IP) != 0 {\n            hasher.update(e.resolve_variable(V_LOCAL_IP).to_string().as_bytes());\n        }\n        hasher.update(&self.rate.period.as_secs().to_be_bytes()[..]);\n        hasher.update(&self.rate.requests.to_be_bytes()[..]);\n        hasher.update(context.as_bytes());\n\n        ThrottleKey {\n            hash: hasher.finalize().into(),\n        }\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn is_allowed(&mut self) -> bool {\n        let throttles = if !self.data.rcpt_to.is_empty() {\n            &self.server.core.smtp.queue.inbound_limiters.rcpt\n        } else if self.data.mail_from.is_some() {\n            &self.server.core.smtp.queue.inbound_limiters.sender\n        } else {\n            &self.server.core.smtp.queue.inbound_limiters.remote\n        };\n\n        for t in throttles {\n            if t.expr.is_empty()\n                || self\n                    .server\n                    .eval_expr(&t.expr, self, \"throttle\", self.data.session_id)\n                    .await\n                    .unwrap_or(false)\n            {\n                if (t.keys & THROTTLE_RCPT_DOMAIN) != 0 {\n                    let d = self\n                        .data\n                        .rcpt_to\n                        .last()\n                        .map(|r| r.domain.as_str())\n                        .unwrap_or_default();\n\n                    if self.data.rcpt_to.iter().filter(|p| p.domain == d).count() > 1 {\n                        continue;\n                    }\n                }\n\n                // Build throttle key\n                let key = t.new_key(self, \"inbound\");\n\n                // Check rate\n                match self\n                    .server\n                    .core\n                    .storage\n                    .lookup\n                    .is_rate_allowed(KV_RATE_LIMIT_SMTP, key.hash.as_slice(), &t.rate, false)\n                    .await\n                {\n                    Ok(Some(_)) => {\n                        trc::event!(\n                            Smtp(SmtpEvent::RateLimitExceeded),\n                            SpanId = self.data.session_id,\n                            Id = t.id.clone(),\n                            Limit = vec![\n                                trc::Value::from(t.rate.requests),\n                                trc::Value::from(t.rate.period)\n                            ],\n                        );\n\n                        return false;\n                    }\n                    Err(err) => {\n                        trc::error!(\n                            err.span_id(self.data.session_id)\n                                .caused_by(trc::location!())\n                        );\n                    }\n                    _ => (),\n                }\n            }\n        }\n\n        true\n    }\n\n    pub async fn throttle_rcpt(&self, rcpt: &str, rate: &Rate, ctx: &str) -> bool {\n        let mut hasher = blake3::Hasher::new();\n        hasher.update(rcpt.as_bytes());\n        hasher.update(ctx.as_bytes());\n        hasher.update(&rate.period.as_secs().to_ne_bytes()[..]);\n        hasher.update(&rate.requests.to_ne_bytes()[..]);\n\n        match self\n            .server\n            .core\n            .storage\n            .lookup\n            .is_rate_allowed(\n                KV_RATE_LIMIT_SMTP,\n                hasher.finalize().as_bytes(),\n                rate,\n                false,\n            )\n            .await\n        {\n            Ok(None) => true,\n            Ok(Some(_)) => false,\n            Err(err) => {\n                trc::error!(\n                    err.span_id(self.data.session_id)\n                        .caused_by(trc::location!())\n                );\n                true\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/auth.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    auth::{\n        AuthRequest,\n        sasl::{sasl_decode_challenge_oauth, sasl_decode_challenge_plain},\n    },\n    listener::SessionStream,\n};\n\nuse directory::Permission;\nuse mail_parser::decoders::base64::base64_decode;\nuse mail_send::Credentials;\nuse smtp_proto::{AUTH_LOGIN, AUTH_OAUTHBEARER, AUTH_PLAIN, AUTH_XOAUTH2, IntoString};\nuse trc::{AuthEvent, SmtpEvent};\n\nuse crate::core::Session;\n\npub struct SaslToken {\n    mechanism: u64,\n    credentials: Credentials<String>,\n}\n\nimpl SaslToken {\n    pub fn from_mechanism(mechanism: u64) -> Option<SaslToken> {\n        match mechanism {\n            AUTH_PLAIN | AUTH_LOGIN => SaslToken {\n                mechanism,\n                credentials: Credentials::Plain {\n                    username: String::new(),\n                    secret: String::new(),\n                },\n            }\n            .into(),\n            AUTH_OAUTHBEARER | AUTH_XOAUTH2 => SaslToken {\n                mechanism,\n                credentials: Credentials::OAuthBearer {\n                    token: String::new(),\n                },\n            }\n            .into(),\n            _ => None,\n        }\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_sasl_response(\n        &mut self,\n        token: &mut SaslToken,\n        response: &[u8],\n    ) -> Result<bool, ()> {\n        if response.is_empty() {\n            match (token.mechanism, &token.credentials) {\n                (AUTH_PLAIN | AUTH_XOAUTH2 | AUTH_OAUTHBEARER, _) => {\n                    self.write(b\"334 Go ahead.\\r\\n\").await?;\n                    return Ok(true);\n                }\n                (AUTH_LOGIN, Credentials::Plain { username, secret }) => {\n                    if username.is_empty() && secret.is_empty() {\n                        self.write(b\"334 VXNlcm5hbWU6\\r\\n\").await?;\n                        return Ok(true);\n                    }\n                }\n                _ => (),\n            }\n        } else if let Some(response) = base64_decode(response) {\n            match (token.mechanism, &mut token.credentials) {\n                (AUTH_PLAIN, _) => {\n                    if let Some(credentials) = sasl_decode_challenge_plain(&response) {\n                        return self.authenticate(credentials).await;\n                    }\n                }\n                (AUTH_LOGIN, Credentials::Plain { username, secret }) => {\n                    return if username.is_empty() {\n                        *username = response.into_string();\n                        self.write(b\"334 UGFzc3dvcmQ6\\r\\n\").await?;\n                        Ok(true)\n                    } else {\n                        *secret = response.into_string();\n                        self.authenticate(std::mem::take(&mut token.credentials))\n                            .await\n                    };\n                }\n                (AUTH_OAUTHBEARER | AUTH_XOAUTH2, _) => {\n                    if let Some(credentials) = sasl_decode_challenge_oauth(&response) {\n                        return self.authenticate(credentials).await;\n                    }\n                }\n                _ => (),\n            }\n        }\n\n        self.auth_error(b\"500 5.5.6 Invalid challenge.\\r\\n\").await\n    }\n\n    pub async fn authenticate(&mut self, credentials: Credentials<String>) -> Result<bool, ()> {\n        if let Some(directory) = &self.params.auth_directory {\n            // Authenticate\n            let result = self\n                .server\n                .authenticate(\n                    &AuthRequest::from_credentials(\n                        credentials,\n                        self.data.session_id,\n                        self.data.remote_ip,\n                    )\n                    .with_directory(directory),\n                )\n                .await\n                .and_then(|access_token| {\n                    access_token\n                        .assert_has_permission(Permission::EmailSend)\n                        .map(|_| access_token)\n                });\n\n            match result {\n                Ok(access_token) => {\n                    self.data.authenticated_as = access_token.into();\n                    self.eval_post_auth_params().await;\n                    self.write(b\"235 2.7.0 Authentication succeeded.\\r\\n\")\n                        .await?;\n                    return Ok(false);\n                }\n                Err(err) => {\n                    let reason = *err.as_ref();\n\n                    trc::error!(err.span_id(self.data.session_id));\n\n                    match reason {\n                        trc::EventType::Auth(trc::AuthEvent::Failed) => {\n                            return self\n                                .auth_error(b\"535 5.7.8 Authentication credentials invalid.\\r\\n\")\n                                .await;\n                        }\n                        trc::EventType::Auth(trc::AuthEvent::TokenExpired) => {\n                            return self.auth_error(b\"535 5.7.8 OAuth token expired.\\r\\n\").await;\n                        }\n                        trc::EventType::Auth(trc::AuthEvent::MissingTotp) => {\n                            return self\n                            .auth_error(\n                                b\"334 5.7.8 Missing TOTP token, try with 'secret$totp_code'.\\r\\n\",\n                            )\n                            .await;\n                        }\n                        trc::EventType::Security(trc::SecurityEvent::Unauthorized) => {\n                            self.write(\n                                concat!(\n                                    \"550 5.7.1 Your account is not authorized \",\n                                    \"to use this service.\\r\\n\"\n                                )\n                                .as_bytes(),\n                            )\n                            .await?;\n                            return Ok(false);\n                        }\n                        trc::EventType::Security(_) => {\n                            return Err(());\n                        }\n                        _ => (),\n                    }\n                }\n            }\n        } else {\n            trc::event!(\n                Smtp(SmtpEvent::MissingAuthDirectory),\n                SpanId = self.data.session_id,\n            );\n        }\n        self.write(b\"454 4.7.0 Temporary authentication failure\\r\\n\")\n            .await?;\n\n        Ok(false)\n    }\n\n    pub async fn auth_error(&mut self, response: &[u8]) -> Result<bool, ()> {\n        tokio::time::sleep(self.params.auth_errors_wait).await;\n        self.data.auth_errors += 1;\n        self.write(response).await?;\n        if self.data.auth_errors < self.params.auth_errors_max {\n            Ok(false)\n        } else {\n            trc::event!(\n                Auth(AuthEvent::TooManyAttempts),\n                SpanId = self.data.session_id,\n            );\n\n            self.write(b\"455 4.3.0 Too many authentication errors, disconnecting.\\r\\n\")\n                .await?;\n            Err(())\n        }\n    }\n\n    pub fn authenticated_as(&self) -> Option<&str> {\n        self.data.authenticated_as.as_ref().map(|token| {\n            if !token.name.is_empty() {\n                token.name.as_str()\n            } else {\n                \"unavailable\"\n            }\n        })\n    }\n\n    pub fn is_authenticated(&self) -> bool {\n        self.data.authenticated_as.is_some()\n    }\n\n    pub fn authenticated_emails(&self) -> &[String] {\n        self.data\n            .authenticated_as\n            .as_ref()\n            .map(|token| token.emails.as_slice())\n            .unwrap_or_default()\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/data.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ArcSeal, AuthResult, DkimSign};\nuse crate::{\n    core::{Session, SessionAddress, State},\n    inbound::milter::Modification,\n    queue::{\n        self, Message, MessageSource, MessageWrapper, QueueEnvelope, RCPT_SPAM_PAYLOAD,\n        quota::HasQueueQuota,\n    },\n    reporting::analysis::AnalyzeReport,\n    scripts::ScriptResult,\n};\nuse common::{\n    config::{\n        smtp::{\n            auth::VerifyStrategy,\n            queue::{QueueExpiry, QueueName},\n            session::Stage,\n        },\n        spamfilter::SpamFilterAction,\n    },\n    listener::SessionStream,\n    psl,\n    scripts::ScriptModification,\n};\nuse mail_auth::{\n    AuthenticatedMessage, AuthenticationResults, DkimResult, DmarcResult, ReceivedSpf,\n    common::{headers::HeaderWriter, verify::VerifySignature},\n    dmarc::{self, verify::DmarcParameters},\n};\nuse mail_builder::headers::{date::Date, message_id::generate_message_id_header};\nuse mail_parser::MessageParser;\nuse sieve::runtime::Variable;\nuse smtp_proto::{\n    MAIL_BY_RETURN, RCPT_NOTIFY_DELAY, RCPT_NOTIFY_FAILURE, RCPT_NOTIFY_NEVER, RCPT_NOTIFY_SUCCESS,\n};\nuse std::{\n    borrow::Cow,\n    time::{Instant, SystemTime},\n};\nuse trc::SmtpEvent;\nuse utils::{DomainPart, config::Rate};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn queue_message(&mut self) -> Cow<'static, [u8]> {\n        // Parse message\n        let raw_message = std::mem::take(&mut self.data.message);\n        let parsed_message = match MessageParser::new()\n            .parse(&raw_message)\n            .filter(|p| p.headers().iter().any(|h| !h.name.is_other()))\n        {\n            Some(parsed_message) => parsed_message,\n            None => {\n                trc::event!(\n                    Smtp(SmtpEvent::MessageParseFailed),\n                    SpanId = self.data.session_id,\n                );\n\n                return (&b\"550 5.7.7 Failed to parse message.\\r\\n\"[..]).into();\n            }\n        };\n\n        // Authenticate message\n        let auth_message = AuthenticatedMessage::from_parsed(\n            &parsed_message,\n            self.server.core.smtp.mail_auth.dkim.strict,\n        );\n        let has_date_header = auth_message.has_date_header();\n        let has_message_id_header = auth_message.has_message_id_header();\n\n        // Loop detection\n        let dc = &self.server.core.smtp.session.data;\n        let ac = &self.server.core.smtp.mail_auth;\n        let rc = &self.server.core.smtp.report;\n        if auth_message.received_headers_count()\n            > self\n                .server\n                .eval_if(&dc.max_received_headers, self, self.data.session_id)\n                .await\n                .unwrap_or(50)\n        {\n            trc::event!(\n                Smtp(SmtpEvent::LoopDetected),\n                SpanId = self.data.session_id,\n                Total = auth_message.received_headers_count(),\n            );\n\n            return (&b\"450 4.4.6 Too many Received headers. Possible loop detected.\\r\\n\"[..])\n                .into();\n        }\n\n        // Verify DKIM\n        let dkim = self\n            .server\n            .eval_if(&ac.dkim.verify, self, self.data.session_id)\n            .await\n            .unwrap_or(VerifyStrategy::Relaxed);\n        let dmarc = self\n            .server\n            .eval_if(&ac.dmarc.verify, self, self.data.session_id)\n            .await\n            .unwrap_or(VerifyStrategy::Relaxed);\n        let dkim_output = if dkim.verify() || dmarc.verify() {\n            let time = Instant::now();\n            let dkim_output = self\n                .server\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .verify_dkim(self.server.inner.cache.build_auth_parameters(&auth_message))\n                .await;\n            let pass = dkim_output\n                .iter()\n                .any(|d| matches!(d.result(), DkimResult::Pass));\n            let strict = dkim.is_strict();\n            let rejected = strict && !pass;\n\n            // Send reports for failed signatures\n            if let Some(rate) = self\n                .server\n                .eval_if::<Rate, _>(&rc.dkim.send, self, self.data.session_id)\n                .await\n            {\n                for output in &dkim_output {\n                    if let Some(rcpt) = output.failure_report_addr() {\n                        self.send_dkim_report(rcpt, &auth_message, &rate, rejected, output)\n                            .await;\n                    }\n                }\n            }\n\n            trc::event!(\n                Smtp(if pass {\n                    SmtpEvent::DkimPass\n                } else {\n                    SmtpEvent::DkimFail\n                }),\n                SpanId = self.data.session_id,\n                Strict = strict,\n                Result = dkim_output.iter().map(trc::Error::from).collect::<Vec<_>>(),\n                Elapsed = time.elapsed(),\n            );\n\n            if rejected {\n                // 'Strict' mode violates the advice of Section 6.1 of RFC6376\n                return if dkim_output\n                    .iter()\n                    .any(|d| matches!(d.result(), DkimResult::TempError(_)))\n                {\n                    (&b\"451 4.7.20 No passing DKIM signatures found.\\r\\n\"[..]).into()\n                } else {\n                    (&b\"550 5.7.20 No passing DKIM signatures found.\\r\\n\"[..]).into()\n                };\n            }\n\n            dkim_output\n        } else {\n            vec![]\n        };\n\n        // Verify ARC\n        let arc = self\n            .server\n            .eval_if(&ac.arc.verify, self, self.data.session_id)\n            .await\n            .unwrap_or(VerifyStrategy::Relaxed);\n        let arc_sealer = self\n            .server\n            .eval_if::<String, _>(&ac.arc.seal, self, self.data.session_id)\n            .await\n            .and_then(|name| self.server.get_arc_sealer(&name, self.data.session_id));\n        let arc_output = if arc.verify() || arc_sealer.is_some() {\n            let time = Instant::now();\n            let arc_output = self\n                .server\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .verify_arc(self.server.inner.cache.build_auth_parameters(&auth_message))\n                .await;\n\n            let strict = arc.is_strict();\n            let pass = matches!(arc_output.result(), DkimResult::Pass | DkimResult::None);\n\n            trc::event!(\n                Smtp(if pass {\n                    SmtpEvent::ArcPass\n                } else {\n                    SmtpEvent::ArcFail\n                }),\n                SpanId = self.data.session_id,\n                Strict = strict,\n                Result = trc::Error::from(arc_output.result()),\n                Elapsed = time.elapsed(),\n            );\n\n            if strict && !pass {\n                return if matches!(arc_output.result(), DkimResult::TempError(_)) {\n                    (&b\"451 4.7.29 ARC validation failed.\\r\\n\"[..]).into()\n                } else {\n                    (&b\"550 5.7.29 ARC validation failed.\\r\\n\"[..]).into()\n                };\n            }\n\n            arc_output.into()\n        } else {\n            None\n        };\n\n        // Build authentication results header\n        let mail_from = self.data.mail_from.as_ref().unwrap();\n        let mut auth_results = AuthenticationResults::new(&self.hostname);\n        if !dkim_output.is_empty() {\n            auth_results = auth_results.with_dkim_results(&dkim_output, auth_message.from())\n        }\n        if let Some(spf_ehlo) = &self.data.spf_ehlo {\n            auth_results = auth_results.with_spf_ehlo_result(\n                spf_ehlo,\n                self.data.remote_ip,\n                &self.data.helo_domain,\n            );\n        }\n        if let Some(spf_mail_from) = &self.data.spf_mail_from {\n            auth_results = auth_results.with_spf_mailfrom_result(\n                spf_mail_from,\n                self.data.remote_ip,\n                &mail_from.address,\n                &self.data.helo_domain,\n            );\n        }\n        if let Some(iprev) = &self.data.iprev {\n            auth_results = auth_results.with_iprev_result(iprev, self.data.remote_ip);\n        }\n\n        // Verify DMARC\n        let is_report = self.is_report();\n        let (dmarc_result, dmarc_policy) = match &self.data.spf_mail_from {\n            Some(spf_output) if dmarc.verify() => {\n                let time = Instant::now();\n                let dmarc_output =\n                    self.server\n                        .core\n                        .smtp\n                        .resolvers\n                        .dns\n                        .verify_dmarc(self.server.inner.cache.build_auth_parameters(\n                            DmarcParameters {\n                                message: &auth_message,\n                                dkim_output: &dkim_output,\n                                rfc5321_mail_from_domain: if !mail_from.domain.is_empty() {\n                                    &mail_from.domain\n                                } else {\n                                    &self.data.helo_domain\n                                },\n                                spf_output,\n                                domain_suffix_fn: |domain| {\n                                    psl::domain_str(domain).unwrap_or(domain)\n                                },\n                            },\n                        ))\n                        .await;\n\n                let pass = matches!(dmarc_output.spf_result(), DmarcResult::Pass)\n                    || matches!(dmarc_output.dkim_result(), DmarcResult::Pass);\n                let strict = dmarc.is_strict();\n                let rejected = strict && dmarc_output.policy() == dmarc::Policy::Reject && !pass;\n                let is_temp_fail = rejected\n                    && matches!(dmarc_output.spf_result(), DmarcResult::TempError(_))\n                    || matches!(dmarc_output.dkim_result(), DmarcResult::TempError(_));\n\n                // Add to DMARC output to the Authentication-Results header\n                auth_results = auth_results.with_dmarc_result(&dmarc_output);\n                let dmarc_result = if pass {\n                    DmarcResult::Pass\n                } else if dmarc_output.spf_result() != &DmarcResult::None {\n                    dmarc_output.spf_result().clone()\n                } else if dmarc_output.dkim_result() != &DmarcResult::None {\n                    dmarc_output.dkim_result().clone()\n                } else {\n                    DmarcResult::None\n                };\n                let dmarc_policy = dmarc_output.policy();\n\n                trc::event!(\n                    Smtp(if pass {\n                        SmtpEvent::DmarcPass\n                    } else {\n                        SmtpEvent::DmarcFail\n                    }),\n                    SpanId = self.data.session_id,\n                    Strict = strict,\n                    Domain = dmarc_output.domain().to_string(),\n                    Policy = dmarc_policy.to_string(),\n                    Result = trc::Error::from(&dmarc_result),\n                    Elapsed = time.elapsed(),\n                );\n\n                // Send DMARC report\n                if dmarc_output.requested_reports() && !is_report {\n                    self.send_dmarc_report(\n                        &auth_message,\n                        &auth_results,\n                        rejected,\n                        dmarc_output,\n                        &dkim_output,\n                        &arc_output,\n                    )\n                    .await;\n                }\n\n                if rejected {\n                    return if is_temp_fail {\n                        (&b\"451 4.7.1 Email temporarily rejected per DMARC policy.\\r\\n\"[..]).into()\n                    } else {\n                        (&b\"550 5.7.1 Email rejected per DMARC policy.\\r\\n\"[..]).into()\n                    };\n                }\n\n                (dmarc_result.into(), dmarc_policy.into())\n            }\n            _ => (None, None),\n        };\n\n        // Analyze reports\n        if is_report {\n            if !rc.analysis.forward {\n                self.server.analyze_report(\n                    mail_parser::Message {\n                        html_body: parsed_message.html_body,\n                        text_body: parsed_message.text_body,\n                        attachments: parsed_message.attachments,\n                        parts: parsed_message\n                            .parts\n                            .into_iter()\n                            .map(|p| p.into_owned())\n                            .collect(),\n                        raw_message: b\"\".into(),\n                    },\n                    self.data.session_id,\n                );\n                self.data.messages_sent += 1;\n                return (b\"250 2.0.0 Message queued for delivery.\\r\\n\"[..]).into();\n            } else {\n                self.server.analyze_report(\n                    mail_parser::Message {\n                        html_body: parsed_message.html_body.clone(),\n                        text_body: parsed_message.text_body.clone(),\n                        attachments: parsed_message.attachments.clone(),\n                        parts: parsed_message\n                            .parts\n                            .iter()\n                            .map(|p| p.clone().into_owned())\n                            .collect(),\n                        raw_message: b\"\".into(),\n                    },\n                    self.data.session_id,\n                );\n            }\n        }\n\n        // Add Received header\n        let message_id = self.server.inner.data.queue_id_gen.generate();\n        let mut headers = Vec::with_capacity(64);\n        if self\n            .server\n            .eval_if(&dc.add_received, self, self.data.session_id)\n            .await\n            .unwrap_or(true)\n        {\n            self.write_received(&mut headers, message_id)\n        }\n\n        // Add authentication results header\n        if self\n            .server\n            .eval_if(&dc.add_auth_results, self, self.data.session_id)\n            .await\n            .unwrap_or(true)\n        {\n            auth_results.write_header(&mut headers);\n        }\n\n        // Add Received-SPF header\n        if let Some(spf_output) = &self.data.spf_mail_from\n            && self\n                .server\n                .eval_if(&dc.add_received_spf, self, self.data.session_id)\n                .await\n                .unwrap_or(true)\n        {\n            ReceivedSpf::new(\n                spf_output,\n                self.data.remote_ip,\n                &self.data.helo_domain,\n                &mail_from.address_lcase,\n                &self.hostname,\n            )\n            .write_header(&mut headers);\n        }\n\n        // ARC Seal\n        if let (Some(arc_sealer), Some(arc_output)) = (arc_sealer, &arc_output)\n            && !dkim_output.is_empty()\n            && arc_output.can_be_sealed()\n        {\n            match arc_sealer.seal(&auth_message, &auth_results, arc_output) {\n                Ok(set) => {\n                    set.write_header(&mut headers);\n                }\n                Err(err) => {\n                    trc::error!(\n                        trc::Error::from(err)\n                            .span_id(self.data.session_id)\n                            .details(\"Failed to ARC seal message\")\n                    );\n                }\n            }\n        }\n\n        // Run SPAM filter\n        let mut train_spam = None;\n        if self.server.core.spam.enabled\n            && self\n                .server\n                .eval_if(&dc.spam_filter, self, self.data.session_id)\n                .await\n                .unwrap_or(true)\n        {\n            match self\n                .spam_classify(\n                    &parsed_message,\n                    &dkim_output,\n                    (&arc_output).into(),\n                    dmarc_result.as_ref(),\n                    dmarc_policy.as_ref(),\n                )\n                .await\n            {\n                SpamFilterAction::Allow(score) => {\n                    // Add headers\n                    headers.extend_from_slice(score.headers.as_bytes());\n                    train_spam = score.train_spam;\n\n                    // Add scores for local recipients\n                    for (is_spam, recipient) in\n                        score.results.into_iter().zip(self.data.rcpt_to.iter_mut())\n                    {\n                        if is_spam {\n                            recipient.flags |= RCPT_SPAM_PAYLOAD;\n                        }\n                    }\n                }\n                SpamFilterAction::Discard => {\n                    self.data.messages_sent += 1;\n                    return (b\"250 2.0.0 Message queued for delivery.\\r\\n\"[..]).into();\n                }\n                SpamFilterAction::Reject => {\n                    self.data.messages_sent += 1;\n                    return (b\"550 5.7.1 Message rejected due to excessive spam score.\\r\\n\"[..])\n                        .into();\n                }\n                SpamFilterAction::Disabled => {}\n            }\n        }\n\n        // Run Milter filters\n        let mut modifications = Vec::new();\n        match self.run_milters(Stage::Data, (&auth_message).into()).await {\n            Ok(modifications_) => {\n                if !modifications_.is_empty() {\n                    modifications = modifications_;\n                }\n            }\n            Err(response) => {\n                return response.into_bytes();\n            }\n        };\n\n        // Run MTA Hooks\n        match self\n            .run_mta_hooks(Stage::Data, (&auth_message).into(), message_id.into())\n            .await\n        {\n            Ok(modifications_) => {\n                if !modifications_.is_empty() {\n                    modifications.retain(|m| !matches!(m, Modification::ReplaceBody { .. }));\n                    modifications.extend(modifications_);\n                }\n            }\n            Err(response) => {\n                return response.into_bytes();\n            }\n        };\n\n        // Apply modifications\n        let mut edited_message = if !modifications.is_empty() {\n            self.data\n                .apply_milter_modifications(modifications, &auth_message)\n        } else {\n            None\n        };\n\n        // Sieve filtering\n        if let Some((script, script_id)) = self\n            .server\n            .eval_if::<String, _>(&dc.script, self, self.data.session_id)\n            .await\n            .and_then(|name| {\n                self.server\n                    .get_trusted_sieve_script(&name, self.data.session_id)\n                    .map(|s| (s, name))\n            })\n        {\n            let params = self\n                .build_script_parameters(\"data\")\n                .with_auth_headers(&headers)\n                .set_variable(\n                    \"arc.result\",\n                    arc_output\n                        .as_ref()\n                        .map(|a| a.result().as_str())\n                        .unwrap_or_default(),\n                )\n                .set_variable(\n                    \"dkim.result\",\n                    dkim_output\n                        .iter()\n                        .find(|r| matches!(r.result(), DkimResult::Pass))\n                        .or_else(|| dkim_output.first())\n                        .map(|r| r.result().as_str())\n                        .unwrap_or_default(),\n                )\n                .set_variable(\n                    \"dkim.domains\",\n                    dkim_output\n                        .iter()\n                        .filter_map(|r| {\n                            if matches!(r.result(), DkimResult::Pass) {\n                                r.signature()\n                                    .map(|s| Variable::from(s.domain().to_lowercase()))\n                            } else {\n                                None\n                            }\n                        })\n                        .collect::<Vec<_>>(),\n                )\n                .set_variable(\n                    \"dmarc.result\",\n                    dmarc_result\n                        .as_ref()\n                        .map(|a| a.as_str())\n                        .unwrap_or_default(),\n                )\n                .set_variable(\n                    \"dmarc.policy\",\n                    dmarc_policy\n                        .as_ref()\n                        .map(|a| a.as_str())\n                        .unwrap_or_default(),\n                )\n                .with_message(parsed_message);\n\n            let modifications = match self.run_script(script_id, script.clone(), params).await {\n                ScriptResult::Accept { modifications } => modifications,\n                ScriptResult::Replace {\n                    message,\n                    modifications,\n                } => {\n                    edited_message = message.into();\n                    modifications\n                }\n                ScriptResult::Reject(message) => {\n                    return message.as_bytes().to_vec().into();\n                }\n                ScriptResult::Discard => {\n                    return (b\"250 2.0.0 Message queued for delivery.\\r\\n\"[..]).into();\n                }\n            };\n\n            // Apply modifications\n            for modification in modifications {\n                match modification {\n                    ScriptModification::AddHeader { name, value } => {\n                        headers.extend_from_slice(name.as_bytes());\n                        headers.extend_from_slice(b\": \");\n                        headers.extend_from_slice(value.as_bytes());\n                        if !value.ends_with('\\n') {\n                            headers.extend_from_slice(b\"\\r\\n\");\n                        }\n                    }\n                    ScriptModification::SetEnvelope { name, value } => {\n                        self.data.apply_envelope_modification(name, value);\n                    }\n                }\n            }\n        }\n\n        // Build message\n        let mail_from = self.data.mail_from.clone().unwrap();\n        let rcpt_to = std::mem::take(&mut self.data.rcpt_to);\n        let mut message = self\n            .build_message(mail_from, rcpt_to, message_id, self.data.session_id)\n            .await;\n\n        // Add Return-Path\n        if self\n            .server\n            .eval_if(&dc.add_return_path, self, self.data.session_id)\n            .await\n            .unwrap_or(true)\n        {\n            headers.extend_from_slice(b\"Return-Path: <\");\n            headers.extend_from_slice(message.message.return_path.as_bytes());\n            headers.extend_from_slice(b\">\\r\\n\");\n        }\n\n        // Add any missing headers\n        if !has_date_header\n            && self\n                .server\n                .eval_if(&dc.add_date, self, self.data.session_id)\n                .await\n                .unwrap_or(true)\n        {\n            headers.extend_from_slice(b\"Date: \");\n            headers.extend_from_slice(Date::now().to_rfc822().as_bytes());\n            headers.extend_from_slice(b\"\\r\\n\");\n        }\n        if !has_message_id_header\n            && self\n                .server\n                .eval_if(&dc.add_message_id, self, self.data.session_id)\n                .await\n                .unwrap_or(true)\n        {\n            headers.extend_from_slice(b\"Message-ID: \");\n            let _ = generate_message_id_header(&mut headers, &self.hostname);\n            headers.extend_from_slice(b\"\\r\\n\");\n        }\n\n        // DKIM sign\n        let raw_message = edited_message.as_deref().unwrap_or(raw_message.as_slice());\n        for signer in self\n            .server\n            .eval_if::<Vec<String>, _>(&ac.dkim.sign, self, self.data.session_id)\n            .await\n            .unwrap_or_default()\n        {\n            if let Some(signer) = self.server.get_dkim_signer(&signer, self.data.session_id) {\n                match signer.sign_chained(&[headers.as_ref(), raw_message]) {\n                    Ok(signature) => {\n                        signature.write_header(&mut headers);\n                    }\n                    Err(err) => {\n                        trc::error!(\n                            trc::Error::from(err)\n                                .span_id(self.data.session_id)\n                                .details(\"Failed to DKIM sign message\")\n                        );\n                    }\n                }\n            }\n        }\n\n        // Update size\n        message.message.size = (raw_message.len() + headers.len()) as u64;\n\n        // Verify queue quota\n        if self.server.has_quota(&mut message).await {\n            // Prepare webhook event\n            let queue_id = message.queue_id;\n\n            // Queue message\n            let source = if !self.is_authenticated() {\n                let dmarc_pass = dmarc_result.is_some_and(|result| result == DmarcResult::Pass);\n\n                #[cfg(feature = \"test_mode\")]\n                {\n                    MessageSource::Unauthenticated {\n                        dmarc_pass: dmarc_pass || message.message.return_path.starts_with(\"dmarc-\"),\n                        train_spam,\n                    }\n                }\n\n                #[cfg(not(feature = \"test_mode\"))]\n                {\n                    MessageSource::Unauthenticated {\n                        dmarc_pass,\n                        train_spam,\n                    }\n                }\n            } else {\n                MessageSource::Authenticated\n            };\n            if message\n                .queue(\n                    Some(&headers),\n                    raw_message,\n                    self.data.session_id,\n                    &self.server,\n                    source,\n                )\n                .await\n            {\n                self.state = State::Accepted(queue_id);\n                self.data.messages_sent += 1;\n                format!(\"250 2.0.0 Message queued with id {queue_id:x}.\\r\\n\")\n                    .into_bytes()\n                    .into()\n            } else {\n                (b\"451 4.3.5 Unable to accept message at this time.\\r\\n\"[..]).into()\n            }\n        } else {\n            (b\"452 4.3.1 Mail system full, try again later.\\r\\n\"[..]).into()\n        }\n    }\n\n    pub async fn build_message(\n        &self,\n        mail_from: SessionAddress,\n        mut rcpt_to: Vec<SessionAddress>,\n        queue_id: u64,\n        span_id: u64,\n    ) -> MessageWrapper {\n        // Build message\n        let created = SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .map_or(0, |d| d.as_secs());\n        let mut message = Message {\n            created,\n            return_path: mail_from.address.to_lowercase_domain().into_boxed_str(),\n            recipients: Vec::with_capacity(rcpt_to.len()),\n            flags: mail_from.flags,\n            priority: self.data.priority,\n            size: 0,\n            env_id: mail_from.dsn_info.map(|i| i.into_boxed_str()),\n            blob_hash: Default::default(),\n            quota_keys: Default::default(),\n            received_from_ip: self.data.remote_ip,\n            received_via_port: self.data.local_port,\n        };\n\n        // Add recipients\n        let future_release = self.data.future_release;\n        rcpt_to.sort_unstable();\n        for rcpt in rcpt_to {\n            message.recipients.push(\n                queue::Recipient::new(rcpt.address)\n                    .with_flags(\n                        if rcpt.flags\n                            & (RCPT_NOTIFY_DELAY\n                                | RCPT_NOTIFY_FAILURE\n                                | RCPT_NOTIFY_SUCCESS\n                                | RCPT_NOTIFY_NEVER)\n                            != 0\n                        {\n                            rcpt.flags\n                        } else {\n                            rcpt.flags | RCPT_NOTIFY_DELAY | RCPT_NOTIFY_FAILURE\n                        },\n                    )\n                    .with_orcpt(rcpt.dsn_info.map(|v| v.into_boxed_str())),\n            );\n\n            let envelope = QueueEnvelope::new(&message, message.recipients.last().unwrap());\n\n            // Set next retry time\n            let retry = if self.data.future_release == 0 {\n                queue::Schedule::now()\n            } else {\n                queue::Schedule::later(future_release)\n            };\n\n            // Resolve queue\n            let queue = self.server.get_queue_or_default(\n                &self\n                    .server\n                    .eval_if::<String, _>(\n                        &self.server.core.smtp.queue.queue,\n                        &envelope,\n                        self.data.session_id,\n                    )\n                    .await\n                    .unwrap_or_else(|| \"default\".to_string()),\n                self.data.session_id,\n            );\n\n            // Set expiration and notification times\n            let num_intervals = std::cmp::max(queue.notify.len(), 1);\n            let next_notify = queue.notify.first().copied().unwrap_or(86400);\n            let (notify, expires) = if self.data.delivery_by == 0 {\n                (\n                    queue::Schedule::later(future_release + next_notify),\n                    match queue.expiry {\n                        QueueExpiry::Ttl(time) => QueueExpiry::Ttl(future_release + time),\n                        QueueExpiry::Attempts(count) => QueueExpiry::Attempts(count),\n                    },\n                )\n            } else if (message.flags & MAIL_BY_RETURN) != 0 {\n                (\n                    queue::Schedule::later(future_release + next_notify),\n                    QueueExpiry::Ttl(self.data.delivery_by as u64),\n                )\n            } else {\n                let (notify, expires) = match queue.expiry {\n                    QueueExpiry::Ttl(expire_secs) => (\n                        (if self.data.delivery_by.is_positive() {\n                            let notify_at = self.data.delivery_by as u64;\n                            if expire_secs > notify_at {\n                                notify_at\n                            } else {\n                                next_notify\n                            }\n                        } else {\n                            let notify_at = -self.data.delivery_by as u64;\n                            if expire_secs > notify_at {\n                                expire_secs - notify_at\n                            } else {\n                                next_notify\n                            }\n                        }),\n                        QueueExpiry::Ttl(expire_secs),\n                    ),\n                    QueueExpiry::Attempts(_) => (\n                        next_notify,\n                        QueueExpiry::Ttl(self.data.delivery_by.unsigned_abs()),\n                    ),\n                };\n\n                let mut notify = queue::Schedule::later(future_release + notify);\n                notify.inner = (num_intervals - 1) as u32; // Disable further notification attempts\n\n                (notify, expires)\n            };\n\n            // Update recipient\n            let recipient = message.recipients.last_mut().unwrap();\n            recipient.retry = retry;\n            recipient.notify = notify;\n            recipient.expires = expires;\n            recipient.queue = queue.virtual_queue;\n        }\n\n        MessageWrapper {\n            queue_id,\n            queue_name: QueueName::default(),\n            is_multi_queue: false,\n            span_id,\n            message,\n        }\n    }\n\n    pub async fn can_send_data(&mut self) -> Result<bool, ()> {\n        if !self.data.rcpt_to.is_empty() {\n            if self.data.messages_sent\n                < self\n                    .server\n                    .eval_if(\n                        &self.server.core.smtp.session.data.max_messages,\n                        self,\n                        self.data.session_id,\n                    )\n                    .await\n                    .unwrap_or(10)\n            {\n                Ok(true)\n            } else {\n                trc::event!(\n                    Smtp(SmtpEvent::TooManyMessages),\n                    SpanId = self.data.session_id,\n                    Limit = self.data.messages_sent\n                );\n\n                self.write(b\"452 4.4.5 Maximum number of messages per session exceeded.\\r\\n\")\n                    .await?;\n                Ok(false)\n            }\n        } else {\n            trc::event!(\n                Smtp(SmtpEvent::RcptToMissing),\n                SpanId = self.data.session_id,\n            );\n\n            self.write(b\"503 5.5.1 RCPT is required first.\\r\\n\").await?;\n            Ok(false)\n        }\n    }\n\n    fn write_received(&self, headers: &mut Vec<u8>, id: u64) {\n        headers.extend_from_slice(b\"Received: from \");\n        headers.extend_from_slice(self.data.helo_domain.as_bytes());\n        headers.extend_from_slice(b\" (\");\n        headers.extend_from_slice(\n            self.data\n                .iprev\n                .as_ref()\n                .and_then(|ir| ir.ptr.as_ref())\n                .and_then(|ptr| ptr.first().map(|s| s.strip_suffix('.').unwrap_or(s)))\n                .unwrap_or(\"unknown\")\n                .as_bytes(),\n        );\n        headers.extend_from_slice(b\" [\");\n        headers.extend_from_slice(self.data.remote_ip.to_string().as_bytes());\n        headers.extend_from_slice(b\"]\");\n        if self.data.asn_geo_data.asn.is_some() || self.data.asn_geo_data.country.is_some() {\n            headers.extend_from_slice(b\" (\");\n            if let Some(asn) = &self.data.asn_geo_data.asn {\n                headers.extend_from_slice(b\"AS\");\n                headers.extend_from_slice(asn.id.to_string().as_bytes());\n                if let Some(name) = &asn.name {\n                    headers.extend_from_slice(b\" \");\n                    headers.extend_from_slice(name.as_bytes());\n                }\n            }\n            if let Some(country) = &self.data.asn_geo_data.country {\n                if self.data.asn_geo_data.asn.is_some() {\n                    headers.extend_from_slice(b\", \");\n                }\n                headers.extend_from_slice(country.as_bytes());\n            }\n            headers.extend_from_slice(b\")\");\n        }\n        headers.extend_from_slice(b\")\\r\\n\\t\");\n        if self.stream.is_tls() {\n            let (version, cipher) = self.stream.tls_version_and_cipher();\n            headers.extend_from_slice(b\"(using \");\n            headers.extend_from_slice(version.as_bytes());\n            headers.extend_from_slice(b\" with cipher \");\n            headers.extend_from_slice(cipher.as_bytes());\n            headers.extend_from_slice(b\")\\r\\n\\t\");\n        }\n        headers.extend_from_slice(b\"by \");\n        headers.extend_from_slice(self.hostname.as_bytes());\n        headers.extend_from_slice(b\" (Stalwart SMTP) with \");\n        headers.extend_from_slice(match (self.stream.is_tls(), !self.is_authenticated()) {\n            (true, true) => b\"ESMTPS\",\n            (true, false) => b\"ESMTPSA\",\n            (false, true) => b\"ESMTP\",\n            (false, false) => b\"ESMTPA\",\n        });\n        headers.extend_from_slice(b\" id \");\n        headers.extend_from_slice(format!(\"{id:X}\").as_bytes());\n        headers.extend_from_slice(b\";\\r\\n\\t\");\n        headers.extend_from_slice(Date::now().to_rfc822().as_bytes());\n        headers.extend_from_slice(b\"\\r\\n\");\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/ehlo.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{core::Session, scripts::ScriptResult};\nuse common::{\n    config::smtp::session::{Mechanism, Stage},\n    listener::SessionStream,\n};\nuse mail_auth::{\n    SpfResult,\n    spf::verify::{HasValidLabels, SpfParameters},\n};\nuse smtp_proto::*;\nuse std::{\n    borrow::Cow,\n    time::{Duration, Instant, SystemTime},\n};\nuse trc::SmtpEvent;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_ehlo(&mut self, domain: Cow<'_, str>, is_extended: bool) -> Result<(), ()> {\n        // Set EHLO domain\n\n        if domain != self.data.helo_domain {\n            // Reject non-FQDN EHLO domains - simply checks that the hostname has at least one dot\n            if self.params.ehlo_reject_non_fqdn && !domain.as_ref().has_valid_labels() {\n                trc::event!(\n                    Smtp(SmtpEvent::InvalidEhlo),\n                    SpanId = self.data.session_id,\n                    Domain = domain.as_ref().to_string(),\n                );\n\n                return self.write(b\"550 5.5.0 Invalid EHLO domain.\\r\\n\").await;\n            }\n\n            trc::event!(\n                Smtp(SmtpEvent::Ehlo),\n                SpanId = self.data.session_id,\n                Domain = domain.as_ref().to_string(),\n            );\n\n            // SPF check\n            let prev_helo_domain =\n                std::mem::replace(&mut self.data.helo_domain, domain.into_owned());\n            if self.params.spf_ehlo.verify() {\n                let time = Instant::now();\n                let spf_output = self\n                    .server\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .verify_spf(self.server.inner.cache.build_auth_parameters(\n                        SpfParameters::verify_ehlo(\n                            self.data.remote_ip,\n                            &self.data.helo_domain,\n                            &self.hostname,\n                        ),\n                    ))\n                    .await;\n\n                trc::event!(\n                    Smtp(if matches!(spf_output.result(), SpfResult::Pass) {\n                        SmtpEvent::SpfEhloPass\n                    } else {\n                        SmtpEvent::SpfEhloFail\n                    }),\n                    SpanId = self.data.session_id,\n                    Domain = self.data.helo_domain.clone(),\n                    Result = trc::Error::from(&spf_output),\n                    Elapsed = time.elapsed(),\n                );\n\n                if self\n                    .handle_spf(&spf_output, self.params.spf_ehlo.is_strict())\n                    .await?\n                {\n                    self.data.spf_ehlo = spf_output.into();\n                } else {\n                    self.data.mail_from = None;\n                    self.data.helo_domain = prev_helo_domain;\n                    return Ok(());\n                }\n            }\n\n            // Sieve filtering\n            if let Some((script, script_id)) = self\n                .server\n                .eval_if::<String, _>(\n                    &self.server.core.smtp.session.ehlo.script,\n                    self,\n                    self.data.session_id,\n                )\n                .await\n                .and_then(|name| {\n                    self.server\n                        .get_trusted_sieve_script(&name, self.data.session_id)\n                        .map(|s| (s, name))\n                })\n                && let ScriptResult::Reject(message) = self\n                    .run_script(\n                        script_id,\n                        script.clone(),\n                        self.build_script_parameters(\"ehlo\"),\n                    )\n                    .await\n            {\n                self.data.mail_from = None;\n                self.data.helo_domain = prev_helo_domain;\n                self.data.spf_ehlo = None;\n                return self.write(message.as_bytes()).await;\n            }\n\n            // Milter filtering\n            if let Err(message) = self.run_milters(Stage::Ehlo, None).await {\n                self.data.mail_from = None;\n                self.data.helo_domain = prev_helo_domain;\n                self.data.spf_ehlo = None;\n                return self.write(message.message.as_bytes()).await;\n            }\n\n            // MTAHook filtering\n            if let Err(message) = self.run_mta_hooks(Stage::Ehlo, None, None).await {\n                self.data.mail_from = None;\n                self.data.helo_domain = prev_helo_domain;\n                self.data.spf_ehlo = None;\n                return self.write(message.message.as_bytes()).await;\n            }\n        }\n\n        // Reset\n        if self.data.mail_from.is_some() {\n            self.reset();\n        }\n\n        if !is_extended {\n            return self\n                .write(format!(\"250 {} you had me at HELO\\r\\n\", self.hostname).as_bytes())\n                .await;\n        }\n\n        let mut response = EhloResponse::new(self.hostname.as_str());\n        response.capabilities =\n            EXT_ENHANCED_STATUS_CODES | EXT_8BIT_MIME | EXT_BINARY_MIME | EXT_SMTP_UTF8;\n        if !self.stream.is_tls() && self.instance.acceptor.is_tls() {\n            response.capabilities |= EXT_START_TLS;\n        }\n        let ec = &self.server.core.smtp.session.extensions;\n        let ac = &self.server.core.smtp.session.auth;\n        let dc = &self.server.core.smtp.session.data;\n\n        // Pipelining\n        if self\n            .server\n            .eval_if(&ec.pipelining, self, self.data.session_id)\n            .await\n            .unwrap_or(true)\n        {\n            response.capabilities |= EXT_PIPELINING;\n        }\n\n        // Chunking\n        if self\n            .server\n            .eval_if(&ec.chunking, self, self.data.session_id)\n            .await\n            .unwrap_or(true)\n        {\n            response.capabilities |= EXT_CHUNKING;\n        }\n\n        // Address Expansion\n        if self\n            .server\n            .eval_if(&ec.expn, self, self.data.session_id)\n            .await\n            .unwrap_or(false)\n        {\n            response.capabilities |= EXT_EXPN;\n        }\n\n        // Recipient Verification\n        if self\n            .server\n            .eval_if(&ec.vrfy, self, self.data.session_id)\n            .await\n            .unwrap_or(false)\n        {\n            response.capabilities |= EXT_VRFY;\n        }\n\n        // Require TLS\n        if self\n            .server\n            .eval_if(&ec.requiretls, self, self.data.session_id)\n            .await\n            .unwrap_or(true)\n        {\n            response.capabilities |= EXT_REQUIRE_TLS;\n        }\n\n        // DSN\n        if self\n            .server\n            .eval_if(&ec.dsn, self, self.data.session_id)\n            .await\n            .unwrap_or(false)\n        {\n            response.capabilities |= EXT_DSN;\n        }\n\n        // Authentication\n        if !self.is_authenticated() {\n            response.auth_mechanisms = self\n                .server\n                .eval_if::<Mechanism, _>(&ac.mechanisms, self, self.data.session_id)\n                .await\n                .unwrap_or_default()\n                .into();\n            if response.auth_mechanisms != 0 {\n                response.capabilities |= EXT_AUTH;\n            }\n        }\n\n        // Future release\n        if let Some(value) = self\n            .server\n            .eval_if::<Duration, _>(&ec.future_release, self, self.data.session_id)\n            .await\n        {\n            response.capabilities |= EXT_FUTURE_RELEASE;\n            response.future_release_interval = value.as_secs();\n            response.future_release_datetime = SystemTime::now()\n                .duration_since(SystemTime::UNIX_EPOCH)\n                .map(|d| d.as_secs())\n                .unwrap_or(0)\n                + value.as_secs();\n        }\n\n        // Deliver By\n        if let Some(value) = self\n            .server\n            .eval_if::<Duration, _>(&ec.deliver_by, self, self.data.session_id)\n            .await\n        {\n            response.capabilities |= EXT_DELIVER_BY;\n            response.deliver_by = value.as_secs();\n        }\n\n        // Priority\n        if let Some(value) = self\n            .server\n            .eval_if::<MtPriority, _>(&ec.mt_priority, self, self.data.session_id)\n            .await\n        {\n            response.capabilities |= EXT_MT_PRIORITY;\n            response.mt_priority = value;\n        }\n\n        // Size\n        response.size = self\n            .server\n            .eval_if(&dc.max_message_size, self, self.data.session_id)\n            .await\n            .unwrap_or(25 * 1024 * 1024);\n        if response.size > 0 {\n            response.capabilities |= EXT_SIZE;\n        }\n\n        // No soliciting\n        if let Some(value) = self\n            .server\n            .eval_if::<String, _>(&ec.no_soliciting, self, self.data.session_id)\n            .await\n        {\n            response.capabilities |= EXT_NO_SOLICITING;\n            response.no_soliciting = if !value.is_empty() {\n                value.to_string().into()\n            } else {\n                None\n            };\n        }\n\n        // Generate response\n        let mut buf = Vec::with_capacity(64);\n        response.write(&mut buf).ok();\n        self.write(&buf).await\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/hooks/client.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::config::smtp::session::MTAHook;\nuse utils::HttpLimitResponse;\n\nuse super::{Request, Response};\n\npub(super) async fn send_mta_hook_request(\n    mta_hook: &MTAHook,\n    request: Request,\n) -> Result<Response, String> {\n    let response = reqwest::Client::builder()\n        .timeout(mta_hook.timeout)\n        .danger_accept_invalid_certs(mta_hook.tls_allow_invalid_certs)\n        .build()\n        .map_err(|err| format!(\"Failed to create HTTP client: {}\", err))?\n        .post(&mta_hook.url)\n        .headers(mta_hook.headers.clone())\n        .body(\n            serde_json::to_string(&request)\n                .map_err(|err| format!(\"Failed to serialize Hook request: {}\", err))?,\n        )\n        .send()\n        .await\n        .map_err(|err| format!(\"Hook request failed: {err}\"))?;\n\n    if response.status().is_success() {\n        serde_json::from_slice(\n            response\n                .bytes_with_limit(mta_hook.max_response_size)\n                .await\n                .map_err(|err| format!(\"Failed to parse Hook response: {}\", err))?\n                .ok_or_else(|| \"Hook response too large\".to_string())?\n                .as_ref(),\n        )\n        .map_err(|err| format!(\"Failed to parse Hook response: {}\", err))\n    } else {\n        Err(format!(\n            \"Hook request failed with code {}: {}\",\n            response.status().as_u16(),\n            response.status().canonical_reason().unwrap_or(\"Unknown\")\n        ))\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/hooks/message.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse ahash::AHashMap;\nuse common::{\n    DAEMON_NAME,\n    config::smtp::session::{MTAHook, Stage},\n    listener::SessionStream,\n};\n\nuse mail_auth::AuthenticatedMessage;\nuse trc::MtaHookEvent;\n\nuse crate::{\n    core::Session,\n    inbound::{\n        FilterResponse,\n        hooks::{\n            Address, Client, Context, Envelope, Message, Protocol, Request, Sasl, Server, Tls,\n        },\n        milter::Modification,\n    },\n    queue::QueueId,\n};\n\nuse super::{Action, Queue, Response, client::send_mta_hook_request};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn run_mta_hooks(\n        &self,\n        stage: Stage,\n        message: Option<&AuthenticatedMessage<'_>>,\n        queue_id: Option<QueueId>,\n    ) -> Result<Vec<Modification>, FilterResponse> {\n        let mta_hooks = &self.server.core.smtp.session.hooks;\n        if mta_hooks.is_empty() {\n            return Ok(Vec::new());\n        }\n\n        let mut modifications = Vec::new();\n        for mta_hook in mta_hooks {\n            if !mta_hook.run_on_stage.contains(&stage)\n                || !self\n                    .server\n                    .eval_if(&mta_hook.enable, self, self.data.session_id)\n                    .await\n                    .unwrap_or(false)\n            {\n                continue;\n            }\n\n            let time = Instant::now();\n            match self.run_mta_hook(stage, mta_hook, message, queue_id).await {\n                Ok(response) => {\n                    trc::event!(\n                        MtaHook(match response.action {\n                            Action::Accept => MtaHookEvent::ActionAccept,\n                            Action::Discard => MtaHookEvent::ActionDiscard,\n                            Action::Reject => MtaHookEvent::ActionReject,\n                            Action::Quarantine => MtaHookEvent::ActionQuarantine,\n                        }),\n                        SpanId = self.data.session_id,\n                        Id = mta_hook.id.clone(),\n                        Elapsed = time.elapsed(),\n                    );\n\n                    let mut new_modifications = Vec::with_capacity(response.modifications.len());\n                    for modification in response.modifications {\n                        new_modifications.push(match modification {\n                            super::Modification::ChangeFrom { value, parameters } => {\n                                Modification::ChangeFrom {\n                                    sender: value,\n                                    args: flatten_parameters(parameters),\n                                }\n                            }\n                            super::Modification::AddRecipient { value, parameters } => {\n                                Modification::AddRcpt {\n                                    recipient: value,\n                                    args: flatten_parameters(parameters),\n                                }\n                            }\n                            super::Modification::DeleteRecipient { value } => {\n                                Modification::DeleteRcpt { recipient: value }\n                            }\n                            super::Modification::ReplaceContents { value } => {\n                                Modification::ReplaceBody {\n                                    value: value.as_bytes().to_vec(),\n                                }\n                            }\n                            super::Modification::AddHeader { name, value } => {\n                                Modification::AddHeader { name, value }\n                            }\n                            super::Modification::InsertHeader { index, name, value } => {\n                                Modification::InsertHeader { index, name, value }\n                            }\n                            super::Modification::ChangeHeader { index, name, value } => {\n                                Modification::ChangeHeader { index, name, value }\n                            }\n                            super::Modification::DeleteHeader { index, name } => {\n                                Modification::ChangeHeader {\n                                    index,\n                                    name,\n                                    value: String::new(),\n                                }\n                            }\n                        });\n                    }\n\n                    if !modifications.is_empty() {\n                        // The message body can only be replaced once, so we need to remove\n                        // any previous replacements.\n                        if new_modifications\n                            .iter()\n                            .any(|m| matches!(m, Modification::ReplaceBody { .. }))\n                        {\n                            modifications\n                                .retain(|m| !matches!(m, Modification::ReplaceBody { .. }));\n                        }\n                        modifications.extend(new_modifications);\n                    } else {\n                        modifications = new_modifications;\n                    }\n\n                    let mut message = match response.action {\n                        Action::Accept => continue,\n                        Action::Discard => FilterResponse::accept(),\n                        Action::Reject => FilterResponse::reject(),\n                        Action::Quarantine => {\n                            modifications.push(Modification::AddHeader {\n                                name: \"X-Quarantine\".into(),\n                                value: \"true\".into(),\n                            });\n                            FilterResponse::accept()\n                        }\n                    };\n\n                    if let Some(response) = response.response {\n                        if let (Some(status), Some(text)) = (response.status, response.message) {\n                            if let Some(enhanced) = response.enhanced_status {\n                                message.message = format!(\"{status} {enhanced} {text}\\r\\n\").into();\n                            } else {\n                                message.message = format!(\"{status} {text}\\r\\n\").into();\n                            }\n                        }\n                        message.disconnect = response.disconnect;\n                    }\n\n                    return Err(message);\n                }\n                Err(err) => {\n                    trc::event!(\n                        MtaHook(MtaHookEvent::Error),\n                        SpanId = self.data.session_id,\n                        Id = mta_hook.id.clone(),\n                        Reason = err,\n                        Elapsed = time.elapsed(),\n                    );\n\n                    if mta_hook.tempfail_on_error {\n                        return Err(FilterResponse::server_failure());\n                    }\n                }\n            }\n        }\n\n        Ok(modifications)\n    }\n\n    pub async fn run_mta_hook(\n        &self,\n        stage: Stage,\n        mta_hook: &MTAHook,\n        message: Option<&AuthenticatedMessage<'_>>,\n        queue_id: Option<QueueId>,\n    ) -> Result<Response, String> {\n        // Build request\n        let (tls_version, tls_cipher) = self.stream.tls_version_and_cipher();\n        let request = Request {\n            context: Context {\n                stage: stage.into(),\n                client: Client {\n                    ip: self.data.remote_ip.to_string(),\n                    port: self.data.remote_port,\n                    ptr: self\n                        .data\n                        .iprev\n                        .as_ref()\n                        .and_then(|ip_rev| ip_rev.ptr.as_ref())\n                        .and_then(|ptrs| ptrs.first())\n                        .map(Into::into),\n                    helo: (!self.data.helo_domain.is_empty())\n                        .then(|| self.data.helo_domain.clone()),\n                    active_connections: 1,\n                },\n                sasl: self.authenticated_as().map(|name| Sasl {\n                    login: name.into(),\n                    method: None,\n                }),\n                tls: (!tls_version.is_empty()).then(|| Tls {\n                    version: tls_version.as_ref().into(),\n                    cipher: tls_cipher.as_ref().into(),\n                    bits: None,\n                    issuer: None,\n                    subject: None,\n                }),\n                server: Server {\n                    name: Some(DAEMON_NAME.into()),\n                    port: self.data.local_port,\n                    ip: self.data.local_ip.to_string().into(),\n                },\n                queue: queue_id.map(|id| Queue {\n                    id: format!(\"{:x}\", id),\n                }),\n                protocol: Protocol { version: 1 },\n            },\n            envelope: self.data.mail_from.as_ref().map(|from| Envelope {\n                from: Address {\n                    address: from.address_lcase.clone(),\n                    parameters: None,\n                },\n                to: self\n                    .data\n                    .rcpt_to\n                    .iter()\n                    .map(|to| Address {\n                        address: to.address_lcase.clone(),\n                        parameters: None,\n                    })\n                    .collect(),\n            }),\n            message: message.map(|message| Message {\n                headers: message\n                    .raw_parsed_headers()\n                    .iter()\n                    .map(|(k, v)| {\n                        (\n                            String::from_utf8_lossy(k).into_owned(),\n                            String::from_utf8_lossy(v).into_owned(),\n                        )\n                    })\n                    .collect(),\n                server_headers: vec![],\n                contents: String::from_utf8_lossy(message.raw_body()).into_owned(),\n                size: message.raw_message().len(),\n            }),\n        };\n\n        send_mta_hook_request(mta_hook, request).await\n    }\n}\n\nfn flatten_parameters(parameters: AHashMap<String, Option<String>>) -> String {\n    let mut arguments = String::new();\n    for (key, value) in parameters {\n        if !arguments.is_empty() {\n            arguments.push(' ');\n        }\n        arguments.push_str(key.as_str());\n        if let Some(value) = value {\n            arguments.push('=');\n            arguments.push_str(value.as_str());\n        }\n    }\n\n    arguments\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/hooks/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod client;\npub mod message;\n\nuse ahash::AHashMap;\n\nuse serde::{Deserialize, Serialize};\n\n#[derive(Serialize, Deserialize)]\npub struct Request {\n    pub context: Context,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub envelope: Option<Envelope>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub message: Option<Message>,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Context {\n    pub stage: Stage,\n    pub client: Client,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub sasl: Option<Sasl>,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub tls: Option<Tls>,\n    pub server: Server,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub queue: Option<Queue>,\n    pub protocol: Protocol,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Sasl {\n    pub login: String,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub method: Option<String>,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Client {\n    pub ip: String,\n    pub port: u16,\n    pub ptr: Option<String>,\n    pub helo: Option<String>,\n    #[serde(rename = \"activeConnections\")]\n    pub active_connections: u32,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Tls {\n    pub version: String,\n    pub cipher: String,\n    #[serde(rename = \"cipherBits\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub bits: Option<u16>,\n    #[serde(rename = \"certIssuer\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub issuer: Option<String>,\n    #[serde(rename = \"certSubject\")]\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub subject: Option<String>,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Server {\n    pub name: Option<String>,\n    pub port: u16,\n    pub ip: Option<String>,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Queue {\n    pub id: String,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Protocol {\n    pub version: u32,\n}\n\n#[derive(Serialize, Deserialize)]\npub enum Stage {\n    #[serde(rename = \"connect\")]\n    Connect,\n    #[serde(rename = \"ehlo\")]\n    Ehlo,\n    #[serde(rename = \"auth\")]\n    Auth,\n    #[serde(rename = \"mail\")]\n    Mail,\n    #[serde(rename = \"rcpt\")]\n    Rcpt,\n    #[serde(rename = \"data\")]\n    Data,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Address {\n    pub address: String,\n    #[serde(skip_serializing_if = \"Option::is_none\")]\n    pub parameters: Option<AHashMap<String, String>>,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Envelope {\n    pub from: Address,\n    pub to: Vec<Address>,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Message {\n    pub headers: Vec<(String, String)>,\n    #[serde(skip_serializing_if = \"Vec::is_empty\")]\n    #[serde(rename = \"serverHeaders\")]\n    #[serde(default)]\n    pub server_headers: Vec<(String, String)>,\n    pub contents: String,\n    pub size: usize,\n}\n\n#[derive(Serialize, Deserialize)]\npub struct Response {\n    pub action: Action,\n    #[serde(default)]\n    pub response: Option<SmtpResponse>,\n    #[serde(default)]\n    pub modifications: Vec<Modification>,\n}\n\n#[derive(Serialize, Deserialize)]\npub enum Action {\n    #[serde(rename = \"accept\")]\n    Accept,\n    #[serde(rename = \"discard\")]\n    Discard,\n    #[serde(rename = \"reject\")]\n    Reject,\n    #[serde(rename = \"quarantine\")]\n    Quarantine,\n}\n\n#[derive(Serialize, Deserialize, Default)]\npub struct SmtpResponse {\n    #[serde(default)]\n    pub status: Option<u16>,\n    #[serde(default)]\n    pub enhanced_status: Option<String>,\n    #[serde(default)]\n    pub message: Option<String>,\n    #[serde(default)]\n    pub disconnect: bool,\n}\n\n#[derive(Serialize, Deserialize, Debug)]\n#[serde(tag = \"type\")]\npub enum Modification {\n    #[serde(rename = \"changeFrom\")]\n    ChangeFrom {\n        value: String,\n        #[serde(default)]\n        parameters: AHashMap<String, Option<String>>,\n    },\n    #[serde(rename = \"addRecipient\")]\n    AddRecipient {\n        value: String,\n        #[serde(default)]\n        parameters: AHashMap<String, Option<String>>,\n    },\n    #[serde(rename = \"deleteRecipient\")]\n    DeleteRecipient { value: String },\n    #[serde(rename = \"replaceContents\")]\n    ReplaceContents { value: String },\n    #[serde(rename = \"addHeader\")]\n    AddHeader { name: String, value: String },\n    #[serde(rename = \"insertHeader\")]\n    InsertHeader {\n        index: u32,\n        name: String,\n        value: String,\n    },\n    #[serde(rename = \"changeHeader\")]\n    ChangeHeader {\n        index: u32,\n        name: String,\n        value: String,\n    },\n    #[serde(rename = \"deleteHeader\")]\n    DeleteHeader { index: u32, name: String },\n}\n\nimpl From<common::config::smtp::session::Stage> for Stage {\n    fn from(value: common::config::smtp::session::Stage) -> Self {\n        match value {\n            common::config::smtp::session::Stage::Connect => Stage::Connect,\n            common::config::smtp::session::Stage::Ehlo => Stage::Ehlo,\n            common::config::smtp::session::Stage::Auth => Stage::Auth,\n            common::config::smtp::session::Stage::Mail => Stage::Mail,\n            common::config::smtp::session::Stage::Rcpt => Stage::Rcpt,\n            common::config::smtp::session::Stage::Data => Stage::Data,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/mail.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{Session, SessionAddress},\n    scripts::ScriptResult,\n};\nuse common::{config::smtp::session::Stage, listener::SessionStream, scripts::ScriptModification};\nuse mail_auth::{IprevOutput, IprevResult, SpfOutput, SpfResult, spf::verify::SpfParameters};\nuse smtp_proto::{MAIL_BY_NOTIFY, MAIL_BY_RETURN, MAIL_REQUIRETLS, MailFrom, MtPriority};\nuse std::{\n    borrow::Cow,\n    time::{Duration, Instant, SystemTime},\n};\nuse trc::SmtpEvent;\nuse utils::{DomainPart, config::Rate};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_mail_from(&mut self, from: MailFrom<Cow<'_, str>>) -> Result<(), ()> {\n        if self.data.helo_domain.is_empty()\n            && (self.params.ehlo_require\n                || self.params.spf_ehlo.verify()\n                || self.params.spf_mail_from.verify())\n        {\n            trc::event!(\n                Smtp(SmtpEvent::DidNotSayEhlo),\n                SpanId = self.data.session_id,\n            );\n\n            return self\n                .write(b\"503 5.5.1 Polite people say EHLO first.\\r\\n\")\n                .await;\n        } else if self.data.mail_from.is_some() {\n            trc::event!(\n                Smtp(SmtpEvent::MultipleMailFrom),\n                SpanId = self.data.session_id,\n            );\n\n            return self\n                .write(b\"503 5.5.1 Multiple MAIL commands not allowed.\\r\\n\")\n                .await;\n        } else if self.params.auth_require && !self.is_authenticated() {\n            trc::event!(\n                Smtp(SmtpEvent::MailFromUnauthenticated),\n                SpanId = self.data.session_id,\n            );\n\n            return self\n                .write(b\"503 5.5.1 You must authenticate first.\\r\\n\")\n                .await;\n        } else if self.data.iprev.is_none() && self.params.iprev.verify() {\n            let time = Instant::now();\n            let iprev = self\n                .server\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .verify_iprev(\n                    self.server\n                        .inner\n                        .cache\n                        .build_auth_parameters(self.data.remote_ip),\n                )\n                .await;\n\n            trc::event!(\n                Smtp(if matches!(iprev.result(), IprevResult::Pass) {\n                    SmtpEvent::IprevPass\n                } else {\n                    SmtpEvent::IprevFail\n                }),\n                SpanId = self.data.session_id,\n                Domain = self.data.helo_domain.clone(),\n                Result = trc::Error::from(&iprev),\n                Elapsed = time.elapsed(),\n            );\n\n            self.data.iprev = iprev.into();\n        }\n\n        // In strict mode reject messages from hosts that fail the reverse DNS lookup check\n        if self.params.iprev.is_strict()\n            && !matches!(\n                &self.data.iprev,\n                Some(IprevOutput {\n                    result: IprevResult::Pass,\n                    ..\n                })\n            )\n        {\n            let message = if matches!(\n                &self.data.iprev,\n                Some(IprevOutput {\n                    result: IprevResult::TempError(_),\n                    ..\n                })\n            ) {\n                &b\"451 4.7.25 Temporary error validating reverse DNS.\\r\\n\"[..]\n            } else {\n                &b\"550 5.7.25 Reverse DNS validation failed.\\r\\n\"[..]\n            };\n\n            return self.write(message).await;\n        }\n\n        let (address, address_lcase, domain) = if !from.address.is_empty() {\n            let address_lcase = from.address.to_lowercase();\n            let domain = address_lcase.domain_part().into();\n            (from.address.into_owned(), address_lcase, domain)\n        } else {\n            (String::new(), String::new(), String::new())\n        };\n\n        let has_dsn = from.env_id.is_some();\n        self.data.mail_from = SessionAddress {\n            address,\n            address_lcase,\n            domain,\n            flags: from.flags,\n            dsn_info: from.env_id.map(|e| e.into_owned()),\n        }\n        .into();\n\n        // Check whether the address is allowed\n        if !self\n            .server\n            .eval_if::<bool, _>(\n                &self.server.core.smtp.session.mail.is_allowed,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .unwrap_or(true)\n        {\n            let mail_from = self.data.mail_from.take().unwrap();\n            trc::event!(\n                Smtp(SmtpEvent::MailFromNotAllowed),\n                From = mail_from.address_lcase,\n                SpanId = self.data.session_id,\n            );\n            return self\n                .write(b\"550 5.7.1 Sender address not allowed.\\r\\n\")\n                .await;\n        }\n\n        // Sieve filtering\n        if let Some((script, script_id)) = self\n            .server\n            .eval_if::<String, _>(\n                &self.server.core.smtp.session.mail.script,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .and_then(|name| {\n                self.server\n                    .get_trusted_sieve_script(&name, self.data.session_id)\n                    .map(|s| (s, name))\n            })\n        {\n            match self\n                .run_script(\n                    script_id,\n                    script.clone(),\n                    self.build_script_parameters(\"mail\"),\n                )\n                .await\n            {\n                ScriptResult::Accept { modifications } => {\n                    if !modifications.is_empty() {\n                        for modification in modifications {\n                            if let ScriptModification::SetEnvelope { name, value } = modification {\n                                self.data.apply_envelope_modification(name, value);\n                            }\n                        }\n                    }\n                }\n                ScriptResult::Reject(message) => {\n                    self.data.mail_from = None;\n                    return self.write(message.as_bytes()).await;\n                }\n                _ => (),\n            }\n        }\n\n        // Milter filtering\n        if let Err(message) = self.run_milters(Stage::Mail, None).await {\n            self.data.mail_from = None;\n            return self.write(message.message.as_bytes()).await;\n        }\n\n        // MTAHook filtering\n        if let Err(message) = self.run_mta_hooks(Stage::Mail, None, None).await {\n            self.data.mail_from = None;\n            return self.write(message.message.as_bytes()).await;\n        }\n\n        // Address rewriting\n        if let Some(new_address) = self\n            .server\n            .eval_if::<String, _>(\n                &self.server.core.smtp.session.mail.rewrite,\n                self,\n                self.data.session_id,\n            )\n            .await\n        {\n            let mail_from = self.data.mail_from.as_mut().unwrap();\n\n            trc::event!(\n                Smtp(SmtpEvent::MailFromRewritten),\n                SpanId = self.data.session_id,\n                Details = mail_from.address_lcase.clone(),\n                From = new_address.clone(),\n            );\n\n            if new_address.contains('@') {\n                mail_from.address_lcase = new_address.to_lowercase();\n                mail_from.domain = mail_from.address_lcase.domain_part().into();\n                mail_from.address = new_address;\n            } else if new_address.is_empty() {\n                mail_from.address_lcase.clear();\n                mail_from.domain.clear();\n                mail_from.address.clear();\n            }\n        }\n\n        // Make sure that the authenticated user is allowed to send from this address\n        match self.authenticated_as() {\n            Some(authenticated_as)\n                if self\n                    .server\n                    .eval_if(\n                        &self.server.core.smtp.session.auth.must_match_sender,\n                        self,\n                        self.data.session_id,\n                    )\n                    .await\n                    .unwrap_or(true) =>\n            {\n                let address_lcase = self.data.mail_from.as_ref().unwrap().address_lcase.as_str();\n                if authenticated_as != address_lcase\n                    && !self.authenticated_emails().iter().any(|e| {\n                        e == address_lcase\n                            || (e.starts_with('@') && address_lcase.ends_with(e.as_str()))\n                    })\n                {\n                    trc::event!(\n                        Smtp(SmtpEvent::MailFromUnauthorized),\n                        SpanId = self.data.session_id,\n                        From = address_lcase.to_string(),\n                        Details = [trc::Value::String(authenticated_as.into())]\n                            .into_iter()\n                            .chain(\n                                self.authenticated_emails()\n                                    .iter()\n                                    .map(|e| trc::Value::String(e.as_str().into()))\n                            )\n                            .collect::<Vec<_>>()\n                    );\n                    self.data.mail_from = None;\n                    return self\n                        .write(b\"501 5.5.4 You are not allowed to send from this address.\\r\\n\")\n                        .await;\n                }\n            }\n            _ => (),\n        }\n\n        // Validate parameters\n        let config = &self.server.core.smtp.session.extensions;\n        let config_data = &self.server.core.smtp.session.data;\n        if (from.flags & MAIL_REQUIRETLS) != 0\n            && !self\n                .server\n                .eval_if(&config.requiretls, self, self.data.session_id)\n                .await\n                .unwrap_or(false)\n        {\n            trc::event!(\n                Smtp(SmtpEvent::RequireTlsDisabled),\n                SpanId = self.data.session_id,\n            );\n            self.data.mail_from = None;\n            return self\n                .write(b\"501 5.5.4 REQUIRETLS has been disabled.\\r\\n\")\n                .await;\n        }\n        if (from.flags & (MAIL_BY_NOTIFY | MAIL_BY_RETURN)) != 0 {\n            if let Some(duration) = self\n                .server\n                .eval_if::<Duration, _>(&config.deliver_by, self, self.data.session_id)\n                .await\n            {\n                if from.by.checked_abs().unwrap_or(0) as u64 <= duration.as_secs()\n                    && (from.by.is_positive() || (from.flags & MAIL_BY_NOTIFY) != 0)\n                {\n                    self.data.delivery_by = from.by;\n                } else {\n                    self.data.mail_from = None;\n\n                    trc::event!(\n                        Smtp(SmtpEvent::DeliverByInvalid),\n                        SpanId = self.data.session_id,\n                        Details = from.by,\n                    );\n\n                    return self\n                        .write(\n                            format!(\n                                \"501 5.5.4 BY parameter exceeds maximum of {} seconds.\\r\\n\",\n                                duration.as_secs()\n                            )\n                            .as_bytes(),\n                        )\n                        .await;\n                }\n            } else {\n                trc::event!(\n                    Smtp(SmtpEvent::DeliverByDisabled),\n                    SpanId = self.data.session_id,\n                );\n                self.data.mail_from = None;\n                return self\n                    .write(b\"501 5.5.4 DELIVERBY extension has been disabled.\\r\\n\")\n                    .await;\n            }\n        }\n        if from.mt_priority != 0 {\n            if self\n                .server\n                .eval_if::<MtPriority, _>(&config.mt_priority, self, self.data.session_id)\n                .await\n                .is_some()\n            {\n                if (-6..6).contains(&from.mt_priority) {\n                    self.data.priority = from.mt_priority as i16;\n                } else {\n                    trc::event!(\n                        Smtp(SmtpEvent::MtPriorityInvalid),\n                        SpanId = self.data.session_id,\n                        Details = from.mt_priority,\n                    );\n                    self.data.mail_from = None;\n                    return self.write(b\"501 5.5.4 Invalid priority value.\\r\\n\").await;\n                }\n            } else {\n                trc::event!(\n                    Smtp(SmtpEvent::MtPriorityDisabled),\n                    SpanId = self.data.session_id,\n                );\n                self.data.mail_from = None;\n                return self\n                    .write(b\"501 5.5.4 MT-PRIORITY extension has been disabled.\\r\\n\")\n                    .await;\n            }\n        }\n        if from.size > 0\n            && from.size\n                > self\n                    .server\n                    .eval_if(&config_data.max_message_size, self, self.data.session_id)\n                    .await\n                    .unwrap_or(25 * 1024 * 1024)\n        {\n            trc::event!(\n                Smtp(SmtpEvent::MessageTooLarge),\n                SpanId = self.data.session_id,\n                Size = from.size,\n            );\n\n            self.data.mail_from = None;\n            return self\n                .write(b\"552 5.3.4 Message too big for system.\\r\\n\")\n                .await;\n        }\n        if from.hold_for != 0 || from.hold_until != 0 {\n            if let Some(max_hold) = self\n                .server\n                .eval_if::<Duration, _>(&config.future_release, self, self.data.session_id)\n                .await\n            {\n                let max_hold = max_hold.as_secs();\n                let hold_for = if from.hold_for != 0 {\n                    from.hold_for\n                } else {\n                    let now = SystemTime::now()\n                        .duration_since(SystemTime::UNIX_EPOCH)\n                        .map_or(0, |d| d.as_secs());\n                    from.hold_until.saturating_sub(now)\n                };\n                if hold_for <= max_hold {\n                    self.data.future_release = hold_for;\n                } else {\n                    trc::event!(\n                        Smtp(SmtpEvent::FutureReleaseInvalid),\n                        SpanId = self.data.session_id,\n                        Details = hold_for,\n                    );\n                    self.data.mail_from = None;\n                    return self\n                        .write(\n                            format!(\n                                \"501 5.5.4 Requested hold time exceeds maximum of {max_hold} seconds.\\r\\n\"\n                            )\n                            .as_bytes(),\n                        )\n                        .await;\n                }\n            } else {\n                trc::event!(\n                    Smtp(SmtpEvent::FutureReleaseDisabled),\n                    SpanId = self.data.session_id,\n                );\n                self.data.mail_from = None;\n                return self\n                    .write(b\"501 5.5.4 FUTURERELEASE extension has been disabled.\\r\\n\")\n                    .await;\n            }\n        }\n        if has_dsn\n            && !self\n                .server\n                .eval_if(&config.dsn, self, self.data.session_id)\n                .await\n                .unwrap_or(false)\n        {\n            trc::event!(Smtp(SmtpEvent::DsnDisabled), SpanId = self.data.session_id,);\n            self.data.mail_from = None;\n            return self\n                .write(b\"501 5.5.4 DSN extension has been disabled.\\r\\n\")\n                .await;\n        }\n\n        if self.is_allowed().await {\n            // Verify SPF\n            if self.params.spf_mail_from.verify() {\n                let time = Instant::now();\n                let mail_from = self.data.mail_from.as_ref().unwrap();\n                let spf_output = if !mail_from.address.is_empty() {\n                    self.server\n                        .core\n                        .smtp\n                        .resolvers\n                        .dns\n                        .check_host(self.server.inner.cache.build_auth_parameters(\n                            SpfParameters::new(\n                                self.data.remote_ip,\n                                &mail_from.domain,\n                                &self.data.helo_domain,\n                                &self.hostname,\n                                &mail_from.address_lcase,\n                            ),\n                        ))\n                        .await\n                } else {\n                    self.server\n                        .core\n                        .smtp\n                        .resolvers\n                        .dns\n                        .check_host(self.server.inner.cache.build_auth_parameters(\n                            SpfParameters::new(\n                                self.data.remote_ip,\n                                &self.data.helo_domain,\n                                &self.data.helo_domain,\n                                &self.hostname,\n                                &format!(\"postmaster@{}\", self.data.helo_domain),\n                            ),\n                        ))\n                        .await\n                };\n\n                trc::event!(\n                    Smtp(if matches!(spf_output.result(), SpfResult::Pass) {\n                        SmtpEvent::SpfFromPass\n                    } else {\n                        SmtpEvent::SpfFromFail\n                    }),\n                    SpanId = self.data.session_id,\n                    Domain = self.data.helo_domain.clone(),\n                    From = if !mail_from.address.is_empty() {\n                        mail_from.address.as_str()\n                    } else {\n                        \"<>\"\n                    }\n                    .to_string(),\n                    Result = trc::Error::from(&spf_output),\n                    Elapsed = time.elapsed(),\n                );\n\n                if self\n                    .handle_spf(&spf_output, self.params.spf_mail_from.is_strict())\n                    .await?\n                {\n                    self.data.spf_mail_from = spf_output.into();\n                } else {\n                    self.data.mail_from = None;\n                    return Ok(());\n                }\n            }\n\n            trc::event!(\n                Smtp(SmtpEvent::MailFrom),\n                SpanId = self.data.session_id,\n                From = self.data.mail_from.as_ref().unwrap().address_lcase.clone(),\n            );\n\n            self.eval_rcpt_params().await;\n            self.write(b\"250 2.1.0 OK\\r\\n\").await\n        } else {\n            trc::event!(\n                Smtp(SmtpEvent::RateLimitExceeded),\n                SpanId = self.data.session_id,\n                From = self.data.mail_from.as_ref().unwrap().address_lcase.clone(),\n            );\n\n            self.data.mail_from = None;\n            self.write(b\"452 4.4.5 Rate limit exceeded, try again later.\\r\\n\")\n                .await\n        }\n    }\n\n    pub async fn handle_spf(&mut self, spf_output: &SpfOutput, strict: bool) -> Result<bool, ()> {\n        let result = match spf_output.result() {\n            SpfResult::Pass => true,\n            SpfResult::TempError if strict => {\n                self.write(b\"451 4.7.24 Temporary SPF validation error.\\r\\n\")\n                    .await?;\n                false\n            }\n            result => {\n                if strict {\n                    self.write(\n                        format!(\"550 5.7.23 SPF validation failed, status: {result}.\\r\\n\")\n                            .as_bytes(),\n                    )\n                    .await?;\n                    false\n                } else {\n                    true\n                }\n            }\n        };\n\n        // Send report\n        if let (Some(recipient), Some(rate)) = (\n            spf_output.report_address(),\n            self.server\n                .eval_if::<Rate, _>(\n                    &self.server.core.smtp.report.spf.send,\n                    self,\n                    self.data.session_id,\n                )\n                .await,\n        ) {\n            // Do not send SPF auth failures to local domains, as they are likely relay attempts (which are blocked later on)\n            match self\n                .server\n                .core\n                .storage\n                .directory\n                .is_local_domain(recipient.domain_part())\n                .await\n            {\n                Ok(true) => return Ok(result),\n                Ok(false) => (),\n                Err(err) => {\n                    trc::error!(\n                        err.caused_by(trc::location!())\n                            .span_id(self.data.session_id)\n                            .details(\"Failed to lookup local domain\")\n                    );\n                }\n            }\n\n            self.send_spf_report(recipient, &rate, !result, spf_output)\n                .await;\n        }\n\n        Ok(result)\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/milter/client.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::config::smtp::session::Milter;\nuse rustls_pki_types::ServerName;\nuse tokio::{\n    io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt},\n    net::TcpStream,\n};\nuse tokio_rustls::{TlsConnector, client::TlsStream};\nuse trc::MilterEvent;\n\nuse super::{\n    protocol::{SMFIC_CONNECT, SMFIC_HELO, SMFIC_MAIL, SMFIC_RCPT},\n    receiver::{FrameResult, Receiver},\n    *,\n};\n\nconst MILTER_CHUNK_SIZE: usize = 65535;\n\nimpl MilterClient<TcpStream> {\n    pub async fn connect(config: &Milter, session_id: u64) -> Result<Self> {\n        tokio::time::timeout(config.timeout_command, async {\n            let mut last_err = Error::Disconnected;\n            for addr in &config.addrs {\n                match TcpStream::connect(addr).await {\n                    Ok(stream) => {\n                        return Ok(MilterClient {\n                            stream,\n                            timeout_cmd: config.timeout_command,\n                            timeout_data: config.timeout_data,\n                            buf: vec![0u8; 8192],\n                            bytes_read: 0,\n                            receiver: Receiver::with_max_frame_len(config.max_frame_len),\n                            options: 0,\n                            version: config.protocol_version,\n                            session_id,\n                            flags_actions: config.flags_actions.unwrap_or(\n                                SMFIF_ADDHDRS\n                                    | SMFIF_CHGBODY\n                                    | SMFIF_ADDRCPT\n                                    | SMFIF_DELRCPT\n                                    | SMFIF_CHGHDRS\n                                    | SMFIF_QUARANTINE\n                                    | SMFIF_CHGFROM\n                                    | SMFIF_ADDRCPT_PAR,\n                            ),\n                            flags_protocol: config.flags_protocol.unwrap_or(0x42),\n                            id: config.id.clone(),\n                        });\n                    }\n                    Err(err) => {\n                        last_err = Error::Io(err);\n                    }\n                }\n            }\n            Err(last_err)\n        })\n        .await\n        .map_err(|_| Error::Timeout)?\n    }\n\n    pub async fn into_tls(\n        self,\n        tls_connector: &TlsConnector,\n        tls_hostname: &str,\n    ) -> Result<MilterClient<TlsStream<TcpStream>>> {\n        tokio::time::timeout(self.timeout_cmd, async {\n            Ok(MilterClient {\n                stream: tls_connector\n                    .connect(\n                        ServerName::try_from(tls_hostname)\n                            .map_err(|_| Error::TLSInvalidName)?\n                            .to_owned(),\n                        self.stream,\n                    )\n                    .await?,\n                buf: self.buf,\n                timeout_cmd: self.timeout_cmd,\n                timeout_data: self.timeout_data,\n                receiver: self.receiver,\n                bytes_read: self.bytes_read,\n                options: self.options,\n                version: self.version,\n                session_id: self.session_id,\n                flags_actions: self.flags_actions,\n                flags_protocol: self.flags_protocol,\n                id: self.id,\n            })\n        })\n        .await\n        .map_err(|_| Error::Timeout)?\n    }\n}\n\nimpl<T: AsyncRead + AsyncWrite + Unpin> MilterClient<T> {\n    pub async fn init(&mut self) -> super::Result<Options> {\n        self.write(Command::OptionNegotiation(Options {\n            version: match self.version {\n                MilterVersion::V2 => 2,\n                MilterVersion::V6 => 6,\n            },\n            actions: self.flags_actions,\n            protocol: self.flags_protocol,\n        }))\n        .await?;\n        match self.read().await? {\n            Response::OptionNegotiation(options) => {\n                self.options = options.protocol;\n                Ok(options)\n            }\n            response => Err(Error::Unexpected(response)),\n        }\n    }\n\n    pub async fn connection(\n        &mut self,\n        hostname: impl AsRef<[u8]>,\n        remote_ip: IpAddr,\n        remote_port: u16,\n        macros: Macros<'_>,\n    ) -> super::Result<Action> {\n        if !self.has_option(SMFIP_NOCONNECT) {\n            self.write(Command::Macro {\n                macros: macros.with_cmd_code(SMFIC_CONNECT),\n            })\n            .await?;\n            self.write(Command::Connect {\n                hostname: hostname.as_ref(),\n                port: remote_port,\n                address: remote_ip,\n            })\n            .await?;\n            if !self.has_option(SMFIP_NR_CONN) {\n                return self.read().await?.into_action();\n            }\n        }\n\n        Ok(Action::Accept)\n    }\n\n    pub async fn helo(\n        &mut self,\n        hostname: impl AsRef<[u8]>,\n        macros: Macros<'_>,\n    ) -> super::Result<Action> {\n        if !self.has_option(SMFIP_NOHELO) {\n            self.write(Command::Macro {\n                macros: macros.with_cmd_code(SMFIC_HELO),\n            })\n            .await?;\n            self.write(Command::Helo {\n                hostname: hostname.as_ref(),\n            })\n            .await?;\n            if !self.has_option(SMFIP_NR_HELO) {\n                return self.read().await?.into_action();\n            }\n        }\n        Ok(Action::Accept)\n    }\n\n    pub async fn mail_from<A, V>(\n        &mut self,\n        addr: A,\n        params: Option<&[V]>,\n        macros: Macros<'_>,\n    ) -> super::Result<Action>\n    where\n        A: AsRef<[u8]>,\n        V: AsRef<[u8]>,\n    {\n        if !self.has_option(SMFIP_NOMAIL) {\n            self.write(Command::Macro {\n                macros: macros.with_cmd_code(SMFIC_MAIL),\n            })\n            .await?;\n            self.write(Command::MailFrom {\n                sender: addr.as_ref(),\n                args: params.map(|params| params.iter().map(|value| value.as_ref()).collect()),\n            })\n            .await?;\n            if !self.has_option(SMFIP_NR_MAIL) {\n                return self.read().await?.into_action();\n            }\n        }\n        Ok(Action::Accept)\n    }\n\n    pub async fn rcpt_to<A, V>(\n        &mut self,\n        addr: A,\n        params: Option<&[V]>,\n        macros: Macros<'_>,\n    ) -> super::Result<Action>\n    where\n        A: AsRef<[u8]>,\n        V: AsRef<[u8]>,\n    {\n        if !self.has_option(SMFIP_NORCPT) {\n            self.write(Command::Macro {\n                macros: macros.with_cmd_code(SMFIC_RCPT),\n            })\n            .await?;\n            self.write(Command::Rcpt {\n                recipient: addr.as_ref(),\n                args: params.map(|params| params.iter().map(|value| value.as_ref()).collect()),\n            })\n            .await?;\n            if !self.has_option(SMFIP_NR_RCPT) {\n                return self.read().await?.into_action();\n            }\n        }\n        Ok(Action::Accept)\n    }\n\n    pub async fn headers<I, H, V>(&mut self, headers: I) -> super::Result<Action>\n    where\n        I: Iterator<Item = (H, V)>,\n        H: AsRef<str>,\n        V: AsRef<str>,\n    {\n        if !self.has_option(SMFIP_NOHDRS) {\n            for (name, value) in headers {\n                self.write(Command::Header {\n                    name: name.as_ref().trim().as_bytes(),\n                    value: value.as_ref().trim().as_bytes(),\n                })\n                .await?;\n                if !self.has_option(SMFIP_NR_HDR) {\n                    match self.read().await? {\n                        Response::Action(Action::Accept | Action::Continue) => (),\n                        Response::Action(action) => return Ok(action),\n                        response => return Err(Error::Unexpected(response)),\n                    }\n                }\n            }\n\n            // Write EndOfHeaders\n            self.write(Command::EndOfHeader).await?;\n            if !self.has_option(SMFIP_NR_EOH) {\n                return self.read().await?.into_action();\n            }\n        }\n        Ok(Action::Accept)\n    }\n\n    pub async fn data(&mut self) -> super::Result<Action> {\n        if matches!(self.version, MilterVersion::V6) && !self.has_option(SMFIP_NODATA) {\n            self.write(Command::Data).await?;\n            if !self.has_option(SMFIP_NR_DATA) {\n                return self.read().await?.into_action();\n            }\n        }\n        Ok(Action::Accept)\n    }\n\n    pub async fn body(&mut self, body: &[u8]) -> super::Result<(Action, Vec<Modification>)> {\n        if !self.has_option(SMFIP_NOBODY) {\n            // Write body chunks\n            for value in body.chunks(MILTER_CHUNK_SIZE) {\n                self.write(Command::Body { value }).await?;\n                if !self.has_option(SMFIP_NR_BODY) {\n                    match self.read().await? {\n                        Response::Action(Action::Accept | Action::Continue)\n                        | Response::Progress => (),\n                        Response::Skip => break,\n                        Response::Action(reject) => {\n                            return Ok((reject, Vec::new()));\n                        }\n                        response => return Err(Error::Unexpected(response)),\n                    }\n                }\n            }\n\n            // Write EndOfBody\n            self.write(Command::EndOfBody).await?;\n\n            // Collect responses\n            let mut modifications = Vec::new();\n            loop {\n                match self.read().await? {\n                    Response::Action(action) => {\n                        return Ok((action, modifications));\n                    }\n                    Response::Modification(modification) => {\n                        modifications.push(modification);\n                    }\n                    Response::Progress => (),\n                    unexpected => {\n                        return Err(Error::Unexpected(unexpected));\n                    }\n                }\n            }\n        } else {\n            Ok((Action::Accept, vec![]))\n        }\n    }\n\n    pub async fn abort(&mut self) -> super::Result<()> {\n        self.write(Command::Abort).await\n    }\n\n    pub async fn quit(&mut self) -> super::Result<()> {\n        self.write(Command::Quit).await\n    }\n\n    async fn write(&mut self, action: Command<'_>) -> super::Result<()> {\n        trc::event!(\n            Milter(MilterEvent::Write),\n            SpanId = self.session_id,\n            Id = self.id.to_string(),\n            Contents = action.to_string(),\n        );\n\n        tokio::time::timeout(self.timeout_cmd, async {\n            self.stream.write_all(action.serialize().as_ref()).await?;\n            self.stream.flush().await.map_err(Error::Io)\n        })\n        .await\n        .map_err(|_| Error::Timeout)?\n    }\n\n    async fn read(&mut self) -> super::Result<Response> {\n        loop {\n            match self.receiver.read_frame(&self.buf[..self.bytes_read]) {\n                FrameResult::Frame(frame) => {\n                    if let Some(response) = Response::deserialize(&frame) {\n                        trc::event!(\n                            Milter(MilterEvent::Read),\n                            SpanId = self.session_id,\n                            Id = self.id.to_string(),\n                            Contents = response.to_string(),\n                        );\n\n                        return Ok(response);\n                    } else {\n                        return Err(Error::FrameInvalid(frame.into_owned()));\n                    }\n                }\n                FrameResult::Incomplete => {\n                    self.bytes_read = tokio::time::timeout(self.timeout_data, async {\n                        self.stream.read(&mut self.buf).await.map_err(Error::Io)\n                    })\n                    .await\n                    .map_err(|_| Error::Timeout)??;\n                    if self.bytes_read == 0 {\n                        return Err(Error::Disconnected);\n                    }\n                }\n                FrameResult::TooLarge(size) => return Err(Error::FrameTooLarge(size)),\n            }\n        }\n    }\n\n    #[inline(always)]\n    fn has_option(&self, opt: u32) -> bool {\n        self.options & opt == opt\n    }\n\n    pub fn with_version(mut self, version: MilterVersion) -> Self {\n        self.version = version;\n        self\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/milter/macros.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, net::IpAddr};\n\nuse super::{Macro, Macros};\n\npub trait IntoMacroValue<'x> {\n    fn into_macro_value(self) -> Cow<'x, [u8]>;\n}\n\nimpl<'x> Macros<'x> {\n    pub fn new() -> Self {\n        Macros::default()\n    }\n\n    pub fn with_cmd_code(mut self, cmd_code: u8) -> Self {\n        self.cmdcode = cmd_code;\n        self\n    }\n\n    pub fn with_macro(mut self, name: &'static [u8], value: impl IntoMacroValue<'x>) -> Self {\n        self.macros.push(Macro {\n            name,\n            value: value.into_macro_value(),\n        });\n        self\n    }\n\n    pub fn with_queue_id(self, queue_id: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"i\", queue_id)\n    }\n\n    pub fn with_local_hostname(self, my_hostname: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"j\", my_hostname)\n    }\n\n    pub fn with_validated_client_name(self, client_name: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"_\", client_name)\n    }\n\n    pub fn with_sasl_login_name(self, sasl_login_name: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{auth_authen}\", sasl_login_name)\n    }\n\n    pub fn with_sasl_sender(self, sasl_sender: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{auth_author}\", sasl_sender)\n    }\n\n    pub fn with_sasl_method(self, sasl_method: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{auth_type}\", sasl_method)\n    }\n\n    pub fn with_client_address(self, client_address: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{client_addr}\", client_address)\n    }\n\n    pub fn with_client_connections(self, client_connections: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{client_connections}\", client_connections)\n    }\n\n    pub fn with_client_name(self, client_name: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{client_name}\", client_name)\n    }\n\n    pub fn with_client_port(self, client_port: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{client_port}\", client_port)\n    }\n\n    pub fn with_client_ptr(self, client_ptr: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{client_ptr}\", client_ptr)\n    }\n\n    pub fn with_cert_issuer(self, cert_issuer: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{cert_issuer}\", cert_issuer)\n    }\n\n    pub fn with_cert_subject(self, cert_subject: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{cert_subject}\", cert_subject)\n    }\n\n    pub fn with_cipher_bits(self, cipher_bits: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{cipher_bits}\", cipher_bits)\n    }\n\n    pub fn with_cipher(self, cipher: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{cipher}\", cipher)\n    }\n\n    pub fn with_daemon_address(self, daemon_address: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{daemon_addr}\", daemon_address)\n    }\n\n    pub fn with_daemon_name(self, daemon_name: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{daemon_name}\", daemon_name)\n    }\n\n    pub fn with_daemon_port(self, daemon_port: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{daemon_port}\", daemon_port)\n    }\n\n    pub fn with_mail_address(self, mail_address: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{mail_addr}\", mail_address)\n    }\n\n    pub fn with_mail_host(self, mail_host_address: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{mail_host}\", mail_host_address)\n    }\n\n    pub fn with_mail_mailer(self, mail_mailer: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{mail_mailer}\", mail_mailer)\n    }\n\n    pub fn with_rcpt_address(self, rcpt_address: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{rcpt_addr}\", rcpt_address)\n    }\n\n    pub fn with_rcpt_host(self, rcpt_host: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{rcpt_host}\", rcpt_host)\n    }\n\n    pub fn with_rcpt_mailer(self, rcpt_mailer: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{rcpt_mailer}\", rcpt_mailer)\n    }\n\n    pub fn with_tls_version(self, tls_version: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{tls_version}\", tls_version)\n    }\n\n    pub fn with_version(self, version: impl IntoMacroValue<'x>) -> Self {\n        self.with_macro(b\"{v}\", version)\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for IpAddr {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Owned(self.to_string().into_bytes())\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for u16 {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Owned(self.to_string().into_bytes())\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for &'x [u8] {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Borrowed(self)\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for &'x str {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Borrowed(self.as_bytes())\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for &'x String {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Borrowed(self.as_bytes())\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for String {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Owned(self.into_bytes())\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for Vec<u8> {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Owned(self)\n    }\n}\n\nimpl<'x> IntoMacroValue<'x> for &'x Vec<u8> {\n    fn into_macro_value(self) -> Cow<'x, [u8]> {\n        Cow::Borrowed(self)\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/milter/message.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Action, Error, Macros, Modification};\nuse crate::{\n    core::{Session, SessionAddress, SessionData},\n    inbound::{FilterResponse, milter::MilterClient},\n};\nuse common::{\n    DAEMON_NAME,\n    config::smtp::session::{Milter, Stage},\n    listener::SessionStream,\n};\nuse mail_auth::AuthenticatedMessage;\nuse smtp_proto::{IntoString, request::parser::Rfc5321Parser};\nuse std::{borrow::Cow, time::Instant};\nuse tokio::io::{AsyncRead, AsyncWrite};\nuse trc::MilterEvent;\nuse utils::DomainPart;\n\nenum Rejection {\n    Action(Action),\n    Error(Error),\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn run_milters(\n        &self,\n        stage: Stage,\n        message: Option<&AuthenticatedMessage<'_>>,\n    ) -> Result<Vec<Modification>, FilterResponse> {\n        let milters = &self.server.core.smtp.session.milters;\n        if milters.is_empty() {\n            return Ok(Vec::new());\n        }\n\n        let mut modifications = Vec::new();\n        for milter in milters {\n            if !milter.run_on_stage.contains(&stage)\n                || !self\n                    .server\n                    .eval_if(&milter.enable, self, self.data.session_id)\n                    .await\n                    .unwrap_or(false)\n            {\n                continue;\n            }\n\n            let time = Instant::now();\n            match self.connect_and_run(milter, message).await {\n                Ok(new_modifications) => {\n                    trc::event!(\n                        Milter(MilterEvent::ActionAccept),\n                        SpanId = self.data.session_id,\n                        Id = milter.id.to_string(),\n                        Elapsed = time.elapsed(),\n                    );\n\n                    if !modifications.is_empty() {\n                        // The message body can only be replaced once, so we need to remove\n                        // any previous replacements.\n                        if new_modifications\n                            .iter()\n                            .any(|m| matches!(m, Modification::ReplaceBody { .. }))\n                        {\n                            modifications\n                                .retain(|m| !matches!(m, Modification::ReplaceBody { .. }));\n                        }\n                        modifications.extend(new_modifications);\n                    } else {\n                        modifications = new_modifications;\n                    }\n                }\n                Err(Rejection::Action(action)) => {\n                    trc::event!(\n                        Milter(match &action {\n                            Action::Discard => MilterEvent::ActionDiscard,\n                            Action::Reject => MilterEvent::ActionReject,\n                            Action::TempFail => MilterEvent::ActionTempFail,\n                            Action::ReplyCode { .. } => {\n                                MilterEvent::ActionReplyCode\n                            }\n                            Action::Shutdown => MilterEvent::ActionShutdown,\n                            Action::ConnectionFailure => MilterEvent::ActionConnectionFailure,\n                            Action::Accept | Action::Continue => unreachable!(),\n                        }),\n                        SpanId = self.data.session_id,\n                        Id = milter.id.to_string(),\n                        Elapsed = time.elapsed(),\n                    );\n\n                    return Err(match action {\n                        Action::Discard => FilterResponse::accept(),\n                        Action::Reject => FilterResponse::reject(),\n                        Action::TempFail => FilterResponse::temp_fail(),\n                        Action::ReplyCode { code, text } => {\n                            let mut response = Vec::with_capacity(text.len() + 6);\n                            response.extend_from_slice(code.as_slice());\n                            response.push(b' ');\n                            response.extend_from_slice(text.as_bytes());\n                            if !text.ends_with('\\n') {\n                                response.extend_from_slice(b\"\\r\\n\");\n                            }\n                            FilterResponse {\n                                message: response.into_string().into(),\n                                disconnect: false,\n                            }\n                        }\n                        Action::Shutdown => FilterResponse::shutdown(),\n                        Action::ConnectionFailure => FilterResponse::default().disconnect(),\n                        Action::Accept | Action::Continue => unreachable!(),\n                    });\n                }\n                Err(Rejection::Error(err)) => {\n                    let (code, details) = match err {\n                        Error::Io(details) => {\n                            (MilterEvent::IoError, trc::Value::from(details.to_string()))\n                        }\n                        Error::FrameTooLarge(size) => {\n                            (MilterEvent::FrameTooLarge, trc::Value::from(size))\n                        }\n                        Error::FrameInvalid(bytes) => {\n                            (MilterEvent::FrameInvalid, trc::Value::from(bytes))\n                        }\n                        Error::Unexpected(response) => (\n                            MilterEvent::UnexpectedResponse,\n                            trc::Value::from(response.to_string()),\n                        ),\n                        Error::Timeout => (MilterEvent::Timeout, trc::Value::None),\n                        Error::TLSInvalidName => (MilterEvent::TlsInvalidName, trc::Value::None),\n                        Error::Disconnected => (MilterEvent::Disconnected, trc::Value::None),\n                    };\n\n                    trc::event!(\n                        Milter(code),\n                        SpanId = self.data.session_id,\n                        Id = milter.id.to_string(),\n                        Details = details,\n                        Elapsed = time.elapsed(),\n                    );\n\n                    if milter.tempfail_on_error {\n                        return Err(FilterResponse::server_failure());\n                    }\n                }\n            }\n        }\n\n        Ok(modifications)\n    }\n\n    async fn connect_and_run(\n        &self,\n        milter: &Milter,\n        message: Option<&AuthenticatedMessage<'_>>,\n    ) -> Result<Vec<Modification>, Rejection> {\n        // Build client\n        let client = MilterClient::connect(milter, self.data.session_id).await?;\n        if !milter.tls {\n            self.run(client, message).await\n        } else {\n            self.run(\n                client\n                    .into_tls(\n                        if !milter.tls_allow_invalid_certs {\n                            &self.server.inner.data.smtp_connectors.pki_verify\n                        } else {\n                            &self.server.inner.data.smtp_connectors.dummy_verify\n                        },\n                        &milter.hostname,\n                    )\n                    .await?,\n                message,\n            )\n            .await\n        }\n    }\n\n    async fn run<S: AsyncRead + AsyncWrite + Unpin>(\n        &self,\n        mut client: MilterClient<S>,\n        message: Option<&AuthenticatedMessage<'_>>,\n    ) -> Result<Vec<Modification>, Rejection> {\n        // Option negotiation\n        client.init().await?;\n\n        // Connect stage\n        let client_ptr = self\n            .data\n            .iprev\n            .as_ref()\n            .and_then(|ip_rev| ip_rev.ptr.as_ref())\n            .and_then(|ptrs| ptrs.first())\n            .map(|s| s.as_str());\n        client\n            .connection(\n                client_ptr.unwrap_or(self.data.helo_domain.as_str()),\n                self.data.remote_ip,\n                self.data.remote_port,\n                Macros::new()\n                    .with_daemon_name(DAEMON_NAME)\n                    .with_local_hostname(&self.hostname)\n                    .with_client_address(self.data.remote_ip)\n                    .with_client_port(self.data.remote_port)\n                    .with_client_ptr(client_ptr.unwrap_or(\"unknown\")),\n            )\n            .await?\n            .assert_continue()?;\n\n        // EHLO/HELO\n        let (tls_version, tls_cipher) = self.stream.tls_version_and_cipher();\n        client\n            .helo(\n                &self.data.helo_domain,\n                Macros::new()\n                    .with_cipher(tls_cipher.as_ref())\n                    .with_tls_version(tls_version.as_ref()),\n            )\n            .await?\n            .assert_continue()?;\n\n        // Mail from\n        if let Some(mail_from) = &self.data.mail_from {\n            let addr = &mail_from.address_lcase;\n            client\n                .mail_from(\n                    &format!(\"<{addr}>\"),\n                    None::<&[&str]>,\n                    if let Some(name) = self.authenticated_as() {\n                        Macros::new()\n                            .with_mail_address(addr)\n                            .with_sasl_login_name(name)\n                    } else {\n                        Macros::new().with_mail_address(addr)\n                    },\n                )\n                .await?\n                .assert_continue()?;\n\n            // Rcpt to\n            for rcpt in &self.data.rcpt_to {\n                client\n                    .rcpt_to(\n                        &format!(\"<{}>\", rcpt.address_lcase),\n                        None::<&[&str]>,\n                        Macros::new().with_rcpt_address(&rcpt.address_lcase),\n                    )\n                    .await?\n                    .assert_continue()?;\n            }\n        }\n\n        if let Some(message) = message {\n            // Data\n            client.data().await?.assert_continue()?;\n\n            // Headers\n            client\n                .headers(message.raw_parsed_headers().iter().map(|(k, v)| {\n                    (\n                        std::str::from_utf8(k).unwrap_or_default(),\n                        std::str::from_utf8(v).unwrap_or_default(),\n                    )\n                }))\n                .await?\n                .assert_continue()?;\n\n            // Message body\n            let (action, modifications) = client.body(message.raw_message()).await?;\n            action.assert_continue()?;\n\n            // Quit\n            let _ = client.quit().await;\n\n            // Return modifications\n            Ok(modifications)\n        } else {\n            // Quit\n            let _ = client.quit().await;\n\n            Ok(Vec::new())\n        }\n    }\n}\n\nimpl SessionData {\n    pub fn apply_milter_modifications(\n        &mut self,\n        modifications: Vec<Modification>,\n        message: &AuthenticatedMessage<'_>,\n    ) -> Option<Vec<u8>> {\n        let mut body = Vec::new();\n        let mut header_changes = Vec::new();\n        let mut needs_rewrite = false;\n\n        for modification in modifications {\n            match modification {\n                Modification::ChangeFrom { sender, mut args } => {\n                    // Change sender\n                    let sender = strip_brackets(&sender);\n                    let address_lcase = sender.to_lowercase();\n                    let mut mail_from = SessionAddress {\n                        domain: address_lcase.domain_part().into(),\n                        address_lcase,\n                        address: sender,\n                        flags: 0,\n                        dsn_info: None,\n                    };\n                    if !args.is_empty() {\n                        args.push('\\n');\n                        match Rfc5321Parser::new(&mut args.as_bytes().iter())\n                            .mail_from_parameters(Cow::Borrowed(\"\"))\n                        {\n                            Ok(addr) => {\n                                mail_from.flags = addr.flags;\n                                mail_from.dsn_info = addr.env_id.map(|e| e.into_owned());\n                            }\n                            Err(err) => {\n                                trc::event!(\n                                    Milter(MilterEvent::ParseError),\n                                    SpanId = self.session_id,\n                                    Details = \"Failed to parse milter mailFrom parameters\",\n                                    Reason = err.to_string(),\n                                );\n                            }\n                        }\n                    }\n                    self.mail_from = Some(mail_from);\n                }\n                Modification::AddRcpt {\n                    recipient,\n                    mut args,\n                } => {\n                    // Add recipient\n                    let recipient = strip_brackets(&recipient);\n                    if recipient.contains('@') {\n                        let address_lcase = recipient.to_lowercase();\n                        let mut rcpt = SessionAddress {\n                            domain: address_lcase.domain_part().into(),\n                            address_lcase,\n                            address: recipient,\n                            flags: 0,\n                            dsn_info: None,\n                        };\n                        if !args.is_empty() {\n                            args.push('\\n');\n                            match Rfc5321Parser::new(&mut args.as_bytes().iter())\n                                .rcpt_to_parameters(Cow::Borrowed(\"\"))\n                            {\n                                Ok(addr) => {\n                                    rcpt.flags = addr.flags;\n                                    rcpt.dsn_info = addr.orcpt.map(|e| e.into_owned());\n                                }\n                                Err(err) => {\n                                    trc::event!(\n                                        Milter(MilterEvent::ParseError),\n                                        SpanId = self.session_id,\n                                        Details = \"Failed to parse milter rcptTo parameters\",\n                                        Reason = err.to_string(),\n                                    );\n                                }\n                            }\n                        }\n\n                        if !self.rcpt_to.contains(&rcpt) {\n                            self.rcpt_to.push(rcpt);\n                        }\n                    }\n                }\n                Modification::DeleteRcpt { recipient } => {\n                    let recipient = strip_brackets(&recipient);\n                    self.rcpt_to.retain(|r| r.address_lcase != recipient);\n                }\n                Modification::ReplaceBody { value } => {\n                    body.extend(value);\n                }\n                Modification::AddHeader { name, value } => {\n                    header_changes.push((0, name, value, false));\n                }\n                Modification::InsertHeader { index, name, value } => {\n                    header_changes.push((index, name, value, false));\n                    needs_rewrite = true;\n                }\n                Modification::ChangeHeader { index, name, value } => {\n                    if value.is_empty()\n                        || message\n                            .raw_parsed_headers()\n                            .iter()\n                            .any(|(n, _)| n.eq_ignore_ascii_case(name.as_bytes()))\n                    {\n                        header_changes.push((index, name, value, true));\n                        needs_rewrite = true;\n                    } else {\n                        header_changes.push((0, name, value, false));\n                    }\n                }\n                Modification::Quarantine { reason } => {\n                    header_changes.push((0, \"X-Quarantine\".into(), reason, false));\n                }\n            }\n        }\n\n        // If there are no header changes return\n        if header_changes.is_empty() {\n            return if !body.is_empty() {\n                let mut new_message = Vec::with_capacity(body.len() + message.raw_headers().len());\n                new_message.extend_from_slice(message.raw_headers());\n                new_message.extend(body);\n                Some(new_message)\n            } else {\n                None\n            };\n        }\n\n        let new_body = if !body.is_empty() {\n            &body[..]\n        } else {\n            message.raw_body()\n        };\n\n        if needs_rewrite {\n            let mut headers = message\n                .raw_parsed_headers()\n                .iter()\n                .map(|(h, v)| (Cow::from(*h), Cow::from(*v)))\n                .collect::<Vec<_>>();\n\n            // Perform changes\n            for (index, header_name, header_value, is_change) in header_changes {\n                if is_change {\n                    let mut header_count = 0;\n                    for (pos, (name, value)) in headers.iter_mut().enumerate() {\n                        if name.eq_ignore_ascii_case(header_name.as_bytes()) {\n                            header_count += 1;\n                            if header_count == index {\n                                if !header_value.is_empty() {\n                                    *value = Cow::from(header_value.as_bytes().to_vec());\n                                } else {\n                                    headers.remove(pos);\n                                }\n                                break;\n                            }\n                        }\n                    }\n                } else {\n                    let mut header_pos = 0;\n                    if index > 0 {\n                        let mut header_count = 0;\n                        for (pos, (name, _)) in headers.iter().enumerate() {\n                            if name.eq_ignore_ascii_case(header_name.as_bytes()) {\n                                header_pos = pos;\n                                header_count += 1;\n                                if header_count == index {\n                                    break;\n                                }\n                            }\n                        }\n                    }\n\n                    headers.insert(\n                        header_pos,\n                        (\n                            Cow::from(header_name.as_bytes().to_vec()),\n                            Cow::from(header_value.as_bytes().to_vec()),\n                        ),\n                    );\n                }\n            }\n\n            // Write new headers\n            let mut new_message = Vec::with_capacity(\n                new_body.len()\n                    + message.raw_headers().len()\n                    + headers\n                        .iter()\n                        .map(|(h, v)| h.len() + v.len() + 4)\n                        .sum::<usize>(),\n            );\n            for (header, value) in headers {\n                new_message.extend_from_slice(header.as_ref());\n                if value.first().is_some_and(|c| c.is_ascii_whitespace()) {\n                    new_message.extend_from_slice(b\":\");\n                } else {\n                    new_message.extend_from_slice(b\": \");\n                }\n                new_message.extend_from_slice(value.as_ref());\n                if value.last().is_none_or(|c| *c != b'\\n') {\n                    new_message.extend_from_slice(b\"\\r\\n\");\n                }\n            }\n            new_message.extend_from_slice(b\"\\r\\n\");\n            new_message.extend(new_body);\n            Some(new_message)\n        } else {\n            let mut new_message = Vec::with_capacity(\n                new_body.len()\n                    + message.raw_headers().len()\n                    + header_changes\n                        .iter()\n                        .map(|(_, h, v, _)| h.len() + v.len() + 4)\n                        .sum::<usize>(),\n            );\n            for (_, header, value, _) in header_changes {\n                new_message.extend_from_slice(header.as_bytes());\n                new_message.extend_from_slice(b\": \");\n                new_message.extend_from_slice(value.as_bytes());\n                if !value.ends_with('\\n') {\n                    new_message.extend_from_slice(b\"\\r\\n\");\n                }\n            }\n            new_message.extend_from_slice(message.raw_headers());\n            new_message.extend(new_body);\n            Some(new_message)\n        }\n    }\n}\n\nimpl Action {\n    fn assert_continue(self) -> Result<(), Rejection> {\n        match self {\n            Action::Continue | Action::Accept => Ok(()),\n            action => Err(Rejection::Action(action)),\n        }\n    }\n}\n\nimpl From<Error> for Rejection {\n    fn from(err: Error) -> Self {\n        Rejection::Error(err)\n    }\n}\n\nfn strip_brackets(addr: &str) -> String {\n    let addr = addr.trim();\n    if let Some(addr) = addr.strip_prefix('<') {\n        if let Some((addr, _)) = addr.rsplit_once('>') {\n            addr.trim().into()\n        } else {\n            addr.trim().into()\n        }\n    } else {\n        addr.into()\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/milter/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, fmt::Display, net::IpAddr, sync::Arc, time::Duration};\n\nuse common::config::smtp::session::MilterVersion;\n\nuse serde::{Deserialize, Serialize};\nuse tokio::io::{AsyncRead, AsyncWrite};\n\nuse self::receiver::Receiver;\n\npub mod client;\npub mod macros;\npub mod message;\npub mod protocol;\npub mod receiver;\n\npub struct MilterClient<T: AsyncRead + AsyncWrite> {\n    stream: T,\n    buf: Vec<u8>,\n    bytes_read: usize,\n    timeout_cmd: Duration,\n    timeout_data: Duration,\n    receiver: Receiver,\n    version: MilterVersion,\n    options: u32,\n    flags_actions: u32,\n    flags_protocol: u32,\n    id: Arc<String>,\n    session_id: u64,\n}\n\n#[derive(Debug)]\npub enum Error {\n    Io(std::io::Error),\n    FrameTooLarge(usize),\n    FrameInvalid(Vec<u8>),\n    Unexpected(Response),\n    Timeout,\n    TLSInvalidName,\n    Disconnected,\n}\n\nimpl From<std::io::Error> for Error {\n    fn from(err: std::io::Error) -> Self {\n        Error::Io(err)\n    }\n}\n\npub enum Command<'x> {\n    Abort,\n    Body {\n        value: &'x [u8],\n    },\n    EndOfBody,\n    Data,\n    Connect {\n        hostname: &'x [u8],\n        port: u16,\n        address: IpAddr,\n    },\n    Macro {\n        macros: Macros<'x>,\n    },\n    Header {\n        name: &'x [u8],\n        value: &'x [u8],\n    },\n    EndOfHeader,\n    Helo {\n        hostname: &'x [u8],\n    },\n    MailFrom {\n        sender: &'x [u8],\n        args: Option<Vec<&'x [u8]>>,\n    },\n    Rcpt {\n        recipient: &'x [u8],\n        args: Option<Vec<&'x [u8]>>,\n    },\n    OptionNegotiation(Options),\n    Quit,\n    QuitNewConnection,\n}\n\n#[derive(Debug)]\npub enum Response {\n    Action(Action),\n    Modification(Modification),\n    Progress,\n    Skip,\n    SetSymbols,\n    OptionNegotiation(Options),\n}\n\n#[derive(Debug)]\npub enum Action {\n    Accept,\n    Continue,\n    Discard,\n    Reject,\n    TempFail,\n    ReplyCode { code: [u8; 3], text: String },\n    Shutdown,\n    ConnectionFailure,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub enum Modification {\n    ChangeFrom {\n        sender: String,\n        args: String,\n    },\n    AddRcpt {\n        recipient: String,\n        args: String,\n    },\n    DeleteRcpt {\n        recipient: String,\n    },\n    ReplaceBody {\n        value: Vec<u8>,\n    },\n    AddHeader {\n        name: String,\n        value: String,\n    },\n    InsertHeader {\n        index: u32,\n        name: String,\n        value: String,\n    },\n    ChangeHeader {\n        index: u32,\n        name: String,\n        value: String,\n    },\n    Quarantine {\n        reason: String,\n    },\n}\n\n#[derive(Debug)]\npub struct Options {\n    pub version: u32,\n    pub actions: u32,\n    pub protocol: u32,\n}\n\n#[derive(Default)]\npub struct Macros<'x> {\n    cmdcode: u8,\n    macros: Vec<Macro<'x>>,\n}\n\npub struct Macro<'x> {\n    name: &'x [u8],\n    value: Cow<'x, [u8]>,\n}\n\npub const SMFIF_NONE: u32 = 0x00000000; /* no flags */\npub const SMFIF_ADDHDRS: u32 = 0x00000001; /* filter may add headers */\npub const SMFIF_CHGBODY: u32 = 0x00000002; /* filter may replace body */\npub const SMFIF_MODBODY: u32 = SMFIF_CHGBODY; /* backwards compatible */\npub const SMFIF_ADDRCPT: u32 = 0x00000004; /* filter may add recipients */\npub const SMFIF_DELRCPT: u32 = 0x00000008; /* filter may delete recipients */\npub const SMFIF_CHGHDRS: u32 = 0x00000010; /* filter may change/delete headers */\npub const SMFIF_QUARANTINE: u32 = 0x00000020; /* filter may quarantine envelope */\npub const SMFIF_CHGFROM: u32 = 0x00000040; /* filter may change \"from\" (envelope sender) */\npub const SMFIF_ADDRCPT_PAR: u32 = 0x00000080; /* add recipients incl. args */\npub const SMFIF_SETSYMLIST: u32 = 0x00000100; /* filter can send set of symbols (macros) that it wants */\n\npub const SMFIP_NOCONNECT: u32 = 0x00000001; /* MTA should not send connect info */\npub const SMFIP_NOHELO: u32 = 0x00000002; /* MTA should not send HELO info */\npub const SMFIP_NOMAIL: u32 = 0x00000004; /* MTA should not send MAIL info */\npub const SMFIP_NORCPT: u32 = 0x00000008; /* MTA should not send RCPT info */\npub const SMFIP_NOBODY: u32 = 0x00000010; /* MTA should not send body */\npub const SMFIP_NOHDRS: u32 = 0x00000020; /* MTA should not send headers */\npub const SMFIP_NOEOH: u32 = 0x00000040; /* MTA should not send EOH */\npub const SMFIP_NR_HDR: u32 = 0x00000080; /* No reply for headers */\npub const SMFIP_NOHREPL: u32 = SMFIP_NR_HDR; /* No reply for headers */\npub const SMFIP_NOUNKNOWN: u32 = 0x00000100; /* MTA should not send unknown commands */\npub const SMFIP_NODATA: u32 = 0x00000200; /* MTA should not send DATA */\npub const SMFIP_SKIP: u32 = 0x00000400; /* MTA understands SMFIS_SKIP */\npub const SMFIP_RCPT_REJ: u32 = 0x00000800; /* MTA should also send rejected RCPTs */\npub const SMFIP_NR_CONN: u32 = 0x00001000; /* No reply for connect */\npub const SMFIP_NR_HELO: u32 = 0x00002000; /* No reply for HELO */\npub const SMFIP_NR_MAIL: u32 = 0x00004000; /* No reply for MAIL */\npub const SMFIP_NR_RCPT: u32 = 0x00008000; /* No reply for RCPT */\npub const SMFIP_NR_DATA: u32 = 0x00010000; /* No reply for DATA */\npub const SMFIP_NR_UNKN: u32 = 0x00020000; /* No reply for UNKN */\npub const SMFIP_NR_EOH: u32 = 0x00040000; /* No reply for eoh */\npub const SMFIP_NR_BODY: u32 = 0x00080000; /* No reply for body chunk */\npub const SMFIP_HDR_LEADSPC: u32 = 0x00100000; /* header value leading space */\npub const SMFIP_MDS_256K: u32 = 0x10000000; /* MILTER_MAX_DATA_SIZE=256K */\npub const SMFIP_MDS_1M: u32 = 0x20000000; /* MILTER_MAX_DATA_SIZE=1M */\n\npub type Result<T> = std::result::Result<T, Error>;\n\nimpl Display for Command<'_> {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Command::Abort => write!(f, \"ABORT\"),\n            Command::Body { value } => write!(f, \"BODY [{} bytes]\", value.len()),\n            Command::EndOfBody => write!(f, \"EOB\"),\n            Command::Connect {\n                hostname,\n                port,\n                address,\n            } => write!(\n                f,\n                \"CONNECT (host: {:?}, port: {}, address: {})\",\n                std::str::from_utf8(hostname).unwrap_or_default(),\n                port,\n                address\n            ),\n            Command::Macro { macros } => {\n                write!(f, \"MACRO (code: {}, params: \", macros.cmdcode)?;\n                for macro_ in &macros.macros {\n                    write!(\n                        f,\n                        \"({:?}, {:?})\",\n                        std::str::from_utf8(macro_.name).unwrap_or_default(),\n                        std::str::from_utf8(macro_.value.as_ref()).unwrap_or_default()\n                    )?;\n                }\n                write!(f, \")\")\n            }\n            Command::Header { name, value } => {\n                write!(\n                    f,\n                    \"HEADER ({}: {:?})\",\n                    std::str::from_utf8(name).unwrap_or_default(),\n                    std::str::from_utf8(value).unwrap_or_default()\n                )\n            }\n            Command::EndOfHeader => write!(f, \"EOH\"),\n            Command::Helo { hostname } => write!(\n                f,\n                \"HELO {:?}\",\n                std::str::from_utf8(hostname).unwrap_or_default()\n            ),\n            Command::MailFrom { sender, args } => {\n                write!(\n                    f,\n                    \"MAIL (from: {}, params: \",\n                    std::str::from_utf8(sender).unwrap_or_default()\n                )?;\n                if let Some(args) = args {\n                    for arg in args {\n                        write!(f, \" {}\", std::str::from_utf8(arg).unwrap_or_default())?;\n                    }\n                }\n                write!(f, \")\")\n            }\n            Command::Rcpt { recipient, args } => {\n                write!(\n                    f,\n                    \"RCPT (to: {}, params: \",\n                    std::str::from_utf8(recipient).unwrap_or_default()\n                )?;\n                if let Some(args) = args {\n                    for arg in args {\n                        write!(f, \" {}\", std::str::from_utf8(arg).unwrap_or_default())?;\n                    }\n                }\n                write!(f, \")\")\n            }\n            Command::OptionNegotiation(opt) => write!(f, \"OPTNEG ({})\", opt),\n            Command::Quit => write!(f, \"QUIT\"),\n            Command::Data => write!(f, \"DATA\"),\n            Command::QuitNewConnection => write!(f, \"QUIT_NC\"),\n        }\n    }\n}\n\nimpl Display for Response {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Response::Action(action) => write!(f, \"ACTION ({})\", action),\n            Response::Modification(modification) => write!(f, \"MODIFICATION ({})\", modification),\n            Response::Progress => write!(f, \"PROGRESS\"),\n            Response::OptionNegotiation(opt) => write!(f, \"OPTNEG ({})\", opt),\n            Response::Skip => write!(f, \"SKIP\"),\n            Response::SetSymbols => write!(f, \"SET_SYMBOLS\"),\n        }\n    }\n}\n\nimpl Display for Action {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Action::Accept => write!(f, \"ACCEPT\"),\n            Action::Continue => write!(f, \"CONTINUE\"),\n            Action::Discard => write!(f, \"DISCARD\"),\n            Action::Reject => write!(f, \"REJECT\"),\n            Action::TempFail => write!(f, \"TEMPFAIL\"),\n            Action::ReplyCode { code, text } => {\n                write!(f, \"REPLYCODE (code: {:?}, text: {})\", code, text)\n            }\n            Action::Shutdown => write!(f, \"SHUTDOWN\"),\n            Action::ConnectionFailure => write!(f, \"CONN_FAIL\"),\n        }\n    }\n}\n\nimpl Display for Modification {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Modification::AddRcpt { recipient, args } => {\n                write!(f, \"ADD_RCPT (recipient: {}, args: {})\", recipient, args)\n            }\n            Modification::DeleteRcpt { recipient } => {\n                write!(f, \"DEL_RCPT (recipient: {})\", recipient)\n            }\n            Modification::ReplaceBody { value } => {\n                write!(f, \"REPLACE_BODY ({} bytes)\", value.len())\n            }\n            Modification::AddHeader { name, value } => {\n                write!(f, \"ADD_HEADER ({}: {})\", name, value)\n            }\n            Modification::ChangeHeader { index, name, value } => {\n                write!(f, \"CHANGE_HEADER (index: {}, {}: {})\", index, name, value)\n            }\n            Modification::Quarantine { reason } => write!(f, \"QUARANTINE ({})\", reason),\n            Modification::ChangeFrom { sender, args } => {\n                write!(f, \"CHANGE_FROM (<{}> {})\", sender, args)\n            }\n            Modification::InsertHeader { index, name, value } => {\n                write!(f, \"INSERT_HEADER (index: {}, {}: {})\", index, name, value)\n            }\n        }\n    }\n}\n\nimpl Display for Options {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"version: {}, actions: [\", self.version,)?;\n\n        if self.actions & SMFIF_ADDHDRS != 0 {\n            write!(f, \"ADDHDRS \")?;\n        }\n        if self.actions & SMFIF_CHGBODY != 0 {\n            write!(f, \"CHGBODY \")?;\n        }\n        if self.actions & SMFIF_CHGHDRS != 0 {\n            write!(f, \"CHGHDRS \")?;\n        }\n        if self.actions & SMFIF_ADDRCPT != 0 {\n            write!(f, \"ADDRCPT \")?;\n        }\n        if self.actions & SMFIF_DELRCPT != 0 {\n            write!(f, \"DELRCPT \")?;\n        }\n        if self.actions & SMFIF_CHGFROM != 0 {\n            write!(f, \"CHGFROM \")?;\n        }\n        if self.actions & SMFIF_QUARANTINE != 0 {\n            write!(f, \"QUARANTINE \")?;\n        }\n        if self.actions & SMFIF_CHGFROM != 0 {\n            write!(f, \"CHGFROM \")?;\n        }\n        if self.actions & SMFIF_ADDRCPT_PAR != 0 {\n            write!(f, \"ADDRCPT_PAR \")?;\n        }\n        if self.actions & SMFIF_SETSYMLIST != 0 {\n            write!(f, \"SETSYMLIST \")?;\n        }\n        write!(f, \"], options: [\",)?;\n\n        if self.protocol & SMFIP_NOCONNECT != 0 {\n            write!(f, \"NOCONNECT \")?;\n        }\n\n        if self.protocol & SMFIP_NOHELO != 0 {\n            write!(f, \"NOHELO \")?;\n        }\n\n        if self.protocol & SMFIP_NOMAIL != 0 {\n            write!(f, \"NOMAIL \")?;\n        }\n\n        if self.protocol & SMFIP_NORCPT != 0 {\n            write!(f, \"NORCPT \")?;\n        }\n\n        if self.protocol & SMFIP_NOBODY != 0 {\n            write!(f, \"NOBODY \")?;\n        }\n\n        if self.protocol & SMFIP_NOHDRS != 0 {\n            write!(f, \"NOHDRS \")?;\n        }\n\n        if self.protocol & SMFIP_NOEOH != 0 {\n            write!(f, \"NOEOH \")?;\n        }\n\n        if self.protocol & SMFIP_NR_HDR != 0 {\n            write!(f, \"NR_HDR \")?;\n        }\n\n        if self.protocol & SMFIP_NOUNKNOWN != 0 {\n            write!(f, \"NOUNKNOWN \")?;\n        }\n\n        if self.protocol & SMFIP_NODATA != 0 {\n            write!(f, \"NODATA \")?;\n        }\n\n        if self.protocol & SMFIP_SKIP != 0 {\n            write!(f, \"SKIP \")?;\n        }\n\n        if self.protocol & SMFIP_RCPT_REJ != 0 {\n            write!(f, \"RCPT_REJ \")?;\n        }\n\n        if self.protocol & SMFIP_NR_CONN != 0 {\n            write!(f, \"NR_CONN \")?;\n        }\n\n        if self.protocol & SMFIP_NR_HELO != 0 {\n            write!(f, \"NR_HELO \")?;\n        }\n\n        if self.protocol & SMFIP_NR_MAIL != 0 {\n            write!(f, \"NR_MAIL \")?;\n        }\n\n        if self.protocol & SMFIP_NR_RCPT != 0 {\n            write!(f, \"NR_RCPT \")?;\n        }\n\n        if self.protocol & SMFIP_NR_DATA != 0 {\n            write!(f, \"NR_DATA \")?;\n        }\n\n        if self.protocol & SMFIP_NR_UNKN != 0 {\n            write!(f, \"NR_UNKN \")?;\n        }\n\n        if self.protocol & SMFIP_NR_EOH != 0 {\n            write!(f, \"NR_EOH \")?;\n        }\n\n        if self.protocol & SMFIP_NR_BODY != 0 {\n            write!(f, \"NR_BODY \")?;\n        }\n\n        if self.protocol & SMFIP_HDR_LEADSPC != 0 {\n            write!(f, \"HDR_LEADSPC \")?;\n        }\n\n        if self.protocol & SMFIP_MDS_256K != 0 {\n            write!(f, \"MDS_256K \")?;\n        }\n\n        if self.protocol & SMFIP_MDS_1M != 0 {\n            write!(f, \"MDS_1M \")?;\n        }\n\n        write!(f, \"]\")\n    }\n}\n\nimpl Display for Error {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Error::Io(err) => write!(f, \"IO error: {}\", err),\n            Error::FrameTooLarge(size) => {\n                write!(f, \"Milter response of {} bytes is too large.\", size)\n            }\n            Error::FrameInvalid(frame) => write!(\n                f,\n                \"Invalid milter response: {:?}\",\n                frame.get(0..100).unwrap_or(frame.as_ref())\n            ),\n            Error::Unexpected(response) => write!(f, \"Unexpected response: {}\", response),\n            Error::Timeout => write!(f, \"Connection timed out\"),\n            Error::TLSInvalidName => write!(f, \"Invalid TLS name\"),\n            Error::Disconnected => write!(f, \"Disconnected unexpectedly\"),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/milter/protocol.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse crate::inbound::milter::Action;\n\nuse super::{Command, Error, Modification, Options, Response};\n\npub const SMFIR_ADDRCPT: u8 = b'+'; /* add recipient */\npub const SMFIR_DELRCPT: u8 = b'-'; /* remove recipient */\npub const SMFIR_ADDRCPT_PAR: u8 = b'2'; /* add recipient (incl. ESMTP args) */\npub const SMFIR_SHUTDOWN: u8 = b'4'; /* 421: shutdown (internal to MTA) */\npub const SMFIR_ACCEPT: u8 = b'a'; /* accept */\npub const SMFIR_REPLBODY: u8 = b'b'; /* replace body (chunk) */\npub const SMFIR_CONTINUE: u8 = b'c'; /* continue */\npub const SMFIR_DISCARD: u8 = b'd'; /* discard */\npub const SMFIR_CHGFROM: u8 = b'e'; /* change envelope sender (from) */\npub const SMFIR_CONN_FAIL: u8 = b'f'; /* cause a connection failure */\npub const SMFIR_ADDHEADER: u8 = b'h'; /* add header */\npub const SMFIR_INSHEADER: u8 = b'i'; /* insert header */\npub const SMFIR_SETSYMLIST: u8 = b'l'; /* set list of symbols (macros) */\npub const SMFIR_CHGHEADER: u8 = b'm'; /* change header */\npub const SMFIR_PROGRESS: u8 = b'p'; /* progress */\npub const SMFIR_QUARANTINE: u8 = b'q'; /* quarantine */\npub const SMFIR_REJECT: u8 = b'r'; /* reject */\npub const SMFIR_SKIP: u8 = b's'; /* skip */\npub const SMFIR_TEMPFAIL: u8 = b't'; /* tempfail */\npub const SMFIR_REPLYCODE: u8 = b'y'; /* reply code etc */\n\npub const SMFIC_ABORT: u8 = b'A'; /* Abort */\npub const SMFIC_BODY: u8 = b'B'; /* Body chunk */\npub const SMFIC_CONNECT: u8 = b'C'; /* Connection information */\npub const SMFIC_MACRO: u8 = b'D'; /* Define macro */\npub const SMFIC_BODYEOB: u8 = b'E'; /* final body chunk (End) */\npub const SMFIC_HELO: u8 = b'H'; /* HELO/EHLO */\npub const SMFIC_QUIT_NC: u8 = b'K'; /* QUIT but new connection follows */\npub const SMFIC_HEADER: u8 = b'L'; /* Header */\npub const SMFIC_MAIL: u8 = b'M'; /* MAIL from */\npub const SMFIC_EOH: u8 = b'N'; /* EOH */\npub const SMFIC_OPTNEG: u8 = b'O'; /* Option negotiation */\npub const SMFIC_QUIT: u8 = b'Q'; /* QUIT */\npub const SMFIC_RCPT: u8 = b'R'; /* RCPT to */\npub const SMFIC_DATA: u8 = b'T'; /* DATA */\npub const SMFIC_UNKNOWN: u8 = b'U'; /* Any unknown command */\n\nimpl Command<'_> {\n    fn build(command: u8, len: u32) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(len as usize + 1 + std::mem::size_of::<u32>());\n        buf.extend_from_slice((len + 1).to_be_bytes().as_ref());\n        buf.push(command);\n        buf\n    }\n\n    pub fn serialize(self) -> Vec<u8> {\n        match self {\n            Command::Abort => Command::build(SMFIC_ABORT, 0),\n            Command::Body { value } => {\n                let mut buf = Command::build(SMFIC_BODY, value.len() as u32);\n                buf.extend(value);\n                buf\n            }\n            Command::EndOfBody => Command::build(SMFIC_BODYEOB, 0),\n            Command::Connect {\n                hostname,\n                port,\n                address,\n            } => {\n                /*\n\n                char\thostname[]\tHostname, NUL terminated\n                char\tfamily\t\tProtocol family (see below)\n                uint16\tport\t\tPort number (SMFIA_INET or SMFIA_INET6 only)\n                char\taddress[]\tIP address (ASCII) or unix socket path, NUL terminated\n\n                */\n\n                let (address, family) = match address {\n                    IpAddr::V4(address) => (address.to_string(), b'4'),\n                    IpAddr::V6(address) => (address.to_string(), b'6'),\n                };\n\n                let mut buf = Command::build(\n                    SMFIC_CONNECT,\n                    hostname.len() as u32 // hostname\n                        + 1 // NUL\n                        + 1 // family\n                        + std::mem::size_of::<u16>() as u32 // port\n                        + address.len() as u32 // address\n                        + 1, // NUL\n                );\n                buf.extend(hostname);\n                buf.push(0x00);\n                buf.push(family);\n                buf.extend(port.to_be_bytes().as_ref());\n                buf.extend(address.as_bytes());\n                buf.push(0x00);\n                buf\n            }\n            Command::Macro { macros } => {\n                let mut buf = Command::build(\n                    SMFIC_MACRO,\n                    macros.macros.iter().fold(1, |acc, macro_| {\n                        acc + macro_.name.len() as u32 + 1 + macro_.value.len() as u32 + 1\n                    }),\n                );\n                buf.push(macros.cmdcode);\n                for macro_ in macros.macros {\n                    buf.extend(macro_.name);\n                    buf.push(0x00);\n                    buf.extend(macro_.value.as_ref());\n                    buf.push(0x00);\n                }\n                buf\n            }\n            Command::Header { name, value } => {\n                let mut buf =\n                    Command::build(SMFIC_HEADER, name.len() as u32 + 1 + value.len() as u32 + 1);\n                buf.extend(name);\n                buf.push(0x00);\n                buf.extend(value);\n                buf.push(0x00);\n                buf\n            }\n            Command::EndOfHeader => Command::build(SMFIC_EOH, 0),\n            Command::Helo { hostname } => {\n                let mut buf = Command::build(SMFIC_HELO, hostname.len() as u32 + 1);\n                buf.extend(hostname);\n                buf.push(0x00);\n                buf\n            }\n            Command::MailFrom { sender, args } => {\n                let mut buf = Command::build(\n                    SMFIC_MAIL,\n                    sender.len() as u32 // sender\n                        + 1 // NUL\n                        + args.as_ref().map_or(0, |args| args.iter().fold(0, |acc, arg| acc + arg.len() as u32) + 1), // args\n                );\n                buf.extend(sender);\n                buf.push(0x00);\n                if let Some(args) = args {\n                    for arg in args {\n                        buf.extend(arg);\n                        buf.push(0x00);\n                    }\n                }\n                buf\n            }\n            Command::Rcpt { recipient, args } => {\n                let mut buf = Command::build(\n                    SMFIC_RCPT,\n                    recipient.len() as u32 // recipient\n                        + 1 // NUL\n                        + args.as_ref().map_or(0, |args| args.iter().fold(0, |acc, arg| acc + arg.len() as u32) + 1), // args\n                );\n                buf.extend(recipient);\n                buf.push(0x00);\n                if let Some(args) = args {\n                    for arg in args {\n                        buf.extend(arg);\n                        buf.push(0x00);\n                    }\n                }\n                buf\n            }\n            Command::OptionNegotiation(opt) => {\n                let mut buf = Command::build(SMFIC_OPTNEG, 3 * std::mem::size_of::<u32>() as u32);\n                buf.extend(opt.version.to_be_bytes().as_ref());\n                buf.extend(opt.actions.to_be_bytes().as_ref());\n                buf.extend(opt.protocol.to_be_bytes().as_ref());\n                buf\n            }\n            Command::Quit => Command::build(SMFIC_QUIT, 0),\n\n            // Version 6\n            Command::Data => Command::build(SMFIC_DATA, 0),\n            Command::QuitNewConnection => Command::build(SMFIC_QUIT_NC, 0),\n        }\n    }\n\n    #[cfg(feature = \"test_mode\")]\n    pub fn deserialize(bytes: &[u8]) -> Command<'_> {\n        let mut reader = PacketReader::new(bytes);\n        match reader.byte() {\n            SMFIC_ABORT => Command::Abort,\n            SMFIC_BODY => Command::Body { value: &bytes[1..] },\n            SMFIC_BODYEOB => Command::EndOfBody,\n            SMFIC_CONNECT => {\n                let hostname = reader.read_nul_terminated().unwrap();\n                let family = reader.byte();\n                let port = reader.read_u16();\n                let address = std::str::from_utf8(reader.read_nul_terminated().unwrap()).unwrap();\n                Command::Connect {\n                    hostname,\n                    port,\n                    address: match family {\n                        b'4' => IpAddr::V4(address.parse().unwrap()),\n                        b'6' => IpAddr::V6(address.parse().unwrap()),\n                        _ => unreachable!(),\n                    },\n                }\n            }\n            SMFIC_MACRO => {\n                let cmdcode = reader.byte();\n                let mut macros = Vec::new();\n                while let Some(name) = reader.read_nul_terminated() {\n                    let value = reader.read_nul_terminated().unwrap();\n                    macros.push(super::Macro {\n                        name,\n                        value: value.into(),\n                    });\n                }\n                Command::Macro {\n                    macros: super::Macros { cmdcode, macros },\n                }\n            }\n            SMFIC_HEADER => {\n                let name = reader.read_nul_terminated().unwrap();\n                let value = reader.read_nul_terminated().unwrap();\n                Command::Header { name, value }\n            }\n            SMFIC_EOH => Command::EndOfHeader,\n            SMFIC_HELO => {\n                let hostname = reader.read_nul_terminated().unwrap();\n                Command::Helo { hostname }\n            }\n            SMFIC_MAIL => {\n                let sender = reader.read_nul_terminated().unwrap();\n                let mut args = Vec::new();\n                while let Some(arg) = reader.read_nul_terminated() {\n                    args.push(arg);\n                }\n                Command::MailFrom {\n                    sender,\n                    args: Some(args),\n                }\n            }\n            SMFIC_RCPT => {\n                let recipient = reader.read_nul_terminated().unwrap();\n                let mut args = Vec::new();\n                while let Some(arg) = reader.read_nul_terminated() {\n                    args.push(arg);\n                }\n                Command::Rcpt {\n                    recipient,\n                    args: Some(args),\n                }\n            }\n            SMFIC_OPTNEG => Command::OptionNegotiation(super::Options {\n                version: reader.read_u32(),\n                actions: reader.read_u32(),\n                protocol: reader.read_u32(),\n            }),\n            SMFIC_QUIT => Command::Quit,\n            SMFIC_DATA => Command::Data,\n            SMFIC_QUIT_NC => Command::QuitNewConnection,\n            c => panic!(\"Unknown command: {}\", char::from(c)),\n        }\n    }\n}\n\nimpl Response {\n    pub fn deserialize(bytes: &[u8]) -> Option<Self> {\n        let frame_len = bytes.len().saturating_sub(1);\n        let mut bytes = bytes.iter();\n        match *bytes.next()? {\n            SMFIR_ADDRCPT => Response::Modification(Modification::AddRcpt {\n                recipient: read_nul_terminated(&mut bytes, frame_len)?,\n                args: String::new(),\n            }),\n            SMFIR_DELRCPT => Response::Modification(Modification::DeleteRcpt {\n                recipient: read_nul_terminated(&mut bytes, frame_len)?,\n            }),\n            SMFIR_ACCEPT => Response::Action(Action::Accept),\n            SMFIR_REPLBODY => {\n                let mut body = Vec::with_capacity(frame_len);\n                body.extend(bytes);\n                Response::Modification(Modification::ReplaceBody { value: body })\n            }\n            SMFIR_CONTINUE => Response::Action(Action::Continue),\n            SMFIR_DISCARD => Response::Action(Action::Discard),\n            SMFIR_ADDHEADER => Response::Modification(Modification::AddHeader {\n                name: read_nul_terminated(&mut bytes, 16)?,\n                value: read_nul_terminated(&mut bytes, frame_len)?,\n            }),\n            SMFIR_CHGHEADER => Response::Modification(Modification::ChangeHeader {\n                index: read_u32(&mut bytes)?,\n                name: read_nul_terminated(&mut bytes, 16)?,\n                value: read_nul_terminated(&mut bytes, frame_len)?,\n            }),\n            SMFIR_PROGRESS => Response::Progress,\n            SMFIR_QUARANTINE => Response::Modification(Modification::Quarantine {\n                reason: read_nul_terminated(&mut bytes, frame_len)?,\n            }),\n            SMFIR_REJECT => Response::Action(Action::Reject),\n            SMFIR_TEMPFAIL => Response::Action(Action::TempFail),\n            SMFIR_REPLYCODE => {\n                let code = [*bytes.next()?, *bytes.next()?, *bytes.next()?];\n                bytes.next()?; // Space\n                Response::Action(Action::ReplyCode {\n                    code,\n                    text: read_nul_terminated(&mut bytes, frame_len)?,\n                })\n            }\n            SMFIC_OPTNEG => Response::OptionNegotiation(Options {\n                version: read_u32(&mut bytes)?,\n                actions: read_u32(&mut bytes)?,\n                protocol: read_u32(&mut bytes)?,\n            }),\n\n            // V6\n            SMFIR_ADDRCPT_PAR => Response::Modification(Modification::AddRcpt {\n                recipient: read_nul_terminated(&mut bytes, frame_len)?,\n                args: read_nul_terminated(&mut bytes, frame_len)?,\n            }),\n            SMFIR_CHGFROM => Response::Modification(Modification::ChangeFrom {\n                sender: read_nul_terminated(&mut bytes, frame_len)?,\n                args: read_nul_terminated(&mut bytes, frame_len)?,\n            }),\n            SMFIR_SKIP => Response::Skip,\n            SMFIR_SETSYMLIST => Response::SetSymbols,\n            SMFIR_SHUTDOWN => Response::Action(Action::Shutdown),\n            SMFIR_CONN_FAIL => Response::Action(Action::ConnectionFailure),\n            SMFIR_INSHEADER => Response::Modification(Modification::InsertHeader {\n                index: read_u32(&mut bytes)?,\n                name: read_nul_terminated(&mut bytes, 16)?,\n                value: read_nul_terminated(&mut bytes, frame_len)?,\n            }),\n            _ => return None,\n        }\n        .into()\n    }\n\n    pub fn can_continue(&self) -> bool {\n        matches!(\n            self,\n            Response::Progress | Response::Action(Action::Accept | Action::Continue)\n        )\n    }\n\n    pub fn into_action(self) -> super::Result<Action> {\n        match self {\n            Response::Action(action) => Ok(action),\n            response => Err(Error::Unexpected(response)),\n        }\n    }\n\n    #[cfg(feature = \"test_mode\")]\n    pub fn serialize(&self) -> Vec<u8> {\n        match self {\n            Response::Action(action) => match action {\n                Action::Accept => Command::build(SMFIR_ACCEPT, 0),\n                Action::Continue => Command::build(SMFIR_CONTINUE, 0),\n                Action::Discard => Command::build(SMFIR_DISCARD, 0),\n                Action::Reject => Command::build(SMFIR_REJECT, 0),\n                Action::TempFail => Command::build(SMFIR_TEMPFAIL, 0),\n                Action::ReplyCode { code, text } => {\n                    let mut buf = Command::build(SMFIR_REPLYCODE, text.len() as u32 + 4 + 1);\n                    buf.extend(code);\n                    buf.push(b' ');\n                    buf.extend(text.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n                Action::Shutdown => Command::build(SMFIR_SHUTDOWN, 0),\n                Action::ConnectionFailure => Command::build(SMFIR_CONN_FAIL, 0),\n            },\n            Response::Modification(modif) => match modif {\n                Modification::ChangeFrom { sender, args } => {\n                    let mut buf =\n                        Command::build(SMFIR_CHGFROM, sender.len() as u32 + args.len() as u32 + 2);\n                    buf.extend(sender.as_bytes());\n                    buf.push(0x00);\n                    buf.extend(args.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n                Modification::AddRcpt { recipient, args } => {\n                    let mut buf = Command::build(\n                        SMFIR_ADDRCPT_PAR,\n                        recipient.len() as u32 + args.len() as u32 + 2,\n                    );\n                    buf.extend(recipient.as_bytes());\n                    buf.push(0x00);\n                    buf.extend(args.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n                Modification::DeleteRcpt { recipient } => {\n                    let mut buf = Command::build(SMFIR_DELRCPT, recipient.len() as u32 + 1);\n                    buf.extend(recipient.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n                Modification::ReplaceBody { value } => {\n                    let mut buf = Command::build(SMFIR_REPLBODY, value.len() as u32);\n                    buf.extend(value);\n                    buf\n                }\n                Modification::AddHeader { name, value } => {\n                    let mut buf =\n                        Command::build(SMFIR_ADDHEADER, name.len() as u32 + value.len() as u32 + 2);\n                    buf.extend(name.as_bytes());\n                    buf.push(0x00);\n                    buf.extend(value.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n                Modification::InsertHeader { index, name, value } => {\n                    let mut buf = Command::build(\n                        SMFIR_INSHEADER,\n                        name.len() as u32\n                            + value.len() as u32\n                            + std::mem::size_of::<u32>() as u32\n                            + 2,\n                    );\n                    buf.extend(index.to_be_bytes().as_ref());\n                    buf.extend(name.as_bytes());\n                    buf.push(0x00);\n                    buf.extend(value.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n                Modification::ChangeHeader { index, name, value } => {\n                    let mut buf = Command::build(\n                        SMFIR_CHGHEADER,\n                        name.len() as u32\n                            + value.len() as u32\n                            + std::mem::size_of::<u32>() as u32\n                            + 2,\n                    );\n                    buf.extend(index.to_be_bytes().as_ref());\n                    buf.extend(name.as_bytes());\n                    buf.push(0x00);\n                    buf.extend(value.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n                Modification::Quarantine { reason } => {\n                    let mut buf = Command::build(SMFIR_QUARANTINE, reason.len() as u32 + 1);\n                    buf.extend(reason.as_bytes());\n                    buf.push(0x00);\n                    buf\n                }\n            },\n            Response::Progress => Command::build(SMFIR_PROGRESS, 0),\n            Response::Skip => Command::build(SMFIR_SKIP, 0),\n            Response::SetSymbols => Command::build(SMFIR_SETSYMLIST, 0),\n            Response::OptionNegotiation(opt) => {\n                let mut buf = Command::build(SMFIC_OPTNEG, 3 * std::mem::size_of::<u32>() as u32);\n                buf.extend(opt.version.to_be_bytes().as_ref());\n                buf.extend(opt.actions.to_be_bytes().as_ref());\n                buf.extend(opt.protocol.to_be_bytes().as_ref());\n                buf\n            }\n        }\n    }\n}\n\nfn read_nul_terminated(bytes: &mut std::slice::Iter<u8>, expected_len: usize) -> Option<String> {\n    let mut buf = Vec::with_capacity(expected_len);\n    loop {\n        match bytes.next()? {\n            0x00 => break,\n            byte => buf.push(*byte),\n        }\n    }\n    String::from_utf8(buf).ok()\n}\n\nfn read_u32(bytes: &mut std::slice::Iter<u8>) -> Option<u32> {\n    let mut buf = [0u8; 4];\n    for byte in buf.iter_mut() {\n        *byte = *bytes.next()?;\n    }\n    Some(u32::from_be_bytes(buf))\n}\n\n#[cfg(feature = \"test_mode\")]\npub struct PacketReader<'x> {\n    bytes: &'x [u8],\n    iter: std::iter::Enumerate<std::slice::Iter<'x, u8>>,\n}\n\n#[cfg(feature = \"test_mode\")]\nimpl<'x> PacketReader<'x> {\n    pub fn new(bytes: &'x [u8]) -> PacketReader<'x> {\n        Self {\n            bytes,\n            iter: bytes.iter().enumerate(),\n        }\n    }\n\n    pub fn byte(&mut self) -> u8 {\n        *self.iter.next().unwrap().1\n    }\n\n    pub fn read_nul_terminated(&mut self) -> Option<&'x [u8]> {\n        let (start_pos, ch) = self.iter.next()?;\n        let mut end_pos = start_pos;\n\n        if *ch != 0x00 {\n            loop {\n                match self.iter.next().unwrap().1 {\n                    0x00 => break,\n                    _ => end_pos += 1,\n                }\n            }\n        }\n\n        Some(&self.bytes[start_pos..end_pos + 1])\n    }\n\n    pub fn read_u32(&mut self) -> u32 {\n        let mut buf = [0u8; 4];\n        for byte in buf.iter_mut() {\n            *byte = self.byte();\n        }\n        u32::from_be_bytes(buf)\n    }\n\n    pub fn read_u16(&mut self) -> u16 {\n        let mut buf = [0u8; 2];\n        for byte in buf.iter_mut() {\n            *byte = self.byte();\n        }\n        u16::from_be_bytes(buf)\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/milter/receiver.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nenum State {\n    Len {\n        buf: [u8; std::mem::size_of::<u32>()],\n        bytes_read: usize,\n    },\n    Frame {\n        buf: Vec<u8>,\n        frame_len: usize,\n    },\n}\n\npub struct Receiver {\n    packet_pos: usize,\n    state: State,\n    max_frame_len: usize,\n}\n\npub enum FrameResult<'x> {\n    Frame(Cow<'x, [u8]>),\n    Incomplete,\n    TooLarge(usize),\n}\n\nimpl Default for State {\n    fn default() -> Self {\n        State::Len {\n            buf: [0; std::mem::size_of::<u32>()],\n            bytes_read: 0,\n        }\n    }\n}\n\nimpl Receiver {\n    pub fn with_max_frame_len(max_frame_len: usize) -> Self {\n        Receiver {\n            packet_pos: 0,\n            state: State::default(),\n            max_frame_len,\n        }\n    }\n\n    pub fn read_frame<'x>(&mut self, packet: &'x [u8]) -> FrameResult<'x> {\n        if !packet.is_empty() {\n            match &mut self.state {\n                State::Len { buf, bytes_read } => {\n                    while *bytes_read < std::mem::size_of::<u32>() {\n                        if let Some(byte) = packet.get(self.packet_pos) {\n                            buf[*bytes_read] = *byte;\n                            *bytes_read += 1;\n                            self.packet_pos += 1;\n                        } else {\n                            self.packet_pos = 0;\n                            return FrameResult::Incomplete;\n                        }\n                    }\n                    let length = u32::from_be_bytes(*buf) as usize;\n                    if length <= self.max_frame_len {\n                        if let Some(frame) = packet.get(self.packet_pos..self.packet_pos + length) {\n                            self.packet_pos += length;\n                            self.state = State::default();\n                            FrameResult::Frame(frame.into())\n                        } else {\n                            let mut buf = Vec::with_capacity(length);\n                            if let Some(bytes_available) = packet.get(self.packet_pos..) {\n                                buf.extend(bytes_available);\n                            }\n                            self.state = State::Frame {\n                                buf,\n                                frame_len: length,\n                            };\n                            self.packet_pos = 0;\n                            FrameResult::Incomplete\n                        }\n                    } else {\n                        FrameResult::TooLarge(length)\n                    }\n                }\n                State::Frame { buf, frame_len } => {\n                    let bytes_pending = *frame_len - buf.len();\n                    if let Some(bytes) =\n                        packet.get(self.packet_pos..self.packet_pos + bytes_pending)\n                    {\n                        let mut buf = std::mem::take(buf);\n                        buf.extend(bytes);\n                        self.packet_pos += bytes_pending;\n                        self.state = State::default();\n                        FrameResult::Frame(buf.into())\n                    } else if let Some(bytes_available) = packet.get(self.packet_pos..) {\n                        buf.extend(bytes_available);\n                        self.packet_pos = 0;\n                        FrameResult::Incomplete\n                    } else {\n                        self.packet_pos = 0;\n                        FrameResult::Incomplete\n                    }\n                }\n            }\n        } else {\n            FrameResult::Incomplete\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse common::config::smtp::auth::{ArcSealer, DkimSigner};\nuse mail_auth::{\n    ArcOutput, AuthenticatedMessage, AuthenticationResults, DkimResult, DmarcResult, IprevResult,\n    SpfResult, arc::ArcSet, dkim::Signature, dmarc::Policy,\n};\n\npub mod auth;\npub mod data;\npub mod ehlo;\npub mod hooks;\npub mod mail;\npub mod milter;\npub mod rcpt;\npub mod session;\npub mod spam;\npub mod spawn;\npub mod vrfy;\n\n#[derive(Debug, Default)]\npub struct FilterResponse {\n    pub message: Cow<'static, str>,\n    pub disconnect: bool,\n}\n\npub trait ArcSeal {\n    fn seal<'x>(\n        &self,\n        message: &'x AuthenticatedMessage,\n        results: &'x AuthenticationResults,\n        arc_output: &'x ArcOutput,\n    ) -> mail_auth::Result<ArcSet<'x>>;\n}\n\nimpl ArcSeal for ArcSealer {\n    fn seal<'x>(\n        &self,\n        message: &'x AuthenticatedMessage,\n        results: &'x AuthenticationResults,\n        arc_output: &'x ArcOutput,\n    ) -> mail_auth::Result<ArcSet<'x>> {\n        match self {\n            ArcSealer::RsaSha256(sealer) => sealer.seal(message, results, arc_output),\n            ArcSealer::Ed25519Sha256(sealer) => sealer.seal(message, results, arc_output),\n        }\n    }\n}\n\npub trait DkimSign {\n    fn sign(&self, message: &[u8]) -> mail_auth::Result<Signature>;\n    fn sign_chained(&self, message: &[&[u8]]) -> mail_auth::Result<Signature>;\n}\n\nimpl DkimSign for DkimSigner {\n    fn sign(&self, message: &[u8]) -> mail_auth::Result<Signature> {\n        match self {\n            DkimSigner::RsaSha256(signer) => signer.sign(message),\n            DkimSigner::Ed25519Sha256(signer) => signer.sign(message),\n        }\n    }\n    fn sign_chained(&self, message: &[&[u8]]) -> mail_auth::Result<Signature> {\n        match self {\n            DkimSigner::RsaSha256(signer) => signer.sign_chained(message.iter().copied()),\n            DkimSigner::Ed25519Sha256(signer) => signer.sign_chained(message.iter().copied()),\n        }\n    }\n}\n\npub trait AuthResult {\n    fn as_str(&self) -> &'static str;\n}\n\nimpl AuthResult for SpfResult {\n    fn as_str(&self) -> &'static str {\n        match self {\n            SpfResult::Pass => \"pass\",\n            SpfResult::Fail => \"fail\",\n            SpfResult::SoftFail => \"softfail\",\n            SpfResult::Neutral => \"neutral\",\n            SpfResult::None => \"none\",\n            SpfResult::TempError => \"temperror\",\n            SpfResult::PermError => \"permerror\",\n        }\n    }\n}\n\nimpl AuthResult for IprevResult {\n    fn as_str(&self) -> &'static str {\n        match self {\n            IprevResult::Pass => \"pass\",\n            IprevResult::Fail(_) => \"fail\",\n            IprevResult::TempError(_) => \"temperror\",\n            IprevResult::PermError(_) => \"permerror\",\n            IprevResult::None => \"none\",\n        }\n    }\n}\n\nimpl AuthResult for DkimResult {\n    fn as_str(&self) -> &'static str {\n        match self {\n            DkimResult::Pass => \"pass\",\n            DkimResult::None => \"none\",\n            DkimResult::Neutral(_) => \"neutral\",\n            DkimResult::Fail(_) => \"fail\",\n            DkimResult::PermError(_) => \"permerror\",\n            DkimResult::TempError(_) => \"temperror\",\n        }\n    }\n}\n\nimpl AuthResult for DmarcResult {\n    fn as_str(&self) -> &'static str {\n        match self {\n            DmarcResult::Pass => \"pass\",\n            DmarcResult::Fail(_) => \"fail\",\n            DmarcResult::TempError(_) => \"temperror\",\n            DmarcResult::PermError(_) => \"permerror\",\n            DmarcResult::None => \"none\",\n        }\n    }\n}\n\nimpl AuthResult for Policy {\n    fn as_str(&self) -> &'static str {\n        match self {\n            Policy::Reject => \"reject\",\n            Policy::Quarantine => \"quarantine\",\n            Policy::None | Policy::Unspecified => \"none\",\n        }\n    }\n}\n\nimpl FilterResponse {\n    pub fn accept() -> Self {\n        Self {\n            message: Cow::Borrowed(\"250 2.0.0 Message queued for delivery.\\r\\n\"),\n            disconnect: false,\n        }\n    }\n\n    pub fn reject() -> Self {\n        Self {\n            message: Cow::Borrowed(\"503 5.5.3 Message rejected.\\r\\n\"),\n            disconnect: false,\n        }\n    }\n\n    pub fn temp_fail() -> Self {\n        Self {\n            message: Cow::Borrowed(\"451 4.3.5 Unable to accept message at this time.\\r\\n\"),\n            disconnect: false,\n        }\n    }\n\n    pub fn shutdown() -> Self {\n        Self {\n            message: Cow::Borrowed(\"421 4.3.0 Server shutting down.\\r\\n\"),\n            disconnect: true,\n        }\n    }\n\n    pub fn server_failure() -> Self {\n        Self {\n            message: Cow::Borrowed(\"451 4.3.5 Unable to accept message at this time.\\r\\n\"),\n            disconnect: false,\n        }\n    }\n\n    pub fn disconnect(self) -> Self {\n        Self {\n            disconnect: true,\n            ..self\n        }\n    }\n\n    pub fn into_bytes(self) -> Cow<'static, [u8]> {\n        match self.message {\n            Cow::Borrowed(s) => Cow::Borrowed(s.as_bytes()),\n            Cow::Owned(s) => Cow::Owned(s.into_bytes()),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/rcpt.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::{Session, SessionAddress},\n    scripts::ScriptResult,\n};\nuse common::{\n    KV_GREYLIST, config::smtp::session::Stage, listener::SessionStream, scripts::ScriptModification,\n};\nuse directory::backend::RcptType;\nuse smtp_proto::{\n    RCPT_NOTIFY_DELAY, RCPT_NOTIFY_FAILURE, RCPT_NOTIFY_NEVER, RCPT_NOTIFY_SUCCESS, RcptTo,\n};\nuse std::borrow::Cow;\nuse store::dispatch::lookup::KeyValue;\nuse trc::{SecurityEvent, SmtpEvent};\nuse utils::DomainPart;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_rcpt_to(&mut self, to: RcptTo<Cow<'_, str>>) -> Result<(), ()> {\n        #[cfg(feature = \"test_mode\")]\n        if self.instance.id.ends_with(\"-debug\") {\n            if to.address.contains(\"fail@\") {\n                return self.write(b\"503 5.5.1 Invalid recipient.\\r\\n\").await;\n            } else if (to.address.contains(\"delay-random@\") && rand::random())\n                || to.address.contains(\"delay@\")\n            {\n                return self.write(b\"451 4.5.3 Try again later.\\r\\n\").await;\n            } else if to.address.contains(\"slow@\") {\n                tokio::time::sleep(std::time::Duration::from_secs(\n                    rand::random::<u64>() % 5 + 5,\n                ))\n                .await;\n            }\n        }\n\n        if self.data.mail_from.is_none() {\n            trc::event!(\n                Smtp(SmtpEvent::MailFromMissing),\n                SpanId = self.data.session_id,\n            );\n            return self.write(b\"503 5.5.1 MAIL is required first.\\r\\n\").await;\n        } else if self.data.rcpt_to.len() >= self.params.rcpt_max {\n            trc::event!(\n                Smtp(SmtpEvent::TooManyRecipients),\n                SpanId = self.data.session_id,\n                Limit = self.params.rcpt_max,\n            );\n            return self.write(b\"455 4.5.3 Too many recipients.\\r\\n\").await;\n        }\n\n        // Verify parameters\n        if ((to.flags\n            & (RCPT_NOTIFY_DELAY | RCPT_NOTIFY_NEVER | RCPT_NOTIFY_SUCCESS | RCPT_NOTIFY_FAILURE)\n            != 0)\n            || to.orcpt.is_some())\n            && !self.params.rcpt_dsn\n        {\n            trc::event!(Smtp(SmtpEvent::DsnDisabled), SpanId = self.data.session_id,);\n            return self\n                .write(b\"501 5.5.4 DSN extension has been disabled.\\r\\n\")\n                .await;\n        }\n\n        // Build RCPT\n        let address_lcase = to.address.to_lowercase();\n        let rcpt = SessionAddress {\n            domain: address_lcase.domain_part().into(),\n            address_lcase,\n            address: to.address.into_owned(),\n            flags: to.flags,\n            dsn_info: to.orcpt.map(|e| e.into_owned()),\n        };\n\n        if self.data.rcpt_to.contains(&rcpt) {\n            trc::event!(\n                Smtp(SmtpEvent::RcptToDuplicate),\n                SpanId = self.data.session_id,\n                To = rcpt.address_lcase,\n            );\n            self.data.rcpt_oks += 1;\n            return self.write(b\"250 2.1.5 OK\\r\\n\").await;\n        }\n        self.data.rcpt_to.push(rcpt);\n\n        // Address rewriting and Sieve filtering\n        let rcpt_config = &self.server.core.smtp.session.rcpt;\n        let rcpt_script = self\n            .server\n            .eval_if::<String, _>(&rcpt_config.script, self, self.data.session_id)\n            .await\n            .and_then(|name| {\n                self.server\n                    .get_trusted_sieve_script(&name, self.data.session_id)\n                    .map(|s| (s.clone(), name))\n            });\n\n        let session_config = &self.server.core.smtp.session;\n        if rcpt_script.is_some()\n            || !rcpt_config.rewrite.is_empty()\n            || session_config\n                .milters\n                .iter()\n                .any(|m| m.run_on_stage.contains(&Stage::Rcpt))\n            || session_config\n                .hooks\n                .iter()\n                .any(|h| h.run_on_stage.contains(&Stage::Rcpt))\n        {\n            // Sieve filtering\n            if let Some((script, script_id)) = rcpt_script {\n                match self\n                    .run_script(\n                        script_id,\n                        script.clone(),\n                        self.build_script_parameters(\"rcpt\"),\n                    )\n                    .await\n                {\n                    ScriptResult::Accept { modifications } => {\n                        if !modifications.is_empty() {\n                            for modification in modifications {\n                                if let ScriptModification::SetEnvelope { name, value } =\n                                    modification\n                                {\n                                    self.data.apply_envelope_modification(name, value);\n                                }\n                            }\n                        }\n                    }\n                    ScriptResult::Reject(message) => {\n                        self.data.rcpt_to.pop();\n                        return self.write(message.as_bytes()).await;\n                    }\n                    _ => (),\n                }\n            }\n\n            // Milter filtering\n            if let Err(message) = self.run_milters(Stage::Rcpt, None).await {\n                self.data.rcpt_to.pop();\n                return self.write(message.message.as_bytes()).await;\n            }\n\n            // MTAHook filtering\n            if let Err(message) = self.run_mta_hooks(Stage::Rcpt, None, None).await {\n                self.data.rcpt_to.pop();\n                return self.write(message.message.as_bytes()).await;\n            }\n\n            // Address rewriting\n            if let Some(new_address) = self\n                .server\n                .eval_if::<String, _>(&rcpt_config.rewrite, self, self.data.session_id)\n                .await\n            {\n                let rcpt = self.data.rcpt_to.last_mut().unwrap();\n\n                trc::event!(\n                    Smtp(SmtpEvent::RcptToRewritten),\n                    SpanId = self.data.session_id,\n                    Details = rcpt.address_lcase.clone(),\n                    To = new_address.clone(),\n                );\n\n                if new_address.contains('@') {\n                    rcpt.address_lcase = new_address.to_lowercase();\n                    rcpt.domain = rcpt.address_lcase.domain_part().into();\n                    rcpt.address = new_address;\n                }\n            }\n\n            // Check for duplicates\n            let rcpt = self.data.rcpt_to.last().unwrap();\n            if self.data.rcpt_to.iter().filter(|r| r == &rcpt).count() > 1 {\n                trc::event!(\n                    Smtp(SmtpEvent::RcptToDuplicate),\n                    SpanId = self.data.session_id,\n                    To = rcpt.address_lcase.clone(),\n                );\n                self.data.rcpt_to.pop();\n                self.data.rcpt_oks += 1;\n                return self.write(b\"250 2.1.5 OK\\r\\n\").await;\n            }\n        }\n\n        // Verify address\n        let rcpt = self.data.rcpt_to.last().unwrap();\n        let mut rcpt_members = None;\n        if let Some(directory) = self\n            .server\n            .eval_if::<String, _>(&rcpt_config.directory, self, self.data.session_id)\n            .await\n            .and_then(|name| self.server.get_directory(&name))\n        {\n            match directory.is_local_domain(&rcpt.domain).await {\n                Ok(true) => {\n                    match self\n                        .server\n                        .rcpt(directory, &rcpt.address_lcase, self.data.session_id)\n                        .await\n                    {\n                        Ok(RcptType::Mailbox) => {}\n                        Ok(RcptType::List(members)) => {\n                            rcpt_members = Some(members);\n                        }\n                        Ok(RcptType::Invalid) => {\n                            trc::event!(\n                                Smtp(SmtpEvent::MailboxDoesNotExist),\n                                SpanId = self.data.session_id,\n                                To = rcpt.address_lcase.clone(),\n                            );\n\n                            let rcpt_to = self.data.rcpt_to.pop().unwrap().address_lcase;\n                            return self\n                                .rcpt_error(b\"550 5.1.2 Mailbox does not exist.\\r\\n\", rcpt_to)\n                                .await;\n                        }\n                        Err(err) => {\n                            trc::error!(\n                                err.span_id(self.data.session_id)\n                                    .caused_by(trc::location!())\n                                    .details(\"Failed to verify address.\")\n                            );\n\n                            self.data.rcpt_to.pop();\n                            return self\n                                .write(b\"451 4.4.3 Unable to verify address at this time.\\r\\n\")\n                                .await;\n                        }\n                    }\n                }\n                Ok(false) => {\n                    if !self\n                        .server\n                        .eval_if(&rcpt_config.relay, self, self.data.session_id)\n                        .await\n                        .unwrap_or(false)\n                    {\n                        trc::event!(\n                            Smtp(SmtpEvent::RelayNotAllowed),\n                            SpanId = self.data.session_id,\n                            To = rcpt.address_lcase.clone(),\n                        );\n\n                        let rcpt_to = self.data.rcpt_to.pop().unwrap().address_lcase;\n                        return self\n                            .rcpt_error(b\"550 5.1.2 Relay not allowed.\\r\\n\", rcpt_to)\n                            .await;\n                    }\n                }\n                Err(err) => {\n                    trc::error!(\n                        err.span_id(self.data.session_id)\n                            .caused_by(trc::location!())\n                            .details(\"Failed to verify address.\")\n                    );\n\n                    self.data.rcpt_to.pop();\n                    return self\n                        .write(b\"451 4.4.3 Unable to verify address at this time.\\r\\n\")\n                        .await;\n                }\n            }\n        } else if !self\n            .server\n            .eval_if(&rcpt_config.relay, self, self.data.session_id)\n            .await\n            .unwrap_or(false)\n        {\n            trc::event!(\n                Smtp(SmtpEvent::RelayNotAllowed),\n                SpanId = self.data.session_id,\n                To = rcpt.address_lcase.clone(),\n            );\n\n            let rcpt_to = self.data.rcpt_to.pop().unwrap().address_lcase;\n            return self\n                .rcpt_error(b\"550 5.1.2 Relay not allowed.\\r\\n\", rcpt_to)\n                .await;\n        }\n\n        if self.is_allowed().await {\n            // Greylist\n            if let Some(greylist_duration) = self\n                .server\n                .core\n                .spam\n                .grey_list_expiry\n                .filter(|_| self.data.authenticated_as.is_none())\n            {\n                let from_addr = self\n                    .data\n                    .mail_from\n                    .as_ref()\n                    .unwrap()\n                    .address_lcase\n                    .as_bytes();\n                let to_addr = self.data.rcpt_to.last().unwrap().address_lcase.as_bytes();\n                let mut key = Vec::with_capacity(from_addr.len() + to_addr.len() + 1);\n                key.push(KV_GREYLIST);\n                key.extend_from_slice(from_addr);\n                key.extend_from_slice(to_addr);\n\n                match self.server.in_memory_store().key_exists(key.clone()).await {\n                    Ok(true) => (),\n                    Ok(false) => {\n                        match self\n                            .server\n                            .in_memory_store()\n                            .key_set(KeyValue::new(key, vec![]).expires(greylist_duration))\n                            .await\n                        {\n                            Ok(_) => {\n                                let rcpt = self.data.rcpt_to.pop().unwrap();\n\n                                trc::event!(\n                                    Smtp(SmtpEvent::RcptToGreylisted),\n                                    SpanId = self.data.session_id,\n                                    To = rcpt.address_lcase,\n                                );\n\n                                return self\n                                    .write(\n                                        concat!(\n                                            \"452 4.2.2 Greylisted, please try \",\n                                            \"again in a few moments.\\r\\n\"\n                                        )\n                                        .as_bytes(),\n                                    )\n                                    .await;\n                            }\n                            Err(err) => {\n                                trc::error!(\n                                    err.span_id(self.data.session_id)\n                                        .caused_by(trc::location!())\n                                        .details(\"Failed to set greylist.\")\n                                );\n                            }\n                        }\n                    }\n                    Err(err) => {\n                        trc::error!(\n                            err.span_id(self.data.session_id)\n                                .caused_by(trc::location!())\n                                .details(\"Failed to check greylist.\")\n                        );\n                    }\n                }\n            }\n\n            trc::event!(\n                Smtp(SmtpEvent::RcptTo),\n                SpanId = self.data.session_id,\n                To = self.data.rcpt_to.last().unwrap().address_lcase.clone(),\n            );\n        } else {\n            trc::event!(\n                Smtp(SmtpEvent::RateLimitExceeded),\n                SpanId = self.data.session_id,\n                To = self.data.rcpt_to.last().unwrap().address_lcase.clone(),\n            );\n\n            self.data.rcpt_to.pop();\n            return self\n                .write(b\"452 4.4.5 Rate limit exceeded, try again later.\\r\\n\")\n                .await;\n        }\n\n        // Expand list\n        if let Some(members) = rcpt_members {\n            let list_addr = self.data.rcpt_to.pop().unwrap();\n            let orcpt = format!(\"rfc822;{}\", list_addr.address_lcase);\n            for member in members {\n                let mut member_addr = SessionAddress::new(member);\n                if !self.data.rcpt_to.contains(&member_addr)\n                    && member_addr.address_lcase != list_addr.address_lcase\n                {\n                    member_addr.dsn_info = orcpt.clone().into();\n                    member_addr.flags = list_addr.flags;\n                    self.data.rcpt_to.push(member_addr);\n                }\n            }\n        }\n\n        self.data.rcpt_oks += 1;\n        self.write(b\"250 2.1.5 OK\\r\\n\").await\n    }\n\n    async fn rcpt_error(&mut self, response: &[u8], rcpt: String) -> Result<(), ()> {\n        tokio::time::sleep(self.params.rcpt_errors_wait).await;\n        self.data.rcpt_errors += 1;\n        let has_too_many_errors = self.data.rcpt_errors >= self.params.rcpt_errors_max;\n\n        match self\n            .server\n            .is_rcpt_fail2banned(self.data.remote_ip, &rcpt)\n            .await\n        {\n            Ok(true) => {\n                trc::event!(\n                    Security(SecurityEvent::AbuseBan),\n                    SpanId = self.data.session_id,\n                    RemoteIp = self.data.remote_ip,\n                    To = rcpt,\n                );\n            }\n            Ok(false) => {\n                if has_too_many_errors {\n                    trc::event!(\n                        Smtp(SmtpEvent::TooManyInvalidRcpt),\n                        SpanId = self.data.session_id,\n                        Limit = self.params.rcpt_errors_max,\n                        To = rcpt,\n                    );\n                }\n            }\n            Err(err) => {\n                trc::error!(\n                    err.span_id(self.data.session_id)\n                        .caused_by(trc::location!())\n                        .details(\"Failed to check if IP should be banned.\")\n                );\n            }\n        }\n\n        if !has_too_many_errors {\n            self.write(response).await\n        } else {\n            self.write(b\"451 4.3.0 Too many errors, disconnecting.\\r\\n\")\n                .await?;\n            Err(())\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    config::{server::ServerProtocol, smtp::session::Mechanism},\n    expr::{self, functions::ResolveVariable, *},\n    listener::SessionStream,\n};\n\nuse compact_str::ToCompactString;\nuse smtp_proto::{\n    request::receiver::{\n        BdatReceiver, DataReceiver, DummyDataReceiver, DummyLineReceiver, LineReceiver,\n        MAX_LINE_LENGTH,\n    },\n    *,\n};\nuse tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt};\nuse trc::{NetworkEvent, SecurityEvent, SmtpEvent};\n\nuse crate::core::{Session, State};\n\nuse super::auth::SaslToken;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn ingest(&mut self, bytes: &[u8]) -> Result<bool, ()> {\n        let mut iter = bytes.iter();\n        let mut state = std::mem::replace(&mut self.state, State::None);\n\n        'outer: loop {\n            match &mut state {\n                State::Request(receiver) => loop {\n                    match receiver.ingest(&mut iter) {\n                        Ok(request) => match request {\n                            Request::Rcpt { to } => {\n                                self.handle_rcpt_to(to).await?;\n                            }\n                            Request::Mail { from } => {\n                                self.handle_mail_from(from).await?;\n                            }\n                            Request::Ehlo { host } => {\n                                if self.instance.protocol == ServerProtocol::Smtp {\n                                    self.handle_ehlo(host, true).await?;\n                                } else {\n                                    trc::event!(\n                                        Smtp(SmtpEvent::LhloExpected),\n                                        SpanId = self.data.session_id,\n                                    );\n\n                                    self.write(b\"500 5.5.1 Invalid command.\\r\\n\").await?;\n                                }\n                            }\n                            Request::Data => {\n                                if self.can_send_data().await? {\n                                    self.write(b\"354 Start mail input; end with <CRLF>.<CRLF>\\r\\n\")\n                                        .await?;\n                                    self.data.message = Vec::with_capacity(1024);\n                                    state = State::Data(DataReceiver::new());\n                                    continue 'outer;\n                                }\n                            }\n                            Request::Bdat {\n                                chunk_size,\n                                is_last,\n                            } => {\n                                state = if chunk_size + self.data.message.len()\n                                    < self.params.max_message_size\n                                {\n                                    if self.data.message.is_empty() {\n                                        self.data.message = Vec::with_capacity(chunk_size);\n                                    } else {\n                                        self.data.message.reserve(chunk_size);\n                                    }\n                                    State::Bdat(BdatReceiver::new(chunk_size, is_last))\n                                } else {\n                                    // Chunk is too large, ignore.\n                                    State::DataTooLarge(DummyDataReceiver::new_bdat(chunk_size))\n                                };\n                                continue 'outer;\n                            }\n                            Request::Auth {\n                                mechanism,\n                                initial_response,\n                            } => {\n                                let auth: u64 = self\n                                    .server\n                                    .eval_if::<Mechanism, _>(\n                                        &self.server.core.smtp.session.auth.mechanisms,\n                                        self,\n                                        self.data.session_id,\n                                    )\n                                    .await\n                                    .unwrap_or_default()\n                                    .into();\n                                if auth == 0 || self.params.auth_directory.is_none() {\n                                    trc::event!(\n                                        Smtp(SmtpEvent::AuthNotAllowed),\n                                        SpanId = self.data.session_id,\n                                    );\n\n                                    self.write(b\"503 5.5.1 AUTH not allowed.\\r\\n\").await?;\n                                } else if let Some(authenticated_as) = self.authenticated_as() {\n                                    trc::event!(\n                                        Smtp(SmtpEvent::AlreadyAuthenticated),\n                                        SpanId = self.data.session_id,\n                                        AccountName = authenticated_as.to_string(),\n                                    );\n\n                                    self.write(b\"503 5.5.1 Already authenticated.\\r\\n\").await?;\n                                } else if let Some(mut token) =\n                                    SaslToken::from_mechanism(mechanism & auth)\n                                {\n                                    if self\n                                        .handle_sasl_response(\n                                            &mut token,\n                                            initial_response.as_bytes(),\n                                        )\n                                        .await?\n                                    {\n                                        state = State::Sasl(LineReceiver::new(token));\n                                        continue 'outer;\n                                    }\n                                } else {\n                                    trc::event!(\n                                        Smtp(SmtpEvent::AuthMechanismNotSupported),\n                                        SpanId = self.data.session_id,\n                                    );\n\n                                    self.write(\n                                        b\"554 5.7.8 Authentication mechanism not supported.\\r\\n\",\n                                    )\n                                    .await?;\n                                }\n                            }\n                            Request::Noop { .. } => {\n                                trc::event!(Smtp(SmtpEvent::Noop), SpanId = self.data.session_id,);\n\n                                self.write(b\"250 2.0.0 OK\\r\\n\").await?;\n                            }\n                            Request::Vrfy { value } => {\n                                self.handle_vrfy(value).await?;\n                            }\n                            Request::Expn { value } => {\n                                self.handle_expn(value).await?;\n                            }\n                            Request::StartTls => {\n                                if !self.stream.is_tls() {\n                                    if self.instance.acceptor.is_tls() {\n                                        trc::event!(\n                                            Smtp(SmtpEvent::StartTls),\n                                            SpanId = self.data.session_id,\n                                        );\n\n                                        self.write(b\"220 2.0.0 Ready to start TLS.\\r\\n\").await?;\n                                        #[cfg(any(test, feature = \"test_mode\"))]\n                                        if self.data.helo_domain.contains(\"badtls\") {\n                                            return Err(());\n                                        }\n                                        self.state = State::default();\n                                        return Ok(false);\n                                    } else {\n                                        trc::event!(\n                                            Smtp(SmtpEvent::StartTlsUnavailable),\n                                            SpanId = self.data.session_id,\n                                        );\n\n                                        self.write(b\"502 5.7.0 TLS not available.\\r\\n\").await?;\n                                    }\n                                } else {\n                                    trc::event!(\n                                        Smtp(SmtpEvent::StartTlsAlready),\n                                        SpanId = self.data.session_id,\n                                    );\n\n                                    self.write(b\"504 5.7.4 Already in TLS mode.\\r\\n\").await?;\n                                }\n                            }\n                            Request::Rset => {\n                                trc::event!(Smtp(SmtpEvent::Rset), SpanId = self.data.session_id,);\n\n                                self.reset();\n                                self.write(b\"250 2.0.0 OK\\r\\n\").await?;\n                            }\n                            Request::Quit => {\n                                trc::event!(Smtp(SmtpEvent::Quit), SpanId = self.data.session_id,);\n\n                                self.write(b\"221 2.0.0 Bye.\\r\\n\").await?;\n                                return Err(());\n                            }\n                            Request::Help { .. } => {\n                                trc::event!(Smtp(SmtpEvent::Help), SpanId = self.data.session_id,);\n\n                                self.write(b\"250 2.0.0 Help can be found at https://stalw.art\\r\\n\")\n                                    .await?;\n                            }\n                            Request::Helo { host } => {\n                                if self.instance.protocol == ServerProtocol::Smtp {\n                                    self.handle_ehlo(host, false).await?;\n                                } else {\n                                    trc::event!(\n                                        Smtp(SmtpEvent::LhloExpected),\n                                        SpanId = self.data.session_id,\n                                    );\n\n                                    self.write(b\"500 5.5.1 Invalid command: LHLO expected.\\r\\n\")\n                                        .await?;\n                                }\n                            }\n                            Request::Lhlo { host } => {\n                                if self.instance.protocol == ServerProtocol::Lmtp {\n                                    self.handle_ehlo(host, true).await?;\n                                } else {\n                                    trc::event!(\n                                        Smtp(SmtpEvent::EhloExpected),\n                                        SpanId = self.data.session_id,\n                                    );\n\n                                    self.write(b\"502 5.5.1 Invalid command: EHLO expected.\\r\\n\")\n                                        .await?;\n                                }\n                            }\n                            cmd @ (Request::Etrn { .. }\n                            | Request::Atrn { .. }\n                            | Request::Burl { .. }) => {\n                                trc::event!(\n                                    Smtp(SmtpEvent::CommandNotImplemented),\n                                    SpanId = self.data.session_id,\n                                    Details = format!(\"{cmd:?}\"),\n                                );\n\n                                self.write(b\"502 5.5.1 Command not implemented.\\r\\n\")\n                                    .await?;\n                            }\n                        },\n                        Err(err) => match err {\n                            Error::NeedsMoreData { .. } => break 'outer,\n                            Error::UnknownCommand | Error::InvalidResponse { .. } => {\n                                // Check for port scanners\n                                if !self.is_authenticated() {\n                                    match self\n                                        .server\n                                        .is_scanner_fail2banned(self.data.remote_ip)\n                                        .await\n                                    {\n                                        Ok(true) => {\n                                            trc::event!(\n                                                Security(SecurityEvent::ScanBan),\n                                                SpanId = self.data.session_id,\n                                                RemoteIp = self.data.remote_ip,\n                                                Reason = \"Invalid SMTP command\",\n                                            );\n\n                                            return Err(());\n                                        }\n                                        Ok(false) => {}\n                                        Err(err) => {\n                                            trc::error!(\n                                                err.span_id(self.data.session_id)\n                                                    .details(\"Failed to check for fail2ban\")\n                                            );\n                                        }\n                                    }\n                                }\n\n                                trc::event!(\n                                    Smtp(SmtpEvent::InvalidCommand),\n                                    SpanId = self.data.session_id,\n                                );\n\n                                self.write(b\"500 5.5.1 Invalid command.\\r\\n\").await?;\n                            }\n                            Error::InvalidSenderAddress => {\n                                trc::event!(\n                                    Smtp(SmtpEvent::InvalidSenderAddress),\n                                    SpanId = self.data.session_id,\n                                );\n\n                                self.write(b\"501 5.1.8 Bad sender's system address.\\r\\n\")\n                                    .await?;\n                            }\n                            Error::InvalidRecipientAddress => {\n                                trc::event!(\n                                    Smtp(SmtpEvent::InvalidRecipientAddress),\n                                    SpanId = self.data.session_id,\n                                );\n\n                                self.write(\n                                    b\"501 5.1.3 Bad destination mailbox address syntax.\\r\\n\",\n                                )\n                                .await?;\n                            }\n                            Error::SyntaxError { syntax } => {\n                                trc::event!(\n                                    Smtp(SmtpEvent::SyntaxError),\n                                    SpanId = self.data.session_id,\n                                    Details = syntax\n                                );\n\n                                if !self.params.ehlo_reject_non_fqdn && syntax.starts_with(\"EHLO \")\n                                {\n                                    self.handle_ehlo(\"null\".into(), true).await?\n                                } else {\n                                    self.write(\n                                        format!(\"501 5.5.2 Syntax error, expected: {syntax}\\r\\n\")\n                                            .as_bytes(),\n                                    )\n                                    .await?;\n                                }\n                            }\n                            Error::InvalidParameter { param } => {\n                                trc::event!(\n                                    Smtp(SmtpEvent::InvalidParameter),\n                                    SpanId = self.data.session_id,\n                                    Details = param\n                                );\n\n                                self.write(\n                                    format!(\"501 5.5.4 Invalid parameter {param:?}.\\r\\n\")\n                                        .as_bytes(),\n                                )\n                                .await?;\n                            }\n                            Error::UnsupportedParameter { param } => {\n                                trc::event!(\n                                    Smtp(SmtpEvent::UnsupportedParameter),\n                                    SpanId = self.data.session_id,\n                                    Details = param.clone()\n                                );\n\n                                self.write(\n                                    format!(\"504 5.5.4 Unsupported parameter {param:?}.\\r\\n\")\n                                        .as_bytes(),\n                                )\n                                .await?;\n                            }\n                            Error::ResponseTooLong => {\n                                state = State::RequestTooLarge(DummyLineReceiver::default());\n                                continue 'outer;\n                            }\n                        },\n                    }\n                },\n                State::Data(receiver) => {\n                    if self.data.message.len() + bytes.len() < self.params.max_message_size {\n                        if receiver.ingest(&mut iter, &mut self.data.message) {\n                            let message = self.queue_message().await;\n                            let num_responses = if self.instance.protocol == ServerProtocol::Smtp {\n                                1\n                            } else {\n                                self.data.rcpt_oks\n                            };\n                            if !message.is_empty() {\n                                for _ in 0..num_responses {\n                                    self.write(message.as_ref()).await?;\n                                }\n                                self.reset();\n                                state = State::default();\n                            } else {\n                                // Disconnect requested\n                                return Err(());\n                            }\n                        } else {\n                            break 'outer;\n                        }\n                    } else {\n                        state = State::DataTooLarge(DummyDataReceiver::new_data(receiver));\n                    }\n                }\n                State::Bdat(receiver) => {\n                    if receiver.ingest(&mut iter, &mut self.data.message) {\n                        if self.can_send_data().await? {\n                            if receiver.is_last {\n                                let message = self.queue_message().await;\n                                if !message.is_empty() {\n                                    let num_responses =\n                                        if self.instance.protocol == ServerProtocol::Smtp {\n                                            1\n                                        } else {\n                                            self.data.rcpt_oks\n                                        };\n                                    for _ in 0..num_responses {\n                                        self.write(message.as_ref()).await?;\n                                    }\n                                    self.reset();\n                                } else {\n                                    // Disconnect requested\n                                    return Err(());\n                                }\n                            } else {\n                                self.write(b\"250 2.6.0 Chunk accepted.\\r\\n\").await?;\n                            }\n                        } else {\n                            self.data.message = Vec::with_capacity(0);\n                        }\n                        state = State::default();\n                    } else {\n                        break 'outer;\n                    }\n                }\n                State::Sasl(receiver) => {\n                    if receiver.ingest(&mut iter) {\n                        if receiver.buf.len() < MAX_LINE_LENGTH {\n                            if self\n                                .handle_sasl_response(&mut receiver.state, &receiver.buf)\n                                .await?\n                            {\n                                receiver.buf.clear();\n                                continue 'outer;\n                            }\n                        } else {\n                            trc::event!(\n                                Smtp(SmtpEvent::AuthExchangeTooLong),\n                                SpanId = self.data.session_id,\n                                Limit = MAX_LINE_LENGTH,\n                            );\n\n                            self.auth_error(\n                                b\"500 5.5.6 Authentication Exchange line is too long.\\r\\n\",\n                            )\n                            .await?;\n                        }\n                        state = State::default();\n                    } else {\n                        break 'outer;\n                    }\n                }\n                State::DataTooLarge(receiver) => {\n                    if receiver.ingest(&mut iter) {\n                        trc::event!(\n                            Smtp(SmtpEvent::MessageTooLarge),\n                            SpanId = self.data.session_id,\n                        );\n\n                        self.data.message = Vec::with_capacity(0);\n                        self.write(b\"552 5.3.4 Message too big for system.\\r\\n\")\n                            .await?;\n                        state = State::default();\n                    } else {\n                        break 'outer;\n                    }\n                }\n                State::RequestTooLarge(receiver) => {\n                    if receiver.ingest(&mut iter) {\n                        trc::event!(\n                            Smtp(SmtpEvent::RequestTooLarge),\n                            SpanId = self.data.session_id,\n                        );\n\n                        self.write(b\"554 5.3.4 Line is too long.\\r\\n\").await?;\n                        state = State::default();\n                    } else {\n                        break 'outer;\n                    }\n                }\n                State::None | State::Accepted(_) => unreachable!(),\n            }\n        }\n        self.state = state;\n\n        Ok(true)\n    }\n}\n\nimpl<T: AsyncWrite + AsyncRead + Unpin> Session<T> {\n    pub fn reset(&mut self) {\n        self.data.mail_from = None;\n        self.data.spf_mail_from = None;\n        self.data.rcpt_to.clear();\n        self.data.message = Vec::with_capacity(0);\n        self.data.priority = 0;\n        self.data.delivery_by = 0;\n        self.data.future_release = 0;\n        self.data.rcpt_oks = 0;\n    }\n\n    #[inline(always)]\n    pub async fn write(&mut self, bytes: &[u8]) -> Result<(), ()> {\n        match self.stream.write_all(bytes).await {\n            Ok(_) => match self.stream.flush().await {\n                Ok(_) => {\n                    trc::event!(\n                        Smtp(SmtpEvent::RawOutput),\n                        SpanId = self.data.session_id,\n                        Size = bytes.len(),\n                        Contents = trc::Value::from_maybe_string(bytes),\n                    );\n\n                    Ok(())\n                }\n                Err(err) => {\n                    trc::event!(\n                        Network(NetworkEvent::FlushError),\n                        SpanId = self.data.session_id,\n                        Reason = err.to_string(),\n                    );\n                    Err(())\n                }\n            },\n            Err(err) => {\n                trc::event!(\n                    Network(NetworkEvent::WriteError),\n                    SpanId = self.data.session_id,\n                    Reason = err.to_string(),\n                );\n\n                Err(())\n            }\n        }\n    }\n\n    #[inline(always)]\n    pub async fn read(&mut self, bytes: &mut [u8]) -> Result<usize, ()> {\n        match self.stream.read(bytes).await {\n            Ok(len) => {\n                trc::event!(\n                    Smtp(SmtpEvent::RawInput),\n                    SpanId = self.data.session_id,\n                    Size = len,\n                    Contents =\n                        String::from_utf8_lossy(bytes.get(0..len).unwrap_or_default()).into_owned(),\n                );\n\n                Ok(len)\n            }\n            Err(err) => {\n                trc::event!(\n                    Network(NetworkEvent::ReadError),\n                    SpanId = self.data.session_id,\n                    Reason = err.to_string(),\n                );\n\n                Err(())\n            }\n        }\n    }\n}\n\nimpl<T: SessionStream> ResolveVariable for Session<T> {\n    fn resolve_variable(&self, variable: u32) -> expr::Variable<'_> {\n        match variable {\n            V_RECIPIENT => self\n                .data\n                .rcpt_to\n                .last()\n                .map(|r| r.address_lcase.as_str())\n                .unwrap_or_default()\n                .into(),\n            V_RECIPIENT_DOMAIN => self\n                .data\n                .rcpt_to\n                .last()\n                .map(|r| r.domain.as_str())\n                .unwrap_or_default()\n                .into(),\n            V_RECIPIENTS => self\n                .data\n                .rcpt_to\n                .iter()\n                .map(|r| Variable::from(r.address_lcase.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SENDER => self\n                .data\n                .mail_from\n                .as_ref()\n                .map(|m| m.address_lcase.as_str())\n                .unwrap_or_default()\n                .into(),\n            V_SENDER_DOMAIN => self\n                .data\n                .mail_from\n                .as_ref()\n                .map(|m| m.domain.as_str())\n                .unwrap_or_default()\n                .into(),\n            V_HELO_DOMAIN => self.data.helo_domain.as_str().into(),\n            V_AUTHENTICATED_AS => self.authenticated_as().unwrap_or_default().into(),\n            V_LISTENER => self.instance.id.as_str().into(),\n            V_REMOTE_IP => self.data.remote_ip_str.as_str().into(),\n            V_REMOTE_PORT => self.data.remote_port.into(),\n            V_LOCAL_IP => self.data.local_ip_str.as_str().into(),\n            V_LOCAL_PORT => self.data.local_port.into(),\n            V_TLS => self.stream.is_tls().into(),\n            V_PRIORITY => self.data.priority.to_compact_string().into(),\n            V_PROTOCOL => self.instance.protocol.as_str().into(),\n            V_ASN => self\n                .data\n                .asn_geo_data\n                .asn\n                .as_ref()\n                .map(|a| a.id)\n                .unwrap_or_default()\n                .into(),\n            V_COUNTRY => self\n                .data\n                .asn_geo_data\n                .country\n                .as_ref()\n                .map(|c| c.as_str())\n                .unwrap_or_default()\n                .into(),\n            _ => expr::Variable::default(),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/spam.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{config::spamfilter::SpamFilterAction, listener::SessionStream};\nuse mail_auth::{ArcOutput, DkimOutput, DmarcResult, dmarc::Policy};\nuse mail_parser::Message;\nuse spam_filter::{\n    SpamFilterInput,\n    analysis::{\n        init::SpamFilterInit,\n        score::{SpamFilterAnalyzeScore, SpamFilterScore},\n    },\n};\n\nuse crate::core::Session;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn spam_classify<'x>(\n        &'x self,\n        message: &'x Message<'x>,\n        dkim_result: &'x [DkimOutput<'x>],\n        arc_result: Option<&'x ArcOutput<'x>>,\n        dmarc_result: Option<&'x DmarcResult>,\n        dmarc_policy: Option<&'x Policy>,\n    ) -> SpamFilterAction<SpamFilterScore> {\n        let server = &self.server;\n        let mut ctx = server.spam_filter_init(self.build_spam_input(\n            message,\n            dkim_result,\n            arc_result,\n            dmarc_result,\n            dmarc_policy,\n        ));\n\n        if !self.is_authenticated() {\n            // Spam classification\n            server.spam_filter_classify(&mut ctx).await\n        } else {\n            // Do not classify authenticated sessions\n            SpamFilterAction::Disabled\n        }\n    }\n\n    pub fn build_spam_input<'x>(\n        &'x self,\n        message: &'x Message<'x>,\n        dkim_result: &'x [DkimOutput<'x>],\n        arc_result: Option<&'x ArcOutput>,\n        dmarc_result: Option<&'x DmarcResult>,\n        dmarc_policy: Option<&'x Policy>,\n    ) -> SpamFilterInput<'x> {\n        SpamFilterInput {\n            message,\n            span_id: self.data.session_id,\n            arc_result,\n            spf_ehlo_result: self.data.spf_ehlo.as_ref(),\n            spf_mail_from_result: self.data.spf_mail_from.as_ref(),\n            dkim_result,\n            dmarc_result,\n            dmarc_policy,\n            iprev_result: self.data.iprev.as_ref(),\n            remote_ip: self.data.remote_ip,\n            ehlo_domain: self.data.helo_domain.as_str().into(),\n            authenticated_as: self.data.authenticated_as.as_ref().map(|a| a.name.as_str()),\n            asn: self.data.asn_geo_data.asn.as_ref().map(|a| a.id),\n            country: self.data.asn_geo_data.country.as_ref().map(|c| c.as_str()),\n            is_tls: self.stream.is_tls(),\n            env_from: self\n                .data\n                .mail_from\n                .as_ref()\n                .map(|m| m.address_lcase.as_str())\n                .unwrap_or_default(),\n            env_from_flags: self\n                .data\n                .mail_from\n                .as_ref()\n                .map(|m| m.flags)\n                .unwrap_or_default(),\n            env_rcpt_to: self\n                .data\n                .rcpt_to\n                .iter()\n                .map(|r| r.address_lcase.as_str())\n                .collect(),\n            is_test: false,\n            is_train: false,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/spawn.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse common::{\n    config::smtp::session::Stage,\n    core::BuildServer,\n    listener::{self, SessionManager, SessionStream},\n};\n\nuse tokio_rustls::server::TlsStream;\nuse trc::{SecurityEvent, SmtpEvent};\n\nuse crate::{\n    core::{Session, SessionData, SessionParameters, SmtpSessionManager, State},\n    scripts::ScriptResult,\n};\n\nimpl SessionManager for SmtpSessionManager {\n    async fn handle<T: SessionStream>(self, session: listener::SessionData<T>) {\n        // Build server and create session\n        let server = self.inner.build_server();\n        let _in_flight = session.in_flight;\n        let mut session = Session {\n            data: SessionData::new(\n                session.local_ip,\n                session.local_port,\n                session.remote_ip,\n                session.remote_port,\n                server.lookup_asn_country(session.remote_ip).await,\n                session.session_id,\n            ),\n            hostname: \"\".into(),\n            server,\n            instance: session.instance,\n            state: State::default(),\n            stream: session.stream,\n            params: SessionParameters::default(),\n        };\n\n        // Enforce throttle\n        if session.is_allowed().await\n            && session.init_conn().await\n            && session.handle_conn().await\n            && session.instance.acceptor.is_tls()\n            && let Ok(mut session) = session.into_tls().await\n        {\n            session.handle_conn().await;\n        }\n    }\n\n    #[allow(clippy::manual_async_fn)]\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send {\n        async {\n            let _ = self\n                .inner\n                .ipc\n                .queue_tx\n                .send(common::ipc::QueueEvent::Stop)\n                .await;\n            let _ = self\n                .inner\n                .ipc\n                .report_tx\n                .send(common::ipc::ReportingEvent::Stop)\n                .await;\n        }\n    }\n}\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn init_conn(&mut self) -> bool {\n        self.eval_session_params().await;\n\n        let config = &self.server.core.smtp.session.connect;\n\n        // Sieve filtering\n        if let Some((script, script_id)) = self\n            .server\n            .eval_if::<String, _>(&config.script, self, self.data.session_id)\n            .await\n            .and_then(|name| {\n                self.server\n                    .get_trusted_sieve_script(&name, self.data.session_id)\n                    .map(|s| (s, name))\n            })\n            && let ScriptResult::Reject(message) = self\n                .run_script(\n                    script_id,\n                    script.clone(),\n                    self.build_script_parameters(\"connect\"),\n                )\n                .await\n        {\n            let _ = self.write(message.as_bytes()).await;\n            return false;\n        }\n\n        // Milter filtering\n        if let Err(message) = self.run_milters(Stage::Connect, None).await {\n            let _ = self.write(message.message.as_bytes()).await;\n            return false;\n        }\n\n        // MTAHook filtering\n        if let Err(message) = self.run_mta_hooks(Stage::Connect, None, None).await {\n            let _ = self.write(message.message.as_bytes()).await;\n            return false;\n        }\n\n        // Obtain hostname\n        self.hostname = self\n            .server\n            .eval_if::<String, _>(&config.hostname, self, self.data.session_id)\n            .await\n            .unwrap_or_default();\n        if self.hostname.is_empty() {\n            trc::event!(\n                Smtp(SmtpEvent::MissingLocalHostname),\n                SpanId = self.data.session_id,\n            );\n            self.hostname = \"localhost\".into();\n        }\n\n        // Obtain greeting\n        let greeting = self\n            .server\n            .eval_if::<String, _>(&config.greeting, self, self.data.session_id)\n            .await\n            .filter(|g| !g.is_empty())\n            .map(|g| format!(\"220 {}\\r\\n\", g))\n            .unwrap_or_else(|| \"220 Stalwart ESMTP at your service.\\r\\n\".to_string());\n\n        if self.write(greeting.as_bytes()).await.is_err() {\n            return false;\n        }\n\n        true\n    }\n\n    pub async fn handle_conn(&mut self) -> bool {\n        let mut buf = vec![0; 8192];\n        let mut shutdown_rx = self.instance.shutdown_rx.clone();\n\n        loop {\n            tokio::select! {\n                result = tokio::time::timeout(\n                    self.params.timeout,\n                    self.read(&mut buf)) => {\n                        match result {\n                            Ok(Ok(bytes_read)) => {\n                                if bytes_read > 0 {\n                                    if Instant::now() < self.data.valid_until && bytes_read <= self.data.bytes_left  {\n                                        self.data.bytes_left -= bytes_read;\n                                        match self.ingest(&buf[..bytes_read]).await {\n                                            Ok(true) => (),\n                                            Ok(false) => {\n                                                return true;\n                                            }\n                                            Err(_) => {\n                                                break;\n                                            }\n                                        }\n                                    } else if bytes_read > self.data.bytes_left {\n                                        self\n                                            .write(format!(\"452 4.7.28 {} Session exceeded transfer quota.\\r\\n\", self.hostname).as_bytes())\n                                            .await\n                                            .ok();\n\n                                        trc::event!(\n                                            Smtp(SmtpEvent::TransferLimitExceeded),\n                                            SpanId = self.data.session_id,\n                                        );\n\n                                        break;\n                                    } else {\n                                        self\n                                            .write(format!(\"421 4.3.2 {} Session open for too long.\\r\\n\", self.hostname).as_bytes())\n                                            .await\n                                            .ok();\n\n                                        match self.server.is_loiter_fail2banned(self.data.remote_ip)\n                                            .await\n                                        {\n                                            Ok(true) => {\n                                                trc::event!(\n                                                    Security(SecurityEvent::LoiterBan),\n                                                    SpanId = self.data.session_id,\n                                                    RemoteIp = self.data.remote_ip,\n                                                );\n                                            }\n                                            Ok(false) => {\n                                                trc::event!(\n                                                    Smtp(SmtpEvent::TimeLimitExceeded),\n                                                    SpanId = self.data.session_id,\n                                                );\n                                            }\n                                            Err(err) => {\n                                                trc::error!(err\n                                                    .span_id(self.data.session_id)\n                                                    .caused_by(trc::location!())\n                                                    .details(\"Failed to check if IP should be banned.\"));\n                                            }\n                                        }\n\n                                        break;\n                                    }\n                                } else {\n                                    trc::event!(\n                                        Network(trc::NetworkEvent::Closed),\n                                        SpanId = self.data.session_id,\n                                        CausedBy = trc::location!()\n                                    );\n\n                                    break;\n                                }\n                            }\n                            Ok(Err(_)) => {\n                                break;\n                            }\n                            Err(_) => {\n                                trc::event!(\n                                    Network(trc::NetworkEvent::Timeout),\n                                    SpanId = self.data.session_id,\n                                    CausedBy = trc::location!()\n                                );\n\n                                self\n                                    .write(format!(\"221 2.0.0 {} Disconnecting inactive client.\\r\\n\", self.hostname).as_bytes())\n                                    .await\n                                    .ok();\n                                break;\n                            }\n                        }\n                },\n                _ = shutdown_rx.changed() => {\n                    trc::event!(\n                        Network(trc::NetworkEvent::Closed),\n                        SpanId = self.data.session_id,\n                        Reason = \"Server shutting down\",\n                        CausedBy = trc::location!()\n                    );\n                    self.write(format!(\"421 4.3.0 {} Server shutting down.\\r\\n\", self.hostname).as_bytes()).await.ok();\n                    break;\n                }\n            };\n        }\n\n        false\n    }\n\n    pub async fn into_tls(self) -> Result<Session<TlsStream<T>>, ()> {\n        Ok(Session {\n            hostname: self.hostname,\n            stream: self\n                .instance\n                .tls_accept(self.stream, self.data.session_id)\n                .await?,\n            state: self.state,\n            data: self.data,\n            instance: self.instance,\n            server: self.server,\n            params: self.params,\n        })\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/inbound/vrfy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::Session;\nuse common::listener::SessionStream;\nuse std::{borrow::Cow, fmt::Write};\nuse trc::SmtpEvent;\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn handle_vrfy(&mut self, address: Cow<'_, str>) -> Result<(), ()> {\n        match self\n            .server\n            .eval_if::<String, _>(\n                &self.server.core.smtp.session.rcpt.directory,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .and_then(|name| self.server.get_directory(&name))\n        {\n            Some(directory) if self.params.can_vrfy => {\n                match self\n                    .server\n                    .vrfy(directory, &address.to_lowercase(), self.data.session_id)\n                    .await\n                {\n                    Ok(values) if !values.is_empty() => {\n                        let mut result = String::with_capacity(32);\n                        for (pos, value) in values.iter().enumerate() {\n                            let _ = write!(\n                                result,\n                                \"250{}{}\\r\\n\",\n                                if pos == values.len() - 1 { \" \" } else { \"-\" },\n                                value\n                            );\n                        }\n\n                        trc::event!(\n                            Smtp(SmtpEvent::Vrfy),\n                            SpanId = self.data.session_id,\n                            To = address.as_ref().to_string(),\n                            Result = values,\n                        );\n\n                        self.write(result.as_bytes()).await\n                    }\n                    Ok(_) => {\n                        trc::event!(\n                            Smtp(SmtpEvent::VrfyNotFound),\n                            SpanId = self.data.session_id,\n                            To = address.as_ref().to_string(),\n                        );\n\n                        self.write(b\"550 5.1.2 Address not found.\\r\\n\").await\n                    }\n                    Err(err) => {\n                        let is_not_supported =\n                            err.matches(trc::EventType::Store(trc::StoreEvent::NotSupported));\n\n                        trc::error!(err.span_id(self.data.session_id).details(\"VRFY failed\"));\n\n                        if !is_not_supported {\n                            self.write(b\"252 2.4.3 Unable to verify address at this time.\\r\\n\")\n                                .await\n                        } else {\n                            self.write(b\"550 5.1.2 Address not found.\\r\\n\").await\n                        }\n                    }\n                }\n            }\n            _ => {\n                trc::event!(\n                    Smtp(SmtpEvent::VrfyDisabled),\n                    SpanId = self.data.session_id,\n                    To = address.as_ref().to_string(),\n                );\n\n                self.write(b\"252 2.5.1 VRFY is disabled.\\r\\n\").await\n            }\n        }\n    }\n\n    pub async fn handle_expn(&mut self, address: Cow<'_, str>) -> Result<(), ()> {\n        match self\n            .server\n            .eval_if::<String, _>(\n                &self.server.core.smtp.session.rcpt.directory,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .and_then(|name| self.server.get_directory(&name))\n        {\n            Some(directory) if self.params.can_expn => {\n                match self\n                    .server\n                    .expn(directory, &address.to_lowercase(), self.data.session_id)\n                    .await\n                {\n                    Ok(values) if !values.is_empty() => {\n                        let mut result = String::with_capacity(32);\n                        for (pos, value) in values.iter().enumerate() {\n                            let _ = write!(\n                                result,\n                                \"250{}{}\\r\\n\",\n                                if pos == values.len() - 1 { \" \" } else { \"-\" },\n                                value\n                            );\n                        }\n\n                        trc::event!(\n                            Smtp(SmtpEvent::Expn),\n                            SpanId = self.data.session_id,\n                            To = address.as_ref().to_string(),\n                            Result = values,\n                        );\n\n                        self.write(result.as_bytes()).await\n                    }\n                    Ok(_) => {\n                        trc::event!(\n                            Smtp(SmtpEvent::ExpnNotFound),\n                            SpanId = self.data.session_id,\n                            To = address.as_ref().to_string(),\n                        );\n\n                        self.write(b\"550 5.1.2 Mailing list not found.\\r\\n\").await\n                    }\n                    Err(err) => {\n                        let is_not_supported =\n                            err.matches(trc::EventType::Store(trc::StoreEvent::NotSupported));\n\n                        trc::error!(err.span_id(self.data.session_id).details(\"VRFY failed\"));\n\n                        if !is_not_supported {\n                            self.write(b\"252 2.4.3 Unable to expand mailing list at this time.\\r\\n\")\n                                .await\n                        } else {\n                            self.write(b\"550 5.1.2 Mailing list not found.\\r\\n\").await\n                        }\n                    }\n                }\n            }\n            _ => {\n                trc::event!(\n                    Smtp(SmtpEvent::ExpnDisabled),\n                    SpanId = self.data.session_id,\n                    To = address.as_ref().to_string(),\n                );\n\n                self.write(b\"252 2.5.1 EXPN is disabled.\\r\\n\").await\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#![warn(clippy::large_futures)]\n\nuse common::{\n    Inner,\n    manager::boot::{BootManager, IpcReceivers},\n};\nuse queue::manager::SpawnQueue;\nuse reporting::scheduler::SpawnReport;\nuse std::sync::Arc;\n\npub mod core;\npub mod inbound;\npub mod outbound;\npub mod queue;\npub mod reporting;\npub mod scripts;\n\npub trait StartQueueManager {\n    fn start_queue_manager(&mut self);\n}\n\npub trait SpawnQueueManager {\n    fn spawn_queue_manager(&mut self, inner: Arc<Inner>);\n}\n\nimpl StartQueueManager for BootManager {\n    fn start_queue_manager(&mut self) {\n        self.ipc_rxs.spawn_queue_manager(self.inner.clone());\n    }\n}\n\nimpl SpawnQueueManager for IpcReceivers {\n    fn spawn_queue_manager(&mut self, inner: Arc<Inner>) {\n        // Spawn queue manager\n        self.queue_rx.take().unwrap().spawn(inner.clone());\n\n        // Spawn report manager\n        self.report_rx.take().unwrap().spawn(inner);\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/client.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::session::SessionParams;\nuse crate::queue::{Error, ErrorDetails, HostResponse, MessageWrapper, Status};\nuse mail_send::{Credentials, smtp::AssertReply};\nuse rustls::ClientConnection;\nuse rustls_pki_types::ServerName;\nuse smtp_proto::{\n    AUTH_CRAM_MD5, AUTH_DIGEST_MD5, AUTH_LOGIN, AUTH_OAUTHBEARER, AUTH_PLAIN, AUTH_XOAUTH2,\n    EXT_START_TLS, EhloResponse, Response,\n    response::{\n        generate::BitToString,\n        parser::{MAX_RESPONSE_LENGTH, ResponseReceiver},\n    },\n};\nuse std::{\n    net::{IpAddr, SocketAddr},\n    time::Duration,\n};\nuse tokio::{\n    io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt},\n    net::{TcpSocket, TcpStream},\n};\nuse tokio_rustls::{TlsConnector, client::TlsStream};\nuse trc::DeliveryEvent;\n\npub struct SmtpClient<T: AsyncRead + AsyncWrite> {\n    pub stream: T,\n    pub timeout: Duration,\n    pub session_id: u64,\n}\n\nimpl<T: AsyncRead + AsyncWrite + Unpin> SmtpClient<T> {\n    pub async fn authenticate<U>(\n        &mut self,\n        credentials: impl AsRef<Credentials<U>>,\n        capabilities: impl AsRef<EhloResponse<String>>,\n    ) -> mail_send::Result<&mut Self>\n    where\n        U: AsRef<str> + PartialEq + Eq + std::hash::Hash,\n    {\n        let credentials = credentials.as_ref();\n        let capabilities = capabilities.as_ref();\n        let mut available_mechanisms = match &credentials {\n            Credentials::Plain { .. } => AUTH_CRAM_MD5 | AUTH_DIGEST_MD5 | AUTH_LOGIN | AUTH_PLAIN,\n            Credentials::OAuthBearer { .. } => AUTH_OAUTHBEARER,\n            Credentials::XOauth2 { .. } => AUTH_XOAUTH2,\n        } & capabilities.auth_mechanisms;\n\n        // Try authenticating from most secure to least secure\n        let mut has_err = None;\n        let mut has_failed = false;\n\n        while available_mechanisms != 0 && !has_failed {\n            let mechanism = 1 << ((63 - available_mechanisms.leading_zeros()) as u64);\n            available_mechanisms ^= mechanism;\n            match self.auth(mechanism, credentials).await {\n                Ok(_) => {\n                    return Ok(self);\n                }\n                Err(err) => match err {\n                    mail_send::Error::UnexpectedReply(reply) => {\n                        has_failed = reply.code() == 535;\n                        has_err = reply.into();\n                    }\n                    mail_send::Error::UnsupportedAuthMechanism => (),\n                    _ => return Err(err),\n                },\n            }\n        }\n\n        if let Some(has_err) = has_err {\n            Err(mail_send::Error::AuthenticationFailed(has_err))\n        } else {\n            Err(mail_send::Error::UnsupportedAuthMechanism)\n        }\n    }\n\n    pub(crate) async fn auth<U>(\n        &mut self,\n        mechanism: u64,\n        credentials: &Credentials<U>,\n    ) -> mail_send::Result<()>\n    where\n        U: AsRef<str> + PartialEq + Eq + std::hash::Hash,\n    {\n        let mut reply = if (mechanism & (AUTH_PLAIN | AUTH_XOAUTH2 | AUTH_OAUTHBEARER)) != 0 {\n            self.cmd(\n                format!(\n                    \"AUTH {} {}\\r\\n\",\n                    mechanism.to_mechanism(),\n                    credentials.encode(mechanism, \"\")?,\n                )\n                .as_bytes(),\n            )\n            .await?\n        } else {\n            self.cmd(format!(\"AUTH {}\\r\\n\", mechanism.to_mechanism()).as_bytes())\n                .await?\n        };\n\n        for _ in 0..3 {\n            match reply.code() {\n                334 => {\n                    reply = self\n                        .cmd(\n                            format!(\"{}\\r\\n\", credentials.encode(mechanism, reply.message())?)\n                                .as_bytes(),\n                        )\n                        .await?;\n                }\n                235 => {\n                    return Ok(());\n                }\n                _ => {\n                    return Err(mail_send::Error::UnexpectedReply(reply));\n                }\n            }\n        }\n\n        Err(mail_send::Error::UnexpectedReply(reply))\n    }\n\n    pub async fn read_greeting(\n        &mut self,\n        hostname: &str,\n    ) -> Result<(), Status<HostResponse<Box<str>>, ErrorDetails>> {\n        tokio::time::timeout(self.timeout, self.read())\n            .await\n            .map_err(|_| Status::timeout(hostname, \"reading greeting\"))?\n            .and_then(|r| r.assert_code(220))\n            .map_err(|err| Status::from_smtp_error(hostname, \"\", err))\n    }\n\n    pub async fn read_smtp_data_response(\n        &mut self,\n        hostname: &str,\n        bdat_cmd: &Option<String>,\n    ) -> Result<Response<String>, Status<HostResponse<Box<str>>, ErrorDetails>> {\n        tokio::time::timeout(self.timeout, self.read())\n            .await\n            .map_err(|_| Status::timeout(hostname, \"reading SMTP DATA response\"))?\n            .map_err(|err| {\n                Status::from_smtp_error(hostname, bdat_cmd.as_deref().unwrap_or(\"DATA\"), err)\n            })\n    }\n\n    pub async fn read_lmtp_data_response(\n        &mut self,\n        hostname: &str,\n        num_responses: usize,\n    ) -> Result<Vec<Response<Box<str>>>, Status<HostResponse<Box<str>>, ErrorDetails>> {\n        tokio::time::timeout(self.timeout, async { self.read_many(num_responses).await })\n            .await\n            .map_err(|_| Status::timeout(hostname, \"reading LMTP DATA responses\"))?\n            .map_err(|err| Status::from_smtp_error(hostname, \"\", err))\n    }\n\n    pub async fn write_chunks(&mut self, chunks: &[&[u8]]) -> Result<(), mail_send::Error> {\n        for chunk in chunks {\n            self.stream\n                .write_all(chunk)\n                .await\n                .map_err(mail_send::Error::from)?;\n        }\n        self.stream.flush().await.map_err(mail_send::Error::from)\n    }\n\n    pub async fn send_message(\n        &mut self,\n        message: &MessageWrapper,\n        bdat_cmd: &Option<String>,\n        params: &SessionParams<'_>,\n    ) -> Result<(), Status<HostResponse<Box<str>>, ErrorDetails>> {\n        match params\n            .server\n            .blob_store()\n            .get_blob(message.message.blob_hash.as_slice(), 0..usize::MAX)\n            .await\n        {\n            Ok(Some(raw_message)) => {\n                tokio::time::timeout(params.conn_strategy.timeout_data, async {\n                    if let Some(bdat_cmd) = bdat_cmd {\n                        trc::event!(\n                            Delivery(DeliveryEvent::RawOutput),\n                            SpanId = self.session_id,\n                            Contents = bdat_cmd.clone(),\n                            Size = bdat_cmd.len()\n                        );\n\n                        self.write_chunks(&[bdat_cmd.as_bytes(), &raw_message])\n                            .await\n                    } else {\n                        trc::event!(\n                            Delivery(DeliveryEvent::RawOutput),\n                            SpanId = self.session_id,\n                            Contents = \"DATA\\r\\n\",\n                            Size = 6\n                        );\n\n                        self.write_chunks(&[b\"DATA\\r\\n\"]).await?;\n                        self.read().await?.assert_code(354)?;\n                        self.write_message(&raw_message)\n                            .await\n                            .map_err(mail_send::Error::from)\n                    }\n                })\n                .await\n                .map_err(|_| Status::timeout(params.hostname, \"sending message\"))?\n                .map_err(|err| {\n                    Status::from_smtp_error(\n                        params.hostname,\n                        bdat_cmd.as_deref().unwrap_or(\"DATA\"),\n                        err,\n                    )\n                })\n            }\n            Ok(None) => {\n                trc::event!(\n                    Queue(trc::QueueEvent::BlobNotFound),\n                    SpanId = message.span_id,\n                    BlobId = message.message.blob_hash.to_hex(),\n                    CausedBy = trc::location!()\n                );\n                Err(Status::TemporaryFailure(ErrorDetails {\n                    entity: \"localhost\".into(),\n                    details: Error::Io(\"Queue system error.\".into()),\n                }))\n            }\n            Err(err) => {\n                trc::error!(\n                    err.span_id(message.span_id)\n                        .details(\"Failed to fetch blobId\")\n                        .caused_by(trc::location!())\n                );\n\n                Err(Status::TemporaryFailure(ErrorDetails {\n                    entity: \"localhost\".into(),\n                    details: Error::Io(\"Queue system error.\".into()),\n                }))\n            }\n        }\n    }\n\n    pub async fn say_helo(\n        &mut self,\n        params: &SessionParams<'_>,\n    ) -> Result<EhloResponse<String>, Status<HostResponse<Box<str>>, ErrorDetails>> {\n        let cmd = if params.is_smtp {\n            format!(\"EHLO {}\\r\\n\", params.local_hostname)\n        } else {\n            format!(\"LHLO {}\\r\\n\", params.local_hostname)\n        };\n\n        trc::event!(\n            Delivery(DeliveryEvent::RawOutput),\n            SpanId = self.session_id,\n            Contents = cmd.clone(),\n            Size = cmd.len()\n        );\n\n        tokio::time::timeout(params.conn_strategy.timeout_ehlo, async {\n            self.stream.write_all(cmd.as_bytes()).await?;\n            self.stream.flush().await?;\n            self.read_ehlo().await\n        })\n        .await\n        .map_err(|_| Status::timeout(params.hostname, \"reading EHLO response\"))?\n        .map_err(|err| Status::from_smtp_error(params.hostname, &cmd, err))\n    }\n\n    pub async fn quit(mut self: SmtpClient<T>) {\n        trc::event!(\n            Delivery(DeliveryEvent::RawOutput),\n            SpanId = self.session_id,\n            Contents = \"QUIT\\r\\n\",\n            Size = 6\n        );\n\n        let _ = tokio::time::timeout(Duration::from_secs(10), async {\n            if self.stream.write_all(b\"QUIT\\r\\n\").await.is_ok() && self.stream.flush().await.is_ok()\n            {\n                let mut buf = [0u8; 128];\n                let _ = self.stream.read(&mut buf).await;\n            }\n        })\n        .await;\n    }\n\n    pub async fn read_ehlo(&mut self) -> mail_send::Result<EhloResponse<String>> {\n        let mut buf = vec![0u8; 8192];\n        let mut buf_concat = Vec::with_capacity(0);\n\n        loop {\n            let br = self.stream.read(&mut buf).await?;\n\n            if br == 0 {\n                return Err(mail_send::Error::UnparseableReply);\n            }\n\n            trc::event!(\n                Delivery(DeliveryEvent::RawInput),\n                SpanId = self.session_id,\n                Contents = trc::Value::from_maybe_string(&buf[..br]),\n                Size = br,\n            );\n\n            let mut iter = if buf_concat.is_empty() {\n                buf[..br].iter()\n            } else if br + buf_concat.len() < MAX_RESPONSE_LENGTH {\n                buf_concat.extend_from_slice(&buf[..br]);\n                buf_concat.iter()\n            } else {\n                return Err(mail_send::Error::UnparseableReply);\n            };\n\n            match EhloResponse::parse(&mut iter) {\n                Ok(reply) => return Ok(reply),\n                Err(err) => match err {\n                    smtp_proto::Error::NeedsMoreData { .. } => {\n                        if buf_concat.is_empty() {\n                            buf_concat = buf[..br].to_vec();\n                        }\n                    }\n                    smtp_proto::Error::InvalidResponse { code } => {\n                        match ResponseReceiver::from_code(code).parse(&mut iter) {\n                            Ok(response) => {\n                                return Err(mail_send::Error::UnexpectedReply(response));\n                            }\n                            Err(smtp_proto::Error::NeedsMoreData { .. }) => {\n                                if buf_concat.is_empty() {\n                                    buf_concat = buf[..br].to_vec();\n                                }\n                            }\n                            Err(_) => return Err(mail_send::Error::UnparseableReply),\n                        }\n                    }\n                    _ => {\n                        return Err(mail_send::Error::UnparseableReply);\n                    }\n                },\n            }\n        }\n    }\n\n    pub async fn read(&mut self) -> mail_send::Result<Response<String>> {\n        let mut buf = vec![0u8; 8192];\n        let mut parser = ResponseReceiver::default();\n\n        loop {\n            let br = self.stream.read(&mut buf).await?;\n\n            if br > 0 {\n                trc::event!(\n                    Delivery(DeliveryEvent::RawInput),\n                    SpanId = self.session_id,\n                    Contents = trc::Value::from_maybe_string(&buf[..br]),\n                    Size = br\n                );\n\n                match parser.parse(&mut buf[..br].iter()) {\n                    Ok(reply) => return Ok(reply),\n                    Err(err) => match err {\n                        smtp_proto::Error::NeedsMoreData { .. } => (),\n                        _ => {\n                            return Err(mail_send::Error::UnparseableReply);\n                        }\n                    },\n                }\n            } else {\n                return Err(mail_send::Error::UnparseableReply);\n            }\n        }\n    }\n\n    pub async fn read_many(&mut self, num: usize) -> mail_send::Result<Vec<Response<Box<str>>>> {\n        let mut buf = vec![0u8; 1024];\n        let mut response = Vec::with_capacity(num);\n        let mut parser = ResponseReceiver::default();\n\n        'outer: loop {\n            let br = self.stream.read(&mut buf).await?;\n\n            if br > 0 {\n                let mut iter = buf[..br].iter();\n\n                trc::event!(\n                    Delivery(DeliveryEvent::RawInput),\n                    SpanId = self.session_id,\n                    Contents = trc::Value::from_maybe_string(&buf[..br]),\n                    Size = br\n                );\n\n                loop {\n                    match parser.parse(&mut iter) {\n                        Ok(reply) => {\n                            response.push(reply.into_box());\n                            if response.len() != num {\n                                parser.reset();\n                            } else {\n                                break 'outer;\n                            }\n                        }\n                        Err(err) => match err {\n                            smtp_proto::Error::NeedsMoreData { .. } => break,\n                            _ => {\n                                return Err(mail_send::Error::UnparseableReply);\n                            }\n                        },\n                    }\n                }\n            } else {\n                return Err(mail_send::Error::UnparseableReply);\n            }\n        }\n\n        Ok(response)\n    }\n\n    /// Sends a command to the SMTP server and waits for a reply.\n    pub async fn cmd(&mut self, cmd: impl AsRef<[u8]>) -> mail_send::Result<Response<String>> {\n        tokio::time::timeout(self.timeout, async {\n            let cmd = cmd.as_ref();\n\n            trc::event!(\n                Delivery(DeliveryEvent::RawOutput),\n                SpanId = self.session_id,\n                Contents = trc::Value::from_maybe_string(cmd),\n                Size = cmd.len()\n            );\n\n            self.stream.write_all(cmd).await?;\n            self.stream.flush().await?;\n            self.read().await\n        })\n        .await\n        .map_err(|_| mail_send::Error::Timeout)?\n    }\n\n    pub async fn write_message(&mut self, message: &[u8]) -> tokio::io::Result<()> {\n        // Transparency procedure\n        let mut is_cr_or_lf = false;\n\n        // As per RFC 5322bis, section 2.3:\n        // CR and LF MUST only occur together as CRLF; they MUST NOT appear\n        // independently in the body.\n        // For this reason, we apply the transparency procedure when there is\n        // a CR or LF followed by a dot.\n\n        trc::event!(\n            Delivery(DeliveryEvent::RawOutput),\n            SpanId = self.session_id,\n            Contents = \"[message]\",\n            Size = message.len() + 5\n        );\n\n        let mut last_pos = 0;\n        for (pos, byte) in message.iter().enumerate() {\n            if *byte == b'.' && is_cr_or_lf {\n                if let Some(bytes) = message.get(last_pos..pos) {\n                    self.stream.write_all(bytes).await?;\n                    self.stream.write_all(b\".\").await?;\n                    last_pos = pos;\n                }\n                is_cr_or_lf = false;\n            } else {\n                is_cr_or_lf = *byte == b'\\n' || *byte == b'\\r';\n            }\n        }\n        if let Some(bytes) = message.get(last_pos..) {\n            self.stream.write_all(bytes).await?;\n        }\n        self.stream.write_all(\"\\r\\n.\\r\\n\".as_bytes()).await?;\n        self.stream.flush().await\n    }\n}\n\nimpl SmtpClient<TcpStream> {\n    /// Upgrade the connection to TLS.\n    pub async fn start_tls(\n        mut self,\n        tls_connector: &TlsConnector,\n        hostname: &str,\n    ) -> mail_send::Result<SmtpClient<TlsStream<TcpStream>>> {\n        // Send STARTTLS command\n        self.cmd(b\"STARTTLS\\r\\n\")\n            .await?\n            .assert_positive_completion()?;\n\n        self.into_tls(tls_connector, hostname).await\n    }\n\n    pub async fn into_tls(\n        self,\n        tls_connector: &TlsConnector,\n        hostname: &str,\n    ) -> mail_send::Result<SmtpClient<TlsStream<TcpStream>>> {\n        tokio::time::timeout(self.timeout, async {\n            Ok(SmtpClient {\n                stream: tls_connector\n                    .connect(\n                        ServerName::try_from(hostname)\n                            .map_err(|_| mail_send::Error::InvalidTLSName)?\n                            .to_owned(),\n                        self.stream,\n                    )\n                    .await\n                    .map_err(|err| {\n                        let kind = err.kind();\n                        if let Some(inner) = err.into_inner() {\n                            match inner.downcast::<rustls::Error>() {\n                                Ok(error) => mail_send::Error::Tls(error),\n                                Err(error) => {\n                                    mail_send::Error::Io(std::io::Error::new(kind, error))\n                                }\n                            }\n                        } else {\n                            mail_send::Error::Io(std::io::Error::new(kind, \"Unspecified\"))\n                        }\n                    })?,\n                timeout: self.timeout,\n                session_id: self.session_id,\n            })\n        })\n        .await\n        .map_err(|_| mail_send::Error::Timeout)?\n    }\n}\n\nimpl SmtpClient<TcpStream> {\n    /// Connects to a remote host address\n    pub async fn connect(\n        remote_addr: SocketAddr,\n        timeout: Duration,\n        session_id: u64,\n    ) -> mail_send::Result<Self> {\n        tokio::time::timeout(timeout, async {\n            Ok(SmtpClient {\n                stream: TcpStream::connect(remote_addr).await?,\n                timeout,\n                session_id,\n            })\n        })\n        .await\n        .map_err(|_| mail_send::Error::Timeout)?\n    }\n\n    /// Connects to a remote host address using the provided local IP\n    pub async fn connect_using(\n        local_ip: IpAddr,\n        remote_addr: SocketAddr,\n        timeout: Duration,\n        session_id: u64,\n    ) -> mail_send::Result<Self> {\n        tokio::time::timeout(timeout, async {\n            let socket = if local_ip.is_ipv4() {\n                TcpSocket::new_v4()?\n            } else {\n                TcpSocket::new_v6()?\n            };\n            socket.bind(SocketAddr::new(local_ip, 0))?;\n\n            Ok(SmtpClient {\n                stream: socket.connect(remote_addr).await?,\n                timeout,\n                session_id,\n            })\n        })\n        .await\n        .map_err(|_| mail_send::Error::Timeout)?\n    }\n\n    pub async fn try_start_tls(\n        mut self,\n        tls_connector: &TlsConnector,\n        hostname: &str,\n        capabilities: &EhloResponse<String>,\n    ) -> StartTlsResult {\n        if capabilities.has_capability(EXT_START_TLS) {\n            match self.cmd(\"STARTTLS\\r\\n\").await {\n                Ok(response) => {\n                    if response.code() == 220 {\n                        match self.into_tls(tls_connector, hostname).await {\n                            Ok(smtp_client) => StartTlsResult::Success { smtp_client },\n                            Err(error) => StartTlsResult::Error { error },\n                        }\n                    } else {\n                        StartTlsResult::Unavailable {\n                            response: response.into_box().into(),\n                            smtp_client: self,\n                        }\n                    }\n                }\n                Err(error) => StartTlsResult::Error { error },\n            }\n        } else {\n            StartTlsResult::Unavailable {\n                smtp_client: self,\n                response: None,\n            }\n        }\n    }\n}\n\nimpl SmtpClient<TlsStream<TcpStream>> {\n    pub fn tls_connection(&self) -> &ClientConnection {\n        self.stream.get_ref().1\n    }\n}\n\n#[allow(clippy::large_enum_variant)]\npub enum StartTlsResult {\n    Success {\n        smtp_client: SmtpClient<TlsStream<TcpStream>>,\n    },\n    Error {\n        error: mail_send::Error,\n    },\n    Unavailable {\n        response: Option<Response<Box<str>>>,\n        smtp_client: SmtpClient<TcpStream>,\n    },\n}\n\npub(crate) trait BoxResponse {\n    fn into_box(self) -> Response<Box<str>>;\n}\n\nimpl BoxResponse for Response<String> {\n    fn into_box(self) -> Response<Box<str>> {\n        Response {\n            code: self.code,\n            esc: self.esc,\n            message: self.message.into_boxed_str(),\n        }\n    }\n}\n\npub(crate) fn from_mail_send_error(error: &mail_send::Error) -> trc::Error {\n    let event = trc::EventType::Smtp(trc::SmtpEvent::Error).into_err();\n    match error {\n        mail_send::Error::Io(err) => event.details(\"I/O Error\").reason(err),\n        mail_send::Error::Tls(err) => event.details(\"TLS Error\").reason(err),\n        mail_send::Error::Base64(err) => event.details(\"Base64 Error\").reason(err),\n        mail_send::Error::Auth(err) => event.details(\"SMTP Authentication Error\").reason(err),\n        mail_send::Error::UnparseableReply => event.details(\"Unparseable SMTP Reply\"),\n        mail_send::Error::UnexpectedReply(reply) => event\n            .details(\"Unexpected SMTP Response\")\n            .ctx(trc::Key::Code, reply.code)\n            .ctx(trc::Key::Reason, reply.message.clone()),\n        mail_send::Error::AuthenticationFailed(reply) => event\n            .details(\"SMTP Authentication Failed\")\n            .ctx(trc::Key::Code, reply.code)\n            .ctx(trc::Key::Reason, reply.message.clone()),\n        mail_send::Error::InvalidTLSName => event.details(\"Invalid TLS Name\"),\n        mail_send::Error::MissingCredentials => event.details(\"Missing Authentication Credentials\"),\n        mail_send::Error::MissingMailFrom => event.details(\"Missing Message Sender\"),\n        mail_send::Error::MissingRcptTo => event.details(\"Missing Message Recipients\"),\n        mail_send::Error::UnsupportedAuthMechanism => {\n            event.details(\"Unsupported Authentication Mechanism\")\n        }\n        mail_send::Error::Timeout => event.details(\"Connection Timeout\"),\n        mail_send::Error::MissingStartTls => event.details(\"STARTTLS not available\"),\n    }\n}\n\npub(crate) fn from_error_status(err: &Status<HostResponse<Box<str>>, ErrorDetails>) -> trc::Error {\n    match err {\n        Status::Scheduled | Status::Completed(_) => {\n            trc::EventType::Smtp(trc::SmtpEvent::Error).into_err()\n        }\n        Status::TemporaryFailure(err) | Status::PermanentFailure(err) => {\n            from_error_details(&err.details)\n        }\n    }\n}\n\npub(crate) fn from_error_details(err: &Error) -> trc::Error {\n    let event = trc::EventType::Smtp(trc::SmtpEvent::Error).into_err();\n    match err {\n        Error::DnsError(err) => event.details(\"DNS Error\").reason(err),\n        Error::UnexpectedResponse(reply) => event\n            .details(\"Unexpected SMTP Response\")\n            .ctx(trc::Key::Code, reply.response.code)\n            .ctx(trc::Key::Details, reply.command.clone())\n            .ctx(trc::Key::Reason, reply.response.message.clone()),\n        Error::ConnectionError(err) => event\n            .details(\"Connection Error\")\n            .ctx(trc::Key::Reason, err.clone()),\n        Error::TlsError(err) => event\n            .details(\"TLS Error\")\n            .ctx(trc::Key::Reason, err.clone()),\n        Error::DaneError(err) => event\n            .details(\"DANE Error\")\n            .ctx(trc::Key::Reason, err.clone()),\n        Error::MtaStsError(err) => event.details(\"MTA-STS Error\").reason(err),\n        Error::RateLimited => event.details(\"Rate Limited\"),\n        Error::ConcurrencyLimited => event.details(\"Concurrency Limited\"),\n        Error::Io(err) => event.details(\"I/O Error\").reason(err),\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/dane/dnssec.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Server,\n    config::smtp::resolver::{Tlsa, TlsaEntry},\n};\nuse mail_auth::{\n    common::resolver::IntoFqdn,\n    hickory_resolver::{\n        Name,\n        proto::rr::rdata::tlsa::{CertUsage, Matching, Selector},\n    },\n};\nuse std::{future::Future, sync::Arc};\n\npub trait TlsaLookup: Sync + Send {\n    fn tlsa_lookup<'x>(\n        &self,\n        key: impl IntoFqdn<'x> + Sync + Send,\n    ) -> impl Future<Output = mail_auth::Result<Option<Arc<Tlsa>>>> + Send;\n}\n\nimpl TlsaLookup for Server {\n    async fn tlsa_lookup<'x>(\n        &self,\n        key: impl IntoFqdn<'x> + Sync + Send,\n    ) -> mail_auth::Result<Option<Arc<Tlsa>>> {\n        let key = key.into_fqdn();\n        if let Some(value) = self.inner.cache.dns_tlsa.get(key.as_ref()) {\n            return Ok(Some(value));\n        }\n\n        #[cfg(any(test, feature = \"test_mode\"))]\n        if true {\n            return mail_auth::common::resolver::mock_resolve(key.as_ref());\n        }\n\n        let mut entries = Vec::new();\n        let tlsa_lookup = self\n            .core\n            .smtp\n            .resolvers\n            .dnssec\n            .resolver\n            .tlsa_lookup(Name::from_str_relaxed(key.as_ref())?)\n            .await?;\n\n        let mut has_end_entities = false;\n        let mut has_intermediates = false;\n        let mut found_insecure = false;\n\n        for record in tlsa_lookup.as_lookup().record_iter() {\n            if let Some(tlsa) = record.data().as_tlsa() {\n                if record.proof().is_secure() {\n                    let is_end_entity = match tlsa.cert_usage() {\n                        CertUsage::DaneEe => true,\n                        CertUsage::DaneTa => false,\n                        _ => continue,\n                    };\n                    if is_end_entity {\n                        has_end_entities = true;\n                    } else {\n                        has_intermediates = true;\n                    }\n                    entries.push(TlsaEntry {\n                        is_end_entity,\n                        is_sha256: match tlsa.matching() {\n                            Matching::Sha256 => true,\n                            Matching::Sha512 => false,\n                            _ => continue,\n                        },\n                        is_spki: match tlsa.selector() {\n                            Selector::Spki => true,\n                            Selector::Full => false,\n                            _ => continue,\n                        },\n                        data: tlsa.cert_data().to_vec(),\n                    });\n                } else {\n                    found_insecure = true;\n                }\n            }\n        }\n\n        if !entries.is_empty() || !found_insecure {\n            let tlsa = Arc::new(Tlsa {\n                entries,\n                has_end_entities,\n                has_intermediates,\n            });\n\n            self.inner.cache.dns_tlsa.insert_with_expiry(\n                key.into_owned(),\n                tlsa.clone(),\n                tlsa_lookup.valid_until(),\n            );\n\n            Ok(Some(tlsa))\n        } else {\n            Ok(None)\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/dane/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod dnssec;\npub mod verify;\n"
  },
  {
    "path": "crates/smtp/src/outbound/dane/verify.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::config::smtp::resolver::Tlsa;\nuse rustls_pki_types::CertificateDer;\nuse sha1::Digest;\nuse sha2::{Sha256, Sha512};\nuse trc::DaneEvent;\nuse x509_parser::prelude::{FromDer, X509Certificate};\n\nuse crate::queue::{Error, ErrorDetails, HostResponse, Status};\n\npub trait TlsaVerify {\n    fn verify(\n        &self,\n        session_id: u64,\n        hostname: &str,\n        certificates: Option<&[CertificateDer<'_>]>,\n    ) -> Result<(), Status<HostResponse<Box<str>>, ErrorDetails>>;\n}\n\nimpl TlsaVerify for Tlsa {\n    fn verify(\n        &self,\n        session_id: u64,\n        hostname: &str,\n        certificates: Option<&[CertificateDer<'_>]>,\n    ) -> Result<(), Status<HostResponse<Box<str>>, ErrorDetails>> {\n        let certificates = if let Some(certificates) = certificates {\n            certificates\n        } else {\n            trc::event!(\n                Dane(DaneEvent::NoCertificatesFound),\n                SpanId = session_id,\n                Hostname = hostname.to_string(),\n            );\n\n            return Err(Status::TemporaryFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::DaneError(\"No certificates were provided by host\".into()),\n            }));\n        };\n\n        let mut matched_end_entity = false;\n        let mut matched_intermediate = false;\n        'outer: for (pos, der_certificate) in certificates.iter().enumerate() {\n            // Parse certificate\n            let certificate = match X509Certificate::from_der(der_certificate.as_ref()) {\n                Ok((_, certificate)) => certificate,\n                Err(err) => {\n                    trc::event!(\n                        Dane(DaneEvent::CertificateParseError),\n                        SpanId = session_id,\n                        Hostname = hostname.to_string(),\n                        Reason = err.to_string(),\n                    );\n\n                    return Err(Status::TemporaryFailure(ErrorDetails {\n                        entity: hostname.into(),\n                        details: Error::DaneError(\"Failed to parse X.509 certificate\".into()),\n                    }));\n                }\n            };\n\n            // Match against TLSA records\n            let is_end_entity = pos == 0;\n            let mut sha256 = [None, None];\n            let mut sha512 = [None, None];\n            for record in self.entries.iter() {\n                if record.is_end_entity == is_end_entity {\n                    let hash: &[u8] = if record.is_sha256 {\n                        &sha256[usize::from(record.is_spki)].get_or_insert_with(|| {\n                            let mut hasher = Sha256::new();\n                            hasher.update(if record.is_spki {\n                                certificate.public_key().raw\n                            } else {\n                                der_certificate.as_ref()\n                            });\n                            hasher.finalize()\n                        })[..]\n                    } else {\n                        &sha512[usize::from(record.is_spki)].get_or_insert_with(|| {\n                            let mut hasher = Sha512::new();\n                            hasher.update(if record.is_spki {\n                                certificate.public_key().raw\n                            } else {\n                                der_certificate.as_ref()\n                            });\n                            hasher.finalize()\n                        })[..]\n                    };\n\n                    if hash == record.data {\n                        trc::event!(\n                            Dane(DaneEvent::TlsaRecordMatch),\n                            SpanId = session_id,\n                            Hostname = hostname.to_string(),\n                            Type = if is_end_entity {\n                                \"end-entity\"\n                            } else {\n                                \"intermediate\"\n                            },\n                            Details = format!(\"{:x?}\", hash),\n                        );\n\n                        if is_end_entity {\n                            matched_end_entity = true;\n                            if !self.has_intermediates {\n                                break 'outer;\n                            }\n                        } else {\n                            matched_intermediate = true;\n                            break 'outer;\n                        }\n                    }\n                }\n            }\n        }\n\n        // DANE is valid if:\n        // - EE matched even if no TA matched\n        // - Both EE and TA matched\n        // - EE is not present and TA matched\n        if (self.has_end_entities && matched_end_entity)\n            || ((self.has_end_entities == matched_end_entity)\n                && (self.has_intermediates == matched_intermediate))\n        {\n            trc::event!(\n                Dane(DaneEvent::AuthenticationSuccess),\n                SpanId = session_id,\n                Hostname = hostname.to_string(),\n            );\n\n            Ok(())\n        } else {\n            trc::event!(\n                Dane(DaneEvent::AuthenticationFailure),\n                SpanId = session_id,\n                Hostname = hostname.to_string(),\n            );\n\n            Err(Status::PermanentFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::DaneError(\"No matching certificates found in TLSA records\".into()),\n            }))\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/delivery.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{NextHop, lookup::ToNextHop, mta_sts, session::SessionParams};\nuse crate::outbound::DeliveryResult;\nuse crate::outbound::client::{\n    SmtpClient, from_error_details, from_error_status, from_mail_send_error,\n};\nuse crate::outbound::dane::dnssec::TlsaLookup;\nuse crate::outbound::lookup::{DnsLookup, SourceIp};\nuse crate::outbound::mta_sts::lookup::MtaStsLookup;\nuse crate::outbound::mta_sts::verify::VerifyPolicy;\nuse crate::outbound::{client::StartTlsResult, dane::verify::TlsaVerify};\nuse crate::queue::dsn::SendDsn;\nuse crate::queue::spool::SmtpSpool;\nuse crate::queue::throttle::IsAllowed;\nuse crate::queue::{\n    Error, FROM_REPORT, HostResponse, MessageWrapper, QueueEnvelope, QueuedMessage, Status,\n};\nuse crate::reporting::SmtpReporting;\nuse crate::{queue::ErrorDetails, reporting::tls::TlsRptOptions};\nuse ahash::AHashMap;\nuse common::Server;\nuse common::config::smtp::queue::RoutingStrategy;\nuse common::config::{server::ServerProtocol, smtp::report::AggregateFrequency};\nuse common::ipc::{PolicyType, QueueEvent, QueueEventStatus, TlsEvent};\nuse compact_str::ToCompactString;\nuse mail_auth::{\n    mta_sts::TlsRpt,\n    report::tlsrpt::{FailureDetails, ResultType},\n};\nuse smtp_proto::MAIL_REQUIRETLS;\nuse std::sync::Arc;\nuse std::{\n    net::{IpAddr, Ipv4Addr, SocketAddr},\n    time::Instant,\n};\nuse store::write::{BatchBuilder, QueueClass, ValueClass, now};\nuse trc::{DaneEvent, DeliveryEvent, MtaStsEvent, ServerEvent, TlsRptEvent};\n\nimpl QueuedMessage {\n    pub fn try_deliver(self, server: Server) {\n        #![allow(clippy::large_futures)]\n        tokio::spawn(async move {\n            // Lock queue event\n            let queue_id = self.queue_id;\n            let status = if server.try_lock_event(queue_id, self.queue_name).await {\n                if let Some(mut message) = server.read_message(queue_id, self.queue_name).await {\n                    // Generate span id\n                    message.span_id = server.inner.data.span_id_gen.generate();\n                    let span_id = message.span_id;\n\n                    trc::event!(\n                        Delivery(DeliveryEvent::AttemptStart),\n                        SpanId = message.span_id,\n                        QueueId = message.queue_id,\n                        QueueName = message.queue_name.to_string(),\n                        From = if !message.message.return_path.is_empty() {\n                            trc::Value::String(message.message.return_path.as_ref().into())\n                        } else {\n                            trc::Value::String(\"<>\".into())\n                        },\n                        To = message\n                            .message\n                            .recipients\n                            .iter()\n                            .filter_map(|r| {\n                                if matches!(\n                                    r.status,\n                                    Status::Scheduled | Status::TemporaryFailure(_)\n                                ) && r.queue == message.queue_name\n                                {\n                                    Some(trc::Value::String(r.address().into()))\n                                } else {\n                                    None\n                                }\n                            })\n                            .collect::<Vec<_>>(),\n                        Size = message.message.size,\n                        Total = message.message.recipients.len(),\n                    );\n\n                    // Attempt delivery\n                    let start_time = Instant::now();\n                    let queue_event = self.deliver_task(server.clone(), message).await;\n\n                    trc::event!(\n                        Delivery(DeliveryEvent::AttemptEnd),\n                        SpanId = span_id,\n                        Elapsed = start_time.elapsed(),\n                    );\n\n                    // Unlock event\n                    server.unlock_event(queue_id, self.queue_name).await;\n\n                    queue_event\n                } else {\n                    // Message no longer exists, delete queue event.\n                    let mut batch = BatchBuilder::new();\n                    batch.clear(ValueClass::Queue(QueueClass::MessageEvent(\n                        store::write::QueueEvent {\n                            due: self.due,\n                            queue_id: self.queue_id,\n                            queue_name: self.queue_name.into_inner(),\n                        },\n                    )));\n\n                    if let Err(err) = server.store().write(batch.build_all()).await {\n                        trc::error!(\n                            err.details(\"Failed to delete queue event.\")\n                                .caused_by(trc::location!())\n                        );\n                    }\n\n                    // Unlock event\n                    server.unlock_event(queue_id, self.queue_name).await;\n\n                    QueueEventStatus::Completed\n                }\n            } else {\n                QueueEventStatus::Locked\n            };\n\n            // Notify queue manager\n            if server\n                .inner\n                .ipc\n                .queue_tx\n                .send(QueueEvent::WorkerDone {\n                    queue_id,\n                    queue_name: self.queue_name,\n                    status,\n                })\n                .await\n                .is_err()\n            {\n                trc::event!(\n                    Server(ServerEvent::ThreadError),\n                    Reason = \"Channel closed.\",\n                    CausedBy = trc::location!(),\n                );\n            }\n        });\n    }\n\n    async fn deliver_task(self, server: Server, mut message: MessageWrapper) -> QueueEventStatus {\n        // Check that the message still has recipients to be delivered\n        let has_pending_delivery = message.has_pending_delivery();\n        let span_id = message.span_id;\n\n        // Send any due Delivery Status Notifications\n        server.send_dsn(&mut message).await;\n\n        match has_pending_delivery {\n            PendingDelivery::Yes(true)\n                if message\n                    .message\n                    .next_delivery_event(self.queue_name.into())\n                    .is_some_and(|due| due <= now()) => {}\n            PendingDelivery::No => {\n                trc::event!(\n                    Delivery(DeliveryEvent::Completed),\n                    SpanId = span_id,\n                    Elapsed = trc::Value::Duration((now() - message.message.created) * 1000)\n                );\n\n                // All message recipients expired, do not re-queue. (DSN has been already sent)\n                message.remove(&server, self.due.into()).await;\n\n                return QueueEventStatus::Completed;\n            }\n            _ => {\n                // Re-queue the message if its not yet due for delivery\n                message.save_changes(&server, self.due.into()).await;\n                return QueueEventStatus::Deferred;\n            }\n        }\n\n        // Throttle sender\n        for throttle in &server.core.smtp.queue.outbound_limiters.sender {\n            if let Err(retry_at) = server\n                .is_allowed(throttle, &message.message, message.span_id)\n                .await\n            {\n                trc::event!(\n                    Delivery(DeliveryEvent::RateLimitExceeded),\n                    Id = throttle.id.clone(),\n                    SpanId = span_id,\n                    NextRetry = trc::Value::Timestamp(retry_at)\n                );\n\n                let now = now();\n                for rcpt in message.message.recipients.iter_mut() {\n                    if matches!(\n                        &rcpt.status,\n                        Status::Scheduled | Status::TemporaryFailure(_)\n                    ) && rcpt.retry.due <= now\n                        && rcpt.queue == message.queue_name\n                    {\n                        rcpt.retry.due = retry_at;\n                        rcpt.status = Status::TemporaryFailure(ErrorDetails {\n                            entity: \"localhost\".into(),\n                            details: Error::RateLimited,\n                        });\n                    }\n                }\n\n                message.save_changes(&server, self.due.into()).await;\n\n                return QueueEventStatus::Deferred;\n            }\n        }\n\n        // Group recipients by route\n        let queue_config = &server.core.smtp.queue;\n        let now_ = now();\n        let mut routes: AHashMap<(&str, &RoutingStrategy), Vec<usize>> = AHashMap::new();\n        for (rcpt_idx, rcpt) in message.message.recipients.iter().enumerate() {\n            if matches!(\n                &rcpt.status,\n                Status::Scheduled | Status::TemporaryFailure(_)\n            ) && rcpt.retry.due <= now_\n                && rcpt.queue == message.queue_name\n            {\n                let envelope = QueueEnvelope::new(&message.message, rcpt);\n                let route = server.get_route_or_default(\n                    &server\n                        .eval_if::<String, _>(&queue_config.route, &envelope, message.span_id)\n                        .await\n                        .unwrap_or_else(|| \"default\".to_string()),\n                    message.span_id,\n                );\n\n                routes\n                    .entry((rcpt.domain_part(), route))\n                    .or_default()\n                    .push(rcpt_idx);\n            }\n        }\n\n        let no_ip = IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0));\n        let mut delivery_results: Vec<DeliveryResult> = Vec::new();\n        'next_route: for ((domain, route), rcpt_idxs) in routes {\n            trc::event!(\n                Delivery(DeliveryEvent::DomainDeliveryStart),\n                SpanId = message.span_id,\n                Domain = domain.to_string(),\n            );\n\n            // Build envelope\n            let mut envelope =\n                QueueEnvelope::new(&message.message, &message.message.recipients[rcpt_idxs[0]]);\n\n            // Throttle recipient domain\n            for throttle in &queue_config.outbound_limiters.rcpt {\n                if let Err(retry_at) = server\n                    .is_allowed(throttle, &envelope, message.span_id)\n                    .await\n                {\n                    trc::event!(\n                        Delivery(DeliveryEvent::RateLimitExceeded),\n                        Id = throttle.id.clone(),\n                        SpanId = span_id,\n                        Domain = domain.to_string(),\n                    );\n\n                    delivery_results.push(DeliveryResult::rate_limited(rcpt_idxs, retry_at));\n                    continue 'next_route;\n                }\n            }\n\n            // Obtain next hop\n            let (mut remote_hosts, mx_config, is_smtp) = match route {\n                RoutingStrategy::Local => {\n                    // Deliver message locally\n                    message\n                        .deliver_local(&rcpt_idxs, &mut delivery_results, &server)\n                        .await;\n                    continue 'next_route;\n                }\n                RoutingStrategy::Mx(mx_config) => (Vec::with_capacity(0), Some(mx_config), true),\n                RoutingStrategy::Relay(relay_config) => (\n                    vec![NextHop::Relay(relay_config)],\n                    None,\n                    relay_config.protocol == ServerProtocol::Smtp,\n                ),\n            };\n\n            // Prepare TLS strategy\n            let mut tls_strategy = server.get_tls_or_default(\n                &server\n                    .eval_if::<String, _>(&queue_config.tls, &envelope, message.span_id)\n                    .await\n                    .unwrap_or_else(|| \"default\".to_string()),\n                message.span_id,\n            );\n\n            // Obtain TLS reporting\n            let tls_report =\n                if is_smtp && mx_config.is_some() && (message.message.flags & FROM_REPORT == 0) {\n                    match server\n                        .eval_if(\n                            &server.core.smtp.report.tls.send,\n                            &envelope,\n                            message.span_id,\n                        )\n                        .await\n                        .unwrap_or(AggregateFrequency::Never)\n                    {\n                        interval @ (AggregateFrequency::Hourly\n                        | AggregateFrequency::Daily\n                        | AggregateFrequency::Weekly) => {\n                            let time = Instant::now();\n                            match server\n                                .core\n                                .smtp\n                                .resolvers\n                                .dns\n                                .txt_lookup::<TlsRpt>(\n                                    format!(\"_smtp._tls.{domain}.\"),\n                                    Some(&server.inner.cache.dns_txt),\n                                )\n                                .await\n                            {\n                                Ok(record) => {\n                                    trc::event!(\n                                        TlsRpt(TlsRptEvent::RecordFetch),\n                                        SpanId = message.span_id,\n                                        Domain = domain.to_string(),\n                                        Details = record\n                                            .rua\n                                            .iter()\n                                            .map(|uri| trc::Value::from(match uri {\n                                                mail_auth::mta_sts::ReportUri::Mail(uri)\n                                                | mail_auth::mta_sts::ReportUri::Http(uri) =>\n                                                    uri.to_string(),\n                                            }))\n                                            .collect::<Vec<_>>(),\n                                        Elapsed = time.elapsed(),\n                                    );\n\n                                    TlsRptOptions { record, interval }.into()\n                                }\n                                Err(mail_auth::Error::DnsRecordNotFound(_)) => {\n                                    trc::event!(\n                                        TlsRpt(TlsRptEvent::RecordNotFound),\n                                        SpanId = message.span_id,\n                                        Domain = domain.to_string(),\n                                        Elapsed = time.elapsed(),\n                                    );\n                                    None\n                                }\n                                Err(err) => {\n                                    trc::event!(\n                                        TlsRpt(TlsRptEvent::RecordFetchError),\n                                        SpanId = message.span_id,\n                                        Domain = domain.to_string(),\n                                        CausedBy = trc::Error::from(err),\n                                        Elapsed = time.elapsed(),\n                                    );\n                                    None\n                                }\n                            }\n                        }\n                        _ => None,\n                    }\n                } else {\n                    None\n                };\n\n            // Obtain MTA-STS policy for domain\n            let mta_sts_policy = if mx_config.is_some() && tls_strategy.try_mta_sts() && is_smtp {\n                let time = Instant::now();\n                match server\n                    .lookup_mta_sts_policy(domain, tls_strategy.timeout_mta_sts)\n                    .await\n                {\n                    Ok(mta_sts_policy) => {\n                        trc::event!(\n                            MtaSts(MtaStsEvent::PolicyFetch),\n                            SpanId = message.span_id,\n                            Domain = domain.to_string(),\n                            Strict = mta_sts_policy.enforce(),\n                            Details = mta_sts_policy\n                                .mx\n                                .iter()\n                                .map(|mx| trc::Value::String(mx.to_compact_string()))\n                                .collect::<Vec<_>>(),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        mta_sts_policy.into()\n                    }\n                    Err(err) => {\n                        // Report MTA-STS error\n                        let strict = tls_strategy.is_mta_sts_required();\n                        if let Some(tls_report) = &tls_report {\n                            match &err {\n                                mta_sts::Error::Dns(mail_auth::Error::DnsRecordNotFound(_)) => {\n                                    if strict {\n                                        server.schedule_report(TlsEvent {\n                                            policy: PolicyType::Sts(None),\n                                            domain: domain.to_string(),\n                                            failure: FailureDetails::new(ResultType::Other)\n                                                .with_failure_reason_code(\n                                                    \"MTA-STS is required and no policy was found.\",\n                                                )\n                                                .into(),\n                                            tls_record: tls_report.record.clone(),\n                                            interval: tls_report.interval,\n                                        })\n                                        .await;\n                                    }\n                                }\n                                mta_sts::Error::Dns(mail_auth::Error::DnsError(_)) => (),\n                                _ => {\n                                    server\n                                        .schedule_report(TlsEvent {\n                                            policy: PolicyType::Sts(None),\n                                            domain: domain.to_string(),\n                                            failure: FailureDetails::new(&err)\n                                                .with_failure_reason_code(err.to_string())\n                                                .into(),\n                                            tls_record: tls_report.record.clone(),\n                                            interval: tls_report.interval,\n                                        })\n                                        .await;\n                                }\n                            }\n                        }\n\n                        match &err {\n                            mta_sts::Error::Dns(mail_auth::Error::DnsRecordNotFound(_)) => {\n                                trc::event!(\n                                    MtaSts(MtaStsEvent::PolicyNotFound),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n                            }\n                            mta_sts::Error::Dns(err) => {\n                                trc::event!(\n                                    MtaSts(MtaStsEvent::PolicyFetchError),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    CausedBy = trc::Error::from(err.clone()),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n                            }\n                            mta_sts::Error::Http(err) => {\n                                trc::event!(\n                                    MtaSts(MtaStsEvent::PolicyFetchError),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Reason = err.to_string(),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n                            }\n                            mta_sts::Error::InvalidPolicy(reason) => {\n                                trc::event!(\n                                    MtaSts(MtaStsEvent::InvalidPolicy),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Reason = reason.clone(),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n                            }\n                        }\n\n                        if strict {\n                            delivery_results.push(DeliveryResult::domain(\n                                Status::from_mta_sts_error(domain, err),\n                                rcpt_idxs,\n                            ));\n                            continue 'next_route;\n                        }\n\n                        None\n                    }\n                }\n            } else {\n                None\n            };\n\n            // Obtain remote hosts list\n            let mx_list;\n            if let Some(mx_config) = mx_config {\n                // Lookup MX\n                let time = Instant::now();\n                mx_list = match server\n                    .core\n                    .smtp\n                    .resolvers\n                    .dns\n                    .mx_lookup(domain, Some(&server.inner.cache.dns_mx))\n                    .await\n                {\n                    Ok(mx) => mx,\n                    Err(mail_auth::Error::DnsRecordNotFound(_)) => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::MxLookupFailed),\n                            SpanId = message.span_id,\n                            Domain = domain.to_string(),\n                            Details = \"No MX records were found, attempting implicit MX.\",\n                            Elapsed = time.elapsed(),\n                        );\n\n                        Arc::new(vec![])\n                    }\n                    Err(err) => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::MxLookupFailed),\n                            SpanId = message.span_id,\n                            Domain = domain.to_string(),\n                            CausedBy = trc::Error::from(err.clone()),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        delivery_results.push(DeliveryResult::domain(\n                            Status::from_mail_auth_error(domain, err),\n                            rcpt_idxs,\n                        ));\n                        continue 'next_route;\n                    }\n                };\n\n                if let Some(remote_hosts_) = mx_list.to_remote_hosts(domain, mx_config) {\n                    trc::event!(\n                        Delivery(DeliveryEvent::MxLookup),\n                        SpanId = message.span_id,\n                        Domain = domain.to_string(),\n                        Details = remote_hosts_\n                            .iter()\n                            .map(|h| trc::Value::String(h.hostname().into()))\n                            .collect::<Vec<_>>(),\n                        Elapsed = time.elapsed(),\n                    );\n                    remote_hosts = remote_hosts_;\n                } else {\n                    trc::event!(\n                        Delivery(DeliveryEvent::NullMx),\n                        SpanId = message.span_id,\n                        Domain = domain.to_string(),\n                        Elapsed = time.elapsed(),\n                    );\n\n                    delivery_results.push(DeliveryResult::domain(\n                        Status::PermanentFailure(ErrorDetails {\n                            entity: domain.into(),\n                            details: Error::DnsError(\n                                \"Domain does not accept messages (null MX)\".into(),\n                            ),\n                        }),\n                        rcpt_idxs,\n                    ));\n                    continue 'next_route;\n                }\n            }\n\n            // Try delivering message\n            let mut last_status: Status<HostResponse<Box<str>>, ErrorDetails> = Status::Scheduled;\n            'next_host: for remote_host in &remote_hosts {\n                // Validate MTA-STS\n                envelope.mx = remote_host.hostname();\n                if let Some(mta_sts_policy) = &mta_sts_policy {\n                    let strict = mta_sts_policy.enforce();\n                    if !mta_sts_policy.verify(envelope.mx) {\n                        // Report MTA-STS failed verification\n                        if let Some(tls_report) = &tls_report {\n                            server\n                                .schedule_report(TlsEvent {\n                                    policy: mta_sts_policy.into(),\n                                    domain: domain.to_string(),\n                                    failure: FailureDetails::new(ResultType::ValidationFailure)\n                                        .with_receiving_mx_hostname(envelope.mx)\n                                        .with_failure_reason_code(\"MX not authorized by policy.\")\n                                        .into(),\n                                    tls_record: tls_report.record.clone(),\n                                    interval: tls_report.interval,\n                                })\n                                .await;\n                        }\n\n                        trc::event!(\n                            MtaSts(MtaStsEvent::NotAuthorized),\n                            SpanId = message.span_id,\n                            Domain = domain.to_string(),\n                            Hostname = envelope.mx.to_string(),\n                            Details = mta_sts_policy\n                                .mx\n                                .iter()\n                                .map(|mx| trc::Value::String(mx.to_compact_string()))\n                                .collect::<Vec<_>>(),\n                            Strict = strict,\n                        );\n\n                        if strict {\n                            last_status = Status::PermanentFailure(ErrorDetails {\n                                entity: envelope.mx.into(),\n                                details: Error::MtaStsError(\n                                    format!(\"MX {:?} not authorized by policy.\", envelope.mx)\n                                        .into_boxed_str(),\n                                ),\n                            });\n                            continue 'next_host;\n                        }\n                    } else {\n                        trc::event!(\n                            MtaSts(MtaStsEvent::Authorized),\n                            SpanId = message.span_id,\n                            Domain = domain.to_string(),\n                            Hostname = envelope.mx.to_string(),\n                            Details = mta_sts_policy\n                                .mx\n                                .iter()\n                                .map(|mx| trc::Value::String(mx.to_compact_string()))\n                                .collect::<Vec<_>>(),\n                            Strict = strict,\n                        );\n                    }\n                }\n\n                // Obtain source and remote IPs\n                let time = Instant::now();\n                let resolve_result = match server.resolve_host(remote_host, &envelope).await {\n                    Ok(result) => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::IpLookup),\n                            SpanId = message.span_id,\n                            Domain = domain.to_string(),\n                            Hostname = envelope.mx.to_string(),\n                            Details = result\n                                .remote_ips\n                                .iter()\n                                .map(|ip| trc::Value::from(*ip))\n                                .collect::<Vec<_>>(),\n                            Limit = remote_host.max_multi_homed(),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        result\n                    }\n                    Err(status) => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::IpLookupFailed),\n                            SpanId = message.span_id,\n                            Domain = domain.to_string(),\n                            Hostname = envelope.mx.to_string(),\n                            Details = status.to_string(),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        last_status = status;\n                        continue 'next_host;\n                    }\n                };\n\n                // Update TLS strategy\n                tls_strategy = server.get_tls_or_default(\n                    &server\n                        .eval_if::<String, _>(&queue_config.tls, &envelope, message.span_id)\n                        .await\n                        .unwrap_or_else(|| \"default\".to_string()),\n                    message.span_id,\n                );\n\n                // Lookup DANE policy\n                let dane_policy = if tls_strategy.try_dane() && is_smtp {\n                    let time = Instant::now();\n                    let strict = tls_strategy.is_dane_required();\n                    match server\n                        .tlsa_lookup(format!(\"_25._tcp.{}.\", envelope.mx))\n                        .await\n                    {\n                        Ok(Some(tlsa)) => {\n                            if tlsa.has_end_entities {\n                                trc::event!(\n                                    Dane(DaneEvent::TlsaRecordFetch),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Hostname = envelope.mx.to_string(),\n                                    Details = format!(\"{tlsa:?}\"),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n\n                                tlsa.into()\n                            } else {\n                                trc::event!(\n                                    Dane(DaneEvent::TlsaRecordInvalid),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Hostname = envelope.mx.to_string(),\n                                    Details = format!(\"{tlsa:?}\"),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n\n                                // Report invalid TLSA record\n                                if let Some(tls_report) = &tls_report {\n                                    server\n                                        .schedule_report(TlsEvent {\n                                            policy: tlsa.into(),\n                                            domain: domain.to_string(),\n                                            failure: FailureDetails::new(ResultType::TlsaInvalid)\n                                                .with_receiving_mx_hostname(envelope.mx)\n                                                .with_failure_reason_code(\"Invalid TLSA record.\")\n                                                .into(),\n                                            tls_record: tls_report.record.clone(),\n                                            interval: tls_report.interval,\n                                        })\n                                        .await;\n                                }\n\n                                if strict {\n                                    last_status = Status::PermanentFailure(ErrorDetails {\n                                        entity: envelope.mx.into(),\n                                        details: Error::DaneError(\n                                            \"No valid TLSA records were found\".into(),\n                                        ),\n                                    });\n                                    continue 'next_host;\n                                }\n                                None\n                            }\n                        }\n                        Ok(None) => {\n                            trc::event!(\n                                Dane(DaneEvent::TlsaRecordNotDnssecSigned),\n                                SpanId = message.span_id,\n                                Domain = domain.to_string(),\n                                Hostname = envelope.mx.to_string(),\n                                Strict = strict,\n                                Elapsed = time.elapsed(),\n                            );\n\n                            if strict {\n                                // Report DANE required\n                                if let Some(tls_report) = &tls_report {\n                                    server\n                                        .schedule_report(TlsEvent {\n                                            policy: PolicyType::Tlsa(None),\n                                            domain: domain.to_string(),\n                                            failure: FailureDetails::new(ResultType::DaneRequired)\n                                                .with_receiving_mx_hostname(envelope.mx)\n                                                .with_failure_reason_code(\n                                                    \"No TLSA DNSSEC records found.\",\n                                                )\n                                                .into(),\n                                            tls_record: tls_report.record.clone(),\n                                            interval: tls_report.interval,\n                                        })\n                                        .await;\n                                }\n\n                                last_status = Status::PermanentFailure(ErrorDetails {\n                                    entity: envelope.mx.into(),\n                                    details: Error::DaneError(\n                                        \"No TLSA DNSSEC records found\".into(),\n                                    ),\n                                });\n                                continue 'next_host;\n                            }\n                            None\n                        }\n                        Err(err) => {\n                            let not_found = matches!(&err, mail_auth::Error::DnsRecordNotFound(_));\n\n                            if not_found {\n                                trc::event!(\n                                    Dane(DaneEvent::TlsaRecordNotFound),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Hostname = envelope.mx.to_string(),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n                            } else {\n                                trc::event!(\n                                    Dane(DaneEvent::TlsaRecordFetchError),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Hostname = envelope.mx.to_string(),\n                                    CausedBy = trc::Error::from(err.clone()),\n                                    Strict = strict,\n                                    Elapsed = time.elapsed(),\n                                );\n                            }\n\n                            if strict {\n                                last_status = if not_found {\n                                    // Report DANE required\n                                    if let Some(tls_report) = &tls_report {\n                                        server\n                                            .schedule_report(TlsEvent {\n                                                policy: PolicyType::Tlsa(None),\n                                                domain: domain.to_string(),\n                                                failure: FailureDetails::new(\n                                                    ResultType::DaneRequired,\n                                                )\n                                                .with_receiving_mx_hostname(envelope.mx)\n                                                .with_failure_reason_code(\n                                                    \"No TLSA records found for MX.\",\n                                                )\n                                                .into(),\n                                                tls_record: tls_report.record.clone(),\n                                                interval: tls_report.interval,\n                                            })\n                                            .await;\n                                    }\n\n                                    Status::PermanentFailure(ErrorDetails {\n                                        entity: envelope.mx.into(),\n                                        details: Error::DaneError(\"No TLSA records found\".into()),\n                                    })\n                                } else {\n                                    Status::from_mail_auth_error(envelope.mx, err)\n                                };\n                                continue 'next_host;\n                            }\n                            None\n                        }\n                    }\n                } else {\n                    None\n                };\n\n                // Try each IP address\n                'next_ip: for remote_ip in resolve_result.remote_ips {\n                    // Throttle remote host\n                    envelope.remote_ip = remote_ip;\n                    for throttle in &queue_config.outbound_limiters.remote {\n                        if let Err(retry_at) = server\n                            .is_allowed(throttle, &envelope, message.span_id)\n                            .await\n                        {\n                            trc::event!(\n                                Delivery(DeliveryEvent::RateLimitExceeded),\n                                SpanId = message.span_id,\n                                Id = throttle.id.clone(),\n                                RemoteIp = remote_ip,\n                            );\n                            delivery_results\n                                .push(DeliveryResult::rate_limited(rcpt_idxs, retry_at));\n                            continue 'next_route;\n                        }\n                    }\n\n                    // Obtain connection parameters\n                    let conn_strategy = server.get_connection_or_default(\n                        &server\n                            .eval_if::<String, _>(\n                                &queue_config.connection,\n                                &envelope,\n                                message.span_id,\n                            )\n                            .await\n                            .unwrap_or_else(|| \"default\".to_string()),\n                        message.span_id,\n                    );\n\n                    // Set source IP, if any\n                    let ip_host = conn_strategy.source_ip(remote_ip.is_ipv4());\n\n                    // Connect\n                    let time = Instant::now();\n                    let mut smtp_client = match if let Some(ip_host) = ip_host {\n                        envelope.local_ip = ip_host.ip;\n                        SmtpClient::connect_using(\n                            ip_host.ip,\n                            SocketAddr::new(remote_ip, remote_host.port()),\n                            conn_strategy.timeout_connect,\n                            span_id,\n                        )\n                        .await\n                    } else {\n                        envelope.local_ip = no_ip;\n                        SmtpClient::connect(\n                            SocketAddr::new(remote_ip, remote_host.port()),\n                            conn_strategy.timeout_connect,\n                            span_id,\n                        )\n                        .await\n                    } {\n                        Ok(smtp_client) => {\n                            trc::event!(\n                                Delivery(DeliveryEvent::Connect),\n                                SpanId = message.span_id,\n                                Domain = domain.to_string(),\n                                Hostname = envelope.mx.to_string(),\n                                LocalIp = envelope.local_ip,\n                                RemoteIp = remote_ip,\n                                RemotePort = remote_host.port(),\n                                Elapsed = time.elapsed(),\n                            );\n\n                            smtp_client\n                        }\n                        Err(err) => {\n                            trc::event!(\n                                Delivery(DeliveryEvent::ConnectError),\n                                SpanId = message.span_id,\n                                Domain = domain.to_string(),\n                                Hostname = envelope.mx.to_string(),\n                                LocalIp = envelope.local_ip,\n                                RemoteIp = remote_ip,\n                                RemotePort = remote_host.port(),\n                                CausedBy = from_mail_send_error(&err),\n                                Elapsed = time.elapsed(),\n                            );\n\n                            last_status = Status::from_smtp_error(envelope.mx, \"\", err);\n                            continue 'next_ip;\n                        }\n                    };\n\n                    // Obtain session parameters\n                    let local_hostname = ip_host\n                        .and_then(|ip| ip.host.as_deref())\n                        .or(conn_strategy.ehlo_hostname.as_deref())\n                        .unwrap_or(server.core.network.server_name.as_str());\n                    let mut params = SessionParams {\n                        session_id: message.span_id,\n                        server: &server,\n                        credentials: remote_host.credentials(),\n                        is_smtp: remote_host.is_smtp(),\n                        hostname: envelope.mx,\n                        local_hostname,\n                        conn_strategy,\n                        capabilities: None,\n                    };\n\n                    // Prepare TLS connector\n                    let is_strict_tls = tls_strategy.is_tls_required()\n                        || (message.message.flags & MAIL_REQUIRETLS) != 0\n                        || mta_sts_policy.is_some()\n                        || dane_policy.is_some();\n                    // As per RFC7671 Section 5.1, DANE-EE(3) allows name mismatch\n                    let tls_connector = if tls_strategy.allow_invalid_certs\n                        || remote_host.allow_invalid_certs()\n                        || dane_policy.as_ref().is_some_and(|t| t.has_end_entities)\n                    {\n                        &server.inner.data.smtp_connectors.dummy_verify\n                    } else {\n                        &server.inner.data.smtp_connectors.pki_verify\n                    };\n\n                    if !remote_host.implicit_tls() {\n                        // Read greeting\n                        smtp_client.timeout = conn_strategy.timeout_greeting;\n                        if let Err(status) = smtp_client.read_greeting(envelope.mx).await {\n                            trc::event!(\n                                Delivery(DeliveryEvent::GreetingFailed),\n                                SpanId = message.span_id,\n                                Domain = domain.to_string(),\n                                Hostname = envelope.mx.to_string(),\n                                Details = status.to_string(),\n                            );\n\n                            last_status = status;\n                            continue 'next_host;\n                        }\n\n                        // Say EHLO\n                        let time = Instant::now();\n                        let capabilities = match smtp_client.say_helo(&params).await {\n                            Ok(capabilities) => {\n                                trc::event!(\n                                    Delivery(DeliveryEvent::Ehlo),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Hostname = envelope.mx.to_string(),\n                                    Details = capabilities.capabilities(),\n                                    Elapsed = time.elapsed(),\n                                );\n\n                                capabilities\n                            }\n                            Err(status) => {\n                                trc::event!(\n                                    Delivery(DeliveryEvent::EhloRejected),\n                                    SpanId = message.span_id,\n                                    Domain = domain.to_string(),\n                                    Hostname = envelope.mx.to_string(),\n                                    Details = status.to_string(),\n                                    Elapsed = time.elapsed(),\n                                );\n\n                                last_status = status;\n                                continue 'next_host;\n                            }\n                        };\n\n                        // Try starting TLS\n                        if tls_strategy.try_start_tls() {\n                            let time = Instant::now();\n                            smtp_client.timeout = tls_strategy.timeout_tls;\n                            match smtp_client\n                                .try_start_tls(tls_connector, envelope.mx, &capabilities)\n                                .await\n                            {\n                                StartTlsResult::Success { smtp_client } => {\n                                    trc::event!(\n                                        Delivery(DeliveryEvent::StartTls),\n                                        SpanId = message.span_id,\n                                        Domain = domain.to_string(),\n                                        Hostname = envelope.mx.to_string(),\n                                        Version = format!(\n                                            \"{:?}\",\n                                            smtp_client\n                                                .tls_connection()\n                                                .protocol_version()\n                                                .unwrap()\n                                        ),\n                                        Details = format!(\n                                            \"{:?}\",\n                                            smtp_client\n                                                .tls_connection()\n                                                .negotiated_cipher_suite()\n                                                .unwrap()\n                                        ),\n                                        Elapsed = time.elapsed(),\n                                    );\n\n                                    // Verify DANE\n                                    if let Some(dane_policy) = &dane_policy\n                                        && let Err(status) = dane_policy.verify(\n                                            message.span_id,\n                                            envelope.mx,\n                                            smtp_client.tls_connection().peer_certificates(),\n                                        )\n                                    {\n                                        // Report DANE verification failure\n                                        if let Some(tls_report) = &tls_report {\n                                            server\n                                                .schedule_report(TlsEvent {\n                                                    policy: dane_policy.into(),\n                                                    domain: domain.to_string(),\n                                                    failure: FailureDetails::new(\n                                                        ResultType::ValidationFailure,\n                                                    )\n                                                    .with_receiving_mx_hostname(envelope.mx)\n                                                    .with_receiving_ip(remote_ip)\n                                                    .with_failure_reason_code(\n                                                        \"No matching certificates found.\",\n                                                    )\n                                                    .into(),\n                                                    tls_record: tls_report.record.clone(),\n                                                    interval: tls_report.interval,\n                                                })\n                                                .await;\n                                        }\n\n                                        last_status = status;\n                                        continue 'next_host;\n                                    }\n\n                                    // Report TLS success\n                                    if let Some(tls_report) = &tls_report {\n                                        server\n                                            .schedule_report(TlsEvent {\n                                                policy: (&mta_sts_policy, &dane_policy).into(),\n                                                domain: domain.to_string(),\n                                                failure: None,\n                                                tls_record: tls_report.record.clone(),\n                                                interval: tls_report.interval,\n                                            })\n                                            .await;\n                                    }\n\n                                    // Deliver message over TLS\n                                    message\n                                        .deliver(\n                                            smtp_client,\n                                            rcpt_idxs,\n                                            &mut delivery_results,\n                                            params,\n                                        )\n                                        .await\n                                }\n                                StartTlsResult::Unavailable {\n                                    response,\n                                    smtp_client,\n                                } => {\n                                    // Report unavailable STARTTLS\n                                    let reason =\n                                        response.as_ref().map(|r| r.to_string()).unwrap_or_else(\n                                            || \"STARTTLS was not advertised by host\".to_string(),\n                                        );\n\n                                    trc::event!(\n                                        Delivery(DeliveryEvent::StartTlsUnavailable),\n                                        SpanId = message.span_id,\n                                        Domain = domain.to_string(),\n                                        Hostname = envelope.mx.to_string(),\n                                        Code = response.as_ref().map(|r| r.code()),\n                                        Details = response\n                                            .as_ref()\n                                            .map(|r| r.message().as_ref())\n                                            .unwrap_or(\"STARTTLS was not advertised by host\")\n                                            .to_string(),\n                                        Elapsed = time.elapsed(),\n                                    );\n\n                                    if let Some(tls_report) = &tls_report {\n                                        server\n                                            .schedule_report(TlsEvent {\n                                                policy: (&mta_sts_policy, &dane_policy).into(),\n                                                domain: domain.to_string(),\n                                                failure: FailureDetails::new(\n                                                    ResultType::StartTlsNotSupported,\n                                                )\n                                                .with_receiving_mx_hostname(envelope.mx)\n                                                .with_receiving_ip(remote_ip)\n                                                .with_failure_reason_code(reason)\n                                                .into(),\n                                                tls_record: tls_report.record.clone(),\n                                                interval: tls_report.interval,\n                                            })\n                                            .await;\n                                    }\n\n                                    if is_strict_tls {\n                                        last_status =\n                                            Status::from_starttls_error(envelope.mx, response);\n                                        continue 'next_host;\n                                    } else {\n                                        // TLS is not required, proceed in plain-text\n                                        params.capabilities = Some(capabilities);\n                                        message\n                                            .deliver(\n                                                smtp_client,\n                                                rcpt_idxs,\n                                                &mut delivery_results,\n                                                params,\n                                            )\n                                            .await\n                                    }\n                                }\n                                StartTlsResult::Error { error } => {\n                                    trc::event!(\n                                        Delivery(DeliveryEvent::StartTlsError),\n                                        SpanId = message.span_id,\n                                        Domain = domain.to_string(),\n                                        Hostname = envelope.mx.to_string(),\n                                        Reason = from_mail_send_error(&error),\n                                        Elapsed = time.elapsed(),\n                                    );\n\n                                    // Report TLS failure\n                                    if let (Some(tls_report), mail_send::Error::Tls(error)) =\n                                        (&tls_report, &error)\n                                    {\n                                        server\n                                            .schedule_report(TlsEvent {\n                                                policy: (&mta_sts_policy, &dane_policy).into(),\n                                                domain: domain.to_string(),\n                                                failure: FailureDetails::new(\n                                                    ResultType::CertificateNotTrusted,\n                                                )\n                                                .with_receiving_mx_hostname(envelope.mx)\n                                                .with_receiving_ip(remote_ip)\n                                                .with_failure_reason_code(error.to_string())\n                                                .into(),\n                                                tls_record: tls_report.record.clone(),\n                                                interval: tls_report.interval,\n                                            })\n                                            .await;\n                                    }\n\n                                    last_status = if is_strict_tls {\n                                        Status::from_tls_error(envelope.mx, error)\n                                    } else {\n                                        Status::from_tls_error(envelope.mx, error).into_temporary()\n                                    };\n                                    continue 'next_host;\n                                }\n                            }\n                        } else {\n                            // TLS has been disabled\n                            trc::event!(\n                                Delivery(DeliveryEvent::StartTlsDisabled),\n                                SpanId = message.span_id,\n                                Domain = domain.to_string(),\n                                Hostname = envelope.mx.to_string(),\n                            );\n\n                            message\n                                .deliver(smtp_client, rcpt_idxs, &mut delivery_results, params)\n                                .await\n                        }\n                    } else {\n                        // Start TLS\n                        smtp_client.timeout = tls_strategy.timeout_tls;\n                        let mut smtp_client =\n                            match smtp_client.into_tls(tls_connector, envelope.mx).await {\n                                Ok(smtp_client) => smtp_client,\n                                Err(error) => {\n                                    trc::event!(\n                                        Delivery(DeliveryEvent::ImplicitTlsError),\n                                        SpanId = message.span_id,\n                                        Domain = domain.to_string(),\n                                        Hostname = envelope.mx.to_string(),\n                                        Reason = from_mail_send_error(&error),\n                                    );\n\n                                    last_status = Status::from_tls_error(envelope.mx, error);\n                                    continue 'next_host;\n                                }\n                            };\n\n                        // Read greeting\n                        smtp_client.timeout = conn_strategy.timeout_greeting;\n                        if let Err(status) = smtp_client.read_greeting(envelope.mx).await {\n                            trc::event!(\n                                Delivery(DeliveryEvent::GreetingFailed),\n                                SpanId = message.span_id,\n                                Domain = domain.to_string(),\n                                Hostname = envelope.mx.to_string(),\n                                Details = from_error_status(&status),\n                            );\n\n                            last_status = status;\n                            continue 'next_host;\n                        }\n\n                        // Deliver message\n                        message\n                            .deliver(smtp_client, rcpt_idxs, &mut delivery_results, params)\n                            .await\n                    }\n\n                    // Continue with the next domain/route\n                    continue 'next_route;\n                }\n            }\n\n            // Update status\n            delivery_results.push(DeliveryResult::domain(last_status, rcpt_idxs));\n        }\n\n        // Apply status changes\n        for delivery_result in delivery_results {\n            match delivery_result {\n                DeliveryResult::Domain { status, rcpt_idxs } => {\n                    for rcpt_idx in rcpt_idxs {\n                        message\n                            .set_rcpt_status(status.clone(), rcpt_idx, &server)\n                            .await;\n                    }\n                }\n                DeliveryResult::Account { status, rcpt_idx } => {\n                    message.set_rcpt_status(status, rcpt_idx, &server).await;\n                }\n                DeliveryResult::RateLimited {\n                    rcpt_idxs,\n                    retry_at,\n                } => {\n                    for rcpt_idx in rcpt_idxs {\n                        message.set_rcpt_rate_limit(rcpt_idx, retry_at);\n                    }\n                }\n            }\n        }\n\n        // Send Delivery Status Notifications\n        server.send_dsn(&mut message).await;\n\n        // Notify queue manager\n        if message.message.next_event(None).is_some() {\n            trc::event!(\n                Queue(trc::QueueEvent::Rescheduled),\n                SpanId = span_id,\n                NextRetry = message\n                    .message\n                    .next_delivery_event(None)\n                    .map(trc::Value::Timestamp),\n                NextDsn = message.message.next_dsn(None).map(trc::Value::Timestamp),\n                Expires = message.message.expires(None).map(trc::Value::Timestamp),\n            );\n\n            // Save changes to disk\n            message.save_changes(&server, self.due.into()).await;\n\n            QueueEventStatus::Deferred\n        } else {\n            trc::event!(\n                Delivery(DeliveryEvent::Completed),\n                SpanId = span_id,\n                Elapsed = trc::Value::Duration((now() - message.message.created) * 1000)\n            );\n\n            // Delete message from queue\n            message.remove(&server, self.due.into()).await;\n\n            QueueEventStatus::Completed\n        }\n    }\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum PendingDelivery {\n    Yes(bool),\n    No,\n}\n\nimpl MessageWrapper {\n    /// Marks as failed all domains that reached their expiration time\n    pub fn has_pending_delivery(&mut self) -> PendingDelivery {\n        let now = now();\n        let mut has_pending_delivery = false;\n        let mut matches_queue = false;\n\n        for rcpt in self.message.recipients.iter_mut() {\n            match &rcpt.status {\n                Status::TemporaryFailure(err) if rcpt.is_expired(self.message.created, now) => {\n                    trc::event!(\n                        Delivery(DeliveryEvent::Failed),\n                        SpanId = self.span_id,\n                        QueueId = self.queue_id,\n                        QueueName = self.queue_name.as_str().to_string(),\n                        To = rcpt.address().to_string(),\n                        Reason = from_error_details(&err.details),\n                        Details = trc::Value::Timestamp(now),\n                        Expires = rcpt\n                            .expiration_time(self.message.created)\n                            .map(trc::Value::Timestamp),\n                        NextRetry = trc::Value::Timestamp(rcpt.retry.due),\n                        NextDsn = trc::Value::Timestamp(rcpt.notify.due),\n                    );\n\n                    rcpt.status =\n                        std::mem::replace(&mut rcpt.status, Status::Scheduled).into_permanent();\n                }\n                Status::Scheduled if rcpt.is_expired(self.message.created, now) => {\n                    trc::event!(\n                        Delivery(DeliveryEvent::Failed),\n                        SpanId = self.span_id,\n                        QueueId = self.queue_id,\n                        QueueName = self.queue_name.as_str().to_string(),\n                        To = rcpt.address().to_string(),\n                        Reason = \"Message expired without any delivery attempts made.\",\n                        Details = trc::Value::Timestamp(now),\n                        Expires = rcpt\n                            .expiration_time(self.message.created)\n                            .map(trc::Value::Timestamp),\n                        NextRetry = trc::Value::Timestamp(rcpt.retry.due),\n                        NextDsn = trc::Value::Timestamp(rcpt.notify.due),\n                    );\n\n                    rcpt.status = Status::PermanentFailure(ErrorDetails {\n                        entity: rcpt.domain_part().into(),\n                        details: Error::Io(\n                            \"Message expired without any delivery attempts made.\".into(),\n                        ),\n                    });\n                }\n                Status::Completed(_) | Status::PermanentFailure(_) => (),\n                _ => {\n                    has_pending_delivery = true;\n                    matches_queue = matches_queue || rcpt.queue == self.queue_name;\n                }\n            }\n        }\n\n        if has_pending_delivery {\n            PendingDelivery::Yes(matches_queue)\n        } else {\n            PendingDelivery::No\n        }\n    }\n\n    pub async fn set_rcpt_status(\n        &mut self,\n        status: Status<HostResponse<Box<str>>, ErrorDetails>,\n        rcpt_idx: usize,\n        server: &Server,\n    ) {\n        let needs_retry = matches!(&status, Status::TemporaryFailure(_) | Status::Scheduled);\n        self.message.recipients[rcpt_idx].status = status;\n\n        if needs_retry {\n            let envelope = QueueEnvelope::new(&self.message, &self.message.recipients[rcpt_idx]);\n            let queue = server.get_queue_or_default(\n                &server\n                    .eval_if::<String, _>(&server.core.smtp.queue.queue, &envelope, self.span_id)\n                    .await\n                    .unwrap_or_else(|| \"default\".to_string()),\n                self.span_id,\n            );\n            let rcpt = &mut self.message.recipients[rcpt_idx];\n            rcpt.retry.due = now()\n                + queue.retry[std::cmp::min(rcpt.retry.inner as usize, queue.retry.len() - 1)];\n            rcpt.retry.inner += 1;\n            rcpt.expires = queue.expiry;\n            rcpt.queue = queue.virtual_queue;\n        }\n    }\n\n    pub fn set_rcpt_rate_limit(&mut self, rcpt_idx: usize, retry_at: u64) {\n        let rcpt = &mut self.message.recipients[rcpt_idx];\n        rcpt.retry.due = retry_at;\n        rcpt.status = Status::TemporaryFailure(ErrorDetails {\n            entity: \"localhost\".into(),\n            details: Error::RateLimited,\n        });\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/local.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    outbound::DeliveryResult,\n    queue::{\n        Error, ErrorDetails, FROM_AUTHENTICATED, FROM_UNAUTHENTICATED_DMARC, HostResponse,\n        MessageSource, MessageWrapper, RCPT_SPAM_PAYLOAD, Status, UnexpectedResponse,\n        quota::HasQueueQuota, spool::SmtpSpool,\n    },\n    reporting::SmtpReporting,\n};\nuse common::Server;\nuse email::message::delivery::{IngestMessage, IngestRecipient, LocalDeliveryStatus, MailDelivery};\nuse smtp_proto::Response;\nuse trc::SieveEvent;\n\nimpl MessageWrapper {\n    pub(super) async fn deliver_local(\n        &self,\n        rcpt_idxs: &[usize],\n        statuses: &mut Vec<DeliveryResult>,\n        server: &Server,\n    ) {\n        // Prepare recipients list\n        let mut pending_recipients = Vec::new();\n        let mut recipients = Vec::new();\n        for &rcpt_idx in rcpt_idxs {\n            let rcpt = &self.message.recipients[rcpt_idx];\n            let rcpt_addr = rcpt.address();\n            recipients.push(IngestRecipient {\n                address: rcpt_addr.to_lowercase(),\n                is_spam: rcpt.flags & RCPT_SPAM_PAYLOAD != 0,\n            });\n            pending_recipients.push((rcpt_idx, rcpt_addr));\n        }\n\n        // Deliver message\n        let delivery_result = server\n            .deliver_message(IngestMessage {\n                sender_address: self.message.return_path.to_string(),\n                sender_authenticated: self.message.flags\n                    & (FROM_UNAUTHENTICATED_DMARC | FROM_AUTHENTICATED)\n                    != 0,\n                recipients,\n                message_blob: self.message.blob_hash.clone(),\n                message_size: self.message.size,\n                session_id: self.span_id,\n            })\n            .await;\n\n        // Process delivery results\n        for ((rcpt_idx, rcpt_addr), result) in\n            pending_recipients.into_iter().zip(delivery_result.status)\n        {\n            let status = match result {\n                LocalDeliveryStatus::Success => Status::Completed(HostResponse {\n                    hostname: \"localhost\".into(),\n                    response: Response {\n                        code: 250,\n                        esc: [2, 1, 5],\n                        message: \"OK\".into(),\n                    },\n                }),\n                LocalDeliveryStatus::TemporaryFailure { reason } => {\n                    Status::TemporaryFailure(ErrorDetails {\n                        entity: \"localhost\".into(),\n                        details: Error::UnexpectedResponse(UnexpectedResponse {\n                            command: format!(\"RCPT TO:<{rcpt_addr}>\").into_boxed_str(),\n                            response: Response {\n                                code: 451,\n                                esc: [4, 3, 0],\n                                message: reason.into(),\n                            },\n                        }),\n                    })\n                }\n                LocalDeliveryStatus::PermanentFailure { code, reason } => {\n                    Status::PermanentFailure(ErrorDetails {\n                        entity: \"localhost\".into(),\n                        details: Error::UnexpectedResponse(UnexpectedResponse {\n                            command: format!(\"RCPT TO:<{rcpt_addr}>\").into_boxed_str(),\n                            response: Response {\n                                code: 550,\n                                esc: code,\n                                message: reason.into(),\n                            },\n                        }),\n                    })\n                }\n            };\n            statuses.push(DeliveryResult::account(status, rcpt_idx));\n        }\n\n        // Process autogenerated messages\n        for autogenerated in delivery_result.autogenerated {\n            let mut message = server.new_message(autogenerated.sender_address, self.span_id);\n            for rcpt in autogenerated.recipients {\n                message.add_recipient(rcpt, server).await;\n            }\n\n            // Sign message\n            let signature = server\n                .sign_message(\n                    &mut message,\n                    &server.core.sieve.sign,\n                    &autogenerated.message,\n                )\n                .await;\n\n            // Queue Message\n            message.message.size =\n                (autogenerated.message.len() + signature.as_ref().map_or(0, |s| s.len())) as u64;\n            if server.has_quota(&mut message).await {\n                message\n                    .queue(\n                        signature.as_deref(),\n                        &autogenerated.message,\n                        self.span_id,\n                        server,\n                        MessageSource::Autogenerated,\n                    )\n                    .await;\n            } else {\n                trc::event!(\n                    Sieve(SieveEvent::QuotaExceeded),\n                    SpanId = self.span_id,\n                    From = message.message.return_path,\n                    To = message\n                        .message\n                        .recipients\n                        .into_iter()\n                        .map(|r| trc::Value::from(r.address().to_string()))\n                        .collect::<Vec<_>>(),\n                );\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::NextHop;\nuse crate::queue::{Error, ErrorDetails, HostResponse, Status};\nuse common::{\n    Server,\n    config::smtp::queue::{ConnectionStrategy, IpAndHost, MxConfig},\n    expr::{V_MX, functions::ResolveVariable},\n};\nuse mail_auth::{IpLookupStrategy, MX};\nuse rand::{Rng, seq::SliceRandom};\nuse std::{future::Future, net::IpAddr, sync::Arc};\n\npub struct IpLookupResult {\n    pub remote_ips: Vec<IpAddr>,\n}\n\npub trait DnsLookup: Sync + Send {\n    fn ip_lookup(\n        &self,\n        key: &str,\n        strategy: IpLookupStrategy,\n        max_results: usize,\n    ) -> impl Future<Output = mail_auth::Result<Vec<IpAddr>>> + Send;\n\n    fn resolve_host(\n        &self,\n        remote_host: &NextHop<'_>,\n        envelope: &impl ResolveVariable,\n    ) -> impl Future<Output = Result<IpLookupResult, Status<HostResponse<Box<str>>, ErrorDetails>>> + Send;\n}\n\nimpl DnsLookup for Server {\n    async fn ip_lookup(\n        &self,\n        key: &str,\n        strategy: IpLookupStrategy,\n        max_results: usize,\n    ) -> mail_auth::Result<Vec<IpAddr>> {\n        let (has_ipv4, has_ipv6, v4_first) = match strategy {\n            IpLookupStrategy::Ipv4Only => (true, false, false),\n            IpLookupStrategy::Ipv6Only => (false, true, false),\n            IpLookupStrategy::Ipv4thenIpv6 => (true, true, true),\n            IpLookupStrategy::Ipv6thenIpv4 => (true, true, false),\n        };\n        let ipv4_addrs = if has_ipv4 {\n            match self\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .ipv4_lookup(key, Some(&self.inner.cache.dns_ipv4))\n                .await\n            {\n                Ok(addrs) => addrs,\n                Err(_) if has_ipv6 => Arc::new(Vec::new()),\n                Err(err) => return Err(err),\n            }\n        } else {\n            Arc::new(Vec::new())\n        };\n\n        if has_ipv6 {\n            let ipv6_addrs = match self\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .ipv6_lookup(key, Some(&self.inner.cache.dns_ipv6))\n                .await\n            {\n                Ok(addrs) => addrs,\n                Err(_) if !ipv4_addrs.is_empty() => Arc::new(Vec::new()),\n                Err(err) => return Err(err),\n            };\n            if v4_first {\n                Ok(ipv4_addrs\n                    .iter()\n                    .copied()\n                    .map(IpAddr::from)\n                    .chain(ipv6_addrs.iter().copied().map(IpAddr::from))\n                    .take(max_results)\n                    .collect())\n            } else {\n                Ok(ipv6_addrs\n                    .iter()\n                    .copied()\n                    .map(IpAddr::from)\n                    .chain(ipv4_addrs.iter().copied().map(IpAddr::from))\n                    .take(max_results)\n                    .collect())\n            }\n        } else {\n            Ok(ipv4_addrs\n                .iter()\n                .take(max_results)\n                .copied()\n                .map(IpAddr::from)\n                .collect())\n        }\n    }\n\n    #[allow(unused_mut)]\n    async fn resolve_host(\n        &self,\n        remote_host: &NextHop<'_>,\n        envelope: &impl ResolveVariable,\n    ) -> Result<IpLookupResult, Status<HostResponse<Box<str>>, ErrorDetails>> {\n        let mut remote_ips = self\n            .ip_lookup(\n                remote_host.fqdn_hostname().as_ref(),\n                remote_host.ip_lookup_strategy(),\n                remote_host.max_multi_homed(),\n            )\n            .await\n            .map_err(|err| {\n                if let mail_auth::Error::DnsRecordNotFound(_) = &err {\n                    if matches!(\n                        remote_host,\n                        NextHop::MX {\n                            is_implicit: true,\n                            ..\n                        }\n                    ) {\n                        Status::PermanentFailure(ErrorDetails {\n                            entity: remote_host.hostname().into(),\n                            details: Error::DnsError(\"no MX record found.\".into()),\n                        })\n                    } else {\n                        Status::PermanentFailure(ErrorDetails {\n                            entity: remote_host.hostname().into(),\n                            details: Error::ConnectionError(\"record not found for MX\".into()),\n                        })\n                    }\n                } else {\n                    Status::TemporaryFailure(ErrorDetails {\n                        entity: remote_host.hostname().into(),\n                        details: Error::ConnectionError(\n                            format!(\"lookup error: {err}\").into_boxed_str(),\n                        ),\n                    })\n                }\n            })?;\n\n        if !remote_ips.is_empty() {\n            #[cfg(not(feature = \"test_mode\"))]\n            if remote_ips.iter().any(|ip| ip.is_loopback()) {\n                remote_ips.retain(|ip| !ip.is_loopback());\n                if remote_ips.is_empty() {\n                    return Err(Status::PermanentFailure(ErrorDetails {\n                        entity: remote_host.hostname().into(),\n                        details: Error::ConnectionError(\"host resolves loopback address\".into()),\n                    }));\n                }\n            }\n\n            Ok(IpLookupResult { remote_ips })\n        } else {\n            Err(Status::TemporaryFailure(ErrorDetails {\n                entity: remote_host.hostname().into(),\n                details: Error::DnsError(\n                    format!(\n                        \"No IP addresses found for {:?}.\",\n                        envelope.resolve_variable(V_MX).to_string()\n                    )\n                    .into_boxed_str(),\n                ),\n            }))\n        }\n    }\n}\n\npub trait SourceIp {\n    fn source_ip(&self, is_v4: bool) -> Option<&IpAndHost>;\n}\n\nimpl SourceIp for ConnectionStrategy {\n    fn source_ip(&self, is_v4: bool) -> Option<&IpAndHost> {\n        let ips = if is_v4 {\n            &self.source_ipv4\n        } else {\n            &self.source_ipv6\n        };\n        match ips.len().cmp(&1) {\n            std::cmp::Ordering::Equal => ips.first(),\n            std::cmp::Ordering::Greater => Some(&ips[rand::rng().random_range(0..ips.len())]),\n            std::cmp::Ordering::Less => None,\n        }\n    }\n}\n\npub trait ToNextHop {\n    fn to_remote_hosts<'x, 'y: 'x>(\n        &'x self,\n        domain: &'y str,\n        config: &'x MxConfig,\n    ) -> Option<Vec<NextHop<'x>>>;\n}\n\nimpl ToNextHop for Vec<MX> {\n    fn to_remote_hosts<'x, 'y: 'x>(\n        &'x self,\n        domain: &'y str,\n        config: &'x MxConfig,\n    ) -> Option<Vec<NextHop<'x>>> {\n        if !self.is_empty() {\n            // Obtain max number of MX hosts to process\n            let mut remote_hosts = Vec::with_capacity(config.max_mx);\n\n            'outer: for mx in self.iter() {\n                if mx.exchanges.len() > 1 {\n                    let mut slice = mx.exchanges.iter().collect::<Vec<_>>();\n                    slice.shuffle(&mut rand::rng());\n                    for remote_host in slice {\n                        remote_hosts.push(NextHop::MX {\n                            host: remote_host.as_str(),\n                            is_implicit: false,\n                            config,\n                        });\n                        if remote_hosts.len() == config.max_mx {\n                            break 'outer;\n                        }\n                    }\n                } else if let Some(remote_host) = mx.exchanges.first() {\n                    // Check for Null MX\n                    if mx.preference == 0 && remote_host == \".\" {\n                        return None;\n                    }\n                    remote_hosts.push(NextHop::MX {\n                        host: remote_host.as_str(),\n                        is_implicit: false,\n                        config,\n                    });\n                    if remote_hosts.len() == config.max_mx {\n                        break;\n                    }\n                }\n            }\n            remote_hosts.into()\n        } else {\n            // If an empty list of MXs is returned, the address is treated as if it was\n            // associated with an implicit MX RR with a preference of 0, pointing to that host.\n            vec![NextHop::MX {\n                host: domain,\n                is_implicit: true,\n                config,\n            }]\n            .into()\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    outbound::client::BoxResponse,\n    queue::{Error, ErrorDetails, HostResponse, Status, UnexpectedResponse},\n};\nuse common::config::{\n    server::ServerProtocol,\n    smtp::queue::{MxConfig, RelayConfig},\n};\nuse mail_auth::IpLookupStrategy;\nuse mail_send::Credentials;\nuse smtp_proto::{Response, Severity};\nuse std::borrow::Cow;\n\npub mod client;\npub mod dane;\npub mod delivery;\npub mod local;\npub mod lookup;\npub mod mta_sts;\npub mod session;\n\npub(super) enum DeliveryResult {\n    Domain {\n        status: Status<HostResponse<Box<str>>, ErrorDetails>,\n        rcpt_idxs: Vec<usize>,\n    },\n    Account {\n        status: Status<HostResponse<Box<str>>, ErrorDetails>,\n        rcpt_idx: usize,\n    },\n    RateLimited {\n        rcpt_idxs: Vec<usize>,\n        retry_at: u64,\n    },\n}\n\nimpl Status<HostResponse<Box<str>>, ErrorDetails> {\n    pub fn from_smtp_error(hostname: &str, command: &str, err: mail_send::Error) -> Self {\n        match err {\n            mail_send::Error::Io(_)\n            | mail_send::Error::Tls(_)\n            | mail_send::Error::Base64(_)\n            | mail_send::Error::UnparseableReply\n            | mail_send::Error::AuthenticationFailed(_)\n            | mail_send::Error::MissingCredentials\n            | mail_send::Error::MissingMailFrom\n            | mail_send::Error::MissingRcptTo\n            | mail_send::Error::Timeout => Status::TemporaryFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::ConnectionError(err.to_string().into_boxed_str()),\n            }),\n\n            mail_send::Error::UnexpectedReply(response) => {\n                if response.severity() == Severity::PermanentNegativeCompletion {\n                    Status::PermanentFailure(ErrorDetails {\n                        entity: hostname.into(),\n                        details: Error::UnexpectedResponse(UnexpectedResponse {\n                            command: command.trim().into(),\n                            response: response.into_box(),\n                        }),\n                    })\n                } else {\n                    Status::TemporaryFailure(ErrorDetails {\n                        entity: hostname.into(),\n                        details: Error::UnexpectedResponse(UnexpectedResponse {\n                            command: command.trim().into(),\n                            response: response.into_box(),\n                        }),\n                    })\n                }\n            }\n\n            mail_send::Error::Auth(_)\n            | mail_send::Error::UnsupportedAuthMechanism\n            | mail_send::Error::InvalidTLSName\n            | mail_send::Error::MissingStartTls => Status::PermanentFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::ConnectionError(err.to_string().into_boxed_str()),\n            }),\n        }\n    }\n\n    pub fn from_starttls_error(hostname: &str, response: Option<Response<Box<str>>>) -> Self {\n        let entity = hostname.into();\n        if let Some(response) = response {\n            if response.severity() == Severity::PermanentNegativeCompletion {\n                Status::PermanentFailure(ErrorDetails {\n                    entity,\n                    details: Error::UnexpectedResponse(UnexpectedResponse {\n                        command: \"STARTTLS\".into(),\n                        response,\n                    }),\n                })\n            } else {\n                Status::TemporaryFailure(ErrorDetails {\n                    entity,\n                    details: Error::UnexpectedResponse(UnexpectedResponse {\n                        command: \"STARTTLS\".into(),\n                        response,\n                    }),\n                })\n            }\n        } else {\n            Status::PermanentFailure(ErrorDetails {\n                entity,\n                details: Error::TlsError(\"STARTTLS not advertised by host.\".into()),\n            })\n        }\n    }\n\n    pub fn from_tls_error(hostname: &str, err: mail_send::Error) -> Self {\n        match err {\n            mail_send::Error::InvalidTLSName => Status::PermanentFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::TlsError(\"Invalid hostname\".into()),\n            }),\n            mail_send::Error::Timeout => Status::TemporaryFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::TlsError(\"TLS handshake timed out\".into()),\n            }),\n            mail_send::Error::Tls(err) => Status::TemporaryFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::TlsError(format!(\"Handshake failed: {err}\").into_boxed_str()),\n            }),\n            mail_send::Error::Io(err) => Status::TemporaryFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::TlsError(format!(\"I/O error: {err}\").into_boxed_str()),\n            }),\n            _ => Status::PermanentFailure(ErrorDetails {\n                entity: hostname.into(),\n                details: Error::TlsError(\"Other TLS error\".into()),\n            }),\n        }\n    }\n\n    pub fn timeout(hostname: &str, stage: &str) -> Self {\n        Status::TemporaryFailure(ErrorDetails {\n            entity: hostname.into(),\n            details: Error::ConnectionError(format!(\"Timeout while {stage}\").into_boxed_str()),\n        })\n    }\n\n    pub fn local_error() -> Self {\n        Status::TemporaryFailure(ErrorDetails {\n            entity: \"localhost\".into(),\n            details: Error::ConnectionError(\"Could not deliver message locally.\".into()),\n        })\n    }\n\n    pub fn from_mail_auth_error(entity: &str, err: mail_auth::Error) -> Self {\n        match &err {\n            mail_auth::Error::DnsRecordNotFound(code) => Status::PermanentFailure(ErrorDetails {\n                entity: entity.into(),\n                details: Error::DnsError(format!(\"Domain not found: {code:?}\").into_boxed_str()),\n            }),\n            _ => Status::TemporaryFailure(ErrorDetails {\n                entity: entity.into(),\n                details: Error::DnsError(err.to_string().into_boxed_str()),\n            }),\n        }\n    }\n\n    pub fn from_mta_sts_error(entity: &str, err: mta_sts::Error) -> Self {\n        match &err {\n            mta_sts::Error::Dns(err) => match err {\n                mail_auth::Error::DnsRecordNotFound(code) => {\n                    Status::PermanentFailure(ErrorDetails {\n                        entity: entity.into(),\n                        details: Error::MtaStsError(\n                            format!(\"Record not found: {code:?}\").into_boxed_str(),\n                        ),\n                    })\n                }\n                mail_auth::Error::InvalidRecordType => Status::PermanentFailure(ErrorDetails {\n                    entity: entity.into(),\n                    details: Error::MtaStsError(\"Failed to parse MTA-STS DNS record.\".into()),\n                }),\n                _ => Status::TemporaryFailure(ErrorDetails {\n                    entity: entity.into(),\n                    details: Error::MtaStsError(\n                        format!(\"DNS lookup error: {err}\").into_boxed_str(),\n                    ),\n                }),\n            },\n            mta_sts::Error::Http(err) => {\n                if err.is_timeout() {\n                    Status::TemporaryFailure(ErrorDetails {\n                        entity: entity.into(),\n                        details: Error::MtaStsError(\"Timeout fetching policy.\".into()),\n                    })\n                } else if err.is_connect() {\n                    Status::TemporaryFailure(ErrorDetails {\n                        entity: entity.into(),\n                        details: Error::MtaStsError(\"Could not reach policy host.\".into()),\n                    })\n                } else if err.is_status()\n                    & err\n                        .status()\n                        .is_some_and(|s| s == reqwest::StatusCode::NOT_FOUND)\n                {\n                    Status::PermanentFailure(ErrorDetails {\n                        entity: entity.into(),\n                        details: Error::MtaStsError(\"Policy not found.\".into()),\n                    })\n                } else {\n                    Status::TemporaryFailure(ErrorDetails {\n                        entity: entity.into(),\n                        details: Error::MtaStsError(\"Failed to fetch policy.\".into()),\n                    })\n                }\n            }\n            mta_sts::Error::InvalidPolicy(err) => Status::PermanentFailure(ErrorDetails {\n                entity: entity.into(),\n                details: Error::MtaStsError(\n                    format!(\"Failed to parse policy: {err}\").into_boxed_str(),\n                ),\n            }),\n        }\n    }\n}\n\n#[derive(Debug)]\npub enum NextHop<'x> {\n    Relay(&'x RelayConfig),\n    MX {\n        is_implicit: bool,\n        host: &'x str,\n        config: &'x MxConfig,\n    },\n}\n\nimpl NextHop<'_> {\n    #[inline(always)]\n    pub fn hostname(&self) -> &str {\n        match self {\n            NextHop::MX { host, .. } => {\n                if let Some(host) = host.strip_suffix('.') {\n                    host\n                } else {\n                    host\n                }\n            }\n            NextHop::Relay(host) => host.address.as_str(),\n        }\n    }\n\n    #[inline(always)]\n    pub fn fqdn_hostname(&self) -> Cow<'_, str> {\n        match self {\n            NextHop::MX { host, .. } => {\n                if !host.ends_with('.') {\n                    format!(\"{host}.\").into()\n                } else {\n                    (*host).into()\n                }\n            }\n            NextHop::Relay(host) => host.address.as_str().into(),\n        }\n    }\n\n    #[inline(always)]\n    pub fn max_multi_homed(&self) -> usize {\n        match self {\n            NextHop::MX { config, .. } => config.max_multi_homed,\n            NextHop::Relay(_) => 10,\n        }\n    }\n\n    #[inline(always)]\n    pub fn ip_lookup_strategy(&self) -> IpLookupStrategy {\n        match self {\n            NextHop::MX { config, .. } => config.ip_lookup_strategy,\n            NextHop::Relay(_) => IpLookupStrategy::Ipv4thenIpv6,\n        }\n    }\n\n    #[inline(always)]\n    fn port(&self) -> u16 {\n        match self {\n            #[cfg(feature = \"test_mode\")]\n            NextHop::MX { .. } => 9925,\n            #[cfg(not(feature = \"test_mode\"))]\n            NextHop::MX { .. } => 25,\n            NextHop::Relay(host) => host.port,\n        }\n    }\n\n    #[inline(always)]\n    fn credentials(&self) -> Option<&Credentials<String>> {\n        match self {\n            NextHop::MX { .. } => None,\n            NextHop::Relay(host) => host.auth.as_ref(),\n        }\n    }\n\n    #[inline(always)]\n    fn allow_invalid_certs(&self) -> bool {\n        #[cfg(feature = \"test_mode\")]\n        {\n            true\n        }\n        #[cfg(not(feature = \"test_mode\"))]\n        match self {\n            NextHop::MX { .. } => false,\n            NextHop::Relay(host) => host.tls_allow_invalid_certs,\n        }\n    }\n\n    #[inline(always)]\n    fn implicit_tls(&self) -> bool {\n        match self {\n            NextHop::MX { .. } => false,\n            NextHop::Relay(host) => host.tls_implicit,\n        }\n    }\n\n    #[inline(always)]\n    fn is_smtp(&self) -> bool {\n        match self {\n            NextHop::MX { .. } => true,\n            NextHop::Relay(host) => host.protocol == ServerProtocol::Smtp,\n        }\n    }\n}\n\nimpl DeliveryResult {\n    pub fn domain(\n        status: Status<HostResponse<Box<str>>, ErrorDetails>,\n        rcpt_idxs: Vec<usize>,\n    ) -> Self {\n        DeliveryResult::Domain { status, rcpt_idxs }\n    }\n\n    pub fn rate_limited(rcpt_idxs: Vec<usize>, retry_at: u64) -> Self {\n        DeliveryResult::RateLimited {\n            rcpt_idxs,\n            retry_at,\n        }\n    }\n\n    pub fn account(status: Status<HostResponse<Box<str>>, ErrorDetails>, rcpt_idx: usize) -> Self {\n        DeliveryResult::Account { status, rcpt_idx }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/mta_sts/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fmt::Display, sync::Arc, time::Duration};\n\n#[cfg(feature = \"test_mode\")]\npub static STS_TEST_POLICY: parking_lot::Mutex<Vec<u8>> = parking_lot::Mutex::new(Vec::new());\n\nuse common::{Server, config::smtp::resolver::Policy};\nuse mail_auth::{mta_sts::MtaSts, report::tlsrpt::ResultType};\n\nuse super::{Error, parse::ParsePolicy};\n\n#[cfg(not(feature = \"test_mode\"))]\nuse utils::HttpLimitResponse;\n\n#[cfg(not(feature = \"test_mode\"))]\nconst MAX_POLICY_SIZE: usize = 1024 * 1024;\n\npub trait MtaStsLookup: Sync + Send {\n    fn lookup_mta_sts_policy(\n        &self,\n        domain: &str,\n        timeout: Duration,\n    ) -> impl std::future::Future<Output = Result<Arc<Policy>, Error>> + Send;\n}\n\n#[allow(unused_variables)]\nimpl MtaStsLookup for Server {\n    async fn lookup_mta_sts_policy(\n        &self,\n        domain: &str,\n        timeout: Duration,\n    ) -> Result<Arc<Policy>, Error> {\n        // Lookup MTA-STS TXT record\n        let record = match self\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .txt_lookup::<MtaSts>(\n                format!(\"_mta-sts.{domain}.\"),\n                Some(&self.inner.cache.dns_txt),\n            )\n            .await\n        {\n            Ok(record) => record,\n            Err(err) => {\n                // Return the cached policy in case of failure\n                return if let Some(value) = self.inner.cache.dbs_mta_sts.get(domain) {\n                    Ok(value)\n                } else {\n                    Err(err.into())\n                };\n            }\n        };\n\n        // Check if the policy has been cached\n        if let Some(value) = self.inner.cache.dbs_mta_sts.get(domain)\n            && value.id == record.id\n        {\n            return Ok(value);\n        }\n\n        // Fetch policy\n        #[cfg(not(feature = \"test_mode\"))]\n        let bytes = reqwest::Client::builder()\n            .user_agent(common::USER_AGENT)\n            .timeout(timeout)\n            .redirect(reqwest::redirect::Policy::none())\n            .build()?\n            .get(format!(\"https://mta-sts.{domain}/.well-known/mta-sts.txt\"))\n            .send()\n            .await?\n            .bytes_with_limit(MAX_POLICY_SIZE)\n            .await?\n            .ok_or_else(|| Error::InvalidPolicy(\"Policy too large\".to_string()))?;\n        #[cfg(feature = \"test_mode\")]\n        let bytes = STS_TEST_POLICY.lock().clone();\n\n        // Parse policy\n        let policy = Arc::new(Policy::parse(\n            std::str::from_utf8(&bytes).map_err(|err| Error::InvalidPolicy(err.to_string()))?,\n            record.id.clone(),\n        )?);\n\n        self.inner.cache.dbs_mta_sts.insert(\n            domain.to_string(),\n            policy.clone(),\n            Duration::from_secs(if (3600..31557600).contains(&policy.max_age) {\n                policy.max_age\n            } else {\n                86400\n            }),\n        );\n\n        Ok(policy)\n    }\n}\n\nimpl From<&Error> for ResultType {\n    fn from(err: &Error) -> Self {\n        match &err {\n            Error::InvalidPolicy(_) => ResultType::StsPolicyInvalid,\n            _ => ResultType::StsPolicyFetchError,\n        }\n    }\n}\n\nimpl Display for Error {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Error::Dns(err) => match err {\n                mail_auth::Error::DnsRecordNotFound(code) => {\n                    write!(f, \"Record not found: {code:?}\")\n                }\n                mail_auth::Error::InvalidRecordType => {\n                    f.write_str(\"Failed to parse MTA-STS DNS record.\")\n                }\n                _ => write!(f, \"DNS lookup error: {err}\"),\n            },\n            Error::Http(err) => {\n                if err.is_timeout() {\n                    f.write_str(\"Timeout fetching policy.\")\n                } else if err.is_connect() {\n                    f.write_str(\"Could not reach policy host.\")\n                } else if err.is_status() && (err.status() == Some(reqwest::StatusCode::NOT_FOUND))\n                {\n                    f.write_str(\"Policy not found.\")\n                } else {\n                    f.write_str(\"Failed to fetch policy.\")\n                }\n            }\n            Error::InvalidPolicy(err) => write!(f, \"Failed to parse policy: {err}\"),\n        }\n    }\n}\n\nimpl From<mail_auth::Error> for Error {\n    fn from(value: mail_auth::Error) -> Self {\n        Error::Dns(value)\n    }\n}\n\nimpl From<reqwest::Error> for Error {\n    fn from(value: reqwest::Error) -> Self {\n        Error::Http(value)\n    }\n}\n\nimpl From<String> for Error {\n    fn from(value: String) -> Self {\n        Error::InvalidPolicy(value)\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/mta_sts/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod lookup;\npub mod parse;\npub mod verify;\n\n#[derive(Debug)]\npub enum Error {\n    Dns(mail_auth::Error),\n    Http(reqwest::Error),\n    InvalidPolicy(String),\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/mta_sts/parse.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::config::smtp::resolver::{Mode, MxPattern, Policy};\n\npub trait ParsePolicy {\n    fn parse(data: &str, id: String) -> Result<Self, String>\n    where\n        Self: Sized;\n}\n\nimpl ParsePolicy for Policy {\n    fn parse(mut data: &str, id: String) -> Result<Policy, String> {\n        let mut mode = Mode::None;\n        let mut max_age: u64 = 86400;\n        let mut mx = Vec::new();\n\n        while !data.is_empty() {\n            if let Some((key, next_data)) = data.split_once(':') {\n                let value = if let Some((value, next_data)) = next_data.split_once('\\n') {\n                    data = next_data;\n                    value.trim()\n                } else {\n                    data = \"\";\n                    next_data.trim()\n                };\n                match key.trim() {\n                    \"mx\" => {\n                        if let Some(suffix) = value.strip_prefix(\"*.\") {\n                            if !suffix.is_empty() {\n                                mx.push(MxPattern::StartsWith(suffix.to_lowercase()));\n                            }\n                        } else if !value.is_empty() {\n                            mx.push(MxPattern::Equals(value.to_lowercase()));\n                        }\n                    }\n                    \"max_age\" => {\n                        if let Ok(value) = value.parse() {\n                            max_age = value;\n                        }\n                    }\n                    \"mode\" => {\n                        mode = match value {\n                            \"enforce\" => Mode::Enforce,\n                            \"testing\" => Mode::Testing,\n                            \"none\" => Mode::None,\n                            _ => return Err(format!(\"Unsupported mode {value:?}.\")),\n                        };\n                    }\n                    \"version\" => {\n                        if !value.eq_ignore_ascii_case(\"STSv1\") {\n                            return Err(format!(\"Unsupported version {value:?}.\"));\n                        }\n                    }\n                    _ => (),\n                }\n            } else {\n                break;\n            }\n        }\n\n        if !mx.is_empty() {\n            Ok(Policy {\n                id,\n                mode,\n                mx,\n                max_age,\n            })\n        } else {\n            Err(\"No 'mx' entries found.\".to_string())\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/mta_sts/verify.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::config::smtp::resolver::{Mode, MxPattern, Policy};\n\npub trait VerifyPolicy {\n    fn verify(&self, mx_host: &str) -> bool;\n    fn enforce(&self) -> bool;\n}\n\nimpl VerifyPolicy for Policy {\n    fn verify(&self, mx_host: &str) -> bool {\n        if self.mode != Mode::None {\n            for mx_pattern in &self.mx {\n                match mx_pattern {\n                    MxPattern::Equals(host) => {\n                        if host == mx_host {\n                            return true;\n                        }\n                    }\n                    MxPattern::StartsWith(domain) => {\n                        if let Some((_, suffix)) = mx_host.split_once('.')\n                            && suffix == domain\n                        {\n                            return true;\n                        }\n                    }\n                }\n            }\n\n            false\n        } else {\n            true\n        }\n    }\n\n    fn enforce(&self) -> bool {\n        self.mode == Mode::Enforce\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/outbound/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::client::SmtpClient;\nuse crate::outbound::DeliveryResult;\nuse crate::outbound::client::{BoxResponse, from_error_status, from_mail_send_error};\nuse crate::queue::{Error, MessageWrapper, Recipient, Status};\nuse crate::queue::{ErrorDetails, HostResponse, UnexpectedResponse};\nuse common::Server;\nuse common::config::smtp::queue::ConnectionStrategy;\nuse mail_send::Credentials;\nuse smtp_proto::{\n    EXT_CHUNKING, EXT_DSN, EXT_REQUIRE_TLS, EXT_SIZE, EXT_SMTP_UTF8, EhloResponse, MAIL_REQUIRETLS,\n    MAIL_RET_FULL, MAIL_RET_HDRS, MAIL_SMTPUTF8, RCPT_NOTIFY_DELAY, RCPT_NOTIFY_FAILURE,\n    RCPT_NOTIFY_NEVER, RCPT_NOTIFY_SUCCESS, Severity,\n};\nuse std::{fmt::Write, time::Instant};\nuse tokio::io::{AsyncRead, AsyncWrite};\nuse trc::DeliveryEvent;\n\npub struct SessionParams<'x> {\n    pub server: &'x Server,\n    pub hostname: &'x str,\n    pub credentials: Option<&'x Credentials<String>>,\n    pub capabilities: Option<EhloResponse<String>>,\n    pub is_smtp: bool,\n    pub local_hostname: &'x str,\n    pub conn_strategy: &'x ConnectionStrategy,\n    pub session_id: u64,\n}\n\nimpl MessageWrapper {\n    pub(super) async fn deliver<T: AsyncRead + AsyncWrite + Unpin>(\n        &self,\n        mut smtp_client: SmtpClient<T>,\n        rcpt_idxs: Vec<usize>,\n        statuses: &mut Vec<DeliveryResult>,\n        mut params: SessionParams<'_>,\n    ) {\n        // Obtain capabilities\n        let time = Instant::now();\n        let capabilities = if let Some(capabilities) = params.capabilities.take() {\n            capabilities\n        } else {\n            match smtp_client.say_helo(&params).await {\n                Ok(capabilities) => {\n                    trc::event!(\n                        Delivery(DeliveryEvent::Ehlo),\n                        SpanId = params.session_id,\n                        Hostname = params.hostname.to_string(),\n                        Details = capabilities.capabilities(),\n                        Elapsed = time.elapsed(),\n                    );\n\n                    capabilities\n                }\n                Err(status) => {\n                    trc::event!(\n                        Delivery(DeliveryEvent::EhloRejected),\n                        SpanId = params.session_id,\n                        Hostname = params.hostname.to_string(),\n                        CausedBy = from_error_status(&status),\n                        Elapsed = time.elapsed(),\n                    );\n                    smtp_client.quit().await;\n                    statuses.push(DeliveryResult::domain(status, rcpt_idxs));\n                    return;\n                }\n            }\n        };\n\n        // Authenticate\n        if let Some(credentials) = params.credentials {\n            let time = Instant::now();\n            if let Err(err) = smtp_client.authenticate(credentials, &capabilities).await {\n                trc::event!(\n                    Delivery(DeliveryEvent::AuthFailed),\n                    SpanId = params.session_id,\n                    Hostname = params.hostname.to_string(),\n                    CausedBy = from_mail_send_error(&err),\n                    Elapsed = time.elapsed(),\n                );\n\n                smtp_client.quit().await;\n                statuses.push(DeliveryResult::domain(\n                    Status::from_smtp_error(params.hostname, \"AUTH ...\", err),\n                    rcpt_idxs,\n                ));\n                return;\n            }\n\n            trc::event!(\n                Delivery(DeliveryEvent::Auth),\n                SpanId = params.session_id,\n                Hostname = params.hostname.to_string(),\n                Elapsed = time.elapsed(),\n            );\n\n            // Refresh capabilities\n            // Disabled as some SMTP servers deauthenticate after EHLO\n            /*capabilities = match say_helo(&mut smtp_client, &params).await {\n                Ok(capabilities) => capabilities,\n                Err(status) => {\n                    trc::event!(\n\n                        context = \"ehlo\",\n                        event = \"rejected\",\n                        mx = &params.hostname,\n                        reason = %status,\n                    );\n                    smtp_client.quit().await;\n                    return status;\n                }\n            };*/\n        }\n\n        // MAIL FROM\n        let time = Instant::now();\n        smtp_client.timeout = params.conn_strategy.timeout_mail;\n        let cmd = self.build_mail_from(&capabilities);\n        match smtp_client.cmd(cmd.as_bytes()).await.and_then(|r| {\n            if r.is_positive_completion() {\n                Ok(r)\n            } else {\n                Err(mail_send::Error::UnexpectedReply(r))\n            }\n        }) {\n            Ok(response) => {\n                trc::event!(\n                    Delivery(DeliveryEvent::MailFrom),\n                    SpanId = params.session_id,\n                    Hostname = params.hostname.to_string(),\n                    From = self.message.return_path.to_string(),\n                    Code = response.code,\n                    Details = response.message.to_string(),\n                    Elapsed = time.elapsed(),\n                );\n            }\n            Err(err) => {\n                trc::event!(\n                    Delivery(DeliveryEvent::MailFromRejected),\n                    SpanId = params.session_id,\n                    Hostname = params.hostname.to_string(),\n                    CausedBy = from_mail_send_error(&err),\n                    Elapsed = time.elapsed(),\n                );\n\n                smtp_client.quit().await;\n                statuses.push(DeliveryResult::domain(\n                    Status::from_smtp_error(params.hostname, &cmd, err),\n                    rcpt_idxs,\n                ));\n                return;\n            }\n        }\n\n        // RCPT TO\n        let mut accepted_rcpts = Vec::new();\n        smtp_client.timeout = params.conn_strategy.timeout_rcpt;\n        for rcpt_idx in &rcpt_idxs {\n            let time = Instant::now();\n            let rcpt = &self.message.recipients[*rcpt_idx];\n            if matches!(\n                &rcpt.status,\n                Status::Completed(_) | Status::PermanentFailure(_)\n            ) {\n                continue;\n            }\n\n            let cmd = self.build_rcpt_to(rcpt, &capabilities);\n            match smtp_client.cmd(cmd.as_bytes()).await {\n                Ok(response) => match response.severity() {\n                    Severity::PositiveCompletion => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::RcptTo),\n                            SpanId = params.session_id,\n                            Hostname = params.hostname.to_string(),\n                            To = rcpt.address().to_string(),\n                            Code = response.code,\n                            Details = response.message.to_string(),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        accepted_rcpts.push((\n                            rcpt,\n                            rcpt_idx,\n                            Status::Completed(HostResponse {\n                                hostname: params.hostname.into(),\n                                response: response.into_box(),\n                            }),\n                        ));\n                    }\n                    severity => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::RcptToRejected),\n                            SpanId = params.session_id,\n                            Hostname = params.hostname.to_string(),\n                            To = rcpt.address().to_string(),\n                            Code = response.code,\n                            Details = response.message.to_string(),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        let response = ErrorDetails {\n                            entity: params.hostname.into(),\n                            details: Error::UnexpectedResponse(UnexpectedResponse {\n                                command: cmd.trim().into(),\n                                response: response.into_box(),\n                            }),\n                        };\n                        statuses.push(DeliveryResult::account(\n                            if severity == Severity::PermanentNegativeCompletion {\n                                Status::PermanentFailure(response)\n                            } else {\n                                Status::TemporaryFailure(response)\n                            },\n                            *rcpt_idx,\n                        ));\n                    }\n                },\n                Err(err) => {\n                    trc::event!(\n                        Delivery(DeliveryEvent::RcptToFailed),\n                        SpanId = params.session_id,\n                        Hostname = params.hostname.to_string(),\n                        To = rcpt.address().to_string(),\n                        CausedBy = from_mail_send_error(&err),\n                        Elapsed = time.elapsed(),\n                    );\n\n                    // Something went wrong, abort.\n                    smtp_client.quit().await;\n                    statuses.push(DeliveryResult::domain(\n                        Status::from_smtp_error(params.hostname, \"\", err),\n                        rcpt_idxs,\n                    ));\n                    return;\n                }\n            }\n        }\n\n        // Send message\n        if !accepted_rcpts.is_empty() {\n            let time = Instant::now();\n            let bdat_cmd = capabilities\n                .has_capability(EXT_CHUNKING)\n                .then(|| format!(\"BDAT {} LAST\\r\\n\", self.message.size));\n\n            if let Err(status) = smtp_client.send_message(self, &bdat_cmd, &params).await {\n                trc::event!(\n                    Delivery(DeliveryEvent::MessageRejected),\n                    SpanId = params.session_id,\n                    Hostname = params.hostname.to_string(),\n                    CausedBy = from_error_status(&status),\n                    Elapsed = time.elapsed(),\n                );\n\n                smtp_client.quit().await;\n                statuses.push(DeliveryResult::domain(status, rcpt_idxs));\n                return;\n            }\n\n            if params.is_smtp {\n                // Handle SMTP response\n                match smtp_client\n                    .read_smtp_data_response(params.hostname, &bdat_cmd)\n                    .await\n                {\n                    Ok(response) => {\n                        // Mark recipients as delivered\n                        if response.code() == 250 {\n                            for (rcpt, rcpt_idx, status) in accepted_rcpts {\n                                trc::event!(\n                                    Delivery(DeliveryEvent::Delivered),\n                                    SpanId = params.session_id,\n                                    Hostname = params.hostname.to_string(),\n                                    To = rcpt.address().to_string(),\n                                    Code = response.code,\n                                    Details = response.message.to_string(),\n                                    Elapsed = time.elapsed(),\n                                );\n\n                                statuses.push(DeliveryResult::account(status, *rcpt_idx));\n                            }\n                        } else {\n                            trc::event!(\n                                Delivery(DeliveryEvent::MessageRejected),\n                                SpanId = params.session_id,\n                                Hostname = params.hostname.to_string(),\n                                Code = response.code,\n                                Details = response.message.to_string(),\n                                Elapsed = time.elapsed(),\n                            );\n\n                            smtp_client.quit().await;\n                            statuses.push(DeliveryResult::domain(\n                                Status::from_smtp_error(\n                                    params.hostname,\n                                    bdat_cmd.as_deref().unwrap_or(\"DATA\"),\n                                    mail_send::Error::UnexpectedReply(response),\n                                ),\n                                rcpt_idxs,\n                            ));\n                            return;\n                        }\n                    }\n                    Err(status) => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::MessageRejected),\n                            SpanId = params.session_id,\n                            Hostname = params.hostname.to_string(),\n                            CausedBy = from_error_status(&status),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        smtp_client.quit().await;\n                        statuses.push(DeliveryResult::domain(status, rcpt_idxs));\n                        return;\n                    }\n                }\n            } else {\n                // Handle LMTP responses\n                match smtp_client\n                    .read_lmtp_data_response(params.hostname, accepted_rcpts.len())\n                    .await\n                {\n                    Ok(responses) => {\n                        for ((rcpt, rcpt_idx, _), response) in\n                            accepted_rcpts.into_iter().zip(responses)\n                        {\n                            let status: Status<HostResponse<Box<str>>, ErrorDetails> =\n                                match response.severity() {\n                                    Severity::PositiveCompletion => {\n                                        trc::event!(\n                                            Delivery(DeliveryEvent::Delivered),\n                                            SpanId = params.session_id,\n                                            Hostname = params.hostname.to_string(),\n                                            To = rcpt.address().to_string(),\n                                            Code = response.code,\n                                            Details = response.message.to_string(),\n                                            Elapsed = time.elapsed(),\n                                        );\n\n                                        Status::Completed(HostResponse {\n                                            hostname: params.hostname.into(),\n                                            response,\n                                        })\n                                    }\n                                    severity => {\n                                        trc::event!(\n                                            Delivery(DeliveryEvent::RcptToRejected),\n                                            SpanId = params.session_id,\n                                            Hostname = params.hostname.to_string(),\n                                            To = rcpt.address().to_string(),\n                                            Code = response.code,\n                                            Details = response.message.to_string(),\n                                            Elapsed = time.elapsed(),\n                                        );\n\n                                        let response = ErrorDetails {\n                                            entity: params.hostname.into(),\n                                            details: Error::UnexpectedResponse(\n                                                UnexpectedResponse {\n                                                    command: bdat_cmd\n                                                        .as_deref()\n                                                        .unwrap_or(\"DATA\")\n                                                        .into(),\n                                                    response,\n                                                },\n                                            ),\n                                        };\n                                        if severity == Severity::PermanentNegativeCompletion {\n                                            Status::PermanentFailure(response)\n                                        } else {\n                                            Status::TemporaryFailure(response)\n                                        }\n                                    }\n                                };\n\n                            statuses.push(DeliveryResult::account(status, *rcpt_idx));\n                        }\n                    }\n                    Err(status) => {\n                        trc::event!(\n                            Delivery(DeliveryEvent::MessageRejected),\n                            SpanId = params.session_id,\n                            Hostname = params.hostname.to_string(),\n                            CausedBy = from_error_status(&status),\n                            Elapsed = time.elapsed(),\n                        );\n\n                        smtp_client.quit().await;\n                        statuses.push(DeliveryResult::domain(status, rcpt_idxs));\n                        return;\n                    }\n                }\n            }\n        }\n\n        smtp_client.quit().await;\n    }\n\n    fn build_mail_from(&self, capabilities: &EhloResponse<String>) -> String {\n        let mut mail_from = String::with_capacity(self.message.return_path.len() + 60);\n        let _ = write!(mail_from, \"MAIL FROM:<{}>\", self.message.return_path);\n        if capabilities.has_capability(EXT_SIZE) {\n            let _ = write!(mail_from, \" SIZE={}\", self.message.size);\n        }\n        if self.has_flag(MAIL_REQUIRETLS) & capabilities.has_capability(EXT_REQUIRE_TLS) {\n            mail_from.push_str(\" REQUIRETLS\");\n        }\n        if self.has_flag(MAIL_SMTPUTF8) & capabilities.has_capability(EXT_SMTP_UTF8) {\n            mail_from.push_str(\" SMTPUTF8\");\n        }\n        if capabilities.has_capability(EXT_DSN) {\n            if self.has_flag(MAIL_RET_FULL) {\n                mail_from.push_str(\" RET=FULL\");\n            } else if self.has_flag(MAIL_RET_HDRS) {\n                mail_from.push_str(\" RET=HDRS\");\n            }\n            if let Some(env_id) = &self.message.env_id {\n                let _ = write!(mail_from, \" ENVID={env_id}\");\n            }\n        }\n\n        mail_from.push_str(\"\\r\\n\");\n        mail_from\n    }\n\n    fn build_rcpt_to(&self, rcpt: &Recipient, capabilities: &EhloResponse<String>) -> String {\n        let mut rcpt_to = String::with_capacity(rcpt.address().len() + 60);\n        let _ = write!(rcpt_to, \"RCPT TO:<{}>\", rcpt.address());\n        if capabilities.has_capability(EXT_DSN) {\n            if rcpt.has_flag(RCPT_NOTIFY_SUCCESS | RCPT_NOTIFY_FAILURE | RCPT_NOTIFY_DELAY) {\n                rcpt_to.push_str(\" NOTIFY=\");\n                let mut add_comma = if rcpt.has_flag(RCPT_NOTIFY_SUCCESS) {\n                    rcpt_to.push_str(\"SUCCESS\");\n                    true\n                } else {\n                    false\n                };\n                if rcpt.has_flag(RCPT_NOTIFY_DELAY) {\n                    if add_comma {\n                        rcpt_to.push(',');\n                    } else {\n                        add_comma = true;\n                    }\n                    rcpt_to.push_str(\"DELAY\");\n                }\n                if rcpt.has_flag(RCPT_NOTIFY_FAILURE) {\n                    if add_comma {\n                        rcpt_to.push(',');\n                    }\n                    rcpt_to.push_str(\"FAILURE\");\n                }\n            } else if rcpt.has_flag(RCPT_NOTIFY_NEVER) {\n                rcpt_to.push_str(\" NOTIFY=NEVER\");\n            }\n        }\n        rcpt_to.push_str(\"\\r\\n\");\n        rcpt_to\n    }\n\n    #[inline(always)]\n    pub fn has_flag(&self, flag: u64) -> bool {\n        (self.message.flags & flag) != 0\n    }\n}\n\nimpl Recipient {\n    #[inline(always)]\n    pub fn has_flag(&self, flag: u64) -> bool {\n        (self.flags & flag) != 0\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/queue/dsn.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::spool::SmtpSpool;\nuse super::{\n    Error, ErrorDetails, HostResponse, Message, MessageSource, QueueEnvelope, RCPT_DSN_SENT,\n    Recipient, Status,\n};\nuse crate::queue::{MessageWrapper, UnexpectedResponse};\nuse crate::reporting::SmtpReporting;\nuse common::Server;\nuse mail_builder::MessageBuilder;\nuse mail_builder::headers::HeaderType;\nuse mail_builder::headers::content_type::ContentType;\nuse mail_builder::mime::{BodyPart, MimePart, make_boundary};\nuse mail_parser::DateTime;\nuse smtp_proto::{\n    RCPT_NOTIFY_DELAY, RCPT_NOTIFY_FAILURE, RCPT_NOTIFY_NEVER, RCPT_NOTIFY_SUCCESS, Response,\n};\nuse std::fmt::Write;\nuse std::future::Future;\nuse store::write::now;\n\npub trait SendDsn: Sync + Send {\n    fn send_dsn(&self, message: &mut MessageWrapper) -> impl Future<Output = ()> + Send;\n    fn log_dsn(&self, message: &MessageWrapper) -> impl Future<Output = ()> + Send;\n}\n\nimpl SendDsn for Server {\n    async fn send_dsn(&self, message: &mut MessageWrapper) {\n        // Send DSN events\n        self.log_dsn(message).await;\n\n        if !message.message.return_path.is_empty() {\n            // Build DSN\n            if let Some(dsn) = message.build_dsn(self).await {\n                let mut dsn_message = self.new_message(\"\", message.span_id);\n                dsn_message\n                    .add_recipient(message.message.return_path.as_ref(), self)\n                    .await;\n\n                // Sign message\n                let signature = self\n                    .sign_message(message, &self.core.smtp.queue.dsn.sign, &dsn)\n                    .await;\n\n                // Queue DSN\n                dsn_message\n                    .queue(\n                        signature.as_deref(),\n                        &dsn,\n                        message.span_id,\n                        self,\n                        MessageSource::Dsn,\n                    )\n                    .await;\n            }\n        } else {\n            // Handle double bounce\n            message.handle_double_bounce();\n        }\n    }\n\n    async fn log_dsn(&self, message: &MessageWrapper) {\n        let now = now();\n\n        for rcpt in &message.message.recipients {\n            if rcpt.has_flag(RCPT_DSN_SENT) {\n                continue;\n            }\n\n            match &rcpt.status {\n                Status::Completed(response) => {\n                    trc::event!(\n                        Delivery(trc::DeliveryEvent::DsnSuccess),\n                        SpanId = message.span_id,\n                        To = rcpt.address.clone(),\n                        Hostname = response.hostname.clone(),\n                        Code = response.response.code,\n                        Details = response.response.message.to_string(),\n                    );\n                }\n                Status::TemporaryFailure(response) if rcpt.notify.due <= now => {\n                    trc::event!(\n                        Delivery(trc::DeliveryEvent::DsnTempFail),\n                        SpanId = message.span_id,\n                        To = rcpt.address.clone(),\n                        Hostname = response.entity.clone(),\n                        Details = response.details.to_string(),\n                        NextRetry = trc::Value::Timestamp(rcpt.retry.due),\n                        Expires = rcpt\n                            .expiration_time(message.message.created)\n                            .map(trc::Value::Timestamp),\n                        Total = rcpt.retry.inner,\n                    );\n                }\n                Status::PermanentFailure(response) => {\n                    trc::event!(\n                        Delivery(trc::DeliveryEvent::DsnPermFail),\n                        SpanId = message.span_id,\n                        To = rcpt.address.clone(),\n                        Hostname = response.entity.clone(),\n                        Details = response.details.to_string(),\n                        Total = rcpt.retry.inner,\n                    );\n                }\n                Status::Scheduled if rcpt.notify.due <= now => {\n                    trc::event!(\n                        Delivery(trc::DeliveryEvent::DsnTempFail),\n                        SpanId = message.span_id,\n                        To = rcpt.address.clone(),\n                        Details = \"Concurrency limited\",\n                        NextRetry = trc::Value::Timestamp(rcpt.retry.due),\n                        Expires = rcpt\n                            .expiration_time(message.message.created)\n                            .map(trc::Value::Timestamp),\n                        Total = rcpt.retry.inner,\n                    );\n                }\n                _ => continue,\n            }\n        }\n    }\n}\n\nconst MAX_HEADER_SIZE: usize = 4096;\n\nimpl MessageWrapper {\n    pub async fn build_dsn(&mut self, server: &Server) -> Option<Vec<u8>> {\n        let config = &server.core.smtp.queue;\n        let now = now();\n\n        let mut txt_success = String::new();\n        let mut txt_delay = String::new();\n        let mut txt_failed = String::new();\n        let mut dsn = String::new();\n\n        for rcpt in &mut self.message.recipients {\n            if rcpt.has_flag(RCPT_DSN_SENT | RCPT_NOTIFY_NEVER) {\n                continue;\n            }\n            match &rcpt.status {\n                Status::Completed(response) => {\n                    rcpt.flags |= RCPT_DSN_SENT;\n                    if !rcpt.has_flag(RCPT_NOTIFY_SUCCESS) {\n                        continue;\n                    }\n                    rcpt.write_dsn(&mut dsn);\n                    rcpt.status.write_dsn(&mut dsn);\n                    response.write_dsn_text(&rcpt.address, &mut txt_success);\n                }\n                Status::TemporaryFailure(response)\n                    if rcpt.notify.due <= now && rcpt.has_flag(RCPT_NOTIFY_DELAY) =>\n                {\n                    rcpt.write_dsn(&mut dsn);\n                    rcpt.status.write_dsn(&mut dsn);\n                    rcpt.write_dsn_will_retry_until(self.message.created, &mut dsn);\n                    response.write_dsn_text(&rcpt.address, &mut txt_delay);\n                }\n                Status::PermanentFailure(response) => {\n                    rcpt.flags |= RCPT_DSN_SENT;\n                    if !rcpt.has_flag(RCPT_NOTIFY_FAILURE) {\n                        continue;\n                    }\n                    rcpt.write_dsn(&mut dsn);\n                    rcpt.status.write_dsn(&mut dsn);\n                    response.write_dsn_text(&rcpt.address, &mut txt_failed);\n                }\n                Status::Scheduled if rcpt.notify.due <= now && rcpt.has_flag(RCPT_NOTIFY_DELAY) => {\n                    // This case should not happen under normal circumstances\n                    rcpt.write_dsn(&mut dsn);\n                    rcpt.status.write_dsn(&mut dsn);\n                    rcpt.write_dsn_will_retry_until(self.message.created, &mut dsn);\n                    ErrorDetails {\n                        entity: \"localhost\".into(),\n                        details: Error::ConcurrencyLimited,\n                    }\n                    .write_dsn_text(&rcpt.address, &mut txt_delay);\n                }\n                _ => continue,\n            }\n\n            dsn.push_str(\"\\r\\n\");\n        }\n\n        // Build text response\n        let txt_len = txt_success.len() + txt_delay.len() + txt_failed.len();\n        if txt_len == 0 {\n            return None;\n        }\n\n        let has_success = !txt_success.is_empty();\n        let has_delay = !txt_delay.is_empty();\n        let has_failure = !txt_failed.is_empty();\n\n        let mut txt = String::with_capacity(txt_len + 128);\n        let (subject, is_mixed) = if has_success && !has_delay && !has_failure {\n            txt.push_str(\n                \"Your message has been successfully delivered to the following recipients:\\r\\n\\r\\n\",\n            );\n            (\"Successfully delivered message\", false)\n        } else if has_delay && !has_success && !has_failure {\n            txt.push_str(\"There was a temporary problem delivering your message to the following recipients:\\r\\n\\r\\n\");\n            (\"Warning: Delay in message delivery\", false)\n        } else if has_failure && !has_success && !has_delay {\n            txt.push_str(\n                \"Your message could not be delivered to the following recipients:\\r\\n\\r\\n\",\n            );\n            (\"Failed to deliver message\", false)\n        } else if has_success {\n            txt.push_str(\"Your message has been partially delivered:\\r\\n\\r\\n\");\n            (\"Partially delivered message\", true)\n        } else {\n            txt.push_str(\"Your message could not be delivered to some recipients:\\r\\n\\r\\n\");\n            (\n                \"Warning: Temporary and permanent failures during message delivery\",\n                true,\n            )\n        };\n\n        if has_success {\n            if is_mixed {\n                txt.push_str(\n                    \"    ----- Delivery to the following addresses was successful -----\\r\\n\",\n                );\n            }\n\n            txt.push_str(&txt_success);\n            txt.push_str(\"\\r\\n\");\n        }\n\n        if has_delay {\n            if is_mixed {\n                txt.push_str(\n                    \"    ----- There was a temporary problem delivering to these addresses -----\\r\\n\",\n                );\n            }\n            txt.push_str(&txt_delay);\n            txt.push_str(\"\\r\\n\");\n        }\n\n        if has_failure {\n            if is_mixed {\n                txt.push_str(\"    ----- Delivery to the following addresses failed -----\\r\\n\");\n            }\n            txt.push_str(&txt_failed);\n            txt.push_str(\"\\r\\n\");\n        }\n\n        // Update next delay notification time\n        if has_delay {\n            let mut changes = Vec::new();\n            for (rcpt_idx, rcpt) in self.message.recipients.iter().enumerate() {\n                if matches!(\n                    &rcpt.status,\n                    Status::TemporaryFailure(_) | Status::Scheduled\n                ) && rcpt.notify.due <= now\n                {\n                    let envelope = QueueEnvelope::new(&self.message, rcpt);\n\n                    let queue_id = server\n                        .eval_if::<String, _>(\n                            &server.core.smtp.queue.queue,\n                            &envelope,\n                            self.span_id,\n                        )\n                        .await\n                        .unwrap_or_else(|| \"default\".to_string());\n                    let queue = server.get_queue_or_default(&queue_id, self.span_id);\n\n                    if let Some(next_notify) =\n                        queue.notify.get((rcpt.notify.inner + 1) as usize).copied()\n                    {\n                        changes.push((rcpt_idx, 1, now + next_notify));\n                    } else {\n                        changes.push((rcpt_idx, 0, u64::MAX));\n                    }\n                }\n            }\n\n            for (rcpt_idx, inner, due) in changes {\n                let rcpt = &mut self.message.recipients[rcpt_idx];\n                rcpt.notify.inner += inner;\n                rcpt.notify.due = due;\n            }\n        }\n\n        // Obtain hostname and sender addresses\n        let from_name = server\n            .eval_if(&config.dsn.name, &self.message, self.span_id)\n            .await\n            .unwrap_or_else(|| String::from(\"Mail Delivery Subsystem\"));\n        let from_addr = server\n            .eval_if(&config.dsn.address, &self.message, self.span_id)\n            .await\n            .unwrap_or_else(|| String::from(\"MAILER-DAEMON@localhost\"));\n        let reporting_mta = server\n            .eval_if(\n                &server.core.smtp.report.submitter,\n                &self.message,\n                self.span_id,\n            )\n            .await\n            .unwrap_or_else(|| String::from(\"localhost\"));\n\n        // Prepare DSN\n        let mut dsn_header = String::with_capacity(dsn.len() + 128);\n        self.message\n            .write_dsn_headers(&mut dsn_header, &reporting_mta);\n        let dsn = dsn_header + dsn.as_str();\n\n        // Fetch up to MAX_HEADER_SIZE bytes of message headers\n        let headers = match server\n            .blob_store()\n            .get_blob(self.message.blob_hash.as_slice(), 0..MAX_HEADER_SIZE)\n            .await\n        {\n            Ok(Some(mut buf)) => {\n                let mut prev_ch = 0;\n                let mut last_lf = buf.len();\n                for (pos, &ch) in buf.iter().enumerate() {\n                    match ch {\n                        b'\\n' => {\n                            last_lf = pos + 1;\n                            if prev_ch != b'\\n' {\n                                prev_ch = ch;\n                            } else {\n                                break;\n                            }\n                        }\n                        b'\\r' => (),\n                        0 => break,\n                        _ => {\n                            prev_ch = ch;\n                        }\n                    }\n                }\n                if last_lf < MAX_HEADER_SIZE {\n                    buf.truncate(last_lf);\n                }\n                String::from_utf8(buf).unwrap_or_default()\n            }\n            Ok(None) => {\n                trc::event!(\n                    Queue(trc::QueueEvent::BlobNotFound),\n                    SpanId = self.span_id,\n                    BlobId = self.message.blob_hash.to_hex(),\n                    CausedBy = trc::location!()\n                );\n\n                String::new()\n            }\n            Err(err) => {\n                trc::error!(\n                    err.span_id(self.span_id)\n                        .details(\"Failed to fetch blobId\")\n                        .caused_by(trc::location!())\n                );\n\n                String::new()\n            }\n        };\n\n        // Build message\n        MessageBuilder::new()\n            .from((from_name.as_str(), from_addr.as_str()))\n            .header(\n                \"To\",\n                HeaderType::Text(self.message.return_path.as_ref().into()),\n            )\n            .header(\"Auto-Submitted\", HeaderType::Text(\"auto-generated\".into()))\n            .message_id(format!(\"<{}@{}>\", make_boundary(\".\"), reporting_mta))\n            .subject(subject)\n            .body(MimePart::new(\n                ContentType::new(\"multipart/report\").attribute(\"report-type\", \"delivery-status\"),\n                BodyPart::Multipart(vec![\n                    MimePart::new(ContentType::new(\"text/plain\"), BodyPart::Text(txt.into())),\n                    MimePart::new(\n                        ContentType::new(\"message/delivery-status\"),\n                        BodyPart::Text(dsn.into()),\n                    ),\n                    MimePart::new(\n                        ContentType::new(\"message/rfc822\"),\n                        BodyPart::Text(headers.into()),\n                    ),\n                ]),\n            ))\n            .write_to_vec()\n            .unwrap_or_default()\n            .into()\n    }\n\n    fn handle_double_bounce(&mut self) {\n        let mut is_double_bounce = Vec::with_capacity(0);\n        let now = now();\n\n        for rcpt in &mut self.message.recipients {\n            if !rcpt.has_flag(RCPT_DSN_SENT | RCPT_NOTIFY_NEVER)\n                && let Status::PermanentFailure(err) = &rcpt.status\n            {\n                rcpt.flags |= RCPT_DSN_SENT;\n                let mut dsn = String::new();\n                err.write_dsn_text(&rcpt.address, &mut dsn);\n                is_double_bounce.push(dsn);\n            }\n\n            if rcpt.notify.due <= now {\n                rcpt.notify.due = rcpt\n                    .expiration_time(self.message.created)\n                    .map(|d| d + 10)\n                    .unwrap_or(u64::MAX);\n            }\n        }\n\n        if !is_double_bounce.is_empty() {\n            trc::event!(\n                Delivery(trc::DeliveryEvent::DoubleBounce),\n                SpanId = self.span_id,\n                To = is_double_bounce\n            );\n        }\n    }\n}\n\nimpl HostResponse<Box<str>> {\n    fn write_dsn_text(&self, addr: &str, dsn: &mut String) {\n        let _ = write!(\n            dsn,\n            \"<{}> (delivered to '{}' with code {} ({}.{}.{}) '\",\n            addr,\n            self.hostname,\n            self.response.code,\n            self.response.esc[0],\n            self.response.esc[1],\n            self.response.esc[2]\n        );\n        self.response.write_response(dsn);\n        dsn.push_str(\"')\\r\\n\");\n    }\n}\n\nimpl UnexpectedResponse {\n    fn write_dsn_text(&self, host: &str, addr: &str, dsn: &mut String) {\n        let _ = write!(dsn, \"<{addr}> (host '{host}' rejected \");\n\n        if !self.command.is_empty() {\n            let _ = write!(dsn, \"command '{}'\", self.command);\n        } else {\n            dsn.push_str(\"transaction\");\n        }\n\n        let _ = write!(\n            dsn,\n            \" with code {} ({}.{}.{}) '\",\n            self.response.code, self.response.esc[0], self.response.esc[1], self.response.esc[2]\n        );\n        self.response.write_response(dsn);\n        dsn.push_str(\"')\\r\\n\");\n    }\n}\n\nimpl ErrorDetails {\n    fn write_dsn_text(&self, addr: &str, dsn: &mut String) {\n        let entity = self.entity.as_ref();\n        match &self.details {\n            Error::UnexpectedResponse(response) => {\n                response.write_dsn_text(entity, addr, dsn);\n            }\n            Error::DnsError(err) => {\n                let _ = write!(dsn, \"<{addr}> (failed to lookup '{entity}': {err})\\r\\n\",);\n            }\n            Error::ConnectionError(details) => {\n                let _ = write!(\n                    dsn,\n                    \"<{addr}> (connection to '{entity}' failed: {details})\\r\\n\",\n                );\n            }\n            Error::TlsError(details) => {\n                let _ = write!(dsn, \"<{addr}> (TLS error from '{entity}': {details})\\r\\n\",);\n            }\n            Error::DaneError(details) => {\n                let _ = write!(\n                    dsn,\n                    \"<{addr}> (DANE failed to authenticate '{entity}': {details})\\r\\n\",\n                );\n            }\n            Error::MtaStsError(details) => {\n                let _ = write!(\n                    dsn,\n                    \"<{addr}> (MTA-STS failed to authenticate '{entity}': {details})\\r\\n\",\n                );\n            }\n            Error::RateLimited => {\n                let _ = write!(dsn, \"<{addr}> (rate limited)\\r\\n\");\n            }\n            Error::ConcurrencyLimited => {\n                let _ = write!(\n                    dsn,\n                    \"<{addr}> (too many concurrent connections to remote server)\\r\\n\",\n                );\n            }\n            Error::Io(err) => {\n                let _ = write!(dsn, \"<{addr}> (queue error: {err})\\r\\n\");\n            }\n        }\n    }\n}\n\nimpl Message {\n    fn write_dsn_headers(&self, dsn: &mut String, reporting_mta: &str) {\n        let _ = write!(dsn, \"Reporting-MTA: dns;{reporting_mta}\\r\\n\");\n        dsn.push_str(\"Arrival-Date: \");\n        dsn.push_str(&DateTime::from_timestamp(self.created as i64).to_rfc822());\n        dsn.push_str(\"\\r\\n\");\n        if let Some(env_id) = &self.env_id {\n            let _ = write!(dsn, \"Original-Envelope-Id: {env_id}\\r\\n\");\n        }\n        dsn.push_str(\"\\r\\n\");\n    }\n}\n\nimpl Recipient {\n    fn write_dsn(&self, dsn: &mut String) {\n        if let Some(orcpt) = &self.orcpt {\n            let _ = write!(dsn, \"Original-Recipient: rfc822;{orcpt}\\r\\n\");\n        }\n        let _ = write!(dsn, \"Final-Recipient: rfc822;{}\\r\\n\", self.address);\n    }\n\n    fn write_dsn_will_retry_until(&self, created: u64, dsn: &mut String) {\n        if let Some(expires) = self.expiration_time(created)\n            && expires > now()\n        {\n            dsn.push_str(\"Will-Retry-Until: \");\n            dsn.push_str(&DateTime::from_timestamp(expires as i64).to_rfc822());\n            dsn.push_str(\"\\r\\n\");\n        }\n    }\n}\n\nimpl<T, E> Status<T, E> {\n    pub fn into_permanent(self) -> Self {\n        match self {\n            Status::TemporaryFailure(v) => Status::PermanentFailure(v),\n            v => v,\n        }\n    }\n\n    pub fn into_temporary(self) -> Self {\n        match self {\n            Status::PermanentFailure(err) => Status::TemporaryFailure(err),\n            other => other,\n        }\n    }\n\n    pub fn is_permanent(&self) -> bool {\n        matches!(self, Status::PermanentFailure(_))\n    }\n\n    fn write_dsn_action(&self, dsn: &mut String) {\n        dsn.push_str(\"Action: \");\n        dsn.push_str(match self {\n            Status::Completed(_) => \"delivered\",\n            Status::PermanentFailure(_) => \"failed\",\n            Status::TemporaryFailure(_) | Status::Scheduled => \"delayed\",\n        });\n        dsn.push_str(\"\\r\\n\");\n    }\n}\n\nimpl Status<HostResponse<Box<str>>, ErrorDetails> {\n    fn write_dsn(&self, dsn: &mut String) {\n        self.write_dsn_action(dsn);\n        self.write_dsn_status(dsn);\n        self.write_dsn_diagnostic(dsn);\n        self.write_dsn_remote_mta(dsn);\n    }\n\n    fn write_dsn_status(&self, dsn: &mut String) {\n        dsn.push_str(\"Status: \");\n        match self {\n            Status::Completed(response) => {\n                response.response.write_dsn_status(dsn);\n            }\n            Status::TemporaryFailure(err) | Status::PermanentFailure(err) => {\n                if let Error::UnexpectedResponse(response) = &err.details {\n                    response.response.write_dsn_status(dsn);\n                } else {\n                    dsn.push_str(if matches!(self, Status::PermanentFailure(_)) {\n                        \"5.0.0\"\n                    } else {\n                        \"4.0.0\"\n                    });\n                }\n            }\n            Status::Scheduled => {\n                dsn.push_str(\"4.0.0\");\n            }\n        }\n        dsn.push_str(\"\\r\\n\");\n    }\n\n    fn write_dsn_remote_mta(&self, dsn: &mut String) {\n        match self {\n            Status::Completed(response) => {\n                dsn.push_str(\"Remote-MTA: dns;\");\n                dsn.push_str(&response.hostname);\n                dsn.push_str(\"\\r\\n\");\n            }\n            Status::TemporaryFailure(err) | Status::PermanentFailure(err) => match &err.details {\n                Error::UnexpectedResponse(_)\n                | Error::ConnectionError(_)\n                | Error::TlsError(_)\n                | Error::DaneError(_) => {\n                    dsn.push_str(\"Remote-MTA: dns;\");\n                    dsn.push_str(&err.entity);\n                    dsn.push_str(\"\\r\\n\");\n                }\n                _ => (),\n            },\n            Status::Scheduled => (),\n        }\n    }\n\n    fn write_dsn_diagnostic(&self, dsn: &mut String) {\n        if let Status::PermanentFailure(err) | Status::TemporaryFailure(err) = self\n            && let Error::UnexpectedResponse(response) = &err.details\n        {\n            response.response.write_dsn_diagnostic(dsn);\n        }\n    }\n}\n\nimpl WriteDsn for Response<Box<str>> {\n    fn write_dsn_status(&self, dsn: &mut String) {\n        if self.esc[0] > 0 {\n            let _ = write!(dsn, \"{}.{}.{}\", self.esc[0], self.esc[1], self.esc[2]);\n        } else {\n            let _ = write!(\n                dsn,\n                \"{}.{}.{}\",\n                self.code / 100,\n                (self.code / 10) % 10,\n                self.code % 10\n            );\n        }\n    }\n\n    fn write_dsn_diagnostic(&self, dsn: &mut String) {\n        let _ = write!(dsn, \"Diagnostic-Code: smtp;{} \", self.code);\n        self.write_response(dsn);\n        dsn.push_str(\"\\r\\n\");\n    }\n\n    fn write_response(&self, dsn: &mut String) {\n        for ch in self.message.chars() {\n            if ch != '\\n' && ch != '\\r' {\n                dsn.push(ch);\n            }\n        }\n    }\n}\n\ntrait WriteDsn {\n    fn write_dsn_status(&self, dsn: &mut String);\n    fn write_dsn_diagnostic(&self, dsn: &mut String);\n    fn write_response(&self, dsn: &mut String);\n}\n"
  },
  {
    "path": "crates/smtp/src/queue/manager.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Message, QueueId, Status, spool::SmtpSpool};\nuse crate::queue::{Recipient, spool::LOCK_EXPIRY};\nuse ahash::AHashMap;\nuse common::{\n    Inner,\n    config::smtp::queue::{QueueExpiry, QueueName},\n    core::BuildServer,\n    ipc::{QueueEvent, QueueEventStatus},\n};\nuse rand::{Rng, seq::SliceRandom};\nuse std::{\n    collections::hash_map::Entry,\n    sync::{Arc, atomic::Ordering},\n    time::{Duration, Instant},\n};\nuse store::write::now;\nuse tokio::sync::mpsc;\n\npub struct Queue {\n    pub core: Arc<Inner>,\n    pub locked: AHashMap<(QueueId, QueueName), LockedMessage>,\n    pub locked_revision: u64,\n    pub stats: AHashMap<QueueName, QueueStats>,\n    pub next_refresh: Instant,\n    pub rx: mpsc::Receiver<QueueEvent>,\n    pub is_paused: bool,\n}\n\n#[derive(Debug)]\npub struct QueueStats {\n    pub in_flight: usize,\n    pub max_in_flight: usize,\n    pub last_warning: Instant,\n}\n\n#[derive(Debug)]\npub struct LockedMessage {\n    pub expires: u64,\n    pub revision: u64,\n}\n\nimpl SpawnQueue for mpsc::Receiver<QueueEvent> {\n    fn spawn(self, core: Arc<Inner>) {\n        tokio::spawn(async move {\n            Queue::new(core, self).start().await;\n        });\n    }\n}\n\nconst BACK_PRESSURE_WARN_INTERVAL: Duration = Duration::from_secs(60);\n\nimpl Queue {\n    pub fn new(core: Arc<Inner>, rx: mpsc::Receiver<QueueEvent>) -> Self {\n        Queue {\n            core,\n            locked: AHashMap::with_capacity(128),\n            locked_revision: 0,\n            stats: AHashMap::new(),\n            next_refresh: Instant::now() + Duration::from_secs(1),\n            is_paused: false,\n            rx,\n        }\n    }\n\n    pub async fn start(&mut self) {\n        loop {\n            let mut refresh_queue;\n\n            match tokio::time::timeout(\n                self.next_refresh.duration_since(Instant::now()),\n                self.rx.recv(),\n            )\n            .await\n            {\n                Ok(Some(event)) => {\n                    refresh_queue = self.handle_event(event).await;\n\n                    while let Ok(event) = self.rx.try_recv() {\n                        refresh_queue = self.handle_event(event).await || refresh_queue;\n                    }\n                }\n                Err(_) => {\n                    refresh_queue = true;\n                }\n                Ok(None) => {\n                    break;\n                }\n            };\n\n            if !self.is_paused {\n                // Deliver scheduled messages\n                if refresh_queue || self.next_refresh <= Instant::now() {\n                    // Process queue events\n                    let server = self.core.build_server();\n                    let mut queue_events = server.next_event(self).await;\n\n                    if queue_events.messages.len() > 3 {\n                        queue_events.messages.shuffle(&mut rand::rng());\n                    }\n\n                    for queue_event in &queue_events.messages {\n                        // Fetch queue stats\n                        let stats = match self.stats.get_mut(&queue_event.queue_name) {\n                            Some(stats) => stats,\n                            None => {\n                                let queue_config =\n                                    server.get_virtual_queue_or_default(&queue_event.queue_name);\n                                self.stats.insert(\n                                    queue_event.queue_name,\n                                    QueueStats::new(queue_config.threads),\n                                );\n                                self.stats.get_mut(&queue_event.queue_name).unwrap()\n                            }\n                        };\n\n                        // Enforce concurrency limits\n                        if stats.has_capacity() {\n                            // Deliver message\n                            stats.in_flight += 1;\n                            queue_event.try_deliver(server.clone());\n                        } else {\n                            if stats.last_warning.elapsed() >= BACK_PRESSURE_WARN_INTERVAL {\n                                stats.last_warning = Instant::now();\n                                trc::event!(\n                                    Queue(trc::QueueEvent::BackPressure),\n                                    Reason = \"Processing capacity for this queue exceeded.\",\n                                    QueueName = queue_event.queue_name.to_string(),\n                                    Limit = stats.max_in_flight,\n                                );\n                            }\n                            self.locked\n                                .remove(&(queue_event.queue_id, queue_event.queue_name));\n                        }\n                    }\n\n                    // Remove expired locks\n                    let now = now();\n                    self.locked.retain(|_, locked| {\n                        locked.expires > now && locked.revision == self.locked_revision\n                    });\n\n                    self.next_refresh = Instant::now()\n                        + Duration::from_secs(queue_events.next_refresh.saturating_sub(now));\n                }\n            } else {\n                // Queue is paused\n                self.next_refresh = Instant::now() + Duration::from_secs(86400);\n            }\n        }\n    }\n\n    async fn handle_event(&mut self, event: QueueEvent) -> bool {\n        match event {\n            QueueEvent::WorkerDone {\n                queue_id,\n                queue_name,\n                status,\n            } => {\n                let queue_stats = self.stats.get_mut(&queue_name).unwrap();\n                queue_stats.in_flight -= 1;\n\n                match status {\n                    QueueEventStatus::Completed => {\n                        self.locked.remove(&(queue_id, queue_name));\n                        !self.locked.is_empty() || !queue_stats.has_capacity()\n                    }\n                    QueueEventStatus::Locked => {\n                        let expires = LOCK_EXPIRY + rand::rng().random_range(5..10);\n                        let due_in = Instant::now() + Duration::from_secs(expires);\n                        if due_in < self.next_refresh {\n                            self.next_refresh = due_in;\n                        }\n\n                        self.locked.insert(\n                            (queue_id, queue_name),\n                            LockedMessage {\n                                expires: now() + expires,\n                                revision: self.locked_revision,\n                            },\n                        );\n                        self.locked.len() > 1 || !queue_stats.has_capacity()\n                    }\n                    QueueEventStatus::Deferred => {\n                        self.locked.remove(&(queue_id, queue_name));\n                        true\n                    }\n                }\n            }\n            QueueEvent::Refresh => true,\n            QueueEvent::Paused(paused) => {\n                self.core\n                    .data\n                    .queue_status\n                    .store(!paused, Ordering::Relaxed);\n                self.is_paused = paused;\n                false\n            }\n            QueueEvent::ReloadSettings => {\n                let server = self.core.build_server();\n                for (name, settings) in &server.core.smtp.queue.virtual_queues {\n                    if let Some(stats) = self.stats.get_mut(name) {\n                        stats.max_in_flight = settings.threads;\n                    } else {\n                        self.stats.insert(*name, QueueStats::new(settings.threads));\n                    }\n                }\n\n                false\n            }\n            QueueEvent::Stop => {\n                self.rx.close();\n                self.is_paused = true;\n                false\n            }\n        }\n    }\n}\n\nimpl Message {\n    pub fn next_event(&self, queue: Option<QueueName>) -> Option<u64> {\n        let mut next_event = None;\n\n        for rcpt in &self.recipients {\n            if matches!(rcpt.status, Status::Scheduled | Status::TemporaryFailure(_))\n                && queue.is_none_or(|q| rcpt.queue == q)\n            {\n                let mut earlier_event = std::cmp::min(rcpt.retry.due, rcpt.notify.due);\n\n                if let Some(expires) = rcpt.expiration_time(self.created) {\n                    earlier_event = std::cmp::min(earlier_event, expires);\n                }\n\n                if let Some(next_event) = &mut next_event {\n                    if earlier_event < *next_event {\n                        *next_event = earlier_event;\n                    }\n                } else {\n                    next_event = Some(earlier_event);\n                }\n            }\n        }\n\n        next_event\n    }\n\n    pub fn next_delivery_event(&self, queue: Option<QueueName>) -> Option<u64> {\n        let mut next_delivery = None;\n\n        for rcpt in self.recipients.iter().filter(|rcpt| {\n            matches!(rcpt.status, Status::Scheduled | Status::TemporaryFailure(_))\n                && queue.is_none_or(|q| rcpt.queue == q)\n        }) {\n            if let Some(next_delivery) = &mut next_delivery {\n                if rcpt.retry.due < *next_delivery {\n                    *next_delivery = rcpt.retry.due;\n                }\n            } else {\n                next_delivery = Some(rcpt.retry.due);\n            }\n        }\n\n        next_delivery\n    }\n\n    pub fn next_dsn(&self, queue: Option<QueueName>) -> Option<u64> {\n        let mut next_dsn = None;\n\n        for rcpt in self.recipients.iter().filter(|rcpt| {\n            matches!(rcpt.status, Status::Scheduled | Status::TemporaryFailure(_))\n                && queue.is_none_or(|q| rcpt.queue == q)\n        }) {\n            if let Some(next_dsn) = &mut next_dsn {\n                if rcpt.notify.due < *next_dsn {\n                    *next_dsn = rcpt.notify.due;\n                }\n            } else {\n                next_dsn = Some(rcpt.notify.due);\n            }\n        }\n\n        next_dsn\n    }\n\n    pub fn expires(&self, queue: Option<QueueName>) -> Option<u64> {\n        let mut expires = None;\n\n        for rcpt in self.recipients.iter().filter(|d| {\n            matches!(d.status, Status::Scheduled | Status::TemporaryFailure(_))\n                && queue.is_none_or(|q| d.queue == q)\n        }) {\n            if let Some(rcpt_expires) = rcpt.expiration_time(self.created) {\n                if let Some(expires) = &mut expires {\n                    if rcpt_expires > *expires {\n                        *expires = rcpt_expires;\n                    }\n                } else {\n                    expires = Some(rcpt_expires)\n                }\n            }\n        }\n\n        expires\n    }\n\n    pub fn next_events(&self) -> AHashMap<QueueName, u64> {\n        let mut next_events = AHashMap::new();\n\n        for rcpt in &self.recipients {\n            if matches!(rcpt.status, Status::Scheduled | Status::TemporaryFailure(_)) {\n                let mut earlier_event = std::cmp::min(rcpt.retry.due, rcpt.notify.due);\n\n                if let Some(expires) = rcpt.expiration_time(self.created) {\n                    earlier_event = std::cmp::min(earlier_event, expires);\n                }\n\n                match next_events.entry(rcpt.queue) {\n                    Entry::Occupied(mut entry) => {\n                        let entry = entry.get_mut();\n                        if earlier_event < *entry {\n                            *entry = earlier_event;\n                        }\n                    }\n                    Entry::Vacant(entry) => {\n                        entry.insert(earlier_event);\n                    }\n                }\n            }\n        }\n\n        next_events\n    }\n}\n\nimpl Recipient {\n    pub fn expiration_time(&self, created: u64) -> Option<u64> {\n        match self.expires {\n            QueueExpiry::Ttl(time) => Some(created + time),\n            QueueExpiry::Attempts(_) => None,\n        }\n    }\n\n    pub fn is_expired(&self, created: u64, now: u64) -> bool {\n        match self.expires {\n            QueueExpiry::Ttl(time) => created + time <= now,\n            QueueExpiry::Attempts(count) => self.retry.inner >= count,\n        }\n    }\n}\n\npub trait SpawnQueue {\n    fn spawn(self, core: Arc<Inner>);\n}\n\nimpl QueueStats {\n    fn new(max_in_flight: usize) -> Self {\n        QueueStats {\n            in_flight: 0,\n            max_in_flight,\n            last_warning: Instant::now() - BACK_PRESSURE_WARN_INTERVAL,\n        }\n    }\n\n    #[inline]\n    pub fn has_capacity(&self) -> bool {\n        self.in_flight < self.max_in_flight\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/queue/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    config::smtp::queue::{QueueExpiry, QueueName},\n    expr::{self, functions::ResolveVariable, *},\n};\nuse compact_str::ToCompactString;\nuse smtp_proto::Response;\nuse std::{\n    fmt::Display,\n    net::{IpAddr, Ipv4Addr},\n    time::{Duration, Instant, SystemTime},\n};\nuse store::write::now;\nuse types::blob_hash::BlobHash;\nuse utils::DomainPart;\n\npub mod dsn;\npub mod manager;\npub mod quota;\npub mod spool;\npub mod throttle;\n\npub type QueueId = u64;\n\n#[derive(Debug, Clone, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, serde::Deserialize)]\npub struct Schedule<T> {\n    pub due: u64,\n    pub inner: T,\n}\n\n#[derive(Debug, Clone, Copy)]\npub struct QueuedMessage {\n    pub due: u64,\n    pub queue_id: QueueId,\n    pub queue_name: QueueName,\n}\n\n#[derive(Debug, Clone, Copy)]\npub enum MessageSource {\n    Authenticated,\n    Unauthenticated {\n        dmarc_pass: bool,\n        train_spam: Option<bool>,\n    },\n    Dsn,\n    Report,\n    Autogenerated,\n}\n\n#[derive(rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, Debug, Clone, PartialEq, Eq)]\npub struct Message {\n    pub created: u64,\n    pub blob_hash: BlobHash,\n\n    pub return_path: Box<str>,\n    pub recipients: Vec<Recipient>,\n\n    pub received_from_ip: IpAddr,\n    pub received_via_port: u16,\n\n    pub flags: u64,\n    pub env_id: Option<Box<str>>,\n    pub priority: i16,\n\n    pub size: u64,\n    pub quota_keys: Box<[QuotaKey]>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct MessageWrapper {\n    pub queue_id: QueueId,\n    pub queue_name: QueueName,\n    pub is_multi_queue: bool,\n    pub span_id: u64,\n    pub message: Message,\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    serde::Deserialize,\n)]\npub enum QuotaKey {\n    Size { key: Box<[u8]>, id: u64 },\n    Count { key: Box<[u8]>, id: u64 },\n}\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    serde::Deserialize,\n)]\npub struct Recipient {\n    pub address: Box<str>,\n\n    pub retry: Schedule<u32>,\n    pub notify: Schedule<u32>,\n    pub expires: QueueExpiry,\n\n    pub queue: QueueName,\n    pub status: Status<HostResponse<Box<str>>, ErrorDetails>,\n    pub flags: u64,\n    pub orcpt: Option<Box<str>>,\n}\n\npub const FROM_AUTHENTICATED: u64 = 1 << 32;\npub const FROM_UNAUTHENTICATED: u64 = 1 << 33;\npub const FROM_UNAUTHENTICATED_DMARC: u64 = 1 << 34;\npub const FROM_DSN: u64 = 1 << 35;\npub const FROM_REPORT: u64 = 1 << 36;\npub const FROM_AUTOGENERATED: u64 = 1 << 37;\n\npub const RCPT_DSN_SENT: u64 = 1 << 32;\n//pub const RCPT_STATUS_CHANGED: u64 = 1 << 33;\npub const RCPT_SPAM_PAYLOAD: u64 = 1 << 34;\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Serialize,\n    serde::Deserialize,\n)]\npub enum Status<T, E> {\n    #[serde(rename = \"scheduled\")]\n    Scheduled,\n    #[serde(rename = \"completed\")]\n    Completed(T),\n    #[serde(rename = \"temp_fail\")]\n    TemporaryFailure(E),\n    #[serde(rename = \"perm_fail\")]\n    PermanentFailure(E),\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct HostResponse<T> {\n    pub hostname: T,\n    pub response: Response<Box<str>>,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n    Default,\n)]\npub enum Error {\n    DnsError(Box<str>),\n    UnexpectedResponse(UnexpectedResponse),\n    ConnectionError(Box<str>),\n    TlsError(Box<str>),\n    DaneError(Box<str>),\n    MtaStsError(Box<str>),\n    RateLimited,\n    #[default]\n    ConcurrencyLimited,\n    Io(Box<str>),\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Deserialize,\n)]\npub struct UnexpectedResponse {\n    pub command: Box<str>,\n    pub response: Response<Box<str>>,\n}\n\n#[derive(\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Default,\n    serde::Deserialize,\n)]\npub struct ErrorDetails {\n    pub entity: Box<str>,\n    pub details: Error,\n}\n\nimpl<T> Ord for Schedule<T> {\n    fn cmp(&self, other: &Self) -> std::cmp::Ordering {\n        other.due.cmp(&self.due)\n    }\n}\n\nimpl<T> PartialOrd for Schedule<T> {\n    fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {\n        Some(self.cmp(other))\n    }\n}\n\nimpl<T> PartialEq for Schedule<T> {\n    fn eq(&self, other: &Self) -> bool {\n        self.due == other.due\n    }\n}\n\nimpl<T> Eq for Schedule<T> {}\n\nimpl<T: Default> Schedule<T> {\n    pub fn now() -> Self {\n        Schedule {\n            due: now(),\n            inner: T::default(),\n        }\n    }\n\n    pub fn later(duration: u64) -> Self {\n        Schedule {\n            due: now() + duration,\n            inner: T::default(),\n        }\n    }\n}\n\npub struct QueueEnvelope<'x> {\n    pub message: &'x Message,\n    pub domain: &'x str,\n    pub mx: &'x str,\n    pub rcpt: &'x Recipient,\n    pub remote_ip: IpAddr,\n    pub local_ip: IpAddr,\n}\n\nimpl<'x> QueueEnvelope<'x> {\n    pub fn new(message: &'x Message, rcpt: &'x Recipient) -> Self {\n        Self {\n            message,\n            domain: rcpt.address.domain_part(),\n            rcpt,\n            mx: \"\",\n            remote_ip: IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),\n            local_ip: IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),\n        }\n    }\n}\n\nimpl<'x> ResolveVariable for QueueEnvelope<'x> {\n    fn resolve_variable(&self, variable: u32) -> expr::Variable<'x> {\n        match variable {\n            V_SENDER => self.message.return_path.as_ref().into(),\n            V_SENDER_DOMAIN => self.message.return_path.domain_part().into(),\n            V_RECIPIENT_DOMAIN => self.domain.into(),\n            V_RECIPIENT => self.rcpt.address.as_ref().into(),\n            V_RECIPIENTS => self\n                .message\n                .recipients\n                .iter()\n                .map(|r| Variable::from(r.address.as_ref()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_QUEUE_RETRY_NUM => self.rcpt.retry.inner.into(),\n            V_QUEUE_NOTIFY_NUM => self.rcpt.notify.inner.into(),\n            V_QUEUE_EXPIRES_IN => match &self.rcpt.expires {\n                QueueExpiry::Ttl(time) => (*time + self.message.created).saturating_sub(now()),\n                QueueExpiry::Attempts(count) => {\n                    (count.saturating_sub(self.rcpt.retry.inner)) as u64\n                }\n            }\n            .into(),\n            V_QUEUE_LAST_STATUS => self.rcpt.status.to_compact_string().into(),\n            V_QUEUE_LAST_ERROR => match &self.rcpt.status {\n                Status::Scheduled | Status::Completed(_) => \"none\",\n                Status::TemporaryFailure(err) | Status::PermanentFailure(err) => {\n                    match &err.details {\n                        Error::DnsError(_) => \"dns\",\n                        Error::UnexpectedResponse(_) => \"unexpected-reply\",\n                        Error::ConnectionError(_) => \"connection\",\n                        Error::TlsError(_) => \"tls\",\n                        Error::DaneError(_) => \"dane\",\n                        Error::MtaStsError(_) => \"mta-sts\",\n                        Error::RateLimited => \"rate\",\n                        Error::ConcurrencyLimited => \"concurrency\",\n                        Error::Io(_) => \"io\",\n                    }\n                }\n            }\n            .into(),\n            V_QUEUE_NAME => self.rcpt.queue.as_str().into(),\n            V_QUEUE_AGE => now().saturating_sub(self.message.created).into(),\n            V_SOURCE => if (self.message.flags & FROM_AUTHENTICATED) != 0 {\n                \"authenticated\"\n            } else if (self.message.flags & FROM_UNAUTHENTICATED_DMARC) != 0 {\n                \"dmarc_pass\"\n            } else if (self.message.flags & FROM_UNAUTHENTICATED) != 0 {\n                \"unauthenticated\"\n            } else if (self.message.flags & FROM_DSN) != 0 {\n                \"dsn\"\n            } else if (self.message.flags & FROM_REPORT) != 0 {\n                \"report\"\n            } else if (self.message.flags & FROM_AUTOGENERATED) != 0 {\n                \"autogenerated\"\n            } else {\n                \"unknown\"\n            }\n            .into(),\n            V_MX => self.mx.into(),\n            V_PRIORITY => self.message.priority.into(),\n            V_REMOTE_IP => self.remote_ip.to_compact_string().into(),\n            V_LOCAL_IP => self.local_ip.to_compact_string().into(),\n            V_RECEIVED_FROM_IP => self.message.received_from_ip.to_compact_string().into(),\n            V_RECEIVED_VIA_PORT => self.message.received_via_port.into(),\n            V_SIZE => self.message.size.into(),\n            _ => \"\".into(),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl ResolveVariable for Message {\n    fn resolve_variable(&self, variable: u32) -> expr::Variable<'_> {\n        match variable {\n            V_SENDER => self.return_path.as_ref().into(),\n            V_SENDER_DOMAIN => self.return_path.domain_part().into(),\n            V_RECIPIENTS => self\n                .recipients\n                .iter()\n                .map(|r| Variable::from(r.address.as_ref()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_PRIORITY => self.priority.into(),\n            _ => \"\".into(),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\npub struct RecipientDomain<'x>(&'x str);\n\nimpl<'x> RecipientDomain<'x> {\n    pub fn new(domain: &'x str) -> Self {\n        Self(domain)\n    }\n}\n\nimpl<'x> ResolveVariable for RecipientDomain<'x> {\n    fn resolve_variable(&self, variable: u32) -> expr::Variable<'x> {\n        match variable {\n            V_RECIPIENT_DOMAIN => self.0.into(),\n            _ => \"\".into(),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\n#[inline(always)]\npub fn instant_to_timestamp(now: Instant, time: Instant) -> u64 {\n    SystemTime::now()\n        .duration_since(SystemTime::UNIX_EPOCH)\n        .map_or(0, |d| d.as_secs())\n        + time.checked_duration_since(now).map_or(0, |d| d.as_secs())\n}\n\nimpl Recipient {\n    pub fn new(address: impl AsRef<str>) -> Self {\n        Recipient {\n            address: address.to_lowercase_domain().into_boxed_str(),\n            status: Status::Scheduled,\n            flags: 0,\n            orcpt: None,\n            retry: Schedule::now(),\n            notify: Schedule::now(),\n            expires: QueueExpiry::Attempts(0),\n            queue: QueueName::default(),\n        }\n    }\n\n    pub fn with_flags(mut self, flags: u64) -> Self {\n        self.flags = flags;\n        self\n    }\n\n    pub fn with_orcpt(mut self, orcpt: Option<Box<str>>) -> Self {\n        self.orcpt = orcpt;\n        self\n    }\n\n    pub fn address(&self) -> &str {\n        &self.address\n    }\n\n    pub fn domain_part(&self) -> &str {\n        self.address.domain_part()\n    }\n}\n\nimpl ArchivedRecipient {\n    pub fn address(&self) -> &str {\n        self.address.as_ref()\n    }\n\n    pub fn domain_part(&self) -> &str {\n        self.address.domain_part()\n    }\n}\n\npub trait InstantFromTimestamp {\n    fn to_instant(&self) -> Instant;\n}\n\nimpl InstantFromTimestamp for u64 {\n    fn to_instant(&self) -> Instant {\n        let timestamp = *self;\n        let current_timestamp = SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .map_or(0, |d| d.as_secs());\n        if timestamp > current_timestamp {\n            Instant::now() + Duration::from_secs(timestamp - current_timestamp)\n        } else {\n            Instant::now()\n        }\n    }\n}\n\nimpl Display for Error {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Error::UnexpectedResponse(response) => {\n                write!(\n                    f,\n                    \"Unexpected response for {}: {}\",\n                    response.command, response.response\n                )\n            }\n            Error::DnsError(err) => {\n                write!(f, \"DNS lookup failed: {err}\")\n            }\n            Error::ConnectionError(details) => {\n                write!(f, \"Connection failed: {details}\",)\n            }\n            Error::TlsError(details) => {\n                write!(f, \"TLS error: {details}\",)\n            }\n            Error::DaneError(details) => {\n                write!(f, \"DANE authentication failure: {details}\",)\n            }\n            Error::MtaStsError(details) => {\n                write!(f, \"MTA-STS auth failed: {details}\")\n            }\n            Error::RateLimited => {\n                write!(f, \"Rate limited\")\n            }\n            Error::ConcurrencyLimited => {\n                write!(f, \"Too many concurrent connections to remote server\")\n            }\n            Error::Io(err) => {\n                write!(f, \"Queue error: {err}\")\n            }\n        }\n    }\n}\n\nimpl Display for ArchivedError {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            ArchivedError::UnexpectedResponse(response) => {\n                write!(\n                    f,\n                    \"Unexpected response for {}: {}\",\n                    response.command, response.response\n                )\n            }\n            ArchivedError::DnsError(err) => {\n                write!(f, \"DNS lookup failed: {err}\")\n            }\n            ArchivedError::ConnectionError(details) => {\n                write!(f, \"Connection failed: {details}\",)\n            }\n            ArchivedError::TlsError(details) => {\n                write!(f, \"TLS error: {details}\",)\n            }\n            ArchivedError::DaneError(details) => {\n                write!(f, \"DANE authentication failure: {details}\",)\n            }\n            ArchivedError::MtaStsError(details) => {\n                write!(f, \"MTA-STS auth failed: {details}\")\n            }\n            ArchivedError::RateLimited => {\n                write!(f, \"Rate limited\")\n            }\n            ArchivedError::ConcurrencyLimited => {\n                write!(f, \"Too many concurrent connections to remote server\")\n            }\n            ArchivedError::Io(err) => {\n                write!(f, \"Queue error: {err}\")\n            }\n        }\n    }\n}\n\nimpl Display for Status<HostResponse<Box<str>>, ErrorDetails> {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Status::Scheduled => write!(f, \"Scheduled\"),\n            Status::Completed(response) => write!(f, \"Delivered: {}\", response.response),\n            Status::TemporaryFailure(err) => {\n                write!(f, \"Temporary Failure for {}: {}\", err.entity, err.details)\n            }\n            Status::PermanentFailure(err) => {\n                write!(f, \"Permanent Failure for {}: {}\", err.entity, err.details)\n            }\n        }\n    }\n}\n\nimpl Display for ArchivedErrorDetails {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"Error for {}: {}\", self.entity, self.details)\n    }\n}\n\n/*\n\npub trait DisplayArchivedResponse {\n    fn to_string(&self) -> String;\n}\n\nimpl DisplayArchivedResponse for ArchivedResponse<Box<str>> {\n    fn to_string(&self) -> String {\n        format!(\n            \"Code: {}, Enhanced code: {}.{}.{}, Message: {}\",\n            self.code, self.esc[0], self.esc[1], self.esc[2], self.message,\n        )\n    }\n}\n*/\n"
  },
  {
    "path": "crates/smtp/src/queue/quota.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{QueueEnvelope, QuotaKey, Status};\nuse crate::{core::throttle::NewKey, queue::MessageWrapper};\nuse ahash::AHashSet;\nuse common::{Server, config::smtp::queue::QueueQuota, expr::functions::ResolveVariable};\nuse std::future::Future;\nuse store::{\n    ValueKey,\n    write::{BatchBuilder, QueueClass, ValueClass},\n};\nuse trc::QueueEvent;\nuse utils::DomainPart;\n\npub trait HasQueueQuota: Sync + Send {\n    fn has_quota(&self, message: &mut MessageWrapper) -> impl Future<Output = bool> + Send;\n    fn check_quota<'x>(\n        &'x self,\n        quota: &'x QueueQuota,\n        envelope: &impl ResolveVariable,\n        size: u64,\n        id: u64,\n        refs: &mut Vec<QuotaKey>,\n        session_id: u64,\n    ) -> impl Future<Output = bool> + Send;\n}\n\nimpl HasQueueQuota for Server {\n    async fn has_quota(&self, message: &mut MessageWrapper) -> bool {\n        let mut quota_keys = Vec::new();\n\n        if !self.core.smtp.queue.quota.sender.is_empty() {\n            for quota in &self.core.smtp.queue.quota.sender {\n                if !self\n                    .check_quota(\n                        quota,\n                        &message.message,\n                        message.message.size,\n                        0,\n                        &mut quota_keys,\n                        message.span_id,\n                    )\n                    .await\n                {\n                    trc::event!(\n                        Queue(QueueEvent::QuotaExceeded),\n                        SpanId = message.span_id,\n                        Id = quota.id.clone(),\n                        Type = \"Sender\"\n                    );\n\n                    return false;\n                }\n            }\n        }\n\n        if !self.core.smtp.queue.quota.rcpt_domain.is_empty() {\n            let mut seen_domains = AHashSet::new();\n            for quota in &self.core.smtp.queue.quota.rcpt_domain {\n                for (rcpt_idx, rcpt) in message.message.recipients.iter().enumerate() {\n                    if seen_domains.insert(rcpt.address.domain_part())\n                        && !self\n                            .check_quota(\n                                quota,\n                                &QueueEnvelope::new(&message.message, rcpt),\n                                message.message.size,\n                                ((rcpt_idx + 1) << 32) as u64,\n                                &mut quota_keys,\n                                message.span_id,\n                            )\n                            .await\n                    {\n                        trc::event!(\n                            Queue(QueueEvent::QuotaExceeded),\n                            SpanId = message.span_id,\n                            Id = quota.id.clone(),\n                            Type = \"Domain\"\n                        );\n\n                        return false;\n                    }\n                }\n            }\n        }\n\n        for quota in &self.core.smtp.queue.quota.rcpt {\n            for (rcpt_idx, rcpt) in message.message.recipients.iter().enumerate() {\n                if !self\n                    .check_quota(\n                        quota,\n                        &QueueEnvelope::new(&message.message, rcpt),\n                        message.message.size,\n                        (rcpt_idx + 1) as u64,\n                        &mut quota_keys,\n                        message.span_id,\n                    )\n                    .await\n                {\n                    trc::event!(\n                        Queue(QueueEvent::QuotaExceeded),\n                        SpanId = message.span_id,\n                        Id = quota.id.clone(),\n                        Type = \"Recipient\"\n                    );\n\n                    return false;\n                }\n            }\n        }\n\n        message.message.quota_keys = quota_keys.into_boxed_slice();\n\n        true\n    }\n\n    async fn check_quota<'x>(\n        &'x self,\n        quota: &'x QueueQuota,\n        envelope: &impl ResolveVariable,\n        size: u64,\n        id: u64,\n        refs: &mut Vec<QuotaKey>,\n        session_id: u64,\n    ) -> bool {\n        if !quota.expr.is_empty()\n            && self\n                .eval_expr(&quota.expr, envelope, \"check_quota\", session_id)\n                .await\n                .unwrap_or(false)\n        {\n            let key = quota.new_key(envelope, \"\");\n            if let Some(max_size) = quota.size {\n                let used_size = self\n                    .core\n                    .storage\n                    .data\n                    .get_counter(ValueKey::from(ValueClass::Queue(QueueClass::QuotaSize(\n                        key.as_ref().to_vec(),\n                    ))))\n                    .await\n                    .unwrap_or(0) as u64;\n                if used_size + size > max_size {\n                    return false;\n                } else {\n                    refs.push(QuotaKey::Size {\n                        key: key.as_ref().into(),\n                        id,\n                    });\n                }\n            }\n\n            if let Some(max_messages) = quota.messages {\n                let total_messages = self\n                    .core\n                    .storage\n                    .data\n                    .get_counter(ValueKey::from(ValueClass::Queue(QueueClass::QuotaCount(\n                        key.as_ref().to_vec(),\n                    ))))\n                    .await\n                    .unwrap_or(0) as u64;\n                if total_messages + 1 > max_messages {\n                    return false;\n                } else {\n                    refs.push(QuotaKey::Count {\n                        key: key.as_ref().into(),\n                        id,\n                    });\n                }\n            }\n        }\n        true\n    }\n}\n\nimpl MessageWrapper {\n    pub fn release_quota(&mut self, batch: &mut BatchBuilder) {\n        if self.message.quota_keys.is_empty() {\n            return;\n        }\n        let mut quota_ids = Vec::with_capacity(self.message.recipients.len());\n\n        let mut seen_domains = AHashSet::new();\n        for (pos, rcpt) in self.message.recipients.iter().enumerate() {\n            if matches!(\n                &rcpt.status,\n                Status::Completed(_) | Status::PermanentFailure(_)\n            ) {\n                if seen_domains.insert(rcpt.address.domain_part()) {\n                    quota_ids.push(((pos + 1) as u64) << 32);\n                }\n                quota_ids.push((pos + 1) as u64);\n            }\n        }\n\n        if !quota_ids.is_empty() {\n            let mut quota_keys = Vec::new();\n            for quota_key in std::mem::take(&mut self.message.quota_keys) {\n                match quota_key {\n                    QuotaKey::Count { id, key } if quota_ids.contains(&id) => {\n                        batch.add(\n                            ValueClass::Queue(QueueClass::QuotaCount(key.into_vec())),\n                            -1,\n                        );\n                    }\n                    QuotaKey::Size { id, key } if quota_ids.contains(&id) => {\n                        batch.add(\n                            ValueClass::Queue(QueueClass::QuotaSize(key.into_vec())),\n                            -(self.message.size as i64),\n                        );\n                    }\n                    _ => {\n                        quota_keys.push(quota_key);\n                    }\n                }\n            }\n            self.message.quota_keys = quota_keys.into_boxed_slice();\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/queue/spool.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    ArchivedMessage, ArchivedStatus, Message, MessageSource, QueueEnvelope, QueueId, QueuedMessage,\n    QuotaKey, Recipient, Schedule, Status,\n};\nuse crate::queue::manager::{LockedMessage, Queue};\nuse crate::queue::{\n    FROM_AUTHENTICATED, FROM_AUTOGENERATED, FROM_DSN, FROM_REPORT, FROM_UNAUTHENTICATED,\n    FROM_UNAUTHENTICATED_DMARC, MessageWrapper,\n};\nuse common::config::smtp::queue::QueueName;\nuse common::ipc::QueueEvent;\nuse common::{KV_LOCK_QUEUE_MESSAGE, Server};\nuse std::borrow::Cow;\nuse std::collections::hash_map::Entry;\nuse std::future::Future;\nuse std::net::{IpAddr, Ipv4Addr};\nuse std::time::SystemTime;\nuse store::write::key::DeserializeBigEndian;\nuse store::write::serialize::rkyv_deserialize;\nuse store::write::{\n    AlignedBytes, Archive, Archiver, BatchBuilder, BlobLink, BlobOp, MergeResult, Params,\n    QueueClass, ValueClass, now,\n};\nuse store::{Deserialize, IterateParams, Serialize, U64_LEN, ValueKey};\nuse trc::{AddContext, ServerEvent, SpamEvent};\nuse types::blob_hash::BlobHash;\nuse utils::DomainPart;\n\npub const LOCK_EXPIRY: u64 = 10 * 60; // 10 minutes\npub const QUEUE_REFRESH: u64 = 5 * 60; // 5 minutes\nconst INFINITE_LOCK: u64 = 60 * 60 * 24 * 365; // 1 year\n\npub struct QueuedMessages {\n    pub messages: Vec<QueuedMessage>,\n    pub next_refresh: u64,\n}\n\npub trait SmtpSpool: Sync + Send {\n    fn new_message(&self, return_path: impl AsRef<str>, span_id: u64) -> MessageWrapper;\n\n    fn next_event(&self, queue: &mut Queue) -> impl Future<Output = QueuedMessages> + Send;\n\n    fn try_lock_event(\n        &self,\n        queue_id: QueueId,\n        queue_name: QueueName,\n    ) -> impl Future<Output = bool> + Send;\n\n    fn unlock_event(\n        &self,\n        queue_id: QueueId,\n        queue_name: QueueName,\n    ) -> impl Future<Output = ()> + Send;\n\n    fn read_message(\n        &self,\n        id: QueueId,\n        queue_name: QueueName,\n    ) -> impl Future<Output = Option<MessageWrapper>> + Send;\n\n    fn read_message_archive(\n        &self,\n        id: QueueId,\n    ) -> impl Future<Output = trc::Result<Option<Archive<AlignedBytes>>>> + Send;\n}\n\nimpl SmtpSpool for Server {\n    fn new_message(&self, return_path: impl AsRef<str>, span_id: u64) -> MessageWrapper {\n        let created = SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .map_or(0, |d| d.as_secs());\n\n        MessageWrapper {\n            queue_id: self.inner.data.queue_id_gen.generate(),\n            queue_name: QueueName::default(),\n            is_multi_queue: false,\n            span_id,\n            message: Message {\n                created,\n                return_path: return_path.to_lowercase_domain().into_boxed_str(),\n                recipients: Vec::with_capacity(1),\n                flags: 0,\n                env_id: None,\n                priority: 0,\n                size: 0,\n                blob_hash: Default::default(),\n                quota_keys: Default::default(),\n                received_from_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),\n                received_via_port: 0,\n            },\n        }\n    }\n\n    async fn next_event(&self, queue: &mut Queue) -> QueuedMessages {\n        let now = now();\n        let from_key = ValueKey::from(ValueClass::Queue(QueueClass::MessageEvent(\n            store::write::QueueEvent {\n                due: 0,\n                queue_id: 0,\n                queue_name: [0; 8],\n            },\n        )));\n        let to_key = ValueKey::from(ValueClass::Queue(QueueClass::MessageEvent(\n            store::write::QueueEvent {\n                due: now + QUEUE_REFRESH,\n                queue_id: u64::MAX,\n                queue_name: [u8::MAX; 8],\n            },\n        )));\n\n        let mut events = QueuedMessages {\n            messages: Vec::new(),\n            next_refresh: now + QUEUE_REFRESH,\n        };\n\n        queue.locked_revision += 1;\n        let result = self\n            .store()\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending().no_values(),\n                |key, _| {\n                    let due = key.deserialize_be_u64(0)?;\n\n                    if due <= now {\n                        let queue_id = key.deserialize_be_u64(U64_LEN)?;\n                        let queue_name = key\n                            .get(U64_LEN + U64_LEN..)\n                            .and_then(QueueName::from_bytes)\n                            .ok_or_else(|| {\n                                trc::StoreEvent::DataCorruption\n                                    .caused_by(trc::location!())\n                                    .ctx(trc::Key::Key, key)\n                            })?;\n\n                        let add_event = queue\n                            .stats\n                            .get(&queue_name)\n                            .is_none_or(|stats| stats.has_capacity())\n                            && match queue.locked.entry((queue_id, queue_name)) {\n                                Entry::Occupied(mut entry) => {\n                                    let locked = entry.get_mut();\n                                    locked.revision = queue.locked_revision;\n                                    if locked.expires <= now {\n                                        locked.expires = now + INFINITE_LOCK;\n\n                                        true\n                                    } else {\n                                        if locked.expires < events.next_refresh {\n                                            events.next_refresh = locked.expires;\n                                        }\n\n                                        false\n                                    }\n                                }\n                                Entry::Vacant(entry) => {\n                                    entry.insert(LockedMessage {\n                                        expires: now + INFINITE_LOCK,\n                                        revision: queue.locked_revision,\n                                    });\n                                    true\n                                }\n                            };\n\n                        if add_event {\n                            events.messages.push(QueuedMessage {\n                                due,\n                                queue_id,\n                                queue_name,\n                            });\n                        }\n\n                        Ok(true)\n                    } else {\n                        if due < events.next_refresh {\n                            events.next_refresh = due;\n                        }\n                        Ok(false)\n                    }\n                },\n            )\n            .await;\n\n        if let Err(err) = result {\n            trc::error!(\n                err.details(\"Failed to read queue.\")\n                    .caused_by(trc::location!())\n            );\n        }\n\n        events\n    }\n\n    async fn try_lock_event(&self, queue_id: QueueId, queue_name: QueueName) -> bool {\n        match self\n            .in_memory_store()\n            .try_lock(\n                KV_LOCK_QUEUE_MESSAGE,\n                &lock_id(queue_id, queue_name),\n                LOCK_EXPIRY,\n            )\n            .await\n        {\n            Ok(result) => {\n                if !result {\n                    trc::event!(\n                        Queue(trc::QueueEvent::Locked),\n                        QueueId = queue_id,\n                        QueueName = queue_name.to_string()\n                    );\n                }\n                result\n            }\n            Err(err) => {\n                trc::error!(\n                    err.details(\"Failed to lock event.\")\n                        .caused_by(trc::location!())\n                );\n                false\n            }\n        }\n    }\n\n    async fn unlock_event(&self, queue_id: QueueId, queue_name: QueueName) {\n        if let Err(err) = self\n            .in_memory_store()\n            .remove_lock(KV_LOCK_QUEUE_MESSAGE, &lock_id(queue_id, queue_name))\n            .await\n        {\n            trc::error!(\n                err.details(\"Failed to unlock event.\")\n                    .caused_by(trc::location!())\n            );\n        }\n    }\n\n    async fn read_message(\n        &self,\n        queue_id: QueueId,\n        queue_name: QueueName,\n    ) -> Option<MessageWrapper> {\n        match self\n            .read_message_archive(queue_id)\n            .await\n            .and_then(|a| match a {\n                Some(a) => a.deserialize::<Message>().map(Some),\n                None => Ok(None),\n            }) {\n            Ok(Some(message)) => Some(MessageWrapper {\n                is_multi_queue: message.recipients.iter().any(|rcpt| {\n                    matches!(rcpt.status, Status::Scheduled | Status::TemporaryFailure(_))\n                        && rcpt.queue != queue_name\n                }),\n                queue_id,\n                queue_name,\n                span_id: 0,\n                message,\n            }),\n            Ok(None) => None,\n            Err(err) => {\n                trc::error!(\n                    err.details(\"Failed to read message.\")\n                        .caused_by(trc::location!())\n                );\n\n                None\n            }\n        }\n    }\n\n    async fn read_message_archive(\n        &self,\n        id: QueueId,\n    ) -> trc::Result<Option<Archive<AlignedBytes>>> {\n        self.store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Queue(\n                QueueClass::Message(id),\n            )))\n            .await\n    }\n}\n\nfn lock_id(queue_id: QueueId, queue_name: QueueName) -> [u8; 16] {\n    let mut id = [0; 16];\n    id[..8].copy_from_slice(&queue_id.to_be_bytes());\n    id[8..].copy_from_slice(queue_name.as_ref());\n    id\n}\n\nimpl MessageWrapper {\n    pub async fn queue(\n        mut self,\n        raw_headers: Option<&[u8]>,\n        raw_message: &[u8],\n        session_id: u64,\n        server: &Server,\n        source: MessageSource,\n    ) -> bool {\n        // Set flags\n        let (flags, event, train_spam) = match source {\n            MessageSource::Authenticated => (\n                FROM_AUTHENTICATED,\n                trc::QueueEvent::QueueMessageAuthenticated,\n                None,\n            ),\n            MessageSource::Unauthenticated {\n                dmarc_pass: true,\n                train_spam,\n            } => (\n                FROM_UNAUTHENTICATED_DMARC,\n                trc::QueueEvent::QueueMessage,\n                train_spam,\n            ),\n            MessageSource::Unauthenticated {\n                dmarc_pass: false,\n                train_spam,\n            } => (\n                FROM_UNAUTHENTICATED,\n                trc::QueueEvent::QueueMessage,\n                train_spam,\n            ),\n            MessageSource::Dsn => (FROM_DSN, trc::QueueEvent::QueueDsn, None),\n            MessageSource::Report => (FROM_REPORT, trc::QueueEvent::QueueReport, None),\n            MessageSource::Autogenerated => (\n                FROM_AUTOGENERATED,\n                trc::QueueEvent::QueueAutogenerated,\n                None,\n            ),\n        };\n        self.message.flags |= flags;\n\n        // Write blob\n        let message = if let Some(raw_headers) = raw_headers {\n            let mut message = Vec::with_capacity(raw_headers.len() + raw_message.len());\n            message.extend_from_slice(raw_headers);\n            message.extend_from_slice(raw_message);\n            Cow::Owned(message)\n        } else {\n            raw_message.into()\n        };\n        self.message.blob_hash = BlobHash::generate(message.as_ref());\n\n        // Generate id\n        if self.message.size == 0 {\n            self.message.size = message.len() as u64;\n        }\n\n        // Reserve and write blob\n        let mut batch = BatchBuilder::new();\n        let now = now();\n        let reserve_until = now + 120;\n        batch.set(\n            BlobOp::Link {\n                hash: self.message.blob_hash.clone(),\n                to: BlobLink::Temporary {\n                    until: reserve_until,\n                },\n            },\n            vec![],\n        );\n        if let Err(err) = server.store().write(batch.build_all()).await {\n            trc::error!(\n                err.details(\"Failed to write to store.\")\n                    .span_id(session_id)\n                    .caused_by(trc::location!())\n            );\n\n            return false;\n        }\n        if let Err(err) = server\n            .blob_store()\n            .put_blob(self.message.blob_hash.as_slice(), message.as_ref())\n            .await\n        {\n            trc::error!(\n                err.details(\"Failed to write blob.\")\n                    .span_id(session_id)\n                    .caused_by(trc::location!())\n            );\n\n            return false;\n        }\n\n        trc::event!(\n            Queue(event),\n            SpanId = session_id,\n            QueueId = self.queue_id,\n            From = if !self.message.return_path.is_empty() {\n                trc::Value::String(self.message.return_path.as_ref().into())\n            } else {\n                trc::Value::String(\"<>\".into())\n            },\n            To = self\n                .message\n                .recipients\n                .iter()\n                .map(|r| trc::Value::String(r.address.as_ref().into()))\n                .collect::<Vec<_>>(),\n            Size = self.message.size,\n            NextRetry = self\n                .message\n                .next_delivery_event(None)\n                .map(trc::Value::Timestamp),\n            NextDsn = self.message.next_dsn(None).map(trc::Value::Timestamp),\n            Expires = self.message.expires(None).map(trc::Value::Timestamp),\n        );\n\n        // Write message to queue\n        let mut batch = BatchBuilder::new();\n\n        // Reserve quotas\n        for quota_key in &self.message.quota_keys {\n            match quota_key {\n                QuotaKey::Count { key, .. } => {\n                    batch.add(ValueClass::Queue(QueueClass::QuotaCount(key.to_vec())), 1);\n                }\n                QuotaKey::Size { key, .. } => {\n                    batch.add(\n                        ValueClass::Queue(QueueClass::QuotaSize(key.to_vec())),\n                        self.message.size as i64,\n                    );\n                }\n            }\n        }\n\n        for (queue_name, due) in self.message.next_events() {\n            batch.set(\n                ValueClass::Queue(QueueClass::MessageEvent(store::write::QueueEvent {\n                    due,\n                    queue_id: self.queue_id,\n                    queue_name: queue_name.into_inner(),\n                })),\n                Vec::new(),\n            );\n        }\n\n        if let Some(is_spam) = train_spam\n            && let Some(config) = &server.core.spam.classifier\n        {\n            let hold_period = now + config.hold_samples_for;\n\n            batch\n                .set(\n                    BlobOp::Link {\n                        hash: self.message.blob_hash.clone(),\n                        to: BlobLink::Temporary { until: hold_period },\n                    },\n                    vec![BlobLink::SPAM_SAMPLE_LINK],\n                )\n                .set(\n                    BlobOp::SpamSample {\n                        hash: self.message.blob_hash.clone(),\n                        until: hold_period,\n                    },\n                    vec![u8::from(is_spam), 1],\n                );\n\n            trc::event!(\n                Spam(SpamEvent::TrainSampleAdded),\n                Details = if is_spam { \"spam\" } else { \"ham\" },\n                Expires = trc::Value::Timestamp(hold_period),\n                SpanId = self.span_id,\n            );\n        }\n\n        batch\n            .clear(BlobOp::Link {\n                hash: self.message.blob_hash.clone(),\n                to: BlobLink::Temporary {\n                    until: reserve_until,\n                },\n            })\n            .set(\n                BlobOp::Link {\n                    hash: self.message.blob_hash.clone(),\n                    to: BlobLink::Id { id: self.queue_id },\n                },\n                vec![],\n            )\n            .set(\n                BlobOp::Commit {\n                    hash: self.message.blob_hash.clone(),\n                },\n                vec![],\n            )\n            .set(\n                ValueClass::Queue(QueueClass::Message(self.queue_id)),\n                match Archiver::new(self.message).serialize() {\n                    Ok(data) => data,\n                    Err(err) => {\n                        trc::error!(\n                            err.details(\"Failed to serialize message.\")\n                                .span_id(session_id)\n                                .caused_by(trc::location!())\n                        );\n                        return false;\n                    }\n                },\n            );\n\n        if let Err(err) = server.store().write(batch.build_all()).await {\n            trc::error!(\n                err.details(\"Failed to write to store.\")\n                    .span_id(session_id)\n                    .caused_by(trc::location!())\n            );\n\n            return false;\n        }\n\n        // Queue the message\n        if server\n            .inner\n            .ipc\n            .queue_tx\n            .send(QueueEvent::Refresh)\n            .await\n            .is_err()\n        {\n            trc::event!(\n                Server(ServerEvent::ThreadError),\n                Reason = \"Channel closed.\",\n                CausedBy = trc::location!(),\n                SpanId = session_id,\n            );\n        }\n\n        true\n    }\n\n    pub async fn add_recipient(&mut self, rcpt: impl AsRef<str>, server: &Server) {\n        // Resolve queue\n        self.message.recipients.push(Recipient::new(rcpt));\n        let queue = server.get_queue_or_default(\n            &server\n                .eval_if::<String, _>(\n                    &server.core.smtp.queue.queue,\n                    &QueueEnvelope::new(&self.message, self.message.recipients.last().unwrap()),\n                    self.span_id,\n                )\n                .await\n                .unwrap_or_else(|| \"default\".to_string()),\n            self.span_id,\n        );\n\n        // Update expiration\n        let now = now();\n        let recipient = self.message.recipients.last_mut().unwrap();\n        recipient.notify = Schedule::later(queue.notify.first().copied().unwrap_or(86400) + now);\n        recipient.expires = queue.expiry;\n        recipient.queue = queue.virtual_queue;\n    }\n\n    pub async fn save_changes(mut self, server: &Server, prev_event: Option<u64>) -> bool {\n        // Release quota for completed deliveries\n        let mut batch = BatchBuilder::new();\n        self.release_quota(&mut batch);\n\n        // Update message queue\n        if let Some(prev_event) = prev_event {\n            batch.clear(ValueClass::Queue(QueueClass::MessageEvent(\n                store::write::QueueEvent {\n                    due: prev_event,\n                    queue_id: self.queue_id,\n                    queue_name: self.queue_name.into_inner(),\n                },\n            )));\n        }\n        for (queue_name, due) in self.message.next_events() {\n            batch.set(\n                ValueClass::Queue(QueueClass::MessageEvent(store::write::QueueEvent {\n                    due,\n                    queue_id: self.queue_id,\n                    queue_name: queue_name.into_inner(),\n                })),\n                Vec::new(),\n            );\n        }\n\n        let message_bytes = match Archiver::new(self.message).serialize() {\n            Ok(data) => data,\n            Err(err) => {\n                trc::error!(\n                    err.details(\"Failed to serialize message.\")\n                        .span_id(self.span_id)\n                        .caused_by(trc::location!())\n                );\n                return false;\n            }\n        };\n        if self.is_multi_queue {\n            batch.merge_fnc(\n                ValueClass::Queue(QueueClass::Message(self.queue_id)),\n                Params::with_capacity(3)\n                    .with_u64(self.queue_id)\n                    .with_bytes(self.queue_name.into_inner().to_vec())\n                    .with_bytes(message_bytes),\n                |params, _, bytes| {\n                    let mut cur_message = <Archive<AlignedBytes> as Deserialize>::deserialize(\n                        bytes.ok_or_else(|| {\n                            trc::StoreEvent::NotFound\n                                .into_err()\n                                .details(\"Message no longer exists.\")\n                                .caused_by(trc::location!())\n                                .ctx(trc::Key::QueueId, params.u64(0))\n                        })?,\n                    )\n                    .and_then(|archive| archive.deserialize::<Message>())\n                    .caused_by(trc::location!())?;\n\n                    let new_message_ =\n                        <Archive<AlignedBytes> as Deserialize>::deserialize(params.bytes(2))\n                            .caused_by(trc::location!())?;\n                    let new_message = new_message_\n                        .unarchive::<Message>()\n                        .caused_by(trc::location!())?;\n\n                    if cur_message.blob_hash.as_slice() == new_message.blob_hash.0.as_slice()\n                        && cur_message.recipients.len() == new_message.recipients.len()\n                    {\n                        let queue_name = params.bytes(1);\n                        for (rcpt_idx, rcpt) in new_message\n                            .recipients\n                            .iter()\n                            .enumerate()\n                            .filter(|(_, rcpt)| rcpt.queue.as_slice() == queue_name)\n                        {\n                            cur_message.recipients[rcpt_idx] =\n                                rkyv_deserialize(rcpt).caused_by(trc::location!())?;\n                        }\n\n                        Archiver::new(cur_message)\n                            .serialize()\n                            .caused_by(trc::location!())\n                            .map(MergeResult::Update)\n                    } else {\n                        Err(trc::StoreEvent::UnexpectedError\n                            .into_err()\n                            .details(\"Message blob hash or recipient count mismatch.\")\n                            .caused_by(trc::location!())\n                            .ctx(trc::Key::QueueId, params.u64(0)))\n                    }\n                },\n            );\n        } else {\n            batch.set(\n                ValueClass::Queue(QueueClass::Message(self.queue_id)),\n                message_bytes,\n            );\n        }\n\n        if let Err(err) = server.store().write(batch.build_all()).await {\n            trc::error!(\n                err.details(\"Failed to save changes.\")\n                    .span_id(self.span_id)\n                    .caused_by(trc::location!())\n            );\n            false\n        } else {\n            true\n        }\n    }\n\n    pub async fn remove(self, server: &Server, prev_event: Option<u64>) -> bool {\n        let mut batch = BatchBuilder::new();\n\n        if let Some(prev_event) = prev_event {\n            batch.clear(ValueClass::Queue(QueueClass::MessageEvent(\n                store::write::QueueEvent {\n                    due: prev_event,\n                    queue_id: self.queue_id,\n                    queue_name: self.queue_name.into_inner(),\n                },\n            )));\n        } else {\n            for (queue_name, due) in self.message.next_events() {\n                batch.clear(ValueClass::Queue(QueueClass::MessageEvent(\n                    store::write::QueueEvent {\n                        due,\n                        queue_id: self.queue_id,\n                        queue_name: queue_name.into_inner(),\n                    },\n                )));\n            }\n        }\n\n        // Release all quotas\n        for quota_key in self.message.quota_keys {\n            match quota_key {\n                QuotaKey::Count { key, .. } => {\n                    batch.add(ValueClass::Queue(QueueClass::QuotaCount(key.to_vec())), -1);\n                }\n                QuotaKey::Size { key, .. } => {\n                    batch.add(\n                        ValueClass::Queue(QueueClass::QuotaSize(key.to_vec())),\n                        -(self.message.size as i64),\n                    );\n                }\n            }\n        }\n\n        batch\n            .clear(BlobOp::Link {\n                hash: self.message.blob_hash.clone(),\n                to: BlobLink::Id { id: self.queue_id },\n            })\n            .clear(ValueClass::Queue(QueueClass::Message(self.queue_id)));\n\n        if let Err(err) = server.store().write(batch.build_all()).await {\n            trc::error!(\n                err.details(\"Failed to write to update queue.\")\n                    .span_id(self.span_id)\n                    .caused_by(trc::location!())\n            );\n            false\n        } else {\n            true\n        }\n    }\n\n    pub fn has_domain(&self, domains: &[String]) -> bool {\n        self.message.recipients.iter().any(|r| {\n            let domain = r.address.domain_part();\n            domains.iter().any(|dd| dd == domain)\n        }) || self\n            .message\n            .return_path\n            .rsplit_once('@')\n            .is_some_and(|(_, domain)| domains.iter().any(|dd| dd == domain))\n    }\n}\n\nimpl ArchivedMessage {\n    pub fn has_domain(&self, domains: &[String]) -> bool {\n        self.recipients.iter().any(|r| {\n            let domain = r.address.domain_part();\n            domains.iter().any(|dd| dd == domain)\n        }) || self\n            .return_path\n            .rsplit_once('@')\n            .is_some_and(|(_, domain)| domains.iter().any(|dd| dd == domain))\n    }\n\n    pub fn next_delivery_event(&self, queue: Option<QueueName>) -> Option<u64> {\n        let mut next_delivery = None;\n\n        for rcpt in self.recipients.iter().filter(|d| {\n            matches!(\n                d.status,\n                ArchivedStatus::Scheduled | ArchivedStatus::TemporaryFailure(_)\n            ) && queue.is_none_or(|q| d.queue == q)\n        }) {\n            let retry_due = rcpt.retry.due.to_native();\n            if let Some(next_delivery) = &mut next_delivery {\n                if retry_due < *next_delivery {\n                    *next_delivery = retry_due;\n                }\n            } else {\n                next_delivery = Some(retry_due);\n            }\n        }\n\n        next_delivery\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/queue/throttle.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::core::throttle::NewKey;\nuse common::{\n    KV_RATE_LIMIT_SMTP, Server, config::smtp::QueueRateLimiter, expr::functions::ResolveVariable,\n};\nuse std::future::Future;\nuse store::write::now;\n\npub trait IsAllowed: Sync + Send {\n    fn is_allowed<'x>(\n        &'x self,\n        throttle: &'x QueueRateLimiter,\n        envelope: &impl ResolveVariable,\n        session_id: u64,\n    ) -> impl Future<Output = Result<(), u64>> + Send;\n}\n\nimpl IsAllowed for Server {\n    async fn is_allowed<'x>(\n        &'x self,\n        throttle: &'x QueueRateLimiter,\n        envelope: &impl ResolveVariable,\n        session_id: u64,\n    ) -> Result<(), u64> {\n        if throttle.expr.is_empty()\n            || self\n                .eval_expr(&throttle.expr, envelope, \"throttle\", session_id)\n                .await\n                .unwrap_or(false)\n        {\n            let key = throttle.new_key(envelope, \"outbound\");\n\n            match self\n                .core\n                .storage\n                .lookup\n                .is_rate_allowed(KV_RATE_LIMIT_SMTP, key.as_ref(), &throttle.rate, false)\n                .await\n            {\n                Ok(Some(next_refill)) => {\n                    trc::event!(\n                        Queue(trc::QueueEvent::RateLimitExceeded),\n                        SpanId = session_id,\n                        Id = throttle.id.clone(),\n                        Limit = vec![\n                            trc::Value::from(throttle.rate.requests),\n                            trc::Value::from(throttle.rate.period)\n                        ],\n                    );\n\n                    return Err(now() + next_refill);\n                }\n                Err(err) => {\n                    trc::error!(err.span_id(session_id).caused_by(trc::location!()));\n                }\n                _ => (),\n            }\n        }\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/reporting/analysis.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashMap;\nuse common::Server;\nuse mail_auth::{\n    flate2::read::GzDecoder,\n    report::{ActionDisposition, DmarcResult, Feedback, Report, tlsrpt::TlsReport},\n    zip,\n};\nuse mail_parser::{Message, MimeHeaders, PartType};\nuse std::{\n    borrow::Cow,\n    collections::hash_map::Entry,\n    io::{Cursor, Read},\n};\nuse store::{\n    Serialize,\n    write::{Archiver, BatchBuilder, ReportClass, ValueClass, now},\n};\nuse trc::IncomingReportEvent;\n\nenum Compression {\n    None,\n    Gzip,\n    Zip,\n}\n\nenum Format<D, T, A> {\n    Dmarc(D),\n    Tls(T),\n    Arf(A),\n}\n\nstruct ReportData<'x> {\n    compression: Compression,\n    format: Format<(), (), ()>,\n    data: &'x [u8],\n}\n\n#[derive(\n    rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, serde::Serialize, serde::Deserialize,\n)]\npub struct IncomingReport<T> {\n    pub from: String,\n    pub to: Vec<String>,\n    pub subject: String,\n    pub report: T,\n}\n\npub trait AnalyzeReport: Sync + Send {\n    fn analyze_report(&self, message: Message<'static>, session_id: u64);\n}\n\nimpl AnalyzeReport for Server {\n    fn analyze_report(&self, message: Message<'static>, session_id: u64) {\n        let core = self.clone();\n        tokio::spawn(async move {\n            let from: String = message\n                .from()\n                .and_then(|a| a.last())\n                .and_then(|a| a.address())\n                .unwrap_or_default()\n                .into();\n            let to: Vec<String> = message.to().map_or_else(Vec::new, |a| {\n                a.iter()\n                    .filter_map(|a| a.address())\n                    .map(|a| a.into())\n                    .collect()\n            });\n            let subject: String = message.subject().unwrap_or_default().into();\n            let mut reports = Vec::new();\n\n            for part in &message.parts {\n                match &part.body {\n                    PartType::Text(report) => {\n                        if part\n                            .content_type()\n                            .and_then(|ct| ct.subtype())\n                            .is_some_and(|t| t.eq_ignore_ascii_case(\"xml\"))\n                            || part\n                                .attachment_name()\n                                .and_then(|n| n.rsplit_once('.'))\n                                .is_some_and(|(_, e)| e.eq_ignore_ascii_case(\"xml\"))\n                        {\n                            reports.push(ReportData {\n                                compression: Compression::None,\n                                format: Format::Dmarc(()),\n                                data: report.as_bytes(),\n                            });\n                        } else if part.is_content_type(\"message\", \"feedback-report\") {\n                            reports.push(ReportData {\n                                compression: Compression::None,\n                                format: Format::Arf(()),\n                                data: report.as_bytes(),\n                            });\n                        }\n                    }\n                    PartType::Binary(report) | PartType::InlineBinary(report) => {\n                        if part.is_content_type(\"message\", \"feedback-report\") {\n                            reports.push(ReportData {\n                                compression: Compression::None,\n                                format: Format::Arf(()),\n                                data: report.as_ref(),\n                            });\n                            continue;\n                        }\n\n                        let subtype = part\n                            .content_type()\n                            .and_then(|ct| ct.subtype())\n                            .unwrap_or(\"\");\n                        let attachment_name = part.attachment_name();\n                        let ext = attachment_name\n                            .and_then(|f| f.rsplit_once('.'))\n                            .map_or(\"\", |(_, e)| e);\n                        let tls_parts = subtype.rsplit_once('+');\n                        let compression = match (tls_parts.map(|(_, c)| c).unwrap_or(subtype), ext)\n                        {\n                            (\"gzip\", _) => Compression::Gzip,\n                            (\"zip\", _) => Compression::Zip,\n                            (_, \"gz\") => Compression::Gzip,\n                            (_, \"zip\") => Compression::Zip,\n                            _ => Compression::None,\n                        };\n                        let format = match (tls_parts.map(|(c, _)| c).unwrap_or(subtype), ext) {\n                            (\"xml\", _) => Format::Dmarc(()),\n                            (\"tlsrpt\", _) | (_, \"json\") => Format::Tls(()),\n                            _ => {\n                                if attachment_name\n                                    .is_some_and(|n| n.contains(\".xml\") || n.contains('!'))\n                                {\n                                    Format::Dmarc(())\n                                } else {\n                                    continue;\n                                }\n                            }\n                        };\n\n                        reports.push(ReportData {\n                            compression,\n                            format,\n                            data: report.as_ref(),\n                        });\n                    }\n                    _ => (),\n                }\n            }\n\n            for report in reports {\n                let data = match report.compression {\n                    Compression::None => Cow::Borrowed(report.data),\n                    Compression::Gzip => {\n                        let mut file = GzDecoder::new(report.data);\n                        let mut buf = Vec::new();\n                        if let Err(err) = file.read_to_end(&mut buf) {\n                            trc::event!(\n                                IncomingReport(IncomingReportEvent::DecompressError),\n                                SpanId = session_id,\n                                From = from.to_string(),\n                                Reason = err.to_string(),\n                                CausedBy = trc::location!()\n                            );\n\n                            continue;\n                        }\n                        Cow::Owned(buf)\n                    }\n                    Compression::Zip => {\n                        let mut archive = match zip::ZipArchive::new(Cursor::new(report.data)) {\n                            Ok(archive) => archive,\n                            Err(err) => {\n                                trc::event!(\n                                    IncomingReport(IncomingReportEvent::DecompressError),\n                                    SpanId = session_id,\n                                    From = from.to_string(),\n                                    Reason = err.to_string(),\n                                    CausedBy = trc::location!()\n                                );\n\n                                continue;\n                            }\n                        };\n                        let mut buf = Vec::with_capacity(0);\n                        for i in 0..archive.len() {\n                            match archive.by_index(i) {\n                                Ok(mut file) => {\n                                    buf = Vec::with_capacity(file.compressed_size() as usize);\n                                    if let Err(err) = file.read_to_end(&mut buf) {\n                                        trc::event!(\n                                            IncomingReport(IncomingReportEvent::DecompressError),\n                                            SpanId = session_id,\n                                            From = from.to_string(),\n                                            Reason = err.to_string(),\n                                            CausedBy = trc::location!()\n                                        );\n                                    }\n                                    break;\n                                }\n                                Err(err) => {\n                                    trc::event!(\n                                        IncomingReport(IncomingReportEvent::DecompressError),\n                                        SpanId = session_id,\n                                        From = from.to_string(),\n                                        Reason = err.to_string(),\n                                        CausedBy = trc::location!()\n                                    );\n                                }\n                            }\n                        }\n                        Cow::Owned(buf)\n                    }\n                };\n\n                let report = match report.format {\n                    Format::Dmarc(_) => match Report::parse_xml(&data) {\n                        Ok(report) => {\n                            // Log\n                            report.log();\n                            Format::Dmarc(report)\n                        }\n                        Err(err) => {\n                            trc::event!(\n                                IncomingReport(IncomingReportEvent::DmarcParseFailed),\n                                SpanId = session_id,\n                                From = from.to_string(),\n                                Reason = err,\n                                CausedBy = trc::location!()\n                            );\n\n                            continue;\n                        }\n                    },\n                    Format::Tls(_) => match TlsReport::parse_json(&data) {\n                        Ok(report) => {\n                            // Log\n                            report.log();\n                            Format::Tls(report)\n                        }\n                        Err(err) => {\n                            trc::event!(\n                                IncomingReport(IncomingReportEvent::TlsRpcParseFailed),\n                                SpanId = session_id,\n                                From = from.to_string(),\n                                Reason = format!(\"{err:?}\"),\n                                CausedBy = trc::location!()\n                            );\n\n                            continue;\n                        }\n                    },\n                    Format::Arf(_) => match Feedback::parse_arf(&data) {\n                        Some(report) => {\n                            // Log\n                            report.log();\n                            Format::Arf(report.into_owned())\n                        }\n                        None => {\n                            trc::event!(\n                                IncomingReport(IncomingReportEvent::ArfParseFailed),\n                                SpanId = session_id,\n                                From = from.to_string(),\n                                CausedBy = trc::location!()\n                            );\n\n                            continue;\n                        }\n                    },\n                };\n\n                // Store report\n                if let Some(expires_in) = &core.core.smtp.report.analysis.store {\n                    let expires = now() + expires_in.as_secs();\n                    let id = core.inner.data.queue_id_gen.generate();\n\n                    let mut batch = BatchBuilder::new();\n                    match report {\n                        Format::Dmarc(report) => {\n                            batch.set(\n                                ValueClass::Report(ReportClass::Dmarc { id, expires }),\n                                Archiver::new(IncomingReport {\n                                    from,\n                                    to,\n                                    subject,\n                                    report,\n                                })\n                                .serialize()\n                                .unwrap_or_default(),\n                            );\n                        }\n                        Format::Tls(report) => {\n                            batch.set(\n                                ValueClass::Report(ReportClass::Tls { id, expires }),\n                                Archiver::new(IncomingReport {\n                                    from,\n                                    to,\n                                    subject,\n                                    report,\n                                })\n                                .serialize()\n                                .unwrap_or_default(),\n                            );\n                        }\n                        Format::Arf(report) => {\n                            batch.set(\n                                ValueClass::Report(ReportClass::Arf { id, expires }),\n                                Archiver::new(IncomingReport {\n                                    from,\n                                    to,\n                                    subject,\n                                    report,\n                                })\n                                .serialize()\n                                .unwrap_or_default(),\n                            );\n                        }\n                    }\n                    if let Err(err) = core.core.storage.data.write(batch.build_all()).await {\n                        trc::error!(\n                            err.span_id(session_id)\n                                .caused_by(trc::location!())\n                                .details(\"Failed to write report\")\n                        );\n                    }\n                }\n                return;\n            }\n        });\n    }\n}\n\ntrait LogReport {\n    fn log(&self);\n}\n\nimpl LogReport for Report {\n    fn log(&self) {\n        let mut dmarc_pass = 0;\n        let mut dmarc_quarantine = 0;\n        let mut dmarc_reject = 0;\n        let mut dmarc_none = 0;\n        let mut dkim_pass = 0;\n        let mut dkim_fail = 0;\n        let mut dkim_none = 0;\n        let mut spf_pass = 0;\n        let mut spf_fail = 0;\n        let mut spf_none = 0;\n\n        for record in self.records() {\n            let count = std::cmp::min(record.count(), 1);\n\n            match record.action_disposition() {\n                ActionDisposition::Pass => {\n                    dmarc_pass += count;\n                }\n                ActionDisposition::Quarantine => {\n                    dmarc_quarantine += count;\n                }\n                ActionDisposition::Reject => {\n                    dmarc_reject += count;\n                }\n                ActionDisposition::None | ActionDisposition::Unspecified => {\n                    dmarc_none += count;\n                }\n            }\n            match record.dmarc_dkim_result() {\n                DmarcResult::Pass => {\n                    dkim_pass += count;\n                }\n                DmarcResult::Fail => {\n                    dkim_fail += count;\n                }\n                DmarcResult::Unspecified => {\n                    dkim_none += count;\n                }\n            }\n            match record.dmarc_spf_result() {\n                DmarcResult::Pass => {\n                    spf_pass += count;\n                }\n                DmarcResult::Fail => {\n                    spf_fail += count;\n                }\n                DmarcResult::Unspecified => {\n                    spf_none += count;\n                }\n            }\n        }\n\n        trc::event!(\n            IncomingReport(\n                if (dmarc_reject + dmarc_quarantine + dkim_fail + spf_fail) > 0 {\n                    IncomingReportEvent::DmarcReportWithWarnings\n                } else {\n                    IncomingReportEvent::DmarcReport\n                }\n            ),\n            RangeFrom = trc::Value::Timestamp(self.date_range_begin()),\n            RangeTo = trc::Value::Timestamp(self.date_range_end()),\n            Domain = self.domain().to_string(),\n            From = self.email().to_string(),\n            Id = self.report_id().to_string(),\n            DmarcPass = dmarc_pass,\n            DmarcQuarantine = dmarc_quarantine,\n            DmarcReject = dmarc_reject,\n            DmarcNone = dmarc_none,\n            DkimPass = dkim_pass,\n            DkimFail = dkim_fail,\n            DkimNone = dkim_none,\n            SpfPass = spf_pass,\n            SpfFail = spf_fail,\n            SpfNone = spf_none,\n        );\n    }\n}\n\nimpl LogReport for TlsReport {\n    fn log(&self) {\n        for policy in self.policies.iter().take(5) {\n            let mut details = AHashMap::with_capacity(policy.failure_details.len());\n            for failure in &policy.failure_details {\n                let num_failures = std::cmp::min(1, failure.failed_session_count);\n                match details.entry(failure.result_type) {\n                    Entry::Occupied(mut e) => {\n                        *e.get_mut() += num_failures;\n                    }\n                    Entry::Vacant(e) => {\n                        e.insert(num_failures);\n                    }\n                }\n            }\n\n            trc::event!(\n                IncomingReport(if policy.summary.total_failure > 0 {\n                    IncomingReportEvent::TlsReportWithWarnings\n                } else {\n                    IncomingReportEvent::TlsReport\n                }),\n                RangeFrom =\n                    trc::Value::Timestamp(self.date_range.start_datetime.to_timestamp() as u64),\n                RangeTo = trc::Value::Timestamp(self.date_range.end_datetime.to_timestamp() as u64),\n                Domain = policy.policy.policy_domain.clone(),\n                From = self.contact_info.as_deref().unwrap_or_default().to_string(),\n                Id = self.report_id.clone(),\n                Policy = format!(\"{:?}\", policy.policy.policy_type),\n                TotalSuccesses = policy.summary.total_success,\n                TotalFailures = policy.summary.total_failure,\n                Details = format!(\"{details:?}\"),\n            );\n        }\n    }\n}\n\nimpl LogReport for Feedback<'_> {\n    fn log(&self) {\n        trc::event!(\n            IncomingReport(match self.feedback_type() {\n                mail_auth::report::FeedbackType::Abuse => IncomingReportEvent::AbuseReport,\n                mail_auth::report::FeedbackType::AuthFailure =>\n                    IncomingReportEvent::AuthFailureReport,\n                mail_auth::report::FeedbackType::Fraud => IncomingReportEvent::FraudReport,\n                mail_auth::report::FeedbackType::NotSpam => IncomingReportEvent::NotSpamReport,\n                mail_auth::report::FeedbackType::Other => IncomingReportEvent::OtherReport,\n                mail_auth::report::FeedbackType::Virus => IncomingReportEvent::VirusReport,\n            }),\n            RangeFrom = trc::Value::Timestamp(\n                self.arrival_date()\n                    .map(|d| d as u64)\n                    .unwrap_or_else(|| { now() })\n            ),\n            Domain = self\n                .reported_domain()\n                .iter()\n                .map(|d| trc::Value::String(d.as_ref().into()))\n                .collect::<Vec<_>>(),\n            Hostname = self.reporting_mta().map(|d| trc::Value::String(d.into())),\n            Url = self\n                .reported_uri()\n                .iter()\n                .map(|d| trc::Value::String(d.as_ref().into()))\n                .collect::<Vec<_>>(),\n            RemoteIp = self.source_ip(),\n            Total = self.incidents(),\n            Result = format!(\"{:?}\", self.delivery_result()),\n            Details = self\n                .authentication_results()\n                .iter()\n                .map(|d| trc::Value::String(d.as_ref().into()))\n                .collect::<Vec<_>>(),\n        );\n    }\n}\n\nimpl<T> IncomingReport<T> {\n    pub fn has_domain(&self, domain: &[String]) -> bool {\n        self.to\n            .iter()\n            .any(|to| domain.iter().any(|d| to.ends_with(d.as_str())))\n            || domain.iter().any(|d| self.from.ends_with(d.as_str()))\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/reporting/dkim.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::listener::SessionStream;\n\nuse mail_auth::{\n    AuthenticatedMessage, AuthenticationResults, DkimOutput, common::verify::VerifySignature,\n};\nuse trc::OutgoingReportEvent;\nuse utils::config::Rate;\n\nuse crate::{core::Session, reporting::SmtpReporting};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn send_dkim_report(\n        &self,\n        rcpt: &str,\n        message: &AuthenticatedMessage<'_>,\n        rate: &Rate,\n        rejected: bool,\n        output: &DkimOutput<'_>,\n    ) {\n        // Generate report\n        let signature = if let Some(signature) = output.signature() {\n            signature\n        } else {\n            return;\n        };\n\n        // Throttle recipient\n        if !self.throttle_rcpt(rcpt, rate, \"dkim\").await {\n            trc::event!(\n                OutgoingReport(OutgoingReportEvent::DkimRateLimited),\n                SpanId = self.data.session_id,\n                To = rcpt.to_string(),\n                Limit = vec![\n                    trc::Value::from(rate.requests),\n                    trc::Value::from(rate.period)\n                ],\n            );\n\n            return;\n        }\n\n        let config = &self.server.core.smtp.report.dkim;\n        let from_addr = self\n            .server\n            .eval_if(&config.address, self, self.data.session_id)\n            .await\n            .unwrap_or_else(|| \"MAILER-DAEMON@localhost\".to_string());\n        let mut report = Vec::with_capacity(128);\n        self.new_auth_failure(output.result().into(), rejected)\n            .with_authentication_results(\n                AuthenticationResults::new(&self.hostname)\n                    .with_dkim_result(output, message.from())\n                    .to_string(),\n            )\n            .with_dkim_domain(signature.domain())\n            .with_dkim_selector(signature.selector())\n            .with_dkim_identity(signature.identity())\n            .with_headers(std::str::from_utf8(message.raw_headers()).unwrap_or_default())\n            .write_rfc5322(\n                (\n                    self.server\n                        .eval_if(&config.name, self, self.data.session_id)\n                        .await\n                        .unwrap_or_else(|| \"Mail Delivery Subsystem\".to_string())\n                        .as_str(),\n                    from_addr.as_str(),\n                ),\n                rcpt,\n                &self\n                    .server\n                    .eval_if(&config.subject, self, self.data.session_id)\n                    .await\n                    .unwrap_or_else(|| \"DKIM Report\".to_string()),\n                &mut report,\n            )\n            .ok();\n\n        trc::event!(\n            OutgoingReport(OutgoingReportEvent::DkimReport),\n            SpanId = self.data.session_id,\n            From = from_addr.to_string(),\n            To = rcpt.to_string(),\n        );\n\n        // Send report\n        self.server\n            .send_report(\n                &from_addr,\n                [rcpt].into_iter(),\n                report,\n                &config.sign,\n                true,\n                self.data.session_id,\n            )\n            .await;\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/reporting/dmarc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{AggregateTimestamp, SerializedSize};\nuse crate::{core::Session, queue::RecipientDomain, reporting::SmtpReporting};\nuse ahash::AHashMap;\nuse common::{\n    Server,\n    config::smtp::report::AggregateFrequency,\n    ipc::{DmarcEvent, ToHash},\n    listener::SessionStream,\n};\nuse compact_str::ToCompactString;\nuse mail_auth::{\n    ArcOutput, AuthenticatedMessage, AuthenticationResults, DkimOutput, DkimResult, DmarcOutput,\n    SpfResult,\n    common::verify::VerifySignature,\n    dmarc::{self, URI},\n    report::{AuthFailureType, IdentityAlignment, PolicyPublished, Record, Report, SPFDomainScope},\n};\nuse std::{collections::hash_map::Entry, future::Future};\nuse store::{\n    Deserialize, IterateParams, Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, QueueClass, ReportEvent, ValueClass},\n};\nuse trc::{AddContext, OutgoingReportEvent};\nuse utils::{DomainPart, config::Rate};\n\n#[derive(\n    Debug,\n    PartialEq,\n    Eq,\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    serde::Serialize,\n    serde::Deserialize,\n)]\npub struct DmarcFormat {\n    pub rua: Vec<URI>,\n    pub policy: PolicyPublished,\n    pub records: Vec<Record>,\n}\n\nimpl<T: SessionStream> Session<T> {\n    #[allow(clippy::too_many_arguments)]\n    pub async fn send_dmarc_report(\n        &self,\n        message: &AuthenticatedMessage<'_>,\n        auth_results: &AuthenticationResults<'_>,\n        rejected: bool,\n        dmarc_output: DmarcOutput,\n        dkim_output: &[DkimOutput<'_>],\n        arc_output: &Option<ArcOutput<'_>>,\n    ) {\n        let dmarc_record = dmarc_output.dmarc_record_cloned().unwrap();\n        let config = &self.server.core.smtp.report.dmarc;\n\n        // Send failure report\n        if let (Some(failure_rate), Some(report_options)) = (\n            self.server\n                .eval_if::<Rate, _>(&config.send, self, self.data.session_id)\n                .await,\n            dmarc_output.failure_report(),\n        ) {\n            // Verify that any external reporting addresses are authorized\n            let rcpts = match self\n                .server\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .verify_dmarc_report_address(\n                    dmarc_output.domain(),\n                    dmarc_record.ruf(),\n                    Some(&self.server.inner.cache.dns_txt),\n                )\n                .await\n            {\n                Some(rcpts) => {\n                    if !rcpts.is_empty() {\n                        let mut new_rcpts = Vec::with_capacity(rcpts.len());\n\n                        for rcpt in rcpts {\n                            if self.throttle_rcpt(rcpt.uri(), &failure_rate, \"dmarc\").await {\n                                new_rcpts.push(rcpt.uri());\n                            }\n                        }\n\n                        new_rcpts\n                    } else {\n                        if !dmarc_record.ruf().is_empty() {\n                            trc::event!(\n                                OutgoingReport(OutgoingReportEvent::UnauthorizedReportingAddress),\n                                SpanId = self.data.session_id,\n                                Url = dmarc_record\n                                    .ruf()\n                                    .iter()\n                                    .map(|u| trc::Value::String(u.uri().to_compact_string()))\n                                    .collect::<Vec<_>>(),\n                            );\n                        }\n                        vec![]\n                    }\n                }\n                None => {\n                    trc::event!(\n                        OutgoingReport(OutgoingReportEvent::ReportingAddressValidationError),\n                        SpanId = self.data.session_id,\n                        Url = dmarc_record\n                            .ruf()\n                            .iter()\n                            .map(|u| trc::Value::String(u.uri().to_compact_string()))\n                            .collect::<Vec<_>>(),\n                    );\n\n                    vec![]\n                }\n            };\n\n            // Throttle recipient\n            if !rcpts.is_empty() {\n                let mut report = Vec::with_capacity(128);\n                let from_addr = self\n                    .server\n                    .eval_if(&config.address, self, self.data.session_id)\n                    .await\n                    .unwrap_or_else(|| \"MAILER-DAEMON@localhost\".to_compact_string());\n                let mut auth_failure = self\n                    .new_auth_failure(AuthFailureType::Dmarc, rejected)\n                    .with_authentication_results(auth_results.to_string())\n                    .with_headers(std::str::from_utf8(message.raw_headers()).unwrap_or_default());\n\n                // Report the first failed signature\n                let dkim_failed = if let (\n                    dmarc::Report::Dkim\n                    | dmarc::Report::DkimSpf\n                    | dmarc::Report::All\n                    | dmarc::Report::Any,\n                    Some(signature),\n                ) = (\n                    &report_options,\n                    dkim_output.iter().find_map(|o| {\n                        let s = o.signature()?;\n                        if !matches!(o.result(), DkimResult::Pass) {\n                            Some(s)\n                        } else {\n                            None\n                        }\n                    }),\n                ) {\n                    auth_failure = auth_failure\n                        .with_dkim_domain(signature.domain())\n                        .with_dkim_selector(signature.selector())\n                        .with_dkim_identity(signature.identity());\n                    true\n                } else {\n                    false\n                };\n\n                // Report SPF failure\n                let spf_failed = if let (\n                    dmarc::Report::Spf\n                    | dmarc::Report::DkimSpf\n                    | dmarc::Report::All\n                    | dmarc::Report::Any,\n                    Some(output),\n                ) = (\n                    &report_options,\n                    self.data\n                        .spf_ehlo\n                        .as_ref()\n                        .and_then(|s| {\n                            if s.result() != SpfResult::Pass {\n                                s.into()\n                            } else {\n                                None\n                            }\n                        })\n                        .or_else(|| {\n                            self.data.spf_mail_from.as_ref().and_then(|s| {\n                                if s.result() != SpfResult::Pass {\n                                    s.into()\n                                } else {\n                                    None\n                                }\n                            })\n                        }),\n                ) {\n                    auth_failure =\n                        auth_failure.with_spf_dns(format!(\"txt : {} : v=SPF1\", output.domain()));\n                    // TODO use DNS record\n                    true\n                } else {\n                    false\n                };\n\n                auth_failure\n                    .with_identity_alignment(if dkim_failed && spf_failed {\n                        IdentityAlignment::DkimSpf\n                    } else if dkim_failed {\n                        IdentityAlignment::Dkim\n                    } else {\n                        IdentityAlignment::Spf\n                    })\n                    .write_rfc5322(\n                        (\n                            self.server\n                                .eval_if(&config.name, self, self.data.session_id)\n                                .await\n                                .unwrap_or_else(|| \"Mail Delivery Subsystem\".to_compact_string())\n                                .as_str(),\n                            from_addr.as_str(),\n                        ),\n                        &rcpts.join(\", \"),\n                        &self\n                            .server\n                            .eval_if(&config.subject, self, self.data.session_id)\n                            .await\n                            .unwrap_or_else(|| \"DMARC Report\".to_compact_string()),\n                        &mut report,\n                    )\n                    .ok();\n\n                trc::event!(\n                    OutgoingReport(OutgoingReportEvent::DmarcReport),\n                    SpanId = self.data.session_id,\n                    From = from_addr.to_string(),\n                    To = rcpts\n                        .iter()\n                        .map(|a| trc::Value::String(a.to_compact_string()))\n                        .collect::<Vec<_>>(),\n                );\n\n                // Send report\n                self.server\n                    .send_report(\n                        &from_addr,\n                        rcpts.into_iter(),\n                        report,\n                        &config.sign,\n                        true,\n                        self.data.session_id,\n                    )\n                    .await;\n            } else {\n                trc::event!(\n                    OutgoingReport(OutgoingReportEvent::DmarcRateLimited),\n                    SpanId = self.data.session_id,\n                    Limit = vec![\n                        trc::Value::from(failure_rate.requests),\n                        trc::Value::from(failure_rate.period)\n                    ],\n                );\n            }\n        }\n\n        // Send aggregate reports\n        let interval = self\n            .server\n            .eval_if(\n                &self.server.core.smtp.report.dmarc_aggregate.send,\n                self,\n                self.data.session_id,\n            )\n            .await\n            .unwrap_or(AggregateFrequency::Never);\n\n        if matches!(interval, AggregateFrequency::Never) || dmarc_record.rua().is_empty() {\n            return;\n        }\n\n        // Create DMARC report record\n        let mut report_record = Record::new()\n            .with_dmarc_output(&dmarc_output)\n            .with_dkim_output(dkim_output)\n            .with_source_ip(self.data.remote_ip)\n            .with_header_from(message.from().domain_part())\n            .with_envelope_from(\n                self.data\n                    .mail_from\n                    .as_ref()\n                    .map(|mf| mf.domain.as_str())\n                    .unwrap_or_else(|| self.data.helo_domain.as_str()),\n            );\n        if let Some(spf_ehlo) = &self.data.spf_ehlo {\n            report_record = report_record.with_spf_output(spf_ehlo, SPFDomainScope::Helo);\n        }\n        if let Some(spf_mail_from) = &self.data.spf_mail_from {\n            report_record = report_record.with_spf_output(spf_mail_from, SPFDomainScope::MailFrom);\n        }\n        if let Some(arc_output) = arc_output {\n            report_record = report_record.with_arc_output(arc_output);\n        }\n\n        // Submit DMARC report event\n        self.server\n            .schedule_report(DmarcEvent {\n                domain: dmarc_output.into_domain(),\n                report_record,\n                dmarc_record,\n                interval,\n            })\n            .await;\n    }\n}\n\npub trait DmarcReporting: Sync + Send {\n    fn send_dmarc_aggregate_report(&self, event: ReportEvent) -> impl Future<Output = ()> + Send;\n    fn generate_dmarc_aggregate_report(\n        &self,\n        event: &ReportEvent,\n        rua: &mut Vec<URI>,\n        serialized_size: Option<&mut serde_json::Serializer<SerializedSize>>,\n        span_id: u64,\n    ) -> impl Future<Output = trc::Result<Option<Report>>> + Send;\n    fn delete_dmarc_report(&self, event: ReportEvent) -> impl Future<Output = ()> + Send;\n    fn schedule_dmarc(&self, event: Box<DmarcEvent>) -> impl Future<Output = ()> + Send;\n}\n\nimpl DmarcReporting for Server {\n    async fn send_dmarc_aggregate_report(&self, event: ReportEvent) {\n        let span_id = self.inner.data.span_id_gen.generate();\n\n        trc::event!(\n            OutgoingReport(OutgoingReportEvent::DmarcAggregateReport),\n            SpanId = span_id,\n            ReportId = event.seq_id,\n            Domain = event.domain.clone(),\n            RangeFrom = trc::Value::Timestamp(event.seq_id),\n            RangeTo = trc::Value::Timestamp(event.due),\n        );\n\n        // Generate report\n        let mut serialized_size = serde_json::Serializer::new(SerializedSize::new(\n            self.eval_if(\n                &self.core.smtp.report.dmarc_aggregate.max_size,\n                &RecipientDomain::new(event.domain.as_str()),\n                span_id,\n            )\n            .await\n            .unwrap_or(25 * 1024 * 1024),\n        ));\n        let mut rua = Vec::new();\n        let report = match self\n            .generate_dmarc_aggregate_report(&event, &mut rua, Some(&mut serialized_size), span_id)\n            .await\n        {\n            Ok(Some(report)) => report,\n            Ok(None) => {\n                trc::event!(\n                    OutgoingReport(OutgoingReportEvent::NotFound),\n                    SpanId = span_id,\n                    CausedBy = trc::location!()\n                );\n\n                return;\n            }\n            Err(err) => {\n                trc::error!(err.span_id(span_id).details(\"Failed to read DMARC report\"));\n                return;\n            }\n        };\n\n        // Verify external reporting addresses\n        let rua = match self\n            .core\n            .smtp\n            .resolvers\n            .dns\n            .verify_dmarc_report_address(&event.domain, &rua, Some(&self.inner.cache.dns_txt))\n            .await\n        {\n            Some(rcpts) => {\n                if !rcpts.is_empty() {\n                    rcpts\n                        .into_iter()\n                        .map(|u| u.uri().to_string())\n                        .collect::<Vec<_>>()\n                } else {\n                    trc::event!(\n                        OutgoingReport(OutgoingReportEvent::UnauthorizedReportingAddress),\n                        SpanId = span_id,\n                        Url = rua\n                            .iter()\n                            .map(|u| trc::Value::String(u.uri().to_compact_string()))\n                            .collect::<Vec<_>>(),\n                    );\n\n                    self.delete_dmarc_report(event).await;\n                    return;\n                }\n            }\n            None => {\n                trc::event!(\n                    OutgoingReport(OutgoingReportEvent::ReportingAddressValidationError),\n                    SpanId = span_id,\n                    Url = rua\n                        .iter()\n                        .map(|u| trc::Value::String(u.uri().to_compact_string()))\n                        .collect::<Vec<_>>(),\n                );\n\n                self.delete_dmarc_report(event).await;\n                return;\n            }\n        };\n\n        // Serialize report\n        let config = &self.core.smtp.report.dmarc_aggregate;\n        let from_addr = self\n            .eval_if(\n                &config.address,\n                &RecipientDomain::new(event.domain.as_str()),\n                span_id,\n            )\n            .await\n            .unwrap_or_else(|| \"MAILER-DAEMON@localhost\".to_compact_string());\n        let mut message = Vec::with_capacity(2048);\n        let _ = report.write_rfc5322(\n            &self\n                .eval_if(\n                    &self.core.smtp.report.submitter,\n                    &RecipientDomain::new(event.domain.as_str()),\n                    span_id,\n                )\n                .await\n                .unwrap_or_else(|| \"localhost\".to_compact_string()),\n            (\n                self.eval_if(\n                    &config.name,\n                    &RecipientDomain::new(event.domain.as_str()),\n                    span_id,\n                )\n                .await\n                .unwrap_or_else(|| \"Mail Delivery Subsystem\".to_compact_string())\n                .as_str(),\n                from_addr.as_str(),\n            ),\n            rua.iter().map(|a| a.as_str()),\n            &mut message,\n        );\n\n        // Send report\n        self.send_report(\n            &from_addr,\n            rua.iter(),\n            message,\n            &config.sign,\n            false,\n            event.seq_id,\n        )\n        .await;\n\n        self.delete_dmarc_report(event).await;\n    }\n\n    async fn generate_dmarc_aggregate_report(\n        &self,\n        event: &ReportEvent,\n        rua: &mut Vec<URI>,\n        mut serialized_size: Option<&mut serde_json::Serializer<SerializedSize>>,\n        span_id: u64,\n    ) -> trc::Result<Option<Report>> {\n        // Deserialize report\n        let dmarc = match self\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Queue(\n                QueueClass::DmarcReportHeader(event.clone()),\n            )))\n            .await?\n        {\n            Some(dmarc) => dmarc.deserialize::<DmarcFormat>()?,\n            None => {\n                return Ok(None);\n            }\n        };\n        let _ = std::mem::replace(rua, dmarc.rua);\n\n        // Create report\n        let config = &self.core.smtp.report.dmarc_aggregate;\n        let mut report = Report::new()\n            .with_policy_published(dmarc.policy)\n            .with_date_range_begin(event.seq_id)\n            .with_date_range_end(event.due)\n            .with_report_id(format!(\"{}_{}\", event.policy_hash, event.seq_id))\n            .with_email(\n                self.eval_if(\n                    &config.address,\n                    &RecipientDomain::new(event.domain.as_str()),\n                    span_id,\n                )\n                .await\n                .unwrap_or_else(|| \"MAILER-DAEMON@localhost\".to_compact_string()),\n            );\n        if let Some(org_name) = self\n            .eval_if::<String, _>(\n                &config.org_name,\n                &RecipientDomain::new(event.domain.as_str()),\n                span_id,\n            )\n            .await\n        {\n            report = report.with_org_name(org_name);\n        }\n        if let Some(contact_info) = self\n            .eval_if::<String, _>(\n                &config.contact_info,\n                &RecipientDomain::new(event.domain.as_str()),\n                span_id,\n            )\n            .await\n        {\n            report = report.with_extra_contact_info(contact_info);\n        }\n\n        if let Some(serialized_size) = serialized_size.as_deref_mut() {\n            let _ = serde::Serialize::serialize(&report, serialized_size);\n        }\n\n        // Group duplicates\n        let from_key = ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportEvent(\n            ReportEvent {\n                due: event.due,\n                policy_hash: event.policy_hash,\n                seq_id: 0,\n                domain: event.domain.clone(),\n            },\n        )));\n        let to_key = ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportEvent(\n            ReportEvent {\n                due: event.due,\n                policy_hash: event.policy_hash,\n                seq_id: u64::MAX,\n                domain: event.domain.clone(),\n            },\n        )));\n        let mut record_map = AHashMap::with_capacity(dmarc.records.len());\n        self.core\n            .storage\n            .data\n            .iterate(IterateParams::new(from_key, to_key).ascending(), |_, v| {\n                let archive = <Archive<AlignedBytes> as Deserialize>::deserialize(v)?;\n\n                match record_map.entry(archive.deserialize::<Record>()?) {\n                    Entry::Occupied(mut e) => {\n                        *e.get_mut() += 1;\n                        Ok(true)\n                    }\n                    Entry::Vacant(e) => {\n                        if serialized_size\n                            .as_deref_mut()\n                            .is_none_or(|serialized_size| {\n                                serde::Serialize::serialize(e.key(), serialized_size).is_ok()\n                            })\n                        {\n                            e.insert(1u32);\n                            Ok(true)\n                        } else {\n                            Ok(false)\n                        }\n                    }\n                }\n            })\n            .await\n            .caused_by(trc::location!())?;\n\n        for (record, count) in record_map {\n            report = report.with_record(record.with_count(count));\n        }\n\n        Ok(Some(report))\n    }\n\n    async fn delete_dmarc_report(&self, event: ReportEvent) {\n        let from_key = ReportEvent {\n            due: event.due,\n            policy_hash: event.policy_hash,\n            seq_id: 0,\n            domain: event.domain.clone(),\n        };\n        let to_key = ReportEvent {\n            due: event.due,\n            policy_hash: event.policy_hash,\n            seq_id: u64::MAX,\n            domain: event.domain.clone(),\n        };\n\n        if let Err(err) = self\n            .core\n            .storage\n            .data\n            .delete_range(\n                ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportEvent(from_key))),\n                ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportEvent(to_key))),\n            )\n            .await\n        {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to delete DMARC report\")\n            );\n            return;\n        }\n\n        let mut batch = BatchBuilder::new();\n        batch.clear(ValueClass::Queue(QueueClass::DmarcReportHeader(event)));\n        if let Err(err) = self.core.storage.data.write(batch.build_all()).await {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to delete DMARC report\")\n            );\n        }\n    }\n\n    async fn schedule_dmarc(&self, event: Box<DmarcEvent>) {\n        let created = event.interval.to_timestamp();\n        let deliver_at = created + event.interval.as_secs();\n        let mut report_event = ReportEvent {\n            due: deliver_at,\n            policy_hash: event.dmarc_record.to_hash(),\n            seq_id: created,\n            domain: event.domain,\n        };\n\n        // Write policy if missing\n        let mut builder = BatchBuilder::new();\n        if self\n            .core\n            .storage\n            .data\n            .get_value::<()>(ValueKey::from(ValueClass::Queue(\n                QueueClass::DmarcReportHeader(report_event.clone()),\n            )))\n            .await\n            .unwrap_or_default()\n            .is_none()\n        {\n            // Serialize report\n            let entry = DmarcFormat {\n                rua: event.dmarc_record.rua().to_vec(),\n                policy: PolicyPublished::from_record(\n                    report_event.domain.to_string(),\n                    &event.dmarc_record,\n                ),\n                records: vec![],\n            };\n\n            // Write report\n            builder.set(\n                ValueClass::Queue(QueueClass::DmarcReportHeader(report_event.clone())),\n                match Archiver::new(entry).serialize() {\n                    Ok(data) => data.to_vec(),\n                    Err(err) => {\n                        trc::error!(\n                            err.caused_by(trc::location!())\n                                .details(\"Failed to serialize DMARC report\")\n                        );\n                        return;\n                    }\n                },\n            );\n        }\n\n        // Write entry\n        report_event.seq_id = self.inner.data.queue_id_gen.generate();\n        builder.set(\n            ValueClass::Queue(QueueClass::DmarcReportEvent(report_event)),\n            match Archiver::new(event.report_record).serialize() {\n                Ok(data) => data.to_vec(),\n                Err(err) => {\n                    trc::error!(\n                        err.caused_by(trc::location!())\n                            .details(\"Failed to serialize DMARC report\")\n                    );\n                    return;\n                }\n            },\n        );\n\n        if let Err(err) = self.core.storage.data.write(builder.build_all()).await {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to write DMARC report\")\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/reporting/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    core::Session,\n    inbound::DkimSign,\n    queue::{MessageSource, MessageWrapper, spool::SmtpSpool},\n};\nuse common::{\n    Server, USER_AGENT,\n    config::smtp::report::{AddressMatch, AggregateFrequency},\n    expr::if_block::IfBlock,\n    ipc::ReportingEvent,\n};\nuse mail_auth::{\n    common::headers::HeaderWriter,\n    report::{AuthFailureType, DeliveryResult, Feedback, FeedbackType},\n};\nuse mail_parser::DateTime;\nuse std::{future::Future, io, time::SystemTime};\nuse store::write::{ReportEvent, key::KeySerializer};\nuse tokio::io::{AsyncRead, AsyncWrite};\n\npub mod analysis;\npub mod dkim;\npub mod dmarc;\npub mod scheduler;\npub mod spf;\npub mod tls;\n\nimpl<T: AsyncWrite + AsyncRead + Unpin> Session<T> {\n    pub fn new_auth_failure(&self, ft: AuthFailureType, rejected: bool) -> Feedback<'_> {\n        Feedback::new(FeedbackType::AuthFailure)\n            .with_auth_failure(ft)\n            .with_arrival_date(\n                SystemTime::now()\n                    .duration_since(SystemTime::UNIX_EPOCH)\n                    .map_or(0, |d| d.as_secs()) as i64,\n            )\n            .with_source_ip(self.data.remote_ip)\n            .with_reporting_mta(&self.hostname)\n            .with_user_agent(USER_AGENT)\n            .with_delivery_result(if rejected {\n                DeliveryResult::Reject\n            } else {\n                DeliveryResult::Unspecified\n            })\n    }\n\n    pub fn is_report(&self) -> bool {\n        for addr_match in &self.server.core.smtp.report.analysis.addresses {\n            for addr in &self.data.rcpt_to {\n                match addr_match {\n                    AddressMatch::StartsWith(prefix) if addr.address_lcase.starts_with(prefix) => {\n                        return true;\n                    }\n                    AddressMatch::EndsWith(suffix) if addr.address_lcase.ends_with(suffix) => {\n                        return true;\n                    }\n                    AddressMatch::Equals(value) if addr.address_lcase.eq(value) => return true,\n                    _ => (),\n                }\n            }\n        }\n\n        false\n    }\n}\n\npub trait SmtpReporting: Sync + Send {\n    fn send_report(\n        &self,\n        from_addr: &str,\n        rcpts: impl Iterator<Item = impl AsRef<str> + Sync + Send> + Sync + Send,\n        report: Vec<u8>,\n        sign_config: &IfBlock,\n        deliver_now: bool,\n        parent_session_id: u64,\n    ) -> impl Future<Output = ()> + Send;\n\n    fn send_autogenerated(\n        &self,\n        from_addr: impl AsRef<str> + Sync + Send,\n        rcpts: impl Iterator<Item = impl AsRef<str> + Sync + Send> + Sync + Send,\n        raw_message: Vec<u8>,\n        sign_config: Option<&IfBlock>,\n        parent_session_id: u64,\n    ) -> impl Future<Output = ()> + Send;\n\n    fn schedule_report(\n        &self,\n        report: impl Into<ReportingEvent> + Sync + Send,\n    ) -> impl Future<Output = ()> + Send;\n\n    fn sign_message(\n        &self,\n        message: &mut MessageWrapper,\n        config: &IfBlock,\n        bytes: &[u8],\n    ) -> impl Future<Output = Option<Vec<u8>>> + Send;\n}\n\nimpl SmtpReporting for Server {\n    async fn send_report(\n        &self,\n        from_addr: &str,\n        rcpts: impl Iterator<Item = impl AsRef<str> + Sync + Send> + Sync + Send,\n        report: Vec<u8>,\n        sign_config: &IfBlock,\n        deliver_now: bool,\n        parent_session_id: u64,\n    ) {\n        // Build message\n        let mut message = self.new_message(from_addr, parent_session_id);\n        for rcpt_ in rcpts {\n            message.add_recipient(rcpt_.as_ref(), self).await;\n        }\n\n        // Sign message\n        let signature = self.sign_message(&mut message, sign_config, &report).await;\n\n        // Schedule delivery at a random time between now and the next 3 hours\n        if !deliver_now {\n            #[cfg(not(feature = \"test_mode\"))]\n            {\n                use common::config::smtp::queue::QueueExpiry;\n                use rand::Rng;\n\n                let delivery_time = rand::rng().random_range(0u64..10800u64);\n                for rcpt in &mut message.message.recipients {\n                    rcpt.retry.due += delivery_time;\n                    rcpt.notify.due += delivery_time;\n                    if let QueueExpiry::Ttl(expires) = &mut rcpt.expires {\n                        *expires += delivery_time;\n                    }\n                }\n            }\n        }\n\n        // Queue message\n        message\n            .queue(\n                signature.as_deref(),\n                &report,\n                parent_session_id,\n                self,\n                MessageSource::Report,\n            )\n            .await;\n    }\n\n    async fn send_autogenerated(\n        &self,\n        from_addr: impl AsRef<str> + Sync + Send,\n        rcpts: impl Iterator<Item = impl AsRef<str> + Sync + Send> + Sync + Send,\n        raw_message: Vec<u8>,\n        sign_config: Option<&IfBlock>,\n        parent_session_id: u64,\n    ) {\n        // Build message\n        let mut message = self.new_message(from_addr.as_ref(), parent_session_id);\n        for rcpt in rcpts {\n            message.add_recipient(rcpt, self).await;\n        }\n\n        // Sign message\n        let signature = if let Some(sign_config) = sign_config {\n            self.sign_message(&mut message, sign_config, &raw_message)\n                .await\n        } else {\n            None\n        };\n\n        // Queue message\n        message\n            .queue(\n                signature.as_deref(),\n                &raw_message,\n                parent_session_id,\n                self,\n                MessageSource::Autogenerated,\n            )\n            .await;\n    }\n\n    async fn schedule_report(&self, report: impl Into<ReportingEvent> + Sync + Send) {\n        if self.inner.ipc.report_tx.send(report.into()).await.is_err() {\n            trc::event!(\n                Server(trc::ServerEvent::ThreadError),\n                CausedBy = trc::location!(),\n                Details = \"Failed to send event to ReportScheduler\"\n            );\n        }\n    }\n\n    async fn sign_message(\n        &self,\n        message: &mut MessageWrapper,\n        config: &IfBlock,\n        bytes: &[u8],\n    ) -> Option<Vec<u8>> {\n        let signers = self\n            .eval_if::<Vec<String>, _>(config, &message.message, message.span_id)\n            .await\n            .unwrap_or_default();\n        if !signers.is_empty() {\n            let mut headers = Vec::with_capacity(64);\n            for signer in signers.iter() {\n                if let Some(signer) = self.get_dkim_signer(signer, message.span_id) {\n                    match signer.sign(bytes) {\n                        Ok(signature) => {\n                            signature.write_header(&mut headers);\n                        }\n                        Err(err) => {\n                            trc::error!(\n                                trc::Error::from(err)\n                                    .span_id(message.span_id)\n                                    .details(\"Failed to sign message\")\n                                    .caused_by(trc::location!())\n                            );\n                        }\n                    }\n                }\n            }\n            if !headers.is_empty() {\n                return Some(headers);\n            }\n        }\n        None\n    }\n}\n\npub trait AggregateTimestamp {\n    fn to_timestamp(&self) -> u64;\n    fn to_timestamp_(&self, dt: DateTime) -> u64;\n    fn as_secs(&self) -> u64;\n    fn due(&self) -> u64;\n}\n\nimpl AggregateTimestamp for AggregateFrequency {\n    fn to_timestamp(&self) -> u64 {\n        self.to_timestamp_(DateTime::from_timestamp(\n            SystemTime::now()\n                .duration_since(SystemTime::UNIX_EPOCH)\n                .map_or(0, |d| d.as_secs()) as i64,\n        ))\n    }\n\n    fn to_timestamp_(&self, mut dt: DateTime) -> u64 {\n        (match self {\n            AggregateFrequency::Hourly => {\n                dt.minute = 0;\n                dt.second = 0;\n                dt.to_timestamp()\n            }\n            AggregateFrequency::Daily => {\n                dt.hour = 0;\n                dt.minute = 0;\n                dt.second = 0;\n                dt.to_timestamp()\n            }\n            AggregateFrequency::Weekly => {\n                let dow = dt.day_of_week();\n                dt.hour = 0;\n                dt.minute = 0;\n                dt.second = 0;\n                dt.to_timestamp() - (86400 * dow as i64)\n            }\n            AggregateFrequency::Never => dt.to_timestamp(),\n        }) as u64\n    }\n\n    fn as_secs(&self) -> u64 {\n        match self {\n            AggregateFrequency::Hourly => 3600,\n            AggregateFrequency::Daily => 86400,\n            AggregateFrequency::Weekly => 7 * 86400,\n            AggregateFrequency::Never => 0,\n        }\n    }\n\n    fn due(&self) -> u64 {\n        self.to_timestamp() + self.as_secs()\n    }\n}\n\npub struct SerializedSize {\n    bytes_left: usize,\n}\n\nimpl SerializedSize {\n    pub fn new(bytes_left: usize) -> Self {\n        Self { bytes_left }\n    }\n}\n\nimpl io::Write for SerializedSize {\n    fn write(&mut self, buf: &[u8]) -> io::Result<usize> {\n        //let c = print!(\" (left: {}, buf: {})\", self.bytes_left, buf.len());\n        let buf_len = buf.len();\n        if buf_len <= self.bytes_left {\n            self.bytes_left -= buf_len;\n            Ok(buf_len)\n        } else {\n            Err(io::Error::other(\"Size exceeded\"))\n        }\n    }\n\n    fn flush(&mut self) -> io::Result<()> {\n        Ok(())\n    }\n}\n\npub trait ReportLock {\n    fn tls_lock(&self) -> Vec<u8>;\n    fn dmarc_lock(&self) -> Vec<u8>;\n}\n\nimpl ReportLock for ReportEvent {\n    fn tls_lock(&self) -> Vec<u8> {\n        KeySerializer::new(self.domain.len() + std::mem::size_of::<u64>() + 1)\n            .write(0u8)\n            .write(self.due)\n            .write(self.domain.as_bytes())\n            .finalize()\n    }\n\n    fn dmarc_lock(&self) -> Vec<u8> {\n        KeySerializer::new(self.domain.len() + (std::mem::size_of::<u64>() * 2) + 1)\n            .write(1u8)\n            .write(self.due)\n            .write(self.policy_hash)\n            .write(self.domain.as_bytes())\n            .finalize()\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/reporting/scheduler.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashMap;\nuse common::{Inner, KV_LOCK_QUEUE_REPORT, Server, core::BuildServer, ipc::ReportingEvent};\n\nuse std::{\n    future::Future,\n    sync::Arc,\n    time::{Duration, SystemTime},\n};\nuse store::{\n    Deserialize, IterateParams, Store, ValueKey,\n    write::{BatchBuilder, QueueClass, ReportEvent, ValueClass, now},\n};\nuse tokio::sync::mpsc;\n\nuse crate::queue::spool::LOCK_EXPIRY;\n\nuse super::{AggregateTimestamp, ReportLock, dmarc::DmarcReporting, tls::TlsReporting};\n\npub const REPORT_REFRESH: Duration = Duration::from_secs(86400);\n\nimpl SpawnReport for mpsc::Receiver<ReportingEvent> {\n    fn spawn(mut self, inner: Arc<Inner>) {\n        tokio::spawn(async move {\n            let mut next_wake_up = REPORT_REFRESH;\n            let mut refresh_queue = true;\n\n            loop {\n                let server = inner.build_server();\n\n                if refresh_queue {\n                    // Read events\n                    let events = next_report_event(server.store()).await;\n                    let now = now();\n                    next_wake_up = events\n                        .last()\n                        .and_then(|e| {\n                            e.due()\n                                .filter(|due| *due > now)\n                                .map(|due| Duration::from_secs(due - now))\n                        })\n                        .unwrap_or(REPORT_REFRESH);\n\n                    if events\n                        .first()\n                        .and_then(|e| e.due())\n                        .is_some_and(|due| due <= now)\n                    {\n                        let server_ = server.clone();\n                        tokio::spawn(async move {\n                            let mut tls_reports = AHashMap::new();\n                            for report_event in events {\n                                match report_event {\n                                    QueueClass::DmarcReportHeader(event) if event.due <= now => {\n                                        let lock_name = event.dmarc_lock();\n                                        if server_.try_lock_report(&lock_name).await {\n                                            server_.send_dmarc_aggregate_report(event).await;\n                                            server_.unlock_report(&lock_name).await;\n                                        }\n                                    }\n                                    QueueClass::TlsReportHeader(event) if event.due <= now => {\n                                        tls_reports\n                                            .entry(event.domain.clone())\n                                            .or_insert_with(Vec::new)\n                                            .push(event);\n                                    }\n                                    _ => (),\n                                }\n                            }\n\n                            for (_, tls_report) in tls_reports {\n                                let lock_name = tls_report.first().unwrap().tls_lock();\n                                if server_.try_lock_report(&lock_name).await {\n                                    server_.send_tls_aggregate_report(tls_report).await;\n                                    server_.unlock_report(&lock_name).await;\n                                }\n                            }\n                        });\n                    }\n                }\n\n                match tokio::time::timeout(next_wake_up, self.recv()).await {\n                    Ok(Some(event)) => {\n                        refresh_queue = false;\n\n                        match event {\n                            ReportingEvent::Dmarc(event) => {\n                                next_wake_up = std::cmp::min(\n                                    next_wake_up,\n                                    Duration::from_secs(event.interval.due().saturating_sub(now())),\n                                );\n                                server.schedule_dmarc(event).await;\n                            }\n                            ReportingEvent::Tls(event) => {\n                                next_wake_up = std::cmp::min(\n                                    next_wake_up,\n                                    Duration::from_secs(event.interval.due().saturating_sub(now())),\n                                );\n                                server.schedule_tls(event).await;\n                            }\n                            ReportingEvent::Stop => break,\n                        }\n                    }\n                    Ok(None) => break,\n                    Err(_) => {\n                        refresh_queue = true;\n                    }\n                }\n            }\n        });\n    }\n}\n\nasync fn next_report_event(store: &Store) -> Vec<QueueClass> {\n    let now = now();\n    let from_key = ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportHeader(\n        ReportEvent {\n            due: 0,\n            policy_hash: 0,\n            seq_id: 0,\n            domain: String::new(),\n        },\n    )));\n    let to_key = ValueKey::from(ValueClass::Queue(QueueClass::TlsReportHeader(\n        ReportEvent {\n            due: now + REPORT_REFRESH.as_secs(),\n            policy_hash: 0,\n            seq_id: 0,\n            domain: String::new(),\n        },\n    )));\n\n    let mut events = Vec::new();\n    let mut old_locks = Vec::new();\n    let result = store\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending().no_values(),\n            |key, _| {\n                let event = ReportEvent::deserialize(key)?;\n\n                // TODO - REMOVEME - Part of v0.11 migration\n                if event.seq_id == 0 {\n                    old_locks.push(if *key.last().unwrap() == 0 {\n                        QueueClass::DmarcReportHeader(event)\n                    } else {\n                        QueueClass::TlsReportHeader(event)\n                    });\n                    return Ok(true);\n                }\n\n                let do_continue = event.due <= now;\n                events.push(if *key.last().unwrap() == 0 {\n                    QueueClass::DmarcReportHeader(event)\n                } else {\n                    QueueClass::TlsReportHeader(event)\n                });\n                Ok(do_continue)\n            },\n        )\n        .await;\n\n    // TODO - REMOVEME - Part of v0.11 migration\n    if !old_locks.is_empty() {\n        let mut batch = BatchBuilder::new();\n        for event in old_locks {\n            batch.clear(ValueClass::Queue(event));\n        }\n        if let Err(err) = store.write(batch.build_all()).await {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to remove old report events\")\n            );\n        }\n    }\n\n    if let Err(err) = result {\n        trc::error!(\n            err.caused_by(trc::location!())\n                .details(\"Failed to read from store\")\n        );\n    }\n\n    events\n}\n\npub trait LockReport: Sync + Send {\n    fn try_lock_report(&self, lock: &[u8]) -> impl Future<Output = bool> + Send;\n\n    fn unlock_report(&self, lock: &[u8]) -> impl Future<Output = ()> + Send;\n}\n\nimpl LockReport for Server {\n    async fn try_lock_report(&self, key: &[u8]) -> bool {\n        match self\n            .in_memory_store()\n            .try_lock(KV_LOCK_QUEUE_REPORT, key, LOCK_EXPIRY)\n            .await\n        {\n            Ok(result) => {\n                if !result {\n                    trc::event!(\n                        OutgoingReport(trc::OutgoingReportEvent::Locked),\n                        Expires = trc::Value::Timestamp(now() + LOCK_EXPIRY),\n                        Key = key\n                    );\n                }\n                result\n            }\n            Err(err) => {\n                trc::error!(\n                    err.details(\"Failed to lock report.\")\n                        .caused_by(trc::location!())\n                );\n                false\n            }\n        }\n    }\n\n    async fn unlock_report(&self, key: &[u8]) {\n        if let Err(err) = self\n            .in_memory_store()\n            .remove_lock(KV_LOCK_QUEUE_REPORT, key)\n            .await\n        {\n            trc::error!(\n                err.details(\"Failed to unlock event.\")\n                    .caused_by(trc::location!())\n            );\n        }\n    }\n}\n\npub trait ToTimestamp {\n    fn to_timestamp(&self) -> u64;\n}\n\nimpl ToTimestamp for Duration {\n    fn to_timestamp(&self) -> u64 {\n        SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .map_or(0, |d| d.as_secs())\n            + self.as_secs()\n    }\n}\n\npub trait SpawnReport {\n    fn spawn(self, core: Arc<Inner>);\n}\n"
  },
  {
    "path": "crates/smtp/src/reporting/spf.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::listener::SessionStream;\n\nuse mail_auth::{AuthenticationResults, SpfOutput, report::AuthFailureType};\nuse trc::OutgoingReportEvent;\nuse utils::config::Rate;\n\nuse crate::{core::Session, reporting::SmtpReporting};\n\nimpl<T: SessionStream> Session<T> {\n    pub async fn send_spf_report(\n        &self,\n        rcpt: &str,\n        rate: &Rate,\n        rejected: bool,\n        output: &SpfOutput,\n    ) {\n        // Throttle recipient\n        if !self.throttle_rcpt(rcpt, rate, \"spf\").await {\n            trc::event!(\n                OutgoingReport(OutgoingReportEvent::SpfRateLimited),\n                SpanId = self.data.session_id,\n                To = rcpt.to_string(),\n                Limit = vec![\n                    trc::Value::from(rate.requests),\n                    trc::Value::from(rate.period)\n                ],\n            );\n\n            return;\n        }\n\n        // Generate report\n        let config = &self.server.core.smtp.report.spf;\n        let from_addr = self\n            .server\n            .eval_if(&config.address, self, self.data.session_id)\n            .await\n            .unwrap_or_else(|| \"MAILER-DAEMON@localhost\".to_string());\n        let mut report = Vec::with_capacity(128);\n        self.new_auth_failure(AuthFailureType::Spf, rejected)\n            .with_authentication_results(\n                if let Some(mail_from) = &self.data.mail_from {\n                    AuthenticationResults::new(&self.hostname).with_spf_mailfrom_result(\n                        output,\n                        self.data.remote_ip,\n                        &mail_from.address,\n                        &self.data.helo_domain,\n                    )\n                } else {\n                    AuthenticationResults::new(&self.hostname).with_spf_ehlo_result(\n                        output,\n                        self.data.remote_ip,\n                        &self.data.helo_domain,\n                    )\n                }\n                .to_string(),\n            )\n            .with_spf_dns(format!(\"txt : {} : v=SPF1\", output.domain())) // TODO use DNS record\n            .write_rfc5322(\n                (\n                    self.server\n                        .eval_if(&config.name, self, self.data.session_id)\n                        .await\n                        .unwrap_or_else(|| \"Mailer Daemon\".to_string())\n                        .as_str(),\n                    from_addr.as_str(),\n                ),\n                rcpt,\n                &self\n                    .server\n                    .eval_if(&config.subject, self, self.data.session_id)\n                    .await\n                    .unwrap_or_else(|| \"SPF Report\".to_string()),\n                &mut report,\n            )\n            .ok();\n\n        trc::event!(\n            OutgoingReport(OutgoingReportEvent::SpfReport),\n            SpanId = self.data.session_id,\n            To = rcpt.to_string(),\n            From = from_addr.to_string(),\n        );\n\n        // Send report\n        self.server\n            .send_report(\n                &from_addr,\n                [rcpt].into_iter(),\n                report,\n                &config.sign,\n                true,\n                self.data.session_id,\n            )\n            .await;\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/reporting/tls.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{AggregateTimestamp, SerializedSize};\nuse crate::{queue::RecipientDomain, reporting::SmtpReporting};\nuse ahash::AHashMap;\nuse common::{\n    Server, USER_AGENT,\n    config::smtp::{\n        report::AggregateFrequency,\n        resolver::{Mode, MxPattern},\n    },\n    ipc::{TlsEvent, ToHash},\n};\nuse mail_auth::{\n    flate2::{Compression, write::GzEncoder},\n    mta_sts::{ReportUri, TlsRpt},\n    report::tlsrpt::{\n        DateRange, FailureDetails, Policy, PolicyDetails, PolicyType, Summary, TlsReport,\n    },\n};\nuse mail_parser::DateTime;\nuse reqwest::header::CONTENT_TYPE;\nuse std::fmt::Write;\nuse std::{collections::hash_map::Entry, future::Future, sync::Arc, time::Duration};\nuse store::{\n    Deserialize, IterateParams, Serialize, ValueKey,\n    write::{AlignedBytes, Archive, Archiver, BatchBuilder, QueueClass, ReportEvent, ValueClass},\n};\nuse trc::{AddContext, OutgoingReportEvent};\n\n#[derive(Debug, Clone)]\npub struct TlsRptOptions {\n    pub record: Arc<TlsRpt>,\n    pub interval: AggregateFrequency,\n}\n\n#[derive(Debug, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive, serde::Serialize)]\npub struct TlsFormat {\n    pub rua: Vec<ReportUri>,\n    pub policy: PolicyDetails,\n    pub records: Vec<Option<FailureDetails>>,\n}\n\n#[cfg(feature = \"test_mode\")]\npub static TLS_HTTP_REPORT: parking_lot::Mutex<Vec<u8>> = parking_lot::Mutex::new(Vec::new());\n\npub trait TlsReporting: Sync + Send {\n    fn send_tls_aggregate_report(\n        &self,\n        events: Vec<ReportEvent>,\n    ) -> impl Future<Output = ()> + Send;\n    fn generate_tls_aggregate_report(\n        &self,\n        events: &[ReportEvent],\n        rua: &mut Vec<ReportUri>,\n        serialized_size: Option<&mut serde_json::Serializer<SerializedSize>>,\n        span_id: u64,\n    ) -> impl Future<Output = trc::Result<Option<TlsReport>>> + Send;\n    fn schedule_tls(&self, event: Box<TlsEvent>) -> impl Future<Output = ()> + Send;\n    fn delete_tls_report(&self, events: Vec<ReportEvent>) -> impl Future<Output = ()> + Send;\n}\n\nimpl TlsReporting for Server {\n    async fn send_tls_aggregate_report(&self, events: Vec<ReportEvent>) {\n        let (domain_name, event_from, event_to) = events\n            .first()\n            .map(|e| (e.domain.as_str(), e.seq_id, e.due))\n            .unwrap();\n\n        let span_id = self.inner.data.span_id_gen.generate();\n\n        trc::event!(\n            OutgoingReport(OutgoingReportEvent::TlsAggregate),\n            SpanId = span_id,\n            ReportId = event_from,\n            Domain = domain_name.to_string(),\n            RangeFrom = trc::Value::Timestamp(event_from),\n            RangeTo = trc::Value::Timestamp(event_to),\n        );\n\n        // Generate report\n        let mut rua = Vec::new();\n        let mut serialized_size = serde_json::Serializer::new(SerializedSize::new(\n            self.eval_if(\n                &self.core.smtp.report.tls.max_size,\n                &RecipientDomain::new(domain_name),\n                span_id,\n            )\n            .await\n            .unwrap_or(25 * 1024 * 1024),\n        ));\n        let report = match self\n            .generate_tls_aggregate_report(&events, &mut rua, Some(&mut serialized_size), span_id)\n            .await\n        {\n            Ok(Some(report)) => report,\n            Ok(None) => {\n                // This should not happen\n                trc::event!(\n                    OutgoingReport(OutgoingReportEvent::NotFound),\n                    SpanId = span_id,\n                    CausedBy = trc::location!()\n                );\n                self.delete_tls_report(events).await;\n                return;\n            }\n            Err(err) => {\n                trc::error!(\n                    err.span_id(span_id)\n                        .caused_by(trc::location!())\n                        .details(\"Failed to read TLS report\")\n                );\n                return;\n            }\n        };\n\n        // Compress and serialize report\n        let json = report.to_json();\n        let mut e = GzEncoder::new(Vec::with_capacity(json.len()), Compression::default());\n        let json = match std::io::Write::write_all(&mut e, json.as_bytes()).and_then(|_| e.finish())\n        {\n            Ok(report) => report,\n            Err(err) => {\n                trc::event!(\n                    OutgoingReport(OutgoingReportEvent::SubmissionError),\n                    SpanId = span_id,\n                    Reason = err.to_string(),\n                    Details = \"Failed to compress report\"\n                );\n\n                self.delete_tls_report(events).await;\n                return;\n            }\n        };\n\n        // Try delivering report over HTTP\n        let mut rcpts = Vec::with_capacity(rua.len());\n        for uri in &rua {\n            match uri {\n                ReportUri::Http(uri) => {\n                    if let Ok(client) = reqwest::Client::builder()\n                        .user_agent(USER_AGENT)\n                        .timeout(Duration::from_secs(2 * 60))\n                        .build()\n                    {\n                        #[cfg(feature = \"test_mode\")]\n                        if uri == \"https://127.0.0.1/tls\" {\n                            TLS_HTTP_REPORT.lock().extend_from_slice(&json);\n                            self.delete_tls_report(events).await;\n                            return;\n                        }\n\n                        match client\n                            .post(uri)\n                            .header(CONTENT_TYPE, \"application/tlsrpt+gzip\")\n                            .body(json.to_vec())\n                            .send()\n                            .await\n                        {\n                            Ok(response) => {\n                                if response.status().is_success() {\n                                    trc::event!(\n                                        OutgoingReport(OutgoingReportEvent::HttpSubmission),\n                                        SpanId = span_id,\n                                        Url = uri.to_string(),\n                                        Code = response.status().as_u16(),\n                                    );\n\n                                    self.delete_tls_report(events).await;\n                                    return;\n                                } else {\n                                    trc::event!(\n                                        OutgoingReport(OutgoingReportEvent::SubmissionError),\n                                        SpanId = span_id,\n                                        Url = uri.to_string(),\n                                        Code = response.status().as_u16(),\n                                        Details = \"Invalid HTTP response\"\n                                    );\n                                }\n                            }\n                            Err(err) => {\n                                trc::event!(\n                                    OutgoingReport(OutgoingReportEvent::SubmissionError),\n                                    SpanId = span_id,\n                                    Url = uri.to_string(),\n                                    Reason = err.to_string(),\n                                    Details = \"HTTP submission error\"\n                                );\n                            }\n                        }\n                    }\n                }\n                ReportUri::Mail(mailto) => {\n                    rcpts.push(mailto.as_str());\n                }\n            }\n        }\n\n        // Deliver report over SMTP\n        if !rcpts.is_empty() {\n            let config = &self.core.smtp.report.tls;\n            let from_addr = self\n                .eval_if(&config.address, &RecipientDomain::new(domain_name), span_id)\n                .await\n                .unwrap_or_else(|| \"MAILER-DAEMON@localhost\".to_string());\n            let mut message = Vec::with_capacity(2048);\n            let _ = report.write_rfc5322_from_bytes(\n                domain_name,\n                &self\n                    .eval_if(\n                        &self.core.smtp.report.submitter,\n                        &RecipientDomain::new(domain_name),\n                        span_id,\n                    )\n                    .await\n                    .unwrap_or_else(|| \"localhost\".to_string()),\n                (\n                    self.eval_if(&config.name, &RecipientDomain::new(domain_name), span_id)\n                        .await\n                        .unwrap_or_else(|| \"Mail Delivery Subsystem\".to_string())\n                        .as_str(),\n                    from_addr.as_str(),\n                ),\n                rcpts.iter().copied(),\n                &json,\n                &mut message,\n            );\n\n            // Send report\n            self.send_report(\n                &from_addr,\n                rcpts.iter(),\n                message,\n                &config.sign,\n                false,\n                span_id,\n            )\n            .await;\n        } else {\n            trc::event!(\n                OutgoingReport(OutgoingReportEvent::NoRecipientsFound),\n                SpanId = span_id,\n            );\n        }\n        self.delete_tls_report(events).await;\n    }\n\n    async fn generate_tls_aggregate_report(\n        &self,\n        events: &[ReportEvent],\n        rua: &mut Vec<ReportUri>,\n        mut serialized_size: Option<&mut serde_json::Serializer<SerializedSize>>,\n        span_id: u64,\n    ) -> trc::Result<Option<TlsReport>> {\n        let (domain_name, event_from, event_to, policy) = events\n            .first()\n            .map(|e| (e.domain.as_str(), e.seq_id, e.due, e.policy_hash))\n            .unwrap();\n        let config = &self.core.smtp.report.tls;\n        let mut report = TlsReport {\n            organization_name: self\n                .eval_if::<String, _>(\n                    &config.org_name,\n                    &RecipientDomain::new(domain_name),\n                    span_id,\n                )\n                .await\n                .clone(),\n            date_range: DateRange {\n                start_datetime: DateTime::from_timestamp(event_from as i64),\n                end_datetime: DateTime::from_timestamp(event_to as i64),\n            },\n            contact_info: self\n                .eval_if::<String, _>(\n                    &config.contact_info,\n                    &RecipientDomain::new(domain_name),\n                    span_id,\n                )\n                .await\n                .clone(),\n            report_id: format!(\"{}_{}\", event_from, policy),\n            policies: Vec::with_capacity(events.len()),\n        };\n\n        if let Some(serialized_size) = serialized_size.as_deref_mut() {\n            let _ = serde::Serialize::serialize(&report, serialized_size);\n        }\n\n        for event in events {\n            let tls = if let Some(tls) = self\n                .store()\n                .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::Queue(\n                    QueueClass::TlsReportHeader(event.clone()),\n                )))\n                .await?\n            {\n                tls.deserialize::<TlsFormat>()?\n            } else {\n                continue;\n            };\n\n            if let Some(serialized_size) = serialized_size.as_deref_mut()\n                && serde::Serialize::serialize(&tls, serialized_size).is_err()\n            {\n                continue;\n            }\n\n            // Group duplicates\n            let mut total_success = 0;\n            let mut total_failure = 0;\n            let from_key =\n                ValueKey::from(ValueClass::Queue(QueueClass::TlsReportEvent(ReportEvent {\n                    due: event.due,\n                    policy_hash: event.policy_hash,\n                    seq_id: 0,\n                    domain: event.domain.clone(),\n                })));\n            let to_key =\n                ValueKey::from(ValueClass::Queue(QueueClass::TlsReportEvent(ReportEvent {\n                    due: event.due,\n                    policy_hash: event.policy_hash,\n                    seq_id: u64::MAX,\n                    domain: event.domain.clone(),\n                })));\n            let mut record_map = AHashMap::new();\n            self.core\n                .storage\n                .data\n                .iterate(IterateParams::new(from_key, to_key).ascending(), |_, v| {\n                    let archive = <Archive<AlignedBytes> as Deserialize>::deserialize(v)?;\n                    if let Some(failure_details) =\n                        archive.deserialize::<Option<FailureDetails>>()?\n                    {\n                        match record_map.entry(failure_details) {\n                            Entry::Occupied(mut e) => {\n                                total_failure += 1;\n                                *e.get_mut() += 1;\n                                Ok(true)\n                            }\n                            Entry::Vacant(e) => {\n                                if serialized_size\n                                    .as_deref_mut()\n                                    .is_none_or(|serialized_size| {\n                                        serde::Serialize::serialize(e.key(), serialized_size)\n                                            .is_ok()\n                                    })\n                                {\n                                    total_failure += 1;\n                                    e.insert(1u32);\n                                    Ok(true)\n                                } else {\n                                    Ok(false)\n                                }\n                            }\n                        }\n                    } else {\n                        total_success += 1;\n                        Ok(true)\n                    }\n                })\n                .await\n                .caused_by(trc::location!())?;\n\n            // Add policy\n            report.policies.push(Policy {\n                policy: tls.policy,\n                summary: Summary {\n                    total_success,\n                    total_failure,\n                },\n                failure_details: record_map\n                    .into_iter()\n                    .map(|(mut r, count)| {\n                        r.failed_session_count = count;\n                        r\n                    })\n                    .collect(),\n            });\n\n            // Add report URIs\n            for entry in tls.rua {\n                if !rua.contains(&entry) {\n                    rua.push(entry);\n                }\n            }\n        }\n\n        Ok(if !report.policies.is_empty() {\n            Some(report)\n        } else {\n            None\n        })\n    }\n\n    async fn schedule_tls(&self, event: Box<TlsEvent>) {\n        let created = event.interval.to_timestamp();\n        let deliver_at = created + event.interval.as_secs();\n        let mut report_event = ReportEvent {\n            due: deliver_at,\n            policy_hash: event.policy.to_hash(),\n            seq_id: created,\n            domain: event.domain,\n        };\n\n        // Write policy if missing\n        let mut builder = BatchBuilder::new();\n        if self\n            .core\n            .storage\n            .data\n            .get_value::<()>(ValueKey::from(ValueClass::Queue(\n                QueueClass::TlsReportHeader(report_event.clone()),\n            )))\n            .await\n            .unwrap_or_default()\n            .is_none()\n        {\n            // Serialize report\n            let mut policy = PolicyDetails {\n                policy_type: PolicyType::NoPolicyFound,\n                policy_string: vec![],\n                policy_domain: report_event.domain.clone(),\n                mx_host: vec![],\n            };\n\n            match event.policy {\n                common::ipc::PolicyType::Tlsa(tlsa) => {\n                    policy.policy_type = PolicyType::Tlsa;\n                    if let Some(tlsa) = tlsa {\n                        for entry in &tlsa.entries {\n                            policy.policy_string.push(format!(\n                                \"{} {} {} {}\",\n                                if entry.is_end_entity { 3 } else { 2 },\n                                i32::from(entry.is_spki),\n                                if entry.is_sha256 { 1 } else { 2 },\n                                entry\n                                    .data\n                                    .iter()\n                                    .fold(String::with_capacity(64), |mut s, b| {\n                                        write!(s, \"{b:02X}\").ok();\n                                        s\n                                    })\n                            ));\n                        }\n                    }\n                }\n                common::ipc::PolicyType::Sts(sts) => {\n                    policy.policy_type = PolicyType::Sts;\n                    if let Some(sts) = sts {\n                        policy.policy_string.push(\"version: STSv1\".to_string());\n                        policy.policy_string.push(format!(\n                            \"mode: {}\",\n                            match sts.mode {\n                                Mode::Enforce => \"enforce\",\n                                Mode::Testing => \"testing\",\n                                Mode::None => \"none\",\n                            }\n                        ));\n                        policy\n                            .policy_string\n                            .push(format!(\"max_age: {}\", sts.max_age));\n                        for mx in &sts.mx {\n                            let mx = match mx {\n                                MxPattern::Equals(mx) => mx.to_string(),\n                                MxPattern::StartsWith(mx) => format!(\"*.{mx}\"),\n                            };\n                            policy.policy_string.push(format!(\"mx: {mx}\"));\n                            policy.mx_host.push(mx);\n                        }\n                    }\n                }\n                _ => (),\n            }\n\n            // Create report entry\n            let entry = TlsFormat {\n                rua: event.tls_record.rua.clone(),\n                policy,\n                records: vec![],\n            };\n\n            // Write report\n            builder.set(\n                ValueClass::Queue(QueueClass::TlsReportHeader(report_event.clone())),\n                match Archiver::new(entry).serialize() {\n                    Ok(data) => data.to_vec(),\n                    Err(err) => {\n                        trc::error!(\n                            err.caused_by(trc::location!())\n                                .details(\"Failed to serialize TLS report\")\n                        );\n                        return;\n                    }\n                },\n            );\n        }\n\n        // Write entry\n        report_event.seq_id = self.inner.data.queue_id_gen.generate();\n        builder.set(\n            ValueClass::Queue(QueueClass::TlsReportEvent(report_event)),\n            match Archiver::new(event.failure).serialize() {\n                Ok(data) => data.to_vec(),\n                Err(err) => {\n                    trc::error!(\n                        err.caused_by(trc::location!())\n                            .details(\"Failed to serialize TLS report\")\n                    );\n                    return;\n                }\n            },\n        );\n\n        if let Err(err) = self.core.storage.data.write(builder.build_all()).await {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to write TLS report\")\n            );\n        }\n    }\n\n    async fn delete_tls_report(&self, events: Vec<ReportEvent>) {\n        let mut batch = BatchBuilder::new();\n\n        for event in events {\n            let from_key = ReportEvent {\n                due: event.due,\n                policy_hash: event.policy_hash,\n                seq_id: 0,\n                domain: event.domain.clone(),\n            };\n            let to_key = ReportEvent {\n                due: event.due,\n                policy_hash: event.policy_hash,\n                seq_id: u64::MAX,\n                domain: event.domain.clone(),\n            };\n\n            // Remove report events\n            if let Err(err) = self\n                .core\n                .storage\n                .data\n                .delete_range(\n                    ValueKey::from(ValueClass::Queue(QueueClass::TlsReportEvent(from_key))),\n                    ValueKey::from(ValueClass::Queue(QueueClass::TlsReportEvent(to_key))),\n                )\n                .await\n            {\n                trc::error!(\n                    err.caused_by(trc::location!())\n                        .details(\"Failed to delete TLS reports\")\n                );\n\n                return;\n            }\n\n            // Remove report header\n            batch.clear(ValueClass::Queue(QueueClass::TlsReportHeader(event)));\n        }\n\n        if let Err(err) = self.core.storage.data.write(batch.build_all()).await {\n            trc::error!(\n                err.caused_by(trc::location!())\n                    .details(\"Failed to delete TLS reports\")\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/scripts/envelope.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse sieve::Envelope;\nuse smtp_proto::{\n    MAIL_BY_NOTIFY, MAIL_BY_RETURN, MAIL_BY_TRACE, MAIL_RET_FULL, MAIL_RET_HDRS, RCPT_NOTIFY_DELAY,\n    RCPT_NOTIFY_FAILURE, RCPT_NOTIFY_NEVER, RCPT_NOTIFY_SUCCESS,\n};\nuse utils::DomainPart;\n\nuse crate::core::{SessionAddress, SessionData};\n\nimpl SessionData {\n    pub fn apply_envelope_modification(&mut self, envelope: Envelope, value: String) {\n        match envelope {\n            Envelope::From => {\n                let (address, address_lcase, domain) = if value.contains('@') {\n                    let address_lcase = value.to_lowercase();\n                    let domain = address_lcase.domain_part().into();\n                    (value, address_lcase, domain)\n                } else if value.is_empty() {\n                    (String::new(), String::new(), String::new())\n                } else {\n                    return;\n                };\n                if let Some(mail_from) = &mut self.mail_from {\n                    mail_from.address = address;\n                    mail_from.address_lcase = address_lcase;\n                    mail_from.domain = domain;\n                } else {\n                    self.mail_from = SessionAddress {\n                        address,\n                        address_lcase,\n                        domain,\n                        flags: 0,\n                        dsn_info: None,\n                    }\n                    .into();\n                }\n            }\n            Envelope::To => {\n                if value.contains('@') {\n                    let address_lcase = value.to_lowercase();\n                    let domain = address_lcase.domain_part().into();\n                    if let Some(rcpt_to) = self.rcpt_to.last_mut() {\n                        rcpt_to.address = value;\n                        rcpt_to.address_lcase = address_lcase;\n                        rcpt_to.domain = domain;\n                    } else {\n                        self.rcpt_to.push(SessionAddress {\n                            address: value,\n                            address_lcase,\n                            domain,\n                            flags: 0,\n                            dsn_info: None,\n                        });\n                    }\n                }\n            }\n            Envelope::ByMode => {\n                if let Some(mail_from) = &mut self.mail_from {\n                    mail_from.flags &= !(MAIL_BY_NOTIFY | MAIL_BY_RETURN);\n                    if value == \"N\" {\n                        mail_from.flags |= MAIL_BY_NOTIFY;\n                    } else if value == \"R\" {\n                        mail_from.flags |= MAIL_BY_RETURN;\n                    }\n                }\n            }\n            Envelope::ByTrace => {\n                if let Some(mail_from) = &mut self.mail_from {\n                    if value == \"T\" {\n                        mail_from.flags |= MAIL_BY_TRACE;\n                    } else {\n                        mail_from.flags &= !MAIL_BY_TRACE;\n                    }\n                }\n            }\n            Envelope::Notify => {\n                if let Some(rcpt_to) = self.rcpt_to.last_mut() {\n                    rcpt_to.flags &= !(RCPT_NOTIFY_DELAY\n                        | RCPT_NOTIFY_FAILURE\n                        | RCPT_NOTIFY_SUCCESS\n                        | RCPT_NOTIFY_NEVER);\n                    if value == \"NEVER\" {\n                        rcpt_to.flags |= RCPT_NOTIFY_NEVER;\n                    } else {\n                        for value in value.split(',') {\n                            match value.trim() {\n                                \"SUCCESS\" => rcpt_to.flags |= RCPT_NOTIFY_SUCCESS,\n                                \"FAILURE\" => rcpt_to.flags |= RCPT_NOTIFY_FAILURE,\n                                \"DELAY\" => rcpt_to.flags |= RCPT_NOTIFY_DELAY,\n                                _ => (),\n                            }\n                        }\n                    }\n                }\n            }\n            Envelope::Ret => {\n                if let Some(mail_from) = &mut self.mail_from {\n                    mail_from.flags &= !(MAIL_RET_FULL | MAIL_RET_HDRS);\n                    if value == \"FULL\" {\n                        mail_from.flags |= MAIL_RET_FULL;\n                    } else if value == \"HDRS\" {\n                        mail_from.flags |= MAIL_RET_HDRS;\n                    }\n                }\n            }\n            Envelope::Orcpt => {\n                if let Some(rcpt_to) = self.rcpt_to.last_mut() {\n                    rcpt_to.dsn_info = value.into();\n                }\n            }\n            Envelope::Envid => {\n                if let Some(mail_from) = &mut self.mail_from {\n                    mail_from.dsn_info = value.into();\n                }\n            }\n            Envelope::ByTimeAbsolute | Envelope::ByTimeRelative => (),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/scripts/event_loop.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    inbound::DkimSign,\n    queue::{MessageSource, quota::HasQueueQuota, spool::SmtpSpool},\n};\nuse common::{Server, config::smtp::queue::QueueExpiry, scripts::plugins::PluginContext};\nuse mail_auth::common::headers::HeaderWriter;\nuse mail_parser::{Encoding, Message, MessagePart, PartType};\nuse sieve::{\n    Event, Input, MatchAs, Recipient, Sieve,\n    compiler::grammar::actions::action_redirect::{ByMode, ByTime, Notify, NotifyItem, Ret},\n};\nuse smtp_proto::{\n    MAIL_BY_TRACE, MAIL_RET_FULL, MAIL_RET_HDRS, RCPT_NOTIFY_DELAY, RCPT_NOTIFY_FAILURE,\n    RCPT_NOTIFY_NEVER, RCPT_NOTIFY_SUCCESS,\n};\nuse std::{borrow::Cow, future::Future, sync::Arc, time::Instant};\nuse trc::SieveEvent;\n\nuse super::{ScriptModification, ScriptParameters, ScriptResult};\n\npub trait RunScript: Sync + Send {\n    fn run_script(\n        &self,\n        script_id: String,\n        script: Arc<Sieve>,\n        params: ScriptParameters<'_>,\n    ) -> impl Future<Output = ScriptResult> + Send;\n}\n\nimpl RunScript for Server {\n    async fn run_script(\n        &self,\n        script_id: String,\n        script: Arc<Sieve>,\n        params: ScriptParameters<'_>,\n    ) -> ScriptResult {\n        // Create filter instance\n        let time = Instant::now();\n        let mut instance = self\n            .core\n            .sieve\n            .trusted_runtime\n            .filter_parsed(params.message.unwrap_or_else(|| Message {\n                parts: vec![MessagePart {\n                    headers: vec![],\n                    is_encoding_problem: false,\n                    body: PartType::Text(\"\".into()),\n                    encoding: Encoding::None,\n                    offset_header: 0,\n                    offset_body: 0,\n                    offset_end: 0,\n                }],\n                raw_message: b\"\"[..].into(),\n                ..Default::default()\n            }))\n            .with_vars_env(params.variables)\n            .with_envelope_list(params.envelope)\n            .with_user_address(&params.from_addr)\n            .with_user_full_name(&params.from_name);\n        let mut input = Input::script(\"__script\", script);\n        let mut messages: Vec<Vec<u8>> = Vec::new();\n        let session_id = params.session_id;\n\n        let mut reject_reason = None;\n        let mut modifications = vec![];\n        let mut keep_id = usize::MAX;\n\n        // Start event loop\n        while let Some(result) = instance.run(input) {\n            match result {\n                Ok(event) => match event {\n                    Event::IncludeScript { name, optional } => {\n                        let name_ = name.as_str().to_lowercase();\n                        if let Some(script) = self.core.sieve.trusted_scripts.get(&name_) {\n                            input = Input::script(name, script.clone());\n                        } else if optional {\n                            input = false.into();\n                        } else {\n                            trc::event!(\n                                Sieve(SieveEvent::ScriptNotFound),\n                                Id = script_id.clone(),\n                                SpanId = session_id,\n                                Details = name_,\n                            );\n                            break;\n                        }\n                    }\n                    Event::ListContains {\n                        lists,\n                        values,\n                        match_as,\n                    } => {\n                        input = false.into();\n                        'outer: for list in lists {\n                            if let Some(store) = self.core.storage.lookups.get(&list) {\n                                for value in &values {\n                                    if let Ok(true) = store\n                                        .key_exists(if !matches!(match_as, MatchAs::Lowercase) {\n                                            value.clone()\n                                        } else {\n                                            value.to_lowercase()\n                                        })\n                                        .await\n                                    {\n                                        input = true.into();\n                                        break 'outer;\n                                    }\n                                }\n                            } else {\n                                trc::event!(\n                                    Sieve(SieveEvent::ListNotFound),\n                                    Id = script_id.clone(),\n                                    SpanId = session_id,\n                                    Details = list,\n                                );\n                            }\n                        }\n                    }\n                    Event::Function { id, arguments } => {\n                        input = self\n                            .core\n                            .run_plugin(\n                                id,\n                                PluginContext {\n                                    session_id,\n                                    server: self,\n                                    message: instance.message(),\n                                    modifications: &mut modifications,\n                                    access_token: params.access_token,\n                                    arguments,\n                                },\n                            )\n                            .await;\n                    }\n                    Event::Keep { message_id, .. } => {\n                        keep_id = message_id;\n                        input = true.into();\n                    }\n                    Event::Discard => {\n                        keep_id = usize::MAX - 1;\n                        input = true.into();\n                    }\n                    Event::Reject { reason, .. } => {\n                        reject_reason = reason.into();\n                        input = true.into();\n                    }\n                    Event::SendMessage {\n                        recipient,\n                        notify,\n                        return_of_content,\n                        by_time,\n                        message_id,\n                    } => {\n                        // Build message\n                        let mut message = self.new_message(params.return_path.as_str(), session_id);\n                        match recipient {\n                            Recipient::Address(rcpt) => {\n                                message.add_recipient(rcpt, self).await;\n                            }\n                            Recipient::Group(rcpt_list) => {\n                                for rcpt in rcpt_list {\n                                    message.add_recipient(rcpt, self).await;\n                                }\n                            }\n                            Recipient::List(list) => {\n                                trc::event!(\n                                    Sieve(SieveEvent::NotSupported),\n                                    Id = script_id.clone(),\n                                    SpanId = session_id,\n                                    Details = list,\n                                    Reason = \"Sending to lists is not supported.\",\n                                );\n                            }\n                        }\n\n                        // Set notify flags\n                        let mut flags = 0;\n                        match notify {\n                            Notify::Never => {\n                                flags = RCPT_NOTIFY_NEVER;\n                            }\n                            Notify::Items(items) => {\n                                for item in items {\n                                    flags |= match item {\n                                        NotifyItem::Success => RCPT_NOTIFY_SUCCESS,\n                                        NotifyItem::Failure => RCPT_NOTIFY_FAILURE,\n                                        NotifyItem::Delay => RCPT_NOTIFY_DELAY,\n                                    };\n                                }\n                            }\n                            Notify::Default => (),\n                        }\n                        if flags > 0 {\n                            for rcpt in &mut message.message.recipients {\n                                rcpt.flags |= flags;\n                            }\n                        }\n\n                        // Set ByTime flags\n                        match by_time {\n                            ByTime::Relative {\n                                rlimit,\n                                mode,\n                                trace,\n                            } => {\n                                if trace {\n                                    message.message.flags |= MAIL_BY_TRACE;\n                                }\n                                match mode {\n                                    ByMode::Notify => {\n                                        for domain in &mut message.message.recipients {\n                                            domain.notify.due += rlimit;\n                                        }\n                                    }\n                                    ByMode::Return => {\n                                        for domain in &mut message.message.recipients {\n                                            domain.notify.due += rlimit;\n                                        }\n                                    }\n                                    ByMode::Default => (),\n                                }\n                            }\n                            ByTime::Absolute {\n                                alimit,\n                                mode,\n                                trace,\n                            } => {\n                                if trace {\n                                    message.message.flags |= MAIL_BY_TRACE;\n                                }\n                                match mode {\n                                    ByMode::Notify => {\n                                        for domain in &mut message.message.recipients {\n                                            domain.notify.due = alimit as u64;\n                                        }\n                                    }\n                                    ByMode::Return => {\n                                        let expires =\n                                            (alimit as u64).saturating_sub(message.message.created);\n                                        if expires > 0 {\n                                            for domain in &mut message.message.recipients {\n                                                domain.expires = QueueExpiry::Ttl(expires);\n                                            }\n                                        }\n                                    }\n                                    ByMode::Default => (),\n                                }\n                            }\n                            ByTime::None => (),\n                        };\n\n                        // Set ret\n                        match return_of_content {\n                            Ret::Full => {\n                                message.message.flags |= MAIL_RET_FULL;\n                            }\n                            Ret::Hdrs => {\n                                message.message.flags |= MAIL_RET_HDRS;\n                            }\n                            Ret::Default => (),\n                        }\n\n                        // Queue message\n                        let is_forward = message_id == 0;\n                        let raw_message = if !is_forward {\n                            messages.get(message_id - 1).map(|m| m.as_slice())\n                        } else {\n                            instance.message().raw_message().into()\n                        };\n                        if let Some(raw_message) = raw_message.filter(|m| !m.is_empty()) {\n                            let headers = if !params.sign.is_empty() {\n                                let mut headers = Vec::new();\n\n                                for dkim in &params.sign {\n                                    if let Some(dkim) = self.get_dkim_signer(dkim, session_id) {\n                                        match dkim.sign(raw_message) {\n                                            Ok(signature) => {\n                                                signature.write_header(&mut headers);\n                                            }\n                                            Err(err) => {\n                                                trc::error!(\n                                                    trc::Error::from(err)\n                                                        .span_id(session_id)\n                                                        .caused_by(trc::location!())\n                                                        .details(\"DKIM sign failed\")\n                                                );\n                                            }\n                                        }\n                                    }\n                                }\n\n                                if is_forward {\n                                    headers.extend_from_slice(params.headers.unwrap_or_default());\n                                }\n\n                                Some(Cow::Owned(headers))\n                            } else if is_forward {\n                                params.headers.map(Cow::Borrowed)\n                            } else {\n                                None\n                            };\n\n                            if self.has_quota(&mut message).await {\n                                message\n                                    .queue(\n                                        headers.as_deref(),\n                                        raw_message,\n                                        session_id,\n                                        self,\n                                        MessageSource::Autogenerated,\n                                    )\n                                    .await;\n                            } else {\n                                trc::event!(\n                                    Sieve(SieveEvent::QuotaExceeded),\n                                    SpanId = session_id,\n                                    Id = script_id.clone(),\n                                    From = message.message.return_path,\n                                    To = message\n                                        .message\n                                        .recipients\n                                        .into_iter()\n                                        .map(|r| trc::Value::from(r.address().to_string()))\n                                        .collect::<Vec<_>>(),\n                                );\n                            }\n                        }\n\n                        input = true.into();\n                    }\n                    Event::CreatedMessage { message, .. } => {\n                        messages.push(message);\n                        input = true.into();\n                    }\n                    Event::SetEnvelope { envelope, value } => {\n                        modifications.push(ScriptModification::SetEnvelope {\n                            name: envelope,\n                            value,\n                        });\n                        input = true.into();\n                    }\n                    unsupported => {\n                        trc::event!(\n                            Sieve(SieveEvent::NotSupported),\n                            Id = script_id.clone(),\n                            SpanId = session_id,\n                            Reason = \"Unsupported event\",\n                            Details = format!(\"{unsupported:?}\"),\n                        );\n                        break;\n                    }\n                },\n                Err(err) => {\n                    trc::event!(\n                        Sieve(SieveEvent::RuntimeError),\n                        Id = script_id.clone(),\n                        SpanId = session_id,\n                        Reason = err.to_string(),\n                    );\n                    break;\n                }\n            }\n        }\n\n        // Keep id\n        // 0 = use original message\n        // MAX = implicit keep\n        // MAX - 1 = discard message\n\n        if keep_id == 0 {\n            trc::event!(\n                Sieve(SieveEvent::ActionAccept),\n                SpanId = session_id,\n                Id = script_id,\n                Elapsed = time.elapsed(),\n            );\n\n            ScriptResult::Accept { modifications }\n        } else if let Some(mut reject_reason) = reject_reason {\n            trc::event!(\n                Sieve(SieveEvent::ActionReject),\n                Id = script_id,\n                SpanId = session_id,\n                Details = reject_reason.clone(),\n                Elapsed = time.elapsed(),\n            );\n\n            if !reject_reason.ends_with('\\n') {\n                reject_reason.push_str(\"\\r\\n\");\n            }\n            let mut reject_bytes = reject_reason.as_bytes().iter();\n            if matches!(reject_bytes.next(), Some(ch) if ch.is_ascii_digit())\n                && matches!(reject_bytes.next(), Some(ch) if ch.is_ascii_digit())\n                && matches!(reject_bytes.next(), Some(ch) if ch.is_ascii_digit())\n                && matches!(reject_bytes.next(), Some(ch) if ch == &b' ' )\n            {\n                ScriptResult::Reject(reject_reason)\n            } else {\n                ScriptResult::Reject(format!(\"503 5.5.3 {reject_reason}\"))\n            }\n        } else if keep_id != usize::MAX - 1 {\n            if let Some(message) = messages.into_iter().nth(keep_id - 1) {\n                trc::event!(\n                    Sieve(SieveEvent::ActionAccept),\n                    SpanId = session_id,\n                    Id = script_id,\n                    Elapsed = time.elapsed(),\n                );\n\n                ScriptResult::Replace {\n                    message,\n                    modifications,\n                }\n            } else {\n                trc::event!(\n                    Sieve(SieveEvent::ActionAcceptReplace),\n                    SpanId = session_id,\n                    Id = script_id,\n                    Elapsed = time.elapsed(),\n                );\n\n                ScriptResult::Accept { modifications }\n            }\n        } else {\n            trc::event!(\n                Sieve(SieveEvent::ActionDiscard),\n                SpanId = session_id,\n                Id = script_id,\n                Elapsed = time.elapsed()\n            );\n\n            ScriptResult::Discard\n        }\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/scripts/exec.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{sync::Arc, time::SystemTime};\n\nuse common::listener::SessionStream;\n\nuse mail_auth::common::resolver::ToReverseName;\nuse sieve::{Envelope, Sieve, runtime::Variable};\nuse smtp_proto::*;\n\nuse crate::{core::Session, inbound::AuthResult};\n\nuse super::{ScriptParameters, ScriptResult, event_loop::RunScript};\n\nimpl<T: SessionStream> Session<T> {\n    pub fn build_script_parameters(&self, stage: &'static str) -> ScriptParameters<'_> {\n        let (tls_version, tls_cipher) = self.stream.tls_version_and_cipher();\n        let mut params = ScriptParameters::new()\n            .set_variable(\"remote_ip\", self.data.remote_ip.to_string())\n            .set_variable(\"remote_ip.reverse\", self.data.remote_ip.to_reverse_name())\n            .set_variable(\"helo_domain\", self.data.helo_domain.as_str().to_lowercase())\n            .set_variable(\n                \"authenticated_as\",\n                self.authenticated_as().unwrap_or_default().to_string(),\n            )\n            .set_variable(\n                \"now\",\n                SystemTime::now()\n                    .duration_since(SystemTime::UNIX_EPOCH)\n                    .map_or(0, |d| d.as_secs()),\n            )\n            .set_variable(\n                \"asn\",\n                self.data\n                    .asn_geo_data\n                    .asn\n                    .as_ref()\n                    .map(|r| r.id)\n                    .unwrap_or_default(),\n            )\n            .set_variable(\n                \"country\",\n                self.data\n                    .asn_geo_data\n                    .country\n                    .as_ref()\n                    .map(|r| r.as_str())\n                    .unwrap_or_default(),\n            )\n            .set_variable(\n                \"spf.result\",\n                self.data\n                    .spf_mail_from\n                    .as_ref()\n                    .map(|r| r.result().as_str())\n                    .unwrap_or_default(),\n            )\n            .set_variable(\n                \"spf_ehlo.result\",\n                self.data\n                    .spf_ehlo\n                    .as_ref()\n                    .map(|r| r.result().as_str())\n                    .unwrap_or_default(),\n            )\n            .set_variable(\"tls.version\", tls_version)\n            .set_variable(\"tls.cipher\", tls_cipher)\n            .set_variable(\"stage\", stage);\n        if let Some(ip_rev) = &self.data.iprev {\n            params = params.set_variable(\"iprev.result\", ip_rev.result().as_str());\n            if let Some(ptr) = ip_rev.ptr.as_ref().and_then(|addrs| addrs.first()) {\n                params = params.set_variable(\n                    \"iprev.ptr\",\n                    ptr.strip_suffix('.').unwrap_or(ptr).to_lowercase(),\n                );\n            }\n        }\n\n        if let Some(mail_from) = &self.data.mail_from {\n            params\n                .envelope\n                .push((Envelope::From, mail_from.address_lcase.to_string().into()));\n            if let Some(env_id) = &mail_from.dsn_info {\n                params\n                    .envelope\n                    .push((Envelope::Envid, env_id.as_str().to_lowercase().into()));\n            }\n\n            if stage != \"data\" {\n                if let Some(rcpt) = self.data.rcpt_to.last() {\n                    params\n                        .envelope\n                        .push((Envelope::To, rcpt.address_lcase.to_string().into()));\n                    if let Some(orcpt) = &rcpt.dsn_info {\n                        params\n                            .envelope\n                            .push((Envelope::Orcpt, orcpt.as_str().to_lowercase().into()));\n                    }\n                }\n            } else {\n                // Build recipients list\n                let mut recipients = vec![];\n                for rcpt in &self.data.rcpt_to {\n                    recipients.push(Variable::from(rcpt.address_lcase.to_string()));\n                }\n                params.envelope.push((Envelope::To, recipients.into()));\n            }\n\n            if (mail_from.flags & MAIL_RET_FULL) != 0 {\n                params.envelope.push((Envelope::Ret, \"FULL\".into()));\n            } else if (mail_from.flags & MAIL_RET_HDRS) != 0 {\n                params.envelope.push((Envelope::Ret, \"HDRS\".into()));\n            }\n            if (mail_from.flags & MAIL_BY_NOTIFY) != 0 {\n                params.envelope.push((Envelope::ByMode, \"N\".into()));\n            } else if (mail_from.flags & MAIL_BY_RETURN) != 0 {\n                params.envelope.push((Envelope::ByMode, \"R\".into()));\n            }\n\n            if (mail_from.flags & MAIL_BODY_7BIT) != 0 {\n                params = params.set_variable(\"param.body\", \"7bit\");\n            } else if (mail_from.flags & MAIL_BODY_8BITMIME) != 0 {\n                params = params.set_variable(\"param.body\", \"8bitmime\");\n            } else if (mail_from.flags & MAIL_BODY_BINARYMIME) != 0 {\n                params = params.set_variable(\"param.body\", \"binarymime\");\n            }\n\n            if (mail_from.flags & MAIL_SMTPUTF8) != 0 {\n                params = params.set_variable(\"param.smtputf8\", Variable::Integer(1));\n            }\n            if (mail_from.flags & MAIL_REQUIRETLS) != 0 {\n                params = params.set_variable(\"param.requiretls\", Variable::Integer(1));\n            }\n        }\n\n        params\n    }\n\n    pub async fn run_script(\n        &self,\n        script_id: String,\n        script: Arc<Sieve>,\n        params: ScriptParameters<'_>,\n    ) -> ScriptResult {\n        self.server\n            .run_script(\n                script_id,\n                script,\n                params\n                    .with_session_id(self.data.session_id)\n                    .with_envelope(&self.server, self, self.data.session_id)\n                    .await,\n            )\n            .await\n    }\n}\n"
  },
  {
    "path": "crates/smtp/src/scripts/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse ahash::AHashMap;\nuse common::{\n    Server, auth::AccessToken, expr::functions::ResolveVariable, scripts::ScriptModification,\n};\n\nuse mail_parser::Message;\nuse sieve::{Envelope, runtime::Variable};\n\npub mod envelope;\npub mod event_loop;\npub mod exec;\n\n#[derive(Debug, serde::Serialize)]\npub enum ScriptResult {\n    Accept {\n        modifications: Vec<ScriptModification>,\n    },\n    Replace {\n        message: Vec<u8>,\n        modifications: Vec<ScriptModification>,\n    },\n    Reject(String),\n    Discard,\n}\n\npub struct ScriptParameters<'x> {\n    message: Option<Message<'x>>,\n    headers: Option<&'x [u8]>,\n    variables: AHashMap<Cow<'static, str>, Variable>,\n    envelope: Vec<(Envelope, Variable)>,\n    from_addr: String,\n    from_name: String,\n    return_path: String,\n    sign: Vec<String>,\n    access_token: Option<&'x AccessToken>,\n    session_id: u64,\n}\n\nimpl<'x> ScriptParameters<'x> {\n    pub fn new() -> Self {\n        ScriptParameters {\n            variables: AHashMap::with_capacity(10),\n            envelope: Vec::with_capacity(6),\n            message: None,\n            headers: None,\n            from_addr: Default::default(),\n            from_name: Default::default(),\n            return_path: Default::default(),\n            sign: Default::default(),\n            access_token: None,\n            session_id: Default::default(),\n        }\n    }\n\n    pub async fn with_envelope(\n        mut self,\n        server: &Server,\n        vars: &impl ResolveVariable,\n        session_id: u64,\n    ) -> Self {\n        for (variable, expr) in [\n            (&mut self.from_addr, &server.core.sieve.from_addr),\n            (&mut self.from_name, &server.core.sieve.from_name),\n            (&mut self.return_path, &server.core.sieve.return_path),\n        ] {\n            if let Some(value) = server.eval_if(expr, vars, session_id).await {\n                *variable = value;\n            }\n        }\n        if let Some(value) = server\n            .eval_if(&server.core.sieve.sign, vars, session_id)\n            .await\n        {\n            self.sign = value;\n        }\n        self\n    }\n\n    pub fn with_message(self, message: Message<'x>) -> Self {\n        Self {\n            message: message.into(),\n            ..self\n        }\n    }\n\n    pub fn with_auth_headers(self, headers: &'x [u8]) -> Self {\n        Self {\n            headers: headers.into(),\n            ..self\n        }\n    }\n\n    pub fn set_variable(\n        mut self,\n        name: impl Into<Cow<'static, str>>,\n        value: impl Into<Variable>,\n    ) -> Self {\n        self.variables.insert(name.into(), value.into());\n        self\n    }\n\n    pub fn set_envelope(mut self, envelope: Envelope, value: impl Into<Variable>) -> Self {\n        self.envelope.push((envelope, value.into()));\n        self\n    }\n\n    pub fn with_access_token(mut self, access_token: &'x AccessToken) -> Self {\n        self.access_token = Some(access_token);\n        self\n    }\n\n    pub fn with_session_id(mut self, session_id: u64) -> Self {\n        self.session_id = session_id;\n        self\n    }\n}\n\nimpl Default for ScriptParameters<'_> {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/Cargo.toml",
    "content": "[package]\nname = \"spam-filter\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path = \"../utils\" }\ntypes = { path = \"../types\" }\nnlp = { path = \"../nlp\" }\nstore = { path = \"../store\" }\ntrc = { path = \"../trc\" }\ncommon = { path =  \"../common\" }\nsmtp-proto = { version = \"0.2\", features = [\"rkyv\"] }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nmail-builder = { version = \"0.4\" }\nmail-auth = { version = \"0.7.1\" }\nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\ntokio = { version = \"1.47\", features = [\"net\", \"macros\"] }\npsl = \"2\"\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\nidna = \"1.0\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\", \"stream\"]}\ndecancer = \"3.0.1\"\nunicode-security = \"0.1.0\"\ninfer = \"0.19\"\nsha1 = \"0.10\"\nsha2 = \"0.10.6\"\ncompact_str = \"0.9.0\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nunicode-general-category = \"1.1.0\"\nunicode-normalization = \"0.1.25\"\n\n[features]\ntest_mode = []\nenterprise = []\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/classifier.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{SpamFilterContext, modules::classifier::SpamClassifier};\nuse common::Server;\nuse std::future::Future;\n\npub trait SpamFilterAnalyzeClassify: Sync + Send {\n    fn spam_filter_analyze_classify(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n\n    fn spam_filter_analyze_spam_trap(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = bool> + Send;\n}\n\nimpl SpamFilterAnalyzeClassify for Server {\n    async fn spam_filter_analyze_classify(&self, ctx: &mut SpamFilterContext<'_>) {\n        if self.core.spam.classifier.is_some()\n            && !ctx.result.has_tag(\"SPAM_TRAP\")\n            && let Err(err) = self.spam_classify(ctx).await\n        {\n            trc::error!(err.span_id(ctx.input.span_id).caused_by(trc::location!()));\n        }\n    }\n\n    async fn spam_filter_analyze_spam_trap(&self, ctx: &mut SpamFilterContext<'_>) -> bool {\n        if let Some(store) = self.get_in_memory_store(\"spam-traps\") {\n            for addr in &ctx.output.env_to_addr {\n                match store.key_exists(addr.address.as_str()).await {\n                    Ok(true) => {\n                        ctx.result.add_tag(\"SPAM_TRAP\");\n                        return true;\n                    }\n                    Ok(false) => (),\n                    Err(err) => {\n                        trc::error!(err.span_id(ctx.input.span_id).caused_by(trc::location!()));\n                    }\n                }\n            }\n        }\n\n        false\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/date.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse mail_parser::HeaderName;\nuse store::write::now;\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeDate: Sync + Send {\n    fn spam_filter_analyze_date(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeDate for Server {\n    async fn spam_filter_analyze_date(&self, ctx: &mut SpamFilterContext<'_>) {\n        match ctx\n            .input\n            .message\n            .header(HeaderName::Date)\n            .map(|h| h.as_datetime())\n        {\n            Some(Some(date)) => {\n                let date = date.to_timestamp();\n                if date != 0 {\n                    let date_diff = now() as i64 - date;\n\n                    if date_diff > 86400 {\n                        // Older than a day\n                        ctx.result.add_tag(\"DATE_IN_PAST\");\n                    } else if -date_diff > 7200 {\n                        //# More than 2 hours in the future\n                        ctx.result.add_tag(\"DATE_IN_FUTURE\");\n                    }\n                } else {\n                    ctx.result.add_tag(\"INVALID_DATE\");\n                }\n            }\n            Some(None) => {\n                ctx.result.add_tag(\"INVALID_DATE\");\n            }\n\n            None => {\n                ctx.result.add_tag(\"MISSING_DATE\");\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/dmarc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse mail_auth::{DkimResult, DmarcResult, SpfResult, dmarc::Policy};\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeDmarc: Sync + Send {\n    fn spam_filter_analyze_dmarc(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeDmarc for Server {\n    async fn spam_filter_analyze_dmarc(&self, ctx: &mut SpamFilterContext<'_>) {\n        ctx.result.add_tag(\n            ctx.input\n                .spf_mail_from_result\n                .map_or(\"SPF_NA\", |r| match r.result() {\n                    SpfResult::Pass => \"SPF_ALLOW\",\n                    SpfResult::Fail => \"SPF_FAIL\",\n                    SpfResult::SoftFail => \"SPF_SOFTFAIL\",\n                    SpfResult::Neutral => \"SPF_NEUTRAL\",\n                    SpfResult::TempError => \"SPF_DNSFAIL\",\n                    SpfResult::PermError => \"SPF_PERMFAIL\",\n                    SpfResult::None => \"SPF_NA\",\n                }),\n        );\n\n        ctx.result.add_tag(\n            match ctx\n                .input\n                .dkim_result\n                .iter()\n                .find(|r| matches!(r.result(), DkimResult::Pass))\n                .or_else(|| ctx.input.dkim_result.first())\n                .map(|r| r.result())\n                .unwrap_or(&DkimResult::None)\n            {\n                DkimResult::Pass => \"DKIM_ALLOW\",\n                DkimResult::Fail(_) => \"DKIM_REJECT\",\n                DkimResult::PermError(_) => \"DKIM_PERMFAIL\",\n                DkimResult::TempError(_) => \"DKIM_TEMPFAIL\",\n                DkimResult::Neutral(_) | DkimResult::None => \"DKIM_NA\",\n            },\n        );\n\n        ctx.result\n            .add_tag(ctx.input.arc_result.map_or(\"ARC_NA\", |r| match r.result() {\n                DkimResult::Pass => \"ARC_ALLOW\",\n                DkimResult::Fail(_) => \"ARC_REJECT\",\n                DkimResult::PermError(_) => \"ARC_INVALID\",\n                DkimResult::TempError(_) => \"ARC_DNSFAIL\",\n                DkimResult::Neutral(_) | DkimResult::None => \"ARC_NA\",\n            }));\n\n        ctx.result\n            .add_tag(ctx.input.dmarc_result.map_or(\"DMARC_NA\", |r| match r {\n                DmarcResult::Pass => \"DMARC_POLICY_ALLOW\",\n                DmarcResult::TempError(_) => \"DMARC_DNSFAIL\",\n                DmarcResult::PermError(_) => \"DMARC_BAD_POLICY\",\n                DmarcResult::None => \"DMARC_NA\",\n                DmarcResult::Fail(_) => ctx.input.dmarc_policy.map_or(\n                    \"DMARC_POLICY_SOFTFAIL\",\n                    |p| match p {\n                        Policy::Quarantine => \"DMARC_POLICY_QUARANTINE\",\n                        Policy::Reject => \"DMARC_POLICY_REJECT\",\n                        Policy::Unspecified | Policy::None => \"DMARC_POLICY_SOFTFAIL\",\n                    },\n                ),\n            }));\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/domain.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ElementLocation, is_trusted_domain};\nuse crate::{\n    Email, Hostname, Recipient, SpamFilterContext, TextPart,\n    modules::{\n        dnsbl::check_dnsbl,\n        expression::StringResolver,\n        html::{A, HREF, HtmlToken},\n    },\n};\nuse common::{\n    Server,\n    config::spamfilter::{Element, Location},\n};\nuse mail_auth::DkimResult;\nuse mail_parser::{HeaderName, HeaderValue, Host, parsers::MessageStream};\nuse nlp::tokenizers::types::TokenType;\nuse std::{collections::HashSet, future::Future};\n\npub trait SpamFilterAnalyzeDomain: Sync + Send {\n    fn spam_filter_analyze_domain(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeDomain for Server {\n    async fn spam_filter_analyze_domain(&self, ctx: &mut SpamFilterContext<'_>) {\n        // Obtain email addresses and domains\n        let mut domains: HashSet<ElementLocation<String>> = HashSet::new();\n        let mut emails: HashSet<ElementLocation<Recipient>> = HashSet::new();\n\n        // Add DKIM domains\n        for dkim in ctx.input.dkim_result {\n            if dkim.result() == &DkimResult::Pass\n                && let Some(domain) = dkim.signature().map(|s| &s.d)\n            {\n                domains.insert(ElementLocation::new(\n                    domain.to_lowercase(),\n                    Location::HeaderDkimPass,\n                ));\n            }\n        }\n\n        // Add Received headers\n        for header in ctx.input.message.headers() {\n            match (&header.name, &header.value) {\n                (HeaderName::Received, HeaderValue::Received(received)) => {\n                    for host in [&received.from, &received.helo, &received.by]\n                        .into_iter()\n                        .flatten()\n                    {\n                        if let Host::Name(name) = host {\n                            let host = Hostname::new(name.as_ref());\n\n                            if host.sld.is_some() {\n                                domains.insert(ElementLocation::new(\n                                    host.fqdn,\n                                    Location::HeaderReceived,\n                                ));\n                            }\n                        }\n                    }\n                }\n                (HeaderName::MessageId, value) => {\n                    if let Some(mid_domain) = value\n                        .as_text()\n                        .and_then(|s| s.rsplit_once('@'))\n                        .and_then(|(_, d)| {\n                            let host = Hostname::new(d);\n                            if host.sld.is_some() { Some(host) } else { None }\n                        })\n                    {\n                        domains.insert(ElementLocation::new(mid_domain.fqdn, Location::HeaderMid));\n                    }\n                }\n                (HeaderName::Other(name), _)\n                    if name.eq_ignore_ascii_case(\"Disposition-Notification-To\") =>\n                {\n                    if let Some(address) = MessageStream::new(\n                        ctx.input\n                            .message\n                            .raw_message\n                            .get(header.offset_start as usize..header.offset_end as usize)\n                            .unwrap_or_default(),\n                    )\n                    .parse_address()\n                    .as_address()\n                    {\n                        for addr in address.iter() {\n                            if let Some(email) = addr.address() {\n                                emails.insert(ElementLocation::new(\n                                    Recipient {\n                                        email: Email::new(email),\n                                        name: None,\n                                    },\n                                    Location::HeaderDnt,\n                                ));\n                            }\n                        }\n                    }\n                }\n                _ => (),\n            }\n        }\n\n        // Add EHLO domain\n        if ctx.output.ehlo_host.sld.is_some() {\n            domains.insert(ElementLocation::new(\n                ctx.output.ehlo_host.fqdn.clone(),\n                Location::Ehlo,\n            ));\n        }\n\n        // Add PTR\n        if let Some(ptr) = &ctx.output.iprev_ptr {\n            domains.insert(ElementLocation::new(ptr.clone(), Location::Tcp));\n        }\n\n        // Add From, Envelope From and Reply-To\n        emails.insert(ElementLocation::new(\n            ctx.output.from.clone(),\n            Location::HeaderFrom,\n        ));\n        if let Some(reply_to) = &ctx.output.reply_to {\n            emails.insert(ElementLocation::new(\n                reply_to.clone(),\n                Location::HeaderReplyTo,\n            ));\n        }\n        emails.insert(ElementLocation::new(\n            Recipient {\n                email: ctx.output.env_from_addr.clone(),\n                name: None,\n            },\n            Location::EnvelopeFrom,\n        ));\n\n        // Add emails found in the message\n        for (part_id, part) in ctx.output.text_parts.iter().enumerate() {\n            let part_id = part_id as u32;\n            let is_body = ctx.input.message.text_body.contains(&part_id)\n                || ctx.input.message.html_body.contains(&part_id);\n            let tokens = match part {\n                TextPart::Plain { tokens, .. } => tokens,\n                TextPart::Html {\n                    tokens,\n                    html_tokens,\n                    ..\n                } => {\n                    emails.extend(html_tokens.iter().filter_map(|token| {\n                        if let HtmlToken::StartTag {\n                            name: A,\n                            attributes,\n                            ..\n                        } = token\n                        {\n                            attributes.iter().find_map(|(attr, value)| {\n                                if *attr == HREF {\n                                    let value = value.as_deref()?.strip_prefix(\"mailto:\")?;\n                                    let email =\n                                        Email::new(value.split_once('?').map_or(value, |(e, _)| e));\n\n                                    if email.is_valid() {\n                                        return Some(ElementLocation::new(\n                                            Recipient { email, name: None },\n                                            if is_body {\n                                                Location::BodyHtml\n                                            } else {\n                                                Location::Attachment\n                                            },\n                                        ));\n                                    }\n                                }\n                                None\n                            })\n                        } else {\n                            None\n                        }\n                    }));\n                    tokens\n                }\n                TextPart::None => continue,\n            };\n\n            for token in tokens {\n                if let TokenType::Email(email) = token {\n                    if !ctx.input.is_train && is_body && !ctx.result.has_tag(\"RCPT_IN_BODY\") {\n                        for rcpt in ctx.output.all_recipients() {\n                            if rcpt.email.address == email.address {\n                                ctx.result.add_tag(\"RCPT_IN_BODY\");\n                                break;\n                            }\n                        }\n                    }\n\n                    if email.is_valid() {\n                        emails.insert(ElementLocation::new(\n                            Recipient {\n                                email: email.clone(),\n                                name: None,\n                            },\n                            if is_body {\n                                Location::BodyText\n                            } else {\n                                Location::Attachment\n                            },\n                        ));\n                    }\n                }\n            }\n        }\n\n        if !ctx.input.is_train {\n            // Validate email\n            for email in &emails {\n                // Skip trusted domains\n                if !email.element.email.is_valid()\n                    || is_trusted_domain(\n                        self,\n                        &email.element.email.domain_part.fqdn,\n                        ctx.input.span_id,\n                    )\n                    .await\n                {\n                    continue;\n                }\n\n                // Check Email DNSBL\n                check_dnsbl(self, ctx, &email.element, Element::Email, email.location).await;\n\n                domains.insert(ElementLocation::new(\n                    email.element.email.domain_part.fqdn.clone(),\n                    email.location,\n                ));\n            }\n\n            // Validate domains\n            for domain in &domains {\n                // Skip trusted domains\n                if !is_trusted_domain(self, &domain.element, ctx.input.span_id).await {\n                    // Check Domain DNSBL\n                    check_dnsbl(\n                        self,\n                        ctx,\n                        &StringResolver(domain.element.as_str()),\n                        Element::Domain,\n                        domain.location,\n                    )\n                    .await;\n                }\n            }\n        }\n        ctx.output.emails = emails;\n        ctx.output.domains = domains;\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/ehlo.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeEhlo: Sync + Send {\n    fn spam_filter_analyze_ehlo(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeEhlo for Server {\n    async fn spam_filter_analyze_ehlo(&self, ctx: &mut SpamFilterContext<'_>) {\n        if let Some(ehlo_ip) = ctx.output.ehlo_host.ip {\n            // Helo host is bare ip\n            ctx.result.add_tag(\"HELO_BAREIP\");\n\n            if ehlo_ip != ctx.input.remote_ip {\n                // Helo A IP != hostname IP\n                ctx.result.add_tag(\"HELO_IP_A\");\n            }\n        } else if ctx.output.ehlo_host.sld.is_some() {\n            if ctx\n                .output\n                .iprev_ptr\n                .as_ref()\n                .is_some_and(|ptr| *ptr != ctx.output.ehlo_host.fqdn)\n            {\n                // Helo does not match reverse IP\n                ctx.result.add_tag(\"HELO_IPREV_MISMATCH\");\n            }\n\n            if matches!(\n                (\n                    self.dns_exists_ip(&ctx.output.ehlo_host.fqdn).await,\n                    self.dns_exists_mx(&ctx.output.ehlo_host.fqdn).await\n                ),\n                (Ok(false), Ok(false))\n            ) {\n                // Helo no resolve to A or MX\n                ctx.result.add_tag(\"HELO_NORES_A_OR_MX\");\n            }\n        } else {\n            if ctx.output.ehlo_host.fqdn.contains(\"user\") {\n                // Helo host contains 'user'\n                ctx.result.add_tag(\"RCVD_HELO_USER\");\n            }\n\n            // Helo not FQDN\n            ctx.result.add_tag(\"HELO_NOT_FQDN\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/from.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{Email, SpamFilterContext};\nuse common::Server;\nuse mail_parser::HeaderName;\nuse nlp::tokenizers::types::{TokenType, TypesTokenizer};\nuse smtp_proto::{MAIL_BODY_8BITMIME, MAIL_BODY_BINARYMIME, MAIL_SMTPUTF8};\nuse std::future::Future;\n\npub trait SpamFilterAnalyzeFrom: Sync + Send {\n    fn spam_filter_analyze_from(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeFrom for Server {\n    async fn spam_filter_analyze_from(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut from_count = 0;\n        let mut from_raw = b\"\".as_slice();\n        let mut crt = None;\n        let mut dnt = None;\n\n        for header in ctx.input.message.headers() {\n            match &header.name {\n                HeaderName::From => {\n                    from_count += 1;\n                    from_raw = ctx\n                        .input\n                        .message\n                        .raw_message()\n                        .get(header.offset_start as usize..header.offset_end as usize)\n                        .unwrap_or_default();\n                }\n                HeaderName::Other(name) => {\n                    if name.eq_ignore_ascii_case(\"X-Confirm-Reading-To\") {\n                        crt = ctx\n                            .input\n                            .header_as_address(header)\n                            .map(|s| s.to_lowercase());\n                    } else if name.eq_ignore_ascii_case(\"Disposition-Notification-To\") {\n                        dnt = ctx\n                            .input\n                            .header_as_address(header)\n                            .map(|s| s.to_lowercase());\n                    }\n                }\n                _ => {}\n            }\n        }\n\n        match from_count {\n            0 => {\n                ctx.result.add_tag(\"MISSING_FROM\");\n            }\n            1 => {}\n            _ => {\n                ctx.result.add_tag(\"MULTIPLE_FROM\");\n            }\n        }\n\n        let env_from_empty = ctx.output.env_from_addr.address.is_empty();\n        let from_addr = &ctx.output.from.email;\n        let from_name = ctx.output.from.name.as_deref().unwrap_or_default();\n        if from_count > 0 {\n            // Validate address\n            let from_addr_is_valid = from_addr.is_valid();\n            if !from_addr_is_valid {\n                ctx.result.add_tag(\"FROM_INVALID\");\n            }\n\n            // Validate from name\n            let from_name_trimmed = from_name.trim();\n            if from_name_trimmed.is_empty() {\n                ctx.result.add_tag(\"FROM_NO_DN\");\n            } else if from_name_trimmed == from_addr.address {\n                ctx.result.add_tag(\"FROM_DN_EQ_ADDR\");\n            } else {\n                if from_addr_is_valid {\n                    ctx.result.add_tag(\"FROM_HAS_DN\");\n                }\n\n                if from_name_trimmed.contains('@')\n                    && let Some(from_name_addr) = TypesTokenizer::new(from_name_trimmed)\n                        .tokenize_numbers(false)\n                        .tokenize_urls(false)\n                        .tokenize_urls_without_scheme(false)\n                        .tokenize_emails(true)\n                        .filter_map(|t| match t.word {\n                            TokenType::Email(email) => {\n                                let email = Email::new(email);\n                                email.is_valid().then_some(email)\n                            }\n                            _ => None,\n                        })\n                        .next()\n                {\n                    if (from_addr_is_valid\n                        && from_name_addr.domain_part.sld != from_addr.domain_part.sld)\n                        || (!env_from_empty\n                            && ctx.output.env_from_addr.domain_part.sld\n                                != from_name_addr.domain_part.sld)\n                        || (env_from_empty\n                            && ctx.output.ehlo_host.sld != from_name_addr.domain_part.sld)\n                    {\n                        ctx.result.add_tag(\"SPOOF_DISPLAY_NAME\");\n                    } else {\n                        ctx.result.add_tag(\"FROM_NEQ_DISPLAY_NAME\");\n                    }\n                }\n            }\n\n            // Check sender\n            if ctx.output.env_from_postmaster {\n                ctx.result.add_tag(\"FROM_BOUNCE\");\n            }\n\n            if !env_from_empty && ctx.output.env_from_addr.address == from_addr.address {\n                ctx.result.add_tag(\"FROM_EQ_ENV_FROM\");\n            } else if from_addr_is_valid {\n                if from_addr.domain_part.sld == ctx.output.ehlo_host.sld {\n                    ctx.result.add_tag(\"FROMTLD_EQ_ENV_FROMTLD\");\n                } else if !ctx.output.env_from_postmaster {\n                    ctx.result.add_tag(\"FORGED_SENDER\");\n                    ctx.result.add_tag(\"FROM_NEQ_ENV_FROM\");\n                }\n            }\n\n            // Validate FROM/TO relationship\n            if ctx.output.recipients_to.len() + ctx.output.recipients_cc.len() == 1 {\n                let rcpt = ctx\n                    .output\n                    .recipients_to\n                    .first()\n                    .or_else(|| ctx.output.recipients_cc.first())\n                    .unwrap();\n                if rcpt.email.address == from_addr.address {\n                    ctx.result.add_tag(\"TO_EQ_FROM\");\n                } else if rcpt.email.domain_part.fqdn == from_addr.domain_part.fqdn {\n                    ctx.result.add_tag(\"TO_DOM_EQ_FROM_DOM\");\n                }\n            }\n\n            // Validate encoding\n            let from_raw_utf8 = std::str::from_utf8(from_raw);\n            if !from_raw.is_ascii() {\n                if (ctx.input.env_from_flags\n                    & (MAIL_SMTPUTF8 | MAIL_BODY_8BITMIME | MAIL_BODY_BINARYMIME))\n                    == 0\n                {\n                    ctx.result.add_tag(\"FROM_NEEDS_ENCODING\");\n                }\n\n                if from_raw_utf8.is_err() {\n                    ctx.result.add_tag(\"INVALID_FROM_8BIT\");\n                }\n            }\n\n            // Validate unnecessary encoding\n            let from_raw_utf8 = from_raw_utf8.unwrap_or_default();\n            if from_name.is_ascii()\n                && from_addr.address.is_ascii()\n                && from_raw_utf8.contains(\"=?\")\n                && from_raw_utf8.contains(\"?=\")\n            {\n                if from_raw_utf8.contains(\"?q?\") || from_raw_utf8.contains(\"?Q?\") {\n                    // From header is unnecessarily encoded in quoted-printable\n                    ctx.result.add_tag(\"FROM_EXCESS_QP\");\n                } else if from_raw_utf8.contains(\"?b?\") || from_raw_utf8.contains(\"?B?\") {\n                    // From header is unnecessarily encoded in base64\n                    ctx.result.add_tag(\"FROM_EXCESS_BASE64\");\n                }\n            }\n\n            // Validate space in FROM\n            if !from_name.is_empty()\n                && !from_addr.address.is_empty()\n                && from_raw_utf8\n                    .as_bytes()\n                    .iter()\n                    .position(|&b| b == b'<')\n                    .and_then(|v| from_raw_utf8.as_bytes().get(v - 1))\n                    .is_none_or(|v| !v.is_ascii_whitespace())\n            {\n                ctx.result.add_tag(\"NO_SPACE_IN_FROM\");\n            }\n\n            // Check whether read confirmation address is different to from address\n            if let Some(crt) = crt\n                && crt != from_addr.address\n            {\n                ctx.result.add_tag(\"HEADER_RCONFIRM_MISMATCH\");\n            }\n        }\n\n        if !env_from_empty {\n            // Validate envelope address\n            if ctx.output.env_from_addr.is_valid() {\n                // Mail from no resolve to A or MX\n                if matches!(\n                    (\n                        self.dns_exists_ip(&ctx.output.env_from_addr.domain_part.fqdn)\n                            .await,\n                        self.dns_exists_mx(&ctx.output.env_from_addr.domain_part.fqdn)\n                            .await\n                    ),\n                    (Ok(false), Ok(false))\n                ) {\n                    // Helo no resolve to A or MX\n                    ctx.result.add_tag(\"FROMHOST_NORES_A_OR_MX\");\n                }\n            } else {\n                ctx.result.add_tag(\"ENV_FROM_INVALID\");\n            }\n\n            // Check whether disposition notification address is different to return path\n            if let Some(dnt) = dnt\n                && dnt != ctx.output.env_from_addr.address\n            {\n                ctx.result.add_tag(\"HEADER_FORGED_MDN\");\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/headers.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse mail_parser::HeaderName;\nuse store::ahash::AHashSet;\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeHeaders: Sync + Send {\n    fn spam_filter_analyze_headers(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeHeaders for Server {\n    async fn spam_filter_analyze_headers(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut list_score = 0.0;\n        let mut unique_headers = AHashSet::new();\n        let raw_message = ctx.input.message.raw_message();\n\n        for header in ctx.input.message.headers() {\n            // Add header exists tag\n            let hdr_name = header.name();\n            let mut tag: String = String::with_capacity(hdr_name.len() + 5);\n            tag.push_str(\"X_HDR_\");\n            for ch in hdr_name.chars() {\n                if ch.is_ascii_alphanumeric() {\n                    tag.push(ch.to_ascii_uppercase());\n                } else if ch == '-' {\n                    tag.push('_');\n                } else {\n                    tag.push(' ');\n                }\n            }\n            ctx.result.add_tag(tag);\n\n            match &header.name {\n                HeaderName::ContentType\n                | HeaderName::ContentTransferEncoding\n                | HeaderName::Date\n                | HeaderName::From\n                | HeaderName::Sender\n                | HeaderName::To\n                | HeaderName::Cc\n                | HeaderName::Bcc\n                | HeaderName::ReplyTo\n                | HeaderName::Subject\n                | HeaderName::MessageId\n                | HeaderName::References\n                | HeaderName::InReplyTo => {\n                    if !unique_headers.insert(header.name.clone()) {\n                        ctx.result.add_tag(\"MULTIPLE_UNIQUE_HEADERS\");\n                    }\n\n                    let mut value = raw_message\n                        .get(header.offset_start as usize..)\n                        .unwrap_or_default()\n                        .iter();\n                    loop {\n                        match value.next() {\n                            Some(b' ' | b'\\t') => {\n                                break;\n                            }\n                            Some(b'\\r' | b'\\n') => {}\n                            _ => {\n                                ctx.result.add_tag(\"HEADER_EMPTY_DELIMITER\");\n                                break;\n                            }\n                        }\n                    }\n                }\n                HeaderName::ListArchive\n                | HeaderName::ListOwner\n                | HeaderName::ListHelp\n                | HeaderName::ListPost => {\n                    list_score += 0.125;\n                }\n                HeaderName::ListId => {\n                    list_score += 0.5125;\n                }\n                HeaderName::ListSubscribe => {\n                    list_score += 0.25;\n                }\n                HeaderName::ListUnsubscribe => {\n                    list_score += 0.25;\n                    ctx.result.add_tag(\"HAS_LIST_UNSUB\");\n                }\n                HeaderName::Other(name) => {\n                    let value = header\n                        .value()\n                        .as_text()\n                        .unwrap_or_default()\n                        .trim()\n                        .to_lowercase();\n\n                    if name.eq_ignore_ascii_case(\"Precedence\") {\n                        if value == \"bulk\" {\n                            list_score += 0.25;\n                            ctx.result.add_tag(\"PRECEDENCE_BULK\");\n                        } else if value == \"list\" {\n                            list_score += 0.25;\n                        }\n                    } else if name.eq_ignore_ascii_case(\"X-Loop\") {\n                        list_score += 0.125;\n                    } else if name.eq_ignore_ascii_case(\"X-Priority\") {\n                        match value.parse::<i32>().unwrap_or(i32::MAX) {\n                            0 => {\n                                ctx.result.add_tag(\"HAS_X_PRIO_ZERO\");\n                            }\n                            1 => {\n                                ctx.result.add_tag(\"HAS_X_PRIO_ONE\");\n                            }\n                            2 => {\n                                ctx.result.add_tag(\"HAS_X_PRIO_TWO\");\n                            }\n                            3 | 4 => {\n                                ctx.result.add_tag(\"HAS_X_PRIO_THREE\");\n                            }\n                            4..=10000 => {\n                                ctx.result.add_tag(\"HAS_X_PRIO_FIVE\");\n                            }\n                            _ => {}\n                        }\n                    }\n                }\n                _ => {}\n            }\n        }\n\n        if list_score >= 1.0 {\n            ctx.result.add_tag(\"MAILLIST\");\n        }\n\n        if unique_headers.is_empty() {\n            ctx.result.add_tag(\"MISSING_ESSENTIAL_HEADERS\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/html.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse hyper::Uri;\nuse mail_parser::MimeHeaders;\nuse nlp::tokenizers::types::{TokenType, TypesTokenizer};\n\nuse crate::{Hostname, SpamFilterContext, TextPart, modules::html::*};\n\npub trait SpamFilterAnalyzeHtml: Sync + Send {\n    fn spam_filter_analyze_html(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\n#[derive(Debug)]\nstruct Href {\n    url_parsed: Option<Uri>,\n    host: Option<Hostname>,\n}\n\nimpl SpamFilterAnalyzeHtml for Server {\n    async fn spam_filter_analyze_html(&self, ctx: &mut SpamFilterContext<'_>) {\n        // Message only has text/html MIME parts\n        if ctx.input.message.content_type().is_some_and(|ct| {\n            ct.ctype().eq_ignore_ascii_case(\"text\")\n                && ct\n                    .subtype()\n                    .unwrap_or_default()\n                    .eq_ignore_ascii_case(\"html\")\n        }) {\n            ctx.result.add_tag(\"MIME_HTML_ONLY\");\n        }\n\n        for (part_id, part) in ctx.output.text_parts.iter().enumerate() {\n            let part_id = part_id as u32;\n            let is_body_part = ctx.input.message.text_body.contains(&part_id)\n                || ctx.input.message.html_body.contains(&part_id);\n\n            let (html_tokens, tokens) = if let TextPart::Html {\n                html_tokens,\n                tokens,\n                ..\n            } = part\n            {\n                (html_tokens, tokens)\n            } else {\n                continue;\n            };\n\n            let mut has_link_to_img = false;\n            let mut last_href: Option<Href> = None;\n            let mut html_img_words = 0;\n            let mut in_head: i32 = 0;\n            let mut in_body: i32 = 0;\n\n            for token in html_tokens {\n                match token {\n                    HtmlToken::StartTag {\n                        name,\n                        attributes,\n                        is_self_closing,\n                    } => match *name {\n                        A => {\n                            if let Some(attr) = attributes.iter().find_map(|(attr, value)| {\n                                if *attr == HREF {\n                                    value.as_deref()\n                                } else {\n                                    None\n                                }\n                            }) {\n                                let url = attr.trim().to_lowercase();\n                                let url_parsed = url.parse::<Uri>().ok();\n                                let href = Href {\n                                    host: url_parsed\n                                        .as_ref()\n                                        .and_then(|uri| uri.host().map(Hostname::new)),\n                                    url_parsed,\n                                };\n\n                                if is_body_part\n                                    && attr.starts_with(\"data:\")\n                                    && attr.contains(\";base64,\")\n                                {\n                                    // Has Data URI encoding\n                                    ctx.result.add_tag(\"HAS_DATA_URI\");\n                                    if attr.contains(\"text/\") {\n                                        //  Uses Data URI encoding to obfuscate plain or HTML in base64\n                                        ctx.result.add_tag(\"DATA_URI_OBFU\");\n                                    }\n                                } else if href.host.as_ref().is_some_and(|h| h.ip.is_some()) {\n                                    // HTML anchor points to an IP address\n                                    ctx.result.add_tag(\"HTTP_TO_IP\");\n                                }\n\n                                if !*is_self_closing {\n                                    last_href = Some(href);\n                                }\n                            }\n                        }\n                        IMG if is_body_part => {\n                            let mut img_width = 800;\n                            let mut img_height = 600;\n\n                            for (attr, value) in attributes {\n                                if let Some(value) =\n                                    value.as_deref().map(|v| v.trim()).filter(|v| !v.is_empty())\n                                {\n                                    let dimension = match *attr {\n                                        WIDTH => &mut img_width,\n                                        HEIGHT => &mut img_height,\n                                        SRC => {\n                                            let src = value.to_ascii_lowercase();\n                                            if src.starts_with(\"data:\") && src.contains(\";base64,\")\n                                            {\n                                                // Has Data URI encoding\n                                                ctx.result.add_tag(\"HAS_DATA_URI\");\n                                            } else if src.starts_with(\"https://\")\n                                                || src.starts_with(\"http://\")\n                                            {\n                                                // Has external image\n                                                ctx.result.add_tag(\"HAS_EXTERNAL_IMG\");\n                                            }\n                                            continue;\n                                        }\n                                        _ => {\n                                            continue;\n                                        }\n                                    };\n                                    if let Some(pct) = value.strip_suffix('%') {\n                                        if let Ok(pct) = pct.trim().parse::<u64>() {\n                                            *dimension = (*dimension * pct) / 100;\n                                        }\n                                    } else if let Ok(value) = value.parse::<u64>() {\n                                        *dimension = value;\n                                    }\n                                }\n                            }\n                            let dimensions = img_width + img_height;\n\n                            if last_href.is_some() {\n                                if dimensions >= 210 {\n                                    ctx.result.add_tag(\"HAS_LINK_TO_LARGE_IMG\");\n                                    has_link_to_img = true;\n                                } else {\n                                    ctx.result.add_tag(\"HAS_LINK_TO_IMG\");\n                                }\n                            }\n\n                            if dimensions > 100 {\n                                // We assume that a single picture 100x200 contains approx 3 words of text\n                                html_img_words += dimensions / 100;\n                            }\n                        }\n                        META => {\n                            let mut has_equiv_refresh = false;\n                            let mut has_content_url = false;\n\n                            for (attr, value) in attributes {\n                                if let Some(value) =\n                                    value.as_deref().map(|v| v.trim()).filter(|v| !v.is_empty())\n                                {\n                                    if *attr == HTTP_EQUIV {\n                                        if value.eq_ignore_ascii_case(\"refresh\") {\n                                            has_equiv_refresh = true;\n                                        }\n                                    } else if *attr == CONTENT\n                                        && value.to_ascii_lowercase().contains(\"url=\")\n                                    {\n                                        has_content_url = true;\n                                    }\n                                }\n                            }\n\n                            if has_equiv_refresh && has_content_url {\n                                // HTML meta refresh tag\n                                ctx.result.add_tag(\"HTML_META_REFRESH_URL\");\n                            }\n                        }\n                        LINK if is_body_part => {\n                            let mut has_rel_style = false;\n                            let mut has_href_css = false;\n\n                            for (attr, value) in attributes {\n                                if let Some(value) =\n                                    value.as_deref().map(|v| v.trim()).filter(|v| !v.is_empty())\n                                {\n                                    if *attr == REL {\n                                        if value.to_ascii_lowercase().contains(\"stylesheet\") {\n                                            has_rel_style = true;\n                                        }\n                                    } else if *attr == HREF\n                                        && value.to_ascii_lowercase().contains(\".css\")\n                                    {\n                                        has_href_css = true;\n                                    }\n                                }\n                            }\n\n                            if has_rel_style || has_href_css {\n                                // Has external CSS\n                                ctx.result.add_tag(\"EXT_CSS\");\n                            }\n                        }\n                        HEAD if !*is_self_closing => {\n                            in_head += 1;\n                        }\n                        BODY if !*is_self_closing => {\n                            in_body += 1;\n                        }\n                        _ => {}\n                    },\n                    HtmlToken::EndTag { name } => match *name {\n                        A => {\n                            last_href = None;\n                        }\n                        HEAD => {\n                            in_head -= 1;\n                        }\n                        BODY => {\n                            in_body -= 1;\n                        }\n                        _ => (),\n                    },\n                    HtmlToken::Text { text } if in_head == 0 => {\n                        if let Some((href_url, href_host)) = last_href\n                            .as_ref()\n                            .and_then(|href| Some((href.url_parsed.as_ref()?, href.host.as_ref()?)))\n                        {\n                            for token in TypesTokenizer::new(text.as_ref())\n                                .tokenize_numbers(false)\n                                .tokenize_urls(true)\n                                .tokenize_urls_without_scheme(true)\n                                .tokenize_emails(true)\n                            {\n                                let text_url = match token.word {\n                                    TokenType::Url(url) => url.to_lowercase(),\n                                    TokenType::UrlNoScheme(url) => {\n                                        format!(\"http://{}\", url.to_lowercase())\n                                    }\n                                    _ => continue,\n                                };\n                                let text_url_parsed =\n                                    if let Ok(text_url_parsed) = text_url.parse::<Uri>() {\n                                        text_url_parsed\n                                    } else {\n                                        continue;\n                                    };\n\n                                if href_url.scheme().map(|s| s.as_str()).unwrap_or_default()\n                                    == \"http\"\n                                    && text_url_parsed\n                                        .scheme()\n                                        .map(|s| s.as_str())\n                                        .unwrap_or_default()\n                                        == \"https\"\n                                {\n                                    // The anchor text contains a distinct scheme compared to the target URL\n                                    ctx.result.add_tag(\"HTTP_TO_HTTPS\");\n                                }\n\n                                if let Some(text_url_host) = text_url_parsed.host() {\n                                    let text_url_host = Hostname::new(text_url_host);\n\n                                    if text_url_host.sld_or_default() != href_host.sld_or_default()\n                                    {\n                                        // The anchor text contains a different domain than the target URL\n                                        ctx.result.add_tag(\"PHISHING\");\n                                    }\n                                }\n                            }\n                        }\n                    }\n                    _ => (),\n                }\n            }\n\n            if is_body_part {\n                if in_head != 0 || in_body != 0 {\n                    // HTML tags are not properly closed\n                    ctx.result.add_tag(\"HTML_UNBALANCED_TAG\");\n                }\n\n                let mut html_words = 0;\n                let mut html_uris = 0;\n                let mut html_text_chars = 0;\n\n                for token in tokens {\n                    match token {\n                        TokenType::Alphabetic(s) | TokenType::Alphanumeric(s) => {\n                            html_words += 1;\n                            html_text_chars += s.len();\n                        }\n                        TokenType::Email(s) => {\n                            html_words += 1;\n                            html_text_chars += s.address.len();\n                        }\n                        TokenType::Url(_) | TokenType::UrlNoScheme(_) => {\n                            html_uris += 1;\n                        }\n                        _ => (),\n                    }\n                }\n\n                match html_text_chars {\n                    0..1024 => {\n                        ctx.result.add_tag(\"HTML_SHORT_1\");\n                    }\n                    1024..1536 => {\n                        ctx.result.add_tag(\"HTML_SHORT_2\");\n                    }\n                    1536..2048 => {\n                        ctx.result.add_tag(\"HTML_SHORT_3\");\n                    }\n                    _ => (),\n                }\n\n                if (!has_link_to_img || html_text_chars >= 2048)\n                    && (html_img_words as f64 / (html_words as f64 + html_img_words as f64) > 0.5)\n                {\n                    // Message contains more images than text\n                    ctx.result.add_tag(\"HTML_TEXT_IMG_RATIO\");\n                }\n\n                if html_uris > 0 && html_words == 0 {\n                    // Message only contains URIs in HTML\n                    ctx.result.add_tag(\"BODY_URI_ONLY\");\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/init.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Server;\n\nuse mail_auth::DmarcResult;\nuse mail_parser::{HeaderName, PartType, parsers::fields::thread::thread_name};\nuse nlp::tokenizers::types::{TokenType, TypesTokenizer};\n\nuse crate::{\n    Email, Hostname, IpParts, Recipient, SpamFilterContext, SpamFilterInput, SpamFilterOutput,\n    SpamFilterResult, TextPart,\n    modules::html::{HEAD, HtmlToken, html_to_tokens},\n};\n\nuse super::url::UrlParts;\n\npub trait SpamFilterInit {\n    fn spam_filter_init<'x>(&self, input: SpamFilterInput<'x>) -> SpamFilterContext<'x>;\n}\n\nconst POSTMASTER_ADDRESSES: [&str; 3] = [\"postmaster\", \"mailer-daemon\", \"root\"];\n\nimpl SpamFilterInit for Server {\n    fn spam_filter_init<'x>(&self, mut input: SpamFilterInput<'x>) -> SpamFilterContext<'x> {\n        let mut subject = \"\";\n        let mut from = None;\n        let mut reply_to = None;\n        let mut recipients_to = Vec::new();\n        let mut recipients_cc = Vec::new();\n        let mut recipients_bcc = Vec::new();\n        let mut found_spam_status = false;\n\n        for header in input.message.headers() {\n            match &header.name {\n                HeaderName::To | HeaderName::Cc | HeaderName::Bcc => {\n                    if let Some(addrs) = header.value().as_address() {\n                        for addr in addrs.iter() {\n                            let rcpt = Recipient {\n                                email: Email::new(addr.address().unwrap_or_default()),\n                                name: addr.name().and_then(|s| {\n                                    let s = s.trim();\n                                    if !s.is_empty() {\n                                        Some(s.to_lowercase())\n                                    } else {\n                                        None\n                                    }\n                                }),\n                            };\n                            if header.name == HeaderName::To {\n                                recipients_to.push(rcpt);\n                            } else if header.name == HeaderName::Cc {\n                                recipients_cc.push(rcpt);\n                            } else {\n                                recipients_bcc.push(rcpt);\n                            }\n                        }\n                    }\n                }\n                HeaderName::ReplyTo => {\n                    reply_to = header\n                        .value()\n                        .as_address()\n                        .and_then(|addrs| addrs.first())\n                        .and_then(|addr| {\n                            Some(Recipient {\n                                email: Email::new(addr.address()?),\n                                name: addr.name().and_then(|s| {\n                                    let s = s.trim();\n                                    if !s.is_empty() {\n                                        Some(s.to_lowercase())\n                                    } else {\n                                        None\n                                    }\n                                }),\n                            })\n                        });\n                }\n                HeaderName::Subject => {\n                    subject = header.value().as_text().unwrap_or_default();\n                }\n                HeaderName::From => {\n                    from = header.value().as_address().and_then(|addrs| addrs.first());\n                }\n                HeaderName::Other(name)\n                    if input.is_train && !found_spam_status && name.eq(\"X-Spam-Result\") =>\n                {\n                    for token in header\n                        .value()\n                        .as_text()\n                        .unwrap_or_default()\n                        .split_ascii_whitespace()\n                    {\n                        if let Some(dmarc) = token.strip_prefix(\"DMARC_\") {\n                            input.dmarc_result = if dmarc == \"POLICY_ALLOW\" {\n                                Some(&DmarcResult::Pass)\n                            } else {\n                                Some(&DmarcResult::None)\n                            };\n                        } else if let Some(asn) = token\n                            .strip_prefix(\"SOURCE_ASN_\")\n                            .and_then(|v| v.parse().ok())\n                        {\n                            input.asn = Some(asn);\n                        }\n                    }\n\n                    found_spam_status = true;\n                }\n                _ => {}\n            }\n        }\n\n        // Tokenize subject\n        let subject_tokens = TypesTokenizer::new(subject)\n            .tokenize_numbers(false)\n            .tokenize_urls(true)\n            .tokenize_urls_without_scheme(true)\n            .tokenize_emails(true)\n            .map(|t| match t.word {\n                TokenType::Alphabetic(s) => TokenType::Alphabetic(s.into()),\n                TokenType::Alphanumeric(s) => TokenType::Alphanumeric(s.into()),\n                TokenType::Integer(s) => TokenType::Integer(s.into()),\n                TokenType::Other(s) => TokenType::Other(s),\n                TokenType::Punctuation(s) => TokenType::Punctuation(s),\n                TokenType::Space => TokenType::Space,\n                TokenType::Url(url) => TokenType::Url(UrlParts::new(url)),\n                TokenType::UrlNoHost(s) => TokenType::UrlNoHost(s.into()),\n                TokenType::UrlNoScheme(s) => {\n                    TokenType::UrlNoScheme(UrlParts::new(format!(\"https://{}\", s.trim())))\n                }\n                TokenType::IpAddr(i) => TokenType::IpAddr(IpParts::new(i)),\n                TokenType::Email(e) => TokenType::Email(Email::new(e)),\n                TokenType::Float(s) => TokenType::Float(s.into()),\n            })\n            .collect::<Vec<_>>();\n\n        // Tokenize and convert text parts\n        let mut text_parts = Vec::new();\n        let mut text_parts_nested = Vec::new();\n        let mut message_stack = Vec::new();\n        let mut message_iter = input.message.parts.iter();\n\n        loop {\n            while let Some(part) = message_iter.next() {\n                let is_main_message = message_stack.is_empty();\n                let text_part = match &part.body {\n                    PartType::Text(text) => TextPart::Plain {\n                        text_body: text.as_ref(),\n                        tokens: TypesTokenizer::new(text.as_ref())\n                            .tokenize_numbers(false)\n                            .tokenize_urls(true)\n                            .tokenize_urls_without_scheme(true)\n                            .tokenize_emails(true)\n                            .map(|t| match t.word {\n                                TokenType::Alphabetic(s) => TokenType::Alphabetic(s.into()),\n                                TokenType::Alphanumeric(s) => TokenType::Alphanumeric(s.into()),\n                                TokenType::Integer(s) => TokenType::Integer(s.into()),\n                                TokenType::Other(s) => TokenType::Other(s),\n                                TokenType::Punctuation(s) => TokenType::Punctuation(s),\n                                TokenType::Space => TokenType::Space,\n                                TokenType::Url(url) => TokenType::Url(UrlParts::new(url)),\n                                TokenType::UrlNoHost(s) => TokenType::UrlNoHost(s.into()),\n                                TokenType::UrlNoScheme(s) => TokenType::UrlNoScheme(UrlParts::new(\n                                    format!(\"https://{}\", s.trim()),\n                                )),\n                                TokenType::IpAddr(i) => TokenType::IpAddr(IpParts::new(i)),\n                                TokenType::Email(e) => TokenType::Email(Email::new(e)),\n                                TokenType::Float(s) => TokenType::Float(s.into()),\n                            })\n                            .collect::<Vec<_>>(),\n                    },\n                    PartType::Html(html) => {\n                        let html_tokens = html_to_tokens(html);\n                        let text_body_len = html_tokens\n                            .iter()\n                            .filter_map(|t| match t {\n                                HtmlToken::Text { text } => text.len().into(),\n                                _ => None,\n                            })\n                            .sum();\n                        let mut text_body = String::with_capacity(text_body_len);\n                        let mut in_head = false;\n                        for token in &html_tokens {\n                            match token {\n                                HtmlToken::StartTag { name: HEAD, .. } => {\n                                    in_head = true;\n                                }\n                                HtmlToken::EndTag { name: HEAD } => {\n                                    in_head = false;\n                                }\n                                HtmlToken::Text { text } if !in_head => {\n                                    if !text_body.is_empty()\n                                        && !text_body.ends_with(' ')\n                                        && !text.starts_with(' ')\n                                    {\n                                        text_body.push(' ');\n                                    }\n                                    text_body.push_str(text)\n                                }\n                                _ => {}\n                            }\n                        }\n\n                        TextPart::Html {\n                            tokens: TypesTokenizer::new(&text_body)\n                                .tokenize_numbers(false)\n                                .tokenize_urls(true)\n                                .tokenize_urls_without_scheme(true)\n                                .tokenize_emails(true)\n                                .map(|t| match t.word {\n                                    TokenType::Alphabetic(s) => {\n                                        TokenType::Alphabetic(s.to_string().into())\n                                    }\n                                    TokenType::Alphanumeric(s) => {\n                                        TokenType::Alphanumeric(s.to_string().into())\n                                    }\n                                    TokenType::Integer(s) => {\n                                        TokenType::Integer(s.to_string().into())\n                                    }\n                                    TokenType::Other(s) => TokenType::Other(s),\n                                    TokenType::Punctuation(s) => TokenType::Punctuation(s),\n                                    TokenType::Space => TokenType::Space,\n                                    TokenType::Url(url) => {\n                                        TokenType::Url(UrlParts::new(url.to_string()))\n                                    }\n                                    TokenType::UrlNoHost(s) => {\n                                        TokenType::UrlNoHost(s.to_string().into())\n                                    }\n                                    TokenType::UrlNoScheme(s) => TokenType::UrlNoScheme(\n                                        UrlParts::new(format!(\"https://{}\", s.trim())),\n                                    ),\n                                    TokenType::IpAddr(i) => TokenType::IpAddr(IpParts::new(i)),\n                                    TokenType::Email(e) => TokenType::Email(Email::new(e)),\n                                    TokenType::Float(s) => TokenType::Float(s.to_string().into()),\n                                })\n                                .collect::<Vec<_>>(),\n                            html_tokens,\n                            text_body,\n                        }\n                    }\n                    PartType::Message(message) => {\n                        message_stack.push(message_iter);\n                        message_iter = message.parts.iter();\n                        TextPart::None\n                    }\n                    _ => TextPart::None,\n                };\n\n                if is_main_message {\n                    text_parts.push(text_part);\n                } else if !matches!(text_part, TextPart::None) {\n                    text_parts_nested.push(text_part);\n                }\n            }\n\n            if let Some(iter) = message_stack.pop() {\n                message_iter = iter;\n            } else {\n                break;\n            }\n        }\n        text_parts.extend(text_parts_nested);\n\n        let subject_thread = thread_name(subject).to_string();\n        let env_from_addr = Email::new(input.env_from);\n        SpamFilterContext {\n            output: SpamFilterOutput {\n                ehlo_host: Hostname::new(input.ehlo_domain.unwrap_or(\"unknown\")),\n                iprev_ptr: input.iprev_result.and_then(|r| {\n                    r.ptr\n                        .as_ref()\n                        .and_then(|ptr| ptr.first())\n                        .map(|ptr| (ptr.strip_suffix('.').unwrap_or(ptr)).to_lowercase())\n                }),\n                env_from_postmaster: env_from_addr.address.is_empty()\n                    || POSTMASTER_ADDRESSES.contains(&env_from_addr.local_part.as_str()),\n                env_from_addr,\n                env_to_addr: input\n                    .env_rcpt_to\n                    .iter()\n                    .map(|rcpt| Email::new(rcpt))\n                    .collect(),\n                from: Recipient {\n                    email: Email::new(from.and_then(|f| f.address()).unwrap_or_default()),\n                    name: from.and_then(|f| f.name()).map(|name| name.to_lowercase()),\n                },\n                reply_to,\n                subject_thread_lc: subject_thread.trim().to_lowercase(),\n                subject_thread,\n                subject_lc: subject.trim().to_lowercase(),\n                subject: subject.to_string(),\n                subject_tokens,\n                recipients_to,\n                recipients_cc,\n                recipients_bcc,\n                text_parts,\n                ips: Default::default(),\n                emails: Default::default(),\n                urls: Default::default(),\n                domains: Default::default(),\n            },\n            input,\n            result: SpamFilterResult::default(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/ip.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ElementLocation;\nuse crate::{IpParts, SpamFilterContext, TextPart, modules::dnsbl::check_dnsbl};\nuse common::{\n    Server,\n    config::spamfilter::{Element, IpResolver, Location},\n};\nuse mail_auth::IprevResult;\nuse mail_parser::{HeaderName, HeaderValue, Host};\nuse nlp::tokenizers::types::TokenType;\nuse std::future::Future;\nuse store::ahash::AHashSet;\n\npub trait SpamFilterAnalyzeIp: Sync + Send {\n    fn spam_filter_analyze_ip(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeIp for Server {\n    async fn spam_filter_analyze_ip(&self, ctx: &mut SpamFilterContext<'_>) {\n        // IP Address RBL\n        let mut ips = AHashSet::new();\n\n        ips.insert(ElementLocation::new(ctx.input.remote_ip, Location::Tcp));\n\n        // Obtain IP addresses from Received headers\n        for header in ctx.input.message.headers() {\n            if let (HeaderName::Received, HeaderValue::Received(received)) =\n                (&header.name, &header.value)\n            {\n                if let Some(ip) = received.from_ip()\n                    && !ip.is_loopback()\n                    && !self.is_ip_allowed(&ip)\n                {\n                    ips.insert(ElementLocation::new(ip, Location::HeaderReceived));\n                }\n                for host in [&received.from, &received.helo, &received.by]\n                    .into_iter()\n                    .flatten()\n                {\n                    if let Host::IpAddr(ip) = host\n                        && !ip.is_loopback()\n                        && !self.is_ip_allowed(ip)\n                    {\n                        ips.insert(ElementLocation::new(*ip, Location::HeaderReceived));\n                    }\n                }\n            }\n        }\n\n        // Obtain IP addresses from the message body\n        for (part_id, part) in ctx.output.text_parts.iter().enumerate() {\n            let part_id = part_id as u32;\n            let is_body = ctx.input.message.text_body.contains(&part_id)\n                || ctx.input.message.html_body.contains(&part_id);\n            match part {\n                TextPart::Plain { tokens, .. } | TextPart::Html { tokens, .. } => {\n                    ips.extend(tokens.iter().filter_map(|t| {\n                        if let TokenType::IpAddr(ip) = t {\n                            ip.ip.map(|ip| {\n                                ElementLocation::new(\n                                    ip,\n                                    if is_body {\n                                        Location::BodyText\n                                    } else {\n                                        Location::Attachment\n                                    },\n                                )\n                            })\n                        } else {\n                            None\n                        }\n                    }))\n                }\n\n                TextPart::None => (),\n            }\n        }\n\n        // Validate IP addresses\n        for ip in &ips {\n            if ip.element.is_loopback()\n                || ip.element.is_multicast()\n                || ip.element.is_unspecified()\n                || self.is_ip_allowed(&ip.element)\n            {\n                continue;\n            } else if self.is_ip_blocked(&ip.element) {\n                ctx.result.add_tag(\"IP_BLOCKED\");\n                continue;\n            }\n\n            check_dnsbl(\n                self,\n                ctx,\n                &IpResolver::new(ip.element),\n                Element::Ip,\n                ip.location,\n            )\n            .await;\n        }\n        ctx.output.ips = ips;\n\n        // Reverse DNS validation\n        if let Some(iprev) = ctx.input.iprev_result {\n            match &iprev.result {\n                IprevResult::TempError(_) => ctx.result.add_tag(\"RDNS_DNSFAIL\"),\n                IprevResult::Fail(_) | IprevResult::PermError(_) => ctx.result.add_tag(\"RDNS_NONE\"),\n                IprevResult::Pass | IprevResult::None => (),\n            }\n        }\n\n        // Add ASN\n        if let Some(asn_id) = &ctx.input.asn {\n            ctx.result.add_tag(format!(\"SOURCE_ASN_{asn_id}\"));\n        }\n    }\n}\n\nimpl IpParts {\n    pub fn new(text: &str) -> IpParts {\n        IpParts {\n            ip: text.parse().ok(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/llm.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n */\n\nuse std::{future::Future, time::Instant};\n\nuse common::Server;\nuse trc::AiEvent;\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeLlm: Sync + Send {\n    fn spam_filter_analyze_llm(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeLlm for Server {\n    async fn spam_filter_analyze_llm(&self, ctx: &mut SpamFilterContext<'_>) {\n        if let Some(config) = self\n            .core\n            .enterprise\n            .as_ref()\n            .and_then(|c| c.spam_filter_llm.as_ref())\n        {\n            let time = Instant::now();\n            let body = if let Some(body) = ctx.text_body() {\n                body\n            } else {\n                return;\n            };\n            let prompt = format!(\n                \"{}\\n\\nSubject: {}\\n\\n{}\",\n                config.prompt, ctx.output.subject, body\n            );\n\n            match config\n                .model\n                .send_request(prompt, config.temperature.into())\n                .await\n            {\n                Ok(response) => {\n                    trc::event!(\n                        Ai(AiEvent::LlmResponse),\n                        Id = config.model.id.clone(),\n                        Details = response.clone(),\n                        Elapsed = time.elapsed(),\n                        SpanId = ctx.input.span_id,\n                    );\n\n                    let mut category = None;\n                    let mut confidence = None;\n                    let mut explanation = None;\n\n                    for (idx, value) in response.split(config.separator).enumerate() {\n                        let value = value.trim();\n                        if !value.is_empty() {\n                            if idx == config.index_category {\n                                let value = value.to_uppercase();\n                                if config.categories.contains(value.as_str()) {\n                                    category = Some(value);\n                                }\n                            } else if config.index_confidence.is_some_and(|i| i == idx) {\n                                let value = value.to_uppercase();\n                                if config.confidence.contains(value.as_str()) {\n                                    confidence = Some(value);\n                                }\n                            } else if config.index_explanation.is_some_and(|i| i == idx) {\n                                let explanation = explanation.get_or_insert_with(|| {\n                                    String::with_capacity(std::cmp::min(value.len(), 255))\n                                });\n\n                                for value in value.chars() {\n                                    if !value.is_whitespace() {\n                                        explanation.push(value);\n                                    } else {\n                                        explanation.push(' ');\n                                    }\n                                    if explanation.len() == 255 {\n                                        break;\n                                    }\n                                }\n                            }\n                        }\n                    }\n\n                    let category = match (category, confidence) {\n                        (Some(category), Some(confidence)) => {\n                            ctx.result.add_tag(format!(\"LLM_{category}_{confidence}\"));\n                            category\n                        }\n                        (Some(category), None) => {\n                            ctx.result.add_tag(format!(\"LLM_{category}\"));\n                            category\n                        }\n                        _ => return,\n                    };\n\n                    if let Some(explanation) = explanation {\n                        ctx.result.llm_result = Some((category, explanation));\n                    }\n                }\n                Err(err) => {\n                    trc::error!(err.span_id(ctx.input.span_id));\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/messageid.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse mail_parser::HeaderName;\n\nuse crate::{Hostname, SpamFilterContext};\n\npub trait SpamFilterAnalyzeMid: Sync + Send {\n    fn spam_filter_analyze_message_id(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeMid for Server {\n    async fn spam_filter_analyze_message_id(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut mid = \"\";\n        let mut mid_raw = \"\";\n\n        for header in ctx.input.message.headers() {\n            if let (HeaderName::MessageId, value) = (&header.name, &header.value) {\n                mid = value.as_text().unwrap_or_default();\n                mid_raw = std::str::from_utf8(\n                    &ctx.input.message.raw_message()\n                        [header.offset_start as usize..header.offset_end as usize],\n                )\n                .unwrap_or_default()\n                .trim();\n                break;\n            }\n        }\n\n        if !mid.is_empty() {\n            let mid = mid.to_lowercase();\n            if let Some(mid_host) = mid.rsplit_once('@').map(|(_, host)| Hostname::new(host)) {\n                if mid_host.ip.is_some() {\n                    if mid_host.fqdn.starts_with('[') {\n                        ctx.result.add_tag(\"MID_RHS_IP_LITERAL\");\n                    } else {\n                        ctx.result.add_tag(\"MID_BARE_IP\");\n                    }\n                } else if !mid_host.fqdn.contains('.') {\n                    ctx.result.add_tag(\"MID_RHS_NOT_FQDN\");\n                } else if mid_host.fqdn.starts_with(\"www.\") {\n                    ctx.result.add_tag(\"MID_RHS_WWW\");\n                }\n\n                if !mid_raw.is_ascii() || mid_raw.contains('(') || mid.starts_with('@') {\n                    ctx.result.add_tag(\"INVALID_MSGID\");\n                }\n\n                if mid_host.fqdn.len() > 255 {\n                    ctx.result.add_tag(\"MID_RHS_TOO_LONG\");\n                }\n\n                // From address present in Message-ID checks\n                for (part, sender) in [\n                    (\"FROM\", &ctx.output.from.email),\n                    (\"ENV_FROM\", &ctx.output.env_from_addr),\n                ] {\n                    if !sender.address.is_empty() {\n                        if mid.contains(sender.address.as_str()) {\n                            ctx.result.add_tag(format!(\"MID_CONTAINS_{part}\"));\n                        } else if mid_host.fqdn == sender.domain_part.fqdn {\n                            ctx.result.add_tag(format!(\"MID_RHS_MATCH_{part}\"));\n                        } else if matches!((&mid_host.sld, &sender.domain_part.sld), (Some(mid_sld), Some(sender_sld)) if mid_sld == sender_sld)\n                        {\n                            ctx.result.add_tag(format!(\"MID_RHS_MATCH_{part}TLD\"));\n                        }\n                    }\n                }\n\n                // To/Cc addresses present in Message-ID checks\n                for rcpt in ctx.output.all_recipients() {\n                    if mid.contains(rcpt.email.address.as_str()) {\n                        ctx.result.add_tag(\"MID_CONTAINS_TO\");\n                    } else if mid_host.fqdn == rcpt.email.domain_part.fqdn {\n                        ctx.result.add_tag(\"MID_RHS_MATCH_TO\");\n                    }\n                }\n            } else {\n                ctx.result.add_tag(\"INVALID_MSGID\");\n            }\n\n            if !mid_raw.starts_with('<') || !mid_raw.contains('>') {\n                ctx.result.add_tag(\"MID_MISSING_BRACKETS\");\n            }\n        } else {\n            ctx.result.add_tag(\"MISSING_MID\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/mime.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{collections::HashSet, future::Future, vec};\n\nuse common::{\n    Server,\n    scripts::{\n        IsMixedCharset,\n        functions::{array::cosine_similarity, unicode::CharUtils},\n    },\n};\nuse mail_parser::{HeaderName, MimeHeaders, PartType};\nuse nlp::tokenizers::types::TokenType;\n\nuse crate::{SpamFilterContext, TextPart};\n\npub trait SpamFilterAnalyzeMime: Sync + Send {\n    fn spam_filter_analyze_mime(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeMime for Server {\n    async fn spam_filter_analyze_mime(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut has_mime_version = false;\n        let mut has_ct = false;\n        let mut has_cte = false;\n        let mut had_cd = false;\n        let mut is_plain_text = false;\n\n        for header in ctx.input.message.headers() {\n            match &header.name {\n                HeaderName::MimeVersion => {\n                    if ctx\n                        .input\n                        .message\n                        .raw_message()\n                        .get(header.offset_field as usize..header.offset_start as usize - 1)\n                        != Some(b\"MIME-Version\")\n                    {\n                        ctx.result.add_tag(\"MV_CASE\");\n                    }\n                    has_mime_version = true;\n                }\n                HeaderName::ContentType => {\n                    has_ct = true;\n\n                    if let Some(ct) = header.value().as_content_type() {\n                        if ct.ctype().eq_ignore_ascii_case(\"multipart\")\n                            && ct\n                                .subtype()\n                                .is_some_and(|s| s.eq_ignore_ascii_case(\"report\"))\n                            && ct.attribute(\"report-type\").is_some_and(|a| {\n                                a.eq_ignore_ascii_case(\"delivery-status\")\n                                    || a.eq_ignore_ascii_case(\"disposition-notification\")\n                            })\n                        {\n                            // Message is a DSN\n                            ctx.result.add_tag(\"IS_DSN\");\n                        }\n\n                        is_plain_text = ct.ctype().eq_ignore_ascii_case(\"text\")\n                            && ct\n                                .subtype()\n                                .unwrap_or_default()\n                                .eq_ignore_ascii_case(\"plain\");\n                    }\n                }\n                HeaderName::ContentTransferEncoding => {\n                    has_cte = true;\n                }\n                HeaderName::ContentDisposition => {\n                    had_cd = true;\n                }\n                _ => (),\n            }\n        }\n\n        if !has_mime_version && (has_ct || has_cte) {\n            ctx.result.add_tag(\"MISSING_MIME_VERSION\");\n        }\n        if has_ct && !is_plain_text && !has_cte && !had_cd && !has_mime_version {\n            // Only Content-Type header without other MIME headers\n            ctx.result.add_tag(\"MIME_HEADER_CTYPE_ONLY\");\n        }\n        let raw_message = ctx.input.message.raw_message();\n\n        let mut has_text_part = false;\n        let mut is_encrypted = false;\n        let mut is_encrypted_smime = false;\n        let mut is_encrypted_pgp = false;\n\n        let mut num_parts = 0;\n        let mut num_parts_size = 0;\n\n        for (part_id, part) in ctx.input.message.parts.iter().enumerate() {\n            let part_id = part_id as u32;\n            let mut ct = None;\n            let mut cd = None;\n            let mut ct_type = String::new();\n            let mut ct_subtype = String::new();\n            let mut cte = String::new();\n            let mut is_attachment = ctx.input.message.attachments.contains(&part_id);\n            let mut has_content_id = false;\n\n            for header in part.headers() {\n                match &header.name {\n                    HeaderName::ContentType => {\n                        if let Some(ct_) = header.value().as_content_type() {\n                            ct_type = ct_.ctype().to_ascii_lowercase();\n                            ct_subtype = ct_.subtype().unwrap_or_default().to_ascii_lowercase();\n                            ct = Some(ct_);\n                        }\n\n                        if ct_type.is_empty() {\n                            // Content-Type header can't be parsed\n                            ctx.result.add_tag(\"BROKEN_CONTENT_TYPE\");\n                        } else if (ct_type == \"message\" && ct_subtype == \"rfc822\")\n                            || (ct_type == \"text\" && ct_subtype == \"rfc822-headers\")\n                        {\n                            // Message has parts\n                            ctx.result.add_tag(\"HAS_MESSAGE_PARTS\");\n                        }\n\n                        if raw_message\n                            .get(header.offset_start as usize..header.offset_end as usize)\n                            .and_then(|s| s.trim_ascii_end().last())\n                            == Some(&b';')\n                        {\n                            // Content-Type header ends with a semi-colon\n                            ctx.result.add_tag(\"CT_EXTRA_SEMI\");\n                        }\n                    }\n                    HeaderName::ContentTransferEncoding => {\n                        let cte_ = header.value().as_text().unwrap_or_default();\n                        cte = cte_.to_ascii_lowercase();\n\n                        if cte != cte_ {\n                            ctx.result.add_tag(\"CTE_CASE\");\n                        }\n                    }\n                    HeaderName::ContentDisposition => {\n                        cd = header.value().as_content_type();\n                    }\n                    HeaderName::ContentId => {\n                        has_content_id = true;\n                    }\n                    _ => (),\n                }\n            }\n\n            match ct_type.as_str() {\n                \"multipart\" => {\n                    let part_ids = match &part.body {\n                        PartType::Multipart(parts) => parts.as_slice(),\n                        _ => &[],\n                    };\n\n                    match ct_subtype.as_str() {\n                        \"alternative\" => {\n                            let mut has_plain_part = false;\n                            let mut has_html_part = false;\n\n                            let mut text_part_words = vec![];\n                            let mut text_part_uris = 0;\n\n                            let mut html_part_words = vec![];\n                            let mut html_part_uris = 0;\n\n                            for text_part in part_ids\n                                .iter()\n                                .map(|id| &ctx.output.text_parts[*id as usize])\n                            {\n                                let (tokens, words, uri_count) = match text_part {\n                                    TextPart::Plain { tokens, .. } if !has_plain_part => {\n                                        has_plain_part = true;\n                                        (tokens, &mut text_part_words, &mut text_part_uris)\n                                    }\n                                    TextPart::Html { tokens, .. } if !has_html_part => {\n                                        has_html_part = true;\n                                        (tokens, &mut html_part_words, &mut html_part_uris)\n                                    }\n                                    _ => continue,\n                                };\n\n                                let mut uris = HashSet::new();\n                                for token in tokens {\n                                    match token {\n                                        TokenType::Alphabetic(v) | TokenType::Alphanumeric(v) => {\n                                            words.push(v.as_ref());\n                                        }\n                                        TokenType::Url(v) => {\n                                            if let Some(host) =\n                                                v.url_parsed.as_ref().map(|uri| &uri.host)\n                                            {\n                                                uris.insert(host.sld_or_default());\n                                            }\n                                        }\n                                        _ => (),\n                                    }\n                                }\n\n                                *uri_count = uris.len();\n                            }\n\n                            //  Multipart message mostly text/html MIME\n                            if has_html_part {\n                                if !has_plain_part {\n                                    ctx.result.add_tag(\"MIME_MA_MISSING_TEXT\");\n                                }\n                            } else if has_plain_part {\n                                ctx.result.add_tag(\"MIME_MA_MISSING_HTML\");\n                            }\n\n                            // HTML and text parts are different\n                            if has_plain_part\n                                && has_html_part\n                                && (!text_part_words.is_empty() || !html_part_words.is_empty())\n                                && cosine_similarity(&text_part_words, &html_part_words) < 0.95\n                            {\n                                ctx.result.add_tag(\"PARTS_DIFFER\");\n                            }\n\n                            // Odd URI count between parts\n                            if text_part_uris != html_part_uris {\n                                ctx.result.add_tag(\"URI_COUNT_ODD\");\n                            }\n                        }\n                        \"mixed\" => {\n                            let mut num_text_parts = 0;\n                            let mut has_other_parts = false;\n\n                            for (sub_part_id, sub_part) in part_ids\n                                .iter()\n                                .map(|id| (*id, &ctx.input.message.parts[*id as usize]))\n                            {\n                                let ctype = sub_part\n                                    .content_type()\n                                    .map(|ct| ct.ctype())\n                                    .unwrap_or_default();\n\n                                if ctype.eq_ignore_ascii_case(\"text\")\n                                    && !ctx.input.message.attachments.contains(&sub_part_id)\n                                {\n                                    num_text_parts += 1;\n                                } else if !ctype.eq_ignore_ascii_case(\"multipart\") {\n                                    has_other_parts = true;\n                                }\n                            }\n\n                            // Found multipart/mixed without non-textual part\n                            if !has_other_parts && num_text_parts < 3 {\n                                ctx.result.add_tag(\"CTYPE_MIXED_BOGUS\");\n                            }\n                        }\n                        \"encrypted\" => {\n                            is_encrypted = true;\n                        }\n                        _ => (),\n                    }\n\n                    continue;\n                }\n                \"text\" => {\n                    let mut is_7bit = false;\n                    match cte.as_str() {\n                        \"\" | \"7bit\" => {\n                            if raw_message\n                                .get(\n                                    part.raw_body_offset() as usize..part.raw_end_offset() as usize,\n                                )\n                                .is_some_and(|bytes| !bytes.is_ascii())\n                            {\n                                // MIME text part claims to be ASCII but isn't\n                                ctx.result.add_tag(\"BAD_CTE_7BIT\");\n                            }\n                            is_7bit = true;\n                        }\n                        \"base64\" => {\n                            if part.contents().is_ascii() {\n                                // Has text part encoded in base64 that does not contain any 8bit characters\n                                ctx.result.add_tag(\"MIME_BASE64_TEXT_BOGUS\");\n                            } else {\n                                // Has text part encoded in base64\n                                ctx.result.add_tag(\"MIME_BASE64_TEXT\");\n                            }\n                        }\n                        _ => (),\n                    }\n\n                    if !is_7bit\n                        && ct_subtype == \"plain\"\n                        && ct\n                            .and_then(|ct| ct.attribute(\"charset\"))\n                            .is_none_or(|c| c.is_empty())\n                    {\n                        // Charset header is missing\n                        ctx.result.add_tag(\"MISSING_CHARSET\");\n                    }\n\n                    if ctx\n                        .output\n                        .text_parts\n                        .get(part_id as usize)\n                        .filter(|_| {\n                            ctx.input.message.text_body.contains(&part_id)\n                                || ctx.input.message.html_body.contains(&part_id)\n                        })\n                        .is_some_and(|p| match p {\n                            TextPart::Plain { text_body, .. } => text_body.is_mixed_charset(),\n                            TextPart::Html { text_body, .. } => text_body.is_mixed_charset(),\n                            TextPart::None => false,\n                        })\n                    {\n                        // Text part contains multiple scripts\n                        ctx.result.add_tag(\"MIXED_CHARSET\");\n                    }\n\n                    has_text_part = true;\n                }\n                \"application\" => match ct_subtype.as_str() {\n                    \"pkcs7-mime\" => {\n                        ctx.result.add_tag(\"ENCRYPTED_SMIME\");\n                        is_attachment = false;\n                        is_encrypted_smime = true;\n                    }\n                    \"pkcs7-signature\" => {\n                        ctx.result.add_tag(\"SIGNED_SMIME\");\n                        is_attachment = false;\n                    }\n                    \"pgp-encrypted\" => {\n                        ctx.result.add_tag(\"ENCRYPTED_PGP\");\n                        is_attachment = false;\n                        is_encrypted_pgp = true;\n                    }\n                    \"pgp-signature\" => {\n                        ctx.result.add_tag(\"SIGNED_PGP\");\n                        is_attachment = false;\n                    }\n                    \"octet-stream\" => {\n                        if !is_encrypted\n                            && !has_content_id\n                            && cd.is_none_or(|cd| {\n                                !cd.c_type.eq_ignore_ascii_case(\"attachment\")\n                                    && !cd.has_attribute(\"filename\")\n                            })\n                        {\n                            ctx.result.add_tag(\"CTYPE_MISSING_DISPOSITION\");\n                        }\n                    }\n                    _ => (),\n                },\n                _ => (),\n            }\n\n            num_parts += 1;\n            num_parts_size += part.len();\n\n            let ct_full = format!(\"{ct_type}/{ct_subtype}\");\n\n            if is_attachment {\n                // Has a MIME attachment\n                ctx.result.add_tag(\"HAS_ATTACHMENT\");\n                if ct_full != \"application/octet-stream\"\n                    && let Some(t) = infer::get(part.contents())\n                {\n                    if t.mime_type() == ct_full {\n                        // Known content-type\n                        ctx.result.add_tag(\"MIME_GOOD\");\n                    } else {\n                        // Known bad content-type\n                        ctx.result.add_tag(\"MIME_BAD\");\n                    }\n                }\n            }\n\n            // Analyze attachment name\n            if let Some(attach_name) = part.attachment_name() {\n                if attach_name.chars().any(|c| c.is_obscured()) {\n                    // Attachment name contains zero-width space\n                    ctx.result.add_tag(\"MIME_BAD_UNICODE\");\n                }\n                let attach_name = attach_name.trim().to_lowercase();\n                if let Some((name, ext)) = attach_name.rsplit_once('.').and_then(|(name, ext)| {\n                    Some((name, self.core.spam.lists.file_extensions.get(ext)?))\n                }) {\n                    let sub_ext = name\n                        .rsplit_once('.')\n                        .and_then(|(_, ext)| self.core.spam.lists.file_extensions.get(ext));\n\n                    if ext.is_bad {\n                        // Attachment has a bad extension\n                        if sub_ext.is_some_and(|e| e.is_bad) {\n                            ctx.result.add_tag(\"MIME_DOUBLE_BAD_EXTENSION\");\n                        } else {\n                            ctx.result.add_tag(\"MIME_BAD_EXTENSION\");\n                        }\n                    }\n\n                    if ext.is_archive && sub_ext.is_some_and(|e| e.is_archive) {\n                        // Archive in archive\n                        ctx.result.add_tag(\"MIME_ARCHIVE_IN_ARCHIVE\");\n                    }\n\n                    if !ext.known_types.is_empty()\n                        && ct_full != \"application/octet-stream\"\n                        && !ext.known_types.contains(&ct_full)\n                    {\n                        // Invalid attachment mime type\n                        ctx.result.add_tag(\"MIME_BAD_ATTACHMENT\");\n                    }\n                }\n            }\n        }\n\n        match num_parts_size {\n            0 => {\n                // Message contains no parts\n                ctx.result.add_tag(\"COMPLETELY_EMPTY\");\n            }\n            1..64 if num_parts == 1 => {\n                // Message contains only one short part\n                ctx.result.add_tag(\"SINGLE_SHORT_PART\");\n            }\n            _ => (),\n        }\n\n        if has_text_part && (is_encrypted_pgp || is_encrypted_smime) {\n            // Message contains both text and encrypted parts\n            ctx.result.add_tag(\"BOGUS_ENCRYPTED_AND_TEXT\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    Recipient, SpamFilterContext, SpamFilterInput, SpamFilterOutput, SpamFilterResult, TextPart,\n};\nuse common::{Server, config::spamfilter::Location};\nuse mail_parser::{Header, parsers::MessageStream};\nuse std::{\n    borrow::Cow,\n    hash::{Hash, Hasher},\n};\n\npub mod classifier;\npub mod date;\npub mod dmarc;\npub mod domain;\npub mod ehlo;\npub mod from;\npub mod headers;\npub mod html;\npub mod init;\npub mod ip;\npub mod messageid;\npub mod mime;\npub mod pyzor;\npub mod received;\npub mod recipient;\npub mod replyto;\npub mod rules;\npub mod score;\npub mod subject;\npub mod url;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\npub mod llm;\n// SPDX-SnippetEnd\n\nimpl SpamFilterInput<'_> {\n    pub fn header_as_address(&self, header: &Header<'_>) -> Option<Cow<'_, str>> {\n        self.message\n            .raw_message()\n            .get(header.offset_start as usize..header.offset_end as usize)\n            .map(|bytes| MessageStream::new(bytes).parse_address())\n            .and_then(|addr| addr.into_address())\n            .and_then(|addr| addr.into_list().into_iter().next())\n            .and_then(|addr| addr.address)\n    }\n}\n\nimpl SpamFilterOutput<'_> {\n    pub fn all_recipients(&self) -> impl Iterator<Item = &Recipient> {\n        self.recipients_to\n            .iter()\n            .chain(self.recipients_cc.iter())\n            .chain(self.recipients_bcc.iter())\n    }\n}\n\nimpl SpamFilterContext<'_> {\n    pub fn text_body(&self) -> Option<&str> {\n        self.input\n            .message\n            .text_body\n            .first()\n            .or_else(|| self.input.message.html_body.first())\n            .and_then(|idx| self.output.text_parts.get(*idx as usize))\n            .and_then(|part| match part {\n                TextPart::Plain { text_body, .. } => Some(*text_body),\n                TextPart::Html { text_body, .. } => Some(text_body.as_str()),\n                TextPart::None => None,\n            })\n    }\n}\n\nimpl SpamFilterResult {\n    pub fn add_tag(&mut self, tag: impl Into<String>) {\n        self.tags.insert(tag.into());\n    }\n\n    pub fn has_tag(&self, tag: impl AsRef<str>) -> bool {\n        self.tags.contains(tag.as_ref())\n    }\n}\n\n#[derive(Debug)]\npub struct ElementLocation<T> {\n    pub element: T,\n    pub location: Location,\n}\n\nimpl<T: Hash> Hash for ElementLocation<T> {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.element.hash(state);\n    }\n}\n\nimpl<T: PartialEq> PartialEq for ElementLocation<T> {\n    fn eq(&self, other: &Self) -> bool {\n        self.element.eq(&other.element)\n    }\n}\n\nimpl<T: Eq> Eq for ElementLocation<T> {}\n\nimpl<T> ElementLocation<T> {\n    pub fn new(element: T, location: impl Into<Location>) -> Self {\n        Self {\n            element,\n            location: location.into(),\n        }\n    }\n}\n\npub(crate) async fn is_trusted_domain(server: &Server, domain: &str, span_id: u64) -> bool {\n    if let Some(store) = server.core.storage.lookups.get(\"trusted-domains\") {\n        match store.key_exists(domain).await {\n            Ok(true) => return true,\n            Ok(false) => (),\n            Err(err) => {\n                trc::error!(err.span_id(span_id).caused_by(trc::location!()));\n            }\n        }\n    }\n\n    match server.core.storage.directory.is_local_domain(domain).await {\n        Ok(result) => result,\n        Err(err) => {\n            trc::error!(err.span_id(span_id).caused_by(trc::location!()));\n            false\n        }\n    }\n}\n\npub(crate) async fn is_url_redirector(server: &Server, url: &str, span_id: u64) -> bool {\n    if let Some(store) = server.core.storage.lookups.get(\"url-redirectors\") {\n        match store.key_exists(url).await {\n            Ok(result) => result,\n            Err(err) => {\n                trc::error!(err.span_id(span_id).caused_by(trc::location!()));\n                false\n            }\n        }\n    } else {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/pyzor.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{future::Future, time::Instant};\n\nuse common::Server;\n\nuse crate::{SpamFilterContext, modules::pyzor::pyzor_check};\n\npub trait SpamFilterAnalyzePyzor: Sync + Send {\n    fn spam_filter_analyze_pyzor(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzePyzor for Server {\n    async fn spam_filter_analyze_pyzor(&self, ctx: &mut SpamFilterContext<'_>) {\n        if let Some(config) = &self.core.spam.pyzor {\n            let time = Instant::now();\n            match pyzor_check(ctx.input.message, config).await {\n                Ok(Some(result)) => {\n                    let is_spam = result.code == 200\n                        && result.count > config.min_count\n                        && (result.wl_count < config.min_wl_count\n                            || (result.wl_count as f64 / result.count as f64) < config.ratio);\n                    if is_spam {\n                        ctx.result.add_tag(\"PYZOR\");\n                    }\n                    trc::event!(\n                        Spam(trc::SpamEvent::Pyzor),\n                        Result = is_spam,\n                        Details = vec![\n                            trc::Value::from(result.code),\n                            trc::Value::from(result.count),\n                            trc::Value::from(result.wl_count)\n                        ],\n                        SpanId = ctx.input.span_id,\n                        Elapsed = time.elapsed()\n                    );\n                }\n                Ok(None) => {}\n                Err(err) => {\n                    trc::error!(\n                        err.span_id(ctx.input.span_id)\n                            .ctx(trc::Key::Elapsed, time.elapsed())\n                    );\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/received.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse mail_parser::{HeaderName, Host};\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeReceived: Sync + Send {\n    fn spam_filter_analyze_received(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeReceived for Server {\n    async fn spam_filter_analyze_received(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut rcvd_count = 0;\n        let mut rcvd_from_ip = 0;\n        let mut tls_count = 0;\n\n        for header in ctx.input.message.headers() {\n            if let HeaderName::Received = &header.name {\n                if !ctx\n                    .input\n                    .message\n                    .raw_message()\n                    .get(header.offset_start as usize..header.offset_end as usize)\n                    .unwrap_or_default()\n                    .is_ascii()\n                {\n                    // Received headers have non-ASCII characters\n                    ctx.result.add_tag(\"RCVD_ILLEGAL_CHARS\");\n                }\n\n                if let Some(received) = header.value().as_received() {\n                    let helo_domain = received.from().or_else(|| received.helo());\n                    let ip_rev = received.from_iprev();\n\n                    if matches!(&helo_domain, Some(Host::Name(hostname)) if hostname.eq_ignore_ascii_case(\"user\"))\n                    {\n                        // HELO domain is \"user\"\n                        ctx.result.add_tag(\"RCVD_HELO_USER\");\n                    } else if let (Some(Host::Name(helo_domain)), Some(ip_rev)) =\n                        (helo_domain, ip_rev)\n                        && helo_domain.to_lowercase() != ip_rev.to_lowercase()\n                    {\n                        // HELO domain does not match PTR record\n                        ctx.result.add_tag(\"FORGED_RCVD_TRAIL\");\n                    }\n\n                    if let Some(delivered_for) = received.for_().map(|s| s.to_lowercase())\n                        && ctx\n                            .output\n                            .all_recipients()\n                            .any(|r| r.email.address == delivered_for)\n                    {\n                        // Recipient appears on Received trail\n                        ctx.result.add_tag(\"PREVIOUSLY_DELIVERED\");\n                    }\n\n                    if matches!(received.from, Some(Host::IpAddr(_))) {\n                        // Received from an IP address rather than a FQDN\n                        rcvd_from_ip += 1;\n                    }\n\n                    if received.tls_version().is_some() {\n                        // Received with TLS\n                        tls_count += 1;\n                    }\n                } else {\n                    // Received header is not RFC 5322 compliant\n                    ctx.result.add_tag(\"RCVD_UNPARSABLE\");\n                }\n\n                rcvd_count += 1;\n            }\n        }\n\n        if rcvd_from_ip >= 2 || (rcvd_from_ip == 1 && ctx.output.ehlo_host.ip.is_some()) {\n            // Has two or more Received headers containing bare IP addresses\n            ctx.result.add_tag(\"RCVD_DOUBLE_IP_SPAM\");\n        }\n\n        // Received from an authenticated user\n        if ctx.input.authenticated_as.is_some() {\n            ctx.result.add_tag(\"RCVD_VIA_SMTP_AUTH\");\n        }\n\n        // Received with TLS checks\n        if rcvd_count > 0 && rcvd_count == tls_count && ctx.input.is_tls {\n            ctx.result.add_tag(\"RCVD_TLS_ALL\");\n        } else if ctx.input.is_tls {\n            ctx.result.add_tag(\"RCVD_TLS_LAST\");\n        } else {\n            ctx.result.add_tag(\"RCVD_NO_TLS_LAST\");\n        }\n\n        match rcvd_count {\n            0 => {\n                ctx.result.add_tag(\"RCVD_COUNT_ZERO\");\n            }\n            1 => {\n                ctx.result.add_tag(\"RCVD_COUNT_ONE\");\n            }\n            2 => {\n                ctx.result.add_tag(\"RCVD_COUNT_TWO\");\n            }\n            3 => {\n                ctx.result.add_tag(\"RCVD_COUNT_THREE\");\n            }\n            4 | 5 => {\n                ctx.result.add_tag(\"RCVD_COUNT_FIVE\");\n            }\n            6 | 7 => {\n                ctx.result.add_tag(\"RCVD_COUNT_SEVEN\");\n            }\n            8..=12 => {\n                ctx.result.add_tag(\"RCVD_COUNT_TWELVE\");\n            }\n            _ => {}\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/recipient.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::{Server, scripts::functions::text::levenshtein_distance};\nuse mail_parser::HeaderName;\nuse smtp_proto::{MAIL_BODY_8BITMIME, MAIL_BODY_BINARYMIME, MAIL_SMTPUTF8};\nuse store::ahash::HashSet;\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeRecipient: Sync + Send {\n    fn spam_filter_analyze_recipient(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeRecipient for Server {\n    async fn spam_filter_analyze_recipient(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut to_raw = b\"\".as_slice();\n        let mut cc_raw = b\"\".as_slice();\n        let mut bcc_raw = b\"\".as_slice();\n        let mut has_list_unsubscribe = false;\n        let mut has_list_id = false;\n\n        for header in ctx.input.message.headers() {\n            match &header.name {\n                HeaderName::To | HeaderName::Cc | HeaderName::Bcc => {\n                    let raw = ctx\n                        .input\n                        .message\n                        .raw_message()\n                        .get(header.offset_start as usize..header.offset_end as usize)\n                        .unwrap_or_default();\n                    match header.name {\n                        HeaderName::To => to_raw = raw,\n                        HeaderName::Cc => cc_raw = raw,\n                        HeaderName::Bcc => bcc_raw = raw,\n                        _ => unreachable!(),\n                    }\n                }\n                HeaderName::ListUnsubscribe => {\n                    has_list_unsubscribe = true;\n                }\n                HeaderName::ListId => {\n                    has_list_id = true;\n                }\n                _ => {}\n            }\n        }\n\n        if to_raw.is_empty() {\n            ctx.result.add_tag(\"MISSING_TO\");\n        }\n\n        let to_raw_utf8 = std::str::from_utf8(to_raw);\n        let cc_raw_utf8 = std::str::from_utf8(cc_raw);\n        let bcc_raw_utf8 = std::str::from_utf8(bcc_raw);\n\n        for (raw, raw_utf8, recipients) in [\n            (to_raw, &to_raw_utf8, &ctx.output.recipients_to),\n            (cc_raw, &cc_raw_utf8, &ctx.output.recipients_cc),\n            (bcc_raw, &bcc_raw_utf8, &ctx.output.recipients_bcc),\n        ] {\n            if !raw.is_empty() {\n                // Validate non-ASCII characters in recipient headers\n                if !raw.is_ascii() {\n                    if (ctx.input.env_from_flags\n                        & (MAIL_SMTPUTF8 | MAIL_BODY_8BITMIME | MAIL_BODY_BINARYMIME))\n                        == 0\n                    {\n                        ctx.result.add_tag(\"TO_NEEDS_ENCODING\");\n                    }\n\n                    if raw_utf8.is_err() {\n                        ctx.result.add_tag(\"INVALID_TO_8BIT\");\n                    }\n                }\n\n                // Validate unnecessary encoding in recipient headers\n                let raw_utf8 = raw_utf8.unwrap_or_default();\n                if recipients.iter().all(|rcpt| {\n                    rcpt.name.as_ref().is_none_or(|name| name.is_ascii())\n                        && rcpt.email.address.is_ascii()\n                }) && raw_utf8.contains(\"=?\")\n                    && raw_utf8.contains(\"?=\")\n                {\n                    if raw_utf8.contains(\"?q?\") || raw_utf8.contains(\"?Q?\") {\n                        // To header is unnecessarily encoded in quoted-printable\n                        ctx.result.add_tag(\"TO_EXCESS_QP\");\n                    } else if raw_utf8.contains(\"?b?\") || raw_utf8.contains(\"?B?\") {\n                        // To header is unnecessarily encoded in base64\n                        ctx.result.add_tag(\"TO_EXCESS_BASE64\");\n                    }\n                }\n\n                // Check for spaces in recipient addresses\n                for token in raw_utf8.split('<') {\n                    if let Some((addr, _)) = token.split_once('>')\n                        && (addr.starts_with(' ') || addr.ends_with(' '))\n                    {\n                        ctx.result.add_tag(\"TO_WRAPPED_IN_SPACES\");\n                        break;\n                    }\n                }\n            }\n        }\n\n        let unique_recipients = ctx\n            .output\n            .all_recipients()\n            .filter(|rcpt| !rcpt.email.address.is_empty())\n            .collect::<HashSet<_>>();\n        let rcpt_count = unique_recipients.len();\n\n        match unique_recipients.len() {\n            0 => {\n                ctx.result.add_tag(\"RCPT_COUNT_ZERO\");\n                return;\n            }\n            1 => {\n                ctx.result.add_tag(\"RCPT_COUNT_ONE\");\n            }\n            2 => {\n                ctx.result.add_tag(\"RCPT_COUNT_TWO\");\n            }\n            3 => {\n                ctx.result.add_tag(\"RCPT_COUNT_THREE\");\n            }\n            4 | 5 => {\n                ctx.result.add_tag(\"RCPT_COUNT_FIVE\");\n            }\n            6 | 7 => {\n                ctx.result.add_tag(\"RCPT_COUNT_SEVEN\");\n            }\n            8..=12 => {\n                ctx.result.add_tag(\"RCPT_COUNT_TWELVE\");\n            }\n            13.. => {\n                ctx.result.add_tag(\"RCPT_COUNT_GT_50\");\n            }\n        }\n\n        let mut to_dn_eq_addr_count = 0;\n        let mut to_dn_count = 0;\n        let mut to_match_envrcpt = 0;\n\n        for rcpt in &unique_recipients {\n            // Validate name\n            if let Some(rcpt_name) = &rcpt.name {\n                if *rcpt_name == rcpt.email.address {\n                    to_dn_eq_addr_count += 1;\n                } else {\n                    to_dn_count += 1;\n                }\n            }\n\n            // Recipient is present in envelope\n            if ctx.output.env_to_addr.contains(&rcpt.email) {\n                to_match_envrcpt += 1;\n            }\n\n            // Check if the local part is present in the subject\n            if !rcpt.email.local_part.is_empty() {\n                if ctx.output.subject_lc.contains(rcpt.email.address.as_str()) {\n                    ctx.result.add_tag(\"RCPT_IN_SUBJECT\");\n                } else if rcpt.email.local_part.len() > 3\n                    && ctx\n                        .output\n                        .subject_lc\n                        .contains(rcpt.email.local_part.as_str())\n                {\n                    ctx.result.add_tag(\"RCPT_LOCAL_IN_SUBJECT\");\n                }\n            }\n        }\n\n        if to_dn_count == 0 && to_dn_eq_addr_count == 0 {\n            ctx.result.add_tag(\"TO_DN_NONE\");\n        } else if to_dn_count == rcpt_count {\n            ctx.result.add_tag(\"TO_DN_ALL\");\n        } else if to_dn_count > 0 {\n            ctx.result.add_tag(\"TO_DN_SOME\");\n        }\n\n        if to_dn_eq_addr_count == rcpt_count {\n            ctx.result.add_tag(\"TO_DN_EQ_ADDR_ALL\");\n        } else if to_dn_eq_addr_count > 0 {\n            ctx.result.add_tag(\"TO_DN_EQ_ADDR_SOME\");\n        }\n\n        if to_match_envrcpt == rcpt_count {\n            ctx.result.add_tag(\"TO_MATCH_ENVRCPT_ALL\");\n        } else {\n            if to_match_envrcpt > 0 {\n                ctx.result.add_tag(\"TO_MATCH_ENVRCPT_SOME\");\n            }\n\n            if !has_list_id && !has_list_unsubscribe {\n                for env_rcpt in &ctx.output.env_to_addr {\n                    if !unique_recipients.iter().any(|rcpt| rcpt.email == *env_rcpt)\n                        && env_rcpt != &ctx.output.env_from_addr\n                    {\n                        ctx.result.add_tag(\"FORGED_RECIPIENTS\");\n                        break;\n                    }\n                }\n            }\n        }\n\n        // Message from bounce and over 1 recipient\n        if rcpt_count > 1 && ctx.output.env_from_postmaster {\n            ctx.result.add_tag(\"RCPT_BOUNCEMOREONE\");\n        }\n\n        let rcpts = ctx\n            .output\n            .recipients_to\n            .iter()\n            .chain(ctx.output.recipients_cc.iter())\n            .collect::<Vec<_>>();\n\n        let mut is_sorted = false;\n        if rcpts.len() >= 6 {\n            // Check if the recipients list is sorted\n            let mut sorted = true;\n            for i in 1..rcpts.len() {\n                if rcpts[i - 1].email.address > rcpts[i].email.address {\n                    sorted = false;\n                    break;\n                }\n            }\n            if sorted {\n                ctx.result.add_tag(\"SORTED_RECIPS\");\n                is_sorted = true;\n            }\n        }\n\n        if !is_sorted && rcpt_count >= 5 {\n            // Look for similar recipients\n            let mut hits = 0;\n            let mut combinations = 0;\n            for i in 0..rcpts.len() {\n                for j in i + 1..rcpts.len() {\n                    let a = &rcpts[i].email;\n                    let b = &rcpts[j].email;\n\n                    if levenshtein_distance(&a.local_part, &b.local_part) < 3\n                        || (a.domain_part.fqdn != b.domain_part.fqdn\n                            && levenshtein_distance(&a.domain_part.fqdn, &b.domain_part.fqdn) < 4)\n                    {\n                        hits += 1;\n                    }\n                    combinations += 1;\n                }\n            }\n\n            if hits as f64 / combinations as f64 > 0.65 {\n                ctx.result.add_tag(\"SUSPICIOUS_RECIPS\");\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/replyto.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse mail_parser::HeaderName;\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeReplyTo: Sync + Send {\n    fn spam_filter_analyze_reply_to(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeReplyTo for Server {\n    async fn spam_filter_analyze_reply_to(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut reply_to_raw = b\"\".as_slice();\n        let mut is_from_list = false;\n\n        for header in ctx.input.message.headers() {\n            match &header.name {\n                HeaderName::ReplyTo => {\n                    reply_to_raw = ctx\n                        .input\n                        .message\n                        .raw_message()\n                        .get(header.offset_start as usize..header.offset_end as usize)\n                        .unwrap_or_default();\n                }\n                HeaderName::ListUnsubscribe | HeaderName::ListId => {\n                    is_from_list = true;\n                }\n\n                HeaderName::Other(name) => {\n                    if !is_from_list {\n                        is_from_list = name.eq_ignore_ascii_case(\"X-To-Get-Off-This-List\")\n                            || name.eq_ignore_ascii_case(\"X-List\")\n                            || name.eq_ignore_ascii_case(\"Auto-Submitted\");\n                    }\n                }\n                _ => {}\n            }\n        }\n\n        if reply_to_raw.is_empty() {\n            return;\n        }\n\n        if let Some(reply_to) = &ctx.output.reply_to {\n            let reply_to_name = reply_to.name.as_deref().unwrap_or_default();\n            ctx.result.add_tag(\"HAS_REPLYTO\");\n\n            if reply_to.email == ctx.output.from.email {\n                ctx.result.add_tag(\"REPLYTO_EQ_FROM\");\n            } else {\n                if reply_to.email.domain_part.sld == ctx.output.from.email.domain_part.sld {\n                    ctx.result.add_tag(\"REPLYTO_DOM_EQ_FROM_DOM\");\n                } else {\n                    if !is_from_list\n                        && ctx\n                            .output\n                            .all_recipients()\n                            .any(|r| r.email == reply_to.email)\n                    {\n                        ctx.result.add_tag(\"REPLYTO_EQ_TO_ADDR\");\n                    } else {\n                        ctx.result.add_tag(\"REPLYTO_DOM_NEQ_FROM_DOM\");\n                    }\n\n                    if !(is_from_list\n                        || ctx\n                            .output\n                            .recipients_to\n                            .iter()\n                            .any(|r| r.email == ctx.output.from.email)\n                        || ctx\n                            .output\n                            .env_to_addr\n                            .iter()\n                            .any(|r| r.domain_part.sld == ctx.output.from.email.domain_part.sld)\n                        || ctx.output.env_to_addr.len() == 1\n                            && ctx.output.env_to_addr.contains(&ctx.output.from.email))\n                    {\n                        ctx.result.add_tag(\"SPOOF_REPLYTO\");\n                    }\n                }\n\n                if !reply_to_name.is_empty()\n                    && reply_to_name == ctx.output.from.name.as_deref().unwrap_or_default()\n                {\n                    ctx.result.add_tag(\"REPLYTO_DN_EQ_FROM_DN\");\n                }\n            }\n\n            if reply_to.email == ctx.output.env_from_addr {\n                ctx.result.add_tag(\"REPLYTO_ADDR_EQ_FROM\");\n            }\n\n            // Validate unnecessary encoding\n            let reply_to_raw_utf8 = std::str::from_utf8(reply_to_raw).unwrap_or_default();\n            if reply_to.email.address.is_ascii()\n                && reply_to_name.is_ascii()\n                && reply_to_raw_utf8.contains(\"=?\")\n                && reply_to_raw_utf8.contains(\"?=\")\n            {\n                if reply_to_raw_utf8.contains(\"?q?\") || reply_to_raw_utf8.contains(\"?Q?\") {\n                    // Reply-To header is unnecessarily encoded in quoted-printable\n                    ctx.result.add_tag(\"REPLYTO_EXCESS_QP\");\n                } else if reply_to_raw_utf8.contains(\"?b?\") || reply_to_raw_utf8.contains(\"?B?\") {\n                    // Reply-To header is unnecessarily encoded in base64\n                    ctx.result.add_tag(\"REPLYTO_EXCESS_BASE64\");\n                }\n            }\n        } else {\n            ctx.result.add_tag(\"REPLYTO_UNPARSABLE\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/rules.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::{\n    Server,\n    config::spamfilter::{IpResolver, Location},\n};\n\nuse crate::{\n    SpamFilterContext, TextPart,\n    modules::expression::{EmailHeader, SpamFilterResolver, StringResolver},\n};\n\npub trait SpamFilterAnalyzeRules: Sync + Send {\n    fn spam_filter_analyze_rules(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeRules for Server {\n    async fn spam_filter_analyze_rules(&self, ctx: &mut SpamFilterContext<'_>) {\n        if !self.core.spam.rules.url.is_empty() {\n            for url in &ctx.output.urls {\n                for rule in &self.core.spam.rules.url {\n                    if let Some(tag) = self\n                        .eval_if::<String, _>(\n                            rule,\n                            &SpamFilterResolver::new(ctx, &url.element, url.location),\n                            ctx.input.span_id,\n                        )\n                        .await\n                    {\n                        ctx.result.tags.insert(tag);\n                    }\n                }\n            }\n        }\n\n        if !self.core.spam.rules.domain.is_empty() {\n            for domain in &ctx.output.domains {\n                let resolver = StringResolver(domain.element.as_str());\n\n                for rule in &self.core.spam.rules.domain {\n                    if let Some(tag) = self\n                        .eval_if::<String, _>(\n                            rule,\n                            &SpamFilterResolver::new(ctx, &resolver, domain.location),\n                            ctx.input.span_id,\n                        )\n                        .await\n                    {\n                        ctx.result.tags.insert(tag);\n                    }\n                }\n            }\n        }\n\n        if !self.core.spam.rules.email.is_empty() {\n            for email in &ctx.output.emails {\n                for rule in &self.core.spam.rules.email {\n                    if let Some(tag) = self\n                        .eval_if::<String, _>(\n                            rule,\n                            &SpamFilterResolver::new(ctx, &email.element, email.location),\n                            ctx.input.span_id,\n                        )\n                        .await\n                    {\n                        ctx.result.tags.insert(tag);\n                    }\n                }\n            }\n\n            for (rcpt, location) in [\n                (&ctx.output.recipients_to, Location::HeaderTo),\n                (&ctx.output.recipients_cc, Location::HeaderCc),\n                (&ctx.output.recipients_bcc, Location::HeaderBcc),\n            ] {\n                for email in rcpt {\n                    for rule in &self.core.spam.rules.email {\n                        if let Some(tag) = self\n                            .eval_if::<String, _>(\n                                rule,\n                                &SpamFilterResolver::new(ctx, email, location),\n                                ctx.input.span_id,\n                            )\n                            .await\n                        {\n                            ctx.result.tags.insert(tag);\n                        }\n                    }\n                }\n            }\n        }\n\n        if !self.core.spam.rules.ip.is_empty() {\n            for ip in &ctx.output.ips {\n                let ip_resolver = IpResolver::new(ip.element);\n\n                for rule in &self.core.spam.rules.ip {\n                    if let Some(tag) = self\n                        .eval_if::<String, _>(\n                            rule,\n                            &SpamFilterResolver::new(ctx, &ip_resolver, ip.location),\n                            ctx.input.span_id,\n                        )\n                        .await\n                    {\n                        ctx.result.tags.insert(tag);\n                    }\n                }\n            }\n        }\n\n        if !self.core.spam.rules.header.is_empty() {\n            for header in ctx.input.message.headers() {\n                let raw = String::from_utf8_lossy(\n                    ctx.input\n                        .message\n                        .raw_message()\n                        .get(header.offset_start as usize..header.offset_end as usize)\n                        .unwrap_or_default(),\n                );\n                let header_resolver = EmailHeader {\n                    header,\n                    raw: raw.as_ref(),\n                };\n\n                for rule in &self.core.spam.rules.header {\n                    if let Some(tag) = self\n                        .eval_if::<String, _>(\n                            rule,\n                            &SpamFilterResolver::new(ctx, &header_resolver, Location::BodyText),\n                            ctx.input.span_id,\n                        )\n                        .await\n                    {\n                        ctx.result.tags.insert(tag);\n                    }\n                }\n            }\n        }\n\n        if !self.core.spam.rules.body.is_empty() {\n            for (idx, part) in ctx.output.text_parts.iter().enumerate() {\n                let text = match part {\n                    TextPart::Plain { text_body, .. } => *text_body,\n                    TextPart::Html { text_body, .. } => text_body.as_str(),\n                    TextPart::None => continue,\n                };\n                let idx = idx as u32;\n                let location = if ctx.input.message.text_body.contains(&idx) {\n                    Location::BodyText\n                } else if ctx.input.message.html_body.contains(&idx) {\n                    Location::BodyHtml\n                } else {\n                    Location::Attachment\n                };\n                let string_resolver = StringResolver(text);\n\n                for rule in &self.core.spam.rules.body {\n                    if let Some(tag) = self\n                        .eval_if::<String, _>(\n                            rule,\n                            &SpamFilterResolver::new(ctx, &string_resolver, location),\n                            ctx.input.span_id,\n                        )\n                        .await\n                    {\n                        ctx.result.tags.insert(tag);\n                    }\n                }\n            }\n        }\n\n        if !self.core.spam.rules.any.is_empty() {\n            let dummy_resolver = StringResolver(\"\");\n            for rule in &self.core.spam.rules.any {\n                if let Some(tag) = self\n                    .eval_if::<String, _>(\n                        rule,\n                        &SpamFilterResolver::new(ctx, &dummy_resolver, Location::BodyText),\n                        ctx.input.span_id,\n                    )\n                    .await\n                {\n                    ctx.result.tags.insert(tag);\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/score.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    SpamFilterContext,\n    analysis::{\n        classifier::SpamFilterAnalyzeClassify, date::SpamFilterAnalyzeDate,\n        dmarc::SpamFilterAnalyzeDmarc, domain::SpamFilterAnalyzeDomain,\n        ehlo::SpamFilterAnalyzeEhlo, from::SpamFilterAnalyzeFrom,\n        headers::SpamFilterAnalyzeHeaders, html::SpamFilterAnalyzeHtml, ip::SpamFilterAnalyzeIp,\n        messageid::SpamFilterAnalyzeMid, mime::SpamFilterAnalyzeMime,\n        pyzor::SpamFilterAnalyzePyzor, received::SpamFilterAnalyzeReceived,\n        recipient::SpamFilterAnalyzeRecipient, replyto::SpamFilterAnalyzeReplyTo,\n        rules::SpamFilterAnalyzeRules, subject::SpamFilterAnalyzeSubject,\n        url::SpamFilterAnalyzeUrl,\n    },\n};\nuse common::{Server, config::spamfilter::SpamFilterAction};\nuse std::{fmt::Write, future::Future, vec};\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\nuse crate::analysis::llm::SpamFilterAnalyzeLlm;\n// SPDX-SnippetEnd\n\npub trait SpamFilterAnalyzeScore: Sync + Send {\n    fn spam_filter_finalize(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = SpamFilterAction<SpamFilterScore>> + Send;\n\n    fn spam_filter_classify(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = SpamFilterAction<SpamFilterScore>> + Send;\n}\n\n#[derive(Debug, Default)]\npub struct SpamFilterScore {\n    pub results: Vec<bool>,\n    pub headers: String,\n    pub train_spam: Option<bool>,\n    pub score: f32,\n}\n\nimpl SpamFilterAnalyzeScore for Server {\n    async fn spam_filter_finalize(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> SpamFilterAction<SpamFilterScore> {\n        // Calculate final score\n        let mut results = vec![];\n        let mut header_len = 60;\n        let mut is_spam_trap = false;\n        let mut rbl_count = 0;\n\n        for tag in &ctx.result.tags {\n            let score = match self.core.spam.lists.scores.get(tag) {\n                Some(SpamFilterAction::Allow(score)) => *score,\n                Some(SpamFilterAction::Discard) => {\n                    return SpamFilterAction::Discard;\n                }\n                Some(SpamFilterAction::Reject) => {\n                    return SpamFilterAction::Reject;\n                }\n                None | Some(SpamFilterAction::Disabled) => 0.0,\n            };\n            if tag == \"SPAM_TRAP\" {\n                is_spam_trap = true;\n            } else if score > 1.0 && tag.starts_with(\"RBL_\") {\n                rbl_count += 1;\n            }\n            ctx.result.score += score;\n            header_len += tag.len() + 10;\n            if score != 0.0 || !tag.starts_with(\"X_\") {\n                results.push((tag.as_str(), score));\n            }\n        }\n\n        let mut final_score = ctx.result.score;\n        let mut avg_confidence: f32 = 0.0;\n        let mut total_results = 0;\n        let mut user_results = vec![\n            ctx.result.score >= self.core.spam.scores.spam_threshold;\n            ctx.input.env_rcpt_to.len()\n        ];\n        if !ctx.result.classifier_confidence.is_empty() {\n            for (idx, &confidence) in ctx.result.classifier_confidence.iter().enumerate() {\n                if let Some(confidence) = confidence {\n                    avg_confidence += confidence;\n                    total_results += 1;\n\n                    let user_score = self\n                        .core\n                        .spam\n                        .lists\n                        .scores\n                        .get(confidence.spam_tag())\n                        .and_then(|v| v.as_score())\n                        .copied()\n                        .unwrap_or_default();\n\n                    user_results[idx] =\n                        ctx.result.score + user_score >= self.core.spam.scores.spam_threshold;\n                }\n            }\n\n            if total_results > 0 {\n                avg_confidence /= total_results as f32;\n\n                let tag = avg_confidence.spam_tag();\n                let score = self\n                    .core\n                    .spam\n                    .lists\n                    .scores\n                    .get(tag)\n                    .and_then(|v| v.as_score())\n                    .copied()\n                    .unwrap_or_default();\n                results.push((tag, score));\n                final_score += score;\n            }\n        }\n\n        if self.core.spam.scores.reject_threshold > 0.0\n            && final_score >= self.core.spam.scores.reject_threshold\n        {\n            SpamFilterAction::Reject\n        } else if self.core.spam.scores.discard_threshold > 0.0\n            && final_score >= self.core.spam.scores.discard_threshold\n        {\n            SpamFilterAction::Discard\n        } else {\n            let mut headers = String::with_capacity(header_len + 40);\n            results.sort_by(|a, b| a.1.partial_cmp(&b.1).unwrap().then_with(|| a.0.cmp(b.0)));\n            headers.push_str(\"X-Spam-Result: \");\n            for (idx, (tag, score)) in results.into_iter().enumerate() {\n                if idx > 0 {\n                    headers.push_str(\",\\r\\n\\t\");\n                }\n                let _ = write!(&mut headers, \"{} ({:.2})\", tag, score);\n            }\n            headers.push_str(\"\\r\\n\");\n\n            if let Some((category, explanation)) = &ctx.result.llm_result {\n                let _ = write!(&mut headers, \"X-Spam-LLM: {category} ({explanation})\\r\\n\",);\n            }\n\n            let is_spam = final_score >= self.core.spam.scores.spam_threshold;\n            let class = if is_spam { \"spam\" } else { \"ham\" };\n\n            if avg_confidence != 0.0 {\n                let _ = write!(\n                    &mut headers,\n                    \"X-Spam-Score: {class}, score={final_score:.2}, avg_confidence={avg_confidence:.2}\\r\\n\",\n                );\n            } else {\n                let _ = write!(\n                    &mut headers,\n                    \"X-Spam-Score: {class}, score={final_score:.2}\\r\\n\",\n                );\n            }\n\n            // Autolearn SPAM\n            let mut train_spam = None;\n            if is_spam\n                && self.core.spam.classifier.as_ref().is_some_and(|c| {\n                    (c.auto_learn_spam_trap && is_spam_trap)\n                        || (c.auto_learn_spam_rbl_count > 0\n                            && rbl_count >= c.auto_learn_spam_rbl_count)\n                })\n            {\n                train_spam = Some(true);\n            }\n\n            SpamFilterAction::Allow(SpamFilterScore {\n                results: user_results,\n                headers,\n                train_spam,\n                score: final_score,\n            })\n        }\n    }\n\n    async fn spam_filter_classify(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> SpamFilterAction<SpamFilterScore> {\n        // IP address analysis\n        self.spam_filter_analyze_ip(ctx).await;\n\n        // DMARC/SPF/DKIM/ARC analysis\n        self.spam_filter_analyze_dmarc(ctx).await;\n\n        // EHLO hostname analysis\n        self.spam_filter_analyze_ehlo(ctx).await;\n\n        // Generic header analysis\n        self.spam_filter_analyze_headers(ctx).await;\n\n        // Received headers analysis\n        self.spam_filter_analyze_received(ctx).await;\n\n        // Message-ID analysis\n        self.spam_filter_analyze_message_id(ctx).await;\n\n        // Date header analysis\n        self.spam_filter_analyze_date(ctx).await;\n\n        // Subject analysis\n        self.spam_filter_analyze_subject(ctx).await;\n\n        // From and Envelope From analysis\n        self.spam_filter_analyze_from(ctx).await;\n\n        // Reply-To analysis\n        self.spam_filter_analyze_reply_to(ctx).await;\n\n        // Recipient analysis\n        self.spam_filter_analyze_recipient(ctx).await;\n\n        // E-mail and domain analysis\n        self.spam_filter_analyze_domain(ctx).await;\n\n        // URL analysis\n        self.spam_filter_analyze_url(ctx).await;\n\n        // MIME part analysis\n        self.spam_filter_analyze_mime(ctx).await;\n\n        // HTML content analysis\n        self.spam_filter_analyze_html(ctx).await;\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n\n        // LLM classification\n        #[cfg(feature = \"enterprise\")]\n        self.spam_filter_analyze_llm(ctx).await;\n\n        // SPDX-SnippetEnd\n\n        // Spam trap\n        self.spam_filter_analyze_spam_trap(ctx).await;\n\n        // Pyzor checks\n        self.spam_filter_analyze_pyzor(ctx).await;\n\n        // Model classification\n        self.spam_filter_analyze_classify(ctx).await;\n\n        // User-defined rules\n        self.spam_filter_analyze_rules(ctx).await;\n\n        // Final score calculation\n        self.spam_filter_finalize(ctx).await\n    }\n}\n\npub trait ConfidenceStore {\n    fn spam_tag(&self) -> &'static str;\n}\n\nimpl ConfidenceStore for f32 {\n    fn spam_tag(&self) -> &'static str {\n        match *self {\n            p if p < 0.15 => \"PROB_HAM_HIGH\",\n            p if p < 0.25 => \"PROB_HAM_MEDIUM\",\n            p if p < 0.40 => \"PROB_HAM_LOW\",\n            p if p < 0.60 => \"PROB_SPAM_UNCERTAIN\",\n            p if p < 0.75 => \"PROB_SPAM_LOW\",\n            p if p < 0.85 => \"PROB_SPAM_MEDIUM\",\n            p => {\n                if p.is_finite() {\n                    \"PROB_SPAM_HIGH\"\n                } else {\n                    \"PROB_SPAM_UNCERTAIN\"\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/subject.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::future::Future;\n\nuse common::Server;\nuse mail_parser::HeaderName;\nuse nlp::tokenizers::types::TokenType;\nuse smtp_proto::{MAIL_BODY_8BITMIME, MAIL_BODY_BINARYMIME, MAIL_SMTPUTF8};\n\nuse crate::SpamFilterContext;\n\npub trait SpamFilterAnalyzeSubject: Sync + Send {\n    fn spam_filter_analyze_subject(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\nimpl SpamFilterAnalyzeSubject for Server {\n    async fn spam_filter_analyze_subject(&self, ctx: &mut SpamFilterContext<'_>) {\n        let mut subject_raw = b\"\".as_slice();\n\n        for header in ctx.input.message.headers() {\n            if header.name == HeaderName::Subject {\n                subject_raw = ctx\n                    .input\n                    .message\n                    .raw_message()\n                    .get(header.offset_start as usize..header.offset_end as usize)\n                    .unwrap_or_default();\n                break;\n            }\n        }\n\n        if subject_raw.is_empty() {\n            // Missing subject header\n            ctx.result.add_tag(\"MISSING_SUBJECT\");\n            return;\n        }\n\n        let mut word_count = 0;\n        let mut upper_count = 0;\n        let mut lower_count = 0;\n\n        let mut last_ch = ' ';\n        let mut is_ascii = true;\n\n        for ch in ctx.output.subject_thread.chars() {\n            if !ch.is_whitespace() {\n                if last_ch.is_whitespace() {\n                    word_count += 1;\n                }\n\n                match ch {\n                    '$' | '€' | '£' | '¥' | '₹' | '₽' | '₿' => {\n                        ctx.result.add_tag(\"SUBJECT_HAS_CURRENCY\");\n                    }\n                    _ => {\n                        if ch.is_alphabetic() {\n                            if ch.is_uppercase() {\n                                upper_count += 1;\n                            } else {\n                                lower_count += 1;\n                            }\n                        }\n                    }\n                }\n            }\n\n            if !ch.is_ascii() {\n                is_ascii = false;\n            }\n\n            last_ch = ch;\n        }\n\n        if ctx.output.subject_lc.is_empty() {\n            // Subject is empty\n            ctx.result.add_tag(\"EMPTY_SUBJECT\");\n        } else if ctx.output.subject.ends_with(' ') {\n            // Subject ends with whitespace\n            ctx.result.add_tag(\"SUBJECT_ENDS_SPACES\");\n        } else if ctx.output.subject\n            == \"XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X\"\n        {\n            ctx.result.add_tag(\"GTUBE_TEST\");\n        }\n\n        if ctx.output.subject_thread.len() >= 10\n            && word_count > 1\n            && upper_count > 2\n            && lower_count == 0\n        {\n            // Subject contains mostly capital letters\n            ctx.result.add_tag(\"SUBJ_ALL_CAPS\");\n        }\n\n        for token in &ctx.output.subject_tokens {\n            match token {\n                TokenType::Url(url) => {\n                    // Subject contains URL\n                    ctx.result.add_tag(\"URL_IN_SUBJECT\");\n\n                    if let Some(url_parsed) = &url.url_parsed {\n                        let host = url_parsed.host.sld_or_default();\n                        for rcpt in ctx.output.all_recipients() {\n                            if rcpt.email.domain_part.sld_or_default() == host {\n                                ctx.result.add_tag(\"RCPT_DOMAIN_IN_SUBJECT\");\n                                break;\n                            }\n                        }\n                    }\n                }\n                TokenType::UrlNoScheme(url) => {\n                    if let Some(url_parsed) = &url.url_parsed {\n                        let host = url_parsed.host.sld_or_default();\n                        for rcpt in ctx.output.all_recipients() {\n                            if rcpt.email.domain_part.sld_or_default() == host {\n                                ctx.result.add_tag(\"RCPT_DOMAIN_IN_SUBJECT\");\n                                break;\n                            }\n                        }\n                    }\n                }\n                TokenType::Email(email) => {\n                    // Subject contains recipient\n                    if ctx.output.env_to_addr.contains(email)\n                        || ctx\n                            .output\n                            .all_recipients()\n                            .any(|r| r.email.address == email.address)\n                    {\n                        ctx.result.add_tag(\"RCPT_IN_SUBJECT\");\n                    } else {\n                        let host = email.domain_part.sld_or_default();\n                        for rcpt in ctx.output.all_recipients() {\n                            if rcpt.email.address == email.address {\n                                ctx.result.add_tag(\"RCPT_IN_SUBJECT\");\n                                break;\n                            } else if rcpt.email.domain_part.sld_or_default() == host {\n                                ctx.result.add_tag(\"RCPT_DOMAIN_IN_SUBJECT\");\n                                break;\n                            }\n                        }\n                    }\n                }\n                _ => {}\n            }\n        }\n\n        // Validate encoding\n        let subject_raw_utf8 = std::str::from_utf8(subject_raw);\n        if !subject_raw.is_ascii() {\n            if (ctx.input.env_from_flags\n                & (MAIL_SMTPUTF8 | MAIL_BODY_8BITMIME | MAIL_BODY_BINARYMIME))\n                == 0\n            {\n                ctx.result.add_tag(\"SUBJECT_NEEDS_ENCODING\");\n            }\n\n            if subject_raw_utf8.is_err() {\n                ctx.result.add_tag(\"INVALID_SUBJECT_8BIT\");\n            }\n        }\n\n        // Validate unnecessary encoding\n        let subject_raw_utf8 = subject_raw_utf8.unwrap_or_default();\n        if is_ascii && subject_raw_utf8.contains(\"=?\") && subject_raw_utf8.contains(\"?=\") {\n            if subject_raw_utf8.contains(\"?q?\") || subject_raw_utf8.contains(\"?Q?\") {\n                // Subject header is unnecessarily encoded in quoted-printable\n                ctx.result.add_tag(\"SUBJ_EXCESS_QP\");\n            } else if subject_raw_utf8.contains(\"?b?\") || subject_raw_utf8.contains(\"?B?\") {\n                // Subject header is unnecessarily encoded in base64\n                ctx.result.add_tag(\"SUBJ_EXCESS_BASE64\");\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/analysis/url.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ElementLocation, is_trusted_domain, is_url_redirector};\nuse crate::modules::dnsbl::check_dnsbl;\nuse crate::modules::expression::StringResolver;\nuse crate::modules::html::SRC;\nuse crate::{\n    Hostname, SpamFilterContext, TextPart,\n    modules::html::{A, HREF, HtmlToken},\n};\nuse common::Server;\nuse common::config::spamfilter::{Element, IpResolver, Location};\nuse common::scripts::IsMixedCharset;\nuse common::scripts::functions::unicode::CharUtils;\nuse hyper::{Uri, header::LOCATION};\nuse nlp::tokenizers::types::TokenType;\nuse reqwest::redirect::Policy;\nuse std::collections::HashSet;\nuse std::hash::{Hash, Hasher};\nuse std::{borrow::Cow, future::Future, time::Duration};\n\npub trait SpamFilterAnalyzeUrl: Sync + Send {\n    fn spam_filter_analyze_url(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = ()> + Send;\n}\n\n#[derive(Clone, Debug)]\npub struct UrlParts<'x> {\n    pub url: String,\n    pub url_original: Cow<'x, str>,\n    pub url_parsed: Option<UrlParsed>,\n}\n\n#[derive(Clone, Debug)]\npub struct UrlParsed {\n    pub parts: Uri,\n    pub host: Hostname,\n}\n\nimpl SpamFilterAnalyzeUrl for Server {\n    async fn spam_filter_analyze_url(&self, ctx: &mut SpamFilterContext<'_>) {\n        // Extract URLs\n        let mut urls: HashSet<ElementLocation<UrlParts<'static>>> =\n            HashSet::from_iter(ctx.output.subject_tokens.iter().filter_map(|t| match t {\n                TokenType::Url(url) | TokenType::UrlNoScheme(url) => Some(ElementLocation::new(\n                    url.to_owned(),\n                    Location::HeaderSubject,\n                )),\n                _ => None,\n            }));\n        for (part_id, part) in ctx.output.text_parts.iter().enumerate() {\n            let part_id = part_id as u32;\n            let is_body = ctx.input.message.text_body.contains(&part_id)\n                || ctx.input.message.html_body.contains(&part_id);\n\n            let tokens = match part {\n                TextPart::Plain { tokens, .. } => tokens,\n                TextPart::Html {\n                    html_tokens,\n                    tokens,\n                    ..\n                } => {\n                    for token in html_tokens {\n                        if let HtmlToken::StartTag { attributes, .. } = token {\n                            for (attr, value) in attributes {\n                                match value {\n                                    Some(value) if [HREF, SRC].contains(attr) => {\n                                        urls.insert(ElementLocation::new(\n                                            UrlParts::new(value.trim().to_string()),\n                                            if is_body {\n                                                Location::BodyHtml\n                                            } else {\n                                                Location::Attachment\n                                            },\n                                        ));\n                                    }\n                                    _ => {}\n                                }\n                            }\n                        }\n                    }\n                    tokens\n                }\n                TextPart::None => &[][..],\n            };\n\n            for token in tokens {\n                match token {\n                    TokenType::Url(url) | TokenType::UrlNoScheme(url) => {\n                        if !ctx.input.is_train\n                            && is_body\n                            && !ctx.result.has_tag(\"RCPT_DOMAIN_IN_BODY\")\n                            && let Some(url_parsed) = &url.url_parsed\n                        {\n                            let host = url_parsed.host.sld_or_default();\n                            for rcpt in ctx.output.all_recipients() {\n                                if rcpt.email.domain_part.sld_or_default() == host {\n                                    ctx.result.add_tag(\"RCPT_DOMAIN_IN_BODY\");\n                                    break;\n                                }\n                            }\n                        }\n\n                        urls.insert(ElementLocation::new(\n                            url.to_owned(),\n                            if is_body {\n                                Location::BodyHtml\n                            } else {\n                                Location::Attachment\n                            },\n                        ));\n                    }\n                    _ => {}\n                }\n            }\n\n            if is_body && !ctx.input.is_train {\n                let is_single = match part {\n                    TextPart::Plain { tokens, .. } => is_single_url(tokens),\n                    TextPart::Html {\n                        html_tokens,\n                        tokens,\n                        ..\n                    } => is_single_html_url(html_tokens, tokens),\n                    TextPart::None => false,\n                };\n\n                if is_single {\n                    ctx.result.add_tag(\"URL_ONLY\");\n                }\n            }\n        }\n\n        if !ctx.input.is_train {\n            let mut redirected_urls = HashSet::new();\n            for url in &urls {\n                for ch in url.element.url.chars() {\n                    if ch.is_zwsp() {\n                        ctx.result.add_tag(\"ZERO_WIDTH_SPACE_URL\");\n                    }\n\n                    if ch.is_obscured() {\n                        ctx.result.add_tag(\"SUSPICIOUS_URL\");\n                    }\n                }\n\n                // Skip non-URLs such as 'data:' and 'mailto:'\n                if !url.element.url.contains(\"://\") {\n                    continue;\n                }\n\n                // Obtain parse url\n                let url_parsed = if let Some(url_parsed) = &url.element.url_parsed {\n                    url_parsed\n                } else {\n                    // URL could not be parsed\n                    ctx.result.add_tag(\"UNPARSABLE_URL\");\n                    continue;\n                };\n                let host_sld = url_parsed.host.sld_or_default();\n\n                // Skip local and trusted domains\n                if is_trusted_domain(self, host_sld, ctx.input.span_id).await {\n                    continue;\n                }\n\n                if let Some(ip) = url_parsed.host.ip {\n                    // Check IP DNSBL\n                    check_dnsbl(self, ctx, &IpResolver::new(ip), Element::Ip, url.location).await;\n                } else if is_url_redirector(self, host_sld, ctx.input.span_id).await {\n                    // Check for redirectors\n                    ctx.result.add_tag(\"REDIRECTOR_URL\");\n\n                    if !ctx.result.has_tag(\"URL_REDIRECTOR_NESTED\") {\n                        let mut redirect_count = 1;\n                        let mut url_redirect = Cow::Borrowed(url.element.url.as_str());\n\n                        while redirect_count <= 3 {\n                            match http_get_header(\n                                url_redirect.as_ref(),\n                                LOCATION,\n                                Duration::from_secs(5),\n                            )\n                            .await\n                            {\n                                Ok(Some(location)) => {\n                                    let location = UrlParts::new(location);\n                                    if let Some(location_parsed) = &location.url_parsed {\n                                        if is_url_redirector(\n                                            self,\n                                            location_parsed.host.sld_or_default(),\n                                            ctx.input.span_id,\n                                        )\n                                        .await\n                                        {\n                                            url_redirect = Cow::Owned(location.url);\n                                            redirect_count += 1;\n                                            continue;\n                                        } else {\n                                            redirected_urls.insert(ElementLocation::new(\n                                                location,\n                                                url.location,\n                                            ));\n                                        }\n                                    }\n                                }\n                                Ok(None) => {}\n                                Err(err) => {\n                                    trc::error!(err.span_id(ctx.input.span_id));\n                                }\n                            }\n                            break;\n                        }\n\n                        if redirect_count > 3 {\n                            ctx.result.add_tag(\"URL_REDIRECTOR_NESTED\");\n                        }\n                    }\n                }\n            }\n\n            urls.extend(redirected_urls);\n\n            for (el, url_parsed) in urls.iter().filter_map(|el| {\n                el.element\n                    .url_parsed\n                    .as_ref()\n                    .map(|url_parsed| (el, url_parsed))\n            }) {\n                let host = &url_parsed.host;\n\n                if host.ip.is_none() {\n                    if !host.fqdn.is_ascii() {\n                        if let Ok(cured_host) =\n                            decancer::cure(&host.fqdn, decancer::Options::default())\n                        {\n                            let cured_host = cured_host.to_string();\n                            if cured_host != host.fqdn\n                                && matches!(self.dns_exists_ip(&cured_host).await, Ok(true))\n                            {\n                                ctx.result.add_tag(\"HOMOGRAPH_URL\");\n                            }\n                        }\n\n                        if host.fqdn.is_mixed_charset() {\n                            ctx.result.add_tag(\"MIXED_CHARSET_URL\");\n                        }\n                    }\n\n                    // Check Domain DNSBL\n                    if let Some(sld) = &host.sld {\n                        check_dnsbl(\n                            self,\n                            ctx,\n                            &StringResolver(sld),\n                            Element::Domain,\n                            el.location,\n                        )\n                        .await;\n                    }\n                } else {\n                    // URL is an ip address\n                    ctx.result.add_tag(\"SUSPICIOUS_URL\");\n                }\n\n                // Check URL DNSBL\n                check_dnsbl(self, ctx, &el.element, Element::Url, el.location).await;\n            }\n        }\n\n        // Update context\n        ctx.output.urls = urls;\n    }\n}\n\n#[allow(unreachable_code)]\n#[allow(unused_variables)]\nasync fn http_get_header(\n    url: &str,\n    header: hyper::header::HeaderName,\n    timeout: Duration,\n) -> trc::Result<Option<String>> {\n    #[cfg(feature = \"test_mode\")]\n    {\n        return if url.contains(\"redirect.\") {\n            Ok(url.split_once(\"/?\").unwrap().1.to_string().into())\n        } else {\n            Ok(None)\n        };\n    }\n    reqwest::Client::builder()\n        .user_agent(\"Mozilla/5.0 (X11; Linux i686; rv:109.0) Gecko/20100101 Firefox/118.0\")\n        .timeout(timeout)\n        .redirect(Policy::none())\n        .danger_accept_invalid_certs(true)\n        .build()\n        .map_err(|err| {\n            trc::SieveEvent::RuntimeError\n                .into_err()\n                .reason(err)\n                .details(\"Failed to build request\")\n        })?\n        .get(url)\n        .send()\n        .await\n        .map_err(|err| {\n            trc::SieveEvent::RuntimeError\n                .into_err()\n                .reason(err)\n                .details(\"Failed to send request\")\n        })\n        .map(|response| {\n            response\n                .headers()\n                .get(header)\n                .and_then(|h| h.to_str().ok())\n                .map(|h| h.to_string())\n        })\n}\n\nfn is_single_url<T, E, U, I>(tokens: &[TokenType<T, E, U, I>]) -> bool {\n    let mut url_count = 0;\n    let mut word_count = 0;\n\n    for token in tokens {\n        match token {\n            TokenType::Alphabetic(_)\n            | TokenType::Alphanumeric(_)\n            | TokenType::Integer(_)\n            | TokenType::Email(_)\n            | TokenType::Float(_) => {\n                word_count += 1;\n            }\n            TokenType::Url(_) | TokenType::UrlNoScheme(_) => {\n                url_count += 1;\n            }\n            _ => {}\n        }\n    }\n\n    url_count == 1 && word_count <= 1\n}\n\nfn is_single_html_url<T, E, U, I>(\n    html_tokens: &[HtmlToken],\n    tokens: &[TokenType<T, E, U, I>],\n) -> bool {\n    let mut url_count = 0;\n    let mut word_count = 0;\n\n    for token in tokens {\n        match token {\n            TokenType::Alphabetic(_)\n            | TokenType::Alphanumeric(_)\n            | TokenType::Integer(_)\n            | TokenType::Email(_)\n            | TokenType::Float(_) => {\n                word_count += 1;\n            }\n            TokenType::Url(_) | TokenType::UrlNoScheme(_) => {\n                url_count += 1;\n            }\n            _ => {}\n        }\n    }\n\n    if word_count > 1 || url_count != 1 {\n        return false;\n    }\n\n    url_count = 0;\n\n    for token in html_tokens {\n        if matches!(token, HtmlToken::StartTag { name, attributes, .. } if *name == A && attributes.iter().any(|(k, _)| *k == HREF))\n        {\n            url_count += 1;\n        }\n    }\n\n    url_count == 1\n}\n\nimpl PartialEq for UrlParts<'_> {\n    fn eq(&self, other: &Self) -> bool {\n        self.url == other.url\n    }\n}\n\nimpl Eq for UrlParts<'_> {}\n\nimpl Hash for UrlParts<'_> {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.url.hash(state);\n    }\n}\n\nimpl<'x> UrlParts<'x> {\n    pub fn new(url: impl Into<Cow<'x, str>>) -> Self {\n        let url_original = url.into();\n        let url = url_original.trim().to_lowercase();\n\n        Self {\n            url_parsed: url.parse::<Uri>().ok().and_then(|url_parsed| {\n                if url_parsed.host().is_some() {\n                    Some(UrlParsed {\n                        host: Hostname::new(url_parsed.host().unwrap()),\n                        parts: url_parsed,\n                    })\n                } else {\n                    None\n                }\n            }),\n            url,\n            url_original,\n        }\n    }\n\n    pub fn to_owned(&self) -> UrlParts<'static> {\n        UrlParts {\n            url: self.url.clone(),\n            url_original: Cow::Owned(self.url_original.clone().into_owned()),\n            url_parsed: self.url_parsed.clone(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod analysis;\npub mod modules;\n\nuse analysis::ElementLocation;\nuse analysis::url::UrlParts;\nuse mail_auth::{ArcOutput, DkimOutput, DmarcResult, IprevOutput, SpfOutput, dmarc::Policy};\nuse mail_parser::Message;\nuse modules::html::HtmlToken;\nuse nlp::tokenizers::types::TokenType;\nuse std::borrow::Cow;\nuse std::collections::HashSet;\nuse std::hash::{Hash, Hasher};\nuse std::net::{IpAddr, Ipv4Addr};\nuse store::ahash::AHashSet;\n\npub struct SpamFilterInput<'x> {\n    pub message: &'x Message<'x>,\n    pub span_id: u64,\n\n    // Sender authentication\n    pub arc_result: Option<&'x ArcOutput<'x>>,\n    pub spf_ehlo_result: Option<&'x SpfOutput>,\n    pub spf_mail_from_result: Option<&'x SpfOutput>,\n    pub dkim_result: &'x [DkimOutput<'x>],\n    pub dmarc_result: Option<&'x DmarcResult>,\n    pub dmarc_policy: Option<&'x Policy>,\n    pub iprev_result: Option<&'x IprevOutput>,\n\n    // Session details\n    pub remote_ip: IpAddr,\n    pub ehlo_domain: Option<&'x str>,\n    pub authenticated_as: Option<&'x str>,\n    pub asn: Option<u32>,\n    pub country: Option<&'x str>,\n\n    // TLS\n    pub is_tls: bool,\n\n    // Envelope\n    pub env_from: &'x str,\n    pub env_from_flags: u64,\n    pub env_rcpt_to: Vec<&'x str>,\n\n    pub is_train: bool,\n    pub is_test: bool,\n}\n\npub struct SpamFilterOutput<'x> {\n    pub ehlo_host: Hostname,\n    pub iprev_ptr: Option<String>,\n\n    pub env_from_addr: Email,\n    pub env_from_postmaster: bool,\n    pub env_to_addr: HashSet<Email>,\n    pub from: Recipient,\n    pub recipients_to: Vec<Recipient>,\n    pub recipients_cc: Vec<Recipient>,\n    pub recipients_bcc: Vec<Recipient>,\n    pub reply_to: Option<Recipient>,\n\n    pub subject: String,\n    pub subject_lc: String,\n    pub subject_thread: String,\n    pub subject_thread_lc: String,\n    pub subject_tokens: Vec<TokenType<Cow<'x, str>, Email, UrlParts<'x>, IpParts>>,\n\n    pub ips: AHashSet<ElementLocation<IpAddr>>,\n    pub urls: HashSet<ElementLocation<UrlParts<'x>>>,\n    pub emails: HashSet<ElementLocation<Recipient>>,\n    pub domains: HashSet<ElementLocation<String>>,\n\n    pub text_parts: Vec<TextPart<'x>>,\n}\n\n#[derive(Debug)]\npub struct IpParts {\n    ip: Option<IpAddr>,\n}\n\npub enum TextPart<'x> {\n    Plain {\n        text_body: &'x str,\n        tokens: Vec<TokenType<Cow<'x, str>, Email, UrlParts<'x>, IpParts>>,\n    },\n    Html {\n        html_tokens: Vec<HtmlToken>,\n        text_body: String,\n        tokens: Vec<TokenType<Cow<'x, str>, Email, UrlParts<'x>, IpParts>>,\n    },\n    None,\n}\n\n#[derive(Debug, Default)]\npub struct SpamFilterResult {\n    pub tags: AHashSet<String>,\n    pub classifier_confidence: Vec<Option<f32>>,\n    pub score: f32,\n    pub rbl_ip_checks: usize,\n    pub rbl_domain_checks: usize,\n    pub rbl_url_checks: usize,\n    pub rbl_email_checks: usize,\n    pub llm_result: Option<(String, String)>,\n}\n\npub struct SpamFilterContext<'x> {\n    pub input: SpamFilterInput<'x>,\n    pub output: SpamFilterOutput<'x>,\n    pub result: SpamFilterResult,\n}\n\n#[derive(Debug, Clone)]\npub struct Hostname {\n    pub fqdn: String,\n    pub ip: Option<IpAddr>,\n    pub sld: Option<String>,\n}\n\n#[derive(Debug, Clone)]\npub struct Email {\n    pub address: String,\n    pub local_part: String,\n    pub domain_part: Hostname,\n}\n\n#[derive(Debug, Clone)]\npub struct Recipient {\n    pub email: Email,\n    pub name: Option<String>,\n}\n\nimpl<'x> SpamFilterInput<'x> {\n    pub fn from_message(message: &'x Message<'x>, span_id: u64) -> Self {\n        Self {\n            message,\n            span_id,\n            arc_result: None,\n            spf_ehlo_result: None,\n            spf_mail_from_result: None,\n            dkim_result: &[],\n            dmarc_result: None,\n            dmarc_policy: None,\n            iprev_result: None,\n            remote_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),\n            ehlo_domain: None,\n            authenticated_as: None,\n            asn: None,\n            country: None,\n            is_tls: true,\n            env_from: \"\",\n            env_from_flags: 0,\n            env_rcpt_to: vec![],\n            is_test: false,\n            is_train: false,\n        }\n    }\n\n    pub fn train_mode(mut self) -> Self {\n        self.is_train = true;\n        self\n    }\n}\n\nimpl PartialEq for Hostname {\n    fn eq(&self, other: &Self) -> bool {\n        self.fqdn.eq(&other.fqdn)\n    }\n}\n\nimpl Eq for Hostname {}\n\nimpl PartialEq for Email {\n    fn eq(&self, other: &Self) -> bool {\n        self.address.eq(&other.address)\n    }\n}\n\nimpl Eq for Email {}\n\nimpl Hash for Hostname {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.fqdn.hash(state)\n    }\n}\n\nimpl Hash for Email {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.address.hash(state)\n    }\n}\n\nimpl Email {\n    pub fn classifier_parts(&self) -> Option<(&str, &str)> {\n        // Returns (local@, @domain)\n        if self.is_valid() {\n            let at_pos = self.address.find('@')?;\n            Some((&self.address[..=at_pos], &self.address[at_pos..]))\n        } else {\n            None\n        }\n    }\n\n    pub fn is_valid(&self) -> bool {\n        self.domain_part.sld.is_some() && !self.local_part.is_empty()\n    }\n}\n\nimpl PartialEq for Recipient {\n    fn eq(&self, other: &Self) -> bool {\n        self.email.eq(&other.email)\n    }\n}\n\nimpl Eq for Recipient {}\n\nimpl Hash for Recipient {\n    fn hash<H: Hasher>(&self, state: &mut H) {\n        self.email.hash(state)\n    }\n}\n\nimpl PartialOrd for Email {\n    fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {\n        Some(self.cmp(other))\n    }\n}\n\nimpl PartialOrd for Recipient {\n    fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {\n        Some(self.cmp(other))\n    }\n}\n\nimpl Ord for Email {\n    fn cmp(&self, other: &Self) -> std::cmp::Ordering {\n        self.address.cmp(&other.address)\n    }\n}\n\nimpl Ord for Recipient {\n    fn cmp(&self, other: &Self) -> std::cmp::Ordering {\n        self.email.cmp(&other.email)\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/modules/classifier.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::analysis::domain::SpamFilterAnalyzeDomain;\nuse crate::analysis::init::SpamFilterInit;\nuse crate::analysis::is_trusted_domain;\nuse crate::analysis::url::SpamFilterAnalyzeUrl;\nuse crate::modules::html::{A, ALT, HREF, HtmlToken, IMG, SRC, TITLE};\nuse crate::{Email, SpamFilterContext, TextPart};\nuse crate::{Hostname, SpamFilterInput};\nuse common::config::spamfilter;\nuse common::manager::{SPAM_CLASSIFIER_KEY, SPAM_TRAINER_KEY};\nuse common::{Server, config::spamfilter::Location, ipc::BroadcastEvent};\nuse mail_auth::DmarcResult;\nuse mail_parser::{MessageParser, MimeHeaders};\nuse nlp::classifier::feature::{\n    CcfhFeature, CcfhFeatureBuilder, FeatureBuilder, FhFeature, FhFeatureBuilder, Sample,\n    UnprocessedFeature,\n};\nuse nlp::classifier::ftrl::Ftrl;\nuse nlp::classifier::reservoir::SampleReservoir;\nuse nlp::classifier::train::{CcfhTrainer, FhTrainer};\nuse nlp::tokenizers::types::TypesTokenizer;\nuse nlp::tokenizers::{stream::WordStemTokenizer, types::TokenType};\nuse std::time::Instant;\nuse std::{\n    borrow::Cow,\n    collections::{HashMap, hash_map::Entry},\n    hash::{Hash, RandomState},\n    sync::Arc,\n};\nuse store::rand::seq::SliceRandom;\nuse store::write::{BlobLink, now};\nuse store::{\n    Deserialize, IterateParams, Serialize, U32_LEN, U64_LEN, ValueKey,\n    write::{\n        AlignedBytes, Archive, Archiver, BatchBuilder, BlobOp, ValueClass,\n        key::DeserializeBigEndian,\n    },\n};\nuse tokio::sync::{mpsc, oneshot};\nuse trc::{AddContext, SpamEvent};\nuse types::blob_hash::BlobHash;\nuse unicode_general_category::{GeneralCategory, get_general_category};\nuse unicode_normalization::UnicodeNormalization;\nuse unicode_security::mixed_script::AugmentedScriptSet;\n\npub trait SpamClassifier {\n    fn spam_train(&self, retrain: bool) -> impl Future<Output = trc::Result<()>> + Send;\n\n    fn spam_classify(\n        &self,\n        ctx: &mut SpamFilterContext<'_>,\n    ) -> impl Future<Output = trc::Result<()>> + Send;\n\n    fn spam_build_tokens<'x>(\n        &self,\n        ctx: &'x SpamFilterContext<'_>,\n    ) -> impl Future<Output = Tokens<'x>> + Send;\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Clone, PartialEq, Eq, Debug)]\npub struct TrainingSample {\n    hash: BlobHash,\n    account_id: u32,\n}\n\nstruct TrainingTask {\n    sample: TrainingSample,\n    is_spam: bool,\n    is_replay: bool,\n    remove: Option<u64>,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug)]\npub struct SpamTrainer {\n    pub trainer: SpamTrainerClass,\n    pub reservoir: SampleReservoir<TrainingSample>,\n    pub last_sample_expiry: u64,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug)]\npub enum SpamTrainerClass {\n    FtrlFh(Box<FhTrainer<Ftrl>>),\n    FtrlCfh(Box<CcfhTrainer<Ftrl, Ftrl>>),\n}\n\nimpl SpamClassifier for Server {\n    async fn spam_train(&self, retrain: bool) -> trc::Result<()> {\n        let Some(config) = &self.core.spam.classifier else {\n            return Ok(());\n        };\n\n        let _permit = self\n            .inner\n            .ipc\n            .train_task_controller\n            .try_run()\n            .ok_or_else(|| {\n                trc::EventType::Spam(SpamEvent::TrainCompleted)\n                    .reason(\"Spam training task is already running\")\n                    .caused_by(trc::location!())\n            })?;\n\n        let started = Instant::now();\n        trc::event!(Spam(SpamEvent::TrainStarted));\n\n        // Fetch or build trainer\n        let mut trainer = if !retrain\n            && let Some(trainer) = self\n                .blob_store()\n                .get_blob(SPAM_TRAINER_KEY, 0..usize::MAX)\n                .await\n                .and_then(|archive| match archive {\n                    Some(archive) => <Archive<AlignedBytes> as Deserialize>::deserialize(&archive)\n                        .and_then(|archive| archive.deserialize_untrusted::<SpamTrainer>())\n                        .map(Some),\n                    None => Ok(None),\n                })\n                .caused_by(trc::location!())?\n        {\n            trainer\n        } else {\n            SpamTrainer {\n                trainer: match &config.i_params {\n                    Some(i_params) => SpamTrainerClass::FtrlCfh(Box::new(CcfhTrainer::new(\n                        Ftrl::new(config.w_params.feature_hash_size),\n                        Ftrl::new(i_params.feature_hash_size).with_initial_weights(0.5),\n                    ))),\n                    None => SpamTrainerClass::FtrlFh(Box::new(FhTrainer::new(Ftrl::new(\n                        config.w_params.feature_hash_size,\n                    )))),\n                },\n                reservoir: SampleReservoir::default(),\n                last_sample_expiry: 0,\n            }\n        };\n\n        // Update hyperparameters\n        match (&mut trainer.trainer, &config.i_params) {\n            (SpamTrainerClass::FtrlFh(trainer), None) => {\n                trainer.optimizer_mut().set_hyperparams(\n                    config.w_params.alpha,\n                    config.w_params.beta,\n                    config.w_params.l1_ratio,\n                    config.w_params.l2_ratio,\n                );\n            }\n            (SpamTrainerClass::FtrlCfh(trainer), Some(i_params)) => {\n                trainer.w_optimizer_mut().set_hyperparams(\n                    config.w_params.alpha,\n                    config.w_params.beta,\n                    config.w_params.l1_ratio,\n                    config.w_params.l2_ratio,\n                );\n                trainer.i_optimizer_mut().set_hyperparams(\n                    i_params.alpha,\n                    i_params.beta,\n                    i_params.l1_ratio,\n                    i_params.l2_ratio,\n                );\n            }\n            _ => {}\n        }\n\n        // Fetch blob hashes for samples\n        let mut samples = Vec::new();\n        let mut remove_entries = false;\n        let from_key = ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Blob(BlobOp::SpamSample {\n                hash: BlobHash::default(),\n                until: trainer.last_sample_expiry + 1,\n            }),\n        };\n        let to_key = ValueKey {\n            account_id: u32::MAX,\n            collection: u8::MAX,\n            document_id: u32::MAX,\n            class: ValueClass::Blob(BlobOp::SpamSample {\n                hash: BlobHash::new_max(),\n                until: u64::MAX,\n            }),\n        };\n        let mut spam_count = 0;\n        let mut ham_count = 0;\n        self.store()\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending(),\n                |key, value| {\n                    let until = key.deserialize_be_u64(1)?;\n                    let account_id = key.deserialize_be_u32(U64_LEN + 1)?;\n                    let hash = BlobHash::try_from_hash_slice(\n                        key.get(U64_LEN + U32_LEN + 1..).ok_or_else(|| {\n                            trc::Error::corrupted_key(key, value.into(), trc::location!())\n                        })?,\n                    )\n                    .unwrap();\n                    let (Some(is_spam), Some(hold)) = (value.first(), value.get(1)) else {\n                        return Err(trc::Error::corrupted_key(\n                            key,\n                            value.into(),\n                            trc::location!(),\n                        ));\n                    };\n\n                    let do_remove = *hold == 0;\n                    let is_spam = *is_spam == 1;\n                    let sample = TrainingSample { hash, account_id };\n\n                    // Add to reservoir\n                    if !do_remove {\n                        trainer.reservoir.update_reservoir(\n                            &sample,\n                            is_spam,\n                            config.reservoir_capacity,\n                        );\n                    } else {\n                        trainer.reservoir.update_counts(is_spam);\n                    }\n\n                    samples.push(TrainingTask {\n                        sample,\n                        is_spam,\n                        is_replay: false,\n                        remove: do_remove.then_some(until),\n                    });\n\n                    remove_entries |= do_remove;\n\n                    // Update trainer stats\n                    trainer.last_sample_expiry = until;\n                    if is_spam {\n                        spam_count += 1;\n                    } else {\n                        ham_count += 1;\n                    }\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        if samples.is_empty() {\n            trc::event!(\n                Spam(SpamEvent::TrainCompleted),\n                Total = 0,\n                Elapsed = started.elapsed()\n            );\n\n            return Ok(());\n        } else if (trainer.reservoir.ham.total_seen < config.min_ham_samples)\n            || (trainer.reservoir.spam.total_seen < config.min_spam_samples)\n        {\n            trc::event!(\n                Spam(SpamEvent::ModelNotReady),\n                Reason = \"Not enough samples for training\",\n                Details = vec![\n                    trc::Value::from(trainer.reservoir.ham.total_seen),\n                    trc::Value::from(trainer.reservoir.spam.total_seen)\n                ],\n                Limit = vec![\n                    trc::Value::from(config.min_ham_samples),\n                    trc::Value::from(config.min_spam_samples)\n                ],\n                Elapsed = started.elapsed()\n            );\n\n            return Ok(());\n        }\n\n        // Balance classes if needed\n        if spam_count > ham_count {\n            // We have too much spam this time. We need to replay old HAM.\n            samples.extend(\n                trainer\n                    .reservoir\n                    .replay_samples((spam_count - ham_count) as usize, false)\n                    .map(|sample| TrainingTask {\n                        sample: sample.clone(),\n                        is_spam: false,\n                        is_replay: true,\n                        remove: None,\n                    }),\n            );\n        } else if ham_count > spam_count {\n            // We have too much ham this time. We need to replay old SPAM.\n            samples.extend(\n                trainer\n                    .reservoir\n                    .replay_samples((ham_count - spam_count) as usize, true)\n                    .map(|sample| TrainingTask {\n                        sample: sample.clone(),\n                        is_spam: true,\n                        is_replay: true,\n                        remove: None,\n                    }),\n            );\n        }\n\n        let num_samples = samples.len();\n        samples.shuffle(&mut store::rand::rng());\n\n        // Spawn training task\n        let epochs = match trainer\n            .reservoir\n            .ham\n            .total_seen\n            .min(trainer.reservoir.spam.total_seen)\n        {\n            0..=50 => 3,   // Bootstrap\n            51..=200 => 2, // Refinement\n            _ => 1,        // Full online training\n        };\n        let task = trainer.trainer.spawn(epochs)?;\n        let is_fh = matches!(task, TrainTask::Fh { .. });\n\n        // Train\n        for chunk in samples.chunks(128) {\n            let mut fh_samples = if is_fh {\n                Vec::with_capacity(chunk.len())\n            } else {\n                Vec::new()\n            };\n            let mut ccfh_samples = if !is_fh {\n                Vec::with_capacity(chunk.len())\n            } else {\n                Vec::new()\n            };\n\n            for sample in chunk {\n                let account_id = if sample.sample.account_id != u32::MAX {\n                    Some(sample.sample.account_id)\n                } else {\n                    None\n                };\n                let Some(raw_message) = self\n                    .blob_store()\n                    .get_blob(sample.sample.hash.as_slice(), 0..usize::MAX)\n                    .await\n                    .caused_by(trc::location!())?\n                else {\n                    if sample.is_replay {\n                        trainer\n                            .reservoir\n                            .remove_sample(&sample.sample, sample.is_spam);\n                    } else {\n                        trc::event!(\n                            Spam(SpamEvent::TrainSampleNotFound),\n                            Reason = \"Blob not found\",\n                            AccountId = account_id,\n                            BlobId = sample.sample.hash.to_hex(),\n                        );\n                    }\n                    continue;\n                };\n\n                // Build features\n                let Some(message) = MessageParser::new().parse(&raw_message) else {\n                    if sample.is_replay {\n                        trainer\n                            .reservoir\n                            .remove_sample(&sample.sample, sample.is_spam);\n                    }\n                    trc::event!(\n                        Spam(SpamEvent::TrainSampleNotFound),\n                        Reason = \"Failed to parse message\",\n                        AccountId = account_id,\n                        BlobId = sample.sample.hash.to_hex(),\n                    );\n                    continue;\n                };\n                let mut ctx =\n                    self.spam_filter_init(SpamFilterInput::from_message(&message, 0).train_mode());\n                self.spam_filter_analyze_domain(&mut ctx).await;\n                self.spam_filter_analyze_url(&mut ctx).await;\n                let mut tokens = self.spam_build_tokens(&ctx).await.0;\n\n                match &task {\n                    TrainTask::Fh { builder, .. } => {\n                        if config.log_scale {\n                            builder.scale(&mut tokens);\n                        }\n                        fh_samples.push(Sample::new(\n                            builder.build(&tokens, account_id, config.l2_normalize),\n                            sample.is_spam,\n                        ));\n                    }\n                    TrainTask::Ccfh { builder, .. } => {\n                        if config.log_scale {\n                            builder.scale(&mut tokens);\n                        }\n                        ccfh_samples.push(Sample::new(\n                            builder.build(&tokens, account_id, config.l2_normalize),\n                            sample.is_spam,\n                        ));\n                    }\n                }\n\n                // Look for stop requests\n                if self.inner.ipc.train_task_controller.should_stop() {\n                    trc::event!(\n                        Spam(SpamEvent::TrainCompleted),\n                        Reason = \"Training task was stopped\",\n                        Total = fh_samples.len() + ccfh_samples.len(),\n                        Elapsed = started.elapsed()\n                    );\n                    return Ok(());\n                }\n            }\n\n            // Send batch for training\n            let (done_tx, done_rx) = oneshot::channel::<()>();\n            match &task {\n                TrainTask::Fh { batch_tx, .. } => {\n                    batch_tx\n                        .send(FhTrainJob {\n                            samples: fh_samples,\n                            done: done_tx,\n                        })\n                        .await\n                        .map_err(|err| {\n                            trc::EventType::Server(trc::ServerEvent::ThreadError)\n                                .reason(err)\n                                .details(\"Spam train task failed\")\n                                .caused_by(trc::location!())\n                        })?;\n                }\n                TrainTask::Ccfh { batch_tx, .. } => {\n                    batch_tx\n                        .send(CcfhTrainJob {\n                            samples: ccfh_samples,\n                            done: done_tx,\n                        })\n                        .await\n                        .map_err(|err| {\n                            trc::EventType::Server(trc::ServerEvent::ThreadError)\n                                .reason(err)\n                                .details(\"Spam train task failed\")\n                                .caused_by(trc::location!())\n                        })?;\n                }\n            }\n\n            done_rx.await.map_err(|err| {\n                trc::EventType::Server(trc::ServerEvent::ThreadError)\n                    .reason(err)\n                    .details(\"Spam train task failed\")\n                    .caused_by(trc::location!())\n            })?;\n        }\n\n        // Take ownership of trainer\n        trainer.trainer = match task {\n            TrainTask::Fh {\n                batch_tx,\n                trainer_rx,\n                ..\n            } => {\n                drop(batch_tx);\n                SpamTrainerClass::FtrlFh(trainer_rx.await.map_err(|err| {\n                    trc::EventType::Server(trc::ServerEvent::ThreadError)\n                        .reason(err)\n                        .details(\"Spam train task failed\")\n                        .caused_by(trc::location!())\n                })?)\n            }\n            TrainTask::Ccfh {\n                batch_tx,\n                trainer_rx,\n                ..\n            } => {\n                drop(batch_tx);\n                SpamTrainerClass::FtrlCfh(trainer_rx.await.map_err(|err| {\n                    trc::EventType::Server(trc::ServerEvent::ThreadError)\n                        .reason(err)\n                        .details(\"Spam train task failed\")\n                        .caused_by(trc::location!())\n                })?)\n            }\n        };\n\n        // Store updated trainer and classifier\n        let ham_count = trainer.reservoir.ham.total_seen;\n        let spam_count = trainer.reservoir.spam.total_seen;\n        let classifier = Archiver::new(match &trainer.trainer {\n            SpamTrainerClass::FtrlFh(fh_trainer) => spamfilter::SpamClassifier::FhClassifier {\n                classifier: fh_trainer.build_classifier(),\n                last_trained_at: now(),\n            },\n            SpamTrainerClass::FtrlCfh(ccfh_trainer) => spamfilter::SpamClassifier::CcfhClassifier {\n                classifier: ccfh_trainer.build_classifier(),\n                last_trained_at: now(),\n            },\n        });\n        self.blob_store()\n            .put_blob(\n                SPAM_TRAINER_KEY,\n                &Archiver::new(trainer)\n                    .serialize()\n                    .caused_by(trc::location!())?,\n            )\n            .await\n            .caused_by(trc::location!())?;\n        self.blob_store()\n            .put_blob(\n                SPAM_CLASSIFIER_KEY,\n                &classifier.serialize().caused_by(trc::location!())?,\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n        self.inner\n            .data\n            .spam_classifier\n            .store(Arc::new(classifier.inner));\n        self.cluster_broadcast(BroadcastEvent::ReloadSpamFilter)\n            .await;\n\n        trc::event!(\n            Spam(SpamEvent::TrainCompleted),\n            Total = num_samples,\n            Details = vec![trc::Value::from(ham_count), trc::Value::from(spam_count)],\n            Elapsed = started.elapsed()\n        );\n\n        // Remove samples marked for deletion\n        if remove_entries {\n            let mut batch = BatchBuilder::new();\n            for sample in samples {\n                if let Some(until) = sample.remove {\n                    batch\n                        .with_account_id(sample.sample.account_id)\n                        .clear(BlobOp::Link {\n                            hash: sample.sample.hash.clone(),\n                            to: BlobLink::Temporary { until },\n                        })\n                        .clear(BlobOp::SpamSample {\n                            hash: sample.sample.hash,\n                            until,\n                        });\n                    if batch.is_large_batch() {\n                        self.store()\n                            .write(batch.build_all())\n                            .await\n                            .caused_by(trc::location!())?;\n                        batch = BatchBuilder::new();\n                    }\n                }\n            }\n            if !batch.is_empty() {\n                self.store()\n                    .write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n        }\n\n        Ok(())\n    }\n\n    async fn spam_classify(&self, ctx: &mut SpamFilterContext<'_>) -> trc::Result<()> {\n        let classifier = self.inner.data.spam_classifier.load_full();\n        let Some(config) = &self.core.spam.classifier else {\n            return Ok(());\n        };\n\n        let started = Instant::now();\n        match classifier.as_ref() {\n            spamfilter::SpamClassifier::FhClassifier { classifier, .. } => {\n                let mut classifier_confidence = Vec::with_capacity(ctx.input.env_rcpt_to.len());\n                let mut has_prediction = false;\n                let mut tokens = self.spam_build_tokens(ctx).await.0;\n                let feature_builder = classifier.feature_builder();\n                if config.log_scale {\n                    feature_builder.scale(&mut tokens);\n                }\n\n                for rcpt in &ctx.input.env_rcpt_to {\n                    let prediction = if let Some(account_id) = self\n                        .directory()\n                        .email_to_id(rcpt)\n                        .await\n                        .caused_by(trc::location!())?\n                    {\n                        has_prediction = true;\n                        classifier\n                            .predict_proba_sample(&feature_builder.build(\n                                &tokens,\n                                account_id.into(),\n                                config.l2_normalize,\n                            ))\n                            .into()\n                    } else {\n                        None\n                    };\n                    classifier_confidence.push(prediction);\n                }\n\n                if has_prediction {\n                    ctx.result.classifier_confidence = classifier_confidence;\n                } else {\n                    // None of the recipients are local, default to global model prediction\n                    let prediction = classifier.predict_proba_sample(&feature_builder.build(\n                        &tokens,\n                        None,\n                        config.l2_normalize,\n                    ));\n                    ctx.result.classifier_confidence =\n                        vec![prediction.into(); ctx.input.env_rcpt_to.len()];\n                }\n            }\n            spamfilter::SpamClassifier::CcfhClassifier { classifier, .. } => {\n                let mut classifier_confidence = Vec::with_capacity(ctx.input.env_rcpt_to.len());\n                let mut has_prediction = false;\n                let mut tokens = self.spam_build_tokens(ctx).await.0;\n                let feature_builder = classifier.feature_builder();\n                if config.log_scale {\n                    feature_builder.scale(&mut tokens);\n                }\n\n                for rcpt in &ctx.input.env_rcpt_to {\n                    let prediction = if let Some(account_id) = self\n                        .directory()\n                        .email_to_id(rcpt)\n                        .await\n                        .caused_by(trc::location!())?\n                    {\n                        has_prediction = true;\n                        classifier\n                            .predict_proba_sample(&feature_builder.build(\n                                &tokens,\n                                account_id.into(),\n                                config.l2_normalize,\n                            ))\n                            .into()\n                    } else {\n                        None\n                    };\n                    classifier_confidence.push(prediction);\n                }\n\n                if has_prediction {\n                    ctx.result.classifier_confidence = classifier_confidence;\n                } else {\n                    // None of the recipients are local, default to global model prediction\n                    let prediction = classifier.predict_proba_sample(&feature_builder.build(\n                        &tokens,\n                        None,\n                        config.l2_normalize,\n                    ));\n                    ctx.result.classifier_confidence =\n                        vec![prediction.into(); ctx.input.env_rcpt_to.len()];\n                }\n            }\n            spamfilter::SpamClassifier::Disabled => {\n                return Ok(());\n            }\n        }\n\n        trc::event!(\n            Spam(SpamEvent::Classify),\n            Result = ctx\n                .result\n                .classifier_confidence\n                .iter()\n                .zip(ctx.input.env_rcpt_to.iter())\n                .map(|(v, rcpt)| trc::Value::Array(vec![\n                    trc::Value::from(rcpt.to_string()),\n                    trc::Value::from(*v)\n                ]))\n                .collect::<Vec<_>>(),\n            SpanId = ctx.input.span_id,\n            Elapsed = started.elapsed()\n        );\n\n        Ok(())\n    }\n\n    async fn spam_build_tokens<'x>(&self, ctx: &'x SpamFilterContext<'_>) -> Tokens<'x> {\n        let mut tokens = Tokens::default();\n\n        // Add From addresses\n        if ctx\n            .input\n            .dmarc_result\n            .as_ref()\n            .is_some_and(|result| **result != DmarcResult::Pass)\n        {\n            tokens.insert(Token::Sender { value: \"!\".into() });\n        }\n        for email in [&ctx.output.env_from_addr, &ctx.output.from.email] {\n            tokens.insert_email(email, true);\n        }\n\n        // Add Email addresses\n        for email in &ctx.output.emails {\n            let is_sender = match &email.location {\n                Location::HeaderReplyTo | Location::HeaderDnt => true,\n                Location::BodyText\n                | Location::BodyHtml\n                | Location::Attachment\n                | Location::HeaderSubject => false,\n                _ => continue,\n            };\n\n            if is_sender\n                || !is_trusted_domain(\n                    self,\n                    email.element.email.domain_part.sld_or_default(),\n                    ctx.input.span_id,\n                )\n                .await\n            {\n                tokens.insert_email(&email.element.email, is_sender);\n            }\n        }\n\n        // Add URLs\n        for url in &ctx.output.urls {\n            if let Some(url) = &url.element.url_parsed\n                && !is_trusted_domain(self, url.host.sld_or_default(), ctx.input.span_id).await\n            {\n                if let Some(host) = &url.host.sld {\n                    tokens.insert(Token::Url { value: host.into() });\n                    if host != &url.host.fqdn {\n                        tokens.insert(Token::Url {\n                            value: url.host.fqdn.as_str().into(),\n                        });\n                    }\n                } else {\n                    tokens.insert(Token::Url {\n                        value: url.host.fqdn.as_str().into(),\n                    });\n                }\n                for token in url\n                    .parts\n                    .path()\n                    .split(['/', '.', '_'])\n                    .filter(|v| v.chars().all(|ch| ch.is_alphabetic()))\n                {\n                    if token.len() > 2 {\n                        let token = truncate_word(token, MAX_TOKEN_LENGTH);\n                        tokens.insert(Token::Url {\n                            value: format!(\"_{token}\").into(),\n                        });\n                    }\n                }\n            }\n        }\n\n        // Add hostnames\n        for domain in &ctx.output.domains {\n            if matches!(\n                domain.location,\n                Location::HeaderReceived | Location::HeaderMid | Location::Ehlo | Location::Tcp\n            ) {\n                let host = Hostname::new(&domain.element);\n                let host_sld = host.sld_or_default();\n\n                if !is_trusted_domain(self, host_sld, ctx.input.span_id).await {\n                    if !host_sld.is_empty() && host_sld != host.fqdn {\n                        tokens.insert(Token::Hostname {\n                            value: host_sld.to_string().into(),\n                        });\n                    }\n\n                    tokens.insert(Token::Hostname {\n                        value: host.fqdn.into(),\n                    });\n                }\n            }\n        }\n\n        // Add ASN\n        if let Some(asn) = ctx.input.asn {\n            tokens.insert(Token::Asn {\n                number: asn.to_be_bytes(),\n            });\n        }\n\n        // Add MIME and attachment indicators\n        for part in &ctx.input.message.parts {\n            if let Some(name) = part.attachment_name()\n                && let Some((name, ext)) = name.rsplit_once('.')\n            {\n                if !ext.is_empty() {\n                    tokens.insert(Token::Attachment {\n                        value: lower_prefix(\"!\", truncate_word(ext, MAX_TOKEN_LENGTH)).into(),\n                    });\n                }\n                let name = name.to_lowercase();\n                let word_tokenizer = WordStemTokenizer::new(&name);\n                for token in TypesTokenizer::new(&name) {\n                    if let TokenType::Alphabetic(word) = token.word {\n                        word_tokenizer.tokenize(word, |token| {\n                            tokens.insert(Token::Attachment {\n                                value: format!(\n                                    \"_{}\",\n                                    truncate_word(token.as_ref(), MAX_TOKEN_LENGTH)\n                                )\n                                .into(),\n                            });\n                        });\n                    }\n                }\n            }\n\n            if let Some(ct) = part.content_type() {\n                let mut ct_lower = String::with_capacity(\n                    ct.c_type.len() + ct.c_subtype.as_ref().map_or(0, |s| s.len()),\n                );\n                for ch in ct.c_type.chars() {\n                    ct_lower.push(ch.to_ascii_lowercase());\n                }\n                if let Some(st) = &ct.c_subtype {\n                    ct_lower.push('/');\n                    for ch in st.chars() {\n                        ct_lower.push(ch.to_ascii_lowercase());\n                    }\n                }\n\n                tokens.insert(Token::MimeType { value: ct_lower });\n            }\n        }\n\n        // Tokenize the subject\n        for token in &ctx.output.subject_tokens {\n            tokens.insert_type(\n                &WordStemTokenizer::new(&ctx.output.subject_thread_lc),\n                token,\n                false,\n            );\n        }\n\n        // Tokenize the text parts\n        let body_idx = ctx\n            .input\n            .message\n            .html_body\n            .first()\n            .or_else(|| ctx.input.message.text_body.first())\n            .map(|idx| *idx as usize);\n        let mut alt_tokens = Tokens::default();\n        for (idx, part) in ctx.output.text_parts.iter().enumerate() {\n            let is_body = Some(idx) == body_idx;\n            if is_body\n                || (!ctx.input.message.text_body.contains(&(idx as u32))\n                    && !ctx.input.message.html_body.contains(&(idx as u32)))\n            {\n                tokens.insert_text_part(part, is_body);\n            } else {\n                alt_tokens.insert_text_part(part, false);\n            }\n        }\n        if !alt_tokens.0.is_empty() {\n            for (token, count) in alt_tokens.0.into_iter() {\n                if let Entry::Vacant(entry) = tokens.0.entry(token) {\n                    entry.insert(count);\n                }\n            }\n        }\n\n        tokens\n    }\n}\n\nstruct FhTrainJob {\n    samples: Vec<Sample<FhFeature>>,\n    done: oneshot::Sender<()>,\n}\n\nstruct CcfhTrainJob {\n    samples: Vec<Sample<CcfhFeature>>,\n    done: oneshot::Sender<()>,\n}\n\nenum TrainTask {\n    Fh {\n        batch_tx: mpsc::Sender<FhTrainJob>,\n        trainer_rx: oneshot::Receiver<Box<FhTrainer<Ftrl>>>,\n        builder: FhFeatureBuilder,\n    },\n    Ccfh {\n        batch_tx: mpsc::Sender<CcfhTrainJob>,\n        trainer_rx: oneshot::Receiver<Box<CcfhTrainer<Ftrl, Ftrl>>>,\n        builder: CcfhFeatureBuilder,\n    },\n}\n\nimpl SpamTrainerClass {\n    fn spawn(self, num_epochs: usize) -> trc::Result<TrainTask> {\n        match self {\n            SpamTrainerClass::FtrlFh(mut trainer) => {\n                let builder = trainer.feature_builder();\n                let (batch_tx, mut batch_rx) = mpsc::channel::<FhTrainJob>(1);\n                let (trainer_tx, trainer_rx) = oneshot::channel();\n\n                std::thread::Builder::new()\n                    .name(\"FTRL Train Task\".into())\n                    .spawn(move || {\n                        while let Some(mut job) = batch_rx.blocking_recv() {\n                            trainer.fit(&mut job.samples, num_epochs);\n                            let _ = job.done.send(());\n                        }\n                        // Send trainer back when done\n                        let _ = trainer_tx.send(trainer);\n                    })\n                    .map_err(|err| {\n                        trc::EventType::Server(trc::ServerEvent::ThreadError)\n                            .reason(err)\n                            .details(\"Failed to spawn spam train task\")\n                            .caused_by(trc::location!())\n                    })?;\n\n                Ok(TrainTask::Fh {\n                    batch_tx,\n                    trainer_rx,\n                    builder,\n                })\n            }\n            SpamTrainerClass::FtrlCfh(mut trainer) => {\n                let builder = trainer.feature_builder();\n                let (batch_tx, mut batch_rx) = mpsc::channel::<CcfhTrainJob>(1);\n                let (trainer_tx, trainer_rx) = oneshot::channel();\n\n                std::thread::Builder::new()\n                    .name(\"FTRL Train Task\".into())\n                    .spawn(move || {\n                        while let Some(mut job) = batch_rx.blocking_recv() {\n                            trainer.fit(&mut job.samples, num_epochs);\n                            let _ = job.done.send(());\n                        }\n                        // Send trainer back when done\n                        let _ = trainer_tx.send(trainer);\n                    })\n                    .map_err(|err| {\n                        trc::EventType::Server(trc::ServerEvent::ThreadError)\n                            .reason(err)\n                            .details(\"Failed to spawn spam train task\")\n                            .caused_by(trc::location!())\n                    })?;\n\n                Ok(TrainTask::Ccfh {\n                    batch_tx,\n                    trainer_rx,\n                    builder,\n                })\n            }\n        }\n    }\n}\n\nconst MAX_TOKEN_LENGTH: usize = 16;\n\n#[derive(\n    Debug, Clone, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize, PartialOrd, Ord,\n)]\n#[serde(tag = \"type\", rename_all = \"snake_case\")]\npub enum Token<'x> {\n    Word { value: Cow<'x, str> },\n    Number { code: [u8; 2] },\n    Alphanumeric { code: [u8; 4] },\n    UnicodeCategory { value: &'x str },\n    Sender { value: Cow<'x, str> },\n    Asn { number: [u8; 4] },\n    Url { value: Cow<'x, str> },\n    Email { value: Cow<'x, str> },\n    Hostname { value: Cow<'x, str> },\n    Attachment { value: Cow<'x, str> },\n    MimeType { value: String },\n    HtmlImage { src: &'x str },\n    HtmlAnchor { href: &'x str },\n}\n\n#[derive(Debug)]\npub struct Tokens<'x>(pub HashMap<Token<'x>, f32, RandomState>);\n\nimpl<'x> Tokens<'x> {\n    fn insert_text_part(&mut self, part: &'x TextPart<'x>, is_body: bool) {\n        match part {\n            TextPart::Plain { text_body, tokens } => {\n                let word_tokenizer = WordStemTokenizer::new(text_body);\n\n                for token in tokens {\n                    self.insert_type(&word_tokenizer, token, is_body);\n                }\n\n                if is_body\n                    && (tokens.is_empty()\n                        || !tokens.iter().any(|t| matches!(t, TokenType::Alphabetic(_))))\n                {\n                    self.insert(Token::Word {\n                        value: \"_null\".into(),\n                    });\n                }\n            }\n            TextPart::Html {\n                text_body,\n                tokens,\n                html_tokens,\n            } => {\n                let word_tokenizer = WordStemTokenizer::new(text_body);\n\n                for token in tokens {\n                    self.insert_type(&word_tokenizer, token, is_body);\n                }\n\n                if is_body {\n                    if tokens.is_empty()\n                        || !tokens.iter().any(|t| matches!(t, TokenType::Alphabetic(_)))\n                    {\n                        self.insert(Token::Word {\n                            value: \"_null\".into(),\n                        });\n                    }\n\n                    for token in html_tokens {\n                        if let HtmlToken::StartTag {\n                            name: A | IMG,\n                            attributes,\n                            ..\n                        } = token\n                        {\n                            for (name, value) in attributes {\n                                match (*name, value) {\n                                    (ALT | TITLE, Some(value)) => {\n                                        for token in TypesTokenizer::new(value) {\n                                            self.insert_type(&word_tokenizer, &token.word, is_body);\n                                        }\n                                    }\n                                    (SRC, Some(value)) => {\n                                        self.insert(Token::HtmlImage {\n                                            src: value.split_once(':').unwrap_or_default().0,\n                                        });\n                                    }\n                                    (HREF, Some(value)) => {\n                                        self.insert(Token::HtmlAnchor {\n                                            href: value.split_once(':').unwrap_or_default().0,\n                                        });\n                                    }\n                                    _ => {}\n                                }\n                            }\n                        }\n                    }\n                }\n            }\n            TextPart::None => (),\n        }\n    }\n\n    fn insert_type<T: AsRef<str>, E, U, I>(\n        &mut self,\n        word_tokenizer: &WordStemTokenizer,\n        token: &TokenType<T, E, U, I>,\n        is_body: bool,\n    ) {\n        match token {\n            TokenType::Alphabetic(word) => {\n                let word = word.as_ref();\n                let mut set: Option<AugmentedScriptSet> = None;\n                let mut has_confusables = false;\n                let mut upper_count = 0;\n                for ch in word.chars() {\n                    if ch.is_uppercase() {\n                        upper_count += 1;\n                    }\n\n                    has_confusables |=\n                        !ch.is_ascii() && !std::iter::once(ch).nfc().eq(std::iter::once(ch).nfkc());\n                    set.get_or_insert_default().intersect_with(ch.into());\n                }\n                let is_mixed_script = set.is_some_and(|set| set.is_empty());\n\n                if (is_mixed_script || has_confusables)\n                    && let Ok(cured_word) = decancer::cure(word, decancer::Options::default())\n                {\n                    if word.len() > MAX_TOKEN_LENGTH {\n                        self.insert(Token::Word {\n                            value: truncate_word(cured_word.as_str(), MAX_TOKEN_LENGTH)\n                                .to_string()\n                                .into(),\n                        });\n                    } else {\n                        self.insert(Token::Word {\n                            value: String::from(cured_word).into(),\n                        });\n                    }\n                } else {\n                    let word = word.to_lowercase();\n                    word_tokenizer.tokenize(&word, |token| {\n                        self.insert(Token::Word {\n                            value: truncate_word(token.as_ref(), MAX_TOKEN_LENGTH)\n                                .to_string()\n                                .into(),\n                        });\n                    });\n                }\n\n                if is_body && word.len() == upper_count && word.len() > 3 {\n                    self.insert(Token::Word {\n                        value: \"_allcaps\".into(),\n                    });\n                }\n            }\n            TokenType::Alphanumeric(word) => {\n                self.insert(Token::from_alphanumeric(word.as_ref()));\n            }\n            TokenType::UrlNoHost(url) => {\n                for token in url\n                    .as_ref()\n                    .to_lowercase()\n                    .split(['/', '.', '_'])\n                    .filter(|v| v.chars().all(|ch| ch.is_alphabetic()))\n                {\n                    if token.len() > 2 {\n                        let token = truncate_word(token, MAX_TOKEN_LENGTH);\n                        self.insert(Token::Url {\n                            value: format!(\"_{token}\").into(),\n                        });\n                    }\n                }\n            }\n            TokenType::Other(ch) | TokenType::Punctuation(ch) => {\n                let category = get_general_category(*ch);\n                if !matches!(\n                    category,\n                    GeneralCategory::ClosePunctuation\n                        | GeneralCategory::ConnectorPunctuation\n                        | GeneralCategory::DashPunctuation\n                        | GeneralCategory::FinalPunctuation\n                        | GeneralCategory::InitialPunctuation\n                        | GeneralCategory::OpenPunctuation\n                        | GeneralCategory::OtherPunctuation\n                        | GeneralCategory::SpaceSeparator\n                ) {\n                    self.insert(Token::UnicodeCategory {\n                        value: category.abbreviation(),\n                    });\n                }\n            }\n            TokenType::Integer(word) => {\n                self.insert(Token::from_number(false, word.as_ref()));\n            }\n            TokenType::Float(word) => {\n                self.insert(Token::from_number(true, word.as_ref()));\n            }\n            TokenType::IpAddr(_) => {\n                self.insert(Token::Url {\n                    value: \"!ip\".into(),\n                });\n            }\n            TokenType::Email(_)\n            | TokenType::Url(_)\n            | TokenType::UrlNoScheme(_)\n            | TokenType::Space => {}\n        }\n    }\n\n    fn insert(&mut self, token: Token<'x>) {\n        *self.0.entry(token).or_insert(0.0) += 1.0;\n    }\n\n    fn insert_if_missing(&mut self, token: Token<'x>) {\n        self.0.entry(token).or_insert(1.0);\n    }\n\n    fn insert_email(&mut self, email: &'x Email, is_sender: bool) {\n        if !email.address.is_empty() {\n            if is_sender {\n                self.insert_if_missing(Token::Sender {\n                    value: email.address.as_str().into(),\n                });\n                self.insert_if_missing(Token::Sender {\n                    value: email.domain_part.fqdn.as_str().into(),\n                });\n                if let Some(sld) = &email.domain_part.sld\n                    && sld != &email.domain_part.fqdn\n                {\n                    self.insert_if_missing(Token::Sender { value: sld.into() });\n                }\n            } else {\n                self.insert_if_missing(Token::Email {\n                    value: email.address.as_str().into(),\n                });\n                self.insert_if_missing(Token::Email {\n                    value: email.domain_part.fqdn.as_str().into(),\n                });\n                if let Some(sld) = &email.domain_part.sld\n                    && !sld.is_empty()\n                    && sld != &email.domain_part.fqdn\n                {\n                    self.insert_if_missing(Token::Email { value: sld.into() });\n                }\n            }\n        }\n    }\n}\n\nimpl Token<'static> {\n    fn from_alphanumeric(s: &str) -> Self {\n        let mut is_hex = true;\n        let mut is_ascii = true;\n        let mut digit_count = 0;\n\n        for ch in s.chars() {\n            match ch {\n                'a'..='f' | 'A'..='F' => {}\n                '0'..='9' => {\n                    digit_count += 1;\n                }\n                _ => {\n                    is_ascii &= ch.is_ascii();\n                    is_hex = false;\n                }\n            }\n        }\n\n        if is_hex {\n            Token::Number {\n                code: [b'X', s.len().min(u8::MAX as usize) as u8],\n            }\n        } else if !is_ascii {\n            let word: String = if let Ok(cured) = decancer::cure(s, decancer::Options::default()) {\n                cured\n                    .as_str()\n                    .chars()\n                    .filter(|ch| ch.is_alphabetic())\n                    .take(MAX_TOKEN_LENGTH)\n                    .collect()\n            } else {\n                s.chars()\n                    .filter(|ch| ch.is_alphabetic())\n                    .flat_map(|ch| ch.to_lowercase())\n                    .take(MAX_TOKEN_LENGTH)\n                    .collect()\n            };\n\n            Token::Word { value: word.into() }\n        } else if s.len() > 3 && digit_count == 1 {\n            let word: String = s\n                .chars()\n                .filter(|ch| ch.is_alphabetic())\n                .flat_map(|ch| ch.to_lowercase())\n                .take(MAX_TOKEN_LENGTH)\n                .collect();\n            Token::Word { value: word.into() }\n        } else {\n            // Character class counts\n            let mut upper = 0u32;\n            let mut lower = 0u32;\n            let mut digit = 0u32;\n            let mut len = 0;\n            let mut char_types = Vec::with_capacity(len);\n            for c in s.chars() {\n                let char_type = CharType::from_char(c);\n                char_types.push(char_type);\n                match char_type {\n                    CharType::Upper => upper += 1,\n                    CharType::Lower => lower += 1,\n                    CharType::Digit => digit += 1,\n                    CharType::Other => (),\n                }\n                len += 1;\n            }\n\n            // Determine dominant composition\n            let composition = match (upper > 0, lower > 0, digit > 0) {\n                (true, false, false) => b'U',  // UPPERCASE only\n                (false, true, false) => b'L',  // lowercase only\n                (false, false, true) => b'D',  // digits only\n                (true, true, false) => b'A',   // Alphabetic mixed case\n                (true, false, true) => b'H',   // Upper + digits (common in codes)\n                (false, true, true) => b'M',   // lower + digits (common in identifiers)\n                (true, true, true) => b'X',    // eXtreme mix - all three\n                (false, false, false) => b'E', // empty/invalid\n            };\n\n            // Length bucket (log-ish scale)\n            let len_code = match len {\n                1 => b'1',\n                2 => b'2',\n                3 => b'3',\n                4 => b'4',\n                5..=6 => b'5',\n                7..=8 => b'6',\n                9..=12 => b'7',\n                13..=16 => b'8',\n                17..=32 => b'9',\n                _ => b'Z',\n            };\n\n            // Ratio encoding (which class dominates)\n            let max_count = upper.max(lower).max(digit);\n            let dominance = (max_count * 100) / len.min(1) as u32;\n            let ratio = match dominance {\n                0..=50 => b'B',  // Balanced\n                51..=75 => b'P', // Partial dominance\n                76..=99 => b'D', // Dominant\n                _ => b'O',       // One class only (100%)\n            };\n\n            // Run code\n            let mut run_count = 0;\n            if len > 1 {\n                let mut prev_type = char_types[0];\n                for &current_type in char_types.iter().skip(1) {\n                    if current_type != prev_type {\n                        run_count += 1;\n                        prev_type = current_type;\n                    }\n                }\n            }\n            let run_ratio = (run_count as f64) / ((len - 1) as f64);\n            let run_code = match run_ratio {\n                r if r <= 0.1 => b'0', // Very long runs (e.g., AAAABBBB)\n                r if r <= 0.3 => b'1', // Moderate runs\n                r if r <= 0.5 => b'2', // Balanced runs/alternation\n                r if r <= 0.7 => b'3', // High alternation\n                _ => b'4',             // Near maximum alternation (e.g., A1A1A1)\n            };\n\n            Token::Alphanumeric {\n                code: [composition, len_code, ratio, run_code],\n            }\n        }\n    }\n\n    fn from_number(is_float: bool, num: &str) -> Self {\n        Token::Number {\n            code: [\n                if num.starts_with(\"-\") {\n                    if is_float { b'F' } else { b'I' }\n                } else if is_float {\n                    b'f'\n                } else {\n                    b'i'\n                },\n                num.as_bytes()\n                    .iter()\n                    .filter(|c| c.is_ascii_digit())\n                    .count()\n                    .min(u8::MAX as usize) as u8,\n            ],\n        }\n    }\n}\n\nfn lower_prefix(prefix: &str, value: &str) -> String {\n    let mut result = String::with_capacity(prefix.len() + value.len());\n    result.push_str(prefix);\n    for ch in value.chars() {\n        for lower_ch in ch.to_lowercase() {\n            result.push(lower_ch);\n        }\n    }\n    result\n}\n\nfn truncate_word(word: &str, max_len: usize) -> &str {\n    if word.len() <= max_len {\n        word\n    } else {\n        let mut pos = 0;\n        for (count, (idx, _)) in word.char_indices().enumerate() {\n            pos = idx;\n            if count == max_len {\n                break;\n            }\n        }\n        &word[..pos]\n    }\n}\n\nimpl UnprocessedFeature for Token<'_> {\n    fn prefix(&self) -> u16 {\n        match self {\n            Token::Word { .. } => 0,\n            Token::Number { .. } => 1,\n            Token::Alphanumeric { .. } => 2,\n            Token::UnicodeCategory { .. } => 3,\n            Token::Sender { .. } => 4,\n            Token::Asn { .. } => 5,\n            Token::Url { .. } => 6,\n            Token::Email { .. } => 7,\n            Token::Hostname { .. } => 8,\n            Token::Attachment { .. } => 9,\n            Token::MimeType { .. } => 10,\n            Token::HtmlImage { .. } => 11,\n            Token::HtmlAnchor { .. } => 12,\n        }\n    }\n\n    fn value(&self) -> &[u8] {\n        match self {\n            Token::Word { value } => value.as_bytes(),\n            Token::Number { code } => code,\n            Token::Alphanumeric { code } => code,\n            Token::UnicodeCategory { value } => value.as_bytes(),\n            Token::Sender { value } => value.as_bytes(),\n            Token::Asn { number } => number,\n            Token::Url { value } => value.as_bytes(),\n            Token::Email { value } => value.as_bytes(),\n            Token::Hostname { value } => value.as_bytes(),\n            Token::Attachment { value } => value.as_bytes(),\n            Token::MimeType { value } => value.as_bytes(),\n            Token::HtmlImage { src } => src.as_bytes(),\n            Token::HtmlAnchor { href } => href.as_bytes(),\n        }\n    }\n}\n\n#[derive(Debug, PartialEq, Eq, Clone, Copy)]\nenum CharType {\n    Upper,\n    Lower,\n    Digit,\n    Other,\n}\n\nimpl CharType {\n    fn from_char(c: char) -> CharType {\n        match c {\n            'A'..='Z' => CharType::Upper,\n            'a'..='z' => CharType::Lower,\n            '0'..='9' => CharType::Digit,\n            _ => CharType::Other,\n        }\n    }\n}\n\nimpl<'x> Default for Tokens<'x> {\n    fn default() -> Self {\n        Tokens(HashMap::with_capacity(128))\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/modules/dnsbl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    net::Ipv4Addr,\n    sync::Arc,\n    time::{Duration, Instant},\n};\n\nuse common::{\n    Server,\n    config::spamfilter::{DnsBlServer, Element, IpResolver, Location},\n    expr::functions::ResolveVariable,\n};\n\nuse mail_auth::{Error, common::resolver::IntoFqdn};\nuse trc::SpamEvent;\n\nuse crate::SpamFilterContext;\n\nuse super::expression::SpamFilterResolver;\n\npub(crate) async fn check_dnsbl(\n    server: &Server,\n    ctx: &mut SpamFilterContext<'_>,\n    resolver: &impl ResolveVariable,\n    scope: Element,\n    location: Location,\n) {\n    let (mut checks, max_checks) = match scope {\n        Element::Email => (\n            ctx.result.rbl_email_checks,\n            server.core.spam.dnsbl.max_email_checks,\n        ),\n        Element::Ip => (\n            ctx.result.rbl_ip_checks,\n            server.core.spam.dnsbl.max_ip_checks,\n        ),\n        Element::Url => (\n            ctx.result.rbl_url_checks,\n            server.core.spam.dnsbl.max_url_checks,\n        ),\n        Element::Domain => (\n            ctx.result.rbl_domain_checks,\n            server.core.spam.dnsbl.max_domain_checks,\n        ),\n        Element::Header | Element::Body | Element::Any => unreachable!(),\n    };\n\n    for dnsbl in &server.core.spam.dnsbl.servers {\n        if dnsbl.scope == scope\n            && checks < max_checks\n            && let Some(tag) = is_dnsbl(\n                server,\n                dnsbl,\n                SpamFilterResolver::new(ctx, resolver, location),\n                scope,\n                &mut checks,\n            )\n            .await\n        {\n            ctx.result.add_tag(tag);\n        }\n    }\n\n    match scope {\n        Element::Email => ctx.result.rbl_email_checks = checks,\n        Element::Ip => ctx.result.rbl_ip_checks = checks,\n        Element::Url => ctx.result.rbl_url_checks = checks,\n        Element::Domain => ctx.result.rbl_domain_checks = checks,\n        Element::Header | Element::Body | Element::Any => unreachable!(),\n    }\n}\n\nasync fn is_dnsbl(\n    server: &Server,\n    config: &DnsBlServer,\n    resolver: SpamFilterResolver<'_, impl ResolveVariable>,\n    element: Element,\n    checks: &mut usize,\n) -> Option<String> {\n    let time = Instant::now();\n    let zone = server\n        .eval_if::<String, _>(&config.zone, &resolver, resolver.ctx.input.span_id)\n        .await?;\n\n    #[cfg(feature = \"test_mode\")]\n    {\n        if zone.contains(\".11.20.\") {\n            let parts = zone.split('.').collect::<Vec<_>>();\n\n            return if config.tags.if_then.iter().any(|i| i.expr.items.len() == 3) && parts[0] != \"2\"\n            {\n                None\n            } else {\n                server\n                    .eval_if(\n                        &config.tags,\n                        &SpamFilterResolver::new(\n                            resolver.ctx,\n                            &IpResolver::new(\n                                format!(\"127.0.{}.{}\", parts[1], parts[0]).parse().unwrap(),\n                            ),\n                            resolver.location,\n                        ),\n                        resolver.ctx.input.span_id,\n                    )\n                    .await\n            };\n        }\n    }\n\n    let result = match server.inner.cache.dns_rbl.get(zone.as_str()) {\n        Some(Some(result)) => result,\n        Some(None) => return None,\n        None => {\n            *checks += 1;\n\n            match server\n                .core\n                .smtp\n                .resolvers\n                .dns\n                .ipv4_lookup_raw((&zone).into_fqdn().as_ref())\n                .await\n            {\n                Ok(result) => {\n                    trc::event!(\n                        Spam(SpamEvent::Dnsbl),\n                        Hostname = zone.clone(),\n                        Result = result\n                            .entry\n                            .iter()\n                            .map(|ip| trc::Value::from(ip.to_string()))\n                            .collect::<Vec<_>>(),\n                        Details = element.as_str(),\n                        Elapsed = time.elapsed()\n                    );\n\n                    let entry = Arc::new(IpResolver::new(\n                        result\n                            .entry\n                            .iter()\n                            .copied()\n                            .next()\n                            .unwrap_or(Ipv4Addr::BROADCAST)\n                            .into(),\n                    ));\n\n                    server.inner.cache.dns_rbl.insert_with_expiry(\n                        zone.to_string(),\n                        Some(entry.clone()),\n                        result.expires,\n                    );\n\n                    entry\n                }\n                Err(Error::DnsRecordNotFound(_)) => {\n                    trc::event!(\n                        Spam(SpamEvent::Dnsbl),\n                        Hostname = zone.clone(),\n                        Result = trc::Value::None,\n                        Details = element.as_str(),\n                        Elapsed = time.elapsed()\n                    );\n\n                    server.inner.cache.dns_rbl.insert(\n                        zone.to_string(),\n                        None,\n                        Duration::from_secs(86400),\n                    );\n\n                    return None;\n                }\n                Err(err) => {\n                    trc::event!(\n                        Spam(SpamEvent::DnsblError),\n                        Hostname = zone,\n                        Elapsed = time.elapsed(),\n                        Details = element.as_str(),\n                        CausedBy = err.to_string()\n                    );\n\n                    return None;\n                }\n            }\n        }\n    };\n\n    server\n        .eval_if(\n            &config.tags,\n            &SpamFilterResolver::new(resolver.ctx, result.as_ref(), resolver.location),\n            resolver.ctx.input.span_id,\n        )\n        .await\n}\n"
  },
  {
    "path": "crates/spam-filter/src/modules/expression.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    config::spamfilter::*,\n    expr::{StringCow, Variable, functions::ResolveVariable},\n};\nuse compact_str::{CompactString, ToCompactString, format_compact};\nuse mail_parser::{Header, HeaderValue};\nuse nlp::tokenizers::types::TokenType;\n\nuse crate::{Recipient, SpamFilterContext, TextPart, analysis::url::UrlParts};\n\npub(crate) struct SpamFilterResolver<'x, T: ResolveVariable> {\n    pub ctx: &'x SpamFilterContext<'x>,\n    pub item: &'x T,\n    pub location: Location,\n}\n\nimpl<'x, T: ResolveVariable> SpamFilterResolver<'x, T> {\n    pub fn new(ctx: &'x SpamFilterContext<'x>, item: &'x T, location: Location) -> Self {\n        Self {\n            ctx,\n            item,\n            location,\n        }\n    }\n}\n\nimpl<T: ResolveVariable> ResolveVariable for SpamFilterResolver<'_, T> {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        match variable {\n            0..100 => self.item.resolve_variable(variable),\n            V_SPAM_REMOTE_IP => self.ctx.input.remote_ip.to_compact_string().into(),\n            V_SPAM_REMOTE_IP_PTR => self\n                .ctx\n                .output\n                .iprev_ptr\n                .as_deref()\n                .unwrap_or_default()\n                .into(),\n            V_SPAM_EHLO_DOMAIN => self.ctx.output.ehlo_host.fqdn.as_str().into(),\n            V_SPAM_AUTH_AS => self.ctx.input.authenticated_as.unwrap_or_default().into(),\n            V_SPAM_ASN => self.ctx.input.asn.unwrap_or_default().into(),\n            V_SPAM_COUNTRY => self.ctx.input.country.unwrap_or_default().into(),\n            V_SPAM_IS_TLS => self.ctx.input.is_tls.into(),\n            V_SPAM_ENV_FROM => self.ctx.output.env_from_addr.address.as_str().into(),\n            V_SPAM_ENV_FROM_LOCAL => self.ctx.output.env_from_addr.local_part.as_str().into(),\n            V_SPAM_ENV_FROM_DOMAIN => self\n                .ctx\n                .output\n                .env_from_addr\n                .domain_part\n                .fqdn\n                .as_str()\n                .into(),\n            V_SPAM_ENV_TO => self\n                .ctx\n                .output\n                .env_to_addr\n                .iter()\n                .map(|e| Variable::from(e.address.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_FROM => self.ctx.output.from.email.address.as_str().into(),\n            V_SPAM_FROM_NAME => self\n                .ctx\n                .output\n                .from\n                .name\n                .as_deref()\n                .unwrap_or_default()\n                .into(),\n            V_SPAM_FROM_LOCAL => self.ctx.output.from.email.local_part.as_str().into(),\n            V_SPAM_FROM_DOMAIN => self.ctx.output.from.email.domain_part.fqdn.as_str().into(),\n            V_SPAM_REPLY_TO => self\n                .ctx\n                .output\n                .reply_to\n                .as_ref()\n                .map(|r| r.email.address.as_str())\n                .unwrap_or_default()\n                .into(),\n            V_SPAM_REPLY_TO_NAME => self\n                .ctx\n                .output\n                .reply_to\n                .as_ref()\n                .and_then(|r| r.name.as_deref())\n                .unwrap_or_default()\n                .into(),\n            V_SPAM_REPLY_TO_LOCAL => self\n                .ctx\n                .output\n                .reply_to\n                .as_ref()\n                .map(|r| r.email.local_part.as_str())\n                .unwrap_or_default()\n                .into(),\n            V_SPAM_REPLY_TO_DOMAIN => self\n                .ctx\n                .output\n                .reply_to\n                .as_ref()\n                .map(|r| r.email.domain_part.fqdn.as_str())\n                .unwrap_or_default()\n                .into(),\n            V_SPAM_TO => self\n                .ctx\n                .output\n                .recipients_to\n                .iter()\n                .map(|r| Variable::from(r.email.address.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_TO_NAME => self\n                .ctx\n                .output\n                .recipients_to\n                .iter()\n                .filter_map(|r| Variable::from(r.name.as_deref()?).into())\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_TO_LOCAL => self\n                .ctx\n                .output\n                .recipients_to\n                .iter()\n                .map(|r| Variable::from(r.email.local_part.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_TO_DOMAIN => self\n                .ctx\n                .output\n                .recipients_to\n                .iter()\n                .map(|r| Variable::from(r.email.domain_part.fqdn.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_CC => self\n                .ctx\n                .output\n                .recipients_cc\n                .iter()\n                .map(|r| Variable::from(r.email.address.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_CC_NAME => self\n                .ctx\n                .output\n                .recipients_cc\n                .iter()\n                .filter_map(|r| Variable::from(r.name.as_deref()?).into())\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_CC_LOCAL => self\n                .ctx\n                .output\n                .recipients_cc\n                .iter()\n                .map(|r| Variable::from(r.email.local_part.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_CC_DOMAIN => self\n                .ctx\n                .output\n                .recipients_cc\n                .iter()\n                .map(|r| Variable::from(r.email.domain_part.fqdn.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_BCC => self\n                .ctx\n                .output\n                .recipients_bcc\n                .iter()\n                .map(|r| Variable::from(r.email.address.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_BCC_NAME => self\n                .ctx\n                .output\n                .recipients_bcc\n                .iter()\n                .filter_map(|r| Variable::from(r.name.as_deref()?).into())\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_BCC_LOCAL => self\n                .ctx\n                .output\n                .recipients_bcc\n                .iter()\n                .map(|r| Variable::from(r.email.local_part.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_BCC_DOMAIN => self\n                .ctx\n                .output\n                .recipients_bcc\n                .iter()\n                .map(|r| Variable::from(r.email.domain_part.fqdn.as_str()))\n                .collect::<Vec<_>>()\n                .into(),\n            V_SPAM_BODY_TEXT => self.ctx.text_body().unwrap_or_default().into(),\n            V_SPAM_BODY_HTML => self\n                .ctx\n                .input\n                .message\n                .html_body\n                .first()\n                .and_then(|idx| self.ctx.output.text_parts.get(*idx as usize))\n                .map(|part| {\n                    if let TextPart::Html { text_body, .. } = part {\n                        text_body.as_str()\n                    } else {\n                        \"\"\n                    }\n                })\n                .unwrap_or_default()\n                .into(),\n            V_SPAM_BODY_RAW => Variable::from(CompactString::from_utf8_lossy(\n                self.ctx.input.message.raw_message(),\n            )),\n            V_SPAM_SUBJECT => self.ctx.output.subject_lc.as_str().into(),\n            V_SPAM_SUBJECT_THREAD => self.ctx.output.subject_thread_lc.as_str().into(),\n            V_SPAM_LOCATION => self.location.as_str().into(),\n            V_WORDS_SUBJECT => self\n                .ctx\n                .output\n                .subject_tokens\n                .iter()\n                .filter_map(|w| match w {\n                    TokenType::Alphabetic(w)\n                    | TokenType::Alphanumeric(w)\n                    | TokenType::Integer(w)\n                    | TokenType::Float(w) => Some(Variable::from(w.as_ref())),\n                    _ => None,\n                })\n                .collect::<Vec<_>>()\n                .into(),\n            V_WORDS_BODY => self\n                .ctx\n                .input\n                .message\n                .html_body\n                .first()\n                .and_then(|idx| self.ctx.output.text_parts.get(*idx as usize))\n                .map(|part| match part {\n                    TextPart::Plain { tokens, .. } | TextPart::Html { tokens, .. } => tokens\n                        .iter()\n                        .filter_map(|w| match w {\n                            TokenType::Alphabetic(w)\n                            | TokenType::Alphanumeric(w)\n                            | TokenType::Integer(w)\n                            | TokenType::Float(w) => Some(Variable::from(w.as_ref())),\n                            _ => None,\n                        })\n                        .collect::<Vec<_>>(),\n                    TextPart::None => vec![],\n                })\n                .unwrap_or_default()\n                .into(),\n            _ => Variable::Integer(0),\n        }\n    }\n\n    fn resolve_global(&self, variable: &str) -> Variable<'_> {\n        Variable::Integer(self.ctx.result.tags.contains(variable).into())\n    }\n}\n\npub(crate) struct EmailHeader<'x> {\n    pub header: &'x Header<'x>,\n    pub raw: &'x str,\n}\n\nimpl ResolveVariable for EmailHeader<'_> {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        match variable {\n            V_HEADER_NAME => self.header.name().into(),\n            V_HEADER_NAME_LOWER => CompactString::from_str_to_lowercase(self.header.name()).into(),\n            V_HEADER_VALUE | V_HEADER_VALUE_LOWER | V_HEADER_PROPERTY => match &self.header.value {\n                HeaderValue::Text(text) => {\n                    if variable == V_HEADER_VALUE_LOWER {\n                        CompactString::from_str_to_lowercase(text).into()\n                    } else {\n                        text.as_ref().into()\n                    }\n                }\n                HeaderValue::TextList(list) => Variable::Array(\n                    list.iter()\n                        .map(|text| {\n                            Variable::String(if variable == V_HEADER_VALUE_LOWER {\n                                StringCow::Owned(CompactString::from_str_to_lowercase(text))\n                            } else {\n                                StringCow::Borrowed(text.as_ref())\n                            })\n                        })\n                        .collect(),\n                ),\n                HeaderValue::Address(address) => Variable::Array(if variable == 1 {\n                    address\n                        .iter()\n                        .filter_map(|a| {\n                            a.address.as_ref().map(|text| {\n                                Variable::String(if variable == V_HEADER_VALUE_LOWER {\n                                    StringCow::Owned(CompactString::from_str_to_lowercase(text))\n                                } else {\n                                    StringCow::Borrowed(text.as_ref())\n                                })\n                            })\n                        })\n                        .collect()\n                } else {\n                    address\n                        .iter()\n                        .filter_map(|a| {\n                            a.name.as_ref().map(|text| {\n                                Variable::String(if variable == V_HEADER_VALUE_LOWER {\n                                    StringCow::Owned(CompactString::from_str_to_lowercase(text))\n                                } else {\n                                    StringCow::Borrowed(text.as_ref())\n                                })\n                            })\n                        })\n                        .collect()\n                }),\n                HeaderValue::DateTime(date_time) => {\n                    CompactString::new(date_time.to_rfc3339()).into()\n                }\n                HeaderValue::ContentType(ct) => {\n                    if variable != V_HEADER_PROPERTY {\n                        if let Some(st) = ct.subtype() {\n                            format_compact!(\"{}/{}\", ct.ctype(), st).into()\n                        } else {\n                            ct.ctype().into()\n                        }\n                    } else {\n                        Variable::Array(\n                            ct.attributes()\n                                .map(|attr| {\n                                    attr.iter()\n                                        .map(|attr| {\n                                            Variable::from(format_compact!(\n                                                \"{}={}\", attr.name, attr.value\n                                            ))\n                                        })\n                                        .collect::<Vec<_>>()\n                                })\n                                .unwrap_or_default(),\n                        )\n                    }\n                }\n                HeaderValue::Received(_) => {\n                    if variable == V_HEADER_VALUE_LOWER {\n                        CompactString::from_str_to_lowercase(self.raw.trim()).into()\n                    } else {\n                        self.raw.trim().into()\n                    }\n                }\n                HeaderValue::Empty => \"\".into(),\n            },\n            V_HEADER_RAW => self.raw.into(),\n            V_HEADER_RAW_LOWER => CompactString::from_str_to_lowercase(self.raw).into(),\n            _ => Variable::Integer(0),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl ResolveVariable for Recipient {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        match variable {\n            V_RCPT_EMAIL => Variable::from(self.email.address.as_str()),\n            V_RCPT_NAME => Variable::from(self.name.as_deref().unwrap_or_default()),\n            V_RCPT_LOCAL => Variable::from(self.email.local_part.as_str()),\n            V_RCPT_DOMAIN => Variable::from(self.email.domain_part.fqdn.as_str()),\n            V_RCPT_DOMAIN_SLD => Variable::from(self.email.domain_part.sld_or_default()),\n            _ => Variable::Integer(0),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl ResolveVariable for UrlParts<'_> {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        match variable {\n            V_URL_FULL => Variable::from(self.url.as_str()),\n            V_URL_PATH_QUERY => Variable::from(\n                self.url_parsed\n                    .as_ref()\n                    .and_then(|p| p.parts.path_and_query().map(|p| p.as_str()))\n                    .unwrap_or_default(),\n            ),\n            V_URL_PATH => Variable::from(\n                self.url_parsed\n                    .as_ref()\n                    .map(|p| p.parts.path())\n                    .unwrap_or_default(),\n            ),\n            V_URL_QUERY => Variable::from(\n                self.url_parsed\n                    .as_ref()\n                    .and_then(|p| p.parts.query())\n                    .unwrap_or_default(),\n            ),\n            V_URL_SCHEME => Variable::from(\n                self.url_parsed\n                    .as_ref()\n                    .and_then(|p| p.parts.scheme_str())\n                    .unwrap_or_default(),\n            ),\n            V_URL_AUTHORITY => Variable::from(\n                self.url_parsed\n                    .as_ref()\n                    .and_then(|p| p.parts.authority().map(|a| a.as_str()))\n                    .unwrap_or_default(),\n            ),\n            V_URL_HOST => Variable::from(\n                self.url_parsed\n                    .as_ref()\n                    .map(|p| p.host.fqdn.as_str())\n                    .unwrap_or_default(),\n            ),\n            V_URL_HOST_SLD => Variable::from(\n                self.url_parsed\n                    .as_ref()\n                    .map(|p| p.host.sld_or_default())\n                    .unwrap_or_default(),\n            ),\n            V_URL_PORT => Variable::Integer(\n                self.url_parsed\n                    .as_ref()\n                    .and_then(|p| p.parts.port_u16())\n                    .unwrap_or(0) as _,\n            ),\n            _ => Variable::Integer(0),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\npub struct StringResolver<'x>(pub &'x str);\n\nimpl ResolveVariable for StringResolver<'_> {\n    fn resolve_variable(&self, _: u32) -> Variable<'_> {\n        Variable::from(self.0)\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\npub struct StringListResolver<'x>(pub &'x [String]);\n\nimpl ResolveVariable for StringListResolver<'_> {\n    fn resolve_variable(&self, _: u32) -> Variable<'_> {\n        Variable::Array(self.0.iter().map(|v| Variable::from(v.as_str())).collect())\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/modules/html.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mail_parser::decoders::html::add_html_token;\n\n#[derive(Debug, Eq, PartialEq, Clone, serde::Serialize, serde::Deserialize)]\n#[serde(tag = \"type\")]\npub enum HtmlToken {\n    StartTag {\n        name: u64,\n        attributes: Vec<(u64, Option<String>)>,\n        is_self_closing: bool,\n    },\n    EndTag {\n        name: u64,\n    },\n    Comment {\n        text: String,\n    },\n    Text {\n        text: String,\n    },\n}\n\npub(crate) const A: u64 = b'a' as u64;\npub(crate) const IMG: u64 = (b'i' as u64) | ((b'm' as u64) << 8) | ((b'g' as u64) << 16);\npub(crate) const HEAD: u64 =\n    (b'h' as u64) | ((b'e' as u64) << 8) | ((b'a' as u64) << 16) | ((b'd' as u64) << 24);\npub(crate) const BODY: u64 =\n    (b'b' as u64) | ((b'o' as u64) << 8) | ((b'd' as u64) << 16) | ((b'y' as u64) << 24);\npub(crate) const META: u64 =\n    (b'm' as u64) | ((b'e' as u64) << 8) | ((b't' as u64) << 16) | ((b'a' as u64) << 24);\npub(crate) const LINK: u64 =\n    (b'l' as u64) | ((b'i' as u64) << 8) | ((b'n' as u64) << 16) | ((b'k' as u64) << 24);\npub(crate) const ALT: u64 = (b'a' as u64) | ((b'l' as u64) << 8) | ((b't' as u64) << 16);\npub(crate) const TITLE: u64 = (b't' as u64)\n    | ((b'i' as u64) << 8)\n    | ((b't' as u64) << 16)\n    | ((b'l' as u64) << 24)\n    | ((b'e' as u64) << 32);\n\npub(crate) const HREF: u64 =\n    (b'h' as u64) | ((b'r' as u64) << 8) | ((b'e' as u64) << 16) | ((b'f' as u64) << 24);\npub(crate) const SRC: u64 = (b's' as u64) | ((b'r' as u64) << 8) | ((b'c' as u64) << 16);\npub(crate) const WIDTH: u64 = (b'w' as u64)\n    | ((b'i' as u64) << 8)\n    | ((b'd' as u64) << 16)\n    | ((b't' as u64) << 24)\n    | ((b'h' as u64) << 32);\npub(crate) const HEIGHT: u64 = (b'h' as u64)\n    | ((b'e' as u64) << 8)\n    | ((b'i' as u64) << 16)\n    | ((b'g' as u64) << 24)\n    | ((b'h' as u64) << 32)\n    | ((b't' as u64) << 40);\npub(crate) const REL: u64 = (b'r' as u64) | ((b'e' as u64) << 8) | ((b'l' as u64) << 16);\npub(crate) const CONTENT: u64 = (b'c' as u64)\n    | ((b'o' as u64) << 8)\n    | ((b'n' as u64) << 16)\n    | ((b't' as u64) << 24)\n    | ((b'e' as u64) << 32)\n    | ((b'n' as u64) << 40)\n    | ((b't' as u64) << 48);\npub(crate) const HTTP_EQUIV: u64 = (b'h' as u64)\n    | ((b't' as u64) << 8)\n    | ((b't' as u64) << 16)\n    | ((b'p' as u64) << 24)\n    | ((b'-' as u64) << 32)\n    | ((b'e' as u64) << 40)\n    | ((b'q' as u64) << 48)\n    | ((b'u' as u64) << 56);\n\npub fn html_to_tokens(input: &str) -> Vec<HtmlToken> {\n    let input = input.as_bytes();\n    let mut iter = input.iter().enumerate().peekable();\n    let mut tags = vec![];\n\n    let mut is_token_start = true;\n    let mut is_after_space = false;\n    let mut is_new_line = true;\n\n    let mut token_start = 0;\n    let mut token_end = 0;\n\n    let mut text = String::with_capacity(16);\n\n    while let Some((mut pos, &ch)) = iter.next() {\n        match ch {\n            b'<' => {\n                if !is_token_start {\n                    add_html_token(\n                        &mut text,\n                        &input[token_start..token_end + 1],\n                        is_after_space,\n                    );\n                    is_after_space = false;\n                    is_token_start = true;\n                }\n                if !text.is_empty() {\n                    tags.push(HtmlToken::Text {\n                        text: text.as_str().into(),\n                    });\n                    text.clear();\n                }\n\n                while matches!(iter.peek(), Some(&(_, &ch)) if ch.is_ascii_whitespace()) {\n                    pos += 1;\n                    iter.next();\n                }\n\n                if matches!(input.get(pos + 1..pos + 4), Some(b\"!--\")) {\n                    let mut comment = Vec::new();\n                    let mut last_ch: u8 = 0;\n                    for (_, &ch) in iter.by_ref() {\n                        match ch {\n                            b'>' if comment.len() > 2\n                                && matches!(comment.last(), Some(b'-'))\n                                && matches!(comment.get(comment.len() - 2), Some(b'-')) =>\n                            {\n                                break;\n                            }\n                            b' ' | b'\\t' | b'\\r' | b'\\n' => {\n                                if last_ch != b' ' {\n                                    comment.push(b' ');\n                                } else {\n                                    last_ch = b' ';\n                                }\n                                continue;\n                            }\n                            _ => {\n                                comment.push(ch);\n                            }\n                        }\n                        last_ch = ch;\n                    }\n                    tags.push(HtmlToken::Comment {\n                        text: String::from_utf8(comment).unwrap_or_default(),\n                    });\n                } else {\n                    let mut is_end_tag = false;\n                    loop {\n                        match iter.peek() {\n                            Some(&(_, &b'/')) => {\n                                is_end_tag = true;\n                                //pos += 1;\n                                iter.next();\n                            }\n                            Some((_, ch)) if ch.is_ascii_whitespace() => {\n                                //pos += 1;\n                                iter.next();\n                            }\n                            _ => break,\n                        }\n                    }\n\n                    let mut in_quote = false;\n                    let mut is_self_closing = false;\n\n                    let mut key: u64 = 0;\n                    let mut shift = 0;\n\n                    let mut tag = 0;\n                    let mut attributes: Vec<(u64, Option<String>)> = vec![];\n\n                    'outer: while let Some((_, &ch)) = iter.next() {\n                        match ch {\n                            b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' if shift < 64 => {\n                                key |= (ch as u64) << shift;\n                                shift += 8;\n                            }\n                            b'A'..=b'Z' if shift < 64 => {\n                                key |= ((ch - b'A' + b'a') as u64) << shift;\n                                shift += 8;\n                            }\n                            b'/' if !in_quote => {\n                                is_self_closing = true;\n                            }\n                            b'>' if !in_quote => {\n                                if shift != 0 {\n                                    if tag == 0 {\n                                        tag = key;\n                                    } else {\n                                        attributes.push((key, None));\n                                    }\n                                }\n                                break;\n                            }\n                            b'\"' => {\n                                in_quote = !in_quote;\n                            }\n                            b'=' if !in_quote => {\n                                while matches!(iter.peek(), Some(&(_, &ch)) if ch.is_ascii_whitespace())\n                                {\n                                    iter.next();\n                                }\n\n                                if shift != 0 {\n                                    attributes.push((key, None));\n                                    key = 0;\n                                    shift = 0;\n                                }\n\n                                let mut value = vec![];\n\n                                for (_, &ch) in iter.by_ref() {\n                                    match ch {\n                                        b'>' if !in_quote => {\n                                            if !value.is_empty() {\n                                                let value =\n                                                    String::from_utf8(value).unwrap_or_default();\n                                                if let Some((_, v)) = attributes.last_mut() {\n                                                    *v = value.into();\n                                                } else {\n                                                    // Broken attribute\n                                                    attributes.push((0, Some(value)));\n                                                }\n                                            }\n                                            break 'outer;\n                                        }\n                                        b'\"' => {\n                                            if in_quote {\n                                                in_quote = false;\n                                                break;\n                                            } else {\n                                                in_quote = true;\n                                            }\n                                        }\n                                        b' ' | b'\\t' | b'\\r' | b'\\n' if !in_quote => {\n                                            break;\n                                        }\n                                        _ => {\n                                            value.push(ch);\n                                        }\n                                    }\n                                }\n\n                                if !value.is_empty() {\n                                    let value = String::from_utf8(value).unwrap_or_default();\n                                    if let Some((_, v)) = attributes.last_mut() {\n                                        *v = value.into();\n                                    } else {\n                                        // Broken attribute\n                                        attributes.push((0, Some(value)));\n                                    }\n                                }\n                            }\n                            b' ' | b'\\t' | b'\\r' | b'\\n' => {\n                                if shift != 0 {\n                                    if tag == 0 {\n                                        tag = key;\n                                    } else {\n                                        attributes.push((key, None));\n                                    }\n                                    key = 0;\n                                    shift = 0;\n                                }\n                            }\n                            _ => {}\n                        }\n                    }\n\n                    if tag != 0 {\n                        if is_end_tag {\n                            tags.push(HtmlToken::EndTag { name: tag });\n                        } else {\n                            tags.push(HtmlToken::StartTag {\n                                name: tag,\n                                attributes,\n                                is_self_closing,\n                            });\n                        }\n                    }\n                }\n                continue;\n            }\n            b' ' | b'\\t' | b'\\r' | b'\\n' => {\n                if !is_token_start {\n                    add_html_token(\n                        &mut text,\n                        &input[token_start..token_end + 1],\n                        is_after_space && !is_new_line,\n                    );\n                    is_new_line = false;\n                }\n                is_after_space = true;\n                is_token_start = true;\n                continue;\n            }\n            b'&' if !is_token_start => {\n                add_html_token(\n                    &mut text,\n                    &input[token_start..token_end + 1],\n                    is_after_space && !is_new_line,\n                );\n                is_new_line = false;\n                is_token_start = true;\n                is_after_space = false;\n            }\n            b';' if !is_token_start => {\n                add_html_token(\n                    &mut text,\n                    &input[token_start..pos + 1],\n                    is_after_space && !is_new_line,\n                );\n                is_token_start = true;\n                is_after_space = false;\n                is_new_line = false;\n                continue;\n            }\n            _ => (),\n        }\n\n        if is_token_start {\n            token_start = pos;\n            is_token_start = false;\n        }\n        token_end = pos;\n    }\n\n    if !is_token_start {\n        add_html_token(\n            &mut text,\n            &input[token_start..token_end + 1],\n            is_after_space && !is_new_line,\n        );\n    }\n    if !text.is_empty() {\n        tags.push(HtmlToken::Text {\n            text: text.as_str().into(),\n        });\n    }\n\n    tags\n}\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn test_html_to_tokens_text() {\n        let input = \"Hello, world!\";\n        let tokens = html_to_tokens(input);\n        assert_eq!(\n            tokens,\n            vec![HtmlToken::Text {\n                text: \"Hello, world!\".into()\n            }]\n        );\n    }\n\n    #[test]\n    fn test_html_to_tokens_start_tag() {\n        let input = \"<div>\";\n        let tokens = html_to_tokens(input);\n        assert_eq!(\n            tokens,\n            vec![HtmlToken::StartTag {\n                name: 7760228,\n                attributes: vec![],\n                is_self_closing: false\n            }]\n        );\n    }\n\n    #[test]\n    fn test_html_to_tokens_end_tag() {\n        let input = \"</div>\";\n        let tokens = html_to_tokens(input);\n        assert_eq!(tokens, vec![HtmlToken::EndTag { name: 7760228 }]);\n    }\n\n    #[test]\n    fn test_html_to_tokens_comment() {\n        let input = \"<!-- This is a comment -->\";\n        let tokens = html_to_tokens(input);\n        assert_eq!(\n            tokens,\n            vec![HtmlToken::Comment {\n                text: \"!-- This is a comment --\".into()\n            }]\n        );\n    }\n\n    #[test]\n    fn test_html_to_tokens_mixed() {\n        let input = \"<div>Hello, <span>&quot; world &quot; </span>!</div>\";\n        let tokens = html_to_tokens(input);\n        assert_eq!(\n            tokens,\n            vec![\n                HtmlToken::StartTag {\n                    name: 7760228,\n                    attributes: vec![],\n                    is_self_closing: false\n                },\n                HtmlToken::Text {\n                    text: \"Hello,\".into()\n                },\n                HtmlToken::StartTag {\n                    name: 1851879539,\n                    attributes: vec![],\n                    is_self_closing: false\n                },\n                HtmlToken::Text {\n                    text: \" \\\" world \\\"\".into()\n                },\n                HtmlToken::EndTag { name: 1851879539 },\n                HtmlToken::Text { text: \" !\".into() },\n                HtmlToken::EndTag { name: 7760228 }\n            ]\n        );\n    }\n\n    #[test]\n    fn test_html_to_tokens_with_attributes() {\n        let input = r#\"<input type=\"text\" value=\"test\"><single/><one attr/><a b=1 b c=\"123\">\"#;\n        let tokens = html_to_tokens(input);\n        assert_eq!(\n            tokens,\n            vec![\n                HtmlToken::StartTag {\n                    name: 500186508905,\n                    attributes: vec![\n                        (1701869940, Some(\"text\".into())),\n                        (435761734006, Some(\"test\".into()))\n                    ],\n                    is_self_closing: false\n                },\n                HtmlToken::StartTag {\n                    name: 111516266162547,\n                    attributes: vec![],\n                    is_self_closing: true\n                },\n                HtmlToken::StartTag {\n                    name: 6647407,\n                    attributes: vec![(1920234593, None)],\n                    is_self_closing: true\n                },\n                HtmlToken::StartTag {\n                    name: 97,\n                    attributes: vec![(98, Some(\"1\".into())), (98, None), (99, Some(\"123\".into()))],\n                    is_self_closing: false\n                }\n            ]\n        );\n    }\n}\n"
  },
  {
    "path": "crates/spam-filter/src/modules/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod classifier;\npub mod dnsbl;\npub mod expression;\npub mod html;\npub mod pyzor;\npub mod sanitize;\n"
  },
  {
    "path": "crates/spam-filter/src/modules/pyzor.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    borrow::Cow,\n    io::Write,\n    net::SocketAddr,\n    time::{Duration, SystemTime},\n};\n\nuse common::config::spamfilter::PyzorConfig;\nuse mail_parser::{Message, PartType, decoders::html::add_html_token};\nuse nlp::tokenizers::types::{TokenType, TypesTokenizer};\nuse sha1::{Digest, Sha1};\nuse tokio::net::UdpSocket;\n\nconst MIN_LINE_LENGTH: usize = 8;\nconst ATOMIC_NUM_LINES: usize = 4;\nconst DIGEST_SPEC: &[(usize, usize)] = &[(20, 3), (60, 3)];\n\n#[derive(Default, Debug, PartialEq, Eq)]\npub(crate) struct PyzorResponse {\n    pub code: u32,\n    pub count: u64,\n    pub wl_count: u64,\n}\n\npub(crate) async fn pyzor_check(\n    message: &Message<'_>,\n    config: &PyzorConfig,\n) -> trc::Result<Option<PyzorResponse>> {\n    // Make sure there is at least one text part\n    if !message\n        .parts\n        .iter()\n        .any(|p| matches!(p.body, PartType::Text(_) | PartType::Html(_)))\n    {\n        return Ok(None);\n    }\n\n    // Hash message\n    let request = message.pyzor_check_message();\n\n    #[cfg(feature = \"test_mode\")]\n    {\n        if request.contains(\"b5b476f0b5ba6e1c038361d3ded5818dd39c90a2\") {\n            return Ok(PyzorResponse {\n                code: 200,\n                count: 1000,\n                wl_count: 0,\n            }\n            .into());\n        } else if request.contains(\"d67d4b8bfc3860449e3418bb6017e2612f3e2a99\") {\n            return Ok(PyzorResponse {\n                code: 200,\n                count: 60,\n                wl_count: 10,\n            }\n            .into());\n        } else if request.contains(\"81763547012b75e57a20d18ce0b93014208cdfdb\") {\n            return Ok(PyzorResponse {\n                code: 200,\n                count: 50,\n                wl_count: 20,\n            }\n            .into());\n        }\n    }\n\n    // Send message to address\n    pyzor_send_message(config.address, config.timeout, &request)\n        .await\n        .map(Into::into)\n        .map_err(|err| {\n            trc::SpamEvent::PyzorError\n                .into_err()\n                .ctx(trc::Key::Url, config.address.to_string())\n                .reason(err)\n                .details(\"Pyzor failed\")\n        })\n}\n\nasync fn pyzor_send_message(\n    addr: SocketAddr,\n    timeout: Duration,\n    message: &str,\n) -> std::io::Result<PyzorResponse> {\n    let socket = UdpSocket::bind(\"0.0.0.0:0\").await?;\n    tokio::time::timeout(timeout, socket.send_to(message.as_bytes(), addr)).await??;\n\n    let mut buffer = vec![0u8; 1024];\n    let (size, _) = tokio::time::timeout(timeout, socket.recv_from(&mut buffer)).await??;\n\n    let raw_response = std::str::from_utf8(&buffer[..size])\n        .map_err(|err| std::io::Error::new(std::io::ErrorKind::InvalidData, err))?;\n    let mut response = PyzorResponse {\n        code: u32::MAX,\n        count: u64::MAX,\n        wl_count: u64::MAX,\n    };\n\n    for line in raw_response.lines() {\n        if let Some((k, v)) = line.split_once(':') {\n            if k.eq_ignore_ascii_case(\"code\") {\n                response.code = v.trim().parse().map_err(|_| {\n                    std::io::Error::new(\n                        std::io::ErrorKind::InvalidData,\n                        format!(\"Invalid line: {raw_response}\"),\n                    )\n                })?;\n            } else if k.eq_ignore_ascii_case(\"count\") {\n                response.count = v.trim().parse().map_err(|_| {\n                    std::io::Error::new(\n                        std::io::ErrorKind::InvalidData,\n                        format!(\"Invalid line: {raw_response}\"),\n                    )\n                })?;\n            } else if k.eq_ignore_ascii_case(\"wl-count\") {\n                response.wl_count = v.trim().parse().map_err(|_| {\n                    std::io::Error::new(\n                        std::io::ErrorKind::InvalidData,\n                        format!(\"Invalid line: {raw_response}\"),\n                    )\n                })?;\n            }\n        }\n    }\n\n    if response.code != u32::MAX && response.count != u64::MAX && response.wl_count != u64::MAX {\n        Ok(response)\n    } else {\n        Err(std::io::Error::new(\n            std::io::ErrorKind::InvalidData,\n            format!(\"Invalid response: {raw_response}\"),\n        ))\n    }\n}\n\ntrait PyzorDigest<W: Write> {\n    fn pyzor_digest(&self, writer: W) -> W;\n}\n\npub trait PyzorCheck {\n    fn pyzor_check_message(&self) -> String;\n}\n\nimpl<W: Write> PyzorDigest<W> for Message<'_> {\n    fn pyzor_digest(&self, writer: W) -> W {\n        let parts = self\n            .parts\n            .iter()\n            .filter_map(|part| match &part.body {\n                PartType::Text(text) => Some(text.as_ref().into()),\n                PartType::Html(html) => Some(html_to_text(html.as_ref()).into()),\n                _ => None,\n            })\n            .collect::<Vec<Cow<str>>>();\n\n        pyzor_digest(writer, parts.iter().flat_map(|text| text.lines()))\n    }\n}\n\nimpl PyzorCheck for Message<'_> {\n    fn pyzor_check_message(&self) -> String {\n        let time = SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .map_or(0, |d| d.as_secs());\n\n        pyzor_create_message(\n            self,\n            time,\n            (time & 0xFFFF) as u16 ^ ((time >> 16) & 0xFFFF) as u16,\n        )\n    }\n}\n\nfn pyzor_create_message(message: &Message<'_>, time: u64, thread: u16) -> String {\n    // Hash message\n    let hash = message.pyzor_digest(Sha1::new()).finalize();\n    // Hash key\n    let mut hash_key = Sha1::new();\n    hash_key.update(\"anonymous:\".as_bytes());\n    let hash_key = hash_key.finalize();\n\n    // Hash message\n    let message = format!(\n        \"Op: check\\nOp-Digest: {hash:x}\\nThread: {thread}\\nPV: 2.1\\nUser: anonymous\\nTime: {time}\"\n    );\n    let mut msg_hash = Sha1::new();\n    msg_hash.update(message.as_bytes());\n    let msg_hash = msg_hash.finalize();\n\n    // Sign\n    let mut sig = Sha1::new();\n    sig.update(msg_hash);\n    sig.update(format!(\":{time}:{hash_key:x}\"));\n    let sig = sig.finalize();\n\n    format!(\"{message}\\nSig: {sig:x}\\n\")\n}\n\nfn pyzor_digest<'x, I, W>(mut writer: W, lines: I) -> W\nwhere\n    I: Iterator<Item = &'x str>,\n    W: Write,\n{\n    let mut result = Vec::with_capacity(16);\n\n    for line in lines {\n        let mut clean_line = String::with_capacity(line.len());\n        let mut token_start = usize::MAX;\n        let mut token_end = usize::MAX;\n\n        let add_line = |line: &mut String, span: &str| {\n            if !span.contains(char::from(0)) {\n                if span.len() < 10 {\n                    line.push_str(span);\n                }\n            } else {\n                let span = span.replace(char::from(0), \"\");\n                if span.len() < 10 {\n                    line.push_str(&span);\n                }\n            }\n        };\n\n        for token in TypesTokenizer::new(line) {\n            match token.word {\n                TokenType::Alphabetic(_)\n                | TokenType::Alphanumeric(_)\n                | TokenType::Integer(_)\n                | TokenType::Float(_)\n                | TokenType::Other(_)\n                | TokenType::Punctuation(_) => {\n                    if token_start == usize::MAX {\n                        token_start = token.from;\n                    }\n                    token_end = token.to;\n                }\n                TokenType::Space\n                | TokenType::Url(_)\n                | TokenType::UrlNoScheme(_)\n                | TokenType::UrlNoHost(_)\n                | TokenType::IpAddr(_)\n                | TokenType::Email(_) => {\n                    if token_start != usize::MAX {\n                        add_line(&mut clean_line, &line[token_start..token_end]);\n                        token_start = usize::MAX;\n                        token_end = usize::MAX;\n                    }\n                }\n            }\n        }\n\n        if token_start != usize::MAX {\n            add_line(&mut clean_line, &line[token_start..token_end]);\n        }\n\n        if clean_line.len() >= MIN_LINE_LENGTH {\n            result.push(clean_line);\n        }\n    }\n\n    if result.len() > ATOMIC_NUM_LINES {\n        for (offset, length) in DIGEST_SPEC {\n            for i in 0..*length {\n                if let Some(line) = result.get((*offset * result.len() / 100) + i) {\n                    let _ = writer.write_all(line.as_bytes());\n                }\n            }\n        }\n    } else {\n        for line in result {\n            let _ = writer.write_all(line.as_bytes());\n        }\n    }\n\n    writer\n}\n\nfn html_to_text(input: &str) -> String {\n    let mut result = String::with_capacity(input.len());\n    let input = input.as_bytes();\n\n    let mut in_tag = false;\n    let mut in_comment = false;\n    let mut in_style = false;\n    let mut in_script = false;\n\n    let mut is_token_start = true;\n    let mut is_after_space = false;\n    let mut is_tag_close = false;\n\n    let mut token_start = 0;\n    let mut token_end = 0;\n\n    let mut tag_token_pos = 0;\n    let mut comment_pos = 0;\n\n    for (pos, ch) in input.iter().enumerate() {\n        if !in_comment {\n            match ch {\n                b'<' => {\n                    if !(in_tag || in_style || in_script || is_token_start) {\n                        add_html_token(\n                            &mut result,\n                            &input[token_start..token_end + 1],\n                            is_after_space,\n                        );\n                        is_after_space = false;\n                    }\n\n                    tag_token_pos = 0;\n                    in_tag = true;\n                    is_token_start = true;\n                    is_tag_close = false;\n                    continue;\n                }\n                b'>' if in_tag => {\n                    if tag_token_pos == 1\n                        && let Some(tag) = input.get(token_start..token_end + 1)\n                    {\n                        if tag.eq_ignore_ascii_case(b\"style\") {\n                            in_style = !is_tag_close;\n                        } else if tag.eq_ignore_ascii_case(b\"script\") {\n                            in_script = !is_tag_close;\n                        }\n                    }\n\n                    in_tag = false;\n                    is_token_start = true;\n                    is_after_space = !result.is_empty();\n\n                    continue;\n                }\n                b'/' if in_tag => {\n                    if tag_token_pos == 0 {\n                        is_tag_close = true;\n                    }\n                    continue;\n                }\n                b'!' if in_tag && tag_token_pos == 0 => {\n                    if let Some(b\"--\") = input.get(pos + 1..pos + 3) {\n                        in_comment = true;\n                        continue;\n                    }\n                }\n                b' ' | b'\\t' | b'\\r' | b'\\n' => {\n                    if !(in_tag || in_style || in_script) {\n                        if !is_token_start {\n                            add_html_token(\n                                &mut result,\n                                &input[token_start..token_end + 1],\n                                is_after_space,\n                            );\n                        }\n                        is_after_space = true;\n                    }\n\n                    is_token_start = true;\n                    continue;\n                }\n                b'&' if !(in_tag || is_token_start || in_style || in_script) => {\n                    add_html_token(\n                        &mut result,\n                        &input[token_start..token_end + 1],\n                        is_after_space,\n                    );\n                    is_token_start = true;\n                    is_after_space = false;\n                }\n                b';' if !(in_tag || is_token_start || in_style || in_script) => {\n                    add_html_token(&mut result, &input[token_start..pos + 1], is_after_space);\n                    is_token_start = true;\n                    is_after_space = false;\n                    continue;\n                }\n                _ => (),\n            }\n            if is_token_start {\n                token_start = pos;\n                is_token_start = false;\n                if in_tag {\n                    tag_token_pos += 1;\n                }\n            }\n            token_end = pos;\n        } else {\n            match ch {\n                b'-' => comment_pos += 1,\n                b'>' if comment_pos == 2 => {\n                    comment_pos = 0;\n                    in_comment = false;\n                    in_tag = false;\n                    is_token_start = true;\n                }\n                _ => comment_pos = 0,\n            }\n        }\n    }\n\n    if !(in_tag || is_token_start || in_style || in_script) {\n        add_html_token(\n            &mut result,\n            &input[token_start..token_end + 1],\n            is_after_space,\n        );\n    }\n\n    result.shrink_to_fit();\n    result\n}\n\n#[cfg(test)]\nmod test {\n    use std::time::Duration;\n\n    use mail_parser::MessageParser;\n    use sha1::Digest;\n    use sha1::Sha1;\n\n    use super::pyzor_create_message;\n    use super::pyzor_send_message;\n    use super::{PyzorDigest, html_to_text, pyzor_digest};\n\n    use super::PyzorResponse;\n\n    #[ignore]\n    #[tokio::test]\n    async fn send_message() {\n        assert_eq!(\n            pyzor_send_message(\n                \"public.pyzor.org:24441\".parse().unwrap(),\n                Duration::from_secs(10),\n                concat!(\n                    \"Op: check\\n\",\n                    \"Op-Digest: b2c27325a034c581df0c9ef37e4a0d63208a3e7e\\n\",\n                    \"Thread: 49005\\n\",\n                    \"PV: 2.1\\n\",\n                    \"User: anonymous\\n\",\n                    \"Time: 1697468672\\n\",\n                    \"Sig: 9cf4571b85d3887fdd0d4f444fd0c164e0290722\\n\"\n                ),\n            )\n            .await\n            .unwrap(),\n            PyzorResponse {\n                code: 200,\n                count: 0,\n                wl_count: 0\n            }\n        );\n    }\n\n    #[test]\n    fn message_pyzor() {\n        let message = pyzor_create_message(\n            &MessageParser::new().parse(HTML_TEXT_STYLE_SCRIPT).unwrap(),\n            1697468672,\n            49005,\n        );\n\n        assert_eq!(\n            message,\n            concat!(\n                \"Op: check\\n\",\n                \"Op-Digest: b2c27325a034c581df0c9ef37e4a0d63208a3e7e\\n\",\n                \"Thread: 49005\\n\",\n                \"PV: 2.1\\n\",\n                \"User: anonymous\\n\",\n                \"Time: 1697468672\\n\",\n                \"Sig: 9cf4571b85d3887fdd0d4f444fd0c164e0290722\\n\"\n            )\n        );\n    }\n\n    #[test]\n    fn digest_pyzor() {\n        // HTML stripping\n        assert_eq!(html_to_text(HTML_RAW), HTML_RAW_STRIPED);\n\n        // Token stripping\n        for strip_me in [\n            \"t@abc.com\",\n            \"t1@abc.com\",\n            \"t+a@abc.com\",\n            \"t.a@abc.com\",\n            \"0A2D3f%a#S\",\n            \"3sddkf9jdkd9\",\n            \"@@#@@@@@@@@@\",\n            \"http://spammer.com/special-offers?buy=now\",\n        ] {\n            assert_eq!(\n                String::from_utf8(pyzor_digest(\n                    Vec::new(),\n                    format!(\"Test {strip_me} Test2\").lines(),\n                ))\n                .unwrap(),\n                \"TestTest2\"\n            );\n        }\n\n        // Test short lines\n        assert_eq!(\n            String::from_utf8(pyzor_digest(\n                Vec::new(),\n                concat!(\"This line is included\\n\", \"not this\\n\", \"This also\").lines(),\n            ))\n            .unwrap(),\n            \"ThislineisincludedThisalso\"\n        );\n\n        // Test atomic\n        assert_eq!(\n            String::from_utf8(pyzor_digest(\n                Vec::new(),\n                \"All this message\\nShould be included\\nIn the digest\".lines(),\n            ))\n            .unwrap(),\n            \"AllthismessageShouldbeincludedInthedigest\"\n        );\n\n        // Test spec\n        let mut text = String::new();\n        for i in 0..100 {\n            text += format!(\"Line{i} test test test\\n\").as_str();\n        }\n        let mut expected = String::new();\n        for i in [20, 21, 22, 60, 61, 62] {\n            expected += format!(\"Line{i}testtesttest\").as_str();\n        }\n        assert_eq!(\n            String::from_utf8(pyzor_digest(Vec::new(), text.lines(),)).unwrap(),\n            expected\n        );\n\n        // Test email parsing\n        for (input, expected) in [\n            (\n                HTML_TEXT,\n                concat!(\n                    \"Emailspam,alsoknownasjunkemailorbulkemail,isasubset\",\n                    \"ofspaminvolvingnearlyidenticalmessagessenttonumerous\",\n                    \"byemail.Clickingonlinksinspamemailmaysendusersto\",\n                    \"byemail.Clickingonlinksinspamemailmaysendusersto\",\n                    \"phishingwebsitesorsitesthatarehostingmalware.\",\n                    \"Emailspam.Emailspam,alsoknownasjunkemailorbulkemail,\",\n                    \"isasubsetofspaminvolvingnearlyidenticalmessage\",\n                    \"ssenttonumerousbyemail.Clickingonlinksinspamemailmaysenduse\",\n                    \"rstophishingwebsitesorsitesthatarehostingmalware.\"\n                ),\n            ),\n            (HTML_TEXT_STYLE_SCRIPT, \"Thisisatest.Thisisatest.\"),\n            (TEXT_ATTACHMENT, \"Thisisatestmailing\"),\n            (TEXT_ATTACHMENT_W_NULL, \"Thisisatestmailing\"),\n            (TEXT_ATTACHMENT_W_MULTIPLE_NULLS, \"Thisisatestmailing\"),\n            (TEXT_ATTACHMENT_W_SUBJECT_NULL, \"Thisisatestmailing\"),\n            (TEXT_ATTACHMENT_W_CONTENTTYPE_NULL, \"Thisisatestmailing\"),\n        ] {\n            assert_eq!(\n                String::from_utf8(\n                    MessageParser::new()\n                        .parse(input)\n                        .unwrap()\n                        .pyzor_digest(Vec::new(),)\n                )\n                .unwrap(),\n                expected,\n                \"failed for {input}\"\n            )\n        }\n\n        // Test SHA hash\n        assert_eq!(\n            format!(\n                \"{:x}\",\n                MessageParser::new()\n                    .parse(HTML_TEXT_STYLE_SCRIPT)\n                    .unwrap()\n                    .pyzor_digest(Sha1::new(),)\n                    .finalize()\n            ),\n            \"b2c27325a034c581df0c9ef37e4a0d63208a3e7e\",\n        )\n    }\n\n    const HTML_TEXT: &str = r#\"MIME-Version: 1.0\nSender: chirila@gapps.spamexperts.com\nReceived: by 10.216.157.70 with HTTP; Thu, 16 Jan 2014 00:43:31 -0800 (PST)\nDate: Thu, 16 Jan 2014 10:43:31 +0200\nDelivered-To: chirila@gapps.spamexperts.com\nX-Google-Sender-Auth: ybCmONS9U9D6ZUfjx-9_tY-hF2Q\nMessage-ID: <CAK-mJS8sE-V6qtspzzZ+bZ1eSUE_FNMt3K-5kBOG-z3NMgU_Rg@mail.gmail.com>\nSubject: Test\nFrom: Alexandru Chirila <chirila@spamexperts.com>\nTo: Alexandru Chirila <chirila@gapps.spamexperts.com>\nContent-Type: multipart/alternative; boundary=001a11c25ff293069304f0126bfd\n\n--001a11c25ff293069304f0126bfd\nContent-Type: text/plain; charset=ISO-8859-1\n\nEmail spam.\n\nEmail spam, also known as junk email or unsolicited bulk email, is a subset\nof electronic spam involving nearly identical messages sent to numerous\nrecipients by email. Clicking on links in spam email may send users to\nphishing web sites or sites that are hosting malware.\n\n--001a11c25ff293069304f0126bfd\nContent-Type: text/html; charset=ISO-8859-1\nContent-Transfer-Encoding: quoted-printable\n\n<div dir=3D\"ltr\"><div>Email spam.</div><div><br></div><div>Email spam, also=\n known as junk email or unsolicited bulk email, is a subset of electronic s=\npam involving nearly identical messages sent to numerous recipients by emai=\nl. Clicking on links in spam email may send users to phishing web sites or =\nsites that are hosting malware.</div>\n</div>\n\n--001a11c25ff293069304f0126bfd--\n\"#;\n\n    const HTML_TEXT_STYLE_SCRIPT: &str = r#\"MIME-Version: 1.0\nSender: chirila@gapps.spamexperts.com\nReceived: by 10.216.157.70 with HTTP; Thu, 16 Jan 2014 00:43:31 -0800 (PST)\nDate: Thu, 16 Jan 2014 10:43:31 +0200\nDelivered-To: chirila@gapps.spamexperts.com\nX-Google-Sender-Auth: ybCmONS9U9D6ZUfjx-9_tY-hF2Q\nMessage-ID: <CAK-mJS8sE-V6qtspzzZ+bZ1eSUE_FNMt3K-5kBOG-z3NMgU_Rg@mail.gmail.com>\nSubject: Test\nFrom: Alexandru Chirila <chirila@spamexperts.com>\nTo: Alexandru Chirila <chirila@gapps.spamexperts.com>\nContent-Type: multipart/alternative; boundary=001a11c25ff293069304f0126bfd\n\n--001a11c25ff293069304f0126bfd\nContent-Type: text/plain; charset=ISO-8859-1\n\nThis is a test.\n\n--001a11c25ff293069304f0126bfd\nContent-Type: text/html; charset=ISO-8859-1\nContent-Transfer-Encoding: quoted-printable\n\n<div dir=3D\"ltr\">\n<style> This is my style.</style>\n<script> This is my script.</script>\n<div>This is a test.</div>\n</div>\n\n--001a11c25ff293069304f0126bfd--\n\"#;\n\n    const TEXT_ATTACHMENT: &str = r#\"MIME-Version: 1.0\nReceived: by 10.76.127.40 with HTTP; Fri, 17 Jan 2014 02:21:43 -0800 (PST)\nDate: Fri, 17 Jan 2014 12:21:43 +0200\nDelivered-To: chirila.s.alexandru@gmail.com\nMessage-ID: <CALTHOsuHFaaatiXJKU=LdDCo4NmD_h49yvG2RDsWw17D0-NXJg@mail.gmail.com>\nSubject: Test\nFrom: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nTo: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nContent-Type: multipart/mixed; boundary=f46d040a62c49bb1c804f027e8cc\n\n--f46d040a62c49bb1c804f027e8cc\nContent-Type: multipart/alternative; boundary=f46d040a62c49bb1c404f027e8ca\n\n--f46d040a62c49bb1c404f027e8ca\nContent-Type: text/plain; charset=ISO-8859-1\n\nThis is a test mailing\n\n--f46d040a62c49bb1c404f027e8ca--\n--f46d040a62c49bb1c804f027e8cc\nContent-Type: image/png; name=\"tar.png\"\nContent-Disposition: attachment; filename=\"tar.png\"\nContent-Transfer-Encoding: base64\nX-Attachment-Id: f_hqjas5ad0\n\niVBORw0KGgoAAAANSUhEUgAAAskAAADlCAAAAACErzVVAAAACXBIWXMAAAsTAAALEwEAmpwYAAAD\nQmCC\n--f46d040a62c49bb1c804f027e8cc--\"#;\n\n    const TEXT_ATTACHMENT_W_NULL: &str = \"MIME-Version: 1.0\nReceived: by 10.76.127.40 with HTTP; Fri, 17 Jan 2014 02:21:43 -0800 (PST)\nDate: Fri, 17 Jan 2014 12:21:43 +0200\nDelivered-To: chirila.s.alexandru@gmail.com\nMessage-ID: <CALTHOsuHFaaatiXJKU=LdDCo4NmD_h49yvG2RDsWw17D0-NXJg@mail.gmail.com>\nSubject: Test\nFrom: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nTo: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nContent-Type: multipart/mixed; boundary=f46d040a62c49bb1c804f027e8cc\n\n--f46d040a62c49bb1c804f027e8cc\nContent-Type: multipart/alternative; boundary=f46d040a62c49bb1c404f027e8ca\n\n--f46d040a62c49bb1c404f027e8ca\nContent-Type: text/plain; charset=ISO-8859-1\n\nThis is a test ma\\0iling\n--f46d040a62c49bb1c804f027e8cc--\";\n\n    const TEXT_ATTACHMENT_W_MULTIPLE_NULLS: &str = \"MIME-Version: 1.0\nReceived: by 10.76.127.40 with HTTP; Fri, 17 Jan 2014 02:21:43 -0800 (PST)\nDate: Fri, 17 Jan 2014 12:21:43 +0200\nDelivered-To: chirila.s.alexandru@gmail.com\nMessage-ID: <CALTHOsuHFaaatiXJKU=LdDCo4NmD_h49yvG2RDsWw17D0-NXJg@mail.gmail.com>\nSubject: Test\nFrom: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nTo: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nContent-Type: multipart/mixed; boundary=f46d040a62c49bb1c804f027e8cc\n\n--f46d040a62c49bb1c804f027e8cc\nContent-Type: multipart/alternative; boundary=f46d040a62c49bb1c404f027e8ca\n\n--f46d040a62c49bb1c404f027e8ca\nContent-Type: text/plain; charset=ISO-8859-1\n\nThis is a test ma\\0\\0\\0iling\n--f46d040a62c49bb1c804f027e8cc--\";\n\n    const TEXT_ATTACHMENT_W_SUBJECT_NULL: &str = \"MIME-Version: 1.0\nReceived: by 10.76.127.40 with HTTP; Fri, 17 Jan 2014 02:21:43 -0800 (PST)\nDate: Fri, 17 Jan 2014 12:21:43 +0200\nDelivered-To: chirila.s.alexandru@gmail.com\nMessage-ID: <CALTHOsuHFaaatiXJKU=LdDCo4NmD_h49yvG2RDsWw17D0-NXJg@mail.gmail.com>\nSubject: Te\\0\\0\\0st\nFrom: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nTo: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nContent-Type: multipart/mixed; boundary=f46d040a62c49bb1c804f027e8cc\n\n--f46d040a62c49bb1c804f027e8cc\nContent-Type: multipart/alternative; boundary=f46d040a62c49bb1c404f027e8ca\n\n--f46d040a62c49bb1c404f027e8ca\nContent-Type: text/plain; charset=ISO-8859-1\n\nThis is a test mailing\n--f46d040a62c49bb1c804f027e8cc--\";\n\n    const TEXT_ATTACHMENT_W_CONTENTTYPE_NULL: &str = \"MIME-Version: 1.0\nReceived: by 10.76.127.40 with HTTP; Fri, 17 Jan 2014 02:21:43 -0800 (PST)\nDate: Fri, 17 Jan 2014 12:21:43 +0200\nDelivered-To: chirila.s.alexandru@gmail.com\nMessage-ID: <CALTHOsuHFaaatiXJKU=LdDCo4NmD_h49yvG2RDsWw17D0-NXJg@mail.gmail.com>\nSubject: Test\nFrom: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nTo: Alexandru Chirila <chirila.s.alexandru@gmail.com>\nContent-Type: multipart/mixed; boundary=f46d040a62c49bb1c804f027e8cc\n\n--f46d040a62c49bb1c804f027e8cc\nContent-Type: multipart/alternative; boundary=f46d040a62c49bb1c404f027e8ca\n\n--f46d040a62c49bb1c404f027e8ca\nContent-Type: text/plain; charset=\\\"iso-8859-1\\0\\0\\0\\\"\n\nThis is a test mailing\n--f46d040a62c49bb1c804f027e8cc--\";\n\n    const HTML_RAW: &str = r#\"<html><head><title>Email spam</title></head><body>\n<p><b>Email spam</b>, also known as <b>junk email</b> \nor <b>unsolicited bulk email</b> (<i>UBE</i>), is a subset of \n<a href=\"/wiki/Spam_(electronic)\" title=\"Spam (electronic)\">electronic spam</a> \ninvolving nearly identical messages sent to numerous recipients by <a href=\"/wiki/Email\" title=\"Email\">\nemail</a>. Clicking on <a href=\"/wiki/Html_email#Security_vulnerabilities\" title=\"Html email\" class=\"mw-redirect\">\nlinks in spam email</a> may send users to <a href=\"/wiki/Phishing\" title=\"Phishing\">phishing</a> \nweb sites or sites that are hosting <a href=\"/wiki/Malware\" title=\"Malware\">malware</a>.</body></html>\"#;\n\n    const HTML_RAW_STRIPED: &str = concat!(\n        \"Email spam Email spam , also known as junk email or unsolicited bulk email ( UBE ),\",\n        \" is a subset of electronic spam involving nearly identical messages sent to numerous recipients by email\",\n        \" . Clicking on links in spam email may send users to phishing web sites or sites that are hosting malware .\"\n    );\n}\n"
  },
  {
    "path": "crates/spam-filter/src/modules/sanitize.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse crate::{Email, Hostname};\n\nimpl Hostname {\n    pub fn new(host: &str) -> Self {\n        let mut fqdn = host.trim_end_matches('.').to_lowercase();\n\n        // Decode punycode\n        if fqdn.contains(\"xn--\") {\n            let mut decoded = String::with_capacity(fqdn.len());\n\n            for part in fqdn.split('.') {\n                if !decoded.is_empty() {\n                    decoded.push('.');\n                }\n\n                if let Some(puny) = part\n                    .strip_prefix(\"xn--\")\n                    .and_then(idna::punycode::decode_to_string)\n                {\n                    decoded.push_str(&puny);\n                } else {\n                    decoded.push_str(part);\n                }\n            }\n\n            fqdn = decoded;\n        }\n\n        let ip = fqdn\n            .strip_prefix('[')\n            .and_then(|ip| ip.strip_suffix(']'))\n            .unwrap_or(&fqdn)\n            .parse::<IpAddr>()\n            .ok();\n\n        Hostname {\n            sld: if ip.is_none() {\n                psl::domain(fqdn.as_bytes()).and_then(|domain| {\n                    if domain.suffix().typ().is_some() {\n                        std::str::from_utf8(domain.as_bytes()).ok().map(Into::into)\n                    } else {\n                        None\n                    }\n                })\n            } else {\n                None\n            },\n            ip,\n            fqdn,\n        }\n    }\n}\n\nimpl Email {\n    pub fn new(address: &str) -> Self {\n        let address = address.to_lowercase();\n        let (local_part, domain) = address.rsplit_once('@').unwrap_or_default();\n\n        Email {\n            local_part: local_part.into(),\n            domain_part: Hostname::new(domain),\n            address,\n        }\n    }\n}\n\nimpl Hostname {\n    pub fn sld_or_default(&self) -> &str {\n        self.sld.as_deref().unwrap_or(self.fqdn.as_str())\n    }\n}\n"
  },
  {
    "path": "crates/store/Cargo.toml",
    "content": "[package]\nname = \"store\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path = \"../utils\" }\ntypes = { path = \"../types\" }\nnlp = { path = \"../nlp\" }\ntrc = { path = \"../trc\" }\nrocksdb = { version = \"0.24\", optional = true, features = [\"multi-threaded-cf\"] }\nfoundationdb = { version = \"0.9.2\", features = [\"embedded-fdb-include\", \"fdb-7_3\"], optional = true }\nrusqlite = { version = \"0.37\", features = [\"bundled\"], optional = true }\n#rust-s3 = { version = \"0.37\", default-features = false, features = [\"tokio-rustls-tls\"], optional = true }\nrust-s3 = { version = \"0.35\", default-features = false, features = [\"tokio-rustls-tls\", \"no-verify-ssl\"], optional = true }\nasync-nats = { version = \"0.44\", default-features = false, features = [\"server_2_10\", \"server_2_11\", \"ring\"], optional = true }\nazure_core = { version = \"0.21.0\", optional = true }\nazure_storage = { version = \"0.21.0\", default-features = false, features = [\"enable_reqwest_rustls\", \"hmac_rust\"], optional = true }\nazure_storage_blobs = { version = \"0.21.0\", default-features = false, features = [\"enable_reqwest_rustls\", \"hmac_rust\"], optional = true }\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\", \"stream\"]}\ntokio = { version = \"1.47\", features = [\"sync\", \"fs\", \"io-util\"] }\nr2d2 = { version = \"0.8.10\", optional = true }\nfutures = { version = \"0.3\", optional = true }\nrand = \"0.9.0\"\nroaring = \"0.11\"\nrayon = { version = \"1.11\", optional = true }\nserde = { version = \"1.0\", features = [\"derive\"]}\nahash = { version = \"0.8.2\", features = [\"serde\"] }\nxxhash-rust = { version = \"0.8.5\", features = [\"xxh3\"] }\nfarmhash = \"1.1.5\"\nparking_lot = \"0.12\"\nlru-cache = { version = \"0.1.2\", optional = true }\nnum_cpus = { version = \"1.17\", optional = true }\nblake3 = \"1.8\"\nlz4_flex = { version = \"0.12\", default-features = false }\ndeadpool-postgres = { version = \"0.14\", optional = true }\ntokio-postgres = { version = \"0.7.10\", features = [\"with-serde_json-1\"], optional = true }\ntokio-rustls = { version = \"0.26\", optional = true, default-features = false, features = [\"ring\", \"tls12\"] }\nrustls = { version = \"0.23.5\", optional = true, default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pki-types = { version = \"1\", optional = true }\nring = { version = \"0.17\", optional = true }\nbytes = { version = \"1.10\", optional = true }\nmysql_async = { version = \"0.36\", default-features = false, features = [\"default-rustls-ring\", \"minimal\"], optional = true }\nserde_json = { version = \"1.0.64\" }\nregex = \"1.12\"\nflate2 = \"1.1\"\nredis = { version = \"0.32\", features = [ \"tokio-comp\", \"tokio-rustls-comp\", \"tls-rustls-insecure\", \"tls-rustls-webpki-roots\", \"cluster-async\"], optional = true }\ndeadpool = { version = \"0.12\", features = [\"managed\"], optional = true }\narc-swap = \"1.6.0\"\nbitpacking = \"0.9.2\"\nmemchr = { version = \"2.7\" }\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\nzenoh = { version = \"1.3.4\", default-features = false, features = [\"auth_pubkey\", \"transport_multilink\", \"transport_compression\", \"transport_quic\", \"transport_tcp\", \"transport_tls\", \"transport_udp\"], optional = true }\nrdkafka = { version = \"0.38\", features = [\"cmake-build\"], optional = true }\nrustls_021 = { package = \"rustls\", version = \"0.21\", default-features = false, features = [\"dangerous_configuration\"], optional = true }\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n\n[features]\n# Data Stores\nrocks = [\"rocksdb\", \"rayon\", \"num_cpus\"]\nsqlite = [\"rusqlite\", \"rayon\", \"r2d2\", \"num_cpus\", \"lru-cache\"]\npostgres = [\"tokio-postgres\", \"deadpool\", \"deadpool-postgres\", \"tokio-rustls\", \"rustls\", \"ring\", \"rustls-pki-types\", \"futures\", \"bytes\"]\nmysql = [\"mysql_async\", \"futures\"]\nfoundation = [\"foundationdb\", \"futures\"]\nfdb-chunked-bm = []\n\n# Blob stores\ns3 = [\"rust-s3\", \"rustls_021\"]\nazure = [\"azure_core\", \"azure_storage\", \"azure_storage_blobs\"]\n\n# In-memory stores\nredis = [\"dep:redis\", \"deadpool\", \"futures\"]\n\n# Pubsub\nnats = [\"async-nats\"]\nzenoh = [\"dep:zenoh\"]\nkafka = [\"rdkafka\"]\n\nenterprise = []\ntest_mode = []\n"
  },
  {
    "path": "crates/store/src/backend/azure/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fmt::Display, io::Write, ops::Range, time::Duration};\n\nuse azure_core::error::ErrorKind;\nuse azure_core::{ExponentialRetryOptions, RetryOptions, StatusCode, TransportOptions};\nuse azure_storage::StorageCredentials;\nuse azure_storage_blobs::prelude::{ClientBuilder, ContainerClient};\nuse futures::stream::StreamExt;\nuse std::sync::Arc;\nuse utils::{\n    codec::base32_custom::Base32Writer,\n    config::{Config, utils::AsKey},\n};\n\npub struct AzureStore {\n    client: ContainerClient,\n    prefix: Option<String>,\n}\n\nimpl AzureStore {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n\n        let storage_account = config\n            .value_require((&prefix, \"storage-account\"))?\n            .to_string();\n        let container = config.value_require((&prefix, \"container\"))?.to_string();\n\n        let credentials = match (\n            config.value((&prefix, \"azure-access-key\")),\n            config.value((&prefix, \"sas-token\")),\n        ) {\n            (Some(access_key), None) => {\n                StorageCredentials::access_key(storage_account.clone(), access_key.to_string())\n            }\n            (None, Some(sas_token)) => match StorageCredentials::sas_token(sas_token) {\n                Ok(cred) => cred,\n                Err(err) => {\n                    config.new_build_error(\n                        prefix.as_str(),\n                        format!(\"Failed to create credentials: {err:?}\"),\n                    );\n                    return None;\n                }\n            },\n            _ => {\n                config.new_build_error(\n                    prefix.as_str(),\n                    concat!(\n                        \"Failed to create credentials: exactly one of \",\n                        \"'azure-access-key' and 'sas-token' must be specified\"\n                    ),\n                );\n                return None;\n            }\n        };\n\n        let timeout = config\n            .property_or_default::<Duration>((&prefix, \"timeout\"), \"30s\")\n            .unwrap_or_else(|| Duration::from_secs(30));\n        let transport = match reqwest::Client::builder().timeout(timeout).build() {\n            Ok(client) => Arc::new(client),\n            Err(err) => {\n                config.new_build_error(\n                    prefix.as_str(),\n                    format!(\"Failed to create HTTP client: {err:?}\"),\n                );\n                return None;\n            }\n        };\n\n        // Take the configured number of retries and multiply by 2. This is intended to match the\n        // precedent set by the S3 back end, where we do the indicated number of retries,\n        // ourselves, but internally the rust-s3 crate is also retrying each of our requests up to\n        // one additional time, itself. So our retries, and the S3 backend's retries, are\n        // comparable to each other.\n        let max_retries: u32 = config\n            .property_or_default((&prefix, \"max-retries\"), \"3\")\n            .unwrap_or(3)\n            * 2;\n\n        Some(AzureStore {\n            client: ClientBuilder::new(storage_account, credentials)\n                .transport(TransportOptions::new(transport))\n                .retry(RetryOptions::exponential(\n                    ExponentialRetryOptions::default().max_retries(max_retries),\n                ))\n                .container_client(container),\n            prefix: config.value((&prefix, \"key-prefix\")).map(|s| s.to_string()),\n        })\n    }\n\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let blob_client = self.client.blob_client(self.build_key(key));\n\n        let mut stream = blob_client.get();\n        let mut buf = if range.end == usize::MAX {\n            // Let's turn this into a proper RangeFrom.\n            stream = stream.range(range.start..);\n            // We don't know how big to expect the result to be.\n            Vec::new()\n        } else {\n            stream = stream.range(range.clone());\n            Vec::with_capacity(range.end - range.start)\n        };\n        let mut stream = stream.into_stream();\n\n        while let Some(response) = stream.next().await {\n            let err = match response {\n                Ok(chunks) => {\n                    let mut chunks = chunks.data;\n                    let mut err = None;\n                    while let Some(chunk) = chunks.next().await {\n                        match chunk {\n                            Ok(ref data) => {\n                                buf.extend(data);\n                            }\n                            Err(e) => {\n                                err = Some(e);\n                                break;\n                            }\n                        }\n                    }\n                    err\n                }\n                Err(e) => Some(e),\n            };\n\n            if let Some(e) = err {\n                return if matches!(\n                    e.kind(),\n                    ErrorKind::HttpResponse {\n                        status: StatusCode::NotFound,\n                        ..\n                    }\n                ) {\n                    Ok(None)\n                } else {\n                    Err(trc::StoreEvent::AzureError.reason(e))\n                };\n            }\n        }\n\n        Ok(Some(buf))\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let blob_client = self.client.blob_client(self.build_key(key));\n\n        // We unfortunately have to make a copy of `data`. This is because the Azure SDK wants to\n        // coerce the body into a value of type azure_core::Body, which doesn't have a lifetime\n        // parameter and so cannot hold any non-static references (directly or indirectly).\n        let data = data.to_vec();\n\n        blob_client\n            .put_block_blob(data)\n            .into_future()\n            .await\n            .map_err(into_error)?;\n\n        Ok(())\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let blob_client = self.client.blob_client(self.build_key(key));\n\n        if let Err(e) = blob_client.delete().into_future().await {\n            if matches!(\n                e.kind(),\n                ErrorKind::HttpResponse {\n                    status: StatusCode::NotFound,\n                    ..\n                }\n            ) {\n                Ok(false)\n            } else {\n                Err(trc::StoreEvent::AzureError.reason(e))\n            }\n        } else {\n            Ok(true)\n        }\n    }\n\n    fn build_key(&self, key: &[u8]) -> String {\n        if let Some(prefix) = &self.prefix {\n            let mut writer =\n                Base32Writer::with_raw_capacity(prefix.len() + (key.len().div_ceil(4) * 5));\n            writer.push_string(prefix);\n            writer.write_all(key).unwrap();\n            writer.finalize()\n        } else {\n            Base32Writer::from_bytes(key).finalize()\n        }\n    }\n}\n\n#[inline(always)]\nfn into_error(err: impl Display) -> trc::Error {\n    trc::StoreEvent::AzureError.reason(err)\n}\n"
  },
  {
    "path": "crates/store/src/backend/composite/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\n#[cfg(any(feature = \"postgres\", feature = \"mysql\"))]\npub mod read_replica;\npub mod sharded_blob;\npub mod sharded_lookup;\n"
  },
  {
    "path": "crates/store/src/backend/composite/read_replica.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse crate::{\n    Deserialize, IterateParams, Key, Store, Stores, ValueKey,\n    search::{IndexDocument, SearchComparator, SearchDocumentId, SearchFilter, SearchQuery},\n    write::{AssignedIds, Batch, SearchIndex, ValueClass},\n};\nuse std::{\n    future::Future,\n    ops::Range,\n    sync::atomic::{AtomicUsize, Ordering},\n};\nuse utils::config::{Config, utils::AsKey};\n\npub struct SQLReadReplica {\n    primary: Store,\n    replicas: Vec<Store>,\n    last_used_replica: AtomicUsize,\n}\n\nimpl SQLReadReplica {\n    pub async fn open(\n        config: &mut Config,\n        prefix: impl AsKey,\n        stores: &Stores,\n        create_store_tables: bool,\n        create_search_tables: bool,\n    ) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let primary_id = config.value_require((&prefix, \"primary\"))?.to_string();\n        let replica_ids = config\n            .values((&prefix, \"replicas\"))\n            .map(|(_, v)| v.to_string())\n            .collect::<Vec<_>>();\n\n        let primary = if let Some(store) = stores.stores.get(&primary_id) {\n            if store.is_pg_or_mysql() {\n                store.clone()\n            } else {\n                config.new_build_error(\n                    (&prefix, \"primary\"),\n                    \"Primary store must be a PostgreSQL or MySQL store\",\n                );\n                return None;\n            }\n        } else {\n            config.new_build_error(\n                (&prefix, \"primary\"),\n                format!(\"Primary store {primary_id} not found\"),\n            );\n            return None;\n        };\n        let mut replicas = Vec::with_capacity(replica_ids.len());\n        for replica_id in replica_ids {\n            if let Some(store) = stores.stores.get(&replica_id) {\n                if store.is_pg_or_mysql() {\n                    replicas.push(store.clone());\n                } else {\n                    config.new_build_error(\n                        (&prefix, \"replicas\"),\n                        \"Replica store must be a PostgreSQL or MySQL store\",\n                    );\n                    return None;\n                }\n            } else {\n                config.new_build_error(\n                    (&prefix, \"replicas\"),\n                    format!(\"Replica store {replica_id} not found\"),\n                );\n                return None;\n            }\n        }\n        if !replicas.is_empty() {\n            if create_store_tables {\n                let result = match &primary {\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.create_storage_tables().await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.create_storage_tables().await,\n                    _ => panic!(\"Invalid store type\"),\n                };\n\n                if let Err(err) = result {\n                    config.new_build_error(\n                        (&prefix, \"primary\"),\n                        format!(\"Failed to create tables: {err}\"),\n                    );\n                }\n            }\n\n            if create_search_tables {\n                let result = match &primary {\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.create_search_tables().await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.create_search_tables().await,\n                    _ => panic!(\"Invalid store type\"),\n                };\n\n                if let Err(err) = result {\n                    config.new_build_warning(\n                        (&prefix, \"primary\"),\n                        format!(\"Failed to create search tables: {err}\"),\n                    );\n                }\n            }\n\n            Some(Self {\n                primary,\n                replicas,\n                last_used_replica: AtomicUsize::new(0),\n            })\n        } else {\n            config.new_build_error((&prefix, \"replicas\"), \"No replica stores specified\");\n            None\n        }\n    }\n\n    async fn run_op<'x, F, T, R>(&'x self, f: F) -> trc::Result<T>\n    where\n        F: Fn(&'x Store) -> R,\n        R: Future<Output = trc::Result<T>>,\n        T: 'static,\n    {\n        let mut last_error = None;\n        for store in [\n            &self.replicas\n                [self.last_used_replica.fetch_add(1, Ordering::Relaxed) % self.replicas.len()],\n            &self.primary,\n        ] {\n            match f(store).await {\n                Ok(result) => return Ok(result),\n                Err(err) => {\n                    if err.is_assertion_failure() {\n                        return Err(err);\n                    } else {\n                        last_error = Some(err);\n                    }\n                }\n            }\n        }\n\n        Err(last_error.unwrap())\n    }\n\n    pub async fn get_blob(&self, key: &[u8], range: Range<usize>) -> trc::Result<Option<Vec<u8>>> {\n        self.run_op(move |store| {\n            let range = range.clone();\n\n            async move {\n                match store {\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.get_blob(key, range).await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.get_blob(key, range).await,\n                    _ => panic!(\"Invalid store type\"),\n                }\n            }\n        })\n        .await\n    }\n\n    pub async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.put_blob(key, data).await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.put_blob(key, data).await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n\n    pub async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.delete_blob(key).await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.delete_blob(key).await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n\n    pub async fn get_value<U>(&self, key: impl Key) -> trc::Result<Option<U>>\n    where\n        U: Deserialize + 'static,\n    {\n        self.run_op(move |store| {\n            let key = key.clone();\n\n            async move {\n                match store {\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.get_value(key).await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.get_value(key).await,\n                    _ => panic!(\"Invalid store type\"),\n                }\n            }\n        })\n        .await\n    }\n\n    pub async fn iterate<T: Key>(\n        &self,\n        params: IterateParams<T>,\n        mut cb: impl for<'x> FnMut(&'x [u8], &'x [u8]) -> trc::Result<bool> + Sync + Send,\n    ) -> trc::Result<()> {\n        let mut last_error = None;\n        for store in [\n            &self.replicas\n                [self.last_used_replica.fetch_add(1, Ordering::Relaxed) % self.replicas.len()],\n            &self.primary,\n        ] {\n            match match store {\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => store.iterate(params.clone(), &mut cb).await,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => store.iterate(params.clone(), &mut cb).await,\n                _ => panic!(\"Invalid store type\"),\n            } {\n                Ok(result) => return Ok(result),\n                Err(err) => {\n                    last_error = Some(err);\n                }\n            }\n        }\n\n        Err(last_error.unwrap())\n    }\n\n    pub async fn get_counter(\n        &self,\n        key: impl Into<ValueKey<ValueClass>> + Sync + Send,\n    ) -> trc::Result<i64> {\n        let key = key.into();\n        self.run_op(move |store| {\n            let key = key.clone();\n\n            async move {\n                match store {\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.get_counter(key).await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.get_counter(key).await,\n                    _ => panic!(\"Invalid store type\"),\n                }\n            }\n        })\n        .await\n    }\n\n    pub async fn write(&self, batch: Batch<'_>) -> trc::Result<AssignedIds> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.write(batch).await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.write(batch).await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n\n    pub async fn delete_range(&self, from: impl Key, to: impl Key) -> trc::Result<()> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.delete_range(from, to).await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.delete_range(from, to).await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n\n    pub async fn purge_store(&self) -> trc::Result<()> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.purge_store().await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.purge_store().await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n\n    pub async fn index(&self, documents: Vec<IndexDocument>) -> trc::Result<()> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.index(documents).await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.index(documents).await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n\n    pub async fn unindex(&self, query: SearchQuery) -> trc::Result<u64> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.unindex(query).await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.unindex(query).await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n\n    pub async fn query<R: SearchDocumentId>(\n        &self,\n        index: SearchIndex,\n        filters: &[SearchFilter],\n        sort: &[SearchComparator],\n    ) -> trc::Result<Vec<R>> {\n        match &self.primary {\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(store) => store.query(index, filters, sort).await,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(store) => store.query(index, filters, sort).await,\n            _ => panic!(\"Invalid store type\"),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/composite/sharded_blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse std::ops::Range;\n\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::{BlobBackend, Store, Stores};\n\npub struct ShardedBlob {\n    pub stores: Vec<BlobBackend>,\n}\n\nimpl ShardedBlob {\n    pub fn open(config: &mut Config, prefix: impl AsKey, stores: &Stores) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let store_ids = config\n            .values((&prefix, \"stores\"))\n            .map(|(_, v)| v.to_string())\n            .collect::<Vec<_>>();\n\n        let mut blob_stores = Vec::with_capacity(store_ids.len());\n        for store_id in store_ids {\n            if let Some(store) = stores.blob_stores.get(&store_id) {\n                blob_stores.push(store.backend.clone());\n            } else {\n                config.new_build_error(\n                    (&prefix, \"stores\"),\n                    format!(\"Blob store {store_id} not found\"),\n                );\n                return None;\n            }\n        }\n        if !blob_stores.is_empty() {\n            Some(Self {\n                stores: blob_stores,\n            })\n        } else {\n            config.new_build_error((&prefix, \"stores\"), \"No blob stores specified\");\n            None\n        }\n    }\n\n    #[inline(always)]\n    fn get_store(&self, key: &[u8]) -> &BlobBackend {\n        &self.stores[xxhash_rust::xxh3::xxh3_64(key) as usize % self.stores.len()]\n    }\n\n    pub async fn get_blob(\n        &self,\n        key: &[u8],\n        read_range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        Box::pin(async move {\n            match self.get_store(key) {\n                BlobBackend::Store(store) => match store {\n                    #[cfg(feature = \"sqlite\")]\n                    Store::SQLite(store) => store.get_blob(key, read_range).await,\n                    #[cfg(feature = \"foundation\")]\n                    Store::FoundationDb(store) => store.get_blob(key, read_range).await,\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.get_blob(key, read_range).await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.get_blob(key, read_range).await,\n                    #[cfg(feature = \"rocks\")]\n                    Store::RocksDb(store) => store.get_blob(key, read_range).await,\n                    // SPDX-SnippetBegin\n                    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                    // SPDX-License-Identifier: LicenseRef-SEL\n                    #[cfg(all(\n                        feature = \"enterprise\",\n                        any(feature = \"postgres\", feature = \"mysql\")\n                    ))]\n                    Store::SQLReadReplica(store) => store.get_blob(key, read_range).await,\n                    // SPDX-SnippetEnd\n                    Store::None => Err(trc::StoreEvent::NotConfigured.into()),\n                },\n                BlobBackend::Fs(store) => store.get_blob(key, read_range).await,\n                #[cfg(feature = \"s3\")]\n                BlobBackend::S3(store) => store.get_blob(key, read_range).await,\n                #[cfg(feature = \"azure\")]\n                BlobBackend::Azure(store) => store.get_blob(key, read_range).await,\n                BlobBackend::Sharded(_) => unimplemented!(),\n            }\n        })\n        .await\n    }\n\n    pub async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        Box::pin(async move {\n            match self.get_store(key) {\n                BlobBackend::Store(store) => match store {\n                    #[cfg(feature = \"sqlite\")]\n                    Store::SQLite(store) => store.put_blob(key, data).await,\n                    #[cfg(feature = \"foundation\")]\n                    Store::FoundationDb(store) => store.put_blob(key, data).await,\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.put_blob(key, data).await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.put_blob(key, data).await,\n                    #[cfg(feature = \"rocks\")]\n                    Store::RocksDb(store) => store.put_blob(key, data).await,\n                    // SPDX-SnippetBegin\n                    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                    // SPDX-License-Identifier: LicenseRef-SEL\n                    #[cfg(all(\n                        feature = \"enterprise\",\n                        any(feature = \"postgres\", feature = \"mysql\")\n                    ))]\n                    // SPDX-SnippetEnd\n                    Store::SQLReadReplica(store) => store.put_blob(key, data).await,\n                    Store::None => Err(trc::StoreEvent::NotConfigured.into()),\n                },\n                BlobBackend::Fs(store) => store.put_blob(key, data).await,\n                #[cfg(feature = \"s3\")]\n                BlobBackend::S3(store) => store.put_blob(key, data).await,\n                #[cfg(feature = \"azure\")]\n                BlobBackend::Azure(store) => store.put_blob(key, data).await,\n                BlobBackend::Sharded(_) => unimplemented!(),\n            }\n        })\n        .await\n    }\n\n    pub async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        Box::pin(async move {\n            match self.get_store(key) {\n                BlobBackend::Store(store) => match store {\n                    #[cfg(feature = \"sqlite\")]\n                    Store::SQLite(store) => store.delete_blob(key).await,\n                    #[cfg(feature = \"foundation\")]\n                    Store::FoundationDb(store) => store.delete_blob(key).await,\n                    #[cfg(feature = \"postgres\")]\n                    Store::PostgreSQL(store) => store.delete_blob(key).await,\n                    #[cfg(feature = \"mysql\")]\n                    Store::MySQL(store) => store.delete_blob(key).await,\n                    #[cfg(feature = \"rocks\")]\n                    Store::RocksDb(store) => store.delete_blob(key).await,\n                    // SPDX-SnippetBegin\n                    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                    // SPDX-License-Identifier: LicenseRef-SEL\n                    #[cfg(all(\n                        feature = \"enterprise\",\n                        any(feature = \"postgres\", feature = \"mysql\")\n                    ))]\n                    Store::SQLReadReplica(store) => store.delete_blob(key).await,\n                    // SPDX-SnippetEnd\n                    Store::None => Err(trc::StoreEvent::NotConfigured.into()),\n                },\n                BlobBackend::Fs(store) => store.delete_blob(key).await,\n                #[cfg(feature = \"s3\")]\n                BlobBackend::S3(store) => store.delete_blob(key).await,\n                #[cfg(feature = \"azure\")]\n                BlobBackend::Azure(store) => store.delete_blob(key).await,\n                BlobBackend::Sharded(_) => unimplemented!(),\n            }\n        })\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/composite/sharded_lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::{\n    Deserialize, InMemoryStore, Stores, Value,\n    dispatch::lookup::{KeyValue, LookupKey},\n};\n\n#[derive(Debug)]\npub struct ShardedInMemory {\n    pub stores: Vec<InMemoryStore>,\n}\n\nimpl ShardedInMemory {\n    pub fn open(config: &mut Config, prefix: impl AsKey, stores: &Stores) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let store_ids = config\n            .values((&prefix, \"stores\"))\n            .map(|(_, v)| v.to_string())\n            .collect::<Vec<_>>();\n\n        let mut in_memory_stores = Vec::with_capacity(store_ids.len());\n        for store_id in store_ids {\n            if let Some(store) = stores\n                .in_memory_stores\n                .get(&store_id)\n                .filter(|store| store.is_redis())\n            {\n                in_memory_stores.push(store.clone());\n            } else {\n                config.new_build_error(\n                    (&prefix, \"stores\"),\n                    format!(\"In-memory store {store_id} not found\"),\n                );\n                return None;\n            }\n        }\n        if !in_memory_stores.is_empty() {\n            Some(Self {\n                stores: in_memory_stores,\n            })\n        } else {\n            config.new_build_error((&prefix, \"stores\"), \"No in-memory stores specified\");\n            None\n        }\n    }\n\n    #[inline(always)]\n    fn get_store(&self, key: &[u8]) -> &InMemoryStore {\n        &self.stores[xxhash_rust::xxh3::xxh3_64(key) as usize % self.stores.len()]\n    }\n\n    pub async fn key_set(&self, kv: KeyValue<Vec<u8>>) -> trc::Result<()> {\n        Box::pin(async move {\n            match self.get_store(&kv.key) {\n                #[cfg(feature = \"redis\")]\n                InMemoryStore::Redis(store) => store.key_set(&kv.key, &kv.value, kv.expires).await,\n                InMemoryStore::Static(_) => Err(trc::StoreEvent::NotSupported.into_err()),\n                _ => Err(trc::StoreEvent::NotSupported.into_err()),\n            }\n        })\n        .await\n    }\n\n    pub async fn counter_incr(&self, kv: KeyValue<i64>) -> trc::Result<i64> {\n        Box::pin(async move {\n            match self.get_store(&kv.key) {\n                #[cfg(feature = \"redis\")]\n                InMemoryStore::Redis(store) => store.key_incr(&kv.key, kv.value, kv.expires).await,\n                InMemoryStore::Static(_) => Err(trc::StoreEvent::NotSupported.into_err()),\n                _ => Err(trc::StoreEvent::NotSupported.into_err()),\n            }\n        })\n        .await\n    }\n\n    pub async fn key_delete(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<()> {\n        let key_ = key.into();\n        let key = key_.as_bytes();\n        Box::pin(async move {\n            match self.get_store(key) {\n                #[cfg(feature = \"redis\")]\n                InMemoryStore::Redis(store) => store.key_delete(key).await,\n                InMemoryStore::Static(_) => Err(trc::StoreEvent::NotSupported.into_err()),\n                _ => Err(trc::StoreEvent::NotSupported.into_err()),\n            }\n        })\n        .await\n    }\n\n    pub async fn counter_delete(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<()> {\n        let key_ = key.into();\n        let key = key_.as_bytes();\n        Box::pin(async move {\n            match self.get_store(key) {\n                #[cfg(feature = \"redis\")]\n                InMemoryStore::Redis(store) => store.key_delete(key).await,\n                InMemoryStore::Static(_) => Err(trc::StoreEvent::NotSupported.into_err()),\n                _ => Err(trc::StoreEvent::NotSupported.into_err()),\n            }\n        })\n        .await\n    }\n\n    #[allow(unused_variables)]\n    pub async fn key_delete_prefix(&self, prefix: &[u8]) -> trc::Result<()> {\n        Box::pin(async move {\n            #[cfg(feature = \"redis\")]\n            for store in &self.stores {\n                match store {\n                    InMemoryStore::Redis(store) => store.key_delete_prefix(prefix).await?,\n                    InMemoryStore::Static(_) => {\n                        return Err(trc::StoreEvent::NotSupported.into_err());\n                    }\n                    _ => return Err(trc::StoreEvent::NotSupported.into_err()),\n                }\n            }\n\n            Ok(())\n        })\n        .await\n    }\n\n    pub async fn key_get<T: Deserialize + From<Value<'static>> + std::fmt::Debug + 'static>(\n        &self,\n        key: impl Into<LookupKey<'_>>,\n    ) -> trc::Result<Option<T>> {\n        let key_ = key.into();\n        let key = key_.as_bytes();\n        Box::pin(async move {\n            match self.get_store(key) {\n                #[cfg(feature = \"redis\")]\n                InMemoryStore::Redis(store) => store.key_get(key).await,\n                InMemoryStore::Static(_) => Err(trc::StoreEvent::NotSupported.into_err()),\n                _ => Err(trc::StoreEvent::NotSupported.into_err()),\n            }\n        })\n        .await\n    }\n\n    pub async fn counter_get(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<i64> {\n        let key_ = key.into();\n        let key = key_.as_bytes();\n        Box::pin(async move {\n            match self.get_store(key) {\n                #[cfg(feature = \"redis\")]\n                InMemoryStore::Redis(store) => store.counter_get(key).await,\n                InMemoryStore::Static(_) => Err(trc::StoreEvent::NotSupported.into_err()),\n                _ => Err(trc::StoreEvent::NotSupported.into_err()),\n            }\n        })\n        .await\n    }\n\n    pub async fn key_exists(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<bool> {\n        let key_ = key.into();\n        let key = key_.as_bytes();\n        Box::pin(async move {\n            match self.get_store(key) {\n                #[cfg(feature = \"redis\")]\n                InMemoryStore::Redis(store) => store.key_exists(key).await,\n                InMemoryStore::Static(_) => Err(trc::StoreEvent::NotSupported.into_err()),\n                _ => Err(trc::StoreEvent::NotSupported.into_err()),\n            }\n        })\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/elastic/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    backend::elastic::ElasticSearchStore,\n    search::{\n        CalendarSearchField, ContactSearchField, EmailSearchField, SearchableField,\n        TracingSearchField,\n    },\n};\nuse reqwest::{Error, Response, Url};\nuse serde_json::{Value, json};\nuse utils::config::{Config, http::build_http_client, utils::AsKey};\n\nimpl ElasticSearchStore {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let client = build_http_client(config, prefix.clone(), \"application/json\".into())?;\n        let prefix = prefix.as_key();\n        let url = config\n            .value_require((&prefix, \"url\"))?\n            .trim_end_matches(\"/\")\n            .to_string();\n        Url::parse(&url)\n            .map_err(|e| config.new_parse_error((&prefix, \"url\"), format!(\"Invalid URL: {e}\",)))\n            .ok()?;\n\n        let es = Self { client, url };\n\n        let shards = config\n            .property_or_default((&prefix, \"index.shards\"), \"3\")\n            .unwrap_or(3);\n        let replicas = config\n            .property_or_default((&prefix, \"index.replicas\"), \"0\")\n            .unwrap_or(0);\n        let with_source = config\n            .property_or_default((&prefix, \"index.include-source\"), \"false\")\n            .unwrap_or(false);\n\n        if let Err(err) = es.create_indexes(shards, replicas, with_source).await {\n            config.new_build_error(prefix.as_str(), err.to_string());\n        }\n\n        Some(es)\n    }\n\n    pub async fn create_indexes(\n        &self,\n        shards: usize,\n        replicas: usize,\n        with_source: bool,\n    ) -> trc::Result<()> {\n        self.create_index::<EmailSearchField>(shards, replicas, with_source)\n            .await?;\n        self.create_index::<CalendarSearchField>(shards, replicas, with_source)\n            .await?;\n        self.create_index::<ContactSearchField>(shards, replicas, with_source)\n            .await?;\n        self.create_index::<TracingSearchField>(shards, replicas, with_source)\n            .await?;\n        Ok(())\n    }\n\n    async fn create_index<T: SearchableField>(\n        &self,\n        shards: usize,\n        replicas: usize,\n        with_source: bool,\n    ) -> trc::Result<()> {\n        let mut mappings = serde_json::Map::new();\n        mappings.insert(\n            \"properties\".to_string(),\n            Value::Object(\n                T::primary_keys()\n                    .iter()\n                    .chain(T::all_fields())\n                    .map(|field| (field.field_name().to_string(), field.es_schema()))\n                    .collect::<serde_json::Map<String, Value>>(),\n            ),\n        );\n        if !with_source {\n            mappings.insert(\"_source\".to_string(), json!({ \"enabled\": false }));\n        }\n        let body = json!({\n          \"mappings\": mappings,\n          \"settings\": {\n            \"index.number_of_shards\": shards,\n            \"index.number_of_replicas\": replicas,\n            \"analysis\": {\n              \"analyzer\": {\n                \"default\": {\n                  \"type\": \"custom\",\n                  \"tokenizer\": \"standard\",\n                  \"filter\": [\"lowercase\", \"stemmer\"]\n                }\n              }\n            }\n          }\n        });\n\n        let response = self\n            .client\n            .put(format!(\"{}/{}\", self.url, T::index().index_name()))\n            .body(body.to_string())\n            .send()\n            .await\n            .map_err(|err| {\n                trc::StoreEvent::ElasticsearchError\n                    .reason(err)\n                    .details(\"Failed to create index\")\n            })?;\n\n        match response.status().as_u16() {\n            200..300 => Ok(()),\n            status @ (400..500) => {\n                let text = response.text().await.unwrap_or_default();\n                if text.contains(\"resource_already_exists_exception\") {\n                    // Index already exists, ignore\n                    Ok(())\n                } else {\n                    Err(trc::StoreEvent::ElasticsearchError\n                        .reason(text)\n                        .ctx(trc::Key::Code, status))\n                }\n            }\n            status => {\n                let text = response.text().await.unwrap_or_default();\n                Err(trc::StoreEvent::ElasticsearchError\n                    .reason(text)\n                    .ctx(trc::Key::Code, status))\n            }\n        }\n    }\n\n    #[cfg(feature = \"test_mode\")]\n    pub async fn drop_indexes(&self) -> trc::Result<()> {\n        use crate::write::SearchIndex;\n\n        for index in &[\n            SearchIndex::Email,\n            SearchIndex::Calendar,\n            SearchIndex::Contacts,\n            SearchIndex::Tracing,\n        ] {\n            assert_success(\n                self.client\n                    .delete(format!(\"{}/{}\", self.url, index.index_name()))\n                    .send()\n                    .await,\n            )\n            .await\n            .map(|_| ())?;\n        }\n\n        Ok(())\n    }\n}\n\npub(crate) async fn assert_success(response: Result<Response, Error>) -> trc::Result<Response> {\n    match response {\n        Ok(response) => {\n            let status = response.status();\n            if status.is_success() {\n                Ok(response)\n            } else {\n                Err(trc::StoreEvent::ElasticsearchError\n                    .reason(response.text().await.unwrap_or_default())\n                    .ctx(trc::Key::Code, status.as_u16()))\n            }\n        }\n        Err(err) => Err(trc::StoreEvent::ElasticsearchError.reason(err)),\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/elastic/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::search::*;\nuse reqwest::Client;\nuse serde::{Deserialize, Deserializer};\nuse serde_json::{Value, json};\n\npub mod main;\npub mod search;\n\npub struct ElasticSearchStore {\n    client: Client,\n    url: String,\n}\n\n#[derive(Debug, Deserialize)]\npub struct SearchResponse {\n    pub hits: Hits,\n}\n\n#[derive(Debug, Deserialize)]\npub struct Hits {\n    pub total: Total,\n    pub hits: Vec<Hit>,\n}\n\n#[derive(Debug, Deserialize)]\npub struct Total {\n    pub value: u64,\n}\n\n#[derive(Debug, Deserialize)]\npub struct Hit {\n    #[serde(rename = \"_id\", deserialize_with = \"deserialize_string_to_u64\")]\n    pub id: u64,\n    pub sort: Option<Value>,\n}\n\n#[derive(Debug, Deserialize)]\npub struct DeleteByQueryResponse {\n    pub deleted: u64,\n}\n\nimpl SearchField {\n    pub fn es_schema(&self) -> Value {\n        match self {\n            SearchField::AccountId\n            | SearchField::DocumentId\n            | SearchField::Email(EmailSearchField::Size) => json!({\n              \"type\": \"integer\"\n            }),\n            SearchField::Id\n            | SearchField::Email(EmailSearchField::SentAt | EmailSearchField::ReceivedAt)\n            | SearchField::Calendar(CalendarSearchField::Start)\n            | SearchField::Tracing(TracingSearchField::QueueId | TracingSearchField::EventType) => {\n                json!({\n                  \"type\": \"long\"\n                })\n            }\n            SearchField::Email(EmailSearchField::HasAttachment) => json!({\n              \"type\": \"boolean\"\n            }),\n            SearchField::Calendar(CalendarSearchField::Uid)\n            | SearchField::Contact(ContactSearchField::Uid) => json!({\n              \"type\": \"keyword\",\n            }),\n            SearchField::Email(\n                EmailSearchField::From | EmailSearchField::To | EmailSearchField::Subject,\n            ) => json!({\n              \"type\": \"text\",\n              \"fields\": {\n                \"keyword\": {\n                  \"type\": \"keyword\"\n                }\n              }\n            }),\n            SearchField::Email(EmailSearchField::Headers) => {\n                json!({\n                  \"type\": \"object\",\n                  \"enabled\": true\n                })\n            }\n            #[cfg(feature = \"test_mode\")]\n            SearchField::Email(EmailSearchField::Bcc | EmailSearchField::Cc) => {\n                json!({\n                  \"type\": \"text\",\n                  \"fields\": {\n                    \"keyword\": {\n                      \"type\": \"keyword\"\n                    }\n                  }\n                })\n            }\n            #[cfg(not(feature = \"test_mode\"))]\n            SearchField::Email(EmailSearchField::Bcc | EmailSearchField::Cc) => {\n                json!({\n                  \"type\": \"text\"\n                })\n            }\n            SearchField::Email(EmailSearchField::Body | EmailSearchField::Attachment)\n            | SearchField::Calendar(\n                CalendarSearchField::Title\n                | CalendarSearchField::Description\n                | CalendarSearchField::Location\n                | CalendarSearchField::Owner\n                | CalendarSearchField::Attendee,\n            )\n            | SearchField::Contact(\n                ContactSearchField::Member\n                | ContactSearchField::Kind\n                | ContactSearchField::Name\n                | ContactSearchField::Nickname\n                | ContactSearchField::Organization\n                | ContactSearchField::Email\n                | ContactSearchField::Phone\n                | ContactSearchField::OnlineService\n                | ContactSearchField::Address\n                | ContactSearchField::Note,\n            )\n            | SearchField::File(FileSearchField::Name | FileSearchField::Content)\n            | SearchField::Tracing(TracingSearchField::Keywords) => json!({\n              \"type\": \"text\"\n            }),\n        }\n    }\n}\n\nfn deserialize_string_to_u64<'de, D>(deserializer: D) -> Result<u64, D::Error>\nwhere\n    D: Deserializer<'de>,\n{\n    <&str>::deserialize(deserializer)?\n        .parse::<u64>()\n        .map_err(serde::de::Error::custom)\n}\n"
  },
  {
    "path": "crates/store/src/backend/elastic/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    backend::elastic::{\n        DeleteByQueryResponse, ElasticSearchStore, SearchResponse, main::assert_success,\n    },\n    search::{\n        IndexDocument, SearchComparator, SearchDocumentId, SearchField, SearchFilter,\n        SearchOperator, SearchQuery, SearchValue,\n    },\n    write::SearchIndex,\n};\nuse serde_json::{Map, Value, json};\nuse std::fmt::Write;\n\nimpl ElasticSearchStore {\n    pub async fn index(&self, documents: Vec<IndexDocument>) -> trc::Result<()> {\n        let mut request = String::with_capacity(512);\n\n        for document in documents {\n            let id = if let (Some(SearchValue::Uint(account_id)), Some(SearchValue::Uint(doc_id))) = (\n                document.fields.get(&SearchField::AccountId),\n                document.fields.get(&SearchField::DocumentId),\n            ) {\n                *account_id << 32 | *doc_id\n            } else if let Some(SearchValue::Uint(id)) = document.fields.get(&SearchField::Id) {\n                *id\n            } else {\n                debug_assert!(false, \"Document is missing required ID fields\");\n                continue;\n            };\n\n            let _ = writeln!(\n                &mut request,\n                \"{{\\\"index\\\":{{\\\"_index\\\":\\\"{}\\\",\\\"_id\\\":{id}}}}}\",\n                document.index.index_name()\n            );\n            json_serialize(&mut request, &document);\n            request.push('\\n');\n        }\n\n        assert_success(\n            self.client\n                .post(format!(\"{}/_bulk\", self.url))\n                .body(request)\n                .send()\n                .await,\n        )\n        .await\n        .map(|_| ())\n    }\n\n    pub async fn query<R: SearchDocumentId>(\n        &self,\n        index: SearchIndex,\n        filters: &[SearchFilter],\n        sort: &[SearchComparator],\n    ) -> trc::Result<Vec<R>> {\n        let mut search_after: Option<Value> = None;\n        let mut results = Vec::new();\n        let mut has_more = true;\n\n        while has_more {\n            let query = Map::from_iter(\n                [\n                    Some((\"query\".to_string(), build_query(filters))),\n                    Some((\"size\".to_string(), Value::from(10_000))),\n                    Some((\"_source\".to_string(), Value::from(false))),\n                    Some((\n                        \"sort\".to_string(),\n                        build_sort(sort, R::field().field_name()),\n                    )),\n                    search_after\n                        .take()\n                        .map(|sa| (\"search_after\".to_string(), sa)),\n                ]\n                .into_iter()\n                .flatten(),\n            );\n\n            let response = assert_success(\n                self.client\n                    .post(format!(\"{}/{}/_search\", self.url, index.index_name()))\n                    .body(serde_json::to_string(&query).unwrap_or_default())\n                    .send()\n                    .await,\n            )\n            .await?;\n\n            let text = response\n                .text()\n                .await\n                .map_err(|err| trc::StoreEvent::ElasticsearchError.reason(err))?;\n\n            let response = serde_json::from_str::<SearchResponse>(&text).map_err(|err| {\n                trc::StoreEvent::ElasticsearchError\n                    .reason(err)\n                    .details(text)\n            })?;\n\n            has_more = response.hits.hits.len() == 10_000\n                && response.hits.hits.last().unwrap().sort.is_some();\n\n            for hit in response.hits.hits {\n                search_after = hit.sort;\n                results.push(R::from_u64(hit.id));\n            }\n        }\n\n        Ok(results)\n    }\n\n    pub async fn unindex(&self, filter: SearchQuery) -> trc::Result<u64> {\n        if filter.filters.is_empty() {\n            return Err(trc::StoreEvent::ElasticsearchError\n                .reason(\"Unindex operation requires at least one filter\"));\n        }\n\n        let query = json!({\n            \"query\": build_query(&filter.filters),\n        });\n\n        let response = assert_success(\n            self.client\n                .post(format!(\n                    \"{}/{}/_delete_by_query\",\n                    self.url,\n                    filter.index.index_name()\n                ))\n                .body(serde_json::to_string(&query).unwrap_or_default())\n                .send()\n                .await,\n        )\n        .await?;\n\n        let response_body = response\n            .text()\n            .await\n            .map_err(|err| trc::StoreEvent::ElasticsearchError.reason(err))?;\n\n        serde_json::from_str::<DeleteByQueryResponse>(&response_body)\n            .map(|delete_response| delete_response.deleted)\n            .map_err(|err| trc::StoreEvent::ElasticsearchError.reason(err))\n    }\n\n    pub async fn refresh_index(&self, index: SearchIndex) -> trc::Result<()> {\n        let url = format!(\"{}/{}/_refresh\", self.url, index.index_name());\n\n        assert_success(self.client.post(url).send().await)\n            .await\n            .map(|_| ())\n    }\n}\n\nfn build_query(filters: &[SearchFilter]) -> Value {\n    if filters.is_empty() {\n        return json!({ \"match_all\": {} });\n    }\n\n    let mut stack = Vec::new();\n    let mut conditions = Vec::new();\n    let mut logical_op = &SearchFilter::And;\n\n    for filter in filters {\n        match filter {\n            SearchFilter::Operator { field, op, value } => {\n                if field.is_text() && matches!(op, SearchOperator::Equal | SearchOperator::Contains)\n                {\n                    let SearchValue::Text { value, .. } = value else {\n                        debug_assert!(false, \"Invalid value type for text field\");\n                        continue;\n                    };\n\n                    if op != &SearchOperator::Equal {\n                        conditions.push(json!({\n                            \"match\": { field.field_name(): {\n                                \"query\": value,\n                                \"operator\": \"and\"\n                            } }\n                        }));\n                    } else {\n                        conditions.push(json!({\n                            \"match_phrase\": { field.field_name(): value }\n                        }));\n                    }\n                } else {\n                    let value = match value {\n                        SearchValue::Text { value, .. } => json!(value),\n                        SearchValue::Int(value) => json!(value),\n                        SearchValue::Uint(value) => json!(value),\n                        SearchValue::Boolean(value) => json!(value),\n                        SearchValue::KeyValues(kv) => {\n                            let (key, value) = kv.iter().next().unwrap();\n\n                            let cond = if !value.is_empty() {\n                                if op == &SearchOperator::Equal {\n                                    json!({\n                                        \"term\": {\n                                            format!(\"{}.{}.keyword\", field.field_name(), key): value\n                                        }\n                                    })\n                                } else {\n                                    json!({\n                                        \"match\": {\n                                            format!(\"{}.{}\", field.field_name(), key): value\n                                        }\n                                    })\n                                }\n                            } else {\n                                json!({\n                                    \"exists\": { \"field\": format!(\"{}.{}\", field.field_name(), key) }\n                                })\n                            };\n\n                            conditions.push(cond);\n                            continue;\n                        }\n                    };\n\n                    let cond = match op {\n                        SearchOperator::Equal | SearchOperator::Contains => json!({\n                            \"term\": { field.field_name(): value }\n                        }),\n                        op => {\n                            let op = match op {\n                                SearchOperator::LowerThan => \"lt\",\n                                SearchOperator::LowerEqualThan => \"lte\",\n                                SearchOperator::GreaterThan => \"gt\",\n                                SearchOperator::GreaterEqualThan => \"gte\",\n                                _ => unreachable!(),\n                            };\n\n                            json!({\n                                \"range\": { field.field_name(): { op: value } }\n                            })\n                        }\n                    };\n\n                    conditions.push(cond);\n                }\n            }\n\n            SearchFilter::And | SearchFilter::Or | SearchFilter::Not => {\n                stack.push((logical_op, conditions));\n                logical_op = filter;\n                conditions = Vec::new();\n            }\n            SearchFilter::End => {\n                if let Some((prev_logical_op, mut prev_conditions)) = stack.pop() {\n                    if !conditions.is_empty() {\n                        match logical_op {\n                            SearchFilter::And => {\n                                prev_conditions.push(json!({ \"bool\": { \"must\": conditions } }));\n                            }\n                            SearchFilter::Or => {\n                                prev_conditions.push(json!({ \"bool\": { \"should\": conditions } }));\n                            }\n                            SearchFilter::Not => {\n                                prev_conditions.push(json!({ \"bool\": { \"must_not\": conditions } }));\n                            }\n                            _ => unreachable!(),\n                        }\n                    }\n                    logical_op = prev_logical_op;\n                    conditions = prev_conditions;\n                }\n            }\n            SearchFilter::DocumentSet(_) => {\n                debug_assert!(\n                    false,\n                    \"DocumentSet filters are not supported in this backend\"\n                );\n                continue;\n            }\n        }\n    }\n\n    debug_assert!(\n        !conditions.is_empty(),\n        \"No conditions were built for the query\"\n    );\n\n    if conditions.len() == 1 {\n        conditions.pop().unwrap()\n    } else {\n        json!({ \"bool\": { \"must\": conditions } })\n    }\n}\n\nfn build_sort(sort: &[SearchComparator], tie_breaker: &str) -> Value {\n    Value::Array(\n        sort.iter()\n            .filter_map(|comp| match comp {\n                SearchComparator::Field { field, ascending } => {\n                    let field = if field.is_text() {\n                        format!(\"{}.keyword\", field.field_name())\n                    } else {\n                        field.field_name().to_string()\n                    };\n\n                    Some(json!({\n                        field: if *ascending { \"asc\" } else { \"desc\" }\n                    }))\n                }\n                _ => None,\n            })\n            .chain([json!({\n                tie_breaker: \"asc\"\n            })])\n            .collect(),\n    )\n}\n\nfn json_serialize(request: &mut String, document: &IndexDocument) {\n    request.push('{');\n    for (idx, (k, v)) in document.fields.iter().enumerate() {\n        if idx > 0 {\n            request.push(',');\n        }\n\n        let _ = write!(request, \"{:?}:\", k.field_name());\n        match v {\n            SearchValue::Text { value, .. } => {\n                json_serialize_str(request, value);\n            }\n            SearchValue::KeyValues(map) => {\n                request.push('{');\n                for (i, (key, value)) in map.iter().enumerate() {\n                    if i > 0 {\n                        request.push(',');\n                    }\n                    json_serialize_str(request, key);\n                    request.push(':');\n                    json_serialize_str(request, value);\n                }\n                request.push('}');\n            }\n            SearchValue::Int(v) => {\n                let _ = write!(request, \"{}\", v);\n            }\n            SearchValue::Uint(v) => {\n                let _ = write!(request, \"{}\", v);\n            }\n            SearchValue::Boolean(v) => {\n                let _ = write!(request, \"{}\", v);\n            }\n        }\n    }\n    request.push('}');\n}\n\nfn json_serialize_str(request: &mut String, value: &str) {\n    request.push('\"');\n    for c in value.chars() {\n        match c {\n            '\"' => request.push_str(\"\\\\\\\"\"),\n            '\\\\' => request.push_str(\"\\\\\\\\\"),\n            '\\n' => request.push_str(\"\\\\n\"),\n            '\\r' => request.push_str(\"\\\\r\"),\n            '\\t' => request.push_str(\"\\\\t\"),\n            '\\u{0008}' => request.push_str(\"\\\\b\"), // backspace\n            '\\u{000C}' => request.push_str(\"\\\\f\"), // form feed\n            _ => {\n                if !c.is_control() {\n                    request.push(c);\n                } else {\n                    let _ = write!(request, \"\\\\u{:04x}\", c as u32);\n                }\n            }\n        }\n    }\n    request.push('\"');\n}\n"
  },
  {
    "path": "crates/store/src/backend/foundationdb/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{FdbStore, MAX_VALUE_SIZE};\nuse crate::{\n    IterateParams, SUBSPACE_BLOBS,\n    backend::foundationdb::into_error,\n    write::{AnyKey, key::KeySerializer},\n};\nuse std::ops::Range;\nuse trc::AddContext;\nuse types::blob_hash::BLOB_HASH_LEN;\n\nimpl FdbStore {\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let block_start = range.start / MAX_VALUE_SIZE;\n        let bytes_start = range.start % MAX_VALUE_SIZE;\n        let block_end = (range.end / MAX_VALUE_SIZE) + 1;\n\n        let begin = KeySerializer::new(key.len() + 2)\n            .write(key)\n            .write(block_start as u16)\n            .finalize();\n        let end = KeySerializer::new(key.len() + 2)\n            .write(key)\n            .write(block_end as u16)\n            .finalize();\n        let key_len = begin.len();\n\n        let mut blob_data: Option<Vec<u8>> = None;\n        let blob_range = range.end - range.start;\n\n        self.iterate(\n            IterateParams::new(\n                AnyKey {\n                    subspace: SUBSPACE_BLOBS,\n                    key: begin,\n                },\n                AnyKey {\n                    subspace: SUBSPACE_BLOBS,\n                    key: end,\n                },\n            ),\n            |key, value| {\n                if key.len() == key_len {\n                    if let Some(blob_data) = &mut blob_data {\n                        blob_data.extend_from_slice(\n                            value\n                                .get(\n                                    ..std::cmp::min(\n                                        blob_range.saturating_sub(blob_data.len()),\n                                        value.len(),\n                                    ),\n                                )\n                                .unwrap_or(&[]),\n                        );\n                        if blob_data.len() == blob_range {\n                            return Ok(false);\n                        }\n                    } else {\n                        let blob_size = if blob_range <= (5 * (1 << 20)) {\n                            blob_range\n                        } else if value.len() == MAX_VALUE_SIZE {\n                            MAX_VALUE_SIZE * 2\n                        } else {\n                            value.len()\n                        };\n                        let mut blob_data_ = Vec::with_capacity(blob_size);\n                        blob_data_.extend_from_slice(\n                            value\n                                .get(\n                                    bytes_start\n                                        ..std::cmp::min(bytes_start + blob_range, value.len()),\n                                )\n                                .unwrap_or(&[]),\n                        );\n                        let is_done = blob_data_.len() == blob_range;\n                        blob_data = blob_data_.into();\n                        if is_done {\n                            return Ok(false);\n                        }\n                    }\n                }\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        Ok(blob_data)\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        const N_CHUNKS: usize = (1 << 5) - 1;\n        let last_chunk = std::cmp::max(\n            (data.len() / MAX_VALUE_SIZE)\n                + if !data.len().is_multiple_of(MAX_VALUE_SIZE) {\n                    1\n                } else {\n                    0\n                },\n            1,\n        ) - 1;\n        let mut trx = self.db.create_trx().map_err(into_error)?;\n\n        for (chunk_pos, chunk_bytes) in data.chunks(MAX_VALUE_SIZE).enumerate() {\n            trx.set(\n                &KeySerializer::new(key.len() + 3)\n                    .write(SUBSPACE_BLOBS)\n                    .write(key)\n                    .write(chunk_pos as u16)\n                    .finalize(),\n                chunk_bytes,\n            );\n            if chunk_pos == last_chunk || (chunk_pos > 0 && chunk_pos % N_CHUNKS == 0) {\n                self.commit(trx, false).await?;\n                if chunk_pos < last_chunk {\n                    trx = self.db.create_trx().map_err(into_error)?;\n                } else {\n                    break;\n                }\n            }\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        if key.len() < BLOB_HASH_LEN {\n            return Ok(false);\n        }\n\n        let trx = self.db.create_trx().map_err(into_error)?;\n        trx.clear_range(\n            &KeySerializer::new(key.len() + 3)\n                .write(SUBSPACE_BLOBS)\n                .write(key)\n                .write(0u16)\n                .finalize(),\n            &KeySerializer::new(key.len() + 3)\n                .write(SUBSPACE_BLOBS)\n                .write(key)\n                .write(u16::MAX)\n                .finalize(),\n        );\n\n        self.commit(trx, false).await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/foundationdb/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse foundationdb::{Database, api, options::DatabaseOption};\nuse utils::config::{Config, utils::AsKey};\n\nuse super::FdbStore;\n\nimpl FdbStore {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let guard = unsafe {\n            api::FdbApiBuilder::default()\n                .build()\n                .map_err(|err| {\n                    config.new_build_error(\n                        prefix.as_str(),\n                        format!(\"Failed to boot FoundationDB: {err:?}\"),\n                    )\n                })\n                .ok()?\n                .boot()\n                .map_err(|err| {\n                    config.new_build_error(\n                        prefix.as_str(),\n                        format!(\"Failed to boot FoundationDB: {err:?}\"),\n                    )\n                })\n                .ok()?\n        };\n\n        let db = Database::new(config.value((&prefix, \"cluster-file\")))\n            .map_err(|err| {\n                config.new_build_error(\n                    prefix.as_str(),\n                    format!(\"Failed to create FoundationDB database: {err:?}\"),\n                )\n            })\n            .ok()?;\n\n        if let Some(value) = config\n            .property::<Option<Duration>>((&prefix, \"transaction.timeout\"))\n            .unwrap_or_default()\n        {\n            db.set_option(DatabaseOption::TransactionTimeout(value.as_millis() as i32))\n                .map_err(|err| {\n                    config.new_build_error(\n                        (&prefix, \"transaction.timeout\"),\n                        format!(\"Failed to set option: {err:?}\"),\n                    )\n                })\n                .ok()?;\n        }\n        if let Some(value) = config.property((&prefix, \"transaction.retry-limit\")) {\n            db.set_option(DatabaseOption::TransactionRetryLimit(value))\n                .map_err(|err| {\n                    config.new_build_error(\n                        (&prefix, \"transaction.retry-limit\"),\n                        format!(\"Failed to set option: {err:?}\"),\n                    )\n                })\n                .ok()?;\n        }\n        if let Some(value) = config\n            .property::<Option<Duration>>((&prefix, \"transaction.max-retry-delay\"))\n            .unwrap_or_default()\n        {\n            db.set_option(DatabaseOption::TransactionMaxRetryDelay(\n                value.as_millis() as i32\n            ))\n            .map_err(|err| {\n                config.new_build_error(\n                    (&prefix, \"transaction.max-retry-delay\"),\n                    format!(\"Failed to set option: {err:?}\"),\n                )\n            })\n            .ok()?;\n        }\n        if let Some(value) = config.property((&prefix, \"ids.machine\")) {\n            db.set_option(DatabaseOption::MachineId(value))\n                .map_err(|err| {\n                    config.new_build_error(\n                        (&prefix, \"ids.machine\"),\n                        format!(\"Failed to set option: {err:?}\"),\n                    )\n                })\n                .ok()?;\n        }\n        if let Some(value) = config.property((&prefix, \"ids.datacenter\")) {\n            db.set_option(DatabaseOption::DatacenterId(value))\n                .map_err(|err| {\n                    config.new_build_error(\n                        (&prefix, \"ids.datacenter\"),\n                        format!(\"Failed to set option: {err:?}\"),\n                    )\n                })\n                .ok()?;\n        }\n\n        Some(Self {\n            guard,\n            db,\n            version: Default::default(),\n        })\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/foundationdb/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse foundationdb::{Database, FdbError, api::NetworkAutoStop};\nuse std::time::{Duration, Instant};\n\npub mod blob;\npub mod main;\npub mod read;\npub mod write;\n\nconst MAX_VALUE_SIZE: usize = 100000;\npub const TRANSACTION_EXPIRY: Duration = Duration::from_secs(1);\n\n#[allow(dead_code)]\npub struct FdbStore {\n    db: Database,\n    guard: NetworkAutoStop,\n    version: parking_lot::Mutex<ReadVersion>,\n}\n\npub(crate) struct ReadVersion {\n    version: i64,\n    expires: Instant,\n}\n\nimpl ReadVersion {\n    pub fn new(version: i64) -> Self {\n        Self {\n            version,\n            expires: Instant::now() + TRANSACTION_EXPIRY,\n        }\n    }\n\n    pub fn is_expired(&self) -> bool {\n        self.expires < Instant::now()\n    }\n}\n\nimpl Default for ReadVersion {\n    fn default() -> Self {\n        Self {\n            version: 0,\n            expires: Instant::now(),\n        }\n    }\n}\n\n#[inline(always)]\nfn into_error(error: FdbError) -> trc::Error {\n    trc::StoreEvent::FoundationdbError\n        .reason(error.message())\n        .ctx(trc::Key::Code, error.code())\n}\n"
  },
  {
    "path": "crates/store/src/backend/foundationdb/read.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{FdbStore, MAX_VALUE_SIZE, ReadVersion, into_error};\nuse crate::{\n    Deserialize, IterateParams, Key, ValueKey, WITH_SUBSPACE,\n    backend::deserialize_i64_le,\n    write::{ValueClass, key::KeySerializer},\n};\nuse foundationdb::{\n    KeySelector, RangeOption, Transaction,\n    future::FdbSlice,\n    options::{self},\n};\nuse futures::TryStreamExt;\n\n#[allow(dead_code)]\npub(crate) enum ChunkedValue {\n    Single(FdbSlice),\n    Chunked { n_chunks: u8, bytes: Vec<u8> },\n    None,\n}\n\nstruct ChunkedValueCollector {\n    key: Vec<u8>,\n    bytes: Vec<u8>,\n}\n\nimpl FdbStore {\n    pub(crate) async fn get_value<U>(&self, key: impl Key) -> trc::Result<Option<U>>\n    where\n        U: Deserialize,\n    {\n        let key = key.serialize(WITH_SUBSPACE);\n        let trx = self.read_trx().await?;\n\n        match read_chunked_value(&key, &trx, true).await? {\n            ChunkedValue::Single(bytes) => U::deserialize(&bytes).map(Some),\n            ChunkedValue::Chunked { bytes, .. } => U::deserialize_owned(bytes).map(Some),\n            ChunkedValue::None => Ok(None),\n        }\n    }\n\n    pub(crate) async fn iterate<T: Key>(\n        &self,\n        params: IterateParams<T>,\n        mut cb: impl for<'x> FnMut(&'x [u8], &'x [u8]) -> trc::Result<bool> + Sync + Send,\n    ) -> trc::Result<()> {\n        let begin = params.begin.serialize(WITH_SUBSPACE);\n        let end = params.end.serialize(WITH_SUBSPACE);\n\n        if !params.first {\n            let mut last_key = vec![];\n            let mut chunked_key: Option<ChunkedValueCollector> = None;\n\n            'outer: loop {\n                let begin_selector = if last_key.is_empty() {\n                    KeySelector::first_greater_or_equal(&begin)\n                } else {\n                    KeySelector::first_greater_than(&last_key)\n                };\n\n                let trx = self.read_trx().await?;\n                let mut values = trx.get_ranges(\n                    RangeOption {\n                        begin: begin_selector,\n                        end: KeySelector::first_greater_than(&end),\n                        mode: options::StreamingMode::WantAll,\n                        reverse: !params.ascending,\n                        ..Default::default()\n                    },\n                    true,\n                );\n\n                let mut last_key_ = vec![];\n                loop {\n                    match values.try_next().await {\n                        Ok(Some(values)) => {\n                            let mut key = &[] as &[u8];\n                            for value in values.iter() {\n                                key = value.key();\n\n                                // Check whether we are collecting a chunked value\n                                let cb_key = key.get(1..).unwrap_or_default();\n                                let cb_value = value.value();\n\n                                if let Some(chunk) = &mut chunked_key {\n                                    if chunk.key.len() + 1 == cb_key.len()\n                                        && cb_key[..chunk.key.len()] == chunk.key[..]\n                                    {\n                                        // This is a chunk of the current value\n                                        chunk.bytes.extend_from_slice(cb_value);\n                                        continue;\n                                    } else {\n                                        // Return collected chunked value\n                                        if !cb(&chunk.key, &chunk.bytes)? {\n                                            return Ok(());\n                                        }\n\n                                        // Reset collector\n                                        chunked_key = None;\n                                    }\n                                }\n\n                                if cb_value.len() < MAX_VALUE_SIZE {\n                                    if !cb(cb_key, cb_value)? {\n                                        return Ok(());\n                                    }\n                                } else {\n                                    // Start collecting chunked value\n                                    chunked_key = Some(ChunkedValueCollector {\n                                        key: cb_key.to_vec(),\n                                        bytes: cb_value.to_vec(),\n                                    });\n                                }\n                            }\n                            if values.more() {\n                                last_key_ = key.to_vec();\n                            }\n                        }\n                        Ok(None) => {\n                            // Return any chunked value collected\n                            if let Some(chunked_key) = chunked_key.take() {\n                                cb(&chunked_key.key, &chunked_key.bytes)?;\n                            }\n\n                            break 'outer;\n                        }\n                        Err(e) => {\n                            if e.code() == 1007 && !last_key_.is_empty() {\n                                // Transaction is too old to perform reads or be committed\n                                drop(values);\n                                last_key = last_key_;\n                                continue 'outer;\n                            } else {\n                                return Err(into_error(e));\n                            }\n                        }\n                    }\n                }\n            }\n        } else {\n            let trx = self.read_trx().await?;\n            let mut values = trx.get_ranges_keyvalues(\n                RangeOption {\n                    begin: KeySelector::first_greater_or_equal(&begin),\n                    end: KeySelector::first_greater_than(&end),\n                    mode: options::StreamingMode::Small,\n                    reverse: !params.ascending,\n                    ..Default::default()\n                },\n                true,\n            );\n\n            if let Some(value) = values.try_next().await.map_err(into_error)? {\n                cb(value.key().get(1..).unwrap_or_default(), value.value())?;\n            }\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn get_counter(\n        &self,\n        key: impl Into<ValueKey<ValueClass>> + Sync + Send,\n    ) -> trc::Result<i64> {\n        let key = key.into().serialize(WITH_SUBSPACE);\n        if let Some(bytes) = self\n            .read_trx()\n            .await?\n            .get(&key, true)\n            .await\n            .map_err(into_error)?\n        {\n            deserialize_i64_le(&key, &bytes)\n        } else {\n            Ok(0)\n        }\n    }\n\n    pub(crate) async fn read_trx(&self) -> trc::Result<Transaction> {\n        let (is_expired, mut read_version) = {\n            let version = self.version.lock();\n            (version.is_expired(), version.version)\n        };\n        let trx = self.db.create_trx().map_err(into_error)?;\n\n        if is_expired {\n            read_version = trx.get_read_version().await.map_err(into_error)?;\n            *self.version.lock() = ReadVersion::new(read_version);\n        } else {\n            trx.set_read_version(read_version);\n        }\n\n        Ok(trx)\n    }\n}\n\npub(crate) async fn read_chunked_value(\n    key: &[u8],\n    trx: &Transaction,\n    snapshot: bool,\n) -> trc::Result<ChunkedValue> {\n    if let Some(bytes) = trx.get(key, snapshot).await.map_err(into_error)? {\n        if bytes.len() < MAX_VALUE_SIZE {\n            Ok(ChunkedValue::Single(bytes))\n        } else {\n            let mut value = Vec::with_capacity(bytes.len() * 2);\n            value.extend_from_slice(&bytes);\n            let mut key = KeySerializer::new(key.len() + 1)\n                .write(key)\n                .write(0u8)\n                .finalize();\n\n            while let Some(bytes) = trx.get(&key, snapshot).await.map_err(into_error)? {\n                value.extend_from_slice(&bytes);\n                *key.last_mut().unwrap() += 1;\n            }\n\n            Ok(ChunkedValue::Chunked {\n                bytes: value,\n                n_chunks: *key.last().unwrap(),\n            })\n        }\n    } else {\n        Ok(ChunkedValue::None)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/foundationdb/write.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    FdbStore, MAX_VALUE_SIZE, ReadVersion, into_error,\n    read::{ChunkedValue, read_chunked_value},\n};\nuse crate::{\n    backend::deserialize_i64_le,\n    write::{\n        AssignedIds, Batch, DirectoryClass, MAX_COMMIT_ATTEMPTS, MAX_COMMIT_TIME, MergeResult,\n        Operation, TaskQueueClass, TelemetryClass, ValueClass, ValueOp, key::KeySerializer,\n    },\n    *,\n};\nuse foundationdb::{\n    FdbError, KeySelector, RangeOption, Transaction,\n    options::{self, MutationType},\n};\nuse futures::TryStreamExt;\nuse rand::Rng;\nuse std::{\n    cmp::Ordering,\n    time::{Duration, Instant},\n};\nuse trc::AddContext;\n\nimpl FdbStore {\n    pub(crate) async fn write(&self, batch: Batch<'_>) -> trc::Result<AssignedIds> {\n        let start = Instant::now();\n        let mut retry_count = 0;\n        let has_changes = !batch.changes.is_empty();\n\n        loop {\n            let mut account_id = u32::MAX;\n            let mut collection = u8::MAX;\n            let mut document_id = u32::MAX;\n            let mut change_id = 0u64;\n            let mut result = AssignedIds::default();\n\n            let trx = self.db.create_trx().map_err(into_error)?;\n\n            if has_changes {\n                for &account_id in batch.changes.keys() {\n                    debug_assert!(account_id != u32::MAX);\n                    let key = ValueClass::ChangeId.serialize(account_id, 0, 0, WITH_SUBSPACE);\n                    let change_id =\n                        if let Some(bytes) = trx.get(&key, false).await.map_err(into_error)? {\n                            deserialize_i64_le(&key, &bytes)? + 1\n                        } else {\n                            1\n                        };\n                    trx.set(&key, &change_id.to_le_bytes()[..]);\n                    result.push_change_id(account_id, change_id as u64);\n                }\n            }\n\n            for op in batch.ops.iter_mut() {\n                match op {\n                    Operation::AccountId {\n                        account_id: account_id_,\n                    } => {\n                        account_id = *account_id_;\n                        if has_changes {\n                            change_id = result.set_current_change_id(account_id)?;\n                        }\n                    }\n                    Operation::Collection {\n                        collection: collection_,\n                    } => {\n                        collection = u8::from(*collection_);\n                    }\n                    Operation::DocumentId {\n                        document_id: document_id_,\n                    } => {\n                        document_id = *document_id_;\n                    }\n                    Operation::Value { class, op } => {\n                        let mut key =\n                            class.serialize(account_id, collection, document_id, WITH_SUBSPACE);\n\n                        match op {\n                            ValueOp::Set(value) => {\n                                if !chunk_value(&trx, &mut key, value) {\n                                    trx.cancel();\n                                    return Err(trc::StoreEvent::FoundationdbError\n                                        .ctx(trc::Key::Reason, \"Value is too large\"));\n                                }\n                            }\n                            ValueOp::SetFnc(set_op) => {\n                                let value = (set_op.fnc)(&set_op.params, &result)?;\n                                if !chunk_value(&trx, &mut key, &value) {\n                                    trx.cancel();\n                                    return Err(trc::StoreEvent::FoundationdbError\n                                        .ctx(trc::Key::Reason, \"Value is too large\"));\n                                }\n                            }\n                            ValueOp::MergeFnc(merge_op) => {\n                                let (merge_result, is_chunked) =\n                                    match read_chunked_value(&key, &trx, false)\n                                        .await\n                                        .caused_by(trc::location!())?\n                                    {\n                                        ChunkedValue::Single(slice) => (\n                                            (merge_op.fnc)(\n                                                &merge_op.params,\n                                                &result,\n                                                Some(slice.as_ref()),\n                                            )?,\n                                            false,\n                                        ),\n                                        ChunkedValue::Chunked { bytes, .. } => (\n                                            (merge_op.fnc)(\n                                                &merge_op.params,\n                                                &result,\n                                                Some(bytes.as_ref()),\n                                            )?,\n                                            true,\n                                        ),\n                                        ChunkedValue::None => (\n                                            (merge_op.fnc)(&merge_op.params, &result, None)?,\n                                            false,\n                                        ),\n                                    };\n\n                                match merge_result {\n                                    MergeResult::Update(value) => {\n                                        if !chunk_value(&trx, &mut key, &value) {\n                                            trx.cancel();\n                                            return Err(trc::StoreEvent::FoundationdbError\n                                                .ctx(trc::Key::Reason, \"Value is too large\"));\n                                        }\n                                    }\n                                    MergeResult::Delete => {\n                                        if is_chunked {\n                                            trx.clear_range(\n                                                &key,\n                                                &KeySerializer::new(key.len() + 1)\n                                                    .write(key.as_slice())\n                                                    .write(u8::MAX)\n                                                    .finalize(),\n                                            );\n                                        } else {\n                                            trx.clear(&key);\n                                        }\n                                    }\n                                    MergeResult::Skip => (),\n                                }\n                            }\n\n                            ValueOp::AtomicAdd(by) => {\n                                trx.atomic_op(&key, &by.to_le_bytes()[..], MutationType::Add);\n                            }\n                            ValueOp::AddAndGet(by) => {\n                                let num = if let Some(bytes) =\n                                    trx.get(&key, false).await.map_err(into_error)?\n                                {\n                                    deserialize_i64_le(&key, &bytes)? + *by\n                                } else {\n                                    *by\n                                };\n                                trx.set(&key, &num.to_le_bytes()[..]);\n                                result.push_counter_id(num);\n                            }\n                            ValueOp::Clear => {\n                                if matches!(\n                                    key[0],\n                                    SUBSPACE_DIRECTORY\n                                        | SUBSPACE_TASK_QUEUE\n                                        | SUBSPACE_IN_MEMORY_VALUE\n                                        | SUBSPACE_PROPERTY\n                                        | SUBSPACE_QUEUE_MESSAGE\n                                        | SUBSPACE_REPORT_OUT\n                                        | SUBSPACE_REPORT_IN\n                                        | SUBSPACE_TELEMETRY_SPAN\n                                        | SUBSPACE_SEARCH_INDEX\n                                        | SUBSPACE_LOGS\n                                ) && matches!(\n                                    class,\n                                    ValueClass::Property(_)\n                                        | ValueClass::Queue(_)\n                                        | ValueClass::Report(_)\n                                        | ValueClass::Directory(DirectoryClass::Principal(_))\n                                        | ValueClass::ShareNotification { .. }\n                                        | ValueClass::Telemetry(TelemetryClass::Metric { .. })\n                                        | ValueClass::TaskQueue(TaskQueueClass::SendImip {\n                                            is_payload: true,\n                                            ..\n                                        })\n                                        | ValueClass::InMemory(_)\n                                ) {\n                                    trx.clear_range(\n                                        &key,\n                                        &KeySerializer::new(key.len() + 1)\n                                            .write(key.as_slice())\n                                            .write(u8::MAX)\n                                            .finalize(),\n                                    );\n                                } else {\n                                    trx.clear(&key);\n                                }\n                            }\n                        }\n                    }\n                    Operation::Index { field, key, set } => {\n                        let key = IndexKey {\n                            account_id,\n                            collection,\n                            document_id,\n                            field: *field,\n                            key: &*key,\n                        }\n                        .serialize(WITH_SUBSPACE);\n\n                        if *set {\n                            trx.set(&key, &[]);\n                        } else {\n                            trx.clear(&key);\n                        }\n                    }\n                    Operation::Log { collection, set } => {\n                        let key = LogKey {\n                            account_id,\n                            collection: u8::from(*collection),\n                            change_id,\n                        }\n                        .serialize(WITH_SUBSPACE);\n\n                        trx.set(&key, set);\n                    }\n                    Operation::AssertValue {\n                        class,\n                        assert_value,\n                    } => {\n                        let key =\n                            class.serialize(account_id, collection, document_id, WITH_SUBSPACE);\n\n                        let matches = match read_chunked_value(&key, &trx, false).await {\n                            Ok(ChunkedValue::Single(bytes)) => assert_value.matches(bytes.as_ref()),\n                            Ok(ChunkedValue::Chunked { bytes, .. }) => {\n                                assert_value.matches(bytes.as_ref())\n                            }\n                            Ok(ChunkedValue::None) => assert_value.is_none(),\n                            Err(_) => false,\n                        };\n\n                        if !matches {\n                            trx.cancel();\n                            return Err(trc::StoreEvent::AssertValueFailed.into());\n                        }\n                    }\n                }\n            }\n\n            if self\n                .commit(\n                    trx,\n                    retry_count < MAX_COMMIT_ATTEMPTS && start.elapsed() < MAX_COMMIT_TIME,\n                )\n                .await?\n            {\n                return Ok(result);\n            } else {\n                let backoff = rand::rng().random_range(50..=100);\n                tokio::time::sleep(Duration::from_millis(backoff)).await;\n                retry_count += 1;\n            }\n        }\n    }\n\n    pub(crate) async fn commit(&self, trx: Transaction, will_retry: bool) -> trc::Result<bool> {\n        match trx.commit().await {\n            Ok(result) => {\n                let commit_version = result.committed_version().map_err(into_error)?;\n                let mut version = self.version.lock();\n                if commit_version > version.version {\n                    *version = ReadVersion::new(commit_version);\n                }\n                Ok(true)\n            }\n            Err(err) => {\n                if will_retry {\n                    err.on_error().await.map_err(into_error)?;\n                    Ok(false)\n                } else {\n                    Err(into_error(FdbError::from(err)))\n                }\n            }\n        }\n    }\n\n    pub(crate) async fn purge_store(&self) -> trc::Result<()> {\n        // Obtain all zero counters\n        let mut delete_keys = Vec::new();\n        for subspace in [SUBSPACE_COUNTER, SUBSPACE_QUOTA, SUBSPACE_IN_MEMORY_COUNTER] {\n            let trx = self.db.create_trx().map_err(into_error)?;\n            let from_key = [subspace, 0u8];\n            let to_key = [subspace, u8::MAX, u8::MAX, u8::MAX, u8::MAX, u8::MAX];\n\n            let mut values = trx.get_ranges_keyvalues(\n                RangeOption {\n                    begin: KeySelector::first_greater_or_equal(&from_key[..]),\n                    end: KeySelector::first_greater_or_equal(&to_key[..]),\n                    mode: options::StreamingMode::WantAll,\n                    reverse: false,\n                    ..Default::default()\n                },\n                true,\n            );\n\n            while let Some(value) = values.try_next().await.map_err(into_error)? {\n                if value.value().iter().all(|byte| *byte == 0) {\n                    delete_keys.push(value.key().to_vec());\n                }\n            }\n        }\n\n        if delete_keys.is_empty() {\n            return Ok(());\n        }\n\n        // Delete keys\n        let integer = 0i64.to_le_bytes();\n        for chunk in delete_keys.chunks(1024) {\n            let mut retry_count = 0;\n            loop {\n                let trx = self.db.create_trx().map_err(into_error)?;\n                for key in chunk {\n                    trx.atomic_op(key, &integer, MutationType::CompareAndClear);\n                }\n\n                if self.commit(trx, retry_count < MAX_COMMIT_ATTEMPTS).await? {\n                    break;\n                } else {\n                    retry_count += 1;\n                }\n            }\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn delete_range(&self, from: impl Key, to: impl Key) -> trc::Result<()> {\n        let from = from.serialize(WITH_SUBSPACE);\n        let to = to.serialize(WITH_SUBSPACE);\n\n        let trx = self.db.create_trx().map_err(into_error)?;\n        trx.clear_range(&from, &to);\n        self.commit(trx, false).await.map(|_| ())\n    }\n}\n\nfn chunk_value(trx: &Transaction, key: &mut Vec<u8>, value: &[u8]) -> bool {\n    if !value.is_empty() && value.len() > MAX_VALUE_SIZE {\n        for (pos, chunk) in value.chunks(MAX_VALUE_SIZE).enumerate() {\n            match pos.cmp(&1) {\n                Ordering::Less => {}\n                Ordering::Equal => {\n                    key.push(0);\n                }\n                Ordering::Greater => {\n                    if pos < u8::MAX as usize {\n                        *key.last_mut().unwrap() += 1;\n                    } else {\n                        return false;\n                    }\n                }\n            }\n            trx.set(key, chunk);\n        }\n    } else {\n        trx.set(key, value.as_ref());\n    }\n\n    true\n}\n"
  },
  {
    "path": "crates/store/src/backend/fs/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{io::SeekFrom, ops::Range, path::PathBuf};\n\nuse tokio::{\n    fs::{self, File},\n    io::{AsyncReadExt, AsyncSeekExt, AsyncWriteExt},\n};\nuse utils::{\n    codec::base32_custom::Base32Writer,\n    config::{Config, utils::AsKey},\n};\n\npub struct FsStore {\n    path: PathBuf,\n    hash_levels: usize,\n}\n\nimpl FsStore {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let path = PathBuf::from(config.value_require((&prefix, \"path\"))?);\n        if !path.exists() {\n            fs::create_dir_all(&path)\n                .await\n                .map_err(|e| {\n                    config.new_build_error(\n                        (&prefix, \"path\"),\n                        format!(\"Failed to create directory: {e}\"),\n                    )\n                })\n                .ok()?;\n        }\n\n        Some(FsStore {\n            path,\n            hash_levels: std::cmp::min(\n                config\n                    .property_or_default((&prefix, \"depth\"), \"2\")\n                    .unwrap_or(2),\n                5,\n            ),\n        })\n    }\n\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let blob_path = self.build_path(key);\n        let blob_size = match fs::metadata(&blob_path).await {\n            Ok(m) => m.len() as usize,\n            Err(_) => return Ok(None),\n        };\n        let mut blob = File::open(&blob_path).await.map_err(into_error)?;\n\n        Ok(Some(if range.start != 0 || range.end != usize::MAX {\n            let from_offset = if range.start < blob_size {\n                range.start\n            } else {\n                0\n            };\n            let mut buf = vec![0; (std::cmp::min(range.end, blob_size) - from_offset) as usize];\n\n            if from_offset > 0 {\n                blob.seek(SeekFrom::Start(from_offset as u64))\n                    .await\n                    .map_err(into_error)?;\n            }\n            blob.read_exact(&mut buf).await.map_err(into_error)?;\n            buf\n        } else {\n            let mut buf = Vec::with_capacity(blob_size as usize);\n            blob.read_to_end(&mut buf).await.map_err(into_error)?;\n            buf\n        }))\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let blob_path = self.build_path(key);\n\n        if fs::metadata(&blob_path)\n            .await\n            .map_or(true, |m| m.len() as usize != data.len())\n        {\n            fs::create_dir_all(blob_path.parent().unwrap())\n                .await\n                .map_err(into_error)?;\n            let mut blob_file = File::create(&blob_path).await.map_err(into_error)?;\n            blob_file.write_all(data).await.map_err(into_error)?;\n            blob_file.flush().await.map_err(into_error)?;\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let blob_path = self.build_path(key);\n        if fs::metadata(&blob_path).await.is_ok() {\n            fs::remove_file(&blob_path).await.map_err(into_error)?;\n            Ok(true)\n        } else {\n            Ok(false)\n        }\n    }\n\n    fn build_path(&self, key: &[u8]) -> PathBuf {\n        let mut path = self.path.clone();\n\n        for byte in key.iter().take(self.hash_levels) {\n            path.push(format!(\"{:x}\", byte));\n        }\n        path.push(Base32Writer::from_bytes(key).finalize());\n        path\n    }\n}\n\nfn into_error(err: std::io::Error) -> trc::Error {\n    trc::StoreEvent::FilesystemError.reason(err)\n}\n"
  },
  {
    "path": "crates/store/src/backend/http/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    collections::hash_map::Entry,\n    sync::atomic::{AtomicBool, AtomicU64},\n    time::Duration,\n};\n\nuse ahash::AHashMap;\nuse arc_swap::ArcSwap;\nuse utils::config::Config;\n\nuse crate::{InMemoryStore, Stores};\n\nuse super::{HttpStore, HttpStoreConfig, HttpStoreFormat};\n\nimpl Stores {\n    pub fn parse_http_stores(&mut self, config: &mut Config, is_reload: bool) {\n        // Parse remote lists\n        for id in config.sub_keys(\"http-lookup\", \".url\") {\n            let id_ = id.as_str();\n            if !config\n                .property_or_default((\"http-lookup\", id_, \"enable\"), \"true\")\n                .unwrap_or(true)\n            {\n                continue;\n            }\n\n            let format = match config\n                .value_require((\"http-lookup\", id_, \"format\"))\n                .unwrap_or_default()\n            {\n                \"list\" => HttpStoreFormat::List,\n                \"csv\" => HttpStoreFormat::Csv {\n                    index_key: config\n                        .property_require((\"http-lookup\", id_, \"index.key\"))\n                        .unwrap_or(0),\n                    index_value: config.property((\"http-lookup\", id_, \"index.value\")),\n                    separator: config\n                        .property_or_default::<String>((\"http-lookup\", id_, \"separator\"), \",\")\n                        .unwrap_or_default()\n                        .chars()\n                        .next()\n                        .unwrap_or(','),\n                    skip_first: config\n                        .property_or_default::<bool>((\"http-lookup\", id_, \"skip-first\"), \"false\")\n                        .unwrap_or(false),\n                },\n                other => {\n                    let message = format!(\"Invalid format: {other:?}\");\n                    config.new_build_error((\"http-lookup\", id_, \"format\"), message);\n                    continue;\n                }\n            };\n\n            let http_config = HttpStoreConfig {\n                url: config\n                    .value_require((\"http-lookup\", id_, \"url\"))\n                    .unwrap_or_default()\n                    .to_string(),\n                retry: config\n                    .property_or_default::<Duration>((\"http-lookup\", id_, \"retry\"), \"1h\")\n                    .unwrap_or(Duration::from_secs(3600))\n                    .as_secs(),\n                refresh: config\n                    .property_or_default::<Duration>((\"http-lookup\", id_, \"refresh\"), \"12h\")\n                    .unwrap_or(Duration::from_secs(43200))\n                    .as_secs(),\n                timeout: config\n                    .property_or_default::<Duration>((\"http-lookup\", id_, \"timeout\"), \"30s\")\n                    .unwrap_or(Duration::from_secs(30)),\n                gzipped: config\n                    .property_or_default::<bool>((\"http-lookup\", id_, \"gzipped\"), \"false\")\n                    .unwrap_or_default(),\n                max_size: config\n                    .property_or_default::<usize>((\"http-lookup\", id_, \"limits.size\"), \"104857600\")\n                    .unwrap_or(104857600),\n                max_entries: config\n                    .property_or_default::<usize>((\"http-lookup\", id_, \"limits.entries\"), \"100000\")\n                    .unwrap_or(100000),\n                max_entry_size: config\n                    .property_or_default::<usize>((\"http-lookup\", id_, \"limits.entry-size\"), \"512\")\n                    .unwrap_or(512),\n                format,\n                id,\n            };\n\n            match self.in_memory_stores.entry(http_config.id.clone()) {\n                Entry::Vacant(entry) => {\n                    let store = HttpStore {\n                        entries: ArcSwap::from_pointee(AHashMap::new()),\n                        expires: AtomicU64::new(0),\n                        in_flight: AtomicBool::new(false),\n                        config: http_config,\n                    };\n\n                    entry.insert(InMemoryStore::Http(store.into()));\n                }\n                Entry::Occupied(e) if !is_reload => {\n                    config.new_build_error(\n                        (\"http-lookup\", e.key().as_str()),\n                        \"An in-memory store with this id already exists\",\n                    );\n                }\n                _ => {}\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/http/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    io::{BufRead, BufReader},\n    sync::{Arc, atomic::Ordering},\n    time::Instant,\n};\n\nuse ahash::AHashMap;\nuse compact_str::ToCompactString;\nuse rand::seq::IndexedRandom;\nuse utils::HttpLimitResponse;\n\nuse crate::{Value, backend::http::HttpStoreFormat, write::now};\n\nuse super::HttpStore;\n\nconst BROWSER_USER_AGENTS: [&str; 5] = [\n    \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36\",\n    \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/120.0.0.0 Safari/537.36\",\n    \"Mozilla/5.0 (Macintosh; Intel Mac OS X 14_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.1 Safari/605.1.15\",\n    \"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0\",\n    \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36\",\n];\n\npub(crate) trait HttpStoreGet {\n    fn get(&self, key: &str) -> Option<Value<'static>>;\n    fn contains(&self, key: &str) -> bool;\n    fn refresh(&self);\n}\n\nimpl HttpStoreGet for Arc<HttpStore> {\n    fn get(&self, key: &str) -> Option<Value<'static>> {\n        self.refresh();\n        self.entries.load().get(key).cloned()\n    }\n\n    fn contains(&self, key: &str) -> bool {\n        #[cfg(feature = \"test_mode\")]\n        {\n            if self.config.url.contains(\"phishtank.com\")\n                || self.config.url.contains(\"openphish.com\")\n            {\n                return (self.config.url.contains(\"open\") && key.contains(\"open\"))\n                    || (self.config.url.contains(\"tank\") && key.contains(\"tank\"));\n            } else if self.config.url.contains(\"disposable.github.io\") {\n                return key.ends_with(\"guerrillamail.com\") || key.ends_with(\"disposable.org\");\n            } else if self.config.url.contains(\"free_email_provider_domains.txt\") {\n                return key.ends_with(\"gmail.com\")\n                    || key.ends_with(\"googlemail.com\")\n                    || key.ends_with(\"yahoomail.com\")\n                    || key.ends_with(\"outlook.com\")\n                    || key.ends_with(\"freemail.org\");\n            }\n        }\n\n        self.refresh();\n        self.entries.load().contains_key(key)\n    }\n\n    fn refresh(&self) {\n        if self.expires.load(Ordering::Relaxed) <= now() {\n            let in_flight = self.in_flight.swap(true, Ordering::Relaxed);\n            if !in_flight {\n                let this = self.clone();\n                tokio::spawn(async move {\n                    let expires = match this.try_refresh().await {\n                        Ok(list) => {\n                            this.entries.store(list.into());\n                            this.config.refresh\n                        }\n                        Err(err) => {\n                            trc::error!(err);\n                            this.config.retry\n                        }\n                    };\n\n                    this.expires.store(now() + expires, Ordering::Relaxed);\n                    this.in_flight.store(false, Ordering::Relaxed);\n                });\n            }\n        }\n    }\n}\n\nimpl HttpStore {\n    async fn try_refresh(&self) -> trc::Result<AHashMap<String, Value<'static>>> {\n        let time = Instant::now();\n        let agent = BROWSER_USER_AGENTS.choose(&mut rand::rng()).unwrap();\n        let response = reqwest::Client::builder()\n            .timeout(self.config.timeout)\n            .user_agent(*agent)\n            .build()\n            .unwrap_or_default()\n            .get(&self.config.url)\n            .send()\n            .await\n            .map_err(|err| {\n                trc::StoreEvent::HttpStoreError\n                    .into_err()\n                    .reason(err)\n                    .ctx(trc::Key::Url, self.config.url.to_compact_string())\n                    .details(\"Failed to build request\")\n            })?;\n\n        if !response.status().is_success() {\n            trc::bail!(\n                trc::StoreEvent::HttpStoreError\n                    .into_err()\n                    .ctx(trc::Key::Code, response.status().as_u16())\n                    .ctx(trc::Key::Url, self.config.url.to_compact_string())\n                    .ctx(trc::Key::Elapsed, time.elapsed())\n                    .details(\"Failed to fetch HTTP list\")\n            );\n        }\n\n        let bytes = response\n            .bytes_with_limit(self.config.max_size)\n            .await\n            .map_err(|err| {\n                trc::StoreEvent::HttpStoreError\n                    .into_err()\n                    .reason(err)\n                    .ctx(trc::Key::Url, self.config.url.to_compact_string())\n                    .ctx(trc::Key::Elapsed, time.elapsed())\n                    .details(\"Failed to fetch resource\")\n            })?\n            .ok_or_else(|| {\n                trc::StoreEvent::HttpStoreError\n                    .into_err()\n                    .ctx(trc::Key::Url, self.config.url.to_compact_string())\n                    .ctx(trc::Key::Elapsed, time.elapsed())\n                    .details(\"Resource is too large\")\n            })?;\n\n        let reader: Box<dyn std::io::Read + Sync + Send> = if self.config.gzipped {\n            Box::new(flate2::read::GzDecoder::new(&bytes[..]))\n        } else {\n            Box::new(&bytes[..])\n        };\n\n        let mut entries = AHashMap::new();\n        for (pos, line) in BufReader::new(reader).lines().enumerate() {\n            let line_ = line.map_err(|err| {\n                trc::StoreEvent::HttpStoreError\n                    .into_err()\n                    .reason(err)\n                    .ctx(trc::Key::Url, self.config.url.to_compact_string())\n                    .ctx(trc::Key::Elapsed, time.elapsed())\n                    .details(\"Failed to read line\")\n            })?;\n\n            match &self.config.format {\n                HttpStoreFormat::List => {\n                    let line = line_.trim();\n                    if !line.is_empty() {\n                        entries.insert(line.to_string(), Value::Integer(1));\n                    }\n                }\n                HttpStoreFormat::Csv {\n                    index_key,\n                    index_value,\n                    separator,\n                    skip_first,\n                } if pos > 0 || !*skip_first => {\n                    let mut in_quote = false;\n                    let mut col_num = 0;\n                    let mut last_ch = ' ';\n\n                    let mut entry_key: String = String::new();\n                    let mut entry_value: String = String::new();\n\n                    for ch in line_.chars() {\n                        match ch {\n                            '\"' if last_ch != '\\\\' => {\n                                in_quote = !in_quote;\n                            }\n                            '\\\\' if last_ch != '\\\\' => (),\n                            _ => {\n                                if ch == *separator && !in_quote {\n                                    if col_num == *index_key && index_value.is_none() {\n                                        break;\n                                    } else {\n                                        col_num += 1;\n                                    }\n                                } else if col_num == *index_key {\n                                    entry_key.push(ch);\n                                    if entry_key.len() > self.config.max_entry_size {\n                                        break;\n                                    }\n                                } else if index_value.is_some_and(|v| col_num == v) {\n                                    entry_value.push(ch);\n                                    if entry_value.len() > self.config.max_entry_size {\n                                        break;\n                                    }\n                                }\n                            }\n                        }\n\n                        last_ch = ch;\n                    }\n\n                    if !entry_key.is_empty() {\n                        let entry_value = if !entry_value.is_empty() {\n                            Value::Text(entry_value.into())\n                        } else {\n                            Value::Integer(1)\n                        };\n                        entries.insert(entry_key, entry_value);\n                    }\n                }\n                _ => (),\n            }\n\n            if entries.len() == self.config.max_entries {\n                break;\n            }\n        }\n\n        trc::event!(\n            Store(trc::StoreEvent::HttpStoreFetch),\n            Url = self.config.url.to_compact_string(),\n            Total = entries.len(),\n            Elapsed = time.elapsed(),\n        );\n\n        Ok(entries)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/http/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod config;\npub mod lookup;\n\nuse std::{\n    sync::atomic::{AtomicBool, AtomicU64},\n    time::Duration,\n};\n\nuse ahash::AHashMap;\nuse arc_swap::ArcSwap;\n\nuse crate::Value;\n\n#[derive(Debug, Clone)]\npub struct HttpStoreConfig {\n    pub id: String,\n    pub url: String,\n    pub retry: u64,\n    pub refresh: u64,\n    pub timeout: Duration,\n    pub gzipped: bool,\n    pub max_size: usize,\n    pub max_entries: usize,\n    pub max_entry_size: usize,\n    pub format: HttpStoreFormat,\n}\n\n#[derive(Debug, Clone)]\npub enum HttpStoreFormat {\n    List,\n    Csv {\n        index_key: u32,\n        index_value: Option<u32>,\n        separator: char,\n        skip_first: bool,\n    },\n}\n\n#[derive(Debug)]\npub struct HttpStore {\n    pub entries: ArcSwap<AHashMap<String, Value<'static>>>,\n    pub expires: AtomicU64,\n    pub in_flight: AtomicBool,\n    pub config: HttpStoreConfig,\n}\n"
  },
  {
    "path": "crates/store/src/backend/kafka/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse rdkafka::{\n    ClientConfig, ClientContext, TopicPartitionList,\n    consumer::{BaseConsumer, ConsumerContext, Rebalance, StreamConsumer},\n    error::KafkaResult,\n    producer::FutureProducer,\n};\nuse std::{fmt::Debug, time::Duration};\nuse utils::config::{Config, utils::AsKey};\n\npub mod pubsub;\n\npub(super) type LoggingConsumer = StreamConsumer<CustomContext>;\n\npub struct KafkaPubSub {\n    consumer_builder: ClientConfig,\n    producer: FutureProducer,\n}\n\nimpl KafkaPubSub {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let brokers = config\n            .values((&prefix, \"brokers\"))\n            .map(|(_, v)| v.to_string())\n            .collect::<Vec<_>>();\n        if brokers.is_empty() {\n            config.new_build_error((&prefix, \"brokers\"), \"No Kafka brokers specified\");\n            return None;\n        }\n\n        let mut consumer_builder = ClientConfig::new();\n\n        consumer_builder\n            .set(\n                \"group.id\",\n                config.value_require_non_empty((&prefix, \"group-id\"))?,\n            )\n            .set(\n                \"bootstrap.servers\",\n                config.value_require_non_empty((&prefix, \"brokers\"))?,\n            )\n            .set(\"enable.partition.eof\", \"false\")\n            .set(\n                \"session.timeout.ms\",\n                config\n                    .property_or_default((&prefix, \"timeout.session\"), \"5s\")\n                    .unwrap_or(Duration::from_secs(5))\n                    .as_millis()\n                    .to_string(),\n            )\n            .set(\"enable.auto.commit\", \"true\");\n\n        let producer = ClientConfig::new()\n            .set(\n                \"bootstrap.servers\",\n                config.value_require_non_empty((&prefix, \"brokers\"))?,\n            )\n            .set(\n                \"message.timeout.ms\",\n                config\n                    .property_or_default((&prefix, \"timeout.message\"), \"5s\")\n                    .unwrap_or(Duration::from_secs(5))\n                    .as_millis()\n                    .to_string(),\n            )\n            .create()\n            .map_err(|err| {\n                config.new_build_error(\n                    (&prefix, \"config\"),\n                    format!(\"Failed to create Kafka producer: {}\", err),\n                );\n            })\n            .ok()?;\n\n        KafkaPubSub {\n            consumer_builder,\n            producer,\n        }\n        .into()\n    }\n}\n\nimpl Debug for KafkaPubSub {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.debug_struct(\"KafkaPubSub\").finish()\n    }\n}\n\npub(super) struct CustomContext;\n\nimpl ClientContext for CustomContext {}\n\nimpl ConsumerContext for CustomContext {\n    fn pre_rebalance(&self, _: &BaseConsumer<Self>, _: &Rebalance) {}\n\n    fn post_rebalance(&self, _: &BaseConsumer<Self>, _: &Rebalance) {}\n\n    fn commit_callback(&self, _: KafkaResult<()>, _: &TopicPartitionList) {}\n}\n"
  },
  {
    "path": "crates/store/src/backend/kafka/pubsub.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse super::{CustomContext, KafkaPubSub, LoggingConsumer};\nuse crate::dispatch::pubsub::{Msg, PubSubStream};\nuse rdkafka::{\n    Message,\n    consumer::{CommitMode, Consumer, StreamConsumer},\n    producer::FutureRecord,\n};\nuse trc::{ClusterEvent, Error, EventType};\n\npub struct KafkaPubSubStream {\n    subs: LoggingConsumer,\n}\n\nimpl KafkaPubSub {\n    pub async fn publish(&self, topic: &'static str, message: Vec<u8>) -> trc::Result<()> {\n        self.producer\n            .send(\n                FutureRecord::<(), [u8]>::to(topic).payload(message.as_slice()),\n                Duration::from_secs(0),\n            )\n            .await\n            .map(|_| ())\n            .map_err(|(err, _)| {\n                Error::new(EventType::Cluster(ClusterEvent::PublisherError)).reason(err)\n            })\n    }\n\n    pub async fn subscribe(&self, topic: &'static str) -> trc::Result<PubSubStream> {\n        let subs: StreamConsumer<CustomContext> = self\n            .consumer_builder\n            .create_with_context(CustomContext)\n            .map_err(|err| {\n                Error::new(EventType::Cluster(ClusterEvent::SubscriberError)).reason(err)\n            })?;\n        subs.subscribe(&[topic]).map_err(|err| {\n            Error::new(EventType::Cluster(ClusterEvent::SubscriberError)).reason(err)\n        })?;\n\n        Ok(PubSubStream::Kafka(KafkaPubSubStream { subs }))\n    }\n}\n\nimpl KafkaPubSubStream {\n    pub async fn next(&mut self) -> Option<Msg> {\n        let msg = self.subs.recv().await.ok()?;\n        let _ = self.subs.commit_message(&msg, CommitMode::Async);\n        Msg::Kafka(msg.payload().unwrap_or_default().to_vec()).into()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/meili/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    backend::meili::{MeiliSearchStore, Task, TaskStatus, TaskUid},\n    search::{\n        CalendarSearchField, ContactSearchField, EmailSearchField, SearchableField,\n        TracingSearchField,\n    },\n};\nuse reqwest::{Error, Response, Url};\nuse serde_json::{Value, json};\nuse std::time::Duration;\nuse utils::config::{Config, http::build_http_client, utils::AsKey};\n\nimpl MeiliSearchStore {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let client = build_http_client(config, prefix.clone(), \"application/json\".into())?;\n        let prefix = prefix.as_key();\n        let url = config\n            .value_require((&prefix, \"url\"))?\n            .trim_end_matches(\"/\")\n            .to_string();\n        Url::parse(&url)\n            .map_err(|e| config.new_parse_error((&prefix, \"url\"), format!(\"Invalid URL: {e}\",)))\n            .ok()?;\n        let task_poll_interval = config\n            .property_or_default::<Duration>((&prefix, \"task.poll-interval\"), \"500ms\")\n            .unwrap_or(Duration::from_millis(500));\n        let task_poll_retries = config\n            .property_or_default::<usize>((&prefix, \"task.poll-retries\"), \"120\")\n            .unwrap_or(120);\n        let task_fail_on_timeout = config\n            .property_or_default::<bool>((&prefix, \"task.fail-on-timeout\"), \"true\")\n            .unwrap_or(true);\n\n        let ms = Self {\n            client,\n            url,\n            task_poll_interval: Duration::from_millis(500),\n            task_poll_retries: 120,\n            task_fail_on_timeout: true,\n        };\n\n        if let Err(err) = ms.create_indexes().await {\n            config.new_build_error(prefix.as_str(), err.to_string());\n        }\n\n        Some(Self {\n            client: ms.client,\n            url: ms.url,\n            task_poll_interval,\n            task_poll_retries,\n            task_fail_on_timeout,\n        })\n    }\n\n    pub async fn create_indexes(&self) -> trc::Result<()> {\n        self.create_index::<EmailSearchField>().await?;\n        self.create_index::<CalendarSearchField>().await?;\n        self.create_index::<ContactSearchField>().await?;\n        self.create_index::<TracingSearchField>().await?;\n        Ok(())\n    }\n\n    async fn create_index<T: SearchableField>(&self) -> trc::Result<()> {\n        let index_name = T::index().index_name();\n        let response = assert_success(\n            self.client\n                .post(format!(\"{}/indexes\", self.url))\n                .body(\n                    json!({\n                        \"uid\": index_name,\n                        \"primaryKey\": \"id\",\n                    })\n                    .to_string(),\n                )\n                .send()\n                .await,\n        )\n        .await?;\n\n        if !self.wait_for_task(response).await? {\n            // Index already exists\n            return Ok(());\n        }\n\n        let mut searchable = Vec::new();\n        let mut filterable = Vec::new();\n        let mut sortable = Vec::new();\n\n        for field in T::all_fields() {\n            if field.is_indexed() {\n                sortable.push(Value::String(field.field_name().to_string()));\n            }\n            if field.is_text() {\n                searchable.push(Value::String(field.field_name().to_string()));\n            } else {\n                filterable.push(Value::String(field.field_name().to_string()));\n            }\n        }\n\n        for key in T::primary_keys() {\n            filterable.push(Value::String(key.field_name().to_string()));\n        }\n\n        #[cfg(feature = \"test_mode\")]\n        filterable.push(Value::String(\"bcc\".into()));\n\n        if !searchable.is_empty() {\n            self.update_index_settings(\n                index_name,\n                \"searchable-attributes\",\n                Value::Array(searchable),\n            )\n            .await?;\n        }\n\n        if !filterable.is_empty() {\n            self.update_index_settings(\n                index_name,\n                \"filterable-attributes\",\n                Value::Array(filterable),\n            )\n            .await?;\n        }\n\n        if !sortable.is_empty() {\n            self.update_index_settings(index_name, \"sortable-attributes\", Value::Array(sortable))\n                .await?;\n        }\n\n        Ok(())\n    }\n\n    async fn update_index_settings(\n        &self,\n        index_uid: &str,\n        setting: &str,\n        value: Value,\n    ) -> trc::Result<bool> {\n        let response = assert_success(\n            self.client\n                .put(format!(\n                    \"{}/indexes/{}/settings/{}\",\n                    self.url, index_uid, setting\n                ))\n                .body(value.to_string())\n                .send()\n                .await,\n        )\n        .await?;\n        self.wait_for_task(response).await\n    }\n\n    #[cfg(feature = \"test_mode\")]\n    pub async fn drop_indexes(&self) -> trc::Result<()> {\n        use crate::write::SearchIndex;\n\n        for index in &[\n            SearchIndex::Email,\n            SearchIndex::Calendar,\n            SearchIndex::Contacts,\n            SearchIndex::Tracing,\n        ] {\n            let response = self\n                .client\n                .delete(format!(\"{}/indexes/{}\", self.url, index.index_name()))\n                .send()\n                .await\n                .map_err(|err| trc::StoreEvent::MeilisearchError.reason(err))?;\n\n            match response.status().as_u16() {\n                200..=299 => {\n                    self.wait_for_task(response).await?;\n                }\n                400..=499 => {\n                    // Index does not exist\n                    return Ok(());\n                }\n                _ => {\n                    let status = response.status();\n                    let msg = response.text().await.unwrap_or_default();\n                    return Err(trc::StoreEvent::MeilisearchError\n                        .reason(msg)\n                        .ctx(trc::Key::Code, status.as_u16()));\n                }\n            }\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn wait_for_task(&self, response: Response) -> trc::Result<bool> {\n        let response_body = response.text().await.map_err(|err| {\n            trc::StoreEvent::MeilisearchError\n                .reason(err)\n                .details(\"Request failed\")\n        })?;\n        let task_uid = serde_json::from_str::<TaskUid>(&response_body)\n            .map_err(|err| trc::StoreEvent::MeilisearchError.reason(err))?\n            .task_uid;\n\n        let mut loop_count = 0;\n        let url = format!(\"{}/tasks/{}\", self.url, task_uid);\n\n        while loop_count < self.task_poll_retries {\n            let resp = assert_success(self.client.get(&url).send().await).await?;\n\n            let text = resp\n                .text()\n                .await\n                .map_err(|err| trc::StoreEvent::MeilisearchError.reason(err))?;\n\n            let task = serde_json::from_str::<Task>(&text).map_err(|err| {\n                trc::StoreEvent::MeilisearchError\n                    .reason(err)\n                    .details(text.clone())\n            })?;\n\n            match task.status {\n                TaskStatus::Succeeded => return Ok(true),\n                TaskStatus::Failed => {\n                    let (code, message) = task\n                        .error\n                        .map(|e| (e.code, Some(e.message)))\n                        .unwrap_or((None, None));\n                    return if matches!(code.as_deref(), Some(\"index_already_exists\")) {\n                        Ok(false)\n                    } else {\n                        Err(trc::StoreEvent::MeilisearchError\n                            .reason(\"Meilisearch task failed.\")\n                            .id(task_uid)\n                            .code(code)\n                            .details(message))\n                    };\n                }\n                TaskStatus::Canceled => {\n                    return Err(trc::StoreEvent::MeilisearchError\n                        .reason(\"Meilisearch task was canceled\")\n                        .id(task_uid));\n                }\n                TaskStatus::Enqueued | TaskStatus::Processing => {\n                    loop_count += 1;\n                    tokio::time::sleep(self.task_poll_interval).await;\n                }\n                TaskStatus::Unknown => {\n                    return Err(trc::StoreEvent::MeilisearchError\n                        .reason(\"Meilisearch task returned an unknown status\")\n                        .id(task_uid)\n                        .details(text));\n                }\n            }\n        }\n\n        if self.task_fail_on_timeout {\n            Err(trc::StoreEvent::MeilisearchError\n                .reason(\"Timed out waiting for Meilisearch task\")\n                .id(task_uid))\n        } else {\n            Ok(true)\n        }\n    }\n}\n\npub(crate) async fn assert_success(response: Result<Response, Error>) -> trc::Result<Response> {\n    match response {\n        Ok(response) => {\n            let status = response.status();\n            if status.is_success() {\n                Ok(response)\n            } else {\n                Err(trc::StoreEvent::MeilisearchError\n                    .reason(response.text().await.unwrap_or_default())\n                    .ctx(trc::Key::Code, status.as_u16()))\n            }\n        }\n        Err(err) => Err(trc::StoreEvent::MeilisearchError.reason(err)),\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/meili/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse reqwest::Client;\nuse serde::Deserialize;\nuse std::time::Duration;\n\npub mod main;\npub mod search;\n\npub struct MeiliSearchStore {\n    client: Client,\n    url: String,\n    task_poll_interval: Duration,\n    task_poll_retries: usize,\n    task_fail_on_timeout: bool,\n}\n\n#[derive(Debug, Deserialize)]\npub(crate) struct TaskUid {\n    #[serde(rename = \"taskUid\")]\n    pub task_uid: u64,\n}\n\n#[derive(Debug, Deserialize)]\nstruct TaskError {\n    message: String,\n    #[serde(default)]\n    code: Option<String>,\n}\n\n#[derive(Debug, Deserialize)]\nstruct Task {\n    //#[serde(rename = \"uid\")]\n    //uid: u64,\n    status: TaskStatus,\n    #[serde(default)]\n    error: Option<TaskError>,\n}\n\n#[derive(Debug, Deserialize)]\n#[serde(rename_all = \"lowercase\")]\nenum TaskStatus {\n    Enqueued,\n    Processing,\n    Succeeded,\n    Failed,\n    Canceled,\n    #[serde(other)]\n    Unknown,\n}\n\n#[derive(Debug, Deserialize)]\nstruct MeiliSearchResponse {\n    hits: Vec<MeiliHit>,\n    //#[allow(dead_code)]\n    //#[serde(default, rename = \"estimatedTotalHits\")]\n    //estimated_total_hits: Option<u64>,\n}\n\n#[derive(Debug, Deserialize)]\nstruct MeiliHit {\n    id: u64,\n}\n"
  },
  {
    "path": "crates/store/src/backend/meili/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashSet;\nuse serde_json::{Map, Value, json};\n\nuse crate::{\n    backend::meili::{MeiliSearchResponse, MeiliSearchStore, main::assert_success},\n    search::*,\n    write::SearchIndex,\n};\nuse std::fmt::{Display, Write};\n\nimpl MeiliSearchStore {\n    pub async fn index(&self, documents: Vec<IndexDocument>) -> trc::Result<()> {\n        let mut index_documents: [String; 5] = [\n            String::new(),\n            String::new(),\n            String::new(),\n            String::new(),\n            String::new(),\n        ];\n\n        for document in documents {\n            let request = &mut index_documents[document.index.array_pos()];\n            if !request.is_empty() {\n                request.push(',');\n            } else {\n                request.reserve(1024);\n                request.push('[');\n            }\n\n            json_serialize(request, &document);\n        }\n\n        for (mut payload, index) in index_documents.into_iter().zip([\n            SearchIndex::Email,\n            SearchIndex::Calendar,\n            SearchIndex::Contacts,\n            SearchIndex::Tracing,\n            SearchIndex::File,\n        ]) {\n            if payload.is_empty() {\n                continue;\n            }\n\n            payload.push(']');\n\n            let response = assert_success(\n                self.client\n                    .put(format!(\n                        \"{}/indexes/{}/documents\",\n                        self.url,\n                        index.index_name()\n                    ))\n                    .body(payload)\n                    .send()\n                    .await,\n            )\n            .await?;\n            self.wait_for_task(response).await?;\n        }\n\n        Ok(())\n    }\n\n    pub async fn query<R: SearchDocumentId>(\n        &self,\n        index: SearchIndex,\n        filters: &[SearchFilter],\n        sort: &[SearchComparator],\n    ) -> trc::Result<Vec<R>> {\n        let filter_group = build_query(filters);\n\n        let mut body = Map::new();\n        body.insert(\"limit\".to_string(), Value::from(10_000));\n        body.insert(\"offset\".to_string(), Value::from(0));\n        body.insert(\n            \"attributesToRetrieve\".to_string(),\n            Value::Array(vec![Value::String(\"id\".to_string())]),\n        );\n\n        if !filter_group.filter.is_empty() {\n            body.insert(\"filter\".to_string(), Value::String(filter_group.filter));\n        }\n\n        if !filter_group.q.is_empty() {\n            body.insert(\"q\".to_string(), Value::String(filter_group.q));\n        }\n\n        if !sort.is_empty() {\n            let sort_arr: Vec<Value> = sort\n                .iter()\n                .filter_map(|comp| match comp {\n                    SearchComparator::Field { field, ascending } => Some(Value::String(format!(\n                        \"{}:{}\",\n                        field.field_name(),\n                        if *ascending { \"asc\" } else { \"desc\" }\n                    ))),\n                    _ => None,\n                })\n                .collect();\n            if !sort_arr.is_empty() {\n                body.insert(\"sort\".to_string(), Value::Array(sort_arr));\n            }\n        }\n\n        let resp = assert_success(\n            self.client\n                .post(format!(\n                    \"{}/indexes/{}/search\",\n                    self.url,\n                    index.index_name()\n                ))\n                .body(Value::Object(body).to_string())\n                .send()\n                .await,\n        )\n        .await?;\n\n        let text = resp\n            .text()\n            .await\n            .map_err(|err| trc::StoreEvent::MeilisearchError.reason(err))?;\n\n        serde_json::from_str::<MeiliSearchResponse>(&text)\n            .map(|results| {\n                results\n                    .hits\n                    .into_iter()\n                    .map(|hit| R::from_u64(hit.id))\n                    .collect()\n            })\n            .map_err(|err| trc::StoreEvent::MeilisearchError.reason(err).details(text))\n    }\n\n    pub async fn unindex(&self, filter: SearchQuery) -> trc::Result<u64> {\n        let filter_group = build_query(&filter.filters);\n\n        if filter_group.filter.is_empty() {\n            return Err(trc::StoreEvent::MeilisearchError.reason(\n                \"Meilisearch delete-by-filter requires structured (non-text) filters only\",\n            ));\n        }\n\n        let url = format!(\n            \"{}/indexes/{}/documents/delete\",\n            self.url,\n            filter.index.index_name()\n        );\n\n        let response = assert_success(\n            self.client\n                .post(url)\n                .body(json!({ \"filter\": filter_group.filter }).to_string())\n                .send()\n                .await,\n        )\n        .await?;\n\n        self.wait_for_task(response).await?;\n\n        Ok(0)\n    }\n}\n\n#[derive(Default, Debug)]\nstruct FilterGroup {\n    q: String,\n    filter: String,\n}\n\nfn build_query(filters: &[SearchFilter]) -> FilterGroup {\n    if filters.is_empty() {\n        return FilterGroup::default();\n    }\n    let mut operator_stack = Vec::new();\n    let mut operator = &SearchFilter::And;\n    let mut is_first = true;\n    let mut filter = String::new();\n    let mut queries = AHashSet::new();\n\n    for f in filters {\n        match f {\n            SearchFilter::Operator { field, op, value } => {\n                if field.is_text() && matches!(op, SearchOperator::Equal | SearchOperator::Contains)\n                {\n                    let value = match value {\n                        SearchValue::Text { value, .. } => value,\n                        _ => {\n                            debug_assert!(\n                                false,\n                                \"Text field search with non-text value is not supported\"\n                            );\n                            \"\"\n                        }\n                    };\n\n                    if matches!(op, SearchOperator::Equal) {\n                        queries.insert(format!(\"{value:?}\"));\n                    } else {\n                        for token in value.split_whitespace() {\n                            queries.insert(token.to_string());\n                        }\n                    }\n                } else {\n                    if !filter.is_empty() && !filter.ends_with('(') {\n                        match operator {\n                            SearchFilter::And => filter.push_str(\" AND \"),\n                            SearchFilter::Or => filter.push_str(\" OR \"),\n                            _ => (),\n                        }\n                    }\n\n                    match value {\n                        SearchValue::Text { value, .. } => {\n                            filter.push_str(field.field_name());\n                            filter.push(' ');\n                            op.write_meli_op(&mut filter, format!(\"{value:?}\"));\n                        }\n                        SearchValue::KeyValues(kv) => {\n                            let (key, value) = kv.iter().next().unwrap();\n                            filter.push_str(field.field_name());\n                            filter.push('.');\n                            filter.push_str(key);\n                            filter.push(' ');\n                            op.write_meli_op(&mut filter, format!(\"{value:?}\"));\n                        }\n                        SearchValue::Int(v) => {\n                            filter.push_str(field.field_name());\n                            filter.push(' ');\n                            op.write_meli_op(&mut filter, v);\n                        }\n                        SearchValue::Uint(v) => {\n                            filter.push_str(field.field_name());\n                            filter.push(' ');\n                            op.write_meli_op(&mut filter, v);\n                        }\n                        SearchValue::Boolean(v) => {\n                            filter.push_str(field.field_name());\n                            filter.push(' ');\n                            op.write_meli_op(&mut filter, v);\n                        }\n                    }\n                }\n            }\n            SearchFilter::And | SearchFilter::Or => {\n                if !filter.is_empty() && !filter.ends_with('(') {\n                    match operator {\n                        SearchFilter::And => filter.push_str(\" AND \"),\n                        SearchFilter::Or => filter.push_str(\" OR \"),\n                        _ => (),\n                    }\n                }\n\n                operator_stack.push((operator, is_first));\n                operator = f;\n                is_first = true;\n                filter.push('(');\n            }\n            SearchFilter::Not => {\n                if !filter.is_empty() && !filter.ends_with('(') {\n                    match operator {\n                        SearchFilter::And => filter.push_str(\" AND \"),\n                        SearchFilter::Or => filter.push_str(\" OR \"),\n                        _ => (),\n                    }\n                }\n\n                operator_stack.push((operator, is_first));\n                operator = &SearchFilter::And;\n                is_first = true;\n                filter.push_str(\"NOT (\");\n            }\n            SearchFilter::End => {\n                let p = operator_stack.pop().unwrap_or((&SearchFilter::And, true));\n                operator = p.0;\n                is_first = p.1;\n\n                if !filter.ends_with('(') {\n                    filter.push(')');\n                } else {\n                    filter.pop();\n                    if filter.ends_with(\"NOT \") {\n                        let len = filter.len();\n                        filter.truncate(len - 4);\n                    }\n                    if filter.ends_with(\" AND \") {\n                        let len = filter.len();\n                        filter.truncate(len - 5);\n                        is_first = true;\n                    } else if filter.ends_with(\" OR \") {\n                        let len = filter.len();\n                        filter.truncate(len - 4);\n                        is_first = true;\n                    }\n                }\n            }\n            SearchFilter::DocumentSet(_) => {\n                debug_assert!(false, \"DocumentSet filters are not supported\")\n            }\n        }\n    }\n\n    let mut q = String::new();\n    if !queries.is_empty() {\n        for (idx, term) in queries.into_iter().enumerate() {\n            if idx > 0 {\n                q.push(' ');\n            }\n            q.push_str(&term);\n        }\n    }\n\n    FilterGroup { q, filter }\n}\n\nimpl SearchOperator {\n    fn write_meli_op(&self, query: &mut String, value: impl Display) {\n        match self {\n            SearchOperator::LowerThan => {\n                let _ = write!(query, \"< {value}\");\n            }\n            SearchOperator::LowerEqualThan => {\n                let _ = write!(query, \"<= {value}\");\n            }\n            SearchOperator::GreaterThan => {\n                let _ = write!(query, \"> {value}\");\n            }\n            SearchOperator::GreaterEqualThan => {\n                let _ = write!(query, \">= {value}\");\n            }\n            SearchOperator::Equal | SearchOperator::Contains => {\n                let _ = write!(query, \"= {value}\");\n            }\n        }\n    }\n}\n\nfn json_serialize(request: &mut String, document: &IndexDocument) {\n    let mut id = 0u64;\n    let mut is_first = true;\n    request.push('{');\n    for (k, v) in document.fields.iter() {\n        match k {\n            SearchField::AccountId => {\n                if let SearchValue::Uint(account_id) = v {\n                    id |= account_id << 32;\n                }\n            }\n            SearchField::DocumentId => {\n                if let SearchValue::Uint(doc_id) = v {\n                    id |= doc_id;\n                }\n            }\n            SearchField::Id => {\n                if let SearchValue::Uint(doc_id) = v {\n                    id = *doc_id;\n                }\n                continue;\n            }\n            _ => {}\n        }\n\n        if !is_first {\n            request.push(',');\n        } else {\n            is_first = false;\n        }\n\n        let _ = write!(request, \"{:?}:\", k.field_name());\n        match v {\n            SearchValue::Text { value, .. } => {\n                json_serialize_str(request, value);\n            }\n            SearchValue::KeyValues(map) => {\n                request.push('{');\n                for (i, (key, value)) in map.iter().enumerate() {\n                    if i > 0 {\n                        request.push(',');\n                    }\n                    json_serialize_str(request, key);\n                    request.push(':');\n                    json_serialize_str(request, value);\n                }\n                request.push('}');\n            }\n            SearchValue::Int(v) => {\n                let _ = write!(request, \"{}\", v);\n            }\n            SearchValue::Uint(v) => {\n                let _ = write!(request, \"{}\", v);\n            }\n            SearchValue::Boolean(v) => {\n                let _ = write!(request, \"{}\", v);\n            }\n        }\n    }\n\n    /*if id == 0 {\n        debug_assert!(false, \"Document is missing required ID fields\");\n    }*/\n\n    let _ = write!(request, \",\\\"id\\\":{id}}}\");\n}\n\nfn json_serialize_str(request: &mut String, value: &str) {\n    request.push('\"');\n    for c in value.chars() {\n        match c {\n            '\"' => request.push_str(\"\\\\\\\"\"),\n            '\\\\' => request.push_str(\"\\\\\\\\\"),\n            '\\n' => request.push_str(\"\\\\n\"),\n            '\\r' => request.push_str(\"\\\\r\"),\n            '\\t' => request.push_str(\"\\\\t\"),\n            '\\u{0008}' => request.push_str(\"\\\\b\"), // backspace\n            '\\u{000C}' => request.push_str(\"\\\\f\"), // form feed\n            _ => {\n                if !c.is_control() {\n                    request.push(c);\n                } else {\n                    let _ = write!(request, \"\\\\u{:04x}\", c as u32);\n                }\n            }\n        }\n    }\n    request.push('\"');\n}\n\nimpl SearchIndex {\n    #[inline(always)]\n    fn array_pos(&self) -> usize {\n        match self {\n            SearchIndex::Email => 0,\n            SearchIndex::Calendar => 1,\n            SearchIndex::Contacts => 2,\n            SearchIndex::Tracing => 3,\n            SearchIndex::File => 4,\n            SearchIndex::InMemory => unreachable!(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/memory/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::collections::hash_map::Entry;\n\nuse ahash::AHashMap;\nuse utils::{config::Config, glob::GlobMap};\n\nuse crate::{InMemoryStore, Stores, Value};\n\npub type StaticMemoryStore = GlobMap<Value<'static>>;\n\nimpl Stores {\n    pub fn parse_static_stores(&mut self, config: &mut Config, is_reload: bool) {\n        let mut lookups = AHashMap::new();\n        let mut errors = Vec::new();\n\n        for (key, value) in config.iterate_prefix(\"lookup\") {\n            if let Some((id, key)) = key\n                .split_once('.')\n                .filter(|(id, key)| !id.is_empty() && !key.is_empty())\n            {\n                // Detect value type\n                let value = if !value.is_empty() {\n                    let mut has_integers = false;\n                    let mut has_floats = false;\n                    let mut has_others = false;\n\n                    for (pos, ch) in value.as_bytes().iter().enumerate() {\n                        match ch {\n                            b'.' if !has_floats && has_integers => {\n                                has_floats = true;\n                            }\n                            b'0'..=b'9' => {\n                                has_integers = true;\n                            }\n                            b'-' if pos == 0 && value.len() > 1 => {}\n                            _ => {\n                                has_others = true;\n                            }\n                        }\n                    }\n\n                    if has_others {\n                        if value == \"true\" {\n                            Value::Integer(1.into())\n                        } else if value == \"false\" {\n                            Value::Integer(0.into())\n                        } else {\n                            Value::Text(value.to_string().into())\n                        }\n                    } else if has_floats {\n                        value\n                            .parse()\n                            .map(Value::Float)\n                            .unwrap_or_else(|_| Value::Text(value.to_string().into()))\n                    } else {\n                        value\n                            .parse()\n                            .map(Value::Integer)\n                            .unwrap_or_else(|_| Value::Text(value.to_string().into()))\n                    }\n                } else {\n                    Value::Text(\"\".into())\n                };\n\n                // Add entry\n                lookups\n                    .entry(id.to_string())\n                    .or_insert_with(StaticMemoryStore::default)\n                    .insert(key, value);\n            } else {\n                errors.push(key.to_string());\n            }\n        }\n\n        for error in errors {\n            config.new_parse_error(error, \"Invalid lookup key format\");\n        }\n\n        for (id, store) in lookups {\n            match self.in_memory_stores.entry(id) {\n                Entry::Vacant(entry) => {\n                    entry.insert(InMemoryStore::Static(store.into()));\n                }\n                Entry::Occupied(e) if !is_reload => {\n                    config.new_build_error(\n                        (\"lookup\", e.key().as_str()),\n                        \"An in-memory store with this id already exists\",\n                    );\n                }\n                _ => {}\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[cfg(feature = \"azure\")]\npub mod azure;\npub mod elastic;\n#[cfg(feature = \"foundation\")]\npub mod foundationdb;\npub mod fs;\npub mod http;\n#[cfg(feature = \"kafka\")]\npub mod kafka;\npub mod meili;\npub mod memory;\n#[cfg(feature = \"mysql\")]\npub mod mysql;\n#[cfg(feature = \"nats\")]\npub mod nats;\n#[cfg(feature = \"postgres\")]\npub mod postgres;\n#[cfg(feature = \"redis\")]\npub mod redis;\n#[cfg(feature = \"rocks\")]\npub mod rocksdb;\n#[cfg(feature = \"s3\")]\npub mod s3;\n#[cfg(feature = \"sqlite\")]\npub mod sqlite;\n#[cfg(feature = \"zenoh\")]\npub mod zenoh;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\npub mod composite;\n// SPDX-SnippetEnd\n\npub const MAX_TOKEN_LENGTH: usize = (u8::MAX >> 1) as usize;\npub const MAX_TOKEN_MASK: usize = MAX_TOKEN_LENGTH - 1;\n\n#[allow(dead_code)]\nfn deserialize_i64_le(key: &[u8], bytes: &[u8]) -> trc::Result<i64> {\n    Ok(i64::from_le_bytes(bytes[..].try_into().map_err(|_| {\n        trc::Error::corrupted_key(key, bytes.into(), trc::location!())\n    })?))\n}\n"
  },
  {
    "path": "crates/store/src/backend/mysql/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::ops::Range;\n\nuse mysql_async::prelude::Queryable;\n\nuse super::{MysqlStore, into_error};\n\nimpl MysqlStore {\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn\n            .prep(\"SELECT v FROM t WHERE k = ?\")\n            .await\n            .map_err(into_error)?;\n        conn.exec_first::<Vec<u8>, _, _>(&s, (key,))\n            .await\n            .map(|bytes| {\n                if range.start == 0 && range.end == usize::MAX {\n                    bytes\n                } else {\n                    bytes.map(|bytes| {\n                        bytes\n                            .get(range.start..std::cmp::min(bytes.len(), range.end))\n                            .unwrap_or_default()\n                            .to_vec()\n                    })\n                }\n            })\n            .map_err(into_error)\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn\n            .prep(\"INSERT INTO t (k, v) VALUES (?, ?) ON DUPLICATE KEY UPDATE v = VALUES(v)\")\n            .await\n            .map_err(into_error)?;\n        conn.exec_drop(&s, (key, data))\n            .await\n            .map_err(into_error)\n            .map(|_| ())\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn\n            .prep(\"DELETE FROM t WHERE k = ?\")\n            .await\n            .map_err(into_error)?;\n        conn.exec_iter(&s, (key,))\n            .await\n            .map_err(into_error)\n            .map(|hits| hits.affected_rows() > 0)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/mysql/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse mysql_async::{Params, Row, prelude::Queryable};\n\nuse crate::{IntoRows, QueryResult, QueryType, Value};\n\nuse super::{MysqlStore, into_error};\n\nimpl MysqlStore {\n    pub(crate) async fn sql_query<T: QueryResult>(\n        &self,\n        query: &str,\n        params: &[Value<'_>],\n    ) -> trc::Result<T> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn.prep(query).await.map_err(into_error)?;\n        let params = Params::Positional(params.iter().map(Into::into).collect());\n\n        match T::query_type() {\n            QueryType::Execute => conn.exec_drop(s, params).await.map_or_else(\n                |e| Err(into_error(e)),\n                |_| Ok(T::from_exec(conn.affected_rows() as usize)),\n            ),\n            QueryType::Exists => conn\n                .exec_first::<Row, _, _>(s, params)\n                .await\n                .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_exists(r.is_some()))),\n            QueryType::QueryOne => conn\n                .exec_first::<Row, _, _>(s, params)\n                .await\n                .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_query_one(r))),\n            QueryType::QueryAll => conn\n                .exec::<Row, _, _>(s, params)\n                .await\n                .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_query_all(r))),\n        }\n    }\n}\n\nimpl From<crate::Value<'_>> for mysql_async::Value {\n    fn from(value: crate::Value) -> Self {\n        match value {\n            crate::Value::Integer(i) => mysql_async::Value::Int(i),\n            crate::Value::Bool(b) => mysql_async::Value::Int(b as i64),\n            crate::Value::Float(f) => mysql_async::Value::Double(f),\n            crate::Value::Text(t) => mysql_async::Value::Bytes(t.into_owned().into_bytes()),\n            crate::Value::Blob(b) => mysql_async::Value::Bytes(b.into_owned()),\n            crate::Value::Null => mysql_async::Value::NULL,\n        }\n    }\n}\n\nimpl From<mysql_async::Value> for crate::Value<'static> {\n    fn from(value: mysql_async::Value) -> Self {\n        match value {\n            mysql_async::Value::Int(i) => Self::Integer(i),\n            mysql_async::Value::UInt(i) => Self::Integer(i as i64),\n            mysql_async::Value::Double(f) => Self::Float(f),\n            mysql_async::Value::Bytes(b) => String::from_utf8(b).map_or_else(\n                |e| Self::Blob(e.into_bytes().into()),\n                |s| Self::Text(s.into()),\n            ),\n            mysql_async::Value::NULL => Self::Null,\n            mysql_async::Value::Float(f) => Self::Float(f as f64),\n            mysql_async::Value::Date(_, _, _, _, _, _, _)\n            | mysql_async::Value::Time(_, _, _, _, _, _) => Self::Text(value.as_sql(true).into()),\n        }\n    }\n}\n\nimpl IntoRows for Vec<mysql_async::Row> {\n    fn into_rows(self) -> crate::Rows {\n        crate::Rows {\n            rows: self\n                .into_iter()\n                .map(|r| crate::Row {\n                    values: r\n                        .unwrap_raw()\n                        .into_iter()\n                        .flatten()\n                        .map(Into::into)\n                        .collect(),\n                })\n                .collect(),\n        }\n    }\n\n    fn into_named_rows(self) -> crate::NamedRows {\n        crate::NamedRows {\n            names: self\n                .first()\n                .map(|r| r.columns().iter().map(|c| c.name_str().into()).collect())\n                .unwrap_or_default(),\n            rows: self\n                .into_iter()\n                .map(|r| crate::Row {\n                    values: r\n                        .unwrap_raw()\n                        .into_iter()\n                        .flatten()\n                        .map(Into::into)\n                        .collect(),\n                })\n                .collect(),\n        }\n    }\n\n    fn into_row(self) -> Option<crate::Row> {\n        unreachable!()\n    }\n}\n\nimpl IntoRows for Option<mysql_async::Row> {\n    fn into_row(self) -> Option<crate::Row> {\n        self.map(|row| crate::Row {\n            values: row\n                .unwrap_raw()\n                .into_iter()\n                .flatten()\n                .map(Into::into)\n                .collect(),\n        })\n    }\n\n    fn into_rows(self) -> crate::Rows {\n        unreachable!()\n    }\n\n    fn into_named_rows(self) -> crate::NamedRows {\n        unreachable!()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/mysql/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse mysql_async::{\n    Conn, OptsBuilder, Pool, PoolConstraints, PoolOpts, SslOpts, prelude::Queryable,\n};\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::{\n    backend::mysql::MysqlSearchField,\n    search::{\n        CalendarSearchField, ContactSearchField, EmailSearchField, SearchableField,\n        TracingSearchField,\n    },\n    *,\n};\n\nuse super::{MysqlStore, into_error};\n\nimpl MysqlStore {\n    pub async fn open(\n        config: &mut Config,\n        prefix: impl AsKey,\n        create_store_tables: bool,\n        create_search_tables: bool,\n    ) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let mut opts = OptsBuilder::default()\n            .ip_or_hostname(config.value_require((&prefix, \"host\"))?.to_string())\n            .user(config.value((&prefix, \"user\")).map(|s| s.to_string()))\n            .pass(config.value((&prefix, \"password\")).map(|s| s.to_string()))\n            .db_name(\n                config\n                    .value_require((&prefix, \"database\"))?\n                    .to_string()\n                    .into(),\n            )\n            .max_allowed_packet(config.property((&prefix, \"max-allowed-packet\")))\n            .wait_timeout(\n                config\n                    .property::<Option<Duration>>((&prefix, \"timeout\"))\n                    .unwrap_or_default()\n                    .map(|t| t.as_secs() as usize),\n            )\n            .client_found_rows(true);\n        if let Some(port) = config.property((&prefix, \"port\")) {\n            opts = opts.tcp_port(port);\n        }\n\n        if config\n            .property_or_default::<bool>((&prefix, \"tls.enable\"), \"false\")\n            .unwrap_or_default()\n        {\n            let allow_invalid = config\n                .property_or_default::<bool>((&prefix, \"tls.allow-invalid-certs\"), \"false\")\n                .unwrap_or_default();\n            opts = opts.ssl_opts(Some(\n                SslOpts::default()\n                    .with_danger_accept_invalid_certs(allow_invalid)\n                    .with_danger_skip_domain_validation(allow_invalid),\n            ));\n        }\n\n        // Configure connection pool\n        let mut pool_min = PoolConstraints::default().min();\n        let mut pool_max = PoolConstraints::default().max();\n        if let Some(n_size) = config\n            .property::<usize>((&prefix, \"pool.min-connections\"))\n            .filter(|&n| n > 0)\n        {\n            pool_min = n_size;\n        }\n        if let Some(n_size) = config\n            .property::<usize>((&prefix, \"pool.max-connections\"))\n            .filter(|&n| n > 0)\n        {\n            pool_max = n_size;\n        }\n        opts = opts.pool_opts(\n            PoolOpts::default().with_constraints(PoolConstraints::new(pool_min, pool_max).unwrap()),\n        );\n\n        let db = Self {\n            conn_pool: Pool::new(opts),\n        };\n\n        if create_store_tables && let Err(err) = db.create_storage_tables().await {\n            config.new_build_error(prefix.as_str(), format!(\"Failed to create tables: {err}\"));\n        }\n\n        if create_search_tables && let Err(err) = db.create_search_tables().await {\n            config.new_build_warning(\n                prefix.as_str(),\n                format!(\"Failed to create search tables: {err}\"),\n            );\n        }\n\n        Some(db)\n    }\n\n    pub(crate) async fn create_storage_tables(&self) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n\n        for table in [\n            SUBSPACE_ACL,\n            SUBSPACE_DIRECTORY,\n            SUBSPACE_TASK_QUEUE,\n            SUBSPACE_BLOB_EXTRA,\n            SUBSPACE_BLOB_LINK,\n            SUBSPACE_IN_MEMORY_VALUE,\n            SUBSPACE_PROPERTY,\n            SUBSPACE_SETTINGS,\n            SUBSPACE_QUEUE_MESSAGE,\n            SUBSPACE_QUEUE_EVENT,\n            SUBSPACE_REPORT_OUT,\n            SUBSPACE_REPORT_IN,\n            SUBSPACE_LOGS,\n            SUBSPACE_TELEMETRY_SPAN,\n            SUBSPACE_TELEMETRY_METRIC,\n        ] {\n            let table = char::from(table);\n            conn.query_drop(format!(\n                \"CREATE TABLE IF NOT EXISTS {table} (\n                    k TINYBLOB,\n                    v MEDIUMBLOB NOT NULL,\n                    PRIMARY KEY (k(255))\n                ) ENGINE=InnoDB\"\n            ))\n            .await\n            .map_err(into_error)?;\n        }\n\n        conn.query_drop(format!(\n            \"CREATE TABLE IF NOT EXISTS {} (\n                k TINYBLOB,\n                v LONGBLOB NOT NULL,\n                PRIMARY KEY (k(255))\n            ) ENGINE=InnoDB\",\n            char::from(SUBSPACE_BLOBS),\n        ))\n        .await\n        .map_err(into_error)?;\n\n        for table in [SUBSPACE_INDEXES] {\n            let table = char::from(table);\n            conn.query_drop(format!(\n                \"CREATE TABLE IF NOT EXISTS {table} (\n                    k BLOB,\n                    PRIMARY KEY (k(400))\n                ) ENGINE=InnoDB\"\n            ))\n            .await\n            .map_err(into_error)?;\n        }\n\n        for table in [SUBSPACE_COUNTER, SUBSPACE_QUOTA, SUBSPACE_IN_MEMORY_COUNTER] {\n            conn.query_drop(format!(\n                \"CREATE TABLE IF NOT EXISTS {} (\n                k TINYBLOB,\n                v BIGINT NOT NULL DEFAULT 0,\n                PRIMARY KEY (k(255))\n            ) ENGINE=InnoDB\",\n                char::from(table)\n            ))\n            .await\n            .map_err(into_error)?;\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn create_search_tables(&self) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n\n        create_search_tables::<EmailSearchField>(&mut conn).await?;\n        create_search_tables::<CalendarSearchField>(&mut conn).await?;\n        create_search_tables::<ContactSearchField>(&mut conn).await?;\n        //create_search_tables::<FileSearchField>(&mut conn).await?;\n        create_search_tables::<TracingSearchField>(&mut conn).await?;\n\n        Ok(())\n    }\n}\n\nasync fn create_search_tables<T: SearchableField + MysqlSearchField + 'static>(\n    conn: &mut Conn,\n) -> trc::Result<()> {\n    let table_name = T::index().mysql_table();\n    let mut query = format!(\"CREATE TABLE IF NOT EXISTS {} (\", table_name);\n\n    // Add primary key columns\n    let pkeys = T::primary_keys();\n    for pkey in pkeys {\n        query.push_str(&format!(\"{} {}, \", pkey.column(), pkey.column_type()));\n    }\n\n    // Add other columns\n    for field in T::all_fields() {\n        query.push_str(&format!(\"{} {}, \", field.column(), field.column_type()));\n    }\n\n    // Add primary key constraint\n    query.push_str(\"PRIMARY KEY (\");\n    for (i, pkey) in pkeys.iter().enumerate() {\n        if i > 0 {\n            query.push_str(\", \");\n        }\n        query.push_str(pkey.column());\n    }\n    query.push_str(\")) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci\");\n\n    conn.query_drop(&query).await.map_err(into_error)?;\n\n    // Create indexes\n    for field in T::all_fields() {\n        if field.is_text() {\n            let column_name = field.column();\n            let create_index_query = format!(\n                \"CREATE FULLTEXT INDEX fts_{table_name}_{column_name} ON {table_name}({column_name})\",\n            );\n\n            let _ = conn.query_drop(&create_index_query).await;\n        }\n\n        if field.is_indexed() {\n            let column_name = field.column();\n            let create_index_query = format!(\n                \"CREATE INDEX idx_{table_name}_{column_name} ON {table_name}({column_name})\",\n            );\n            let _ = conn.query_drop(&create_index_query).await;\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/store/src/backend/mysql/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    search::{\n        CalendarSearchField, ContactSearchField, EmailSearchField, FileSearchField, SearchField,\n        TracingSearchField,\n    },\n    write::SearchIndex,\n};\nuse mysql_async::Pool;\nuse std::fmt::Display;\n\npub mod blob;\npub mod lookup;\npub mod main;\npub mod read;\npub mod search;\npub mod write;\n\npub struct MysqlStore {\n    pub(crate) conn_pool: Pool,\n}\n\n#[inline(always)]\nfn into_error(err: impl Display) -> trc::Error {\n    trc::StoreEvent::MysqlError.reason(err)\n}\n\nimpl SearchIndex {\n    pub fn mysql_table(&self) -> &'static str {\n        match self {\n            SearchIndex::Email => \"s_email\",\n            SearchIndex::Calendar => \"s_cal\",\n            SearchIndex::Contacts => \"s_card\",\n            SearchIndex::File => \"s_file\",\n            SearchIndex::Tracing => \"s_trace\",\n            SearchIndex::InMemory => \"\",\n        }\n    }\n}\n\ntrait MysqlSearchField {\n    fn column(&self) -> &'static str;\n    fn column_type(&self) -> &'static str;\n}\n\nimpl MysqlSearchField for EmailSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            EmailSearchField::From => \"fadr\",\n            EmailSearchField::To => \"tadr\",\n            EmailSearchField::Cc => \"cc\",\n            EmailSearchField::Bcc => \"bcc\",\n            EmailSearchField::Subject => \"subj\",\n            EmailSearchField::Body => \"body\",\n            EmailSearchField::Attachment => \"atta\",\n            EmailSearchField::ReceivedAt => \"rcvd\",\n            EmailSearchField::SentAt => \"sent\",\n            EmailSearchField::Size => \"size\",\n            EmailSearchField::HasAttachment => \"hatt\",\n            EmailSearchField::Headers => \"hdrs\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            EmailSearchField::ReceivedAt | EmailSearchField::SentAt => \"BIGINT\",\n            EmailSearchField::Size => \"INT\",\n            EmailSearchField::HasAttachment => \"BOOLEAN\",\n            EmailSearchField::Headers => \"JSON\",\n            EmailSearchField::From => \"TEXT\",\n            EmailSearchField::To => \"TEXT\",\n            EmailSearchField::Cc => \"TEXT\",\n            EmailSearchField::Bcc => \"TEXT\",\n            EmailSearchField::Subject => \"TEXT\",\n            EmailSearchField::Body => \"MEDIUMTEXT\",\n            EmailSearchField::Attachment => \"MEDIUMTEXT\",\n        }\n    }\n}\n\nimpl MysqlSearchField for CalendarSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            CalendarSearchField::Title => \"titl\",\n            CalendarSearchField::Description => \"dscd\",\n            CalendarSearchField::Location => \"locn\",\n            CalendarSearchField::Owner => \"ownr\",\n            CalendarSearchField::Attendee => \"atnd\",\n            CalendarSearchField::Start => \"strt\",\n            CalendarSearchField::Uid => \"uid\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            CalendarSearchField::Start => \"BIGINT NOT NULL\",\n            _ => \"TEXT\",\n        }\n    }\n}\n\nimpl MysqlSearchField for ContactSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            ContactSearchField::Member => \"mmbr\",\n            ContactSearchField::Name => \"name\",\n            ContactSearchField::Nickname => \"nick\",\n            ContactSearchField::Organization => \"orgn\",\n            ContactSearchField::Email => \"eml\",\n            ContactSearchField::Phone => \"phon\",\n            ContactSearchField::OnlineService => \"olsv\",\n            ContactSearchField::Address => \"addr\",\n            ContactSearchField::Note => \"note\",\n            ContactSearchField::Kind => \"kind\",\n            ContactSearchField::Uid => \"uid\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            ContactSearchField::Kind | ContactSearchField::Uid => \"TEXT\",\n            _ => \"TEXT\",\n        }\n    }\n}\n\nimpl MysqlSearchField for FileSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            FileSearchField::Name => \"name\",\n            FileSearchField::Content => \"body\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            FileSearchField::Name => \"TEXT\",\n            FileSearchField::Content => \"MEDIUMTEXT\",\n        }\n    }\n}\nimpl MysqlSearchField for TracingSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            TracingSearchField::QueueId => \"qid\",\n            TracingSearchField::EventType => \"etyp\",\n            TracingSearchField::Keywords => \"kwds\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            TracingSearchField::EventType => \"BIGINT\",\n            TracingSearchField::QueueId => \"BIGINT\",\n            TracingSearchField::Keywords => \"TEXT\",\n        }\n    }\n}\n\nimpl MysqlSearchField for SearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            SearchField::AccountId => \"accid\",\n            SearchField::DocumentId => \"docid\",\n            SearchField::Id => \"id\",\n            SearchField::Email(field) => field.column(),\n            SearchField::Calendar(field) => field.column(),\n            SearchField::Contact(field) => field.column(),\n            SearchField::File(field) => field.column(),\n            SearchField::Tracing(field) => field.column(),\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            SearchField::AccountId => \"INT NOT NULL\",\n            SearchField::DocumentId => \"INT NOT NULL\",\n            SearchField::Id => \"BIGINT NOT NULL\",\n            SearchField::Email(field) => field.column_type(),\n            SearchField::Calendar(field) => field.column_type(),\n            SearchField::Contact(field) => field.column_type(),\n            SearchField::File(field) => field.column_type(),\n            SearchField::Tracing(field) => field.column_type(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/mysql/read.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{MysqlStore, into_error};\nuse crate::{Deserialize, IterateParams, Key, ValueKey, write::ValueClass};\nuse futures::TryStreamExt;\nuse mysql_async::{Row, prelude::Queryable};\n\nimpl MysqlStore {\n    pub(crate) async fn get_value<U>(&self, key: impl Key) -> trc::Result<Option<U>>\n    where\n        U: Deserialize + 'static,\n    {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn\n            .prep(format!(\n                \"SELECT v FROM {} WHERE k = ?\",\n                char::from(key.subspace())\n            ))\n            .await\n            .map_err(into_error)?;\n        let key = key.serialize(0);\n        conn.exec_first::<Vec<u8>, _, _>(&s, (key,))\n            .await\n            .map_err(into_error)\n            .and_then(|r| {\n                if let Some(r) = r {\n                    Ok(Some(U::deserialize_owned(r)?))\n                } else {\n                    Ok(None)\n                }\n            })\n    }\n\n    pub(crate) async fn iterate<T: Key>(\n        &self,\n        params: IterateParams<T>,\n        mut cb: impl for<'x> FnMut(&'x [u8], &'x [u8]) -> trc::Result<bool> + Sync + Send,\n    ) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let table = char::from(params.begin.subspace());\n        let begin = params.begin.serialize(0);\n        let end = params.end.serialize(0);\n        let keys = if params.values { \"k, v\" } else { \"k\" };\n\n        let s = conn\n            .prep(&match (params.first, params.ascending) {\n                (true, true) => {\n                    format!(\n                        \"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k ASC LIMIT 1\"\n                    )\n                }\n                (true, false) => {\n                    format!(\n                        \"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k DESC LIMIT 1\"\n                    )\n                }\n                (false, true) => {\n                    format!(\"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k ASC\")\n                }\n                (false, false) => {\n                    format!(\"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k DESC\")\n                }\n            })\n            .await\n            .map_err(into_error)?;\n        let mut rows = conn\n            .exec_stream::<Row, _, _>(&s, (begin, end))\n            .await\n            .map_err(into_error)?;\n\n        if params.values {\n            while let Some(mut row) = rows.try_next().await.map_err(into_error)? {\n                let value = row\n                    .take_opt::<Vec<u8>, _>(1)\n                    .unwrap_or_else(|| Ok(vec![]))\n                    .map_err(into_error)?;\n                let key = row\n                    .take_opt::<Vec<u8>, _>(0)\n                    .unwrap_or_else(|| Ok(vec![]))\n                    .map_err(into_error)?;\n\n                if !cb(&key, &value)? {\n                    break;\n                }\n            }\n        } else {\n            while let Some(mut row) = rows.try_next().await.map_err(into_error)? {\n                if !cb(\n                    &row.take_opt::<Vec<u8>, _>(0)\n                        .unwrap_or_else(|| Ok(vec![]))\n                        .map_err(into_error)?,\n                    b\"\",\n                )? {\n                    break;\n                }\n            }\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn get_counter(\n        &self,\n        key: impl Into<ValueKey<ValueClass>> + Sync + Send,\n    ) -> trc::Result<i64> {\n        let key = key.into();\n        let table = char::from(key.subspace());\n        let key = key.serialize(0);\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn\n            .prep(format!(\"SELECT v FROM {table} WHERE k = ?\"))\n            .await\n            .map_err(into_error)?;\n        match conn.exec_first::<i64, _, _>(&s, (key,)).await {\n            Ok(Some(num)) => Ok(num),\n            Ok(None) => Ok(0),\n            Err(e) => Err(into_error(e)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/mysql/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    backend::{\n        MAX_TOKEN_LENGTH,\n        mysql::{MysqlSearchField, MysqlStore, into_error},\n    },\n    search::{\n        IndexDocument, SearchComparator, SearchDocumentId, SearchFilter, SearchOperator,\n        SearchQuery, SearchValue,\n    },\n    write::SearchIndex,\n};\nuse mysql_async::{IsolationLevel, TxOpts, Value, prelude::Queryable};\nuse nlp::tokenizers::word::WordTokenizer;\nuse std::fmt::Write;\n\nimpl MysqlStore {\n    pub async fn index(&self, documents: Vec<IndexDocument>) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let mut tx_opts = TxOpts::default();\n        tx_opts\n            .with_consistent_snapshot(false)\n            .with_isolation_level(IsolationLevel::ReadCommitted);\n        let mut trx = conn.start_transaction(tx_opts).await.map_err(into_error)?;\n\n        for document in documents {\n            let index = document.index;\n            let primary_keys = index.primary_keys();\n            let all_fields = index.all_fields();\n            let mut fields = document.fields;\n            let mut values = Vec::with_capacity(fields.len() + 2);\n            let mut query = format!(\"INSERT INTO {} (\", index.mysql_table());\n\n            for (i, field) in primary_keys.iter().chain(all_fields).enumerate() {\n                if i > 0 {\n                    query.push(',');\n                }\n                query.push_str(field.column());\n            }\n\n            query.push_str(\") VALUES (\");\n\n            for (i, field) in primary_keys.iter().chain(all_fields).enumerate() {\n                if i > 0 {\n                    query.push(',');\n                }\n\n                if let Some(value) = fields.remove(field) {\n                    query.push('?');\n                    values.push(value);\n                } else {\n                    query.push_str(\"NULL\");\n                }\n            }\n\n            query.push_str(\") ON DUPLICATE KEY UPDATE \");\n            for (i, field) in all_fields.iter().enumerate() {\n                if i > 0 {\n                    query.push(',');\n                }\n                let column = field.column();\n                let _ = write!(&mut query, \"{column} = VALUES({column})\");\n            }\n\n            let s = trx.prep(&query).await.map_err(into_error)?;\n\n            trx.exec_drop(&s, values).await.map_err(into_error)?;\n        }\n\n        trx.commit().await.map_err(into_error)\n    }\n\n    pub async fn query<R: SearchDocumentId>(\n        &self,\n        index: SearchIndex,\n        filters: &[SearchFilter],\n        sort: &[SearchComparator],\n    ) -> trc::Result<Vec<R>> {\n        let mut query = format!(\n            \"SELECT {} FROM {}\",\n            R::field().column(),\n            index.mysql_table()\n        );\n        let params = build_filter(&mut query, filters);\n        if !sort.is_empty() {\n            build_sort(&mut query, sort);\n        }\n\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn.prep(query).await.map_err(into_error)?;\n\n        conn.exec::<i64, _, _>(s, params)\n            .await\n            .map(|r| r.into_iter().map(|r| R::from_u64(r as u64)).collect())\n            .map_err(into_error)\n    }\n\n    pub async fn unindex(&self, filter: SearchQuery) -> trc::Result<u64> {\n        let mut query = format!(\"DELETE FROM {} \", filter.index.mysql_table());\n        let params = build_filter(&mut query, &filter.filters);\n\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        let s = conn.prep(query).await.map_err(into_error)?;\n\n        conn.exec_drop(s, params)\n            .await\n            .map(|_| conn.affected_rows() as u64)\n            .map_err(into_error)\n    }\n}\n\nfn build_filter(query: &mut String, filters: &[SearchFilter]) -> Vec<Value> {\n    if filters.is_empty() {\n        return Vec::new();\n    }\n    query.push_str(\" WHERE \");\n    let mut operator_stack = Vec::new();\n    let mut operator = &SearchFilter::And;\n    let mut is_first = true;\n    let mut values: Vec<Value> = Vec::new();\n\n    for filter in filters {\n        match filter {\n            SearchFilter::Operator { field, op, value } => {\n                if !is_first {\n                    match operator {\n                        SearchFilter::And => query.push_str(\" AND \"),\n                        SearchFilter::Or => query.push_str(\" OR \"),\n                        _ => (),\n                    }\n                } else {\n                    is_first = false;\n                }\n\n                if field.is_text() && matches!(op, SearchOperator::Equal | SearchOperator::Contains)\n                {\n                    let (value, mode) = match (value, op) {\n                        (SearchValue::Text { value, .. }, SearchOperator::Equal) => {\n                            (Value::Bytes(format!(\"{value:?}\").into_bytes()), \"BOOLEAN\")\n                        }\n                        (SearchValue::Text { value, .. }, ..) => {\n                            let mut text_query = String::with_capacity(value.len() + 1);\n\n                            for item in WordTokenizer::new(value, MAX_TOKEN_LENGTH) {\n                                if !text_query.is_empty() {\n                                    text_query.push(' ');\n                                }\n                                text_query.push('+');\n                                text_query.push_str(&item.word);\n                            }\n\n                            (Value::Bytes(text_query.into_bytes()), \"BOOLEAN\")\n                        }\n                        _ => {\n                            debug_assert!(false, \"Invalid search value for text field\");\n                            continue;\n                        }\n                    };\n                    let _ = write!(query, \"MATCH({}) AGAINST(? IN {mode} MODE)\", field.column());\n                    values.push(value);\n                } else if let SearchValue::KeyValues(kv) = value {\n                    let (key, value) = kv.iter().next().unwrap();\n\n                    values.push(Value::Bytes(format!(\"$.{key:?}\").into_bytes()));\n\n                    if !value.is_empty() {\n                        if op == &SearchOperator::Equal {\n                            let _ = write!(query, \"JSON_EXTRACT({}, ?) = ?\", field.column());\n                            values.push(Value::Bytes(value.as_bytes().to_vec()));\n                        } else {\n                            let _ = write!(query, \"JSON_EXTRACT({}, ?) LIKE ?\", field.column(),);\n                            values.push(Value::Bytes(format!(\"%{value}%\").into_bytes()));\n                        }\n                    } else {\n                        let _ = write!(query, \"JSON_CONTAINS_PATH({}, 'one', ?)\", field.column(),);\n                    }\n                } else {\n                    query.push_str(field.column());\n                    query.push(' ');\n                    op.write_mysql(query);\n                    values.push(to_mysql(value));\n                }\n            }\n            SearchFilter::And | SearchFilter::Or => {\n                if !is_first {\n                    match operator {\n                        SearchFilter::And => query.push_str(\" AND \"),\n                        SearchFilter::Or => query.push_str(\" OR \"),\n                        _ => (),\n                    }\n                } else {\n                    is_first = false;\n                }\n\n                operator_stack.push((operator, is_first));\n                operator = filter;\n                is_first = true;\n                query.push('(');\n            }\n            SearchFilter::Not => {\n                if !is_first {\n                    match operator {\n                        SearchFilter::And => query.push_str(\" AND \"),\n                        SearchFilter::Or => query.push_str(\" OR \"),\n                        _ => (),\n                    }\n                } else {\n                    is_first = false;\n                }\n\n                operator_stack.push((operator, is_first));\n                operator = &SearchFilter::And;\n                is_first = true;\n                query.push_str(\"NOT (\");\n            }\n            SearchFilter::End => {\n                let p = operator_stack.pop().unwrap_or((&SearchFilter::And, true));\n                operator = p.0;\n                is_first = p.1;\n                query.push(')');\n            }\n            SearchFilter::DocumentSet(_) => {\n                debug_assert!(\n                    false,\n                    \"DocumentSet filters are not supported in Postgres backend\"\n                )\n            }\n        }\n    }\n\n    values\n}\n\nfn build_sort(query: &mut String, sort: &[SearchComparator]) {\n    query.push_str(\" ORDER BY \");\n    for (i, comparator) in sort.iter().enumerate() {\n        if i > 0 {\n            query.push_str(\", \");\n        }\n        match comparator {\n            SearchComparator::Field { field, ascending } => {\n                query.push_str(field.column());\n                if *ascending {\n                    query.push_str(\" ASC\");\n                } else {\n                    query.push_str(\" DESC\");\n                }\n            }\n            SearchComparator::DocumentSet { .. } | SearchComparator::SortedSet { .. } => {\n                debug_assert!(\n                    false,\n                    \"DocumentSet and SortedSet comparators are not supported \"\n                );\n            }\n        }\n    }\n}\n\nimpl SearchOperator {\n    fn write_mysql(&self, query: &mut String) {\n        match self {\n            SearchOperator::LowerThan => {\n                let _ = write!(query, \"< ?\");\n            }\n            SearchOperator::LowerEqualThan => {\n                let _ = write!(query, \"<= ?\");\n            }\n            SearchOperator::GreaterThan => {\n                let _ = write!(query, \"> ?\");\n            }\n            SearchOperator::GreaterEqualThan => {\n                let _ = write!(query, \">= ?\");\n            }\n            SearchOperator::Equal => {\n                let _ = write!(query, \"= ?\");\n            }\n            SearchOperator::Contains => {\n                let _ = write!(query, \"LIKE '%' CONCAT('%', ?, '%')\");\n            }\n        }\n    }\n}\n\nimpl From<SearchValue> for Value {\n    fn from(value: SearchValue) -> Self {\n        match value {\n            SearchValue::Text { mut value, .. } => {\n                // Truncate values larger than 16MB to avoid MySQL errors\n                if value.len() > 16_777_214 {\n                    let pos = value.floor_char_boundary(16_777_214);\n                    value.truncate(pos);\n                }\n\n                Value::Bytes(value.into_bytes())\n            }\n            SearchValue::KeyValues(vec_map) => serde_json::to_string(&vec_map)\n                .map(|v| Value::Bytes(v.into_bytes()))\n                .unwrap_or(Value::NULL),\n            SearchValue::Int(i) => Value::Int(i),\n            SearchValue::Uint(i) => Value::Int(i as i64),\n            SearchValue::Boolean(b) => Value::Int(b as i64),\n        }\n    }\n}\n\nfn to_mysql(value: &SearchValue) -> Value {\n    match value {\n        SearchValue::Text { value, .. } => Value::Bytes(value.as_bytes().to_vec()),\n        SearchValue::KeyValues(vec_map) => serde_json::to_string(&vec_map)\n            .map(|v| Value::Bytes(v.into_bytes()))\n            .unwrap_or(Value::NULL),\n        SearchValue::Int(i) => Value::Int(*i),\n        SearchValue::Uint(i) => Value::Int(*i as i64),\n        SearchValue::Boolean(b) => Value::Int(*b as i64),\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/mysql/write.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{MysqlStore, into_error};\nuse crate::{\n    IndexKey, Key, LogKey, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER, SUBSPACE_QUOTA,\n    write::{\n        AssignedIds, Batch, MAX_COMMIT_ATTEMPTS, MAX_COMMIT_TIME, MergeResult, Operation,\n        ValueClass, ValueOp,\n    },\n};\nuse ahash::AHashMap;\nuse mysql_async::{Conn, Error, IsolationLevel, TxOpts, params, prelude::Queryable};\nuse rand::Rng;\nuse std::time::{Duration, Instant};\n\n#[derive(Debug)]\nenum CommitError {\n    Mysql(mysql_async::Error),\n    Internal(trc::Error),\n    //Retry,\n}\n\nimpl MysqlStore {\n    pub(crate) async fn write(&self, mut batch: Batch<'_>) -> trc::Result<AssignedIds> {\n        let start = Instant::now();\n        let mut retry_count = 0;\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n\n        loop {\n            let err = match self.write_trx(&mut conn, &mut batch).await {\n                Ok(result) => {\n                    return Ok(result);\n                }\n                Err(err) => err,\n            };\n\n            let _ = conn.query_drop(\"ROLLBACK;\").await;\n\n            match err {\n                CommitError::Mysql(Error::Server(err))\n                    if [1062, 1213].contains(&err.code)\n                        && retry_count < MAX_COMMIT_ATTEMPTS\n                        && start.elapsed() < MAX_COMMIT_TIME => {}\n                /*CommitError::Retry => {\n                    if retry_count > MAX_COMMIT_ATTEMPTS || start.elapsed() > MAX_COMMIT_TIME {\n                        return Err(trc::StoreEvent::AssertValueFailed\n                            .into_err()\n                            .caused_by(trc::location!()));\n                    }\n                }*/\n                CommitError::Mysql(err) => {\n                    return Err(into_error(err));\n                }\n                CommitError::Internal(err) => {\n                    return Err(err);\n                }\n            }\n\n            let backoff = rand::rng().random_range(50..=300);\n            tokio::time::sleep(Duration::from_millis(backoff)).await;\n            retry_count += 1;\n        }\n    }\n\n    async fn write_trx(\n        &self,\n        conn: &mut Conn,\n        batch: &mut Batch<'_>,\n    ) -> Result<AssignedIds, CommitError> {\n        let has_changes = !batch.changes.is_empty();\n        let mut account_id = u32::MAX;\n        let mut collection = u8::MAX;\n        let mut document_id = u32::MAX;\n        let mut change_id = 0u64;\n        let mut asserted_values = AHashMap::new();\n        let mut tx_opts = TxOpts::default();\n        tx_opts\n            .with_consistent_snapshot(false)\n            .with_isolation_level(IsolationLevel::ReadCommitted);\n        let mut trx = conn.start_transaction(tx_opts).await?;\n        let mut result = AssignedIds::default();\n\n        if has_changes {\n            for &account_id in batch.changes.keys() {\n                let key = ValueClass::ChangeId.serialize(account_id, 0, 0, 0);\n                let s = trx\n                    .prep(concat!(\n                        \"INSERT INTO n (k, v) VALUES (:k, LAST_INSERT_ID(1)) \",\n                        \"ON DUPLICATE KEY UPDATE v = LAST_INSERT_ID(v + 1)\"\n                    ))\n                    .await?;\n                trx.exec_drop(&s, params! {\"k\" => key}).await?;\n                let s = trx.prep(\"SELECT LAST_INSERT_ID()\").await?;\n                let change_id = trx.exec_first::<i64, _, _>(&s, ()).await?.ok_or_else(|| {\n                    mysql_async::Error::Io(mysql_async::IoError::Io(std::io::Error::other(\n                        \"LAST_INSERT_ID() did not return a value\",\n                    )))\n                })?;\n                result.push_change_id(account_id, change_id as u64);\n            }\n        }\n\n        for op in batch.ops.iter_mut() {\n            match op {\n                Operation::AccountId {\n                    account_id: account_id_,\n                } => {\n                    account_id = *account_id_;\n                    if has_changes {\n                        change_id = result.set_current_change_id(account_id)?;\n                    }\n                }\n                Operation::Collection {\n                    collection: collection_,\n                } => {\n                    collection = u8::from(*collection_);\n                }\n                Operation::DocumentId {\n                    document_id: document_id_,\n                } => {\n                    document_id = *document_id_;\n                }\n                Operation::Value { class, op } => {\n                    let key = class.serialize(account_id, collection, document_id, 0);\n                    let table = char::from(class.subspace(collection));\n\n                    match op {\n                        ValueOp::Set(value) => {\n                            let exists = asserted_values.get(&key);\n                            let s = if let Some(exists) = exists {\n                                if *exists {\n                                    trx.prep(format!(\"UPDATE {} SET v = :v WHERE k = :k\", table))\n                                        .await?\n                                } else {\n                                    trx.prep(format!(\n                                        \"INSERT INTO {} (k, v) VALUES (:k, :v)\",\n                                        table\n                                    ))\n                                    .await?\n                                }\n                            } else {\n                                trx\n                            .prep(\n                                format!(\"INSERT INTO {} (k, v) VALUES (:k, :v) ON DUPLICATE KEY UPDATE v = VALUES(v)\", table),\n                            )\n                            .await?\n                            };\n\n                            match trx\n                                .exec_drop(&s, params! {\"k\" => key, \"v\" => &*value})\n                                .await\n                            {\n                                Ok(_) => {\n                                    if trx.affected_rows() == 0 {\n                                        trx.rollback().await?;\n                                        return Err(trc::StoreEvent::AssertValueFailed\n                                            .into_err()\n                                            .caused_by(trc::location!())\n                                            .into());\n                                    }\n                                }\n                                Err(err) => {\n                                    trx.rollback().await?;\n                                    return Err(err.into());\n                                }\n                            }\n                        }\n                        ValueOp::SetFnc(set_op) => {\n                            let value = (set_op.fnc)(&set_op.params, &result)?;\n                            let exists = asserted_values.get(&key);\n                            let s = if let Some(exists) = exists {\n                                if *exists {\n                                    trx.prep(format!(\"UPDATE {} SET v = :v WHERE k = :k\", table))\n                                        .await?\n                                } else {\n                                    trx.prep(format!(\n                                        \"INSERT INTO {} (k, v) VALUES (:k, :v)\",\n                                        table\n                                    ))\n                                    .await?\n                                }\n                            } else {\n                                trx\n                            .prep(\n                                format!(\"INSERT INTO {} (k, v) VALUES (:k, :v) ON DUPLICATE KEY UPDATE v = VALUES(v)\", table),\n                            )\n                            .await?\n                            };\n\n                            match trx.exec_drop(&s, params! {\"k\" => key, \"v\" => &value}).await {\n                                Ok(_) => {\n                                    if trx.affected_rows() == 0 {\n                                        trx.rollback().await?;\n                                        return Err(trc::StoreEvent::AssertValueFailed\n                                            .into_err()\n                                            .caused_by(trc::location!())\n                                            .into());\n                                    }\n                                }\n                                Err(err) => {\n                                    trx.rollback().await?;\n                                    return Err(err.into());\n                                }\n                            }\n                        }\n                        ValueOp::MergeFnc(merge_op) => {\n                            let s = trx\n                                .prep(format!(\"SELECT v FROM {} WHERE k = ? FOR UPDATE\", table))\n                                .await?;\n                            let (exists, merge_result) = trx\n                                .exec_first::<Vec<u8>, _, _>(&s, (&key,))\n                                .await?\n                                .map(|bytes| {\n                                    (merge_op.fnc)(&merge_op.params, &result, Some(bytes.as_ref()))\n                                        .map(|v| (true, v))\n                                        .map_err(CommitError::from)\n                                })\n                                .unwrap_or_else(|| {\n                                    (merge_op.fnc)(&merge_op.params, &result, None)\n                                        .map(|v| (false, v))\n                                        .map_err(CommitError::from)\n                                })?;\n\n                            let s = if exists {\n                                trx.prep(format!(\"UPDATE {} SET v = :v WHERE k = :k\", table))\n                                    .await?\n                            } else {\n                                trx.prep(format!(\"INSERT INTO {} (k, v) VALUES (:k, :v)\", table))\n                                    .await?\n                            };\n\n                            match merge_result {\n                                MergeResult::Update(value) => {\n                                    if let Err(err) =\n                                        trx.exec_drop(&s, params! {\"k\" => key, \"v\" => &value}).await\n                                    {\n                                        trx.rollback().await?;\n                                        return Err(err.into());\n                                    }\n                                }\n                                MergeResult::Delete if exists => {\n                                    // Update asserted value\n                                    if let Some(exists) = asserted_values.get_mut(&key) {\n                                        *exists = false;\n                                    }\n\n                                    let s = trx\n                                        .prep(format!(\"DELETE FROM {} WHERE k = ?\", table))\n                                        .await?;\n                                    trx.exec_drop(&s, (key,)).await?;\n                                }\n                                _ => (),\n                            }\n                        }\n\n                        ValueOp::AtomicAdd(by) => {\n                            if *by >= 0 {\n                                let s = trx\n                                    .prep(format!(\n                                        concat!(\n                                            \"INSERT INTO {} (k, v) VALUES (?, ?) \",\n                                            \"ON DUPLICATE KEY UPDATE v = v + VALUES(v)\"\n                                        ),\n                                        table\n                                    ))\n                                    .await?;\n                                trx.exec_drop(&s, (key, &*by)).await?;\n                            } else {\n                                let s = trx\n                                    .prep(format!(\"UPDATE {table} SET v = v + ? WHERE k = ?\"))\n                                    .await?;\n                                trx.exec_drop(&s, (&*by, key)).await?;\n                            }\n                        }\n                        ValueOp::AddAndGet(by) => {\n                            let s = trx\n                                .prep(format!(\n                                    concat!(\n                                        \"INSERT INTO {} (k, v) VALUES (:k, LAST_INSERT_ID(:v)) \",\n                                        \"ON DUPLICATE KEY UPDATE v = LAST_INSERT_ID(v + :v)\"\n                                    ),\n                                    table\n                                ))\n                                .await?;\n                            trx.exec_drop(&s, params! {\"k\" => key, \"v\" => &*by}).await?;\n                            let s = trx.prep(\"SELECT LAST_INSERT_ID()\").await?;\n                            result.push_counter_id(\n                                trx.exec_first::<i64, _, _>(&s, ()).await?.ok_or_else(|| {\n                                    mysql_async::Error::Io(mysql_async::IoError::Io(\n                                        std::io::Error::other(\n                                            \"LAST_INSERT_ID() did not return a value\",\n                                        ),\n                                    ))\n                                })?,\n                            );\n                        }\n                        ValueOp::Clear => {\n                            // Update asserted value\n                            if let Some(exists) = asserted_values.get_mut(&key) {\n                                *exists = false;\n                            }\n\n                            let s = trx\n                                .prep(format!(\"DELETE FROM {} WHERE k = ?\", table))\n                                .await?;\n                            trx.exec_drop(&s, (key,)).await?;\n                        }\n                    }\n                }\n                Operation::Index { field, key, set } => {\n                    let key = IndexKey {\n                        account_id,\n                        collection,\n                        document_id,\n                        field: *field,\n                        key: &*key,\n                    }\n                    .serialize(0);\n\n                    let s = if *set {\n                        trx.prep(\"INSERT IGNORE INTO i (k) VALUES (?)\").await?\n                    } else {\n                        trx.prep(\"DELETE FROM i WHERE k = ?\").await?\n                    };\n                    trx.exec_drop(&s, (key,)).await?;\n                }\n                Operation::Log { collection, set } => {\n                    let key = LogKey {\n                        account_id,\n                        collection: u8::from(*collection),\n                        change_id,\n                    }\n                    .serialize(0);\n\n                    let s = trx\n                        .prep(\"INSERT INTO l (k, v) VALUES (?, ?) ON DUPLICATE KEY UPDATE v = VALUES(v)\")\n                        .await?;\n\n                    trx.exec_drop(&s, (key, &*set)).await?;\n                }\n                Operation::AssertValue {\n                    class,\n                    assert_value,\n                } => {\n                    let key = class.serialize(account_id, collection, document_id, 0);\n                    let table = char::from(class.subspace(collection));\n\n                    let s = trx\n                        .prep(format!(\"SELECT v FROM {} WHERE k = ? FOR UPDATE\", table))\n                        .await?;\n                    let (exists, matches) = trx\n                        .exec_first::<Vec<u8>, _, _>(&s, (&key,))\n                        .await?\n                        .map(|bytes| (true, assert_value.matches(&bytes)))\n                        .unwrap_or_else(|| (false, assert_value.is_none()));\n                    if !matches {\n                        trx.rollback().await?;\n                        return Err(trc::StoreEvent::AssertValueFailed\n                            .into_err()\n                            .caused_by(trc::location!())\n                            .into());\n                    }\n                    asserted_values.insert(key, exists);\n                }\n            }\n        }\n\n        trx.commit().await.map(|_| result).map_err(Into::into)\n    }\n\n    pub(crate) async fn purge_store(&self) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n        for subspace in [SUBSPACE_QUOTA, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER] {\n            let s = conn\n                .prep(format!(\"DELETE FROM {} WHERE v = 0\", char::from(subspace),))\n                .await\n                .map_err(into_error)?;\n            conn.exec_drop(&s, ()).await.map_err(into_error)?;\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn delete_range(&self, from: impl Key, to: impl Key) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;\n\n        let s = conn\n            .prep(format!(\n                \"DELETE FROM {} WHERE k >= ? AND k < ?\",\n                char::from(from.subspace()),\n            ))\n            .await\n            .map_err(into_error)?;\n        conn.exec_drop(&s, (&from.serialize(0), &to.serialize(0)))\n            .await\n            .map_err(into_error)\n    }\n}\n\nimpl From<trc::Error> for CommitError {\n    fn from(err: trc::Error) -> Self {\n        CommitError::Internal(err)\n    }\n}\n\nimpl From<mysql_async::Error> for CommitError {\n    fn from(err: mysql_async::Error) -> Self {\n        CommitError::Mysql(err)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/nats/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse async_nats::Client;\nuse utils::config::{Config, utils::AsKey};\n\npub mod pubsub;\n\n#[derive(Debug)]\npub struct NatsPubSub {\n    client: Client,\n}\n\nimpl NatsPubSub {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let urls = config\n            .values((&prefix, \"address\"))\n            .map(|(_, v)| v.to_string())\n            .collect::<Vec<_>>();\n        if urls.is_empty() {\n            config.new_build_error((&prefix, \"address\"), \"No Nats addresses specified\");\n            return None;\n        }\n\n        let mut opts = async_nats::ConnectOptions::new()\n            .max_reconnects(\n                config\n                    .property_or_default::<Option<usize>>((&prefix, \"max-reconnects\"), \"false\")\n                    .unwrap_or_default(),\n            )\n            .connection_timeout(\n                config\n                    .property_or_default((&prefix, \"timeout.connection\"), \"5s\")\n                    .unwrap_or_else(|| Duration::from_secs(5)),\n            )\n            .request_timeout(\n                config\n                    .property_or_default::<Option<Duration>>((&prefix, \"timeout.request\"), \"10s\")\n                    .unwrap_or_else(|| Some(Duration::from_secs(10))),\n            )\n            .ping_interval(\n                config\n                    .property_or_default((&prefix, \"ping-interval\"), \"60s\")\n                    .unwrap_or_else(|| Duration::from_secs(5)),\n            )\n            .client_capacity(\n                config\n                    .property_or_default((&prefix, \"capacity.client\"), \"2048\")\n                    .unwrap_or(2048),\n            )\n            .subscription_capacity(\n                config\n                    .property_or_default((&prefix, \"capacity.subscription\"), \"65536\")\n                    .unwrap_or(65536),\n            )\n            .read_buffer_capacity(\n                config\n                    .property_or_default((&prefix, \"capacity.read-buffer\"), \"65535\")\n                    .unwrap_or(65535),\n            )\n            .require_tls(\n                config\n                    .property_or_default((&prefix, \"tls.enable\"), \"false\")\n                    .unwrap_or_default(),\n            );\n\n        if config\n            .property_or_default((&prefix, \"no-echo\"), \"true\")\n            .unwrap_or(true)\n        {\n            opts = opts.no_echo();\n        }\n\n        if let (Some(user), Some(pass)) = (\n            config.value((&prefix, \"user\")),\n            config.value((&prefix, \"password\")),\n        ) {\n            opts = opts.user_and_password(user.to_string(), pass.to_string());\n        } else if let Some(credentials) = config.value((&prefix, \"credentials\")) {\n            opts = opts\n                .credentials(credentials)\n                .map_err(|err| {\n                    config.new_build_error(\n                        (&prefix, \"credentials\"),\n                        format!(\"Failed to parse Nats credentials: {}\", err),\n                    );\n                })\n                .ok()?;\n        }\n\n        async_nats::connect_with_options(urls, opts)\n            .await\n            .map_err(|err| {\n                config.new_build_error(\n                    (&prefix, \"urls\"),\n                    format!(\"Failed to connect to Nats: {}\", err),\n                );\n            })\n            .map(|client| NatsPubSub { client })\n            .ok()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/nats/pubsub.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::NatsPubSub;\nuse crate::dispatch::pubsub::{Msg, PubSubStream};\nuse futures::StreamExt;\nuse trc::{ClusterEvent, Error, EventType};\n\npub struct NatsPubSubStream {\n    subs: async_nats::Subscriber,\n}\n\nimpl NatsPubSub {\n    pub async fn publish(&self, topic: &'static str, message: Vec<u8>) -> trc::Result<()> {\n        self.client\n            .publish(topic, message.into())\n            .await\n            .map_err(|err| Error::new(EventType::Cluster(ClusterEvent::PublisherError)).reason(err))\n    }\n\n    pub async fn subscribe(&self, topic: &'static str) -> trc::Result<PubSubStream> {\n        self.client\n            .subscribe(topic)\n            .await\n            .map(|subs| PubSubStream::Nats(NatsPubSubStream { subs }))\n            .map_err(|err| {\n                Error::new(EventType::Cluster(ClusterEvent::SubscriberError)).reason(err)\n            })\n    }\n}\n\nimpl NatsPubSubStream {\n    pub async fn next(&mut self) -> Option<Msg> {\n        self.subs.next().await.map(Msg::Nats)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::ops::Range;\n\nuse crate::backend::postgres::into_pool_error;\n\nuse super::{PostgresStore, into_error};\n\nimpl PostgresStore {\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn\n            .prepare_cached(\"SELECT v FROM t WHERE k = $1\")\n            .await\n            .map_err(into_error)?;\n        conn.query_opt(&s, &[&key])\n            .await\n            .and_then(|row| {\n                if let Some(row) = row {\n                    Ok(Some(if range.start == 0 && range.end == usize::MAX {\n                        row.try_get::<_, Vec<u8>>(0)?\n                    } else {\n                        let bytes = row.try_get::<_, &[u8]>(0)?;\n                        bytes\n                            .get(range.start..std::cmp::min(bytes.len(), range.end))\n                            .unwrap_or_default()\n                            .to_vec()\n                    }))\n                } else {\n                    Ok(None)\n                }\n            })\n            .map_err(into_error)\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn\n            .prepare_cached(\n                \"INSERT INTO t (k, v) VALUES ($1, $2) ON CONFLICT (k) DO UPDATE SET v = EXCLUDED.v\",\n            )\n            .await\n            .map_err(into_error)?;\n        conn.execute(&s, &[&key, &data])\n            .await\n            .map_err(into_error)\n            .map(|_| ())\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn\n            .prepare_cached(\"DELETE FROM t WHERE k = $1\")\n            .await\n            .map_err(into_error)?;\n        conn.execute(&s, &[&key])\n            .await\n            .map_err(into_error)\n            .map(|hits| hits > 0)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{QueryResult, QueryType, backend::postgres::into_pool_error};\n\nuse bytes::BytesMut;\nuse futures::{TryStreamExt, pin_mut};\nuse tokio_postgres::types::{FromSql, ToSql, Type};\n\nuse crate::IntoRows;\n\nuse super::{PostgresStore, into_error};\n\nimpl PostgresStore {\n    pub(crate) async fn sql_query<T: QueryResult>(\n        &self,\n        query: &str,\n        params_: &[crate::Value<'_>],\n    ) -> trc::Result<T> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn.prepare_cached(query).await.map_err(into_error)?;\n        let params = params_\n            .iter()\n            .map(|v| v as &(dyn tokio_postgres::types::ToSql + Sync))\n            .collect::<Vec<_>>();\n\n        match T::query_type() {\n            QueryType::Execute => conn\n                .execute(&s, params.as_slice())\n                .await\n                .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_exec(r as usize))),\n            QueryType::Exists => {\n                let rows = conn\n                    .query_raw(&s, params.into_iter())\n                    .await\n                    .map_err(into_error)?;\n                pin_mut!(rows);\n                rows.try_next()\n                    .await\n                    .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_exists(r.is_some())))\n            }\n            QueryType::QueryOne => conn\n                .query_opt(&s, params.as_slice())\n                .await\n                .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_query_one(r))),\n            QueryType::QueryAll => conn\n                .query(&s, params.as_slice())\n                .await\n                .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_query_all(r))),\n        }\n    }\n}\n\nimpl ToSql for crate::Value<'_> {\n    fn to_sql(\n        &self,\n        ty: &tokio_postgres::types::Type,\n        out: &mut BytesMut,\n    ) -> Result<tokio_postgres::types::IsNull, Box<dyn std::error::Error + Sync + Send>>\n    where\n        Self: Sized,\n    {\n        match self {\n            crate::Value::Integer(v) => match *ty {\n                Type::CHAR => (*v as i8).to_sql(ty, out),\n                Type::INT2 => (*v as i16).to_sql(ty, out),\n                Type::INT4 => (*v as i32).to_sql(ty, out),\n                _ => v.to_sql(ty, out),\n            },\n            crate::Value::Bool(v) => v.to_sql(ty, out),\n            crate::Value::Float(v) => {\n                if matches!(ty, &Type::FLOAT4) {\n                    (*v as f32).to_sql(ty, out)\n                } else {\n                    v.to_sql(ty, out)\n                }\n            }\n            crate::Value::Text(v) => v.to_sql(ty, out),\n            crate::Value::Blob(v) => v.to_sql(ty, out),\n            crate::Value::Null => None::<String>.to_sql(ty, out),\n        }\n    }\n\n    fn accepts(_: &tokio_postgres::types::Type) -> bool\n    where\n        Self: Sized,\n    {\n        true\n    }\n\n    fn to_sql_checked(\n        &self,\n        ty: &tokio_postgres::types::Type,\n        out: &mut BytesMut,\n    ) -> Result<tokio_postgres::types::IsNull, Box<dyn std::error::Error + Sync + Send>> {\n        match self {\n            crate::Value::Integer(v) => match *ty {\n                Type::CHAR => (*v as i8).to_sql_checked(ty, out),\n                Type::INT2 => (*v as i16).to_sql_checked(ty, out),\n                Type::INT4 => (*v as i32).to_sql_checked(ty, out),\n                _ => v.to_sql_checked(ty, out),\n            },\n            crate::Value::Bool(v) => v.to_sql_checked(ty, out),\n            crate::Value::Float(v) => {\n                if matches!(ty, &Type::FLOAT4) {\n                    (*v as f32).to_sql_checked(ty, out)\n                } else {\n                    v.to_sql_checked(ty, out)\n                }\n            }\n            crate::Value::Text(v) => v.to_sql_checked(ty, out),\n            crate::Value::Blob(v) => v.to_sql_checked(ty, out),\n            crate::Value::Null => None::<String>.to_sql_checked(ty, out),\n        }\n    }\n}\n\nimpl IntoRows for Vec<tokio_postgres::Row> {\n    fn into_rows(self) -> crate::Rows {\n        crate::Rows {\n            rows: self\n                .into_iter()\n                .map(|r| crate::Row {\n                    values: (0..r.len())\n                        .map(|idx| r.try_get(idx).unwrap_or(crate::Value::Null))\n                        .collect(),\n                })\n                .collect(),\n        }\n    }\n\n    fn into_named_rows(self) -> crate::NamedRows {\n        crate::NamedRows {\n            names: self\n                .first()\n                .map(|r| r.columns().iter().map(|c| c.name().to_string()).collect())\n                .unwrap_or_default(),\n            rows: self\n                .into_iter()\n                .map(|r| crate::Row {\n                    values: (0..r.len())\n                        .map(|idx| r.try_get(idx).unwrap_or(crate::Value::Null))\n                        .collect(),\n                })\n                .collect(),\n        }\n    }\n\n    fn into_row(self) -> Option<crate::Row> {\n        unreachable!()\n    }\n}\n\nimpl IntoRows for Option<tokio_postgres::Row> {\n    fn into_row(self) -> Option<crate::Row> {\n        self.map(|row| crate::Row {\n            values: (0..row.len())\n                .map(|idx| row.try_get(idx).unwrap_or(crate::Value::Null))\n                .collect(),\n        })\n    }\n\n    fn into_rows(self) -> crate::Rows {\n        unreachable!()\n    }\n\n    fn into_named_rows(self) -> crate::NamedRows {\n        unreachable!()\n    }\n}\n\nimpl FromSql<'_> for crate::Value<'static> {\n    fn from_sql(\n        ty: &tokio_postgres::types::Type,\n        raw: &'_ [u8],\n    ) -> Result<Self, Box<dyn std::error::Error + Sync + Send>> {\n        match ty {\n            &Type::VARCHAR | &Type::TEXT | &Type::BPCHAR | &Type::NAME | &Type::UNKNOWN => {\n                String::from_sql(ty, raw).map(|s| crate::Value::Text(s.into()))\n            }\n            &Type::BOOL => bool::from_sql(ty, raw).map(crate::Value::Bool),\n            &Type::CHAR => i8::from_sql(ty, raw).map(|v| crate::Value::Integer(v as i64)),\n            &Type::INT2 => i16::from_sql(ty, raw).map(|v| crate::Value::Integer(v as i64)),\n            &Type::INT4 => i32::from_sql(ty, raw).map(|v| crate::Value::Integer(v as i64)),\n            &Type::INT8 | &Type::OID => i64::from_sql(ty, raw).map(crate::Value::Integer),\n            &Type::FLOAT4 | &Type::FLOAT8 => f64::from_sql(ty, raw).map(crate::Value::Float),\n            ty if (ty.name() == \"citext\"\n                || ty.name() == \"ltree\"\n                || ty.name() == \"lquery\"\n                || ty.name() == \"ltxtquery\") =>\n            {\n                String::from_sql(ty, raw).map(|s| crate::Value::Text(s.into()))\n            }\n            _ => Vec::<u8>::from_sql(ty, raw).map(|b| crate::Value::Blob(b.into())),\n        }\n    }\n\n    fn accepts(_: &tokio_postgres::types::Type) -> bool {\n        true\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{PostgresStore, into_error};\nuse crate::{\n    backend::postgres::{PsqlSearchField, into_pool_error, tls::MakeRustlsConnect},\n    search::{\n        CalendarSearchField, ContactSearchField, EmailSearchField, SearchableField,\n        TracingSearchField,\n    },\n    *,\n};\nuse deadpool::managed::Object;\nuse deadpool_postgres::{Config, Manager, ManagerConfig, PoolConfig, RecyclingMethod, Runtime};\nuse nlp::language::Language;\nuse std::time::Duration;\nuse tokio_postgres::NoTls;\nuse utils::{config::utils::AsKey, rustls_client_config};\n\nimpl PostgresStore {\n    pub async fn open(\n        config: &mut utils::config::Config,\n        prefix: impl AsKey,\n        create_store_tables: bool,\n        create_search_tables: bool,\n    ) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let mut cfg = Config::new();\n        cfg.dbname = config\n            .value_require((&prefix, \"database\"))?\n            .to_string()\n            .into();\n        cfg.host = config.value((&prefix, \"host\")).map(|s| s.to_string());\n        cfg.user = config.value((&prefix, \"user\")).map(|s| s.to_string());\n        cfg.password = config.value((&prefix, \"password\")).map(|s| s.to_string());\n        cfg.port = config.property((&prefix, \"port\"));\n        cfg.connect_timeout = config\n            .property::<Option<Duration>>((&prefix, \"timeout\"))\n            .unwrap_or_default();\n        cfg.options = config.value((&prefix, \"options\")).map(|s| s.to_string());\n        cfg.manager = Some(ManagerConfig {\n            recycling_method: RecyclingMethod::Clean,\n        });\n        if let Some(max_conn) = config.property::<usize>((&prefix, \"pool.max-connections\")) {\n            cfg.pool = PoolConfig::new(max_conn).into();\n        }\n        let mut db = Self {\n            conn_pool: if config\n                .property_or_default::<bool>((&prefix, \"tls.enable\"), \"false\")\n                .unwrap_or_default()\n            {\n                cfg.create_pool(\n                    Some(Runtime::Tokio1),\n                    MakeRustlsConnect::new(rustls_client_config(\n                        config\n                            .property_or_default((&prefix, \"tls.allow-invalid-certs\"), \"false\")\n                            .unwrap_or_default(),\n                    )),\n                )\n            } else {\n                cfg.create_pool(Some(Runtime::Tokio1), NoTls)\n            }\n            .map_err(|e| {\n                config.new_build_error(\n                    prefix.as_str(),\n                    format!(\"Failed to create connection pool: {e}\"),\n                )\n            })\n            .ok()?,\n            languages: config\n                .properties::<Language>((&prefix, \"languages\"))\n                .into_iter()\n                .map(|(_, v)| v)\n                .collect(),\n        };\n\n        if db.languages.is_empty() {\n            db.languages.insert(Language::English);\n        }\n\n        if create_store_tables && let Err(err) = db.create_storage_tables().await {\n            config.new_build_error(prefix.as_str(), format!(\"Failed to create tables: {err}\"));\n        }\n\n        if create_search_tables && let Err(err) = db.create_search_tables().await {\n            config.new_build_warning(\n                prefix.as_str(),\n                format!(\"Failed to create search tables: {err}\"),\n            );\n        }\n\n        Some(db)\n    }\n\n    pub(crate) async fn create_storage_tables(&self) -> trc::Result<()> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n\n        for table in [\n            SUBSPACE_ACL,\n            SUBSPACE_DIRECTORY,\n            SUBSPACE_TASK_QUEUE,\n            SUBSPACE_BLOB_EXTRA,\n            SUBSPACE_BLOB_LINK,\n            SUBSPACE_IN_MEMORY_VALUE,\n            SUBSPACE_PROPERTY,\n            SUBSPACE_SETTINGS,\n            SUBSPACE_QUEUE_MESSAGE,\n            SUBSPACE_QUEUE_EVENT,\n            SUBSPACE_REPORT_OUT,\n            SUBSPACE_REPORT_IN,\n            SUBSPACE_LOGS,\n            SUBSPACE_BLOBS,\n            SUBSPACE_TELEMETRY_SPAN,\n            SUBSPACE_TELEMETRY_METRIC,\n        ] {\n            let table = char::from(table);\n            conn.execute(\n                &format!(\n                    \"CREATE TABLE IF NOT EXISTS {table} (\n                        k BYTEA PRIMARY KEY,\n                        v BYTEA NOT NULL\n                    )\"\n                ),\n                &[],\n            )\n            .await\n            .map_err(into_error)?;\n        }\n\n        for table in [SUBSPACE_INDEXES] {\n            let table = char::from(table);\n            conn.execute(\n                &format!(\n                    \"CREATE TABLE IF NOT EXISTS {table} (\n                        k BYTEA PRIMARY KEY\n                    )\"\n                ),\n                &[],\n            )\n            .await\n            .map_err(into_error)?;\n        }\n\n        for table in [SUBSPACE_COUNTER, SUBSPACE_QUOTA, SUBSPACE_IN_MEMORY_COUNTER] {\n            conn.execute(\n                &format!(\n                    \"CREATE TABLE IF NOT EXISTS {} (\n                    k BYTEA PRIMARY KEY,\n                    v BIGINT NOT NULL DEFAULT 0\n                )\",\n                    char::from(table)\n                ),\n                &[],\n            )\n            .await\n            .map_err(into_error)?;\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn create_search_tables(&self) -> trc::Result<()> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n\n        create_search_tables::<EmailSearchField>(&conn).await?;\n        create_search_tables::<CalendarSearchField>(&conn).await?;\n        create_search_tables::<ContactSearchField>(&conn).await?;\n        //create_search_tables::<FileSearchField>(&conn).await?;\n        create_search_tables::<TracingSearchField>(&conn).await?;\n\n        Ok(())\n    }\n}\n\nasync fn create_search_tables<T: SearchableField + PsqlSearchField + 'static>(\n    conn: &Object<Manager>,\n) -> trc::Result<()> {\n    let table_name = T::index().psql_table();\n    let mut query = format!(\"CREATE TABLE IF NOT EXISTS {} (\", table_name);\n\n    // Add primary key columns\n    let pkeys = T::primary_keys();\n    for pkey in pkeys {\n        query.push_str(&format!(\"{} {}, \", pkey.column(), pkey.column_type()));\n    }\n\n    // Add other columns\n    for field in T::all_fields() {\n        query.push_str(&format!(\"{} {}\", field.column(), field.column_type()));\n        if let Some(sort_type) = field.sort_column_type() {\n            query.push_str(&format!(\", {} {}\", field.sort_column().unwrap(), sort_type));\n        }\n        query.push_str(\", \");\n    }\n\n    // Add primary key constraint\n    query.push_str(\"PRIMARY KEY (\");\n    for (i, pkey) in pkeys.iter().enumerate() {\n        if i > 0 {\n            query.push_str(\", \");\n        }\n        query.push_str(pkey.column());\n    }\n    query.push_str(\"))\");\n\n    conn.execute(&query, &[]).await.map_err(into_error)?;\n\n    // Create indexes\n    for field in T::all_fields() {\n        if field.is_text() || field.is_json() {\n            let column_name = field.column();\n            let create_index_query = format!(\n                \"CREATE INDEX IF NOT EXISTS gin_{table_name}_{column_name} ON {table_name} USING GIN({column_name})\",\n            );\n            conn.execute(&create_index_query, &[])\n                .await\n                .map_err(into_error)?;\n        }\n\n        if field.is_indexed() {\n            let column_name = field.sort_column().unwrap_or(field.column());\n            let create_index_query = format!(\n                \"CREATE INDEX IF NOT EXISTS idx_{table_name}_{column_name} ON {table_name}({column_name})\",\n            );\n            conn.execute(&create_index_query, &[])\n                .await\n                .map_err(into_error)?;\n        }\n    }\n\n    Ok(())\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    search::{\n        CalendarSearchField, ContactSearchField, EmailSearchField, FileSearchField, SearchField,\n        TracingSearchField,\n    },\n    write::SearchIndex,\n};\nuse ahash::AHashSet;\nuse deadpool_postgres::Pool;\nuse nlp::language::Language;\n\npub mod blob;\npub mod lookup;\npub mod main;\npub mod read;\npub mod search;\npub mod tls;\npub mod write;\n\npub struct PostgresStore {\n    pub(crate) conn_pool: Pool,\n    pub(crate) languages: AHashSet<Language>,\n}\n\n#[inline(always)]\nfn into_error(err: tokio_postgres::error::Error) -> trc::Error {\n    let mut local_err = trc::StoreEvent::PostgresqlError.reason(err.to_string());\n    if let Some(db_err) = err.as_db_error() {\n        local_err = local_err.code(db_err.code().code().to_string());\n        if let Some(detail) = db_err.detail() {\n            local_err = local_err.details(detail.to_string());\n        }\n\n        if let Some(hint) = db_err.hint() {\n            local_err = local_err.caused_by(hint.to_string());\n        }\n    }\n    local_err\n}\n\n#[inline(always)]\nfn into_pool_error(err: deadpool::managed::PoolError<tokio_postgres::Error>) -> trc::Error {\n    trc::StoreEvent::PostgresqlError.reason(err)\n}\n\nimpl SearchIndex {\n    pub fn psql_table(&self) -> &'static str {\n        match self {\n            SearchIndex::Email => \"s_email\",\n            SearchIndex::Calendar => \"s_cal\",\n            SearchIndex::Contacts => \"s_card\",\n            SearchIndex::File => \"s_file\",\n            SearchIndex::Tracing => \"s_trace\",\n            SearchIndex::InMemory => \"\",\n        }\n    }\n}\n\ntrait PsqlSearchField {\n    fn column(&self) -> &'static str;\n    fn column_type(&self) -> &'static str;\n    fn sort_column_type(&self) -> Option<&'static str>;\n    fn sort_column(&self) -> Option<&'static str>;\n}\n\nimpl PsqlSearchField for EmailSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            EmailSearchField::From => \"fadr\",\n            EmailSearchField::To => \"tadr\",\n            EmailSearchField::Cc => \"cc\",\n            EmailSearchField::Bcc => \"bcc\",\n            EmailSearchField::Subject => \"subj\",\n            EmailSearchField::Body => \"body\",\n            EmailSearchField::Attachment => \"atta\",\n            EmailSearchField::ReceivedAt => \"rcvd\",\n            EmailSearchField::SentAt => \"sent\",\n            EmailSearchField::Size => \"size\",\n            EmailSearchField::HasAttachment => \"hatt\",\n            EmailSearchField::Headers => \"hdrs\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            EmailSearchField::ReceivedAt | EmailSearchField::SentAt => \"BIGINT\",\n            EmailSearchField::Size => \"INTEGER\",\n            EmailSearchField::HasAttachment => \"BOOLEAN\",\n            EmailSearchField::Headers => \"JSONB\",\n            _ => \"TSVECTOR\",\n        }\n    }\n\n    fn sort_column_type(&self) -> Option<&'static str> {\n        match self {\n            EmailSearchField::From | EmailSearchField::To | EmailSearchField::Subject => {\n                Some(\"TEXT\")\n            }\n            #[cfg(feature = \"test_mode\")]\n            EmailSearchField::Cc | EmailSearchField::Bcc => Some(\"TEXT\"),\n            _ => None,\n        }\n    }\n\n    fn sort_column(&self) -> Option<&'static str> {\n        match self {\n            EmailSearchField::From => Some(\"s_fr\"),\n            EmailSearchField::To => Some(\"s_to\"),\n            EmailSearchField::Subject => Some(\"s_sj\"),\n            #[cfg(feature = \"test_mode\")]\n            EmailSearchField::Bcc => Some(\"s_bc\"),\n            #[cfg(feature = \"test_mode\")]\n            EmailSearchField::Cc => Some(\"s_cc\"),\n            _ => None,\n        }\n    }\n}\n\nimpl PsqlSearchField for CalendarSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            CalendarSearchField::Title => \"titl\",\n            CalendarSearchField::Description => \"dscd\",\n            CalendarSearchField::Location => \"locn\",\n            CalendarSearchField::Owner => \"ownr\",\n            CalendarSearchField::Attendee => \"atnd\",\n            CalendarSearchField::Start => \"strt\",\n            CalendarSearchField::Uid => \"uid\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            CalendarSearchField::Start => \"BIGINT\",\n            CalendarSearchField::Uid => \"TEXT\",\n            _ => \"TSVECTOR\",\n        }\n    }\n\n    fn sort_column_type(&self) -> Option<&'static str> {\n        None\n    }\n\n    fn sort_column(&self) -> Option<&'static str> {\n        None\n    }\n}\n\nimpl PsqlSearchField for ContactSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            ContactSearchField::Member => \"mmbr\",\n            ContactSearchField::Name => \"name\",\n            ContactSearchField::Nickname => \"nick\",\n            ContactSearchField::Organization => \"orgn\",\n            ContactSearchField::Email => \"eml\",\n            ContactSearchField::Phone => \"phon\",\n            ContactSearchField::OnlineService => \"olsv\",\n            ContactSearchField::Address => \"addr\",\n            ContactSearchField::Note => \"note\",\n            ContactSearchField::Kind => \"kind\",\n            ContactSearchField::Uid => \"uid\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            ContactSearchField::Kind | ContactSearchField::Uid => \"TEXT\",\n            _ => \"TSVECTOR\",\n        }\n    }\n\n    fn sort_column_type(&self) -> Option<&'static str> {\n        None\n    }\n\n    fn sort_column(&self) -> Option<&'static str> {\n        None\n    }\n}\n\nimpl PsqlSearchField for FileSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            FileSearchField::Name => \"name\",\n            FileSearchField::Content => \"body\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        \"TSVECTOR\"\n    }\n\n    fn sort_column_type(&self) -> Option<&'static str> {\n        None\n    }\n\n    fn sort_column(&self) -> Option<&'static str> {\n        None\n    }\n}\nimpl PsqlSearchField for TracingSearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            TracingSearchField::QueueId => \"qid\",\n            TracingSearchField::EventType => \"etyp\",\n            TracingSearchField::Keywords => \"kwds\",\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            TracingSearchField::EventType => \"BIGINT\",\n            TracingSearchField::QueueId => \"BIGINT\",\n            TracingSearchField::Keywords => \"TSVECTOR\",\n        }\n    }\n\n    fn sort_column_type(&self) -> Option<&'static str> {\n        None\n    }\n\n    fn sort_column(&self) -> Option<&'static str> {\n        None\n    }\n}\n\nimpl PsqlSearchField for SearchField {\n    fn column(&self) -> &'static str {\n        match self {\n            SearchField::AccountId => \"accid\",\n            SearchField::DocumentId => \"docid\",\n            SearchField::Id => \"id\",\n            SearchField::Email(field) => field.column(),\n            SearchField::Calendar(field) => field.column(),\n            SearchField::Contact(field) => field.column(),\n            SearchField::File(field) => field.column(),\n            SearchField::Tracing(field) => field.column(),\n        }\n    }\n\n    fn column_type(&self) -> &'static str {\n        match self {\n            SearchField::AccountId => \"INTEGER NOT NULL\",\n            SearchField::DocumentId => \"INTEGER NOT NULL\",\n            SearchField::Id => \"BIGINT NOT NULL\",\n            SearchField::Email(field) => field.column_type(),\n            SearchField::Calendar(field) => field.column_type(),\n            SearchField::Contact(field) => field.column_type(),\n            SearchField::File(field) => field.column_type(),\n            SearchField::Tracing(field) => field.column_type(),\n        }\n    }\n\n    fn sort_column_type(&self) -> Option<&'static str> {\n        match self {\n            SearchField::Email(field) => field.sort_column_type(),\n            SearchField::Calendar(field) => field.sort_column_type(),\n            SearchField::Contact(field) => field.sort_column_type(),\n            SearchField::File(field) => field.sort_column_type(),\n            SearchField::Tracing(field) => field.sort_column_type(),\n            SearchField::AccountId | SearchField::DocumentId | SearchField::Id => None,\n        }\n    }\n\n    fn sort_column(&self) -> Option<&'static str> {\n        match self {\n            SearchField::Email(field) => field.sort_column(),\n            SearchField::Calendar(field) => field.sort_column(),\n            SearchField::Contact(field) => field.sort_column(),\n            SearchField::File(field) => field.sort_column(),\n            SearchField::Tracing(field) => field.sort_column(),\n            SearchField::AccountId | SearchField::DocumentId | SearchField::Id => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/read.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{PostgresStore, into_error};\nuse crate::{\n    Deserialize, IterateParams, Key, ValueKey, backend::postgres::into_pool_error,\n    write::ValueClass,\n};\nuse futures::{TryStreamExt, pin_mut};\n\nimpl PostgresStore {\n    pub(crate) async fn get_value<U>(&self, key: impl Key) -> trc::Result<Option<U>>\n    where\n        U: Deserialize + 'static,\n    {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn\n            .prepare_cached(&format!(\n                \"SELECT v FROM {} WHERE k = $1\",\n                char::from(key.subspace())\n            ))\n            .await\n            .map_err(into_error)?;\n        let key = key.serialize(0);\n        conn.query_opt(&s, &[&key])\n            .await\n            .map_err(into_error)\n            .and_then(|r| {\n                if let Some(r) = r {\n                    Ok(Some(U::deserialize(r.get(0))?))\n                } else {\n                    Ok(None)\n                }\n            })\n    }\n\n    pub(crate) async fn iterate<T: Key>(\n        &self,\n        params: IterateParams<T>,\n        mut cb: impl for<'x> FnMut(&'x [u8], &'x [u8]) -> trc::Result<bool> + Sync + Send,\n    ) -> trc::Result<()> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let table = char::from(params.begin.subspace());\n        let begin = params.begin.serialize(0);\n        let end = params.end.serialize(0);\n        let keys = if params.values { \"k, v\" } else { \"k\" };\n\n        let s = conn\n            .prepare_cached(&match (params.first, params.ascending) {\n                (true, true) => {\n                    format!(\n                        \"SELECT {keys} FROM {table} WHERE k >= $1 AND k <= $2 ORDER BY k ASC LIMIT 1\"\n                    )\n                }\n                (true, false) => {\n                    format!(\n                    \"SELECT {keys} FROM {table} WHERE k >= $1 AND k <= $2 ORDER BY k DESC LIMIT 1\"\n                )\n                }\n                (false, true) => {\n                    format!(\"SELECT {keys} FROM {table} WHERE k >= $1 AND k <= $2 ORDER BY k ASC\")\n                }\n                (false, false) => {\n                    format!(\"SELECT {keys} FROM {table} WHERE k >= $1 AND k <= $2 ORDER BY k DESC\")\n                }\n            })\n            .await.map_err(into_error)?;\n        let rows = conn\n            .query_raw(&s, &[&begin, &end])\n            .await\n            .map_err(into_error)?;\n\n        pin_mut!(rows);\n\n        if params.values {\n            while let Some(row) = rows.try_next().await.map_err(into_error)? {\n                let key = row.try_get::<_, &[u8]>(0).map_err(into_error)?;\n                let value = row.try_get::<_, &[u8]>(1).map_err(into_error)?;\n\n                if !cb(key, value)? {\n                    break;\n                }\n            }\n        } else {\n            while let Some(row) = rows.try_next().await.map_err(into_error)? {\n                if !cb(row.try_get::<_, &[u8]>(0).map_err(into_error)?, b\"\")? {\n                    break;\n                }\n            }\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn get_counter(\n        &self,\n        key: impl Into<ValueKey<ValueClass>> + Sync + Send,\n    ) -> trc::Result<i64> {\n        let key = key.into();\n        let table = char::from(key.subspace());\n        let key = key.serialize(0);\n\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn\n            .prepare_cached(&format!(\"SELECT v FROM {table} WHERE k = $1\"))\n            .await\n            .map_err(into_error)?;\n        match conn.query_opt(&s, &[&key]).await {\n            Ok(Some(row)) => row.try_get(0).map_err(into_error),\n            Ok(None) => Ok(0),\n            Err(e) => Err(into_error(e)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    backend::postgres::{PostgresStore, PsqlSearchField, into_error, into_pool_error},\n    search::{\n        IndexDocument, SearchComparator, SearchDocumentId, SearchFilter, SearchOperator,\n        SearchQuery, SearchValue,\n    },\n    write::SearchIndex,\n};\nuse nlp::language::Language;\nuse std::fmt::Write;\nuse tokio_postgres::{\n    IsolationLevel,\n    types::{FromSql, ToSql, Type, WrongType},\n};\n\nimpl PostgresStore {\n    pub async fn index(&self, documents: Vec<IndexDocument>) -> trc::Result<()> {\n        let mut conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let trx = conn\n            .build_transaction()\n            .isolation_level(IsolationLevel::ReadCommitted)\n            .start()\n            .await\n            .map_err(into_error)?;\n\n        for document in documents {\n            let index = document.index;\n            let primary_keys = index.primary_keys();\n            let all_fields = index.all_fields();\n            let fields = document.fields;\n            let mut values = Vec::with_capacity(fields.len() + 2);\n            let mut query = format!(\"INSERT INTO {} (\", index.psql_table());\n\n            for (i, field) in primary_keys.iter().chain(all_fields).enumerate() {\n                if i > 0 {\n                    query.push(',');\n                }\n                query.push_str(field.column());\n\n                if let Some(sort_column) = field.sort_column() {\n                    query.push(',');\n                    query.push_str(sort_column);\n                }\n            }\n\n            query.push_str(\") VALUES (\");\n\n            for (i, field) in primary_keys.iter().chain(all_fields).enumerate() {\n                if i > 0 {\n                    query.push(',');\n                }\n\n                if let Some(value) = fields.get(field) {\n                    let value_ref = format!(\"${}\", values.len() + 1);\n                    let (text_len, language) = if let SearchValue::Text { value, language } = value\n                    {\n                        (\n                            value.len(),\n                            if self.languages.contains(language) {\n                                pg_lang(language).unwrap_or(\"simple\")\n                            } else {\n                                \"simple\"\n                            },\n                        )\n                    } else {\n                        (0, \"simple\")\n                    };\n\n                    if field.is_text() {\n                        let _ = write!(&mut query, \"to_tsvector('{language}',{value_ref})\");\n                    } else if text_len > 512 {\n                        query.push_str(\"left(\");\n                        query.push_str(&value_ref);\n                        query.push_str(\",512)\");\n                    } else {\n                        query.push_str(&value_ref);\n                    }\n\n                    if field.sort_column().is_some() {\n                        if text_len > 255 {\n                            query.push_str(\",left(\");\n                            query.push_str(&value_ref);\n                            query.push_str(\",255)\");\n                        } else {\n                            query.push(',');\n                            query.push_str(&value_ref);\n                        }\n                    }\n\n                    values.push(value as &(dyn ToSql + Sync));\n                } else {\n                    query.push_str(\"NULL\");\n                    if field.sort_column().is_some() {\n                        query.push_str(\",NULL\");\n                    }\n                }\n            }\n\n            query.push_str(\") ON CONFLICT (\");\n            for (i, pkey) in primary_keys.iter().enumerate() {\n                if i > 0 {\n                    query.push(',');\n                }\n                query.push_str(pkey.column());\n            }\n            query.push_str(\") DO UPDATE SET \");\n            for (i, field) in all_fields.iter().enumerate() {\n                if i > 0 {\n                    query.push(',');\n                }\n                let column = field.column();\n                let _ = write!(&mut query, \"{column} = EXCLUDED.{column}\");\n            }\n\n            trx.execute(&query, &values).await.map_err(into_error)?;\n        }\n\n        trx.commit().await.map_err(into_error)\n    }\n\n    pub async fn query<R: SearchDocumentId>(\n        &self,\n        index: SearchIndex,\n        filters: &[SearchFilter],\n        sort: &[SearchComparator],\n    ) -> trc::Result<Vec<R>> {\n        let mut query = format!(\"SELECT {} FROM {}\", R::field().column(), index.psql_table());\n        let params = self.build_filter(&mut query, filters);\n        if !sort.is_empty() {\n            build_sort(&mut query, sort);\n        }\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn.prepare_cached(&query).await.map_err(into_error)?;\n\n        conn.query(&s, params.as_slice())\n            .await\n            .and_then(|rows| {\n                rows.into_iter()\n                    .map(|row| row.try_get::<_, DocId>(0).map(|v| R::from_u64(v.0)))\n                    .collect::<Result<Vec<R>, _>>()\n            })\n            .map_err(into_error)\n    }\n\n    pub async fn unindex(&self, filter: SearchQuery) -> trc::Result<u64> {\n        debug_assert!(!filter.filters.is_empty());\n        let mut query = format!(\"DELETE FROM {} \", filter.index.psql_table());\n        let params = self.build_filter(&mut query, &filter.filters);\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let s = conn.prepare_cached(&query).await.map_err(into_error)?;\n\n        conn.execute(&s, params.as_slice())\n            .await\n            .map_err(into_error)\n    }\n\n    fn build_filter<'x>(\n        &self,\n        query: &mut String,\n        filters: &'x [SearchFilter],\n    ) -> Vec<&'x (dyn ToSql + Sync)> {\n        if filters.is_empty() {\n            return Vec::new();\n        }\n        query.push_str(\" WHERE \");\n        let mut operator_stack = Vec::new();\n        let mut operator = &SearchFilter::And;\n        let mut is_first = true;\n        let mut values = Vec::new();\n\n        for filter in filters {\n            match filter {\n                SearchFilter::Operator { field, op, value } => {\n                    if !is_first {\n                        match operator {\n                            SearchFilter::And => query.push_str(\" AND \"),\n                            SearchFilter::Or => query.push_str(\" OR \"),\n                            _ => (),\n                        }\n                    } else {\n                        is_first = false;\n                    }\n                    let value_pos = values.len() + 1;\n                    if field.is_text()\n                        && matches!(op, SearchOperator::Equal | SearchOperator::Contains)\n                    {\n                        query.push_str(field.column());\n                        query.push(' ');\n\n                        let language = match &value {\n                            SearchValue::Text { language, .. }\n                                if self.languages.contains(language) =>\n                            {\n                                pg_lang(language).unwrap_or(\"simple\")\n                            }\n                            _ => \"simple\",\n                        };\n                        let method = match op {\n                            SearchOperator::Equal => \"phraseto_tsquery\",\n                            _ => \"plainto_tsquery\",\n                        };\n                        let _ = write!(query, \"@@ {method}('{language}', ${value_pos})\");\n                        values.push(value as &(dyn ToSql + Sync));\n                    } else if let SearchValue::KeyValues(kv) = value {\n                        query.push_str(field.column());\n                        query.push(' ');\n\n                        let (key, value) = kv.iter().next().unwrap();\n                        values.push(key as &(dyn ToSql + Sync));\n\n                        if !value.is_empty() {\n                            let _ = write!(query, \"->> ${value_pos} \");\n                            op.write_pqsql(query, values.len() + 1);\n                            values.push(value as &(dyn ToSql + Sync));\n                        } else {\n                            let _ = write!(query, \" ? ${value_pos}\");\n                        }\n                    } else {\n                        query.push_str(field.sort_column().unwrap_or(field.column()));\n                        query.push(' ');\n\n                        op.write_pqsql(query, value_pos);\n                        values.push(value as &(dyn ToSql + Sync));\n                    }\n                }\n                SearchFilter::And | SearchFilter::Or => {\n                    if !is_first {\n                        match operator {\n                            SearchFilter::And => query.push_str(\" AND \"),\n                            SearchFilter::Or => query.push_str(\" OR \"),\n                            _ => (),\n                        }\n                    } else {\n                        is_first = false;\n                    }\n\n                    operator_stack.push((operator, is_first));\n                    operator = filter;\n                    is_first = true;\n                    query.push('(');\n                }\n                SearchFilter::Not => {\n                    if !is_first {\n                        match operator {\n                            SearchFilter::And => query.push_str(\" AND \"),\n                            SearchFilter::Or => query.push_str(\" OR \"),\n                            _ => (),\n                        }\n                    } else {\n                        is_first = false;\n                    }\n\n                    operator_stack.push((operator, is_first));\n                    operator = &SearchFilter::And;\n                    is_first = true;\n                    query.push_str(\"NOT (\");\n                }\n                SearchFilter::End => {\n                    let p = operator_stack.pop().unwrap_or((&SearchFilter::And, true));\n                    operator = p.0;\n                    is_first = p.1;\n                    query.push(')');\n                }\n                SearchFilter::DocumentSet(_) => {\n                    debug_assert!(\n                        false,\n                        \"DocumentSet filters are not supported in Postgres backend\"\n                    )\n                }\n            }\n        }\n\n        values\n    }\n}\n\nfn build_sort(query: &mut String, sort: &[SearchComparator]) {\n    query.push_str(\" ORDER BY \");\n    for (i, comparator) in sort.iter().enumerate() {\n        if i > 0 {\n            query.push_str(\", \");\n        }\n        match comparator {\n            SearchComparator::Field { field, ascending } => {\n                query.push_str(field.sort_column().unwrap_or(field.column()));\n                if *ascending {\n                    query.push_str(\" ASC\");\n                } else {\n                    query.push_str(\" DESC\");\n                }\n            }\n            SearchComparator::DocumentSet { .. } | SearchComparator::SortedSet { .. } => {\n                debug_assert!(\n                    false,\n                    \"DocumentSet and SortedSet comparators are not supported \"\n                );\n            }\n        }\n    }\n}\n\nimpl ToSql for SearchValue {\n    fn to_sql(\n        &self,\n        ty: &tokio_postgres::types::Type,\n        out: &mut bytes::BytesMut,\n    ) -> Result<tokio_postgres::types::IsNull, Box<dyn std::error::Error + Sync + Send>>\n    where\n        Self: Sized,\n    {\n        match self {\n            SearchValue::Text { value, .. } => {\n                // Truncate large text fields to avoid Postgres errors (see https://www.postgresql.org/docs/current/textsearch-limitations.html)\n\n                if value.len() > 650_000 {\n                    (&value[..value.floor_char_boundary(650_000)]).to_sql(ty, out)\n                } else {\n                    value.to_sql(ty, out)\n                }\n            }\n            SearchValue::Int(v) => match *ty {\n                Type::INT4 => (*v as i32).to_sql(ty, out),\n                _ => v.to_sql(ty, out),\n            },\n            SearchValue::Uint(v) => match *ty {\n                Type::INT4 => (*v as i32).to_sql(ty, out),\n                _ => (*v as i64).to_sql(ty, out),\n            },\n            SearchValue::Boolean(v) => v.to_sql(ty, out),\n            SearchValue::KeyValues(kv) => {\n                serde_json::to_value(kv).unwrap_or_default().to_sql(ty, out)\n            }\n        }\n    }\n\n    fn accepts(_: &tokio_postgres::types::Type) -> bool\n    where\n        Self: Sized,\n    {\n        true\n    }\n\n    fn to_sql_checked(\n        &self,\n        ty: &tokio_postgres::types::Type,\n        out: &mut bytes::BytesMut,\n    ) -> Result<tokio_postgres::types::IsNull, Box<dyn std::error::Error + Sync + Send>> {\n        match self {\n            SearchValue::Text { value, .. } => {\n                // Truncate large text fields to avoid Postgres errors (see https://www.postgresql.org/docs/current/textsearch-limitations.html)\n\n                if value.len() > 650_000 {\n                    (&value[..value.floor_char_boundary(650_000)]).to_sql_checked(ty, out)\n                } else {\n                    value.to_sql_checked(ty, out)\n                }\n            }\n            SearchValue::Int(v) => match *ty {\n                Type::INT4 => (*v as i32).to_sql_checked(ty, out),\n                _ => v.to_sql_checked(ty, out),\n            },\n            SearchValue::Uint(v) => match *ty {\n                Type::INT4 => (*v as i32).to_sql_checked(ty, out),\n                _ => (*v as i64).to_sql_checked(ty, out),\n            },\n            SearchValue::Boolean(v) => v.to_sql_checked(ty, out),\n            SearchValue::KeyValues(kv) => serde_json::to_value(kv)\n                .unwrap_or_default()\n                .to_sql_checked(ty, out),\n        }\n    }\n}\n\nstruct DocId(u64);\n\nimpl FromSql<'_> for DocId {\n    fn from_sql(\n        ty: &tokio_postgres::types::Type,\n        raw: &'_ [u8],\n    ) -> Result<Self, Box<dyn std::error::Error + Sync + Send>> {\n        match ty {\n            &Type::INT4 => i32::from_sql(ty, raw).map(|v| DocId(v as u64)),\n            &Type::INT8 | &Type::OID => i64::from_sql(ty, raw).map(|v| DocId(v as u64)),\n            _ => Err(Box::new(WrongType::new::<DocId>(ty.clone()))),\n        }\n    }\n\n    fn accepts(typ: &Type) -> bool {\n        matches!(typ, &Type::INT4 | &Type::INT8 | &Type::OID)\n    }\n}\n\nimpl SearchOperator {\n    fn write_pqsql(&self, query: &mut String, value_pos: usize) {\n        match self {\n            SearchOperator::LowerThan => {\n                let _ = write!(query, \"< ${value_pos}\");\n            }\n            SearchOperator::LowerEqualThan => {\n                let _ = write!(query, \"<= ${value_pos}\");\n            }\n            SearchOperator::GreaterThan => {\n                let _ = write!(query, \"> ${value_pos}\");\n            }\n            SearchOperator::GreaterEqualThan => {\n                let _ = write!(query, \">= ${value_pos}\");\n            }\n            SearchOperator::Equal => {\n                let _ = write!(query, \"= ${value_pos}\");\n            }\n            SearchOperator::Contains => {\n                let _ = write!(query, \"LIKE '%' || ${value_pos} || '%'\");\n            }\n        }\n    }\n}\n\n#[inline(always)]\nfn pg_lang(lang: &Language) -> Option<&'static str> {\n    match lang {\n        Language::Esperanto => None,\n        Language::English => Some(\"english\"),\n        Language::Russian => Some(\"russian\"),\n        Language::Mandarin => None,\n        Language::Spanish => Some(\"spanish\"),\n        Language::Portuguese => Some(\"portuguese\"),\n        Language::Italian => Some(\"italian\"),\n        Language::Bengali => None,\n        Language::French => Some(\"french\"),\n        Language::German => Some(\"german\"),\n        Language::Ukrainian => None,\n        Language::Georgian => None,\n        Language::Arabic => Some(\"arabic\"),\n        Language::Hindi => Some(\"hindi\"),\n        Language::Japanese => None,\n        Language::Hebrew => None,\n        Language::Yiddish => Some(\"yiddish\"),\n        Language::Polish => Some(\"polish\"),\n        Language::Amharic => None,\n        Language::Javanese => None,\n        Language::Korean => None,\n        Language::Bokmal => Some(\"norwegian\"), // Norwegian covers Bokmål\n        Language::Danish => Some(\"danish\"),\n        Language::Swedish => Some(\"swedish\"),\n        Language::Finnish => Some(\"finnish\"),\n        Language::Turkish => Some(\"turkish\"),\n        Language::Dutch => Some(\"dutch\"),\n        Language::Hungarian => Some(\"hungarian\"),\n        Language::Czech => Some(\"czech\"),\n        Language::Greek => Some(\"greek\"),\n        Language::Bulgarian => None,\n        Language::Belarusian => None,\n        Language::Marathi => None,\n        Language::Kannada => None,\n        Language::Romanian => Some(\"romanian\"),\n        Language::Slovene => None,\n        Language::Croatian => None,\n        Language::Serbian => Some(\"serbian\"),\n        Language::Macedonian => None,\n        Language::Lithuanian => Some(\"lithuanian\"),\n        Language::Latvian => None,\n        Language::Estonian => None,\n        Language::Tamil => Some(\"tamil\"),\n        Language::Vietnamese => None,\n        Language::Urdu => None,\n        Language::Thai => None,\n        Language::Gujarati => None,\n        Language::Uzbek => None,\n        Language::Punjabi => None,\n        Language::Azerbaijani => None,\n        Language::Indonesian => Some(\"indonesian\"),\n        Language::Telugu => None,\n        Language::Persian => None,\n        Language::Malayalam => None,\n        Language::Oriya => None,\n        Language::Burmese => None,\n        Language::Nepali => Some(\"nepali\"),\n        Language::Sinhalese => None,\n        Language::Khmer => None,\n        Language::Turkmen => None,\n        Language::Akan => None,\n        Language::Zulu => None,\n        Language::Shona => None,\n        Language::Afrikaans => None,\n        Language::Latin => None,\n        Language::Slovak => None,\n        Language::Catalan => Some(\"catalan\"),\n        Language::Tagalog => None,\n        Language::Armenian => Some(\"armenian\"),\n        Language::Unknown | Language::None => None,\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/tls.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n// Credits: https://github.com/jbg/tokio-postgres-rustls\n\nuse std::{\n    convert::TryFrom,\n    future::Future,\n    io,\n    pin::Pin,\n    sync::Arc,\n    task::{Context, Poll},\n};\n\nuse futures::future::{FutureExt, TryFutureExt};\nuse ring::digest;\nuse rustls::ClientConfig;\nuse rustls_pki_types::ServerName;\nuse tokio::io::{AsyncRead, AsyncWrite, ReadBuf};\nuse tokio_postgres::tls::{ChannelBinding, MakeTlsConnect, TlsConnect};\nuse tokio_rustls::{TlsConnector, client::TlsStream};\n\n#[derive(Clone)]\npub struct MakeRustlsConnect {\n    config: Arc<ClientConfig>,\n}\n\nimpl MakeRustlsConnect {\n    pub fn new(config: ClientConfig) -> Self {\n        Self {\n            config: Arc::new(config),\n        }\n    }\n}\n\nimpl<S> MakeTlsConnect<S> for MakeRustlsConnect\nwhere\n    S: AsyncRead + AsyncWrite + Unpin + Send + 'static,\n{\n    type Stream = RustlsStream<S>;\n    type TlsConnect = RustlsConnect;\n    type Error = io::Error;\n\n    fn make_tls_connect(&mut self, hostname: &str) -> io::Result<RustlsConnect> {\n        ServerName::try_from(hostname.to_string())\n            .map(|dns_name| {\n                RustlsConnect(Some(RustlsConnectData {\n                    hostname: dns_name,\n                    connector: Arc::clone(&self.config).into(),\n                }))\n            })\n            .or(Ok(RustlsConnect(None)))\n    }\n}\n\npub struct RustlsConnect(Option<RustlsConnectData>);\n\nstruct RustlsConnectData {\n    hostname: ServerName<'static>,\n    connector: TlsConnector,\n}\n\nimpl<S> TlsConnect<S> for RustlsConnect\nwhere\n    S: AsyncRead + AsyncWrite + Unpin + Send + 'static,\n{\n    type Stream = RustlsStream<S>;\n    type Error = io::Error;\n    type Future = Pin<Box<dyn Future<Output = io::Result<RustlsStream<S>>> + Send>>;\n\n    fn connect(self, stream: S) -> Self::Future {\n        match self.0 {\n            None => Box::pin(core::future::ready(Err(io::ErrorKind::InvalidInput.into()))),\n            Some(c) => c\n                .connector\n                .connect(c.hostname, stream)\n                .map_ok(|s| RustlsStream(Box::pin(s)))\n                .boxed(),\n        }\n    }\n}\n\npub struct RustlsStream<S>(Pin<Box<TlsStream<S>>>);\n\nimpl<S> tokio_postgres::tls::TlsStream for RustlsStream<S>\nwhere\n    S: AsyncRead + AsyncWrite + Unpin,\n{\n    fn channel_binding(&self) -> ChannelBinding {\n        let (_, session) = self.0.get_ref();\n        match session.peer_certificates() {\n            Some(certs) if !certs.is_empty() => {\n                let sha256 = digest::digest(&digest::SHA256, certs[0].as_ref());\n                ChannelBinding::tls_server_end_point(sha256.as_ref().into())\n            }\n            _ => ChannelBinding::none(),\n        }\n    }\n}\n\nimpl<S> AsyncRead for RustlsStream<S>\nwhere\n    S: AsyncRead + AsyncWrite + Unpin,\n{\n    fn poll_read(\n        mut self: Pin<&mut Self>,\n        cx: &mut Context,\n        buf: &mut ReadBuf<'_>,\n    ) -> Poll<tokio::io::Result<()>> {\n        self.0.as_mut().poll_read(cx, buf)\n    }\n}\n\nimpl<S> AsyncWrite for RustlsStream<S>\nwhere\n    S: AsyncRead + AsyncWrite + Unpin,\n{\n    fn poll_write(\n        mut self: Pin<&mut Self>,\n        cx: &mut Context,\n        buf: &[u8],\n    ) -> Poll<tokio::io::Result<usize>> {\n        self.0.as_mut().poll_write(cx, buf)\n    }\n\n    fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll<tokio::io::Result<()>> {\n        self.0.as_mut().poll_flush(cx)\n    }\n\n    fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context) -> Poll<tokio::io::Result<()>> {\n        self.0.as_mut().poll_shutdown(cx)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/postgres/write.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{PostgresStore, into_error};\nuse crate::{\n    IndexKey, Key, LogKey, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER, SUBSPACE_QUOTA,\n    backend::postgres::into_pool_error,\n    write::{\n        AssignedIds, Batch, MAX_COMMIT_ATTEMPTS, MAX_COMMIT_TIME, MergeResult, Operation,\n        ValueClass, ValueOp,\n    },\n};\nuse ahash::AHashMap;\nuse deadpool_postgres::Object;\nuse rand::Rng;\nuse std::time::{Duration, Instant};\nuse tokio_postgres::{IsolationLevel, error::SqlState};\n\n#[derive(Debug)]\nenum CommitError {\n    Postgres(tokio_postgres::Error),\n    Internal(trc::Error),\n    //Retry,\n}\n\nimpl PostgresStore {\n    pub(crate) async fn write(&self, mut batch: Batch<'_>) -> trc::Result<AssignedIds> {\n        let mut conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n        let start = Instant::now();\n        let mut retry_count = 0;\n\n        loop {\n            match self.write_trx(&mut conn, &mut batch).await {\n                Ok(result) => {\n                    return Ok(result);\n                }\n                Err(err) => {\n                    match err {\n                        CommitError::Postgres(err) => match err.code() {\n                            Some(\n                                &SqlState::T_R_SERIALIZATION_FAILURE\n                                | &SqlState::T_R_DEADLOCK_DETECTED,\n                            ) if retry_count < MAX_COMMIT_ATTEMPTS\n                                && start.elapsed() < MAX_COMMIT_TIME => {}\n                            Some(&SqlState::UNIQUE_VIOLATION) => {\n                                return Err(trc::StoreEvent::AssertValueFailed\n                                    .into_err()\n                                    .reason(\"Unique violation\")\n                                    .caused_by(trc::location!()));\n                            }\n                            _ => return Err(into_error(err)),\n                        },\n                        CommitError::Internal(err) => return Err(err),\n                        /*CommitError::Retry => {\n                            if retry_count > MAX_COMMIT_ATTEMPTS\n                                || start.elapsed() > MAX_COMMIT_TIME\n                            {\n                                return Err(trc::StoreEvent::AssertValueFailed\n                                    .into_err()\n                                    .caused_by(trc::location!()));\n                            }\n                        }*/\n                    }\n\n                    let backoff = rand::rng().random_range(50..=300);\n                    tokio::time::sleep(Duration::from_millis(backoff)).await;\n                    retry_count += 1;\n                }\n            }\n        }\n    }\n\n    async fn write_trx(\n        &self,\n        conn: &mut Object,\n        batch: &mut Batch<'_>,\n    ) -> Result<AssignedIds, CommitError> {\n        let mut account_id = u32::MAX;\n        let mut collection = u8::MAX;\n        let mut document_id = u32::MAX;\n        let mut change_id = 0u64;\n        let mut asserted_values = AHashMap::new();\n        let trx = conn\n            .build_transaction()\n            .isolation_level(IsolationLevel::ReadCommitted)\n            .start()\n            .await?;\n        let mut result = AssignedIds::default();\n        let has_changes = !batch.changes.is_empty();\n\n        if has_changes {\n            for &account_id in batch.changes.keys() {\n                let key = ValueClass::ChangeId.serialize(account_id, 0, 0, 0);\n                let s = trx\n                    .prepare_cached(concat!(\n                        \"INSERT INTO n (k, v) VALUES ($1, 1) \",\n                        \"ON CONFLICT(k) DO UPDATE SET v = n.v + 1 RETURNING v\"\n                    ))\n                    .await?;\n                let change_id = trx\n                    .query_one(&s, &[&key])\n                    .await\n                    .and_then(|row| row.try_get::<_, i64>(0))?;\n                result.push_change_id(account_id, change_id as u64);\n            }\n        }\n\n        for op in batch.ops.iter_mut() {\n            match op {\n                Operation::AccountId {\n                    account_id: account_id_,\n                } => {\n                    account_id = *account_id_;\n                    if has_changes {\n                        change_id = result.set_current_change_id(account_id)?;\n                    }\n                }\n                Operation::Collection {\n                    collection: collection_,\n                } => {\n                    collection = u8::from(*collection_);\n                }\n                Operation::DocumentId {\n                    document_id: document_id_,\n                } => {\n                    document_id = *document_id_;\n                }\n                Operation::Value { class, op } => {\n                    let key = class.serialize(account_id, collection, document_id, 0);\n                    let table = char::from(class.subspace(collection));\n\n                    match op {\n                        ValueOp::Set(value) => {\n                            let s = if let Some(exists) = asserted_values.get(&key) {\n                                if *exists {\n                                    trx.prepare_cached(&format!(\n                                        \"UPDATE {} SET v = $2 WHERE k = $1\",\n                                        table\n                                    ))\n                                    .await?\n                                } else {\n                                    trx.prepare_cached(&format!(\n                                        \"INSERT INTO {} (k, v) VALUES ($1, $2)\",\n                                        table\n                                    ))\n                                    .await?\n                                }\n                            } else {\n                                trx.prepare_cached(&format!(\n                                    concat!(\n                                        \"INSERT INTO {} (k, v) VALUES ($1, $2) \",\n                                        \"ON CONFLICT (k) DO UPDATE SET v = EXCLUDED.v\"\n                                    ),\n                                    table\n                                ))\n                                .await?\n                            };\n\n                            if trx.execute(&s, &[&key, &(*value)]).await? == 0 {\n                                return Err(trc::StoreEvent::AssertValueFailed\n                                    .into_err()\n                                    .caused_by(trc::location!())\n                                    .into());\n                            }\n                        }\n                        ValueOp::SetFnc(set_op) => {\n                            let value = (set_op.fnc)(&set_op.params, &result)?;\n\n                            let s = if let Some(exists) = asserted_values.get(&key) {\n                                if *exists {\n                                    trx.prepare_cached(&format!(\n                                        \"UPDATE {} SET v = $2 WHERE k = $1\",\n                                        table\n                                    ))\n                                    .await?\n                                } else {\n                                    trx.prepare_cached(&format!(\n                                        \"INSERT INTO {} (k, v) VALUES ($1, $2)\",\n                                        table\n                                    ))\n                                    .await?\n                                }\n                            } else {\n                                trx.prepare_cached(&format!(\n                                    concat!(\n                                        \"INSERT INTO {} (k, v) VALUES ($1, $2) \",\n                                        \"ON CONFLICT (k) DO UPDATE SET v = EXCLUDED.v\"\n                                    ),\n                                    table\n                                ))\n                                .await?\n                            };\n\n                            if trx.execute(&s, &[&key, &value]).await? == 0 {\n                                return Err(trc::StoreEvent::AssertValueFailed\n                                    .into_err()\n                                    .caused_by(trc::location!())\n                                    .into());\n                            }\n                        }\n                        ValueOp::MergeFnc(merge_op) => {\n                            let s = trx\n                                .prepare_cached(&format!(\n                                    \"SELECT v FROM {} WHERE k = $1 FOR UPDATE\",\n                                    table\n                                ))\n                                .await?;\n                            let (exists, merge_result) = trx\n                                .query_opt(&s, &[&key])\n                                .await?\n                                .map(|row| {\n                                    row.try_get::<_, &[u8]>(0)\n                                        .map_err(CommitError::from)\n                                        .and_then(|v| {\n                                            (merge_op.fnc)(&merge_op.params, &result, Some(v))\n                                                .map(|v| (true, v))\n                                                .map_err(CommitError::from)\n                                        })\n                                })\n                                .unwrap_or_else(|| {\n                                    (merge_op.fnc)(&merge_op.params, &result, None)\n                                        .map(|v| (false, v))\n                                        .map_err(CommitError::from)\n                                })?;\n\n                            match merge_result {\n                                MergeResult::Update(value) => {\n                                    let s = if exists {\n                                        trx.prepare_cached(&format!(\n                                            \"UPDATE {} SET v = $2 WHERE k = $1\",\n                                            table\n                                        ))\n                                        .await?\n                                    } else {\n                                        trx.prepare_cached(&format!(\n                                            \"INSERT INTO {} (k, v) VALUES ($1, $2)\",\n                                            table\n                                        ))\n                                        .await?\n                                    };\n\n                                    trx.execute(&s, &[&key, &value]).await?;\n                                }\n                                MergeResult::Delete if exists => {\n                                    let s = trx\n                                        .prepare_cached(&format!(\n                                            \"DELETE FROM {} WHERE k = $1\",\n                                            table\n                                        ))\n                                        .await?;\n                                    trx.execute(&s, &[&key]).await?;\n\n                                    // Update asserted value\n                                    if let Some(exists) = asserted_values.get_mut(&key) {\n                                        *exists = false;\n                                    }\n                                }\n                                _ => (),\n                            }\n                        }\n                        ValueOp::AtomicAdd(by) => {\n                            if *by >= 0 {\n                                let s = trx\n                                    .prepare_cached(&format!(\n                                        concat!(\n                                            \"INSERT INTO {} (k, v) VALUES ($1, $2) \",\n                                            \"ON CONFLICT(k) DO UPDATE SET v = {}.v + EXCLUDED.v\"\n                                        ),\n                                        table, table\n                                    ))\n                                    .await?;\n                                trx.execute(&s, &[&key, &*by]).await?;\n                            } else {\n                                let s = trx\n                                    .prepare_cached(&format!(\n                                        \"UPDATE {table} SET v = v + $1 WHERE k = $2\"\n                                    ))\n                                    .await?;\n                                trx.execute(&s, &[&*by, &key]).await?;\n                            }\n                        }\n                        ValueOp::AddAndGet(by) => {\n                            let s = trx\n                                .prepare_cached(&format!(\n                                    concat!(\n                                    \"INSERT INTO {} (k, v) VALUES ($1, $2) \",\n                                    \"ON CONFLICT(k) DO UPDATE SET v = {}.v + EXCLUDED.v RETURNING v\"\n                                ),\n                                    table, table\n                                ))\n                                .await?;\n                            result.push_counter_id(\n                                trx.query_one(&s, &[&key, &*by])\n                                    .await\n                                    .and_then(|row| row.try_get::<_, i64>(0))?,\n                            );\n                        }\n                        ValueOp::Clear => {\n                            let s = trx\n                                .prepare_cached(&format!(\"DELETE FROM {} WHERE k = $1\", table))\n                                .await?;\n                            trx.execute(&s, &[&key]).await?;\n\n                            // Update asserted value\n                            if let Some(exists) = asserted_values.get_mut(&key) {\n                                *exists = false;\n                            }\n                        }\n                    }\n                }\n                Operation::Index { field, key, set } => {\n                    let key = IndexKey {\n                        account_id,\n                        collection,\n                        document_id,\n                        field: *field,\n                        key: &*key,\n                    }\n                    .serialize(0);\n\n                    let s = if *set {\n                        trx.prepare_cached(\n                            \"INSERT INTO i (k) VALUES ($1) ON CONFLICT (k) DO NOTHING\",\n                        )\n                        .await?\n                    } else {\n                        trx.prepare_cached(\"DELETE FROM i WHERE k = $1\").await?\n                    };\n                    trx.execute(&s, &[&key]).await?;\n                }\n                Operation::Log { collection, set } => {\n                    let key = LogKey {\n                        account_id,\n                        collection: u8::from(*collection),\n                        change_id,\n                    }\n                    .serialize(0);\n\n                    let s = trx\n                        .prepare_cached(concat!(\n                            \"INSERT INTO l (k, v) VALUES ($1, $2) \",\n                            \"ON CONFLICT (k) DO UPDATE SET v = EXCLUDED.v\"\n                        ))\n                        .await?;\n\n                    trx.execute(&s, &[&key, &*set]).await?;\n                }\n                Operation::AssertValue {\n                    class,\n                    assert_value,\n                } => {\n                    let key = class.serialize(account_id, collection, document_id, 0);\n                    let table = char::from(class.subspace(collection));\n\n                    let s = trx\n                        .prepare_cached(&format!(\"SELECT v FROM {} WHERE k = $1 FOR UPDATE\", table))\n                        .await?;\n                    let (exists, matches) = trx\n                        .query_opt(&s, &[&key])\n                        .await?\n                        .map(|row| {\n                            row.try_get::<_, &[u8]>(0)\n                                .map_or((true, false), |v| (true, assert_value.matches(v)))\n                        })\n                        .unwrap_or_else(|| (false, assert_value.is_none()));\n                    if !matches {\n                        return Err(trc::StoreEvent::AssertValueFailed\n                            .into_err()\n                            .caused_by(trc::location!())\n                            .into());\n                    }\n                    asserted_values.insert(key, exists);\n                }\n            }\n        }\n\n        trx.commit().await.map(|_| result).map_err(Into::into)\n    }\n\n    pub(crate) async fn purge_store(&self) -> trc::Result<()> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n\n        for subspace in [SUBSPACE_QUOTA, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER] {\n            let s = conn\n                .prepare_cached(&format!(\"DELETE FROM {} WHERE v = 0\", char::from(subspace),))\n                .await\n                .map_err(into_error)?;\n            conn.execute(&s, &[])\n                .await\n                .map(|_| ())\n                .map_err(into_error)?\n        }\n\n        Ok(())\n    }\n\n    pub(crate) async fn delete_range(&self, from: impl Key, to: impl Key) -> trc::Result<()> {\n        let conn = self.conn_pool.get().await.map_err(into_pool_error)?;\n\n        let s = conn\n            .prepare_cached(&format!(\n                \"DELETE FROM {} WHERE k >= $1 AND k < $2\",\n                char::from(from.subspace()),\n            ))\n            .await\n            .map_err(into_error)?;\n        conn.execute(&s, &[&from.serialize(0), &to.serialize(0)])\n            .await\n            .map(|_| ())\n            .map_err(into_error)\n    }\n}\n\nimpl From<trc::Error> for CommitError {\n    fn from(err: trc::Error) -> Self {\n        CommitError::Internal(err)\n    }\n}\n\nimpl From<tokio_postgres::Error> for CommitError {\n    fn from(err: tokio_postgres::Error) -> Self {\n        CommitError::Postgres(err)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/redis/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse redis::AsyncCommands;\n\nuse crate::Deserialize;\n\nuse super::{RedisPool, RedisStore, into_error};\n\nimpl RedisStore {\n    pub async fn key_set(&self, key: &[u8], value: &[u8], expires: Option<u64>) -> trc::Result<()> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                self.key_set_(\n                    pool.get().await.map_err(into_error)?.as_mut(),\n                    key,\n                    value,\n                    expires,\n                )\n                .await\n            }\n            RedisPool::Cluster(pool) => {\n                self.key_set_(\n                    pool.get().await.map_err(into_error)?.as_mut(),\n                    key,\n                    value,\n                    expires,\n                )\n                .await\n            }\n        }\n    }\n\n    pub async fn key_incr(&self, key: &[u8], value: i64, expires: Option<u64>) -> trc::Result<i64> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                self.key_incr_(\n                    pool.get().await.map_err(into_error)?.as_mut(),\n                    key,\n                    value,\n                    expires,\n                )\n                .await\n            }\n            RedisPool::Cluster(pool) => {\n                self.key_incr_(\n                    pool.get().await.map_err(into_error)?.as_mut(),\n                    key,\n                    value,\n                    expires,\n                )\n                .await\n            }\n        }\n    }\n\n    pub async fn key_delete(&self, key: &[u8]) -> trc::Result<()> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                self.key_delete_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n            RedisPool::Cluster(pool) => {\n                self.key_delete_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n        }\n    }\n\n    pub async fn key_delete_prefix(&self, prefix: &[u8]) -> trc::Result<()> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                self.key_delete_prefix_(pool.get().await.map_err(into_error)?.as_mut(), prefix)\n                    .await\n            }\n            RedisPool::Cluster(pool) => {\n                self.key_delete_prefix_(pool.get().await.map_err(into_error)?.as_mut(), prefix)\n                    .await\n            }\n        }\n    }\n\n    pub async fn key_get<T: Deserialize + std::fmt::Debug + 'static>(\n        &self,\n        key: &[u8],\n    ) -> trc::Result<Option<T>> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                self.key_get_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n            RedisPool::Cluster(pool) => {\n                self.key_get_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n        }\n    }\n\n    pub async fn counter_get(&self, key: &[u8]) -> trc::Result<i64> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                self.counter_get_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n            RedisPool::Cluster(pool) => {\n                self.counter_get_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n        }\n    }\n\n    pub async fn key_exists(&self, key: &[u8]) -> trc::Result<bool> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                self.key_exists_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n            RedisPool::Cluster(pool) => {\n                self.key_exists_(pool.get().await.map_err(into_error)?.as_mut(), key)\n                    .await\n            }\n        }\n    }\n\n    async fn key_get_<T: Deserialize + std::fmt::Debug + 'static>(\n        &self,\n        conn: &mut impl AsyncCommands,\n        key: &[u8],\n    ) -> trc::Result<Option<T>> {\n        if let Some(value) = redis::cmd(\"GET\")\n            .arg(key)\n            .query_async::<Option<Vec<u8>>>(conn)\n            .await\n            .map_err(into_error)?\n        {\n            T::deserialize_owned(value).map(Some)\n        } else {\n            Ok(None)\n        }\n    }\n\n    async fn counter_get_(&self, conn: &mut impl AsyncCommands, key: &[u8]) -> trc::Result<i64> {\n        redis::cmd(\"GET\")\n            .arg(key)\n            .query_async::<Option<i64>>(conn)\n            .await\n            .map(|x| x.unwrap_or(0))\n            .map_err(into_error)\n    }\n\n    async fn key_exists_(&self, conn: &mut impl AsyncCommands, key: &[u8]) -> trc::Result<bool> {\n        conn.exists(key).await.map_err(into_error)\n    }\n\n    async fn key_set_(\n        &self,\n        conn: &mut impl AsyncCommands,\n        key: &[u8],\n        value: &[u8],\n        expires: Option<u64>,\n    ) -> trc::Result<()> {\n        if let Some(expires) = expires {\n            conn.set_ex(key, value, expires).await.map_err(into_error)\n        } else {\n            conn.set(key, value).await.map_err(into_error)\n        }\n    }\n\n    async fn key_incr_(\n        &self,\n        conn: &mut impl AsyncCommands,\n        key: &[u8],\n        value: i64,\n        expires: Option<u64>,\n    ) -> trc::Result<i64> {\n        if let Some(expires) = expires {\n            redis::pipe()\n                .atomic()\n                .incr(key, value)\n                .expire(key, expires as i64)\n                .ignore()\n                .query_async::<Vec<i64>>(conn)\n                .await\n                .map_err(into_error)\n                .map(|v| v.first().copied().unwrap_or(0))\n        } else {\n            conn.incr(key, value).await.map_err(into_error)\n        }\n    }\n\n    async fn key_delete_(&self, conn: &mut impl AsyncCommands, key: &[u8]) -> trc::Result<()> {\n        conn.del(key).await.map_err(into_error)\n    }\n\n    async fn key_delete_prefix_(\n        &self,\n        conn: &mut impl AsyncCommands,\n        prefix: &[u8],\n    ) -> trc::Result<()> {\n        let mut pattern = Vec::with_capacity(prefix.len() + 1);\n        pattern.extend_from_slice(prefix);\n        pattern.push(b'*');\n\n        let mut cursor = 0;\n        loop {\n            let (new_cursor, keys): (u64, Vec<Vec<u8>>) = redis::cmd(\"SCAN\")\n                .cursor_arg(cursor)\n                .arg(\"MATCH\")\n                .arg(&pattern)\n                .arg(\"COUNT\")\n                .arg(100)\n                .query_async(conn)\n                .await\n                .map_err(into_error)?;\n\n            if !keys.is_empty() {\n                conn.del::<_, ()>(&keys).await.map_err(into_error)?;\n            }\n\n            if new_cursor != 0 {\n                cursor = new_cursor;\n            } else {\n                return Ok(());\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/redis/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fmt::Display, time::Duration};\n\nuse deadpool::{\n    Runtime,\n    managed::{Manager, Pool},\n};\nuse redis::{\n    Client, ProtocolVersion,\n    cluster::{ClusterClient, ClusterClientBuilder},\n};\nuse utils::config::{Config, utils::AsKey};\n\npub mod lookup;\npub mod pool;\npub mod pubsub;\n\n#[derive(Debug)]\npub struct RedisStore {\n    pool: RedisPool,\n}\n\nstruct RedisConnectionManager {\n    client: Client,\n    timeout: Duration,\n}\n\nstruct RedisClusterConnectionManager {\n    client: ClusterClient,\n    timeout: Duration,\n}\n\nenum RedisPool {\n    Single(Pool<RedisConnectionManager>),\n    Cluster(Pool<RedisClusterConnectionManager>),\n}\n\nimpl RedisStore {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let urls = config\n            .values((&prefix, \"urls\"))\n            .map(|(_, v)| v.to_string())\n            .collect::<Vec<_>>();\n        if urls.is_empty() {\n            config.new_build_error((&prefix, \"urls\"), \"No Redis URLs specified\");\n            return None;\n        }\n\n        Some(\n            match config.value((&prefix, \"redis-type\")).unwrap_or(\"single\") {\n                \"single\" => {\n                    let client = Client::open(urls.into_iter().next().unwrap())\n                        .map_err(|err| {\n                            config.new_build_error(\n                                prefix.as_str(),\n                                format!(\"Failed to open Redis client: {err:?}\"),\n                            )\n                        })\n                        .ok()?;\n                    let timeout = config\n                        .property_or_default((&prefix, \"timeout\"), \"10s\")\n                        .unwrap_or_default();\n\n                    Self {\n                        pool: RedisPool::Single(\n                            build_pool(config, &prefix, RedisConnectionManager { client, timeout })\n                                .map_err(|err| {\n                                    config.new_build_error(\n                                        prefix.as_str(),\n                                        format!(\"Failed to build Redis pool: {err:?}\"),\n                                    )\n                                })\n                                .ok()?,\n                        ),\n                    }\n                }\n                \"cluster\" => {\n                    let mut builder = ClusterClientBuilder::new(urls.into_iter());\n                    if let Some(value) = config.property((&prefix, \"user\")) {\n                        builder = builder.username(value);\n                    }\n                    if let Some(value) = config.property((&prefix, \"password\")) {\n                        builder = builder.password(value);\n                    }\n                    if let Some(value) = config.property((&prefix, \"retry.total\")) {\n                        builder = builder.retries(value);\n                    }\n                    if let Some(value) = config\n                        .property::<Option<Duration>>((&prefix, \"retry.max-wait\"))\n                        .unwrap_or_default()\n                    {\n                        builder = builder.max_retry_wait(value.as_millis() as u64);\n                    }\n                    if let Some(value) = config\n                        .property::<Option<Duration>>((&prefix, \"retry.min-wait\"))\n                        .unwrap_or_default()\n                    {\n                        builder = builder.min_retry_wait(value.as_millis() as u64);\n                    }\n                    if let Some(true) = config.property::<bool>((&prefix, \"read-from-replicas\")) {\n                        builder = builder.read_from_replicas();\n                    }\n                    if config\n                        .value((&prefix, \"protocol-version\"))\n                        .unwrap_or(\"resp2\")\n                        == \"resp3\"\n                    {\n                        builder = builder.use_protocol(ProtocolVersion::RESP3);\n                    }\n\n                    let client = builder\n                        .build()\n                        .map_err(|err| {\n                            config.new_build_error(\n                                prefix.as_str(),\n                                format!(\"Failed to open Redis client: {err:?}\"),\n                            )\n                        })\n                        .ok()?;\n                    let timeout = config\n                        .property_or_default::<Duration>((&prefix, \"timeout\"), \"10s\")\n                        .unwrap_or_else(|| Duration::from_secs(10));\n\n                    Self {\n                        pool: RedisPool::Cluster(\n                            build_pool(\n                                config,\n                                &prefix,\n                                RedisClusterConnectionManager { client, timeout },\n                            )\n                            .map_err(|err| {\n                                config.new_build_error(\n                                    prefix.as_str(),\n                                    format!(\"Failed to build Redis pool: {err:?}\"),\n                                )\n                            })\n                            .ok()?,\n                        ),\n                    }\n                }\n                invalid => {\n                    let err = format!(\"Invalid Redis type {invalid:?}\");\n                    config.new_parse_error((&prefix, \"redis-type\"), err);\n                    return None;\n                }\n            },\n        )\n    }\n}\n\nfn build_pool<M: Manager>(\n    config: &mut Config,\n    prefix: &str,\n    manager: M,\n) -> Result<Pool<M>, String> {\n    Pool::builder(manager)\n        .runtime(Runtime::Tokio1)\n        .max_size(\n            config\n                .property_or_default((prefix, \"pool.max-connections\"), \"10\")\n                .unwrap_or(10),\n        )\n        .create_timeout(\n            config\n                .property_or_default::<Option<Duration>>((prefix, \"pool.create-timeout\"), \"30s\")\n                .unwrap_or_default(),\n        )\n        .wait_timeout(\n            config\n                .property_or_default::<Option<Duration>>((prefix, \"pool.wait-timeout\"), \"30s\")\n                .unwrap_or_default(),\n        )\n        .recycle_timeout(\n            config\n                .property_or_default::<Option<Duration>>((prefix, \"pool.recycle-timeout\"), \"30s\")\n                .unwrap_or_default(),\n        )\n        .build()\n        .map_err(|err| {\n            format!(\n                \"Failed to build pool for {prefix:?}: {err}\",\n                prefix = prefix,\n                err = err\n            )\n        })\n}\n\n#[inline(always)]\nfn into_error(err: impl Display) -> trc::Error {\n    trc::StoreEvent::RedisError.reason(err)\n}\n\nimpl std::fmt::Debug for RedisPool {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Self::Single(_) => f.debug_tuple(\"Single\").finish(),\n            Self::Cluster(_) => f.debug_tuple(\"Cluster\").finish(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/redis/pool.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse deadpool::managed;\nuse redis::{\n    aio::{ConnectionLike, MultiplexedConnection},\n    cluster_async::ClusterConnection,\n};\n\nuse super::{RedisClusterConnectionManager, RedisConnectionManager, into_error};\n\nimpl managed::Manager for RedisConnectionManager {\n    type Type = MultiplexedConnection;\n    type Error = trc::Error;\n\n    async fn create(&self) -> Result<MultiplexedConnection, trc::Error> {\n        match tokio::time::timeout(self.timeout, self.client.get_multiplexed_tokio_connection())\n            .await\n        {\n            Ok(conn) => conn.map_err(into_error),\n            Err(_) => Err(trc::StoreEvent::RedisError.ctx(trc::Key::Details, \"Connection Timeout\")),\n        }\n    }\n\n    async fn recycle(\n        &self,\n        conn: &mut MultiplexedConnection,\n        _: &managed::Metrics,\n    ) -> managed::RecycleResult<trc::Error> {\n        conn.req_packed_command(&redis::cmd(\"PING\"))\n            .await\n            .map(|_| ())\n            .map_err(|err| managed::RecycleError::Backend(into_error(err)))\n    }\n}\n\nimpl managed::Manager for RedisClusterConnectionManager {\n    type Type = ClusterConnection;\n    type Error = trc::Error;\n\n    async fn create(&self) -> Result<ClusterConnection, trc::Error> {\n        match tokio::time::timeout(self.timeout, self.client.get_async_connection()).await {\n            Ok(conn) => conn.map_err(into_error),\n            Err(_) => Err(trc::StoreEvent::RedisError.ctx(trc::Key::Details, \"Connection Timeout\")),\n        }\n    }\n\n    async fn recycle(\n        &self,\n        conn: &mut ClusterConnection,\n        _: &managed::Metrics,\n    ) -> managed::RecycleResult<trc::Error> {\n        conn.req_packed_command(&redis::cmd(\"PING\"))\n            .await\n            .map(|_| ())\n            .map_err(|err| managed::RecycleError::Backend(into_error(err)))\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/redis/pubsub.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{RedisPool, RedisStore, into_error};\nuse crate::dispatch::pubsub::{Msg, PubSubStream};\nuse futures::StreamExt;\nuse redis::{AsyncCommands, PushInfo, cluster::ClusterConfig, cluster_async::ClusterConnection};\nuse tokio::sync::mpsc::UnboundedReceiver;\n\npub struct RedisPubSubStream {\n    stream: redis::aio::PubSubStream,\n}\n\npub struct RedisClusterPubSubStream {\n    _conn: ClusterConnection,\n    rx: UnboundedReceiver<PushInfo>,\n}\n\nimpl RedisStore {\n    pub async fn publish(&self, topic: &'static str, message: Vec<u8>) -> trc::Result<()> {\n        match &self.pool {\n            RedisPool::Single(pool) => pool\n                .get()\n                .await\n                .map_err(into_error)?\n                .as_mut()\n                .publish(topic, message)\n                .await\n                .map_err(into_error),\n            RedisPool::Cluster(pool) => pool\n                .get()\n                .await\n                .map_err(into_error)?\n                .as_mut()\n                .publish(topic, message)\n                .await\n                .map_err(into_error),\n        }\n    }\n\n    pub async fn subscribe(&self, topic: &'static str) -> trc::Result<PubSubStream> {\n        match &self.pool {\n            RedisPool::Single(pool) => {\n                let mut pubsub = pool\n                    .manager()\n                    .client\n                    .get_async_pubsub()\n                    .await\n                    .map_err(into_error)?;\n                pubsub.subscribe(topic).await.map_err(into_error)?;\n\n                Ok(PubSubStream::Redis(RedisPubSubStream {\n                    stream: pubsub.into_on_message(),\n                }))\n            }\n            RedisPool::Cluster(pool) => {\n                let (tx, rx) = tokio::sync::mpsc::unbounded_channel();\n\n                let mut _conn = pool\n                    .manager()\n                    .client\n                    .get_async_connection_with_config(ClusterConfig::default().set_push_sender(tx))\n                    .await\n                    .map_err(into_error)?;\n\n                _conn.subscribe(topic).await.map_err(into_error)?;\n\n                Ok(PubSubStream::RedisCluster(RedisClusterPubSubStream {\n                    _conn,\n                    rx,\n                }))\n            }\n        }\n    }\n}\n\nimpl RedisPubSubStream {\n    pub async fn next(&mut self) -> Option<Msg> {\n        self.stream.next().await.map(Msg::Redis)\n    }\n}\n\nimpl RedisClusterPubSubStream {\n    pub async fn next(&mut self) -> Option<Msg> {\n        loop {\n            if let Some(msg) = redis::Msg::from_push_info(self.rx.recv().await?) {\n                return Some(Msg::Redis(msg));\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/rocksdb/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::ops::Range;\n\nuse super::{CF_BLOBS, RocksDbStore, into_error};\n\nimpl RocksDbStore {\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let db = self.db.clone();\n        self.spawn_worker(move || {\n            db.get_pinned_cf(&db.cf_handle(CF_BLOBS).unwrap(), key)\n                .map(|obj| {\n                    obj.map(|bytes| {\n                        if range.start == 0 && range.end == usize::MAX {\n                            bytes.to_vec()\n                        } else {\n                            bytes\n                                .get(range.start..std::cmp::min(bytes.len(), range.end))\n                                .unwrap_or_default()\n                                .to_vec()\n                        }\n                    })\n                })\n                .map_err(into_error)\n        })\n        .await\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let db = self.db.clone();\n        self.spawn_worker(move || {\n            db.put_cf(&db.cf_handle(CF_BLOBS).unwrap(), key, data)\n                .map_err(into_error)\n        })\n        .await\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let db = self.db.clone();\n        self.spawn_worker(move || {\n            db.delete_cf(&db.cf_handle(CF_BLOBS).unwrap(), key)\n                .map_err(into_error)\n                .map(|_| true)\n        })\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/rocksdb/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{CF_BLOBS, RocksDbStore};\nuse crate::*;\nuse rocksdb::{ColumnFamilyDescriptor, MergeOperands, OptimisticTransactionDB, Options};\nuse std::path::PathBuf;\nuse tokio::sync::oneshot;\nuse utils::config::{Config, utils::AsKey};\n\nimpl RocksDbStore {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        // Create the database directory if it doesn't exist\n        let idx_path: PathBuf = PathBuf::from(config.value_require((&prefix, \"path\"))?);\n        std::fs::create_dir_all(&idx_path)\n            .map_err(|err| {\n                config.new_build_error(\n                    (&prefix, \"path\"),\n                    format!(\n                        \"Failed to create database directory {}: {:?}\",\n                        idx_path.display(),\n                        err\n                    ),\n                )\n            })\n            .ok()?;\n\n        let mut cfs = Vec::new();\n\n        // Counters\n        for subspace in [SUBSPACE_COUNTER, SUBSPACE_QUOTA, SUBSPACE_IN_MEMORY_COUNTER] {\n            let mut cf_opts = Options::default();\n            cf_opts.set_merge_operator_associative(\"merge\", numeric_value_merge);\n            cfs.push(ColumnFamilyDescriptor::new(\n                std::str::from_utf8(&[subspace]).unwrap(),\n                cf_opts,\n            ));\n        }\n\n        // Blobs\n        let mut cf_opts = Options::default();\n        cf_opts.set_enable_blob_files(true);\n        cf_opts.set_min_blob_size(\n            config\n                .property_or_default((&prefix, \"min-blob-size\"), \"16834\")\n                .unwrap_or(16834),\n        );\n        cfs.push(ColumnFamilyDescriptor::new(CF_BLOBS, cf_opts));\n\n        // Other cfs\n        for subspace in [\n            SUBSPACE_INDEXES,\n            SUBSPACE_ACL,\n            SUBSPACE_DIRECTORY,\n            SUBSPACE_TASK_QUEUE,\n            SUBSPACE_BLOB_EXTRA,\n            SUBSPACE_BLOB_LINK,\n            SUBSPACE_IN_MEMORY_VALUE,\n            SUBSPACE_PROPERTY,\n            SUBSPACE_SETTINGS,\n            SUBSPACE_QUEUE_MESSAGE,\n            SUBSPACE_QUEUE_EVENT,\n            SUBSPACE_REPORT_OUT,\n            SUBSPACE_REPORT_IN,\n            SUBSPACE_LOGS,\n            SUBSPACE_BLOBS,\n            SUBSPACE_TELEMETRY_SPAN,\n            SUBSPACE_TELEMETRY_METRIC,\n            SUBSPACE_SEARCH_INDEX,\n            LEGACY_SUBSPACE_BITMAP_ID,\n            LEGACY_SUBSPACE_BITMAP_TAG,\n            LEGACY_SUBSPACE_BITMAP_TEXT,\n            LEGACY_SUBSPACE_FTS_INDEX,\n            LEGACY_SUBSPACE_TELEMETRY_INDEX,\n        ] {\n            let cf_opts = Options::default();\n            cfs.push(ColumnFamilyDescriptor::new(\n                std::str::from_utf8(&[subspace]).unwrap(),\n                cf_opts,\n            ));\n        }\n\n        let mut db_opts = Options::default();\n        db_opts.create_missing_column_families(true);\n        db_opts.create_if_missing(true);\n        db_opts.set_max_background_jobs(std::cmp::max(num_cpus::get() as i32, 3));\n        db_opts.increase_parallelism(std::cmp::max(num_cpus::get() as i32, 3));\n        db_opts.set_level_zero_file_num_compaction_trigger(1);\n        db_opts.set_level_compaction_dynamic_level_bytes(true);\n        //db_opts.set_keep_log_file_num(100);\n        //db_opts.set_max_successive_merges(100);\n        db_opts.set_write_buffer_size(\n            config\n                .property_or_default((&prefix, \"write-buffer-size\"), \"134217728\")\n                .unwrap_or(134217728),\n        );\n\n        Some(RocksDbStore {\n            db: OptimisticTransactionDB::open_cf_descriptors(&db_opts, idx_path, cfs)\n                .map_err(|err| {\n                    config.new_build_error(\n                        prefix.as_str(),\n                        format!(\"Failed to open database: {:?}\", err),\n                    )\n                })\n                .ok()?\n                .into(),\n            worker_pool: rayon::ThreadPoolBuilder::new()\n                .num_threads(std::cmp::max(\n                    config\n                        .property::<usize>((&prefix, \"pool.workers\"))\n                        .filter(|v| *v > 0)\n                        .unwrap_or_else(num_cpus::get),\n                    4,\n                ))\n                .build()\n                .map_err(|err| {\n                    config.new_build_error(\n                        (&prefix, \"pool.workers\"),\n                        format!(\"Failed to build worker pool: {:?}\", err),\n                    )\n                })\n                .ok()?,\n        })\n    }\n\n    pub async fn spawn_worker<U, V>(&self, mut f: U) -> trc::Result<V>\n    where\n        U: FnMut() -> trc::Result<V> + Send,\n        V: Sync + Send + 'static,\n    {\n        let (tx, rx) = oneshot::channel();\n\n        self.worker_pool.scope(|s| {\n            s.spawn(|_| {\n                tx.send(f()).ok();\n            });\n        });\n\n        match rx.await {\n            Ok(result) => result,\n            Err(err) => Err(trc::EventType::Server(trc::ServerEvent::ThreadError).reason(err)),\n        }\n    }\n}\n\npub fn numeric_value_merge(\n    _key: &[u8],\n    value: Option<&[u8]>,\n    operands: &MergeOperands,\n) -> Option<Vec<u8>> {\n    let mut value = if let Some(value) = value {\n        i64::from_le_bytes(value.try_into().ok()?)\n    } else {\n        0\n    };\n\n    for op in operands.iter() {\n        value += i64::from_le_bytes(op.try_into().ok()?);\n    }\n\n    let mut bytes = Vec::with_capacity(std::mem::size_of::<i64>());\n    bytes.extend_from_slice(&value.to_le_bytes());\n    Some(bytes)\n}\n"
  },
  {
    "path": "crates/store/src/backend/rocksdb/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse rocksdb::{BoundColumnFamily, MultiThreaded, OptimisticTransactionDB};\n\nuse crate::{SUBSPACE_BLOBS, SUBSPACE_INDEXES, SUBSPACE_LOGS};\n\npub mod blob;\npub mod main;\npub mod read;\npub mod write;\n\nstatic CF_LOGS: &str = unsafe { std::str::from_utf8_unchecked(&[SUBSPACE_LOGS]) };\nstatic CF_INDEXES: &str = unsafe { std::str::from_utf8_unchecked(&[SUBSPACE_INDEXES]) };\nstatic CF_BLOBS: &str = unsafe { std::str::from_utf8_unchecked(&[SUBSPACE_BLOBS]) };\n\npub(crate) trait CfHandle {\n    fn subspace_handle(&self, subspace: u8) -> Arc<BoundColumnFamily<'_>>;\n}\n\nimpl CfHandle for OptimisticTransactionDB<MultiThreaded> {\n    #[inline(always)]\n    fn subspace_handle(&self, subspace: u8) -> Arc<BoundColumnFamily<'_>> {\n        let subspace = &[subspace];\n        self.cf_handle(unsafe { std::str::from_utf8_unchecked(subspace) })\n            .unwrap()\n    }\n}\n\npub struct RocksDbStore {\n    db: Arc<OptimisticTransactionDB<MultiThreaded>>,\n    worker_pool: rayon::ThreadPool,\n}\n\n#[inline(always)]\nfn into_error(err: rocksdb::Error) -> trc::Error {\n    trc::StoreEvent::RocksdbError.reason(err)\n}\n"
  },
  {
    "path": "crates/store/src/backend/rocksdb/read.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{RocksDbStore, into_error};\nuse crate::{\n    Deserialize, IterateParams, Key, ValueKey, backend::rocksdb::CfHandle, write::ValueClass,\n};\nuse rocksdb::{Direction, IteratorMode};\n\nimpl RocksDbStore {\n    pub(crate) async fn get_value<U>(&self, key: impl Key) -> trc::Result<Option<U>>\n    where\n        U: Deserialize + 'static,\n    {\n        let db = self.db.clone();\n        self.spawn_worker(move || {\n            db.get_pinned_cf(\n                &db.cf_handle(std::str::from_utf8(&[key.subspace()]).unwrap())\n                    .unwrap(),\n                key.serialize(0),\n            )\n            .map_err(into_error)\n            .and_then(|value| {\n                if let Some(value) = value {\n                    U::deserialize(&value).map(Some)\n                } else {\n                    Ok(None)\n                }\n            })\n        })\n        .await\n    }\n\n    pub(crate) async fn iterate<T: Key>(\n        &self,\n        params: IterateParams<T>,\n        mut cb: impl for<'x> FnMut(&'x [u8], &'x [u8]) -> trc::Result<bool> + Sync + Send,\n    ) -> trc::Result<()> {\n        let db = self.db.clone();\n\n        self.spawn_worker(move || {\n            let cf = db.subspace_handle(params.begin.subspace());\n            let begin = params.begin.serialize(0);\n            let end = params.end.serialize(0);\n            let it_mode = if params.ascending {\n                IteratorMode::From(&begin, Direction::Forward)\n            } else {\n                IteratorMode::From(&end, Direction::Reverse)\n            };\n\n            for row in db.iterator_cf(&cf, it_mode) {\n                let (key, value) = row.map_err(into_error)?;\n                if key.as_ref() < begin.as_slice()\n                    || key.as_ref() > end.as_slice()\n                    || !cb(&key, &value)?\n                    || params.first\n                {\n                    break;\n                }\n            }\n\n            Ok(())\n        })\n        .await\n    }\n\n    pub(crate) async fn get_counter(\n        &self,\n        key: impl Into<ValueKey<ValueClass>> + Sync + Send,\n    ) -> trc::Result<i64> {\n        let key = key.into();\n        let db = self.db.clone();\n        self.spawn_worker(move || {\n            let cf = self.db.subspace_handle(key.subspace());\n            let key = key.serialize(0);\n\n            db.get_pinned_cf(&cf, &key)\n                .map_err(into_error)\n                .and_then(|bytes| {\n                    Ok(if let Some(bytes) = bytes {\n                        i64::from_le_bytes(bytes[..].try_into().map_err(|_| {\n                            trc::Error::corrupted_key(&key, (&bytes[..]).into(), trc::location!())\n                        })?)\n                    } else {\n                        0\n                    })\n                })\n        })\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/rocksdb/write.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{CF_INDEXES, CF_LOGS, CfHandle, RocksDbStore, into_error};\nuse crate::{\n    Deserialize, IndexKey, Key, LogKey, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER,\n    SUBSPACE_QUOTA,\n    backend::deserialize_i64_le,\n    write::{\n        AssignedIds, Batch, MAX_COMMIT_ATTEMPTS, MAX_COMMIT_TIME, MergeResult, Operation,\n        ValueClass, ValueOp,\n    },\n};\nuse rand::Rng;\nuse rocksdb::{\n    BoundColumnFamily, ErrorKind, IteratorMode, OptimisticTransactionDB,\n    OptimisticTransactionOptions, WriteOptions,\n};\nuse std::{\n    sync::Arc,\n    thread::sleep,\n    time::{Duration, Instant},\n};\n\nimpl RocksDbStore {\n    pub(crate) async fn write(&self, mut batch: Batch<'_>) -> trc::Result<AssignedIds> {\n        let db = self.db.clone();\n\n        self.spawn_worker(move || {\n            let mut txn = RocksDBTransaction {\n                db: &db,\n                cf_indexes: db.cf_handle(CF_INDEXES).unwrap(),\n                cf_logs: db.cf_handle(CF_LOGS).unwrap(),\n                txn_opts: OptimisticTransactionOptions::default(),\n                batch: &mut batch,\n            };\n            txn.txn_opts.set_snapshot(true);\n\n            // Begin write\n            let mut retry_count = 0;\n            let start = Instant::now();\n            loop {\n                match txn.commit() {\n                    Ok(result) => {\n                        return Ok(result);\n                    }\n                    Err(CommitError::Internal(err)) => return Err(err),\n                    Err(CommitError::RocksDB(err)) => match err.kind() {\n                        ErrorKind::Busy | ErrorKind::MergeInProgress | ErrorKind::TryAgain\n                            if retry_count < MAX_COMMIT_ATTEMPTS\n                                && start.elapsed() < MAX_COMMIT_TIME =>\n                        {\n                            let backoff = rand::rng().random_range(50..=300);\n                            sleep(Duration::from_millis(backoff));\n                            retry_count += 1;\n                        }\n                        _ => return Err(into_error(err)),\n                    },\n                }\n            }\n        })\n        .await\n    }\n\n    pub(crate) async fn delete_range(&self, from: impl Key, to: impl Key) -> trc::Result<()> {\n        let db = self.db.clone();\n        self.spawn_worker(move || {\n            db.delete_range_cf(\n                &db.cf_handle(std::str::from_utf8(&[from.subspace()]).unwrap())\n                    .unwrap(),\n                from.serialize(0),\n                to.serialize(0),\n            )\n            .map_err(into_error)\n        })\n        .await\n    }\n\n    pub(crate) async fn purge_store(&self) -> trc::Result<()> {\n        let db = self.db.clone();\n        self.spawn_worker(move || {\n            for subspace in [SUBSPACE_QUOTA, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER] {\n                let cf = db\n                    .cf_handle(std::str::from_utf8(&[subspace]).unwrap())\n                    .unwrap();\n\n                let mut delete_keys = Vec::new();\n\n                for row in db.iterator_cf(&cf, IteratorMode::Start) {\n                    let (key, value) = row.map_err(into_error)?;\n\n                    if i64::deserialize(&value)? == 0 {\n                        delete_keys.push(key);\n                    }\n                }\n\n                let txn_opts = OptimisticTransactionOptions::default();\n                for key in delete_keys {\n                    let txn = db.transaction_opt(&WriteOptions::default(), &txn_opts);\n                    if txn\n                        .get_pinned_for_update_cf(&cf, &key, true)\n                        .map_err(into_error)?\n                        .map(|value| i64::deserialize(&value).map(|v| v == 0).unwrap_or(false))\n                        .unwrap_or(false)\n                    {\n                        txn.delete_cf(&cf, key).map_err(into_error)?;\n                        txn.commit().map_err(into_error)?;\n                    } else {\n                        txn.rollback().map_err(into_error)?;\n                    }\n                }\n            }\n\n            Ok(())\n        })\n        .await\n    }\n}\n\nstruct RocksDBTransaction<'x, 'y> {\n    db: &'x OptimisticTransactionDB,\n    cf_indexes: Arc<BoundColumnFamily<'x>>,\n    cf_logs: Arc<BoundColumnFamily<'x>>,\n    txn_opts: OptimisticTransactionOptions,\n    batch: &'x mut Batch<'y>,\n}\n\nenum CommitError {\n    Internal(trc::Error),\n    RocksDB(rocksdb::Error),\n}\n\nimpl RocksDBTransaction<'_, '_> {\n    fn commit(&mut self) -> Result<AssignedIds, CommitError> {\n        let mut account_id = u32::MAX;\n        let mut collection = u8::MAX;\n        let mut document_id = u32::MAX;\n        let mut change_id = 0u64;\n        let mut result = AssignedIds::default();\n        let has_changes = !self.batch.changes.is_empty();\n\n        let txn = self\n            .db\n            .transaction_opt(&WriteOptions::default(), &self.txn_opts);\n\n        if has_changes {\n            let cf = self.db.cf_handle(\"n\").unwrap();\n            for &account_id in self.batch.changes.keys() {\n                let key = ValueClass::ChangeId.serialize(account_id, 0, 0, 0);\n                let change_id = txn\n                    .get_pinned_for_update_cf(&cf, &key, true)\n                    .map_err(CommitError::from)\n                    .and_then(|bytes| {\n                        if let Some(bytes) = bytes {\n                            deserialize_i64_le(&key, &bytes)\n                                .map(|v| v + 1)\n                                .map_err(CommitError::from)\n                        } else {\n                            Ok(1)\n                        }\n                    })?;\n                txn.put_cf(&cf, &key, &change_id.to_le_bytes()[..])?;\n                result.push_change_id(account_id, change_id as u64);\n            }\n        }\n\n        for op in self.batch.ops.iter_mut() {\n            match op {\n                Operation::AccountId {\n                    account_id: account_id_,\n                } => {\n                    account_id = *account_id_;\n                    if has_changes {\n                        change_id = result.set_current_change_id(account_id)?;\n                    }\n                }\n                Operation::Collection {\n                    collection: collection_,\n                } => {\n                    collection = u8::from(*collection_);\n                }\n                Operation::DocumentId {\n                    document_id: document_id_,\n                } => {\n                    document_id = *document_id_;\n                }\n                Operation::Value { class, op } => {\n                    let key = class.serialize(account_id, collection, document_id, 0);\n                    let cf = self.db.subspace_handle(class.subspace(collection));\n\n                    match op {\n                        ValueOp::Set(value) => {\n                            txn.put_cf(&cf, &key, value)?;\n                        }\n                        ValueOp::SetFnc(set_op) => {\n                            let value = (set_op.fnc)(&set_op.params, &result)?;\n\n                            txn.put_cf(&cf, &key, value)?;\n                        }\n                        ValueOp::MergeFnc(merge_op) => {\n                            let merge_result = (merge_op.fnc)(\n                                &merge_op.params,\n                                &result,\n                                txn.get_pinned_for_update_cf(&cf, &key, true)?.as_deref(),\n                            )?;\n\n                            match merge_result {\n                                MergeResult::Update(value) => {\n                                    txn.put_cf(&cf, &key, value)?;\n                                }\n                                MergeResult::Delete => {\n                                    txn.delete_cf(&cf, &key)?;\n                                }\n                                MergeResult::Skip => (),\n                            }\n                        }\n                        ValueOp::AtomicAdd(by) => {\n                            txn.merge_cf(&cf, &key, &by.to_le_bytes()[..])?;\n                        }\n                        ValueOp::AddAndGet(by) => {\n                            let num = txn\n                                .get_pinned_for_update_cf(&cf, &key, true)\n                                .map_err(CommitError::from)\n                                .and_then(|bytes| {\n                                    if let Some(bytes) = bytes {\n                                        deserialize_i64_le(&key, &bytes)\n                                            .map(|v| v + *by)\n                                            .map_err(CommitError::from)\n                                    } else {\n                                        Ok(*by)\n                                    }\n                                })?;\n                            txn.put_cf(&cf, &key, &num.to_le_bytes()[..])?;\n                            result.push_counter_id(num);\n                        }\n                        ValueOp::Clear => {\n                            txn.delete_cf(&cf, &key)?;\n                        }\n                    }\n                }\n                Operation::Index { field, key, set } => {\n                    let key = IndexKey {\n                        account_id,\n                        collection,\n                        document_id,\n                        field: *field,\n                        key: &*key,\n                    }\n                    .serialize(0);\n\n                    if *set {\n                        txn.put_cf(&self.cf_indexes, &key, [])?;\n                    } else {\n                        txn.delete_cf(&self.cf_indexes, &key)?;\n                    }\n                }\n                Operation::Log { collection, set } => {\n                    let key = LogKey {\n                        account_id,\n                        collection: u8::from(*collection),\n                        change_id,\n                    }\n                    .serialize(0);\n\n                    txn.put_cf(&self.cf_logs, &key, set)?;\n                }\n                Operation::AssertValue {\n                    class,\n                    assert_value,\n                } => {\n                    let key = class.serialize(account_id, collection, document_id, 0);\n                    let cf = self.db.subspace_handle(class.subspace(collection));\n\n                    let matches = txn\n                        .get_pinned_for_update_cf(&cf, &key, true)?\n                        .map(|value| assert_value.matches(&value))\n                        .unwrap_or_else(|| assert_value.is_none());\n\n                    if !matches {\n                        txn.rollback()?;\n                        return Err(CommitError::Internal(\n                            trc::StoreEvent::AssertValueFailed.into(),\n                        ));\n                    }\n                }\n            }\n        }\n\n        txn.commit().map(|_| result).map_err(Into::into)\n    }\n}\n\nimpl From<rocksdb::Error> for CommitError {\n    fn from(err: rocksdb::Error) -> Self {\n        CommitError::RocksDB(err)\n    }\n}\n\nimpl From<trc::Error> for CommitError {\n    fn from(err: trc::Error) -> Self {\n        CommitError::Internal(err)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/s3/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse s3::{Bucket, Region, creds::Credentials};\nuse std::{fmt::Display, io::Write, ops::Range, time::Duration};\nuse utils::{\n    codec::base32_custom::Base32Writer,\n    config::{Config, utils::AsKey},\n};\n\npub struct S3Store {\n    bucket: Box<Bucket>,\n    prefix: Option<String>,\n    max_retries: u32,\n}\n\nimpl S3Store {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        // Obtain region and endpoint from config\n        let prefix = prefix.as_key();\n        let region = config.value_require((&prefix, \"region\"))?.to_string();\n        let region = if let Some(endpoint) = config.value((&prefix, \"endpoint\")) {\n            Region::Custom {\n                region: region.to_string(),\n                endpoint: endpoint.to_string(),\n            }\n        } else {\n            region.parse().unwrap()\n        };\n        let credentials = Credentials::new(\n            config.value((&prefix, \"access-key\")),\n            config.value((&prefix, \"secret-key\")),\n            config.value((&prefix, \"security-token\")),\n            config.value((&prefix, \"session-token\")),\n            config.value((&prefix, \"profile\")),\n        )\n        .map_err(|err| {\n            config.new_build_error(\n                prefix.as_str(),\n                format!(\"Failed to create credentials: {err:?}\"),\n            )\n        })\n        .ok()?;\n        let timeout = config\n            .property_or_default::<Duration>((&prefix, \"timeout\"), \"30s\")\n            .unwrap_or_else(|| Duration::from_secs(30));\n        /*let allow_invalid = config\n        .property_or_default::<bool>((&prefix, \"tls.allow-invalid\"), \"false\")\n        .unwrap_or_default();*/\n\n        Some(S3Store {\n            bucket: Bucket::new(\n                config.value_require((&prefix, \"bucket\"))?,\n                region,\n                credentials,\n            )\n            .map_err(|err| {\n                config.new_build_error(prefix.as_str(), format!(\"Failed to create bucket: {err:?}\"))\n            })\n            .ok()?\n            .with_path_style()\n            /*.set_dangereous_config(allow_invalid, allow_invalid)\n            .map_err(|err| {\n                config.new_build_error(prefix.as_str(), format!(\"Failed to create bucket: {err:?}\"))\n            })\n            .ok()?*/\n            .with_request_timeout(timeout)\n            .map_err(|err| {\n                config.new_build_error(prefix.as_str(), format!(\"Failed to create bucket: {err:?}\"))\n            })\n            .ok()?,\n            max_retries: config\n                .property_or_default((&prefix, \"max-retries\"), \"3\")\n                .unwrap_or(3),\n            prefix: config.value((&prefix, \"key-prefix\")).map(|s| s.to_string()),\n        })\n    }\n\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let path = self.build_key(key);\n        let mut retries_left = self.max_retries;\n\n        loop {\n            let response = if range.start != 0 || range.end != usize::MAX {\n                self.bucket\n                    .get_object_range(\n                        &path,\n                        range.start as u64,\n                        Some(range.end.saturating_sub(1) as u64),\n                    )\n                    .await\n            } else {\n                self.bucket.get_object(&path).await\n            }\n            .map_err(into_error)?;\n\n            match response.status_code() {\n                200..=299 => return Ok(Some(response.to_vec())),\n                404 => return Ok(None),\n                500..=599 if retries_left > 0 => {\n                    // wait backoff\n                    tokio::time::sleep(Duration::from_secs(\n                        1 << (self.max_retries - retries_left).min(6),\n                    ))\n                    .await;\n\n                    retries_left -= 1;\n                }\n                code => {\n                    return Err(trc::StoreEvent::S3Error\n                        .reason(String::from_utf8_lossy(response.as_slice()))\n                        .ctx(trc::Key::Code, code));\n                }\n            }\n        }\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let mut retries_left = self.max_retries;\n\n        loop {\n            let response = self\n                .bucket\n                .put_object(self.build_key(key), data)\n                .await\n                .map_err(into_error)?;\n\n            match response.status_code() {\n                200..=299 => return Ok(()),\n                500..=599 if retries_left > 0 => {\n                    // wait backoff\n                    tokio::time::sleep(Duration::from_secs(\n                        1 << (self.max_retries - retries_left).min(6),\n                    ))\n                    .await;\n\n                    retries_left -= 1;\n                }\n                code => {\n                    return Err(trc::StoreEvent::S3Error\n                        .reason(String::from_utf8_lossy(response.as_slice()))\n                        .ctx(trc::Key::Code, code));\n                }\n            }\n        }\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let mut retries_left = self.max_retries;\n\n        loop {\n            let response = self\n                .bucket\n                .delete_object(self.build_key(key))\n                .await\n                .map_err(into_error)?;\n\n            match response.status_code() {\n                200..=299 => return Ok(true),\n                404 => return Ok(false),\n                500..=599 if retries_left > 0 => {\n                    // wait backoff\n                    tokio::time::sleep(Duration::from_secs(\n                        1 << (self.max_retries - retries_left).min(6),\n                    ))\n                    .await;\n\n                    retries_left -= 1;\n                }\n                code => {\n                    return Err(trc::StoreEvent::S3Error\n                        .reason(String::from_utf8_lossy(response.as_slice()))\n                        .ctx(trc::Key::Code, code));\n                }\n            }\n        }\n    }\n\n    fn build_key(&self, key: &[u8]) -> String {\n        if let Some(prefix) = &self.prefix {\n            let mut writer =\n                Base32Writer::with_raw_capacity(prefix.len() + (key.len().div_ceil(4) * 5));\n            writer.push_string(prefix);\n            writer.write_all(key).unwrap();\n            writer.finalize()\n        } else {\n            Base32Writer::from_bytes(key).finalize()\n        }\n    }\n}\n\n#[inline(always)]\nfn into_error(err: impl Display) -> trc::Error {\n    trc::StoreEvent::S3Error.reason(err)\n}\n"
  },
  {
    "path": "crates/store/src/backend/sqlite/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::ops::Range;\n\nuse rusqlite::OptionalExtension;\n\nuse super::{SqliteStore, into_error};\n\nimpl SqliteStore {\n    pub(crate) async fn get_blob(\n        &self,\n        key: &[u8],\n        range: Range<usize>,\n    ) -> trc::Result<Option<Vec<u8>>> {\n        let conn = self.conn_pool.get().map_err(into_error)?;\n        self.spawn_worker(move || {\n            let mut result = conn\n                .prepare_cached(\"SELECT v FROM t WHERE k = ?\")\n                .map_err(into_error)?;\n            result\n                .query_row([&key], |row| {\n                    Ok({\n                        let bytes = row.get_ref(0)?.as_bytes()?;\n                        if range.start == 0 && range.end == usize::MAX {\n                            bytes.to_vec()\n                        } else {\n                            bytes\n                                .get(range.start..std::cmp::min(bytes.len(), range.end))\n                                .unwrap_or_default()\n                                .to_vec()\n                        }\n                    })\n                })\n                .optional()\n                .map_err(into_error)\n        })\n        .await\n    }\n\n    pub(crate) async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let conn = self.conn_pool.get().map_err(into_error)?;\n        self.spawn_worker(move || {\n            conn.prepare_cached(\"INSERT OR REPLACE INTO t (k, v) VALUES (?, ?)\")\n                .map_err(into_error)?\n                .execute([key, data])\n                .map_err(into_error)\n                .map(|_| ())\n        })\n        .await\n    }\n\n    pub(crate) async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let conn = self.conn_pool.get().map_err(into_error)?;\n        self.spawn_worker(move || {\n            conn.prepare_cached(\"DELETE FROM t WHERE k = ?\")\n                .map_err(into_error)?\n                .execute([key])\n                .map_err(into_error)\n                .map(|_| true)\n        })\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/sqlite/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse rusqlite::{Row, Rows, ToSql, types::FromSql};\n\nuse crate::{IntoRows, QueryResult, QueryType, Value};\n\nuse super::{SqliteStore, into_error};\n\nimpl SqliteStore {\n    pub(crate) async fn sql_query<T: QueryResult>(\n        &self,\n        query: &str,\n        params_: &[Value<'_>],\n    ) -> trc::Result<T> {\n        let conn = self.conn_pool.get().map_err(into_error)?;\n        self.spawn_worker(move || {\n            let mut s = conn.prepare_cached(query).map_err(into_error)?;\n            let params = params_\n                .iter()\n                .map(|v| v as &dyn rusqlite::types::ToSql)\n                .collect::<Vec<_>>();\n\n            match T::query_type() {\n                QueryType::Execute => s\n                    .execute(params.as_slice())\n                    .map_or_else(|e| Err(into_error(e)), |r| Ok(T::from_exec(r))),\n                QueryType::Exists => s\n                    .exists(params.as_slice())\n                    .map(T::from_exists)\n                    .map_err(into_error),\n                QueryType::QueryOne => s\n                    .query(params.as_slice())\n                    .and_then(|mut rows| Ok(T::from_query_one(rows.next()?)))\n                    .map_err(into_error),\n                QueryType::QueryAll => Ok(T::from_query_all(\n                    s.query(params.as_slice()).map_err(into_error)?,\n                )),\n            }\n        })\n        .await\n    }\n}\n\nimpl ToSql for Value<'_> {\n    fn to_sql(&self) -> rusqlite::Result<rusqlite::types::ToSqlOutput<'_>> {\n        match self {\n            Value::Integer(value) => value.to_sql(),\n            Value::Bool(value) => value.to_sql(),\n            Value::Float(value) => value.to_sql(),\n            Value::Text(value) => value.to_sql(),\n            Value::Blob(value) => value.to_sql(),\n            Value::Null => Ok(rusqlite::types::ToSqlOutput::Owned(\n                rusqlite::types::Value::Null,\n            )),\n        }\n    }\n}\n\nimpl FromSql for Value<'static> {\n    fn column_result(value: rusqlite::types::ValueRef<'_>) -> rusqlite::types::FromSqlResult<Self> {\n        Ok(match value {\n            rusqlite::types::ValueRef::Null => Value::Null,\n            rusqlite::types::ValueRef::Integer(v) => Value::Integer(v),\n            rusqlite::types::ValueRef::Real(v) => Value::Float(v),\n            rusqlite::types::ValueRef::Text(v) => {\n                Value::Text(String::from_utf8_lossy(v).into_owned().into())\n            }\n            rusqlite::types::ValueRef::Blob(v) => Value::Blob(v.to_vec().into()),\n        })\n    }\n}\n\nimpl IntoRows for Rows<'_> {\n    fn into_rows(mut self) -> crate::Rows {\n        let column_count = self.as_ref().map(|s| s.column_count()).unwrap_or_default();\n        let mut rows = crate::Rows { rows: Vec::new() };\n\n        while let Ok(Some(row)) = self.next() {\n            rows.rows.push(crate::Row {\n                values: (0..column_count)\n                    .map(|idx| row.get::<_, Value>(idx).unwrap_or(Value::Null))\n                    .collect(),\n            });\n        }\n\n        rows\n    }\n\n    fn into_named_rows(mut self) -> crate::NamedRows {\n        let (column_count, names) = self\n            .as_ref()\n            .map(|s| {\n                (\n                    s.column_count(),\n                    s.column_names()\n                        .into_iter()\n                        .map(String::from)\n                        .collect::<Vec<_>>(),\n                )\n            })\n            .unwrap_or((0, Vec::new()));\n\n        let mut rows = crate::NamedRows {\n            names,\n            rows: Vec::new(),\n        };\n\n        while let Ok(Some(row)) = self.next() {\n            rows.rows.push(crate::Row {\n                values: (0..column_count)\n                    .map(|idx| row.get::<_, Value>(idx).unwrap_or(Value::Null))\n                    .collect(),\n            });\n        }\n\n        rows\n    }\n\n    fn into_row(self) -> Option<crate::Row> {\n        unreachable!()\n    }\n}\n\nimpl IntoRows for Option<&Row<'_>> {\n    fn into_row(self) -> Option<crate::Row> {\n        self.map(|row| crate::Row {\n            values: (0..row.as_ref().column_count())\n                .map(|idx| row.get::<_, Value>(idx).unwrap_or(Value::Null))\n                .collect(),\n        })\n    }\n\n    fn into_rows(self) -> crate::Rows {\n        unreachable!()\n    }\n\n    fn into_named_rows(self) -> crate::NamedRows {\n        unreachable!()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/sqlite/main.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse r2d2::Pool;\nuse tokio::sync::oneshot;\nuse utils::config::{Config, utils::AsKey};\n\nuse crate::*;\n\nuse super::{SqliteStore, into_error, pool::SqliteConnectionManager};\n\nimpl SqliteStore {\n    pub fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let db = Self {\n            conn_pool: Pool::builder()\n                .max_size(\n                    config\n                        .property((&prefix, \"pool.max-connections\"))\n                        .unwrap_or_else(|| (num_cpus::get() * 4) as u32),\n                )\n                .build(\n                    SqliteConnectionManager::file(config.value_require((&prefix, \"path\"))?)\n                        .with_init(|c| {\n                            c.execute_batch(concat!(\n                                \"PRAGMA journal_mode = WAL; \",\n                                \"PRAGMA synchronous = NORMAL; \",\n                                \"PRAGMA temp_store = memory;\",\n                                \"PRAGMA busy_timeout = 30000;\"\n                            ))\n                        }),\n                )\n                .map_err(|err| {\n                    config.new_build_error(\n                        prefix.as_str(),\n                        format!(\"Failed to build connection pool: {err}\"),\n                    )\n                })\n                .ok()?,\n            worker_pool: rayon::ThreadPoolBuilder::new()\n                .num_threads(std::cmp::max(\n                    config\n                        .property::<usize>((&prefix, \"pool.workers\"))\n                        .filter(|v| *v > 0)\n                        .unwrap_or_else(num_cpus::get),\n                    4,\n                ))\n                .build()\n                .map_err(|err| {\n                    config.new_build_error(\n                        prefix.as_str(),\n                        format!(\"Failed to build worker pool: {err}\"),\n                    )\n                })\n                .ok()?,\n        };\n\n        if let Err(err) = db.create_tables() {\n            config.new_build_error(prefix.as_str(), format!(\"Failed to create tables: {err}\"));\n        }\n\n        Some(db)\n    }\n\n    #[cfg(feature = \"test_mode\")]\n    pub fn open_memory() -> trc::Result<Self> {\n        use super::into_error;\n\n        let db = Self {\n            conn_pool: Pool::builder()\n                .max_size(1)\n                .build(SqliteConnectionManager::memory())\n                .map_err(into_error)?,\n            worker_pool: rayon::ThreadPoolBuilder::new()\n                .num_threads(num_cpus::get())\n                .build()\n                .map_err(|err| {\n                    into_error(err).ctx(trc::Key::Reason, \"Failed to build worker pool\")\n                })?,\n        };\n        db.create_tables()?;\n        Ok(db)\n    }\n\n    pub(super) fn create_tables(&self) -> trc::Result<()> {\n        let conn = self.conn_pool.get().map_err(into_error)?;\n\n        for table in [\n            SUBSPACE_ACL,\n            SUBSPACE_DIRECTORY,\n            SUBSPACE_TASK_QUEUE,\n            SUBSPACE_BLOB_EXTRA,\n            SUBSPACE_BLOB_LINK,\n            SUBSPACE_IN_MEMORY_VALUE,\n            SUBSPACE_PROPERTY,\n            SUBSPACE_SETTINGS,\n            SUBSPACE_QUEUE_MESSAGE,\n            SUBSPACE_QUEUE_EVENT,\n            SUBSPACE_REPORT_OUT,\n            SUBSPACE_REPORT_IN,\n            SUBSPACE_LOGS,\n            SUBSPACE_BLOBS,\n            SUBSPACE_TELEMETRY_SPAN,\n            SUBSPACE_TELEMETRY_METRIC,\n            SUBSPACE_SEARCH_INDEX,\n        ] {\n            let table = char::from(table);\n            conn.execute(\n                &format!(\n                    \"CREATE TABLE IF NOT EXISTS {table} (\n                        k BLOB PRIMARY KEY,\n                        v BLOB NOT NULL\n                    )\"\n                ),\n                [],\n            )\n            .map_err(into_error)?;\n        }\n\n        let table = char::from(SUBSPACE_INDEXES);\n        conn.execute(\n            &format!(\n                \"CREATE TABLE IF NOT EXISTS {table} (\n                        k BLOB PRIMARY KEY\n                )\"\n            ),\n            [],\n        )\n        .map_err(into_error)?;\n\n        for table in [SUBSPACE_COUNTER, SUBSPACE_QUOTA, SUBSPACE_IN_MEMORY_COUNTER] {\n            conn.execute(\n                &format!(\n                    \"CREATE TABLE IF NOT EXISTS {} (\n                    k BLOB PRIMARY KEY,\n                    v INTEGER NOT NULL DEFAULT 0\n                )\",\n                    char::from(table)\n                ),\n                [],\n            )\n            .map_err(into_error)?;\n        }\n\n        Ok(())\n    }\n\n    pub async fn spawn_worker<U, V>(&self, mut f: U) -> trc::Result<V>\n    where\n        U: FnMut() -> trc::Result<V> + Send,\n        V: Sync + Send + 'static,\n    {\n        let (tx, rx) = oneshot::channel();\n\n        self.worker_pool.scope(|s| {\n            s.spawn(|_| {\n                tx.send(f()).ok();\n            });\n        });\n\n        match rx.await {\n            Ok(result) => result,\n            Err(err) => Err(trc::EventType::Server(trc::ServerEvent::ThreadError).reason(err)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/sqlite/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse r2d2::Pool;\n\nuse self::pool::SqliteConnectionManager;\n\npub mod blob;\npub mod lookup;\npub mod main;\npub mod pool;\npub mod read;\npub mod write;\n\npub struct SqliteStore {\n    pub(crate) conn_pool: Pool<SqliteConnectionManager>,\n    pub(crate) worker_pool: rayon::ThreadPool,\n}\n\n#[inline(always)]\nfn into_error(err: impl Display) -> trc::Error {\n    trc::StoreEvent::SqliteError.reason(err)\n}\n"
  },
  {
    "path": "crates/store/src/backend/sqlite/pool.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse rusqlite::{Connection, Error, OpenFlags};\nuse std::fmt;\nuse std::path::{Path, PathBuf};\n\n#[derive(Debug)]\nenum Source {\n    File(PathBuf),\n    Memory,\n}\n\ntype InitFn = dyn Fn(&mut Connection) -> Result<(), rusqlite::Error> + Send + Sync + 'static;\n\n/// An `r2d2::ManageConnection` for `rusqlite::Connection`s.\npub struct SqliteConnectionManager {\n    source: Source,\n    flags: OpenFlags,\n    init: Option<Box<InitFn>>,\n}\n\nimpl fmt::Debug for SqliteConnectionManager {\n    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {\n        let mut builder = f.debug_struct(\"SqliteConnectionManager\");\n        let _ = builder.field(\"source\", &self.source);\n        let _ = builder.field(\"flags\", &self.source);\n        let _ = builder.field(\"init\", &self.init.as_ref().map(|_| \"InitFn\"));\n        builder.finish()\n    }\n}\n\nimpl SqliteConnectionManager {\n    /// Creates a new `SqliteConnectionManager` from file.\n    ///\n    /// See `rusqlite::Connection::open`\n    pub fn file<P: AsRef<Path>>(path: P) -> Self {\n        Self {\n            source: Source::File(path.as_ref().to_path_buf()),\n            flags: OpenFlags::default(),\n            init: None,\n        }\n    }\n\n    /// Creates a new `SqliteConnectionManager` from memory.\n    pub fn memory() -> Self {\n        Self {\n            source: Source::Memory,\n            flags: OpenFlags::default(),\n            init: None,\n        }\n    }\n\n    /// Converts `SqliteConnectionManager` into one that sets OpenFlags upon\n    /// connection creation.\n    ///\n    /// See `rustqlite::OpenFlags` for a list of available flags.\n    pub fn with_flags(self, flags: OpenFlags) -> Self {\n        Self { flags, ..self }\n    }\n\n    /// Converts `SqliteConnectionManager` into one that calls an initialization\n    /// function upon connection creation. Could be used to set PRAGMAs, for\n    /// example.\n    ///\n    /// ### Example\n    ///\n    /// Make a `SqliteConnectionManager` that sets the `foreign_keys` pragma to\n    /// true for every connection.\n    ///\n    /// ```rust,no_run\n    /// # use r2d2_sqlite::{SqliteConnectionManager};\n    /// let manager = SqliteConnectionManager::file(\"app.db\")\n    ///     .with_init(|c| c.execute_batch(\"PRAGMA foreign_keys=1;\"));\n    /// ```\n    pub fn with_init<F>(self, init: F) -> Self\n    where\n        F: Fn(&mut Connection) -> Result<(), rusqlite::Error> + Send + Sync + 'static,\n    {\n        let init: Option<Box<InitFn>> = Some(Box::new(init));\n        Self { init, ..self }\n    }\n}\n\nfn sleeper(_: i32) -> bool {\n    std::thread::sleep(std::time::Duration::from_millis(200));\n    true\n}\n\nimpl r2d2::ManageConnection for SqliteConnectionManager {\n    type Connection = Connection;\n    type Error = rusqlite::Error;\n\n    fn connect(&self) -> Result<Connection, Error> {\n        match self.source {\n            Source::File(ref path) => Connection::open_with_flags(path, self.flags),\n            Source::Memory => Connection::open_in_memory_with_flags(self.flags),\n        }\n        .and_then(|mut c| {\n            c.busy_handler(Some(sleeper))?;\n            match self.init {\n                None => Ok(c),\n                Some(ref init) => init(&mut c).map(|_| c),\n            }\n        })\n    }\n\n    fn is_valid(&self, conn: &mut Connection) -> Result<(), Error> {\n        conn.execute_batch(\"\")\n    }\n\n    fn has_broken(&self, _: &mut Connection) -> bool {\n        false\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/sqlite/read.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{SqliteStore, into_error};\nuse crate::{Deserialize, IterateParams, Key, ValueKey, write::ValueClass};\nuse rusqlite::OptionalExtension;\n\nimpl SqliteStore {\n    pub(crate) async fn get_value<U>(&self, key: impl Key) -> trc::Result<Option<U>>\n    where\n        U: Deserialize + 'static,\n    {\n        let conn = self.conn_pool.get().map_err(into_error)?;\n        self.spawn_worker(move || {\n            let mut result = conn\n                .prepare_cached(&format!(\n                    \"SELECT v FROM {} WHERE k = ?\",\n                    char::from(key.subspace())\n                ))\n                .map_err(into_error)?;\n            let key = key.serialize(0);\n            result\n                .query_row([&key], |row| {\n                    U::deserialize(row.get_ref(0)?.as_bytes()?)\n                        .map_err(|err| rusqlite::Error::ToSqlConversionFailure(err.into()))\n                })\n                .optional()\n                .map_err(into_error)\n        })\n        .await\n    }\n\n    pub(crate) async fn iterate<T: Key>(\n        &self,\n        params: IterateParams<T>,\n        mut cb: impl for<'x> FnMut(&'x [u8], &'x [u8]) -> trc::Result<bool> + Sync + Send,\n    ) -> trc::Result<()> {\n        let conn = self.conn_pool.get().map_err(into_error)?;\n\n        self.spawn_worker(move || {\n            let table = char::from(params.begin.subspace());\n            let begin = params.begin.serialize(0);\n            let end = params.end.serialize(0);\n            let keys = if params.values { \"k, v\" } else { \"k\" };\n\n            let mut query = conn\n                .prepare_cached(&match (params.first, params.ascending) {\n                    (true, true) => {\n                        format!(\n                        \"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k ASC LIMIT 1\"\n                    )\n                    }\n                    (true, false) => {\n                        format!(\n                        \"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k DESC LIMIT 1\"\n                    )\n                    }\n                    (false, true) => {\n                        format!(\"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k ASC\")\n                    }\n                    (false, false) => {\n                        format!(\n                            \"SELECT {keys} FROM {table} WHERE k >= ? AND k <= ? ORDER BY k DESC\"\n                        )\n                    }\n                })\n                .map_err(into_error)?;\n            let mut rows = query.query([&begin, &end]).map_err(into_error)?;\n\n            if params.values {\n                while let Some(row) = rows.next().map_err(into_error)? {\n                    let key = row\n                        .get_ref(0)\n                        .map_err(into_error)?\n                        .as_bytes()\n                        .map_err(into_error)?;\n                    let value = row\n                        .get_ref(1)\n                        .map_err(into_error)?\n                        .as_bytes()\n                        .map_err(into_error)?;\n\n                    if !cb(key, value)? {\n                        break;\n                    }\n                }\n            } else {\n                while let Some(row) = rows.next().map_err(into_error)? {\n                    if !cb(\n                        row.get_ref(0)\n                            .map_err(into_error)?\n                            .as_bytes()\n                            .map_err(into_error)?,\n                        b\"\",\n                    )? {\n                        break;\n                    }\n                }\n            }\n\n            Ok(())\n        })\n        .await\n    }\n\n    pub(crate) async fn get_counter(\n        &self,\n        key: impl Into<ValueKey<ValueClass>> + Sync + Send,\n    ) -> trc::Result<i64> {\n        let key = key.into();\n        let table = char::from(key.subspace());\n        let key = key.serialize(0);\n        let conn = self.conn_pool.get().map_err(into_error)?;\n        self.spawn_worker(move || {\n            match conn\n                .prepare_cached(&format!(\"SELECT v FROM {table} WHERE k = ?\"))\n                .map_err(into_error)?\n                .query_row([&key], |row| row.get::<_, i64>(0))\n            {\n                Ok(value) => Ok(value),\n                Err(rusqlite::Error::QueryReturnedNoRows) => Ok(0),\n                Err(e) => Err(into_error(e)),\n            }\n        })\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/sqlite/write.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{SqliteStore, into_error};\nuse crate::{\n    IndexKey, Key, LogKey, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER, SUBSPACE_QUOTA,\n    write::{AssignedIds, Batch, MergeResult, Operation, ValueClass, ValueOp},\n};\nuse rusqlite::{OptionalExtension, TransactionBehavior, params};\nuse trc::AddContext;\n\nimpl SqliteStore {\n    pub(crate) async fn write(&self, batch: Batch<'_>) -> trc::Result<AssignedIds> {\n        let mut conn = self\n            .conn_pool\n            .get()\n            .map_err(into_error)\n            .caused_by(trc::location!())?;\n        self.spawn_worker(move || {\n            let mut account_id = u32::MAX;\n            let mut collection = u8::MAX;\n            let mut document_id = u32::MAX;\n            let mut change_id = 0u64;\n            let trx = conn\n                .transaction_with_behavior(TransactionBehavior::Immediate)\n                .map_err(into_error)\n                .caused_by(trc::location!())?;\n            let mut result = AssignedIds::default();\n            let has_changes = !batch.changes.is_empty();\n\n            if has_changes {\n                for &account_id in batch.changes.keys() {\n                    let key = ValueClass::ChangeId.serialize(account_id, 0, 0, 0);\n                    let change_id = trx\n                        .prepare_cached(concat!(\n                            \"INSERT INTO n (k, v) VALUES (?, ?) \",\n                            \"ON CONFLICT(k) DO UPDATE SET v = v + \",\n                            \"excluded.v RETURNING v\"\n                        ))\n                        .map_err(into_error)\n                        .caused_by(trc::location!())?\n                        .query_row(params![&key, &1i64], |row| row.get::<_, i64>(0))\n                        .map_err(into_error)\n                        .caused_by(trc::location!())?;\n                    result.push_change_id(account_id, change_id as u64);\n                }\n            }\n\n            for op in batch.ops.iter_mut() {\n                match op {\n                    Operation::AccountId {\n                        account_id: account_id_,\n                    } => {\n                        account_id = *account_id_;\n                        if has_changes {\n                            change_id = result.set_current_change_id(account_id)?;\n                        }\n                    }\n                    Operation::Collection {\n                        collection: collection_,\n                    } => {\n                        collection = u8::from(*collection_);\n                    }\n                    Operation::DocumentId {\n                        document_id: document_id_,\n                    } => {\n                        document_id = *document_id_;\n                    }\n                    Operation::Value { class, op } => {\n                        let key = class.serialize(account_id, collection, document_id, 0);\n                        let table = char::from(class.subspace(collection));\n\n                        match op {\n                            ValueOp::Set(value) => {\n                                trx.prepare_cached(&format!(\n                                    \"INSERT OR REPLACE INTO {} (k, v) VALUES (?, ?)\",\n                                    table\n                                ))\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?\n                                .execute([&key, value])\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?;\n                            }\n                            ValueOp::SetFnc(set_op) => {\n                                let value = (set_op.fnc)(&set_op.params, &result)?;\n                                trx.prepare_cached(&format!(\n                                    \"INSERT OR REPLACE INTO {} (k, v) VALUES (?, ?)\",\n                                    table\n                                ))\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?\n                                .execute([&key, &value])\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?;\n                            }\n                            ValueOp::MergeFnc(merge_op) => {\n                                let merge_result = trx\n                                    .prepare_cached(&format!(\"SELECT v FROM {} WHERE k = ?\", table))\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?\n                                    .query_row([&key], |row| {\n                                        Ok((merge_op.fnc)(\n                                            &merge_op.params,\n                                            &result,\n                                            Some(row.get_ref(0)?.as_bytes()?),\n                                        ))\n                                    })\n                                    .optional()\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?\n                                    .unwrap_or_else(|| {\n                                        (merge_op.fnc)(&merge_op.params, &result, None)\n                                    })?;\n\n                                match merge_result {\n                                    MergeResult::Update(value) => {\n                                        trx.prepare_cached(&format!(\n                                            \"INSERT OR REPLACE INTO {} (k, v) VALUES (?, ?)\",\n                                            table\n                                        ))\n                                        .map_err(into_error)\n                                        .caused_by(trc::location!())?\n                                        .execute([&key, &value])\n                                        .map_err(into_error)\n                                        .caused_by(trc::location!())?;\n                                    }\n                                    MergeResult::Delete => {\n                                        trx.prepare_cached(&format!(\n                                            \"DELETE FROM {} WHERE k = ?\",\n                                            table\n                                        ))\n                                        .map_err(into_error)\n                                        .caused_by(trc::location!())?\n                                        .execute([&key])\n                                        .map_err(into_error)\n                                        .caused_by(trc::location!())?;\n                                    }\n                                    MergeResult::Skip => (),\n                                }\n                            }\n                            ValueOp::AtomicAdd(by) => {\n                                if *by >= 0 {\n                                    trx.prepare_cached(&format!(\n                                        concat!(\n                                            \"INSERT INTO {} (k, v) VALUES (?, ?) \",\n                                            \"ON CONFLICT(k) DO UPDATE SET v = v + excluded.v\"\n                                        ),\n                                        table\n                                    ))\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?\n                                    .execute(params![&key, *by])\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?;\n                                } else {\n                                    trx.prepare_cached(&format!(\n                                        \"UPDATE {table} SET v = v + ? WHERE k = ?\"\n                                    ))\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?\n                                    .execute(params![*by, &key])\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?;\n                                }\n                            }\n                            ValueOp::AddAndGet(by) => {\n                                result.push_counter_id(\n                                    trx.prepare_cached(&format!(\n                                        concat!(\n                                            \"INSERT INTO {} (k, v) VALUES (?, ?) \",\n                                            \"ON CONFLICT(k) DO UPDATE SET v = v + \",\n                                            \"excluded.v RETURNING v\"\n                                        ),\n                                        table\n                                    ))\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?\n                                    .query_row(params![&key, &*by], |row| row.get::<_, i64>(0))\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?,\n                                );\n                            }\n                            ValueOp::Clear => {\n                                trx.prepare_cached(&format!(\"DELETE FROM {} WHERE k = ?\", table))\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?\n                                    .execute([&key])\n                                    .map_err(into_error)\n                                    .caused_by(trc::location!())?;\n                            }\n                        }\n                    }\n                    Operation::Index { field, key, set } => {\n                        let key = IndexKey {\n                            account_id,\n                            collection,\n                            document_id,\n                            field: *field,\n                            key: &*key,\n                        }\n                        .serialize(0);\n\n                        if *set {\n                            trx.prepare_cached(\"INSERT OR IGNORE INTO i (k) VALUES (?)\")\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?\n                                .execute([&key])\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?;\n                        } else {\n                            trx.prepare_cached(\"DELETE FROM i WHERE k = ?\")\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?\n                                .execute([&key])\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?;\n                        }\n                    }\n                    Operation::Log { collection, set } => {\n                        let key = LogKey {\n                            account_id,\n                            collection: u8::from(*collection),\n                            change_id,\n                        }\n                        .serialize(0);\n\n                        trx.prepare_cached(\"INSERT OR REPLACE INTO l (k, v) VALUES (?, ?)\")\n                            .map_err(into_error)\n                            .caused_by(trc::location!())?\n                            .execute([&key, set])\n                            .map_err(into_error)\n                            .caused_by(trc::location!())?;\n                    }\n                    Operation::AssertValue {\n                        class,\n                        assert_value,\n                    } => {\n                        let key = class.serialize(account_id, collection, document_id, 0);\n                        let table = char::from(class.subspace(collection));\n\n                        let matches = trx\n                            .prepare_cached(&format!(\"SELECT v FROM {} WHERE k = ?\", table))\n                            .map_err(into_error)\n                            .caused_by(trc::location!())?\n                            .query_row([&key], |row| {\n                                Ok(assert_value.matches(row.get_ref(0)?.as_bytes()?))\n                            })\n                            .optional()\n                            .map_err(into_error)\n                            .caused_by(trc::location!())?\n                            .unwrap_or_else(|| assert_value.is_none());\n                        if !matches {\n                            trx.rollback()\n                                .map_err(into_error)\n                                .caused_by(trc::location!())?;\n                            return Err(trc::StoreEvent::AssertValueFailed\n                                .into_err()\n                                .caused_by(trc::location!()));\n                        }\n                    }\n                }\n            }\n\n            trx.commit().map(|_| result).map_err(into_error)\n        })\n        .await\n    }\n\n    pub(crate) async fn purge_store(&self) -> trc::Result<()> {\n        let conn = self\n            .conn_pool\n            .get()\n            .map_err(into_error)\n            .caused_by(trc::location!())?;\n        self.spawn_worker(move || {\n            for subspace in [SUBSPACE_QUOTA, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER] {\n                conn.prepare_cached(&format!(\"DELETE FROM {} WHERE v = 0\", char::from(subspace),))\n                    .map_err(into_error)\n                    .caused_by(trc::location!())?\n                    .execute([])\n                    .map_err(into_error)\n                    .caused_by(trc::location!())?;\n            }\n\n            Ok(())\n        })\n        .await\n    }\n\n    pub(crate) async fn delete_range(&self, from: impl Key, to: impl Key) -> trc::Result<()> {\n        let conn = self\n            .conn_pool\n            .get()\n            .map_err(into_error)\n            .caused_by(trc::location!())?;\n        self.spawn_worker(move || {\n            conn.prepare_cached(&format!(\n                \"DELETE FROM {} WHERE k >= ? AND k < ?\",\n                char::from(from.subspace()),\n            ))\n            .map_err(into_error)\n            .caused_by(trc::location!())?\n            .execute([from.serialize(0), to.serialize(0)])\n            .map_err(into_error)\n            .caused_by(trc::location!())?;\n\n            Ok(())\n        })\n        .await\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/zenoh/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse utils::config::{Config, utils::AsKey};\npub mod pubsub;\n\n#[derive(Debug)]\npub struct ZenohPubSub {\n    session: zenoh::Session,\n}\n\nimpl ZenohPubSub {\n    pub async fn open(config: &mut Config, prefix: impl AsKey) -> Option<Self> {\n        let prefix = prefix.as_key();\n        let zenoh_config =\n            zenoh::Config::from_json5(config.value_require_non_empty((&prefix, \"config\"))?)\n                .map_err(|err| {\n                    config.new_build_error(\n                        (&prefix, \"config\"),\n                        format!(\"Invalid zenoh config: {}\", err),\n                    );\n                })\n                .ok()?;\n        zenoh::open(zenoh_config)\n            .await\n            .map_err(|err| {\n                config.new_build_error(\n                    (&prefix, \"config\"),\n                    format!(\"Failed to create zenoh session: {}\", err),\n                );\n            })\n            .map(|session| ZenohPubSub { session })\n            .ok()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/backend/zenoh/pubsub.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ZenohPubSub;\nuse crate::dispatch::pubsub::{Msg, PubSubStream};\nuse trc::{ClusterEvent, Error, EventType};\n\npub struct ZenohPubSubStream {\n    subs: zenoh::pubsub::Subscriber<zenoh::handlers::FifoChannelHandler<zenoh::sample::Sample>>,\n}\n\nimpl ZenohPubSub {\n    pub async fn publish(&self, topic: &'static str, message: Vec<u8>) -> trc::Result<()> {\n        self.session\n            .declare_publisher(topic)\n            .await\n            .map_err(|err| {\n                Error::new(EventType::Cluster(ClusterEvent::PublisherError)).reason(err)\n            })?\n            .put(message)\n            .await\n            .map_err(|err| Error::new(EventType::Cluster(ClusterEvent::PublisherError)).reason(err))\n    }\n\n    pub async fn subscribe(&self, topic: &'static str) -> trc::Result<PubSubStream> {\n        self.session\n            .declare_subscriber(topic)\n            .await\n            .map(|subs| PubSubStream::Zenoh(ZenohPubSubStream { subs }))\n            .map_err(|err| {\n                Error::new(EventType::Cluster(ClusterEvent::SubscriberError)).reason(err)\n            })\n    }\n}\n\nimpl ZenohPubSubStream {\n    pub async fn next(&mut self) -> Option<Msg> {\n        self.subs\n            .recv_async()\n            .await\n            .map(|sample| Msg::Zenoh(sample.payload().to_bytes().into_owned()))\n            .ok()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    BlobStore, CompressionAlgo, InMemoryStore, PurgeSchedule, PurgeStore, Store, Stores,\n    backend::{elastic::ElasticSearchStore, fs::FsStore, meili::MeiliSearchStore},\n};\nuse utils::config::{Config, cron::SimpleCron, utils::ParseValue};\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\nenum CompositeStore {\n    #[cfg(any(feature = \"postgres\", feature = \"mysql\"))]\n    SQLReadReplica(String),\n    ShardedBlob(String),\n    ShardedInMemory(String),\n}\n// SPDX-SnippetEnd\n\nimpl Stores {\n    pub async fn parse_all(config: &mut Config, is_reload: bool) -> Self {\n        let mut stores = Self::parse(config).await;\n        stores.parse_in_memory(config, is_reload).await;\n        stores\n    }\n\n    pub async fn parse(config: &mut Config) -> Self {\n        let mut stores = Self::default();\n        stores.parse_stores(config).await;\n        stores\n    }\n\n    pub async fn parse_stores(&mut self, config: &mut Config) {\n        let is_reload = !self.stores.is_empty();\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n        #[cfg(feature = \"enterprise\")]\n        let mut composite_stores = Vec::new();\n        // SPDX-SnippetEnd\n\n        for store_id in config.sub_keys(\"store\", \".type\") {\n            let id = store_id.as_str();\n            // Parse store\n            #[cfg(feature = \"test_mode\")]\n            {\n                if config\n                    .property_or_default::<bool>((\"store\", id, \"disable\"), \"false\")\n                    .unwrap_or(false)\n                {\n                    continue;\n                }\n            }\n            let protocol = if let Some(protocol) = config.value_require((\"store\", id, \"type\")) {\n                protocol.to_ascii_lowercase()\n            } else {\n                continue;\n            };\n            let prefix = (\"store\", id);\n            let compression_algo = config\n                .property_or_default::<CompressionAlgo>((\"store\", id, \"compression\"), \"none\")\n                .unwrap_or(CompressionAlgo::None);\n\n            match protocol.as_str() {\n                #[cfg(feature = \"rocks\")]\n                \"rocksdb\" => {\n                    // Avoid opening the same store twice\n                    if is_reload\n                        && self\n                            .stores\n                            .values()\n                            .any(|store| matches!(store, Store::RocksDb(_)))\n                    {\n                        continue;\n                    }\n\n                    if let Some(db) = crate::backend::rocksdb::RocksDbStore::open(config, prefix)\n                        .await\n                        .map(Store::from)\n                    {\n                        self.stores.insert(store_id.clone(), db.clone());\n                        self.search_stores\n                            .insert(store_id.clone(), db.clone().into());\n                        self.blob_stores.insert(\n                            store_id.clone(),\n                            BlobStore::from(db.clone()).with_compression(compression_algo),\n                        );\n                        self.in_memory_stores.insert(store_id, db.into());\n                    }\n                }\n                #[cfg(feature = \"foundation\")]\n                \"foundationdb\" => {\n                    // Avoid opening the same store twice\n                    if is_reload\n                        && self\n                            .stores\n                            .values()\n                            .any(|store| matches!(store, Store::FoundationDb(_)))\n                    {\n                        continue;\n                    }\n\n                    if let Some(db) = crate::backend::foundationdb::FdbStore::open(config, prefix)\n                        .await\n                        .map(Store::from)\n                    {\n                        self.stores.insert(store_id.clone(), db.clone());\n                        self.search_stores\n                            .insert(store_id.clone(), db.clone().into());\n                        self.blob_stores.insert(\n                            store_id.clone(),\n                            BlobStore::from(db.clone()).with_compression(compression_algo),\n                        );\n                        self.in_memory_stores.insert(store_id, db.into());\n                    }\n                }\n                #[cfg(feature = \"postgres\")]\n                \"postgresql\" => {\n                    if let Some(db) = crate::backend::postgres::PostgresStore::open(\n                        config,\n                        prefix,\n                        config.is_active_store(id),\n                        config.is_active_search_store(id),\n                    )\n                    .await\n                    .map(Store::from)\n                    {\n                        self.stores.insert(store_id.clone(), db.clone());\n                        self.search_stores\n                            .insert(store_id.clone(), db.clone().into());\n                        self.blob_stores.insert(\n                            store_id.clone(),\n                            BlobStore::from(db.clone()).with_compression(compression_algo),\n                        );\n                        self.in_memory_stores.insert(store_id.clone(), db.into());\n                    }\n                }\n                #[cfg(feature = \"mysql\")]\n                \"mysql\" => {\n                    if let Some(db) = crate::backend::mysql::MysqlStore::open(\n                        config,\n                        prefix,\n                        config.is_active_store(id),\n                        config.is_active_search_store(id),\n                    )\n                    .await\n                    .map(Store::from)\n                    {\n                        self.stores.insert(store_id.clone(), db.clone());\n                        self.search_stores\n                            .insert(store_id.clone(), db.clone().into());\n                        self.blob_stores.insert(\n                            store_id.clone(),\n                            BlobStore::from(db.clone()).with_compression(compression_algo),\n                        );\n                        self.in_memory_stores.insert(store_id.clone(), db.into());\n                    }\n                }\n                #[cfg(feature = \"sqlite\")]\n                \"sqlite\" => {\n                    // Avoid opening the same store twice\n                    if is_reload\n                        && self\n                            .stores\n                            .values()\n                            .any(|store| matches!(store, Store::SQLite(_)))\n                    {\n                        continue;\n                    }\n\n                    if let Some(db) =\n                        crate::backend::sqlite::SqliteStore::open(config, prefix).map(Store::from)\n                    {\n                        self.stores.insert(store_id.clone(), db.clone());\n                        self.search_stores\n                            .insert(store_id.clone(), db.clone().into());\n                        self.blob_stores.insert(\n                            store_id.clone(),\n                            BlobStore::from(db.clone()).with_compression(compression_algo),\n                        );\n                        self.in_memory_stores.insert(store_id.clone(), db.into());\n                    }\n                }\n                \"fs\" => {\n                    if let Some(db) = FsStore::open(config, prefix).await.map(BlobStore::from) {\n                        self.blob_stores\n                            .insert(store_id, db.with_compression(compression_algo));\n                    }\n                }\n                #[cfg(feature = \"s3\")]\n                \"s3\" => {\n                    if let Some(db) = crate::backend::s3::S3Store::open(config, prefix)\n                        .await\n                        .map(BlobStore::from)\n                    {\n                        self.blob_stores\n                            .insert(store_id, db.with_compression(compression_algo));\n                    }\n                }\n                \"elasticsearch\" => {\n                    if let Some(db) = ElasticSearchStore::open(config, prefix)\n                        .await\n                        .map(crate::SearchStore::from)\n                    {\n                        self.search_stores.insert(store_id, db);\n                    }\n                }\n                \"meilisearch\" => {\n                    if let Some(db) = MeiliSearchStore::open(config, prefix)\n                        .await\n                        .map(crate::SearchStore::from)\n                    {\n                        self.search_stores.insert(store_id, db);\n                    }\n                }\n                #[cfg(feature = \"redis\")]\n                \"redis\" => {\n                    if let Some(db) = crate::backend::redis::RedisStore::open(config, prefix)\n                        .await\n                        .map(std::sync::Arc::new)\n                    {\n                        self.in_memory_stores\n                            .insert(store_id.clone(), InMemoryStore::Redis(db.clone()));\n                        self.pubsub_stores\n                            .insert(store_id, crate::PubSubStore::Redis(db));\n                    }\n                }\n                #[cfg(feature = \"nats\")]\n                \"nats\" => {\n                    if let Some(db) = crate::backend::nats::NatsPubSub::open(config, prefix)\n                        .await\n                        .map(std::sync::Arc::new)\n                    {\n                        self.pubsub_stores\n                            .insert(store_id, crate::PubSubStore::Nats(db));\n                    }\n                }\n                #[cfg(feature = \"zenoh\")]\n                \"zenoh\" => {\n                    if let Some(db) = crate::backend::zenoh::ZenohPubSub::open(config, prefix)\n                        .await\n                        .map(std::sync::Arc::new)\n                    {\n                        self.pubsub_stores\n                            .insert(store_id, crate::PubSubStore::Zenoh(db));\n                    }\n                }\n                #[cfg(feature = \"kafka\")]\n                \"kafka\" => {\n                    if let Some(db) = crate::backend::kafka::KafkaPubSub::open(config, prefix)\n                        .await\n                        .map(std::sync::Arc::new)\n                    {\n                        self.pubsub_stores\n                            .insert(store_id, crate::PubSubStore::Kafka(db));\n                    }\n                }\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(feature = \"enterprise\")]\n                \"sql-read-replica\" => {\n                    #[cfg(any(feature = \"postgres\", feature = \"mysql\"))]\n                    composite_stores.push(CompositeStore::SQLReadReplica(store_id));\n                }\n                #[cfg(feature = \"enterprise\")]\n                \"distributed-blob\" | \"sharded-blob\" => {\n                    composite_stores.push(CompositeStore::ShardedBlob(store_id));\n                }\n                #[cfg(feature = \"enterprise\")]\n                \"sharded-in-memory\" => {\n                    composite_stores.push(CompositeStore::ShardedInMemory(store_id));\n                }\n                // SPDX-SnippetEnd\n                #[cfg(feature = \"azure\")]\n                \"azure\" => {\n                    if let Some(db) = crate::backend::azure::AzureStore::open(config, prefix)\n                        .await\n                        .map(BlobStore::from)\n                    {\n                        self.blob_stores\n                            .insert(store_id, db.with_compression(compression_algo));\n                    }\n                }\n                unknown => {\n                    config.new_parse_warning(\n                        (\"store\", id, \"type\"),\n                        format!(\"Unknown directory type: {unknown:?}\"),\n                    );\n                }\n            }\n        }\n\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n        #[cfg(feature = \"enterprise\")]\n        for composite_store in composite_stores {\n            match composite_store {\n                #[cfg(any(feature = \"postgres\", feature = \"mysql\"))]\n                CompositeStore::SQLReadReplica(id) => {\n                    let prefix = (\"store\", id.as_str());\n                    if let Some(db) = crate::backend::composite::read_replica::SQLReadReplica::open(\n                        config,\n                        prefix,\n                        self,\n                        config.is_active_store(&id),\n                        config.is_active_search_store(&id),\n                    )\n                    .await\n                    {\n                        let db = Store::SQLReadReplica(db.into());\n                        self.stores.insert(id.to_string(), db.clone());\n                        self.search_stores.insert(id.to_string(), db.clone().into());\n                        self.blob_stores.insert(\n                            id.to_string(),\n                            BlobStore::from(db.clone()).with_compression(\n                                config\n                                    .property_or_default::<CompressionAlgo>(\n                                        (\"store\", id.as_str(), \"compression\"),\n                                        \"none\",\n                                    )\n                                    .unwrap_or(CompressionAlgo::None),\n                            ),\n                        );\n                        self.in_memory_stores.insert(id, db.into());\n                    }\n                }\n                CompositeStore::ShardedBlob(id) => {\n                    let prefix = (\"store\", id.as_str());\n                    if let Some(db) = crate::backend::composite::sharded_blob::ShardedBlob::open(\n                        config, prefix, self,\n                    ) {\n                        let store = BlobStore {\n                            backend: crate::BlobBackend::Sharded(db.into()),\n                            compression: config\n                                .property_or_default::<CompressionAlgo>(\n                                    (\"store\", id.as_str(), \"compression\"),\n                                    \"none\",\n                                )\n                                .unwrap_or(CompressionAlgo::None),\n                        };\n                        self.blob_stores.insert(id, store);\n                    }\n                }\n                CompositeStore::ShardedInMemory(id) => {\n                    let prefix = (\"store\", id.as_str());\n                    if let Some(db) =\n                        crate::backend::composite::sharded_lookup::ShardedInMemory::open(\n                            config, prefix, self,\n                        )\n                    {\n                        self.in_memory_stores\n                            .insert(id, InMemoryStore::Sharded(db.into()));\n                    }\n                }\n            }\n        }\n\n        // SPDX-SnippetEnd\n    }\n\n    pub async fn parse_in_memory(&mut self, config: &mut Config, is_reload: bool) {\n        // Parse memory stores\n        self.parse_static_stores(config, is_reload);\n\n        // Parse http stores\n        self.parse_http_stores(config, is_reload);\n\n        // Parse purge schedules\n        if let Some(store) = config\n            .value(\"storage.data\")\n            .and_then(|store_id| self.stores.get(store_id))\n        {\n            let store_id = config.value(\"storage.data\").unwrap().to_string();\n            self.purge_schedules.push(PurgeSchedule {\n                cron: config\n                    .property_or_default::<SimpleCron>(\n                        (\"store\", store_id.as_str(), \"purge.frequency\"),\n                        \"0 3 *\",\n                    )\n                    .unwrap_or_else(|| SimpleCron::parse_value(\"0 3 *\").unwrap()),\n                store_id,\n                store: PurgeStore::Data(store.clone()),\n            });\n\n            if let Some(blob_store) = config\n                .value(\"storage.blob\")\n                .and_then(|blob_store_id| self.blob_stores.get(blob_store_id))\n            {\n                let store_id = config.value(\"storage.blob\").unwrap().to_string();\n                self.purge_schedules.push(PurgeSchedule {\n                    cron: config\n                        .property_or_default::<SimpleCron>(\n                            (\"store\", store_id.as_str(), \"purge.frequency\"),\n                            \"0 4 *\",\n                        )\n                        .unwrap_or_else(|| SimpleCron::parse_value(\"0 4 *\").unwrap()),\n                    store_id,\n                    store: PurgeStore::Blobs {\n                        store: store.clone(),\n                        blob_store: blob_store.clone(),\n                    },\n                });\n            }\n        }\n        for (store_id, store) in &self.in_memory_stores {\n            if matches!(store, InMemoryStore::Store(_))\n                && config.is_active_in_memory_store(store_id)\n            {\n                self.purge_schedules.push(PurgeSchedule {\n                    cron: config\n                        .property_or_default::<SimpleCron>(\n                            (\"store\", store_id.as_str(), \"purge.frequency\"),\n                            \"0 5 *\",\n                        )\n                        .unwrap_or_else(|| SimpleCron::parse_value(\"0 5 *\").unwrap()),\n                    store_id: store_id.clone(),\n                    store: PurgeStore::Lookup(store.clone()),\n                });\n            }\n        }\n    }\n}\n\n#[allow(dead_code)]\ntrait IsActiveStore {\n    fn is_active_store(&self, id: &str) -> bool;\n    fn is_active_in_memory_store(&self, id: &str) -> bool;\n    fn is_active_search_store(&self, id: &str) -> bool;\n}\n\nimpl IsActiveStore for Config {\n    fn is_active_store(&self, id: &str) -> bool {\n        for key in [\n            \"storage.data\",\n            \"storage.blob\",\n            \"storage.lookup\",\n            \"tracing.history.store\",\n            \"metrics.history.store\",\n        ] {\n            if let Some(store_id) = self.value(key)\n                && store_id == id\n            {\n                return true;\n            }\n        }\n\n        false\n    }\n\n    fn is_active_search_store(&self, id: &str) -> bool {\n        self.value(\"storage.fts\")\n            .is_some_and(|store_id| store_id == id)\n    }\n\n    fn is_active_in_memory_store(&self, id: &str) -> bool {\n        self.value(\"storage.lookup\")\n            .is_some_and(|store_id| store_id == id)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/dispatch/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, ops::Range, time::Instant};\n\nuse trc::{AddContext, StoreEvent};\nuse utils::config::utils::ParseValue;\n\nuse crate::{BlobBackend, BlobStore, CompressionAlgo, Store};\n\nimpl BlobStore {\n    pub async fn get_blob(&self, key: &[u8], range: Range<usize>) -> trc::Result<Option<Vec<u8>>> {\n        let read_range = match self.compression {\n            CompressionAlgo::None => range.clone(),\n            CompressionAlgo::Lz4 => 0..usize::MAX,\n        };\n        let start_time = Instant::now();\n        let result = match &self.backend {\n            BlobBackend::Store(store) => match store {\n                #[cfg(feature = \"sqlite\")]\n                Store::SQLite(store) => store.get_blob(key, read_range).await,\n                #[cfg(feature = \"foundation\")]\n                Store::FoundationDb(store) => store.get_blob(key, read_range).await,\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => store.get_blob(key, read_range).await,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => store.get_blob(key, read_range).await,\n                #[cfg(feature = \"rocks\")]\n                Store::RocksDb(store) => store.get_blob(key, read_range).await,\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(store) => store.get_blob(key, read_range).await,\n                // SPDX-SnippetEnd\n                Store::None => Err(trc::StoreEvent::NotConfigured.into()),\n            },\n            BlobBackend::Fs(store) => store.get_blob(key, read_range).await,\n            #[cfg(feature = \"s3\")]\n            BlobBackend::S3(store) => store.get_blob(key, read_range).await,\n            #[cfg(feature = \"azure\")]\n            BlobBackend::Azure(store) => store.get_blob(key, read_range).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            BlobBackend::Sharded(store) => store.get_blob(key, read_range).await,\n            // SPDX-SnippetEnd\n        };\n\n        trc::event!(\n            Store(StoreEvent::BlobRead),\n            Key = key,\n            Elapsed = start_time.elapsed(),\n            Size = result\n                .as_ref()\n                .map_or(0, |data| data.as_ref().map_or(0, |data| data.len())),\n        );\n\n        let decompressed = match self.compression {\n            CompressionAlgo::Lz4 => match result.caused_by(trc::location!())? {\n                Some(data)\n                    if data.last().copied().unwrap_or_default()\n                        == CompressionAlgo::Lz4.marker() =>\n                {\n                    lz4_flex::decompress_size_prepended(\n                        data.get(..data.len() - 1).unwrap_or_default(),\n                    )\n                    .map_err(|err| {\n                        trc::StoreEvent::DecompressError\n                            .reason(err)\n                            .ctx(trc::Key::Key, key)\n                            .ctx(trc::Key::CausedBy, trc::location!())\n                    })?\n                }\n                Some(data) => {\n                    trc::event!(Store(StoreEvent::BlobMissingMarker), Key = key,);\n                    data\n                }\n                None => return Ok(None),\n            },\n            _ => return result,\n        };\n\n        if range.end > decompressed.len() {\n            Ok(Some(decompressed))\n        } else {\n            Ok(Some(\n                decompressed\n                    .get(range.start..range.end)\n                    .unwrap_or_default()\n                    .to_vec(),\n            ))\n        }\n    }\n\n    pub async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        let data: Cow<[u8]> = match self.compression {\n            CompressionAlgo::None => data.into(),\n            CompressionAlgo::Lz4 => {\n                let mut compressed = lz4_flex::compress_prepend_size(data);\n                compressed.push(CompressionAlgo::Lz4.marker());\n                compressed.into()\n            }\n        };\n\n        let start_time = Instant::now();\n        let result = match &self.backend {\n            BlobBackend::Store(store) => match store {\n                #[cfg(feature = \"sqlite\")]\n                Store::SQLite(store) => store.put_blob(key, data.as_ref()).await,\n                #[cfg(feature = \"foundation\")]\n                Store::FoundationDb(store) => store.put_blob(key, data.as_ref()).await,\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => store.put_blob(key, data.as_ref()).await,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => store.put_blob(key, data.as_ref()).await,\n                #[cfg(feature = \"rocks\")]\n                Store::RocksDb(store) => store.put_blob(key, data.as_ref()).await,\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(store) => store.put_blob(key, data.as_ref()).await,\n                // SPDX-SnippetEnd\n                Store::None => Err(trc::StoreEvent::NotConfigured.into()),\n            },\n            BlobBackend::Fs(store) => store.put_blob(key, data.as_ref()).await,\n            #[cfg(feature = \"s3\")]\n            BlobBackend::S3(store) => store.put_blob(key, data.as_ref()).await,\n            #[cfg(feature = \"azure\")]\n            BlobBackend::Azure(store) => store.put_blob(key, data.as_ref()).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            BlobBackend::Sharded(store) => store.put_blob(key, data.as_ref()).await,\n            // SPDX-SnippetEnd\n        }\n        .caused_by(trc::location!());\n\n        trc::event!(\n            Store(StoreEvent::BlobWrite),\n            Key = key,\n            Elapsed = start_time.elapsed(),\n            Size = data.len(),\n        );\n\n        result\n    }\n\n    pub async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        let start_time = Instant::now();\n        let result = match &self.backend {\n            BlobBackend::Store(store) => match store {\n                #[cfg(feature = \"sqlite\")]\n                Store::SQLite(store) => store.delete_blob(key).await,\n                #[cfg(feature = \"foundation\")]\n                Store::FoundationDb(store) => store.delete_blob(key).await,\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => store.delete_blob(key).await,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => store.delete_blob(key).await,\n                #[cfg(feature = \"rocks\")]\n                Store::RocksDb(store) => store.delete_blob(key).await,\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(store) => store.delete_blob(key).await,\n                // SPDX-SnippetEnd\n                Store::None => Err(trc::StoreEvent::NotConfigured.into()),\n            },\n            BlobBackend::Fs(store) => store.delete_blob(key).await,\n            #[cfg(feature = \"s3\")]\n            BlobBackend::S3(store) => store.delete_blob(key).await,\n            #[cfg(feature = \"azure\")]\n            BlobBackend::Azure(store) => store.delete_blob(key).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            BlobBackend::Sharded(store) => store.delete_blob(key).await,\n            // SPDX-SnippetEnd\n        }\n        .caused_by(trc::location!());\n\n        trc::event!(\n            Store(StoreEvent::BlobWrite),\n            Key = key,\n            Elapsed = start_time.elapsed(),\n        );\n\n        result\n    }\n\n    pub fn with_compression(self, compression: CompressionAlgo) -> Self {\n        Self {\n            backend: self.backend,\n            compression,\n        }\n    }\n}\n\nconst MAGIC_MARKER: u8 = 0xa0;\n\nimpl CompressionAlgo {\n    pub fn marker(&self) -> u8 {\n        match self {\n            CompressionAlgo::Lz4 => MAGIC_MARKER | 0x01,\n            //CompressionAlgo::Zstd => MAGIC_MARKER | 0x02,\n            CompressionAlgo::None => 0,\n        }\n    }\n}\n\nimpl ParseValue for CompressionAlgo {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        match value {\n            \"lz4\" => Ok(CompressionAlgo::Lz4),\n            //\"zstd\" => Ok(CompressionAlgo::Zstd),\n            \"none\" | \"false\" | \"disable\" | \"disabled\" => Ok(CompressionAlgo::None),\n            algo => Err(format!(\"Invalid compression algorithm: {algo}\",)),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/dispatch/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse trc::AddContext;\nuse utils::config::Rate;\n\n#[allow(unused_imports)]\nuse crate::{\n    Deserialize, InMemoryStore, IterateParams, QueryResult, Store, U64_LEN, Value, ValueKey,\n    write::{\n        BatchBuilder, Operation, ValueClass, ValueOp,\n        key::{DeserializeBigEndian, KeySerializer},\n        now,\n    },\n};\nuse crate::{\n    SerializeInfallible,\n    backend::http::lookup::HttpStoreGet,\n    write::{InMemoryClass, assert::AssertValue},\n};\n\npub struct KeyValue<T> {\n    pub key: Vec<u8>,\n    pub value: T,\n    pub expires: Option<u64>,\n}\n\nimpl InMemoryStore {\n    pub async fn key_set(&self, kv: KeyValue<Vec<u8>>) -> trc::Result<()> {\n        match self {\n            InMemoryStore::Store(store) => {\n                let mut batch = BatchBuilder::new();\n                batch.any_op(Operation::Value {\n                    class: ValueClass::InMemory(InMemoryClass::Key(kv.key)),\n                    op: ValueOp::Set(\n                        KeySerializer::new(kv.value.len() + U64_LEN)\n                            .write(kv.expires.map_or(u64::MAX, |expires| now() + expires))\n                            .write(kv.value.as_slice())\n                            .finalize(),\n                    ),\n                });\n                store.write(batch.build_all()).await.map(|_| ())\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.key_set(&kv.key, &kv.value, kv.expires).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.key_set(kv).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {\n                Err(trc::StoreEvent::NotSupported.into_err())\n            }\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn counter_incr(&self, kv: KeyValue<i64>, return_value: bool) -> trc::Result<i64> {\n        match self {\n            InMemoryStore::Store(store) => {\n                let mut batch = BatchBuilder::new();\n\n                if let Some(expires) = kv.expires {\n                    batch.any_op(Operation::Value {\n                        class: ValueClass::InMemory(InMemoryClass::Key(kv.key.clone())),\n                        op: ValueOp::Set(\n                            KeySerializer::new(U64_LEN * 2)\n                                .write(0u64)\n                                .write(now() + expires)\n                                .finalize(),\n                        ),\n                    });\n                }\n\n                if return_value {\n                    batch.any_op(Operation::Value {\n                        class: ValueClass::InMemory(InMemoryClass::Counter(kv.key)),\n                        op: ValueOp::AddAndGet(kv.value),\n                    });\n\n                    store\n                        .write(batch.build_all())\n                        .await\n                        .and_then(|r| r.last_counter_id())\n                } else {\n                    batch.any_op(Operation::Value {\n                        class: ValueClass::InMemory(InMemoryClass::Counter(kv.key)),\n                        op: ValueOp::AtomicAdd(kv.value),\n                    });\n\n                    store.write(batch.build_all()).await.map(|_| 0)\n                }\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.key_incr(&kv.key, kv.value, kv.expires).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.counter_incr(kv).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {\n                Err(trc::StoreEvent::NotSupported.into_err())\n            }\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn key_delete(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<()> {\n        match self {\n            InMemoryStore::Store(store) => {\n                let mut batch = BatchBuilder::new();\n                batch.any_op(Operation::Value {\n                    class: ValueClass::InMemory(InMemoryClass::Key(key.into().into_bytes())),\n                    op: ValueOp::Clear,\n                });\n                store.write(batch.build_all()).await.map(|_| ())\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.key_delete(key.into().as_bytes()).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.key_delete(key).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {\n                Err(trc::StoreEvent::NotSupported.into_err())\n            }\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn counter_delete(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<()> {\n        match self {\n            InMemoryStore::Store(store) => {\n                let mut batch = BatchBuilder::new();\n                batch.any_op(Operation::Value {\n                    class: ValueClass::InMemory(InMemoryClass::Counter(key.into().into_bytes())),\n                    op: ValueOp::Clear,\n                });\n                store.write(batch.build_all()).await.map(|_| ())\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.key_delete(key.into().as_bytes()).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.counter_delete(key).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {\n                Err(trc::StoreEvent::NotSupported.into_err())\n            }\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn key_delete_prefix(&self, prefix: &[u8]) -> trc::Result<()> {\n        match self {\n            InMemoryStore::Store(store) => {\n                if prefix.is_empty() {\n                    return Ok(());\n                }\n\n                let from_range = prefix.to_vec();\n                let mut to_range = Vec::with_capacity(prefix.len() + 3);\n                to_range.extend_from_slice(prefix);\n                to_range.extend_from_slice([u8::MAX, u8::MAX, u8::MAX].as_ref());\n\n                store\n                    .delete_range(\n                        ValueKey::from(ValueClass::InMemory(InMemoryClass::Counter(\n                            from_range.clone(),\n                        ))),\n                        ValueKey::from(ValueClass::InMemory(InMemoryClass::Counter(\n                            to_range.clone(),\n                        ))),\n                    )\n                    .await?;\n\n                store\n                    .delete_range(\n                        ValueKey::from(ValueClass::InMemory(InMemoryClass::Key(from_range))),\n                        ValueKey::from(ValueClass::InMemory(InMemoryClass::Key(to_range))),\n                    )\n                    .await\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.key_delete_prefix(prefix).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.key_delete_prefix(prefix).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {\n                Err(trc::StoreEvent::NotSupported.into_err())\n            }\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn key_get<T: Deserialize + From<Value<'static>> + std::fmt::Debug + 'static>(\n        &self,\n        key: impl Into<LookupKey<'_>>,\n    ) -> trc::Result<Option<T>> {\n        match self {\n            InMemoryStore::Store(store) => store\n                .get_value::<LookupValue<T>>(ValueKey::from(ValueClass::InMemory(\n                    InMemoryClass::Key(key.into().into_bytes()),\n                )))\n                .await\n                .map(|value| value.and_then(|v| v.into())),\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.key_get(key.into().as_bytes()).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.key_get(key).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(store) => Ok(store\n                .get(key.into().as_str())\n                .map(|value| T::from(value.clone()))),\n            InMemoryStore::Http(store) => {\n                Ok(store.get(key.into().as_str()).map(|value| T::from(value)))\n            }\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn counter_get(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<i64> {\n        match self {\n            InMemoryStore::Store(store) => {\n                store\n                    .get_counter(ValueKey::from(ValueClass::InMemory(\n                        InMemoryClass::Counter(key.into().into_bytes()),\n                    )))\n                    .await\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.counter_get(key.into().as_bytes()).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.counter_get(key).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {\n                Err(trc::StoreEvent::NotSupported.into_err())\n            }\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn key_exists(&self, key: impl Into<LookupKey<'_>>) -> trc::Result<bool> {\n        match self {\n            InMemoryStore::Store(store) => store\n                .get_value::<LookupValue<()>>(ValueKey::from(ValueClass::InMemory(\n                    InMemoryClass::Key(key.into().into_bytes()),\n                )))\n                .await\n                .map(|value| matches!(value, Some(LookupValue::Value(())))),\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store.key_exists(key.into().as_bytes()).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store.key_exists(key).await,\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(store) => Ok(store.get(key.into().as_str()).is_some()),\n            InMemoryStore::Http(store) => Ok(store.contains(key.into().as_str())),\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn is_rate_allowed(\n        &self,\n        prefix: u8,\n        key: &[u8],\n        rate: &Rate,\n        soft_check: bool,\n    ) -> trc::Result<Option<u64>> {\n        let now = now();\n        let range_start = now / rate.period.as_secs();\n        let range_end = (range_start * rate.period.as_secs()) + rate.period.as_secs();\n        let expires_in = range_end - now;\n\n        let mut bucket = Vec::with_capacity(key.len() + U64_LEN + 1);\n        bucket.push(prefix);\n        bucket.extend_from_slice(key);\n        bucket.extend_from_slice(range_start.to_be_bytes().as_slice());\n\n        let requests = if !soft_check {\n            self.counter_incr(KeyValue::new(bucket, 1).expires(expires_in), true)\n                .await\n                .caused_by(trc::location!())?\n        } else {\n            self.counter_get(bucket).await.caused_by(trc::location!())? + 1\n        };\n\n        if requests <= rate.requests as i64 {\n            Ok(None)\n        } else {\n            Ok(Some(expires_in))\n        }\n    }\n\n    pub async fn try_lock(&self, prefix: u8, key: &[u8], duration: u64) -> trc::Result<bool> {\n        match self {\n            InMemoryStore::Store(store) => {\n                let key = KeyValue::<()>::build_key(prefix, key);\n                let lock_expiry = match store\n                    .get_value::<u64>(ValueKey::from(ValueClass::InMemory(InMemoryClass::Key(\n                        key.clone(),\n                    ))))\n                    .await\n                {\n                    Ok(lock_expiry) => lock_expiry,\n                    Err(err)\n                        if err.matches(trc::EventType::Store(trc::StoreEvent::DataCorruption)) =>\n                    {\n                        // TODO remove in 1.0\n                        let mut batch = BatchBuilder::new();\n                        batch.any_op(Operation::Value {\n                            class: ValueClass::InMemory(InMemoryClass::Key(key.clone())),\n                            op: ValueOp::Clear,\n                        });\n                        store\n                            .write(batch.build_all())\n                            .await\n                            .caused_by(trc::location!())?;\n                        None\n                    }\n                    Err(err) => {\n                        return Err(err\n                            .details(\"Failed to read lock.\")\n                            .caused_by(trc::location!()));\n                    }\n                };\n\n                let now = now();\n                if lock_expiry.is_some_and(|expiry| expiry > now) {\n                    return Ok(false);\n                }\n\n                let key: ValueClass = ValueClass::InMemory(InMemoryClass::Key(key));\n                let mut batch = BatchBuilder::new();\n                batch.assert_value(\n                    key.clone(),\n                    match lock_expiry {\n                        Some(value) => AssertValue::U64(value),\n                        None => AssertValue::None,\n                    },\n                );\n                batch.set(key.clone(), (now + duration).serialize());\n                match store.write(batch.build_all()).await {\n                    Ok(_) => Ok(true),\n                    Err(err) if err.is_assertion_failure() => Ok(false),\n                    Err(err) => Err(err\n                        .details(\"Failed to lock event.\")\n                        .caused_by(trc::location!())),\n                }\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(store) => store\n                .key_incr(&KeyValue::<()>::build_key(prefix, key), 1, duration.into())\n                .await\n                .map(|count| count == 1),\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(store) => store\n                .counter_incr(KeyValue::with_prefix(prefix, key, 1).expires(duration))\n                .await\n                .map(|count| count == 1),\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {\n                Err(trc::StoreEvent::NotSupported.into_err())\n            }\n        }\n    }\n\n    pub async fn remove_lock(&self, prefix: u8, key: &[u8]) -> trc::Result<()> {\n        self.key_delete(KeyValue::<()>::build_key(prefix, key))\n            .await\n    }\n\n    pub async fn purge_in_memory_store(&self) -> trc::Result<()> {\n        match self {\n            InMemoryStore::Store(store) => {\n                // Delete expired keys and counters\n                let from_key = ValueKey::from(ValueClass::InMemory(InMemoryClass::Key(vec![0u8])));\n                let to_key =\n                    ValueKey::from(ValueClass::InMemory(InMemoryClass::Key(vec![u8::MAX; 10])));\n\n                let current_time = now();\n                let mut expired_keys = Vec::new();\n                let mut expired_counters = Vec::new();\n                store\n                    .iterate(IterateParams::new(from_key, to_key), |key, value| {\n                        let expiry = value.deserialize_be_u64(0).caused_by(trc::location!())?;\n                        if expiry == 0 {\n                            if value\n                                .deserialize_be_u64(U64_LEN)\n                                .caused_by(trc::location!())?\n                                <= current_time\n                            {\n                                expired_counters.push(key.to_vec());\n                            }\n                        } else if expiry <= current_time {\n                            expired_keys.push(key.to_vec());\n                        }\n                        Ok(true)\n                    })\n                    .await\n                    .caused_by(trc::location!())?;\n\n                if !expired_keys.is_empty() {\n                    let mut batch = BatchBuilder::new();\n                    for key in expired_keys {\n                        batch.any_op(Operation::Value {\n                            class: ValueClass::InMemory(InMemoryClass::Key(key)),\n                            op: ValueOp::Clear,\n                        });\n                        if batch.is_large_batch() {\n                            store\n                                .write(batch.build_all())\n                                .await\n                                .caused_by(trc::location!())?;\n                            batch = BatchBuilder::new();\n                        }\n                    }\n                    if !batch.is_empty() {\n                        store\n                            .write(batch.build_all())\n                            .await\n                            .caused_by(trc::location!())?;\n                    }\n                }\n\n                if !expired_counters.is_empty() {\n                    let mut batch = BatchBuilder::new();\n                    for key in expired_counters {\n                        batch.any_op(Operation::Value {\n                            class: ValueClass::InMemory(InMemoryClass::Counter(key.clone())),\n                            op: ValueOp::Clear,\n                        });\n                        batch.any_op(Operation::Value {\n                            class: ValueClass::InMemory(InMemoryClass::Key(key)),\n                            op: ValueOp::Clear,\n                        });\n                        if batch.is_large_batch() {\n                            store\n                                .write(batch.build_all())\n                                .await\n                                .caused_by(trc::location!())?;\n                            batch = BatchBuilder::new();\n                        }\n                    }\n                    if !batch.is_empty() {\n                        store\n                            .write(batch.build_all())\n                            .await\n                            .caused_by(trc::location!())?;\n                    }\n                }\n            }\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(_) => {}\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(feature = \"enterprise\")]\n            InMemoryStore::Sharded(_) => {}\n            // SPDX-SnippetEnd\n            InMemoryStore::Static(_) | InMemoryStore::Http(_) => {}\n        }\n\n        Ok(())\n    }\n\n    pub fn is_sql(&self) -> bool {\n        match self {\n            InMemoryStore::Store(store) => store.is_sql(),\n            _ => false,\n        }\n    }\n\n    pub fn is_redis(&self) -> bool {\n        match self {\n            #[cfg(feature = \"redis\")]\n            InMemoryStore::Redis(_) => true,\n            InMemoryStore::Static(_) => false,\n            _ => false,\n        }\n    }\n}\n\npub enum LookupKey<'x> {\n    String(String),\n    StringRef(&'x str),\n    Bytes(Vec<u8>),\n    BytesRef(&'x [u8]),\n}\n\nimpl<'x> From<&'x str> for LookupKey<'x> {\n    fn from(key: &'x str) -> Self {\n        LookupKey::StringRef(key)\n    }\n}\n\nimpl<'x> From<&'x String> for LookupKey<'x> {\n    fn from(key: &'x String) -> Self {\n        LookupKey::StringRef(key.as_str())\n    }\n}\n\nimpl<'x> From<&'x [u8]> for LookupKey<'x> {\n    fn from(key: &'x [u8]) -> Self {\n        LookupKey::BytesRef(key)\n    }\n}\n\nimpl<'x> From<Cow<'x, str>> for LookupKey<'x> {\n    fn from(key: Cow<'x, str>) -> Self {\n        match key {\n            Cow::Borrowed(key) => LookupKey::StringRef(key),\n            Cow::Owned(key) => LookupKey::String(key),\n        }\n    }\n}\n\nimpl From<String> for LookupKey<'static> {\n    fn from(key: String) -> Self {\n        LookupKey::String(key)\n    }\n}\n\nimpl From<Vec<u8>> for LookupKey<'static> {\n    fn from(key: Vec<u8>) -> Self {\n        LookupKey::Bytes(key)\n    }\n}\n\nimpl LookupKey<'_> {\n    pub fn as_str(&self) -> &str {\n        match self {\n            LookupKey::String(string) => string,\n            LookupKey::StringRef(string) => string,\n            LookupKey::Bytes(bytes) => std::str::from_utf8(bytes).unwrap_or_default(),\n            LookupKey::BytesRef(bytes) => std::str::from_utf8(bytes).unwrap_or_default(),\n        }\n    }\n\n    pub fn into_bytes(self) -> Vec<u8> {\n        match self {\n            LookupKey::String(string) => string.into_bytes(),\n            LookupKey::StringRef(string) => string.as_bytes().to_vec(),\n            LookupKey::Bytes(bytes) => bytes,\n            LookupKey::BytesRef(bytes) => bytes.to_vec(),\n        }\n    }\n\n    pub fn as_bytes(&self) -> &[u8] {\n        match self {\n            LookupKey::String(string) => string.as_bytes(),\n            LookupKey::StringRef(string) => string.as_bytes(),\n            LookupKey::Bytes(bytes) => bytes.as_slice(),\n            LookupKey::BytesRef(bytes) => bytes,\n        }\n    }\n}\n\nimpl<T> KeyValue<T> {\n    pub fn build_key(prefix: u8, key: impl AsRef<[u8]>) -> Vec<u8> {\n        let key_ = key.as_ref();\n        let mut key = Vec::with_capacity(key_.len() + 1);\n        key.push(prefix);\n        key.extend_from_slice(key_);\n        key\n    }\n\n    pub fn with_prefix(prefix: u8, key: impl AsRef<[u8]>, value: T) -> Self {\n        Self {\n            key: Self::build_key(prefix, key),\n            value,\n            expires: None,\n        }\n    }\n\n    pub fn new(key: impl Into<Vec<u8>>, value: T) -> Self {\n        Self {\n            key: key.into(),\n            value,\n            expires: None,\n        }\n    }\n\n    pub fn expires(mut self, expires: u64) -> Self {\n        self.expires = expires.into();\n        self\n    }\n\n    pub fn expires_opt(mut self, expires: Option<u64>) -> Self {\n        self.expires = expires;\n        self\n    }\n}\n\nenum LookupValue<T> {\n    Value(T),\n    None,\n}\n\nimpl<T: Deserialize> Deserialize for LookupValue<T> {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        bytes.deserialize_be_u64(0).and_then(|expires| {\n            Ok(if expires > now() {\n                LookupValue::Value(\n                    T::deserialize(bytes.get(U64_LEN..).unwrap_or_default())\n                        .caused_by(trc::location!())?,\n                )\n            } else {\n                LookupValue::None\n            })\n        })\n    }\n}\n\nimpl<T> From<LookupValue<T>> for Option<T> {\n    fn from(value: LookupValue<T>) -> Self {\n        match value {\n            LookupValue::Value(value) => Some(value),\n            LookupValue::None => None,\n        }\n    }\n}\n\nimpl From<Value<'static>> for String {\n    fn from(value: Value<'static>) -> Self {\n        match value {\n            Value::Text(string) => string.into_owned(),\n            Value::Blob(bytes) => String::from_utf8_lossy(bytes.as_ref()).into_owned(),\n            Value::Bool(boolean) => boolean.to_string(),\n            Value::Null => String::new(),\n            Value::Integer(num) => num.to_string(),\n            Value::Float(num) => num.to_string(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/dispatch/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse roaring::RoaringBitmap;\n\nuse crate::Store;\n\npub mod blob;\npub mod lookup;\npub mod pubsub;\npub mod search;\npub mod store;\n\nimpl Store {\n    pub fn id(&self) -> &'static str {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(_) => \"sqlite\",\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(_) => \"foundationdb\",\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(_) => \"postgresql\",\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(_) => \"mysql\",\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(_) => \"rocksdb\",\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(_) => \"read_replica\",\n            // SPDX-SnippetEnd\n            Self::None => \"none\",\n        }\n    }\n}\n\n#[allow(clippy::len_without_is_empty)]\npub trait DocumentSet: Sync + Send {\n    fn min(&self) -> u32;\n    fn max(&self) -> u32;\n    fn contains(&self, id: u32) -> bool;\n    fn len(&self) -> usize;\n    fn iterate(&self) -> impl Iterator<Item = u32>;\n}\n\nimpl DocumentSet for RoaringBitmap {\n    fn min(&self) -> u32 {\n        self.min().unwrap_or(0)\n    }\n\n    fn max(&self) -> u32 {\n        self.max().map(|m| m + 1).unwrap_or(0)\n    }\n\n    fn contains(&self, id: u32) -> bool {\n        self.contains(id)\n    }\n\n    fn len(&self) -> usize {\n        self.len() as usize\n    }\n\n    fn iterate(&self) -> impl Iterator<Item = u32> {\n        self.iter()\n    }\n}\n\nimpl DocumentSet for Vec<u32> {\n    fn contains(&self, id: u32) -> bool {\n        self.binary_search(&id).is_ok()\n    }\n\n    fn min(&self) -> u32 {\n        self.first().copied().unwrap_or(0)\n    }\n\n    fn max(&self) -> u32 {\n        self.last().copied().map(|m| m + 1).unwrap_or(0)\n    }\n\n    fn len(&self) -> usize {\n        self.len()\n    }\n\n    fn iterate(&self) -> impl Iterator<Item = u32> {\n        self.iter().copied()\n    }\n}\n\nimpl DocumentSet for () {\n    fn min(&self) -> u32 {\n        0\n    }\n\n    fn max(&self) -> u32 {\n        u32::MAX\n    }\n\n    fn contains(&self, _: u32) -> bool {\n        true\n    }\n\n    fn len(&self) -> usize {\n        0\n    }\n\n    fn iterate(&self) -> impl Iterator<Item = u32> {\n        std::iter::empty()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/dispatch/pubsub.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::PubSubStore;\n\npub enum PubSubStream {\n    #[cfg(feature = \"redis\")]\n    Redis(crate::backend::redis::pubsub::RedisPubSubStream),\n    #[cfg(feature = \"redis\")]\n    RedisCluster(crate::backend::redis::pubsub::RedisClusterPubSubStream),\n    #[cfg(feature = \"nats\")]\n    Nats(crate::backend::nats::pubsub::NatsPubSubStream),\n    #[cfg(feature = \"zenoh\")]\n    Zenoh(crate::backend::zenoh::pubsub::ZenohPubSubStream),\n    #[cfg(feature = \"kafka\")]\n    Kafka(crate::backend::kafka::pubsub::KafkaPubSubStream),\n    #[cfg(not(any(feature = \"redis\", feature = \"nats\")))]\n    Unimplemented,\n}\n\npub enum Msg {\n    #[cfg(feature = \"redis\")]\n    Redis(redis::Msg),\n    #[cfg(feature = \"nats\")]\n    Nats(async_nats::Message),\n    #[cfg(feature = \"zenoh\")]\n    Zenoh(Vec<u8>),\n    #[cfg(feature = \"kafka\")]\n    Kafka(Vec<u8>),\n    #[cfg(not(any(feature = \"redis\", feature = \"nats\")))]\n    Unimplemented,\n}\n\n#[allow(unused_variables)]\nimpl PubSubStore {\n    pub async fn publish(&self, topic: &'static str, message: Vec<u8>) -> trc::Result<()> {\n        match self {\n            #[cfg(feature = \"redis\")]\n            PubSubStore::Redis(store) => store.publish(topic, message).await,\n            #[cfg(feature = \"nats\")]\n            PubSubStore::Nats(store) => store.publish(topic, message).await,\n            #[cfg(feature = \"zenoh\")]\n            PubSubStore::Zenoh(store) => store.publish(topic, message).await,\n            #[cfg(feature = \"kafka\")]\n            PubSubStore::Kafka(store) => store.publish(topic, message).await,\n            PubSubStore::None => Err(trc::StoreEvent::NotSupported.into_err()),\n        }\n    }\n\n    pub async fn subscribe(&self, topic: &'static str) -> trc::Result<PubSubStream> {\n        match self {\n            #[cfg(feature = \"redis\")]\n            PubSubStore::Redis(store) => store.subscribe(topic).await,\n            #[cfg(feature = \"nats\")]\n            PubSubStore::Nats(store) => store.subscribe(topic).await,\n            #[cfg(feature = \"zenoh\")]\n            PubSubStore::Zenoh(store) => store.subscribe(topic).await,\n            #[cfg(feature = \"kafka\")]\n            PubSubStore::Kafka(store) => store.subscribe(topic).await,\n            PubSubStore::None => Err(trc::StoreEvent::NotSupported.into_err()),\n        }\n    }\n\n    pub fn is_none(&self) -> bool {\n        matches!(self, PubSubStore::None)\n    }\n}\n\nimpl PubSubStream {\n    pub async fn next(&mut self) -> Option<Msg> {\n        match self {\n            #[cfg(feature = \"redis\")]\n            PubSubStream::Redis(stream) => stream.next().await,\n            #[cfg(feature = \"redis\")]\n            PubSubStream::RedisCluster(stream) => stream.next().await,\n            #[cfg(feature = \"nats\")]\n            PubSubStream::Nats(stream) => stream.next().await,\n            #[cfg(feature = \"zenoh\")]\n            PubSubStream::Zenoh(stream) => stream.next().await,\n            #[cfg(feature = \"kafka\")]\n            PubSubStream::Kafka(stream) => stream.next().await,\n            #[cfg(not(any(feature = \"redis\", feature = \"nats\")))]\n            PubSubStream::Unimplemented => None,\n        }\n    }\n}\n\nimpl Msg {\n    pub fn payload(&self) -> &[u8] {\n        match self {\n            #[cfg(feature = \"redis\")]\n            Msg::Redis(msg) => msg.get_payload_bytes(),\n            #[cfg(feature = \"nats\")]\n            Msg::Nats(msg) => msg.payload.as_ref(),\n            #[cfg(feature = \"zenoh\")]\n            Msg::Zenoh(msg) => msg.as_slice(),\n            #[cfg(feature = \"kafka\")]\n            Msg::Kafka(msg) => msg.as_slice(),\n            #[cfg(not(any(feature = \"redis\", feature = \"nats\")))]\n            Msg::Unimplemented => &[],\n        }\n    }\n\n    pub fn topic(&self) -> &str {\n        match self {\n            #[cfg(feature = \"redis\")]\n            Msg::Redis(msg) => msg.get_channel_name(),\n            #[cfg(feature = \"nats\")]\n            Msg::Nats(msg) => msg.subject.as_str(),\n            #[cfg(feature = \"zenoh\")]\n            Msg::Zenoh(_) => \"\",\n            #[cfg(feature = \"kafka\")]\n            Msg::Kafka(_) => \"\",\n            #[cfg(not(any(feature = \"redis\", feature = \"nats\")))]\n            Msg::Unimplemented => \"\",\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/dispatch/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse trc::AddContext;\n\nuse crate::{\n    SearchStore, Store,\n    search::{\n        IndexDocument, SearchComparator, SearchField, SearchFilter, SearchOperator, SearchQuery,\n        SearchValue,\n        split::{SplitFilter, split_filters},\n    },\n    write::SearchIndex,\n};\nuse std::cmp::Ordering;\n\nimpl SearchStore {\n    pub async fn query_account(&self, query: SearchQuery) -> trc::Result<Vec<u32>> {\n        // Pre-filter by mask\n        if query.mask.is_empty() {\n            return Ok(vec![]);\n        }\n\n        // If the store does not support FTS, use the internal FTS store\n        if let Some(store) = self.internal_fts() {\n            return store.query_account(query).await;\n        }\n\n        // If all filters and comparators are external, delegate to the underlying store\n        let mut account_id = u32::MAX;\n        let mut has_local_filters = false;\n        let mut has_external_filters = false;\n        for filter in &query.filters {\n            match filter {\n                SearchFilter::Operator {\n                    field: SearchField::AccountId,\n                    op: SearchOperator::Equal,\n                    value: SearchValue::Uint(id),\n                } => {\n                    account_id = *id as u32;\n                }\n                SearchFilter::DocumentSet(_) => {\n                    has_local_filters = true;\n                }\n                SearchFilter::Operator { .. } => {\n                    has_external_filters = true;\n                }\n                _ => (),\n            }\n        }\n\n        if account_id == u32::MAX {\n            return Err(trc::StoreEvent::UnexpectedError\n                .reason(\"Account ID filter is required for account queries\")\n                .caused_by(trc::location!()));\n        }\n\n        if !has_local_filters && !has_external_filters && query.comparators.is_empty() {\n            return Ok(query.mask.iter().collect());\n        }\n\n        if !has_local_filters && query.comparators.iter().all(|c| c.is_external()) {\n            return self\n                .sub_query(query.index, &query.filters, &query.comparators)\n                .await\n                .map(|results| {\n                    if !results.is_empty() || has_external_filters {\n                        results\n                            .into_iter()\n                            .filter(|id| query.mask.contains(*id))\n                            .collect()\n                    } else {\n                        // Database sort is broken, return masked results\n                        query.mask.iter().collect()\n                    }\n                })\n                .caused_by(trc::location!());\n        }\n\n        let filters = if has_external_filters {\n            // Split filters\n            let split_filters = split_filters(query.filters).ok_or_else(|| {\n                trc::StoreEvent::UnexpectedError\n                    .reason(\"Invalid filter query\")\n                    .caused_by(trc::location!())\n            })?;\n\n            let mut filters = Vec::with_capacity(split_filters.len());\n            for split_filter in split_filters {\n                match split_filter {\n                    SplitFilter::External(external) => {\n                        // Execute sub-query\n                        filters.push(SearchFilter::DocumentSet(\n                            self.sub_query(query.index, &external, &[])\n                                .await?\n                                .into_iter()\n                                .collect(),\n                        ));\n                    }\n                    SplitFilter::Internal(filter) => {\n                        filters.push(filter);\n                    }\n                }\n            }\n\n            filters\n        } else {\n            query.filters\n        };\n\n        // Merge results locally\n        let results = SearchQuery::new(query.index)\n            .with_filters(filters)\n            .with_mask(query.mask)\n            .filter();\n\n        let total_results = results.results().len();\n        match total_results.cmp(&1) {\n            Ordering::Equal => Ok(vec![results.results().min().unwrap()]),\n            Ordering::Less => Ok(vec![]),\n            Ordering::Greater => {\n                if !query.comparators.is_empty() {\n                    let mut local = Vec::with_capacity(query.comparators.len());\n                    let mut external = Vec::with_capacity(query.comparators.len());\n                    let mut external_first = false;\n                    for (pos, comparator) in query.comparators.into_iter().enumerate() {\n                        if comparator.is_external() {\n                            external.push(comparator);\n                            if pos == 0 {\n                                external_first = true;\n                            }\n                        } else {\n                            local.push(comparator);\n                        }\n                    }\n\n                    if !external.is_empty() {\n                        let mut results = results.results().clone();\n                        let filters = vec![\n                            SearchFilter::Operator {\n                                field: SearchField::AccountId,\n                                op: SearchOperator::Equal,\n                                value: SearchValue::Uint(account_id as u64),\n                            },\n                            SearchFilter::Operator {\n                                field: SearchField::DocumentId,\n                                op: SearchOperator::GreaterEqualThan,\n                                value: SearchValue::Uint(results.min().unwrap() as u64),\n                            },\n                            SearchFilter::Operator {\n                                field: SearchField::DocumentId,\n                                op: SearchOperator::LowerEqualThan,\n                                value: SearchValue::Uint(results.max().unwrap() as u64),\n                            },\n                        ];\n\n                        let mut ordered_results = Vec::with_capacity(total_results as usize);\n                        for ordered_result in\n                            self.sub_query(query.index, &filters, &external).await?\n                        {\n                            if results.remove(ordered_result) {\n                                ordered_results.push(ordered_result);\n                            }\n                        }\n                        // Add any remaining results not yet in the index\n                        ordered_results.extend(results.into_iter());\n\n                        if local.is_empty() {\n                            return Ok(ordered_results);\n                        }\n\n                        let comparator = SearchComparator::SortedSet {\n                            set: ordered_results\n                                .into_iter()\n                                .enumerate()\n                                .map(|(pos, id)| (id, pos as u32))\n                                .collect(),\n                            ascending: true,\n                        };\n\n                        if external_first {\n                            local.insert(0, comparator);\n                        } else {\n                            local.push(comparator);\n                        }\n                    }\n\n                    Ok(results.with_comparators(local).into_sorted())\n                } else {\n                    Ok(results.results().iter().collect())\n                }\n            }\n        }\n    }\n\n    async fn sub_query(\n        &self,\n        index: SearchIndex,\n        filters: &[SearchFilter],\n        sort: &[SearchComparator],\n    ) -> trc::Result<Vec<u32>> {\n        match self {\n            SearchStore::Store(store) => match store {\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => store.query(index, filters, sort).await,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => store.query(index, filters, sort).await,\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(store) => store.query(index, filters, sort).await,\n                // SPDX-SnippetEnd\n                _ => unreachable!(),\n            },\n            SearchStore::ElasticSearch(store) => store.query(index, filters, sort).await,\n            SearchStore::MeiliSearch(store) => store.query(index, filters, sort).await,\n        }\n    }\n\n    pub async fn query_global(&self, query: SearchQuery) -> trc::Result<Vec<u64>> {\n        match self {\n            SearchStore::Store(store) => match store {\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => {\n                    store\n                        .query(query.index, &query.filters, &query.comparators)\n                        .await\n                }\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => {\n                    store\n                        .query(query.index, &query.filters, &query.comparators)\n                        .await\n                }\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(store) => {\n                    store\n                        .query(query.index, &query.filters, &query.comparators)\n                        .await\n                }\n                // SPDX-SnippetEnd\n                store => store.query_global(query).await,\n            },\n            SearchStore::ElasticSearch(store) => {\n                store\n                    .query(query.index, &query.filters, &query.comparators)\n                    .await\n            }\n            SearchStore::MeiliSearch(store) => {\n                store\n                    .query(query.index, &query.filters, &query.comparators)\n                    .await\n            }\n        }\n    }\n\n    pub async fn index(&self, documents: Vec<IndexDocument>) -> trc::Result<()> {\n        match self {\n            SearchStore::Store(store) => match store {\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => store.index(documents).await,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => store.index(documents).await,\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(store) => store.index(documents).await,\n                // SPDX-SnippetEnd\n                store => store.index(documents).await,\n            },\n            SearchStore::ElasticSearch(store) => store.index(documents).await,\n            SearchStore::MeiliSearch(store) => store.index(documents).await,\n        }\n    }\n\n    pub async fn unindex(&self, query: SearchQuery) -> trc::Result<u64> {\n        match self {\n            SearchStore::Store(store) => match store {\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(store) => store.unindex(query).await,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(store) => store.unindex(query).await,\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(store) => store.unindex(query).await,\n                // SPDX-SnippetEnd\n                store => store.unindex(query).await.map(|_| 0),\n            },\n            SearchStore::ElasticSearch(store) => store.unindex(query).await,\n            SearchStore::MeiliSearch(store) => store.unindex(query).await,\n        }\n    }\n\n    pub fn internal_fts(&self) -> Option<&Store> {\n        match self {\n            SearchStore::Store(store) => match store {\n                #[cfg(feature = \"postgres\")]\n                Store::PostgreSQL(_) => None,\n                #[cfg(feature = \"mysql\")]\n                Store::MySQL(_) => None,\n                // SPDX-SnippetBegin\n                // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n                // SPDX-License-Identifier: LicenseRef-SEL\n                #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n                Store::SQLReadReplica(_) => None,\n                // SPDX-SnippetEnd\n                store => Some(store),\n            },\n            _ => None,\n        }\n    }\n\n    pub fn is_mysql(&self) -> bool {\n        match self {\n            #[cfg(feature = \"mysql\")]\n            SearchStore::Store(Store::MySQL(_)) => true,\n            _ => false,\n        }\n    }\n\n    pub fn is_postgres(&self) -> bool {\n        match self {\n            #[cfg(feature = \"postgres\")]\n            SearchStore::Store(Store::PostgreSQL(_)) => true,\n            _ => false,\n        }\n    }\n\n    pub fn is_elasticsearch(&self) -> bool {\n        matches!(self, SearchStore::ElasticSearch(_))\n    }\n\n    pub fn is_meilisearch(&self) -> bool {\n        matches!(self, SearchStore::MeiliSearch(_))\n    }\n}\n\nimpl SearchFilter {\n    pub fn is_external(&self) -> bool {\n        matches!(self, SearchFilter::Operator { .. })\n    }\n}\n\nimpl SearchComparator {\n    pub fn is_external(&self) -> bool {\n        matches!(self, SearchComparator::Field { .. })\n    }\n}\n"
  },
  {
    "path": "crates/store/src/dispatch/store.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::DocumentSet;\nuse crate::{\n    Deserialize, IterateParams, Key, QueryResult, SUBSPACE_BLOB_EXTRA, SUBSPACE_COUNTER,\n    SUBSPACE_INDEXES, SUBSPACE_LOGS, Store, U32_LEN, Value, ValueKey,\n    write::{\n        AnyClass, AnyKey, AssignedIds, Batch, BatchBuilder, Operation, ReportClass, ValueClass,\n        ValueOp,\n        key::{DeserializeBigEndian, KeySerializer},\n        now,\n    },\n};\nuse compact_str::ToCompactString;\nuse std::{ops::Range, time::Instant};\nuse trc::{AddContext, StoreEvent};\nuse types::collection::Collection;\n\nimpl Store {\n    pub async fn get_value<U>(&self, key: impl Key) -> trc::Result<Option<U>>\n    where\n        U: Deserialize + 'static,\n    {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.get_value(key).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.get_value(key).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.get_value(key).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.get_value(key).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.get_value(key).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.get_value(key).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn iterate<T: Key>(\n        &self,\n        params: IterateParams<T>,\n        cb: impl for<'x> FnMut(&'x [u8], &'x [u8]) -> trc::Result<bool> + Sync + Send,\n    ) -> trc::Result<()> {\n        let start_time = Instant::now();\n        let result = match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.iterate(params, cb).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.iterate(params, cb).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.iterate(params, cb).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.iterate(params, cb).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.iterate(params, cb).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.iterate(params, cb).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!());\n\n        trc::event!(\n            Store(StoreEvent::DataIterate),\n            Elapsed = start_time.elapsed(),\n        );\n\n        result\n    }\n\n    pub async fn get_counter(\n        &self,\n        key: impl Into<ValueKey<ValueClass>> + Sync + Send,\n    ) -> trc::Result<i64> {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.get_counter(key).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.get_counter(key).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.get_counter(key).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.get_counter(key).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.get_counter(key).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.get_counter(key).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!())\n    }\n\n    #[allow(unreachable_patterns)]\n    #[allow(unused_variables)]\n    pub async fn sql_query<T: QueryResult + std::fmt::Debug>(\n        &self,\n        query: &str,\n        params: Vec<Value<'_>>,\n    ) -> trc::Result<T> {\n        let result = match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.sql_query(query, &params).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.sql_query(query, &params).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.sql_query(query, &params).await,\n            _ => Err(trc::StoreEvent::NotSupported.into_err()),\n        };\n\n        trc::event!(\n            Store(trc::StoreEvent::SqlQuery),\n            Details = query.to_compact_string(),\n            Value = params.as_slice(),\n            Result = &result,\n        );\n\n        result.caused_by(trc::location!())\n    }\n\n    pub async fn write(&self, batch: Batch<'_>) -> trc::Result<AssignedIds> {\n        let start_time = Instant::now();\n        let ops = batch.ops.len();\n\n        let result = match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.write(batch).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.write(batch).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.write(batch).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.write(batch).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.write(batch).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.write(batch).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        };\n\n        trc::event!(\n            Store(StoreEvent::DataWrite),\n            Elapsed = start_time.elapsed(),\n            Total = ops,\n        );\n\n        result\n    }\n\n    pub async fn assign_document_ids(\n        &self,\n        account_id: u32,\n        collection: Collection,\n        num_ids: u64,\n    ) -> trc::Result<u32> {\n        // Increment UID next\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account_id)\n            .with_collection(collection)\n            .add_and_get(ValueClass::DocumentId, num_ids as i64);\n        self.write(batch.build_all()).await.and_then(|v| {\n            v.last_counter_id().map(|id| {\n                debug_assert!(id >= num_ids as i64, \"{} < {}\", id, num_ids);\n                id as u32\n            })\n        })\n    }\n\n    pub async fn purge_store(&self) -> trc::Result<()> {\n        // Delete expired reports\n        let now = now();\n        self.delete_range(\n            ValueKey::from(ValueClass::Report(ReportClass::Dmarc { id: 0, expires: 0 })),\n            ValueKey::from(ValueClass::Report(ReportClass::Dmarc {\n                id: u64::MAX,\n                expires: now,\n            })),\n        )\n        .await\n        .caused_by(trc::location!())?;\n        self.delete_range(\n            ValueKey::from(ValueClass::Report(ReportClass::Tls { id: 0, expires: 0 })),\n            ValueKey::from(ValueClass::Report(ReportClass::Tls {\n                id: u64::MAX,\n                expires: now,\n            })),\n        )\n        .await\n        .caused_by(trc::location!())?;\n        self.delete_range(\n            ValueKey::from(ValueClass::Report(ReportClass::Arf { id: 0, expires: 0 })),\n            ValueKey::from(ValueClass::Report(ReportClass::Arf {\n                id: u64::MAX,\n                expires: now,\n            })),\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.purge_store().await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.purge_store().await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.purge_store().await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.purge_store().await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.purge_store().await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.purge_store().await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn delete_range(&self, from: impl Key, to: impl Key) -> trc::Result<()> {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.delete_range(from, to).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.delete_range(from, to).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.delete_range(from, to).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.delete_range(from, to).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.delete_range(from, to).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.delete_range(from, to).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn delete_documents(\n        &self,\n        subspace: u8,\n        account_id: u32,\n        collection: u8,\n        collection_offset: Option<usize>,\n        document_ids: &impl DocumentSet,\n    ) -> trc::Result<()> {\n        // Serialize keys\n        let (from_key, to_key) = if collection_offset.is_some() {\n            (\n                KeySerializer::new(U32_LEN + 2)\n                    .write(account_id)\n                    .write(collection),\n                KeySerializer::new(U32_LEN + 2)\n                    .write(account_id)\n                    .write(collection + 1),\n            )\n        } else {\n            (\n                KeySerializer::new(U32_LEN).write(account_id),\n                KeySerializer::new(U32_LEN).write(account_id + 1),\n            )\n        };\n\n        // Find keys to delete\n        let mut delete_keys = Vec::new();\n        self.iterate(\n            IterateParams::new(\n                AnyKey {\n                    subspace,\n                    key: from_key.finalize(),\n                },\n                AnyKey {\n                    subspace,\n                    key: to_key.finalize(),\n                },\n            )\n            .no_values(),\n            |key, _| {\n                if collection_offset.is_none_or(|offset| {\n                    key.get(key.len() - U32_LEN - offset).copied() == Some(collection)\n                }) {\n                    let document_id = key.deserialize_be_u32(key.len() - U32_LEN)?;\n                    if document_ids.contains(document_id) {\n                        delete_keys.push(key.to_vec());\n                    }\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        // Remove keys\n        let mut batch = BatchBuilder::new();\n\n        for key in delete_keys {\n            if batch.is_large_batch() {\n                self.write(std::mem::take(&mut batch).build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n            batch.any_op(Operation::Value {\n                class: ValueClass::Any(AnyClass { subspace, key }),\n                op: ValueOp::Clear,\n            });\n        }\n\n        if !batch.is_empty() {\n            self.write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(())\n    }\n\n    pub async fn danger_destroy_account(&self, account_id: u32) -> trc::Result<()> {\n        for subspace in [\n            SUBSPACE_LOGS,\n            SUBSPACE_INDEXES,\n            SUBSPACE_COUNTER,\n            SUBSPACE_BLOB_EXTRA,\n        ] {\n            self.delete_range(\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U32_LEN).write(account_id).finalize(),\n                },\n                AnyKey {\n                    subspace,\n                    key: KeySerializer::new(U32_LEN).write(account_id + 1).finalize(),\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n        }\n\n        for (from_class, to_class) in [\n            (ValueClass::Acl(account_id), ValueClass::Acl(account_id + 1)),\n            (ValueClass::Property(0), ValueClass::Property(0)),\n        ] {\n            self.delete_range(\n                ValueKey {\n                    account_id,\n                    collection: 0,\n                    document_id: 0,\n                    class: from_class,\n                },\n                ValueKey {\n                    account_id: account_id + 1,\n                    collection: 0,\n                    document_id: 0,\n                    class: to_class,\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n        }\n\n        Ok(())\n    }\n\n    pub async fn get_blob(&self, key: &[u8], range: Range<usize>) -> trc::Result<Option<Vec<u8>>> {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.get_blob(key, range).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.get_blob(key, range).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.get_blob(key, range).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.get_blob(key, range).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.get_blob(key, range).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.get_blob(key, range).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn put_blob(&self, key: &[u8], data: &[u8]) -> trc::Result<()> {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.put_blob(key, data).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.put_blob(key, data).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.put_blob(key, data).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.put_blob(key, data).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.put_blob(key, data).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.put_blob(key, data).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!())\n    }\n\n    pub async fn delete_blob(&self, key: &[u8]) -> trc::Result<bool> {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(store) => store.delete_blob(key).await,\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(store) => store.delete_blob(key).await,\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(store) => store.delete_blob(key).await,\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(store) => store.delete_blob(key).await,\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(store) => store.delete_blob(key).await,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(store) => store.delete_blob(key).await,\n            // SPDX-SnippetEnd\n            Self::None => Err(trc::StoreEvent::NotConfigured.into()),\n        }\n        .caused_by(trc::location!())\n    }\n}\n"
  },
  {
    "path": "crates/store/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod backend;\npub mod config;\npub mod dispatch;\npub mod query;\npub mod search;\npub mod write;\n\npub use ahash;\npub use blake3;\npub use parking_lot;\npub use rand;\npub use rkyv;\npub use roaring;\npub use xxhash_rust;\n\nuse ahash::AHashMap;\nuse backend::{fs::FsStore, http::HttpStore, memory::StaticMemoryStore};\nuse std::{borrow::Cow, sync::Arc};\nuse utils::config::cron::SimpleCron;\nuse write::ValueClass;\n\nuse crate::backend::{elastic::ElasticSearchStore, meili::MeiliSearchStore};\n\npub trait Deserialize: Sized + Sync + Send {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self>;\n    fn deserialize_owned(bytes: Vec<u8>) -> trc::Result<Self> {\n        Self::deserialize(&bytes)\n    }\n}\n\npub trait Serialize {\n    fn serialize(&self) -> trc::Result<Vec<u8>>;\n}\n\npub trait SerializeInfallible {\n    fn serialize(&self) -> Vec<u8>;\n}\n\n// Key serialization flags\npub(crate) const WITH_SUBSPACE: u32 = 1;\n\npub trait Key: Sync + Send + Clone {\n    fn serialize(&self, flags: u32) -> Vec<u8>;\n    fn subspace(&self) -> u8;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\npub struct IndexKey<T: AsRef<[u8]>> {\n    pub account_id: u32,\n    pub collection: u8,\n    pub document_id: u32,\n    pub field: u8,\n    pub key: T,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub struct IndexKeyPrefix {\n    pub account_id: u32,\n    pub collection: u8,\n    pub field: u8,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub struct ValueKey<T: AsRef<ValueClass>> {\n    pub account_id: u32,\n    pub collection: u8,\n    pub document_id: u32,\n    pub class: T,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub struct LogKey {\n    pub account_id: u32,\n    pub collection: u8,\n    pub change_id: u64,\n}\n\npub const U64_LEN: usize = std::mem::size_of::<u64>();\npub const U32_LEN: usize = std::mem::size_of::<u32>();\npub const U16_LEN: usize = std::mem::size_of::<u16>();\n\npub const SUBSPACE_ACL: u8 = b'a';\npub const SUBSPACE_DIRECTORY: u8 = b'd';\npub const SUBSPACE_TASK_QUEUE: u8 = b'f';\npub const SUBSPACE_INDEXES: u8 = b'i';\npub const SUBSPACE_BLOB_EXTRA: u8 = b'j';\npub const SUBSPACE_BLOB_LINK: u8 = b'k';\npub const SUBSPACE_BLOBS: u8 = b't';\npub const SUBSPACE_LOGS: u8 = b'l';\npub const SUBSPACE_COUNTER: u8 = b'n';\npub const SUBSPACE_IN_MEMORY_VALUE: u8 = b'm';\npub const SUBSPACE_IN_MEMORY_COUNTER: u8 = b'y';\npub const SUBSPACE_PROPERTY: u8 = b'p';\npub const SUBSPACE_SETTINGS: u8 = b's';\npub const SUBSPACE_QUEUE_MESSAGE: u8 = b'e';\npub const SUBSPACE_QUEUE_EVENT: u8 = b'q';\npub const SUBSPACE_QUOTA: u8 = b'u';\npub const SUBSPACE_REPORT_OUT: u8 = b'h';\npub const SUBSPACE_REPORT_IN: u8 = b'r';\npub const SUBSPACE_TELEMETRY_SPAN: u8 = b'o';\npub const SUBSPACE_TELEMETRY_METRIC: u8 = b'x';\npub const SUBSPACE_SEARCH_INDEX: u8 = b'z';\n\n// TODO: Remove in v1.0\npub const LEGACY_SUBSPACE_BITMAP_ID: u8 = b'b';\npub const LEGACY_SUBSPACE_BITMAP_TAG: u8 = b'c';\npub const LEGACY_SUBSPACE_BITMAP_TEXT: u8 = b'v';\npub const LEGACY_SUBSPACE_FTS_INDEX: u8 = b'g';\npub const LEGACY_SUBSPACE_TELEMETRY_INDEX: u8 = b'w';\n\n#[derive(Clone)]\npub struct IterateParams<T: Key> {\n    begin: T,\n    end: T,\n    first: bool,\n    ascending: bool,\n    values: bool,\n}\n\n#[derive(Clone, Default)]\npub struct Stores {\n    pub stores: AHashMap<String, Store>,\n    pub blob_stores: AHashMap<String, BlobStore>,\n    pub search_stores: AHashMap<String, SearchStore>,\n    pub in_memory_stores: AHashMap<String, InMemoryStore>,\n    pub pubsub_stores: AHashMap<String, PubSubStore>,\n    pub purge_schedules: Vec<PurgeSchedule>,\n}\n\n#[derive(Clone, Default)]\npub enum Store {\n    #[cfg(feature = \"sqlite\")]\n    SQLite(Arc<backend::sqlite::SqliteStore>),\n    #[cfg(feature = \"foundation\")]\n    FoundationDb(Arc<backend::foundationdb::FdbStore>),\n    #[cfg(feature = \"postgres\")]\n    PostgreSQL(Arc<backend::postgres::PostgresStore>),\n    #[cfg(feature = \"mysql\")]\n    MySQL(Arc<backend::mysql::MysqlStore>),\n    #[cfg(feature = \"rocks\")]\n    RocksDb(Arc<backend::rocksdb::RocksDbStore>),\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n    SQLReadReplica(Arc<backend::composite::read_replica::SQLReadReplica>),\n    // SPDX-SnippetEnd\n    #[default]\n    None,\n}\n\n#[derive(Clone)]\npub struct BlobStore {\n    pub backend: BlobBackend,\n    pub compression: CompressionAlgo,\n}\n\n#[derive(Clone, Copy, Debug)]\npub enum CompressionAlgo {\n    None,\n    Lz4,\n}\n\n#[derive(Clone)]\npub enum BlobBackend {\n    Store(Store),\n    Fs(Arc<FsStore>),\n    #[cfg(feature = \"s3\")]\n    S3(Arc<backend::s3::S3Store>),\n    #[cfg(feature = \"azure\")]\n    Azure(Arc<backend::azure::AzureStore>),\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    Sharded(Arc<backend::composite::sharded_blob::ShardedBlob>),\n    // SPDX-SnippetEnd\n}\n\n#[derive(Clone)]\npub enum SearchStore {\n    Store(Store),\n    ElasticSearch(Arc<ElasticSearchStore>),\n    MeiliSearch(Arc<MeiliSearchStore>),\n}\n\n#[derive(Clone, Debug)]\npub enum InMemoryStore {\n    Store(Store),\n    #[cfg(feature = \"redis\")]\n    Redis(Arc<backend::redis::RedisStore>),\n    Http(Arc<HttpStore>),\n    Static(Arc<StaticMemoryStore>),\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    Sharded(Arc<backend::composite::sharded_lookup::ShardedInMemory>),\n    // SPDX-SnippetEnd\n}\n\n#[derive(Clone, Default)]\npub enum PubSubStore {\n    #[cfg(feature = \"redis\")]\n    Redis(Arc<backend::redis::RedisStore>),\n    #[cfg(feature = \"nats\")]\n    Nats(Arc<backend::nats::NatsPubSub>),\n    #[cfg(feature = \"zenoh\")]\n    Zenoh(Arc<backend::zenoh::ZenohPubSub>),\n    #[cfg(feature = \"kafka\")]\n    Kafka(Arc<backend::kafka::KafkaPubSub>),\n    #[default]\n    None,\n}\n\n#[cfg(feature = \"sqlite\")]\nimpl From<backend::sqlite::SqliteStore> for Store {\n    fn from(store: backend::sqlite::SqliteStore) -> Self {\n        Self::SQLite(Arc::new(store))\n    }\n}\n\n#[cfg(feature = \"foundation\")]\nimpl From<backend::foundationdb::FdbStore> for Store {\n    fn from(store: backend::foundationdb::FdbStore) -> Self {\n        Self::FoundationDb(Arc::new(store))\n    }\n}\n\n#[cfg(feature = \"postgres\")]\nimpl From<backend::postgres::PostgresStore> for Store {\n    fn from(store: backend::postgres::PostgresStore) -> Self {\n        Self::PostgreSQL(Arc::new(store))\n    }\n}\n\n#[cfg(feature = \"mysql\")]\nimpl From<backend::mysql::MysqlStore> for Store {\n    fn from(store: backend::mysql::MysqlStore) -> Self {\n        Self::MySQL(Arc::new(store))\n    }\n}\n\n#[cfg(feature = \"rocks\")]\nimpl From<backend::rocksdb::RocksDbStore> for Store {\n    fn from(store: backend::rocksdb::RocksDbStore) -> Self {\n        Self::RocksDb(Arc::new(store))\n    }\n}\n\nimpl From<FsStore> for BlobStore {\n    fn from(store: FsStore) -> Self {\n        BlobStore {\n            backend: BlobBackend::Fs(Arc::new(store)),\n            compression: CompressionAlgo::None,\n        }\n    }\n}\n\n#[cfg(feature = \"s3\")]\nimpl From<backend::s3::S3Store> for BlobStore {\n    fn from(store: backend::s3::S3Store) -> Self {\n        BlobStore {\n            backend: BlobBackend::S3(Arc::new(store)),\n            compression: CompressionAlgo::None,\n        }\n    }\n}\n\n#[cfg(feature = \"azure\")]\nimpl From<backend::azure::AzureStore> for BlobStore {\n    fn from(store: backend::azure::AzureStore) -> Self {\n        BlobStore {\n            backend: BlobBackend::Azure(Arc::new(store)),\n            compression: CompressionAlgo::None,\n        }\n    }\n}\n\nimpl From<ElasticSearchStore> for SearchStore {\n    fn from(store: ElasticSearchStore) -> Self {\n        Self::ElasticSearch(Arc::new(store))\n    }\n}\n\nimpl From<MeiliSearchStore> for SearchStore {\n    fn from(store: MeiliSearchStore) -> Self {\n        Self::MeiliSearch(Arc::new(store))\n    }\n}\n\n#[cfg(feature = \"redis\")]\nimpl From<backend::redis::RedisStore> for InMemoryStore {\n    fn from(store: backend::redis::RedisStore) -> Self {\n        Self::Redis(Arc::new(store))\n    }\n}\n\nimpl From<Store> for SearchStore {\n    fn from(store: Store) -> Self {\n        Self::Store(store)\n    }\n}\n\nimpl From<Store> for BlobStore {\n    fn from(store: Store) -> Self {\n        BlobStore {\n            backend: BlobBackend::Store(store),\n            compression: CompressionAlgo::None,\n        }\n    }\n}\n\nimpl From<Store> for InMemoryStore {\n    fn from(store: Store) -> Self {\n        Self::Store(store)\n    }\n}\n\nimpl Default for BlobStore {\n    fn default() -> Self {\n        Self {\n            backend: BlobBackend::Store(Store::None),\n            compression: CompressionAlgo::None,\n        }\n    }\n}\n\nimpl Default for InMemoryStore {\n    fn default() -> Self {\n        Self::Store(Store::None)\n    }\n}\n\nimpl Default for SearchStore {\n    fn default() -> Self {\n        Self::Store(Store::None)\n    }\n}\n\n#[derive(Clone)]\npub enum PurgeStore {\n    Data(Store),\n    Blobs { store: Store, blob_store: BlobStore },\n    Lookup(InMemoryStore),\n}\n\n#[derive(Clone)]\npub struct PurgeSchedule {\n    pub cron: SimpleCron,\n    pub store_id: String,\n    pub store: PurgeStore,\n}\n\n#[derive(Clone, Debug, PartialEq)]\npub enum Value<'x> {\n    Integer(i64),\n    Bool(bool),\n    Float(f64),\n    Text(Cow<'x, str>),\n    Blob(Cow<'x, [u8]>),\n    Null,\n}\n\nimpl Eq for Value<'_> {}\n\nimpl<'x> Value<'x> {\n    pub fn to_str<'y: 'x>(&'y self) -> Cow<'x, str> {\n        match self {\n            Value::Text(s) => s.as_ref().into(),\n            Value::Integer(i) => Cow::Owned(i.to_string()),\n            Value::Bool(b) => Cow::Owned(b.to_string()),\n            Value::Float(f) => Cow::Owned(f.to_string()),\n            Value::Blob(b) => String::from_utf8_lossy(b.as_ref()),\n            Value::Null => Cow::Borrowed(\"\"),\n        }\n    }\n}\n\n#[derive(Clone, Debug)]\npub struct Row {\n    pub values: Vec<Value<'static>>,\n}\n\n#[derive(Clone, Debug)]\npub struct Rows {\n    pub rows: Vec<Row>,\n}\n\n#[derive(Clone, Debug)]\npub struct NamedRows {\n    pub names: Vec<String>,\n    pub rows: Vec<Row>,\n}\n\n#[derive(Clone, Copy)]\npub enum QueryType {\n    Execute,\n    Exists,\n    QueryAll,\n    QueryOne,\n}\n\npub trait QueryResult: Sync + Send + 'static {\n    fn from_exec(items: usize) -> Self;\n    fn from_exists(exists: bool) -> Self;\n    fn from_query_one(items: impl IntoRows) -> Self;\n    fn from_query_all(items: impl IntoRows) -> Self;\n\n    fn query_type() -> QueryType;\n}\n\npub trait IntoRows {\n    fn into_row(self) -> Option<Row>;\n    fn into_rows(self) -> Rows;\n    fn into_named_rows(self) -> NamedRows;\n}\n\nimpl QueryResult for Option<Row> {\n    fn query_type() -> QueryType {\n        QueryType::QueryOne\n    }\n\n    fn from_exec(_: usize) -> Self {\n        unreachable!()\n    }\n\n    fn from_exists(_: bool) -> Self {\n        unreachable!()\n    }\n\n    fn from_query_all(_: impl IntoRows) -> Self {\n        unreachable!()\n    }\n\n    fn from_query_one(items: impl IntoRows) -> Self {\n        items.into_row()\n    }\n}\n\nimpl QueryResult for Rows {\n    fn query_type() -> QueryType {\n        QueryType::QueryAll\n    }\n\n    fn from_exec(_: usize) -> Self {\n        unreachable!()\n    }\n\n    fn from_exists(_: bool) -> Self {\n        unreachable!()\n    }\n\n    fn from_query_all(items: impl IntoRows) -> Self {\n        items.into_rows()\n    }\n\n    fn from_query_one(_: impl IntoRows) -> Self {\n        unreachable!()\n    }\n}\n\nimpl QueryResult for NamedRows {\n    fn query_type() -> QueryType {\n        QueryType::QueryAll\n    }\n\n    fn from_exec(_: usize) -> Self {\n        unreachable!()\n    }\n\n    fn from_exists(_: bool) -> Self {\n        unreachable!()\n    }\n\n    fn from_query_all(items: impl IntoRows) -> Self {\n        items.into_named_rows()\n    }\n\n    fn from_query_one(_: impl IntoRows) -> Self {\n        unreachable!()\n    }\n}\n\nimpl QueryResult for bool {\n    fn query_type() -> QueryType {\n        QueryType::Exists\n    }\n\n    fn from_exec(_: usize) -> Self {\n        unreachable!()\n    }\n\n    fn from_exists(exists: bool) -> Self {\n        exists\n    }\n\n    fn from_query_all(_: impl IntoRows) -> Self {\n        unreachable!()\n    }\n\n    fn from_query_one(_: impl IntoRows) -> Self {\n        unreachable!()\n    }\n}\n\nimpl QueryResult for usize {\n    fn query_type() -> QueryType {\n        QueryType::Execute\n    }\n\n    fn from_exec(items: usize) -> Self {\n        items\n    }\n\n    fn from_exists(_: bool) -> Self {\n        unreachable!()\n    }\n\n    fn from_query_all(_: impl IntoRows) -> Self {\n        unreachable!()\n    }\n\n    fn from_query_one(_: impl IntoRows) -> Self {\n        unreachable!()\n    }\n}\n\nimpl<'x> From<&'x str> for Value<'x> {\n    fn from(value: &'x str) -> Self {\n        Self::Text(value.into())\n    }\n}\n\nimpl From<String> for Value<'_> {\n    fn from(value: String) -> Self {\n        Self::Text(value.into())\n    }\n}\n\nimpl<'x> From<&'x String> for Value<'x> {\n    fn from(value: &'x String) -> Self {\n        Self::Text(value.into())\n    }\n}\n\nimpl<'x> From<Cow<'x, str>> for Value<'x> {\n    fn from(value: Cow<'x, str>) -> Self {\n        Self::Text(value)\n    }\n}\n\nimpl From<bool> for Value<'_> {\n    fn from(value: bool) -> Self {\n        Self::Bool(value)\n    }\n}\n\nimpl From<i64> for Value<'_> {\n    fn from(value: i64) -> Self {\n        Self::Integer(value)\n    }\n}\n\nimpl From<Value<'static>> for i64 {\n    fn from(value: Value<'static>) -> Self {\n        if let Value::Integer(value) = value {\n            value\n        } else {\n            0\n        }\n    }\n}\n\nimpl From<u64> for Value<'_> {\n    fn from(value: u64) -> Self {\n        Self::Integer(value as i64)\n    }\n}\n\nimpl From<u32> for Value<'_> {\n    fn from(value: u32) -> Self {\n        Self::Integer(value as i64)\n    }\n}\n\nimpl From<f64> for Value<'_> {\n    fn from(value: f64) -> Self {\n        Self::Float(value)\n    }\n}\n\nimpl<'x> From<&'x [u8]> for Value<'x> {\n    fn from(value: &'x [u8]) -> Self {\n        Self::Blob(value.into())\n    }\n}\n\nimpl From<Vec<u8>> for Value<'_> {\n    fn from(value: Vec<u8>) -> Self {\n        Self::Blob(value.into())\n    }\n}\n\nimpl Value<'_> {\n    pub fn into_string(self) -> String {\n        match self {\n            Value::Text(s) => s.into_owned(),\n            Value::Integer(i) => i.to_string(),\n            Value::Bool(b) => b.to_string(),\n            Value::Float(f) => f.to_string(),\n            Value::Blob(b) => String::from_utf8_lossy(b.as_ref()).into_owned(),\n            Value::Null => \"\".into(),\n        }\n    }\n\n    pub fn into_lower_string(self) -> String {\n        match self {\n            Value::Text(s) => s.as_ref().to_lowercase(),\n            Value::Integer(i) => i.to_string(),\n            Value::Bool(b) => b.to_string(),\n            Value::Float(f) => f.to_string(),\n            Value::Blob(b) => String::from_utf8_lossy(b.as_ref()).to_lowercase(),\n            Value::Null => \"\".into(),\n        }\n    }\n}\n\nimpl From<Row> for Vec<String> {\n    fn from(value: Row) -> Self {\n        value.values.into_iter().map(|v| v.into_string()).collect()\n    }\n}\n\nimpl From<Row> for Vec<u32> {\n    fn from(value: Row) -> Self {\n        value\n            .values\n            .into_iter()\n            .filter_map(|v| {\n                if let Value::Integer(v) = v {\n                    Some(v as u32)\n                } else {\n                    None\n                }\n            })\n            .collect()\n    }\n}\n\nimpl From<Rows> for Vec<String> {\n    fn from(value: Rows) -> Self {\n        value\n            .rows\n            .into_iter()\n            .flat_map(|v| v.values.into_iter().map(|v| v.into_string()))\n            .collect()\n    }\n}\n\nimpl From<Rows> for Vec<u32> {\n    fn from(value: Rows) -> Self {\n        value\n            .rows\n            .into_iter()\n            .flat_map(|v| {\n                v.values.into_iter().filter_map(|v| {\n                    if let Value::Integer(v) = v {\n                        Some(v as u32)\n                    } else {\n                        None\n                    }\n                })\n            })\n            .collect()\n    }\n}\n\nimpl Store {\n    #[inline(always)]\n    pub fn is_none(&self) -> bool {\n        matches!(self, Self::None)\n    }\n\n    #[inline(always)]\n    pub fn is_sql(&self) -> bool {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Store::SQLite(_) => true,\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(_) => true,\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(_) => true,\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Store::SQLReadReplica(_) => true,\n            // SPDX-SnippetEnd\n            _ => false,\n        }\n    }\n\n    #[inline(always)]\n    pub fn is_pg_or_mysql(&self) -> bool {\n        match self {\n            #[cfg(feature = \"mysql\")]\n            Store::MySQL(_) => true,\n            #[cfg(feature = \"postgres\")]\n            Store::PostgreSQL(_) => true,\n            _ => false,\n        }\n    }\n\n    #[inline(always)]\n    pub fn is_foundationdb(&self) -> bool {\n        match self {\n            #[cfg(feature = \"foundation\")]\n            Store::FoundationDb(_) => true,\n            _ => false,\n        }\n    }\n\n    // SPDX-SnippetBegin\n    // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n    // SPDX-License-Identifier: LicenseRef-SEL\n    #[cfg(feature = \"enterprise\")]\n    pub fn is_enterprise_store(&self) -> bool {\n        match self {\n            #[cfg(any(feature = \"postgres\", feature = \"mysql\"))]\n            Store::SQLReadReplica(_) => true,\n            _ => false,\n        }\n    }\n    // SPDX-SnippetEnd\n\n    #[cfg(not(feature = \"enterprise\"))]\n    pub fn is_enterprise_store(&self) -> bool {\n        false\n    }\n}\n\nimpl std::fmt::Debug for Store {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            #[cfg(feature = \"sqlite\")]\n            Self::SQLite(_) => f.debug_tuple(\"SQLite\").finish(),\n            #[cfg(feature = \"foundation\")]\n            Self::FoundationDb(_) => f.debug_tuple(\"FoundationDb\").finish(),\n            #[cfg(feature = \"postgres\")]\n            Self::PostgreSQL(_) => f.debug_tuple(\"PostgreSQL\").finish(),\n            #[cfg(feature = \"mysql\")]\n            Self::MySQL(_) => f.debug_tuple(\"MySQL\").finish(),\n            #[cfg(feature = \"rocks\")]\n            Self::RocksDb(_) => f.debug_tuple(\"RocksDb\").finish(),\n\n            // SPDX-SnippetBegin\n            // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n            // SPDX-License-Identifier: LicenseRef-SEL\n            #[cfg(all(feature = \"enterprise\", any(feature = \"postgres\", feature = \"mysql\")))]\n            Self::SQLReadReplica(_) => f.debug_tuple(\"SQLReadReplica\").finish(),\n            // SPDX-SnippetEnd\n            Self::None => f.debug_tuple(\"None\").finish(),\n        }\n    }\n}\n\nimpl From<Value<'_>> for trc::Value {\n    fn from(value: Value) -> Self {\n        match value {\n            Value::Integer(v) => trc::Value::Int(v),\n            Value::Bool(v) => trc::Value::Bool(v),\n            Value::Float(v) => trc::Value::Float(v),\n            Value::Text(v) => trc::Value::String(match v {\n                Cow::Borrowed(v) => v.into(),\n                Cow::Owned(v) => v.into(),\n            }),\n            Value::Blob(v) => trc::Value::Bytes(v.into_owned()),\n            Value::Null => trc::Value::None,\n        }\n    }\n}\n\nimpl From<Value<'static>> for () {\n    fn from(_: Value<'static>) -> Self {\n        unreachable!()\n    }\n}\n\nimpl Stores {\n    pub fn disable_enterprise_only(&mut self) {\n        // SPDX-SnippetBegin\n        // SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n        // SPDX-License-Identifier: LicenseRef-SEL\n        #[cfg(feature = \"enterprise\")]\n        {\n            #[cfg(any(feature = \"postgres\", feature = \"mysql\"))]\n            self.stores\n                .retain(|_, store| !matches!(store, Store::SQLReadReplica(_)));\n            self.blob_stores\n                .retain(|_, store| !matches!(store.backend, BlobBackend::Sharded(_)));\n        }\n        // SPDX-SnippetEnd\n    }\n}\n"
  },
  {
    "path": "crates/store/src/query/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashSet;\nuse trc::AddContext;\nuse types::collection::Collection;\n\nuse crate::{\n    Deserialize, IterateParams, Store, U32_LEN, ValueKey,\n    write::{BatchBuilder, ValueClass, key::DeserializeBigEndian},\n};\n\npub enum AclQuery {\n    SharedWith {\n        grant_account_id: u32,\n        to_account_id: u32,\n        to_collection: u8,\n    },\n    HasAccess {\n        grant_account_id: u32,\n    },\n}\n\n#[derive(Debug)]\npub struct AclItem {\n    pub to_account_id: u32,\n    pub to_collection: Collection,\n    pub to_document_id: u32,\n    pub permissions: u64,\n}\n\nimpl Store {\n    pub async fn acl_query(&self, query: AclQuery) -> trc::Result<Vec<AclItem>> {\n        let mut results = Vec::new();\n        let (from_key, to_key) = match query {\n            AclQuery::SharedWith {\n                grant_account_id,\n                to_account_id,\n                to_collection,\n            } => {\n                let from_key = ValueKey {\n                    account_id: to_account_id,\n                    collection: to_collection,\n                    document_id: 0,\n                    class: ValueClass::Acl(grant_account_id),\n                };\n                let mut to_key = from_key.clone();\n                to_key.document_id = u32::MAX;\n\n                (from_key, to_key)\n            }\n            AclQuery::HasAccess { grant_account_id } => (\n                ValueKey {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 0,\n                    class: ValueClass::Acl(grant_account_id),\n                },\n                ValueKey {\n                    account_id: u32::MAX,\n                    collection: u8::MAX,\n                    document_id: u32::MAX,\n                    class: ValueClass::Acl(grant_account_id),\n                },\n            ),\n        };\n\n        self.iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                results.push(AclItem::deserialize(key)?.with_permissions(u64::deserialize(value)?));\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())\n        .map(|_| results)\n    }\n\n    pub async fn acl_revoke_all(&self, account_id: u32) -> trc::Result<AHashSet<u32>> {\n        let from_key = ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Acl(0),\n        };\n        let to_key = ValueKey {\n            account_id: u32::MAX,\n            collection: u8::MAX,\n            document_id: u32::MAX,\n            class: ValueClass::Acl(u32::MAX),\n        };\n\n        let mut delete_keys = Vec::new();\n        let mut revoked_accounts = AHashSet::new();\n        self.iterate(\n            IterateParams::new(from_key, to_key).ascending().no_values(),\n            |key, _| {\n                if account_id == key.deserialize_be_u32(U32_LEN)? {\n                    let owner_account_id = key.deserialize_be_u32(0)?;\n                    revoked_accounts.insert(owner_account_id);\n                    delete_keys.push((owner_account_id, AclItem::deserialize(key)?));\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        // Remove permissions\n        let mut batch = BatchBuilder::new();\n        batch.with_account_id(account_id);\n        let mut last_collection = Collection::None;\n        for (revoke_account_id, acl_item) in delete_keys.into_iter() {\n            if batch.is_large_batch() {\n                self.write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n                batch = BatchBuilder::new();\n                batch.with_account_id(account_id);\n                last_collection = Collection::None;\n            }\n            if acl_item.to_collection != last_collection {\n                batch.with_collection(acl_item.to_collection);\n                last_collection = acl_item.to_collection;\n            }\n            batch\n                .with_document(acl_item.to_document_id)\n                .acl_revoke(revoke_account_id);\n        }\n        if !batch.is_empty() {\n            self.write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        Ok(revoked_accounts)\n    }\n}\n\nimpl Deserialize for AclItem {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(AclItem {\n            to_account_id: bytes.deserialize_be_u32(U32_LEN)?,\n            to_collection: bytes\n                .get(U32_LEN * 2)\n                .map(|b| Collection::from(*b))\n                .ok_or_else(|| trc::StoreEvent::DataCorruption.caused_by(trc::location!()))?,\n            to_document_id: bytes.deserialize_be_u32((U32_LEN * 2) + 1)?,\n            permissions: 0,\n        })\n    }\n}\n\nimpl AclItem {\n    fn with_permissions(mut self, permissions: u64) -> Self {\n        self.permissions = permissions;\n        self\n    }\n}\n"
  },
  {
    "path": "crates/store/src/query/log.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse trc::AddContext;\nuse types::collection::{SyncCollection, VanishedCollection};\nuse utils::codec::leb128::Leb128Iterator;\n\nuse crate::{\n    IterateParams, LogKey, Store, U32_LEN, U64_LEN,\n    write::{LogCollection, key::DeserializeBigEndian},\n};\n\n#[derive(Debug, PartialEq, Eq, Clone, Copy)]\npub enum Change {\n    InsertContainer(u64),\n    UpdateContainer(u64),\n    UpdateContainerProperty(u64),\n    DeleteContainer(u64),\n    InsertItem(u64),\n    UpdateItem(u64),\n    DeleteItem(u64),\n}\n\n#[derive(Debug)]\npub struct Changes {\n    pub changes: Vec<Change>,\n    pub from_change_id: u64,\n    pub to_change_id: u64,\n    pub container_change_id: Option<u64>,\n    pub item_change_id: Option<u64>,\n    pub is_truncated: bool,\n}\n\n#[derive(Debug, Clone, Copy)]\npub enum Query {\n    All,\n    Since(u64),\n    SinceInclusive(u64),\n    RangeInclusive(u64, u64),\n}\n\npub trait DeserializeVanished: Sized + Sync + Send {\n    fn deserialize_vanished<'x>(bytes: &mut impl Iterator<Item = &'x u8>) -> Option<Self>;\n}\n\nimpl Default for Changes {\n    fn default() -> Self {\n        Self {\n            changes: Vec::with_capacity(10),\n            from_change_id: 0,\n            to_change_id: 0,\n            container_change_id: None,\n            item_change_id: None,\n            is_truncated: false,\n        }\n    }\n}\n\nimpl Store {\n    pub async fn changes(\n        &self,\n        account_id: u32,\n        collection_: LogCollection,\n        query: Query,\n    ) -> trc::Result<Changes> {\n        let is_share_log = matches!(\n            collection_,\n            LogCollection::Sync(SyncCollection::ShareNotification)\n        );\n        let collection = u8::from(collection_);\n\n        let (is_inclusive, from_change_id, to_change_id) = match query {\n            Query::All => (true, 0, u64::MAX),\n            Query::Since(change_id) => (false, change_id, u64::MAX),\n            Query::SinceInclusive(change_id) => (true, change_id, u64::MAX),\n            Query::RangeInclusive(from_change_id, to_change_id) => {\n                (true, from_change_id, to_change_id)\n            }\n        };\n        let from_key = LogKey {\n            account_id,\n            collection,\n            change_id: from_change_id,\n        };\n        let to_key = LogKey {\n            account_id,\n            collection,\n            change_id: to_change_id,\n        };\n\n        let mut changelog = Changes::default();\n\n        self.iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let change_id = key.deserialize_be_u64(key.len() - U64_LEN)?;\n                if is_inclusive || change_id != from_change_id {\n                    if value.is_empty() {\n                        changelog.is_truncated = true;\n                        return Ok(true);\n                    }\n                    if changelog.changes.is_empty() {\n                        changelog.from_change_id = change_id;\n                    }\n                    changelog.to_change_id = change_id;\n                    if !is_share_log {\n                        let (has_container_changes, has_item_changes) =\n                            changelog.deserialize(value).ok_or_else(|| {\n                                trc::Error::corrupted_key(key, value.into(), trc::location!())\n                            })?;\n                        if has_container_changes {\n                            changelog.container_change_id = Some(change_id);\n                        }\n                        if has_item_changes {\n                            changelog.item_change_id = Some(change_id);\n                        }\n                    } else {\n                        changelog.changes.push(Change::InsertItem(change_id));\n                    }\n                } else {\n                    changelog.from_change_id = change_id;\n                    changelog.to_change_id = change_id;\n                }\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        Ok(changelog)\n    }\n\n    pub async fn vanished<T: DeserializeVanished>(\n        &self,\n        account_id: u32,\n        collection: LogCollection,\n        query: Query,\n    ) -> trc::Result<Vec<T>> {\n        let collection = u8::from(collection);\n        let (is_inclusive, from_change_id, to_change_id) = match query {\n            Query::All => (true, 0, u64::MAX),\n            Query::Since(change_id) => (false, change_id, u64::MAX),\n            Query::SinceInclusive(change_id) => (true, change_id, u64::MAX),\n            Query::RangeInclusive(from_change_id, to_change_id) => {\n                (true, from_change_id, to_change_id)\n            }\n        };\n        let from_key = LogKey {\n            account_id,\n            collection,\n            change_id: from_change_id,\n        };\n        let to_key = LogKey {\n            account_id,\n            collection,\n            change_id: to_change_id,\n        };\n\n        let mut vanished = Vec::default();\n\n        self.iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let change_id = key.deserialize_be_u64(key.len() - U64_LEN)?;\n                if is_inclusive || change_id != from_change_id {\n                    let mut iter = value.iter().peekable();\n\n                    while iter.peek().is_some() {\n                        if let Some(item) = T::deserialize_vanished(&mut iter) {\n                            vanished.push(item);\n                        } else {\n                            return Err(trc::Error::corrupted_key(\n                                key,\n                                value.into(),\n                                trc::location!(),\n                            ));\n                        }\n                    }\n                }\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        Ok(vanished)\n    }\n\n    pub async fn get_last_change_id(\n        &self,\n        account_id: u32,\n        collection: LogCollection,\n    ) -> trc::Result<Option<u64>> {\n        let collection = u8::from(collection);\n        let from_key = LogKey {\n            account_id,\n            collection,\n            change_id: 0,\n        };\n        let to_key = LogKey {\n            account_id,\n            collection,\n            change_id: u64::MAX,\n        };\n\n        let mut last_change_id = None;\n\n        self.iterate(\n            IterateParams::new(from_key, to_key)\n                .descending()\n                .no_values()\n                .only_first(),\n            |key, _| {\n                last_change_id = key.deserialize_be_u64(key.len() - U64_LEN)?.into();\n                Ok(false)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        Ok(last_change_id)\n    }\n}\n\nimpl From<VanishedCollection> for LogCollection {\n    fn from(value: VanishedCollection) -> Self {\n        LogCollection::Vanished(value)\n    }\n}\n\nimpl From<SyncCollection> for LogCollection {\n    fn from(value: SyncCollection) -> Self {\n        LogCollection::Sync(value)\n    }\n}\n\nimpl Changes {\n    pub fn deserialize(&mut self, bytes: &[u8]) -> Option<(bool, bool)> {\n        let mut bytes_it = bytes.iter();\n\n        let container_inserts: usize = bytes_it.next_leb128()?;\n        let container_updates: usize = bytes_it.next_leb128()?;\n        let container_property_changes: usize = bytes_it.next_leb128()?;\n        let container_deletes: usize = bytes_it.next_leb128()?;\n\n        let item_inserts: usize = bytes_it.next_leb128()?;\n        let item_updates: usize = bytes_it.next_leb128()?;\n        let item_deletes: usize = bytes_it.next_leb128()?;\n\n        let has_container_changes =\n            container_inserts + container_updates + container_property_changes + container_deletes\n                > 0;\n        let has_item_changes = item_inserts + item_updates + item_deletes > 0;\n\n        if container_inserts > 0 {\n            for _ in 0..container_inserts {\n                self.changes\n                    .push(Change::InsertContainer(bytes_it.next_leb128()?));\n            }\n        }\n\n        if container_updates > 0 || container_property_changes > 0 {\n            'update_outer: for change_pos in 0..(container_updates + container_property_changes) {\n                let id = bytes_it.next_leb128()?;\n                let mut is_property_change = change_pos >= container_updates;\n\n                for (idx, change) in self.changes.iter().enumerate() {\n                    match change {\n                        Change::InsertContainer(insert_id) if *insert_id == id => {\n                            // Item updated after inserted, no need to count this change.\n                            continue 'update_outer;\n                        }\n                        Change::UpdateContainer(update_id) if *update_id == id => {\n                            // Move update to the front\n                            is_property_change = false;\n                            self.changes.remove(idx);\n                            break;\n                        }\n                        Change::UpdateContainerProperty(update_id) if *update_id == id => {\n                            // Move update to the front\n                            self.changes.remove(idx);\n                            break;\n                        }\n                        _ => (),\n                    }\n                }\n\n                self.changes.push(if !is_property_change {\n                    Change::UpdateContainer(id)\n                } else {\n                    Change::UpdateContainerProperty(id)\n                });\n            }\n        }\n\n        if container_deletes > 0 {\n            'delete_outer: for _ in 0..container_deletes {\n                let id = bytes_it.next_leb128()?;\n\n                'delete_inner: for (idx, change) in self.changes.iter().enumerate() {\n                    match change {\n                        Change::InsertContainer(insert_id) if *insert_id == id => {\n                            self.changes.remove(idx);\n                            continue 'delete_outer;\n                        }\n                        Change::UpdateContainer(update_id) if *update_id == id => {\n                            self.changes.remove(idx);\n                            break 'delete_inner;\n                        }\n                        _ => (),\n                    }\n                }\n\n                self.changes.push(Change::DeleteContainer(id));\n            }\n        }\n\n        // Item changes\n        if item_inserts > 0 {\n            for _ in 0..item_inserts {\n                self.changes\n                    .push(Change::InsertItem(bytes_it.next_leb128()?));\n            }\n        }\n\n        if item_updates > 0 {\n            'update_outer: for _ in 0..item_updates {\n                let id = bytes_it.next_leb128()?;\n\n                for (idx, change) in self.changes.iter().enumerate() {\n                    match change {\n                        Change::InsertItem(insert_id) if *insert_id == id => {\n                            // Item updated after inserted, no need to count this change.\n                            continue 'update_outer;\n                        }\n                        Change::UpdateItem(update_id) if *update_id == id => {\n                            // Move update to the front\n                            self.changes.remove(idx);\n                            break;\n                        }\n                        _ => (),\n                    }\n                }\n\n                self.changes.push(Change::UpdateItem(id));\n            }\n        }\n\n        if item_deletes > 0 {\n            'delete_outer: for _ in 0..item_deletes {\n                let id = bytes_it.next_leb128()?;\n\n                'delete_inner: for (idx, change) in self.changes.iter().enumerate() {\n                    match change {\n                        Change::InsertItem(insert_id) if *insert_id == id => {\n                            self.changes.remove(idx);\n                            continue 'delete_outer;\n                        }\n                        Change::UpdateItem(update_id) if *update_id == id => {\n                            self.changes.remove(idx);\n                            break 'delete_inner;\n                        }\n                        _ => (),\n                    }\n                }\n\n                self.changes.push(Change::DeleteItem(id));\n            }\n        }\n\n        Some((has_container_changes, has_item_changes))\n    }\n}\n\nimpl Changes {\n    pub fn total_container_changes(&self) -> usize {\n        self.changes\n            .iter()\n            .filter(|change| change.is_container_change())\n            .count()\n    }\n\n    pub fn total_item_changes(&self) -> usize {\n        self.changes\n            .iter()\n            .filter(|change| change.is_item_change())\n            .count()\n    }\n}\n\nimpl Change {\n    pub fn item_id(&self) -> Option<u64> {\n        match self {\n            Change::InsertItem(id) => Some(*id),\n            Change::UpdateItem(id) => Some(*id),\n            Change::DeleteItem(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    pub fn container_id(&self) -> Option<u64> {\n        match self {\n            Change::InsertContainer(id) => Some(*id),\n            Change::UpdateContainer(id) => Some(*id),\n            Change::UpdateContainerProperty(id) => Some(*id),\n            Change::DeleteContainer(id) => Some(*id),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_item_id(self) -> Option<u64> {\n        match self {\n            Change::InsertItem(id) => Some(id),\n            Change::UpdateItem(id) => Some(id),\n            Change::DeleteItem(id) => Some(id),\n            _ => None,\n        }\n    }\n\n    pub fn try_unwrap_container_id(self) -> Option<u64> {\n        match self {\n            Change::InsertContainer(id) => Some(id),\n            Change::UpdateContainer(id) => Some(id),\n            Change::UpdateContainerProperty(id) => Some(id),\n            Change::DeleteContainer(id) => Some(id),\n            _ => None,\n        }\n    }\n\n    pub fn is_container_change(&self) -> bool {\n        matches!(\n            self,\n            Change::InsertContainer(_)\n                | Change::UpdateContainer(_)\n                | Change::UpdateContainerProperty(_)\n                | Change::DeleteContainer(_)\n        )\n    }\n\n    pub fn is_item_change(&self) -> bool {\n        matches!(\n            self,\n            Change::InsertItem(_) | Change::UpdateItem(_) | Change::DeleteItem(_)\n        )\n    }\n}\n\nimpl DeserializeVanished for u64 {\n    fn deserialize_vanished<'x>(bytes: &mut impl Iterator<Item = &'x u8>) -> Option<Self> {\n        let mut num = [0u8; U64_LEN];\n        for i in num.iter_mut() {\n            *i = *bytes.next()?;\n        }\n        Some(u64::from_be_bytes(num))\n    }\n}\n\nimpl DeserializeVanished for (u32, u32) {\n    fn deserialize_vanished<'x>(bytes: &mut impl Iterator<Item = &'x u8>) -> Option<Self> {\n        let mut num1 = [0u8; U32_LEN];\n        let mut num2 = [0u8; U32_LEN];\n        for i in num1.iter_mut().chain(num2.iter_mut()) {\n            *i = *bytes.next()?;\n        }\n        Some((u32::from_be_bytes(num1), u32::from_be_bytes(num2)))\n    }\n}\n\nimpl DeserializeVanished for String {\n    fn deserialize_vanished<'x>(bytes: &mut impl Iterator<Item = &'x u8>) -> Option<Self> {\n        let mut name = Vec::with_capacity(16);\n\n        loop {\n            let byte = bytes.next()?;\n            if *byte != 0 {\n                name.push(*byte);\n            } else {\n                break;\n            }\n        }\n\n        String::from_utf8(name).ok()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/query/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod log;\n\nuse crate::{IterateParams, Key};\n\nimpl<T: Key> IterateParams<T> {\n    pub fn new(begin: T, end: T) -> Self {\n        IterateParams {\n            begin,\n            end,\n            first: false,\n            ascending: true,\n            values: true,\n        }\n    }\n\n    pub fn set_ascending(mut self, ascending: bool) -> Self {\n        self.ascending = ascending;\n        self\n    }\n\n    pub fn set_values(mut self, values: bool) -> Self {\n        self.values = values;\n        self\n    }\n\n    pub fn ascending(mut self) -> Self {\n        self.ascending = true;\n        self\n    }\n\n    pub fn descending(mut self) -> Self {\n        self.ascending = false;\n        self\n    }\n\n    pub fn only_first(mut self) -> Self {\n        self.first = true;\n        self\n    }\n\n    pub fn no_values(mut self) -> Self {\n        self.values = false;\n        self\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/bm_u32.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    IterateParams, Store, U32_LEN, ValueKey,\n    search::*,\n    write::{\n        SEARCH_INDEX_MAX_FIELD_LEN, SearchIndex, SearchIndexClass, SearchIndexField, SearchIndexId,\n        SearchIndexType, ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n    },\n};\nuse ahash::AHashMap;\nuse roaring::RoaringBitmap;\nuse std::{\n    collections::hash_map::Entry,\n    ops::{BitAndAssign, BitOrAssign},\n};\nuse trc::AddContext;\nuse utils::cheeky_hash::CheekyHash;\n\n#[derive(Default)]\npub(super) struct BitmapCache {\n    cache: AHashMap<(CheekyHash, u8), Option<RoaringBitmap>>,\n}\n\nimpl BitmapCache {\n    pub async fn merge_bitmaps(\n        &mut self,\n        store: &Store,\n        index: SearchIndex,\n        account_id: u32,\n        hashes: impl Iterator<Item = CheekyHash>,\n        field: u8,\n        is_union: bool,\n    ) -> trc::Result<Option<RoaringBitmap>> {\n        let mut result = RoaringBitmap::new();\n        for (idx, hash) in hashes.enumerate() {\n            match self.cache.entry((hash, field)) {\n                Entry::Occupied(entry) => {\n                    if let Some(bm) = entry.get() {\n                        if is_union {\n                            result.bitor_assign(bm);\n                        } else if idx == 0 {\n                            result = bm.clone();\n                        } else {\n                            result.bitand_assign(bm);\n                            if result.is_empty() {\n                                return Ok(None);\n                            }\n                        }\n                    } else if !is_union {\n                        return Ok(None);\n                    }\n                }\n                Entry::Vacant(entry) => {\n                    let from_key = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: 0,\n                        },\n                        typ: SearchIndexType::Term { hash, field },\n                    }));\n                    let to_key = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: u32::MAX,\n                        },\n                        typ: SearchIndexType::Term { hash, field },\n                    }));\n                    let key_len = (U32_LEN * 2) + hash.len() + 2;\n                    let mut documents = RoaringBitmap::new();\n                    store\n                        .iterate(\n                            IterateParams::new(from_key, to_key).no_values().ascending(),\n                            |key, _| {\n                                if key.len() == key_len {\n                                    documents.insert(key.deserialize_be_u32(key.len() - U32_LEN)?);\n                                }\n\n                                Ok(true)\n                            },\n                        )\n                        .await\n                        .caused_by(trc::location!())?;\n\n                    if !documents.is_empty() {\n                        if is_union {\n                            result.bitor_assign(&documents);\n                        } else if idx == 0 {\n                            result = documents.clone();\n                        } else {\n                            result.bitand_assign(&documents);\n                            if result.is_empty() {\n                                entry.insert(Some(documents));\n                                return Ok(None);\n                            }\n                        }\n                        entry.insert(Some(documents));\n                    } else if !is_union {\n                        entry.insert(None);\n                        return Ok(None);\n                    }\n                }\n            }\n        }\n\n        if !result.is_empty() {\n            Ok(Some(result))\n        } else {\n            Ok(None)\n        }\n    }\n}\n\npub(crate) async fn range_to_bitmap(\n    store: &Store,\n    index: SearchIndex,\n    account_id: u32,\n    field_id: u8,\n    match_value: &[u8],\n    op: SearchOperator,\n) -> trc::Result<Option<RoaringBitmap>> {\n    let ((from_value, from_doc_id, from_field), (end_value, end_doc_id, end_field)) = match op {\n        SearchOperator::LowerThan => ((&[][..], 0, field_id), (match_value, 0, field_id)),\n        SearchOperator::LowerEqualThan => {\n            ((&[][..], 0, field_id), (match_value, u32::MAX, field_id))\n        }\n        SearchOperator::GreaterThan => (\n            (match_value, u32::MAX, field_id),\n            (&[][..], u32::MAX, field_id + 1),\n        ),\n        SearchOperator::GreaterEqualThan => (\n            (match_value, 0, field_id),\n            (&[][..], u32::MAX, field_id + 1),\n        ),\n        SearchOperator::Equal | SearchOperator::Contains => (\n            (match_value, 0, field_id),\n            (match_value, u32::MAX, field_id),\n        ),\n    };\n\n    let begin = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n        index,\n        id: SearchIndexId::Account {\n            account_id,\n            document_id: from_doc_id,\n        },\n        typ: SearchIndexType::Index {\n            field: SearchIndexField {\n                field_id: from_field,\n                data: from_value.to_vec(),\n            },\n        },\n    }));\n\n    let end = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n        index,\n        id: SearchIndexId::Account {\n            account_id,\n            document_id: end_doc_id,\n        },\n        typ: SearchIndexType::Index {\n            field: SearchIndexField {\n                field_id: end_field,\n                data: end_value.to_vec(),\n            },\n        },\n    }));\n\n    let mut bm = RoaringBitmap::new();\n    let prefix = KeySerializer::new(U32_LEN + 2)\n        .write(index.as_u8() | 1 << 6)\n        .write(account_id)\n        .write(field_id)\n        .finalize();\n    let prefix_len = prefix.len();\n\n    store\n        .iterate(\n            IterateParams::new(begin, end).no_values().ascending(),\n            |key, _| {\n                if !key.starts_with(&prefix) {\n                    return Ok(false);\n                }\n\n                let id_pos = key.len() - U32_LEN;\n                let value = key\n                    .get(prefix_len..id_pos)\n                    .ok_or_else(|| trc::Error::corrupted_key(key, None, trc::location!()))?;\n\n                let matches = match op {\n                    SearchOperator::LowerThan => value < match_value,\n                    SearchOperator::LowerEqualThan => value <= match_value,\n                    SearchOperator::GreaterThan => value > match_value,\n                    SearchOperator::GreaterEqualThan => value >= match_value,\n                    SearchOperator::Equal | SearchOperator::Contains => value == match_value,\n                };\n\n                if matches {\n                    bm.insert(key.deserialize_be_u32(id_pos)?);\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    if !bm.is_empty() {\n        Ok(Some(bm))\n    } else {\n        Ok(None)\n    }\n}\n\npub(crate) async fn sort_order(\n    store: &Store,\n    index: SearchIndex,\n    account_id: u32,\n    field_id: u8,\n) -> trc::Result<AHashMap<u32, u32>> {\n    let begin = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n        index,\n        id: SearchIndexId::Account {\n            account_id,\n            document_id: 0,\n        },\n        typ: SearchIndexType::Index {\n            field: SearchIndexField {\n                field_id,\n                data: vec![0u8],\n            },\n        },\n    }));\n    let end = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n        index,\n        id: SearchIndexId::Account {\n            account_id,\n            document_id: u32::MAX,\n        },\n        typ: SearchIndexType::Index {\n            field: SearchIndexField {\n                field_id,\n                data: vec![u8::MAX; SEARCH_INDEX_MAX_FIELD_LEN],\n            },\n        },\n    }));\n\n    let mut last_value = Vec::new();\n    let mut results = AHashMap::new();\n    let mut pos = 0;\n    store\n        .iterate(\n            IterateParams::new(begin, end).no_values().ascending(),\n            |key, _| {\n                let value = key\n                    .get(U32_LEN + 2..key.len() - U32_LEN)\n                    .ok_or_else(|| trc::Error::corrupted_key(key, None, trc::location!()))?;\n                if value != last_value {\n                    pos += 1;\n                    last_value = value.to_vec();\n                }\n\n                results.insert(key.deserialize_be_u32(key.len() - U32_LEN)?, pos);\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    Ok(results)\n}\n"
  },
  {
    "path": "crates/store/src/search/bm_u64.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    IterateParams, Store, U64_LEN, ValueKey,\n    search::*,\n    write::{\n        SearchIndex, SearchIndexClass, SearchIndexField, SearchIndexId, SearchIndexType,\n        ValueClass,\n        key::{DeserializeBigEndian, KeySerializer},\n    },\n};\nuse ahash::AHashMap;\nuse roaring::RoaringTreemap;\nuse std::{\n    collections::hash_map::Entry,\n    ops::{BitAndAssign, BitOrAssign},\n};\nuse trc::AddContext;\nuse utils::cheeky_hash::CheekyHash;\n\n#[derive(Default)]\npub(super) struct TreemapCache {\n    cache: AHashMap<(CheekyHash, u8), Option<RoaringTreemap>>,\n}\n\nimpl TreemapCache {\n    pub async fn merge_treemaps(\n        &mut self,\n        store: &Store,\n        index: SearchIndex,\n        hashes: impl Iterator<Item = CheekyHash>,\n        field: u8,\n        is_union: bool,\n    ) -> trc::Result<Option<RoaringTreemap>> {\n        let mut result = RoaringTreemap::new();\n        for (idx, hash) in hashes.enumerate() {\n            match self.cache.entry((hash, field)) {\n                Entry::Occupied(entry) => {\n                    if let Some(bm) = entry.get() {\n                        if is_union {\n                            result.bitor_assign(bm);\n                        } else if idx == 0 {\n                            result = bm.clone();\n                        } else {\n                            result.bitand_assign(bm);\n                            if result.is_empty() {\n                                return Ok(None);\n                            }\n                        }\n                    } else if !is_union {\n                        return Ok(None);\n                    }\n                }\n                Entry::Vacant(entry) => {\n                    let from_key = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Global { id: 0 },\n                        typ: SearchIndexType::Term { hash, field },\n                    }));\n                    let to_key = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Global { id: u64::MAX },\n                        typ: SearchIndexType::Term { hash, field },\n                    }));\n                    let key_len = U64_LEN + hash.len() + 2;\n                    let mut documents = RoaringTreemap::new();\n                    store\n                        .iterate(\n                            IterateParams::new(from_key, to_key).no_values().ascending(),\n                            |key, _| {\n                                if key.len() == key_len {\n                                    documents.insert(key.deserialize_be_u64(key.len() - U64_LEN)?);\n                                }\n\n                                Ok(true)\n                            },\n                        )\n                        .await\n                        .caused_by(trc::location!())?;\n\n                    if !documents.is_empty() {\n                        if is_union {\n                            result.bitor_assign(&documents);\n                        } else if idx == 0 {\n                            result = documents.clone();\n                        } else {\n                            result.bitand_assign(&documents);\n                            if result.is_empty() {\n                                entry.insert(Some(documents));\n                                return Ok(None);\n                            }\n                        }\n                        entry.insert(Some(documents));\n                    } else if !is_union {\n                        entry.insert(None);\n                        return Ok(None);\n                    }\n                }\n            }\n        }\n\n        if !result.is_empty() {\n            Ok(Some(result))\n        } else {\n            Ok(None)\n        }\n    }\n}\n\npub(crate) async fn range_to_treemap(\n    store: &Store,\n    index: SearchIndex,\n    field_id: u8,\n    match_value: &[u8],\n    op: SearchOperator,\n) -> trc::Result<Option<RoaringTreemap>> {\n    let ((from_value, from_id, from_field), (end_value, end_id, end_field)) = match op {\n        SearchOperator::LowerThan => ((&[][..], 0, field_id), (match_value, 0, field_id)),\n        SearchOperator::LowerEqualThan => {\n            ((&[][..], 0, field_id), (match_value, u64::MAX, field_id))\n        }\n        SearchOperator::GreaterThan => (\n            (match_value, u64::MAX, field_id),\n            (&[][..], u64::MAX, field_id + 1),\n        ),\n        SearchOperator::GreaterEqualThan => (\n            (match_value, 0, field_id),\n            (&[][..], u64::MAX, field_id + 1),\n        ),\n        SearchOperator::Equal | SearchOperator::Contains => (\n            (match_value, 0, field_id),\n            (match_value, u64::MAX, field_id),\n        ),\n    };\n\n    let begin = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n        index,\n        id: SearchIndexId::Global { id: from_id },\n        typ: SearchIndexType::Index {\n            field: SearchIndexField {\n                field_id: from_field,\n                data: from_value.to_vec(),\n            },\n        },\n    }));\n\n    let end = ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n        index,\n        id: SearchIndexId::Global { id: end_id },\n        typ: SearchIndexType::Index {\n            field: SearchIndexField {\n                field_id: end_field,\n                data: end_value.to_vec(),\n            },\n        },\n    }));\n\n    let mut bm = RoaringTreemap::new();\n    let prefix = KeySerializer::new(U64_LEN + 2)\n        .write(index.as_u8() | 1 << 6)\n        .write(field_id)\n        .finalize();\n    let prefix_len = prefix.len();\n\n    store\n        .iterate(\n            IterateParams::new(begin, end).no_values().ascending(),\n            |key, _| {\n                if !key.starts_with(&prefix) {\n                    return Ok(false);\n                }\n\n                let id_pos = key.len() - U64_LEN;\n                let value = key\n                    .get(prefix_len..id_pos)\n                    .ok_or_else(|| trc::Error::corrupted_key(key, None, trc::location!()))?;\n\n                let matches = match op {\n                    SearchOperator::LowerThan => value < match_value,\n                    SearchOperator::LowerEqualThan => value <= match_value,\n                    SearchOperator::GreaterThan => value > match_value,\n                    SearchOperator::GreaterEqualThan => value >= match_value,\n                    SearchOperator::Equal | SearchOperator::Contains => value == match_value,\n                };\n\n                if matches {\n                    bm.insert(key.deserialize_be_u64(id_pos)?);\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n    if !bm.is_empty() {\n        Ok(Some(bm))\n    } else {\n        Ok(None)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/document.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::search::*;\n\nimpl IndexDocument {\n    pub fn new(index: SearchIndex) -> Self {\n        Self {\n            fields: Default::default(),\n            index,\n        }\n    }\n\n    pub fn with_account_id(mut self, account_id: u32) -> Self {\n        self.fields\n            .insert(SearchField::AccountId, SearchValue::Uint(account_id as u64));\n        self\n    }\n\n    pub fn with_document_id(mut self, document_id: u32) -> Self {\n        self.fields.insert(\n            SearchField::DocumentId,\n            SearchValue::Uint(document_id as u64),\n        );\n        self\n    }\n\n    pub fn with_id(mut self, id: u64) -> Self {\n        self.fields.insert(SearchField::Id, SearchValue::Uint(id));\n        self\n    }\n\n    pub fn index_text(&mut self, field: impl Into<SearchField>, value: &str, language: Language) {\n        match self.fields.entry(field.into()) {\n            Entry::Occupied(mut entry) => {\n                if let SearchValue::Text {\n                    value: existing_value,\n                    ..\n                } = entry.get_mut()\n                {\n                    existing_value.push(' ');\n                    sanitize_text_to_buf(existing_value, value);\n                }\n            }\n            Entry::Vacant(entry) => {\n                entry.insert(SearchValue::Text {\n                    value: sanitize_text(value),\n                    language,\n                });\n            }\n        }\n    }\n\n    pub fn index_bool(&mut self, field: impl Into<SearchField>, value: bool) {\n        self.fields\n            .insert(field.into(), SearchValue::Boolean(value));\n    }\n\n    pub fn index_integer<N: Into<i64>>(&mut self, field: impl Into<SearchField>, value: N) {\n        self.fields\n            .insert(field.into(), SearchValue::Int(value.into()));\n    }\n\n    pub fn index_unsigned<N: Into<u64>>(&mut self, field: impl Into<SearchField>, value: N) {\n        self.fields\n            .insert(field.into(), SearchValue::Uint(value.into()));\n    }\n\n    pub fn index_keyword(&mut self, field: impl Into<SearchField>, value: impl AsRef<str>) {\n        self.fields.insert(\n            field.into(),\n            SearchValue::Text {\n                value: sanitize_text(value.as_ref()),\n                language: Language::None,\n            },\n        );\n    }\n\n    pub fn insert_key_value(\n        &mut self,\n        field: impl Into<SearchField>,\n        key: impl AsRef<str>,\n        value: impl AsRef<str>,\n    ) {\n        let search_field = field.into();\n        let key = key\n            .as_ref()\n            .chars()\n            .filter(|ch| !ch.is_control())\n            .map(|ch| ch.to_ascii_lowercase())\n            .collect::<String>();\n        let value = value.as_ref();\n\n        match self.fields.entry(search_field) {\n            Entry::Occupied(mut entry) => {\n                if let SearchValue::KeyValues(existing_key_values) = entry.get_mut() {\n                    if let Some(existing_value) = existing_key_values.get_mut(&key) {\n                        existing_value.push(' ');\n                        sanitize_text_to_buf(existing_value, value);\n                    } else {\n                        existing_key_values.append(key, sanitize_text(value));\n                    }\n                }\n            }\n            Entry::Vacant(entry) => {\n                let mut new_key_values = VecMap::new();\n                new_key_values.append(key, sanitize_text(value));\n                entry.insert(SearchValue::KeyValues(new_key_values));\n            }\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.fields.is_empty()\n    }\n\n    pub fn has_field(&self, field: &SearchField) -> bool {\n        self.fields.contains_key(field)\n    }\n\n    pub fn fields(&self) -> impl Iterator<Item = (&SearchField, &SearchValue)> {\n        self.fields.iter()\n    }\n\n    pub fn set_unknown_language(&mut self, lang: Language) {\n        for value in self.fields.values_mut() {\n            if let SearchValue::Text { language, .. } = value\n                && language.is_unknown()\n            {\n                *language = lang;\n            }\n        }\n    }\n}\n\nimpl SearchFilter {\n    pub fn cond(\n        field: impl Into<SearchField>,\n        op: SearchOperator,\n        value: impl Into<SearchValue>,\n    ) -> Self {\n        SearchFilter::Operator {\n            field: field.into(),\n            op,\n            value: value.into(),\n        }\n    }\n\n    pub fn eq(field: impl Into<SearchField>, value: impl Into<SearchValue>) -> Self {\n        SearchFilter::Operator {\n            field: field.into(),\n            op: SearchOperator::Equal,\n            value: value.into(),\n        }\n    }\n\n    pub fn lt(field: impl Into<SearchField>, value: impl Into<SearchValue>) -> Self {\n        SearchFilter::Operator {\n            field: field.into(),\n            op: SearchOperator::LowerThan,\n            value: value.into(),\n        }\n    }\n\n    pub fn le(field: impl Into<SearchField>, value: impl Into<SearchValue>) -> Self {\n        SearchFilter::Operator {\n            field: field.into(),\n            op: SearchOperator::LowerEqualThan,\n            value: value.into(),\n        }\n    }\n\n    pub fn gt(field: impl Into<SearchField>, value: impl Into<SearchValue>) -> Self {\n        SearchFilter::Operator {\n            field: field.into(),\n            op: SearchOperator::GreaterThan,\n            value: value.into(),\n        }\n    }\n\n    pub fn ge(field: impl Into<SearchField>, value: impl Into<SearchValue>) -> Self {\n        SearchFilter::Operator {\n            field: field.into(),\n            op: SearchOperator::GreaterEqualThan,\n            value: value.into(),\n        }\n    }\n\n    pub fn has_text_detect(\n        field: impl Into<SearchField>,\n        text: impl Into<String>,\n        default_language: Language,\n    ) -> Self {\n        let (text, language) = Language::detect(text.into(), default_language);\n        Self::has_text(field, text, language)\n    }\n\n    pub fn has_text(\n        field: impl Into<SearchField>,\n        text: impl Into<String>,\n        language: Language,\n    ) -> Self {\n        let text = text.into();\n        let (is_exact, text) = if let Some(text) = text\n            .strip_prefix('\"')\n            .and_then(|t| t.strip_suffix('\"'))\n            .or_else(|| text.strip_prefix('\\'').and_then(|t| t.strip_suffix('\\'')))\n        {\n            (true, text.to_string())\n        } else {\n            (false, text)\n        };\n\n        if !matches!(language, Language::None) && is_exact {\n            SearchFilter::Operator {\n                field: field.into(),\n                op: SearchOperator::Equal,\n                value: SearchValue::Text {\n                    value: text,\n                    language,\n                },\n            }\n        } else {\n            SearchFilter::Operator {\n                field: field.into(),\n                op: SearchOperator::Contains,\n                value: SearchValue::Text {\n                    value: text,\n                    language,\n                },\n            }\n        }\n    }\n\n    #[inline(always)]\n    pub fn has_english_text(field: impl Into<SearchField>, text: impl Into<String>) -> Self {\n        Self::has_text(field, text, Language::English)\n    }\n\n    #[inline(always)]\n    pub fn has_keyword(field: impl Into<SearchField>, text: impl Into<String>) -> Self {\n        Self::has_text(field, text, Language::None)\n    }\n\n    pub fn is_in_set(set: RoaringBitmap) -> Self {\n        SearchFilter::DocumentSet(set)\n    }\n}\n\nimpl SearchComparator {\n    pub fn field(field: impl Into<SearchField>, ascending: bool) -> Self {\n        Self::Field {\n            field: field.into(),\n            ascending,\n        }\n    }\n\n    pub fn set(set: RoaringBitmap, ascending: bool) -> Self {\n        Self::DocumentSet { set, ascending }\n    }\n\n    pub fn sorted_set(set: AHashMap<u32, u32>, ascending: bool) -> Self {\n        Self::SortedSet { set, ascending }\n    }\n\n    pub fn ascending(field: impl Into<SearchField>) -> Self {\n        Self::Field {\n            field: field.into(),\n            ascending: true,\n        }\n    }\n\n    pub fn descending(field: impl Into<SearchField>) -> Self {\n        Self::Field {\n            field: field.into(),\n            ascending: false,\n        }\n    }\n}\n\n#[inline(always)]\nfn write_sanitized(out: &mut String, text: &str) {\n    let mut last_is_space = true;\n    for ch in text.chars() {\n        match ch {\n            ' ' | '\\x09'..='\\x0d' => {\n                if !last_is_space {\n                    out.push(' ');\n                    last_is_space = true;\n                }\n            }\n            '\\0'..='\\x1f' | '\\x7f'..='\\u{9f}' => {}\n            ch => {\n                out.push(ch);\n                last_is_space = false;\n            }\n        }\n    }\n}\n\n#[inline(always)]\nfn sanitize_text_to_buf(out: &mut String, text: &str) {\n    out.reserve_exact(text.len());\n    write_sanitized(out, text);\n}\n\n#[inline(always)]\nfn sanitize_text(text: &str) -> String {\n    let mut out = String::with_capacity(text.len());\n    write_sanitized(&mut out, text);\n    out\n}\n"
  },
  {
    "path": "crates/store/src/search/fields.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::search::*;\n\nimpl SearchableField for EmailSearchField {\n    fn index() -> SearchIndex {\n        SearchIndex::Email\n    }\n\n    fn primary_keys() -> &'static [SearchField] {\n        &[SearchField::AccountId, SearchField::DocumentId]\n    }\n\n    fn all_fields() -> &'static [SearchField] {\n        &[\n            SearchField::Email(EmailSearchField::From),\n            SearchField::Email(EmailSearchField::To),\n            SearchField::Email(EmailSearchField::Cc),\n            SearchField::Email(EmailSearchField::Bcc),\n            SearchField::Email(EmailSearchField::Subject),\n            SearchField::Email(EmailSearchField::Body),\n            SearchField::Email(EmailSearchField::Attachment),\n            SearchField::Email(EmailSearchField::ReceivedAt),\n            SearchField::Email(EmailSearchField::SentAt),\n            SearchField::Email(EmailSearchField::Size),\n            SearchField::Email(EmailSearchField::HasAttachment),\n            SearchField::Email(EmailSearchField::Headers),\n        ]\n    }\n\n    fn is_indexed(&self) -> bool {\n        #[cfg(not(feature = \"test_mode\"))]\n        {\n            matches!(\n                self,\n                EmailSearchField::From\n                    | EmailSearchField::To\n                    | EmailSearchField::Subject\n                    | EmailSearchField::ReceivedAt\n                    | EmailSearchField::SentAt\n                    | EmailSearchField::Size\n                    | EmailSearchField::HasAttachment,\n            )\n        }\n\n        #[cfg(feature = \"test_mode\")]\n        {\n            matches!(\n                self,\n                EmailSearchField::From\n                    | EmailSearchField::To\n                    | EmailSearchField::Subject\n                    | EmailSearchField::ReceivedAt\n                    | EmailSearchField::SentAt\n                    | EmailSearchField::Size\n                    | EmailSearchField::HasAttachment\n                    | EmailSearchField::Bcc\n                    | EmailSearchField::Cc\n            )\n        }\n    }\n\n    fn is_text(&self) -> bool {\n        matches!(\n            self,\n            EmailSearchField::From\n                | EmailSearchField::To\n                | EmailSearchField::Cc\n                | EmailSearchField::Bcc\n                | EmailSearchField::Subject\n                | EmailSearchField::Body\n                | EmailSearchField::Attachment,\n        )\n    }\n}\n\nimpl SearchableField for CalendarSearchField {\n    fn index() -> SearchIndex {\n        SearchIndex::Calendar\n    }\n\n    fn primary_keys() -> &'static [SearchField] {\n        &[SearchField::AccountId, SearchField::DocumentId]\n    }\n\n    fn all_fields() -> &'static [SearchField] {\n        &[\n            SearchField::Calendar(CalendarSearchField::Title),\n            SearchField::Calendar(CalendarSearchField::Description),\n            SearchField::Calendar(CalendarSearchField::Location),\n            SearchField::Calendar(CalendarSearchField::Owner),\n            SearchField::Calendar(CalendarSearchField::Attendee),\n            SearchField::Calendar(CalendarSearchField::Start),\n            SearchField::Calendar(CalendarSearchField::Uid),\n        ]\n    }\n\n    fn is_indexed(&self) -> bool {\n        matches!(self, CalendarSearchField::Start | CalendarSearchField::Uid)\n    }\n\n    fn is_text(&self) -> bool {\n        !self.is_indexed()\n    }\n}\n\nimpl SearchableField for ContactSearchField {\n    fn index() -> SearchIndex {\n        SearchIndex::Contacts\n    }\n\n    fn primary_keys() -> &'static [SearchField] {\n        &[SearchField::AccountId, SearchField::DocumentId]\n    }\n\n    fn all_fields() -> &'static [SearchField] {\n        &[\n            SearchField::Contact(ContactSearchField::Member),\n            SearchField::Contact(ContactSearchField::Kind),\n            SearchField::Contact(ContactSearchField::Name),\n            SearchField::Contact(ContactSearchField::Nickname),\n            SearchField::Contact(ContactSearchField::Organization),\n            SearchField::Contact(ContactSearchField::Email),\n            SearchField::Contact(ContactSearchField::Phone),\n            SearchField::Contact(ContactSearchField::OnlineService),\n            SearchField::Contact(ContactSearchField::Address),\n            SearchField::Contact(ContactSearchField::Note),\n            SearchField::Contact(ContactSearchField::Uid),\n        ]\n    }\n\n    fn is_indexed(&self) -> bool {\n        matches!(self, ContactSearchField::Uid | ContactSearchField::Kind)\n    }\n\n    fn is_text(&self) -> bool {\n        !self.is_indexed()\n    }\n}\n\nimpl SearchableField for FileSearchField {\n    fn index() -> SearchIndex {\n        SearchIndex::File\n    }\n\n    fn primary_keys() -> &'static [SearchField] {\n        &[SearchField::AccountId, SearchField::DocumentId]\n    }\n\n    fn all_fields() -> &'static [SearchField] {\n        &[\n            SearchField::File(FileSearchField::Name),\n            SearchField::File(FileSearchField::Content),\n        ]\n    }\n\n    fn is_indexed(&self) -> bool {\n        false\n    }\n\n    fn is_text(&self) -> bool {\n        true\n    }\n}\n\nimpl SearchableField for TracingSearchField {\n    fn index() -> SearchIndex {\n        SearchIndex::Tracing\n    }\n\n    fn primary_keys() -> &'static [SearchField] {\n        &[SearchField::Id]\n    }\n\n    fn all_fields() -> &'static [SearchField] {\n        &[\n            SearchField::Tracing(TracingSearchField::EventType),\n            SearchField::Tracing(TracingSearchField::QueueId),\n            SearchField::Tracing(TracingSearchField::Keywords),\n        ]\n    }\n\n    fn is_indexed(&self) -> bool {\n        matches!(\n            self,\n            TracingSearchField::QueueId | TracingSearchField::EventType\n        )\n    }\n\n    fn is_text(&self) -> bool {\n        matches!(self, TracingSearchField::Keywords)\n    }\n}\n\nimpl SearchField {\n    pub(crate) fn is_indexed(&self) -> bool {\n        match self {\n            SearchField::Email(field) => field.is_indexed(),\n            SearchField::Calendar(field) => field.is_indexed(),\n            SearchField::Contact(field) => field.is_indexed(),\n            SearchField::File(field) => field.is_indexed(),\n            SearchField::Tracing(field) => field.is_indexed(),\n            SearchField::AccountId | SearchField::DocumentId | SearchField::Id => false,\n        }\n    }\n\n    pub(crate) fn is_text(&self) -> bool {\n        match self {\n            SearchField::Email(field) => field.is_text(),\n            SearchField::Calendar(field) => field.is_text(),\n            SearchField::Contact(field) => field.is_text(),\n            SearchField::File(field) => field.is_text(),\n            SearchField::Tracing(field) => field.is_text(),\n            SearchField::AccountId | SearchField::DocumentId | SearchField::Id => false,\n        }\n    }\n\n    pub(crate) fn is_json(&self) -> bool {\n        matches!(self, SearchField::Email(EmailSearchField::Headers))\n    }\n}\n\nimpl SearchIndex {\n    pub fn all_fields(&self) -> &[SearchField] {\n        match self {\n            SearchIndex::Email => EmailSearchField::all_fields(),\n            SearchIndex::Calendar => CalendarSearchField::all_fields(),\n            SearchIndex::Contacts => ContactSearchField::all_fields(),\n            SearchIndex::File => FileSearchField::all_fields(),\n            SearchIndex::Tracing => TracingSearchField::all_fields(),\n            SearchIndex::InMemory => unreachable!(),\n        }\n    }\n\n    pub fn primary_keys(&self) -> &'static [SearchField] {\n        match self {\n            SearchIndex::Email => EmailSearchField::primary_keys(),\n            SearchIndex::Calendar => CalendarSearchField::primary_keys(),\n            SearchIndex::Contacts => ContactSearchField::primary_keys(),\n            SearchIndex::File => FileSearchField::primary_keys(),\n            SearchIndex::Tracing => TracingSearchField::primary_keys(),\n            SearchIndex::InMemory => unreachable!(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/index.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    Deserialize, IterateParams, Store, U64_LEN, ValueKey,\n    search::{\n        IndexDocument, SearchField, SearchFilter, SearchOperator, SearchQuery, SearchValue,\n        term::{TermIndex, TermIndexBuilder},\n    },\n    write::{\n        AlignedBytes, Archive, BatchBuilder, SEARCH_INDEX_MAX_FIELD_LEN, SearchIndex,\n        SearchIndexClass, SearchIndexField, SearchIndexId, SearchIndexType, ValueClass,\n        key::DeserializeBigEndian,\n    },\n};\nuse ahash::AHashMap;\nuse trc::AddContext;\nuse utils::cheeky_hash::CheekyHash;\n\nimpl Store {\n    pub(crate) async fn index(&self, documents: Vec<IndexDocument>) -> trc::Result<()> {\n        let truncate_at = if self.is_foundationdb() { 1_048_576 } else { 0 };\n\n        for document in documents {\n            let mut batch = BatchBuilder::new();\n            let index = document.index;\n            let mut old_term_index = None;\n\n            if matches!(index, SearchIndex::Calendar | SearchIndex::Contacts) {\n                let mut account_id = None;\n                let mut document_id = None;\n                for (field, value) in &document.fields {\n                    if let SearchValue::Uint(id) = value {\n                        match field {\n                            SearchField::AccountId => {\n                                account_id = Some(*id as u32);\n                            }\n                            SearchField::DocumentId => {\n                                document_id = Some(*id as u32);\n                            }\n                            _ => {}\n                        }\n                    }\n                }\n\n                if let (Some(account_id), Some(document_id)) = (account_id, document_id)\n                    && let Some(archive) = self\n                        .get_value::<Archive<AlignedBytes>>(ValueKey::from(\n                            ValueClass::SearchIndex(SearchIndexClass {\n                                index,\n                                id: SearchIndexId::Account {\n                                    account_id,\n                                    document_id,\n                                },\n                                typ: SearchIndexType::Document,\n                            }),\n                        ))\n                        .await\n                        .caused_by(trc::location!())?\n                {\n                    old_term_index = Some(archive);\n                }\n            }\n\n            let term_index_builder = TermIndexBuilder::build(document, truncate_at);\n            if let Some(old_term_index) = old_term_index {\n                let old_term_index = old_term_index\n                    .unarchive::<TermIndex>()\n                    .caused_by(trc::location!())?;\n                term_index_builder\n                    .index\n                    .merge_index(&mut batch, index, term_index_builder.id, old_term_index)\n                    .caused_by(trc::location!())?;\n            } else {\n                term_index_builder\n                    .index\n                    .write_index(&mut batch, index, term_index_builder.id)\n                    .caused_by(trc::location!())?;\n            }\n\n            let mut commit_points = batch.commit_points();\n            for commit_point in commit_points.iter() {\n                let batch = batch.build_one(commit_point);\n                self.write(batch).await.caused_by(trc::location!())?;\n            }\n        }\n        Ok(())\n    }\n\n    pub(crate) async fn unindex(&self, query: SearchQuery) -> trc::Result<()> {\n        let index = query.index;\n        let mut account_documents: AHashMap<u32, Vec<u32>> = AHashMap::new();\n        let mut ids = vec![];\n        let mut to_id = None;\n        let mut last_account_id = None;\n\n        for filter in query.filters {\n            match filter {\n                SearchFilter::Operator { field, op, value } => match (field, value) {\n                    (SearchField::AccountId, SearchValue::Uint(id))\n                        if op == SearchOperator::Equal =>\n                    {\n                        last_account_id = Some(id as u32);\n                        account_documents.entry(id as u32).or_default();\n                    }\n                    (SearchField::DocumentId, SearchValue::Uint(id))\n                        if op == SearchOperator::Equal && last_account_id.is_some() =>\n                    {\n                        account_documents\n                            .get_mut(&last_account_id.unwrap())\n                            .unwrap()\n                            .push(id as u32);\n                    }\n                    (SearchField::Id, SearchValue::Uint(id)) => match op {\n                        SearchOperator::LowerThan => {\n                            to_id = Some(id.saturating_sub(1));\n                        }\n                        SearchOperator::LowerEqualThan => {\n                            to_id = Some(id);\n                        }\n                        SearchOperator::Equal => {\n                            ids.push(id);\n                        }\n                        _ => {\n                            return Err(trc::StoreEvent::UnexpectedError\n                                .into_err()\n                                .reason(\"Unsupported operator for Id field\"));\n                        }\n                    },\n                    filter => {\n                        return Err(trc::StoreEvent::UnexpectedError\n                            .into_err()\n                            .details(format!(\"Unsupported unindex filter {filter:?}\")));\n                    }\n                },\n                SearchFilter::And | SearchFilter::Or | SearchFilter::End => {}\n                SearchFilter::Not | SearchFilter::DocumentSet(_) => {\n                    return Err(trc::StoreEvent::UnexpectedError\n                        .into_err()\n                        .details(format!(\"Unsupported unindex filter {filter:?}\")));\n                }\n            }\n        }\n\n        // Delete by account and document ids\n        for (account_id, document_ids) in account_documents {\n            if !document_ids.is_empty() {\n                for document_id in document_ids {\n                    let Some(archive) = self\n                        .get_value::<Archive<AlignedBytes>>(ValueKey::from(\n                            ValueClass::SearchIndex(SearchIndexClass {\n                                index,\n                                id: SearchIndexId::Account {\n                                    account_id,\n                                    document_id,\n                                },\n                                typ: SearchIndexType::Document,\n                            }),\n                        ))\n                        .await\n                        .caused_by(trc::location!())?\n                    else {\n                        continue;\n                    };\n                    let term_index = archive\n                        .unarchive::<TermIndex>()\n                        .caused_by(trc::location!())?;\n                    let mut batch = BatchBuilder::new();\n                    term_index.delete_index(\n                        &mut batch,\n                        index,\n                        SearchIndexId::Account {\n                            account_id,\n                            document_id,\n                        },\n                    );\n                    self.write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n            } else {\n                // Delete all documents for the account\n                self.delete_range(\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: 0,\n                        },\n                        typ: SearchIndexType::Document,\n                    })),\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: u32::MAX,\n                        },\n                        typ: SearchIndexType::Document,\n                    })),\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n                self.delete_range(\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: 0,\n                        },\n                        typ: SearchIndexType::Index {\n                            field: SearchIndexField {\n                                field_id: 0,\n                                data: vec![0u8],\n                            },\n                        },\n                    })),\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: u32::MAX,\n                        },\n                        typ: SearchIndexType::Index {\n                            field: SearchIndexField {\n                                field_id: u8::MAX,\n                                data: vec![u8::MAX; SEARCH_INDEX_MAX_FIELD_LEN],\n                            },\n                        },\n                    })),\n                )\n                .await\n                .caused_by(trc::location!())?;\n\n                self.delete_range(\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: 0,\n                        },\n                        typ: SearchIndexType::Term {\n                            hash: CheekyHash::NULL,\n                            field: 0,\n                        },\n                    })),\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Account {\n                            account_id,\n                            document_id: u32::MAX,\n                        },\n                        typ: SearchIndexType::Term {\n                            hash: CheekyHash::FULL,\n                            field: u8::MAX,\n                        },\n                    })),\n                )\n                .await\n                .caused_by(trc::location!())?;\n            }\n        }\n\n        // Delete by global ids\n        for id in ids {\n            let Some(archive) = self\n                .get_value::<Archive<AlignedBytes>>(ValueKey::from(ValueClass::SearchIndex(\n                    SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Global { id },\n                        typ: SearchIndexType::Document,\n                    },\n                )))\n                .await\n                .caused_by(trc::location!())?\n            else {\n                continue;\n            };\n            let term_index = archive\n                .unarchive::<TermIndex>()\n                .caused_by(trc::location!())?;\n            let mut batch = BatchBuilder::new();\n            term_index.delete_index(&mut batch, index, SearchIndexId::Global { id });\n            self.write(batch.build_all())\n                .await\n                .caused_by(trc::location!())?;\n        }\n\n        // Delete ranges\n        if let Some(to_id) = to_id {\n            let mut batches = Vec::new();\n            self.iterate(\n                IterateParams::new(\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Global { id: 0 },\n                        typ: SearchIndexType::Document,\n                    })),\n                    ValueKey::from(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id: SearchIndexId::Global { id: to_id },\n                        typ: SearchIndexType::Document,\n                    })),\n                ),\n                |key, value| {\n                    let archive = <Archive<AlignedBytes> as Deserialize>::deserialize(value)?;\n                    let term_index = archive.unarchive::<TermIndex>()?;\n                    let mut batch = BatchBuilder::new();\n                    term_index.delete_index(\n                        &mut batch,\n                        index,\n                        SearchIndexId::Global {\n                            id: key.deserialize_be_u64(key.len() - U64_LEN)?,\n                        },\n                    );\n                    batches.push(batch);\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n            for mut batch in batches {\n                self.write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n        }\n\n        Ok(())\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/local.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::search::*;\nuse roaring::RoaringBitmap;\n\nstruct State {\n    pub op: SearchFilter,\n    pub bm: Option<RoaringBitmap>,\n}\n\nimpl SearchQuery {\n    pub fn new(index: SearchIndex) -> Self {\n        Self {\n            index,\n            filters: Vec::new(),\n            comparators: Vec::new(),\n            mask: RoaringBitmap::new(),\n        }\n    }\n\n    pub fn with_filters(mut self, filters: Vec<SearchFilter>) -> Self {\n        if self.filters.is_empty() {\n            self.filters = filters;\n        } else {\n            self.filters.extend(filters);\n        }\n        self\n    }\n\n    pub fn with_comparators(mut self, comparators: Vec<SearchComparator>) -> Self {\n        if self.comparators.is_empty() {\n            self.comparators = comparators;\n        } else {\n            self.comparators.extend(comparators);\n        }\n        self\n    }\n\n    pub fn with_filter(mut self, filter: SearchFilter) -> Self {\n        self.filters.push(filter);\n        self\n    }\n\n    pub fn add_filter(&mut self, filter: SearchFilter) -> &mut Self {\n        self.filters.push(filter);\n        self\n    }\n\n    pub fn with_comparator(mut self, comparator: SearchComparator) -> Self {\n        self.comparators.push(comparator);\n        self\n    }\n\n    pub fn with_mask(mut self, mask: RoaringBitmap) -> Self {\n        self.mask = mask;\n        self\n    }\n\n    pub fn with_account_id(mut self, account_id: u32) -> Self {\n        self.filters.push(SearchFilter::cond(\n            SearchField::AccountId,\n            SearchOperator::Equal,\n            SearchValue::Uint(account_id as u64),\n        ));\n        self\n    }\n\n    pub fn filter(self) -> QueryResults {\n        if self.filters.is_empty() {\n            return QueryResults {\n                results: self.mask,\n                comparators: self.comparators,\n            };\n        }\n        let mut state: State = State {\n            op: SearchFilter::And,\n            bm: None,\n        };\n        let mut stack = Vec::new();\n        let mut filters = self.filters.into_iter().peekable();\n        let mask = self.mask;\n\n        while let Some(filter) = filters.next() {\n            let mut result = match filter {\n                SearchFilter::DocumentSet(set) => Some(set),\n                op @ (SearchFilter::And | SearchFilter::Or | SearchFilter::Not) => {\n                    stack.push(state);\n                    state = State { op, bm: None };\n                    continue;\n                }\n                SearchFilter::End => {\n                    if let Some(prev_state) = stack.pop() {\n                        let bm = state.bm;\n                        state = prev_state;\n                        bm\n                    } else {\n                        break;\n                    }\n                }\n                SearchFilter::Operator { .. } => {\n                    continue;\n                }\n            };\n\n            // Apply logical operation\n            if let Some(dest) = &mut state.bm {\n                match state.op {\n                    SearchFilter::And => {\n                        if let Some(result) = result {\n                            dest.bitand_assign(result);\n                        } else {\n                            dest.clear();\n                        }\n                    }\n                    SearchFilter::Or => {\n                        if let Some(result) = result {\n                            dest.bitor_assign(result);\n                        }\n                    }\n                    SearchFilter::Not => {\n                        if let Some(mut result) = result {\n                            result.bitxor_assign(&mask);\n                            dest.bitand_assign(result);\n                        }\n                    }\n                    _ => unreachable!(),\n                }\n            } else if let Some(ref mut result_) = result {\n                if let SearchFilter::Not = state.op {\n                    result_.bitxor_assign(&mask);\n                }\n                state.bm = result;\n            } else if let SearchFilter::Not = state.op {\n                state.bm = Some(mask.clone());\n            } else {\n                state.bm = Some(RoaringBitmap::new());\n            }\n\n            // And short-circuit\n            if matches!(state.op, SearchFilter::And) && state.bm.as_ref().unwrap().is_empty() {\n                while let Some(filter) = filters.peek() {\n                    if matches!(filter, SearchFilter::End) {\n                        break;\n                    } else {\n                        filters.next();\n                    }\n                }\n            }\n        }\n\n        // AND with mask\n        let mut results = state.bm.unwrap_or_default();\n        results.bitand_assign(&mask);\n        QueryResults {\n            results,\n            comparators: self.comparators,\n        }\n    }\n}\n\nimpl QueryResults {\n    pub fn new(results: RoaringBitmap, comparators: Vec<SearchComparator>) -> Self {\n        Self {\n            results,\n            comparators,\n        }\n    }\n\n    pub fn with_comparators(mut self, comparators: Vec<SearchComparator>) -> Self {\n        if self.comparators.is_empty() {\n            self.comparators = comparators;\n        } else {\n            self.comparators.extend(comparators);\n        }\n        self\n    }\n\n    pub fn results(&self) -> &RoaringBitmap {\n        &self.results\n    }\n\n    pub fn update_results(&mut self, results: RoaringBitmap) {\n        self.results = results;\n    }\n\n    pub fn into_bitmap(self) -> RoaringBitmap {\n        self.results\n    }\n\n    pub fn into_sorted(self) -> Vec<u32> {\n        let comparators = self.comparators;\n        let mut results = self.results.into_iter().collect::<Vec<u32>>();\n\n        if !results.is_empty() && !comparators.is_empty() {\n            results.sort_by(|a, b| {\n                for comparator in &comparators {\n                    let (a, b, is_ascending) = match comparator {\n                        SearchComparator::DocumentSet { set, ascending } => (\n                            !set.contains(*a) as u32,\n                            !set.contains(*b) as u32,\n                            *ascending,\n                        ),\n                        SearchComparator::SortedSet { set, ascending } => (\n                            *set.get(a).unwrap_or(&u32::MAX),\n                            *set.get(b).unwrap_or(&u32::MAX),\n                            *ascending,\n                        ),\n                        SearchComparator::Field { .. } => continue,\n                    };\n\n                    let ordering = if is_ascending { a.cmp(&b) } else { b.cmp(&a) };\n\n                    if ordering != Ordering::Equal {\n                        return ordering;\n                    }\n                }\n                Ordering::Equal\n            });\n        }\n\n        results\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod bm_u32;\npub mod bm_u64;\npub mod document;\npub mod fields;\npub mod index;\npub mod local;\npub mod query;\npub mod split;\npub mod term;\n\nuse crate::write::SearchIndex;\nuse ahash::AHashMap;\nuse nlp::language::Language;\nuse roaring::RoaringBitmap;\nuse std::cmp::Ordering;\nuse std::collections::hash_map::Entry;\nuse std::fmt::Display;\nuse std::ops::{BitAndAssign, BitOrAssign, BitXorAssign};\nuse utils::config::utils::ParseValue;\nuse utils::map::vec_map::VecMap;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum SearchOperator {\n    LowerThan,\n    LowerEqualThan,\n    GreaterThan,\n    GreaterEqualThan,\n    Equal,\n    Contains,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\npub enum SearchField {\n    AccountId,\n    DocumentId,\n    Id,\n    Email(EmailSearchField),\n    Calendar(CalendarSearchField),\n    Contact(ContactSearchField),\n    File(FileSearchField),\n    Tracing(TracingSearchField),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\npub enum EmailSearchField {\n    From,\n    To,\n    Cc,\n    Bcc,\n    Subject,\n    Body,\n    Attachment,\n    ReceivedAt,\n    SentAt,\n    Size,\n    HasAttachment,\n    Headers,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum CalendarSearchField {\n    Title,\n    Description,\n    Location,\n    Owner,\n    Attendee,\n    Start,\n    Uid,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum ContactSearchField {\n    Member,\n    Kind,\n    Name,\n    Nickname,\n    Organization,\n    Email,\n    Phone,\n    OnlineService,\n    Address,\n    Note,\n    Uid,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum FileSearchField {\n    Name,\n    Content,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum TracingSearchField {\n    EventType,\n    QueueId,\n    Keywords,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum SearchValue {\n    Text { value: String, language: Language },\n    KeyValues(VecMap<String, String>),\n    Int(i64),\n    Uint(u64),\n    Boolean(bool),\n}\n\npub trait SearchDocumentId: Sized + Copy + Display {\n    fn from_u64(id: u64) -> Self;\n    fn field() -> SearchField;\n}\n\n#[derive(Debug)]\npub struct SearchQuery {\n    pub(crate) index: SearchIndex,\n    pub(crate) filters: Vec<SearchFilter>,\n    pub(crate) comparators: Vec<SearchComparator>,\n    pub(crate) mask: RoaringBitmap,\n}\n\n#[derive(Debug, PartialEq, Clone, Default)]\npub enum SearchFilter {\n    Operator {\n        field: SearchField,\n        op: SearchOperator,\n        value: SearchValue,\n    },\n    DocumentSet(RoaringBitmap),\n    And,\n    Or,\n    Not,\n    #[default]\n    End,\n}\n\n#[derive(Debug)]\npub enum SearchComparator {\n    Field {\n        field: SearchField,\n        ascending: bool,\n    },\n    DocumentSet {\n        set: RoaringBitmap,\n        ascending: bool,\n    },\n    SortedSet {\n        set: AHashMap<u32, u32>,\n        ascending: bool,\n    },\n}\n\n#[derive(Debug)]\npub struct IndexDocument {\n    pub(crate) index: SearchIndex,\n    pub(crate) fields: AHashMap<SearchField, SearchValue>,\n}\n\n#[derive(Debug)]\npub struct QueryResults {\n    results: RoaringBitmap,\n    comparators: Vec<SearchComparator>,\n}\n\nimpl From<EmailSearchField> for SearchField {\n    fn from(field: EmailSearchField) -> Self {\n        SearchField::Email(field)\n    }\n}\n\nimpl From<CalendarSearchField> for SearchField {\n    fn from(field: CalendarSearchField) -> Self {\n        SearchField::Calendar(field)\n    }\n}\n\nimpl From<ContactSearchField> for SearchField {\n    fn from(field: ContactSearchField) -> Self {\n        SearchField::Contact(field)\n    }\n}\n\nimpl From<FileSearchField> for SearchField {\n    fn from(field: FileSearchField) -> Self {\n        SearchField::File(field)\n    }\n}\n\nimpl From<TracingSearchField> for SearchField {\n    fn from(field: TracingSearchField) -> Self {\n        SearchField::Tracing(field)\n    }\n}\n\nimpl From<u64> for SearchValue {\n    fn from(value: u64) -> Self {\n        SearchValue::Uint(value)\n    }\n}\n\nimpl From<i64> for SearchValue {\n    fn from(value: i64) -> Self {\n        SearchValue::Int(value)\n    }\n}\n\nimpl From<u32> for SearchValue {\n    fn from(value: u32) -> Self {\n        SearchValue::Uint(value as u64)\n    }\n}\n\nimpl From<i32> for SearchValue {\n    fn from(value: i32) -> Self {\n        SearchValue::Int(value as i64)\n    }\n}\n\nimpl From<usize> for SearchValue {\n    fn from(value: usize) -> Self {\n        SearchValue::Uint(value as u64)\n    }\n}\n\nimpl From<bool> for SearchValue {\n    fn from(value: bool) -> Self {\n        SearchValue::Boolean(value)\n    }\n}\n\nimpl From<String> for SearchValue {\n    fn from(value: String) -> Self {\n        SearchValue::Text {\n            value,\n            language: Language::None,\n        }\n    }\n}\n\nimpl SearchDocumentId for u32 {\n    fn from_u64(id: u64) -> Self {\n        id as u32\n    }\n\n    fn field() -> SearchField {\n        SearchField::DocumentId\n    }\n}\n\nimpl SearchDocumentId for u64 {\n    fn from_u64(id: u64) -> Self {\n        id\n    }\n\n    fn field() -> SearchField {\n        SearchField::Id\n    }\n}\n\npub trait SearchableField: Sized {\n    fn index() -> SearchIndex;\n    fn primary_keys() -> &'static [SearchField];\n    fn all_fields() -> &'static [SearchField];\n    fn is_indexed(&self) -> bool;\n    fn is_text(&self) -> bool;\n}\n\nimpl ParseValue for SearchField {\n    fn parse_value(value: &str) -> utils::config::Result<Self> {\n        Ok(match value {\n            // Email\n            \"email-from\" => Self::Email(EmailSearchField::From),\n            \"email-to\" => Self::Email(EmailSearchField::To),\n            \"email-cc\" => Self::Email(EmailSearchField::Cc),\n            \"email-bcc\" => Self::Email(EmailSearchField::Bcc),\n            \"email-subject\" => Self::Email(EmailSearchField::Subject),\n            \"email-body\" => Self::Email(EmailSearchField::Body),\n            \"email-attachment\" => Self::Email(EmailSearchField::Attachment),\n            \"email-received-at\" => Self::Email(EmailSearchField::ReceivedAt),\n            \"email-sent-at\" => Self::Email(EmailSearchField::SentAt),\n            \"email-size\" => Self::Email(EmailSearchField::Size),\n            \"email-has-attachment\" => Self::Email(EmailSearchField::HasAttachment),\n            \"email-headers\" => Self::Email(EmailSearchField::Headers),\n\n            // Calendar\n            \"cal-title\" => Self::Calendar(CalendarSearchField::Title),\n            \"cal-desc\" => Self::Calendar(CalendarSearchField::Description),\n            \"cal-location\" => Self::Calendar(CalendarSearchField::Location),\n            \"cal-owner\" => Self::Calendar(CalendarSearchField::Owner),\n            \"cal-attendee\" => Self::Calendar(CalendarSearchField::Attendee),\n            \"cal-start\" => Self::Calendar(CalendarSearchField::Start),\n            \"cal-uid\" => Self::Calendar(CalendarSearchField::Uid),\n\n            // Contact\n            \"contact-member\" => Self::Contact(ContactSearchField::Member),\n            \"contact-kind\" => Self::Contact(ContactSearchField::Kind),\n            \"contact-name\" => Self::Contact(ContactSearchField::Name),\n            \"contact-nickname\" => Self::Contact(ContactSearchField::Nickname),\n            \"contact-org\" => Self::Contact(ContactSearchField::Organization),\n            \"contact-email\" => Self::Contact(ContactSearchField::Email),\n            \"contact-phone\" => Self::Contact(ContactSearchField::Phone),\n            \"contact-online-service\" => Self::Contact(ContactSearchField::OnlineService),\n            \"contact-address\" => Self::Contact(ContactSearchField::Address),\n            \"contact-note\" => Self::Contact(ContactSearchField::Note),\n            \"contact-uid\" => Self::Contact(ContactSearchField::Uid),\n\n            // File\n            \"file-name\" => Self::File(FileSearchField::Name),\n            \"file-content\" => Self::File(FileSearchField::Content),\n\n            // Tracing\n            \"trace-event-type\" => Self::Tracing(TracingSearchField::EventType),\n            \"trace-queue-id\" => Self::Tracing(TracingSearchField::QueueId),\n            \"trace-keywords\" => Self::Tracing(TracingSearchField::Keywords),\n\n            _ => return Err(format!(\"Unknown search field: {value}\")),\n        })\n    }\n}\n\nimpl Eq for SearchFilter {}\n\nimpl SearchIndex {\n    pub fn index_name(&self) -> &'static str {\n        match self {\n            SearchIndex::Email => \"st_email\",\n            SearchIndex::Calendar => \"st_calendar\",\n            SearchIndex::Contacts => \"st_contact\",\n            SearchIndex::File => \"st_file\",\n            SearchIndex::Tracing => \"st_tracing\",\n            SearchIndex::InMemory => unreachable!(),\n        }\n    }\n}\n\nimpl SearchField {\n    pub fn field_name(&self) -> &'static str {\n        match self {\n            SearchField::AccountId => \"acc_id\",\n            SearchField::DocumentId => \"doc_id\",\n            SearchField::Id => \"id\",\n            SearchField::Email(field) => match field {\n                EmailSearchField::From => \"from\",\n                EmailSearchField::To => \"to\",\n                EmailSearchField::Cc => \"cc\",\n                EmailSearchField::Bcc => \"bcc\",\n                EmailSearchField::Subject => \"subj\",\n                EmailSearchField::Body => \"body\",\n                EmailSearchField::Attachment => \"attach\",\n                EmailSearchField::ReceivedAt => \"rcvd\",\n                EmailSearchField::SentAt => \"sent\",\n                EmailSearchField::Size => \"size\",\n                EmailSearchField::HasAttachment => \"has_att\",\n                EmailSearchField::Headers => \"headers\",\n            },\n            SearchField::Calendar(field) => match field {\n                CalendarSearchField::Title => \"title\",\n                CalendarSearchField::Description => \"desc\",\n                CalendarSearchField::Location => \"loc\",\n                CalendarSearchField::Owner => \"owner\",\n                CalendarSearchField::Attendee => \"attendee\",\n                CalendarSearchField::Start => \"start\",\n                CalendarSearchField::Uid => \"uid\",\n            },\n            SearchField::Contact(field) => match field {\n                ContactSearchField::Member => \"member\",\n                ContactSearchField::Kind => \"kind\",\n                ContactSearchField::Name => \"name\",\n                ContactSearchField::Nickname => \"nick\",\n                ContactSearchField::Organization => \"org\",\n                ContactSearchField::Email => \"email\",\n                ContactSearchField::Phone => \"phone\",\n                ContactSearchField::OnlineService => \"online\",\n                ContactSearchField::Address => \"addr\",\n                ContactSearchField::Note => \"note\",\n                ContactSearchField::Uid => \"uid\",\n            },\n            SearchField::File(field) => match field {\n                FileSearchField::Name => \"name\",\n                FileSearchField::Content => \"content\",\n            },\n            SearchField::Tracing(field) => match field {\n                TracingSearchField::EventType => \"ev_type\",\n                TracingSearchField::QueueId => \"queue_id\",\n                TracingSearchField::Keywords => \"keywords\",\n            },\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    Store,\n    backend::MAX_TOKEN_LENGTH,\n    search::{\n        QueryResults, SearchComparator, SearchField, SearchFilter, SearchOperator, SearchQuery,\n        SearchValue,\n        bm_u32::{BitmapCache, range_to_bitmap, sort_order},\n        bm_u64::{TreemapCache, range_to_treemap},\n    },\n    write::SEARCH_INDEX_MAX_FIELD_LEN,\n};\nuse nlp::{language::stemmer::Stemmer, tokenizers::space::SpaceTokenizer};\nuse roaring::{RoaringBitmap, RoaringTreemap};\nuse std::ops::{BitAndAssign, BitOrAssign, BitXorAssign};\nuse utils::cheeky_hash::CheekyHash;\n\nimpl Store {\n    pub(crate) async fn query_account(&self, query: SearchQuery) -> trc::Result<Vec<u32>> {\n        struct State {\n            pub op: SearchFilter,\n            pub bm: Option<RoaringBitmap>,\n        }\n        let mut state: State = State {\n            op: SearchFilter::And,\n            bm: None,\n        };\n        let mut stack = Vec::new();\n        let mask = query.mask;\n        let mut bitmaps = BitmapCache::default();\n        let mut account_id = u32::MAX;\n\n        for filter in &query.filters {\n            if let SearchFilter::Operator {\n                field: SearchField::AccountId,\n                value: SearchValue::Uint(id),\n                ..\n            } = filter\n            {\n                account_id = *id as u32;\n                break;\n            }\n        }\n\n        if account_id == u32::MAX {\n            return Err(trc::StoreEvent::UnexpectedError\n                .into_err()\n                .details(\"Account ID must be specified before other filters\"));\n        }\n\n        let mut results;\n        if query.filters.len() > 1 {\n            let mut filters = query.filters.into_iter().peekable();\n            while let Some(filter) = filters.next() {\n                let mut result = match filter {\n                    SearchFilter::Operator { field, op, value } => {\n                        if matches!(field, SearchField::AccountId) {\n                            continue;\n                        }\n\n                        if field.is_text()\n                            && matches!(op, SearchOperator::Contains | SearchOperator::Equal)\n                        {\n                            let (value, language) = match value {\n                                SearchValue::Text { value, language } => (value, language),\n                                _ => {\n                                    return Err(trc::StoreEvent::UnexpectedError\n                                        .into_err()\n                                        .details(\"Expected text value for text field\"));\n                                }\n                            };\n\n                            if op == SearchOperator::Equal {\n                                bitmaps\n                                    .merge_bitmaps(\n                                        self,\n                                        query.index,\n                                        account_id,\n                                        language\n                                            .tokenize_text(&value, MAX_TOKEN_LENGTH)\n                                            .map(|token| CheekyHash::new(token.word.as_bytes())),\n                                        field.u8_id(),\n                                        false,\n                                    )\n                                    .await?\n                            } else {\n                                let mut result = RoaringBitmap::new();\n                                for token in Stemmer::new(&value, language, MAX_TOKEN_LENGTH) {\n                                    let mut tokens = Vec::with_capacity(3);\n                                    tokens.push(CheekyHash::new(token.word.as_bytes()));\n                                    tokens.push(CheekyHash::new(\n                                        format!(\"{}*\", token.word).as_bytes(),\n                                    ));\n                                    if let Some(stemmed_word) = token.stemmed_word {\n                                        tokens.push(CheekyHash::new(\n                                            format!(\"{stemmed_word}*\").as_bytes(),\n                                        ));\n                                    }\n                                    let union = bitmaps\n                                        .merge_bitmaps(\n                                            self,\n                                            query.index,\n                                            account_id,\n                                            tokens.into_iter(),\n                                            field.u8_id(),\n                                            true,\n                                        )\n                                        .await?;\n                                    if let Some(union) = union {\n                                        if result.is_empty() {\n                                            result = union;\n                                        } else {\n                                            result.bitand_assign(&union);\n                                            if result.is_empty() {\n                                                break;\n                                            }\n                                        }\n                                    } else {\n                                        result.clear();\n                                        break;\n                                    }\n                                }\n                                if !result.is_empty() {\n                                    Some(result)\n                                } else {\n                                    None\n                                }\n                            }\n                        } else if field.is_json() {\n                            let (key, value) = match value {\n                                SearchValue::KeyValues(kv) => kv.into_iter().next().unwrap(),\n                                _ => {\n                                    return Err(trc::StoreEvent::UnexpectedError\n                                        .into_err()\n                                        .details(\"Expected text value for text field\"));\n                                }\n                            };\n\n                            if !value.is_empty() {\n                                bitmaps\n                                    .merge_bitmaps(\n                                        self,\n                                        query.index,\n                                        account_id,\n                                        SpaceTokenizer::new(value.as_str(), MAX_TOKEN_LENGTH).map(\n                                            |value| {\n                                                CheekyHash::new(format!(\"{key} {value}\").as_bytes())\n                                            },\n                                        ),\n                                        field.u8_id(),\n                                        true,\n                                    )\n                                    .await?\n                            } else {\n                                bitmaps\n                                    .merge_bitmaps(\n                                        self,\n                                        query.index,\n                                        account_id,\n                                        [CheekyHash::new(key.as_bytes())].into_iter(),\n                                        field.u8_id(),\n                                        false,\n                                    )\n                                    .await?\n                            }\n                        } else if field.is_indexed() {\n                            let value = match value {\n                                SearchValue::Text { value, .. } => {\n                                    let mut value = value.into_bytes();\n                                    value.truncate(SEARCH_INDEX_MAX_FIELD_LEN);\n                                    value\n                                }\n                                SearchValue::Int(v) => (v as u64).to_be_bytes().to_vec(),\n                                SearchValue::Uint(v) => v.to_be_bytes().to_vec(),\n                                SearchValue::Boolean(v) => vec![v as u8],\n                                SearchValue::KeyValues(_) => {\n                                    return Err(trc::StoreEvent::UnexpectedError\n                                        .into_err()\n                                        .details(\"Expected non key-value for non-text field\"));\n                                }\n                            };\n\n                            range_to_bitmap(\n                                self,\n                                query.index,\n                                account_id,\n                                field.u8_id(),\n                                &value,\n                                op,\n                            )\n                            .await?\n                        } else {\n                            return Err(trc::StoreEvent::UnexpectedError\n                                .into_err()\n                                .details(format!(\"Field {field:?} is not indexed\")));\n                        }\n                    }\n                    SearchFilter::DocumentSet(bitmap) => Some(bitmap),\n                    op @ (SearchFilter::And | SearchFilter::Or | SearchFilter::Not) => {\n                        stack.push(state);\n                        state = State { op, bm: None };\n                        continue;\n                    }\n                    SearchFilter::End => {\n                        if let Some(prev_state) = stack.pop() {\n                            let bm = state.bm;\n                            state = prev_state;\n                            bm\n                        } else {\n                            break;\n                        }\n                    }\n                };\n\n                // Apply logical operation\n                if let Some(dest) = &mut state.bm {\n                    match state.op {\n                        SearchFilter::And => {\n                            if let Some(result) = result {\n                                dest.bitand_assign(result);\n                            } else {\n                                dest.clear();\n                            }\n                        }\n                        SearchFilter::Or => {\n                            if let Some(result) = result {\n                                dest.bitor_assign(result);\n                            }\n                        }\n                        SearchFilter::Not => {\n                            if let Some(mut result) = result {\n                                result.bitxor_assign(&mask);\n                                dest.bitand_assign(result);\n                            }\n                        }\n                        _ => unreachable!(),\n                    }\n                } else if let Some(result_) = &mut result {\n                    if let SearchFilter::Not = state.op {\n                        result_.bitxor_assign(&mask);\n                    }\n                    state.bm = result;\n                } else if let SearchFilter::Not = state.op {\n                    state.bm = Some(mask.clone());\n                } else {\n                    state.bm = Some(RoaringBitmap::new());\n                }\n\n                // And short circuit\n                if matches!(state.op, SearchFilter::And) && state.bm.as_ref().unwrap().is_empty() {\n                    while let Some(filter) = filters.peek() {\n                        if matches!(filter, SearchFilter::End) {\n                            break;\n                        } else {\n                            filters.next();\n                        }\n                    }\n                }\n            }\n\n            results = state.bm.unwrap_or_default();\n            results.bitand_assign(&mask);\n        } else {\n            results = mask;\n        }\n\n        if results.len() > 1 && !query.comparators.is_empty() {\n            let mut comparators = Vec::with_capacity(query.comparators.len());\n            for comparator in query.comparators {\n                let comparator = match comparator {\n                    SearchComparator::Field { field, ascending } => SearchComparator::SortedSet {\n                        set: sort_order(self, query.index, account_id, field.u8_id()).await?,\n                        ascending,\n                    },\n                    _ => comparator,\n                };\n\n                comparators.push(comparator);\n            }\n\n            Ok(QueryResults::new(results, comparators).into_sorted())\n        } else {\n            Ok(results.into_iter().collect::<Vec<_>>())\n        }\n    }\n\n    pub(crate) async fn query_global(&self, query: SearchQuery) -> trc::Result<Vec<u64>> {\n        struct State {\n            pub op: SearchFilter,\n            pub bm: Option<RoaringTreemap>,\n        }\n        let mut state: State = State {\n            op: SearchFilter::And,\n            bm: None,\n        };\n        let mut stack = Vec::new();\n        let mut filters = query.filters.into_iter().peekable();\n        let mut bitmaps = TreemapCache::default();\n\n        while let Some(filter) = filters.next() {\n            let result = match filter {\n                SearchFilter::Operator { field, op, value } => {\n                    if field.is_text() {\n                        let value = match value {\n                            SearchValue::Text { value, .. } => value,\n                            _ => {\n                                return Err(trc::StoreEvent::UnexpectedError\n                                    .into_err()\n                                    .details(\"Expected text value for text field\"));\n                            }\n                        };\n\n                        bitmaps\n                            .merge_treemaps(\n                                self,\n                                query.index,\n                                SpaceTokenizer::new(value.as_str(), MAX_TOKEN_LENGTH)\n                                    .map(|word| CheekyHash::new(word.as_bytes())),\n                                field.u8_id(),\n                                false,\n                            )\n                            .await?\n                    } else if field.is_indexed() || matches!(field, SearchField::Id) {\n                        let value = match value {\n                            SearchValue::Text { value, .. } => value.into_bytes(),\n                            SearchValue::Int(v) => (v as u64).to_be_bytes().to_vec(),\n                            SearchValue::Uint(v) => v.to_be_bytes().to_vec(),\n                            SearchValue::Boolean(v) => vec![v as u8],\n                            SearchValue::KeyValues(_) => {\n                                return Err(trc::StoreEvent::UnexpectedError\n                                    .into_err()\n                                    .details(\"Expected non key-value for non-text field\"));\n                            }\n                        };\n\n                        range_to_treemap(self, query.index, field.u8_id(), &value, op).await?\n                    } else {\n                        return Err(trc::StoreEvent::UnexpectedError\n                            .into_err()\n                            .details(format!(\"Field {field:?} is not indexed\")));\n                    }\n                }\n                SearchFilter::DocumentSet(_) | SearchFilter::Not => {\n                    return Err(trc::StoreEvent::UnexpectedError\n                        .into_err()\n                        .details(\"Unsupported filter in global search\"));\n                }\n                op @ (SearchFilter::And | SearchFilter::Or) => {\n                    stack.push(state);\n                    state = State { op, bm: None };\n                    continue;\n                }\n                SearchFilter::End => {\n                    if let Some(prev_state) = stack.pop() {\n                        let bm = state.bm;\n                        state = prev_state;\n                        bm\n                    } else {\n                        break;\n                    }\n                }\n            };\n\n            // Apply logical operation\n            if let Some(dest) = &mut state.bm {\n                match state.op {\n                    SearchFilter::And => {\n                        if let Some(result) = result {\n                            dest.bitand_assign(result);\n                        } else {\n                            dest.clear();\n                        }\n                    }\n                    SearchFilter::Or => {\n                        if let Some(result) = result {\n                            dest.bitor_assign(result);\n                        }\n                    }\n                    _ => unreachable!(),\n                }\n            } else if result.is_some() {\n                state.bm = result;\n            } else {\n                state.bm = Some(RoaringTreemap::new());\n            }\n\n            // And short circuit\n            if matches!(state.op, SearchFilter::And) && state.bm.as_ref().unwrap().is_empty() {\n                while let Some(filter) = filters.peek() {\n                    if matches!(filter, SearchFilter::End) {\n                        break;\n                    } else {\n                        filters.next();\n                    }\n                }\n            }\n        }\n\n        if query.comparators.iter().all(|c| {\n            matches!(\n                c,\n                SearchComparator::Field {\n                    field: SearchField::Id,\n                    ascending: false\n                }\n            )\n        }) {\n            Ok(state\n                .bm\n                .unwrap_or_default()\n                .into_iter()\n                .rev()\n                .collect::<Vec<_>>())\n        } else {\n            Ok(state.bm.unwrap_or_default().into_iter().collect::<Vec<_>>())\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/split.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::search::*;\n\n#[derive(Debug, PartialEq, Eq)]\npub(crate) enum SplitFilter {\n    Internal(SearchFilter),\n    External(Vec<SearchFilter>),\n}\n\npub(crate) fn split_filters(filters_in: Vec<SearchFilter>) -> Option<Vec<SplitFilter>> {\n    let mut account_id = u64::MAX;\n    let mut filters: Vec<SearchFilter> = Vec::with_capacity(filters_in.len());\n    let mut op_stack = Vec::new();\n    let mut document_sets: AHashMap<usize, RoaringBitmap> = AHashMap::new();\n    let mut operators: AHashMap<usize, Vec<SearchFilter>> = AHashMap::new();\n\n    for filter in filters_in {\n        match filter {\n            op @ (SearchFilter::And | SearchFilter::Or | SearchFilter::Not) => {\n                op_stack.push(op.clone());\n                filters.push(op);\n            }\n            SearchFilter::End => {\n                if let Some(ops) = operators.remove(&op_stack.len()) {\n                    filters.extend(ops);\n                }\n                if let Some(docs) = document_sets.remove(&op_stack.len()) {\n                    filters.push(SearchFilter::DocumentSet(docs));\n                }\n                filters.push(SearchFilter::End);\n                op_stack.pop()?;\n            }\n            SearchFilter::Operator {\n                field: SearchField::AccountId,\n                value: SearchValue::Uint(id),\n                ..\n            } => {\n                account_id = id;\n            }\n            SearchFilter::Operator { .. } => {\n                operators.entry(op_stack.len()).or_default().push(filter);\n            }\n            SearchFilter::DocumentSet(docs) => match document_sets.entry(op_stack.len()) {\n                Entry::Occupied(mut entry) => {\n                    if matches!(op_stack.last(), Some(SearchFilter::Or)) {\n                        entry.get_mut().bitor_assign(&docs);\n                    } else {\n                        entry.get_mut().bitand_assign(&docs);\n                    }\n                }\n                Entry::Vacant(entry) => {\n                    entry.insert(docs);\n                }\n            },\n        }\n    }\n\n    if let Some(ops) = operators.remove(&0) {\n        filters.extend(ops);\n    }\n    if let Some(docs) = document_sets.remove(&0) {\n        filters.push(SearchFilter::DocumentSet(docs));\n    }\n\n    if account_id == u64::MAX {\n        return None;\n    }\n\n    let mut split: Vec<SplitFilter> = Vec::new();\n    let mut i = 0;\n\n    'outer: while i < filters.len() {\n        let mut j = i;\n        let mut depth = 0;\n\n        while j < filters.len() {\n            match &filters[j] {\n                SearchFilter::And | SearchFilter::Or | SearchFilter::Not => {\n                    depth += 1;\n                }\n                SearchFilter::End => {\n                    depth -= 1;\n                    if depth < 0 {\n                        if j > i {\n                            break;\n                        } else {\n                            split.push(SplitFilter::Internal(SearchFilter::End));\n                            i += 1;\n                            continue 'outer;\n                        }\n                    }\n                }\n                SearchFilter::Operator { .. } => {}\n                SearchFilter::DocumentSet(_) => {\n                    if depth == 0 && j > i {\n                        break;\n                    } else {\n                        split.push(SplitFilter::Internal(std::mem::take(&mut filters[i])));\n                        i += 1;\n                        continue 'outer;\n                    }\n                }\n            }\n            j += 1;\n        }\n\n        let mut external_filters = vec![SearchFilter::Operator {\n            field: SearchField::AccountId,\n            op: SearchOperator::Equal,\n            value: SearchValue::Uint(account_id),\n        }];\n        let add_or =\n            matches!(split.last(), Some(SplitFilter::Internal(SearchFilter::Or))) && j > i + 1;\n        if add_or {\n            external_filters.push(SearchFilter::Or);\n        }\n        external_filters.extend(&mut filters[i..j].iter_mut().map(std::mem::take));\n        if add_or {\n            external_filters.push(SearchFilter::End);\n        }\n        split.push(SplitFilter::External(external_filters));\n\n        i = j;\n    }\n\n    Some(split)\n}\n\n// Test cases\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn test_split_filters_exhaustive() {\n        let test_cases: Vec<(&str, Vec<SearchFilter>, Vec<SplitFilter>)> = vec![\n            // Test 1: Operator followed by document set at depth 0\n            (\n                \"Operator then document set at depth 0\",\n                vec![account_id(42), other_op(\"test\"), doc_set(&[1, 2, 3])],\n                vec![\n                    SplitFilter::External(vec![account_id(42), other_op(\"test\")]),\n                    SplitFilter::Internal(doc_set(&[1, 2, 3])),\n                ],\n            ),\n            // Test 2: Document set followed by operator at depth 0\n            (\n                \"Document set then operator at depth 0\",\n                vec![account_id(42), doc_set(&[1, 2, 3]), other_op(\"test\")],\n                vec![\n                    SplitFilter::External(vec![account_id(42), other_op(\"test\")]),\n                    SplitFilter::Internal(doc_set(&[1, 2, 3])),\n                ],\n            ),\n            // Test 3: Multiple document sets with operator in between\n            (\n                \"Multiple document sets at depth 0 with operator\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1, 2]),\n                    other_op(\"middle\"),\n                    doc_set(&[2, 4]),\n                ],\n                vec![\n                    SplitFilter::External(vec![account_id(42), other_op(\"middle\")]),\n                    SplitFilter::Internal(doc_set(&[2])),\n                ],\n            ),\n            // Test 4: Document set at depth 0, then AND group\n            (\n                \"Document set then AND group\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1, 2]),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::And,\n                        other_op(\"a\"),\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                ],\n            ),\n            // Test 5: AND group followed by document set at depth 0\n            (\n                \"AND group then document set\",\n                vec![\n                    account_id(42),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    doc_set(&[1, 2]),\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::And,\n                        other_op(\"a\"),\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                ],\n            ),\n            // Test 6: Operator at depth 0, then OR group, then document set\n            (\n                \"Operator, OR group, then document set\",\n                vec![\n                    account_id(42),\n                    other_op(\"pre\"),\n                    SearchFilter::Or,\n                    other_op(\"a\"),\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    doc_set(&[1, 2, 3]),\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::Or,\n                        other_op(\"a\"),\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                        other_op(\"pre\"),\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2, 3])),\n                ],\n            ),\n            // Test 7: Document set, OR group, operator\n            (\n                \"Document set, OR group, operator\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1, 2]),\n                    SearchFilter::Or,\n                    other_op(\"a\"),\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    other_op(\"post\"),\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::Or,\n                        other_op(\"a\"),\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                        other_op(\"post\"),\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                ],\n            ),\n            // Test 8: Multiple OR branches with document sets between\n            (\n                \"Multiple OR branches with document sets between\",\n                vec![\n                    account_id(42),\n                    SearchFilter::Or,\n                    other_op(\"a\"),\n                    SearchFilter::End,\n                    doc_set(&[1, 2]),\n                    SearchFilter::Or,\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    doc_set(&[1, 2, 5, 6]),\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::Or,\n                        other_op(\"a\"),\n                        SearchFilter::End,\n                        SearchFilter::Or,\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                ],\n            ),\n            // Test 9: Document sets at different depths - depth 0 and inside AND\n            (\n                \"Document sets at different depths in AND\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1, 2]),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    doc_set(&[2, 3]),\n                    SearchFilter::End,\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::And),\n                    SplitFilter::External(vec![account_id(42), other_op(\"a\")]),\n                    SplitFilter::Internal(doc_set(&[2, 3])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                ],\n            ),\n            // Test 10: Operator, AND group with doc set inside, operator\n            (\n                \"Operator, AND(operator, doc_set), operator\",\n                vec![\n                    account_id(42),\n                    other_op(\"pre\"),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    doc_set(&[1, 2, 3]),\n                    SearchFilter::End,\n                    other_op(\"post\"),\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::And),\n                    SplitFilter::External(vec![account_id(42), other_op(\"a\")]),\n                    SplitFilter::Internal(doc_set(&[1, 2, 3])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::External(vec![account_id(42), other_op(\"pre\"), other_op(\"post\")]),\n                ],\n            ),\n            // Test 11: Document set, nested groups, document set\n            (\n                \"Doc set, AND(OR(a,b)), doc set\",\n                vec![\n                    account_id(42),\n                    SearchFilter::Or,\n                    doc_set(&[1, 2]),\n                    other_op(\"c\"),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    doc_set(&[3, 4]),\n                    SearchFilter::End,\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::Or),\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::Or,\n                        SearchFilter::And,\n                        other_op(\"a\"),\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                        other_op(\"c\"),\n                        SearchFilter::End,\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2, 3, 4])),\n                    SplitFilter::Internal(SearchFilter::End),\n                ],\n            ),\n            // Test 12: OR with nested AND containing document sets, followed by operator\n            (\n                \"OR(AND(doc_set, doc_set), operator) followed by operator\",\n                vec![\n                    account_id(42),\n                    SearchFilter::Or,\n                    SearchFilter::And,\n                    doc_set(&[1, 2]),\n                    doc_set(&[2, 3]),\n                    SearchFilter::End,\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    other_op(\"post\"),\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::Or),\n                    SplitFilter::Internal(SearchFilter::And),\n                    SplitFilter::Internal(doc_set(&[2])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::External(vec![account_id(42), other_op(\"b\")]),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::External(vec![account_id(42), other_op(\"post\")]),\n                ],\n            ),\n            // Test 13: Complex: doc set, AND group, doc set, OR group, doc set\n            (\n                \"Complex: doc, AND, doc, OR, doc\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1, 2, 3]),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    SearchFilter::End,\n                    doc_set(&[1, 2, 3, 5]),\n                    SearchFilter::Or,\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    doc_set(&[1, 2, 3, 6]),\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::And,\n                        other_op(\"a\"),\n                        SearchFilter::End,\n                        SearchFilter::Or,\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2, 3])),\n                ],\n            ),\n            // Test 14: Operator, NOT group, document set\n            (\n                \"Operator, NOT(operator), document set\",\n                vec![\n                    account_id(42),\n                    other_op(\"pre\"),\n                    SearchFilter::Not,\n                    other_op(\"a\"),\n                    SearchFilter::End,\n                    doc_set(&[1, 2]),\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::Not,\n                        other_op(\"a\"),\n                        SearchFilter::End,\n                        other_op(\"pre\"),\n                    ]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                ],\n            ),\n            // Test 15: Document set, NOT group, operator\n            (\n                \"Document set, NOT(operator), operator\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1, 2]),\n                    SearchFilter::Not,\n                    other_op(\"a\"),\n                    doc_set(&[3, 4]),\n                    SearchFilter::End,\n                    other_op(\"post\"),\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::Not),\n                    SplitFilter::External(vec![account_id(42), other_op(\"a\")]),\n                    SplitFilter::Internal(doc_set(&[3, 4])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::External(vec![account_id(42), other_op(\"post\")]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                ],\n            ),\n            // Test 16: Alternating doc sets and operators\n            (\n                \"Alternating: doc, op, doc, op, doc\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1]),\n                    other_op(\"a\"),\n                    doc_set(&[1, 2]),\n                    other_op(\"b\"),\n                    doc_set(&[1, 3]),\n                ],\n                vec![\n                    SplitFilter::External(vec![account_id(42), other_op(\"a\"), other_op(\"b\")]),\n                    SplitFilter::Internal(doc_set(&[1])),\n                ],\n            ),\n            // Test 17: Multiple operators, then OR group with doc set inside, then doc set\n            (\n                \"Multiple ops, OR(op, doc_set), doc\",\n                vec![\n                    account_id(42),\n                    other_op(\"a\"),\n                    SearchFilter::Or,\n                    other_op(\"c\"),\n                    doc_set(&[1, 2]),\n                    SearchFilter::End,\n                    other_op(\"b\"),\n                    doc_set(&[3, 4]),\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::Or),\n                    SplitFilter::External(vec![account_id(42), other_op(\"c\")]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::External(vec![account_id(42), other_op(\"a\"), other_op(\"b\")]),\n                    SplitFilter::Internal(doc_set(&[3, 4])),\n                ],\n            ),\n            // Test 18: Doc set before and after nested OR(AND(op))\n            (\n                \"Doc, OR(AND(op)), doc\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1]),\n                    SearchFilter::Or,\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    other_op(\"c\"),\n                    SearchFilter::End,\n                    other_op(\"b\"),\n                    SearchFilter::End,\n                    doc_set(&[2]),\n                ],\n                vec![\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::Or,\n                        SearchFilter::And,\n                        other_op(\"a\"),\n                        other_op(\"c\"),\n                        SearchFilter::End,\n                        other_op(\"b\"),\n                        SearchFilter::End,\n                    ]),\n                    SplitFilter::Internal(doc_set(&[])),\n                ],\n            ),\n            // Test 19: AND group with doc set, operator between, OR group with doc set\n            (\n                \"AND(op, doc), op, OR(op, doc)\",\n                vec![\n                    account_id(42),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    doc_set(&[1, 2]),\n                    SearchFilter::End,\n                    other_op(\"middle\"),\n                    SearchFilter::Or,\n                    other_op(\"b\"),\n                    other_op(\"c\"),\n                    doc_set(&[3, 4]),\n                    SearchFilter::End,\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::And),\n                    SplitFilter::External(vec![account_id(42), other_op(\"a\")]),\n                    SplitFilter::Internal(doc_set(&[1, 2])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::Internal(SearchFilter::Or),\n                    SplitFilter::External(vec![\n                        account_id(42),\n                        SearchFilter::Or,\n                        other_op(\"b\"),\n                        other_op(\"c\"),\n                        SearchFilter::End,\n                    ]),\n                    SplitFilter::Internal(doc_set(&[3, 4])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::External(vec![account_id(42), other_op(\"middle\")]),\n                ],\n            ),\n            // Test 20: Deep nesting with document sets at multiple levels\n            (\n                \"Deep nesting: doc, AND(doc, OR(doc, AND(op, doc)))\",\n                vec![\n                    account_id(42),\n                    doc_set(&[1]),\n                    SearchFilter::And,\n                    doc_set(&[2]),\n                    SearchFilter::Or,\n                    doc_set(&[3]),\n                    SearchFilter::And,\n                    other_op(\"a\"),\n                    doc_set(&[4]),\n                    SearchFilter::End,\n                    SearchFilter::End,\n                    SearchFilter::End,\n                ],\n                vec![\n                    SplitFilter::Internal(SearchFilter::And),\n                    SplitFilter::Internal(SearchFilter::Or),\n                    SplitFilter::Internal(SearchFilter::And),\n                    SplitFilter::External(vec![account_id(42), other_op(\"a\")]),\n                    SplitFilter::Internal(doc_set(&[4])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::Internal(doc_set(&[3])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::Internal(doc_set(&[2])),\n                    SplitFilter::Internal(SearchFilter::End),\n                    SplitFilter::Internal(doc_set(&[1])),\n                ],\n            ),\n        ];\n\n        for (description, input, expected) in test_cases {\n            println!(\"------ Running test: {} ------\", description);\n            let result = split_filters(input.clone());\n            assert!(result.is_some(), \"Test '{}' returned None\", description);\n\n            let result = result.unwrap();\n            if result != expected {\n                print_split_filter_code(&result);\n            }\n            assert_eq!(result, expected, \"Test '{description}' failed\",);\n        }\n    }\n\n    fn account_id(id: u64) -> SearchFilter {\n        SearchFilter::Operator {\n            field: SearchField::AccountId,\n            op: SearchOperator::Equal,\n            value: SearchValue::Uint(id),\n        }\n    }\n\n    fn other_op(value: &str) -> SearchFilter {\n        SearchFilter::Operator {\n            field: SearchField::DocumentId,\n            op: SearchOperator::Equal,\n            value: SearchValue::Text {\n                value: value.to_string(),\n                language: Language::None,\n            },\n        }\n    }\n\n    fn doc_set(ids: &[u32]) -> SearchFilter {\n        let mut bitmap = RoaringBitmap::new();\n        for id in ids {\n            bitmap.insert(*id);\n        }\n        SearchFilter::DocumentSet(bitmap)\n    }\n\n    fn print_split_filter_code(splits: &[SplitFilter]) {\n        println!(\"vec![\");\n        for split in splits {\n            match split {\n                SplitFilter::Internal(filter) => {\n                    print!(\"    SplitFilter::Internal(\");\n                    print_search_filter_code(filter, 0);\n                    println!(\"),\");\n                }\n                SplitFilter::External(filters) => {\n                    println!(\"    SplitFilter::External(vec![\");\n                    for filter in filters {\n                        print!(\"        \");\n                        print_search_filter_code(filter, 2);\n                        println!(\",\");\n                    }\n                    println!(\"    ]),\");\n                }\n            }\n        }\n        println!(\"]\");\n    }\n\n    fn print_search_filter_code(filter: &SearchFilter, indent_level: usize) {\n        let indent = \"    \".repeat(indent_level);\n        match filter {\n            SearchFilter::Operator { field, op, value } => match (field, op, value) {\n                (SearchField::AccountId, SearchOperator::Equal, SearchValue::Uint(id)) => {\n                    print!(\"account_id({})\", id);\n                }\n                (\n                    SearchField::DocumentId,\n                    SearchOperator::Equal,\n                    SearchValue::Text { value, .. },\n                ) => {\n                    print!(\"other_op(\\\"{}\\\")\", value);\n                }\n                _ => {\n                    println!(\"SearchFilter::Operator {{\");\n                    println!(\"{}    field: {:?},\", indent, field);\n                    println!(\"{}    op: {:?},\", indent, op);\n                    println!(\"{}    value: {:?},\", indent, value);\n                    print!(\"{}}}\", indent);\n                }\n            },\n            SearchFilter::DocumentSet(bitmap) => {\n                let ids: Vec<u32> = bitmap.iter().collect();\n                if ids.is_empty() {\n                    print!(\"doc_set(&[])\");\n                } else if ids.len() <= 5 {\n                    print!(\"doc_set(&[\");\n                    for (i, id) in ids.iter().enumerate() {\n                        if i > 0 {\n                            print!(\", \");\n                        }\n                        print!(\"{}\", id);\n                    }\n                    print!(\"])\");\n                } else {\n                    // For large bitmaps, create inline\n                    println!(\"{{\");\n                    println!(\"{}    let mut bitmap = RoaringBitmap::new();\", indent);\n                    for id in ids {\n                        println!(\"{}    bitmap.insert({});\", indent, id);\n                    }\n                    print!(\"{}    doc_set_bitmap(bitmap)\", indent);\n                    println!();\n                    print!(\"{}}}\", indent);\n                }\n            }\n            SearchFilter::And => print!(\"SearchFilter::And\"),\n            SearchFilter::Or => print!(\"SearchFilter::Or\"),\n            SearchFilter::Not => print!(\"SearchFilter::Not\"),\n            SearchFilter::End => print!(\"SearchFilter::End\"),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/search/term.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    Serialize,\n    backend::MAX_TOKEN_LENGTH,\n    search::*,\n    write::{\n        Archiver, BatchBuilder, SEARCH_INDEX_MAX_FIELD_LEN, SearchIndexClass, SearchIndexField,\n        SearchIndexId, SearchIndexType, ValueClass,\n    },\n};\nuse ahash::AHashSet;\nuse nlp::{\n    language::stemmer::Stemmer,\n    tokenizers::{space::SpaceTokenizer, word::WordTokenizer},\n};\nuse utils::{\n    cheeky_hash::{CheekyBTreeMap, CheekyHash},\n    map::bitmap::BitPop,\n};\n\n#[derive(Debug, PartialEq, Eq, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\npub(crate) struct TermIndex {\n    terms: Vec<Term>,\n    fields: Vec<SearchIndexField>,\n}\n\n#[derive(Debug, PartialEq, Eq, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\npub(crate) struct Term {\n    hash: CheekyHash,\n    fields: u32,\n}\n\npub(crate) struct TermIndexBuilder {\n    pub(crate) index: TermIndex,\n    pub(crate) id: SearchIndexId,\n}\n\nimpl TermIndexBuilder {\n    pub fn build(document: IndexDocument, truncate_at: usize) -> Self {\n        let mut terms: CheekyBTreeMap<u32> = CheekyBTreeMap::new();\n        let mut fields: Vec<SearchIndexField> = Vec::new();\n        let mut account_id = None;\n        let mut document_id = None;\n        let mut id = None;\n\n        for (field, value) in document.fields {\n            match field {\n                SearchField::Id => {\n                    if let SearchValue::Uint(v) = value {\n                        fields.push(SearchIndexField {\n                            field_id: field.u8_id(),\n                            data: v.to_be_bytes().to_vec(),\n                        });\n                        id = Some(v);\n                    }\n\n                    continue;\n                }\n                SearchField::AccountId => {\n                    if let SearchValue::Uint(v) = value {\n                        account_id = Some(v);\n                    }\n                    continue;\n                }\n                SearchField::DocumentId => {\n                    if let SearchValue::Uint(v) = value {\n                        document_id = Some(v);\n                    }\n                    continue;\n                }\n                _ => {}\n            }\n\n            let field = match value {\n                SearchValue::Text { value, language } => {\n                    if field.is_text() {\n                        let value = if truncate_at > 0 && value.len() > truncate_at {\n                            let pos = value.floor_char_boundary(truncate_at);\n                            &value[..pos]\n                        } else {\n                            &value\n                        };\n\n                        match language {\n                            Language::Unknown => {\n                                for token in WordTokenizer::new(value, MAX_TOKEN_LENGTH) {\n                                    terms\n                                        .entry(CheekyHash::new(token.word.as_bytes()))\n                                        .or_default()\n                                        .bit_push(field.u8_id());\n                                }\n                            }\n                            Language::None => {\n                                for token in SpaceTokenizer::new(value, MAX_TOKEN_LENGTH) {\n                                    terms\n                                        .entry(CheekyHash::new(token.as_bytes()))\n                                        .or_default()\n                                        .bit_push(field.u8_id());\n                                }\n                            }\n                            _ => {\n                                for token in Stemmer::new(value, language, MAX_TOKEN_LENGTH) {\n                                    terms\n                                        .entry(CheekyHash::new(token.word.as_bytes()))\n                                        .or_default()\n                                        .bit_push(field.u8_id());\n\n                                    if let Some(stemmed_word) = token.stemmed_word {\n                                        terms\n                                            .entry(CheekyHash::new(\n                                                format!(\"{}*\", stemmed_word).as_bytes(),\n                                            ))\n                                            .or_default()\n                                            .bit_push(field.u8_id());\n                                    }\n                                }\n                            }\n                        }\n                    }\n\n                    if field.is_indexed() {\n                        let mut data = value.into_bytes();\n                        data.truncate(SEARCH_INDEX_MAX_FIELD_LEN);\n\n                        SearchIndexField {\n                            field_id: field.u8_id(),\n                            data,\n                        }\n                    } else {\n                        continue;\n                    }\n                }\n                SearchValue::KeyValues(map) => {\n                    for (key, value) in map {\n                        terms\n                            .entry(CheekyHash::new(key.as_bytes()))\n                            .or_default()\n                            .bit_push(field.u8_id());\n                        for token in SpaceTokenizer::new(value.as_str(), MAX_TOKEN_LENGTH) {\n                            terms\n                                .entry(CheekyHash::new(format!(\"{key} {token}\").as_bytes()))\n                                .or_default()\n                                .bit_push(field.u8_id());\n                        }\n                    }\n\n                    continue;\n                }\n                SearchValue::Int(v) => SearchIndexField {\n                    field_id: field.u8_id(),\n                    data: (v as u64).to_be_bytes().to_vec(),\n                },\n                SearchValue::Uint(v) => SearchIndexField {\n                    field_id: field.u8_id(),\n                    data: v.to_be_bytes().to_vec(),\n                },\n                SearchValue::Boolean(v) => SearchIndexField {\n                    field_id: field.u8_id(),\n                    data: vec![v as u8],\n                },\n            };\n\n            fields.push(field);\n        }\n\n        TermIndexBuilder {\n            index: TermIndex {\n                terms: terms\n                    .into_iter()\n                    .map(|(k, v)| Term { hash: k, fields: v })\n                    .collect(),\n                fields,\n            },\n            id: match (account_id, document_id, id) {\n                (Some(account_id), Some(document_id), None) => SearchIndexId::Account {\n                    account_id: account_id as u32,\n                    document_id: document_id as u32,\n                },\n                (None, None, Some(id)) => SearchIndexId::Global { id },\n                _ => {\n                    debug_assert!(\n                        false,\n                        \"Invalid combination of AccountId {account_id:?}, DocumentId {document_id:?} and Id {id:?} fields\"\n                    );\n                    SearchIndexId::Global { id: 0 }\n                }\n            },\n        }\n    }\n}\n\nimpl TermIndex {\n    pub fn write_index(\n        self,\n        batch: &mut BatchBuilder,\n        index: SearchIndex,\n        id: SearchIndexId,\n    ) -> trc::Result<()> {\n        let archive = Archiver::new(self);\n        batch\n            .set(\n                ValueClass::SearchIndex(SearchIndexClass {\n                    index,\n                    id,\n                    typ: SearchIndexType::Document,\n                }),\n                archive.serialize()?,\n            )\n            .commit_point();\n\n        for term in archive.inner.terms {\n            let mut fields = term.fields;\n            while let Some(field) = fields.bit_pop() {\n                batch\n                    .set(\n                        ValueClass::SearchIndex(SearchIndexClass {\n                            index,\n                            id,\n                            typ: SearchIndexType::Term {\n                                hash: term.hash,\n                                field,\n                            },\n                        }),\n                        vec![],\n                    )\n                    .commit_point();\n            }\n        }\n\n        for field in archive.inner.fields {\n            batch\n                .set(\n                    ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id,\n                        typ: SearchIndexType::Index { field },\n                    }),\n                    vec![],\n                )\n                .commit_point();\n        }\n\n        Ok(())\n    }\n\n    pub fn merge_index(\n        self,\n        batch: &mut BatchBuilder,\n        index: SearchIndex,\n        id: SearchIndexId,\n        old_term: &ArchivedTermIndex,\n    ) -> trc::Result<()> {\n        let archive = Archiver::new(self);\n        batch\n            .set(\n                ValueClass::SearchIndex(SearchIndexClass {\n                    index,\n                    id,\n                    typ: SearchIndexType::Document,\n                }),\n                archive.serialize()?,\n            )\n            .commit_point();\n\n        let mut old_terms = AHashSet::with_capacity(old_term.terms.len());\n        let mut old_fields = AHashSet::with_capacity(old_term.fields.len());\n        for term in old_term.terms.iter() {\n            let mut fields = term.fields.to_native();\n            while let Some(field) = fields.bit_pop() {\n                old_terms.insert(SearchIndexType::Term {\n                    hash: term.hash.to_native(),\n                    field,\n                });\n            }\n        }\n        for field in old_term.fields.iter() {\n            old_fields.insert(SearchIndexField {\n                field_id: field.field_id,\n                data: field.data.to_vec(),\n            });\n        }\n\n        for term in archive.inner.terms {\n            let mut fields = term.fields;\n            while let Some(field) = fields.bit_pop() {\n                let typ = SearchIndexType::Term {\n                    hash: term.hash,\n                    field,\n                };\n\n                if !old_terms.remove(&typ) {\n                    batch\n                        .set(\n                            ValueClass::SearchIndex(SearchIndexClass { index, id, typ }),\n                            vec![],\n                        )\n                        .commit_point();\n                }\n            }\n        }\n\n        for field in archive.inner.fields {\n            if !old_fields.remove(&field) {\n                batch\n                    .set(\n                        ValueClass::SearchIndex(SearchIndexClass {\n                            index,\n                            id,\n                            typ: SearchIndexType::Index { field },\n                        }),\n                        vec![],\n                    )\n                    .commit_point();\n            }\n        }\n\n        for typ in old_terms {\n            batch\n                .clear(ValueClass::SearchIndex(SearchIndexClass { index, id, typ }))\n                .commit_point();\n        }\n\n        for field in old_fields {\n            batch\n                .clear(ValueClass::SearchIndex(SearchIndexClass {\n                    index,\n                    id,\n                    typ: SearchIndexType::Index { field },\n                }))\n                .commit_point();\n        }\n\n        Ok(())\n    }\n}\n\nimpl ArchivedTermIndex {\n    pub fn delete_index(&self, batch: &mut BatchBuilder, index: SearchIndex, id: SearchIndexId) {\n        batch\n            .clear(ValueClass::SearchIndex(SearchIndexClass {\n                index,\n                id,\n                typ: SearchIndexType::Document,\n            }))\n            .commit_point();\n\n        for term in self.terms.iter() {\n            let mut fields = term.fields.to_native();\n            while let Some(field) = fields.bit_pop() {\n                batch\n                    .clear(ValueClass::SearchIndex(SearchIndexClass {\n                        index,\n                        id,\n                        typ: SearchIndexType::Term {\n                            hash: term.hash.to_native(),\n                            field,\n                        },\n                    }))\n                    .commit_point();\n            }\n        }\n\n        for field in self.fields.iter() {\n            batch\n                .clear(ValueClass::SearchIndex(SearchIndexClass {\n                    index,\n                    id,\n                    typ: SearchIndexType::Index {\n                        field: SearchIndexField {\n                            field_id: field.field_id,\n                            data: field.data.to_vec(),\n                        },\n                    },\n                }))\n                .commit_point();\n        }\n    }\n}\n\nimpl SearchIndex {\n    pub(crate) fn as_u8(&self) -> u8 {\n        match self {\n            SearchIndex::Email => 0,\n            SearchIndex::Calendar => 1,\n            SearchIndex::Contacts => 2,\n            SearchIndex::File => 3,\n            SearchIndex::Tracing => 4,\n            SearchIndex::InMemory => unreachable!(),\n        }\n    }\n}\n\nimpl SearchField {\n    pub(crate) fn u8_id(&self) -> u8 {\n        match self {\n            SearchField::AccountId => 0,\n            SearchField::DocumentId => 1,\n            SearchField::Id => 2,\n            SearchField::Email(field) => match field {\n                EmailSearchField::From => 3,\n                EmailSearchField::To => 4,\n                EmailSearchField::Cc => 5,\n                EmailSearchField::Bcc => 6,\n                EmailSearchField::Subject => 7,\n                EmailSearchField::Body => 8,\n                EmailSearchField::Attachment => 9,\n                EmailSearchField::ReceivedAt => 10,\n                EmailSearchField::SentAt => 11,\n                EmailSearchField::Size => 12,\n                EmailSearchField::HasAttachment => 13,\n                EmailSearchField::Headers => 14,\n            },\n            SearchField::Calendar(field) => match field {\n                CalendarSearchField::Title => 3,\n                CalendarSearchField::Description => 4,\n                CalendarSearchField::Location => 5,\n                CalendarSearchField::Owner => 6,\n                CalendarSearchField::Attendee => 7,\n                CalendarSearchField::Start => 8,\n                CalendarSearchField::Uid => 9,\n            },\n            SearchField::Contact(field) => match field {\n                ContactSearchField::Member => 3,\n                ContactSearchField::Kind => 4,\n                ContactSearchField::Name => 5,\n                ContactSearchField::Nickname => 6,\n                ContactSearchField::Organization => 7,\n                ContactSearchField::Email => 8,\n                ContactSearchField::Phone => 9,\n                ContactSearchField::OnlineService => 10,\n                ContactSearchField::Address => 11,\n                ContactSearchField::Note => 12,\n                ContactSearchField::Uid => 13,\n            },\n            SearchField::File(field) => match field {\n                FileSearchField::Name => 3,\n                FileSearchField::Content => 4,\n            },\n            SearchField::Tracing(field) => match field {\n                TracingSearchField::EventType => 3,\n                TracingSearchField::QueueId => 4,\n                TracingSearchField::Keywords => 5,\n            },\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/assert.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{Archive, ArchiveVersion};\nuse crate::{U32_LEN, U64_LEN};\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum AssertValue {\n    U32(u32),\n    U64(u64),\n    Archive(ArchiveVersion),\n    Some,\n    None,\n}\n\npub trait ToAssertValue {\n    fn to_assert_value(&self) -> AssertValue;\n}\n\nimpl ToAssertValue for AssertValue {\n    fn to_assert_value(&self) -> AssertValue {\n        *self\n    }\n}\n\nimpl ToAssertValue for () {\n    fn to_assert_value(&self) -> AssertValue {\n        AssertValue::None\n    }\n}\n\nimpl ToAssertValue for u64 {\n    fn to_assert_value(&self) -> AssertValue {\n        AssertValue::U64(*self)\n    }\n}\n\nimpl ToAssertValue for u32 {\n    fn to_assert_value(&self) -> AssertValue {\n        AssertValue::U32(*self)\n    }\n}\n\nimpl<T> ToAssertValue for Archive<T> {\n    fn to_assert_value(&self) -> AssertValue {\n        AssertValue::Archive(self.version)\n    }\n}\n\nimpl<T> ToAssertValue for &Archive<T> {\n    fn to_assert_value(&self) -> AssertValue {\n        AssertValue::Archive(self.version)\n    }\n}\n\nimpl AssertValue {\n    pub fn matches(&self, bytes: &[u8]) -> bool {\n        match self {\n            AssertValue::U32(v) => bytes\n                .get(bytes.len() - U32_LEN..)\n                .is_some_and(|b| b == v.to_be_bytes()),\n\n            AssertValue::U64(v) => bytes\n                .get(bytes.len() - U64_LEN..)\n                .is_some_and(|b| b == v.to_be_bytes()),\n            AssertValue::Archive(v) => match v {\n                ArchiveVersion::Versioned { hash, .. } => bytes\n                    .get(bytes.len() - U32_LEN - U64_LEN - 1..bytes.len() - U64_LEN - 1)\n                    .is_some_and(|b| b == hash.to_be_bytes()),\n                ArchiveVersion::Hashed { hash } => bytes\n                    .get(bytes.len() - U32_LEN - 1..bytes.len() - 1)\n                    .is_some_and(|b| b == hash.to_be_bytes()),\n                ArchiveVersion::Unversioned => false,\n            },\n            AssertValue::None => false,\n            AssertValue::Some => true,\n        }\n    }\n\n    pub fn is_none(&self) -> bool {\n        matches!(self, AssertValue::None)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/batch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    Batch, BatchBuilder, ChangedCollection, IntoOperations, Operation, ValueClass, ValueOp,\n    assert::ToAssertValue, log::VanishedItem,\n};\nuse crate::{\n    SerializeInfallible, U32_LEN,\n    write::{LogCollection, MergeFnc, MergeOperation, Params, SetFnc, SetOperation},\n};\nuse types::{\n    collection::{Collection, SyncCollection, VanishedCollection},\n    field::FieldType,\n};\nuse utils::map::vec_map::VecMap;\n\nimpl BatchBuilder {\n    pub fn new() -> Self {\n        Self {\n            ops: Vec::with_capacity(32),\n            current_account_id: None,\n            current_collection: None,\n            current_document_id: None,\n            changes: Default::default(),\n            changed_collections: Default::default(),\n            batch_size: 0,\n            batch_ops: 0,\n            has_assertions: false,\n            commit_points: Vec::new(),\n        }\n    }\n\n    pub fn with_account_id(&mut self, account_id: u32) -> &mut Self {\n        if self\n            .current_account_id\n            .is_none_or(|current_account_id| current_account_id != account_id)\n        {\n            self.current_account_id = account_id.into();\n            self.ops.push(Operation::AccountId { account_id });\n        }\n        self\n    }\n\n    pub fn with_collection(&mut self, collection: Collection) -> &mut Self {\n        let collection_ = Some(collection);\n        if collection_ != self.current_collection {\n            self.current_collection = collection_;\n            self.ops.push(Operation::Collection { collection });\n        }\n        self\n    }\n\n    pub fn with_document(&mut self, document_id: u32) -> &mut Self {\n        self.ops.push(Operation::DocumentId { document_id });\n        self.current_document_id = Some(document_id);\n        self.has_assertions = false;\n        self\n    }\n\n    pub fn assert_value(\n        &mut self,\n        class: impl Into<ValueClass>,\n        value: impl ToAssertValue,\n    ) -> &mut Self {\n        self.ops.push(Operation::AssertValue {\n            class: class.into(),\n            assert_value: value.to_assert_value(),\n        });\n        self.batch_ops += 1;\n        self.has_assertions = true;\n        self\n    }\n\n    pub fn index(&mut self, field: impl FieldType, value: impl Into<Vec<u8>>) -> &mut Self {\n        let field = field.into();\n        let value = value.into();\n        let value_len = value.len();\n\n        self.ops.push(Operation::Index {\n            field,\n            key: value,\n            set: true,\n        });\n        self.batch_size += (U32_LEN * 3) + value_len;\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn unindex(&mut self, field: impl FieldType, value: impl Into<Vec<u8>>) -> &mut Self {\n        let field = field.into();\n        let value = value.into();\n        let value_len = value.len();\n\n        self.ops.push(Operation::Index {\n            field,\n            key: value,\n            set: false,\n        });\n        self.batch_size += (U32_LEN * 3) + value_len;\n        self.batch_ops += 1;\n        self\n    }\n\n    #[inline(always)]\n    pub fn tag(&mut self, field: impl FieldType) -> &mut Self {\n        self.index(field, vec![])\n    }\n\n    #[inline(always)]\n    pub fn untag(&mut self, field: impl FieldType) -> &mut Self {\n        self.unindex(field, vec![])\n    }\n\n    pub fn add(&mut self, class: impl Into<ValueClass>, value: i64) -> &mut Self {\n        let class = class.into();\n        self.batch_size += class.serialized_size() + std::mem::size_of::<i64>();\n        self.ops.push(Operation::Value {\n            class,\n            op: ValueOp::AtomicAdd(value),\n        });\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn add_and_get(&mut self, class: impl Into<ValueClass>, value: i64) -> &mut Self {\n        let class = class.into();\n        self.batch_size += class.serialized_size() + (std::mem::size_of::<i64>() * 2);\n        self.ops.push(Operation::Value {\n            class,\n            op: ValueOp::AddAndGet(value),\n        });\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn set(&mut self, class: impl Into<ValueClass>, value: impl Into<Vec<u8>>) -> &mut Self {\n        let class = class.into();\n        let value = value.into();\n        self.batch_size += class.serialized_size() + value.len();\n        self.ops.push(Operation::Value {\n            class,\n            op: ValueOp::Set(value),\n        });\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn set_fnc(\n        &mut self,\n        class: impl Into<ValueClass>,\n        params: Params,\n        fnc: SetFnc,\n    ) -> &mut Self {\n        self.ops.push(Operation::Value {\n            class: class.into(),\n            op: ValueOp::SetFnc(SetOperation { fnc, params }),\n        });\n        self\n    }\n\n    pub fn merge_fnc(\n        &mut self,\n        class: impl Into<ValueClass>,\n        params: Params,\n        fnc: MergeFnc,\n    ) -> &mut Self {\n        self.ops.push(Operation::Value {\n            class: class.into(),\n            op: ValueOp::MergeFnc(MergeOperation { fnc, params }),\n        });\n        self\n    }\n\n    pub fn clear(&mut self, class: impl Into<ValueClass>) -> &mut Self {\n        let class = class.into();\n        self.batch_size += class.serialized_size();\n        self.ops.push(Operation::Value {\n            class,\n            op: ValueOp::Clear,\n        });\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn acl_grant(&mut self, grant_account_id: u32, op: Vec<u8>) -> &mut Self {\n        self.batch_size += (U32_LEN * 3) + op.len();\n        self.ops.push(Operation::Value {\n            class: ValueClass::Acl(grant_account_id),\n            op: ValueOp::Set(op),\n        });\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn acl_revoke(&mut self, grant_account_id: u32) -> &mut Self {\n        self.batch_size += U32_LEN * 3;\n        self.ops.push(Operation::Value {\n            class: ValueClass::Acl(grant_account_id),\n            op: ValueOp::Clear,\n        });\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn log_item_insert(\n        &mut self,\n        collection: SyncCollection,\n        prefix: Option<u32>,\n    ) -> &mut Self {\n        if let (Some(account_id), Some(document_id)) =\n            (self.current_account_id, self.current_document_id)\n        {\n            self.changes.get_mut_or_insert(account_id).log_item_insert(\n                collection,\n                prefix,\n                document_id,\n            );\n        }\n        self\n    }\n\n    pub fn log_item_update(\n        &mut self,\n        collection: SyncCollection,\n        prefix: Option<u32>,\n    ) -> &mut Self {\n        if let (Some(account_id), Some(document_id)) =\n            (self.current_account_id, self.current_document_id)\n        {\n            self.changes.get_mut_or_insert(account_id).log_item_update(\n                collection,\n                prefix,\n                document_id,\n            );\n        }\n        self\n    }\n\n    pub fn log_item_delete(\n        &mut self,\n        collection: SyncCollection,\n        prefix: Option<u32>,\n    ) -> &mut Self {\n        if let (Some(account_id), Some(document_id)) =\n            (self.current_account_id, self.current_document_id)\n        {\n            self.changes.get_mut_or_insert(account_id).log_item_delete(\n                collection,\n                prefix,\n                document_id,\n            );\n        }\n        self\n    }\n\n    pub fn log_container_insert(&mut self, collection: SyncCollection) -> &mut Self {\n        if let (Some(account_id), Some(document_id)) =\n            (self.current_account_id, self.current_document_id)\n        {\n            self.changes\n                .get_mut_or_insert(account_id)\n                .log_container_insert(collection, document_id);\n        }\n        self\n    }\n\n    pub fn log_container_update(&mut self, collection: SyncCollection) -> &mut Self {\n        if let (Some(account_id), Some(document_id)) =\n            (self.current_account_id, self.current_document_id)\n        {\n            self.changes\n                .get_mut_or_insert(account_id)\n                .log_container_update(collection, document_id);\n        }\n        self\n    }\n\n    pub fn log_container_delete(&mut self, collection: SyncCollection) -> &mut Self {\n        if let (Some(account_id), Some(document_id)) =\n            (self.current_account_id, self.current_document_id)\n        {\n            self.changes\n                .get_mut_or_insert(account_id)\n                .log_container_delete(collection, document_id);\n        }\n        self\n    }\n\n    pub fn log_container_property_change(\n        &mut self,\n        collection: SyncCollection,\n        document_id: u32,\n    ) -> &mut Self {\n        if let Some(account_id) = self.current_account_id {\n            self.changes\n                .get_mut_or_insert(account_id)\n                .log_container_property_update(collection, document_id);\n        }\n        self\n    }\n\n    pub fn log_vanished_item(\n        &mut self,\n        collection: VanishedCollection,\n        item: impl Into<VanishedItem>,\n    ) -> &mut Self {\n        if let Some(account_id) = self.current_account_id {\n            let item = item.into();\n            self.batch_size += item.serialized_size();\n            self.changes\n                .get_mut_or_insert(account_id)\n                .log_vanished_item(collection, item);\n        }\n        self\n    }\n\n    pub fn log_share_notification(\n        &mut self,\n        notification_id: u64,\n        notify_account_id: u32,\n        value: impl SerializeInfallible,\n    ) -> &mut Self {\n        self.changed_collections\n            .get_mut_or_insert(notify_account_id)\n            .share_notification_id = Some(notification_id);\n        self.set(\n            ValueClass::ShareNotification {\n                notification_id,\n                notify_account_id,\n            },\n            value.serialize(),\n        )\n    }\n\n    fn serialize_changes(&mut self) {\n        if !self.changes.is_empty() {\n            for (account_id, changelog) in std::mem::take(&mut self.changes) {\n                self.with_account_id(account_id);\n\n                // Serialize changes\n                for (collection, changes) in changelog.changes.into_iter() {\n                    let cc = self.changed_collections.get_mut_or_insert(account_id);\n                    if changes.has_container_changes() {\n                        cc.changed_containers.insert(collection);\n                    }\n                    if changes.has_item_changes() {\n                        cc.changed_items.insert(collection);\n                    }\n\n                    self.ops.push(Operation::Log {\n                        collection: LogCollection::Sync(collection),\n                        set: changes.serialize(),\n                    });\n                }\n\n                // Serialize vanished items\n                for (collection, vanished) in changelog.vanished.into_iter() {\n                    self.ops.push(Operation::Log {\n                        collection: LogCollection::Vanished(collection),\n                        set: vanished.serialize(),\n                    });\n                }\n            }\n        }\n    }\n\n    pub fn commit_point(&mut self) -> &mut Self {\n        if self.is_large_batch() {\n            self.serialize_changes();\n            self.commit_points.push(self.ops.len());\n            self.batch_ops = 0;\n            self.batch_size = 0;\n            if let Some(account_id) = self.current_account_id {\n                self.ops.push(Operation::AccountId { account_id });\n            }\n            if let Some(collection) = self.current_collection {\n                self.ops.push(Operation::Collection { collection });\n            }\n        }\n        self\n    }\n\n    #[inline]\n    pub fn is_large_batch(&self) -> bool {\n        self.batch_size > 5_000_000 || self.batch_ops > 1000\n    }\n\n    pub fn any_op(&mut self, op: Operation) -> &mut Self {\n        self.ops.push(op);\n        self.batch_ops += 1;\n        self\n    }\n\n    pub fn custom(&mut self, value: impl IntoOperations) -> trc::Result<&mut Self> {\n        value.build(self)?;\n        Ok(self)\n    }\n\n    pub fn last_account_id(&self) -> Option<u32> {\n        self.current_account_id\n    }\n\n    pub fn last_collection(&self) -> Option<Collection> {\n        self.current_collection\n    }\n\n    pub fn last_document_id(&self) -> Option<u32> {\n        self.current_document_id\n    }\n\n    pub fn commit_points(&mut self) -> CommitPointIterator {\n        self.serialize_changes();\n        CommitPointIterator {\n            commit_points: std::mem::take(&mut self.commit_points),\n            commit_point_last: self.ops.len(),\n            offset_start: 0,\n        }\n    }\n\n    pub fn build_one(&mut self, commit_point: CommitPoint) -> Batch<'_> {\n        Batch {\n            changes: &self.changed_collections,\n            ops: &mut self.ops[commit_point.offset_start..commit_point.offset_end],\n        }\n    }\n\n    pub fn build_all(&mut self) -> Batch<'_> {\n        self.serialize_changes();\n        Batch {\n            changes: &self.changed_collections,\n            ops: self.ops.as_mut_slice(),\n        }\n    }\n\n    pub fn changes(self) -> Option<VecMap<u32, ChangedCollection>> {\n        if self.has_changes() {\n            Some(self.changed_collections)\n        } else {\n            None\n        }\n    }\n\n    pub fn has_changes(&self) -> bool {\n        !self.changed_collections.is_empty()\n    }\n\n    pub fn ops(&self) -> &[Operation] {\n        self.ops.as_slice()\n    }\n\n    pub fn len(&self) -> usize {\n        self.batch_size\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.batch_ops == 0\n    }\n}\n\npub struct CommitPointIterator {\n    commit_points: Vec<usize>,\n    commit_point_last: usize,\n    offset_start: usize,\n}\n\npub struct CommitPoint {\n    pub offset_start: usize,\n    pub offset_end: usize,\n}\n\nimpl CommitPointIterator {\n    pub fn iter(&mut self) -> impl Iterator<Item = CommitPoint> {\n        self.commit_points\n            .iter()\n            .copied()\n            .chain([self.commit_point_last])\n            .map(|offset_end| {\n                let point = CommitPoint {\n                    offset_start: self.offset_start,\n                    offset_end,\n                };\n                self.offset_start = offset_end;\n                point\n            })\n    }\n}\n\nimpl Batch<'_> {\n    pub fn is_atomic(&self) -> bool {\n        !self.ops.iter().any(|op| {\n            matches!(\n                op,\n                Operation::AssertValue { .. }\n                    | Operation::Value {\n                        op: ValueOp::AddAndGet(_),\n                        ..\n                    }\n            )\n        })\n    }\n\n    pub fn first_account_id(&self) -> Option<u32> {\n        self.ops.iter().find_map(|op| match op {\n            Operation::AccountId { account_id } => Some(*account_id),\n            _ => None,\n        })\n    }\n}\n\nimpl Default for BatchBuilder {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/bitpack.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse bitpacking::{BitPacker, BitPacker1x, BitPacker4x, BitPacker8x};\nuse utils::codec::leb128::Leb128Reader;\n\nuse super::key::KeySerializer;\n\n#[derive(Default)]\npub struct BitpackIterator<'x> {\n    pub(crate) bytes: &'x [u8],\n    pub(crate) bytes_offset: usize,\n    pub(crate) chunk: Vec<u32>,\n    pub(crate) chunk_offset: usize,\n    pub items_left: u32,\n}\n\n#[derive(Clone, Copy)]\npub(crate) struct BitBlockPacker {\n    bitpacker_1: BitPacker1x,\n    bitpacker_4: BitPacker4x,\n    bitpacker_8: BitPacker8x,\n    block_len: usize,\n}\n\nimpl KeySerializer {\n    pub fn bitpack_sorted(self, items: &[u32]) -> Self {\n        let mut serializer = self;\n        let mut bitpacker = BitBlockPacker::new();\n        let mut compressed = vec![0u8; 4 * BitPacker8x::BLOCK_LEN];\n\n        let mut pos = 0;\n        let len = items.len();\n        let mut initial_value = None;\n\n        serializer = serializer.write_leb128(len as u32);\n\n        while pos < len {\n            let block_len = match len - pos {\n                0..=31 => {\n                    for val in &items[pos..] {\n                        serializer = serializer.write_leb128(*val);\n                    }\n                    break;\n                }\n                32..=127 => BitPacker1x::BLOCK_LEN,\n                128..=255 => BitPacker4x::BLOCK_LEN,\n                _ => BitPacker8x::BLOCK_LEN,\n            };\n\n            let chunk = &items[pos..pos + block_len];\n            bitpacker.block_len(block_len);\n            let num_bits: u8 = bitpacker.num_bits_strictly_sorted(initial_value, chunk);\n            let compressed_len = bitpacker.compress_strictly_sorted(\n                initial_value,\n                chunk,\n                &mut compressed[..],\n                num_bits,\n            );\n            serializer = serializer\n                .write(num_bits)\n                .write(&compressed[..compressed_len]);\n            initial_value = chunk[chunk.len() - 1].into();\n\n            pos += block_len;\n        }\n        serializer\n    }\n}\n\nimpl<'x> BitpackIterator<'x> {\n    pub fn from_bytes_and_offset(bytes: &'x [u8], bytes_offset: usize, items_left: u32) -> Self {\n        BitpackIterator {\n            bytes,\n            bytes_offset,\n            items_left,\n            ..Default::default()\n        }\n    }\n\n    pub fn new(bytes: &'x [u8]) -> Option<Self> {\n        bytes\n            .read_leb128::<u32>()\n            .map(|(items_left, bytes_offset)| BitpackIterator {\n                bytes,\n                bytes_offset,\n                items_left,\n                ..Default::default()\n            })\n    }\n}\n\nimpl Iterator for BitpackIterator<'_> {\n    type Item = u32;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        if let Some(item) = self.chunk.get(self.chunk_offset) {\n            self.chunk_offset += 1;\n            return Some(*item);\n        }\n        let block_len = match self.items_left {\n            0 => return None,\n            1..=31 => {\n                self.items_left -= 1;\n                let (item, bytes_read) = self.bytes.get(self.bytes_offset..)?.read_leb128()?;\n                self.bytes_offset += bytes_read;\n                return Some(item);\n            }\n            32..=127 => BitPacker1x::BLOCK_LEN,\n            128..=255 => BitPacker4x::BLOCK_LEN,\n            _ => BitPacker8x::BLOCK_LEN,\n        };\n\n        let bitpacker = BitBlockPacker::with_block_len(block_len);\n        let num_bits = *self.bytes.get(self.bytes_offset)?;\n        let bytes_read = ((num_bits as usize) * block_len / 8) + 1;\n        let initial_value = self.chunk.last().copied();\n\n        self.chunk = vec![0u32; block_len];\n        self.chunk_offset = 1;\n\n        bitpacker.decompress_strictly_sorted(\n            initial_value,\n            self.bytes\n                .get(self.bytes_offset + 1..self.bytes_offset + bytes_read)?,\n            &mut self.chunk[..],\n            num_bits,\n        );\n\n        self.bytes_offset += bytes_read;\n        self.items_left -= block_len as u32;\n        self.chunk.first().copied()\n    }\n}\n\nimpl BitBlockPacker {\n    pub fn with_block_len(block_len: usize) -> Self {\n        BitBlockPacker {\n            bitpacker_1: BitPacker1x::new(),\n            bitpacker_4: BitPacker4x::new(),\n            bitpacker_8: BitPacker8x::new(),\n            block_len,\n        }\n    }\n\n    pub fn block_len(&mut self, num: usize) {\n        self.block_len = num;\n    }\n}\n\nimpl BitPacker for BitBlockPacker {\n    const BLOCK_LEN: usize = 0;\n\n    fn new() -> Self {\n        BitBlockPacker {\n            bitpacker_1: BitPacker1x::new(),\n            bitpacker_4: BitPacker4x::new(),\n            bitpacker_8: BitPacker8x::new(),\n            block_len: 1,\n        }\n    }\n\n    fn compress(&self, decompressed: &[u32], compressed: &mut [u8], num_bits: u8) -> usize {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => self\n                .bitpacker_8\n                .compress(decompressed, compressed, num_bits),\n            BitPacker4x::BLOCK_LEN => self\n                .bitpacker_4\n                .compress(decompressed, compressed, num_bits),\n            _ => self\n                .bitpacker_1\n                .compress(decompressed, compressed, num_bits),\n        }\n    }\n\n    fn compress_sorted(\n        &self,\n        initial: u32,\n        decompressed: &[u32],\n        compressed: &mut [u8],\n        num_bits: u8,\n    ) -> usize {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => {\n                self.bitpacker_8\n                    .compress_sorted(initial, decompressed, compressed, num_bits)\n            }\n            BitPacker4x::BLOCK_LEN => {\n                self.bitpacker_4\n                    .compress_sorted(initial, decompressed, compressed, num_bits)\n            }\n            _ => self\n                .bitpacker_1\n                .compress_sorted(initial, decompressed, compressed, num_bits),\n        }\n    }\n\n    fn decompress(&self, compressed: &[u8], decompressed: &mut [u32], num_bits: u8) -> usize {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => {\n                self.bitpacker_8\n                    .decompress(compressed, decompressed, num_bits)\n            }\n            BitPacker4x::BLOCK_LEN => {\n                self.bitpacker_4\n                    .decompress(compressed, decompressed, num_bits)\n            }\n            _ => self\n                .bitpacker_1\n                .decompress(compressed, decompressed, num_bits),\n        }\n    }\n\n    fn decompress_sorted(\n        &self,\n        initial: u32,\n        compressed: &[u8],\n        decompressed: &mut [u32],\n        num_bits: u8,\n    ) -> usize {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => {\n                self.bitpacker_8\n                    .decompress_sorted(initial, compressed, decompressed, num_bits)\n            }\n            BitPacker4x::BLOCK_LEN => {\n                self.bitpacker_4\n                    .decompress_sorted(initial, compressed, decompressed, num_bits)\n            }\n            _ => self\n                .bitpacker_1\n                .decompress_sorted(initial, compressed, decompressed, num_bits),\n        }\n    }\n\n    fn num_bits(&self, decompressed: &[u32]) -> u8 {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => self.bitpacker_8.num_bits(decompressed),\n            BitPacker4x::BLOCK_LEN => self.bitpacker_4.num_bits(decompressed),\n            _ => self.bitpacker_1.num_bits(decompressed),\n        }\n    }\n\n    fn num_bits_sorted(&self, initial: u32, decompressed: &[u32]) -> u8 {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => self.bitpacker_8.num_bits_sorted(initial, decompressed),\n            BitPacker4x::BLOCK_LEN => self.bitpacker_4.num_bits_sorted(initial, decompressed),\n            _ => self.bitpacker_1.num_bits_sorted(initial, decompressed),\n        }\n    }\n\n    fn compress_strictly_sorted(\n        &self,\n        initial: Option<u32>,\n        decompressed: &[u32],\n        compressed: &mut [u8],\n        num_bits: u8,\n    ) -> usize {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => self.bitpacker_8.compress_strictly_sorted(\n                initial,\n                decompressed,\n                compressed,\n                num_bits,\n            ),\n            BitPacker4x::BLOCK_LEN => self.bitpacker_4.compress_strictly_sorted(\n                initial,\n                decompressed,\n                compressed,\n                num_bits,\n            ),\n            _ => self.bitpacker_1.compress_strictly_sorted(\n                initial,\n                decompressed,\n                compressed,\n                num_bits,\n            ),\n        }\n    }\n\n    fn decompress_strictly_sorted(\n        &self,\n        initial: Option<u32>,\n        compressed: &[u8],\n        decompressed: &mut [u32],\n        num_bits: u8,\n    ) -> usize {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => self.bitpacker_8.decompress_strictly_sorted(\n                initial,\n                compressed,\n                decompressed,\n                num_bits,\n            ),\n            BitPacker4x::BLOCK_LEN => self.bitpacker_4.decompress_strictly_sorted(\n                initial,\n                compressed,\n                decompressed,\n                num_bits,\n            ),\n            _ => self.bitpacker_1.decompress_strictly_sorted(\n                initial,\n                compressed,\n                decompressed,\n                num_bits,\n            ),\n        }\n    }\n\n    fn num_bits_strictly_sorted(&self, initial: Option<u32>, decompressed: &[u32]) -> u8 {\n        match self.block_len {\n            BitPacker8x::BLOCK_LEN => self\n                .bitpacker_8\n                .num_bits_strictly_sorted(initial, decompressed),\n            BitPacker4x::BLOCK_LEN => self\n                .bitpacker_4\n                .num_bits_strictly_sorted(initial, decompressed),\n            _ => self\n                .bitpacker_1\n                .num_bits_strictly_sorted(initial, decompressed),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n\n    use super::*;\n\n    #[test]\n    fn bitpack_roundtrip() {\n        for num_positions in [\n            1,\n            10,\n            BitPacker1x::BLOCK_LEN,\n            BitPacker4x::BLOCK_LEN,\n            BitPacker8x::BLOCK_LEN,\n            BitPacker8x::BLOCK_LEN + BitPacker4x::BLOCK_LEN + BitPacker1x::BLOCK_LEN,\n            BitPacker8x::BLOCK_LEN + BitPacker4x::BLOCK_LEN + BitPacker1x::BLOCK_LEN + 1,\n            (BitPacker8x::BLOCK_LEN * 3)\n                + (BitPacker4x::BLOCK_LEN * 3)\n                + (BitPacker1x::BLOCK_LEN * 3)\n                + 1,\n            (BitPacker8x::BLOCK_LEN * 32) + 1,\n        ] {\n            let serialized = KeySerializer::new(num_positions * std::mem::size_of::<u32>())\n                .bitpack_sorted(\n                    &(0..num_positions)\n                        .map(|i| (i * i) as u32)\n                        .collect::<Vec<_>>(),\n                )\n                .finalize();\n\n            println!(\n                \"Testing block {num_positions} with {} size...\",\n                serialized.len()\n            );\n\n            let mut iter = BitpackIterator::new(&serialized).unwrap();\n\n            assert_eq!(\n                iter.items_left, num_positions as u32,\n                \"failed for num_positions: {}\",\n                num_positions\n            );\n\n            for i in 0..num_positions {\n                assert_eq!(\n                    iter.next(),\n                    Some((i * i) as u32),\n                    \"failed for position: {}\",\n                    i\n                );\n            }\n            assert_eq!(iter.next(), None, \"expected end of iterator\");\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Instant;\n\nuse super::{BlobOp, Operation, ValueClass, ValueOp, key::DeserializeBigEndian, now};\nuse crate::{\n    BlobStore, IterateParams, Store, U32_LEN, U64_LEN, ValueKey,\n    write::{BatchBuilder, BlobLink},\n};\nuse trc::{AddContext, PurgeEvent};\nuse types::{\n    blob::BlobClass,\n    blob_hash::{BLOB_HASH_LEN, BlobHash},\n};\n\n#[derive(Debug, PartialEq, Eq)]\npub struct BlobQuota {\n    pub bytes: usize,\n    pub count: usize,\n}\n\nimpl Store {\n    pub async fn blob_exists(&self, hash: impl AsRef<BlobHash> + Sync + Send) -> trc::Result<bool> {\n        self.get_value::<()>(ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Blob(BlobOp::Commit {\n                hash: hash.as_ref().clone(),\n            }),\n        })\n        .await\n        .map(|v| v.is_some())\n        .caused_by(trc::location!())\n    }\n\n    pub async fn blob_quota(&self, account_id: u32) -> trc::Result<BlobQuota> {\n        let from_key = ValueKey {\n            account_id,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Blob(BlobOp::Quota {\n                hash: BlobHash::default(),\n                until: 0,\n            }),\n        };\n        let to_key = ValueKey {\n            account_id: account_id + 1,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Blob(BlobOp::Quota {\n                hash: BlobHash::default(),\n                until: u64::MAX,\n            }),\n        };\n\n        let now = now();\n        let mut quota = BlobQuota { bytes: 0, count: 0 };\n\n        self.iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let until = key.deserialize_be_u64(key.len() - U64_LEN)?;\n                if until > now {\n                    let bytes = value.deserialize_be_u32(0)?;\n                    if bytes > 0 {\n                        quota.bytes += bytes as usize;\n                        quota.count += 1;\n                    }\n                }\n                Ok(true)\n            },\n        )\n        .await\n        .caused_by(trc::location!())?;\n\n        Ok(quota)\n    }\n\n    pub async fn blob_has_access(\n        &self,\n        hash: impl AsRef<BlobHash> + Sync + Send,\n        class: impl AsRef<BlobClass> + Sync + Send,\n    ) -> trc::Result<bool> {\n        let key = match class.as_ref() {\n            BlobClass::Reserved {\n                account_id,\n                expires,\n            } if *expires > now() => ValueKey {\n                account_id: *account_id,\n                collection: 0,\n                document_id: 0,\n                class: ValueClass::Blob(BlobOp::Link {\n                    hash: hash.as_ref().clone(),\n                    to: BlobLink::Temporary { until: *expires },\n                }),\n            },\n            BlobClass::Linked {\n                account_id,\n                collection,\n                document_id,\n            } => ValueKey {\n                account_id: *account_id,\n                collection: *collection,\n                document_id: *document_id,\n                class: ValueClass::Blob(BlobOp::Link {\n                    hash: hash.as_ref().clone(),\n                    to: BlobLink::Document,\n                }),\n            },\n            _ => return Ok(false),\n        };\n\n        self.get_value::<()>(key).await.map(|v| v.is_some())\n    }\n\n    pub async fn purge_blobs(&self, blob_store: BlobStore) -> trc::Result<()> {\n        let mut total_active = 0;\n        let mut total_deleted = 0;\n        let started = Instant::now();\n\n        for byte in 0..=u8::MAX {\n            // Validate linked blobs\n            let mut from_hash = BlobHash::default();\n            let mut to_hash = BlobHash::new_max();\n            from_hash.0[0] = byte;\n            to_hash.0[0] = byte;\n            let from_key = ValueKey {\n                account_id: 0,\n                collection: 0,\n                document_id: 0,\n                class: ValueClass::Blob(BlobOp::Commit { hash: from_hash }),\n            };\n            let to_key = ValueKey {\n                account_id: u32::MAX,\n                collection: u8::MAX,\n                document_id: u32::MAX,\n                class: ValueClass::Blob(BlobOp::Link {\n                    hash: to_hash,\n                    to: BlobLink::Document,\n                }),\n            };\n\n            let mut state = BlobPurgeState::new();\n            self.iterate(\n                IterateParams::new(from_key, to_key).ascending(),\n                |key, value| {\n                    let hash =\n                        BlobHash::try_from_hash_slice(key.get(0..BLOB_HASH_LEN).ok_or_else(\n                            || trc::Error::corrupted_key(key, value.into(), trc::location!()),\n                        )?)\n                        .unwrap();\n\n                    state.update_hash(hash);\n                    state.process_key(key, value)?;\n\n                    Ok(true)\n                },\n            )\n            .await\n            .caused_by(trc::location!())?;\n\n            state.finalize(BlobHash::default());\n\n            // Delete expired or unlinked blobs\n            for (_, op) in &state.delete_keys {\n                if let BlobOp::Commit { hash } = op {\n                    blob_store\n                        .delete_blob(hash.as_ref())\n                        .await\n                        .caused_by(trc::location!())?;\n                }\n            }\n\n            // Delete hashes\n            let mut batch = BatchBuilder::new();\n            for (account_id, op) in state.delete_keys {\n                if batch.is_large_batch() {\n                    self.write(batch.build_all())\n                        .await\n                        .caused_by(trc::location!())?;\n                    batch = BatchBuilder::new();\n                }\n\n                if let Some(account_id) = account_id {\n                    batch.with_account_id(account_id);\n                }\n\n                batch.any_op(Operation::Value {\n                    class: ValueClass::Blob(op),\n                    op: ValueOp::Clear,\n                });\n            }\n            if !batch.is_empty() {\n                self.write(batch.build_all())\n                    .await\n                    .caused_by(trc::location!())?;\n            }\n\n            total_active += state.total_active - 1; // Exclude default hash\n            total_deleted += state.total_deleted;\n        }\n\n        trc::event!(\n            Purge(PurgeEvent::BlobCleanup),\n            Expires = total_deleted,\n            Total = total_active,\n            Elapsed = started.elapsed()\n        );\n\n        Ok(())\n    }\n}\n\nstruct BlobPurgeState {\n    last_hash: BlobHash,\n    last_hash_is_linked: bool,\n    delete_keys: Vec<(Option<u32>, BlobOp)>,\n    spam_train_samples: Vec<(u32, u64)>,\n    now: u64,\n    total_deleted: u64,\n    total_active: u64,\n}\n\nimpl BlobPurgeState {\n    fn new() -> Self {\n        Self {\n            last_hash: BlobHash::default(),\n            last_hash_is_linked: true, // Avoid deleting non-existing last_hash on first iteration\n            delete_keys: Vec::new(),\n            spam_train_samples: Vec::new(),\n            now: now(),\n            total_deleted: 0,\n            total_active: 0,\n        }\n    }\n\n    pub fn update_hash(&mut self, hash: BlobHash) {\n        if self.last_hash != hash {\n            self.finalize(hash);\n            self.last_hash_is_linked = false;\n        }\n    }\n\n    pub fn finalize(&mut self, new_hash: BlobHash) {\n        if !self.last_hash_is_linked {\n            self.total_deleted += 1;\n            self.delete_keys.push((\n                None,\n                BlobOp::Commit {\n                    hash: std::mem::replace(&mut self.last_hash, new_hash),\n                },\n            ));\n        } else {\n            self.total_active += 1;\n            if !self.spam_train_samples.is_empty() {\n                if self.spam_train_samples.len() > 1 {\n                    // Sort by account_id ascending, then until descending\n                    self.spam_train_samples\n                        .sort_unstable_by(|(a_id, a_until), (b_id, b_until)| {\n                            a_id.cmp(b_id).then_with(|| b_until.cmp(a_until))\n                        });\n                    let mut samples = self.spam_train_samples.iter().peekable();\n                    while let Some((account_id, _)) = samples.next() {\n                        // Keep only the latest sample per account\n                        while let Some((next_account_id, next_until)) = samples.peek() {\n                            if next_account_id == account_id {\n                                self.delete_keys.push((\n                                    Some(*account_id),\n                                    BlobOp::SpamSample {\n                                        hash: self.last_hash.clone(),\n                                        until: *next_until,\n                                    },\n                                ));\n                                self.delete_keys.push((\n                                    Some(*account_id),\n                                    BlobOp::Link {\n                                        hash: self.last_hash.clone(),\n                                        to: BlobLink::Temporary { until: *next_until },\n                                    },\n                                ));\n                                samples.next();\n                            } else {\n                                break;\n                            }\n                        }\n                    }\n                }\n\n                self.spam_train_samples.clear();\n            }\n            self.last_hash = new_hash;\n        }\n    }\n\n    pub fn process_key(&mut self, key: &[u8], value: &[u8]) -> trc::Result<()> {\n        const TEMP_LINK: usize = BLOB_HASH_LEN + U32_LEN + U64_LEN;\n        const DOC_LINK: usize = BLOB_HASH_LEN + U64_LEN + 1;\n        const ID_LINK: usize = BLOB_HASH_LEN + U64_LEN;\n\n        match key.len() {\n            BLOB_HASH_LEN => {\n                // Main blob entry\n                Ok(())\n            }\n            TEMP_LINK => {\n                // Temporary link\n                let until = key.deserialize_be_u64(BLOB_HASH_LEN + U32_LEN)?;\n                if until <= self.now {\n                    let account_id = key.deserialize_be_u32(BLOB_HASH_LEN)?;\n                    self.delete_keys.push((\n                        Some(account_id),\n                        BlobOp::Link {\n                            hash: self.last_hash.clone(),\n                            to: BlobLink::Temporary { until },\n                        },\n                    ));\n                    match value.first().copied() {\n                        Some(BlobLink::QUOTA_LINK) => {\n                            self.delete_keys.push((\n                                Some(account_id),\n                                BlobOp::Quota {\n                                    hash: self.last_hash.clone(),\n                                    until,\n                                },\n                            ));\n                        }\n                        Some(BlobLink::UNDELETE_LINK) => {\n                            self.delete_keys.push((\n                                Some(account_id),\n                                BlobOp::Undelete {\n                                    hash: self.last_hash.clone(),\n                                    until,\n                                },\n                            ));\n                        }\n                        Some(BlobLink::SPAM_SAMPLE_LINK) => {\n                            self.delete_keys.push((\n                                Some(account_id),\n                                BlobOp::SpamSample {\n                                    hash: self.last_hash.clone(),\n                                    until,\n                                },\n                            ));\n                        }\n                        _ => {}\n                    }\n                } else {\n                    // Delete attempts to train the same message multiple times\n                    if matches!(value.first(), Some(&BlobLink::SPAM_SAMPLE_LINK)) {\n                        let account_id = key.deserialize_be_u32(BLOB_HASH_LEN)?;\n                        self.spam_train_samples.push((account_id, until));\n                    }\n\n                    self.last_hash_is_linked = true;\n                }\n                Ok(())\n            }\n            DOC_LINK | ID_LINK => {\n                // Document/Id link\n                self.last_hash_is_linked = true;\n                Ok(())\n            }\n            _ => Err(trc::Error::corrupted_key(\n                key,\n                value.into(),\n                trc::location!(),\n            )),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/key.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{\n    AnyKey, BlobOp, DirectoryClass, InMemoryClass, QueueClass, ReportClass, ReportEvent,\n    TaskQueueClass, TelemetryClass, ValueClass,\n};\nuse crate::{\n    Deserialize, IndexKey, IndexKeyPrefix, Key, LogKey, SUBSPACE_ACL, SUBSPACE_BLOB_EXTRA,\n    SUBSPACE_BLOB_LINK, SUBSPACE_COUNTER, SUBSPACE_DIRECTORY, SUBSPACE_IN_MEMORY_COUNTER,\n    SUBSPACE_IN_MEMORY_VALUE, SUBSPACE_INDEXES, SUBSPACE_LOGS, SUBSPACE_PROPERTY,\n    SUBSPACE_QUEUE_EVENT, SUBSPACE_QUEUE_MESSAGE, SUBSPACE_QUOTA, SUBSPACE_REPORT_IN,\n    SUBSPACE_REPORT_OUT, SUBSPACE_SEARCH_INDEX, SUBSPACE_SETTINGS, SUBSPACE_TASK_QUEUE,\n    SUBSPACE_TELEMETRY_METRIC, SUBSPACE_TELEMETRY_SPAN, U16_LEN, U32_LEN, U64_LEN, ValueKey,\n    WITH_SUBSPACE,\n    write::{BlobLink, IndexPropertyClass, SearchIndex, SearchIndexId, SearchIndexType},\n};\nuse std::convert::TryInto;\nuse types::{blob_hash::BLOB_HASH_LEN, collection::SyncCollection, field::Field};\nuse utils::codec::leb128::Leb128_;\n\npub struct KeySerializer {\n    pub buf: Vec<u8>,\n}\n\npub trait KeySerialize {\n    fn serialize(&self, buf: &mut Vec<u8>);\n}\n\npub trait DeserializeBigEndian {\n    fn deserialize_be_u16(&self, index: usize) -> trc::Result<u16>;\n    fn deserialize_be_u32(&self, index: usize) -> trc::Result<u32>;\n    fn deserialize_be_u64(&self, index: usize) -> trc::Result<u64>;\n}\n\nimpl KeySerializer {\n    pub fn new(capacity: usize) -> Self {\n        Self {\n            buf: Vec::with_capacity(capacity),\n        }\n    }\n\n    pub fn write<T: KeySerialize>(mut self, value: T) -> Self {\n        value.serialize(&mut self.buf);\n        self\n    }\n\n    pub fn write_leb128<T: Leb128_>(mut self, value: T) -> Self {\n        T::to_leb128_bytes(value, &mut self.buf);\n        self\n    }\n\n    pub fn finalize(self) -> Vec<u8> {\n        self.buf\n    }\n}\n\nimpl KeySerialize for u8 {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.push(*self);\n    }\n}\n\nimpl KeySerialize for &str {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(self.as_bytes());\n    }\n}\n\nimpl KeySerialize for &String {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(self.as_bytes());\n    }\n}\n\nimpl KeySerialize for &[u8] {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(self);\n    }\n}\n\nimpl KeySerialize for u32 {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(&self.to_be_bytes());\n    }\n}\n\nimpl KeySerialize for u16 {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(&self.to_be_bytes());\n    }\n}\n\nimpl KeySerialize for u64 {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        buf.extend_from_slice(&self.to_be_bytes());\n    }\n}\n\nimpl DeserializeBigEndian for &[u8] {\n    fn deserialize_be_u16(&self, index: usize) -> trc::Result<u16> {\n        self.get(index..index + U16_LEN)\n            .ok_or_else(|| {\n                trc::StoreEvent::DataCorruption\n                    .caused_by(trc::location!())\n                    .ctx(trc::Key::Value, *self)\n            })\n            .and_then(|bytes| {\n                bytes.try_into().map_err(|_| {\n                    trc::StoreEvent::DataCorruption\n                        .caused_by(trc::location!())\n                        .ctx(trc::Key::Value, *self)\n                })\n            })\n            .map(u16::from_be_bytes)\n    }\n\n    fn deserialize_be_u32(&self, index: usize) -> trc::Result<u32> {\n        self.get(index..index + U32_LEN)\n            .ok_or_else(|| {\n                trc::StoreEvent::DataCorruption\n                    .caused_by(trc::location!())\n                    .ctx(trc::Key::Value, *self)\n            })\n            .and_then(|bytes| {\n                bytes.try_into().map_err(|_| {\n                    trc::StoreEvent::DataCorruption\n                        .caused_by(trc::location!())\n                        .ctx(trc::Key::Value, *self)\n                })\n            })\n            .map(u32::from_be_bytes)\n    }\n\n    fn deserialize_be_u64(&self, index: usize) -> trc::Result<u64> {\n        self.get(index..index + U64_LEN)\n            .ok_or_else(|| {\n                trc::StoreEvent::DataCorruption\n                    .caused_by(trc::location!())\n                    .ctx(trc::Key::Value, *self)\n            })\n            .and_then(|bytes| {\n                bytes.try_into().map_err(|_| {\n                    trc::StoreEvent::DataCorruption\n                        .caused_by(trc::location!())\n                        .ctx(trc::Key::Value, *self)\n                })\n            })\n            .map(u64::from_be_bytes)\n    }\n}\n\nimpl<T: AsRef<ValueClass>> ValueKey<T> {\n    pub fn with_document_id(self, document_id: u32) -> Self {\n        Self {\n            document_id,\n            ..self\n        }\n    }\n\n    pub fn is_counter(&self) -> bool {\n        self.class.as_ref().is_counter(self.collection)\n    }\n}\n\nimpl ValueKey<ValueClass> {\n    pub fn property(\n        account_id: u32,\n        collection: impl Into<u8>,\n        document_id: u32,\n        field: impl Into<u8>,\n    ) -> ValueKey<ValueClass> {\n        ValueKey {\n            account_id,\n            collection: collection.into(),\n            document_id,\n            class: ValueClass::Property(field.into()),\n        }\n    }\n\n    pub fn archive(\n        account_id: u32,\n        collection: impl Into<u8>,\n        document_id: u32,\n    ) -> ValueKey<ValueClass> {\n        ValueKey {\n            account_id,\n            collection: collection.into(),\n            document_id,\n            class: ValueClass::Property(Field::ARCHIVE.into()),\n        }\n    }\n}\n\nimpl Key for IndexKeyPrefix {\n    fn serialize(&self, flags: u32) -> Vec<u8> {\n        {\n            if (flags & WITH_SUBSPACE) != 0 {\n                KeySerializer::new(std::mem::size_of::<IndexKeyPrefix>() + 1)\n                    .write(crate::SUBSPACE_INDEXES)\n            } else {\n                KeySerializer::new(std::mem::size_of::<IndexKeyPrefix>())\n            }\n        }\n        .write(self.account_id)\n        .write(self.collection)\n        .write(self.field)\n        .finalize()\n    }\n\n    fn subspace(&self) -> u8 {\n        SUBSPACE_INDEXES\n    }\n}\n\nimpl IndexKeyPrefix {\n    pub fn len() -> usize {\n        U32_LEN + 2\n    }\n}\n\nimpl Key for LogKey {\n    fn subspace(&self) -> u8 {\n        SUBSPACE_LOGS\n    }\n\n    fn serialize(&self, flags: u32) -> Vec<u8> {\n        {\n            if (flags & WITH_SUBSPACE) != 0 {\n                KeySerializer::new(std::mem::size_of::<LogKey>() + 1).write(crate::SUBSPACE_LOGS)\n            } else {\n                KeySerializer::new(std::mem::size_of::<LogKey>())\n            }\n        }\n        .write(self.account_id)\n        .write(self.collection)\n        .write(self.change_id)\n        .finalize()\n    }\n}\n\nimpl<T: AsRef<ValueClass> + Sync + Send + Clone> Key for ValueKey<T> {\n    fn subspace(&self) -> u8 {\n        self.class.as_ref().subspace(self.collection)\n    }\n\n    fn serialize(&self, flags: u32) -> Vec<u8> {\n        self.class\n            .as_ref()\n            .serialize(self.account_id, self.collection, self.document_id, flags)\n    }\n}\n\nimpl ValueClass {\n    pub fn serialize(\n        &self,\n        account_id: u32,\n        collection: u8,\n        document_id: u32,\n        flags: u32,\n    ) -> Vec<u8> {\n        let serializer = if (flags & WITH_SUBSPACE) != 0 {\n            KeySerializer::new(self.serialized_size() + 2).write(self.subspace(collection))\n        } else {\n            KeySerializer::new(self.serialized_size() + 1)\n        };\n\n        match self {\n            ValueClass::Property(property) => serializer\n                .write(account_id)\n                .write(collection)\n                .write(*property)\n                .write(document_id),\n            ValueClass::IndexProperty(property) => match property {\n                IndexPropertyClass::Hash { property, hash } => serializer\n                    .write(account_id)\n                    .write(collection)\n                    .write(*property)\n                    .write(hash.as_bytes())\n                    .write(document_id),\n                IndexPropertyClass::Integer { property, value } => serializer\n                    .write(account_id)\n                    .write(collection)\n                    .write(*property)\n                    .write(*value)\n                    .write(document_id),\n            },\n            ValueClass::Acl(grant_account_id) => serializer\n                .write(*grant_account_id)\n                .write(account_id)\n                .write(collection)\n                .write(document_id),\n            ValueClass::TaskQueue(task) => match task {\n                TaskQueueClass::UpdateIndex {\n                    index,\n                    is_insert,\n                    due,\n                } => serializer\n                    .write(due.inner())\n                    .write(account_id)\n                    .write(if *is_insert { 7u8 } else { 8u8 })\n                    .write(document_id)\n                    .write(index.to_u8()),\n                TaskQueueClass::SendAlarm {\n                    due,\n                    event_id,\n                    alarm_id,\n                    is_email_alert,\n                } => serializer\n                    .write(due.inner())\n                    .write(account_id)\n                    .write(if *is_email_alert { 3u8 } else { 6u8 })\n                    .write(document_id)\n                    .write(*event_id)\n                    .write(*alarm_id),\n                TaskQueueClass::SendImip { due, is_payload } => {\n                    if !*is_payload {\n                        serializer\n                            .write(due.inner())\n                            .write(account_id)\n                            .write(4u8)\n                            .write(document_id)\n                    } else {\n                        serializer\n                            .write(u64::MAX)\n                            .write(account_id)\n                            .write(5u8)\n                            .write(document_id)\n                            .write(due.inner())\n                    }\n                }\n                TaskQueueClass::MergeThreads { due } => serializer\n                    .write(due.inner())\n                    .write(account_id)\n                    .write(9u8)\n                    .write(document_id),\n            },\n            ValueClass::Blob(op) => match op {\n                BlobOp::Commit { hash } => serializer.write::<&[u8]>(hash.as_ref()),\n                BlobOp::Link { hash, to } => match to {\n                    BlobLink::Id { id } => serializer.write::<&[u8]>(hash.as_ref()).write(*id),\n                    BlobLink::Document => serializer\n                        .write::<&[u8]>(hash.as_ref())\n                        .write(account_id)\n                        .write(collection)\n                        .write(document_id),\n                    BlobLink::Temporary { until } => serializer\n                        .write::<&[u8]>(hash.as_ref())\n                        .write(account_id)\n                        .write(*until),\n                },\n                BlobOp::Quota { hash, until } => serializer\n                    .write(BlobLink::QUOTA_LINK)\n                    .write(account_id)\n                    .write::<&[u8]>(hash.as_ref())\n                    .write(*until),\n                BlobOp::Undelete { hash, until } => serializer\n                    .write(BlobLink::UNDELETE_LINK)\n                    .write(account_id)\n                    .write::<&[u8]>(hash.as_ref())\n                    .write(*until),\n                BlobOp::SpamSample { hash, until } => serializer\n                    .write(BlobLink::SPAM_SAMPLE_LINK)\n                    .write(*until)\n                    .write(account_id)\n                    .write::<&[u8]>(hash.as_ref()),\n            },\n            ValueClass::Config(key) => serializer.write(key.as_slice()),\n            ValueClass::InMemory(lookup) => match lookup {\n                InMemoryClass::Key(key) => serializer.write(key.as_slice()),\n                InMemoryClass::Counter(key) => serializer.write(key.as_slice()),\n            },\n            ValueClass::Directory(directory) => match directory {\n                DirectoryClass::NameToId(name) => serializer.write(0u8).write(name.as_slice()),\n                DirectoryClass::EmailToId(email) => serializer.write(1u8).write(email.as_slice()),\n                DirectoryClass::Principal(uid) => serializer.write(2u8).write_leb128(*uid),\n                DirectoryClass::UsedQuota(uid) => serializer.write(4u8).write_leb128(*uid),\n                DirectoryClass::MemberOf {\n                    principal_id,\n                    member_of,\n                } => serializer.write(5u8).write(*principal_id).write(*member_of),\n                DirectoryClass::Members {\n                    principal_id,\n                    has_member,\n                } => serializer\n                    .write(6u8)\n                    .write(*principal_id)\n                    .write(*has_member),\n                DirectoryClass::Index { word, principal_id } => serializer\n                    .write(7u8)\n                    .write(word.as_slice())\n                    .write(*principal_id),\n            },\n            ValueClass::Queue(queue) => match queue {\n                QueueClass::Message(queue_id) => serializer.write(*queue_id),\n                QueueClass::MessageEvent(event) => serializer\n                    .write(event.due)\n                    .write(event.queue_id)\n                    .write(event.queue_name.as_slice()),\n                QueueClass::DmarcReportHeader(event) => serializer\n                    .write(0u8)\n                    .write(event.due)\n                    .write(event.domain.as_bytes())\n                    .write(event.policy_hash)\n                    .write(event.seq_id)\n                    .write(0u8),\n                QueueClass::TlsReportHeader(event) => serializer\n                    .write(0u8)\n                    .write(event.due)\n                    .write(event.domain.as_bytes())\n                    .write(event.policy_hash)\n                    .write(event.seq_id)\n                    .write(1u8),\n                QueueClass::DmarcReportEvent(event) => serializer\n                    .write(1u8)\n                    .write(event.due)\n                    .write(event.domain.as_bytes())\n                    .write(event.policy_hash)\n                    .write(event.seq_id),\n                QueueClass::TlsReportEvent(event) => serializer\n                    .write(2u8)\n                    .write(event.due)\n                    .write(event.domain.as_bytes())\n                    .write(event.policy_hash)\n                    .write(event.seq_id),\n                QueueClass::QuotaCount(key) => serializer.write(0u8).write(key.as_slice()),\n                QueueClass::QuotaSize(key) => serializer.write(1u8).write(key.as_slice()),\n            },\n            ValueClass::Report(report) => match report {\n                ReportClass::Tls { id, expires } => {\n                    serializer.write(0u8).write(*expires).write(*id)\n                }\n                ReportClass::Dmarc { id, expires } => {\n                    serializer.write(1u8).write(*expires).write(*id)\n                }\n                ReportClass::Arf { id, expires } => {\n                    serializer.write(2u8).write(*expires).write(*id)\n                }\n            },\n            ValueClass::Telemetry(telemetry) => match telemetry {\n                TelemetryClass::Span { span_id } => serializer.write(*span_id),\n                TelemetryClass::Metric {\n                    timestamp,\n                    metric_id,\n                    node_id,\n                } => serializer\n                    .write(*timestamp)\n                    .write_leb128(*metric_id)\n                    .write_leb128(*node_id),\n            },\n            ValueClass::DocumentId => serializer.write(account_id).write(collection),\n            ValueClass::ChangeId => serializer.write(account_id),\n            ValueClass::ShareNotification {\n                notification_id,\n                notify_account_id,\n            } => serializer\n                .write(*notify_account_id)\n                .write(u8::from(SyncCollection::ShareNotification))\n                .write(*notification_id),\n            ValueClass::SearchIndex(index) => match &index.typ {\n                SearchIndexType::Term { field, hash } => {\n                    let class = index.index.as_u8();\n                    match &index.id {\n                        SearchIndexId::Account {\n                            account_id,\n                            document_id,\n                        } => serializer\n                            .write(class)\n                            .write(*account_id)\n                            .write(hash.payload())\n                            .write(hash.payload_len())\n                            .write(*field)\n                            .write(*document_id),\n                        SearchIndexId::Global { id } => serializer\n                            .write(class)\n                            .write(hash.payload())\n                            .write(hash.payload_len())\n                            .write(*field)\n                            .write(*id),\n                    }\n                }\n                SearchIndexType::Index { field } => {\n                    let class = index.index.as_u8() | 1 << 6;\n                    match &index.id {\n                        SearchIndexId::Account {\n                            account_id,\n                            document_id,\n                        } => serializer\n                            .write(class)\n                            .write(*account_id)\n                            .write(field.field_id)\n                            .write(field.data.as_slice())\n                            .write(*document_id),\n                        SearchIndexId::Global { id } => serializer\n                            .write(class)\n                            .write(field.field_id)\n                            .write(field.data.as_slice())\n                            .write(*id),\n                    }\n                }\n                SearchIndexType::Document => {\n                    let class = index.index.as_u8() | 2 << 6;\n                    match &index.id {\n                        SearchIndexId::Account {\n                            account_id,\n                            document_id,\n                        } => serializer\n                            .write(class)\n                            .write(*account_id)\n                            .write(*document_id),\n                        SearchIndexId::Global { id } => serializer.write(class).write(*id),\n                    }\n                }\n            },\n            ValueClass::Any(any) => serializer.write(any.key.as_slice()),\n        }\n        .finalize()\n    }\n}\n\nimpl BlobLink {\n    pub const QUOTA_LINK: u8 = 0;\n    pub const UNDELETE_LINK: u8 = 1;\n    pub const SPAM_SAMPLE_LINK: u8 = 2;\n}\n\nimpl<T: AsRef<[u8]> + Sync + Send + Clone> Key for IndexKey<T> {\n    fn subspace(&self) -> u8 {\n        SUBSPACE_INDEXES\n    }\n\n    fn serialize(&self, flags: u32) -> Vec<u8> {\n        let key = self.key.as_ref();\n        {\n            if (flags & WITH_SUBSPACE) != 0 {\n                KeySerializer::new(std::mem::size_of::<IndexKey<T>>() + key.len() + 1)\n                    .write(crate::SUBSPACE_INDEXES)\n            } else {\n                KeySerializer::new(std::mem::size_of::<IndexKey<T>>() + key.len())\n            }\n        }\n        .write(self.account_id)\n        .write(self.collection)\n        .write(self.field)\n        .write(key)\n        .write(self.document_id)\n        .finalize()\n    }\n}\n\nimpl<T: AsRef<[u8]> + Sync + Send + Clone> Key for AnyKey<T> {\n    fn serialize(&self, flags: u32) -> Vec<u8> {\n        let key = self.key.as_ref();\n        if (flags & WITH_SUBSPACE) != 0 {\n            KeySerializer::new(key.len() + 1).write(self.subspace)\n        } else {\n            KeySerializer::new(key.len())\n        }\n        .write(key)\n        .finalize()\n    }\n\n    fn subspace(&self) -> u8 {\n        self.subspace\n    }\n}\n\nimpl ValueClass {\n    pub fn serialized_size(&self) -> usize {\n        match self {\n            ValueClass::Property(_) => U32_LEN * 2 + 3,\n            ValueClass::IndexProperty(p) => match p {\n                IndexPropertyClass::Hash { hash, .. } => U32_LEN * 2 + 3 + hash.len(),\n                IndexPropertyClass::Integer { .. } => U32_LEN * 2 + 3 + U64_LEN,\n            },\n            ValueClass::Acl(_) => U32_LEN * 3 + 2,\n            ValueClass::InMemory(InMemoryClass::Counter(v) | InMemoryClass::Key(v))\n            | ValueClass::Config(v) => v.len(),\n            ValueClass::Directory(d) => match d {\n                DirectoryClass::NameToId(v) | DirectoryClass::EmailToId(v) => v.len(),\n                DirectoryClass::Principal(_) | DirectoryClass::UsedQuota(_) => U32_LEN,\n                DirectoryClass::Members { .. } | DirectoryClass::MemberOf { .. } => U32_LEN * 2,\n                DirectoryClass::Index { word, .. } => word.len() + U32_LEN,\n            },\n            ValueClass::Blob(op) => match op {\n                BlobOp::Commit { .. } => BLOB_HASH_LEN,\n                BlobOp::Link { to, .. } => {\n                    BLOB_HASH_LEN\n                        + match to {\n                            BlobLink::Id { .. } => U64_LEN,\n                            BlobLink::Document => U32_LEN * 2 + 1,\n                            BlobLink::Temporary { .. } => U32_LEN + U64_LEN,\n                        }\n                }\n                BlobOp::Quota { .. } | BlobOp::Undelete { .. } => {\n                    BLOB_HASH_LEN + U32_LEN + U64_LEN + 1\n                }\n                BlobOp::SpamSample { .. } => BLOB_HASH_LEN + U32_LEN + 2,\n            },\n            ValueClass::TaskQueue(e) => match e {\n                TaskQueueClass::UpdateIndex { .. } => (U64_LEN * 2) + 2,\n                TaskQueueClass::SendAlarm { .. } | TaskQueueClass::MergeThreads { .. } => {\n                    U64_LEN + (U32_LEN * 3) + 1\n                }\n                TaskQueueClass::SendImip { is_payload, .. } => {\n                    if *is_payload {\n                        (U64_LEN * 2) + (U32_LEN * 2) + 1\n                    } else {\n                        U64_LEN + (U32_LEN * 2) + 1\n                    }\n                }\n            },\n            ValueClass::Queue(q) => match q {\n                QueueClass::Message(_) => U64_LEN,\n                QueueClass::MessageEvent(_) => U64_LEN * 3,\n                QueueClass::DmarcReportEvent(event) | QueueClass::TlsReportEvent(event) => {\n                    event.domain.len() + U64_LEN * 3\n                }\n                QueueClass::DmarcReportHeader(event) | QueueClass::TlsReportHeader(event) => {\n                    event.domain.len() + (U64_LEN * 3) + 1\n                }\n                QueueClass::QuotaCount(v) | QueueClass::QuotaSize(v) => v.len(),\n            },\n            ValueClass::Report(_) => U64_LEN * 2 + 1,\n            ValueClass::Telemetry(telemetry) => match telemetry {\n                TelemetryClass::Span { .. } => U64_LEN + 1,\n                TelemetryClass::Metric { .. } => U64_LEN * 2 + 1,\n            },\n            ValueClass::DocumentId => U32_LEN + 1,\n            ValueClass::ChangeId => U32_LEN,\n            ValueClass::ShareNotification { .. } => U32_LEN + U64_LEN + 1,\n            ValueClass::SearchIndex(v) => match &v.typ {\n                SearchIndexType::Term { hash, .. } => U64_LEN + hash.len() + 2,\n                SearchIndexType::Index { field, .. } => 1 + field.data.len() + U64_LEN,\n                SearchIndexType::Document => match &v.id {\n                    SearchIndexId::Account { .. } => 1 + U32_LEN * 2,\n                    SearchIndexId::Global { .. } => 1 + U64_LEN,\n                },\n            },\n            ValueClass::Any(v) => v.key.len(),\n        }\n    }\n\n    pub fn subspace(&self, collection: u8) -> u8 {\n        match self {\n            ValueClass::Property(field) => {\n                if *field == 84 && collection == 1 {\n                    SUBSPACE_COUNTER\n                } else {\n                    SUBSPACE_PROPERTY\n                }\n            }\n            ValueClass::IndexProperty { .. } => SUBSPACE_PROPERTY,\n            ValueClass::Acl(_) => SUBSPACE_ACL,\n            ValueClass::TaskQueue { .. } => SUBSPACE_TASK_QUEUE,\n            ValueClass::Blob(op) => match op {\n                BlobOp::Commit { .. } | BlobOp::Link { .. } => SUBSPACE_BLOB_LINK,\n                BlobOp::Quota { .. } | BlobOp::Undelete { .. } | BlobOp::SpamSample { .. } => {\n                    SUBSPACE_BLOB_EXTRA\n                }\n            },\n            ValueClass::Config(_) => SUBSPACE_SETTINGS,\n            ValueClass::InMemory(lookup) => match lookup {\n                InMemoryClass::Key(_) => SUBSPACE_IN_MEMORY_VALUE,\n                InMemoryClass::Counter(_) => SUBSPACE_IN_MEMORY_COUNTER,\n            },\n            ValueClass::Directory(directory) => match directory {\n                DirectoryClass::UsedQuota(_) => SUBSPACE_QUOTA,\n                _ => SUBSPACE_DIRECTORY,\n            },\n            ValueClass::Queue(queue) => match queue {\n                QueueClass::Message(_) => SUBSPACE_QUEUE_MESSAGE,\n                QueueClass::MessageEvent(_) => SUBSPACE_QUEUE_EVENT,\n                QueueClass::DmarcReportHeader(_)\n                | QueueClass::TlsReportHeader(_)\n                | QueueClass::DmarcReportEvent(_)\n                | QueueClass::TlsReportEvent(_) => SUBSPACE_REPORT_OUT,\n                QueueClass::QuotaCount(_) | QueueClass::QuotaSize(_) => SUBSPACE_QUOTA,\n            },\n            ValueClass::Report(_) => SUBSPACE_REPORT_IN,\n            ValueClass::Telemetry(telemetry) => match telemetry {\n                TelemetryClass::Span { .. } => SUBSPACE_TELEMETRY_SPAN,\n                TelemetryClass::Metric { .. } => SUBSPACE_TELEMETRY_METRIC,\n            },\n            ValueClass::DocumentId | ValueClass::ChangeId => SUBSPACE_COUNTER,\n            ValueClass::ShareNotification { .. } => SUBSPACE_LOGS,\n            ValueClass::SearchIndex(_) => SUBSPACE_SEARCH_INDEX,\n            ValueClass::Any(any) => any.subspace,\n        }\n    }\n\n    pub fn is_counter(&self, collection: u8) -> bool {\n        match self {\n            ValueClass::Directory(DirectoryClass::UsedQuota(_))\n            | ValueClass::InMemory(InMemoryClass::Counter(_))\n            | ValueClass::Queue(QueueClass::QuotaCount(_) | QueueClass::QuotaSize(_))\n            | ValueClass::DocumentId\n            | ValueClass::ChangeId => true,\n            ValueClass::Property(84) if collection == 1 => true, // TODO: Find a more elegant way to do this\n            _ => false,\n        }\n    }\n}\n\nimpl From<ValueClass> for ValueKey<ValueClass> {\n    fn from(class: ValueClass) -> Self {\n        ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class,\n        }\n    }\n}\n\nimpl From<DirectoryClass> for ValueKey<ValueClass> {\n    fn from(value: DirectoryClass) -> Self {\n        ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Directory(value),\n        }\n    }\n}\n\nimpl From<DirectoryClass> for ValueClass {\n    fn from(value: DirectoryClass) -> Self {\n        ValueClass::Directory(value)\n    }\n}\n\nimpl From<BlobOp> for ValueClass {\n    fn from(value: BlobOp) -> Self {\n        ValueClass::Blob(value)\n    }\n}\n\nimpl Deserialize for ReportEvent {\n    fn deserialize(key: &[u8]) -> trc::Result<Self> {\n        Ok(ReportEvent {\n            due: key.deserialize_be_u64(1)?,\n            policy_hash: key.deserialize_be_u64(key.len() - (U64_LEN * 2 + 1))?,\n            seq_id: key.deserialize_be_u64(key.len() - (U64_LEN + 1))?,\n            domain: key\n                .get(U64_LEN + 1..key.len() - (U64_LEN * 2 + 1))\n                .and_then(|domain| std::str::from_utf8(domain).ok())\n                .map(|s| s.to_string())\n                .ok_or_else(|| {\n                    trc::StoreEvent::DataCorruption\n                        .caused_by(trc::location!())\n                        .ctx(trc::Key::Key, key)\n                })?,\n        })\n    }\n}\n\nimpl SearchIndex {\n    pub fn to_u8(&self) -> u8 {\n        match self {\n            SearchIndex::Email => 0,\n            SearchIndex::Calendar => 1,\n            SearchIndex::Contacts => 2,\n            SearchIndex::File => 3,\n            SearchIndex::Tracing => 4,\n            SearchIndex::InMemory => unreachable!(),\n        }\n    }\n\n    pub fn try_from_u8(value: u8) -> Option<Self> {\n        match value {\n            0 => Some(SearchIndex::Email),\n            1 => Some(SearchIndex::Calendar),\n            2 => Some(SearchIndex::Contacts),\n            3 => Some(SearchIndex::File),\n            4 => Some(SearchIndex::Tracing),\n            _ => None,\n        }\n    }\n\n    pub fn name(&self) -> &'static str {\n        match self {\n            SearchIndex::Email => \"email\",\n            SearchIndex::Calendar => \"calendar\",\n            SearchIndex::Contacts => \"contacts\",\n            SearchIndex::File => \"file\",\n            SearchIndex::Tracing => \"tracing\",\n            SearchIndex::InMemory => \"in_memory\",\n        }\n    }\n\n    pub fn try_from_str(value: &str) -> Option<Self> {\n        match value {\n            \"email\" => Some(SearchIndex::Email),\n            \"calendar\" => Some(SearchIndex::Calendar),\n            \"contacts\" => Some(SearchIndex::Contacts),\n            \"file\" => Some(SearchIndex::File),\n            \"tracing\" => Some(SearchIndex::Tracing),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/log.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{SerializeInfallible, U64_LEN};\nuse ahash::AHashSet;\nuse types::collection::{SyncCollection, VanishedCollection};\nuse utils::{codec::leb128::Leb128Vec, map::vec_map::VecMap};\n\nuse super::key::KeySerializer;\n\n#[derive(Default, Debug)]\npub(crate) struct ChangeLogBuilder {\n    pub changes: VecMap<SyncCollection, Changes>,\n    pub vanished: VecMap<VanishedCollection, VanishedItems>,\n}\n\n#[derive(Clone, Debug, PartialEq, Eq, Hash)]\npub enum VanishedItem {\n    Name(String),\n    Id(u64),\n    IdPair(u32, u32),\n}\n\n#[derive(Default, Debug)]\npub(crate) struct VanishedItems(Vec<VanishedItem>);\n\n#[derive(Default, Debug)]\npub struct Changes {\n    pub item_inserts: AHashSet<u64>,\n    pub item_updates: AHashSet<u64>,\n    pub item_deletes: AHashSet<u64>,\n\n    pub container_inserts: AHashSet<u32>,\n    pub container_updates: AHashSet<u32>,\n    pub container_deletes: AHashSet<u32>,\n    pub container_property_changes: AHashSet<u32>,\n}\n\nimpl ChangeLogBuilder {\n    pub fn log_container_insert(&mut self, collection: SyncCollection, document_id: u32) {\n        let changes = self.changes.get_mut_or_insert(collection);\n        if changes.container_deletes.remove(&document_id) {\n            changes.container_updates.insert(document_id);\n        } else {\n            changes.container_inserts.insert(document_id);\n        }\n    }\n\n    pub fn log_item_insert(\n        &mut self,\n        collection: SyncCollection,\n        prefix: Option<u32>,\n        document_id: u32,\n    ) {\n        let id = build_id(prefix, document_id);\n        let changes = self.changes.get_mut_or_insert(collection);\n        if changes.item_deletes.remove(&id) {\n            changes.item_updates.insert(id);\n        } else {\n            changes.item_inserts.insert(id);\n        }\n    }\n\n    pub fn log_container_update(&mut self, collection: SyncCollection, document_id: u32) {\n        self.changes\n            .get_mut_or_insert(collection)\n            .container_updates\n            .insert(document_id);\n    }\n\n    pub fn log_container_property_update(&mut self, collection: SyncCollection, document_id: u32) {\n        self.changes\n            .get_mut_or_insert(collection)\n            .container_property_changes\n            .insert(document_id);\n    }\n\n    pub fn log_item_update(\n        &mut self,\n        collection: SyncCollection,\n        prefix: Option<u32>,\n        document_id: u32,\n    ) {\n        self.changes\n            .get_mut_or_insert(collection)\n            .item_updates\n            .insert(build_id(prefix, document_id));\n    }\n\n    pub fn log_container_delete(&mut self, collection: SyncCollection, document_id: u32) {\n        let changes = self.changes.get_mut_or_insert(collection);\n        let id = document_id;\n        changes.container_updates.remove(&id);\n        changes.container_property_changes.remove(&id);\n        changes.container_deletes.insert(id);\n    }\n\n    pub fn log_item_delete(\n        &mut self,\n        collection: SyncCollection,\n        prefix: Option<u32>,\n        document_id: u32,\n    ) {\n        let changes = self.changes.get_mut_or_insert(collection);\n        let id = build_id(prefix, document_id);\n        changes.item_updates.remove(&id);\n        changes.item_deletes.insert(id);\n    }\n\n    pub fn log_vanished_item(\n        &mut self,\n        collection: VanishedCollection,\n        item: impl Into<VanishedItem>,\n    ) {\n        self.vanished\n            .get_mut_or_insert(collection)\n            .0\n            .push(item.into());\n    }\n}\n\n#[inline(always)]\nfn build_id(prefix: Option<u32>, document_id: u32) -> u64 {\n    if let Some(prefix) = prefix {\n        ((prefix as u64) << 32) | document_id as u64\n    } else {\n        document_id as u64\n    }\n}\n\nimpl Changes {\n    pub fn has_container_changes(&self) -> bool {\n        !self.container_inserts.is_empty()\n            || !self.container_updates.is_empty()\n            || !self.container_property_changes.is_empty()\n            || !self.container_deletes.is_empty()\n    }\n\n    pub fn has_item_changes(&self) -> bool {\n        !self.item_inserts.is_empty()\n            || !self.item_updates.is_empty()\n            || !self.item_deletes.is_empty()\n    }\n}\n\nimpl SerializeInfallible for Changes {\n    fn serialize(&self) -> Vec<u8> {\n        let mut buf = Vec::with_capacity(\n            1 + (self.item_inserts.len()\n                + self.item_updates.len()\n                + self.item_deletes.len()\n                + self.container_inserts.len()\n                + self.container_updates.len()\n                + self.container_property_changes.len()\n                + self.container_deletes.len()\n                + 4)\n                * std::mem::size_of::<usize>(),\n        );\n\n        buf.push_leb128(self.container_inserts.len());\n        buf.push_leb128(self.container_updates.len());\n        buf.push_leb128(self.container_property_changes.len());\n        buf.push_leb128(self.container_deletes.len());\n        buf.push_leb128(self.item_inserts.len());\n        buf.push_leb128(self.item_updates.len());\n        buf.push_leb128(self.item_deletes.len());\n\n        for list in [\n            &self.container_inserts,\n            &self.container_updates,\n            &self.container_property_changes,\n            &self.container_deletes,\n        ] {\n            for id in list {\n                buf.push_leb128(*id);\n            }\n        }\n        for list in [&self.item_inserts, &self.item_updates, &self.item_deletes] {\n            for id in list {\n                buf.push_leb128(*id);\n            }\n        }\n\n        buf\n    }\n}\n\nimpl From<String> for VanishedItem {\n    fn from(value: String) -> Self {\n        VanishedItem::Name(value)\n    }\n}\n\nimpl From<u64> for VanishedItem {\n    fn from(value: u64) -> Self {\n        VanishedItem::Id(value)\n    }\n}\n\nimpl From<(u32, u32)> for VanishedItem {\n    fn from(value: (u32, u32)) -> Self {\n        VanishedItem::Id((value.0 as u64) << 32 | value.1 as u64)\n    }\n}\n\nimpl VanishedItem {\n    pub fn serialized_size(&self) -> usize {\n        match self {\n            VanishedItem::Name(name) => name.len() + 1,\n            VanishedItem::Id(_) | VanishedItem::IdPair(..) => U64_LEN,\n        }\n    }\n}\n\nimpl SerializeInfallible for VanishedItems {\n    fn serialize(&self) -> Vec<u8> {\n        let mut buf = KeySerializer::new(64);\n\n        for item in &self.0 {\n            buf = match item {\n                VanishedItem::Name(name) => buf.write(name.as_bytes()).write(0u8),\n                VanishedItem::Id(id) => buf.write(id.to_be_bytes().as_slice()),\n                VanishedItem::IdPair(a, b) => buf\n                    .write(a.to_be_bytes().as_slice())\n                    .write(b.to_be_bytes().as_slice()),\n            };\n        }\n\n        buf.finalize()\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse self::assert::AssertValue;\nuse crate::backend::MAX_TOKEN_LENGTH;\nuse log::ChangeLogBuilder;\nuse nlp::tokenizers::word::WordTokenizer;\nuse rkyv::util::AlignedVec;\nuse std::{\n    collections::HashSet,\n    hash::Hash,\n    time::{Duration, SystemTime},\n};\nuse types::{\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection, VanishedCollection},\n    field::{\n        CalendarEventField, CalendarNotificationField, ContactField, EmailField,\n        EmailSubmissionField, Field, MailboxField, PrincipalField, SieveField,\n    },\n};\nuse utils::{\n    cheeky_hash::CheekyHash,\n    map::{bitmap::Bitmap, vec_map::VecMap},\n};\n\npub mod assert;\npub mod batch;\npub mod bitpack;\npub mod blob;\npub mod key;\npub mod log;\npub mod serialize;\n\npub(crate) const ARCHIVE_ALIGNMENT: usize = 16;\n\n#[derive(Debug, Clone)]\npub struct Archive<T> {\n    pub inner: T,\n    pub version: ArchiveVersion,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum ArchiveVersion {\n    Versioned { change_id: u64, hash: u32 },\n    Hashed { hash: u32 },\n    Unversioned,\n}\n\n#[derive(Debug, Clone)]\npub enum AlignedBytes {\n    Aligned(AlignedVec<ARCHIVE_ALIGNMENT>),\n    Vec(Vec<u8>),\n}\n\npub struct Archiver<T>\nwhere\n    T: rkyv::Archive\n        + for<'a> rkyv::Serialize<\n            rkyv::api::high::HighSerializer<\n                rkyv::util::AlignedVec,\n                rkyv::ser::allocator::ArenaHandle<'a>,\n                rkyv::rancor::Error,\n            >,\n        >,\n{\n    pub inner: T,\n    pub flags: u8,\n}\n\n#[derive(Debug, Default)]\npub struct AssignedIds {\n    pub ids: Vec<AssignedId>,\n    current_change_id: Option<u64>,\n}\n\n#[derive(Debug)]\npub enum AssignedId {\n    Counter(i64),\n    ChangeId(ChangeId),\n}\n\n#[derive(Debug, Clone, Copy)]\npub struct ChangeId {\n    pub account_id: u32,\n    pub change_id: u64,\n}\n\n#[cfg(not(feature = \"test_mode\"))]\npub(crate) const MAX_COMMIT_ATTEMPTS: u32 = 10;\n#[cfg(not(feature = \"test_mode\"))]\npub(crate) const MAX_COMMIT_TIME: Duration = Duration::from_secs(10);\n\n#[cfg(feature = \"test_mode\")]\npub(crate) const MAX_COMMIT_ATTEMPTS: u32 = 1000;\n#[cfg(feature = \"test_mode\")]\npub(crate) const MAX_COMMIT_TIME: Duration = Duration::from_secs(3600);\n\n#[derive(Debug)]\npub struct Batch<'x> {\n    pub(crate) changes: &'x VecMap<u32, ChangedCollection>,\n    pub(crate) ops: &'x mut [Operation],\n}\n\n#[derive(Debug)]\npub struct BatchBuilder {\n    current_account_id: Option<u32>,\n    current_collection: Option<Collection>,\n    current_document_id: Option<u32>,\n    changes: VecMap<u32, ChangeLogBuilder>,\n    changed_collections: VecMap<u32, ChangedCollection>,\n    has_assertions: bool,\n    batch_size: usize,\n    batch_ops: usize,\n    commit_points: Vec<usize>,\n    ops: Vec<Operation>,\n}\n\n#[derive(Debug, Default)]\npub struct ChangedCollection {\n    pub changed_containers: Bitmap<SyncCollection>,\n    pub changed_items: Bitmap<SyncCollection>,\n    pub share_notification_id: Option<u64>,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash)]\npub enum Operation {\n    AccountId {\n        account_id: u32,\n    },\n    Collection {\n        collection: Collection,\n    },\n    DocumentId {\n        document_id: u32,\n    },\n    AssertValue {\n        class: ValueClass,\n        assert_value: AssertValue,\n    },\n    Value {\n        class: ValueClass,\n        op: ValueOp,\n    },\n    Index {\n        field: u8,\n        key: Vec<u8>,\n        set: bool,\n    },\n    Log {\n        collection: LogCollection,\n        set: Vec<u8>,\n    },\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Clone, Copy)]\npub enum LogCollection {\n    Sync(SyncCollection),\n    Vanished(VanishedCollection),\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum ValueClass {\n    Property(u8),\n    IndexProperty(IndexPropertyClass),\n    Acl(u32),\n    InMemory(InMemoryClass),\n    TaskQueue(TaskQueueClass),\n    Directory(DirectoryClass),\n    Blob(BlobOp),\n    Config(Vec<u8>),\n    Queue(QueueClass),\n    Report(ReportClass),\n    Telemetry(TelemetryClass),\n    SearchIndex(SearchIndexClass),\n    Any(AnyClass),\n    ShareNotification {\n        notification_id: u64,\n        notify_account_id: u32,\n    },\n    DocumentId,\n    ChangeId,\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum IndexPropertyClass {\n    Hash { property: u8, hash: CheekyHash },\n    Integer { property: u8, value: u64 },\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub struct SearchIndexClass {\n    pub index: SearchIndex,\n    pub id: SearchIndexId,\n    pub typ: SearchIndexType,\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum SearchIndexType {\n    Term { field: u8, hash: CheekyHash },\n    Index { field: SearchIndexField },\n    Document,\n}\n\npub(crate) const SEARCH_INDEX_MAX_FIELD_LEN: usize = 128;\n\n#[derive(Debug, PartialEq, Eq, Clone, Hash, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive)]\npub struct SearchIndexField {\n    pub(crate) field_id: u8,\n    pub(crate) data: Vec<u8>,\n}\n\n#[derive(Debug, PartialEq, Clone, Copy, Eq, Hash)]\npub enum SearchIndexId {\n    Account { account_id: u32, document_id: u32 },\n    Global { id: u64 },\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum TaskQueueClass {\n    UpdateIndex {\n        due: TaskEpoch,\n        index: SearchIndex,\n        is_insert: bool,\n    },\n    SendAlarm {\n        due: TaskEpoch,\n        event_id: u16,\n        alarm_id: u16,\n        is_email_alert: bool,\n    },\n    SendImip {\n        due: TaskEpoch,\n        is_payload: bool,\n    },\n    MergeThreads {\n        due: TaskEpoch,\n    },\n}\n\n#[derive(Debug, PartialEq, Clone, Copy, Eq, Hash)]\n#[repr(transparent)]\npub struct TaskEpoch(pub(crate) u64);\n\n#[derive(Debug, PartialEq, Clone, Copy, Eq, Hash)]\npub enum SearchIndex {\n    Email,\n    Calendar,\n    Contacts,\n    File,\n    Tracing,\n    InMemory,\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub struct AnyClass {\n    pub subspace: u8,\n    pub key: Vec<u8>,\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum InMemoryClass {\n    Key(Vec<u8>),\n    Counter(Vec<u8>),\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum DirectoryClass {\n    NameToId(Vec<u8>),\n    EmailToId(Vec<u8>),\n    Index { word: Vec<u8>, principal_id: u32 },\n    MemberOf { principal_id: u32, member_of: u32 },\n    Members { principal_id: u32, has_member: u32 },\n    Principal(u32),\n    UsedQuota(u32),\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum QueueClass {\n    Message(u64),\n    MessageEvent(QueueEvent),\n    DmarcReportHeader(ReportEvent),\n    DmarcReportEvent(ReportEvent),\n    TlsReportHeader(ReportEvent),\n    TlsReportEvent(ReportEvent),\n    QuotaCount(Vec<u8>),\n    QuotaSize(Vec<u8>),\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum ReportClass {\n    Tls { id: u64, expires: u64 },\n    Dmarc { id: u64, expires: u64 },\n    Arf { id: u64, expires: u64 },\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum TelemetryClass {\n    Span {\n        span_id: u64,\n    },\n    Metric {\n        timestamp: u64,\n        metric_id: u64,\n        node_id: u64,\n    },\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub struct QueueEvent {\n    pub due: u64,\n    pub queue_id: u64,\n    pub queue_name: [u8; 8],\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub struct ReportEvent {\n    pub due: u64,\n    pub policy_hash: u64,\n    pub seq_id: u64,\n    pub domain: String,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash, Default)]\npub enum ValueOp {\n    Set(Vec<u8>),\n    SetFnc(SetOperation),\n    MergeFnc(MergeOperation),\n    AtomicAdd(i64),\n    AddAndGet(i64),\n    #[default]\n    Clear,\n}\n\npub enum MergeResult {\n    Update(Vec<u8>),\n    Skip,\n    Delete,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\npub enum Param {\n    I64(i64),\n    U64(u64),\n    String(String),\n    Bytes(Vec<u8>),\n    Bool(bool),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\n#[repr(transparent)]\npub struct Params(Vec<Param>);\n\npub type SetFnc = fn(&Params, &AssignedIds) -> trc::Result<Vec<u8>>;\npub type MergeFnc = fn(&Params, &AssignedIds, Option<&[u8]>) -> trc::Result<MergeResult>;\n\n#[derive(Debug, Clone)]\npub struct MergeOperation {\n    pub(crate) fnc: MergeFnc,\n    pub(crate) params: Params,\n}\n\n#[derive(Debug, Clone)]\npub struct SetOperation {\n    pub(crate) fnc: SetFnc,\n    pub(crate) params: Params,\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum BlobOp {\n    Commit { hash: BlobHash },\n    Link { hash: BlobHash, to: BlobLink },\n    Quota { hash: BlobHash, until: u64 },\n    Undelete { hash: BlobHash, until: u64 },\n    SpamSample { hash: BlobHash, until: u64 },\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub enum BlobLink {\n    Id { id: u64 },\n    Document,\n    Temporary { until: u64 },\n}\n\n#[derive(Debug, PartialEq, Clone, Eq, Hash)]\npub struct AnyKey<T: AsRef<[u8]>> {\n    pub subspace: u8,\n    pub key: T,\n}\n\npub trait TokenizeText {\n    fn tokenize_into(&self, tokens: &mut HashSet<String>);\n    fn to_tokens(&self) -> HashSet<String>;\n}\n\nimpl TokenizeText for &str {\n    fn tokenize_into(&self, tokens: &mut HashSet<String>) {\n        for token in WordTokenizer::new(self, MAX_TOKEN_LENGTH) {\n            tokens.insert(token.word.into_owned());\n        }\n    }\n\n    fn to_tokens(&self) -> HashSet<String> {\n        let mut tokens = HashSet::new();\n        self.tokenize_into(&mut tokens);\n        tokens\n    }\n}\n\npub trait IntoOperations {\n    fn build(self, batch: &mut BatchBuilder) -> trc::Result<()>;\n}\n\n#[inline(always)]\npub fn now() -> u64 {\n    SystemTime::now()\n        .duration_since(SystemTime::UNIX_EPOCH)\n        .map_or(0, |d| d.as_secs())\n}\n\nimpl AsRef<ValueClass> for ValueClass {\n    fn as_ref(&self) -> &ValueClass {\n        self\n    }\n}\n\nimpl AssignedIds {\n    pub fn push_counter_id(&mut self, id: i64) {\n        self.ids.push(AssignedId::Counter(id));\n    }\n\n    pub fn push_change_id(&mut self, account_id: u32, change_id: u64) {\n        self.ids.push(AssignedId::ChangeId(ChangeId {\n            account_id,\n            change_id,\n        }));\n    }\n\n    pub fn last_change_id(&self, account_id: u32) -> trc::Result<u64> {\n        self.ids\n            .iter()\n            .filter_map(|id| match id {\n                AssignedId::ChangeId(change_id) if change_id.account_id == account_id => {\n                    Some(change_id.change_id)\n                }\n                _ => None,\n            })\n            .next_back()\n            .ok_or_else(|| {\n                trc::StoreEvent::UnexpectedError\n                    .caused_by(trc::location!())\n                    .ctx(trc::Key::Reason, \"No change ids were created\")\n            })\n    }\n\n    pub fn current_change_id(&self) -> trc::Result<u64> {\n        self.current_change_id.ok_or_else(|| {\n            trc::StoreEvent::UnexpectedError\n                .caused_by(trc::location!())\n                .ctx(trc::Key::Reason, \"No current change id is set\")\n        })\n    }\n\n    pub(crate) fn set_current_change_id(&mut self, account_id: u32) -> trc::Result<u64> {\n        let change_id = self.last_change_id(account_id)?;\n        self.current_change_id = Some(change_id);\n        Ok(change_id)\n    }\n\n    pub fn last_counter_id(&self) -> trc::Result<i64> {\n        self.ids\n            .iter()\n            .filter_map(|id| match id {\n                AssignedId::Counter(counter_id) => Some(*counter_id),\n                _ => None,\n            })\n            .next_back()\n            .ok_or_else(|| {\n                trc::StoreEvent::UnexpectedError\n                    .caused_by(trc::location!())\n                    .ctx(trc::Key::Reason, \"No counter ids were created\")\n            })\n    }\n}\n\nimpl QueueClass {\n    pub fn due(&self) -> Option<u64> {\n        match self {\n            QueueClass::DmarcReportHeader(report_event) => report_event.due.into(),\n            QueueClass::TlsReportHeader(report_event) => report_event.due.into(),\n            _ => None,\n        }\n    }\n}\n\nimpl<T: AsRef<[u8]>> AsRef<[u8]> for Archive<T> {\n    fn as_ref(&self) -> &[u8] {\n        self.inner.as_ref()\n    }\n}\n\nimpl ArchiveVersion {\n    pub fn hash(&self) -> Option<u32> {\n        match self {\n            ArchiveVersion::Versioned { hash, .. } => Some(*hash),\n            ArchiveVersion::Hashed { hash } => Some(*hash),\n            ArchiveVersion::Unversioned => None,\n        }\n    }\n\n    pub fn change_id(&self) -> Option<u64> {\n        match self {\n            ArchiveVersion::Versioned { change_id, .. } => Some(*change_id),\n            _ => None,\n        }\n    }\n}\n\nimpl From<LogCollection> for u8 {\n    fn from(value: LogCollection) -> Self {\n        match value {\n            LogCollection::Sync(col) => col as u8,\n            LogCollection::Vanished(col) => col as u8,\n        }\n    }\n}\n\nimpl From<ContactField> for ValueClass {\n    fn from(value: ContactField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<CalendarEventField> for ValueClass {\n    fn from(value: CalendarEventField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<CalendarNotificationField> for ValueClass {\n    fn from(value: CalendarNotificationField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<EmailField> for ValueClass {\n    fn from(value: EmailField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<MailboxField> for ValueClass {\n    fn from(value: MailboxField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<PrincipalField> for ValueClass {\n    fn from(value: PrincipalField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<SieveField> for ValueClass {\n    fn from(value: SieveField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<EmailSubmissionField> for ValueClass {\n    fn from(value: EmailSubmissionField) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl From<Field> for ValueClass {\n    fn from(value: Field) -> Self {\n        ValueClass::Property(value.into())\n    }\n}\n\nimpl PartialEq for MergeOperation {\n    fn eq(&self, other: &Self) -> bool {\n        self.params == other.params\n    }\n}\n\nimpl Eq for MergeOperation {}\n\nimpl PartialEq for SetOperation {\n    fn eq(&self, other: &Self) -> bool {\n        self.params == other.params\n    }\n}\n\nimpl Eq for SetOperation {}\n\nimpl Hash for MergeOperation {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.params.hash(state);\n    }\n}\n\nimpl Hash for SetOperation {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.params.hash(state);\n    }\n}\n\nimpl SetOperation {\n    pub fn params(&self) -> &Params {\n        &self.params\n    }\n}\n\nimpl MergeOperation {\n    pub fn params(&self) -> &Params {\n        &self.params\n    }\n}\n\nimpl Params {\n    pub fn with_capacity(capacity: usize) -> Self {\n        Self(Vec::with_capacity(capacity))\n    }\n\n    pub fn new() -> Self {\n        Self(Vec::new())\n    }\n\n    pub fn with_i64(mut self, value: i64) -> Self {\n        self.0.push(Param::I64(value));\n        self\n    }\n\n    pub fn with_u64(mut self, value: u64) -> Self {\n        self.0.push(Param::U64(value));\n        self\n    }\n\n    pub fn with_string(mut self, value: String) -> Self {\n        self.0.push(Param::String(value));\n        self\n    }\n\n    pub fn with_str(mut self, value: &str) -> Self {\n        self.0.push(Param::String(value.to_string()));\n        self\n    }\n\n    pub fn with_bytes(mut self, value: Vec<u8>) -> Self {\n        self.0.push(Param::Bytes(value));\n        self\n    }\n\n    pub fn with_bool(mut self, value: bool) -> Self {\n        self.0.push(Param::Bool(value));\n        self\n    }\n\n    pub fn i64(&self, idx: usize) -> i64 {\n        match &self.0[idx] {\n            Param::I64(v) => *v,\n            _ => panic!(\"Param at index {} is not an i64\", idx),\n        }\n    }\n\n    pub fn u64(&self, idx: usize) -> u64 {\n        match &self.0[idx] {\n            Param::U64(v) => *v,\n            _ => panic!(\"Param at index {} is not a u64\", idx),\n        }\n    }\n\n    pub fn string(&self, idx: usize) -> &str {\n        match &self.0[idx] {\n            Param::String(v) => v.as_str(),\n            _ => panic!(\"Param at index {} is not a String\", idx),\n        }\n    }\n\n    pub fn bytes(&self, idx: usize) -> &[u8] {\n        match &self.0[idx] {\n            Param::Bytes(v) => v.as_slice(),\n            _ => panic!(\"Param at index {} is not Bytes\", idx),\n        }\n    }\n\n    pub fn bool(&self, idx: usize) -> bool {\n        match &self.0[idx] {\n            Param::Bool(v) => *v,\n            _ => panic!(\"Param at index {} is not a bool\", idx),\n        }\n    }\n\n    pub fn len(&self) -> usize {\n        self.0.len()\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.0.is_empty()\n    }\n\n    pub fn as_slice(&self) -> &[Param] {\n        &self.0\n    }\n}\n\nimpl Default for Params {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n\nimpl AsRef<[Param]> for Params {\n    fn as_ref(&self) -> &[Param] {\n        &self.0\n    }\n}\n\nimpl TaskEpoch {\n    /*\n      Structure of the 64-bit epoch:\n       4 bytes: seconds since custom epoch (1632280000)\n       2 bytes: attempt number\n       2 bytes: sequence id\n    */\n\n    const EPOCH_OFFSET: u64 = 1632280000;\n\n    pub fn now() -> Self {\n        Self::new(now())\n    }\n\n    pub fn new(timestamp: u64) -> Self {\n        Self(timestamp.saturating_sub(Self::EPOCH_OFFSET) << 32)\n    }\n\n    pub fn with_attempt(mut self, attempt: u16) -> Self {\n        self.0 |= (attempt as u64) << 16;\n        self\n    }\n\n    pub fn with_sequence_id(mut self, sequence_id: u16) -> Self {\n        self.0 |= sequence_id as u64;\n        self\n    }\n\n    pub fn with_random_sequence_id(self) -> Self {\n        self.with_sequence_id(rand::random())\n    }\n\n    pub fn due(&self) -> u64 {\n        (self.0 >> 32) + Self::EPOCH_OFFSET\n    }\n\n    pub fn attempt(&self) -> u16 {\n        (self.0 >> 16) as u16\n    }\n\n    pub fn sequence_id(&self) -> u16 {\n        self.0 as u16\n    }\n\n    pub fn inner(&self) -> u64 {\n        self.0\n    }\n\n    pub fn from_inner(inner: u64) -> Self {\n        Self(inner)\n    }\n}\n"
  },
  {
    "path": "crates/store/src/write/serialize.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{ARCHIVE_ALIGNMENT, AlignedBytes, Archive, ArchiveVersion, Archiver};\nuse crate::{Deserialize, Serialize, SerializeInfallible, U32_LEN, U64_LEN, Value};\nuse compact_str::format_compact;\nuse rkyv::util::AlignedVec;\nuse roaring::{RoaringBitmap, RoaringTreemap};\n\nconst MAGIC_MARKER: u8 = 1 << 7;\nconst VERSIONED: u8 = 1 << 6;\nconst HASHED: u8 = 1 << 5;\nconst LZ4_COMPRESSED: u8 = 1 << 4;\n\nconst COMPRESS_WATERMARK: usize = 8192;\n\nfn validate_marker_and_contents(bytes: &[u8]) -> Option<(bool, &[u8], ArchiveVersion)> {\n    let (marker, contents) = bytes\n        .split_last()\n        .filter(|(marker, _)| (**marker & MAGIC_MARKER) != 0)?;\n    let is_uncompressed = (marker & LZ4_COMPRESSED) == 0;\n    if marker & VERSIONED != 0 {\n        let (contents, change_id) = contents\n            .split_at_checked(contents.len() - U64_LEN)\n            .and_then(|(contents, change_id)| {\n                change_id\n                    .try_into()\n                    .ok()\n                    .map(|change_id| (contents, u64::from_be_bytes(change_id)))\n            })?;\n        contents\n            .split_at_checked(contents.len() - U32_LEN)\n            .and_then(|(contents, archive_hash)| {\n                let hash = xxhash_rust::xxh3::xxh3_64(contents) as u32;\n                if hash.to_be_bytes().as_slice() == archive_hash {\n                    Some((\n                        is_uncompressed,\n                        contents,\n                        ArchiveVersion::Versioned { change_id, hash },\n                    ))\n                } else {\n                    None\n                }\n            })\n    } else if marker & HASHED != 0 {\n        contents\n            .split_at_checked(contents.len() - U32_LEN)\n            .and_then(|(contents, archive_hash)| {\n                let hash = xxhash_rust::xxh3::xxh3_64(contents) as u32;\n                if hash.to_be_bytes().as_slice() == archive_hash {\n                    Some((is_uncompressed, contents, ArchiveVersion::Hashed { hash }))\n                } else {\n                    None\n                }\n            })\n    } else {\n        Some((is_uncompressed, contents, ArchiveVersion::Unversioned))\n    }\n}\n\nimpl Deserialize for Archive<AlignedBytes> {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        let (is_uncompressed, contents, version) =\n            validate_marker_and_contents(bytes).ok_or_else(|| {\n                trc::StoreEvent::DataCorruption\n                    .into_err()\n                    .details(\"Archive integrity compromised\")\n                    .ctx(trc::Key::Value, bytes)\n                    .caused_by(trc::location!())\n            })?;\n\n        if is_uncompressed {\n            let mut bytes = AlignedVec::with_capacity(contents.len());\n            bytes.extend_from_slice(contents);\n            Ok(Archive {\n                version,\n                inner: AlignedBytes::Aligned(bytes),\n            })\n        } else {\n            aligned_lz4_deflate(contents).map(|inner| Archive { version, inner })\n        }\n    }\n\n    fn deserialize_owned(mut bytes: Vec<u8>) -> trc::Result<Self> {\n        let (is_uncompressed, contents, version) = validate_marker_and_contents(&bytes)\n            .ok_or_else(|| {\n                trc::StoreEvent::DataCorruption\n                    .into_err()\n                    .details(\"Archive integrity compromised\")\n                    .ctx(trc::Key::Value, bytes.as_slice())\n                    .caused_by(trc::location!())\n            })?;\n\n        if is_uncompressed {\n            bytes.truncate(contents.len());\n            if bytes.as_ptr().addr() & (ARCHIVE_ALIGNMENT - 1) == 0 {\n                Ok(Archive {\n                    version,\n                    inner: AlignedBytes::Vec(bytes),\n                })\n            } else {\n                let mut aligned = AlignedVec::with_capacity(bytes.len());\n                aligned.extend_from_slice(&bytes);\n                Ok(Archive {\n                    version,\n                    inner: AlignedBytes::Aligned(aligned),\n                })\n            }\n        } else {\n            aligned_lz4_deflate(contents).map(|inner| Archive { version, inner })\n        }\n    }\n}\n\n#[inline]\nfn aligned_lz4_deflate(archive: &[u8]) -> trc::Result<AlignedBytes> {\n    lz4_flex::block::uncompressed_size(archive)\n        .and_then(|(uncompressed_size, archive)| {\n            let mut bytes = AlignedVec::with_capacity(uncompressed_size);\n            unsafe {\n                // SAFETY: `new_len` is equal to `capacity` and vector is initialized by lz4_flex.\n                bytes.set_len(uncompressed_size);\n            }\n            lz4_flex::decompress_into(archive, &mut bytes)?;\n            Ok(AlignedBytes::Aligned(bytes))\n        })\n        .map_err(|err| {\n            trc::StoreEvent::DecompressError\n                .ctx(trc::Key::Value, archive)\n                .caused_by(trc::location!())\n                .reason(err)\n        })\n}\n\nimpl<T> Serialize for Archiver<T>\nwhere\n    T: rkyv::Archive\n        + for<'a> rkyv::Serialize<\n            rkyv::api::high::HighSerializer<\n                rkyv::util::AlignedVec,\n                rkyv::ser::allocator::ArenaHandle<'a>,\n                rkyv::rancor::Error,\n            >,\n        >,\n{\n    fn serialize(&self) -> trc::Result<Vec<u8>> {\n        rkyv::to_bytes::<rkyv::rancor::Error>(&self.inner)\n            .map_err(|err| {\n                trc::StoreEvent::DeserializeError\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n            .map(|input| {\n                let input = input.as_ref();\n                let input_len = input.len();\n                let version_offset = ((self.flags & VERSIONED != 0) as usize) * U64_LEN;\n                let mut bytes = if input_len > COMPRESS_WATERMARK {\n                    let mut bytes = vec![\n                        self.flags | LZ4_COMPRESSED;\n                        lz4_flex::block::get_maximum_output_size(input_len)\n                            + (U32_LEN * 2)\n                            + version_offset\n                            + 1\n                    ];\n\n                    // Compress the data\n                    let compressed_len =\n                        lz4_flex::compress_into(input, &mut bytes[U32_LEN..]).unwrap();\n\n                    if compressed_len < input_len {\n                        // Prepend the length of the uncompressed data\n                        bytes[..U32_LEN].copy_from_slice(&(input_len as u32).to_le_bytes());\n\n                        if self.flags & HASHED != 0 {\n                            // Hash the compressed data including the length\n                            let hash =\n                                xxhash_rust::xxh3::xxh3_64(&bytes[..compressed_len + U32_LEN])\n                                    as u32;\n\n                            // Add the hash\n                            bytes[compressed_len + U32_LEN..compressed_len + (U32_LEN * 2)]\n                                .copy_from_slice(&hash.to_be_bytes());\n\n                            // Truncate to the actual size\n                            bytes.truncate(compressed_len + (U32_LEN * 2) + version_offset + 1);\n                        } else {\n                            // Truncate to the actual size\n                            bytes.truncate(compressed_len + U32_LEN + 1);\n                        }\n\n                        return bytes;\n                    }\n                    bytes.clear();\n                    bytes\n                } else {\n                    Vec::with_capacity(input_len + U32_LEN + version_offset + 1)\n                };\n\n                bytes.extend_from_slice(input);\n                if self.flags & HASHED != 0 {\n                    bytes.extend_from_slice(\n                        &(xxhash_rust::xxh3::xxh3_64(input) as u32).to_be_bytes(),\n                    );\n                }\n                if version_offset != 0 {\n                    bytes.extend_from_slice(0u64.to_be_bytes().as_slice());\n                }\n                bytes.push(self.flags);\n                bytes\n            })\n    }\n}\n\nimpl Archive<AlignedBytes> {\n    #[inline]\n    pub fn as_bytes(&self) -> &[u8] {\n        match &self.inner {\n            AlignedBytes::Vec(bytes) => bytes.as_slice(),\n            AlignedBytes::Aligned(bytes) => bytes.as_slice(),\n        }\n    }\n\n    pub fn unarchive<T>(&self) -> trc::Result<&<T as rkyv::Archive>::Archived>\n    where\n        T: rkyv::Archive,\n        T::Archived: for<'a> rkyv::bytecheck::CheckBytes<\n                rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>,\n            > + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        let bytes = self.as_bytes();\n        if self.version != ArchiveVersion::Unversioned {\n            if bytes.len() >= std::mem::size_of::<T::Archived>() {\n                // SAFETY: Trusted input with integrity hash\n                Ok(unsafe { rkyv::access_unchecked::<T::Archived>(bytes) })\n            } else {\n                Err(trc::StoreEvent::DataCorruption\n                    .into_err()\n                    .details(format_compact!(\n                        \"Archive size mismatch, expected {} bytes but got {} bytes.\",\n                        std::mem::size_of::<T::Archived>(),\n                        bytes.len()\n                    ))\n                    .ctx(trc::Key::Value, bytes)\n                    .caused_by(trc::location!()))\n            }\n        } else {\n            rkyv::access::<T::Archived, rkyv::rancor::Error>(bytes).map_err(|err| {\n                trc::StoreEvent::DeserializeError\n                    .ctx(trc::Key::Value, self.as_bytes())\n                    .details(\"Archive access failed\")\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n        }\n    }\n\n    pub fn unarchive_untrusted<T>(&self) -> trc::Result<&<T as rkyv::Archive>::Archived>\n    where\n        T: rkyv::Archive,\n        T::Archived: for<'a> rkyv::bytecheck::CheckBytes<\n                rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>,\n            > + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        let bytes = self.as_bytes();\n        if bytes.len() >= std::mem::size_of::<T::Archived>() {\n            rkyv::access::<T::Archived, rkyv::rancor::Error>(bytes).map_err(|err| {\n                trc::StoreEvent::DeserializeError\n                    .ctx(trc::Key::Value, self.as_bytes())\n                    .details(\"Archive access failed\")\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n        } else {\n            Err(trc::StoreEvent::DataCorruption\n                .into_err()\n                .details(format_compact!(\n                    \"Archive size mismatch, expected {} bytes but got {} bytes.\",\n                    std::mem::size_of::<T::Archived>(),\n                    bytes.len()\n                ))\n                .ctx(trc::Key::Value, bytes)\n                .caused_by(trc::location!()))\n        }\n    }\n\n    pub fn deserialize<T>(&self) -> trc::Result<T>\n    where\n        T: rkyv::Archive,\n        T::Archived: for<'a> rkyv::bytecheck::CheckBytes<\n                rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>,\n            > + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        self.unarchive::<T>().and_then(|input| {\n            rkyv::deserialize(input).map_err(|err| {\n                trc::StoreEvent::DeserializeError\n                    .ctx(trc::Key::Value, self.as_bytes())\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n        })\n    }\n\n    pub fn deserialize_untrusted<T>(&self) -> trc::Result<T>\n    where\n        T: rkyv::Archive,\n        T::Archived: for<'a> rkyv::bytecheck::CheckBytes<\n                rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>,\n            > + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        self.unarchive_untrusted::<T>().and_then(|input| {\n            rkyv::deserialize(input).map_err(|err| {\n                trc::StoreEvent::DeserializeError\n                    .ctx(trc::Key::Value, self.as_bytes())\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n        })\n    }\n\n    pub fn to_unarchived<T>(&self) -> trc::Result<Archive<&<T as rkyv::Archive>::Archived>>\n    where\n        T: rkyv::Archive,\n        T::Archived: for<'a> rkyv::bytecheck::CheckBytes<\n                rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>,\n            > + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        self.unarchive::<T>().map(|inner| Archive {\n            version: self.version,\n            inner,\n        })\n    }\n\n    pub fn into_deserialized<T>(&self) -> trc::Result<Archive<T>>\n    where\n        T: rkyv::Archive,\n        T::Archived: for<'a> rkyv::bytecheck::CheckBytes<\n                rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>,\n            > + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        self.deserialize::<T>().map(|inner| Archive {\n            version: self.version,\n            inner,\n        })\n    }\n\n    pub fn into_inner(self) -> Vec<u8> {\n        let mut bytes = match self.inner {\n            AlignedBytes::Vec(bytes) => bytes,\n            AlignedBytes::Aligned(bytes) => bytes.to_vec(),\n        };\n        match self.version {\n            ArchiveVersion::Versioned { change_id, hash } => {\n                bytes.extend_from_slice(&change_id.to_be_bytes());\n                bytes.extend_from_slice(&hash.to_be_bytes());\n                bytes.push(MAGIC_MARKER | VERSIONED | HASHED);\n            }\n            ArchiveVersion::Hashed { hash } => {\n                bytes.extend_from_slice(&hash.to_be_bytes());\n                bytes.push(MAGIC_MARKER | HASHED);\n            }\n            ArchiveVersion::Unversioned => {\n                bytes.push(MAGIC_MARKER);\n            }\n        }\n        bytes\n    }\n\n    pub fn extract_hash(bytes: &[u8]) -> Option<u32> {\n        let marker = *bytes.last()?;\n        if marker & VERSIONED != 0 {\n            bytes\n                .get(bytes.len() - U32_LEN - U64_LEN - 1..bytes.len() - U64_LEN - 1)\n                .and_then(|slice| slice.try_into().ok().map(u32::from_be_bytes))\n        } else if marker & HASHED != 0 {\n            bytes\n                .get(bytes.len() - U32_LEN - 1..bytes.len() - 1)\n                .and_then(|slice| slice.try_into().ok().map(u32::from_be_bytes))\n        } else {\n            None\n        }\n    }\n}\n\nimpl<T> Archiver<T>\nwhere\n    T: rkyv::Archive\n        + for<'a> rkyv::Serialize<\n            rkyv::api::high::HighSerializer<\n                rkyv::util::AlignedVec,\n                rkyv::ser::allocator::ArenaHandle<'a>,\n                rkyv::rancor::Error,\n            >,\n        >,\n{\n    pub fn new(inner: T) -> Self {\n        Self {\n            inner,\n            flags: MAGIC_MARKER | HASHED,\n        }\n    }\n\n    pub fn into_inner(self) -> T {\n        self.inner\n    }\n\n    pub fn with_version(self) -> Self {\n        Self {\n            inner: self.inner,\n            flags: self.flags | VERSIONED,\n        }\n    }\n\n    pub fn untrusted(self) -> Self {\n        Self {\n            inner: self.inner,\n            flags: MAGIC_MARKER,\n        }\n    }\n\n    pub fn serialize_versioned(self) -> trc::Result<(u64, Vec<u8>)> {\n        self.with_version()\n            .serialize()\n            .map(|bytes| ((bytes.len() - U64_LEN - 1) as u64, bytes))\n    }\n}\n\nimpl<T> Archive<&T>\nwhere\n    T: rkyv::Portable\n        + for<'a> rkyv::bytecheck::CheckBytes<rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>>\n        + Sync\n        + Send,\n{\n    pub fn to_deserialized<V>(&self) -> trc::Result<Archive<V>>\n    where\n        T: rkyv::Deserialize<V, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        rkyv::deserialize::<V, rkyv::rancor::Error>(self.inner)\n            .map_err(|err| {\n                trc::StoreEvent::DeserializeError\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n            .map(|inner| Archive {\n                version: self.version,\n                inner,\n            })\n    }\n\n    pub fn deserialize<V>(&self) -> trc::Result<V>\n    where\n        T: rkyv::Deserialize<V, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n    {\n        rkyv::deserialize::<V, rkyv::rancor::Error>(self.inner).map_err(|err| {\n            trc::StoreEvent::DeserializeError\n                .caused_by(trc::location!())\n                .reason(err)\n        })\n    }\n}\n\n#[inline]\npub fn rkyv_deserialize<T, V>(input: &T) -> trc::Result<V>\nwhere\n    T: rkyv::Portable\n        + for<'a> rkyv::bytecheck::CheckBytes<rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>>\n        + Sync\n        + Send\n        + rkyv::Deserialize<V, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n{\n    rkyv::deserialize::<V, rkyv::rancor::Error>(input).map_err(|err| {\n        trc::StoreEvent::DeserializeError\n            .caused_by(trc::location!())\n            .reason(err)\n    })\n}\n\npub fn rkyv_unarchive<T>(input: &[u8]) -> trc::Result<&<T as rkyv::Archive>::Archived>\nwhere\n    T: rkyv::Archive,\n    T::Archived: for<'a> rkyv::bytecheck::CheckBytes<rkyv::api::high::HighValidator<'a, rkyv::rancor::Error>>\n        + rkyv::Deserialize<T, rkyv::api::high::HighDeserializer<rkyv::rancor::Error>>,\n{\n    rkyv::access::<T::Archived, rkyv::rancor::Error>(input).map_err(|err| {\n        trc::StoreEvent::DataCorruption\n            .caused_by(trc::location!())\n            .ctx(trc::Key::Value, input)\n            .reason(err)\n    })\n}\n\nimpl SerializeInfallible for u32 {\n    fn serialize(&self) -> Vec<u8> {\n        self.to_be_bytes().to_vec()\n    }\n}\n\nimpl SerializeInfallible for u64 {\n    fn serialize(&self) -> Vec<u8> {\n        self.to_be_bytes().to_vec()\n    }\n}\n\nimpl SerializeInfallible for i64 {\n    fn serialize(&self) -> Vec<u8> {\n        self.to_be_bytes().to_vec()\n    }\n}\n\nimpl SerializeInfallible for u16 {\n    fn serialize(&self) -> Vec<u8> {\n        self.to_be_bytes().to_vec()\n    }\n}\n\nimpl SerializeInfallible for f64 {\n    fn serialize(&self) -> Vec<u8> {\n        self.to_be_bytes().to_vec()\n    }\n}\n\nimpl SerializeInfallible for &str {\n    fn serialize(&self) -> Vec<u8> {\n        self.as_bytes().to_vec()\n    }\n}\n\nimpl Deserialize for String {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(String::from_utf8_lossy(bytes).into_owned())\n    }\n\n    fn deserialize_owned(bytes: Vec<u8>) -> trc::Result<Self> {\n        Ok(String::from_utf8(bytes)\n            .unwrap_or_else(|err| String::from_utf8_lossy(err.as_bytes()).into_owned()))\n    }\n}\n\nimpl Deserialize for u64 {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(u64::from_be_bytes(bytes.try_into().map_err(|_| {\n            trc::StoreEvent::DataCorruption.caused_by(trc::location!())\n        })?))\n    }\n}\n\nimpl Deserialize for i64 {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(i64::from_be_bytes(bytes.try_into().map_err(|_| {\n            trc::StoreEvent::DataCorruption.caused_by(trc::location!())\n        })?))\n    }\n}\n\nimpl Deserialize for u32 {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        Ok(u32::from_be_bytes(bytes.try_into().map_err(|_| {\n            trc::StoreEvent::DataCorruption.caused_by(trc::location!())\n        })?))\n    }\n}\n\nimpl Deserialize for () {\n    fn deserialize(_bytes: &[u8]) -> trc::Result<Self> {\n        Ok(())\n    }\n}\n\nimpl<T> From<Value<'static>> for Archive<T> {\n    fn from(_: Value<'static>) -> Self {\n        unimplemented!()\n    }\n}\n\nimpl Default for Archive<AlignedBytes> {\n    fn default() -> Self {\n        Archive {\n            version: ArchiveVersion::Unversioned,\n            inner: AlignedBytes::Aligned(AlignedVec::new()),\n        }\n    }\n}\n\nimpl Serialize for RoaringBitmap {\n    fn serialize(&self) -> trc::Result<Vec<u8>> {\n        let mut bytes = Vec::with_capacity(self.serialized_size());\n        self.serialize_into(&mut bytes)\n            .map_err(|err| {\n                trc::StoreEvent::UnexpectedError\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n            .map(|_| bytes)\n    }\n}\n\nimpl Deserialize for RoaringBitmap {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        RoaringBitmap::deserialize_from(bytes).map_err(|err| {\n            trc::StoreEvent::DeserializeError\n                .caused_by(trc::location!())\n                .reason(err)\n        })\n    }\n}\n\nimpl Serialize for RoaringTreemap {\n    fn serialize(&self) -> trc::Result<Vec<u8>> {\n        let mut bytes = Vec::with_capacity(self.serialized_size());\n        self.serialize_into(&mut bytes)\n            .map_err(|err| {\n                trc::StoreEvent::UnexpectedError\n                    .caused_by(trc::location!())\n                    .reason(err)\n            })\n            .map(|_| bytes)\n    }\n}\n\nimpl Deserialize for RoaringTreemap {\n    fn deserialize(bytes: &[u8]) -> trc::Result<Self> {\n        RoaringTreemap::deserialize_from(bytes).map_err(|err| {\n            trc::StoreEvent::DeserializeError\n                .caused_by(trc::location!())\n                .reason(err)\n        })\n    }\n}\n"
  },
  {
    "path": "crates/trc/Cargo.toml",
    "content": "[package]\nname = \"trc\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nevent_macro = { path = \"./event-macro\" }\nmail-auth = { version = \"0.7.1\" }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\"] } \nbase64 = \"0.22.1\"\nserde = \"1.0\"\nserde_json = \"1.0.120\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\"]}\nrtrb = \"0.3.1\"\nparking_lot = \"0.12.3\"\ntokio = { version = \"1.47\", features = [\"net\", \"macros\"] }\nahash = \"0.8.11\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\n\n[features]\ntest_mode = []\nenterprise = []\n\n[dev-dependencies]\n"
  },
  {
    "path": "crates/trc/event-macro/Cargo.toml",
    "content": "[package]\nname = \"event_macro\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[lib]\nproc-macro = true\n\n[dependencies]\nsyn = { version = \"2.0\", features = [\"full\"] }\nquote = \"1.0\"\nproc-macro2 = \"1.0\"\n"
  },
  {
    "path": "crates/trc/event-macro/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse proc_macro::TokenStream;\nuse quote::quote;\nuse syn::{\n    Data, DeriveInput, Expr, ExprPath, Fields, Ident, Token, parse::Parse, parse_macro_input,\n};\n\nstatic mut GLOBAL_ID_COUNTER: usize = 0;\n\n#[proc_macro_attribute]\npub fn event_type(_attr: TokenStream, item: TokenStream) -> TokenStream {\n    let input = parse_macro_input!(item as DeriveInput);\n    let name = &input.ident;\n    let name_str = name.to_string();\n    let prefix = to_snake_case(name_str.strip_suffix(\"Event\").unwrap_or(&name_str));\n\n    let enum_variants = match &input.data {\n        Data::Enum(data_enum) => &data_enum.variants,\n        _ => panic!(\"This macro only works with enums\"),\n    };\n\n    let mut variant_ids = Vec::new();\n    let mut variant_names = Vec::new();\n    let mut event_names = Vec::new();\n\n    for variant in enum_variants {\n        unsafe {\n            variant_ids.push(GLOBAL_ID_COUNTER);\n            GLOBAL_ID_COUNTER += 1;\n        }\n        let variant_name = &variant.ident;\n        event_names.push(format!(\n            \"{prefix}.{}\",\n            to_snake_case(&variant_name.to_string())\n        ));\n        variant_names.push(variant_name);\n    }\n\n    let id_fn = quote! {\n        pub const fn id(&self) -> usize {\n            match self {\n                #(Self::#variant_names => #variant_ids,)*\n            }\n        }\n    };\n\n    let name_fn = quote! {\n        pub fn name(&self) -> &'static str {\n            match self {\n                #(Self::#variant_names => #event_names,)*\n            }\n        }\n    };\n\n    let parse_fn = quote! {\n        pub fn try_parse(name: &str) -> Option<Self> {\n            match name {\n                #(#event_names => Some(Self::#variant_names),)*\n                _ => None,\n            }\n        }\n    };\n\n    let variants_fn = quote! {\n        pub const fn variants() -> &'static [Self] {\n            &[\n                #(#name::#variant_names,)*\n            ]\n        }\n    };\n\n    let expanded = quote! {\n        #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\n        pub enum #name {\n            #(#variant_names),*\n        }\n\n        impl #name {\n            #id_fn\n            #name_fn\n            #parse_fn\n            #variants_fn\n        }\n    };\n\n    TokenStream::from(expanded)\n}\n\n#[proc_macro_attribute]\npub fn event_family(_attr: TokenStream, item: TokenStream) -> TokenStream {\n    let input = parse_macro_input!(item as DeriveInput);\n    let name = &input.ident;\n\n    let variants = match &input.data {\n        Data::Enum(data_enum) => &data_enum.variants,\n        _ => panic!(\"EventType must be an enum\"),\n    };\n\n    let variant_idents: Vec<_> = variants.iter().map(|v| &v.ident).collect();\n\n    let event_types: Vec<_> = variants\n        .iter()\n        .map(|v| match &v.fields {\n            Fields::Unnamed(fields) => &fields.unnamed[0],\n            _ => panic!(\"EventType variants must be unnamed and contain a single type\"),\n        })\n        .map(|f| &f.ty)\n        .collect();\n\n    let variant_names: Vec<_> = variant_idents\n        .iter()\n        .map(|ident| {\n            let name_str = ident.to_string();\n            to_snake_case(name_str.strip_suffix(\"Event\").unwrap_or(&name_str))\n        })\n        .collect();\n\n    let expanded = quote! {\n        pub enum #name {\n            #(#variant_idents(#event_types)),*\n        }\n\n        impl #name {\n            pub const fn id(&self) -> usize {\n                match self {\n                    #(#name::#variant_idents(e) => e.id()),*\n                }\n            }\n\n            pub fn name(&self) -> &'static str {\n                match self {\n                    #(#name::#variant_idents(e) => e.name()),*\n                }\n            }\n\n            pub fn try_parse(name: &str) -> Option<Self> {\n                match name.trim().split_once('.')?.0 {\n                #(\n                    #variant_names =>  <#event_types>::try_parse(&name).map(#name::#variant_idents),\n                )*\n                    _ => None,\n                }\n            }\n\n            pub const fn variants() -> [#name; crate::TOTAL_EVENT_COUNT] {\n                let mut variants = [crate::EventType::Eval(crate::EvalEvent::Error); crate::TOTAL_EVENT_COUNT];\n                #(\n                    {\n                        let sub_variants = <#event_types>::variants();\n                        let mut i = 0;\n                        while i < sub_variants.len() {\n                            variants[sub_variants[i].id()] = #name::#variant_idents(sub_variants[i]);\n                            i += 1;\n                        }\n\n                    }\n                )*\n                variants\n            }\n        }\n    };\n\n    TokenStream::from(expanded)\n}\n\n#[proc_macro_attribute]\npub fn key_names(_attr: TokenStream, item: TokenStream) -> TokenStream {\n    let input = parse_macro_input!(item as DeriveInput);\n    let name = &input.ident;\n\n    let enum_variants = match &input.data {\n        Data::Enum(data_enum) => &data_enum.variants,\n        _ => panic!(\"This macro only works with enums\"),\n    };\n\n    let mut variant_names = Vec::new();\n    let mut camel_case_names = Vec::new();\n    let mut snake_case_names = Vec::new();\n\n    for variant in enum_variants.iter() {\n        let variant_name = &variant.ident;\n        variant_names.push(variant_name);\n        snake_case_names.push(to_snake_case(&variant_name.to_string()));\n        camel_case_names.push(\n            variant_name\n                .to_string()\n                .char_indices()\n                .map(|(i, c)| if i == 0 { c.to_ascii_lowercase() } else { c })\n                .collect::<String>(),\n        );\n    }\n\n    let id_fn = quote! {\n        pub fn id(&self) -> &'static str {\n            match self {\n                #(Self::#variant_names => #snake_case_names,)*\n            }\n        }\n    };\n\n    let name_fn = quote! {\n        pub fn name(&self) -> &'static str {\n            match self {\n                #(Self::#variant_names => #camel_case_names,)*\n            }\n        }\n    };\n\n    let parse_fn = quote! {\n        pub fn try_parse(name: &str) -> Option<Self> {\n            match name {\n                #(#snake_case_names => Some(Self::#variant_names),)*\n                _ => None,\n            }\n        }\n    };\n\n    let expanded = quote! {\n        #input\n\n        impl #name {\n            #name_fn\n            #id_fn\n            #parse_fn\n        }\n    };\n\n    TokenStream::from(expanded)\n}\n\n#[proc_macro]\npub fn total_event_count(_item: TokenStream) -> TokenStream {\n    let count = unsafe { GLOBAL_ID_COUNTER };\n    let expanded = quote! {\n        #count\n    };\n    TokenStream::from(expanded)\n}\n\nfn to_snake_case(name: &str) -> String {\n    let mut out = String::with_capacity(name.len());\n    for (idx, ch) in name.char_indices() {\n        if ch.is_ascii_uppercase() {\n            if idx > 0 {\n                out.push('-');\n            }\n            out.push(ch.to_ascii_lowercase());\n        } else {\n            out.push(ch);\n        }\n    }\n    out\n}\n\nstruct EventMacroInput {\n    event: Ident,\n    param: Expr,\n    key_values: Vec<(Ident, Expr)>,\n}\n\nimpl Parse for EventMacroInput {\n    fn parse(input: syn::parse::ParseStream) -> syn::Result<Self> {\n        let event: Ident = input.parse()?;\n        let content;\n        syn::parenthesized!(content in input);\n        let param: Expr = content.parse()?;\n\n        let mut key_values = Vec::new();\n        while !input.is_empty() {\n            input.parse::<Token![,]>()?;\n            if input.is_empty() {\n                break;\n            }\n            let key: Ident = input.parse()?;\n            input.parse::<Token![=]>()?;\n            let value: Expr = input.parse()?;\n            key_values.push((key, value));\n        }\n\n        Ok(EventMacroInput {\n            event,\n            param,\n            key_values,\n        })\n    }\n}\n\n#[proc_macro]\npub fn event(input: TokenStream) -> TokenStream {\n    let EventMacroInput {\n        event,\n        param,\n        key_values,\n    } = parse_macro_input!(input as EventMacroInput);\n\n    let key_value_tokens = key_values.iter().map(|(key, value)| {\n        quote! {\n            (trc::Key::#key, trc::Value::from(#value))\n        }\n    });\n    // This avoids having to evaluate expensive values when we know we are not interested in the event\n    let key_value_metric_tokens = key_values.iter().filter_map(|(key, value)| {\n        if key.is_metric_key() {\n            Some(quote! {\n                (trc::Key::#key, trc::Value::from(#value))\n            })\n        } else {\n            None\n        }\n    });\n\n    let expanded = if matches!(&param, Expr::Path(ExprPath { path, .. })  if path.segments.len() > 1 && path.segments.last().unwrap().arguments.is_empty() )\n    {\n        quote! {{\n            const ET: trc::EventType = trc::EventType::#event(#param);\n            const ET_ID: usize = ET.id();\n            if trc::Collector::has_interest(ET_ID) {\n                let keys = vec![#(#key_value_tokens),*];\n                if trc::Collector::is_metric(ET_ID) {\n                    trc::Collector::record_metric(ET, ET_ID, &keys);\n                }\n                trc::Event::with_keys(ET, keys).send();\n            } else if trc::Collector::is_metric(ET_ID) {\n                trc::Collector::record_metric(ET, ET_ID, &[#(#key_value_metric_tokens),*]);\n            }\n        }}\n    } else {\n        quote! {{\n            let et = trc::EventType::#event(#param);\n            let et_id = et.id();\n            if trc::Collector::has_interest(et_id) {\n                let keys = vec![#(#key_value_tokens),*];\n                if trc::Collector::is_metric(et_id) {\n                    trc::Collector::record_metric(et, et_id, &keys);\n                }\n                trc::Event::with_keys(et, keys).send();\n            } else if trc::Collector::is_metric(et_id) {\n                trc::Collector::record_metric(et, et_id, &[#(#key_value_metric_tokens),*]);\n            }\n        }}\n    };\n\n    TokenStream::from(expanded)\n}\n\ntrait IsMetricKey {\n    fn is_metric_key(&self) -> bool;\n}\n\nimpl IsMetricKey for Ident {\n    fn is_metric_key(&self) -> bool {\n        matches!(\n            self.to_string().as_ref(),\n            \"Total\"\n                | \"Elapsed\"\n                | \"Size\"\n                | \"TotalSuccesses\"\n                | \"TotalFailures\"\n                | \"DmarcPass\"\n                | \"DmarcQuarantine\"\n                | \"DmarcReject\"\n                | \"DmarcNone\"\n                | \"DkimPass\"\n                | \"DkimFail\"\n                | \"DkimNone\"\n                | \"SpfPass\"\n                | \"SpfFail\"\n                | \"SpfNone\"\n                | \"Protocol\"\n                | \"Code\"\n        )\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/atomics/array.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::atomic::{AtomicU32, AtomicU64, Ordering};\n\npub struct AtomicU32Array<const N: usize>([AtomicU32; N]);\npub struct AtomicU64Array<const N: usize>([AtomicU64; N]);\n\nimpl<const N: usize> AtomicU32Array<N> {\n    #[allow(clippy::new_without_default)]\n    #[allow(clippy::declare_interior_mutable_const)]\n    pub const fn new() -> Self {\n        Self({\n            const INIT: AtomicU32 = AtomicU32::new(0);\n            let mut array = [INIT; N];\n            let mut i = 0;\n            while i < N {\n                array[i] = AtomicU32::new(0);\n                i += 1;\n            }\n            array\n        })\n    }\n\n    #[inline(always)]\n    pub fn get(&self, index: usize) -> u32 {\n        self.0[index].load(Ordering::Relaxed)\n    }\n\n    #[inline(always)]\n    pub fn set(&self, index: usize, value: u32) {\n        self.0[index].store(value, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn add(&self, index: usize, value: u32) {\n        self.0[index].fetch_add(value, Ordering::Relaxed);\n    }\n\n    pub fn inner(&self) -> &[AtomicU32; N] {\n        &self.0\n    }\n}\n\nimpl<const N: usize> AtomicU64Array<N> {\n    #[allow(clippy::new_without_default)]\n    #[allow(clippy::declare_interior_mutable_const)]\n    pub const fn new() -> Self {\n        Self({\n            const INIT: AtomicU64 = AtomicU64::new(0);\n            let mut array = [INIT; N];\n            let mut i = 0;\n            while i < N {\n                array[i] = AtomicU64::new(0);\n                i += 1;\n            }\n            array\n        })\n    }\n\n    #[inline(always)]\n    pub fn get(&self, index: usize) -> u64 {\n        self.0[index].load(Ordering::Relaxed)\n    }\n\n    #[inline(always)]\n    pub fn set(&self, index: usize, value: u64) {\n        self.0[index].store(value, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn add(&self, index: usize, value: u64) {\n        self.0[index].fetch_add(value, Ordering::Relaxed);\n    }\n\n    pub fn inner(&self) -> &[AtomicU64; N] {\n        &self.0\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/atomics/bitset.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::atomic::{AtomicUsize, Ordering};\n\nuse crate::ipc::{USIZE_BITS, USIZE_BITS_MASK, bitset::Bitset};\n\npub struct AtomicBitset<const N: usize>([AtomicUsize; N]);\n\nimpl<const N: usize> AtomicBitset<N> {\n    #[allow(clippy::new_without_default)]\n    #[allow(clippy::declare_interior_mutable_const)]\n    pub const fn new() -> Self {\n        Self({\n            const INIT: AtomicUsize = AtomicUsize::new(0);\n            let mut array = [INIT; N];\n            let mut i = 0;\n            while i < N {\n                array[i] = AtomicUsize::new(0);\n                i += 1;\n            }\n            array\n        })\n    }\n\n    #[inline(always)]\n    pub fn set(&self, index: impl Into<usize>) {\n        let index = index.into();\n        self.0[index / USIZE_BITS].fetch_or(1 << (index & USIZE_BITS_MASK), Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn clear(&self, index: impl Into<usize>) {\n        let index = index.into();\n        self.0[index / USIZE_BITS].fetch_and(!(1 << (index & USIZE_BITS_MASK)), Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn get(&self, index: impl Into<usize>) -> bool {\n        let index = index.into();\n        self.0[index / USIZE_BITS].load(Ordering::Relaxed) & (1 << (index & USIZE_BITS_MASK)) != 0\n    }\n\n    pub fn update(&self, bitset: impl AsRef<Bitset<N>>) {\n        let bitset = bitset.as_ref();\n        for i in 0..N {\n            self.0[i].store(bitset.0[i], Ordering::Relaxed);\n        }\n    }\n\n    pub fn union(&self, bitset: impl AsRef<Bitset<N>>) {\n        let bitset = bitset.as_ref();\n        for i in 0..N {\n            self.0[i].fetch_or(bitset.0[i], Ordering::Relaxed);\n        }\n    }\n\n    pub fn clear_all(&self) {\n        for i in 0..N {\n            self.0[i].store(0, Ordering::Relaxed);\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        for i in 0..N {\n            if self.0[i].load(Ordering::Relaxed) != 0 {\n                return false;\n            }\n        }\n        true\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    const TEST_SIZE: usize = 1000;\n    type TestBitset = AtomicBitset<{ TEST_SIZE.div_ceil(USIZE_BITS) }>;\n    static BITSET: TestBitset = TestBitset::new();\n\n    #[test]\n    fn test_atomic_bitset() {\n        for i in 0..TEST_SIZE {\n            assert!(!BITSET.get(i), \"Bit {} should be unset in new BITSET\", i);\n        }\n\n        for i in 0..TEST_SIZE {\n            assert!(!BITSET.get(i), \"Bit {} should be initially unset\", i);\n            BITSET.set(i);\n            assert!(BITSET.get(i), \"Bit {} should be set after setting\", i);\n        }\n\n        BITSET.clear_all();\n\n        for i in 0..TEST_SIZE {\n            BITSET.set(i);\n            assert!(BITSET.get(i), \"Bit {} should be set before clearing\", i);\n            BITSET.clear(i);\n            assert!(!BITSET.get(i), \"Bit {} should be unset after clearing\", i);\n        }\n\n        BITSET.clear_all();\n\n        // Set even bits\n        for i in (0..TEST_SIZE).step_by(2) {\n            BITSET.set(i);\n        }\n\n        // Check all bits\n        for i in 0..TEST_SIZE {\n            if i % 2 == 0 {\n                assert!(BITSET.get(i), \"Even bit {} should be set\", i);\n            } else {\n                assert!(!BITSET.get(i), \"Odd bit {} should be unset\", i);\n            }\n        }\n\n        // Clear even bits and set odd bits\n        for i in 0..TEST_SIZE {\n            if i % 2 == 0 {\n                BITSET.clear(i);\n            } else {\n                BITSET.set(i);\n            }\n        }\n\n        // Check all bits again\n        for i in 0..TEST_SIZE {\n            if i % 2 == 0 {\n                assert!(!BITSET.get(i), \"Even bit {} should now be unset\", i);\n            } else {\n                assert!(BITSET.get(i), \"Odd bit {} should now be set\", i);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/atomics/counter.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::atomic::{AtomicU64, Ordering};\n\npub struct AtomicCounter {\n    id: &'static str,\n    description: &'static str,\n    unit: &'static str,\n    value: AtomicU64,\n}\n\nimpl AtomicCounter {\n    pub const fn new(id: &'static str, description: &'static str, unit: &'static str) -> Self {\n        Self {\n            id,\n            description,\n            unit,\n            value: AtomicU64::new(0),\n        }\n    }\n\n    #[inline(always)]\n    pub fn increment(&self) {\n        self.value.fetch_add(1, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn increment_by(&self, value: u64) {\n        self.value.fetch_add(value, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn decrement(&self) {\n        self.value.fetch_sub(1, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn decrement_by(&self, value: u64) {\n        self.value.fetch_sub(value, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn get(&self) -> u64 {\n        self.value.load(Ordering::Relaxed)\n    }\n\n    pub fn id(&self) -> &'static str {\n        self.id\n    }\n\n    pub fn description(&self) -> &'static str {\n        self.description\n    }\n\n    pub fn unit(&self) -> &'static str {\n        self.unit\n    }\n\n    pub fn is_active(&self) -> bool {\n        self.value.load(Ordering::Relaxed) > 0\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/atomics/gauge.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::atomic::{AtomicU64, Ordering};\n\nuse crate::MetricType;\n\npub struct AtomicGauge {\n    id: MetricType,\n    value: AtomicU64,\n}\n\nimpl AtomicGauge {\n    pub const fn new(id: MetricType) -> Self {\n        Self {\n            id,\n            value: AtomicU64::new(0),\n        }\n    }\n\n    #[inline(always)]\n    pub fn increment(&self) {\n        self.value.fetch_add(1, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn set(&self, value: u64) {\n        self.value.store(value, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn decrement(&self) {\n        self.value.fetch_sub(1, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn get(&self) -> u64 {\n        self.value.load(Ordering::Relaxed)\n    }\n\n    #[inline(always)]\n    pub fn add(&self, value: u64) {\n        self.value.fetch_add(value, Ordering::Relaxed);\n    }\n\n    #[inline(always)]\n    pub fn subtract(&self, value: u64) {\n        self.value.fetch_sub(value, Ordering::Relaxed);\n    }\n\n    pub fn id(&self) -> MetricType {\n        self.id\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/atomics/histogram.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::atomic::{AtomicU64, Ordering};\n\nuse crate::MetricType;\n\nuse super::array::AtomicU32Array;\n\npub struct AtomicHistogram<const N: usize> {\n    id: MetricType,\n    buckets: AtomicU32Array<N>,\n    upper_bounds: [u64; N],\n    sum: AtomicU64,\n    count: AtomicU64,\n    min: AtomicU64,\n    max: AtomicU64,\n}\n\nimpl<const N: usize> AtomicHistogram<N> {\n    pub const fn new(id: MetricType, upper_bounds: [u64; N]) -> Self {\n        Self {\n            buckets: AtomicU32Array::new(),\n            upper_bounds,\n            sum: AtomicU64::new(0),\n            count: AtomicU64::new(0),\n            min: AtomicU64::new(u64::MAX),\n            max: AtomicU64::new(0),\n            id,\n        }\n    }\n\n    pub fn observe(&self, value: u64) {\n        self.sum.fetch_add(value, Ordering::Relaxed);\n        self.count.fetch_add(1, Ordering::Relaxed);\n        self.min.fetch_min(value, Ordering::Relaxed);\n        self.max.fetch_max(value, Ordering::Relaxed);\n\n        for (idx, upper_bound) in self.upper_bounds.iter().enumerate() {\n            if value < *upper_bound {\n                self.buckets.add(idx, 1);\n                return;\n            }\n        }\n\n        unreachable!()\n    }\n\n    pub fn id(&self) -> MetricType {\n        self.id\n    }\n\n    pub fn sum(&self) -> u64 {\n        self.sum.load(Ordering::Relaxed)\n    }\n\n    pub fn count(&self) -> u64 {\n        self.count.load(Ordering::Relaxed)\n    }\n\n    pub fn average(&self) -> f64 {\n        let sum = self.sum();\n        let count = self.count();\n        if count > 0 {\n            sum as f64 / count as f64\n        } else {\n            0.0\n        }\n    }\n\n    pub fn min(&self) -> Option<u64> {\n        let min = self.min.load(Ordering::Relaxed);\n        if min != u64::MAX { Some(min) } else { None }\n    }\n\n    pub fn max(&self) -> Option<u64> {\n        let max = self.max.load(Ordering::Relaxed);\n        if max != 0 { Some(max) } else { None }\n    }\n\n    pub fn buckets_iter(&self) -> impl IntoIterator<Item = u64> + '_ {\n        self.buckets\n            .inner()\n            .iter()\n            .map(|bucket| bucket.load(Ordering::Relaxed) as u64)\n    }\n\n    pub fn buckets_vec(&self) -> Vec<u64> {\n        let mut vec = Vec::with_capacity(N);\n        for bucket in self.buckets.inner().iter() {\n            vec.push(bucket.load(Ordering::Relaxed) as u64);\n        }\n        vec\n    }\n\n    pub fn buckets_len(&self) -> usize {\n        N\n    }\n\n    pub fn upper_bounds_iter(&self) -> impl IntoIterator<Item = u64> + '_ {\n        self.upper_bounds.iter().copied()\n    }\n\n    pub fn upper_bounds_vec(&self) -> Vec<f64> {\n        let mut vec = Vec::with_capacity(N - 1);\n        for upper_bound in self.upper_bounds.iter().take(N - 1) {\n            vec.push(*upper_bound as f64);\n        }\n        vec\n    }\n\n    pub fn is_active(&self) -> bool {\n        self.count.load(Ordering::Relaxed) > 0\n    }\n\n    pub const fn new_message_sizes(id: MetricType) -> AtomicHistogram<12> {\n        AtomicHistogram::new(\n            id,\n            [\n                500,         // 500 bytes\n                1_000,       // 1 KB\n                10_000,      // 10 KB\n                100_000,     // 100 KB\n                1_000_000,   // 1 MB\n                5_000_000,   // 5 MB\n                10_000_000,  // 10 MB\n                25_000_000,  // 25 MB\n                50_000_000,  // 50 MB\n                100_000_000, // 100 MB\n                500_000_000, // 500 MB\n                u64::MAX,    // Catch-all for any larger sizes\n            ],\n        )\n    }\n\n    pub const fn new_short_durations(id: MetricType) -> AtomicHistogram<12> {\n        AtomicHistogram::new(\n            id,\n            [\n                5,        // 5 milliseconds\n                10,       // 10 milliseconds\n                50,       // 50 milliseconds\n                100,      // 100 milliseconds\n                500,      // 0.5 seconds\n                1_000,    // 1 second\n                2_000,    // 2 seconds\n                5_000,    // 5 seconds\n                10_000,   // 10 seconds\n                30_000,   // 30 seconds\n                60_000,   // 1 minute\n                u64::MAX, // Catch-all for any longer durations\n            ],\n        )\n    }\n\n    pub const fn new_medium_durations(id: MetricType) -> AtomicHistogram<12> {\n        AtomicHistogram::new(\n            id,\n            [\n                250,\n                500,\n                1_000,\n                5_000,\n                10_000, // For quick connections (seconds)\n                60_000,\n                (60 * 5) * 1_000,\n                (60 * 10) * 1_000,\n                (60 * 30) * 1_000, // For medium-length connections (minutes)\n                (60 * 60) * 1_000,\n                (60 * 60 * 5) * 1_000,\n                u64::MAX, // For extreme cases (8 hours and 1 day)\n            ],\n        )\n    }\n\n    pub const fn new_long_durations(id: MetricType) -> AtomicHistogram<12> {\n        AtomicHistogram::new(\n            id,\n            [\n                1_000,       // 1 second\n                30_000,      // 30 seconds\n                300_000,     // 5 minutes\n                600_000,     // 10 minutes\n                1_800_000,   // 30 minutes\n                3_600_000,   // 1 hour\n                14_400_000,  // 5 hours\n                28_800_000,  // 8 hours\n                43_200_000,  // 12 hours\n                86_400_000,  // 1 day\n                604_800_000, // 1 week\n                u64::MAX,    // Catch-all for any longer durations\n            ],\n        )\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/atomics/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod array;\npub mod bitset;\npub mod counter;\npub mod gauge;\npub mod histogram;\n"
  },
  {
    "path": "crates/trc/src/event/conv.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, fmt::Debug, str::FromStr, time::Duration};\n\nuse compact_str::{CompactString, ToCompactString, format_compact};\nuse mail_auth::common::verify::VerifySignature;\n\nuse crate::*;\n\nimpl AsRef<EventType> for Error {\n    fn as_ref(&self) -> &EventType {\n        &self.0.inner\n    }\n}\n\nimpl From<&'static str> for Value {\n    fn from(value: &'static str) -> Self {\n        Self::String(CompactString::const_new(value))\n    }\n}\n\nimpl From<String> for Value {\n    fn from(value: String) -> Self {\n        Self::String(CompactString::from_string_buffer(value))\n    }\n}\n\nimpl From<CompactString> for Value {\n    fn from(value: CompactString) -> Self {\n        Self::String(value)\n    }\n}\n\nimpl From<Box<str>> for Value {\n    fn from(value: Box<str>) -> Self {\n        Self::String(CompactString::from(value))\n    }\n}\n\nimpl From<u64> for Value {\n    fn from(value: u64) -> Self {\n        Self::UInt(value)\n    }\n}\n\nimpl From<i64> for Value {\n    fn from(value: i64) -> Self {\n        Self::Int(value)\n    }\n}\n\nimpl From<f64> for Value {\n    fn from(value: f64) -> Self {\n        Self::Float(value)\n    }\n}\n\nimpl From<f32> for Value {\n    fn from(value: f32) -> Self {\n        Self::Float(value.into())\n    }\n}\n\nimpl From<u16> for Value {\n    fn from(value: u16) -> Self {\n        Self::UInt(value.into())\n    }\n}\n\nimpl From<i32> for Value {\n    fn from(value: i32) -> Self {\n        Self::Int(value.into())\n    }\n}\n\nimpl From<u32> for Value {\n    fn from(value: u32) -> Self {\n        Self::UInt(value.into())\n    }\n}\n\nimpl From<usize> for Value {\n    fn from(value: usize) -> Self {\n        Self::UInt(value as u64)\n    }\n}\n\nimpl From<bool> for Value {\n    fn from(value: bool) -> Self {\n        Self::Bool(value)\n    }\n}\n\nimpl From<IpAddr> for Value {\n    fn from(value: IpAddr) -> Self {\n        match value {\n            IpAddr::V4(ip) => Value::Ipv4(ip),\n            IpAddr::V6(ip) => Value::Ipv6(ip),\n        }\n    }\n}\n\nimpl<T: Into<Value>> From<Option<T>> for Value {\n    fn from(value: Option<T>) -> Self {\n        match value {\n            Some(value) => value.into(),\n            None => Self::None,\n        }\n    }\n}\n\nimpl From<Duration> for Value {\n    fn from(value: Duration) -> Self {\n        Self::Duration(value.as_millis() as u64)\n    }\n}\n\nimpl From<Error> for Value {\n    fn from(value: Error) -> Self {\n        Self::Event(value)\n    }\n}\n\nimpl From<EventType> for Error {\n    fn from(value: EventType) -> Self {\n        Error::new(value)\n    }\n}\n\nimpl From<StoreEvent> for Error {\n    fn from(value: StoreEvent) -> Self {\n        Error::new(EventType::Store(value))\n    }\n}\n\nimpl From<AuthEvent> for Error {\n    fn from(value: AuthEvent) -> Self {\n        Error::new(EventType::Auth(value))\n    }\n}\n\nimpl From<Vec<u8>> for Value {\n    fn from(value: Vec<u8>) -> Self {\n        Self::Bytes(value)\n    }\n}\n\nimpl From<&[u8]> for Value {\n    fn from(value: &[u8]) -> Self {\n        Self::Bytes(value.to_vec())\n    }\n}\n\nimpl From<Cow<'static, str>> for Value {\n    fn from(value: Cow<'static, str>) -> Self {\n        match value {\n            Cow::Borrowed(value) => Self::String(CompactString::const_new(value)),\n            Cow::Owned(value) => Self::String(value.into()),\n        }\n    }\n}\n\nimpl<T> From<&crate::Result<T>> for Value\nwhere\n    T: Debug,\n{\n    fn from(value: &crate::Result<T>) -> Self {\n        match value {\n            Ok(value) => format_compact!(\"{:?}\", value).into(),\n            Err(err) => Value::Event(err.clone()),\n        }\n    }\n}\n\nimpl<T> From<Vec<T>> for Value\nwhere\n    T: Into<Value>,\n{\n    fn from(value: Vec<T>) -> Self {\n        Self::Array(value.into_iter().map(Into::into).collect())\n    }\n}\n\nimpl<T> From<&[T]> for Value\nwhere\n    T: Into<Value> + Clone,\n{\n    fn from(value: &[T]) -> Self {\n        Self::Array(value.iter().map(|v| v.clone().into()).collect())\n    }\n}\n\nimpl EventType {\n    pub fn from_io_error(self, err: std::io::Error) -> Error {\n        self.reason(err).details(\"I/O error\")\n    }\n\n    pub fn from_json_error(self, err: serde_json::Error) -> Error {\n        self.reason(err).details(\"JSON deserialization failed\")\n    }\n\n    pub fn from_base64_error(self, err: base64::DecodeError) -> Error {\n        self.reason(err).details(\"Base64 decoding failed\")\n    }\n\n    pub fn from_http_error(self, err: reqwest::Error) -> Error {\n        self.into_err()\n            .ctx_opt(\n                Key::Url,\n                err.url().map(|url| url.as_ref().to_compact_string()),\n            )\n            .ctx_opt(Key::Code, err.status().map(|status| status.as_u16()))\n            .reason(err)\n    }\n\n    pub fn from_http_str_error(self, err: reqwest::header::ToStrError) -> Error {\n        self.reason(err)\n            .details(\"Failed to convert header to string\")\n    }\n}\n\nimpl From<mail_auth::Error> for Error {\n    fn from(err: mail_auth::Error) -> Self {\n        match err {\n            mail_auth::Error::ParseError => {\n                EventType::MailAuth(MailAuthEvent::ParseError).into_err()\n            }\n            mail_auth::Error::MissingParameters => {\n                EventType::MailAuth(MailAuthEvent::MissingParameters).into_err()\n            }\n            mail_auth::Error::NoHeadersFound => {\n                EventType::MailAuth(MailAuthEvent::NoHeadersFound).into_err()\n            }\n            mail_auth::Error::CryptoError(details) => EventType::MailAuth(MailAuthEvent::Crypto)\n                .into_err()\n                .details(CompactString::from(details)),\n            mail_auth::Error::Io(details) => EventType::MailAuth(MailAuthEvent::Io)\n                .into_err()\n                .details(CompactString::from(details)),\n            mail_auth::Error::Base64 => EventType::MailAuth(MailAuthEvent::Base64).into_err(),\n            mail_auth::Error::UnsupportedVersion => {\n                EventType::Dkim(DkimEvent::UnsupportedVersion).into_err()\n            }\n            mail_auth::Error::UnsupportedAlgorithm => {\n                EventType::Dkim(DkimEvent::UnsupportedAlgorithm).into_err()\n            }\n            mail_auth::Error::UnsupportedCanonicalization => {\n                EventType::Dkim(DkimEvent::UnsupportedCanonicalization).into_err()\n            }\n            mail_auth::Error::UnsupportedKeyType => {\n                EventType::Dkim(DkimEvent::UnsupportedKeyType).into_err()\n            }\n            mail_auth::Error::FailedBodyHashMatch => {\n                EventType::Dkim(DkimEvent::FailedBodyHashMatch).into_err()\n            }\n            mail_auth::Error::FailedVerification => {\n                EventType::Dkim(DkimEvent::FailedVerification).into_err()\n            }\n            mail_auth::Error::FailedAuidMatch => {\n                EventType::Dkim(DkimEvent::FailedAuidMatch).into_err()\n            }\n            mail_auth::Error::RevokedPublicKey => {\n                EventType::Dkim(DkimEvent::RevokedPublicKey).into_err()\n            }\n            mail_auth::Error::IncompatibleAlgorithms => {\n                EventType::Dkim(DkimEvent::IncompatibleAlgorithms).into_err()\n            }\n            mail_auth::Error::SignatureExpired => {\n                EventType::Dkim(DkimEvent::SignatureExpired).into_err()\n            }\n            mail_auth::Error::SignatureLength => {\n                EventType::Dkim(DkimEvent::SignatureLength).into_err()\n            }\n            mail_auth::Error::DnsError(details) => EventType::MailAuth(MailAuthEvent::DnsError)\n                .into_err()\n                .details(CompactString::from(details)),\n            mail_auth::Error::DnsRecordNotFound(code) => {\n                EventType::MailAuth(MailAuthEvent::DnsRecordNotFound)\n                    .into_err()\n                    .code(code.to_str())\n            }\n            mail_auth::Error::ArcChainTooLong => EventType::Arc(ArcEvent::ChainTooLong).into_err(),\n            mail_auth::Error::ArcInvalidInstance(instance) => {\n                EventType::Arc(ArcEvent::InvalidInstance).ctx(Key::Id, instance)\n            }\n            mail_auth::Error::ArcInvalidCV => EventType::Arc(ArcEvent::InvalidCv).into_err(),\n            mail_auth::Error::ArcHasHeaderTag => EventType::Arc(ArcEvent::HasHeaderTag).into_err(),\n            mail_auth::Error::ArcBrokenChain => EventType::Arc(ArcEvent::BrokenChain).into_err(),\n            mail_auth::Error::NotAligned => {\n                EventType::MailAuth(MailAuthEvent::PolicyNotAligned).into_err()\n            }\n            mail_auth::Error::InvalidRecordType => {\n                EventType::MailAuth(MailAuthEvent::DnsInvalidRecordType).into_err()\n            }\n        }\n    }\n}\n\nimpl From<&mail_auth::DkimResult> for Error {\n    fn from(value: &mail_auth::DkimResult) -> Self {\n        match value.clone() {\n            mail_auth::DkimResult::Pass => Error::new(EventType::Dkim(DkimEvent::Pass)),\n            mail_auth::DkimResult::Neutral(err) => {\n                Error::new(EventType::Dkim(DkimEvent::Neutral)).caused_by(Error::from(err))\n            }\n            mail_auth::DkimResult::Fail(err) => {\n                Error::new(EventType::Dkim(DkimEvent::Fail)).caused_by(Error::from(err))\n            }\n            mail_auth::DkimResult::PermError(err) => {\n                Error::new(EventType::Dkim(DkimEvent::PermError)).caused_by(Error::from(err))\n            }\n            mail_auth::DkimResult::TempError(err) => {\n                Error::new(EventType::Dkim(DkimEvent::TempError)).caused_by(Error::from(err))\n            }\n            mail_auth::DkimResult::None => Error::new(EventType::Dkim(DkimEvent::None)),\n        }\n    }\n}\n\nimpl From<&mail_auth::DmarcResult> for Error {\n    fn from(value: &mail_auth::DmarcResult) -> Self {\n        match value.clone() {\n            mail_auth::DmarcResult::Pass => Error::new(EventType::Dmarc(DmarcEvent::Pass)),\n            mail_auth::DmarcResult::Fail(err) => {\n                Error::new(EventType::Dmarc(DmarcEvent::Fail)).caused_by(Error::from(err))\n            }\n            mail_auth::DmarcResult::PermError(err) => {\n                Error::new(EventType::Dmarc(DmarcEvent::PermError)).caused_by(Error::from(err))\n            }\n            mail_auth::DmarcResult::TempError(err) => {\n                Error::new(EventType::Dmarc(DmarcEvent::TempError)).caused_by(Error::from(err))\n            }\n            mail_auth::DmarcResult::None => Error::new(EventType::Dmarc(DmarcEvent::None)),\n        }\n    }\n}\n\nimpl From<&mail_auth::DkimOutput<'_>> for Error {\n    fn from(value: &mail_auth::DkimOutput<'_>) -> Self {\n        Error::from(value.result()).ctx_opt(\n            Key::Domain,\n            value.signature().map(|s| s.domain().to_compact_string()),\n        )\n    }\n}\n\nimpl From<&mail_auth::IprevOutput> for Error {\n    fn from(value: &mail_auth::IprevOutput) -> Self {\n        match value.result().clone() {\n            mail_auth::IprevResult::Pass => Error::new(EventType::Iprev(IprevEvent::Pass)),\n            mail_auth::IprevResult::Fail(err) => {\n                Error::new(EventType::Iprev(IprevEvent::Fail)).caused_by(Error::from(err))\n            }\n            mail_auth::IprevResult::PermError(err) => {\n                Error::new(EventType::Iprev(IprevEvent::PermError)).caused_by(Error::from(err))\n            }\n            mail_auth::IprevResult::TempError(err) => {\n                Error::new(EventType::Iprev(IprevEvent::TempError)).caused_by(Error::from(err))\n            }\n            mail_auth::IprevResult::None => Error::new(EventType::Iprev(IprevEvent::None)),\n        }\n        .ctx_opt(\n            Key::Details,\n            value.ptr.as_ref().map(|s| {\n                s.iter()\n                    .map(|v| Value::String(v.into()))\n                    .collect::<Vec<_>>()\n            }),\n        )\n    }\n}\n\nimpl From<&mail_auth::SpfOutput> for Error {\n    fn from(value: &mail_auth::SpfOutput) -> Self {\n        Error::new(EventType::Spf(match value.result() {\n            mail_auth::SpfResult::Pass => SpfEvent::Pass,\n            mail_auth::SpfResult::Fail => SpfEvent::Fail,\n            mail_auth::SpfResult::SoftFail => SpfEvent::SoftFail,\n            mail_auth::SpfResult::Neutral => SpfEvent::Neutral,\n            mail_auth::SpfResult::PermError => SpfEvent::PermError,\n            mail_auth::SpfResult::TempError => SpfEvent::TempError,\n            mail_auth::SpfResult::None => SpfEvent::None,\n        }))\n        .ctx_opt(\n            Key::Details,\n            value.explanation().map(|s| s.to_compact_string()),\n        )\n    }\n}\n\nimpl From<rkyv::rancor::Error> for Error {\n    fn from(value: rkyv::rancor::Error) -> Self {\n        Error::new(EventType::Store(StoreEvent::DeserializeError))\n            .reason(value)\n            .details(\"Rkyv de/serialization failed\")\n    }\n}\n\npub trait AssertSuccess\nwhere\n    Self: Sized,\n{\n    fn assert_success(\n        self,\n        cause: EventType,\n    ) -> impl std::future::Future<Output = crate::Result<Self>> + Send;\n}\n\nimpl AssertSuccess for reqwest::Response {\n    async fn assert_success(self, cause: EventType) -> crate::Result<Self> {\n        let status = self.status();\n        if status.is_success() {\n            Ok(self)\n        } else {\n            Err(cause\n                .ctx(Key::Code, status.as_u16())\n                .details(\"HTTP request failed\")\n                .ctx_opt(Key::Reason, self.text().await.map(CompactString::from).ok()))\n        }\n    }\n}\n\nimpl FromStr for EventType {\n    type Err = ();\n\n    fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {\n        EventType::try_parse(s).ok_or(())\n    }\n}\n\nimpl FromStr for Key {\n    type Err = ();\n\n    fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {\n        Key::try_parse(s).ok_or(())\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/event/description.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::*;\n\nimpl EventType {\n    pub fn description(&self) -> &'static str {\n        match self {\n            EventType::Store(event) => event.description(),\n            EventType::Jmap(event) => event.description(),\n            EventType::Imap(event) => event.description(),\n            EventType::ManageSieve(event) => event.description(),\n            EventType::Pop3(event) => event.description(),\n            EventType::Smtp(event) => event.description(),\n            EventType::Network(event) => event.description(),\n            EventType::Limit(event) => event.description(),\n            EventType::Manage(event) => event.description(),\n            EventType::Auth(event) => event.description(),\n            EventType::Config(event) => event.description(),\n            EventType::Resource(event) => event.description(),\n            EventType::Sieve(event) => event.description(),\n            EventType::Spam(event) => event.description(),\n            EventType::Server(event) => event.description(),\n            EventType::Purge(event) => event.description(),\n            EventType::Eval(event) => event.description(),\n            EventType::Acme(event) => event.description(),\n            EventType::Http(event) => event.description(),\n            EventType::Arc(event) => event.description(),\n            EventType::Dkim(event) => event.description(),\n            EventType::Dmarc(event) => event.description(),\n            EventType::Iprev(event) => event.description(),\n            EventType::Dane(event) => event.description(),\n            EventType::Spf(event) => event.description(),\n            EventType::MailAuth(event) => event.description(),\n            EventType::Tls(event) => event.description(),\n            EventType::PushSubscription(event) => event.description(),\n            EventType::Cluster(event) => event.description(),\n            EventType::Housekeeper(event) => event.description(),\n            EventType::TaskQueue(event) => event.description(),\n            EventType::Milter(event) => event.description(),\n            EventType::MtaHook(event) => event.description(),\n            EventType::Delivery(event) => event.description(),\n            EventType::Queue(event) => event.description(),\n            EventType::TlsRpt(event) => event.description(),\n            EventType::MtaSts(event) => event.description(),\n            EventType::IncomingReport(event) => event.description(),\n            EventType::OutgoingReport(event) => event.description(),\n            EventType::Telemetry(event) => event.description(),\n            EventType::MessageIngest(event) => event.description(),\n            EventType::Security(event) => event.description(),\n            EventType::Ai(event) => event.description(),\n            EventType::WebDav(event) => event.description(),\n            EventType::Calendar(event) => event.description(),\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            EventType::Store(event) => event.explain(),\n            EventType::Jmap(event) => event.explain(),\n            EventType::Imap(event) => event.explain(),\n            EventType::ManageSieve(event) => event.explain(),\n            EventType::Pop3(event) => event.explain(),\n            EventType::Smtp(event) => event.explain(),\n            EventType::Network(event) => event.explain(),\n            EventType::Limit(event) => event.explain(),\n            EventType::Manage(event) => event.explain(),\n            EventType::Auth(event) => event.explain(),\n            EventType::Config(event) => event.explain(),\n            EventType::Resource(event) => event.explain(),\n            EventType::Sieve(event) => event.explain(),\n            EventType::Spam(event) => event.explain(),\n            EventType::Server(event) => event.explain(),\n            EventType::Purge(event) => event.explain(),\n            EventType::Eval(event) => event.explain(),\n            EventType::Acme(event) => event.explain(),\n            EventType::Http(event) => event.explain(),\n            EventType::Arc(event) => event.explain(),\n            EventType::Dkim(event) => event.explain(),\n            EventType::Dmarc(event) => event.explain(),\n            EventType::Iprev(event) => event.explain(),\n            EventType::Dane(event) => event.explain(),\n            EventType::Spf(event) => event.explain(),\n            EventType::MailAuth(event) => event.explain(),\n            EventType::Tls(event) => event.explain(),\n            EventType::PushSubscription(event) => event.explain(),\n            EventType::Cluster(event) => event.explain(),\n            EventType::Housekeeper(event) => event.explain(),\n            EventType::TaskQueue(event) => event.explain(),\n            EventType::Milter(event) => event.explain(),\n            EventType::MtaHook(event) => event.explain(),\n            EventType::Delivery(event) => event.explain(),\n            EventType::Queue(event) => event.explain(),\n            EventType::TlsRpt(event) => event.explain(),\n            EventType::MtaSts(event) => event.explain(),\n            EventType::IncomingReport(event) => event.explain(),\n            EventType::OutgoingReport(event) => event.explain(),\n            EventType::Telemetry(event) => event.explain(),\n            EventType::MessageIngest(event) => event.explain(),\n            EventType::Security(event) => event.explain(),\n            EventType::Ai(event) => event.explain(),\n            EventType::WebDav(event) => event.explain(),\n            EventType::Calendar(event) => event.explain(),\n        }\n    }\n}\n\nimpl HttpEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            HttpEvent::Error => \"HTTP error occurred\",\n            HttpEvent::RequestUrl => \"HTTP request URL\",\n            HttpEvent::RequestBody => \"HTTP request body\",\n            HttpEvent::ResponseBody => \"HTTP response body\",\n            HttpEvent::XForwardedMissing => \"X-Forwarded-For header is missing\",\n            HttpEvent::ConnectionStart => \"HTTP connection started\",\n            HttpEvent::ConnectionEnd => \"HTTP connection ended\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            HttpEvent::Error => \"An error occurred during an HTTP request\",\n            HttpEvent::RequestUrl => \"The URL of an HTTP request\",\n            HttpEvent::RequestBody => \"The body of an HTTP request\",\n            HttpEvent::ResponseBody => \"The body of an HTTP response\",\n            HttpEvent::XForwardedMissing => \"The X-Forwarded-For header is missing\",\n            HttpEvent::ConnectionStart => \"An HTTP connection was started\",\n            HttpEvent::ConnectionEnd => \"An HTTP connection was ended\",\n        }\n    }\n}\n\nimpl ClusterEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ClusterEvent::SubscriberStart => \"PubSub subscriber started\",\n            ClusterEvent::SubscriberStop => \"PubSub subscriber stopped\",\n            ClusterEvent::SubscriberError => \"PubSub subscriber error\",\n            ClusterEvent::SubscriberDisconnected => \"PubSub subscriber disconnected\",\n            ClusterEvent::PublisherStart => \"PubSub publisher started\",\n            ClusterEvent::PublisherStop => \"PubSub publisher stopped\",\n            ClusterEvent::PublisherError => \"PubSub publisher error\",\n            ClusterEvent::MessageReceived => \"PubSub message received\",\n            ClusterEvent::MessageSkipped => \"PubSub message skipped\",\n            ClusterEvent::MessageInvalid => \"Invalid PubSub message\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ClusterEvent::SubscriberStart => \"The PubSub subscriber has started\",\n            ClusterEvent::SubscriberStop => \"The PubSub subscriber has stopped\",\n            ClusterEvent::SubscriberError => \"An error occurred while subscribing to PubSub\",\n            ClusterEvent::SubscriberDisconnected => \"The PubSub subscriber has disconnected\",\n            ClusterEvent::PublisherStart => \"The PubSub publisher has started\",\n            ClusterEvent::PublisherStop => \"The PubSub publisher has stopped\",\n            ClusterEvent::PublisherError => \"An error occurred while publishing to PubSub\",\n            ClusterEvent::MessageReceived => \"A message was received from the PubSub server\",\n            ClusterEvent::MessageSkipped => \"A message originating from this node was skipped\",\n            ClusterEvent::MessageInvalid => {\n                \"An invalid message was received from the PubSub server\"\n            }\n        }\n    }\n}\n\nimpl HousekeeperEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            HousekeeperEvent::Start => \"Housekeeper process started\",\n            HousekeeperEvent::Stop => \"Housekeeper process stopped\",\n            HousekeeperEvent::Schedule => \"Housekeeper task scheduled\",\n            HousekeeperEvent::Run => \"Housekeeper task run\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            HousekeeperEvent::Start => \"The housekeeper process has started\",\n            HousekeeperEvent::Stop => \"The housekeeper process has stopped\",\n            HousekeeperEvent::Schedule => \"A housekeeper task has been scheduled\",\n            HousekeeperEvent::Run => \"A housekeeper task is running\",\n        }\n    }\n}\n\nimpl TaskQueueEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            TaskQueueEvent::TaskAcquired => \"Task acquired from queue\",\n            TaskQueueEvent::TaskLocked => \"Task is locked by another process\",\n            TaskQueueEvent::BlobNotFound => \"Blob not found for task\",\n            TaskQueueEvent::MetadataNotFound => \"Metadata not found for task\",\n            TaskQueueEvent::TaskIgnored => \"Task ignored based on current server roles\",\n            TaskQueueEvent::TaskFailed => \"Task failed during processing\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            TaskQueueEvent::TaskAcquired => \"A task has been acquired from the queue\",\n            TaskQueueEvent::TaskLocked => \"The task id is locked by another process\",\n            TaskQueueEvent::BlobNotFound => \"The requested blob was not found for task\",\n            TaskQueueEvent::MetadataNotFound => \"The metadata was not found for task\",\n            TaskQueueEvent::TaskIgnored => \"The task was ignored based on the current server roles\",\n            TaskQueueEvent::TaskFailed => \"The task failed during processing\",\n        }\n    }\n}\n\nimpl ImapEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ImapEvent::GetAcl => \"IMAP GET ACL command\",\n            ImapEvent::SetAcl => \"IMAP SET ACL command\",\n            ImapEvent::MyRights => \"IMAP MYRIGHTS command\",\n            ImapEvent::ListRights => \"IMAP LISTRIGHTS command\",\n            ImapEvent::Append => \"IMAP APPEND command\",\n            ImapEvent::Capabilities => \"IMAP CAPABILITIES command\",\n            ImapEvent::Id => \"IMAP ID command\",\n            ImapEvent::Close => \"IMAP CLOSE command\",\n            ImapEvent::Copy => \"IMAP COPY command\",\n            ImapEvent::Move => \"IMAP MOVE command\",\n            ImapEvent::CreateMailbox => \"IMAP CREATE mailbox command\",\n            ImapEvent::DeleteMailbox => \"IMAP DELETE mailbox command\",\n            ImapEvent::RenameMailbox => \"IMAP RENAME mailbox command\",\n            ImapEvent::Enable => \"IMAP ENABLE command\",\n            ImapEvent::Expunge => \"IMAP EXPUNGE command\",\n            ImapEvent::Fetch => \"IMAP FETCH command\",\n            ImapEvent::IdleStart => \"IMAP IDLE start\",\n            ImapEvent::IdleStop => \"IMAP IDLE stop\",\n            ImapEvent::List => \"IMAP LIST command\",\n            ImapEvent::Lsub => \"IMAP LSUB command\",\n            ImapEvent::Logout => \"IMAP LOGOUT command\",\n            ImapEvent::Namespace => \"IMAP NAMESPACE command\",\n            ImapEvent::Noop => \"IMAP NOOP command\",\n            ImapEvent::Search => \"IMAP SEARCH command\",\n            ImapEvent::Sort => \"IMAP SORT command\",\n            ImapEvent::Select => \"IMAP SELECT command\",\n            ImapEvent::Status => \"IMAP STATUS command\",\n            ImapEvent::Store => \"IMAP STORE command\",\n            ImapEvent::Subscribe => \"IMAP SUBSCRIBE command\",\n            ImapEvent::Unsubscribe => \"IMAP UNSUBSCRIBE command\",\n            ImapEvent::Thread => \"IMAP THREAD command\",\n            ImapEvent::Error => \"IMAP error occurred\",\n            ImapEvent::RawInput => \"Raw IMAP input received\",\n            ImapEvent::RawOutput => \"Raw IMAP output sent\",\n            ImapEvent::ConnectionStart => \"IMAP connection started\",\n            ImapEvent::ConnectionEnd => \"IMAP connection ended\",\n            ImapEvent::GetQuota => \"IMAP GETQUOTA command\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ImapEvent::GetAcl => \"Client requested mailbox ACL\",\n            ImapEvent::SetAcl => \"Client set mailbox ACL\",\n            ImapEvent::MyRights => \"Client requested mailbox rights\",\n            ImapEvent::ListRights => \"Client requested mailbox rights list\",\n            ImapEvent::Append => \"Client appended a message to a mailbox\",\n            ImapEvent::Capabilities => \"Client requested server capabilities\",\n            ImapEvent::Id => \"Client sent an ID command\",\n            ImapEvent::Close => \"Client closed a mailbox\",\n            ImapEvent::Copy => \"Client copied messages between mailboxes\",\n            ImapEvent::Move => \"Client moved messages between mailboxes\",\n            ImapEvent::CreateMailbox => \"Client created a mailbox\",\n            ImapEvent::DeleteMailbox => \"Client deleted a mailbox\",\n            ImapEvent::RenameMailbox => \"Client renamed a mailbox\",\n            ImapEvent::Enable => \"Client enabled an extension\",\n            ImapEvent::Expunge => \"Client expunged messages\",\n            ImapEvent::Fetch => \"Client fetched messages\",\n            ImapEvent::IdleStart => \"Client started IDLE\",\n            ImapEvent::IdleStop => \"Client stopped IDLE\",\n            ImapEvent::List => \"Client listed mailboxes\",\n            ImapEvent::Lsub => \"Client listed subscribed mailboxes\",\n            ImapEvent::Logout => \"Client logged out\",\n            ImapEvent::Namespace => \"Client requested namespace\",\n            ImapEvent::Noop => \"Client sent a NOOP command\",\n            ImapEvent::Search => \"Client searched for messages\",\n            ImapEvent::Sort => \"Client sorted messages\",\n            ImapEvent::Select => \"Client selected a mailbox\",\n            ImapEvent::Status => \"Client requested mailbox status\",\n            ImapEvent::Store => \"Client stored flags\",\n            ImapEvent::Subscribe => \"Client subscribed to a mailbox\",\n            ImapEvent::Unsubscribe => \"Client unsubscribed from a mailbox\",\n            ImapEvent::Thread => \"Client requested message threads\",\n            ImapEvent::Error => \"An error occurred during an IMAP command\",\n            ImapEvent::RawInput => \"Raw IMAP input received\",\n            ImapEvent::RawOutput => \"Raw IMAP output sent\",\n            ImapEvent::ConnectionStart => \"IMAP connection started\",\n            ImapEvent::ConnectionEnd => \"IMAP connection ended\",\n            ImapEvent::GetQuota => \"Client requested mailbox quota\",\n        }\n    }\n}\n\nimpl Pop3Event {\n    pub fn description(&self) -> &'static str {\n        match self {\n            Pop3Event::Delete => \"POP3 DELETE command\",\n            Pop3Event::Reset => \"POP3 RESET command\",\n            Pop3Event::Quit => \"POP3 QUIT command\",\n            Pop3Event::Fetch => \"POP3 FETCH command\",\n            Pop3Event::List => \"POP3 LIST command\",\n            Pop3Event::ListMessage => \"POP3 LIST specific message command\",\n            Pop3Event::Uidl => \"POP3 UIDL command\",\n            Pop3Event::UidlMessage => \"POP3 UIDL specific message command\",\n            Pop3Event::Stat => \"POP3 STAT command\",\n            Pop3Event::Noop => \"POP3 NOOP command\",\n            Pop3Event::Capabilities => \"POP3 CAPABILITIES command\",\n            Pop3Event::StartTls => \"POP3 STARTTLS command\",\n            Pop3Event::Utf8 => \"POP3 UTF8 command\",\n            Pop3Event::Error => \"POP3 error occurred\",\n            Pop3Event::RawInput => \"Raw POP3 input received\",\n            Pop3Event::RawOutput => \"Raw POP3 output sent\",\n            Pop3Event::ConnectionStart => \"POP3 connection started\",\n            Pop3Event::ConnectionEnd => \"POP3 connection ended\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            Pop3Event::Delete => \"Client deleted a message\",\n            Pop3Event::Reset => \"Client reset the session\",\n            Pop3Event::Quit => \"Client quit the session\",\n            Pop3Event::Fetch => \"Client fetched a message\",\n            Pop3Event::List => \"Client listed messages\",\n            Pop3Event::ListMessage => \"Client listed a specific message\",\n            Pop3Event::Uidl => \"Client requested unique identifiers\",\n            Pop3Event::UidlMessage => \"Client requested a specific unique identifier\",\n            Pop3Event::Stat => \"Client requested mailbox status\",\n            Pop3Event::Noop => \"Client sent a NOOP command\",\n            Pop3Event::Capabilities => \"Client requested server capabilities\",\n            Pop3Event::StartTls => \"Client requested TLS\",\n            Pop3Event::Utf8 => \"Client requested UTF-8 support\",\n            Pop3Event::Error => \"An error occurred during a POP3 command\",\n            Pop3Event::RawInput => \"Raw POP3 input received\",\n            Pop3Event::RawOutput => \"Raw POP3 output sent\",\n            Pop3Event::ConnectionStart => \"POP3 connection started\",\n            Pop3Event::ConnectionEnd => \"POP3 connection ended\",\n        }\n    }\n}\n\nimpl ManageSieveEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ManageSieveEvent::CreateScript => \"ManageSieve CREATE script command\",\n            ManageSieveEvent::UpdateScript => \"ManageSieve UPDATE script command\",\n            ManageSieveEvent::GetScript => \"ManageSieve GET script command\",\n            ManageSieveEvent::DeleteScript => \"ManageSieve DELETE script command\",\n            ManageSieveEvent::RenameScript => \"ManageSieve RENAME script command\",\n            ManageSieveEvent::CheckScript => \"ManageSieve CHECK script command\",\n            ManageSieveEvent::HaveSpace => \"ManageSieve HAVESPACE command\",\n            ManageSieveEvent::ListScripts => \"ManageSieve LIST scripts command\",\n            ManageSieveEvent::SetActive => \"ManageSieve SET ACTIVE command\",\n            ManageSieveEvent::Capabilities => \"ManageSieve CAPABILITIES command\",\n            ManageSieveEvent::StartTls => \"ManageSieve STARTTLS command\",\n            ManageSieveEvent::Unauthenticate => \"ManageSieve UNAUTHENTICATE command\",\n            ManageSieveEvent::Logout => \"ManageSieve LOGOUT command\",\n            ManageSieveEvent::Noop => \"ManageSieve NOOP command\",\n            ManageSieveEvent::Error => \"ManageSieve error occurred\",\n            ManageSieveEvent::RawInput => \"Raw ManageSieve input received\",\n            ManageSieveEvent::RawOutput => \"Raw ManageSieve output sent\",\n            ManageSieveEvent::ConnectionStart => \"ManageSieve connection started\",\n            ManageSieveEvent::ConnectionEnd => \"ManageSieve connection ended\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ManageSieveEvent::CreateScript => \"Client created a script\",\n            ManageSieveEvent::UpdateScript => \"Client updated a script\",\n            ManageSieveEvent::GetScript => \"Client fetched a script\",\n            ManageSieveEvent::DeleteScript => \"Client deleted a script\",\n            ManageSieveEvent::RenameScript => \"Client renamed a script\",\n            ManageSieveEvent::CheckScript => \"Client checked a script\",\n            ManageSieveEvent::HaveSpace => \"Client checked for space\",\n            ManageSieveEvent::ListScripts => \"Client listed scripts\",\n            ManageSieveEvent::SetActive => \"Client set an active script\",\n            ManageSieveEvent::Capabilities => \"Client requested server capabilities\",\n            ManageSieveEvent::StartTls => \"Client requested TLS\",\n            ManageSieveEvent::Unauthenticate => \"Client unauthenticated\",\n            ManageSieveEvent::Logout => \"Client logged out\",\n            ManageSieveEvent::Noop => \"Client sent a NOOP command\",\n            ManageSieveEvent::Error => \"An error occurred during a ManageSieve command\",\n            ManageSieveEvent::RawInput => \"Raw ManageSieve input received\",\n            ManageSieveEvent::RawOutput => \"Raw ManageSieve output sent\",\n            ManageSieveEvent::ConnectionStart => \"ManageSieve connection started\",\n            ManageSieveEvent::ConnectionEnd => \"ManageSieve connection ended\",\n        }\n    }\n}\n\nimpl SmtpEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            SmtpEvent::Error => \"SMTP error occurred\",\n            SmtpEvent::IdNotFound => \"Strategy not found\",\n            SmtpEvent::ConcurrencyLimitExceeded => \"Concurrency limit exceeded\",\n            SmtpEvent::TransferLimitExceeded => \"Transfer limit exceeded\",\n            SmtpEvent::RateLimitExceeded => \"Rate limit exceeded\",\n            SmtpEvent::TimeLimitExceeded => \"Time limit exceeded\",\n            SmtpEvent::MissingAuthDirectory => \"Missing auth directory\",\n            SmtpEvent::MessageParseFailed => \"Message parsing failed\",\n            SmtpEvent::MessageTooLarge => \"Message too large\",\n            SmtpEvent::LoopDetected => \"Mail loop detected\",\n            SmtpEvent::DkimPass => \"DKIM verification passed\",\n            SmtpEvent::DkimFail => \"DKIM verification failed\",\n            SmtpEvent::ArcPass => \"ARC verification passed\",\n            SmtpEvent::ArcFail => \"ARC verification failed\",\n            SmtpEvent::SpfEhloPass => \"SPF EHLO check passed\",\n            SmtpEvent::SpfEhloFail => \"SPF EHLO check failed\",\n            SmtpEvent::SpfFromPass => \"SPF From check passed\",\n            SmtpEvent::SpfFromFail => \"SPF From check failed\",\n            SmtpEvent::DmarcPass => \"DMARC check passed\",\n            SmtpEvent::DmarcFail => \"DMARC check failed\",\n            SmtpEvent::IprevPass => \"IPREV check passed\",\n            SmtpEvent::IprevFail => \"IPREV check failed\",\n            SmtpEvent::TooManyMessages => \"Too many messages\",\n            SmtpEvent::Ehlo => \"SMTP EHLO command\",\n            SmtpEvent::InvalidEhlo => \"Invalid EHLO command\",\n            SmtpEvent::DidNotSayEhlo => \"Client did not say EHLO\",\n            SmtpEvent::EhloExpected => \"EHLO command expected\",\n            SmtpEvent::LhloExpected => \"LHLO command expected\",\n            SmtpEvent::MailFromUnauthenticated => \"MAIL FROM without authentication\",\n            SmtpEvent::MailFromUnauthorized => \"MAIL FROM unauthorized\",\n            SmtpEvent::MailFromRewritten => \"MAIL FROM address rewritten\",\n            SmtpEvent::MailFromMissing => \"MAIL FROM address missing\",\n            SmtpEvent::MailFromNotAllowed => \"MAIL FROM not allowed\",\n            SmtpEvent::MailFrom => \"SMTP MAIL FROM command\",\n            SmtpEvent::MultipleMailFrom => \"Multiple MAIL FROM commands\",\n            SmtpEvent::MailboxDoesNotExist => \"Mailbox does not exist\",\n            SmtpEvent::RelayNotAllowed => \"Relay not allowed\",\n            SmtpEvent::RcptTo => \"SMTP RCPT TO command\",\n            SmtpEvent::RcptToDuplicate => \"Duplicate RCPT TO\",\n            SmtpEvent::RcptToRewritten => \"RCPT TO address rewritten\",\n            SmtpEvent::RcptToMissing => \"RCPT TO address missing\",\n            SmtpEvent::RcptToGreylisted => \"RCPT TO greylisted\",\n            SmtpEvent::TooManyRecipients => \"Too many recipients\",\n            SmtpEvent::TooManyInvalidRcpt => \"Too many invalid recipients\",\n            SmtpEvent::RawInput => \"Raw SMTP input received\",\n            SmtpEvent::RawOutput => \"Raw SMTP output sent\",\n            SmtpEvent::MissingLocalHostname => \"Missing local hostname\",\n            SmtpEvent::Vrfy => \"SMTP VRFY command\",\n            SmtpEvent::VrfyNotFound => \"VRFY address not found\",\n            SmtpEvent::VrfyDisabled => \"VRFY command disabled\",\n            SmtpEvent::Expn => \"SMTP EXPN command\",\n            SmtpEvent::ExpnNotFound => \"EXPN address not found\",\n            SmtpEvent::ExpnDisabled => \"EXPN command disabled\",\n            SmtpEvent::RequireTlsDisabled => \"REQUIRETLS extension disabled\",\n            SmtpEvent::DeliverByDisabled => \"DELIVERBY extension disabled\",\n            SmtpEvent::DeliverByInvalid => \"Invalid DELIVERBY parameter\",\n            SmtpEvent::FutureReleaseDisabled => \"FUTURE RELEASE extension disabled\",\n            SmtpEvent::FutureReleaseInvalid => \"Invalid FUTURE RELEASE parameter\",\n            SmtpEvent::MtPriorityDisabled => \"MT-PRIORITY extension disabled\",\n            SmtpEvent::MtPriorityInvalid => \"Invalid MT-PRIORITY parameter\",\n            SmtpEvent::DsnDisabled => \"DSN extension disabled\",\n            SmtpEvent::AuthNotAllowed => \"Authentication not allowed\",\n            SmtpEvent::AuthMechanismNotSupported => \"Auth mechanism not supported\",\n            SmtpEvent::AuthExchangeTooLong => \"Auth exchange too long\",\n            SmtpEvent::AlreadyAuthenticated => \"Already authenticated\",\n            SmtpEvent::Noop => \"SMTP NOOP command\",\n            SmtpEvent::StartTls => \"SMTP STARTTLS command\",\n            SmtpEvent::StartTlsUnavailable => \"STARTTLS unavailable\",\n            SmtpEvent::StartTlsAlready => \"TLS already active\",\n            SmtpEvent::Rset => \"SMTP RSET command\",\n            SmtpEvent::Quit => \"SMTP QUIT command\",\n            SmtpEvent::Help => \"SMTP HELP command\",\n            SmtpEvent::CommandNotImplemented => \"Command not implemented\",\n            SmtpEvent::InvalidCommand => \"Invalid command\",\n            SmtpEvent::InvalidSenderAddress => \"Invalid sender address\",\n            SmtpEvent::InvalidRecipientAddress => \"Invalid recipient address\",\n            SmtpEvent::InvalidParameter => \"Invalid parameter\",\n            SmtpEvent::UnsupportedParameter => \"Unsupported parameter\",\n            SmtpEvent::SyntaxError => \"Syntax error\",\n            SmtpEvent::RequestTooLarge => \"Request too large\",\n            SmtpEvent::ConnectionStart => \"SMTP connection started\",\n            SmtpEvent::ConnectionEnd => \"SMTP connection ended\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            SmtpEvent::Error => \"An error occurred during an SMTP command\",\n            SmtpEvent::IdNotFound => \"The strategy ID was not found in the configuration\",\n            SmtpEvent::ConcurrencyLimitExceeded => \"The concurrency limit was exceeded\",\n            SmtpEvent::TransferLimitExceeded => {\n                \"The remote host transferred more data than allowed\"\n            }\n            SmtpEvent::RateLimitExceeded => \"The rate limit was exceeded\",\n            SmtpEvent::TimeLimitExceeded => \"The remote host kept the SMTP session open too long\",\n            SmtpEvent::MissingAuthDirectory => \"The auth directory was missing\",\n            SmtpEvent::MessageParseFailed => \"Failed to parse the message\",\n            SmtpEvent::MessageTooLarge => \"The message was rejected because it was too large\",\n            SmtpEvent::LoopDetected => {\n                \"A mail loop was detected, the message contains too many Received headers\"\n            }\n            SmtpEvent::DkimPass => \"Successful DKIM verification\",\n            SmtpEvent::DkimFail => \"Failed to verify DKIM signature\",\n            SmtpEvent::ArcPass => \"Successful ARC verification\",\n            SmtpEvent::ArcFail => \"Failed to verify ARC signature\",\n            SmtpEvent::SpfEhloPass => \"EHLO identity passed SPF check\",\n            SmtpEvent::SpfEhloFail => \"EHLO identity failed SPF check\",\n            SmtpEvent::SpfFromPass => \"MAIL FROM identity passed SPF check\",\n            SmtpEvent::SpfFromFail => \"MAIL FROM identity failed SPF check\",\n            SmtpEvent::DmarcPass => \"Successful DMARC verification\",\n            SmtpEvent::DmarcFail => \"Failed to verify DMARC policy\",\n            SmtpEvent::IprevPass => \"Reverse IP check passed\",\n            SmtpEvent::IprevFail => \"Reverse IP check failed\",\n            SmtpEvent::TooManyMessages => {\n                \"The remote server exceeded the number of messages allowed per session\"\n            }\n            SmtpEvent::Ehlo => \"The remote server sent an EHLO command\",\n            SmtpEvent::InvalidEhlo => \"The remote server sent an invalid EHLO command\",\n            SmtpEvent::DidNotSayEhlo => \"The remote server did not send EHLO command\",\n            SmtpEvent::EhloExpected => {\n                \"The remote server sent a LHLO command while EHLO was expected\"\n            }\n            SmtpEvent::LhloExpected => {\n                \"The remote server sent an EHLO command while LHLO was expected\"\n            }\n            SmtpEvent::MailFromUnauthenticated => {\n                \"The remote client did not authenticate before sending MAIL FROM\"\n            }\n            SmtpEvent::MailFromUnauthorized => {\n                \"The remote client is not authorized to send mail from the given address\"\n            }\n            SmtpEvent::MailFromRewritten => \"The envelope sender address was rewritten\",\n            SmtpEvent::MailFromMissing => {\n                \"The remote client issued an RCPT TO command before MAIL FROM\"\n            }\n            SmtpEvent::MailFromNotAllowed => {\n                \"The remote client is not allowed to send mail from this address\"\n            }\n            SmtpEvent::MailFrom => \"The remote client sent a MAIL FROM command\",\n            SmtpEvent::MultipleMailFrom => \"The remote client already sent a MAIL FROM command\",\n            SmtpEvent::MailboxDoesNotExist => \"The mailbox does not exist on the server\",\n            SmtpEvent::RelayNotAllowed => \"The server does not allow relaying\",\n            SmtpEvent::RcptTo => \"The remote client sent an RCPT TO command\",\n            SmtpEvent::RcptToDuplicate => {\n                \"The remote client already sent an RCPT TO command for this recipient\"\n            }\n            SmtpEvent::RcptToRewritten => \"The envelope recipient address was rewritten\",\n            SmtpEvent::RcptToMissing => \"The remote client issued a DATA command before RCPT TO\",\n            SmtpEvent::RcptToGreylisted => \"The recipient was greylisted\",\n            SmtpEvent::TooManyRecipients => {\n                \"The remote client exceeded the number of recipients allowed\"\n            }\n            SmtpEvent::TooManyInvalidRcpt => {\n                \"The remote client exceeded the number of invalid RCPT TO commands allowed\"\n            }\n            SmtpEvent::RawInput => \"Raw SMTP input received\",\n            SmtpEvent::RawOutput => \"Raw SMTP output sent\",\n            SmtpEvent::MissingLocalHostname => \"The local hostname is missing in the configuration\",\n            SmtpEvent::Vrfy => \"The remote client sent a VRFY command\",\n            SmtpEvent::VrfyNotFound => {\n                \"The remote client sent a VRFY command for an address that was not found\"\n            }\n            SmtpEvent::VrfyDisabled => \"The VRFY command is disabled\",\n            SmtpEvent::Expn => \"The remote client sent an EXPN command\",\n            SmtpEvent::ExpnNotFound => {\n                \"The remote client sent an EXPN command for an address that was not found\"\n            }\n            SmtpEvent::ExpnDisabled => \"The EXPN command is disabled\",\n            SmtpEvent::RequireTlsDisabled => \"The REQUIRETLS extension is disabled\",\n            SmtpEvent::DeliverByDisabled => \"The DELIVERBY extension is disabled\",\n            SmtpEvent::DeliverByInvalid => \"The DELIVERBY parameter is invalid\",\n            SmtpEvent::FutureReleaseDisabled => \"The FUTURE RELEASE extension is disabled\",\n            SmtpEvent::FutureReleaseInvalid => \"The FUTURE RELEASE parameter is invalid\",\n            SmtpEvent::MtPriorityDisabled => \"The MT-PRIORITY extension is disabled\",\n            SmtpEvent::MtPriorityInvalid => \"The MT-PRIORITY parameter is invalid\",\n            SmtpEvent::DsnDisabled => \"The DSN extension is disabled\",\n            SmtpEvent::AuthNotAllowed => \"Authentication is not allowed on this listener\",\n            SmtpEvent::AuthMechanismNotSupported => {\n                \"The requested authentication mechanism is not supported\"\n            }\n            SmtpEvent::AuthExchangeTooLong => \"The authentication exchange was too long\",\n            SmtpEvent::AlreadyAuthenticated => \"The client is already authenticated\",\n            SmtpEvent::Noop => \"The remote client sent a NOOP command\",\n            SmtpEvent::StartTls => \"The remote client requested a TLS connection\",\n            SmtpEvent::StartTlsUnavailable => {\n                \"The remote client requested a TLS connection but it is not available\"\n            }\n            SmtpEvent::Rset => \"The remote client sent a RSET command\",\n            SmtpEvent::Quit => \"The remote client sent a QUIT command\",\n            SmtpEvent::Help => \"The remote client sent a HELP command\",\n            SmtpEvent::CommandNotImplemented => {\n                \"The server does not implement the requested command\"\n            }\n            SmtpEvent::InvalidCommand => \"The remote client sent an invalid command\",\n            SmtpEvent::InvalidSenderAddress => \"The specified sender address is invalid\",\n            SmtpEvent::InvalidRecipientAddress => \"The specified recipient address is invalid\",\n            SmtpEvent::InvalidParameter => \"The command contained an invalid parameter\",\n            SmtpEvent::UnsupportedParameter => \"The command contained an unsupported parameter\",\n            SmtpEvent::SyntaxError => \"The command contained a syntax error\",\n            SmtpEvent::RequestTooLarge => \"The request was too large\",\n            SmtpEvent::ConnectionStart => \"A new SMTP connection was started\",\n            SmtpEvent::ConnectionEnd => \"The SMTP connection was ended\",\n            SmtpEvent::StartTlsAlready => \"TLS is already active\",\n        }\n    }\n}\n\nimpl DeliveryEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            DeliveryEvent::AttemptStart => \"Delivery attempt started\",\n            DeliveryEvent::AttemptEnd => \"Delivery attempt ended\",\n            DeliveryEvent::Completed => \"Delivery completed\",\n            DeliveryEvent::Failed => \"Delivery failed\",\n            DeliveryEvent::DomainDeliveryStart => \"New delivery attempt for domain\",\n            DeliveryEvent::MxLookup => \"MX record lookup\",\n            DeliveryEvent::MxLookupFailed => \"MX record lookup failed\",\n            DeliveryEvent::IpLookup => \"IP address lookup\",\n            DeliveryEvent::IpLookupFailed => \"IP address lookup failed\",\n            DeliveryEvent::NullMx => \"Null MX record found\",\n            DeliveryEvent::Connect => \"Connecting to remote server\",\n            DeliveryEvent::ConnectError => \"Connection error\",\n            DeliveryEvent::MissingOutboundHostname => \"Missing outbound hostname in configuration\",\n            DeliveryEvent::GreetingFailed => \"SMTP greeting failed\",\n            DeliveryEvent::Ehlo => \"SMTP EHLO command\",\n            DeliveryEvent::EhloRejected => \"SMTP EHLO rejected\",\n            DeliveryEvent::Auth => \"SMTP authentication\",\n            DeliveryEvent::AuthFailed => \"SMTP authentication failed\",\n            DeliveryEvent::MailFrom => \"SMTP MAIL FROM command\",\n            DeliveryEvent::MailFromRejected => \"SMTP MAIL FROM rejected\",\n            DeliveryEvent::Delivered => \"Message delivered\",\n            DeliveryEvent::RcptTo => \"SMTP RCPT TO command\",\n            DeliveryEvent::RcptToRejected => \"SMTP RCPT TO rejected\",\n            DeliveryEvent::RcptToFailed => \"SMTP RCPT TO failed\",\n            DeliveryEvent::MessageRejected => \"Message rejected by remote server\",\n            DeliveryEvent::StartTls => \"SMTP STARTTLS command\",\n            DeliveryEvent::StartTlsUnavailable => \"STARTTLS unavailable\",\n            DeliveryEvent::StartTlsError => \"STARTTLS error\",\n            DeliveryEvent::StartTlsDisabled => \"STARTTLS disabled\",\n            DeliveryEvent::ImplicitTlsError => \"Implicit TLS error\",\n            DeliveryEvent::ConcurrencyLimitExceeded => \"Concurrency limit exceeded\",\n            DeliveryEvent::RateLimitExceeded => \"Rate limit exceeded\",\n            DeliveryEvent::DoubleBounce => \"Discarding message after double bounce\",\n            DeliveryEvent::DsnSuccess => \"DSN success notification\",\n            DeliveryEvent::DsnTempFail => \"DSN temporary failure notification\",\n            DeliveryEvent::DsnPermFail => \"DSN permanent failure notification\",\n            DeliveryEvent::RawInput => \"Raw SMTP input received\",\n            DeliveryEvent::RawOutput => \"Raw SMTP output sent\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            DeliveryEvent::AttemptStart => \"A new delivery attempt for the message has started\",\n            DeliveryEvent::AttemptEnd => \"The delivery attempt has ended\",\n            DeliveryEvent::Completed => \"Delivery was completed for all recipients\",\n            DeliveryEvent::Failed => \"Message delivery failed due to a temporary error\",\n            DeliveryEvent::DomainDeliveryStart => \"A new delivery attempt for a domain has started\",\n            DeliveryEvent::MxLookup => \"Looking up MX records for the domain\",\n            DeliveryEvent::MxLookupFailed => \"Failed to look up MX records for the domain\",\n            DeliveryEvent::IpLookup => \"Looking up IP address for the domain\",\n            DeliveryEvent::IpLookupFailed => \"Failed to look up IP address for the domain\",\n            DeliveryEvent::NullMx => \"The domain has a null MX record, delivery is impossible\",\n            DeliveryEvent::Connect => \"Connecting to the remote server\",\n            DeliveryEvent::ConnectError => \"Error connecting to the remote server\",\n            DeliveryEvent::MissingOutboundHostname => {\n                \"The outbound hostname is missing in the configuration\"\n            }\n            DeliveryEvent::GreetingFailed => {\n                \"Failed to read the SMTP greeting from the remote server\"\n            }\n            DeliveryEvent::Ehlo => \"The EHLO command was sent to the remote server\",\n            DeliveryEvent::EhloRejected => \"The remote server rejected the EHLO command\",\n            DeliveryEvent::Auth => \"Authenticating with the remote server\",\n            DeliveryEvent::AuthFailed => \"Authentication with the remote server failed\",\n            DeliveryEvent::MailFrom => \"The MAIL FROM command was sent to the remote server\",\n            DeliveryEvent::MailFromRejected => \"The remote server rejected the MAIL FROM command\",\n            DeliveryEvent::Delivered => \"The message was delivered to the recipient\",\n            DeliveryEvent::RcptTo => \"The RCPT TO command was sent to the remote server\",\n            DeliveryEvent::RcptToRejected => \"The remote server rejected the RCPT TO command\",\n            DeliveryEvent::RcptToFailed => {\n                \"Failed to send the RCPT TO command to the remote server\"\n            }\n            DeliveryEvent::MessageRejected => \"The remote server rejected the message\",\n            DeliveryEvent::StartTls => \"Requesting a TLS connection with the remote server\",\n            DeliveryEvent::StartTlsUnavailable => \"The remote server does not support STARTTLS\",\n            DeliveryEvent::StartTlsError => \"It was not possible to establish a TLS connection\",\n            DeliveryEvent::StartTlsDisabled => {\n                \"STARTTLS has been disabled in the configuration for this host\"\n            }\n            DeliveryEvent::ImplicitTlsError => \"Error starting implicit TLS\",\n            DeliveryEvent::ConcurrencyLimitExceeded => {\n                \"The concurrency limit was exceeded for the remote host\"\n            }\n            DeliveryEvent::RateLimitExceeded => \"The rate limit was exceeded for the remote host\",\n            DeliveryEvent::DoubleBounce => \"The message was discarded after a double bounce\",\n            DeliveryEvent::DsnSuccess => \"A success delivery status notification was created\",\n            DeliveryEvent::DsnTempFail => {\n                \"A temporary failure delivery status notification was created\"\n            }\n            DeliveryEvent::DsnPermFail => {\n                \"A permanent failure delivery status notification was created\"\n            }\n            DeliveryEvent::RawInput => \"Raw SMTP input received\",\n            DeliveryEvent::RawOutput => \"Raw SMTP output sent\",\n        }\n    }\n}\n\nimpl QueueEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            QueueEvent::Rescheduled => \"Message rescheduled for delivery\",\n            QueueEvent::Locked => \"Queue event is locked by another process\",\n            QueueEvent::BlobNotFound => \"Message blob not found\",\n            QueueEvent::RateLimitExceeded => \"Rate limit exceeded\",\n            QueueEvent::ConcurrencyLimitExceeded => \"Concurrency limit exceeded\",\n            QueueEvent::QuotaExceeded => \"Quota exceeded\",\n            QueueEvent::QueueMessage => \"Queued message for delivery\",\n            QueueEvent::QueueMessageAuthenticated => \"Queued message submission for delivery\",\n            QueueEvent::QueueReport => \"Queued report for delivery\",\n            QueueEvent::QueueDsn => \"Queued DSN for delivery\",\n            QueueEvent::QueueAutogenerated => \"Queued autogenerated message for delivery\",\n            QueueEvent::BackPressure => \"Queue backpressure detected\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            QueueEvent::Rescheduled => \"The message was rescheduled for delivery\",\n            QueueEvent::Locked => \"The queue event is locked by another process\",\n            QueueEvent::BlobNotFound => \"The message blob was not found\",\n            QueueEvent::RateLimitExceeded => \"The queue rate limit was exceeded\",\n            QueueEvent::ConcurrencyLimitExceeded => \"The queue concurrency limit was exceeded\",\n            QueueEvent::QuotaExceeded => \"The queue quota was exceeded\",\n            QueueEvent::QueueMessage => \"A new message was queued for delivery\",\n            QueueEvent::QueueMessageAuthenticated => {\n                \"A new message was queued for delivery from an authenticated client\"\n            }\n            QueueEvent::QueueReport => \"A new report was queued for delivery\",\n            QueueEvent::QueueDsn => \"A delivery status notification was queued for delivery\",\n            QueueEvent::QueueAutogenerated => \"A system generated message was queued for delivery\",\n            QueueEvent::BackPressure => {\n                \"Queue congested, processing can't keep up with incoming message rate\"\n            }\n        }\n    }\n}\n\nimpl IncomingReportEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            IncomingReportEvent::DmarcReport => \"DMARC report received\",\n            IncomingReportEvent::DmarcReportWithWarnings => \"DMARC report received with warnings\",\n            IncomingReportEvent::TlsReport => \"TLS report received\",\n            IncomingReportEvent::TlsReportWithWarnings => \"TLS report received with warnings\",\n            IncomingReportEvent::AbuseReport => \"Abuse report received\",\n            IncomingReportEvent::AuthFailureReport => \"Authentication failure report received\",\n            IncomingReportEvent::FraudReport => \"Fraud report received\",\n            IncomingReportEvent::NotSpamReport => \"Not spam report received\",\n            IncomingReportEvent::VirusReport => \"Virus report received\",\n            IncomingReportEvent::OtherReport => \"Other type of report received\",\n            IncomingReportEvent::MessageParseFailed => \"Failed to parse incoming report message\",\n            IncomingReportEvent::DmarcParseFailed => \"Failed to parse DMARC report\",\n            IncomingReportEvent::TlsRpcParseFailed => \"Failed to parse TLS RPC report\",\n            IncomingReportEvent::ArfParseFailed => \"Failed to parse ARF report\",\n            IncomingReportEvent::DecompressError => \"Error decompressing report\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            IncomingReportEvent::DmarcReport => \"A DMARC report has been received\",\n            IncomingReportEvent::DmarcReportWithWarnings => {\n                \"A DMARC report with warnings has been received\"\n            }\n            IncomingReportEvent::TlsReport => \"A TLS report has been received\",\n            IncomingReportEvent::TlsReportWithWarnings => {\n                \"A TLS report with warnings has been received\"\n            }\n            IncomingReportEvent::AbuseReport => \"An abuse report has been received\",\n            IncomingReportEvent::AuthFailureReport => {\n                \"An authentication failure report has been received\"\n            }\n            IncomingReportEvent::FraudReport => \"A fraud report has been received\",\n            IncomingReportEvent::NotSpamReport => \"A not spam report has been received\",\n            IncomingReportEvent::VirusReport => \"A virus report has been received\",\n            IncomingReportEvent::OtherReport => \"An unknown type of report has been received\",\n            IncomingReportEvent::MessageParseFailed => {\n                \"Failed to parse the incoming report message\"\n            }\n            IncomingReportEvent::DmarcParseFailed => \"Failed to parse the DMARC report\",\n            IncomingReportEvent::TlsRpcParseFailed => \"Failed to parse the TLS RPC report\",\n            IncomingReportEvent::ArfParseFailed => \"Failed to parse the ARF report\",\n            IncomingReportEvent::DecompressError => \"Error decompressing the report\",\n        }\n    }\n}\n\nimpl OutgoingReportEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            OutgoingReportEvent::SpfReport => \"SPF report sent\",\n            OutgoingReportEvent::SpfRateLimited => \"SPF report rate limited\",\n            OutgoingReportEvent::DkimReport => \"DKIM report sent\",\n            OutgoingReportEvent::DkimRateLimited => \"DKIM report rate limited\",\n            OutgoingReportEvent::DmarcReport => \"DMARC report sent\",\n            OutgoingReportEvent::DmarcRateLimited => \"DMARC report rate limited\",\n            OutgoingReportEvent::DmarcAggregateReport => \"DMARC aggregate is being prepared\",\n            OutgoingReportEvent::TlsAggregate => \"TLS aggregate report is being prepared\",\n            OutgoingReportEvent::HttpSubmission => \"Report submitted via HTTP\",\n            OutgoingReportEvent::UnauthorizedReportingAddress => \"Unauthorized reporting address\",\n            OutgoingReportEvent::ReportingAddressValidationError => {\n                \"Error validating reporting address\"\n            }\n            OutgoingReportEvent::NotFound => \"Report not found\",\n            OutgoingReportEvent::SubmissionError => \"Error submitting report\",\n            OutgoingReportEvent::NoRecipientsFound => \"No recipients found for report\",\n            OutgoingReportEvent::Locked => \"Report is locked by another process\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            OutgoingReportEvent::SpfReport => \"An SPF report has been sent\",\n            OutgoingReportEvent::SpfRateLimited => \"The SPF report was rate limited\",\n            OutgoingReportEvent::DkimReport => \"A DKIM report has been sent\",\n            OutgoingReportEvent::DkimRateLimited => \"The DKIM report was rate limited\",\n            OutgoingReportEvent::DmarcReport => \"A DMARC report has been sent\",\n            OutgoingReportEvent::DmarcRateLimited => \"The DMARC report was rate limited\",\n            OutgoingReportEvent::DmarcAggregateReport => \"A DMARC aggregate report will be sent\",\n            OutgoingReportEvent::TlsAggregate => \"A TLS aggregate report will be sent\",\n            OutgoingReportEvent::HttpSubmission => \"The report was submitted via HTTP\",\n            OutgoingReportEvent::UnauthorizedReportingAddress => {\n                \"The reporting address is not authorized to send reports\"\n            }\n            OutgoingReportEvent::ReportingAddressValidationError => {\n                \"Error validating the reporting address\"\n            }\n            OutgoingReportEvent::NotFound => \"The report was not found\",\n            OutgoingReportEvent::SubmissionError => \"Error submitting the report\",\n            OutgoingReportEvent::NoRecipientsFound => \"No recipients found for the report\",\n            OutgoingReportEvent::Locked => \"The report is locked by another process\",\n        }\n    }\n}\n\nimpl MtaStsEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            MtaStsEvent::Authorized => \"Host authorized by MTA-STS policy\",\n            MtaStsEvent::NotAuthorized => \"Host not authorized by MTA-STS policy\",\n            MtaStsEvent::PolicyFetch => \"Fetched MTA-STS policy\",\n            MtaStsEvent::PolicyNotFound => \"MTA-STS policy not found\",\n            MtaStsEvent::PolicyFetchError => \"Error fetching MTA-STS policy\",\n            MtaStsEvent::InvalidPolicy => \"Invalid MTA-STS policy\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            MtaStsEvent::Authorized => \"The host is authorized by the MTA-STS policy\",\n            MtaStsEvent::NotAuthorized => \"The host is not authorized by the MTA-STS policy\",\n            MtaStsEvent::PolicyFetch => \"The MTA-STS policy has been fetched\",\n            MtaStsEvent::PolicyNotFound => \"An MTA-STS policy was not found\",\n            MtaStsEvent::PolicyFetchError => \"An error occurred while fetching the MTA-STS policy\",\n            MtaStsEvent::InvalidPolicy => \"The MTA-STS policy is invalid\",\n        }\n    }\n}\n\nimpl TlsRptEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            TlsRptEvent::RecordFetch => \"Fetched TLS-RPT record\",\n            TlsRptEvent::RecordFetchError => \"Error fetching TLS-RPT record\",\n            TlsRptEvent::RecordNotFound => \"TLS-RPT record not found\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            TlsRptEvent::RecordFetch => \"The TLS-RPT record has been fetched\",\n            TlsRptEvent::RecordFetchError => \"An error occurred while fetching the TLS-RPT record\",\n            TlsRptEvent::RecordNotFound => \"No TLS-RPT records were found\",\n        }\n    }\n}\n\nimpl DaneEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            DaneEvent::AuthenticationSuccess => \"DANE authentication successful\",\n            DaneEvent::AuthenticationFailure => \"DANE authentication failed\",\n            DaneEvent::NoCertificatesFound => \"No certificates found for DANE\",\n            DaneEvent::CertificateParseError => \"Error parsing certificate for DANE\",\n            DaneEvent::TlsaRecordMatch => \"TLSA record match found\",\n            DaneEvent::TlsaRecordFetch => \"Fetching TLSA record\",\n            DaneEvent::TlsaRecordFetchError => \"Error fetching TLSA record\",\n            DaneEvent::TlsaRecordNotFound => \"TLSA record not found\",\n            DaneEvent::TlsaRecordNotDnssecSigned => \"TLSA record not DNSSEC signed\",\n            DaneEvent::TlsaRecordInvalid => \"Invalid TLSA record\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            DaneEvent::AuthenticationSuccess => \"Successful DANE authentication\",\n            DaneEvent::AuthenticationFailure => \"Failed DANE authentication\",\n            DaneEvent::NoCertificatesFound => \"No certificates were found for DANE\",\n            DaneEvent::CertificateParseError => \"An error occurred while parsing the certificate\",\n            DaneEvent::TlsaRecordMatch => \"A TLSA record match was found\",\n            DaneEvent::TlsaRecordFetch => \"The TLSA record has been fetched\",\n            DaneEvent::TlsaRecordFetchError => \"An error occurred while fetching the TLSA record\",\n            DaneEvent::TlsaRecordNotFound => \"The TLSA record was not found\",\n            DaneEvent::TlsaRecordNotDnssecSigned => \"The TLSA record is not DNSSEC signed\",\n            DaneEvent::TlsaRecordInvalid => \"The TLSA record is invalid\",\n        }\n    }\n}\n\nimpl MilterEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            MilterEvent::Read => \"Reading from Milter\",\n            MilterEvent::Write => \"Writing to Milter\",\n            MilterEvent::ActionAccept => \"Milter action: Accept\",\n            MilterEvent::ActionDiscard => \"Milter action: Discard\",\n            MilterEvent::ActionReject => \"Milter action: Reject\",\n            MilterEvent::ActionTempFail => \"Milter action: Temporary failure\",\n            MilterEvent::ActionReplyCode => \"Milter action: Reply code\",\n            MilterEvent::ActionConnectionFailure => \"Milter action: Connection failure\",\n            MilterEvent::ActionShutdown => \"Milter action: Shutdown\",\n            MilterEvent::IoError => \"Milter I/O error\",\n            MilterEvent::FrameTooLarge => \"Milter frame too large\",\n            MilterEvent::FrameInvalid => \"Invalid Milter frame\",\n            MilterEvent::UnexpectedResponse => \"Unexpected Milter response\",\n            MilterEvent::Timeout => \"Milter timeout\",\n            MilterEvent::TlsInvalidName => \"Invalid TLS name for Milter\",\n            MilterEvent::Disconnected => \"Milter disconnected\",\n            MilterEvent::ParseError => \"Milter parse error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            MilterEvent::Read => \"Reading from the Milter\",\n            MilterEvent::Write => \"Writing to the Milter\",\n            MilterEvent::ActionAccept => \"The Milter requested to accept the message\",\n            MilterEvent::ActionDiscard => \"The Milter requested to discard the message\",\n            MilterEvent::ActionReject => \"The Milter requested to reject the message\",\n            MilterEvent::ActionTempFail => \"The Milter requested to temporarily fail the message\",\n            MilterEvent::ActionReplyCode => \"The Milter requested a reply code\",\n            MilterEvent::ActionConnectionFailure => \"The Milter requested a connection failure\",\n            MilterEvent::ActionShutdown => \"The Milter requested a shutdown\",\n            MilterEvent::IoError => \"An I/O error occurred with the Milter\",\n            MilterEvent::FrameTooLarge => \"The Milter frame was too large\",\n            MilterEvent::FrameInvalid => \"The Milter frame was invalid\",\n            MilterEvent::UnexpectedResponse => {\n                \"An unexpected response was received from the Milter\"\n            }\n            MilterEvent::Timeout => \"A timeout occurred with the Milter\",\n            MilterEvent::TlsInvalidName => \"The Milter TLS name is invalid\",\n            MilterEvent::Disconnected => \"The Milter disconnected\",\n            MilterEvent::ParseError => \"An error occurred while parsing the Milter response\",\n        }\n    }\n}\n\nimpl MtaHookEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            MtaHookEvent::ActionAccept => \"MTA hook action: Accept\",\n            MtaHookEvent::ActionDiscard => \"MTA hook action: Discard\",\n            MtaHookEvent::ActionReject => \"MTA hook action: Reject\",\n            MtaHookEvent::ActionQuarantine => \"MTA hook action: Quarantine\",\n            MtaHookEvent::Error => \"MTA hook error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            MtaHookEvent::ActionAccept => \"The MTA hook requested to accept the message\",\n            MtaHookEvent::ActionDiscard => \"The MTA hook requested to discard the message\",\n            MtaHookEvent::ActionReject => \"The MTA hook requested to reject the message\",\n            MtaHookEvent::ActionQuarantine => \"The MTA hook requested to quarantine the message\",\n            MtaHookEvent::Error => \"An error occurred with the MTA hook\",\n        }\n    }\n}\n\nimpl PushSubscriptionEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            PushSubscriptionEvent::Success => \"Push subscription successful\",\n            PushSubscriptionEvent::Error => \"Push subscription error\",\n            PushSubscriptionEvent::NotFound => \"Push subscription not found\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            PushSubscriptionEvent::Success => \"The push subscription was successful\",\n            PushSubscriptionEvent::Error => \"An error occurred with the push subscription\",\n            PushSubscriptionEvent::NotFound => \"The push subscription was not found\",\n        }\n    }\n}\n\nimpl SpamEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            SpamEvent::Pyzor => \"Pyzor success\",\n            SpamEvent::PyzorError => \"Pyzor error\",\n            SpamEvent::Classify => \"Classifying message for spam\",\n            SpamEvent::Dnsbl => \"DNSBL query\",\n            SpamEvent::DnsblError => \"Error querying DNSBL\",\n            SpamEvent::TrainStarted => \"Spam classifier training started\",\n            SpamEvent::TrainCompleted => \"Spam classifier training completed\",\n            SpamEvent::TrainSampleAdded => \"New training sample added\",\n            SpamEvent::TrainSampleNotFound => \"Training sample not found\",\n            SpamEvent::ModelLoaded => \"Spam classifier model loaded\",\n            SpamEvent::ModelNotReady => \"Spam classifier model not ready\",\n            SpamEvent::ModelNotFound => \"Spam classifier model not found\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            SpamEvent::PyzorError => \"An error occurred with Pyzor\",\n            SpamEvent::Classify => \"The message is being classified for spam\",\n            SpamEvent::Pyzor => \"Pyzor query successful\",\n            SpamEvent::Dnsbl => \"The DNSBL query was successful\",\n            SpamEvent::DnsblError => \"An error occurred while querying the DNSBL\",\n            SpamEvent::TrainStarted => \"SGD logistic regression training has started\",\n            SpamEvent::TrainCompleted => \"SGD logistic regression training has completed\",\n            SpamEvent::TrainSampleAdded => \"A new training sample has been added\",\n            SpamEvent::TrainSampleNotFound => \"A training sample was not found\",\n            SpamEvent::ModelLoaded => \"The spam classifier model has been loaded\",\n            SpamEvent::ModelNotReady => {\n                \"The spam classifier model has not been trained with enough data\"\n            }\n            SpamEvent::ModelNotFound => \"The spam classifier model has not been trained yet\",\n        }\n    }\n}\n\nimpl SieveEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            SieveEvent::ActionAccept => \"Sieve action: Accept\",\n            SieveEvent::ActionAcceptReplace => \"Sieve action: Accept and replace\",\n            SieveEvent::ActionDiscard => \"Sieve action: Discard\",\n            SieveEvent::ActionReject => \"Sieve action: Reject\",\n            SieveEvent::SendMessage => \"Sieve sending message\",\n            SieveEvent::MessageTooLarge => \"Sieve message too large\",\n            SieveEvent::ScriptNotFound => \"Sieve script not found\",\n            SieveEvent::ListNotFound => \"Sieve list not found\",\n            SieveEvent::RuntimeError => \"Sieve runtime error\",\n            SieveEvent::UnexpectedError => \"Unexpected Sieve error\",\n            SieveEvent::NotSupported => \"Sieve action not supported\",\n            SieveEvent::QuotaExceeded => \"Sieve quota exceeded\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            SieveEvent::ActionAccept => \"The Sieve script requested to accept the message\",\n            SieveEvent::ActionAcceptReplace => {\n                \"The Sieve script requested to accept the message and replace its contents\"\n            }\n            SieveEvent::ActionDiscard => \"The Sieve script requested to discard the message\",\n            SieveEvent::ActionReject => \"The Sieve script requested to reject the message\",\n            SieveEvent::SendMessage => \"The Sieve script is sending a message\",\n            SieveEvent::MessageTooLarge => \"The Sieve message is too large\",\n            SieveEvent::ScriptNotFound => \"The Sieve script was not found\",\n            SieveEvent::ListNotFound => \"The Sieve list was not found\",\n            SieveEvent::RuntimeError => \"A runtime error occurred with the Sieve script\",\n            SieveEvent::UnexpectedError => \"An unexpected error occurred with the Sieve script\",\n            SieveEvent::NotSupported => \"The Sieve action is not supported\",\n            SieveEvent::QuotaExceeded => \"The Sieve quota was exceeded\",\n        }\n    }\n}\n\nimpl TlsEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            TlsEvent::Handshake => \"TLS handshake\",\n            TlsEvent::HandshakeError => \"TLS handshake error\",\n            TlsEvent::NotConfigured => \"TLS not configured\",\n            TlsEvent::CertificateNotFound => \"TLS certificate not found\",\n            TlsEvent::NoCertificatesAvailable => \"No TLS certificates available\",\n            TlsEvent::MultipleCertificatesAvailable => \"Multiple TLS certificates available\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            TlsEvent::Handshake => \"Successful TLS handshake\",\n            TlsEvent::HandshakeError => \"An error occurred during the TLS handshake\",\n            TlsEvent::NotConfigured => \"TLS is not configured\",\n            TlsEvent::CertificateNotFound => \"The TLS certificate was not found\",\n            TlsEvent::NoCertificatesAvailable => \"No TLS certificates are available\",\n            TlsEvent::MultipleCertificatesAvailable => \"Multiple TLS certificates are available\",\n        }\n    }\n}\n\nimpl NetworkEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            NetworkEvent::ListenStart => \"Network listener started\",\n            NetworkEvent::ListenStop => \"Network listener stopped\",\n            NetworkEvent::ListenError => \"Network listener error\",\n            NetworkEvent::BindError => \"Network bind error\",\n            NetworkEvent::ReadError => \"Network read error\",\n            NetworkEvent::WriteError => \"Network write error\",\n            NetworkEvent::FlushError => \"Network flush error\",\n            NetworkEvent::AcceptError => \"Network accept error\",\n            NetworkEvent::SplitError => \"Network split error\",\n            NetworkEvent::Timeout => \"Network timeout\",\n            NetworkEvent::Closed => \"Network connection closed\",\n            NetworkEvent::ProxyError => \"Proxy protocol error\",\n            NetworkEvent::SetOptError => \"Network set option error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            NetworkEvent::ListenStart => \"The network listener has started\",\n            NetworkEvent::ListenStop => \"The network listener has stopped\",\n            NetworkEvent::ListenError => \"An error occurred with the network listener\",\n            NetworkEvent::BindError => \"An error occurred while binding the network listener\",\n            NetworkEvent::ReadError => \"An error occurred while reading from the network\",\n            NetworkEvent::WriteError => \"An error occurred while writing to the network\",\n            NetworkEvent::FlushError => \"An error occurred while flushing the network\",\n            NetworkEvent::AcceptError => \"An error occurred while accepting a network connection\",\n            NetworkEvent::SplitError => \"An error occurred while splitting the network connection\",\n            NetworkEvent::Timeout => \"A network timeout occurred\",\n            NetworkEvent::Closed => \"The network connection was closed\",\n            NetworkEvent::ProxyError => \"An error occurred with the proxy protocol\",\n            NetworkEvent::SetOptError => \"An error occurred while setting network options\",\n        }\n    }\n}\n\nimpl ServerEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ServerEvent::Startup => {\n                concat!(\"Starting Stalwart Server v\", env!(\"CARGO_PKG_VERSION\"))\n            }\n            ServerEvent::Shutdown => {\n                concat!(\"Shutting down Stalwart Server v\", env!(\"CARGO_PKG_VERSION\"))\n            }\n            ServerEvent::StartupError => \"Server startup error\",\n            ServerEvent::ThreadError => \"Server thread error\",\n            ServerEvent::Licensing => \"Server licensing event\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ServerEvent::Startup => \"Stalwart Server has started\",\n            ServerEvent::Shutdown => \"Stalwart Server is shutting down\",\n            ServerEvent::StartupError => \"An error occurred while starting the server\",\n            ServerEvent::ThreadError => \"An error occurred with a server thread\",\n            ServerEvent::Licensing => \"A licensing event occurred\",\n        }\n    }\n}\n\nimpl TelemetryEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            TelemetryEvent::Alert => \"Alert triggered\",\n            TelemetryEvent::LogError => \"Log collector error\",\n            TelemetryEvent::WebhookError => \"Webhook collector error\",\n            TelemetryEvent::JournalError => \"Journal collector error\",\n            TelemetryEvent::OtelExporterError => \"OpenTelemetry exporter error\",\n            TelemetryEvent::OtelMetricsExporterError => \"OpenTelemetry metrics exporter error\",\n            TelemetryEvent::PrometheusExporterError => \"Prometheus exporter error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            TelemetryEvent::Alert => \"An alert was triggered\",\n            TelemetryEvent::LogError => \"An error occurred with the log collector\",\n            TelemetryEvent::WebhookError => \"An error occurred with the webhook collector\",\n            TelemetryEvent::JournalError => \"An error occurred with the journal collector\",\n            TelemetryEvent::OtelExporterError => {\n                \"An error occurred with the OpenTelemetry exporter\"\n            }\n            TelemetryEvent::OtelMetricsExporterError => {\n                \"An error occurred with the OpenTelemetry metrics exporter\"\n            }\n            TelemetryEvent::PrometheusExporterError => {\n                \"An error occurred with the Prometheus exporter\"\n            }\n        }\n    }\n}\n\nimpl AcmeEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            AcmeEvent::AuthStart => \"ACME authentication started\",\n            AcmeEvent::AuthPending => \"ACME authentication pending\",\n            AcmeEvent::AuthValid => \"ACME authentication valid\",\n            AcmeEvent::AuthCompleted => \"ACME authentication completed\",\n            AcmeEvent::AuthError => \"ACME authentication error\",\n            AcmeEvent::AuthTooManyAttempts => \"Too many ACME authentication attempts\",\n            AcmeEvent::ProcessCert => \"Processing ACME certificate\",\n            AcmeEvent::OrderStart => \"ACME order started\",\n            AcmeEvent::OrderProcessing => \"ACME order processing\",\n            AcmeEvent::OrderCompleted => \"ACME order completed\",\n            AcmeEvent::OrderReady => \"ACME order ready\",\n            AcmeEvent::OrderValid => \"ACME order valid\",\n            AcmeEvent::OrderInvalid => \"ACME order invalid\",\n            AcmeEvent::RenewBackoff => \"ACME renew backoff\",\n            AcmeEvent::DnsRecordCreated => \"ACME DNS record created\",\n            AcmeEvent::DnsRecordCreationFailed => \"ACME DNS record creation failed\",\n            AcmeEvent::DnsRecordDeletionFailed => \"ACME DNS record deletion failed\",\n            AcmeEvent::DnsRecordNotPropagated => \"ACME DNS record not propagated\",\n            AcmeEvent::DnsRecordLookupFailed => \"ACME DNS record lookup failed\",\n            AcmeEvent::DnsRecordPropagated => \"ACME DNS record propagated\",\n            AcmeEvent::DnsRecordPropagationTimeout => \"ACME DNS record propagation timeout\",\n            AcmeEvent::ClientSuppliedSni => \"ACME client supplied SNI\",\n            AcmeEvent::ClientMissingSni => \"ACME client missing SNI\",\n            AcmeEvent::TlsAlpnReceived => \"ACME TLS ALPN received\",\n            AcmeEvent::TlsAlpnError => \"ACME TLS ALPN error\",\n            AcmeEvent::TokenNotFound => \"ACME token not found\",\n            AcmeEvent::Error => \"ACME error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            AcmeEvent::AuthStart => \"ACME authentication has started\",\n            AcmeEvent::AuthPending => \"ACME authentication is pending\",\n            AcmeEvent::AuthValid => \"ACME authentication is valid\",\n            AcmeEvent::AuthCompleted => \"ACME authentication has completed\",\n            AcmeEvent::AuthError => \"An error occurred with ACME authentication\",\n            AcmeEvent::AuthTooManyAttempts => \"Too many ACME authentication attempts\",\n            AcmeEvent::ProcessCert => \"Processing the ACME certificate\",\n            AcmeEvent::OrderStart => \"ACME order has started\",\n            AcmeEvent::OrderProcessing => \"ACME order is processing\",\n            AcmeEvent::OrderCompleted => \"ACME order has completed\",\n            AcmeEvent::OrderReady => \"ACME order is ready\",\n            AcmeEvent::OrderValid => \"ACME order is valid\",\n            AcmeEvent::OrderInvalid => \"ACME order is invalid\",\n            AcmeEvent::RenewBackoff => \"ACME renew backoff\",\n            AcmeEvent::DnsRecordCreated => \"ACME DNS record has been created\",\n            AcmeEvent::DnsRecordCreationFailed => \"Failed to create ACME DNS record\",\n            AcmeEvent::DnsRecordDeletionFailed => \"Failed to delete ACME DNS record\",\n            AcmeEvent::DnsRecordNotPropagated => \"ACME DNS record has not propagated\",\n            AcmeEvent::DnsRecordLookupFailed => \"Failed to look up ACME DNS record\",\n            AcmeEvent::DnsRecordPropagated => \"ACME DNS record has propagated\",\n            AcmeEvent::DnsRecordPropagationTimeout => \"ACME DNS record propagation timeout\",\n            AcmeEvent::ClientSuppliedSni => \"ACME client supplied SNI\",\n            AcmeEvent::ClientMissingSni => \"ACME client missing SNI\",\n            AcmeEvent::TlsAlpnReceived => \"ACME TLS ALPN received\",\n            AcmeEvent::TlsAlpnError => \"ACME TLS ALPN error\",\n            AcmeEvent::TokenNotFound => \"ACME token not found\",\n            AcmeEvent::Error => \"An error occurred with ACME\",\n        }\n    }\n}\n\nimpl PurgeEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            PurgeEvent::Started => \"Purge started\",\n            PurgeEvent::Finished => \"Purge finished\",\n            PurgeEvent::Running => \"Purge running\",\n            PurgeEvent::Error => \"Purge error\",\n            PurgeEvent::InProgress => \"Active purge in progress\",\n            PurgeEvent::AutoExpunge => \"Auto-expunge executed\",\n            PurgeEvent::BlobCleanup => \"Blob storage cleanup completed\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            PurgeEvent::Started => \"The purge has started\",\n            PurgeEvent::Finished => \"The purge has finished\",\n            PurgeEvent::Running => \"The purge is running\",\n            PurgeEvent::Error => \"An error occurred with the purge\",\n            PurgeEvent::InProgress => \"An active purge is in progress\",\n            PurgeEvent::AutoExpunge => \"Auto-expunge has been executed\",\n            PurgeEvent::BlobCleanup => \"Blob storage cleanup has completed\",\n        }\n    }\n}\n\nimpl EvalEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            EvalEvent::Result => \"Expression evaluation result\",\n            EvalEvent::Error => \"Expression evaluation error\",\n            EvalEvent::DirectoryNotFound => \"Directory not found while evaluating expression\",\n            EvalEvent::StoreNotFound => \"Store not found while evaluating expression\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            EvalEvent::Result => \"The expression evaluation has a result\",\n            EvalEvent::Error => \"An error occurred while evaluating the expression\",\n            EvalEvent::DirectoryNotFound => {\n                \"The directory was not found while evaluating the expression\"\n            }\n            EvalEvent::StoreNotFound => \"The store was not found while evaluating the expression\",\n        }\n    }\n}\n\nimpl ConfigEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ConfigEvent::ParseError => \"Configuration parse error\",\n            ConfigEvent::BuildError => \"Configuration build error\",\n            ConfigEvent::MacroError => \"Configuration macro error\",\n            ConfigEvent::WriteError => \"Configuration write error\",\n            ConfigEvent::FetchError => \"Configuration fetch error\",\n            ConfigEvent::DefaultApplied => \"Default configuration applied\",\n            ConfigEvent::MissingSetting => \"Missing configuration setting\",\n            ConfigEvent::UnusedSetting => \"Unused configuration setting\",\n            ConfigEvent::ParseWarning => \"Configuration parse warning\",\n            ConfigEvent::BuildWarning => \"Configuration build warning\",\n            ConfigEvent::ImportExternal => \"Importing external configuration\",\n            ConfigEvent::AlreadyUpToDate => \"Configuration already up to date\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ConfigEvent::ParseError => \"An error occurred while parsing the configuration\",\n            ConfigEvent::BuildError => \"An error occurred while building the configuration\",\n            ConfigEvent::MacroError => \"An error occurred with a configuration macro\",\n            ConfigEvent::WriteError => \"An error occurred while writing the configuration\",\n            ConfigEvent::FetchError => \"An error occurred while fetching the configuration\",\n            ConfigEvent::DefaultApplied => \"The default configuration has been applied\",\n            ConfigEvent::MissingSetting => \"A configuration setting is missing\",\n            ConfigEvent::UnusedSetting => \"A configuration setting is unused\",\n            ConfigEvent::ParseWarning => \"A warning occurred while parsing the configuration\",\n            ConfigEvent::BuildWarning => \"A warning occurred while building the configuration\",\n            ConfigEvent::ImportExternal => \"An external configuration is being imported\",\n            ConfigEvent::AlreadyUpToDate => \"The configuration is already up to date\",\n        }\n    }\n}\n\nimpl ArcEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ArcEvent::ChainTooLong => \"ARC chain too long\",\n            ArcEvent::InvalidInstance => \"Invalid ARC instance\",\n            ArcEvent::InvalidCv => \"Invalid ARC CV\",\n            ArcEvent::HasHeaderTag => \"ARC has header tag\",\n            ArcEvent::BrokenChain => \"Broken ARC chain\",\n            ArcEvent::SealerNotFound => \"ARC sealer not found\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ArcEvent::ChainTooLong => \"The ARC chain is too long\",\n            ArcEvent::InvalidInstance => \"The ARC instance is invalid\",\n            ArcEvent::InvalidCv => \"The ARC CV is invalid\",\n            ArcEvent::HasHeaderTag => \"The ARC has a header tag\",\n            ArcEvent::BrokenChain => \"The ARC chain is broken\",\n            ArcEvent::SealerNotFound => \"The ARC sealer was not found\",\n        }\n    }\n}\n\nimpl DkimEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            DkimEvent::Pass => \"DKIM verification passed\",\n            DkimEvent::Neutral => \"DKIM verification neutral\",\n            DkimEvent::Fail => \"DKIM verification failed\",\n            DkimEvent::PermError => \"DKIM permanent error\",\n            DkimEvent::TempError => \"DKIM temporary error\",\n            DkimEvent::None => \"No DKIM signature\",\n            DkimEvent::UnsupportedVersion => \"Unsupported DKIM version\",\n            DkimEvent::UnsupportedAlgorithm => \"Unsupported DKIM algorithm\",\n            DkimEvent::UnsupportedCanonicalization => \"Unsupported DKIM canonicalization\",\n            DkimEvent::UnsupportedKeyType => \"Unsupported DKIM key type\",\n            DkimEvent::FailedBodyHashMatch => \"DKIM body hash mismatch\",\n            DkimEvent::FailedVerification => \"DKIM verification failed\",\n            DkimEvent::FailedAuidMatch => \"DKIM AUID mismatch\",\n            DkimEvent::RevokedPublicKey => \"DKIM public key revoked\",\n            DkimEvent::IncompatibleAlgorithms => \"Incompatible DKIM algorithms\",\n            DkimEvent::SignatureExpired => \"DKIM signature expired\",\n            DkimEvent::SignatureLength => \"DKIM signature length issue\",\n            DkimEvent::SignerNotFound => \"DKIM signer not found\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            DkimEvent::Pass => \"DKIM verification has passed\",\n            DkimEvent::Neutral => \"DKIM verification is neutral\",\n            DkimEvent::Fail => \"DKIM verification has failed\",\n            DkimEvent::PermError => \"A permanent error occurred with DKIM\",\n            DkimEvent::TempError => \"A temporary error occurred with DKIM\",\n            DkimEvent::None => \"No DKIM signature was found\",\n            DkimEvent::UnsupportedVersion => \"The DKIM version is unsupported\",\n            DkimEvent::UnsupportedAlgorithm => \"The DKIM algorithm is unsupported\",\n            DkimEvent::UnsupportedCanonicalization => \"The DKIM canonicalization is unsupported\",\n            DkimEvent::UnsupportedKeyType => \"The DKIM key type is unsupported\",\n            DkimEvent::FailedBodyHashMatch => \"The DKIM body hash does not match\",\n            DkimEvent::FailedVerification => \"The DKIM verification has failed\",\n            DkimEvent::FailedAuidMatch => \"The DKIM AUID does not match\",\n            DkimEvent::RevokedPublicKey => \"The DKIM public key has been revoked\",\n            DkimEvent::IncompatibleAlgorithms => \"The DKIM algorithms are incompatible\",\n            DkimEvent::SignatureExpired => \"The DKIM signature has expired\",\n            DkimEvent::SignatureLength => \"The DKIM signature length is incorrect\",\n            DkimEvent::SignerNotFound => \"The DKIM signer was not found\",\n        }\n    }\n}\n\nimpl SpfEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            SpfEvent::Pass => \"SPF check passed\",\n            SpfEvent::Fail => \"SPF check failed\",\n            SpfEvent::SoftFail => \"SPF soft fail\",\n            SpfEvent::Neutral => \"SPF neutral result\",\n            SpfEvent::TempError => \"SPF temporary error\",\n            SpfEvent::PermError => \"SPF permanent error\",\n            SpfEvent::None => \"No SPF record\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            SpfEvent::Pass => \"The SPF check has passed\",\n            SpfEvent::Fail => \"The SPF check has failed\",\n            SpfEvent::SoftFail => \"The SPF check has soft failed\",\n            SpfEvent::Neutral => \"The SPF result is neutral\",\n            SpfEvent::TempError => \"A temporary error occurred with SPF\",\n            SpfEvent::PermError => \"A permanent error occurred with SPF\",\n            SpfEvent::None => \"No SPF record was found\",\n        }\n    }\n}\n\nimpl DmarcEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            DmarcEvent::Pass => \"DMARC check passed\",\n            DmarcEvent::Fail => \"DMARC check failed\",\n            DmarcEvent::PermError => \"DMARC permanent error\",\n            DmarcEvent::TempError => \"DMARC temporary error\",\n            DmarcEvent::None => \"No DMARC record\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            DmarcEvent::Pass => \"The DMARC check has passed\",\n            DmarcEvent::Fail => \"The DMARC check has failed\",\n            DmarcEvent::PermError => \"A permanent error occurred with DMARC\",\n            DmarcEvent::TempError => \"A temporary error occurred with DMARC\",\n            DmarcEvent::None => \"No DMARC record was found\",\n        }\n    }\n}\n\nimpl IprevEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            IprevEvent::Pass => \"IPREV check passed\",\n            IprevEvent::Fail => \"IPREV check failed\",\n            IprevEvent::PermError => \"IPREV permanent error\",\n            IprevEvent::TempError => \"IPREV temporary error\",\n            IprevEvent::None => \"No IPREV record\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            IprevEvent::Pass => \"The IPREV check has passed\",\n            IprevEvent::Fail => \"The IPREV check has failed\",\n            IprevEvent::PermError => \"A permanent error occurred with IPREV\",\n            IprevEvent::TempError => \"A temporary error occurred with IPREV\",\n            IprevEvent::None => \"No IPREV record was found\",\n        }\n    }\n}\n\nimpl MailAuthEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            MailAuthEvent::ParseError => \"Mail authentication parse error\",\n            MailAuthEvent::MissingParameters => \"Missing mail authentication parameters\",\n            MailAuthEvent::NoHeadersFound => \"No headers found in message\",\n            MailAuthEvent::Crypto => \"Crypto error during mail authentication\",\n            MailAuthEvent::Io => \"I/O error during mail authentication\",\n            MailAuthEvent::Base64 => \"Base64 error during mail authentication\",\n            MailAuthEvent::DnsError => \"DNS error\",\n            MailAuthEvent::DnsRecordNotFound => \"DNS record not found\",\n            MailAuthEvent::DnsInvalidRecordType => \"Invalid DNS record type\",\n            MailAuthEvent::PolicyNotAligned => \"Policy not aligned\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            MailAuthEvent::ParseError => \"An error occurred while parsing mail authentication\",\n            MailAuthEvent::MissingParameters => \"Mail authentication parameters are missing\",\n            MailAuthEvent::NoHeadersFound => \"No headers were found in the message\",\n            MailAuthEvent::Crypto => \"A crypto error occurred during mail authentication\",\n            MailAuthEvent::Io => \"An I/O error occurred during mail authentication\",\n            MailAuthEvent::Base64 => \"A base64 error occurred during mail authentication\",\n            MailAuthEvent::DnsError => \"A DNS error occurred\",\n            MailAuthEvent::DnsRecordNotFound => \"The DNS record was not found\",\n            MailAuthEvent::DnsInvalidRecordType => \"The DNS record type is invalid\",\n            MailAuthEvent::PolicyNotAligned => \"The policy is not aligned\",\n        }\n    }\n}\n\nimpl StoreEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            StoreEvent::AssertValueFailed => \"Another process modified the record\",\n            StoreEvent::FoundationdbError => \"FoundationDB error\",\n            StoreEvent::MysqlError => \"MySQL error\",\n            StoreEvent::PostgresqlError => \"PostgreSQL error\",\n            StoreEvent::RocksdbError => \"RocksDB error\",\n            StoreEvent::SqliteError => \"SQLite error\",\n            StoreEvent::LdapError => \"LDAP error\",\n            StoreEvent::ElasticsearchError => \"ElasticSearch error\",\n            StoreEvent::RedisError => \"Redis error\",\n            StoreEvent::S3Error => \"S3 error\",\n            StoreEvent::AzureError => \"Azure error\",\n            StoreEvent::FilesystemError => \"Filesystem error\",\n            StoreEvent::PoolError => \"Connection pool error\",\n            StoreEvent::DataCorruption => \"Data corruption detected\",\n            StoreEvent::DecompressError => \"Decompression error\",\n            StoreEvent::DeserializeError => \"Deserialization error\",\n            StoreEvent::NotFound => \"Record not found in database\",\n            StoreEvent::NotConfigured => \"Store not configured\",\n            StoreEvent::NotSupported => \"Operation not supported by store\",\n            StoreEvent::UnexpectedError => \"Unexpected store error\",\n            StoreEvent::CryptoError => \"Store crypto error\",\n            StoreEvent::BlobMissingMarker => \"Blob missing marker\",\n            StoreEvent::SqlQuery => \"SQL query executed\",\n            StoreEvent::LdapQuery => \"LDAP query executed\",\n            StoreEvent::LdapWarning => \"LDAP authentication warning\",\n            StoreEvent::DataWrite => \"Write batch operation\",\n            StoreEvent::BlobRead => \"Blob read operation\",\n            StoreEvent::BlobWrite => \"Blob write operation\",\n            StoreEvent::BlobDelete => \"Blob delete operation\",\n            StoreEvent::DataIterate => \"Data store iteration operation\",\n            StoreEvent::HttpStoreFetch => \"HTTP store updated\",\n            StoreEvent::HttpStoreError => \"Error updating HTTP store\",\n            StoreEvent::CacheMiss => \"Cache miss\",\n            StoreEvent::CacheHit => \"Cache hit\",\n            StoreEvent::CacheStale => \"Cache is stale\",\n            StoreEvent::CacheUpdate => \"Cache update\",\n            StoreEvent::MeilisearchError => \"Meilisearch error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            StoreEvent::AssertValueFailed => \"Another process modified the record\",\n            StoreEvent::FoundationdbError => \"A FoundationDB error occurred\",\n            StoreEvent::MysqlError => \"A MySQL error occurred\",\n            StoreEvent::PostgresqlError => \"A PostgreSQL error occurred\",\n            StoreEvent::RocksdbError => \"A RocksDB error occurred\",\n            StoreEvent::SqliteError => \"An SQLite error occurred\",\n            StoreEvent::LdapError => \"An LDAP error occurred\",\n            StoreEvent::ElasticsearchError => \"An ElasticSearch error occurred\",\n            StoreEvent::RedisError => \"A Redis error occurred\",\n            StoreEvent::S3Error => \"An S3 error occurred\",\n            StoreEvent::AzureError => \"An Azure error occurred\",\n            StoreEvent::FilesystemError => \"A filesystem error occurred\",\n            StoreEvent::PoolError => \"A connection pool error occurred\",\n            StoreEvent::DataCorruption => \"Data corruption was detected\",\n            StoreEvent::DecompressError => \"A decompression error occurred\",\n            StoreEvent::DeserializeError => \"A deserialization error occurred\",\n            StoreEvent::NotFound => \"The record was not found in the database\",\n            StoreEvent::NotConfigured => \"The store is not configured\",\n            StoreEvent::NotSupported => \"The operation is not supported by the store\",\n            StoreEvent::UnexpectedError => \"An unexpected store error occurred\",\n            StoreEvent::CryptoError => \"A store crypto error occurred\",\n            StoreEvent::BlobMissingMarker => \"The blob is missing a marker\",\n            StoreEvent::SqlQuery => \"An SQL query was executed\",\n            StoreEvent::LdapQuery => \"An LDAP query was executed\",\n            StoreEvent::LdapWarning => \"An LDAP authentication warning occurred\",\n            StoreEvent::DataWrite => \"A write batch operation was executed\",\n            StoreEvent::BlobRead => \"A blob read operation was executed\",\n            StoreEvent::BlobWrite => \"A blob write operation was executed\",\n            StoreEvent::BlobDelete => \"A blob delete operation was executed\",\n            StoreEvent::DataIterate => \"A data store iteration operation was executed\",\n            StoreEvent::HttpStoreFetch => \"The HTTP store was updated\",\n            StoreEvent::HttpStoreError => \"An error occurred while updating the HTTP store\",\n            StoreEvent::CacheMiss => \"No cache entry found for the account\",\n            StoreEvent::CacheHit => \"Cache entry found for the account, no update needed\",\n            StoreEvent::CacheStale => \"Cache is too old, rebuilding\",\n            StoreEvent::CacheUpdate => \"Cache updated with latest database changes\",\n            StoreEvent::MeilisearchError => \"A Meilisearch error occurred\",\n        }\n    }\n}\n\nimpl MessageIngestEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            MessageIngestEvent::Ham => \"Message ingested\",\n            MessageIngestEvent::Spam => \"Possible spam message ingested\",\n            MessageIngestEvent::ImapAppend => \"Message appended via IMAP\",\n            MessageIngestEvent::JmapAppend => \"Message appended via JMAP\",\n            MessageIngestEvent::Duplicate => \"Skipping duplicate message\",\n            MessageIngestEvent::Error => \"Message ingestion error\",\n            MessageIngestEvent::FtsIndex => \"Full-text search index updated\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            MessageIngestEvent::Ham => \"The message has been ingested\",\n            MessageIngestEvent::Spam => \"A possible spam message has been ingested\",\n            MessageIngestEvent::ImapAppend => \"The message has been appended via IMAP\",\n            MessageIngestEvent::JmapAppend => \"The message has been appended via JMAP\",\n            MessageIngestEvent::Duplicate => \"The message is a duplicate and has been skipped\",\n            MessageIngestEvent::Error => \"An error occurred while ingesting the message\",\n            MessageIngestEvent::FtsIndex => \"The full-text search index has been updated\",\n        }\n    }\n}\n\nimpl JmapEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            JmapEvent::MethodCall => \"JMAP method call\",\n            JmapEvent::InvalidArguments => \"Invalid JMAP arguments\",\n            JmapEvent::RequestTooLarge => \"JMAP request too large\",\n            JmapEvent::StateMismatch => \"JMAP state mismatch\",\n            JmapEvent::AnchorNotFound => \"JMAP anchor not found\",\n            JmapEvent::UnsupportedFilter => \"Unsupported JMAP filter\",\n            JmapEvent::UnsupportedSort => \"Unsupported JMAP sort\",\n            JmapEvent::UnknownMethod => \"Unknown JMAP method\",\n            JmapEvent::InvalidResultReference => \"Invalid JMAP result reference\",\n            JmapEvent::Forbidden => \"JMAP operation forbidden\",\n            JmapEvent::AccountNotFound => \"JMAP account not found\",\n            JmapEvent::AccountNotSupportedByMethod => \"JMAP account not supported by method\",\n            JmapEvent::AccountReadOnly => \"JMAP account is read-only\",\n            JmapEvent::NotFound => \"JMAP resource not found\",\n            JmapEvent::CannotCalculateChanges => \"Cannot calculate JMAP changes\",\n            JmapEvent::UnknownDataType => \"Unknown JMAP data type\",\n            JmapEvent::UnknownCapability => \"Unknown JMAP capability\",\n            JmapEvent::NotJson => \"JMAP request is not JSON\",\n            JmapEvent::NotRequest => \"JMAP input is not a request\",\n            JmapEvent::WebsocketStart => \"JMAP WebSocket connection started\",\n            JmapEvent::WebsocketStop => \"JMAP WebSocket connection stopped\",\n            JmapEvent::WebsocketError => \"JMAP WebSocket error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            JmapEvent::MethodCall => \"A JMAP method call has been made\",\n            JmapEvent::InvalidArguments => \"The JMAP arguments are invalid\",\n            JmapEvent::RequestTooLarge => \"The JMAP request is too large\",\n            JmapEvent::StateMismatch => \"The JMAP state is mismatched\",\n            JmapEvent::AnchorNotFound => \"The JMAP anchor was not found\",\n            JmapEvent::UnsupportedFilter => \"The JMAP filter is unsupported\",\n            JmapEvent::UnsupportedSort => \"The JMAP sort is unsupported\",\n            JmapEvent::UnknownMethod => \"The JMAP method is unknown\",\n            JmapEvent::InvalidResultReference => \"The JMAP result reference is invalid\",\n            JmapEvent::Forbidden => \"The JMAP operation is forbidden\",\n            JmapEvent::AccountNotFound => \"The JMAP account was not found\",\n            JmapEvent::AccountNotSupportedByMethod => {\n                \"The JMAP account is not supported by the method\"\n            }\n            JmapEvent::AccountReadOnly => \"The JMAP account is read-only\",\n            JmapEvent::NotFound => \"The JMAP resource was not found\",\n            JmapEvent::CannotCalculateChanges => \"Cannot calculate JMAP changes\",\n            JmapEvent::UnknownDataType => \"The JMAP data type is unknown\",\n            JmapEvent::UnknownCapability => \"The JMAP capability is unknown\",\n            JmapEvent::NotJson => \"The JMAP request is not JSON\",\n            JmapEvent::NotRequest => \"The JMAP input is not a request\",\n            JmapEvent::WebsocketStart => \"The JMAP WebSocket connection has started\",\n            JmapEvent::WebsocketStop => \"The JMAP WebSocket connection has stopped\",\n            JmapEvent::WebsocketError => \"An error occurred with the JMAP WebSocket connection\",\n        }\n    }\n}\n\nimpl LimitEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            LimitEvent::SizeRequest => \"Request size limit reached\",\n            LimitEvent::SizeUpload => \"Upload size limit reached\",\n            LimitEvent::CallsIn => \"Incoming calls limit reached\",\n            LimitEvent::ConcurrentRequest => \"Concurrent request limit reached\",\n            LimitEvent::ConcurrentUpload => \"Concurrent upload limit reached\",\n            LimitEvent::ConcurrentConnection => \"Concurrent connection limit reached\",\n            LimitEvent::Quota => \"Quota limit reached\",\n            LimitEvent::BlobQuota => \"Blob quota limit reached\",\n            LimitEvent::TooManyRequests => \"Too many requests\",\n            LimitEvent::TenantQuota => \"Tenant quota limit reached\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            LimitEvent::SizeRequest => \"The request size limit has been reached\",\n            LimitEvent::SizeUpload => \"The upload size limit has been reached\",\n            LimitEvent::CallsIn => \"The incoming calls limit has been reached\",\n            LimitEvent::ConcurrentRequest => \"The concurrent request limit has been reached\",\n            LimitEvent::ConcurrentUpload => \"The concurrent upload limit has been reached\",\n            LimitEvent::ConcurrentConnection => \"The concurrent connection limit has been reached\",\n            LimitEvent::Quota => \"The quota limit has been reached\",\n            LimitEvent::BlobQuota => \"The blob quota limit has been reached\",\n            LimitEvent::TooManyRequests => \"Too many requests have been made\",\n            LimitEvent::TenantQuota => \"One of the tenant quota limits has been reached\",\n        }\n    }\n}\n\nimpl ManageEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ManageEvent::MissingParameter => \"Missing parameter\",\n            ManageEvent::AlreadyExists => \"Record already exists\",\n            ManageEvent::AssertFailed => \"Assertion failed\",\n            ManageEvent::NotFound => \"Resource not found\",\n            ManageEvent::NotSupported => \"Management operation not supported\",\n            ManageEvent::Error => \"Management error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ManageEvent::MissingParameter => \"A parameter is missing\",\n            ManageEvent::AlreadyExists => \"A record with the same name already exists\",\n            ManageEvent::AssertFailed => \"A management assertion has failed\",\n            ManageEvent::NotFound => \"The managed resource was not found\",\n            ManageEvent::NotSupported => \"The management operation is not supported\",\n            ManageEvent::Error => \"A management error occurred\",\n        }\n    }\n}\n\nimpl AuthEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            AuthEvent::Success => \"Authentication successful\",\n            AuthEvent::Failed => \"Authentication failed\",\n            AuthEvent::MissingTotp => \"Missing TOTP for authentication\",\n            AuthEvent::TooManyAttempts => \"Too many authentication attempts\",\n            AuthEvent::Error => \"Authentication error\",\n            AuthEvent::TokenExpired => \"OAuth token expired\",\n            AuthEvent::ClientRegistration => \"OAuth Client registration\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            AuthEvent::Success => \"Successful authentication\",\n            AuthEvent::Failed => \"Failed authentication\",\n            AuthEvent::MissingTotp => \"TOTP is missing for authentication\",\n            AuthEvent::TooManyAttempts => \"Too many authentication attempts have been made\",\n            AuthEvent::Error => \"An error occurred with authentication\",\n            AuthEvent::TokenExpired => \"OAuth authentication token has expired\",\n            AuthEvent::ClientRegistration => \"OAuth client successfully registered\",\n        }\n    }\n}\n\nimpl ResourceEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            ResourceEvent::NotFound => \"Resource not found\",\n            ResourceEvent::BadParameters => \"Bad resource parameters\",\n            ResourceEvent::Error => \"Resource error\",\n            ResourceEvent::DownloadExternal => \"Downloading external resource\",\n            ResourceEvent::WebadminUnpacked => \"Webadmin resource unpacked\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            ResourceEvent::NotFound => \"The resource was not found\",\n            ResourceEvent::BadParameters => \"The resource parameters are bad\",\n            ResourceEvent::Error => \"An error occurred with the resource\",\n            ResourceEvent::DownloadExternal => \"The external resource is being downloaded\",\n            ResourceEvent::WebadminUnpacked => \"The webadmin resource has been unpacked\",\n        }\n    }\n}\n\nimpl SecurityEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            SecurityEvent::AuthenticationBan => \"Banned due to authentication errors\",\n            SecurityEvent::AbuseBan => \"Banned due to abuse\",\n            SecurityEvent::LoiterBan => \"Banned due to loitering\",\n            SecurityEvent::IpBlocked => \"Blocked IP address\",\n            SecurityEvent::ScanBan => \"Banned due to scan\",\n            SecurityEvent::Unauthorized => \"Unauthorized access\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            SecurityEvent::AuthenticationBan => {\n                \"IP address was banned due to multiple authentication errors\"\n            }\n            SecurityEvent::AbuseBan => {\n                \"IP address was banned due to abuse, such as RCPT TO attacks\"\n            }\n            SecurityEvent::ScanBan => \"IP address was banned due to exploit scanning\",\n            SecurityEvent::LoiterBan => \"IP address was banned due to multiple loitering events\",\n            SecurityEvent::IpBlocked => \"Rejected connection from blocked IP address\",\n            SecurityEvent::Unauthorized => \"Account does not have permission to access resource\",\n        }\n    }\n}\n\nimpl AiEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            AiEvent::LlmResponse => \"LLM response\",\n            AiEvent::ApiError => \"AI API error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            AiEvent::LlmResponse => \"An LLM response has been received\",\n            AiEvent::ApiError => \"An AI API error occurred\",\n        }\n    }\n}\n\nimpl WebDavEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            WebDavEvent::Propfind => \"WebDAV PROPFIND request\",\n            WebDavEvent::Proppatch => \"WebDAV PROPPATCH request\",\n            WebDavEvent::Get => \"WebDAV GET request\",\n            WebDavEvent::Report => \"WebDAV REPORT request\",\n            WebDavEvent::Mkcol => \"WebDAV MKCOL request\",\n            WebDavEvent::Delete => \"WebDAV DELETE request\",\n            WebDavEvent::Put => \"WebDAV PUT request\",\n            WebDavEvent::Post => \"WebDAV POST request\",\n            WebDavEvent::Patch => \"WebDAV PATCH request\",\n            WebDavEvent::Copy => \"WebDAV COPY request\",\n            WebDavEvent::Move => \"WebDAV MOVE request\",\n            WebDavEvent::Lock => \"WebDAV LOCK request\",\n            WebDavEvent::Unlock => \"WebDAV UNLOCK request\",\n            WebDavEvent::Acl => \"WebDAV ACL request\",\n            WebDavEvent::Error => \"WebDAV error\",\n            WebDavEvent::Head => \"WebDAV HEAD request\",\n            WebDavEvent::Mkcalendar => \"WebDAV MKCALENDAR request\",\n            WebDavEvent::Options => \"WebDAV OPTIONS request\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            WebDavEvent::Propfind => \"A PROPFIND request has been made to the server\",\n            WebDavEvent::Proppatch => \"A PROPPATCH request has been made to the server\",\n            WebDavEvent::Get => \"A GET request has been made to the server\",\n            WebDavEvent::Report => \"A REPORT request has been made to the server\",\n            WebDavEvent::Mkcol => \"A MKCOL request has been made to the server\",\n            WebDavEvent::Delete => \"A DELETE request has been made to the server\",\n            WebDavEvent::Put => \"A PUT request has been made to the server\",\n            WebDavEvent::Post => \"A POST request has been made to the server\",\n            WebDavEvent::Patch => \"A PATCH request has been made to the server\",\n            WebDavEvent::Copy => \"A COPY request has been made to the server\",\n            WebDavEvent::Move => \"A MOVE request has been made to the server\",\n            WebDavEvent::Lock => \"A LOCK request has been made to the server\",\n            WebDavEvent::Unlock => \"An UNLOCK request has been made to the server\",\n            WebDavEvent::Acl => {\n                \"An ACL request has been made to the\n                server\"\n            }\n            WebDavEvent::Error => \"An error occurred with the WebDAV request\",\n            WebDavEvent::Head => \"A HEAD request has been made to the server\",\n            WebDavEvent::Mkcalendar => \"A MKCALENDAR request has been made to the server\",\n            WebDavEvent::Options => \"An OPTIONS request has been made to the server\",\n        }\n    }\n}\n\nimpl CalendarEvent {\n    pub fn description(&self) -> &'static str {\n        match self {\n            CalendarEvent::RuleExpansionError => \"Calendar rule expansion error\",\n            CalendarEvent::AlarmSent => \"Calendar alarm sent\",\n            CalendarEvent::AlarmSkipped => \"Calendar alarm skipped\",\n            CalendarEvent::AlarmRecipientOverride => \"Calendar alarm recipient overriden\",\n            CalendarEvent::AlarmFailed => \"Calendar alarm could not be sent\",\n            CalendarEvent::ItipMessageSent => \"Calendar iTIP message sent\",\n            CalendarEvent::ItipMessageReceived => \"Calendar iTIP message received\",\n            CalendarEvent::ItipMessageError => \"iTIP message error\",\n        }\n    }\n\n    pub fn explain(&self) -> &'static str {\n        match self {\n            CalendarEvent::RuleExpansionError => {\n                \"An error occurred while expanding calendar recurrences\"\n            }\n            CalendarEvent::AlarmSent => \"A calendar alarm has been sent to the recipient\",\n            CalendarEvent::AlarmSkipped => \"A calendar alarm was skipped\",\n            CalendarEvent::AlarmRecipientOverride => \"A calendar alarm recipient was overridden\",\n            CalendarEvent::AlarmFailed => \"A calendar alarm could not be sent to the recipient\",\n            CalendarEvent::ItipMessageSent => \"A calendar iTIP message has been sent\",\n            CalendarEvent::ItipMessageReceived => \"A calendar iTIP/iMIP message has been received\",\n            CalendarEvent::ItipMessageError => {\n                \"An error occurred while processing an iTIP/iMIP message\"\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/event/level.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{cmp::Ordering, fmt::Display, str::FromStr};\n\nuse super::*;\n\nimpl EventType {\n    pub fn level(&self) -> Level {\n        match self {\n            EventType::Store(event) => match event {\n                StoreEvent::DataWrite\n                | StoreEvent::DataIterate\n                | StoreEvent::BlobRead\n                | StoreEvent::BlobWrite\n                | StoreEvent::BlobDelete\n                | StoreEvent::SqlQuery\n                | StoreEvent::LdapQuery => Level::Trace,\n                StoreEvent::CacheMiss\n                | StoreEvent::CacheHit\n                | StoreEvent::CacheStale\n                | StoreEvent::CacheUpdate\n                | StoreEvent::NotFound\n                | StoreEvent::HttpStoreFetch\n                | StoreEvent::LdapWarning => Level::Debug,\n                StoreEvent::AssertValueFailed\n                | StoreEvent::FoundationdbError\n                | StoreEvent::MysqlError\n                | StoreEvent::PostgresqlError\n                | StoreEvent::RocksdbError\n                | StoreEvent::SqliteError\n                | StoreEvent::LdapError\n                | StoreEvent::ElasticsearchError\n                | StoreEvent::MeilisearchError\n                | StoreEvent::RedisError\n                | StoreEvent::S3Error\n                | StoreEvent::AzureError\n                | StoreEvent::FilesystemError\n                | StoreEvent::PoolError\n                | StoreEvent::DataCorruption\n                | StoreEvent::DecompressError\n                | StoreEvent::DeserializeError\n                | StoreEvent::NotConfigured\n                | StoreEvent::NotSupported\n                | StoreEvent::UnexpectedError\n                | StoreEvent::CryptoError => Level::Error,\n                StoreEvent::BlobMissingMarker | StoreEvent::HttpStoreError => Level::Warn,\n            },\n            EventType::Jmap(_) => Level::Debug,\n            EventType::Imap(event) => match event {\n                ImapEvent::ConnectionStart | ImapEvent::ConnectionEnd => Level::Debug,\n                ImapEvent::GetAcl\n                | ImapEvent::SetAcl\n                | ImapEvent::MyRights\n                | ImapEvent::ListRights\n                | ImapEvent::Append\n                | ImapEvent::Capabilities\n                | ImapEvent::Id\n                | ImapEvent::Close\n                | ImapEvent::Copy\n                | ImapEvent::Move\n                | ImapEvent::CreateMailbox\n                | ImapEvent::DeleteMailbox\n                | ImapEvent::RenameMailbox\n                | ImapEvent::Enable\n                | ImapEvent::Expunge\n                | ImapEvent::Fetch\n                | ImapEvent::List\n                | ImapEvent::Lsub\n                | ImapEvent::Logout\n                | ImapEvent::Namespace\n                | ImapEvent::Noop\n                | ImapEvent::Search\n                | ImapEvent::Sort\n                | ImapEvent::Select\n                | ImapEvent::Status\n                | ImapEvent::Store\n                | ImapEvent::Subscribe\n                | ImapEvent::Unsubscribe\n                | ImapEvent::Thread\n                | ImapEvent::Error\n                | ImapEvent::IdleStart\n                | ImapEvent::IdleStop\n                | ImapEvent::GetQuota => Level::Debug,\n                ImapEvent::RawInput | ImapEvent::RawOutput => Level::Trace,\n            },\n            EventType::ManageSieve(event) => match event {\n                ManageSieveEvent::ConnectionStart | ManageSieveEvent::ConnectionEnd => Level::Debug,\n                ManageSieveEvent::CreateScript\n                | ManageSieveEvent::UpdateScript\n                | ManageSieveEvent::GetScript\n                | ManageSieveEvent::DeleteScript\n                | ManageSieveEvent::RenameScript\n                | ManageSieveEvent::CheckScript\n                | ManageSieveEvent::HaveSpace\n                | ManageSieveEvent::ListScripts\n                | ManageSieveEvent::SetActive\n                | ManageSieveEvent::Capabilities\n                | ManageSieveEvent::StartTls\n                | ManageSieveEvent::Unauthenticate\n                | ManageSieveEvent::Logout\n                | ManageSieveEvent::Noop\n                | ManageSieveEvent::Error => Level::Debug,\n                ManageSieveEvent::RawInput | ManageSieveEvent::RawOutput => Level::Trace,\n            },\n            EventType::Pop3(event) => match event {\n                Pop3Event::ConnectionStart | Pop3Event::ConnectionEnd => Level::Debug,\n                Pop3Event::Delete\n                | Pop3Event::Reset\n                | Pop3Event::Quit\n                | Pop3Event::Fetch\n                | Pop3Event::List\n                | Pop3Event::ListMessage\n                | Pop3Event::Uidl\n                | Pop3Event::UidlMessage\n                | Pop3Event::Stat\n                | Pop3Event::Noop\n                | Pop3Event::Capabilities\n                | Pop3Event::StartTls\n                | Pop3Event::Utf8\n                | Pop3Event::Error => Level::Debug,\n                Pop3Event::RawInput | Pop3Event::RawOutput => Level::Trace,\n            },\n            EventType::Smtp(event) => match event {\n                SmtpEvent::ConnectionStart | SmtpEvent::ConnectionEnd => Level::Debug,\n                SmtpEvent::DidNotSayEhlo\n                | SmtpEvent::EhloExpected\n                | SmtpEvent::LhloExpected\n                | SmtpEvent::MailFromUnauthenticated\n                | SmtpEvent::MailFromUnauthorized\n                | SmtpEvent::MailFromRewritten\n                | SmtpEvent::MailFromMissing\n                | SmtpEvent::MultipleMailFrom\n                | SmtpEvent::MailFromNotAllowed\n                | SmtpEvent::RcptToDuplicate\n                | SmtpEvent::RcptToRewritten\n                | SmtpEvent::RcptToMissing\n                | SmtpEvent::RequireTlsDisabled\n                | SmtpEvent::DeliverByDisabled\n                | SmtpEvent::DeliverByInvalid\n                | SmtpEvent::FutureReleaseDisabled\n                | SmtpEvent::FutureReleaseInvalid\n                | SmtpEvent::MtPriorityDisabled\n                | SmtpEvent::MtPriorityInvalid\n                | SmtpEvent::DsnDisabled\n                | SmtpEvent::AuthExchangeTooLong\n                | SmtpEvent::AlreadyAuthenticated\n                | SmtpEvent::Noop\n                | SmtpEvent::StartTls\n                | SmtpEvent::StartTlsUnavailable\n                | SmtpEvent::StartTlsAlready\n                | SmtpEvent::Rset\n                | SmtpEvent::Quit\n                | SmtpEvent::Help\n                | SmtpEvent::CommandNotImplemented\n                | SmtpEvent::InvalidCommand\n                | SmtpEvent::InvalidSenderAddress\n                | SmtpEvent::InvalidRecipientAddress\n                | SmtpEvent::InvalidParameter\n                | SmtpEvent::UnsupportedParameter\n                | SmtpEvent::SyntaxError\n                | SmtpEvent::Error => Level::Debug,\n                SmtpEvent::MissingLocalHostname | SmtpEvent::IdNotFound => Level::Warn,\n                SmtpEvent::ConcurrencyLimitExceeded\n                | SmtpEvent::TransferLimitExceeded\n                | SmtpEvent::RateLimitExceeded\n                | SmtpEvent::TimeLimitExceeded\n                | SmtpEvent::MissingAuthDirectory\n                | SmtpEvent::MessageParseFailed\n                | SmtpEvent::MessageTooLarge\n                | SmtpEvent::LoopDetected\n                | SmtpEvent::DkimPass\n                | SmtpEvent::DkimFail\n                | SmtpEvent::ArcPass\n                | SmtpEvent::ArcFail\n                | SmtpEvent::SpfEhloPass\n                | SmtpEvent::SpfEhloFail\n                | SmtpEvent::SpfFromPass\n                | SmtpEvent::SpfFromFail\n                | SmtpEvent::DmarcPass\n                | SmtpEvent::DmarcFail\n                | SmtpEvent::IprevPass\n                | SmtpEvent::IprevFail\n                | SmtpEvent::TooManyMessages\n                | SmtpEvent::Ehlo\n                | SmtpEvent::InvalidEhlo\n                | SmtpEvent::MailFrom\n                | SmtpEvent::MailboxDoesNotExist\n                | SmtpEvent::RelayNotAllowed\n                | SmtpEvent::RcptTo\n                | SmtpEvent::RcptToGreylisted\n                | SmtpEvent::TooManyInvalidRcpt\n                | SmtpEvent::Vrfy\n                | SmtpEvent::VrfyNotFound\n                | SmtpEvent::VrfyDisabled\n                | SmtpEvent::Expn\n                | SmtpEvent::ExpnNotFound\n                | SmtpEvent::AuthNotAllowed\n                | SmtpEvent::AuthMechanismNotSupported\n                | SmtpEvent::ExpnDisabled\n                | SmtpEvent::RequestTooLarge\n                | SmtpEvent::TooManyRecipients => Level::Info,\n                SmtpEvent::RawInput | SmtpEvent::RawOutput => Level::Trace,\n            },\n            EventType::Network(event) => match event {\n                NetworkEvent::ReadError\n                | NetworkEvent::WriteError\n                | NetworkEvent::FlushError\n                | NetworkEvent::Closed => Level::Trace,\n                NetworkEvent::Timeout | NetworkEvent::AcceptError => Level::Debug,\n                NetworkEvent::ListenStart | NetworkEvent::ListenStop => Level::Info,\n                NetworkEvent::ListenError\n                | NetworkEvent::BindError\n                | NetworkEvent::SetOptError\n                | NetworkEvent::SplitError => Level::Error,\n                NetworkEvent::ProxyError => Level::Warn,\n            },\n            EventType::Limit(cause) => match cause {\n                LimitEvent::SizeRequest => Level::Debug,\n                LimitEvent::SizeUpload => Level::Debug,\n                LimitEvent::CallsIn => Level::Debug,\n                LimitEvent::ConcurrentRequest => Level::Debug,\n                LimitEvent::ConcurrentUpload => Level::Debug,\n                LimitEvent::ConcurrentConnection => Level::Warn,\n                LimitEvent::Quota => Level::Debug,\n                LimitEvent::BlobQuota => Level::Debug,\n                LimitEvent::TooManyRequests => Level::Warn,\n                LimitEvent::TenantQuota => Level::Info,\n            },\n            EventType::Manage(_) => Level::Debug,\n            EventType::Auth(cause) => match cause {\n                AuthEvent::Failed | AuthEvent::TokenExpired => Level::Debug,\n                AuthEvent::MissingTotp => Level::Trace,\n                AuthEvent::TooManyAttempts => Level::Warn,\n                AuthEvent::Error => Level::Error,\n                AuthEvent::Success | AuthEvent::ClientRegistration => Level::Info,\n            },\n            EventType::Config(cause) => match cause {\n                ConfigEvent::ParseError\n                | ConfigEvent::BuildError\n                | ConfigEvent::MacroError\n                | ConfigEvent::WriteError\n                | ConfigEvent::FetchError => Level::Error,\n                ConfigEvent::DefaultApplied\n                | ConfigEvent::MissingSetting\n                | ConfigEvent::UnusedSetting\n                | ConfigEvent::AlreadyUpToDate => Level::Debug,\n                ConfigEvent::ParseWarning | ConfigEvent::BuildWarning => Level::Warn,\n                ConfigEvent::ImportExternal => Level::Info,\n            },\n            EventType::Resource(cause) => match cause {\n                ResourceEvent::NotFound => Level::Debug,\n                ResourceEvent::BadParameters | ResourceEvent::Error => Level::Error,\n                ResourceEvent::DownloadExternal | ResourceEvent::WebadminUnpacked => Level::Info,\n            },\n            EventType::Arc(event) => match event {\n                ArcEvent::ChainTooLong\n                | ArcEvent::InvalidInstance\n                | ArcEvent::InvalidCv\n                | ArcEvent::HasHeaderTag\n                | ArcEvent::BrokenChain => Level::Debug,\n                ArcEvent::SealerNotFound => Level::Warn,\n            },\n            EventType::Dkim(event) => match event {\n                DkimEvent::SignerNotFound => Level::Warn,\n                _ => Level::Debug,\n            },\n            EventType::MailAuth(_) => Level::Debug,\n            EventType::Purge(event) => match event {\n                PurgeEvent::Started => Level::Debug,\n                PurgeEvent::Finished => Level::Debug,\n                PurgeEvent::Running => Level::Info,\n                PurgeEvent::Error => Level::Error,\n                PurgeEvent::BlobCleanup => Level::Info,\n                PurgeEvent::InProgress | PurgeEvent::AutoExpunge => Level::Debug,\n            },\n            EventType::Eval(event) => match event {\n                EvalEvent::Error | EvalEvent::StoreNotFound => Level::Debug,\n                EvalEvent::Result => Level::Trace,\n                EvalEvent::DirectoryNotFound => Level::Warn,\n            },\n            EventType::Server(event) => match event {\n                ServerEvent::Startup | ServerEvent::Shutdown | ServerEvent::Licensing => {\n                    Level::Info\n                }\n                ServerEvent::StartupError | ServerEvent::ThreadError => Level::Error,\n            },\n            EventType::Acme(event) => match event {\n                AcmeEvent::DnsRecordCreated\n                | AcmeEvent::DnsRecordPropagated\n                | AcmeEvent::TlsAlpnReceived\n                | AcmeEvent::AuthStart\n                | AcmeEvent::AuthPending\n                | AcmeEvent::AuthValid\n                | AcmeEvent::AuthCompleted\n                | AcmeEvent::ProcessCert\n                | AcmeEvent::OrderProcessing\n                | AcmeEvent::OrderReady\n                | AcmeEvent::OrderValid\n                | AcmeEvent::OrderStart\n                | AcmeEvent::OrderCompleted => Level::Info,\n                AcmeEvent::Error => Level::Error,\n                AcmeEvent::OrderInvalid\n                | AcmeEvent::AuthError\n                | AcmeEvent::AuthTooManyAttempts\n                | AcmeEvent::TokenNotFound\n                | AcmeEvent::DnsRecordPropagationTimeout\n                | AcmeEvent::TlsAlpnError\n                | AcmeEvent::DnsRecordCreationFailed => Level::Warn,\n                AcmeEvent::RenewBackoff\n                | AcmeEvent::DnsRecordDeletionFailed\n                | AcmeEvent::ClientSuppliedSni\n                | AcmeEvent::ClientMissingSni\n                | AcmeEvent::DnsRecordNotPropagated\n                | AcmeEvent::DnsRecordLookupFailed => Level::Debug,\n            },\n            EventType::Tls(event) => match event {\n                TlsEvent::Handshake => Level::Info,\n                TlsEvent::HandshakeError | TlsEvent::CertificateNotFound => Level::Debug,\n                TlsEvent::NotConfigured => Level::Error,\n                TlsEvent::NoCertificatesAvailable | TlsEvent::MultipleCertificatesAvailable => {\n                    Level::Warn\n                }\n            },\n            EventType::Sieve(event) => match event {\n                SieveEvent::NotSupported\n                | SieveEvent::QuotaExceeded\n                | SieveEvent::ListNotFound\n                | SieveEvent::ScriptNotFound\n                | SieveEvent::MessageTooLarge => Level::Warn,\n                SieveEvent::SendMessage => Level::Info,\n                SieveEvent::UnexpectedError => Level::Error,\n                SieveEvent::ActionAccept\n                | SieveEvent::RuntimeError\n                | SieveEvent::ActionAcceptReplace\n                | SieveEvent::ActionDiscard\n                | SieveEvent::ActionReject => Level::Debug,\n            },\n            EventType::Spam(event) => match event {\n                SpamEvent::Pyzor\n                | SpamEvent::PyzorError\n                | SpamEvent::Dnsbl\n                | SpamEvent::DnsblError\n                | SpamEvent::Classify\n                | SpamEvent::TrainSampleAdded => Level::Debug,\n                SpamEvent::TrainSampleNotFound => Level::Warn,\n                SpamEvent::TrainStarted\n                | SpamEvent::TrainCompleted\n                | SpamEvent::ModelLoaded\n                | SpamEvent::ModelNotReady\n                | SpamEvent::ModelNotFound => Level::Info,\n            },\n            EventType::Http(event) => match event {\n                HttpEvent::ConnectionStart | HttpEvent::ConnectionEnd => Level::Debug,\n                HttpEvent::XForwardedMissing => Level::Warn,\n                HttpEvent::Error | HttpEvent::RequestUrl => Level::Debug,\n                HttpEvent::RequestBody | HttpEvent::ResponseBody => Level::Trace,\n            },\n            EventType::PushSubscription(event) => match event {\n                PushSubscriptionEvent::Error | PushSubscriptionEvent::NotFound => Level::Debug,\n                PushSubscriptionEvent::Success => Level::Trace,\n            },\n            EventType::Cluster(event) => match event {\n                ClusterEvent::SubscriberStart\n                | ClusterEvent::SubscriberStop\n                | ClusterEvent::PublisherStart\n                | ClusterEvent::PublisherStop => Level::Info,\n                ClusterEvent::SubscriberDisconnected => Level::Warn,\n                ClusterEvent::MessageReceived | ClusterEvent::MessageSkipped => Level::Trace,\n                ClusterEvent::PublisherError\n                | ClusterEvent::SubscriberError\n                | ClusterEvent::MessageInvalid => Level::Error,\n            },\n            EventType::Housekeeper(event) => match event {\n                HousekeeperEvent::Start | HousekeeperEvent::Stop => Level::Info,\n                HousekeeperEvent::Run | HousekeeperEvent::Schedule => Level::Debug,\n            },\n            EventType::TaskQueue(event) => match event {\n                TaskQueueEvent::BlobNotFound\n                | TaskQueueEvent::TaskAcquired\n                | TaskQueueEvent::TaskLocked\n                | TaskQueueEvent::TaskIgnored\n                | TaskQueueEvent::MetadataNotFound => Level::Debug,\n                TaskQueueEvent::TaskFailed => Level::Warn,\n            },\n            EventType::Dmarc(_) => Level::Debug,\n            EventType::Spf(_) => Level::Debug,\n            EventType::Iprev(_) => Level::Debug,\n            EventType::Milter(event) => match event {\n                MilterEvent::Read | MilterEvent::Write => Level::Trace,\n                MilterEvent::ActionAccept\n                | MilterEvent::ActionDiscard\n                | MilterEvent::ActionReject\n                | MilterEvent::ActionTempFail\n                | MilterEvent::ActionReplyCode\n                | MilterEvent::ActionConnectionFailure\n                | MilterEvent::ActionShutdown => Level::Info,\n                MilterEvent::IoError\n                | MilterEvent::FrameTooLarge\n                | MilterEvent::FrameInvalid\n                | MilterEvent::UnexpectedResponse\n                | MilterEvent::Timeout\n                | MilterEvent::TlsInvalidName\n                | MilterEvent::Disconnected\n                | MilterEvent::ParseError => Level::Warn,\n            },\n            EventType::MtaHook(event) => match event {\n                MtaHookEvent::ActionAccept\n                | MtaHookEvent::ActionDiscard\n                | MtaHookEvent::ActionReject\n                | MtaHookEvent::ActionQuarantine => Level::Info,\n                MtaHookEvent::Error => Level::Warn,\n            },\n            EventType::Dane(event) => match event {\n                DaneEvent::AuthenticationSuccess\n                | DaneEvent::AuthenticationFailure\n                | DaneEvent::NoCertificatesFound\n                | DaneEvent::CertificateParseError\n                | DaneEvent::TlsaRecordMatch\n                | DaneEvent::TlsaRecordFetch\n                | DaneEvent::TlsaRecordFetchError\n                | DaneEvent::TlsaRecordNotFound\n                | DaneEvent::TlsaRecordNotDnssecSigned\n                | DaneEvent::TlsaRecordInvalid => Level::Info,\n            },\n            EventType::Delivery(event) => match event {\n                DeliveryEvent::AttemptStart\n                | DeliveryEvent::AttemptEnd\n                | DeliveryEvent::Completed\n                | DeliveryEvent::Failed\n                | DeliveryEvent::DomainDeliveryStart\n                | DeliveryEvent::MxLookupFailed\n                | DeliveryEvent::IpLookupFailed\n                | DeliveryEvent::NullMx\n                | DeliveryEvent::Connect\n                | DeliveryEvent::ConnectError\n                | DeliveryEvent::GreetingFailed\n                | DeliveryEvent::EhloRejected\n                | DeliveryEvent::AuthFailed\n                | DeliveryEvent::MailFromRejected\n                | DeliveryEvent::Delivered\n                | DeliveryEvent::RcptToRejected\n                | DeliveryEvent::RcptToFailed\n                | DeliveryEvent::MessageRejected\n                | DeliveryEvent::StartTls\n                | DeliveryEvent::StartTlsUnavailable\n                | DeliveryEvent::StartTlsError\n                | DeliveryEvent::StartTlsDisabled\n                | DeliveryEvent::ImplicitTlsError\n                | DeliveryEvent::DoubleBounce => Level::Info,\n                DeliveryEvent::ConcurrencyLimitExceeded\n                | DeliveryEvent::RateLimitExceeded\n                | DeliveryEvent::MissingOutboundHostname => Level::Warn,\n                DeliveryEvent::DsnSuccess\n                | DeliveryEvent::DsnTempFail\n                | DeliveryEvent::DsnPermFail => Level::Info,\n                DeliveryEvent::MxLookup\n                | DeliveryEvent::IpLookup\n                | DeliveryEvent::Ehlo\n                | DeliveryEvent::Auth\n                | DeliveryEvent::MailFrom\n                | DeliveryEvent::RcptTo => Level::Debug,\n                DeliveryEvent::RawInput | DeliveryEvent::RawOutput => Level::Trace,\n            },\n            EventType::Queue(event) => match event {\n                QueueEvent::BackPressure => Level::Warn,\n                QueueEvent::QueueMessage\n                | QueueEvent::QueueMessageAuthenticated\n                | QueueEvent::QueueReport\n                | QueueEvent::QueueDsn\n                | QueueEvent::QueueAutogenerated\n                | QueueEvent::RateLimitExceeded\n                | QueueEvent::ConcurrencyLimitExceeded\n                | QueueEvent::Rescheduled\n                | QueueEvent::QuotaExceeded => Level::Info,\n                QueueEvent::Locked | QueueEvent::BlobNotFound => Level::Debug,\n            },\n            EventType::TlsRpt(event) => match event {\n                TlsRptEvent::RecordFetch\n                | TlsRptEvent::RecordFetchError\n                | TlsRptEvent::RecordNotFound => Level::Info,\n            },\n            EventType::MtaSts(event) => match event {\n                MtaStsEvent::PolicyFetch\n                | MtaStsEvent::PolicyNotFound\n                | MtaStsEvent::PolicyFetchError\n                | MtaStsEvent::InvalidPolicy\n                | MtaStsEvent::NotAuthorized\n                | MtaStsEvent::Authorized => Level::Info,\n            },\n            EventType::IncomingReport(event) => match event {\n                IncomingReportEvent::DmarcReportWithWarnings\n                | IncomingReportEvent::TlsReportWithWarnings => Level::Warn,\n                IncomingReportEvent::DmarcReport\n                | IncomingReportEvent::TlsReport\n                | IncomingReportEvent::AbuseReport\n                | IncomingReportEvent::AuthFailureReport\n                | IncomingReportEvent::FraudReport\n                | IncomingReportEvent::NotSpamReport\n                | IncomingReportEvent::VirusReport\n                | IncomingReportEvent::OtherReport\n                | IncomingReportEvent::MessageParseFailed\n                | IncomingReportEvent::DmarcParseFailed\n                | IncomingReportEvent::TlsRpcParseFailed\n                | IncomingReportEvent::ArfParseFailed\n                | IncomingReportEvent::DecompressError => Level::Info,\n            },\n            EventType::OutgoingReport(event) => match event {\n                OutgoingReportEvent::Locked | OutgoingReportEvent::NotFound => Level::Info,\n                OutgoingReportEvent::SpfReport\n                | OutgoingReportEvent::SpfRateLimited\n                | OutgoingReportEvent::DkimReport\n                | OutgoingReportEvent::DkimRateLimited\n                | OutgoingReportEvent::DmarcReport\n                | OutgoingReportEvent::DmarcRateLimited\n                | OutgoingReportEvent::DmarcAggregateReport\n                | OutgoingReportEvent::TlsAggregate\n                | OutgoingReportEvent::HttpSubmission\n                | OutgoingReportEvent::UnauthorizedReportingAddress\n                | OutgoingReportEvent::ReportingAddressValidationError\n                | OutgoingReportEvent::SubmissionError\n                | OutgoingReportEvent::NoRecipientsFound => Level::Info,\n            },\n            EventType::Telemetry(_) => Level::Warn,\n            EventType::MessageIngest(event) => match event {\n                MessageIngestEvent::Ham\n                | MessageIngestEvent::Spam\n                | MessageIngestEvent::ImapAppend\n                | MessageIngestEvent::JmapAppend\n                | MessageIngestEvent::Duplicate\n                | MessageIngestEvent::FtsIndex => Level::Info,\n                MessageIngestEvent::Error => Level::Error,\n            },\n            EventType::Security(_) => Level::Info,\n            EventType::Ai(event) => match event {\n                AiEvent::LlmResponse => Level::Trace,\n                AiEvent::ApiError => Level::Warn,\n            },\n            EventType::WebDav(_) => Level::Debug,\n            EventType::Calendar(event) => match event {\n                CalendarEvent::ItipMessageSent\n                | CalendarEvent::ItipMessageReceived\n                | CalendarEvent::AlarmSent => Level::Info,\n                CalendarEvent::AlarmFailed => Level::Warn,\n                CalendarEvent::RuleExpansionError\n                | CalendarEvent::AlarmSkipped\n                | CalendarEvent::AlarmRecipientOverride\n                | CalendarEvent::ItipMessageError => Level::Debug,\n            },\n        }\n    }\n}\n\nimpl PartialOrd for Level {\n    #[inline(always)]\n    fn partial_cmp(&self, other: &Level) -> Option<Ordering> {\n        Some(self.cmp(other))\n    }\n\n    #[inline(always)]\n    fn lt(&self, other: &Level) -> bool {\n        (*other as usize) < (*self as usize)\n    }\n\n    #[inline(always)]\n    fn le(&self, other: &Level) -> bool {\n        (*other as usize) <= (*self as usize)\n    }\n\n    #[inline(always)]\n    fn gt(&self, other: &Level) -> bool {\n        (*other as usize) > (*self as usize)\n    }\n\n    #[inline(always)]\n    fn ge(&self, other: &Level) -> bool {\n        (*other as usize) >= (*self as usize)\n    }\n}\n\nimpl Ord for Level {\n    #[inline(always)]\n    fn cmp(&self, other: &Self) -> Ordering {\n        (*other as usize).cmp(&(*self as usize))\n    }\n}\n\nimpl FromStr for Level {\n    type Err = String;\n\n    fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {\n        match s.to_ascii_lowercase().as_str() {\n            \"disable\" => Ok(Self::Disable),\n            \"trace\" => Ok(Self::Trace),\n            \"debug\" => Ok(Self::Debug),\n            \"info\" => Ok(Self::Info),\n            \"warn\" => Ok(Self::Warn),\n            \"error\" => Ok(Self::Error),\n            _ => Err(s.to_string()),\n        }\n    }\n}\n\nimpl Level {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Self::Disable => \"DISABLE\",\n            Self::Trace => \"TRACE\",\n            Self::Debug => \"DEBUG\",\n            Self::Info => \"INFO\",\n            Self::Warn => \"WARN\",\n            Self::Error => \"ERROR\",\n        }\n    }\n\n    pub fn is_contained(&self, other: Self) -> bool {\n        *self >= other && other != Level::Disable && *self != Level::Disable\n    }\n}\n\nimpl Display for Level {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        self.as_str().fmt(f)\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/event/metrics.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::MetricType;\n\nimpl MetricType {\n    pub fn name(&self) -> &'static str {\n        match self {\n            Self::MessageIngestionTime => \"message-ingest.time\",\n            Self::MessageFtsIndexTime => \"message-ingest.index-time\",\n            Self::DeliveryTotalTime => \"delivery.total-time\",\n            Self::DeliveryTime => \"delivery.attempt-time\",\n            Self::MessageSize => \"message.size\",\n            Self::MessageAuthSize => \"message.authenticated-size\",\n            Self::ReportOutgoingSize => \"outgoing-report.size\",\n            Self::StoreReadTime => \"store.data-read-time\",\n            Self::StoreWriteTime => \"store.data-write-time\",\n            Self::BlobReadTime => \"store.blob-read-time\",\n            Self::BlobWriteTime => \"store.blob-write-time\",\n            Self::DnsLookupTime => \"dns.lookup-time\",\n            Self::HttpRequestTime => \"http.request-time\",\n            Self::ImapRequestTime => \"imap.request-time\",\n            Self::Pop3RequestTime => \"pop3.request-time\",\n            Self::SmtpRequestTime => \"smtp.request-time\",\n            Self::SieveRequestTime => \"sieve.request-time\",\n            Self::HttpActiveConnections => \"http.active-connections\",\n            Self::ImapActiveConnections => \"imap.active-connections\",\n            Self::Pop3ActiveConnections => \"pop3.active-connections\",\n            Self::SmtpActiveConnections => \"smtp.active-connections\",\n            Self::SieveActiveConnections => \"sieve.active-connections\",\n            Self::DeliveryActiveConnections => \"delivery.active-connections\",\n            Self::ServerMemory => \"server.memory\",\n            Self::QueueCount => \"queue.count\",\n            Self::UserCount => \"user.count\",\n            Self::DomainCount => \"domain.count\",\n        }\n    }\n\n    pub fn description(&self) -> &'static str {\n        match self {\n            Self::MessageIngestionTime => \"Message ingestion time\",\n            Self::MessageFtsIndexTime => \"Message full-text indexing time\",\n            Self::DeliveryTotalTime => \"Total message delivery time from submission to delivery\",\n            Self::DeliveryTime => \"Message delivery time\",\n            Self::MessageSize => \"Received message size\",\n            Self::MessageAuthSize => \"Received message size from authenticated users\",\n            Self::ReportOutgoingSize => \"Outgoing report size\",\n            Self::StoreReadTime => \"Data store read time\",\n            Self::StoreWriteTime => \"Data store write time\",\n            Self::BlobReadTime => \"Blob store read time\",\n            Self::BlobWriteTime => \"Blob store write time\",\n            Self::DnsLookupTime => \"DNS lookup time\",\n            Self::HttpRequestTime => \"HTTP request duration\",\n            Self::ImapRequestTime => \"IMAP request duration\",\n            Self::Pop3RequestTime => \"POP3 request duration\",\n            Self::SmtpRequestTime => \"SMTP request duration\",\n            Self::SieveRequestTime => \"ManageSieve request duration\",\n            Self::HttpActiveConnections => \"Active HTTP connections\",\n            Self::ImapActiveConnections => \"Active IMAP connections\",\n            Self::Pop3ActiveConnections => \"Active POP3 connections\",\n            Self::SmtpActiveConnections => \"Active SMTP connections\",\n            Self::SieveActiveConnections => \"Active ManageSieve connections\",\n            Self::DeliveryActiveConnections => \"Active delivery connections\",\n            Self::ServerMemory => \"Server memory usage\",\n            Self::QueueCount => \"Total number of messages in the queue\",\n            Self::UserCount => \"Total number of users\",\n            Self::DomainCount => \"Total number of domains\",\n        }\n    }\n\n    pub fn unit(&self) -> &'static str {\n        match self {\n            Self::MessageIngestionTime\n            | Self::MessageFtsIndexTime\n            | Self::DeliveryTotalTime\n            | Self::DeliveryTime\n            | Self::StoreReadTime\n            | Self::StoreWriteTime\n            | Self::BlobReadTime\n            | Self::BlobWriteTime\n            | Self::DnsLookupTime\n            | Self::HttpRequestTime\n            | Self::ImapRequestTime\n            | Self::Pop3RequestTime\n            | Self::SmtpRequestTime\n            | Self::SieveRequestTime => \"milliseconds\",\n            Self::MessageSize\n            | Self::MessageAuthSize\n            | Self::ReportOutgoingSize\n            | Self::ServerMemory => \"bytes\",\n            Self::HttpActiveConnections\n            | Self::ImapActiveConnections\n            | Self::Pop3ActiveConnections\n            | Self::SmtpActiveConnections\n            | Self::SieveActiveConnections\n            | Self::DeliveryActiveConnections => \"connections\",\n            Self::QueueCount => \"messages\",\n            Self::UserCount => \"users\",\n            Self::DomainCount => \"domains\",\n        }\n    }\n\n    pub fn code(&self) -> u64 {\n        match self {\n            Self::MessageIngestionTime => 0,\n            Self::MessageFtsIndexTime => 1,\n            Self::DeliveryTotalTime => 2,\n            Self::DeliveryTime => 3,\n            Self::MessageSize => 4,\n            Self::MessageAuthSize => 5,\n            Self::ReportOutgoingSize => 6,\n            Self::StoreReadTime => 7,\n            Self::StoreWriteTime => 8,\n            Self::BlobReadTime => 9,\n            Self::BlobWriteTime => 10,\n            Self::DnsLookupTime => 11,\n            Self::HttpRequestTime => 12,\n            Self::ImapRequestTime => 13,\n            Self::Pop3RequestTime => 14,\n            Self::SmtpRequestTime => 15,\n            Self::SieveRequestTime => 16,\n            Self::HttpActiveConnections => 17,\n            Self::ImapActiveConnections => 18,\n            Self::Pop3ActiveConnections => 19,\n            Self::SmtpActiveConnections => 20,\n            Self::SieveActiveConnections => 21,\n            Self::DeliveryActiveConnections => 22,\n            Self::ServerMemory => 23,\n            Self::QueueCount => 24,\n            Self::UserCount => 25,\n            Self::DomainCount => 26,\n        }\n    }\n\n    pub fn from_code(code: u64) -> Option<Self> {\n        match code {\n            0 => Some(Self::MessageIngestionTime),\n            1 => Some(Self::MessageFtsIndexTime),\n            2 => Some(Self::DeliveryTotalTime),\n            3 => Some(Self::DeliveryTime),\n            4 => Some(Self::MessageSize),\n            5 => Some(Self::MessageAuthSize),\n            6 => Some(Self::ReportOutgoingSize),\n            7 => Some(Self::StoreReadTime),\n            8 => Some(Self::StoreWriteTime),\n            9 => Some(Self::BlobReadTime),\n            10 => Some(Self::BlobWriteTime),\n            11 => Some(Self::DnsLookupTime),\n            12 => Some(Self::HttpRequestTime),\n            13 => Some(Self::ImapRequestTime),\n            14 => Some(Self::Pop3RequestTime),\n            15 => Some(Self::SmtpRequestTime),\n            16 => Some(Self::SieveRequestTime),\n            17 => Some(Self::HttpActiveConnections),\n            18 => Some(Self::ImapActiveConnections),\n            19 => Some(Self::Pop3ActiveConnections),\n            20 => Some(Self::SmtpActiveConnections),\n            21 => Some(Self::SieveActiveConnections),\n            22 => Some(Self::DeliveryActiveConnections),\n            23 => Some(Self::ServerMemory),\n            24 => Some(Self::QueueCount),\n            25 => Some(Self::UserCount),\n            26 => Some(Self::DomainCount),\n            _ => None,\n        }\n    }\n\n    pub fn try_parse(name: &str) -> Option<Self> {\n        match name {\n            \"message-ingest.time\" => Some(Self::MessageIngestionTime),\n            \"message-ingest.index-time\" => Some(Self::MessageFtsIndexTime),\n            \"delivery.total-time\" => Some(Self::DeliveryTotalTime),\n            \"delivery.attempt-time\" => Some(Self::DeliveryTime),\n            \"message.size\" => Some(Self::MessageSize),\n            \"message.authenticated-size\" => Some(Self::MessageAuthSize),\n            \"outgoing-report.size\" => Some(Self::ReportOutgoingSize),\n            \"store.data-read-time\" => Some(Self::StoreReadTime),\n            \"store.data-write-time\" => Some(Self::StoreWriteTime),\n            \"store.blob-read-time\" => Some(Self::BlobReadTime),\n            \"store.blob-write-time\" => Some(Self::BlobWriteTime),\n            \"dns.lookup-time\" => Some(Self::DnsLookupTime),\n            \"http.request-time\" => Some(Self::HttpRequestTime),\n            \"imap.request-time\" => Some(Self::ImapRequestTime),\n            \"pop3.request-time\" => Some(Self::Pop3RequestTime),\n            \"smtp.request-time\" => Some(Self::SmtpRequestTime),\n            \"sieve.request-time\" => Some(Self::SieveRequestTime),\n            \"http.active-connections\" => Some(Self::HttpActiveConnections),\n            \"imap.active-connections\" => Some(Self::ImapActiveConnections),\n            \"pop3.active-connections\" => Some(Self::Pop3ActiveConnections),\n            \"smtp.active-connections\" => Some(Self::SmtpActiveConnections),\n            \"sieve.active-connections\" => Some(Self::SieveActiveConnections),\n            \"delivery.active-connections\" => Some(Self::DeliveryActiveConnections),\n            \"server.memory\" => Some(Self::ServerMemory),\n            \"queue.count\" => Some(Self::QueueCount),\n            \"user.count\" => Some(Self::UserCount),\n            \"domain.count\" => Some(Self::DomainCount),\n            _ => None,\n        }\n    }\n\n    pub fn variants() -> &'static [Self] {\n        &[\n            Self::MessageIngestionTime,\n            Self::MessageFtsIndexTime,\n            Self::DeliveryTotalTime,\n            Self::DeliveryTime,\n            Self::MessageSize,\n            Self::MessageAuthSize,\n            Self::ReportOutgoingSize,\n            Self::StoreReadTime,\n            Self::StoreWriteTime,\n            Self::BlobReadTime,\n            Self::BlobWriteTime,\n            Self::DnsLookupTime,\n            Self::HttpRequestTime,\n            Self::ImapRequestTime,\n            Self::Pop3RequestTime,\n            Self::SmtpRequestTime,\n            Self::SieveRequestTime,\n            Self::HttpActiveConnections,\n            Self::ImapActiveConnections,\n            Self::Pop3ActiveConnections,\n            Self::SmtpActiveConnections,\n            Self::SieveActiveConnections,\n            Self::DeliveryActiveConnections,\n            Self::ServerMemory,\n            Self::QueueCount,\n            Self::UserCount,\n            Self::DomainCount,\n        ]\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/event/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod conv;\npub mod description;\npub mod level;\npub mod metrics;\n\nuse compact_str::ToCompactString;\nuse std::fmt::Display;\n\nuse crate::*;\n\nimpl<T> Event<T> {\n    pub fn with_capacity(inner: T, capacity: usize) -> Self {\n        Self {\n            inner,\n            keys: Vec::with_capacity(capacity),\n        }\n    }\n\n    pub fn with_keys(inner: T, keys: Vec<(Key, Value)>) -> Self {\n        Self { inner, keys }\n    }\n\n    pub fn new(inner: T) -> Self {\n        Self {\n            inner,\n            keys: Vec::with_capacity(5),\n        }\n    }\n\n    pub fn value(&self, key: Key) -> Option<&Value> {\n        self.keys\n            .iter()\n            .find_map(|(k, v)| if *k == key { Some(v) } else { None })\n    }\n\n    pub fn value_as_str(&self, key: Key) -> Option<&str> {\n        self.value(key).and_then(|v| v.as_str())\n    }\n\n    pub fn value_as_uint(&self, key: Key) -> Option<u64> {\n        self.value(key).and_then(|v| v.to_uint())\n    }\n\n    pub fn take_value(&mut self, key: Key) -> Option<Value> {\n        self.keys.iter_mut().find_map(|(k, v)| {\n            if *k == key {\n                Some(std::mem::take(v))\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn into_boxed(self) -> Box<Self> {\n        Box::new(self)\n    }\n}\n\nimpl Error {\n    #[inline(always)]\n    pub fn new(inner: EventType) -> Self {\n        Error(Box::new(Event::new(inner)))\n    }\n\n    #[inline(always)]\n    pub fn set_ctx(&mut self, key: Key, value: impl Into<Value>) {\n        self.0.keys.push((key, value.into()));\n    }\n\n    #[inline(always)]\n    pub fn ctx(mut self, key: Key, value: impl Into<Value>) -> Self {\n        self.0.keys.push((key, value.into()));\n        self\n    }\n\n    #[inline(always)]\n    pub fn ctx_unique(mut self, key: Key, value: impl Into<Value>) -> Self {\n        if self.0.keys.iter().all(|(k, _)| *k != key) {\n            self.0.keys.push((key, value.into()));\n        }\n        self\n    }\n\n    #[inline(always)]\n    pub fn ctx_opt(self, key: Key, value: Option<impl Into<Value>>) -> Self {\n        match value {\n            Some(value) => self.ctx(key, value),\n            None => self,\n        }\n    }\n\n    #[inline(always)]\n    pub fn matches(&self, inner: EventType) -> bool {\n        self.0.inner == inner\n    }\n\n    #[inline(always)]\n    pub fn event_type(&self) -> EventType {\n        self.0.inner\n    }\n\n    #[inline(always)]\n    pub fn span_id(self, session_id: u64) -> Self {\n        self.ctx(Key::SpanId, session_id)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Self {\n        self.ctx(Key::CausedBy, error)\n    }\n\n    #[inline(always)]\n    pub fn details(self, error: impl Into<Value>) -> Self {\n        self.ctx(Key::Details, error)\n    }\n\n    #[inline(always)]\n    pub fn code(self, error: impl Into<Value>) -> Self {\n        self.ctx(Key::Code, error)\n    }\n\n    #[inline(always)]\n    pub fn id(self, error: impl Into<Value>) -> Self {\n        self.ctx(Key::Id, error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Self {\n        self.ctx(Key::Reason, error.to_compact_string())\n    }\n\n    #[inline(always)]\n    pub fn document_id(self, id: u32) -> Self {\n        self.ctx(Key::DocumentId, id)\n    }\n\n    #[inline(always)]\n    pub fn account_id(self, id: u32) -> Self {\n        self.ctx(Key::AccountId, id)\n    }\n\n    #[inline(always)]\n    pub fn collection(self, id: impl Into<u8>) -> Self {\n        self.ctx(Key::Collection, id.into() as u64)\n    }\n\n    #[inline(always)]\n    pub fn wrap(self, cause: EventType) -> Self {\n        Error::new(cause).caused_by(self)\n    }\n\n    #[inline(always)]\n    pub fn keys(&self) -> &[(Key, Value)] {\n        &self.0.keys\n    }\n\n    #[inline(always)]\n    pub fn value(&self, key: Key) -> Option<&Value> {\n        self.0.value(key)\n    }\n\n    #[inline(always)]\n    pub fn value_as_str(&self, key: Key) -> Option<&str> {\n        self.0.value_as_str(key)\n    }\n\n    #[inline(always)]\n    pub fn value_as_uint(&self, key: Key) -> Option<u64> {\n        self.0.value_as_uint(key)\n    }\n\n    #[inline(always)]\n    pub fn take_value(&mut self, key: Key) -> Option<Value> {\n        self.0.take_value(key)\n    }\n\n    #[inline(always)]\n    pub fn is_assertion_failure(&self) -> bool {\n        self.0.inner == EventType::Store(StoreEvent::AssertValueFailed)\n    }\n\n    pub fn key(&self, key: Key) -> Option<&Value> {\n        self.0\n            .keys\n            .iter()\n            .find_map(|(k, v)| if *k == key { Some(v) } else { None })\n    }\n\n    #[inline(always)]\n    pub fn is_jmap_method_error(&self) -> bool {\n        !matches!(\n            self.0.inner,\n            EventType::Jmap(\n                JmapEvent::UnknownCapability | JmapEvent::NotJson | JmapEvent::NotRequest\n            )\n        )\n    }\n\n    #[inline(always)]\n    pub fn must_disconnect(&self) -> bool {\n        matches!(\n            self.0.inner,\n            EventType::Network(_)\n                | EventType::Auth(AuthEvent::TooManyAttempts)\n                | EventType::Limit(LimitEvent::ConcurrentRequest | LimitEvent::TooManyRequests)\n                | EventType::Security(_)\n        )\n    }\n\n    #[inline(always)]\n    pub fn should_write_err(&self) -> bool {\n        !matches!(self.0.inner, EventType::Network(_) | EventType::Security(_))\n    }\n\n    pub fn corrupted_key(key: &[u8], value: Option<&[u8]>, caused_by: &'static str) -> Error {\n        EventType::Store(StoreEvent::DataCorruption)\n            .ctx(Key::Key, key)\n            .ctx_opt(Key::Value, value)\n            .ctx(Key::CausedBy, caused_by)\n    }\n}\n\nimpl Event<EventDetails> {\n    pub fn span_id(&self) -> Option<u64> {\n        for (key, value) in &self.keys {\n            match (key, value) {\n                (Key::SpanId, Value::UInt(value)) => return Some(*value),\n                (Key::SpanId, Value::Int(value)) => return Some(*value as u64),\n                _ => {}\n            }\n        }\n\n        None\n    }\n}\n\nimpl EventType {\n    #[inline(always)]\n    pub fn is_span_start(&self) -> bool {\n        matches!(\n            self,\n            EventType::Smtp(SmtpEvent::ConnectionStart)\n                | EventType::Imap(ImapEvent::ConnectionStart)\n                | EventType::ManageSieve(ManageSieveEvent::ConnectionStart)\n                | EventType::Pop3(Pop3Event::ConnectionStart)\n                | EventType::Http(HttpEvent::ConnectionStart)\n                | EventType::Delivery(DeliveryEvent::AttemptStart)\n        )\n    }\n\n    #[inline(always)]\n    pub fn is_span_end(&self) -> bool {\n        matches!(\n            self,\n            EventType::Smtp(SmtpEvent::ConnectionEnd)\n                | EventType::Imap(ImapEvent::ConnectionEnd)\n                | EventType::ManageSieve(ManageSieveEvent::ConnectionEnd)\n                | EventType::Pop3(Pop3Event::ConnectionEnd)\n                | EventType::Http(HttpEvent::ConnectionEnd)\n                | EventType::Delivery(DeliveryEvent::AttemptEnd)\n        )\n    }\n\n    pub fn is_raw_io(&self) -> bool {\n        matches!(\n            self,\n            EventType::Imap(ImapEvent::RawInput | ImapEvent::RawOutput)\n                | EventType::Smtp(SmtpEvent::RawInput | SmtpEvent::RawOutput)\n                | EventType::Pop3(Pop3Event::RawInput | Pop3Event::RawOutput)\n                | EventType::ManageSieve(ManageSieveEvent::RawInput | ManageSieveEvent::RawOutput)\n                | EventType::Delivery(DeliveryEvent::RawInput | DeliveryEvent::RawOutput)\n                | EventType::Milter(MilterEvent::Read | MilterEvent::Write)\n        )\n    }\n\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(self)\n    }\n\n    pub fn message(&self) -> &'static str {\n        match self {\n            EventType::Store(cause) => cause.message(),\n            EventType::Jmap(cause) => cause.message(),\n            EventType::Imap(_) => \"IMAP error\",\n            EventType::ManageSieve(_) => \"ManageSieve error\",\n            EventType::Pop3(_) => \"POP3 error\",\n            EventType::Smtp(_) => \"SMTP error\",\n            EventType::Network(_) => \"Network error\",\n            EventType::Limit(cause) => cause.message(),\n            EventType::Manage(cause) => cause.message(),\n            EventType::Auth(cause) => cause.message(),\n            EventType::Config(_) => \"Configuration error\",\n            EventType::Resource(cause) => cause.message(),\n            EventType::Security(_) => \"Insufficient permissions\",\n            _ => \"Internal server error\",\n        }\n    }\n}\n\nimpl StoreEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Store(self))\n    }\n\n    pub fn message(&self) -> &'static str {\n        match self {\n            Self::AssertValueFailed => \"Another process has modified the value\",\n            Self::BlobMissingMarker => \"Blob is missing marker\",\n            Self::FoundationdbError => \"FoundationDB error\",\n            Self::MysqlError => \"MySQL error\",\n            Self::PostgresqlError => \"PostgreSQL error\",\n            Self::RocksdbError => \"RocksDB error\",\n            Self::SqliteError => \"SQLite error\",\n            Self::LdapError => \"LDAP error\",\n            Self::ElasticsearchError => \"ElasticSearch error\",\n            Self::RedisError => \"Redis error\",\n            Self::S3Error => \"S3 error\",\n            Self::AzureError => \"Azure error\",\n            Self::FilesystemError => \"Filesystem error\",\n            Self::PoolError => \"Connection pool error\",\n            Self::DataCorruption => \"Data corruption\",\n            Self::DecompressError => \"Decompression error\",\n            Self::DeserializeError => \"Deserialization error\",\n            Self::NotFound => \"Not found\",\n            Self::NotConfigured => \"Not configured\",\n            Self::NotSupported => \"Operation not supported\",\n            Self::UnexpectedError => \"Unexpected error\",\n            Self::CryptoError => \"Crypto error\",\n            _ => \"Store error\",\n        }\n    }\n}\n\nimpl SecurityEvent {\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Security(self))\n    }\n}\n\nimpl AuthEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Auth(self))\n    }\n\n    pub fn message(&self) -> &'static str {\n        match self {\n            Self::Failed => \"Authentication failed\",\n            Self::MissingTotp => concat!(\n                \"A TOTP code is required to authenticate this account. \",\n                \"Try authenticating again using 'secret$totp_token'.\"\n            ),\n            Self::TooManyAttempts => \"Too many authentication attempts\",\n            _ => \"Authentication error\",\n        }\n    }\n}\n\nimpl ManageEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Manage(self))\n    }\n\n    pub fn message(&self) -> &'static str {\n        match self {\n            Self::MissingParameter => \"Missing parameter\",\n            Self::AlreadyExists => \"Already exists\",\n            Self::AssertFailed => \"Assertion failed\",\n            Self::NotFound => \"Not found\",\n            Self::NotSupported => \"Operation not supported\",\n            Self::Error => \"Management API Error\",\n        }\n    }\n}\n\nimpl JmapEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Jmap(self))\n    }\n\n    pub fn message(&self) -> &'static str {\n        match self {\n            Self::InvalidArguments => \"Invalid arguments\",\n            Self::RequestTooLarge => \"Request too large\",\n            Self::StateMismatch => \"State mismatch\",\n            Self::AnchorNotFound => \"Anchor not found\",\n            Self::UnsupportedFilter => \"Unsupported filter\",\n            Self::UnsupportedSort => \"Unsupported sort\",\n            Self::UnknownMethod => \"Unknown method\",\n            Self::InvalidResultReference => \"Invalid result reference\",\n            Self::Forbidden => \"Forbidden\",\n            Self::AccountNotFound => \"Account not found\",\n            Self::AccountNotSupportedByMethod => \"Account not supported by method\",\n            Self::AccountReadOnly => \"Account read-only\",\n            Self::NotFound => \"Not found\",\n            Self::CannotCalculateChanges => \"Cannot calculate changes\",\n            Self::UnknownDataType => \"Unknown data type\",\n            Self::UnknownCapability => \"Unknown capability\",\n            Self::NotJson => \"Not JSON\",\n            Self::NotRequest => \"Not a request\",\n            _ => \"Other message\",\n        }\n    }\n}\n\nimpl LimitEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Limit(self))\n    }\n\n    pub fn message(&self) -> &'static str {\n        match self {\n            Self::SizeRequest => \"Request too large\",\n            Self::SizeUpload => \"Upload too large\",\n            Self::CallsIn => \"Too many calls in\",\n            Self::ConcurrentRequest => \"Too many concurrent requests\",\n            Self::ConcurrentConnection => \"Too many concurrent connections\",\n            Self::ConcurrentUpload => \"Too many concurrent uploads\",\n            Self::Quota => \"Quota exceeded\",\n            Self::BlobQuota => \"Blob quota exceeded\",\n            Self::TooManyRequests => \"Too many requests\",\n            Self::TenantQuota => \"Tenant quota exceeded\",\n        }\n    }\n}\n\nimpl ResourceEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Resource(self))\n    }\n\n    pub fn message(&self) -> &'static str {\n        match self {\n            Self::NotFound => \"Not found\",\n            Self::BadParameters => \"Bad parameters\",\n            Self::Error => \"Resource error\",\n            _ => \"Other status\",\n        }\n    }\n}\n\nimpl SmtpEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Smtp(self))\n    }\n}\n\nimpl SieveEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Sieve(self))\n    }\n}\n\nimpl SpamEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Spam(self))\n    }\n}\n\nimpl ImapEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Imap(self))\n    }\n\n    #[inline(always)]\n    pub fn caused_by(self, error: impl Into<Value>) -> Error {\n        self.into_err().caused_by(error)\n    }\n\n    #[inline(always)]\n    pub fn reason(self, error: impl Display) -> Error {\n        self.into_err().reason(error)\n    }\n}\n\nimpl Pop3Event {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Pop3(self))\n    }\n}\n\nimpl ManageSieveEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::ManageSieve(self))\n    }\n}\n\nimpl NetworkEvent {\n    #[inline(always)]\n    pub fn ctx(self, key: Key, value: impl Into<Value>) -> Error {\n        self.into_err().ctx(key, value)\n    }\n\n    #[inline(always)]\n    pub fn into_err(self) -> Error {\n        Error::new(EventType::Network(self))\n    }\n}\n\nimpl Value {\n    pub fn from_maybe_string(value: &[u8]) -> Self {\n        if let Ok(value) = std::str::from_utf8(value) {\n            Self::String(value.into())\n        } else {\n            Self::Bytes(value.to_vec())\n        }\n    }\n\n    pub fn to_uint(&self) -> Option<u64> {\n        match self {\n            Self::UInt(value) => Some(*value),\n            Self::Int(value) => Some(*value as u64),\n            _ => None,\n        }\n    }\n\n    pub fn as_str(&self) -> Option<&str> {\n        match self {\n            Self::String(value) => Some(value.as_str()),\n            _ => None,\n        }\n    }\n\n    pub fn into_string(self) -> Option<CompactString> {\n        match self {\n            Self::String(value) => Some(value),\n            _ => None,\n        }\n    }\n}\n\nimpl<T> AddContext<T> for Result<T> {\n    #[inline(always)]\n    fn caused_by(self, location: &'static str) -> Result<T> {\n        match self {\n            Ok(value) => Ok(value),\n            Err(mut err) => {\n                err.set_ctx(Key::CausedBy, location);\n                Err(err)\n            }\n        }\n    }\n\n    #[inline(always)]\n    fn add_context<F>(self, f: F) -> Result<T>\n    where\n        F: FnOnce(Error) -> Error,\n    {\n        match self {\n            Ok(value) => Ok(value),\n            Err(err) => Err(f(err)),\n        }\n    }\n}\n\nimpl std::error::Error for Error {}\nimpl Eq for Error {}\nimpl PartialEq for Error {\n    fn eq(&self, other: &Self) -> bool {\n        if self.0.inner == other.0.inner && self.0.keys.len() == other.0.keys.len() {\n            for kv in self.0.keys.iter() {\n                if !other.0.keys.iter().any(|okv| kv == okv) {\n                    return false;\n                }\n            }\n\n            true\n        } else {\n            false\n        }\n    }\n}\n\nimpl PartialEq for Value {\n    fn eq(&self, other: &Self) -> bool {\n        match (self, other) {\n            (Self::String(l0), Self::String(r0)) => l0 == r0,\n            (Self::UInt(l0), Self::UInt(r0)) => l0 == r0,\n            (Self::Int(l0), Self::Int(r0)) => l0 == r0,\n            (Self::Float(l0), Self::Float(r0)) => l0 == r0,\n            (Self::Bytes(l0), Self::Bytes(r0)) => l0 == r0,\n            (Self::Bool(l0), Self::Bool(r0)) => l0 == r0,\n            (Self::Ipv4(l0), Self::Ipv4(r0)) => l0 == r0,\n            (Self::Ipv6(l0), Self::Ipv6(r0)) => l0 == r0,\n            (Self::Event(l0), Self::Event(r0)) => l0 == r0,\n            (Self::Array(l0), Self::Array(r0)) => l0 == r0,\n            _ => false,\n        }\n    }\n}\n\nimpl Eq for Value {}\n\nimpl From<EventType> for usize {\n    fn from(value: EventType) -> Self {\n        value.id()\n    }\n}\n\nimpl AsRef<Event<EventDetails>> for Event<EventDetails> {\n    fn as_ref(&self) -> &Event<EventDetails> {\n        self\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/ipc/bitset.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{USIZE_BITS, USIZE_BITS_MASK};\n\n#[derive(Clone, Debug, PartialEq, Eq)]\npub struct Bitset<const N: usize>(pub(crate) [usize; N]);\n\nimpl<const N: usize> Bitset<N> {\n    #[allow(clippy::new_without_default)]\n    pub const fn new() -> Self {\n        Self([0; N])\n    }\n\n    pub const fn all() -> Self {\n        Self([usize::MAX; N])\n    }\n\n    #[inline(always)]\n    pub fn set(&mut self, index: impl Into<usize>) {\n        let index = index.into();\n        self.0[index / USIZE_BITS] |= 1 << (index & USIZE_BITS_MASK);\n    }\n\n    #[inline(always)]\n    pub fn clear(&mut self, index: impl Into<usize>) {\n        let index = index.into();\n        self.0[index / USIZE_BITS] &= !(1 << (index & USIZE_BITS_MASK));\n    }\n\n    #[inline(always)]\n    pub fn get(&self, index: impl Into<usize>) -> bool {\n        let index = index.into();\n        self.0[index / USIZE_BITS] & (1 << (index & USIZE_BITS_MASK)) != 0\n    }\n\n    pub fn union(&mut self, other: &Self) {\n        for i in 0..N {\n            self.0[i] |= other.0[i];\n        }\n    }\n\n    pub fn intersection(&mut self, other: &Self) {\n        for i in 0..N {\n            self.0[i] &= other.0[i];\n        }\n    }\n\n    pub fn difference(&mut self, other: &Self) {\n        for i in 0..N {\n            self.0[i] &= !other.0[i];\n        }\n    }\n\n    pub fn clear_all(&mut self) {\n        for i in 0..N {\n            self.0[i] = 0;\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        for i in 0..N {\n            if self.0[i] != 0 {\n                return false;\n            }\n        }\n        true\n    }\n\n    pub fn inner(&self) -> &[usize; N] {\n        &self.0\n    }\n}\n\nimpl<const N: usize> Default for Bitset<N> {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/ipc/channel.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    cell::UnsafeCell,\n    sync::{\n        Arc,\n        atomic::{AtomicU64, Ordering},\n    },\n};\n\nuse rtrb::{Consumer, Producer, PushError, RingBuffer};\n\nuse crate::{\n    Error, Event, EventType,\n    ipc::collector::{COLLECTOR_THREAD, COLLECTOR_UPDATES, Update},\n};\n\nuse super::collector::{Collector, CollectorThread};\n\npub(crate) static CHANNEL_FLAGS: AtomicU64 = AtomicU64::new(0);\npub(crate) const CHANNEL_SIZE: usize = 10240;\npub(crate) const CHANNEL_UPDATE_MARKER: u64 = 1 << 63;\n\nthread_local! {\n    static EVENT_TX: UnsafeCell<Sender> = {\n        // Create channel.\n        let (tx, rx) = RingBuffer::new(CHANNEL_SIZE);\n\n        // Register receiver with collector.\n        COLLECTOR_UPDATES.lock().push(Update::RegisterReceiver { receiver: Receiver { rx } });\n\n        // Spawn collector thread.\n        let collector = COLLECTOR_THREAD.clone();\n        CHANNEL_FLAGS.fetch_or(CHANNEL_UPDATE_MARKER, Ordering::Relaxed);\n        collector.thread().unpark();\n\n        // Return sender.\n        UnsafeCell::new(Sender {\n            tx,\n            collector,\n            overflow: Vec::with_capacity(0),\n        })\n    };\n}\n\npub struct Sender {\n    tx: Producer<Event<EventType>>,\n    collector: Arc<CollectorThread>,\n    overflow: Vec<Event<EventType>>,\n}\n\npub struct Receiver {\n    rx: Consumer<Event<EventType>>,\n}\n\n#[derive(Debug)]\npub struct ChannelError;\n\nimpl Sender {\n    pub fn send(&mut self, event: Event<EventType>) -> Result<(), ChannelError> {\n        while let Some(event) = self.overflow.pop() {\n            if let Err(PushError::Full(event)) = self.tx.push(event) {\n                self.overflow.push(event);\n                break;\n            }\n        }\n\n        if let Err(PushError::Full(event)) = self.tx.push(event) {\n            if self.overflow.len() <= CHANNEL_SIZE * 2 {\n                self.overflow.push(event);\n            } else {\n                return Err(ChannelError);\n            }\n        }\n\n        Ok(())\n    }\n}\n\nimpl Receiver {\n    pub fn try_recv(&mut self) -> Result<Option<Event<EventType>>, ChannelError> {\n        match self.rx.pop() {\n            Ok(event) => Ok(Some(event)),\n            Err(_) => {\n                if !self.rx.is_abandoned() {\n                    Ok(None)\n                } else {\n                    Err(ChannelError)\n                }\n            }\n        }\n    }\n}\n\nimpl Event<EventType> {\n    pub fn send(self) {\n        // SAFETY: EVENT_TX is thread-local.\n        let _ = EVENT_TX.try_with(|tx| unsafe {\n            let tx = &mut *tx.get();\n            if tx.send(self).is_ok() {\n                CHANNEL_FLAGS.fetch_add(1, Ordering::Relaxed);\n                tx.collector.thread().unpark();\n            }\n        });\n    }\n\n    pub fn send_with_metrics(self) {\n        Collector::record_metric(self.inner, self.inner.id(), &self.keys);\n        self.send();\n    }\n}\n\nimpl Error {\n    pub fn send(self) {\n        self.0.send();\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/ipc/collector.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    sync::{Arc, LazyLock, atomic::Ordering},\n    thread::{Builder, JoinHandle, park},\n    time::SystemTime,\n};\n\nuse ahash::AHashMap;\nuse atomics::bitset::AtomicBitset;\nuse ipc::{\n    USIZE_BITS,\n    channel::{CHANNEL_FLAGS, CHANNEL_UPDATE_MARKER, Receiver},\n    subscriber::{Interests, Subscriber},\n};\nuse parking_lot::Mutex;\n\nuse crate::*;\n\npub(crate) type GlobalInterests = AtomicBitset<{ TOTAL_EVENT_COUNT.div_ceil(USIZE_BITS) }>;\n\npub(crate) static TRACE_INTERESTS: GlobalInterests = GlobalInterests::new();\npub(crate) type CollectorThread = JoinHandle<()>;\npub(crate) static ACTIVE_SUBSCRIBERS: Mutex<Vec<String>> = Mutex::new(Vec::new());\npub(crate) static COLLECTOR_UPDATES: Mutex<Vec<Update>> = Mutex::new(Vec::new());\n\npub(crate) const EVENT_TYPES: [EventType; TOTAL_EVENT_COUNT] = EventType::variants();\n\n#[allow(clippy::enum_variant_names)]\npub(crate) enum Update {\n    RegisterReceiver {\n        receiver: Receiver,\n    },\n    RegisterSubscriber {\n        subscriber: Subscriber,\n    },\n    UnregisterSubscriber {\n        id: String,\n    },\n    UpdateSubscriber {\n        id: String,\n        interests: Interests,\n        lossy: bool,\n    },\n    UpdateLevels {\n        levels: AHashMap<EventType, Level>,\n    },\n    Shutdown,\n}\n\npub struct Collector {\n    receivers: Vec<Receiver>,\n    subscribers: Vec<Subscriber>,\n    levels: [Level; TOTAL_EVENT_COUNT],\n    active_spans: AHashMap<u64, Arc<Event<EventDetails>>>,\n}\n\nconst HTTP_CONN_START: usize = EventType::Http(HttpEvent::ConnectionStart).id();\nconst HTTP_CONN_END: usize = EventType::Http(HttpEvent::ConnectionEnd).id();\nconst IMAP_CONN_START: usize = EventType::Imap(ImapEvent::ConnectionStart).id();\nconst IMAP_CONN_END: usize = EventType::Imap(ImapEvent::ConnectionEnd).id();\nconst POP3_CONN_START: usize = EventType::Pop3(Pop3Event::ConnectionStart).id();\nconst POP3_CONN_END: usize = EventType::Pop3(Pop3Event::ConnectionEnd).id();\nconst SMTP_CONN_START: usize = EventType::Smtp(SmtpEvent::ConnectionStart).id();\nconst SMTP_CONN_END: usize = EventType::Smtp(SmtpEvent::ConnectionEnd).id();\nconst MANAGE_SIEVE_CONN_START: usize =\n    EventType::ManageSieve(ManageSieveEvent::ConnectionStart).id();\nconst MANAGE_SIEVE_CONN_END: usize = EventType::ManageSieve(ManageSieveEvent::ConnectionEnd).id();\nconst EV_ATTEMPT_START: usize = EventType::Delivery(DeliveryEvent::AttemptStart).id();\nconst EV_ATTEMPT_END: usize = EventType::Delivery(DeliveryEvent::AttemptEnd).id();\n\nconst STALE_SPAN_CHECK_WATERMARK: usize = 8000;\nconst SPAN_MAX_HOLD: u64 = 60 * 60 * 24; // 1 day\n\npub(crate) static COLLECTOR_THREAD: LazyLock<Arc<CollectorThread>> = LazyLock::new(|| {\n    Arc::new(\n        Builder::new()\n            .name(\"stalwart-collector\".to_string())\n            .spawn(move || {\n                Collector::default().collect();\n            })\n            .expect(\"Failed to start event collector\"),\n    )\n});\n\nimpl Collector {\n    fn collect(&mut self) {\n        let mut do_continue = true;\n\n        // Update\n        self.update();\n\n        while do_continue {\n            match CHANNEL_FLAGS.swap(0, Ordering::Relaxed) {\n                0 => {\n                    park();\n                }\n                CHANNEL_UPDATE_MARKER..=u64::MAX => {\n                    do_continue = self.update();\n                }\n                _ => {}\n            }\n\n            // Collect all events\n            let mut closed_rxs = Vec::new();\n            for (rx_idx, rx) in self.receivers.iter_mut().enumerate() {\n                let timestamp = SystemTime::now()\n                    .duration_since(SystemTime::UNIX_EPOCH)\n                    .map_or(0, |d| d.as_secs());\n\n                loop {\n                    match rx.try_recv() {\n                        Ok(Some(event)) => {\n                            // Build event\n                            let event_id = event.inner.id();\n                            let mut event = Event {\n                                inner: EventDetails {\n                                    level: self.levels[event_id],\n                                    typ: event.inner,\n                                    timestamp,\n                                    span: None,\n                                },\n                                keys: event.keys,\n                            };\n\n                            // Track spans\n                            let event = match event_id {\n                                HTTP_CONN_START\n                                | IMAP_CONN_START\n                                | POP3_CONN_START\n                                | SMTP_CONN_START\n                                | MANAGE_SIEVE_CONN_START\n                                | EV_ATTEMPT_START => {\n                                    let event = Arc::new(event);\n                                    self.active_spans.insert(\n                                        event.span_id().unwrap_or_else(|| {\n                                            panic!(\"Missing span ID: {event:?}\")\n                                        }),\n                                        event.clone(),\n                                    );\n\n                                    if self.active_spans.len() > STALE_SPAN_CHECK_WATERMARK {\n                                        self.active_spans.retain(|_, span| {\n                                            timestamp.saturating_sub(span.inner.timestamp)\n                                                < SPAN_MAX_HOLD\n                                        });\n                                    }\n                                    event\n                                }\n\n                                HTTP_CONN_END\n                                | IMAP_CONN_END\n                                | POP3_CONN_END\n                                | SMTP_CONN_END\n                                | MANAGE_SIEVE_CONN_END\n                                | EV_ATTEMPT_END => {\n                                    if let Some(span) = self\n                                        .active_spans\n                                        .remove(&event.span_id().expect(\"Missing span ID\"))\n                                    {\n                                        event.inner.span = Some(span.clone());\n                                    } else {\n                                        #[cfg(debug_assertions)]\n                                        {\n                                            if event.span_id().unwrap() != 0 {\n                                                eprintln!(\"Unregistered span ID: {event:?}\");\n                                            }\n                                        }\n                                    }\n                                    Arc::new(event)\n                                }\n                                _ => {\n                                    if let Some(span_id) = event.span_id() {\n                                        if let Some(span) = self.active_spans.get(&span_id) {\n                                            event.inner.span = Some(span.clone());\n                                        } else {\n                                            #[cfg(debug_assertions)]\n                                            {\n                                                if span_id != 0 {\n                                                    eprintln!(\"Unregistered span ID: {event:?}\");\n                                                }\n                                            }\n                                        }\n                                    }\n\n                                    Arc::new(event)\n                                }\n                            };\n\n                            // Send to subscribers\n                            for subscriber in self.subscribers.iter_mut() {\n                                subscriber.push_event(event_id, event.clone());\n                            }\n                        }\n                        Ok(None) => {\n                            break;\n                        }\n                        Err(_) => {\n                            closed_rxs.push(rx_idx); // Channel is closed, remove.\n                            break;\n                        }\n                    }\n                }\n            }\n\n            if do_continue {\n                // Remove closed receivers (should be rare in Tokio)\n                if !closed_rxs.is_empty() {\n                    let mut receivers = Vec::with_capacity(self.receivers.len() - closed_rxs.len());\n                    for (rx_idx, rx) in self.receivers.drain(..).enumerate() {\n                        if !closed_rxs.contains(&rx_idx) {\n                            receivers.push(rx);\n                        }\n                    }\n                    self.receivers = receivers;\n                }\n\n                // Send batched events\n                if !self.subscribers.is_empty() {\n                    self.subscribers\n                        .retain_mut(|subscriber| subscriber.send_batch().is_ok());\n                }\n            }\n        }\n\n        // Send remaining events\n        for mut subscriber in self.subscribers.drain(..) {\n            let _ = subscriber.send_batch();\n        }\n    }\n\n    fn update(&mut self) -> bool {\n        for update in COLLECTOR_UPDATES.lock().drain(..) {\n            match update {\n                Update::RegisterReceiver { receiver } => {\n                    self.receivers.push(receiver);\n                }\n                Update::RegisterSubscriber { subscriber } => {\n                    ACTIVE_SUBSCRIBERS.lock().push(subscriber.id.clone());\n                    self.subscribers.push(subscriber);\n                }\n                Update::UnregisterSubscriber { id } => {\n                    ACTIVE_SUBSCRIBERS.lock().retain(|s| s != &id);\n                    self.subscribers.retain(|s| s.id != id);\n                }\n                Update::UpdateSubscriber {\n                    id,\n                    interests,\n                    lossy,\n                } => {\n                    for subscriber in self.subscribers.iter_mut() {\n                        if subscriber.id == id {\n                            subscriber.interests = interests;\n                            subscriber.lossy = lossy;\n                            break;\n                        }\n                    }\n                }\n                Update::UpdateLevels { levels } => {\n                    for event in EVENT_TYPES.iter() {\n                        let event_id = event.id();\n                        if let Some(level) = levels.get(event) {\n                            self.levels[event_id] = *level;\n                        } else {\n                            self.levels[event_id] = event.level();\n                        }\n                    }\n                }\n                Update::Shutdown => return false,\n            }\n        }\n\n        true\n    }\n\n    pub fn set_interests(mut interests: Interests) {\n        if !interests.is_empty() {\n            for event_type in EVENT_TYPES.iter() {\n                if event_type.is_span_start() || event_type.is_span_end() {\n                    interests.set(*event_type);\n                }\n            }\n        }\n\n        TRACE_INTERESTS.update(interests);\n    }\n\n    pub fn union_interests(interests: Interests) {\n        TRACE_INTERESTS.union(interests);\n    }\n\n    #[inline(always)]\n    pub fn has_interest(event: impl Into<usize>) -> bool {\n        TRACE_INTERESTS.get(event)\n    }\n\n    pub fn get_subscribers() -> Vec<String> {\n        ACTIVE_SUBSCRIBERS.lock().clone()\n    }\n\n    pub fn update_custom_levels(levels: AHashMap<EventType, Level>) {\n        COLLECTOR_UPDATES\n            .lock()\n            .push(Update::UpdateLevels { levels });\n    }\n\n    pub fn update_subscriber(id: String, interests: Interests, lossy: bool) {\n        COLLECTOR_UPDATES.lock().push(Update::UpdateSubscriber {\n            id,\n            interests,\n            lossy,\n        });\n    }\n\n    pub fn remove_subscriber(id: String) {\n        COLLECTOR_UPDATES\n            .lock()\n            .push(Update::UnregisterSubscriber { id });\n    }\n\n    pub fn shutdown() {\n        COLLECTOR_UPDATES.lock().push(Update::Shutdown);\n        Collector::reload();\n    }\n\n    pub fn is_enabled() -> bool {\n        !TRACE_INTERESTS.is_empty()\n    }\n\n    pub fn reload() {\n        CHANNEL_FLAGS.fetch_or(CHANNEL_UPDATE_MARKER, Ordering::Relaxed);\n        COLLECTOR_THREAD.thread().unpark();\n    }\n}\n\nimpl Default for Collector {\n    fn default() -> Self {\n        let mut c = Collector {\n            subscribers: Vec::new(),\n            levels: [Level::Disable; TOTAL_EVENT_COUNT],\n            active_spans: AHashMap::new(),\n            receivers: Vec::new(),\n        };\n\n        for event in EVENT_TYPES.iter() {\n            let event_id = event.id();\n            c.levels[event_id] = event.level();\n        }\n\n        c\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/ipc/metrics.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::atomic::Ordering;\n\nuse atomics::{array::AtomicU32Array, gauge::AtomicGauge, histogram::AtomicHistogram};\nuse ipc::{\n    collector::{Collector, EVENT_TYPES, GlobalInterests},\n    subscriber::Interests,\n};\n\nuse crate::*;\n\npub(crate) static METRIC_INTERESTS: GlobalInterests = GlobalInterests::new();\n\nstatic EVENT_COUNTERS: AtomicU32Array<TOTAL_EVENT_COUNT> = AtomicU32Array::new();\nstatic CONNECTION_METRICS: [ConnectionMetrics; TOTAL_CONN_TYPES] = init_conn_metrics();\n\nstatic MESSAGE_INGESTION_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<10>::new_short_durations(MetricType::MessageIngestionTime);\nstatic MESSAGE_INDEX_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<10>::new_short_durations(MetricType::MessageFtsIndexTime);\nstatic MESSAGE_DELIVERY_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<18>::new_long_durations(MetricType::DeliveryTotalTime);\n\nstatic MESSAGE_INCOMING_SIZE: AtomicHistogram<12> =\n    AtomicHistogram::<12>::new_message_sizes(MetricType::MessageSize);\nstatic MESSAGE_SUBMISSION_SIZE: AtomicHistogram<12> =\n    AtomicHistogram::<12>::new_message_sizes(MetricType::MessageAuthSize);\nstatic MESSAGE_OUT_REPORT_SIZE: AtomicHistogram<12> =\n    AtomicHistogram::<12>::new_message_sizes(MetricType::ReportOutgoingSize);\n\nstatic STORE_DATA_READ_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<10>::new_short_durations(MetricType::StoreReadTime);\nstatic STORE_DATA_WRITE_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<10>::new_short_durations(MetricType::StoreWriteTime);\nstatic STORE_BLOB_READ_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<10>::new_short_durations(MetricType::BlobReadTime);\nstatic STORE_BLOB_WRITE_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<10>::new_short_durations(MetricType::BlobWriteTime);\n\nstatic DNS_LOOKUP_TIME: AtomicHistogram<12> =\n    AtomicHistogram::<10>::new_short_durations(MetricType::DnsLookupTime);\n\nstatic SERVER_MEMORY: AtomicGauge = AtomicGauge::new(MetricType::ServerMemory);\nstatic QUEUE_COUNT: AtomicGauge = AtomicGauge::new(MetricType::QueueCount);\nstatic USER_COUNT: AtomicGauge = AtomicGauge::new(MetricType::UserCount);\nstatic DOMAIN_COUNT: AtomicGauge = AtomicGauge::new(MetricType::DomainCount);\n\nconst CONN_SMTP_IN: usize = 0;\nconst CONN_SMTP_OUT: usize = 1;\nconst CONN_IMAP: usize = 2;\nconst CONN_POP3: usize = 3;\nconst CONN_HTTP: usize = 4;\nconst CONN_SIEVE: usize = 5;\nconst TOTAL_CONN_TYPES: usize = 6;\n\npub struct ConnectionMetrics {\n    pub active_connections: AtomicGauge,\n    pub elapsed: AtomicHistogram<12>,\n}\n\npub struct EventCounter {\n    id: EventType,\n    value: u32,\n}\n\nimpl Collector {\n    pub fn record_metric(event: EventType, event_id: usize, keys: &[(Key, Value)]) {\n        // Increment the event counter\n        if !event.is_span_end() && !event.is_raw_io() {\n            EVENT_COUNTERS.add(event_id, 1);\n        }\n\n        // Extract variables\n        let mut elapsed = 0;\n        let mut size = 0;\n        for (key, value) in keys {\n            match (key, value) {\n                (Key::Elapsed, Value::Duration(d)) => elapsed = *d,\n                (Key::Size, Value::UInt(s)) => size = *s,\n                _ => {}\n            }\n        }\n\n        match event {\n            EventType::Smtp(SmtpEvent::ConnectionStart) => {\n                let conn = &CONNECTION_METRICS[CONN_SMTP_IN];\n                conn.active_connections.increment();\n            }\n            EventType::Smtp(SmtpEvent::ConnectionEnd) => {\n                let conn = &CONNECTION_METRICS[CONN_SMTP_IN];\n                conn.active_connections.decrement();\n                conn.elapsed.observe(elapsed);\n            }\n            EventType::Imap(ImapEvent::ConnectionStart) => {\n                let conn = &CONNECTION_METRICS[CONN_IMAP];\n                conn.active_connections.increment();\n            }\n            EventType::Imap(ImapEvent::ConnectionEnd) => {\n                let conn = &CONNECTION_METRICS[CONN_IMAP];\n                conn.active_connections.decrement();\n                conn.elapsed.observe(elapsed);\n            }\n            EventType::Pop3(Pop3Event::ConnectionStart) => {\n                let conn = &CONNECTION_METRICS[CONN_POP3];\n                conn.active_connections.increment();\n            }\n            EventType::Pop3(Pop3Event::ConnectionEnd) => {\n                let conn = &CONNECTION_METRICS[CONN_POP3];\n                conn.active_connections.decrement();\n                conn.elapsed.observe(elapsed);\n            }\n            EventType::Http(HttpEvent::ConnectionStart) => {\n                let conn = &CONNECTION_METRICS[CONN_HTTP];\n                conn.active_connections.increment();\n            }\n            EventType::Http(HttpEvent::ConnectionEnd) => {\n                let conn = &CONNECTION_METRICS[CONN_HTTP];\n                conn.active_connections.decrement();\n                conn.elapsed.observe(elapsed);\n            }\n            EventType::ManageSieve(ManageSieveEvent::ConnectionStart) => {\n                let conn = &CONNECTION_METRICS[CONN_SIEVE];\n                conn.active_connections.increment();\n            }\n            EventType::ManageSieve(ManageSieveEvent::ConnectionEnd) => {\n                let conn = &CONNECTION_METRICS[CONN_SIEVE];\n                conn.active_connections.decrement();\n                conn.elapsed.observe(elapsed);\n            }\n            EventType::Delivery(DeliveryEvent::AttemptStart) => {\n                let conn = &CONNECTION_METRICS[CONN_SMTP_OUT];\n                conn.active_connections.increment();\n            }\n            EventType::Delivery(DeliveryEvent::AttemptEnd) => {\n                let conn = &CONNECTION_METRICS[CONN_SMTP_OUT];\n                conn.active_connections.decrement();\n                conn.elapsed.observe(elapsed);\n            }\n            EventType::Delivery(DeliveryEvent::Completed) => {\n                QUEUE_COUNT.decrement();\n                MESSAGE_DELIVERY_TIME.observe(elapsed);\n            }\n            EventType::Delivery(\n                DeliveryEvent::MxLookup | DeliveryEvent::IpLookup | DeliveryEvent::NullMx,\n            )\n            | EventType::TlsRpt(_)\n            | EventType::MtaSts(_)\n            | EventType::Dane(_) => {\n                if elapsed > 0 {\n                    DNS_LOOKUP_TIME.observe(elapsed);\n                }\n            }\n            EventType::MessageIngest(\n                MessageIngestEvent::Ham\n                | MessageIngestEvent::Spam\n                | MessageIngestEvent::ImapAppend\n                | MessageIngestEvent::JmapAppend,\n            ) => {\n                MESSAGE_INGESTION_TIME.observe(elapsed);\n            }\n            EventType::Queue(QueueEvent::QueueMessage) => {\n                MESSAGE_INCOMING_SIZE.observe(size);\n                QUEUE_COUNT.increment();\n            }\n            EventType::Queue(QueueEvent::QueueMessageAuthenticated) => {\n                MESSAGE_SUBMISSION_SIZE.observe(size);\n                QUEUE_COUNT.increment();\n            }\n            EventType::Queue(QueueEvent::QueueReport) => {\n                MESSAGE_OUT_REPORT_SIZE.observe(size);\n                QUEUE_COUNT.increment();\n            }\n            EventType::Queue(QueueEvent::QueueAutogenerated | QueueEvent::QueueDsn) => {\n                QUEUE_COUNT.increment();\n            }\n            EventType::MessageIngest(MessageIngestEvent::FtsIndex) => {\n                MESSAGE_INDEX_TIME.observe(elapsed);\n            }\n            EventType::Store(StoreEvent::BlobWrite) => {\n                STORE_BLOB_WRITE_TIME.observe(elapsed);\n            }\n            EventType::Store(StoreEvent::BlobRead) => {\n                STORE_BLOB_READ_TIME.observe(elapsed);\n            }\n            EventType::Store(StoreEvent::DataWrite) => {\n                STORE_DATA_WRITE_TIME.observe(elapsed);\n            }\n            EventType::Store(StoreEvent::DataIterate) => {\n                STORE_DATA_READ_TIME.observe(elapsed);\n            }\n\n            _ => {}\n        }\n    }\n\n    #[inline(always)]\n    pub fn is_metric(event: impl Into<usize>) -> bool {\n        METRIC_INTERESTS.get(event)\n    }\n\n    pub fn set_metrics(interests: Interests) {\n        METRIC_INTERESTS.update(interests);\n    }\n\n    pub fn collect_counters(_is_enterprise: bool) -> impl Iterator<Item = EventCounter> {\n        EVENT_COUNTERS\n            .inner()\n            .iter()\n            .enumerate()\n            .filter_map(|(event_id, value)| {\n                let value = value.load(Ordering::Relaxed);\n                if value > 0 {\n                    Some(EventCounter {\n                        id: EVENT_TYPES[event_id],\n                        value,\n                    })\n                } else {\n                    None\n                }\n            })\n    }\n\n    pub fn collect_gauges(is_enterprise: bool) -> impl Iterator<Item = &'static AtomicGauge> {\n        static E_GAUGES: &[&AtomicGauge] =\n            &[&SERVER_MEMORY, &QUEUE_COUNT, &USER_COUNT, &DOMAIN_COUNT];\n        static C_GAUGES: &[&AtomicGauge] = &[&SERVER_MEMORY, &USER_COUNT, &DOMAIN_COUNT];\n\n        if is_enterprise { E_GAUGES } else { C_GAUGES }\n            .iter()\n            .copied()\n            .chain(CONNECTION_METRICS.iter().map(|m| &m.active_connections))\n    }\n\n    pub fn collect_histograms(\n        is_enterprise: bool,\n    ) -> impl Iterator<Item = &'static AtomicHistogram<12>> {\n        static E_HISTOGRAMS: &[&AtomicHistogram<12>] = &[\n            &MESSAGE_INGESTION_TIME,\n            &MESSAGE_INDEX_TIME,\n            &MESSAGE_DELIVERY_TIME,\n            &MESSAGE_INCOMING_SIZE,\n            &MESSAGE_SUBMISSION_SIZE,\n            &MESSAGE_OUT_REPORT_SIZE,\n            &STORE_DATA_READ_TIME,\n            &STORE_DATA_WRITE_TIME,\n            &STORE_BLOB_READ_TIME,\n            &STORE_BLOB_WRITE_TIME,\n            &DNS_LOOKUP_TIME,\n        ];\n        static C_HISTOGRAMS: &[&AtomicHistogram<12>] = &[\n            &MESSAGE_DELIVERY_TIME,\n            &MESSAGE_INCOMING_SIZE,\n            &MESSAGE_SUBMISSION_SIZE,\n        ];\n\n        if is_enterprise {\n            E_HISTOGRAMS\n        } else {\n            C_HISTOGRAMS\n        }\n        .iter()\n        .copied()\n        .chain(CONNECTION_METRICS.iter().map(|m| &m.elapsed))\n        .filter(|h| h.is_active())\n    }\n\n    #[inline(always)]\n    pub fn read_event_metric(metric_id: usize) -> u32 {\n        EVENT_COUNTERS.get(metric_id)\n    }\n\n    pub fn read_metric(metric_type: MetricType) -> f64 {\n        match metric_type {\n            MetricType::ServerMemory => SERVER_MEMORY.get() as f64,\n            MetricType::MessageIngestionTime => MESSAGE_INGESTION_TIME.average(),\n            MetricType::MessageFtsIndexTime => MESSAGE_INDEX_TIME.average(),\n            MetricType::MessageSize => MESSAGE_INCOMING_SIZE.average(),\n            MetricType::MessageAuthSize => MESSAGE_SUBMISSION_SIZE.average(),\n            MetricType::DeliveryTotalTime => MESSAGE_DELIVERY_TIME.average(),\n            MetricType::DeliveryTime => CONNECTION_METRICS[CONN_SMTP_OUT].elapsed.average(),\n            MetricType::DeliveryActiveConnections => {\n                CONNECTION_METRICS[CONN_SMTP_OUT].active_connections.get() as f64\n            }\n            MetricType::QueueCount => QUEUE_COUNT.get() as f64,\n            MetricType::ReportOutgoingSize => MESSAGE_OUT_REPORT_SIZE.average(),\n            MetricType::StoreReadTime => STORE_DATA_READ_TIME.average(),\n            MetricType::StoreWriteTime => STORE_DATA_WRITE_TIME.average(),\n            MetricType::BlobReadTime => STORE_BLOB_READ_TIME.average(),\n            MetricType::BlobWriteTime => STORE_BLOB_WRITE_TIME.average(),\n            MetricType::DnsLookupTime => DNS_LOOKUP_TIME.average(),\n            MetricType::HttpActiveConnections => {\n                CONNECTION_METRICS[CONN_HTTP].active_connections.get() as f64\n            }\n            MetricType::HttpRequestTime => CONNECTION_METRICS[CONN_HTTP].elapsed.average(),\n            MetricType::ImapActiveConnections => {\n                CONNECTION_METRICS[CONN_IMAP].active_connections.get() as f64\n            }\n            MetricType::ImapRequestTime => CONNECTION_METRICS[CONN_IMAP].elapsed.average(),\n            MetricType::Pop3ActiveConnections => {\n                CONNECTION_METRICS[CONN_POP3].active_connections.get() as f64\n            }\n            MetricType::Pop3RequestTime => CONNECTION_METRICS[CONN_POP3].elapsed.average(),\n            MetricType::SmtpActiveConnections => {\n                CONNECTION_METRICS[CONN_SMTP_IN].active_connections.get() as f64\n            }\n            MetricType::SmtpRequestTime => CONNECTION_METRICS[CONN_SMTP_IN].elapsed.average(),\n            MetricType::SieveActiveConnections => {\n                CONNECTION_METRICS[CONN_SIEVE].active_connections.get() as f64\n            }\n            MetricType::SieveRequestTime => CONNECTION_METRICS[CONN_SIEVE].elapsed.average(),\n            MetricType::UserCount => USER_COUNT.get() as f64,\n            MetricType::DomainCount => DOMAIN_COUNT.get() as f64,\n        }\n    }\n\n    pub fn update_gauge(metric_type: MetricType, value: u64) {\n        match metric_type {\n            MetricType::ServerMemory => SERVER_MEMORY.set(value),\n            MetricType::QueueCount => QUEUE_COUNT.set(value),\n            MetricType::UserCount => USER_COUNT.set(value),\n            MetricType::DomainCount => DOMAIN_COUNT.set(value),\n            _ => {}\n        }\n    }\n\n    pub fn update_event_counter(event_type: EventType, value: u32) {\n        EVENT_COUNTERS.add(event_type.into(), value);\n    }\n\n    pub fn update_histogram(metric_type: MetricType, value: u64) {\n        match metric_type {\n            MetricType::MessageIngestionTime => MESSAGE_INGESTION_TIME.observe(value),\n            MetricType::MessageFtsIndexTime => MESSAGE_INDEX_TIME.observe(value),\n            MetricType::DeliveryTotalTime => MESSAGE_DELIVERY_TIME.observe(value),\n            MetricType::DeliveryTime => CONNECTION_METRICS[CONN_SMTP_OUT].elapsed.observe(value),\n            MetricType::DnsLookupTime => DNS_LOOKUP_TIME.observe(value),\n            _ => {}\n        }\n    }\n}\n\nimpl EventCounter {\n    pub fn id(&self) -> EventType {\n        self.id\n    }\n\n    pub fn value(&self) -> u64 {\n        self.value as u64\n    }\n}\n\nimpl ConnectionMetrics {\n    #[allow(clippy::new_without_default)]\n    pub const fn new() -> Self {\n        Self {\n            active_connections: AtomicGauge::new(MetricType::BlobReadTime),\n            elapsed: AtomicHistogram::<18>::new_medium_durations(MetricType::BlobReadTime),\n        }\n    }\n}\n\n#[allow(clippy::declare_interior_mutable_const)]\nconst fn init_conn_metrics() -> [ConnectionMetrics; TOTAL_CONN_TYPES] {\n    const INIT: ConnectionMetrics = ConnectionMetrics::new();\n    let mut array = [INIT; TOTAL_CONN_TYPES];\n    let mut i = 0;\n    while i < TOTAL_CONN_TYPES {\n        let metric = match i {\n            CONN_HTTP => &[\n                MetricType::HttpRequestTime,\n                MetricType::HttpActiveConnections,\n            ],\n            CONN_IMAP => &[\n                MetricType::ImapRequestTime,\n                MetricType::ImapActiveConnections,\n            ],\n            CONN_POP3 => &[\n                MetricType::Pop3RequestTime,\n                MetricType::Pop3ActiveConnections,\n            ],\n            CONN_SMTP_IN => &[\n                MetricType::SmtpRequestTime,\n                MetricType::SmtpActiveConnections,\n            ],\n            CONN_SMTP_OUT => &[\n                MetricType::DeliveryTime,\n                MetricType::DeliveryActiveConnections,\n            ],\n            CONN_SIEVE => &[\n                MetricType::SieveRequestTime,\n                MetricType::SieveActiveConnections,\n            ],\n            _ => &[MetricType::BlobReadTime, MetricType::BlobReadTime],\n        };\n\n        array[i] = ConnectionMetrics {\n            elapsed: AtomicHistogram::<18>::new_medium_durations(metric[0]),\n            active_connections: AtomicGauge::new(metric[1]),\n        };\n        i += 1;\n    }\n    array\n}\n\nimpl EventType {\n    pub fn is_metric(&self) -> bool {\n        match self {\n            EventType::Server(ServerEvent::ThreadError) => true,\n            EventType::Purge(PurgeEvent::Error) => true,\n            EventType::Eval(\n                EvalEvent::Error | EvalEvent::StoreNotFound | EvalEvent::DirectoryNotFound,\n            ) => true,\n            EventType::Acme(\n                AcmeEvent::TlsAlpnError\n                | AcmeEvent::OrderCompleted\n                | AcmeEvent::AuthError\n                | AcmeEvent::AuthTooManyAttempts\n                | AcmeEvent::DnsRecordCreationFailed\n                | AcmeEvent::DnsRecordDeletionFailed\n                | AcmeEvent::DnsRecordPropagationTimeout\n                | AcmeEvent::ClientMissingSni\n                | AcmeEvent::TokenNotFound\n                | AcmeEvent::DnsRecordLookupFailed\n                | AcmeEvent::OrderInvalid\n                | AcmeEvent::Error,\n            ) => true,\n            EventType::Store(\n                StoreEvent::AssertValueFailed\n                | StoreEvent::FoundationdbError\n                | StoreEvent::MysqlError\n                | StoreEvent::PostgresqlError\n                | StoreEvent::RocksdbError\n                | StoreEvent::SqliteError\n                | StoreEvent::LdapError\n                | StoreEvent::ElasticsearchError\n                | StoreEvent::RedisError\n                | StoreEvent::S3Error\n                | StoreEvent::AzureError\n                | StoreEvent::FilesystemError\n                | StoreEvent::PoolError\n                | StoreEvent::DataCorruption\n                | StoreEvent::DecompressError\n                | StoreEvent::DeserializeError\n                | StoreEvent::NotFound\n                | StoreEvent::NotConfigured\n                | StoreEvent::NotSupported\n                | StoreEvent::UnexpectedError\n                | StoreEvent::CryptoError\n                | StoreEvent::BlobMissingMarker\n                | StoreEvent::DataWrite\n                | StoreEvent::DataIterate\n                | StoreEvent::BlobRead\n                | StoreEvent::BlobWrite\n                | StoreEvent::BlobDelete\n                | StoreEvent::HttpStoreError,\n            ) => true,\n            EventType::MessageIngest(_) => true,\n            EventType::Jmap(\n                JmapEvent::MethodCall\n                | JmapEvent::WebsocketStart\n                | JmapEvent::WebsocketError\n                | JmapEvent::UnsupportedFilter\n                | JmapEvent::UnsupportedSort\n                | JmapEvent::Forbidden\n                | JmapEvent::NotJson\n                | JmapEvent::NotRequest\n                | JmapEvent::InvalidArguments\n                | JmapEvent::RequestTooLarge\n                | JmapEvent::UnknownMethod,\n            ) => true,\n            EventType::Imap(ImapEvent::ConnectionStart | ImapEvent::ConnectionEnd) => true,\n            EventType::ManageSieve(\n                ManageSieveEvent::ConnectionStart | ManageSieveEvent::ConnectionEnd,\n            ) => true,\n            EventType::Pop3(Pop3Event::ConnectionStart | Pop3Event::ConnectionEnd) => true,\n            EventType::Smtp(\n                SmtpEvent::ConnectionStart\n                | SmtpEvent::ConnectionEnd\n                | SmtpEvent::Error\n                | SmtpEvent::ConcurrencyLimitExceeded\n                | SmtpEvent::TransferLimitExceeded\n                | SmtpEvent::RateLimitExceeded\n                | SmtpEvent::TimeLimitExceeded\n                | SmtpEvent::MessageParseFailed\n                | SmtpEvent::MessageTooLarge\n                | SmtpEvent::LoopDetected\n                | SmtpEvent::DkimPass\n                | SmtpEvent::DkimFail\n                | SmtpEvent::ArcPass\n                | SmtpEvent::ArcFail\n                | SmtpEvent::SpfEhloPass\n                | SmtpEvent::SpfEhloFail\n                | SmtpEvent::SpfFromPass\n                | SmtpEvent::SpfFromFail\n                | SmtpEvent::DmarcPass\n                | SmtpEvent::DmarcFail\n                | SmtpEvent::IprevPass\n                | SmtpEvent::IprevFail\n                | SmtpEvent::TooManyMessages\n                | SmtpEvent::InvalidEhlo\n                | SmtpEvent::DidNotSayEhlo\n                | SmtpEvent::MailFromUnauthenticated\n                | SmtpEvent::MailFromUnauthorized\n                | SmtpEvent::MailFromMissing\n                | SmtpEvent::MultipleMailFrom\n                | SmtpEvent::MailboxDoesNotExist\n                | SmtpEvent::RelayNotAllowed\n                | SmtpEvent::RcptToDuplicate\n                | SmtpEvent::RcptToMissing\n                | SmtpEvent::TooManyRecipients\n                | SmtpEvent::TooManyInvalidRcpt\n                | SmtpEvent::AuthMechanismNotSupported\n                | SmtpEvent::AuthExchangeTooLong\n                | SmtpEvent::CommandNotImplemented\n                | SmtpEvent::InvalidCommand\n                | SmtpEvent::SyntaxError\n                | SmtpEvent::RequestTooLarge,\n            ) => true,\n            EventType::Http(\n                HttpEvent::Error\n                | HttpEvent::RequestBody\n                | HttpEvent::ResponseBody\n                | HttpEvent::XForwardedMissing,\n            ) => true,\n            EventType::Network(NetworkEvent::Timeout) => true,\n            EventType::Security(_) => true,\n            EventType::Limit(_) => true,\n            EventType::Manage(_) => false,\n            EventType::Auth(\n                AuthEvent::Success\n                | AuthEvent::Failed\n                | AuthEvent::TooManyAttempts\n                | AuthEvent::Error,\n            ) => true,\n            EventType::Config(_) => false,\n            EventType::Resource(\n                ResourceEvent::NotFound | ResourceEvent::BadParameters | ResourceEvent::Error,\n            ) => true,\n            EventType::Arc(\n                ArcEvent::ChainTooLong\n                | ArcEvent::InvalidInstance\n                | ArcEvent::InvalidCv\n                | ArcEvent::HasHeaderTag\n                | ArcEvent::BrokenChain,\n            ) => true,\n            EventType::Dkim(_) => true,\n            EventType::Dmarc(_) => true,\n            EventType::Iprev(_) => true,\n            EventType::Dane(\n                DaneEvent::AuthenticationSuccess\n                | DaneEvent::AuthenticationFailure\n                | DaneEvent::NoCertificatesFound\n                | DaneEvent::CertificateParseError\n                | DaneEvent::TlsaRecordFetchError\n                | DaneEvent::TlsaRecordNotFound\n                | DaneEvent::TlsaRecordNotDnssecSigned\n                | DaneEvent::TlsaRecordInvalid,\n            ) => true,\n            EventType::Spf(_) => true,\n            EventType::MailAuth(_) => true,\n            EventType::Tls(TlsEvent::HandshakeError) => true,\n            EventType::Sieve(\n                SieveEvent::ActionAccept\n                | SieveEvent::ActionAcceptReplace\n                | SieveEvent::ActionDiscard\n                | SieveEvent::ActionReject\n                | SieveEvent::SendMessage\n                | SieveEvent::MessageTooLarge\n                | SieveEvent::RuntimeError\n                | SieveEvent::UnexpectedError\n                | SieveEvent::NotSupported\n                | SieveEvent::QuotaExceeded,\n            ) => true,\n            EventType::Spam(\n                SpamEvent::PyzorError\n                | SpamEvent::TrainCompleted\n                | SpamEvent::TrainSampleAdded\n                | SpamEvent::Classify\n                | SpamEvent::ModelNotReady\n                | SpamEvent::DnsblError,\n            ) => true,\n            EventType::PushSubscription(_) => true,\n            EventType::Cluster(\n                ClusterEvent::SubscriberError\n                | ClusterEvent::PublisherError\n                | ClusterEvent::SubscriberDisconnected,\n            ) => true,\n            EventType::Housekeeper(_) => false,\n            EventType::TaskQueue(\n                TaskQueueEvent::BlobNotFound | TaskQueueEvent::MetadataNotFound,\n            ) => true,\n            EventType::Milter(\n                MilterEvent::ActionAccept\n                | MilterEvent::ActionDiscard\n                | MilterEvent::ActionReject\n                | MilterEvent::ActionTempFail\n                | MilterEvent::ActionReplyCode\n                | MilterEvent::ActionConnectionFailure\n                | MilterEvent::ActionShutdown,\n            ) => true,\n            EventType::MtaHook(_) => true,\n            EventType::Delivery(\n                DeliveryEvent::AttemptStart\n                | DeliveryEvent::Completed\n                | DeliveryEvent::AttemptEnd\n                | DeliveryEvent::MxLookupFailed\n                | DeliveryEvent::IpLookupFailed\n                | DeliveryEvent::NullMx\n                | DeliveryEvent::GreetingFailed\n                | DeliveryEvent::EhloRejected\n                | DeliveryEvent::AuthFailed\n                | DeliveryEvent::MailFromRejected\n                | DeliveryEvent::Delivered\n                | DeliveryEvent::RcptToRejected\n                | DeliveryEvent::RcptToFailed\n                | DeliveryEvent::MessageRejected\n                | DeliveryEvent::StartTlsUnavailable\n                | DeliveryEvent::StartTlsError\n                | DeliveryEvent::StartTlsDisabled\n                | DeliveryEvent::ImplicitTlsError\n                | DeliveryEvent::ConcurrencyLimitExceeded\n                | DeliveryEvent::RateLimitExceeded\n                | DeliveryEvent::DoubleBounce\n                | DeliveryEvent::DsnSuccess\n                | DeliveryEvent::DsnTempFail\n                | DeliveryEvent::DsnPermFail,\n            ) => true,\n            EventType::Queue(\n                QueueEvent::QueueMessage\n                | QueueEvent::QueueMessageAuthenticated\n                | QueueEvent::QueueReport\n                | QueueEvent::QueueDsn\n                | QueueEvent::QueueAutogenerated\n                | QueueEvent::Rescheduled\n                | QueueEvent::BlobNotFound\n                | QueueEvent::RateLimitExceeded\n                | QueueEvent::ConcurrencyLimitExceeded\n                | QueueEvent::QuotaExceeded,\n            ) => true,\n            EventType::TlsRpt(_) => false,\n            EventType::MtaSts(\n                MtaStsEvent::Authorized | MtaStsEvent::NotAuthorized | MtaStsEvent::InvalidPolicy,\n            ) => true,\n            EventType::IncomingReport(_) => true,\n            EventType::OutgoingReport(\n                OutgoingReportEvent::SpfReport\n                | OutgoingReportEvent::SpfRateLimited\n                | OutgoingReportEvent::DkimReport\n                | OutgoingReportEvent::DkimRateLimited\n                | OutgoingReportEvent::DmarcReport\n                | OutgoingReportEvent::DmarcRateLimited\n                | OutgoingReportEvent::DmarcAggregateReport\n                | OutgoingReportEvent::TlsAggregate\n                | OutgoingReportEvent::HttpSubmission\n                | OutgoingReportEvent::UnauthorizedReportingAddress\n                | OutgoingReportEvent::ReportingAddressValidationError\n                | OutgoingReportEvent::NotFound\n                | OutgoingReportEvent::SubmissionError\n                | OutgoingReportEvent::NoRecipientsFound,\n            ) => true,\n            EventType::Telemetry(\n                TelemetryEvent::LogError\n                | TelemetryEvent::WebhookError\n                | TelemetryEvent::OtelExporterError\n                | TelemetryEvent::OtelMetricsExporterError\n                | TelemetryEvent::PrometheusExporterError\n                | TelemetryEvent::JournalError,\n            ) => true,\n            EventType::Calendar(\n                CalendarEvent::AlarmSent\n                | CalendarEvent::AlarmFailed\n                | CalendarEvent::ItipMessageReceived\n                | CalendarEvent::ItipMessageSent\n                | CalendarEvent::ItipMessageError,\n            ) => true,\n            _ => false,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/ipc/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod bitset;\npub mod channel;\npub mod collector;\npub mod metrics;\npub mod subscriber;\n\npub(crate) const USIZE_BITS: usize = std::mem::size_of::<usize>() * 8;\npub(crate) const USIZE_BITS_MASK: usize = USIZE_BITS - 1;\n"
  },
  {
    "path": "crates/trc/src/ipc/subscriber.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse tokio::sync::mpsc::{self, error::TrySendError};\n\nuse crate::{Event, EventDetails, EventType, Level, TOTAL_EVENT_COUNT};\n\nuse super::{\n    USIZE_BITS,\n    bitset::Bitset,\n    channel::ChannelError,\n    collector::{COLLECTOR_UPDATES, Collector, Update},\n};\n\nconst MAX_BATCH_SIZE: usize = 32768;\n\npub type Interests = Box<Bitset<{ TOTAL_EVENT_COUNT.div_ceil(USIZE_BITS) }>>;\npub type EventBatch = Vec<Arc<Event<EventDetails>>>;\n\n#[derive(Debug)]\npub(crate) struct Subscriber {\n    pub id: String,\n    pub interests: Interests,\n    pub tx: mpsc::Sender<EventBatch>,\n    pub lossy: bool,\n    pub batch: EventBatch,\n}\n\npub struct SubscriberBuilder {\n    pub id: String,\n    pub interests: Interests,\n    pub lossy: bool,\n}\n\nimpl Subscriber {\n    #[inline(always)]\n    pub fn push_event(&mut self, event_id: usize, trace: Arc<Event<EventDetails>>) {\n        if self.interests.get(event_id) {\n            self.batch.push(trace);\n        }\n    }\n\n    pub fn send_batch(&mut self) -> Result<(), ChannelError> {\n        if !self.batch.is_empty() {\n            match self\n                .tx\n                .try_send(std::mem::replace(&mut self.batch, Vec::with_capacity(128)))\n            {\n                Ok(_) => Ok(()),\n                Err(TrySendError::Full(mut events)) => {\n                    if self.lossy && events.len() > MAX_BATCH_SIZE {\n                        events.retain(|e| e.inner.level == Level::Error);\n                        if events.len() > MAX_BATCH_SIZE {\n                            events.truncate(MAX_BATCH_SIZE);\n                        }\n                    }\n                    self.batch = events;\n                    Ok(())\n                }\n                Err(TrySendError::Closed(_)) => Err(ChannelError),\n            }\n        } else {\n            Ok(())\n        }\n    }\n}\n\nimpl SubscriberBuilder {\n    pub fn new(id: String) -> Self {\n        Self {\n            id,\n            interests: Default::default(),\n            lossy: true,\n        }\n    }\n\n    pub fn with_default_interests(mut self, level: Level) -> Self {\n        for event in EventType::variants() {\n            if event.level() >= level {\n                self.interests.set(event);\n            }\n        }\n        self\n    }\n\n    pub fn with_interests(mut self, interests: Interests) -> Self {\n        self.interests = interests;\n        self\n    }\n\n    pub fn set_interests(mut self, interest: impl IntoIterator<Item = impl Into<usize>>) -> Self {\n        for level in interest {\n            self.interests.set(level);\n        }\n        self\n    }\n\n    pub fn with_lossy(mut self, lossy: bool) -> Self {\n        self.lossy = lossy;\n        self\n    }\n\n    pub fn register(self) -> (mpsc::Sender<EventBatch>, mpsc::Receiver<EventBatch>) {\n        let (tx, rx) = mpsc::channel(8192);\n\n        COLLECTOR_UPDATES.lock().push(Update::RegisterSubscriber {\n            subscriber: Subscriber {\n                id: self.id,\n                interests: self.interests,\n                tx: tx.clone(),\n                lossy: self.lossy,\n                batch: Vec::new(),\n            },\n        });\n\n        // Notify collector\n        Collector::reload();\n\n        (tx, rx)\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod atomics;\npub mod event;\npub mod ipc;\npub mod macros;\npub mod serializers;\n\nuse std::{\n    net::{IpAddr, Ipv4Addr, Ipv6Addr},\n    sync::Arc,\n};\n\npub use crate::ipc::collector::Collector;\nuse compact_str::CompactString;\npub use event_macro::event;\n\nuse event_macro::{event_family, event_type, key_names, total_event_count};\n\npub type Result<T> = std::result::Result<T, Error>;\n\n#[derive(Debug, Clone)]\n#[repr(transparent)]\npub struct Error(Box<Event<EventType>>);\n\n#[derive(Debug, Clone)]\npub struct Event<T> {\n    pub inner: T,\n    pub keys: Vec<(Key, Value)>,\n}\n\n#[derive(Debug, Clone)]\npub struct EventDetails {\n    pub typ: EventType,\n    pub timestamp: u64,\n    pub level: Level,\n    pub span: Option<Arc<Event<EventDetails>>>,\n}\n\n#[derive(Copy, Clone, Debug, Hash, Eq, PartialEq)]\n#[repr(usize)]\npub enum Level {\n    Trace = 0,\n    Debug = 1,\n    Info = 2,\n    Warn = 3,\n    Error = 4,\n    Disable = 5,\n}\n\n#[derive(Debug, Default, Clone)]\npub enum Value {\n    String(CompactString),\n    UInt(u64),\n    Int(i64),\n    Float(f64),\n    Timestamp(u64),\n    Duration(u64),\n    Bytes(Vec<u8>),\n    Bool(bool),\n    Ipv4(Ipv4Addr),\n    Ipv6(Ipv6Addr),\n    Event(Error),\n    Array(Vec<Value>),\n    #[default]\n    None,\n}\n\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash)]\n#[key_names]\npub enum Key {\n    AccountName,\n    AccountId,\n    BlobId,\n    #[default]\n    CausedBy,\n    ChangeId,\n    Code,\n    Collection,\n    Contents,\n    Details,\n    DkimFail,\n    DkimNone,\n    DkimPass,\n    DmarcNone,\n    DmarcPass,\n    DmarcQuarantine,\n    DmarcReject,\n    DocumentId,\n    Domain,\n    Due,\n    Elapsed,\n    Expires,\n    From,\n    Hostname,\n    Id,\n    Key,\n    Limit,\n    ListenerId,\n    LocalIp,\n    LocalPort,\n    MailboxName,\n    MailboxId,\n    MessageId,\n    NextDsn,\n    NextRetry,\n    Path,\n    Policy,\n    QueueId,\n    RangeFrom,\n    RangeTo,\n    Reason,\n    RemoteIp,\n    RemotePort,\n    ReportId,\n    Result,\n    Size,\n    Source,\n    SpanId,\n    SpfFail,\n    SpfNone,\n    SpfPass,\n    Strict,\n    Tls,\n    To,\n    Total,\n    TotalFailures,\n    TotalSuccesses,\n    Type,\n    Uid,\n    UidNext,\n    UidValidity,\n    Url,\n    ValidFrom,\n    ValidTo,\n    Value,\n    Version,\n    QueueName,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\n#[event_family]\npub enum EventType {\n    Server(ServerEvent),\n    Purge(PurgeEvent),\n    Eval(EvalEvent),\n    Acme(AcmeEvent),\n    Store(StoreEvent),\n    MessageIngest(MessageIngestEvent),\n    Jmap(JmapEvent),\n    Imap(ImapEvent),\n    ManageSieve(ManageSieveEvent),\n    Pop3(Pop3Event),\n    Smtp(SmtpEvent),\n    Http(HttpEvent),\n    Network(NetworkEvent),\n    Limit(LimitEvent),\n    Manage(ManageEvent),\n    Auth(AuthEvent),\n    Config(ConfigEvent),\n    Resource(ResourceEvent),\n    Arc(ArcEvent),\n    Dkim(DkimEvent),\n    Dmarc(DmarcEvent),\n    Iprev(IprevEvent),\n    Dane(DaneEvent),\n    Spf(SpfEvent),\n    MailAuth(MailAuthEvent),\n    Tls(TlsEvent),\n    Sieve(SieveEvent),\n    Spam(SpamEvent),\n    PushSubscription(PushSubscriptionEvent),\n    Cluster(ClusterEvent),\n    Housekeeper(HousekeeperEvent),\n    TaskQueue(TaskQueueEvent),\n    Milter(MilterEvent),\n    MtaHook(MtaHookEvent),\n    Delivery(DeliveryEvent),\n    Queue(QueueEvent),\n    TlsRpt(TlsRptEvent),\n    MtaSts(MtaStsEvent),\n    IncomingReport(IncomingReportEvent),\n    OutgoingReport(OutgoingReportEvent),\n    Telemetry(TelemetryEvent),\n    Security(SecurityEvent),\n    Ai(AiEvent),\n    WebDav(WebDavEvent),\n    Calendar(CalendarEvent),\n}\n\n#[event_type]\npub enum HttpEvent {\n    ConnectionStart,\n    ConnectionEnd,\n    Error,\n    RequestUrl,\n    RequestBody,\n    ResponseBody,\n    XForwardedMissing,\n}\n\n#[event_type]\npub enum SecurityEvent {\n    AuthenticationBan,\n    AbuseBan,\n    ScanBan,\n    LoiterBan,\n    IpBlocked,\n    Unauthorized,\n}\n\n#[event_type]\npub enum ClusterEvent {\n    SubscriberStart,\n    SubscriberStop,\n    SubscriberError,\n    SubscriberDisconnected,\n    PublisherStart,\n    PublisherStop,\n    PublisherError,\n    MessageReceived,\n    MessageSkipped,\n    MessageInvalid,\n}\n\n#[event_type]\npub enum HousekeeperEvent {\n    Start,\n    Stop,\n    Schedule,\n    Run,\n}\n\n#[event_type]\npub enum TaskQueueEvent {\n    TaskAcquired,\n    TaskLocked,\n    TaskIgnored,\n    TaskFailed,\n    BlobNotFound,\n    MetadataNotFound,\n}\n\n#[event_type]\npub enum ImapEvent {\n    ConnectionStart,\n    ConnectionEnd,\n\n    // Commands\n    GetAcl,\n    SetAcl,\n    MyRights,\n    ListRights,\n    Append,\n    Capabilities,\n    Id,\n    Close,\n    Copy,\n    Move,\n    CreateMailbox,\n    DeleteMailbox,\n    RenameMailbox,\n    Enable,\n    Expunge,\n    Fetch,\n    IdleStart,\n    IdleStop,\n    List,\n    Lsub,\n    Logout,\n    Namespace,\n    Noop,\n    Search,\n    Sort,\n    Select,\n    Status,\n    Store,\n    Subscribe,\n    Unsubscribe,\n    Thread,\n    GetQuota,\n\n    // Errors\n    Error,\n\n    // Debugging\n    RawInput,\n    RawOutput,\n}\n\n#[event_type]\npub enum Pop3Event {\n    ConnectionStart,\n    ConnectionEnd,\n\n    // Commands\n    Delete,\n    Reset,\n    Quit,\n    Fetch,\n    List,\n    ListMessage,\n    Uidl,\n    UidlMessage,\n    Stat,\n    Noop,\n    Capabilities,\n    StartTls,\n    Utf8,\n\n    // Errors\n    Error,\n\n    // Debugging\n    RawInput,\n    RawOutput,\n}\n\n#[event_type]\npub enum ManageSieveEvent {\n    ConnectionStart,\n    ConnectionEnd,\n\n    // Commands\n    CreateScript,\n    UpdateScript,\n    GetScript,\n    DeleteScript,\n    RenameScript,\n    CheckScript,\n    HaveSpace,\n    ListScripts,\n    SetActive,\n    Capabilities,\n    StartTls,\n    Unauthenticate,\n    Logout,\n    Noop,\n\n    // Errors\n    Error,\n\n    // Debugging\n    RawInput,\n    RawOutput,\n}\n\n#[event_type]\npub enum SmtpEvent {\n    ConnectionStart,\n    ConnectionEnd,\n    Error,\n    IdNotFound,\n    ConcurrencyLimitExceeded,\n    TransferLimitExceeded,\n    RateLimitExceeded,\n    TimeLimitExceeded,\n    MissingAuthDirectory,\n    MessageParseFailed,\n    MessageTooLarge,\n    LoopDetected,\n    DkimPass,\n    DkimFail,\n    ArcPass,\n    ArcFail,\n    SpfEhloPass,\n    SpfEhloFail,\n    SpfFromPass,\n    SpfFromFail,\n    DmarcPass,\n    DmarcFail,\n    IprevPass,\n    IprevFail,\n    TooManyMessages,\n    Ehlo,\n    InvalidEhlo,\n    DidNotSayEhlo,\n    EhloExpected,\n    LhloExpected,\n    MailFromUnauthenticated,\n    MailFromUnauthorized,\n    MailFromNotAllowed,\n    MailFromRewritten,\n    MailFromMissing,\n    MailFrom,\n    MultipleMailFrom,\n    MailboxDoesNotExist,\n    RelayNotAllowed,\n    RcptTo,\n    RcptToDuplicate,\n    RcptToRewritten,\n    RcptToMissing,\n    RcptToGreylisted,\n    TooManyRecipients,\n    TooManyInvalidRcpt,\n    RawInput,\n    RawOutput,\n    MissingLocalHostname,\n    Vrfy,\n    VrfyNotFound,\n    VrfyDisabled,\n    Expn,\n    ExpnNotFound,\n    ExpnDisabled,\n    RequireTlsDisabled,\n    DeliverByDisabled,\n    DeliverByInvalid,\n    FutureReleaseDisabled,\n    FutureReleaseInvalid,\n    MtPriorityDisabled,\n    MtPriorityInvalid,\n    DsnDisabled,\n    AuthNotAllowed,\n    AuthMechanismNotSupported,\n    AuthExchangeTooLong,\n    AlreadyAuthenticated,\n    Noop,\n    StartTls,\n    StartTlsUnavailable,\n    StartTlsAlready,\n    Rset,\n    Quit,\n    Help,\n    CommandNotImplemented,\n    InvalidCommand,\n    InvalidSenderAddress,\n    InvalidRecipientAddress,\n    InvalidParameter,\n    UnsupportedParameter,\n    SyntaxError,\n    RequestTooLarge,\n}\n\n#[event_type]\npub enum DeliveryEvent {\n    AttemptStart,\n    AttemptEnd,\n    Completed,\n    Failed,\n    DomainDeliveryStart,\n    MxLookup,\n    MxLookupFailed,\n    IpLookup,\n    IpLookupFailed,\n    NullMx,\n    Connect,\n    ConnectError,\n    MissingOutboundHostname,\n    GreetingFailed,\n    Ehlo,\n    EhloRejected,\n    Auth,\n    AuthFailed,\n    MailFrom,\n    MailFromRejected,\n    Delivered,\n    RcptTo,\n    RcptToRejected,\n    RcptToFailed,\n    MessageRejected,\n    StartTls,\n    StartTlsUnavailable,\n    StartTlsError,\n    StartTlsDisabled,\n    ImplicitTlsError,\n    ConcurrencyLimitExceeded,\n    RateLimitExceeded,\n    DoubleBounce,\n    DsnSuccess,\n    DsnTempFail,\n    DsnPermFail,\n    RawInput,\n    RawOutput,\n}\n\n#[event_type]\npub enum QueueEvent {\n    QueueMessage,\n    QueueMessageAuthenticated,\n    QueueReport,\n    QueueDsn,\n    QueueAutogenerated,\n    Rescheduled,\n    Locked,\n    BlobNotFound,\n    RateLimitExceeded,\n    ConcurrencyLimitExceeded,\n    QuotaExceeded,\n    BackPressure,\n}\n\n#[event_type]\npub enum IncomingReportEvent {\n    DmarcReport,\n    DmarcReportWithWarnings,\n    TlsReport,\n    TlsReportWithWarnings,\n    AbuseReport,\n    AuthFailureReport,\n    FraudReport,\n    NotSpamReport,\n    VirusReport,\n    OtherReport,\n    MessageParseFailed,\n    DmarcParseFailed,\n    TlsRpcParseFailed,\n    ArfParseFailed,\n    DecompressError,\n}\n\n#[event_type]\npub enum OutgoingReportEvent {\n    SpfReport,\n    SpfRateLimited,\n    DkimReport,\n    DkimRateLimited,\n    DmarcReport,\n    DmarcRateLimited,\n    DmarcAggregateReport,\n    TlsAggregate,\n    HttpSubmission,\n    UnauthorizedReportingAddress,\n    ReportingAddressValidationError,\n    NotFound,\n    SubmissionError,\n    NoRecipientsFound,\n    Locked,\n}\n\n#[event_type]\npub enum MtaStsEvent {\n    Authorized,\n    NotAuthorized,\n    PolicyFetch,\n    PolicyNotFound,\n    PolicyFetchError,\n    InvalidPolicy,\n}\n\n#[event_type]\npub enum TlsRptEvent {\n    RecordFetch,\n    RecordFetchError,\n    RecordNotFound,\n}\n\n#[event_type]\npub enum DaneEvent {\n    AuthenticationSuccess,\n    AuthenticationFailure,\n    NoCertificatesFound,\n    CertificateParseError,\n    TlsaRecordMatch,\n    TlsaRecordFetch,\n    TlsaRecordFetchError,\n    TlsaRecordNotFound,\n    TlsaRecordNotDnssecSigned,\n    TlsaRecordInvalid,\n}\n\n#[event_type]\npub enum MilterEvent {\n    Read,\n    Write,\n    ActionAccept,\n    ActionDiscard,\n    ActionReject,\n    ActionTempFail,\n    ActionReplyCode,\n    ActionConnectionFailure,\n    ActionShutdown,\n    IoError,\n    FrameTooLarge,\n    FrameInvalid,\n    UnexpectedResponse,\n    Timeout,\n    TlsInvalidName,\n    Disconnected,\n    ParseError,\n}\n\n#[event_type]\npub enum MtaHookEvent {\n    ActionAccept,\n    ActionDiscard,\n    ActionReject,\n    ActionQuarantine,\n    Error,\n}\n\n#[event_type]\npub enum PushSubscriptionEvent {\n    Success,\n    Error,\n    NotFound,\n}\n\n#[event_type]\npub enum SpamEvent {\n    Pyzor,\n    PyzorError,\n    Dnsbl,\n    DnsblError,\n    TrainStarted,\n    TrainCompleted,\n    TrainSampleAdded,\n    TrainSampleNotFound,\n    Classify,\n    ModelLoaded,\n    ModelNotReady,\n    ModelNotFound,\n}\n\n#[event_type]\npub enum SieveEvent {\n    ActionAccept,\n    ActionAcceptReplace,\n    ActionDiscard,\n    ActionReject,\n    SendMessage,\n    MessageTooLarge,\n    ScriptNotFound,\n    ListNotFound,\n    RuntimeError,\n    UnexpectedError,\n    NotSupported,\n    QuotaExceeded,\n}\n\n#[event_type]\npub enum TlsEvent {\n    Handshake,\n    HandshakeError,\n    NotConfigured,\n    CertificateNotFound,\n    NoCertificatesAvailable,\n    MultipleCertificatesAvailable,\n}\n\n#[event_type]\npub enum NetworkEvent {\n    ListenStart,\n    ListenStop,\n    ListenError,\n    BindError,\n    ReadError,\n    WriteError,\n    FlushError,\n    AcceptError,\n    SplitError,\n    Timeout,\n    Closed,\n    ProxyError,\n    SetOptError,\n}\n\n#[event_type]\npub enum ServerEvent {\n    Startup,\n    Shutdown,\n    StartupError,\n    ThreadError,\n    Licensing,\n}\n\n#[event_type]\npub enum TelemetryEvent {\n    Alert,\n    LogError,\n    WebhookError,\n    OtelExporterError,\n    OtelMetricsExporterError,\n    PrometheusExporterError,\n    JournalError,\n}\n\n#[event_type]\npub enum AcmeEvent {\n    AuthStart,\n    AuthPending,\n    AuthValid,\n    AuthCompleted,\n    AuthError,\n    AuthTooManyAttempts,\n    ProcessCert,\n    OrderStart,\n    OrderProcessing,\n    OrderCompleted,\n    OrderReady,\n    OrderValid,\n    OrderInvalid,\n    RenewBackoff,\n    DnsRecordCreated,\n    DnsRecordCreationFailed,\n    DnsRecordDeletionFailed,\n    DnsRecordNotPropagated,\n    DnsRecordLookupFailed,\n    DnsRecordPropagated,\n    DnsRecordPropagationTimeout,\n    ClientSuppliedSni,\n    ClientMissingSni,\n    TlsAlpnReceived,\n    TlsAlpnError,\n    TokenNotFound,\n    Error,\n}\n\n#[event_type]\npub enum PurgeEvent {\n    Started,\n    Finished,\n    Running,\n    Error,\n    InProgress,\n    AutoExpunge,\n    BlobCleanup,\n}\n\n#[event_type]\npub enum EvalEvent {\n    Result,\n    Error,\n    DirectoryNotFound,\n    StoreNotFound,\n}\n\n#[event_type]\npub enum ConfigEvent {\n    ParseError,\n    BuildError,\n    MacroError,\n    WriteError,\n    FetchError,\n    DefaultApplied,\n    MissingSetting,\n    UnusedSetting,\n    ParseWarning,\n    BuildWarning,\n    ImportExternal,\n    AlreadyUpToDate,\n}\n\n#[event_type]\npub enum ArcEvent {\n    ChainTooLong,\n    InvalidInstance,\n    InvalidCv,\n    HasHeaderTag,\n    BrokenChain,\n    SealerNotFound,\n}\n\n#[event_type]\npub enum DkimEvent {\n    Pass,\n    Neutral,\n    Fail,\n    PermError,\n    TempError,\n    None,\n    UnsupportedVersion,\n    UnsupportedAlgorithm,\n    UnsupportedCanonicalization,\n    UnsupportedKeyType,\n    FailedBodyHashMatch,\n    FailedVerification,\n    FailedAuidMatch,\n    RevokedPublicKey,\n    IncompatibleAlgorithms,\n    SignatureExpired,\n    SignatureLength,\n    SignerNotFound,\n}\n\n#[event_type]\npub enum SpfEvent {\n    Pass,\n    Fail,\n    SoftFail,\n    Neutral,\n    TempError,\n    PermError,\n    None,\n}\n\n#[event_type]\npub enum DmarcEvent {\n    Pass,\n    Fail,\n    PermError,\n    TempError,\n    None,\n}\n\n#[event_type]\npub enum IprevEvent {\n    Pass,\n    Fail,\n    PermError,\n    TempError,\n    None,\n}\n\n#[event_type]\npub enum MailAuthEvent {\n    ParseError,\n    MissingParameters,\n    NoHeadersFound,\n    Crypto,\n    Io,\n    Base64,\n    DnsError,\n    DnsRecordNotFound,\n    DnsInvalidRecordType,\n    PolicyNotAligned,\n}\n\n#[event_type]\npub enum StoreEvent {\n    // Errors\n    AssertValueFailed,\n    FoundationdbError,\n    MysqlError,\n    PostgresqlError,\n    RocksdbError,\n    SqliteError,\n    LdapError,\n    ElasticsearchError,\n    MeilisearchError,\n    RedisError,\n    S3Error,\n    AzureError,\n    FilesystemError,\n    PoolError,\n    DataCorruption,\n    DecompressError,\n    DeserializeError,\n    NotFound,\n    NotConfigured,\n    NotSupported,\n    UnexpectedError,\n    CryptoError,\n    HttpStoreError,\n\n    // Caching\n    CacheMiss,\n    CacheHit,\n    CacheStale,\n    CacheUpdate,\n\n    // Warnings\n    BlobMissingMarker,\n\n    // Traces\n    DataWrite,\n    DataIterate,\n    BlobRead,\n    BlobWrite,\n    BlobDelete,\n    SqlQuery,\n    LdapQuery,\n    LdapWarning,\n    HttpStoreFetch,\n}\n\n#[event_type]\npub enum MessageIngestEvent {\n    // Events\n    Ham,\n    Spam,\n    ImapAppend,\n    JmapAppend,\n    Duplicate,\n    Error,\n    FtsIndex,\n}\n\n#[event_type]\npub enum JmapEvent {\n    // Calls\n    MethodCall,\n\n    // Method errors\n    InvalidArguments,\n    RequestTooLarge,\n    StateMismatch,\n    AnchorNotFound,\n    UnsupportedFilter,\n    UnsupportedSort,\n    UnknownMethod,\n    InvalidResultReference,\n    Forbidden,\n    AccountNotFound,\n    AccountNotSupportedByMethod,\n    AccountReadOnly,\n    NotFound,\n    CannotCalculateChanges,\n    UnknownDataType,\n\n    // Request errors\n    UnknownCapability,\n    NotJson,\n    NotRequest,\n\n    // Not JMAP standard\n    WebsocketStart,\n    WebsocketStop,\n    WebsocketError,\n}\n\n#[event_type]\npub enum LimitEvent {\n    SizeRequest,\n    SizeUpload,\n    CallsIn,\n    ConcurrentRequest,\n    ConcurrentUpload,\n    ConcurrentConnection, // Used by listener\n    Quota,\n    BlobQuota,\n    TenantQuota,\n    TooManyRequests,\n}\n\n#[event_type]\npub enum ManageEvent {\n    MissingParameter,\n    AlreadyExists,\n    AssertFailed,\n    NotFound,\n    NotSupported,\n    Error,\n}\n\n#[event_type]\npub enum AuthEvent {\n    Success,\n    Failed,\n    TokenExpired,\n    MissingTotp,\n    TooManyAttempts,\n    ClientRegistration,\n    Error,\n}\n\n#[event_type]\npub enum ResourceEvent {\n    NotFound,\n    BadParameters,\n    Error,\n    DownloadExternal,\n    WebadminUnpacked,\n}\n\n#[event_type]\npub enum AiEvent {\n    LlmResponse,\n    ApiError,\n}\n\n#[event_type]\npub enum WebDavEvent {\n    // Requests\n    Propfind,\n    Proppatch,\n    Get,\n    Head,\n    Report,\n    Mkcol,\n    Mkcalendar,\n    Delete,\n    Put,\n    Post,\n    Patch,\n    Copy,\n    Move,\n    Lock,\n    Unlock,\n    Acl,\n    Options,\n\n    // Errors\n    Error,\n}\n\n#[event_type]\npub enum CalendarEvent {\n    RuleExpansionError,\n    AlarmSent,\n    AlarmSkipped,\n    AlarmRecipientOverride,\n    AlarmFailed,\n    ItipMessageSent,\n    ItipMessageReceived,\n    ItipMessageError,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub enum MetricType {\n    ServerMemory,\n    MessageIngestionTime,\n    MessageFtsIndexTime,\n    MessageSize,\n    MessageAuthSize,\n    DeliveryTotalTime,\n    DeliveryTime,\n    DeliveryActiveConnections,\n    QueueCount,\n    ReportOutgoingSize,\n    StoreReadTime,\n    StoreWriteTime,\n    BlobReadTime,\n    BlobWriteTime,\n    DnsLookupTime,\n    HttpActiveConnections,\n    HttpRequestTime,\n    ImapActiveConnections,\n    ImapRequestTime,\n    Pop3ActiveConnections,\n    Pop3RequestTime,\n    SmtpActiveConnections,\n    SmtpRequestTime,\n    SieveActiveConnections,\n    SieveRequestTime,\n    UserCount,\n    DomainCount,\n}\n\npub const TOTAL_EVENT_COUNT: usize = total_event_count!();\n\npub trait AddContext<T> {\n    fn caused_by(self, location: &'static str) -> Result<T>;\n    fn add_context<F>(self, f: F) -> Result<T>\n    where\n        F: FnOnce(Error) -> Error;\n}\n"
  },
  {
    "path": "crates/trc/src/macros.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[macro_export]\nmacro_rules! location {\n    () => {{ concat!(file!(), \":\", line!()) }};\n}\n\n#[macro_export]\nmacro_rules! bail {\n    ($err:expr $(,)?) => {\n        return Err($err);\n    };\n}\n\n#[macro_export]\nmacro_rules! error {\n    ($err:expr $(,)?) => {\n        let err = $err;\n        let event_id = err.as_ref().id();\n\n        if $crate::Collector::is_metric(event_id) {\n            $crate::Collector::record_metric(*err.as_ref(), event_id, err.keys());\n        }\n        if $crate::Collector::has_interest(event_id) {\n            err.send();\n        }\n    };\n}\n"
  },
  {
    "path": "crates/trc/src/serializers/binary.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse crate::*;\nuse compact_str::format_compact;\nuse std::net::{Ipv4Addr, Ipv6Addr};\n\nconst VERSION: u8 = 1;\n\npub fn serialize_events<'x>(\n    events: impl IntoIterator<Item = &'x Event<EventDetails>>,\n    num_events: usize,\n) -> Vec<u8> {\n    let mut buf = Vec::with_capacity(num_events * 64);\n    buf.push(VERSION);\n    leb128_write(&mut buf, num_events as u64);\n    for event in events {\n        event.serialize(&mut buf);\n    }\n    buf\n}\n\npub fn deserialize_events(bytes: &[u8]) -> crate::Result<Vec<Event<EventDetails>>> {\n    let mut iter = bytes.iter();\n    if *iter.next().ok_or_else(|| {\n        StoreEvent::DataCorruption\n            .caused_by(crate::location!())\n            .details(\"EOF while reading version\")\n    })? != VERSION\n    {\n        crate::bail!(\n            StoreEvent::DataCorruption\n                .caused_by(crate::location!())\n                .details(\"Invalid version\")\n        );\n    }\n    let len = leb128_read(&mut iter).ok_or_else(|| {\n        StoreEvent::DataCorruption\n            .caused_by(crate::location!())\n            .details(\"EOF while size\")\n    })? as usize;\n    let mut events = Vec::with_capacity(len);\n    for n in 0..len {\n        events.push(Event::deserialize(&mut iter).ok_or_else(|| {\n            StoreEvent::DataCorruption\n                .caused_by(crate::location!())\n                .details(format_compact!(\"Failed to deserialize event {n}\"))\n        })?);\n    }\n    Ok(events)\n}\n\npub fn deserialize_single_event(bytes: &[u8]) -> crate::Result<Event<EventDetails>> {\n    let mut iter = bytes.iter();\n    if *iter.next().ok_or_else(|| {\n        StoreEvent::DataCorruption\n            .caused_by(crate::location!())\n            .details(\"EOF while reading version\")\n    })? != VERSION\n    {\n        crate::bail!(\n            StoreEvent::DataCorruption\n                .caused_by(crate::location!())\n                .details(\"Invalid version\")\n        );\n    }\n    let _ = leb128_read(&mut iter).ok_or_else(|| {\n        StoreEvent::DataCorruption\n            .caused_by(crate::location!())\n            .details(\"EOF while size\")\n    })?;\n    Event::deserialize(&mut iter).ok_or_else(|| {\n        StoreEvent::DataCorruption\n            .caused_by(crate::location!())\n            .details(\"Failed to deserialize event\")\n    })\n}\n\nimpl Event<EventDetails> {\n    pub fn serialize(&self, buf: &mut Vec<u8>) {\n        leb128_write(buf, self.inner.typ.code());\n        buf.extend_from_slice(self.inner.timestamp.to_le_bytes().as_ref());\n        leb128_write(buf, self.keys.len() as u64);\n        for (k, v) in &self.keys {\n            leb128_write(buf, k.code());\n            v.serialize(buf);\n        }\n    }\n    pub fn deserialize<'x>(iter: &mut impl Iterator<Item = &'x u8>) -> Option<Self> {\n        let typ = EventType::from_code(leb128_read(iter)?)?;\n        let timestamp = u64::from_le_bytes([\n            *iter.next()?,\n            *iter.next()?,\n            *iter.next()?,\n            *iter.next()?,\n            *iter.next()?,\n            *iter.next()?,\n            *iter.next()?,\n            *iter.next()?,\n        ]);\n        let keys_len = leb128_read(iter)?;\n        let mut keys = Vec::with_capacity(keys_len as usize);\n        for _ in 0..keys_len {\n            let key = Key::from_code(leb128_read(iter)?)?;\n            let value = Value::deserialize(iter)?;\n            keys.push((key, value));\n        }\n        Some(Event {\n            inner: EventDetails {\n                typ,\n                timestamp,\n                level: Level::Info,\n                span: None,\n            },\n            keys,\n        })\n    }\n}\n\nimpl Value {\n    fn serialize(&self, buf: &mut Vec<u8>) {\n        match self {\n            Value::String(v) => {\n                buf.push(0u8);\n                leb128_write(buf, v.len() as u64);\n                buf.extend(v.as_bytes());\n            }\n            Value::UInt(v) => {\n                buf.push(1u8);\n                leb128_write(buf, *v);\n            }\n            Value::Int(v) => {\n                buf.push(2u8);\n                buf.extend(&v.to_le_bytes());\n            }\n            Value::Float(v) => {\n                buf.push(3u8);\n                buf.extend(&v.to_le_bytes());\n            }\n            Value::Timestamp(v) => {\n                buf.push(4u8);\n                buf.extend(&v.to_le_bytes());\n            }\n            Value::Duration(v) => {\n                buf.push(5u8);\n                leb128_write(buf, *v);\n            }\n            Value::Bytes(v) => {\n                buf.push(6u8);\n                leb128_write(buf, v.len() as u64);\n                buf.extend(v);\n            }\n            Value::Bool(true) => {\n                buf.push(7u8);\n            }\n            Value::Bool(false) => {\n                buf.push(8u8);\n            }\n            Value::Ipv4(v) => {\n                buf.push(9u8);\n                buf.extend(&v.octets());\n            }\n            Value::Ipv6(v) => {\n                buf.push(10u8);\n                buf.extend(&v.octets());\n            }\n            Value::Event(v) => {\n                buf.push(11u8);\n                leb128_write(buf, v.0.inner.code());\n                leb128_write(buf, v.0.keys.len() as u64);\n                for (k, v) in &v.0.keys {\n                    leb128_write(buf, k.code());\n                    v.serialize(buf);\n                }\n            }\n            Value::Array(v) => {\n                buf.push(12u8);\n                leb128_write(buf, v.len() as u64);\n                for value in v {\n                    value.serialize(buf);\n                }\n            }\n            Value::None => {\n                buf.push(13u8);\n            }\n        }\n    }\n\n    fn deserialize<'x>(iter: &mut impl Iterator<Item = &'x u8>) -> Option<Self> {\n        match iter.next()? {\n            0 => {\n                let mut buf = vec![0u8; leb128_read(iter)? as usize];\n                for byte in buf.iter_mut() {\n                    *byte = *iter.next()?;\n                }\n                Some(Value::String(CompactString::from_utf8(buf).ok()?))\n            }\n            1 => Some(Value::UInt(leb128_read(iter)?)),\n            2 => {\n                let mut buf = [0u8; std::mem::size_of::<i64>()];\n                for byte in buf.iter_mut() {\n                    *byte = *iter.next()?;\n                }\n                Some(Value::Int(i64::from_le_bytes(buf)))\n            }\n            3 => {\n                let mut buf = [0u8; std::mem::size_of::<f64>()];\n                for byte in buf.iter_mut() {\n                    *byte = *iter.next()?;\n                }\n                Some(Value::Float(f64::from_le_bytes(buf)))\n            }\n            4 => {\n                let mut buf = [0u8; std::mem::size_of::<u64>()];\n                for byte in buf.iter_mut() {\n                    *byte = *iter.next()?;\n                }\n                Some(Value::Timestamp(u64::from_le_bytes(buf)))\n            }\n            5 => Some(Value::Duration(leb128_read(iter)?)),\n            6 => {\n                let mut buf = vec![0u8; leb128_read(iter)? as usize];\n                for byte in buf.iter_mut() {\n                    *byte = *iter.next()?;\n                }\n                Some(Value::Bytes(buf))\n            }\n            7 => Some(Value::Bool(true)),\n            8 => Some(Value::Bool(false)),\n            9 => {\n                let mut buf = [0u8; 4];\n                for byte in buf.iter_mut() {\n                    *byte = *iter.next()?;\n                }\n                Some(Value::Ipv4(Ipv4Addr::from(buf)))\n            }\n            10 => {\n                let mut buf = [0u8; 16];\n                for byte in buf.iter_mut() {\n                    *byte = *iter.next()?;\n                }\n                Some(Value::Ipv6(Ipv6Addr::from(buf)))\n            }\n            11 => {\n                let code = EventType::from_code(leb128_read(iter)?)?;\n                let keys_len = leb128_read(iter)?;\n                let mut keys = Vec::with_capacity(keys_len as usize);\n                for _ in 0..keys_len {\n                    let key = Key::from_code(leb128_read(iter)?)?;\n                    let value = Value::deserialize(iter)?;\n                    keys.push((key, value));\n                }\n                Some(Value::Event(Error(\n                    Event::with_keys(code, keys).into_boxed(),\n                )))\n            }\n            12 => {\n                let len = leb128_read(iter)?;\n                let mut values = Vec::with_capacity(len as usize);\n                for _ in 0..len {\n                    values.push(Value::deserialize(iter)?);\n                }\n                Some(Value::Array(values))\n            }\n            13 => Some(Value::None),\n            _ => None,\n        }\n    }\n}\n\nfn leb128_write(buf: &mut Vec<u8>, mut value: u64) {\n    loop {\n        if value < 0x80 {\n            buf.push(value as u8);\n            break;\n        } else {\n            buf.push(((value & 0x7f) | 0x80) as u8);\n            value >>= 7;\n        }\n    }\n}\n\nfn leb128_read<'x>(iter: &mut impl Iterator<Item = &'x u8>) -> Option<u64> {\n    let mut result = 0;\n\n    for shift in [0, 7, 14, 21, 28, 35, 42, 49, 56, 63] {\n        let byte = iter.next()?;\n\n        if (byte & 0x80) == 0 {\n            result |= (*byte as u64) << shift;\n            return Some(result);\n        } else {\n            result |= ((byte & 0x7F) as u64) << shift;\n        }\n    }\n\n    None\n}\n\nimpl EventType {\n    pub fn code(&self) -> u64 {\n        match self {\n            EventType::Acme(AcmeEvent::AuthCompleted) => 0,\n            EventType::Acme(AcmeEvent::AuthError) => 1,\n            EventType::Acme(AcmeEvent::AuthPending) => 2,\n            EventType::Acme(AcmeEvent::AuthStart) => 3,\n            EventType::Acme(AcmeEvent::AuthTooManyAttempts) => 4,\n            EventType::Acme(AcmeEvent::AuthValid) => 5,\n            EventType::Acme(AcmeEvent::ClientMissingSni) => 6,\n            EventType::Acme(AcmeEvent::ClientSuppliedSni) => 7,\n            EventType::Acme(AcmeEvent::DnsRecordCreated) => 8,\n            EventType::Acme(AcmeEvent::DnsRecordCreationFailed) => 9,\n            EventType::Acme(AcmeEvent::DnsRecordDeletionFailed) => 10,\n            EventType::Acme(AcmeEvent::DnsRecordLookupFailed) => 11,\n            EventType::Acme(AcmeEvent::DnsRecordNotPropagated) => 12,\n            EventType::Acme(AcmeEvent::DnsRecordPropagated) => 13,\n            EventType::Acme(AcmeEvent::DnsRecordPropagationTimeout) => 14,\n            EventType::Acme(AcmeEvent::Error) => 15,\n            EventType::Acme(AcmeEvent::OrderCompleted) => 16,\n            EventType::Acme(AcmeEvent::OrderInvalid) => 17,\n            EventType::Acme(AcmeEvent::OrderProcessing) => 18,\n            EventType::Acme(AcmeEvent::OrderReady) => 19,\n            EventType::Acme(AcmeEvent::OrderStart) => 20,\n            EventType::Acme(AcmeEvent::OrderValid) => 21,\n            EventType::Acme(AcmeEvent::ProcessCert) => 22,\n            EventType::Acme(AcmeEvent::RenewBackoff) => 23,\n            EventType::Acme(AcmeEvent::TlsAlpnError) => 24,\n            EventType::Acme(AcmeEvent::TlsAlpnReceived) => 25,\n            EventType::Acme(AcmeEvent::TokenNotFound) => 26,\n            EventType::Arc(ArcEvent::BrokenChain) => 27,\n            EventType::Arc(ArcEvent::ChainTooLong) => 28,\n            EventType::Arc(ArcEvent::HasHeaderTag) => 29,\n            EventType::Arc(ArcEvent::InvalidCv) => 30,\n            EventType::Arc(ArcEvent::InvalidInstance) => 31,\n            EventType::Arc(ArcEvent::SealerNotFound) => 32,\n            EventType::Security(SecurityEvent::AuthenticationBan) => 33,\n            EventType::Auth(AuthEvent::Error) => 34,\n            EventType::Auth(AuthEvent::Failed) => 35,\n            EventType::Auth(AuthEvent::MissingTotp) => 36,\n            EventType::Auth(AuthEvent::Success) => 37,\n            EventType::Auth(AuthEvent::TooManyAttempts) => 38,\n            EventType::Cluster(ClusterEvent::SubscriberStart) => 39,\n            EventType::Cluster(ClusterEvent::SubscriberStop) => 40,\n            EventType::Cluster(ClusterEvent::SubscriberError) => 41,\n            EventType::Cluster(ClusterEvent::SubscriberDisconnected) => 42,\n            EventType::Cluster(ClusterEvent::PublisherStart) => 43,\n            EventType::Cluster(ClusterEvent::PublisherStop) => 44,\n            EventType::Cluster(ClusterEvent::PublisherError) => 45,\n            EventType::Cluster(ClusterEvent::MessageReceived) => 46,\n            EventType::Cluster(ClusterEvent::MessageSkipped) => 47,\n            EventType::Cluster(ClusterEvent::MessageInvalid) => 49,\n            EventType::Config(ConfigEvent::AlreadyUpToDate) => 53,\n            EventType::Config(ConfigEvent::BuildError) => 54,\n            EventType::Config(ConfigEvent::BuildWarning) => 55,\n            EventType::Config(ConfigEvent::DefaultApplied) => 56,\n            EventType::Config(ConfigEvent::FetchError) => 58,\n            EventType::Config(ConfigEvent::ImportExternal) => 59,\n            EventType::Config(ConfigEvent::MacroError) => 60,\n            EventType::Config(ConfigEvent::MissingSetting) => 61,\n            EventType::Config(ConfigEvent::ParseError) => 62,\n            EventType::Config(ConfigEvent::ParseWarning) => 63,\n            EventType::Config(ConfigEvent::UnusedSetting) => 64,\n            EventType::Config(ConfigEvent::WriteError) => 65,\n            EventType::Dane(DaneEvent::AuthenticationFailure) => 66,\n            EventType::Dane(DaneEvent::AuthenticationSuccess) => 67,\n            EventType::Dane(DaneEvent::CertificateParseError) => 68,\n            EventType::Dane(DaneEvent::NoCertificatesFound) => 69,\n            EventType::Dane(DaneEvent::TlsaRecordFetch) => 70,\n            EventType::Dane(DaneEvent::TlsaRecordFetchError) => 71,\n            EventType::Dane(DaneEvent::TlsaRecordInvalid) => 72,\n            EventType::Dane(DaneEvent::TlsaRecordMatch) => 73,\n            EventType::Dane(DaneEvent::TlsaRecordNotDnssecSigned) => 74,\n            EventType::Dane(DaneEvent::TlsaRecordNotFound) => 75,\n            EventType::Delivery(DeliveryEvent::AttemptEnd) => 76,\n            EventType::Delivery(DeliveryEvent::AttemptStart) => 77,\n            EventType::Delivery(DeliveryEvent::Auth) => 78,\n            EventType::Delivery(DeliveryEvent::AuthFailed) => 79,\n            EventType::Delivery(DeliveryEvent::Completed) => 80,\n            EventType::Delivery(DeliveryEvent::ConcurrencyLimitExceeded) => 81,\n            EventType::Delivery(DeliveryEvent::Connect) => 82,\n            EventType::Delivery(DeliveryEvent::ConnectError) => 83,\n            EventType::Delivery(DeliveryEvent::Delivered) => 84,\n            EventType::Delivery(DeliveryEvent::DomainDeliveryStart) => 85,\n            EventType::Delivery(DeliveryEvent::DoubleBounce) => 86,\n            EventType::Delivery(DeliveryEvent::DsnPermFail) => 87,\n            EventType::Delivery(DeliveryEvent::DsnSuccess) => 88,\n            EventType::Delivery(DeliveryEvent::DsnTempFail) => 89,\n            EventType::Delivery(DeliveryEvent::Ehlo) => 90,\n            EventType::Delivery(DeliveryEvent::EhloRejected) => 91,\n            EventType::Delivery(DeliveryEvent::Failed) => 92,\n            EventType::Delivery(DeliveryEvent::GreetingFailed) => 93,\n            EventType::Delivery(DeliveryEvent::ImplicitTlsError) => 94,\n            EventType::Delivery(DeliveryEvent::IpLookup) => 95,\n            EventType::Delivery(DeliveryEvent::IpLookupFailed) => 96,\n            EventType::Delivery(DeliveryEvent::MailFrom) => 97,\n            EventType::Delivery(DeliveryEvent::MailFromRejected) => 98,\n            EventType::Delivery(DeliveryEvent::MessageRejected) => 99,\n            EventType::Delivery(DeliveryEvent::MissingOutboundHostname) => 100,\n            EventType::Delivery(DeliveryEvent::MxLookup) => 101,\n            EventType::Delivery(DeliveryEvent::MxLookupFailed) => 102,\n            EventType::Delivery(DeliveryEvent::NullMx) => 103,\n            EventType::Delivery(DeliveryEvent::RateLimitExceeded) => 104,\n            EventType::Delivery(DeliveryEvent::RawInput) => 105,\n            EventType::Delivery(DeliveryEvent::RawOutput) => 106,\n            EventType::Delivery(DeliveryEvent::RcptTo) => 107,\n            EventType::Delivery(DeliveryEvent::RcptToFailed) => 108,\n            EventType::Delivery(DeliveryEvent::RcptToRejected) => 109,\n            EventType::Delivery(DeliveryEvent::StartTls) => 110,\n            EventType::Delivery(DeliveryEvent::StartTlsDisabled) => 111,\n            EventType::Delivery(DeliveryEvent::StartTlsError) => 112,\n            EventType::Delivery(DeliveryEvent::StartTlsUnavailable) => 113,\n            EventType::Dkim(DkimEvent::Fail) => 114,\n            EventType::Dkim(DkimEvent::FailedAuidMatch) => 115,\n            EventType::Dkim(DkimEvent::FailedBodyHashMatch) => 116,\n            EventType::Dkim(DkimEvent::FailedVerification) => 117,\n            EventType::Dkim(DkimEvent::IncompatibleAlgorithms) => 118,\n            EventType::Dkim(DkimEvent::Neutral) => 119,\n            EventType::Dkim(DkimEvent::None) => 120,\n            EventType::Dkim(DkimEvent::Pass) => 121,\n            EventType::Dkim(DkimEvent::PermError) => 122,\n            EventType::Dkim(DkimEvent::RevokedPublicKey) => 123,\n            EventType::Dkim(DkimEvent::SignatureExpired) => 124,\n            EventType::Dkim(DkimEvent::SignatureLength) => 125,\n            EventType::Dkim(DkimEvent::SignerNotFound) => 126,\n            EventType::Dkim(DkimEvent::TempError) => 127,\n            EventType::Dkim(DkimEvent::UnsupportedAlgorithm) => 128,\n            EventType::Dkim(DkimEvent::UnsupportedCanonicalization) => 129,\n            EventType::Dkim(DkimEvent::UnsupportedKeyType) => 130,\n            EventType::Dkim(DkimEvent::UnsupportedVersion) => 131,\n            EventType::Dmarc(DmarcEvent::Fail) => 132,\n            EventType::Dmarc(DmarcEvent::None) => 133,\n            EventType::Dmarc(DmarcEvent::Pass) => 134,\n            EventType::Dmarc(DmarcEvent::PermError) => 135,\n            EventType::Dmarc(DmarcEvent::TempError) => 136,\n            EventType::Eval(EvalEvent::DirectoryNotFound) => 137,\n            EventType::Eval(EvalEvent::Error) => 138,\n            EventType::Eval(EvalEvent::Result) => 139,\n            EventType::Eval(EvalEvent::StoreNotFound) => 140,\n            EventType::TaskQueue(TaskQueueEvent::BlobNotFound) => 141,\n            EventType::MessageIngest(MessageIngestEvent::FtsIndex) => 142,\n            EventType::Spam(SpamEvent::TrainSampleAdded) => 143,\n            EventType::TaskQueue(TaskQueueEvent::TaskLocked) => 144,\n            EventType::TaskQueue(TaskQueueEvent::MetadataNotFound) => 145,\n            EventType::Housekeeper(HousekeeperEvent::Run) => 146,\n            EventType::Housekeeper(HousekeeperEvent::Schedule) => 149,\n            EventType::Housekeeper(HousekeeperEvent::Start) => 150,\n            EventType::Housekeeper(HousekeeperEvent::Stop) => 151,\n            EventType::Http(HttpEvent::ConnectionEnd) => 152,\n            EventType::Http(HttpEvent::ConnectionStart) => 153,\n            EventType::Http(HttpEvent::Error) => 154,\n            EventType::Http(HttpEvent::RequestBody) => 155,\n            EventType::Http(HttpEvent::RequestUrl) => 156,\n            EventType::Http(HttpEvent::ResponseBody) => 157,\n            EventType::Http(HttpEvent::XForwardedMissing) => 158,\n            EventType::Imap(ImapEvent::Append) => 159,\n            EventType::Imap(ImapEvent::Capabilities) => 160,\n            EventType::Imap(ImapEvent::Close) => 161,\n            EventType::Imap(ImapEvent::ConnectionEnd) => 162,\n            EventType::Imap(ImapEvent::ConnectionStart) => 163,\n            EventType::Imap(ImapEvent::Copy) => 164,\n            EventType::Imap(ImapEvent::CreateMailbox) => 165,\n            EventType::Imap(ImapEvent::DeleteMailbox) => 166,\n            EventType::Imap(ImapEvent::Enable) => 167,\n            EventType::Imap(ImapEvent::Error) => 168,\n            EventType::Imap(ImapEvent::Expunge) => 169,\n            EventType::Imap(ImapEvent::Fetch) => 170,\n            EventType::Imap(ImapEvent::GetAcl) => 171,\n            EventType::Imap(ImapEvent::Id) => 172,\n            EventType::Imap(ImapEvent::IdleStart) => 173,\n            EventType::Imap(ImapEvent::IdleStop) => 174,\n            EventType::Imap(ImapEvent::List) => 175,\n            EventType::Imap(ImapEvent::ListRights) => 176,\n            EventType::Imap(ImapEvent::Logout) => 177,\n            EventType::Imap(ImapEvent::Lsub) => 178,\n            EventType::Imap(ImapEvent::Move) => 179,\n            EventType::Imap(ImapEvent::MyRights) => 180,\n            EventType::Imap(ImapEvent::Namespace) => 181,\n            EventType::Imap(ImapEvent::Noop) => 182,\n            EventType::Imap(ImapEvent::RawInput) => 183,\n            EventType::Imap(ImapEvent::RawOutput) => 184,\n            EventType::Imap(ImapEvent::RenameMailbox) => 185,\n            EventType::Imap(ImapEvent::Search) => 186,\n            EventType::Imap(ImapEvent::Select) => 187,\n            EventType::Imap(ImapEvent::SetAcl) => 188,\n            EventType::Imap(ImapEvent::Sort) => 189,\n            EventType::Imap(ImapEvent::Status) => 190,\n            EventType::Imap(ImapEvent::Store) => 191,\n            EventType::Imap(ImapEvent::Subscribe) => 192,\n            EventType::Imap(ImapEvent::Thread) => 193,\n            EventType::Imap(ImapEvent::Unsubscribe) => 194,\n            EventType::IncomingReport(IncomingReportEvent::AbuseReport) => 195,\n            EventType::IncomingReport(IncomingReportEvent::ArfParseFailed) => 196,\n            EventType::IncomingReport(IncomingReportEvent::AuthFailureReport) => 197,\n            EventType::IncomingReport(IncomingReportEvent::DecompressError) => 198,\n            EventType::IncomingReport(IncomingReportEvent::DmarcParseFailed) => 199,\n            EventType::IncomingReport(IncomingReportEvent::DmarcReport) => 200,\n            EventType::IncomingReport(IncomingReportEvent::DmarcReportWithWarnings) => 201,\n            EventType::IncomingReport(IncomingReportEvent::FraudReport) => 202,\n            EventType::IncomingReport(IncomingReportEvent::MessageParseFailed) => 203,\n            EventType::IncomingReport(IncomingReportEvent::NotSpamReport) => 204,\n            EventType::IncomingReport(IncomingReportEvent::OtherReport) => 205,\n            EventType::IncomingReport(IncomingReportEvent::TlsReport) => 206,\n            EventType::IncomingReport(IncomingReportEvent::TlsReportWithWarnings) => 207,\n            EventType::IncomingReport(IncomingReportEvent::TlsRpcParseFailed) => 208,\n            EventType::IncomingReport(IncomingReportEvent::VirusReport) => 209,\n            EventType::Iprev(IprevEvent::Fail) => 210,\n            EventType::Iprev(IprevEvent::None) => 211,\n            EventType::Iprev(IprevEvent::Pass) => 212,\n            EventType::Iprev(IprevEvent::PermError) => 213,\n            EventType::Iprev(IprevEvent::TempError) => 214,\n            EventType::Jmap(JmapEvent::AccountNotFound) => 215,\n            EventType::Jmap(JmapEvent::AccountNotSupportedByMethod) => 216,\n            EventType::Jmap(JmapEvent::AccountReadOnly) => 217,\n            EventType::Jmap(JmapEvent::AnchorNotFound) => 218,\n            EventType::Jmap(JmapEvent::CannotCalculateChanges) => 219,\n            EventType::Jmap(JmapEvent::Forbidden) => 220,\n            EventType::Jmap(JmapEvent::InvalidArguments) => 221,\n            EventType::Jmap(JmapEvent::InvalidResultReference) => 222,\n            EventType::Jmap(JmapEvent::MethodCall) => 223,\n            EventType::Jmap(JmapEvent::NotFound) => 224,\n            EventType::Jmap(JmapEvent::NotJson) => 225,\n            EventType::Jmap(JmapEvent::NotRequest) => 226,\n            EventType::Jmap(JmapEvent::RequestTooLarge) => 227,\n            EventType::Jmap(JmapEvent::StateMismatch) => 228,\n            EventType::Jmap(JmapEvent::UnknownCapability) => 229,\n            EventType::Jmap(JmapEvent::UnknownDataType) => 230,\n            EventType::Jmap(JmapEvent::UnknownMethod) => 231,\n            EventType::Jmap(JmapEvent::UnsupportedFilter) => 232,\n            EventType::Jmap(JmapEvent::UnsupportedSort) => 233,\n            EventType::Jmap(JmapEvent::WebsocketError) => 234,\n            EventType::Jmap(JmapEvent::WebsocketStart) => 235,\n            EventType::Jmap(JmapEvent::WebsocketStop) => 236,\n            EventType::Limit(LimitEvent::BlobQuota) => 237,\n            EventType::Limit(LimitEvent::CallsIn) => 238,\n            EventType::Limit(LimitEvent::ConcurrentConnection) => 239,\n            EventType::Limit(LimitEvent::ConcurrentRequest) => 240,\n            EventType::Limit(LimitEvent::ConcurrentUpload) => 241,\n            EventType::Limit(LimitEvent::Quota) => 242,\n            EventType::Limit(LimitEvent::SizeRequest) => 243,\n            EventType::Limit(LimitEvent::SizeUpload) => 244,\n            EventType::Limit(LimitEvent::TooManyRequests) => 245,\n            EventType::MailAuth(MailAuthEvent::Base64) => 246,\n            EventType::MailAuth(MailAuthEvent::Crypto) => 247,\n            EventType::MailAuth(MailAuthEvent::DnsError) => 248,\n            EventType::MailAuth(MailAuthEvent::DnsInvalidRecordType) => 249,\n            EventType::MailAuth(MailAuthEvent::DnsRecordNotFound) => 250,\n            EventType::MailAuth(MailAuthEvent::Io) => 251,\n            EventType::MailAuth(MailAuthEvent::MissingParameters) => 252,\n            EventType::MailAuth(MailAuthEvent::NoHeadersFound) => 253,\n            EventType::MailAuth(MailAuthEvent::ParseError) => 254,\n            EventType::MailAuth(MailAuthEvent::PolicyNotAligned) => 255,\n            EventType::ManageSieve(ManageSieveEvent::Capabilities) => 256,\n            EventType::ManageSieve(ManageSieveEvent::CheckScript) => 257,\n            EventType::ManageSieve(ManageSieveEvent::ConnectionEnd) => 258,\n            EventType::ManageSieve(ManageSieveEvent::ConnectionStart) => 259,\n            EventType::ManageSieve(ManageSieveEvent::CreateScript) => 260,\n            EventType::ManageSieve(ManageSieveEvent::DeleteScript) => 261,\n            EventType::ManageSieve(ManageSieveEvent::Error) => 262,\n            EventType::ManageSieve(ManageSieveEvent::GetScript) => 263,\n            EventType::ManageSieve(ManageSieveEvent::HaveSpace) => 264,\n            EventType::ManageSieve(ManageSieveEvent::ListScripts) => 265,\n            EventType::ManageSieve(ManageSieveEvent::Logout) => 266,\n            EventType::ManageSieve(ManageSieveEvent::Noop) => 267,\n            EventType::ManageSieve(ManageSieveEvent::RawInput) => 268,\n            EventType::ManageSieve(ManageSieveEvent::RawOutput) => 269,\n            EventType::ManageSieve(ManageSieveEvent::RenameScript) => 270,\n            EventType::ManageSieve(ManageSieveEvent::SetActive) => 271,\n            EventType::ManageSieve(ManageSieveEvent::StartTls) => 272,\n            EventType::ManageSieve(ManageSieveEvent::Unauthenticate) => 273,\n            EventType::ManageSieve(ManageSieveEvent::UpdateScript) => 274,\n            EventType::Manage(ManageEvent::AlreadyExists) => 275,\n            EventType::Manage(ManageEvent::AssertFailed) => 276,\n            EventType::Manage(ManageEvent::Error) => 277,\n            EventType::Manage(ManageEvent::MissingParameter) => 278,\n            EventType::Manage(ManageEvent::NotFound) => 279,\n            EventType::Manage(ManageEvent::NotSupported) => 280,\n            EventType::MessageIngest(MessageIngestEvent::Duplicate) => 281,\n            EventType::MessageIngest(MessageIngestEvent::Error) => 282,\n            EventType::MessageIngest(MessageIngestEvent::Ham) => 283,\n            EventType::MessageIngest(MessageIngestEvent::ImapAppend) => 284,\n            EventType::MessageIngest(MessageIngestEvent::JmapAppend) => 285,\n            EventType::MessageIngest(MessageIngestEvent::Spam) => 286,\n            EventType::Milter(MilterEvent::ActionAccept) => 287,\n            EventType::Milter(MilterEvent::ActionConnectionFailure) => 288,\n            EventType::Milter(MilterEvent::ActionDiscard) => 289,\n            EventType::Milter(MilterEvent::ActionReject) => 290,\n            EventType::Milter(MilterEvent::ActionReplyCode) => 291,\n            EventType::Milter(MilterEvent::ActionShutdown) => 292,\n            EventType::Milter(MilterEvent::ActionTempFail) => 293,\n            EventType::Milter(MilterEvent::Disconnected) => 294,\n            EventType::Milter(MilterEvent::FrameInvalid) => 295,\n            EventType::Milter(MilterEvent::FrameTooLarge) => 296,\n            EventType::Milter(MilterEvent::IoError) => 297,\n            EventType::Milter(MilterEvent::ParseError) => 298,\n            EventType::Milter(MilterEvent::Read) => 299,\n            EventType::Milter(MilterEvent::Timeout) => 300,\n            EventType::Milter(MilterEvent::TlsInvalidName) => 301,\n            EventType::Milter(MilterEvent::UnexpectedResponse) => 302,\n            EventType::Milter(MilterEvent::Write) => 303,\n            EventType::MtaHook(MtaHookEvent::ActionAccept) => 304,\n            EventType::MtaHook(MtaHookEvent::ActionDiscard) => 305,\n            EventType::MtaHook(MtaHookEvent::ActionQuarantine) => 306,\n            EventType::MtaHook(MtaHookEvent::ActionReject) => 307,\n            EventType::MtaHook(MtaHookEvent::Error) => 308,\n            EventType::MtaSts(MtaStsEvent::Authorized) => 309,\n            EventType::MtaSts(MtaStsEvent::InvalidPolicy) => 310,\n            EventType::MtaSts(MtaStsEvent::NotAuthorized) => 311,\n            EventType::MtaSts(MtaStsEvent::PolicyFetch) => 312,\n            EventType::MtaSts(MtaStsEvent::PolicyFetchError) => 313,\n            EventType::MtaSts(MtaStsEvent::PolicyNotFound) => 314,\n            EventType::Network(NetworkEvent::AcceptError) => 315,\n            EventType::Network(NetworkEvent::BindError) => 316,\n            EventType::Network(NetworkEvent::Closed) => 317,\n            EventType::Security(SecurityEvent::IpBlocked) => 318,\n            EventType::Network(NetworkEvent::FlushError) => 319,\n            EventType::Network(NetworkEvent::ListenError) => 320,\n            EventType::Network(NetworkEvent::ListenStart) => 321,\n            EventType::Network(NetworkEvent::ListenStop) => 322,\n            EventType::Network(NetworkEvent::ProxyError) => 323,\n            EventType::Network(NetworkEvent::ReadError) => 324,\n            EventType::Network(NetworkEvent::SetOptError) => 325,\n            EventType::Network(NetworkEvent::SplitError) => 326,\n            EventType::Network(NetworkEvent::Timeout) => 327,\n            EventType::Network(NetworkEvent::WriteError) => 328,\n            EventType::OutgoingReport(OutgoingReportEvent::DkimRateLimited) => 329,\n            EventType::OutgoingReport(OutgoingReportEvent::DkimReport) => 330,\n            EventType::OutgoingReport(OutgoingReportEvent::DmarcAggregateReport) => 331,\n            EventType::OutgoingReport(OutgoingReportEvent::DmarcRateLimited) => 332,\n            EventType::OutgoingReport(OutgoingReportEvent::DmarcReport) => 333,\n            EventType::OutgoingReport(OutgoingReportEvent::HttpSubmission) => 334,\n            EventType::OutgoingReport(OutgoingReportEvent::Locked) => 337,\n            EventType::OutgoingReport(OutgoingReportEvent::NoRecipientsFound) => 338,\n            EventType::OutgoingReport(OutgoingReportEvent::NotFound) => 339,\n            EventType::OutgoingReport(OutgoingReportEvent::ReportingAddressValidationError) => 340,\n            EventType::OutgoingReport(OutgoingReportEvent::SpfRateLimited) => 341,\n            EventType::OutgoingReport(OutgoingReportEvent::SpfReport) => 342,\n            EventType::OutgoingReport(OutgoingReportEvent::SubmissionError) => 343,\n            EventType::OutgoingReport(OutgoingReportEvent::TlsAggregate) => 344,\n            EventType::OutgoingReport(OutgoingReportEvent::UnauthorizedReportingAddress) => 345,\n            EventType::Pop3(Pop3Event::Capabilities) => 346,\n            EventType::Pop3(Pop3Event::ConnectionEnd) => 347,\n            EventType::Pop3(Pop3Event::ConnectionStart) => 348,\n            EventType::Pop3(Pop3Event::Delete) => 349,\n            EventType::Pop3(Pop3Event::Error) => 350,\n            EventType::Pop3(Pop3Event::Fetch) => 351,\n            EventType::Pop3(Pop3Event::List) => 352,\n            EventType::Pop3(Pop3Event::ListMessage) => 353,\n            EventType::Pop3(Pop3Event::Noop) => 354,\n            EventType::Pop3(Pop3Event::Quit) => 355,\n            EventType::Pop3(Pop3Event::RawInput) => 356,\n            EventType::Pop3(Pop3Event::RawOutput) => 357,\n            EventType::Pop3(Pop3Event::Reset) => 358,\n            EventType::Pop3(Pop3Event::StartTls) => 359,\n            EventType::Pop3(Pop3Event::Stat) => 360,\n            EventType::Pop3(Pop3Event::Uidl) => 361,\n            EventType::Pop3(Pop3Event::UidlMessage) => 362,\n            EventType::Pop3(Pop3Event::Utf8) => 363,\n            EventType::Purge(PurgeEvent::AutoExpunge) => 364,\n            EventType::Purge(PurgeEvent::Error) => 365,\n            EventType::Purge(PurgeEvent::Finished) => 366,\n            EventType::Purge(PurgeEvent::InProgress) => 367,\n            EventType::Purge(PurgeEvent::Running) => 368,\n            EventType::Purge(PurgeEvent::Started) => 369,\n            EventType::Purge(PurgeEvent::BlobCleanup) => 370,\n            EventType::PushSubscription(PushSubscriptionEvent::Error) => 371,\n            EventType::PushSubscription(PushSubscriptionEvent::NotFound) => 372,\n            EventType::PushSubscription(PushSubscriptionEvent::Success) => 373,\n            EventType::Queue(QueueEvent::BlobNotFound) => 374,\n            EventType::Queue(QueueEvent::ConcurrencyLimitExceeded) => 375,\n            EventType::Queue(QueueEvent::Locked) => 377,\n            EventType::Queue(QueueEvent::QueueAutogenerated) => 378,\n            EventType::Queue(QueueEvent::QueueDsn) => 379,\n            EventType::Queue(QueueEvent::QueueMessage) => 380,\n            EventType::Queue(QueueEvent::QueueMessageAuthenticated) => 381,\n            EventType::Queue(QueueEvent::QueueReport) => 382,\n            EventType::Queue(QueueEvent::QuotaExceeded) => 383,\n            EventType::Queue(QueueEvent::RateLimitExceeded) => 384,\n            EventType::Queue(QueueEvent::Rescheduled) => 385,\n            EventType::Resource(ResourceEvent::BadParameters) => 386,\n            EventType::Resource(ResourceEvent::DownloadExternal) => 387,\n            EventType::Resource(ResourceEvent::Error) => 388,\n            EventType::Resource(ResourceEvent::NotFound) => 389,\n            EventType::Resource(ResourceEvent::WebadminUnpacked) => 390,\n            EventType::Server(ServerEvent::Licensing) => 391,\n            EventType::Server(ServerEvent::Shutdown) => 392,\n            EventType::Server(ServerEvent::Startup) => 393,\n            EventType::Server(ServerEvent::StartupError) => 394,\n            EventType::Server(ServerEvent::ThreadError) => 395,\n            EventType::Sieve(SieveEvent::ActionAccept) => 396,\n            EventType::Sieve(SieveEvent::ActionAcceptReplace) => 397,\n            EventType::Sieve(SieveEvent::ActionDiscard) => 398,\n            EventType::Sieve(SieveEvent::ActionReject) => 399,\n            EventType::Sieve(SieveEvent::ListNotFound) => 400,\n            EventType::Sieve(SieveEvent::MessageTooLarge) => 401,\n            EventType::Sieve(SieveEvent::NotSupported) => 402,\n            EventType::Sieve(SieveEvent::QuotaExceeded) => 403,\n            EventType::Sieve(SieveEvent::RuntimeError) => 404,\n            EventType::Sieve(SieveEvent::ScriptNotFound) => 405,\n            EventType::Sieve(SieveEvent::SendMessage) => 406,\n            EventType::Sieve(SieveEvent::UnexpectedError) => 407,\n            EventType::Smtp(SmtpEvent::AlreadyAuthenticated) => 408,\n            EventType::Smtp(SmtpEvent::ArcFail) => 409,\n            EventType::Smtp(SmtpEvent::ArcPass) => 410,\n            EventType::Smtp(SmtpEvent::AuthExchangeTooLong) => 411,\n            EventType::Smtp(SmtpEvent::AuthMechanismNotSupported) => 412,\n            EventType::Smtp(SmtpEvent::AuthNotAllowed) => 413,\n            EventType::Smtp(SmtpEvent::CommandNotImplemented) => 414,\n            EventType::Smtp(SmtpEvent::ConcurrencyLimitExceeded) => 415,\n            EventType::Smtp(SmtpEvent::ConnectionEnd) => 416,\n            EventType::Smtp(SmtpEvent::ConnectionStart) => 417,\n            EventType::Smtp(SmtpEvent::DeliverByDisabled) => 418,\n            EventType::Smtp(SmtpEvent::DeliverByInvalid) => 419,\n            EventType::Smtp(SmtpEvent::DidNotSayEhlo) => 420,\n            EventType::Smtp(SmtpEvent::DkimFail) => 421,\n            EventType::Smtp(SmtpEvent::DkimPass) => 422,\n            EventType::Smtp(SmtpEvent::DmarcFail) => 423,\n            EventType::Smtp(SmtpEvent::DmarcPass) => 424,\n            EventType::Smtp(SmtpEvent::DsnDisabled) => 425,\n            EventType::Smtp(SmtpEvent::Ehlo) => 426,\n            EventType::Smtp(SmtpEvent::EhloExpected) => 427,\n            EventType::Smtp(SmtpEvent::Error) => 428,\n            EventType::Smtp(SmtpEvent::Expn) => 429,\n            EventType::Smtp(SmtpEvent::ExpnDisabled) => 430,\n            EventType::Smtp(SmtpEvent::ExpnNotFound) => 431,\n            EventType::Smtp(SmtpEvent::FutureReleaseDisabled) => 432,\n            EventType::Smtp(SmtpEvent::FutureReleaseInvalid) => 433,\n            EventType::Smtp(SmtpEvent::Help) => 434,\n            EventType::Smtp(SmtpEvent::InvalidCommand) => 435,\n            EventType::Smtp(SmtpEvent::InvalidEhlo) => 436,\n            EventType::Smtp(SmtpEvent::InvalidParameter) => 437,\n            EventType::Smtp(SmtpEvent::InvalidRecipientAddress) => 438,\n            EventType::Smtp(SmtpEvent::InvalidSenderAddress) => 439,\n            EventType::Smtp(SmtpEvent::IprevFail) => 440,\n            EventType::Smtp(SmtpEvent::IprevPass) => 441,\n            EventType::Smtp(SmtpEvent::LhloExpected) => 442,\n            EventType::Smtp(SmtpEvent::LoopDetected) => 443,\n            EventType::Smtp(SmtpEvent::MailFrom) => 444,\n            EventType::Smtp(SmtpEvent::MailFromMissing) => 445,\n            EventType::Smtp(SmtpEvent::MailFromRewritten) => 446,\n            EventType::Smtp(SmtpEvent::MailFromUnauthenticated) => 447,\n            EventType::Smtp(SmtpEvent::MailFromUnauthorized) => 448,\n            EventType::Smtp(SmtpEvent::MailboxDoesNotExist) => 449,\n            EventType::Smtp(SmtpEvent::MessageParseFailed) => 450,\n            EventType::Smtp(SmtpEvent::MessageTooLarge) => 451,\n            EventType::Smtp(SmtpEvent::MissingAuthDirectory) => 452,\n            EventType::Smtp(SmtpEvent::MissingLocalHostname) => 453,\n            EventType::Smtp(SmtpEvent::MtPriorityDisabled) => 454,\n            EventType::Smtp(SmtpEvent::MtPriorityInvalid) => 455,\n            EventType::Smtp(SmtpEvent::MultipleMailFrom) => 456,\n            EventType::Smtp(SmtpEvent::Noop) => 457,\n            EventType::Smtp(SmtpEvent::Quit) => 460,\n            EventType::Smtp(SmtpEvent::RateLimitExceeded) => 461,\n            EventType::Smtp(SmtpEvent::RawInput) => 462,\n            EventType::Smtp(SmtpEvent::RawOutput) => 463,\n            EventType::Smtp(SmtpEvent::RcptTo) => 464,\n            EventType::Smtp(SmtpEvent::RcptToDuplicate) => 465,\n            EventType::Smtp(SmtpEvent::RcptToMissing) => 466,\n            EventType::Smtp(SmtpEvent::RcptToRewritten) => 467,\n            EventType::Smtp(SmtpEvent::RelayNotAllowed) => 468,\n            EventType::Smtp(SmtpEvent::IdNotFound) => 469,\n            EventType::Smtp(SmtpEvent::RequestTooLarge) => 470,\n            EventType::Smtp(SmtpEvent::RequireTlsDisabled) => 471,\n            EventType::Smtp(SmtpEvent::Rset) => 472,\n            EventType::Smtp(SmtpEvent::SpfEhloFail) => 473,\n            EventType::Smtp(SmtpEvent::SpfEhloPass) => 474,\n            EventType::Smtp(SmtpEvent::SpfFromFail) => 475,\n            EventType::Smtp(SmtpEvent::SpfFromPass) => 476,\n            EventType::Smtp(SmtpEvent::StartTls) => 477,\n            EventType::Smtp(SmtpEvent::StartTlsAlready) => 478,\n            EventType::Smtp(SmtpEvent::StartTlsUnavailable) => 479,\n            EventType::Smtp(SmtpEvent::SyntaxError) => 480,\n            EventType::Smtp(SmtpEvent::TimeLimitExceeded) => 481,\n            EventType::Smtp(SmtpEvent::TooManyInvalidRcpt) => 482,\n            EventType::Smtp(SmtpEvent::TooManyMessages) => 483,\n            EventType::Smtp(SmtpEvent::TooManyRecipients) => 484,\n            EventType::Smtp(SmtpEvent::TransferLimitExceeded) => 485,\n            EventType::Smtp(SmtpEvent::UnsupportedParameter) => 486,\n            EventType::Smtp(SmtpEvent::Vrfy) => 487,\n            EventType::Smtp(SmtpEvent::VrfyDisabled) => 488,\n            EventType::Smtp(SmtpEvent::VrfyNotFound) => 489,\n            EventType::Spam(SpamEvent::Classify) => 490,\n            EventType::Spam(SpamEvent::TrainSampleNotFound) => 491,\n            EventType::Store(StoreEvent::HttpStoreFetch) => 492,\n            EventType::Store(StoreEvent::HttpStoreError) => 493,\n            EventType::Spam(SpamEvent::PyzorError) => 494,\n            EventType::Spam(SpamEvent::TrainCompleted) => 495,\n            EventType::Spam(SpamEvent::ModelNotReady) => 496,\n            EventType::Spam(SpamEvent::ModelNotFound) => 497,\n            EventType::Spf(SpfEvent::Fail) => 498,\n            EventType::Spf(SpfEvent::Neutral) => 499,\n            EventType::Spf(SpfEvent::None) => 500,\n            EventType::Spf(SpfEvent::Pass) => 501,\n            EventType::Spf(SpfEvent::PermError) => 502,\n            EventType::Spf(SpfEvent::SoftFail) => 503,\n            EventType::Spf(SpfEvent::TempError) => 504,\n            EventType::Store(StoreEvent::AssertValueFailed) => 505,\n            EventType::Store(StoreEvent::BlobDelete) => 506,\n            EventType::Store(StoreEvent::BlobMissingMarker) => 507,\n            EventType::Store(StoreEvent::BlobRead) => 508,\n            EventType::Store(StoreEvent::BlobWrite) => 509,\n            EventType::Store(StoreEvent::CryptoError) => 510,\n            EventType::Store(StoreEvent::DataCorruption) => 511,\n            EventType::Store(StoreEvent::DataIterate) => 512,\n            EventType::Store(StoreEvent::DataWrite) => 513,\n            EventType::Store(StoreEvent::DecompressError) => 514,\n            EventType::Store(StoreEvent::DeserializeError) => 515,\n            EventType::Store(StoreEvent::ElasticsearchError) => 516,\n            EventType::Store(StoreEvent::FilesystemError) => 517,\n            EventType::Store(StoreEvent::FoundationdbError) => 518,\n            EventType::Store(StoreEvent::LdapWarning) => 519,\n            EventType::Store(StoreEvent::LdapError) => 520,\n            EventType::Store(StoreEvent::LdapQuery) => 521,\n            EventType::Store(StoreEvent::MysqlError) => 522,\n            EventType::Store(StoreEvent::NotConfigured) => 523,\n            EventType::Store(StoreEvent::NotFound) => 524,\n            EventType::Store(StoreEvent::NotSupported) => 525,\n            EventType::Store(StoreEvent::PoolError) => 526,\n            EventType::Store(StoreEvent::PostgresqlError) => 527,\n            EventType::Store(StoreEvent::RedisError) => 528,\n            EventType::Store(StoreEvent::RocksdbError) => 529,\n            EventType::Store(StoreEvent::S3Error) => 530,\n            EventType::Store(StoreEvent::SqlQuery) => 531,\n            EventType::Store(StoreEvent::SqliteError) => 532,\n            EventType::Store(StoreEvent::UnexpectedError) => 533,\n            EventType::Telemetry(TelemetryEvent::JournalError) => 534,\n            EventType::Telemetry(TelemetryEvent::LogError) => 535,\n            EventType::Telemetry(TelemetryEvent::OtelExporterError) => 536,\n            EventType::Telemetry(TelemetryEvent::OtelMetricsExporterError) => 537,\n            EventType::Telemetry(TelemetryEvent::PrometheusExporterError) => 538,\n            EventType::Telemetry(TelemetryEvent::WebhookError) => 539,\n            EventType::TlsRpt(TlsRptEvent::RecordFetch) => 540,\n            EventType::TlsRpt(TlsRptEvent::RecordFetchError) => 541,\n            EventType::Tls(TlsEvent::CertificateNotFound) => 542,\n            EventType::Tls(TlsEvent::Handshake) => 543,\n            EventType::Tls(TlsEvent::HandshakeError) => 544,\n            EventType::Tls(TlsEvent::MultipleCertificatesAvailable) => 545,\n            EventType::Tls(TlsEvent::NoCertificatesAvailable) => 546,\n            EventType::Tls(TlsEvent::NotConfigured) => 547,\n            EventType::Telemetry(TelemetryEvent::Alert) => 548,\n            EventType::Security(SecurityEvent::AbuseBan) => 549,\n            EventType::Security(SecurityEvent::LoiterBan) => 550,\n            EventType::Smtp(SmtpEvent::MailFromNotAllowed) => 551,\n            EventType::Security(SecurityEvent::Unauthorized) => 552,\n            EventType::Limit(LimitEvent::TenantQuota) => 553,\n            EventType::Auth(AuthEvent::TokenExpired) => 554,\n            EventType::Auth(AuthEvent::ClientRegistration) => 555,\n            EventType::Ai(AiEvent::LlmResponse) => 556,\n            EventType::Ai(AiEvent::ApiError) => 557,\n            EventType::Security(SecurityEvent::ScanBan) => 558,\n            EventType::Store(StoreEvent::AzureError) => 559,\n            EventType::TlsRpt(TlsRptEvent::RecordNotFound) => 560,\n            EventType::Smtp(SmtpEvent::RcptToGreylisted) => 561,\n            EventType::Spam(SpamEvent::Dnsbl) => 562,\n            EventType::Spam(SpamEvent::DnsblError) => 563,\n            EventType::Spam(SpamEvent::Pyzor) => 564,\n            EventType::Queue(QueueEvent::BackPressure) => 48,\n            EventType::Imap(ImapEvent::GetQuota) => 57,\n            EventType::WebDav(WebDavEvent::Propfind) => 147,\n            EventType::WebDav(WebDavEvent::Proppatch) => 148,\n            EventType::WebDav(WebDavEvent::Get) => 335,\n            EventType::WebDav(WebDavEvent::Report) => 336,\n            EventType::WebDav(WebDavEvent::Mkcol) => 376,\n            EventType::WebDav(WebDavEvent::Delete) => 458,\n            EventType::WebDav(WebDavEvent::Put) => 459,\n            EventType::WebDav(WebDavEvent::Post) => 565,\n            EventType::WebDav(WebDavEvent::Patch) => 566,\n            EventType::WebDav(WebDavEvent::Copy) => 567,\n            EventType::WebDav(WebDavEvent::Move) => 568,\n            EventType::WebDav(WebDavEvent::Lock) => 569,\n            EventType::WebDav(WebDavEvent::Unlock) => 570,\n            EventType::WebDav(WebDavEvent::Acl) => 571,\n            EventType::WebDav(WebDavEvent::Error) => 572,\n            EventType::WebDav(WebDavEvent::Options) => 573,\n            EventType::WebDav(WebDavEvent::Head) => 574,\n            EventType::WebDav(WebDavEvent::Mkcalendar) => 575,\n            EventType::Calendar(CalendarEvent::RuleExpansionError) => 576,\n            EventType::Store(StoreEvent::CacheMiss) => 50,\n            EventType::Store(StoreEvent::CacheHit) => 51,\n            EventType::Store(StoreEvent::CacheStale) => 52,\n            EventType::Store(StoreEvent::CacheUpdate) => 577,\n            EventType::TaskQueue(TaskQueueEvent::TaskAcquired) => 578,\n            EventType::Calendar(CalendarEvent::AlarmSent) => 579,\n            EventType::Calendar(CalendarEvent::AlarmSkipped) => 580,\n            EventType::Calendar(CalendarEvent::AlarmRecipientOverride) => 581,\n            EventType::Calendar(CalendarEvent::AlarmFailed) => 582,\n            EventType::Calendar(CalendarEvent::ItipMessageSent) => 583,\n            EventType::Calendar(CalendarEvent::ItipMessageReceived) => 584,\n            EventType::Calendar(CalendarEvent::ItipMessageError) => 585,\n            EventType::TaskQueue(TaskQueueEvent::TaskIgnored) => 586,\n            EventType::TaskQueue(TaskQueueEvent::TaskFailed) => 587,\n            EventType::Spam(SpamEvent::TrainStarted) => 588,\n            EventType::Spam(SpamEvent::ModelLoaded) => 589,\n            EventType::Store(StoreEvent::MeilisearchError) => 590,\n        }\n    }\n\n    pub fn from_code(code: u64) -> Option<Self> {\n        match code {\n            0 => Some(EventType::Acme(AcmeEvent::AuthCompleted)),\n            1 => Some(EventType::Acme(AcmeEvent::AuthError)),\n            2 => Some(EventType::Acme(AcmeEvent::AuthPending)),\n            3 => Some(EventType::Acme(AcmeEvent::AuthStart)),\n            4 => Some(EventType::Acme(AcmeEvent::AuthTooManyAttempts)),\n            5 => Some(EventType::Acme(AcmeEvent::AuthValid)),\n            6 => Some(EventType::Acme(AcmeEvent::ClientMissingSni)),\n            7 => Some(EventType::Acme(AcmeEvent::ClientSuppliedSni)),\n            8 => Some(EventType::Acme(AcmeEvent::DnsRecordCreated)),\n            9 => Some(EventType::Acme(AcmeEvent::DnsRecordCreationFailed)),\n            10 => Some(EventType::Acme(AcmeEvent::DnsRecordDeletionFailed)),\n            11 => Some(EventType::Acme(AcmeEvent::DnsRecordLookupFailed)),\n            12 => Some(EventType::Acme(AcmeEvent::DnsRecordNotPropagated)),\n            13 => Some(EventType::Acme(AcmeEvent::DnsRecordPropagated)),\n            14 => Some(EventType::Acme(AcmeEvent::DnsRecordPropagationTimeout)),\n            15 => Some(EventType::Acme(AcmeEvent::Error)),\n            16 => Some(EventType::Acme(AcmeEvent::OrderCompleted)),\n            17 => Some(EventType::Acme(AcmeEvent::OrderInvalid)),\n            18 => Some(EventType::Acme(AcmeEvent::OrderProcessing)),\n            19 => Some(EventType::Acme(AcmeEvent::OrderReady)),\n            20 => Some(EventType::Acme(AcmeEvent::OrderStart)),\n            21 => Some(EventType::Acme(AcmeEvent::OrderValid)),\n            22 => Some(EventType::Acme(AcmeEvent::ProcessCert)),\n            23 => Some(EventType::Acme(AcmeEvent::RenewBackoff)),\n            24 => Some(EventType::Acme(AcmeEvent::TlsAlpnError)),\n            25 => Some(EventType::Acme(AcmeEvent::TlsAlpnReceived)),\n            26 => Some(EventType::Acme(AcmeEvent::TokenNotFound)),\n            27 => Some(EventType::Arc(ArcEvent::BrokenChain)),\n            28 => Some(EventType::Arc(ArcEvent::ChainTooLong)),\n            29 => Some(EventType::Arc(ArcEvent::HasHeaderTag)),\n            30 => Some(EventType::Arc(ArcEvent::InvalidCv)),\n            31 => Some(EventType::Arc(ArcEvent::InvalidInstance)),\n            32 => Some(EventType::Arc(ArcEvent::SealerNotFound)),\n            33 => Some(EventType::Security(SecurityEvent::AuthenticationBan)),\n            34 => Some(EventType::Auth(AuthEvent::Error)),\n            35 => Some(EventType::Auth(AuthEvent::Failed)),\n            36 => Some(EventType::Auth(AuthEvent::MissingTotp)),\n            37 => Some(EventType::Auth(AuthEvent::Success)),\n            38 => Some(EventType::Auth(AuthEvent::TooManyAttempts)),\n            39 => Some(EventType::Cluster(ClusterEvent::SubscriberStart)),\n            40 => Some(EventType::Cluster(ClusterEvent::SubscriberStop)),\n            41 => Some(EventType::Cluster(ClusterEvent::SubscriberError)),\n            42 => Some(EventType::Cluster(ClusterEvent::SubscriberDisconnected)),\n            43 => Some(EventType::Cluster(ClusterEvent::PublisherStart)),\n            44 => Some(EventType::Cluster(ClusterEvent::PublisherStop)),\n            45 => Some(EventType::Cluster(ClusterEvent::PublisherError)),\n            46 => Some(EventType::Cluster(ClusterEvent::MessageReceived)),\n            47 => Some(EventType::Cluster(ClusterEvent::MessageSkipped)),\n            49 => Some(EventType::Cluster(ClusterEvent::MessageInvalid)),\n            53 => Some(EventType::Config(ConfigEvent::AlreadyUpToDate)),\n            54 => Some(EventType::Config(ConfigEvent::BuildError)),\n            55 => Some(EventType::Config(ConfigEvent::BuildWarning)),\n            56 => Some(EventType::Config(ConfigEvent::DefaultApplied)),\n            58 => Some(EventType::Config(ConfigEvent::FetchError)),\n            59 => Some(EventType::Config(ConfigEvent::ImportExternal)),\n            60 => Some(EventType::Config(ConfigEvent::MacroError)),\n            61 => Some(EventType::Config(ConfigEvent::MissingSetting)),\n            62 => Some(EventType::Config(ConfigEvent::ParseError)),\n            63 => Some(EventType::Config(ConfigEvent::ParseWarning)),\n            64 => Some(EventType::Config(ConfigEvent::UnusedSetting)),\n            65 => Some(EventType::Config(ConfigEvent::WriteError)),\n            66 => Some(EventType::Dane(DaneEvent::AuthenticationFailure)),\n            67 => Some(EventType::Dane(DaneEvent::AuthenticationSuccess)),\n            68 => Some(EventType::Dane(DaneEvent::CertificateParseError)),\n            69 => Some(EventType::Dane(DaneEvent::NoCertificatesFound)),\n            70 => Some(EventType::Dane(DaneEvent::TlsaRecordFetch)),\n            71 => Some(EventType::Dane(DaneEvent::TlsaRecordFetchError)),\n            72 => Some(EventType::Dane(DaneEvent::TlsaRecordInvalid)),\n            73 => Some(EventType::Dane(DaneEvent::TlsaRecordMatch)),\n            74 => Some(EventType::Dane(DaneEvent::TlsaRecordNotDnssecSigned)),\n            75 => Some(EventType::Dane(DaneEvent::TlsaRecordNotFound)),\n            76 => Some(EventType::Delivery(DeliveryEvent::AttemptEnd)),\n            77 => Some(EventType::Delivery(DeliveryEvent::AttemptStart)),\n            78 => Some(EventType::Delivery(DeliveryEvent::Auth)),\n            79 => Some(EventType::Delivery(DeliveryEvent::AuthFailed)),\n            80 => Some(EventType::Delivery(DeliveryEvent::Completed)),\n            81 => Some(EventType::Delivery(DeliveryEvent::ConcurrencyLimitExceeded)),\n            82 => Some(EventType::Delivery(DeliveryEvent::Connect)),\n            83 => Some(EventType::Delivery(DeliveryEvent::ConnectError)),\n            84 => Some(EventType::Delivery(DeliveryEvent::Delivered)),\n            85 => Some(EventType::Delivery(DeliveryEvent::DomainDeliveryStart)),\n            86 => Some(EventType::Delivery(DeliveryEvent::DoubleBounce)),\n            87 => Some(EventType::Delivery(DeliveryEvent::DsnPermFail)),\n            88 => Some(EventType::Delivery(DeliveryEvent::DsnSuccess)),\n            89 => Some(EventType::Delivery(DeliveryEvent::DsnTempFail)),\n            90 => Some(EventType::Delivery(DeliveryEvent::Ehlo)),\n            91 => Some(EventType::Delivery(DeliveryEvent::EhloRejected)),\n            92 => Some(EventType::Delivery(DeliveryEvent::Failed)),\n            93 => Some(EventType::Delivery(DeliveryEvent::GreetingFailed)),\n            94 => Some(EventType::Delivery(DeliveryEvent::ImplicitTlsError)),\n            95 => Some(EventType::Delivery(DeliveryEvent::IpLookup)),\n            96 => Some(EventType::Delivery(DeliveryEvent::IpLookupFailed)),\n            97 => Some(EventType::Delivery(DeliveryEvent::MailFrom)),\n            98 => Some(EventType::Delivery(DeliveryEvent::MailFromRejected)),\n            99 => Some(EventType::Delivery(DeliveryEvent::MessageRejected)),\n            100 => Some(EventType::Delivery(DeliveryEvent::MissingOutboundHostname)),\n            101 => Some(EventType::Delivery(DeliveryEvent::MxLookup)),\n            102 => Some(EventType::Delivery(DeliveryEvent::MxLookupFailed)),\n            103 => Some(EventType::Delivery(DeliveryEvent::NullMx)),\n            104 => Some(EventType::Delivery(DeliveryEvent::RateLimitExceeded)),\n            105 => Some(EventType::Delivery(DeliveryEvent::RawInput)),\n            106 => Some(EventType::Delivery(DeliveryEvent::RawOutput)),\n            107 => Some(EventType::Delivery(DeliveryEvent::RcptTo)),\n            108 => Some(EventType::Delivery(DeliveryEvent::RcptToFailed)),\n            109 => Some(EventType::Delivery(DeliveryEvent::RcptToRejected)),\n            110 => Some(EventType::Delivery(DeliveryEvent::StartTls)),\n            111 => Some(EventType::Delivery(DeliveryEvent::StartTlsDisabled)),\n            112 => Some(EventType::Delivery(DeliveryEvent::StartTlsError)),\n            113 => Some(EventType::Delivery(DeliveryEvent::StartTlsUnavailable)),\n            114 => Some(EventType::Dkim(DkimEvent::Fail)),\n            115 => Some(EventType::Dkim(DkimEvent::FailedAuidMatch)),\n            116 => Some(EventType::Dkim(DkimEvent::FailedBodyHashMatch)),\n            117 => Some(EventType::Dkim(DkimEvent::FailedVerification)),\n            118 => Some(EventType::Dkim(DkimEvent::IncompatibleAlgorithms)),\n            119 => Some(EventType::Dkim(DkimEvent::Neutral)),\n            120 => Some(EventType::Dkim(DkimEvent::None)),\n            121 => Some(EventType::Dkim(DkimEvent::Pass)),\n            122 => Some(EventType::Dkim(DkimEvent::PermError)),\n            123 => Some(EventType::Dkim(DkimEvent::RevokedPublicKey)),\n            124 => Some(EventType::Dkim(DkimEvent::SignatureExpired)),\n            125 => Some(EventType::Dkim(DkimEvent::SignatureLength)),\n            126 => Some(EventType::Dkim(DkimEvent::SignerNotFound)),\n            127 => Some(EventType::Dkim(DkimEvent::TempError)),\n            128 => Some(EventType::Dkim(DkimEvent::UnsupportedAlgorithm)),\n            129 => Some(EventType::Dkim(DkimEvent::UnsupportedCanonicalization)),\n            130 => Some(EventType::Dkim(DkimEvent::UnsupportedKeyType)),\n            131 => Some(EventType::Dkim(DkimEvent::UnsupportedVersion)),\n            132 => Some(EventType::Dmarc(DmarcEvent::Fail)),\n            133 => Some(EventType::Dmarc(DmarcEvent::None)),\n            134 => Some(EventType::Dmarc(DmarcEvent::Pass)),\n            135 => Some(EventType::Dmarc(DmarcEvent::PermError)),\n            136 => Some(EventType::Dmarc(DmarcEvent::TempError)),\n            137 => Some(EventType::Eval(EvalEvent::DirectoryNotFound)),\n            138 => Some(EventType::Eval(EvalEvent::Error)),\n            139 => Some(EventType::Eval(EvalEvent::Result)),\n            140 => Some(EventType::Eval(EvalEvent::StoreNotFound)),\n            141 => Some(EventType::TaskQueue(TaskQueueEvent::BlobNotFound)),\n            142 => Some(EventType::MessageIngest(MessageIngestEvent::FtsIndex)),\n            143 => Some(EventType::Spam(SpamEvent::TrainSampleAdded)),\n            144 => Some(EventType::TaskQueue(TaskQueueEvent::TaskLocked)),\n            145 => Some(EventType::TaskQueue(TaskQueueEvent::MetadataNotFound)),\n            146 => Some(EventType::Housekeeper(HousekeeperEvent::Run)),\n            149 => Some(EventType::Housekeeper(HousekeeperEvent::Schedule)),\n            150 => Some(EventType::Housekeeper(HousekeeperEvent::Start)),\n            151 => Some(EventType::Housekeeper(HousekeeperEvent::Stop)),\n            152 => Some(EventType::Http(HttpEvent::ConnectionEnd)),\n            153 => Some(EventType::Http(HttpEvent::ConnectionStart)),\n            154 => Some(EventType::Http(HttpEvent::Error)),\n            155 => Some(EventType::Http(HttpEvent::RequestBody)),\n            156 => Some(EventType::Http(HttpEvent::RequestUrl)),\n            157 => Some(EventType::Http(HttpEvent::ResponseBody)),\n            158 => Some(EventType::Http(HttpEvent::XForwardedMissing)),\n            159 => Some(EventType::Imap(ImapEvent::Append)),\n            160 => Some(EventType::Imap(ImapEvent::Capabilities)),\n            161 => Some(EventType::Imap(ImapEvent::Close)),\n            162 => Some(EventType::Imap(ImapEvent::ConnectionEnd)),\n            163 => Some(EventType::Imap(ImapEvent::ConnectionStart)),\n            164 => Some(EventType::Imap(ImapEvent::Copy)),\n            165 => Some(EventType::Imap(ImapEvent::CreateMailbox)),\n            166 => Some(EventType::Imap(ImapEvent::DeleteMailbox)),\n            167 => Some(EventType::Imap(ImapEvent::Enable)),\n            168 => Some(EventType::Imap(ImapEvent::Error)),\n            169 => Some(EventType::Imap(ImapEvent::Expunge)),\n            170 => Some(EventType::Imap(ImapEvent::Fetch)),\n            171 => Some(EventType::Imap(ImapEvent::GetAcl)),\n            172 => Some(EventType::Imap(ImapEvent::Id)),\n            173 => Some(EventType::Imap(ImapEvent::IdleStart)),\n            174 => Some(EventType::Imap(ImapEvent::IdleStop)),\n            175 => Some(EventType::Imap(ImapEvent::List)),\n            176 => Some(EventType::Imap(ImapEvent::ListRights)),\n            177 => Some(EventType::Imap(ImapEvent::Logout)),\n            178 => Some(EventType::Imap(ImapEvent::Lsub)),\n            179 => Some(EventType::Imap(ImapEvent::Move)),\n            180 => Some(EventType::Imap(ImapEvent::MyRights)),\n            181 => Some(EventType::Imap(ImapEvent::Namespace)),\n            182 => Some(EventType::Imap(ImapEvent::Noop)),\n            183 => Some(EventType::Imap(ImapEvent::RawInput)),\n            184 => Some(EventType::Imap(ImapEvent::RawOutput)),\n            185 => Some(EventType::Imap(ImapEvent::RenameMailbox)),\n            186 => Some(EventType::Imap(ImapEvent::Search)),\n            187 => Some(EventType::Imap(ImapEvent::Select)),\n            188 => Some(EventType::Imap(ImapEvent::SetAcl)),\n            189 => Some(EventType::Imap(ImapEvent::Sort)),\n            190 => Some(EventType::Imap(ImapEvent::Status)),\n            191 => Some(EventType::Imap(ImapEvent::Store)),\n            192 => Some(EventType::Imap(ImapEvent::Subscribe)),\n            193 => Some(EventType::Imap(ImapEvent::Thread)),\n            194 => Some(EventType::Imap(ImapEvent::Unsubscribe)),\n            195 => Some(EventType::IncomingReport(IncomingReportEvent::AbuseReport)),\n            196 => Some(EventType::IncomingReport(\n                IncomingReportEvent::ArfParseFailed,\n            )),\n            197 => Some(EventType::IncomingReport(\n                IncomingReportEvent::AuthFailureReport,\n            )),\n            198 => Some(EventType::IncomingReport(\n                IncomingReportEvent::DecompressError,\n            )),\n            199 => Some(EventType::IncomingReport(\n                IncomingReportEvent::DmarcParseFailed,\n            )),\n            200 => Some(EventType::IncomingReport(IncomingReportEvent::DmarcReport)),\n            201 => Some(EventType::IncomingReport(\n                IncomingReportEvent::DmarcReportWithWarnings,\n            )),\n            202 => Some(EventType::IncomingReport(IncomingReportEvent::FraudReport)),\n            203 => Some(EventType::IncomingReport(\n                IncomingReportEvent::MessageParseFailed,\n            )),\n            204 => Some(EventType::IncomingReport(\n                IncomingReportEvent::NotSpamReport,\n            )),\n            205 => Some(EventType::IncomingReport(IncomingReportEvent::OtherReport)),\n            206 => Some(EventType::IncomingReport(IncomingReportEvent::TlsReport)),\n            207 => Some(EventType::IncomingReport(\n                IncomingReportEvent::TlsReportWithWarnings,\n            )),\n            208 => Some(EventType::IncomingReport(\n                IncomingReportEvent::TlsRpcParseFailed,\n            )),\n            209 => Some(EventType::IncomingReport(IncomingReportEvent::VirusReport)),\n            210 => Some(EventType::Iprev(IprevEvent::Fail)),\n            211 => Some(EventType::Iprev(IprevEvent::None)),\n            212 => Some(EventType::Iprev(IprevEvent::Pass)),\n            213 => Some(EventType::Iprev(IprevEvent::PermError)),\n            214 => Some(EventType::Iprev(IprevEvent::TempError)),\n            215 => Some(EventType::Jmap(JmapEvent::AccountNotFound)),\n            216 => Some(EventType::Jmap(JmapEvent::AccountNotSupportedByMethod)),\n            217 => Some(EventType::Jmap(JmapEvent::AccountReadOnly)),\n            218 => Some(EventType::Jmap(JmapEvent::AnchorNotFound)),\n            219 => Some(EventType::Jmap(JmapEvent::CannotCalculateChanges)),\n            220 => Some(EventType::Jmap(JmapEvent::Forbidden)),\n            221 => Some(EventType::Jmap(JmapEvent::InvalidArguments)),\n            222 => Some(EventType::Jmap(JmapEvent::InvalidResultReference)),\n            223 => Some(EventType::Jmap(JmapEvent::MethodCall)),\n            224 => Some(EventType::Jmap(JmapEvent::NotFound)),\n            225 => Some(EventType::Jmap(JmapEvent::NotJson)),\n            226 => Some(EventType::Jmap(JmapEvent::NotRequest)),\n            227 => Some(EventType::Jmap(JmapEvent::RequestTooLarge)),\n            228 => Some(EventType::Jmap(JmapEvent::StateMismatch)),\n            229 => Some(EventType::Jmap(JmapEvent::UnknownCapability)),\n            230 => Some(EventType::Jmap(JmapEvent::UnknownDataType)),\n            231 => Some(EventType::Jmap(JmapEvent::UnknownMethod)),\n            232 => Some(EventType::Jmap(JmapEvent::UnsupportedFilter)),\n            233 => Some(EventType::Jmap(JmapEvent::UnsupportedSort)),\n            234 => Some(EventType::Jmap(JmapEvent::WebsocketError)),\n            235 => Some(EventType::Jmap(JmapEvent::WebsocketStart)),\n            236 => Some(EventType::Jmap(JmapEvent::WebsocketStop)),\n            237 => Some(EventType::Limit(LimitEvent::BlobQuota)),\n            238 => Some(EventType::Limit(LimitEvent::CallsIn)),\n            239 => Some(EventType::Limit(LimitEvent::ConcurrentConnection)),\n            240 => Some(EventType::Limit(LimitEvent::ConcurrentRequest)),\n            241 => Some(EventType::Limit(LimitEvent::ConcurrentUpload)),\n            242 => Some(EventType::Limit(LimitEvent::Quota)),\n            243 => Some(EventType::Limit(LimitEvent::SizeRequest)),\n            244 => Some(EventType::Limit(LimitEvent::SizeUpload)),\n            245 => Some(EventType::Limit(LimitEvent::TooManyRequests)),\n            246 => Some(EventType::MailAuth(MailAuthEvent::Base64)),\n            247 => Some(EventType::MailAuth(MailAuthEvent::Crypto)),\n            248 => Some(EventType::MailAuth(MailAuthEvent::DnsError)),\n            249 => Some(EventType::MailAuth(MailAuthEvent::DnsInvalidRecordType)),\n            250 => Some(EventType::MailAuth(MailAuthEvent::DnsRecordNotFound)),\n            251 => Some(EventType::MailAuth(MailAuthEvent::Io)),\n            252 => Some(EventType::MailAuth(MailAuthEvent::MissingParameters)),\n            253 => Some(EventType::MailAuth(MailAuthEvent::NoHeadersFound)),\n            254 => Some(EventType::MailAuth(MailAuthEvent::ParseError)),\n            255 => Some(EventType::MailAuth(MailAuthEvent::PolicyNotAligned)),\n            256 => Some(EventType::ManageSieve(ManageSieveEvent::Capabilities)),\n            257 => Some(EventType::ManageSieve(ManageSieveEvent::CheckScript)),\n            258 => Some(EventType::ManageSieve(ManageSieveEvent::ConnectionEnd)),\n            259 => Some(EventType::ManageSieve(ManageSieveEvent::ConnectionStart)),\n            260 => Some(EventType::ManageSieve(ManageSieveEvent::CreateScript)),\n            261 => Some(EventType::ManageSieve(ManageSieveEvent::DeleteScript)),\n            262 => Some(EventType::ManageSieve(ManageSieveEvent::Error)),\n            263 => Some(EventType::ManageSieve(ManageSieveEvent::GetScript)),\n            264 => Some(EventType::ManageSieve(ManageSieveEvent::HaveSpace)),\n            265 => Some(EventType::ManageSieve(ManageSieveEvent::ListScripts)),\n            266 => Some(EventType::ManageSieve(ManageSieveEvent::Logout)),\n            267 => Some(EventType::ManageSieve(ManageSieveEvent::Noop)),\n            268 => Some(EventType::ManageSieve(ManageSieveEvent::RawInput)),\n            269 => Some(EventType::ManageSieve(ManageSieveEvent::RawOutput)),\n            270 => Some(EventType::ManageSieve(ManageSieveEvent::RenameScript)),\n            271 => Some(EventType::ManageSieve(ManageSieveEvent::SetActive)),\n            272 => Some(EventType::ManageSieve(ManageSieveEvent::StartTls)),\n            273 => Some(EventType::ManageSieve(ManageSieveEvent::Unauthenticate)),\n            274 => Some(EventType::ManageSieve(ManageSieveEvent::UpdateScript)),\n            275 => Some(EventType::Manage(ManageEvent::AlreadyExists)),\n            276 => Some(EventType::Manage(ManageEvent::AssertFailed)),\n            277 => Some(EventType::Manage(ManageEvent::Error)),\n            278 => Some(EventType::Manage(ManageEvent::MissingParameter)),\n            279 => Some(EventType::Manage(ManageEvent::NotFound)),\n            280 => Some(EventType::Manage(ManageEvent::NotSupported)),\n            281 => Some(EventType::MessageIngest(MessageIngestEvent::Duplicate)),\n            282 => Some(EventType::MessageIngest(MessageIngestEvent::Error)),\n            283 => Some(EventType::MessageIngest(MessageIngestEvent::Ham)),\n            284 => Some(EventType::MessageIngest(MessageIngestEvent::ImapAppend)),\n            285 => Some(EventType::MessageIngest(MessageIngestEvent::JmapAppend)),\n            286 => Some(EventType::MessageIngest(MessageIngestEvent::Spam)),\n            287 => Some(EventType::Milter(MilterEvent::ActionAccept)),\n            288 => Some(EventType::Milter(MilterEvent::ActionConnectionFailure)),\n            289 => Some(EventType::Milter(MilterEvent::ActionDiscard)),\n            290 => Some(EventType::Milter(MilterEvent::ActionReject)),\n            291 => Some(EventType::Milter(MilterEvent::ActionReplyCode)),\n            292 => Some(EventType::Milter(MilterEvent::ActionShutdown)),\n            293 => Some(EventType::Milter(MilterEvent::ActionTempFail)),\n            294 => Some(EventType::Milter(MilterEvent::Disconnected)),\n            295 => Some(EventType::Milter(MilterEvent::FrameInvalid)),\n            296 => Some(EventType::Milter(MilterEvent::FrameTooLarge)),\n            297 => Some(EventType::Milter(MilterEvent::IoError)),\n            298 => Some(EventType::Milter(MilterEvent::ParseError)),\n            299 => Some(EventType::Milter(MilterEvent::Read)),\n            300 => Some(EventType::Milter(MilterEvent::Timeout)),\n            301 => Some(EventType::Milter(MilterEvent::TlsInvalidName)),\n            302 => Some(EventType::Milter(MilterEvent::UnexpectedResponse)),\n            303 => Some(EventType::Milter(MilterEvent::Write)),\n            304 => Some(EventType::MtaHook(MtaHookEvent::ActionAccept)),\n            305 => Some(EventType::MtaHook(MtaHookEvent::ActionDiscard)),\n            306 => Some(EventType::MtaHook(MtaHookEvent::ActionQuarantine)),\n            307 => Some(EventType::MtaHook(MtaHookEvent::ActionReject)),\n            308 => Some(EventType::MtaHook(MtaHookEvent::Error)),\n            309 => Some(EventType::MtaSts(MtaStsEvent::Authorized)),\n            310 => Some(EventType::MtaSts(MtaStsEvent::InvalidPolicy)),\n            311 => Some(EventType::MtaSts(MtaStsEvent::NotAuthorized)),\n            312 => Some(EventType::MtaSts(MtaStsEvent::PolicyFetch)),\n            313 => Some(EventType::MtaSts(MtaStsEvent::PolicyFetchError)),\n            314 => Some(EventType::MtaSts(MtaStsEvent::PolicyNotFound)),\n            315 => Some(EventType::Network(NetworkEvent::AcceptError)),\n            316 => Some(EventType::Network(NetworkEvent::BindError)),\n            317 => Some(EventType::Network(NetworkEvent::Closed)),\n            318 => Some(EventType::Security(SecurityEvent::IpBlocked)),\n            319 => Some(EventType::Network(NetworkEvent::FlushError)),\n            320 => Some(EventType::Network(NetworkEvent::ListenError)),\n            321 => Some(EventType::Network(NetworkEvent::ListenStart)),\n            322 => Some(EventType::Network(NetworkEvent::ListenStop)),\n            323 => Some(EventType::Network(NetworkEvent::ProxyError)),\n            324 => Some(EventType::Network(NetworkEvent::ReadError)),\n            325 => Some(EventType::Network(NetworkEvent::SetOptError)),\n            326 => Some(EventType::Network(NetworkEvent::SplitError)),\n            327 => Some(EventType::Network(NetworkEvent::Timeout)),\n            328 => Some(EventType::Network(NetworkEvent::WriteError)),\n            329 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::DkimRateLimited,\n            )),\n            330 => Some(EventType::OutgoingReport(OutgoingReportEvent::DkimReport)),\n            331 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::DmarcAggregateReport,\n            )),\n            332 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::DmarcRateLimited,\n            )),\n            333 => Some(EventType::OutgoingReport(OutgoingReportEvent::DmarcReport)),\n            334 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::HttpSubmission,\n            )),\n            337 => Some(EventType::OutgoingReport(OutgoingReportEvent::Locked)),\n            338 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::NoRecipientsFound,\n            )),\n            339 => Some(EventType::OutgoingReport(OutgoingReportEvent::NotFound)),\n            340 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::ReportingAddressValidationError,\n            )),\n            341 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::SpfRateLimited,\n            )),\n            342 => Some(EventType::OutgoingReport(OutgoingReportEvent::SpfReport)),\n            343 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::SubmissionError,\n            )),\n            344 => Some(EventType::OutgoingReport(OutgoingReportEvent::TlsAggregate)),\n            345 => Some(EventType::OutgoingReport(\n                OutgoingReportEvent::UnauthorizedReportingAddress,\n            )),\n            346 => Some(EventType::Pop3(Pop3Event::Capabilities)),\n            347 => Some(EventType::Pop3(Pop3Event::ConnectionEnd)),\n            348 => Some(EventType::Pop3(Pop3Event::ConnectionStart)),\n            349 => Some(EventType::Pop3(Pop3Event::Delete)),\n            350 => Some(EventType::Pop3(Pop3Event::Error)),\n            351 => Some(EventType::Pop3(Pop3Event::Fetch)),\n            352 => Some(EventType::Pop3(Pop3Event::List)),\n            353 => Some(EventType::Pop3(Pop3Event::ListMessage)),\n            354 => Some(EventType::Pop3(Pop3Event::Noop)),\n            355 => Some(EventType::Pop3(Pop3Event::Quit)),\n            356 => Some(EventType::Pop3(Pop3Event::RawInput)),\n            357 => Some(EventType::Pop3(Pop3Event::RawOutput)),\n            358 => Some(EventType::Pop3(Pop3Event::Reset)),\n            359 => Some(EventType::Pop3(Pop3Event::StartTls)),\n            360 => Some(EventType::Pop3(Pop3Event::Stat)),\n            361 => Some(EventType::Pop3(Pop3Event::Uidl)),\n            362 => Some(EventType::Pop3(Pop3Event::UidlMessage)),\n            363 => Some(EventType::Pop3(Pop3Event::Utf8)),\n            364 => Some(EventType::Purge(PurgeEvent::AutoExpunge)),\n            365 => Some(EventType::Purge(PurgeEvent::Error)),\n            366 => Some(EventType::Purge(PurgeEvent::Finished)),\n            367 => Some(EventType::Purge(PurgeEvent::InProgress)),\n            368 => Some(EventType::Purge(PurgeEvent::Running)),\n            369 => Some(EventType::Purge(PurgeEvent::Started)),\n            370 => Some(EventType::Purge(PurgeEvent::BlobCleanup)),\n            371 => Some(EventType::PushSubscription(PushSubscriptionEvent::Error)),\n            372 => Some(EventType::PushSubscription(PushSubscriptionEvent::NotFound)),\n            373 => Some(EventType::PushSubscription(PushSubscriptionEvent::Success)),\n            374 => Some(EventType::Queue(QueueEvent::BlobNotFound)),\n            375 => Some(EventType::Queue(QueueEvent::ConcurrencyLimitExceeded)),\n            377 => Some(EventType::Queue(QueueEvent::Locked)),\n            378 => Some(EventType::Queue(QueueEvent::QueueAutogenerated)),\n            379 => Some(EventType::Queue(QueueEvent::QueueDsn)),\n            380 => Some(EventType::Queue(QueueEvent::QueueMessage)),\n            381 => Some(EventType::Queue(QueueEvent::QueueMessageAuthenticated)),\n            382 => Some(EventType::Queue(QueueEvent::QueueReport)),\n            383 => Some(EventType::Queue(QueueEvent::QuotaExceeded)),\n            384 => Some(EventType::Queue(QueueEvent::RateLimitExceeded)),\n            385 => Some(EventType::Queue(QueueEvent::Rescheduled)),\n            386 => Some(EventType::Resource(ResourceEvent::BadParameters)),\n            387 => Some(EventType::Resource(ResourceEvent::DownloadExternal)),\n            388 => Some(EventType::Resource(ResourceEvent::Error)),\n            389 => Some(EventType::Resource(ResourceEvent::NotFound)),\n            390 => Some(EventType::Resource(ResourceEvent::WebadminUnpacked)),\n            391 => Some(EventType::Server(ServerEvent::Licensing)),\n            392 => Some(EventType::Server(ServerEvent::Shutdown)),\n            393 => Some(EventType::Server(ServerEvent::Startup)),\n            394 => Some(EventType::Server(ServerEvent::StartupError)),\n            395 => Some(EventType::Server(ServerEvent::ThreadError)),\n            396 => Some(EventType::Sieve(SieveEvent::ActionAccept)),\n            397 => Some(EventType::Sieve(SieveEvent::ActionAcceptReplace)),\n            398 => Some(EventType::Sieve(SieveEvent::ActionDiscard)),\n            399 => Some(EventType::Sieve(SieveEvent::ActionReject)),\n            400 => Some(EventType::Sieve(SieveEvent::ListNotFound)),\n            401 => Some(EventType::Sieve(SieveEvent::MessageTooLarge)),\n            402 => Some(EventType::Sieve(SieveEvent::NotSupported)),\n            403 => Some(EventType::Sieve(SieveEvent::QuotaExceeded)),\n            404 => Some(EventType::Sieve(SieveEvent::RuntimeError)),\n            405 => Some(EventType::Sieve(SieveEvent::ScriptNotFound)),\n            406 => Some(EventType::Sieve(SieveEvent::SendMessage)),\n            407 => Some(EventType::Sieve(SieveEvent::UnexpectedError)),\n            408 => Some(EventType::Smtp(SmtpEvent::AlreadyAuthenticated)),\n            409 => Some(EventType::Smtp(SmtpEvent::ArcFail)),\n            410 => Some(EventType::Smtp(SmtpEvent::ArcPass)),\n            411 => Some(EventType::Smtp(SmtpEvent::AuthExchangeTooLong)),\n            412 => Some(EventType::Smtp(SmtpEvent::AuthMechanismNotSupported)),\n            413 => Some(EventType::Smtp(SmtpEvent::AuthNotAllowed)),\n            414 => Some(EventType::Smtp(SmtpEvent::CommandNotImplemented)),\n            415 => Some(EventType::Smtp(SmtpEvent::ConcurrencyLimitExceeded)),\n            416 => Some(EventType::Smtp(SmtpEvent::ConnectionEnd)),\n            417 => Some(EventType::Smtp(SmtpEvent::ConnectionStart)),\n            418 => Some(EventType::Smtp(SmtpEvent::DeliverByDisabled)),\n            419 => Some(EventType::Smtp(SmtpEvent::DeliverByInvalid)),\n            420 => Some(EventType::Smtp(SmtpEvent::DidNotSayEhlo)),\n            421 => Some(EventType::Smtp(SmtpEvent::DkimFail)),\n            422 => Some(EventType::Smtp(SmtpEvent::DkimPass)),\n            423 => Some(EventType::Smtp(SmtpEvent::DmarcFail)),\n            424 => Some(EventType::Smtp(SmtpEvent::DmarcPass)),\n            425 => Some(EventType::Smtp(SmtpEvent::DsnDisabled)),\n            426 => Some(EventType::Smtp(SmtpEvent::Ehlo)),\n            427 => Some(EventType::Smtp(SmtpEvent::EhloExpected)),\n            428 => Some(EventType::Smtp(SmtpEvent::Error)),\n            429 => Some(EventType::Smtp(SmtpEvent::Expn)),\n            430 => Some(EventType::Smtp(SmtpEvent::ExpnDisabled)),\n            431 => Some(EventType::Smtp(SmtpEvent::ExpnNotFound)),\n            432 => Some(EventType::Smtp(SmtpEvent::FutureReleaseDisabled)),\n            433 => Some(EventType::Smtp(SmtpEvent::FutureReleaseInvalid)),\n            434 => Some(EventType::Smtp(SmtpEvent::Help)),\n            435 => Some(EventType::Smtp(SmtpEvent::InvalidCommand)),\n            436 => Some(EventType::Smtp(SmtpEvent::InvalidEhlo)),\n            437 => Some(EventType::Smtp(SmtpEvent::InvalidParameter)),\n            438 => Some(EventType::Smtp(SmtpEvent::InvalidRecipientAddress)),\n            439 => Some(EventType::Smtp(SmtpEvent::InvalidSenderAddress)),\n            440 => Some(EventType::Smtp(SmtpEvent::IprevFail)),\n            441 => Some(EventType::Smtp(SmtpEvent::IprevPass)),\n            442 => Some(EventType::Smtp(SmtpEvent::LhloExpected)),\n            443 => Some(EventType::Smtp(SmtpEvent::LoopDetected)),\n            444 => Some(EventType::Smtp(SmtpEvent::MailFrom)),\n            445 => Some(EventType::Smtp(SmtpEvent::MailFromMissing)),\n            446 => Some(EventType::Smtp(SmtpEvent::MailFromRewritten)),\n            447 => Some(EventType::Smtp(SmtpEvent::MailFromUnauthenticated)),\n            448 => Some(EventType::Smtp(SmtpEvent::MailFromUnauthorized)),\n            449 => Some(EventType::Smtp(SmtpEvent::MailboxDoesNotExist)),\n            450 => Some(EventType::Smtp(SmtpEvent::MessageParseFailed)),\n            451 => Some(EventType::Smtp(SmtpEvent::MessageTooLarge)),\n            452 => Some(EventType::Smtp(SmtpEvent::MissingAuthDirectory)),\n            453 => Some(EventType::Smtp(SmtpEvent::MissingLocalHostname)),\n            454 => Some(EventType::Smtp(SmtpEvent::MtPriorityDisabled)),\n            455 => Some(EventType::Smtp(SmtpEvent::MtPriorityInvalid)),\n            456 => Some(EventType::Smtp(SmtpEvent::MultipleMailFrom)),\n            457 => Some(EventType::Smtp(SmtpEvent::Noop)),\n            460 => Some(EventType::Smtp(SmtpEvent::Quit)),\n            461 => Some(EventType::Smtp(SmtpEvent::RateLimitExceeded)),\n            462 => Some(EventType::Smtp(SmtpEvent::RawInput)),\n            463 => Some(EventType::Smtp(SmtpEvent::RawOutput)),\n            464 => Some(EventType::Smtp(SmtpEvent::RcptTo)),\n            465 => Some(EventType::Smtp(SmtpEvent::RcptToDuplicate)),\n            466 => Some(EventType::Smtp(SmtpEvent::RcptToMissing)),\n            467 => Some(EventType::Smtp(SmtpEvent::RcptToRewritten)),\n            468 => Some(EventType::Smtp(SmtpEvent::RelayNotAllowed)),\n            469 => Some(EventType::Smtp(SmtpEvent::IdNotFound)),\n            470 => Some(EventType::Smtp(SmtpEvent::RequestTooLarge)),\n            471 => Some(EventType::Smtp(SmtpEvent::RequireTlsDisabled)),\n            472 => Some(EventType::Smtp(SmtpEvent::Rset)),\n            473 => Some(EventType::Smtp(SmtpEvent::SpfEhloFail)),\n            474 => Some(EventType::Smtp(SmtpEvent::SpfEhloPass)),\n            475 => Some(EventType::Smtp(SmtpEvent::SpfFromFail)),\n            476 => Some(EventType::Smtp(SmtpEvent::SpfFromPass)),\n            477 => Some(EventType::Smtp(SmtpEvent::StartTls)),\n            478 => Some(EventType::Smtp(SmtpEvent::StartTlsAlready)),\n            479 => Some(EventType::Smtp(SmtpEvent::StartTlsUnavailable)),\n            480 => Some(EventType::Smtp(SmtpEvent::SyntaxError)),\n            481 => Some(EventType::Smtp(SmtpEvent::TimeLimitExceeded)),\n            482 => Some(EventType::Smtp(SmtpEvent::TooManyInvalidRcpt)),\n            483 => Some(EventType::Smtp(SmtpEvent::TooManyMessages)),\n            484 => Some(EventType::Smtp(SmtpEvent::TooManyRecipients)),\n            485 => Some(EventType::Smtp(SmtpEvent::TransferLimitExceeded)),\n            486 => Some(EventType::Smtp(SmtpEvent::UnsupportedParameter)),\n            487 => Some(EventType::Smtp(SmtpEvent::Vrfy)),\n            488 => Some(EventType::Smtp(SmtpEvent::VrfyDisabled)),\n            489 => Some(EventType::Smtp(SmtpEvent::VrfyNotFound)),\n            490 => Some(EventType::Spam(SpamEvent::Classify)),\n            491 => Some(EventType::Spam(SpamEvent::TrainSampleNotFound)),\n            492 => Some(EventType::Store(StoreEvent::HttpStoreFetch)),\n            493 => Some(EventType::Store(StoreEvent::HttpStoreError)),\n            494 => Some(EventType::Spam(SpamEvent::PyzorError)),\n            495 => Some(EventType::Spam(SpamEvent::TrainCompleted)),\n            496 => Some(EventType::Spam(SpamEvent::ModelNotReady)),\n            497 => Some(EventType::Spam(SpamEvent::ModelNotFound)),\n            498 => Some(EventType::Spf(SpfEvent::Fail)),\n            499 => Some(EventType::Spf(SpfEvent::Neutral)),\n            500 => Some(EventType::Spf(SpfEvent::None)),\n            501 => Some(EventType::Spf(SpfEvent::Pass)),\n            502 => Some(EventType::Spf(SpfEvent::PermError)),\n            503 => Some(EventType::Spf(SpfEvent::SoftFail)),\n            504 => Some(EventType::Spf(SpfEvent::TempError)),\n            505 => Some(EventType::Store(StoreEvent::AssertValueFailed)),\n            506 => Some(EventType::Store(StoreEvent::BlobDelete)),\n            507 => Some(EventType::Store(StoreEvent::BlobMissingMarker)),\n            508 => Some(EventType::Store(StoreEvent::BlobRead)),\n            509 => Some(EventType::Store(StoreEvent::BlobWrite)),\n            510 => Some(EventType::Store(StoreEvent::CryptoError)),\n            511 => Some(EventType::Store(StoreEvent::DataCorruption)),\n            512 => Some(EventType::Store(StoreEvent::DataIterate)),\n            513 => Some(EventType::Store(StoreEvent::DataWrite)),\n            514 => Some(EventType::Store(StoreEvent::DecompressError)),\n            515 => Some(EventType::Store(StoreEvent::DeserializeError)),\n            516 => Some(EventType::Store(StoreEvent::ElasticsearchError)),\n            517 => Some(EventType::Store(StoreEvent::FilesystemError)),\n            518 => Some(EventType::Store(StoreEvent::FoundationdbError)),\n            519 => Some(EventType::Store(StoreEvent::LdapWarning)),\n            520 => Some(EventType::Store(StoreEvent::LdapError)),\n            521 => Some(EventType::Store(StoreEvent::LdapQuery)),\n            522 => Some(EventType::Store(StoreEvent::MysqlError)),\n            523 => Some(EventType::Store(StoreEvent::NotConfigured)),\n            524 => Some(EventType::Store(StoreEvent::NotFound)),\n            525 => Some(EventType::Store(StoreEvent::NotSupported)),\n            526 => Some(EventType::Store(StoreEvent::PoolError)),\n            527 => Some(EventType::Store(StoreEvent::PostgresqlError)),\n            528 => Some(EventType::Store(StoreEvent::RedisError)),\n            529 => Some(EventType::Store(StoreEvent::RocksdbError)),\n            530 => Some(EventType::Store(StoreEvent::S3Error)),\n            531 => Some(EventType::Store(StoreEvent::SqlQuery)),\n            532 => Some(EventType::Store(StoreEvent::SqliteError)),\n            533 => Some(EventType::Store(StoreEvent::UnexpectedError)),\n            534 => Some(EventType::Telemetry(TelemetryEvent::JournalError)),\n            535 => Some(EventType::Telemetry(TelemetryEvent::LogError)),\n            536 => Some(EventType::Telemetry(TelemetryEvent::OtelExporterError)),\n            537 => Some(EventType::Telemetry(\n                TelemetryEvent::OtelMetricsExporterError,\n            )),\n            538 => Some(EventType::Telemetry(\n                TelemetryEvent::PrometheusExporterError,\n            )),\n            539 => Some(EventType::Telemetry(TelemetryEvent::WebhookError)),\n            540 => Some(EventType::TlsRpt(TlsRptEvent::RecordFetch)),\n            541 => Some(EventType::TlsRpt(TlsRptEvent::RecordFetchError)),\n            542 => Some(EventType::Tls(TlsEvent::CertificateNotFound)),\n            543 => Some(EventType::Tls(TlsEvent::Handshake)),\n            544 => Some(EventType::Tls(TlsEvent::HandshakeError)),\n            545 => Some(EventType::Tls(TlsEvent::MultipleCertificatesAvailable)),\n            546 => Some(EventType::Tls(TlsEvent::NoCertificatesAvailable)),\n            547 => Some(EventType::Tls(TlsEvent::NotConfigured)),\n            548 => Some(EventType::Telemetry(TelemetryEvent::Alert)),\n            549 => Some(EventType::Security(SecurityEvent::AbuseBan)),\n            550 => Some(EventType::Security(SecurityEvent::LoiterBan)),\n            551 => Some(EventType::Smtp(SmtpEvent::MailFromNotAllowed)),\n            552 => Some(EventType::Security(SecurityEvent::Unauthorized)),\n            553 => Some(EventType::Limit(LimitEvent::TenantQuota)),\n            554 => Some(EventType::Auth(AuthEvent::TokenExpired)),\n            555 => Some(EventType::Auth(AuthEvent::ClientRegistration)),\n            556 => Some(EventType::Ai(AiEvent::LlmResponse)),\n            557 => Some(EventType::Ai(AiEvent::ApiError)),\n            558 => Some(EventType::Security(SecurityEvent::ScanBan)),\n            559 => Some(EventType::Store(StoreEvent::AzureError)),\n            560 => Some(EventType::TlsRpt(TlsRptEvent::RecordNotFound)),\n            561 => Some(EventType::Smtp(SmtpEvent::RcptToGreylisted)),\n            562 => Some(EventType::Spam(SpamEvent::Dnsbl)),\n            563 => Some(EventType::Spam(SpamEvent::DnsblError)),\n            564 => Some(EventType::Spam(SpamEvent::Pyzor)),\n            48 => Some(EventType::Queue(QueueEvent::BackPressure)),\n            57 => Some(EventType::Imap(ImapEvent::GetQuota)),\n            147 => Some(EventType::WebDav(WebDavEvent::Propfind)),\n            148 => Some(EventType::WebDav(WebDavEvent::Proppatch)),\n            335 => Some(EventType::WebDav(WebDavEvent::Get)),\n            336 => Some(EventType::WebDav(WebDavEvent::Report)),\n            376 => Some(EventType::WebDav(WebDavEvent::Mkcol)),\n            458 => Some(EventType::WebDav(WebDavEvent::Delete)),\n            459 => Some(EventType::WebDav(WebDavEvent::Put)),\n            565 => Some(EventType::WebDav(WebDavEvent::Post)),\n            566 => Some(EventType::WebDav(WebDavEvent::Patch)),\n            567 => Some(EventType::WebDav(WebDavEvent::Copy)),\n            568 => Some(EventType::WebDav(WebDavEvent::Move)),\n            569 => Some(EventType::WebDav(WebDavEvent::Lock)),\n            570 => Some(EventType::WebDav(WebDavEvent::Unlock)),\n            571 => Some(EventType::WebDav(WebDavEvent::Acl)),\n            572 => Some(EventType::WebDav(WebDavEvent::Error)),\n            573 => Some(EventType::WebDav(WebDavEvent::Options)),\n            574 => Some(EventType::WebDav(WebDavEvent::Head)),\n            575 => Some(EventType::WebDav(WebDavEvent::Mkcalendar)),\n            576 => Some(EventType::Calendar(CalendarEvent::RuleExpansionError)),\n            50 => Some(EventType::Store(StoreEvent::CacheMiss)),\n            51 => Some(EventType::Store(StoreEvent::CacheHit)),\n            52 => Some(EventType::Store(StoreEvent::CacheStale)),\n            577 => Some(EventType::Store(StoreEvent::CacheUpdate)),\n            578 => Some(EventType::TaskQueue(TaskQueueEvent::TaskAcquired)),\n            579 => Some(EventType::Calendar(CalendarEvent::AlarmSent)),\n            580 => Some(EventType::Calendar(CalendarEvent::AlarmSkipped)),\n            581 => Some(EventType::Calendar(CalendarEvent::AlarmRecipientOverride)),\n            582 => Some(EventType::Calendar(CalendarEvent::AlarmFailed)),\n            583 => Some(EventType::Calendar(CalendarEvent::ItipMessageSent)),\n            584 => Some(EventType::Calendar(CalendarEvent::ItipMessageReceived)),\n            585 => Some(EventType::Calendar(CalendarEvent::ItipMessageError)),\n            586 => Some(EventType::TaskQueue(TaskQueueEvent::TaskIgnored)),\n            587 => Some(EventType::TaskQueue(TaskQueueEvent::TaskFailed)),\n            588 => Some(EventType::Spam(SpamEvent::TrainStarted)),\n            589 => Some(EventType::Spam(SpamEvent::ModelLoaded)),\n            590 => Some(EventType::Store(StoreEvent::MeilisearchError)),\n            _ => None,\n        }\n    }\n}\n\nimpl Key {\n    fn code(&self) -> u64 {\n        match self {\n            Key::AccountName => 0,\n            Key::AccountId => 1,\n            Key::BlobId => 2,\n            Key::CausedBy => 3,\n            Key::ChangeId => 4,\n            Key::Code => 5,\n            Key::Collection => 6,\n            Key::Contents => 7,\n            Key::Details => 8,\n            Key::DkimFail => 9,\n            Key::DkimNone => 10,\n            Key::DkimPass => 11,\n            Key::DmarcNone => 12,\n            Key::DmarcPass => 13,\n            Key::DmarcQuarantine => 14,\n            Key::DmarcReject => 15,\n            Key::DocumentId => 16,\n            Key::Domain => 17,\n            Key::Due => 18,\n            Key::Elapsed => 19,\n            Key::Expires => 20,\n            Key::From => 21,\n            Key::Hostname => 22,\n            Key::Id => 23,\n            Key::Key => 24,\n            Key::Limit => 25,\n            Key::ListenerId => 26,\n            Key::LocalIp => 27,\n            Key::LocalPort => 28,\n            Key::MailboxName => 29,\n            Key::MailboxId => 30,\n            Key::MessageId => 31,\n            Key::NextDsn => 32,\n            Key::NextRetry => 33,\n            Key::Path => 34,\n            Key::Policy => 35,\n            Key::QueueId => 36,\n            Key::RangeFrom => 37,\n            Key::RangeTo => 38,\n            Key::Reason => 39,\n            Key::RemoteIp => 40,\n            Key::RemotePort => 41,\n            Key::ReportId => 42,\n            Key::Result => 43,\n            Key::Size => 44,\n            Key::Source => 45,\n            Key::SpanId => 46,\n            Key::SpfFail => 47,\n            Key::SpfNone => 48,\n            Key::SpfPass => 49,\n            Key::Strict => 50,\n            Key::Tls => 51,\n            Key::To => 52,\n            Key::Total => 53,\n            Key::TotalFailures => 54,\n            Key::TotalSuccesses => 55,\n            Key::Type => 56,\n            Key::Uid => 57,\n            Key::UidNext => 58,\n            Key::UidValidity => 59,\n            Key::Url => 60,\n            Key::ValidFrom => 61,\n            Key::ValidTo => 62,\n            Key::Value => 63,\n            Key::Version => 64,\n            Key::QueueName => 65,\n        }\n    }\n\n    fn from_code(code: u64) -> Option<Self> {\n        match code {\n            0 => Some(Key::AccountName),\n            1 => Some(Key::AccountId),\n            2 => Some(Key::BlobId),\n            3 => Some(Key::CausedBy),\n            4 => Some(Key::ChangeId),\n            5 => Some(Key::Code),\n            6 => Some(Key::Collection),\n            7 => Some(Key::Contents),\n            8 => Some(Key::Details),\n            9 => Some(Key::DkimFail),\n            10 => Some(Key::DkimNone),\n            11 => Some(Key::DkimPass),\n            12 => Some(Key::DmarcNone),\n            13 => Some(Key::DmarcPass),\n            14 => Some(Key::DmarcQuarantine),\n            15 => Some(Key::DmarcReject),\n            16 => Some(Key::DocumentId),\n            17 => Some(Key::Domain),\n            18 => Some(Key::Due),\n            19 => Some(Key::Elapsed),\n            20 => Some(Key::Expires),\n            21 => Some(Key::From),\n            22 => Some(Key::Hostname),\n            23 => Some(Key::Id),\n            24 => Some(Key::Key),\n            25 => Some(Key::Limit),\n            26 => Some(Key::ListenerId),\n            27 => Some(Key::LocalIp),\n            28 => Some(Key::LocalPort),\n            29 => Some(Key::MailboxName),\n            30 => Some(Key::MailboxId),\n            31 => Some(Key::MessageId),\n            32 => Some(Key::NextDsn),\n            33 => Some(Key::NextRetry),\n            34 => Some(Key::Path),\n            35 => Some(Key::Policy),\n            36 => Some(Key::QueueId),\n            37 => Some(Key::RangeFrom),\n            38 => Some(Key::RangeTo),\n            39 => Some(Key::Reason),\n            40 => Some(Key::RemoteIp),\n            41 => Some(Key::RemotePort),\n            42 => Some(Key::ReportId),\n            43 => Some(Key::Result),\n            44 => Some(Key::Size),\n            45 => Some(Key::Source),\n            46 => Some(Key::SpanId),\n            47 => Some(Key::SpfFail),\n            48 => Some(Key::SpfNone),\n            49 => Some(Key::SpfPass),\n            50 => Some(Key::Strict),\n            51 => Some(Key::Tls),\n            52 => Some(Key::To),\n            53 => Some(Key::Total),\n            54 => Some(Key::TotalFailures),\n            55 => Some(Key::TotalSuccesses),\n            56 => Some(Key::Type),\n            57 => Some(Key::Uid),\n            58 => Some(Key::UidNext),\n            59 => Some(Key::UidValidity),\n            60 => Some(Key::Url),\n            61 => Some(Key::ValidFrom),\n            62 => Some(Key::ValidTo),\n            63 => Some(Key::Value),\n            64 => Some(Key::Version),\n            65 => Some(Key::QueueName),\n            _ => None,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/serializers/json.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{Error, Event, EventDetails, Key, Value};\nuse ahash::AHashSet;\nuse base64::{Engine, engine::general_purpose::STANDARD};\nuse mail_parser::DateTime;\nuse serde::{\n    Serialize, Serializer,\n    ser::{SerializeMap, SerializeSeq},\n};\n\nstruct Keys<'x> {\n    keys: &'x [(Key, Value)],\n    span_keys: &'x [(Key, Value)],\n}\n\npub struct JsonEventSerializer<T> {\n    inner: T,\n    with_id: bool,\n    with_spans: bool,\n    with_description: bool,\n    with_explanation: bool,\n}\n\nimpl<T> JsonEventSerializer<T> {\n    pub fn new(inner: T) -> Self {\n        Self {\n            inner,\n            with_id: false,\n            with_spans: false,\n            with_description: false,\n            with_explanation: false,\n        }\n    }\n\n    pub fn with_id(mut self) -> Self {\n        self.with_id = true;\n        self\n    }\n\n    pub fn with_spans(mut self) -> Self {\n        self.with_spans = true;\n        self\n    }\n\n    pub fn with_description(mut self) -> Self {\n        self.with_description = true;\n        self\n    }\n\n    pub fn with_explanation(mut self) -> Self {\n        self.with_explanation = true;\n        self\n    }\n\n    pub fn into_inner(self) -> T {\n        self.inner\n    }\n}\n\nimpl<T: AsRef<Event<EventDetails>>> Serialize for JsonEventSerializer<Vec<T>> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        let mut seq = serializer.serialize_seq(Some(self.inner.len()))?;\n        for event in &self.inner {\n            seq.serialize_element(&JsonEventSerializer {\n                inner: event,\n                with_id: self.with_id,\n                with_spans: self.with_spans,\n                with_description: self.with_description,\n                with_explanation: self.with_explanation,\n            })?;\n        }\n        seq.end()\n    }\n}\n\nimpl<T: AsRef<Event<EventDetails>>> Serialize for JsonEventSerializer<T> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        let event = self.inner.as_ref();\n        let mut map = serializer.serialize_map(None)?;\n        if self.with_id {\n            map.serialize_entry(\n                \"id\",\n                &format!(\"{}{}\", event.inner.timestamp, event.inner.typ.id()),\n            )?;\n        }\n        if self.with_description {\n            map.serialize_entry(\"text\", event.inner.typ.description())?;\n        }\n        if self.with_explanation {\n            map.serialize_entry(\"details\", event.inner.typ.explain())?;\n        }\n        map.serialize_entry(\n            \"createdAt\",\n            &DateTime::from_timestamp(event.inner.timestamp as i64).to_rfc3339(),\n        )?;\n        map.serialize_entry(\"type\", event.inner.typ.name())?;\n        map.serialize_entry(\n            \"data\",\n            &JsonEventSerializer {\n                inner: Keys {\n                    keys: event.keys.as_slice(),\n                    span_keys: event\n                        .inner\n                        .span\n                        .as_ref()\n                        .map(|s| &s.keys[..])\n                        .unwrap_or(&[]),\n                },\n                with_spans: self.with_spans,\n                with_description: self.with_description,\n                with_explanation: self.with_explanation,\n                with_id: self.with_id,\n            },\n        )?;\n        map.end()\n    }\n}\n\nimpl Serialize for JsonEventSerializer<Keys<'_>> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        let keys_len = self.inner.keys.len() + self.inner.span_keys.len();\n        let mut seen_keys = AHashSet::with_capacity(keys_len);\n        let mut keys = serializer.serialize_map(Some(keys_len))?;\n        for (key, value) in self.inner.keys.iter().chain(self.inner.span_keys.iter()) {\n            if !matches!(value, Value::None)\n                && (self.with_spans || !matches!(key, Key::SpanId))\n                && seen_keys.insert(*key)\n            {\n                keys.serialize_entry(\n                    key.name(),\n                    &JsonEventSerializer {\n                        inner: value,\n                        with_spans: self.with_spans,\n                        with_description: self.with_description,\n                        with_explanation: self.with_explanation,\n                        with_id: self.with_id,\n                    },\n                )?;\n            }\n        }\n        keys.end()\n    }\n}\n\nimpl Serialize for JsonEventSerializer<&Error> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        let mut map = serializer.serialize_map(None)?;\n        map.serialize_entry(\"type\", self.inner.0.inner.name())?;\n        if self.with_description {\n            map.serialize_entry(\"text\", self.inner.0.inner.description())?;\n        }\n        if self.with_explanation {\n            map.serialize_entry(\"details\", self.inner.0.inner.explain())?;\n        }\n        map.serialize_entry(\n            \"data\",\n            &JsonEventSerializer {\n                inner: Keys {\n                    keys: self.inner.0.keys.as_slice(),\n                    span_keys: &[],\n                },\n                with_spans: self.with_spans,\n                with_description: self.with_description,\n                with_explanation: self.with_explanation,\n                with_id: self.with_id,\n            },\n        )?;\n        map.end()\n    }\n}\n\nimpl Serialize for JsonEventSerializer<&Value> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        match &self.inner {\n            Value::String(value) => value.serialize(serializer),\n            Value::UInt(value) => value.serialize(serializer),\n            Value::Int(value) => value.serialize(serializer),\n            Value::Float(value) => value.serialize(serializer),\n            Value::Timestamp(value) => DateTime::from_timestamp(*value as i64)\n                .to_rfc3339()\n                .serialize(serializer),\n            Value::Duration(value) => value.serialize(serializer),\n            Value::Bytes(value) => STANDARD.encode(value).serialize(serializer),\n            Value::Bool(value) => value.serialize(serializer),\n            Value::Ipv4(value) => value.serialize(serializer),\n            Value::Ipv6(value) => value.serialize(serializer),\n            Value::Event(value) => JsonEventSerializer {\n                inner: value,\n                with_spans: self.with_spans,\n                with_description: self.with_description,\n                with_explanation: self.with_explanation,\n                with_id: self.with_id,\n            }\n            .serialize(serializer),\n            Value::Array(value) => JsonEventSerializer {\n                inner: value,\n                with_spans: self.with_spans,\n                with_description: self.with_description,\n                with_explanation: self.with_explanation,\n                with_id: self.with_id,\n            }\n            .serialize(serializer),\n            Value::None => unreachable!(),\n        }\n    }\n}\n\nimpl Serialize for JsonEventSerializer<&Vec<Value>> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: Serializer,\n    {\n        let mut seq = serializer.serialize_seq(Some(self.inner.len()))?;\n        for value in self.inner {\n            seq.serialize_element(&JsonEventSerializer {\n                inner: value,\n                with_spans: self.with_spans,\n                with_description: self.with_description,\n                with_explanation: self.with_explanation,\n                with_id: self.with_id,\n            })?;\n        }\n        seq.end()\n    }\n}\n"
  },
  {
    "path": "crates/trc/src/serializers/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod json;\npub mod text;\n\n// SPDX-SnippetBegin\n// SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n// SPDX-License-Identifier: LicenseRef-SEL\n#[cfg(feature = \"enterprise\")]\npub mod binary;\n// SPDX-SnippetEnd\n"
  },
  {
    "path": "crates/trc/src/serializers/text.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\nuse mail_parser::DateTime;\nuse tokio::io::{AsyncWrite, AsyncWriteExt};\n\nuse crate::{Error, Event, EventDetails, Key, Level, Value};\nuse base64::{Engine, engine::general_purpose::STANDARD};\n\npub struct FmtWriter<T: AsyncWrite + Unpin> {\n    writer: T,\n    ansi: bool,\n    multiline: bool,\n}\n\n#[allow(dead_code)]\nenum Color {\n    Black,\n    Red,\n    Green,\n    Yellow,\n    Blue,\n    Magenta,\n    Cyan,\n    White,\n}\n\nimpl<T: AsyncWrite + Unpin> FmtWriter<T> {\n    pub fn new(writer: T) -> Self {\n        Self {\n            writer,\n            ansi: false,\n            multiline: false,\n        }\n    }\n\n    pub fn with_ansi(self, ansi: bool) -> Self {\n        Self { ansi, ..self }\n    }\n\n    pub fn with_multiline(self, multiline: bool) -> Self {\n        Self { multiline, ..self }\n    }\n\n    pub async fn write(&mut self, event: &Event<EventDetails>) -> std::io::Result<()> {\n        // Write timestamp\n        if self.ansi {\n            self.writer\n                .write_all(Color::White.as_code().as_bytes())\n                .await?;\n        }\n        self.writer\n            .write_all(\n                DateTime::from_timestamp(event.inner.timestamp as i64)\n                    .to_rfc3339()\n                    .as_bytes(),\n            )\n            .await?;\n        if self.ansi {\n            self.writer.write_all(Color::reset().as_bytes()).await?;\n        }\n        self.writer.write_all(\" \".as_bytes()).await?;\n\n        // Write level\n        if self.ansi {\n            self.writer\n                .write_all(\n                    match event.inner.level {\n                        Level::Error => Color::Red,\n                        Level::Warn => Color::Yellow,\n                        Level::Info => Color::Green,\n                        Level::Debug => Color::Blue,\n                        Level::Trace => Color::Magenta,\n                        Level::Disable => return Ok(()),\n                    }\n                    .as_code_bold()\n                    .as_bytes(),\n                )\n                .await?;\n        }\n        self.writer\n            .write_all(event.inner.level.as_str().as_bytes())\n            .await?;\n        if self.ansi {\n            self.writer.write_all(Color::reset().as_bytes()).await?;\n        }\n        self.writer.write_all(\" \".as_bytes()).await?;\n\n        // Write message\n        if self.ansi {\n            self.writer\n                .write_all(Color::White.as_code_bold().as_bytes())\n                .await?;\n        }\n        self.writer\n            .write_all(event.inner.typ.description().as_bytes())\n            .await?;\n        if self.ansi {\n            self.writer.write_all(Color::reset().as_bytes()).await?;\n        }\n        self.writer.write_all(\" (\".as_bytes()).await?;\n        self.writer\n            .write_all(event.inner.typ.name().as_bytes())\n            .await?;\n\n        self.writer\n            .write_all(if self.multiline { \")\\n\" } else { \") \" }.as_bytes())\n            .await?;\n\n        // Write keys\n        if let Some(parent_event) = &event.inner.span {\n            self.write_keys(&parent_event.keys, &event.keys, 1).await?;\n        } else {\n            self.write_keys(&[], &event.keys, 1).await?;\n        }\n\n        if !self.multiline {\n            self.writer.write_all(\"\\n\".as_bytes()).await?;\n        }\n\n        Ok(())\n    }\n\n    async fn write_keys(\n        &mut self,\n        span_keys: &[(Key, Value)],\n        keys: &[(Key, Value)],\n        indent: usize,\n    ) -> std::io::Result<()> {\n        Box::pin(async move {\n            let mut is_first = true;\n            for (key, value) in span_keys.iter().chain(keys.iter()) {\n                if matches!(key, Key::SpanId) {\n                    continue;\n                } else if is_first {\n                    is_first = false;\n                } else if !self.multiline {\n                    self.writer.write_all(\", \".as_bytes()).await?;\n                }\n\n                // Write key\n                if self.multiline {\n                    for _ in 0..indent {\n                        self.writer.write_all(\"\\t\".as_bytes()).await?;\n                    }\n                }\n                if self.ansi {\n                    self.writer\n                        .write_all(Color::Cyan.as_code().as_bytes())\n                        .await?;\n                }\n                self.writer.write_all(key.name().as_bytes()).await?;\n                if self.ansi {\n                    self.writer.write_all(Color::reset().as_bytes()).await?;\n                }\n\n                // Write value\n                self.writer.write_all(\" = \".as_bytes()).await?;\n                self.write_value(value, indent).await?;\n\n                if self.multiline && !matches!(value, Value::Event(_)) {\n                    self.writer.write_all(\"\\n\".as_bytes()).await?;\n                }\n            }\n\n            Ok(())\n        })\n        .await\n    }\n\n    async fn write_value(&mut self, value: &Value, indent: usize) -> std::io::Result<()> {\n        Box::pin(async move {\n            match value {\n                Value::String(v) => {\n                    self.writer.write_all(\"\\\"\".as_bytes()).await?;\n                    for ch in v.as_bytes() {\n                        match ch {\n                            b'\\r' => {\n                                self.writer.write_all(\"\\\\r\".as_bytes()).await?;\n                            }\n                            b'\\n' => {\n                                self.writer.write_all(\"\\\\n\".as_bytes()).await?;\n                            }\n                            b'\\t' => {\n                                self.writer.write_all(\"\\\\t\".as_bytes()).await?;\n                            }\n                            b'\\\\' => {\n                                self.writer.write_all(\"\\\\\\\\\".as_bytes()).await?;\n                            }\n                            _ => {\n                                self.writer.write_all(&[*ch]).await?;\n                            }\n                        }\n                    }\n                    self.writer.write_all(\"\\\"\".as_bytes()).await?;\n                }\n                Value::UInt(v) => {\n                    self.writer.write_all(v.to_string().as_bytes()).await?;\n                }\n                Value::Int(v) => {\n                    self.writer.write_all(v.to_string().as_bytes()).await?;\n                }\n                Value::Float(v) => {\n                    self.writer.write_all(v.to_string().as_bytes()).await?;\n                }\n                Value::Timestamp(v) => {\n                    self.writer\n                        .write_all(DateTime::from_timestamp(*v as i64).to_rfc3339().as_bytes())\n                        .await?;\n                }\n                Value::Duration(v) => {\n                    self.writer.write_all(v.to_string().as_bytes()).await?;\n                    self.writer.write_all(\"ms\".as_bytes()).await?;\n                }\n                Value::Bytes(bytes) => {\n                    self.writer.write_all(\"base64:\".as_bytes()).await?;\n                    self.writer\n                        .write_all(STANDARD.encode(bytes).as_bytes())\n                        .await?;\n                }\n                Value::Bool(true) => {\n                    self.writer.write_all(\"true\".as_bytes()).await?;\n                }\n                Value::Bool(false) => {\n                    self.writer.write_all(\"false\".as_bytes()).await?;\n                }\n                Value::Ipv4(v) => {\n                    self.writer.write_all(v.to_string().as_bytes()).await?;\n                }\n                Value::Ipv6(v) => {\n                    self.writer.write_all(v.to_string().as_bytes()).await?;\n                }\n                Value::Event(e) => {\n                    self.writer\n                        .write_all(e.0.inner.description().as_bytes())\n                        .await?;\n                    self.writer.write_all(\" (\".as_bytes()).await?;\n                    self.writer.write_all(e.0.inner.name().as_bytes()).await?;\n                    self.writer.write_all(\")\".as_bytes()).await?;\n                    if !e.0.keys.is_empty() {\n                        self.writer\n                            .write_all(if self.multiline { \"\\n\" } else { \" { \" }.as_bytes())\n                            .await?;\n\n                        self.write_keys(&e.0.keys, &[], indent + 1).await?;\n\n                        if !self.multiline {\n                            self.writer.write_all(\" }\".as_bytes()).await?;\n                        }\n                    } else if self.multiline {\n                        self.writer.write_all(\"\\n\".as_bytes()).await?;\n                    }\n                }\n                Value::Array(arr) => {\n                    self.writer.write_all(\"[\".as_bytes()).await?;\n                    for (pos, value) in arr.iter().enumerate() {\n                        if pos > 0 {\n                            self.writer.write_all(\", \".as_bytes()).await?;\n                        }\n                        self.write_value(value, indent).await?;\n                    }\n                    self.writer.write_all(\"]\".as_bytes()).await?;\n                }\n                Value::None => {\n                    self.writer.write_all(\"(null)\".as_bytes()).await?;\n                }\n            }\n\n            Ok(())\n        })\n        .await\n    }\n\n    pub async fn flush(&mut self) -> std::io::Result<()> {\n        self.writer.flush().await\n    }\n\n    pub fn update_writer(&mut self, writer: T) {\n        self.writer = writer;\n    }\n}\n\nimpl Color {\n    pub fn as_code(&self) -> &'static str {\n        match self {\n            Color::Black => \"\\x1b[30m\",\n            Color::Red => \"\\x1b[31m\",\n            Color::Green => \"\\x1b[32m\",\n            Color::Yellow => \"\\x1b[33m\",\n            Color::Blue => \"\\x1b[34m\",\n            Color::Magenta => \"\\x1b[35m\",\n            Color::Cyan => \"\\x1b[36m\",\n            Color::White => \"\\x1b[37m\",\n        }\n    }\n\n    pub fn as_code_bold(&self) -> &'static str {\n        match self {\n            Color::Black => \"\\x1b[30;1m\",\n            Color::Red => \"\\x1b[31;1m\",\n            Color::Green => \"\\x1b[32;1m\",\n            Color::Yellow => \"\\x1b[33;1m\",\n            Color::Blue => \"\\x1b[34;1m\",\n            Color::Magenta => \"\\x1b[35;1m\",\n            Color::Cyan => \"\\x1b[36;1m\",\n            Color::White => \"\\x1b[37;1m\",\n        }\n    }\n\n    pub fn reset() -> &'static str {\n        \"\\x1b[0m\"\n    }\n}\n\nimpl Display for Value {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Value::String(value) => value.fmt(f),\n            Value::UInt(value) => value.fmt(f),\n            Value::Int(value) => value.fmt(f),\n            Value::Float(value) => value.fmt(f),\n            Value::Timestamp(value) => value.fmt(f),\n            Value::Duration(value) => value.fmt(f),\n            Value::Bytes(value) => STANDARD.encode(value).fmt(f),\n            Value::Bool(value) => value.fmt(f),\n            Value::Ipv4(value) => value.fmt(f),\n            Value::Ipv6(value) => value.fmt(f),\n            Value::Event(value) => {\n                \"{\".fmt(f)?;\n                value.fmt(f)?;\n                \"}\".fmt(f)\n            }\n            Value::Array(value) => {\n                f.write_str(\"[\")?;\n                for (i, value) in value.iter().enumerate() {\n                    if i > 0 {\n                        f.write_str(\", \")?;\n                    }\n                    value.fmt(f)?;\n                }\n                f.write_str(\"]\")\n            }\n            Value::None => \"(null)\".fmt(f),\n        }\n    }\n}\n\nimpl Display for Error {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        self.0.inner.description().fmt(f)?;\n        \" (\".fmt(f)?;\n        self.0.inner.name().fmt(f)?;\n        \")\".fmt(f)?;\n\n        if !self.0.keys.is_empty() {\n            f.write_str(\": \")?;\n            for (i, (key, value)) in self.0.keys.iter().enumerate() {\n                if i > 0 {\n                    f.write_str(\", \")?;\n                }\n                key.name().fmt(f)?;\n                f.write_str(\" = \")?;\n                value.fmt(f)?;\n            }\n        }\n\n        Ok(())\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use crate::{EventType, Level};\n\n    #[allow(dead_code)]\n    fn to_camel_case(name: &str) -> String {\n        let mut out = String::with_capacity(name.len());\n        let mut upper = true;\n        for ch in name.chars() {\n            if ch.is_alphanumeric() {\n                if upper {\n                    out.push(ch.to_ascii_uppercase());\n                    upper = false;\n                } else {\n                    out.push(ch);\n                }\n            } else {\n                upper = true;\n            }\n        }\n        out\n    }\n\n    #[allow(dead_code)]\n    fn event_to_class(name: &str) -> String {\n        let (group, name) = name.split_once('.').unwrap();\n        let group = to_camel_case(group);\n        format!(\n            \"EventType::{}({}Event::{})\",\n            group,\n            group,\n            to_camel_case(name)\n        )\n    }\n\n    #[allow(dead_code)]\n    fn event_to_webadmin_class(name: &str) -> String {\n        let (group, name) = name.split_once('.').unwrap();\n        format!(\"{}{}\", to_camel_case(group), to_camel_case(name))\n    }\n\n    #[test]\n    fn print_all_events() {\n        assert!(!Level::Disable.is_contained(Level::Warn));\n        assert!(Level::Trace.is_contained(Level::Error));\n        assert!(Level::Trace.is_contained(Level::Debug));\n        assert!(!Level::Error.is_contained(Level::Trace));\n        assert!(!Level::Debug.is_contained(Level::Trace));\n\n        let mut names = Vec::with_capacity(100);\n\n        for event in EventType::variants() {\n            names.push((event.name(), event.description(), event.level().as_str()));\n            assert_eq!(EventType::try_parse(event.name()).unwrap(), event);\n        }\n\n        // sort by name\n        names.sort_by(|a, b| a.0.cmp(b.0));\n\n        for (name, description, level) in names {\n            //println!(\"{:?},\", name);\n            println!(\"|`{name}`|{description}|`{level}`|\")\n        }\n\n        //for (pos, (name, _, _)) in names.iter().enumerate() {\n            //println!(\"{:?},\", name);\n            //println!(\"{} => Some({}),\", pos, event_to_class(name));\n            //println!(\"{} => {},\", event_to_class(name), pos);\n            /*println!(\n                \"#[serde(rename = \\\"{name}\\\")]\\n{},\",\n                event_to_webadmin_class(name)\n            );*/\n        //}\n    }\n}\n"
  },
  {
    "path": "crates/types/Cargo.toml",
    "content": "[package]\nname = \"types\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\nutils = { path = \"../utils\" }\ntrc = { path = \"../trc\" }\njmap-tools = { version = \"0.1\" }\nhashify = \"0.2\"\nserde = { version = \"1.0\", features = [\"derive\"]}\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = { version = \"0.9.0\", features = [\"rkyv\", \"serde\"] }\nblake3 = \"1.3.3\"\n\n\n[features]\ntest_mode = []\n\n"
  },
  {
    "path": "crates/types/src/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::{self, Display};\nuse utils::map::bitmap::{Bitmap, BitmapItem};\n\n#[derive(\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    PartialOrd,\n    Ord,\n    Hash,\n    Copy,\n)]\n#[rkyv(compare(PartialEq), derive(Debug))]\n#[repr(u8)]\npub enum Acl {\n    Read = 0,\n    Modify = 1,\n    Delete = 2,\n    ReadItems = 3,\n    AddItems = 4,\n    ModifyItems = 5,\n    RemoveItems = 6,\n    CreateChild = 7,\n    Share = 8,\n    Submit = 9,\n    SchedulingReadFreeBusy = 10,\n    SchedulingInvite = 11,\n    SchedulingReply = 12,\n    ModifyItemsOwn = 13,\n    ModifyPrivateProperties = 14,\n    ModifyRSVP = 15,\n    None = 16,\n}\n\n#[derive(\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    serde::Serialize,\n    Default,\n)]\n#[rkyv(compare(PartialEq), derive(Debug))]\npub struct AclGrant {\n    pub account_id: u32,\n    pub grants: Bitmap<Acl>,\n}\n\nimpl Acl {\n    fn as_str(&self) -> &'static str {\n        match self {\n            Acl::Read => \"read\",\n            Acl::Modify => \"modify\",\n            Acl::Delete => \"delete\",\n            Acl::ReadItems => \"readItems\",\n            Acl::AddItems => \"addItems\",\n            Acl::ModifyItems => \"modifyItems\",\n            Acl::RemoveItems => \"removeItems\",\n            Acl::CreateChild => \"createChild\",\n            Acl::Share => \"share\",\n            Acl::Submit => \"submit\",\n            Acl::ModifyItemsOwn => \"modifyItemsOwn\",\n            Acl::ModifyPrivateProperties => \"modifyPrivateProperties\",\n            Acl::None => \"\",\n            Acl::SchedulingReadFreeBusy => \"schedulingReadFreeBusy\",\n            Acl::SchedulingInvite => \"schedulingInvite\",\n            Acl::SchedulingReply => \"schedulingReply\",\n            Acl::ModifyRSVP => \"modifyRSVP\",\n        }\n    }\n}\n\nimpl Display for Acl {\n    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {\n        write!(f, \"{}\", self.as_str())\n    }\n}\n\nimpl serde::Serialize for Acl {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.as_str())\n    }\n}\n\nimpl BitmapItem for Acl {\n    fn max() -> u64 {\n        Acl::None as u64\n    }\n\n    fn is_valid(&self) -> bool {\n        !matches!(self, Acl::None)\n    }\n}\n\nimpl From<Acl> for u64 {\n    fn from(value: Acl) -> Self {\n        value as u64\n    }\n}\n\nimpl From<u64> for Acl {\n    fn from(value: u64) -> Self {\n        match value {\n            0 => Acl::Read,\n            1 => Acl::Modify,\n            2 => Acl::Delete,\n            3 => Acl::ReadItems,\n            4 => Acl::AddItems,\n            5 => Acl::ModifyItems,\n            6 => Acl::RemoveItems,\n            7 => Acl::CreateChild,\n            8 => Acl::Share,\n            9 => Acl::Submit,\n            10 => Acl::SchedulingReadFreeBusy,\n            11 => Acl::SchedulingInvite,\n            12 => Acl::SchedulingReply,\n            13 => Acl::ModifyItemsOwn,\n            14 => Acl::ModifyPrivateProperties,\n            15 => Acl::ModifyRSVP,\n            _ => Acl::None,\n        }\n    }\n}\n\nimpl From<&ArchivedAclGrant> for AclGrant {\n    fn from(value: &ArchivedAclGrant) -> Self {\n        Self {\n            account_id: u32::from(value.account_id),\n            grants: (&value.grants).into(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/types/src/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_tools::{Element, Property, Value};\nuse std::{borrow::Borrow, str::FromStr, time::SystemTime};\nuse utils::codec::{\n    base32_custom::{Base32Reader, Base32Writer},\n    leb128::{Leb128Iterator, Leb128Writer},\n};\n\nuse crate::blob_hash::BlobHash;\n\nconst B_LINKED: u8 = 0x10;\nconst B_RESERVED: u8 = 0x20;\n\n#[derive(Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]\npub enum BlobClass {\n    Reserved {\n        account_id: u32,\n        expires: u64,\n    },\n    Linked {\n        account_id: u32,\n        collection: u8,\n        document_id: u32,\n    },\n}\n\nimpl Default for BlobClass {\n    fn default() -> Self {\n        BlobClass::Reserved {\n            account_id: 0,\n            expires: 0,\n        }\n    }\n}\n\nimpl AsRef<BlobClass> for BlobClass {\n    fn as_ref(&self) -> &BlobClass {\n        self\n    }\n}\n\nimpl BlobClass {\n    pub fn account_id(&self) -> u32 {\n        match self {\n            BlobClass::Reserved { account_id, .. } | BlobClass::Linked { account_id, .. } => {\n                *account_id\n            }\n        }\n    }\n\n    pub fn is_valid(&self) -> bool {\n        match self {\n            BlobClass::Reserved { expires, .. } => {\n                *expires\n                    > SystemTime::now()\n                        .duration_since(SystemTime::UNIX_EPOCH)\n                        .map_or(0, |d| d.as_secs())\n            }\n            BlobClass::Linked { .. } => true,\n        }\n    }\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub struct BlobId {\n    pub hash: BlobHash,\n    pub class: BlobClass,\n    pub section: Option<BlobSection>,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]\npub struct BlobSection {\n    pub offset_start: usize,\n    pub size: usize,\n    pub encoding: u8,\n}\n\nimpl FromStr for BlobId {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        BlobId::from_base32(s).ok_or(())\n    }\n}\n\nimpl BlobId {\n    pub fn new(hash: BlobHash, class: BlobClass) -> Self {\n        BlobId {\n            hash,\n            class,\n            section: None,\n        }\n    }\n\n    pub fn new_section(\n        hash: BlobHash,\n        class: BlobClass,\n        offset_start: usize,\n        offset_end: usize,\n        encoding: impl Into<u8>,\n    ) -> Self {\n        BlobId {\n            hash,\n            class,\n            section: BlobSection {\n                offset_start,\n                size: offset_end - offset_start,\n                encoding: encoding.into(),\n            }\n            .into(),\n        }\n    }\n\n    pub fn with_section_size(mut self, size: usize) -> Self {\n        self.section.get_or_insert_with(Default::default).size = size;\n        self\n    }\n\n    #[inline]\n    pub fn from_base32(value: impl AsRef<[u8]>) -> Option<Self> {\n        BlobId::from_iter(&mut Base32Reader::new(value.as_ref()))\n    }\n\n    #[allow(clippy::should_implement_trait)]\n    pub fn from_iter<T, U>(it: &mut T) -> Option<Self>\n    where\n        T: Iterator<Item = U> + Leb128Iterator<U>,\n        U: Borrow<u8>,\n    {\n        let class = *it.next()?.borrow();\n        let encoding = class & 0x0F;\n\n        let mut hash = BlobHash::default();\n        for byte in hash.as_mut().iter_mut() {\n            *byte = *it.next()?.borrow();\n        }\n\n        let account_id: u32 = it.next_leb128()?;\n\n        BlobId {\n            hash,\n            class: if (class & B_LINKED) != 0 {\n                BlobClass::Linked {\n                    account_id,\n                    collection: *it.next()?.borrow(),\n                    document_id: it.next_leb128()?,\n                }\n            } else {\n                BlobClass::Reserved {\n                    account_id,\n                    expires: it.next_leb128()?,\n                }\n            },\n            section: if encoding != 0 {\n                BlobSection {\n                    offset_start: it.next_leb128()?,\n                    size: it.next_leb128()?,\n                    encoding: encoding - 1,\n                }\n                .into()\n            } else {\n                None\n            },\n        }\n        .into()\n    }\n\n    fn serialize_as(&self, writer: &mut impl Leb128Writer) {\n        let marker = self\n            .section\n            .as_ref()\n            .map_or(0, |section| section.encoding + 1)\n            | if matches!(\n                self,\n                BlobId {\n                    class: BlobClass::Linked { .. },\n                    ..\n                }\n            ) {\n                B_LINKED\n            } else {\n                B_RESERVED\n            };\n\n        let _ = writer.write(&[marker]);\n        let _ = writer.write(self.hash.as_ref());\n\n        match &self.class {\n            BlobClass::Reserved {\n                account_id,\n                expires,\n            } => {\n                let _ = writer.write_leb128(*account_id);\n                let _ = writer.write_leb128(*expires);\n            }\n            BlobClass::Linked {\n                account_id,\n                collection,\n                document_id,\n            } => {\n                let _ = writer.write_leb128(*account_id);\n                let _ = writer.write(&[*collection]);\n                let _ = writer.write_leb128(*document_id);\n            }\n        }\n\n        if let Some(section) = &self.section {\n            let _ = writer.write_leb128(section.offset_start);\n            let _ = writer.write_leb128(section.size);\n        }\n    }\n\n    pub fn start_offset(&self) -> usize {\n        if let Some(section) = &self.section {\n            section.offset_start\n        } else {\n            0\n        }\n    }\n}\n\nimpl serde::Serialize for BlobId {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.to_string().as_str())\n    }\n}\n\nimpl std::fmt::Display for BlobId {\n    #[allow(clippy::unused_io_amount)]\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let mut writer = Base32Writer::with_capacity(std::mem::size_of::<BlobId>() * 2);\n        self.serialize_as(&mut writer);\n        f.write_str(&writer.finalize())\n    }\n}\n\nimpl<'x, P: Property, E: Element + From<BlobId>> From<BlobId> for Value<'x, P, E> {\n    fn from(id: BlobId) -> Self {\n        Value::Element(E::from(id))\n    }\n}\n"
  },
  {
    "path": "crates/types/src/blob_hash.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub const BLOB_HASH_LEN: usize = 32;\n\n#[derive(\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    Clone,\n    Debug,\n    Default,\n    PartialEq,\n    Eq,\n    Hash,\n    PartialOrd,\n    Ord,\n    serde::Serialize,\n    serde::Deserialize,\n)]\n#[rkyv(derive(Debug))]\n#[repr(transparent)]\npub struct BlobHash(pub [u8; BLOB_HASH_LEN]);\n\nimpl BlobHash {\n    pub fn new_max() -> Self {\n        BlobHash([u8::MAX; BLOB_HASH_LEN])\n    }\n\n    pub fn generate(value: impl AsRef<[u8]>) -> Self {\n        BlobHash(blake3::hash(value.as_ref()).into())\n    }\n\n    pub fn try_from_hash_slice(value: &[u8]) -> Result<BlobHash, std::array::TryFromSliceError> {\n        value.try_into().map(BlobHash)\n    }\n\n    pub fn as_slice(&self) -> &[u8] {\n        self.0.as_ref()\n    }\n\n    pub fn to_hex(&self) -> String {\n        let mut hex = String::with_capacity(BLOB_HASH_LEN * 2);\n        for byte in self.0.iter() {\n            hex.push_str(&format!(\"{:02x}\", byte));\n        }\n        hex\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.0 == [0; BLOB_HASH_LEN]\n    }\n}\n\nimpl From<&ArchivedBlobHash> for BlobHash {\n    fn from(value: &ArchivedBlobHash) -> Self {\n        BlobHash(value.0)\n    }\n}\n\nimpl AsRef<BlobHash> for BlobHash {\n    fn as_ref(&self) -> &BlobHash {\n        self\n    }\n}\n\nimpl From<BlobHash> for Vec<u8> {\n    fn from(value: BlobHash) -> Self {\n        value.0.to_vec()\n    }\n}\n\nimpl AsRef<[u8]> for BlobHash {\n    fn as_ref(&self) -> &[u8] {\n        self.0.as_ref()\n    }\n}\n\nimpl AsMut<[u8]> for BlobHash {\n    fn as_mut(&mut self) -> &mut [u8] {\n        self.0.as_mut()\n    }\n}\n"
  },
  {
    "path": "crates/types/src/collection.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::type_state::DataType;\nuse compact_str::CompactString;\nuse std::{\n    fmt::{self, Display, Formatter},\n    str::FromStr,\n};\nuse utils::map::bitmap::BitmapItem;\n\n#[derive(\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    Debug,\n    Clone,\n    Copy,\n    Hash,\n    PartialEq,\n    Eq,\n    Default,\n)]\n#[repr(u8)]\npub enum Collection {\n    Email = 0,\n    Mailbox = 1,\n    Thread = 2,\n    Identity = 3,\n    EmailSubmission = 4,\n    SieveScript = 5,\n    PushSubscription = 6,\n    Principal = 7,\n    Calendar = 8,\n    CalendarEvent = 9,\n    AddressBook = 10,\n    ContactCard = 11,\n    FileNode = 12,\n    CalendarEventNotification = 13,\n    #[default]\n    None = 14,\n}\n\n#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, Default)]\n#[repr(u8)]\npub enum SyncCollection {\n    Email = 0,\n    Thread = 1,\n    Calendar = 2,\n    AddressBook = 3,\n    FileNode = 4,\n    Identity = 5,\n    EmailSubmission = 6,\n    SieveScript = 7,\n    CalendarEventNotification = 8,\n    ShareNotification = 9,\n    #[default]\n    None = 10,\n}\n\n#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq)]\n#[repr(u8)]\npub enum VanishedCollection {\n    Email = 251,\n    Calendar = 252,\n    AddressBook = 253,\n    FileNode = 254,\n}\n\nimpl Collection {\n    pub const MAX: usize = Collection::None as usize;\n\n    pub fn main_collection(&self) -> Collection {\n        match self {\n            Collection::Email => Collection::Mailbox,\n            Collection::CalendarEvent => Collection::Calendar,\n            Collection::ContactCard => Collection::AddressBook,\n            _ => *self,\n        }\n    }\n\n    pub fn parent_collection(&self) -> Option<Collection> {\n        match self {\n            Collection::Email => Some(Collection::Mailbox),\n            Collection::CalendarEvent => Some(Collection::Calendar),\n            Collection::ContactCard => Some(Collection::AddressBook),\n            Collection::FileNode => Some(Collection::FileNode),\n            Collection::CalendarEventNotification => Some(Collection::CalendarEventNotification),\n            _ => None,\n        }\n    }\n\n    pub fn child_collection(&self) -> Option<Collection> {\n        match self {\n            Collection::Mailbox => Some(Collection::Email),\n            Collection::Calendar => Some(Collection::CalendarEvent),\n            Collection::AddressBook => Some(Collection::ContactCard),\n            Collection::FileNode => Some(Collection::FileNode),\n            Collection::CalendarEventNotification => Some(Collection::CalendarEventNotification),\n            _ => None,\n        }\n    }\n}\n\nimpl SyncCollection {\n    pub fn collection(&self, is_container: bool) -> Collection {\n        match self {\n            SyncCollection::Email => {\n                if is_container {\n                    Collection::Mailbox\n                } else {\n                    Collection::Email\n                }\n            }\n            SyncCollection::Thread => Collection::Thread,\n            SyncCollection::Calendar => {\n                if is_container {\n                    Collection::Calendar\n                } else {\n                    Collection::CalendarEvent\n                }\n            }\n            SyncCollection::AddressBook => {\n                if is_container {\n                    Collection::AddressBook\n                } else {\n                    Collection::ContactCard\n                }\n            }\n            SyncCollection::FileNode => Collection::FileNode,\n            SyncCollection::Identity => Collection::Identity,\n            SyncCollection::EmailSubmission => Collection::EmailSubmission,\n            SyncCollection::SieveScript => Collection::SieveScript,\n            SyncCollection::CalendarEventNotification => Collection::CalendarEventNotification,\n            SyncCollection::ShareNotification | SyncCollection::None => Collection::None,\n        }\n    }\n\n    pub fn vanished_collection(&self) -> Option<VanishedCollection> {\n        match self {\n            SyncCollection::Email => Some(VanishedCollection::Email),\n            SyncCollection::Calendar => Some(VanishedCollection::Calendar),\n            SyncCollection::AddressBook => Some(VanishedCollection::AddressBook),\n            SyncCollection::FileNode => Some(VanishedCollection::FileNode),\n            _ => None,\n        }\n    }\n}\n\nimpl From<Collection> for SyncCollection {\n    fn from(v: Collection) -> Self {\n        match v {\n            Collection::Email => SyncCollection::Email,\n            Collection::Mailbox => SyncCollection::Email,\n            Collection::Thread => SyncCollection::Thread,\n            Collection::Identity => SyncCollection::Identity,\n            Collection::EmailSubmission => SyncCollection::EmailSubmission,\n            Collection::SieveScript => SyncCollection::SieveScript,\n            Collection::PushSubscription => SyncCollection::None,\n            Collection::Principal => SyncCollection::None,\n            Collection::Calendar => SyncCollection::Calendar,\n            Collection::CalendarEvent => SyncCollection::Calendar,\n            Collection::CalendarEventNotification => SyncCollection::CalendarEventNotification,\n            Collection::AddressBook => SyncCollection::AddressBook,\n            Collection::ContactCard => SyncCollection::AddressBook,\n            Collection::FileNode => SyncCollection::FileNode,\n            _ => SyncCollection::None,\n        }\n    }\n}\n\nimpl From<u8> for Collection {\n    fn from(v: u8) -> Self {\n        match v {\n            0 => Collection::Email,\n            1 => Collection::Mailbox,\n            2 => Collection::Thread,\n            3 => Collection::Identity,\n            4 => Collection::EmailSubmission,\n            5 => Collection::SieveScript,\n            6 => Collection::PushSubscription,\n            7 => Collection::Principal,\n            8 => Collection::Calendar,\n            9 => Collection::CalendarEvent,\n            10 => Collection::AddressBook,\n            11 => Collection::ContactCard,\n            12 => Collection::FileNode,\n            13 => Collection::CalendarEventNotification,\n            _ => Collection::None,\n        }\n    }\n}\n\nimpl From<u8> for SyncCollection {\n    fn from(v: u8) -> Self {\n        match v {\n            0 => SyncCollection::Email,\n            1 => SyncCollection::Thread,\n            2 => SyncCollection::Calendar,\n            3 => SyncCollection::AddressBook,\n            4 => SyncCollection::FileNode,\n            5 => SyncCollection::Identity,\n            6 => SyncCollection::EmailSubmission,\n            7 => SyncCollection::SieveScript,\n            8 => SyncCollection::CalendarEventNotification,\n            9 => SyncCollection::ShareNotification,\n            _ => SyncCollection::None,\n        }\n    }\n}\n\nimpl From<u64> for SyncCollection {\n    fn from(v: u64) -> Self {\n        match v {\n            0 => SyncCollection::Email,\n            1 => SyncCollection::Thread,\n            2 => SyncCollection::Calendar,\n            3 => SyncCollection::AddressBook,\n            4 => SyncCollection::FileNode,\n            5 => SyncCollection::Identity,\n            6 => SyncCollection::EmailSubmission,\n            7 => SyncCollection::SieveScript,\n            8 => SyncCollection::CalendarEventNotification,\n            9 => SyncCollection::ShareNotification,\n            _ => SyncCollection::None,\n        }\n    }\n}\n\nimpl From<u64> for Collection {\n    fn from(v: u64) -> Self {\n        match v {\n            0 => Collection::Email,\n            1 => Collection::Mailbox,\n            2 => Collection::Thread,\n            3 => Collection::Identity,\n            4 => Collection::EmailSubmission,\n            5 => Collection::SieveScript,\n            6 => Collection::PushSubscription,\n            7 => Collection::Principal,\n            8 => Collection::Calendar,\n            9 => Collection::CalendarEvent,\n            10 => Collection::AddressBook,\n            11 => Collection::ContactCard,\n            12 => Collection::FileNode,\n            13 => Collection::CalendarEventNotification,\n            _ => Collection::None,\n        }\n    }\n}\n\nimpl From<Collection> for u8 {\n    fn from(v: Collection) -> Self {\n        v as u8\n    }\n}\n\nimpl From<SyncCollection> for u8 {\n    fn from(v: SyncCollection) -> Self {\n        v as u8\n    }\n}\n\nimpl From<SyncCollection> for u64 {\n    fn from(v: SyncCollection) -> Self {\n        v as u64\n    }\n}\n\nimpl From<VanishedCollection> for u8 {\n    fn from(v: VanishedCollection) -> Self {\n        v as u8\n    }\n}\n\nimpl From<Collection> for u64 {\n    fn from(collection: Collection) -> u64 {\n        collection as u64\n    }\n}\n\nimpl TryFrom<Collection> for DataType {\n    type Error = ();\n\n    fn try_from(value: Collection) -> Result<Self, Self::Error> {\n        match value {\n            Collection::Email => Ok(DataType::Email),\n            Collection::Mailbox => Ok(DataType::Mailbox),\n            Collection::Thread => Ok(DataType::Thread),\n            Collection::Identity => Ok(DataType::Identity),\n            Collection::EmailSubmission => Ok(DataType::EmailSubmission),\n            Collection::SieveScript => Ok(DataType::SieveScript),\n            Collection::PushSubscription => Ok(DataType::PushSubscription),\n            Collection::Principal => Ok(DataType::Principal),\n            Collection::Calendar => Ok(DataType::Calendar),\n            Collection::CalendarEvent => Ok(DataType::CalendarEvent),\n            Collection::AddressBook => Ok(DataType::AddressBook),\n            Collection::ContactCard => Ok(DataType::ContactCard),\n            Collection::FileNode => Ok(DataType::FileNode),\n            Collection::CalendarEventNotification => Ok(DataType::CalendarEventNotification),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl TryFrom<DataType> for Collection {\n    type Error = ();\n\n    fn try_from(value: DataType) -> Result<Self, Self::Error> {\n        match value {\n            DataType::Email => Ok(Collection::Email),\n            DataType::Mailbox => Ok(Collection::Mailbox),\n            DataType::Thread => Ok(Collection::Thread),\n            DataType::Identity => Ok(Collection::Identity),\n            DataType::EmailSubmission => Ok(Collection::EmailSubmission),\n            DataType::SieveScript => Ok(Collection::SieveScript),\n            DataType::PushSubscription => Ok(Collection::PushSubscription),\n            DataType::Principal => Ok(Collection::Principal),\n            DataType::Calendar => Ok(Collection::Calendar),\n            DataType::CalendarEvent => Ok(Collection::CalendarEvent),\n            DataType::AddressBook => Ok(Collection::AddressBook),\n            DataType::ContactCard => Ok(Collection::ContactCard),\n            DataType::FileNode => Ok(Collection::FileNode),\n            DataType::CalendarEventNotification => Ok(Collection::CalendarEventNotification),\n            _ => Err(()),\n        }\n    }\n}\n\nimpl Display for Collection {\n    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {\n        self.as_str().fmt(f)\n    }\n}\n\nimpl Collection {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            Collection::PushSubscription => \"pushSubscription\",\n            Collection::Email => \"email\",\n            Collection::Mailbox => \"mailbox\",\n            Collection::Thread => \"thread\",\n            Collection::Identity => \"identity\",\n            Collection::EmailSubmission => \"emailSubmission\",\n            Collection::SieveScript => \"sieveScript\",\n            Collection::Principal => \"principal\",\n            Collection::Calendar => \"calendar\",\n            Collection::CalendarEvent => \"calendarEvent\",\n            Collection::AddressBook => \"addressBook\",\n            Collection::ContactCard => \"contactCard\",\n            Collection::FileNode => \"fileNode\",\n            Collection::CalendarEventNotification => \"calendarEventNotification\",\n            Collection::None => \"\",\n        }\n    }\n\n    pub fn as_config_case(&self) -> &'static str {\n        match self {\n            Collection::PushSubscription => \"push-subscription\",\n            Collection::Email => \"email\",\n            Collection::Mailbox => \"mailbox\",\n            Collection::Thread => \"thread\",\n            Collection::Identity => \"identity\",\n            Collection::EmailSubmission => \"email-submission\",\n            Collection::SieveScript => \"sieve-script\",\n            Collection::Principal => \"principal\",\n            Collection::Calendar => \"calendar\",\n            Collection::CalendarEvent => \"calendar-event\",\n            Collection::AddressBook => \"address-book\",\n            Collection::ContactCard => \"contact-card\",\n            Collection::FileNode => \"file-node\",\n            Collection::CalendarEventNotification => \"calendar-event-notification\",\n            Collection::None => \"\",\n        }\n    }\n}\n\nimpl FromStr for Collection {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        hashify::tiny_map!(s.as_bytes(),\n            \"pushSubscription\" => Collection::PushSubscription,\n            \"email\" => Collection::Email,\n            \"mailbox\" => Collection::Mailbox,\n            \"thread\" => Collection::Thread,\n            \"identity\" => Collection::Identity,\n            \"emailSubmission\" => Collection::EmailSubmission,\n            \"sieveScript\" => Collection::SieveScript,\n            \"principal\" => Collection::Principal,\n            \"calendar\" => Collection::Calendar,\n            \"calendarEvent\" => Collection::CalendarEvent,\n            \"addressBook\" => Collection::AddressBook,\n            \"contactCard\" => Collection::ContactCard,\n            \"fileNode\" => Collection::FileNode,\n            \"calendarEventNotification\" => Collection::CalendarEventNotification,\n        )\n        .ok_or(())\n    }\n}\n\nimpl From<Collection> for trc::Value {\n    fn from(value: Collection) -> Self {\n        trc::Value::String(CompactString::const_new(value.as_str()))\n    }\n}\n\nimpl BitmapItem for Collection {\n    fn max() -> u64 {\n        Collection::None as u64\n    }\n\n    fn is_valid(&self) -> bool {\n        !matches!(self, Collection::None)\n    }\n}\n\nimpl BitmapItem for SyncCollection {\n    fn max() -> u64 {\n        SyncCollection::None as u64\n    }\n\n    fn is_valid(&self) -> bool {\n        !matches!(self, SyncCollection::None)\n    }\n}\n\nimpl SyncCollection {\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            SyncCollection::Email => \"email\",\n            SyncCollection::Thread => \"thread\",\n            SyncCollection::Calendar => \"calendar\",\n            SyncCollection::AddressBook => \"addressBook\",\n            SyncCollection::FileNode => \"fileNode\",\n            SyncCollection::Identity => \"identity\",\n            SyncCollection::EmailSubmission => \"emailSubmission\",\n            SyncCollection::SieveScript => \"sieveScript\",\n            SyncCollection::CalendarEventNotification => \"calendarEventNotification\",\n            SyncCollection::ShareNotification => \"shareNotification\",\n            SyncCollection::None => \"\",\n        }\n    }\n}\n"
  },
  {
    "path": "crates/types/src/dead_property.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[derive(Debug, Clone, PartialEq, Eq, rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)]\n#[cfg_attr(feature = \"test_mode\", derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(feature = \"test_mode\", serde(tag = \"type\", content = \"data\"))]\n#[rkyv(derive(Debug))]\npub enum DeadPropertyTag {\n    ElementStart(DeadElementTag),\n    ElementEnd,\n    Text(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)]\n#[cfg_attr(feature = \"test_mode\", derive(serde::Serialize, serde::Deserialize))]\n#[rkyv(derive(Debug))]\npub struct DeadElementTag {\n    pub name: String,\n    pub attrs: Option<String>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, rkyv::Archive, rkyv::Serialize, rkyv::Deserialize)]\n#[cfg_attr(feature = \"test_mode\", derive(serde::Serialize, serde::Deserialize))]\n#[cfg_attr(feature = \"test_mode\", serde(transparent))]\n#[rkyv(derive(Debug))]\n#[repr(transparent)]\npub struct DeadProperty(pub Vec<DeadPropertyTag>);\n\nimpl From<&ArchivedDeadProperty> for DeadProperty {\n    fn from(value: &ArchivedDeadProperty) -> Self {\n        DeadProperty(value.0.iter().map(|tag| tag.into()).collect::<Vec<_>>())\n    }\n}\n\nimpl From<&ArchivedDeadPropertyTag> for DeadPropertyTag {\n    fn from(tag: &ArchivedDeadPropertyTag) -> Self {\n        match tag {\n            ArchivedDeadPropertyTag::ElementStart(tag) => DeadPropertyTag::ElementStart(tag.into()),\n            ArchivedDeadPropertyTag::ElementEnd => DeadPropertyTag::ElementEnd,\n            ArchivedDeadPropertyTag::Text(tag) => DeadPropertyTag::Text(tag.to_string()),\n        }\n    }\n}\n\nimpl From<&ArchivedDeadElementTag> for DeadElementTag {\n    fn from(tag: &ArchivedDeadElementTag) -> Self {\n        DeadElementTag {\n            name: tag.name.to_string(),\n            attrs: tag.attrs.as_ref().map(|s| s.to_string()),\n        }\n    }\n}\n\nimpl ArchivedDeadProperty {\n    pub fn find_tag(&self, needle: &str) -> Option<DeadProperty> {\n        let mut depth: u32 = 0;\n        let mut tags = Vec::new();\n        let mut found_tag = false;\n\n        for tag in self.0.iter() {\n            match tag {\n                ArchivedDeadPropertyTag::ElementStart(start) => {\n                    if depth == 0 && start.name == needle {\n                        found_tag = true;\n                    } else if found_tag {\n                        tags.push(tag.into());\n                    }\n\n                    depth += 1;\n                }\n                ArchivedDeadPropertyTag::ElementEnd => {\n                    if found_tag {\n                        if depth == 1 {\n                            break;\n                        } else {\n                            tags.push(tag.into());\n                        }\n                    }\n                    depth = depth.saturating_sub(1);\n                }\n                ArchivedDeadPropertyTag::Text(_) => {\n                    if found_tag {\n                        tags.push(tag.into());\n                    }\n                }\n            }\n        }\n\n        if found_tag {\n            Some(DeadProperty(tags))\n        } else {\n            None\n        }\n    }\n}\n\nimpl DeadProperty {\n    pub fn remove_element(&mut self, element: &DeadElementTag) {\n        let mut depth = 0;\n        let mut remove = false;\n        self.0.retain(|item| match item {\n            DeadPropertyTag::ElementStart(tag) => {\n                if depth == 0 && !remove && tag.name == element.name {\n                    remove = true;\n                }\n                depth += 1;\n\n                !remove\n            }\n            DeadPropertyTag::ElementEnd => {\n                depth -= 1;\n                if remove && depth == 0 {\n                    remove = false;\n                    false\n                } else {\n                    !remove\n                }\n            }\n            _ => !remove,\n        });\n    }\n\n    pub fn add_element(&mut self, element: DeadElementTag, values: Vec<DeadPropertyTag>) {\n        self.0.push(DeadPropertyTag::ElementStart(element));\n        self.0.extend(values);\n        self.0.push(DeadPropertyTag::ElementEnd);\n    }\n\n    pub fn size(&self) -> usize {\n        let mut size = 0;\n        for item in &self.0 {\n            match item {\n                DeadPropertyTag::ElementStart(tag) => {\n                    size += tag.size();\n                }\n                DeadPropertyTag::ElementEnd => {\n                    size += 1;\n                }\n                DeadPropertyTag::Text(text) => {\n                    size += text.len();\n                }\n            }\n        }\n        size\n    }\n}\n\nimpl ArchivedDeadProperty {\n    pub fn size(&self) -> usize {\n        let mut size = 0;\n        for item in self.0.iter() {\n            match item {\n                ArchivedDeadPropertyTag::ElementStart(tag) => {\n                    size += tag.size();\n                }\n                ArchivedDeadPropertyTag::ElementEnd => {\n                    size += 1;\n                }\n                ArchivedDeadPropertyTag::Text(text) => {\n                    size += text.len();\n                }\n            }\n        }\n        size\n    }\n}\n\nimpl DeadElementTag {\n    pub fn new(name: String, attrs: Option<String>) -> Self {\n        DeadElementTag { name, attrs }\n    }\n\n    pub fn size(&self) -> usize {\n        self.name.len() + self.attrs.as_ref().map_or(0, |attrs| attrs.len())\n    }\n}\n\nimpl ArchivedDeadElementTag {\n    pub fn size(&self) -> usize {\n        self.name.len() + self.attrs.as_ref().map_or(0, |attrs| attrs.len())\n    }\n}\n\nimpl Default for DeadProperty {\n    fn default() -> Self {\n        DeadProperty(Vec::with_capacity(4))\n    }\n}\n"
  },
  {
    "path": "crates/types/src/field.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nconst ARCHIVE_FIELD: u8 = 50;\n\npub trait FieldType: Into<u8> + Copy + std::fmt::Debug + PartialEq + Eq {}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(transparent)]\npub struct Field(u8);\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum ContactField {\n    Uid,\n    Email,\n    Archive,\n    CreatedToUpdated,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum CalendarEventField {\n    Uid,\n    Archive,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum CalendarNotificationField {\n    CreatedToId,\n    Archive,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum EmailField {\n    Archive,\n    Metadata,\n    Threading,\n    DeletedAt,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum MailboxField {\n    UidCounter,\n    Archive,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum SieveField {\n    Name,\n    Ids,\n    Archive,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum EmailSubmissionField {\n    Archive,\n    Metadata,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum IdentityField {\n    Archive,\n    DocumentId,\n}\n\n#[derive(Clone, Copy, Debug, PartialEq, Eq)]\n#[repr(u8)]\npub enum PrincipalField {\n    Archive,\n    EncryptionKeys,\n    ParticipantIdentities,\n    DefaultCalendarId,\n    DefaultAddressBookId,\n    ActiveScriptId,\n    PushSubscriptions,\n}\n\nimpl From<ContactField> for u8 {\n    fn from(value: ContactField) -> Self {\n        match value {\n            ContactField::Uid => 0,\n            ContactField::Email => 1,\n            ContactField::CreatedToUpdated => 2,\n            ContactField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<CalendarEventField> for u8 {\n    fn from(value: CalendarEventField) -> Self {\n        match value {\n            CalendarEventField::Uid => 0,\n            CalendarEventField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<CalendarNotificationField> for u8 {\n    fn from(value: CalendarNotificationField) -> Self {\n        match value {\n            CalendarNotificationField::CreatedToId => 0,\n            CalendarNotificationField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<EmailField> for u8 {\n    fn from(value: EmailField) -> Self {\n        match value {\n            EmailField::Metadata => 71,\n            EmailField::Threading => 90,\n            EmailField::DeletedAt => 91,\n            EmailField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<MailboxField> for u8 {\n    fn from(value: MailboxField) -> Self {\n        match value {\n            MailboxField::UidCounter => 84,\n            MailboxField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<SieveField> for u8 {\n    fn from(value: SieveField) -> Self {\n        match value {\n            SieveField::Name => 13,\n            SieveField::Ids => 84,\n            SieveField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<EmailSubmissionField> for u8 {\n    fn from(value: EmailSubmissionField) -> Self {\n        match value {\n            EmailSubmissionField::Metadata => 49,\n            EmailSubmissionField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<PrincipalField> for u8 {\n    fn from(value: PrincipalField) -> Self {\n        match value {\n            PrincipalField::ParticipantIdentities => 45,\n            PrincipalField::EncryptionKeys => 46,\n            PrincipalField::DefaultCalendarId => 47,\n            PrincipalField::DefaultAddressBookId => 48,\n            PrincipalField::ActiveScriptId => 49,\n            PrincipalField::PushSubscriptions => 44,\n            PrincipalField::Archive => ARCHIVE_FIELD,\n        }\n    }\n}\n\nimpl From<IdentityField> for u8 {\n    fn from(value: IdentityField) -> Self {\n        match value {\n            IdentityField::Archive => ARCHIVE_FIELD,\n            IdentityField::DocumentId => 51,\n        }\n    }\n}\n\nimpl From<Field> for u8 {\n    fn from(value: Field) -> Self {\n        value.0\n    }\n}\n\nimpl From<ContactField> for Field {\n    fn from(value: ContactField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<CalendarEventField> for Field {\n    fn from(value: CalendarEventField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<CalendarNotificationField> for Field {\n    fn from(value: CalendarNotificationField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<EmailField> for Field {\n    fn from(value: EmailField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<MailboxField> for Field {\n    fn from(value: MailboxField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<PrincipalField> for Field {\n    fn from(value: PrincipalField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<SieveField> for Field {\n    fn from(value: SieveField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<EmailSubmissionField> for Field {\n    fn from(value: EmailSubmissionField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl From<IdentityField> for Field {\n    fn from(value: IdentityField) -> Self {\n        Field(u8::from(value))\n    }\n}\n\nimpl Field {\n    pub const ARCHIVE: Field = Field(ARCHIVE_FIELD);\n\n    pub fn new(value: u8) -> Self {\n        Field(value)\n    }\n\n    pub fn inner(&self) -> u8 {\n        self.0\n    }\n}\n\nimpl FieldType for Field {}\nimpl FieldType for ContactField {}\nimpl FieldType for CalendarEventField {}\nimpl FieldType for CalendarNotificationField {}\nimpl FieldType for EmailField {}\nimpl FieldType for MailboxField {}\nimpl FieldType for PrincipalField {}\nimpl FieldType for SieveField {}\nimpl FieldType for EmailSubmissionField {}\nimpl FieldType for IdentityField {}\n"
  },
  {
    "path": "crates/types/src/id.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::DocumentId;\nuse jmap_tools::{Element, Property, Value};\nuse std::{ops::Deref, str::FromStr};\nuse utils::codec::base32_custom::{BASE32_ALPHABET, BASE32_INVERSE};\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash, Copy, PartialOrd, Ord)]\n#[repr(transparent)]\npub struct Id(u64);\n\nimpl Default for Id {\n    fn default() -> Self {\n        Id(u64::MAX)\n    }\n}\n\nimpl FromStr for Id {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        let mut id = 0;\n\n        for &ch in s.as_bytes() {\n            let i = BASE32_INVERSE[ch as usize];\n            if i != u8::MAX {\n                id = (id << 5) | i as u64;\n            } else {\n                return Err(());\n            }\n        }\n\n        Ok(Id(id))\n    }\n}\n\nimpl Id {\n    pub fn new(id: u64) -> Self {\n        Self(id)\n    }\n\n    pub fn singleton() -> Self {\n        Self::new(20080258862541)\n    }\n\n    // From https://github.com/archer884/crockford by J/A <archer884@gmail.com>\n    // License: MIT/Apache 2.0\n    pub fn as_string(&self) -> String {\n        match self.0 {\n            0 => \"a\".to_string(),\n            mut n => {\n                // Used for the initial shift.\n                const QUAD_SHIFT: usize = 60;\n                const QUAD_RESET: usize = 4;\n\n                // Used for all subsequent shifts.\n                const FIVE_SHIFT: usize = 59;\n                const FIVE_RESET: usize = 5;\n\n                // After we clear the four most significant bits, the four least significant bits will be\n                // replaced with 0001. We can then know to stop once the four most significant bits are,\n                // likewise, 0001.\n                const STOP_BIT: u64 = 1 << QUAD_SHIFT;\n\n                let mut buf = String::with_capacity(7);\n\n                // Start by getting the most significant four bits. We get four here because these would be\n                // leftovers when starting from the least significant bits. In either case, tag the four least\n                // significant bits with our stop bit.\n                match (n >> QUAD_SHIFT) as usize {\n                    // Eat leading zero-bits. This should not be done if the first four bits were non-zero.\n                    // Additionally, we *must* do this in increments of five bits.\n                    0 => {\n                        n <<= QUAD_RESET;\n                        n |= 1;\n                        n <<= n.leading_zeros() / 5 * 5;\n                    }\n\n                    // Write value of first four bytes.\n                    i => {\n                        n <<= QUAD_RESET;\n                        n |= 1;\n                        buf.push(char::from(BASE32_ALPHABET[i]));\n                    }\n                }\n\n                // From now until we reach the stop bit, take the five most significant bits and then shift\n                // left by five bits.\n                while n != STOP_BIT {\n                    buf.push(char::from(BASE32_ALPHABET[(n >> FIVE_SHIFT) as usize]));\n                    n <<= FIVE_RESET;\n                }\n\n                buf\n            }\n        }\n    }\n\n    #[inline(always)]\n    pub fn from_parts(prefix_id: DocumentId, doc_id: DocumentId) -> Id {\n        Id(((prefix_id as u64) << 32) | doc_id as u64)\n    }\n\n    #[inline(always)]\n    pub fn id(&self) -> u64 {\n        self.0\n    }\n\n    #[inline(always)]\n    pub fn document_id(&self) -> DocumentId {\n        (self.0 & 0xFFFFFFFF) as DocumentId\n    }\n\n    #[inline(always)]\n    pub fn prefix_id(&self) -> DocumentId {\n        (self.0 >> 32) as DocumentId\n    }\n\n    #[inline(always)]\n    pub fn is_singleton(&self) -> bool {\n        self.0 == 20080258862541\n    }\n\n    #[inline(always)]\n    pub fn is_valid(&self) -> bool {\n        self.0 != u64::MAX\n    }\n}\n\nimpl From<u64> for Id {\n    fn from(id: u64) -> Self {\n        Id(id)\n    }\n}\n\nimpl From<u32> for Id {\n    fn from(id: u32) -> Self {\n        Id(id as u64)\n    }\n}\n\nimpl From<Id> for u64 {\n    fn from(id: Id) -> Self {\n        id.0\n    }\n}\n\nimpl From<&Id> for u64 {\n    fn from(id: &Id) -> Self {\n        id.0\n    }\n}\n\nimpl From<(u32, u32)> for Id {\n    fn from(id: (u32, u32)) -> Self {\n        Id::from_parts(id.0, id.1)\n    }\n}\n\nimpl Deref for Id {\n    type Target = u64;\n\n    fn deref(&self) -> &Self::Target {\n        &self.0\n    }\n}\n\nimpl AsRef<u64> for Id {\n    fn as_ref(&self) -> &u64 {\n        &self.0\n    }\n}\n\nimpl From<Id> for u32 {\n    fn from(id: Id) -> Self {\n        id.document_id()\n    }\n}\n\nimpl From<Id> for String {\n    fn from(id: Id) -> Self {\n        id.as_string()\n    }\n}\n\nimpl serde::Serialize for Id {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        serializer.serialize_str(self.as_string().as_str())\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for Id {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        Id::from_str(<&str>::deserialize(deserializer)?)\n            .map_err(|_| serde::de::Error::custom(\"invalid JMAP ID\"))\n    }\n}\n\nimpl std::fmt::Display for Id {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.write_str(&self.as_string())\n    }\n}\n\nimpl<'x, P: Property, E: Element + From<Id>> From<Id> for Value<'x, P, E> {\n    fn from(id: Id) -> Self {\n        Value::Element(E::from(id))\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use std::str::FromStr;\n\n    use crate::id::Id;\n\n    #[test]\n    fn parse_jmap_id() {\n        for number in [\n            0,\n            1,\n            10,\n            1000,\n            Id::singleton().id(),\n            u64::MAX / 2,\n            u64::MAX - 1,\n            u64::MAX,\n        ] {\n            let id = Id::from(number);\n            assert_eq!(Id::from_str(&id.to_string()).unwrap(), id);\n        }\n\n        Id::from_str(\"p333333333333p333333333333\").unwrap();\n    }\n}\n"
  },
  {
    "path": "crates/types/src/keyword.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_tools::{Element, Property, Value};\nuse std::{fmt::Display, str::FromStr};\n\npub const SEEN: usize = 0;\npub const DRAFT: usize = 1;\npub const FLAGGED: usize = 2;\npub const ANSWERED: usize = 3;\npub const RECENT: usize = 4;\npub const IMPORTANT: usize = 5;\npub const PHISHING: usize = 6;\npub const JUNK: usize = 7;\npub const NOTJUNK: usize = 8;\npub const DELETED: usize = 9;\npub const FORWARDED: usize = 10;\npub const MDN_SENT: usize = 11;\npub const AUTOSENT: usize = 12;\npub const CANUNSUBSCRIBE: usize = 13;\npub const FOLLOWED: usize = 14;\npub const HASATTACHMENT: usize = 15;\npub const HASMEMO: usize = 16;\npub const HASNOATTACHMENT: usize = 17;\npub const IMPORTED: usize = 18;\npub const ISTRUSTED: usize = 19;\npub const MAILFLAGBIT0: usize = 20;\npub const MAILFLAGBIT1: usize = 21;\npub const MAILFLAGBIT2: usize = 22;\npub const MASKEDEMAIL: usize = 23;\npub const MEMO: usize = 24;\npub const MUTED: usize = 25;\npub const NEW: usize = 26;\npub const NOTIFY: usize = 27;\npub const UNSUBSCRIBED: usize = 28;\npub const OTHER: usize = 29;\n\n#[derive(\n    rkyv::Serialize,\n    rkyv::Deserialize,\n    rkyv::Archive,\n    Debug,\n    Clone,\n    PartialEq,\n    Eq,\n    Hash,\n    Default,\n    PartialOrd,\n    Ord,\n    serde::Serialize,\n)]\n#[serde(untagged)]\n#[rkyv(derive(PartialEq), compare(PartialEq))]\npub enum Keyword {\n    Other(Box<str>),\n    #[serde(rename(serialize = \"$seen\"))]\n    Seen,\n    #[serde(rename(serialize = \"$draft\"))]\n    Draft,\n    #[serde(rename(serialize = \"$flagged\"))]\n    Flagged,\n    #[serde(rename(serialize = \"$answered\"))]\n    Answered,\n    #[default]\n    #[serde(rename(serialize = \"$recent\"))]\n    Recent,\n    #[serde(rename(serialize = \"$important\"))]\n    Important,\n    #[serde(rename(serialize = \"$phishing\"))]\n    Phishing,\n    #[serde(rename(serialize = \"$junk\"))]\n    Junk,\n    #[serde(rename(serialize = \"$notjunk\"))]\n    NotJunk,\n    #[serde(rename(serialize = \"$deleted\"))]\n    Deleted,\n    #[serde(rename(serialize = \"$forwarded\"))]\n    Forwarded,\n    #[serde(rename(serialize = \"$mdnsent\"))]\n    MdnSent,\n    #[serde(rename(serialize = \"$autosent\"))]\n    Autosent,\n    #[serde(rename(serialize = \"$canunsubscribe\"))]\n    CanUnsubscribe,\n    #[serde(rename(serialize = \"$followed\"))]\n    Followed,\n    #[serde(rename(serialize = \"$hasattachment\"))]\n    HasAttachment,\n    #[serde(rename(serialize = \"$hasmemo\"))]\n    HasMemo,\n    #[serde(rename(serialize = \"$hasnoattachment\"))]\n    HasNoAttachment,\n    #[serde(rename(serialize = \"$imported\"))]\n    Imported,\n    #[serde(rename(serialize = \"$istrusted\"))]\n    IsTrusted,\n    #[serde(rename(serialize = \"$MailFlagBit0\"))]\n    MailFlagBit0,\n    #[serde(rename(serialize = \"$MailFlagBit1\"))]\n    MailFlagBit1,\n    #[serde(rename(serialize = \"$MailFlagBit2\"))]\n    MailFlagBit2,\n    #[serde(rename(serialize = \"$maskedemail\"))]\n    MaskedEmail,\n    #[serde(rename(serialize = \"$memo\"))]\n    Memo,\n    #[serde(rename(serialize = \"$muted\"))]\n    Muted,\n    #[serde(rename(serialize = \"$new\"))]\n    New,\n    #[serde(rename(serialize = \"$notify\"))]\n    Notify,\n    #[serde(rename(serialize = \"$unsubscribed\"))]\n    Unsubscribed,\n}\n\nimpl Keyword {\n    pub const MAX_LENGTH: usize = 128;\n\n    pub fn parse(value: &str) -> Self {\n        Self::try_parse(value)\n            .unwrap_or_else(|| Keyword::Other(value.chars().take(Keyword::MAX_LENGTH).collect()))\n    }\n\n    pub fn from_other(value: String) -> Self {\n        if value.len() <= Keyword::MAX_LENGTH {\n            Keyword::Other(value.into_boxed_str())\n        } else {\n            Keyword::Other(value.chars().take(Keyword::MAX_LENGTH).collect())\n        }\n    }\n\n    pub fn from_boxed_other(value: Box<str>) -> Self {\n        if value.len() <= Keyword::MAX_LENGTH {\n            Keyword::Other(value)\n        } else {\n            Keyword::Other(value.chars().take(Keyword::MAX_LENGTH).collect())\n        }\n    }\n\n    pub fn try_parse(value: &str) -> Option<Self> {\n        value\n            .split_at_checked(1)\n            .filter(|(prefix, _)| matches!(*prefix, \"$\" | \"\\\\\"))\n            .and_then(|(_, rest)| {\n                hashify::tiny_map_ignore_case!(rest.as_bytes(),\n                    \"seen\" => Keyword::Seen,\n                    \"draft\" => Keyword::Draft,\n                    \"flagged\" => Keyword::Flagged,\n                    \"answered\" => Keyword::Answered,\n                    \"recent\" => Keyword::Recent,\n                    \"important\" => Keyword::Important,\n                    \"phishing\" => Keyword::Phishing,\n                    \"junk\" => Keyword::Junk,\n                    \"notjunk\" => Keyword::NotJunk,\n                    \"deleted\" => Keyword::Deleted,\n                    \"forwarded\" => Keyword::Forwarded,\n                    \"mdnsent\" => Keyword::MdnSent,\n                    \"autosent\" => Keyword::Autosent,\n                    \"canunsubscribe\" => Keyword::CanUnsubscribe,\n                    \"followed\" => Keyword::Followed,\n                    \"hasattachment\" => Keyword::HasAttachment,\n                    \"hasmemo\" => Keyword::HasMemo,\n                    \"hasnoattachment\" => Keyword::HasNoAttachment,\n                    \"imported\" => Keyword::Imported,\n                    \"istrusted\" => Keyword::IsTrusted,\n                    \"mailflagbit0\" => Keyword::MailFlagBit0,\n                    \"mailflagbit1\" => Keyword::MailFlagBit1,\n                    \"mailflagbit2\" => Keyword::MailFlagBit2,\n                    \"maskedemail\" => Keyword::MaskedEmail,\n                    \"memo\" => Keyword::Memo,\n                    \"muted\" => Keyword::Muted,\n                    \"new\" => Keyword::New,\n                    \"notify\" => Keyword::Notify,\n                    \"unsubscribed\" => Keyword::Unsubscribed,\n                )\n            })\n    }\n\n    pub fn id(&self) -> Result<u32, &str> {\n        match self {\n            Keyword::Seen => Ok(SEEN as u32),\n            Keyword::Draft => Ok(DRAFT as u32),\n            Keyword::Flagged => Ok(FLAGGED as u32),\n            Keyword::Answered => Ok(ANSWERED as u32),\n            Keyword::Recent => Ok(RECENT as u32),\n            Keyword::Important => Ok(IMPORTANT as u32),\n            Keyword::Phishing => Ok(PHISHING as u32),\n            Keyword::Junk => Ok(JUNK as u32),\n            Keyword::NotJunk => Ok(NOTJUNK as u32),\n            Keyword::Deleted => Ok(DELETED as u32),\n            Keyword::Forwarded => Ok(FORWARDED as u32),\n            Keyword::MdnSent => Ok(MDN_SENT as u32),\n            Keyword::Autosent => Ok(AUTOSENT as u32),\n            Keyword::CanUnsubscribe => Ok(CANUNSUBSCRIBE as u32),\n            Keyword::Followed => Ok(FOLLOWED as u32),\n            Keyword::HasAttachment => Ok(HASATTACHMENT as u32),\n            Keyword::HasMemo => Ok(HASMEMO as u32),\n            Keyword::HasNoAttachment => Ok(HASNOATTACHMENT as u32),\n            Keyword::Imported => Ok(IMPORTED as u32),\n            Keyword::IsTrusted => Ok(ISTRUSTED as u32),\n            Keyword::MailFlagBit0 => Ok(MAILFLAGBIT0 as u32),\n            Keyword::MailFlagBit1 => Ok(MAILFLAGBIT1 as u32),\n            Keyword::MailFlagBit2 => Ok(MAILFLAGBIT2 as u32),\n            Keyword::MaskedEmail => Ok(MASKEDEMAIL as u32),\n            Keyword::Memo => Ok(MEMO as u32),\n            Keyword::Muted => Ok(MUTED as u32),\n            Keyword::New => Ok(NEW as u32),\n            Keyword::Notify => Ok(NOTIFY as u32),\n            Keyword::Unsubscribed => Ok(UNSUBSCRIBED as u32),\n            Keyword::Other(string) => Err(string.as_ref()),\n        }\n    }\n\n    pub fn into_id(self) -> Result<u32, Box<str>> {\n        match self {\n            Keyword::Seen => Ok(SEEN as u32),\n            Keyword::Draft => Ok(DRAFT as u32),\n            Keyword::Flagged => Ok(FLAGGED as u32),\n            Keyword::Answered => Ok(ANSWERED as u32),\n            Keyword::Recent => Ok(RECENT as u32),\n            Keyword::Important => Ok(IMPORTANT as u32),\n            Keyword::Phishing => Ok(PHISHING as u32),\n            Keyword::Junk => Ok(JUNK as u32),\n            Keyword::NotJunk => Ok(NOTJUNK as u32),\n            Keyword::Deleted => Ok(DELETED as u32),\n            Keyword::Forwarded => Ok(FORWARDED as u32),\n            Keyword::MdnSent => Ok(MDN_SENT as u32),\n            Keyword::Autosent => Ok(AUTOSENT as u32),\n            Keyword::CanUnsubscribe => Ok(CANUNSUBSCRIBE as u32),\n            Keyword::Followed => Ok(FOLLOWED as u32),\n            Keyword::HasAttachment => Ok(HASATTACHMENT as u32),\n            Keyword::HasMemo => Ok(HASMEMO as u32),\n            Keyword::HasNoAttachment => Ok(HASNOATTACHMENT as u32),\n            Keyword::Imported => Ok(IMPORTED as u32),\n            Keyword::IsTrusted => Ok(ISTRUSTED as u32),\n            Keyword::MailFlagBit0 => Ok(MAILFLAGBIT0 as u32),\n            Keyword::MailFlagBit1 => Ok(MAILFLAGBIT1 as u32),\n            Keyword::MailFlagBit2 => Ok(MAILFLAGBIT2 as u32),\n            Keyword::MaskedEmail => Ok(MASKEDEMAIL as u32),\n            Keyword::Memo => Ok(MEMO as u32),\n            Keyword::Muted => Ok(MUTED as u32),\n            Keyword::New => Ok(NEW as u32),\n            Keyword::Notify => Ok(NOTIFY as u32),\n            Keyword::Unsubscribed => Ok(UNSUBSCRIBED as u32),\n            Keyword::Other(string) => Err(string),\n        }\n    }\n\n    pub fn try_from_id(id: usize) -> Result<Self, usize> {\n        match id {\n            SEEN => Ok(Keyword::Seen),\n            DRAFT => Ok(Keyword::Draft),\n            FLAGGED => Ok(Keyword::Flagged),\n            ANSWERED => Ok(Keyword::Answered),\n            RECENT => Ok(Keyword::Recent),\n            IMPORTANT => Ok(Keyword::Important),\n            PHISHING => Ok(Keyword::Phishing),\n            JUNK => Ok(Keyword::Junk),\n            NOTJUNK => Ok(Keyword::NotJunk),\n            DELETED => Ok(Keyword::Deleted),\n            FORWARDED => Ok(Keyword::Forwarded),\n            MDN_SENT => Ok(Keyword::MdnSent),\n            AUTOSENT => Ok(Keyword::Autosent),\n            CANUNSUBSCRIBE => Ok(Keyword::CanUnsubscribe),\n            FOLLOWED => Ok(Keyword::Followed),\n            HASATTACHMENT => Ok(Keyword::HasAttachment),\n            HASMEMO => Ok(Keyword::HasMemo),\n            HASNOATTACHMENT => Ok(Keyword::HasNoAttachment),\n            IMPORTED => Ok(Keyword::Imported),\n            ISTRUSTED => Ok(Keyword::IsTrusted),\n            MAILFLAGBIT0 => Ok(Keyword::MailFlagBit0),\n            MAILFLAGBIT1 => Ok(Keyword::MailFlagBit1),\n            MAILFLAGBIT2 => Ok(Keyword::MailFlagBit2),\n            MASKEDEMAIL => Ok(Keyword::MaskedEmail),\n            MEMO => Ok(Keyword::Memo),\n            MUTED => Ok(Keyword::Muted),\n            NEW => Ok(Keyword::New),\n            NOTIFY => Ok(Keyword::Notify),\n            UNSUBSCRIBED => Ok(Keyword::Unsubscribed),\n            _ => Err(id),\n        }\n    }\n}\n\nimpl From<String> for Keyword {\n    fn from(value: String) -> Self {\n        Keyword::try_parse(&value).unwrap_or_else(|| Keyword::from_other(value))\n    }\n}\n\nimpl Display for Keyword {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Keyword::Seen => write!(f, \"$seen\"),\n            Keyword::Draft => write!(f, \"$draft\"),\n            Keyword::Flagged => write!(f, \"$flagged\"),\n            Keyword::Answered => write!(f, \"$answered\"),\n            Keyword::Recent => write!(f, \"$recent\"),\n            Keyword::Important => write!(f, \"$important\"),\n            Keyword::Phishing => write!(f, \"$phishing\"),\n            Keyword::Junk => write!(f, \"$junk\"),\n            Keyword::NotJunk => write!(f, \"$notjunk\"),\n            Keyword::Deleted => write!(f, \"$deleted\"),\n            Keyword::Forwarded => write!(f, \"$forwarded\"),\n            Keyword::MdnSent => write!(f, \"$mdnsent\"),\n            Keyword::Autosent => write!(f, \"$autosent\"),\n            Keyword::CanUnsubscribe => write!(f, \"$canunsubscribe\"),\n            Keyword::Followed => write!(f, \"$followed\"),\n            Keyword::HasAttachment => write!(f, \"$hasattachment\"),\n            Keyword::HasMemo => write!(f, \"$hasmemo\"),\n            Keyword::HasNoAttachment => write!(f, \"$hasnoattachment\"),\n            Keyword::Imported => write!(f, \"$imported\"),\n            Keyword::IsTrusted => write!(f, \"$istrusted\"),\n            Keyword::MailFlagBit0 => write!(f, \"$MailFlagBit0\"),\n            Keyword::MailFlagBit1 => write!(f, \"$MailFlagBit1\"),\n            Keyword::MailFlagBit2 => write!(f, \"$MailFlagBit2\"),\n            Keyword::MaskedEmail => write!(f, \"$maskedemail\"),\n            Keyword::Memo => write!(f, \"$memo\"),\n            Keyword::Muted => write!(f, \"$muted\"),\n            Keyword::New => write!(f, \"$new\"),\n            Keyword::Notify => write!(f, \"$notify\"),\n            Keyword::Unsubscribed => write!(f, \"$unsubscribed\"),\n            Keyword::Other(s) => write!(f, \"{}\", s),\n        }\n    }\n}\n\nimpl Display for ArchivedKeyword {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            ArchivedKeyword::Seen => write!(f, \"$seen\"),\n            ArchivedKeyword::Draft => write!(f, \"$draft\"),\n            ArchivedKeyword::Flagged => write!(f, \"$flagged\"),\n            ArchivedKeyword::Answered => write!(f, \"$answered\"),\n            ArchivedKeyword::Recent => write!(f, \"$recent\"),\n            ArchivedKeyword::Important => write!(f, \"$important\"),\n            ArchivedKeyword::Phishing => write!(f, \"$phishing\"),\n            ArchivedKeyword::Junk => write!(f, \"$junk\"),\n            ArchivedKeyword::NotJunk => write!(f, \"$notjunk\"),\n            ArchivedKeyword::Deleted => write!(f, \"$deleted\"),\n            ArchivedKeyword::Forwarded => write!(f, \"$forwarded\"),\n            ArchivedKeyword::MdnSent => write!(f, \"$mdnsent\"),\n            ArchivedKeyword::Autosent => write!(f, \"$autosent\"),\n            ArchivedKeyword::CanUnsubscribe => write!(f, \"$canunsubscribe\"),\n            ArchivedKeyword::Followed => write!(f, \"$followed\"),\n            ArchivedKeyword::HasAttachment => write!(f, \"$hasattachment\"),\n            ArchivedKeyword::HasMemo => write!(f, \"$hasmemo\"),\n            ArchivedKeyword::HasNoAttachment => write!(f, \"$hasnoattachment\"),\n            ArchivedKeyword::Imported => write!(f, \"$imported\"),\n            ArchivedKeyword::IsTrusted => write!(f, \"$istrusted\"),\n            ArchivedKeyword::MailFlagBit0 => write!(f, \"$MailFlagBit0\"),\n            ArchivedKeyword::MailFlagBit1 => write!(f, \"$MailFlagBit1\"),\n            ArchivedKeyword::MailFlagBit2 => write!(f, \"$MailFlagBit2\"),\n            ArchivedKeyword::MaskedEmail => write!(f, \"$maskedemail\"),\n            ArchivedKeyword::Memo => write!(f, \"$memo\"),\n            ArchivedKeyword::Muted => write!(f, \"$muted\"),\n            ArchivedKeyword::New => write!(f, \"$new\"),\n            ArchivedKeyword::Notify => write!(f, \"$notify\"),\n            ArchivedKeyword::Unsubscribed => write!(f, \"$unsubscribed\"),\n            ArchivedKeyword::Other(s) => write!(f, \"{}\", s),\n        }\n    }\n}\n\nimpl From<Keyword> for Vec<u8> {\n    fn from(keyword: Keyword) -> Self {\n        match keyword {\n            Keyword::Seen => vec![SEEN as u8],\n            Keyword::Draft => vec![DRAFT as u8],\n            Keyword::Flagged => vec![FLAGGED as u8],\n            Keyword::Answered => vec![ANSWERED as u8],\n            Keyword::Recent => vec![RECENT as u8],\n            Keyword::Important => vec![IMPORTANT as u8],\n            Keyword::Phishing => vec![PHISHING as u8],\n            Keyword::Junk => vec![JUNK as u8],\n            Keyword::NotJunk => vec![NOTJUNK as u8],\n            Keyword::Deleted => vec![DELETED as u8],\n            Keyword::Forwarded => vec![FORWARDED as u8],\n            Keyword::MdnSent => vec![MDN_SENT as u8],\n            Keyword::Autosent => vec![AUTOSENT as u8],\n            Keyword::CanUnsubscribe => vec![CANUNSUBSCRIBE as u8],\n            Keyword::Followed => vec![FOLLOWED as u8],\n            Keyword::HasAttachment => vec![HASATTACHMENT as u8],\n            Keyword::HasMemo => vec![HASMEMO as u8],\n            Keyword::HasNoAttachment => vec![HASNOATTACHMENT as u8],\n            Keyword::Imported => vec![IMPORTED as u8],\n            Keyword::IsTrusted => vec![ISTRUSTED as u8],\n            Keyword::MailFlagBit0 => vec![MAILFLAGBIT0 as u8],\n            Keyword::MailFlagBit1 => vec![MAILFLAGBIT1 as u8],\n            Keyword::MailFlagBit2 => vec![MAILFLAGBIT2 as u8],\n            Keyword::MaskedEmail => vec![MASKEDEMAIL as u8],\n            Keyword::Memo => vec![MEMO as u8],\n            Keyword::Muted => vec![MUTED as u8],\n            Keyword::New => vec![NEW as u8],\n            Keyword::Notify => vec![NOTIFY as u8],\n            Keyword::Unsubscribed => vec![UNSUBSCRIBED as u8],\n            Keyword::Other(string) => string.as_bytes().to_vec(),\n        }\n    }\n}\n\nimpl FromStr for Keyword {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        Ok(Keyword::parse(s))\n    }\n}\n\nimpl ArchivedKeyword {\n    pub fn id(&self) -> Result<u32, &str> {\n        match self {\n            ArchivedKeyword::Seen => Ok(SEEN as u32),\n            ArchivedKeyword::Draft => Ok(DRAFT as u32),\n            ArchivedKeyword::Flagged => Ok(FLAGGED as u32),\n            ArchivedKeyword::Answered => Ok(ANSWERED as u32),\n            ArchivedKeyword::Recent => Ok(RECENT as u32),\n            ArchivedKeyword::Important => Ok(IMPORTANT as u32),\n            ArchivedKeyword::Phishing => Ok(PHISHING as u32),\n            ArchivedKeyword::Junk => Ok(JUNK as u32),\n            ArchivedKeyword::NotJunk => Ok(NOTJUNK as u32),\n            ArchivedKeyword::Deleted => Ok(DELETED as u32),\n            ArchivedKeyword::Forwarded => Ok(FORWARDED as u32),\n            ArchivedKeyword::MdnSent => Ok(MDN_SENT as u32),\n            ArchivedKeyword::Autosent => Ok(AUTOSENT as u32),\n            ArchivedKeyword::CanUnsubscribe => Ok(CANUNSUBSCRIBE as u32),\n            ArchivedKeyword::Followed => Ok(FOLLOWED as u32),\n            ArchivedKeyword::HasAttachment => Ok(HASATTACHMENT as u32),\n            ArchivedKeyword::HasMemo => Ok(HASMEMO as u32),\n            ArchivedKeyword::HasNoAttachment => Ok(HASNOATTACHMENT as u32),\n            ArchivedKeyword::Imported => Ok(IMPORTED as u32),\n            ArchivedKeyword::IsTrusted => Ok(ISTRUSTED as u32),\n            ArchivedKeyword::MailFlagBit0 => Ok(MAILFLAGBIT0 as u32),\n            ArchivedKeyword::MailFlagBit1 => Ok(MAILFLAGBIT1 as u32),\n            ArchivedKeyword::MailFlagBit2 => Ok(MAILFLAGBIT2 as u32),\n            ArchivedKeyword::MaskedEmail => Ok(MASKEDEMAIL as u32),\n            ArchivedKeyword::Memo => Ok(MEMO as u32),\n            ArchivedKeyword::Muted => Ok(MUTED as u32),\n            ArchivedKeyword::New => Ok(NEW as u32),\n            ArchivedKeyword::Notify => Ok(NOTIFY as u32),\n            ArchivedKeyword::Unsubscribed => Ok(UNSUBSCRIBED as u32),\n            ArchivedKeyword::Other(string) => Err(string.as_ref()),\n        }\n    }\n\n    pub fn to_native(&self) -> Keyword {\n        match self {\n            ArchivedKeyword::Seen => Keyword::Seen,\n            ArchivedKeyword::Draft => Keyword::Draft,\n            ArchivedKeyword::Flagged => Keyword::Flagged,\n            ArchivedKeyword::Answered => Keyword::Answered,\n            ArchivedKeyword::Recent => Keyword::Recent,\n            ArchivedKeyword::Important => Keyword::Important,\n            ArchivedKeyword::Phishing => Keyword::Phishing,\n            ArchivedKeyword::Junk => Keyword::Junk,\n            ArchivedKeyword::NotJunk => Keyword::NotJunk,\n            ArchivedKeyword::Deleted => Keyword::Deleted,\n            ArchivedKeyword::Forwarded => Keyword::Forwarded,\n            ArchivedKeyword::MdnSent => Keyword::MdnSent,\n            ArchivedKeyword::Autosent => Keyword::Autosent,\n            ArchivedKeyword::CanUnsubscribe => Keyword::CanUnsubscribe,\n            ArchivedKeyword::Followed => Keyword::Followed,\n            ArchivedKeyword::HasAttachment => Keyword::HasAttachment,\n            ArchivedKeyword::HasMemo => Keyword::HasMemo,\n            ArchivedKeyword::HasNoAttachment => Keyword::HasNoAttachment,\n            ArchivedKeyword::Imported => Keyword::Imported,\n            ArchivedKeyword::IsTrusted => Keyword::IsTrusted,\n            ArchivedKeyword::MailFlagBit0 => Keyword::MailFlagBit0,\n            ArchivedKeyword::MailFlagBit1 => Keyword::MailFlagBit1,\n            ArchivedKeyword::MailFlagBit2 => Keyword::MailFlagBit2,\n            ArchivedKeyword::MaskedEmail => Keyword::MaskedEmail,\n            ArchivedKeyword::Memo => Keyword::Memo,\n            ArchivedKeyword::Muted => Keyword::Muted,\n            ArchivedKeyword::New => Keyword::New,\n            ArchivedKeyword::Notify => Keyword::Notify,\n            ArchivedKeyword::Unsubscribed => Keyword::Unsubscribed,\n            ArchivedKeyword::Other(other) => Keyword::Other(other.as_ref().into()),\n        }\n    }\n}\n\nimpl From<&ArchivedKeyword> for Keyword {\n    fn from(value: &ArchivedKeyword) -> Self {\n        match value {\n            ArchivedKeyword::Seen => Keyword::Seen,\n            ArchivedKeyword::Draft => Keyword::Draft,\n            ArchivedKeyword::Flagged => Keyword::Flagged,\n            ArchivedKeyword::Answered => Keyword::Answered,\n            ArchivedKeyword::Recent => Keyword::Recent,\n            ArchivedKeyword::Important => Keyword::Important,\n            ArchivedKeyword::Phishing => Keyword::Phishing,\n            ArchivedKeyword::Junk => Keyword::Junk,\n            ArchivedKeyword::NotJunk => Keyword::NotJunk,\n            ArchivedKeyword::Deleted => Keyword::Deleted,\n            ArchivedKeyword::Forwarded => Keyword::Forwarded,\n            ArchivedKeyword::MdnSent => Keyword::MdnSent,\n            ArchivedKeyword::Autosent => Keyword::Autosent,\n            ArchivedKeyword::CanUnsubscribe => Keyword::CanUnsubscribe,\n            ArchivedKeyword::Followed => Keyword::Followed,\n            ArchivedKeyword::HasAttachment => Keyword::HasAttachment,\n            ArchivedKeyword::HasMemo => Keyword::HasMemo,\n            ArchivedKeyword::HasNoAttachment => Keyword::HasNoAttachment,\n            ArchivedKeyword::Imported => Keyword::Imported,\n            ArchivedKeyword::IsTrusted => Keyword::IsTrusted,\n            ArchivedKeyword::MailFlagBit0 => Keyword::MailFlagBit0,\n            ArchivedKeyword::MailFlagBit1 => Keyword::MailFlagBit1,\n            ArchivedKeyword::MailFlagBit2 => Keyword::MailFlagBit2,\n            ArchivedKeyword::MaskedEmail => Keyword::MaskedEmail,\n            ArchivedKeyword::Memo => Keyword::Memo,\n            ArchivedKeyword::Muted => Keyword::Muted,\n            ArchivedKeyword::New => Keyword::New,\n            ArchivedKeyword::Notify => Keyword::Notify,\n            ArchivedKeyword::Unsubscribed => Keyword::Unsubscribed,\n            ArchivedKeyword::Other(string) => Keyword::Other(string.as_ref().into()),\n        }\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for Keyword {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        Ok(Keyword::parse(<&str>::deserialize(deserializer)?))\n    }\n}\n\nimpl<'x, P: Property, E: Element + From<Keyword>> From<Keyword> for Value<'x, P, E> {\n    fn from(id: Keyword) -> Self {\n        Value::Element(E::from(id))\n    }\n}\n"
  },
  {
    "path": "crates/types/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod blob;\npub mod blob_hash;\npub mod collection;\npub mod dead_property;\npub mod field;\npub mod id;\npub mod keyword;\npub mod semver;\npub mod special_use;\npub mod type_state;\n\npub type DocumentId = u32;\npub type ChangeId = u64;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\n#[cfg_attr(feature = \"test_mode\", derive(serde::Serialize, serde::Deserialize))]\npub struct TimeRange {\n    pub start: i64,\n    pub end: i64,\n}\n\nimpl TimeRange {\n    pub fn new(start: i64, end: i64) -> Self {\n        Self { start, end }\n    }\n\n    pub fn is_in_range(&self, match_overlap: bool, start: i64, end: i64) -> bool {\n        if !match_overlap {\n            // RFC4791#9.9: (start <  DTEND AND end > DTSTART)\n            self.start < end && self.end > start\n        } else {\n            // RFC4791#9.9: ((start <  DUE) OR (start <= DTSTART)) AND ((end > DTSTART) OR (end >= DUE))\n            ((start < self.end) || (start <= self.start)) && (end > self.start || end >= self.end)\n        }\n    }\n}\n\nimpl Default for TimeRange {\n    fn default() -> Self {\n        Self {\n            start: i64::MIN,\n            end: i64::MAX,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/types/src/semver.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Display;\n\n#[derive(Clone, Debug, Default, PartialEq, Eq, Hash, PartialOrd, Ord)]\n#[repr(transparent)]\npub struct Semver(u64);\n\nimpl Semver {\n    pub fn current() -> Self {\n        env!(\"CARGO_PKG_VERSION\").try_into().unwrap()\n    }\n\n    pub fn new(major: u16, minor: u16, patch: u16) -> Self {\n        let mut version: u64 = 0;\n        version |= (major as u64) << 32;\n        version |= (minor as u64) << 16;\n        version |= patch as u64;\n        Semver(version)\n    }\n\n    pub fn unpack(&self) -> (u16, u16, u16) {\n        let version = self.0;\n        let major = ((version >> 32) & 0xFFFF) as u16;\n        let minor = ((version >> 16) & 0xFFFF) as u16;\n        let patch = (version & 0xFFFF) as u16;\n        (major, minor, patch)\n    }\n\n    pub fn major(&self) -> u16 {\n        (self.0 >> 32) as u16\n    }\n\n    pub fn minor(&self) -> u16 {\n        (self.0 >> 16) as u16\n    }\n\n    pub fn patch(&self) -> u16 {\n        self.0 as u16\n    }\n\n    pub fn is_valid(&self) -> bool {\n        self.0 > 0\n    }\n}\n\nimpl AsRef<u64> for Semver {\n    fn as_ref(&self) -> &u64 {\n        &self.0\n    }\n}\n\nimpl From<u64> for Semver {\n    fn from(value: u64) -> Self {\n        Semver(value)\n    }\n}\n\nimpl TryFrom<&str> for Semver {\n    type Error = ();\n\n    fn try_from(value: &str) -> Result<Self, Self::Error> {\n        let mut parts = value.splitn(3, '.');\n        let major = parts.next().ok_or(())?.parse().map_err(|_| ())?;\n        let minor = parts.next().ok_or(())?.parse().map_err(|_| ())?;\n        let patch = parts.next().ok_or(())?.parse().map_err(|_| ())?;\n        Ok(Semver::new(major, minor, patch))\n    }\n}\n\nimpl Display for Semver {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let (major, minor, patch) = self.unpack();\n        write!(f, \"{major}.{minor}.{patch}\")\n    }\n}\n"
  },
  {
    "path": "crates/types/src/special_use.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_tools::{Element, Property, Value};\nuse utils::config::utils::ParseValue;\n\n#[derive(\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    Clone,\n    Copy,\n    PartialEq,\n    Eq,\n    Hash,\n    Debug,\n    PartialOrd,\n    Ord,\n)]\n#[rkyv(derive(Debug))]\npub enum SpecialUse {\n    Inbox,\n    Trash,\n    Junk,\n    Drafts,\n    Archive,\n    Sent,\n    Shared,\n    Important,\n    None,\n    Memos,\n    Scheduled,\n    Snoozed,\n}\n\nimpl SpecialUse {\n    pub fn parse(s: &str) -> Option<Self> {\n        hashify::tiny_map_ignore_case!(s.as_bytes(),\n            b\"inbox\" => SpecialUse::Inbox,\n            b\"trash\" => SpecialUse::Trash,\n            b\"junk\" => SpecialUse::Junk,\n            b\"drafts\" => SpecialUse::Drafts,\n            b\"archive\" => SpecialUse::Archive,\n            b\"sent\" => SpecialUse::Sent,\n            b\"shared\" => SpecialUse::Shared,\n            b\"important\" => SpecialUse::Important,\n            b\"memos\" => SpecialUse::Memos,\n            b\"scheduled\" => SpecialUse::Scheduled,\n            b\"snoozed\" => SpecialUse::Snoozed,\n        )\n    }\n\n    pub fn as_str(&self) -> Option<&'static str> {\n        match self {\n            SpecialUse::Inbox => Some(\"inbox\"),\n            SpecialUse::Trash => Some(\"trash\"),\n            SpecialUse::Junk => Some(\"junk\"),\n            SpecialUse::Drafts => Some(\"drafts\"),\n            SpecialUse::Archive => Some(\"archive\"),\n            SpecialUse::Sent => Some(\"sent\"),\n            SpecialUse::Shared => Some(\"shared\"),\n            SpecialUse::Important => Some(\"important\"),\n            SpecialUse::Memos => Some(\"memos\"),\n            SpecialUse::Scheduled => Some(\"scheduled\"),\n            SpecialUse::Snoozed => Some(\"snoozed\"),\n            SpecialUse::None => None,\n        }\n    }\n}\n\nimpl ArchivedSpecialUse {\n    pub fn as_str(&self) -> Option<&'static str> {\n        match self {\n            ArchivedSpecialUse::Inbox => Some(\"inbox\"),\n            ArchivedSpecialUse::Trash => Some(\"trash\"),\n            ArchivedSpecialUse::Junk => Some(\"junk\"),\n            ArchivedSpecialUse::Drafts => Some(\"drafts\"),\n            ArchivedSpecialUse::Archive => Some(\"archive\"),\n            ArchivedSpecialUse::Sent => Some(\"sent\"),\n            ArchivedSpecialUse::Shared => Some(\"shared\"),\n            ArchivedSpecialUse::Important => Some(\"important\"),\n            ArchivedSpecialUse::Memos => Some(\"memos\"),\n            ArchivedSpecialUse::Scheduled => Some(\"scheduled\"),\n            ArchivedSpecialUse::Snoozed => Some(\"snoozed\"),\n            ArchivedSpecialUse::None => None,\n        }\n    }\n}\n\nimpl From<&ArchivedSpecialUse> for SpecialUse {\n    fn from(value: &ArchivedSpecialUse) -> Self {\n        match value {\n            ArchivedSpecialUse::Inbox => SpecialUse::Inbox,\n            ArchivedSpecialUse::Trash => SpecialUse::Trash,\n            ArchivedSpecialUse::Junk => SpecialUse::Junk,\n            ArchivedSpecialUse::Drafts => SpecialUse::Drafts,\n            ArchivedSpecialUse::Archive => SpecialUse::Archive,\n            ArchivedSpecialUse::Sent => SpecialUse::Sent,\n            ArchivedSpecialUse::Shared => SpecialUse::Shared,\n            ArchivedSpecialUse::Important => SpecialUse::Important,\n            ArchivedSpecialUse::Memos => SpecialUse::Memos,\n            ArchivedSpecialUse::Scheduled => SpecialUse::Scheduled,\n            ArchivedSpecialUse::Snoozed => SpecialUse::Snoozed,\n            ArchivedSpecialUse::None => SpecialUse::None,\n        }\n    }\n}\n\nimpl ParseValue for SpecialUse {\n    fn parse_value(value: &str) -> Result<Self, String> {\n        SpecialUse::parse(value).ok_or_else(|| format!(\"Unknown folder role {:?}\", value))\n    }\n}\n\nimpl<'x, P: Property, E: Element + From<SpecialUse>> From<SpecialUse> for Value<'x, P, E> {\n    fn from(id: SpecialUse) -> Self {\n        Value::Element(E::from(id))\n    }\n}\n"
  },
  {
    "path": "crates/types/src/type_state.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::collection::SyncCollection;\nuse jmap_tools::{Element, Property, Value};\nuse serde::Serialize;\nuse std::{fmt::Display, str::FromStr};\nuse utils::map::bitmap::{Bitmap, BitmapItem};\n\n#[derive(Debug, Eq, PartialEq, Hash, Clone, Copy, Serialize, PartialOrd, Ord)]\n#[repr(u8)]\npub enum DataType {\n    #[serde(rename = \"Email\")]\n    Email = 0,\n    #[serde(rename = \"EmailDelivery\")]\n    EmailDelivery = 1,\n    #[serde(rename = \"EmailSubmission\")]\n    EmailSubmission = 2,\n    #[serde(rename = \"Mailbox\")]\n    Mailbox = 3,\n    #[serde(rename = \"Thread\")]\n    Thread = 4,\n    #[serde(rename = \"Identity\")]\n    Identity = 5,\n    #[serde(rename = \"Core\")]\n    Core = 6,\n    #[serde(rename = \"PushSubscription\")]\n    PushSubscription = 7,\n    #[serde(rename = \"SearchSnippet\")]\n    SearchSnippet = 8,\n    #[serde(rename = \"VacationResponse\")]\n    VacationResponse = 9,\n    #[serde(rename = \"MDN\")]\n    Mdn = 10,\n    #[serde(rename = \"Quota\")]\n    Quota = 11,\n    #[serde(rename = \"SieveScript\")]\n    SieveScript = 12,\n    #[serde(rename = \"Calendar\")]\n    Calendar = 13,\n    #[serde(rename = \"CalendarEvent\")]\n    CalendarEvent = 14,\n    #[serde(rename = \"CalendarEventNotification\")]\n    CalendarEventNotification = 15,\n    #[serde(rename = \"AddressBook\")]\n    AddressBook = 16,\n    #[serde(rename = \"ContactCard\")]\n    ContactCard = 17,\n    #[serde(rename = \"FileNode\")]\n    FileNode = 18,\n    #[serde(rename = \"Principal\")]\n    Principal = 19,\n    #[serde(rename = \"ShareNotification\")]\n    ShareNotification = 20,\n    #[serde(rename = \"ParticipantIdentity\")]\n    ParticipantIdentity = 21,\n    #[serde(rename = \"CalendarAlert\")]\n    CalendarAlert = 22,\n    None = 23,\n}\n\n#[derive(Debug, Clone, Copy)]\npub struct StateChange {\n    pub account_id: u32,\n    pub change_id: u64,\n    pub types: Bitmap<DataType>,\n}\n\nimpl StateChange {\n    pub fn new(account_id: u32) -> Self {\n        Self {\n            account_id,\n            change_id: 0,\n            types: Default::default(),\n        }\n    }\n\n    pub fn set_change(&mut self, type_state: DataType) {\n        self.types.insert(type_state);\n    }\n\n    pub fn with_change(mut self, type_state: DataType) -> Self {\n        self.set_change(type_state);\n        self\n    }\n\n    pub fn with_change_id(mut self, change_id: u64) -> Self {\n        self.change_id = change_id;\n        self\n    }\n\n    pub fn has_changes(&self) -> bool {\n        !self.types.is_empty()\n    }\n}\n\nimpl BitmapItem for DataType {\n    fn max() -> u64 {\n        DataType::None as u64\n    }\n\n    fn is_valid(&self) -> bool {\n        !matches!(self, DataType::None)\n    }\n}\n\nimpl From<u64> for DataType {\n    fn from(value: u64) -> Self {\n        match value {\n            0 => DataType::Email,\n            1 => DataType::EmailDelivery,\n            2 => DataType::EmailSubmission,\n            3 => DataType::Mailbox,\n            4 => DataType::Thread,\n            5 => DataType::Identity,\n            6 => DataType::Core,\n            7 => DataType::PushSubscription,\n            8 => DataType::SearchSnippet,\n            9 => DataType::VacationResponse,\n            10 => DataType::Mdn,\n            11 => DataType::Quota,\n            12 => DataType::SieveScript,\n            13 => DataType::Calendar,\n            14 => DataType::CalendarEvent,\n            15 => DataType::CalendarEventNotification,\n            16 => DataType::AddressBook,\n            17 => DataType::ContactCard,\n            18 => DataType::FileNode,\n            19 => DataType::Principal,\n            20 => DataType::ShareNotification,\n            21 => DataType::ParticipantIdentity,\n            22 => DataType::CalendarAlert,\n            _ => {\n                debug_assert!(false, \"Invalid type_state value: {}\", value);\n                DataType::None\n            }\n        }\n    }\n}\n\nimpl From<DataType> for u64 {\n    fn from(type_state: DataType) -> u64 {\n        type_state as u64\n    }\n}\n\nimpl DataType {\n    pub fn try_from_sync(value: SyncCollection, is_container: bool) -> Option<Self> {\n        match (value, is_container) {\n            (SyncCollection::Email, false) => DataType::Email.into(),\n            (SyncCollection::Email, true) => DataType::Mailbox.into(),\n            (SyncCollection::Thread, _) => DataType::Thread.into(),\n            (SyncCollection::Calendar, true) => DataType::Calendar.into(),\n            (SyncCollection::Calendar, false) => DataType::CalendarEvent.into(),\n            (SyncCollection::AddressBook, true) => DataType::AddressBook.into(),\n            (SyncCollection::AddressBook, false) => DataType::ContactCard.into(),\n            (SyncCollection::FileNode, _) => DataType::FileNode.into(),\n            (SyncCollection::Identity, _) => DataType::Identity.into(),\n            (SyncCollection::EmailSubmission, _) => DataType::EmailSubmission.into(),\n            (SyncCollection::SieveScript, _) => DataType::SieveScript.into(),\n            _ => None,\n        }\n    }\n}\n\nimpl DataType {\n    pub fn parse(value: &str) -> Option<Self> {\n        hashify::tiny_map!(value.as_bytes(),\n            b\"Email\" => DataType::Email,\n            b\"EmailDelivery\" => DataType::EmailDelivery,\n            b\"EmailSubmission\" => DataType::EmailSubmission,\n            b\"Mailbox\" => DataType::Mailbox,\n            b\"Thread\" => DataType::Thread,\n            b\"Identity\" => DataType::Identity,\n            b\"Core\" => DataType::Core,\n            b\"PushSubscription\" => DataType::PushSubscription,\n            b\"SearchSnippet\" => DataType::SearchSnippet,\n            b\"VacationResponse\" => DataType::VacationResponse,\n            b\"MDN\" => DataType::Mdn,\n            b\"Quota\" => DataType::Quota,\n            b\"SieveScript\" => DataType::SieveScript,\n            b\"Calendar\" => DataType::Calendar,\n            b\"CalendarEvent\" => DataType::CalendarEvent,\n            b\"CalendarEventNotification\" => DataType::CalendarEventNotification,\n            b\"AddressBook\" => DataType::AddressBook,\n            b\"ContactCard\" => DataType::ContactCard,\n            b\"FileNode\" => DataType::FileNode,\n            b\"Principal\" => DataType::Principal,\n            b\"ShareNotification\" => DataType::ShareNotification,\n            b\"ParticipantIdentity\" => DataType::ParticipantIdentity,\n            b\"CalendarAlert\" => DataType::CalendarAlert,\n        )\n    }\n\n    pub fn as_str(&self) -> &'static str {\n        match self {\n            DataType::Email => \"Email\",\n            DataType::EmailDelivery => \"EmailDelivery\",\n            DataType::EmailSubmission => \"EmailSubmission\",\n            DataType::Mailbox => \"Mailbox\",\n            DataType::Thread => \"Thread\",\n            DataType::Identity => \"Identity\",\n            DataType::Core => \"Core\",\n            DataType::PushSubscription => \"PushSubscription\",\n            DataType::SearchSnippet => \"SearchSnippet\",\n            DataType::VacationResponse => \"VacationResponse\",\n            DataType::Mdn => \"MDN\",\n            DataType::Quota => \"Quota\",\n            DataType::SieveScript => \"SieveScript\",\n            DataType::Calendar => \"Calendar\",\n            DataType::CalendarEvent => \"CalendarEvent\",\n            DataType::CalendarEventNotification => \"CalendarEventNotification\",\n            DataType::AddressBook => \"AddressBook\",\n            DataType::ContactCard => \"ContactCard\",\n            DataType::FileNode => \"FileNode\",\n            DataType::Principal => \"Principal\",\n            DataType::ShareNotification => \"ShareNotification\",\n            DataType::ParticipantIdentity => \"ParticipantIdentity\",\n            DataType::CalendarAlert => \"CalendarAlert\",\n            DataType::None => \"\",\n        }\n    }\n}\n\nimpl FromStr for DataType {\n    type Err = ();\n\n    fn from_str(s: &str) -> Result<Self, Self::Err> {\n        DataType::parse(s).ok_or(())\n    }\n}\n\nimpl Display for DataType {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        write!(f, \"{}\", self.as_str())\n    }\n}\n\nimpl<'de> serde::Deserialize<'de> for DataType {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        DataType::parse(<&str>::deserialize(deserializer)?)\n            .ok_or_else(|| serde::de::Error::custom(\"invalid JMAP data type\"))\n    }\n}\n\nimpl<'x, P: Property, E: Element + From<DataType>> From<DataType> for Value<'x, P, E> {\n    fn from(id: DataType) -> Self {\n        Value::Element(E::from(id))\n    }\n}\n"
  },
  {
    "path": "crates/utils/Cargo.toml",
    "content": "[package]\nname = \"utils\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[dependencies]\ntrc = { path = \"../trc\" }\nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pemfile = \"2.0\"\nrustls-pki-types = { version = \"1\" }\ntokio = { version = \"1.47\", features = [\"net\", \"macros\", \"signal\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\nserde = { version = \"1.0\", features = [\"derive\"]}\nmail-auth = { version = \"0.7.1\" }\nsmtp-proto = { version = \"0.2\" }\nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nahash = { version = \"0.8\", features = [\"serde\"] }\nchrono = \"0.4\"\nrand = \"0.9.0\"\nwebpki-roots = { version = \"1.0\"}\nring = { version = \"0.17\" }\nbase64 = \"0.22\"\nserde_json = \"1.0\"\nrcgen = \"0.14\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"http2\", \"stream\"]}\nx509-parser = \"0.18\"\npem = \"3.0\"\nparking_lot = \"0.12\"\nfutures = \"0.3\"\nregex = \"1.7.0\"\nblake3 = \"1.3.3\"\nhttp-body-util = \"0.1.0\"\nform_urlencoded = \"1.1.0\"\npsl = \"2\"\nquick_cache = \"0.6.9\"\nfast-float = \"0.2.0\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\nxxhash-rust = { version = \"0.8.5\", features = [\"xxh3\"] }\nfarmhash = \"1.1.5\"\nnohash-hasher = \"0.2.0\"\n\n[target.'cfg(unix)'.dependencies]\nprivdrop = \"0.5.3\"\n\n[features]\ntest_mode = []\n\n[dev-dependencies]\ntokio = { version = \"1.47\", features = [\"full\"] }\n"
  },
  {
    "path": "crates/utils/proc-macros/Cargo.toml",
    "content": "[package]\nname = \"proc_macros\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[lib]\nproc-macro = true\n\n[dependencies]\nsyn = { version = \"2.0\", features = [\"full\"] }\nquote = \"1.0\"\nproc-macro2 = \"1.0\"\n"
  },
  {
    "path": "crates/utils/proc-macros/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse proc_macro::TokenStream;\nuse quote::quote;\nuse syn::{Data, DeriveInput, parse_macro_input};\n\n#[proc_macro_derive(EnumMethods)]\npub fn enum_id(input: TokenStream) -> TokenStream {\n    let input = parse_macro_input!(input as DeriveInput);\n    let name = &input.ident;\n\n    let variants = match input.data {\n        Data::Enum(ref data) => &data.variants,\n        _ => panic!(\"EnumMethods only works on enums\"),\n    };\n\n    let variant_count = variants.len();\n    let variant_names: Vec<_> = variants.iter().map(|v| &v.ident).collect();\n    let variant_ids: Vec<u32> = (0..(variant_count as u32)).collect();\n    let snake_case_names: Vec<String> = variant_names\n        .iter()\n        .map(|name| to_snake_case(&name.to_string()))\n        .collect();\n\n    let expanded = quote! {\n        impl #name {\n            pub const COUNT: usize = #variant_count;\n\n            pub const fn id(&self) -> u32 {\n                match self {\n                    #(#name::#variant_names => #variant_ids,)*\n                }\n            }\n\n            pub fn from_id(id: u32) -> Option<Self> {\n                match id {\n                    #(#variant_ids => Some(#name::#variant_names),)*\n                    _ => None,\n                }\n            }\n\n            pub fn name(&self) -> &'static str {\n                match self {\n                    #(#name::#variant_names => #snake_case_names,)*\n                }\n            }\n\n            pub fn from_name(name: &str) -> Option<Self> {\n                match name {\n                    #(#snake_case_names => Some(#name::#variant_names),)*\n                    _ => None,\n                }\n            }\n        }\n    };\n\n    TokenStream::from(expanded)\n}\n\nfn to_snake_case(s: &str) -> String {\n    let mut result = String::new();\n    for (i, ch) in s.char_indices() {\n        if ch.is_uppercase() {\n            if i > 0 {\n                result.push('-');\n            }\n            result.push(ch.to_ascii_lowercase());\n        } else {\n            result.push(ch);\n        }\n    }\n    result\n}\n"
  },
  {
    "path": "crates/utils/src/bimap.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Borrow, hash::Hash, rc::Rc};\n\nuse ahash::AHashMap;\n\n#[derive(Debug)]\n#[repr(transparent)]\nstruct StringRef<T: IdBimapItem>(Rc<T>);\n\n#[derive(Debug)]\n#[repr(transparent)]\nstruct IdRef<T: IdBimapItem>(Rc<T>);\n\n#[derive(Debug, Default)]\npub struct IdBimap<T: IdBimapItem> {\n    id_to_name: AHashMap<IdRef<T>, Rc<T>>,\n    name_to_id: AHashMap<StringRef<T>, Rc<T>>,\n}\n\nimpl<T: IdBimapItem> IdBimap<T> {\n    pub fn with_capacity(capacity: usize) -> Self {\n        Self {\n            id_to_name: AHashMap::with_capacity(capacity),\n            name_to_id: AHashMap::with_capacity(capacity),\n        }\n    }\n\n    pub fn insert(&mut self, item: T) {\n        let item = Rc::new(item);\n        self.id_to_name.insert(IdRef(item.clone()), item.clone());\n        self.name_to_id.insert(StringRef(item.clone()), item);\n    }\n\n    pub fn by_name(&self, name: &str) -> Option<&T> {\n        self.name_to_id.get(name).map(|v| v.as_ref())\n    }\n\n    pub fn by_id(&self, id: u32) -> Option<&T> {\n        self.id_to_name.get(&id).map(|v| v.as_ref())\n    }\n\n    pub fn iter(&self) -> impl Iterator<Item = &T> {\n        self.name_to_id.values().map(|v| v.as_ref())\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.name_to_id.is_empty()\n    }\n}\n\n// SAFETY: Safe because Rc<> are never returned from the struct\nunsafe impl<T: IdBimapItem> Send for IdBimap<T> {}\nunsafe impl<T: IdBimapItem> Sync for IdBimap<T> {}\n\npub trait IdBimapItem: std::fmt::Debug {\n    fn id(&self) -> &u32;\n    fn name(&self) -> &str;\n}\n\nimpl<T: IdBimapItem> Borrow<str> for StringRef<T> {\n    fn borrow(&self) -> &str {\n        self.0.name()\n    }\n}\n\nimpl<T: IdBimapItem> Borrow<u32> for IdRef<T> {\n    fn borrow(&self) -> &u32 {\n        self.0.id()\n    }\n}\n\nimpl<T: IdBimapItem> PartialEq for StringRef<T> {\n    fn eq(&self, other: &Self) -> bool {\n        self.0.name() == other.0.name()\n    }\n}\n\nimpl<T: IdBimapItem> Eq for StringRef<T> {}\n\nimpl<T: IdBimapItem> PartialEq for IdRef<T> {\n    fn eq(&self, other: &Self) -> bool {\n        self.0.id() == other.0.id()\n    }\n}\n\nimpl<T: IdBimapItem> Eq for IdRef<T> {}\n\nimpl<T: IdBimapItem> Hash for StringRef<T> {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.0.name().hash(state)\n    }\n}\n\nimpl<T: IdBimapItem> Hash for IdRef<T> {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.0.id().hash(state)\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/cache.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::config::Config;\nuse mail_auth::{MX, ResolverCache, Txt};\nuse quick_cache::{\n    Equivalent, Weighter,\n    sync::{DefaultLifecycle, PlaceholderGuard},\n};\nuse std::{\n    borrow::Borrow,\n    hash::Hash,\n    net::{IpAddr, Ipv4Addr, Ipv6Addr},\n    sync::Arc,\n    time::{Duration, Instant},\n};\n\npub struct Cache<K: Eq + Hash + CacheItemWeight, V: Clone + CacheItemWeight>(\n    quick_cache::sync::Cache<K, V, CacheItemWeighter>,\n);\n\npub struct CacheWithTtl<K: Eq + Hash + CacheItemWeight, V: Clone + CacheItemWeight>(\n    quick_cache::sync::Cache<K, TtlEntry<V>, CacheItemWeighter>,\n);\n\n#[derive(Clone)]\npub struct TtlEntry<V: Clone + CacheItemWeight> {\n    value: V,\n    expires: Instant,\n}\n\nimpl<K: Eq + Hash + CacheItemWeight, V: Clone + CacheItemWeight> Cache<K, V> {\n    pub fn from_config(\n        config: &mut Config,\n        key: &str,\n        max_weight: u64,\n        estimated_weight: u64,\n    ) -> Self {\n        let weight_capacity = config\n            .property((\"cache\", key, \"size\"))\n            .unwrap_or(max_weight);\n        let estimated_items_capacity = config\n            .property((\"cache\", key, \"capacity\"))\n            .unwrap_or_else(|| weight_capacity as usize / estimated_weight as usize);\n\n        Self::new(estimated_items_capacity, weight_capacity)\n    }\n\n    pub fn new(estimated_items_capacity: usize, weight_capacity: u64) -> Self {\n        Self(quick_cache::sync::Cache::with_weighter(\n            estimated_items_capacity,\n            weight_capacity,\n            CacheItemWeighter,\n        ))\n    }\n\n    #[inline(always)]\n    pub fn get<Q>(&self, key: &Q) -> Option<V>\n    where\n        Q: Hash + Equivalent<K> + ?Sized,\n    {\n        self.0.get(key)\n    }\n\n    #[inline(always)]\n    pub async fn get_value_or_guard_async<'a, Q>(\n        &'a self,\n        key: &Q,\n    ) -> Result<\n        V,\n        PlaceholderGuard<'a, K, V, CacheItemWeighter, ahash::RandomState, DefaultLifecycle<K, V>>,\n    >\n    where\n        Q: Hash + Equivalent<K> + ToOwned<Owned = K> + ?Sized,\n    {\n        self.0.get_value_or_guard_async(key).await\n    }\n\n    #[inline(always)]\n    pub fn insert(&self, key: K, value: V) {\n        self.0.insert(key, value);\n    }\n\n    #[inline(always)]\n    pub fn remove<Q>(&self, key: &Q) -> Option<V>\n    where\n        Q: Hash + Equivalent<K> + ?Sized,\n    {\n        self.0.remove(key).map(|(_, v)| v)\n    }\n\n    #[inline(always)]\n    pub fn clear(&self) {\n        self.0.clear();\n    }\n}\n\nimpl<K: Eq + Hash + CacheItemWeight, V: Clone + CacheItemWeight> CacheWithTtl<K, V> {\n    pub fn from_config(\n        config: &mut Config,\n        key: &str,\n        max_weight: u64,\n        estimated_weight: u64,\n    ) -> Self {\n        let weight_capacity = config\n            .property((\"cache\", key, \"size\"))\n            .unwrap_or(max_weight);\n        let estimated_items_capacity = config\n            .property((\"cache\", key, \"capacity\"))\n            .unwrap_or_else(|| weight_capacity as usize / estimated_weight as usize);\n\n        Self::new(estimated_items_capacity, weight_capacity)\n    }\n\n    pub fn new(estimated_items_capacity: usize, weight_capacity: u64) -> Self {\n        Self(quick_cache::sync::Cache::with_weighter(\n            estimated_items_capacity,\n            weight_capacity,\n            CacheItemWeighter,\n        ))\n    }\n\n    #[inline(always)]\n    pub fn get<Q>(&self, key: &Q) -> Option<V>\n    where\n        Q: Hash + Equivalent<K> + ?Sized,\n    {\n        self.0.get(key).and_then(|v| {\n            if v.expires > Instant::now() {\n                Some(v.value)\n            } else {\n                self.0.remove(key);\n                None\n            }\n        })\n    }\n\n    #[inline(always)]\n    pub async fn get_value_or_guard_async<'a, Q>(\n        &'a self,\n        key: &Q,\n    ) -> Result<\n        V,\n        PlaceholderGuard<\n            'a,\n            K,\n            TtlEntry<V>,\n            CacheItemWeighter,\n            ahash::RandomState,\n            DefaultLifecycle<K, TtlEntry<V>>,\n        >,\n    >\n    where\n        Q: Hash + Equivalent<K> + ToOwned<Owned = K> + ?Sized,\n    {\n        match self.0.get_value_or_guard_async(key).await {\n            Ok(value) => {\n                if value.expires > Instant::now() {\n                    Ok(value.value)\n                } else {\n                    self.0.remove(key);\n                    self.0.get_value_or_guard_async(key).await.map(|v| v.value)\n                }\n            }\n            Err(err) => Err(err),\n        }\n    }\n\n    #[inline(always)]\n    pub fn insert(&self, key: K, value: V, expires: Duration) {\n        self.0.insert(key, TtlEntry::new(value, expires));\n    }\n\n    #[inline(always)]\n    pub fn insert_with_expiry(&self, key: K, value: V, expires: Instant) {\n        self.0.insert(key, TtlEntry::with_expiry(value, expires));\n    }\n\n    #[inline(always)]\n    pub fn remove<Q>(&self, key: &Q) -> Option<V>\n    where\n        Q: Hash + Equivalent<K> + ?Sized,\n    {\n        self.0.remove(key).map(|(_, v)| v.value)\n    }\n\n    #[inline(always)]\n    pub fn clear(&self) {\n        self.0.clear();\n    }\n}\n\n#[derive(Clone)]\npub struct CacheItemWeighter;\n\nimpl<K: CacheItemWeight, V: CacheItemWeight> Weighter<K, V> for CacheItemWeighter {\n    fn weight(&self, key: &K, val: &V) -> u64 {\n        key.weight() + val.weight()\n    }\n}\n\npub trait CacheItemWeight {\n    fn weight(&self) -> u64;\n}\n\nimpl<T: Clone + CacheItemWeight> CacheItemWeight for TtlEntry<T> {\n    fn weight(&self) -> u64 {\n        self.value.weight() + std::mem::size_of::<Instant>() as u64\n    }\n}\n\nimpl<T: Clone + CacheItemWeight> CacheItemWeight for Option<T> {\n    fn weight(&self) -> u64 {\n        match self {\n            Some(v) => v.weight(),\n            None => std::mem::size_of::<usize>() as u64,\n        }\n    }\n}\n\nimpl<T: CacheItemWeight> CacheItemWeight for Arc<T> {\n    fn weight(&self) -> u64 {\n        self.as_ref().weight()\n    }\n}\n\nimpl CacheItemWeight for u64 {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<u64>() as u64\n    }\n}\n\nimpl CacheItemWeight for String {\n    fn weight(&self) -> u64 {\n        self.len() as u64 + std::mem::size_of::<String>() as u64\n    }\n}\n\nimpl CacheItemWeight for u32 {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<u32>() as u64\n    }\n}\n\nimpl CacheItemWeight for Vec<IpAddr> {\n    fn weight(&self) -> u64 {\n        (self.len() * std::mem::size_of::<IpAddr>()) as u64\n            + std::mem::size_of::<Vec<IpAddr>>() as u64\n    }\n}\n\nimpl CacheItemWeight for Vec<Ipv4Addr> {\n    fn weight(&self) -> u64 {\n        (self.len() * std::mem::size_of::<Ipv4Addr>()) as u64\n            + std::mem::size_of::<Vec<Ipv4Addr>>() as u64\n    }\n}\n\nimpl CacheItemWeight for Vec<Ipv6Addr> {\n    fn weight(&self) -> u64 {\n        (self.len() * std::mem::size_of::<Ipv6Addr>()) as u64\n            + std::mem::size_of::<Vec<Ipv6Addr>>() as u64\n    }\n}\n\nimpl CacheItemWeight for Vec<MX> {\n    fn weight(&self) -> u64 {\n        self.iter()\n            .map(|mx| {\n                mx.exchanges\n                    .iter()\n                    .map(|e| e.len() + std::mem::size_of::<MX>())\n                    .sum::<usize>()\n            })\n            .sum::<usize>() as u64\n            + std::mem::size_of::<Vec<MX>>() as u64\n    }\n}\n\nimpl CacheItemWeight for Vec<String> {\n    fn weight(&self) -> u64 {\n        self.iter().map(|s| s.len()).sum::<usize>() as u64\n            + std::mem::size_of::<Vec<String>>() as u64\n    }\n}\n\nimpl CacheItemWeight for Txt {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<Txt>() as u64\n    }\n}\n\nimpl CacheItemWeight for IpAddr {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<IpAddr>() as u64\n    }\n}\n\nimpl CacheItemWeight for bool {\n    fn weight(&self) -> u64 {\n        std::mem::size_of::<bool>() as u64\n    }\n}\n\nimpl<T: Clone + CacheItemWeight> TtlEntry<T> {\n    pub fn new(value: T, expires: Duration) -> Self {\n        Self {\n            value,\n            expires: Instant::now() + expires,\n        }\n    }\n\n    pub fn with_expiry(value: T, expires: Instant) -> Self {\n        Self { value, expires }\n    }\n}\n\nimpl<K: Eq + Hash + CacheItemWeight, V: Clone + CacheItemWeight> ResolverCache<K, V>\n    for CacheWithTtl<K, V>\n{\n    fn get<Q>(&self, key: &Q) -> Option<V>\n    where\n        K: Borrow<Q>,\n        Q: Hash + Eq + ?Sized,\n    {\n        CacheWithTtl::get(self, key)\n    }\n\n    fn remove<Q>(&self, key: &Q) -> Option<V>\n    where\n        K: Borrow<Q>,\n        Q: Hash + Eq + ?Sized,\n    {\n        CacheWithTtl::remove(self, key)\n    }\n\n    fn insert(&self, key: K, value: V, expires: Instant) {\n        self.0.insert(key, TtlEntry::with_expiry(value, expires));\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/chained_bytes.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, ops::Range};\n\n#[derive(Debug, Clone)]\npub struct ChainedBytes<'x> {\n    first: &'x [u8],\n    last: &'x [u8],\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum SliceRange<'x> {\n    Single(&'x [u8]),\n    Split(&'x [u8], &'x [u8]),\n    None,\n}\n\nimpl<'x> ChainedBytes<'x> {\n    pub fn new(first: &'x [u8]) -> Self {\n        Self { first, last: &[] }\n    }\n\n    pub fn append(&mut self, bytes: &'x [u8]) {\n        self.last = bytes;\n    }\n\n    pub fn with_last(mut self, bytes: &'x [u8]) -> Self {\n        self.last = bytes;\n        self\n    }\n\n    pub fn get(&self, index: Range<usize>) -> Option<Cow<'x, [u8]>> {\n        let start = index.start;\n        let end = index.end;\n\n        if let Some(bytes) = self.first.get(start..end) {\n            Some(Cow::Borrowed(bytes))\n        } else if start >= self.first.len() {\n            self.last\n                .get(start - self.first.len()..end - self.first.len())\n                .map(Cow::Borrowed)\n        } else if let (Some(first), Some(last)) = (\n            self.first.get(start..),\n            self.last.get(..end - self.first.len()),\n        ) {\n            let mut vec = vec![0u8; first.len() + last.len()];\n            vec[..first.len()].copy_from_slice(first);\n            vec[first.len()..].copy_from_slice(last);\n            Some(Cow::Owned(vec))\n        } else {\n            None\n        }\n    }\n\n    pub fn get_slice_range(&self, index: Range<usize>) -> SliceRange<'x> {\n        let start = index.start;\n        let end = index.end;\n\n        if let Some(bytes) = self.first.get(start..end) {\n            SliceRange::Single(bytes)\n        } else if start >= self.first.len() {\n            self.last\n                .get(start - self.first.len()..end - self.first.len())\n                .map(SliceRange::Single)\n                .unwrap_or(SliceRange::None)\n        } else if let (Some(first), Some(last)) = (\n            self.first.get(start..),\n            self.last.get(..end - self.first.len()),\n        ) {\n            SliceRange::Split(first, last)\n        } else {\n            SliceRange::None\n        }\n    }\n\n    pub fn get_full_range(&self) -> SliceRange<'x> {\n        if self.last.is_empty() {\n            SliceRange::Single(self.first)\n        } else {\n            SliceRange::Split(self.first, self.last)\n        }\n    }\n\n    pub fn to_bytes(&self) -> Vec<u8> {\n        let mut bytes = vec![0u8; self.first.len() + self.last.len()];\n        bytes[..self.first.len()].copy_from_slice(self.first);\n        bytes[self.first.len()..].copy_from_slice(self.last);\n        bytes\n    }\n\n    pub fn len(&self) -> usize {\n        self.first.len() + self.last.len()\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.len() == 0\n    }\n}\n\nimpl<'x> SliceRange<'x> {\n    pub fn len(&self) -> usize {\n        match self {\n            SliceRange::Single(bytes) => bytes.len(),\n            SliceRange::Split(first, last) => first.len() + last.len(),\n            SliceRange::None => 0,\n        }\n    }\n\n    pub fn try_into_bytes(self) -> Option<Cow<'x, [u8]>> {\n        match self {\n            SliceRange::Single(bytes) => Some(Cow::Borrowed(bytes)),\n            SliceRange::Split(first, last) => {\n                let mut vec = vec![0u8; first.len() + last.len()];\n                vec[..first.len()].copy_from_slice(first);\n                vec[first.len()..].copy_from_slice(last);\n                Some(Cow::Owned(vec))\n            }\n            SliceRange::None => None,\n        }\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.len() == 0\n    }\n\n    fn into_pairs(self) -> (&'x [u8], &'x [u8]) {\n        match self {\n            SliceRange::Single(bytes) => (bytes, &[][..]),\n            SliceRange::Split(first, last) => (first, last),\n            SliceRange::None => (&[][..], &[][..]),\n        }\n    }\n\n    pub fn is_none(&self) -> bool {\n        matches!(self, SliceRange::None)\n    }\n\n    pub fn is_some(&self) -> bool {\n        !self.is_none()\n    }\n}\n\nimpl<'x> IntoIterator for SliceRange<'x> {\n    type Item = &'x u8;\n    type IntoIter = std::iter::Chain<std::slice::Iter<'x, u8>, std::slice::Iter<'x, u8>>;\n\n    fn into_iter(self) -> Self::IntoIter {\n        let (first, last) = self.into_pairs();\n\n        first.iter().chain(last.iter())\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/cheeky_hash.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse nohash_hasher::IsEnabled;\nuse std::{\n    collections::{BTreeMap, HashMap, HashSet},\n    fmt::Debug,\n    hash::Hash,\n};\n\n// A hash that can cheekily store small inputs directly without hashing them.\n#[derive(\n    Copy, Clone, PartialEq, Eq, PartialOrd, Ord, rkyv::Serialize, rkyv::Deserialize, rkyv::Archive,\n)]\n#[repr(transparent)]\npub struct CheekyHash([u8; HASH_SIZE]);\n\nconst HASH_SIZE: usize = std::mem::size_of::<u64>() * 2;\nconst HASH_PAYLOAD: usize = HASH_SIZE - 1;\n\npub type CheekyHashSet = HashSet<CheekyHash, nohash_hasher::BuildNoHashHasher<CheekyHash>>;\npub type CheekyHashMap<V> = HashMap<CheekyHash, V, nohash_hasher::BuildNoHashHasher<CheekyHash>>;\npub type CheekyBTreeMap<V> = BTreeMap<CheekyHash, V>;\n\nimpl CheekyHash {\n    pub const HASH_SIZE: usize = HASH_SIZE;\n    pub const NULL: CheekyHash = CheekyHash([0u8; HASH_SIZE]);\n    pub const FULL: CheekyHash = CheekyHash([u8::MAX; HASH_SIZE]);\n\n    pub fn new(bytes: impl AsRef<[u8]>) -> Self {\n        let mut hash = [0u8; HASH_SIZE];\n        let bytes = bytes.as_ref();\n\n        if bytes.len() <= HASH_PAYLOAD {\n            hash[0] = bytes.len() as u8;\n            hash[1..1 + bytes.len()].copy_from_slice(bytes);\n        } else {\n            let h1 = xxhash_rust::xxh3::xxh3_64(bytes).to_be_bytes();\n            let h2 = farmhash::fingerprint64(bytes).to_be_bytes();\n            hash[0] = bytes.len().min(u8::MAX as usize) as u8;\n            hash[1..1 + std::mem::size_of::<u64>()].copy_from_slice(&h1);\n            hash[1 + std::mem::size_of::<u64>()..]\n                .copy_from_slice(&h2[..std::mem::size_of::<u64>() - 1]);\n        }\n\n        CheekyHash(hash)\n    }\n\n    pub fn deserialize(bytes: &[u8]) -> Option<Self> {\n        let len = *bytes.first()?;\n        let mut hash = [0u8; HASH_SIZE];\n        let hash_len = 1 + (len as usize).min(HASH_PAYLOAD);\n\n        hash[0] = len;\n        hash[1..hash_len].copy_from_slice(bytes.get(1..hash_len)?);\n        Some(CheekyHash(hash))\n    }\n\n    #[allow(clippy::len_without_is_empty)]\n    #[inline(always)]\n    pub fn len(&self) -> usize {\n        (self.0[0] as usize).min(HASH_PAYLOAD) + 1\n    }\n\n    #[inline(always)]\n    pub fn as_bytes(&self) -> &[u8] {\n        &self.0[..self.len()]\n    }\n\n    #[inline(always)]\n    pub fn as_raw_bytes(&self) -> &[u8; HASH_SIZE] {\n        &self.0\n    }\n\n    pub fn into_inner(self) -> [u8; HASH_SIZE] {\n        self.0\n    }\n\n    pub fn payload(&self) -> &[u8] {\n        let len = self.0[0] as usize;\n        if len <= HASH_PAYLOAD {\n            &self.0[1..1 + len]\n        } else {\n            &self.0[1..]\n        }\n    }\n\n    pub fn payload_len(&self) -> u8 {\n        self.0[0]\n    }\n}\n\nimpl AsRef<[u8]> for CheekyHash {\n    fn as_ref(&self) -> &[u8] {\n        self.as_bytes()\n    }\n}\n\nimpl Hash for CheekyHash {\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        let len = self.0[0] as usize;\n        if len <= HASH_PAYLOAD {\n            state.write_u64(xxhash_rust::xxh3::xxh3_64(&self.0[1..1 + len]));\n        } else {\n            state.write_u64(u64::from_be_bytes(\n                self.0[1..1 + std::mem::size_of::<u64>()]\n                    .try_into()\n                    .unwrap(),\n            ));\n        }\n    }\n}\n\nimpl IsEnabled for CheekyHash {}\n\nimpl ArchivedCheekyHash {\n    #[inline(always)]\n    pub fn as_raw_bytes(&self) -> &[u8; HASH_SIZE] {\n        &self.0\n    }\n\n    #[inline(always)]\n    pub fn as_bytes(&self) -> &[u8] {\n        let len = self.0[0] as usize;\n        &self.0[..1 + len.min(HASH_PAYLOAD)]\n    }\n\n    #[inline(always)]\n    pub fn to_native(&self) -> CheekyHash {\n        CheekyHash(self.0)\n    }\n}\n\nimpl Debug for CheekyHash {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        let len = self.payload_len();\n        let payload = self.payload();\n        let payload_str = if len <= HASH_PAYLOAD as u8 {\n            std::str::from_utf8(payload).unwrap_or(\"<non-utf8>\")\n        } else {\n            \"<hashed data>\"\n        };\n\n        f.debug_struct(\"CheekyHash\")\n            .field(\"length\", &len)\n            .field(\"bytes\", &payload_str)\n            .finish()\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn test_cheeky_hash_all() {\n        // Test 1: Empty input\n        let hash_empty = CheekyHash::new([]);\n        assert_eq!(\n            hash_empty.as_bytes()[0],\n            0,\n            \"Empty input should have length 0\"\n        );\n        assert_eq!(\n            hash_empty.as_bytes().len(),\n            1,\n            \"Empty input should only have length byte\"\n        );\n\n        // Test 2: Single byte input\n        let hash_single = CheekyHash::new([42]);\n        assert_eq!(\n            hash_single.as_bytes()[0],\n            1,\n            \"Single byte should have length 1\"\n        );\n        assert_eq!(\n            hash_single.as_bytes()[1],\n            42,\n            \"Single byte value should be preserved\"\n        );\n        assert_eq!(hash_single.as_bytes().len(), 2);\n\n        // Test 3: Small input (less than HASH_LEN)\n        let small_data = b\"hello\";\n        let hash_small = CheekyHash::new(small_data);\n        assert_eq!(hash_small.as_bytes()[0], 5, \"Length should be 5\");\n        assert_eq!(\n            &hash_small.as_bytes()[1..6],\n            small_data,\n            \"Small data should be stored directly\"\n        );\n        assert_eq!(hash_small.as_bytes().len(), 6);\n\n        // Test 4: Input exactly at HASH_PAYLOAD boundary\n        let boundary_data = vec![1u8; HASH_PAYLOAD - 1];\n        let hash_boundary = CheekyHash::new(&boundary_data);\n        assert_eq!(\n            hash_boundary.as_bytes()[0],\n            (HASH_PAYLOAD - 1) as u8,\n            \"Length should be HASH_LEN\"\n        );\n        assert_eq!(\n            &hash_boundary.as_bytes()[1..],\n            &boundary_data[..],\n            \"Boundary data should be stored directly\"\n        );\n\n        // Test 5: Large input (greater than HASH_LEN) - uses hashing\n        let large_data = vec![7u8; HASH_SIZE];\n        let hash_large = CheekyHash::new(&large_data);\n        assert_eq!(\n            hash_large.as_bytes()[0],\n            HASH_SIZE as u8,\n            \"Large data should have length byte set to HASH_LEN\"\n        );\n        assert_eq!(\n            hash_large.as_bytes().len(),\n            HASH_SIZE,\n            \"Large data hash should be full length\"\n        );\n        // Verify it's actually hashed (not raw data)\n        assert_ne!(\n            &hash_large.as_bytes()[1..],\n            &large_data[..HASH_PAYLOAD],\n            \"Large data should be hashed, not stored directly\"\n        );\n\n        // Test 6: AsRef<[u8]> trait\n        let hash = CheekyHash::new(b\"test\");\n        let bytes_ref: &[u8] = hash.as_ref();\n        assert_eq!(bytes_ref, hash.as_bytes(), \"AsRef should match as_bytes\");\n\n        // Test 7: Copy, Clone, PartialEq traits\n        let hash1 = CheekyHash::new(b\"identical\");\n        let hash2 = hash1; // Copy\n        assert_eq!(hash1, hash2, \"Copied hashes should be equal\");\n\n        // Test 8: Different inputs produce different hashes\n        let hash_a = CheekyHash::new(b\"abc\");\n        let hash_b = CheekyHash::new(b\"def\");\n        assert_ne!(\n            hash_a, hash_b,\n            \"Different inputs should produce different hashes\"\n        );\n\n        // Test 9: Same input produces same hash (deterministic)\n        let hash_x1 = CheekyHash::new(b\"deterministic\");\n        let hash_x2 = CheekyHash::new(b\"deterministic\");\n        assert_eq!(\n            hash_x1, hash_x2,\n            \"Same input should produce identical hashes\"\n        );\n\n        // Test 10: Large inputs with different content produce different hashes\n        let large1 = vec![1u8; 100];\n        let large2 = vec![2u8; 100];\n        let hash_large1 = CheekyHash::new(&large1);\n        let hash_large2 = CheekyHash::new(&large2);\n        assert_ne!(\n            hash_large1, hash_large2,\n            \"Different large inputs should produce different hashes\"\n        );\n\n        // Test 11: Hash trait (can be used in HashMap/HashSet)\n        use std::collections::HashMap;\n        let mut map = HashMap::new();\n        let key = CheekyHash::new(b\"key\");\n        map.insert(key, \"value\");\n        assert_eq!(\n            map.get(&key),\n            Some(&\"value\"),\n            \"CheekyHash should work as HashMap key\"\n        );\n\n        // Test 12: Debug trait\n        let hash = CheekyHash::new(b\"debug\");\n        let debug_str = format!(\"{:?}\", hash);\n        assert!(\n            debug_str.contains(\"CheekyHash\"),\n            \"Debug output should contain type name\"\n        );\n\n        // Test 13: CheekyHashSet and CheekyHashMap\n        let mut cheeky_set: CheekyHashSet = CheekyHashSet::default();\n        cheeky_set.insert(CheekyHash::new(b\"set_item\"));\n        assert!(cheeky_set.contains(&CheekyHash::new(b\"set_item\")));\n        let mut cheeky_map: CheekyHashMap<&str> = CheekyHashMap::default();\n        cheeky_map.insert(CheekyHash::new(b\"map_key\"), \"map_value\");\n        assert_eq!(\n            cheeky_map.get(&CheekyHash::new(b\"map_key\")),\n            Some(&\"map_value\")\n        );\n\n        println!(\"All CheekyHash tests passed!\");\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/codec/base32_custom.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{io::Write, slice::Iter};\n\nuse super::leb128::{Leb128Iterator, Leb128Writer};\n\npub static BASE32_ALPHABET: &[u8] = b\"abcdefghijklmnopqrstuvwxyz792013\";\npub static BASE32_INVERSE: [u8; 256] = [\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 29, 30, 28, 31, 255, 255, 255, 26, 255, 27,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 10, 11,\n    12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 255, 255, 255, 255, 255, 255, 0, 1, 2,\n    3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n    255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,\n];\n\npub struct Base32Writer {\n    last_byte: u8,\n    pos: usize,\n    result: String,\n}\n\nimpl Base32Writer {\n    pub fn from_bytes(bytes: impl AsRef<[u8]>) -> Self {\n        let bytes = bytes.as_ref();\n        let mut writer = Base32Writer::with_capacity(bytes.len());\n        writer.write_all(bytes).unwrap();\n        writer\n    }\n\n    pub fn with_capacity(capacity: usize) -> Self {\n        Self::with_raw_capacity(capacity.div_ceil(4) * 5)\n    }\n\n    pub fn with_raw_capacity(capacity: usize) -> Self {\n        Base32Writer {\n            result: String::with_capacity(capacity),\n            last_byte: 0,\n            pos: 0,\n        }\n    }\n\n    pub fn push_char(&mut self, ch: char) {\n        self.result.push(ch);\n    }\n\n    pub fn push_string(&mut self, string: &str) {\n        self.result.push_str(string);\n    }\n\n    fn push_byte(&mut self, byte: u8, is_remainder: bool) {\n        let (ch1, ch2) = match self.pos % 5 {\n            0 => ((byte & 0xF8) >> 3, u8::MAX),\n            1 => (\n                (((self.last_byte & 0x07) << 2) | ((byte & 0xC0) >> 6)),\n                ((byte & 0x3E) >> 1),\n            ),\n            2 => (\n                (((self.last_byte & 0x01) << 4) | ((byte & 0xF0) >> 4)),\n                u8::MAX,\n            ),\n            3 => (\n                (((self.last_byte & 0x0F) << 1) | (byte >> 7)),\n                ((byte & 0x7C) >> 2),\n            ),\n            4 => (\n                (((self.last_byte & 0x03) << 3) | ((byte & 0xE0) >> 5)),\n                (byte & 0x1F),\n            ),\n            _ => unreachable!(),\n        };\n\n        self.result.push(char::from(BASE32_ALPHABET[ch1 as usize]));\n        if !is_remainder {\n            if ch2 != u8::MAX {\n                self.result.push(char::from(BASE32_ALPHABET[ch2 as usize]));\n            }\n            self.last_byte = byte;\n            self.pos += 1;\n        }\n    }\n\n    pub fn finalize(mut self) -> String {\n        if !self.pos.is_multiple_of(5) {\n            self.push_byte(0, true);\n        }\n\n        self.result\n    }\n}\n\nimpl std::io::Write for Base32Writer {\n    fn write(&mut self, bytes: &[u8]) -> std::io::Result<usize> {\n        let start_pos = self.pos;\n\n        for &byte in bytes {\n            self.push_byte(byte, false);\n        }\n\n        Ok(self.pos - start_pos)\n    }\n\n    fn flush(&mut self) -> std::io::Result<()> {\n        Ok(())\n    }\n}\n\n#[derive(Debug)]\npub struct Base32Reader<'x> {\n    bytes: Iter<'x, u8>,\n    last_byte: u8,\n    pos: usize,\n}\n\nimpl<'x> Base32Reader<'x> {\n    pub fn new(bytes: &'x [u8]) -> Self {\n        Base32Reader {\n            bytes: bytes.iter(),\n            pos: 0,\n            last_byte: 0,\n        }\n    }\n\n    #[allow(clippy::should_implement_trait)]\n    pub fn from_iter(bytes: Iter<'x, u8>) -> Self {\n        Base32Reader {\n            bytes,\n            pos: 0,\n            last_byte: 0,\n        }\n    }\n\n    #[inline(always)]\n    fn map_byte(&mut self) -> Option<u8> {\n        match self.bytes.next() {\n            Some(&byte) => match BASE32_INVERSE[byte as usize] {\n                byte if byte != u8::MAX => {\n                    self.last_byte = byte;\n                    Some(byte)\n                }\n                _ => None,\n            },\n            _ => None,\n        }\n    }\n}\n\nimpl Iterator for Base32Reader<'_> {\n    type Item = u8;\n    fn next(&mut self) -> Option<Self::Item> {\n        let pos = self.pos % 5;\n        let last_byte = self.last_byte;\n        let byte = self.map_byte()?;\n        self.pos += 1;\n\n        match pos {\n            0 => ((byte << 3) | (self.map_byte().unwrap_or(0) >> 2)).into(),\n            1 => ((last_byte << 6) | (byte << 1) | (self.map_byte().unwrap_or(0) >> 4)).into(),\n            2 => ((last_byte << 4) | (byte >> 1)).into(),\n            3 => ((last_byte << 7) | (byte << 2) | (self.map_byte().unwrap_or(0) >> 3)).into(),\n            4 => ((last_byte << 5) | byte).into(),\n            _ => None,\n        }\n    }\n}\n\nimpl Leb128Iterator<u8> for Base32Reader<'_> {}\nimpl Leb128Writer for Base32Writer {}\n\n#[cfg(test)]\nmod tests {\n    use std::io::Write;\n\n    use crate::codec::base32_custom::{Base32Reader, Base32Writer};\n\n    #[test]\n    fn base32_roundtrip() {\n        let mut bytes = Vec::with_capacity(100);\n        for byte in 0..100 {\n            bytes.push((100 - byte) as u8);\n            let mut writer = Base32Writer::with_capacity(10);\n            writer.write_all(&bytes).unwrap();\n            let result = writer.finalize();\n\n            let mut bytes_result = Vec::new();\n            for byte in Base32Reader::new(result.as_bytes()) {\n                bytes_result.push(byte);\n            }\n\n            assert_eq!(bytes, bytes_result);\n        }\n\n        for bytes in [\n            vec![0],\n            vec![32, 43, 55, 99, 43, 55],\n            vec![84, 4, 43, 77, 62, 55, 92],\n            vec![84, 4, 43, 77, 62, 55, 92],\n        ] {\n            let mut writer = Base32Writer::with_capacity(10);\n            writer.write_all(&bytes).unwrap();\n            let result = writer.finalize();\n\n            let mut bytes_result = Vec::new();\n            for byte in Base32Reader::new(result.as_bytes()) {\n                bytes_result.push(byte);\n            }\n\n            assert_eq!(bytes, bytes_result);\n        }\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/codec/leb128.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#![allow(dead_code)]\n\nuse std::{borrow::Borrow, io::Write};\n\npub trait Leb128_ {\n    fn to_leb128_writer(self, out: &mut impl Write) -> std::io::Result<usize>;\n    fn to_leb128_bytes(self, out: &mut Vec<u8>);\n    fn from_leb128_bytes_pos(slice: &[u8]) -> Option<(Self, usize)>\n    where\n        Self: std::marker::Sized;\n    fn from_leb128_bytes(slice: &[u8]) -> Option<Self>\n    where\n        Self: std::marker::Sized;\n    fn from_leb128_it<T, I>(it: T) -> Option<Self>\n    where\n        Self: std::marker::Sized,\n        T: Iterator<Item = I>,\n        I: Borrow<u8>;\n}\n\npub trait Leb128Vec<T: Leb128_> {\n    fn push_leb128(&mut self, value: T);\n}\n\npub trait Leb128Writer: Write + Sized {\n    #[inline(always)]\n    fn write_leb128<T: Leb128_>(&mut self, value: T) -> std::io::Result<usize> {\n        T::to_leb128_writer(value, self)\n    }\n}\n\nimpl<T: Leb128_> Leb128Vec<T> for Vec<u8> {\n    #[inline(always)]\n    fn push_leb128(&mut self, value: T) {\n        T::to_leb128_bytes(value, self);\n    }\n}\n\npub trait Leb128Iterator<I>: Iterator<Item = I>\nwhere\n    I: Borrow<u8>,\n{\n    #[inline(always)]\n    fn next_leb128<T: Leb128_>(&mut self) -> Option<T> {\n        T::from_leb128_it(self)\n    }\n\n    #[inline(always)]\n    fn skip_leb128(&mut self) -> Option<()> {\n        for byte in self {\n            if (byte.borrow() & 0x80) == 0 {\n                return Some(());\n            }\n        }\n        None\n    }\n}\n\npub trait Leb128Reader: AsRef<[u8]> {\n    #[inline(always)]\n    fn read_leb128<T: Leb128_>(&self) -> Option<(T, usize)> {\n        T::from_leb128_bytes_pos(self.as_ref())\n    }\n\n    #[inline(always)]\n    fn skip_leb128(&self) -> Option<usize> {\n        for (pos, byte) in self.as_ref().iter().enumerate() {\n            if (byte & 0x80) == 0 {\n                return (pos + 1).into();\n            }\n        }\n        None\n    }\n}\n\nimpl Leb128Reader for &[u8] {}\nimpl Leb128Reader for Vec<u8> {}\nimpl Leb128Reader for Box<[u8]> {}\nimpl<'x> Leb128Iterator<&'x u8> for std::slice::Iter<'x, u8> {}\n\n// Based on leb128.rs from rustc\nmacro_rules! impl_unsigned_leb128 {\n    ($int_ty:ident, $shifts:expr) => {\n        impl Leb128_ for $int_ty {\n            #[inline(always)]\n            fn to_leb128_writer(self, out: &mut impl Write) -> std::io::Result<usize> {\n                let mut value = self;\n                let mut bytes_written = 0;\n                loop {\n                    if value < 0x80 {\n                        bytes_written += out.write(&[value as u8])?;\n                        break;\n                    } else {\n                        bytes_written += out.write(&[((value & 0x7f) | 0x80) as u8])?;\n                        value >>= 7;\n                    }\n                }\n                Ok(bytes_written)\n            }\n\n            #[inline(always)]\n            fn to_leb128_bytes(self, out: &mut Vec<u8>) {\n                let mut value = self;\n                loop {\n                    if value < 0x80 {\n                        out.push(value as u8);\n                        break;\n                    } else {\n                        out.push(((value & 0x7f) | 0x80) as u8);\n                        value >>= 7;\n                    }\n                }\n            }\n\n            #[inline(always)]\n            fn from_leb128_bytes_pos(slice: &[u8]) -> Option<($int_ty, usize)> {\n                let mut result = 0;\n\n                for (shift, (pos, &byte)) in $shifts.into_iter().zip(slice.iter().enumerate()) {\n                    if (byte & 0x80) == 0 {\n                        result |= (byte as $int_ty) << shift;\n                        return Some((result, pos + 1));\n                    } else {\n                        result |= ((byte & 0x7F) as $int_ty) << shift;\n                    }\n                }\n\n                None\n            }\n\n            #[inline(always)]\n            fn from_leb128_bytes(slice: &[u8]) -> Option<$int_ty> {\n                let mut result = 0;\n\n                for (shift, &byte) in $shifts.into_iter().zip(slice.iter()) {\n                    if (byte & 0x80) == 0 {\n                        result |= (byte as $int_ty) << shift;\n                        return Some(result);\n                    } else {\n                        result |= ((byte & 0x7F) as $int_ty) << shift;\n                    }\n                }\n\n                None\n            }\n\n            #[inline(always)]\n            fn from_leb128_it<T, I>(it: T) -> Option<$int_ty>\n            where\n                T: Iterator<Item = I>,\n                I: Borrow<u8>,\n            {\n                let mut result = 0;\n\n                for (shift, byte_) in $shifts.into_iter().zip(it) {\n                    let byte = byte_.borrow();\n\n                    if (byte & 0x80) == 0 {\n                        result |= (*byte as $int_ty) << shift;\n                        return Some(result);\n                    } else {\n                        result |= ((byte & 0x7F) as $int_ty) << shift;\n                    }\n                }\n\n                None\n            }\n        }\n    };\n}\n\nimpl_unsigned_leb128!(u8, [0]);\nimpl_unsigned_leb128!(u16, [0, 7, 14]);\nimpl_unsigned_leb128!(u32, [0, 7, 14, 21, 28]);\nimpl_unsigned_leb128!(u64, [0, 7, 14, 21, 28, 35, 42, 49, 56, 63]);\nimpl_unsigned_leb128!(usize, [0, 7, 14, 21, 28, 35, 42, 49, 56, 63]);\n\nimpl Leb128Writer for Vec<u8> {\n    fn write_leb128<T: Leb128_>(&mut self, value: T) -> std::io::Result<usize> {\n        T::to_leb128_writer(value, self)\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/codec/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod base32_custom;\npub mod leb128;\n"
  },
  {
    "path": "crates/utils/src/config/cron.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse chrono::{Datelike, Local, TimeDelta, TimeZone, Timelike};\n\nuse super::utils::ParseValue;\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum SimpleCron {\n    Day { hour: u32, minute: u32 },\n    Week { day: u32, hour: u32, minute: u32 },\n    Hour { minute: u32 },\n}\n\nimpl SimpleCron {\n    pub fn time_to_next(&self) -> Duration {\n        let now = Local::now();\n        let next = match self {\n            SimpleCron::Day { hour, minute } => {\n                let next = Local\n                    .with_ymd_and_hms(now.year(), now.month(), now.day(), *hour, *minute, 0)\n                    .earliest()\n                    .unwrap_or_else(|| now - TimeDelta::try_seconds(1).unwrap_or_default());\n                if next <= now {\n                    next + TimeDelta::try_days(1).unwrap_or_default()\n                } else {\n                    next\n                }\n            }\n            SimpleCron::Week { day, hour, minute } => {\n                let next = Local\n                    .with_ymd_and_hms(now.year(), now.month(), now.day(), *hour, *minute, 0)\n                    .earliest()\n                    .unwrap_or_else(|| now - TimeDelta::try_seconds(1).unwrap_or_default());\n                if next <= now {\n                    next + TimeDelta::try_days(\n                        (7 - now.weekday().number_from_monday() + *day).into(),\n                    )\n                    .unwrap_or_default()\n                } else {\n                    next\n                }\n            }\n            SimpleCron::Hour { minute } => {\n                let next = Local\n                    .with_ymd_and_hms(now.year(), now.month(), now.day(), now.hour(), *minute, 0)\n                    .earliest()\n                    .unwrap_or_else(|| now - TimeDelta::try_seconds(1).unwrap_or_default());\n                if next <= now {\n                    next + TimeDelta::try_hours(1).unwrap_or_default()\n                } else {\n                    next\n                }\n            }\n        };\n\n        (next - now).to_std().unwrap_or_else(|_| self.as_duration())\n    }\n\n    pub fn as_duration(&self) -> Duration {\n        match self {\n            SimpleCron::Day { .. } => Duration::from_secs(24 * 60 * 60),\n            SimpleCron::Week { .. } => Duration::from_secs(7 * 24 * 60 * 60),\n            SimpleCron::Hour { .. } => Duration::from_secs(60 * 60),\n        }\n    }\n}\n\nimpl ParseValue for SimpleCron {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        let mut hour = 0;\n        let mut minute = 0;\n\n        for (pos, value) in value.split(' ').enumerate() {\n            if pos == 0 {\n                minute = value\n                    .parse::<u32>()\n                    .map_err(|_| \"Invalid cron key: failed to parse cron minute\".to_string())?;\n                if !(0..=59).contains(&minute) {\n                    return Err(format!(\n                        \"Invalid cron key: failed to parse minute, invalid value: {minute}\"\n                    ));\n                }\n            } else if pos == 1 {\n                if value\n                    .as_bytes()\n                    .first()\n                    .ok_or_else(|| \"Invalid cron key: failed to parse cron hour\".to_string())?\n                    == &b'*'\n                {\n                    return Ok(SimpleCron::Hour { minute });\n                } else {\n                    hour = value\n                        .parse::<u32>()\n                        .map_err(|_| \"Invalid cron key: failed to parse cron hour\".to_string())?;\n                    if !(0..=23).contains(&hour) {\n                        return Err(format!(\n                            \"Invalid cron key: failed to parse hour, invalid value: {hour}\"\n                        ));\n                    }\n                }\n            } else if pos == 2 {\n                if value\n                    .as_bytes()\n                    .first()\n                    .ok_or_else(|| \"Invalid cron key: failed to parse cron weekday\".to_string())?\n                    == &b'*'\n                {\n                    return Ok(SimpleCron::Day { hour, minute });\n                } else {\n                    let day = value.parse::<u32>().map_err(|_| {\n                        \"Invalid cron key: failed to parse cron weekday\".to_string()\n                    })?;\n                    if !(1..=7).contains(&hour) {\n                        return Err(format!(\n                            \"Invalid cron key: failed to parse weekday, invalid value: {}, range is 1 (Monday) to 7 (Sunday).\",\n                            hour,\n                        ));\n                    }\n\n                    return Ok(SimpleCron::Week { day, hour, minute });\n                }\n            }\n        }\n\n        Err(\"Invalid cron key: parse cron expression.\".to_string())\n    }\n}\n\nimpl Default for SimpleCron {\n    fn default() -> Self {\n        SimpleCron::Hour { minute: 0 }\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/config/http.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::config::{Config, utils::AsKey};\nuse base64::{Engine, engine::general_purpose};\nuse reqwest::{\n    Client,\n    header::{AUTHORIZATION, CONTENT_TYPE, HeaderMap, HeaderName, HeaderValue, USER_AGENT},\n};\nuse std::{str::FromStr, time::Duration};\n\npub fn build_http_client(\n    config: &mut Config,\n    prefix: impl AsKey,\n    content_type: Option<&str>,\n) -> Option<Client> {\n    let mut headers = parse_http_headers(config, prefix.clone());\n    headers.insert(USER_AGENT, \"Stalwart/1.0.0\".parse().unwrap());\n\n    if let Some(content_type) = content_type {\n        headers.insert(CONTENT_TYPE, HeaderValue::from_str(content_type).unwrap());\n    }\n\n    let prefix = prefix.as_key();\n    match Client::builder()\n        .connect_timeout(\n            config\n                .property_or_default::<Duration>((&prefix, \"timeout\"), \"30s\")\n                .unwrap_or(Duration::from_secs(30)),\n        )\n        .danger_accept_invalid_certs(\n            config\n                .property_or_default::<bool>((&prefix, \"tls.allow-invalid-certs\"), \"false\")\n                .unwrap_or(false),\n        )\n        .default_headers(headers)\n        .build()\n    {\n        Ok(client) => Some(client),\n        Err(err) => {\n            config.new_build_error(&prefix, format!(\"Failed to build HTTP client: {err}\"));\n            None\n        }\n    }\n}\n\npub fn parse_http_headers(config: &mut Config, prefix: impl AsKey) -> HeaderMap {\n    let prefix = prefix.as_key();\n    let mut headers = HeaderMap::new();\n\n    for (header, value) in config\n        .values((&prefix, \"headers\"))\n        .map(|(_, v)| {\n            if let Some((k, v)) = v.split_once(':') {\n                Ok((\n                    HeaderName::from_str(k.trim()).map_err(|err| {\n                        format!(\"Invalid header found in property \\\"{prefix}.headers\\\": {err}\",)\n                    })?,\n                    HeaderValue::from_str(v.trim()).map_err(|err| {\n                        format!(\"Invalid header found in property \\\"{prefix}.headers\\\": {err}\",)\n                    })?,\n                ))\n            } else {\n                Err(format!(\n                    \"Invalid header found in property \\\"{prefix}.headers\\\": {v}\",\n                ))\n            }\n        })\n        .collect::<Result<Vec<(HeaderName, HeaderValue)>, String>>()\n        .map_err(|e| config.new_parse_error((&prefix, \"headers\"), e))\n        .unwrap_or_default()\n    {\n        headers.insert(header, value);\n    }\n\n    if let (Some(name), Some(secret)) = (\n        config.value((&prefix, \"auth.username\")),\n        config.value((&prefix, \"auth.secret\")),\n    ) {\n        headers.insert(\n            AUTHORIZATION,\n            format!(\n                \"Basic {}\",\n                general_purpose::STANDARD.encode(format!(\"{}:{}\", name, secret))\n            )\n            .parse()\n            .unwrap(),\n        );\n    } else if let Some(token) = config.value((&prefix, \"auth.token\")) {\n        headers.insert(AUTHORIZATION, format!(\"Bearer {}\", token).parse().unwrap());\n    }\n\n    headers\n}\n"
  },
  {
    "path": "crates/utils/src/config/ipmask.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr};\n\nuse rustls::{SupportedCipherSuite, crypto::ring::cipher_suite::*};\n\nuse super::utils::ParseValue;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum IpAddrMask {\n    V4 { addr: Ipv4Addr, mask: u32 },\n    V6 { addr: Ipv6Addr, mask: u128 },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum IpAddrOrMask {\n    Ip(IpAddr),\n    Mask(IpAddrMask),\n}\n\nimpl IpAddrMask {\n    pub fn matches(&self, remote: &IpAddr) -> bool {\n        match self {\n            IpAddrMask::V4 { addr, mask } => match *mask {\n                u32::MAX => match remote {\n                    IpAddr::V4(remote) => addr == remote,\n                    IpAddr::V6(remote) => {\n                        if let Some(remote) = remote.to_ipv4_mapped() {\n                            addr == &remote\n                        } else {\n                            false\n                        }\n                    }\n                },\n                0 => {\n                    matches!(remote, IpAddr::V4(_))\n                }\n                _ => {\n                    u32::from_be_bytes(match remote {\n                        IpAddr::V4(ip) => ip.octets(),\n                        IpAddr::V6(ip) => {\n                            if let Some(ip) = ip.to_ipv4() {\n                                ip.octets()\n                            } else {\n                                return false;\n                            }\n                        }\n                    }) & mask\n                        == u32::from_be_bytes(addr.octets()) & mask\n                }\n            },\n            IpAddrMask::V6 { addr, mask } => match *mask {\n                u128::MAX => match remote {\n                    IpAddr::V6(remote) => remote == addr,\n                    IpAddr::V4(remote) => &remote.to_ipv6_mapped() == addr,\n                },\n                0 => {\n                    matches!(remote, IpAddr::V6(_))\n                }\n                _ => {\n                    u128::from_be_bytes(match remote {\n                        IpAddr::V6(ip) => ip.octets(),\n                        IpAddr::V4(ip) => ip.to_ipv6_mapped().octets(),\n                    }) & mask\n                        == u128::from_be_bytes(addr.octets()) & mask\n                }\n            },\n        }\n    }\n}\n\nimpl ParseValue for IpAddrMask {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        if let Some((addr, mask)) = value.rsplit_once('/') {\n            if let (Ok(addr), Ok(mask)) =\n                (addr.trim().parse::<IpAddr>(), mask.trim().parse::<u32>())\n            {\n                match addr {\n                    IpAddr::V4(addr) if (8..=32).contains(&mask) => {\n                        return Ok(IpAddrMask::V4 {\n                            addr,\n                            mask: u32::MAX << (32 - mask),\n                        });\n                    }\n                    IpAddr::V6(addr) if (8..=128).contains(&mask) => {\n                        return Ok(IpAddrMask::V6 {\n                            addr,\n                            mask: u128::MAX << (128 - mask),\n                        });\n                    }\n                    _ => (),\n                }\n            }\n        } else {\n            match value.trim().parse::<IpAddr>() {\n                Ok(IpAddr::V4(addr)) => {\n                    return Ok(IpAddrMask::V4 {\n                        addr,\n                        mask: u32::MAX,\n                    });\n                }\n                Ok(IpAddr::V6(addr)) => {\n                    return Ok(IpAddrMask::V6 {\n                        addr,\n                        mask: u128::MAX,\n                    });\n                }\n                _ => (),\n            }\n        }\n\n        Err(format!(\"Invalid IP address {:?}\", value,))\n    }\n}\n\nimpl ParseValue for IpAddrOrMask {\n    fn parse_value(ip: &str) -> super::Result<Self> {\n        if ip.contains('/') {\n            IpAddrMask::parse_value(ip).map(IpAddrOrMask::Mask)\n        } else {\n            IpAddr::parse_value(ip).map(IpAddrOrMask::Ip)\n        }\n    }\n}\n\nimpl ParseValue for SocketAddr {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid socket address {:?}.\", value,))\n    }\n}\n\nimpl ParseValue for SupportedCipherSuite {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        Ok(match value {\n            // TLS1.3 suites\n            \"TLS13_AES_256_GCM_SHA384\" => TLS13_AES_256_GCM_SHA384,\n            \"TLS13_AES_128_GCM_SHA256\" => TLS13_AES_128_GCM_SHA256,\n            \"TLS13_CHACHA20_POLY1305_SHA256\" => TLS13_CHACHA20_POLY1305_SHA256,\n            // TLS1.2 suites\n            \"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\" => TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,\n            \"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256\" => TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,\n            \"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256\" => {\n                TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256\n            }\n            \"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\" => TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,\n            \"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\" => TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,\n            \"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256\" => {\n                TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256\n            }\n            cipher => return Err(format!(\"Unsupported TLS cipher suite {:?}\", cipher,)),\n        })\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn test_ipaddrmask() {\n        for (mask, ip) in [\n            (\"10.0.0.0/8\", \"10.30.20.11\"),\n            (\"10.0.0.0/8\", \"10.0.13.73\"),\n            (\"192.168.1.1\", \"192.168.1.1\"),\n        ] {\n            let mask = IpAddrMask::parse_value(mask).unwrap();\n            let ip = ip.parse::<IpAddr>().unwrap();\n            assert!(mask.matches(&ip));\n        }\n\n        for (mask, ip) in [\n            (\"10.0.0.0/8\", \"11.30.20.11\"),\n            (\"192.168.1.1\", \"193.168.1.1\"),\n        ] {\n            let mask = IpAddrMask::parse_value(mask).unwrap();\n            let ip = ip.parse::<IpAddr>().unwrap();\n            assert!(!mask.matches(&ip));\n        }\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/config/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod cron;\npub mod http;\npub mod ipmask;\npub mod parser;\npub mod utils;\n\nuse ahash::AHashMap;\nuse compact_str::CompactString;\nuse serde::Serialize;\nuse std::{collections::BTreeMap, time::Duration};\n\n#[derive(Debug, Default, Serialize)]\npub struct Config {\n    #[serde(skip)]\n    pub keys: BTreeMap<String, String>,\n    pub warnings: AHashMap<String, ConfigWarning>,\n    pub errors: AHashMap<String, ConfigError>,\n    #[cfg(debug_assertions)]\n    #[serde(skip)]\n    pub keys_read: parking_lot::Mutex<ahash::AHashSet<String>>,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Serialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum ConfigWarning {\n    Missing,\n    AppliedDefault { default: String },\n    Unread { value: String },\n    Build { error: String },\n    Parse { error: String },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Serialize)]\n#[serde(tag = \"type\")]\n#[serde(rename_all = \"camelCase\")]\npub enum ConfigError {\n    Parse { error: String },\n    Build { error: String },\n    Macro { error: String },\n}\n\n#[derive(Debug, Default, PartialEq, Eq)]\npub struct ConfigKey {\n    pub key: String,\n    pub value: String,\n}\n\n#[derive(Debug, Default, PartialEq, Eq, Clone)]\npub struct Rate {\n    pub requests: u64,\n    pub period: Duration,\n}\n\npub type Result<T> = std::result::Result<T, String>;\n\nimpl Config {\n    pub async fn resolve_macros(&mut self, classes: &[&str]) {\n        for macro_class in classes {\n            self.resolve_macro_type(macro_class).await;\n        }\n    }\n\n    pub async fn resolve_all_macros(&mut self) {\n        self.resolve_macros(&[\"env\", \"file\", \"cfg\"]).await;\n    }\n\n    async fn resolve_macro_type(&mut self, class: &str) {\n        let macro_start = format!(\"%{{{class}:\");\n        let mut replacements = AHashMap::new();\n        'outer: for (key, value) in &self.keys {\n            if value.contains(&macro_start) && value.contains(\"}%\") {\n                let mut result = String::with_capacity(value.len());\n                let mut snippet: &str = value.as_str();\n\n                loop {\n                    if let Some((suffix, macro_name)) = snippet.split_once(&macro_start) {\n                        if !suffix.is_empty() {\n                            result.push_str(suffix);\n                        }\n                        if let Some((location, rest)) = macro_name.split_once(\"}%\") {\n                            match class {\n                                \"cfg\" => {\n                                    if let Some(value) = replacements\n                                        .get(location)\n                                        .or_else(|| self.keys.get(location))\n                                    {\n                                        result.push_str(value);\n                                    } else {\n                                        self.errors.insert(\n                                            key.clone(),\n                                            ConfigError::Macro {\n                                                error: format!(\"Unknown key {location:?}\"),\n                                            },\n                                        );\n                                    }\n                                }\n                                \"env\" => match std::env::var(location) {\n                                    Ok(value) => {\n                                        result.push_str(&value);\n                                    }\n                                    Err(_) => {\n                                        self.errors.insert(\n                                                key.clone(),\n                                                ConfigError::Macro { error : format!(\n                                                    \"Failed to obtain environment variable {location:?}\"\n                                                )},\n                                            );\n                                    }\n                                },\n                                \"file\" => {\n                                    let file_name = location.strip_prefix(\"//\").unwrap_or(location);\n                                    match tokio::fs::read(file_name).await {\n                                        Ok(value) => match String::from_utf8(value) {\n                                            Ok(value) => {\n                                                result.push_str(&value);\n                                            }\n                                            Err(err) => {\n                                                self.errors.insert(\n                                                    key.clone(),\n                                                    ConfigError::Macro {\n                                                        error: format!(\n                                                        \"Failed to read file {file_name:?}: {err}\"\n                                                    ),\n                                                    },\n                                                );\n                                                continue 'outer;\n                                            }\n                                        },\n                                        Err(err) => {\n                                            self.errors.insert(\n                                                key.clone(),\n                                                ConfigError::Macro {\n                                                    error: format!(\n                                                        \"Failed to read file {file_name:?}: {err}\"\n                                                    ),\n                                                },\n                                            );\n                                            continue 'outer;\n                                        }\n                                    }\n                                }\n                                _ => {\n                                    unreachable!()\n                                }\n                            };\n\n                            snippet = rest;\n                        }\n                    } else {\n                        result.push_str(snippet);\n                        break;\n                    }\n                }\n\n                replacements.insert(key.clone(), result);\n            }\n        }\n\n        if !replacements.is_empty() {\n            for (key, value) in replacements {\n                self.keys.insert(key, value);\n            }\n        }\n    }\n\n    pub fn update(&mut self, settings: Vec<(String, String)>) {\n        self.keys.extend(settings);\n    }\n\n    pub fn log_errors(&self) {\n        for (key, err) in &self.errors {\n            let (cause, message) = match err {\n                ConfigError::Parse { error } => (\n                    trc::ConfigEvent::ParseError,\n                    format!(\"Failed to parse setting {key:?}: {error}\"),\n                ),\n                ConfigError::Build { error } => (\n                    trc::ConfigEvent::BuildError,\n                    format!(\"Build error for key {key:?}: {error}\"),\n                ),\n                ConfigError::Macro { error } => (\n                    trc::ConfigEvent::MacroError,\n                    format!(\"Macro expansion error for setting {key:?}: {error}\"),\n                ),\n            };\n\n            trc::error!(\n                trc::EventType::Config(cause)\n                    .into_err()\n                    .details(CompactString::from(message))\n            );\n        }\n    }\n\n    pub fn log_warnings(&mut self) {\n        #[cfg(debug_assertions)]\n        self.warn_unread_keys();\n\n        for (key, warn) in &self.warnings {\n            let (cause, message) = match warn {\n                ConfigWarning::AppliedDefault { default } => (\n                    trc::ConfigEvent::DefaultApplied,\n                    format!(\"WARNING: Missing setting {key:?}, applied default {default:?}\"),\n                ),\n                ConfigWarning::Missing => (\n                    trc::ConfigEvent::MissingSetting,\n                    format!(\"WARNING: Missing setting {key:?}\"),\n                ),\n                ConfigWarning::Unread { value } => (\n                    trc::ConfigEvent::UnusedSetting,\n                    format!(\"WARNING: Unused setting {key:?} with value {value:?}\"),\n                ),\n                ConfigWarning::Parse { error } => (\n                    trc::ConfigEvent::ParseWarning,\n                    format!(\"WARNING: Failed to parse {key:?}: {error}\"),\n                ),\n                ConfigWarning::Build { error } => (\n                    trc::ConfigEvent::BuildWarning,\n                    format!(\"WARNING for {key:?}: {error}\"),\n                ),\n            };\n\n            trc::error!(\n                trc::EventType::Config(cause)\n                    .into_err()\n                    .details(CompactString::from(message))\n            );\n        }\n    }\n}\n\nimpl Clone for Config {\n    fn clone(&self) -> Self {\n        Self {\n            keys: self.keys.clone(),\n            warnings: self.warnings.clone(),\n            errors: self.errors.clone(),\n            #[cfg(debug_assertions)]\n            keys_read: Default::default(),\n        }\n    }\n}\n\nimpl PartialEq for Config {\n    fn eq(&self, other: &Self) -> bool {\n        self.keys == other.keys && self.warnings == other.warnings && self.errors == other.errors\n    }\n}\n\nimpl Eq for Config {}\n\nimpl From<(String, String)> for ConfigKey {\n    fn from((key, value): (String, String)) -> Self {\n        Self { key, value }\n    }\n}\n\nimpl From<(&str, &str)> for ConfigKey {\n    fn from((key, value): (&str, &str)) -> Self {\n        Self {\n            key: key.to_string(),\n            value: value.to_string(),\n        }\n    }\n}\n\nimpl From<(&str, String)> for ConfigKey {\n    fn from((key, value): (&str, String)) -> Self {\n        Self {\n            key: key.to_string(),\n            value,\n        }\n    }\n}\n\nimpl From<(String, &str)> for ConfigKey {\n    fn from((key, value): (String, &str)) -> Self {\n        Self {\n            key,\n            value: value.to_string(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/config/parser.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    collections::{BTreeMap, btree_map::Entry},\n    iter::Peekable,\n    str::Chars,\n};\n\nuse super::{Config, Result};\nuse std::fmt::Write;\n\nconst MAX_NEST_LEVEL: usize = 10;\n\n// Simple TOML parser for Stalwart Server configuration files.\nimpl Config {\n    pub fn new(toml: impl AsRef<str>) -> Result<Self> {\n        let mut config = Config::default();\n        config.parse(toml.as_ref())?;\n        Ok(config)\n    }\n\n    pub fn parse(&mut self, toml: &str) -> Result<()> {\n        let mut parser = TomlParser::new(&mut self.keys, toml);\n        let mut table_name = String::new();\n        let mut last_array_name = String::new();\n        let mut last_array_pos = 0;\n\n        while parser.seek_next_char() {\n            match parser.peek_char()? {\n                '[' => {\n                    parser.next_char(true, false)?;\n                    table_name.clear();\n                    let mut is_array = match parser.next_char(true, false)? {\n                        '[' => true,\n                        ch => {\n                            table_name.push(ch);\n                            false\n                        }\n                    };\n                    let mut in_quote = false;\n                    let mut last_ch = char::from(0);\n                    loop {\n                        let ch = parser.next_char(!in_quote, false)?;\n                        match ch {\n                            '\\\"' if !in_quote || last_ch != '\\\\' => {\n                                in_quote = !in_quote;\n                            }\n                            '\\\\' if in_quote => (),\n                            ']' if !in_quote => {\n                                if table_name.is_empty() {\n                                    return Err(format!(\n                                        \"Empty table name at line {}.\",\n                                        parser.line\n                                    ));\n                                }\n                                if is_array {\n                                    if table_name == last_array_name {\n                                        last_array_pos += 1;\n                                    } else {\n                                        last_array_pos = 0;\n                                        last_array_name = table_name.to_string();\n                                    }\n                                    is_array = false;\n                                    write!(table_name, \".{last_array_pos:04}\").ok();\n                                } else {\n                                    break;\n                                }\n                            }\n                            _ => {\n                                if !in_quote {\n                                    if ch.is_alphanumeric() || ['.', '-', '_'].contains(&ch) {\n                                        table_name.push(ch.to_ascii_lowercase());\n                                    } else {\n                                        return Err(format!(\n                                            \"Unexpected character {:?} at line {}.\",\n                                            ch, parser.line\n                                        ));\n                                    }\n                                } else {\n                                    table_name.push(ch);\n                                }\n                            }\n                        }\n                        last_ch = ch;\n                    }\n                    parser.skip_line();\n                }\n                'a'..='z' | 'A'..='Z' | '0'..='9' | '\\\"' => {\n                    let (key, _) = parser.key(\n                        if !table_name.is_empty() {\n                            format!(\"{table_name}.\")\n                        } else {\n                            String::with_capacity(10)\n                        },\n                        false,\n                    )?;\n                    parser.value(key, &['\\n'], 0)?;\n                }\n                '#' => {\n                    parser.skip_line();\n                }\n                ch => {\n                    let ch = *ch;\n                    return Err(format!(\n                        \"Unexpected character {:?} at line {}.\",\n                        ch, parser.line\n                    ));\n                }\n            }\n        }\n\n        Ok(())\n    }\n}\n\nstruct TomlParser<'x, 'y> {\n    keys: &'y mut BTreeMap<String, String>,\n    iter: Peekable<Chars<'x>>,\n    line: usize,\n}\n\nimpl<'x, 'y> TomlParser<'x, 'y> {\n    fn new(keys: &'y mut BTreeMap<String, String>, toml: &'x str) -> Self {\n        Self {\n            keys,\n            iter: toml.chars().peekable(),\n            line: 1,\n        }\n    }\n\n    fn seek_next_char(&mut self) -> bool {\n        while let Some(ch) = self.iter.peek() {\n            match ch {\n                '\\n' => {\n                    self.iter.next();\n                    self.line += 1;\n                }\n                '\\r' | ' ' | '\\t' => {\n                    self.iter.next();\n                }\n                '#' => {\n                    self.skip_line();\n                }\n                _ => {\n                    return true;\n                }\n            }\n        }\n\n        false\n    }\n\n    fn peek_char(&mut self) -> Result<&char> {\n        self.iter.peek().ok_or_else(|| \"\".to_string())\n    }\n\n    fn next_char(&mut self, skip_wsp: bool, allow_lf: bool) -> Result<char> {\n        for ch in &mut self.iter {\n            match ch {\n                '\\r' => (),\n                ' ' | '\\t' if skip_wsp => (),\n                '\\n' => {\n                    return if allow_lf {\n                        self.line += 1;\n                        Ok(ch)\n                    } else {\n                        Err(format!(\"Unexpected end of line at line: {}\", self.line))\n                    };\n                }\n                _ => {\n                    return Ok(ch);\n                }\n            }\n        }\n        Err(format!(\"Unexpected EOF at line: {}\", self.line))\n    }\n\n    fn skip_line(&mut self) {\n        for ch in &mut self.iter {\n            if ch == '\\n' {\n                self.line += 1;\n                break;\n            }\n        }\n    }\n\n    #[allow(clippy::while_let_on_iterator)]\n    fn key(&mut self, mut key: String, in_curly: bool) -> Result<(String, char)> {\n        let start_key_len = key.len();\n        while let Some(ch) = self.iter.next() {\n            match ch {\n                '=' => {\n                    if start_key_len != key.len() {\n                        return Ok((key, ch));\n                    } else {\n                        return Err(format!(\"Empty key at line: {}\", self.line));\n                    }\n                }\n                ',' | '}' if in_curly => {\n                    if start_key_len != key.len() {\n                        return Ok((key, ch));\n                    } else {\n                        return Err(format!(\"Empty key at line: {}\", self.line));\n                    }\n                }\n                /*'a'..='z' | '.' | 'A'..='Z' | '0'..='9' | '_' | '-' => {\n                    key.push(ch);\n                }*/\n                '\\\"' => {\n                    let mut last_ch = char::from(0);\n                    while let Some(ch) = self.iter.next() {\n                        match ch {\n                            '\\\\' => (),\n                            '\\\"' if last_ch != '\\\\' => {\n                                break;\n                            }\n                            '\\n' => {\n                                return Err(format!(\n                                    \"Unexpected end of line while parsing quoted key at line: {}\",\n                                    self.line\n                                ));\n                            }\n                            _ => {\n                                key.push(ch);\n                            }\n                        }\n                        last_ch = ch;\n                    }\n                }\n                ' ' | '\\t' | '\\r' => (),\n                '\\n' => {\n                    if start_key_len == key.len() {\n                        self.line += 1;\n                    } else {\n                        return Err(format!(\n                            \"Unexpected end of line while parsing key {:?} at line: {}\",\n                            key, self.line\n                        ));\n                    }\n                }\n                _ => {\n                    key.push(ch);\n                }\n            }\n        }\n        Err(format!(\"Unexpected EOF at line: {}\", self.line))\n    }\n\n    fn value(&mut self, key: String, stop_chars: &[char], nest_level: usize) -> Result<char> {\n        if nest_level == MAX_NEST_LEVEL {\n            return Err(format!(\"Too many nested structures at line {}.\", self.line));\n        }\n        match self.next_char(true, false)? {\n            '[' => {\n                let mut array_pos = 0;\n                self.seek_next_char();\n                loop {\n                    match self.value(\n                        format!(\"{key}.{array_pos:04}\"),\n                        &[',', ']'],\n                        nest_level + 1,\n                    )? {\n                        ',' => {\n                            self.seek_next_char();\n                            array_pos += 1;\n                        }\n                        ']' => break,\n                        ch => {\n                            return Err(format!(\n                                \"Unexpected character {:?} found in array for property {:?} at line {}.\",\n                                ch, key, self.line\n                            ));\n                        }\n                    }\n                }\n            }\n            '{' => {\n                let base_key = format!(\"{key}.\");\n                let base_key_len = base_key.len();\n\n                loop {\n                    let (sub_key, stop_char) = self.key(base_key.clone(), true)?;\n                    match stop_char {\n                        '=' => {\n                            // Key value\n                            self.seek_next_char();\n\n                            match self.value(sub_key, &[',', '}'], nest_level + 1)? {\n                                ',' => {\n                                    self.seek_next_char();\n                                }\n                                '}' => break,\n                                ch => {\n                                    return Err(format!(\n                                        \"Unexpected character {:?} found in inline table for property {:?} at line {}.\",\n                                        ch, key, self.line\n                                    ));\n                                }\n                            }\n                        }\n                        ',' => {\n                            // Set\n                            if sub_key.len() > base_key_len {\n                                self.insert_key(sub_key, String::new())?;\n                            }\n                        }\n                        '}' => {\n                            // Set\n                            if sub_key.len() > base_key_len {\n                                self.insert_key(sub_key, String::new())?;\n                            }\n                            break;\n                        }\n                        _ => unreachable!(),\n                    }\n                }\n            }\n            qch @ ('\\'' | '\\\"') => {\n                let mut value = String::new();\n                if matches!(self.iter.peek(), Some(ch) if ch == &qch) {\n                    self.iter.next();\n                    if matches!(self.iter.peek(), Some(ch) if ch == &qch) {\n                        self.iter.next();\n                        if matches!(self.iter.peek(), Some(ch) if ch == &'\\n') {\n                            self.iter.next();\n                            self.line += 1;\n                        }\n\n                        let mut last_ch = char::from(0);\n                        let mut prev_last_ch = char::from(0);\n                        loop {\n                            let ch = self.next_char(false, true)?;\n                            if !(ch == qch && last_ch == qch && prev_last_ch == qch) {\n                                value.push(ch);\n                                prev_last_ch = last_ch;\n                                last_ch = ch;\n                            } else {\n                                value.truncate(value.len() - 2);\n                                break;\n                            }\n                        }\n                    }\n                } else {\n                    let mut last_ch = char::from(0);\n\n                    loop {\n                        let ch = self.next_char(false, true)?;\n                        match ch {\n                            '\\\\' if last_ch != '\\\\' => (),\n                            't' if last_ch == '\\\\' => {\n                                value.push('\\t');\n                            }\n                            'r' if last_ch == '\\\\' => {\n                                value.push('\\r');\n                            }\n                            'n' if last_ch == '\\\\' => {\n                                value.push('\\n');\n                            }\n                            ch => {\n                                if ch != qch || last_ch == '\\\\' {\n                                    value.push(ch);\n                                } else {\n                                    break;\n                                }\n                            }\n                        }\n                        last_ch = ch;\n                    }\n                }\n\n                self.insert_key(key, value)?;\n            }\n            ch if ch.is_alphanumeric() || ['.', '+', '-'].contains(&ch) => {\n                let mut value = String::with_capacity(4);\n                value.push(ch);\n                while let Some(ch) = self.iter.peek() {\n                    if ch.is_alphanumeric() || ['.', '+', '-'].contains(ch) {\n                        value.push(self.next_char(true, false)?);\n                    } else {\n                        break;\n                    }\n                }\n                self.insert_key(key, value)?;\n            }\n            ch => {\n                return if stop_chars.contains(&ch) {\n                    Ok(ch)\n                } else {\n                    Err(format!(\n                        \"Expected {:?} but found {:?} in value at line {}.\",\n                        stop_chars, ch, self.line\n                    ))\n                };\n            }\n        }\n\n        loop {\n            match self.next_char(true, true)? {\n                '#' => {\n                    self.skip_line();\n                    if stop_chars.contains(&'\\n') {\n                        return Ok('\\n');\n                    }\n                }\n                ch if stop_chars.contains(&ch) => {\n                    return Ok(ch);\n                }\n                '\\n' if !stop_chars.contains(&'\\n') => (),\n                ch => {\n                    return Err(format!(\n                        \"Expected {:?} but found {:?} in value at line {}.\",\n                        stop_chars, ch, self.line\n                    ));\n                }\n            }\n        }\n    }\n\n    fn insert_key(&mut self, key: String, mut value: String) -> Result<()> {\n        match self.keys.entry(key) {\n            Entry::Vacant(e) => {\n                value.shrink_to_fit();\n                e.insert(value);\n                Ok(())\n            }\n            Entry::Occupied(e) => Err(format!(\n                \"Duplicate key {:?} at line {}.\",\n                e.key(),\n                self.line\n            )),\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use std::{collections::BTreeMap, fs, path::PathBuf};\n\n    use crate::config::Config;\n\n    #[test]\n    fn toml_parse() {\n        let file = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n            .parent()\n            .unwrap()\n            .parent()\n            .unwrap()\n            .to_path_buf()\n            .join(\"tests\")\n            .join(\"resources\")\n            .join(\"smtp\")\n            .join(\"config\")\n            .join(\"toml-parser.toml\");\n\n        let mut config = Config::default();\n        config.parse(&fs::read_to_string(file).unwrap()).unwrap();\n        let expected = BTreeMap::from_iter(\n            [\n                (\"arrays.colors.0000\", \"red\"),\n                (\"arrays.colors.0001\", \"yellow\"),\n                (\"arrays.colors.0002\", \"green\"),\n                (\"arrays.contributors.0000\", \"Foo Bar <foo@example.com>\"),\n                (\"arrays.contributors.0001.email\", \"bazqux@example.com\"),\n                (\"arrays.contributors.0001.name\", \"Baz Qux\"),\n                (\"arrays.contributors.0001.url\", \"https://example.com/bazqux\"),\n                (\"arrays.integers.0000\", \"1\"),\n                (\"arrays.integers.0001\", \"2\"),\n                (\"arrays.integers.0002\", \"3\"),\n                (\"arrays.integers2.0000\", \"1\"),\n                (\"arrays.integers2.0001\", \"2\"),\n                (\"arrays.integers2.0002\", \"3\"),\n                (\"arrays.integers3.0000\", \"4\"),\n                (\"arrays.integers3.0001\", \"5\"),\n                (\"arrays.nested_arrays_of_ints.0000.0000\", \"1\"),\n                (\"arrays.nested_arrays_of_ints.0000.0001\", \"2\"),\n                (\"arrays.nested_arrays_of_ints.0001.0000\", \"3\"),\n                (\"arrays.nested_arrays_of_ints.0001.0001\", \"4\"),\n                (\"arrays.nested_arrays_of_ints.0001.0002\", \"5\"),\n                (\"arrays.nested_mixed_array.0000.0000\", \"1\"),\n                (\"arrays.nested_mixed_array.0000.0001\", \"2\"),\n                (\"arrays.nested_mixed_array.0001.0000\", \"a\"),\n                (\"arrays.nested_mixed_array.0001.0001\", \"b\"),\n                (\"arrays.nested_mixed_array.0001.0002\", \"c\"),\n                (\"arrays.numbers.0000\", \"0.1\"),\n                (\"arrays.numbers.0001\", \"0.2\"),\n                (\"arrays.numbers.0002\", \"0.5\"),\n                (\"arrays.numbers.0003\", \"1\"),\n                (\"arrays.numbers.0004\", \"2\"),\n                (\"arrays.numbers.0005\", \"5\"),\n                (\"arrays.string_array.0000\", \"all\"),\n                (\"arrays.string_array.0001\", \"strings\"),\n                (\"arrays.string_array.0002\", \"are the same\"),\n                (\"arrays.string_array.0003\", \"type\"),\n                (\"database.data.0000.0000\", \"delta\"),\n                (\"database.data.0000.0001\", \"phi\"),\n                (\"database.data.0001.0000\", \"3.14\"),\n                (\"database.enabled\", \"true\"),\n                (\"database.ports.0000\", \"8000\"),\n                (\"database.ports.0001\", \"8001\"),\n                (\"database.ports.0002\", \"8002\"),\n                (\"database.temp_targets.case\", \"72.0\"),\n                (\"database.temp_targets.cpu\", \"79.5\"),\n                (\"products.0000.name\", \"Hammer\"),\n                (\"products.0000.sku\", \"738594937\"),\n                (\"products.0002.color\", \"gray\"),\n                (\"products.0002.name\", \"Nail\"),\n                (\"products.0002.sku\", \"284758393\"),\n                (\"servers.127.0.0.1\", \"value\"),\n                (\"servers.alpha.ip\", \"10.0.0.1\"),\n                (\"servers.alpha.role\", \"frontend\"),\n                (\"servers.beta.ip\", \"10.0.0.2\"),\n                (\"servers.beta.role\", \"backend\"),\n                (\"servers.character encoding\", \"value\"),\n                (\n                    \"strings.my \\\"string\\\" test.lines\",\n                    concat!(\n                        \"The first newline is\\ntrimmed in raw strings.\\n\",\n                        \"All other whitespace\\nis preserved.\\n\"\n                    ),\n                ),\n                (\"strings.my \\\"string\\\" test.str1\", \"I'm a string.\"),\n                (\"strings.my \\\"string\\\" test.str2\", \"You can \\\"quote\\\" me.\"),\n                (\"strings.my \\\"string\\\" test.str3\", \"Name\\tTabs\\nNew Line.\"),\n                (\"env.var1\", \"utils\"),\n                (\"env.var2\", \"utils\"),\n                (\"sets.integer.1\", \"\"),\n                (\"sets.integers.1\", \"\"),\n                (\"sets.integers.2\", \"\"),\n                (\"sets.integers.3\", \"\"),\n                (\"sets.string.red\", \"\"),\n                (\"sets.strings.red\", \"\"),\n                (\"sets.strings.yellow\", \"\"),\n                (\"sets.strings.green\", \"\"),\n            ]\n            .map(|(k, v)| (k.to_string(), v.to_string())),\n        );\n\n        if config.keys != expected {\n            for (key, value) in &config.keys {\n                if let Some(expected_value) = expected.get(key) {\n                    if value != expected_value {\n                        panic!(\n                            \"Expected value {:?} for key {:?} but found {:?}.\",\n                            expected_value, key, value\n                        );\n                    }\n                } else {\n                    panic!(\n                        \"Unexpected key {:?} found in config with value {:?}.\",\n                        key, value\n                    );\n                }\n            }\n\n            for (key, value) in &expected {\n                if let Some(config_value) = config.keys.get(key) {\n                    if value != config_value {\n                        panic!(\n                            \"Expected value {:?} for key {:?} but found {:?}.\",\n                            value, key, config_value\n                        );\n                    }\n                } else {\n                    panic!(\n                        \"Expected key {:?} not found in config with value {:?}.\",\n                        key, value\n                    );\n                }\n            }\n        }\n\n        assert_eq!(\n            config.set_values(\"sets.strings\").collect::<Vec<_>>(),\n            vec![\"green\", \"red\", \"yellow\"]\n        );\n\n        assert_eq!(\n            config.sub_keys(\"sets.strings\", \"\"),\n            vec![\"green\", \"red\", \"yellow\"]\n        );\n\n        assert_eq!(config.sub_keys(\"sets\", \".red\"), vec![\"string\", \"strings\"]);\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/config/utils.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    net::{IpAddr, Ipv4Addr, Ipv6Addr},\n    path::PathBuf,\n    str::FromStr,\n    time::Duration,\n};\n\nuse mail_auth::{\n    IpLookupStrategy,\n    common::crypto::{Algorithm, HashAlgorithm},\n    dkim::Canonicalization,\n};\nuse smtp_proto::MtPriority;\n\nuse super::{Config, ConfigError, ConfigWarning, Rate};\n\nimpl Config {\n    pub fn property<T: ParseValue>(&mut self, key: impl AsKey) -> Option<T> {\n        let key = key.as_key();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(key.clone());\n\n        if let Some(value) = self.keys.get(&key) {\n            match T::parse_value(value) {\n                Ok(value) => Some(value),\n                Err(err) => {\n                    self.new_parse_error(key, err);\n                    None\n                }\n            }\n        } else {\n            None\n        }\n    }\n\n    pub fn property_or_default<T: ParseValue>(\n        &mut self,\n        key: impl AsKey,\n        default: &str,\n    ) -> Option<T> {\n        let key = key.as_key();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(key.clone());\n\n        let value = match self.keys.get(&key) {\n            Some(value) => value.as_str(),\n            None => default,\n        };\n        match T::parse_value(value) {\n            Ok(value) => Some(value),\n            Err(err) => {\n                self.new_parse_error(key, err);\n                None\n            }\n        }\n    }\n\n    pub fn property_or_else<T: ParseValue>(\n        &mut self,\n        key: impl AsKey,\n        or_else: impl AsKey,\n        default: &str,\n    ) -> Option<T> {\n        let key = key.as_key();\n        let value = match self.value_or_else(key.as_str(), or_else.clone()) {\n            Some(value) => value,\n            None => default,\n        };\n\n        match T::parse_value(value) {\n            Ok(value) => Some(value),\n            Err(err) => {\n                self.new_parse_error(key, err);\n                None\n            }\n        }\n    }\n\n    pub fn property_require<T: ParseValue>(&mut self, key: impl AsKey) -> Option<T> {\n        let key = key.as_key();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(key.clone());\n\n        if let Some(value) = self.keys.get(&key) {\n            match T::parse_value(value) {\n                Ok(value) => Some(value),\n                Err(err) => {\n                    self.new_parse_error(key, err);\n                    None\n                }\n            }\n        } else {\n            self.new_parse_error(key, \"Missing property\");\n            None\n        }\n    }\n\n    pub fn sub_keys(&self, prefix: impl AsKey, suffix: &str) -> Vec<String> {\n        let mut last_key = \"\";\n        let prefix = prefix.as_prefix();\n\n        self.keys\n            .keys()\n            .filter_map(move |key| {\n                let key = key.strip_prefix(&prefix)?;\n                let key = if !suffix.is_empty() {\n                    key.strip_suffix(suffix)?\n                } else if let Some((key, _)) = key.split_once('.') {\n                    key\n                } else {\n                    key\n                };\n                if last_key != key {\n                    last_key = key;\n                    Some(key.to_string())\n                } else {\n                    None\n                }\n            })\n            .collect()\n    }\n\n    pub fn sub_keys_with_suffixes(&self, prefix: impl AsKey, suffixes: &[&str]) -> Vec<String> {\n        let mut last_key = \"\";\n        let prefix = prefix.as_prefix();\n\n        self.keys\n            .keys()\n            .filter_map(move |key| {\n                let key = key.strip_prefix(&prefix)?;\n                let key = suffixes\n                    .iter()\n                    .filter_map(|suffix| key.strip_suffix(suffix))\n                    .next()?;\n                if last_key != key {\n                    last_key = key;\n                    Some(key.to_string())\n                } else {\n                    None\n                }\n            })\n            .collect()\n    }\n\n    pub fn prefix<'x, 'y: 'x>(&'y self, prefix: impl AsKey) -> impl Iterator<Item = &'x str> + 'x {\n        let prefix = prefix.as_prefix();\n        self.keys\n            .keys()\n            .filter_map(move |key| key.strip_prefix(&prefix))\n    }\n\n    pub fn set_values<'x, 'y: 'x>(\n        &'y self,\n        prefix: impl AsKey,\n    ) -> impl Iterator<Item = &'x str> + 'x {\n        let prefix = prefix.as_prefix();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(prefix.clone());\n\n        self.keys\n            .keys()\n            .filter_map(move |key| key.strip_prefix(&prefix))\n    }\n\n    pub fn properties<T: ParseValue>(&mut self, prefix: impl AsKey) -> Vec<(String, T)> {\n        let full_prefix = prefix.as_key();\n        let prefix = prefix.as_prefix();\n        let mut results = Vec::new();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(prefix.clone());\n\n        for (key, value) in &self.keys {\n            if key.starts_with(&prefix) || key == &full_prefix {\n                match T::parse_value(value) {\n                    Ok(value) => {\n                        results.push((key.to_string(), value));\n                    }\n                    Err(error) => {\n                        self.errors\n                            .insert(key.to_string(), ConfigError::Parse { error });\n                    }\n                }\n            }\n        }\n\n        results\n    }\n\n    pub fn value(&self, key: impl AsKey) -> Option<&str> {\n        let key = key.as_key();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(key.clone());\n\n        self.keys.get(&key).map(|s| s.as_str())\n    }\n\n    pub fn contains_key(&self, key: impl AsKey) -> bool {\n        self.keys.contains_key(&key.as_key())\n    }\n\n    pub fn value_require(&mut self, key: impl AsKey) -> Option<&str> {\n        let key = key.as_key();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(key.clone());\n\n        if let Some(value) = self.keys.get(&key) {\n            Some(value.as_str())\n        } else {\n            self.errors.insert(\n                key,\n                ConfigError::Parse {\n                    error: \"Missing property\".to_string(),\n                },\n            );\n            None\n        }\n    }\n\n    pub fn value_require_non_empty(&mut self, key: impl AsKey) -> Option<&str> {\n        let key = key.as_key();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(key.clone());\n\n        if let Some(value) = self.keys.get(&key).and_then(|v| {\n            let v = v.trim();\n            if !v.is_empty() { Some(v) } else { None }\n        }) {\n            Some(value)\n        } else {\n            self.errors.insert(\n                key,\n                ConfigError::Parse {\n                    error: \"Missing property\".to_string(),\n                },\n            );\n            None\n        }\n    }\n\n    pub fn try_parse_value<T: ParseValue>(&mut self, key: impl AsKey, value: &str) -> Option<T> {\n        match T::parse_value(value) {\n            Ok(value) => Some(value),\n            Err(error) => {\n                self.errors\n                    .insert(key.as_key(), ConfigError::Parse { error });\n                None\n            }\n        }\n    }\n\n    pub fn value_or_else(&self, key: impl AsKey, or_else: impl AsKey) -> Option<&str> {\n        let key = key.as_key();\n\n        #[cfg(debug_assertions)]\n        {\n            self.keys_read.lock().insert(key.clone());\n            self.keys_read.lock().insert(or_else.clone().as_key());\n        }\n\n        self.keys\n            .get(&key)\n            .or_else(|| self.keys.get(&or_else.as_key()))\n            .map(|s| s.as_str())\n    }\n\n    pub fn values(&self, prefix: impl AsKey) -> impl Iterator<Item = (&str, &str)> {\n        let full_prefix = prefix.as_key();\n        let prefix = prefix.as_prefix();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(prefix.clone());\n\n        self.keys.iter().filter_map(move |(key, value)| {\n            if key.starts_with(&prefix) || key == &full_prefix {\n                (key.as_str(), value.as_str()).into()\n            } else {\n                None\n            }\n        })\n    }\n\n    pub fn iterate_prefix(&self, prefix: impl AsKey) -> impl Iterator<Item = (&str, &str)> {\n        let prefix = prefix.as_prefix();\n\n        #[cfg(debug_assertions)]\n        self.keys_read.lock().insert(prefix.clone());\n\n        self.keys\n            .iter()\n            .filter_map(move |(key, value)| Some((key.strip_prefix(&prefix)?, value.as_str())))\n    }\n\n    pub fn values_or_else(\n        &self,\n        prefix: impl AsKey,\n        or_else: impl AsKey,\n    ) -> impl Iterator<Item = (&str, &str)> {\n        let mut prefix = prefix.as_prefix();\n\n        #[cfg(debug_assertions)]\n        {\n            self.keys_read.lock().insert(prefix.clone());\n            self.keys_read.lock().insert(or_else.clone().as_prefix());\n        }\n\n        self.values(if self.keys.keys().any(|k| k.starts_with(&prefix)) {\n            prefix.truncate(prefix.len() - 1);\n            prefix\n        } else {\n            or_else.as_key()\n        })\n    }\n\n    pub fn has_prefix(&self, prefix: impl AsKey) -> bool {\n        let prefix = prefix.as_prefix();\n        self.keys.keys().any(|k| k.starts_with(&prefix))\n    }\n\n    pub fn new_parse_error(&mut self, key: impl AsKey, details: impl Into<String>) {\n        self.errors.insert(\n            key.as_key(),\n            ConfigError::Parse {\n                error: details.into(),\n            },\n        );\n    }\n\n    pub fn new_build_error(&mut self, key: impl AsKey, details: impl Into<String>) {\n        self.errors.insert(\n            key.as_key(),\n            ConfigError::Build {\n                error: details.into(),\n            },\n        );\n    }\n\n    pub fn new_parse_warning(&mut self, key: impl AsKey, details: impl Into<String>) {\n        self.warnings.insert(\n            key.as_key(),\n            ConfigWarning::Parse {\n                error: details.into(),\n            },\n        );\n    }\n\n    pub fn new_build_warning(&mut self, key: impl AsKey, details: impl Into<String>) {\n        self.warnings.insert(\n            key.as_key(),\n            ConfigWarning::Build {\n                error: details.into(),\n            },\n        );\n    }\n\n    pub fn new_missing_property(&mut self, key: impl AsKey) {\n        self.warnings.insert(key.as_key(), ConfigWarning::Missing);\n    }\n\n    #[cfg(debug_assertions)]\n    pub fn warn_unread_keys(&mut self) {\n        let mut keys = self.keys.clone();\n\n        for key in self.keys_read.lock().iter() {\n            if let Some(base_key) = key.strip_suffix('.') {\n                keys.remove(base_key);\n                keys.retain(|k, _| !k.starts_with(key));\n            } else {\n                keys.remove(key);\n            }\n        }\n\n        for (key, value) in keys {\n            self.warnings.insert(key, ConfigWarning::Unread { value });\n        }\n    }\n}\n\npub trait ParseValue: Sized {\n    fn parse_value(value: &str) -> super::Result<Self>;\n}\n\nimpl<T: ParseValue> ParseValue for Option<T> {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        if !value.is_empty()\n            && !value.eq_ignore_ascii_case(\"false\")\n            && !value.eq_ignore_ascii_case(\"disable\")\n            && !value.eq_ignore_ascii_case(\"disabled\")\n            && !value.eq_ignore_ascii_case(\"never\")\n            && !value.eq(\"0\")\n        {\n            T::parse_value(value).map(Some)\n        } else {\n            Ok(None)\n        }\n    }\n}\n\nimpl ParseValue for String {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        Ok(value.to_string())\n    }\n}\n\nimpl ParseValue for u64 {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid integer value {:?}.\", value,))\n    }\n}\n\nimpl ParseValue for f64 {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid floating point value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for u16 {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid integer value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for i16 {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid integer value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for u32 {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid integer value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for i32 {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid integer value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for f32 {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid floating point value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for IpAddr {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid IP address value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for usize {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid integer value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for bool {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid boolean value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for Ipv4Addr {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid IPv4 value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for Ipv6Addr {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        value\n            .parse()\n            .map_err(|_| format!(\"Invalid IPv6 value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for PathBuf {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        let path = PathBuf::from(value);\n\n        if path.exists() {\n            Ok(path)\n        } else {\n            Err(format!(\"Directory {} does not exist.\", path.display()))\n        }\n    }\n}\n\nimpl ParseValue for MtPriority {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        match value.to_ascii_lowercase().as_str() {\n            \"mixer\" => Ok(MtPriority::Mixer),\n            \"stanag4406\" => Ok(MtPriority::Stanag4406),\n            \"nsep\" => Ok(MtPriority::Nsep),\n            _ => Err(format!(\"Invalid priority value {:?}.\", value)),\n        }\n    }\n}\n\nimpl ParseValue for Canonicalization {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        match value {\n            \"relaxed\" => Ok(Canonicalization::Relaxed),\n            \"simple\" => Ok(Canonicalization::Simple),\n            _ => Err(format!(\"Invalid canonicalization value {:?}.\", value)),\n        }\n    }\n}\n\nimpl ParseValue for IpLookupStrategy {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        Ok(match value.to_lowercase().as_str() {\n            \"ipv4_only\" => IpLookupStrategy::Ipv4Only,\n            \"ipv6_only\" => IpLookupStrategy::Ipv6Only,\n            //\"ipv4_and_ipv6\" => IpLookupStrategy::Ipv4AndIpv6,\n            \"ipv6_then_ipv4\" => IpLookupStrategy::Ipv6thenIpv4,\n            \"ipv4_then_ipv6\" => IpLookupStrategy::Ipv4thenIpv6,\n            _ => return Err(format!(\"Invalid IP lookup strategy {:?}.\", value)),\n        })\n    }\n}\n\nimpl ParseValue for Algorithm {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        match value {\n            \"ed25519-sha256\" | \"ed25519-sha-256\" => Ok(Algorithm::Ed25519Sha256),\n            \"rsa-sha-256\" | \"rsa-sha256\" => Ok(Algorithm::RsaSha256),\n            \"rsa-sha-1\" | \"rsa-sha1\" => Ok(Algorithm::RsaSha1),\n            _ => Err(format!(\"Invalid algorithm {:?}.\", value)),\n        }\n    }\n}\n\nimpl ParseValue for HashAlgorithm {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        match value {\n            \"sha256\" | \"sha-256\" => Ok(HashAlgorithm::Sha256),\n            \"sha-1\" | \"sha1\" => Ok(HashAlgorithm::Sha1),\n            _ => Err(format!(\"Invalid hash algorithm {:?}.\", value)),\n        }\n    }\n}\n\nimpl ParseValue for Duration {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        let mut digits = String::new();\n        let mut multiplier = String::new();\n\n        for ch in value.chars() {\n            if ch.is_ascii_digit() {\n                digits.push(ch);\n            } else if !ch.is_ascii_whitespace() {\n                multiplier.push(ch.to_ascii_lowercase());\n            }\n        }\n\n        let multiplier = match multiplier.as_str() {\n            \"d\" => 24 * 60 * 60 * 1000,\n            \"h\" => 60 * 60 * 1000,\n            \"m\" => 60 * 1000,\n            \"s\" => 1000,\n            \"ms\" | \"\" => 1,\n            _ => return Err(format!(\"Invalid duration value {:?}.\", value)),\n        };\n\n        digits\n            .parse::<u64>()\n            .ok()\n            .and_then(|num| {\n                if num > 0 {\n                    Some(Duration::from_millis(num * multiplier))\n                } else {\n                    None\n                }\n            })\n            .ok_or_else(|| format!(\"Invalid duration value {:?}.\", value))\n    }\n}\n\nimpl ParseValue for Rate {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        if let Some((requests, period)) = value.split_once('/') {\n            Ok(Rate {\n                requests: requests\n                    .trim()\n                    .parse::<u64>()\n                    .ok()\n                    .and_then(|r| if r > 0 { Some(r) } else { None })\n                    .ok_or_else(|| format!(\"Invalid rate value {:?}.\", value))?,\n                period: std::cmp::max(Duration::parse_value(period)?, Duration::from_secs(1)),\n            })\n        } else if [\"false\", \"none\", \"unlimited\"].contains(&value) {\n            Ok(Rate::default())\n        } else {\n            Err(format!(\"Invalid rate value {:?}.\", value))\n        }\n    }\n}\n\nimpl ParseValue for trc::Level {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        trc::Level::from_str(value).map_err(|err| format!(\"Invalid log level: {err}\"))\n    }\n}\n\nimpl ParseValue for trc::EventType {\n    fn parse_value(value: &str) -> super::Result<Self> {\n        trc::EventType::try_parse(value).ok_or_else(|| format!(\"Unknown event type: {value}\"))\n    }\n}\n\nimpl ParseValue for () {\n    fn parse_value(_: &str) -> super::Result<Self> {\n        Ok(())\n    }\n}\n\npub trait AsKey: Clone {\n    fn as_key(&self) -> String;\n    fn as_prefix(&self) -> String;\n}\n\nimpl AsKey for String {\n    fn as_key(&self) -> String {\n        self.to_string()\n    }\n\n    fn as_prefix(&self) -> String {\n        format!(\"{self}.\")\n    }\n}\n\nimpl AsKey for &String {\n    fn as_key(&self) -> String {\n        self.to_string()\n    }\n\n    fn as_prefix(&self) -> String {\n        format!(\"{self}.\")\n    }\n}\n\nimpl AsKey for &str {\n    fn as_key(&self) -> String {\n        self.to_string()\n    }\n\n    fn as_prefix(&self) -> String {\n        format!(\"{self}.\")\n    }\n}\n\nimpl<A, B> AsKey for (A, B)\nwhere\n    A: AsRef<str> + Clone,\n    B: AsRef<str> + Clone,\n{\n    fn as_key(&self) -> String {\n        format!(\"{}.{}\", self.0.as_ref(), self.1.as_ref(),)\n    }\n\n    fn as_prefix(&self) -> String {\n        format!(\"{}.{}.\", self.0.as_ref(), self.1.as_ref(),)\n    }\n}\n\nimpl<A, B, C> AsKey for (A, B, C)\nwhere\n    A: AsRef<str> + Clone,\n    B: AsRef<str> + Clone,\n    C: AsRef<str> + Clone,\n{\n    fn as_key(&self) -> String {\n        format!(\n            \"{}.{}.{}\",\n            self.0.as_ref(),\n            self.1.as_ref(),\n            self.2.as_ref()\n        )\n    }\n\n    fn as_prefix(&self) -> String {\n        format!(\n            \"{}.{}.{}.\",\n            self.0.as_ref(),\n            self.1.as_ref(),\n            self.2.as_ref()\n        )\n    }\n}\n\nimpl<A, B, C, D> AsKey for (A, B, C, D)\nwhere\n    A: AsRef<str> + Clone,\n    B: AsRef<str> + Clone,\n    C: AsRef<str> + Clone,\n    D: AsRef<str> + Clone,\n{\n    fn as_key(&self) -> String {\n        format!(\n            \"{}.{}.{}.{}\",\n            self.0.as_ref(),\n            self.1.as_ref(),\n            self.2.as_ref(),\n            self.3.as_ref()\n        )\n    }\n\n    fn as_prefix(&self) -> String {\n        format!(\n            \"{}.{}.{}.{}.\",\n            self.0.as_ref(),\n            self.1.as_ref(),\n            self.2.as_ref(),\n            self.3.as_ref()\n        )\n    }\n}\n\nimpl<A, B, C, D, E> AsKey for (A, B, C, D, E)\nwhere\n    A: AsRef<str> + Clone,\n    B: AsRef<str> + Clone,\n    C: AsRef<str> + Clone,\n    D: AsRef<str> + Clone,\n    E: AsRef<str> + Clone,\n{\n    fn as_key(&self) -> String {\n        format!(\n            \"{}.{}.{}.{}.{}\",\n            self.0.as_ref(),\n            self.1.as_ref(),\n            self.2.as_ref(),\n            self.3.as_ref(),\n            self.4.as_ref()\n        )\n    }\n\n    fn as_prefix(&self) -> String {\n        format!(\n            \"{}.{}.{}.{}.{}.\",\n            self.0.as_ref(),\n            self.1.as_ref(),\n            self.2.as_ref(),\n            self.3.as_ref(),\n            self.4.as_ref()\n        )\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use std::net::IpAddr;\n\n    use crate::config::Config;\n\n    #[test]\n    fn toml_utils() {\n        let toml = r#\"\n[queues.\"z\"]\nretry = [0, 1, 15, 60, 90]\nvalue = \"hi\"\n\n[queues.\"x\"]\nretry = [3, 60]\nvalue = \"hi 2\"\n\n[queues.a]\nretry = [1, 2, 3, 4]\nvalue = \"hi 3\"\n\n[servers.\"my relay\"]\nhostname = \"mx.example.org\"\n\n[[servers.\"my relay\".transaction.auth.limits]]\nidle = 10\n\n[[servers.\"my relay\".transaction.auth.limits]]\nidle = 20\n\n[servers.\"submissions\"]\nhostname = \"submit.example.org\"\nip = \"a:b::1:1\"\n\"#;\n        let mut config = Config::default();\n        config.parse(toml).unwrap();\n\n        assert_eq!(config.sub_keys(\"queues\", \"\"), [\"a\", \"x\", \"z\"]);\n        assert_eq!(config.sub_keys(\"servers\", \"\"), [\"my relay\", \"submissions\"]);\n        assert_eq!(\n            config.sub_keys(\"queues.z.retry\", \"\"),\n            [\"0000\", \"0001\", \"0002\", \"0003\", \"0004\"]\n        );\n        assert_eq!(\n            config\n                .property::<u32>(\"servers.my relay.transaction.auth.limits.0001.idle\")\n                .unwrap(),\n            20\n        );\n        assert_eq!(\n            config\n                .property::<IpAddr>((\"servers\", \"submissions\", \"ip\"))\n                .unwrap(),\n            \"a:b::1:1\".parse::<IpAddr>().unwrap()\n        );\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/glob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::borrow::Cow;\n\nuse ahash::{AHashMap, AHashSet};\nuse serde::Deserialize;\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct GlobPattern {\n    pattern: Vec<PatternChar>,\n    to_lower: bool,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum PatternChar {\n    WildcardMany { num: usize, match_pos: usize },\n    WildcardSingle { match_pos: usize },\n    Char { char: char, match_pos: usize },\n}\n\nimpl GlobPattern {\n    pub fn compile(pattern: &str, to_lower: bool) -> Self {\n        let mut chars = Vec::new();\n        let mut is_escaped = false;\n        let mut str = pattern.chars().peekable();\n\n        while let Some(char) = str.next() {\n            match char {\n                '*' if !is_escaped => {\n                    let mut num = 1;\n                    while let Some('*') = str.peek() {\n                        num += 1;\n                        str.next();\n                    }\n                    chars.push(PatternChar::WildcardMany { num, match_pos: 0 });\n                }\n                '?' if !is_escaped => {\n                    chars.push(PatternChar::WildcardSingle { match_pos: 0 });\n                }\n                '\\\\' if !is_escaped => {\n                    is_escaped = true;\n                    continue;\n                }\n                _ => {\n                    if is_escaped {\n                        is_escaped = false;\n                    }\n                    if to_lower && char.is_uppercase() {\n                        for char in char.to_lowercase() {\n                            chars.push(PatternChar::Char { char, match_pos: 0 });\n                        }\n                    } else {\n                        chars.push(PatternChar::Char { char, match_pos: 0 });\n                    }\n                }\n            }\n        }\n\n        GlobPattern {\n            pattern: chars,\n            to_lower,\n        }\n    }\n\n    pub fn try_compile(pattern: &str, to_lower: bool) -> Result<Self, String> {\n        // Detect if the key is a glob pattern\n        let mut last_ch = '\\0';\n        let mut has_escape = false;\n        let mut is_glob = false;\n        for ch in pattern.chars() {\n            match ch {\n                '\\\\' => {\n                    has_escape = true;\n                }\n                '*' | '?' if last_ch != '\\\\' => {\n                    is_glob = true;\n                }\n                _ => {}\n            }\n\n            last_ch = ch;\n        }\n\n        if is_glob {\n            Ok(GlobPattern::compile(pattern, to_lower))\n        } else {\n            Err(if has_escape {\n                pattern.replace('\\\\', \"\")\n            } else {\n                pattern.to_string()\n            })\n        }\n    }\n\n    // Credits: Algorithm ported from https://research.swtch.com/glob\n    pub fn matches(&self, value: &str) -> bool {\n        let value = if self.to_lower {\n            value.to_lowercase().chars().collect::<Vec<_>>()\n        } else {\n            value.chars().collect::<Vec<_>>()\n        };\n\n        let mut px = 0;\n        let mut nx = 0;\n        let mut next_px = 0;\n        let mut next_nx = 0;\n\n        while px < self.pattern.len() || nx < value.len() {\n            match self.pattern.get(px) {\n                Some(PatternChar::Char { char, .. }) => {\n                    if matches!(value.get(nx), Some(nc) if nc == char ) {\n                        px += 1;\n                        nx += 1;\n                        continue;\n                    }\n                }\n                Some(PatternChar::WildcardSingle { .. }) => {\n                    if nx < value.len() {\n                        px += 1;\n                        nx += 1;\n                        continue;\n                    }\n                }\n                Some(PatternChar::WildcardMany { .. }) => {\n                    next_px = px;\n                    next_nx = nx + 1;\n                    px += 1;\n                    continue;\n                }\n                _ => (),\n            }\n            if 0 < next_nx && next_nx <= value.len() {\n                px = next_px;\n                nx = next_nx;\n                continue;\n            }\n            return false;\n        }\n        true\n    }\n}\n\n#[derive(Debug, Clone, Default)]\npub struct GlobSet {\n    entries: AHashSet<String>,\n    patterns: Vec<GlobPattern>,\n}\n\n#[derive(Debug, Clone)]\npub struct GlobMap<V> {\n    entries: AHashMap<String, V>,\n    patterns: Vec<(GlobPattern, V)>,\n}\n\nimpl GlobSet {\n    pub fn new() -> Self {\n        GlobSet::default()\n    }\n\n    pub fn insert(&mut self, pattern: &str) {\n        match GlobPattern::try_compile(pattern, false) {\n            Ok(glob) => {\n                self.patterns.push(glob);\n            }\n            Err(entry) => {\n                self.entries.insert(entry);\n            }\n        }\n    }\n\n    pub fn contains(&self, key: &str) -> bool {\n        self.entries.contains(key) || self.patterns.iter().any(|pattern| pattern.matches(key))\n    }\n}\n\nimpl<V> GlobMap<V> {\n    pub fn new() -> Self {\n        GlobMap {\n            entries: AHashMap::new(),\n            patterns: Vec::new(),\n        }\n    }\n\n    pub fn insert(&mut self, pattern: &str, value: V) {\n        match GlobPattern::try_compile(pattern, false) {\n            Ok(glob) => {\n                self.patterns.push((glob, value));\n            }\n            Err(entry) => {\n                self.entries.insert(entry, value);\n            }\n        }\n    }\n\n    pub fn get(&self, key: &str) -> Option<&V> {\n        self.entries.get(key).or_else(|| {\n            self.patterns\n                .iter()\n                .find_map(|(pattern, value)| pattern.matches(key).then_some(value))\n        })\n    }\n}\n\nimpl<V> Default for GlobMap<V> {\n    fn default() -> Self {\n        GlobMap::new()\n    }\n}\n\nimpl<'de> Deserialize<'de> for GlobPattern {\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        Ok(GlobPattern::compile(\n            <Cow<&str>>::deserialize(deserializer)?.as_ref(),\n            true,\n        ))\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod bimap;\npub mod cache;\npub mod chained_bytes;\npub mod cheeky_hash;\npub mod codec;\npub mod config;\npub mod glob;\npub mod map;\npub mod snowflake;\npub mod template;\npub mod topological;\npub mod url_params;\n\nuse compact_str::ToCompactString;\nuse futures::StreamExt;\nuse reqwest::Response;\nuse rustls::{\n    ClientConfig, RootCertStore, SignatureScheme,\n    client::danger::{HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier},\n};\nuse rustls_pki_types::TrustAnchor;\nuse std::sync::Arc;\n\npub trait HttpLimitResponse: Sync + Send {\n    fn bytes_with_limit(\n        self,\n        limit: usize,\n    ) -> impl std::future::Future<Output = reqwest::Result<Option<Vec<u8>>>> + Send;\n}\n\nimpl HttpLimitResponse for Response {\n    async fn bytes_with_limit(self, limit: usize) -> reqwest::Result<Option<Vec<u8>>> {\n        if self\n            .content_length()\n            .is_some_and(|len| len as usize > limit)\n        {\n            return Ok(None);\n        }\n\n        let mut bytes = Vec::with_capacity(std::cmp::min(limit, 1024));\n        let mut stream = self.bytes_stream();\n\n        while let Some(chunk) = stream.next().await {\n            let chunk = chunk?;\n            if bytes.len() + chunk.len() > limit {\n                return Ok(None);\n            }\n            bytes.extend_from_slice(&chunk);\n        }\n\n        Ok(Some(bytes))\n    }\n}\n\npub trait UnwrapFailure<T> {\n    fn failed(self, action: &str) -> T;\n}\n\nimpl<T> UnwrapFailure<T> for Option<T> {\n    fn failed(self, message: &str) -> T {\n        match self {\n            Some(result) => result,\n            None => {\n                trc::event!(\n                    Server(trc::ServerEvent::StartupError),\n                    Details = message.to_compact_string()\n                );\n                eprintln!(\"{message}\");\n                std::process::exit(1);\n            }\n        }\n    }\n}\n\nimpl<T, E: std::fmt::Display> UnwrapFailure<T> for Result<T, E> {\n    fn failed(self, message: &str) -> T {\n        match self {\n            Ok(result) => result,\n            Err(err) => {\n                trc::event!(\n                    Server(trc::ServerEvent::StartupError),\n                    Details = message.to_compact_string(),\n                    Reason = err.to_compact_string()\n                );\n\n                #[cfg(feature = \"test_mode\")]\n                panic!(\"{message}: {err}\");\n\n                #[cfg(not(feature = \"test_mode\"))]\n                {\n                    eprintln!(\"{message}: {err}\");\n                    std::process::exit(1);\n                }\n            }\n        }\n    }\n}\n\npub fn failed(message: &str) -> ! {\n    trc::event!(\n        Server(trc::ServerEvent::StartupError),\n        Details = message.to_compact_string(),\n    );\n    eprintln!(\"{message}\");\n    std::process::exit(1);\n}\n\npub async fn wait_for_shutdown() {\n    #[cfg(not(target_env = \"msvc\"))]\n    let signal = {\n        use tokio::signal::unix::{SignalKind, signal};\n\n        let mut h_term = signal(SignalKind::terminate()).failed(\"start signal handler\");\n        let mut h_int = signal(SignalKind::interrupt()).failed(\"start signal handler\");\n\n        tokio::select! {\n            _ = h_term.recv() => \"SIGTERM\",\n            _ = h_int.recv() => \"SIGINT\",\n        }\n    };\n\n    #[cfg(target_env = \"msvc\")]\n    let signal = {\n        match tokio::signal::ctrl_c().await {\n            Ok(()) => \"SIGINT\",\n            Err(err) => {\n                trc::event!(\n                    Server(trc::ServerEvent::ThreadError),\n                    Details = \"Unable to listen for shutdown signal\",\n                    Reason = err.to_string(),\n                );\n                \"Error\"\n            }\n        }\n    };\n\n    trc::event!(Server(trc::ServerEvent::Shutdown), CausedBy = signal);\n}\n\npub fn rustls_client_config(allow_invalid_certs: bool) -> ClientConfig {\n    let config = ClientConfig::builder();\n\n    if !allow_invalid_certs {\n        let mut root_cert_store = RootCertStore::empty();\n\n        root_cert_store.extend(webpki_roots::TLS_SERVER_ROOTS.iter().map(|ta| TrustAnchor {\n            subject: ta.subject.clone(),\n            subject_public_key_info: ta.subject_public_key_info.clone(),\n            name_constraints: ta.name_constraints.clone(),\n        }));\n\n        config\n            .with_root_certificates(root_cert_store)\n            .with_no_client_auth()\n    } else {\n        config\n            .dangerous()\n            .with_custom_certificate_verifier(Arc::new(DummyVerifier {}))\n            .with_no_client_auth()\n    }\n}\n\npub trait DomainPart {\n    fn to_lowercase_domain(&self) -> String;\n    fn domain_part(&self) -> &str;\n    fn try_domain_part(&self) -> Option<&str>;\n    fn try_local_part(&self) -> Option<&str>;\n}\n\nimpl<T: AsRef<str>> DomainPart for T {\n    fn to_lowercase_domain(&self) -> String {\n        let address = self.as_ref();\n        if let Some((local, domain)) = address.rsplit_once('@') {\n            let mut address = String::with_capacity(address.len());\n            address.push_str(local);\n            address.push('@');\n            for ch in domain.chars() {\n                for ch in ch.to_lowercase() {\n                    address.push(ch);\n                }\n            }\n            address\n        } else {\n            address.to_string()\n        }\n    }\n\n    #[inline(always)]\n    fn try_domain_part(&self) -> Option<&str> {\n        self.as_ref().rsplit_once('@').map(|(_, d)| d)\n    }\n\n    #[inline(always)]\n    fn try_local_part(&self) -> Option<&str> {\n        self.as_ref().rsplit_once('@').map(|(l, _)| l)\n    }\n\n    #[inline(always)]\n    fn domain_part(&self) -> &str {\n        self.as_ref()\n            .rsplit_once('@')\n            .map(|(_, d)| d)\n            .unwrap_or_default()\n    }\n}\n\n#[derive(Debug)]\nstruct DummyVerifier;\n\nimpl ServerCertVerifier for DummyVerifier {\n    fn verify_server_cert(\n        &self,\n        _end_entity: &rustls_pki_types::CertificateDer<'_>,\n        _intermediates: &[rustls_pki_types::CertificateDer<'_>],\n        _server_name: &rustls_pki_types::ServerName<'_>,\n        _ocsp_response: &[u8],\n        _now: rustls_pki_types::UnixTime,\n    ) -> Result<ServerCertVerified, rustls::Error> {\n        Ok(ServerCertVerified::assertion())\n    }\n\n    fn verify_tls12_signature(\n        &self,\n        _message: &[u8],\n        _cert: &rustls_pki_types::CertificateDer<'_>,\n        _dss: &rustls::DigitallySignedStruct,\n    ) -> Result<HandshakeSignatureValid, rustls::Error> {\n        Ok(HandshakeSignatureValid::assertion())\n    }\n\n    fn verify_tls13_signature(\n        &self,\n        _message: &[u8],\n        _cert: &rustls_pki_types::CertificateDer<'_>,\n        _dss: &rustls::DigitallySignedStruct,\n    ) -> Result<HandshakeSignatureValid, rustls::Error> {\n        Ok(HandshakeSignatureValid::assertion())\n    }\n\n    fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {\n        vec![\n            SignatureScheme::RSA_PKCS1_SHA1,\n            SignatureScheme::ECDSA_SHA1_Legacy,\n            SignatureScheme::RSA_PKCS1_SHA256,\n            SignatureScheme::ECDSA_NISTP256_SHA256,\n            SignatureScheme::RSA_PKCS1_SHA384,\n            SignatureScheme::ECDSA_NISTP384_SHA384,\n            SignatureScheme::RSA_PKCS1_SHA512,\n            SignatureScheme::ECDSA_NISTP521_SHA512,\n            SignatureScheme::RSA_PSS_SHA256,\n            SignatureScheme::RSA_PSS_SHA384,\n            SignatureScheme::RSA_PSS_SHA512,\n            SignatureScheme::ED25519,\n            SignatureScheme::ED448,\n        ]\n    }\n}\n\n// Basic email sanitizer\npub fn sanitize_email(email: &str) -> Option<String> {\n    let mut result = String::with_capacity(email.len());\n    let mut found_local = false;\n    let mut found_domain = false;\n    let mut last_ch = char::from(0);\n\n    for ch in email.chars() {\n        if !ch.is_whitespace() {\n            if ch == '@' {\n                if !result.is_empty() && !found_local {\n                    found_local = true;\n                } else {\n                    return None;\n                }\n            } else if ch == '.' {\n                if !(last_ch.is_alphanumeric() || last_ch == '-' || last_ch == '_') {\n                    return None;\n                } else if found_local {\n                    found_domain = true;\n                }\n            }\n            last_ch = ch;\n            for ch in ch.to_lowercase() {\n                result.push(ch);\n            }\n        }\n    }\n\n    if found_domain\n        && last_ch != '.'\n        && psl::domain(result.as_bytes()).is_some_and(|d| d.suffix().typ().is_some())\n    {\n        Some(result)\n    } else {\n        None\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/map/bitmap.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::ops::Deref;\n\n#[derive(\n    Debug,\n    rkyv::Archive,\n    rkyv::Deserialize,\n    rkyv::Serialize,\n    serde::Serialize,\n    serde::Deserialize,\n    Clone,\n    Copy,\n    PartialOrd,\n    Ord,\n    PartialEq,\n    Eq,\n    Hash,\n)]\n#[rkyv(compare(PartialEq), derive(Debug))]\n#[repr(transparent)]\npub struct Bitmap<T: BitmapItem> {\n    pub bitmap: u64,\n    #[serde(skip)]\n    #[rkyv(omit_bounds)]\n    _state: std::marker::PhantomData<T>,\n}\n\npub trait BitmapItem: From<u64> + Into<u64> + Sized + Copy {\n    fn max() -> u64;\n    fn is_valid(&self) -> bool;\n}\n\npub trait BitPop {\n    fn bit_push(&mut self, item: u8);\n    fn bit_pop(&mut self) -> Option<u8>;\n}\n\nimpl<T: BitmapItem> Bitmap<T> {\n    pub fn new() -> Self {\n        Self::default()\n    }\n\n    #[inline(always)]\n    pub fn all() -> Self {\n        Self {\n            bitmap: u64::MAX >> (64 - T::max()),\n            _state: std::marker::PhantomData,\n        }\n    }\n\n    #[inline(always)]\n    pub fn union(&mut self, items: &Bitmap<T>) {\n        self.bitmap |= items.bitmap;\n    }\n\n    #[inline(always)]\n    pub fn union_raw(&mut self, items: impl Into<u64>) {\n        self.bitmap |= items.into();\n    }\n\n    #[inline(always)]\n    pub fn intersection(&mut self, items: &Bitmap<T>) {\n        self.bitmap &= items.bitmap;\n    }\n\n    #[inline(always)]\n    pub fn insert(&mut self, item: T) {\n        debug_assert!(item.is_valid());\n        self.bitmap |= 1 << item.into();\n    }\n\n    pub fn insert_many(&mut self, items: impl IntoIterator<Item = T>) {\n        for item in items.into_iter() {\n            self.insert(item);\n        }\n    }\n\n    #[inline(always)]\n    pub fn with_item(mut self, item: T) -> Self {\n        self.insert(item);\n        self\n    }\n\n    #[inline(always)]\n    pub fn remove(&mut self, item: T) {\n        debug_assert!(item.is_valid());\n        self.bitmap ^= 1 << item.into();\n    }\n\n    #[inline(always)]\n    pub fn pop(&mut self) -> Option<T> {\n        if self.bitmap != 0 {\n            let item = 63 - self.bitmap.leading_zeros();\n            self.bitmap ^= 1 << item;\n            Some((item as u64).into())\n        } else {\n            None\n        }\n    }\n\n    #[inline(always)]\n    pub fn contains(&self, item: T) -> bool {\n        self.bitmap & (1 << item.into()) != 0\n    }\n\n    #[inline(always)]\n    pub fn contains_any(&self, items: impl Iterator<Item = T>) -> bool {\n        for item in items {\n            if self.bitmap & (1 << item.into()) != 0 {\n                return true;\n            }\n        }\n        false\n    }\n\n    #[inline(always)]\n    pub fn contains_all(&self, items: impl Iterator<Item = T>) -> bool {\n        if !self.is_empty() {\n            for item in items {\n                if self.bitmap & (1 << item.into()) == 0 {\n                    return false;\n                }\n            }\n            true\n        } else {\n            false\n        }\n    }\n\n    #[inline(always)]\n    pub fn is_empty(&self) -> bool {\n        self.bitmap == 0\n    }\n\n    #[inline(always)]\n    pub fn clear(&mut self) -> Self {\n        let bitmap = self.bitmap;\n        self.bitmap = 0;\n        Bitmap {\n            bitmap,\n            _state: std::marker::PhantomData,\n        }\n    }\n\n    pub fn into_inner(self) -> u64 {\n        self.bitmap\n    }\n}\n\nimpl BitPop for u32 {\n    fn bit_push(&mut self, item: u8) {\n        *self |= 1 << item;\n    }\n\n    fn bit_pop(&mut self) -> Option<u8> {\n        if *self != 0 {\n            let item = 31 - self.leading_zeros();\n            *self ^= 1 << item;\n            Some(item as u8)\n        } else {\n            None\n        }\n    }\n}\n\nimpl BitPop for u64 {\n    fn bit_push(&mut self, item: u8) {\n        *self |= 1 << item;\n    }\n\n    fn bit_pop(&mut self) -> Option<u8> {\n        if *self != 0 {\n            let item = 63 - self.leading_zeros();\n            *self ^= 1 << item;\n            Some(item as u8)\n        } else {\n            None\n        }\n    }\n}\n\nimpl<T: BitmapItem> From<ArchivedBitmap<T>> for Bitmap<T> {\n    fn from(value: ArchivedBitmap<T>) -> Self {\n        Self {\n            bitmap: value.bitmap.into(),\n            _state: std::marker::PhantomData,\n        }\n    }\n}\n\nimpl<T: BitmapItem> From<&ArchivedBitmap<T>> for Bitmap<T> {\n    fn from(value: &ArchivedBitmap<T>) -> Self {\n        Self {\n            bitmap: value.bitmap.into(),\n            _state: std::marker::PhantomData,\n        }\n    }\n}\n\nimpl<T: BitmapItem> From<u64> for Bitmap<T> {\n    fn from(value: u64) -> Self {\n        Self {\n            bitmap: value,\n            _state: std::marker::PhantomData,\n        }\n    }\n}\n\nimpl<T: BitmapItem> AsRef<u64> for Bitmap<T> {\n    fn as_ref(&self) -> &u64 {\n        &self.bitmap\n    }\n}\n\nimpl<T: BitmapItem> Deref for Bitmap<T> {\n    type Target = u64;\n\n    fn deref(&self) -> &Self::Target {\n        &self.bitmap\n    }\n}\n\nimpl<T: BitmapItem> From<Bitmap<T>> for u64 {\n    fn from(value: Bitmap<T>) -> Self {\n        value.bitmap\n    }\n}\n\nimpl<T: BitmapItem> Iterator for Bitmap<T> {\n    type Item = T;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        if self.bitmap != 0 {\n            let item = 63 - self.bitmap.leading_zeros();\n            self.bitmap ^= 1 << item;\n            Some((item as u64).into())\n        } else {\n            None\n        }\n    }\n}\n\nimpl<T: BitmapItem> From<Vec<T>> for Bitmap<T> {\n    fn from(values: Vec<T>) -> Self {\n        let mut bitmap = Bitmap::default();\n        for value in values {\n            if value.is_valid() {\n                bitmap.insert(value);\n            }\n        }\n        bitmap\n    }\n}\n\nimpl<T: BitmapItem> FromIterator<T> for Bitmap<T> {\n    fn from_iter<U: IntoIterator<Item = T>>(iter: U) -> Self {\n        let mut bitmap = Bitmap::new();\n        for value in iter {\n            if value.is_valid() {\n                bitmap.insert(value);\n            }\n        }\n        bitmap\n    }\n}\n\nimpl<T: BitmapItem> From<&Vec<T>> for Bitmap<T> {\n    fn from(values: &Vec<T>) -> Self {\n        let mut bitmap = Bitmap::default();\n        for value in values {\n            if value.is_valid() {\n                bitmap.insert(*value);\n            }\n        }\n        bitmap\n    }\n}\n\nimpl<T: BitmapItem> From<T> for Bitmap<T> {\n    fn from(value: T) -> Self {\n        let mut bitmap = Bitmap::default();\n        bitmap.insert(value);\n        bitmap\n    }\n}\n\nimpl<T: BitmapItem> From<Bitmap<T>> for Vec<T> {\n    fn from(values: Bitmap<T>) -> Self {\n        let mut list = Vec::new();\n        for item in values {\n            list.push(item);\n        }\n        list\n    }\n}\n\nimpl<T: BitmapItem> Default for Bitmap<T> {\n    fn default() -> Self {\n        Bitmap {\n            bitmap: 0,\n            _state: std::marker::PhantomData,\n        }\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/map/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod bitmap;\npub mod mutex_map;\npub mod vec_map;\n"
  },
  {
    "path": "crates/utils/src/map/mutex_map.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse core::hash::Hash;\nuse std::hash::Hasher;\n\nuse ahash::AHasher;\nuse tokio::sync::{Mutex, MutexGuard};\n\npub struct MutexMap<T: Default> {\n    map: Box<[Mutex<T>]>,\n    mask: u64,\n    hasher: AHasher,\n}\n\npub struct MutexMapLockError;\npub type Result<T> = std::result::Result<T, MutexMapLockError>;\n\n#[allow(clippy::mutex_atomic)]\nimpl<T: Default> MutexMap<T> {\n    pub fn with_capacity(size: usize) -> MutexMap<T> {\n        let size = size.next_power_of_two();\n        MutexMap {\n            map: (0..size)\n                .map(|_| T::default().into())\n                .collect::<Vec<Mutex<T>>>()\n                .into_boxed_slice(),\n            mask: (size - 1) as u64,\n            hasher: AHasher::default(),\n        }\n    }\n\n    pub async fn lock<U>(&self, key: U) -> MutexGuard<'_, T>\n    where\n        U: Into<u64> + Copy,\n    {\n        let hash = key.into() & self.mask;\n        self.map[hash as usize].lock().await\n    }\n\n    /*pub async fn try_lock<U>(&self, key: U, timeout: Duration) -> Option<MutexGuard<'_, T>>\n    where\n        U: Into<u64> + Copy,\n    {\n        let hash = key.into() & self.mask;\n        self.map[hash as usize].try_lock(timeout).await\n    }*/\n\n    pub async fn lock_hash<U>(&self, key: U) -> MutexGuard<'_, T>\n    where\n        U: Hash,\n    {\n        let mut hasher = self.hasher.clone();\n        key.hash(&mut hasher);\n        let hash = hasher.finish() & self.mask;\n        self.map[hash as usize].lock().await\n    }\n\n    /*pub async fn try_lock_hash<U>(&self, key: U, timeout: Duration) -> Option<MutexGuard<'_, T>>\n    where\n        U: Hash,\n    {\n        let mut hasher = self.hasher.clone();\n        key.hash(&mut hasher);\n        let hash = hasher.finish() & self.mask;\n        self.map[hash as usize].try_lock_for(timeout).await\n    }*/\n}\n"
  },
  {
    "path": "crates/utils/src/map/vec_map.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse rkyv::Archive;\nuse serde::{Deserialize, Serialize, ser::SerializeMap};\nuse std::{borrow::Borrow, cmp::Ordering, fmt, hash::Hash};\n\n// A map implemented using vectors\n// used for small datasets of less than 20 items\n// and when deserializing from JSON\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq)]\npub struct VecMap<K: Eq + PartialEq, V> {\n    inner: Vec<KeyValue<K, V>>,\n}\n\n#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Clone, PartialEq, Eq, Hash)]\npub struct KeyValue<K: Eq + PartialEq, V> {\n    key: K,\n    value: V,\n}\n\nimpl<K: Eq + PartialEq, V> Default for VecMap<K, V> {\n    fn default() -> Self {\n        VecMap { inner: Vec::new() }\n    }\n}\n\nimpl<K: Eq + PartialEq, V> VecMap<K, V> {\n    pub fn new() -> Self {\n        Self::default()\n    }\n\n    pub fn with_capacity(capacity: usize) -> Self {\n        Self {\n            inner: Vec::with_capacity(capacity),\n        }\n    }\n\n    #[inline(always)]\n    pub fn set(&mut self, key: K, value: V) -> bool {\n        if let Some(kv) = self.inner.iter_mut().find(|kv| kv.key == key) {\n            kv.value = value;\n            false\n        } else {\n            self.inner.push(KeyValue { key, value });\n            true\n        }\n    }\n\n    #[inline(always)]\n    pub fn append(&mut self, key: K, value: V) {\n        self.inner.push(KeyValue { key, value });\n    }\n\n    #[inline(always)]\n    pub fn with_append(mut self, key: K, value: V) -> Self {\n        self.append(key, value);\n        self\n    }\n\n    #[inline(always)]\n    pub fn insert(&mut self, idx: usize, key: K, value: V) {\n        self.inner.insert(idx, KeyValue { key, value });\n    }\n\n    #[inline(always)]\n    pub fn get<Q: ?Sized>(&self, key: &Q) -> Option<&V>\n    where\n        K: Borrow<Q> + PartialEq<Q>,\n    {\n        self.inner.iter().find_map(|kv| {\n            if &kv.key == key {\n                Some(&kv.value)\n            } else {\n                None\n            }\n        })\n    }\n\n    #[inline(always)]\n    pub fn get_mut(&mut self, key: &K) -> Option<&mut V> {\n        self.inner.iter_mut().find_map(|kv| {\n            if &kv.key == key {\n                Some(&mut kv.value)\n            } else {\n                None\n            }\n        })\n    }\n\n    #[inline(always)]\n    pub fn contains_key(&self, key: &K) -> bool {\n        self.inner.iter().any(|kv| kv.key == *key)\n    }\n\n    #[inline(always)]\n    pub fn remove<Q: ?Sized>(&mut self, key: &Q) -> Option<V>\n    where\n        K: Borrow<Q> + PartialEq<Q>,\n    {\n        self.inner\n            .iter()\n            .position(|kv| kv.key == *key)\n            .map(|pos| self.inner.remove(pos).value)\n    }\n\n    #[inline(always)]\n    pub fn remove_all(&mut self, key: &K) {\n        self.inner.retain(|kv| kv.key != *key);\n    }\n\n    #[inline(always)]\n    pub fn remove_entry(&mut self, key: &K) -> Option<(K, V)> {\n        self.inner.iter().position(|k| &k.key == key).map(|pos| {\n            let kv = self.inner.remove(pos);\n            (kv.key, kv.value)\n        })\n    }\n\n    #[inline(always)]\n    pub fn swap_remove(&mut self, index: usize) -> V {\n        self.inner.swap_remove(index).value\n    }\n\n    #[inline(always)]\n    pub fn is_empty(&self) -> bool {\n        self.inner.is_empty()\n    }\n\n    #[inline(always)]\n    pub fn len(&self) -> usize {\n        self.inner.len()\n    }\n\n    #[inline(always)]\n    pub fn clear(&mut self) {\n        self.inner.clear();\n    }\n\n    #[inline(always)]\n    pub fn iter(&self) -> impl Iterator<Item = (&K, &V)> {\n        self.inner.iter().map(|kv| (&kv.key, &kv.value))\n    }\n\n    #[inline(always)]\n    pub fn iter_by_key<'x, 'y: 'x>(&'x self, key: &'y K) -> impl Iterator<Item = &'x V> + 'x {\n        self.inner.iter().filter_map(move |kv| {\n            if &kv.key == key {\n                Some(&kv.value)\n            } else {\n                None\n            }\n        })\n    }\n\n    #[inline(always)]\n    pub fn iter_mut(&mut self) -> impl Iterator<Item = (&mut K, &mut V)> {\n        self.inner.iter_mut().map(|kv| (&mut kv.key, &mut kv.value))\n    }\n\n    #[inline(always)]\n    pub fn iter_mut_by_key<'x, 'y: 'x>(\n        &'x mut self,\n        key: &'y K,\n    ) -> impl Iterator<Item = &'x mut V> + 'x {\n        self.inner.iter_mut().filter_map(move |kv| {\n            if &kv.key == key {\n                Some(&mut kv.value)\n            } else {\n                None\n            }\n        })\n    }\n\n    #[inline(always)]\n    pub fn keys(&self) -> impl Iterator<Item = &K> {\n        self.inner.iter().map(|kv| &kv.key)\n    }\n\n    #[inline(always)]\n    pub fn values(&self) -> impl Iterator<Item = &V> {\n        self.inner.iter().map(|kv| &kv.value)\n    }\n\n    #[inline(always)]\n    pub fn values_mut(&mut self) -> impl Iterator<Item = &mut V> {\n        self.inner.iter_mut().map(|kv| &mut kv.value)\n    }\n\n    pub fn get_mut_or_insert_with(&mut self, key: K, fnc: impl FnOnce() -> V) -> &mut V {\n        if let Some(pos) = self.inner.iter().position(|kv| kv.key == key) {\n            &mut self.inner[pos].value\n        } else {\n            self.inner.push(KeyValue { key, value: fnc() });\n            &mut self.inner.last_mut().unwrap().value\n        }\n    }\n\n    pub fn with_key_value(mut self, key: K, value: V) -> Self {\n        self.append(key, value);\n        self\n    }\n\n    pub fn sort_unstable(&mut self)\n    where\n        K: Ord,\n        V: Ord,\n    {\n        self.inner.sort_unstable_by(|a, b| match a.key.cmp(&b.key) {\n            Ordering::Equal => a.value.cmp(&b.value),\n            cmp => cmp,\n        });\n    }\n}\n\nimpl<K: Eq + PartialEq, V: Default> VecMap<K, V> {\n    pub fn get_mut_or_insert(&mut self, key: K) -> &mut V {\n        if let Some(pos) = self.inner.iter().position(|kv| kv.key == key) {\n            &mut self.inner[pos].value\n        } else {\n            self.inner.push(KeyValue {\n                key,\n                value: V::default(),\n            });\n            &mut self.inner.last_mut().unwrap().value\n        }\n    }\n}\n\nimpl<K: Archive + Eq + PartialEq, V: Archive> ArchivedVecMap<K, V> {\n    pub fn len(&self) -> usize {\n        self.inner.len()\n    }\n\n    pub fn is_empty(&self) -> bool {\n        self.inner.is_empty()\n    }\n\n    #[inline(always)]\n    pub fn iter(\n        &self,\n    ) -> impl Iterator<\n        Item = (\n            &<K as rkyv::Archive>::Archived,\n            &<V as rkyv::Archive>::Archived,\n        ),\n    > {\n        self.inner.iter().map(|kv| (&kv.key, &kv.value))\n    }\n}\n\nimpl<K: Eq + PartialEq, V> IntoIterator for VecMap<K, V> {\n    type Item = (K, V);\n\n    type IntoIter =\n        std::iter::Map<std::vec::IntoIter<KeyValue<K, V>>, fn(KeyValue<K, V>) -> (K, V)>;\n\n    fn into_iter(self) -> Self::IntoIter {\n        self.inner.into_iter().map(|kv| (kv.key, kv.value))\n    }\n}\n\nimpl<'x, K: Eq + PartialEq, V> IntoIterator for &'x VecMap<K, V> {\n    type Item = (&'x K, &'x V);\n\n    type IntoIter = std::iter::Map<\n        std::slice::Iter<'x, KeyValue<K, V>>,\n        fn(&'x KeyValue<K, V>) -> (&'x K, &'x V),\n    >;\n\n    fn into_iter(self) -> Self::IntoIter {\n        self.inner.iter().map(|kv| (&kv.key, &kv.value))\n    }\n}\n\nimpl<K, V> Hash for VecMap<K, V>\nwhere\n    K: Eq + PartialEq + Hash,\n    V: Hash,\n{\n    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {\n        self.inner.hash(state);\n    }\n}\n\nimpl<K: Eq + PartialEq, V> FromIterator<(K, V)> for VecMap<K, V> {\n    fn from_iter<T>(iter: T) -> Self\n    where\n        T: IntoIterator<Item = (K, V)>,\n    {\n        let mut map = VecMap::new();\n        for (k, v) in iter {\n            map.append(k, v);\n        }\n        map\n    }\n}\n\nstruct VecMapVisitor<K, V> {\n    phantom: std::marker::PhantomData<(K, V)>,\n}\n\nimpl<'de, K: Eq + PartialEq + Deserialize<'de>, V: Deserialize<'de>> serde::de::Visitor<'de>\n    for VecMapVisitor<K, V>\n{\n    type Value = VecMap<K, V>;\n\n    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {\n        formatter.write_str(\"a valid map\")\n    }\n\n    fn visit_map<A>(self, mut map: A) -> Result<Self::Value, A::Error>\n    where\n        A: serde::de::MapAccess<'de>,\n    {\n        // Duplicates are not checked during deserialization\n        let mut vec_map = VecMap::new();\n        while let Some(key) = map.next_key::<K>()? {\n            vec_map.append(key, map.next_value()?);\n        }\n        Ok(vec_map)\n    }\n}\n\nimpl<'de, K: Eq + PartialEq + Deserialize<'de>, V: Deserialize<'de>> Deserialize<'de>\n    for VecMap<K, V>\n{\n    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>\n    where\n        D: serde::Deserializer<'de>,\n    {\n        deserializer.deserialize_map(VecMapVisitor {\n            phantom: std::marker::PhantomData,\n        })\n    }\n}\n\nimpl<K: Eq + PartialEq + Serialize, V: Serialize> Serialize for VecMap<K, V> {\n    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    where\n        S: serde::Serializer,\n    {\n        let mut map = serializer.serialize_map(self.len().into())?;\n\n        for (key, value) in self {\n            map.serialize_entry(key, value)?\n        }\n\n        map.end()\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/snowflake.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    sync::atomic::{AtomicU64, Ordering},\n    time::{Duration, SystemTime},\n};\n\n#[derive(Debug)]\npub struct SnowflakeIdGenerator {\n    epoch: SystemTime,\n    node_id: u64,\n    sequence: AtomicU64,\n}\n\nconst SEQUENCE_LEN: u64 = 12;\nconst NODE_ID_LEN: u64 = 9;\n\nconst SEQUENCE_MASK: u64 = (1 << SEQUENCE_LEN) - 1;\nconst NODE_ID_MASK: u64 = (1 << NODE_ID_LEN) - 1;\n\nconst DEFAULT_EPOCH: u64 = 1632280000; // 52 years after UNIX_EPOCH\n//const DEFAULT_EPOCH_MS: u128 = (DEFAULT_EPOCH as u128) * 1000; // 52 years after UNIX_EPOCH in milliseconds\n\n/*\n\nID characteristics:\n\n- 43 bits for milliseconds since January 1st, 2022: 2^43 / (1000 * 60 * 60 * 24 * 365) = 278.92 years (from year 2022 until 2300)\n- 9 bits for a node id: 2^9 = 512 nodes\n- 12 bits for a sequence number: 2^12 = 4096 ids per millisecond\n\n*/\n\nimpl SnowflakeIdGenerator {\n    pub fn new() -> Self {\n        Self::with_node_id(rand::random::<u64>())\n    }\n\n    pub fn from_duration(period: Duration) -> Option<u64> {\n        (SystemTime::UNIX_EPOCH + Duration::from_secs(DEFAULT_EPOCH))\n            .elapsed()\n            .ok()\n            .and_then(|elapsed| elapsed.checked_sub(period))\n            .map(|elapsed| (elapsed.as_millis() as u64) << (SEQUENCE_LEN + NODE_ID_LEN))\n    }\n\n    pub fn from_timestamp(timestamp: u64) -> Option<u64> {\n        SystemTime::now()\n            .duration_since(SystemTime::UNIX_EPOCH)\n            .ok()\n            .and_then(|now| now.as_secs().checked_sub(timestamp))\n            .and_then(|diff| Self::from_duration(Duration::from_secs(diff)))\n    }\n\n    pub fn from_sequence_and_node_id(sequence: u64, node_id: Option<u64>) -> Option<u64> {\n        let node_id = node_id.unwrap_or_else(rand::random::<u64>);\n        let sequence = sequence & SEQUENCE_MASK;\n\n        (SystemTime::UNIX_EPOCH + Duration::from_secs(DEFAULT_EPOCH))\n            .elapsed()\n            .ok()\n            .map(|elapsed| {\n                ((elapsed.as_millis() as u64) << (SEQUENCE_LEN + NODE_ID_LEN))\n                    | (sequence << NODE_ID_LEN)\n                    | (node_id & NODE_ID_MASK)\n            })\n    }\n\n    pub fn to_timestamp(id: u64) -> u64 {\n        (id >> (SEQUENCE_LEN + NODE_ID_LEN)) / 1000 + DEFAULT_EPOCH\n    }\n\n    pub fn with_node_id(node_id: u64) -> Self {\n        Self {\n            epoch: SystemTime::UNIX_EPOCH + Duration::from_secs(DEFAULT_EPOCH), // 52 years after UNIX_EPOCH\n            node_id,\n            sequence: 0.into(),\n        }\n    }\n\n    #[inline(always)]\n    pub fn past_id(&self, period: Duration) -> Option<u64> {\n        self.epoch\n            .elapsed()\n            .ok()\n            .and_then(|elapsed| elapsed.checked_sub(period))\n            .map(|elapsed| (elapsed.as_millis() as u64) << (SEQUENCE_LEN + NODE_ID_LEN))\n    }\n\n    pub fn is_valid(&self) -> bool {\n        self.epoch.elapsed().is_ok()\n    }\n\n    #[inline(always)]\n    pub fn generate(&self) -> u64 {\n        let elapsed = self\n            .epoch\n            .elapsed()\n            .map(|e| e.as_millis())\n            .unwrap_or_default() as u64;\n        let sequence = self.sequence.fetch_add(1, Ordering::Relaxed) & SEQUENCE_MASK;\n\n        (elapsed << (SEQUENCE_LEN + NODE_ID_LEN))\n            | (sequence << NODE_ID_LEN)\n            | (self.node_id & NODE_ID_MASK)\n    }\n}\n\nimpl Default for SnowflakeIdGenerator {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n\nimpl Clone for SnowflakeIdGenerator {\n    fn clone(&self) -> Self {\n        Self {\n            epoch: self.epoch,\n            node_id: self.node_id,\n            sequence: 0.into(),\n        }\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/suffixlist.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::io::Read;\n\nuse ahash::AHashSet;\nuse mail_auth::flate2::read::GzDecoder;\n\nuse crate::config::Config;\n\n#[derive(Debug, Clone, Default)]\npub struct PublicSuffix {\n    pub suffixes: AHashSet<String>,\n    pub exceptions: AHashSet<String>,\n    pub wildcards: Vec<String>,\n}\n\n#[derive(PartialEq, Eq, Clone, Copy)]\npub enum DomainPart {\n    Sld,\n    Tld,\n    Host,\n}\n\nimpl PublicSuffix {\n    pub fn contains(&self, suffix: &str) -> bool {\n        self.suffixes.contains(suffix)\n            || (!self.exceptions.contains(suffix)\n                && self.wildcards.iter().any(|w| suffix.ends_with(w)))\n    }\n\n    pub fn domain_part(&self, domain: &str, part: DomainPart) -> Option<String> {\n        let d = domain.trim().to_lowercase();\n        let mut seen_dot = false;\n        for (pos, ch) in d.as_bytes().iter().enumerate().rev() {\n            if *ch == b'.' {\n                if seen_dot {\n                    let maybe_domain =\n                        std::str::from_utf8(&d.as_bytes()[pos + 1..]).unwrap_or_default();\n                    if !self.contains(maybe_domain) {\n                        return if part == DomainPart::Sld {\n                            maybe_domain\n                        } else {\n                            std::str::from_utf8(&d.as_bytes()[..pos]).unwrap_or_default()\n                        }\n                        .to_string()\n                        .into();\n                    }\n                } else if part == DomainPart::Tld {\n                    return std::str::from_utf8(&d.as_bytes()[pos + 1..])\n                        .unwrap_or_default()\n                        .to_string()\n                        .into();\n                } else {\n                    seen_dot = true;\n                }\n            }\n        }\n\n        if seen_dot {\n            if part == DomainPart::Sld {\n                d.into()\n            } else {\n                None\n            }\n        } else if part == DomainPart::Host {\n            d.into()\n        } else {\n            None\n        }\n    }\n}\n\nimpl From<&str> for PublicSuffix {\n    fn from(list: &str) -> Self {\n        let mut ps = PublicSuffix::default();\n        for line in list.lines() {\n            let line = line.trim().to_lowercase();\n            if !line.starts_with(\"//\") {\n                if let Some(domain) = line.strip_prefix('*') {\n                    ps.wildcards.push(domain.to_string());\n                } else if let Some(domain) = line.strip_prefix('!') {\n                    ps.exceptions.insert(domain.to_string());\n                } else {\n                    ps.suffixes.insert(line.to_string());\n                }\n            }\n        }\n        ps.suffixes.insert(\"onion\".to_string());\n        ps\n    }\n}\n\nimpl PublicSuffix {\n    #[allow(unused_variables)]\n    pub async fn parse(config: &mut Config, key: &str) -> PublicSuffix {\n        let mut values = config\n            .values(key)\n            .map(|(_, s)| s.to_string())\n            .collect::<Vec<_>>();\n        if values.is_empty() {\n            values = vec![\n                \"https://publicsuffix.org/list/public_suffix_list.dat\".to_string(),\n                \"https://raw.githubusercontent.com/publicsuffix/list/master/public_suffix_list.dat\"\n                    .to_string(),\n            ]\n        }\n\n        for (idx, value) in values.into_iter().enumerate() {\n            let bytes = if value.starts_with(\"https://\") || value.starts_with(\"http://\") {\n                let result = match reqwest::get(&value).await {\n                    Ok(r) => {\n                        if r.status().is_success() {\n                            r.bytes().await\n                        } else {\n                            config.new_build_warning(\n                                format!(\"{value}.{idx}\"),\n                                format!(\n                                    \"Failed to fetch public suffixes from {value:?}: Status {status}\",\n                                    value = value,\n                                    status = r.status()\n                                ),\n                            );\n                            continue;\n                        }\n                    }\n                    Err(err) => Err(err),\n                };\n\n                match result {\n                    Ok(bytes) => bytes.to_vec(),\n                    Err(err) => {\n                        config.new_build_warning(\n                            format!(\"{value}.{idx}\"),\n                            format!(\"Failed to fetch public suffixes from {value:?}: {err}\",),\n                        );\n                        continue;\n                    }\n                }\n            } else if let Some(filename) = value.strip_prefix(\"file://\") {\n                match std::fs::read(filename) {\n                    Ok(bytes) => bytes,\n                    Err(err) => {\n                        config.new_build_warning(\n                            format!(\"{value}.{idx}\"),\n                            format!(\"Failed to read public suffixes from {value:?}: {err}\",),\n                        );\n                        continue;\n                    }\n                }\n            } else {\n                config.new_parse_error(key, format!(\"Invalid public suffix file {value:?}\"));\n                continue;\n            };\n            let bytes = if value.ends_with(\".gz\") {\n                match GzDecoder::new(&bytes[..])\n                    .bytes()\n                    .collect::<Result<Vec<_>, _>>()\n                {\n                    Ok(bytes) => bytes,\n                    Err(err) => {\n                        config.new_build_warning(\n                            format!(\"{value}.{idx}\"),\n                            format!(\n                                \"Failed to decompress public suffixes from {value:?}: {err}\",\n                                value = value,\n                                err = err\n                            ),\n                        );\n                        continue;\n                    }\n                }\n            } else {\n                bytes\n            };\n\n            match String::from_utf8(bytes) {\n                Ok(list) => {\n                    return PublicSuffix::from(list.as_str());\n                }\n                Err(err) => {\n                    config.new_build_warning(\n                        format!(\"{value}.{idx}\"),\n                        format!(\n                            \"Failed to parse public suffixes from {value:?}: {err}\",\n                            value = value,\n                            err = err\n                        ),\n                    );\n                }\n            }\n        }\n\n        #[cfg(not(feature = \"test_mode\"))]\n        config.new_build_warning(key, \"Failed to parse public suffixes from any source.\");\n\n        PublicSuffix::default()\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/template.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashMap;\nuse std::{hash::Hash, str::FromStr};\n\n#[derive(Debug, Clone, PartialEq, Eq, Default)]\npub struct Template<T> {\n    pub items: Vec<TemplateItem<T>>,\n    pub size: usize,\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum TemplateItem<T> {\n    Static(String),\n    Variable { name: T, escape: bool },\n    If { variable: T, block_end: usize },\n    ForEach { variable: T, block_end: usize },\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum Variable<T: Eq + Hash, V: AsRef<str>> {\n    Single(V),\n    Block(Vec<AHashMap<T, V>>),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub struct Variables<T: Eq + Hash, V: AsRef<str>> {\n    pub items: AHashMap<T, Variable<T, V>>,\n}\n\nimpl<T: FromStr + Eq + Hash + std::fmt::Debug> Template<T> {\n    pub fn parse(mut template: &str) -> Result<Self, String> {\n        let mut items = Vec::new();\n        let mut block_stack = vec![];\n        let mut size = 0;\n\n        loop {\n            if let Some((start, end)) = template.split_once(\"{{\") {\n                if !start.is_empty() {\n                    items.push(TemplateItem::Static(start.to_string()));\n                    size += start.len();\n                }\n                let (var, rest) = end.split_once(\"}}\").ok_or(\"Unmatched {{\")?;\n                template = rest;\n                let var = var.trim();\n                if let Some(var_name) = var.strip_prefix(\"#\").map(|v| v.trim()) {\n                    let (is_each, var_name) = if let Some(each) = var_name.strip_prefix(\"each \") {\n                        (true, each)\n                    } else if let Some(if_cond) = var_name.strip_prefix(\"if \") {\n                        (false, if_cond)\n                    } else {\n                        return Err(format!(\"Invalid block start: {}\", var_name));\n                    };\n                    let var = T::from_str(var_name)\n                        .map_err(|_| format!(\"Invalid variable: {}\", var_name))?;\n\n                    block_stack.push((var_name, items.len()));\n\n                    if is_each {\n                        items.push(TemplateItem::ForEach {\n                            variable: var,\n                            block_end: 0,\n                        });\n                    } else {\n                        items.push(TemplateItem::If {\n                            variable: var,\n                            block_end: 0,\n                        });\n                    }\n                } else if let Some(var_name) = var.strip_prefix(\"/\").map(|v| v.trim()) {\n                    let (is_each, var_name) = if let Some(each) = var_name.strip_prefix(\"each \") {\n                        (true, each)\n                    } else if let Some(if_cond) = var_name.strip_prefix(\"if \") {\n                        (false, if_cond)\n                    } else {\n                        return Err(format!(\"Invalid block end: {}\", var_name));\n                    };\n\n                    if let Some((expected_name, if_pos)) = block_stack.pop() {\n                        if expected_name != var_name {\n                            return Err(format!(\n                                \"Block end does not match start: expected {}, got {}\",\n                                expected_name, var_name\n                            ));\n                        }\n                        let block_end_idx = items.len();\n                        match &mut items[if_pos] {\n                            TemplateItem::If { block_end, .. } if !is_each => {\n                                *block_end = block_end_idx;\n                            }\n                            TemplateItem::ForEach { block_end, .. } if is_each => {\n                                *block_end = block_end_idx;\n                            }\n                            _ => {\n                                return Err(format!(\n                                    \"Block end does not match start type for {}\",\n                                    var_name\n                                ));\n                            }\n                        }\n                    }\n                } else {\n                    let (name, escape) = var.strip_prefix(\"!\").map_or((var, true), |v| (v, false));\n                    let name =\n                        T::from_str(name).map_err(|_| format!(\"Invalid variable: {}\", name))?;\n                    items.push(TemplateItem::Variable { name, escape });\n                }\n            } else {\n                if !template.is_empty() {\n                    items.push(TemplateItem::Static(template.to_string()));\n                    size += template.len();\n                }\n                break;\n            }\n        }\n\n        if block_stack.is_empty() {\n            Ok(Template { items, size })\n        } else {\n            Err(format!(\"Unmatched {{: {}\", block_stack.last().unwrap().0))\n        }\n    }\n\n    pub fn eval<V>(&self, variables: &Variables<T, V>) -> String\n    where\n        V: AsRef<str>,\n    {\n        let mut result = String::with_capacity(self.size);\n        let mut items = self.items.iter().enumerate();\n        let mut base_offset = 0;\n\n        while let Some((idx, item)) = items.next() {\n            let idx = idx + base_offset;\n            match item {\n                TemplateItem::Static(s) => result.push_str(s),\n                TemplateItem::Variable { name, escape } => {\n                    if let Some(Variable::Single(variable)) = variables.items.get(name) {\n                        if *escape {\n                            html_escape(&mut result, variable.as_ref())\n                        } else {\n                            result.push_str(variable.as_ref());\n                        }\n                    }\n                }\n                TemplateItem::If {\n                    variable,\n                    block_end,\n                } => {\n                    if !variables.items.contains_key(variable) {\n                        items = self.items[*block_end..].iter().enumerate();\n                        base_offset = *block_end;\n                    }\n                }\n                TemplateItem::ForEach {\n                    variable,\n                    block_end,\n                } => {\n                    if let Some(Variable::Block(entries)) = variables.items.get(variable) {\n                        let slice = &self.items[idx + 1..*block_end];\n                        for entry in entries {\n                            let mut slice = slice.iter();\n                            while let Some(sub_item) = slice.next() {\n                                match sub_item {\n                                    TemplateItem::Static(s) => result.push_str(s),\n                                    TemplateItem::Variable { name, escape } => {\n                                        if let Some(variable) = entry.get(name) {\n                                            if *escape {\n                                                html_escape(&mut result, variable.as_ref())\n                                            } else {\n                                                result.push_str(variable.as_ref());\n                                            }\n                                        }\n                                    }\n                                    TemplateItem::If {\n                                        variable,\n                                        block_end: start_pos,\n                                    } => {\n                                        if !entry.contains_key(variable) {\n                                            slice = self.items[*start_pos..*block_end].iter();\n                                        }\n                                    }\n                                    _ => {}\n                                }\n                            }\n                        }\n                    }\n                    items = self.items[*block_end..].iter().enumerate();\n                    base_offset = *block_end;\n                }\n            }\n        }\n\n        result\n    }\n}\n\nfn html_escape(result: &mut String, input: &str) {\n    for c in input.chars() {\n        match c {\n            '&' => result.push_str(\"&amp;\"),\n            '<' => result.push_str(\"&lt;\"),\n            '>' => result.push_str(\"&gt;\"),\n            '\"' => result.push_str(\"&quot;\"),\n            '\\'' => result.push_str(\"&#39;\"),\n            _ => result.push(c),\n        }\n    }\n}\n\nimpl<T: Eq + Hash, V: AsRef<str>> Variables<T, V> {\n    pub fn new() -> Self {\n        Self {\n            items: AHashMap::new(),\n        }\n    }\n\n    pub fn insert_single(&mut self, key: T, value: V) {\n        self.items.insert(key, Variable::Single(value));\n    }\n\n    pub fn insert_block<V1, V2>(&mut self, key: T, value: V1)\n    where\n        V1: IntoIterator<Item = V2>,\n        V2: IntoIterator<Item = (T, V)>,\n    {\n        self.items.insert(\n            key,\n            Variable::Block(value.into_iter().map(AHashMap::from_iter).collect()),\n        );\n    }\n}\n\nimpl<T: Eq + Hash, V: AsRef<str>> Default for Variables<T, V> {\n    fn default() -> Self {\n        Self::new()\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn test_simple_variable_substitution() {\n        let template = Template::parse(\"Hello {{name}}!\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"name\".to_string(), \"World\".to_string());\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Hello World!\");\n    }\n\n    #[test]\n    fn test_multiple_variables() {\n        let template = Template::parse(\"{{greeting}} {{name}}, today is {{day}}\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"greeting\".to_string(), \"Hello\".to_string());\n        vars.insert_single(\"name\".to_string(), \"Alice\".to_string());\n        vars.insert_single(\"day\".to_string(), \"Monday\".to_string());\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Hello Alice, today is Monday\");\n    }\n\n    #[test]\n    fn test_missing_variable() {\n        let template = Template::parse(\"Hello {{name}}!\").unwrap();\n        let vars = Variables::<String, String>::new();\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Hello !\");\n    }\n\n    #[test]\n    fn test_static_text_only() {\n        let template = Template::parse(\"This is just static text\").unwrap();\n        let vars = Variables::<String, String>::new();\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"This is just static text\");\n    }\n\n    #[test]\n    fn test_empty_template() {\n        let template = Template::parse(\"\").unwrap();\n        let vars = Variables::<String, String>::new();\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"\");\n    }\n\n    #[test]\n    fn test_if_block_with_existing_variable() {\n        let template =\n            Template::parse(\"{{#if show_message}}Hello World!{{/if show_message}}\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"show_message\".to_string(), \"true\".to_string());\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Hello World!\");\n    }\n\n    #[test]\n    fn test_if_block_with_missing_variable() {\n        let template =\n            Template::parse(\"{{#if show_message}}Hello World!{{/if show_message}}\").unwrap();\n        let vars = Variables::<String, String>::new();\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"\");\n    }\n\n    #[test]\n    fn test_if_block_with_content_and_variables() {\n        let template = Template::parse(\n            \"{{#if notifications}}You have notifications: {{count}}{{/if notifications}}\",\n        )\n        .unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"notifications\".to_string(), \"true\".to_string());\n        vars.insert_single(\"count\".to_string(), \"5\".to_string());\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"You have notifications: 5\");\n    }\n\n    #[test]\n    fn test_foreach_block_basic() {\n        let template = Template::parse(\"{{#each items}}{{name}} {{/each items}}\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n\n        let items = vec![\n            vec![(\"name\".to_string(), \"Item1\".to_string())],\n            vec![(\"name\".to_string(), \"Item2\".to_string())],\n            vec![(\"name\".to_string(), \"Item3\".to_string())],\n        ];\n        vars.insert_block(\"items\".to_string(), items);\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Item1 Item2 Item3 \");\n    }\n\n    #[test]\n    fn test_foreach_block_multiple_variables() {\n        let template = Template::parse(\n            \"{{#each notifications}}* {{name}} at {{time}}\\n{{/each notifications}}\",\n        )\n        .unwrap();\n        let mut vars = Variables::<String, String>::new();\n\n        let notifications = vec![\n            vec![\n                (\"name\".to_string(), \"Meeting\".to_string()),\n                (\"time\".to_string(), \"10:00\".to_string()),\n            ],\n            vec![\n                (\"name\".to_string(), \"Call\".to_string()),\n                (\"time\".to_string(), \"14:30\".to_string()),\n            ],\n        ];\n        vars.insert_block(\"notifications\".to_string(), notifications);\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"* Meeting at 10:00\\n* Call at 14:30\\n\");\n    }\n\n    #[test]\n    fn test_foreach_block_empty() {\n        let template = Template::parse(\"{{#each items}}{{name}}{{/each items}}\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_block(\"items\".to_string(), Vec::<Vec<(String, String)>>::new());\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"\");\n    }\n\n    #[test]\n    fn test_foreach_block_missing_variable() {\n        let template = Template::parse(\"{{#each items}}{{name}}{{/each items}}\").unwrap();\n        let vars = Variables::<String, String>::new();\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"\");\n    }\n\n    #[test]\n    fn test_complex_template_example() {\n        let template_str = r#\"Hello {{name}},\n\n{{#if notifications}}You have the following notifications:\n{{#each notifications}}* {{name}} at {{time}}\n{{/each notifications}}{{/if notifications}}\nBest regards\"#;\n\n        let template = Template::parse(template_str).unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"name\".to_string(), \"Alice\".to_string());\n        vars.insert_single(\"notifications\".to_string(), \"true\".to_string());\n\n        let notifications = vec![\n            vec![\n                (\"name\".to_string(), \"Team Meeting\".to_string()),\n                (\"time\".to_string(), \"09:00\".to_string()),\n            ],\n            vec![\n                (\"name\".to_string(), \"Doctor Appointment\".to_string()),\n                (\"time\".to_string(), \"15:30\".to_string()),\n            ],\n        ];\n        vars.insert_block(\"notifications\".to_string(), notifications);\n\n        let result = template.eval(&vars);\n        let expected = r#\"Hello Alice,\n\nYou have the following notifications:\n* Team Meeting at 09:00\n* Doctor Appointment at 15:30\n\nBest regards\"#;\n\n        assert_eq!(result, expected);\n    }\n\n    #[test]\n    fn test_complex_template_no_notifications() {\n        let template_str = r#\"Hello {{name}},\n\n{{#if notifications}}\nYou have the following notifications:\n{{#each notifications}}\n* {{name}} at {{time}}\n{{/each notifications}}{{/if notifications}}\nBest regards\"#;\n\n        let template = Template::parse(template_str).unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"name\".to_string(), \"Bob\".to_string());\n\n        let result = template.eval(&vars);\n        let expected = r#\"Hello Bob,\n\n\nBest regards\"#;\n\n        assert_eq!(result, expected);\n    }\n\n    #[test]\n    fn test_whitespace_handling() {\n        let template = Template::parse(\"{{ name }}\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"name\".to_string(), \"Test\".to_string());\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Test\");\n    }\n\n    #[test]\n    fn test_whitespace_in_blocks() {\n        let template = Template::parse(\"{{# if condition }}Content{{/ if condition }}\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n        vars.insert_single(\"condition\".to_string(), \"true\".to_string());\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Content\");\n    }\n\n    // Error handling tests\n    #[test]\n    fn test_unmatched_opening_brace() {\n        let result = Template::<String>::parse(\"Hello {{name\");\n        assert!(result.is_err());\n        assert!(result.unwrap_err().contains(\"Unmatched {{\"));\n    }\n\n    #[test]\n    fn test_invalid_block_start() {\n        let result = Template::<String>::parse(\"{{#invalid block}}{{/invalid block}}\");\n        assert!(result.is_err());\n        assert!(result.unwrap_err().contains(\"Invalid block start\"));\n    }\n\n    #[test]\n    fn test_invalid_block_end() {\n        let result = Template::<String>::parse(\"{{#if test}}{{\\\\/invalid block}}\");\n        assert!(result.is_err());\n        assert!(result.unwrap_err().contains(\"Unmatched\"));\n    }\n\n    #[test]\n    fn test_mismatched_block_names() {\n        let result = Template::<String>::parse(\"{{#if test}}{{/if different}}\");\n        assert!(result.is_err());\n        assert!(\n            result\n                .unwrap_err()\n                .contains(\"Block end does not match start\")\n        );\n    }\n\n    #[test]\n    fn test_mismatched_block_types() {\n        let result = Template::<String>::parse(\"{{#if test}}{{/each test}}\");\n        assert!(result.is_err());\n        assert!(\n            result\n                .unwrap_err()\n                .contains(\"Block end does not match start\")\n        );\n    }\n\n    #[test]\n    fn test_consecutive_braces() {\n        let template = Template::parse(\"{{}}\").unwrap();\n        let vars = Variables::<String, String>::new();\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"\");\n    }\n\n    #[test]\n    fn test_foreach_with_missing_inner_variables() {\n        let template =\n            Template::parse(\"{{#each items}}{{name}}: {{missing}}{{/each items}}\").unwrap();\n        let mut vars = Variables::<String, String>::new();\n\n        let items = vec![\n            vec![(\"name\".to_string(), \"Item1\".to_string())],\n            vec![(\"name\".to_string(), \"Item2\".to_string())],\n        ];\n        vars.insert_block(\"items\".to_string(), items);\n\n        let result = template.eval(&vars);\n        assert_eq!(result, \"Item1: Item2: \");\n    }\n\n    /*#[test]\n    fn test_full() {\n        // Load static html in memory from resources/email-templates/calendar-alarm.html\n        let template_str = include_str!(\"../../../resources/email-templates/calendar-alarm.html\");\n        let template: Template<CalendarTemplateVariable> = Template::parse(template_str).unwrap();\n\n        let mut vars = Variables::<CalendarTemplateVariable, String>::new();\n        vars.insert_single(\n            CalendarTemplateVariable::PageTitle,\n            \"Test Event\".to_string(),\n        );\n        vars.insert_single(CalendarTemplateVariable::Header, \"Event Header\".to_string());\n        vars.insert_single(CalendarTemplateVariable::Footer, \"Event Footer\".to_string());\n        vars.insert_single(\n            CalendarTemplateVariable::EventTitle,\n            \"Meeting with Team\".to_string(),\n        );\n        vars.insert_single(\n            CalendarTemplateVariable::EventDescription,\n            \"Discuss project updates\".to_string(),\n        );\n        vars.insert_single(\n            CalendarTemplateVariable::EventDetails,\n            \"Details about the event\".to_string(),\n        );\n        vars.insert_single(\n            CalendarTemplateVariable::ActionUrl,\n            \"http://example.com/action\".to_string(),\n        );\n        vars.insert_single(\n            CalendarTemplateVariable::ActionName,\n            \"Join Meeting\".to_string(),\n        );\n        vars.insert_single(\n            CalendarTemplateVariable::AttendeesTitle,\n            \"Attendees\".to_string(),\n        );\n        vars.insert_block(\n            CalendarTemplateVariable::EventDetails,\n            vec![\n                vec![\n                    (CalendarTemplateVariable::Key, \"Location\".to_string()),\n                    (\n                        CalendarTemplateVariable::Value,\n                        \"Conference Room A\".to_string(),\n                    ),\n                ],\n                vec![\n                    (CalendarTemplateVariable::Key, \"Time\".to_string()),\n                    (\n                        CalendarTemplateVariable::Value,\n                        \"10:00 AM - 11:00 AM\".to_string(),\n                    ),\n                ],\n            ],\n        );\n        vars.insert_block(\n            CalendarTemplateVariable::Attendees,\n            vec![\n                vec![\n                    (CalendarTemplateVariable::Key, \"Alice\".to_string()),\n                    (\n                        CalendarTemplateVariable::Value,\n                        \"alice@domain.org\".to_string(),\n                    ),\n                ],\n                vec![\n                    (CalendarTemplateVariable::Key, \"Bob\".to_string()),\n                    (\n                        CalendarTemplateVariable::Value,\n                        \"bob@domain.org\".to_string(),\n                    ),\n                ],\n            ],\n        );\n        let result = template.eval(&vars);\n        // Write result to test.html\n        std::fs::write(\"test.html\", result).expect(\"Unable to write file\");\n    }*/\n}\n"
  },
  {
    "path": "crates/utils/src/topological.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashMap;\nuse std::{collections::VecDeque, hash::Hash};\n\n#[derive(Debug)]\npub struct TopologicalSort<T: Copy + Eq + Hash> {\n    edges: AHashMap<T, Vec<T>>,\n    count: AHashMap<T, usize>,\n}\n\nimpl<T: Copy + Eq + Hash + std::fmt::Debug> TopologicalSort<T> {\n    pub fn with_capacity(capacity: usize) -> Self {\n        Self {\n            edges: AHashMap::with_capacity(capacity),\n            count: AHashMap::with_capacity(capacity),\n        }\n    }\n\n    pub fn insert(&mut self, from: T, to: T) {\n        self.count.entry(from).or_insert(0);\n        self.edges.entry(from).or_default().push(to);\n        *self.count.entry(to).or_insert(0) += 1;\n    }\n\n    pub fn into_iterator(mut self) -> TopologicalSortIterator<T> {\n        let mut no_edges = VecDeque::with_capacity(self.count.len());\n        self.count.retain(|node, count| {\n            if *count == 0 {\n                no_edges.push_back(*node);\n                false\n            } else {\n                true\n            }\n        });\n\n        TopologicalSortIterator {\n            edges: self.edges,\n            count: self.count,\n            no_edges,\n        }\n    }\n}\n\n#[derive(Debug)]\npub struct TopologicalSortIterator<T: Copy + Eq + Hash> {\n    edges: AHashMap<T, Vec<T>>,\n    count: AHashMap<T, usize>,\n    no_edges: VecDeque<T>,\n}\n\nimpl<T: Copy + Eq + Hash> Iterator for TopologicalSortIterator<T> {\n    type Item = T;\n\n    fn next(&mut self) -> Option<Self::Item> {\n        let no_edge = self.no_edges.pop_back()?;\n\n        if let Some(edges) = self.edges.get(&no_edge) {\n            for neighbor in edges {\n                if let Some(count) = self.count.get_mut(neighbor) {\n                    *count -= 1;\n                    if *count == 0 {\n                        self.count.remove(neighbor);\n                        self.no_edges.push_front(*neighbor);\n                    }\n                }\n            }\n        }\n\n        Some(no_edge)\n    }\n}\n\nimpl<T: Copy + Eq + Hash> TopologicalSortIterator<T> {\n    pub fn is_valid(&self) -> bool {\n        self.count.is_empty()\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n\n    #[test]\n    fn test_topological_sort() {\n        let mut sort = TopologicalSort::with_capacity(6);\n        sort.insert(1, 2);\n        sort.insert(1, 3);\n        sort.insert(2, 4);\n        sort.insert(3, 4);\n        sort.insert(4, 5);\n        sort.insert(5, 6);\n\n        let mut iter = sort.into_iterator();\n        assert_eq!(iter.next(), Some(1));\n        assert_eq!(iter.next(), Some(2));\n        assert_eq!(iter.next(), Some(3));\n        assert_eq!(iter.next(), Some(4));\n        assert_eq!(iter.next(), Some(5));\n        assert_eq!(iter.next(), Some(6));\n        assert_eq!(iter.next(), None);\n        assert!(iter.is_valid(), \"{:?}\", iter);\n    }\n\n    #[test]\n    fn test_topological_sort_cycle() {\n        let mut sort = TopologicalSort::with_capacity(6);\n        sort.insert(1, 2);\n        sort.insert(2, 3);\n        sort.insert(3, 1);\n\n        let mut iter = sort.into_iterator();\n        assert_eq!(iter.next(), None);\n        assert!(!iter.is_valid());\n    }\n}\n"
  },
  {
    "path": "crates/utils/src/url_params.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, collections::HashMap};\n\n#[derive(Default)]\npub struct UrlParams<'x> {\n    params: HashMap<Cow<'x, str>, Cow<'x, str>>,\n}\n\nimpl<'x> UrlParams<'x> {\n    pub fn new(query: Option<&'x str>) -> Self {\n        if let Some(query) = query {\n            Self {\n                params: form_urlencoded::parse(query.as_bytes())\n                    .filter(|(_, value)| !value.is_empty())\n                    .collect(),\n            }\n        } else {\n            Self::default()\n        }\n    }\n\n    pub fn get(&self, key: &str) -> Option<&str> {\n        self.params.get(key).map(|v| v.as_ref())\n    }\n\n    pub fn has_key(&self, key: &str) -> bool {\n        self.params.contains_key(key)\n    }\n\n    pub fn parse<T>(&self, key: &str) -> Option<T>\n    where\n        T: std::str::FromStr,\n    {\n        self.get(key).and_then(|v| v.parse().ok())\n    }\n\n    pub fn into_inner(self) -> HashMap<Cow<'x, str>, Cow<'x, str>> {\n        self.params\n    }\n}\n"
  },
  {
    "path": "docker-bake.hcl",
    "content": "variable \"TARGET\" {\n  default = \"$TARGET\"\n}\nvariable \"GHCR_REPO\" {\n  default = \"$GHCR_REPO\"\n}\nvariable \"BUILD_ENV\" {\n  default = \"$BUILD_ENV\"\n}\nvariable \"SUFFIX\" {\n  default = \"$SUFFIX\"\n}\nvariable \"DOCKER_PLATFORM\" {\n  default = \"$DOCKER_PLATFORM\"\n}\ntarget \"docker-metadata-action\" {}\ntarget \"build\" {\n  secret = [\n    \"type=env,id=ACTIONS_RESULTS_URL\",\n    \"type=env,id=ACTIONS_RUNTIME_TOKEN\"\n  ]\n  args = {\n    TARGET = \"${TARGET}\"\n    BUILD_ENV = equal(\"\", \"${BUILD_ENV}\") ? null : \"${BUILD_ENV}\"\n  }\n  target = \"binaries\"\n  cache-from = [\n    \"type=registry,ref=${GHCR_REPO}-buildcache:${TARGET}\"\n  ]\n  cache-to = [\n    \"type=registry,ref=${GHCR_REPO}-buildcache:${TARGET},mode=max,compression=zstd,compression-level=9,force-compression=true,oci-mediatypes=true,image-manifest=false\"\n  ]\n  context = \"./\"\n  dockerfile = \"Dockerfile.build\"\n  output = [\"./artifact\"]\n}\ntarget \"image\" {\n  inherits = [\"build\",\"docker-metadata-action\"]\n  cache-to = [\"\"]\n  cache-from = [\n    \"type=registry,ref=${GHCR_REPO}-buildcache:${TARGET}\"\n  ]\n  target = equal(\"\", \"${SUFFIX}\") ? \"gnu\" : \"musl\"\n  platforms = [\n    \"${DOCKER_PLATFORM}\"\n  ]\n  output = [\n    \"\"\n  ]\n}\n"
  },
  {
    "path": "install.sh",
    "content": "#!/usr/bin/env sh\n# shellcheck shell=dash\n\n#\n# SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n#\n# SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n#\n\n# Stalwart install script -- based on the rustup installation script.\n\nset -e\nset -u\n\nreadonly BASE_URL=\"https://github.com/stalwartlabs/stalwart/releases/latest/download\"\n\nmain() {\n    downloader --check\n    need_cmd uname\n    need_cmd mktemp\n    need_cmd chmod\n    need_cmd mkdir\n    need_cmd rm\n    need_cmd rmdir\n    need_cmd tar\n\n    # Make sure we are running as root\n    if [ \"$(id -u)\" -ne 0 ] ; then\n        err \"❌ Install failed: This program needs to run as root.\"\n    fi\n\n    # Detect OS\n    local _os=\"unknown\"\n    local _uname=\"$(uname)\"\n    _account=\"stalwart\"\n    if [ \"${_uname}\" = \"Linux\" ]; then\n        _os=\"linux\"\n    elif [ \"${_uname}\" = \"Darwin\" ]; then\n        _os=\"macos\"\n        _account=\"_stalwart\"\n    fi\n\n    # Read arguments\n    local _dir=\"/opt/stalwart\"\n\n    # Default component setting\n    local _component=\"stalwart\"\n\n    # Loop through the arguments\n    for arg in \"$@\"; do\n        case \"$arg\" in\n            --fdb)\n                _component=\"stalwart-foundationdb\"\n                ;;\n            *)\n                if [ -n \"$arg\" ]; then\n                    _dir=$arg\n                fi\n                ;;\n        esac\n    done\n\n    # Detect platform architecture\n    get_architecture || return 1\n    local _arch=\"$RETVAL\"\n    assert_nz \"$_arch\" \"arch\"\n\n    # Create directories\n    ensure mkdir -p \"$_dir\" \"$_dir/bin\" \"$_dir/etc\" \"$_dir/logs\"\n\n    # Download latest binary\n    say \"⏳ Downloading ${_component} for ${_arch}...\"\n    local _file=\"${_dir}/bin/stalwart.tar.gz\"\n    local _url=\"${BASE_URL}/${_component}-${_arch}.tar.gz\"\n    ensure mkdir -p \"$_dir\"\n    ensure downloader \"$_url\" \"$_file\" \"$_arch\"\n    ensure tar zxvf \"$_file\" -C \"$_dir/bin\"\n    if [ \"$_component\" = \"stalwart-foundationdb\" ]; then\n        ignore mv \"$_dir/bin/stalwart-foundationdb\" \"$_dir/bin/stalwart\"\n    fi\n    ignore chmod +x \"$_dir/bin/stalwart\"\n    ignore rm \"$_file\"\n\n    # Create system account\n    if ! id -u ${_account} > /dev/null 2>&1; then\n        say \"🖥️  Creating '${_account}' account...\"\n        if [ \"${_os}\" = \"macos\" ]; then\n            local _last_uid=\"$(dscacheutil -q user | grep uid | awk '{print $2}' | sort -n | tail -n 1)\"\n            local _last_gid=\"$(dscacheutil -q group | grep gid | awk '{print $2}' | sort -n | tail -n 1)\"\n            local _uid=\"$((_last_uid+1))\"\n            local _gid=\"$((_last_gid+1))\"\n\n            ensure dscl /Local/Default -create Groups/_stalwart\n            ensure dscl /Local/Default -create Groups/_stalwart Password \\*\n            ensure dscl /Local/Default -create Groups/_stalwart PrimaryGroupID $_gid\n            ensure dscl /Local/Default -create Groups/_stalwart RealName \"Stalwart service\"\n            ensure dscl /Local/Default -create Groups/_stalwart RecordName _stalwart stalwart\n\n            ensure dscl /Local/Default -create Users/_stalwart\n            ensure dscl /Local/Default -create Users/_stalwart NFSHomeDirectory /Users/_stalwart\n            ensure dscl /Local/Default -create Users/_stalwart Password \\*\n            ensure dscl /Local/Default -create Users/_stalwart PrimaryGroupID $_gid\n            ensure dscl /Local/Default -create Users/_stalwart RealName \"Stalwart service\"\n            ensure dscl /Local/Default -create Users/_stalwart RecordName _stalwart stalwart\n            ensure dscl /Local/Default -create Users/_stalwart UniqueID $_uid\n            ensure dscl /Local/Default -create Users/_stalwart UserShell /bin/bash\n\n            ensure dscl /Local/Default -delete /Users/_stalwart AuthenticationAuthority\n            ensure dscl /Local/Default -delete /Users/_stalwart PasswordPolicyOptions\n        else\n            ensure useradd ${_account} -s /usr/sbin/nologin -M -r -U\n        fi\n    fi\n\n    # Run init\n    ignore $_dir/bin/stalwart --init \"$_dir\"\n\n    # Set permissions\n    say \"🔐 Setting permissions...\"\n    ensure chown -R ${_account}:${_account} \"$_dir\"\n    ensure chmod -R 755 \"$_dir\"\n    ensure chmod 700 \"$_dir/etc/config.toml\"\n\n    # Create service file\n    say \"🚀 Starting service...\"\n    if [ \"${_os}\" = \"linux\" ]; then\n        local _issystemdlinux=$(command -v systemctl)\n        if [ -n \"$_issystemdlinux\" ]; then\n            create_service_linux_systemd \"$_dir\"\n        else\n            create_service_linux_initd \"$_dir\"\n        fi\n    elif [ \"${_os}\" = \"macos\" ]; then\n        create_service_macos \"$_dir\"\n    fi\n\n    # Installation complete\n    local _host=$(hostname -f)\n    say \"🎉 Installation complete! Continue the setup at http://$_host:8080/login\"\n\n    return 0\n}\n\n# Functions to create service files\ncreate_service_linux_systemd() {\n    local _dir=\"$1\"\n    cat <<EOF | sed \"s|__PATH__|$_dir|g\" > /etc/systemd/system/stalwart.service\n[Unit]\nDescription=Stalwart\nConflicts=postfix.service sendmail.service exim4.service\nConditionPathExists=__PATH__/etc/config.toml\nAfter=network-online.target\n\n[Service]\nType=simple\nLimitNOFILE=65536\nKillMode=process\nKillSignal=SIGINT\nRestart=on-failure\nRestartSec=5\nExecStart=__PATH__/bin/stalwart --config=__PATH__/etc/config.toml\nSyslogIdentifier=stalwart\nUser=stalwart\nGroup=stalwart\nAmbientCapabilities=CAP_NET_BIND_SERVICE\n\n[Install]\nWantedBy=multi-user.target\nEOF\n    systemctl daemon-reload\n    systemctl enable stalwart.service\n    systemctl restart stalwart.service\n}\n\ncreate_service_linux_initd() {\n    local _dir=\"$1\"\n    cat <<\"EOF\" | sed \"s|__PATH__|$_dir|g\" > /etc/init.d/stalwart\n#!/bin/sh\n### BEGIN INIT INFO\n# Provides:          stalwart\n# Required-Start:    $network\n# Required-Stop:     $network\n# Default-Start:     2 3 4 5\n# Default-Stop:      0 1 6\n# Short-Description: Stalwart Server\n# Description:       Starts and stops the Stalwart Server\n# Conflicts:         postfix sendmail\n### END INIT INFO\n\nPATH=/sbin:/usr/sbin:/bin:/usr/bin\n\n. /lib/init/vars.sh\n. /lib/lsb/init-functions\n\n# Service Config\nDAEMON=__PATH__/bin/stalwart\nDAEMON_ARGS=\"--config=__PATH__/etc/config.toml\"\nPIDFILE=/var/run/stalwart.pid\nULIMIT_NOFILE=65536\n\n# Exit if the package is not installed\n[ -x \"$DAEMON\" ] || exit 0\n\n# Exit if config file doesn't exist\n[ -f \"__PATH__/etc/config.toml\" ] || exit 0\n\n# Read configuration variable file if it is present\n[ -r /etc/default/stalwart ] && . /etc/default/stalwart\n\n# Increase file descriptor limit\nulimit -n $ULIMIT_NOFILE\n\ndo_start()\n{\n    # Return\n    #   0 if daemon has been started\n    #   1 if daemon was already running\n    #   2 if daemon could not be started\n    start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \\\n        || return 1\n    start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON \\\n        --background --make-pidfile --chuid stalwart:stalwart \\\n        -- $DAEMON_ARGS \\\n        || return 2\n}\n\ndo_stop()\n{\n    # Return\n    #   0 if daemon has been stopped\n    #   1 if daemon was already stopped\n    #   2 if daemon could not be stopped\n    #   other if a failure occurred\n    start-stop-daemon --stop --quiet --retry=INT/30/KILL/5 --pidfile $PIDFILE --name stalwart\n    RETVAL=\"$?\"\n    [ \"$RETVAL\" = 2 ] && return 2\n    # Wait for children to finish too if this is a daemon that forks\n    # and if the daemon is only ever run from this initscript.\n    start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON\n    [ \"$?\" = 2 ] && return 2\n    # Many daemons don't delete their pidfiles when they exit.\n    rm -f $PIDFILE\n    return \"$RETVAL\"\n}\n\ncase \"$1\" in\n  start)\n    [ \"$VERBOSE\" != no ] && log_daemon_msg \"Starting Stalwart Server\" \"stalwart\"\n    do_start\n    case \"$?\" in\n        0|1) [ \"$VERBOSE\" != no ] && log_end_msg 0 ;;\n        2) [ \"$VERBOSE\" != no ] && log_end_msg 1 ;;\n    esac\n    ;;\n  stop)\n    [ \"$VERBOSE\" != no ] && log_daemon_msg \"Stopping Stalwart Server\" \"stalwart\"\n    do_stop\n    case \"$?\" in\n        0|1) [ \"$VERBOSE\" != no ] && log_end_msg 0 ;;\n        2) [ \"$VERBOSE\" != no ] && log_end_msg 1 ;;\n    esac\n    ;;\n  status)\n    status_of_proc \"$DAEMON\" \"stalwart\" && exit 0 || exit $?\n    ;;\n  restart)\n    log_daemon_msg \"Restarting Stalwart Server\" \"stalwart\"\n    do_stop\n    case \"$?\" in\n      0|1)\n        do_start\n        case \"$?\" in\n            0) log_end_msg 0 ;;\n            1) log_end_msg 1 ;; # Old process is still running\n            *) log_end_msg 1 ;; # Failed to start\n        esac\n        ;;\n      *)\n        # Failed to stop\n        log_end_msg 1\n        ;;\n    esac\n    ;;\n  *)\n    echo \"Usage: /etc/init.d/stalwart {start|stop|status|restart}\" >&2\n    exit 3\n    ;;\nesac\n\nexit 0\nEOF\n    chmod +x /etc/init.d/stalwart\n\n    cat <<EOF > /etc/default/stalwart\n# Configuration for Stalwart init script being run during\n# the boot sequence\n\n# Set to 'yes' to enable additional verbosity\n#VERBOSE=no\nEOF\n    update-rc.d stalwart defaults\n    service stalwart start\n}\n\ncreate_service_macos() {\n    local _dir=\"$1\"\n    cat <<EOF | sed \"s|__PATH__|$_dir|g\" > /Library/LaunchAgents/stalwart.mail.plist\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE plist PUBLIC \"-//Apple Computer//DTD PLIST 1.0//EN\"\n    \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n<plist version=\"1.0\">\n    <dict>\n        <key>Label</key>\n        <string>stalwart.mail</string>\n        <key>ServiceDescription</key>\n        <string>Stalwart</string>\n        <key>ProgramArguments</key>\n        <array>\n            <string>__PATH__/bin/stalwart</string>\n            <string>--config=__PATH__/etc/config.toml</string>\n        </array>\n        <key>RunAtLoad</key>\n        <true/>\n        <key>KeepAlive</key>\n        <true/>\n    </dict>\n</plist>\nEOF\n    launchctl load /Library/LaunchAgents/stalwart.mail.plist\n    launchctl enable system/stalwart.mail\n    launchctl start system/stalwart.mail\n}\n\n\nget_architecture() {\n    local _ostype _cputype _bitness _arch _clibtype\n    _ostype=\"$(uname -s)\"\n    _cputype=\"$(uname -m)\"\n    _clibtype=\"gnu\"\n\n    if [ \"$_ostype\" = Linux ]; then\n        if [ \"$(uname -o)\" = Android ]; then\n            _ostype=Android\n        fi\n        if ldd --version 2>&1 | grep -q 'musl'; then\n            _clibtype=\"musl\"\n        fi\n    fi\n\n    if [ \"$_ostype\" = Darwin ] && [ \"$_cputype\" = i386 ]; then\n        # Darwin `uname -m` lies\n        if sysctl hw.optional.x86_64 | grep -q ': 1'; then\n            _cputype=x86_64\n        fi\n    fi\n\n    if [ \"$_ostype\" = SunOS ]; then\n        # Both Solaris and illumos presently announce as \"SunOS\" in \"uname -s\"\n        # so use \"uname -o\" to disambiguate.  We use the full path to the\n        # system uname in case the user has coreutils uname first in PATH,\n        # which has historically sometimes printed the wrong value here.\n        if [ \"$(/usr/bin/uname -o)\" = illumos ]; then\n            _ostype=illumos\n        fi\n\n        # illumos systems have multi-arch userlands, and \"uname -m\" reports the\n        # machine hardware name; e.g., \"i86pc\" on both 32- and 64-bit x86\n        # systems.  Check for the native (widest) instruction set on the\n        # running kernel:\n        if [ \"$_cputype\" = i86pc ]; then\n            _cputype=\"$(isainfo -n)\"\n        fi\n    fi\n\n    case \"$_ostype\" in\n\n        Android)\n            _ostype=linux-android\n            ;;\n\n        Linux)\n            check_proc\n            _ostype=unknown-linux-$_clibtype\n            _bitness=$(get_bitness)\n            ;;\n\n        FreeBSD)\n            _ostype=unknown-freebsd\n            ;;\n\n        NetBSD)\n            _ostype=unknown-netbsd\n            ;;\n\n        DragonFly)\n            _ostype=unknown-dragonfly\n            ;;\n\n        Darwin)\n            _ostype=apple-darwin\n            ;;\n\n        illumos)\n            _ostype=unknown-illumos\n            ;;\n\n        MINGW* | MSYS* | CYGWIN* | Windows_NT)\n            _ostype=pc-windows-gnu\n            ;;\n\n        *)\n            err \"unrecognized OS type: $_ostype\"\n            ;;\n\n    esac\n\n    case \"$_cputype\" in\n\n        i386 | i486 | i686 | i786 | x86)\n            _cputype=i686\n            ;;\n\n        xscale | arm)\n            _cputype=arm\n            if [ \"$_ostype\" = \"linux-android\" ]; then\n                _ostype=linux-androideabi\n            fi\n            ;;\n\n        armv6l)\n            _cputype=arm\n            if [ \"$_ostype\" = \"linux-android\" ]; then\n                _ostype=linux-androideabi\n            else\n                _ostype=\"${_ostype}eabihf\"\n            fi\n            ;;\n\n        armv7l | armv8l)\n            _cputype=armv7\n            if [ \"$_ostype\" = \"linux-android\" ]; then\n                _ostype=linux-androideabi\n            else\n                _ostype=\"${_ostype}eabihf\"\n            fi\n            ;;\n\n        aarch64 | arm64)\n            _cputype=aarch64\n            ;;\n\n        x86_64 | x86-64 | x64 | amd64)\n            _cputype=x86_64\n            ;;\n\n        mips)\n            _cputype=$(get_endianness mips '' el)\n            ;;\n\n        mips64)\n            if [ \"$_bitness\" -eq 64 ]; then\n                # only n64 ABI is supported for now\n                _ostype=\"${_ostype}abi64\"\n                _cputype=$(get_endianness mips64 '' el)\n            fi\n            ;;\n\n        ppc)\n            _cputype=powerpc\n            ;;\n\n        ppc64)\n            _cputype=powerpc64\n            ;;\n\n        ppc64le)\n            _cputype=powerpc64le\n            ;;\n\n        s390x)\n            _cputype=s390x\n            ;;\n        riscv64)\n            _cputype=riscv64gc\n            ;;\n        *)\n            err \"unknown CPU type: $_cputype\"\n\n    esac\n\n    # Detect 64-bit linux with 32-bit userland\n    if [ \"${_ostype}\" = unknown-linux-gnu ] && [ \"${_bitness}\" -eq 32 ]; then\n        case $_cputype in\n            x86_64)\n                if [ -n \"${RUSTUP_CPUTYPE:-}\" ]; then\n                    _cputype=\"$RUSTUP_CPUTYPE\"\n                else {\n                    # 32-bit executable for amd64 = x32\n                    if is_host_amd64_elf; then {\n                         echo \"This host is running an x32 userland; as it stands, x32 support is poor,\" 1>&2\n                         echo \"and there isn't a native toolchain -- you will have to install\" 1>&2\n                         echo \"multiarch compatibility with i686 and/or amd64, then select one\" 1>&2\n                         echo \"by re-running this script with the RUSTUP_CPUTYPE environment variable\" 1>&2\n                         echo \"set to i686 or x86_64, respectively.\" 1>&2\n                         echo 1>&2\n                         echo \"You will be able to add an x32 target after installation by running\" 1>&2\n                         echo \"  rustup target add x86_64-unknown-linux-gnux32\" 1>&2\n                         exit 1\n                    }; else\n                        _cputype=i686\n                    fi\n                }; fi\n                ;;\n            mips64)\n                _cputype=$(get_endianness mips '' el)\n                ;;\n            powerpc64)\n                _cputype=powerpc\n                ;;\n            aarch64)\n                _cputype=armv7\n                if [ \"$_ostype\" = \"linux-android\" ]; then\n                    _ostype=linux-androideabi\n                else\n                    _ostype=\"${_ostype}eabihf\"\n                fi\n                ;;\n            riscv64gc)\n                err \"riscv64 with 32-bit userland unsupported\"\n                ;;\n        esac\n    fi\n\n    # Detect armv7 but without the CPU features Rust needs in that build,\n    # and fall back to arm.\n    # See https://github.com/rust-lang/rustup.rs/issues/587.\n    if [ \"$_ostype\" = \"unknown-linux-gnueabihf\" ] && [ \"$_cputype\" = armv7 ]; then\n        if ensure grep '^Features' /proc/cpuinfo | grep -q -v neon; then\n            # At least one processor does not have NEON.\n            _cputype=arm\n        fi\n    fi\n\n    _arch=\"${_cputype}-${_ostype}\"\n\n    RETVAL=\"$_arch\"\n}\n\ncheck_proc() {\n    # Check for /proc by looking for the /proc/self/exe link\n    # This is only run on Linux\n    if ! test -L /proc/self/exe ; then\n        err \"fatal: Unable to find /proc/self/exe.  Is /proc mounted?  Installation cannot proceed without /proc.\"\n    fi\n}\n\nget_bitness() {\n    need_cmd head\n    # Architecture detection without dependencies beyond coreutils.\n    # ELF files start out \"\\x7fELF\", and the following byte is\n    #   0x01 for 32-bit and\n    #   0x02 for 64-bit.\n    # The printf builtin on some shells like dash only supports octal\n    # escape sequences, so we use those.\n    local _current_exe_head\n    _current_exe_head=$(head -c 5 /proc/self/exe )\n    if [ \"$_current_exe_head\" = \"$(printf '\\177ELF\\001')\" ]; then\n        echo 32\n    elif [ \"$_current_exe_head\" = \"$(printf '\\177ELF\\002')\" ]; then\n        echo 64\n    else\n        err \"unknown platform bitness\"\n    fi\n}\n\nis_host_amd64_elf() {\n    need_cmd head\n    need_cmd tail\n    # ELF e_machine detection without dependencies beyond coreutils.\n    # Two-byte field at offset 0x12 indicates the CPU,\n    # but we're interested in it being 0x3E to indicate amd64, or not that.\n    local _current_exe_machine\n    _current_exe_machine=$(head -c 19 /proc/self/exe | tail -c 1)\n    [ \"$_current_exe_machine\" = \"$(printf '\\076')\" ]\n}\n\nget_endianness() {\n    local cputype=$1\n    local suffix_eb=$2\n    local suffix_el=$3\n\n    # detect endianness without od/hexdump, like get_bitness() does.\n    need_cmd head\n    need_cmd tail\n\n    local _current_exe_endianness\n    _current_exe_endianness=\"$(head -c 6 /proc/self/exe | tail -c 1)\"\n    if [ \"$_current_exe_endianness\" = \"$(printf '\\001')\" ]; then\n        echo \"${cputype}${suffix_el}\"\n    elif [ \"$_current_exe_endianness\" = \"$(printf '\\002')\" ]; then\n        echo \"${cputype}${suffix_eb}\"\n    else\n        err \"unknown platform endianness\"\n    fi\n}\n\nsay() {\n    printf '%s\\n' \"$1\"\n}\n\nerr() {\n    say \"$1\" >&2\n    exit 1\n}\n\nneed_cmd() {\n    if ! check_cmd \"$1\"; then\n        err \"need '$1' (command not found)\"\n    fi\n}\n\ncheck_cmd() {\n    command -v \"$1\" > /dev/null 2>&1\n}\n\nassert_nz() {\n    if [ -z \"$1\" ]; then err \"assert_nz $2\"; fi\n}\n\n# Run a command that should never fail. If the command fails execution\n# will immediately terminate with an error showing the failing\n# command.\nensure() {\n    if ! \"$@\"; then err \"command failed: $*\"; fi\n}\n\n# This wraps curl or wget. Try curl first, if not installed,\n# use wget instead.\ndownloader() {\n    local _dld\n    local _ciphersuites\n    local _err\n    local _status\n    local _retry\n    if check_cmd curl; then\n        _dld=curl\n    elif check_cmd wget; then\n        _dld=wget\n    else\n        _dld='curl or wget' # to be used in error message of need_cmd\n    fi\n\n    if [ \"$1\" = --check ]; then\n        need_cmd \"$_dld\"\n    elif [ \"$_dld\" = curl ]; then\n        check_curl_for_retry_support\n        _retry=\"$RETVAL\"\n        get_ciphersuites_for_curl\n        _ciphersuites=\"$RETVAL\"\n        if [ -n \"$_ciphersuites\" ]; then\n            _err=$(curl $_retry --proto '=https' --tlsv1.2 --ciphers \"$_ciphersuites\" --silent --show-error --fail --location \"$1\" --output \"$2\" 2>&1)\n            _status=$?\n        else\n            echo \"Warning: Not enforcing strong cipher suites for TLS, this is potentially less secure\"\n            if ! check_help_for \"$3\" curl --proto --tlsv1.2; then\n                echo \"Warning: Not enforcing TLS v1.2, this is potentially less secure\"\n                _err=$(curl $_retry --silent --show-error --fail --location \"$1\" --output \"$2\" 2>&1)\n                _status=$?\n            else\n                _err=$(curl $_retry --proto '=https' --tlsv1.2 --silent --show-error --fail --location \"$1\" --output \"$2\" 2>&1)\n                _status=$?\n            fi\n        fi\n        if [ -n \"$_err\" ]; then\n            if echo \"$_err\" | grep -q 404; then\n                err \"❌  Binary for platform '$3' not found, this platform may be unsupported.\"\n            else\n                echo \"$_err\" >&2\n            fi\n        fi\n        return $_status\n    elif [ \"$_dld\" = wget ]; then\n        if [ \"$(wget -V 2>&1|head -2|tail -1|cut -f1 -d\" \")\" = \"BusyBox\" ]; then\n            echo \"Warning: using the BusyBox version of wget.  Not enforcing strong cipher suites for TLS or TLS v1.2, this is potentially less secure\"\n            _err=$(wget \"$1\" -O \"$2\" 2>&1)\n            _status=$?\n        else\n            get_ciphersuites_for_wget\n            _ciphersuites=\"$RETVAL\"\n            if [ -n \"$_ciphersuites\" ]; then\n                _err=$(wget --https-only --secure-protocol=TLSv1_2 --ciphers \"$_ciphersuites\" \"$1\" -O \"$2\" 2>&1)\n                _status=$?\n            else\n                echo \"Warning: Not enforcing strong cipher suites for TLS, this is potentially less secure\"\n                if ! check_help_for \"$3\" wget --https-only --secure-protocol; then\n                    echo \"Warning: Not enforcing TLS v1.2, this is potentially less secure\"\n                    _err=$(wget \"$1\" -O \"$2\" 2>&1)\n                    _status=$?\n                else\n                    _err=$(wget --https-only --secure-protocol=TLSv1_2 \"$1\" -O \"$2\" 2>&1)\n                    _status=$?\n                fi\n            fi\n        fi\n        if [ -n \"$_err\" ]; then\n            if echo \"$_err\" | grep -q ' 404 Not Found'; then\n                err \"❌  Binary for platform '$3' not found, this platform may be unsupported.\"\n            else\n                echo \"$_err\" >&2\n            fi\n        fi\n        return $_status\n    else\n        err \"Unknown downloader\"   # should not reach here\n    fi\n}\n\n# Check if curl supports the --retry flag, then pass it to the curl invocation.\ncheck_curl_for_retry_support() {\n  local _retry_supported=\"\"\n  # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n  if check_help_for \"notspecified\" \"curl\" \"--retry\"; then\n    _retry_supported=\"--retry 3\"\n  fi\n\n  RETVAL=\"$_retry_supported\"\n\n}\n\ncheck_help_for() {\n    local _arch\n    local _cmd\n    local _arg\n    _arch=\"$1\"\n    shift\n    _cmd=\"$1\"\n    shift\n\n    local _category\n    if \"$_cmd\" --help | grep -q 'For all options use the manual or \"--help all\".'; then\n      _category=\"all\"\n    else\n      _category=\"\"\n    fi\n\n    case \"$_arch\" in\n\n        *darwin*)\n        if check_cmd sw_vers; then\n            case $(sw_vers -productVersion) in\n                10.*)\n                    # If we're running on macOS, older than 10.13, then we always\n                    # fail to find these options to force fallback\n                    if [ \"$(sw_vers -productVersion | cut -d. -f2)\" -lt 13 ]; then\n                        # Older than 10.13\n                        echo \"Warning: Detected macOS platform older than 10.13\"\n                        return 1\n                    fi\n                    ;;\n                11.*)\n                    # We assume Big Sur will be OK for now\n                    ;;\n                *)\n                    # Unknown product version, warn and continue\n                    echo \"Warning: Detected unknown macOS major version: $(sw_vers -productVersion)\"\n                    echo \"Warning TLS capabilities detection may fail\"\n                    ;;\n            esac\n        fi\n        ;;\n\n    esac\n\n    for _arg in \"$@\"; do\n        if ! \"$_cmd\" --help $_category | grep -q -- \"$_arg\"; then\n            return 1\n        fi\n    done\n\n    true # not strictly needed\n}\n\n# Return cipher suite string specified by user, otherwise return strong TLS 1.2-1.3 cipher suites\n# if support by local tools is detected. Detection currently supports these curl backends:\n# GnuTLS and OpenSSL (possibly also LibreSSL and BoringSSL). Return value can be empty.\nget_ciphersuites_for_curl() {\n    if [ -n \"${RUSTUP_TLS_CIPHERSUITES-}\" ]; then\n        # user specified custom cipher suites, assume they know what they're doing\n        RETVAL=\"$RUSTUP_TLS_CIPHERSUITES\"\n        return\n    fi\n\n    local _openssl_syntax=\"no\"\n    local _gnutls_syntax=\"no\"\n    local _backend_supported=\"yes\"\n    if curl -V | grep -q ' OpenSSL/'; then\n        _openssl_syntax=\"yes\"\n    elif curl -V | grep -iq ' LibreSSL/'; then\n        _openssl_syntax=\"yes\"\n    elif curl -V | grep -iq ' BoringSSL/'; then\n        _openssl_syntax=\"yes\"\n    elif curl -V | grep -iq ' GnuTLS/'; then\n        _gnutls_syntax=\"yes\"\n    else\n        _backend_supported=\"no\"\n    fi\n\n    local _args_supported=\"no\"\n    if [ \"$_backend_supported\" = \"yes\" ]; then\n        # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n        if check_help_for \"notspecified\" \"curl\" \"--tlsv1.2\" \"--ciphers\" \"--proto\"; then\n            _args_supported=\"yes\"\n        fi\n    fi\n\n    local _cs=\"\"\n    if [ \"$_args_supported\" = \"yes\" ]; then\n        if [ \"$_openssl_syntax\" = \"yes\" ]; then\n            _cs=$(get_strong_ciphersuites_for \"openssl\")\n        elif [ \"$_gnutls_syntax\" = \"yes\" ]; then\n            _cs=$(get_strong_ciphersuites_for \"gnutls\")\n        fi\n    fi\n\n    RETVAL=\"$_cs\"\n}\n\n# Return cipher suite string specified by user, otherwise return strong TLS 1.2-1.3 cipher suites\n# if support by local tools is detected. Detection currently supports these wget backends:\n# GnuTLS and OpenSSL (possibly also LibreSSL and BoringSSL). Return value can be empty.\nget_ciphersuites_for_wget() {\n    if [ -n \"${RUSTUP_TLS_CIPHERSUITES-}\" ]; then\n        # user specified custom cipher suites, assume they know what they're doing\n        RETVAL=\"$RUSTUP_TLS_CIPHERSUITES\"\n        return\n    fi\n\n    local _cs=\"\"\n    if wget -V | grep -q '\\-DHAVE_LIBSSL'; then\n        # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n        if check_help_for \"notspecified\" \"wget\" \"TLSv1_2\" \"--ciphers\" \"--https-only\" \"--secure-protocol\"; then\n            _cs=$(get_strong_ciphersuites_for \"openssl\")\n        fi\n    elif wget -V | grep -q '\\-DHAVE_LIBGNUTLS'; then\n        # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n        if check_help_for \"notspecified\" \"wget\" \"TLSv1_2\" \"--ciphers\" \"--https-only\" \"--secure-protocol\"; then\n            _cs=$(get_strong_ciphersuites_for \"gnutls\")\n        fi\n    fi\n\n    RETVAL=\"$_cs\"\n}\n\n# Return strong TLS 1.2-1.3 cipher suites in OpenSSL or GnuTLS syntax. TLS 1.2\n# excludes non-ECDHE and non-AEAD cipher suites. DHE is excluded due to bad\n# DH params often found on servers (see RFC 7919). Sequence matches or is\n# similar to Firefox 68 ESR with weak cipher suites disabled via about:config.\n# $1 must be openssl or gnutls.\nget_strong_ciphersuites_for() {\n    if [ \"$1\" = \"openssl\" ]; then\n        # OpenSSL is forgiving of unknown values, no problems with TLS 1.3 values on versions that don't support it yet.\n        echo \"TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384\"\n    elif [ \"$1\" = \"gnutls\" ]; then\n        # GnuTLS isn't forgiving of unknown values, so this may require a GnuTLS version that supports TLS 1.3 even if wget doesn't.\n        # Begin with SECURE128 (and higher) then remove/add to build cipher suites. Produces same 9 cipher suites as OpenSSL but in slightly different order.\n        echo \"SECURE128:-VERS-SSL3.0:-VERS-TLS1.0:-VERS-TLS1.1:-VERS-DTLS-ALL:-CIPHER-ALL:-MAC-ALL:-KX-ALL:+AEAD:+ECDHE-ECDSA:+ECDHE-RSA:+AES-128-GCM:+CHACHA20-POLY1305:+AES-256-GCM\"\n    fi\n}\n\n# This is just for indicating that commands' results are being\n# intentionally ignored. Usually, because it's being executed\n# as part of error handling.\nignore() {\n    \"$@\"\n}\n\nmain \"$@\" || exit 1\n"
  },
  {
    "path": "resources/apparmor.d/stalwart-mail",
    "content": "#include <tunables/global>\n\nprofile stalwart flags=(attach_disconnected) {\n  #include <abstractions/base>\n  #include <abstractions/nameservice>\n  #include <abstractions/openssl>\n\n  # Allow network access\n  network inet stream,\n  network inet6 stream,\n  network inet dgram,\n  network inet6 dgram,\n\n  # Outgoing access to port 25 and 443\n  network tcp,\n  network udp,\n  owner /proc/*/net/if_inet6 r,\n  owner /proc/*/net/ipv6_route r,\n\n  # Full write access to /opt/stalwart\n  /opt/stalwart/** rwk,\n\n  # Allow creating directories under /tmp\n  /tmp/ r,\n  /tmp/** rwk,\n\n  # Allow binding to specific ports\n  network inet stream bind port 25,\n  network inet stream bind port 587,\n  network inet stream bind port 465,\n  network inet stream bind port 143,\n  network inet stream bind port 993,\n  network inet stream bind port 110,\n  network inet stream bind port 995,\n  network inet stream bind port 4190,\n  network inet stream bind port 443,\n  network inet stream bind port 8080,\n  network inet6 stream bind port 25,\n  network inet6 stream bind port 587,\n  network inet6 stream bind port 465,\n  network inet6 stream bind port 143,\n  network inet6 stream bind port 993,\n  network inet6 stream bind port 110,\n  network inet6 stream bind port 995,\n  network inet6 stream bind port 4190,\n  network inet6 stream bind port 443,\n  network inet6 stream bind port 8080,\n\n  # Allow UDP port 7911\n  network inet dgram bind port 7911,\n  network inet6 dgram bind port 7911,\n\n  # Basic system access\n  /usr/bin/stalwart rix,\n  /etc/stalwart/** r,\n  /var/log/stalwart/** w,\n\n  # Additional permissions might be needed depending on specific requirements\n}\n"
  },
  {
    "path": "resources/config/config.toml",
    "content": "#############################################\n# Stalwart Configuration File   \n#############################################\n\n[server.listener.\"smtp\"]\nbind = [\"[::]:25\"]\nprotocol = \"smtp\"\n\n[server.listener.\"submission\"]\nbind = [\"[::]:587\"]\nprotocol = \"smtp\"\n\n[server.listener.\"submissions\"]\nbind = [\"[::]:465\"]\nprotocol = \"smtp\"\ntls.implicit = true\n\n[server.listener.\"imap\"]\nbind = [\"[::]:143\"]\nprotocol = \"imap\"\n\n[server.listener.\"imaptls\"]\nbind = [\"[::]:993\"]\nprotocol = \"imap\"\ntls.implicit = true\n\n[server.listener.pop3]\nbind = \"[::]:110\"\nprotocol = \"pop3\"\n\n[server.listener.pop3s]\nbind = \"[::]:995\"\nprotocol = \"pop3\"\ntls.implicit = true\n\n[server.listener.\"sieve\"]\nbind = [\"[::]:4190\"]\nprotocol = \"managesieve\"\n\n[server.listener.\"https\"]\nprotocol = \"http\"\nbind = [\"[::]:443\"]\ntls.implicit = true\n\n[storage]\ndata = \"rocksdb\"\nfts = \"rocksdb\"\nblob = \"rocksdb\"\nlookup = \"rocksdb\"\ndirectory = \"internal\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"%{env:STALWART_PATH}%/data\"\ncompression = \"lz4\"\n\n[directory.\"internal\"]\ntype = \"internal\"\nstore = \"rocksdb\"\n\n[tracer.\"stdout\"]\ntype = \"stdout\"\nlevel = \"info\"\nansi = false\nenable = true\n\n#[server.run-as]\n#user = \"stalwart\"\n#group = \"stalwart\"\n\n[authentication.fallback-admin]\nuser = \"admin\"\nsecret = \"%{env:ADMIN_SECRET}%\"\n"
  },
  {
    "path": "resources/docker/Dockerfile.fdb",
    "content": "FROM debian:trixie-slim AS chef\nRUN apt-get update && \\\n    export DEBIAN_FRONTEND=noninteractive && \\\n    apt-get install -yq \\\n    build-essential \\\n    cmake \\\n    clang \\\n    curl \\\n    protobuf-compiler \\\n    adduser\nENV RUSTUP_HOME=/opt/rust/rustup \\\n    PATH=/home/root/.cargo/bin:/opt/rust/cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\nRUN curl https://sh.rustup.rs -sSf | \\\n    env CARGO_HOME=/opt/rust/cargo \\\n    sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path && \\\n    env CARGO_HOME=/opt/rust/cargo \\\n    rustup component add rustfmt\nRUN curl -LO https://github.com/apple/foundationdb/releases/download/7.3.69/foundationdb-clients_7.3.69-1_amd64.deb && \\\n    dpkg -i foundationdb-clients_7.3.69-1_amd64.deb\nRUN env CARGO_HOME=/opt/rust/cargo cargo install cargo-chef && \\\n    rm -rf /opt/rust/cargo/registry/\nWORKDIR /app\n\nFROM chef AS planner\nCOPY Cargo.toml .\nCOPY Cargo.lock .\nCOPY crates/ crates/\nCOPY resources/ resources/\nCOPY tests/ tests/\nRUN cargo chef prepare --recipe-path recipe.json\n\nFROM chef AS builder\nCOPY --from=planner /app/recipe.json recipe.json\nRUN cargo chef cook --release --recipe-path recipe.json\nCOPY Cargo.toml .\nCOPY Cargo.lock .\nCOPY crates/ crates/\nCOPY resources/ resources/\nCOPY tests/ tests/\nRUN cargo build -p stalwart --no-default-features --features \"foundationdb elastic s3 redis azure nats enterprise\" --release\n\nFROM debian:trixie-slim AS runtime\n\nCOPY --from=builder /app/target/release/stalwart /usr/local/bin/stalwart\nRUN apt-get update -y && apt-get install -yq --no-install-recommends ca-certificates curl adduser\nRUN curl -LO https://github.com/apple/foundationdb/releases/download/7.3.69/foundationdb-clients_7.3.69-1_amd64.deb && \\\n    dpkg -i foundationdb-clients_7.3.69-1_amd64.deb\nRUN useradd stalwart -s /sbin/nologin -M\nRUN mkdir -p /opt/stalwart\nRUN chown stalwart:stalwart /opt/stalwart\n\nENTRYPOINT [\"/usr/local/bin/stalwart\", \"--config\", \"/opt/stalwart/etc/config.toml\"]\n"
  },
  {
    "path": "resources/docker/download.sh",
    "content": "#!/usr/bin/env sh\n# shellcheck shell=dash\n\n# Stalwart install script -- based on the rustup installation script.\n\nset -e\nset -u\n\nreadonly BASE_URL=\"https://github.com/stalwartlabs/stalwart/releases/latest/download\"\n\nmain() {\n    downloader --check\n    need_cmd uname\n    need_cmd mktemp\n    need_cmd chmod\n    need_cmd mkdir\n    need_cmd rm\n    need_cmd rmdir\n    need_cmd tar\n\n    # Make sure we are running as root\n    if [ \"$(id -u)\" -ne 0 ] ; then\n        err \"❌ Install failed: This program needs to run as root.\"\n    fi\n\n    # Detect OS\n    local _os=\"unknown\"\n    local _uname=\"$(uname)\"\n    _account=\"stalwart\"\n    if [ \"${_uname}\" = \"Linux\" ]; then\n        _os=\"linux\"\n    elif [ \"${_uname}\" = \"Darwin\" ]; then\n        _os=\"macos\"\n        _account=\"_stalwart\"\n    fi\n\n    # Default component setting\n    local _component=\"stalwart\"\n    local _dir=\"/usr/local/bin\"\n\n    # Detect platform architecture\n    get_architecture || return 1\n    local _arch=\"$RETVAL\"\n    assert_nz \"$_arch\" \"arch\"\n\n    # Download latest binary\n    say \"⏳ Downloading ${_component} for ${_arch}...\"\n    local _file=\"${_dir}/stalwart.tar.gz\"\n    local _url=\"${BASE_URL}/${_component}-${_arch}.tar.gz\"\n    ensure downloader \"$_url\" \"$_file\" \"$_arch\"\n    ensure tar zxvf \"$_file\" -C \"$_dir\"\n    ignore chmod +x \"$_dir/stalwart\"\n    ignore rm \"$_file\"\n\n    return 0\n}\n\nget_architecture() {\n    local _ostype _cputype _bitness _arch _clibtype\n    _ostype=\"$(uname -s)\"\n    _cputype=\"$(uname -m)\"\n    _clibtype=\"gnu\"\n\n    if [ \"$_ostype\" = Linux ]; then\n        if [ \"$(uname -o)\" = Android ]; then\n            _ostype=Android\n        fi\n        if ldd --version 2>&1 | grep -q 'musl'; then\n            _clibtype=\"musl\"\n        fi\n    fi\n\n    if [ \"$_ostype\" = Darwin ] && [ \"$_cputype\" = i386 ]; then\n        # Darwin `uname -m` lies\n        if sysctl hw.optional.x86_64 | grep -q ': 1'; then\n            _cputype=x86_64\n        fi\n    fi\n\n    if [ \"$_ostype\" = SunOS ]; then\n        # Both Solaris and illumos presently announce as \"SunOS\" in \"uname -s\"\n        # so use \"uname -o\" to disambiguate.  We use the full path to the\n        # system uname in case the user has coreutils uname first in PATH,\n        # which has historically sometimes printed the wrong value here.\n        if [ \"$(/usr/bin/uname -o)\" = illumos ]; then\n            _ostype=illumos\n        fi\n\n        # illumos systems have multi-arch userlands, and \"uname -m\" reports the\n        # machine hardware name; e.g., \"i86pc\" on both 32- and 64-bit x86\n        # systems.  Check for the native (widest) instruction set on the\n        # running kernel:\n        if [ \"$_cputype\" = i86pc ]; then\n            _cputype=\"$(isainfo -n)\"\n        fi\n    fi\n\n    case \"$_ostype\" in\n\n        Android)\n            _ostype=linux-android\n            ;;\n\n        Linux)\n            check_proc\n            _ostype=unknown-linux-$_clibtype\n            _bitness=$(get_bitness)\n            ;;\n\n        FreeBSD)\n            _ostype=unknown-freebsd\n            ;;\n\n        NetBSD)\n            _ostype=unknown-netbsd\n            ;;\n\n        DragonFly)\n            _ostype=unknown-dragonfly\n            ;;\n\n        Darwin)\n            _ostype=apple-darwin\n            ;;\n\n        illumos)\n            _ostype=unknown-illumos\n            ;;\n\n        MINGW* | MSYS* | CYGWIN* | Windows_NT)\n            _ostype=pc-windows-gnu\n            ;;\n\n        *)\n            err \"unrecognized OS type: $_ostype\"\n            ;;\n\n    esac\n\n    case \"$_cputype\" in\n\n        i386 | i486 | i686 | i786 | x86)\n            _cputype=i686\n            ;;\n\n        xscale | arm)\n            _cputype=arm\n            if [ \"$_ostype\" = \"linux-android\" ]; then\n                _ostype=linux-androideabi\n            fi\n            ;;\n\n        armv6l)\n            _cputype=arm\n            if [ \"$_ostype\" = \"linux-android\" ]; then\n                _ostype=linux-androideabi\n            else\n                _ostype=\"${_ostype}eabihf\"\n            fi\n            ;;\n\n        armv7l | armv8l)\n            _cputype=armv7\n            if [ \"$_ostype\" = \"linux-android\" ]; then\n                _ostype=linux-androideabi\n            else\n                _ostype=\"${_ostype}eabihf\"\n            fi\n            ;;\n\n        aarch64 | arm64)\n            _cputype=aarch64\n            ;;\n\n        x86_64 | x86-64 | x64 | amd64)\n            _cputype=x86_64\n            ;;\n\n        mips)\n            _cputype=$(get_endianness mips '' el)\n            ;;\n\n        mips64)\n            if [ \"$_bitness\" -eq 64 ]; then\n                # only n64 ABI is supported for now\n                _ostype=\"${_ostype}abi64\"\n                _cputype=$(get_endianness mips64 '' el)\n            fi\n            ;;\n\n        ppc)\n            _cputype=powerpc\n            ;;\n\n        ppc64)\n            _cputype=powerpc64\n            ;;\n\n        ppc64le)\n            _cputype=powerpc64le\n            ;;\n\n        s390x)\n            _cputype=s390x\n            ;;\n        riscv64)\n            _cputype=riscv64gc\n            ;;\n        *)\n            err \"unknown CPU type: $_cputype\"\n\n    esac\n\n    # Detect 64-bit linux with 32-bit userland\n    if [ \"${_ostype}\" = unknown-linux-gnu ] && [ \"${_bitness}\" -eq 32 ]; then\n        case $_cputype in\n            x86_64)\n                if [ -n \"${RUSTUP_CPUTYPE:-}\" ]; then\n                    _cputype=\"$RUSTUP_CPUTYPE\"\n                else {\n                    # 32-bit executable for amd64 = x32\n                    if is_host_amd64_elf; then {\n                         echo \"This host is running an x32 userland; as it stands, x32 support is poor,\" 1>&2\n                         echo \"and there isn't a native toolchain -- you will have to install\" 1>&2\n                         echo \"multiarch compatibility with i686 and/or amd64, then select one\" 1>&2\n                         echo \"by re-running this script with the RUSTUP_CPUTYPE environment variable\" 1>&2\n                         echo \"set to i686 or x86_64, respectively.\" 1>&2\n                         echo 1>&2\n                         echo \"You will be able to add an x32 target after installation by running\" 1>&2\n                         echo \"  rustup target add x86_64-unknown-linux-gnux32\" 1>&2\n                         exit 1\n                    }; else\n                        _cputype=i686\n                    fi\n                }; fi\n                ;;\n            mips64)\n                _cputype=$(get_endianness mips '' el)\n                ;;\n            powerpc64)\n                _cputype=powerpc\n                ;;\n            aarch64)\n                _cputype=armv7\n                if [ \"$_ostype\" = \"linux-android\" ]; then\n                    _ostype=linux-androideabi\n                else\n                    _ostype=\"${_ostype}eabihf\"\n                fi\n                ;;\n            riscv64gc)\n                err \"riscv64 with 32-bit userland unsupported\"\n                ;;\n        esac\n    fi\n\n    # Detect armv7 but without the CPU features Rust needs in that build,\n    # and fall back to arm.\n    # See https://github.com/rust-lang/rustup.rs/issues/587.\n    if [ \"$_ostype\" = \"unknown-linux-gnueabihf\" ] && [ \"$_cputype\" = armv7 ]; then\n        if ensure grep '^Features' /proc/cpuinfo | grep -q -v neon; then\n            # At least one processor does not have NEON.\n            _cputype=arm\n        fi\n    fi\n\n    _arch=\"${_cputype}-${_ostype}\"\n\n    RETVAL=\"$_arch\"\n}\n\ncheck_proc() {\n    # Check for /proc by looking for the /proc/self/exe link\n    # This is only run on Linux\n    if ! test -L /proc/self/exe ; then\n        err \"fatal: Unable to find /proc/self/exe.  Is /proc mounted?  Installation cannot proceed without /proc.\"\n    fi\n}\n\nget_bitness() {\n    need_cmd head\n    # Architecture detection without dependencies beyond coreutils.\n    # ELF files start out \"\\x7fELF\", and the following byte is\n    #   0x01 for 32-bit and\n    #   0x02 for 64-bit.\n    # The printf builtin on some shells like dash only supports octal\n    # escape sequences, so we use those.\n    local _current_exe_head\n    _current_exe_head=$(head -c 5 /proc/self/exe )\n    if [ \"$_current_exe_head\" = \"$(printf '\\177ELF\\001')\" ]; then\n        echo 32\n    elif [ \"$_current_exe_head\" = \"$(printf '\\177ELF\\002')\" ]; then\n        echo 64\n    else\n        err \"unknown platform bitness\"\n    fi\n}\n\nis_host_amd64_elf() {\n    need_cmd head\n    need_cmd tail\n    # ELF e_machine detection without dependencies beyond coreutils.\n    # Two-byte field at offset 0x12 indicates the CPU,\n    # but we're interested in it being 0x3E to indicate amd64, or not that.\n    local _current_exe_machine\n    _current_exe_machine=$(head -c 19 /proc/self/exe | tail -c 1)\n    [ \"$_current_exe_machine\" = \"$(printf '\\076')\" ]\n}\n\nget_endianness() {\n    local cputype=$1\n    local suffix_eb=$2\n    local suffix_el=$3\n\n    # detect endianness without od/hexdump, like get_bitness() does.\n    need_cmd head\n    need_cmd tail\n\n    local _current_exe_endianness\n    _current_exe_endianness=\"$(head -c 6 /proc/self/exe | tail -c 1)\"\n    if [ \"$_current_exe_endianness\" = \"$(printf '\\001')\" ]; then\n        echo \"${cputype}${suffix_el}\"\n    elif [ \"$_current_exe_endianness\" = \"$(printf '\\002')\" ]; then\n        echo \"${cputype}${suffix_eb}\"\n    else\n        err \"unknown platform endianness\"\n    fi\n}\n\nsay() {\n    printf '%s\\n' \"$1\"\n}\n\nerr() {\n    say \"$1\" >&2\n    exit 1\n}\n\nneed_cmd() {\n    if ! check_cmd \"$1\"; then\n        err \"need '$1' (command not found)\"\n    fi\n}\n\ncheck_cmd() {\n    command -v \"$1\" > /dev/null 2>&1\n}\n\nassert_nz() {\n    if [ -z \"$1\" ]; then err \"assert_nz $2\"; fi\n}\n\n# Run a command that should never fail. If the command fails execution\n# will immediately terminate with an error showing the failing\n# command.\nensure() {\n    if ! \"$@\"; then err \"command failed: $*\"; fi\n}\n\n# This wraps curl or wget. Try curl first, if not installed,\n# use wget instead.\ndownloader() {\n    local _dld\n    local _ciphersuites\n    local _err\n    local _status\n    local _retry\n    if check_cmd curl; then\n        _dld=curl\n    elif check_cmd wget; then\n        _dld=wget\n    else\n        _dld='curl or wget' # to be used in error message of need_cmd\n    fi\n\n    if [ \"$1\" = --check ]; then\n        need_cmd \"$_dld\"\n    elif [ \"$_dld\" = curl ]; then\n        check_curl_for_retry_support\n        _retry=\"$RETVAL\"\n        get_ciphersuites_for_curl\n        _ciphersuites=\"$RETVAL\"\n        if [ -n \"$_ciphersuites\" ]; then\n            _err=$(curl $_retry --proto '=https' --tlsv1.2 --ciphers \"$_ciphersuites\" --silent --show-error --fail --location \"$1\" --output \"$2\" 2>&1)\n            _status=$?\n        else\n            echo \"Warning: Not enforcing strong cipher suites for TLS, this is potentially less secure\"\n            if ! check_help_for \"$3\" curl --proto --tlsv1.2; then\n                echo \"Warning: Not enforcing TLS v1.2, this is potentially less secure\"\n                _err=$(curl $_retry --silent --show-error --fail --location \"$1\" --output \"$2\" 2>&1)\n                _status=$?\n            else\n                _err=$(curl $_retry --proto '=https' --tlsv1.2 --silent --show-error --fail --location \"$1\" --output \"$2\" 2>&1)\n                _status=$?\n            fi\n        fi\n        if [ -n \"$_err\" ]; then\n            if echo \"$_err\" | grep -q 404; then\n                err \"❌  Binary for platform '$3' not found, this platform may be unsupported.\"\n            else\n                echo \"$_err\" >&2\n            fi\n        fi\n        return $_status\n    elif [ \"$_dld\" = wget ]; then\n        if [ \"$(wget -V 2>&1|head -2|tail -1|cut -f1 -d\" \")\" = \"BusyBox\" ]; then\n            echo \"Warning: using the BusyBox version of wget.  Not enforcing strong cipher suites for TLS or TLS v1.2, this is potentially less secure\"\n            _err=$(wget \"$1\" -O \"$2\" 2>&1)\n            _status=$?\n        else\n            get_ciphersuites_for_wget\n            _ciphersuites=\"$RETVAL\"\n            if [ -n \"$_ciphersuites\" ]; then\n                _err=$(wget --https-only --secure-protocol=TLSv1_2 --ciphers \"$_ciphersuites\" \"$1\" -O \"$2\" 2>&1)\n                _status=$?\n            else\n                echo \"Warning: Not enforcing strong cipher suites for TLS, this is potentially less secure\"\n                if ! check_help_for \"$3\" wget --https-only --secure-protocol; then\n                    echo \"Warning: Not enforcing TLS v1.2, this is potentially less secure\"\n                    _err=$(wget \"$1\" -O \"$2\" 2>&1)\n                    _status=$?\n                else\n                    _err=$(wget --https-only --secure-protocol=TLSv1_2 \"$1\" -O \"$2\" 2>&1)\n                    _status=$?\n                fi\n            fi\n        fi\n        if [ -n \"$_err\" ]; then\n            if echo \"$_err\" | grep -q ' 404 Not Found'; then\n                err \"❌  Binary for platform '$3' not found, this platform may be unsupported.\"\n            else\n                echo \"$_err\" >&2\n            fi\n        fi\n        return $_status\n    else\n        err \"Unknown downloader\"   # should not reach here\n    fi\n}\n\n# Check if curl supports the --retry flag, then pass it to the curl invocation.\ncheck_curl_for_retry_support() {\n  local _retry_supported=\"\"\n  # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n  if check_help_for \"notspecified\" \"curl\" \"--retry\"; then\n    _retry_supported=\"--retry 3\"\n  fi\n\n  RETVAL=\"$_retry_supported\"\n\n}\n\ncheck_help_for() {\n    local _arch\n    local _cmd\n    local _arg\n    _arch=\"$1\"\n    shift\n    _cmd=\"$1\"\n    shift\n\n    local _category\n    if \"$_cmd\" --help | grep -q 'For all options use the manual or \"--help all\".'; then\n      _category=\"all\"\n    else\n      _category=\"\"\n    fi\n\n    case \"$_arch\" in\n\n        *darwin*)\n        if check_cmd sw_vers; then\n            case $(sw_vers -productVersion) in\n                10.*)\n                    # If we're running on macOS, older than 10.13, then we always\n                    # fail to find these options to force fallback\n                    if [ \"$(sw_vers -productVersion | cut -d. -f2)\" -lt 13 ]; then\n                        # Older than 10.13\n                        echo \"Warning: Detected macOS platform older than 10.13\"\n                        return 1\n                    fi\n                    ;;\n                11.*)\n                    # We assume Big Sur will be OK for now\n                    ;;\n                *)\n                    # Unknown product version, warn and continue\n                    echo \"Warning: Detected unknown macOS major version: $(sw_vers -productVersion)\"\n                    echo \"Warning TLS capabilities detection may fail\"\n                    ;;\n            esac\n        fi\n        ;;\n\n    esac\n\n    for _arg in \"$@\"; do\n        if ! \"$_cmd\" --help $_category | grep -q -- \"$_arg\"; then\n            return 1\n        fi\n    done\n\n    true # not strictly needed\n}\n\n# Return cipher suite string specified by user, otherwise return strong TLS 1.2-1.3 cipher suites\n# if support by local tools is detected. Detection currently supports these curl backends:\n# GnuTLS and OpenSSL (possibly also LibreSSL and BoringSSL). Return value can be empty.\nget_ciphersuites_for_curl() {\n    if [ -n \"${RUSTUP_TLS_CIPHERSUITES-}\" ]; then\n        # user specified custom cipher suites, assume they know what they're doing\n        RETVAL=\"$RUSTUP_TLS_CIPHERSUITES\"\n        return\n    fi\n\n    local _openssl_syntax=\"no\"\n    local _gnutls_syntax=\"no\"\n    local _backend_supported=\"yes\"\n    if curl -V | grep -q ' OpenSSL/'; then\n        _openssl_syntax=\"yes\"\n    elif curl -V | grep -iq ' LibreSSL/'; then\n        _openssl_syntax=\"yes\"\n    elif curl -V | grep -iq ' BoringSSL/'; then\n        _openssl_syntax=\"yes\"\n    elif curl -V | grep -iq ' GnuTLS/'; then\n        _gnutls_syntax=\"yes\"\n    else\n        _backend_supported=\"no\"\n    fi\n\n    local _args_supported=\"no\"\n    if [ \"$_backend_supported\" = \"yes\" ]; then\n        # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n        if check_help_for \"notspecified\" \"curl\" \"--tlsv1.2\" \"--ciphers\" \"--proto\"; then\n            _args_supported=\"yes\"\n        fi\n    fi\n\n    local _cs=\"\"\n    if [ \"$_args_supported\" = \"yes\" ]; then\n        if [ \"$_openssl_syntax\" = \"yes\" ]; then\n            _cs=$(get_strong_ciphersuites_for \"openssl\")\n        elif [ \"$_gnutls_syntax\" = \"yes\" ]; then\n            _cs=$(get_strong_ciphersuites_for \"gnutls\")\n        fi\n    fi\n\n    RETVAL=\"$_cs\"\n}\n\n# Return cipher suite string specified by user, otherwise return strong TLS 1.2-1.3 cipher suites\n# if support by local tools is detected. Detection currently supports these wget backends:\n# GnuTLS and OpenSSL (possibly also LibreSSL and BoringSSL). Return value can be empty.\nget_ciphersuites_for_wget() {\n    if [ -n \"${RUSTUP_TLS_CIPHERSUITES-}\" ]; then\n        # user specified custom cipher suites, assume they know what they're doing\n        RETVAL=\"$RUSTUP_TLS_CIPHERSUITES\"\n        return\n    fi\n\n    local _cs=\"\"\n    if wget -V | grep -q '\\-DHAVE_LIBSSL'; then\n        # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n        if check_help_for \"notspecified\" \"wget\" \"TLSv1_2\" \"--ciphers\" \"--https-only\" \"--secure-protocol\"; then\n            _cs=$(get_strong_ciphersuites_for \"openssl\")\n        fi\n    elif wget -V | grep -q '\\-DHAVE_LIBGNUTLS'; then\n        # \"unspecified\" is for arch, allows for possibility old OS using macports, homebrew, etc.\n        if check_help_for \"notspecified\" \"wget\" \"TLSv1_2\" \"--ciphers\" \"--https-only\" \"--secure-protocol\"; then\n            _cs=$(get_strong_ciphersuites_for \"gnutls\")\n        fi\n    fi\n\n    RETVAL=\"$_cs\"\n}\n\n# Return strong TLS 1.2-1.3 cipher suites in OpenSSL or GnuTLS syntax. TLS 1.2\n# excludes non-ECDHE and non-AEAD cipher suites. DHE is excluded due to bad\n# DH params often found on servers (see RFC 7919). Sequence matches or is\n# similar to Firefox 68 ESR with weak cipher suites disabled via about:config.\n# $1 must be openssl or gnutls.\nget_strong_ciphersuites_for() {\n    if [ \"$1\" = \"openssl\" ]; then\n        # OpenSSL is forgiving of unknown values, no problems with TLS 1.3 values on versions that don't support it yet.\n        echo \"TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384\"\n    elif [ \"$1\" = \"gnutls\" ]; then\n        # GnuTLS isn't forgiving of unknown values, so this may require a GnuTLS version that supports TLS 1.3 even if wget doesn't.\n        # Begin with SECURE128 (and higher) then remove/add to build cipher suites. Produces same 9 cipher suites as OpenSSL but in slightly different order.\n        echo \"SECURE128:-VERS-SSL3.0:-VERS-TLS1.0:-VERS-TLS1.1:-VERS-DTLS-ALL:-CIPHER-ALL:-MAC-ALL:-KX-ALL:+AEAD:+ECDHE-ECDSA:+ECDHE-RSA:+AES-128-GCM:+CHACHA20-POLY1305:+AES-256-GCM\"\n    fi\n}\n\n# This is just for indicating that commands' results are being\n# intentionally ignored. Usually, because it's being executed\n# as part of error handling.\nignore() {\n    \"$@\"\n}\n\nmain \"$@\" || exit 1\n"
  },
  {
    "path": "resources/docker/entrypoint.sh",
    "content": "#!/usr/bin/env sh\n# shellcheck shell=dash\n\n# If the configuration file does not exist initialize it.\nif [ ! -f /opt/stalwart/etc/config.toml ]; then\n    /usr/local/bin/stalwart --init /opt/stalwart\nfi\n\n# If the configuration file exists, start the server.\nexec /usr/local/bin/stalwart --config /opt/stalwart/etc/config.toml\n"
  },
  {
    "path": "resources/html-templates/calendar-alarm.html",
    "content": "<!doctype html>\n<html xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:v=\"urn:schemas-microsoft-com:vml\"\n  xmlns:o=\"urn:schemas-microsoft-com:office:office\">\n\n<head>\n  <title>{{page_title}}</title><!--[if !mso]><!-->\n  <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\"><!--<![endif]-->\n  <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\">\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n  <style type=\"text/css\">\n    #outlook a {\n      padding: 0;\n    }\n\n    body {\n      margin: 0;\n      padding: 0;\n      -webkit-text-size-adjust: 100%;\n      -ms-text-size-adjust: 100%;\n    }\n\n    table,\n    td {\n      border-collapse: collapse;\n      mso-table-lspace: 0pt;\n      mso-table-rspace: 0pt;\n    }\n\n    img {\n      border: 0;\n      height: auto;\n      line-height: 100%;\n      outline: none;\n      text-decoration: none;\n      -ms-interpolation-mode: bicubic;\n    }\n\n    p {\n      display: block;\n      margin: 13px 0;\n    }\n  </style><!--[if mso]>\n        <noscript>\n        <xml>\n        <o:OfficeDocumentSettings>\n          <o:AllowPNG/>\n          <o:PixelsPerInch>96</o:PixelsPerInch>\n        </o:OfficeDocumentSettings>\n        </xml>\n        </noscript>\n        <![endif]--><!--[if lte mso 11]>\n        <style type=\"text/css\">\n          .mj-outlook-group-fix { width:100% !important; }\n        </style>\n        <![endif]-->\n  <style type=\"text/css\">\n    @media only screen and (min-width:480px) {\n      .mj-column-per-100 {\n        width: 100% !important;\n        max-width: 100%;\n      }\n    }\n  </style>\n  <style media=\"screen and (min-width:480px)\">\n    .moz-text-html .mj-column-per-100 {\n      width: 100% !important;\n      max-width: 100%;\n    }\n  </style>\n  <style type=\"text/css\">\n    @media only screen and (max-width:480px) {\n      table.mj-full-width-mobile {\n        width: 100% !important;\n      }\n\n      td.mj-full-width-mobile {\n        width: auto !important;\n      }\n    }\n  </style>\n  <style type=\"text/css\">\n    .event-detail {\n      font-weight: bold;\n      color: #2c5aa0;\n    }\n\n    .guest-list {\n      background-color: #f8f9fa;\n      padding: 10px;\n      border-radius: 4px;\n      margin-top: 5px;\n    }\n\n    :root {\n      color-scheme: light only;\n    }\n  </style>\n</head>\n\n<body style=\"word-spacing:normal;background-color:#f4f4f4;\">\n  <div style=\"background-color:#f4f4f4;\">\n    <!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px;\" width=\"600\" bgcolor=\"#ffffff\" ><tr><td style=\"line-height:0px;font-size:0px;mso-line-height-rule:exactly;\"><![endif]-->\n    <div style=\"background-color:#ffffff;margin:0px auto;max-width:600px;\">\n      <table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n        style=\"background-color:#ffffff;width:100%;\">\n        <tbody>\n          <tr>\n            <td style=\"direction:ltr;font-size:0px;padding:20px 0;text-align:center;\">\n              <!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:600px;\" ><![endif]-->\n              <div class=\"mj-column-per-100 mj-outlook-group-fix\"\n                style=\"font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;\">\n                <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"vertical-align:top;\"\n                  width=\"100%\">\n                  <tbody>\n                    <tr>\n                      <td align=\"center\" style=\"font-size:0px;padding:10px 25px;word-break:break-word;\">\n                        <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                          style=\"border-collapse:collapse;border-spacing:0px;\">\n                          <tbody>\n                            <tr>\n                              <td style=\"width:200px;\"><img alt=\"Logo\" height=\"auto\" src=\"cid:{{logo_cid}}\"\n                                  style=\"border:0;display:block;outline:none;text-decoration:none;height:auto;width:100%;font-size:13px;\"\n                                  width=\"200\"></td>\n                            </tr>\n                          </tbody>\n                        </table>\n                      </td>\n                    </tr>\n                  </tbody>\n                </table>\n              </div><!--[if mso | IE]></td></tr></table><![endif]-->\n            </td>\n          </tr>\n        </tbody>\n      </table>\n    </div>\n    <!--[if mso | IE]></td></tr></table><![endif]--><!-- Main Content --><!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px;\" width=\"600\" bgcolor=\"#ffffff\" ><tr><td style=\"line-height:0px;font-size:0px;mso-line-height-rule:exactly;\"><![endif]-->\n    <div style=\"background-color:#ffffff;margin:0px auto;max-width:600px;\">\n      <table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n        style=\"background-color:#ffffff;width:100%;\">\n        <tbody>\n          <tr>\n            <td style=\"direction:ltr;font-size:0px;padding:0 20px;text-align:center;\">\n              <!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:560px;\" ><![endif]-->\n              <div class=\"mj-column-per-100 mj-outlook-group-fix\"\n                style=\"font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;\">\n                <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"vertical-align:top;\"\n                  width=\"100%\">\n                  <tbody>\n                    <tr>\n                      <td align=\"center\" style=\"font-size:0px;padding:20px 0;word-break:break-word;\">\n                        <div\n                          style=\"font-family:Arial, sans-serif;font-size:20px;font-weight:bold;line-height:1.5;text-align:center;color:#2c5aa0;\">\n                          {{header}}</div>\n                      </td>\n                    </tr>\n                    <tr>\n                      <td align=\"left\" style=\"font-size:0px;padding:10px 0 5px 0;word-break:break-word;\">\n                        <div\n                          style=\"font-family:Arial, sans-serif;font-size:24px;font-weight:bold;line-height:1.5;text-align:left;color:#333333;\">\n                          {{event_title}}</div>\n                      </td>\n                    </tr>\n                    {{#if event_description}}\n                    <tr>\n                      <td align=\"left\" style=\"font-size:0px;padding:0 0 20px 0;word-break:break-word;\">\n                        <div\n                          style=\"font-family:Arial, sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#666666;\">\n                          {{event_description}}</div>\n                      </td>\n                    </tr>\n                    {{/if event_description}}\n                    {{#each event_details}}\n                    <tr>\n                      <td align=\"left\" style=\"font-size:0px;padding:5px 0;word-break:break-word;\">\n                        <div\n                          style=\"font-family:Arial, sans-serif;font-size:14px;line-height:1.5;text-align:left;color:#333333;\">\n                          <span class=\"event-detail\" style=\"font-weight: bold; color: #2c5aa0;\">{{key}}:</span>\n                          {{value}}\n                        </div>\n                      </td>\n                    </tr>\n                    {{/each event_details}}\n                    {{#if attendees}}\n                    <tr>\n                      <td align=\"left\" style=\"font-size:0px;padding:15px 0 5px 0;word-break:break-word;\">\n                        <div\n                          style=\"font-family:Arial, sans-serif;font-size:16px;font-weight:bold;line-height:1.5;text-align:left;color:#333333;\">\n                          {{attendees_title}}:</div>\n                      </td>\n                    </tr>\n                    <tr>\n                      <td align=\"left\" style=\"font-size:0px;padding:0 0 20px 0;word-break:break-word;\">\n                        <div\n                          style=\"font-family:Arial, sans-serif;font-size:14px;line-height:1.5;text-align:left;color:#333333;\">\n                          <div class=\"guest-list\"\n                            style=\"background-color: #f8f9fa; padding: 10px; border-radius: 4px; margin-top: 5px;\">\n                            {{#each attendees}}• {{key}} &lt;{{value}}&gt;<br>{{/each attendees}}</div>\n                        </div>\n                      </td>\n                    </tr>\n                    {{/if attendees}}\n                    <tr>\n                      <td align=\"center\" vertical-align=\"middle\"\n                        style=\"font-size:0px;padding:15px 30px;word-break:break-word;\">\n                        <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                          style=\"border-collapse:separate;line-height:100%;\">\n                          <tr>\n                            <td align=\"center\" bgcolor=\"#2c5aa0\" role=\"presentation\"\n                              style=\"border:none;border-radius:6px;cursor:auto;mso-padding-alt:10px 25px;background:#2c5aa0;\"\n                              valign=\"middle\"><a href=\"{{action_url}}\"\n                                style=\"display:inline-block;background:#2c5aa0;color:#ffffff;font-family:Arial, sans-serif;font-size:16px;font-weight:bold;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:6px;\"\n                                target=\"_blank\">{{action_name}}</a></td>\n                          </tr>\n                        </table>\n                      </td>\n                    </tr>\n                  </tbody>\n                </table>\n              </div><!--[if mso | IE]></td></tr></table><![endif]-->\n            </td>\n          </tr>\n        </tbody>\n      </table>\n    </div>\n    <!--[if mso | IE]></td></tr></table><![endif]--><!-- Footer --><!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px;\" width=\"600\" bgcolor=\"#f8f9fa\" ><tr><td style=\"line-height:0px;font-size:0px;mso-line-height-rule:exactly;\"><![endif]-->\n    <div style=\"background-color:#f8f9fa;margin:0px auto;max-width:600px;\">\n      <table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n        style=\"background-color:#f8f9fa;width:100%;\">\n        <tbody>\n          <tr>\n            <td style=\"direction:ltr;font-size:0px;padding:20px;text-align:center;\">\n              <!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:560px;\" ><![endif]-->\n              <div class=\"mj-column-per-100 mj-outlook-group-fix\"\n                style=\"font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;\">\n                <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"vertical-align:top;\"\n                  width=\"100%\">\n                  <tbody>\n                    <tr>\n                      <td align=\"center\" style=\"font-size:0px;padding:0 0 15px 0;word-break:break-word;\">\n                        <p style=\"border-top:solid 1px #e9ecef;font-size:1px;margin:0px auto;width:100%;\"></p><!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" style=\"border-top:solid 1px #e9ecef;font-size:1px;margin:0px auto;width:560px;\" role=\"presentation\" width=\"560px\" ><tr><td style=\"height:0;line-height:0;\"> &nbsp;\n</td></tr></table><![endif]-->\n                      </td>\n                    </tr>\n                    <tr>\n                      <td align=\"center\" style=\"font-size:0px;padding:10px 25px;word-break:break-word;\">\n                        <div\n                          style=\"font-family:Arial, sans-serif;font-size:12px;line-height:1.4;text-align:center;color:#6c757d;\">\n                          {{footer}}</div>\n                      </td>\n                    </tr>\n                  </tbody>\n                </table>\n              </div><!--[if mso | IE]></td></tr></table><![endif]-->\n            </td>\n          </tr>\n        </tbody>\n      </table>\n    </div><!--[if mso | IE]></td></tr></table><![endif]-->\n  </div>\n</body>\n\n</html>"
  },
  {
    "path": "resources/html-templates/calendar-alarm.html.min",
    "content": "<!doctypehtml><html xmlns=http://www.w3.org/1999/xhtml xmlns:o=urn:schemas-microsoft-com:office:office xmlns:v=urn:schemas-microsoft-com:vml><title>{{page_title}}</title><!--[if !mso]><!--><meta content=\"IE=edge\"http-equiv=X-UA-Compatible><!--<![endif]--><meta content=\"text/html; charset=UTF-8\"http-equiv=Content-Type><meta content=\"width=device-width,initial-scale=1\"name=viewport><style>#outlook a{padding:0}body{margin:0;padding:0;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}table,td{border-collapse:collapse;mso-table-lspace:0;mso-table-rspace:0}img{border:0;height:auto;line-height:100%;outline:0;text-decoration:none;-ms-interpolation-mode:bicubic}p{display:block;margin:13px 0}</style><!--[if mso]><noscript><xml><o:officedocumentsettings><o:allowpng><o:pixelsperinch>96</o:pixelsperinch></o:officedocumentsettings></xml></noscript><![endif]--><!--[if lte mso 11]><style>.mj-outlook-group-fix{width:100%!important}</style><![endif]--><style>@media only screen and (min-width:480px){.mj-column-per-100{width:100%!important;max-width:100%}}</style><style media=\"screen and (min-width:480px)\">.moz-text-html .mj-column-per-100{width:100%!important;max-width:100%}</style><style>@media only screen and (max-width:480px){table.mj-full-width-mobile{width:100%!important}td.mj-full-width-mobile{width:auto!important}}</style><style>.event-detail{font-weight:700;color:#2c5aa0}.guest-list{background-color:#f8f9fa;padding:10px;border-radius:4px;margin-top:5px}:root{color-scheme:light only}</style><body style=word-spacing:normal;background-color:#f4f4f4><div style=background-color:#f4f4f4><!--[if mso | IE]><table border=0 cellpadding=0 cellspacing=0 align=center style=width:600px width=600 bgcolor=#ffffff><tr><td style=line-height:0;font-size:0;mso-line-height-rule:exactly><![endif]--><div style=\"background-color:#fff;margin:0 auto;max-width:600px\"><table border=0 cellpadding=0 cellspacing=0 role=presentation style=background-color:#fff;width:100% align=center><tr><td style=\"direction:ltr;font-size:0;padding:20px 0;text-align:center\"><!--[if mso | IE]><table border=0 cellpadding=0 cellspacing=0 role=presentation><tr><td style=vertical-align:top;width:600px><![endif]--><div style=font-size:0;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100% class=\"mj-column-per-100 mj-outlook-group-fix\"><table border=0 cellpadding=0 cellspacing=0 role=presentation style=vertical-align:top width=100%><tr><td style=\"font-size:0;padding:10px 25px;word-break:break-word\"align=center><table border=0 cellpadding=0 cellspacing=0 role=presentation style=border-collapse:collapse;border-spacing:0><tr><td style=width:200px><img alt=Logo height=auto src=cid:{{logo_cid}} style=border:0;display:block;outline:0;text-decoration:none;height:auto;width:100%;font-size:13px width=200></table></table></div><!--[if mso | IE]><![endif]--></table></div><!--[if mso | IE]><![endif]--><!--[if mso | IE]><table border=0 cellpadding=0 cellspacing=0 align=center style=width:600px width=600 bgcolor=#ffffff><tr><td style=line-height:0;font-size:0;mso-line-height-rule:exactly><![endif]--><div style=\"background-color:#fff;margin:0 auto;max-width:600px\"><table border=0 cellpadding=0 cellspacing=0 role=presentation style=background-color:#fff;width:100% align=center><tr><td style=\"direction:ltr;font-size:0;padding:0 20px;text-align:center\"><!--[if mso | IE]><table border=0 cellpadding=0 cellspacing=0 role=presentation><tr><td style=vertical-align:top;width:560px><![endif]--><div style=font-size:0;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100% class=\"mj-column-per-100 mj-outlook-group-fix\"><table border=0 cellpadding=0 cellspacing=0 role=presentation style=vertical-align:top width=100%><tr><td style=\"font-size:0;padding:20px 0;word-break:break-word\"align=center><div style=font-family:Arial,sans-serif;font-size:20px;font-weight:700;line-height:1.5;text-align:center;color:#2c5aa0>{{header}}</div><tr><td style=\"font-size:0;padding:10px 0 5px 0;word-break:break-word\"align=left><div style=font-family:Arial,sans-serif;font-size:24px;font-weight:700;line-height:1.5;text-align:left;color:#333>{{event_title}}</div></tr>{{#if event_description}}<tr><td style=\"font-size:0;padding:0 0 20px 0;word-break:break-word\"align=left><div style=font-family:Arial,sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#666>{{event_description}}</div></tr>{{/if event_description}} {{#each event_details}}<tr><td style=\"font-size:0;padding:5px 0;word-break:break-word\"align=left><div style=font-family:Arial,sans-serif;font-size:14px;line-height:1.5;text-align:left;color:#333><span class=event-detail style=font-weight:700;color:#2c5aa0>{{key}}:</span> {{value}}</div></tr>{{/each event_details}} {{#if attendees}}<tr><td style=\"font-size:0;padding:15px 0 5px 0;word-break:break-word\"align=left><div style=font-family:Arial,sans-serif;font-size:16px;font-weight:700;line-height:1.5;text-align:left;color:#333>{{attendees_title}}:</div><tr><td style=\"font-size:0;padding:0 0 20px 0;word-break:break-word\"align=left><div style=font-family:Arial,sans-serif;font-size:14px;line-height:1.5;text-align:left;color:#333><div style=background-color:#f8f9fa;padding:10px;border-radius:4px;margin-top:5px class=guest-list>{{#each attendees}}• {{key}} &lt;{{value}}><br>{{/each attendees}}</div></div></tr>{{/if attendees}}<tr><td style=\"font-size:0;padding:15px 30px;word-break:break-word\"align=center vertical-align=middle><table border=0 cellpadding=0 cellspacing=0 role=presentation style=border-collapse:separate;line-height:100%><tr><td style=\"border:none;border-radius:6px;cursor:auto;mso-padding-alt:10px 25px;background:#2c5aa0\"align=center bgcolor=#2c5aa0 role=presentation valign=middle><a href={{action_url}} style=\"display:inline-block;background:#2c5aa0;color:#fff;font-family:Arial,sans-serif;font-size:16px;font-weight:700;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0;border-radius:6px\"target=_blank>{{action_name}}</a></table></table></div><!--[if mso | IE]><![endif]--></table></div><!--[if mso | IE]><![endif]--><!--[if mso | IE]><table border=0 cellpadding=0 cellspacing=0 align=center style=width:600px width=600 bgcolor=#f8f9fa><tr><td style=line-height:0;font-size:0;mso-line-height-rule:exactly><![endif]--><div style=\"background-color:#f8f9fa;margin:0 auto;max-width:600px\"><table border=0 cellpadding=0 cellspacing=0 role=presentation style=background-color:#f8f9fa;width:100% align=center><tr><td style=direction:ltr;font-size:0;padding:20px;text-align:center><!--[if mso | IE]><table border=0 cellpadding=0 cellspacing=0 role=presentation><tr><td style=vertical-align:top;width:560px><![endif]--><div style=font-size:0;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100% class=\"mj-column-per-100 mj-outlook-group-fix\"><table border=0 cellpadding=0 cellspacing=0 role=presentation style=vertical-align:top width=100%><tr><td style=\"font-size:0;padding:0 0 15px 0;word-break:break-word\"align=center><p style=\"border-top:solid 1px #e9ecef;font-size:1px;margin:0 auto;width:100%\"></p><!--[if mso | IE]><table border=0 cellpadding=0 cellspacing=0 role=presentation style=\"border-top:solid 1px #e9ecef;font-size:1px;margin:0 auto;width:560px\"align=center width=560px><tr><td style=height:0;line-height:0> </table><![endif]--><tr><td style=\"font-size:0;padding:10px 25px;word-break:break-word\"align=center><div style=font-family:Arial,sans-serif;font-size:12px;line-height:1.4;text-align:center;color:#6c757d>{{footer}}</div></table></div><!--[if mso | IE]><![endif]--></table></div><!--[if mso | IE]><![endif]--></div>"
  },
  {
    "path": "resources/html-templates/calendar-alarm.mjml",
    "content": "<mjml>\n  <mj-head>\n    <mj-title>{{title}}</mj-title>\n    <mj-attributes>\n      <mj-all font-family=\"Arial, sans-serif\" />\n      <mj-text font-size=\"14px\" color=\"#333333\" line-height=\"1.5\" />\n      <mj-section background-color=\"#ffffff\" />\n    </mj-attributes>\n    <mj-style>\n      :root {\n        color-scheme: light only;\n      }\n      .event-detail {\n      font-weight: bold;\n      color: #2c5aa0;\n      }\n      .guest-list {\n      background-color: #f8f9fa;\n      padding: 10px;\n      border-radius: 4px;\n      margin-top: 5px;\n      }\n    </mj-style>\n  </mj-head>\n  <mj-body background-color=\"#f4f4f4\">\n    <mj-section background-color=\"#ffffff\" padding=\"20px 0\">\n      <mj-column>\n        <mj-image src=\"https://stalw.art/img/logo-dark@2x.png\" alt=\"Stalwart Logo\" width=\"200px\" align=\"center\" />\n      </mj-column>\n    </mj-section>\n\n    <mj-section background-color=\"#ffffff\" padding=\"0 20px\">\n      <mj-column>\n        <mj-text align=\"center\" font-size=\"20px\" font-weight=\"bold\" color=\"#2c5aa0\" padding=\"20px 0\">\n          {{upcoming_event}}\n        </mj-text>\n\n        <mj-text font-size=\"24px\" font-weight=\"bold\" color=\"#333333\" padding=\"10px 0 5px 0\">\n          {{event_title}}\n        </mj-text>\n\n        <mj-text font-size=\"14px\" color=\"#666666\" padding=\"0 0 20px 0\" line-height=\"1.4\">\n          {{event_description}}\n        </mj-text>\n\n        <mj-text padding=\"5px 0\">\n          <span class=\"event-detail\">{{field_name}}:</span> {{field_value}}\n        </mj-text>\n\n        <mj-text font-size=\"16px\" font-weight=\"bold\" color=\"#333333\" padding=\"15px 0 5px 0\">\n          {{attendees}}:\n        </mj-text>\n\n        <mj-text padding=\"0 0 20px 0\">\n          <div class=\"guest-list\">\n            • {{guest_name}} ({{guest_email}})<br />\n          </div>\n        </mj-text>\n\n        <mj-button background-color=\"#2c5aa0\" color=\"#ffffff\" font-size=\"16px\" font-weight=\"bold\" border-radius=\"6px\" padding=\"15px 30px\" href=\"{{event_url}}\" align=\"center\">\n          {{open_button}}\n        </mj-button>\n      </mj-column>\n    </mj-section>\n\n    <mj-section background-color=\"#f8f9fa\" padding=\"20px\">\n      <mj-column>\n        <mj-divider border-color=\"#e9ecef\" border-width=\"1px\" padding=\"0 0 15px 0\" />\n        <mj-text font-size=\"12px\" color=\"#6c757d\" align=\"center\" line-height=\"1.4\">\n         {{footer}}\n        </mj-text>\n      </mj-column>\n    </mj-section>\n  </mj-body>\n</mjml>"
  },
  {
    "path": "resources/html-templates/calendar-invite.html",
    "content": "<!doctype html>\n<html xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:v=\"urn:schemas-microsoft-com:vml\"\n    xmlns:o=\"urn:schemas-microsoft-com:office:office\">\n\n<head>\n    <title>{{page_title}}</title>\n    <!--[if !mso]><!-->\n    <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\">\n    <!--<![endif]-->\n    <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\">\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n    <style type=\"text/css\">\n        #outlook a {\n            padding: 0;\n        }\n\n        body {\n            margin: 0;\n            padding: 0;\n            -webkit-text-size-adjust: 100%;\n            -ms-text-size-adjust: 100%;\n        }\n\n        table,\n        td {\n            border-collapse: collapse;\n            mso-table-lspace: 0pt;\n            mso-table-rspace: 0pt;\n        }\n\n        img {\n            border: 0;\n            height: auto;\n            line-height: 100%;\n            outline: none;\n            text-decoration: none;\n            -ms-interpolation-mode: bicubic;\n        }\n\n        p {\n            display: block;\n            margin: 13px 0;\n        }\n\n        .color-info {\n            background: #4CAF50;\n            background-color: #4CAF50;\n        }\n\n        .color-warning {\n            background: #FF9800;\n            background-color: #FF9800;\n        }\n\n        .color-danger {\n            background: #f44336;\n            background-color: #f44336;\n        }\n\n        :root {\n            color-scheme: light only;\n        }\n    </style>\n    <!--[if mso]>\n        <noscript>\n        <xml>\n        <o:OfficeDocumentSettings>\n          <o:AllowPNG/>\n          <o:PixelsPerInch>96</o:PixelsPerInch>\n        </o:OfficeDocumentSettings>\n        </xml>\n        </noscript>\n        <![endif]-->\n    <!--[if lte mso 11]>\n        <style type=\"text/css\">\n          .mj-outlook-group-fix { width:100% !important; }\n        </style>\n        <![endif]-->\n    <style type=\"text/css\">\n        @media only screen and (min-width:480px) {\n            .mj-column-per-100 {\n                width: 100% !important;\n                max-width: 100%;\n            }\n\n            .mj-column-per-33-3 {\n                width: 33.3% !important;\n                max-width: 33.3%;\n            }\n        }\n    </style>\n    <style media=\"screen and (min-width:480px)\">\n        .moz-text-html .mj-column-per-100 {\n            width: 100% !important;\n            max-width: 100%;\n        }\n\n        .moz-text-html .mj-column-per-33-3 {\n            width: 33.3% !important;\n            max-width: 33.3%;\n        }\n    </style>\n    <style type=\"text/css\">\n        @media only screen and (max-width:480px) {\n            table.mj-full-width-mobile {\n                width: 100% !important;\n            }\n\n            td.mj-full-width-mobile {\n                width: auto !important;\n            }\n        }\n    </style>\n</head>\n\n<body style=\"word-spacing:normal;background-color:#f4f4f4;\">\n    <div style=\"background-color:#f4f4f4;\">\n        <!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px;\" width=\"600\" bgcolor=\"#4CAF50\" ><tr><td style=\"line-height:0px;font-size:0px;mso-line-height-rule:exactly;\"><![endif]-->\n        {{#if header}}\n        <div class=\"color-{{color}}\" style=\"margin:0px auto;max-width:600px;\">\n            <table class=\"color-{{color}}\" align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                style=\"width:100%;\">\n                <tbody>\n                    <tr>\n                        <td style=\"direction:ltr;font-size:0px;padding:5px;text-align:center;\">\n                            <!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:590px;\" ><![endif]-->\n                            <div class=\"mj-column-per-100 mj-outlook-group-fix\"\n                                style=\"font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;\">\n                                <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                                    style=\"vertical-align:top;\" width=\"100%\">\n                                    <tbody>\n                                        <tr>\n                                            <td align=\"center\"\n                                                style=\"font-size:0px;padding:10px 25px;word-break:break-word;\">\n                                                <div\n                                                    style=\"font-family:Arial, sans-serif;font-size:16px;font-weight:bold;line-height:1.4;text-align:center;color:white;\">\n                                                    {{header}}</div>\n                                            </td>\n                                        </tr>\n                                    </tbody>\n                                </table>\n                            </div>\n                            <!--[if mso | IE]></td></tr></table><![endif]-->\n                        </td>\n                    </tr>\n                </tbody>\n            </table>\n        </div>\n        {{/if header}}\n\n        <!--[if mso | IE]></td></tr></table><![endif]-->\n        <!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px;\" width=\"600\" bgcolor=\"white\" ><tr><td style=\"line-height:0px;font-size:0px;mso-line-height-rule:exactly;\"><![endif]-->\n        <div style=\"background:white;background-color:white;margin:0px auto;max-width:600px;\">\n            <table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                style=\"background:white;background-color:white;width:100%;\">\n                <tbody>\n                    <tr>\n                        <td style=\"direction:ltr;font-size:0px;padding:5px 5px 5px 5px;text-align:center;\">\n                            <!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:590px;\" ><![endif]-->\n                            <div class=\"mj-column-per-100 mj-outlook-group-fix\"\n                                style=\"font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;\">\n                                <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                                    style=\"vertical-align:top;\" width=\"100%\">\n                                    <tbody>\n                                        <tr>\n                                            <td align=\"center\"\n                                                style=\"font-size:0px;padding:10px 25px;padding-bottom:20px;word-break:break-word;\">\n                                                <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                                                    style=\"border-collapse:collapse;border-spacing:0px;\">\n                                                    <tbody>\n                                                        <tr>\n                                                            <td style=\"width:200px;\">\n                                                                <img height=\"auto\" src=\"{{logo_cid}}\"\n                                                                    style=\"border:0;display:block;outline:none;text-decoration:none;height:auto;width:100%;font-size:13px;\"\n                                                                    width=\"200\">\n                                                            </td>\n                                                        </tr>\n                                                    </tbody>\n                                                </table>\n                                            </td>\n                                        </tr>\n                                        {{#each event_details}}\n                                        <tr>\n                                            <td align=\"left\" class=\"event-detail\"\n                                                style=\"margin-bottom: 16px; font-size: 0px; padding: 10px 25px; word-break: break-word;\">\n                                                <div\n                                                    style=\"font-family:Arial, sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#333333;\">\n                                                    <div class=\"event-detail-label\"\n                                                        style=\"font-weight: bold; color: #666666; margin-bottom: 4px;\">\n                                                        {{key}}\n                                                        {{#if changed}}<span class=\"changed-pill\"\n                                                            style=\"background-color: #4CAF50; color: white; padding: 4px 8px; border-radius: 12px; font-size: 8px; font-weight: bold; margin-left: 8px; display: inline-block;\">{{changed}}</span>\n                                                        {{/if changed}}\n                                                    </div>\n                                                    {{#if old_value}}<div class=\"strikethrough\"\n                                                        style=\"text-decoration: line-through; color: #999999;\">\n                                                        {{old_value}}</div>{{/if old_value}}\n                                                    <div style=\"display: flex; align-items: center;\">\n                                                        <span>{{value}}</span>\n                                                    </div>\n                                                </div>\n                                            </td>\n                                        </tr>\n                                        {{/each event_details}}\n                                        {{#if attendees}}\n                                        <tr>\n                                            <td align=\"left\" class=\"event-detail\"\n                                                style=\"margin-bottom: 16px; font-size: 0px; padding: 10px 25px; padding-bottom: 30px; word-break: break-word;\">\n                                                <div\n                                                    style=\"font-family:Arial, sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#333333;\">\n                                                    <div class=\"event-detail-label\"\n                                                        style=\"font-weight: bold; color: #666666; margin-bottom: 4px;\">\n                                                        {{attendees_title}}</div>\n                                                    {{#each attendees}}<div class=\"guest-item\"\n                                                        style=\"margin-bottom: 4px;\">• {{key}}\n                                                        ({{value}})</div>{{/each attendees}}\n\n                                                </div>\n                                            </td>\n                                        </tr>\n                                        {{/if attendees}}\n                                        {{#if rsvp}}\n                                        <tr>\n                                            <td align=\"center\"\n                                                style=\"font-size:0px;padding:10px 0;word-break:break-word;\">\n                                                <p\n                                                    style=\"border-top:solid 1px #e0e0e0;font-size:1px;margin:0px auto;width:100%;\">\n                                                </p>\n                                                <!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" style=\"border-top:solid 1px #e0e0e0;font-size:1px;margin:0px auto;width:590px;\" role=\"presentation\" width=\"590px\" ><tr><td style=\"height:0;line-height:0;\"> &nbsp;\n</td></tr></table><![endif]-->\n                                            </td>\n                                        </tr>\n                                        <tr>\n                                            <td align=\"left\"\n                                                style=\"font-size:0px;padding:10px 0px 5px 20px;word-break:break-word;\">\n                                                <div\n                                                    style=\"font-family:Arial, sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#333333;\">\n                                                    {{rsvp}}\n                                                </div>\n                                            </td>\n                                        </tr>\n                                        {{/if rsvp}}\n                                    </tbody>\n                                </table>\n                            </div>\n                            <!--[if mso | IE]></td></tr></table><![endif]-->\n                        </td>\n                    </tr>\n                </tbody>\n            </table>\n        </div>\n        <!--[if mso | IE]></td></tr></table><![endif]-->\n        {{#if rsvp}}\n        <!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px;\" width=\"600\" bgcolor=\"white\" ><tr><td style=\"line-height:0px;font-size:0px;mso-line-height-rule:exactly;\"><![endif]-->\n        <div style=\"background:white;background-color:white;margin:0px auto;max-width:600px;\">\n            <table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                style=\"background:white;background-color:white;width:100%;\">\n                <tbody>\n                    <tr>\n                        <td style=\"direction:ltr;font-size:0px;padding:0 5px 5px 5px;text-align:center;\">\n                            <!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"width:590px;\" ><![endif]-->\n                            <div class=\"mj-column-per-100 mj-outlook-group-fix\"\n                                style=\"font-size:0;line-height:0;text-align:left;display:inline-block;width:100%;direction:ltr;\">\n                                <!--[if mso | IE]><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" ><tr><![endif]-->\n\n                                {{#each actions}}\n                                <!--[if mso | IE]><td style=\"vertical-align:top;width:196px;\" ><![endif]-->\n                                <div class=\"mj-column-per-33-3 mj-outlook-group-fix\"\n                                    style=\"font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:33.3%;\">\n                                    <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                                        style=\"vertical-align:top;\" width=\"100%\">\n                                        <tbody>\n                                            <tr>\n                                                <td align=\"left\" vertical-align=\"middle\"\n                                                    style=\"font-size:0px;padding:10px 25px;word-break:break-word;\">\n                                                    <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\"\n                                                        role=\"presentation\"\n                                                        style=\"border-collapse:separate;width:100%;line-height:100%;\">\n                                                        <tr>\n                                                            <td align=\"center\" bgcolor=\"#4CAF50\" role=\"presentation\"\n                                                                class=\"color-{{color}}\"\n                                                                style=\"border:none;border-radius:4px;cursor:auto;mso-padding-alt:8px 4px;\"\n                                                                valign=\"middle\">\n                                                                <a href=\"{{!action_url}}\" class=\"color-{{color}}\"\n                                                                    style=\"display:inline-block;color:white;font-family:Arial, sans-serif;font-size:11px;font-weight:bold;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:8px 4px;mso-padding-alt:0px;border-radius:4px;\"\n                                                                    target=\"_blank\"> {{action_name}} </a>\n                                                            </td>\n                                                        </tr>\n                                                    </table>\n                                                </td>\n                                            </tr>\n                                        </tbody>\n                                    </table>\n                                </div>\n                                <!--[if mso | IE]></td><![endif]-->\n                                {{/each actions}}\n                                <!--[if mso | IE]></tr></table><![endif]-->\n                            </div>\n                            <!--[if mso | IE]></td></tr></table><![endif]-->\n                        </td>\n                    </tr>\n                </tbody>\n            </table>\n        </div>\n        <!--[if mso | IE]></td></tr></table><![endif]-->\n        {{/if rsvp}}\n        {{#if footer}}\n        <!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px;\" width=\"600\" bgcolor=\"#f8f8f8\" ><tr><td style=\"line-height:0px;font-size:0px;mso-line-height-rule:exactly;\"><![endif]-->\n        <div style=\"background:#f8f8f8;background-color:#f8f8f8;margin:0px auto;max-width:600px;\">\n            <table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                style=\"background:#f8f8f8;background-color:#f8f8f8;width:100%;\">\n                <tbody>\n                    <tr>\n                        <td style=\"direction:ltr;font-size:0px;padding:20px;text-align:center;\">\n                            <!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:560px;\" ><![endif]-->\n                            <div class=\"mj-column-per-100 mj-outlook-group-fix\"\n                                style=\"font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;\">\n                                <table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"\n                                    style=\"vertical-align:top;\" width=\"100%\">\n                                    <tbody>\n                                        {{#each footer}}\n                                        <tr>\n                                            <td align=\"center\"\n                                                style=\"font-size:0px;padding:10px 25px;word-break:break-word;\">\n                                                <div\n                                                    style=\"font-family:Arial, sans-serif;font-size:12px;line-height:1.5;text-align:center;color:#666666;\">\n                                                    {{key}}</div>\n                                            </td>\n                                        </tr>\n                                        {{/each footer}}\n                                    </tbody>\n                                </table>\n                            </div>\n                            <!--[if mso | IE]></td></tr></table><![endif]-->\n                        </td>\n                    </tr>\n                </tbody>\n            </table>\n        </div>\n        <!--[if mso | IE]></td></tr></table><![endif]-->\n        {{/if footer}}\n    </div>\n</body>\n\n</html>"
  },
  {
    "path": "resources/html-templates/calendar-invite.html.min",
    "content": "<!doctype html><html xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:v=\"urn:schemas-microsoft-com:vml\" xmlns:o=\"urn:schemas-microsoft-com:office:office\"><head><title>{{page_title}}</title><!--[if !mso]><!--><meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\"><!--<![endif]--><meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"><style type=\"text/css\">#outlook a{padding:0}body{margin:0;padding:0;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}table,td{border-collapse:collapse;mso-table-lspace:0;mso-table-rspace:0}img{border:0;height:auto;line-height:100%;outline:0;text-decoration:none;-ms-interpolation-mode:bicubic}p{display:block;margin:13px 0}.color-info{background:#4caf50;background-color:#4caf50}.color-warning{background:#ff9800;background-color:#ff9800}.color-danger{background:#f44336;background-color:#f44336}:root{color-scheme:light only}</style><!--[if mso]><noscript><xml><o:officedocumentsettings><o:allowpng><o:pixelsperinch>96</o:pixelsperinch></o:officedocumentsettings></xml></noscript><![endif]--><!--[if lte mso 11]><style type=\"text/css\">.mj-outlook-group-fix{width:100%!important}</style><![endif]--><style type=\"text/css\">@media only screen and (min-width:480px){.mj-column-per-100{width:100%!important;max-width:100%}.mj-column-per-33-3{width:33.3%!important;max-width:33.3%}}</style><style media=\"screen and (min-width:480px)\">.moz-text-html .mj-column-per-100{width:100%!important;max-width:100%}.moz-text-html .mj-column-per-33-3{width:33.3%!important;max-width:33.3%}</style><style type=\"text/css\">@media only screen and (max-width:480px){table.mj-full-width-mobile{width:100%!important}td.mj-full-width-mobile{width:auto!important}}</style></head><body style=\"word-spacing:normal;background-color:#f4f4f4\"><div style=\"background-color:#f4f4f4\"><!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px\" width=\"600\" bgcolor=\"#4CAF50\"><tr><td style=\"line-height:0;font-size:0;mso-line-height-rule:exactly\"><![endif]--> {{#if header}}<div class=\"color-{{color}}\" style=\"margin:0 auto;max-width:600px\"><table class=\"color-{{color}}\" align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"width:100%\"><tbody><tr><td style=\"direction:ltr;font-size:0;padding:5px;text-align:center\"><!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:590px\"><![endif]--><div class=\"mj-column-per-100 mj-outlook-group-fix\" style=\"font-size:0;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%\"><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"vertical-align:top\" width=\"100%\"><tbody><tr><td align=\"center\" style=\"font-size:0;padding:10px 25px;word-break:break-word\"><div style=\"font-family:Arial,sans-serif;font-size:16px;font-weight:700;line-height:1.4;text-align:center;color:#fff\">{{header}}</div></td></tr></tbody></table></div><!--[if mso | IE]><![endif]--></td></tr></tbody></table></div>{{/if header}}<!--[if mso | IE]><![endif]--><!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px\" width=\"600\" bgcolor=\"white\"><tr><td style=\"line-height:0;font-size:0;mso-line-height-rule:exactly\"><![endif]--><div style=\"background:#fff;background-color:#fff;margin:0 auto;max-width:600px\"><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"background:#fff;background-color:#fff;width:100%\"><tbody><tr><td style=\"direction:ltr;font-size:0;padding:5px 5px 5px 5px;text-align:center\"><!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:590px\"><![endif]--><div class=\"mj-column-per-100 mj-outlook-group-fix\" style=\"font-size:0;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%\"><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"vertical-align:top\" width=\"100%\"><tbody><tr><td align=\"center\" style=\"font-size:0;padding:10px 25px;padding-bottom:20px;word-break:break-word\"><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"border-collapse:collapse;border-spacing:0\"><tbody><tr><td style=\"width:200px\"><img height=\"auto\" src=\"{{logo_cid}}\" style=\"border:0;display:block;outline:0;text-decoration:none;height:auto;width:100%;font-size:13px\" width=\"200\"></td></tr></tbody></table></td></tr>{{#each event_details}}<tr><td align=\"left\" class=\"event-detail\" style=\"margin-bottom:16px;font-size:0;padding:10px 25px;word-break:break-word\"><div style=\"font-family:Arial,sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#333\"><div class=\"event-detail-label\" style=\"font-weight:700;color:#666;margin-bottom:4px\">{{key}} {{#if changed}}<span class=\"changed-pill\" style=\"background-color:#4caf50;color:#fff;padding:4px 8px;border-radius:12px;font-size:8px;font-weight:700;margin-left:8px;display:inline-block\">{{changed}}</span>{{/if changed}}</div>{{#if old_value}}<div class=\"strikethrough\" style=\"text-decoration:line-through;color:#999\">{{old_value}}</div>{{/if old_value}}<div style=\"display:flex;align-items:center\"><span>{{value}}</span></div></div></td></tr>{{/each event_details}} {{#if attendees}}<tr><td align=\"left\" class=\"event-detail\" style=\"margin-bottom:16px;font-size:0;padding:10px 25px;padding-bottom:30px;word-break:break-word\"><div style=\"font-family:Arial,sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#333\"><div class=\"event-detail-label\" style=\"font-weight:700;color:#666;margin-bottom:4px\">{{attendees_title}}</div>{{#each attendees}}<div class=\"guest-item\" style=\"margin-bottom:4px\">• {{key}} ({{value}})</div>{{/each attendees}}</div></td></tr>{{/if attendees}} {{#if rsvp}}<tr><td align=\"center\" style=\"font-size:0;padding:10px 0;word-break:break-word\"><p style=\"border-top:solid 1px #e0e0e0;font-size:1px;margin:0 auto;width:100%\"></p><!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" style=\"border-top:solid 1px #e0e0e0;font-size:1px;margin:0 auto;width:590px\" role=\"presentation\" width=\"590px\"><tr><td style=\"height:0;line-height:0\">&nbsp;</td></tr></table><![endif]--></td></tr><tr><td align=\"left\" style=\"font-size:0;padding:10px 0 5px 20px;word-break:break-word\"><div style=\"font-family:Arial,sans-serif;font-size:14px;line-height:1.4;text-align:left;color:#333\">{{rsvp}}</div></td></tr>{{/if rsvp}}</tbody></table></div><!--[if mso | IE]><![endif]--></td></tr></tbody></table></div><!--[if mso | IE]><![endif]--> {{#if rsvp}}<!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px\" width=\"600\" bgcolor=\"white\"><tr><td style=\"line-height:0;font-size:0;mso-line-height-rule:exactly\"><![endif]--><div style=\"background:#fff;background-color:#fff;margin:0 auto;max-width:600px\"><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"background:#fff;background-color:#fff;width:100%\"><tbody><tr><td style=\"direction:ltr;font-size:0;padding:0 5px 5px 5px;text-align:center\"><!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"width:590px\"><![endif]--><div class=\"mj-column-per-100 mj-outlook-group-fix\" style=\"font-size:0;line-height:0;text-align:left;display:inline-block;width:100%;direction:ltr\"><!--[if mso | IE]><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\"><tr><![endif]--> {{#each actions}}<!--[if mso | IE]><td style=\"vertical-align:top;width:196px\"><![endif]--><div class=\"mj-column-per-33-3 mj-outlook-group-fix\" style=\"font-size:0;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:33.3%\"><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"vertical-align:top\" width=\"100%\"><tbody><tr><td align=\"left\" vertical-align=\"middle\" style=\"font-size:0;padding:10px 25px;word-break:break-word\"><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"border-collapse:separate;width:100%;line-height:100%\"><tr><td align=\"center\" bgcolor=\"#4CAF50\" role=\"presentation\" class=\"color-{{color}}\" style=\"border:none;border-radius:4px;cursor:auto;mso-padding-alt:8px 4px\" valign=\"middle\"><a href=\"{{!action_url}}\" class=\"color-{{color}}\" style=\"display:inline-block;color:#fff;font-family:Arial,sans-serif;font-size:11px;font-weight:700;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:8px 4px;mso-padding-alt:0;border-radius:4px\" target=\"_blank\">{{action_name}}</a></td></tr></table></td></tr></tbody></table></div><!--[if mso | IE]><![endif]--> {{/each actions}}<!--[if mso | IE]><![endif]--></div><!--[if mso | IE]><![endif]--></td></tr></tbody></table></div><!--[if mso | IE]><![endif]--> {{/if rsvp}} {{#if footer}}<!--[if mso | IE]><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"\" style=\"width:600px\" width=\"600\" bgcolor=\"#f8f8f8\"><tr><td style=\"line-height:0;font-size:0;mso-line-height-rule:exactly\"><![endif]--><div style=\"background:#f8f8f8;background-color:#f8f8f8;margin:0 auto;max-width:600px\"><table align=\"center\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"background:#f8f8f8;background-color:#f8f8f8;width:100%\"><tbody><tr><td style=\"direction:ltr;font-size:0;padding:20px;text-align:center\"><!--[if mso | IE]><table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tr><td class=\"\" style=\"vertical-align:top;width:560px\"><![endif]--><div class=\"mj-column-per-100 mj-outlook-group-fix\" style=\"font-size:0;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%\"><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" role=\"presentation\" style=\"vertical-align:top\" width=\"100%\"><tbody>{{#each footer}}<tr><td align=\"center\" style=\"font-size:0;padding:10px 25px;word-break:break-word\"><div style=\"font-family:Arial,sans-serif;font-size:12px;line-height:1.5;text-align:center;color:#666\">{{key}}</div></td></tr>{{/each footer}}</tbody></table></div><!--[if mso | IE]><![endif]--></td></tr></tbody></table></div><!--[if mso | IE]><![endif]--> {{/if footer}}</div></body></html>"
  },
  {
    "path": "resources/html-templates/calendar-invite.mjml",
    "content": "<mjml>\n  <mj-head>\n    <mj-title>Event Changed</mj-title>\n    <mj-preview>An event you're attending has been updated</mj-preview>\n    <mj-attributes>\n      <mj-all font-family=\"Arial, sans-serif\" />\n      <mj-text font-size=\"14px\" color=\"#333333\" line-height=\"1.4\" />\n      <mj-button background-color=\"#4CAF50\" color=\"white\" font-size=\"11px\" font-weight=\"bold\" border-radius=\"4px\" align=\"left\"/>\n    </mj-attributes>\n    <mj-style inline=\"inline\">\n      .changed-pill {\n      background-color: #4CAF50;\n      color: white;\n      padding: 4px 8px;\n      border-radius: 12px;\n      font-size: 8px;\n      font-weight: bold;\n      margin-left: 8px;\n      display: inline-block;\n      }\n      .event-detail {\n      margin-bottom: 16px;\n      }\n      .event-detail-label {\n      font-weight: bold;\n      color: #666666;\n      margin-bottom: 4px;\n      }\n      .guest-item {\n      margin-bottom: 4px;\n      }\n      .strikethrough {\n      text-decoration: line-through;\n      color: #999999;\n      }\n    </mj-style>\n  </mj-head>\n  <mj-body background-color=\"#f4f4f4\">\n    <!-- Header Alert -->\n    <mj-section background-color=\"#4CAF50\" padding=\"5px\">\n      <mj-column>\n        <mj-text align=\"center\" color=\"white\" font-size=\"16px\" font-weight=\"bold\">\n          This event has been updated\n        </mj-text>\n      </mj-column>\n    </mj-section>\n\n    <!-- Main Content -->\n    <mj-section background-color=\"white\" padding=\"5px 5px 5px 5px\">\n      <mj-column>\n        <!-- Logo -->\n        <mj-image src=\"https://stalw.art/img/logo-dark@2x.png\" alt=\"Stalwart Logo\" width=\"200px\" align=\"center\" padding-bottom=\"20px\" />\n\n        <!-- Title -->\n        <mj-text css-class=\"event-detail\">\n          <div class=\"event-detail-label\">Title</div>\n          <div style=\"display: flex; align-items: center;\">\n            <span>Annual Team Building Workshop 2025</span>\n\n          </div>\n        </mj-text>\n\n        <!-- Description -->\n        <mj-text css-class=\"event-detail\">\n          <div class=\"event-detail-label\">Description</div>\n          <div>Join us for our annual team building workshop featuring interactive activities, networking sessions, and strategic planning for the upcoming year. This year's theme focuses on collaboration and innovation.</div>\n        </mj-text>\n\n        <!-- When -->\n        <mj-text css-class=\"event-detail\">\n          <div class=\"event-detail-label\">When <span class=\"changed-pill\">CHANGED</span></div>\n          <div class=\"strikethrough\">Friday, March 15, 2025 9:00 AM - 5:00 PM (PST)</div>\n          <div>Friday, March 15, 2025 9:00 AM - 5:00 PM (PST)</div>\n        </mj-text>\n\n        <!-- Location -->\n        <mj-text css-class=\"event-detail\">\n          <div class=\"event-detail-label\">Location</div>\n          <div>Grand Conference Center<br>123 Business Park Drive<br>San Francisco, CA 94105</div>\n        </mj-text>\n\n        <!-- Guest List -->\n        <mj-text css-class=\"event-detail\" padding-bottom=\"30px\">\n          <div class=\"event-detail-label\">Guest List</div>\n          <div class=\"guest-item\">• John Smith (john.smith@company.com)</div>\n          <div style=\"margin-top: 8px; color: #666666; font-style: italic;\">+ 12 more attendees</div>\n        </mj-text>\n\n        <!-- RSVP legend -->\n        <mj-divider border-color=\"#e0e0e0\" border-width=\"1px\" padding=\"10px 0\" />\n\n        <mj-text font-size=\"14px\" color=\"#333333\" padding=\"10px 0px 5px 20px\">\n          <b>Reply</b> as email@domain.com for this event series:\n        </mj-text>\n\n      </mj-column>\n    </mj-section>\n\n    <!-- RSVP buttons -->\n    <mj-section background-color=\"white\" padding=\"0 5px 5px 5px\">\n      <mj-group>\n        <mj-column width=\"33.3%\">\n          <mj-button background-color=\"#4CAF50\" href=\"#\" width=\"100%\" inner-padding=\"8px 4px\">\n            YES\n          </mj-button>\n        </mj-column>\n        <mj-column width=\"33.3%\">\n          <mj-button background-color=\"#f44336\" href=\"#\" width=\"100%\" inner-padding=\"8px 4px\">\n            NO\n          </mj-button>\n        </mj-column>\n        <mj-column width=\"33.3%\">\n          <mj-button background-color=\"#FF9800\" href=\"#\" width=\"100%\" inner-padding=\"8px 4px\">\n            MAYBE\n          </mj-button>\n        </mj-column>\n      </mj-group>\n    </mj-section>\n\n    <!-- Footer -->\n    <mj-section background-color=\"#f8f8f8\" padding=\"20px\">\n      <mj-column>\n        <mj-text font-size=\"12px\" color=\"#666666\" align=\"center\" line-height=\"1.5\">\n          You’re receiving this e-mail as you're listed as a participant for this event.\n        </mj-text>\n        <mj-text font-size=\"12px\" color=\"#666666\" align=\"center\" line-height=\"1.5\" padding-top=\"10px\">\n          Forwarding this e-mail could allow any recipient to reply to the organizer, join the guest list, extend the invitation to others, or alter your RSVP.\n        </mj-text>\n      </mj-column>\n    </mj-section>\n  </mj-body>\n</mjml>"
  },
  {
    "path": "resources/locales/i18n.yml",
    "content": "# SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n# SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n\ncalendar.alarm_subject_prefix:\n  en: Notification\n  es: Notificación\n  fr: Notification\n  de: Benachrichtigung\n  it: Notifica\n  pt: Notificação\n  nl: Melding\n  da: Notifikation\n  ca: Notificació\n  el: Ειδοποίηση\n  sv: Notifikation\n  pl: Powiadomienie\n\ncalendar.alarm_header:\n  en: You have an upcoming event\n  es: Tienes un evento próximo\n  fr: Vous avez un événement à venir\n  de: Sie haben einen bevorstehenden Termin\n  it: Hai un evento in programma\n  pt: Você tem um evento próximo\n  nl: U heeft een aankomende gebeurtenis\n  da: Du har en kommende begivenhed\n  ca: Tens un esdeveniment d'aquí poc\n  el: Έχετε μία επερχόμενη εκδήλωση\n  sv: Du har en kommande händelse\n  pl: Masz nadchodzące wydarzenie\n\ncalendar.alarm_footer:\n  en: You are receiving this email because you have enabled calendar notifications. To stop receiving these emails, login to the self-service portal and disable event notifications.\n  es: Recibe este correo porque ha habilitado las notificaciones de calendario. Para dejar de recibir estos correos, inicie sesión en el portal de autoservicio y desactive las notificaciones de eventos.\n  fr: Vous recevez cet e-mail car vous avez activé les notifications de calendrier. Pour arrêter de recevoir ces e-mails, connectez-vous au portail libre-service et désactivez les notifications d'événements.\n  de: Sie erhalten diese E-Mail, weil Sie Kalender-Benachrichtigungen aktiviert haben. Um diese E-Mails nicht mehr zu erhalten, melden Sie sich im Self-Service-Portal an und deaktivieren Sie Ereignisbenachrichtigungen.\n  it: Ricevi questa email perché hai abilitato le notifiche del calendario. Per smettere di ricevere queste email, accedi al portale self-service e disabilita le notifiche degli eventi.\n  pt: Você está recebendo este e-mail porque habilitou as notificações do calendário. Para parar de receber estes e-mails, faça login no portal de autoatendimento e desative as notificações de eventos.\n  nl: U ontvangt deze e-mail omdat u kalendernotificaties heeft ingeschakeld. Om deze e-mails niet meer te ontvangen, logt u in op de selfservice-portal en schakelt u gebeurtenismeldingen uit.\n  da: Du modtager denne e-mail, fordi du har aktiveret kalendernotifikationer. Du kan deaktivere notifikationer i selvbetjeningsportalen.\n  ca: Estàs revent aquest correu perquè tens habilitada les notificacions del calendari. Per no rebre més aquests correus, identificat al portal d'autoservei i desactiva les notificacions d'esdeveniments.\n  el: Λαμβάνεται αυτό το e-mail γιατί έχετε ένεργοποιήσει τις ειδοποιήσεις ημερολογίου. Για διακοπή της λήψης τους, μπείτε στη πύλη αυτοεξυπηρέτησης και απενεργοποιείστε τις ειδοποιήσεις εκδηλώσεων.\n  sv: Du får det här mejlet för att du har slagit på kalendernotifikationer. För att sluta få dessa mejl, stäng av inställningen i självbetjäningsportalen.\n  pl: Otrzymujesz tę wiadomość, ponieważ włączyłeś powiadomienia kalendarza. Aby przestać je otrzymywać, zaloguj się do portalu samoobsługowego i wyłącz powiadomienia o wydarzeniach.\n\ncalendar.alarm_open:\n  en: View Event\n  es: Ver Evento\n  fr: Voir l'Événement\n  de: Termin Anzeigen\n  it: Visualizza Evento\n  pt: Ver Evento\n  nl: Gebeurtenis Bekijken\n  da: Vis begivenhed\n  ca: Veure esdeveniment\n  el: Προβολή Εκδήλωσης\n  sv: Visa händelse\n  pl: Wyświetl wydarzenie\n\ncalendar.organizer:\n  en: Organizer\n  es: Organizador\n  fr: Organisateur\n  de: Organisator\n  it: Organizzatore\n  pt: Organizador\n  nl: Organisator\n  da: Arrangør\n  ca: Organitzador\n  el: Διοργανωτής\n  sv: Arrangör\n  pl: Organizator\n\ncalendar.attendees:\n  en: Guests\n  es: Invitados\n  fr: Invités\n  de: Gäste\n  it: Ospiti\n  pt: Convidados\n  nl: Gasten\n  da: Gæster\n  ca: Invitats\n  el: Συμμετέχωντες\n  sv: Gäster\n  pl: Goście\n\ncalendar.start:\n  en: Start\n  es: Inicio\n  fr: Début\n  de: Beginn\n  it: Inizio\n  pt: Início\n  nl: Begin\n  da: Start\n  ca: Inici\n  el: Έναρξη\n  sv: Start\n  pl: Start\n\ncalendar.end:\n  en: End\n  es: Fin\n  fr: Fin\n  de: Ende\n  it: Fine\n  pt: Fim\n  nl: Einde\n  da: Slut\n  ca: Fi\n  el: Λήξη\n  sv: Slut\n  pl: Koniec\n\ncalendar.location:\n  en: Location\n  es: Ubicación\n  fr: Lieu\n  de: Ort\n  it: Luogo\n  pt: Local\n  nl: Locatie\n  da: Lokation\n  ca: Lloc\n  el: Τοποθεσία\n  sv: Plats\n  pl: Lokalizacja\n\ncalendar.date_template:\n  # English: \"Sun May 25, 2025 9:30am\"\n  en: \"%a %b %-d, %Y %-I:%M%P\"\n  # Spanish: \"dom 25 may 2025 9:30h\" (day month year hour and minute)\n  es: \"%a %-d %b %Y %-H:%Mh\"\n  # French: \"dim 25 mai 2025 9:30h\" (day month year hour and minute)\n  fr: \"%a %-d %b %Y %-H:%Mh\"\n  # German: \"So 25. Mai 2025 9:30 Uhr\" (day date month year hour and minute)\n  de: \"%a %-d. %b %Y %-H:%M Uhr\"\n  # Italian: \"dom 25 mag 2025 ore 9:30\" (day date month year hour and minute)\n  it: \"%a %-d %b %Y ore %-H:%M\"\n  # Portuguese: \"dom 25 mai 2025 9:30h\" (day date month year hour and minute)\n  pt: \"%a %-d %b %Y %-H:%Mh\"\n  # Dutch: \"zo 25 mei 2025 9:30u\" (weekday date month year hour and minute)\n  nl: \"%a %-d %b %Y %-H:%Mu\"\n  # Danish: \"søn 25. maj 2025 kl. 9:30\" (weekday date month year hour and minute)\n  da: \"%a %-d. %b %Y kl. %-H:%M\"\n  # Catalan: \"diu 25 mai 2025 9:30h\" (day month year hour and minute)\n  ca: \"%a %-d %b %Y %-H:%Mh\"\n  # Greek: \"Κυρ 25 Μαϊ 2025 9:30 η ώρα\" (day month year hour and minute)\n  el: \"%a %-d %b %Y %-H:%M η ώρα\"\n  # Svenska: \"sön 25 maj 2025 kl. 9:30\" (weekday date month year hour and minute)\n  sv: \"%a %-d %b %Y kl. %-H:%M\"\n  # Polish: \"nie 25 maj 2025 09:00\" (weekday day month year hour and minute)\n  pl: \"%a %d %b %Y %H:%M\"\n\ncalendar.date_template_long:\n  # English: \"Sunday May 25, 2025 9:30am\"\n  en: \"%A %B %-d, %Y %-I:%M%P\"\n  # Spanish: \"domingo 25 mayo 2025 9:30h\" (day month year hour and minute)\n  es: \"%A %-d %B %Y %-H:%Mh\"\n  # French: \"dimanche 25 mai 2025 9:30h\" (day month year hour and minute)\n  fr: \"%A %-d %B %Y %-H:%Mh\"\n  # German: \"Sonntag 25. Mai 2025 9:30 Uhr\" (day date month year hour and minute)\n  de: \"%A %-d. %B %Y %-H:%M Uhr\"\n  # Italian: \"domenica 25 maggio 2025 ore 9:30\" (day date month year hour and minute)\n  it: \"%A %-d %B %Y ore %-H:%M\"\n  # Portuguese: \"domingo 25 maio 2025 9:30h\" (day date month year hour and minute)\n  pt: \"%A %-d %B %Y %-H:%Mh\"\n  # Dutch: \"zondag 25 mei 2025 9:30u\" (weekday date month year hour and minute)\n  nl: \"%A %-d %B %Y %-H:%Mu\"\n  # Danish: \"søndag 25. maj 2025 kl. 9:30\" (weekday date month year hour and minute)\n  da: \"%A %-d. %B %Y kl. %-H:%M\"\n  # Catalan: \"diumenge 25 maig 2025 9:30h\" (day month year hour and minute)\n  ca: \"%A %-d %B %Y %-H:%Mh\"\n  # Greek: \"Κυριακή 25 Μαΐου 2025 9:30 η ώρα\" (day month year hour and minute)\n  el: \"%A %-d %B %Y %-H:%M η ώρα\"\n\n  # Svenska: \"söndag 25 maj 2025 kl. 9:30\" (weekday date month year hour and minute)\n  sv: \"%A %-d. %B %Y kl. %-H:%M\"\n\n  # Polish: \"niedziela 25 maj 2025 09:00\" (weekday day month year hour and minute)\n  pl: \"%A %d %b %Y %H:%M\"\n\ncalendar.invitation:\n  en: Invitation\n  es: Invitación\n  fr: Invitation\n  de: Einladung\n  it: Invito\n  pt: Convite\n  nl: Uitnodiging\n  da: Invitation\n  ca: Invitació\n  el: Πρόσκληση\n  sv: Inbjudan\n  pl: Zaproszenie\n\ncalendar.updated_invitation:\n  en: Updated invitation\n  es: Invitación actualizada\n  fr: Invitation mise à jour\n  de: Aktualisierte Einladung\n  it: Invito aggiornato\n  pt: Convite atualizado\n  nl: Bijgewerkte uitnodiging\n  da: Opdateret invitation\n  ca: Invitació actualitzada\n  el: Ενημερωμένη πρόσκληση\n  sv: Uppdaterad inbjudan\n  pl: Zaktualizowane zaproszenie\n\ncalendar.event_updated:\n  en: This event has been updated\n  es: Este evento ha sido actualizado\n  fr: Cet événement a été mis à jour\n  de: Dieses Ereignis wurde aktualisiert\n  it: Questo evento è stato aggiornato\n  pt: Este evento foi atualizado\n  nl: Dit evenement is bijgewerkt\n  da: Denne begivenhed er blevet opdateret\n  ca: Aquest esdeveniment s'ha actualitzat\n  el: Αυτή η εκδήλωση ενημερώθηκε\n  sv: Den här händelsen har blivit uppdaterad\n  pl: To wydarzenie zostało zaktualizowane\n\ncalendar.cancelled:\n  en: Cancelled\n  es: Cancelado\n  fr: Annulé\n  de: Abgesagt\n  it: Annullato\n  pt: Cancelado\n  nl: Geannuleerd\n  da: Aflyst\n  ca: Cancel·lat\n  el: Ακυρώθηκε\n  sv: Inställd\n  pl: Anulowane\n\ncalendar.event_cancelled:\n  en: This event has been canceled\n  es: Este evento ha sido cancelado\n  fr: Cet événement a été annulé\n  de: Dieses Ereignis wurde abgesagt\n  it: Questo evento è stato annullato\n  pt: Este evento foi cancelado\n  nl: Dit evenement is geannuleerd\n  da: Denne begivenhed er blevet aflyst\n  ca: Aquest esdeveniment s'ha cancel·lat\n  el: Αυτή η εκδήλωση ακυρώθηκε\n  sv: Den här händelsen har blivit inställd\n  pl: To wydarzenie zostało anulowane\n\ncalendar.accepted:\n  en: Accepted\n  es: Aceptado\n  fr: Accepté\n  de: Angenommen\n  it: Accettato\n  pt: Aceito\n  nl: Geaccepteerd\n  da: Accepteret\n  ca: Acceptat\n  el: Αποδοχή\n  sv: Accepterat\n  pl: Zaakceptowano\n\ncalendar.participant_accepted:\n  en: Participant $name accepted the invitation\n  es: El participante $name aceptó la invitación\n  fr: Le participant $name a accepté l'invitation\n  de: Teilnehmer $name hat die Einladung angenommen\n  it: Il partecipante $name ha accettato l'invito\n  pt: O participante $name aceitou o convite\n  nl: Deelnemer $name heeft de uitnodiging geaccepteerd\n  da: Deltager $name accepterede invitationen\n  ca: El participatn $name a acceptat la invitació\n  el: Το συμμετέχων πρόσωπο $name αποδέχτηκε τη πρόσκληση\n  sv: Deltagaren $name har tackat ja till inbjudan\n  pl: Uczestnik $name zaakceptował zaproszenie\n\ncalendar.declined:\n  en: Declined\n  es: Rechazado\n  fr: Refusé\n  de: Abgelehnt\n  it: Rifiutato\n  pt: Recusado\n  nl: Afgewezen\n  da: Afvist\n  ca: Rebutjat\n  el: Απορρίφθηκε\n  sv: Avböjt\n  pl: Odrzucono\n\ncalendar.participant_declined:\n  en: Participant $name declined the invitation\n  es: El participante $name rechazó la invitación\n  fr: Le participant $name a refusé l'invitation\n  de: Teilnehmer $name hat die Einladung abgelehnt\n  it: Il partecipante $name ha rifiutato l'invito\n  pt: O participante $name recusou o convite\n  nl: Deelnemer $name heeft de uitnodiging afgewezen\n  da: Deltager $name afslog invitationen\n  ca: El participatn $name ha rebutjat la invitació\n  el: Το συμμετέχων πρόσωπο $name απέρριψε τη πρόσκληση\n  sv: Deltagaren $name har tackat nej till inbjudan\n  pl: Uczestnik $name odrzucił zaproszenie\n\ncalendar.tentative:\n  en: Tentative\n  es: Provisional\n  fr: Provisoire\n  de: Vorläufig\n  it: Provvisorio\n  pt: Provisório\n  nl: Voorlopig\n  da: Foreløbig\n  ca: Provisional\n  el: Μη δεσμευτικά\n  sv: Möjligen\n  pl: Wstępnie\n\ncalendar.participant_tentative:\n  en: Participant $name tentatively accepted the invitation\n  es: El participante $name aceptó provisionalmente la invitación\n  fr: Le participant $name a accepté provisoirement l'invitation\n  de: Teilnehmer $name hat die Einladung vorläufig angenommen\n  it: Il partecipante $name ha accettato provvisoriamente l'invito\n  pt: O participante $name aceitou provisoriamente o convite\n  nl: Deelnemer $name heeft de uitnodiging voorlopig geaccepteerd\n  da: Deltager $name accepterede foreløbigt invitationen\n  ca: El participan $name accepta provisionalment la invitació a l'esdeveniment\n  el: Το συμμετέχων πρόσωπο $name αποδέχτηκε μη δεσμευτικά τη πρόσκληση\n  sv: Deltagaren $name har preliminärt tackat ja till inbjudan\n  pl: Uczestnik $name wstępnie zaakceptował zaproszenie\n\ncalendar.delegated:\n  en: Delegated\n  es: Delegado\n  fr: Délégué\n  de: Delegiert\n  it: Delegato\n  pt: Delegado\n  nl: Gedelegeerd\n  da: Delegeret\n  ca: Delegat\n  el: Ανατέθηκε\n  sv: Delegerat\n  pl: Delegowane\n\ncalendar.participant_delegated:\n  en: Participant $name delegated the invitation\n  es: El participante $name delegó la invitación\n  fr: Le participant $name a délégué l'invitation\n  de: Teilnehmer $name hat die Einladung delegiert\n  it: Il partecipante $name ha delegato l'invito\n  pt: O participante $name delegou o convite\n  nl: Deelnemer $name heeft de uitnodiging gedelegeerd\n  da: Deltager $name delegerede invitationen\n  ca: El participant $name ha delegat la invitació\n  el: Το συμμετέχων πρόσωπο $name ανέθεσε τη πρόσκληση\n  sv: Deltagaren $name delegerade inbjudan\n  pl: Uczestnik $name przekazał zaproszenie\n\ncalendar.reply:\n  en: Reply\n  es: Respuesta\n  fr: Réponse\n  de: Antwort\n  it: Risposta\n  pt: Resposta\n  nl: Antwoord\n  da: Svar\n  ca: Resposta\n  el: Απάντηση\n  sv: Svar\n  pl: Odpowiedź\n\ncalendar.participant_reply:\n  en: Participant $name replied to the invitation\n  es: El participante $name respondió a la invitación\n  fr: Le participant $name a répondu à l'invitation\n  de: Teilnehmer $name hat auf die Einladung geantwortet\n  it: Il partecipante $name ha risposto all'invito\n  pt: O participante $name respondeu ao convite\n  nl: Deelnemer $name heeft gereageerd op de uitnodiging\n  da: Deltager $name svarede på invitationen\n  ca: El participant $name ha respost a la invitació\n  el: Το συμμετέχων πρόσωπο $name απάντησε στη πρόσκληση\n  sv: Deltagaren $name har svarat på inbjudan\n  pl: Uczestnik $name odpowiedział na zaproszenie\n\ncalendar.summary:\n  en: Summary\n  es: Resumen\n  fr: Résumé\n  de: Zusammenfassung\n  it: Riepilogo\n  pt: Resumo\n  nl: Samenvatting\n  da: Resumé\n  ca: Resum\n  el: Περίληψη\n  sv: Sammanfattning\n  pl: Podsumowanie\n\ncalendar.description:\n  en: Description\n  es: Descripción\n  fr: Description\n  de: Beschreibung\n  it: Descrizione\n  pt: Descrição\n  nl: Beschrijving\n  da: Beskrivelse\n  ca: Descripció\n  el: Περιγραφή\n  sv: Beskrivning\n  pl: Opis\n\ncalendar.when:\n  en: When\n  es: Cuándo\n  fr: Quand\n  de: Wann\n  it: Quando\n  pt: Quando\n  nl: Wanneer\n  da: Hvornår\n  ca: Quan\n  el: Πότε\n  sv: När\n  pl: Kiedy\n\ncalendar.changed:\n  en: Changed\n  es: Cambiado\n  fr: Modifié\n  de: Geändert\n  it: Modificato\n  pt: Alterado\n  nl: Gewijzigd\n  da: Ændret\n  ca: Canviat\n  el: Μεταβλήθηκε\n  sv: Ändrat\n  pl: Zmieniono\n\ncalendar.reply_as:\n  en: Reply as $name for this event series\n  es: Responder como $name para esta serie de eventos\n  fr: Répondre en tant que $name pour cette série d'événements\n  de: Als $name für diese Ereignisserie antworten\n  it: Rispondi come $name per questa serie di eventi\n  pt: Responder como $name para esta série de eventos\n  nl: Antwoord als $name voor deze evenementenreeks\n  da: Svar som $name for denne begivenhedsserie\n  ca: Respondre com a $name per aquesta serie d'esdeveniments\n  el: Απάντηση ως $name για αυτή τη σειρά εκδηλώσεων\n  sv: Svara som $name på den här inbjudan\n  pl: Odpowiedz jako $name dla tej serii wydarzeń\n\ncalendar.yes:\n  en: Yes\n  es: Sí\n  fr: Oui\n  de: Ja\n  it: Sì\n  pt: Sim\n  nl: Ja\n  da: Ja\n  ca: Sí\n  el: Ναι\n  sv: Ja\n  pl: Tak\n\ncalendar.no:\n  en: No\n  es: No\n  fr: Non\n  de: Nein\n  it: No\n  pt: Não\n  nl: Nee\n  da: Nej\n  ca: No\n  el: Όχι\n  sv: Nej\n  pl: Nie\n\ncalendar.maybe:\n  en: Maybe\n  es: Quizás\n  fr: Peut-être\n  de: Vielleicht\n  it: Forse\n  pt: Talvez\n  nl: Misschien\n  da: Måske\n  ca: Potser\n  el: Ίσως\n  sv: Kanske\n  pl: Może\n\ncalendar.imip_footer_1:\n  en: You're receiving this e-mail as you're listed as a participant for this event.\n  es: Recibes este correo electrónico porque estás registrado como participante de este evento.\n  fr: Vous recevez cet e-mail car vous êtes inscrit comme participant à cet événement.\n  de: Sie erhalten diese E-Mail, weil Sie als Teilnehmer für dieses Ereignis aufgeführt sind.\n  it: Ricevi questa e-mail perché sei elencato come partecipante a questo evento.\n  pt: Você está recebendo este e-mail porque está listado como participante deste evento.\n  nl: U ontvangt deze e-mail omdat u staat vermeld als deelnemer aan dit evenement.\n  da: Du modtager denne e-mail, fordi du er opført som deltager i denne begivenhed.\n  ca: Reps aquest correu electrònic perquè ets registrat com a participant d'aquest esdeveniment.\n  el: Λαμβάνετε αυτό το e-mail καθώς είστε στη λίστα συμμετεχόντων αυτής της εκδήλωσης.\n  sv: Du får det här mejlet för att du är uppskriven som deltagare i den här händelsen\n  pl: Otrzymujesz tego e-maila, ponieważ jesteś uczestnikiem tego wydarzenia.\n\ncalendar.imip_footer_2:\n  en: Forwarding this e-mail could allow any recipient to reply to the organizer, join the guest list, extend the invitation to others, or alter your RSVP.\n  es: Reenviar este correo electrónico podría permitir que cualquier destinatario responda al organizador, se una a la lista de invitados, extienda la invitación a otros o modifique tu confirmación de asistencia.\n  fr: Le transfert de cet e-mail pourrait permettre à tout destinataire de répondre à l'organisateur, de rejoindre la liste des invités, d'étendre l'invitation à d'autres ou de modifier votre RSVP.\n  de: Das Weiterleiten dieser E-Mail könnte es jedem Empfänger ermöglichen, dem Organisator zu antworten, der Gästeliste beizutreten, die Einladung an andere weiterzugeben oder Ihre Zusage zu ändern.\n  it: L'inoltro di questa e-mail potrebbe consentire a qualsiasi destinatario di rispondere all'organizzatore, unirsi alla lista degli ospiti, estendere l'invito ad altri o modificare la tua conferma di partecipazione.\n  pt: Encaminhar este e-mail pode permitir que qualquer destinatário responda ao organizador, se junte à lista de convidados, estenda o convite a outros ou altere sua confirmação de presença.\n  nl: Het doorsturen van deze e-mail kan elke ontvanger in staat stellen om te reageren op de organisator, deel te nemen aan de gastenlijst, de uitnodiging uit te breiden naar anderen, of uw RSVP te wijzigen.\n  da: Videresendelse af denne e-mail kan give andre mulighed for at svare arrangøren, tilslutte sig gæstelisten, videreformidle invitationen eller ændre din tilmelding.\n  ca: Reenviar aquest correu electrònic podria fer que qualsevol destinatari respongues a l'organitzador, afegir-se a la llista de convidats, estengui la invitació a d'altres o modificar la teva confirmació d'assistència.\n  el: Προωθόντας αυτό το e-mail μπορεί να επιτρέψει σε οποιοδήποτε παραλήπτη να απαντήση στον οργανωτή, να μπει στη λίστα επισκεπτών, να επεκτείνει τη πρόσκληση σε άλλους ή να αλλάξει την παρουσία σας.\n  sv: Att vidarebefordra det här mejlet kan göra det möjligt för vilken mottagare som helst att skicka svar till arrangören, lägga till sig själv på gästlistan, skicka inbjudan vidare till andra, samt ändra ditt svar.\n  pl: Przekazanie tej wiadomości e-mail może umożliwić każdemu odbiorcy odpowiedź do organizatora, dołączenie do listy gości, przesłanie zaproszenia innym osobom lub zmianę Twojej odpowiedzi RSVP.\n\ncalendar.rsvp_recorded:\n  en: Your RSVP has been recorded.\n  es: Tu confirmación de asistencia ha sido registrada.\n  fr: Votre RSVP a été enregistré.\n  de: Ihre Zusage wurde aufgezeichnet.\n  it: La tua conferma di partecipazione è stata registrata.\n  pt: Sua confirmação de presença foi registrada.\n  nl: Uw RSVP is geregistreerd.\n  da: Din tilmelding er blevet registreret.\n  ca: La teva confirmació d'assistència ha sigut registrada.\n  el: Η κοινοποίηση της παρουσίας σας καταγράφηκε.\n  sv: Ditt svar har registrerats.\n  pl: Twoja odpowiedź RSVP została zarejestrowana.\n\ncalendar.rsvp_failed:\n  en: Failed to record your RSVP.\n  es: No se pudo registrar tu confirmación de asistencia.\n  fr: Impossible d'enregistrer votre RSVP.\n  de: Ihre Zusage konnte nicht aufgezeichnet werden.\n  it: Impossibile registrare la tua conferma di partecipazione.\n  pt: Falha ao registrar sua confirmação de presença.\n  nl: Kan uw RSVP niet registreren.\n  da: Kunne ikke registrere din tilmelding.\n  ca: No s'ha pogut registrar la teva confirmació d'assistència.\n  el: Αποτυχία κατα τη καταγραφή της παρουσίας σας\n  sv: Det gick inte att registrera ditt svar.\n  pl: Nie udało się zarejestrować Twojej odpowiedzi RSVP.\n\ncalendar.event_not_found:\n  en: The event you are trying to RSVP to was not found.\n  es: No se encontró el evento al que intentas confirmar asistencia.\n  fr: L'événement auquel vous essayez de répondre n'a pas été trouvé.\n  de: Das Ereignis, für das Sie eine Zusage geben möchten, wurde nicht gefunden.\n  it: L'evento a cui stai cercando di confermare la partecipazione non è stato trovato.\n  pt: O evento para o qual você está tentando confirmar presença não foi encontrado.\n  nl: Het evenement waarvoor u probeert te reageren is niet gevonden.\n  da: Begivenheden du forsøger at tilmelde dig blev ikke fundet.\n  ca: L'esdeveniment al que intentes confirmar l'assistència no es troba.\n  el: Η εκδήλωση που θέλετε να κοινοποιήσετε την παρουσία σας δεν υπάρχει.\n  sv: Händelsen du försöker OSA till kunde inte hittas.\n  pl: Wydarzenie, na które próbujesz odpowiedzieć RSVP, nie zostało znalezione.\n\ncalendar.invalid_rsvp:\n  en: The RSVP request was invalid or malformed.\n  es: La solicitud de confirmación de asistencia era inválida o estaba mal formada.\n  fr: La demande de RSVP était invalide ou mal formée.\n  de: Die Zusage-Anfrage war ungültig oder fehlerhaft.\n  it: La richiesta di conferma di partecipazione era non valida o mal formata.\n  pt: A solicitação de confirmação de presença era inválida ou mal formada.\n  nl: Het RSVP-verzoek was ongeldig of onjuist gevormd.\n  da: Tilmeldingsanmodningen var ugyldig eller forkert udformet.\n  ca: La sol·licitud de confirmació d'assistència no era vàlida o estava mal formada.\n  el: Η αίτητη για κοινοποίηση παρουσίας είναι άκυρη ή έχει πρόβλημα.\n  sv: Svaret på inbjudan var ogiltigt eller i fel format.\n  pl: Żądanie RSVP było nieprawidłowe lub źle sformułowane.\n\ncalendar.not_participant:\n  en: You are no longer a participant in this event.\n  es: Ya no eres participante de este evento.\n  fr: Vous n'êtes plus participant à cet événement.\n  de: Sie sind kein Teilnehmer dieses Ereignisses mehr.\n  it: Non sei più un partecipante a questo evento.\n  pt: Você não é mais um participante deste evento.\n  nl: U bent geen deelnemer meer aan dit evenement.\n  da: Du deltager ikke længere i denne begivenhed.\n  ca: Ja no ets un participant d'aquest esdeveniment.\n  el: Δε συμμετέχετε πια σε αυτή την εκδήλωση.\n  sv: Du är inte längre en deltagare i den här händelse.\n  pl: Nie jesteś już uczestnikiem tego wydarzenia.\n"
  },
  {
    "path": "resources/scripts/ossify.py",
    "content": "#!/usr/bin/env python3\n\"\"\"\nStalwart SEL code remover\n\nThis script removes SEL code from the Stalwart codebase by:\n1. Removing entire .rs files that contain \"SPDX-License-Identifier: LicenseRef-SEL\" in their first comment\n2. Removing SEL snippets marked with SPDX-SnippetBegin/End from mixed files\n\nUsage: python ossify.py <stalwart_repository>/crates\n\"\"\"\n\nimport os\nimport sys\nimport re\nimport argparse\nfrom pathlib import Path\nfrom typing import List, Tuple, Optional\n\n\ndef find_first_comment_block(content: str) -> Optional[str]:\n    \"\"\"\n    Find the first comment block in a Rust file.\n    Returns the comment content or None if no comment block is found.\n    \"\"\"\n    # Remove leading whitespace and find the first comment\n    lines = content.strip().split('\\n')\n    \n    if not lines:\n        return None\n    \n    first_line = lines[0].strip()\n    \n    # Check for block comment starting with /*\n    if first_line.startswith('/*'):\n        comment_lines = []\n        in_comment = True\n        \n        for line in lines:\n            if in_comment:\n                comment_lines.append(line)\n                if '*/' in line:\n                    break\n        \n        return '\\n'.join(comment_lines)\n    \n    # Check for line comments starting with //\n    elif first_line.startswith('//'):\n        comment_lines = []\n        \n        for line in lines:\n            stripped = line.strip()\n            if stripped.startswith('//'):\n                comment_lines.append(line)\n            elif stripped == '':\n                comment_lines.append(line)  # Keep empty lines within comment block\n            else:\n                break  # Stop at first non-comment, non-empty line\n        \n        return '\\n'.join(comment_lines)\n    \n    return None\n\n\ndef should_remove_file(file_path: str) -> bool:\n    \"\"\"\n    Check if a .rs file should be completely removed based on its first comment.\n    Returns True if the file contains \"SPDX-License-Identifier: LicenseRef-SEL\" in the first comment.\n    \"\"\"\n    try:\n        with open(file_path, 'r', encoding='utf-8') as f:\n            content = f.read()\n        \n        first_comment = find_first_comment_block(content)\n        if first_comment and 'SPDX-License-Identifier: LicenseRef-SEL' in first_comment:\n            return True\n            \n    except Exception as e:\n        print(f\"Error reading file {file_path}: {e}\")\n    \n    return False\n\n\ndef remove_proprietary_snippets(content: str) -> Tuple[str, int]:\n    \"\"\"\n    Remove proprietary snippets from file content.\n    Returns tuple of (modified_content, number_of_snippets_removed)\n    \"\"\"\n    snippets_removed = 0\n    \n    # Pattern to match SPDX snippets that contain LicenseRef-SEL\n    # We look for SPDX-SnippetBegin, then check if the snippet contains LicenseRef-SEL,\n    # and if so, remove everything until SPDX-SnippetEnd\n    \n    lines = content.split('\\n')\n    result_lines = []\n    i = 0\n    \n    while i < len(lines):\n        line = lines[i]\n        \n        # Check if this line starts a snippet\n        if '// SPDX-SnippetBegin' in line:\n            # Look ahead to see if this snippet contains LicenseRef-SEL\n            snippet_start = i\n            snippet_lines = []\n            j = i\n            \n            # Collect the snippet lines until we find SnippetEnd or reach end of file\n            while j < len(lines):\n                snippet_lines.append(lines[j])\n                if '// SPDX-SnippetEnd' in lines[j]:\n                    break\n                j += 1\n            \n            # Check if this snippet contains LicenseRef-SEL\n            snippet_content = '\\n'.join(snippet_lines)\n            if 'SPDX-License-Identifier: LicenseRef-SEL' in snippet_content:\n                # Remove this snippet\n                snippets_removed += 1\n                i = j + 1  # Skip past the SnippetEnd line\n                continue\n            else:\n                # Keep this snippet as it's not proprietary\n                result_lines.append(line)\n                i += 1\n        else:\n            result_lines.append(line)\n            i += 1\n    \n    return '\\n'.join(result_lines), snippets_removed\n\n\ndef process_rust_file(file_path: str, dry_run: bool = False) -> dict:\n    \"\"\"\n    Process a single Rust file, removing proprietary content.\n    Returns a dictionary with processing results.\n    \"\"\"\n    result = {\n        'file': file_path,\n        'action': 'none',\n        'snippets_removed': 0,\n        'error': None\n    }\n    \n    try:\n        # Check if the entire file should be removed\n        if should_remove_file(file_path):\n            result['action'] = 'file_removed'\n            if not dry_run:\n                os.remove(file_path)\n            return result\n        \n        # Process snippets in the file\n        with open(file_path, 'r', encoding='utf-8') as f:\n            original_content = f.read()\n        \n        modified_content, snippets_removed = remove_proprietary_snippets(original_content)\n        \n        if snippets_removed > 0:\n            result['action'] = 'snippets_removed'\n            result['snippets_removed'] = snippets_removed\n            \n            if not dry_run:\n                with open(file_path, 'w', encoding='utf-8') as f:\n                    f.write(modified_content)\n        \n    except Exception as e:\n        result['error'] = str(e)\n    \n    return result\n\n\ndef find_rust_files(directory: str) -> List[str]:\n    \"\"\"Find all .rs files in the given directory recursively.\"\"\"\n    rust_files = []\n    \n    for root, dirs, files in os.walk(directory):\n        for file in files:\n            if file.endswith('.rs'):\n                rust_files.append(os.path.join(root, file))\n    \n    return rust_files\n\n\ndef main():\n    parser = argparse.ArgumentParser(\n        description='Remove Enterprise licensed code from Stalwart codebase'\n    )\n    parser.add_argument(\n        'directory',\n        help='Directory containing Stalwart code to process'\n    )\n    parser.add_argument(\n        '--dry-run',\n        action='store_true',\n        help='Show what would be done without making changes'\n    )\n    parser.add_argument(\n        '--verbose',\n        action='store_true',\n        help='Show detailed output for each file'\n    )\n    \n    args = parser.parse_args()\n    \n    if not os.path.isdir(args.directory):\n        print(f\"Error: {args.directory} is not a valid directory\")\n        sys.exit(1)\n    \n    print(f\"Processing Rust files in: {args.directory}\")\n    if args.dry_run:\n        print(\"DRY RUN MODE - No changes will be made\")\n    print()\n    \n    rust_files = find_rust_files(args.directory)\n    \n    if not rust_files:\n        print(\"No .rs files found in the specified directory\")\n        return\n    \n    print(f\"Found {len(rust_files)} Rust files\")\n    print()\n    \n    files_removed = 0\n    files_with_snippets_removed = 0\n    total_snippets_removed = 0\n    errors = []\n    \n    for file_path in rust_files:\n        result = process_rust_file(file_path, args.dry_run)\n        \n        if result['error']:\n            errors.append(f\"{file_path}: {result['error']}\")\n            continue\n        \n        if result['action'] == 'file_removed':\n            files_removed += 1\n            if args.verbose or args.dry_run:\n                action_text = \"Would remove\" if args.dry_run else \"Removed\"\n                print(f\"{action_text} file: {file_path}\")\n        \n        elif result['action'] == 'snippets_removed':\n            files_with_snippets_removed += 1\n            total_snippets_removed += result['snippets_removed']\n            if args.verbose or args.dry_run:\n                action_text = \"Would remove\" if args.dry_run else \"Removed\"\n                print(f\"{action_text} {result['snippets_removed']} snippet(s) from: {file_path}\")\n    \n    # Summary\n    print(\"\\nSummary:\")\n    action_text = \"Would be\" if args.dry_run else \"Were\"\n    print(f\"- {files_removed} files {action_text.lower()} completely removed\")\n    print(f\"- {total_snippets_removed} proprietary snippets {action_text.lower()} removed from {files_with_snippets_removed} files\")\n    \n    if errors:\n        print(f\"- {len(errors)} errors occurred:\")\n        for error in errors:\n            print(f\"  {error}\")\n    \n    if args.dry_run:\n        print(\"\\nRun without --dry-run to apply changes\")\n\n\nif __name__ == '__main__':\n    main()\n"
  },
  {
    "path": "resources/systemd/stalwart-mail.service",
    "content": "[Unit]\nDescription=Stalwart Server\nConflicts=postfix.service sendmail.service exim4.service\nConditionPathExists=__PATH__/etc/config.toml\nAfter=network-online.target\n\n[Service]\nType=simple\nLimitNOFILE=65536\nKillMode=process\nKillSignal=SIGINT\nRestart=on-failure\nRestartSec=5\nExecStart=__PATH__/bin/stalwart --config=__PATH__/etc/config.toml\nSyslogIdentifier=stalwart\nUser=stalwart\nGroup=stalwart\nAmbientCapabilities=CAP_NET_BIND_SERVICE\n\n[Install]\nWantedBy=multi-user.target\n"
  },
  {
    "path": "resources/systemd/stalwart.mail.plist",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE plist PUBLIC \"-//Apple Computer//DTD PLIST 1.0//EN\"\n    \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n<plist version=\"1.0\">\n    <dict>\n        <key>Label</key>\n        <string>stalwart.mail</string>\n        <key>ServiceDescription</key>\n        <string>Stalwart</string>\n        <key>ProgramArguments</key>\n        <array>\n            <string>__PATH__/bin/stalwart</string>\n            <string>--config=__PATH__/etc/config.toml</string>\n        </array>\n        <key>RunAtLoad</key>\n        <true/>\n        <key>KeepAlive</key>\n        <true/>\n    </dict>\n</plist>\n"
  },
  {
    "path": "tests/Cargo.toml",
    "content": "[package]\nname = \"tests\"\nversion = \"0.15.5\"\nedition = \"2024\"\n\n[features]\n#default = [\"sqlite\", \"postgres\", \"mysql\", \"rocks\", \"s3\", \"redis\", \"nats\", \"azure\", \"foundationdb\"]\ndefault = [\"sqlite\", \"postgres\", \"mysql\", \"rocks\", \"s3\", \"redis\", \"foundationdb\"]\n#default = [\"rocks\", \"foundationdb\"]\nsqlite = [\"store/sqlite\"]\nfoundationdb = [\"store/foundation\", \"common/foundation\"]\npostgres = [\"store/postgres\"]\nmysql = [\"store/mysql\"]\nrocks = [\"store/rocks\"]\ns3 = [\"store/s3\"]\nredis = [\"store/redis\"]\nnats = [\"store/nats\"]\nazure = [\"store/azure\"]\n\n[dev-dependencies]\nstore = { path = \"../crates/store\", features = [\"test_mode\", \"enterprise\"] }\nnlp = { path = \"../crates/nlp\" }\ndirectory = { path = \"../crates/directory\", features = [\"test_mode\", \"enterprise\"] }\njmap = { path = \"../crates/jmap\", features = [\"test_mode\", \"enterprise\"] }\njmap_proto = { path = \"../crates/jmap-proto\" }\nimap = { path = \"../crates/imap\", features = [\"test_mode\"] }\nimap_proto = { path = \"../crates/imap-proto\" }\ntypes = { path = \"../crates/types\" }\ndav = { path = \"../crates/dav\", features = [\"test_mode\"] }\ndav-proto = { path = \"../crates/dav-proto\", features = [\"test_mode\"] }\ncalcard = { version = \"0.3\", features = [\"rkyv\"] }\ngroupware = { path = \"../crates/groupware\", features = [\"test_mode\"] }\nhttp = { path = \"../crates/http\", features = [\"test_mode\", \"enterprise\"] }\nhttp_proto = { path = \"../crates/http-proto\" }\nservices = { path = \"../crates/services\", features = [\"test_mode\", \"enterprise\"] }\npop3 = { path = \"../crates/pop3\", features = [\"test_mode\"] }\nsmtp = { path = \"../crates/smtp\", features = [\"test_mode\", \"enterprise\"] }\ncommon = { path = \"../crates/common\", features = [\"test_mode\", \"enterprise\"] }\nemail = { path = \"../crates/email\", features = [\"test_mode\", \"enterprise\"] }\nspam-filter = { path = \"../crates/spam-filter\", features = [\"test_mode\", \"enterprise\"] }\nmigration = { path = \"../crates/migration\", features = [\"test_mode\", \"enterprise\"] }\ntrc = { path = \"../crates/trc\", features = [\"enterprise\"] }\nmanagesieve = { path = \"../crates/managesieve\", features = [\"test_mode\", \"enterprise\"] }\nsmtp-proto = { version = \"0.2\" }\nmail-send = { version = \"0.5\", default-features = false, features = [\"cram-md5\", \"ring\", \"tls12\"] }\nmail-auth = { version = \"0.7.1\", features = [\"test\"] }\nmail-parser = { version = \"0.11\", features = [\"full_encoding\", \"rkyv\"] } \nmail-builder = \"0.4.4\"\nsieve-rs = { version = \"0.7\", features = [\"rkyv\"] } \nutils = { path = \"../crates/utils\", features = [\"test_mode\"] }\njmap-client = { version = \"0.4\", features = [\"websockets\", \"debug\", \"async\"] } \ntokio = { version = \"1.47\", features = [\"full\"] }\ntokio-rustls = { version = \"0.26\", default-features = false, features = [\"ring\", \"tls12\"] }\nrustls = { version = \"0.23.5\", default-features = false, features = [\"std\", \"ring\", \"tls12\"] }\nrustls-pemfile = \"2.0\"\nrustls-pki-types = { version = \"1\" }\ncsv = \"1.1\"\nrayon = { version = \"1.5.1\" }\nflate2 = { version = \"1.0.17\", features = [\"zlib\"], default-features = false }\nserde = { version = \"1.0\", features = [\"derive\"]}\nserde_json = \"1.0\"\nreqwest = { version = \"0.12\", default-features = false, features = [\"rustls-tls-webpki-roots\", \"multipart\", \"http2\"]}\nbytes = \"1.4.0\"\nfutures = \"0.3\"\nece = \"2.2\"\nhyper = { version = \"1.0.1\", features = [\"server\", \"http1\", \"http2\"] }\nhyper-util = { version = \"0.1.1\", features = [\"tokio\"] }\nhttp-body-util = \"0.1.0\"\nbase64 = \"0.22\"\nahash = { version = \"0.8\" }\nserial_test = \"3.0.0\"\nnum_cpus = \"1.15.0\"\nasync-trait = \"0.1.68\"\nchrono = \"0.4\"\nring = { version = \"0.17\" }\nbiscuit = \"0.7.0\"\nform_urlencoded = \"1.1.0\"\nrkyv = { version = \"0.8.10\", features = [\"little_endian\"] }\ncompact_str = \"0.9.0\"\nquick-xml = \"0.38\"\n\n[target.'cfg(not(target_env = \"msvc\"))'.dependencies]\njemallocator = \"0.5.0\"\n"
  },
  {
    "path": "tests/resources/acme/Docker.pebble",
    "content": "FROM golang:1.18-alpine as builder\n\nENV CGO_ENABLED=0\n\nWORKDIR /pebble-src\n\nRUN apk update && apk add --no-cache git\nRUN git clone https://github.com/letsencrypt/pebble/ /pebble-src\nRUN go build -o /go/bin/pebble ./cmd/pebble\n\n## main\nFROM alpine:3.15.4\n\nCOPY --from=builder /go/bin/pebble /usr/bin/pebble\nCOPY --from=builder /pebble-src/test/ /test/\n\nCMD [ \"/usr/bin/pebble\" ]\n\nEXPOSE 14000\nEXPOSE 15000\n\n# Build:\n# docker build -f Docker.pebble -t pebble\n\n# Run:\n# docker run -d -p 14000:14000 -p 15000:15000 pebble\n# docker run -d --name pebble -p 14000:14000 -p 15000:15000 pebble pebble -config /test/config/pebble-config.json -strict\n\n"
  },
  {
    "path": "tests/resources/acme/config.toml",
    "content": "acme.pebble.contact = \"postmaster@example.org\"\nacme.pebble.directory = \"https://localhost:14000/dir\"\n#acme.pebble.domains = \"mail.example.org\"\nacme.pebble.renew-before = \"30d\"\n\nacme.pebble.challenge = \"tls-alpn-01\"\n#acme.pebble.challenge = \"http-01\"\n#acme.pebble.challenge = \"dns-01\"\n\nacme.pebble.domains = \"*.example.org\"\nacme.pebble.provider = \"cloudflare\"\nacme.pebble.secret = \"<KEY>\"\n\nauthentication.fallback-admin.secret = \"secret\"\nauthentication.fallback-admin.user = \"admin\"\nconfig.local-keys.0 = \"*\"\ndirectory.internal.store = \"rocksdb\"\ndirectory.internal.type = \"internal\"\nlookup.default.hostname = \"mail.example.org\"\nlookup.default.domain = \"example.org\"\noauth.key = \"0Wn7rO4UdmBoE8mp3cDcD9Qlpz3na74z7fGRoSuq8fVsGPelLl3KrHomBN8h2biA\"\nqueue.quota.size.enable = true\nqueue.quota.size.messages = 100000\nqueue.quota.size.size = 10737418240\nreport.analysis.addresses = \"postmaster@*\"\nserver.http.permissive-cors = true\nserver.listener.http.bind = \"[::]:5002\"\nserver.listener.http.protocol = \"http\"\nserver.listener.https.bind = \"[::]:5001\"\nserver.listener.https.protocol = \"http\"\nserver.listener.https.tls.implicit = true\nserver.listener.imap.bind = \"[::]:143\"\nserver.listener.imap.protocol = \"imap\"\nserver.listener.imaptls.bind = \"[::]:993\"\nserver.listener.imaptls.protocol = \"imap\"\nserver.listener.imaptls.tls.implicit = true\nserver.listener.sieve.bind = \"[::]:4190\"\nserver.listener.sieve.protocol = \"managesieve\"\nserver.listener.smtp.bind = \"[::]:25\"\nserver.listener.smtp.protocol = \"smtp\"\nserver.listener.submission.bind = \"[::]:587\"\nserver.listener.submission.protocol = \"smtp\"\nserver.listener.submissions.bind = \"[::]:465\"\nserver.listener.submissions.protocol = \"smtp\"\nserver.listener.submissions.tls.implicit = true\nstorage.blob = \"rocksdb\"\nstorage.data = \"rocksdb\"\nstorage.directory = \"internal\"\nstorage.fts = \"rocksdb\"\nstorage.lookup = \"rocksdb\"\nstore.rocksdb.compression = \"lz4\"\nstore.rocksdb.path = \"/tmp/stalwart-temp-data\"\nstore.rocksdb.type = \"rocksdb\"\ntracer.stdout.ansi = true\ntracer.stdout.enable = true\ntracer.stdout.level = \"trace\"\ntracer.stdout.type = \"stdout\"\nversion.spam-filter = 1.0\n"
  },
  {
    "path": "tests/resources/acme/docker-compose-pebble.yaml",
    "content": "# docker-compose -f docker-compose-pebble.yaml up\n# curl --request POST --data '{\"ip\":\"192.168.5.2\"}' http://localhost:8055/set-default-ipv4\n# HTTPS port should be 5001\n# HTTP port should be 5002\n# Directory https://localhost:14000/dir\n\nversion: '3'\nservices:\n  pebble:\n    image: letsencrypt/pebble:latest\n    command: pebble -config /test/config/pebble-config.json -strict -dnsserver 10.30.50.3:8053 #-dnsserver 8.8.8.8:53\n    ports:\n      - 14000:14000 # HTTPS ACME API\n      - 15000:15000 # HTTPS Management API\n    networks:\n      acmenet:\n        ipv4_address: 10.30.50.2\n  challtestsrv:\n    image: letsencrypt/pebble-challtestsrv:latest\n    command: pebble-challtestsrv -defaultIPv6 \"\" -defaultIPv4 10.30.50.3\n    ports:\n      - 8055:8055 # HTTP Management API\n    networks:\n      acmenet:\n        ipv4_address: 10.30.50.3\n\nnetworks:\n  acmenet:\n    driver: bridge\n    ipam:\n      driver: default\n      config:\n        - subnet: 10.30.50.0/24\n"
  },
  {
    "path": "tests/resources/acme/test_acme.sh",
    "content": "#!/bin/sh\n\nrm -Rf /tmp/stalwart-temp-data\nmkdir -p /tmp/stalwart-temp-data\ncp ./tests/resources/acme/config.toml /tmp/stalwart-temp-data/config.toml\n\ncurl --request POST --data '{\"ip\":\"192.168.5.2\"}' http://localhost:8055/set-default-ipv4\n\ncargo run -p stalwart --no-default-features --features \"sqlite foundationdb postgres mysql rocks elastic s3 redis\" -- --config=/tmp/stalwart-temp-data/config.toml\n"
  },
  {
    "path": "tests/resources/crypto/cert_mixed.pem",
    "content": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nxjMEZMYfNhYJKwYBBAHaRw8BAQdAYyTN1HzqapLw8xwkCGwa0OjsgT/JqhcB/+Dy\nGa1fsBrNG0pvaG4gRG9lIDxqb2huQGV4YW1wbGUub3JnPsKJBBMWCAAxFiEEg836\npwbXpuQ/THMtpJwd4oBfIrUFAmTGHzYCGwMECwkIBwUVCAkKCwUWAgMBAAAKCRCk\nnB3igF8itYhyAQD2jEdeYa3gyQ47X9YWZTK1wEJkN8W9//V1fYl2XQwqlQEA0qBv\nAi6nUh99oDw+/zQ8DFIKdeb5Ti4tu/X58PdpiQ7OOARkxh82EgorBgEEAZdVAQUB\nAQdAvXz2FbFN0DovQF/ACnZyczTsSIQp0mvmF1PE+aijbC8DAQgHwngEGBYIACAW\nIQSDzfqnBtem5D9Mcy2knB3igF8itQUCZMYfNgIbDAAKCRCknB3igF8itRnoAQC3\nGzPmgx7TnB+SexPuJV/DoKSMJ0/X+hbEFcZkulxaDQEAh+xiJCvf+ZNAKw6kFhsL\nUuZhEDktxnY6Ehz3aB7FawA=\n=KGrr\n-----END PGP PUBLIC KEY BLOCK-----\n\n-----BEGIN CERTIFICATE-----\nMIIDbjCCAlagAwIBAgIUZ4K0WXNSS8H0cUcZavD9EYqqTAswDQYJKoZIhvcNAQEN\nBQAwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0\neTAgFw0xOTExMjAwNjU0MThaGA8yMDUyMDkyNzA2NTQxOFowGTEXMBUGA1UEAxMO\nQWxpY2UgTG92ZWxhY2UwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDD\n7q35ZdG2JAzzJGNZDZ9sV7AKh0hlRfoFjTZN5m4RegQAYSyag43ouWi1xRN0avf0\nUTYrwjK04qRdV7GzCACoEKq/xiNUOsjfJXzbCublN3fZMOXDshKKBqThlK75SjA9\nCzxg7ejGoiY/iidk0e91neK30SCCaBTJlfR2ZDrPk73IPMeksxoTatfF9hw9dDA+\n/Hi1yptN/aG0Q/s9icFrxr6y2zQXsjuQPmjMZgj10aD9cazWVgRYCgflhmA0V1uQ\nl1wobYU8DAVxVn+GgabqyjGQMoythIK0Gn5+ofwxXXUM/zbU+g6+1ISdoXxRRFtq\n2GzbIqkAHZZQm+BbnFrhAgMBAAGjgZcwgZQwDAYDVR0TAQH/BAIwADAeBgNVHREE\nFzAVgRNhbGljZUBzbWltZS5leGFtcGxlMBMGA1UdJQQMMAoGCCsGAQUFBwMEMA8G\nA1UdDwEB/wQFAwMHoAAwHQYDVR0OBBYEFKwuVFqk/VUYry7oZkQ40SXR1wB5MB8G\nA1UdIwQYMBaAFLdSTXPAiD2yw3paDPOU9/eAonfbMA0GCSqGSIb3DQEBDQUAA4IB\nAQB76o4Yz7yrVSFcpXqLrcGtdI4q93aKCXECCCzNQLp4yesh6brqaZHNJtwYcJ5T\nqbUym9hJ70iJE4jGNN+yAZR1ltte0HFKYIBKM4EJumG++2hqbUaLz4tl06BHaQPC\nv/9NiNY7q9R9c/B6s1YzHhwqkWht2a+AtgJ4BkpG+g+MmZMQV/Ao7RwLFKJ9OlMW\nLBmEXFcpIJN0HpPasT0nEl/MmotSu+8RnClAi3yFfyTKb+8rD7VxuyXetqDZ6dU/\n9/iqD/SZS7OQIjywtd343mACz3B1RlFxMHSA6dQAf2btGumqR0KiAp3KkYRAePoa\nJqYkB7Zad06ngFl0G0FHON+7\n-----END CERTIFICATE-----\n"
  },
  {
    "path": "tests/resources/crypto/cert_pgp.pem",
    "content": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nxsFNBGTGHwkBEADRB5EEtfsnUwgF2ZRg6h1fp2E8LNhv4lb9AWersI8KNFoWM6qx\nBk/MfEpgILSPdW3g7PWHOxPV/hxjtStFHfbU/Ye5VvfbkU49faIPiw1V3MQJJ171\ncN6kgMnABfdixNiutDkHP4f34ABrEqexX2myOP+btxL24gI/N9UpOD5PiKTyKR7i\nGwNpi+O022rs/KvjlWR7iSJ4vk7bGFfTNHvWI6dZworey1tZoTIZ0CgvgMeB/F1q\nOOa0FvrJdNYR227RpHmICqFqTptNZ2EfdkJ6QUXW7bZ9dWgL36ds9QPJOGcG3c5i\nJebeX5YdJnniBefiWjfZElcqh/N6SqVuEwoTLyMCnMZ6gjNMn6tddwPH24kavZhT\np6+vhTHmyq8XBqK/XEt9r+clSfg2hi5s7GO7hQV+W26xRjX7sQJY41PfzkgYJ0BM\n6+w09X1ZO/iMjEp44t2rd3xSudwGYhlbazXbdB+OJaa3RtyjOAeFgY8OyNlODx3V\nxXLtF+104HGSL7nkpBsu6LLighSgEEF2Vok43grr0omyb1NPhWoAZhM8sT5iv5gW\nfKvB1O13c+hDc/iGTAvcrtdLLnF2Cs+6HD7r7zPPM4L6DrD1+oQt510H/oOEE5NZ\nwIS9CmBf0txqwk7n1U5V95lonaCK9nfoKeQ1fKl/tu01dCeERRbMXG2nCQARAQAB\nzRtKb2huIERvZSA8am9obkBleGFtcGxlLm9yZz7CwYcEEwEIADEWIQQWwx1eM+Aa\no8okGzL45grMTSggxQUCZMYfCQIbAwQLCQgHBRUICQoLBRYCAwEAAAoJEPjmCsxN\nKCDFWP4QAI3eS5nPxmU0AC9/h8jeKNgjgpENroNQZKeWZQ8x4PfncDRkcbsJfT7Y\nIVZl4zw6gFKY5EoB1s1KkYJxPgYsqicmKNiR7Tnzabb3mzomU48FKaIyVCBzFUnJ\nYMroL/rm7QhoW2WWLvT+CPCPway/tA3By8Be/YOjhavJ8mf1W3rPzt87/4Vo6erf\nyzL0lN+FQmmhKfT4j42jF4SMSyyC2yzvfC7PT49u+KUKQm/LpQsfKHpwXZ/VI6+X\nGtZjTqsc+uglJYRo69oosImLzieA/ST1ltjmUutZQOSvlQFpDUEFrMej8XZ0qsrf\n0gP2iwxyl0vkhV8c6wO6CacDHPivvQEHed9H1PNGn3DBfKb7Mq/jado2DapRtJg3\n2OH0F0HTvQ0uNKl30xMUcwGQB0cKOlaFtksZT1LsosQPhtPLpFy1TuWaXOInpQLq\nJmNVcTbydOsCKq0mb6bgGcvhElC1q39tclKP3rOEDOnJ8hE6wYNaMGrt6WSKr3Tt\nh52M6KwTXOuMAecMvpDBSS3UFEVQ+T5puzInDTkjINxmj23ip+swA1x3HH2IgNrO\nVJ7O20oEf0+qC47R5rTRUxrvh/U0U3DRE5xt2J2T3xetFDT2mnQv0jcyMg/UlXXv\nGpGVfwNkvN0Cxmb1tFiBNLKCcPVizxq4MLrwx+MVfQBaRCwjJrUszsFNBGTGHwoB\nEACr5lA+j5pH0Er6Q76btbS4q9JgNjDNrjKJwX9brdBY1oXIUeBqCW9ekoqDTFpn\nxA5EFGJvPO++/0ZCa+zXE4IAcXS9+I9HVBouenPYBLETnXK0Phws+OCLoe0cAIvG\ne9Xo9VrHcGXCs9tJruVSAW3NF04YejHmnHNfEuD8mbaUdxVn5zc23w/2gLaY/ABL\nZfNV8XZw0jBVBm3YXS3Ob3uIO+RvsNqBgnhGYN/C51QI9hdxXWUDlD1vdRacXmcI\nLDCYC3w6u8caxL0ktXTS4zwN+hEu7jHxBNiKcovCeIF5VZ5NcPpp6+6Y+vNdmmXw\n+lWNwAzj3ah6iu+y25LKSsz+7IkCh5liOwwYohO+YI7SjtTD+gL9HiHYAIO+PtBh\n7GudmUwFoARu/q54hE4ThpzkeOzJzPqGkM/CzmwdKKM3u81ze+72ptJOqVKbFEsQ\n3+RURrIAfyYyeJj4VVCfHNzrRRVpARZc9hJm1AXefxPnDN9dxbikjQgbg5UxrKaJ\ncjVU+go5CH5lg2D1LRGfKqTJtfiWFPjtztNgMp/SeslkhhFXsyJ0RJDcU8VfRBrO\nDBnZvPnZi4nLaWCL1LdHA8Y9EJgSwVOsfdRqL/Xk9qxqgl5R8m8lsNKZN2EYkfMN\n4Vd+/8UBbmibHYoGIQi7UlNSPthc0XQcRzFen+3H4sg5kQARAQABwsF2BBgBCAAg\nFiEEFsMdXjPgGqPKJBsy+OYKzE0oIMUFAmTGHwsCGwwACgkQ+OYKzE0oIMXn4hAA\nlUWeF7tDdyENsOYyhsbtLIuLipYe6orHFY5m68NNOoLWwqEeTvutJgFeDT4WxYi0\nPJaNQYFPyGVyg7N0hCx5cGwajdnwGpb5zpSNyvG2Yes9I1O/u7+FFrbSwOuo61t1\nscGa8YlgTKoyGc9cwxl5U8krrlEwXTWQ/qF1Gq2wHG23wm1D2d2PXFDRvw3gPxJn\nyWkrx5k26ru1kguM7XFVyRi7B+uG4vdvMlxMBXM3jpH1CJRr82VvzYPv7f05Z5To\nC7XDqHpWKx3+AQvh/ZsSBpBhzK8qaixysMwnawe05rOPydWvsLlnMCGManKVnq9Y\nWek1P2dwYT9zuroBR5nmrECY+xVWk7vhsDasKsYlQ/LdDyzSL7qh0Vq3DjcoHxLI\nuL7qQ3O0YRcKGfmQibpKdDzvIqA+48Nfh2nDnTxvfuwOxb41zdLTZQftaSXc0Xwd\nHgquBAFbRDr5TyWlUUc8iACowKkk01pEPc8coxPCp6F/hz6kgmebRevzs7sxwrS7\naUWycSls783JC7WO267DRD30FNx+9S7SY4ECzhDGjLdne6wIoib1L9SFkk1AAKb3\nm2+6BB/HxCXtMqi95pFeCjV99bp+PBqoifx9SlFYZq9qcGDr/jyrdG8V2Wf/HF4n\nK8RIPxB+daAPMLTpj4WBhNquSE6mRQvABEf0GPi2eLA=\n=0TDv\n-----END PGP PUBLIC KEY BLOCK-----\n\n\n"
  },
  {
    "path": "tests/resources/crypto/cert_smime.pem",
    "content": "-----BEGIN CERTIFICATE-----\nMIIHbTCCBVWgAwIBAgIQFxA+3j2KHLXKBlGT58pDazANBgkqhkiG9w0BAQsFADBr\nMQswCQYDVQQGEwJJVDEOMAwGA1UEBwwFTWlsYW4xIzAhBgNVBAoMGkFjdGFsaXMg\nUy5wLkEuLzAzMzU4NTIwOTY3MScwJQYDVQQDDB5BY3RhbGlzIEF1dGhlbnRpY2F0\naW9uIFJvb3QgQ0EwHhcNMjAwNzA2MDg0NTQ3WhcNMzAwOTIyMTEyMjAyWjCBgTEL\nMAkGA1UEBhMCSVQxEDAOBgNVBAgMB0JlcmdhbW8xGTAXBgNVBAcMEFBvbnRlIFNh\nbiBQaWV0cm8xFzAVBgNVBAoMDkFjdGFsaXMgUy5wLkEuMSwwKgYDVQQDDCNBY3Rh\nbGlzIENsaWVudCBBdXRoZW50aWNhdGlvbiBDQSBHMzCCAiIwDQYJKoZIhvcNAQEB\nBQADggIPADCCAgoCggIBAO3mh5ahwaS27cJCVfc/Dw8iYF8T4KZDiIZJkXkcGy8a\nUA/cRgHu9ro6hsxRYe/ED4AIcSlarRh82HqtFSVQs4ZwikQW1V/icCIS91C2IVAG\na1YlKfedqgweqky+bBniUvRevVT0keZOqRTcO5hw007dL6FhYNmlZBt5IaJs1V6I\nniRjokOHR++qWgrUGy5LefY6ACs9gZ8Bi0OMK9PZ37pibeQCsdmMRytl4Ej7JVWe\nM/BtNIIprHwO1LY0/8InpGOmdG+5LC6xHLzg53B0HvVUqzUQNePUhNwJZFmmTP46\nFXovxmH4/SuY5IkXop0eJqjN+dxRHHizngYUk1EaTHUOcLFy4vQ0kxgbjb+GsNg6\nM2/6gZZIRk78JPdpotIwHnBNtkp9wPVH61NqdcP7kbPkyLXkNMTtAfydpmNnGqqH\nLEvUrK4iBpUPG9C09KOjm9OyhrT2uf5SLzJsee9g79r/rw4hAgcsZtR3YI6fCbRO\nJncmD+hgbHCck+9TWcNc1x5xZMgm8UXmoPamkkfceAlVV49QQ5jUTgqneTQHyF1F\n2ExXmf47pEIoJMVxloRIXywQuB2uqcIs8/X6tfsMDynFmhfT/0mTrgQ6xt9DIsgm\nWuuhvZhLReWS7oeKxnyqscuGeTMXnLs7fjGZq0inyhnlznhA/4rl+WdNjNaO4jEv\nAgMBAAGjggH0MIIB8DAPBgNVHRMBAf8EBTADAQH/MB8GA1UdIwQYMBaAFFLYiDrI\nn3hm7YnzezhwlMkCAjbQMEEGCCsGAQUFBwEBBDUwMzAxBggrBgEFBQcwAYYlaHR0\ncDovL29jc3AwNS5hY3RhbGlzLml0L1ZBL0FVVEgtUk9PVDBFBgNVHSAEPjA8MDoG\nBFUdIAAwMjAwBggrBgEFBQcCARYkaHR0cHM6Ly93d3cuYWN0YWxpcy5pdC9hcmVh\nLWRvd25sb2FkMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDBDCB4wYDVR0f\nBIHbMIHYMIGWoIGToIGQhoGNbGRhcDovL2xkYXAwNS5hY3RhbGlzLml0L2NuJTNk\nQWN0YWxpcyUyMEF1dGhlbnRpY2F0aW9uJTIwUm9vdCUyMENBLG8lM2RBY3RhbGlz\nJTIwUy5wLkEuJTJmMDMzNTg1MjA5NjcsYyUzZElUP2NlcnRpZmljYXRlUmV2b2Nh\ndGlvbkxpc3Q7YmluYXJ5MD2gO6A5hjdodHRwOi8vY3JsMDUuYWN0YWxpcy5pdC9S\nZXBvc2l0b3J5L0FVVEgtUk9PVC9nZXRMYXN0Q1JMMB0GA1UdDgQWBBS+l6mqhL+A\nvxBTfQky+eEuMhvPdzAOBgNVHQ8BAf8EBAMCAQYwDQYJKoZIhvcNAQELBQADggIB\nACab5xtZDXSzEgPp51X3hICFzULDO2EcV8em5hLfSCKxZR9amCnjcODVfMbaKfdU\nZXtevMIIZmHgkz9dBan7ijGbJXjZCPP29zwZGSyCjpfadg5s9hnNCN1r3DGwIHfy\nLgbcfffDyV/2wW+XTGbhldnazZsX892q+srRmC8XnX4ygg+eWL/AkHDenvbFuTlJ\nvUyd5I7e1nb3dYXMObPu24ZTQ9/K1hSQbs7pqecaptTUjoIDpBUpSp4Us+h1I4MA\nWonemKYoPS9f0y65JrRCKcfsKSI+1kwPSanDDMiydKzeo46XrS0hlA5NzQjqUJ7U\nsuGvPtDvknqc0v03nNXBnUjejYtvwO3sEDXdUW5m9kjNqlQZXzdHumZJVqPUGKTW\ncn9Hf3d7qbCmmxPXjQoNUuHg56fLCanZWkEO4SP1GAgIA7SyJu/yffv0ts7sBFrS\nTD3L2mCAXM3Y8BfblvvDSf2bvySm/fPe9brmuzrCXsTxUQc1+/z5ydvzV3E3cLnU\noSXP6XfXNyEVO6sPkcUSnISHM798xLkCTB5EkjPCjPE2zs4v9L9JVOkkskvW6RnW\nWccdfR3fELNHL/kep8re6IbbYs8Hn5GM0Ohs8CMDPYEox+QX/6/SnOfyaqqSilBo\nnMQBstsymBBgdEKO+tTHHCMnJQVvZn7jRQ20wXgxMrvN\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIFuzCCA6OgAwIBAgIIVwoRl0LE48wwDQYJKoZIhvcNAQELBQAwazELMAkGA1UE\nBhMCSVQxDjAMBgNVBAcMBU1pbGFuMSMwIQYDVQQKDBpBY3RhbGlzIFMucC5BLi8w\nMzM1ODUyMDk2NzEnMCUGA1UEAwweQWN0YWxpcyBBdXRoZW50aWNhdGlvbiBSb290\nIENBMB4XDTExMDkyMjExMjIwMloXDTMwMDkyMjExMjIwMlowazELMAkGA1UEBhMC\nSVQxDjAMBgNVBAcMBU1pbGFuMSMwIQYDVQQKDBpBY3RhbGlzIFMucC5BLi8wMzM1\nODUyMDk2NzEnMCUGA1UEAwweQWN0YWxpcyBBdXRoZW50aWNhdGlvbiBSb290IENB\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAp8bEpSmkLO/lGMWwUKNv\nUTufClrJwkg4CsIcoBh/kbWHuUA/3R1oHwiD1S0eiKD4j1aPbZkCkpAW1V8IbInX\n4ay8IMKx4INRimlNAJZaby/ARH6jDuSRzVju3PvHHkVH3Se5CAGfpiEd9UEtL0z9\nKK3giq0itFZljoZUj5NDKd45RnijMCO6zfB9E1fAXdKDa0hMxKufgFpbOr3JpyI/\ngCczWw63igxdBzcIy2zSekciRDXFzMwujt0q7bd9Zg1fYVEiVRvjRuPjPdA1Yprb\nrxTIW6HMiRvhMCb8oJsfgadHHwTrozmSBp+Z07/T6k9QnBn+locePGX2oxgkg4YQ\n51Q+qDp2JE+BIcXjDwL4k5RHILv+1A7TaLndxHqEguNTVHnd25zS8gebLra8Pu2F\nbe8lEfKXGkJh90qX6IuxEAf6ZYGyojnP9zz/GPvG8VqLWeICrHuS0E4UT1lF9gxe\nKF+w6D9Fz8+vm2/7hNN3WpVvrJSEnu68wEqPSpP4RCHiMUVhUE4Q2OM1fEwZtN4F\nv6MGn8i1zeQf1xcGDXqVdFUNaBr8EBtiZJ1t4JWgw5QHVw0U5r0F+7if5t+L4sbn\nfpb2U8WANFAoWPASUHEXMLrmeGO89LKtmyuy/uE5jF66CyCU3nuDuP/jVo23Eek7\njPKxwV2dpAtMK9myGPW1n0sCAwEAAaNjMGEwHQYDVR0OBBYEFFLYiDrIn3hm7Ynz\nezhwlMkCAjbQMA8GA1UdEwEB/wQFMAMBAf8wHwYDVR0jBBgwFoAUUtiIOsifeGbt\nifN7OHCUyQICNtAwDgYDVR0PAQH/BAQDAgEGMA0GCSqGSIb3DQEBCwUAA4ICAQAL\ne3KHwGCmSUyIWOYdiPcUZEim2FgKDk8TNd81HdTtBjHIgT5q1d07GjLukD0R0i70\njsNjLiNmsGe+b7bAEzlgqqI0JZN1Ut6nna0Oh4lScWoWPBkdg/iaKWW+9D+a2fDz\nWochcYBNy+A4mz+7+uAwTc+G02UQGRjRlwKxK3JCaKygvU5a2hi/a5iB0P2avl4V\nSM0RFbnAKVy06Ij3Pjaut2L9HmLecHgQHEhb2rykOLpn7VU+Xlff1ANATIGk0k9j\npwlCCRT8AKnCgHNPLsBA2RF7SOp6AsDT6ygBJlh0wcBzIm2Tlf05fbsq4/aC4yyX\nX04fkZT6/iyj2HYauE2yOE+b+h1IYHkm4vP9qdCa6HCPSXrW5b0KDtst842/6+Ok\nfcvHlXHo2qN8xcL4dJIEG4aspCJTQLas/kx2z/uUMsA1n3Y/buWQbqCmJqK4LL7R\nK4X9p2jIugErsWx0Hbhzlefut8cl8ABMALJ+tguLHPPAUJ4lueAI3jZm/zel0btU\nZCzJJ7VLkn5l/9Mt4blOvH+kQSGQQXemOR/qnuOf0GZvBeyqdn6/axag67XH/JJU\nLysRJyU3eExRarDzzFhdFPFqSBX/wge2sY0PjlxQRrM9vwGYT7JZVEc+NHt4bVaT\nLnPqZih4zR0Uv6CPLy64Lo7yFIrM6bV8+2ydDKXhlg==\n-----END CERTIFICATE-----\n\n\n-----BEGIN CERTIFICATE-----\nMIIDbjCCAlagAwIBAgIUZ4K0WXNSS8H0cUcZavD9EYqqTAswDQYJKoZIhvcNAQEN\nBQAwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0\neTAgFw0xOTExMjAwNjU0MThaGA8yMDUyMDkyNzA2NTQxOFowGTEXMBUGA1UEAxMO\nQWxpY2UgTG92ZWxhY2UwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDD\n7q35ZdG2JAzzJGNZDZ9sV7AKh0hlRfoFjTZN5m4RegQAYSyag43ouWi1xRN0avf0\nUTYrwjK04qRdV7GzCACoEKq/xiNUOsjfJXzbCublN3fZMOXDshKKBqThlK75SjA9\nCzxg7ejGoiY/iidk0e91neK30SCCaBTJlfR2ZDrPk73IPMeksxoTatfF9hw9dDA+\n/Hi1yptN/aG0Q/s9icFrxr6y2zQXsjuQPmjMZgj10aD9cazWVgRYCgflhmA0V1uQ\nl1wobYU8DAVxVn+GgabqyjGQMoythIK0Gn5+ofwxXXUM/zbU+g6+1ISdoXxRRFtq\n2GzbIqkAHZZQm+BbnFrhAgMBAAGjgZcwgZQwDAYDVR0TAQH/BAIwADAeBgNVHREE\nFzAVgRNhbGljZUBzbWltZS5leGFtcGxlMBMGA1UdJQQMMAoGCCsGAQUFBwMEMA8G\nA1UdDwEB/wQFAwMHoAAwHQYDVR0OBBYEFKwuVFqk/VUYry7oZkQ40SXR1wB5MB8G\nA1UdIwQYMBaAFLdSTXPAiD2yw3paDPOU9/eAonfbMA0GCSqGSIb3DQEBDQUAA4IB\nAQB76o4Yz7yrVSFcpXqLrcGtdI4q93aKCXECCCzNQLp4yesh6brqaZHNJtwYcJ5T\nqbUym9hJ70iJE4jGNN+yAZR1ltte0HFKYIBKM4EJumG++2hqbUaLz4tl06BHaQPC\nv/9NiNY7q9R9c/B6s1YzHhwqkWht2a+AtgJ4BkpG+g+MmZMQV/Ao7RwLFKJ9OlMW\nLBmEXFcpIJN0HpPasT0nEl/MmotSu+8RnClAi3yFfyTKb+8rD7VxuyXetqDZ6dU/\n9/iqD/SZS7OQIjywtd343mACz3B1RlFxMHSA6dQAf2btGumqR0KiAp3KkYRAePoa\nJqYkB7Zad06ngFl0G0FHON+7\n-----END CERTIFICATE-----\n"
  },
  {
    "path": "tests/resources/crypto/is_encrypted.txt",
    "content": "Subject: TRUE\nContent-Type: multipart/encrypted;\n        protocol=\"application/pgp-encrypted\";\n        boundary=\"17778885806d6b28_46555a54adda6fa5_d83705f326a6b951\"\n\nbody\n!!!\nSubject: FALSE\nContent-Type: multipart/signed;\n        protocol=\"application/pkcs7-signature\";\n        boundary=\"17778885806d6b28_46555a54adda6fa5_d83705f326a6b951\"\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/pkcs7-mime;\n        name=\"smime.p7m\";\n        smime-type=enveloped-data\nContent-Disposition: attachment;\n        filename=\"smime.p7m\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/pkcs7-signature;\n        name=\"smime.p7s\"; smime-type=\"signed-data\"\nContent-Disposition: attachment;\n        filename=\"smime.p7s\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/octet-stream;\n        name=\"smime.p7m\"\nContent-Disposition: attachment;\n        filename=\"smime.p7m\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/octet-stream;\n        name=\"smime.p7m\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/octet-stream\nContent-Disposition: attachment;\n        filename=\"smime.p7m\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/octet-stream\nContent-Disposition: attachment;\n        filename=\"smime.p7s\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/octet-stream\nContent-Disposition: attachment;\n        filename=\"smime.p7c\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: TRUE\nContent-Type: application/octet-stream\nContent-Disposition: attachment;\n        filename=\"smime.p7z\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: FALSE\nContent-Type: application/octet-stream\nContent-Disposition: attachment;\n        filename=\"file.gz\"\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: FALSE\nContent-Type: application/octet-stream\nContent-Disposition: attachment\nContent-Transfer-Encoding: base64\n\nbody\n!!!\nSubject: FALSE\nContent-Type: multipart/mixed;\n        boundary=\"17778885806d6b28_46555a54adda6fa5_d83705f326a6b951\"\n\nbody\n!!!\nSubject: TRUE\nContent-Type: multipart/mixed; boundary=\"----sinikael-?=_1-17364333937490.28304631087789\"\n\n------sinikael-?=_1-17364333937490.28304631087789\nContent-Type: text/plain\nContent-Transfer-Encoding: quoted-printable\n\n\n-----BEGIN PGP MESSAGE-----\nVersion: Mailvelope v6.0.1\nComment: https://mailvelope.com\n\nwcFMAy3n9oegk2NSARAApgl4nuztm5uxuCe+7ISW9Ox+315/pCHZmOUibORX\nxGEbUs+ZgTqKIdYMet3+zSS70ykgjZDvh+cLm1oGmD5ZzLxSMjhgLpqrA0tG\n3GB3gus844irLGBjVwgTuwAKhRz/CYaqlKGOloQkvcwRwy3N8UY8KG/WqD63\ng9WQKYc+8+cR4wfQ0PSzZL6HexdT2LE2mWgKxStwDbbkwZumR0qZy4OvyI6O\nzFCZSaz3F3VnDGbQgo5hK8Iynf25sC54P/E0ykdG6QpJZPo/ASoWePv0XyT6\nwRiqq2cx366P6lxB3CJyjn+qRCfcXz0XXYejHmuI++S6ucNS4JcIhpuoi+tv\n8b7pSy/6yQEKLyUhPfUmvFLxBm10bu8E9uhsmZnpiSnBsEGkLm/l9TFbOOSd\n9nK0NeEVfqfIi25bxVJtRcYryf7BpjmgEdKlmF8Efeb1RpaI8IwVc6E7AhYB\nc1vz1gN6BQ5ePtlHDFKxFV3jIqBKck8h2GSFGEYOXyTT+3tgFkqv/SuAUed9\nX0bCjy7v578nsNieoyy/133X1+mu7L8rdADEAHhU/WLQ9VY/xu6GhNA0vzU8\nyVQfSEJ1DJ4awU9Cn1mrbVg3vD/jpQzs67Sd33gC8xMqA9ZFOmXrnR4V1wkz\nY8XobC+UY6o9qn/Oz5mEtZG8rItYJ/bPOSSlFP6kXVrBwUwD8GquMWN8AUEB\nD/9IVPN/qCtuhF+9jIjrxiw+agmeqUPWBEmLLZYL6xivP7Bn5bOJb6wwgU4e\nIRIoh3Q5AKLvqN26BPePM16yZ6Z6qxTP/CFT5mOQQbj+3TH6fYD1ui74FoyR\n+szTfntIdUOShKqogSKoRytMXheR+4TW7/D525ohRUsz71pfdCxqYC+k/vXr\nffl7lkv/V7GkPJ5fT+CantgLqXVevCIKOMVDVBT6G92t8Zn0klqv+MfVsZ3V\n6FbwcqS+GxsCNezSdT1+K2yPja0fZffQOHD6nLDyIZmcr8KrCmP1MPY5LKJw\nbMVCbYz0CrRl9d6JJGZOq+yapVOc0CDxWmxhGJP3M9OiCvGoPz3l+fDdNJ0f\ndmxEv8c/I5t+AELxLMBa4aPuGE3pZQj4zyzhC3gnK5n9bqFecHl2+Kz+aVsi\n4B1xZXIPT+XmlXM4rgCtuMlKZ6YkGRWiAnC8jok/k4iPaN7mfKbos0xwa187\nhaMjKKFVP8YvXlYpGeuvP3C2/oBxwsw9WihqL2tLTEE60ZgDEKWcZ3zwVKjl\nbSNeDsWXlj+vIletdDyH/AMhyDOnUDc4bTkRW7W8ntq3fi8XmpQjHzW02BEd\nWq1Y/bV7nlh97Y7d3z5zlLhQnKrrP5IyQdOkd2WGpU39cDZe6XE6arQZKGVT\nupnuX3Q48Hm6oSga+cydngjShtLCFQF/MFCZVYydjhe+KtzZCYBZ8nCMRxcV\nPTuACJtUqOR/z5aPgRjM14accCYDwJ0Dm51h5LuEzYFDJjcdFbVSVfvnCy5a\nb9h6VBCpWieI6ubAzYKKJxGwp/OcoaQyYPjwlNQyTSb7omZPsAtFpTALNXbU\nCIM2cFnD/MMvgEHCuqPZoKh0YdQxYV50p36Wa/rMvTfW+S1T2neWGD6kMDWZ\nI651K+HBCQV6INrJk5ugvCt26/l8+Y75cnGEKbkyJgt+NGjULX3jjPawhvwN\n0EKx8hYwASuJ3E71iAXga7oSMRemDk4ZXoz+PDngoWtcoXcdQNMiVymOt6iZ\nP5ImSQXkId1XZYjS3+1t+Sk8WgUi5kvu4WP3hm1Ovjd1kXTy3lqXpfci23si\neZofjs9DLldkV5xtOwWVx/B1UkqvtD2BuwcDWgvGEm2LEn4/eXkHUTzHLs4j\nT9s454HasmVfd+4/koBdD6ASDqFf5Ue14OonsXvIvLop2CgUHxDVffr1onSk\nK9r0kKxe9iAFZ8KYpf2Do/FPTL8BIXjHC2vfJ05wl262xs7uNMdgWm4+NP/q\nuVNOoXByZIDEPixx7jimKhlEIZx36AbsI+OB5cV1Pr5SSbO3w205ALol+xKN\nTPJiaQDH8qNXX+9j4JfO8uWvDmhPUe83bm/mHg4wf/39+PNQptwmhLH9iXtC\nCzqgmr1vUNd/8dLgQuqVn/R3uUbGeWmKD9cqURWcvumbq+rFWPl1N7T7VRfc\nXGDnMBNBmLke2YO8XUY561lqKekfS/2fekSdz005sI2cnGmy5LLh+kPN6opz\n0pA2kSskbfSQqJmEdjKQU3oGEzEgwvOXjFwT7cChevgLEmDBkQXTDuEeo3Kb\nKnwaNPnodzaNXAfZh/NylEgcUFjEx6crZMN3ztl8zsvAvP4P0BPmixZ89G80\n5fkw+PwLe2vPtuObvuY+ezbJGb1jV0tWZFXF\n=3DaQyM\n-----END PGP MESSAGE-----\n\n------sinikael-?=_1-17364333937490.28304631087789--\n!!!\nSubject: TRUE\nContent-Type: text/plain\nContent-Transfer-Encoding: quoted-printable\n\n\n\n-----BEGIN PGP MESSAGE-----\nVersion: Mailvelope v6.0.1\nComment: https://mailvelope.com\n\nwcFMAy3n9oegk2NSARAApgl4nuztm5uxuCe+7ISW9Ox+315/pCHZmOUibORX\nxGEbUs+ZgTqKIdYMet3+zSS70ykgjZDvh+cLm1oGmD5ZzLxSMjhgLpqrA0tG\n3GB3gus844irLGBjVwgTuwAKhRz/CYaqlKGOloQkvcwRwy3N8UY8KG/WqD63\ng9WQKYc+8+cR4wfQ0PSzZL6HexdT2LE2mWgKxStwDbbkwZumR0qZy4OvyI6O\nzFCZSaz3F3VnDGbQgo5hK8Iynf25sC54P/E0ykdG6QpJZPo/ASoWePv0XyT6\nwRiqq2cx366P6lxB3CJyjn+qRCfcXz0XXYejHmuI++S6ucNS4JcIhpuoi+tv\n8b7pSy/6yQEKLyUhPfUmvFLxBm10bu8E9uhsmZnpiSnBsEGkLm/l9TFbOOSd\n9nK0NeEVfqfIi25bxVJtRcYryf7BpjmgEdKlmF8Efeb1RpaI8IwVc6E7AhYB\nc1vz1gN6BQ5ePtlHDFKxFV3jIqBKck8h2GSFGEYOXyTT+3tgFkqv/SuAUed9\nX0bCjy7v578nsNieoyy/133X1+mu7L8rdADEAHhU/WLQ9VY/xu6GhNA0vzU8\nyVQfSEJ1DJ4awU9Cn1mrbVg3vD/jpQzs67Sd33gC8xMqA9ZFOmXrnR4V1wkz\nY8XobC+UY6o9qn/Oz5mEtZG8rItYJ/bPOSSlFP6kXVrBwUwD8GquMWN8AUEB\nD/9IVPN/qCtuhF+9jIjrxiw+agmeqUPWBEmLLZYL6xivP7Bn5bOJb6wwgU4e\nIRIoh3Q5AKLvqN26BPePM16yZ6Z6qxTP/CFT5mOQQbj+3TH6fYD1ui74FoyR\n+szTfntIdUOShKqogSKoRytMXheR+4TW7/D525ohRUsz71pfdCxqYC+k/vXr\nffl7lkv/V7GkPJ5fT+CantgLqXVevCIKOMVDVBT6G92t8Zn0klqv+MfVsZ3V\n6FbwcqS+GxsCNezSdT1+K2yPja0fZffQOHD6nLDyIZmcr8KrCmP1MPY5LKJw\nbMVCbYz0CrRl9d6JJGZOq+yapVOc0CDxWmxhGJP3M9OiCvGoPz3l+fDdNJ0f\ndmxEv8c/I5t+AELxLMBa4aPuGE3pZQj4zyzhC3gnK5n9bqFecHl2+Kz+aVsi\n4B1xZXIPT+XmlXM4rgCtuMlKZ6YkGRWiAnC8jok/k4iPaN7mfKbos0xwa187\nhaMjKKFVP8YvXlYpGeuvP3C2/oBxwsw9WihqL2tLTEE60ZgDEKWcZ3zwVKjl\nbSNeDsWXlj+vIletdDyH/AMhyDOnUDc4bTkRW7W8ntq3fi8XmpQjHzW02BEd\nWq1Y/bV7nlh97Y7d3z5zlLhQnKrrP5IyQdOkd2WGpU39cDZe6XE6arQZKGVT\nupnuX3Q48Hm6oSga+cydngjShtLCFQF/MFCZVYydjhe+KtzZCYBZ8nCMRxcV\nPTuACJtUqOR/z5aPgRjM14accCYDwJ0Dm51h5LuEzYFDJjcdFbVSVfvnCy5a\nb9h6VBCpWieI6ubAzYKKJxGwp/OcoaQyYPjwlNQyTSb7omZPsAtFpTALNXbU\nCIM2cFnD/MMvgEHCuqPZoKh0YdQxYV50p36Wa/rMvTfW+S1T2neWGD6kMDWZ\nI651K+HBCQV6INrJk5ugvCt26/l8+Y75cnGEKbkyJgt+NGjULX3jjPawhvwN\n0EKx8hYwASuJ3E71iAXga7oSMRemDk4ZXoz+PDngoWtcoXcdQNMiVymOt6iZ\nP5ImSQXkId1XZYjS3+1t+Sk8WgUi5kvu4WP3hm1Ovjd1kXTy3lqXpfci23si\neZofjs9DLldkV5xtOwWVx/B1UkqvtD2BuwcDWgvGEm2LEn4/eXkHUTzHLs4j\nT9s454HasmVfd+4/koBdD6ASDqFf5Ue14OonsXvIvLop2CgUHxDVffr1onSk\nK9r0kKxe9iAFZ8KYpf2Do/FPTL8BIXjHC2vfJ05wl262xs7uNMdgWm4+NP/q\nuVNOoXByZIDEPixx7jimKhlEIZx36AbsI+OB5cV1Pr5SSbO3w205ALol+xKN\nTPJiaQDH8qNXX+9j4JfO8uWvDmhPUe83bm/mHg4wf/39+PNQptwmhLH9iXtC\nCzqgmr1vUNd/8dLgQuqVn/R3uUbGeWmKD9cqURWcvumbq+rFWPl1N7T7VRfc\nXGDnMBNBmLke2YO8XUY561lqKekfS/2fekSdz005sI2cnGmy5LLh+kPN6opz\n0pA2kSskbfSQqJmEdjKQU3oGEzEgwvOXjFwT7cChevgLEmDBkQXTDuEeo3Kb\nKnwaNPnodzaNXAfZh/NylEgcUFjEx6crZMN3ztl8zsvAvP4P0BPmixZ89G80\n5fkw+PwLe2vPtuObvuY+ezbJGb1jV0tWZFXF\n=3DaQyM\n-----END PGP MESSAGE-----\n"
  },
  {
    "path": "tests/resources/imap/000.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 262 7\n   )(\n      \"text\" \"plain\" (\n         \"charset\" \"US-ASCII\"\n      ) NIL NIL \"7bit\" 111 3\n   )(\n      (\n         \"text\" \"basic\" (\n            \"charset\" \"us-ascii\"\n         ) NIL NIL \"base64\" 85 2\n      )(\n         \"text\" \"jpeg\" (\n            \"charset\" \"us-ascii\"\n         ) NIL NIL \"base64\" 44 1\n      ) \"parallel\"\n   )(\n      \"text\" \"enriched\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 140 5\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL NIL 223 (\n         NIL \"(subject in US-ASCII)\" (\n            (\n               NIL NIL \"unknown\" \"localhost\"\n            )\n         ) (\n            (\n               NIL NIL \"unknown\" \"localhost\"\n            )\n         ) (\n            (\n               NIL NIL \"unknown\" \"localhost\"\n            )\n         ) NIL NIL NIL NIL NIL\n      ) (\n         \"text\" \"plain\" (\n            \"charset\" \"ISO-8859-1\"\n         ) NIL NIL \"Quoted-printable\" 48 1\n      ) 0\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 262 7 \"1ba9f4410aca245ffe18870d1767eaf1\" NIL NIL NIL\n   )(\n      \"text\" \"plain\" (\n         \"charset\" \"US-ASCII\"\n      ) NIL NIL \"7bit\" 111 3 \"457a84e4830817334703ecead7a71bdf\" NIL NIL NIL\n   )(\n      (\n         \"text\" \"basic\" (\n            \"charset\" \"us-ascii\"\n         ) NIL NIL \"base64\" 85 2 \"c6e3404e683fdeb88e96dfed0abbddad\" NIL NIL NIL\n      )(\n         \"text\" \"jpeg\" (\n            \"charset\" \"us-ascii\"\n         ) NIL NIL \"base64\" 44 1 \"88217c7613d6252c87ba0cd279f1e93c\" NIL NIL NIL\n      ) \"parallel\" (\n         \"boundary\" \"unique-boundary-2\"\n      ) NIL NIL NIL\n   )(\n      \"text\" \"enriched\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 140 5 \"4a5cd280a875e791bd6d767dcf60c43d\" NIL NIL NIL\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL NIL 223 (\n         NIL \"(subject in US-ASCII)\" (\n            (\n               NIL NIL \"unknown\" \"localhost\"\n            )\n         ) (\n            (\n               NIL NIL \"unknown\" \"localhost\"\n            )\n         ) (\n            (\n               NIL NIL \"unknown\" \"localhost\"\n            )\n         ) NIL NIL NIL NIL NIL\n      ) (\n         \"text\" \"plain\" (\n            \"charset\" \"ISO-8859-1\"\n         ) NIL NIL \"Quoted-printable\" 48 1 \"632b0aae19d3c55d420c0dc8cebaf049\" NIL NIL NIL\n      ) 0 \"40ceb8762dcb7270c37f1395e91aa893\" NIL NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"unique-boundary-1\"\n   ) NIL NIL NIL\n)\n\nBODY[] {1854}\r\nMIME-Version: 1.0\nFrom: Nathaniel Borenstein <nsb@nsb.fv.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Fri, 07 Oct 1994 16:15:05 -0700 (PDT)\nSubject: A multipart example\nContent-Type: multipart/mixed;\n            boundary=unique-boundary-1\n\nThis is the preamble area of a multipart message.\nMail readers that understand multipart format\nshould ignore this preamble.\n\nIf you are reading this text, you might want to\nconsider changing to a mail reader that understands\nhow to properly display multipart messages.\n\n--unique-boundary-1\n\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\n--unique-boundary-1\nContent-type: text/plain; charset=US-ASCII\n\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\n--unique-boundary-1\nContent-Type: multipart/parallel; boundary=unique-boundary-2\n\n--unique-boundary-2\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n--unique-boundary-2\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\n... base64-encoded image data goes here ...\n\n--unique-boundary-2--\n\n--unique-boundary-1\nContent-type: text/enriched\n\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\n--unique-boundary-1\nContent-Type: message/rfc822\n\nFrom: (mailbox in US-ASCII)\nTo: (address in US-ASCII)\nSubject: (subject in US-ASCII)\nContent-Type: Text/plain; charset=ISO-8859-1\nContent-Transfer-Encoding: Quoted-printable\n\n... Additional text in ISO-8859-1 goes here ...\n\n--unique-boundary-1--\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 1854\n----------------------------------\nBODY[HEADER] {239}\r\nMIME-Version: 1.0\nFrom: Nathaniel Borenstein <nsb@nsb.fv.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Fri, 07 Oct 1994 16:15:05 -0700 (PDT)\nSubject: A multipart example\nContent-Type: multipart/mixed;\n            boundary=unique-boundary-1\n\n\n----------------------------------\nBODY[TEXT] {1615}\r\nThis is the preamble area of a multipart message.\nMail readers that understand multipart format\nshould ignore this preamble.\n\nIf you are reading this text, you might want to\nconsider changing to a mail reader that understands\nhow to properly display multipart messages.\n\n--unique-boundary-1\n\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\n--unique-boundary-1\nContent-type: text/plain; charset=US-ASCII\n\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\n--unique-boundary-1\nContent-Type: multipart/parallel; boundary=unique-boundary-2\n\n--unique-boundary-2\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n--unique-boundary-2\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\n... base64-encoded image data goes here ...\n\n--unique-boundary-2--\n\n--unique-boundary-1\nContent-type: text/enriched\n\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\n--unique-boundary-1\nContent-Type: message/rfc822\n\nFrom: (mailbox in US-ASCII)\nTo: (address in US-ASCII)\nSubject: (subject in US-ASCII)\nContent-Type: Text/plain; charset=ISO-8859-1\nContent-Transfer-Encoding: Quoted-printable\n\n... Additional text in ISO-8859-1 goes here ...\n\n--unique-boundary-1--\n\n\n----------------------------------\nBODY[MIME] {72}\r\nContent-Type: multipart/mixed;\n            boundary=unique-boundary-1\n\r\n\n----------------------------------\nBODY[1] {262}\r\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\nBINARY[1] {262}\r\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\nBINARY.SIZE[1] 262\n----------------------------------\nBODY[1.HEADER] {1}\r\n\n\n----------------------------------\nBODY[1.TEXT] {262}\r\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\n----------------------------------\nBODY[1.MIME] {2}\r\n\r\n\n----------------------------------\nBODY[1.1] {262}\r\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\nBINARY[1.1] {262}\r\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\nBINARY.SIZE[1.1] 262\n----------------------------------\nBODY[2] {111}\r\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\nBINARY[2] {111}\r\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\nBINARY.SIZE[2] 111\n----------------------------------\nBODY[2.HEADER] {44}\r\nContent-type: text/plain; charset=US-ASCII\n\n\n----------------------------------\nBODY[2.TEXT] {111}\r\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\n----------------------------------\nBODY[2.MIME] {45}\r\nContent-Type: text/plain; charset=US-ASCII\n\r\n\n----------------------------------\nBODY[2.1] {111}\r\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\nBINARY[2.1] {111}\r\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\nBINARY.SIZE[2.1] 111\n----------------------------------\nBODY[3] {314}\r\n--unique-boundary-2\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n--unique-boundary-2\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\n... base64-encoded image data goes here ...\n\n--unique-boundary-2--\n\nBINARY[3] {16}\r\n[binary content]\nBINARY.SIZE[3] 376\n----------------------------------\nBODY[3.HEADER] {62}\r\nContent-Type: multipart/parallel; boundary=unique-boundary-2\n\n\n----------------------------------\nBODY[3.TEXT] {314}\r\n--unique-boundary-2\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n--unique-boundary-2\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\n... base64-encoded image data goes here ...\n\n--unique-boundary-2--\n\n----------------------------------\nBODY[3.MIME] {63}\r\nContent-Type: multipart/parallel; boundary=unique-boundary-2\n\r\n\n----------------------------------\nBODY[3.1] {85}\r\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n* NO [UNKNOWN-CTE] Failed to decode part 3.1 of message 0.\r\n\nBINARY.SIZE[3.1] 85\n----------------------------------\nBODY[3.1.HEADER] {61}\r\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\n\n----------------------------------\nBODY[3.1.TEXT] {85}\r\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n----------------------------------\nBODY[3.1.MIME] {62}\r\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\r\n\n----------------------------------\nBODY[3.1.1] {85}\r\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n* NO [UNKNOWN-CTE] Failed to decode part 3.1.1 of message 0.\r\n\nBINARY.SIZE[3.1.1] 85\n----------------------------------\nBODY[3.2] {44}\r\n... base64-encoded image data goes here ...\n\n* NO [UNKNOWN-CTE] Failed to decode part 3.2 of message 0.\r\n\nBINARY.SIZE[3.2] 44\n----------------------------------\nBODY[3.2.HEADER] {60}\r\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\n\n----------------------------------\nBODY[3.2.TEXT] {44}\r\n... base64-encoded image data goes here ...\n\n----------------------------------\nBODY[3.2.MIME] {61}\r\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\r\n\n----------------------------------\nBODY[3.2.1] {44}\r\n... base64-encoded image data goes here ...\n\n* NO [UNKNOWN-CTE] Failed to decode part 3.2.1 of message 0.\r\n\nBINARY.SIZE[3.2.1] 44\n----------------------------------\nBODY[4] {140}\r\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\nBINARY[4] {140}\r\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\nBINARY.SIZE[4] 140\n----------------------------------\nBODY[4.HEADER] {29}\r\nContent-type: text/enriched\n\n\n----------------------------------\nBODY[4.TEXT] {140}\r\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\n----------------------------------\nBODY[4.MIME] {30}\r\nContent-Type: text/enriched\n\r\n\n----------------------------------\nBODY[4.1] {140}\r\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\nBINARY[4.1] {140}\r\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\nBINARY.SIZE[4.1] 140\n----------------------------------\nBODY[5] {223}\r\nFrom: (mailbox in US-ASCII)\nTo: (address in US-ASCII)\nSubject: (subject in US-ASCII)\nContent-Type: Text/plain; charset=ISO-8859-1\nContent-Transfer-Encoding: Quoted-printable\n\n... Additional text in ISO-8859-1 goes here ...\n\nBINARY[5] {16}\r\n[binary content]\nBINARY.SIZE[5] 223\n----------------------------------\nBODY[5.HEADER] {175}\r\nFrom: (mailbox in US-ASCII)\nTo: (address in US-ASCII)\nSubject: (subject in US-ASCII)\nContent-Type: Text/plain; charset=ISO-8859-1\nContent-Transfer-Encoding: Quoted-printable\n\n\n----------------------------------\nBODY[5.TEXT] {48}\r\n... Additional text in ISO-8859-1 goes here ...\n\n----------------------------------\nBODY[5.MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[5.1] {48}\r\n... Additional text in ISO-8859-1 goes here ...\n\nBINARY[5.1] {48}\r\n... Additional text in ISO-8859-1 goes here ...\n\nBINARY.SIZE[5.1] 48\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {79}\r\nFrom: Nathaniel Borenstein <nsb@nsb.fv.com>\nTo: Ned Freed <ned@innosoft.com>\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\naniel Borenstein <nsb@nsb\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {211}\r\nMIME-Version: 1.0\nFrom: Nathaniel Borenstein <nsb@nsb.fv.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Fri, 07 Oct 1994 16:15:05 -0700 (PDT)\nContent-Type: multipart/mixed;\n            boundary=unique-boundary-1\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\non: 1.0\nFrom: Nathaniel B\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/000.txt",
    "content": "MIME-Version: 1.0\nFrom: Nathaniel Borenstein <nsb@nsb.fv.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Fri, 07 Oct 1994 16:15:05 -0700 (PDT)\nSubject: A multipart example\nContent-Type: multipart/mixed;\n            boundary=unique-boundary-1\n\nThis is the preamble area of a multipart message.\nMail readers that understand multipart format\nshould ignore this preamble.\n\nIf you are reading this text, you might want to\nconsider changing to a mail reader that understands\nhow to properly display multipart messages.\n\n--unique-boundary-1\n\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\n--unique-boundary-1\nContent-type: text/plain; charset=US-ASCII\n\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\n--unique-boundary-1\nContent-Type: multipart/parallel; boundary=unique-boundary-2\n\n--unique-boundary-2\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n--unique-boundary-2\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\n... base64-encoded image data goes here ...\n\n--unique-boundary-2--\n\n--unique-boundary-1\nContent-type: text/enriched\n\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\n--unique-boundary-1\nContent-Type: message/rfc822\n\nFrom: (mailbox in US-ASCII)\nTo: (address in US-ASCII)\nSubject: (subject in US-ASCII)\nContent-Type: Text/plain; charset=ISO-8859-1\nContent-Transfer-Encoding: Quoted-printable\n\n... Additional text in ISO-8859-1 goes here ...\n\n--unique-boundary-1--\n\n"
  },
  {
    "path": "tests/resources/imap/001.imap",
    "content": "BODY (\n   (\n      \"message\" \"external-body\" (\n         \"name\" \"BodyFormats.ps\" \"site\" \"thumper.bellcore.com\" \"mode\" \"image\" \"access-type\" \"ANON-FTP\" \"directory\" \"pub\" \"expiration\" \"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n      ) NIL NIL NIL 79\n   )(\n      \"message\" \"external-body\" (\n         \"access-type\" \"local-file\" \"name\" \"/u/nsb/writing/rfcs/RFC-MIME.ps\" \"site\" \"thumper.bellcore.com\" \"expiration\" \"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n      ) NIL NIL NIL 79\n   )(\n      \"message\" \"external-body\" (\n         \"access-type\" \"mail-server\" \"server\" \"listserv@bogus.bitnet\" \"expiration\" \"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n      ) NIL NIL NIL 97\n   ) \"alternative\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"message\" \"external-body\" (\n         \"name\" \"BodyFormats.ps\" \"site\" \"thumper.bellcore.com\" \"mode\" \"image\" \"access-type\" \"ANON-FTP\" \"directory\" \"pub\" \"expiration\" \"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n      ) NIL NIL NIL 79 \"13a120642a010037cbc238c999349116\" NIL NIL NIL\n   )(\n      \"message\" \"external-body\" (\n         \"access-type\" \"local-file\" \"name\" \"/u/nsb/writing/rfcs/RFC-MIME.ps\" \"site\" \"thumper.bellcore.com\" \"expiration\" \"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n      ) NIL NIL NIL 79 \"13a120642a010037cbc238c999349116\" NIL NIL NIL\n   )(\n      \"message\" \"external-body\" (\n         \"access-type\" \"mail-server\" \"server\" \"listserv@bogus.bitnet\" \"expiration\" \"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n      ) NIL NIL NIL 97 \"6f6c2486f2516ed45542404973c1cebb\" NIL NIL NIL\n   ) \"alternative\" (\n      \"boundary\" \"42\"\n   ) NIL NIL NIL\n)\n\nBODY[] {1109}\r\nFrom: Whomever\nTo: Someone\nDate: Whenever\nSubject: whatever\nMIME-Version: 1.0\nMessage-ID: <id1@host.com>\nContent-Type: multipart/alternative; boundary=42\nContent-ID: <id001@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body; name=\"BodyFormats.ps\";\n            site=\"thumper.bellcore.com\"; mode=\"image\";\n            access-type=ANON-FTP; directory=\"pub\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body; access-type=local-file;\n            name=\"/u/nsb/writing/rfcs/RFC-MIME.ps\";\n            site=\"thumper.bellcore.com\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body;\n            access-type=mail-server\n            server=\"listserv@bogus.bitnet\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nget RFC-MIME.DOC\n\n--42--\n\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 1109\n----------------------------------\nBODY[HEADER] {198}\r\nFrom: Whomever\nTo: Someone\nDate: Whenever\nSubject: whatever\nMIME-Version: 1.0\nMessage-ID: <id1@host.com>\nContent-Type: multipart/alternative; boundary=42\nContent-ID: <id001@guppylake.bellcore.com>\n\n\n----------------------------------\nBODY[TEXT] {911}\r\n--42\nContent-Type: message/external-body; name=\"BodyFormats.ps\";\n            site=\"thumper.bellcore.com\"; mode=\"image\";\n            access-type=ANON-FTP; directory=\"pub\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body; access-type=local-file;\n            name=\"/u/nsb/writing/rfcs/RFC-MIME.ps\";\n            site=\"thumper.bellcore.com\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body;\n            access-type=mail-server\n            server=\"listserv@bogus.bitnet\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nget RFC-MIME.DOC\n\n--42--\n\n\n\n----------------------------------\nBODY[MIME] {94}\r\nContent-Type: multipart/alternative; boundary=42\nContent-ID: <id001@guppylake.bellcore.com>\n\r\n\n----------------------------------\nBODY[1] {79}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nBINARY[1] {16}\r\n[binary content]\nBINARY.SIZE[1] 79\n----------------------------------\nBODY[1.HEADER] {230}\r\nContent-Type: message/external-body; name=\"BodyFormats.ps\";\n            site=\"thumper.bellcore.com\"; mode=\"image\";\n            access-type=ANON-FTP; directory=\"pub\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\n\n----------------------------------\nBODY[1.TEXT] {79}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n----------------------------------\nBODY[1.MIME] {231}\r\nContent-Type: message/external-body; name=\"BodyFormats.ps\";\n            site=\"thumper.bellcore.com\"; mode=\"image\";\n            access-type=ANON-FTP; directory=\"pub\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\r\n\n----------------------------------\nBODY[1.1] {79}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nBINARY[1.1] {16}\r\n[binary content]\nBINARY.SIZE[1.1] 79\n----------------------------------\nBODY[2] {79}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nBINARY[2] {16}\r\n[binary content]\nBINARY.SIZE[2] 79\n----------------------------------\nBODY[2.HEADER] {218}\r\nContent-Type: message/external-body; access-type=local-file;\n            name=\"/u/nsb/writing/rfcs/RFC-MIME.ps\";\n            site=\"thumper.bellcore.com\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\n\n----------------------------------\nBODY[2.TEXT] {79}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n----------------------------------\nBODY[2.MIME] {219}\r\nContent-Type: message/external-body; access-type=local-file;\n            name=\"/u/nsb/writing/rfcs/RFC-MIME.ps\";\n            site=\"thumper.bellcore.com\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\r\n\n----------------------------------\nBODY[2.1] {79}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nBINARY[2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.1] 79\n----------------------------------\nBODY[3] {97}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nget RFC-MIME.DOC\n\nBINARY[3] {16}\r\n[binary content]\nBINARY.SIZE[3] 97\n----------------------------------\nBODY[3.HEADER] {181}\r\nContent-Type: message/external-body;\n            access-type=mail-server\n            server=\"listserv@bogus.bitnet\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\n\n----------------------------------\nBODY[3.TEXT] {97}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nget RFC-MIME.DOC\n\n----------------------------------\nBODY[3.MIME] {182}\r\nContent-Type: message/external-body;\n            access-type=mail-server\n            server=\"listserv@bogus.bitnet\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\r\n\n----------------------------------\nBODY[3.1] {97}\r\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nget RFC-MIME.DOC\n\nBINARY[3.1] {16}\r\n[binary content]\nBINARY.SIZE[3.1] 97\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {29}\r\nFrom: Whomever\nTo: Someone\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {19}\r\never\nTo: Someone\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {181}\r\nFrom: Whomever\nTo: Someone\nDate: Whenever\nMIME-Version: 1.0\nMessage-ID: <id1@host.com>\nContent-Type: multipart/alternative; boundary=42\nContent-ID: <id001@guppylake.bellcore.com>\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\never\nTo: Someone\nDate: Wh\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/001.txt",
    "content": "From: Whomever\nTo: Someone\nDate: Whenever\nSubject: whatever\nMIME-Version: 1.0\nMessage-ID: <id1@host.com>\nContent-Type: multipart/alternative; boundary=42\nContent-ID: <id001@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body; name=\"BodyFormats.ps\";\n            site=\"thumper.bellcore.com\"; mode=\"image\";\n            access-type=ANON-FTP; directory=\"pub\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body; access-type=local-file;\n            name=\"/u/nsb/writing/rfcs/RFC-MIME.ps\";\n            site=\"thumper.bellcore.com\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\n--42\nContent-Type: message/external-body;\n            access-type=mail-server\n            server=\"listserv@bogus.bitnet\";\n            expiration=\"Fri, 14 Jun 1991 19:13:14 -0400 (EDT)\"\n\nContent-type: application/postscript\nContent-ID: <id42@guppylake.bellcore.com>\n\nget RFC-MIME.DOC\n\n--42--\n\n\n"
  },
  {
    "path": "tests/resources/imap/002.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 61 5\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL \"7bit\" 1979 (\n         \"Thu, 13 Aug 1998 17:42:41 +1000\" \"Map of Argentina with Description\" (\n            (\n               \"Bill Clinton\" NIL \"president\" \"whitehouse.gov\"\n            )\n         ) (\n            (\n               \"Bill Clinton\" NIL \"president\" \"whitehouse.gov\"\n            )\n         ) (\n            (\n               \"Bill Clinton\" NIL \"president\" \"whitehouse.gov\"\n            )\n         ) (\n            (\n               \"A1 Gore (The Enforcer)\" NIL \"vice-president\" \"whitehouse.gov\"\n            )\n         ) NIL NIL NIL \"<199804130742.RAA20366@mai1host.whitehouse.gov>\"\n      ) (\n         (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 355 12\n         )(\n            \"image\" \"gif\" (\n               \"name\" \"map_of_Argentina.gif\"\n            ) NIL NIL \"base64\" 389\n         ) \"mixed\"\n      ) 0\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 61 5 \"63aa55d7eacd3e7d4af551d3d7f8fe46\" NIL NIL NIL\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL \"7bit\" 1979 (\n         \"Thu, 13 Aug 1998 17:42:41 +1000\" \"Map of Argentina with Description\" (\n            (\n               \"Bill Clinton\" NIL \"president\" \"whitehouse.gov\"\n            )\n         ) (\n            (\n               \"Bill Clinton\" NIL \"president\" \"whitehouse.gov\"\n            )\n         ) (\n            (\n               \"Bill Clinton\" NIL \"president\" \"whitehouse.gov\"\n            )\n         ) (\n            (\n               \"A1 Gore (The Enforcer)\" NIL \"vice-president\" \"whitehouse.gov\"\n            )\n         ) NIL NIL NIL \"<199804130742.RAA20366@mai1host.whitehouse.gov>\"\n      ) (\n         (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 355 12 \"47027313db254547987c7eb4be377593\" NIL NIL NIL\n         )(\n            \"image\" \"gif\" (\n               \"name\" \"map_of_Argentina.gif\"\n            ) NIL NIL \"base64\" 389 \"5359a63540c1a8eae06a27e8760e1501\" (\n               \"inline\" (\n                  \"fi1ename\" \"map_of_Argentina.gif\"\n               )\n            ) NIL NIL\n         ) \"mixed\" (\n            \"boundary\" \"DC8------------DC8638F443D87A7F0726DEF7\"\n         ) NIL NIL NIL\n      ) 0 \"1522079b4114146b37ea1af171484a1a\" (\n         \"inline\" NIL\n      ) NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n   ) NIL NIL NIL\n)\n\nBODY[] {2652}\r\nFrom:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\nSubject: [Fwd: Map of Argentina with Description]\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\nThis is a multi-part message in MIME format.\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\nContent-Disposition: inline\n\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\n--D7F------------D7FD5A0B8AB9C65CCDBFA872--\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 2652\n----------------------------------\nBODY[HEADER] {270}\r\nFrom:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\nSubject: [Fwd: Map of Argentina with Description]\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\n\n----------------------------------\nBODY[TEXT] {2382}\r\nThis is a multi-part message in MIME format.\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\nContent-Disposition: inline\n\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\n--D7F------------D7FD5A0B8AB9C65CCDBFA872--\n\n\n----------------------------------\nBODY[MIME] {98}\r\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\r\n\n----------------------------------\nBODY[1] {61}\r\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\nBINARY[1] {61}\r\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\nBINARY.SIZE[1] 61\n----------------------------------\nBODY[1.HEADER] {76}\r\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\n\n----------------------------------\nBODY[1.TEXT] {61}\r\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\n----------------------------------\nBODY[1.MIME] {77}\r\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\r\n\n----------------------------------\nBODY[1.1] {61}\r\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\nBINARY[1.1] {61}\r\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\nBINARY.SIZE[1.1] 61\n----------------------------------\nBODY[2] {1979}\r\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\nBINARY[2] {16}\r\n[binary content]\nBINARY.SIZE[2] 1979\n----------------------------------\nBODY[2.HEADER] {835}\r\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\n\n----------------------------------\nBODY[2.TEXT] {1144}\r\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\n----------------------------------\nBODY[2.MIME] {91}\r\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[2.1] {355}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\nBINARY[2.1] {355}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\nBINARY.SIZE[2.1] 355\n----------------------------------\nBODY[2.1.HEADER] {76}\r\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\n\n----------------------------------\nBODY[2.1.TEXT] {355}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n----------------------------------\nBODY[2.1.MIME] {77}\r\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\r\n\n----------------------------------\nBODY[2.1.1] {355}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\nBINARY[2.1.1] {355}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\nBINARY.SIZE[2.1.1] 355\n----------------------------------\nBODY[2.2] {389}\r\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\nBINARY[2.2] {16}\r\n[binary content]\nBINARY.SIZE[2.2] 288\n----------------------------------\nBODY[2.2.HEADER] {149}\r\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\n\n----------------------------------\nBODY[2.2.TEXT] {389}\r\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n----------------------------------\nBODY[2.2.MIME] {150}\r\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\r\n\n----------------------------------\nBODY[2.2.1] {389}\r\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\nBINARY[2.2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.2.1] 288\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {125}\r\nFrom:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\nGore <vice-president@whit\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {221}\r\nFrom:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\nGore <vice-president@whit\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/002.txt",
    "content": "From:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\nSubject: [Fwd: Map of Argentina with Description]\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\nThis is a multi-part message in MIME format.\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\nContent-Disposition: inline\n\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\n--D7F------------D7FD5A0B8AB9C65CCDBFA872--\n\n"
  },
  {
    "path": "tests/resources/imap/003.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 48 1\n   )(\n      \"text\" \"enriched\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 69 2\n   )(\n      \"application\" \"x-whatever\" NIL NIL NIL NIL 51\n   ) \"alternative\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 48 1 \"2229d79e5de40eae37c43fe934adbd24\" NIL NIL NIL\n   )(\n      \"text\" \"enriched\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 69 2 \"1aba0fd91c6544b8626008f78b83c3f9\" NIL NIL NIL\n   )(\n      \"application\" \"x-whatever\" NIL NIL NIL NIL 51 \"7f7ef645554f637854b8acdd853aa612\" NIL NIL NIL\n   ) \"alternative\" (\n      \"boundary\" \"boundary42\"\n   ) NIL NIL NIL\n)\n\nBODY[] {565}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Mon, 22 Mar 1993 09:41:09 -0800 (PST)\nSubject: Formatted text mail\nMIME-Version: 1.0\nContent-Type: multipart/alternative; boundary=boundary42\n\n--boundary42\nContent-Type: text/plain; charset=us-ascii\n\n... plain text version of message goes here ...\n\n--boundary42\nContent-Type: text/enriched\n\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\n--boundary42\nContent-Type: application/x-whatever\n\n... fanciest version of same message goes here ...\n\n--boundary42--\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 565\n----------------------------------\nBODY[HEADER] {228}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Mon, 22 Mar 1993 09:41:09 -0800 (PST)\nSubject: Formatted text mail\nMIME-Version: 1.0\nContent-Type: multipart/alternative; boundary=boundary42\n\n\n----------------------------------\nBODY[TEXT] {337}\r\n--boundary42\nContent-Type: text/plain; charset=us-ascii\n\n... plain text version of message goes here ...\n\n--boundary42\nContent-Type: text/enriched\n\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\n--boundary42\nContent-Type: application/x-whatever\n\n... fanciest version of same message goes here ...\n\n--boundary42--\n\n\n----------------------------------\nBODY[MIME] {59}\r\nContent-Type: multipart/alternative; boundary=boundary42\n\r\n\n----------------------------------\nBODY[1] {48}\r\n... plain text version of message goes here ...\n\nBINARY[1] {48}\r\n... plain text version of message goes here ...\n\nBINARY.SIZE[1] 48\n----------------------------------\nBODY[1.HEADER] {44}\r\nContent-Type: text/plain; charset=us-ascii\n\n\n----------------------------------\nBODY[1.TEXT] {48}\r\n... plain text version of message goes here ...\n\n----------------------------------\nBODY[1.MIME] {45}\r\nContent-Type: text/plain; charset=us-ascii\n\r\n\n----------------------------------\nBODY[1.1] {48}\r\n... plain text version of message goes here ...\n\nBINARY[1.1] {48}\r\n... plain text version of message goes here ...\n\nBINARY.SIZE[1.1] 48\n----------------------------------\nBODY[2] {69}\r\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\nBINARY[2] {69}\r\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\nBINARY.SIZE[2] 69\n----------------------------------\nBODY[2.HEADER] {29}\r\nContent-Type: text/enriched\n\n\n----------------------------------\nBODY[2.TEXT] {69}\r\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\n----------------------------------\nBODY[2.MIME] {30}\r\nContent-Type: text/enriched\n\r\n\n----------------------------------\nBODY[2.1] {69}\r\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\nBINARY[2.1] {69}\r\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\nBINARY.SIZE[2.1] 69\n----------------------------------\nBODY[3] {51}\r\n... fanciest version of same message goes here ...\n\nBINARY[3] {16}\r\n[binary content]\nBINARY.SIZE[3] 51\n----------------------------------\nBODY[3.HEADER] {38}\r\nContent-Type: application/x-whatever\n\n\n----------------------------------\nBODY[3.TEXT] {51}\r\n... fanciest version of same message goes here ...\n\n----------------------------------\nBODY[3.MIME] {39}\r\nContent-Type: application/x-whatever\n\r\n\n----------------------------------\nBODY[3.1] {51}\r\n... fanciest version of same message goes here ...\n\nBINARY[3.1] {16}\r\n[binary content]\nBINARY.SIZE[3.1] 51\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {81}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\naniel Borenstein <nsb@bel\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {200}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Mon, 22 Mar 1993 09:41:09 -0800 (PST)\nMIME-Version: 1.0\nContent-Type: multipart/alternative; boundary=boundary42\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\naniel Borenstein <nsb@bel\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/003.txt",
    "content": "From: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Mon, 22 Mar 1993 09:41:09 -0800 (PST)\nSubject: Formatted text mail\nMIME-Version: 1.0\nContent-Type: multipart/alternative; boundary=boundary42\n\n--boundary42\nContent-Type: text/plain; charset=us-ascii\n\n... plain text version of message goes here ...\n\n--boundary42\nContent-Type: text/enriched\n\n... RFC 1896 text/enriched version of same message\n    goes here ...\n\n--boundary42\nContent-Type: application/x-whatever\n\n... fanciest version of same message goes here ...\n\n--boundary42--\n\n"
  },
  {
    "path": "tests/resources/imap/004.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 45 1\n   )(\n      (\n         \"message\" NIL NIL NIL NIL NIL 100 (\n            \"Fri, 26 Mar 1993 11:13:32 +0200\" \"my opinion\" (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 22 1\n         ) 0\n      )(\n         \"message\" NIL NIL NIL NIL NIL 125 (\n            \"Fri, 26 Mar 1993 10:07:13 -0500\" \"my different opinion\" (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 31 1\n         ) 0\n      ) \"digest\"\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 45 1 \"f7f7e917e54763330bc648984459b38e\" NIL NIL NIL\n   )(\n      (\n         \"message\" NIL NIL NIL NIL NIL 100 (\n            \"Fri, 26 Mar 1993 11:13:32 +0200\" \"my opinion\" (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 22 1 \"1ec4e1f0175dc661a12854b33879ce9b\" NIL NIL NIL\n         ) 0 \"ef1527984c599523c90fb184a8fdfd61\" NIL NIL NIL\n      )(\n         \"message\" NIL NIL NIL NIL NIL 125 (\n            \"Fri, 26 Mar 1993 10:07:13 -0500\" \"my different opinion\" (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 31 1 \"7a0001277b1197e7029ad48b54919b62\" NIL NIL NIL\n         ) 0 \"9609bccef55fba1208046885cdcc9db8\" NIL NIL NIL\n      ) \"digest\" (\n         \"boundary\" \"---- next message ----\"\n      ) NIL NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"---- main boundary ----\"\n   ) NIL NIL NIL\n)\n\nBODY[] {728}\r\nFrom: Moderator-Address\nTo: Recipient-List\nDate: Mon, 22 Mar 1994 13:34:51 +0000\nSubject: Internet Digest, volume 42\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n            boundary=\"---- main boundary ----\"\n\n------ main boundary ----\n\n...Introductory text or table of contents...\n\n------ main boundary ----\nContent-Type: multipart/digest;\n            boundary=\"---- next message ----\"\n\n------ next message ----\n\nFrom: someone-else\nDate: Fri, 26 Mar 1993 11:13:32 +0200\nSubject: my opinion\n\n...body goes here ...\n\n------ next message ----\n\nFrom: someone-else-again\nDate: Fri, 26 Mar 1993 10:07:13 -0500\nSubject: my different opinion\n\n... another body goes here ...\n\n------ next message ------\n\n------ main boundary ------\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 728\n----------------------------------\nBODY[HEADER] {214}\r\nFrom: Moderator-Address\nTo: Recipient-List\nDate: Mon, 22 Mar 1994 13:34:51 +0000\nSubject: Internet Digest, volume 42\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n            boundary=\"---- main boundary ----\"\n\n\n----------------------------------\nBODY[TEXT] {514}\r\n------ main boundary ----\n\n...Introductory text or table of contents...\n\n------ main boundary ----\nContent-Type: multipart/digest;\n            boundary=\"---- next message ----\"\n\n------ next message ----\n\nFrom: someone-else\nDate: Fri, 26 Mar 1993 11:13:32 +0200\nSubject: my opinion\n\n...body goes here ...\n\n------ next message ----\n\nFrom: someone-else-again\nDate: Fri, 26 Mar 1993 10:07:13 -0500\nSubject: my different opinion\n\n... another body goes here ...\n\n------ next message ------\n\n------ main boundary ------\n\n\n----------------------------------\nBODY[MIME] {80}\r\nContent-Type: multipart/mixed;\n            boundary=\"---- main boundary ----\"\n\r\n\n----------------------------------\nBODY[1] {45}\r\n...Introductory text or table of contents...\n\nBINARY[1] {45}\r\n...Introductory text or table of contents...\n\nBINARY.SIZE[1] 45\n----------------------------------\nBODY[1.HEADER] {1}\r\n\n\n----------------------------------\nBODY[1.TEXT] {45}\r\n...Introductory text or table of contents...\n\n----------------------------------\nBODY[1.MIME] {2}\r\n\r\n\n----------------------------------\nBODY[1.1] {45}\r\n...Introductory text or table of contents...\n\nBINARY[1.1] {45}\r\n...Introductory text or table of contents...\n\nBINARY.SIZE[1.1] 45\n----------------------------------\nBODY[2] {306}\r\n------ next message ----\n\nFrom: someone-else\nDate: Fri, 26 Mar 1993 11:13:32 +0200\nSubject: my opinion\n\n...body goes here ...\n\n------ next message ----\n\nFrom: someone-else-again\nDate: Fri, 26 Mar 1993 10:07:13 -0500\nSubject: my different opinion\n\n... another body goes here ...\n\n------ next message ------\n\nBINARY[2] {16}\r\n[binary content]\nBINARY.SIZE[2] 385\n----------------------------------\nBODY[2.HEADER] {79}\r\nContent-Type: multipart/digest;\n            boundary=\"---- next message ----\"\n\n\n----------------------------------\nBODY[2.TEXT] {306}\r\n------ next message ----\n\nFrom: someone-else\nDate: Fri, 26 Mar 1993 11:13:32 +0200\nSubject: my opinion\n\n...body goes here ...\n\n------ next message ----\n\nFrom: someone-else-again\nDate: Fri, 26 Mar 1993 10:07:13 -0500\nSubject: my different opinion\n\n... another body goes here ...\n\n------ next message ------\n\n----------------------------------\nBODY[2.MIME] {80}\r\nContent-Type: multipart/digest;\n            boundary=\"---- next message ----\"\n\r\n\n----------------------------------\nBODY[2.1] {100}\r\nFrom: someone-else\nDate: Fri, 26 Mar 1993 11:13:32 +0200\nSubject: my opinion\n\n...body goes here ...\n\nBINARY[2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.1] 100\n----------------------------------\nBODY[2.1.HEADER] {78}\r\nFrom: someone-else\nDate: Fri, 26 Mar 1993 11:13:32 +0200\nSubject: my opinion\n\n\n----------------------------------\nBODY[2.1.TEXT] {22}\r\n...body goes here ...\n\n----------------------------------\nBODY[2.1.MIME] {2}\r\n\r\n\n----------------------------------\nBODY[2.1.1] {22}\r\n...body goes here ...\n\nBINARY[2.1.1] {22}\r\n...body goes here ...\n\nBINARY.SIZE[2.1.1] 22\n----------------------------------\nBODY[2.2] {125}\r\nFrom: someone-else-again\nDate: Fri, 26 Mar 1993 10:07:13 -0500\nSubject: my different opinion\n\n... another body goes here ...\n\nBINARY[2.2] {16}\r\n[binary content]\nBINARY.SIZE[2.2] 125\n----------------------------------\nBODY[2.2.HEADER] {94}\r\nFrom: someone-else-again\nDate: Fri, 26 Mar 1993 10:07:13 -0500\nSubject: my different opinion\n\n\n----------------------------------\nBODY[2.2.TEXT] {31}\r\n... another body goes here ...\n\n----------------------------------\nBODY[2.2.MIME] {2}\r\n\r\n\n----------------------------------\nBODY[2.2.1] {31}\r\n... another body goes here ...\n\nBINARY[2.2.1] {31}\r\n... another body goes here ...\n\nBINARY.SIZE[2.2.1] 31\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {45}\r\nFrom: Moderator-Address\nTo: Recipient-List\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\nrator-Address\nTo: Recipie\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {179}\r\nFrom: Moderator-Address\nTo: Recipient-List\nDate: Mon, 22 Mar 1994 13:34:51 +0000\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n            boundary=\"---- main boundary ----\"\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\nrator-Address\nTo: Recipie\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/004.txt",
    "content": "From: Moderator-Address\nTo: Recipient-List\nDate: Mon, 22 Mar 1994 13:34:51 +0000\nSubject: Internet Digest, volume 42\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n            boundary=\"---- main boundary ----\"\n\n------ main boundary ----\n\n...Introductory text or table of contents...\n\n------ main boundary ----\nContent-Type: multipart/digest;\n            boundary=\"---- next message ----\"\n\n------ next message ----\n\nFrom: someone-else\nDate: Fri, 26 Mar 1993 11:13:32 +0200\nSubject: my opinion\n\n...body goes here ...\n\n------ next message ----\n\nFrom: someone-else-again\nDate: Fri, 26 Mar 1993 10:07:13 -0500\nSubject: my different opinion\n\n... another body goes here ...\n\n------ next message ------\n\n------ main boundary ------\n\n"
  },
  {
    "path": "tests/resources/imap/005.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 79 1\n   )(\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 76 2\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 79 1 \"b35878dedb7cd0aa6934f90df9d517b0\" NIL NIL NIL\n   )(\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 76 2 \"d3905bfd2a53ad0a4438d01bf271677a\" NIL NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"simple boundary\"\n   ) NIL NIL NIL\n)\n\nBODY[] {691}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Sun, 21 Mar 1993 23:56:48 -0800 (PST)\nSubject: Sample message\nMIME-Version: 1.0\nContent-type: multipart/mixed; boundary=\"simple boundary\"\n\nThis is the preamble.  It is to be ignored, though it\nis a handy place for composition agents to include an\nexplanatory note to non-MIME conformant readers.\n\n--simple boundary\n\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\n--simple boundary\nContent-type: text/plain; charset=us-ascii\n\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\n--simple boundary--\n\nThis is the epilogue.  It is also to be ignored.\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 691\n----------------------------------\nBODY[HEADER] {224}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Sun, 21 Mar 1993 23:56:48 -0800 (PST)\nSubject: Sample message\nMIME-Version: 1.0\nContent-type: multipart/mixed; boundary=\"simple boundary\"\n\n\n----------------------------------\nBODY[TEXT] {467}\r\nThis is the preamble.  It is to be ignored, though it\nis a handy place for composition agents to include an\nexplanatory note to non-MIME conformant readers.\n\n--simple boundary\n\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\n--simple boundary\nContent-type: text/plain; charset=us-ascii\n\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\n--simple boundary--\n\nThis is the epilogue.  It is also to be ignored.\n\n\n----------------------------------\nBODY[MIME] {60}\r\nContent-Type: multipart/mixed; boundary=\"simple boundary\"\n\r\n\n----------------------------------\nBODY[1] {79}\r\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\nBINARY[1] {79}\r\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\nBINARY.SIZE[1] 79\n----------------------------------\nBODY[1.HEADER] {1}\r\n\n\n----------------------------------\nBODY[1.TEXT] {79}\r\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\n----------------------------------\nBODY[1.MIME] {2}\r\n\r\n\n----------------------------------\nBODY[1.1] {79}\r\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\nBINARY[1.1] {79}\r\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\nBINARY.SIZE[1.1] 79\n----------------------------------\nBODY[2] {76}\r\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\nBINARY[2] {76}\r\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\nBINARY.SIZE[2] 76\n----------------------------------\nBODY[2.HEADER] {44}\r\nContent-type: text/plain; charset=us-ascii\n\n\n----------------------------------\nBODY[2.TEXT] {76}\r\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\n----------------------------------\nBODY[2.MIME] {45}\r\nContent-Type: text/plain; charset=us-ascii\n\r\n\n----------------------------------\nBODY[2.1] {76}\r\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\nBINARY[2.1] {76}\r\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\nBINARY.SIZE[2.1] 76\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {81}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\naniel Borenstein <nsb@bel\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {201}\r\nFrom: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Sun, 21 Mar 1993 23:56:48 -0800 (PST)\nMIME-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"simple boundary\"\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\naniel Borenstein <nsb@bel\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/005.txt",
    "content": "From: Nathaniel Borenstein <nsb@bellcore.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Sun, 21 Mar 1993 23:56:48 -0800 (PST)\nSubject: Sample message\nMIME-Version: 1.0\nContent-type: multipart/mixed; boundary=\"simple boundary\"\n\nThis is the preamble.  It is to be ignored, though it\nis a handy place for composition agents to include an\nexplanatory note to non-MIME conformant readers.\n\n--simple boundary\n\nThis is implicitly typed plain US-ASCII text.\nIt does NOT end with a linebreak.\n--simple boundary\nContent-type: text/plain; charset=us-ascii\n\nThis is explicitly typed plain US-ASCII text.\nIt DOES end with a linebreak.\n\n--simple boundary--\n\nThis is the epilogue.  It is also to be ignored.\n\n"
  },
  {
    "path": "tests/resources/imap/006.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"utf-8\"\n      ) NIL NIL \"quoted-printable\" 87 2\n   )(\n      \"text\" \"html\" (\n         \"charset\" \"utf-8\"\n      ) NIL NIL \"quoted-printable\" 93 2\n   ) \"alternative\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"utf-8\"\n      ) NIL NIL \"quoted-printable\" 87 2 \"ba404f7e85d80b41eddae75d5087186d\" (\n         \"inline\" NIL\n      ) NIL NIL\n   )(\n      \"text\" \"html\" (\n         \"charset\" \"utf-8\"\n      ) NIL NIL \"quoted-printable\" 93 2 \"bb2724458c0a6183195d922d711523c0\" (\n         \"inline\" NIL\n      ) NIL NIL\n   ) \"alternative\" (\n      \"boundary\" \"boundary-string\"\n   ) NIL NIL NIL\n)\n\nBODY[] {617}\r\nFrom: sender@example.com\nTo: recipient@example.com\nSubject: Multipart Email Example\nContent-Type: multipart/alternative; boundary=\"boundary-string\"\n\n--boundary-string\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\n--boundary-string\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\n--boundary-string--\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 617\n----------------------------------\nBODY[HEADER] {149}\r\nFrom: sender@example.com\nTo: recipient@example.com\nSubject: Multipart Email Example\nContent-Type: multipart/alternative; boundary=\"boundary-string\"\n\n\n----------------------------------\nBODY[TEXT] {468}\r\n--boundary-string\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\n--boundary-string\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\n--boundary-string--\n\n\n----------------------------------\nBODY[MIME] {66}\r\nContent-Type: multipart/alternative; boundary=\"boundary-string\"\n\r\n\n----------------------------------\nBODY[1] {87}\r\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\nBINARY[1] {87}\r\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\nBINARY.SIZE[1] 87\n----------------------------------\nBODY[1.HEADER] {115}\r\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\n\n----------------------------------\nBODY[1.TEXT] {87}\r\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\n----------------------------------\nBODY[1.MIME] {116}\r\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[1.1] {87}\r\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\nBINARY[1.1] {87}\r\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\nBINARY.SIZE[1.1] 87\n----------------------------------\nBODY[2] {93}\r\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\nBINARY[2] {93}\r\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\nBINARY.SIZE[2] 93\n----------------------------------\nBODY[2.HEADER] {114}\r\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\n\n----------------------------------\nBODY[2.TEXT] {93}\r\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\n----------------------------------\nBODY[2.MIME] {115}\r\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[2.1] {93}\r\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\nBINARY[2.1] {93}\r\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\nBINARY.SIZE[2.1] 93\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {53}\r\nFrom: sender@example.com\nTo: recipient@example.com\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\ner@example.com\nTo: recipi\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {117}\r\nFrom: sender@example.com\nTo: recipient@example.com\nContent-Type: multipart/alternative; boundary=\"boundary-string\"\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\ner@example.com\nTo: recipi\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/006.txt",
    "content": "From: sender@example.com\nTo: recipient@example.com\nSubject: Multipart Email Example\nContent-Type: multipart/alternative; boundary=\"boundary-string\"\n\n--boundary-string\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\n--boundary-string\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\n--boundary-string--\n\n"
  },
  {
    "path": "tests/resources/imap/007.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 1 0\n   )(\n      (\n         (\n            (\n               \"text\" \"plain\" (\n                  \"charset\" \"us-ascii\"\n               ) NIL NIL \"7bit\" 1 0\n            )(\n               \"image\" \"jpeg\" NIL NIL NIL NIL 1\n            )(\n               \"text\" \"plain\" (\n                  \"charset\" \"us-ascii\"\n               ) NIL NIL \"7bit\" 1 0\n            ) \"mixed\"\n         )(\n            (\n               \"text\" \"html\" (\n                  \"charset\" \"us-ascii\"\n               ) NIL NIL \"7bit\" 14 0\n            )(\n               \"image\" \"jpeg\" NIL NIL NIL NIL 1\n            ) \"related\"\n         ) \"alternative\"\n      )(\n         \"image\" \"jpeg\" NIL NIL NIL NIL 1\n      )(\n         \"application\" \"x-excel\" NIL NIL NIL NIL 1\n      )(\n         \"message\" \"rfc822\" NIL NIL NIL NIL 13 (\n            NIL \"J\" (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 1 0\n         ) 0\n      ) \"mixed\"\n   )(\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 1 0\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 1 0 \"7fc56270e7a70fa81a5935b72eacbe29\" (\n         \"inline\" NIL\n      ) NIL NIL\n   )(\n      (\n         (\n            (\n               \"text\" \"plain\" (\n                  \"charset\" \"us-ascii\"\n               ) NIL NIL \"7bit\" 1 0 \"9d5ed678fe57bcca610140957afab571\" (\n                  \"inline\" NIL\n               ) NIL NIL\n            )(\n               \"image\" \"jpeg\" NIL NIL NIL NIL 1 \"0d61f8370cad1d412f80b84d143e1257\" (\n                  \"inline\" NIL\n               ) NIL NIL\n            )(\n               \"text\" \"plain\" (\n                  \"charset\" \"us-ascii\"\n               ) NIL NIL \"7bit\" 1 0 \"f623e75af30e62bbd73d6df5b50bb7b5\" (\n                  \"inline\" NIL\n               ) NIL NIL\n            ) \"mixed\" (\n               \"boundary\" \"4\"\n            ) NIL NIL NIL\n         )(\n            (\n               \"text\" \"html\" (\n                  \"charset\" \"us-ascii\"\n               ) NIL NIL \"7bit\" 14 0 \"ece7293a99c7e12f8d044b683e5c2f33\" NIL NIL NIL\n            )(\n               \"image\" \"jpeg\" NIL NIL NIL NIL 1 \"800618943025315f869e4e1f09471012\" NIL NIL NIL\n            ) \"related\" (\n               \"boundary\" \"5\"\n            ) NIL NIL NIL\n         ) \"alternative\" (\n            \"boundary\" \"3\"\n         ) NIL NIL NIL\n      )(\n         \"image\" \"jpeg\" NIL NIL NIL NIL 1 \"dfcf28d0734569a6a693bc8194de62bf\" (\n            \"attachment\" NIL\n         ) NIL NIL\n      )(\n         \"application\" \"x-excel\" NIL NIL NIL NIL 1 \"c1d9f50f86825a1a2302ec2449c17196\" NIL NIL NIL\n      )(\n         \"message\" \"rfc822\" NIL NIL NIL NIL 13 (\n            NIL \"J\" (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) (\n               (\n                  NIL NIL \"unknown\" \"localhost\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 1 0 \"ff44570aca8241914870afbc310cdb85\" NIL NIL NIL\n         ) 0 \"9c9888d1b2c167dd33f7542df5a65aa7\" NIL NIL NIL\n      ) \"mixed\" (\n         \"boundary\" \"2\"\n      ) NIL NIL NIL\n   )(\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 1 0 \"a5f3c6a11b03839d46af9fb43c97c188\" (\n         \"inline\" NIL\n      ) NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"1\"\n   ) NIL NIL NIL\n)\n\nBODY[] {871}\r\nSubject: RFC 8621 Section 4.1.4 test\nContent-Type: multipart/mixed; boundary=\"1\"\n\n--1\nContent-Type: text/plain\nContent-Disposition: inline\n\nA\n--1  \nContent-Type: multipart/mixed; boundary=\"2\"\n\n--2\nContent-Type: multipart/alternative; boundary=\"3\"\n\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\n--2   \nContent-Type: image/jpeg\nContent-Disposition: attachment\n\nG\n--2  \nContent-Type: application/x-excel\n\nH\n--2  \nContent-Type: message/rfc822\n\nSubject: J\n\nJ\n--2--\n\n--1  \nContent-Type: text/plain\nContent-Disposition: inline\n\nK\n--1--\n\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 871\n----------------------------------\nBODY[HEADER] {82}\r\nSubject: RFC 8621 Section 4.1.4 test\nContent-Type: multipart/mixed; boundary=\"1\"\n\n\n----------------------------------\nBODY[TEXT] {789}\r\n--1\nContent-Type: text/plain\nContent-Disposition: inline\n\nA\n--1  \nContent-Type: multipart/mixed; boundary=\"2\"\n\n--2\nContent-Type: multipart/alternative; boundary=\"3\"\n\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\n--2   \nContent-Type: image/jpeg\nContent-Disposition: attachment\n\nG\n--2  \nContent-Type: application/x-excel\n\nH\n--2  \nContent-Type: message/rfc822\n\nSubject: J\n\nJ\n--2--\n\n--1  \nContent-Type: text/plain\nContent-Disposition: inline\n\nK\n--1--\n\n\n----------------------------------\nBODY[MIME] {46}\r\nContent-Type: multipart/mixed; boundary=\"1\"\n\r\n\n----------------------------------\nBODY[1] {1}\r\nA\nBINARY[1] {1}\r\nA\nBINARY.SIZE[1] 1\n----------------------------------\nBODY[1.HEADER] {54}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\n\n----------------------------------\nBODY[1.TEXT] {1}\r\nA\n----------------------------------\nBODY[1.MIME] {55}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[1.1] {1}\r\nA\nBINARY[1.1] {1}\r\nA\nBINARY.SIZE[1.1] 1\n----------------------------------\nBODY[2] {608}\r\n--2\nContent-Type: multipart/alternative; boundary=\"3\"\n\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\n--2   \nContent-Type: image/jpeg\nContent-Disposition: attachment\n\nG\n--2  \nContent-Type: application/x-excel\n\nH\n--2  \nContent-Type: message/rfc822\n\nSubject: J\n\nJ\n--2--\n\nBINARY[2] {16}\r\n[binary content]\nBINARY.SIZE[2] 653\n----------------------------------\nBODY[2.HEADER] {45}\r\nContent-Type: multipart/mixed; boundary=\"2\"\n\n\n----------------------------------\nBODY[2.TEXT] {608}\r\n--2\nContent-Type: multipart/alternative; boundary=\"3\"\n\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\n--2   \nContent-Type: image/jpeg\nContent-Disposition: attachment\n\nG\n--2  \nContent-Type: application/x-excel\n\nH\n--2  \nContent-Type: message/rfc822\n\nSubject: J\n\nJ\n--2--\n\n----------------------------------\nBODY[2.MIME] {46}\r\nContent-Type: multipart/mixed; boundary=\"2\"\n\r\n\n----------------------------------\nBODY[2.1] {386}\r\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\nBINARY[2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.1] 437\n----------------------------------\nBODY[2.1.HEADER] {51}\r\nContent-Type: multipart/alternative; boundary=\"3\"\n\n\n----------------------------------\nBODY[2.1.TEXT] {386}\r\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\n----------------------------------\nBODY[2.1.MIME] {52}\r\nContent-Type: multipart/alternative; boundary=\"3\"\n\r\n\n----------------------------------\nBODY[2.1.1] {190}\r\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\nBINARY[2.1.1] {16}\r\n[binary content]\nBINARY.SIZE[2.1.1] 235\n----------------------------------\nBODY[2.1.1.HEADER] {45}\r\nContent-Type: multipart/mixed; boundary=\"4\"\n\n\n----------------------------------\nBODY[2.1.1.TEXT] {190}\r\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n----------------------------------\nBODY[2.1.1.MIME] {46}\r\nContent-Type: multipart/mixed; boundary=\"4\"\n\r\n\n----------------------------------\nBODY[2.1.1.1] {1}\r\nB\nBINARY[2.1.1.1] {1}\r\nB\nBINARY.SIZE[2.1.1.1] 1\n----------------------------------\nBODY[2.1.1.1.HEADER] {54}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\n\n----------------------------------\nBODY[2.1.1.1.TEXT] {1}\r\nB\n----------------------------------\nBODY[2.1.1.1.MIME] {55}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[2.1.1.1.1] {1}\r\nB\nBINARY[2.1.1.1.1] {1}\r\nB\nBINARY.SIZE[2.1.1.1.1] 1\n----------------------------------\nBODY[2.1.1.2] {1}\r\nC\nBINARY[2.1.1.2] {16}\r\n[binary content]\nBINARY.SIZE[2.1.1.2] 1\n----------------------------------\nBODY[2.1.1.2.HEADER] {54}\r\nContent-Type: image/jpeg\nContent-Disposition: inline\n\n\n----------------------------------\nBODY[2.1.1.2.TEXT] {1}\r\nC\n----------------------------------\nBODY[2.1.1.2.MIME] {55}\r\nContent-Type: image/jpeg\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[2.1.1.2.1] {1}\r\nC\nBINARY[2.1.1.2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.1.1.2.1] 1\n----------------------------------\nBODY[2.1.1.3] {1}\r\nD\nBINARY[2.1.1.3] {1}\r\nD\nBINARY.SIZE[2.1.1.3] 1\n----------------------------------\nBODY[2.1.1.3.HEADER] {54}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\n\n----------------------------------\nBODY[2.1.1.3.TEXT] {1}\r\nD\n----------------------------------\nBODY[2.1.1.3.MIME] {55}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[2.1.1.3.1] {1}\r\nD\nBINARY[2.1.1.3.1] {1}\r\nD\nBINARY.SIZE[2.1.1.3.1] 1\n----------------------------------\nBODY[2.1.2] {86}\r\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\nBINARY[2.1.2] {16}\r\n[binary content]\nBINARY.SIZE[2.1.2] 133\n----------------------------------\nBODY[2.1.2.HEADER] {47}\r\nContent-Type: multipart/related; boundary=\"5\"\n\n\n----------------------------------\nBODY[2.1.2.TEXT] {86}\r\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n----------------------------------\nBODY[2.1.2.MIME] {48}\r\nContent-Type: multipart/related; boundary=\"5\"\n\r\n\n----------------------------------\nBODY[2.1.2.1] {14}\r\n<html>E</html>\nBINARY[2.1.2.1] {14}\r\n<html>E</html>\nBINARY.SIZE[2.1.2.1] 14\n----------------------------------\nBODY[2.1.2.1.HEADER] {25}\r\nContent-Type: text/html\n\n\n----------------------------------\nBODY[2.1.2.1.TEXT] {14}\r\n<html>E</html>\n----------------------------------\nBODY[2.1.2.1.MIME] {26}\r\nContent-Type: text/html\n\r\n\n----------------------------------\nBODY[2.1.2.1.1] {14}\r\n<html>E</html>\nBINARY[2.1.2.1.1] {14}\r\n<html>E</html>\nBINARY.SIZE[2.1.2.1.1] 14\n----------------------------------\nBODY[2.1.2.2] {1}\r\nF\nBINARY[2.1.2.2] {16}\r\n[binary content]\nBINARY.SIZE[2.1.2.2] 1\n----------------------------------\nBODY[2.1.2.2.HEADER] {26}\r\nContent-Type: image/jpeg\n\n\n----------------------------------\nBODY[2.1.2.2.TEXT] {1}\r\nF\n----------------------------------\nBODY[2.1.2.2.MIME] {27}\r\nContent-Type: image/jpeg\n\r\n\n----------------------------------\nBODY[2.1.2.2.1] {1}\r\nF\nBINARY[2.1.2.2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.1.2.2.1] 1\n----------------------------------\nBODY[2.2] {1}\r\nG\nBINARY[2.2] {16}\r\n[binary content]\nBINARY.SIZE[2.2] 1\n----------------------------------\nBODY[2.2.HEADER] {58}\r\nContent-Type: image/jpeg\nContent-Disposition: attachment\n\n\n----------------------------------\nBODY[2.2.TEXT] {1}\r\nG\n----------------------------------\nBODY[2.2.MIME] {59}\r\nContent-Type: image/jpeg\nContent-Disposition: attachment\n\r\n\n----------------------------------\nBODY[2.2.1] {1}\r\nG\nBINARY[2.2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.2.1] 1\n----------------------------------\nBODY[2.3] {1}\r\nH\nBINARY[2.3] {16}\r\n[binary content]\nBINARY.SIZE[2.3] 1\n----------------------------------\nBODY[2.3.HEADER] {35}\r\nContent-Type: application/x-excel\n\n\n----------------------------------\nBODY[2.3.TEXT] {1}\r\nH\n----------------------------------\nBODY[2.3.MIME] {36}\r\nContent-Type: application/x-excel\n\r\n\n----------------------------------\nBODY[2.3.1] {1}\r\nH\nBINARY[2.3.1] {16}\r\n[binary content]\nBINARY.SIZE[2.3.1] 1\n----------------------------------\nBODY[2.4] {13}\r\nSubject: J\n\nJ\nBINARY[2.4] {16}\r\n[binary content]\nBINARY.SIZE[2.4] 13\n----------------------------------\nBODY[2.4.HEADER] {12}\r\nSubject: J\n\n\n----------------------------------\nBODY[2.4.TEXT] {1}\r\nJ\n----------------------------------\nBODY[2.4.MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[2.4.1] {1}\r\nJ\nBINARY[2.4.1] {1}\r\nJ\nBINARY.SIZE[2.4.1] 1\n----------------------------------\nBODY[3] {1}\r\nK\nBINARY[3] {1}\r\nK\nBINARY.SIZE[3] 1\n----------------------------------\nBODY[3.HEADER] {54}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\n\n----------------------------------\nBODY[3.TEXT] {1}\r\nK\n----------------------------------\nBODY[3.MIME] {55}\r\nContent-Type: text/plain\nContent-Disposition: inline\n\r\n\n----------------------------------\nBODY[3.1] {1}\r\nK\nBINARY[3.1] {1}\r\nK\nBINARY.SIZE[3.1] 1\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {2}\r\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {0}\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {46}\r\nContent-Type: multipart/mixed; boundary=\"1\"\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\npe: multipart/mixed; boun\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/007.txt",
    "content": "Subject: RFC 8621 Section 4.1.4 test\nContent-Type: multipart/mixed; boundary=\"1\"\n\n--1\nContent-Type: text/plain\nContent-Disposition: inline\n\nA\n--1  \nContent-Type: multipart/mixed; boundary=\"2\"\n\n--2\nContent-Type: multipart/alternative; boundary=\"3\"\n\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\n--2   \nContent-Type: image/jpeg\nContent-Disposition: attachment\n\nG\n--2  \nContent-Type: application/x-excel\n\nH\n--2  \nContent-Type: message/rfc822\n\nSubject: J\n\nJ\n--2--\n\n--1  \nContent-Type: text/plain\nContent-Disposition: inline\n\nK\n--1--\n\n"
  },
  {
    "path": "tests/resources/imap/008.imap",
    "content": "BODY (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"utf-8\"\n      ) NIL NIL \"7bit\" 54 0\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL \"base64\" 1179 (\n         \"Tue, 14 Dec 2021 11:48:25 +0100\" \"HTML test\" (\n            (\n               \"Name\" NIL \"email\" \"example.com\"\n            )\n         ) (\n            (\n               \"Name\" NIL \"email\" \"example.com\"\n            )\n         ) (\n            (\n               \"Name\" NIL \"email\" \"example.com\"\n            )\n         ) (\n            (\n               \"email@example.com\" NIL \"email\" \"example.com\"\n            )\n         ) NIL NIL NIL \"<random-message-id@example.com>\"\n      ) (\n         (\n            \"text\" \"plain\" (\n               \"charset\" \"utf-8\" \"format\" \"flowed\"\n            ) NIL NIL \"7bit\" 30 0\n         )(\n            \"text\" \"html\" (\n               \"charset\" \"utf-8\"\n            ) NIL NIL \"7bit\" 173 8\n         ) \"alternative\"\n      ) 0\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"plain\" (\n         \"charset\" \"utf-8\"\n      ) NIL NIL \"7bit\" 54 0 \"e377afc895a2c4c0d17b378f355de59e\" NIL NIL NIL\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL \"base64\" 1179 (\n         \"Tue, 14 Dec 2021 11:48:25 +0100\" \"HTML test\" (\n            (\n               \"Name\" NIL \"email\" \"example.com\"\n            )\n         ) (\n            (\n               \"Name\" NIL \"email\" \"example.com\"\n            )\n         ) (\n            (\n               \"Name\" NIL \"email\" \"example.com\"\n            )\n         ) (\n            (\n               \"email@example.com\" NIL \"email\" \"example.com\"\n            )\n         ) NIL NIL NIL \"<random-message-id@example.com>\"\n      ) (\n         (\n            \"text\" \"plain\" (\n               \"charset\" \"utf-8\" \"format\" \"flowed\"\n            ) NIL NIL \"7bit\" 30 0 \"6891396510cbadf4e2cfe31aee5bd25f\" NIL NIL NIL\n         )(\n            \"text\" \"html\" (\n               \"charset\" \"utf-8\"\n            ) NIL NIL \"7bit\" 173 8 \"1a04bd2ec90f44a42792eacdc14fe8ea\" NIL NIL NIL\n         ) \"alternative\" (\n            \"boundary\" \"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"\n         ) NIL \"en-US\" NIL\n      ) 0 \"eb1ac6e049a544c2bda1b7787e03db50\" (\n         \"attachment\" (\n            \"filename\" \"attached_email.eml\"\n         )\n      ) NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\"\n   ) NIL NIL NIL\n)\n\nBODY[] {1649}\r\nContent-Type: multipart/mixed;\n boundary=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\n\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 7bit\n\nThis is a message with a base64 encoded attached email\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Disposition: attachment; filename=\"attached_email.eml\"\nContent-Type: message/rfc822\nContent-Transfer-Encoding: base64\n\nVG86ICJlbWFpbEBleGFtcGxlLmNvbSIgPGVtYWlsQGV4YW1wbGUuY29tPg0KRnJvbTogTmFtZSA8\nZW1haWxAZXhhbXBsZS5jb20+DQpTdWJqZWN0OiBIVE1MIHRlc3QNCk1lc3NhZ2UtSUQ6IDxyYW5k\nb20tbWVzc2FnZS1pZEBleGFtcGxlLmNvbT4NCkRhdGU6IFR1ZSwgMTQgRGVjIDIwMjEgMTE6NDg6\nMjUgKzAxMDANCk1JTUUtVmVyc2lvbjogMS4wDQpDb250ZW50LVR5cGU6IG11bHRpcGFydC9hbHRl\ncm5hdGl2ZTsNCiBib3VuZGFyeT0iYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYSINCkNvbnRlbnQtTGFuZ3VhZ2U6IGVuLVVTDQoNClRoaXMgaXMgYSBtdWx0aS1wYXJ0IG1l\nc3NhZ2UgaW4gTUlNRSBmb3JtYXQuDQotLWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWENCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD11dGYtODsgZm9ybWF0\nPWZsb3dlZA0KQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdA0KDQpUaGlzIGlzIGFuICpI\nVE1MKiB0ZXN0IG1lc3NhZ2UNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYQ0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgNCkNvbnRlbnQtVHJh\nbnNmZXItRW5jb2Rpbmc6IDdiaXQNCg0KPGh0bWw+DQogIDxoZWFkPg0KICAgIDxtZXRhIGh0dHAt\nZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04Ij4N\nCiAgPC9oZWFkPg0KICA8Ym9keT4NCiAgICBUaGlzIGlzIGFuIDxiPkhUTUw8L2I+IHRlc3QgbWVz\nc2FnZQ0KICA8L2JvZHk+DQo8L2h0bWw+DQoNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWFhYWFhYS0tDQo=\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb--\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 1649\n----------------------------------\nBODY[HEADER] {83}\r\nContent-Type: multipart/mixed;\n boundary=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\n\n\n----------------------------------\nBODY[TEXT] {1566}\r\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 7bit\n\nThis is a message with a base64 encoded attached email\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Disposition: attachment; filename=\"attached_email.eml\"\nContent-Type: message/rfc822\nContent-Transfer-Encoding: base64\n\nVG86ICJlbWFpbEBleGFtcGxlLmNvbSIgPGVtYWlsQGV4YW1wbGUuY29tPg0KRnJvbTogTmFtZSA8\nZW1haWxAZXhhbXBsZS5jb20+DQpTdWJqZWN0OiBIVE1MIHRlc3QNCk1lc3NhZ2UtSUQ6IDxyYW5k\nb20tbWVzc2FnZS1pZEBleGFtcGxlLmNvbT4NCkRhdGU6IFR1ZSwgMTQgRGVjIDIwMjEgMTE6NDg6\nMjUgKzAxMDANCk1JTUUtVmVyc2lvbjogMS4wDQpDb250ZW50LVR5cGU6IG11bHRpcGFydC9hbHRl\ncm5hdGl2ZTsNCiBib3VuZGFyeT0iYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYSINCkNvbnRlbnQtTGFuZ3VhZ2U6IGVuLVVTDQoNClRoaXMgaXMgYSBtdWx0aS1wYXJ0IG1l\nc3NhZ2UgaW4gTUlNRSBmb3JtYXQuDQotLWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWENCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD11dGYtODsgZm9ybWF0\nPWZsb3dlZA0KQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdA0KDQpUaGlzIGlzIGFuICpI\nVE1MKiB0ZXN0IG1lc3NhZ2UNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYQ0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgNCkNvbnRlbnQtVHJh\nbnNmZXItRW5jb2Rpbmc6IDdiaXQNCg0KPGh0bWw+DQogIDxoZWFkPg0KICAgIDxtZXRhIGh0dHAt\nZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04Ij4N\nCiAgPC9oZWFkPg0KICA8Ym9keT4NCiAgICBUaGlzIGlzIGFuIDxiPkhUTUw8L2I+IHRlc3QgbWVz\nc2FnZQ0KICA8L2JvZHk+DQo8L2h0bWw+DQoNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWFhYWFhYS0tDQo=\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb--\n\n----------------------------------\nBODY[MIME] {84}\r\nContent-Type: multipart/mixed;\n boundary=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\n\r\n\n----------------------------------\nBODY[1] {54}\r\nThis is a message with a base64 encoded attached email\nBINARY[1] {54}\r\nThis is a message with a base64 encoded attached email\nBINARY.SIZE[1] 54\n----------------------------------\nBODY[1.HEADER] {73}\r\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 7bit\n\n\n----------------------------------\nBODY[1.TEXT] {54}\r\nThis is a message with a base64 encoded attached email\n----------------------------------\nBODY[1.MIME] {74}\r\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 7bit\n\r\n\n----------------------------------\nBODY[1.1] {54}\r\nThis is a message with a base64 encoded attached email\nBINARY[1.1] {54}\r\nThis is a message with a base64 encoded attached email\nBINARY.SIZE[1.1] 54\n----------------------------------\nBODY[2] {1179}\r\nVG86ICJlbWFpbEBleGFtcGxlLmNvbSIgPGVtYWlsQGV4YW1wbGUuY29tPg0KRnJvbTogTmFtZSA8\nZW1haWxAZXhhbXBsZS5jb20+DQpTdWJqZWN0OiBIVE1MIHRlc3QNCk1lc3NhZ2UtSUQ6IDxyYW5k\nb20tbWVzc2FnZS1pZEBleGFtcGxlLmNvbT4NCkRhdGU6IFR1ZSwgMTQgRGVjIDIwMjEgMTE6NDg6\nMjUgKzAxMDANCk1JTUUtVmVyc2lvbjogMS4wDQpDb250ZW50LVR5cGU6IG11bHRpcGFydC9hbHRl\ncm5hdGl2ZTsNCiBib3VuZGFyeT0iYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYSINCkNvbnRlbnQtTGFuZ3VhZ2U6IGVuLVVTDQoNClRoaXMgaXMgYSBtdWx0aS1wYXJ0IG1l\nc3NhZ2UgaW4gTUlNRSBmb3JtYXQuDQotLWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWENCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD11dGYtODsgZm9ybWF0\nPWZsb3dlZA0KQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdA0KDQpUaGlzIGlzIGFuICpI\nVE1MKiB0ZXN0IG1lc3NhZ2UNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYQ0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgNCkNvbnRlbnQtVHJh\nbnNmZXItRW5jb2Rpbmc6IDdiaXQNCg0KPGh0bWw+DQogIDxoZWFkPg0KICAgIDxtZXRhIGh0dHAt\nZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04Ij4N\nCiAgPC9oZWFkPg0KICA8Ym9keT4NCiAgICBUaGlzIGlzIGFuIDxiPkhUTUw8L2I+IHRlc3QgbWVz\nc2FnZQ0KICA8L2JvZHk+DQo8L2h0bWw+DQoNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWFhYWFhYS0tDQo=\nBINARY[2] {16}\r\n[binary content]\nBINARY.SIZE[2] 872\n----------------------------------\nBODY[2.HEADER] {319}\r\nTo: \"email@example.com\" <email@example.com>\r\nFrom: Name <email@example.com>\r\nSubject: HTML test\r\nMessage-ID: <random-message-id@example.com>\r\nDate: Tue, 14 Dec 2021 11:48:25 +0100\r\nMIME-Version: 1.0\r\nContent-Type: multipart/alternative;\r\n boundary=\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"\r\nContent-Language: en-US\r\n\r\n\n----------------------------------\nBODY[2.TEXT] {553}\r\nThis is a multi-part message in MIME format.\r\n--aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\r\nContent-Type: text/plain; charset=utf-8; format=flowed\r\nContent-Transfer-Encoding: 7bit\r\n\r\nThis is an *HTML* test message\r\n--aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\r\nContent-Type: text/html; charset=utf-8\r\nContent-Transfer-Encoding: 7bit\r\n\r\n<html>\r\n  <head>\r\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\r\n  </head>\r\n  <body>\r\n    This is an <b>HTML</b> test message\r\n  </body>\r\n</html>\r\n\r\n--aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa--\r\n\n----------------------------------\nBODY[2.MIME] {128}\r\nContent-Disposition: attachment; filename=\"attached_email.eml\"\nContent-Type: message/rfc822\nContent-Transfer-Encoding: base64\n\r\n\n----------------------------------\nBODY[2.1] {30}\r\nThis is an *HTML* test message\nBINARY[2.1] {30}\r\nThis is an *HTML* test message\nBINARY.SIZE[2.1] 30\n----------------------------------\nBODY[2.1.HEADER] {91}\r\nContent-Type: text/plain; charset=utf-8; format=flowed\r\nContent-Transfer-Encoding: 7bit\r\n\r\n\n----------------------------------\nBODY[2.1.TEXT] {30}\r\nThis is an *HTML* test message\n----------------------------------\nBODY[2.1.MIME] {91}\r\nContent-Type: text/plain; charset=utf-8; format=flowed\r\nContent-Transfer-Encoding: 7bit\r\n\r\n\n----------------------------------\nBODY[2.1.1] {30}\r\nThis is an *HTML* test message\nBINARY[2.1.1] {30}\r\nThis is an *HTML* test message\nBINARY.SIZE[2.1.1] 30\n----------------------------------\nBODY[2.2] {173}\r\n<html>\r\n  <head>\r\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\r\n  </head>\r\n  <body>\r\n    This is an <b>HTML</b> test message\r\n  </body>\r\n</html>\r\n\nBINARY[2.2] {173}\r\n<html>\r\n  <head>\r\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\r\n  </head>\r\n  <body>\r\n    This is an <b>HTML</b> test message\r\n  </body>\r\n</html>\r\n\nBINARY.SIZE[2.2] 173\n----------------------------------\nBODY[2.2.HEADER] {75}\r\nContent-Type: text/html; charset=utf-8\r\nContent-Transfer-Encoding: 7bit\r\n\r\n\n----------------------------------\nBODY[2.2.TEXT] {173}\r\n<html>\r\n  <head>\r\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\r\n  </head>\r\n  <body>\r\n    This is an <b>HTML</b> test message\r\n  </body>\r\n</html>\r\n\n----------------------------------\nBODY[2.2.MIME] {75}\r\nContent-Type: text/html; charset=utf-8\r\nContent-Transfer-Encoding: 7bit\r\n\r\n\n----------------------------------\nBODY[2.2.1] {173}\r\n<html>\r\n  <head>\r\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\r\n  </head>\r\n  <body>\r\n    This is an <b>HTML</b> test message\r\n  </body>\r\n</html>\r\n\nBINARY[2.2.1] {173}\r\n<html>\r\n  <head>\r\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\r\n  </head>\r\n  <body>\r\n    This is an <b>HTML</b> test message\r\n  </body>\r\n</html>\r\n\nBINARY.SIZE[2.2.1] 173\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {2}\r\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {0}\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {84}\r\nContent-Type: multipart/mixed;\n boundary=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\npe: multipart/mixed;\n bou\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/008.txt",
    "content": "Content-Type: multipart/mixed;\n boundary=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\n\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 7bit\n\nThis is a message with a base64 encoded attached email\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Disposition: attachment; filename=\"attached_email.eml\"\nContent-Type: message/rfc822\nContent-Transfer-Encoding: base64\n\nVG86ICJlbWFpbEBleGFtcGxlLmNvbSIgPGVtYWlsQGV4YW1wbGUuY29tPg0KRnJvbTogTmFtZSA8\nZW1haWxAZXhhbXBsZS5jb20+DQpTdWJqZWN0OiBIVE1MIHRlc3QNCk1lc3NhZ2UtSUQ6IDxyYW5k\nb20tbWVzc2FnZS1pZEBleGFtcGxlLmNvbT4NCkRhdGU6IFR1ZSwgMTQgRGVjIDIwMjEgMTE6NDg6\nMjUgKzAxMDANCk1JTUUtVmVyc2lvbjogMS4wDQpDb250ZW50LVR5cGU6IG11bHRpcGFydC9hbHRl\ncm5hdGl2ZTsNCiBib3VuZGFyeT0iYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYSINCkNvbnRlbnQtTGFuZ3VhZ2U6IGVuLVVTDQoNClRoaXMgaXMgYSBtdWx0aS1wYXJ0IG1l\nc3NhZ2UgaW4gTUlNRSBmb3JtYXQuDQotLWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWENCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD11dGYtODsgZm9ybWF0\nPWZsb3dlZA0KQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdA0KDQpUaGlzIGlzIGFuICpI\nVE1MKiB0ZXN0IG1lc3NhZ2UNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYQ0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgNCkNvbnRlbnQtVHJh\nbnNmZXItRW5jb2Rpbmc6IDdiaXQNCg0KPGh0bWw+DQogIDxoZWFkPg0KICAgIDxtZXRhIGh0dHAt\nZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04Ij4N\nCiAgPC9oZWFkPg0KICA8Ym9keT4NCiAgICBUaGlzIGlzIGFuIDxiPkhUTUw8L2I+IHRlc3QgbWVz\nc2FnZQ0KICA8L2JvZHk+DQo8L2h0bWw+DQoNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWFhYWFhYS0tDQo=\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb--\n"
  },
  {
    "path": "tests/resources/imap/009.imap",
    "content": "BODY (\n   (\n      \"text\" \"html\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"base64\" 239 3\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL NIL 723 (\n         NIL \"Exporting my book about coffee tables\" (\n            (\n               \"Cosmo Kramer\" NIL \"kramer\" \"kramerica.com\"\n            )\n         ) (\n            (\n               \"Cosmo Kramer\" NIL \"kramer\" \"kramerica.com\"\n            )\n         ) (\n            (\n               \"Cosmo Kramer\" NIL \"kramer\" \"kramerica.com\"\n            )\n         ) NIL NIL NIL NIL NIL\n      ) (\n         (\n            \"text\" \"plain\" (\n               \"charset\" \"utf-16\"\n            ) NIL NIL \"quoted-printable\" 228 3\n         )(\n            \"image\" \"gif\" (\n               \"name\" \"Book about ☕ tables.gif\"\n            ) NIL NIL \"Base64\" 56\n         ) \"mixed\"\n      ) 0\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"html\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"base64\" 239 3 \"07aab44e51c5f1833a5d19f2e1804c4b\" NIL NIL NIL\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL NIL 723 (\n         NIL \"Exporting my book about coffee tables\" (\n            (\n               \"Cosmo Kramer\" NIL \"kramer\" \"kramerica.com\"\n            )\n         ) (\n            (\n               \"Cosmo Kramer\" NIL \"kramer\" \"kramerica.com\"\n            )\n         ) (\n            (\n               \"Cosmo Kramer\" NIL \"kramer\" \"kramerica.com\"\n            )\n         ) NIL NIL NIL NIL NIL\n      ) (\n         (\n            \"text\" \"plain\" (\n               \"charset\" \"utf-16\"\n            ) NIL NIL \"quoted-printable\" 228 3 \"3a942a99cdd8a099ae107d3867ec20fb\" NIL NIL NIL\n         )(\n            \"image\" \"gif\" (\n               \"name\" \"Book about ☕ tables.gif\"\n            ) NIL NIL \"Base64\" 56 \"d40fa7f401e9dc2df56cbb740d65ff52\" (\n               \"attachment\" NIL\n            ) NIL NIL\n         ) \"mixed\" (\n            \"boundary\" \"giddyup\"\n         ) NIL NIL NIL\n      ) 0 \"cdb0382a03a15601fb1b3c7422521620\" NIL NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"festivus\"\n   ) NIL NIL NIL\n)\n\nBODY[] {1457}\r\nFrom: Art Vandelay <art@vandelay.com> (Vandelay Industries)\nTo: \"Colleagues\": \"James Smythe\" <james@vandelay.com>; Friends:\n    jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?= <john@example.com>;\nDate: Sat, 20 Nov 2021 14:22:01 -0800\nSubject: Why not both importing AND exporting? =?utf-8?b?4pi6?=\nContent-Type: multipart/mixed; boundary=\"festivus\";\n\n--festivus\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: base64\n\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\n--festivus\nContent-Type: message/rfc822\n\nFrom: \"Cosmo Kramer\" <kramer@kramerica.com>\nSubject: Exporting my book about coffee tables\nContent-Type: multipart/mixed; boundary=\"giddyup\";\n\n--giddyup\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n--giddyup\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n--giddyup--\n--festivus--\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 1457\n----------------------------------\nBODY[HEADER] {349}\r\nFrom: Art Vandelay <art@vandelay.com> (Vandelay Industries)\nTo: \"Colleagues\": \"James Smythe\" <james@vandelay.com>; Friends:\n    jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?= <john@example.com>;\nDate: Sat, 20 Nov 2021 14:22:01 -0800\nSubject: Why not both importing AND exporting? =?utf-8?b?4pi6?=\nContent-Type: multipart/mixed; boundary=\"festivus\";\n\n\n----------------------------------\nBODY[TEXT] {1108}\r\n--festivus\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: base64\n\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\n--festivus\nContent-Type: message/rfc822\n\nFrom: \"Cosmo Kramer\" <kramer@kramerica.com>\nSubject: Exporting my book about coffee tables\nContent-Type: multipart/mixed; boundary=\"giddyup\";\n\n--giddyup\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n--giddyup\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n--giddyup--\n--festivus--\n\n----------------------------------\nBODY[MIME] {54}\r\nContent-Type: multipart/mixed; boundary=\"festivus\";\n\r\n\n----------------------------------\nBODY[1] {239}\r\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\nBINARY[1] {175}\r\n<html><p>I was thinking about quitting the &ldquo;exporting&rdquo; to focus just on the &ldquo;importing&rdquo;,</p><p>but then I thought, why not do both? &#x263A;</p></html>\nBINARY.SIZE[1] 175\n----------------------------------\nBODY[1.HEADER] {79}\r\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: base64\n\n\n----------------------------------\nBODY[1.TEXT] {239}\r\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\n----------------------------------\nBODY[1.MIME] {80}\r\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: base64\n\r\n\n----------------------------------\nBODY[1.1] {239}\r\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\nBINARY[1.1] {175}\r\n<html><p>I was thinking about quitting the &ldquo;exporting&rdquo; to focus just on the &ldquo;importing&rdquo;,</p><p>but then I thought, why not do both? &#x263A;</p></html>\nBINARY.SIZE[1.1] 175\n----------------------------------\nBODY[2] {723}\r\nFrom: \"Cosmo Kramer\" <kramer@kramerica.com>\nSubject: Exporting my book about coffee tables\nContent-Type: multipart/mixed; boundary=\"giddyup\";\n\n--giddyup\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n--giddyup\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n--giddyup--\nBINARY[2] {16}\r\n[binary content]\nBINARY.SIZE[2] 723\n----------------------------------\nBODY[2.HEADER] {143}\r\nFrom: \"Cosmo Kramer\" <kramer@kramerica.com>\nSubject: Exporting my book about coffee tables\nContent-Type: multipart/mixed; boundary=\"giddyup\";\n\n\n----------------------------------\nBODY[2.TEXT] {580}\r\n--giddyup\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n--giddyup\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n--giddyup--\n----------------------------------\nBODY[2.MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[2.1] {228}\r\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\nBINARY[2.1] {101}\r\nℌ𝔢𝔩𝔭 𝔪𝔢 𝔢𝔵𝔭𝔬𝔯𝔱 𝔪𝔶 𝔟𝔬𝔬𝔨 𝔭𝔩𝔢𝔞𝔰𝔢!\nBINARY.SIZE[2.1] 101\n----------------------------------\nBODY[2.1.HEADER] {88}\r\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n\n----------------------------------\nBODY[2.1.TEXT] {228}\r\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n----------------------------------\nBODY[2.1.MIME] {89}\r\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\r\n\n----------------------------------\nBODY[2.1.1] {228}\r\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\nBINARY[2.1.1] {101}\r\nℌ𝔢𝔩𝔭 𝔪𝔢 𝔢𝔵𝔭𝔬𝔯𝔱 𝔪𝔶 𝔟𝔬𝔬𝔨 𝔭𝔩𝔢𝔞𝔰𝔢!\nBINARY.SIZE[2.1.1] 101\n----------------------------------\nBODY[2.2] {56}\r\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\nBINARY[2.2] {16}\r\n[binary content]\nBINARY.SIZE[2.2] 42\n----------------------------------\nBODY[2.2.HEADER] {175}\r\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\n\n----------------------------------\nBODY[2.2.TEXT] {56}\r\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n----------------------------------\nBODY[2.2.MIME] {176}\r\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\r\n\n----------------------------------\nBODY[2.2.1] {56}\r\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\nBINARY[2.2.1] {16}\r\n[binary content]\nBINARY.SIZE[2.2.1] 42\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {196}\r\nFrom: Art Vandelay <art@vandelay.com> (Vandelay Industries)\nTo: \"Colleagues\": \"James Smythe\" <james@vandelay.com>; Friends:\n    jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?= <john@example.com>;\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\nVandelay <art@vandelay.co\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {286}\r\nFrom: Art Vandelay <art@vandelay.com> (Vandelay Industries)\nTo: \"Colleagues\": \"James Smythe\" <james@vandelay.com>; Friends:\n    jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?= <john@example.com>;\nDate: Sat, 20 Nov 2021 14:22:01 -0800\nContent-Type: multipart/mixed; boundary=\"festivus\";\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\nVandelay <art@vandelay.co\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/009.txt",
    "content": "From: Art Vandelay <art@vandelay.com> (Vandelay Industries)\nTo: \"Colleagues\": \"James Smythe\" <james@vandelay.com>; Friends:\n    jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?= <john@example.com>;\nDate: Sat, 20 Nov 2021 14:22:01 -0800\nSubject: Why not both importing AND exporting? =?utf-8?b?4pi6?=\nContent-Type: multipart/mixed; boundary=\"festivus\";\n\n--festivus\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: base64\n\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\n--festivus\nContent-Type: message/rfc822\n\nFrom: \"Cosmo Kramer\" <kramer@kramerica.com>\nSubject: Exporting my book about coffee tables\nContent-Type: multipart/mixed; boundary=\"giddyup\";\n\n--giddyup\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n--giddyup\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n--giddyup--\n--festivus--\n"
  },
  {
    "path": "tests/resources/imap/010.imap",
    "content": "BODY (\n   \"message\" \"rfc822\" NIL NIL NIL NIL 88 (\n      \"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) NIL NIL NIL NIL NIL\n   ) (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 12 1\n   ) 0\n)\n\nBODYSTRUCTURE (\n   \"message\" \"rfc822\" NIL NIL NIL NIL 88 (\n      \"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) NIL NIL NIL NIL NIL\n   ) (\n      \"text\" \"plain\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 12 1 \"f0ef7081e1539ac00ef5b761b4fb01b3\" NIL NIL NIL\n   ) 0 \"dfea5a78f321b331e6d7983a1f9cc6b7\" NIL NIL NIL\n)\n\nBODY[] {196}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 88\n----------------------------------\nBODY[HEADER] {108}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\n\n----------------------------------\nBODY[TEXT] {88}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n----------------------------------\nBODY[MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[1] {88}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\nBINARY[1] {16}\r\n[binary content]\nBINARY.SIZE[1] 88\n----------------------------------\nBODY[1.HEADER] {76}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\n\n----------------------------------\nBODY[1.TEXT] {12}\r\nHello world\n\n----------------------------------\nBODY[1.MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[1.1] {12}\r\nHello world\n\nBINARY[1.1] {12}\r\nHello world\n\nBINARY.SIZE[1.1] 12\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {24}\r\nFrom: user@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {14}\r\n@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {109}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMIME-Version: 1.0\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\n@domain.org\nDate: Sat, 24\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/010.txt",
    "content": "From: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n"
  },
  {
    "path": "tests/resources/imap/011.imap",
    "content": "BODY (\n   \"message\" \"rfc822\" NIL NIL NIL NIL 271 (\n      \"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) NIL NIL NIL NIL NIL\n   ) (\n      (\n         \"message\" NIL NIL NIL NIL NIL 42 (\n            NIL \"m1\" (\n               (\n                  NIL NIL \"m1\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m1\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m1\" \"example.com\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 8 1\n         ) 0\n      )(\n         \"message\" NIL NIL NIL NIL NIL 42 (\n            NIL \"m2\" (\n               (\n                  NIL NIL \"m2\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m2\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m2\" \"example.com\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 8 1\n         ) 0\n      ) \"digest\"\n   ) 0\n)\n\nBODYSTRUCTURE (\n   \"message\" \"rfc822\" NIL NIL NIL NIL 271 (\n      \"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) (\n         (\n            NIL NIL \"sub\" \"domain.org\"\n         )\n      ) NIL NIL NIL NIL NIL\n   ) (\n      (\n         \"message\" NIL NIL NIL NIL NIL 42 (\n            NIL \"m1\" (\n               (\n                  NIL NIL \"m1\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m1\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m1\" \"example.com\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 8 1 \"8dc313ad8cf1d82dbe8d46f5f0d3d79c\" NIL NIL NIL\n         ) 0 \"702907ad1c165219425153a8f0a5f578\" NIL NIL NIL\n      )(\n         \"message\" NIL NIL NIL NIL NIL 42 (\n            NIL \"m2\" (\n               (\n                  NIL NIL \"m2\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m2\" \"example.com\"\n               )\n            ) (\n               (\n                  NIL NIL \"m2\" \"example.com\"\n               )\n            ) NIL NIL NIL NIL NIL\n         ) (\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 8 1 \"f344a10ee7adfdcfc29650b6e31601d8\" NIL NIL NIL\n         ) 0 \"0c79449f982ccecbc258d902cd989f69\" NIL NIL NIL\n      ) \"digest\" (\n         \"boundary\" \"foo\"\n      ) NIL NIL NIL\n   ) 0 \"4935800d6cfad87d931093820097206a\" NIL NIL NIL\n)\n\nBODY[] {379}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nX-Mime: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\nepilogue\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 260\n----------------------------------\nBODY[HEADER] {108}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\n\n----------------------------------\nBODY[TEXT] {271}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nX-Mime: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\nepilogue\n\n----------------------------------\nBODY[MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[1] {271}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nX-Mime: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\nepilogue\n\nBINARY[1] {16}\r\n[binary content]\nBINARY.SIZE[1] 260\n----------------------------------\nBODY[1.HEADER] {123}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\n\n----------------------------------\nBODY[1.TEXT] {137}\r\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nX-Mime: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n----------------------------------\nBODY[1.MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[1.1] {42}\r\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\nBINARY[1.1] {16}\r\n[binary content]\nBINARY.SIZE[1.1] 42\n----------------------------------\nBODY[1.1.HEADER] {34}\r\nFrom: m1@example.com\nSubject: m1\n\n\n----------------------------------\nBODY[1.1.TEXT] {8}\r\nm1 body\n\n----------------------------------\nBODY[1.1.MIME] {2}\r\n\r\n\n----------------------------------\nBODY[1.1.1] {8}\r\nm1 body\n\nBINARY[1.1.1] {8}\r\nm1 body\n\nBINARY.SIZE[1.1.1] 8\n----------------------------------\nBODY[1.2] {42}\r\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\nBINARY[1.2] {16}\r\n[binary content]\nBINARY.SIZE[1.2] 42\n----------------------------------\nBODY[1.2.HEADER] {34}\r\nFrom: m2@example.com\nSubject: m2\n\n\n----------------------------------\nBODY[1.2.TEXT] {8}\r\nm2 body\n\n----------------------------------\nBODY[1.2.MIME] {2}\r\n\r\n\n----------------------------------\nBODY[1.2.1] {8}\r\nm2 body\n\nBINARY[1.2.1] {8}\r\nm2 body\n\nBINARY.SIZE[1.2.1] 8\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {24}\r\nFrom: user@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {14}\r\n@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {109}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMIME-Version: 1.0\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\n@domain.org\nDate: Sat, 24\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/011.txt",
    "content": "From: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nX-Mime: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\nepilogue\n"
  },
  {
    "path": "tests/resources/imap/012.imap",
    "content": "BODY (\n   (\n      \"text\" \"x-myown\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 6 1\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL NIL 280 (\n         \"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" (\n            (\n               NIL NIL \"sub\" \"domain.org\"\n            )\n         ) (\n            (\n               NIL NIL \"sub\" \"domain.org\"\n            )\n         ) (\n            (\n               NIL NIL \"sub\" \"domain.org\"\n            )\n         ) NIL NIL NIL NIL NIL\n      ) (\n         (\n            \"text\" \"html\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 19 1\n         )(\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 20 1\n         ) \"alternative\"\n      ) 0\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"x-myown\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"7bit\" 6 1 \"b1946ac92492d2347c6235b4d2611184\" NIL NIL NIL\n   )(\n      \"message\" \"rfc822\" NIL NIL NIL NIL 280 (\n         \"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" (\n            (\n               NIL NIL \"sub\" \"domain.org\"\n            )\n         ) (\n            (\n               NIL NIL \"sub\" \"domain.org\"\n            )\n         ) (\n            (\n               NIL NIL \"sub\" \"domain.org\"\n            )\n         ) NIL NIL NIL NIL NIL\n      ) (\n         (\n            \"text\" \"html\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 19 1 \"35c5b687e359e8ce7be1f1ecafd9b475\" NIL NIL NIL\n         )(\n            \"text\" \"plain\" (\n               \"charset\" \"us-ascii\"\n            ) NIL NIL \"7bit\" 20 1 \"deeff770fadb17c664d431b97bcd05c5\" NIL NIL NIL\n         ) \"alternative\" (\n            \"boundary\" \"sub1\"\n         ) NIL NIL NIL\n      ) 0 \"735ed696bc05fdf6840de404781d5d77\" NIL NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"foo bar\"\n   ) NIL NIL NIL\n)\n\nBODY[] {565}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 565\n----------------------------------\nBODY[HEADER] {130}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\n\n----------------------------------\nBODY[TEXT] {435}\r\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n\n----------------------------------\nBODY[MIME] {53}\r\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\r\n\n----------------------------------\nBODY[1] {6}\r\nhello\n\nBINARY[1] {6}\r\nhello\n\nBINARY.SIZE[1] 6\n----------------------------------\nBODY[1.HEADER] {46}\r\nContent-Type: text/x-myown; charset=us-ascii\n\n\n----------------------------------\nBODY[1.TEXT] {6}\r\nhello\n\n----------------------------------\nBODY[1.MIME] {47}\r\nContent-Type: text/x-myown; charset=us-ascii\n\r\n\n----------------------------------\nBODY[1.1] {6}\r\nhello\n\nBINARY[1.1] {6}\r\nhello\n\nBINARY.SIZE[1.1] 6\n----------------------------------\nBODY[2] {280}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\nBINARY[2] {16}\r\n[binary content]\nBINARY.SIZE[2] 280\n----------------------------------\nBODY[2.HEADER] {129}\r\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\n\n----------------------------------\nBODY[2.TEXT] {151}\r\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n----------------------------------\nBODY[2.MIME] {31}\r\nContent-Type: message/rfc822\n\r\n\n----------------------------------\nBODY[2.1] {19}\r\n<p>Hello world</p>\n\nBINARY[2.1] {19}\r\n<p>Hello world</p>\n\nBINARY.SIZE[2.1] 19\n----------------------------------\nBODY[2.1.HEADER] {25}\r\nContent-Type: text/html\n\n\n----------------------------------\nBODY[2.1.TEXT] {19}\r\n<p>Hello world</p>\n\n----------------------------------\nBODY[2.1.MIME] {26}\r\nContent-Type: text/html\n\r\n\n----------------------------------\nBODY[2.1.1] {19}\r\n<p>Hello world</p>\n\nBINARY[2.1.1] {19}\r\n<p>Hello world</p>\n\nBINARY.SIZE[2.1.1] 19\n----------------------------------\nBODY[2.2] {20}\r\nHello another world\n\nBINARY[2.2] {20}\r\nHello another world\n\nBINARY.SIZE[2.2] 20\n----------------------------------\nBODY[2.2.HEADER] {26}\r\nContent-Type: text/plain\n\n\n----------------------------------\nBODY[2.2.TEXT] {20}\r\nHello another world\n\n----------------------------------\nBODY[2.2.MIME] {27}\r\nContent-Type: text/plain\n\r\n\n----------------------------------\nBODY[2.2.1] {20}\r\nHello another world\n\nBINARY[2.2.1] {20}\r\nHello another world\n\nBINARY.SIZE[2.2.1] 20\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {24}\r\nFrom: user@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {14}\r\n@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {131}\r\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMIME-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\n@domain.org\nDate: Sat, 24\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/012.txt",
    "content": "From: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n"
  },
  {
    "path": "tests/resources/imap/013.imap",
    "content": "BODY (\n   \"text\" \"plain\" (\n      \"charset\" \"us-ascii\"\n   ) NIL NIL \"7bit\" 356 13\n)\n\nBODYSTRUCTURE (\n   \"text\" \"plain\" (\n      \"charset\" \"us-ascii\"\n   ) NIL NIL \"7bit\" 356 13 \"77beb490b61fa7ed17f997c4124a57bf\" NIL NIL NIL\n)\n\nBODY[] {697}\r\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n\nBINARY[] {356}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n\nBINARY.SIZE[] 356\n----------------------------------\nBODY[HEADER] {341}\r\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\n\n----------------------------------\nBODY[TEXT] {356}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n\n----------------------------------\nBODY[MIME] {77}\r\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\r\n\n----------------------------------\nBODY[1] {356}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n\nBINARY[1] {356}\r\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n\nBINARY.SIZE[1] 356\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {107}\r\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {25}\r\n Clinton <president@white\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {298}\r\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-ID: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nMIME-Version: 1.0\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\n 13 Aug 1998 17:42:41 +10\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/013.txt",
    "content": "Date: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n"
  },
  {
    "path": "tests/resources/imap/014.imap",
    "content": "BODY (\n   (\n      \"text\" \"x-myown\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"quoted-printable\" 79 15\n   ) \"mixed\"\n)\n\nBODYSTRUCTURE (\n   (\n      \"text\" \"x-myown\" (\n         \"charset\" \"us-ascii\"\n      ) NIL NIL \"quoted-printable\" 79 15 \"22839bb2efefde05dda98625a9ed8875\" NIL NIL NIL\n   ) \"mixed\" (\n      \"boundary\" \"foo bar\"\n   ) NIL NIL NIL\n)\n\nBODY[] {404}\r\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\n\nhello  \nbar=\n\nfoo\t=\nbar\nfoo\t \t= \n=62\nfoo = \t\nbar\nfoo =\n=62\nfoo  \nbar=\n\nfoo_bar\n\n--foo bar--\nRoot MIME epilogue\n\nBINARY[] {16}\r\n[binary content]\nBINARY.SIZE[] 404\n----------------------------------\nBODY[HEADER] {173}\r\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\n\n----------------------------------\nBODY[TEXT] {231}\r\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\n\nhello  \nbar=\n\nfoo\t=\nbar\nfoo\t \t= \n=62\nfoo = \t\nbar\nfoo =\n=62\nfoo  \nbar=\n\nfoo_bar\n\n--foo bar--\nRoot MIME epilogue\n\n----------------------------------\nBODY[MIME] {53}\r\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\r\n\n----------------------------------\nBODY[1] {79}\r\nhello  \nbar=\n\nfoo\t=\nbar\nfoo\t \t= \n=62\nfoo = \t\nbar\nfoo =\n=62\nfoo  \nbar=\n\nfoo_bar\n\nBINARY[1] {56}\r\nhello\nbar\nfoo\tbar\nfoo\t \tb\nfoo bar\nfoo b\nfoo\nbar\nfoo_bar\n\nBINARY.SIZE[1] 56\n----------------------------------\nBODY[1.HEADER] {90}\r\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\n\n\n----------------------------------\nBODY[1.TEXT] {79}\r\nhello  \nbar=\n\nfoo\t=\nbar\nfoo\t \t= \n=62\nfoo = \t\nbar\nfoo =\n=62\nfoo  \nbar=\n\nfoo_bar\n\n----------------------------------\nBODY[1.MIME] {91}\r\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\n\r\n\n----------------------------------\nBODY[1.1] {79}\r\nhello  \nbar=\n\nfoo\t=\nbar\nfoo\t \t= \n=62\nfoo = \t\nbar\nfoo =\n=62\nfoo  \nbar=\n\nfoo_bar\n\nBINARY[1.1] {56}\r\nhello\nbar\nfoo\tbar\nfoo\t \tb\nfoo bar\nfoo b\nfoo\nbar\nfoo_bar\n\nBINARY.SIZE[1.1] 56\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)] {24}\r\nFrom: user@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS (FROM TO)]<10> {14}\r\n@domain.org\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)] {174}\r\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMIME-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\r\n\n----------------------------------\nBODY[HEADER.FIELDS.NOT (SUBJECT CC)]<10> {25}\r\ndomain  Fri Feb 22 17:06:\n----------------------------------\n"
  },
  {
    "path": "tests/resources/imap/014.txt",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\n\nhello  \nbar=\n\nfoo\t=\nbar\nfoo\t \t= \n=62\nfoo = \t\nbar\nfoo =\n=62\nfoo  \nbar=\n\nfoo_bar\n\n--foo bar--\nRoot MIME epilogue\n"
  },
  {
    "path": "tests/resources/imap-test/append",
    "content": "connections: 3\nstate: created\n\n1 ok select $mailbox\n2 ok select $mailbox\n\n# Two connections have mailbox SELECTed, one doesn't.\n# The \\recent flags can be given to either one of the SELECTed connections,\n# but never for the 3rd. We rely on mailbox state tracking to catch duplicate\n# \\recent flags (which is why there are two FETCH FLAGS commands).\n\n1 ok append $mailbox (\\seen \\flagged)\n* 1 exists \n2 ok noop\n* 1 exists\n3 ok status $mailbox (messages unseen recent)\n* status $mailbox (messages 1 unseen 0 recent 0)\n1 ok fetch 1 (uid flags)\n* 1 fetch (uid $uid1 flags (\\seen \\flagged))\n2 ok fetch 1 (uid flags)\n* 1 fetch (uid $uid1 flags (\\seen \\flagged))\n\n2 ok append\n* 2 exists\n1 ok noop\n* 2 exists\n3 ok status $mailbox (messages unseen recent)\n* status $mailbox (messages 2 unseen 1 recent 0)\n1 ok fetch 2 (uid flags)\n2 ok fetch 2 (uid flags)\n\n3 ok append\n3 ok status $mailbox (messages unseen)\n* status $mailbox (messages 3 unseen 2)\n2 ok noop\n* 3 exists\n1 ok noop\n* 3 exists\n1 ok fetch 3 (uid flags)\n2 ok fetch 3 (uid flags)\n\n1 append ${mailbox}nonexistent\nno [trycreate]\n"
  },
  {
    "path": "tests/resources/imap-test/append-binary",
    "content": "capabilities: BINARY\nstate: created\n\nok append $mailbox ~{{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: application/octet-stream\nContent-Transfer-Encoding: binary\n\nnil\n\n\n\n}}}\n\nok select $mailbox\n\n# should have been converted to base64, or something\nok fetch 1 (body.peek[1])\n! 1 fetch (body[1] {{{\nnil\n\n}}})\n! 1 fetch (body[1] {{{\nnil\n\n}}})\n! 1 fetch (body[1] ~{{{\nnil\n\n\n}}})\n\nok fetch 1 (binary.size[1] binary.peek[1])\n! 1 fetch (binary.size[1] 6 binary[1] ~{{{\nnil\n\n}}})\n"
  },
  {
    "path": "tests/resources/imap-test/atoms",
    "content": "state: auth\n\n# Don't confuse with ~{literal8}\nok list \"\" ~foo\nno select ~foo\n\n# atom-specials   = \"(\" / \")\" / \"{\" / SP / CTL / list-wildcards /\n#                   quoted-specials / resp-specials\n# quoted-specials = DQUOTE / \"\\\"\n# resp-specials   = \"]\"\n# list-wildcards  = \"%\" / \"*\"\nno select !#$$&'+,-.0123456789:;<=>?@^_`|[}\nok list \"\" !#$$&'+,-.0123456789:;<=>?@^_`|[}%*\n"
  },
  {
    "path": "tests/resources/imap-test/broken/search-intdate",
    "content": "# this test assumes that server stores INTERNALDATE timestamps in\n# EET/EEST timezones (or stores/fetches them using the same timezones as\n# they were APPENDed with)\n\n# 1) Timezone changes from EET +0200 -> EEST +0300\n# 1a) BEFORE\nok search before 24-mar-2007\n* search\nok search before 25-mar-2007\n* search 1\nok search before 26-mar-2007\n* search 1 2 3 4 5 6\nok search before 27-mar-2007\n* search 1 2 3 4 5 6 7\n\n# 1b) ON\nok search on 23-mar-2007\n* search\nok search on 24-mar-2007\n* search 1\nok search on 25-mar-2007\n* search 2 3 4 5 6\nok search on 26-mar-2007\n* search 7\n\n# 1c) SINCE\nok search 1:7 since 24-mar-2007\n* search 1 2 3 4 5 6 7\nok search 1:7 since 25-mar-2007\n* search 2 3 4 5 6 7\nok search 1:7 since 26-mar-2007\n* search 7\nok search 1:7 since 27-mar-2007\n* search\n\n# 2) Timezone changes from EEST +0300 -> EET +0200\n# 2a) BEFORE\nok search 8:* before 27-oct-2007\n* search\nok search 8:* before 28-oct-2007\n* search 8\nok search 8:* before 29-oct-2007\n* search 8 9 10 11 12 13 14 15\nok search 8:* before 30-oct-2007\n* search 8 9 10 11 12 13 14 15 16\n\n# 2b) ON\nok search 8:* on 26-oct-2007\n* search\nok search 8:* on 27-oct-2007\n* search 8\nok search 8:* on 28-oct-2007\n* search 9 10 11 12 13 14 15\nok search 8:* on 29-oct-2007\n* search 16\n\n# 2c) SINCE\nok search 8:* since 27-oct-2007\n* search 8 9 10 11 12 13 14 15 16\nok search 8:* since 28-oct-2007\n* search 9 10 11 12 13 14 15 16\nok search 8:* since 29-oct-2007\n* search 16\nok search 8:* since 30-oct-2007\n* search\n\n# 3) Try a couple of NOTs\nok search 1:7 not before 26-mar-2007\n* search 7\nok search 1:7 not on 25-mar-2007\n* search 1 7\nok search 8:* not since 28-oct-2007\n* search 8\nok search 8:* not on 28-oct-2007\n* search 8 16\n"
  },
  {
    "path": "tests/resources/imap-test/broken/search-intdate.mbox",
    "content": "From user@domain  Sat Mar 24 23:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Mar 25 00:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Mar 25 01:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Mar 25 02:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Mar 25 04:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Mar 25 23:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Mon Mar 26 00:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sat Oct 27 23:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Oct 28 00:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Oct 28 01:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Oct 28 02:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Oct 28 03:00:00 2007 +0300\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Oct 28 03:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Oct 28 04:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Sun Oct 28 23:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Mon Oct 29 00:00:00 2007 +0200\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\n"
  },
  {
    "path": "tests/resources/imap-test/catenate",
    "content": "state: created\ncapabilities: CATENATE\n\n\"\" delete ${mailbox}2\n\nok append $mailbox (\\seen \\flagged) catenate (text {{{\nFrom: foo@example.com\n\nHello world body\n\n}}})\n\nok append $mailbox (\\seen \\flagged) catenate (text {{{\nFrom: foo2@example.com\n\nLookslike: header\n\nAnother body\n\n}}})\n\nok select $mailbox\n* ok [uidvalidity $uidv]\n\nok fetch 1:2 (uid body.peek[])\n* 1 fetch (uid $uid body[] {{{\nFrom: foo@example.com\n\nHello world body\n\n}}})\n* 2 fetch (uid $uid2 body[] {{{\nFrom: foo2@example.com\n\nLookslike: header\n\nAnother body\n\n}}})\n\nok create ${mailbox}2\n\nok append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url/;uid=$uid/;section=header\" text {{{\nbody1\n\n}}})\n\nok append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url;uidvalidity=$uidv/;uid=$uid/;section=header\" text {{{\nbody2\n\n}}})\n\nok append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url/;uid=$uid2/;section=text\" text {{{\nbody3\n\n}}})\n\nok append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url/;uid=$uid2/;section=1\" text {{{\nbody4\n\n}}})\n\nok append ${mailbox}2 (\\seen \\flagged) catenate (text {{{\nFrom: new@example.com\nSubject: test header\n\n\n}}} url \"/$mailbox_url/;uid=$uid/;section=1\" text {{{\n\nsuffix\n\n}}})\n\nok append ${mailbox}2 (\\seen \\flagged) catenate (text {{{\nFrom: hdr1@example.com\n\n}}} text {{{\nSubject: hdr2\n\n}}} text {{{\n\nbody6\n\n}}})\n\nok append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url/;uid=$uid2/;partial=100000.1\" text {{{\nHdr: foo\n\nbody7\n\n}}})\n\n#\n# Try invalid URLs\n#\n\nno append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" text {{{\nbody\n\n}}})\n\nno append ${mailbox}2 (\\seen \\flagged) catenate (text {{{\nhdr: 1\n\n}}} url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\")\n\nno append ${mailbox}2 (\\seen \\flagged) catenate (text {{{\nhdr: 1\n\n}}} url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" text {{{\nhdr: 2\n\n}}})\n\nno append ${mailbox}2 (\\seen \\flagged) catenate (text {{{\nhdr: 1\n\n}}} url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" text {{{\nhdr: 2\n\n}}} url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" text {{{\nhdr: 3\n\n}}})\n\nno append ${mailbox}2 (\\seen \\flagged) catenate (text {{{\nhdr: 1\n\n}}} url \"/$mailbox_url/;uid=$uid/;section=header\" text {{{\nhdr: 2\n\n}}} url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" text {{{\nhdr: 3\n\n}}})\n\nno append ${mailbox}2 (\\seen \\flagged) catenate (text {{{\nhdr: 1\n\n}}} url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" text {{{\nhdr: 2\n\n}}} url \"/$mailbox_url/;uid=$uid/;section=header\" text {{{\nhdr: 3\n\n}}})\n\nno append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" url \"/$mailbox_url/;uid=$uid/;section=header\")\nno append ${mailbox}2 (\\seen \\flagged) catenate (url \"/$mailbox_url/;uid=$uid/;section=header\" url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\")\n\n#\n# verify previous appends\n#\nok select ${mailbox}2\nok fetch 1:* body.peek[]\n* 1 fetch (body[] {{{\nFrom: foo@example.com\n\nbody1\n\n}}})\n* 2 fetch (body[] {{{\nFrom: foo@example.com\n\nbody2\n\n}}})\n* 3 fetch (body[] {{{\nLookslike: header\n\nAnother body\nbody3\n\n}}})\n* 4 fetch (body[] {{{\nLookslike: header\n\nAnother body\nbody4\n\n}}})\n* 5 fetch (body[] {{{\nFrom: new@example.com\nSubject: test header\n\nHello world body\n\nsuffix\n\n}}})\n* 6 fetch (body[] {{{\nFrom: hdr1@example.com\nSubject: hdr2\n\nbody6\n\n}}})\n* 7 fetch (body[] {{{\nHdr: foo\n\nbody7\n\n}}})\n\n\n#\n# Try appending to nonexistent mailbox\n#\n\nappend ${mailbox}nonexistent catenate (url \"/$mailbox_url/;uid=$uid/;section=1\")\nno [trycreate]\n\nappend ${mailbox}nonexistent catenate (url \"/$mailbox_url/;uid=$uid/;section=1\" text {{{\nhello\n}}})\nno [trycreate]\n\nappend ${mailbox}nonexistent catenate (text {{{\nhello\n}}} url \"/$mailbox_url/;uid=$uid/;section=1\")\nno [trycreate]\n\nappend ${mailbox}nonexistent (\\seen \\flagged) catenate (text {{{\nhdr1: 1\n\n}}} url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\" text {{{\nhdr2: 2\n\n}}} url \"/$mailbox_url/;uid=$uid/;section=header\" text {{{\nhdr3: 3\n\n}}})\nno [trycreate]\n\nok noop\n"
  },
  {
    "path": "tests/resources/imap-test/catenate-multiappend",
    "content": "state: created\ncapabilities: CATENATE MULTIAPPEND\n\nok append $mailbox catenate (text {{{\nFrom: foo@example.com\n\nHello world body\n\n}}}) catenate (text {{{\nFrom: bar@example.com\n\nSecond body\n\n}}})\n\nok select $mailbox\n* ok [uidvalidity $uidv]\n\nok fetch 1:2 (uid body.peek[])\n* 1 fetch (uid $uid body[] {{{\nFrom: foo@example.com\n\nHello world body\n\n}}})\n* 2 fetch (uid $uid2 body[] {{{\nFrom: bar@example.com\n\nSecond body\n\n}}})\n\nok append ${mailbox} catenate (url \"/$mailbox_url/;uid=$uid/;section=header\" text {{{\nbody1\n\n}}}) catenate (url \"/$mailbox_url/;uid=$uid2/;section=header\" text {{{\nbody2\n\n}}})\n\nok append ${mailbox} catenate (url \"/$mailbox_url/;uid=$uid\") catenate (url \"/$mailbox_url/;uid=$uid2/;section=header\" text {{{\nbody3\n\n}}})\n\nok append ${mailbox} catenate (url \"/$mailbox_url/;uid=$uid\") {{{\nNew: Message\n\nbody4\n\n}}}\n\nok append ${mailbox} catenate (url \"/$mailbox_url/;uid=$uid\") (\\answered) {{{\nNew: Message\n\nbody5\n\n}}}\n\nok noop\n\nok fetch 3:* body.peek[]\n* 3 fetch (body[] {{{\nFrom: foo@example.com\n\nbody1\n\n}}})\n* 4 fetch (body[] {{{\nFrom: bar@example.com\n\nbody2\n\n}}})\n* 5 fetch (body[] {{{\nFrom: foo@example.com\n\nHello world body\n\n}}})\n* 6 fetch (body[] {{{\nFrom: bar@example.com\n\nbody3\n\n}}})\n* 7 fetch (body[] {{{\nFrom: foo@example.com\n\nHello world body\n\n}}})\n* 8 fetch (body[] {{{\nNew: Message\n\nbody4\n\n}}})\n* 9 fetch (body[] {{{\nFrom: foo@example.com\n\nHello world body\n\n}}})\n* 10 fetch (body[] {{{\nNew: Message\n\nbody5\n\n}}})\n\n#\n# Try invalid URLs\n#\n\nno append ${mailbox} (\\seen \\flagged) catenate (url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\") catenate (url \"/$mailbox_url/;uid=$uid\")\n\nno append ${mailbox} (\\seen \\flagged) catenate (url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\") catenate (url \"/$mailbox_url/;uid=$uid\" text {{{\nhello\n}}})\n\nno append ${mailbox} (\\seen \\flagged) catenate (url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\") catenate (text {{{\nhello\n}}})\n\nno append ${mailbox} (\\seen \\flagged) catenate (url \"/$mailbox_url;uidvalidity=12345/;uid=$uid/;section=header\") {{{\nhello\n}}}\n\nok noop\n"
  },
  {
    "path": "tests/resources/imap-test/close",
    "content": "connections: 2\nmessages: 3\n\n1 ok store 1,3 +flags \\deleted\n1 ok close\n\n2 ok noop\n* $1 expunge\n* $3 expunge\n"
  },
  {
    "path": "tests/resources/imap-test/copy",
    "content": "connections: 2\nstate: created\n\n1 ok select ${mailbox}\n1 ok append\n1 ok append\n1 ok append\n1 ok append\n1 ok append\n# make sure the server sees the appended messages\n1 ok check\n\n2 \"\" delete ${mailbox}2\n2 ok create ${mailbox}2\n2 ok select ${mailbox}2\n\n1 ok store 1 flags (\\seen)\n1 ok store 2 flags (\\answered \\flagged)\n1 ok store 5 flags (\\flagged $$keyword1 $$keyword2)\n\n1 ok fetch 1:5 (internaldate)\n* 1 fetch (internaldate $date1)\n* 2 fetch (internaldate $date2)\n* 4 fetch (internaldate $date4)\n* 5 fetch (internaldate $date5)\n1 ok copy 1:2,4 ${mailbox}2\n\n2 ok noop\n* 3 exists\n#* 3 recent\n\n2 ok fetch 1:3 (flags internaldate)\n? 1 fetch (flags (\\seen) internaldate $date1)\n? 2 fetch (flags (\\answered \\flagged) internaldate $date2)\n? 3 fetch (flags () internaldate $date4)\n\n# keywords aren't required to be created on COPY, so help the server here\n2 ok store 3 +flags ($$keyword1 $$keyword2)\n\n1 ok copy 5 ${mailbox}2\n2 ok noop\n* 4 exists\n\n2 ok fetch 4 (flags internaldate)\n* 4 fetch (flags (\\flagged $$keyword1 $$keyword2) internaldate $date5)\n\n2 ok close\n2 \"\" delete ${mailbox}2\n"
  },
  {
    "path": "tests/resources/imap-test/default.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user1@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s1\n\nbody1\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user2@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s22\n\nbody22\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user3@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s333\n\nbody33\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user4@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s4444\n\nbody4444\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user5@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s55555\n\nbody55555\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user6@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s666666\n\nbody666666\n\n"
  },
  {
    "path": "tests/resources/imap-test/esearch",
    "content": "capabilities: ESEARCH\nmessages: all\n\nok store 4 +flags \\deleted\nok expunge\n\n# SEARCH ALL\n\nok search return (all) all\n* esearch (tag $tag) all 1:6\nok search return () all\n* esearch (tag $tag) all 1:6\nok search return (min) all\n* esearch (tag $tag) min 1\nok search return (max) all\n* esearch (tag $tag) max 6\nok search return (count) all\n* esearch (tag $tag) count 6\n\n# UID SEARCH ALL\n\nok fetch 1:* UID\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n* 3 fetch (uid $uid3)\n* 4 fetch (uid $uid4)\n* 5 fetch (uid $uid5)\n* 6 fetch (uid $uid6)\n\nok uid search return (all) all\n* esearch (tag $tag) uid all $uid1:$uid3,$uid4:$uid6\nok uid search return () all\n* esearch (tag $tag) uid all $uid1:$uid3,$uid4:$uid6\nok uid search return (min) all\n* esearch (tag $tag) uid min $uid1\nok uid search return (max) all\n* esearch (tag $tag) uid max $uid6\nok uid search return (count) all\n* esearch (tag $tag) uid count 6\n\n# \\Seen flag test\n\nok store 2,4 +flags \\seen\n\nok uid search return (all) seen\n* esearch (tag $tag) uid all $uid2,$uid4\nok uid search return () seen\n* esearch (tag $tag) uid all $uid2,$uid4\nok uid search return (min) seen\n* esearch (tag $tag) uid min $uid2\nok uid search return (max) seen\n* esearch (tag $tag) uid max $uid4\nok uid search return (count) seen\n* esearch (tag $tag) uid count 2\n\n# nonexistent\n\nok search return () 1000\n* esearch (tag $tag)\nok search return (min) 1000\n* esearch (tag $tag)\nok search return (max) 1000\n* esearch (tag $tag)\nok search return (count) 1000\n* esearch (tag $tag) count 0\n\n# UID nonexistent\n\nok uid search return () 1000\n* esearch (tag $tag) uid\nok uid search return (min) 1000\n* esearch (tag $tag) uid\nok uid search return (max) 1000\n* esearch (tag $tag) uid\nok uid search return (count) 1000\n* esearch (tag $tag) uid count 0\n"
  },
  {
    "path": "tests/resources/imap-test/esearch-condstore",
    "content": "capabilities: ESEARCH CONDSTORE\nmessages: 4\n\n# ENABLE is valid only in authenticated state. we could do enable+select\n# manually here also, but lets just make sure that switching it on via\n# condstore-enabling command works as well (since that is valid)\nok fetch 1 modseq\n* ok [highestmodseq $highestmodseq]\n\nok store 1 +flags \\seen\n* 1 fetch (modseq ($modseq1))\nok store 3 +flags \\seen\n* 3 fetch (modseq ($modseq3))\nok store 2 +flags \\seen\n* 2 fetch (modseq ($modseq2))\nok store 4 +flags \\seen\n* 4 fetch (modseq ($modseq4))\n\nok search return (min) 1:3 modseq \"/flags/\\\\seen\" all $highestmodseq\n* esearch (tag $tag) min 1 modseq $modseq4\n\nok search return (max) 1:3 modseq \"/flags/\\\\seen\" all $highestmodseq\n* esearch (tag $tag) max 3 modseq $modseq4\n\nok search return () 1:3 modseq \"/flags/\\\\seen\" all $highestmodseq\n* esearch (tag $tag) all 1:3 modseq $modseq4\n\nok search return (min max) 2:3 modseq \"/flags/\\\\seen\" all $highestmodseq\n* esearch (tag $tag) min 2 max 3 modseq $modseq4\n\nok search return (all) 2:4 modseq \"/flags/\\\\seen\" all $modseq3\n* esearch (tag $tag) all 2,4 modseq $modseq4\n\nok search return (all) 2:4 modseq \"/flags/\\\\seen\" all $modseq2\n* esearch (tag $tag) all 4 modseq $modseq4\n\nok search return (all) 2:4 modseq \"/flags/\\\\seen\" all $modseq4\n* esearch (tag $tag) modseq $modseq4\n"
  },
  {
    "path": "tests/resources/imap-test/esearch.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:21 2008\n\n1\n\nFrom user@domain  Fri Feb 22 17:06:22 2008\n\n2\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\n\n3\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\n\n4\n\nFrom user@domain  Fri Feb 22 17:06:25 2008\n\n5\n\nFrom user@domain  Fri Feb 22 17:06:25 2008\n\n6\n\nFrom user@domain  Fri Feb 22 17:06:25 2008\n\n7\n\n"
  },
  {
    "path": "tests/resources/imap-test/esort",
    "content": "capabilities: ESORT\nmessages: 5\n\nok sort return (min) (arrival) us-ascii 1:5\n* esearch (tag $tag) min 1\nok sort return (max) (arrival) us-ascii 1:5\n* esearch (tag $tag) max 5\nok sort return (all) (arrival) us-ascii 1:5\n* esearch (tag $tag) all 1:5\nok sort return (count) (arrival) us-ascii 1:5\n* esearch (tag $tag) count 5\n"
  },
  {
    "path": "tests/resources/imap-test/expunge",
    "content": "connections: 2\nmessages: 8\n\n# get UIDs\n1 ok fetch 1:4 uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n* 3 fetch (uid $uid3)\n* 4 fetch (uid $uid4)\n\n# 1) test that expunges work ok and session 2 fetch sees 1's flag changes.\n1 ok store 1,3 flags \\deleted\n* 1 fetch (flags (\\deleted))\n* 3 fetch (flags (\\deleted))\n1 ok store 2,4 flags \\seen\n* 2 fetch (flags (\\seen))\n* 4 fetch (flags (\\seen))\n1 ok expunge\n* $1 expunge\n* $3 expunge\n\n2 ok fetch 2,4 (uid)\n* 2 fetch (uid $uid2)\n* 4 fetch (uid $uid4)\n2 ok fetch 2,4 (uid)\n* 2 fetch (uid $uid2)\n* 4 fetch (uid $uid4)\n2 ok check\n* $1 expunge\n* $3 expunge\n2 ok fetch 1:2 (uid flags)\n* 1 fetch (uid $uid2 flags (\\seen))\n* 2 fetch (uid $uid4 flags (\\seen))\n\n# 2) test that session 2 can update flags while some messages are expunged\n1 ok store 2 +flags \\deleted\n1 ok expunge\n\n2 ok store 1 flags \\answered\n* 1 fetch (flags (\\answered))\n1 ok check\n* 1 fetch (flags (\\answered))\n2 ok noop\n* 2 expunge\n\n# 3) check notices flag changes correctly with expunges\n1 ok store 1,3 +flags \\deleted\n1 ok store 2,4 flags \\flagged\n1 ok expunge\n2 ok check\n* $1 expunge\n* $2 fetch (flags (\\flagged))\n* $3 expunge\n* $4 fetch (flags (\\flagged))\n\n# 4) expunging while message is already expunged\n1 ok store 1 +flags \\deleted\n2 ok store 1 +flags \\deleted\n1 ok expunge\n* 1 expunge\n2 ok expunge\n* 1 expunge\n"
  },
  {
    "path": "tests/resources/imap-test/expunge2",
    "content": "connections: 2\nmessages: 6\n\n1 ok fetch 2,4 uid\n* 2 fetch (uid $uid2)\n* 4 fetch (uid $uid4)\n\n# UID FETCH\n1 ok store 1,3 +flags \\deleted\n1 ok expunge\n* $1 expunge\n* $3 expunge\n\n2 ok uid fetch $uid2,$uid4 uid\n* $2 fetch (uid $uid2)\n* $4 fetch (uid $uid4)\n2 ok noop\n\n# UID STORE\n1 ok store 1 +flags \\deleted\n1 ok expunge\n* 1 expunge\n\n2 ok uid store $uid4 flags \\seen\n* $2 fetch (uid $uid4 flags (\\seen))\n2 ok noop\n\n# Make sure CHECK works just as well as NOOP\n1 ok store 1 +flags \\deleted\n1 ok expunge\n* 1 expunge\n2 ok check\n* 1 expunge\n\n# Make sure FETCH, STORE and SEARCH don't trigger EXPUNGE\n1 ok store 1 +flags \\deleted\n1 ok expunge\n\n2 ok fetch 2 flags\n! $1 expunge\n2 ok store 2 flags (\\seen)\n! $1 expunge\n2 ok search all\n! $1 expunge\n"
  },
  {
    "path": "tests/resources/imap-test/fetch",
    "content": "messages: 3\n\nok status $mailbox (uidnext)\n* status $mailbox (uidnext $uidnext)\n\nok fetch 1:3 uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n* 3 fetch (uid $uid3)\n\nok fetch 3:1 uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n* 3 fetch (uid $uid3)\n\nok fetch 1,* uid\n* 1 fetch (uid $uid1)\n* 3 fetch (uid $uid3)\n\nok fetch * uid\n* 3 fetch (uid $uid3)\n\nok fetch *:1 uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n* 3 fetch (uid $uid3)\n\nok uid fetch $uidnext:* uid\n* 3 fetch (uid $uid3)\n\n# break seq=uid map\nok store 2 flags \\deleted\nok expunge\n* 2 expunge\n\nok uid fetch $uidnext:* uid\n* 2 fetch (uid $uid3)\nok uid fetch 1:* uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid3)\nok uid fetch $uid2 uid\n! 1 fetch (uid $uid1)\n! 2 fetch (uid $uid3)\nok uid fetch $uid1,* uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid3)\n\n# test macros\nok fetch 1 full\n* 1 fetch (flags () internaldate $intdate1 rfc822.size $size1 envelope ($!unordered) body ($!unordered))\nok fetch 1 all\n* 1 fetch (flags () internaldate $intdate1 rfc822.size $size1 envelope ($!unordered))\nok fetch 1 fast\n* 1 fetch (flags () internaldate $intdate1 rfc822.size $size1)\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-binary-mime",
    "content": "capabilities: BINARY\nmessages: all\n\n# This is the fetch-body-mime test, except with body[] changed to binary[].\n# The idea is to verify that binary[] works correctly when it doesn't actually\n# have to convert anything.\n\nok fetch 1 (binary.peek[])\n* 1 fetch (binary[] {{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n\n}}})\n\nok fetch 1 (binary.size[])\n* 1 fetch (binary.size[] 602)\n\nok fetch 1 (binary.size[1])\n* 1 fetch (binary.size[1] 7)\n\nok fetch 1 (binary.peek[1])\n* 1 fetch (binary[1] {{{\nhello\n\n}}})\n\nok fetch 1 (binary.peek[2])\n* 1 fetch (binary[2] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n}}})\n\nok fetch 1 (binary.size[2])\n* 1 fetch (binary.size[2] 298)\n\nok fetch 1 (binary.size[2.1])\n* 1 fetch (binary.size[2.1] 20)\n\nok fetch 1 (binary.peek[2.1])\n* 1 fetch (binary[2.1] {{{\n<p>Hello world</p>\n\n}}})\n\nok fetch 1 (binary.peek[2.2])\n* 1 fetch (binary[2.2] {{{\nHello another world\n\n}}})\n\nok fetch 1 (binary.size[2.2])\n* 1 fetch (binary.size[2.2] 21)\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-binary-mime-base64",
    "content": "capabilities: BINARY\nmessages: all\n\nok fetch 1 (binary.size[1] binary.size[5])\n* 1 fetch (binary.size[1] 11 binary.size[5] 30)\n\nok fetch 1 (binary.peek[1] binary.peek[2] binary.peek[3] binary.peek[4] binary.peek[5])\n* 1 fetch (binary[1] {{{\nhello world\n}}} binary[2] {{{\nhello world\n}}} binary[3] {{{\nhello to you too\n}}} binary[4] {{{\nhello to everyone!\n}}} binary[5] ~{{{\nabcdefg\rhijkl\r\nmno\npqrstuvqxyz\n}}})\n\nok fetch 1 (binary.size[1] binary.peek[1])\n* 1 fetch (binary.size[1] 11 binary[1] {{{\nhello world\n}}})\n\nok fetch 1 (binary.size[3] binary.peek[2])\n* 1 fetch (binary.size[3] 16 binary[2] {{{\nhello world\n}}})\n\nok fetch 1 (binary.size[2] binary.size[3] binary.size[4])\n* 1 fetch (binary.size[2] 11 binary.size[3] 16 binary.size[4] 18)\n\nok fetch 1 (binary.peek[5]<0.7>)\n* 1 fetch (binary[5]<0> ~{{{\nabcdefg\n}}})\n\nok fetch 1 (binary.peek[5]<0.8>)\n* 1 fetch (binary[5]<0> ~{{{\nabcdefg\r\n}}})\n\nok fetch 1 (binary.peek[5]<0.10>)\n* 1 fetch (binary[5]<0> ~{{{\nabcdefg\rhi\n}}})\n\nok fetch 1 (binary.peek[5]<10.10>)\n* 1 fetch (binary[5]<10> ~{{{\njkl\r\nmno\np\n}}})\n\nok fetch 1 (binary.peek[5]<5.10>)\n* 1 fetch (binary[5]<5> ~{{{\nfg\rhijkl\r\n\n}}})\n\nok fetch 1 (binary.peek[5]<15.100>)\n* 1 fetch (binary[5]<15> ~{{{\nmno\npqrstuvqxyz\n}}})\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-binary-mime-base64.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: base64\n\n\naGVs\nbG8gd29y\nbGQ=\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: base64\n\n  aGVs    \n bG8g  \n   d29y  \t \n    bGQ= \n\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: base64\n\naGVsbG8g\ndG8geW91IHRv\nbw==\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: base64\n\naGVsbG8gdG8gZXZlcnlvbmUh\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: base64\n\nYWJjZGVmZw1oaWprbA0KbW5vCnBxcnN0dXZxeHl6\n\n--foo bar--\nRoot MIME epilogue\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-binary-mime-qp",
    "content": "capabilities: BINARY\nmessages: all\n\nok fetch 1 (binary.size[1])\n* 1 fetch (binary.size[1] 65)\n\nok fetch 1 (binary.peek[1])\n* 1 fetch (binary[1] {{{\nhello\nbar\nfoo\tbar\nfoo\t \tb\nfoo bar\nfoo b\nfoo\nbar\nfoo_bar\n\n}}})\n\nok fetch 1 (binary.peek[1]<0.10>)\n* 1 fetch (binary[1]<0> {{{\nhello\nbar\n}}})\n\nok fetch 1 (binary.peek[1]<10.10>)\n* 1 fetch (binary[1]<10> ~{{{\n\r\nfoo\tbar\r\n}}})\n\nok fetch 1 (binary.peek[1]<20.10>)\n* 1 fetch (binary[1]<20> ~{{{\n\nfoo\t \tb\r\n\n}}})\n\nok fetch 1 (binary.peek[1]<15.10>)\n* 1 fetch (binary[1]<15> ~{{{\n\tbar\r\nfoo\t\n}}})\n\nok fetch 1 (binary.peek[1]<40.100>)\n* 1 fetch (binary[1]<40> {{{\noo b\nfoo\nbar\nfoo_bar\n\n}}})\n\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-binary-mime-qp.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\n\nhello  \nbar=\n\nfoo\t=\nbar\nfoo\t \t= \n=62\nfoo = \t\nbar\nfoo =\n=62\nfoo  \nbar=\n\nfoo_bar\n\n--foo bar--\nRoot MIME epilogue\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-binary-mime.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body",
    "content": "messages: all\n\n# header and body fetches\nok fetch 1 rfc822.header\n* 1 fetch (rfc822.header $hdr1)\nok fetch 1 body.peek[header]\n* 1 fetch (body[header] $hdr1)\n\nok fetch 1 (flags body.peek[text])\n* 1 fetch (flags () body[text] {{{\nbody1\n\n\n}}})\nok fetch 1 rfc822.text\n* 1 fetch (rfc822.text {{{\nbody1\n\n\n}}})\n* 1 fetch (flags (\\seen))\n\nok fetch 2 (flags body.peek[])\n* 2 fetch (flags () body[] $full2)\nok fetch 2 rfc822\n* 2 fetch (rfc822 $full2)\n* 2 fetch (flags (\\seen))\n\nok fetch 3 (body[])\n* 3 fetch (body[] $full3)\n* 3 fetch (flags (\\seen))\nok fetch 4 (body[header])\n* 4 fetch (body[header] $hdr4)\n* 4 fetch (flags (\\seen))\n\n# partial fetches\nok fetch 2 body.peek[text]<0.3>\n* 2 fetch (body[text]<0> \"bod\")\nok fetch 2 body.peek[text]<3.3>\n* 2 fetch (body[text]<3> \"y22\")\n\nok fetch 3 body.peek[text]<0.1>\n* 3 fetch (body[text]<0> \"b\")\nok fetch 3 body.peek[text]<5.1>\n* 3 fetch (body[text]<5> \"3\")\nok fetch 3 body.peek[text]<5.2>\n* 3 fetch (body[text]<5> ~{{{\n3\r\n}}})\nok fetch 3 body.peek[text]<5.3>\n* 3 fetch (body[text]<5> ~{{{\n3\r\n\n}}})\nok fetch 3 body.peek[text]<6.1>\n* 3 fetch (body[text]<6> ~{{{\n\r\n}}})\nok fetch 3 body.peek[text]<6.2>\n* 3 fetch (body[text]<6> ~{{{\n\r\n\n}}})\nok fetch 3 body.peek[text]<7.1>\n* 3 fetch (body[text]<7> ~{{{\n\n\n}}})\n\n# header fields\nok fetch 1 body.peek[header.fields (from)]\n* 1 fetch (body[header.fields (from)] {{{\nFrom: User1 <user1@domain.org>\n\n\n}}})\n\nok fetch 1 (body.peek[header.fields (from)])\n* 1 fetch (body[header.fields (from)] {{{\nFrom: User1 <user1@domain.org>\n\n\n}}})\n\nok fetch 1 (body.peek[header.fields (from from)])\n* 1 fetch (body[header.fields (from from)] {{{\nFrom: User1 <user1@domain.org>\n\n\n}}})\n\nok fetch 1 body.peek[header.fields (from subject)]\n* 1 fetch (body[header.fields (from subject)] {{{\nFrom: User1 <user1@domain.org>\nSubject: s1\n\n\n}}})\n\nok fetch 1 body.peek[header.fields.not (date)]\n! 1 fetch (body[header.fields.not (date)] {{{\nFrom: User1 <user1@domain.org>\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s1\n\n\n}}})\n\nok fetch 1 body.peek[header.fields.not (date date)]\n! 1 fetch (body[header.fields.not (date date)] {{{\nFrom: User1 <user1@domain.org>\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s1\n\n\n}}})\n\nok fetch 1 body.peek[header.fields (xyz)]\n* 1 fetch (body[header.fields (xyz)] {{{\n\n\n}}})\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-message-rfc822",
    "content": "messages: all\n\nok fetch 1 (body.peek[])\n* 1 fetch (body[] {{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[text])\n* 1 fetch (body[text] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1])\n* 1 fetch (body[1] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1.1])\n* 1 fetch (body[1.1] {{{\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1.header])\n* 1 fetch (body[1.header] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\n\n}}})\n\nok fetch 1 (body.peek[1.header.fields (from subject X-foo)])\n* 1 fetch (body[1.header.fields (from subject X-foo)] {{{\nFrom: sub@domain.org\nSubject: submsg\n\n\n}}})\n\nok fetch 1 (body.peek[1.header.fields.not (from subject X-foo)])\n* 1 fetch (body[1.header.fields.not (from subject X-foo)] {{{\nDate: Sun, 12 Aug 2012 12:34:56 +0300\n\n\n}}})\n\nok fetch 1 (body.peek[1.text])\n* 1 fetch (body[1.text] {{{\nHello world\n\n}}})\n\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-message-rfc822-mime",
    "content": "messages: all\n\nok fetch 1 (body.peek[])\n* 1 fetch (body[] {{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nContent-Custom: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\n\n}}})\n\nok fetch 1 (body.peek[text])\n* 1 fetch (body[text] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nContent-Custom: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\n\n}}})\n\nok fetch 1 (body.peek[1])\n* 1 fetch (body[1] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nContent-Custom: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\n\n}}})\n\nok fetch 1 (body.peek[1.header])\n* 1 fetch (body[1.header] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\n\n}}})\n\nok fetch 1 (body.peek[1.text])\n* 1 fetch (body[1.text] {{{\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nContent-Custom: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n}}})\n\nok fetch 1 (body.peek[1.1])\n* 1 fetch (body[1.1] {{{\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n}}})\n\nok fetch 1 (body.peek[1.1.MIME])\n* 1 fetch (body[1.1.MIME] {{{\n\n\n}}})\n\nok fetch 1 (body.peek[1.1.HEADER])\n* 1 fetch (body[1.1.HEADER] {{{\nFrom: m1@example.com\nSubject: m1\n\n\n}}})\n\nok fetch 1 (body.peek[1.1.TEXT])\n* 1 fetch (body[1.1.TEXT] {{{\nm1 body\n\n}}})\n\nok fetch 1 (body.peek[1.2])\n* 1 fetch (body[1.2] {{{\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n}}})\n\nok fetch 1 (body.peek[1.2.MIME])\n* 1 fetch (body[1.2.MIME] {{{\nContent-Custom: m2 header\n\n\n}}})\n\nok fetch 1 (body.peek[1.2.HEADER])\n* 1 fetch (body[1.2.HEADER] {{{\nFrom: m2@example.com\nSubject: m2\n\n\n}}})\n\nok fetch 1 (body.peek[1.2.TEXT])\n* 1 fetch (body[1.2.TEXT] {{{\nm2 body\n\n}}})\n\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-message-rfc822-mime.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/digest; boundary=\"foo\"\n\nprologue\n\n--foo \n\nFrom: m1@example.com\nSubject: m1\n\nm1 body\n\n--foo \nContent-Custom: m2 header\n\nFrom: m2@example.com\nSubject: m2\n\nm2 body\n\n--foo--\n\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-message-rfc822-x2",
    "content": "messages: all\n\nok fetch 1 (body.peek[])\n* 1 fetch (body[] {{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: user2@domain.org\nDate: Fri, 23 Mar 2007 11:22:33 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[text])\n* 1 fetch (body[text] {{{\nFrom: user2@domain.org\nDate: Fri, 23 Mar 2007 11:22:33 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1])\n* 1 fetch (body[1] {{{\nFrom: user2@domain.org\nDate: Fri, 23 Mar 2007 11:22:33 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1.1])\n* 1 fetch (body[1.1] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1.header])\n* 1 fetch (body[1.header] {{{\nFrom: user2@domain.org\nDate: Fri, 23 Mar 2007 11:22:33 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\n\n}}})\n\nok fetch 1 (body.peek[1.text])\n* 1 fetch (body[1.text] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1.1.1])\n* 1 fetch (body[1.1.1] {{{\nHello world\n\n}}})\n\nok fetch 1 (body.peek[1.1.header])\n* 1 fetch (body[1.1.header] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\n\n}}})\n\nok fetch 1 (body.peek[1.1.text])\n* 1 fetch (body[1.1.text] {{{\nHello world\n\n}}})\n\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-message-rfc822-x2.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: user2@domain.org\nDate: Fri, 23 Mar 2007 11:22:33 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-message-rfc822.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\n\nHello world\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-mime",
    "content": "messages: all\n\nok fetch 1 (body.peek[])\n* 1 fetch (body[] {{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n\n}}})\n\nok fetch 1 (body.peek[text])\n* 1 fetch (body[text] {{{\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n\n}}})\n\nok fetch 1 (body.peek[1])\n* 1 fetch (body[1] {{{\nhello\n\n}}})\n\nok fetch 1 (body.peek[1.mime])\n* 1 fetch (body[1.mime] {{{\nContent-Type: text/x-myown; charset=us-ascii\n\n\n}}})\n\nok fetch 1 (body.peek[2])\n* 1 fetch (body[2] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n}}})\n\nok fetch 1 (body.peek[2.mime])\n* 1 fetch (body[2.mime] {{{\nContent-Type: message/rfc822\n\n\n}}})\n\nok fetch 1 (body.peek[2.header])\n* 1 fetch (body[2.header] {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\n\n}}})\n\nok fetch 1 (body.peek[2.header.fields (from subject X-foo)])\n* 1 fetch (body[2.header.fields (from subject X-foo)] {{{\nFrom: sub@domain.org\nSubject: submsg\n\n\n}}})\n\nok fetch 1 (body.peek[2.header.fields.not (from subject X-foo)])\n* 1 fetch (body[2.header.fields.not (from subject X-foo)] {{{\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\n\n}}})\n\nok fetch 1 (body.peek[2.text])\n* 1 fetch (body[2.text] {{{\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n}}})\n\nok fetch 1 (body.peek[2.1.mime])\n* 1 fetch (body[2.1.mime] {{{\nContent-Type: text/html\n\n\n}}})\n\nok fetch 1 (body.peek[2.1])\n* 1 fetch (body[2.1] {{{\n<p>Hello world</p>\n\n}}})\n\nok fetch 1 (body.peek[2.2.mime])\n* 1 fetch (body[2.2.mime] {{{\nContent-Type: text/plain\n\n\n}}})\n\nok fetch 1 (body.peek[2.2])\n* 1 fetch (body[2.2] {{{\nHello another world\n\n}}})\n\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body-mime.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/html\n\n<p>Hello world</p>\n\n--sub1\nContent-Type: text/plain\n\nHello another world\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-body.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: User1 <user1@domain.org>\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s1\n\nbody1\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: User2 <user2@domain.org>\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s22\n\nbody22\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: User3 <user3@domain.org>\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s333\n\nbody33\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: User4 <user4@domain.org>\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: s4444\n\nbody4444\n\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-bodystructure",
    "content": "messages: all\n\nok fetch 1:* body\n* 1 FETCH (BODY ((\"TEXT\" \"x-MYOWN\" (\"CHARSET\" \"us-ascii\") NIL NIL \"7BIT\" 7 1) \"MIXED\"))\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-bodystructure.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\n--foo bar\nContent-Type: text/x-myown; charset=us-ascii\n\nhello\n\n--foo bar--\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-envelope",
    "content": "messages: all\n\n# date, subject, from, sender, reply-to, to, cc, bcc, in-reply-to, message-id\nok fetch 1:* envelope\n* 1 FETCH (ENVELOPE (\"Thu, 15 Feb 2007 01:02:03 +0200\" \"subject header\" ((\"From Real\" NIL \"fromuser\" \"fromdomain.org\")) ((\"Sender Real\" NIL \"senderuser\" \"senderdomain.org\")) ((\"ReplyTo Real\" NIL \"replytouser\" \"replytodomain.org\")) ((\"To Real\" NIL \"touser\" \"todomain.org\")) ((\"Cc Real\" NIL \"ccuser\" \"ccdomain.org\")) ((\"Bcc Real\" NIL \"bccuser\" \"bccdomain.org\")) \"<reply@to.id>\" \"<msg@id>\"))\n* 2 FETCH (ENVELOPE (\"Thu, 15 Feb 2007 01:02:03 +0200\" NIL ((NIL NIL \"user\" \"domain\")) ((NIL NIL \"user\" \"domain\")) ((NIL NIL \"user\" \"domain\")) NIL NIL NIL NIL NIL))\n* 3 FETCH (ENVELOPE (\"Thu, 15 Feb 2007 01:02:03 +0200\" NIL ((NIL NIL \"user\" \"domain\")) ((NIL NIL \"user\" \"domain\")) ((NIL NIL \"user\" \"domain\")) NIL NIL NIL NIL NIL))\n* 4 FETCH (ENVELOPE (\"Thu, 15 Feb 2007 01:02:03 +0200\" NIL ((\"Real Name\" NIL \"user\" \"domain\")) ((\"Real Name\" NIL \"user\" \"domain\")) ((\"Real Name\" NIL \"user\" \"domain\")) ((NIL NIL \"group\" NIL)(NIL NIL \"g1\" \"d1.org\")(NIL NIL \"g2\" \"d2.org\")(NIL NIL NIL NIL)(NIL NIL \"group2\" NIL)(NIL NIL \"g3\" \"d3.org\")(NIL NIL NIL NIL)) ((NIL NIL \"group\" NIL)(NIL NIL NIL NIL)(NIL NIL \"group2\" NIL)(NIL NIL NIL NIL)) NIL NIL NIL))\n* 5 FETCH (ENVELOPE (\"Thu, 15 Feb 2007 01:02:03 +0200\" NIL ((\"Real Name\" NIL \"user\" \"domain\")) ((\"Real Name\" NIL \"user\" \"domain\")) ((\"Real Name\" NIL \"user\" \"domain\")) NIL NIL NIL NIL NIL))\n* 6 FETCH (ENVELOPE (\"Thu, 15 Feb 2007 01:02:03 +0200\" NIL ((NIL \"@route\" \"user\" \"domain\")) ((NIL \"@route\" \"user\" \"domain\")) ((NIL \"@route\" \"user\" \"domain\")) NIL NIL NIL NIL NIL))\n"
  },
  {
    "path": "tests/resources/imap-test/fetch-envelope.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nMessage-ID: <msg@id>\nIn-Reply-To: <reply@to.id>\nDate: Thu, 15 Feb 2007 01:02:03 +0200\nSubject: subject header\nFrom: From Real <fromuser@fromdomain.org>\nTo: To Real <touser@todomain.org>\nCc: Cc Real <ccuser@ccdomain.org>\nBcc: Bcc Real <bccuser@bccdomain.org>\nSender: Sender Real <senderuser@senderdomain.org>\nReply-To: ReplyTo Real <replytouser@replytodomain.org>\n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Thu, 15 Feb 2007 01:02:03 +0200\nFrom: user@domain\n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Thu, 15 Feb 2007 01:02:03 +0200\nFrom: user@domain\n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Thu, 15 Feb 2007 01:02:03 +0200\nFrom: user@domain (Real Name)\nTo: group: g1@d1.org, g2@d2.org;, group2: g3@d3.org;\nCc: group:;, group2: (foo) ;\n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Thu, 15 Feb 2007 01:02:03 +0200\nFrom: user@domain (Real Name)\nSender: \nReply-To: \n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Thu, 15 Feb 2007 01:02:03 +0200\nFrom: <@route:user@domain>\n\nbody\n"
  },
  {
    "path": "tests/resources/imap-test/id",
    "content": "capabilities: id\nstate: nonauth\n\nok ID (\"a23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"b23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"c23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"d23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"e23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"f23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"g23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"h23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"i23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"j23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"k23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"l23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"m23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"n23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"o23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"p23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"q23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"r23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"s23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"t23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"u23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"v23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"w23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"x23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"y23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"z23456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"aa3456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"ab3456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"ac3456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\" \"ad3456789012345678901234567890\" \"1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\")\n"
  },
  {
    "path": "tests/resources/imap-test/list",
    "content": "connections: 3\nstate: auth\n\n# get the separator\n1 ok list \"\" \"\"\n* list () $sep $root\n# it should be \\noselect, but don't fail everything if it doesn't exist\n* list (\\noselect) $sep $root\n\n1 ok create $mailbox${sep}\n1 ok create $mailbox${sep}test\n2 ok list \"\" $mailbox${sep}%\n* list () $sep $mailbox${sep}test\n\n2 ok create $mailbox${sep}test2\n1 ok list \"\" $mailbox${sep}%\n* list () $sep $mailbox${sep}test\n* list () $sep $mailbox${sep}test2\n\n3 ok create $mailbox${sep}test3${sep}\n2 ok create $mailbox${sep}test3${sep}test4${sep}\n2 ok create $mailbox${sep}test3${sep}test4${sep}test5\n2 ok list \"\" $mailbox${sep}%\n* list () $sep $mailbox${sep}test\n* list () $sep $mailbox${sep}test2\n* list () $sep $mailbox${sep}test3\n! list () $sep $mailbox${sep}test3${sep}test4\n! list () $sep $mailbox${sep}test3${sep}test4${sep}test5\n\n3 ok list \"\" $mailbox${sep}%\n* list () $sep $mailbox${sep}test\n* list () $sep $mailbox${sep}test2\n* list () $sep $mailbox${sep}test3\n\n2 ok list \"\" $mailbox${sep}%${sep}%\n! list () $sep $mailbox${sep}test\n! list () $sep $mailbox${sep}test2\n! list () $sep $mailbox${sep}test3\n* list () $sep $mailbox${sep}test3${sep}test4\n! list () $sep $mailbox${sep}test3${sep}test4${sep}test5\n\n3 ok list \"\" $mailbox${sep}*\n* list () $sep $mailbox${sep}test\n* list () $sep $mailbox${sep}test2\n* list () $sep $mailbox${sep}test3\n* list () $sep $mailbox${sep}test3${sep}test4${sep}test5\n\n3 ok list $mailbox${sep} *\n* list () $sep $mailbox${sep}test\n* list () $sep $mailbox${sep}test2\n* list () $sep $mailbox${sep}test3\n* list () $sep $mailbox${sep}test3${sep}test4${sep}test5\n\n2 ok list \"\" $mailbox${sep}*test4\n! list () $sep $mailbox${sep}test\n! list () $sep $mailbox${sep}test2\n! list () $sep $mailbox${sep}test3\n* list () $sep $mailbox${sep}test3${sep}test4\n! list () $sep $mailbox${sep}test3${sep}test4${sep}test5\n\n2 ok list \"\" $mailbox${sep}*test*\n* list () $sep $mailbox${sep}test\n* list () $sep $mailbox${sep}test2\n* list () $sep $mailbox${sep}test3\n* list () $sep $mailbox${sep}test3${sep}test4${sep}test5\n\n2 ok list \"\" $mailbox${sep}%3${sep}%\n* list () $sep $mailbox${sep}test3${sep}test4\n2 ok list \"\" $mailbox${sep}%3${sep}%4\n* list () $sep $mailbox${sep}test3${sep}test4\n2 ok list \"\" $mailbox${sep}%t*4\n* list () $sep $mailbox${sep}test3${sep}test4\n\n2 ok delete $mailbox${sep}test2\n1 ok list \"\" $mailbox${sep}*\n! list () $sep $mailbox${sep}test2\n\n1 ok list \"\" INBOX\n* list () $inboxsep INBOX\n"
  },
  {
    "path": "tests/resources/imap-test/listext",
    "content": "state: auth\ncapabilities: LIST-EXTENDED\n\n# get the separator\nok list \"\" \"\"\n* list () $sep \"\"\n\nok create $mailbox${sep}\nok create $mailbox${sep}test1\nok create $mailbox${sep}test2${sep}\nok create $mailbox${sep}test2${sep}test1\nok create $mailbox${sep}test3\nok create $mailbox${sep}test4${sep}test5\nok create $mailbox${sep}test4${sep}test52\nok create $mailbox${sep}test6\nok create $mailbox${sep}test7${sep}test7\nok create $mailbox${sep}test8${sep}test8\nok create $mailbox${sep}test8${sep}test9\n\nok subscribe $mailbox${sep}test1\nok subscribe $mailbox${sep}test2${sep}test1\nok subscribe $mailbox${sep}test6\nok subscribe $mailbox${sep}test7${sep}test7\nok subscribe $mailbox${sep}test8${sep}test8\nok subscribe $mailbox${sep}test8${sep}test9\n\nok delete $mailbox${sep}test6\n\n# \"\" isn't a special case with LIST-EXTENDED\n#ok list () \"\" \"\"\n#! list () $sep \"\"\n#ok list \"\" (\"\")\n#! list () $sep \"\"\n#ok list \"\" \"\" return ()\n#! list () $sep \"\"\n\n# test multiple patterns\nok list \"\" ($mailbox${sep}*2 $mailbox${sep}test3)\n! list () $sep $mailbox${sep}test1\n* list () $sep $mailbox${sep}test2\n! list () $sep $mailbox${sep}test2${sep}test1\n* list () $sep $mailbox${sep}test3\n! list () $sep $mailbox${sep}test4\n! list () $sep $mailbox${sep}test4${sep}test5\n* list () $sep $mailbox${sep}test4${sep}test52\n! list () $sep $mailbox${sep}test6\n\n# test errors\nbad list (imaptest) \"\" \"\"\nbad list (recursivematch) \"\" \"\"\nbad list (recursivematch remote) \"\" \"\"\nbad list \"\" \"\" return (imaptest)\n\nok list (remote) \"\" %\n\nok list (subscribed) \"\" $mailbox${sep}*\n* list (\\subscribed) $sep $mailbox${sep}test1\n* list (\\subscribed) $sep $mailbox${sep}test2${sep}test1\n#* list ($!unordered $!ban=\\noselect \\subscribed \\nonexistent) $sep $mailbox${sep}test6\n* list (\\subscribed) $sep $mailbox${sep}test7${sep}test7\n* list (\\subscribed) $sep $mailbox${sep}test8${sep}test9\n\nok list (subscribed recursivematch) \"\" $mailbox${sep}test2% return (children)\n* list (\\haschildren) $sep $mailbox${sep}test2 (childinfo (subscribed))\n! list (\\subscribed) $sep $mailbox${sep}test2\n! list (\\subscribed) $sep $mailbox${sep}test2${sep}test1\n\n# don't test for \\hasnochildren, because it may be \\noinferiors instead\nok list \"\" $mailbox${sep}% return (children subscribed)\n* list ($!unordered $!ban=\\haschildren \\subscribed) $sep $mailbox${sep}test1\n* list ($!unordered $!ban=\\subscribed \\haschildren) $sep $mailbox${sep}test2\n* list ($!unordered $!ban=\\subscribed $!ban=\\haschildren) $sep $mailbox${sep}test3\n* list ($!unordered $!ban=\\subscribed \\haschildren) $sep $mailbox${sep}test4\n* list ($!unordered $!ban=\\subscribed \\haschildren) $sep $mailbox${sep}test7\n* list ($!unordered $!ban=\\subscribed \\haschildren) $sep $mailbox${sep}test8\n\nok list (subscribed recursivematch) \"\" $mailbox*test7\n! list () $sep $mailbox${sep}test7\n* list (\\subscribed) $sep $mailbox${sep}test7${sep}test7\n\nok list (subscribed recursivematch) \"\" $mailbox*test8\n* list ($!unordered $!ban=\\subscribed) $sep $mailbox${sep}test8 (childinfo (\"subscribed\"))\n* list (\\subscribed) $sep $mailbox${sep}test8${sep}test8\n\nok list (subscribed recursivematch) \"\" ($mailbox${sep}test2)\n* list () $sep $mailbox${sep}test2 (childinfo (\"subscribed\"))\n! list () $sep $mailbox${sep}test2${sep}test1\n\nok list (subscribed recursivematch) \"\" ($mailbox${sep}test2 $mailbox${sep}test2${sep}test1)\n* list () $sep $mailbox${sep}test2 (childinfo (\"subscribed\"))\n* list (\\subscribed) $sep $mailbox${sep}test2${sep}test1\n\nok list \"\" ($mailbox${sep}test7 $mailbox${sep}test7${sep}test7)\n#This isn't really an error, although it's non-optimal: ! list (\\nonexistent) $sep $mailbox${sep}test7\n* list () $sep $mailbox${sep}test7${sep}test7\n"
  },
  {
    "path": "tests/resources/imap-test/logout",
    "content": "state: nonauth\nconnections: 2\n\n1 ok noop\n! bye\n2 ok noop\n! bye\n\n1 ok logout\n* bye\n\n2 ok noop\n2 ok logout\n* bye\n"
  },
  {
    "path": "tests/resources/imap-test/move",
    "content": "capabilities: MOVE\nstate: created\n\n# - assumes COPYUID is sent untagged before expunges.\n# - assumes MOVE to mailbox itself changes message UID\n\nok append\nok append\nok append\nok create ${mailbox}2\n\nok select ${mailbox}2\n* 0 exists\n* ok [uidvalidity $uidvalidity_dest]\n* ok [uidnext $uidnext_dest1]\n\n# MOVE:\nok select $mailbox\n\nok fetch 1:* uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n* 3 fetch (uid $uid3)\n\nok move 1 ${mailbox}2\n* ok [copyuid $uidvalidity_dest $uid1 $uidnext_dest1]\n* 1 expunge\n\n# UID MOVE:\nok select ${mailbox}2\n* 1 exists\n* ok [uidvalidity $uidvalidity_dest]\n* ok [uidnext $uidnext_dest2]\n\nok select $mailbox\n* ok [uidvalidity $uidvalidity]\n* ok [uidnext $uidnext1]\n\nok uid move $uid2 ${mailbox}2\n* ok [copyuid $uidvalidity_dest $uid2 $uidnext_dest2]\n* 1 expunge\n\n# MOVE to same mailbox:\n#ok move 1 $mailbox\n#* ok [copyuid $uidvalidity $uid3 $uidnext1]\n#* 1 expunge\n#* 1 exists\n"
  },
  {
    "path": "tests/resources/imap-test/multiappend",
    "content": "capabilities: MULTIAPPEND\nstate: created\n\nok append $mailbox {{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: mail1\n\nbody1\n}}} {{{\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: mail2\n\nbody2\n}}}\n\nok select $mailbox\n\nok fetch 1:* body.peek[header.fields (subject)]\n* 1 fetch (body[header.fields (subject)] {{{\nSubject: mail1\n\n\n}}})\n* 2 fetch (body[header.fields (subject)] {{{\nSubject: mail2\n\n\n}}})\n"
  },
  {
    "path": "tests/resources/imap-test/mutf7",
    "content": "state: auth\n\n# get the separator\nok list \"\" \"\"\n* list () $sep $root\n\nok create \"$mailbox${sep}p&AOQA5A-\"\nok list \"\" \"$mailbox${sep}p&AOQA5A-\"\n* list () $sep \"$mailbox${sep}p&AOQA5A-\"\n\nok status \"$mailbox${sep}p&AOQA5A-\" (messages)\n* status \"$mailbox${sep}p&AOQA5A-\" (messages 0)\n"
  },
  {
    "path": "tests/resources/imap-test/nil",
    "content": "messages: 1\n\nok search subject NIL\n* search 1\n\nok search body nil\n* search 1\n\nok list \"\" NIL\n\nok store 1 +flags NIL\nok fetch 1 flags\n* 1 fetch (flags (NIL))\n"
  },
  {
    "path": "tests/resources/imap-test/nil.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: NiL\n\nnil\n"
  },
  {
    "path": "tests/resources/imap-test/notify",
    "content": "capabilities: notify qresync\n\nok create ${mailbox}2\nok append ${mailbox}2\n\n#\n# Check that initial STATUS notifications are sent when needed\n#\n\nok notify set status (mailboxes ${mailbox}2 (MessageExpunge))\n* status ${mailbox}2 (messages 1)\n\nok notify set status (mailboxes ${mailbox}2 (MessageNew))\n* status ${mailbox}2 (messages 1 uidnext $uidnext uidvalidity $uidvalidity)\n\nok notify set status (mailboxes ${mailbox}2 (FlagChange))\n* status ${mailbox}2 (uidvalidity $uidvalidity highestmodseq $hmodseq)\n\nok notify set (mailboxes ${mailbox}2 (MessageNew MessageExpunge FlagChange))\n! status ${mailbox}2 ()\n"
  },
  {
    "path": "tests/resources/imap-test/pipeline",
    "content": "state: auth\n\ntag1 create ${mailbox}\ntag2 create ${mailbox}2\ntag3 create ${mailbox}3\ntag4 create ${mailbox}4\ntag1 ok\ntag2 ok\ntag3 ok\ntag4 ok\n\ntag1 status ${mailbox} (messages)\ntag2 status ${mailbox}2 (messages)\ntag3 status ${mailbox}3 (messages)\ntag4 status ${mailbox}4 (messages)\n* status ${mailbox} (messages 0)\n* status ${mailbox}2 (messages 0)\n* status ${mailbox}3 (messages 0)\n* status ${mailbox}4 (messages 0)\ntag1 ok\ntag2 ok\ntag3 ok\ntag4 ok\n"
  },
  {
    "path": "tests/resources/imap-test/search-addresses",
    "content": "messages: all\n\n# full address searching\nok search from user-from@domain.org\n* search 1 2 3 4 6 7\nok search to user-to@domain.org\n* search 1 2 3 4\nok search cc user-cc@domain.org\n* search 1 2 3 4 5\nok search bcc user-bcc@domain.org\n* search 1 2 3 4\n\n# realname searching\nok search from realfrom\n* search 2 4 6 7\nok search to realto\n* search 2 4\nok search cc realcc\n* search 2 4\nok search bcc realbcc\n* search 2 4\n\n# existence searches\nok search header from \"\"\n* search 1 2 3 4 5 6 7\nok search header to \"\"\n* search 1 2 3 4 6 7\nok search header cc \"\"\n* search 1 2 3 4 5\nok search header bcc \"\"\n* search 1 2 3 4\n\n# substring address searches\n#ok search from ser-fro\n#* search 1 2 3 4 5 6 7\n#ok search to ser-t\n#* search 1 2 3 4\n#ok search cc ser-c\n#* search 1 2 3 4 5\n#ok search bcc ser-bc\n#* search 1 2 3 4\n\n# substring realname searches\n#ok search from ealfro\n#* search 2 4 6 7\n#ok search to ealt\n#* search 2 4\n#ok search cc ealc\n#* search 2 4\n#ok search bcc ealbc\n#* search 2 4\n\n# multiple addresses\nok search from user-from1\n* search 5\nok search from user-from2\n* search 5\n\n# groups\nok search to groupname\n* search 6 7\nok search to groupname2\n* search 6\nok search to groupuser1\n* search 6\nok search to groupuser2\n* search 6\nok search to groupuser3\n* search 6\nok search to groupuser4\n* search\n"
  },
  {
    "path": "tests/resources/imap-test/search-addresses.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user-from@domain.org\nTo: user-to@domain.org\nCc: user-cc@domain.org\nBcc: user-bcc@domain.org\n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: RealFrom <user-from@domain.org>\nTo: RealTo <user-to@domain.org>\nCc: RealCc <user-cc@domain.org>\nBcc: RealBcc <user-bcc@domain.org>\n\nbody2\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: <user-from (comment)@ (comment) domain.org>\nTo: <user-to@domain.org>\nCc: <user-cc@domain.org>\nBcc: <user-bcc@domain.org>\n\nbody3\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user-from@domain.org (RealFrom)\nTo: user-to@domain.org (RealTo)\nCc: user-cc@domain.org (RealCc)\nBcc: user-bcc@domain.org (RealBcc)\n\nbody4\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: user-from1@domain.org, user-from2@domain.org\nCc: user-cc@domain.org\n\nbody5\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: RealFrom <user-from@domain.org>\nTo: groupname: groupuser1@domain.org, groupuser2@domain.org;,\n  groupname2: groupuser3@domain.org;\n\nbody6\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nFrom: RealFrom <user-from@domain.org>\nTo: groupname:;\n\nbody7\n"
  },
  {
    "path": "tests/resources/imap-test/search-body",
    "content": "messages: all\n\n# search full words first\nok search text asdfghjkl\n* search 1 2\nok search text zxcvbnm\n* search 1 3 4\nok search text qwertyuiop\n* search 2 4\n\nok search body asdfghjkl\n* search 1 2\nok search body zxcvbnm\n* search 1 3 4\nok search body qwertyuiop\n* search 4\n\n# search substrings\n#ok search text sdfghjk\n#* search 1 2\n#ok search text xcvbn\n#* search 1 3 4\n#ok search text wertyuio\n#* search 2 3 4\n\n#ok search body sdfghjk\n#* search 1 2\n#ok search body xcvbn\n#* search 1 3 4\n#ok search body wertyuio\n#* search 4\n"
  },
  {
    "path": "tests/resources/imap-test/search-body.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nasdfghjkl zxcvbnm\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nSubject: qwertyuiop\n\nasdfghjkl\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nX-Header: qwertyuiop\n\nzxcvbnm\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nzxcvbnm qwertyuiop\n"
  },
  {
    "path": "tests/resources/imap-test/search-context-update",
    "content": "capabilities: CONTEXT=SEARCH\nstate: created\n\nok append\nok append\nok append\nok append\nok append\n\nok select $mailbox\n\nok search return (update) body body seen\n* esearch (tag $searchtag)\n\nok store 1,2,4 +flags \\seen\n* esearch (tag $searchtag) addto ($pos 1:2,4)\n\nok store 1,2 -flags \\seen\n* esearch (tag $searchtag) removefrom ($pos2 1:2)\n\nok store 3:5 +flags \\deleted\nok expunge\n* esearch (tag $searchtag) removefrom ($pos3 $4)\n* $3 expunge\n* $4 expunge\n* $5 expunge\n\nok append $mailbox (\\seen)\n* 3 exists\n* esearch (tag $searchtag) addto ($pos4 3)\n\nok cancelupdate \"$searchtag\"\n\nok store 1 +flags \\seen\n! esearch (tag $searchtag) addto ($pos5 1)\n"
  },
  {
    "path": "tests/resources/imap-test/search-context-update2",
    "content": "capabilities: CONTEXT=SEARCH\nmessages: 5\n\nok store 1,2 +flags \\seen\n\nok search return (update all) or seen subject s22\n* esearch (tag $searchtag) all 1:2\n\nok store 3:5 +flags \\seen\n* esearch (tag $searchtag) addto ($pos 3:5)\n\nok store 1:5 -flags \\seen\n* esearch (tag $searchtag) removefrom ($pos2 1,3:5)\n\nok store 3,4 +flags \\seen\n* esearch (tag $searchtag) addto ($pos3 3:4)\n\nok store 3 +flags \\deleted\nok expunge\n* 3 expunge\n* esearch (tag $searchtag) removefrom ($pos4 3)\n"
  },
  {
    "path": "tests/resources/imap-test/search-context-update3",
    "content": "capabilities: CONTEXT=SEARCH\nmessages: 6\n\nok store 1 +flags \\deleted\nok expunge\n\nok fetch 1:5 uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n* 3 fetch (uid $uid3)\n* 4 fetch (uid $uid4)\n* 5 fetch (uid $uid5)\n\nok store 1,2 +flags \\seen\n\nok uid search return (update all) or seen subject s33\n* esearch (tag $searchtag) uid all $uid1:$uid2\n\nok store 3:5 +flags \\seen\n* esearch (tag $searchtag) uid addto ($pos $uid3:$uid5)\n\nok store 1:5 -flags \\seen\n* esearch (tag $searchtag) uid removefrom ($pos2 $uid1,$uid3:$uid5)\n\nok uid store $uid3,$uid4 +flags \\seen\n* esearch (tag $searchtag) uid addto ($pos3 $uid3:$uid4)\n\nok store 3 +flags \\deleted\nok expunge\n* 3 expunge\n* esearch (tag $searchtag) uid removefrom ($pos4 $uid3)\n"
  },
  {
    "path": "tests/resources/imap-test/search-date",
    "content": "messages: all\n\n# 1) Timezone changes from EET +0200 -> EEST +0300\n# 1a) SENTBEFORE\nok search sentbefore 24-mar-2007\n* search\nok search sentbefore 25-mar-2007\n* search 1\nok search sentbefore 27-mar-2007\n* search 1 2 3 4 5 6 7\n\n# 1b) SENTON\nok search senton 23-mar-2007\n* search\nok search senton 24-mar-2007\n* search 1\nok search senton 25-mar-2007\n* search 2 3 4 5 6\nok search senton 26-mar-2007\n* search 7\n\n# 1c) SENTSINCE\nok search 1:7 sentsince 24-mar-2007\n* search 1 2 3 4 5 6 7\nok search 1:7 sentsince 25-mar-2007\n* search 2 3 4 5 6 7\nok search 1:7 sentsince 26-mar-2007\n* search 7\nok search 1:7 sentsince 27-mar-2007\n* search\n\n# 2) Timezone changes from EEST +0300 -> EET +0200\n# 2a) SENTBEFORE\nok search 8:* sentbefore 27-oct-2007\n* search\nok search 8:* sentbefore 28-oct-2007\n* search 8\nok search 8:* sentbefore 29-oct-2007\n* search 8 9 10 11 12 13 14 15\nok search 8:* sentbefore 30-oct-2007\n* search 8 9 10 11 12 13 14 15 16\n\n# 2b) SENTON\nok search 8:* senton 26-oct-2007\n* search\nok search 8:* senton 27-oct-2007\n* search 8\nok search 8:* senton 28-oct-2007\n* search 9 10 11 12 13 14 15\nok search 8:* senton 29-oct-2007\n* search 16\n\n# 2c) SENTSINCE\nok search 8:* sentsince 27-oct-2007\n* search 8 9 10 11 12 13 14 15 16\nok search 8:* sentsince 28-oct-2007\n* search 9 10 11 12 13 14 15 16\nok search 8:* sentsince 29-oct-2007\n* search 16\nok search 8:* sentsince 30-oct-2007\n* search\n\n# 3) Try a couple of NOTs\nok search 1:7 not sentbefore 26-mar-2007\n* search 7\nok search 1:7 not senton 25-mar-2007\n* search 1 7\nok search 8:* not sentsince 28-oct-2007\n* search 8\nok search 8:* not senton 28-oct-2007\n* search 8 16\n"
  },
  {
    "path": "tests/resources/imap-test/search-date.mbox",
    "content": "From user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sat, 24 Mar 2007 23:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 25 Mar 2007 00:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 25 Mar 2007 01:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 25 Mar 2007 02:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 25 Mar 2007 04:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 25 Mar 2007 23:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Mon, 26 Mar 2007 00:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sat, 27 Oct 2007 03:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 28 Oct 2007 00:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 28 Oct 2007 01:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 28 Oct 2007 02:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 28 Oct 2007 03:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 28 Oct 2007 03:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 28 Oct 2007 04:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Sun, 28 Oct 2007 23:00:00 +0000\n\nbody\n\nFrom user@domain  Sat Mar 24 23:00:00 2007 +0000\nDate: Mon, 29 Oct 2007 00:00:00 +0000\n\nbody\n\n"
  },
  {
    "path": "tests/resources/imap-test/search-flags",
    "content": "messages: 5\n\nok store 1 flags ($$hello)\nok store 2 flags (\\seen \\flagged)\nok store 3 flags (\\answered $$hello)\nok store 4 flags (\\flagged \\draft)\nok store 5 flags (\\deleted \\answered)\n\nok search answered\n* search 3 5\nok search unanswered\n* search 1 2 4\n\nok search deleted\n* search 5\nok search undeleted\n* search 1 2 3 4\n\nok search draft\n* search 4\nok search undraft\n* search 1 2 3 5\n\nok search flagged\n* search 2 4\nok search unflagged\n* search 1 3 5\n\nok search seen\n* search 2\nok search unseen\n* search 1 3 4 5\n\nok search keyword $$hello\n* search 1 3\nok search unkeyword $$hello\n* search 2 4 5\n\n#ok search new\n#* search 1 3 4 5\n\n#ok search old\n#* search\n\n#ok search recent\n#* search 1 2 3 4 5\n\nok store 1:* flags (\\seen)\nok store 2 +flags (\\flagged)\nok search seen flagged\n* search 2\nok search seen not flagged\n* search 1 3 4 5\nok search not seen flagged\n* search\nok search not seen not flagged\n* search\n\nok store 1:* flags (\\deleted)\nok store 2 +flags (\\flagged)\nok search deleted flagged\n* search 2\nok search deleted not flagged\n* search 1 3 4 5\nok search not deleted flagged\n* search\nok search not deleted not flagged\n* search\n\nok store 1:* flags (\\seen \\deleted)\nok store 2 +flags (\\flagged)\nok search seen flagged\n* search 2\nok search seen not flagged\n* search 1 3 4 5\nok search not seen flagged\n* search\nok search not seen not flagged\n* search\nok search seen deleted flagged\n* search 2\nok search seen deleted not flagged\n* search 1 3 4 5\nok search not seen deleted flagged\n* search\nok search not seen deleted not flagged\n* search\nok search seen not deleted flagged\n* search\nok search seen not deleted not flagged\n* search\nok search not seen not deleted flagged\n* search\nok search not seen not deleted not flagged\n* search\n"
  },
  {
    "path": "tests/resources/imap-test/search-header",
    "content": "messages: all\n\n# just check that this returns ok. it's not really specified in RFC, so\n# don't verify the result.\nok search subject \"\"\n\n# subject\nok search subject hello\n* search 1\nok search subject beautiful\n* search 1\nok search subject world\n* search 1\nok search subject \"hello beautiful\"\n* search 1\nok search subject \"hello beautiful world\"\n* search 1\n#ok search subject \"eautiful worl\"\n#* search 1\n\n# header\nok search header subject \"\"\n* search 1\nok search not header subject \"\"\n* search 2\n#ok search header x-extra \"\"\n#* search 2\n#ok search not header x-extra \"\"\n#* search 1\n#ok search header x-extra hello\n#* search 2\n#ok search header x-extra \"hello beautiful\"\n#* search 2\n#ok search header x-extra \"eautiful head\"\n#* search 2\n#ok search header x-extra \"another\"\n#* search 2\n"
  },
  {
    "path": "tests/resources/imap-test/search-header.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nSubject: hello beautiful world\n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nX-Extra: hello beautiful header\nX-Extra: another one\n\nbody\n"
  },
  {
    "path": "tests/resources/imap-test/search-partial",
    "content": "capabilities: CONTEXT=SEARCH\nmessages: all\n\nok search return (partial 1:1) all\n* esearch (tag $tag) partial (1:1 1)\nok search return (partial 2:4) all\n* esearch (tag $tag) partial (2:4 2:4)\nok search return (partial 4:2) all\n* esearch (tag $tag) partial (2:4 2:4)\nok search return (partial 1:6) all\n* esearch (tag $tag) partial (1:6 1:5)\nok search return (partial 6:6) all\n* esearch (tag $tag) partial (6:6 nil)\n\nok search return (partial 1:3) 1:2,4:5\n* esearch (tag $tag) partial (1:3 1:2,4)\nok search return (partial 2:3) 1:2,4:5\n* esearch (tag $tag) partial (2:3 2,4)\nok search return (partial 2:4) 1:2,4:5\n* esearch (tag $tag) partial (2:4 2,4:5)\nok search return (partial 2:10) 1:2,4:5\n* esearch (tag $tag) partial (2:10 2,4:5)\n\n# UID partials\n\nok fetch 1 uid\n* 1 fetch (uid $uid1)\nok fetch 2 uid\n* 2 fetch (uid $uid2)\nok fetch 3 uid\n* 3 fetch (uid $uid3)\nok fetch 5 uid\n* 5 fetch (uid $uid5)\n\nok uid search return (partial 1:1) all\n* esearch (tag $tag) uid partial (1:1 $uid1)\n\nok uid search return (partial 2:2) all\n* esearch (tag $tag) uid partial (2:2 $uid2)\n\nok store 2 +flags \\deleted\nok expunge\n\nok uid search return (partial 2:2) all\n* esearch (tag $tag) uid partial (2:2 $uid3)\n\nok uid search return (partial 5:10) all\n* esearch (tag $tag) uid partial (5:10 nil)\n\n# broken results\n\nbad search return (partial 1) all\nbad search return (partial 1:*) all\nbad search return (partial *:1) all\n"
  },
  {
    "path": "tests/resources/imap-test/search-partial.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:21 2008\n\n1\n\nFrom user@domain  Fri Feb 22 17:06:22 2008\n\n2\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\n\n3\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\n\n4\n\nFrom user@domain  Fri Feb 22 17:06:25 2008\n\n5\n"
  },
  {
    "path": "tests/resources/imap-test/search-sets",
    "content": "messages: 6\n\nok uid search all\n* search $uid1 $uid2 $uid3 $uid4 $uid5 $uid6\nok status $mailbox (uidnext)\n* status $mailbox (uidnext $uidnext)\n\n# break seq=uid mapping\nok store 2 +flags \\deleted\nok expunge\n* 2 expunge\n\nok search all\n* search 1 2 3 4 5\nok uid search all\n* search $uid1 $uid3 $uid4 $uid5 $uid6\n\nok search 1:3,5\n* search 1 2 3 5\nok search 4:2\n* search 2 3 4\nok search uid $uid1:$uid3,$uid5\n* search 1 2 4\nok search uid $uid4:$uid2\n* search 2 3\n\nok search 1:3 not uid $uid3\n* search 1 3\nok search not 2,4\n* search 1 3 5\nok search or 1 uid $uid3\n* search 1 2\n\nok search *\n* search 5\nok search uid *\n* search 5\nok search uid $uidnext:*\n* search 5\nok search *:3\n* search 3 4 5\nok search 1,4,*\n* search 1 4 5\n\n# These are in a bit of a grey area. Most servers allow them, but it's not\n# explicitly defined in the RFC that they're legal:\n#ok search 6:*\n#* search 5\n#ok search *:6\n#* search 5\n\nok search (3) uid $uid4\n* search 3\nok search (uid $uid4) 3\n* search 3\nok search 3 (uid $uid4)\n* search 3\n\nok uid search uid 1:4294967295\n* search $uid1 $uid3 $uid4 $uid5 $uid6\nok uid search uid $uidnext:4294967295\n* search \n"
  },
  {
    "path": "tests/resources/imap-test/search-size",
    "content": "messages: all\n\n# get the middle size. don't trust precalculated values in case server\n# modifies the message while APPENDing it. Messages 3 and 4 have different\n# RFC822.SIZE, but with servers that store linefeeds as LFs they have the\n# same file size. This test catches if they search using file size.\nok fetch 1:4 rfc822.size\n* 3 fetch (rfc822.size $size)\n\nok search smaller $size\n* search 1 2\nok search larger $size\n* search 4\nok search not smaller $size\n* search 3 4\nok search not larger $size\n* search 1 2 3\n\nok search not smaller $size not larger $size\n* search 3\nok search or smaller $size larger $size\n* search 1 2 4\n"
  },
  {
    "path": "tests/resources/imap-test/search-size.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n1\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n22\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n333\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n4\n4\n\n"
  },
  {
    "path": "tests/resources/imap-test/select",
    "content": "state: created\n\nok append\nok append\n\n# first open read-only so recent flags don't get lost\nexamine $mailbox\n* 2 exists\n#* 2 recent\n#* ok [unseen 1]\n* ok [uidvalidity $uidvalidity]\n* ok [uidnext $uidnext]\nok [read-only]\n\nok close\n\n# check that STATUS replies with the same values\nok status $mailbox (messages uidnext uidvalidity unseen)\n* status $mailbox (messages 2 uidnext $uidnext uidvalidity $uidvalidity unseen 2)\n\n# then try read-write\nselect $mailbox\n* 2 exists\n#* 2 recent\n#* ok [unseen 1]\n* ok [uidvalidity $uidvalidity]\n* ok [uidnext $uidnext]\nok [read-write]\n\nok close\n\n#ok status $mailbox (recent)\n#* status $mailbox (recent 0)\n"
  },
  {
    "path": "tests/resources/imap-test/select.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\nbody2\n\n"
  },
  {
    "path": "tests/resources/imap-test/sort-addresses",
    "content": "capabilities: SORT\nmessages: all\n\nok sort (from) us-ascii all\n* sort 3 2 1\nok sort (to) us-ascii all\n* sort 3 2 1\nok sort (cc) us-ascii all\n* sort 2 1 3\n\nok sort (reverse from) us-ascii all\n* sort 1 2 3\nok sort (reverse to) us-ascii all\n* sort 1 2 3\nok sort (reverse cc) us-ascii all\n* sort 3 1 2\n\nok sort (from reverse arrival) us-ascii all\n* sort 3 2 1\nok sort (to reverse arrival) us-ascii all\n* sort 3 2 1\nok sort (cc reverse arrival) us-ascii all\n* sort 2 1 3\n"
  },
  {
    "path": "tests/resources/imap-test/sort-addresses.mbox",
    "content": "From user@domain  Fri Feb 22 00:06:23 2008\nFrom: user2@domain2.org\nTo: user2@domain2.org\nCc: user2@domain2.org (foo bar)\n\n1\n\nFrom user@domain  Fri Feb 22 01:06:23 2008\nFrom: user2@domain1.org\nTo: user2@domain1.org\nCc: blah@domain1.org\n\n2\n\nFrom user@domain  Fri Feb 22 02:06:23 2008\nFrom: user1@domain1.org\nTo: user1@domain1.org\nCc: user1@domain1.org\n\n3\n\n"
  },
  {
    "path": "tests/resources/imap-test/sort-arrival",
    "content": "capabilities: SORT\nmessages: all\n\nok sort (arrival) us-ascii all\n* sort 1 3 2 4 5\n\nok sort (reverse arrival) us-ascii all\n* sort 5 4 2 3 1\n\nok sort (arrival reverse size) us-ascii all\n* sort 1 3 5 4 2\n"
  },
  {
    "path": "tests/resources/imap-test/sort-arrival.mbox",
    "content": "From user@domain  Fri Feb 22 00:00:00 2008\n\n1\n\nFrom user@domain  Fri Feb 22 02:00:00 2008\n\n2\n\nFrom user@domain  Fri Feb 22 01:00:00 2008\n\n33\n\nFrom user@domain  Fri Feb 22 02:00:00 2008\n\n44\n\nFrom user@domain  Fri Feb 22 02:00:00 2008\n\n5555\n\n"
  },
  {
    "path": "tests/resources/imap-test/sort-date",
    "content": "capabilities: SORT\nmessages: all\n\nok sort (date) us-ascii all\n* sort 1 3 7 5 2 4 6\n\nok sort (reverse date) us-ascii all\n* sort 6 4 2 5 7 3 1\n\nok sort (date reverse size) us-ascii all\n* sort 1 7 3 5 6 4 2\n"
  },
  {
    "path": "tests/resources/imap-test/sort-date.mbox",
    "content": "From user@domain  Fri Feb 22 23:00:00 2008 +0200\nDate: Fri, 22 Feb 2008 00:00:00 +0200\n\n1\n\nFrom user@domain  Fri Feb 22 22:00:00 2008 +0200\nDate: Fri, 22 Feb 2008 02:00:00 +0200\n\n2\n\nFrom user@domain  Fri Feb 22 21:00:00 2008 +0200\nDate: Fri, 22 Feb 2008 01:00:00 +0200\n\n33\n\nFrom user@domain  Fri Feb 22 20:00:00 2008 +0200\nDate: Fri, 22 Feb 2008 02:00:00 +0200\n\n44\n\nFrom user@domain  Fri Feb 22 01:30:23 2008 +0200\nDate: Fri, 22 Feb 2008 01:30:23 +0200\nSubject: foo\n\n55555\n\nFrom user@domain  Fri Feb 22 18:00:00 2008 +0200\nDate: Fri, 22 Feb 2008 02:00:00 +0200\n\n6666\n\nFrom user@domain  Fri Feb 22 01:00:00 2008 +0200\nDate: Fri, 22 Feb 2008 01:00:00 +0200\nSubject: foo bar foo bar foo bar fooo\n\n777\n\n"
  },
  {
    "path": "tests/resources/imap-test/sort-display-from",
    "content": "capabilities: SORT=DISPLAY\nmessages: all\n\nok sort (displayfrom) us-ascii all\n* sort 2 3 4 1 5\n\nok sort (reverse displayfrom) us-ascii all\n* sort 5 1 4 3 2\n"
  },
  {
    "path": "tests/resources/imap-test/sort-display-from.mbox",
    "content": "From user@domain  Fri Feb 22 23:00:00 2008 +0200\nFrom: foo bar <a@b.c>\n\n1\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nFrom: b@c.d\n\n2\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nFrom: d\n\n3\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nFrom: =?iso-8859-1?q?foo_aar?= <e@f.g>\n\n4\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nFrom: =?iso-8859-1?q?foo_car?= <f@g.h>\n\n5\n"
  },
  {
    "path": "tests/resources/imap-test/sort-display-to",
    "content": "capabilities: SORT=DISPLAY\nmessages: all\n\nok sort (displayto) us-ascii all\n* sort 8 7 2 3 4 1 5 6\n\nok sort (reverse displayto) us-ascii all\n* sort 6 5 1 4 3 2 7 8\n"
  },
  {
    "path": "tests/resources/imap-test/sort-display-to.mbox",
    "content": "From user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: foo bar <a@b.c>\n\n1\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: b@c.d\n\n2\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: d\n\n3\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: =?iso-8859-1?q?foo_aar?= <e@f.g>\n\n4\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: =?iso-8859-1?q?foo_car?= <f@g.h>\n\n5\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: a: xx@xx.org, yy@yy.org;\n\n6\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: z: aa@aa.org, bb@bb.org;\n\n7\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nTo: empty:;\n\n8\n"
  },
  {
    "path": "tests/resources/imap-test/sort-partial",
    "content": "capabilities: CONTEXT=SORT\nmessages: all\n\nok sort return (partial 1:1) (arrival) us-ascii all\n* esearch (tag $tag) partial (1:1 1)\nok sort return (partial 2:4) (arrival) us-ascii all\n* esearch (tag $tag) partial (2:4 2:4)\nok sort return (partial 4:2) (arrival) us-ascii all\n* esearch (tag $tag) partial (2:4 2:4)\nok sort return (partial 2:4) (reverse arrival) us-ascii all\n* esearch (tag $tag) partial (2:4 3:4,2)\nok sort return (partial 1:6) (arrival) us-ascii all\n* esearch (tag $tag) partial (1:6 1:5)\nok sort return (partial 1:6) (reverse arrival) us-ascii all\n* esearch (tag $tag) partial (1:6 5,3:4,2,1)\n\nbad sort return (partial 1) (arrival) us-ascii all\nbad sort return (partial 1:*) (arrival) us-ascii all\nbad sort return (partial *:1) (arrival) us-ascii all\n"
  },
  {
    "path": "tests/resources/imap-test/sort-partial.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:21 2008\n\n1\n\nFrom user@domain  Fri Feb 22 17:06:22 2008\n\n2\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\n\n3\n\nFrom user@domain  Fri Feb 22 17:06:23 2008\n\n4\n\nFrom user@domain  Fri Feb 22 17:06:25 2008\n\n5\n"
  },
  {
    "path": "tests/resources/imap-test/sort-size",
    "content": "capabilities: SORT\nmessages: all\n\nok sort (size) us-ascii all\n* sort 1 8 2 6 3 5 4 7\n\nok sort (reverse size) us-ascii all\n* sort 7 4 5 3 6 2 8 1\n\nok sort (size reverse arrival) us-ascii all\n* sort 8 1 6 2 5 3 7 4\n"
  },
  {
    "path": "tests/resources/imap-test/sort-size.mbox",
    "content": "From user@domain  Fri Feb 22 00:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n1\n\nFrom user@domain  Fri Feb 22 01:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n22\n\nFrom user@domain  Fri Feb 22 02:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n333\n\nFrom user@domain  Fri Feb 22 03:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n4\n4\n\nFrom user@domain  Fri Feb 22 04:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n555\n\nFrom user@domain  Fri Feb 22 05:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n66\n\nFrom user@domain  Fri Feb 22 06:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n7777\n\nFrom user@domain  Fri Feb 22 07:06:23 2008\nDate: Sat, 24 Mar 2007 23:00:00 +0200\n\n8\n\n"
  },
  {
    "path": "tests/resources/imap-test/sort-subject",
    "content": "capabilities: SORT\nmessages: all\n\nok sort (subject) us-ascii all\n* sort 9 10 14 2 12 15 8 4 1 3 5 11 6 7 13\n\nok sort (reverse subject) us-ascii all\n* sort 13 7 6 11 5 3 1 4 8 15 12 2 14 10 9\n\nok sort (subject reverse size) us-ascii all\n* sort 10 9 14 12 2 15 8 4 1 11 5 3 6 13 7\n"
  },
  {
    "path": "tests/resources/imap-test/sort-subject.mbox",
    "content": "From user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: a\n\n1\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: C\n\n2\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: b\n\n3\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: _\n\n4\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: [foo] Fwd: [bar] Re: fw: b (fWd)  (fwd)\n\n55\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: b (a)\n\n66\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: Re: [FWD: c]\n\n77\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: [xyz]\n\n88\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\n\n9\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: =?iso-8859-1?q?_?=\n\n10\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: Re: =?utf-8?q?b?=\n\n11\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: =?iso-8859-1?q?RE:_C?=\n\n12\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: =?us-ascii?b?UmU6IGM=?=\n\n13\n\nThe subject is 'Re: c'.\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: Ad: Re: Ad: Re: Ad: x\n\n14\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: re: [fwd: [fwd: re: [fwd: babylon]]]\n\n15\n"
  },
  {
    "path": "tests/resources/imap-test/store",
    "content": "connections: 2\nmessages: 4\n\n# simple tests\n1 ok store 1:3 flags (\\seen)\n* 1 fetch (flags (\\seen))\n* 2 fetch (flags (\\seen))\n* 3 fetch (flags (\\seen))\n\n2 ok check\n* 1 fetch (flags (\\seen))\n* 2 fetch (flags (\\seen))\n* 3 fetch (flags (\\seen))\n\n1 ok store 4 -flags \\seen\n1 ok store 3 flags (\\draft)\n* 3 fetch (flags (\\draft))\n\n# keywords\n1 ok store 2,4 +flags ($$hello $$world)\n* 2 fetch (flags (\\seen $$hello $$world))\n* 4 fetch (flags ($$hello $$world))\n\n# check that two sessions don't overwrite each others' changes\n1 ok store 1 +flags (\\answered)\n2 ok store 1 -flags (\\seen)\n1 ok check\n2 ok check\n\n1 ok fetch 1 flags\n* 1 fetch (flags (\\answered))\n2 ok fetch 1 flags\n* 1 fetch (flags (\\answered))\n"
  },
  {
    "path": "tests/resources/imap-test/subscribe",
    "content": "connections: 2\nstate: auth\n\n# get the separator\n1 ok list \"\" \"\"\n* list () $sep $root\n\n1 \"\" unsubscribe $mailbox${sep}test\n1 \"\" unsubscribe $mailbox${sep}test2\n1 \"\" unsubscribe $mailbox${sep}test2${sep}test\n1 \"\" unsubscribe $mailbox${sep}test3${sep}test3\n\n1 ok create $mailbox${sep}\n1 ok create $mailbox${sep}test\n1 ok create $mailbox${sep}test2${sep}\n1 ok create $mailbox${sep}test3${sep}\n1 ok create $mailbox${sep}test2${sep}test\n1 ok create $mailbox${sep}test3${sep}test3\n# create the test2 mailbox only if server supports inferior mailboxes\n1 \"\" create $mailbox${sep}test2\n\n1 ok subscribe $mailbox${sep}test\n1 ok subscribe $mailbox${sep}test2${sep}test\n1 ok subscribe $mailbox${sep}test3${sep}test3\n\n2 ok lsub \"\" $mailbox${sep}%\n* lsub () $sep $mailbox${sep}test\n#* lsub (\\noselect) $sep $mailbox${sep}test2\n#* lsub (\\noselect) $sep $mailbox${sep}test3\n! lsub (\\noselect) $sep $mailbox${sep}test2${sep}test\n! lsub (\\noselect) $sep $mailbox${sep}test3${sep}test3\n\n2 ok lsub \"\" *test\n* lsub () $sep $mailbox${sep}test\n* lsub () $sep $mailbox${sep}test2${sep}test\n! lsub () $sep $mailbox${sep}test3${sep}test3\n\n1 ok unsubscribe $mailbox${sep}test\n2 ok lsub \"\" $mailbox${sep}%\n! lsub () $sep $mailbox${sep}test\n\n1 ok unsubscribe $mailbox${sep}test2${sep}test\n2 ok lsub \"\" $mailbox${sep}*\n! lsub () $sep $mailbox${sep}test\n* lsub () $sep $mailbox${sep}test3${sep}test3\n\n1 ok unsubscribe $mailbox${sep}test3${sep}test3\n2 ok lsub \"\" $mailbox${sep}%\n! lsub () $sep $mailbox${sep}test3\n"
  },
  {
    "path": "tests/resources/imap-test/thread",
    "content": "capabilities: THREAD=REFERENCES\nmessages: all\n\nok thread references us-ascii all\n* thread (1 2 3)\n\nok store 1 +flags \\deleted\nok expunge\n\nok thread references us-ascii all\n* thread (1 2)\n"
  },
  {
    "path": "tests/resources/imap-test/thread-orderedsubject",
    "content": "capabilities: THREAD=ORDEREDSUBJECT\nmessages: all\n\nok thread orderedsubject us-ascii all\n* THREAD (1)(2 (7)(12)(13))(3 (5)(11))(4)(6)(8)(9 10)(14)(15)\n"
  },
  {
    "path": "tests/resources/imap-test/thread-orderedsubject.mbox",
    "content": "From user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: a\n\n1\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: C\n\n2\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: b\n\n3\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: _\n\n4\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: [foo] Fwd: [bar] Re: fw: b (fWd)  (fwd)\n\n55\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: b (a)\n\n66\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: Re: [FWD: c]\n\n77\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: [xyz]\n\n88\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\n\n9\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: =?iso-8859-1?q?_?=\n\n10\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: Re: =?utf-8?q?b?=\n\n11\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: =?iso-8859-1?q?RE:_C?=\n\n12\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: =?us-ascii?b?UmU6IGM=?=\n\n13\n\nThe subject is 'Re: c'.\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: Ad: Re: Ad: Re: Ad: x\n\n14\n\nFrom user@domain  Fri Feb 22 23:00:00 2008 +0200\nSubject: re: [fwd: [fwd: re: [fwd: babylon]]]\n\n15\n"
  },
  {
    "path": "tests/resources/imap-test/thread-orderedsubject2",
    "content": "capabilities: THREAD=ORDEREDSUBJECT\nmessages: all\n\nok thread orderedsubject us-ascii all\n* THREAD (4 (2)(8)(6))(3 (1)(7)(5))\n"
  },
  {
    "path": "tests/resources/imap-test/thread-orderedsubject2.mbox",
    "content": "From user@domain  Fri Feb 22 23:00:01 2008 +0200\nSubject: a\nDate: Fri, 22 Feb 2008 22:00:10 +0200\n\n1\n\nFrom user@domain  Fri Feb 22 23:00:02 2008 +0200\nDate: Fri, 22 Feb 2008 22:00:09 +0200\nSubject: b\n\n2\n\nFrom user@domain  Fri Feb 22 23:00:03 2008 +0200\nDate: Fri, 22 Feb 2008 22:00:08 +0200\nSubject: a\n\n3\n\nFrom user@domain  Fri Feb 22 23:00:04 2008 +0200\nDate: Fri, 22 Feb 2008 22:00:07 +0200\nSubject: b\n\n4\n\nFrom user@domain  Fri Feb 22 22:00:30 2008 +0200\nSubject: a\n\n5\n\nFrom user@domain  Fri Feb 22 22:00:29 2008 +0200\nSubject: b\n\n6\n\nFrom user@domain  Fri Feb 22 22:00:28 2008 +0200\nSubject: a\n\n7\n\nFrom user@domain  Fri Feb 22 22:00:27 2008 +0200\nSubject: b\n\n8\n\n"
  },
  {
    "path": "tests/resources/imap-test/thread.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nMessage-ID: <a@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <b@b>\nReferences: <a@b> <c@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 16:06:25 2008\nMessage-Id: <c@b>\nReferences: <b@b>\n\nbody\n"
  },
  {
    "path": "tests/resources/imap-test/thread2",
    "content": "capabilities: THREAD=REFERENCES\nmessages: all\n\nok thread references us-ascii all\n* thread (1)(2)\n"
  },
  {
    "path": "tests/resources/imap-test/thread2.mbox",
    "content": "From user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <a@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <a@c>\n\nbody\n"
  },
  {
    "path": "tests/resources/imap-test/thread3",
    "content": "capabilities: THREAD=REFERENCES\nmessages: all\n\nok thread references us-ascii all\n* THREAD (1 2)\n"
  },
  {
    "path": "tests/resources/imap-test/thread3.mbox",
    "content": "From user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <a@b>\nSubject: foo\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:24 2008\nMessage-Id: <a@b>\nSubject: Re: foo\n\nbody\n"
  },
  {
    "path": "tests/resources/imap-test/thread4",
    "content": "capabilities: THREAD=REFERENCES\nmessages: all\n\nok thread references us-ascii all\n* THREAD (1 2)\n"
  },
  {
    "path": "tests/resources/imap-test/thread4.mbox",
    "content": "From user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <a@b>\nSubject: foo\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:24 2008\nMessage-Id: <b@b>\nReferences: <a@b>\nSubject: Re: foo\n\nbody\n"
  },
  {
    "path": "tests/resources/imap-test/thread5",
    "content": "capabilities: THREAD=REFERENCES\nstate: created\n\nok append\nok append\nok append\nok append\n\nok select $mailbox\nok thread references us-ascii all\n* THREAD (1 2 3 4)\n\nok store 1,2 +flags \\deleted\nok expunge\nok thread references us-ascii all\n* THREAD (1 2)\n\nok append\nok thread references us-ascii all\n* THREAD (1 2 3)\n"
  },
  {
    "path": "tests/resources/imap-test/thread5.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nMessage-ID: <1@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <2@b>\nIn-Reply-To: <1@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <3@b>\nIn-Reply-To: <1@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <4@b>\nIn-Reply-To: <2@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <5@b>\nIn-Reply-To: <2@b>\n\nbody\n\n"
  },
  {
    "path": "tests/resources/imap-test/thread6",
    "content": "capabilities: THREAD=REFERENCES\nmessages: all\n\nok thread references us-ascii all\n* THREAD (1 2)\n"
  },
  {
    "path": "tests/resources/imap-test/thread6.mbox",
    "content": "From user@domain  Fri Feb 22 15:06:23 2008\nReferences: <a@b>\n\nbody\n\nFrom user@domain  Fri Feb 22 15:06:24 2008\nReferences: <a@b>\n\nbody\n"
  },
  {
    "path": "tests/resources/imap-test/thread7",
    "content": "capabilities: THREAD=REFERENCES\nmessages: all\n\nok thread references us-ascii all\n* thread (1 2 3 4)\n\nok store 2 +flags \\deleted\nok expunge\n\nok thread references us-ascii all\n* thread (1 2 3)\n"
  },
  {
    "path": "tests/resources/imap-test/thread7.mbox",
    "content": "From user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <a@b>\n\nbody1\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <b@b>\nReferences: <a@b>\n\nbody2\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <c@b>\nReferences: <b@b>\n\nbody3\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <b@b>\nReferences: <a@b>\n\nbody4 (2 duplicate)\n"
  },
  {
    "path": "tests/resources/imap-test/thread8",
    "content": "capabilities: THREAD=REFERENCES\nstate: created\n\nok append\nok append\nok select $mailbox\n\nok thread references us-ascii all\n* thread (1 2)\n\nok store 2 +flags \\deleted\nok expunge\n\nok thread references us-ascii all\n* thread (1)\n\nok append\n\nok thread references us-ascii all\n* thread (1 2)\n"
  },
  {
    "path": "tests/resources/imap-test/thread8.mbox",
    "content": "From user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <a@b>\n\nbody1\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <b@b>\nReferences: <a@b>\n\nbody2\n\nFrom user@domain  Fri Feb 22 15:06:23 2008\nMessage-Id: <b@b>\nReferences: <a@b>\n\nbody3\n"
  },
  {
    "path": "tests/resources/imap-test/uidplus",
    "content": "capabilities: UIDPLUS\nstate: created\n\nok select ${mailbox}\nok append\n\n\"\" delete ${mailbox}2\nok create ${mailbox}2\n\nappend ${mailbox}2\nok [appenduid $uidvalidity $uid]\n\ncopy 1 ${mailbox}2\nok [copyuid $uidvalidity $srcuid $uid2]\n\nok select ${mailbox}2\n* ok [uidvalidity $uidvalidity]\nok fetch 1:2 uid\n* 1 fetch (uid $uid)\n* 2 fetch (uid $uid2)\n\nok close\n\"\" delete ${mailbox}2\n"
  },
  {
    "path": "tests/resources/imap-test/uidvalidity",
    "content": "connections: 2\nstate: created\n\n1 ok append\n1 ok status $mailbox (uidvalidity uidnext)\n* status $mailbox (uidvalidity $uidvalidity uidnext $uidnext)\n1 ok delete $mailbox\n\n2 ok create $mailbox\n2 ok append $mailbox\n\n#1 ok status $mailbox (uidvalidity uidnext)\n#! status $mailbox (uidvalidity $uidvalidity uidnext $uidnext)\n"
  },
  {
    "path": "tests/resources/imap-test/uidvalidity-rename",
    "content": "connections: 2\nstate: auth\n\n1 ok create ${mailbox}\n2 ok create ${mailbox}2\n\n1 ok append\n2 ok append\n\n1 ok status $mailbox (uidvalidity uidnext)\n* status $mailbox (uidvalidity $uidvalidity uidnext $uidnext)\n\n1 ok rename ${mailbox} ${mailbox}3\n2 ok rename ${mailbox}2 ${mailbox}\n\n#1 ok status $mailbox (uidvalidity uidnext)\n#! status $mailbox (uidvalidity $uidvalidity uidnext $uidnext)\n\n1 \"\" delete ${mailbox}3\n"
  },
  {
    "path": "tests/resources/imap-test/urlauth",
    "content": "capabilities: urlauth\nmessages: 2\n\nok fetch 1:2 (uid body[])\n* 1 fetch (uid $uid1 body[] $body1)\n* 2 fetch (uid $uid2 body[] $body2)\n\nok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid1;urlauth=user+$user\" INTERNAL\n* GENURLAUTH $mail_url1\n\nok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid2;urlauth=user+$user\" INTERNAL\n* GENURLAUTH $mail_url2\n\nok URLFETCH $mail_url1\n* URLFETCH $mail_url1 $body1\n\nok URLFETCH $mail_url2\n* URLFETCH $mail_url2 $body2\n\nok resetkey $mailbox\n\nok URLFETCH $mail_url1\n* URLFETCH $mail_url1 NIL\n\nok URLFETCH $mail_url2\n* URLFETCH $mail_url2 NIL\n"
  },
  {
    "path": "tests/resources/imap-test/urlauth-binary",
    "content": "capabilities: urlauth urlauth=binary\nstate: created\n\nok select ${mailbox}\nok append\nok append\n\nok fetch 1:2 uid\n* 1 fetch (uid $uid1)\n* 2 fetch (uid $uid2)\n\nok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid1/;section=1;urlauth=user+$user\" INTERNAL\n* GENURLAUTH $mail_url1\nok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid1/;section=1.1;urlauth=user+$user\" INTERNAL\n* GENURLAUTH $mail_url1sub\n\nok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid2/;section=1;urlauth=user+$user\" INTERNAL\n* GENURLAUTH $mail_url2\nok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid2/;section=1.1;urlauth=user+$user\" INTERNAL\n* GENURLAUTH $mail_url2sub\n\nok URLFETCH ($mail_url1 binary)\n* URLFETCH $mail_url1 (binary {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: binary\n\nhello world\n--sub1\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: binary\n\nhello world\n--sub1--\nSub MIME epilogue\n\n}}})\n\nok URLFETCH ($mail_url1sub binary)\n* URLFETCH $mail_url1sub (binary {{{\nhello world\n}}})\n\nok URLFETCH ($mail_url1 bodypartstructure)\n* URLFETCH $mail_url1 (BODYPARTSTRUCTURE (\"message\" \"rfc822\" NIL NIL NIL \"7bit\" 437 (\"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" ((NIL NIL \"sub\" \"domain.org\")) ((NIL NIL \"sub\" \"domain.org\")) ((NIL NIL \"sub\" \"domain.org\")) NIL NIL NIL NIL NIL) ((\"text\" \"x-myown\" (\"charset\" \"us-ascii\") NIL NIL \"base64\" 22 3 NIL NIL NIL NIL)(\"text\" \"x-myown\" (\"charset\" \"us-ascii\") NIL NIL \"base64\" 47 5 NIL NIL NIL NIL) \"alternative\" (\"boundary\" \"sub1\") NIL NIL NIL) 26 NIL NIL NIL NIL))\n\nok URLFETCH ($mail_url1sub bodypartstructure)\n* URLFETCH $mail_url1sub (BODYPARTSTRUCTURE (\"text\" \"x-myown\" (\"charset\" \"us-ascii\") NIL NIL \"base64\" 22 3 NIL NIL NIL NIL))\n\n\nok URLFETCH ($mail_url2 binary bodypartstructure)\n* URLFETCH $mail_url2 (BODYPARTSTRUCTURE (\"message\" \"rfc822\" NIL NIL NIL \"7bit\" 390 (\"Sun, 12 Aug 2012 12:34:56 +0300\" \"submsg\" ((NIL NIL \"sub\" \"domain.org\")) ((NIL NIL \"sub\" \"domain.org\")) ((NIL NIL \"sub\" \"domain.org\")) NIL NIL NIL NIL NIL) ((\"text\" \"x-myown\" (\"charset\" \"us-ascii\") NIL NIL \"base64\" 11 0 NIL NIL NIL NIL)(\"text\" \"x-myown\" (\"charset\" \"us-ascii\") NIL NIL \"base64\" 11 0 NIL NIL NIL NIL) \"alternative\" (\"boundary\" \"sub1\") NIL NIL NIL) 18 NIL NIL NIL NIL) binary {{{\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: binary\n\nhello world\n--sub1\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: binary\n\nhello world\n--sub1--\nSub MIME epilogue\n\n}}})\n\nok URLFETCH ($mail_url1sub binary)\n* URLFETCH $mail_url1sub (binary {{{\nhello world\n}}})\n\nok URLFETCH ($mail_url2sub binary bodypartstructure)\n* URLFETCH $mail_url2sub (BODYPARTSTRUCTURE (\"text\" \"x-myown\" (\"charset\" \"us-ascii\") NIL NIL \"base64\" 11 0 NIL NIL NIL NIL) BINARY {{{\nhello world\n}}})\n"
  },
  {
    "path": "tests/resources/imap-test/urlauth-binary.mbox",
    "content": "From user@domain  Fri Feb 22 17:06:23 2008\nFrom: user@domain.org\nDate: Sat, 24 Mar 2007 23:00:00 +0200\nMime-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"foo\n bar\"\n\nRoot MIME prologue\n\n--foo bar\nContent-Type: message/rfc822\n\nFrom: sub@domain.org\nDate: Sun, 12 Aug 2012 12:34:56 +0300\nSubject: submsg\nContent-Type: multipart/alternative; boundary=\"sub1\"\n\nSub MIME prologue\n--sub1\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: base64\n\naGVs\nbG8gd29y\nbGQ=\n\n--sub1\nContent-Type: text/x-myown; charset=us-ascii\nContent-Transfer-Encoding: base64\n\n  aGVs    \n bG8g  \n   d29y  \t \n    bGQ= \n\n\n--sub1--\nSub MIME epilogue\n\n--foo bar--\nRoot MIME epilogue\n"
  },
  {
    "path": "tests/resources/imap-test/urlauth2",
    "content": "capabilities: urlauth\nconnections: 2\nuser 2: $user2\n\n1 ok fetch 1:2 (uid body[])\n* 1 fetch (uid $uid1 body[] $body1)\n* 2 fetch (uid $uid2 body[] $body2)\n\n1 ok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid1;urlauth=authuser\" INTERNAL\n* GENURLAUTH $mail_url1\n\n1 ok GENURLAUTH \"imap://$username@$domain/$mailbox_url/;uid=$uid2;urlauth=authuser\" INTERNAL\n* GENURLAUTH $mail_url2\n\n2 ok URLFETCH $mail_url1\n* URLFETCH $mail_url1 $body1\n\n2 ok URLFETCH $mail_url2\n* URLFETCH $mail_url2 $body2\n\n1 ok resetkey $mailbox\n\n2 ok URLFETCH $mail_url1\n* URLFETCH $mail_url1 NIL\n\n2 ok URLFETCH $mail_url2\n* URLFETCH $mail_url2 NIL\n"
  },
  {
    "path": "tests/resources/itip/google_calendar.txt",
    "content": "# Send initial request\n> put a@gmail.com 1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nBEGIN:VCALENDAR\nPRODID:-//Google Inc//Google Calendar 70.9054//EN\nVERSION:2.0\nCALSCALE:GREGORIAN\nBEGIN:VTIMEZONE\nTZID:America/Los_Angeles\nX-LIC-LOCATION:America/Los_Angeles\nBEGIN:DAYLIGHT\nTZOFFSETFROM:-0800\nTZOFFSETTO:-0700\nTZNAME:PDT\nDTSTART:19700308T020000\nRRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU\nEND:DAYLIGHT\nBEGIN:STANDARD\nTZOFFSETFROM:-0700\nTZOFFSETTO:-0800\nTZNAME:PST\nDTSTART:19701101T020000\nRRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nDTSTART;TZID=America/Los_Angeles:20250619T060000\nDTEND;TZID=America/Los_Angeles:20250619T064500\nRRULE:FREQ=WEEKLY;WKST=SU;COUNT=2;BYDAY=TH\nDTSTAMP:20250616T182403Z\nORGANIZER;CN=John Doe:mailto:a@gmail.com\nUID:1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=\n TRUE;CN=b@gmail.com;X-NUM-GUESTS=0:mailto:b@gmail.com\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE\n ;CN=John Doe;X-NUM-GUESTS=0:mailto:a@gmail.com\nX-MICROSOFT-CDO-OWNERAPPTID:299828133\nCREATED:20250616T182358Z\nLAST-MODIFIED:20250616T182358Z\nLOCATION:\nSEQUENCE:0\nSTATUS:CONFIRMED\nSUMMARY:Meet me maybe\nDESCRIPTION:This is the event description\nTRANSP:OPAQUE\nBEGIN:VALARM\nACTION:DISPLAY\nDESCRIPTION:This is an event reminder\nTRIGGER:-P0DT0H10M0S\nEND:VALARM\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@gmail.com\nto: b@gmail.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@gmail.com\", name: Some(\"John Doe\"), is_organizer: true }, ItipParticipant { email: \"b@gmail.com\", name: Some(\"b@gmail.com\"), is_organizer: false }])\nsummary.description: Text(\"This is the event description\")\nsummary.dtstart: Time(ItipTime { start: 1750338000, tz_id: 148 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Weekly, until: None, count: Some(2), interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: None, weekday: Thursday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: Some(Sunday), rscale: None, skip: None })\nsummary.summary: Text(\"Meet me maybe\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nX-MICROSOFT-CDO-OWNERAPPTID:299828133\nDESCRIPTION:This is the event description\nLOCATION:\nSTATUS:CONFIRMED\nSUMMARY:Meet me maybe\nDTEND;TZID=America/Los_Angeles:20250619T064500\nDTSTART;TZID=America/Los_Angeles:20250619T060000\nTRANSP:OPAQUE\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;\n CN=b@gmail.com;X-NUM-GUESTS=0:mailto:b@gmail.com\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE;\n CN=\"John Doe\";X-NUM-GUESTS=0:mailto:a@gmail.com\nORGANIZER;CN=\"John Doe\":mailto:a@gmail.com\nUID:1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nRRULE:FREQ=WEEKLY;COUNT=2;BYDAY=TH;WKST=SU\nCREATED:20250616T182358Z\nDTSTAMP:0\nLAST-MODIFIED:20250616T182358Z\nSEQUENCE:1\nBEGIN:VALARM\nDESCRIPTION:This is an event reminder\nACTION:DISPLAY\nTRIGGER:-PT10M\nEND:VALARM\nEND:VEVENT\nBEGIN:VTIMEZONE\nX-LIC-LOCATION:America/Los_Angeles\nTZID:America/Los_Angeles\nBEGIN:DAYLIGHT\nDTSTART:19700308T020000\nTZNAME:PDT\nTZOFFSETFROM:-0800\nTZOFFSETTO:-0700\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:19701101T020000\nTZNAME:PST\nTZOFFSETFROM:-0700\nTZOFFSETTO:-0800\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nEND:VTIMEZONE\nEND:VCALENDAR\n\n# Send iTIP\n> send\n\n# Update the event, expect no changes\n> put a@gmail.com 1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nBEGIN:VCALENDAR\nPRODID:-//Google Inc//Google Calendar 70.9054//EN\nVERSION:2.0\nCALSCALE:GREGORIAN\nBEGIN:VTIMEZONE\nTZID:America/Los_Angeles\nX-LIC-LOCATION:America/Los_Angeles\nBEGIN:DAYLIGHT\nTZOFFSETFROM:-0800\nTZOFFSETTO:-0700\nTZNAME:PDT\nDTSTART:19700308T020000\nRRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU\nEND:DAYLIGHT\nBEGIN:STANDARD\nTZOFFSETFROM:-0700\nTZOFFSETTO:-0800\nTZNAME:PST\nDTSTART:19701101T020000\nRRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nDTSTART;TZID=America/Los_Angeles:20250619T060000\nDTEND;TZID=America/Los_Angeles:20250619T064500\nRRULE:FREQ=WEEKLY;WKST=SU;COUNT=2;BYDAY=TH\nDTSTAMP:20250616T182416Z\nORGANIZER;CN=John Doe:mailto:a@gmail.com\nUID:1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=\n TRUE;CN=b@gmail.com;X-NUM-GUESTS=0:mailto:b@gmail.com\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE\n ;CN=John Doe;X-NUM-GUESTS=0:mailto:a@gmail.com\nX-MICROSOFT-CDO-OWNERAPPTID:299828133\nCREATED:20250616T182358Z\nLAST-MODIFIED:20250616T182415Z\nLOCATION:\nSEQUENCE:1\nSTATUS:CONFIRMED\nSUMMARY:Meet me maybe\nDESCRIPTION:This is the updated event description\nTRANSP:OPAQUE\nBEGIN:VALARM\nACTION:DISPLAY\nDESCRIPTION:This is an event reminder\nTRIGGER:-P0DT0H10M0S\nEND:VALARM\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@gmail.com\nto: b@gmail.com\nsummary: update REQUEST\nsummary.attendee: Participants([ItipParticipant { email: \"a@gmail.com\", name: Some(\"John Doe\"), is_organizer: true }, ItipParticipant { email: \"b@gmail.com\", name: Some(\"b@gmail.com\"), is_organizer: false }])\nsummary.description: Text(\"This is the updated event description\")\nsummary.dtstart: Time(ItipTime { start: 1750338000, tz_id: 148 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Weekly, until: None, count: Some(2), interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: None, weekday: Thursday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: Some(Sunday), rscale: None, skip: None })\nsummary.summary: Text(\"Meet me maybe\")\n~summary.description: Text(\"This is the event description\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nX-MICROSOFT-CDO-OWNERAPPTID:299828133\nDESCRIPTION:This is the updated event description\nLOCATION:\nSTATUS:CONFIRMED\nSUMMARY:Meet me maybe\nDTEND;TZID=America/Los_Angeles:20250619T064500\nDTSTART;TZID=America/Los_Angeles:20250619T060000\nTRANSP:OPAQUE\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;\n CN=b@gmail.com;X-NUM-GUESTS=0:mailto:b@gmail.com\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE;\n CN=\"John Doe\";X-NUM-GUESTS=0:mailto:a@gmail.com\nORGANIZER;CN=\"John Doe\":mailto:a@gmail.com\nUID:1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nRRULE:FREQ=WEEKLY;COUNT=2;BYDAY=TH;WKST=SU\nCREATED:20250616T182358Z\nDTSTAMP:0\nLAST-MODIFIED:20250616T182415Z\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VTIMEZONE\nX-LIC-LOCATION:America/Los_Angeles\nTZID:America/Los_Angeles\nBEGIN:DAYLIGHT\nDTSTART:19700308T020000\nTZNAME:PDT\nTZOFFSETFROM:-0800\nTZOFFSETTO:-0700\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:19701101T020000\nTZNAME:PST\nTZOFFSETFROM:-0700\nTZOFFSETTO:-0800\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nEND:VTIMEZONE\nEND:VCALENDAR\n\n> send\n\n# Make sure the original alarms are preserved\n> get b@gmail.com 1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nX-MICROSOFT-CDO-OWNERAPPTID:299828133\nDESCRIPTION:This is the updated event description\nLOCATION:\nSTATUS:CONFIRMED\nSUMMARY:Meet me maybe\nDTEND;TZID=America/Los_Angeles:20250619T064500\nDTSTART;TZID=America/Los_Angeles:20250619T060000\nTRANSP:OPAQUE\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;\n CN=b@gmail.com;X-NUM-GUESTS=0:mailto:b@gmail.com\nATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE;\n CN=\"John Doe\";X-NUM-GUESTS=0:mailto:a@gmail.com\nORGANIZER;CN=\"John Doe\":mailto:a@gmail.com\nUID:1qnf39p0m9h6n1cm9qa9d8mocv@google.com\nRRULE:FREQ=WEEKLY;COUNT=2;BYDAY=TH;WKST=SU\nCREATED:20250616T182358Z\nDTSTAMP:0\nLAST-MODIFIED:20250616T182358Z\nSEQUENCE:1\nBEGIN:VALARM\nDESCRIPTION:This is an event reminder\nACTION:DISPLAY\nTRIGGER:-PT10M\nEND:VALARM\nEND:VEVENT\nBEGIN:VTIMEZONE\nX-LIC-LOCATION:America/Los_Angeles\nTZID:America/Los_Angeles\nBEGIN:DAYLIGHT\nDTSTART:19700308T020000\nTZNAME:PDT\nTZOFFSETFROM:-0800\nTZOFFSETTO:-0700\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:19701101T020000\nTZNAME:PST\nTZOFFSETFROM:-0700\nTZOFFSETTO:-0800\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nEND:VTIMEZONE\nEND:VCALENDAR\n\n\n\n"
  },
  {
    "path": "tests/resources/itip/itip_incoming.txt",
    "content": "# iTIP tests\n\n# Recipient is not organizer or attendee\n> itip x@example.com y@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nMETHOD:REQUEST\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=C:mailto:c@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;RSVP=FALSE;CUTYPE=ROOM:conf_big@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> send\nNotOrganizerNorAttendee\n\n> reset\n\n# Refreshing an event (1)\n> put a@example.com 123456789@example.com\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:123456789@example.com\nSEQUENCE:0\nRDATE:19980304T180000Z\nRDATE:19980311T180000Z\nRDATE:19980318T180000Z\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:b@example.com\nSUMMARY:Review Accounts\nDTSTART:19980304T180000Z\nDTEND:19980304T200000Z\nDTSTAMP:19980303T193000Z\nLOCATION:Conference Room A\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n# Refreshing an event (2)\n> expect\nfrom: a@example.com\nto: b@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 889034400, tz_id: 32768 })\nsummary.location: Text(\"Conference Room A\")\nsummary.summary: Text(\"Review Accounts\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:Conference Room A\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nDTEND:19980304T200000Z\nDTSTART:19980304T180000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nRDATE:19980304T180000Z\nRDATE:19980311T180000Z\nRDATE:19980318T180000Z\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Refreshing an event (3)\n> send\n\n# Refreshing an event (4)\n> itip b@example.com a@example.com\nBEGIN:VCALENDAR\nMETHOD:REFRESH\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:123456789@example.com\nSEQUENCE:0\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:b@example.com\nDTSTART:19980304T180000Z\nEND:VEVENT\nEND:VCALENDAR\n\n# Refreshing an event (5)\n> send\nfrom: a@example.com\nto: b@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 889034400, tz_id: 32768 })\nsummary.location: Text(\"Conference Room A\")\nsummary.summary: Text(\"Review Accounts\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:Conference Room A\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nDTEND:19980304T200000Z\nDTSTART:19980304T180000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nRDATE:19980304T180000Z\nRDATE:19980311T180000Z\nRDATE:19980318T180000Z\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n\n\n"
  },
  {
    "path": "tests/resources/itip/put_validation.txt",
    "content": "# Scheduling invalid actions\n\n# Event has no scheduling information\n> put x@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nNoSchedulingInfo\n\n> reset\n\n# X is not the organizer\n> put x@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nNotOrganizerNorAttendee\n\n> reset\n\n# No attendees\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nNothingToSend\n\n> reset\n\n\n# Organizer with client scheduling agent\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER;SCHEDULE-AGENT=CLIENT:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nOtherSchedulingAgent\n\n> reset\n\n# Single participant with client scheduling agent\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;SCHEDULE-AGENT=CLIENT;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nNothingToSend\n\n> reset\n\n# Only send updates to clients with server scheduling agent\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:d@example.com\nATTENDEE;SCHEDULE-AGENT=CLIENT;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: d@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: Some(\"B\"), is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"B\"), is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867787200, tz_id: 32768 })\nsummary.summary: Text(\"Conference\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Conference\nDTEND:19970701T2100000Z\nDTSTART:19970701T200000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B;PARTSTAT=NEEDS-ACTION:mailto:b@exa\n mple.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B;PARTSTAT=NEEDS-ACTION:mailto:d@exa\n mple.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Writing the same event again should not send a new request\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:d@example.com\nATTENDEE;SCHEDULE-AGENT=CLIENT;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nNothingToSend\n\n# Adding a comment and a custom property should not send a new request\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:d@example.com\nATTENDEE;SCHEDULE-AGENT=CLIENT;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nCOMMENT:This is a comment\nX-EXAMPLE:This is an example\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nNothingToSend\n\n> reset\n\n# Multiple object types should be rejected\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VTODO\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VTODO\nEND:VCALENDAR\n\n> expect\nMultipleObjectTypes\n\n> reset\n\n# Multiple organizers should be rejected\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nORGANIZER:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=D:mailto:d@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nMultipleOrganizer\n\n> reset\n\n# Different UIDs should be rejected\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:other-uid@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nMultipleUid\n\n> reset\n\n# Multiple object instances should be rejected\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nRECURRENCE-ID:19970701T200000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nRECURRENCE-ID:19970701T200000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nMultipleObjectInstances\n\n"
  },
  {
    "path": "tests/resources/itip/rfc5546_event_recurring.txt",
    "content": "# RFC5546 - Group Event Request\n\n# A sample meeting request is sent from \"A\" to \"B\", \"C\", and \"D\".\n> put a@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:guid-1@example.com\nSEQUENCE:0\nRRULE:FREQ=MONTHLY;BYMONTHDAY=1;UNTIL=19980901T210000Z\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970601T210000Z\nDTEND:19970601T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970526T083000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com, d@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.description: Text(\"IETF-C&S Conference Call\")\nsummary.dtstart: Time(ItipTime { start: 865198800, tz_id: 32768 })\nsummary.location: Text(\"Conference Call\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: Some(PartialDateTime { year: Some(1998), month: Some(9), day: Some(1), hour: Some(21), minute: Some(0), second: Some(0), tz_hour: Some(0), tz_minute: Some(0), tz_minus: false }), count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [1], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"IETF Calendaring Working Group Meeting\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970601T220000Z\nDTSTART:19970601T210000Z\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nUID:guid-1@example.com\nRRULE:FREQ=MONTHLY;UNTIL=19980901T210000Z;BYMONTHDAY=1\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP request to attendees\n> send\n\n# Change a recurrence instance\n> put a@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:guid-1@example.com\nSEQUENCE:2\nRRULE:FREQ=MONTHLY;BYMONTHDAY=1;UNTIL=19980901T210000Z\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970601T210000Z\nDTEND:19970601T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970526T083000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nRECURRENCE-ID:19970701T210000Z\nSEQUENCE:1\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970703T210000Z\nDTEND:19970703T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970626T093000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com, d@example.com\nsummary: update ADD\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.description: Text(\"IETF-C&S Conference Call\")\nsummary.dtstart: Time(ItipTime { start: 865198800, tz_id: 32768 })\nsummary.location: Text(\"Conference Call\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: Some(PartialDateTime { year: Some(1998), month: Some(9), day: Some(1), hour: Some(21), minute: Some(0), second: Some(0), tz_hour: Some(0), tz_minute: Some(0), tz_minus: false }), count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [1], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"IETF Calendaring Working Group Meeting\")\nBEGIN:VCALENDAR\nMETHOD:ADD\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970703T220000Z\nDTSTART:19970703T210000Z\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970701T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP update to attendees\n> send\n\n# Cancel a recurrence instance only for B\n> put a@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:guid-1@example.com\nSEQUENCE:3\nRRULE:FREQ=MONTHLY;BYMONTHDAY=1;UNTIL=19980901T210000Z\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970601T210000Z\nDTEND:19970601T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970526T083000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nRECURRENCE-ID:19970701T210000Z\nSEQUENCE:2\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970703T210000Z\nDTEND:19970703T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970626T093000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nRECURRENCE-ID:19970801T210000Z\nSEQUENCE:2\nSTATUS:CANCELLED\nDTSTAMP:19970721T093000Z\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com\nsummary: cancel\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.description: Text(\"IETF-C&S Conference Call\")\nsummary.dtstart: Time(ItipTime { start: 865198800, tz_id: 32768 })\nsummary.location: Text(\"Conference Call\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: Some(PartialDateTime { year: Some(1998), month: Some(9), day: Some(1), hour: Some(21), minute: Some(0), second: Some(0), tz_hour: Some(0), tz_minute: Some(0), tz_minus: false }), count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [1], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"IETF Calendaring Working Group Meeting\")\nBEGIN:VCALENDAR\nMETHOD:CANCEL\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CANCELLED\nATTENDEE:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970801T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP cancellation to B\n> send\n\n# Make sure B has the cancelled event\n> get b@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970601T220000Z\nDTSTART:19970601T210000Z\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nUID:guid-1@example.com\nRRULE:FREQ=MONTHLY;UNTIL=19980901T210000Z;BYMONTHDAY=1\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970703T220000Z\nDTSTART:19970703T210000Z\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970701T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nBEGIN:VEVENT\nSTATUS:CANCELLED\nATTENDEE:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970801T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n\n# Change all future instances\n> put a@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:guid-1@example.com\nSEQUENCE:4\nRRULE:FREQ=MONTHLY;BYMONTHDAY=1;UNTIL=19980901T210000Z\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970601T210000Z\nDTEND:19970601T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970526T083000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nRECURRENCE-ID:19970701T210000Z\nSEQUENCE:3\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970703T210000Z\nDTEND:19970703T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970626T093000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nRECURRENCE-ID:19970801T210000Z\nSEQUENCE:3\nSTATUS:CANCELLED\nDTSTAMP:19970721T093000Z\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nSEQUENCE:3\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:b@example.com\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nATTENDEE;RSVP=TRUE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Discussion\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970901T210000Z\nDTEND:19970901T220000Z\nLOCATION:Building 32, Microsoft, Seattle, WA\nDTSTAMP:19970526T083000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com, d@example.com\nsummary: update ADD\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.description: Text(\"IETF-C&S Conference Call\")\nsummary.dtstart: Time(ItipTime { start: 865198800, tz_id: 32768 })\nsummary.location: Text(\"Conference Call\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: Some(PartialDateTime { year: Some(1998), month: Some(9), day: Some(1), hour: Some(21), minute: Some(0), second: Some(0), tz_hour: Some(0), tz_minute: Some(0), tz_minus: false }), count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [1], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"IETF Calendaring Working Group Meeting\")\nBEGIN:VCALENDAR\nMETHOD:ADD\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Discussion\nLOCATION:Building 32\\, Microsoft\\, Seattle\\, WA\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970901T220000Z\nDTSTART:19970901T210000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:4\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP update to attendees\n> send\n\n# Make sure B has the complete event including all updates\n> get b@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970601T220000Z\nDTSTART:19970601T210000Z\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nUID:guid-1@example.com\nRRULE:FREQ=MONTHLY;UNTIL=19980901T210000Z;BYMONTHDAY=1\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970703T220000Z\nDTSTART:19970703T210000Z\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970701T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nBEGIN:VEVENT\nSTATUS:CANCELLED\nATTENDEE:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970801T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Discussion\nLOCATION:Building 32\\, Microsoft\\, Seattle\\, WA\nSTATUS:CONFIRMED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970901T220000Z\nDTSTART:19970901T210000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:4\nEND:VEVENT\nEND:VCALENDAR\n\n# Cancel the recurring event\n> put a@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:guid-1@example.com\nSEQUENCE:4\nRRULE:FREQ=MONTHLY;BYMONTHDAY=1;UNTIL=19980901T210000Z\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970601T210000Z\nDTEND:19970601T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970526T083000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nRECURRENCE-ID:19970701T210000Z\nSEQUENCE:3\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Conference Call\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970703T210000Z\nDTEND:19970703T220000Z\nLOCATION:Conference Call\nDTSTAMP:19970626T093000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE:mailto:b@example.com\nRECURRENCE-ID:19970801T210000Z\nSEQUENCE:3\nSTATUS:CANCELLED\nDTSTAMP:19970721T093000Z\nEND:VEVENT\nBEGIN:VEVENT\nUID:guid-1@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nSEQUENCE:3\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:b@example.com\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nATTENDEE;RSVP=TRUE:mailto:d@example.com\nDESCRIPTION:IETF-C&S Discussion\nCLASS:PUBLIC\nSUMMARY:IETF Calendaring Working Group Meeting\nDTSTART:19970901T210000Z\nDTEND:19970901T220000Z\nLOCATION:Building 32, Microsoft, Seattle, WA\nDTSTAMP:19970526T083000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n# Cancel a recurring event\n> delete a@example.com guid-1@example.com\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com, d@example.com\nsummary: cancel\nsummary.description: Text(\"IETF-C&S Conference Call\")\nsummary.dtstart: Time(ItipTime { start: 865198800, tz_id: 32768 })\nsummary.location: Text(\"Conference Call\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: Some(PartialDateTime { year: Some(1998), month: Some(9), day: Some(1), hour: Some(21), minute: Some(0), second: Some(0), tz_hour: Some(0), tz_minute: Some(0), tz_minus: false }), count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [1], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"IETF Calendaring Working Group Meeting\")\nBEGIN:VCALENDAR\nMETHOD:CANCEL\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CANCELLED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970601T220000Z\nDTSTART:19970601T210000Z\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:5\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP cancellation to attendees\n> send\n\n# Make sure all instances in B were deleted\n> get b@example.com guid-1@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CANCELLED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970601T220000Z\nDTSTART:19970601T210000Z\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nUID:guid-1@example.com\nRRULE:FREQ=MONTHLY;UNTIL=19980901T210000Z;BYMONTHDAY=1\nDTSTAMP:0\nEND:VEVENT\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Conference Call\nLOCATION:Conference Call\nSTATUS:CANCELLED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970703T220000Z\nDTSTART:19970703T210000Z\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970701T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nBEGIN:VEVENT\nSTATUS:CANCELLED\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19970801T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nBEGIN:VEVENT\nCLASS:PUBLIC\nDESCRIPTION:IETF-C&S Discussion\nLOCATION:Building 32\\, Microsoft\\, Seattle\\, WA\nSTATUS:CANCELLED\nSUMMARY:IETF Calendaring Working Group Meeting\nDTEND:19970901T220000Z\nDTSTART:19970901T210000Z\nATTENDEE:mailto:b@example.com\nATTENDEE:mailto:c@example.com\nATTENDEE:mailto:d@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nUID:guid-1@example.com\nDTSTAMP:0\nSEQUENCE:4\nEND:VEVENT\nEND:VCALENDAR\n\n# Add a new series of instances to the recurring event\n> put a@example.com 123456789@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:123456789@example.com\nSEQUENCE:0\nRRULE:WKST=SU;BYDAY=TU;FREQ=WEEKLY\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:b@example.com\nSUMMARY:Review Accounts\nDTSTART:19980303T210000Z\nDTEND:19980303T220000Z\nLOCATION:The White Room\nDTSTAMP:19980301T093000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nBEGIN:VEVENT\nUID:123456789@example.com\nSEQUENCE:2\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nSUMMARY:Review Accounts\nDTSTAMP:19980303T193000Z\nLOCATION:The Red Room\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 888958800, tz_id: 32768 })\nsummary.location: Text(\"The White Room\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Weekly, until: None, count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: None, weekday: Tuesday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: Some(Sunday), rscale: None, skip: None })\nsummary.summary: Text(\"Review Accounts\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:The White Room\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nDTEND:19980303T220000Z\nDTSTART:19980303T210000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nRRULE:FREQ=WEEKLY;BYDAY=TU;WKST=SU\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VEVENT\nLOCATION:The Red Room\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nUID:123456789@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP request to B and C\n> send\n\n# Add a new series of instances to the recurring event (update)\n> put a@example.com 123456789@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nUID:123456789@example.com\nSEQUENCE:2\nRRULE:WKST=SU;BYDAY=TU,TH;FREQ=WEEKLY\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:b@example.com\nSUMMARY:Review Accounts\nDTSTART:19980303T210000Z\nDTEND:19980303T220000Z\nDTSTAMP:19980303T193000Z\nLOCATION:The White Room\nSTATUS:CONFIRMED\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com\nsummary: update REQUEST\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 888958800, tz_id: 32768 })\nsummary.location: Text(\"The White Room\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Weekly, until: None, count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: None, weekday: Tuesday }, ICalendarDay { ordwk: None, weekday: Thursday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: Some(Sunday), rscale: None, skip: None })\nsummary.summary: Text(\"Review Accounts\")\n~summary.rrule: Rrule(ICalendarRecurrenceRule { freq: Weekly, until: None, count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: None, weekday: Tuesday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: Some(Sunday), rscale: None, skip: None })\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:The White Room\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nDTEND:19980303T220000Z\nDTSTART:19980303T210000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nRRULE:FREQ=WEEKLY;BYDAY=TU,TH;WKST=SU\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n================================\nfrom: a@example.com\nto: c@example.com\nsummary: cancel\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 888958800, tz_id: 32768 })\nsummary.location: Text(\"The White Room\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Weekly, until: None, count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: None, weekday: Tuesday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: Some(Sunday), rscale: None, skip: None })\nsummary.summary: Text(\"Review Accounts\")\nBEGIN:VCALENDAR\nMETHOD:CANCEL\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:The Red Room\nSTATUS:CANCELLED\nSUMMARY:Review Accounts\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nUID:123456789@example.com\nDTSTAMP:0\nSEQUENCE:4\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP request to B and cancellation to C\n> send\n\n# Make sure B has the updated event\n> get b@example.com 123456789@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:The White Room\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nDTEND:19980303T220000Z\nDTSTART:19980303T210000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nRRULE:FREQ=WEEKLY;BYDAY=TU,TH;WKST=SU\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n\n# Make sure C has the updated event\n> get c@example.com 123456789@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:The Red Room\nSTATUS:CANCELLED\nSUMMARY:Review Accounts\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID;THISANDFUTURE:19970901T210000Z\nUID:123456789@example.com\nDTSTAMP:0\nEND:VEVENT\nBEGIN:VEVENT\nLOCATION:The White Room\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nDTEND:19980303T220000Z\nDTSTART:19980303T210000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nRRULE:FREQ=WEEKLY;BYDAY=TU;WKST=SU\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Delete the event from B\n> delete b@example.com 123456789@example.com\n\n> expect\nfrom: b@example.com\nto: a@example.com\nsummary: rsvp DECLINED\nsummary.dtstart: Time(ItipTime { start: 888958800, tz_id: 32768 })\nsummary.location: Text(\"The White Room\")\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Weekly, until: None, count: None, interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: None, weekday: Tuesday }, ICalendarDay { ordwk: None, weekday: Thursday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: Some(Sunday), rscale: None, skip: None })\nsummary.summary: Text(\"Review Accounts\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Review Accounts\nDTEND:19980303T220000Z\nDTSTART:19980303T210000Z\nATTENDEE;PARTSTAT=DECLINED:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n\n> send\n\n# Make sure A has the cancellation from B\n> get a@example.com 123456789@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nLOCATION:The White Room\nSTATUS:CONFIRMED\nSUMMARY:Review Accounts\nDTEND:19980303T220000Z\nDTSTART:19980303T210000Z\nATTENDEE;PARTSTAT=DECLINED:mailto:b@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nORGANIZER:mailto:a@example.com\nUID:123456789@example.com\nRRULE:FREQ=WEEKLY;BYDAY=TU,TH;WKST=SU\nDTSTAMP:1\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n\n"
  },
  {
    "path": "tests/resources/itip/rfc5546_event_single.txt",
    "content": "# RFC5546 - Group Event Request\n\n# A sample meeting request is sent from \"A\" to \"B\", \"C\", and \"D\".\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=C:mailto:c@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;RSVP=FALSE;CUTYPE=ROOM:conf_big@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nDTSTAMP:19970611T190000Z\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:0\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com, d@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: Some(\"B\"), is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: Some(\"C\"), is_organizer: false }, ItipParticipant { email: \"conf_big@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867787200, tz_id: 32768 })\nsummary.summary: Text(\"Conference\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Conference\nDTEND:19970701T2100000Z\nDTSTART:19970701T200000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nATTENDEE;RSVP=FALSE;CUTYPE=ROOM:conf_big@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B;PARTSTAT=NEEDS-ACTION:mailto:b@exa\n mple.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=C;PARTSTAT=NEEDS-ACTION:mailto:c@exa\n mple.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP request to B, C, and D\n> send\n\n# Make sure B receives the request\n> get b@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Conference\nDTEND:19970701T2100000Z\nDTSTART:19970701T200000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nATTENDEE;RSVP=FALSE;CUTYPE=ROOM:conf_big@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B;PARTSTAT=NEEDS-ACTION:mailto:b@exa\n mple.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=C;PARTSTAT=NEEDS-ACTION:mailto:c@exa\n mple.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# B accepts the request\n> put b@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nBEGIN:VEVENT\nDTSTAMP:19970701T200000Z\nSEQUENCE:1\nUID:calsrv.example.com-873970198738777@example.com\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B;PARTSTAT=ACCEPTED:mailto:b@exa\n mple.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=C;PARTSTAT=NEEDS-ACTION:mailto:c@exa\n mple.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=FALSE;CUTYPE=ROOM:conf_big@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nDTSTART:19970701T200000Z\nDTEND:19970701T2100000Z\nSUMMARY:Conference\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: b@example.com\nto: a@example.com\nsummary: rsvp ACCEPTED\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: Some(\"B\"), is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: Some(\"C\"), is_organizer: false }, ItipParticipant { email: \"conf_big@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867787200, tz_id: 32768 })\nsummary.summary: Text(\"Conference\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Conference\nDTEND:19970701T2100000Z\nDTSTART:19970701T200000Z\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B;PARTSTAT=ACCEPTED:mailto:b@example\n .com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:1\nREQUEST-STATUS:2.0;Success\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP reply to A\n> send\n\n# Make sure A receives the reply\n> get a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Conference\nDTEND:19970701T2100000Z\nDTSTART:19970701T200000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED;CN=A:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nATTENDEE;RSVP=FALSE;CUTYPE=ROOM:conf_big@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=B;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=\n 2.0:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=C:mailto:c@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:1\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# A moved the event to a new time\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:c@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nDTSTART:19970701T180000Z\nDTEND:19970701T190000Z\nSUMMARY:Phone Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:1\nDTSTAMP:19970613T190000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com, d@example.com\nsummary: update REQUEST\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\n~summary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: Some(\"B\"), is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: Some(\"C\"), is_organizer: false }, ItipParticipant { email: \"conf_big@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\n~summary.dtstart: Time(ItipTime { start: 867787200, tz_id: 32768 })\n~summary.summary: Text(\"Conference\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:c@example.\n com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP request to B, C, and D\n> send\n\n# Make sure C receives the request\n> get c@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE:mailto:e@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:c@example.\n com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# \"C\" delegates presence at the meeting to \"E\".\n> put c@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nBEGIN:VEVENT\nDTSTAMP:19970701T180000Z\nUID:calsrv.example.com-873970198738777@example.com\nORGANIZER:mailto:a@example.com\nSTATUS:CONFIRMED\nSEQUENCE:2\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@example.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\":mailto:e@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nDTSTART:19970701T180000Z\nDTEND:19970701T190000Z\nSUMMARY:Phone Conference\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: c@example.com\nto: a@example.com\nsummary: rsvp DELEGATED\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@examp\n le.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\":mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:2\nREQUEST-STATUS:2.0;Success\nEND:VEVENT\nEND:VCALENDAR\n================================\nfrom: c@example.com\nto: e@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@examp\n le.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=NEEDS-ACTIO\n N:mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n> send\n\n# Make sure E receives the request\n> get e@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@ex\n ample.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=NEEDS-AC\n TION:mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Make sure A receives the request\n> get a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;CUTYPE=INDIVIDUAL;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@exam\n ple.com\";SCHEDULE-STATUS=2.0:mailto:c@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\":mailto:e@example.c\n om\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:2\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Delegate E accepts the request\n> put e@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nBEGIN:VEVENT\nDTSTAMP:19970701T180000Z\nSEQUENCE:2\nUID:calsrv.example.com-873970198738777@example.com\nORGANIZER:mailto:a@example.com\nSTATUS:CONFIRMED\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@ex\n ample.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=ACCEPTED:mailto:e@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nDTSTART:19970701T180000Z\nDTEND:19970701T190000Z\nSUMMARY:Phone Conference\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: e@example.com\nto: a@example.com, c@example.com\nsummary: rsvp ACCEPTED\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@examp\n le.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=ACCEPTED:mai\n lto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:2\nREQUEST-STATUS:2.0;Success\nEND:VEVENT\nEND:VCALENDAR\n\n> send\n\n# Make sure A receives the reply\n> get a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;CUTYPE=INDIVIDUAL;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@exam\n ple.com\";SCHEDULE-STATUS=2.0:mailto:c@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=ACCEPTED;\n SCHEDULE-STATUS=2.0:mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:2\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Make sure C receives the reply\n> get c@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@ex\n ample.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=ACCEPTED;\n SCHEDULE-STATUS=2.0:mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:3\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Delegate E declines the request\n> put e@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nBEGIN:VEVENT\nDTSTAMP:19970701T180000Z\nSEQUENCE:2\nUID:calsrv.example.com-873970198738777@example.com\nORGANIZER:mailto:a@example.com\nSTATUS:CONFIRMED\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@ex\n ample.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=DECLINED:mailto:e@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nDTSTART:19970701T180000Z\nDTEND:19970701T190000Z\nSUMMARY:Phone Conference\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: e@example.com\nto: a@example.com, c@example.com\nsummary: rsvp DECLINED\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@examp\n le.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=DECLINED:mai\n lto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:2\nREQUEST-STATUS:2.0;Success\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP reply to A and C\n> send\n\n# Make sure C receives the reply\n> get c@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example.com\":mailto:c@ex\n ample.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=DECLINED;\n SCHEDULE-STATUS=2.0:mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:3\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Make sure A receives the reply\n> get a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;CUTYPE=INDIVIDUAL;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@exam\n ple.com\";SCHEDULE-STATUS=2.0:mailto:c@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=DECLINED;\n SCHEDULE-STATUS=2.0:mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:2\nSEQUENCE:2\nEND:VEVENT\nEND:VCALENDAR\n\n# Remove B and expect an iTIP cancelation\n> put a@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VEVENT\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;CUTYPE=INDIVIDUAL;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@exam\n ple.com\";SCHEDULE-STATUS=2.0:mailto:c@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=DECLINED;\n SCHEDULE-STATUS=2.0:mailto:e@example.com\nDTSTART:19970701T180000Z\nDTEND:19970701T190000Z\nSUMMARY:Phone Conference\nUID:calsrv.example.com-873970198738777@example.com\nSEQUENCE:2\nDTSTAMP:19970701T180000Z\nSTATUS:CONFIRMED\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: c@example.com, d@example.com\nsummary: update REQUEST\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\n~summary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CONFIRMED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;CUTYPE=INDIVIDUAL;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example\n .com\":mailto:c@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal;PARTSTAT=NEEDS-ACTION:mailto:d@e\n xample.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=DECLINED:mai\n lto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n================================\nfrom: a@example.com\nto: b@example.com\nsummary: cancel\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"conf@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: Some(\"Hal\"), is_organizer: false }, ItipParticipant { email: \"e@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\nBEGIN:VCALENDAR\nMETHOD:CANCEL\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CANCELLED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:3\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP cancelation\n> send\n\n# Make sure B receives the cancelation\n> get b@example.com calsrv.example.com-873970198738777@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CANCELLED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nEND:VEVENT\nEND:VCALENDAR\n\n# Delete the event from A's calendar and expect an iTIP cancelation\n> delete a@example.com calsrv.example.com-873970198738777@example.com\n\n> expect\nfrom: a@example.com\nto: c@example.com, d@example.com\nsummary: cancel\nsummary.dtstart: Time(ItipTime { start: 867780000, tz_id: 32768 })\nsummary.summary: Text(\"Phone Conference\")\nBEGIN:VCALENDAR\nMETHOD:CANCEL\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSTATUS:CANCELLED\nSUMMARY:Phone Conference\nDTEND:19970701T190000Z\nDTSTART:19970701T180000Z\nATTENDEE;CUTYPE=INDIVIDUAL;PARTSTAT=DELEGATED;DELEGATED-TO=\"mailto:e@example\n .com\";SCHEDULE-STATUS=2.0:mailto:c@example.com\nATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:mailto:a@example.com\nATTENDEE;ROLE=NON-PARTICIPANT;RSVP=FALSE;CUTYPE=ROOM:mailto:conf@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;CN=Hal:mailto:d@example.com\nATTENDEE;RSVP=TRUE;DELEGATED-FROM=\"mailto:c@example.com\";PARTSTAT=DECLINED;\n SCHEDULE-STATUS=2.0:mailto:e@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777@example.com\nDTSTAMP:0\nSEQUENCE:4\nEND:VEVENT\nEND:VCALENDAR\n\n\n"
  },
  {
    "path": "tests/resources/itip/rfc5546_todo.txt",
    "content": "# RFC5546 - Todo Request\n\n# A sample todo is sent from \"A\" to \"B\", \"C\", and \"D\".\n> put a@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nMETHOD:REQUEST\nVERSION:2.0\nBEGIN:VTODO\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:b@example.com\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nATTENDEE;RSVP=TRUE:mailto:d@example.com\nDTSTART:19970701T170000Z\nDUE:19970722T170000Z\nPRIORITY:1\nSUMMARY:Create the requirements document\nUID:calsrv.example.com-873970198738777-00@example.com\nSEQUENCE:0\nDTSTAMP:19970717T200000Z\nSTATUS:NEEDS-ACTION\nEND:VTODO\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, c@example.com, d@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867776400, tz_id: 32768 })\nsummary.summary: Text(\"Create the requirements document\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nPRIORITY:1\nSTATUS:NEEDS-ACTION\nSUMMARY:Create the requirements document\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=NEEDS-ACTION:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n# Send iTIP request to the attendees\n> send\n\n# Make sure B received the todo\n> get b@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nPRIORITY:1\nSTATUS:NEEDS-ACTION\nSUMMARY:Create the requirements document\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=NEEDS-ACTION:mailto:a@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:b@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:c@example.com\nATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n# \"B\" accepts the to-do.\n> put b@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nORGANIZER:mailto:a@example.com\nATTENDEE;PARTSTAT=ACCEPTED:mailto:b@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nCOMMENT:I'll send you my input by email\nSEQUENCE:1\nPRIORITY:1\nSTATUS:IN-PROCESS\nDTSTART:19970701T170000Z\nDUE:19970722T170000Z\nDTSTAMP:19970717T203000Z\nEND:VTODO\nEND:VCALENDAR\n\n> expect\nfrom: b@example.com\nto: a@example.com\nsummary: rsvp ACCEPTED\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867776400, tz_id: 32768 })\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nSTATUS:IN-PROCESS\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;PARTSTAT=ACCEPTED:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:1\nREQUEST-STATUS:2.0;Success\nEND:VTODO\nEND:VCALENDAR\n\n# Send iTIP reply to the organizer\n> send\n\n# Make sure \"A\" received the reply\n> get a@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nPRIORITY:1\nSTATUS:IN-PROCESS\nSUMMARY:Create the requirements document\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:b@example.com\nATTENDEE;ROLE=CHAIR:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nATTENDEE;RSVP=TRUE:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:1\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n# \"B\" updates percent completion of the to-do.\n> put b@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nORGANIZER:mailto:a@example.com\nPERCENT-COMPLETE:75\nATTENDEE;PARTSTAT=IN-PROCESS:mailto:b@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nCOMMENT:I'll send you my input by email\nSEQUENCE:1\nPRIORITY:1\nSTATUS:IN-PROCESS\nDTSTART:19970701T170000Z\nDUE:19970722T170000Z\nDTSTAMP:19970717T203000Z\nEND:VTODO\nEND:VCALENDAR\n\n> expect\nfrom: b@example.com\nto: a@example.com\nsummary: rsvp IN-PROCESS\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867776400, tz_id: 32768 })\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nPERCENT-COMPLETE:75\nSTATUS:IN-PROCESS\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;PARTSTAT=IN-PROCESS:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:1\nREQUEST-STATUS:2.0;Success\nEND:VTODO\nEND:VCALENDAR\n\n# Send iTIP reply to the organizer\n> send\n\n# Make sure \"A\" received the reply\n> get a@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nPERCENT-COMPLETE:75\nPRIORITY:1\nSTATUS:IN-PROCESS\nSUMMARY:Create the requirements document\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;PARTSTAT=IN-PROCESS;SCHEDULE-STATUS=2.0:mailto:b@example.com\nATTENDEE;ROLE=CHAIR:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nATTENDEE;RSVP=TRUE:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:1\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n# \"D\" completed the to-do.\n> put d@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nORGANIZER:mailto:a@example.com\nATTENDEE;PARTSTAT=COMPLETED:mailto:d@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nCOMMENT:I'll send you my input by email\nSEQUENCE:1\nPRIORITY:1\nDTSTART:19970701T170000Z\nDUE:19970722T170000Z\nDTSTAMP:19970717T203000Z\nEND:VTODO\nEND:VCALENDAR\n\n> expect\nfrom: d@example.com\nto: a@example.com\nsummary: rsvp COMPLETED\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867776400, tz_id: 32768 })\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;PARTSTAT=COMPLETED:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:1\nREQUEST-STATUS:2.0;Success\nEND:VTODO\nEND:VCALENDAR\n\n# Send iTIP reply to the organizer\n> send\n\n# Make sure \"A\" received the reply\n> get a@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nPERCENT-COMPLETE:75\nPRIORITY:1\nSTATUS:IN-PROCESS\nSUMMARY:Create the requirements document\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;PARTSTAT=COMPLETED;SCHEDULE-STATUS=2.0:mailto:d@example.com\nATTENDEE;PARTSTAT=IN-PROCESS;SCHEDULE-STATUS=2.0:mailto:b@example.com\nATTENDEE;ROLE=CHAIR:mailto:a@example.com\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:1\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n# Recurring to-do request\n> put a@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nORGANIZER:mailto:a@example.com\nATTENDEE;ROLE=CHAIR:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:d@example.com\nRRULE:FREQ=MONTHLY;COUNT=10;BYDAY=1FR\nDTSTART:19980101T100000Z\nDUE:19980103T100000Z\nSUMMARY:Send Status Reports to Area Managers\nUID:calsrv.example.com-873970198738777-00@example.com\nSEQUENCE:0\nDTSTAMP:19970717T200000Z\nSTATUS:NEEDS-ACTION\nPRIORITY:1\nEND:VTODO\nEND:VCALENDAR\n\n> expect\nfrom: a@example.com\nto: b@example.com, d@example.com\nsummary: update REQUEST\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 883648800, tz_id: 32768 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: None, count: Some(10), interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: Some(1), weekday: Friday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"Send Status Reports to Area Managers\")\n~summary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\n~summary.dtstart: Time(ItipTime { start: 867776400, tz_id: 32768 })\n~summary.summary: Text(\"Create the requirements document\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nPRIORITY:1\nSTATUS:NEEDS-ACTION\nSUMMARY:Send Status Reports to Area Managers\nDUE:19980103T100000Z\nDTSTART:19980101T100000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=NEEDS-ACTION:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:d@example.\n com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nRRULE:FREQ=MONTHLY;COUNT=10;BYDAY=1FR\nDTSTAMP:0\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n================================\nfrom: a@example.com\nto: c@example.com\nsummary: cancel\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"c@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 867776400, tz_id: 32768 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: None, count: Some(10), interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: Some(1), weekday: Friday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"Create the requirements document\")\nBEGIN:VCALENDAR\nMETHOD:CANCEL\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nSTATUS:CANCELLED\nSUMMARY:Create the requirements document\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:2\nEND:VTODO\nEND:VCALENDAR\n\n> send\n\n# Make sure \"C\" received the cancel request\n> get c@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nPRIORITY:1\nSTATUS:CANCELLED\nSUMMARY:Create the requirements document\nDUE:19970722T170000Z\nDTSTART:19970701T170000Z\nATTENDEE;RSVP=TRUE:mailto:c@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nEND:VTODO\nEND:VCALENDAR\n\n# Make sure \"B\" received the updated to-do\n> get b@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nCOMMENT:I'll send you my input by email\nPRIORITY:1\nSTATUS:NEEDS-ACTION\nSUMMARY:Send Status Reports to Area Managers\nDUE:19980103T100000Z\nDTSTART:19980101T100000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=NEEDS-ACTION:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:d@example.\n com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nRRULE:FREQ=MONTHLY;COUNT=10;BYDAY=1FR\nDTSTAMP:0\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n# Reply to an instance of a recurring to-do\n> put b@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nCOMMENT:I'll send you my input by email\nPRIORITY:1\nSTATUS:NEEDS-ACTION\nSUMMARY:Send Status Reports to Area Managers\nDUE:19980103T100000Z\nDTSTART:19980101T100000Z\nATTENDEE;ROLE=CHAIR;PARTSTAT=NEEDS-ACTION:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:b@example.\n com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION:mailto:d@example.\n com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nRRULE:FREQ=MONTHLY;COUNT=10;BYDAY=1FR\nDTSTAMP:19970717T233000Z\nSEQUENCE:1\nEND:VTODO\nBEGIN:VTODO\nORGANIZER:mailto:a@example.com\nATTENDEE;PARTSTAT=IN-PROCESS:mailto:b@example.com\nPERCENT-COMPLETE:75\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:19970717T233000Z\nRECURRENCE-ID:19980101T170000Z\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n> expect\nfrom: b@example.com\nto: a@example.com\nsummary: rsvp NEEDS-ACTION\nsummary.attendee: Participants([ItipParticipant { email: \"a@example.com\", name: None, is_organizer: true }, ItipParticipant { email: \"b@example.com\", name: None, is_organizer: false }, ItipParticipant { email: \"d@example.com\", name: None, is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 883648800, tz_id: 32768 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Monthly, until: None, count: Some(10), interval: None, bysecond: [], byminute: [], byhour: [], byday: [ICalendarDay { ordwk: Some(1), weekday: Friday }], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"Send Status Reports to Area Managers\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VTODO\nPERCENT-COMPLETE:75\nATTENDEE;PARTSTAT=IN-PROCESS:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19980101T170000Z\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:1\nREQUEST-STATUS:2.0;Success\nEND:VTODO\nEND:VCALENDAR\n\n> send\n\n# Make sure \"A\" received the reply\n> get a@example.com calsrv.example.com-873970198738777-00@example.com\nBEGIN:VCALENDAR\nPRODID:-//Example/ExampleCalendarClient//EN\nVERSION:2.0\nBEGIN:VTODO\nPRIORITY:1\nSTATUS:NEEDS-ACTION\nSUMMARY:Send Status Reports to Area Managers\nDUE:19980103T100000Z\nDTSTART:19980101T100000Z\nATTENDEE;ROLE=CHAIR:mailto:a@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:b@example.com\nATTENDEE;RSVP=TRUE;CUTYPE=INDIVIDUAL:mailto:d@example.com\nORGANIZER:mailto:a@example.com\nUID:calsrv.example.com-873970198738777-00@example.com\nRRULE:FREQ=MONTHLY;COUNT=10;BYDAY=1FR\nDTSTAMP:1\nSEQUENCE:1\nEND:VTODO\nBEGIN:VTODO\nPERCENT-COMPLETE:75\nATTENDEE;PARTSTAT=IN-PROCESS:mailto:b@example.com\nORGANIZER:mailto:a@example.com\nRECURRENCE-ID:19980101T170000Z\nUID:calsrv.example.com-873970198738777-00@example.com\nDTSTAMP:0\nSEQUENCE:1\nEND:VTODO\nEND:VCALENDAR\n\n\n"
  },
  {
    "path": "tests/resources/itip/rfc6638_recurring.txt",
    "content": "# RFC6638 - Recurring Event Scheduling\n\n# Organizer invites participant to a recurring event\n> put cyrus@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nRRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nRRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nEND:VTIMEZONE\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:0\nDTSTAMP:20090602T185254Z\nDTSTART;TZID=America/Montreal:20090601T150000\nDTEND;TZID=America/Montreal:20090601T160000\nRRULE:FREQ=DAILY;INTERVAL=1;COUNT=5\nTRANSP:OPAQUE\nSUMMARY:Review Internet-Draft\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@example.com\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: cyrus@example.com\nto: bernard@example.net\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"cyrus@example.com\", name: Some(\"Cyrus Daboo\"), is_organizer: true }, ItipParticipant { email: \"bernard@example.net\", name: Some(\"Bernard Desruisseaux\"), is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 1243882800, tz_id: 167 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Daily, until: None, count: Some(5), interval: Some(1), bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"Review Internet-Draft\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Review Internet-Draft\nDTEND;TZID=America/Montreal:20090601T160000\nDTSTART;TZID=America/Montreal:20090601T150000\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;\n RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nRRULE:FREQ=DAILY;COUNT=5;INTERVAL=1\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n\n# Send iTIP message to participant\n> send\n\n# Make sure the participant receives the event\n> get bernard@example.net 9263504FD3AD\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Review Internet-Draft\nDTEND;TZID=America/Montreal:20090601T160000\nDTSTART;TZID=America/Montreal:20090601T150000\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;RSVP=\n TRUE;PARTSTAT=NEEDS-ACTION:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nRRULE:FREQ=DAILY;COUNT=5;INTERVAL=1\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n\n# Participant declines an instance of the recurring event\n> put bernard@example.net 9263504FD3AD\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nRRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nRRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nEND:VTIMEZONE\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:1\nDTSTAMP:20090602T185254Z\nDTSTART;TZID=America/Montreal:20090601T150000\nDTEND;TZID=America/Montreal:20090601T160000\nRRULE:FREQ=DAILY;INTERVAL=1;COUNT=5\nTRANSP:OPAQUE\nSUMMARY:Review Internet-Draft\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@example.com\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nEND:VEVENT\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:1\nDTSTAMP:20090603T183823Z\nRECURRENCE-ID;TZID=America/Montreal:20090602T150000\nDTSTART;TZID=America/Montreal:20090602T150000\nDTEND;TZID=America/Montreal:20090602T160000\nTRANSP:TRANSPARENT\nSUMMARY:Review Internet-Draft\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@example.com\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=DECLINED;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: bernard@example.net\nto: cyrus@example.com\nsummary: rsvp ACCEPTED\nsummary.attendee: Participants([ItipParticipant { email: \"cyrus@example.com\", name: Some(\"Cyrus Daboo\"), is_organizer: true }, ItipParticipant { email: \"bernard@example.net\", name: Some(\"Bernard Desruisseaux\"), is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 1243882800, tz_id: 167 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Daily, until: None, count: Some(5), interval: Some(1), bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"Review Internet-Draft\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Review Internet-Draft\nDTEND;TZID=America/Montreal:20090601T160000\nDTSTART;TZID=America/Montreal:20090601T150000\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nREQUEST-STATUS:2.0;Success\nEND:VEVENT\nBEGIN:VEVENT\nSUMMARY:Review Internet-Draft\nDTEND;TZID=America/Montreal:20090602T160000\nDTSTART;TZID=America/Montreal:20090602T150000\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=DECLINED;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nRECURRENCE-ID;TZID=America/Montreal:20090602T150000\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nREQUEST-STATUS:2.0;Success\nEND:VEVENT\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n\n# Send iTIP message to organizer\n> send\n\n# Organizer receives the response\n> get cyrus@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Review Internet-Draft\nDTEND;TZID=America/Montreal:20090601T160000\nDTSTART;TZID=America/Montreal:20090601T150000\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;RSVP=\n TRUE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nRRULE:FREQ=DAILY;COUNT=5;INTERVAL=1\nDTSTAMP:1\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VEVENT\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=DECLINED;ROLE=\n REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nRECURRENCE-ID;TZID=America/Montreal:20090602T150000\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n\n# Participant removes an instance of the recurring event\n> put bernard@example.net 9263504FD3AD\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nRRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nRRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nEND:VTIMEZONE\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:1\nDTSTAMP:20090602T185254Z\nDTSTART;TZID=America/Montreal:20090601T150000\nDTEND;TZID=America/Montreal:20090601T160000\nRRULE:FREQ=DAILY;INTERVAL=1;COUNT=5\nEXDATE;TZID=America/Montreal:20090603T150000\nTRANSP:OPAQUE\nSUMMARY:Review Internet-Draft\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@example.com\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nEND:VEVENT\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:1\nDTSTAMP:20090603T183823Z\nRECURRENCE-ID;TZID=America/Montreal:20090602T150000\nDTSTART;TZID=America/Montreal:20090602T150000\nDTEND;TZID=America/Montreal:20090602T160000\nTRANSP:TRANSPARENT\nSUMMARY:Review Internet-Draft\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@example.com\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=DECLINED;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: bernard@example.net\nto: cyrus@example.com\nsummary: rsvp DECLINED\nsummary.attendee: Participants([ItipParticipant { email: \"cyrus@example.com\", name: Some(\"Cyrus Daboo\"), is_organizer: true }, ItipParticipant { email: \"bernard@example.net\", name: Some(\"Bernard Desruisseaux\"), is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 1243882800, tz_id: 167 })\nsummary.rrule: Rrule(ICalendarRecurrenceRule { freq: Daily, until: None, count: Some(5), interval: Some(1), bysecond: [], byminute: [], byhour: [], byday: [], bymonthday: [], byyearday: [], byweekno: [], bymonth: [], bysetpos: [], wkst: None, rscale: None, skip: None })\nsummary.summary: Text(\"Review Internet-Draft\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Review Internet-Draft\nDTEND;TZID=America/Montreal:20090601T160000\nDTSTART;TZID=America/Montreal:20090601T150000\nATTENDEE;PARTSTAT=DECLINED:mailto:bernard@example.net\nORGANIZER:mailto:cyrus@example.com\nRECURRENCE-ID;TZID=America/Montreal:20090603T150000\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n\n# Send iTIP message to organizer\n> send\n\n# Organizer receives the response\n> get cyrus@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VEVENT\nATTENDEE;PARTSTAT=DECLINED:mailto:bernard@example.net\nORGANIZER:mailto:cyrus@example.com\nRECURRENCE-ID;TZID=America/Montreal:20090603T150000\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VEVENT\nSUMMARY:Review Internet-Draft\nDTEND;TZID=America/Montreal:20090601T160000\nDTSTART;TZID=America/Montreal:20090601T150000\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;RSVP=\n TRUE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nRRULE:FREQ=DAILY;COUNT=5;INTERVAL=1\nDTSTAMP:1\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VEVENT\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=DECLINED;ROLE=\n REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nRECURRENCE-ID;TZID=America/Montreal:20090602T150000\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nBEGIN:VTIMEZONE\nTZID:America/Montreal\nBEGIN:STANDARD\nDTSTART:20071104T020000\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:20070311T020000\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nRRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n\n\n\n\n"
  },
  {
    "path": "tests/resources/itip/rfc6638_single.txt",
    "content": "# RFC6638 - Simple Event Scheduling\n\n# Organizer Inviting Multiple Attendees\n> put cyrus@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:0\nDTSTAMP:20090602T185254Z\nDTSTART:20090602T160000Z\nDTEND:20090602T170000Z\nTRANSP:OPAQUE\nSUMMARY:Lunch\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@example.com\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:wilfredo@example.com\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nATTENDEE;CN=\"Mike Douglass\";CUTYPE=INDIVIDUAL;RSVP=TRUE:mailto:mike@example.org\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: cyrus@example.com\nto: bernard@example.net, mike@example.org, wilfredo@example.com\nsummary: invite\nsummary.attendee: Participants([ItipParticipant { email: \"cyrus@example.com\", name: Some(\"Cyrus Daboo\"), is_organizer: true }, ItipParticipant { email: \"bernard@example.net\", name: Some(\"Bernard Desruisseaux\"), is_organizer: false }, ItipParticipant { email: \"mike@example.org\", name: Some(\"Mike Douglass\"), is_organizer: false }, ItipParticipant { email: \"wilfredo@example.com\", name: Some(\"Wilfredo Sanchez Vega\"), is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 1243958400, tz_id: 32768 })\nsummary.summary: Text(\"Lunch\")\nBEGIN:VCALENDAR\nMETHOD:REQUEST\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Lunch\nDTEND:20090602T170000Z\nDTSTART:20090602T160000Z\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nATTENDEE;CN=\"Mike Douglass\";CUTYPE=INDIVIDUAL;RSVP=TRUE;PARTSTAT=NEEDS-ACTI\n ON:mailto:mike@example.org\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:wilfredo@example.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Make sure the sequence number is updated\n> get cyrus@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Lunch\nDTEND:20090602T170000Z\nDTSTART:20090602T160000Z\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nATTENDEE;CN=\"Mike Douglass\";CUTYPE=INDIVIDUAL;RSVP=TRUE:mailto:mike@example.\n org\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:wilfredo@example.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nDTSTAMP:1\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Send iTIP message to attendees\n> send\n\n# Make sure the message was received by the attendees\n> get wilfredo@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Lunch\nDTEND:20090602T170000Z\nDTSTART:20090602T160000Z\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nATTENDEE;CN=\"Mike Douglass\";CUTYPE=INDIVIDUAL;RSVP=TRUE;PARTSTAT=NEEDS-ACTI\n ON:mailto:mike@example.org\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:wilfredo@example.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n# Wilfredo accepts the invitation\n> put wilfredo@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:0\nDTSTAMP:20090602T185254Z\nDTSTART:20090602T160000Z\nDTEND:20090602T170000Z\nTRANSP:OPAQUE\nSUMMARY:Lunch\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@example.com\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:wilfredo@example.com\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nATTENDEE;CN=\"Mike Douglass\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:mike@example.org\nBEGIN:VALARM\nTRIGGER:-PT15M\nACTION:DISPLAY\nDESCRIPTION:Reminder\nEND:VALARM\nEND:VEVENT\nEND:VCALENDAR\n\n# Make sure the sequence number is not updated\n> get wilfredo@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Lunch\nDTEND:20090602T170000Z\nDTSTART:20090602T160000Z\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nATTENDEE;CN=\"Mike Douglass\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;RSVP=TR\n UE:mailto:mike@example.org\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED;ROLE=\n REQ-PARTICIPANT;RSVP=TRUE:mailto:wilfredo@example.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nDTSTAMP:1\nSEQUENCE:0\nBEGIN:VALARM\nDESCRIPTION:Reminder\nACTION:DISPLAY\nTRIGGER:-PT15M\nEND:VALARM\nEND:VEVENT\nEND:VCALENDAR\n\n> expect\nfrom: wilfredo@example.com\nto: cyrus@example.com\nsummary: rsvp ACCEPTED\nsummary.attendee: Participants([ItipParticipant { email: \"cyrus@example.com\", name: Some(\"Cyrus Daboo\"), is_organizer: true }, ItipParticipant { email: \"bernard@example.net\", name: Some(\"Bernard Desruisseaux\"), is_organizer: false }, ItipParticipant { email: \"mike@example.org\", name: Some(\"Mike Douglass\"), is_organizer: false }, ItipParticipant { email: \"wilfredo@example.com\", name: Some(\"Wilfredo Sanchez Vega\"), is_organizer: false }])\nsummary.dtstart: Time(ItipTime { start: 1243958400, tz_id: 32768 })\nsummary.summary: Text(\"Lunch\")\nBEGIN:VCALENDAR\nMETHOD:REPLY\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Lunch\nDTEND:20090602T170000Z\nDTSTART:20090602T160000Z\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:wilfredo@example.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nDTSTAMP:0\nSEQUENCE:0\nREQUEST-STATUS:2.0;Success\nEND:VEVENT\nEND:VCALENDAR\n\n# Send ITIP message to organizer\n> send\n\n# Make sure the message was received by the organizer\n> get cyrus@example.com 9263504FD3AD\nBEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VEVENT\nSUMMARY:Lunch\nDTEND:20090602T170000Z\nDTSTART:20090602T160000Z\nTRANSP:OPAQUE\nATTENDEE;CN=\"Bernard Desruisseaux\";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;\n ROLE=REQ-PARTICIPANT;RSVP=TRUE:mailto:bernard@example.net\nATTENDEE;CN=\"Cyrus Daboo\";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cyrus@e\n xample.com\nATTENDEE;CN=\"Mike Douglass\";CUTYPE=INDIVIDUAL;RSVP=TRUE:mailto:mike@example.\n org\nATTENDEE;CN=\"Wilfredo Sanchez Vega\";CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;\n RSVP=TRUE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:wilfredo@example.com\nORGANIZER;CN=\"Cyrus Daboo\":mailto:cyrus@example.com\nUID:9263504FD3AD\nDTSTAMP:1\nSEQUENCE:1\nEND:VEVENT\nEND:VCALENDAR\n\n\n"
  },
  {
    "path": "tests/resources/jmap/email_get/headers.eml",
    "content": "From: Art Vandelay <art@vandelay.com> (Vandelay Industries)\nTo: \"  James Smythe\" <james@example.com>, Friends:\n      jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?=\n      <john@example.com>;\nCc: List 1: addr1@test.com, addr2@test.com; List 2: addr3@test.com, \n       addr4@test.com; addr5@test.com, addr6@test.com\nCc: =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: addr1@test.com, \n       addr2@test.com; =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: \n       addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\nBcc: Greg Vaudreuil <gvaudre@NRI.Reston.VA.US>, Ned Freed\n        <ned@innosoft.com>, Keith Moore <moore@cs.utk.edu>\nBcc: ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se\nDate: Tue, 1 Jul 2003 10:52:37 +0200\nResent-Date: Tue, 2 Jul 2005 11:52:37 +0200\nResent-Date: Tue, 3 Jul 2005 12:52:37 +0300\nResent-Date: Tue, 4 Jul 2005 13:52:37 +0400\nMessage-ID: <5678.21-Nov-1997@example.com>\nReferences: <1234@local.machine.example>\n            <3456@example.net>\nReferences: <789@local.machine.example>\n            <abcd@example.net>\nKeywords: multipart, alternative, example\nList-Post: <mailto:moderator@host.com> (Postings are Moderated)\nList-Subscribe: (Use this command to join the list)\n       <mailto:list-manager@host.com?body=subscribe%20list>\nList-Subscribe: <ftp://ftp.host.com/list.txt> (FTP),  \n               <mailto:list@host.com?subject=subscribe>\nList-Owner: <http://www.host.com/list.cgi?cmd=sub&lst=list>,\n       <mailto:list-manager@host.com?body=subscribe%20list>\nList-Unsubscribe: (Use this command to get off the list)\n         <mailto:list-manager@host.com?body=unsubscribe%20list>\nSubject: Why not both importing AND exporting? =?utf-8?b?4pi6?=\nX-Address-Single: =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>\nX-Address: Mary Smith <mary@example.net>\nX-Address: John Doe <jdoe@machine.example>\nX-AddressList-Single: Mary Smith <mary@x.test>, jdoe@example.org, Who? <one@y.test>\nX-AddressList: =?US-ASCII*EN?Q?Keith_Moore?= <moore@cs.utk.edu>, \n                John =?US-ASCII*EN?Q?Doe?= <moore@cs.utk.edu>\nX-AddressList: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>,\n                =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>\nX-AddressesGroup-Single: A Group(Some people)\n            :Chris Jones <c@(Chris's host.)public.example>,\n            joe@example.org,     John <jdoe@one.test> (my dear\n            friend); (the end of the group)\nX-AddressesGroup: A Group:Ed Jones <c@a.test>,joe@where.test,John <jdoe@one.test>;\nX-AddressesGroup: \"List 1\": addr1@test.com, addr2@test.com; \"List 2\": \n            addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\nX-List-Single: <mailto:x-moderator@host.com> (X-Postings are Moderated)\nX-List: <http://www.mylist.com/list>,\n       <mailto:list@mylist.com>\nX-List: <http://www.mylist2.com/list2>,\n       <mailto:list2@mylist2.com>\nX-Text-Single: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=\n              =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=\nX-Text: =?iso-8859-1?q?this=20is=20some=20text?=\nX-Text: =?ISO-8859-1?Q?a?= =?ISO-8859-2?Q?_b?=\nX-Date-Single: Tue, 5 Jul 2006 13:52:37 -0500\nX-Date: Sat, 20 Nov 2021 14:22:01 -0800\nX-Date: Sun, 21 Nov 2021 15:23:02 -0900\nX-Id-Single: <myid@example.com> <myid2@example.com>\nX-Id: <myid3@example.com>\nX-Id: <myid4@example.com> <myid5@example.com>\nContent-Type: multipart/mixed; boundary=\"festivus\";\n\n--festivus\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: base64\nX-Custom-Header: 123\n\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\n--festivus\nContent-Type: message/rfc822\nX-Custom-Header-2: 345\n\nFrom: \"Cosmo Kramer\" <kramer@kramerica.com>\nSubject: Exporting my book about coffee tables\nContent-Type: multipart/mixed; boundary=\"giddyup\";\n\n--giddyup\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n--giddyup\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n--giddyup--\n--festivus--\n"
  },
  {
    "path": "tests/resources/jmap/email_get/headers.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 4535,\n  \"receivedAt\": \"2113-09-16T10:13:20Z\",\n  \"messageId\": [\n    \"5678.21-Nov-1997@example.com\"\n  ],\n  \"references\": [\n    \"789@local.machine.example\",\n    \"abcd@example.net\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Art Vandelay (Vandelay Industries)\",\n      \"email\": \"art@vandelay.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"  James Smythe\",\n      \"email\": \"james@example.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"jane@example.com\"\n    },\n    {\n      \"name\": \"John Smîth\",\n      \"email\": \"john@example.com\"\n    }\n  ],\n  \"cc\": [\n    {\n      \"name\": null,\n      \"email\": \"addr1@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr2@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr3@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr4@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr5@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr6@test.com\"\n    }\n  ],\n  \"bcc\": [\n    {\n      \"name\": null,\n      \"email\": \"ietf-822@dimacs.rutgers.edu\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"ojarnef@admin.kth.se\"\n    }\n  ],\n  \"subject\": \"Why not both importing AND exporting? ☺\",\n  \"sentAt\": \"2003-07-01T08:52:37Z\",\n  \"bodyStructure\": {\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"size\": 175,\n        \"type\": \"text/html\",\n        \"charset\": \"us-ascii\"\n      },\n      {\n        \"partId\": \"2\",\n        \"size\": 723,\n        \"type\": \"message/rfc822\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"<html><p>I was thinking about quitting the &ldquo;exporting&rdquo; to focus just on the &ldquo;im...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"size\": 175,\n      \"type\": \"text/html\",\n      \"charset\": \"us-ascii\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"1\",\n      \"size\": 175,\n      \"type\": \"text/html\",\n      \"charset\": \"us-ascii\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"2\",\n      \"size\": 723,\n      \"type\": \"message/rfc822\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"I was thinking about quitting the “exporting” to focus just on the “importing”,\\nbut then I thought, why not do both? ☺\\n\",\n  \"header:Bcc\": \" ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se\",\n  \"header:Bcc:all\": [\n    \" Greg Vaudreuil <gvaudre@NRI.Reston.VA.US>, Ned Freed\\n        <ned@innosoft.com>, Keith Moore <moore@cs.utk.edu>\",\n    \" ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se\"\n  ],\n  \"header:Bcc:asAddresses\": [\n    {\n      \"name\": null,\n      \"email\": \"ietf-822@dimacs.rutgers.edu\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"ojarnef@admin.kth.se\"\n    }\n  ],\n  \"header:Bcc:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Greg Vaudreuil\",\n        \"email\": \"gvaudre@NRI.Reston.VA.US\"\n      },\n      {\n        \"name\": \"Ned Freed\",\n        \"email\": \"ned@innosoft.com\"\n      },\n      {\n        \"name\": \"Keith Moore\",\n        \"email\": \"moore@cs.utk.edu\"\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"email\": \"ietf-822@dimacs.rutgers.edu\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"ojarnef@admin.kth.se\"\n      }\n    ]\n  ],\n  \"header:Bcc:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"ietf-822@dimacs.rutgers.edu\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"ojarnef@admin.kth.se\"\n        }\n      ]\n    }\n  ],\n  \"header:Bcc:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Greg Vaudreuil\",\n            \"email\": \"gvaudre@NRI.Reston.VA.US\"\n          },\n          {\n            \"name\": \"Ned Freed\",\n            \"email\": \"ned@innosoft.com\"\n          },\n          {\n            \"name\": \"Keith Moore\",\n            \"email\": \"moore@cs.utk.edu\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"ietf-822@dimacs.rutgers.edu\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"ojarnef@admin.kth.se\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:Cc\": \" =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: addr1@test.com, \\n       addr2@test.com; =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: \\n       addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\",\n  \"header:Cc:all\": [\n    \" List 1: addr1@test.com, addr2@test.com; List 2: addr3@test.com, \\n       addr4@test.com; addr5@test.com, addr6@test.com\",\n    \" =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: addr1@test.com, \\n       addr2@test.com; =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: \\n       addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\"\n  ],\n  \"header:Cc:asAddresses\": [\n    {\n      \"name\": null,\n      \"email\": \"addr1@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr2@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr3@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr4@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr5@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr6@test.com\"\n    }\n  ],\n  \"header:Cc:asAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"email\": \"addr1@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr2@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr3@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr4@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr5@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr6@test.com\"\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"email\": \"addr1@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr2@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr3@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr4@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr5@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr6@test.com\"\n      }\n    ]\n  ],\n  \"header:Cc:asGroupedAddresses\": [\n    {\n      \"name\": \"Thís ís válíd ÚTF8\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr1@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr2@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Thís ís válíd ÚTF8\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr3@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr4@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr5@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr6@test.com\"\n        }\n      ]\n    }\n  ],\n  \"header:Cc:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": \"List 1\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr1@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr2@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"List 2\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr3@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr4@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr5@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr6@test.com\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": \"Thís ís válíd ÚTF8\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr1@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr2@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"Thís ís válíd ÚTF8\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr3@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr4@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr5@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr6@test.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:Date\": \" Tue, 1 Jul 2003 10:52:37 +0200\",\n  \"header:Date:all\": [\n    \" Tue, 1 Jul 2003 10:52:37 +0200\"\n  ],\n  \"header:Date:asDate\": \"2003-07-01T08:52:37Z\",\n  \"header:Date:asDate:all\": [\n    \"2003-07-01T08:52:37Z\"\n  ],\n  \"header:From\": \" Art Vandelay <art@vandelay.com> (Vandelay Industries)\",\n  \"header:From:all\": [\n    \" Art Vandelay <art@vandelay.com> (Vandelay Industries)\"\n  ],\n  \"header:From:asAddresses\": [\n    {\n      \"name\": \"Art Vandelay (Vandelay Industries)\",\n      \"email\": \"art@vandelay.com\"\n    }\n  ],\n  \"header:From:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Art Vandelay (Vandelay Industries)\",\n        \"email\": \"art@vandelay.com\"\n      }\n    ]\n  ],\n  \"header:From:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"Art Vandelay (Vandelay Industries)\",\n          \"email\": \"art@vandelay.com\"\n        }\n      ]\n    }\n  ],\n  \"header:From:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Art Vandelay (Vandelay Industries)\",\n            \"email\": \"art@vandelay.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:Keywords\": \" multipart, alternative, example\",\n  \"header:Keywords:all\": [\n    \" multipart, alternative, example\"\n  ],\n  \"header:Keywords:asText\": \"multipart, alternative, example\",\n  \"header:Keywords:asText:all\": [\n    \"multipart, alternative, example\"\n  ],\n  \"header:List-Owner\": \" <http://www.host.com/list.cgi?cmd=sub&lst=list>,\\n       <mailto:list-manager@host.com?body=subscribe%20list>\",\n  \"header:List-Owner:all\": [\n    \" <http://www.host.com/list.cgi?cmd=sub&lst=list>,\\n       <mailto:list-manager@host.com?body=subscribe%20list>\"\n  ],\n  \"header:List-Owner:asURLs\": [\n    \"http://www.host.com/list.cgi?cmd=sub&lst=list\",\n    \"mailto:list-manager@host.com?body=subscribe%20list\"\n  ],\n  \"header:List-Owner:asURLs:all\": [\n    [\n      \"http://www.host.com/list.cgi?cmd=sub&lst=list\",\n      \"mailto:list-manager@host.com?body=subscribe%20list\"\n    ]\n  ],\n  \"header:List-Post\": \" <mailto:moderator@host.com> (Postings are Moderated)\",\n  \"header:List-Post:all\": [\n    \" <mailto:moderator@host.com> (Postings are Moderated)\"\n  ],\n  \"header:List-Post:asURLs\": [\n    \"mailto:moderator@host.com\"\n  ],\n  \"header:List-Post:asURLs:all\": [\n    [\n      \"mailto:moderator@host.com\"\n    ]\n  ],\n  \"header:List-Subscribe\": \" <ftp://ftp.host.com/list.txt> (FTP),  \\n               <mailto:list@host.com?subject=subscribe>\",\n  \"header:List-Subscribe:all\": [\n    \" (Use this command to join the list)\\n       <mailto:list-manager@host.com?body=subscribe%20list>\",\n    \" <ftp://ftp.host.com/list.txt> (FTP),  \\n               <mailto:list@host.com?subject=subscribe>\"\n  ],\n  \"header:List-Subscribe:asURLs\": [\n    \"ftp://ftp.host.com/list.txt\",\n    \"mailto:list@host.com?subject=subscribe\"\n  ],\n  \"header:List-Subscribe:asURLs:all\": [\n    [\n      \"mailto:list-manager@host.com?body=subscribe%20list\"\n    ],\n    [\n      \"ftp://ftp.host.com/list.txt\",\n      \"mailto:list@host.com?subject=subscribe\"\n    ]\n  ],\n  \"header:List-Unsubscribe\": \" (Use this command to get off the list)\\n         <mailto:list-manager@host.com?body=unsubscribe%20list>\",\n  \"header:List-Unsubscribe:all\": [\n    \" (Use this command to get off the list)\\n         <mailto:list-manager@host.com?body=unsubscribe%20list>\"\n  ],\n  \"header:List-Unsubscribe:asURLs\": [\n    \"mailto:list-manager@host.com?body=unsubscribe%20list\"\n  ],\n  \"header:List-Unsubscribe:asURLs:all\": [\n    [\n      \"mailto:list-manager@host.com?body=unsubscribe%20list\"\n    ]\n  ],\n  \"header:Message-ID\": \" <5678.21-Nov-1997@example.com>\",\n  \"header:Message-ID:all\": [\n    \" <5678.21-Nov-1997@example.com>\"\n  ],\n  \"header:Message-ID:asMessageIds\": [\n    \"5678.21-Nov-1997@example.com\"\n  ],\n  \"header:Message-ID:asMessageIds:all\": [\n    [\n      \"5678.21-Nov-1997@example.com\"\n    ]\n  ],\n  \"header:References\": \" <789@local.machine.example>\\n            <abcd@example.net>\",\n  \"header:References:all\": [\n    \" <1234@local.machine.example>\\n            <3456@example.net>\",\n    \" <789@local.machine.example>\\n            <abcd@example.net>\"\n  ],\n  \"header:References:asMessageIds\": [\n    \"789@local.machine.example\",\n    \"abcd@example.net\"\n  ],\n  \"header:References:asMessageIds:all\": [\n    [\n      \"1234@local.machine.example\",\n      \"3456@example.net\"\n    ],\n    [\n      \"789@local.machine.example\",\n      \"abcd@example.net\"\n    ]\n  ],\n  \"header:Resent-Date\": \" Tue, 4 Jul 2005 13:52:37 +0400\",\n  \"header:Resent-Date:all\": [\n    \" Tue, 2 Jul 2005 11:52:37 +0200\",\n    \" Tue, 3 Jul 2005 12:52:37 +0300\",\n    \" Tue, 4 Jul 2005 13:52:37 +0400\"\n  ],\n  \"header:Resent-Date:asDate\": \"2005-07-04T09:52:37Z\",\n  \"header:Resent-Date:asDate:all\": [\n    \"2005-07-02T09:52:37Z\",\n    \"2005-07-03T09:52:37Z\",\n    \"2005-07-04T09:52:37Z\"\n  ],\n  \"header:Subject\": \" Why not both importing AND exporting? =?utf-8?b?4pi6?=\",\n  \"header:Subject:all\": [\n    \" Why not both importing AND exporting? =?utf-8?b?4pi6?=\"\n  ],\n  \"header:Subject:asText\": \"Why not both importing AND exporting? ☺\",\n  \"header:Subject:asText:all\": [\n    \"Why not both importing AND exporting? ☺\"\n  ],\n  \"header:To\": \" \\\"  James Smythe\\\" <james@example.com>, Friends:\\n      jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?=\\n      <john@example.com>;\",\n  \"header:To:all\": [\n    \" \\\"  James Smythe\\\" <james@example.com>, Friends:\\n      jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?=\\n      <john@example.com>;\"\n  ],\n  \"header:To:asAddresses\": [\n    {\n      \"name\": \"  James Smythe\",\n      \"email\": \"james@example.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"jane@example.com\"\n    },\n    {\n      \"name\": \"John Smîth\",\n      \"email\": \"john@example.com\"\n    }\n  ],\n  \"header:To:asAddresses:all\": [\n    [\n      {\n        \"name\": \"  James Smythe\",\n        \"email\": \"james@example.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"jane@example.com\"\n      },\n      {\n        \"name\": \"John Smîth\",\n        \"email\": \"john@example.com\"\n      }\n    ]\n  ],\n  \"header:To:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"  James Smythe\",\n          \"email\": \"james@example.com\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Friends\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"jane@example.com\"\n        },\n        {\n          \"name\": \"John Smîth\",\n          \"email\": \"john@example.com\"\n        }\n      ]\n    }\n  ],\n  \"header:To:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"  James Smythe\",\n            \"email\": \"james@example.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"Friends\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"jane@example.com\"\n          },\n          {\n            \"name\": \"John Smîth\",\n            \"email\": \"john@example.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-Address\": \" John Doe <jdoe@machine.example>\",\n  \"header:X-Address:all\": [\n    \" Mary Smith <mary@example.net>\",\n    \" John Doe <jdoe@machine.example>\"\n  ],\n  \"header:X-Address:asAddresses\": [\n    {\n      \"name\": \"John Doe\",\n      \"email\": \"jdoe@machine.example\"\n    }\n  ],\n  \"header:X-Address:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Mary Smith\",\n        \"email\": \"mary@example.net\"\n      }\n    ],\n    [\n      {\n        \"name\": \"John Doe\",\n        \"email\": \"jdoe@machine.example\"\n      }\n    ]\n  ],\n  \"header:X-Address:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"John Doe\",\n          \"email\": \"jdoe@machine.example\"\n        }\n      ]\n    }\n  ],\n  \"header:X-Address:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Mary Smith\",\n            \"email\": \"mary@example.net\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@machine.example\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-Address-Single\": \" =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>\",\n  \"header:X-Address-Single:all\": [\n    \" =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>\"\n  ],\n  \"header:X-Address-Single:asAddresses\": [\n    {\n      \"name\": \"André Pirard\",\n      \"email\": \"PIRARD@vm1.ulg.ac.be\"\n    }\n  ],\n  \"header:X-Address-Single:asAddresses:all\": [\n    [\n      {\n        \"name\": \"André Pirard\",\n        \"email\": \"PIRARD@vm1.ulg.ac.be\"\n      }\n    ]\n  ],\n  \"header:X-Address-Single:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"André Pirard\",\n          \"email\": \"PIRARD@vm1.ulg.ac.be\"\n        }\n      ]\n    }\n  ],\n  \"header:X-Address-Single:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"André Pirard\",\n            \"email\": \"PIRARD@vm1.ulg.ac.be\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressList\": \" =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>,\\n                =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>\",\n  \"header:X-AddressList:all\": [\n    \" =?US-ASCII*EN?Q?Keith_Moore?= <moore@cs.utk.edu>, \\n                John =?US-ASCII*EN?Q?Doe?= <moore@cs.utk.edu>\",\n    \" =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>,\\n                =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>\"\n  ],\n  \"header:X-AddressList:asAddresses\": [\n    {\n      \"name\": \"Keld Jørn Simonsen\",\n      \"email\": \"keld@dkuug.dk\"\n    },\n    {\n      \"name\": \"Olle Järnefors\",\n      \"email\": \"ojarnef@admin.kth.se\"\n    }\n  ],\n  \"header:X-AddressList:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Keith Moore\",\n        \"email\": \"moore@cs.utk.edu\"\n      },\n      {\n        \"name\": \"John Doe\",\n        \"email\": \"moore@cs.utk.edu\"\n      }\n    ],\n    [\n      {\n        \"name\": \"Keld Jørn Simonsen\",\n        \"email\": \"keld@dkuug.dk\"\n      },\n      {\n        \"name\": \"Olle Järnefors\",\n        \"email\": \"ojarnef@admin.kth.se\"\n      }\n    ]\n  ],\n  \"header:X-AddressList:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"Keld Jørn Simonsen\",\n          \"email\": \"keld@dkuug.dk\"\n        },\n        {\n          \"name\": \"Olle Järnefors\",\n          \"email\": \"ojarnef@admin.kth.se\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressList:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Keith Moore\",\n            \"email\": \"moore@cs.utk.edu\"\n          },\n          {\n            \"name\": \"John Doe\",\n            \"email\": \"moore@cs.utk.edu\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Keld Jørn Simonsen\",\n            \"email\": \"keld@dkuug.dk\"\n          },\n          {\n            \"name\": \"Olle Järnefors\",\n            \"email\": \"ojarnef@admin.kth.se\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressList-Single\": \" Mary Smith <mary@x.test>, jdoe@example.org, Who? <one@y.test>\",\n  \"header:X-AddressList-Single:all\": [\n    \" Mary Smith <mary@x.test>, jdoe@example.org, Who? <one@y.test>\"\n  ],\n  \"header:X-AddressList-Single:asAddresses\": [\n    {\n      \"name\": \"Mary Smith\",\n      \"email\": \"mary@x.test\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"jdoe@example.org\"\n    },\n    {\n      \"name\": \"Who?\",\n      \"email\": \"one@y.test\"\n    }\n  ],\n  \"header:X-AddressList-Single:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Mary Smith\",\n        \"email\": \"mary@x.test\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"jdoe@example.org\"\n      },\n      {\n        \"name\": \"Who?\",\n        \"email\": \"one@y.test\"\n      }\n    ]\n  ],\n  \"header:X-AddressList-Single:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"Mary Smith\",\n          \"email\": \"mary@x.test\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"jdoe@example.org\"\n        },\n        {\n          \"name\": \"Who?\",\n          \"email\": \"one@y.test\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressList-Single:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Mary Smith\",\n            \"email\": \"mary@x.test\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"jdoe@example.org\"\n          },\n          {\n            \"name\": \"Who?\",\n            \"email\": \"one@y.test\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup\": \" \\\"List 1\\\": addr1@test.com, addr2@test.com; \\\"List 2\\\": \\n            addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\",\n  \"header:X-AddressesGroup:all\": [\n    \" A Group:Ed Jones <c@a.test>,joe@where.test,John <jdoe@one.test>;\",\n    \" \\\"List 1\\\": addr1@test.com, addr2@test.com; \\\"List 2\\\": \\n            addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\"\n  ],\n  \"header:X-AddressesGroup:asAddresses\": [\n    {\n      \"name\": null,\n      \"email\": \"addr1@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr2@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr3@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr4@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr5@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr6@test.com\"\n    }\n  ],\n  \"header:X-AddressesGroup:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Ed Jones\",\n        \"email\": \"c@a.test\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"joe@where.test\"\n      },\n      {\n        \"name\": \"John\",\n        \"email\": \"jdoe@one.test\"\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"email\": \"addr1@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr2@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr3@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr4@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr5@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr6@test.com\"\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup:asGroupedAddresses\": [\n    {\n      \"name\": \"List 1\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr1@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr2@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": \"List 2\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr3@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr4@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr5@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr6@test.com\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressesGroup:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": \"A Group\",\n        \"addresses\": [\n          {\n            \"name\": \"Ed Jones\",\n            \"email\": \"c@a.test\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"joe@where.test\"\n          },\n          {\n            \"name\": \"John\",\n            \"email\": \"jdoe@one.test\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": \"List 1\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr1@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr2@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"List 2\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr3@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr4@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr5@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr6@test.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup-Single\": \" A Group(Some people)\\n            :Chris Jones <c@(Chris's host.)public.example>,\\n            joe@example.org,     John <jdoe@one.test> (my dear\\n            friend); (the end of the group)\",\n  \"header:X-AddressesGroup-Single:all\": [\n    \" A Group(Some people)\\n            :Chris Jones <c@(Chris's host.)public.example>,\\n            joe@example.org,     John <jdoe@one.test> (my dear\\n            friend); (the end of the group)\"\n  ],\n  \"header:X-AddressesGroup-Single:asAddresses\": [\n    {\n      \"name\": \"Chris Jones (Chris's host.)\",\n      \"email\": \"c@public.example\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"joe@example.org\"\n    },\n    {\n      \"name\": \"John (my dear friend)\",\n      \"email\": \"jdoe@one.test\"\n    },\n    {\n      \"name\": \"the end of the group\",\n      \"email\": \"\"\n    }\n  ],\n  \"header:X-AddressesGroup-Single:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Chris Jones (Chris's host.)\",\n        \"email\": \"c@public.example\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"joe@example.org\"\n      },\n      {\n        \"name\": \"John (my dear friend)\",\n        \"email\": \"jdoe@one.test\"\n      },\n      {\n        \"name\": \"the end of the group\",\n        \"email\": \"\"\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup-Single:asGroupedAddresses\": [\n    {\n      \"name\": \"A Group (Some people)\",\n      \"addresses\": [\n        {\n          \"name\": \"Chris Jones (Chris's host.)\",\n          \"email\": \"c@public.example\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"joe@example.org\"\n        },\n        {\n          \"name\": \"John (my dear friend)\",\n          \"email\": \"jdoe@one.test\"\n        }\n      ]\n    },\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"the end of the group\",\n          \"email\": \"\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressesGroup-Single:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": \"A Group (Some people)\",\n        \"addresses\": [\n          {\n            \"name\": \"Chris Jones (Chris's host.)\",\n            \"email\": \"c@public.example\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"joe@example.org\"\n          },\n          {\n            \"name\": \"John (my dear friend)\",\n            \"email\": \"jdoe@one.test\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"the end of the group\",\n            \"email\": \"\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-Date\": \" Sun, 21 Nov 2021 15:23:02 -0900\",\n  \"header:X-Date:all\": [\n    \" Sat, 20 Nov 2021 14:22:01 -0800\",\n    \" Sun, 21 Nov 2021 15:23:02 -0900\"\n  ],\n  \"header:X-Date:asDate\": \"2021-11-22T00:23:02Z\",\n  \"header:X-Date:asDate:all\": [\n    \"2021-11-20T22:22:01Z\",\n    \"2021-11-22T00:23:02Z\"\n  ],\n  \"header:X-Date-Single\": \" Tue, 5 Jul 2006 13:52:37 -0500\",\n  \"header:X-Date-Single:all\": [\n    \" Tue, 5 Jul 2006 13:52:37 -0500\"\n  ],\n  \"header:X-Date-Single:asDate\": \"2006-07-05T18:52:37Z\",\n  \"header:X-Date-Single:asDate:all\": [\n    \"2006-07-05T18:52:37Z\"\n  ],\n  \"header:X-Id\": \" <myid4@example.com> <myid5@example.com>\",\n  \"header:X-Id:all\": [\n    \" <myid3@example.com>\",\n    \" <myid4@example.com> <myid5@example.com>\"\n  ],\n  \"header:X-Id:asMessageIds\": [\n    \"myid4@example.com\",\n    \"myid5@example.com\"\n  ],\n  \"header:X-Id:asMessageIds:all\": [\n    [\n      \"myid3@example.com\"\n    ],\n    [\n      \"myid4@example.com\",\n      \"myid5@example.com\"\n    ]\n  ],\n  \"header:X-Id-Single\": \" <myid@example.com> <myid2@example.com>\",\n  \"header:X-Id-Single:all\": [\n    \" <myid@example.com> <myid2@example.com>\"\n  ],\n  \"header:X-Id-Single:asMessageIds\": [\n    \"myid@example.com\",\n    \"myid2@example.com\"\n  ],\n  \"header:X-Id-Single:asMessageIds:all\": [\n    [\n      \"myid@example.com\",\n      \"myid2@example.com\"\n    ]\n  ],\n  \"header:X-List\": \" <http://www.mylist2.com/list2>,\\n       <mailto:list2@mylist2.com>\",\n  \"header:X-List:all\": [\n    \" <http://www.mylist.com/list>,\\n       <mailto:list@mylist.com>\",\n    \" <http://www.mylist2.com/list2>,\\n       <mailto:list2@mylist2.com>\"\n  ],\n  \"header:X-List:asURLs\": [\n    \"http://www.mylist2.com/list2\",\n    \"mailto:list2@mylist2.com\"\n  ],\n  \"header:X-List:asURLs:all\": [\n    [\n      \"http://www.mylist.com/list\",\n      \"mailto:list@mylist.com\"\n    ],\n    [\n      \"http://www.mylist2.com/list2\",\n      \"mailto:list2@mylist2.com\"\n    ]\n  ],\n  \"header:X-List-Single\": \" <mailto:x-moderator@host.com> (X-Postings are Moderated)\",\n  \"header:X-List-Single:all\": [\n    \" <mailto:x-moderator@host.com> (X-Postings are Moderated)\"\n  ],\n  \"header:X-List-Single:asURLs\": [\n    \"mailto:x-moderator@host.com\"\n  ],\n  \"header:X-List-Single:asURLs:all\": [\n    [\n      \"mailto:x-moderator@host.com\"\n    ]\n  ],\n  \"header:X-Text\": \" =?ISO-8859-1?Q?a?= =?ISO-8859-2?Q?_b?=\",\n  \"header:X-Text:all\": [\n    \" =?iso-8859-1?q?this=20is=20some=20text?=\",\n    \" =?ISO-8859-1?Q?a?= =?ISO-8859-2?Q?_b?=\"\n  ],\n  \"header:X-Text:asText\": \"a b\",\n  \"header:X-Text:asText:all\": [\n    \"this is some text\",\n    \"a b\"\n  ],\n  \"header:X-Text-Single\": \" =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=\\n              =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=\",\n  \"header:X-Text-Single:all\": [\n    \" =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=\\n              =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=\"\n  ],\n  \"header:X-Text-Single:asText\": \"If you can read this you understand the example.\",\n  \"header:X-Text-Single:asText:all\": [\n    \"If you can read this you understand the example.\"\n  ]\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/message_attachment.eml",
    "content": "From:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\nSubject: [Fwd: Map of Argentina with Description]\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\nThis is a multi-part message in MIME format.\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\nContent-Disposition: inline\n\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\n--D7F------------D7FD5A0B8AB9C65CCDBFA872--\n"
  },
  {
    "path": "tests/resources/jmap/email_get/message_attachment.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 2651,\n  \"receivedAt\": \"2054-01-02T20:53:20Z\",\n  \"from\": [\n    {\n      \"name\": \"Al Gore\",\n      \"email\": \"vice-president@whitehouse.gov\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"White House Transportation Coordinator\",\n      \"email\": \"transport@whitehouse.gov\"\n    }\n  ],\n  \"subject\": \"[Fwd: Map of Argentina with Description]\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"From\",\n        \"value\": \"  Al Gore <vice-president@whitehouse.gov>\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \"  White House Transportation Coordinator\\n     <transport@whitehouse.gov>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" [Fwd: Map of Argentina with Description]\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed;\\n              boundary=\\\"D7F------------D7FD5A0B8AB9C65CCDBFA872\\\"\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 61,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain; charset=us-ascii\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" 7bit\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\"\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 1979,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" message/rfc822\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" 7bit\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline\"\n          }\n        ],\n        \"type\": \"message/rfc822\",\n        \"disposition\": \"inline\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"Fred,\\n\\nFire up Air Force One!  We're going South!\\n\\nThanks,\\nAl\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 61,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=us-ascii\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 61,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=us-ascii\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 1979,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" message/rfc822\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"message/rfc822\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"Fred,\\n\\nFire up Air Force One!  We're going South!\\n\\nThanks,\\nAl\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_alternative.eml",
    "content": "From: sender@example.com\nTo: recipient@example.com\nSubject: Multipart Email Example\nContent-Type: multipart/alternative; boundary=\"boundary-string\"\n\n--boundary-string\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\nPlain text email goes here!\nThis is the fallback if email client does not support HTML\n\n--boundary-string\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\n<h1>This is the HTML Section!</h1>\n<p>This is what displays in most modern email clients</p>\n\n--boundary-string--\n"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_alternative.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 616,\n  \"receivedAt\": \"1989-07-09T15:06:40Z\",\n  \"from\": [\n    {\n      \"name\": null,\n      \"email\": \"sender@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": null,\n      \"email\": \"recipient@example.com\"\n    }\n  ],\n  \"subject\": \"Multipart Email Example\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"From\",\n        \"value\": \" sender@example.com\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" recipient@example.com\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" Multipart Email Example\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/alternative; boundary=\\\"boundary-string\\\"\"\n      }\n    ],\n    \"type\": \"multipart/alternative\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 87,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" quoted-printable\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"utf-8\",\n        \"disposition\": \"inline\"\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 93,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/html; charset=\\\"utf-8\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" quoted-printable\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline\"\n          }\n        ],\n        \"type\": \"text/html\",\n        \"charset\": \"utf-8\",\n        \"disposition\": \"inline\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"Plain text email goes here!\\nThis is the fallback if email client does not support HTML\\n\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"2\": {\n      \"value\": \"<h1>This is the HTML Section!</h1>\\n<p>This is what displays in most modern email clients</p>\\n\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 87,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"utf-8\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 93,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"utf-8\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"attachments\": [],\n  \"hasAttachment\": false,\n  \"preview\": \"Plain text email goes here!\\nThis is the fallback if email client does not support HTML\\n\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_cid.eml",
    "content": "From: \"Doug Sauder\" <doug@example.com>\nTo: \"Joe Blow\" <jblow@example.com>\nSubject: Test message from Microsoft Outlook 00\nDate: Wed, 17 May 2000 19:44:45 -0400\nMessage-ID: <NDBBIAKOPKHFGPLCODIGKEKFCHAA.doug@example.com>\nMIME-Version: 1.0\nContent-Type: multipart/related;\n\tboundary=\"----=_NextPart_000_000C_01BFC038.5A5C8E60\"\nX-Priority: 3 (Normal)\nX-MSMail-Priority: Normal\nX-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\nImportance: Normal\nX-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300\n\nThis is a multi-part message in MIME format.\n\n------=_NextPart_000_000C_01BFC038.5A5C8E60\nContent-Type: multipart/alternative;\n\tboundary=\"----=_NextPart_001_000D_01BFC038.5A5C8E60\"\n\n\n------=_NextPart_001_000D_01BFC038.5A5C8E60\nContent-Type: text/plain;\n\tcharset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\n\nThe Hare and the Tortoise=20\n=20\nA HARE one day ridiculed the short feet and slow pace of the Tortoise, =\nwho replied, laughing:  \"Though you be swift as the wind, I will beat =\nyou in a race.\"  The Hare, believing her assertion to be simply =\nimpossible, assented to the proposal; and they agreed that the Fox =\nshould choose the course and fix the goal.  On the day appointed for the =\nrace the two started together.  The Tortoise never for a moment stopped, =\nbut went on with a slow but steady pace straight to the end of the =\ncourse.  The Hare, lying down by the wayside, fell fast asleep.  At last =\nwaking up, and moving as fast as he could, he saw the Tortoise had =\nreached the goal, and was comfortably dozing after her fatigue. =20\n\n\n=20\nSlow but steady wins the race. =20\n\n\n\n\n------=_NextPart_001_000D_01BFC038.5A5C8E60\nContent-Type: text/html;\n\tcharset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\n<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\">\n<HTML><HEAD>\n<META content=3D\"text/html; charset=3Diso-8859-1\" =\nhttp-equiv=3DContent-Type>\n<META content=3D\"MSHTML 5.00.2314.1000\" name=3DGENERATOR></HEAD>\n<BODY>\n<DIV><FONT face=3DArial size=3D2><BR>The Hare and the Tortoise =\n<BR>&nbsp;<BR>A HARE=20\none day ridiculed the short feet and slow pace of the Tortoise, who =\nreplied,=20\nlaughing:&nbsp; \"Though you be swift as the wind, I will beat you in a=20\nrace.\"&nbsp; The Hare, believing her assertion to be simply impossible, =\nassented=20\nto the proposal; and they agreed that the Fox should choose the course =\nand fix=20\nthe goal.&nbsp; On the day appointed for the race the two started=20\ntogether.&nbsp; The Tortoise never for a moment stopped, but went on =\nwith a slow=20\nbut steady pace straight to the end of the course.&nbsp; The Hare, lying =\ndown by=20\nthe wayside, fell fast asleep.&nbsp; At last waking up, and moving as =\nfast as he=20\ncould, he saw the Tortoise had reached the goal, and was comfortably =\ndozing=20\nafter her fatigue.&nbsp;&nbsp;</FONT></DIV>\n<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>\n<DIV><FONT face=3DArial size=3D2><IMG align=3Dbaseline alt=3D\"blue ball\" =\nborder=3D0=20\nhspace=3D0 src=3D\"cid:823504223@17052000-0f8d\"><BR>&nbsp;<BR>Slow but =\nsteady wins=20\nthe race.&nbsp; </FONT></DIV>\n<DIV>&nbsp;</DIV>\n<DIV><FONT face=3DArial size=3D2><IMG align=3Dbaseline alt=3D\"red ball\" =\nborder=3D0=20\nhspace=3D0 =\nsrc=3D\"cid:823504223@17052000-0f94\"><BR></DIV></FONT></BODY></HTML>\n\n------=_NextPart_001_000D_01BFC038.5A5C8E60--\n\n------=_NextPart_000_000C_01BFC038.5A5C8E60\nContent-Type: image/png;\n\tname=\"blueball.png\"\nContent-Transfer-Encoding: base64\nContent-ID: <823504223@17052000-0f8d>\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAgAABAAABgAAAAA\nCCkAEEIAEEoACDEAEFIIIXMIKXsIKYQIIWsAGFoACDkIIWMQOZwYQqUYQq0YQrUQOaUQMZQAGFIQ\nMYwpUrU5Y8Y5Y84pWs4YSs4YQs4YQr1Ca8Z7nNacvd6Mtd5jlOcxa94hUt4YStYYQsYQMaUAACHO\n5+/n7++cxu9ShO8pWucQOa1Ke86tzt6lzu9ajO8QMZxahNat1ufO7++Mve9Ke+8YOaUYSsaMvee1\n5++Uve8AAClajOdzpe9rnO8IKYwxY+8pWu8IIXsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAADBMg1VAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAGISURBVHicddJtV5swGAbgEk6AJhBSk4bMCUynBSLaqovbrG/bfPn/vyh70lbssceb\nL5xznTsh5BmNhgQoRChwo50EOIohUYLDj4zHhKYQkrEoQdvock4ne0IKMVUpKZLQDeqSTIsv+18P\nyqqWUw2IBsRM7307PPp+fDJrWtnpLDJvewYxnewfnvanZ+fzpmwXijC8KbqEa3Fx2ff91Y95U9XC\nUpaDeQwiMpHXP/v+1++bWVPWQoGFawtjury9vru/f/C1Vi7ezT0WWpQHf/7+u/G71aLThK/MjRxm\nT6KdzZ9fGk9yatMsTgZLl3XVgFRAC6spj/13enssqJVtWVa3NdBSacL8+VZmYqKmdd1CSYoOiMOS\nGwtzlqqlFFIuOqv0a1ZEZrUkWICLLFW266y1KvWE1zV/iDAH1EopnVLCiygZCIomH3NCKX0lnI+B\n1iuuzCGTxwXjnDO4d7NpbX42YJJHkBwmAm2TxwAZg40J3+Xtbv1rgOAZwG0NxW62p+lT+Yi747sD\n/wEUVMzYmWkOvwAAACV0RVh0Q29tbWVudABjbGlwMmdpZiB2LjAuNiBieSBZdmVzIFBpZ3VldDZz\nO7wAAAAASUVORK5CYII=\n\n------=_NextPart_000_000C_01BFC038.5A5C8E60\nContent-Type: image/png;\n\tname=\"redball.png\"\nContent-Transfer-Encoding: base64\nContent-ID: <823504223@17052000-0f94>\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAABAAALAAAVAAAa\nAAAXAAARAAAKAAADAAAcAAAyAABEAABNAABIAAA9AAAjAAAWAAAmAABhAAB7AACGAACHAAB9AAB0\nAABgAAA5AAAUAAAGAAAnAABLAABvAACQAAClAAC7AAC/AACrAAChAACMAABzAABbAAAuAAAIAABM\nAAB3AACZAAC0GRnKODjVPT3bKSndBQW4AACoAAB5AAAxAAAYAAAEAABFAACaAAC7JCTRYWHfhITm\nf3/mVlbqHx/SAAC5AACjAABdAABCAAAoAAAJAABnAAC6Dw/QVFTek5PlrKzpmZntZWXvJSXXAADB\nAACxAACcAABtAABTAAA2AAAbAAAFAABKAACBAADLICDdZ2fonJzrpqbtiorvUVHvFBTRAADDAAC2\nAAB4AABeAABAAAAiAABXAACSAADCAADaGxvoVVXseHjveHjvV1fvJibhAADOAAC3AACnAACVAABH\nAAArAAAPAACdAADFAADhBQXrKCjvPDzvNTXvGxvjAADQAADJAAC1AACXAACEAABsAABPAAASAAAC\nAABiAADpAADvAgLnAADYAADLAAC6AACwAABwAAATAAAkAABYAADIAADTAADNAACzAACDAABuAAAe\nAAB+AADAAACkAACNAAB/AABpAABQAAAwAACRAACpAAC8AACqAACbAABlAABJAAAqAAAOAAA0AACs\nAACvAACtAACmAACJAAB6AABrAABaAAA+AAApAABqAACCAACfAACeAACWAACPAAB8AAAZAAAHAABV\nAACOAACKAAA4AAAQAAA/AAByAACAAABcAAA3AAAsAABmAABDAABWAAAgAAAzAAA8AAA6AAAfAAAM\nAAAdAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAD8LtlFAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAIISURBVHicY2CAg/8QwIABmJhZWFnZ2Dk4MaU5uLh5eHn5+LkFBDlQJf8zC/EIi4iK\niUtI8koJScsgyf5nlpWTV1BUUlZRVVPX4NFk1UJIyghp6+jq6RsYGhmbKJgK85mZW8Dk/rNaSlhZ\n29ja2Ts4Ojkr6Li4urFDNf53N/Ow8vTy9vH18w8IDAoWDQkNC4+ASP5ni4wKio6JjYtPSExKTnFW\nSE1LF4A69n9GZlZ2Tm5efkFhUXFySWlZlEd5RSVY7j+TkGRVdU1tXX1DY1Ozcktpa1t7h2YnOAj+\nd7l1tyo79vT29SdNSJ44SbFVdHIo9xSIHNPUaWqTpifNSJrZnK00S0U1a/acUG5piNz/uXLzVJ2q\nm6dXz584S2WB1cJFi5cshZr539xVftnyFKUVTi2TVjqvyhJLXb1m7TqoHPt6F/HW0g0bN63crGqV\ntWXrtu07BJihcsw71+zanRW8Z89eq337RQ/Ip60xO3gIElX/LbikDm8T36KwbNmRo7O3zpHkPSZw\nHBqL//8flz1x2OOkyKJTi7aqbzutfUZI2gIuF8F2lr/D5dw2+fZdwpl8YVOlI+CJ4/9/joOyYed5\nQzMvhGqnm2V0WiClm///D0lfXHtJ6vLlK9w7rx7vQk5SQJbFtSms1y9evXid7QZacgOxmSxktNzd\ntSwwU+J/VICaCPFIYU3XAJhIOtjf5sfyAAAAJXRFWHRDb21tZW50AGNsaXAyZ2lmIHYuMC42IGJ5\nIFl2ZXMgUGlndWV0NnM7vAAAAABJRU5ErkJggg==\n\n------=_NextPart_000_000C_01BFC038.5A5C8E60--\n"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_cid.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 7477,\n  \"receivedAt\": \"2206-12-09T08:26:40Z\",\n  \"messageId\": [\n    \"NDBBIAKOPKHFGPLCODIGKEKFCHAA.doug@example.com\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Doug Sauder\",\n      \"email\": \"doug@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"Joe Blow\",\n      \"email\": \"jblow@example.com\"\n    }\n  ],\n  \"subject\": \"Test message from Microsoft Outlook 00\",\n  \"sentAt\": \"2000-05-17T23:44:45Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Doug Sauder\\\" <doug@example.com>\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" \\\"Joe Blow\\\" <jblow@example.com>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" Test message from Microsoft Outlook 00\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Wed, 17 May 2000 19:44:45 -0400\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <NDBBIAKOPKHFGPLCODIGKEKFCHAA.doug@example.com>\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/related;\\n\\tboundary=\\\"----=_NextPart_000_000C_01BFC038.5A5C8E60\\\"\"\n      },\n      {\n        \"name\": \"X-Priority\",\n        \"value\": \" 3 (Normal)\"\n      },\n      {\n        \"name\": \"X-MSMail-Priority\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-Mailer\",\n        \"value\": \" Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\"\n      },\n      {\n        \"name\": \"Importance\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-MimeOLE\",\n        \"value\": \" Produced By Microsoft MimeOLE V5.00.2314.1300\"\n      }\n    ],\n    \"type\": \"multipart/related\",\n    \"subParts\": [\n      {\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" multipart/alternative;\\n\\tboundary=\\\"----=_NextPart_001_000D_01BFC038.5A5C8E60\\\"\"\n          }\n        ],\n        \"type\": \"multipart/alternative\",\n        \"subParts\": [\n          {\n            \"partId\": \"2\",\n            \"blobId\": \"blob_0\",\n            \"size\": 761,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" text/plain;\\n\\tcharset=\\\"iso-8859-1\\\"\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" quoted-printable\"\n              }\n            ],\n            \"type\": \"text/plain\",\n            \"charset\": \"iso-8859-1\"\n          },\n          {\n            \"partId\": \"3\",\n            \"blobId\": \"blob_1\",\n            \"size\": 1442,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" text/html;\\n\\tcharset=\\\"iso-8859-1\\\"\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" quoted-printable\"\n              }\n            ],\n            \"type\": \"text/html\",\n            \"charset\": \"iso-8859-1\"\n          }\n        ]\n      },\n      {\n        \"partId\": \"4\",\n        \"blobId\": \"blob_2\",\n        \"size\": 1325,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/png;\\n\\tname=\\\"blueball.png\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          },\n          {\n            \"name\": \"Content-ID\",\n            \"value\": \" <823504223@17052000-0f8d>\"\n          }\n        ],\n        \"name\": \"blueball.png\",\n        \"type\": \"image/png\",\n        \"cid\": \"823504223@17052000-0f8d\"\n      },\n      {\n        \"partId\": \"5\",\n        \"blobId\": \"blob_3\",\n        \"size\": 1453,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/png;\\n\\tname=\\\"redball.png\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          },\n          {\n            \"name\": \"Content-ID\",\n            \"value\": \" <823504223@17052000-0f94>\"\n          }\n        ],\n        \"name\": \"redball.png\",\n        \"type\": \"image/png\",\n        \"cid\": \"823504223@17052000-0f94\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"2\": {\n      \"value\": \"\\nThe Hare and the Tortoise \\n \\nA HARE one day ridiculed the short feet and slow pace of the Tortoi...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    },\n    \"3\": {\n      \"value\": \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 4.0 Transitional//EN\\\">\\n<HTML><HEAD>\\n...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_0\",\n      \"size\": 761,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain;\\n\\tcharset=\\\"iso-8859-1\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"iso-8859-1\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"3\",\n      \"blobId\": \"blob_1\",\n      \"size\": 1442,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html;\\n\\tcharset=\\\"iso-8859-1\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"iso-8859-1\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"4\",\n      \"blobId\": \"blob_2\",\n      \"size\": 1325,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png;\\n\\tname=\\\"blueball.png\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-ID\",\n          \"value\": \" <823504223@17052000-0f8d>\"\n        }\n      ],\n      \"name\": \"blueball.png\",\n      \"type\": \"image/png\",\n      \"cid\": \"823504223@17052000-0f8d\"\n    },\n    {\n      \"partId\": \"5\",\n      \"blobId\": \"blob_3\",\n      \"size\": 1453,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png;\\n\\tname=\\\"redball.png\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-ID\",\n          \"value\": \" <823504223@17052000-0f94>\"\n        }\n      ],\n      \"name\": \"redball.png\",\n      \"type\": \"image/png\",\n      \"cid\": \"823504223@17052000-0f94\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"\\nThe Hare and the Tortoise \\n \\nA HARE one day ridiculed the short feet and slow pace of the Tortoise, who replied, laughing:  \\\"Though you be swift as the wind, I will beat you in a race.\\\"  The Hare, believing her assertion to be simply impossible, assent...\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_mixed.eml",
    "content": "MIME-Version: 1.0\nFrom: Nathaniel Borenstein <nsb@nsb.fv.com>\nTo: Ned Freed <ned@innosoft.com>\nDate: Fri, 07 Oct 1994 16:15:05 -0700 (PDT)\nSubject: A multipart example\nContent-Type: multipart/mixed;\n            boundary=unique-boundary-1\n\nThis is the preamble area of a multipart message.\nMail readers that understand multipart format\nshould ignore this preamble.\n\nIf you are reading this text, you might want to\nconsider changing to a mail reader that understands\nhow to properly display multipart messages.\n\n--unique-boundary-1\n\n... Some text appears here ...\n\n[Note that the blank between the boundary and the start\nof the text in this part means no header fields were\ngiven and this is text in the US-ASCII character set.\nIt could have been done with explicit typing as in the\nnext part.]\n\n--unique-boundary-1\nContent-type: text/plain; charset=US-ASCII\n\nThis could have been part of the previous part, but\nillustrates explicit versus implicit typing of body\nparts.\n\n--unique-boundary-1\nContent-Type: multipart/parallel; boundary=unique-boundary-2\n\n--unique-boundary-2\nContent-Type: audio/basic\nContent-Transfer-Encoding: base64\n\n... base64-encoded 8000 Hz single-channel\n    mu-law-format audio data goes here ...\n\n--unique-boundary-2\nContent-Type: image/jpeg\nContent-Transfer-Encoding: base64\n\n... base64-encoded image data goes here ...\n\n--unique-boundary-2--\n\n--unique-boundary-1\nContent-type: text/enriched\n\nThis is <bold><italic>enriched.</italic></bold>\n<smaller>as defined in RFC 1896</smaller>\n\nIsn't it\n<bigger><bigger>cool?</bigger></bigger>\n\n--unique-boundary-1\nContent-Type: message/rfc822\n\nFrom: (mailbox in US-ASCII)\nTo: (address in US-ASCII)\nSubject: (subject in US-ASCII)\nContent-Type: Text/plain; charset=ISO-8859-1\nContent-Transfer-Encoding: Quoted-printable\n\n... Additional text in ISO-8859-1 goes here ...\n\n--unique-boundary-1--\n"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_mixed.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 1853,\n  \"receivedAt\": \"2028-09-19T18:13:20Z\",\n  \"from\": [\n    {\n      \"name\": \"Nathaniel Borenstein\",\n      \"email\": \"nsb@nsb.fv.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"Ned Freed\",\n      \"email\": \"ned@innosoft.com\"\n    }\n  ],\n  \"subject\": \"A multipart example\",\n  \"sentAt\": \"1994-10-07T23:15:05Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" Nathaniel Borenstein <nsb@nsb.fv.com>\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" Ned Freed <ned@innosoft.com>\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Fri, 07 Oct 1994 16:15:05 -0700 (PDT)\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" A multipart example\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed;\\n            boundary=unique-boundary-1\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 262,\n        \"headers\": [],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\"\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 111,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain; charset=US-ASCII\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"US-ASCII\"\n      },\n      {\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" multipart/parallel; boundary=unique-boundary-2\"\n          }\n        ],\n        \"type\": \"multipart/parallel\",\n        \"subParts\": [\n          {\n            \"partId\": \"4\",\n            \"blobId\": \"blob_2\",\n            \"size\": 85,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" audio/basic\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" base64\"\n              }\n            ],\n            \"type\": \"audio/basic\",\n            \"charset\": \"us-ascii\"\n          },\n          {\n            \"partId\": \"5\",\n            \"blobId\": \"blob_3\",\n            \"size\": 44,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" image/jpeg\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" base64\"\n              }\n            ],\n            \"type\": \"image/jpeg\",\n            \"charset\": \"us-ascii\"\n          }\n        ]\n      },\n      {\n        \"partId\": \"6\",\n        \"blobId\": \"blob_4\",\n        \"size\": 140,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/enriched\"\n          }\n        ],\n        \"type\": \"text/enriched\",\n        \"charset\": \"us-ascii\"\n      },\n      {\n        \"partId\": \"7\",\n        \"blobId\": \"blob_5\",\n        \"size\": 223,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" message/rfc822\"\n          }\n        ],\n        \"type\": \"message/rfc822\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"... Some text appears here ...\\n\\n[Note that the blank between the boundary and the start\\nof the te...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    },\n    \"2\": {\n      \"value\": \"This could have been part of the previous part, but\\nillustrates explicit versus implicit typing o...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 262,\n      \"headers\": [],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 111,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=US-ASCII\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"US-ASCII\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 262,\n      \"headers\": [],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 111,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=US-ASCII\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"US-ASCII\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"4\",\n      \"blobId\": \"blob_2\",\n      \"size\": 85,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" audio/basic\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"audio/basic\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"5\",\n      \"blobId\": \"blob_3\",\n      \"size\": 44,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/jpeg\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"6\",\n      \"blobId\": \"blob_4\",\n      \"size\": 140,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/enriched\"\n        }\n      ],\n      \"type\": \"text/enriched\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"7\",\n      \"blobId\": \"blob_5\",\n      \"size\": 223,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" message/rfc822\"\n        }\n      ],\n      \"type\": \"message/rfc822\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"... Some text appears here ...\\n\\n[Note that the blank between the boundary and the start\\nof the text in this part means no header fields were\\ngiven and this is text in the US-ASCII character set.\\nIt could have been done with explicit typing as in the\\nnex...\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_related.eml",
    "content": "Message-Id: <4.2.0.58.20000519003556.00a918e0@pop.example.com>\nX-Sender: dwsauder@pop.example.com (Unverified)\nX-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.58 \nX-Priority: 2 (High)\nDate: Fri, 19 May 2000 00:36:58 -0400\nTo: Heinz =?iso-8859-1?Q?Muller?= <mueller@example.com>\nFrom: Doug Sauder <dwsauder@example.com>\nSubject: =?iso-8859-1?Q?Die_Hasen_und_die_Frosche?=\nMime-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"=====================_715392540==_\"\n\n--=====================_715392540==_\nContent-Type: multipart/related;\n\ttype=\"multipart/alternative\";\n\tboundary=\"=====================_715392540==_.REL\"\n\n--=====================_715392540==_.REL\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_715392550==_.ALT\"\n\n--=====================_715392550==_.ALT\nContent-Type: text/plain; charset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\nDie Hasen und die Fr=F6sche\n\nDie Hasen klagten einst =FCber ihre mi=DFliche Lage; \"wir leben\", sprach ein=\n Redner, \"in steter Furcht vor Menschen und Tieren, eine Beute der Hunde,=\n der Adler, ja fast aller Raubtiere! Unsere stete Angst ist =E4rger als der=\n Tod selbst. Auf, la=DFt uns ein f=FCr allemal sterben.\"=20\n\nIn einem nahen Teich wollten sie sich nun ers=E4ufen; sie eilten ihm zu;=\n allein das au=DFerordentliche Get=F6se und ihre wunderbare Gestalt=\n erschreckte eine Menge Fr=F6sche, die am Ufer sa=DFen, so sehr, da=DF sie=\n aufs schnellste untertauchten.=20\n\n\"Halt\", rief nun eben dieser Sprecher, \"wir wollen das Ers=E4ufen noch ein=\n wenig aufschieben, denn auch uns f=FCrchten, wie ihr seht, einige Tiere,=\n welche also wohl noch ungl=FCcklicher sein m=FCssen als wir.\"=20\n\n2aa3ed95.png2aa3edd1.png\n--=====================_715392550==_.ALT\nContent-Type: text/html; charset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\n<html>\n<font face=3D\"Arial, Helvetica\" size=3D5 color=3D\"#0000FF\"><b>Die Hasen und =\ndie\nFr=F6sche<br>\n<br>\n</font></b><font face=3D\"Arial, Helvetica\">Die Hasen klagten einst =FCber\nihre mi=DFliche Lage; &quot;wir leben&quot;, sprach ein Redner, &quot;in\nsteter Furcht vor Menschen und Tieren, eine Beute der Hunde, der Adler,\nja fast aller Raubtiere! Unsere stete Angst ist =E4rger als der Tod selbst.\nAuf, la=DFt uns ein f=FCr allemal sterben.&quot; <br>\n<br>\nIn einem nahen Teich wollten sie sich nun ers=E4ufen; sie eilten ihm zu;\nallein das au=DFerordentliche Get=F6se und ihre wunderbare Gestalt\nerschreckte eine Menge Fr=F6sche, die am Ufer sa=DFen, so sehr, da=DF sie au=\nfs\nschnellste untertauchten. <br>\n<br>\n&quot;Halt&quot;, rief nun eben dieser Sprecher, &quot;wir wollen das\nErs=E4ufen noch ein wenig aufschieben, denn auch uns f=FCrchten, wie ihr\nseht, einige Tiere, welche also wohl noch ungl=FCcklicher sein m=FCssen als\nwir.&quot; <br>\n<br>\n</font><img src=3D\"cid:4.2.0.58.20000519003556.00a918e0@pop.example.com.2\"=\n width=3D27 height=3D27 alt=3D\"2aa3ed95.png\"><img=\n src=3D\"cid:4.2.0.58.20000519003556.00a918e0@pop.example.com.3\" width=3D27=\n height=3D27 alt=3D\"2aa3edd1.png\"></html>\n\n--=====================_715392550==_.ALT--\n\n--=====================_715392540==_.REL\nContent-Type: image/png; name=\"2aa3ed95.png\"\nContent-ID: <4.2.0.58.20000519003556.00a918e0@pop.example.com.2>\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; filename=\"2aa3ed95.png\"\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAgAABAAABgAAAAA\nCCkAEEIAEEoACDEAEFIIIXMIKXsIKYQIIWsAGFoACDkIIWMQOZwYQqUYQq0YQrUQOaUQMZQAGFIQ\nMYwpUrU5Y8Y5Y84pWs4YSs4YQs4YQr1Ca8Z7nNacvd6Mtd5jlOcxa94hUt4YStYYQsYQMaUAACHO\n5+/n7++cxu9ShO8pWucQOa1Ke86tzt6lzu9ajO8QMZxahNat1ufO7++Mve9Ke+8YOaUYSsaMvee1\n5++Uve8AAClajOdzpe9rnO8IKYwxY+8pWu8IIXsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAADBMg1VAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAGISURBVHicddJtV5swGAbgEk6AJhBSk4bMCUynBSLaqovbrG/bfPn/vyh70lbssceb\nL5xznTsh5BmNhgQoRChwo50EOIohUYLDj4zHhKYQkrEoQdvock4ne0IKMVUpKZLQDeqSTIsv+18P\nyqqWUw2IBsRM7307PPp+fDJrWtnpLDJvewYxnewfnvanZ+fzpmwXijC8KbqEa3Fx2ff91Y95U9XC\nUpaDeQwiMpHXP/v+1++bWVPWQoGFawtjury9vru/f/C1Vi7ezT0WWpQHf/7+u/G71aLThK/MjRxm\nT6KdzZ9fGk9yatMsTgZLl3XVgFRAC6spj/13enssqJVtWVa3NdBSacL8+VZmYqKmdd1CSYoOiMOS\nGwtzlqqlFFIuOqv0a1ZEZrUkWICLLFW266y1KvWE1zV/iDAH1EopnVLCiygZCIomH3NCKX0lnI+B\n1iuuzCGTxwXjnDO4d7NpbX42YJJHkBwmAm2TxwAZg40J3+Xtbv1rgOAZwG0NxW62p+lT+Yi747sD\n/wEUVMzYmWkOvwAAACV0RVh0Q29tbWVudABjbGlwMmdpZiB2LjAuNiBieSBZdmVzIFBpZ3VldDZz\nO7wAAAAASUVORK5CYII=\n--=====================_715392540==_.REL\nContent-Type: image/png; name=\"2aa3edd1.png\"\nContent-ID: <4.2.0.58.20000519003556.00a918e0@pop.example.com.3>\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; filename=\"2aa3edd1.png\"\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAABAAALAAAVAAAa\nAAAXAAARAAAKAAADAAAcAAAyAABEAABNAABIAAA9AAAjAAAWAAAmAABhAAB7AACGAACHAAB9AAB0\nAABgAAA5AAAUAAAGAAAnAABLAABvAACQAAClAAC7AAC/AACrAAChAACMAABzAABbAAAuAAAIAABM\nAAB3AACZAAC0GRnKODjVPT3bKSndBQW4AACoAAB5AAAxAAAYAAAEAABFAACaAAC7JCTRYWHfhITm\nf3/mVlbqHx/SAAC5AACjAABdAABCAAAoAAAJAABnAAC6Dw/QVFTek5PlrKzpmZntZWXvJSXXAADB\nAACxAACcAABtAABTAAA2AAAbAAAFAABKAACBAADLICDdZ2fonJzrpqbtiorvUVHvFBTRAADDAAC2\nAAB4AABeAABAAAAiAABXAACSAADCAADaGxvoVVXseHjveHjvV1fvJibhAADOAAC3AACnAACVAABH\nAAArAAAPAACdAADFAADhBQXrKCjvPDzvNTXvGxvjAADQAADJAAC1AACXAACEAABsAABPAAASAAAC\nAABiAADpAADvAgLnAADYAADLAAC6AACwAABwAAATAAAkAABYAADIAADTAADNAACzAACDAABuAAAe\nAAB+AADAAACkAACNAAB/AABpAABQAAAwAACRAACpAAC8AACqAACbAABlAABJAAAqAAAOAAA0AACs\nAACvAACtAACmAACJAAB6AABrAABaAAA+AAApAABqAACCAACfAACeAACWAACPAAB8AAAZAAAHAABV\nAACOAACKAAA4AAAQAAA/AAByAACAAABcAAA3AAAsAABmAABDAABWAAAgAAAzAAA8AAA6AAAfAAAM\nAAAdAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAD8LtlFAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAIISURBVHicY2CAg/8QwIABmJhZWFnZ2Dk4MaU5uLh5eHn5+LkFBDlQJf8zC/EIi4iK\niUtI8koJScsgyf5nlpWTV1BUUlZRVVPX4NFk1UJIyghp6+jq6RsYGhmbKJgK85mZW8Dk/rNaSlhZ\n29ja2Ts4Ojkr6Li4urFDNf53N/Ow8vTy9vH18w8IDAoWDQkNC4+ASP5ni4wKio6JjYtPSExKTnFW\nSE1LF4A69n9GZlZ2Tm5efkFhUXFySWlZlEd5RSVY7j+TkGRVdU1tXX1DY1Ozcktpa1t7h2YnOAj+\nd7l1tyo79vT29SdNSJ44SbFVdHIo9xSIHNPUaWqTpifNSJrZnK00S0U1a/acUG5piNz/uXLzVJ2q\nm6dXz584S2WB1cJFi5cshZr539xVftnyFKUVTi2TVjqvyhJLXb1m7TqoHPt6F/HW0g0bN63crGqV\ntWXrtu07BJihcsw71+zanRW8Z89eq337RQ/Ip60xO3gIElX/LbikDm8T36KwbNmRo7O3zpHkPSZw\nHBqL//8flz1x2OOkyKJTi7aqbzutfUZI2gIuF8F2lr/D5dw2+fZdwpl8YVOlI+CJ4/9/joOyYed5\nQzMvhGqnm2V0WiClm///D0lfXHtJ6vLlK9w7rx7vQk5SQJbFtSms1y9evXid7QZacgOxmSxktNzd\ntSwwU+J/VICaCPFIYU3XAJhIOtjf5sfyAAAAJXRFWHRDb21tZW50AGNsaXAyZ2lmIHYuMC42IGJ5\nIFl2ZXMgUGlndWV0NnM7vAAAAABJRU5ErkJggg==\n--=====================_715392540==_.REL--\n\n--=====================_715392540==_\nContent-Type: image/png; name=\"blueball.png\"\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment; filename=\"blueball.png\"\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAgAABAAABgAAAAA\nCCkAEEIAEEoACDEAEFIIIXMIKXsIKYQIIWsAGFoACDkIIWMQOZwYQqUYQq0YQrUQOaUQMZQAGFIQ\nMYwpUrU5Y8Y5Y84pWs4YSs4YQs4YQr1Ca8Z7nNacvd6Mtd5jlOcxa94hUt4YStYYQsYQMaUAACHO\n5+/n7++cxu9ShO8pWucQOa1Ke86tzt6lzu9ajO8QMZxahNat1ufO7++Mve9Ke+8YOaUYSsaMvee1\n5++Uve8AAClajOdzpe9rnO8IKYwxY+8pWu8IIXsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAADBMg1VAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAGISURBVHicddJtV5swGAbgEk6AJhBSk4bMCUynBSLaqovbrG/bfPn/vyh70lbssceb\nL5xznTsh5BmNhgQoRChwo50EOIohUYLDj4zHhKYQkrEoQdvock4ne0IKMVUpKZLQDeqSTIsv+18P\nyqqWUw2IBsRM7307PPp+fDJrWtnpLDJvewYxnewfnvanZ+fzpmwXijC8KbqEa3Fx2ff91Y95U9XC\nUpaDeQwiMpHXP/v+1++bWVPWQoGFawtjury9vru/f/C1Vi7ezT0WWpQHf/7+u/G71aLThK/MjRxm\nT6KdzZ9fGk9yatMsTgZLl3XVgFRAC6spj/13enssqJVtWVa3NdBSacL8+VZmYqKmdd1CSYoOiMOS\nGwtzlqqlFFIuOqv0a1ZEZrUkWICLLFW266y1KvWE1zV/iDAH1EopnVLCiygZCIomH3NCKX0lnI+B\n1iuuzCGTxwXjnDO4d7NpbX42YJJHkBwmAm2TxwAZg40J3+Xtbv1rgOAZwG0NxW62p+lT+Yi747sD\n/wEUVMzYmWkOvwAAACV0RVh0Q29tbWVudABjbGlwMmdpZiB2LjAuNiBieSBZdmVzIFBpZ3VldDZz\nO7wAAAAASUVORK5CYII=\n--=====================_715392540==_\nContent-Type: image/png; name=\"greenball.png\"\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment; filename=\"greenball.png\"\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAAAEAAAGAAAIQAA\nCAAAMQAAQgAAUgAAWgAASgAIYwAIcwAIewAQjAAIawAAOQAAYwAQlAAQnAAhpQAQpQAhrQBCvRhj\nxjFjxjlSxiEpzgAYvQAQrQAYrQAhvQCU1mOt1nuE1lJK3hgh1gAYxgAYtQAAKQBCzhDO55Te563G\n55SU52NS5yEh3gAYzgBS3iGc52vW75y974yE71JC7xCt73ul3nNa7ykh5wAY1gAx5wBS7yFr7zlK\n7xgp5wAp7wAx7wAIhAAQtQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAp1fnZAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAFtSURBVHicddJtV8IgFAdwD2zIgMEE1+NcqdsoK+m5tCyz7/+ZiLmHsyzvq53zO/cy\n+N9ery1bVe9PWQA9z4MQ+H8Yoj7GASZ95IHfaBGmLOSchyIgyOu22mgQSjUcDuNYcoGjLiLK1cHh\n0fHJaTKKOcMItgYxT89OzsfjyTTLC8UF0c2ZNmKquJhczq6ub+YmSVUYRF59GeDastu7+9nD41Nm\nkiJ2jc2J3kAWZ9Pr55fH18XSmRuKUTXUaqHy7O19tfr4NFle/w3YDrWRUIlZrL/W86XJkyJVG9Ea\nEjIx2XyZmZJGioeUaL+2AY8TY8omR6nkLKhu70zjUKVJXsp3quS2DVSJWNh3zzJKCyexI0ZxBP3a\nfE0ElyqOlZJyw8r3BE2SFiJCyxA434SCkg65RhdeQBljQtCg39LWrA90RDDG1EWrYUO23hMANUKR\nRl61E529cR++D2G5LK002dr/qrcfu9u0V3bxn/XdhR/NYeeN0ggsLAAAACV0RVh0Q29tbWVudABj\nbGlwMmdpZiB2LjAuNiBieSBZdmVzIFBpZ3VldDZzO7wAAAAASUVORK5CYII=\n--=====================_715392540==_--\n\n"
  },
  {
    "path": "tests/resources/jmap/email_get/multipart_related.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 11304,\n  \"receivedAt\": \"2328-03-18T08:00:00Z\",\n  \"messageId\": [\n    \"4.2.0.58.20000519003556.00a918e0@pop.example.com\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Doug Sauder\",\n      \"email\": \"dwsauder@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"Heinz Muller\",\n      \"email\": \"mueller@example.com\"\n    }\n  ],\n  \"subject\": \"Die Hasen und die Frosche\",\n  \"sentAt\": \"2000-05-19T04:36:58Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <4.2.0.58.20000519003556.00a918e0@pop.example.com>\"\n      },\n      {\n        \"name\": \"X-Sender\",\n        \"value\": \" dwsauder@pop.example.com (Unverified)\"\n      },\n      {\n        \"name\": \"X-Mailer\",\n        \"value\": \" QUALCOMM Windows Eudora Pro Version 4.2.0.58\"\n      },\n      {\n        \"name\": \"X-Priority\",\n        \"value\": \" 2 (High)\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Fri, 19 May 2000 00:36:58 -0400\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" Heinz =?iso-8859-1?Q?Muller?= <mueller@example.com>\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" Doug Sauder <dwsauder@example.com>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" =?iso-8859-1?Q?Die_Hasen_und_die_Frosche?=\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed;\\n\\tboundary=\\\"=====================_715392540==_\\\"\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" multipart/related;\\n\\ttype=\\\"multipart/alternative\\\";\\n\\tboundary=\\\"=====================_715392540==_.REL\\\"\"\n          }\n        ],\n        \"type\": \"multipart/related\",\n        \"subParts\": [\n          {\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" multipart/alternative;\\n\\tboundary=\\\"=====================_715392550==_.ALT\\\"\"\n              }\n            ],\n            \"type\": \"multipart/alternative\",\n            \"subParts\": [\n              {\n                \"partId\": \"3\",\n                \"blobId\": \"blob_0\",\n                \"size\": 779,\n                \"headers\": [\n                  {\n                    \"name\": \"Content-Type\",\n                    \"value\": \" text/plain; charset=\\\"iso-8859-1\\\"\"\n                  },\n                  {\n                    \"name\": \"Content-Transfer-Encoding\",\n                    \"value\": \" quoted-printable\"\n                  }\n                ],\n                \"type\": \"text/plain\",\n                \"charset\": \"iso-8859-1\"\n              },\n              {\n                \"partId\": \"4\",\n                \"blobId\": \"blob_1\",\n                \"size\": 1154,\n                \"headers\": [\n                  {\n                    \"name\": \"Content-Type\",\n                    \"value\": \" text/html; charset=\\\"iso-8859-1\\\"\"\n                  },\n                  {\n                    \"name\": \"Content-Transfer-Encoding\",\n                    \"value\": \" quoted-printable\"\n                  }\n                ],\n                \"type\": \"text/html\",\n                \"charset\": \"iso-8859-1\"\n              }\n            ]\n          },\n          {\n            \"partId\": \"5\",\n            \"blobId\": \"blob_2\",\n            \"size\": 1325,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" image/png; name=\\\"2aa3ed95.png\\\"\"\n              },\n              {\n                \"name\": \"Content-ID\",\n                \"value\": \" <4.2.0.58.20000519003556.00a918e0@pop.example.com.2>\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" base64\"\n              },\n              {\n                \"name\": \"Content-Disposition\",\n                \"value\": \" inline; filename=\\\"2aa3ed95.png\\\"\"\n              }\n            ],\n            \"name\": \"2aa3ed95.png\",\n            \"type\": \"image/png\",\n            \"disposition\": \"inline\",\n            \"cid\": \"4.2.0.58.20000519003556.00a918e0@pop.example.com.2\"\n          },\n          {\n            \"partId\": \"6\",\n            \"blobId\": \"blob_3\",\n            \"size\": 1453,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" image/png; name=\\\"2aa3edd1.png\\\"\"\n              },\n              {\n                \"name\": \"Content-ID\",\n                \"value\": \" <4.2.0.58.20000519003556.00a918e0@pop.example.com.3>\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" base64\"\n              },\n              {\n                \"name\": \"Content-Disposition\",\n                \"value\": \" inline; filename=\\\"2aa3edd1.png\\\"\"\n              }\n            ],\n            \"name\": \"2aa3edd1.png\",\n            \"type\": \"image/png\",\n            \"disposition\": \"inline\",\n            \"cid\": \"4.2.0.58.20000519003556.00a918e0@pop.example.com.3\"\n          }\n        ]\n      },\n      {\n        \"partId\": \"7\",\n        \"blobId\": \"blob_4\",\n        \"size\": 1325,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/png; name=\\\"blueball.png\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" attachment; filename=\\\"blueball.png\\\"\"\n          }\n        ],\n        \"name\": \"blueball.png\",\n        \"type\": \"image/png\",\n        \"disposition\": \"attachment\"\n      },\n      {\n        \"partId\": \"8\",\n        \"blobId\": \"blob_5\",\n        \"size\": 1298,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/png; name=\\\"greenball.png\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" attachment; filename=\\\"greenball.png\\\"\"\n          }\n        ],\n        \"name\": \"greenball.png\",\n        \"type\": \"image/png\",\n        \"disposition\": \"attachment\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"3\": {\n      \"value\": \"Die Hasen und die Frösche\\n\\nDie Hasen klagten einst über ihre mißliche Lage; \\\"wir leben\\\", sprac...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    },\n    \"4\": {\n      \"value\": \"<html>\\n<font face=\\\"Arial, Helvetica\\\" size=5 color=\\\"#0000FF\\\"><b>Die Hasen und die\\nFrösche<br>\\n...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"3\",\n      \"blobId\": \"blob_0\",\n      \"size\": 779,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"iso-8859-1\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"iso-8859-1\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"4\",\n      \"blobId\": \"blob_1\",\n      \"size\": 1154,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html; charset=\\\"iso-8859-1\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"iso-8859-1\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"5\",\n      \"blobId\": \"blob_2\",\n      \"size\": 1325,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png; name=\\\"2aa3ed95.png\\\"\"\n        },\n        {\n          \"name\": \"Content-ID\",\n          \"value\": \" <4.2.0.58.20000519003556.00a918e0@pop.example.com.2>\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline; filename=\\\"2aa3ed95.png\\\"\"\n        }\n      ],\n      \"name\": \"2aa3ed95.png\",\n      \"type\": \"image/png\",\n      \"disposition\": \"inline\",\n      \"cid\": \"4.2.0.58.20000519003556.00a918e0@pop.example.com.2\"\n    },\n    {\n      \"partId\": \"6\",\n      \"blobId\": \"blob_3\",\n      \"size\": 1453,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png; name=\\\"2aa3edd1.png\\\"\"\n        },\n        {\n          \"name\": \"Content-ID\",\n          \"value\": \" <4.2.0.58.20000519003556.00a918e0@pop.example.com.3>\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline; filename=\\\"2aa3edd1.png\\\"\"\n        }\n      ],\n      \"name\": \"2aa3edd1.png\",\n      \"type\": \"image/png\",\n      \"disposition\": \"inline\",\n      \"cid\": \"4.2.0.58.20000519003556.00a918e0@pop.example.com.3\"\n    },\n    {\n      \"partId\": \"7\",\n      \"blobId\": \"blob_4\",\n      \"size\": 1325,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png; name=\\\"blueball.png\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment; filename=\\\"blueball.png\\\"\"\n        }\n      ],\n      \"name\": \"blueball.png\",\n      \"type\": \"image/png\",\n      \"disposition\": \"attachment\"\n    },\n    {\n      \"partId\": \"8\",\n      \"blobId\": \"blob_5\",\n      \"size\": 1298,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png; name=\\\"greenball.png\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment; filename=\\\"greenball.png\\\"\"\n        }\n      ],\n      \"name\": \"greenball.png\",\n      \"type\": \"image/png\",\n      \"disposition\": \"attachment\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"Die Hasen und die Frösche\\n\\nDie Hasen klagten einst über ihre mißliche Lage; \\\"wir leben\\\", sprach ein Redner, \\\"in steter Furcht vor Menschen und Tieren, eine Beute der Hunde, der Adler, ja fast aller Raubtiere! Unsere stete Angst ist ärger als der Tod...\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/rfc8621.eml",
    "content": "Subject: RFC 8621 Section 4.1.4 test\nContent-Type: multipart/mixed; boundary=\"1\"\n\n--1\nContent-Type: text/plain\nContent-Disposition: inline\n\nA\n--1  \nContent-Type: multipart/mixed; boundary=\"2\"\n\n--2\nContent-Type: multipart/alternative; boundary=\"3\"\n\n--3\nContent-Type: multipart/mixed; boundary=\"4\"\n\n--4\nContent-Type: text/plain\nContent-Disposition: inline\n\nB\n--4  \nContent-Type: image/jpeg\nContent-Disposition: inline\n\nC\n--4  \nContent-Type: text/plain\nContent-Disposition: inline\n\nD\n--4--\n\n--3 \nContent-Type: multipart/related; boundary=\"5\"\n\n--5\nContent-Type: text/html\n\n<html>E</html>\n--5  \nContent-Type: image/jpeg\n\nF\n--5--  \n\n--3-- \n\n--2   \nContent-Type: image/jpeg\nContent-Disposition: attachment\n\nG\n--2  \nContent-Type: application/x-excel\n\nH\n--2  \nContent-Type: message/rfc822\n\nSubject: J\n\nJ\n--2--\n\n--1  \nContent-Type: text/plain\nContent-Disposition: inline\n\nK\n--1--\n"
  },
  {
    "path": "tests/resources/jmap/email_get/rfc8621.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 870,\n  \"receivedAt\": \"1997-07-27T10:40:00Z\",\n  \"subject\": \"RFC 8621 Section 4.1.4 test\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"Subject\",\n        \"value\": \" RFC 8621 Section 4.1.4 test\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed; boundary=\\\"1\\\"\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 1,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\",\n        \"disposition\": \"inline\"\n      },\n      {\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" multipart/mixed; boundary=\\\"2\\\"\"\n          }\n        ],\n        \"type\": \"multipart/mixed\",\n        \"subParts\": [\n          {\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" multipart/alternative; boundary=\\\"3\\\"\"\n              }\n            ],\n            \"type\": \"multipart/alternative\",\n            \"subParts\": [\n              {\n                \"headers\": [\n                  {\n                    \"name\": \"Content-Type\",\n                    \"value\": \" multipart/mixed; boundary=\\\"4\\\"\"\n                  }\n                ],\n                \"type\": \"multipart/mixed\",\n                \"subParts\": [\n                  {\n                    \"partId\": \"5\",\n                    \"blobId\": \"blob_1\",\n                    \"size\": 1,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" text/plain\"\n                      },\n                      {\n                        \"name\": \"Content-Disposition\",\n                        \"value\": \" inline\"\n                      }\n                    ],\n                    \"type\": \"text/plain\",\n                    \"charset\": \"us-ascii\",\n                    \"disposition\": \"inline\"\n                  },\n                  {\n                    \"partId\": \"6\",\n                    \"blobId\": \"blob_2\",\n                    \"size\": 1,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" image/jpeg\"\n                      },\n                      {\n                        \"name\": \"Content-Disposition\",\n                        \"value\": \" inline\"\n                      }\n                    ],\n                    \"type\": \"image/jpeg\",\n                    \"disposition\": \"inline\"\n                  },\n                  {\n                    \"partId\": \"7\",\n                    \"blobId\": \"blob_3\",\n                    \"size\": 1,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" text/plain\"\n                      },\n                      {\n                        \"name\": \"Content-Disposition\",\n                        \"value\": \" inline\"\n                      }\n                    ],\n                    \"type\": \"text/plain\",\n                    \"charset\": \"us-ascii\",\n                    \"disposition\": \"inline\"\n                  }\n                ]\n              },\n              {\n                \"headers\": [\n                  {\n                    \"name\": \"Content-Type\",\n                    \"value\": \" multipart/related; boundary=\\\"5\\\"\"\n                  }\n                ],\n                \"type\": \"multipart/related\",\n                \"subParts\": [\n                  {\n                    \"partId\": \"9\",\n                    \"blobId\": \"blob_4\",\n                    \"size\": 14,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" text/html\"\n                      }\n                    ],\n                    \"type\": \"text/html\",\n                    \"charset\": \"us-ascii\"\n                  },\n                  {\n                    \"partId\": \"10\",\n                    \"blobId\": \"blob_5\",\n                    \"size\": 1,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" image/jpeg\"\n                      }\n                    ],\n                    \"type\": \"image/jpeg\"\n                  }\n                ]\n              }\n            ]\n          },\n          {\n            \"partId\": \"11\",\n            \"blobId\": \"blob_6\",\n            \"size\": 1,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" image/jpeg\"\n              },\n              {\n                \"name\": \"Content-Disposition\",\n                \"value\": \" attachment\"\n              }\n            ],\n            \"type\": \"image/jpeg\",\n            \"disposition\": \"attachment\"\n          },\n          {\n            \"partId\": \"12\",\n            \"blobId\": \"blob_7\",\n            \"size\": 1,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" application/x-excel\"\n              }\n            ],\n            \"type\": \"application/x-excel\"\n          },\n          {\n            \"partId\": \"13\",\n            \"blobId\": \"blob_8\",\n            \"size\": 13,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" message/rfc822\"\n              }\n            ],\n            \"type\": \"message/rfc822\"\n          }\n        ]\n      },\n      {\n        \"partId\": \"14\",\n        \"blobId\": \"blob_9\",\n        \"size\": 1,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\",\n        \"disposition\": \"inline\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"A\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"14\": {\n      \"value\": \"K\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"5\": {\n      \"value\": \"B\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"7\": {\n      \"value\": \"D\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"9\": {\n      \"value\": \"<html>E</html>\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"5\",\n      \"blobId\": \"blob_1\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"6\",\n      \"blobId\": \"blob_2\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"image/jpeg\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"7\",\n      \"blobId\": \"blob_3\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"14\",\n      \"blobId\": \"blob_9\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"9\",\n      \"blobId\": \"blob_4\",\n      \"size\": 14,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"14\",\n      \"blobId\": \"blob_9\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"6\",\n      \"blobId\": \"blob_2\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        }\n      ],\n      \"type\": \"image/jpeg\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"10\",\n      \"blobId\": \"blob_5\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        }\n      ],\n      \"type\": \"image/jpeg\"\n    },\n    {\n      \"partId\": \"11\",\n      \"blobId\": \"blob_6\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment\"\n        }\n      ],\n      \"type\": \"image/jpeg\",\n      \"disposition\": \"attachment\"\n    },\n    {\n      \"partId\": \"12\",\n      \"blobId\": \"blob_7\",\n      \"size\": 1,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" application/x-excel\"\n        }\n      ],\n      \"type\": \"application/x-excel\"\n    },\n    {\n      \"partId\": \"13\",\n      \"blobId\": \"blob_8\",\n      \"size\": 13,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" message/rfc822\"\n        }\n      ],\n      \"type\": \"message/rfc822\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"A\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/single_part.eml",
    "content": "From: \"Doug Sauder\" <doug@example.com>\nTo: =?iso-8859-1?B?SvxyZ2VuIFNjaG38cmdlbg==?= <schmuergen@example.com>\nSubject: =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche_=28Microsoft_Outlook_00=29?=\nDate: Wed, 17 May 2000 19:15:35 -0400\nMessage-ID: <NDBBIAKOPKHFGPLCODIGIEKCCHAA.doug@example.com>\nMIME-Version: 1.0\nContent-Type: text/plain;\n\tcharset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\nX-Priority: 3 (Normal)\nX-MSMail-Priority: Normal\nX-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\nImportance: Normal\nX-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300\n\nDie Hasen und die Fr=F6sche\n\nDie Hasen klagten einst =FCber ihre mi=DFliche Lage; \"wir leben\", sprach =\nein Redner, \"in steter Furcht vor Menschen und Tieren, eine Beute der =\nHunde, der Adler, ja fast aller Raubtiere! Unsere stete Angst ist =\n=E4rger als der Tod selbst. Auf, la=DFt uns ein f=FCr allemal sterben.\"=20\n\nIn einem nahen Teich wollten sie sich nun ers=E4ufen; sie eilten ihm zu; =\nallein das au=DFerordentliche Get=F6se und ihre wunderbare Gestalt =\nerschreckte eine Menge Fr=F6sche, die am Ufer sa=DFen, so sehr, da=DF =\nsie aufs schnellste untertauchten.=20\n\n\"Halt\", rief nun eben dieser Sprecher, \"wir wollen das Ers=E4ufen noch =\nein wenig aufschieben, denn auch uns f=FCrchten, wie ihr seht, einige =\nTiere, welche also wohl noch ungl=FCcklicher sein m=FCssen als wir.\"=20\n"
  },
  {
    "path": "tests/resources/jmap/email_get/single_part.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 1378,\n  \"receivedAt\": \"2013-09-01T01:46:40Z\",\n  \"messageId\": [\n    \"NDBBIAKOPKHFGPLCODIGIEKCCHAA.doug@example.com\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Doug Sauder\",\n      \"email\": \"doug@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"Jürgen Schmürgen\",\n      \"email\": \"schmuergen@example.com\"\n    }\n  ],\n  \"subject\": \"Die Hasen und die Frösche (Microsoft Outlook 00)\",\n  \"sentAt\": \"2000-05-17T23:15:35Z\",\n  \"bodyStructure\": {\n    \"partId\": \"0\",\n    \"blobId\": \"blob_0\",\n    \"size\": 754,\n    \"headers\": [\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Doug Sauder\\\" <doug@example.com>\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" =?iso-8859-1?B?SvxyZ2VuIFNjaG38cmdlbg==?= <schmuergen@example.com>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche_=28Microsoft_Outlook_00=29?=\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Wed, 17 May 2000 19:15:35 -0400\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <NDBBIAKOPKHFGPLCODIGIEKCCHAA.doug@example.com>\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" text/plain;\\n\\tcharset=\\\"iso-8859-1\\\"\"\n      },\n      {\n        \"name\": \"Content-Transfer-Encoding\",\n        \"value\": \" quoted-printable\"\n      },\n      {\n        \"name\": \"X-Priority\",\n        \"value\": \" 3 (Normal)\"\n      },\n      {\n        \"name\": \"X-MSMail-Priority\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-Mailer\",\n        \"value\": \" Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\"\n      },\n      {\n        \"name\": \"Importance\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-MimeOLE\",\n        \"value\": \" Produced By Microsoft MimeOLE V5.00.2314.1300\"\n      }\n    ],\n    \"type\": \"text/plain\",\n    \"charset\": \"iso-8859-1\"\n  },\n  \"bodyValues\": {\n    \"0\": {\n      \"value\": \"Die Hasen und die Frösche\\n\\nDie Hasen klagten einst über ihre mißliche Lage; \\\"wir leben\\\", sprac...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"0\",\n      \"blobId\": \"blob_0\",\n      \"size\": 754,\n      \"headers\": [\n        {\n          \"name\": \"From\",\n          \"value\": \" \\\"Doug Sauder\\\" <doug@example.com>\"\n        },\n        {\n          \"name\": \"To\",\n          \"value\": \" =?iso-8859-1?B?SvxyZ2VuIFNjaG38cmdlbg==?= <schmuergen@example.com>\"\n        },\n        {\n          \"name\": \"Subject\",\n          \"value\": \" =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche_=28Microsoft_Outlook_00=29?=\"\n        },\n        {\n          \"name\": \"Date\",\n          \"value\": \" Wed, 17 May 2000 19:15:35 -0400\"\n        },\n        {\n          \"name\": \"Message-ID\",\n          \"value\": \" <NDBBIAKOPKHFGPLCODIGIEKCCHAA.doug@example.com>\"\n        },\n        {\n          \"name\": \"MIME-Version\",\n          \"value\": \" 1.0\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain;\\n\\tcharset=\\\"iso-8859-1\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        },\n        {\n          \"name\": \"X-Priority\",\n          \"value\": \" 3 (Normal)\"\n        },\n        {\n          \"name\": \"X-MSMail-Priority\",\n          \"value\": \" Normal\"\n        },\n        {\n          \"name\": \"X-Mailer\",\n          \"value\": \" Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\"\n        },\n        {\n          \"name\": \"Importance\",\n          \"value\": \" Normal\"\n        },\n        {\n          \"name\": \"X-MimeOLE\",\n          \"value\": \" Produced By Microsoft MimeOLE V5.00.2314.1300\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"iso-8859-1\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"0\",\n      \"blobId\": \"blob_0\",\n      \"size\": 754,\n      \"headers\": [\n        {\n          \"name\": \"From\",\n          \"value\": \" \\\"Doug Sauder\\\" <doug@example.com>\"\n        },\n        {\n          \"name\": \"To\",\n          \"value\": \" =?iso-8859-1?B?SvxyZ2VuIFNjaG38cmdlbg==?= <schmuergen@example.com>\"\n        },\n        {\n          \"name\": \"Subject\",\n          \"value\": \" =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche_=28Microsoft_Outlook_00=29?=\"\n        },\n        {\n          \"name\": \"Date\",\n          \"value\": \" Wed, 17 May 2000 19:15:35 -0400\"\n        },\n        {\n          \"name\": \"Message-ID\",\n          \"value\": \" <NDBBIAKOPKHFGPLCODIGIEKCCHAA.doug@example.com>\"\n        },\n        {\n          \"name\": \"MIME-Version\",\n          \"value\": \" 1.0\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain;\\n\\tcharset=\\\"iso-8859-1\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        },\n        {\n          \"name\": \"X-Priority\",\n          \"value\": \" 3 (Normal)\"\n        },\n        {\n          \"name\": \"X-MSMail-Priority\",\n          \"value\": \" Normal\"\n        },\n        {\n          \"name\": \"X-Mailer\",\n          \"value\": \" Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\"\n        },\n        {\n          \"name\": \"Importance\",\n          \"value\": \" Normal\"\n        },\n        {\n          \"name\": \"X-MimeOLE\",\n          \"value\": \" Produced By Microsoft MimeOLE V5.00.2314.1300\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"iso-8859-1\"\n    }\n  ],\n  \"attachments\": [],\n  \"hasAttachment\": false,\n  \"preview\": \"Die Hasen und die Frösche\\n\\nDie Hasen klagten einst über ihre mißliche Lage; \\\"wir leben\\\", sprach ein Redner, \\\"in steter Furcht vor Menschen und Tieren, eine Beute der Hunde, der Adler, ja fast aller Raubtiere! Unsere stete Angst ist ärger als der Tod...\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/text_body_missing.eml",
    "content": "From: \"Doug Sauder\" <doug@example.com>\nTo: \"Joe Blow\" <jblow@example.com>\nSubject: Test message from Microsoft Outlook 00\nDate: Wed, 17 May 2000 19:35:05 -0400\nMessage-ID: <NDBBIAKOPKHFGPLCODIGKEKECHAA.doug@example.com>\nMIME-Version: 1.0\nContent-Type: image/png;\n\tname=\"redball.png\"\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment;\n\tfilename=\"redball.png\"\nX-Priority: 3 (Normal)\nX-MSMail-Priority: Normal\nX-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\nImportance: Normal\nX-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAABAAALAAAVAAAa\nAAAXAAARAAAKAAADAAAcAAAyAABEAABNAABIAAA9AAAjAAAWAAAmAABhAAB7AACGAACHAAB9AAB0\nAABgAAA5AAAUAAAGAAAnAABLAABvAACQAAClAAC7AAC/AACrAAChAACMAABzAABbAAAuAAAIAABM\nAAB3AACZAAC0GRnKODjVPT3bKSndBQW4AACoAAB5AAAxAAAYAAAEAABFAACaAAC7JCTRYWHfhITm\nf3/mVlbqHx/SAAC5AACjAABdAABCAAAoAAAJAABnAAC6Dw/QVFTek5PlrKzpmZntZWXvJSXXAADB\nAACxAACcAABtAABTAAA2AAAbAAAFAABKAACBAADLICDdZ2fonJzrpqbtiorvUVHvFBTRAADDAAC2\nAAB4AABeAABAAAAiAABXAACSAADCAADaGxvoVVXseHjveHjvV1fvJibhAADOAAC3AACnAACVAABH\nAAArAAAPAACdAADFAADhBQXrKCjvPDzvNTXvGxvjAADQAADJAAC1AACXAACEAABsAABPAAASAAAC\nAABiAADpAADvAgLnAADYAADLAAC6AACwAABwAAATAAAkAABYAADIAADTAADNAACzAACDAABuAAAe\nAAB+AADAAACkAACNAAB/AABpAABQAAAwAACRAACpAAC8AACqAACbAABlAABJAAAqAAAOAAA0AACs\nAACvAACtAACmAACJAAB6AABrAABaAAA+AAApAABqAACCAACfAACeAACWAACPAAB8AAAZAAAHAABV\nAACOAACKAAA4AAAQAAA/AAByAACAAABcAAA3AAAsAABmAABDAABWAAAgAAAzAAA8AAA6AAAfAAAM\nAAAdAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAD8LtlFAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAIISURBVHicY2CAg/8QwIABmJhZWFnZ2Dk4MaU5uLh5eHn5+LkFBDlQJf8zC/EIi4iK\niUtI8koJScsgyf5nlpWTV1BUUlZRVVPX4NFk1UJIyghp6+jq6RsYGhmbKJgK85mZW8Dk/rNaSlhZ\n29ja2Ts4Ojkr6Li4urFDNf53N/Ow8vTy9vH18w8IDAoWDQkNC4+ASP5ni4wKio6JjYtPSExKTnFW\nSE1LF4A69n9GZlZ2Tm5efkFhUXFySWlZlEd5RSVY7j+TkGRVdU1tXX1DY1Ozcktpa1t7h2YnOAj+\nd7l1tyo79vT29SdNSJ44SbFVdHIo9xSIHNPUaWqTpifNSJrZnK00S0U1a/acUG5piNz/uXLzVJ2q\nm6dXz584S2WB1cJFi5cshZr539xVftnyFKUVTi2TVjqvyhJLXb1m7TqoHPt6F/HW0g0bN63crGqV\ntWXrtu07BJihcsw71+zanRW8Z89eq337RQ/Ip60xO3gIElX/LbikDm8T36KwbNmRo7O3zpHkPSZw\nHBqL//8flz1x2OOkyKJTi7aqbzutfUZI2gIuF8F2lr/D5dw2+fZdwpl8YVOlI+CJ4/9/joOyYed5\nQzMvhGqnm2V0WiClm///D0lfXHtJ6vLlK9w7rx7vQk5SQJbFtSms1y9evXid7QZacgOxmSxktNzd\ntSwwU+J/VICaCPFIYU3XAJhIOtjf5sfyAAAAJXRFWHRDb21tZW50AGNsaXAyZ2lmIHYuMC42IGJ5\nIFl2ZXMgUGlndWV0NnM7vAAAAABJRU5ErkJggg==\n"
  },
  {
    "path": "tests/resources/jmap/email_get/text_body_missing.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 2527,\n  \"receivedAt\": \"2050-01-28T16:26:40Z\",\n  \"messageId\": [\n    \"NDBBIAKOPKHFGPLCODIGKEKECHAA.doug@example.com\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Doug Sauder\",\n      \"email\": \"doug@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"Joe Blow\",\n      \"email\": \"jblow@example.com\"\n    }\n  ],\n  \"subject\": \"Test message from Microsoft Outlook 00\",\n  \"sentAt\": \"2000-05-17T23:35:05Z\",\n  \"bodyStructure\": {\n    \"partId\": \"0\",\n    \"blobId\": \"blob_0\",\n    \"size\": 1453,\n    \"headers\": [\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Doug Sauder\\\" <doug@example.com>\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" \\\"Joe Blow\\\" <jblow@example.com>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" Test message from Microsoft Outlook 00\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Wed, 17 May 2000 19:35:05 -0400\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <NDBBIAKOPKHFGPLCODIGKEKECHAA.doug@example.com>\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" image/png;\\n\\tname=\\\"redball.png\\\"\"\n      },\n      {\n        \"name\": \"Content-Transfer-Encoding\",\n        \"value\": \" base64\"\n      },\n      {\n        \"name\": \"Content-Disposition\",\n        \"value\": \" attachment;\\n\\tfilename=\\\"redball.png\\\"\"\n      },\n      {\n        \"name\": \"X-Priority\",\n        \"value\": \" 3 (Normal)\"\n      },\n      {\n        \"name\": \"X-MSMail-Priority\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-Mailer\",\n        \"value\": \" Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\"\n      },\n      {\n        \"name\": \"Importance\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-MimeOLE\",\n        \"value\": \" Produced By Microsoft MimeOLE V5.00.2314.1300\"\n      }\n    ],\n    \"name\": \"redball.png\",\n    \"type\": \"image/png\",\n    \"disposition\": \"attachment\"\n  },\n  \"bodyValues\": {},\n  \"textBody\": [],\n  \"htmlBody\": [],\n  \"attachments\": [\n    {\n      \"partId\": \"0\",\n      \"blobId\": \"blob_0\",\n      \"size\": 1453,\n      \"headers\": [\n        {\n          \"name\": \"From\",\n          \"value\": \" \\\"Doug Sauder\\\" <doug@example.com>\"\n        },\n        {\n          \"name\": \"To\",\n          \"value\": \" \\\"Joe Blow\\\" <jblow@example.com>\"\n        },\n        {\n          \"name\": \"Subject\",\n          \"value\": \" Test message from Microsoft Outlook 00\"\n        },\n        {\n          \"name\": \"Date\",\n          \"value\": \" Wed, 17 May 2000 19:35:05 -0400\"\n        },\n        {\n          \"name\": \"Message-ID\",\n          \"value\": \" <NDBBIAKOPKHFGPLCODIGKEKECHAA.doug@example.com>\"\n        },\n        {\n          \"name\": \"MIME-Version\",\n          \"value\": \" 1.0\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png;\\n\\tname=\\\"redball.png\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment;\\n\\tfilename=\\\"redball.png\\\"\"\n        },\n        {\n          \"name\": \"X-Priority\",\n          \"value\": \" 3 (Normal)\"\n        },\n        {\n          \"name\": \"X-MSMail-Priority\",\n          \"value\": \" Normal\"\n        },\n        {\n          \"name\": \"X-Mailer\",\n          \"value\": \" Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\"\n        },\n        {\n          \"name\": \"Importance\",\n          \"value\": \" Normal\"\n        },\n        {\n          \"name\": \"X-MimeOLE\",\n          \"value\": \" Produced By Microsoft MimeOLE V5.00.2314.1300\"\n        }\n      ],\n      \"name\": \"redball.png\",\n      \"type\": \"image/png\",\n      \"disposition\": \"attachment\"\n    }\n  ],\n  \"hasAttachment\": true\n}"
  },
  {
    "path": "tests/resources/jmap/email_get/text_body_missing_multipart.eml",
    "content": "From: \"Doug Sauder\" <doug@example.com>\nTo: \"Joe Blow\" <jblow@example.com>\nSubject: Test message from Microsoft Outlook 00\nDate: Wed, 17 May 2000 19:36:13 -0400\nMessage-ID: <NDBBIAKOPKHFGPLCODIGOEKECHAA.doug@example.com>\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"----=_NextPart_000_0004_01BFC037.28F2FA90\"\nX-Priority: 3 (Normal)\nX-MSMail-Priority: Normal\nX-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\nImportance: Normal\nX-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300\n\nThis is a multi-part message in MIME format.\n\n------=_NextPart_000_0004_01BFC037.28F2FA90\nContent-Type: image/png;\n\tname=\"blueball.png\"\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment;\n\tfilename=\"blueball.png\"\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAgAABAAABgAAAAA\nCCkAEEIAEEoACDEAEFIIIXMIKXsIKYQIIWsAGFoACDkIIWMQOZwYQqUYQq0YQrUQOaUQMZQAGFIQ\nMYwpUrU5Y8Y5Y84pWs4YSs4YQs4YQr1Ca8Z7nNacvd6Mtd5jlOcxa94hUt4YStYYQsYQMaUAACHO\n5+/n7++cxu9ShO8pWucQOa1Ke86tzt6lzu9ajO8QMZxahNat1ufO7++Mve9Ke+8YOaUYSsaMvee1\n5++Uve8AAClajOdzpe9rnO8IKYwxY+8pWu8IIXsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAADBMg1VAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAGISURBVHicddJtV5swGAbgEk6AJhBSk4bMCUynBSLaqovbrG/bfPn/vyh70lbssceb\nL5xznTsh5BmNhgQoRChwo50EOIohUYLDj4zHhKYQkrEoQdvock4ne0IKMVUpKZLQDeqSTIsv+18P\nyqqWUw2IBsRM7307PPp+fDJrWtnpLDJvewYxnewfnvanZ+fzpmwXijC8KbqEa3Fx2ff91Y95U9XC\nUpaDeQwiMpHXP/v+1++bWVPWQoGFawtjury9vru/f/C1Vi7ezT0WWpQHf/7+u/G71aLThK/MjRxm\nT6KdzZ9fGk9yatMsTgZLl3XVgFRAC6spj/13enssqJVtWVa3NdBSacL8+VZmYqKmdd1CSYoOiMOS\nGwtzlqqlFFIuOqv0a1ZEZrUkWICLLFW266y1KvWE1zV/iDAH1EopnVLCiygZCIomH3NCKX0lnI+B\n1iuuzCGTxwXjnDO4d7NpbX42YJJHkBwmAm2TxwAZg40J3+Xtbv1rgOAZwG0NxW62p+lT+Yi747sD\n/wEUVMzYmWkOvwAAACV0RVh0Q29tbWVudABjbGlwMmdpZiB2LjAuNiBieSBZdmVzIFBpZ3VldDZz\nO7wAAAAASUVORK5CYII=\n\n------=_NextPart_000_0004_01BFC037.28F2FA90\nContent-Type: image/png;\n\tname=\"redball.png\"\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment;\n\tfilename=\"redball.png\"\n\niVBORw0KGgoAAAANSUhEUgAAABsAAAAbCAMAAAC6CgRnAAADAFBMVEX///8AAAABAAALAAAVAAAa\nAAAXAAARAAAKAAADAAAcAAAyAABEAABNAABIAAA9AAAjAAAWAAAmAABhAAB7AACGAACHAAB9AAB0\nAABgAAA5AAAUAAAGAAAnAABLAABvAACQAAClAAC7AAC/AACrAAChAACMAABzAABbAAAuAAAIAABM\nAAB3AACZAAC0GRnKODjVPT3bKSndBQW4AACoAAB5AAAxAAAYAAAEAABFAACaAAC7JCTRYWHfhITm\nf3/mVlbqHx/SAAC5AACjAABdAABCAAAoAAAJAABnAAC6Dw/QVFTek5PlrKzpmZntZWXvJSXXAADB\nAACxAACcAABtAABTAAA2AAAbAAAFAABKAACBAADLICDdZ2fonJzrpqbtiorvUVHvFBTRAADDAAC2\nAAB4AABeAABAAAAiAABXAACSAADCAADaGxvoVVXseHjveHjvV1fvJibhAADOAAC3AACnAACVAABH\nAAArAAAPAACdAADFAADhBQXrKCjvPDzvNTXvGxvjAADQAADJAAC1AACXAACEAABsAABPAAASAAAC\nAABiAADpAADvAgLnAADYAADLAAC6AACwAABwAAATAAAkAABYAADIAADTAADNAACzAACDAABuAAAe\nAAB+AADAAACkAACNAAB/AABpAABQAAAwAACRAACpAAC8AACqAACbAABlAABJAAAqAAAOAAA0AACs\nAACvAACtAACmAACJAAB6AABrAABaAAA+AAApAABqAACCAACfAACeAACWAACPAAB8AAAZAAAHAABV\nAACOAACKAAA4AAAQAAA/AAByAACAAABcAAA3AAAsAABmAABDAABWAAAgAAAzAAA8AAA6AAAfAAAM\nAAAdAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAD8LtlFAAAAAXRSTlMAQObYZgAAABZ0RVh0U29mdHdhcmUAZ2lmMnBuZyAyLjAu\nMT1evmgAAAIISURBVHicY2CAg/8QwIABmJhZWFnZ2Dk4MaU5uLh5eHn5+LkFBDlQJf8zC/EIi4iK\niUtI8koJScsgyf5nlpWTV1BUUlZRVVPX4NFk1UJIyghp6+jq6RsYGhmbKJgK85mZW8Dk/rNaSlhZ\n29ja2Ts4Ojkr6Li4urFDNf53N/Ow8vTy9vH18w8IDAoWDQkNC4+ASP5ni4wKio6JjYtPSExKTnFW\nSE1LF4A69n9GZlZ2Tm5efkFhUXFySWlZlEd5RSVY7j+TkGRVdU1tXX1DY1Ozcktpa1t7h2YnOAj+\nd7l1tyo79vT29SdNSJ44SbFVdHIo9xSIHNPUaWqTpifNSJrZnK00S0U1a/acUG5piNz/uXLzVJ2q\nm6dXz584S2WB1cJFi5cshZr539xVftnyFKUVTi2TVjqvyhJLXb1m7TqoHPt6F/HW0g0bN63crGqV\ntWXrtu07BJihcsw71+zanRW8Z89eq337RQ/Ip60xO3gIElX/LbikDm8T36KwbNmRo7O3zpHkPSZw\nHBqL//8flz1x2OOkyKJTi7aqbzutfUZI2gIuF8F2lr/D5dw2+fZdwpl8YVOlI+CJ4/9/joOyYed5\nQzMvhGqnm2V0WiClm///D0lfXHtJ6vLlK9w7rx7vQk5SQJbFtSms1y9evXid7QZacgOxmSxktNzd\ntSwwU+J/VICaCPFIYU3XAJhIOtjf5sfyAAAAJXRFWHRDb21tZW50AGNsaXAyZ2lmIHYuMC42IGJ5\nIFl2ZXMgUGlndWV0NnM7vAAAAABJRU5ErkJggg==\n\n------=_NextPart_000_0004_01BFC037.28F2FA90--\n"
  },
  {
    "path": "tests/resources/jmap/email_get/text_body_missing_multipart.json",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"tag\": true\n  },\n  \"size\": 4726,\n  \"receivedAt\": \"2119-10-06T01:46:40Z\",\n  \"messageId\": [\n    \"NDBBIAKOPKHFGPLCODIGOEKECHAA.doug@example.com\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Doug Sauder\",\n      \"email\": \"doug@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"Joe Blow\",\n      \"email\": \"jblow@example.com\"\n    }\n  ],\n  \"subject\": \"Test message from Microsoft Outlook 00\",\n  \"sentAt\": \"2000-05-17T23:36:13Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Doug Sauder\\\" <doug@example.com>\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" \\\"Joe Blow\\\" <jblow@example.com>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" Test message from Microsoft Outlook 00\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Wed, 17 May 2000 19:36:13 -0400\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <NDBBIAKOPKHFGPLCODIGOEKECHAA.doug@example.com>\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed;\\n\\tboundary=\\\"----=_NextPart_000_0004_01BFC037.28F2FA90\\\"\"\n      },\n      {\n        \"name\": \"X-Priority\",\n        \"value\": \" 3 (Normal)\"\n      },\n      {\n        \"name\": \"X-MSMail-Priority\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-Mailer\",\n        \"value\": \" Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)\"\n      },\n      {\n        \"name\": \"Importance\",\n        \"value\": \" Normal\"\n      },\n      {\n        \"name\": \"X-MimeOLE\",\n        \"value\": \" Produced By Microsoft MimeOLE V5.00.2314.1300\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 1325,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/png;\\n\\tname=\\\"blueball.png\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" attachment;\\n\\tfilename=\\\"blueball.png\\\"\"\n          }\n        ],\n        \"name\": \"blueball.png\",\n        \"type\": \"image/png\",\n        \"disposition\": \"attachment\"\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 1453,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/png;\\n\\tname=\\\"redball.png\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" attachment;\\n\\tfilename=\\\"redball.png\\\"\"\n          }\n        ],\n        \"name\": \"redball.png\",\n        \"type\": \"image/png\",\n        \"disposition\": \"attachment\"\n      }\n    ]\n  },\n  \"bodyValues\": {},\n  \"textBody\": [],\n  \"htmlBody\": [],\n  \"attachments\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 1325,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png;\\n\\tname=\\\"blueball.png\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment;\\n\\tfilename=\\\"blueball.png\\\"\"\n        }\n      ],\n      \"name\": \"blueball.png\",\n      \"type\": \"image/png\",\n      \"disposition\": \"attachment\"\n    },\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 1453,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png;\\n\\tname=\\\"redball.png\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment;\\n\\tfilename=\\\"redball.png\\\"\"\n        }\n      ],\n      \"name\": \"redball.png\",\n      \"type\": \"image/png\",\n      \"disposition\": \"attachment\"\n    }\n  ],\n  \"hasAttachment\": true\n}"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment.eml",
    "content": "From:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\nSubject: [Fwd: Map of Argentina with Description]\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\nThis is a multi-part message in MIME format.\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\nContent-Disposition: inline\n\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\n--D7F------------D7FD5A0B8AB9C65CCDBFA872--\n"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment.json",
    "content": "{\n  \"mailboxIds\": null,\n  \"size\": 1979,\n  \"messageId\": [\n    \"199804130742.RAA20366@mai1host.whitehouse.gov\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Bill Clinton\",\n      \"email\": \"president@whitehouse.gov\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"A1 Gore (The Enforcer)\",\n      \"email\": \"vice-president@whitehouse.gov\"\n    }\n  ],\n  \"subject\": \"Map of Argentina with Description\",\n  \"sentAt\": \"1998-08-13T07:42:41Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"Return-Path\",\n        \"value\": \" <president@whitehouse.gov>\"\n      },\n      {\n        \"name\": \"Received\",\n        \"value\": \" from mailhost.whitehouse.gov ([192.168.51.200])\\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\\n        for <vice-president@heartbeat.whitehouse.gov>;\\n        Mon, 13 Aug 1998 l8:14:23 +1000\"\n      },\n      {\n        \"name\": \"Received\",\n        \"value\": \" from the_big_box.whitehouse.gov ([192.168.51.50])\\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Mon, 13 Aug 1998 17:42:41 +1000\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <199804130742.RAA20366@mai1host.whitehouse.gov>\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" Bill Clinton <president@whitehouse.gov>\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \"  Map of Argentina with Description\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed;\\n              boundary=\\\"DC8------------DC8638F443D87A7F0726DEF7\\\"\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 355,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain; charset=us-ascii\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" 7bit\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\"\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 288,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/gif; name=\\\"map_of_Argentina.gif\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          },\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline; fi1ename=\\\"map_of_Argentina.gif\\\"\"\n          }\n        ],\n        \"name\": \"map_of_Argentina.gif\",\n        \"type\": \"image/gif\",\n        \"disposition\": \"inline\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"Hi A1,\\n\\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\\nsome sax music in .au...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 355,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=us-ascii\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 288,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/gif; name=\\\"map_of_Argentina.gif\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline; fi1ename=\\\"map_of_Argentina.gif\\\"\"\n        }\n      ],\n      \"name\": \"map_of_Argentina.gif\",\n      \"type\": \"image/gif\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 355,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=us-ascii\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 288,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/gif; name=\\\"map_of_Argentina.gif\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline; fi1ename=\\\"map_of_Argentina.gif\\\"\"\n        }\n      ],\n      \"name\": \"map_of_Argentina.gif\",\n      \"type\": \"image/gif\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 288,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/gif; name=\\\"map_of_Argentina.gif\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        },\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline; fi1ename=\\\"map_of_Argentina.gif\\\"\"\n        }\n      ],\n      \"name\": \"map_of_Argentina.gif\",\n      \"type\": \"image/gif\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"hasAttachment\": false,\n  \"preview\": \"Hi A1,\\n\\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\\nsome sax music in .au files next week!\\n\\nAnyway, the attached image is really too small to get a good look at\\nArgentina.  Try this for a much better map:\\n\\n     http://www.1one1yp...\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment.part1",
    "content": "Hi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment.part3",
    "content": "<html><body>Hi A1,<br/><br/>I finally figured out this MIME thing.  Pretty cool.  I'll send you<br/>some sax music in .au files next week!<br/><br/>Anyway, the attached image is really too small to get a good look at<br/>Argentina.  Try this for a much better map:<br/><br/>     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm<br/><br/>Then again, shouldn't the CIA have something like that?<br/><br/>Bill</body></html>"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment_b64.eml",
    "content": "Content-Type: multipart/mixed;\n boundary=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\n\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 7bit\n\nThis is a message with a base64 encoded attached email\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nContent-Disposition: attachment; filename=\"attached_email.eml\"\nContent-Type: message/rfc822\nContent-Transfer-Encoding: base64\n\nVG86ICJlbWFpbEBleGFtcGxlLmNvbSIgPGVtYWlsQGV4YW1wbGUuY29tPg0KRnJvbTogTmFtZSA8\nZW1haWxAZXhhbXBsZS5jb20+DQpTdWJqZWN0OiBIVE1MIHRlc3QNCk1lc3NhZ2UtSUQ6IDxyYW5k\nb20tbWVzc2FnZS1pZEBleGFtcGxlLmNvbT4NCkRhdGU6IFR1ZSwgMTQgRGVjIDIwMjEgMTE6NDg6\nMjUgKzAxMDANCk1JTUUtVmVyc2lvbjogMS4wDQpDb250ZW50LVR5cGU6IG11bHRpcGFydC9hbHRl\ncm5hdGl2ZTsNCiBib3VuZGFyeT0iYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYSINCkNvbnRlbnQtTGFuZ3VhZ2U6IGVuLVVTDQoNClRoaXMgaXMgYSBtdWx0aS1wYXJ0IG1l\nc3NhZ2UgaW4gTUlNRSBmb3JtYXQuDQotLWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWENCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNldD11dGYtODsgZm9ybWF0\nPWZsb3dlZA0KQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogN2JpdA0KDQpUaGlzIGlzIGFuICpI\nVE1MKiB0ZXN0IG1lc3NhZ2UNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYQ0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgNCkNvbnRlbnQtVHJh\nbnNmZXItRW5jb2Rpbmc6IDdiaXQNCg0KPGh0bWw+DQogIDxoZWFkPg0KICAgIDxtZXRhIGh0dHAt\nZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04Ij4N\nCiAgPC9oZWFkPg0KICA8Ym9keT4NCiAgICBUaGlzIGlzIGFuIDxiPkhUTUw8L2I+IHRlc3QgbWVz\nc2FnZQ0KICA8L2JvZHk+DQo8L2h0bWw+DQoNCi0tYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh\nYWFhYWFhYWFhYWFhYS0tDQo=\n--bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb--\n"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment_b64.json",
    "content": "{\n  \"mailboxIds\": null,\n  \"size\": 872,\n  \"messageId\": [\n    \"random-message-id@example.com\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Name\",\n      \"email\": \"email@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"email@example.com\",\n      \"email\": \"email@example.com\"\n    }\n  ],\n  \"subject\": \"HTML test\",\n  \"sentAt\": \"2021-12-14T10:48:25Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"To\",\n        \"value\": \" \\\"email@example.com\\\" <email@example.com>\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" Name <email@example.com>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" HTML test\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <random-message-id@example.com>\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Tue, 14 Dec 2021 11:48:25 +0100\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/alternative;\\r\\n boundary=\\\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\\\"\"\n      },\n      {\n        \"name\": \"Content-Language\",\n        \"value\": \" en-US\"\n      }\n    ],\n    \"type\": \"multipart/alternative\",\n    \"language\": [\n      \"en-US\"\n    ],\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 30,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain; charset=utf-8; format=flowed\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" 7bit\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"utf-8\"\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 173,\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/html; charset=utf-8\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" 7bit\"\n          }\n        ],\n        \"type\": \"text/html\",\n        \"charset\": \"utf-8\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"This is an *HTML* test message\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"2\": {\n      \"value\": \"<html>\\n  <head>\\n    <meta http-equiv=\\\"content-type\\\" content=\\\"text/html; charset=UTF-8\\\">\\n  </head>...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 30,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=utf-8; format=flowed\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"utf-8\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 173,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html; charset=utf-8\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"utf-8\"\n    }\n  ],\n  \"attachments\": [],\n  \"hasAttachment\": false,\n  \"preview\": \"This is an *HTML* test message\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment_b64.part1",
    "content": "This is an *HTML* test message"
  },
  {
    "path": "tests/resources/jmap/email_parse/attachment_b64.part2",
    "content": "<html>\r\n  <head>\r\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\r\n  </head>\r\n  <body>\r\n    This is an <b>HTML</b> test message\r\n  </body>\r\n</html>\r\n"
  },
  {
    "path": "tests/resources/jmap/email_parse/headers.eml",
    "content": "From: Art Vandelay <art@vandelay.com> (Vandelay Industries)\nTo: \"  James Smythe\" <james@example.com>, Friends:\n      jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?=\n      <john@example.com>;\nCc: List 1: addr1@test.com, addr2@test.com; List 2: addr3@test.com, \n       addr4@test.com; addr5@test.com, addr6@test.com\nCc: =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: addr1@test.com, \n       addr2@test.com; =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: \n       addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\nBcc: Greg Vaudreuil <gvaudre@NRI.Reston.VA.US>, Ned Freed\n        <ned@innosoft.com>, Keith Moore <moore@cs.utk.edu>\nBcc: ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se\nDate: Tue, 1 Jul 2003 10:52:37 +0200\nResent-Date: Tue, 2 Jul 2005 11:52:37 +0200\nResent-Date: Tue, 3 Jul 2005 12:52:37 +0300\nResent-Date: Tue, 4 Jul 2005 13:52:37 +0400\nMessage-ID: <5678.21-Nov-1997@example.com>\nReferences: <1234@local.machine.example>\n            <3456@example.net>\nReferences: <789@local.machine.example>\n            <abcd@example.net>\nKeywords: multipart, alternative, example\nList-Post: <mailto:moderator@host.com> (Postings are Moderated)\nList-Subscribe: (Use this command to join the list)\n       <mailto:list-manager@host.com?body=subscribe%20list>\nList-Subscribe: <ftp://ftp.host.com/list.txt> (FTP),  \n               <mailto:list@host.com?subject=subscribe>\nList-Owner: <http://www.host.com/list.cgi?cmd=sub&lst=list>,\n       <mailto:list-manager@host.com?body=subscribe%20list>\nList-Unsubscribe: (Use this command to get off the list)\n         <mailto:list-manager@host.com?body=unsubscribe%20list>\nSubject: Why not both importing AND exporting? =?utf-8?b?4pi6?=\nX-Address-Single: =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>\nX-Address: Mary Smith <mary@example.net>\nX-Address: John Doe <jdoe@machine.example>\nX-AddressList-Single: Mary Smith <mary@x.test>, jdoe@example.org, Who? <one@y.test>\nX-AddressList: =?US-ASCII*EN?Q?Keith_Moore?= <moore@cs.utk.edu>, \n                John =?US-ASCII*EN?Q?Doe?= <moore@cs.utk.edu>\nX-AddressList: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>,\n                =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>\nX-AddressesGroup-Single: A Group(Some people)\n            :Chris Jones <c@(Chris's host.)public.example>,\n            joe@example.org,     John <jdoe@one.test> (my dear\n            friend); (the end of the group)\nX-AddressesGroup: A Group:Ed Jones <c@a.test>,joe@where.test,John <jdoe@one.test>;\nX-AddressesGroup: \"List 1\": addr1@test.com, addr2@test.com; \"List 2\": \n            addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\nX-List-Single: <mailto:x-moderator@host.com> (X-Postings are Moderated)\nX-List: <http://www.mylist.com/list>,\n       <mailto:list@mylist.com>\nX-List: <http://www.mylist2.com/list2>,\n       <mailto:list2@mylist2.com>\nX-Text-Single: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=\n              =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=\nX-Text: =?iso-8859-1?q?this=20is=20some=20text?=\nX-Text: =?ISO-8859-1?Q?a?= =?ISO-8859-2?Q?_b?=\nX-Date-Single: Tue, 5 Jul 2006 13:52:37 -0500\nX-Date: Sat, 20 Nov 2021 14:22:01 -0800\nX-Date: Sun, 21 Nov 2021 15:23:02 -0900\nX-Id-Single: <myid@example.com> <myid2@example.com>\nX-Id: <myid3@example.com>\nX-Id: <myid4@example.com> <myid5@example.com>\nContent-Type: multipart/mixed; boundary=\"festivus\";\n\n--festivus\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: base64\nX-Custom-Header: 123\n\nPGh0bWw+PHA+SSB3YXMgdGhpbmtpbmcgYWJvdXQgcXVpdHRpbmcgdGhlICZsZHF1bztle\nHBvcnRpbmcmcmRxdW87IHRvIGZvY3VzIGp1c3Qgb24gdGhlICZsZHF1bztpbXBvcnRpbm\ncmcmRxdW87LDwvcD48cD5idXQgdGhlbiBJIHRob3VnaHQsIHdoeSBub3QgZG8gYm90aD8\ngJiN4MjYzQTs8L3A+PC9odG1sPg==\n--festivus\nContent-Type: message/rfc822\nX-Custom-Header-2: 345\n\nFrom: \"Cosmo Kramer\" <kramer@kramerica.com>\nSubject: Exporting my book about coffee tables\nContent-Type: multipart/mixed; boundary=\"giddyup\";\n\n--giddyup\nContent-Type: text/plain; charset=\"utf-16\"\nContent-Transfer-Encoding: quoted-printable\n\n=FF=FE=0C!5=D8\"=DD5=D8)=DD5=D8-=DD =005=D8*=DD5=D8\"=DD =005=D8\"=\n=DD5=D85=DD5=D8-=DD5=D8,=DD5=D8/=DD5=D81=DD =005=D8*=DD5=D86=DD =\n=005=D8=1F=DD5=D8,=DD5=D8,=DD5=D8(=DD =005=D8-=DD5=D8)=DD5=D8\"=\n=DD5=D8=1E=DD5=D80=DD5=D8\"=DD!=00\n--giddyup\nContent-Type: image/gif; name*1=\"about \"; name*0=\"Book \";\n              name*2*=utf-8''%e2%98%95 tables.gif\nContent-Transfer-Encoding: Base64\nContent-Disposition: attachment\n\nR0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\n--giddyup--\n--festivus--\n"
  },
  {
    "path": "tests/resources/jmap/email_parse/headers.json",
    "content": "{\n  \"mailboxIds\": null,\n  \"messageId\": [\n    \"5678.21-Nov-1997@example.com\"\n  ],\n  \"references\": [\n    \"789@local.machine.example\",\n    \"abcd@example.net\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Art Vandelay (Vandelay Industries)\",\n      \"email\": \"art@vandelay.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"  James Smythe\",\n      \"email\": \"james@example.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"jane@example.com\"\n    },\n    {\n      \"name\": \"John Smîth\",\n      \"email\": \"john@example.com\"\n    }\n  ],\n  \"cc\": [\n    {\n      \"name\": null,\n      \"email\": \"addr1@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr2@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr3@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr4@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr5@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr6@test.com\"\n    }\n  ],\n  \"bcc\": [\n    {\n      \"name\": null,\n      \"email\": \"ietf-822@dimacs.rutgers.edu\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"ojarnef@admin.kth.se\"\n    }\n  ],\n  \"subject\": \"Why not both importing AND exporting? ☺\",\n  \"sentAt\": \"2003-07-01T08:52:37Z\",\n  \"textBody\": [\n    {\n      \"size\": 175,\n      \"type\": \"text/html\",\n      \"charset\": \"us-ascii\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"size\": 175,\n      \"type\": \"text/html\",\n      \"charset\": \"us-ascii\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"size\": 723,\n      \"type\": \"message/rfc822\"\n    }\n  ],\n  \"preview\": \"I was thinking about quitting the “exporting” to focus just on the “importing”,\\nbut then I thought, why not do both? ☺\\n\",\n  \"header:Bcc\": \" ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se\",\n  \"header:Bcc:all\": [\n    \" Greg Vaudreuil <gvaudre@NRI.Reston.VA.US>, Ned Freed\\n        <ned@innosoft.com>, Keith Moore <moore@cs.utk.edu>\",\n    \" ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se\"\n  ],\n  \"header:Bcc:asAddresses\": [\n    {\n      \"name\": null,\n      \"email\": \"ietf-822@dimacs.rutgers.edu\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"ojarnef@admin.kth.se\"\n    }\n  ],\n  \"header:Bcc:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Greg Vaudreuil\",\n        \"email\": \"gvaudre@NRI.Reston.VA.US\"\n      },\n      {\n        \"name\": \"Ned Freed\",\n        \"email\": \"ned@innosoft.com\"\n      },\n      {\n        \"name\": \"Keith Moore\",\n        \"email\": \"moore@cs.utk.edu\"\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"email\": \"ietf-822@dimacs.rutgers.edu\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"ojarnef@admin.kth.se\"\n      }\n    ]\n  ],\n  \"header:Bcc:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"ietf-822@dimacs.rutgers.edu\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"ojarnef@admin.kth.se\"\n        }\n      ]\n    }\n  ],\n  \"header:Bcc:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Greg Vaudreuil\",\n            \"email\": \"gvaudre@NRI.Reston.VA.US\"\n          },\n          {\n            \"name\": \"Ned Freed\",\n            \"email\": \"ned@innosoft.com\"\n          },\n          {\n            \"name\": \"Keith Moore\",\n            \"email\": \"moore@cs.utk.edu\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"ietf-822@dimacs.rutgers.edu\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"ojarnef@admin.kth.se\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:Cc\": \" =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: addr1@test.com, \\n       addr2@test.com; =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: \\n       addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\",\n  \"header:Cc:all\": [\n    \" List 1: addr1@test.com, addr2@test.com; List 2: addr3@test.com, \\n       addr4@test.com; addr5@test.com, addr6@test.com\",\n    \" =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: addr1@test.com, \\n       addr2@test.com; =?utf-8?b?VGjDrXMgw61zIHbDoWzDrWQgw5pURjg=?=: \\n       addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\"\n  ],\n  \"header:Cc:asAddresses\": [\n    {\n      \"name\": null,\n      \"email\": \"addr1@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr2@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr3@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr4@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr5@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr6@test.com\"\n    }\n  ],\n  \"header:Cc:asAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"email\": \"addr1@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr2@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr3@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr4@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr5@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr6@test.com\"\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"email\": \"addr1@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr2@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr3@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr4@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr5@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr6@test.com\"\n      }\n    ]\n  ],\n  \"header:Cc:asGroupedAddresses\": [\n    {\n      \"name\": \"Thís ís válíd ÚTF8\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr1@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr2@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Thís ís válíd ÚTF8\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr3@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr4@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr5@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr6@test.com\"\n        }\n      ]\n    }\n  ],\n  \"header:Cc:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": \"List 1\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr1@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr2@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"List 2\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr3@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr4@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr5@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr6@test.com\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": \"Thís ís válíd ÚTF8\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr1@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr2@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"Thís ís válíd ÚTF8\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr3@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr4@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr5@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr6@test.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:Date\": \" Tue, 1 Jul 2003 10:52:37 +0200\",\n  \"header:Date:all\": [\n    \" Tue, 1 Jul 2003 10:52:37 +0200\"\n  ],\n  \"header:Date:asDate\": \"2003-07-01T08:52:37Z\",\n  \"header:Date:asDate:all\": [\n    \"2003-07-01T08:52:37Z\"\n  ],\n  \"header:From\": \" Art Vandelay <art@vandelay.com> (Vandelay Industries)\",\n  \"header:From:all\": [\n    \" Art Vandelay <art@vandelay.com> (Vandelay Industries)\"\n  ],\n  \"header:From:asAddresses\": [\n    {\n      \"name\": \"Art Vandelay (Vandelay Industries)\",\n      \"email\": \"art@vandelay.com\"\n    }\n  ],\n  \"header:From:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Art Vandelay (Vandelay Industries)\",\n        \"email\": \"art@vandelay.com\"\n      }\n    ]\n  ],\n  \"header:From:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"Art Vandelay (Vandelay Industries)\",\n          \"email\": \"art@vandelay.com\"\n        }\n      ]\n    }\n  ],\n  \"header:From:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Art Vandelay (Vandelay Industries)\",\n            \"email\": \"art@vandelay.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:Keywords\": \" multipart, alternative, example\",\n  \"header:Keywords:all\": [\n    \" multipart, alternative, example\"\n  ],\n  \"header:Keywords:asText\": \"multipart, alternative, example\",\n  \"header:Keywords:asText:all\": [\n    \"multipart, alternative, example\"\n  ],\n  \"header:List-Owner\": \" <http://www.host.com/list.cgi?cmd=sub&lst=list>,\\n       <mailto:list-manager@host.com?body=subscribe%20list>\",\n  \"header:List-Owner:all\": [\n    \" <http://www.host.com/list.cgi?cmd=sub&lst=list>,\\n       <mailto:list-manager@host.com?body=subscribe%20list>\"\n  ],\n  \"header:List-Owner:asURLs\": [\n    \"http://www.host.com/list.cgi?cmd=sub&lst=list\",\n    \"mailto:list-manager@host.com?body=subscribe%20list\"\n  ],\n  \"header:List-Owner:asURLs:all\": [\n    [\n      \"http://www.host.com/list.cgi?cmd=sub&lst=list\",\n      \"mailto:list-manager@host.com?body=subscribe%20list\"\n    ]\n  ],\n  \"header:List-Post\": \" <mailto:moderator@host.com> (Postings are Moderated)\",\n  \"header:List-Post:all\": [\n    \" <mailto:moderator@host.com> (Postings are Moderated)\"\n  ],\n  \"header:List-Post:asURLs\": [\n    \"mailto:moderator@host.com\"\n  ],\n  \"header:List-Post:asURLs:all\": [\n    [\n      \"mailto:moderator@host.com\"\n    ]\n  ],\n  \"header:List-Subscribe\": \" <ftp://ftp.host.com/list.txt> (FTP),  \\n               <mailto:list@host.com?subject=subscribe>\",\n  \"header:List-Subscribe:all\": [\n    \" (Use this command to join the list)\\n       <mailto:list-manager@host.com?body=subscribe%20list>\",\n    \" <ftp://ftp.host.com/list.txt> (FTP),  \\n               <mailto:list@host.com?subject=subscribe>\"\n  ],\n  \"header:List-Subscribe:asURLs\": [\n    \"ftp://ftp.host.com/list.txt\",\n    \"mailto:list@host.com?subject=subscribe\"\n  ],\n  \"header:List-Subscribe:asURLs:all\": [\n    [\n      \"mailto:list-manager@host.com?body=subscribe%20list\"\n    ],\n    [\n      \"ftp://ftp.host.com/list.txt\",\n      \"mailto:list@host.com?subject=subscribe\"\n    ]\n  ],\n  \"header:List-Unsubscribe\": \" (Use this command to get off the list)\\n         <mailto:list-manager@host.com?body=unsubscribe%20list>\",\n  \"header:List-Unsubscribe:all\": [\n    \" (Use this command to get off the list)\\n         <mailto:list-manager@host.com?body=unsubscribe%20list>\"\n  ],\n  \"header:List-Unsubscribe:asURLs\": [\n    \"mailto:list-manager@host.com?body=unsubscribe%20list\"\n  ],\n  \"header:List-Unsubscribe:asURLs:all\": [\n    [\n      \"mailto:list-manager@host.com?body=unsubscribe%20list\"\n    ]\n  ],\n  \"header:Message-ID\": \" <5678.21-Nov-1997@example.com>\",\n  \"header:Message-ID:all\": [\n    \" <5678.21-Nov-1997@example.com>\"\n  ],\n  \"header:Message-ID:asMessageIds\": [\n    \"5678.21-Nov-1997@example.com\"\n  ],\n  \"header:Message-ID:asMessageIds:all\": [\n    [\n      \"5678.21-Nov-1997@example.com\"\n    ]\n  ],\n  \"header:References\": \" <789@local.machine.example>\\n            <abcd@example.net>\",\n  \"header:References:all\": [\n    \" <1234@local.machine.example>\\n            <3456@example.net>\",\n    \" <789@local.machine.example>\\n            <abcd@example.net>\"\n  ],\n  \"header:References:asMessageIds\": [\n    \"789@local.machine.example\",\n    \"abcd@example.net\"\n  ],\n  \"header:References:asMessageIds:all\": [\n    [\n      \"1234@local.machine.example\",\n      \"3456@example.net\"\n    ],\n    [\n      \"789@local.machine.example\",\n      \"abcd@example.net\"\n    ]\n  ],\n  \"header:Resent-Date\": \" Tue, 4 Jul 2005 13:52:37 +0400\",\n  \"header:Resent-Date:all\": [\n    \" Tue, 2 Jul 2005 11:52:37 +0200\",\n    \" Tue, 3 Jul 2005 12:52:37 +0300\",\n    \" Tue, 4 Jul 2005 13:52:37 +0400\"\n  ],\n  \"header:Resent-Date:asDate\": \"2005-07-04T09:52:37Z\",\n  \"header:Resent-Date:asDate:all\": [\n    \"2005-07-02T09:52:37Z\",\n    \"2005-07-03T09:52:37Z\",\n    \"2005-07-04T09:52:37Z\"\n  ],\n  \"header:Subject\": \" Why not both importing AND exporting? =?utf-8?b?4pi6?=\",\n  \"header:Subject:all\": [\n    \" Why not both importing AND exporting? =?utf-8?b?4pi6?=\"\n  ],\n  \"header:Subject:asText\": \"Why not both importing AND exporting? ☺\",\n  \"header:Subject:asText:all\": [\n    \"Why not both importing AND exporting? ☺\"\n  ],\n  \"header:To\": \" \\\"  James Smythe\\\" <james@example.com>, Friends:\\n      jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?=\\n      <john@example.com>;\",\n  \"header:To:all\": [\n    \" \\\"  James Smythe\\\" <james@example.com>, Friends:\\n      jane@example.com, =?UTF-8?Q?John_Sm=C3=AEth?=\\n      <john@example.com>;\"\n  ],\n  \"header:To:asAddresses\": [\n    {\n      \"name\": \"  James Smythe\",\n      \"email\": \"james@example.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"jane@example.com\"\n    },\n    {\n      \"name\": \"John Smîth\",\n      \"email\": \"john@example.com\"\n    }\n  ],\n  \"header:To:asAddresses:all\": [\n    [\n      {\n        \"name\": \"  James Smythe\",\n        \"email\": \"james@example.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"jane@example.com\"\n      },\n      {\n        \"name\": \"John Smîth\",\n        \"email\": \"john@example.com\"\n      }\n    ]\n  ],\n  \"header:To:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"  James Smythe\",\n          \"email\": \"james@example.com\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Friends\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"jane@example.com\"\n        },\n        {\n          \"name\": \"John Smîth\",\n          \"email\": \"john@example.com\"\n        }\n      ]\n    }\n  ],\n  \"header:To:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"  James Smythe\",\n            \"email\": \"james@example.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"Friends\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"jane@example.com\"\n          },\n          {\n            \"name\": \"John Smîth\",\n            \"email\": \"john@example.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-Address\": \" John Doe <jdoe@machine.example>\",\n  \"header:X-Address:all\": [\n    \" Mary Smith <mary@example.net>\",\n    \" John Doe <jdoe@machine.example>\"\n  ],\n  \"header:X-Address:asAddresses\": [\n    {\n      \"name\": \"John Doe\",\n      \"email\": \"jdoe@machine.example\"\n    }\n  ],\n  \"header:X-Address:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Mary Smith\",\n        \"email\": \"mary@example.net\"\n      }\n    ],\n    [\n      {\n        \"name\": \"John Doe\",\n        \"email\": \"jdoe@machine.example\"\n      }\n    ]\n  ],\n  \"header:X-Address:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"John Doe\",\n          \"email\": \"jdoe@machine.example\"\n        }\n      ]\n    }\n  ],\n  \"header:X-Address:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Mary Smith\",\n            \"email\": \"mary@example.net\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@machine.example\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-Address-Single\": \" =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>\",\n  \"header:X-Address-Single:all\": [\n    \" =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>\"\n  ],\n  \"header:X-Address-Single:asAddresses\": [\n    {\n      \"name\": \"André Pirard\",\n      \"email\": \"PIRARD@vm1.ulg.ac.be\"\n    }\n  ],\n  \"header:X-Address-Single:asAddresses:all\": [\n    [\n      {\n        \"name\": \"André Pirard\",\n        \"email\": \"PIRARD@vm1.ulg.ac.be\"\n      }\n    ]\n  ],\n  \"header:X-Address-Single:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"André Pirard\",\n          \"email\": \"PIRARD@vm1.ulg.ac.be\"\n        }\n      ]\n    }\n  ],\n  \"header:X-Address-Single:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"André Pirard\",\n            \"email\": \"PIRARD@vm1.ulg.ac.be\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressList\": \" =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>,\\n                =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>\",\n  \"header:X-AddressList:all\": [\n    \" =?US-ASCII*EN?Q?Keith_Moore?= <moore@cs.utk.edu>, \\n                John =?US-ASCII*EN?Q?Doe?= <moore@cs.utk.edu>\",\n    \" =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>,\\n                =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>\"\n  ],\n  \"header:X-AddressList:asAddresses\": [\n    {\n      \"name\": \"Keld Jørn Simonsen\",\n      \"email\": \"keld@dkuug.dk\"\n    },\n    {\n      \"name\": \"Olle Järnefors\",\n      \"email\": \"ojarnef@admin.kth.se\"\n    }\n  ],\n  \"header:X-AddressList:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Keith Moore\",\n        \"email\": \"moore@cs.utk.edu\"\n      },\n      {\n        \"name\": \"John Doe\",\n        \"email\": \"moore@cs.utk.edu\"\n      }\n    ],\n    [\n      {\n        \"name\": \"Keld Jørn Simonsen\",\n        \"email\": \"keld@dkuug.dk\"\n      },\n      {\n        \"name\": \"Olle Järnefors\",\n        \"email\": \"ojarnef@admin.kth.se\"\n      }\n    ]\n  ],\n  \"header:X-AddressList:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"Keld Jørn Simonsen\",\n          \"email\": \"keld@dkuug.dk\"\n        },\n        {\n          \"name\": \"Olle Järnefors\",\n          \"email\": \"ojarnef@admin.kth.se\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressList:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Keith Moore\",\n            \"email\": \"moore@cs.utk.edu\"\n          },\n          {\n            \"name\": \"John Doe\",\n            \"email\": \"moore@cs.utk.edu\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Keld Jørn Simonsen\",\n            \"email\": \"keld@dkuug.dk\"\n          },\n          {\n            \"name\": \"Olle Järnefors\",\n            \"email\": \"ojarnef@admin.kth.se\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressList-Single\": \" Mary Smith <mary@x.test>, jdoe@example.org, Who? <one@y.test>\",\n  \"header:X-AddressList-Single:all\": [\n    \" Mary Smith <mary@x.test>, jdoe@example.org, Who? <one@y.test>\"\n  ],\n  \"header:X-AddressList-Single:asAddresses\": [\n    {\n      \"name\": \"Mary Smith\",\n      \"email\": \"mary@x.test\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"jdoe@example.org\"\n    },\n    {\n      \"name\": \"Who?\",\n      \"email\": \"one@y.test\"\n    }\n  ],\n  \"header:X-AddressList-Single:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Mary Smith\",\n        \"email\": \"mary@x.test\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"jdoe@example.org\"\n      },\n      {\n        \"name\": \"Who?\",\n        \"email\": \"one@y.test\"\n      }\n    ]\n  ],\n  \"header:X-AddressList-Single:asGroupedAddresses\": [\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"Mary Smith\",\n          \"email\": \"mary@x.test\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"jdoe@example.org\"\n        },\n        {\n          \"name\": \"Who?\",\n          \"email\": \"one@y.test\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressList-Single:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"Mary Smith\",\n            \"email\": \"mary@x.test\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"jdoe@example.org\"\n          },\n          {\n            \"name\": \"Who?\",\n            \"email\": \"one@y.test\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup\": \" \\\"List 1\\\": addr1@test.com, addr2@test.com; \\\"List 2\\\": \\n            addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\",\n  \"header:X-AddressesGroup:all\": [\n    \" A Group:Ed Jones <c@a.test>,joe@where.test,John <jdoe@one.test>;\",\n    \" \\\"List 1\\\": addr1@test.com, addr2@test.com; \\\"List 2\\\": \\n            addr3@test.com, addr4@test.com; addr5@test.com, addr6@test.com\"\n  ],\n  \"header:X-AddressesGroup:asAddresses\": [\n    {\n      \"name\": null,\n      \"email\": \"addr1@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr2@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr3@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr4@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr5@test.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"addr6@test.com\"\n    }\n  ],\n  \"header:X-AddressesGroup:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Ed Jones\",\n        \"email\": \"c@a.test\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"joe@where.test\"\n      },\n      {\n        \"name\": \"John\",\n        \"email\": \"jdoe@one.test\"\n      }\n    ],\n    [\n      {\n        \"name\": null,\n        \"email\": \"addr1@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr2@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr3@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr4@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr5@test.com\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"addr6@test.com\"\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup:asGroupedAddresses\": [\n    {\n      \"name\": \"List 1\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr1@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr2@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": \"List 2\",\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr3@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr4@test.com\"\n        }\n      ]\n    },\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": null,\n          \"email\": \"addr5@test.com\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"addr6@test.com\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressesGroup:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": \"A Group\",\n        \"addresses\": [\n          {\n            \"name\": \"Ed Jones\",\n            \"email\": \"c@a.test\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"joe@where.test\"\n          },\n          {\n            \"name\": \"John\",\n            \"email\": \"jdoe@one.test\"\n          }\n        ]\n      }\n    ],\n    [\n      {\n        \"name\": \"List 1\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr1@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr2@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": \"List 2\",\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr3@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr4@test.com\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": null,\n            \"email\": \"addr5@test.com\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"addr6@test.com\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup-Single\": \" A Group(Some people)\\n            :Chris Jones <c@(Chris's host.)public.example>,\\n            joe@example.org,     John <jdoe@one.test> (my dear\\n            friend); (the end of the group)\",\n  \"header:X-AddressesGroup-Single:all\": [\n    \" A Group(Some people)\\n            :Chris Jones <c@(Chris's host.)public.example>,\\n            joe@example.org,     John <jdoe@one.test> (my dear\\n            friend); (the end of the group)\"\n  ],\n  \"header:X-AddressesGroup-Single:asAddresses\": [\n    {\n      \"name\": \"Chris Jones (Chris's host.)\",\n      \"email\": \"c@public.example\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"joe@example.org\"\n    },\n    {\n      \"name\": \"John (my dear friend)\",\n      \"email\": \"jdoe@one.test\"\n    },\n    {\n      \"name\": \"the end of the group\",\n      \"email\": \"\"\n    }\n  ],\n  \"header:X-AddressesGroup-Single:asAddresses:all\": [\n    [\n      {\n        \"name\": \"Chris Jones (Chris's host.)\",\n        \"email\": \"c@public.example\"\n      },\n      {\n        \"name\": null,\n        \"email\": \"joe@example.org\"\n      },\n      {\n        \"name\": \"John (my dear friend)\",\n        \"email\": \"jdoe@one.test\"\n      },\n      {\n        \"name\": \"the end of the group\",\n        \"email\": \"\"\n      }\n    ]\n  ],\n  \"header:X-AddressesGroup-Single:asGroupedAddresses\": [\n    {\n      \"name\": \"A Group (Some people)\",\n      \"addresses\": [\n        {\n          \"name\": \"Chris Jones (Chris's host.)\",\n          \"email\": \"c@public.example\"\n        },\n        {\n          \"name\": null,\n          \"email\": \"joe@example.org\"\n        },\n        {\n          \"name\": \"John (my dear friend)\",\n          \"email\": \"jdoe@one.test\"\n        }\n      ]\n    },\n    {\n      \"name\": null,\n      \"addresses\": [\n        {\n          \"name\": \"the end of the group\",\n          \"email\": \"\"\n        }\n      ]\n    }\n  ],\n  \"header:X-AddressesGroup-Single:asGroupedAddresses:all\": [\n    [\n      {\n        \"name\": \"A Group (Some people)\",\n        \"addresses\": [\n          {\n            \"name\": \"Chris Jones (Chris's host.)\",\n            \"email\": \"c@public.example\"\n          },\n          {\n            \"name\": null,\n            \"email\": \"joe@example.org\"\n          },\n          {\n            \"name\": \"John (my dear friend)\",\n            \"email\": \"jdoe@one.test\"\n          }\n        ]\n      },\n      {\n        \"name\": null,\n        \"addresses\": [\n          {\n            \"name\": \"the end of the group\",\n            \"email\": \"\"\n          }\n        ]\n      }\n    ]\n  ],\n  \"header:X-Date\": \" Sun, 21 Nov 2021 15:23:02 -0900\",\n  \"header:X-Date:all\": [\n    \" Sat, 20 Nov 2021 14:22:01 -0800\",\n    \" Sun, 21 Nov 2021 15:23:02 -0900\"\n  ],\n  \"header:X-Date:asDate\": \"2021-11-22T00:23:02Z\",\n  \"header:X-Date:asDate:all\": [\n    \"2021-11-20T22:22:01Z\",\n    \"2021-11-22T00:23:02Z\"\n  ],\n  \"header:X-Date-Single\": \" Tue, 5 Jul 2006 13:52:37 -0500\",\n  \"header:X-Date-Single:all\": [\n    \" Tue, 5 Jul 2006 13:52:37 -0500\"\n  ],\n  \"header:X-Date-Single:asDate\": \"2006-07-05T18:52:37Z\",\n  \"header:X-Date-Single:asDate:all\": [\n    \"2006-07-05T18:52:37Z\"\n  ],\n  \"header:X-Id\": \" <myid4@example.com> <myid5@example.com>\",\n  \"header:X-Id:all\": [\n    \" <myid3@example.com>\",\n    \" <myid4@example.com> <myid5@example.com>\"\n  ],\n  \"header:X-Id:asMessageIds\": [\n    \"myid4@example.com\",\n    \"myid5@example.com\"\n  ],\n  \"header:X-Id:asMessageIds:all\": [\n    [\n      \"myid3@example.com\"\n    ],\n    [\n      \"myid4@example.com\",\n      \"myid5@example.com\"\n    ]\n  ],\n  \"header:X-Id-Single\": \" <myid@example.com> <myid2@example.com>\",\n  \"header:X-Id-Single:all\": [\n    \" <myid@example.com> <myid2@example.com>\"\n  ],\n  \"header:X-Id-Single:asMessageIds\": [\n    \"myid@example.com\",\n    \"myid2@example.com\"\n  ],\n  \"header:X-Id-Single:asMessageIds:all\": [\n    [\n      \"myid@example.com\",\n      \"myid2@example.com\"\n    ]\n  ],\n  \"header:X-List\": \" <http://www.mylist2.com/list2>,\\n       <mailto:list2@mylist2.com>\",\n  \"header:X-List:all\": [\n    \" <http://www.mylist.com/list>,\\n       <mailto:list@mylist.com>\",\n    \" <http://www.mylist2.com/list2>,\\n       <mailto:list2@mylist2.com>\"\n  ],\n  \"header:X-List:asURLs\": [\n    \"http://www.mylist2.com/list2\",\n    \"mailto:list2@mylist2.com\"\n  ],\n  \"header:X-List:asURLs:all\": [\n    [\n      \"http://www.mylist.com/list\",\n      \"mailto:list@mylist.com\"\n    ],\n    [\n      \"http://www.mylist2.com/list2\",\n      \"mailto:list2@mylist2.com\"\n    ]\n  ],\n  \"header:X-List-Single\": \" <mailto:x-moderator@host.com> (X-Postings are Moderated)\",\n  \"header:X-List-Single:all\": [\n    \" <mailto:x-moderator@host.com> (X-Postings are Moderated)\"\n  ],\n  \"header:X-List-Single:asURLs\": [\n    \"mailto:x-moderator@host.com\"\n  ],\n  \"header:X-List-Single:asURLs:all\": [\n    [\n      \"mailto:x-moderator@host.com\"\n    ]\n  ],\n  \"header:X-Text\": \" =?ISO-8859-1?Q?a?= =?ISO-8859-2?Q?_b?=\",\n  \"header:X-Text:all\": [\n    \" =?iso-8859-1?q?this=20is=20some=20text?=\",\n    \" =?ISO-8859-1?Q?a?= =?ISO-8859-2?Q?_b?=\"\n  ],\n  \"header:X-Text:asText\": \"a b\",\n  \"header:X-Text:asText:all\": [\n    \"this is some text\",\n    \"a b\"\n  ],\n  \"header:X-Text-Single\": \" =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=\\n              =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=\",\n  \"header:X-Text-Single:all\": [\n    \" =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=\\n              =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=\"\n  ],\n  \"header:X-Text-Single:asText\": \"If you can read this you understand the example.\",\n  \"header:X-Text-Single:asText:all\": [\n    \"If you can read this you understand the example.\"\n  ]\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/headers.eml",
    "content": "Bcc: \"=?utf-8?B?wqFFbCDDsWFuZMO6IGNvbWnDsyDDsW9xdWlzIQ==?=\"\r\n\t<addr1@example.com>\r\nCc: \"=?utf-8?B?0J/RgNC40LLQtdGCLCDQvNC40YA=?=\" <addr0@example.com>\r\nDate: Tue, 10 Jul 2018 01:03:11 +0000\r\nFrom: \"Joe Bloggs\" <joe@example.com>\r\nIn-Reply-To: <other-message-id> <yet-another-message-id>\r\nList-Owner: <http://www.host.com/list.cgi?cmd=sub&lst=list>,\r\n\t<mailto:list-manager@host.com?body=subscribe%20list>\r\nList-Subscribe: <ftp://ftp.host.com/list.txt>,\r\n\t<mailto:list@host.com?subject=subscribe>\r\nList-Subscribe: <mailto:list-manager@host.com?body=subscribe%20list>\r\nMessage-ID: <my-message-id>\r\nReferences: <first-message-id> <second-message-id>\r\nReply-To: \"=?utf-8?B?7JWI64WV7ZWY7IS47JqUIOyEuOqzhA==?=\" <addr2@example.com>, \r\n\t\"=?utf-8?Q?Antoine_de_Saint-Exup=C3=A9ry?=\" <addr3@example.com>\r\nResent-Date: Sat, 2 Jul 2005 09:52:37 +0000\r\nResent-Date: Sun, 3 Jul 2005 09:52:37 +0000\r\nResent-Date: Mon, 4 Jul 2005 09:52:37 +0000\r\nSender: \"=?utf-8?B?44OP44Ot44O844O744Ov44O844Or44OJ?=\" <joe@example.com>\r\nSubject: Headers test\r\nTo: \"Greg Vaudreuil\" <gvaudre@NRI.Reston.VA.US>, \r\n\t\"Ned Freed\" <ned@innosoft.com>, \"Keith Moore\" <moore@cs.utk.edu>\r\nX-AddressesGroup: \"A Group\": \"Ed Jones\" <c@a.test>, \r\n\t<joe@where.test>, \"John\" <jdoe@one.test>;\r\nX-AddressesGroup: \"List 1\": <addr1@test.com>, \r\n\t<addr2@test.com>; \"List 2\": <addr3@test.com>, \r\n\t<addr4@test.com>; <addr5@test.com>, \r\n\t<addr6@test.com>;\r\nX-References: <1234@local.machine.example> <3456@example.net>\r\nX-References: <789@local.machine.example> <abcd@example.net>\r\nX-Text: a b\r\nX-Text: this is some text\r\nMIME-Version: 1.0\r\nContent-Type: multipart/alternative; \r\n\tboundary=\"boundary_0\"\r\n\r\n\r\n--boundary_0\r\nContent-Language: en\r\nContent-Type: text/plain; charset=\"us-ascii\"\r\nX-Header: just a value\r\nX-Text: more text\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nI have the most brilliant plan.  Let me tell you all about it.  What we do i=\r\ns, we\r\n--boundary_0\r\nContent-Location: https://example.com/html-body.html\r\nContent-Type: text/html; charset=\"utf-8\"; name=\"html-body.html\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<!DOCTYPE html><html><head><title></title><style type=3D\"text/css\">div{font-=\r\nsize:16px}</style></head><body><div>I have the most <b>brilliant</b> plan.  =\r\nLet me tell you all about it.  What we do is, we</div></body></html>\r\n--boundary_0--\r\n"
  },
  {
    "path": "tests/resources/jmap/email_set/headers.jmap",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"$draft\": true,\n    \"$seen\": true\n  },\n  \"receivedAt\": \"2018-07-10T01:03:11Z\",\n  \"messageId\": [\n    \"my-message-id\"\n  ],\n  \"inReplyTo\": [\n    \"other-message-id\",\n    \"yet-another-message-id\"\n  ],\n  \"references\": [\n    \"first-message-id\",\n    \"second-message-id\"\n  ],\n  \"sender\": [\n    {\n      \"name\": \"ハロー・ワールド\",\n      \"email\": \"joe@example.com\"\n    }\n  ],\n  \"from\": [\n    {\n      \"name\": \"Joe Bloggs\",\n      \"email\": \"joe@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"Greg Vaudreuil\",\n      \"email\": \"gvaudre@NRI.Reston.VA.US\"\n    },\n    {\n      \"name\": \"Ned Freed\",\n      \"email\": \"ned@innosoft.com\"\n    },\n    {\n      \"name\": \"Keith Moore\",\n      \"email\": \"moore@cs.utk.edu\"\n    }\n  ],\n  \"cc\": [\n    {\n      \"name\": \"Привет, мир\",\n      \"email\": \"addr0@example.com\"\n    }\n  ],\n  \"bcc\": [\n    {\n      \"name\": \"¡El ñandú comió ñoquis!\",\n      \"email\": \"addr1@example.com\"\n    }\n  ],\n  \"replyTo\": [\n    {\n      \"name\": \"안녕하세요 세계\",\n      \"email\": \"addr2@example.com\"\n    },\n    {\n      \"name\": \"Antoine de Saint-Exupéry\",\n      \"email\": \"addr3@example.com\"\n    }\n  ],\n  \"subject\": \"Headers test\",\n  \"sentAt\": \"2018-07-10T01:03:11Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"Bcc\",\n        \"value\": \" \\\"=?utf-8?B?wqFFbCDDsWFuZMO6IGNvbWnDsyDDsW9xdWlzIQ==?=\\\"\\r\\n\\t<addr1@example.com>\"\n      },\n      {\n        \"name\": \"Cc\",\n        \"value\": \" \\\"=?utf-8?B?0J/RgNC40LLQtdGCLCDQvNC40YA=?=\\\" <addr0@example.com>\"\n      },\n      {\n        \"name\": \"Date\",\n        \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Joe Bloggs\\\" <joe@example.com>\"\n      },\n      {\n        \"name\": \"In-Reply-To\",\n        \"value\": \" <other-message-id> <yet-another-message-id>\"\n      },\n      {\n        \"name\": \"List-Owner\",\n        \"value\": \" <http://www.host.com/list.cgi?cmd=sub&lst=list>,\\r\\n\\t<mailto:list-manager@host.com?body=subscribe%20list>\"\n      },\n      {\n        \"name\": \"List-Subscribe\",\n        \"value\": \" <ftp://ftp.host.com/list.txt>,\\r\\n\\t<mailto:list@host.com?subject=subscribe>\"\n      },\n      {\n        \"name\": \"List-Subscribe\",\n        \"value\": \" <mailto:list-manager@host.com?body=subscribe%20list>\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <my-message-id>\"\n      },\n      {\n        \"name\": \"References\",\n        \"value\": \" <first-message-id> <second-message-id>\"\n      },\n      {\n        \"name\": \"Reply-To\",\n        \"value\": \" \\\"=?utf-8?B?7JWI64WV7ZWY7IS47JqUIOyEuOqzhA==?=\\\" <addr2@example.com>, \\r\\n\\t\\\"=?utf-8?Q?Antoine_de_Saint-Exup=C3=A9ry?=\\\" <addr3@example.com>\"\n      },\n      {\n        \"name\": \"Resent-Date\",\n        \"value\": \" Sat, 2 Jul 2005 09:52:37 +0000\"\n      },\n      {\n        \"name\": \"Resent-Date\",\n        \"value\": \" Sun, 3 Jul 2005 09:52:37 +0000\"\n      },\n      {\n        \"name\": \"Resent-Date\",\n        \"value\": \" Mon, 4 Jul 2005 09:52:37 +0000\"\n      },\n      {\n        \"name\": \"Sender\",\n        \"value\": \" \\\"=?utf-8?B?44OP44Ot44O844O744Ov44O844Or44OJ?=\\\" <joe@example.com>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" Headers test\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" \\\"Greg Vaudreuil\\\" <gvaudre@NRI.Reston.VA.US>, \\r\\n\\t\\\"Ned Freed\\\" <ned@innosoft.com>, \\\"Keith Moore\\\" <moore@cs.utk.edu>\"\n      },\n      {\n        \"name\": \"X-AddressesGroup\",\n        \"value\": \" \\\"A Group\\\": \\\"Ed Jones\\\" <c@a.test>, \\r\\n\\t<joe@where.test>, \\\"John\\\" <jdoe@one.test>;\"\n      },\n      {\n        \"name\": \"X-AddressesGroup\",\n        \"value\": \" \\\"List 1\\\": <addr1@test.com>, \\r\\n\\t<addr2@test.com>; \\\"List 2\\\": <addr3@test.com>, \\r\\n\\t<addr4@test.com>; <addr5@test.com>, \\r\\n\\t<addr6@test.com>;\"\n      },\n      {\n        \"name\": \"X-References\",\n        \"value\": \" <1234@local.machine.example> <3456@example.net>\"\n      },\n      {\n        \"name\": \"X-References\",\n        \"value\": \" <789@local.machine.example> <abcd@example.net>\"\n      },\n      {\n        \"name\": \"X-Text\",\n        \"value\": \" a b\"\n      },\n      {\n        \"name\": \"X-Text\",\n        \"value\": \" this is some text\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/alternative; \\r\\n\\tboundary=\\\"boundary_0\\\"\"\n      }\n    ],\n    \"type\": \"multipart/alternative\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 81,\n        \"headers\": [\n          {\n            \"name\": \"Content-Language\",\n            \"value\": \" en\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain; charset=\\\"us-ascii\\\"\"\n          },\n          {\n            \"name\": \"X-Header\",\n            \"value\": \" just a value\"\n          },\n          {\n            \"name\": \"X-Text\",\n            \"value\": \" more text\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" quoted-printable\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\",\n        \"language\": [\n          \"en\"\n        ]\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 218,\n        \"headers\": [\n          {\n            \"name\": \"Content-Location\",\n            \"value\": \" https://example.com/html-body.html\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/html; charset=\\\"utf-8\\\"; name=\\\"html-body.html\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" quoted-printable\"\n          }\n        ],\n        \"name\": \"html-body.html\",\n        \"type\": \"text/html\",\n        \"charset\": \"utf-8\",\n        \"location\": \"https://example.com/html-body.html\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"2\": {\n      \"value\": \"<!DOCTYPE html><html><head><title></title><style type=\\\"text/css\\\">div{font-size:16px}</style>...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 81,\n      \"headers\": [\n        {\n          \"name\": \"Content-Language\",\n          \"value\": \" en\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"us-ascii\\\"\"\n        },\n        {\n          \"name\": \"X-Header\",\n          \"value\": \" just a value\"\n        },\n        {\n          \"name\": \"X-Text\",\n          \"value\": \" more text\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"language\": [\n        \"en\"\n      ]\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 218,\n      \"headers\": [\n        {\n          \"name\": \"Content-Location\",\n          \"value\": \" https://example.com/html-body.html\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html; charset=\\\"utf-8\\\"; name=\\\"html-body.html\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"name\": \"html-body.html\",\n      \"type\": \"text/html\",\n      \"charset\": \"utf-8\",\n      \"location\": \"https://example.com/html-body.html\"\n    }\n  ],\n  \"attachments\": [],\n  \"hasAttachment\": false,\n  \"preview\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/headers.json",
    "content": "{\n    \"keywords\": {\n        \"$seen\": true,\n        \"$draft\": true\n    },\n    \"from\": [\n        {\n            \"name\": \"Joe Bloggs\",\n            \"email\": \"joe@example.com\"\n        }\n    ],\n    \"subject\": \"Headers test\",\n    \"receivedAt\": \"2018-07-10T01:03:11Z\",\n    \"sentAt\": \"2018-07-10T11:03:11+10:00\",\n    \"messageId\": [\n        \"my-message-id\"\n    ],\n    \"inReplyTo\": [\n        \"other-message-id\",\n        \"yet-another-message-id\"\n    ],\n    \"references\": [\n        \"first-message-id\",\n        \"second-message-id\"\n    ],\n    \"sender\": [\n        {\n            \"name\": \"ハロー・ワールド\",\n            \"email\": \"joe@example.com\"\n        }\n    ],\n    \"to\": [\n        {\n            \"email\": \"gvaudre@NRI.Reston.VA.US\",\n            \"name\": \"Greg Vaudreuil\"\n        },\n        {\n            \"email\": \"ned@innosoft.com\",\n            \"name\": \"Ned Freed\"\n        },\n        {\n            \"email\": \"moore@cs.utk.edu\",\n            \"name\": \"Keith Moore\"\n        }\n    ],\n    \"cc\": [\n        {\n            \"name\": \"Привет, мир\",\n            \"email\": \"addr0@example.com\"\n        }\n    ],\n    \"bcc\": [\n        {\n            \"name\": \"¡El ñandú comió ñoquis!\",\n            \"email\": \"addr1@example.com\"\n        }\n    ],\n    \"replyTo\": [\n        {\n            \"name\": \"안녕하세요 세계\",\n            \"email\": \"addr2@example.com\"\n        },\n        {\n            \"name\": \"Antoine de Saint-Exupéry\",\n            \"email\": \"addr3@example.com\"\n        }\n    ],\n    \"header:Resent-Date:asDate:all\": [\n        \"2005-07-02T11:52:37+02:00\",\n        \"2005-07-03T12:52:37+03:00\",\n        \"2005-07-04T13:52:37+04:00\"\n    ],\n    \"header:X-Text:asText:all\": [\n        \"this is some text\",\n        \"a b\"\n    ],\n    \"header:List-Owner:asURLs\": [\n        \"http://www.host.com/list.cgi?cmd=sub&lst=list\",\n        \"mailto:list-manager@host.com?body=subscribe%20list\"\n    ],\n    \"header:List-Subscribe:asURLs:all\": [\n        [\n            \"mailto:list-manager@host.com?body=subscribe%20list\"\n        ],\n        [\n            \"ftp://ftp.host.com/list.txt\",\n            \"mailto:list@host.com?subject=subscribe\"\n        ]\n    ],\n    \"header:X-References:asMessageIds:all\": [\n        [\n            \"1234@local.machine.example\",\n            \"3456@example.net\"\n        ],\n        [\n            \"789@local.machine.example\",\n            \"abcd@example.net\"\n        ]\n    ],\n    \"header:X-AddressesGroup:asGroupedAddresses:all\": [\n        [\n            {\n                \"addresses\": [\n                    {\n                        \"email\": \"c@a.test\",\n                        \"name\": \"Ed Jones\"\n                    },\n                    {\n                        \"email\": \"joe@where.test\",\n                        \"name\": null\n                    },\n                    {\n                        \"email\": \"jdoe@one.test\",\n                        \"name\": \"John\"\n                    }\n                ],\n                \"name\": \"A Group\"\n            }\n        ],\n        [\n            {\n                \"addresses\": [\n                    {\n                        \"email\": \"addr1@test.com\",\n                        \"name\": null\n                    },\n                    {\n                        \"email\": \"addr2@test.com\",\n                        \"name\": null\n                    }\n                ],\n                \"name\": \"List 1\"\n            },\n            {\n                \"addresses\": [\n                    {\n                        \"email\": \"addr3@test.com\",\n                        \"name\": null\n                    },\n                    {\n                        \"email\": \"addr4@test.com\",\n                        \"name\": null\n                    }\n                ],\n                \"name\": \"List 2\"\n            },\n            {\n                \"addresses\": [\n                    {\n                        \"email\": \"addr5@test.com\",\n                        \"name\": null\n                    },\n                    {\n                        \"email\": \"addr6@test.com\",\n                        \"name\": null\n                    }\n                ],\n                \"name\": null\n            }\n        ]\n    ],\n    \"textBody\": [\n        {\n            \"type\": \"text/plain\",\n            \"blobId\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\",\n            \"charset\": \"us-ascii\",\n            \"header:Content-Language\": \"en\",\n            \"header:X-Header\": \"just a value\",\n            \"header:X-Text\": \"more text\"\n        }\n    ],\n    \"htmlBody\": [\n        {\n            \"type\": \"text/html\",\n            \"partId\": \"a49d\",\n            \"location\": \"https://example.com/html-body.html\",\n            \"name\": \"html-body.html\"\n        }\n    ],\n    \"bodyValues\": {\n        \"a49d\": {\n            \"value\": \"<!DOCTYPE html><html><head><title></title><style type=\\\"text/css\\\">div{font-size:16px}</style></head><body><div>I have the most <b>brilliant</b> plan.  Let me tell you all about it.  What we do is, we</div></body></html>\",\n            \"isTruncated\": false\n        }\n    }\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/minimal.eml",
    "content": "Date: Tue, 10 Jul 2018 01:03:11 +0000\r\nMessage-ID: <my-message-id>\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\n\r\n"
  },
  {
    "path": "tests/resources/jmap/email_set/minimal.jmap",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {},\n  \"receivedAt\": \"2018-07-10T01:03:11Z\",\n  \"messageId\": [\n    \"my-message-id\"\n  ],\n  \"sentAt\": \"2018-07-10T01:03:11Z\",\n  \"bodyStructure\": {\n    \"partId\": \"0\",\n    \"blobId\": \"blob_0\",\n    \"size\": 2,\n    \"headers\": [\n      {\n        \"name\": \"Date\",\n        \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <my-message-id>\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n      },\n      {\n        \"name\": \"Content-Transfer-Encoding\",\n        \"value\": \" 7bit\"\n      }\n    ],\n    \"type\": \"text/plain\",\n    \"charset\": \"utf-8\"\n  },\n  \"bodyValues\": {\n    \"0\": {\n      \"value\": \"\\n\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"0\",\n      \"blobId\": \"blob_0\",\n      \"size\": 2,\n      \"headers\": [\n        {\n          \"name\": \"Date\",\n          \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n        },\n        {\n          \"name\": \"Message-ID\",\n          \"value\": \" <my-message-id>\"\n        },\n        {\n          \"name\": \"MIME-Version\",\n          \"value\": \" 1.0\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"utf-8\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"0\",\n      \"blobId\": \"blob_0\",\n      \"size\": 2,\n      \"headers\": [\n        {\n          \"name\": \"Date\",\n          \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n        },\n        {\n          \"name\": \"Message-ID\",\n          \"value\": \" <my-message-id>\"\n        },\n        {\n          \"name\": \"MIME-Version\",\n          \"value\": \" 1.0\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"utf-8\"\n    }\n  ],\n  \"attachments\": [],\n  \"hasAttachment\": false,\n  \"preview\": \"\\n\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/minimal.json",
    "content": "{\n    \"receivedAt\": \"2018-07-10T01:03:11Z\",\n    \"sentAt\": \"2018-07-10T11:03:11+10:00\",\n    \"messageId\": [\n        \"my-message-id\"\n    ]\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/mixed.eml",
    "content": "Date: Sat, 20 Nov 2021 22:22:01 +0000\r\nFrom: \"Art Vandelay (Vandelay Industries)\" <art@vandelay.com>\r\nMessage-ID: <my-message-id>\r\nSubject: =?utf-8?Q?Why_not_both_importing_AND_exporting=3F_=E2=98=BA?=\r\nTo: \"Colleagues\": \"James Smythe\" <james@vandelay.com>; \r\n\t\"Friends\": <jane@example.com>, \r\n\t\"=?utf-8?Q?John_Sm=C3=AEth?=\" <john@example.com>;\r\nMIME-Version: 1.0\r\nContent-Type: multipart/mixed; \r\n\tboundary=\"boundary_0\"\r\n\r\n\r\n--boundary_0\r\nContent-Type: multipart/alternative; \r\n\tboundary=\"boundary_1\"\r\n\r\n\r\n--boundary_1\r\nContent-Language: en\r\nContent-Type: text/plain\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nI was thinking about quitting the =E2=80=9Cexporting=E2=80=9D to focus just =\r\non the =E2=80=9Cimporting=E2=80=9D,\r\nbut then I thought, why not do both? =E2=98=BA\r\n\r\n--boundary_1\r\nContent-Language: en_US\r\nContent-Type: text/html\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<html><p>I was thinking about quitting the &ldquo;exporting&rdquo; to focus =\r\njust on the &ldquo;importing&rdquo;,</p><p>but then I thought, why not do bo=\r\nth? &#x263A;</p></html>\r\n--boundary_1--\r\n\r\n--boundary_0\r\nContent-ID: <cid:1234-5678-9012-3456>\r\nContent-Type: image/png\r\nContent-Transfer-Encoding: base64\r\n\r\naGVyZSBhcmUgdGhlIGVtYmVkZGVkIGltYWdlIGNvbnRlbnRzIQ==\r\n\r\n--boundary_0\r\nContent-Disposition: attachment; filename=\"=?utf-8?Q?Book_about_=E2=98=95_tables.pdf?=\"\r\nContent-Type: x-document/pdf\r\nContent-Transfer-Encoding: base64\r\n\r\nPGh0bWw+PGJvZHk+4oSM8J2UovCdlKnwnZStIPCdlKrwnZSiIPCdlKLwnZS18J2UrfCdlKzwnZSv\r\n8J2UsSDwnZSq8J2UtiDwnZSf8J2UrPCdlKzwnZSoIPCdlK3wnZSp8J2UovCdlJ7wnZSw8J2UoiE8\r\nL2JvZHk+PC9odG1sPg==\r\n\r\n--boundary_0--\r\n"
  },
  {
    "path": "tests/resources/jmap/email_set/mixed.jmap",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"$draft\": true,\n    \"$seen\": true,\n    \"my-tag\": true\n  },\n  \"receivedAt\": \"2021-11-20T22:22:01Z\",\n  \"messageId\": [\n    \"my-message-id\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Art Vandelay (Vandelay Industries)\",\n      \"email\": \"art@vandelay.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"James Smythe\",\n      \"email\": \"james@vandelay.com\"\n    },\n    {\n      \"name\": null,\n      \"email\": \"jane@example.com\"\n    },\n    {\n      \"name\": \"John Smîth\",\n      \"email\": \"john@example.com\"\n    }\n  ],\n  \"subject\": \"Why not both importing AND exporting? ☺\",\n  \"sentAt\": \"2021-11-20T22:22:01Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"Date\",\n        \"value\": \" Sat, 20 Nov 2021 22:22:01 +0000\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Art Vandelay (Vandelay Industries)\\\" <art@vandelay.com>\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <my-message-id>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" =?utf-8?Q?Why_not_both_importing_AND_exporting=3F_=E2=98=BA?=\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" \\\"Colleagues\\\": \\\"James Smythe\\\" <james@vandelay.com>; \\r\\n\\t\\\"Friends\\\": <jane@example.com>, \\r\\n\\t\\\"=?utf-8?Q?John_Sm=C3=AEth?=\\\" <john@example.com>;\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed; \\r\\n\\tboundary=\\\"boundary_0\\\"\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" multipart/alternative; \\r\\n\\tboundary=\\\"boundary_1\\\"\"\n          }\n        ],\n        \"type\": \"multipart/alternative\",\n        \"subParts\": [\n          {\n            \"partId\": \"2\",\n            \"blobId\": \"blob_0\",\n            \"size\": 131,\n            \"headers\": [\n              {\n                \"name\": \"Content-Language\",\n                \"value\": \" en\"\n              },\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" text/plain\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" quoted-printable\"\n              }\n            ],\n            \"type\": \"text/plain\",\n            \"charset\": \"us-ascii\",\n            \"language\": [\n              \"en\"\n            ]\n          },\n          {\n            \"partId\": \"3\",\n            \"blobId\": \"blob_1\",\n            \"size\": 175,\n            \"headers\": [\n              {\n                \"name\": \"Content-Language\",\n                \"value\": \" en_US\"\n              },\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" text/html\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" quoted-printable\"\n              }\n            ],\n            \"type\": \"text/html\",\n            \"charset\": \"us-ascii\",\n            \"language\": [\n              \"en_US\"\n            ]\n          }\n        ]\n      },\n      {\n        \"partId\": \"4\",\n        \"blobId\": \"blob_2\",\n        \"size\": 37,\n        \"headers\": [\n          {\n            \"name\": \"Content-ID\",\n            \"value\": \" <cid:1234-5678-9012-3456>\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" image/png\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          }\n        ],\n        \"type\": \"image/png\",\n        \"cid\": \"cid:1234-5678-9012-3456\"\n      },\n      {\n        \"partId\": \"5\",\n        \"blobId\": \"blob_3\",\n        \"size\": 127,\n        \"headers\": [\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" attachment; filename=\\\"=?utf-8?Q?Book_about_=E2=98=95_tables.pdf?=\\\"\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" x-document/pdf\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" base64\"\n          }\n        ],\n        \"name\": \"Book about ☕ tables.pdf\",\n        \"type\": \"x-document/pdf\",\n        \"disposition\": \"attachment\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"2\": {\n      \"value\": \"I was thinking about quitting the “exporting” to focus just on the “importing”,\\nbut then ...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    },\n    \"3\": {\n      \"value\": \"<html><p>I was thinking about quitting the &ldquo;exporting&rdquo; to focus just on the &ldquo;im...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_0\",\n      \"size\": 131,\n      \"headers\": [\n        {\n          \"name\": \"Content-Language\",\n          \"value\": \" en\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"language\": [\n        \"en\"\n      ]\n    },\n    {\n      \"partId\": \"4\",\n      \"blobId\": \"blob_2\",\n      \"size\": 37,\n      \"headers\": [\n        {\n          \"name\": \"Content-ID\",\n          \"value\": \" <cid:1234-5678-9012-3456>\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/png\",\n      \"cid\": \"cid:1234-5678-9012-3456\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"3\",\n      \"blobId\": \"blob_1\",\n      \"size\": 175,\n      \"headers\": [\n        {\n          \"name\": \"Content-Language\",\n          \"value\": \" en_US\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"us-ascii\",\n      \"language\": [\n        \"en_US\"\n      ]\n    },\n    {\n      \"partId\": \"4\",\n      \"blobId\": \"blob_2\",\n      \"size\": 37,\n      \"headers\": [\n        {\n          \"name\": \"Content-ID\",\n          \"value\": \" <cid:1234-5678-9012-3456>\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/png\",\n      \"cid\": \"cid:1234-5678-9012-3456\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"4\",\n      \"blobId\": \"blob_2\",\n      \"size\": 37,\n      \"headers\": [\n        {\n          \"name\": \"Content-ID\",\n          \"value\": \" <cid:1234-5678-9012-3456>\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/png\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/png\",\n      \"cid\": \"cid:1234-5678-9012-3456\"\n    },\n    {\n      \"partId\": \"5\",\n      \"blobId\": \"blob_3\",\n      \"size\": 127,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment; filename=\\\"=?utf-8?Q?Book_about_=E2=98=95_tables.pdf?=\\\"\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" x-document/pdf\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"name\": \"Book about ☕ tables.pdf\",\n      \"type\": \"x-document/pdf\",\n      \"disposition\": \"attachment\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"I was thinking about quitting the “exporting” to focus just on the “importing”,\\nbut then I thought, why not do both? ☺\\n\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/mixed.json",
    "content": "{\n    \"keywords\": {\n        \"$seen\": true,\n        \"$draft\": true,\n        \"my-tag\": true,\n        \"ignore-me\": false\n    },\n    \"from\": [\n        {\n            \"name\": \"Art Vandelay (Vandelay Industries)\",\n            \"email\": \"art@vandelay.com\"\n        }\n    ],\n    \"header:To:asGroupedAddresses\": [\n        {\n            \"name\": \"Colleagues\",\n            \"addresses\": [\n                {\n                    \"name\": \"James Smythe\",\n                    \"email\": \"james@vandelay.com\"\n                }\n            ]\n        },\n        {\n            \"name\": \"Friends\",\n            \"addresses\": [\n                {\n                    \"email\": \"jane@example.com\"\n                },\n                {\n                    \"name\": \"John Smîth\",\n                    \"email\": \"john@example.com\"\n                }\n            ]\n        }\n    ],\n    \"subject\": \"Why not both importing AND exporting? ☺\",\n    \"receivedAt\": \"2021-11-20T14:22:01-08:00\",\n    \"sentAt\": \"2021-11-20T14:22:01-08:00\",\n    \"messageId\": [\n        \"my-message-id\"\n    ],\n    \"textBody\": [\n        {\n            \"type\": \"text/plain\",\n            \"blobId\": \"I was thinking about quitting the “exporting” to focus just on the “importing”,\\nbut then I thought, why not do both? ☺\\n\",\n            \"header:Content-Language\": \"en\"\n        }\n    ],\n    \"htmlBody\": [\n        {\n            \"type\": \"text/html\",\n            \"blobId\": \"<html><p>I was thinking about quitting the &ldquo;exporting&rdquo; to focus just on the &ldquo;importing&rdquo;,</p><p>but then I thought, why not do both? &#x263A;</p></html>\",\n            \"header:Content-Language\": \"en_US\"\n        }\n    ],\n    \"attachments\": [\n        {\n            \"type\": \"image/png\",\n            \"blobId\": \"here are the embedded image contents!\",\n            \"cid\": \"cid:1234-5678-9012-3456\"\n        },\n        {\n            \"type\": \"x-document/pdf\",\n            \"blobId\": \"<html><body>ℌ𝔢𝔩𝔭 𝔪𝔢 𝔢𝔵𝔭𝔬𝔯𝔱 𝔪𝔶 𝔟𝔬𝔬𝔨 𝔭𝔩𝔢𝔞𝔰𝔢!</body></html>\",\n            \"disposition\": \"attachment\",\n            \"name\": \"Book about ☕ tables.pdf\"\n        }\n    ]\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/nested_body.eml",
    "content": "Date: Tue, 10 Jul 2018 01:03:11 +0000\r\nFrom: \"Joe Bloggs\" <joe@example.com>\r\nMessage-ID: <my-message-id>\r\nSubject: RFC 8621 Section 4.1.4 test\r\nMIME-Version: 1.0\r\nContent-Type: multipart/mixed; \r\n\tboundary=\"boundary_0\"\r\n\r\n\r\n--boundary_0\r\nContent-Disposition: inline\r\nContent-Type: text/plain\r\nContent-Transfer-Encoding: 7bit\r\n\r\nPart A\r\n--boundary_0\r\nContent-Type: multipart/mixed; \r\n\tboundary=\"boundary_1\"\r\n\r\n\r\n--boundary_1\r\nContent-Type: multipart/alternative; \r\n\tboundary=\"boundary_2\"\r\n\r\n\r\n--boundary_2\r\nContent-Type: multipart/mixed; \r\n\tboundary=\"boundary_3\"\r\n\r\n\r\n--boundary_3\r\nContent-Disposition: inline\r\nContent-Type: text/plain\r\nContent-Transfer-Encoding: 7bit\r\n\r\nPart B\r\n--boundary_3\r\nContent-Disposition: inline\r\nContent-Type: image/jpeg\r\nContent-Transfer-Encoding: base64\r\n\r\nUGFydCBD\r\n\r\n--boundary_3\r\nContent-Disposition: inline\r\nContent-Type: text/plain\r\nContent-Transfer-Encoding: 7bit\r\n\r\nPart D\r\n--boundary_3--\r\n\r\n--boundary_2\r\nContent-Type: multipart/related; \r\n\tboundary=\"boundary_4\"\r\n\r\n\r\n--boundary_4\r\nContent-Type: text/html\r\nContent-Transfer-Encoding: 7bit\r\n\r\nPart E\r\n--boundary_4\r\nContent-Type: image/jpeg\r\nContent-Transfer-Encoding: base64\r\n\r\nUGFydCBG\r\n\r\n--boundary_4--\r\n\r\n--boundary_2--\r\n\r\n--boundary_1\r\nContent-Disposition: attachment\r\nContent-Type: image/jpeg\r\nContent-Transfer-Encoding: base64\r\n\r\nUGFydCBH\r\n\r\n--boundary_1\r\nContent-Type: application/x-excel\r\nContent-Transfer-Encoding: base64\r\n\r\nUGFydCBI\r\n\r\n--boundary_1\r\nContent-Type: x-message/rfc822\r\nContent-Transfer-Encoding: base64\r\n\r\nUGFydCBK\r\n\r\n--boundary_1--\r\n\r\n--boundary_0\r\nContent-Disposition: inline\r\nContent-Type: text/plain\r\nContent-Transfer-Encoding: 7bit\r\n\r\nPart K\r\n--boundary_0--\r\n"
  },
  {
    "path": "tests/resources/jmap/email_set/nested_body.jmap",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"$draft\": true,\n    \"$seen\": true\n  },\n  \"receivedAt\": \"2018-07-10T01:03:11Z\",\n  \"messageId\": [\n    \"my-message-id\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Joe Bloggs\",\n      \"email\": \"joe@example.com\"\n    }\n  ],\n  \"subject\": \"RFC 8621 Section 4.1.4 test\",\n  \"sentAt\": \"2018-07-10T01:03:11Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"Date\",\n        \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Joe Bloggs\\\" <joe@example.com>\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <my-message-id>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" RFC 8621 Section 4.1.4 test\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/mixed; \\r\\n\\tboundary=\\\"boundary_0\\\"\"\n      }\n    ],\n    \"type\": \"multipart/mixed\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 6,\n        \"headers\": [\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" 7bit\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\",\n        \"disposition\": \"inline\"\n      },\n      {\n        \"headers\": [\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" multipart/mixed; \\r\\n\\tboundary=\\\"boundary_1\\\"\"\n          }\n        ],\n        \"type\": \"multipart/mixed\",\n        \"subParts\": [\n          {\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" multipart/alternative; \\r\\n\\tboundary=\\\"boundary_2\\\"\"\n              }\n            ],\n            \"type\": \"multipart/alternative\",\n            \"subParts\": [\n              {\n                \"headers\": [\n                  {\n                    \"name\": \"Content-Type\",\n                    \"value\": \" multipart/mixed; \\r\\n\\tboundary=\\\"boundary_3\\\"\"\n                  }\n                ],\n                \"type\": \"multipart/mixed\",\n                \"subParts\": [\n                  {\n                    \"partId\": \"5\",\n                    \"blobId\": \"blob_1\",\n                    \"size\": 6,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Disposition\",\n                        \"value\": \" inline\"\n                      },\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" text/plain\"\n                      },\n                      {\n                        \"name\": \"Content-Transfer-Encoding\",\n                        \"value\": \" 7bit\"\n                      }\n                    ],\n                    \"type\": \"text/plain\",\n                    \"charset\": \"us-ascii\",\n                    \"disposition\": \"inline\"\n                  },\n                  {\n                    \"partId\": \"6\",\n                    \"blobId\": \"blob_2\",\n                    \"size\": 6,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Disposition\",\n                        \"value\": \" inline\"\n                      },\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" image/jpeg\"\n                      },\n                      {\n                        \"name\": \"Content-Transfer-Encoding\",\n                        \"value\": \" base64\"\n                      }\n                    ],\n                    \"type\": \"image/jpeg\",\n                    \"disposition\": \"inline\"\n                  },\n                  {\n                    \"partId\": \"7\",\n                    \"blobId\": \"blob_3\",\n                    \"size\": 6,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Disposition\",\n                        \"value\": \" inline\"\n                      },\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" text/plain\"\n                      },\n                      {\n                        \"name\": \"Content-Transfer-Encoding\",\n                        \"value\": \" 7bit\"\n                      }\n                    ],\n                    \"type\": \"text/plain\",\n                    \"charset\": \"us-ascii\",\n                    \"disposition\": \"inline\"\n                  }\n                ]\n              },\n              {\n                \"headers\": [\n                  {\n                    \"name\": \"Content-Type\",\n                    \"value\": \" multipart/related; \\r\\n\\tboundary=\\\"boundary_4\\\"\"\n                  }\n                ],\n                \"type\": \"multipart/related\",\n                \"subParts\": [\n                  {\n                    \"partId\": \"9\",\n                    \"blobId\": \"blob_4\",\n                    \"size\": 6,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" text/html\"\n                      },\n                      {\n                        \"name\": \"Content-Transfer-Encoding\",\n                        \"value\": \" 7bit\"\n                      }\n                    ],\n                    \"type\": \"text/html\",\n                    \"charset\": \"us-ascii\"\n                  },\n                  {\n                    \"partId\": \"10\",\n                    \"blobId\": \"blob_5\",\n                    \"size\": 6,\n                    \"headers\": [\n                      {\n                        \"name\": \"Content-Type\",\n                        \"value\": \" image/jpeg\"\n                      },\n                      {\n                        \"name\": \"Content-Transfer-Encoding\",\n                        \"value\": \" base64\"\n                      }\n                    ],\n                    \"type\": \"image/jpeg\"\n                  }\n                ]\n              }\n            ]\n          },\n          {\n            \"partId\": \"11\",\n            \"blobId\": \"blob_6\",\n            \"size\": 6,\n            \"headers\": [\n              {\n                \"name\": \"Content-Disposition\",\n                \"value\": \" attachment\"\n              },\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" image/jpeg\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" base64\"\n              }\n            ],\n            \"type\": \"image/jpeg\",\n            \"disposition\": \"attachment\"\n          },\n          {\n            \"partId\": \"12\",\n            \"blobId\": \"blob_7\",\n            \"size\": 6,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" application/x-excel\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" base64\"\n              }\n            ],\n            \"type\": \"application/x-excel\"\n          },\n          {\n            \"partId\": \"13\",\n            \"blobId\": \"blob_8\",\n            \"size\": 6,\n            \"headers\": [\n              {\n                \"name\": \"Content-Type\",\n                \"value\": \" x-message/rfc822\"\n              },\n              {\n                \"name\": \"Content-Transfer-Encoding\",\n                \"value\": \" base64\"\n              }\n            ],\n            \"type\": \"x-message/rfc822\"\n          }\n        ]\n      },\n      {\n        \"partId\": \"14\",\n        \"blobId\": \"blob_9\",\n        \"size\": 6,\n        \"headers\": [\n          {\n            \"name\": \"Content-Disposition\",\n            \"value\": \" inline\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" 7bit\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"us-ascii\",\n        \"disposition\": \"inline\"\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"Part A\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"14\": {\n      \"value\": \"Part K\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"5\": {\n      \"value\": \"Part B\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"7\": {\n      \"value\": \"Part D\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    },\n    \"9\": {\n      \"value\": \"Part E\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"5\",\n      \"blobId\": \"blob_1\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"6\",\n      \"blobId\": \"blob_2\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/jpeg\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"7\",\n      \"blobId\": \"blob_3\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"14\",\n      \"blobId\": \"blob_9\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"9\",\n      \"blobId\": \"blob_4\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"us-ascii\"\n    },\n    {\n      \"partId\": \"14\",\n      \"blobId\": \"blob_9\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" 7bit\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"us-ascii\",\n      \"disposition\": \"inline\"\n    }\n  ],\n  \"attachments\": [\n    {\n      \"partId\": \"6\",\n      \"blobId\": \"blob_2\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" inline\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/jpeg\",\n      \"disposition\": \"inline\"\n    },\n    {\n      \"partId\": \"10\",\n      \"blobId\": \"blob_5\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/jpeg\"\n    },\n    {\n      \"partId\": \"11\",\n      \"blobId\": \"blob_6\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Disposition\",\n          \"value\": \" attachment\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" image/jpeg\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"image/jpeg\",\n      \"disposition\": \"attachment\"\n    },\n    {\n      \"partId\": \"12\",\n      \"blobId\": \"blob_7\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" application/x-excel\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"application/x-excel\"\n    },\n    {\n      \"partId\": \"13\",\n      \"blobId\": \"blob_8\",\n      \"size\": 6,\n      \"headers\": [\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" x-message/rfc822\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" base64\"\n        }\n      ],\n      \"type\": \"x-message/rfc822\"\n    }\n  ],\n  \"hasAttachment\": true,\n  \"preview\": \"Part A\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/nested_body.json",
    "content": "{\n    \"keywords\": {\n        \"$seen\": true,\n        \"$draft\": true\n    },\n    \"from\": [\n        {\n            \"name\": \"Joe Bloggs\",\n            \"email\": \"joe@example.com\"\n        }\n    ],\n    \"subject\": \"RFC 8621 Section 4.1.4 test\",\n    \"receivedAt\": \"2018-07-10T01:03:11Z\",\n    \"sentAt\": \"2018-07-10T11:03:11+10:00\",\n    \"messageId\": [\n        \"my-message-id\"\n    ],\n    \"bodyStructure\": {\n        \"subParts\": [\n            {\n                \"blobId\": \"Part A\",\n                \"disposition\": \"inline\",\n                \"type\": \"text/plain\"\n            },\n            {\n                \"subParts\": [\n                    {\n                        \"subParts\": [\n                            {\n                                \"subParts\": [\n                                    {\n                                        \"blobId\": \"Part B\",\n                                        \"disposition\": \"inline\",\n                                        \"type\": \"text/plain\"\n                                    },\n                                    {\n                                        \"blobId\": \"Part C\",\n                                        \"disposition\": \"inline\",\n                                        \"type\": \"image/jpeg\"\n                                    },\n                                    {\n                                        \"blobId\": \"Part D\",\n                                        \"disposition\": \"inline\",\n                                        \"type\": \"text/plain\"\n                                    }\n                                ],\n                                \"type\": \"multipart/mixed\"\n                            },\n                            {\n                                \"subParts\": [\n                                    {\n                                        \"blobId\": \"Part E\",\n                                        \"type\": \"text/html\"\n                                    },\n                                    {\n                                        \"blobId\": \"Part F\",\n                                        \"type\": \"image/jpeg\"\n                                    }\n                                ],\n                                \"type\": \"multipart/related\"\n                            }\n                        ],\n                        \"type\": \"multipart/alternative\"\n                    },\n                    {\n                        \"blobId\": \"Part G\",\n                        \"disposition\": \"attachment\",\n                        \"type\": \"image/jpeg\"\n                    },\n                    {\n                        \"blobId\": \"Part H\",\n                        \"type\": \"application/x-excel\"\n                    },\n                    {\n                        \"blobId\": \"Part J\",\n                        \"type\": \"x-message/rfc822\"\n                    }\n                ],\n                \"type\": \"multipart/mixed\"\n            },\n            {\n                \"blobId\": \"Part K\",\n                \"disposition\": \"inline\",\n                \"type\": \"text/plain\"\n            }\n        ],\n        \"type\": \"multipart/mixed\"\n    }\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/rfc8621_1.eml",
    "content": "Date: Tue, 10 Jul 2018 01:03:11 +0000\r\nFrom: \"Joe Bloggs\" <joe@example.com>\r\nMessage-ID: <my-message-id>\r\nSubject: World domination\r\nMIME-Version: 1.0\r\nContent-Language: en\r\nContent-Type: text/plain; charset=\"utf-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nI have the most brilliant plan.  Let me tell you all about it.  What we do i=\r\ns, we"
  },
  {
    "path": "tests/resources/jmap/email_set/rfc8621_1.jmap",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"$draft\": true,\n    \"$seen\": true\n  },\n  \"receivedAt\": \"2018-07-10T01:03:11Z\",\n  \"messageId\": [\n    \"my-message-id\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Joe Bloggs\",\n      \"email\": \"joe@example.com\"\n    }\n  ],\n  \"subject\": \"World domination\",\n  \"sentAt\": \"2018-07-10T01:03:11Z\",\n  \"bodyStructure\": {\n    \"partId\": \"0\",\n    \"blobId\": \"blob_0\",\n    \"size\": 81,\n    \"headers\": [\n      {\n        \"name\": \"Date\",\n        \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Joe Bloggs\\\" <joe@example.com>\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <my-message-id>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" World domination\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Language\",\n        \"value\": \" en\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n      },\n      {\n        \"name\": \"Content-Transfer-Encoding\",\n        \"value\": \" quoted-printable\"\n      }\n    ],\n    \"type\": \"text/plain\",\n    \"charset\": \"utf-8\",\n    \"language\": [\n      \"en\"\n    ]\n  },\n  \"bodyValues\": {\n    \"0\": {\n      \"value\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"0\",\n      \"blobId\": \"blob_0\",\n      \"size\": 81,\n      \"headers\": [\n        {\n          \"name\": \"Date\",\n          \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n        },\n        {\n          \"name\": \"From\",\n          \"value\": \" \\\"Joe Bloggs\\\" <joe@example.com>\"\n        },\n        {\n          \"name\": \"Message-ID\",\n          \"value\": \" <my-message-id>\"\n        },\n        {\n          \"name\": \"Subject\",\n          \"value\": \" World domination\"\n        },\n        {\n          \"name\": \"MIME-Version\",\n          \"value\": \" 1.0\"\n        },\n        {\n          \"name\": \"Content-Language\",\n          \"value\": \" en\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"utf-8\",\n      \"language\": [\n        \"en\"\n      ]\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"0\",\n      \"blobId\": \"blob_0\",\n      \"size\": 81,\n      \"headers\": [\n        {\n          \"name\": \"Date\",\n          \"value\": \" Tue, 10 Jul 2018 01:03:11 +0000\"\n        },\n        {\n          \"name\": \"From\",\n          \"value\": \" \\\"Joe Bloggs\\\" <joe@example.com>\"\n        },\n        {\n          \"name\": \"Message-ID\",\n          \"value\": \" <my-message-id>\"\n        },\n        {\n          \"name\": \"Subject\",\n          \"value\": \" World domination\"\n        },\n        {\n          \"name\": \"MIME-Version\",\n          \"value\": \" 1.0\"\n        },\n        {\n          \"name\": \"Content-Language\",\n          \"value\": \" en\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"utf-8\",\n      \"language\": [\n        \"en\"\n      ]\n    }\n  ],\n  \"attachments\": [],\n  \"hasAttachment\": false,\n  \"preview\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/rfc8621_1.json",
    "content": "{\n    \"keywords\": {\n        \"$seen\": true,\n        \"$draft\": true\n    },\n    \"from\": [\n        {\n            \"name\": \"Joe Bloggs\",\n            \"email\": \"joe@example.com\"\n        }\n    ],\n    \"subject\": \"World domination\",\n    \"receivedAt\": \"2018-07-10T01:03:11Z\",\n    \"sentAt\": \"2018-07-10T11:03:11+10:00\",\n    \"messageId\": [\n        \"my-message-id\"\n    ],\n    \"bodyStructure\": {\n        \"type\": \"text/plain\",\n        \"partId\": \"bd48\",\n        \"header:Content-Language\": \"en\"\n    },\n    \"bodyValues\": {\n        \"bd48\": {\n            \"value\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\",\n            \"isTruncated\": false\n        }\n    }\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/rfc8621_2.eml",
    "content": "Date: Tue, 10 Jul 2018 01:05:08 +0000\r\nFrom: \"Joe Bloggs\" <joe@example.com>\r\nMessage-ID: <my-message-id>\r\nSubject: World domination\r\nTo: \"John\" <john@example.com>\r\nMIME-Version: 1.0\r\nContent-Type: multipart/alternative; \r\n\tboundary=\"boundary_0\"\r\n\r\n\r\n--boundary_0\r\nContent-Language: en\r\nContent-Type: text/html; charset=\"utf-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<!DOCTYPE html><html><head><title></title><style type=3D\"text/css\">div{font-=\r\nsize:16px}</style></head><body><div>I have the most <b>brilliant</b> plan.  =\r\nLet me tell you all about it.  What we do is, we</div></body></html>\r\n--boundary_0\r\nContent-Language: en\r\nContent-Type: text/plain; charset=\"utf-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nI have the most brilliant plan.  Let me tell you all about it.  What we do i=\r\ns, we\r\n--boundary_0--\r\n"
  },
  {
    "path": "tests/resources/jmap/email_set/rfc8621_2.jmap",
    "content": "{\n  \"mailboxIds\": {\n    \"a\": true\n  },\n  \"keywords\": {\n    \"$draft\": true,\n    \"$seen\": true\n  },\n  \"receivedAt\": \"2018-07-10T01:05:08Z\",\n  \"messageId\": [\n    \"my-message-id\"\n  ],\n  \"from\": [\n    {\n      \"name\": \"Joe Bloggs\",\n      \"email\": \"joe@example.com\"\n    }\n  ],\n  \"to\": [\n    {\n      \"name\": \"John\",\n      \"email\": \"john@example.com\"\n    }\n  ],\n  \"subject\": \"World domination\",\n  \"sentAt\": \"2018-07-10T01:05:08Z\",\n  \"bodyStructure\": {\n    \"headers\": [\n      {\n        \"name\": \"Date\",\n        \"value\": \" Tue, 10 Jul 2018 01:05:08 +0000\"\n      },\n      {\n        \"name\": \"From\",\n        \"value\": \" \\\"Joe Bloggs\\\" <joe@example.com>\"\n      },\n      {\n        \"name\": \"Message-ID\",\n        \"value\": \" <my-message-id>\"\n      },\n      {\n        \"name\": \"Subject\",\n        \"value\": \" World domination\"\n      },\n      {\n        \"name\": \"To\",\n        \"value\": \" \\\"John\\\" <john@example.com>\"\n      },\n      {\n        \"name\": \"MIME-Version\",\n        \"value\": \" 1.0\"\n      },\n      {\n        \"name\": \"Content-Type\",\n        \"value\": \" multipart/alternative; \\r\\n\\tboundary=\\\"boundary_0\\\"\"\n      }\n    ],\n    \"type\": \"multipart/alternative\",\n    \"subParts\": [\n      {\n        \"partId\": \"1\",\n        \"blobId\": \"blob_0\",\n        \"size\": 218,\n        \"headers\": [\n          {\n            \"name\": \"Content-Language\",\n            \"value\": \" en\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/html; charset=\\\"utf-8\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" quoted-printable\"\n          }\n        ],\n        \"type\": \"text/html\",\n        \"charset\": \"utf-8\",\n        \"language\": [\n          \"en\"\n        ]\n      },\n      {\n        \"partId\": \"2\",\n        \"blobId\": \"blob_1\",\n        \"size\": 81,\n        \"headers\": [\n          {\n            \"name\": \"Content-Language\",\n            \"value\": \" en\"\n          },\n          {\n            \"name\": \"Content-Type\",\n            \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n          },\n          {\n            \"name\": \"Content-Transfer-Encoding\",\n            \"value\": \" quoted-printable\"\n          }\n        ],\n        \"type\": \"text/plain\",\n        \"charset\": \"utf-8\",\n        \"language\": [\n          \"en\"\n        ]\n      }\n    ]\n  },\n  \"bodyValues\": {\n    \"1\": {\n      \"value\": \"<!DOCTYPE html><html><head><title></title><style type=\\\"text/css\\\">div{font-size:16px}</style>...\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": true\n    },\n    \"2\": {\n      \"value\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\",\n      \"isEncodingProblem\": false,\n      \"isTruncated\": false\n    }\n  },\n  \"textBody\": [\n    {\n      \"partId\": \"2\",\n      \"blobId\": \"blob_1\",\n      \"size\": 81,\n      \"headers\": [\n        {\n          \"name\": \"Content-Language\",\n          \"value\": \" en\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/plain; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/plain\",\n      \"charset\": \"utf-8\",\n      \"language\": [\n        \"en\"\n      ]\n    }\n  ],\n  \"htmlBody\": [\n    {\n      \"partId\": \"1\",\n      \"blobId\": \"blob_0\",\n      \"size\": 218,\n      \"headers\": [\n        {\n          \"name\": \"Content-Language\",\n          \"value\": \" en\"\n        },\n        {\n          \"name\": \"Content-Type\",\n          \"value\": \" text/html; charset=\\\"utf-8\\\"\"\n        },\n        {\n          \"name\": \"Content-Transfer-Encoding\",\n          \"value\": \" quoted-printable\"\n        }\n      ],\n      \"type\": \"text/html\",\n      \"charset\": \"utf-8\",\n      \"language\": [\n        \"en\"\n      ]\n    }\n  ],\n  \"attachments\": [],\n  \"hasAttachment\": false,\n  \"preview\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\"\n}"
  },
  {
    "path": "tests/resources/jmap/email_set/rfc8621_2.json",
    "content": "{\n    \"keywords\": {\n        \"$seen\": true,\n        \"$draft\": true\n    },\n    \"from\": [\n        {\n            \"name\": \"Joe Bloggs\",\n            \"email\": \"joe@example.com\"\n        }\n    ],\n    \"to\": [\n        {\n            \"name\": \"John\",\n            \"email\": \"john@example.com\"\n        }\n    ],\n    \"messageId\": [\n        \"my-message-id\"\n    ],\n    \"subject\": \"World domination\",\n    \"receivedAt\": \"2018-07-10T01:05:08Z\",\n    \"sentAt\": \"2018-07-10T11:05:08+10:00\",\n    \"bodyStructure\": {\n        \"type\": \"multipart/alternative\",\n        \"subParts\": [\n            {\n                \"partId\": \"a49d\",\n                \"type\": \"text/html\",\n                \"header:Content-Language\": \"en\"\n            },\n            {\n                \"partId\": \"bd48\",\n                \"type\": \"text/plain\",\n                \"header:Content-Language\": \"en\"\n            }\n        ]\n    },\n    \"bodyValues\": {\n        \"bd48\": {\n            \"value\": \"I have the most brilliant plan.  Let me tell you all about it.  What we do is, we\",\n            \"isTruncated\": false\n        },\n        \"a49d\": {\n            \"value\": \"<!DOCTYPE html><html><head><title></title><style type=\\\"text/css\\\">div{font-size:16px}</style></head><body><div>I have the most <b>brilliant</b> plan.  Let me tell you all about it.  What we do is, we</div></body></html>\",\n            \"isTruncated\": false\n        }\n    }\n}"
  },
  {
    "path": "tests/resources/jmap/email_snippet/html.eml",
    "content": "Message-Id: <4.2.0.58.20000519003052.00a89c40@pop.example.com>\nX-Sender: dwsauder@pop.example.com (Unverified)\nX-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.58 \nX-Priority: 2 (High)\nDate: Fri, 19 May 2000 00:31:00 -0400\nTo: Heinz =?iso-8859-1?Q?M=FCller?= <mueller@example.com>\nFrom: Doug Sauder <dwsauder@example.com>\nSubject: =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche?=\nMime-Version: 1.0\nContent-Language: de\nContent-Type: text/html; charset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\n<html>\n<font face=3D\"Arial, Helvetica\" size=3D5 color=3D\"#0000FF\"><b>Die Hasen und =\ndie\nFr=F6sche<br>\n<br>\n</font></b><font face=3D\"Arial, Helvetica\">Die Hasen klagten einst =FCber\nihre mi=DFliche Lage; &quot;wir leben&quot;, sprach ein Redner, &quot;in\nsteter Furcht vor Menschen und Tieren, eine Beute der Hunde, der Adler,\nja fast aller Raubtiere! Unsere stete Angst ist =E4rger als der Tod selbst.\nAuf, la=DFt uns ein f=FCr allemal sterben.&quot; <br>\n<br>\nIn einem nahen Teich wollten sie sich nun ers=E4ufen; sie eilten ihm zu;\nallein das au=DFerordentliche Get=F6se und ihre wunderbare Gestalt\nerschreckte eine Menge Fr=F6sche, die am Ufer sa=DFen, so sehr, da=DF sie au=\nfs\nschnellste untertauchten. <br>\n<br>\n&quot;Halt&quot;, rief nun eben dieser Sprecher, &quot;wir wollen das\nErs=E4ufen noch ein wenig aufschieben, denn auch uns f=FCrchten, wie ihr\nseht, einige Tiere, welche also wohl noch ungl=FCcklicher sein m=FCssen als\nwir.&quot; <br>\n<br>\n</font></html>\n"
  },
  {
    "path": "tests/resources/jmap/email_snippet/mixed.eml",
    "content": "MIME-Version: 1.0\nDate: Sat, 13 Aug 2021 15:51:01 +0200\nMessage-ID: <CAGxp_yirwLM7DD1xMGvW1qag_rhgw97M7VyxZKwm15AjXVsvxA@mail.gmail.com>\nSubject: Biblioteca de Babel\nFrom: Jorge Luis Borges <jorge@borges.org>\nTo: Julio Cortázar <julio@cortazar.org>\nContent-Language: es\nContent-Type: multipart/alternative; boundary=\"0000000000006adf7205e61fb0a1\"\n\n--0000000000006adf7205e61fb0a1\nContent-Type: text/plain; charset=\"UTF-8\"\nContent-Transfer-Encoding: quoted-printable\n\nEl universo (que otros llaman la *Biblioteca*) se compone de un n=C3=BAmero\nindefinido, y\ntal vez infinito, de galer=C3=ADas hexagonales, con vastos pozos de ventila=\nci=C3=B3n\nen el medio,\ncercados por barandas baj=C3=ADsimas. Desde cualquier hex=C3=A1gono se ven =\nlos pisos\ninferiores y\nsuperiores: interminablemente. La distribuci=C3=B3n de las galer=C3=ADas es\ninvariable. Veinte\nanaqueles, a cinco largos anaqueles por lado, cubren todos los lados menos\ndos; su altura,\nque es la de los pisos, excede apenas la de un bibliotecario normal. Una de\nlas caras libres\nda a un angosto zagu=C3=A1n, que desemboca en otra galer=C3=ADa, id=C3=A9nt=\nica a la\nprimera y a todas. A\nizquierda y a derecha del zagu=C3=A1n hay dos gabinetes min=C3=BAsculos. Un=\no permite\ndormir de\npie; otro, satisfacer las necesidades finales. Por ah=C3=AD pasa la escaler=\na\nespiral, que se abisma\ny se eleva hacia lo remoto. En el zagu=C3=A1n hay un espejo, que fielmente\nduplica las\napariencias. Los hombres suelen inferir de ese espejo que la *Biblioteca*\nno es infinita (si\nlo fuera realmente =C2=BFa qu=C3=A9 esa duplicaci=C3=B3n ilusoria?); yo pre=\nfiero so=C3=B1ar que\nlas superficies\nbru=C3=B1idas figuran y prometen el infinito... La luz procede de unas frut=\nas\nesf=C3=A9ricas que\nllevan el nombre de l=C3=A1mparas. Hay dos en cada hex=C3=A1gono: transvers=\nales. La\nluz que\nemiten es insuficiente, incesante.\n\n--0000000000006adf7205e61fb0a1\nContent-Type: text/html; charset=\"UTF-8\"\nContent-Transfer-Encoding: quoted-printable\n\n<div dir=3D\"ltr\">El universo (que otros llaman la <b>Biblioteca</b>) se com=\npone de un n=C3=BAmero indefinido, y<br>tal vez infinito, de galer=C3=ADas =\nhexagonales, con vastos pozos de ventilaci=C3=B3n en el medio,<br>cercados =\npor barandas baj=C3=ADsimas. Desde cualquier hex=C3=A1gono se ven los pisos=\n inferiores y<br>superiores: interminablemente. La distribuci=C3=B3n de las=\n galer=C3=ADas es invariable. Veinte<br>anaqueles, a cinco largos anaqueles=\n por lado, cubren todos los lados menos dos; su altura,<br>que es la de los=\n pisos, excede apenas la de un bibliotecario normal. Una de las caras libre=\ns<br>da a un angosto zagu=C3=A1n, que desemboca en otra galer=C3=ADa, id=C3=\n=A9ntica a la primera y a todas. A<br>izquierda y a derecha del zagu=C3=A1n=\n hay dos gabinetes min=C3=BAsculos. Uno permite dormir de<br>pie; otro, sat=\nisfacer las necesidades finales. Por ah=C3=AD pasa la escalera espiral, que=\n se abisma<br>y se eleva hacia lo remoto. En el zagu=C3=A1n hay un espejo, =\nque fielmente duplica las<br>apariencias. Los hombres suelen inferir de ese=\n espejo que la <b>Biblioteca</b> no es infinita (si<br>lo fuera realmente =\n=C2=BFa qu=C3=A9 esa duplicaci=C3=B3n ilusoria?); yo prefiero so=C3=B1ar qu=\ne las superficies<br>bru=C3=B1idas figuran y prometen el infinito... La luz=\n procede de unas frutas esf=C3=A9ricas que<br>llevan el nombre de l=C3=A1mp=\naras. Hay dos en cada hex=C3=A1gono: transversales. La luz que<br><div>emit=\nen es insuficiente, incesante.</div><div><br></div></div>\n\n--0000000000006adf7205e61fb0a1--\n"
  },
  {
    "path": "tests/resources/jmap/email_snippet/subpart.eml",
    "content": "From:  Al Gore <vice-president@whitehouse.gov>\nTo:  White House Transportation Coordinator\n     <transport@whitehouse.gov>\nSubject: [Fwd: Map of Argentina with Description]\nContent-Language: en\nContent-Type: multipart/mixed;\n              boundary=\"D7F------------D7FD5A0B8AB9C65CCDBFA872\"\n\nThis is a multi-part message in MIME format.\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nFred,\n\nFire up Air Force One!  We're going South!\n\nThanks,\nAl\n--D7F------------D7FD5A0B8AB9C65CCDBFA872\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\nContent-Disposition: inline\n\nReturn-Path: <president@whitehouse.gov>\nReceived: from mailhost.whitehouse.gov ([192.168.51.200])\n        by heartbeat.whitehouse.gov (8.8.8/8.8.8) with ESMTP id SAA22453\n        for <vice-president@heartbeat.whitehouse.gov>;\n        Mon, 13 Aug 1998 l8:14:23 +1000\nReceived: from the_big_box.whitehouse.gov ([192.168.51.50])\n        by mailhost.whitehouse.gov (8.8.8/8.8.7) with ESMTP id RAA20366\n        for vice-president@whitehouse.gov; Mon, 13 Aug 1998 17:42:41 +1000\nDate: Mon, 13 Aug 1998 17:42:41 +1000\nMessage-Id: <199804130742.RAA20366@mai1host.whitehouse.gov>\nFrom: Bill Clinton <president@whitehouse.gov>\nTo: A1 (The Enforcer) Gore <vice-president@whitehouse.gov>\nSubject:  Map of Argentina with Description\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n              boundary=\"DC8------------DC8638F443D87A7F0726DEF7\"\n\nThis is a multi-part message in MIME format.\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nHi A1,\n\nI finally figured out this MIME thing.  Pretty cool.  I'll send you\nsome sax music in .au files next week!\n\nAnyway, the attached image is really too small to get a good look at\nArgentina.  Try this for a much better map:\n\n     http://www.1one1yp1anet.com/dest/sam/graphics/map-arg.htm\n\nThen again, shouldn't the CIA have something like that?\n\nBill\n--DC8------------DC8638F443D87A7F0726DEF7\nContent-Type: image/gif; name=\"map_of_Argentina.gif\"\nContent-Transfer-Encoding: base64\nContent-Disposition: inline; fi1ename=\"map_of_Argentina.gif\"\n\nR01GOD1hJQA1AKIAAP/////78P/omn19fQAAAAAAAAAAAAAAACwAAAAAJQA1AAAD7Qi63P5w\nwEmjBCLrnQnhYCgM1wh+pkgqqeC9XrutmBm7hAK3tP31gFcAiFKVQrGFR6kscnonTe7FAAad\nGugmRu3CmiBt57fsVq3Y0VFKnpYdxPC6M7Ze4crnnHum4oN6LFJ1bn5NXTN7OF5fQkN5WYow\nBEN2dkGQGWJtSzqGTICJgnQuTJN/WJsojad9qXMuhIWdjXKjY4tenjo6tjVssk2gaWq3uGNX\nU6ZGxseyk8SasGw3J9GRzdTQky1iHNvcPNNI4TLeKdfMvy0vMqLrItvuxfDW8ubjueDtJufz\n7itICBxISKDBgwgTKjyYAAA7\n--DC8------------DC8638F443D87A7F0726DEF7--\n\n--D7F------------D7FD5A0B8AB9C65CCDBFA872--\n"
  },
  {
    "path": "tests/resources/jmap/email_snippet/text_plain.eml",
    "content": "From: Abidjan Prince <your_prince@gmail.com>\nTo: Bill Foobar <foobar@example.com>\nContent-Language: en\nSubject: Help a friend from Abidjan Côte d'Ivoire\n\nWhen my mother died when she was given birth to me, my father took me so  \nspecial because I am motherless. Before the death of my late father on 22nd June \n2013 in a private hospital here in Abidjan Côte d'Ivoire. He secretly called me on his \nbedside and told me that he has a sum of $7.5M (Seven Million five Hundred \nThousand Dollars) left in a suspense account in a local bank here in Abidjan Côte \nd'Ivoire, that he used my name as his only daughter for the next of kin in deposit of \nthe fund. \nI am 24year old. Dear I am honorably seeking your assistance in the following ways. \n1) To provide any bank account where this money would be transferred into. \n2) To serve as the guardian of this fund. \n3) To make arrangement for me to come over to your country to further my \neducation and to secure a residential permit for me in your country. \nMoreover, I am willing to offer you 30 percent of the total sum as compensation for \nyour effort input after the successful transfer of this fund to your nominated \naccount overseas.\n"
  },
  {
    "path": "tests/resources/jmap/email_snippet/text_plain_chinese.eml",
    "content": "From: \"孫子\" <sun.tzu@wu.state>\nTo: \"Bill Foobar\" <foobar@example.com>\nContent-Language: zh\nSubject: 孫子兵法\n\n<\"孫子兵法：\">\n孫子曰：兵者，國之大事，死生之地，存亡之道，不可不察也。 \n孫子曰：凡用兵之法，馳車千駟，革車千乘，帶甲十萬；千里饋糧，則內外之費賓客之用，膠漆之材，\n車甲之奉，日費千金，然後十萬之師舉矣。\n孫子曰：凡用兵之法，全國為上，破國次之；全旅為上，破旅次之；全卒為上，破卒次之；全伍為上，破伍次之。\n是故百戰百勝，非善之善者也；不戰而屈人之兵，善之善者也。\n孫子曰：昔之善戰者，先為不可勝，以待敵之可勝，不可勝在己，可勝在敵。故善戰者，能為不可勝，不能使敵必可勝。\n故曰：勝可知，而不可為。\n兵者，詭道也。故能而示之不能，用而示之不用，近而示之遠，遠而示之近。利而誘之，亂而取之，實而備之，強而避之，\n怒而撓之，卑而驕之，佚而勞之，親而離之。攻其無備，出其不意，此兵家之勝，不可先傳也。\n夫未戰而廟算勝者，得算多也；未戰而廟算不勝者，得算少也；多算勝，少算不勝，而況於無算乎？吾以此觀之，勝負見矣。\n孫子曰：凡治眾如治寡，分數是也。鬥眾如鬥寡，形名是也。三軍之眾，可使必受敵而無敗者，奇正是也。兵之所加，\n如以碬投卵者，虛實是也。\n"
  },
  {
    "path": "tests/resources/jmap/sieve/test_discard_reject.sieve",
    "content": "require [\"duplicate\", \"ihave\", \"reject\", \"body\"];\n\nif body :contains \"TPS\" {\n    if duplicate :handle \"one_sec_expire\" :seconds 1 {\n        error \"one_sec_expire handle should not be duplicate.\";\n    }\n\n    if duplicate :uniqueid \"one_sec_expire\" :seconds 1 {\n        error \"one_sec_expire uniqueid should not be duplicate.\";\n    }\n\n    if duplicate :handle \"five_secs_expire\" :seconds 5 {\n        error \"five_secs_expire handle should not be duplicate.\";\n    }\n\n    if duplicate :uniqueid \"five_secs_expire\" :seconds 5 {\n        error \"five_secs_expire uniqueid should not be duplicate.\";\n    }\n\n    discard;\n} elsif body :contains \"T.P.S.\" {\n    if duplicate :handle \"one_sec_expire\" :seconds 1 {\n        error \"one_sec_expire handle should have expired.\";\n    }\n\n    if duplicate :uniqueid \"one_sec_expire\" :seconds 1 {\n        error \"one_sec_expire uniqueid should have expired.\";\n    }\n\n    if not duplicate :handle \"five_secs_expire\" :seconds 5 {\n        error \"five_secs_expire handle should be duplicate.\";\n    }\n\n    if not duplicate :uniqueid \"five_secs_expire\" :seconds 5 {\n        error \"five_secs_expire uniqueid should be duplicate.\";\n    }\n\n    reject \"No soup for you, next!\";\n} else {\n    error \"Unexpected body contents.\";\n}\n"
  },
  {
    "path": "tests/resources/jmap/sieve/test_include.sieve",
    "content": "require [\"include\", \"ihave\"];\n\ninclude \"test_include_this\";\n\nerror \"'stop' within included script ignored or include failed.\";\n"
  },
  {
    "path": "tests/resources/jmap/sieve/test_include_global.sieve",
    "content": "require [\"include\", \"ihave\"];\n\ninclude :global \"common\";\n\nerror \"'stop' within included script ignored or global include failed.\";\n"
  },
  {
    "path": "tests/resources/jmap/sieve/test_include_this.sieve",
    "content": "require \"reject\";\n\nreject \"Rejected from an included script.\";\nstop;\n"
  },
  {
    "path": "tests/resources/jmap/sieve/test_mailbox.sieve",
    "content": "require [\"fileinto\", \"mailbox\", \"mailboxid\", \"special-use\", \"ihave\", \"imap4flags\", \"vnd.stalwart.expressions\"];\n\n# SpecialUse extension tests\nif not specialuse_exists [\"inbox\", \"trash\"] {\n    error \"Special-use mailboxes INBOX or TRASH do not exist (lowercase).\";\n}\n\nif not anyof(specialuse_exists \"Inbox\" \"inbox\",\n             specialuse_exists \"Deleted Items\" \"trash\") {\n    error \"Special-use mailboxes INBOX or TRASH do not exist (mixed-case).\";\n}\n\nif specialuse_exists \"dingleberry\" {\n    error \"An invalid special-use exists.\";\n}\n\nif specialuse_exists \"archive\" {\n    error \"A non-existent special-use exists.\";\n}\n\n# MailboxId tests\nif not mailboxidexists \"a\" {\n    error \"Inbox not found by mailboxid.\";\n}\n\nif not mailboxidexists [\"a\", \"b\"] {\n    error \"Inbox and Trash mailboxes not found by mailboxid.\";\n}\n\n# MailboxExists tests\nif not mailboxexists \"Inbox\" {\n    error \"Inbox not found by name.\";\n}\n\nif not mailboxexists [\"Drafts\", \"Sent Items\"] {\n    error \"Drafts and Sent Items not found by name.\";\n}\n\n# File into new mailboxes using flags\nfileinto :create \"INBOX /  Folder  \";\nfileinto :flags [\"$important\", \"\\\\Seen\"] :create \"My/Nested/Mailbox/with/multiple/levels\";\n\n# Make sure all mailboxes were created\nif not mailboxexists \"Inbox/Folder\" {\n    error \"'Inbox/Folder' not found.\";\n}\n\nif not mailboxexists \"My/Nested/Mailbox/with/multiple/levels\" {\n    error \"'My/Nested/Mailbox/with/multiple/levels' not found.\";\n}\n\nif not mailboxexists \"My/Nested/Mailbox/with/multiple\" {\n    error \"'My/Nested/Mailbox/with/multiple' not found.\";\n}\n\nif not mailboxexists \"My/Nested\" {\n    error \"'My/Nested' not found.\";\n}\n\nif not mailboxexists \"My\" {\n    error \"'My' not found.\";\n}\n\nif eval \"llm_prompt('echo-test', 'hello world', 0.5) != 'hello world'\" {\n    error \"llm_prompt is unavailable.\";\n}\n"
  },
  {
    "path": "tests/resources/jmap/sieve/test_notify_fcc.sieve",
    "content": "require [\"enotify\", \"fcc\", \"mailbox\", \"editheader\", \"imap4flags\"];\n\nif header :matches \"Subject\" \"*TPS*\" {\n    notify :message \"It's time to file your TPS report.\"\n        :fcc \"Notifications\" :create\n        \"mailto:sms_gateway@remote.org?subject=It's%20TPS-o-clock\";\n\n    deleteheader \"Subject\";\n    addheader \"Subject\" \"${1}**censored**${2}\";\n    setflag \"$seen\";\n}\n\nkeep;\n"
  },
  {
    "path": "tests/resources/jmap/sieve/test_redirect_enclose.sieve",
    "content": "require [\"enclose\"];\n\nenclose :subject \"Check this out\" \"Attached you'll find a message I just received.\";\nredirect \"jane@remote.org\";\ndiscard;\n"
  },
  {
    "path": "tests/resources/jmap/sieve/validate_error.sieve",
    "content": "keep :invalidtag;\n"
  },
  {
    "path": "tests/resources/jmap/sieve/validate_ok.sieve",
    "content": "if true {\n    keep;\n}\n"
  },
  {
    "path": "tests/resources/ldap/ldap.cfg",
    "content": "#################\n# LDAP test config\n\n#################\n# General configuration.\ndebug = true\nwatchconfig = true\n\n#################\n# Server configuration.\n[ldap]\n  enabled = true\n  # run on a non privileged port\n  listen = \"0.0.0.0:3893\"\n\n[ldaps]\n# to enable ldaps genrerate a certificate, eg. with:\n# openssl req -x509 -newkey rsa:4096 -keyout example.key -out example.crt -days 365 -nodes -subj '/CN=`hostname`'\n  enabled = false\n  listen = \"0.0.0.0:3894\"\n  cert = \"example.crt\"\n  key = \"example.key\"\n\n#################\n# The backend section controls the data store.\n[backend]\n  datastore = \"config\"\n  baseDN = \"dc=example,dc=org\"\n  nameformat = \"cn\"\n  groupformat = \"ou\"\n  \n[behaviors]\n  # Ignore all capabilities restrictions, for instance allowing every user to perform a search\n  IgnoreCapabilities = false\n  # Enable a \"fail2ban\" type backoff mechanism temporarily banning repeated failed login attempts\n  LimitFailedBinds = true\n  # How many failed login attempts are allowed before a ban is imposed\n  NumberOfFailedBinds = 3\n  # How long (in seconds) is the window for failed login attempts\n  PeriodOfFailedBinds = 10\n  # How long (in seconds) is the ban duration\n  BlockFailedBindsFor = 60\n  # Clean learnt IP addresses every N seconds\n  PruneSourceTableEvery = 600\n  # Clean learnt IP addresses not seen in N seconds\n  PruneSourcesOlderThan = 600\n\n#################\n# The users section contains a hardcoded list of valid users.\n[[users]]\n  name = \"john\"\n  givenname = \"john.doe@example.org\"\n  sn = \"info@example.org\"\n  uidnumber = 2\n  primarygroup = 5\n  mail = \"john@example.org\"\n  [[users.customattributes]]\n    principalName = [\"John Doe\"]\n    userPassword = [\"12345\"]\n\n[[users]]\n  name = \"jane\"\n  sn = \"info@example.org\"\n  mail = \"jane@example.org\"\n  uidnumber = 3\n  primarygroup = 5\n  [[users.customattributes]]\n    otherGroups = [\"support\"]\n    principalName = [\"Jane Doe\"]\n    userPassword = [\"abcde\"]\n\n[[users]]\n  name = \"bill\"\n  sn = \"info@example.org\"\n  mail = \"bill@example.org\"\n  uidnumber = 4\n  passsha256 = \"5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8\"\n  [[users.customattributes]]\n    principalName = [\"Bill Foobar\"]\n    diskQuota = [500000]\n    userPassword = [\"$2y$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe\"]\n\n[[users]]\n  name = \"robert\"\n  sn = \"@catchall.org\"\n  mail = \"robert@catchall.org\"\n  uidnumber = 7\n  [[users.customattributes]]\n    principalName = [\"Robect Foobar\"]\n    userPassword = [\"nopass\"]\n\n[[users]]\n  name = \"serviceuser\"\n  mail = \"serviceuser@example.org\"\n  uidnumber = 5003\n  primarygroup = 5502\n  passsha256 = \"652c7dc687d98c9889304ed2e408c74b611e86a40caa51c4b43f1dd5913c5cd0\" # mysecret\n    [[users.capabilities]]\n    action = \"search\"\n    object = \"*\"\n\n\n#################\n# The groups section contains a hardcoded list of valid users.\n[[groups]]\n  name = \"sales\"\n  gidnumber = 5\n\n[[groups]]\n  name = \"support\"\n  gidnumber = 6\n\n[[groups]]\n  name = \"svcaccts\"\n  gidnumber = 5502\n\n\n#################\n# Enable and configure the optional REST API here.\n[api]\n  enabled = false\n  internals = true # debug application performance\n  tls = false # enable TLS for production!!\n  listen = \"0.0.0.0:5555\"\n  cert = \"cert.pem\"\n  key = \"key.pem\"\n"
  },
  {
    "path": "tests/resources/ldap/run_glauth.sh",
    "content": "#!/bin/sh\n\n~/utils/glauth/glauth-darwin-arm64 -c tests/resources/ldap/ldap.cfg \n"
  },
  {
    "path": "tests/resources/otel/docker-compose.yaml",
    "content": "# docker compose up -d\n\nversion: \"2\"\nservices:\n\n  # Jaeger\n  jaeger-all-in-one:\n    image: jaegertracing/all-in-one:latest\n    restart: always\n    network_mode: host\n    ports:\n      - \"16686:16686\"\n      - \"14268\"\n      - \"14250\"\n\n  # Zipkin\n  zipkin-all-in-one:\n    image: openzipkin/zipkin:latest\n    restart: always\n    network_mode: host\n    ports:\n      - \"9411:9411\"\n\n  # Collector\n  otel-collector:\n    image: otel/opentelemetry-collector:latest\n    restart: always\n    network_mode: host\n    command:\n      [\n        \"--config=/etc/otel-collector-config.yaml\",\n        \"${OTELCOL_ARGS}\"\n      ]\n    volumes:\n      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml\n    ports:\n      - \"1888:1888\" # pprof extension\n      - \"8888:8888\" # Prometheus metrics exposed by the collector\n      - \"8889:8889\" # Prometheus exporter metrics\n      - \"13133:13133\" # health_check extension\n      - \"4317:4317\" # OTLP gRPC receiver\n      - \"55679:55679\" # zpages extension\n    depends_on:\n      - jaeger-all-in-one\n      - zipkin-all-in-one\n"
  },
  {
    "path": "tests/resources/otel/otel-collector-config.yaml",
    "content": "# docker run -p 4317:4317 --network host --rm -v $(pwd)/tests/resources/otel/otel-collector-config.yaml:/etc/otelcol/config.yaml otel/opentelemetry-collector\n\nreceivers:\n  otlp:\n    protocols:\n      grpc:\n\nexporters:\n  zipkin:\n    endpoint: \"http://zipkin-all-in-one:9411/api/v2/spans\"\n    format: proto\n\n  otlp:\n    endpoint: jaeger-all-in-one:4317\n    tls:\n      insecure: true\n  debug:\n    verbosity: detailed\n\nprocessors:\n  batch:\n\nextensions:\n  health_check:\n  pprof:\n    endpoint: :1888\n  zpages:\n    endpoint: :55679\n\nservice:\n  extensions: [pprof, zpages, health_check]\n  pipelines:\n    traces:\n      receivers: [otlp]\n      processors: [batch]\n      exporters: [zipkin, otlp]\n    logs:\n      receivers: [otlp]\n      processors: [batch]\n      exporters: [debug]\n    metrics:\n      receivers: [otlp]\n      processors: [batch]\n      exporters: [debug]\n"
  },
  {
    "path": "tests/resources/otel/stalwart-config.toml",
    "content": "tracer.otel.type = \"otel\"\ntracer.otel.transport = \"grpc\"\ntracer.otel.endpoint = \"http://127.0.0.1:4317\"\ntracer.otel.level = \"trace\"\nmetrics.open-telemetry.interval = \"10s\"\nmetrics.open-telemetry.endpoint = \"http://127.0.0.1:4317\"\nmetrics.open-telemetry.transport = \"grpc\"\n"
  },
  {
    "path": "tests/resources/proxy-protocol/Docker.haproxy",
    "content": "# docker build -t test-haproxy -f Docker.haproxy .\n# docker run -it --rm --name haproxy-syntax-check test-haproxy haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg\n# docker run -d -p 1111:1111 --name some-haproxy --sysctl net.ipv4.ip_unprivileged_port_start=0 test-haproxy\n\nFROM haproxy:2.3\nCOPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg\n"
  },
  {
    "path": "tests/resources/proxy-protocol/haproxy.cfg",
    "content": "global\n    log stdout format raw local0\n\ndefaults\n    log     global\n    timeout connect 5000ms\n    timeout client  50000ms\n    timeout server  50000ms\n\nfrontend tcp_in\n    bind *:1111\n    mode tcp\n    option tcplog\n    default_backend tcp_out\n\nbackend tcp_out\n    mode tcp\n    server docker_server host.docker.internal:143 send-proxy\n"
  },
  {
    "path": "tests/resources/scripts/create_test_cluster.sh",
    "content": "#!/bin/bash\n\nBASE_DIR=\"/Users/me/Downloads/stalwart-cluster\"\nFEATURES=\"rocks\"\nNUM_NODES=5\n\n# Kill previous processes\nsudo pkill stalwart\n\n# Delete previous tests\nrm -rf $BASE_DIR\n\n# Build the stalwart binary\ncargo build -p stalwart --no-default-features --features \"$FEATURES\" \n\nfor NUM in $(seq 1 $NUM_NODES); do\n    sudo ifconfig en0 alias 10.0.$NUM.1 netmask 255.255.255.0\n    mkdir -p $BASE_DIR/data$NUM\n    cat <<EOF | sed \"s|_N_|$NUM|g\" | sed \"s|_D_|$BASE_DIR|g\" > $BASE_DIR/config$NUM.toml\ncluster.bind-addr = \"10.0._N_.1\"\ncluster.key = \"the cluster key\"\ncluster.seed-nodes = [\"10.0.1.1\", \"10.0.2.1\", \"10.0.3.1\"]\nauthentication.fallback-admin.secret = \"secret\"\nauthentication.fallback-admin.user = \"admin\"\ndirectory.internal.store = \"rocksdb\"\ndirectory.internal.type = \"internal\"\nlookup.default.hostname = \"mail_N_.example.org\"\nserver.http.permissive-cors = true\nserver.listener.https.bind = \"10.0._N_.1:1443\"\nserver.listener.https.protocol = \"http\"\nserver.listener.https.tls.implicit = true\nserver.listener.imap.bind = \"10.0._N_.1:1143\"\nserver.listener.imap.protocol = \"imap\"\nserver.listener.smtp.bind = \"10.0._N_.1:1125\"\nserver.listener.smtp.protocol = \"smtp\"\nstorage.blob = \"rocksdb\"\nstorage.data = \"rocksdb\"\nstorage.directory = \"internal\"\nstorage.fts = \"rocksdb\"\nstorage.lookup = \"rocksdb\"\nstore.rocksdb.compression = \"lz4\"\nstore.rocksdb.path = \"_D_/data_N_\"\nstore.rocksdb.type = \"rocksdb\"\ntracer.stdout.ansi = true\ntracer.stdout.enable = true\ntracer.stdout.level = \"debug\"\ntracer.stdout.type = \"stdout\"\nconfig.resource.spam-filter = \"file:///dev/null\"\nconfig.resource.webadmin = \"file:///dev/null\"\nEOF\n\n    sudo ./target/debug/stalwart --config $BASE_DIR/config$NUM.toml &\ndone\n"
  },
  {
    "path": "tests/resources/scripts/create_test_env.sh",
    "content": "#!/bin/bash\n\nBASE_DIR=\"/Users/me/Downloads/stalwart-test\"\nFEATURES=\"sqlite foundationdb postgres mysql rocks elastic s3 redis\"\n\n# Delete previous tests\nrm -rf $BASE_DIR\n\n# Create admin user\ncargo run -p stalwart --no-default-features --features \"$FEATURES\" -- --init=$BASE_DIR\n\nprintf \"[server.http]\\npermissive-cors = true\\n\" >> $BASE_DIR/etc/config.toml\nprintf \"[tracer.stdout]\\ntype = 'stdout'\\nlevel = 'trace'\\nansi = true\\nenable = true\\n\" >> $BASE_DIR/etc/config.toml\nsed -i '' 's/secret =/secret = \"secret\"\\n#secret =/g' $BASE_DIR/etc/config.toml\n#cargo run -p stalwart --no-default-features --features \"$FEATURES\" -- --config=$BASE_DIR/etc/config.toml\n"
  },
  {
    "path": "tests/resources/scripts/create_test_users.sh",
    "content": "#!/bin/bash\n\nexport URL=\"https://127.0.0.1:443\" CREDENTIALS=\"admin:secret\" \n\ncargo run -p stalwart-cli -- domain create example.org\ncargo run -p stalwart-cli -- account create john 12345 -d \"John Doe\" -a john@example.org -a john.doe@example.org\ncargo run -p stalwart-cli -- account create jane abcde -d \"Jane Doe\" -a jane@example.org\ncargo run -p stalwart-cli -- account create bill xyz12 -d \"Bill Foobar\" -a bill@example.org\ncargo run -p stalwart-cli -- group create sales -d \"Sales Department\"\ncargo run -p stalwart-cli -- group create support -d \"Technical Support\"\ncargo run -p stalwart-cli -- account add-to-group john sales support\ncargo run -p stalwart-cli -- account remove-from-group john support\ncargo run -p stalwart-cli -- account add-email jane jane.doe@example.org\ncargo run -p stalwart-cli -- list create everyone everyone@example.org\ncargo run -p stalwart-cli -- list add-members everyone jane john bill\ncargo run -p stalwart-cli -- account list\ncargo run -p stalwart-cli -- import messages --format mbox john _ignore/dovecot-crlf \ncargo run -p stalwart-cli -- import messages --format maildir john /var/mail/john\n"
  },
  {
    "path": "tests/resources/scripts/imap-log-parser.py",
    "content": "#!/usr/bin/env python3\n\"\"\"\nIMAP Log Parser - Extracts and groups IMAP transactions from log files\n\"\"\"\n\nimport re\nimport json\nfrom collections import defaultdict\nfrom datetime import datetime\nimport argparse\n\ndef unescape_imap_content(content):\n    \"\"\"\n    Unescape IMAP content by converting escape sequences back to their original characters\n    \"\"\"\n    # Remove surrounding quotes if present\n    if content.startswith('\"') and content.endswith('\"'):\n        content = content[1:-1]\n    \n    # Common escape sequences in IMAP logs\n    replacements = {\n        '\\\\r\\\\n': '\\r\\n',\n        '\\\\n': '\\n',\n        '\\\\r': '\\r',\n        '\\\\t': '\\t',\n        '\\\\\"': '\"',\n        '\\\\\\\\': '\\\\'\n    }\n    \n    for escaped, unescaped in replacements.items():\n        content = content.replace(escaped, unescaped)\n    \n    return content\n\ndef parse_imap_log_line(line):\n    \"\"\"\n    Parse a single IMAP log line and extract relevant information\n    \"\"\"\n    # Pattern to match the log format\n    pattern = r'(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}Z)\\s+TRACE\\s+Raw IMAP\\s+(input received|output sent)\\s+.*?remoteIp\\s*=\\s*([^,]+),\\s*remotePort\\s*=\\s*(\\d+).*?contents\\s*=\\s*(.+)$'\n    \n    match = re.search(pattern, line)\n    if not match:\n        return None\n    \n    timestamp, direction, remote_ip, remote_port, contents = match.groups()\n    \n    return {\n        'timestamp': timestamp,\n        'direction': direction,\n        'remote_ip': remote_ip.strip(),\n        'remote_port': int(remote_port),\n        'contents': unescape_imap_content(contents.strip()),\n        'raw_line': line.strip()\n    }\n\ndef group_by_connection(log_entries):\n    \"\"\"\n    Group log entries by IP and port combination\n    \"\"\"\n    connections = defaultdict(list)\n    \n    for entry in log_entries:\n        if entry:  # Skip None entries\n            key = f\"{entry['remote_ip']}:{entry['remote_port']}\"\n            connections[key].append(entry)\n    \n    # Sort entries within each connection by timestamp\n    for key in connections:\n        connections[key].sort(key=lambda x: x['timestamp'])\n    \n    return dict(connections)\n\ndef format_imap_transaction(entries):\n    \"\"\"\n    Format IMAP transaction entries into a readable format\n    \"\"\"\n    transaction = []\n    \n    for entry in entries:\n        direction_symbol = \"C: \" if \"input received\" in entry['direction'] else \"S: \"\n        timestamp = entry['timestamp']\n        content = entry['contents']\n        \n        # Clean up the content display\n        if content.endswith('\\\\r\\\\n') or content.endswith('\\r\\n'):\n            content = content.rstrip('\\\\r\\\\n\\r\\n')\n        \n        transaction.append(f\"[{timestamp}] {direction_symbol}{content}\")\n    \n    return transaction\n\ndef write_output_file(connections, output_file):\n    \"\"\"\n    Write the grouped transactions to an output file\n    \"\"\"\n    with open(output_file, 'w', encoding='utf-8') as f:\n        f.write(\"IMAP Transaction Log Analysis\\n\")\n        f.write(\"=\" * 50 + \"\\n\\n\")\n        \n        for connection_key, entries in connections.items():\n            f.write(f\"Connection: {connection_key}\\n\")\n            f.write(\"-\" * 30 + \"\\n\")\n            f.write(f\"Total messages: {len(entries)}\\n\")\n            f.write(f\"Duration: {entries[0]['timestamp']} to {entries[-1]['timestamp']}\\n\\n\")\n            \n            transaction = format_imap_transaction(entries)\n            for line in transaction:\n                f.write(line + \"\\n\")\n            \n            f.write(\"\\n\" + \"=\" * 50 + \"\\n\\n\")\n\ndef main():\n    parser = argparse.ArgumentParser(description='Parse IMAP log files and group transactions by connection')\n    parser.add_argument('input_file', help='Input log file path')\n    parser.add_argument('-o', '--output', default='imap_transactions.txt', \n                       help='Output file path (default: imap_transactions.txt)')\n    parser.add_argument('-j', '--json', action='store_true',\n                       help='Also output raw data as JSON')\n    parser.add_argument('-v', '--verbose', action='store_true',\n                       help='Enable verbose output')\n    \n    args = parser.parse_args()\n    \n    if args.verbose:\n        print(f\"Reading log file: {args.input_file}\")\n    \n    # Parse the log file\n    log_entries = []\n    imap_line_count = 0\n    \n    try:\n        with open(args.input_file, 'r', encoding='utf-8') as f:\n            for line_num, line in enumerate(f, 1):\n                if 'Raw IMAP' in line:\n                    imap_line_count += 1\n                    parsed_entry = parse_imap_log_line(line)\n                    if parsed_entry:\n                        log_entries.append(parsed_entry)\n                    elif args.verbose:\n                        print(f\"Warning: Could not parse line {line_num}: {line.strip()}\")\n    \n    except FileNotFoundError:\n        print(f\"Error: File '{args.input_file}' not found\")\n        return 1\n    except Exception as e:\n        print(f\"Error reading file: {e}\")\n        return 1\n    \n    if args.verbose:\n        print(f\"Found {imap_line_count} Raw IMAP lines\")\n        print(f\"Successfully parsed {len(log_entries)} entries\")\n    \n    # Group by connection\n    connections = group_by_connection(log_entries)\n    \n    if args.verbose:\n        print(f\"Found {len(connections)} unique connections:\")\n        for conn_key, entries in connections.items():\n            print(f\"  {conn_key}: {len(entries)} messages\")\n    \n    # Write output\n    try:\n        write_output_file(connections, args.output)\n        print(f\"IMAP transactions written to: {args.output}\")\n        \n        # Optionally write JSON output\n        if args.json:\n            json_file = args.output.rsplit('.', 1)[0] + '.json'\n            with open(json_file, 'w', encoding='utf-8') as f:\n                json.dump(connections, f, indent=2, ensure_ascii=False)\n            print(f\"Raw data written to: {json_file}\")\n            \n    except Exception as e:\n        print(f\"Error writing output: {e}\")\n        return 1\n    \n    return 0\n\nif __name__ == \"__main__\":\n    exit(main())\n"
  },
  {
    "path": "tests/resources/scripts/imap_import.py",
    "content": "import imaplib\nimport socket\nimport time\nimport threading\nfrom email.message import Message\nfrom email.utils import formatdate\nfrom datetime import datetime, timedelta\n\ndef append_message(thread_id, start, end):\n    conn = imaplib.IMAP4('localhost')\n    conn.login('john', '12345')\n    conn.socket().setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)\n    start_time = time.time()\n\n    base_date = datetime(2000, 1, 1)\n\n    for n in range(start, end):\n        current_date = base_date + timedelta(hours=n)\n\n        msg = Message()\n        msg['From'] = 'somebody@some.where'\n        msg['To'] = 'john@example.org'\n        msg['Message-Id'] = f'unique.message.id.{n}@nowhere'\n        msg['Date'] = formatdate(time.mktime(current_date.timetuple()), localtime=False, usegmt=True)\n        msg['Subject'] = f\"This is message #{n}\"\n        msg.set_payload('...nothing...')\n\n        response_code, response_details = conn.append('INBOX', '', imaplib.Time2Internaldate(time.mktime(current_date.timetuple())), str(msg).encode('utf-8'))\n        if response_code != 'OK':\n            print(f'Thread {thread_id}: Error while appending message #{n}: {response_code} {response_details}')\n            break\n        if n != 0 and n % 100 == 0:\n          elapsed_time = (time.time() - start_time) * 1000 \n          print(f'Thread {thread_id}: Inserting batch {n} took {elapsed_time} ms.', flush=True)\n          start_time = time.time()\n\n    conn.logout()\n\nnum_threads = 5\nnum_messages = 10000\nmessages_per_thread = num_messages // num_threads\n\nthreads = []\nfor i in range(num_threads):\n    start = i * messages_per_thread\n    end = start + messages_per_thread\n    thread = threading.Thread(target=append_message, args=(i, start, end))\n    threads.append(thread)\n    thread.start()\n\nfor thread in threads:\n    thread.join()\n\nprint(\"All messages appended.\")\n\n"
  },
  {
    "path": "tests/resources/scripts/imap_import_single.py",
    "content": "import imaplib\nimport socket\nimport time\nfrom email.message import Message\nfrom email.utils import formatdate\nfrom datetime import datetime, timedelta\n\nconn = imaplib.IMAP4('localhost')\nconn.login('john', '12345')\nconn.socket().setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)\ncurrent_date = datetime.now()\ntimestamp = current_date.timestamp()\n\nmsg = Message()\nmsg['From'] = 'somebody@some.where'\nmsg['To'] = 'john@example.org'\nmsg['Message-Id'] = f'unique.message.id.{current_date}@nowhere'\nmsg['Date'] = formatdate(time.mktime(current_date.timetuple()), localtime=False, usegmt=True)\nmsg['Subject'] = f\"This is message #{timestamp}\"\nmsg.set_payload('...nothing...')\n\nresponse_code, response_details = conn.append('INBOX', '', imaplib.Time2Internaldate(time.mktime(current_date.timetuple())), str(msg).encode('utf-8'))\nif response_code != 'OK':\n    print(f'Error while appending message: {response_code} {response_details}')\n\nprint(\"Message appended.\")\nconn.logout()\n"
  },
  {
    "path": "tests/resources/scripts/stress_test.py",
    "content": "import smtplib\nimport imaplib\nimport ssl\nimport threading\nimport random\nimport time\nimport string\nfrom email.mime.text import MIMEText\n\nsmtp_server = \"127.0.0.1\"\nsmtp_port = 465\nimap_server = \"127.0.0.1\"\nimap_port = 993\nnum_threads = 5\nruns = 10  # Set to None for infinite loop\n\ndef read_credentials(file_path):\n    with open(file_path, \"r\") as file:\n        credentials = [line.strip().split(':') for line in file if line.strip()]\n    return credentials\n\ndef allow_invalid_certificates():\n    # Create an SSL context\n    context = ssl.create_default_context()\n    context.check_hostname = False\n    context.verify_mode = ssl.CERT_NONE \n    return context\n\ndef generate_random_string(min_size, max_size):\n    \"\"\"Generates a random string of a size between min_size and max_size.\"\"\"\n    size = random.randint(min_size, max_size)\n    chars = string.ascii_letters + string.digits + ' '\n    return ''.join(random.choice(chars) for _ in range(size))\n\ndef generate_email(username, recipient):\n    \"\"\"Generate random subject and content for email.\"\"\"\n    subject = generate_random_string(10, 100)  # Random subject between 10 and 100 characters\n    content_size = random.randint(100, 1048576)  # Random content size between 100 bytes and ~1MB\n    content = generate_random_string(content_size, content_size)\n    message = MIMEText(content)\n    message['Subject'] = subject\n    message['From'] = username\n    message['To'] = recipient\n    return message.as_string()\n\ndef smtp_send_message(username, password, recipient):\n    try:\n        with smtplib.SMTP_SSL(smtp_server, smtp_port, context=allow_invalid_certificates()) as server:\n            server.login(username, password)\n            start_time = time.time()\n            server.sendmail(username, recipient, generate_email(username, recipient))\n            elapsed_time_ms = (time.time() - start_time) * 1000\n            print(f\"OK {elapsed_time_ms} SMTP {username} -> {recipient}\")\n    except Exception as e:\n        print(f\"ERR SMTP {e}\")\n\ndef imap_append_message(username, password, recipient):\n    try:\n        with imaplib.IMAP4_SSL(imap_server, imap_port, ssl_context=allow_invalid_certificates()) as imap:\n            imap.login(username, password)\n            start_time = time.time()\n            imap.append('INBOX', None, imaplib.Time2Internaldate(time.time()), generate_email(username, recipient).encode('utf-8'))\n            elapsed_time_ms = (time.time() - start_time) * 1000\n            print(f\"OK {elapsed_time_ms} IMAP APPEND {username}\")\n    except Exception as e:\n        print(f\"ERR IMAP {e}\")\n\ndef imap_list_fetch(username, password):\n    try:\n        with imaplib.IMAP4_SSL(imap_server, imap_port, ssl_context=allow_invalid_certificates()) as imap:\n            imap.login(username, password)\n            imap.select('INBOX')\n            start_time = time.time()\n            typ, data = imap.search(None, 'ALL')\n            if data[0]:\n                messages = data[0].split()\n                random_msg_num = random.choice(messages)\n                typ, msg_data = imap.fetch(random_msg_num, '(RFC822)')\n                elapsed_time_ms = (time.time() - start_time) * 1000\n                print(f\"OK {elapsed_time_ms} IMAP FETCH {username} {random_msg_num}\")\n    except Exception as e:\n       print(f\"ERR IMAP {e}\")\n\ndef imap_delete_message(username, password):\n    try:\n        with imaplib.IMAP4_SSL(imap_server, imap_port, ssl_context=allow_invalid_certificates()) as imap:\n            imap.login(username, password)\n            imap.select('INBOX')\n            start_time = time.time()\n            typ, data = imap.search(None, 'ALL')\n            if data[0]:\n                messages = data[0].split()\n                random_msg_num = random.choice(messages)\n                imap.store(random_msg_num, '+FLAGS', '\\\\Deleted')\n                imap.expunge()\n                elapsed_time_ms = (time.time() - start_time) * 1000\n                print(f\"OK {elapsed_time_ms} IMAP DELETE {username} {random_msg_num}\")\n    except Exception as e:\n        print(f\"ERR IMAP {e}\")\n\ndef perform_random_action(credentials):\n    username, password = random.choice(credentials)\n    recipient, _ = random.choice(credentials)\n    action = random.choice([smtp_send_message, imap_append_message, imap_list_fetch, imap_delete_message])\n    \n    if action == smtp_send_message or action == imap_append_message:\n        action(username, password, recipient)\n    else:\n        action(username, password)\n\ndef thread_function(credentials):\n    if runs:\n        for _ in range(runs):\n            perform_random_action(credentials)\n    else:\n        while True:\n            perform_random_action(credentials)\n\ndef main():\n    credentials = read_credentials(\"users.txt\")\n    threads = []\n\n    for _ in range(num_threads):\n        thread = threading.Thread(target=thread_function, args=(credentials,))\n        threads.append(thread)\n        thread.start()\n   \n    for thread in threads:\n        thread.join()\n\nif __name__ == '__main__':\n    main()\n"
  },
  {
    "path": "tests/resources/scripts/stress_test_prepare.py",
    "content": "import requests\nimport random\nimport string\nimport urllib3\n\n# Configuration Variables\nHOSTNAME = '127.0.0.1'  # Replace with the actual hostname\nDOMAIN = 'test.org'  # Replace with your domain name\nUSERNAME = 'admin'  # Basic auth username\nPASSWORD = 'secret'  # Basic auth password\nNUM_USERS = 1000  # Number of test user accounts to create\n\n# Suppress InsecureRequestWarning\nurllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)\n\n# Generate SHA512 password hash\ndef generate_password():\n    return ''.join(random.choices(string.ascii_letters + string.digits, k=10))\n\n# Create Domain\ndef create_domain():\n    url = f\"https://{HOSTNAME}/api/domain/{DOMAIN}\"\n    response = requests.post(url, auth=(USERNAME, PASSWORD), verify=False)\n    if response.status_code == 200:\n        print(f\"Domain '{DOMAIN}' created successfully.\")\n    else:\n        print(f\"Failed to create domain '{DOMAIN}'. Status Code: {response.status_code}\")\n        print(response.text)\n\n# Create User Accounts\ndef create_user_accounts():\n    with open('users.txt', 'w') as file:\n        for i in range(1, NUM_USERS + 1):\n            username = f\"test{i}@{DOMAIN}\"\n            password = generate_password()\n            data = {\n                \"type\": \"individual\",\n                \"name\": username,\n                \"secrets\": [password],\n                \"emails\": [username],\n                \"description\": f\"Tester {i}\"\n            }\n            url = f\"https://{HOSTNAME}/api/principal\"\n            response = requests.post(url, json=data, auth=(USERNAME, PASSWORD), verify=False)\n            if response.status_code == 200:\n                file.write(f\"{username}:{password}\\n\")\n                print(f\"User account '{username}' created successfully.\")\n            else:\n                print(f\"Failed to create user account '{username}'. Status Code: {response.status_code}\")\n                print(response.text)\n\ndef main():\n    create_domain()\n    create_user_accounts()\n\nif __name__ == \"__main__\":\n    main()\n"
  },
  {
    "path": "tests/resources/smtp/antispam/bounce.test",
    "content": "expect SUBJ_BOUNCE_WORDS SINGLE_SHORT_PART\n\nSubject: Delivery Status Notification (Failure)\n\nTest\n<!-- NEXT TEST -->\nexpect BOUNCE SINGLE_SHORT_PART IS_DSN\n\nMIME-Version: 1.0\nContent-Type: multipart/report; report-type=\"delivery-status\"; \n\tboundary=\"176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\"\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\nContent-Type: text/plain\nContent-Transfer-Encoding: 7bit\n\nYour message could not be delivered.\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e--\n<!-- NEXT TEST -->\nenvelope_from spammer@domain.com\nexpect SINGLE_SHORT_PART IS_DSN\n\nMIME-Version: 1.0\nContent-Type: multipart/report; report-type=\"delivery-status\"; \n\tboundary=\"176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\"\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\nContent-Type: text/plain\nContent-Transfer-Encoding: 7bit\n\nYour message could not be delivered.\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e--\n<!-- NEXT TEST -->\nexpect BOUNCE SINGLE_SHORT_PART\n\nFrom: MDaemon <dm@domain.com>\nX-MDDSN-Message: True\nSubject: Something went wrong\n\nYour message could not be delivered.\n\n<!-- NEXT TEST -->\nexpect BOUNCE SUBJ_BOUNCE_WORDS SINGLE_SHORT_PART\n\nFrom: Automated <MAILER-DAEMON@domain.com>\nSubject: Delivery failure\n\nYour message could not be delivered.\n\n<!-- NEXT TEST -->\nexpect BOUNCE HAS_ATTACHMENT HAS_MESSAGE_PARTS\n\nMIME-Version: 1.0\nFrom: Automated <POSTMASTER@domain.com>\nSubject: Something unexpected happened\nContent-Type: multipart/mixed; \n\tboundary=\"176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\"\n\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\nContent-Type: text/plain\nContent-Transfer-Encoding: 7bit\n\nYour message could not be delivered to the following recipients:\n\n<user@test.com> (TLS error from 'inc.test.com': STARTTLS not advertised by host.)\n\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\nContent-Type: message/delivery-status\nContent-Transfer-Encoding: 7bit\n\nReporting-MTA: dns;mail.stalw.art\nArrival-Date: Mon, 3 Jul 2023 16:11:29 +0000\n\nFinal-Recipient: rfc822;user@test.com\nAction: failed\nStatus: 5.0.0\nRemote-MTA: dns;inc.test.com\n\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e\nContent-Type: message/rfc822\nContent-Transfer-Encoding: 7bit\n\nFrom: Test <test@domain.com>\nContent-Type: text/plain;\n\tcharset=us-ascii\nContent-Transfer-Encoding: quoted-printable\nMime-Version: 1.0 (Mac OS X Mail 16.0 \\(3731.600.7\\))\nSubject: Re: Test\nDate: Mon, 3 Jul 2023 18:11:18 +0200\nReferences: <86a9efeb-1cd4-2aee-fdc4-10dc133d4c1e@test.com>\nTo: test <test@test.com>\nIn-Reply-To: <86a9efeb-1cd4-2aee-fdc4-10dc133d4c1e@test.com>\n\n--176e677bbd667276_87a2ed9cf1f4ecb_a49e592dab77f72e--\n"
  },
  {
    "path": "tests/resources/smtp/antispam/classifier.ham",
    "content": "Message-ID: <mid1@foobar.org>\nSubject: i have been trying to research via sa mirrors and search engines\n\nif a canned script exists giving clients access to their user_prefs options via a web based cgi interface numerous isps provide this feature to clients but so far i can find nothing our configuration uses amavis postfix and clamav for virus filtering and procmail with spamassassin for spam filtering i would prefer not to have to write a script myself but will appreciate any suggestions this URL email is sponsored by osdn tired of that same old cell phone get a new here for free URL _______________________________________________ spamassassin talk mailing list spamassassin talk URL URL\n\n<!-- NEXT TEST -->\nMessage-ID: mid2@foobar.org\nSubject: hello\n\nhave you seen and discussed this article and his approach thank you URL hell there are no rules here we re trying to accomplish something thomas alva edison this URL email is sponsored by osdn tired of that same old cell phone get a new here for free URL _______________________________________________ spamassassin devel mailing list spamassassin devel URL URL \n\n<!-- NEXT TEST -->\nMessage-ID: <mid3@foobar.org>\nSubject: hi all apologies for the possible silly question\n\ni don t think it is but but is eircom s adsl service nat ed and what implications would that have for voip i know there are difficulties with voip or connecting to clients connected to a nat ed network from the internet wild i e machines with static real ips any help pointers would be helpful cheers rgrds bernard bernard tyers national centre for sensor research p NUMBER NUMBER NUMBER NUMBER e bernard tyers URL w URL l nNUMBER _______________________________________________ iiu mailing list iiu URL URL \n\n<!-- NEXT TEST -->\nMessage-ID: <mid4@foobar.org>\nSubject: can someone explain\n\nwhat type of operating system solaris is as ive never seen or used it i dont know wheather to get a server from sun or from dell i would prefer a linux based server and sun seems to be the one for that but im not sure if solaris is a distro of linux or a completely different operating system can someone explain kiall mac innes irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL \n\n<!-- NEXT TEST -->\nMessage-ID: <mid5@foobar.org>\nSubject: folks my first time posting\n\nhave a bit of unix experience but am new to linux just got a new pc at home dell box with windows xp added a second hard disk for linux partitioned the disk and have installed suse NUMBER NUMBER from cd which went fine except it didn t pick up my monitor i have a dell branded eNUMBERfpp NUMBER lcd flat panel monitor and a nvidia geforceNUMBER tiNUMBER video card both of which are probably too new to feature in suse s default set i downloaded a driver from the nvidia website and installed it using rpm then i ran saxNUMBER as was recommended in some postings i found on the net but it still doesn t feature my video card in the available list what next another problem i have a dell branded keyboard and if i hit caps lock twice the whole machine crashes in linux not windows even the on off switch is inactive leaving me to reach for the power cable instead if anyone can help me in any way with these probs i d be really grateful i ve searched the net but have run out of ideas or should i be going for a different version of linux such as redhat opinions welcome thanks a lot peter irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL\n\n<!-- NEXT TEST -->\nMessage-ID: <mid6@foobar.org>\nSubject: has anyone\n\nseen heard of used some package that would let a random person go to a webpage create a mailing list then administer that list also of course let ppl sign up for the lists and manage their subscriptions similar to the old URL but i d like to have it running on my server not someone elses chris URL \n\n<!-- NEXT TEST -->\nMessage-ID: <mid7@foobar.org>\nSubject: hi thank you for the useful replies\n\ni have found some interesting tutorials in the ibm developer connection URL and URL registration is needed i will post the same message on the web application security list as suggested by someone for now i thing i will use mdNUMBER for password checking i will use the approach described in secure programmin fo linux and unix how to i will separate the authentication module so i can change its implementation at anytime thank you again mario torre please avoid sending me word or powerpoint attachments see URL \n\n<!-- NEXT TEST -->\nMessage-ID: <mid8@foobar.org>\nSubject: hehe sorry\n\nbut if you hit caps lock twice the computer crashes theres one ive never heard before have you tryed dell support yet i think dell computers prefer redhat dell provide some computers pre loaded with red hat i dont know for sure tho so get someone elses opnion as well as mine original message from ilug admin URL mailto ilug admin URL on behalf of peter staunton sent NUMBER august NUMBER NUMBER NUMBER to ilug URL subject ilug newbie seeks advice suse NUMBER NUMBER folks my first time posting have a bit of unix experience but am new to linux just got a new pc at home dell box with windows xp added a second hard disk for linux partitioned the disk and have installed suse NUMBER NUMBER from cd which went fine except it didn t pick up my monitor i have a dell branded eNUMBERfpp NUMBER lcd flat panel monitor and a nvidia geforceNUMBER tiNUMBER video card both of which are probably too new to feature in suse s default set i downloaded a driver from the nvidia website and installed it using rpm then i ran saxNUMBER as was recommended in some postings i found on the net but it still doesn t feature my video card in the available list what next another problem i have a dell branded keyboard and if i hit caps lock twice the whole machine crashes in linux not windows even the on off switch is inactive leaving me to reach for the power cable instead if anyone can help me in any way with these probs i d be really grateful i ve searched the net but have run out of ideas or should i be going for a different version of linux such as redhat opinions welcome thanks a lot peter irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL\n\n<!-- NEXT TEST -->\nMessage-ID: <mid9@foobar.org>\nSubject: it will function as a router\n\nif that is what you wish it even looks like the modem s embedded os is some kind of linux being that it has interesting interfaces like ethNUMBER i don t use it as a router though i just have it do the absolute minimum dsl stuff and do all the really fun stuff like pppoe on my linux box also the manual tells you what the default password is don t forget to run pppoe over the alcatel speedtouch NUMBERi as in my case you have to have a bridge configured in the router modem s software this lists your vci values etc also does anyone know if the high end speedtouch with NUMBER ethernet ports can act as a full router or do i still need to run a pppoe stack on the linux box regards vin irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL \n\n<!-- NEXT TEST -->\nMessage-ID: <mid10@foobar.org>\nSubject: all is it just me\n\nor has there been a massive increase in the amount of email being falsely bounced around the place i ve already received email from a number of people i don t know asking why i am sending them email these can be explained by servers from russia and elsewhere coupled with the false emails i received myself it s really starting to annoy me am i the only one seeing an increase in recent weeks martin martin whelan déise design URL tel NUMBER NUMBER our core product déiseditor allows organisations to publish information to their web site in a fast and cost effective manner there is no need for a full time web developer as the site can be easily updated by the organisations own staff instant updates to keep site information fresh sites which are updated regularly bring users back visit URL for a demonstration déiseditor managing your information _______________________________________________ iiu mailing list iiu URL URL ,0\n<!-- NEXT TEST -->\n"
  },
  {
    "path": "tests/resources/smtp/antispam/classifier.spam",
    "content": "Subject: save up to NUMBER on life insurance\n\nwhy spend more than you have to life quote savings ensuring your family s financial security is very important life quote savings makes buying life insurance simple and affordable we provide free access to the very best companies and the lowest rates life quote savings is fast easy and saves you money let us help you get started with the best values in the country on new coverage you can save hundreds or even thousands of dollars by requesting a free quote from lifequote savings our service will take you less than NUMBER minutes to complete shop and compare save up to NUMBER on all types of life insurance hyperlink click here for your free quote protecting your family is the best investment you ll ever make if you are in receipt of this email in error and or wish to be removed from our list hyperlink please click here and type remove if you reside in any state which prohibits e mail solicitations for insurance please disregard this email\n\n<!-- NEXT TEST -->\nSubject: a powerhouse gifting program\n\nyou don t want to miss get in with the founders the major players are on this one for once be where the players are this is your private invitation experts are calling this the fastest way to huge cash flow ever conceived leverage NUMBER NUMBER into NUMBER NUMBER over and over again the question here is you either want to be wealthy or you don t which one are you i am tossing you a financial lifeline and for your sake i hope you grab onto it and hold on tight for the ride of your life testimonials hear what average people are doing their first few days we ve received NUMBER NUMBER in NUMBER day and we are doing that over and over again q s in al i m a single mother in fl and i ve received NUMBER NUMBER in the last NUMBER days d s in fl i was not sure about this when i sent off my NUMBER NUMBER pledge but i got back NUMBER NUMBER the very next day l l in ky i didn t have the money so i found myself a partner to work this with we have received NUMBER NUMBER over the last NUMBER days i think i made the right decision don t you k c in fl i pick up NUMBER NUMBER my first day and i they gave me free leads and all the training you can too j w in ca announcing we will close your sales for you and help you get a fax blast immediately upon your entry you make the money free leads training don t wait call now fax back to NUMBER NUMBER NUMBER NUMBER or call NUMBER NUMBER NUMBER NUMBER name__________________________________phone___________________________________________ fax_____________________________________email____________________________________________ best time to call_________________________time zone________________________________________ this message is sent in compliance of the new e mail bill per section NUMBER paragraph a NUMBER c of s NUMBER further transmissions by the sender of this email may be stopped at no cost to you by sending a reply to this email address with the word remove in the subject line errors omissions and exceptions excluded this is not spam i have compiled this list from our replicate database relative to seattle marketing group the gigt or turbo team for the sole purpose of these communications your continued inclusion is only by your gracious permission if you wish to not receive this mail from me please send an email to tesrewinter URL with remove in the subject and you will be deleted immediately\n\n<!-- NEXT TEST -->\nSubject: help wanted \n\nwe are a NUMBER year old fortune NUMBER company that is growing at a tremendous rate we are looking for individuals who want to work from home this is an opportunity to make an excellent income no experience is required we will train you so if you are looking to be employed from home with a career that has vast opportunities then go URL we are looking for energetic and self motivated people if that is you than click on the link and fill out the form and one of our employement specialist will contact you to be removed from our link simple go to URL \n\n<!-- NEXT TEST -->\nSubject: tired of the bull out there\n\nwant to stop losing money want a real money maker receive NUMBER NUMBER NUMBER NUMBER today experts are calling this the fastest way to huge cash flow ever conceived a powerhouse gifting program you don t want to miss we work as a team this is your private invitation get in with the founders this is where the big boys play the major players are on this one for once be where the players are this is a system that will drive NUMBER NUMBER s to your doorstep in a short period of time leverage NUMBER NUMBER into NUMBER NUMBER over and over again the question here is you either want to be wealthy or you don t which one are you i am tossing you a financial lifeline and for your sake i hope you grab onto it and hold on tight for the ride of your life testimonials hear what average people are doing their first few days we ve received NUMBER NUMBER in NUMBER day and we are doing that over and over again q s in al i m a single mother in fl and i ve received NUMBER NUMBER in the last NUMBER days d s in fl i was not sure about this when i sent off my NUMBER NUMBER pledge but i got back NUMBER NUMBER the very next day l l in ky i didn t have the money so i found myself a partner to work this with we have received NUMBER NUMBER over the last NUMBER days i think i made the right decision don t you k c in fl i pick up NUMBER NUMBER my first day and i they gave me free leads and all the training you can too j w in ca this will be the most important call you make this year free leads training announcing we will close your sales for you and help you get a fax blast immediately upon your entry you make the money free leads training don t wait call now NUMBER NUMBER NUMBER NUMBER print and fax to NUMBER NUMBER NUMBER NUMBER or send an email requesting more information to successleads URL please include your name and telephone number receive NUMBER NUMBER free leads just for responding a NUMBER NUMBER value name___________________________________ phone___________________________________ fax_____________________________________ email___________________________________ this message is sent in compliance of the new e mail bill per section NUMBER paragraph a NUMBER c of s NUMBER further transmissions by the sender of this email may be stopped at no cost to you by sending a reply to this email address with the word remove in the subject line errors omissions and exceptions excluded this is not spam i have compiled this list from our replicate database relative to seattle marketing group the gigt or turbo team for the sole purpose of these communications your continued inclusion is only by your gracious permission if you wish to not receive this mail from me please send an email to tesrewinter URL with remove in the subject and you will be deleted immediately\n\n<!-- NEXT TEST -->\nSubject: cellular phone accessories \n\nall at below wholesale prices http NUMBER NUMBER NUMBER NUMBER NUMBER sites merchant sales hands free ear buds NUMBER NUMBER phone holsters NUMBER NUMBER booster antennas only NUMBER NUMBER phone cases NUMBER NUMBER car chargers NUMBER NUMBER face plates as low as NUMBER NUMBER lithium ion batteries as low as NUMBER NUMBER http NUMBER NUMBER NUMBER NUMBER NUMBER sites merchant sales click below for accessories on all nokia motorola lg nextel samsung qualcomm ericsson audiovox phones at below wholesale prices http NUMBER NUMBER NUMBER NUMBER NUMBER sites merchant sales if you need assistance please call us NUMBER NUMBER NUMBER to be removed from future mailings please send your remove request to remove me now NUMBER URL thank you and have a super day\n\n<!-- NEXT TEST -->\nSubject: conferencing made easy\n\nonly NUMBER cents per minute including long distance no setup fees no contracts or monthly fees call anytime from anywhere to anywhere connects up to NUMBER participants simplicity in set up and administration operator help available NUMBER NUMBER the highest quality service for the lowest rate in the industry fill out the form below to find out how you can lower your phone bill every month required input field name web address company name state business phone home phone email address type of business to be removed from our distribution lists please hyperlink click here\n\n<!-- NEXT TEST -->\nSubject: dear friend\n\ni am mrs sese seko widow of late president mobutu sese seko of zaire now known as democratic republic of congo drc i am moved to write you this letter this was in confidence considering my presentcircumstance and situation i escaped along with my husband and two of our sons george kongolo and basher out of democratic republic of congo drc to abidjan cote d ivoire where my family and i settled while we later moved to settled in morroco where my husband later died of cancer disease however due to this situation we decided to changed most of my husband s billions of dollars deposited in swiss bank and other countries into other forms of money coded for safe purpose because the new head of state of dr mr laurent kabila has made arrangement with the swiss government and other european countries to freeze all my late husband s treasures deposited in some european countries hence my children and i decided laying low in africa to study the situation till when things gets better like now that president kabila is dead and the son taking over joseph kabila one of my late husband s chateaux in southern france was confiscated by the french government and as such i had to change my identity so that my investment will not be traced and confiscated i have deposited the sum eighteen million united state dollars us NUMBER NUMBER NUMBER NUMBER with a security company for safekeeping the funds are security coded to prevent them from knowing the content what i want you to do is to indicate your interest that you will assist us by receiving the money on our behalf acknowledge this message so that i can introduce you to my son kongolo who has the out modalities for the claim of the said funds i want you to assist in investing this money but i will not want my identity revealed i will also want to buy properties and stock in multi national companies and to engage in other safe and non speculative investments may i at this point emphasise the high level of confidentiality which this business demands and hope you will not betray the trust and confidence which i repose in you in conclusion if you want to assist us my son shall put you in the picture of the business tell you where the funds are currently being maintained and also discuss other modalities including remunerationfor your services for this reason kindly furnish us your contact information that is your personal telephone and fax number for confidential URL regards mrs m sese seko\n\n<!-- NEXT TEST -->\nSubject: lowest rates available for term life insurance\n\ntake a moment and fill out our online form to see the low rate you qualify for save up to NUMBER from regular rates smokers accepted URL representing quality nationwide carriers act now to easily remove your address from the list go to URL please allow NUMBER NUMBER hours for removal\n\n<!-- NEXT TEST -->\nSubject: central bank of nigeria foreign remittance \n\ndept tinubu square lagos nigeria email smith_j URL NUMBERth of august NUMBER attn president ceo strictly private business proposal i am mr johnson s abu the bills and exchange director at the foreignremittance department of the central bank of nigeria i am writingyou this letter to ask for your support and cooperation to carrying thisbusiness opportunity in my department we discovered abandoned the sumof us NUMBER NUMBER NUMBER NUMBER thirty seven million four hundred thousand unitedstates dollars in an account that belong to one of our foreign customers an american late engr john creek junior an oil merchant with the federal government of nigeria who died along with his entire family of a wifeand two children in kenya airbus aNUMBER NUMBER flight kqNUMBER in novemberNUMBER since we heard of his death we have been expecting his next of kin tocome over and put claims for his money as the heir because we cannotrelease the fund from his account unless someone applies for claims asthe next of kin to the deceased as indicated in our banking guidelines unfortunately neither their family member nor distant relative hasappeared to claim the said fund upon this discovery i and other officialsin my department have agreed to make business with you release the totalamount into your account as the heir of the fund since no one came forit or discovered either maintained account with our bank other wisethe fund will be returned to the bank treasury as unclaimed fund we have agreed that our ratio of sharing will be as stated thus NUMBER for you as foreign partner and NUMBER for us the officials in my department upon the successful completion of this transfer my colleague and i willcome to your country and mind our share it is from our NUMBER we intendto import computer accessories into my country as way of recycling thefund to commence this transaction we require you to immediately indicateyour interest by calling me or sending me a fax immediately on the abovetelefax and enclose your private contact telephone fax full nameand address and your designated banking co ordinates to enable us fileletter of claim to the appropriate department for necessary approvalsbefore the transfer can be made note also this transaction must be kept strictly confidential becauseof its nature nb please remember to give me your phone and fax no mr johnson smith abu irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL\n\n<!-- NEXT TEST -->\nSubject: dear stuart\n\nare you tired of searching for love in all the wrong places find love now at URL URL browse through thousands of personals in your area join for free URL search e mail chat use URL to meet cool guys and hot girls go NUMBER on NUMBER or use our private chat rooms click on the link to get started URL find love now you have received this email because you have registerd with emailrewardz or subscribed through one of our marketing partners if you have received this message in error or wish to stop receiving these great offers please click the remove link above to unsubscribe from these mailings please click here URL\n\n<!-- NEXT TEST -->\n"
  },
  {
    "path": "tests/resources/smtp/antispam/classifier.test",
    "content": "envelope_to hello@world.com\nexpect PROB_SPAM_HIGH\n\nSubject: save up to NUMBER on life insurance\n\nwhy spend more than you have to life quote savings ensuring your family s financial security is very important life quote savings makes buying life insurance simple and affordable we provide free access to the very best companies and the lowest rates life quote savings is fast easy and saves you money let us help you get started with the best values in the country on new coverage you can save hundreds or even thousands of dollars by requesting a free quote from lifequote savings our service will take you less than NUMBER minutes to complete shop and compare save up to NUMBER on all types of life insurance hyperlink click here for your free quote protecting your family is the best investment you ll ever make if you are in receipt of this email in error and or wish to be removed from our list hyperlink please click here and type remove if you reside in any state which prohibits e mail solicitations for insurance please disregard this email\n\n<!-- NEXT TEST -->\nenvelope_to hello@world.com\nexpect PROB_HAM_HIGH\n\nSubject: can someone explain\n\nwhat type of operating system solaris is as ive never seen or used it i dont know wheather to get a server from sun or from dell i would prefer a linux based server and sun seems to be the one for that but im not sure if solaris is a distro of linux or a completely different operating system can someone explain kiall mac innes irish linux users group ilug URL URL for un subscription information list maintainer listmaster URL \n<!-- NEXT TEST -->\nenvelope_to hello@world.com\nexpect PROB_SPAM_UNCERTAIN\n\nSubject: Lorem ipsum dolor sit amet, consectetur adipiscing elit\n\nsed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n\n"
  },
  {
    "path": "tests/resources/smtp/antispam/classifier_features.test",
    "content": "From: bill@example.com\nTo: jdoe@example.com\nSubject: TPS Report\n\nI'm going to need those TPS reports ASAP. So, if you could do that, that'd be great.\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"_allcaps\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"asap\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"could\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"go\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"great\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"need\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"report\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"tps\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"bill@example.com\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"example.com\"\n  }\n]\n<!-- NEXT TEST -->\nFrom: Hendrik <hendrik@example.com>\nTo: Harrie <harrie@example.com>\nDate: Sat, 11 Oct 2010 00:31:44 +0200\nSubject: One Two Three Four\nContent-Type: multipart/mixed; boundary=AA\n\nThis is a multi-part message in MIME format.\n--AA\nContent-Type: multipart/mixed; boundary=BB\n\nThis is a multi-part message in MIME format.\n--BB\nContent-Type: text/plain; charset=\"us-ascii\"\n\nThis is the first message part containing\nplain text. \n\n--BB\nContent-Type: text/plain; charset=\"us-ascii\"\n\nThis is another plain text message part.\n\n--BB--\nThis is the end of MIME multipart.\n\n--AA\nContent-Type: text/html; charset=\"us-ascii\"\n\n<html>\n<body>This is a piece of HTML text.</body>\n</html>\n\n--AA--\nThis is the end of  MIME multipart.\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"anoth\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"contain\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"first\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"four\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"html\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"messag\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"one\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"part\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"piec\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"plain\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"text\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"three\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"two\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"example.com\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"hendrik@example.com\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"multipart/mixed\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  }\n]\n<!-- NEXT TEST -->\nContent-Type: text/html; charset=\"utf-8\"\nSubject: IPs in HTML are not urls\n\n<html>\nDas System wurde um 01.01.1970 08:28:00 für die IP-Adresse\n123.123.123.123 gesperrt.<br>\n<br>\nDer Besucher hat versucht, sich mit folgenden Daten anzumelden.<br>\nPartner: 12345678<br>\nPortal: <a href=\"https://www.localhost.de/example.php\" target=\"_blank\">IP-Sperre einsehen</a>\n</html>\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"adress\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"anzumeld\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"besuch\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"dat\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"einseh\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"folgend\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"gesperrt\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"html\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ip\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"partn\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"portal\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sperr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"syst\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"url\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"versucht\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"wurd\"\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      2\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      4\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      8\n    ]\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"!ip\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_example\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_php\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"localhost.de\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"www.localhost.de\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"html_anchor\",\n    \"href\": \"https\"\n  }\n]\n<!-- NEXT TEST -->\nX-Spam-Result: DMARC_POLICY_ALLOW (-0.50),\n    TEST (0.0),\n\tSOURCE_ASN_123 (1.00)\nFrom: Client Services <noreply@tetheer.com>\nTo: user@domain.org\nSubject: Tether Important Update !\nContent-Type: text/html\nContent-Transfer-Encoding: quoted-printable\n\n<!DOCTYPE HTML>\n\n<html><head><title></title>\n<meta http-equiv=3D\"X-UA-Compatible\" content=3D\"IE=3Dedge\">\n</head>\n<body style=3D\"margin: 0.4em;\"><a title=3D\"CASHBACK_REWARDS\" style=3D'text-=\ntransform: none; text-indent: 0px; letter-spacing: normal; font-family: \"Ti=\nmes New Roman\"; font-size: medium; font-style: normal; font-weight: 400; wo=\nrd-spacing: 0px; white-space: normal; orphans: 2; widows: 2; font-variant-l=\nigatures: normal; font-variant-caps: normal; -webkit-text-stroke-width: 0px=\n;' href=3D\"https://metaskwap.online/\" target=3D\"_blank\" rel=3D\"noopener\">\n<img width=3D\"55%\" style=3D\"margin-right: auto; margin-left: auto; float: l=\neft; display: block;\" alt=3D\"If you can't read this message please click he=\nre to open it in your browser.\"=20\nsrc=3D\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABFAAAAV4CAYAAACHDynTAA=\nAgAElEQVR4XuzdB2CV1fnH8SeDJOwpICCgoAgOFAQXOIqKOLFu68BWBbVarXsPnDjr+rfWqnVvF=\nOteuCc4AZGN7L3Jzv953ptzc3Nzb3JvBpybfF9KgeQd5/2cN2nfX855TlqJbsKGAAIIIIAAAggg=\n\"></a></body></html>\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"_allcaps\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"_null\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"browser\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"cashback\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"click\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"import\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"messag\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"open\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"pleas\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"read\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"reward\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"tether\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"updat\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"noreply@tetheer.com\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"tetheer.com\"\n  },\n  {\n    \"type\": \"asn\",\n    \"number\": [\n      0,\n      0,\n      0,\n      123\n    ]\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"metaskwap.online\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"html_image\",\n    \"src\": \"data\"\n  },\n  {\n    \"type\": \"html_anchor\",\n    \"href\": \"https\"\n  }\n]\n<!-- NEXT TEST -->\nFrom: \"BBVA\" <noreply@grupokonecta.net>\nReply-To: noreply@grupokonecta.net\nContent-Type: multipart/alternative; charset=\"UTF-8\"; boundary=\"b1_3d217f30a568faa9ce3dd7dc73399561\"\nContent-Transfer-Encoding: quoted-printable\n\n--b1_3d217f30a568faa9ce3dd7dc73399561\nContent-Type: text/plain; format=flowed; charset=\"UTF-8\"\nContent-Transfer-Encoding: quoted-printable\n\nTarjeta de cr=C3=A9dito BBVA\n\nLa tarjeta de cr=C3=A9dito para viajar con tus consumos\nPedila 100% online y empez=C3=A1 a disfrutar\n\nConocer oferta\nUn mundo de beneficios con las tarjetas de cr=C3=A9dito BBVA\ncompras en cuotas\nCompras en cuotas\nPod=C3=A9s disfrutar hoy de los productos =E2=80=A8que quer=C3=A9s y pagarl=\nos en cuotas\ndescuentos y reintegros\nDescuentos y reintegros\nEntretenimiento, gastronom=C3=ADa, farmacia, ropa =E2=80=A8y m=C3=A1s rubro=\ns con promociones exclusivas\npuntos bbva\nViajes con Puntos BBVA\nVuelos, alojamientos y mucho m=C3=A1s canjeando Puntos BBVA que sum=C3=\n=A1s con tus compras\nConocer oferta\nDescubr=C3=AD la tarjeta que mejor se adapta a vos\nTodas las tarjetas\n\nBlack\n\nPlatinum\n\nGold\n\nInternacional\n\nTodas las tarjetas\nvisa black\nTarjeta Visa Signature\nL=C3=ADmites desde $600.000\n\n15% extra en acumulaci=C3=B3n de Puntos BBVA\nAcceso a salas VIP en aeropuertos\nAsistencia en viajes con cobertura de hasta 250.000 USD\nExtracci=C3=B3n de efectivo en el exterior\nSeguro de robo en cajero y compra protegida\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $200.000\n\n Conocer m=C3=A1s\nmastercard black\nTarjeta Mastercard Black\nL=C3=ADmites desde $600.000\n\n15% extra en acumulaci=C3=B3n de Puntos BBVA\nAcceso a salas VIP en aeropuertos\nAsistencia en viajes con cobertura de hasta 250.000 USD\nExtracci=C3=B3n de efectivo en el exterior\nSeguro de robo en cajero y compra protegida\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $200.000\n Conocer m=C3=A1s\ntarjeta platinum visa\nTarjeta Visa Platinum\nL=C3=ADmites desde $350.000\n\n5% extra en acumulaci=C3=B3n de Puntos BBVA\nAsistencia en viajes con cobertura de hasta 170.000 USD\nExtracci=C3=B3n de efectivo en el exterior\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $120.000\n\n Conocer m=C3=A1s\ntarjeta platinum mastercard\nTarjeta Mastercard Platinum\nL=C3=ADmites desde $350.000\n\n5% extra en acumulaci=C3=B3n de Puntos BBVA\nAsistencia en viajes con cobertura de hasta 50.000 USD y 30.000 EUR\nExtracci=C3=B3n de efectivo en el exterior\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $120.000\n\n Conocer m=C3=A1s\ntarjeta gold visa\nTarjeta Visa Gold\nL=C3=ADmites desde $100.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $20.000\n\n Conocer m=C3=A1s\ntarjeta gold mastercard\nTarjeta Mastercard Gold\nL=C3=ADmites desde $100.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $35.000\n\n Conocer m=C3=A1s\n\nTarjeta Visa Internacional\nL=C3=ADmites desde $10.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $20.000\n\n Conocer m=C3=A1s\n\nTarjeta Mastercard Internacional\nL=C3=ADmites desde $10.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $20.000\n\n Conocer m=C3=A1s\n\n--b1_3d217f30a568faa9ce3dd7dc73399561\nContent-Type: text/html; charset=\"UTF-8\"\nContent-Transfer-Encoding: quoted-printable\n\n<head>\n</head>\n\n<body>\n<div style=3D\"max-width: 600px; margin: 0 auto;\">\n<table border=3D\"0\" cellspacing=3D\"0\" cellpadding=3D\"0\">\n<tbody>\n<tr><th><a href=3D\"http://track.leadsinbx.com/aff_c?offer_id=3D2906&amp;aff=\n_id=3D1980&amp;url_id=3D21948\"> <img src=3D\"https://i.imgur.com/1wg1VQH.jpg=\n\" width=3D\"100%\" alt=3D\"CLICK AQU&Iacute; CLICK AQU&Iacute; CLICK AQU&Iacut=\ne; CLICK AQU&Iacute; CLICK AQU&Iacute; CLICK AQU&Iacute; CLICK AQU&Iacute;\"=\n title=3D\"Es la oportunidad que estabas esperando\" /> </a></th></tr>\n</tbody>\n</table>\n</div>\n</body>\n</html>\n\n--b1_3d217f30a568faa9ce3dd7dc73399561--\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"_allcaps\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"_null\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"acces\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"acumul\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"adapt\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"adicional\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"aeropuert\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"aloj\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"aqu\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"asistent\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"atencion\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bbva\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"benefici\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"black\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"cajer\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"canj\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"click\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"cobertur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"compr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"conoc\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"consult\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"consum\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"cost\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"credit\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"cuot\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"descubr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"descuent\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"disfrut\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"efect\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"empez\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"entreten\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"es\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"esperando\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"estaba\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"eur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"exclus\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"exterior\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"extra\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"extraccion\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"farmaci\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"gastronom\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"gold\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"hoy\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"iacut\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ingres\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"internacional\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"la\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"limit\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"mastercard\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"mejor\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"mensual\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"minim\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"mund\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"necesari\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ofert\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"onlin\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"oportunidad\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"pag\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"pedil\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"personaliz\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"platinum\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"podes\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"product\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"promocion\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"proteg\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"punt\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"que\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"queres\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"reintegr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"resolv\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"rob\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"rop\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"rubr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sal\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"segur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"signatur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sumas\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"tarjet\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"tod\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"usd\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"viaj\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"vip\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"vis\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"vos\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"vuel\"\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      1\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      2\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      3\n    ]\n  },\n  {\n    \"type\": \"unicode_category\",\n    \"value\": \"Sc\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"grupokonecta.net\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"noreply@grupokonecta.net\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_aff\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_jpg\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"i.imgur.com\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"imgur.com\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"leadsinbx.com\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"track.leadsinbx.com\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"multipart/alternative\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  },\n  {\n    \"type\": \"html_image\",\n    \"src\": \"https\"\n  },\n  {\n    \"type\": \"html_anchor\",\n    \"href\": \"http\"\n  }\n]\n<!-- NEXT TEST -->\nFrom: Spammer Systems Iran <marketing@spammer.ir>\nSubject: =?utf-8?b?2KfZgdiy2YjZhtmH4oCM2YfYp9uMINin2LPZhdin2LHYqtix2YXbjNmEIHw=?=\n =?utf-8?b?INiq2YjYs9i52Ycg24zYp9mB2KrZhyDYqtmI2LPYtyDYotix2qnYpw==?=\nMessage-Id: <b43032037c3340adaa1f7fefb9054e34@6abf43e0d73f4ed18bb03142dd21e81c>\nTo: spam@target.org\nReply-To: Spammer Systems Iran <marketing@spammer.ir>\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: quoted-printable\n\nvＥⓡ𝔂 𝔽𝕌Ňℕｙ ţ乇𝕏𝓣\n\n=D8=A7=D9=81=D8=B2=D9=88=D9=86=D9=87=E2=80=8C=D9=87=D8=A7=DB=8C SpammerMail=\n =D8=AA=D9=88=D8=B3=D8=B9=D9=87 =DB=8C=D8=A7=D9=81=D8=AA=D9=87 =D8=AA=D9=88=\n=D8=B3=D8=B7 =D8=A2=D8=B1=DA=A9=D8=A7\n\n=D8=B1=D8=A7=DB=8C=D8=A7=D9=86 =D8=B3=D8=A7=D9=85=D8=A7=D9=86=D9=87 =D8=A2=\n=D8=B1=DA=A9=D8=A7 | =D8=AA=D9=85=D8=A7=D8=B3: 91300476-021 | =D8=A7=DB=8C=\n=D9=85=DB=8C=D9=84: info@spammy.ir\n\n[Telegram]\t[Instagram]\t[LinkedIn]\t[Email]\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"email\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"funny\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"instagram\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"linkedin\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"spammermail\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"telegram\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"text\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"very\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"آرکا\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"اسمارترمی\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"افزونه\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ایمیل\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"تماس\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"توسط\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"توسعه\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"رایان\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"سامانه\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"های\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"یافته\"\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      3\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      8\n    ]\n  },\n  {\n    \"type\": \"unicode_category\",\n    \"value\": \"Cf\"\n  },\n  {\n    \"type\": \"unicode_category\",\n    \"value\": \"Sm\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"marketing@spammer.ir\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"spammer.ir\"\n  },\n  {\n    \"type\": \"email\",\n    \"value\": \"info@spammy.ir\"\n  },\n  {\n    \"type\": \"email\",\n    \"value\": \"spammy.ir\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  }\n]\n<!-- NEXT TEST -->\nReceived: from localhost ([217.61.8.72])\n\tby Consip with ESMTP\n\tid PMLhve2ETFdIAPMLyvhh0c; Sat, 29 Nov 2025 15:55:14 +0100\nReceived: from zspmta-mint02.ad.aruba.it ([127.0.0.1])\n by localhost (zspmta-mint02.ad.aruba.it [127.0.0.1]) (amavis, port 10026)\n with ESMTP id UV6fqMysWqKE; Sat, 29 Nov 2025 15:55:13 +0100 (CET)\nReceived: from zspmbx-mint11.ad.aruba.it (unknown [10.202.133.51])\n\tby zspmta-mint02.ad.aruba.it (Postfix) with ESMTP id 3042B120F77;\n\tSat, 29 Nov 2025 15:54:59 +0100 (CET)\nDate: Sat, 29 Nov 2025 15:54:59 +0100 (CET)\nFrom: gianfranco.mangini@interno.it\nReply-To: \"Hr. Charles Jackson Jr.\" <ferassutti34@gmail.com>\nMessage-ID: <1933878358.10239097.1764428099117.JavaMail.zimbra@interno.it>\nSubject: \nContent-Type: multipart/alternative; \n\tboundary=\"=_bf54163b-f3b6-421f-bc9d-b64439167a39\"\n\n--=_bf54163b-f3b6-421f-bc9d-b64439167a39\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: quoted-printable\n\n\n\nHvorfor har du ikke modtaget donationen p=C3=A5 =E2=82=AC955.000,00 fra hr.=\n Charles Jackson Jr.? Bankdirekt=C3=B8ren informerede mig i g=C3=A5r om, at=\n en af =E2=80=8B=E2=80=8Bmodtagerne ikke havde gjort krav p=C3=A5 donatione=\nn. Efter at have gennemg=C3=A5et mine optegnelser opdagede jeg, at du var b=\nlandt de ber=C3=B8rte, og jeg er meget ked af at h=C3=B8re dette. Bem=C3=A6=\nrk venligst, at der ikke kr=C3=A6ves nogen betaling; et simpelt bekr=C3=A6f=\ntelsesstempel er alt, hvad der skal til for at pengene kan frigives og kred=\niteres din bankkonto inden for 24 timer.=20\n\nBem=C3=A6rk: For yderligere information og for at sikre, at din donation kr=\nediteres inden for 24 timer, anbefaler jeg, at du sender mig dine oplysning=\ner med det samme via e-mail til ferassutti34@gmail.com=20\n\nJeg =C3=B8nsker dig en velsignet m=C3=A5ned med stor succes.=20\nHr. Charles Jackson Jr.=20\n\n--=_bf54163b-f3b6-421f-bc9d-b64439167a39\nContent-Type: text/html; charset=utf-8\nContent-Transfer-Encoding: quoted-printable\n\n<html><body><div style=3D\"font-family: arial, helvetica, sans-serif; font-s=\nize: 12pt; color: #000000\"><div><br></div><div><br></div><div data-marker=\n=3D\"__SIG_PRE__\">Hvorfor har du ikke modtaget donationen p=C3=A5 =E2=82=AC9=\n55.000,00 fra hr. Charles Jackson Jr.? Bankdirekt=C3=B8ren informerede mig =\ni g=C3=A5r om, at en af =E2=80=8B=E2=80=8Bmodtagerne ikke havde gjort krav =\np=C3=A5 donationen. Efter at have gennemg=C3=A5et mine optegnelser opdagede=\n jeg, at du var blandt de ber=C3=B8rte, og jeg er meget ked af at h=C3=B8re=\n dette. Bem=C3=A6rk venligst, at der ikke kr=C3=A6ves nogen betaling; et si=\nmpelt bekr=C3=A6ftelsesstempel er alt, hvad der skal til for at pengene kan=\n frigives og krediteres din bankkonto inden for 24 timer.<br><br>Bem=C3=A6r=\nk: For yderligere information og for at sikre, at din donation krediteres i=\nnden for 24 timer, anbefaler jeg, at du sender mig dine oplysninger med det=\n samme via e-mail til ferassutti34@gmail.com<br><br>Jeg =C3=B8nsker dig en =\nvelsignet m=C3=A5ned med stor succes.<br>Hr. Charles Jackson Jr.</div></div=\n></body></html>\n--=_bf54163b-f3b6-421f-bc9d-b64439167a39--\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"anbefal\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bankdirektør\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bankkonto\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bekræftelsesstem\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bemærk\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"berørt\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"betaling\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bland\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"charl\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"din\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"donation\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"e\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"frigiv\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"gennemgå\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"gjort\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"går\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"hr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"hvorfor\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"hør\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ind\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"inform\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"information\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"jackson\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"jr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"kan\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ked\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"krav\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"kredit\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"kræv\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"mail\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"modtag\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"måned\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"nog\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"opdaged\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"oplysning\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"optegn\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"peng\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sam\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"send\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sikr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"simpelt\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"stor\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"suc\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"tim\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"velsign\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ven\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"via\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"yder\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"ønsk\"\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      2\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      3\n    ]\n  },\n  {\n    \"type\": \"unicode_category\",\n    \"value\": \"Cf\"\n  },\n  {\n    \"type\": \"unicode_category\",\n    \"value\": \"Sc\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"ferassutti34@gmail.com\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"gianfranco.mangini@interno.it\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"gmail.com\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"interno.it\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"aruba.it\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"interno.it\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"zspmbx-mint11.ad.aruba.it\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"zspmta-mint02.ad.aruba.it\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"multipart/alternative\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  }\n]\n<!-- NEXT TEST -->\nDelivered-To: mcfadden@domain.com\nReceived: from gamma.stellaryx.space (unknown [85.120.227.61] (AS6718 NAV COMMUNICATIONS SRL, RO))\n\tby mail.stalw.art (Stalwart SMTP) with ESMTP id 3D6018102E32AFD;\n\tTue, 25 Nov 2025 14:56:37 +0000\nReturn-Path: <102356-235606-568806-22158-mcfadden=domain.com@mail.stellaryx.space>\nContent-Type: multipart/alternative; boundary=\"4521ddb80d67f83dc7585cae40234a09_39856_8ade6\"\nDate: Tue, 25 Nov 2025 15:56:05 +0100\nFrom: \"ZenFluff\" <FluffCoPartner@stellaryx.space>\nReply-To: \"ZenFluff\" <FluffCoPromo@stellaryx.space>\nSubject: Sleep better with FluffCo\nTo: <mcfadden@domain.com>\nMessage-ID: <peiqzch3l7lcfmzv-saxvjtk6vnighwql-39856-8ade6@stellaryx.space>\n\n--4521ddb80d67f83dc7585cae40234a09_39856_8ade6\nContent-Type: text/plain;\nContent-Transfer-Encoding: 8bit\n\nSleep better with FluffCo\n\nhttp://stellaryx.space/Bm6NYOrhicX--9bFn47T2mFlb-Soxhs-FJ8RCnM_SXJyHmebLw\n \nhttp://stellaryx.space/Y46ntHIiWyxTreKwOcyT4txY9f-M-eCwfHuhx0SMoyGjXy1iuA\n\n--4521ddb80d67f83dc7585cae40234a09_39856_8ade6\nContent-Type: text/html;\nContent-Transfer-Encoding: 8bit\n\n<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n<html>\n<head>\n\t<title>Newsletter</title>\n\t<meta content=\"text/html;charset=utf-8\" http-equiv=\"content-Type\">\n</head>\n<body>\n<center><a href=\"http://stellaryx.space/vJeynklMB-MqGKWLe690VqFxB-Ntp0AlJ0fxkvT7yQc5lwBiGQ\"><img src=\"http://stellaryx.space/4de461d36a49b6c16a.jpg\" /><img height=\"1\" src=\"http://www.stellaryx.space/gDLqH1_yixRUwfwFaFPi6qQIY1WYIXRnhpkbB5CIxKEtmAQaWQ\" width=\"1\" /></a>\n<div style=\"padding:10px;width:620px;font-family:Georgia;\"><a href=\"http://stellaryx.space/Bm6NYOrhicX--9bFn47T2mFlb-Soxhs-FJ8RCnM_SXJyHmebLw\" style=\"font-size:26px;font-weight:bold;color:#810B1B;\" target=\"_blank\"><b>Sleep better with FluffCo</b></a><br />\n<br />\n<a href=\"http://stellaryx.space/Bm6NYOrhicX--9bFn47T2mFlb-Soxhs-FJ8RCnM_SXJyHmebLw\" rel=\"sponsored\" target=\"_blank\"><img alt=\" \" src=\"http://stellaryx.space/c3276c4707df73d171.jpg\" /><img alt=\" \" src=\"http://stellaryx.space/de2b61ecd393e24275.jpg\" /></a><br />\n<br />\n<br />\n<br />\n<a href=\"http://stellaryx.space/IUf5_E6B7M10afXhAXm2s-OMucE97vSt_GrBmM2mukYMqBJo3w\" rel=\"sponsored\" target=\"blank\"><img alt=\" \" src=\"http://stellaryx.space/54ed2c4f9cfcf0e61c.jpg\" /> </a><br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<a href=\"http://stellaryx.space/Y46ntHIiWyxTreKwOcyT4txY9f-M-eCwfHuhx0SMoyGjXy1iuA\" http:=\"\" microsoft.com=\"\" rel=\"sponsored\" target=\"_blank\"><img alt=\" \" http:=\"\" microsoft.com=\"\" src=\"http://stellaryx.space/fff6b20f9b31af5b8e.jpg\" /></a><br />\n<br />\n<br />\n<br />\n<br />\n<!--\nEisteddfa cheese frame industrial cat threw tape canal cotton fewer Gurig is a highway useful wolf hamlet located in Ceredigion on its border poem film carried mainly happy written blood clothes with Powys and situated along the A44. texas Its name comes from the Welsh word eisteddfa, meaning seat and the name of St Curig. This gives the meaning of Curigs seat. It arrange secret sang gas leather station is said that Curig rested on the hill here and looked manner habit sugar doesn't spend guess down into the Wye Valley. He decided to build a church in the valley, which is still there today in the village of Llangurig.91193\n\n brown adventure felt apart mill he famous Elvis Rock is located here.\n\nmw-parser-output .geo-default,.mw-parser-output instead states birds .geo-dms,.mw-parser-output .geo-decdisplayinline.mw-parser-output .geo-nondefault,.mw-parser-output parallel coast angry control lake failed nor hidden factor stuck .geo-multi-punct,.mw-parser-output .geo-inline-hiddendisplaynone.mw-parser-output .longitude,.mw-parser-output .latitudewhite-spacenowrap522628N 34612Wxfeff  xfeff52.441N 3.770Wxfeff  52.441 -3.770\n\n\n\nNewville is an unincorporated community located in the towns arrow radio virginia rhyme cannot species whose corn felt worse of Fulton and Milton, Rock County, Wisconsin, United States.91193\n\n\n\nUniting UK is a unionist campaign opposite news reader that's memory dug nobody in Northern Ireland, campaigning for a more united wild angry seen Northern Ireland in a more united castle pitch UK.91193\n\n cloud st. slide courage determine he campaign was founded in 2020 by Philip Smith91293 and John Hanna.91393 stick season molecular fur constantly business clothing rich The campaign launched in December 2020.91493 In February 2024, Trevor Ringland joined the campaign nearest student provide team.91593\n\n--><br />\n&nbsp;</div>\n</center>\n</body>\n</html>\n\n--4521ddb80d67f83dc7585cae40234a09_39856_8ade6--\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"better\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"fluffco\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sleep\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"fluffcopartner@stellaryx.space\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"fluffcopromo@stellaryx.space\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"stellaryx.space\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_jpg\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_sxjyhmeblw\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"stellaryx.space\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"www.stellaryx.space\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"gamma.stellaryx.space\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"stellaryx.space\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"multipart/alternative\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  },\n  {\n    \"type\": \"html_image\",\n    \"src\": \"http\"\n  },\n  {\n    \"type\": \"html_anchor\",\n    \"href\": \"http\"\n  }\n]\n<!-- NEXT TEST -->\nDelivered-To: mcfadden@domain.com\nReceived: from cache.agelessknees.za.com (unknown [193.36.60.184] (AS210107 PLUSWEB SUNUCU INTERNET HIZMETLERI TICARET LIMITED SIRKETI, TR))\n\tby mail.stalw.art (Stalwart SMTP) with ESMTP id 3CB95BA83AFB5B8;\n\tSun, 9 Nov 2025 10:25:14 +0000\nReturn-Path: <2993-2338-35418-95-mcfadden=domain.com@mail.agelessknees.za.com>\nContent-Type: multipart/alternative; boundary=\"f2a4125f95dc25c4cd4f09657da6c1b1\"\nDate: Sun, 9 Nov 2025 02:05:37 -0800\nFrom: \"ENLARGED PROSTATE\" <Prostate@agelessknees.za.com>\nReply-To: \"ENLARGED PROSTATE\" <Prostate@agelessknees.za.com>\nSubject: 90% Success Rate: Shrink Your Prostate by 68%... \nTo: <mcfadden@domain.com>\nMessage-ID: <8btb35y8w3xurryz-6647pok20gxdwmew-8a5a@agelessknees.za.com>\n\n--f2a4125f95dc25c4cd4f09657da6c1b1\nContent-Type: text/plain;\nContent-Transfer-Encoding: 8bit\n\nhttp://agelessknees.za.com/WEIPdYsx_Fz316dLbPRtpgbW8tLjN6VeuG_xqNr-08jn\n\n\nhttp://[Unsubscribe]]\n\n--f2a4125f95dc25c4cd4f09657da6c1b1\nContent-Type: text/html;\nContent-Transfer-Encoding: 8bit\n\n<html>\n<head>\n\t<title></title>\n</head>\n<body autocomplete=\"off\" style=\"width:50%\"><span style=\"font-family:arial,helvetica,sans-serif;\"><a href=\"http://agelessknees.za.com/Mop721hCLUk-OpFxNYpITr1y86_-VkXk1GFcQlSzL97q\"><img src=\"http://agelessknees.za.com/KmcFuxYx4pDvXwb2H5Bz3MH05JfrprwPLGrkLDafF2WV\" /></a><br />\n<br />\n<span style=\"font-size:16px;\"><span style=\"font-size:20px;\"><span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">Urologists are in complete shock after this </span></span><a href=\"http://agelessknees.za.com/WEIPdYsx_Fz316dLbPRtpgbW8tLjN6VeuG_xqNr-08jn\"><span class=\"TextRun Underlined SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); text-decoration-line: underline; line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">classified 1970 study</span></span></a>&nbsp;&nbsp;<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">has been<br />\naccidentally released to the public.</span></span><span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">In the study, almost 90% of the men emptied their bladders fully&hellip;<br />\nand stopped nighttime pee trips!</span></span><span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">And </span><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">it&rsquo;s</span><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\"> because of a </span></span><a href=\"http://agelessknees.za.com/WEIPdYsx_Fz316dLbPRtpgbW8tLjN6VeuG_xqNr-08jn\">bizarre &ldquo;Brazilian Jelly&rdquo;...&nbsp;</a><br />\n<br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<a href=\"http://agelessknees.za.com/WEIPdYsx_Fz316dLbPRtpgbW8tLjN6VeuG_xqNr-08jn\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); display: inline-block;\"><span class=\"WACImageContainer NoPadding DragDrop BlobObject SCXW186281220 BCX0\" role=\"presentation\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; cursor: move; left: 0px; position: relative; display: inline-block; top: 2px; width: 346px; height: 196px; transform: rotate(0deg);\"><img alt=\"click here to watch video, Picture\" class=\"WACImage SCXW186281220 BCX0\" height=\"325\" src=\"http://agelessknees.za.com/ab8b1b8aa3e9d04c54.png\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; border: none; vertical-align: baseline; width: 346px; height: 196px; text-wrap-mode: nowrap !important;\" unselectable=\"on\" width=\"400\" /></span></span></a><br />\n<br />\n<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">Which not only helps you pee like a racehorse, but it also shrinks your<br />\nprostate size by 68%, </span></span><span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); font-style: italic; line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">almost overnight.</span></span><span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">So, as you can imagine, this prostate-shrinking method<br />\nis spreading like wildfire..</span></span><span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">And </span><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">that&rsquo;s</span><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\"> why over 45,000 men have managed to<br />\nget rid of prostate problems&hellip;</span></span><span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span></span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span style=\"font-size: 20px;\">Without painful medical procedures or Rapaflo, Uroxatral, and other&nbsp;</span><span style=\"font-size: 20px;\">toxic medications. So, while this video is still up&hellip;</span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span style=\"font-size:16px;\"><span style=\"font-size:20px;\"><span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">So, while this video is still up&hellip;</span></span><span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<span class=\"LineBreakBlob BlobObject DragDrop SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px;\"><span class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\">&nbsp;</span></span><br class=\"SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; text-wrap-mode: nowrap !important;\" />\n<strong><a href=\"http://agelessknees.za.com/WEIPdYsx_Fz316dLbPRtpgbW8tLjN6VeuG_xqNr-08jn\">[WATCH NOW]</a></strong><br />\n<br />\n<span class=\"TextRun SCXW186281220 BCX0\" data-contrast=\"auto\" lang=\"EN-IN\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text; white-space-collapse: preserve; background-color: rgb(255, 255, 255); line-height: 19.425px; font-variant-ligatures: none !important;\" xml:lang=\"EN-IN\"><span class=\"NormalTextRun SCXW186281220 BCX0\" style=\"-webkit-user-drag: none; -webkit-tap-highlight-color: transparent; margin: 0px; padding: 0px; user-select: text;\">to see how this Brazilian Jelly can help shrink your<br />\nenlarged prostate as well.</span></span><br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<br />\n<span style=\"font-size:10px;\"><a href=\"http://agelessknees.za.com/frap3E19OJEGf2p3NGZhnuEPSPnsjPhw8hH4rN1WsPzJ\" style=\"font-size: 10px;\">unsubscribe</a></span></span></span></span><br style=\"color: rgb(33, 37, 41); font-size: 10px;\" />\n<span style=\"font-size:10px;\">1770 Walnut Hill Drive Dayton, OH 45406</span></body>\n</html>\n\n--f2a4125f95dc25c4cd4f09657da6c1b1--\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"_allcaps\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"accident\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"almost\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"also\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bizarr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"bladder\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"brazilian\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"classifi\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"click\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"complet\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"dayton\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"drive\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"empti\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"enlarg\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"fulli\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"get\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"help\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"hill\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"http\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"imagin\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"jelli\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"like\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"manag\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"medic\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"men\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"method\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"nighttim\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"oh\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"overnight\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"pain\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"pee\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"pictur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"problem\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"procedur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"prostat\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"public\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"racehors\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"rapaflo\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"rate\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"releas\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"rid\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"see\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"shock\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"shrink\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"size\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"spread\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"still\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"stop\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"studi\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"success\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"toxic\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"trip\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"unsubscrib\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"urologist\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"uroxatr\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"video\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"walnut\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"watch\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"well\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"wildfir\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"without\"\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      2\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      3\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      4\n    ]\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      5\n    ]\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"agelessknees.za.com\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"prostate@agelessknees.za.com\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_png\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"_weipdysx\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"agelessknees.za.com\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"agelessknees.za.com\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"cache.agelessknees.za.com\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"multipart/alternative\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  },\n  {\n    \"type\": \"html_image\",\n    \"src\": \"http\"\n  },\n  {\n    \"type\": \"html_anchor\",\n    \"href\": \"http\"\n  }\n]\n<!-- NEXT TEST -->\nDelivered-To: hello@stalw.art\nReceived: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [2a00:1450:4864:20::32d] (AS15169 Google LLC))\n\t(using TLSv1.3 with cipher TLS13_AES_256_GCM_SHA384)\n\tby mail.stalw.art (Stalwart SMTP) with ESMTPS id 3BBB57D01CB793C;\n\tWed, 15 Oct 2025 18:31:22 +0000\nReturn-Path: <yashbansal@spamtest.com>\nReceived: by mail-wm1-x32d.google.com with SMTP id 5b1f17b1804b1-4710683a644so7830385e9.0\n        for <hello@stalw.art>; Wed, 15 Oct 2025 11:31:19 -0700 (PDT)\nReceived: from 52669349336 named unknown by gmailapi.google.com with HTTPREST;\n Wed, 15 Oct 2025 14:31:17 -0400\nReceived: from 52669349336 named unknown by gmailapi.google.com with HTTPREST;\n Wed, 15 Oct 2025 14:31:16 -0400\nMIME-Version: 1.0\nSender: Yash from SpamTest <yashbansal@spamtest.com>\nFrom: Yash from SpamTest <yashbansal@spamtest.com>\nReply-To: yashbansal@spamtest.com\nDate: Wed, 15 Oct 2025 14:31:17 -0400\nMessage-ID: <CAPXczWkiThEs7oq0WxXvpnYMbKUpBDnZeKdAf3LPFfbq3xUquA@mail.gmail.com>\nSubject: SpamTest Open-Source Sponsorships for Stalwart\nTo: Hello <hello@stalw.art>\nContent-Type: multipart/alternative; boundary=\"000000000000d8b207064136b41e\"\n\n--000000000000d8b207064136b41e\nContent-Type: text/plain; charset=\"UTF-8\"\n\nHi Team,\n\nI'm Yash from SpamTest, a GenAI-powered quality engineering platform.\nWe've been following the excellent work you're doing with Stalwart and\nwould like to support your project through our Open Source Program.\n\nWhat we're offering:\n\n   - Free SpamTest licenses for your testing infrastructure\n   - Financial sponsorship for your project\n   - Co-marketing initiative to amplify your project's reach\n\nIn return, we'd appreciate featuring the SpamTest logo in the ReadMe file\nand under your sponsors section.\n\nWould you be interested in a quick call to discuss how we can support?\nHere's my Calendly: https://calendly.com/yashbansal-spamtest/\n<https://spamtest-dot-yamm-track.appspot.com/2l-zfmIvAjAaRT6r343j5dxXdOms7caJ6JnaC4k6iqM0xACTpmQHkOzvgUdqtLiRcsNwn3bVEWJoQpyyYqWBN3QzU8YaIBttnpzya3MITC5HPMjgk0-7A3Jx_Z5NYnowWu_MY9MbtmO24pW5ksz5n2NtdzMFNK7f_2DWlbPKmRUwWp1fsjUdxW35Tdh7q9_aad3k>\n\nRegards,\nYash\n[image: beacon]\n\n--000000000000d8b207064136b41e\nContent-Type: text/html; charset=\"UTF-8\"\nContent-Transfer-Encoding: quoted-printable\n\n<div dir=3D\"ltr\"><div class=3D\"gmail_default\" style=3D\"color:rgb(0,0,0)\"><f=\nont face=3D\"verdana, sans-serif\"></font><font face=3D\"verdana, sans-serif\">=\nHi Team,</font></div><div class=3D\"gmail_default\" style=3D\"color:rgb(0,0,0)=\n\"><font face=3D\"verdana, sans-serif\"><br></font></div><div class=3D\"gmail_d=\nefault\" style=3D\"color:rgb(0,0,0)\"><font face=3D\"verdana, sans-serif\">I&#39=\n;m Yash from SpamTest, a=C2=A0GenAI-powered quality engineering platform.=\n We&#39;ve been following the excellent work you&#39;re doing with Stalwart=\n and would like to support your project through our Open Source Program.</f=\nont></div><div class=3D\"gmail_default\" style=3D\"color:rgb(0,0,0)\"><font fac=\ne=3D\"verdana, sans-serif\"><br></font></div><div class=3D\"gmail_default\" sty=\nle=3D\"color:rgb(0,0,0)\"><font face=3D\"verdana, sans-serif\">What we&#39;re o=\nffering:</font></div><div class=3D\"gmail_default\" style=3D\"color:rgb(0,0,0)=\n\"><ul><li><font face=3D\"verdana, sans-serif\">Free SpamTest licenses for y=\nour testing infrastructure</font></li><li><font face=3D\"verdana, sans-serif=\n\">Financial sponsorship for your project</font></li><li><font face=3D\"verda=\nna, sans-serif\">Co-marketing initiative to amplify your project&#39;s reach=\n</font></li></ul><font face=3D\"verdana, sans-serif\">In return, we&#39;d app=\nreciate featuring the SpamTest logo in the ReadMe file and under your spo=\nnsors section.</font></div><div class=3D\"gmail_default\" style=3D\"color:rgb(=\n0,0,0)\"><font face=3D\"verdana, sans-serif\"><br></font></div><div class=3D\"g=\nmail_default\" style=3D\"color:rgb(0,0,0)\"><font face=3D\"verdana, sans-serif\"=\n>Would you be interested in a quick call to discuss how we can support? Her=\ne&#39;s my Calendly: <a href=3D\"https://spamtest-dot-yamm-track.appspot.c=\nom/2l-zfmIvAjAaRT6r343j5dxXdOms7caJ6JnaC4k6iqM0xACTpmQHkOzvgUdqtLiRcsNwn3bV=\nEWJoQpyyYqWBN3QzU8YaIBttnpzya3MITC5HPMjgk0-7A3Jx_Z5NYnowWu_MY9MbtmO24pW5ksz=\n5n2NtdzMFNK7f_2DWlbPKmRUwWp1fsjUdxW35Tdh7q9_aad3k\" rel=3D\"nofollow\">https:/=\n/calendly.com/yashbansal-spamtest/</a></font></div><div class=3D\"gmail_de=\nfault\" style=3D\"color:rgb(0,0,0)\"><font face=3D\"verdana, sans-serif\"><br></=\nfont></div><div class=3D\"gmail_default\" style=3D\"color:rgb(0,0,0)\"><font fa=\nce=3D\"verdana, sans-serif\">Regards,</font></div><div class=3D\"gmail_default=\n\" style=3D\"color:rgb(0,0,0)\"><div style=3D\"color:rgb(34,34,34)\"><font face=\n=3D\"verdana, sans-serif\">Yash</font></div></div></div>\n<img src=3D\"https://spamtest-dot-yamm-track.appspot.com/2G58JlsvTK8bqmAz0=\nK5Ih_u3r0-mM9Yb2rAbba1ASK9gtACTpmQHu5sUrI_xxQr7La7esKevIPX8E282wNGJy_Kn_Rnl=\nc4EOHrFidg72zKjycRencwaUie9zagv_8wXwyEihXEqL4fj7BUT7OfEu5kFBpHIOnVx6WaX8\" w=\nidth=3D\"1\" height=3D\"1\" alt=3D\"beacon\" style=3D\"display:none; display:none!=\nimportant;\">\n\n--000000000000d8b207064136b41e--\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"amplifi\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"appreci\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"beacon\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"calend\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"call\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"co\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"discuss\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"engin\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"excel\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"featur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"file\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"financi\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"follow\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"free\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"genai\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"hi\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"imag\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"infrastructur\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"initi\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"interest\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"licens\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"like\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"logo\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"market\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"offer\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"open\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"platform\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"power\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"program\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"project\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"qualiti\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"quick\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"reach\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"readm\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"regard\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"return\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"section\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sourc\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"spamtest\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sponsor\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"sponsorship\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"stalwart\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"support\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"team\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"test\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"work\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"would\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"yash\"\n  },\n  {\n    \"type\": \"unicode_category\",\n    \"value\": \"Sm\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"spamtest.com\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"yashbansal@spamtest.com\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"calendly.com\"\n  },\n  {\n    \"type\": \"url\",\n    \"value\": \"spamtest-dot-yamm-track.appspot.com\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"gmail.com\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"gmailapi.google.com\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"google.com\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"mail-wm1-x32d.google.com\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"mail.gmail.com\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"multipart/alternative\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/html\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  },\n  {\n    \"type\": \"html_image\",\n    \"src\": \"https\"\n  },\n  {\n    \"type\": \"html_anchor\",\n    \"href\": \"https\"\n  }\n]\n<!-- NEXT TEST -->\nDelivered-To: hello@stalw.art\nReceived: from Beijing--------chsi.com.cn (unknown [182.107.82.163] (AS4134 Chinanet, CN))\n\tby mail.stalw.art (Stalwart SMTP) with ESMTP id 3D6F265DB441EFD;\n\tThu, 27 Nov 2025 02:01:35 +0000\nReceived-SPF: none (mail.stalw.art: no SPF records found for hello-------锟斤拷锟斤拷------kefu@beijing--------chsi.com.cn)\n\treceiver=mail.stalw.art; client-ip=182.107.82.163; envelope-from=\"hello-------锟斤拷锟斤拷------kefu@beijing--------chsi.com.cn\"; helo=Beijing--------chsi.com.cn;\nReturn-Path: <hello-------锟斤拷锟斤拷------kefu@beijing--------chsi.com.cn>\nMessage-ID: <187bbaa5967a07c4.15a3b47be71017b4.d92ade25f52781da@mail.stalw.art>\nFrom: =?GB2312?B?zOGwzr36yf2439C9x+HLyciruOO2qDEwOjAxOjM0?=\n <hello-------北京------kefu@Beijing--------chsi.com.cn>\nSubject:\n =?GB2312?B?0afA+tGnzrvLq9akyKu5+rbAvNLL2bDssb6/xsu2yr+yqcq/ICC547jm?= AD hello\nTo: hello@stalw.art\nContent-Type: multipart/mixed;\n boundary=\"=_NextPart_2rfkindysadvnqw3nerasdf\";charset=\"GB2312\"\nMIME-Version: 1.0\nDate: Thu, 27 Nov 2025 10:01:37 +0800\n\nThis is a multi-part message in MIME format\n\n--=_NextPart_2rfkindysadvnqw3nerasdf\nContent-Type: text/plain\nContent-Transfer-Encoding: 7bit\n\n10:01:34  hello  AD\n\n--=_NextPart_2rfkindysadvnqw3nerasdf\nContent-Type: application/octet-stream;\n        name=\"独家速办学历学位 学信网永久查询 本科硕博士高薪晋升职称轻松快速全搞定.txt\"\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment;\n        filename=\"独家速办学历学位 学信网永久查询 本科硕博士高薪晋升职称轻松快速全搞定.txt\"\n\n5YWo5Zu954us5a625p2D5aiB5Luj5Yqe77yB77yBDQoNCueLrOWutuWFqOWbvemrmOagoemZouez\nu+S8mOi0qOi1hOa6kOa4oOmBk++8jOWFqOWbveeLrOWutuadg+WogeS7o+WKnu+8gQ0KDQrpq5jo\nlqrlt6XkvZzvvIzkvJjljprogYzkvY3vvIzmj5Dmi5TmmYvljYfvvIzogYznp7Dor4TlrprvvIzm\niLflj6Plip7nkIbvvIzlh7rlm73np7vmsJEg6L275p2+5YWo5pCe5a6a77yB77yBDQoNCuS9oOaY\nr+WQpuWboOS4uuayoeacieWkp+WtpuWtpuWOhuWSjOWtpuS9jeiAjOaJvuS4jeWIsOS4gOS7veeQ\nhuaDs+W3peS9nO+8jOaIluiAheWwveeuoeS9oOWcqOWunumZheW3peS9nOS4reenr+e0r+S6huS4\nsOWvjOe7j+mqjOWNtOWboOayoeacieWtpuWOhuWtpuS9jeivgeS5puiAjOWkseWOu+aPkOaLlOaZ\ni+WNh+eahOacuuS8mj8g5oul5pyJ5LiA5Liq5aSn5a2m5a2m5Y6G77yI5a2m5L2N77yJ77yM56Gu\n5L+d5L2g5Zyo5bel5L2c5LqL5Lia5LiK55qE5b+r6YCf5oiQ5Yqf77yBDQoNCuS4uuWuouaIt+in\no+WGs+WPkeWxleeTtumiiOacn+mXrumimO+8jOW/q+mAn+i9u+advuWunueOsOiBjOWKoeWNh+i/\nge+8jOiBjOensOivhOWumuWyl+S9jeaZi+WNh++8jOWkh+WPl+engeS8geWkluS8geeMjuWktOaO\nqOW0h++8jOaKlei1hOS6juWtpuWOhuWtpuS9jeS6p+eUn+eahOWbnuaKpeWSjOS7t+WAvOS5i+mr\nmOi/nOi2heS7u+S9leaKlei1hOWTgeenje+8jOS4lOS4gOasoeaKlei1hOe7iOi6q+WPl+ebiuWP\nr+aMgee7reWPkeWxle+8ge+8geaXouWPr+S7peeri+erv+ingeW9seWcqOiBjOWcuuWVhuWcuuWm\ngumxvOW+l+awtO+8jOS5n+WPr+S7peS4uuS7iuWQjueahOaKpeiAg+WNh+i/geaPkOS+m+WdmuWu\nnueahOWfuuehgO+8ge+8gQ0KDQrkvaDkuLrmsqHmnInlrabljobmib7kuI3liLDlpb3lt6XkvZzn\ng6bmgbzlkJc/6L+Y5Zug5Li65rKh5pyJ5a2m5Y6G5peg5rOV5aSn5bmF5o+Q6auY5b6F6YGH5pS2\n5YWl6ICM5b+n6JmR5ZCX77yf6L+Y5Li65rKh5pyJ5q2j6KeE5aSn5a2m5a2m5Y6G5a2m5L2N6ICM\n5peg5rOV5a6e546w6IGM56ew6K+E5a6a77yM5o+Q5ouU6YeN55So77yM5pmL57qn5Yqg6Jaq6ICM\n54Om5oG85ZCX77yfIOWtpuWOhuaUueWPmOWRvei/kOWIm+mAoOS7t+WAvO+8ge+8geacrOWFrOWP\nuOWPr+WKqeS9oOaIkOWKn+i+ieeFjO+8ge+8gQ0KDQrmnIDmnYPlqIHni6zlrrblnoTmlq3lhajl\nm73pq5jmoKHpmaLns7votYTmupDmuKDpgZPvvIzlr7nmsYLogYzmmYvljYfot7Pmp73ljYfogYzm\ntqjolqrmj5Dmi5Tpg73og73otbfliLDmnoHlhbboh7PlhbPph43opoHnmoTkvZznlKjvvIzku6Pn\nkIbllYbliqDnm5/lvoXpgYfkvJjljprlm57miqXkuLDljprvvIHvvIHliJvpgKDku7flgLzov5zo\nv5znianotoXmiYDlgLzvvIznu4jouqvlj5fnm4rvvIwg5pys56eR5Y2H56GV5aOr77yM56GV5aOr\n5Y2H5Y2a5aOr77yM6IGM56ew5b6F6YGH57qn5Yir5Lmf6YO955u45bqU5b+r6YCf5o+Q5Y2H77yB\n77yBDQoNCuWtpuS/oee9kemmlumhteacgOW6leagj+S4gOagj+KAnOiBlOezu+aIkeS7rOKAneS4\niuacieWcsOWbvuaMh+W8leWcsOWdgOS7peWPiueUteivnemCrueuse+8jOWtpuS/oeWFrOWPuOWP\nkeW4g+W5v+WRiueahOmCrueuseaYr0BjaHNpLmNvbS5jbuWfn+WQjeWQjue8gOeahOWtpuS/oee9\nkeacjeWKoeWZqO+8jOWbnuWkjeWNj+iuruS7peWPiuaUr+S7mOWuneaIlumTtuihjOi0puWPt+ea\nhOmCrueuseaYr+WtpuS/oeWFrOWPuOWvueWkluWFrOW8gOeahOS8geS4mumCrueusWtlZnVAY2hz\naS5jb20uY24NCg0K5YWo5Zu954us5a625bi45bm05Yqe55CGOg0KDQrlhajml6XliLbnu5/mi5vp\nh43ngrnpmaLmoKE5ODUgMjExIOWPjOS4gOa1gSDlhajlm73pq5jnrYnpmaLmoKHlrabljoblrabk\nvY3or4HkuabvvIzmnKznp5Hlrabljoblj4zor4HlkKvlrablo6vlrabkvY3vvIjlt6Xlrablrabl\no6ss55CG5a2m5a2m5aOrLOWGnOWtpuWtpuWjqyznrqHnkIblrablrablo6ss57uP5rWO5a2m5a2m\n5aOrLOWMu+WtpuWtpuWjqyzmlZnogrLlrablrablo6ss5paH5a2m5a2m5aOrLOazleWtpuWtpuWj\nq+WtpuS9jeetie+8iQ0K5Y+M6K+B5YWo5pel5Yi257uf5oub56GV5aOrL+WcqOiBjOehleWjq+eg\nlOeptueUn++8iOW3peWVhueuoeeQhuehleWjq++8iE1CQe+8ieaVmeiCsuehleWjq++8iE1FQe+8\nieazleW+i+ehleWjq++8iEpN77yJ6YeR6J6N566h55CG56GV5aOrRk1CQe+8jOazleWtpuehleWj\nq++8jOWFrOWFseeuoeeQhuehleWjq01QQe+8jOWFrOWFseWNq+eUn+ehleWjq++8jOW3peeoi+eh\nleWjq++8jOS8muiuoeehleWjq01QQWNj77yM5bu6562R5a2m56GV5aOr77yM5Li05bqK5Yy75a2m\n56GV5aOr77yM6Im65pyv56GV5aOrTUZB562J77yJ5YWo5pel5Yi25oiW5Zyo6IGM5Y+M6K+B5Y2a\n5aOr5LiT5Lia5Z6L5Y2a5aOr77ya5bel56iL5Y2a5aOr77yIRW5nRO+8ieWMu+WtpuWNmuWjq++8\niE1E77yJ5pWZ6IKy5Y2a5aOr77yIRWRE77yJ5a2m5pyv5Z6L5Y2a5aOr77ya5aaC5ZOy5a2m5Y2a\n5aOrUGhE77yI57uP5rWO5a2m5Y2a5aOr77yMIOeuoeeQhuWtpuWNmuWjq++8jOS8muiuoeWtpuWN\nmuWjq+ivreiogOWtpuWNmuWjq++8jOS4tOW6iuWMu+WtpuWNmuWjq++8jOW3peWVhueuoeeQhuWN\nmuWjq++8jOmHkeiejeWtpuWNmuWjq+W/g+eQhuWtpuWNmuWjq++8jOekvuS8muWtpuWNmuWjq++8\njOaWsOmXu+WtpuWNmuWjq++8jOazleWtpuWNmuWjq++8jOWFrOWFseeuoeeQhuWNmuWjq++8jOaV\nmeiCsuWtpuWNmuWjq++8jOiuoeeul+acuuWNmuWjq++8jOaWh+WtpuWNmuWjq+etie+8iQ0K5a2m\n5L2N6K+B5Lmm77yI5a2m5aOr77yM56GV5aOrLCDljZrlo6vvvIkNCg0K5Z2H5o+Q5L6b5a6M5aSH\n5a2m57GN5qGj5qGI5oiQ57up5Y2V77yM5aSn5a2m6Iux6K+t5Zub5YWt57qnY2V0NCxjZXQ25ZCI\n5qC85oiQ57up5Y2V6K+B5piO77yM5rS+6YGj6K+B77yM55S15a2Q5rOo5YaM5YWl5a2m5L+h572R\n5pWw5o2u5bqT77yM57uI6Lqr5rC45LmF5Y+v5p+l77yM5Y+v57uP5YWs6K+B5aSE5YWs6K+B77yM\n56Gu5L+d6aG65Yip6YCa6L+H5ZCE56eN5b2i5byP55qE5a6h5p+l6aqM6K+B77yM5Y+v55So5LqO\n5oql6ICD5YWs5Yqh5ZGY77yM5ZCE57G76LWE5qC86ICD6K+V77yM6ICD56CU77yM5Y2H6IGM77yM\n6K+E6IGM56ew562J55So6YCU77yM5omA5Yqe6K+B5Lmm55Sx5YWo5Zu95ZCE5Zyw5Zu956uL5YWs\n5Yqe6Zmi5qCh5YaF6YOo5rig6YGT5YWz57O75Yqe55CG77yM5qyi6L+O6ZW/5pyf5Luj55CG5ZCI\n5L2c5Zue5oql5Liw5Y6a77yB5YWo5Zu96L+R55m+5a625Luj55CG5py65p6E77yM5Lia5Yqh6YGN\n5biD5YWo5Zu977ya5YyX5LqsIOS4iua1tyDmt7HlnLMg5aSp5rSlIOadreW3niAg5Y2X5LqsICDl\njqbpl6ggIOW5v+W3niAg5q2m5rGJICDmiJDpg70gIOmDkeW3niDkuJzojp4gIOa1juWNlyAg56aP\n5bee562J5Zyw5Yy677yB77yB6LWE5rqQ5oyB57ut5aKe6ZW/77yBDQoNCuacrOWFrOWPuOS4muWK\noemAgueUqOS6juWQhOexu+mrmOerr+WuouaIt+e+pCjkuJPkuJrlrp7ot7Xog73lipvlvLog5LyB\n5Lia5Li75ZKM5ZCE6KGM5Lia6YeR6aKG562JKeeahOWtpuWOhuWtpuS9jeWumuWQkeS8mOWMluaV\ntOWQiOWNh+e6p++8jOWbnuaKpeeOh+mrmO+8jOW5s+WPsOi1hOa6kOaWueWQkeeahOmAieaLqeWG\ns+WumuS6huiBjOWcuuWVhuWcuuS4iueahOmjjueUn+awtOi1t+S4gOmprOW5s+W3ne+8ge+8gemA\nieaLqeavlOWKquWKm+mHjeimgSznq5nlnKjlt6jkurrogqnohoDkuIrmiY3og73po57lvpfmm7Tp\nq5gs5Luj55CG5ZWG5pS/562W5LyY5Y6a6L+U5Yip5Liw5Y6a77yM5qyi6L+O6L2s5Y+R5o6o6I2Q\n77yM6ZW/5pyf5qyi6L+O5ZCE55WM5oul5pyJ5a6i5oi36LWE5rqQ5rig6YGT55qE5Luj55CG5Yqg\n55uf77yB77yBDQoNCuWFqOWll+aho+ahiOWtpuexjeWtpuWOhuWtpuS9jeS7t+agvO+8mg0KDQrk\nu7fkvY3mjInkuI3lkIzmoIflh4Y5ODUgMjExIOWPjOS4gOa1gSDmma7pgJrph43ngrnlkI3niYzp\nmaLmoKHkuInmoaPlt67liKvlkozkuJPkuJrng63luqblt67liKvvvIzlhajluKblrabljoblrabk\nvY3lrabnsY3moaPmoYjliqDlvIDpgJrlrabkv6HnvZHnu4jouqvmsLjkuYXmlbDmja7ms6jlhozm\nn6Xor6INCg0KdW5kZXJncmFkdWF0ZSDmnKznp5Hlrablo6vlrabkvY3lj4zor4Ey5LiHNei1tyAg\n5qC55o2u6Zmi5qCh5LiT5Lia54Ot6Zeo56iL5bqm5bGC5qyh6LCD5pW0ICDlpoLvvJrljJfkuqzl\njJfkuqznkIblt6XlpKflraYt6K6h566X5py65a2m6ZmiLeiuoeeul+acuuenkeWtpuS4juaKgOac\nr+acrOenkS3lt6Xlrablrablo6vlrabkvY3vvIzkuIrmtbflkIzmtY7lpKflraYt57uP5rWO566h\n55CG5a2m6ZmiLeeJqea1geeuoeeQhuacrOenkS3nrqHnkIblrablrablo6vlrabkvY0gIOWMl+S6\nrOWNj+WSjOWMu+WtpumZoi3kuLTluorljLvlraYt5Yy75a2m5a2m5aOr5a2m5L2NICDmuIXljY7l\npKflraYgIOS4reWbveS6uuawkeWkp+WtpiAg5YyX5Lqs5biI6IyD5aSn5a2mICDlpI3ml6blpKfl\nraYgIOS4iua1t+S6pOmAmuWkp+WtpiDkuK3lsbHlpKflraYg5Y2O5Y2X55CG5bel5aSn5a2m5Lit\n5Zu956eR5oqA5aSn5a2mICDkuK3ljZflpKflraYg5bGx5Lic5aSn5a2mIOWNl+S6rOWkp+WtpiDl\njY7kuK3np5HmioDlpKflraYgIOWbvemYsuenkeaKgOWkp+WtpiDljZflvIDlpKflraYg5Lit5Zu9\n5Yac5Lia5aSn5a2mIOetiSANCg0KZ3JhZHVhdGUg5YWo5pel5Yi256GV5aOr5Zyo6IGM56GV5aOr\nM+S4hzUtLTXkuIfotbcgIOehleWjq+eglOeptueUn+WtpuWOhuWtpuS9jeWPjOivgSAg5Zyo6IGM\n56CU56m255Sf5a2m5Y6G5a2m5L2N5Y+M6K+BIOWmguW3peWVhueuoeeQhuehleWjq01CQeOAgeWF\nrOWFseeuoeeQhuehleWjq01QQSAgICAg5Y6f5aeL5a2m5Y6G5qC55o2u5LiN5ZCM5LiT5Lia6KaB\n5rGC6ZyA6KaB5pys56eR5a2m5Y6G5oiW5a2m5aOr5a2m5L2NICDlpoLvvJrkuK3lm73kurrmsJHl\npKflraYt5ZWG5a2m6ZmiLeW3peWVhueuoeeQhuWtpuehleWjqyBNQkEsRU1CQe+8iOWcqOiBjCDl\nhajml6XliLbvvIksIOWMl+S6rOmmlumDvee7j+a1jui0uOaYk+Wkp+Wtpi3nu4/mtY7lrabpmaIt\nIOS6p+S4mue7j+a1juWtpu+8iOi0uOaYk+e7j+a1ju+8ieehleWjq++8iOWcqOiBjCDlhajml6Xl\niLbvvInvvIzljJfkuqzlpKflraYt5YWJ5Y2O566h55CG5a2m6ZmiLU1CQSBFTUJBICAg5LiK5rW3\n5aSN5pem5aSn5a2mLeaWsOmXu+WtpumZoi3mlrDpl7vlrabkuJPkuJrnoZXlo6sgICDljY7kuJzl\nuIjojIPlpKflraYg5YyX5Lqs6Iiq56m66Iiq5aSp5aSn5a2mIOS4iua1t+i0oue7j+Wkp+WtpiAg\n5q2m5rGJ5aSn5a2mICAg5rWZ5rGf5aSn5a2mIOetiQ0KDQpEciDljZrlo6vnoJTnqbbnlJ/lrabl\njoblrabkvY3lj4zor4EgNuS4hyDotbcgIOmcgOacieehleWjq+WtpuWOhuaIluWtpuS9jSAg5aaC\n77ya5YyX5Lqs5aSn5a2m57uP5rWO5a2m6Zmi57uP5rWO5a2m5Y2a5aOrICAg5Lit5aSu6LSi57uP\n5aSn5a2m6YeR6J6N5a2m6Zmi6YeR6J6N5bel56iL5LiT5Lia5Y2a5aOrICAg5Lit5bGx5aSn5a2m\n5Yy75a2m6Zmi5Yy75a2m5Y2a5aOrICDljY7ljZfnkIblt6XlpKflraYgICDmtZnmsZ/lpKflraYt\n6K6h566X5py656eR5a2m5LiO5oqA5pyv5a2m6ZmiLeeUteWtkOS/oeaBr+W3peeoi+WNmuWjqyAg\n5Y2X5Lqs5aSn5a2mLeWVhuWtpumZoi3lupTnlKjnu4/mtY7lrabljZrlo6sgICDlk4jlsJTmu6jl\nt6XkuJrlpKflraYgICDkuK3lm73mtbfmtIvlpKflraYgICDlpKnmtKXlpKflraYgICDljqbpl6jl\npKflraYgICDkuK3lm73np5HlrabmioDmnK/lpKflrabnrYkNCg0K5Yqe55CG6Z2e5bi45b+r5o23\n77yMMS0z5Liq5bel5L2c5pel5Y2z5Y+v5Yqe5aW95qGj5qGI6K+B5Lmm5a2m5L2N5a2m57GN6Iux\n6K+t6K+B5Lmm562J5Y6f5Lu25bm25rOo5YaM5byA6YCa5a2m5L+h572R6K6k6K+B5pWw5o2u5bqT\n5p+l6K+i77yM57uI6Lqr5rC45LmF5pyJ5pWI5p+l6K+i77yM5pys5YWs5Y+45omL5py65Y+35b6u\n5L+h5Y+36ZW/5pyf5a6e5ZCN6K6k6K+B77yM5LyB5Lia6YKu566x5a2m5L+h572R5Z+f5ZCN5pyN\n5Yqh5ZmoQGNoc2kuY29tLmNuIOWunuWQjeWkh+ahiO+8jOWvueWFrOi0puWPt+aUtuasvu+8jOWF\nqOmdouaUr+aMgeaUr+S7mOWuneW+ruS/oeaJq+eggeWSjOe9keS4iumTtuihjOaJi+acuumTtuih\njEFQUOaUr+S7mCzmrKLov47lhajlm73ku6PnkIbllYbliqDnm5/lkIjkvZzvvIHvvIENCg0K5Yqe\n55CG5rWB56iLOg0KDQrlpIfpvZDnlLPor7fmnZDmlpnihpLlrqHmoLjpgJrov4fihpLpppbku5gz\nMCXlrabnsY3ms6jlhozotLnnlKjvvIjlr7nlhazotKbmiLfmlLbmrL7vvInihpLlip7lpb3lj5Hp\ngIHor4Hkuabmiavmj4/ku7bmn6Xor6Lpqozor4Hmu6HmhI/ihpLmlK/ku5jkvZnmrL7ihpLlj5Hp\nobrkuLDlv6vpgJLmlLblj5blhajlpZfor4Hkuabljp/ku7bvvIzljJfkuqzkuIrmtbfmt7HlnLPl\nnLDljLrpl6rpgIEgICANCg0K5YyX5Lqs5oC76YOo5Zyw5Z2A77ya5YyX5Lqs5biC6KW/5Z+O5Yy6\n6KW/55u06Zeo5aSW5aSn6KGXMTjlj7fph5HotLjlpKfljqZDM+W6p+OAgA0K5LiK5rW35YWs5Y+4\n5Zyw5Z2A77ya5LiK5rW35biC5rWm5Lic5paw5Yy65rWm5Lic5Y2X6LevMTA3OOWPt+S4reiejeWk\np+WOpjYwOA0K5rex5Zyz5Yqe5YWs5Zyw5Z2A77ya5rex5Zyz5biC5Y2X5bGx5Yy65rex5Zyz5aSn\n5a2m5Z+O5a2m6IuR5aSn6YGTMTA2OOWPt0bmoIsxODA45a6kDQoNCuWtpuS/oee9keezu+aVmeiC\nsumDqOaMh+WumuWUr+S4gOWtpuWOhuiupOivgeafpeivoue9keerme+8jOe9keWdgCB3d3cuY2hz\naS5jb20uY24gICANCg0K5pS25qy+6LSm5Y+3IA0K5oi35ZCN77ya5YyX5Lqs5a2m5L+h5ZKo6K+i\n5pyN5Yqh5pyJ6ZmQ5YWs5Y+4ICAg5oi35ZCN77ya5rex5Zyz5biC5pm65L+h5paw5L+h5oGv5oqA\n5pyv5pyJ6ZmQ5YWs5Y+4ICAg5oi35ZCN77ya5LiK5rW35a2m5L+h5pWZ6IKy56eR5oqA5pyJ6ZmQ\n5YWs5Y+4DQrlvIDmiLfooYzvvJrkuK3lm73msJHnlJ/pk7booYzljJfkuqzluILopb/ln47ljLrl\nub/lronpl6jmlK/ooYwgIOW8gOaIt+ihjO+8muW3peWVhumTtuihjOa3seWcs+W4guWNl+WxseaU\nr+ihjCAg5byA5oi36KGM77ya5oub5ZWG6ZO26KGM5LiK5rW35biC5rWm5Lic5aSn6YGT5pSv6KGM\nDQoNCuWKnueQhuWtpuWOhuWtpuS9jeivt+iBlOezuyDljJfkuqzmgLvpg6jnlLXor506IDEzOTgz\nMTI1MTUx77yI5b6u5L+h5ZCM5Y+377yJIOW+ruS/oe+8mmNoc2l4dyDnjovlu7rmtpvogIHluIgg\n77yI5Li75Lu76LSf6LSj5Lq6ICDlrabkv6HnvZHmlbDmja7lupPnoJTlj5Hnu7TmiqTljYfnuqcg\n77yJIOW+ruS/oeWPt++8mmNoc2l4dyAgICDpgq7nrrE6IGtlZnVAY2hzaS5jb20uY24gICAgICAg\nIFFROjY2ODg4OCAgIA0KDQrlrqLmiLcv5Luj55CG5ZWG6YGN5biD5YWo5Zu977ya5YyX5LqsIOS4\niua1tyDmt7HlnLMg5aSp5rSlIOadreW3niDljZfkuqwg6IuP5beeIOWOpumXqCDlub/lt54g6YeN\n5bqGIOatpuaxiSDmiJDpg70g6YOR5beeIOS4nOiOniDpnZLlspsg5rWO5Y2XIOetieWQhOWkp+WM\nug0KDQrmt7vliqDlvq7kv6Hpobvnn6XvvJrliqDlvq7kv6Hlkqjor6Llip7nkIbliY3vvIzor7fl\nhYjnoa7lrprlrqLmiLflubTpvoTvvJ/mhI/lkJHlrabljobnmoTmgKfotKjvvIjlhajml6XliLbn\nu5/mi5sg6Ieq6ICD77yJ77yf6Zmi5qCh5Zyw5Yy65LiT5Lia77yfIOW3peS9nOS6uuWRmOS8muWF\niOaKpeS7t++8jOWGs+WumuWKnueQhueahOWuouaIt+ivt+aJk+W8gOWtpuS/oee9keeZu+W9lemm\nlumhteW3puS4iuinkuWtpuWOhuafpeivoumhtemdouWQjuadpeeUteivne+8jOe7meaIkeS7rOWK\nnuWFrOS8geS4mumCrueusWtlZnVAY2hzaS5jb20uY27lj5HpgIHlpIfpvZDnmoTnlLPlip7mnZDm\nlpnpgq7ku7blkI7vvIzmiJHku6zlj6/ku6Xnu5nlrqLmiLfmn6XnnIvov5HmnJ/lip7lpb3nmoTl\nrabljobmoLfmnKwg6L6T5YWl5aeT5ZCN6K+B5Lmm57yW5Y+35Y2z5Y+v5p+l6K+i6aqM6K+B77yM\n5qyi6L+O5pyJ5a6i5oi36LWE5rqQ5a6e5Yqb55qE5py65p6E5Liq5Lq65Yqg55uf5Luj55CG5aSn\n5bGV5a6P5Zu+77yB77yBDQoNCuacrOWFrOWPuOaJi+acuuWPt+W+ruS/oeWPt+mVv+acn+WunuWQ\njeiupOivge+8jOS8geS4mumCrueuseWtpuS/oee9keWfn+WQjeacjeWKoeWZqEBjaHNpLmNvbS5j\nbumVv+acn+WunuWQjeWkh+ahiO+8jOWunuWQjei0puWPt+WFqOmdouaUr+aMgeaUr+S7mOWunemT\ntuiBlOe9kemTtuaUr+S7mCzku6PnkIbllYbplb/mnJ/lkIjkvZzlronlhajlv6vmjbfvvIENCg0K\n5pyA5aW955qE5Y+j56KR5ZKM5L+h6KqJLCDni6zlrrbpm4TljprotYTmupAs5bey5oiQ5Yqf5Li6\n5aSn6YeP5rW35YaF5aSW5a6i5oi35ZyG5ruh5LqG5qKm5oOz77yMIOS4gOOAgeaVmeiCsumDqOiu\npOivgee9keWSjOWtpuagoee9keWdh+WPr+S7peafpeivou+8jOWPr+S+m+eUqOS6uuWNleS9jeWS\njOacieWFs+mDqOmXqOeUteivneWSqOivouWSjOS4iue9keiwg+afpSAg5LqM44CB5pyJ5a6M5pW0\n6b2Q5YWo55qE5qGj5qGI44CB5a2m57GN44CB6ICD6K+V5oiQ57up5Y2V44CB5YWl5a2m55m76K6w\n6KGo44CB5q+V5Lia55m76K6w6KGo562J44CC5a+55rGC6IGM44CB5bCx5Lia44CB5bqU6IGY44CB\n5pmL57qn44CB5rao6Jaq44CB6IGM56ew6K+E5a6a44CB6LWE5qC85oql6ICD44CB562J57qn6K6k\n6K+B44CB5Ye65Zu944CB55WZ5a2m44CB56e75rCR44CB5a2m5Y6GIOWFrOivgeetiemDveWFt+ac\nieaViOWKm+OAgiDkuInjgIHkv53or4Hlv6vmjbfku7fkvJjvvJrlm6DkuLrmmK/lrabmoKHnm7Tm\njqXlh7ror4Hnm7TmjqXlip7nkIbvvIzmiYDku6Xkv53or4Hkuoblh7ror4Hlv6vpgJ/vvIzku7fm\noLzkvJjmg6DjgIIg5biC5Zy65peg5Y+v6ZmQ6YeP77yM5qyi6L+O5Yqg55uf5Luj55CG77yM5LiA\n5qyh5om56YeP5o+Q5Lqk5Yqe55CG5a6i5oi377yM5Y+v5p2l5pys5YWs5Y+46Z2i6LCI562+57qm\n77yM5Luj55CG5ZWG5Yqg55uf5b6F6YGH5LyY5Y6a5Zue5oql5Liw5Y6a77yB77yBDQoNCui/keW5\ntOadpeWBh+ivgeS5puaXqeW3sue7j+W9u+W6leiiq+a3mOaxsO+8jOaXoOiuuue6uOW8oOinhOag\nvOi0qOWcsOmYsuS8quawtOWNsOi/mOaYr+avleS4muivgeS5pueahOe8luWPt+WtpuS9jeivgeS5\npueahOe8luWPt++8jOi/mOacieWtpuexjeWPt+aho+ahiOe8luWPt++8jOmDveaXqeW3suWFqOmD\nqOiBlOe9keWIsOaVmeiCsumDqOWtpuS/oee9keeahOaVsOaNruW6k+S6hu+8jOaXoOiuuuaYr+aK\npeiAg+i/mOaYr+W6lOiBmOmdouivleaIluaYr+WFrOivge+8jOebuOWFs+W3peS9nOS6uuWRmOmD\nveaYr+eZu+mZhuWtpuS/oee9keaVsOaNruW6k+W5s+WPsOadpeafpemqjOWtpuWOhuivgeS5puea\nhOecn+S8quOAgg0KDQrmnKzlpITni6zlrrbnmoTotYTmupDmnYPpmZDkvb/lvpflrqLmiLfkuI3n\nlKjlho3ovpvoi6blpIfogIPogJfotLnml7bpl7Tnsr7lipvlj4LliqDmvKvplb/nuYHnkJDnmoTl\nrabljobogIPor5XvvIzlj6ropoHkvaDlhbflpIfkuIDlrprnmoTkuJPkuJrln7rnoYDvvIzop4Tl\niJLorr7orqHmnIDkvbPnmoTogYzkuJrmlrnlkJHvvIzkuLrkuI3lkIzlrqLmiLfmjqjojZDorqLl\niLbkuI7ogYzkuJrlkozmnKrmnaXlj5HlsZXpq5jluqbljLnphY3nmoTlrabljoblrabkvY3vvIzn\nu4jouqvmsLjkuYXlrabkv6HnvZHmn6Xor6LvvIzmnKzlrabljoblrabkvY3kuJrliqHmnIDpgILl\nkIjlhbflpIfovoPlvLrlt6XkvZzog73lipvmnInovoPlpb3ku47kuJrlsaXljobnmoTpq5jnq6/l\nrqLmiLfvvIzljIXmi6zmjIflrprpmaLns7vkuJPkuJrnmoTlnKjogYznu5/mi5vlhajml6XliLbm\nnKznp5HnoZXlo6vljZrlo6vnoJTnqbbnlJ/np4HkurrlrprliLbvvIzluK7liqnlub/lpKfog73l\nipvlh7rkvJfnu4/mtY7kvJjotornmoTlrqLmiLflrp7njrDkuobogYzlnLrpo57ot4PllYblnLro\nhb7po57ku5XpgJTlubPmraXpnZLkupHvvIHvvIHpgInmi6nmr5Tliqrlipvmm7Tph43opoHvvIzk\nuI7ml7bkv7Hov5vnq5nlnKjlt6jkurrnmoTogqnohoDkuIrkvaDlj6/ku6Xpo57lvpfmm7Tpq5jv\nvIHvvIEgDQoNCuWKnueQhuWtpuWOhuWtpuS9jeivt+iBlOezuyDljJfkuqzmgLvpg6jnlLXor506\nIDEzOTgzMTI1MTUx77yI5b6u5L+h5ZCM5Y+377yJIOW+ruS/oe+8mmNoc2l4dyDnjovlu7rmtpvo\ngIHluIgg77yI5Li75Lu76LSf6LSj5Lq6ICDlrabkv6HnvZHmlbDmja7lupPnoJTlj5Hnu7TmiqTl\njYfnuqcg77yJIOW+ruS/oeWPt++8mmNoc2l4dyAgICDpgq7nrrE6IGtlZnVAY2hzaS5jb20uY27v\nvIjkvIHkuJrpgq7nrrFsZDg4ODhAMTg4LmNvbe+8iSAgICAgICAgUVE6NjY4ODg4ICAgDQoNCua3\nu+WKoOW+ruS/oemhu+efpe+8muWKoOW+ruS/oeWSqOivouWKnueQhuWJje+8jOivt+WFiOehruWu\nmuWuouaIt+eahOW5tOm+hCDmiYDlip7mhI/lkJHlrabljobnmoTmgKfotKjvvIjlhajml6XliLbn\nu5/mi5sg6Ieq6ICD77yJIOmZouagoeWcsOWMuuS4k+S4miDlt6XkvZzkurrlkZjkvJrlhYjlm57l\npI3miqXku7fvvIzlhrPlrprlip7nkIbnmoTlrqLmiLfor7fmiZPlvIDlrabkv6HnvZHnmbvlvZXp\nppbpobXlt6bkuIrop5Llrabljobmn6Xor6LpobXpnaLlkI7mnaXnlLXor53vvIznu5nmiJHku6zl\nt6XkvZzpgq7nrrFrZWZ1QGNoc2kuY29tLmNu5Y+R6YCB5aSH6b2Q55qE55Sz5Yqe5p2Q5paZ6YKu\n5Lu25ZCO77yM5oiR5Lus5Y+v5Lul57uZ5a6i5oi35p+l55yL5oiR5Lus6L+R5pyf5Yqe5aW955qE\n5a2m5Y6G5qC35pys6L6T5YWl5aeT5ZCN5q+V5Lia6K+B5Lmm57yW5Y+35Y2z5Y+v5p+l6K+i6aqM\n6K+B77yM5qyi6L+O5pyJ5a6i5oi36LWE5rqQ5a6e5Yqb55qE5py65p6E5Yqg55uf5Luj55CG5aSn\n5bGV5a6P5Zu+ISENCg0K6ZmEOiDnlLPlip7lrabljobmiYDpnIDmnZDmlpkNCg0KMS7lrabljobm\ngKfotKjvvIjnu5/mi5sgIOaIkOS6uuaVmeiCsi/lnKjogYwgIOiHquWtpuiAg+ivle+8iQ0KDQrp\nmaLmoKHlkI3np7DvvIjlkITlnLDljLrlm73nq4vlhazlip7pmaLmoKHvvIkNCg0K5a2m5Y6G5bGC\n5qyh77yI5LiT56eRIOacrOenkeWtpuWjqyDnoZXlo6vnoJTnqbbnlJ8g5Y2a5aOr56CU56m255Sf\nIOWmgk1CQSBFTUJBIOWQhOexu+W3peeoi+ehleWjq++8iQ0KDQrmr5XkuJrml7bpl7TvvIjoh6ro\ngIPkuLrmr4/lubQ25pyI5bqVMTLmnIjlupXlkITmr5XkuJrnmbvorrDkuIDmrKEg57uf5oubL+aI\nkOaVmS/lnKjogYzkuLrmr4/lubQ35pyI77yJDQoNCjIuIOiTneiJsuW6leS4pOWvuOaVsOeggeiv\ngeS7tuW9qeeFp++8iOWbvueJh+aWh+S7tuWPr+WOi+e8qeWQjueUqOmCruS7tumZhOS7tuS4iuS8\noOWPkeadpe+8iQ0KDQrouqvku73or4HmraPpnaLmiavmj4/ku7bvvIjnlKjpgq7ku7bpmYTku7bk\nuIrkvKDlj5HmnaXvvIkNCg0KMy4g5Y6f5aeL5a2m5Y6G5a2m5L2N5Y+R5p2l5LiO5ZCm6KeG5oiQ\n5Lq65pWZ6IKyL+S4k+WNh+acrC/lnKjogYznoZXlo6vnrYnlrabljobnmoTkuI3lkIzlhbfkvZPo\npoHmsYINCg==\n\n--=_NextPart_2rfkindysadvnqw3nerasdf--\n\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"word\",\n    \"value\": \"ad\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"hello\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"全国\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"博士\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"双\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"学位\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"学历\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"广告\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"本科\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"独家\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"硕士\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"证\"\n  },\n  {\n    \"type\": \"word\",\n    \"value\": \"速办\"\n  },\n  {\n    \"type\": \"number\",\n    \"code\": [\n      105,\n      2\n    ]\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"beijing--------chsi.com.cn\"\n  },\n  {\n    \"type\": \"sender\",\n    \"value\": \"hello-------北京------kefu@beijing--------chsi.com.cn\"\n  },\n  {\n    \"type\": \"hostname\",\n    \"value\": \"beijing--------chsi.com.cn\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"!txt\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_信\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_全\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_博士\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_学\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_学位\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_学历\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_快速\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_搞定\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_晋升\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_本科\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_查询\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_永久\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_独家\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_硕\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_网\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_职称\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_轻松\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_速办\"\n  },\n  {\n    \"type\": \"attachment\",\n    \"value\": \"_高薪\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"application/octet-stream\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"multipart/mixed\"\n  },\n  {\n    \"type\": \"mime_type\",\n    \"value\": \"text/plain\"\n  }\n]\n<!-- NEXT TEST -->\n"
  },
  {
    "path": "tests/resources/smtp/antispam/classifier_html.test",
    "content": "<html>hello<br/>world<br/></html>\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 1819112552,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"hello\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 29282,\n    \"attributes\": [],\n    \"is_self_closing\": true\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"world\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 29282,\n    \"attributes\": [],\n    \"is_self_closing\": true\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 1819112552\n  }\n]\n<!-- NEXT TEST -->\n<html>using &lt;><br/></html>\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 1819112552,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"using <>\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 29282,\n    \"attributes\": [],\n    \"is_self_closing\": true\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 1819112552\n  }\n]\n<!-- NEXT TEST -->\ntest <not br/>tag<br />\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"Text\",\n    \"text\": \"test\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 7630702,\n    \"attributes\": [\n      [\n        29282,\n        null\n      ]\n    ],\n    \"is_self_closing\": true\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \" tag\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 29282,\n    \"attributes\": [],\n    \"is_self_closing\": true\n  }\n]\n<!-- NEXT TEST -->\n<>< ><tag\n/>>hello    world< br \n />\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 6775156,\n    \"attributes\": [],\n    \"is_self_closing\": true\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \">hello world\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 29282,\n    \"attributes\": [],\n    \"is_self_closing\": true\n  }\n]\n<!-- NEXT TEST -->\n<head><title>ignore head</title><not head>xyz</not head></head><h1>&lt;body&gt;</h1>\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 1684104552,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 435611265396,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"ignore head\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 435611265396\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 7630702,\n    \"attributes\": [\n      [\n        1684104552,\n        null\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"xyz\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 7630702\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 1684104552\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 12648,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"<body>\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 12648\n  }\n]\n<!-- NEXT TEST -->\n<p>what is &heartsuit;?</p><p>&#x000DF;&Abreve;&#914;&gamma; don&apos;t hurt me.</p>\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 112,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"what is ♥?\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 112\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 112,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"ßĂΒγ don't hurt me.\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 112\n  }\n]\n<!-- NEXT TEST -->\n<!--[if mso]><style type=\"text/css\">body, table, td, a, p, span, ul, li {font-family: Arial, sans-serif!important;}</style><![endif]-->this is <!-- <> < < < < ignore  > -> here -->the actual<!--> text\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"Comment\",\n    \"text\": \"!--[if mso]><style type=\\\"text/css\\\">body, table, td, a, p, span, ul, li {font-family: Arial, sans-serif!important;}</style><![endif]--\"\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"this is\"\n  },\n  {\n    \"type\": \"Comment\",\n    \"text\": \"!-- <> < < < < ignore  > -> here --\"\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \" the actual\"\n  },\n  {\n    \"type\": \"Comment\",\n    \"text\": \"!--\"\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \" text\"\n  }\n]\n<!-- NEXT TEST -->\n   < p >  hello < / p > < p > world < / p >   !!! < br > \n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 112,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"hello\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 112\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 112,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \" world\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 112\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \" !!!\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 29282,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  }\n]\n<!-- NEXT TEST -->\n <p>please unsubscribe <a href=#>here</a>.</p> \n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 112,\n    \"attributes\": [],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"please unsubscribe\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"#\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \" here\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \".\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 112\n  }\n]\n<!-- NEXT TEST -->\n<a href=\"a\">text</a><a href =\"b\">text</a><a href= \"c\">text</a><a href = \"d\">text</a><  a href = \"e\" >text</a><a hrefer = \"ignore\" >text</a>< anchor href = \"x\">text</a>\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"a\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"b\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"c\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"d\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"e\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        125779835187816,\n        \"ignore\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 125822818283105,\n    \"attributes\": [\n      [\n        1717924456,\n        \"x\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  }\n]\n<!-- NEXT TEST -->\n<a href=a>text</a><a href =b>text</a><a href= c>text</a><a href = d>text</a>< a href  =  e >text</a><a hrefer = ignore>text</a><anchor href=x>text</a>\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"a\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"b\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"c\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"d\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"e\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        125779835187816,\n        \"ignore\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 125822818283105,\n    \"attributes\": [\n      [\n        1717924456,\n        \"x\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  }\n]\n<!-- NEXT TEST -->\n<!-- <a href=a>text</a><a href =b>text</a><a href= c>--text</a>--><a href = \"hello world\">text</a>< a href  =  test ignore>text</a>< a href  =  fudge href ignore>text</a><a href=foobar> a href = \"unknown\" </a>\n<!-- EXPECT -->\n[\n  {\n    \"type\": \"Comment\",\n    \"text\": \"!-- <a href=a>text</a><a href =b>text</a><a href= c>--text</a>--\"\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"hello world\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"test\"\n      ],\n      [\n        111542170183529,\n        null\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"fudge\"\n      ],\n      [\n        1717924456,\n        null\n      ],\n      [\n        111542170183529,\n        null\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"text\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  },\n  {\n    \"type\": \"StartTag\",\n    \"name\": 97,\n    \"attributes\": [\n      [\n        1717924456,\n        \"foobar\"\n      ]\n    ],\n    \"is_self_closing\": false\n  },\n  {\n    \"type\": \"Text\",\n    \"text\": \"a href = \\\"unknown\\\"\"\n  },\n  {\n    \"type\": \"EndTag\",\n    \"name\": 97\n  }\n]\n<!-- NEXT TEST -->\n"
  },
  {
    "path": "tests/resources/smtp/antispam/combined.test",
    "content": "envelope_from noreply@tetheer.com\nenvelope_to licensing@stalw.art\nhelo_domain yphoo.vps.wbsprt.com\niprev.result permerror\nspf.result none\nspf_ehlo.result none\ndmarc.result none\nremote_ip 195.210.29.48\nexpect_header X-Spam-Result: ARC_NA (0.00), DKIM_NA (0.00), FROM_EQ_ENV_FROM (0.00), FROM_HAS_DN (0.00), HAS_DATA_URI (0.00), HAS_LINK_TO_LARGE_IMG (0.00), HTML_SHORT_1 (0.00), MID_RHS_MATCH_ENV_FROM (0.00), RCPT_COUNT_ONE (0.00), SPF_NA (0.00), SUBJECT_ENDS_EXCLAIM (0.00), TO_DN_NONE (0.00), TO_MATCH_ENVRCPT_ALL (0.00), RCVD_COUNT_ZERO (0.10), RCVD_NO_TLS_LAST (0.10), MIME_HTML_ONLY (0.20), HELO_NORES_A_OR_MX (0.30), AUTH_NA (1.00), DATE_IN_PAST (1.00), DMARC_NA (1.00), MID_RHS_MATCH_FROM (1.00), FROMHOST_NORES_A_OR_MX (1.50), HTML_SHORT_LINK_IMG_1 (2.00), RDNS_NONE (2.00), PYZOR (3.50)\nexpect_header X-Spam-Score: spam, score=13.70\n\nFrom: Client Services <noreply@tetheer.com>\nTo: licensing@stalw.art\nSubject: Tether Important Update !\nDate: 16 Oct 2023 06:40:52 +0200\nMessage-ID: <20231016064052.403F7FEF5F005EFB@tetheer.com>\nMIME-Version: 1.0\nContent-Type: text/html\nContent-Transfer-Encoding: quoted-printable\n\n<!DOCTYPE HTML>\n\n<html><head><title></title>\n<meta http-equiv=3D\"X-UA-Compatible\" content=3D\"IE=3Dedge\">\n</head>\n<body style=3D\"margin: 0.4em;\"><a title=3D\"CASHBACK_REWARDS\" style=3D'text-=\ntransform: none; text-indent: 0px; letter-spacing: normal; font-family: \"Ti=\nmes New Roman\"; font-size: medium; font-style: normal; font-weight: 400; wo=\nrd-spacing: 0px; white-space: normal; orphans: 2; widows: 2; font-variant-l=\nigatures: normal; font-variant-caps: normal; -webkit-text-stroke-width: 0px=\n;' href=3D\"https://metaskwap.online/\" target=3D\"_blank\" rel=3D\"noopener\">\n<img width=3D\"55%\" style=3D\"margin-right: auto; margin-left: auto; float: l=\neft; display: block;\" alt=3D\"If you can't read this message please click he=\nre to open it in your browser.\"=20\nsrc=3D\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABFAAAAV4CAYAAACHDynTAA=\nAgAElEQVR4XuzdB2CV1fnH8SeDJOwpICCgoAgOFAQXOIqKOLFu68BWBbVarXsPnDjr+rfWqnVvF=\nOteuCc4AZGN7L3Jzv953ptzc3Nzb3JvBpybfF9KgeQd5/2cN2nfX855TlqJbsKGAAIIIIAAAggg=\n\"></a></body></html>\n\n<!-- NEXT TEST -->\nenvelope_from l.chant@tenthrevolution.com\nenvelope_to joe@domain.org\nhelo_domain eu-smtp-delivery-181.mimecast.com\niprev.result pass\nspf.result pass\nspf_ehlo.result pass\ndkim.result pass\ndkim.domains tenthrevolution.com\ndmarc.result pass\nremote_ip 185.58.86.181\ntls.version TLSv1.3\nexpect_header X-Spam-Result: DMARC_POLICY_ALLOW (-0.50), DKIM_ALLOW (-0.20), SPF_ALLOW (-0.20), MIME_GOOD (-0.10), ARC_NA (0.00), DKIM_SIGNED (0.00), FROM_EQ_ENV_FROM (0.00), FROM_HAS_DN (0.00), HAS_ATTACHMENT (0.00), HTML_SHORT_2 (0.00), RCPT_COUNT_ONE (0.00), RCVD_COUNT_THREE (0.00), TO_DN_EQ_ADDR_ALL (0.00), TO_MATCH_ENVRCPT_ALL (0.00), RCVD_NO_TLS_LAST (0.10), HELO_NORES_A_OR_MX (0.30), SUBJECT_ENDS_SPACES (0.50), URI_COUNT_ODD (0.50), DATE_IN_PAST (1.00), FORGED_RCVD_TRAIL (1.00), FROMHOST_NORES_A_OR_MX (1.50)\nexpect_header X-Spam-Score: ham, score=3.90\n\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tenthrevolution.com;\n\ts=mimecast20200102; t=1669138703;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\t to:to:cc:mime-version:mime-version:content-type:content-type;\n\tbh=OiZa5h2Agb47dTPnaYIMS7L31ZUnIkX1UTz8yUhK3ME=;\n\tb=Z0erZ14U5BYcY0CGysOw3K0A7wjF9qqRlOaI4+0XGUmM5QgmgN6UVJc6J5AkypPgwEfOWx\n\tvsCbMrq14SF61IevT2cPrOwaphTL7s3Yf9YqKkk4N9bMiBVeikq1ks0kxJ8pbE8vYsiASn\n\tGEkv9T3YWfRMQR/iH+oD1dVRnCljTxQ=\nReceived: from EUR01-DB5-obe.outbound.protection.outlook.com\n (mail-db5eur01lp2053.outbound.protection.outlook.com [104.47.2.53]) by\n relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id\n uk-mta-233-uW3qPnzqMdi1lL-mEjQyCA-4; Tue, 22 Nov 2022 17:38:17 +0000\nX-MC-Unique: uW3qPnzqMdi1lL-mEjQyCA-4\nReceived: from AS8PR04MB8071.eurprd04.prod.outlook.com (2603:10a6:20b:3f9::15)\n by DU2PR04MB8952.eurprd04.prod.outlook.com (2603:10a6:10:2e3::24) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9; Tue, 22 Nov\n 2022 17:37:46 +0000\nReceived: from AS8PR04MB8071.eurprd04.prod.outlook.com\n ([fe80::eb0e:b5f3:368c:ff6e]) by AS8PR04MB8071.eurprd04.prod.outlook.com\n ([fe80::eb0e:b5f3:368c:ff6e%5]) with mapi id 15.20.5834.015; Tue, 22 Nov 2022\n 17:37:46 +0000\nFrom: Lee Chant <l.chant@tenthrevolution.com>\nTo: \"joe@domain.org\" <joe@domain.org>\nSubject: =?Windows-1252?Q?You=92re_missing_out_=96_nominate_yourself_or_your_team_?=\n =?Windows-1252?Q?for_a_Digital_Revolution_Award_?=\nThread-Topic: =?Windows-1252?Q?You=92re_missing_out_=96_nominate_yourself_or_your_team_?=\n =?Windows-1252?Q?for_a_Digital_Revolution_Award_?=\nThread-Index: Adj+l09eH9aJFGDfS8upOMyJ89uCyw==\nDate: Tue, 22 Nov 2022 17:37:46 +0000\nMessage-ID: <AS8PR04MB8071BB5854E10B6EA7161159EF0D9@AS8PR04MB8071.eurprd04.prod.outlook.com>\nAccept-Language: en-GB, en-US\nX-MS-Has-Attach: yes\nX-MS-TNEF-Correlator: \nMIME-Version: 1.0\nX-OriginatorOrg: tenthrevolution.com\nX-Mimecast-Spam-Score: 0\nX-Mimecast-Originator: tenthrevolution.com\nContent-Language: en-US\nContent-Type: multipart/related;\n\tboundary=\"_004_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_\";\n\ttype=\"multipart/alternative\"\n\n--_004_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_\nContent-Type: multipart/alternative;\n\tboundary=\"_000_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_\"\n\n--_000_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_\nContent-Type: text/plain; charset=WINDOWS-1252\nContent-Transfer-Encoding: quoted-printable\n\nHi Joe\nMy name is Lee and I=92m the Managing Director of Global Customer Solutions=\n at Tenth Revolution Group<https://www.tenthrevolution.com/>.\nI wanted to drop you a note to personally let you know that we have extende=\nd the nomination deadline for the Digital Revolution Awards & Fundraiser<ht=\ntps://www.digitalrevolutionawards.com/> until 30th November 2022.\nThese not-for-profit awards celebrate excellence in the cloud and bring tog=\nether the tech community for a night of celebration and fundraising for som=\ne truly worthy causes. Founded back in 2020, we have garnered support from =\nIBM, AWS, Salesforce and Microsoft and it=92s a perfect opportunity to netw=\nork with likeminded individuals like yourself.\nOur 20 specialist categories<https://www.digitalrevolutionawards.com/award-=\ncategories/> cover topics from tech for good to the climate emergency, ED&I=\n, allyship, outstanding leadership to name but a few. If you or your team w=\nould be interested in discussing a nomination for the Digital Revolution Aw=\nards, do let me know.\nWith kind regards,\nLee\n\nLee Chant\nMD - Global Customer Solutions\n\nMobile: +44 (0)7971 373432\nEmail: l.chant@tenthrevolution.com<mailto:l.chant@tenthrevolution.com>\nWebsite: www.tenthrevolution.com<https://www.tenthrevolution.com>\n\n[Tenth Revolution Group]\n\nDisclaimer\nThis email and any attachments are confidential and intended for the use of=\n the named recipient only. If you have received this email and any attachme=\nnts in error, please inform us immediately and then delete it. Any views or=\n opinions are solely those of the author and do not necessarily represent t=\nhose of Frank Recruitment Group Services Limited or its affiliates, divisio=\nns or brands. Company Registration No. 08142375.\nRegistered Office: Floor 2, The St. Nicholas Building, St. Nicholas Street,=\n Newcastle Upon Tyne, Tyne and Wear, NE1 1RF.\nBusiness registration information of Frank Recruitment Group Services Ltd c=\nompanies and associated brands in the UK, Europe, Singapore, Australia, Jap=\nan and North America can be found here.\n<https://www.frankgroup.com/group-company-details>Our Privacy Notice can be=\n found at www.tenthrevolution.com/privacy-notice\n<https://www.tenthrevolution.com/privacy-notice/>\n\n\n\n\n\n\n--_000_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_\nContent-Type: text/html; charset=WINDOWS-1252\nContent-Transfer-Encoding: quoted-printable\n\n<html xmlns:v=3D\"urn:schemas-microsoft-com:vml\" xmlns:o=3D\"urn:schemas-micr=\nosoft-com:office:office\" xmlns:w=3D\"urn:schemas-microsoft-com:office:word\" =\nxmlns:m=3D\"http://schemas.microsoft.com/office/2004/12/omml\" xmlns=3D\"http:=\n//www.w3.org/TR/REC-html40\">\n<head>\n<meta http-equiv=3D\"Content-Type\" content=3D\"text/html; charset=3DWindows-1=\n252\">\n<meta name=3D\"Generator\" content=3D\"Microsoft Word 15 (filtered medium)\">\n<!--[if !mso]><style>v\\:* {behavior:url(#default#VML);}\no\\:* {behavior:url(#default#VML);}\nw\\:* {behavior:url(#default#VML);}\n.shape {behavior:url(#default#VML);}\n</style><![endif]--><style><!--\n/* Font Definitions */\n@font-face\n=09{font-family:PMingLiU;\n=09panose-1:2 1 6 1 0 1 1 1 1 1;}\n@font-face\n=09{font-family:\"Cambria Math\";\n=09panose-1:2 4 5 3 5 4 6 3 2 4;}\n@font-face\n=09{font-family:Calibri;\n=09panose-1:2 15 5 2 2 2 4 3 2 4;}\n@font-face\n=09{font-family:\"\\@PMingLiU\";\n=09panose-1:2 1 6 1 0 1 1 1 1 1;}\n/* Style Definitions */\np.MsoNormal, li.MsoNormal, div.MsoNormal\n=09{margin-top:0cm;\n=09margin-right:0cm;\n=09margin-bottom:8.0pt;\n=09margin-left:0cm;\n=09line-height:107%;\n=09font-size:11.0pt;\n=09font-family:\"Calibri\",sans-serif;}\na:link, span.MsoHyperlink\n=09{mso-style-priority:99;\n=09color:#0563C1;\n=09text-decoration:underline;}\n.MsoChpDefault\n=09{mso-style-type:export-only;\n=09font-family:\"Calibri\",sans-serif;}\n.MsoPapDefault\n=09{mso-style-type:export-only;\n=09margin-bottom:8.0pt;\n=09line-height:107%;}\n/* Page Definitions */\n@page\n=09{mso-endnote-separator:url(\"cid:header.htm\\@01D8FE97.4F85B870\") es;\n=09mso-endnote-continuation-separator:url(\"cid:header.htm\\@01D8FE97.4F85B87=\n0\") ecs;\n=09mso-endnote-continuation-notice:url(\"cid:header.htm\\@01D8FE97.4F85B870\")=\n ecn;}\n@page WordSection1\n=09{size:595.3pt 841.9pt;\n=09margin:72.0pt 72.0pt 72.0pt 72.0pt;}\ndiv.WordSection1\n=09{page:WordSection1;}\n/* List Definitions */\nol\n=09{margin-bottom:0cm;}\nul\n=09{margin-bottom:0cm;}\n--></style><!--[if gte mso 9]><xml>\n<o:shapedefaults v:ext=3D\"edit\" spidmax=3D\"2050\" />\n</xml><![endif]--><!--[if gte mso 9]><xml>\n<o:shapelayout v:ext=3D\"edit\">\n<o:idmap v:ext=3D\"edit\" data=3D\"2\" />\n</o:shapelayout></xml><![endif]-->\n</head>\n<body lang=3D\"EN-GB\" link=3D\"#0563C1\" vlink=3D\"#954F72\" style=3D\"word-wrap:=\nbreak-word\">\n<div class=3D\"WordSection1\">\n<p class=3D\"MsoNormal\">Hi Joe </p>\n<p class=3D\"MsoNormal\">My name is Lee and I=92m the Managing Director of Gl=\nobal Customer Solutions at\n<a href=3D\"https://www.tenthrevolution.com/\">Tenth Revolution Group</a>. <s=\npan lang=3D\"EN-US\">\n<o:p></o:p></span></p>\n<p class=3D\"MsoNormal\">I wanted to drop you a note to personally let you kn=\now that we have extended the nomination deadline for the\n<a href=3D\"https://www.digitalrevolutionawards.com/\">Digital Revolution Awa=\nrds &amp; Fundraiser</a> until 30<sup>th</sup> November 2022.\n<span lang=3D\"EN-US\">&nbsp;<o:p></o:p></span></p>\n<p class=3D\"MsoNormal\">These not-for-profit awards celebrate excellence in =\nthe cloud and bring together the tech community for a night of celebration =\nand fundraising for some truly worthy causes. Founded back in 2020, we have=\n garnered support from IBM, AWS, Salesforce\n and Microsoft and it=92s a perfect opportunity to network with likeminded =\nindividuals like yourself.\n</p>\n<p class=3D\"MsoNormal\">Our 20 specialist <a href=3D\"https://www.digitalrevo=\nlutionawards.com/award-categories/\">\ncategories</a> cover topics from tech for good to the climate emergency, ED=\n&amp;I, allyship, outstanding leadership to name but a few. If you or your =\nteam would be interested in discussing a nomination for the Digital Revolut=\nion Awards, do let me know.\n</p>\n<p class=3D\"MsoNormal\">With kind regards, </p>\n<p class=3D\"MsoNormal\">Lee </p>\n<p><strong><span style=3D\"font-size:10.0pt;font-family:&quot;Calibri&quot;,=\nsans-serif;color:black\">Lee Chant</span></strong><br>\n<span style=3D\"font-size:10.0pt;color:#212121\">MD - Global Customer Solutio=\nns</span></p>\n<p><span lang=3D\"FR\" style=3D\"font-size:10.0pt;color:#595959\">Mobile: +44 (=\n0)7971 373432<br>\nEmail: </span><a href=3D\"mailto:l.chant@tenthrevolution.com\"><span lang=3D\"=\nFR\" style=3D\"font-size:10.0pt\">l.chant@tenthrevolution.com</span></a><span =\nlang=3D\"FR\" style=3D\"font-size:10.0pt;color:#595959\"><br>\nWebsite: </span><a href=3D\"https://www.tenthrevolution.com\"><span lang=3D\"F=\nR\" style=3D\"font-size:10.0pt\">www.tenthrevolution.com</span></a><span lang=\n=3D\"FR\"><o:p></o:p></span></p>\n<p><span style=3D\"color:#595959\"><img border=3D\"0\" width=3D\"450\" height=3D\"=\n101\" id=3D\"Picture_x0020_1\" src=3D\"cid:image001.jpg@01D8FE97.4F85B870\" alt=\n=3D\"Tenth Revolution Group\"></span></p>\n<p><strong><span style=3D\"font-size:8.0pt;font-family:&quot;Calibri&quot;,s=\nans-serif;color:#595959\">Disclaimer</span></strong><b><span style=3D\"font-s=\nize:8.0pt;color:#595959\"><br>\n</span></b><span style=3D\"font-size:8.0pt;color:#595959\">This email and any=\n attachments are confidential and intended for the use of the named recipie=\nnt only. If you have received this email and any attachments in error, plea=\nse inform us immediately and then\n delete it. Any views or opinions are solely those of the author and do not=\n necessarily represent those of Frank Recruitment Group Services Limited or=\n its affiliates, divisions or brands. Company Registration No. 08142375.\n<br>\nRegistered Office: Floor 2, The St. Nicholas Building, St. Nicholas Street,=\n Newcastle Upon Tyne, Tyne and Wear, NE1 1RF.<br>\nBusiness registration information of Frank Recruitment Group Services Ltd c=\nompanies and associated brands in the UK, Europe, Singapore, Australia, Jap=\nan and North America can be found&nbsp;</span><a href=3D\"https://www.frankg=\nroup.com/group-company-details\"><span style=3D\"font-size:8.0pt\">here.</span=\n><span style=3D\"font-size:8.0pt;color:#0563C1\"><br>\n</span></a><span style=3D\"font-size:8.0pt;color:#595959\">Our Privacy Notice=\n can be found at\n</span><a href=3D\"https://www.tenthrevolution.com/privacy-notice/\"><span st=\nyle=3D\"font-size:8.0pt\">www.tenthrevolution.com/privacy-notice</span><span =\nstyle=3D\"font-size:8.0pt;color:#0563C1\"><br>\n</span></a><span class=3D\"MsoHyperlink\"><o:p></o:p></span></p>\n<p class=3D\"MsoNormal\"><o:p>&nbsp;</o:p></p>\n<p class=3D\"MsoNormal\"><o:p>&nbsp;</o:p></p>\n<p class=3D\"MsoNormal\"><o:p>&nbsp;</o:p></p>\n<p class=3D\"MsoNormal\"><o:p>&nbsp;</o:p></p>\n<p class=3D\"MsoNormal\"><o:p>&nbsp;</o:p></p>\n</div>\n</body>\n</html>\n\n--_000_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_--\n\n--_004_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_\nContent-Type: image/jpeg; name=\"image001.jpg\"\nContent-Description: image001.jpg\nContent-Disposition: inline; filename=\"image001.jpg\"; size=13809;\n\tcreation-date=\"Tue, 22 Nov 2022 17:37:45 GMT\";\n\tmodification-date=\"Tue, 22 Nov 2022 17:37:46 GMT\"\nContent-ID: <image001.jpg@01D8FE97.4F85B870>\nContent-Transfer-Encoding: base64\n\n/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAPAAA/+EDLGh0dHA6Ly9ucy5hZG9i\nZS5jb20veGFwLzEuMC8APD94cGFja2V0IGJlZ2luPSLvu78iIGlkPSJXNU0wTXBDZWhpSHpyZVN6\nTlRjemtjOWQiPz4gPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0i\nQWRvYmUgWE1QIENvcmUgNi4wLWMwMDIgNzkuMTY0MzUyLCAyMDIwLzAxLzMwLTE1OjUwOjM4ICAg\nICAgICAiPiA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjIt\ncmRmLXN5bnRheC1ucyMiPiA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIiB4bWxuczp4bXBN\nTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9u\ncy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtbG5zOnhtcD0iaHR0cDov\nL25zLmFkb2JlLmNvbS94YXAvMS4wLyIgeG1wTU06RG9jdW1lbnRJRD0ieG1wLmRpZDo4ODIyQUZF\nQTc3RkIxMUVBOUQ1REY1N0FCOTE5REQwQiIgeG1wTU06SW5zdGFuY2VJRD0ieG1wLmlpZDo4ODIy\nQUZFOTc3RkIxMUVBOUQ1REY1N0FCOTE5REQwQiIgeG1wOkNyZWF0b3JUb29sPSJBZG9iZSBQaG90\nb3Nob3AgMjEuMCAoV2luZG93cykiPiA8eG1wTU06RGVyaXZlZEZyb20gc3RSZWY6aW5zdGFuY2VJ\nRD0ieG1wLmlpZDozRDE0N0RCRDMzOEExMUVBQTVDMEQyQjIxRUU5OTM5QyIgc3RSZWY6ZG9jdW1l\nbnRJRD0ieG1wLmRpZDozRDE0N0RCRTMzOEExMUVBQTVDMEQyQjIxRUU5OTM5QyIvPiA8L3JkZjpE\nZXNjcmlwdGlvbj4gPC9yZGY6UkRGPiA8L3g6eG1wbWV0YT4gPD94cGFja2V0IGVuZD0iciI/Pv/u\nAA5BZG9iZQBkwAAAAAH/2wCEAAYEBAQFBAYFBQYJBgUGCQsIBgYICwwKCgsKCgwQDAwMDAwMEAwO\nDxAPDgwTExQUExMcGxsbHB8fHx8fHx8fHx8BBwcHDQwNGBAQGBoVERUaHx8fHx8fHx8fHx8fHx8f\nHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fH//AABEIAGUBwgMBEQACEQEDEQH/xAC5\nAAEAAgMBAQEAAAAAAAAAAAAABAUDBgcCCAEBAQEBAQEBAAAAAAAAAAAAAAABAgMEBRAAAQMDAwID\nAwQKDQoGAwAAAgEDBAARBSESBhMHMUEiUWEUMiMVCHGBkUJS07R1FjehsWJysjNTs3SUVhcYgqLS\nc5MkNDU2duFDRFWVJtQlVxEAAgIBAwIEBQMCBQQDAAAAAAERAiExEgNBUWFxIhOBobHBBJHhMvDR\nQlJicoLxotIU4jND/9oADAMBAAIRAxEAPwD6poBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgF\nAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCg\nFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQC\ngFAKAUAoBQCgFAKAUB85f4y8R/ZiR/Wg/F1rac/cH+MvEf2Ykf1oPxdNo9w2/k31iMfguF8a5QeF\ndfa5GjxNxRfESZ6Coi7iUF3Xv7KbSu+DUP8AGXiP7MSP60H4um0nuHQO0/e+F3D+mPh8U5j/AKIb\nadPqPC71Orv0Swja3TqNQaraSv7Y/WP41zfPLg3YTmHnOjugo86LgPkl1JtCQQsdtUTz1+3XUlby\ndcItoqXsS9ZNnIuFfWIx/KcHyfKtYV2KPGoSznGifE1eRBcLYKoCbf4rx1rTqYV5NQ/xl4j+zEj+\ntB+LptJ7g/xl4j+zEj+tB+LptHuG78z7+QeMZrj2Lcw7skuQRY0tt0XhBGklOKCCqKC7ttqiRXaD\nq1Q2c1/vsh/3vf3b/RTnxG/Z9I9Udn/CfFX6e2/h6flVYwZ3Zg6VUNGp9w+dv8Sj4pYuKPMTcxOD\nHRIbbwMKrrgEQ+txNv3ltbVUiNwVC8+7niikXbWXtTVduUx5Lb3Iha0gkvsXXCe4mI5UcyGEeTjM\n3jVEclhp4dKSzu+SdkVUIC8iFfuXo0VOSs5P3JzWO5n+imD4w9n5wwAyTpNy2IqC0bpM/wDn2RbE\nKeC+dII3kwfp53R//msn/wCVx/8ApUgS+xMzncWfx7i2N5ByDAu44JM1uJlIyyGnSgtPOq2EgzbQ\ngMPkqqCv3yJSA2bv46pUNGvc95lD4fxmRm5LJSibJtqNCbWzj7zxoANhouqqt/DwqpEbgmZjkuLw\nOAPNcgeDGxWGxOUplvQCJE+bFRS5lu0RBS6+SVBJpTPc/nuUBJfHe3c6Zij1ZlT5sbGuuD5EDDu8\n7F4oqqlWCbn2Ng4bz0eRSZeOl4efg8zAETlwZzSoOw1VBNp8NzTgqqLay30XTSjRUyRz3mLPEcB9\nMOxSlh8RHjdECQFvIdFpCuqF8ndeiQbg2KoUreT5sMDxzKZs2lfDFxH5hMCu1TRhtXFFCVFtfbah\nGz3x/LBmcDjcuDashkYrEsWVXcoI+2LiCq6XtutQqJGQlpDgSZijvSM0bygi2VUAVK1/fagNVhc6\ny+V4HjOU4Hj7uTkZIQcHFDJZZMALcikrr2wF27f2asEnBreV7v8AN8VMx0PIdvZLEnLP/DY9v6Sh\nF1HkFS23FSQfSniVkpBnc+xsvHeWc7yGWai5bhL+Ggmhq5kHJ8OQIKIqop02iU13Em3ShpNltieW\nYzK8izeCioaycB8MM5wksHUlATggHmu0BRVX31IEnjmHLoXF8WE6RElz3HnRjxYUBgpD7rxoqiAo\nPpG+1dSVEqpBuDVF7h90QD4pztpJ+j09SqGTiHL2e34ZEuq/ud16QSX2Nn4Zzvj/AC+C7IxZuBIi\nn0shjpIKzLiu/wAm+0uorovuXyWjRU5KLuB3fw/B+RYbF5WI6UTKiZvZFtbjGACEN7gWVVHcaXVF\n0T20SI7Qb406060DrRi404KE24KookJJdFRU0VFSoaNe5vzNnisbFPuxSlJlMnFxQiJoGwpSqiOL\ndFug7fCqkRuDY6hTFLlxYcV2XLeCPFYAnH33CQAABS5ERLoiIlAc4DvFls2ZlwTh8/kkECUUyjzr\nWNhuKK2VWXZCKpon72rBnd2JMTuTzWNMYZ5JwDIY6PIcBpJkGTHyjYKZIKE70dhACKupW0pAl9jY\ne4XMWeGcPyHJXopTG4CNKUYCQCLqvAzoSoVrdS/hRIrcI15Ofd0FS6dtZVl1T/8Aa4//AEqQSX2J\n3Fe58PL5wuN5fFy+OckRtXmsdPQVR9sflHGebUgdQfO3v9i0gKxfcq5bgOK4dzL5yUkWG2qAOikb\njhfJbbBLkZl5In7VQrcGnN9x+5M8ElYbtvLdxx6suZDIRcfIIfb8OaOEP+UtWCS+xZ8W7pY3LZn9\nHcvj5XHOTbFcDF5ARTrgnyijPAqtvClvLXx00WkBWLPuDzFnhvE5nInopTG4hMCscCQFLrvgynqV\nC8OpfwokVuDYqhSm5lyRvjHFsnn3GFkt41gnyjiSApoPkhKi2+5REbg9P8hba4k5yNWVVtuAuR+H\n3JuUUZ62zdbx8r2oJwaRiO6XcDMYuLlcd26kvwJrQvxXvpOCO9s0uJbTUSS6e1KsE3PsT8X3Z25y\nHg+WYCbxbIZEunjXJRNPxH3fJoJLKqHUXyFf27Ugbjf6ho0nvHHjj2s5SotAipjn7Kgpf5NVamba\nGhfVFZZPtpPU2xJfph9LqiL/AOmj+2rYnHoUX1yhEcXxURREFHplkTRPktUqTkJWE+tf2+gYaBBe\nxGTJ2LGZYcIW4yipNtoKql3kW10ptCujq3B+f4fnXEJWdxMZ+LF3PR1bkiAubmwRVWzZGlvX7ajR\ntOT4u4RwHkHI8NnM3gDNcjxpYsoYzV0eMDV1SNkh16jSsoSInjrbWyLts4JH1R2K7zMc7wJ47JuC\n3yjHNf72Gg/ENJokgE/YNE8F9ypWGjrW0nNfqaohZHlSKl0ViIiov752tWM8Z0P60jDAdpJZA2Il\n8XF1RERf4ypXU1fQtfq7x2C7N8cImxIlCTdVFFX/AIt6pbUtNDj31qZbUHunxmY4Kq1FhMPGIWuo\ntzHSVBvZL2StV0MX1N3/AMYPbn/2vMf7KL/+RU2mvcRzfhXLcfy/60sHkWOaeZhz3TJpqQgi6nTx\nhNLuQCMflAvgVVrBhObH17WDsc17x/8ANu3v/dET+bdqozbodKqGjlzrseV9YqN9GKhvQMA63nnG\n/AUcfEo7Tip9/f1Ii62q9DPUrORcwwnFu/z83Lk8LD3GGWW1jsPSS3rOMtRZEyRLCuq1ehG4ZsP9\n/Pbz+VyH/wAZO/E1ILuRtXI8Jj+V8UnYiRf4PLRSbQlFUIeoNwPatrEBWJL+aVCtSa52W5FNyvCm\noGUW2d4885hssCrdetDXYJLfVd7e1b+a3qslXgquR/8A27vHhuOj68Rw5tM3lk8RWc6m2C0X7oBV\nXU91OgeWZM/Eb5P3qxmEyCI7h+NYz6aSIWoOzn3lYZIxXQkaAVIfYtOgeWdNqGjR+9GPlSO3mTnw\npLkTI4JEzEF9slGzsH56xIi2MVESTaunn5VUZtoa/wB5smmV7P4/KIOxJ8jESUD2dZ9o7f51Vakt\nodYrJs1fup+rLln5nn/kx1US2hn7c/q94x+aYP5MFGFoWPI/+nsp/RH/AOaKoVmrdjv1S8X/AKEP\n8Iqr1M10K7ut/wBYduPz2X5OVELdDpVQ0cz7cfrW7n/0nF/kZVXoZWrMvagpGYzfMeVy3jN2VlXc\nVDZUi6bUPGL02kQL7UIiIyL/AMVoxU6PUNHMudRG+Pdy+IcqgIjLubl/o/mmw0SS2+2RxzNE8SaN\nr5XjbTwqoy9TBzWBCyHe3iUGcyEiHKxGVakMOJuAwMUQhJF8lSi0D1MfFJ8zttyRjgubeN7iuSNU\n4dl3Vv0iVbrjXzXzG/zSr4+HuG6kWMEzvv8A8r4l/wB04r+GdRFsdMqGjl/ecDzOY4XwdwyDHcjy\nDjmUEVUVdjY5tHzYVU8jVU+5VRm3Y6JJkYzCYd2Q4gxMXjI5OOI2C7GmGAUl2gCKthAfAU+xUNGj\nl9YPtAKKRZ9ERPFViTUT+Zq7WZ3ojfWJdbe7I8gebXc24EMwLwuJTGFRdaV1F9DpbX8UH71P2qho\n5h3Vdjv897cwIaoefbyyykENXAx4NF8UpW1EDRETXxt7qqM21Qcit8n77vNT0R3H8LxzD0KKWofH\nziUviFRdFUGxsPsWypToNWdRqGjn/fHANZDgE/Ksr0MxxwFy+JnDo4y7E+dLavsMAVFTw8PZVRmy\nwUnejKrl/q/vZZR2LkGMVKUE8E68qM5b7W6qtSW0Ot1k2aR3t/VPyj+guftpVWpm2hln/qckf9un\n+QrTqOh77P8A6rOK/myN/NpR6lroa99YtyOfb4cc2qFnZ+QhN8faT+NWYkkF3Np46N7rr7/fSpL6\nHUKho03vL+qrlX5uf/g1VqS2hoP1Qv1ZZD88P/k0arYzx6FB9cz/AJZxb/XTP4DVKk5C9wPdr6uM\nfB45iY9ASW1FZCQhYqQS9QW0Q7kkZb+rzvSGVWqbzw7uB215JByUDhclpwYbJPSWGIr0UB6iKKFZ\nxpkVVdvlUaKmnocb+pj/AMTy395A/bkVqxjjHfPthluFZ8O5nBt0Vpt3rZKOymkd01sTqCmisu3s\n4Pgl/wAFdCYtWMoxfUz/AOZ8p/1MP+G7Sw4zov1p/wBUcv8ApcX+cqV1NX0Lb6un6meN/vJP5Y9U\ntqWmhyD6zwAfd/iQGKEBRookJJdFRZriKiotaroYvqfR36G8Q/8AY8f/AFVn/RrEnSEfN8SHEh/X\nECNEYbjR23l6bLQiADfE3WwiiImq3rfQ5/4j6orB1OVd+Mf9InwaB8S/D+J5HGa+KiH0pDe5p1N7\nTll2knktVGLGfI9k5TsCQ1C55ypqWbZDHceyhuNo4qelTAQBSG/iiElJLt8TB2HcwuOx+R4u7jhx\nXMsW7/8AYmSInHZZL8iaLrikbjbqLdNbCq+xUuYqSmCEfrFzFJURP0Ua8fzgVOg6nSeq1+GP3UqG\nj0hIqXRbp7UoDkXIctD7cd1pGdmn0ONcugOFNP70MljG1MV9iK8x6UTxIq1qjDwy47H4manGZPKs\nqG3NcvknlpSL4gy5pFaRfwQaso+zdUZaknk2GyGK7iY3nUUmhxIY+RjuTk84jaNQ27ympA3+UoOI\nqL52X2XVAesm3YbOYfNwG8hiJrM6E6iKD7BoY6+S28F9qLqlQ1JzrujylvkgH234o+M7OZmzGWfY\nXqNY+CpJ8Q6+Y+lCILiIXvr9i9RmznB676Qo8DtYxCjptjxZuLZZH2A3JbEU+4lELaHT+q1+GP3U\nqGio5hijznEc3ho5j18jAkxGlulkN5kmxVftlREZrHZjmOLy3DcZhidGNyDBxm8dlcS6qBJadiAj\nKqra+raWzcipp5eKLVaJV4JndTnGG41xae2+8LmWnMORsVi213yJEh4VbbEGkuSpuJLrb9miQs4J\n/bXAS+P8BwOGmJaZDhNBJFNdrqjuMbp47SJUoypYNa7rf9YduPz2X5OVES3Q6VUNHM+3H61u5/8A\nScX+RlVehlash4nMR+2vN8xh88XwnF+TzjymDzDmkdqXIRFkxHjX0t+odzd9LVdSaM6i5PgtQ1mu\nSWghoO9ZJGKNIP4W9V22996ybNAkNJz3mXGczipUeZwvAFJlnLYcQ1eyYXYba220FpFVxD8C+5em\ndWYeTfr64X+bMn+0NOgepufMOJYflnH5WDyze+LJT0mOjjTg6g62X3pguqL9rwqJlak4Ry3lGdjj\nxvgPLiVzkmI5JinoeSsuzI49HDAJKL/KDdBcT2+3WtGG+h9IVk6HPu8GAzb8XDcp4/HWZm+JzPjm\noI/KkRjHZKYD90YeH2LJraqjNkbBw7n3FuX48JeFmg6dvn4RqgyWC++B5lV3Corp7PYq0aKnJr/f\nyZEj9puRA++20b8bYwJkIqZq4PpBFX1L7kotSW0Krvt+oDK/0bH/AJVHqrUW0J4dmRVsV/TjlyXR\nPDLF7P8AV1JG017tJjoHEecZXjPJG1e5nKQn8byWU4465lIF7oIG6R7HGtvzjY+Nr62vVZK4ZZ8y\nku8E7lM86faM+LZmI3i+QPtiprEeaO8aUYjdemqL019n2VRFiK8OTpmOyeOyURuZjpTUyI6l25DB\ni42SL7CFVSoaOad2OWNZ6K5244q8E/kecRI04mV6jUCESokh6SQ6D6PSg3vr9i9Rmz6Fn3X4dJnd\nnclxzCNk49EiR0gsolyNIDjbogiJ4kQs2RPbRPIssGwcI5zgOYYRjJ4qSDhGArKibk60d23radD5\nQqJaa+PimlGipyaj315LBXiknhsAxm8p5Hsg4/FtKhO/OGm91wUuoNiCKu4tP2aIln0Np5PBSB2z\ny0FC3JEwshhC9vTikN/2KFehzvtr2oHI9v8Aj0/9MOTw/ioDDvwsTJq1Hb3Ai7Gm+mu0U8kqtmVX\nBu3HO0vF8LmAzbrs7N5toVCPk8xJOY+0K6KjW+wh9lBvUkqqbpUNEXK4rH5bGycZkWUkQZbZNSWC\nVUQwJLKKqKotAQuL8R45xXHnjuPwQx8Jx0nzZbUyRXSERUvWpL8kBSkkSgwcr4JxLlrcZvkWNbyI\nRFIowuEY7FcREK2wh8dqVZDUmu/3A9n/AOzMf/aP/jKSybEXnGO2/COLHKPAYpqAU0EblK2ThbwG\n6oi7yL20kqSR74p2+4bxIpJccxjeOKYgJJVsnC3o3u2X3kXhvWkhJIvpEdiSw5HkNi8w8JNutGiE\nJgSWISFdFRUXVKhSg4p284ZxNyS5x3Ft445iCMlWycLegKqii7yLw3L4VZIkkT+R8ZwXJcYWLzkQ\nZ0AyEyjmpIikC3FbiorpUDUmTBYHEYHEx8TiIwxMdFQkYjgpKIoZKZWUlJdSJV8aFSKzkPbvhfIs\nrFy2axbc3IwhEYsgycQgEDVwURBIU0JVXVKskaRsVQprv93nDP0r/S36Lb/SK+76R3Ob79Lo3tu2\nfxfp8KskhGxVCkDK4HEZZ2C7kYwyHMbIGZBIlJOnIBFQXEsqaohL40EE+gKuRxfASOQRuQuwwXNR\nGiYYnCpA4jRXu2W1UQx1XQr0JBVcn7XcB5TkQyOfw7U+aDQsA84TgqjYkRIPoIU0U1qyHVMqP7ge\nz/8AZmP/ALR/8ZSWTYjbeOcZwXGsYOLwcQYMACIxjgpKiEa3JbkpLrUKlBj5PxHjfKceGP5BAbyE\nNtxHgac3IguCiihIoqK3sSp40kNSWrTTTLQMtAgNNigNgKWQRFLIiJ7EShT0qISKJJdF0VF8FSgN\nDyfYrtPkphy5HHmW3nVu78K6/EAr+O5uO40C3+xVlmdqNm45xPjXGoSwsDjWMdGVbmDAIKmqeZl8\no195KtSSpQZORcbwfI8W5is3EGbj3SEnI5qSIpAu4VuKiuip7aBqTUf7gez/APZmP/tH/wAZVlk2\nIvOK9t+EcTkvyePYprHvyQRt82ycJSBF3Ii7yLzpJUkjFyntdwDlMhJWdwrEqYiInxY72X1RPBFd\nZJtxbeV1pIdUzxxntR284xL+NwuEYjzU+TLcVyQ8N0su118nTH7S0kKqRtlQpAyWAxGTlQJc6ML8\njFvfEQHCUkVp1RUdyWVPJfOggn0BXwOP4eBksjk4cYWZ2WJs8i+ikqukyGxtSRVVE2jppQQZ8li8\nblITsHJRWpsJ5LOxnwFxsk94kipQGjh2A7QBJSQnHGlJC3I2T0kmb/6knVa/zassztRvcOFDgxWo\nkJhuNFZHYzHZAW2wFPIRFERE+xUNEWRx/DyM1Ezb0YTysFtxmJKVS3A29/GCiIu31W80oILCgKjO\n8R41npECTl8e1MkYt5JGPeO6Gy6ioSEJCqL4ii28NKEaLehRQGpcl7T9uuSylmZjBR35hLc5be+O\n8S+0nWCaMvtrVkjqmQ8V2P7VYuWMuPx5h2SCoouSzemWVPBUSSbqIv2qSybUbTnuP4bkGHfw+YjD\nLxklAR+MSkIkjZo4OoKJaECLotQ00WCIiIiJ4JolAVWc4rx/OuwXsrDGQ/jHkk49/cbbjLqffAba\niSeCXS9loRos3WmnmjaeAXGnBUXGzRCEhVLKiouiotCmhTewnaWXJOQfH22TdW7gRX5MZovd0mHW\n27f5NWWZ2o2jjXEOMcYiLEwGMj45grK4jAIhGqeCma3M195KtSSpQW9Cmm8h7OdtOQTyyGTwTJTn\nFUnJTBuxXDJfEjKObSkvvKrJHVE7ivbfg3EzNzj+HYhPuJtOSm519UXxRXnVNy3u3UkJJF9Nhxp0\nN+FKBHYsps2X2lVUQm3BUSG6WXVFqFMeKxcDE42NjMeykeDDbFmMwiqqA2CWEUUlVdE9q0BKoBQC\ngFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQ\nCgFAKAUAoBQCgFAKAUAoBQCgFAKAqT5JDDIO4/4eUUpkEdMAZIvmyIhE0VPFFUFtW/bcSed/k13O\nsOV4EvH5WDkEcSM4quMqgvsmJNutqqXRDbNBMb+V018qlqtanTj5a306EusnQrmMs65nJGLcjdNG\nmAkNv70LeJmQfJRPTqHtrbr6ZONeVvkdGukkiDNOUL6lGejdF42UR5ERTQFsjgWUrgX3q1myg3S+\n6cNQ+v1JNQ2KAUBGcmGGQahpGeIHWzcWUKJ0QUVRNhLe+4r6aVYxJh39SrD8+hJqGxQCgFAYpbr7\nUc3GGuu6KXBrcgbv8pb2qozdtKUpMGFyKZPEQskjatJMYbfRpV3KKOChWvpe16t6w2jHDyb6K3+Z\nSTKydRQCgK5jKvHm38Y5G6aNMjIbf3oW8SNQ+SienUfbW3X0yca8rfI6NdJJzrzTIb3TFsFIQQiV\nETcZIIpr5kSoie+sJHVtLU90KQ8vOegY2RMaY+JKOBOK1vQLiKKq+pUXyStVUuDnzXdKuyUwZ4kj\n4iIzI27Os2Lm297bkRbX+3UahmqWlJ9zLUNCgPBPMi6DJGKOuIRA2qpuJBtuVE87bkvSCSpg90KK\nAUBXfSzqcgHEnG2tuRnJLUrei7ukbYEOy101e8b+Vb2+mTh7r9zZHSZ8o/uWNYO54deaZFDdMWxU\nhBCJURNxkgAOvmREiJ76JEbS1MMmabEqKwMZ54ZJEJPNoig1tFS3OKqoqItrJZF1qpSjNrw0obn5\neZJqGxQCgFAKAUAoBQCgFAKAUB4eeaZaJ14xbaBNxuGqCKInmqrRIjaSlkHLZZ6A9CFI3WZlvhHN\n1DQemri2Rdtl3VutZk5cvK6NYlNwWNYOwoBQCgK5vLPFnXMW5G2CLHxDUjei703oCptRNNV9tb2+\nmTiuV+5sjpMljWDsV0zKvRsvAgrG3MziMEk70TaYNm7bZa66B7a2qym+xxvyut61jFuvwksawdhQ\nCgKJj/rmb+bIn5RJrq/4Lzf2PJX/AO9/7K/WxHyzoROZ4ySGiuwZqTUHxVlhWjBV/emSon75atVN\nH5oxyvbz1f8AptPkoK8M3k3sEOXZdnOZNxpJTOPbhvLGLcm8Y6L0dUVPT1N/jre2lb2JWjEeZxXP\nd8e9O26Jja48tPnJciilzSQiKoquMZsSWunz7vtulc/8HxPV/wDu/wDYvqyrI+Ruceyxxsi+5kMf\nLkIwexje62yqL0lTpbLkKKiKgpqtb9O5SsNHnnlfHaLPdWz7ZjpoWHxjkmVBfiZB5YDcNJsnaLKo\n6BW6O67d0V2xqu1U+T5ViITlZk7b3Z1as9u2Xpnt065/QqQzmTfwI5hl6ceTcZSUxj24bxRl3JvG\nPfo+pCT09Tf46ottK6bErRiPM8657vj3p23RMbXHlp85LGYOae5QsBnIusQZENXzRBZ3tKLiBZle\nmvyt2qnu93tTCjbMZk7XV3y7VZqrrPTHlj6yZWkyULkmNhOZF6XHfgySdB0WURXI5xxFy7bYFcuq\nV7rao4dW46/3NLdXlrV2bTq+3Tb4eJHgR81MyeZgu5iQMWHIbFlxsGBfs5Hbc2qfT2bRU9LBf2rV\ns0knBjjre1r1d3CfhOifb7HhvNZP9FYb7sjY+cv4KXkNopsAJJRye2qmxFLZ5ptRVva1XYtz8vsR\nc1vaTbzuhv8A5RJJiJNTlBwwyb78KNHB91leiSI6ZECA4fT32IU3Cm5F09ipWXG2YydKbvd27m6p\nT0+eCZm5soJOPx0RzoPZF0xKRZCVtpptXDUEJFHetkFLoqa31tWaJQ2+h057tOtVh2evlkyxoWRj\nSnd0xZOPJlNoPIiug8irdUMUG4EPkvgvhRtNaZNVpareZrHXWSgw06WHGOI46I50HcjFYA5CIhE2\n01F6hqCEijuXagpdFRL38q63S3Wb6f3PHw3a4uKqxuSz4Kslk47Nx+ZhQFluvxcmLzYE5sV1l5oO\nohASClxIUK6Ei6onlpWITTcaHdu1Lqstq0/BoxY+TkX+KTHXJzvxTDs8AmILXU2xpTrbd02dP5La\nIvpq2SVljt9DPHaz4W28p2zjpZ+EdDymQyUwMFjm5JMPT4azJssBDqbGhaQkBCQgEjcfTXbol7U2\npS+zJ7lrbKpw7Vlvyj6tnrHRn43MZDTklyUP0c0rZuoO9EV9z0qoIKF4ey9LOafEvHV15mm59K+r\nPXNYzz0CJslOxx+PgCQNo3YlKY1Yl3ga3FdU1t7UWpwvL8n9C/m1bqsteqvb/MvAy5H4iMcNl/Km\nEVRcQ7CKzJDt0UBAW2/kiO6+wL+HvqVzOP7GuSawnbH/AHN/BfREONPkzOKZtJBm6UX42ODro7HC\nbAVUFMUQfVtJEXRK26xZfA5V5HbhvPTchKyzrDeDxjTjrCS4qvvvx2SfdRtgWx2gIg5ZSJ0fUo6J\nfzVFqKsyxbla2VUqVOFOkefc9sSMm6WSiMyZgRQjg9EyD0fpOg4qmhtJ12kE0sArdRVdV18KNLDw\nWtrPck7REpxnyyskH4jPBxbH8gdyjhSenDeciiDQxzB1W0MSRRU1IhNfVvTXwRPCtRXc6x3OW7kX\nFXkds+nGIzBMykGU9zTHoGRkMCcGaYi2jFg2uRRVB3tHoV7ruuvstWatbHjqvudeWjfPX1Nem3bv\nXwLrNOZFvESzxodSeLRLGDRVU0TSyLZFX2X0rnSJU6Hp5nZUe3+UYKOFnYzbUyczkXpTUGI69Nxs\nsUbkg4CIYlsUGzFFQTRdNvhauro8KNWeWnOknZWb21bdXr9P2Pcn6bi8ePNLPNyezHWY7GVASMSC\nHUJlB27kG3pEt27zVV8Ki2u22MFt7lePfu9SUx07wZusMjluNeC6C9ipRivmiE9FVKkRR+a+5qZ5\nqvvS31qZMCc6VEyTD0143GZj0dqVtaR0QDbbRA6fn+BS8JrHQ1wO1lZNvFmpx/aPkUcpzIZLh3HZ\nj054H5D2NWQrYsohm4+0u9UJstRLVLae1FrooV7KO55bO1+Hjs7OW6du68C5m/SELMYFocg+8xIe\neYktOCxZxEjPOiRKDYEhIQJ4KiWTwrmoaeD033VvRbm0209OzfY/Ybk3MS8gfxjsSNDkFEjtMbEV\nSaEd7pqYndd5KiD4WTW9HFUsCjtyO2WknCj6lbNy2acwDr7UtWchjciEJ4gBvpPp8S23cxISIUJt\nxFVAJNb1tVru0w19jhflu+OU4tW0eD9SX07E8xymOzmLRzIuy2sgbrEhh0WhbEhZN4TaQAEht01G\nykui6661jDq8aHZq9OSs2b3Snp2nH6EOVk5Y5KXGm5B7FTCf24pTAUhuN2HYiOKBCRFqhCRbr+Hl\nWlVRKU9+5ytyvc1azo59P+X+vmbTJV9IzqsKCPoBdJXL7EO3p3W1238a4rU99phxqavGyj7WSwoN\nZF7IBOcOPMcVtPhjJIzjyGyaAI/LasiASpbxrs64eIg8NeVq1Is7bsPto3jHgToZzsw/PdSY7EjR\nZDkSM0wgIqkzYTdcUxO6772HwsnnWXFYwdaO3I7OWknCjw6kXJryVh/j7f0ggyZTpRJwtgHQJUjP\nOk8KECuIadO6Ju238qtdrnBz5fdTot2W4fb+Lc9+nkZUDJN5VjBDkn3AJp2bImOI119m8QbYBRAR\nRLkqqW2+mnuYjdBqLK649z0bbxPloHpuZhzMhiozizZKwDmYontqEjoqrfSNRQEId6gqKuuq3WiS\naTeM5DvetrUXqe2az9PoecDkRlTEFjJPuPtsks3GTwFp9HFUdh7NgKIou5F2+nXSl6wtPiicHJut\nizmM1th/T9jBgMlIkSoTMvIvR8wNyymLlALaHZstyR02JuEXNqiQEvp8VWresJwsdGZ4OR2aTs1f\n/FV+XT49uhM50w67xuUoSHGEFB3i2jdjQjFLFvE1+5as8L9R0/Oq3xPMGLkkeWxBxrQSifkLko+y\nRIEFVFJVTUWhaRUT2fs1eNpt+RPyatVqpl71r+0GZhMjA5HEiuZB6bGnRpBuA+jSbHWCa2kHTBuy\nKjqoqfYqOHWYiDVd1OVJ2dlZPWOkdl4lcxmZOSiSJwy50d1XHhgsx4brjIi0ZAG9UZNHFPbcvVpe\nyWtetuiTjH6nCvM7p2my1iKuMfDJtGNkSJOOiyJLSx5DzLbjzCoqK2ZCikCouvpVbVwsoZ7+OztV\nNqG0al9IZR/AQckGUeayc6Syy7EbRlUFXXkB5gANs7EwCkt1uvpuWld9qVmowj5/uXfHW257rNKM\nd8rTp9sly02TfLm2yMnSDGbVcO24lR5E3FtQUuvuSsP+HxPSlHN/w+5HxK5bM4ZvLt5ByNImAr8N\nkRbVlsCurQGKipHcbb13X8du2raKuIMcW/kpv3Q3ldvD9yK9PezTXE5zJrDdmGbhEKCZApQ3VNB3\nXG/iiKqL9iqq7dy/rU5vkfKuKyxu/wDFkiTkpeAyDrUmS5OgnAkzmurs6oHD2K4CEAjcTF1LXTRU\nqKqssYcx+pu3I+KzTe6u128fT/1IqZTKJj485p+dJyRK0bsNIbwxiEyHqNDdlNoiKrtPffS6qqaV\nrapjEeZz92+1WTs7YxtceWnzk3CvOfSKMsLmkzkjKtZCMJPMBGFoorhILbbhuDdUkDcvnVuv7Fdd\n9dsR8/2PL7N/cd1ZZUfx8/8AV4kqBhAZlPTpbyzJ8gEaceMUERaRb9JttNBC63XVVXzVdKza8qFo\nb4+GG7NzZ/TsiJj8HmcfHDHxckCYxn0R0NhTkttJ8lsXVc6a7U0FSbXTxvWrXTy1k58fBei2q3pX\nhmO0zHyMw4jJJn3Mr8Yz0zYSMkf4c9yAJEYr1Ot43PX01Ny2xBv2re5vlaRp+56wWJm474z4mU3J\nSVIOSnTZJraTi3JPU47dPZUvZOIHBxWpMuZc6R92MFgGMSxIYFxXgedMm0JE+bZVV6bCfuG0VUSl\n77hwfjrjTWsv5dvgR8fg8xj2Ax8TJNji2vTHQ2FOS015Ni6rmxdqaCpNrp43rVrp5ayY4+C9FtVv\nT5ZXhMx8jOWJyC8iHKpLaRgWFjfDKwSlsUkNV6nVtuun4FTctsQb9q3ub5URER95+x+ycTPdz8TK\nBKaBmK06wkdWSIiB8miP5zqil7spt9GnvqKy2wLcVnyK8qEmojvHWfDsecfiMnEm5OUcxlwsgQuC\nKRzFG3AaBob/ADxbh2tpdNNfNKtrJpKNCcfFetrOV6vDwjuU0/GzcbhImJcmtOOzcinRfVhQZRXH\nTlkL4K6e8CUVFEQkvonvrpWydm40X7Hm5OO1KKja9V+2NXbOSZFcy+KyUDHu/AORpxuCrUKOcY29\njROdVRV14SC4IK+GpJWXFk3nB1o78dq1e2LdlHTXVlnmcQuQCObL6xZsN3rxJKCh7T2qBIQKqbgI\nCUSS6fZvWKWjyO/Nw74hxZOUz8ai59W3VfnR1fIdrQtxiRkdUVSIVdUyW2iWNET2LRuvYiryRmyn\nyx9fuQYfF5DGGxkIpglMw+xIEwGVBEEGujtcbVw96GCqhWJPdZUvWnyS241OVPxWqVrOaaOPCM5J\njOJkuZJrIZF8HnYwGERpltW2wVy2813E4pEqDZNUsl9POsuyiEdVxN2VrOY0IKcbygNToLOSBrFz\nHH3kBGLyAWSRG4Auq5s27zVf4u9ltfzrXuLDjJy/9a6Tqrelz0znxn7GVOPTQiYxW5jY5PFNqy1J\nRlekbRCgk2bSuKViQBXQ/lIi+6nuKXjDL/69kqw/VXrGP0n7nuLhsu3nFyr89l1XGBjvRxjkIoIG\nRp0y6qqOp67t32qjutsQWnDdcm92WkafTP8Acl5vGHkYPQadRh4HmJDLqjvFHI7ovDuG43FVCy6p\nWaWhnXn4t9YThyn+jkhv4bMlkI+TZmxwnAwcZ9DjmbSgRoaKA9YSEktZfUqF7q0rqIjBytw33Kya\n3RGmPqYQ41k24GUgt5Fsm8kThq69HUnBJ8EF2+xxoV9o2FLe+r7ilONDK/Gsq2qrfy8O+vVGRzj0\n0o+OMJoN5TGCoMSxZXpG2QoJtuMq4qqJIIqtjRboipTes4wyv8e0Vz669Y+qn7ktIeaOJIB+aysl\n4UBtRjkjIJ5rsV3eRKi/ylvDT25ms6HTZdpy1L8MfX7kB3jWRPi8fBJPZRWAaaWSscl3Axt2ejrJ\nYvRqu77VaXIt26Dk/wAaz4lx7liMx2+JLm4nIuzoOQYlMtzYrL0d1TZI2jF9WyJUBHBIVQmUVPUt\nRWUNdDpfis7Kya3JNad48fAnZCNIkQzZjyFjPrtUH0TdZRJC1G43RbWVL+FYq4Z15Kt1hOGV/wBB\nOyp6TMo40+ox3YgtMtk2KtyFFXN6kZqV+mlk0tr41vfChHH2Ha268PDWnfUwnx7Iu4z6HenieLUE\nYNekqSTYRNvTJzftuo+kjQNU9i61fcUzGTL/AB7Omx29OmmY7TPzgzuYaeufj5NqUy3HjsHFGL0C\nUum6TZl84jqJuuym30WT2LU3rbBp8NvcV01CURHl4+HYxN4PKx5kxYmQBmBOeWQ42rO58DIRE0ad\nU9iIW2/qbW1N6aUrKIuC6s9torZzpn4OfsYmeLym+Mw8OswCfx5xziyUZVB/3UxMENtXCUr7LFYk\n+1VfItzcamV+K1xKk5rEOO3xJEzEZiVLxkopscDx7hPEKRjVDI23Glt8/wClNjvv1S/uqKySajU3\nfhvZ1cr0+Hmu/ie0xM6LLlP4yS003NPrPMPtE6IvbUBTBRNpU3IKbhXz101qbk1noX2rVbdXG7uu\nv6oiy+LPHhUxsWYLZuSBlypTzSuk48j4yFKwuNINzG1vIdErS5My0c7/AIr2bU+stxMuZ7ok5HEZ\nOXPxksJjLf0eSuECxyLqGbRtHZesO0drmia2XzWpWySajU3ycNrWq5Xp8PCO5hn4HJToUvGyJzbm\nPmE7vU2SJ8WnTUlbE1c2+lCsBbdNNNKtbpNOMmeTgtarq36XPTOfj+hZ5bHhkcXLx5mTYS2XGCcD\n5Qo4Kiqp9i9YraGmd+Xj30de6gqHcBnXygPP5Jj4jGuo5GFuKoMrdo2SVwOqpKuxxbbTRE9i1tXq\npxqed/j8j2t2U1eMY0jv49zP9C5OLOlSMVNaYZmn1pEaQwTwi8qIJONqDjKju2puFb666VN6aytD\nXs3rZujSVujU58MoZDCZSS/jHW8g2hY5xX1V6OrhOOE04yqrsdZQR2urZET7dK3SnGo5OG9nVq38\nc6a4a7ruSchinHpsfIxHkjzo4G0hGHUbNpxRUgMUIF+UCKKoui+66VK2xD0OnJxN2Vk4svoR1wUt\nw5kt6btycllI7MhkFAGGxVSRABSJVuRXJVLX3Vd6wowY9huW36monsfp4WdKmtTJ0pvqxmnm4yxm\niaVFfFBI1IjcXRE0FLa+2m9JQg+G1rK1nonELv8AE8phci+7ALIy2nwxziPNq2yQOG4IE2hEROHZ\nPWqqiJr7baU3pTC1Hs2bruae3w/cm5rG/SeKkwep0VfCwu23bSRUUV23S9lTwvWaWhydObj30ddJ\nIORw+bnsxEcnRm3YsgJKqMZxRJW1uI2V9FRPbr9ytVvVTj+v0OXJw3ullYc6f/Iyy8TkH85CyQy2\ngahg438OrBERI9s3+vqoiL836fTp76KyVWoNX4rPkVpWPDvHj4GGPg8tAN5rGT2mse86bwsPsK6b\nJOkpuI0YuNptUiVUQhW32NKrunqsma8F6SqWW191MT2yi4IuhGUjUnekFyKyKZbU1WwoiXX3JXPV\nnomEaXi2sxEwAZyPJxZAsdZKPPxj65Ad3NrskHkRS1sRbPHyr0Wh225/rwPmcSvXj9xOmk5Wfjaf\nsXkPH5SRlo+ccfbZB2KLRQCYLqAJqjiirnVtvQtL7PtVzdkltPXTju7rkmPTpHx1n7H6zgslEhnj\noE8GICqSM3ZU32QNVVQbPeg+m/oUgW3nuqO6blrIXBatdtbRXyyvn+mP1PEjjT6HiRx0sIcXDiqR\nmVZV1VXpEym8lcG47S1S1/3SVVyaz1Jb8Z+na9qppjwjv/XczM4J16W9Lyzzct1yOUQGmm1aaBlx\nUV1EQjcJScUR3LfySye2O+IRpcDbdrucR4R1/U84/EZyG0zCTJg5AY2g0RMXldMfkgTquK2q2S27\np6/Z1pa1XmMk4+HkqlXd6V4Z/WY+RdVzPUKAUAoBQCgFAKAUAoBQGORGjyWSYktA8yaWNpwUMVT3\nit0WqnBm1VZQ1KMMPFYyCpLChsRVOyGrLYN3RPC+1EquzerM04qV/ikvIlVk6CgFAKAUAoBQCgFA\nKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgII4LBjJ+JHHRkk7t/XRltD3\nfhbrXv761vtpJy9jjmdqnyJ1ZOooBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCg\nFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQC\ngFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQCgFAKAUAoBQ\nCgFAKAUAoBQCgFAKA//Z\n--_004_AS8PR04MB8071BB5854E10B6EA7161159EF0D9AS8PR04MB8071eurp_--\n\n<!-- NEXT TEST -->\nenvelope_from noreply@grupokonecta.net\nenvelope_to joe@domain.org\nhelo_domain rheology.yeloweditions.com\niprev.result fail\ndkim.result pass\ndkim.domains rheology.yeloweditions.com\nspf.result softfail\nspf_ehlo.result softfail\ndmarc.result fail\ndmarc.policy reject\nremote_ip 51.89.165.39\ntls.version TLS1_2\nexpect_header X-Spam-Result: DKIM_ALLOW (-0.20), HAS_LIST_UNSUB (-0.01), ARC_NA (0.00), DKIM_SIGNED (0.00), FROM_EQ_ENV_FROM (0.00), FROM_HAS_DN (0.00), HAS_EXTERNAL_IMG (0.00), HAS_LINK_TO_LARGE_IMG (0.00), HAS_REPLYTO (0.00), HTML_SHORT_1 (0.00), MID_RHS_MATCH_ENV_FROM (0.00), RCPT_COUNT_ONE (0.00), REPLYTO_ADDR_EQ_FROM (0.00), REPLYTO_EQ_FROM (0.00), SPF_SOFTFAIL (0.00), TO_DN_NONE (0.00), TO_MATCH_ENVRCPT_ALL (0.00), RCVD_COUNT_ZERO (0.10), RCVD_NO_TLS_LAST (0.10), HELO_NORES_A_OR_MX (0.30), DATE_IN_PAST (1.00), MID_RHS_MATCH_FROM (1.00), PARTS_DIFFER (1.00), FROMHOST_NORES_A_OR_MX (1.50), HTML_SHORT_LINK_IMG_1 (2.00), RDNS_NONE (2.00), VIOLATED_DIRECT_SPF (3.50), DMARC_POLICY_REJECT (4.00)\nexpect_header X-Spam-Score: spam, score=16.29\n\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=sectionalism; d=grupokonecta.net;\n h=To:Subject:Message-ID:Date:From:Reply-To:MIME-Version:List-Unsubscribe:\n Content-Type:Content-Transfer-Encoding; i=noreply@grupokonecta.net;\n bh=dP5ixF9HZsWlj/p3qHOLHJE+ZBdEajVqh0aowOaVxc4=;\n b=nxR6ACs3hInMKEyEK9yQIEgu4RuffMNmnA6a2mC6Z1c9Qbao8AsmuWct1i7bvn5lU1mmDSXx6PwO\n   /w2Z6TSZIW1jZkhQwDo5sJWO/0f26zabNJ3bAsZnN/Yyy0Lf4oUJuaZ97cwHiMPlDIPF++gyuY6s\n   30mpjo29dX3ZCSCkMkcmox55AsQSK2nJ2ZIv8dqep02kiXjldaiW5hIRzHwjeh+REy3Mb5zCuIVJ\n   4dcKomEGK3JsnnY1mdMVHB1ghlVMnms+HSu2AYi7186j0QSdobMxNuBzAKqzDRfunSljG1IL1im8\n   bnZoghgIpHHfEKY5+b5hStqgBQoPTh2CJMAGWNDkdvoFK3m606RRRiKIOP69M4v5a1UjBBoyP7QO\n   AUNbgcjHWQCl83a1ofW2LKSDPcyfj2TVK4yjQlPAH3Q0TJgeig1beio7Ahnz581H22kkyxPtMB56\n   YVA2cG7FvBB40HLmuX1YKu4TAwJ8ZJdCOnt2xZ9lYLb8LuduCFrd6RZH/44lUu/DpnM1lN3Q3oXe\n   moOQCtyUsI77tu7rzy0vgzZME4v0j3S57CYHJktZGvk/3uFHTWhStYlWNk+Ks/rCfkQjSmQV1u+T\n   ozHF7td0rxpD0b8534vjwABr/NwpjlSLIgBvt9d0nBtipYYOumSdXpCj9mB6uZbGVq9K0mdusbY=\nDKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=sectionalism;\n d=rheology.yeloweditions.com;\n h=To:Subject:Message-ID:Date:From:Reply-To:MIME-Version:List-Unsubscribe:\n Content-Type:Content-Transfer-Encoding;\n bh=BwKu50FuSdWv9iToeKQHRPlPqN0=;\n b=UT4zz+mBbtBTs9S/YEiEH7B3N9gklIJBZLKAyaCOvddh0wWb33fT3qyqCKo9Vfyhp+gBHlQ/pfvZ\n   xLRSO8SjfAOLs/0Xs0sy5uVLuNXh5UfdT+IwG5KSgA+NpRvOTmHvfUzFP5DoiykHP0eutX2TxkRh\n   dJHjLIthv+EISFEubGu1ZgFUBReFTCvhZCrYPYJlxze/tmWWChRK7r804EBR3gJiLEvgXQho5Qh5\n   ksdnvvqAAVTHNOWAFbxoojBmNCs/gn0BbXHeJGuR/ZRXUbC1ZzQ4hw/9xNSCHI9n24IoD+vfhldb\n   k2hSVGQisRxn9vTISgDaNjEgFdr+0hUTqSTvrVEDdJYjCmYK/HrJrb8oeh+G/mz30pYQJlJUYIuJ\n   VcGkqXdFQdenNJPPD1UccGCpWblaaSlqY2AVk3mjX08Du2jbuzEFnX2g5UA5okGOmUi3h5C1v3vG\n   zLMuOT3686ZovcbxlW+K1GkOL1IZTkgXiw2X4EDZ6nE191uyVpIi3NTBkRIIWsni2lpDfg9+qq/J\n   /viVYHHmyG7xpB+gmS9r46HZqZYcYA9g5NQuTaYPIzngwlODdoSVsDb9X7/aU2Mhvp+ayq5g5/fa\n   4cbe7mAJMaCifxx5YA2KqkYLqRS3Cm0D2AJGqk9DpkpncHL+g8+jOAtoWCPFa5L+yVnQIexiJqE=\nDomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=sectionalism; d=grupokonecta.net;\n b=k7FzASi0Vd8+owLAAzg7gG2fHkmp7q+kS7fNXMORbpbqrny/kgSjEkfLUDJdmPFyY2PlDQ+/dZcs\n   W6kMg8n/+wKKEMenIz1O0RSCbKoV5N2gchXV1nEevqyZ+Ndf4XwLhHY+Ttlh6dnlMNblWnXZcxjs\n   2bvwWSAVakpbSIbRkYeMHJOfQTw7efqEEhKRqXCWHnHJuC9zkf4C3bUe6WcTjBmlCe12dU+HD18K\n   lkc7TJKwIm81vBgE+HXpVC3YuJsfsNLM4bb/bZmt3yu5SgfEWss6DgOsOa2yNzLWMx6byVhx3zvg\n   u9k+c+seOdotD0Rphz6F2GaW3LJwOz+Uuhntwir3FZ45GEeyn2V0jiXwVpg8L2RREs3bqCUm+THD\n   8p0gSDNbB3SExCuvRmRnvPCogYvO9XGbwX6eh4gl9dGMrmVSxpjnIoJjoAF/AZSPV2FGBhwFWzjr\n   P+lD8QOzpAbKXSArKcDmJU6ehafQW1fknB1colA97NVan2HCj6B1YdaKv7BtPPjh+L1hwVd3mFaD\n   Kp4/hGJnW/UE12ZUaLIFVsioZWf6TcNvFaZQzTlL8U5fShX8iQInbAWh9+c0YGGLzcGcjqUV3BCA\n   ZXLFzE9yrfkpMDPInBqLOXbIiWzIUopFoCPHuEyEDpalYQQx3rXbUKsNnGs8xEipZueJWmdhaUc=;\nTo: joe@domain.org\nSubject: =?UTF-8?B?UGVkw60gb25saW5lIHR1IFRhcmpldGEgQkJWQSAxMDAlIEJPTklGSUNBREEu?=\nMessage-ID: <61805635a4a8f8915503cb518cbbaeecc82ce04e861805635@grupokonecta.net>\nReturn-Path: noreply@grupokonecta.net\nDate: Tue, 10 Oct 2023 11:52:03 +0000\nFrom: \"BBVA\" <noreply@grupokonecta.net>\nReply-To: noreply@grupokonecta.net\nMIME-Version: 1.0\nList-Unsubscribe: <https://notifications.google.com/grVIU61805635/5SRhf6e497d2459dceb342666f9b49a11b7b_WbfS366/FtbW17880.html>,<mailto:abuse@net?subject=NjE4MDU2MzU=>\nContent-Type: multipart/alternative; charset=\"UTF-8\"; boundary=\"b1_3d217f30a568faa9ce3dd7dc73399561\"\nContent-Transfer-Encoding: quoted-printable\n\n--b1_3d217f30a568faa9ce3dd7dc73399561\nContent-Type: text/plain; format=flowed; charset=\"UTF-8\"\nContent-Transfer-Encoding: quoted-printable\n\nTarjeta de cr=C3=A9dito BBVA\n\nLa tarjeta de cr=C3=A9dito para viajar con tus consumos\nPedila 100% online y empez=C3=A1 a disfrutar\n\nConocer oferta\nUn mundo de beneficios con las tarjetas de cr=C3=A9dito BBVA\ncompras en cuotas\nCompras en cuotas\nPod=C3=A9s disfrutar hoy de los productos =E2=80=A8que quer=C3=A9s y pagarl=\nos en cuotas\ndescuentos y reintegros\nDescuentos y reintegros\nEntretenimiento, gastronom=C3=ADa, farmacia, ropa =E2=80=A8y m=C3=A1s rubro=\ns con promociones exclusivas\npuntos bbva\nViajes con Puntos BBVA\nVuelos, alojamientos y mucho m=C3=A1s canjeando Puntos BBVA que sum=C3=\n=A1s con tus compras\nConocer oferta\nDescubr=C3=AD la tarjeta que mejor se adapta a vos\nTodas las tarjetas\n\nBlack\n\nPlatinum\n\nGold\n\nInternacional\n\nTodas las tarjetas\nvisa black\nTarjeta Visa Signature\nL=C3=ADmites desde $600.000\n\n15% extra en acumulaci=C3=B3n de Puntos BBVA\nAcceso a salas VIP en aeropuertos\nAsistencia en viajes con cobertura de hasta 250.000 USD\nExtracci=C3=B3n de efectivo en el exterior\nSeguro de robo en cajero y compra protegida\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $200.000\n\n Conocer m=C3=A1s\nmastercard black\nTarjeta Mastercard Black\nL=C3=ADmites desde $600.000\n\n15% extra en acumulaci=C3=B3n de Puntos BBVA\nAcceso a salas VIP en aeropuertos\nAsistencia en viajes con cobertura de hasta 250.000 USD\nExtracci=C3=B3n de efectivo en el exterior\nSeguro de robo en cajero y compra protegida\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $200.000\n Conocer m=C3=A1s\ntarjeta platinum visa\nTarjeta Visa Platinum\nL=C3=ADmites desde $350.000\n\n5% extra en acumulaci=C3=B3n de Puntos BBVA\nAsistencia en viajes con cobertura de hasta 170.000 USD\nExtracci=C3=B3n de efectivo en el exterior\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $120.000\n\n Conocer m=C3=A1s\ntarjeta platinum mastercard\nTarjeta Mastercard Platinum\nL=C3=ADmites desde $350.000\n\n5% extra en acumulaci=C3=B3n de Puntos BBVA\nAsistencia en viajes con cobertura de hasta 50.000 USD y 30.000 EUR\nExtracci=C3=B3n de efectivo en el exterior\nAtenci=C3=B3n personalizada para resolver tus consultas\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $120.000\n\n Conocer m=C3=A1s\ntarjeta gold visa\nTarjeta Visa Gold\nL=C3=ADmites desde $100.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $20.000\n\n Conocer m=C3=A1s\ntarjeta gold mastercard\nTarjeta Mastercard Gold\nL=C3=ADmites desde $100.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $35.000\n\n Conocer m=C3=A1s\n\nTarjeta Visa Internacional\nL=C3=ADmites desde $10.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $20.000\n\n Conocer m=C3=A1s\n\nTarjeta Mastercard Internacional\nL=C3=ADmites desde $10.000\n\nPuntos BBVA para viajar\nExtracci=C3=B3n de efectivo en el exterior\nTarjetas adicionales sin costo\nEs necesario un ingreso m=C3=ADnimo mensual de $20.000\n\n Conocer m=C3=A1s\n\n--b1_3d217f30a568faa9ce3dd7dc73399561\nContent-Type: text/html; charset=\"UTF-8\"\nContent-Transfer-Encoding: quoted-printable\n\n<head>\n</head>\n\n<body>\n<div style=3D\"max-width: 600px; margin: 0 auto;\">\n<table border=3D\"0\" cellspacing=3D\"0\" cellpadding=3D\"0\">\n<tbody>\n<tr><th><a href=3D\"http://track.leadsinbx.com/aff_c?offer_id=3D2906&amp;aff=\n_id=3D1980&amp;url_id=3D21948\"> <img src=3D\"https://i.imgur.com/1wg1VQH.jpg=\n\" width=3D\"100%\" alt=3D\"CLICK AQU&Iacute; CLICK AQU&Iacute; CLICK AQU&Iacut=\ne; CLICK AQU&Iacute; CLICK AQU&Iacute; CLICK AQU&Iacute; CLICK AQU&Iacute;\"=\n title=3D\"Es la oportunidad que estabas esperando\" /> </a></th></tr>\n</tbody>\n</table>\n</div>\n</body>\n</html>\n\n--b1_3d217f30a568faa9ce3dd7dc73399561--\n\n<!-- NEXT TEST -->\nenvelope_from miah.join@outlook.com\nenvelope_to hello@stalw.art\nhelo_domain HK2PR02CU002.outbound.protection.outlook.com\niprev.result pass\ndkim.result pass\ndkim.domains outlook.com\nspf.result pass\nspf_ehlo.result pass\ndmarc.result pass\ndmarc.policy reject\nremote_ip 52.103.64.5\ntls.version TLS1_2\nexpect_header X-Spam-Result: DMARC_POLICY_ALLOW (-0.50), DKIM_ALLOW (-0.20), SPF_ALLOW (-0.20), ARC_NA (0.00), ARC_SIGNED (0.00), DKIM_SIGNED (0.00), FREEMAIL_FROM (0.00), FROM_EQ_ENV_FROM (0.00), FROM_HAS_DN (0.00), HAS_SEO_WORD (0.00), HAS_X_PRIO_ONE (0.00), HTML_SHORT_1 (0.00), MID_RHS_MATCH_ENV_FROMTLD (0.00), MID_RHS_MATCH_FROMTLD (0.00), RCPT_COUNT_ONE (0.00), RCPT_IN_BODY (0.00), RCVD_COUNT_TWO (0.00), TO_DN_EQ_ADDR_ALL (0.00), TO_MATCH_ENVRCPT_ALL (0.00), RCVD_NO_TLS_LAST (0.10), HELO_NORES_A_OR_MX (0.30), DATE_IN_PAST (1.00), FROMHOST_NORES_A_OR_MX (1.50), SEO_SPAM (5.00)\nexpect_header X-Spam-Score: spam, score=7.00\n\nReturn-Path: <miah.join@outlook.com>\nARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;\n b=sTW55J00fLHM5CSFAdYk6Kpyecib7sSXQWU51a+Eo6514pesoEtpNxM3eYurQfYQY7j+MMcwJ50u9fzJPOUm0JInaQMoDrUWJ5dObEglZtxbN1fpwHLOOP5rjWm+zd9p02jLCCpvoHnu4rIZmog1MO/pCiVRMWemUMzJ2O7mk2zbmode8ryb9tT1ho8XNeCYK9zKmoHwCl2p6TjO4HFQ4SU2hYIWd3//6gfnPDN2qIOgw6Z51zgsEtUYYENIKuHswZFWjt7925Wq380r5Fi+fsaKT8xAWFTq9igFNWKDVU2k7ZL6QlCsXpTRS57rrl1dBYAod1byHHbCOqa+g+VOAA==\nARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector10001;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=SVZW4LFWvuTy4mbM+Q+E+yH3a0DZrHFN3U3wHGbAHMQ=;\n b=NavrW7s0URdfDuEFuzkxV7EUwJtiynvH1o9mzF39USQEfd9l1KyQpzOxo9Po8Dar1qqa/4ECNeUSmetx+NBDArmlTpBak+BKYXAXVRlHheyxILyU/f0RX01+7aifIzLj7LWv7Sx66b9D9/DjaVDbtMvFGPFUzk0JtiATahe7ZU0iKBvsRbGJjS9r0Sq2vHY/SQEUxOxKXUhUQBepSf9k7ibBZK27OhSz9v/jjSDCL/mh5MoOgbq7S8lbxUGS356c/Rm3ZWEInIRcbVqI+P75abEvzRNhRyBDId74h9IZnv+wz9QfGnk8TaFAExRBJ5BzIKlDibTZ+Kzuc+7mvOAKLw==\nARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none;\n dkim=none; arc=none\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com;\n s=selector1;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=SVZW4LFWvuTy4mbM+Q+E+yH3a0DZrHFN3U3wHGbAHMQ=;\n b=Q6QpEIbyyvkSgmfTsPeVdPuTyh+6lA/+qAoEm5k5gEDuyqmLjwVELDsOJQAZzwfQfmxN02O5dpbD0mDWKLFR7Ft//121jF9EV06fbGMNXuuBpfZJ24npu+bPbHs66D7USSMEE6zvuf4bnlhVV0iTTxWwhNEawPfaFpuukvlVO1GtPOjH0SeymOnfHM3LrGSkwYpw5aeEGjrJLFQRSN+k8mD7PyoOkJFFBUyqySWdkRsQ5aw9+7f3wbHbDOb4rqkmkC6fUZSMcqpTSpFFS3fDlQQrcnwhh8ir/tq74AuVYyMoUMns81tExoILI78twEHxyGN2zgLw7K9QojJCf+IIEQ==\nReceived: from SEYPR04MB7496.apcprd04.prod.outlook.com (2603:1096:101:1db::7)\n by PUZPR04MB6246.apcprd04.prod.outlook.com (2603:1096:301:ec::5) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8293.20; Tue, 31 Dec\n 2024 08:24:54 +0000\nReceived: from SEYPR04MB7496.apcprd04.prod.outlook.com\n ([fe80::1aab:cf90:44d8:af4e]) by SEYPR04MB7496.apcprd04.prod.outlook.com\n ([fe80::1aab:cf90:44d8:af4e%6]) with mapi id 15.20.8293.000; Tue, 31 Dec 2024\n 08:24:54 +0000\nFrom: Miah Join <miah.join@outlook.com>\nTo: \"hello@stalw.art\" <hello@stalw.art>\nSubject: SEO-Inquiry\nThread-Topic: SEO-Inquiry\nThread-Index: AdtbXSTUEOZ2HAOVQCqvPiaCPhJ+xw==\nImportance: high\nX-Priority: 1\nSensitivity: private\nDate: Tue, 31 Dec 2024 08:24:22 +0000\nMessage-ID:\n <SEYPR04MB74966F6A34B2B0AC5DA8E6DDFD0A2@SEYPR04MB7496.apcprd04.prod.outlook.com>\nAccept-Language: en-US\nContent-Language: en-US\nX-MS-Has-Attach:\nX-MS-TNEF-Correlator:\nx-ms-publictraffictype: Email\nx-ms-traffictypediagnostic: SEYPR04MB7496:EE_|PUZPR04MB6246:EE_\nx-ms-office365-filtering-correlation-id: d8480a0c-4e45-4278-ad1b-08dd29749a89\nx-ms-exchange-slblob-mailprops:\n SoURN12IA8vYLoDX/njsxmIBKrA5xww7SXj4BirxWi/11KcMwu1z/THqGhGNTFhVj0W89DLwoSD+IqZucSTc96MoKe9ia4xsvj73oWgcR/6suK5u7GoMSUHmwhrgVtaAkaceSYHenT+iTqqsr6L31R3bTGzhR1r7TEKkTLaJO6pXyfeXy8wUEdMEUOpfggfX/BBwA6KLTGvIE9euGA810m3+uPbUX+cE3WNT2yFsiC+H/YFGbcISQcbwlqT/7fAjiNT6fore4X1HnccC1CzArvmHrHF37pjc/JTEQFUxg7woH0GKJtvsUFbDUPvXjKp9rTRqLHTb8fYVSRdeivWB4lJocoW5VqJ9YichWUGp1ELEDuto7T/ourT4i6Q/Z+D+3/Q67J0SAdD8+XAXHfYKG2rq14F+tz4KxbO6EPp7O6HpaHMSbpzps5/sOJ2gmeHucnICmy81mKE9aq2sfJPPdymPBczv4hB7QO+xQDHFNpypNedZ8cTbtW5MlVn18lgmP0BuDBEgU4WcbKu2bMzgl6HJRQctSSldfkkyfGI/URoo69SEmOZiDuuCzhKQQ09ho6xRG3kpsD9DyRAUQyyo3Wq4J1UgLuRyRVFYwPI4zL+e47La1ilj199UD/SJrbHMMTt6itlRBpnH4N7Ev1ELztRfH5VjQ5aG\nx-microsoft-antispam:\n BCL:0;ARA:14566002|8060799006|7092599003|8062599003|5062599005|15080799006|461199028|19110799003|440099028|3412199025|102099032;\nx-microsoft-antispam-message-info:\n =?us-ascii?Q?sSaF+W5EA6WGixAhyYLVcDu/rxF60TPo55Pe5lDbuGq4SQKPFczK5OPYJQHf?=\n =?us-ascii?Q?6w0tFaoHE+b0IsosJe0H/dJXTmVvChLHdVnaTZiPcDKrZVOcLyvm2WnEoemj?=\n =?us-ascii?Q?m21GgioEUF/NQdwaXt6AAYv18zt1GxKufXRYIpLkmKM8gV1XQMXPJteH53ax?=\n =?us-ascii?Q?vifrj3RXruNdM31oe0LrZdIbsbKuvKDSdByf1mB2PVfCgEXIjvfFLgJidnah?=\n =?us-ascii?Q?d30fWN6LcYhQvtVNfpiJwtroUR5oox/HxRtDbU+fkvn0LLoFX7nxdECueURA?=\n =?us-ascii?Q?/67CsMYT5/NKrE+oaYmPjDYkLmS+YDBmWdUyuQDJIQ0Lb7vKisWAOFrweDgc?=\n =?us-ascii?Q?pJUyUtAA5UliC4AO+gI+AH3cv2JaMn+Z7gCf9WpcTCk8PkOEtg4Y1661t4T3?=\n =?us-ascii?Q?rOHV1Q7ibkz6JYCiIVGl1macRWWcIl1GeUWLUiHduVR/ax3PLUYLHYblCHv0?=\n =?us-ascii?Q?XVrSCAAa8+vce6+c7ymVd8uBAvfTT/hfYKwSGR2R6V9clhEDjE5TXsIjx1qc?=\n =?us-ascii?Q?AsCWjvexOp/pWalj3jwJzOE/xKHehhzyIeOJ0kN8snZBJiZpD9VnhYE6Lt8I?=\n =?us-ascii?Q?4ttzIth0+B9mw24tVTPg2cZ3N9yFsznBvKxtuFYl3Oj3YzacJVbwk+XyRsjQ?=\n =?us-ascii?Q?LXmGP3d8NPm2Nhy0yfrt4/wxf2th/e+/I3siWDuHcwJiDFdhvoT/AOuUoFl9?=\n =?us-ascii?Q?5Vs6Amo3CmVo4TGEf+EQNPoLsXjVAMzpk/5TgnKQ2sanXVv1zPGrTT7aErrv?=\n =?us-ascii?Q?m2fE/VhGQOqKiizPYaMaC6W4YPdrAYOOrpZFOgzFfMoNmyDmIfIucke8DJTg?=\n =?us-ascii?Q?K630mHxneVrofGWREkohZiDUe/tQsjp/0Hy2x2792Pg2aumt5nc1Aw/QJrwr?=\n =?us-ascii?Q?VHaKD7O2hL1QLJFbojSSKlWyGQ/CE0DsyUVihC0dWqfdWa8gPlHZ3uzzMN5J?=\n =?us-ascii?Q?BazCEPgYBVnmiQw/ejBmBpm5iKIA0zgNKGva/8tDikzhviHpCe9lijBuNsu3?=\n =?us-ascii?Q?OIfW8bDcJpyKB+g5S7EVo9K+lwYq0i86di3dpDvmSBfB99PFK9kqzaAEfAIU?=\n =?us-ascii?Q?Duh11rVb5jqL5hm7Zhsb9T2kGOiAorAjmrAHQFSyCNO910vF+8s=3D?=\nx-ms-exchange-antispam-messagedata-chunkcount: 1\nx-ms-exchange-antispam-messagedata-0:\n =?us-ascii?Q?9c/kgRHXQSmmVIMMaHuipN3QMSRVkf870g3+luQav+fyRaGeQw2TxAqSMXIQ?=\n =?us-ascii?Q?MqoXTcYcUTW8oy80k0fPln9Hz61fG51WDZ2vzG1wnYvfHer1zkAdp/njN/D2?=\n =?us-ascii?Q?jXEtZZP4w7Fi1Tf3oyRzBLQhMtORyV3RQU/uWIRpjvP/jlu3zfVeIyFtON/q?=\n =?us-ascii?Q?oJqPzYVyxeLv832rNiqEVsAPLOar26932L6xXlandRXwk6WZTN7J76uUF+i0?=\n =?us-ascii?Q?FpKfqvx2IHOT/Qc8dJLf3H/iBywKPBINfmUdTdX83EzX9DYFmCJ0mJHhgmFV?=\n =?us-ascii?Q?2r/JQEN6c+ziSN/sqFBrNvpWaEUGzyB7k4HT+HV9bv/8nx0jVYVdb8Jg/Lxl?=\n =?us-ascii?Q?nt9FxkqJPCdTjYYQwCzM+74raMHz5o3URpFYGfEYsgBBrqZPa6oNgxZBrLU6?=\n =?us-ascii?Q?w/m7VsQI9AbLh/jteycxh6INOvPy6SS3m7+FGFpFjwNAfSQLEyghWO0hyyk3?=\n =?us-ascii?Q?75mJwXHJ53u+J6jMQQPEz6h2SFHN64nrGRbPq6ruSdyEhSj9BdkuEnKb/yAR?=\n =?us-ascii?Q?07pJ7iBfQVTfQ04xXejXErXvrDZGCcVy3/1AOajGLmcW1iTgoRwF3KKTmKct?=\n =?us-ascii?Q?g+/IaxelPpT95uCPhSWREdVLoxuQW6pBltFeXod8ou3YpiOoss2h0EplB4Xa?=\n =?us-ascii?Q?TKPKZSDdPBeCbOwx9GQUhySu57WwarR4Q74+MQwx8zDHdlytk9bLFZrCSCnt?=\n =?us-ascii?Q?OixF+n8R+5U1D+Vub74mpSguv+2efQNt2lwJei3iDd1mC6WRXdGaC4+WdCEt?=\n =?us-ascii?Q?uPRtELZCDsLSUE+097ixYl7uMLCe8nHUFuECfu41T1yX9PPMEmpadpxazQCd?=\n =?us-ascii?Q?USBa0u5BAMvXiZJHfPgf3y255JPI079k+DdhGkhF2cDwLciCHELy5rM52TGT?=\n =?us-ascii?Q?IrCNmfxb/RmOWBVEsvbhV4glTLJlORemnfACjPfh8SzldzzIj2W8zUUrxJQt?=\n =?us-ascii?Q?6bdoxj9FbxsJ3wvYQTDqF9olDDPh/2z1g28uXkknEgvcdU/XvlzE+bz3ACnC?=\n =?us-ascii?Q?yoxmdm3P7/9aIojcU9CeOsnQBDHgBYOxXIK5vNogdrZV9Ew4G2w/gOevLmvA?=\n =?us-ascii?Q?HAbDE1IaI2hhX0zhg2D/QCZZNpvlDgJzESkffQNSOYsIWlHeWBzOJeq7iM0Y?=\n =?us-ascii?Q?pXEltktrRe3doxEsnAYKPirbULWz4u+XJXzAOddWBWiYGNi6ua5wuyv7pyRX?=\n =?us-ascii?Q?CMc1g/sPb+5aGiPpbK4Si7KWv6HpNO3v0ACS0qrmdaaPWSx+wL8tsxO4KBI?=\n =?us-ascii?Q?=3D?=\nContent-Type: multipart/alternative;\n\tboundary=\"_000_SEYPR04MB74966F6A34B2B0AC5DA8E6DDFD0A2SEYPR04MB7496apcp_\"\nMIME-Version: 1.0\nX-OriginatorOrg: outlook.com\nX-MS-Exchange-CrossTenant-AuthAs: Internal\nX-MS-Exchange-CrossTenant-AuthSource: SEYPR04MB7496.apcprd04.prod.outlook.com\nX-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000\nX-MS-Exchange-CrossTenant-Network-Message-Id: d8480a0c-4e45-4278-ad1b-08dd29749a89\nX-MS-Exchange-CrossTenant-originalarrivaltime: 31 Dec 2024 08:24:22.5882\n (UTC)\nX-MS-Exchange-CrossTenant-fromentityheader: Hosted\nX-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa\nX-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000\nX-MS-Exchange-Transport-CrossTenantHeadersStamped: PUZPR04MB6246\n\n--_000_SEYPR04MB74966F6A34B2B0AC5DA8E6DDFD0A2SEYPR04MB7496apcp_\nContent-Type: text/plain; charset=\"us-ascii\"\nContent-Transfer-Encoding: quoted-printable\n\nHello, hello@stalw.art\n\n\n\nI'm just checking with you to see if you're interested in (SEO) search engi=\nne optimization, or if you're interested in Google 1st page for better busi=\nness.\n\n\n\nIf so, I'd love to tell you a little bit more about my abilities and show y=\nou some of my work. I am a very\n\nSkilled SEO expert with various abilities and can (1st Page on Google). any=\nthing.\n\n\n\nI look forward to hearing from you.\n\n\n\nThanks,\n\nMiah\n\n--_000_SEYPR04MB74966F6A34B2B0AC5DA8E6DDFD0A2SEYPR04MB7496apcp_\nContent-Type: text/html; charset=\"us-ascii\"\nContent-Transfer-Encoding: quoted-printable\n\n<html xmlns:v=3D\"urn:schemas-microsoft-com:vml\" xmlns:o=3D\"urn:schemas-micr=\nosoft-com:office:office\" xmlns:w=3D\"urn:schemas-microsoft-com:office:word\" =\nxmlns:m=3D\"http://schemas.microsoft.com/office/2004/12/omml\" xmlns=3D\"http:=\n//www.w3.org/TR/REC-html40\">\n<head>\n<meta http-equiv=3D\"Content-Type\" content=3D\"text/html; charset=3Dus-ascii\"=\n>\n<meta name=3D\"Generator\" content=3D\"Microsoft Word 15 (filtered medium)\">\n<style><!--\n/* Font Definitions */\n@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;}\n@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;}\n/* Style Definitions */\np.MsoNoSpacing, li.MsoNoSpacing, div.MsoNoSpacing\n\t{mso-style-priority:1;\n\tmargin:0in;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ligatures:standardcontextual;}\n.MsoChpDefault\n\t{mso-style-type:export-only;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;}\n.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:10.0pt;\n\tline-height:115%;}\n@page WordSection1\n\t{size:595.3pt 841.9pt;\n\tmargin:1.0in 1.0in 1.0in 1.0in;}\ndiv.WordSection1\n\t{page:WordSection1;}\n--></style><!--[if gte mso 9]><xml>\n<o:shapedefaults v:ext=3D\"edit\" spidmax=3D\"1026\" />\n</xml><![endif]--><!--[if gte mso 9]><xml>\n<o:shapelayout v:ext=3D\"edit\">\n<o:idmap v:ext=3D\"edit\" data=3D\"1\" />\n</o:shapelayout></xml><![endif]-->\n</head>\n<body lang=3D\"EN-US\" style=3D\"word-wrap:break-word\">\n<div class=3D\"WordSection1\">\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\">Hello, hello@stalw.art &nbsp=\n;&nbsp;&nbsp;&nbsp;&nbsp;<o:p></o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\"><o:p>&nbsp;</o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\">I&#8217;m just checking with=\n you to see if you&#8217;re interested in\n</span><b><span lang=3D\"EN-IN\" style=3D\"font-size:12.0pt\">(SEO) search engi=\nne optimization</span></b><span lang=3D\"EN-IN\">, or if you&#8217;re interes=\nted in Google 1st page for better business.&nbsp;\n<o:p></o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\"><o:p>&nbsp;</o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\">If so, I&#8217;d love to tel=\nl you a little bit more about my abilities and show you some of my work. I =\nam a very&nbsp;\n<o:p></o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\">Skilled SEO expert with vari=\nous abilities and can (<b>1st Page on Google</b>). anything.&nbsp;\n<o:p></o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\"><o:p>&nbsp;</o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\">I look forward to hearing fr=\nom you.&nbsp; <o:p>\n</o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\"><o:p>&nbsp;</o:p></span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\">Thanks,</span></p>\n<p class=3D\"MsoNoSpacing\"><span lang=3D\"EN-IN\">Miah</span></p>\n</div>\n</body>\n</html>\n\n--_000_SEYPR04MB74966F6A34B2B0AC5DA8E6DDFD0A2SEYPR04MB7496apcp_--\n\n<!-- NEXT TEST -->\nenvelope_from marketing@landeray.com\nenvelope_to hello@stalw.art\nhelo_domain terminal4.landeray.com\niprev.result pass\ndkim.result pass\ndkim.domains outlook.com\nspf.result pass\nspf_ehlo.result pass\ndmarc.result pass\ndmarc.policy reject\nremote_ip 173.224.123.255\ntls.version TLS1_2\nexpect_header X-Spam-Result: DMARC_POLICY_ALLOW (-0.50), DKIM_ALLOW (-0.20), SPF_ALLOW (-0.20), ARC_NA (0.00), DKIM_SIGNED (0.00), FROM_EQ_ENV_FROM (0.00), FROM_HAS_DN (0.00), HAS_EXTERNAL_IMG (0.00), HAS_REPLYTO (0.00), HAS_X_PRIO_THREE (0.00), HTML_SHORT_1 (0.00), RCPT_COUNT_ONE (0.00), REPLYTO_DN_EQ_FROM_DN (0.00), REPLYTO_DOM_EQ_FROM_DOM (0.00), TO_DN_ALL (0.00), TO_EQ_FROM (0.00), RCVD_COUNT_ZERO (0.10), RCVD_NO_TLS_LAST (0.10), HELO_NORES_A_OR_MX (0.30), MID_RHS_NOT_FQDN (0.50), UNPARSABLE_URL (0.50), DATE_IN_PAST (1.00), FROMHOST_NORES_A_OR_MX (1.50), DIRECT_TO_MX (2.00), FORGED_RECIPIENTS (2.00), SUBJ_ALL_CAPS (3.00)\nexpect_header X-Spam-Score: spam, score=10.10\n\nReturn-Path: <marketing@landeray.com>\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=default; d=landeray.com;\n h=Message-ID:Reply-To:From:To:Subject:Date:MIME-Version:Content-Type;\n i=marketing@landeray.com;\n bh=nlE2QLcR2WpTkMQ4zUwkF7IBIw4eOeAhkKS/HXiMvSs=;\n b=EwWEML/WtTA6nI6CuagI2LRBWAZuRHk6IJiDNA3R77dTpa80gHPkfzF3NBcKcVIiCYcf6i4HCniZ\n   bF4DT8VZrFfLCSykeL2t5FLjYwFcjXjX7qCvyaR0hLfyFRSCDO9RytMw8Q3WoODc/jMXWrdxYpqu\n   phP9D5Q/N8vzSneReGOCV62Me1ss947uc43zYZXvVwuXeyLbU0yPzJtTTEODFXmVp2Ul/M7w963E\n   UxzrNlTd/lEIUXVvxAOL2DKGxY8oXslevI0euO8E8TQms3SlY6LySkDia1h+N+mVpbPPeE3txZhS\n   LJwn2MBX59b+W3aMafAt4Ae/UVOOBlMT7FhhvA==\nMessage-ID: <FA171967DC4F4DC39B03EEF3E42FBDEE@christian>\nReply-To: \"RESERVAS Y CONSULTAS\" <noreply@landeray.com>\nFrom: \"RESERVAS Y CONSULTAS\" <marketing@landeray.com>\nTo: \"Marketing Online\" <marketing@landeray.com>\nSubject: BRISZA ASIA TEMPORADA 2025\nDate: Fri, 3 Jan 2025 14:06:03 +0000\nOrganization: CLIENTE-DISCOTECA\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n\tboundary=\"----=_NextPart_000_0E52_01DB5DE8.9FAFDC60\"\nX-Priority: 3\nX-MSMail-Priority: Normal\nX-Mailer: Microsoft Outlook Express 6.00.2900.5512\nX-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579\n\nThis is a multi-part message in MIME format.\n\n------=_NextPart_000_0E52_01DB5DE8.9FAFDC60\nContent-Type: text/plain;\n\tcharset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\nConsulta v=EDa WhatsApp: https://api.whatsapp.com/send?phone=3D5199504167=\n6&text=3DDeseo realizar una reserva en BRISZA\n\n\nSi no puedes visualizar la imagen haz click aqu=ED https://i.imgur.com/O1=\n2POZ2.png\n\n\n\nSi esta informaci=F3n no es de su inter=E9s. Favor de escribirnos un corr=\neo en blanco a eliminarcorreo2016@gmail.com con el Asunto \"REMOVER\". Grac=\nias.\n\n------=_NextPart_000_0E52_01DB5DE8.9FAFDC60\nContent-Type: text/html;\n\tcharset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\n<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\">\n<HTML><HEAD>\n<META content=3D\"text/html; charset=3Diso-8859-1\" http-equiv=3DContent-Ty=\npe>\n<META name=3DGENERATOR content=3D\"MSHTML 6.00.6000.16788\">\n<STYLE></STYLE>\n</HEAD>\n<BODY bgColor=3D#ffffff>\n<DIV align=3Dleft><FONT face=3DCalibri>Consulta v=EDa WhatsApp: <A=20\nhref=3D\"https://api.whatsapp.com/send?phone=3D51995041676&text=3DDeseo re=\nalizar una reserva en BRISZA\">https://api.whatsapp.com/send?phone=3D51995=\n041676&amp;text=3DDeseo=20\nrealizar una reserva en BRISZA</A></FONT></DIV>\n<DIV align=3Dleft><FONT face=3DCalibri></FONT>&nbsp;</DIV>\n<DIV align=3Dleft><FONT face=3DCalibri></FONT>&nbsp;</DIV>\n<DIV align=3Dcenter><FONT face=3DCalibri>Si no puedes visualizar la image=\nn haz click=20\naqu=ED </FONT><FONT face=3DCalibri><A=20\nhref=3D\"https://i.imgur.com/O12POZ2.png\">https://i.imgur.com/O12POZ2.png<=\n/A></FONT></DIV>\n<DIV align=3Dcenter><FONT size=3D2 face=3DArial></FONT>&nbsp;</DIV>\n<DIV align=3Dcenter><IMG border=3D0 hspace=3D0 alt=3D\"\"=20\nsrc=3D\"https://i.imgur.com/O12POZ2.png\"></DIV>\n<DIV align=3Dcenter>&nbsp;</DIV>\n<DIV align=3Dcenter>\n<DIV align=3Dcenter><SPAN><FONT size=3D3 face=3DCalibri>Si esta informaci=\n=F3n no es de=20\nsu inter=E9s. Favor de escribirnos un correo en blanco a&nbsp;</FONT><A=20\nhref=3D\"mailto:eliminarcorreo2016@gmail.com\"><FONT size=3D3=20\nface=3DCalibri>eliminarcorreo2016@gmail.com</FONT></A></SPAN><SPAN><FONT =\nsize=3D3=20\nface=3DCalibri>&nbsp;con el Asunto \"REMOVER\".=20\nGracias.</FONT></SPAN></DIV></DIV></BODY></HTML>\n\n------=_NextPart_000_0E52_01DB5DE8.9FAFDC60--\n"
  },
  {
    "path": "tests/resources/smtp/antispam/date.test",
    "content": "expect MISSING_DATE\n\nX-Date: Tue, 1 Jul 2003 10:52:37 +0200\n\nTest\n<!-- NEXT TEST -->\nexpect INVALID_DATE\n\nDate: blah blah blah\n\nTest\n<!-- NEXT TEST -->\nexpect DATE_IN_PAST\n\nDate: Tue, 1 Jul 2003 10:52:37 +0200\n\nTest\n<!-- NEXT TEST -->\nexpect DATE_IN_FUTURE\n\nDate: Tue, 1 Jul 2999 10:52:37 +0200\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/dmarc.test",
    "content": "expect DMARC_NA SPF_NA DKIM_NA ARC_NA AUTH_NA\n\nSubject: test\n\nTest\n\n<!-- NEXT TEST -->\nspf.result pass\ndkim.result pass\narc.result pass\ndmarc.result pass\nexpect DKIM_SIGNED ARC_SIGNED DKIM_ALLOW SPF_ALLOW ARC_ALLOW DMARC_POLICY_ALLOW\n\nDKIM-Signature: abc\nARC-Seal: xyz\nSubject: test\n\nTest\n\n<!-- NEXT TEST -->\nspf.result fail\ndkim.result fail\narc.result fail\ndmarc.result fail\ndmarc.policy quarantine\nexpect SPF_FAIL ARC_REJECT DKIM_REJECT DMARC_POLICY_QUARANTINE\n\nSubject: test\n\nTest\n\n<!-- NEXT TEST -->\nspf.result neutral\ndkim.result temperror\narc.result permerror\ndmarc.result fail\ndmarc.policy reject\nexpect DKIM_TEMPFAIL SPF_NEUTRAL ARC_INVALID DMARC_POLICY_REJECT\n\nSubject: test\n\nTest\n\n<!-- NEXT TEST -->\nspf.result softfail\ndkim.result permerror\narc.result temperror\ndmarc.result permerror\nexpect ARC_DNSFAIL DMARC_BAD_POLICY DKIM_PERMFAIL SPF_SOFTFAIL\n\nSubject: test\n\nTest\n<!-- NEXT TEST -->\ndkim.result pass\ndkim.domains spf-dkim-allow.org\nspf.result pass\nexpect DKIM_ALLOW SPF_ALLOW ARC_NA DMARC_NA\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\ndkim.result pass\nspf.result pass\narc.result pass\nexpect DKIM_ALLOW SPF_ALLOW ARC_ALLOW DMARC_NA\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\nspf.result pass\ndkim.result fail\nexpect DKIM_REJECT SPF_ALLOW ARC_NA DMARC_NA\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\nspf.result pass\ndkim.result temperror\nexpect DKIM_TEMPFAIL SPF_ALLOW ARC_NA DMARC_NA\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\ndkim.result pass\ndkim.domains spf-dkim-allow.org\nspf.result fail\nexpect DKIM_ALLOW SPF_FAIL ARC_NA DMARC_NA\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\ndkim.result pass\ndkim.domains spf-dkim-allow.org\nspf.result temperror\nexpect DKIM_ALLOW SPF_DNSFAIL ARC_NA DMARC_NA\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\ndkim.result fail\nspf.result fail\nexpect DKIM_REJECT SPF_FAIL ARC_NA DMARC_NA\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\ndkim.result temperror\nspf.result temperror\nexpect DKIM_TEMPFAIL SPF_DNSFAIL ARC_NA DMARC_NA AUTH_NA_OR_FAIL\n\nFrom: user@spf-dkim-allow.org\nSubject: test\n\nTest\n<!-- NEXT TEST -->\nspf.result pass\ndkim.result pass\narc.result pass\ndmarc.result pass\nenvelope_from hello@stalw.art\nexpect TRUSTED_DOMAIN DMARC_POLICY_ALLOW DKIM_ALLOW SPF_ALLOW ARC_ALLOW\n\nFrom: <hello@stalw.art>\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/from.test",
    "content": "expect MISSING_FROM\n\nX-From: hello@domain.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect MULTIPLE_FROM FROM_EQ_ENV_FROM FROM_NO_DN\n\nFrom: hello@domain.org\nFrom: hello@domain.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from test\nexpect FROM_INVALID ENV_FROM_INVALID\n\nFrom: test\n\nTest\n<!-- NEXT TEST -->\nenvelope_from www-data@domain.org\nexpect FROM_SERVICE_ACCT FROM_HAS_DN FROM_EQ_ENV_FROM\n\nFrom: \"WWW DATA\" <www-data@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect FROM_DN_EQ_ADDR FROM_EQ_ENV_FROM\n\nFrom: \"hello@domain.org\" <hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect SPOOF_DISPLAY_NAME FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"hello@otherdomain.org\" <hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.co.uk\nexpect FROM_NEQ_DISPLAY_NAME FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"hello@other.domain.co.uk\" <hello@domain.co.uk>\n\nTest\n<!-- NEXT TEST -->\nhelo_domain mx.domain.co.uk\nexpect FROMTLD_EQ_ENV_FROMTLD FROM_NEQ_DISPLAY_NAME FROM_HAS_DN FROM_BOUNCE\n\nFrom: \"postmaster@mx.domain.co.uk\" <postmaster@domain.co.uk>\n\nTest\n<!-- NEXT TEST -->\nhelo_domain mx.domain.co.uk\nexpect FROMTLD_EQ_ENV_FROMTLD FROM_HAS_DN FROM_BOUNCE\n\nFrom: \"Mailer Daemon\" <MAILER-DAEMON@domain.co.uk>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from mrspammer@domain.org\nexpect FROM_NAME_HAS_TITLE FROM_NAME_EXCESS_SPACE FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Mr. Money   Maker\" <mrspammer@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello+world@domain.org\nexpect TAGGED_FROM FROM_EQ_ENV_FROM FROM_NO_DN\n\nFrom: hello+world@domain.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect TO_EQ_FROM FROM_EQ_ENV_FROM FROM_NO_DN\n\nFrom: hello@domain.org\nTo: hello@domain.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect FROM_EQ_ENV_FROM FROM_NO_DN\n\nFrom: hello@domain.org\nTo: hello@domain.org, bye@domain.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect FROM_NEEDS_ENCODING FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Hélló\" <hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nparam.smtputf8 1\nenvelope_from hello@domain.org\nexpect FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Hélló\" <hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect FROM_EXCESS_QP FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: =?iso-8859-1?Q?Die_Hasen_und_die_Froesche?= <hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect FROM_EXCESS_BASE64 FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"=?iso-8859-1?B?RGllIEhhc2VuIHVuIGRpZSBGcm9lc2NoZQ==?=\" <hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"=?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche?=\" <hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect NO_SPACE_IN_FROM FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Hello\"<hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Hello\"\n\t<hello@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect HEADER_RCONFIRM_MISMATCH FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Hello\" <hello@domain.org>\nX-Confirm-Reading-To: <bye@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.org\nexpect HEADER_FORGED_MDN FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Hello\" <hello@domain.org>\nDisposition-Notification-To: <bye@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from anonymous@domain.org\nexpect FROM_SERVICE_ACCT WWW_DOT_DOMAIN FROM_EQ_ENV_FROM FROM_HAS_DN\n\nFrom: \"Hello\" <anonymous@domain.org>\nReply-to: <info@www.domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@custom.disposable.org\nexpect FREEMAIL_FROM DISPOSABLE_ENV_FROM FROM_NEQ_ENV_FROM FROM_NO_DN FORGED_SENDER\n\nFrom: hello@gmail.com\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@gmail.com\nexpect DISPOSABLE_FROM FREEMAIL_ENV_FROM FROM_NEQ_ENV_FROM FROM_NO_DN FORGED_SENDER\n\nFrom: hello@custom.disposable.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@nomx.org\nexpect FROMHOST_NORES_A_OR_MX FROM_EQ_ENV_FROM FROM_NO_DN\n\nFrom: hello@nomx.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from baz@domain.org\nexpect SPOOF_DISPLAY_NAME FROM_HAS_DN FROM_EQ_ENV_FROM\n\nFrom: \"Foo (foo@bar.com)\" <baz@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from baz@domain.org\nexpect SPOOF_DISPLAY_NAME FROM_HAS_DN FROM_EQ_ENV_FROM\n\nFrom: Foo (foo@bar.com) <baz@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from baz@domain.org\nexpect SPOOF_DISPLAY_NAME FROM_HAS_DN FROM_EQ_ENV_FROM\n\nFrom: \"Foo foo@bar.com\" <baz@domain.org>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from baz@domain.org\nexpect SPOOF_DISPLAY_NAME FROM_HAS_DN FROM_EQ_ENV_FROM\n\nFrom: \"Foo 'foo@bar.com'\" <baz@domain.org>\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/headers.test",
    "content": "expect HAS_X_PRIO_ONE\n\nX-Priority: 1\nFrom: test@test.com\nTo: test@test.com\n\nTest\n<!-- NEXT TEST -->\nexpect MULTIPLE_UNIQUE_HEADERS HAS_X_PRIO_TWO\n\nX-Mailer: my mailer 1\nX-Priority: 2\nFrom: test@test.com\nFrom: test@test.com\nTo: test@test.com\n\nTest\n<!-- NEXT TEST -->\nexpect XM_CASE HAS_LIST_UNSUB PRECEDENCE_BULK MULTIPLE_UNIQUE_HEADERS\n\nX-mailer: my mailer 1\nList-Unsubscribe: <unsub@list.org>\nPrecedence: bulk\nSubject: first subject\nSubject: second subject\n\nTest\n<!-- NEXT TEST -->\nexpect KLMS_SPAM UNITEDINTERNET_SPAM SPAM_FLAG XM_UA_NO_VERSION\n\nX-Mailer: my mailer\nX-KLMS-AntiSpam-Status: spam\nX-Spam: Yes\nX-UI-Filterresults: JUNK\nSubject: test\n\nTest\n<!-- NEXT TEST -->\nexpect X_PHP_EVAL HIDDEN_SOURCE_OBJ HAS_X_GMSV HAS_X_AS\n\nX-PHP-Script: sendmail.php\nX-PHP-Originating-Script: eval()\nX-Source-Args: ../script\nX-Authenticated-Sender: sender: test@test.org\nX-Get-Message-Sender-Via: authenticated_id: 123\nX-AntiAbuse: 1\nX-Authentication-Warning: 1\nSubject: test\n\nTest\n<!-- NEXT TEST -->\nexpect HEADER_EMPTY_DELIMITER\n\nSubject:test\n\nTest\n<!-- NEXT TEST -->\nexpect \n\nSubject:\n test\n\nTest\n<!-- NEXT TEST -->\nexpect MAILLIST\n\nList-Archive: 1\nList-Owner: 1\nList-Help: 1\nList-Post: 1\nX-Loop: 1\nList-Id: 1\nSubject: test\n\nTest\n<!-- NEXT TEST -->\nexpect MAILLIST HAS_LIST_UNSUB\n\nList-Id: 1\nList-Subscribe: 1\nList-Unsubscribe: 1\nSubject: test\n\nTest\n<!-- NEXT TEST -->\nexpect MISSING_ESSENTIAL_HEADERS\n\nX-Other: test\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/helo.test",
    "content": "helo_domain localhost\nexpect HELO_NOT_FQDN\n\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nhelo_domain user\nexpect RCVD_HELO_USER HELO_NOT_FQDN\n\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nhelo_domain 8.8.8.8\nremote_ip 8.8.8.8\nexpect HELO_BAREIP\n\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nhelo_domain 8.8.8.8\nremote_ip 1.1.1.1\nexpect HELO_IP_A HELO_BAREIP\n\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nhelo_domain domain.org\niprev.ptr domain.org\nremote_ip 1.1.1.1\nexpect \n\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nhelo_domain domain.org\niprev.ptr otherdomain.org\nremote_ip 1.1.1.1\nexpect HELO_IPREV_MISMATCH\n\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nhelo_domain otherdomain.org\niprev.ptr otherdomain.org\nremote_ip 1.1.1.1\nexpect HELO_NORES_A_OR_MX\n\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nhelo_domain otherdomain.org\niprev.ptr otherdomain.net\nremote_ip 1.1.1.1\nexpect HELO_NORES_A_OR_MX HELO_IPREV_MISMATCH\n\nSubject: test\n\ntest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/html.test",
    "content": "expect MIME_HTML_ONLY HTML_SHORT_1\n\nMessage-Id: <4.2.0.58.20000519002557.00a88870@pop.example.com>\nX-Sender: dwsauder@pop.example.com (Unverified)\nX-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.58 \nX-Priority: 2 (High)\nDate: Fri, 19 May 2000 00:29:55 -0400\nTo: Heinz =?iso-8859-1?Q?M=FCller?= <mueller@example.com>\nFrom: Doug Sauder <dwsauder@example.com>\nSubject: =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche?=\nMime-Version: 1.0\nContent-Type: text/html; charset=\"iso-8859-1\"\nContent-Transfer-Encoding: quoted-printable\n\n<html>\n<font face=3D\"Arial, Helvetica\" size=3D5 color=3D\"#0000FF\"><b>Die Hasen und =\ndie\nFr=F6sche<br>\n<br>\n</font></b><font face=3D\"Arial, Helvetica\">Die Hasen klagten einst =FCber\nihre mi=DFliche Lage; &quot;wir leben&quot;, sprach ein Redner, &quot;in\nsteter Furcht vor Menschen und Tieren, eine Beute der Hunde, der Adler,\nja fast aller Raubtiere! Unsere stete Angst ist =E4rger als der Tod selbst.\nAuf, la=DFt uns ein f=FCr allemal sterben.&quot; <br>\n<br>\nIn einem nahen Teich wollten sie sich nun ers=E4ufen; sie eilten ihm zu;\nallein das au=DFerordentliche Get=F6se und ihre wunderbare Gestalt\nerschreckte eine Menge Fr=F6sche, die am Ufer sa=DFen, so sehr, da=DF sie au=\nfs\nschnellste untertauchten. <br>\n<br>\n&quot;Halt&quot;, rief nun eben dieser Sprecher, &quot;wir wollen das\nErs=E4ufen noch ein wenig aufschieben, denn auch uns f=FCrchten, wie ihr\nseht, einige Tiere, welche also wohl noch ungl=FCcklicher sein m=FCssen als\nwir.&quot; <br>\n<br>\n</font></html>\n<!-- NEXT TEST -->\nexpect HTTP_TO_HTTPS HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nhttps://mydomain.com\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</p>\n<a href=\"http://mydomain.com\">https://mydomain.com</a>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect HTTP_TO_IP HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nhttps://mydomain.com\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head></head>\n<body>\n<p>some text</p>\n<a href=\"https://8.8.8.8/phisherino.php\">https://8.8.8.8</a>\n</body>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect EXT_CSS HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nhttps://mydomain.com\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<link href=\"https://domain.com/external.css#test\">\n<p>some text</p>\n<a href=\"https://mydomain.com\">https://mydomain.com</a>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect EXT_CSS HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nhttps://mydomain.com\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<link rel=\"stylesheet\" href=\"https://domain.com/external\">\n<p>some text</p>\n<a href=\"https://mydomain.com\">https://mydomain.com</a>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect HTML_UNBALANCED_TAG HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nhttps://mydomain.com\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<link rel=\"unknown\" href=\"https://domain.com/external\">\n<head>hello\n<a href=\"https://mydomain.com\">https://mydomain.com</a>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect HTML_UNBALANCED_TAG HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nhttps://mydomain.com\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<link rel=\"unknown\" href=\"https://domain.com/external\">\n<body>hello\n<a href=\"https://mydomain.com\">https://mydomain.com</a>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect HTML_SHORT_LINK_IMG_1 HTML_SHORT_1 HAS_LINK_TO_LARGE_IMG\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nTesting\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<html>\n<head><title>Test</title></head>\n<body>\n<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam</p>\n<a href=\"https://herrspammer.com\"><img src=\"cid:spammimg\" width=\"200\" height=\"200\"></a>\n</body>\n</html>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect BODY_URI_ONLY HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nTesting\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<html>\n<head><title>Test</title></head>\n<body>\n<p>http://myurl.com</>\n</body>\n</html>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect HTML_TEXT_IMG_RATIO HTML_SHORT_1\n\nContent-Type: multipart/alternative;\n\tboundary=\"=====================_714967308==_.ALT\"\n\n--=====================_714967308==_.ALT\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\n\nTesting\n\n--=====================_714967308==_.ALT\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<html>\n<head><title>Test</title></head>\n<body>\n<a<img src=\"cid:spammimg1\" width=\"200\" height=\"200\">\n<img src=\"cid:spammimg2\" width=\"200\" height=\"200\">\n<img src=\"cid:spammimg3\" width=\"200\" height=\"200\">\n</body>\n</html>\n\n--=====================_714967308==_.ALT--\n<!-- NEXT TEST -->\nexpect HTML_META_REFRESH_URL MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head>\n<meta http-equiv=\"refresh\" content=\"5; url=https://example.com/\">\n</head>\n<body>\n<p>some text</p>\n</body>\n<!-- NEXT TEST -->\nexpect MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head>\n<meta http-equiv=\"refresh\" content=\"5\">\n</head>\n<body>\n<p>some text</p>\n</body>\n<!-- NEXT TEST -->\nexpect HAS_DATA_URI DATA_URI_OBFU MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head>\n<meta http-equiv=\"refresh\" content=\"5\">\n</head>\n<body>\n<p>some text</p>\n<a href=\"data:text/plain;base64,SGVsbG8sIHdvcmxkIQ==\">Click me for a hello message</a>\n</body>\n<!-- NEXT TEST -->\nexpect HAS_DATA_URI MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head>\n<meta http-equiv=\"refresh\" content=\"5\">\n</head>\n<body>\n<p>some text and a lovely explanation to avoid the text to image ratio tag</p>\n<img src=\"data:image/png;base64,iVBORw0KGg....\" alt=\"Red dot\" />\n<a href=\"data:other\">Click me for a hello message</a>\n</body>\n<!-- NEXT TEST -->\nexpect PHISHING MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head></head><body><p>some text</p>\n<a href=\"https://domain1.com/query\">https://domain2.com/otherquery</a>\n</body>\n<!-- NEXT TEST -->\nexpect MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head></head><body><p>some text</p>\n<a href=\"https://domain1.co.uk/query\">https://subdomain.domain1.co.uk/otherquery</a>\n</body>\n<!-- NEXT TEST -->\nexpect PHISHING MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head></head><body><p>some text</p>\n<a href=\"https://domain1.com/query\">domain2.com/otherquery</a>\n</body>\n<!-- NEXT TEST -->\nexpect MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head></head><body><p>some text</p>\n<a href=\"https://domain1.co.uk/query\">subdomain.domain1.co.uk/otherquery</a>\n</body>\n<!-- NEXT TEST -->\nexpect MIME_HTML_ONLY HTML_SHORT_1\n\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\n\n<head></head><body><p>some text</p>\n<a href=\"https://domain1.co.uk/query\">normal text</a>\n</body>\n"
  },
  {
    "path": "tests/resources/smtp/antispam/ip.test",
    "content": "remote_ip 8.8.8.8\niprev.result temperror\nexpect RDNS_DNSFAIL\n\nSubject: test\n\nTest\n\n<!-- NEXT TEST -->\nremote_ip 8.8.8.8\niprev.result fail\nexpect RDNS_NONE\n\nSubject: test\n\nTest\n\n"
  },
  {
    "path": "tests/resources/smtp/antispam/llm.test",
    "content": "expect LLM_UNSOLICITED_HIGH\n\nSubject: Unsolicited,High,Test\n\nTest\n\n<!-- NEXT TEST -->\nexpect LLM_COMMERCIAL_HIGH\n\nSubject: Commercial,High,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_HARMFUL_HIGH\n\nSubject: Harmful,High,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_LEGITIMATE_HIGH\n\nSubject: Legitimate,High,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_UNSOLICITED_MEDIUM\n\nSubject: Unsolicited,Medium,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_COMMERCIAL_MEDIUM\n\nSubject: Commercial,Medium,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_HARMFUL_MEDIUM\n\nSubject: Harmful,Medium,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_LEGITIMATE_MEDIUM\n\nSubject: Legitimate,Medium,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_UNSOLICITED_LOW\n\nSubject: Unsolicited,Low,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_COMMERCIAL_LOW\n\nSubject: Commercial,Low,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_HARMFUL_LOW\n\nSubject: Harmful,Low,Test\n\nTest\n<!-- NEXT TEST -->\nexpect LLM_LEGITIMATE_LOW\n\nSubject: Legitimate,Low,Test\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/messageid.test",
    "content": "expect MISSING_MID\n\nX-Message-ID: <hello@world.com>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_RHS_IP_LITERAL\n\nMessage-ID: <hello@[127.0.0.1]>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_BARE_IP\n\nMessage-ID: <hello@127.0.0.1>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_RHS_NOT_FQDN\n\nMessage-ID: <hello@domain>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_RHS_WWW\n\nMessage-ID: <hello@www.domain.com>\n\nTest\n<!-- NEXT TEST -->\nexpect INVALID_MSGID\n\nMessage-ID: <@domain.com>\n\nTest\n<!-- NEXT TEST -->\nexpect INVALID_MSGID\n\nMessage-ID: <hélló@domain.com>\n\nTest\n\n<!-- NEXT TEST -->\nexpect INVALID_MSGID\n\nMessage-ID: <hello@domain.com> (hello world)\n\nTest\n<!-- NEXT TEST -->\nexpect MID_RHS_TOO_LONG\n\nMessage-ID: <hello@domaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomaindomain.com>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_MISSING_BRACKETS\n\nMessage-ID: hello@domain.com\n\nTest\n<!-- NEXT TEST -->\nexpect MID_CONTAINS_FROM\n\nFrom: <HELLO@DOMAIN.COM>\nMessage-ID: <hello@DOMAIN.com>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_RHS_MATCH_FROM\n\nFrom: <HELLOWORLD@DOMAIN.COM>\nMessage-ID: <hello@DOMAIN.com>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_RHS_MATCH_FROMTLD\n\nFrom: <hello@domain.co.uk>\nMessage-ID: <1234@host.domain.co.uk>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@domain.co.uk\nexpect MID_RHS_MATCH_ENV_FROMTLD\n\nMessage-ID: <1234@host.domain.co.uk>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_CONTAINS_TO\n\nTo: User <user@domain.com>\nMessage-ID: <user@domain.com>\n\nTest\n<!-- NEXT TEST -->\nexpect MID_RHS_MATCH_TO\n\nFrom: Myself <me@otherdomain.com>\nTo: User <user@domain.com>\nCc: John <john@doe.com>, Jane <jane@doe.com>, Bill <bill@foobar.org>\nMessage-ID: <user@foobar.org>\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/mime.test",
    "content": "expect MISSING_MIME_VERSION SINGLE_SHORT_PART\n\nContent-Type: text/plain; charset=\"us-ascii\"\n\nTest\n<!-- NEXT TEST -->\nexpect MV_CASE SINGLE_SHORT_PART\n\nContent-Type: text/plain; charset=\"us-ascii\"\nMime-Version: 1.0\n\nTest\n<!-- NEXT TEST -->\nexpect CTE_CASE CT_EXTRA_SEMI SINGLE_SHORT_PART\n\nContent-Type: text/plain; charset=\"us-ascii\"; \nContent-Transfer-Encoding: 7Bit\nMIME-Version: 1.0\n\nTest\n<!-- NEXT TEST -->\nexpect BROKEN_CONTENT_TYPE SINGLE_SHORT_PART\n\nContent-Type: ; tag=1\nContent-Transfer-Encoding: 7bit\nMIME-Version: 1.0\n\nTest\n<!-- NEXT TEST -->\nexpect MIME_HEADER_CTYPE_ONLY MISSING_MIME_VERSION SINGLE_SHORT_PART\n\nContent-Type: text/html; charset=\"us-ascii\"\n\nTest\n<!-- NEXT TEST -->\nexpect BAD_CTE_7BIT SINGLE_SHORT_PART\n\nContent-Type: text/plain\nContent-Transfer-Encoding: 7bit\nMIME-Version: 1.0\n\nTéstíng\n<!-- NEXT TEST -->\nexpect MISSING_CHARSET SINGLE_SHORT_PART\n\nContent-Type: text/plain\nContent-Transfer-Encoding: 8bit\nMIME-Version: 1.0\n\nTest\n<!-- NEXT TEST -->\nexpect MIME_BASE64_TEXT_BOGUS SINGLE_SHORT_PART\n\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: base64\nMIME-Version: 1.0\n\naGVsbG8gd29ybGQK\n\n<!-- NEXT TEST -->\nexpect MIME_BASE64_TEXT SINGLE_SHORT_PART\n\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: base64\nMIME-Version: 1.0\n\naMOpbGzDsyB3w7NybGQK\n\n<!-- NEXT TEST -->\nexpect \n\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor \nincididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud \nexercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. \nDuis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore \neu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \nculpa qui officia deserunt mollit anim id est laborum.\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\n<html>\n<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor \nincididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud \nexercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.</p>\n<p>Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore \neu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \nculpa qui officia deserunt mollit anim id est laborum.</p>\n</html>\n\n--boundary--\n<!-- NEXT TEST -->\nexpect MIME_MA_MISSING_TEXT\n\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\n<html>\n<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor \nincididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud \nexercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.</p>\n<p>Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore \neu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \nculpa qui officia deserunt mollit anim id est laborum.</p>\n</html>\n\n--boundary--\n<!-- NEXT TEST -->\nexpect MIME_MA_MISSING_HTML\n\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nLorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor \nincididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud \nexercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. \nDuis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore \neu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \nculpa qui officia deserunt mollit anim id est laborum.\n\n--boundary--\n<!-- NEXT TEST -->\nexpect PARTS_DIFFER\n\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nLorem ipsum dolor sit Ramet, Rcnsectetur Radipiscing elit, Rsed do Reiusmod tempor \nincididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud \nexercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. \nDuis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore \neu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \nculpa qui officia deserunt mollit anim id est laborum.\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\n<html>\n<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor \nincididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud \nexercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.</p>\n<p>Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore \neu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \nculpa qui officia deserunt mollit anim id est laborum.</p>\n</html>\n--boundary--\n<!-- NEXT TEST -->\nexpect URI_COUNT_ODD\n\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nFind me at http://www.example.com or http://www.example.org\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\n<html>\n<p>Find me at http://www.example.com or</p>\n</html>\n--boundary--\n<!-- NEXT TEST -->\nexpect \n\nMIME-Version: 1.0\nContent-Type: multipart/alternative;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nFind me at http://www.example.com or http://www.example.org\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\n<html>\n<p>Find me at http://www.example.com or http://example.org</p>\n</html>\n--boundary--\n<!-- NEXT TEST -->\nexpect CTYPE_MIXED_BOGUS\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\n<html>\n<p>this is a test</p>\n</html>\n--boundary--\n<!-- NEXT TEST -->\nexpect \n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nanother test\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nlast test\n\n--boundary--\n<!-- NEXT TEST -->\nexpect HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: application/octet-stream\nContent-Disposition: attachment\nContent-Transfer-Encoding: 7bit\n\n<html>\n<p>this is a test</p>\n</html>\n--boundary--\n<!-- NEXT TEST -->\nexpect CTYPE_MISSING_DISPOSITION HAS_ATTACHMENT SINGLE_SHORT_PART\n\nContent-Type: application/octet-stream\nMIME-Version: 1.0\n\nTest\n<!-- NEXT TEST -->\nexpect ENCRYPTED_PGP ENCRYPTED_SMIME SIGNED_PGP SIGNED_SMIME HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/encrypted;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: application/pkcs7-mime\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: application/pkcs7-signature\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: application/pgp-encrypted\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: application/pgp-signature\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: application/octet-stream\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary--\n<!-- NEXT TEST -->\nexpect CTYPE_MISSING_DISPOSITION HAS_ATTACHMENT SINGLE_SHORT_PART\n\nContent-Type: application/octet-stream\nMIME-Version: 1.0\n\nTest\n<!-- NEXT TEST -->\nexpect BOGUS_ENCRYPTED_AND_TEXT ENCRYPTED_SMIME \n\nMIME-Version: 1.0\nContent-Type: multipart/encrypted;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: application/pkcs7-mime\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary\nContent-Type: text/html\nContent-Transfer-Encoding: 7bit\n\nthis is a test\n\n--boundary--\n<!-- NEXT TEST -->\nexpect MIXED_CHARSET SINGLE_SHORT_PART\n\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 8bit\nMIME-Version: 1.0\n\nTést 孔子\n\n<!-- NEXT TEST -->\nexpect MIME_BAD_EXTENSION MIME_GOOD HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nsimple text\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Disposition: attachment; filename=\"test.html\"\nContent-Transfer-Encoding: 8bit\n\n<html>\n<p>hello world</p>\n</html>\n--boundary--\n\n<!-- NEXT TEST -->\nexpect MIME_BAD_ATTACHMENT HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nsimple text\n\n--boundary\nContent-Type: text/x-plain; charset=\"utf-8\"\nContent-Disposition: attachment; filename=\"test.txt\"\nContent-Transfer-Encoding: 8bit\n\nhello world\n--boundary--\n\n<!-- NEXT TEST -->\nexpect HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nsimple text\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Disposition: attachment; filename=\"test.txt\"\nContent-Transfer-Encoding: 8bit\n\nhello world\n--boundary--\n\n<!-- NEXT TEST -->\nexpect MIME_DOUBLE_BAD_EXTENSION MIME_GOOD HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nsimple text\n\n--boundary\nContent-Type: text/html; charset=\"utf-8\"\nContent-Disposition: attachment; filename=\"test.html.html\"\nContent-Transfer-Encoding: 8bit\n\n<html>\n<p>hello world</p>\n</html>\n--boundary--\n\n<!-- NEXT TEST -->\nexpect MIME_ARCHIVE_IN_ARCHIVE MIME_GOOD HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nsimple text\n\n--boundary\nContent-Type: application/zip\nContent-Disposition: attachment; filename=\"test.zip.zip\"\nContent-Transfer-Encoding: base64\n\nUEsDBAoAAAAAALN6RlcAAAAAAAAAAAAAAAAIABwAdGVzdC5iaW5VVAkAA+IJI\nGXiCSBldXgLAAEE9QEAAAQUAAAAUEsBAh4DCgAAAAAAs3pGVwAAAAAAAAAAAA\nAAAAgAGAAAAAAAAAAAAKSBAAAAAHRlc3QuYmluVVQFAAPiCSBldXgLAAEE9QE\nAAAQUAAAAUEsFBgAAAAABAAEATgAAAEIAAAAAAA==\n--boundary--\n<!-- NEXT TEST -->\nexpect MIME_BAD HAS_ATTACHMENT\n\nMIME-Version: 1.0\nContent-Type: multipart/mixed;\n\tboundary=\"boundary\"\n\n--boundary\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: 7bit\n\nsimple text\n\n--boundary\nContent-Type: image/png\nContent-Disposition: attachment; filename=\"test.png\"\nContent-Transfer-Encoding: base64\n\nUEsDBAoAAAAAALN6RlcAAAAAAAAAAAAAAAAIABwAdGVzdC5iaW5VVAkAA+IJI\nGXiCSBldXgLAAEE9QEAAAQUAAAAUEsBAh4DCgAAAAAAs3pGVwAAAAAAAAAAAA\nAAAAgAGAAAAAAAAAAAAKSBAAAAAHRlc3QuYmluVVQFAAPiCSBldXgLAAEE9QE\nAAAQUAAAAUEsFBgAAAAABAAEATgAAAEIAAAAAAA==\n--boundary--\n"
  },
  {
    "path": "tests/resources/smtp/antispam/pyzor.test",
    "content": "expect PYZOR\n\nSubject: test\n\nTesta Testb Testc Testd\n\n<!-- NEXT TEST -->\nexpect PYZOR\n\nSubject: test\n\nTest1 Test1 Test2 Test3\n\n<!-- NEXT TEST -->\nexpect \n\nSubject: test\n\nTestX TestY TestZ TestW\n\n<!-- NEXT TEST -->\nexpect \n\nMIME-Version: 1.0\nX-Received: by 2002:a05:6870:c7a6:b0:1e9:8f74:ce15 with SMTP id\n dy38-20020a056870c7a600b001e98f74ce15mr2429202oab.11.1697967052502; Sun, 22\n Oct 2023 02:30:52 -0700 (PDT)\nDate: Sat, 21 Oct 2023 16:59:59 -0700\nMessage-ID: <17434871060391156945@google.com>\nSubject: Report domain: stalw.art Submitter: google.com Report-ID: 17434871060391156945\nFrom: noreply-dmarc-support@google.com\nTo: domains@stalw.art\nContent-Type: application/zip; \n\tname=\"google.com!stalw.art!1697846400!1697932799.zip\"\nContent-Disposition: attachment; \n\tfilename=\"google.com!stalw.art!1697846400!1697932799.zip\"\nContent-Transfer-Encoding: base64\n\nUEsDBAoAAAAIABdJVldhOnFiLAIAAG4JAAAuAAAAZ29vZ2xlLmNvbSFzdGFsdy5hcnQhMTY5Nzg0\nNjQwMCExNjk3OTMyNzk5LnhtbO1WwXKbMBC95ys8vhshDBgYRempX9CeGRkE1hgkjSTs5O8rIgnT\npMl0ptOcfEK83X27+/TwGD09j8PmQpVmgj9uYRRvN5Q3omW8f9z+/PF9V2w3T/gBdZS2R9Kc8cNm\ngxSVQpl6pIa0xJAZs6hQfc3JSHEvRD/QqBEjAgvocuhI2IC5sAzDy64diWp2epIz3bd1mcvzNc9G\nkboR3JDG1Ix3Ap+MkboCwJdGt1JAAOH6ShVI0jzPithyva93xH4N1mJ4SPdpcYBxHu9LCLO8TDME\nbnGXb3eltSK899tY6Eh7xjHMy0OR5mlsuzkkxClvX6PlPjmUpZ2FBzLwO9vSbS0qkmJgzUstp+PA\n9IkugwgrD8fakOEaEWUsmUNcmLRnNmKFgDt4UMvuFZufDpL2IjhFQPp3HQAdENkYDOet5oODeEji\n0s39pxmttI1QYVwlrosgWkyqoTWTGO6zCBYwgmUWJbHV5hYKyY2YuMF7BNwhwL4jvZBhsiK2ITAr\nw7QUmhnrZj/mGlnlzcJIorVNWDTyInQ+sAi12vJNT3trYTfEWsoN65j9lpayEyUtVXWnxLi+rTXs\ned5VIzKZU62ongZzI3wz7OdG8CafGfxK/mW1LR1oY4TCFzqwM9NKEp4kdveALwKsO6OVNP88xUpo\n6843S8/JwUl/Y6qExHEF0yyu0iJPqySuqiQ9fOwt+OXe6uwP291b/8FbTtiv9RZM87u37t76zFsI\n3P46/QJQSwECCgAKAAAACAAXSVZXYTpxYiwCAABuCQAALgAAAAAAAAAAAAAAAAAAAAAAZ29vZ2xl\nLmNvbSFzdGFsdy5hcnQhMTY5Nzg0NjQwMCExNjk3OTMyNzk5LnhtbFBLBQYAAAAAAQABAFwAAAB4\nAgAAAAA=\n"
  },
  {
    "path": "tests/resources/smtp/antispam/rbl.test",
    "content": "remote_ip 20.11.0.1\nexpect RCVD_IN_DNSWL_LOW RBL_SENDERSCORE_REPUT_0\n\nSubject: test\n\ntest\n\n<!-- NEXT TEST -->\nremote_ip 20.11.0.2\nexpect RBL_SENDERSCORE_REPUT_0 RBL_SEM RBL_SPAMHAUS_SBL RBL_BARRACUDA RBL_BLOCKLISTDE RBL_VIRUSFREE_BOTNET RBL_SPAMCOP RCVD_IN_DNSWL_MED\n\nSubject: test\n\ntest\n\n<!-- NEXT TEST -->\nremote_ip 20.11.0.14\nexpect RBL_SENDERSCORE_REPUT_1 RWL_MAILSPIKE_NEUTRAL RECEIVED_SPAMHAUS_SBL RECEIVED_SPAMHAUS_XBL RECEIVED_BLOCKLISTDE RCVD_IN_DNSWL_MED\n\nReceived: from Agni (localhost [20.11.0.5]) (TLS: TLSv1/SSLv3, 168bits,DES-CBC3-SHA) by agni.forevermore.net \n          with esmtp; Mon, 28 Oct 2002 14:48:52 -0800\nReceived: from [20.11.0.14] (79.sub-174-252-72.myvzw.com [20.11.0.8]) by mx.google.com \n          with ESMTPS id m16sm345129qck.28.2011.06.15.07.42.02 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 15 Jun 2011 07:42:08 -0700 (PDT)\nReceived: from user (20.11.0.2) by DB6PR07MB3384.eurprd07.prod.outlook.com ([20.11.0.2]) \n          with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.11; Thu, 13 Sep 2018 14:47:44 +0000\nSubject: test\n\ntest\n\n<!-- NEXT TEST -->\nenvelope_from user@surbl-abuse.com\nexpect URIBL_GREY ABUSE_SURBL DBL_MALWARE SEM_URIBL_FRESH15 SEM_URIBL\n\nFrom: user@uribl-grey.com\nSubject: check my website sh-malware.com/login.php\n\nMy e-mail is spammer@sem-uribl.com\nAnd my website is https://sem-fresh15.com/offers.html\nTry cheating with a trusted domain user@dkimtrusted.org\n\n<!-- NEXT TEST -->\ndkim.result pass\ndkim.domains dkimtrusted.org\nexpect DWL_DNSWL_HI\n\nFrom: user@dkimtrusted.org\nSubject: test\n\ntest\n\n<!-- NEXT TEST -->\nexpect MSBL_EBL MSBL_EBL_GREY\n\nFrom: spammer1@spamcorp.net\nReply-To: User <spammer2@spamcorp.net>\nSubject: test\n\ntest\n\n<!-- NEXT TEST -->\nexpect SURBL_HASHBL_ABUSE SURBL_HASHBL_MALWARE SURBL_HASHBL_PHISH URL_ONLY REDIRECTOR_URL\n\nFrom: spammer@spamcorp.net\nReply-To: User <spammer@spamcorp.net>\nSubject: test\nContent-Type: text/html; charset=\"utf-8\"\n\n<html>\n<a href=\"https://bit.ly/abcde\">test</a>\n<img src=\"https://drive.google.com/path/to/file.exe\">https://lnkiy.in/other/path?query=true</a<\n</html>\n"
  },
  {
    "path": "tests/resources/smtp/antispam/received.test",
    "content": "expect RCVD_COUNT_THREE RCVD_NO_TLS_LAST\n\nReceived: from BAY0-HMR08.bay0.hotmail.com (bay0-hmr08.bay0.hotmail.com [65.54.241.207]) \n          by dogma.slashnull.org (8.11.6/8.11.6) \n          with ESMTP id h2DBpvs24047 for <webmaster@efi.ie>; Thu, 13 Mar 2003 11:51:57 GMT\nReceived: from BAY0-HMR08.bay0.hotmail.com (bay0-hmr08.bay0.hotmail.com [65.54.241.207]) \n          by dogma.slashnull.org (8.11.6/8.11.6) \n          with ESMTP id h2DBpvs24047 for <webmaster@efi.ie>; Thu, 13 Mar 2003 11:51:57 GMT\nReceived: from BAY0-HMR08.bay0.hotmail.com (bay0-hmr08.bay0.hotmail.com [65.54.241.207]) \n          by dogma.slashnull.org (8.11.6/8.11.6) \n          with ESMTP id h2DBpvs24047 for <webmaster@efi.ie>; Thu, 13 Mar 2003 11:51:57 GMT\n\ntest\n<!-- NEXT TEST -->\nauthenticated_as john@doe.com\ntls.version TLSv1.3\nexpect RCVD_VIA_SMTP_AUTH RCVD_COUNT_ONE RCVD_TLS_LAST\n\nReceived: from BAY0-HMR08.bay0.hotmail.com (bay0-hmr08.bay0.hotmail.com [65.54.241.207]) \n          by dogma.slashnull.org (8.11.6/8.11.6) \n          with ESMTP id h2DBpvs24047 for <webmaster@efi.ie>; Thu, 13 Mar 2003 11:51:57 GMT\n\ntest\n<!-- NEXT TEST -->\nexpect RCVD_ILLEGAL_CHARS RCVD_COUNT_ONE RCVD_NO_TLS_LAST\n\nReceived: from BAY0-HMR08.bay0.hótmail.com (bay0-hmr08.bay0.hótmail.com [65.54.241.207]) \n          by dogma.slashnull.org (8.11.6/8.11.6) \n          with ESMTP id h2DBpvs24047 for <webmaster@efi.ie>; Thu, 13 Mar 2003 11:51:57 GMT\n\ntest\n\n<!-- NEXT TEST -->\ntls.version TLVv1.3\nexpect RCVD_TLS_ALL RCVD_HELO_USER RCVD_DOUBLE_IP_SPAM FORGED_RCVD_TRAIL PREVIOUSLY_DELIVERED RCVD_COUNT_FIVE\n\nReceived: from Agni (localhost [::ffff:127.0.0.1]) (TLS: TLSv1/SSLv3, 168bits,DES-CBC3-SHA) by agni.forevermore.net \n          with esmtp; Mon, 28 Oct 2002 14:48:52 -0800\nReceived: from [10.231.252.223] (79.sub-174-252-72.myvzw.com [174.252.72.79]) by mx.google.com \n          with ESMTPS id m16sm345129qck.28.2011.06.15.07.42.02 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 15 Jun 2011 07:42:08 -0700 (PDT)\nReceived: from other.myvzw.com (79.sub-174-252-72.myvzw.com [174.252.72.79]) by mx.google.com \n          with ESMTPS id m16sm345129qck.28.2011.06.15.07.42.02 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 15 Jun 2011 07:42:08 -0700 (PDT)\nReceived: from user (10.175.233.33) by DB6PR07MB3384.eurprd07.prod.outlook.com (10.175.234.11) \n          with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.11; Thu, 13 Sep 2018 14:47:44 +0000\nReceived: from [94.198.96.74] (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) \n          by ietfa.amsl.com (Postfix) with ESMTPS id 10B7AC151535 for <user@domain.com>; Mon, 28 Aug 2023 02:21:23 -0700 (PDT)\nTo: user@domain.com\nSubject: test\n\ntest\n\n<!-- NEXT TEST -->\nexpect DIRECT_TO_MX RCVD_COUNT_ZERO RCVD_NO_TLS_LAST\n\nTo: user@domain.com\nX-Mailer: MUA 1.2\nSubject: test\n\ntest\n<!-- NEXT TEST -->\nexpect RCVD_UNPARSABLE RCVD_NO_TLS_LAST RCVD_COUNT_ONE\n\nTo: user@domain.com\nReceived: invalid\n\ntest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/recipient.test",
    "content": "expect MISSING_TO RCPT_COUNT_ZERO\n\nX-To: hello@world.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_COUNT_ONE TO_DN_ALL\n\nTo: \"Hello World\" <hello@world.com>\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_COUNT_ONE TO_DN_NONE TAGGED_RCPT\n\nTo: hello+there@world.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from user@domain.org\nexpect TO_DN_RECIPIENTS RCPT_COUNT_TWO TO_DN_SOME\n\nTo: \"recipients\" <user@domain.org>\nCc: other@user.org\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_IN_SUBJECT TO_DN_NONE RCPT_COUNT_ONE\n\nTo: hello@world.com\nSubject: Special offer for HELLO@world.com\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_LOCAL_IN_SUBJECT TO_DN_NONE RCPT_COUNT_ONE\n\nTo: hello@world.com\nSubject: Special offer for hello\n\nTest\n<!-- NEXT TEST -->\nenvelope_from \nenvelope_to hello@world.com\nenvelope_to goodbye@world.com\nexpect RCPT_BOUNCEMOREONE TO_MATCH_ENVRCPT_ALL TO_DN_NONE RCPT_COUNT_TWO\n\nTo: hello@world.com\nCc: goodbye@world.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from postmaster@domain.org\nenvelope_to hello@world.com\nenvelope_to goodbye@world.com\nexpect RCPT_BOUNCEMOREONE TO_MATCH_ENVRCPT_SOME TO_DN_NONE RCPT_COUNT_THREE\n\nTo: hello@world.com, test@domain.com\nCc: goodbye@world.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_COUNT_ZERO UNDISC_RCPT\n\nTo: Undisclosed recipients:;\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from list@domain.org\nenvelope_to hello@world.com\nexpect TO_DN_ALL RCPT_COUNT_ONE\n\nList-Id: <list.domain.org>\nTo: \"Mailing List\" <list@domain.org>\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from spammer@domain.org\nenvelope_to hello@world.com\nexpect FORGED_RECIPIENTS TO_NEEDS_ENCODING TO_DN_ALL RCPT_COUNT_ONE\n\nTo: \"Thé Spámmer\" <spammer@domain.org>\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from user@domain.org\nenvelope_to hello@world.com\nenvelope_to user@domain.org\nexpect TO_DN_ALL TO_MATCH_ENVRCPT_ALL RCPT_COUNT_ONE\n\nTo: \"Hello World\" <hello@world.com>\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from user@domain.org\nenvelope_to hello@world.com\nenvelope_to user@domain.org\nexpect TO_EXCESS_QP TO_DN_ALL TO_MATCH_ENVRCPT_ALL RCPT_COUNT_ONE\n\nTo: \"=?iso-8859-1?Q?Die_Hasen_und_die_Froesche?=\" <hello@world.com>\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from user@domain.org\nenvelope_to hello@world.com\nenvelope_to user@domain.org\nexpect TO_EXCESS_BASE64 TO_DN_ALL TO_MATCH_ENVRCPT_ALL RCPT_COUNT_ONE\n\nTo: \"=?iso-8859-1?B?RGllIEhhc2VuIHVuIGRpZSBGcm9lc2NoZQ==?=\" <hello@world.com>\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from test@test.com\nexpect FREEMAIL_TO DISPOSABLE_CC RCPT_COUNT_TWO TO_DN_NONE\n\nTo: user@gmail.com\nCc: otheruser@guerrillamail.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from test@test.com\nexpect FREEMAIL_CC DISPOSABLE_TO DISPOSABLE_BCC RCPT_COUNT_THREE TO_DN_NONE\n\nTo: otheruser@guerrillamail.com\nCc: user@gmail.com\nBcc: some@guerrillamail.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from test@test.com\nexpect SORTED_RECIPS RCPT_COUNT_SEVEN TO_DN_NONE\n\nTo: a@domain.com, b@domain.com, c@domain.com, d@domain.com\nCc: e@domain.com, f@domain.com, g@domain.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from test@test.com\nexpect RCPT_COUNT_SEVEN TO_DN_NONE\n\nTo: tom@domain.com, mark@domain.com, bill@domain.com, peter@domain.com\nCc: jane@domain.com, mary@domain.com, lucy@domain.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from test@test.com\nexpect SUSPICIOUS_RECIPS RCPT_COUNT_SEVEN TO_DN_NONE\n\nTo: tim@domain.com, tom@domain.com, tum@domain.com, tem@domain.com\nCc: tam@domain.com, tron@domain.com, tym@domain.com\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from info@notalist.org\nenvelope_to info@notalist.org\nexpect INFO_TO_INFO_LU RCPT_COUNT_ONE TO_MATCH_ENVRCPT_ALL TO_DN_NONE\n\nFrom: info@notalist.org\nTo: info@notalist.org\nList-Unsubscribe: <info@notalist.org>\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from info@notalist.org\nenvelope_to info@notalist.org\nexpect RCPT_COUNT_ONE TO_MATCH_ENVRCPT_ALL TO_DN_NONE\n\nFrom: info@notalist.org\nTo: info@notalist.org\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@test.org\nenvelope_to user@test.org\nenvelope_to test@test.org\nexpect TO_WRAPPED_IN_SPACES RCPT_COUNT_TWO TO_MATCH_ENVRCPT_ALL TO_DN_NONE\n\nFrom: hello@test.org\nTo: < user@test.org >\nCc: test@test.org\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@test.org\nenvelope_to user@test.org\nenvelope_to test@test.org\nexpect TO_WRAPPED_IN_SPACES RCPT_COUNT_TWO TO_MATCH_ENVRCPT_ALL TO_DN_SOME\n\nFrom: hello@test.org\nTo: user@test.org\nCc: \"Test\" <test@test.org >\nSubject: Hi\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_IN_BODY TO_DN_NONE RCPT_COUNT_ONE\n\nTo: hello@world.com\nSubject: Special offer \n\nAn offer for hello@world.com\n<!-- NEXT TEST -->\nexpect RCPT_DOMAIN_IN_MESSAGE RCPT_IN_BODY RCPT_DOMAIN_IN_SUBJECT RCPT_COUNT_ONE TO_DN_NONE\n\nTo: hello@world.com\nSubject: Message for world.com\n\nAn offer for hello@world.com\n\n<!-- NEXT TEST -->\nexpect RCPT_DOMAIN_IN_MESSAGE RCPT_DOMAIN_IN_BODY RCPT_DOMAIN_IN_SUBJECT RCPT_COUNT_ONE TO_DN_NONE\n\nTo: hello@world.com\nSubject: Message for world.com\n\nAn offer for world.com\n\n"
  },
  {
    "path": "tests/resources/smtp/antispam/replyto.test",
    "content": "expect REPLYTO_UNPARSABLE\n\nReply-to: hello\n\nTest\n<!-- NEXT TEST -->\nexpect REPLYTO_EQ_FROM HAS_REPLYTO \n\nFrom: hello@domain.org\nReply-to: hello@domain.org\n\nTest\n<!-- NEXT TEST -->\nexpect REPLYTO_DOM_EQ_FROM_DOM REPLYTO_DN_EQ_FROM_DN HAS_REPLYTO\n\nFrom: \"Hello\" <hello@host.domain.org.uk>\nReply-to: \"Hello\" <hello@domain.org.uk>\n\nTest\n<!-- NEXT TEST -->\nenvelope_from hello@otherdomain.org.uk\nenvelope_to user@somedomain.com\nexpect REPLYTO_DOM_NEQ_FROM_DOM HAS_REPLYTO\n\nFrom: hello@otherdomain.org.uk\nTo: user@somedomain.com, hello@otherdomain.org.uk\nReply-to: hello@domain.org.uk\n\nTest\n<!-- NEXT TEST -->\nenvelope_from sender@foo.org\nenvelope_to user@somedomain.com\nexpect REPLYTO_EQ_TO_ADDR SPOOF_REPLYTO HAS_REPLYTO\n\nFrom: sender@foo.org\nTo: user@somedomain.com\nReply-to: user@somedomain.com\n\nTest\n<!-- NEXT TEST -->\nenvelope_from list@foo.org\nenvelope_to user@somedomain.com\nexpect REPLYTO_DOM_NEQ_FROM_DOM HAS_REPLYTO\n\nFrom: list@foo.org\nList-Unsubscribe: unsubcribe@foo.org\nTo: user@somedomain.com\nReply-to: user@somedomain.com\n\nTest\n<!-- NEXT TEST -->\nenvelope_from user@foo.org\nenvelope_to other@foo.org\nexpect REPLYTO_DOM_NEQ_FROM_DOM HAS_REPLYTO\n\nFrom: user@foo.org\nTo: otheruser@foo.org\nReply-to: user@otherdomain.org\n\nTest\n<!-- NEXT TEST -->\nenvelope_from user@foo.org\nenvelope_to otheruser@domain.org\nexpect SPOOF_REPLYTO REPLYTO_DOM_NEQ_FROM_DOM HAS_REPLYTO\n\nFrom: user@foo.org\nTo: otheruser@domain.org\nReply-to: user@otherdomain.org\n\nTest\n<!-- NEXT TEST -->\nexpect REPLYTO_EXCESS_QP REPLYTO_DOM_EQ_FROM_DOM HAS_REPLYTO \n\nFrom: hello@domain.org\nReply-to: =?iso-8859-1?Q?Die_Hasen_und_die_Froesche?= <bye@domain.org>\n\nTest\n<!-- NEXT TEST -->\nexpect REPLYTO_EXCESS_BASE64 REPLYTO_DOM_EQ_FROM_DOM HAS_REPLYTO \n\nFrom: hello@domain.org\nReply-to: \"=?iso-8859-1?B?RGllIEhhc2VuIHVuIGRpZSBGcm9lc2NoZQ==?=\" <bye@domain.org>\n\nTest\n<!-- NEXT TEST -->\nexpect REPLYTO_EMAIL_HAS_TITLE REPLYTO_DOM_EQ_FROM_DOM HAS_REPLYTO \n\nFrom: hello@domain.org\nReply-to: \"Mr. Hello\" <bye@domain.org>\n\nTest\n<!-- NEXT TEST -->\nexpect FREEMAIL_REPLY_TO FREEMAIL_FROM REPLYTO_DOM_EQ_FROM_DOM HAS_REPLYTO \n\nFrom: hello@gmail.com\nReply-to: bye@gmail.com\n\nTest\n<!-- NEXT TEST -->\nexpect DISPOSABLE_REPLY_TO DISPOSABLE_FROM REPLYTO_DOM_EQ_FROM_DOM HAS_REPLYTO \n\nFrom: hello@custom.disposable.org\nReply-to: bye@custom.disposable.org\n\nTest\n<!-- NEXT TEST -->\nexpect FREEMAIL_REPLY_TO_NEQ_FROM_DOM FREEMAIL_REPLY_TO FREEMAIL_FROM REPLYTO_DOM_NEQ_FROM_DOM SPOOF_REPLYTO HAS_REPLYTO \n\nFrom: hello@gmail.com\nReply-to: hello@yahoomail.com\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/spamtrap.test",
    "content": "envelope_from spammer@domain.com\nenvelope_to spamtrap@foobar.org\nexpect SPAM_TRAP\n\nSubject: save up to NUMBER on life insurance\n\nwhy spend more than you have to life quote savings ensuring your family s financial security is very important life quote savings makes buying life insurance simple and affordable we provide free access to the very best companies and the lowest rates life quote savings is fast easy and saves you money let us help you get started with the best values in the country on new coverage you can save hundreds or even thousands of dollars by requesting a free quote from lifequote savings our service will take you less than NUMBER minutes to complete shop and compare save up to NUMBER on all types of life insurance hyperlink click here for your free quote protecting your family is the best investment you ll ever make if you are in receipt of this email in error and or wish to be removed from our list hyperlink please click here and type remove if you reside in any state which prohibits e mail solicitations for insurance please disregard this email\n\n"
  },
  {
    "path": "tests/resources/smtp/antispam/subject.test",
    "content": "expect SUBJ_ALL_CAPS\n\nSubject: HELLO WORLD\n\nTest\n<!-- NEXT TEST -->\nexpect LONG_SUBJ\n\nSubject: this is an extremely long subject line that \n    should be truncated to 80 characters and folded\n    in order to be RFC compliant and avoid the SPAM filter\n    that is looking for long subject lines like this one\n    which by the way, it is ridiculously long \n\nTest\n<!-- NEXT TEST -->\nexpect SUBJECT_NEEDS_ENCODING\n\nSubject: thís líné shóúld bé éncódéd\n\nTest\n<!-- NEXT TEST -->\nparam.smtputf8 1\nexpect \n\nSubject: thís líné shóúld bé éncódéd\n\nTest\n<!-- NEXT TEST -->\nparam.8bitmime 1\nexpect \n\nSubject: thís líné shóúld bé éncódéd\n\nTest\n<!-- NEXT TEST -->\nexpect \n\nSubject: =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche?=\n\nTest\n<!-- NEXT TEST -->\nexpect URL_IN_SUBJECT\n\nSubject: check out my url HTTPS://SPAMMER.COM\n\nTest\n<!-- NEXT TEST -->\nexpect URL_IN_SUBJECT\n\nSubject: check out my url HTTP://SPAMMER.COM\n\nTest\n<!-- NEXT TEST -->\nexpect MISSING_SUBJECT\n\nX-Subject: missing subject\n\nTest\n<!-- NEXT TEST -->\nexpect EMPTY_SUBJECT\n\nSubject: \n\nTest\n<!-- NEXT TEST -->\nexpect SUBJ_EXCESS_QP\n\nSubject: =?iso-8859-1?Q?Die_Hasen_und_die_Froesche?=\n\nTest\n<!-- NEXT TEST -->\nexpect SUBJ_EXCESS_BASE64\n\nSubject: =?iso-8859-1?B?RGllIEhhc2VuIHVuIGRpZSBGcm9lc2NoZQ==?=\n\nTest\n<!-- NEXT TEST -->\nexpect FAKE_REPLY\n\nSubject: Re: about your question\n\nTest\n<!-- NEXT TEST -->\nexpect \n\nIn-Reply-To: <id@domain>\nSubject: Re: about your question\n\nTest\n<!-- NEXT TEST -->\nexpect \n\nReferences: <id@domain>\nSubject: Re: about your question\n\nTest\n<!-- NEXT TEST -->\nexpect SUBJECT_ENDS_SPACES\n\nSubject: =?iso-8859-1?Q?Die_Hasen_und_die_Fr=F6sche_?=\n\nTest\n<!-- NEXT TEST -->\nparam.smtputf8 1\nexpect SUBJECT_HAS_CURRENCY SUBJECT_ENDS_EXCLAIM\n\nSubject: You have won £200!\n\nTest\n<!-- NEXT TEST -->\nparam.smtputf8 1\nexpect SUBJECT_HAS_CURRENCY SUBJECT_ENDS_QUESTION\n\nSubject: Have you won $200?\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_IN_SUBJECT\n\nTo: hello@world.org\nSubject: Great offers for hello@world.org\n\nTest\n<!-- NEXT TEST -->\nexpect RCPT_DOMAIN_IN_SUBJECT\n\nTo: hello@world.org\nSubject: Great offers for world.org\n\nTest\n<!-- NEXT TEST -->\nexpect \n\nTo: hello@world.org\nSubject: Question about other@domain.net\n\nTest\n"
  },
  {
    "path": "tests/resources/smtp/antispam/url.test",
    "content": "expect URL_ONLY\n\nSubject: test\n\nhttps://url.org\n<!-- NEXT TEST -->\nexpect \n\nSubject: test\n\nmy site is https://url.org\n<!-- NEXT TEST -->\nexpect SUSPICIOUS_URL\n\nSubject: test\n\nmy site is https://192.168.1.1\n<!-- NEXT TEST -->\nexpect HOMOGRAPH_URL\n\nSubject: test\n\nmy site is https://xn--youtue-tg7b.com\n<!-- NEXT TEST -->\nexpect MIXED_CHARSET_URL\n\nSubject: test\n\nmy site is https://www.xn--1ca81o6aa92e.com/\n<!-- NEXT TEST -->\nexpect UNPARSABLE_URL\n\nSubject: test\n\nlogin to your account at https://bánk.com/\n<!-- NEXT TEST -->\nexpect URL_REDIRECTOR_NESTED REDIRECTOR_URL\n\nSubject: nested redirect\n\nlogin to https://redirect.com/?https://redirect.org/?https://redirect.net/?https://redirect.io/?https://redirect.me/?https://redirect.com\n<!-- NEXT TEST -->\nexpect REDIRECTOR_URL HOMOGRAPH_URL\n\nSubject: redirect to omograph\n\nlogin to https://www.redirect.com/?https://xn--twiter-507b.com\n<!-- NEXT TEST -->\nexpect HAS_ONION_URI HAS_ANON_DOMAIN\n\nSubject: url in title darkweb.onion/login\n\ntest\n<!-- NEXT TEST -->\nexpect HAS_IPFS_GATEWAY_URL HAS_WP_URI URI_HIDDEN_PATH\n\nContent-Type: text/html; charset=\"utf-8\"\nSubject: html test\n\n<link href=\"https://site.com/ipfs/Qm123\">\n<a href=\"https://web.org/../../login.php\"><img src=\"http://site.org/wp-static/img.png\"></a>\n\n<!-- NEXT TEST -->\nexpect HAS_GUC_PROXY_URI HAS_GOOGLE_FIREBASE_URL HAS_GOOGLE_REDIR HAS_ANON_DOMAIN URL_ONLY\n\nContent-Type: text/html; charset=\"utf-8\"\nSubject: mixed urls googleusercontent.com/proxy/url\n\n<a href=\"https://firebasestorage.googleapis.com/content\">google.com/url?otherurl.org</a>\n\n<!-- NEXT TEST -->\nexpect WP_COMPROMISED\n\nSubject: plain test\n\nhttp://url.com/Well-known/../assetlinks.json\nhttp://wp.com/WP-content/content.pdf\n\n<!-- NEXT TEST -->\nexpect HAS_WP_URI\n\nSubject: plain test\n\nhttp://url.com/Well-known/../assetlinks.json\nhttp://wp.com/WP-other/content.pdf\n\n<!-- NEXT TEST -->\nexpect PHISHED_OPENPHISH PHISHED_PHISHTANK\n\nSubject: plain test\n\nhttps://phishing-open.org\nhttps://phishing-tank.com\n\n<!-- NEXT TEST -->\nexpect \n\nSubject: IPs are not urls\n\n192.168.1.1\n\n<!-- NEXT TEST -->\nexpect \n\nContent-Type: text/html; charset=\"utf-8\"\nSubject: IPs in HTML are not urls\n\n<html>\nDas System wurde um 01.01.1970 08:28:00 für die IP-Adresse\n123.123.123.123 gesperrt.<br>\n<br>\nDer Besucher hat versucht, sich mit folgenden Daten anzumelden.<br>\nPartner: 12345678<br>\nPortal: <a href=\"https://www.localhost.de/example.php\" target=\"_blank\">IP-Sperre einsehen</a>\n</html>\n\n<!-- NEXT TEST -->\nexpect RCPT_DOMAIN_IN_BODY\n\nTo: hello@world.com\nSubject: Special offer \n\nAn offer for world.com\n\n"
  },
  {
    "path": "tests/resources/smtp/certs/tls_cert.pem",
    "content": "-----BEGIN CERTIFICATE-----\nMIIFCTCCAvGgAwIBAgIUCgHGQYUqtelbHGVSzCVwBL3fyEUwDQYJKoZIhvcNAQEL\nBQAwFDESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTIyMDUxNjExNDAzNFoXDTIzMDUx\nNjExNDAzNFowFDESMBAGA1UEAwwJbG9jYWxob3N0MIICIjANBgkqhkiG9w0BAQEF\nAAOCAg8AMIICCgKCAgEAtwS0Fzl3SjaCuKEXgZ/fdWbDoj/qDphyNCAKNevQ0+D0\nSTNkWCO04aFSH0zcL8zoD9gokNos0i7OU9//ZhZQmex4V6EFdZn8bFwUWN/scUvW\nHEFXVjtHldO2isZgIxH9LuwRv7KAgkISuWahqerOVDhe7SeQUV0AJGNEh3cT9PZr\ngSY931BxB7n+5k8eoSk8Z1gtBzQzL62kVGpHDKfw8yX8m65owF9eLUBrNzgxmXfC\nxpuHwj7hmVhS09PPKeN/RsFS8PsYO7bo0u8jEKalteumjRT7RyUEbioqfo6ZFOGj\nFHPIq/uKXS9zN1fpoyNh3ur5hMznQhrqlwBM9KlM7GdBJ0pZ3ad0YjT8IL/GnGKR\n85J2WZdLqaQdUZo7nV67FhqdDlNE4MdwiykTMjfmLRXGAVhAzJHKyRKNwmkI2aqe\nS7aqeNgvuDBwY80Q9a2rb5py1Aw+L8yCkUBuHboToDpxSVRDNN8DrWNmmsXnxsOG\nwRDODy4GICKyxlP+RFSM8xWSQ6y9ktS2OfDBm+Eqcw+3pZKhdz2wgxLkUBJ8X1eh\nkJrCA/6LTuhy6m6mMjAfoSOFU7fu88jxaWPgvP7GKyH+LM/t9eucobz2ks5rtSjz\nV4Dc5DCS94/OpVRHwHdaFSPbJKBN9Ev8gnNrAyx/aBPGoHBPG/QUiU7dcUNIPt0C\nAwEAAaNTMFEwHQYDVR0OBBYEFI167IxBmErB11EqiPPqFLa31ZaMMB8GA1UdIwQY\nMBaAFI167IxBmErB11EqiPPqFLa31ZaMMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZI\nhvcNAQELBQADggIBALU00IOiH5ubEauVCmakms5ermNTZfculnhnDfWTLMeh2+a7\nG4cqADErfMhm/mmLbrw33t9s6tCAhQltvewKR40ST9uMPSyiQbYaCXd5DXnuI6Ox\nJtNW+UOWIaMf8abnkdLvREOvb8dVQS1i3xq14tAjY5XgpGwCPP8m54b7N3Q7soLn\ne5PDhPNTnhRIn2RLuYoZmQmMA5fcqEUDYff4epUww7PhrM1QckZligI3566NlGOf\nj1G9JrivBtY0eaJtamIFnGMBT0ThDudxVja2Nv0C2Elry0p4T/o4nc4M67BJ/y1R\nvjNLAgFhbxssemU3lZqSd+pykpJBwDBjFSPrZZmQcbk7H6Uz8V1xr/xuzfw6fA13\nNWZ5vLgP/DQ13sM+XFlxThKfbPMPVe/UCTvfGtNW+3XyBgPntEkR+fNEawQmzbYl\nR+X1ymT9MZnEZqRMf7/UD/SYek1aUJefoew3upjMgxYVvh4F8dqJ+39F+xoFzIA2\n1dDAEMzXtjA3zKhZ2cycZbEzpJvYA3eGLuR16Suqfi4kPvfwK0mOhCxQmpayt7/X\nvuEzW6dPCH8Hgbb0WvsSppGOvhdbDaZFNfFc5eNSxhyKzu3H3ACNImZRtZE+yixx\n0fR8+xz9kDLf8xupV+X9heyFGHSyYU2Lveaevtr2Ij3weLRgJ6LbNALoeKXk\n-----END CERTIFICATE-----\n"
  },
  {
    "path": "tests/resources/smtp/certs/tls_privatekey.pem",
    "content": "-----BEGIN PRIVATE KEY-----\nMIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC3BLQXOXdKNoK4\noReBn991ZsOiP+oOmHI0IAo169DT4PRJM2RYI7ThoVIfTNwvzOgP2CiQ2izSLs5T\n3/9mFlCZ7HhXoQV1mfxsXBRY3+xxS9YcQVdWO0eV07aKxmAjEf0u7BG/soCCQhK5\nZqGp6s5UOF7tJ5BRXQAkY0SHdxP09muBJj3fUHEHuf7mTx6hKTxnWC0HNDMvraRU\nakcMp/DzJfybrmjAX14tQGs3ODGZd8LGm4fCPuGZWFLT088p439GwVLw+xg7tujS\n7yMQpqW166aNFPtHJQRuKip+jpkU4aMUc8ir+4pdL3M3V+mjI2He6vmEzOdCGuqX\nAEz0qUzsZ0EnSlndp3RiNPwgv8acYpHzknZZl0uppB1RmjudXrsWGp0OU0Tgx3CL\nKRMyN+YtFcYBWEDMkcrJEo3CaQjZqp5Ltqp42C+4MHBjzRD1ratvmnLUDD4vzIKR\nQG4duhOgOnFJVEM03wOtY2aaxefGw4bBEM4PLgYgIrLGU/5EVIzzFZJDrL2S1LY5\n8MGb4SpzD7elkqF3PbCDEuRQEnxfV6GQmsID/otO6HLqbqYyMB+hI4VTt+7zyPFp\nY+C8/sYrIf4sz+3165yhvPaSzmu1KPNXgNzkMJL3j86lVEfAd1oVI9skoE30S/yC\nc2sDLH9oE8agcE8b9BSJTt1xQ0g+3QIDAQABAoICABq5oxqpF5RMtXYEgAw7rkPU\nh8jPkHwlIrgd3Z/WGZ53APUXfhWo0ScJiZZsgNKyF0kJBZNxaI4gq5xv3zmnFIoF\nj+Ur7EIqBERGheoceMhqjI9/syMycNeeHM/S/ALjA5ewfT8C7+UVhOpx5DWNxidi\nO+phlp9q9zRZEo69grqIqVYooWxUsMyyCljTQOPDw8BLjfe5VagmsRJqmolslLDM\n4UBSjZVZ18S/3Wgo2oVQia660244BHWCAkZQbbXuNI2+eUAbSoSdxw3WQcaSrywL\nhzyezbqr2yPDIIVuiUgVUt0Ps0P57VCCN07jlYhvCEGnClysFzD+ATefoZ0wg7za\ndQu2E+d166rAjnssyhzcHMn3pxgSdtXD+dQR/xfIGbPABucCupEFqKmhLdMm9+ud\nlHay87qzMpIa8cITJwEQROfXqWAhNUU98pKCOx1SVXBqQC7QVqGQ5solDf0eMSVh\nngQ6Dz2WUI2ty75LteiFwlyTgnU9nyPN0NXsrMEET2BHWre7ufTQqiULtQ7+9BwH\nAMxEKvrQHjMUjdfbXuzdyc5w5mPYJZfFVSQ1HMslx66h9yCpRIsBZvUGvoaP8Tpe\nnQ66FTYRbiOkkdJ7k8DtrnhsJI1oOGjnvj/rvZ8D2pvrlJcIH2AyN3MOL8Jp5Oj1\nnCFt77TwpF92pgl0g9gBAoIBAQDcarmP54QboaIQ9S2gE/4gSVC5i44iDJuSRdI8\nK081RQcWiNzqQXTRc5nqJ7KzLyPiGlg+6rWsBKLos5l4t+MdhhH+KUvk/OtT/g8V\n0NZBNXLIbSb8j8ix4v3/f2qKHN3Co6QOlxb3gFvobKDdoKqUNiSH1zTZ8/Y/BzkM\njqWKhTdaLz6eyzhKfOTA4LO8kJ3VF8HUM1N9/e8Gjorl+gZpJUXUQS0+AIi8W76C\nOwDrVb3BPGVnApQJfWF78h4g20RwXrx/GYUW2vOMcLjXXDV5U7+nobPUoJnLxoZC\n16o88y0Ivan8dBNXsc1epyPvvEqp6MJbAyyVuNeuRJcgYA0BAoIBAQDUkGRV7fLG\nwCr5rNysUO+FKzVtTJnf9KEsqAqUmmVnG4oubxAJJtiB5n2+DT+CtO8Nrtz05BbR\nuxfWm+lbEw6lVMj63bywtp0NdULg7/2t+oq2Svv16KrZIRJttXMkdEiFFmkVAEhX\nl8Fyl6PJPfSMwbPdXEUPUAaNrXweVFffXczHc4W2G212ZzDB0z7QQSgEntbTDFB/\n2Cg5dvuojlM9zw0fuEyLwItZs7n16j/ONZLgBHyroMU9ZPxbnLrVyoZlqtob+RWm\nJu2fSIL9QqG6O4td1TqcUBGvFQYjGvKA+q5fsG26NBJ0Ac48cNK6PS4lMkN3Av2J\nccloYaMEHAXdAoIBAE8WMCy1Ok6byUXiYxOL+OPmyoM40q/e7DcovE2AkLQhZ3Cr\nfPDEucCphPFiexkV8f8fysgQeU0WgMmUH54UBPbD81LJyISKR3nkr875Ftdg8SV/\nHL0EblN9ifuR4U1bHCrJgoUFq2T09oVH7NR44Ju7bZIcIseNZK6qzcp2qGkycXD3\ngLWDX1hCxeV6+qLPFQKvuomEPRH4+jnVDXuFIaW6jPqixDP6BxXmqU2bFDJcmnBq\nVkwGvc1F4qORdUP+yOi05VeJdZqEx1x92aTUXg+BgEQKnjbNxUE7o1L6hQfHjUIU\no5iEoagWkQTEXf2YBwY+EPaNBgNWxnSuAbfJHwECggEBALOF95ezTVWauzD/U6ic\n+o3n/kl/Zn4FJ5KFodn7xCSe18d7uXlhO34KYqx+l+MWWMefpbGWacdcUjfImf93\nSulLgCqP12sP7/iLzp4XUpL7hOeM0NvRU2nqSpwpoUNqik0Mrlc0U+TWoGTduVCf\naMjwV65e3VyfY8mIeclLxqM5n1fcM1OoOnzDjiRE+0n7nYa5eAnq3pn6v4449TZY\nbelH03e0ucFWLtrltesBmj3YdWGJqJlzQOInRhNBfXJOh8+ZynfRmP0o54udPDQV\ncG3PGFd5XPTjkuvhv7sqaSGRlm/um92lWOhtFfdp+i+cuDpmByCef+7zEP19aKZx\n3GkCggEAFTs7KNMfvIEaLH0yQUFeq2gLmtcMofmOmeoIECycN1rG7iJo07lJLIs0\nbVODH8Z0kX8llu3cjGMAH/6R2uugJSxkmFiZKrngTzKmxDPvTCKWR4RFwXH9j8IO\ncPq7FtKN4SgrPy9ciAPdkcGmu3zz/sBKOaoPwvU2PdBRT+v/aoz+GCLXAvzFlKVe\n9/7zdg87ilo8+AtV+71EJeR3kyBPKS9JrWYUKfiams12+uuH4/53rMFZfNCAaZ3Z\n1sdXEO4o3Loc5TX4DbO9FVdBSBe6klEXx4T0QJboO6uBvTBnnRL2SQriJQQFwYT6\nXzVV5pwOxkIDBWDIqMUfwJDChBKfpw==\n-----END PRIVATE KEY-----\n"
  },
  {
    "path": "tests/resources/smtp/config/if-blocks.toml",
    "content": "durations = [\n    {if = \"sender = 'jdoe'\", then = \"5d\"},\n    {if = \"priority = -1 | starts_with(rcpt, 'jane')\", then = \"1h\"},\n    {else = false}\n]\n\nstring-list = [\n    {if = \"sender = 'jdoe'\", then = \"['From', 'To', 'Date']\"},\n    {if = \"priority = -1 | starts_with(rcpt, 'jane')\", then = \"'Other-ID'\"},\n    {else = \"[]\"}\n]\n\nstring-list-bis = [\n    {if = \"sender = 'jdoe'\", then = \"['From', 'To', 'Date']\"},\n    {if = \"priority = -1 | starts_with(rcpt, 'jane')\", then = \"[]\"},\n    {else = \"['ID-Bis']\"}\n]\n\nsingle-value = \"'hello world'\"\n\nbad-if-without-then = [\n    {if = \"sender = 'jdoe'\"},\n    {else = 1}\n]\n\nbad-if-without-else = [\n    {if = \"sender = 'jdoe'\", then = 1}\n]\n\nbad-multiple-else = [\n    {if = \"sender = 'jdoe'\", then = 1},\n    {else = 1},\n    {else = 2}\n]\n"
  },
  {
    "path": "tests/resources/smtp/config/lists.toml",
    "content": "[list]\nlocal-domains = [\"example.org\", \"example.net\"]\nspammer-domains = \"thatdomain.net\"\nlocal-users = \"file://{LIST1}\"\npower-users = [\"file://{LIST1}\", \"file://{LIST2}\"]\n\n[remote.\"lmtp\"]\naddress = 192.168.0.1\nport = 25\nprotocol = \"lmtp\"\nlookup = true\n\n[remote.\"lmtp\".auth]\nusername = \"hello\"\nsecret = \"world\"\n\n[remote.\"lmtp\".tls]\nimplicit = true\nallow-invalid-certs = true\n"
  },
  {
    "path": "tests/resources/smtp/config/rules-dynvalue.toml",
    "content": "\n[envelope]\nrcpt-domain = \"foo.example.org\"\nrcpt = \"user@foo.example.org\"\nsender-domain = \"foo.net\"\nsender = \"bill@foo.net\"\nlocal-ip = \"192.168.9.3\"\nremote-ip = \"A:B:C::D:E\"\nmx = \"mx.somedomain.com\"\nauthenticated-as = \"john@foobar.org\"\npriority = -4\nlistener = \"smtp\"\nhelo-domain = \"hi-domain.net\"\n\n[eval.\"eq\"]\ntest = [\n    {if = \"sender = 'bill@foo.net'\", then = \"sender\"},\n    {else = false}\n]\nexpect = \"bill@foo.net\"\n\n[eval.\"starts-with\"]\ntest = [\n    {if = \"starts_with(rcpt_domain, 'foo')\", then = \"'mx.' + rcpt_domain\"},\n    {else = false}\n]\nexpect = \"mx.foo.example.org\"\n\n[eval.\"regex\"]\ntest = [\n    {if = \"matches('^([^.]+)@([^.]+)\\.(.+)$', rcpt)\", then = \"$1 + '+' + $2 + '@' + $3\"},\n    {else = false}\n]\nexpect = \"user+foo@example.org\"\n\n[eval.\"regex-full\"]\ntest = [\n    {if = \"matches('^([^.]+)@([^.]+)\\.(.+)$', rcpt)\", then = \"rcpt\"},\n    {else = false}\n]\nexpect = \"user@foo.example.org\"\n\n[eval.\"envelope-match\"]\ntest = [\n    {if = \"matches('^([^.]+)@(.+)$', authenticated_as)\", then = \"'rcpt ' + rcpt + ' listener ' + listener + ' ip ' + local_ip + ' priority ' + priority\"},\n    {else = false}\n]\nexpect = \"rcpt user@foo.example.org listener smtp ip 192.168.9.3 priority -4\"\n\n[eval.\"static-match\"]\ntest = [\n    {if = \"matches('^([^.]+)@(.+)$', authenticated_as)\", then = \"'hello world'\"},\n    {else = false}\n]\nexpect = \"hello world\"\n\n[eval.\"no-match\"]\ntest = [\n    {if = \"matches('^([^.]+)@([^.]+)\\.(.+)$org', authenticated_as)\", then = \"'test'\"},\n    {else = false}\n]\nexpect = false\n\n"
  },
  {
    "path": "tests/resources/smtp/config/rules-eval.toml",
    "content": "[envelope]\nrcpt-domain = \"example.org\"\nrcpt = \"user@example.org\"\nsender-domain = \"foo.net\"\nsender = \"bill@foo.net\"\nlocal-ip = \"192.168.9.3\"\nremote-ip = \"A:B:C::D:E\"\nmx = \"mx.somedomain.com\"\nauthenticated-as = \"john@foobar.org\"\npriority = -4\nlistener = \"smtp\"\nhelo-domain = \"hi-domain.net\"\n\n[rule]\n\"eq-true\" = \"rcpt_domain = 'example.org'\"\n\"eq-false\" = \"rcpt_domain = 'example.com'\"\n\"listener-eq-true\" = \"listener = 'smtp'\"\n\"listener-eq-false\" = \"listener = 'smtps'\"\n\"ip-eq-true\" = \"local_ip = '192.168.9.3'\"\n\"ip-eq-false\" = \"remote_ip = 'A:B:C::D:E'\"\n\"ne-true\" = \"!is_empty(authenticated_as)\"\n\"ne-false\" = \"authenticated_as != 'john@foobar.org'\"\n\"starts-with-true\" = \"starts_with(mx, 'mx.some')\"\n\"starts-with-false\" = \"starts_with(mx, 'enchilada')\"\n\"ends-with-true\" = \"ends_with(sender, '@foo.net')\"\n\"ends-with-false\" = \"ends_with(sender, 'chimichanga')\"\n\"regex-true\" = \"matches('^(.+)@(.+)$', sender)\"\n\"regex-false\" = \"matches('/^\\\\S+@\\\\S+\\\\.\\\\S+$/', mx)\"\n\"any-of-true\" = \"authenticated_as != 'john@foobar.org' | rcpt_domain = 'example.org' | starts_with(mx, 'mx.some')\"\n\"any-of-false\" = \"authenticated_as = 'something else' | rcpt_domain = 'something else' | starts_with(mx, 'something else')\"\n\"all-of-true\" =  \"rcpt_domain = 'example.org' & listener = 'smtp' & starts_with(mx, 'mx.some')\"\n\"all-of-false\" = \"rcpt_domain = 'example.org' & listener = 'smtp' & starts_with(mx, 'something else')\"\n\"none-of-true\" = \"!(authenticated_as = 'something else' | rcpt_domain = 'something else' | starts_with(mx, 'something else'))\"\n\"none-of-false\" = \"!(rcpt_domain = 'example.org' | listener = 'smtp' | starts_with(mx, 'mx.some'))\"\n"
  },
  {
    "path": "tests/resources/smtp/config/rules.toml",
    "content": "[rule]\n\"my-nested-rule\" = { any-of = [\n    {if = \"rcpt-domain\", eq = \"example.org\"},\n    {if = \"remote-ip\", eq = \"192.168.0.0/24\"},\n    {all-of = [\n        {if = \"rcpt\", starts-with = \"no-reply@\"},\n        {if = \"sender\", ends-with = \"@domain.org\"},\n        {none-of = [\n            {if = \"priority\", eq = 1},\n            {if = \"priority\", ne = -2},\n        ]}\n    ]}\n]}\n\n[rule.\"simple\"]\nif = \"listener\"\neq = \"smtp\"\n\n[rule.\"is-authenticated\"]\nif = \"authenticated-as\"\nne = \"\"\n\n[[rule.\"expanded\".all-of]]\nif = \"sender-domain\"\nstarts-with = \"example\"\n\n[[rule.\"expanded\".all-of]]\nif = \"sender\"\nin-list = \"test-list\"\n"
  },
  {
    "path": "tests/resources/smtp/config/servers.toml",
    "content": "[server]\nhostname = \"mx.example.org\"\ngreeting = \"Stalwart SMTP - hi there!\"\n\n[server.listener.\"smtp\"]\nbind = [\"127.0.0.1:9925\"]\nprotocol = \"smtp\"\ntls.implicit = false\n\n[server.listener.\"smtps\"]\nbind = [\"127.0.0.1:9465\", \"127.0.0.1:9466\"]\nprotocol = \"smtp\"\nmax-connections = 1024\ntls.implicit = true\ntls.ciphers = [\"TLS13_CHACHA20_POLY1305_SHA256\", \"TLS13_AES_256_GCM_SHA384\"]\nsocket.ttl = 4096\n\n[server.listener.\"submission\"]\ngreeting = \"Stalwart SMTP submission at your service\"\nprotocol = \"smtp\"\nhostname = \"submit.example.org\"\nbind = \"127.0.0.1:9991\"\n#tls.sni = [{subject = \"submit.example.org\", certificate = \"other\"},\n#           {subject = \"submission.example.org\", certificate = \"other\"}]\nsocket.backlog = 2048\n\n[server.tls]\nenable = true\nimplicit = true\ntimeout = 300\ncertificate = \"default\"\n#sni = [{subject = \"other.domain.org\", certificate = \"default\"}]\nprotocols = [\"TLSv1.2\", \"TLSv1.3\"]\nciphers = []\nignore_client_order = true\n\n[server.socket]\nreuse-addr = true\nreuse-port = true\nbacklog = 1024\nttl = 3600\nsend-buffer-size = 65535\nrecv-buffer-size = 65535\nlinger = 1\ntos = 1\n\n[certificate.\"default\"]\ncert = \"%{file:{CERT}}%\"\nprivate-key = \"%{file:{PK}}%\"\n\n[certificate.\"other\"]\ncert = \"%{file:{CERT}}%\"\nprivate-key = \"%{file:{PK}}%\"\n"
  },
  {
    "path": "tests/resources/smtp/config/throttle.toml",
    "content": "[[throttle]]\nmatch = \"remote_ip == '127.0.0.1'\"\nkey = [\"remote_ip\", \"authenticated_as\"]\nrate = \"50/30s\"\nenable = true\n\n[[throttle]]\nkey = \"sender_domain\"\nrate = \"50/30s\"\nenable = true\n\n"
  },
  {
    "path": "tests/resources/smtp/config/toml-parser.toml",
    "content": "[database]\nenabled = true # ignore\nports = [ 8000, 8001, 8002 ] # ignore\ndata = [ [\"delta\", \"phi\"], [3.14] ]\ntemp_targets = { cpu = 79.5, case = 72.0 }\n\n[servers]\n\"127.0.0.1\" = \"value\" # ignore\n\"character encoding\" = \"value\"\n\n[servers.alpha]\nip = \"10.0.0.1\"\nrole = \"frontend\"\n\n[servers.beta]\nip = \"10.0.0.2\"\nrole = \"backend\"\n\n[[products]]\nname = \"Hammer\"\nsku = 738594937\n\n[[products]]  # empty table within the array\n\n[[products]] # ignore\nname = \"Nail\"\nsku = 284758393 # ignore\ncolor = \"gray\"\n\n[strings.\"my \\\"string\\\" test\"]\nstr1 = \"I'm a string.\"\nstr2 = \"You can \\\"quote\\\" me.\"\nstr3 = \"Name\\tTabs\\nNew Line.\"\nlines = '''\nThe first newline is\ntrimmed in raw strings.\nAll other whitespace\nis preserved.\n'''\n\n[sets]\ninteger = { 1 }\nintegers = { 1, 2, 3 }\nstring = { \"red\" }\nstrings = { \"red\", \"yellow\", \"green\" }\n\n[arrays]\nintegers = [ 1, 2, 3 ]\ncolors = [ \"red\", \"yellow\", \"green\" ]\nnested_arrays_of_ints = [ [ 1, 2 ], [3, 4, 5] ]\nnested_mixed_array = [ [ 1, 2 ], [\"a\", \"b\", \"c\"] ]\nstring_array = [ \"all\", 'strings', \"\"\"are the same\"\"\", '''type''' ]\n\n# Mixed-type arrays are allowed\nnumbers = [ 0.1, 0.2, 0.5, 1, 2, 5 ]\nintegers2 = [\n  1, 2, 3 # this is ok\n]\nintegers3 = [\n  4,\n  # comment in the middle\n  5, # this is ok\n]\ncontributors = [\n  \"Foo Bar <foo@example.com>\" ,\n  { name = \"Baz Qux\", email = \"bazqux@example.com\", url = \"https://example.com/bazqux\" }\n]\n\n[env]\nvar1 = !CARGO_PKG_NAME\nvar2 = !CARGO_PKG_NAME #comment\n"
  },
  {
    "path": "tests/resources/smtp/dane/dns.txt",
    "content": "_25._tcp.internet.nl 2 1 1 E1AE9C3DE848ECE1BA72E0D991AE4D0D9EC547C6BAD1DDDAB9D6BEB0A7E0E0D8\n_25._tcp.internet.nl 3 1 1 D6FEA64D4E68CAEAB7CBB2E0F905D7F3CA3308B12FD88C5B469F08AD7E05C7C7\n_25._tcp.mail.ietf.org 3 1 1 0C72AC70B745AC19998811B131D662C9AC69DBDBE7CB23E5B514B56664C5D3D6\n"
  },
  {
    "path": "tests/resources/smtp/dsn/delay.eml",
    "content": "From: \"Mail Delivery Subsystem\" <MAILER-DAEMON@example.org>\r\nTo: sender@foobar.org\r\nAuto-Submitted: auto-generated\r\nSubject: Warning: Delay in message delivery\r\nMIME-Version: 1.0\r\nContent-Type: multipart/report; report-type=\"delivery-status\"; \r\n\tboundary=\"mime_boundary\"\r\n\r\n\r\n--mime_boundary\r\nContent-Type: text/plain; charset=\"utf-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nThere was a temporary problem delivering your message to the following recip=\r\nients:\r\n\r\n<john.doe@example.org> (connection to 'mx.domain.org' failed: Connection tim=\r\neout)\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/delivery-status; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nReporting-MTA: dns;mx.example.org\r\nArrival-Date: <date goes here>\r\n\r\nOriginal-Recipient: rfc822;jdoe@example.org\r\nFinal-Recipient: rfc822;john.doe@example.org\r\nAction: delayed\r\nStatus: 4.0.0\r\nRemote-MTA: dns;mx.domain.org\r\nWill-Retry-Until: <date goes here>\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/rfc822; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nDisclose-recipients: prohibited\r\nFrom: Message Router Submission Agent <AMMGR@corp.timeplex.com>\r\nSubject: Status of: Re: Battery current sense\r\nTo: owner-ups-mib@CS.UTK.EDU\r\nMessage-id: <01HEGJ0WNBY28Y95LN@mr.timeplex.com>\r\nMIME-version: 1.0\r\nContent-Type: text/plain\r\n\r\n\r\n--mime_boundary--\r\n\r\n"
  },
  {
    "path": "tests/resources/smtp/dsn/failure.eml",
    "content": "From: \"Mail Delivery Subsystem\" <MAILER-DAEMON@example.org>\r\nTo: sender@foobar.org\r\nAuto-Submitted: auto-generated\r\nSubject: Failed to deliver message\r\nMIME-Version: 1.0\r\nContent-Type: multipart/report; report-type=\"delivery-status\"; \r\n\tboundary=\"mime_boundary\"\r\n\r\n\r\n--mime_boundary\r\nContent-Type: text/plain; charset=\"utf-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nYour message could not be delivered to the following recipients:\r\n\r\n<foobar@example.org> (host 'mx.example.org' rejected command 'RCPT TO:<fooba=\r\nr@example.org>' with code 550 (5.1.2) 'User does not exist')\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/delivery-status; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nReporting-MTA: dns;mx.example.org\r\nArrival-Date: <date goes here>\r\n\r\nFinal-Recipient: rfc822;foobar@example.org\r\nAction: failed\r\nStatus: 5.1.2\r\nDiagnostic-Code: smtp;550 User does not exist\r\nRemote-MTA: dns;mx.example.org\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/rfc822; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nDisclose-recipients: prohibited\r\nFrom: Message Router Submission Agent <AMMGR@corp.timeplex.com>\r\nSubject: Status of: Re: Battery current sense\r\nTo: owner-ups-mib@CS.UTK.EDU\r\nMessage-id: <01HEGJ0WNBY28Y95LN@mr.timeplex.com>\r\nMIME-version: 1.0\r\nContent-Type: text/plain\r\n\r\n\r\n--mime_boundary--\r\n\r\n"
  },
  {
    "path": "tests/resources/smtp/dsn/mixed.eml",
    "content": "From: \"Mail Delivery Subsystem\" <MAILER-DAEMON@example.org>\r\nTo: sender@foobar.org\r\nAuto-Submitted: auto-generated\r\nSubject: Partially delivered message\r\nMIME-Version: 1.0\r\nContent-Type: multipart/report; report-type=\"delivery-status\"; \r\n\tboundary=\"mime_boundary\"\r\n\r\n\r\n--mime_boundary\r\nContent-Type: text/plain; charset=\"utf-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nYour message has been partially delivered:\r\n\r\n    ----- Delivery to the following addresses was successful -----\r\n<jane@example.org> (delivered to 'mx2.example.org' with code 250 (2.1.5) 'Me=\r\nssage accepted for delivery')\r\n\r\n    ----- There was a temporary problem delivering to these addresses -----\r\n<john.doe@example.org> (connection to 'mx.domain.org' failed: Connection tim=\r\neout)\r\n\r\n    ----- Delivery to the following addresses failed -----\r\n<foobar@example.org> (host 'mx.example.org' rejected command 'RCPT TO:<fooba=\r\nr@example.org>' with code 550 (5.1.2) 'User does not exist')\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/delivery-status; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nReporting-MTA: dns;mx.example.org\r\nArrival-Date: <date goes here>\r\n\r\nFinal-Recipient: rfc822;foobar@example.org\r\nAction: failed\r\nStatus: 5.1.2\r\nDiagnostic-Code: smtp;550 User does not exist\r\nRemote-MTA: dns;mx.example.org\r\n\r\nFinal-Recipient: rfc822;jane@example.org\r\nAction: delivered\r\nStatus: 2.1.5\r\nRemote-MTA: dns;mx2.example.org\r\n\r\nOriginal-Recipient: rfc822;jdoe@example.org\r\nFinal-Recipient: rfc822;john.doe@example.org\r\nAction: delayed\r\nStatus: 4.0.0\r\nRemote-MTA: dns;mx.domain.org\r\nWill-Retry-Until: <date goes here>\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/rfc822; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nDisclose-recipients: prohibited\r\nFrom: Message Router Submission Agent <AMMGR@corp.timeplex.com>\r\nSubject: Status of: Re: Battery current sense\r\nTo: owner-ups-mib@CS.UTK.EDU\r\nMessage-id: <01HEGJ0WNBY28Y95LN@mr.timeplex.com>\r\nMIME-version: 1.0\r\nContent-Type: text/plain\r\n\r\n\r\n--mime_boundary--\r\n\r\n"
  },
  {
    "path": "tests/resources/smtp/dsn/original.txt",
    "content": "Disclose-recipients: prohibited\nDate: Fri, 08 Jul 1994 09:21:25 -0400 (EDT)\nFrom: Message Router Submission Agent <AMMGR@corp.timeplex.com>\nSubject: Status of: Re: Battery current sense\nTo: owner-ups-mib@CS.UTK.EDU\nMessage-id: <01HEGJ0WNBY28Y95LN@mr.timeplex.com>\nMIME-version: 1.0\nContent-Type: text/plain\n\n\n"
  },
  {
    "path": "tests/resources/smtp/dsn/success.eml",
    "content": "From: \"Mail Delivery Subsystem\" <MAILER-DAEMON@example.org>\r\nTo: sender@foobar.org\r\nAuto-Submitted: auto-generated\r\nSubject: Successfully delivered message\r\nMIME-Version: 1.0\r\nContent-Type: multipart/report; report-type=\"delivery-status\"; \r\n\tboundary=\"mime_boundary\"\r\n\r\n\r\n--mime_boundary\r\nContent-Type: text/plain; charset=\"utf-8\"\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\nYour message has been successfully delivered to the following recipients:\r\n\r\n<jane@example.org> (delivered to 'mx2.example.org' with code 250 (2.1.5) 'Me=\r\nssage accepted for delivery')\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/delivery-status; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nReporting-MTA: dns;mx.example.org\r\nArrival-Date: <date goes here>\r\n\r\nFinal-Recipient: rfc822;jane@example.org\r\nAction: delivered\r\nStatus: 2.1.5\r\nRemote-MTA: dns;mx2.example.org\r\n\r\n\r\n--mime_boundary\r\nContent-Type: message/rfc822; charset=\"utf-8\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nDisclose-recipients: prohibited\r\nFrom: Message Router Submission Agent <AMMGR@corp.timeplex.com>\r\nSubject: Status of: Re: Battery current sense\r\nTo: owner-ups-mib@CS.UTK.EDU\r\nMessage-id: <01HEGJ0WNBY28Y95LN@mr.timeplex.com>\r\nMIME-version: 1.0\r\nContent-Type: text/plain\r\n\r\n\r\n--mime_boundary--\r\n\r\n"
  },
  {
    "path": "tests/resources/smtp/lists/test-list1.txt",
    "content": "user1@domain.org\nuser2@domain.org\n"
  },
  {
    "path": "tests/resources/smtp/lists/test-list2.txt",
    "content": "user3@example.net\nuser4@example.net\nuser5@example.net\n"
  },
  {
    "path": "tests/resources/smtp/messages/arc.eml",
    "content": "ARC-Seal: i=2; a=rsa-sha256; s=rsa; d=manchego.org; cv=pass;\n        b=wpAAy6QusmF4O8SeziNaKxXL6EleeBYxQ0HrXl2cDgzHLOvYG0N1Wpz0bpVbA8VgteD2X8XCW\n        yrdlZ5dIPTcCvgfLGLXLRTIcYUdKyfFh5IVEciaUOUsxlSRPpekENZKzdHFkL4j1mAAvpDNJ7Ft\n        OFIp0ku5dACn80g7D4cSEU0=;\nARC-Message-Signature: i=2; a=rsa-sha256; s=rsa; d=manchego.org; c=relaxed/relaxed;\n        h=Subject:To:From:DKIM-Signature; t=1674137914; bh=4ET7siw2kYV7jcN+fzsuYng/\n        sr/BmIzzEjh43dVAv40=; b=V3tMBI1RsyJJY7HUABcebHf0mDJ9odbPm++ZMY5AsCaUYNoSsAm\n        wCf5wYlJQ26KmsluOYXoPwML0a/xvnMXPv6Rs4Z9k4IwzpzhGLsijDXymGPsW3hgq/6ivVTPkwU\n        +pGSCC70rHNrAFFk5P67Ly0tbGYjJ0wZVHBzqL8IJBXK4=;\nARC-Authentication-Results: i=2; manchego.org;\n        dkim=pass header.d=manchego.org header.s=rsa header.b=IN4oMvqq\nAuthentication-Results: manchego.org;\n        dkim=pass header.d=manchego.org header.s=rsa header.b=IN4oMvqq\nARC-Seal: i=1; a=ed25519-sha256; s=ed; d=scamorza.org; cv=none;\n        b=k/MAHECtaer9v4oczoe00a6XMjrxU4QUVVPlZI8XYegbiOgDSaeR6IrwBSKVcN0ELYU+HXlNW\n        RuUGkRuZXQODA==;\nARC-Message-Signature: i=1; a=ed25519-sha256; s=ed; d=scamorza.org; c=relaxed/relaxed;\n        h=Subject:To:From:DKIM-Signature; t=1674137914; bh=4ET7siw2kYV7jcN+fzsuYng/\n        sr/BmIzzEjh43dVAv40=; b=ZVPqB/5+mbOEKIgBsq+S71Sfj2JZUlGmYEA0Ygbj0S1VmTAnsVu\n        FQSInMY4/qcIeqU23BtzMgCFVZfAg5i3zDw==;\nARC-Authentication-Results: i=1; scamorza.org;\n        dkim=pass header.d=manchego.org header.s=rsa header.b=IN4oMvqq\nAuthentication-Results: scamorza.org;\n        dkim=pass header.d=manchego.org header.s=rsa header.b=IN4oMvqq\nDKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=manchego.org; c=relaxed/relaxed;\n        h=Subject:To:From; t=1674137914; bh=4ET7siw2kYV7jcN+fzsuYng/sr/BmIzzEjh43dV\n        Av40=; b=IN4oMvqqxWCEyC38F7fZecYJcnq+7zP3G/xjcI64M3/Dzys2lmQeLYAXipwwYvEa5a\n        VwCcJ7XUX0kSxtr6igC8FIJEDI6UmdvJgMEj/hnEjXR8m4GPrphigjJy7hagaQymBT9WhlzsDPI\n        QRlUVoW0y5v1aDp3KF9bLVCKTELJPM=;\nFrom: queso@manchego.org\nTo: affumicata@scamorza.org\nSubject: Say cheese\n\nWe need to settle which one of us is tastier.\n"
  },
  {
    "path": "tests/resources/smtp/messages/dkim.eml",
    "content": "DKIM-Signature: v=1; a=rsa-sha256; s=default; d=example.com; c=relaxed/relaxed; r=y;\n        h=Subject:To:From; t=1674122129; bh=Xcxymouf0VhlJ7c/vHLAM3LPTUR4LKFKX7PRNni\n        WCEs=; b=m5lYqx81xqAIo4ZBC9FDiFIBrRnnep+taSsutc5MkbBQvf9/Lb54AXHhruEdO2EGkG\n        xUxL1c8QDH3eLz84fPTUgZue84tAsAa0q4gJFIYM5q2/GJvJ6cBvsXKZj82FjRTIz4wlLjzkW7p\n        NdR9C5CID2PO4sW+GymS45F8hwqSj4=;\nDKIM-Signature: v=1; a=ed25519-sha256; s=ed; d=example.com; c=relaxed/relaxed;\n        h=Subject:To:From; t=1674122129; bh=Xcxymouf0VhlJ7c/vHLAM3LPTUR4LKFKX7PRNni\n        WCEs=; b=t8z1AsaxeWek+gMSVojbs2QJu+orzeR4CiHVquJYvXzv+Eb52Wq0fEmaOxoyY1teVL\n        Odp57Vq/zTLjMMZ2hbBQ==;\nFrom: bill@example.com\nTo: jdoe@example.com\nSubject: TPS Report\n\nI'm going to need those TPS reports ASAP. So, if you could do that, that'd be great.\n"
  },
  {
    "path": "tests/resources/smtp/messages/invalid_arc.eml",
    "content": "ARC-Seal: i=1; a=rsa-sha256; s=rsa; d=manchego.org; cv=fail;\n        b=wpAAy6QusmF4O8SeziNaKxXL6EleeBYxQ0HrXl2cDgzHLOvYG0N1Wpz0bpVbA8VgteD2X8XCW\n        yrdlZ5dIPTcCvgfLGLXLRTIcYUdKyfFh5IVEciaUOUsxlSRPpekENZKzdHFkL4j1mAAvpDNJ7Ft\n        OFIp0ku5dACn80g7D4cSEU0=;\nARC-Message-Signature: i=1; a=rsa-sha256; s=rsa; d=manchego.org; c=relaxed/relaxed;\n        h=Subject:To:From:DKIM-Signature; t=1674137914; bh=4ET7siw2kYV7jcN+fzsuYng/\n        sr/BmIzzEjh43dVAv40=; b=V3tMBI1RsyJJY7HUABcebHf0mDJ9odbPm++ZMY5AsCaUYNoSsAm\n        wCf5wYlJQ26KmsluOYXoPwML0a/xvnMXPv6Rs4Z9k4IwzpzhGLsijDXymGPsW3hgq/6ivVTPkwU\n        +pGSCC70rHNrAFFk5P67Ly0tbGYjJ0wZVHBzqL8IJBXK4=;\nARC-Authentication-Results: i=1; manchego.org;\n        dkim=pass header.d=manchego.org header.s=rsa header.b=IN4oMvqq\nDKIM-Signature: v=1; a=rsa-sha256; s=default; d=example.com; c=relaxed/relaxed; r=y;\n        h=Subject:To:From; t=1674122129; bh=Xcxymouf0VhlJ7c/vHLAM3LPTUR4LKFKX7PRNni\n        WCEs=; b=m5lYqx81xqAIo4ZBC9FDiFIBrRnnep+taSsutc5MkbBQvf9/Lb54AXHhruEdO2EGkG\n        xUxL1c8QDH3eLz84fPTUgZue84tAsAa0q4gJFIYM5q2/GJvJ6cBvsXKZj82FjRTIz4wlLjzkW7p\n        NdR9C5CID2PO4sW+GymS45F8hwqSj4=;\nDKIM-Signature: v=1; a=ed25519-sha256; s=ed; d=example.com; c=relaxed/relaxed;\n        h=Subject:To:From; t=1674122129; bh=Xcxymouf0VhlJ7c/vHLAM3LPTUR4LKFKX7PRNni\n        WCEs=; b=t8z1AsaxeWek+gMSVojbs2QJu+orzeR4CiHVquJYvXzv+Eb52Wq0fEmaOxoyY1teVL\n        Odp57Vq/zTLjMMZ2hbBQ==;\nFrom: bill@example.com\nTo: jdoe@example.com\nSubject: TPS Report\n\nI'm going to need those TPS reports ASAP. So, if you could do that, that'd be great.\n"
  },
  {
    "path": "tests/resources/smtp/messages/invalid_dkim.eml",
    "content": "DKIM-Signature: v=1; a=rsa-sha256; s=default; d=example.com; c=relaxed/relaxed; r=y;\n        h=Subject:To:From; t=1674122129; bh=Xcxymouf0VhlJ7c/vHLAM3LPTUR4LKFKX7PRNni\n        WCEs=; b=m5lYqx81xqAIo4ZBC9FDiFIBrRnnep+taSsutc5MkbBQvf9/Lb54AXHhruEdO2EGkG\n        xUxL1c8QDH3eLz84fPTUgZue84tAsAa0q4gJFIYM5q2/GJvJ6cBvsXKZj82FjRTIz4wlLjzkW7p\n        NdR9C5CID2PO4sW+GymS45F8hwqSj4=;\nDKIM-Signature: v=1; a=ed25519-sha256; s=ed; d=example.com; c=relaxed/relaxed;\n        h=Subject:To:From; t=1674122129; bh=Xcxymouf0VhlJ7c/vHLAM3LPTUR4LKFKX7PRNni\n        WCEs=; b=t8z1AsaxeWek+gMSVojbs2QJu+orzeR4CiHVquJYvXzv+Eb52Wq0fEmaOxoyY1teVL\n        Odp57Vq/zTLjMMZ2hbBQ==;\nFrom: bill@example.com\nTo: jdoe@example.com\nSubject: TPS Report\n\nBody hash will not match.\n"
  },
  {
    "path": "tests/resources/smtp/messages/loop.eml",
    "content": "Received: from client1.football.example.com  [192.0.2.1]\n      by submitserver.example.com with SUBMISSION;\n      Fri, 11 Jul 2003 21:01:54 -0700 (PDT)\nReceived: from client1.football.example.com  [192.0.2.1]\n      by submitserver.example.com with SUBMISSION;\n      Fri, 11 Jul 2003 21:01:54 -0700 (PDT)\nReceived: from client1.football.example.com  [192.0.2.1]\n      by submitserver.example.com with SUBMISSION;\n      Fri, 11 Jul 2003 21:01:54 -0700 (PDT)\nReceived: from client1.football.example.com  [192.0.2.1]\n      by submitserver.example.com with SUBMISSION;\n      Fri, 11 Jul 2003 21:01:54 -0700 (PDT)\nFrom: Joe SixPack <joe@football.example.com>\nTo: Suzie Q <suzie@shopping.example.net>\nSubject: Is dinner ready?\nDate: Fri, 11 Jul 2003 21:00:37 -0700 (PDT)\nMessage-ID: <20030712040037.46341.5F8J@football.example.com>\n\nHi.\n\nWe lost the game. Are you hungry yet?\n\nJoe.\n"
  },
  {
    "path": "tests/resources/smtp/messages/multipart.eml",
    "content": "From: Hendrik <hendrik@example.com>\nTo: Harrie <harrie@example.com>\nDate: Sat, 11 Oct 2010 00:31:44 +0200\nSubject: One Two Three Four\nContent-Type: multipart/mixed; boundary=AA\n\nThis is a multi-part message in MIME format.\n--AA\nContent-Type: multipart/mixed; boundary=BB\n\nThis is a multi-part message in MIME format.\n--BB\nContent-Type: text/plain; charset=\"us-ascii\"\n\nThis is the first message part containing\nplain text. \n\n--BB\nContent-Type: text/plain; charset=\"us-ascii\"\n\nThis is another plain text message part.\n\n--BB--\nThis is the end of MIME multipart.\n\n--AA\nContent-Type: text/html; charset=\"us-ascii\"\n\n<html>\n<body>This is a piece of HTML text.</body>\n</html>\n\n--AA--\nThis is the end of  MIME multipart.\n"
  },
  {
    "path": "tests/resources/smtp/messages/no_dkim.eml",
    "content": "Received: from client1.football.example.com  [192.0.2.1]\n      by submitserver.example.com with SUBMISSION;\n      Fri, 11 Jul 2003 21:01:54 -0700 (PDT)\nFrom: Joe SixPack <joe@football.example.com>\nTo: Suzie Q <suzie@shopping.example.net>\nSubject: Is dinner ready?\nDate: Fri, 11 Jul 2003 21:00:37 -0700 (PDT)\nMessage-ID: <20030712040037.46341.5F8J@football.example.com>\n\nHi.\n\nWe lost the game. Are you hungry yet?\n\nJoe.\n"
  },
  {
    "path": "tests/resources/smtp/messages/no_msgid.eml",
    "content": "From: Joe SixPack <joe@football.example.com>\nTo: Suzie Q <suzie@shopping.example.net>\nSubject: Is dinner ready?\n\nHi.\n\nWe lost the game. Are you hungry yet?\n\nJoe.\n"
  },
  {
    "path": "tests/resources/smtp/milter/message.eml",
    "content": "From: John Doe <john@example.org>\r\nTo: Mary Smith <mary.smith@example.org>\r\nReferences: a\r\nReferences: b\r\nX-Mailer: Test\r\nX-1: 1\r\nX-2: 2\r\nX-3: 3\r\nSubject: Saying Hello\r\n\r\nThis is a message just to say hello.\r\n"
  },
  {
    "path": "tests/resources/smtp/milter/message.json",
    "content": "[\n    {\n        \"modifications\": [\n            {\n                \"AddHeader\": {\n                    \"name\": \"X-Hello\",\n                    \"value\": \"World\"\n                }\n            },\n            {\n                \"AddHeader\": {\n                    \"name\": \"X-CR\",\n                    \"value\": \"LF\\r\\n\"\n                }\n            }\n        ],\n        \"result\": \"X-Hello: World\\r\\nX-CR: LF\\r\\nFrom: John Doe <john@example.org>\\r\\nTo: Mary Smith <mary.smith@example.org>\\r\\nReferences: a\\r\\nReferences: b\\r\\nX-Mailer: Test\\r\\nX-1: 1\\r\\nX-2: 2\\r\\nX-3: 3\\r\\nSubject: Saying Hello\\r\\n\\r\\nThis is a message just to say hello.\\r\\n\"\n    },\n    {\n        \"modifications\": [\n            {\n                \"ReplaceBody\": {\n                    \"value\": [\n                        49,\n                        50,\n                        51\n                    ]\n                }\n            }\n        ],\n        \"result\": \"From: John Doe <john@example.org>\\r\\nTo: Mary Smith <mary.smith@example.org>\\r\\nReferences: a\\r\\nReferences: b\\r\\nX-Mailer: Test\\r\\nX-1: 1\\r\\nX-2: 2\\r\\nX-3: 3\\r\\nSubject: Saying Hello\\r\\n\\r\\n123\"\n    },\n    {\n        \"modifications\": [\n            {\n                \"AddHeader\": {\n                    \"name\": \"X-Spam\",\n                    \"value\": \"Yes\"\n                }\n            },\n            {\n                \"ReplaceBody\": {\n                    \"value\": [\n                        49,\n                        50,\n                        51\n                    ]\n                }\n            },\n            {\n                \"ReplaceBody\": {\n                    \"value\": [\n                        52,\n                        53,\n                        54\n                    ]\n                }\n            }\n        ],\n        \"result\": \"X-Spam: Yes\\r\\nFrom: John Doe <john@example.org>\\r\\nTo: Mary Smith <mary.smith@example.org>\\r\\nReferences: a\\r\\nReferences: b\\r\\nX-Mailer: Test\\r\\nX-1: 1\\r\\nX-2: 2\\r\\nX-3: 3\\r\\nSubject: Saying Hello\\r\\n\\r\\n123456\"\n    },\n    {\n        \"modifications\": [\n            {\n                \"ChangeHeader\": {\n                    \"index\": 1,\n                    \"name\": \"References\",\n                    \"value\": \"\"\n                }\n            },\n            {\n                \"ChangeHeader\": {\n                    \"index\": 1,\n                    \"name\": \"References\",\n                    \"value\": \"z\"\n                }\n            },\n            {\n                \"ChangeHeader\": {\n                    \"index\": 1,\n                    \"name\": \"Subject\",\n                    \"value\": \"[SPAM] Saying Hello\"\n                }\n            }\n        ],\n        \"result\": \"From: John Doe <john@example.org>\\r\\nTo: Mary Smith <mary.smith@example.org>\\r\\nReferences: z\\r\\nX-Mailer: Test\\r\\nX-1: 1\\r\\nX-2: 2\\r\\nX-3: 3\\r\\nSubject: [SPAM] Saying Hello\\r\\n\\r\\nThis is a message just to say hello.\\r\\n\"\n    },\n    {\n        \"modifications\": [\n            {\n                \"ChangeHeader\": {\n                    \"index\": 1,\n                    \"name\": \"X-Some-Header\",\n                    \"value\": \"Some Value\"\n                }\n            },\n            {\n                \"InsertHeader\": {\n                    \"index\": 2,\n                    \"name\": \"References\",\n                    \"value\": \"<my-new-ref>\"\n                }\n            },\n            {\n                \"InsertHeader\": {\n                    \"index\": 10,\n                    \"name\": \"X-3\",\n                    \"value\": \"z\"\n                }\n            },\n            {\n                \"ReplaceBody\": {\n                    \"value\": [\n                        52,\n                        53,\n                        54\n                    ]\n                }\n            },\n            {\n                \"ReplaceBody\": {\n                    \"value\": [\n                        49,\n                        50,\n                        51\n                    ]\n                }\n            }\n        ],\n        \"result\": \"X-Some-Header: Some Value\\r\\nFrom: John Doe <john@example.org>\\r\\nTo: Mary Smith <mary.smith@example.org>\\r\\nReferences: a\\r\\nReferences: <my-new-ref>\\r\\nReferences: b\\r\\nX-Mailer: Test\\r\\nX-1: 1\\r\\nX-2: 2\\r\\nX-3: z\\r\\nX-3: 3\\r\\nSubject: Saying Hello\\r\\n\\r\\n456123\"\n    },\n    {\n        \"modifications\": [\n            {\n                \"Quarantine\": {\n                    \"reason\": \"Virus found!\"\n                }\n            },\n            {\n                \"InsertHeader\": {\n                    \"index\": 1,\n                    \"name\": \"References\",\n                    \"value\": \"<my-new-ref>\"\n                }\n            }\n        ],\n        \"result\": \"X-Quarantine: Virus found!\\r\\nFrom: John Doe <john@example.org>\\r\\nTo: Mary Smith <mary.smith@example.org>\\r\\nReferences: <my-new-ref>\\r\\nReferences: a\\r\\nReferences: b\\r\\nX-Mailer: Test\\r\\nX-1: 1\\r\\nX-2: 2\\r\\nX-3: 3\\r\\nSubject: Saying Hello\\r\\n\\r\\nThis is a message just to say hello.\\r\\n\"\n    }\n]"
  },
  {
    "path": "tests/resources/smtp/reports/arf1.eml",
    "content": "From: <abusedesk@example.com>\nDate: Thu, 8 Mar 2005 17:40:36 EDT\nSubject: FW: Earn money\nTo: <abuse@example.net>\nMIME-Version: 1.0\nContent-Type: multipart/report; report-type=feedback-report;\n    boundary=\"part1_13d.2e68ed54_boundary\"\n\n--part1_13d.2e68ed54_boundary\nContent-Type: text/plain; charset=\"US-ASCII\"\nContent-Transfer-Encoding: 7bit\n\nThis is an email abuse report for an email message received from IP\n192.0.2.1 on Thu, 8 Mar 2005 14:00:00 EDT.  For more information\nabout this format please see http://www.mipassoc.org/arf/.\n\n--part1_13d.2e68ed54_boundary\nContent-Type: message/feedback-report\n\nFeedback-Type: abuse\nUser-Agent: SomeGenerator/1.0\nVersion: 1\n\n--part1_13d.2e68ed54_boundary\nContent-Type: message/rfc822\nContent-Disposition: inline\n\nReceived: from mailserver.example.net\n    (mailserver.example.net [192.0.2.1])\n    by example.com with ESMTP id M63d4137594e46;\n    Thu, 08 Mar 2005 14:00:00 -0400\nFrom: <somespammer@example.net>\nTo: <Undisclosed Recipients>\nSubject: Earn money\nMIME-Version: 1.0\nContent-type: text/plain\nMessage-ID: 8787KJKJ3K4J3K4J3K4J3.mail@example.net\nDate: Thu, 02 Sep 2004 12:31:03 -0500\n\nSpam Spam Spam\nSpam Spam Spam\nSpam Spam Spam\nSpam Spam Spam\n--part1_13d.2e68ed54_boundary--\n"
  },
  {
    "path": "tests/resources/smtp/reports/arf2.eml",
    "content": "From: <abusedesk@example.com>\nDate: Thu, 8 Mar 2005 17:40:36 EDT\nSubject: FW: Earn money\nTo: <abuse@example.net>\nMIME-Version: 1.0\nContent-Type: multipart/report; report-type=feedback-report;\n    boundary=\"part1_13d.2e68ed54_boundary\"\n\n--part1_13d.2e68ed54_boundary\nContent-Type: text/plain; charset=\"US-ASCII\"\nContent-Transfer-Encoding: 7bit\n\nThis is an email abuse report for an email message received from IP\n192.0.2.1 on Thu, 8 Mar 2005 14:00:00 EDT.  For more information\nabout this format please see http://www.mipassoc.org/arf/.\n\n--part1_13d.2e68ed54_boundary\nContent-Type: message/feedback-report\n\nFeedback-Type: abuse\nUser-Agent: SomeGenerator/1.0\nVersion: 1\nOriginal-Mail-From: <somespammer@example.net>\nOriginal-Rcpt-To: <user@example.com>\nArrival-Date: Thu, 8 Mar 2005 14:00:00 EDT\nReporting-MTA: dns; mail.example.com\nSource-IP: 192.0.2.1\nAuthentication-Results: mail.example.com;\n                spf=fail smtp.mail=somespammer@example.com\nReported-Domain: example.net\nReported-Uri: http://example.net/earn_money.html\nReported-Uri: mailto:user@example.com\nRemoval-Recipient: user@example.com\n\n--part1_13d.2e68ed54_boundary\nContent-Type: message/rfc822\nContent-Disposition: inline\n\nFrom: <somespammer@example.net>\nReceived: from mailserver.example.net (mailserver.example.net\n    [192.0.2.1]) by example.com with ESMTP id M63d4137594e46;\n    Thu, 08 Mar 2005 14:00:00 -0400\n\nTo: <Undisclosed Recipients>\nSubject: Earn money\nMIME-Version: 1.0\nContent-type: text/plain\nMessage-ID: 8787KJKJ3K4J3K4J3K4J3.mail@example.net\nDate: Thu, 02 Sep 2004 12:31:03 -0500\n\nSpam Spam Spam\nSpam Spam Spam\nSpam Spam Spam\nSpam Spam Spam\n--part1_13d.2e68ed54_boundary--\n"
  },
  {
    "path": "tests/resources/smtp/reports/arf3.eml",
    "content": "From: arf-daemon@example.com\nTo: recipient@example.net\nSubject: This is a test\nDate: Wed, 14 Apr 2010 12:17:45 -0700 (PDT)\nMIME-Version: 1.0\nContent-Type: multipart/report; report-type=feedback-report;\n    boundary=\"part1_13d.2e68ed54_boundary\"\n\n--part1_13d.2e68ed54_boundary\nContent-Type: text/plain; charset=\"US-ASCII\"\nContent-Transfer-Encoding: 7bit\n\nThis is an email abuse report for an email message received\nfrom IP 192.0.2.1 on Wed, 14 Apr 2010 12:15:31 PDT. For more\ninformation about this format please see\nhttp://www.mipassoc.org/arf/.\n\n--part1_13d.2e68ed54_boundary\nContent-Type: message/feedback-report\n\nFeedback-Type: auth-failure\nUser-Agent: SomeDKIMFilter/1.0\nVersion: 1\nOriginal-Mail-From: <randomuser@example.net>\nOriginal-Rcpt-To: <user@example.com>\nReceived-Date: Wed, 14 Apr 2010 12:15:31 -0700 (PDT)\nSource-IP: 192.0.2.1\nAuthentication-Results: mail.example.com; dkim=fail\n    header.d=example.net\nReported-Domain: example.net\nDKIM-Domain: example.net\nAuth-Failure: bodyhash\n\n--part1_13d.2e68ed54_boundary\nContent-Type: message/rfc822\n\nDKIM-Signature: v=1; c=relaxed/simple; a=rsa-sha256;\n    s=testkey; d=example.net; h=From:To:Subject:Date;\n    bh=2jUSOH9NhtVGCQWNr9BrIAPreKQjO6Sn7XIkfJVOzv8=;\n    b=AuUoFEfDxTDkHlLXSZEpZj79LICEps6eda7W3deTVFOk4yAUoqOB\n        4nujc7YopdG5dWLSdNg6xNAZpOPr+kHxt1IrE+NahM6L/LbvaHut\n        KVdkLLkpVaVVQPzeRDI009SO2Il5Lu7rDNH6mZckBdrIx0orEtZV\n        4bmp/YzhwvcubU4=\nReceived: from smtp-out.example.net by mail.example.com\n    with SMTP id o3F52gxO029144;\n    Wed, 14 Apr 2010 12:15:31 -0700 (PDT)\nReceived: from internal-client-001.example.com\n    by mail.example.com\n    with SMTP id o3F3BwdY028431;\n    Wed, 14 Apr 2010 12:12:09 -0700 (PDT)\nFrom: randomuser@example.net\nTo: user@example.com\nDate: Wed, 14 Apr 2010 12:12:09 -0700 (PDT)\nSubject: This is a test\n\nHi, just making sure DKIM is working!\n\n--part1_13d.2e68ed54_boundary--\n"
  },
  {
    "path": "tests/resources/smtp/reports/arf4.eml",
    "content": "Return-Path: <opendmarc@box.mydomain.name>\nReceived: by box.mydomain.name (Postfix, from userid 116)\n\tid CF8FA658E0; Tue,  5 Oct 2021 17:37:02 +1300 (NZDT)\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=box.mydomain.name;\n\ts=mail; t=1633408622;\n\tbh=yDlkGfe4dFwlsFeoKaIHG6xiRgQs2/PqPnLtiPm5ewk=;\n\th=From:To:Date:Subject:From;\n\tb=TwXvJJFoFwJDcb6IKMKsxp2BiRDsrjLOESyQPh/Cc4tRZltVAud/k6f0XP4l5a/T8\n\t kh0iDOGImc0O1WZNFt0MUcwLsfW4qbYjCBtthQDnbPApvv6MJDASwau+wipu5Nrkjc\n\t flg+nMaD97pVgR0LevMoVIWoiy1f5PNC/z0xkY2wnvyoGn91WuDsdocOqyoPo4RmIT\n\t A/f3M4CjOv/QmMEAWBIsa7kAZwf+rNmzDahFOtp2vFLqHt0iZi5vs40fa6O/I0snTM\n\t fRkv2GMZAug7NMU8MN/MhuO87FV6ATZXvB0Kxvsy9z0zZYK7tM1OYHjiCYot45erG3\n\t dlKrYiXsfd3BQ==\nFrom: OpenDMARC Filter <opendmarc@box.mydomain.name>\nTo: postmaster@vericty.interpublication.org\nDate: Tue,  5 Oct 2021 17:37:02 +1300 (NZDT)\nSubject: FW: Wir kaufen dein Auto!\nMIME-Version: 1.0\nContent-Type: multipart/report;\n\treport-type=feedback-report;\n\tboundary=\"box.mydomain.name:8BE2660E72\"\nMessage-Id: <20211005043702.CF8FA658E0@box.mydomain.name>\n\n--box.mydomain.name:8BE2660E72\nContent-Type: text/plain\n\nThis is an authentication failure report for an email message received from IP\n148.163.85.135 on Tue,  5 Oct 2021 17:37:02 +1300 (NZDT).\n\n--box.mydomain.name:8BE2660E72\nContent-Type: message/feedback-report\n\nFeedback-Type: auth-failure\nVersion: 1\nUser-Agent: OpenDMARC-Filter/1.3.2\nAuth-Failure: dmarc\nAuthentication-Results: box.mydomain.name; dmarc=fail header.from=interpublication.org\nOriginal-Envelope-Id: 8BE2660E72\nOriginal-Mail-From: info@interpublication.org\nSource-IP: 148.163.85.135 (sainay.interpublication.org)\nReported-Domain: interpublication.org\n\n--box.mydomain.name:8BE2660E72\nContent-Type: text/rfc822-headers\n\nAuthentication-Results: box.mydomain.name;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key; unprotected) header.d=interpublication.org header.i=@interpublication.org header.b=\"PrsTNnuH\";\n\tdkim-atps=neutral\nReceived: from dslb-002-202-150-127.002.202.pools.vodafone-ip.de (dslb-188-099-080-029.188.099.pools.vodafone-ip.de [188.99.80.29])\n\tby sainay.interpublication.org (Postfix) with ESMTPA id 6BB23A2D3\n\tfor <address@myotherdomain.name>; Tue,  5 Oct 2021 00:36:52 -0400 (EDT)\nDKIM-Filter: OpenDKIM Filter v2.11.0 sainay.interpublication.org 6BB23A2D3\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=interpublication.org; s=default; t=1633408612;\n\tbh=q1/OPSn+VXteY2+DHXqOIgs5LsNCJisEcQIKVW9it6I=;\n\th=From:Subject:To:Reply-To:Date:From;\n\tb=PrsTNnuH8D0Ch3gcWqGmXiYc2Kvu1CHGJBsqS521uYazd3G/urp7MHQvmNwK0r1gS\n\t DR3A3KwGejI5uuqzxDCqz28Mq6AkdTkOjFyXw65MLlsKTQddWTgciVnoqJempa6yzw\n\t PSM5550XqVFqqkNxEcYBUBYEwUdy1tY8rc4zhq8cIrsonQVxJJSbc3cdonICM1kLBV\n\t WASv16p3376ZBcKqFLc8UQ58YQKaFm51VZGEjtabfmWbgOQ7VikFFECDG3aRt8fZa6\n\t D03MrzUSngwPUdcRQZuqS/sApW/a9N2YwdbR51OFzPBr4ypUEIw/qprgBG4BfQQKeS\n\t 1PhinNvVtgQpQ==\nFrom: \"Rolf Bader\" <info@interpublication.org>\nSubject: Wir kaufen dein Auto!\nTo: \"address\" <address@myotherdomain.name>\nContent-Type: multipart/alternative; boundary=\"TD6gM3Blv=_XBZYNFT7dCsH1DHHOKUuSyA\"\nMIME-Version: 1.0\nReply-To: \"Rolf Bader\" <auto24-export@gmx.de>\nOrganization: AutoTEAM24\nDate: Tue, 5 Oct 2021 06:36:51 +0200\n\n--box.mydomain.name:8BE2660E72--\n\n"
  },
  {
    "path": "tests/resources/smtp/reports/arf5.eml",
    "content": "Message-ID: <433689.81121.example@mta.mail.receiver.example>\nFrom: \"SomeISP Antispam Feedback\" <feedback@mail.receiver.example>\nTo: arf-failure@sender.example\nSubject: FW: You have a new bill from your bank\nDate: Sat, 8 Oct 2011 15:15:59 -0500 (CDT)\nMIME-Version: 1.0\nContent-Type: multipart/report;\n    boundary=\"------------Boundary-00=_3BCR4Y7kX93yP9uUPRhg\";\n    report-type=feedback-report\nContent-Transfer-Encoding: 7bit\n\n--------------Boundary-00=_3BCR4Y7kX93yP9uUPRhg\nContent-Type: text/plain; charset=\"us-ascii\"\nContent-Disposition: inline\nContent-Transfer-Encoding: 7bit\n\nThis is an authentication failure report for an email message\nreceived from a.sender.example on 8 Oct 2011 20:15:58 +0000 (GMT).\nFor more information about this format, please see [RFC6591].\n\n--------------Boundary-00=_3BCR4Y7kX93yP9uUPRhg\nContent-Type: message/feedback-report\nContent-Transfer-Encoding: 7bit\n\nFeedback-Type: auth-failure\nUser-Agent: Someisp!Mail-Feedback/1.0\nVersion: 1\nOriginal-Mail-From: anexample.reply@a.sender.example\nOriginal-Envelope-Id: o3F52gxO029144\nAuthentication-Results: mta1011.mail.tp2.receiver.example;\n    dkim=fail (bodyhash) header.d=sender.example\nAuth-Failure: bodyhash\nDKIM-Canonicalized-Body: VGhpcyBpcyBhIG1lc3NhZ2UgYm9keSB0\n    aGF0IGdvdCBtb2RpZmllZCBpbiB0cmFuc2l0LgoKQXQgdGhlIHNhbWU\n    gdGltZSB0aGF0IHRoZSBib2R5aGFzaCBmYWlscyB0byB2ZXJpZnksIH\n    RoZQptZXNzYWdlIGNvbnRlbnQgaXMgY2xlYXJseSBhYnVzaXZlIG9yI\n    HBoaXNoeSwgYXMgdGhlClN1YmplY3QgYWxyZWFkeSBoaW50cy4gIElu\n    ZGVlZCwgdGhpcyBib2R5IGFsc28gY29udGFpbnMKdGhlIGZvbGxvd2l\n    uZyB0ZXh0OgoKICAgUGxlYXNlIGVudGVyIHlvdXIgZnVsbCBiYW5rIG\n    NyZWRlbnRpYWxzIGF0CiAgIGh0dHA6Ly93d3cuc2VuZGVyLmV4YW1wb\n    GUvCgpXZSBhcmUgaW1wbHlpbmcgdGhhdCwgYWx0aG91Z2ggbXVsdGlw\n    bGUgZmFpbHVyZXMKcmVxdWlyZSBtdWx0aXBsZSByZXBvcnRzLCBhIHN\n    pbmdsZSBmYWlsdXJlIGNhbiBiZQpyZXBvcnRlZCBhbG9uZyB3aXRoIH\n    BoaXNoaW5nIGluIGEgc2luZ2xlIHJlcG9ydC4K\nDKIM-Domain: sender.example\nDKIM-Identity: @sender.example\nDKIM-Selector: testkey\nArrival-Date: 8 Oct 2011 20:15:58 +0000 (GMT)\nSource-IP: 192.0.2.1\nReported-Domain: a.sender.example\nReported-URI: http://www.sender.example/\n\n--------------Boundary-00=_3BCR4Y7kX93yP9uUPRhg\nContent-Type: text/rfc822-headers\nContent-Transfer-Encoding: 7bit\n\nAuthentication-Results: mta1011.mail.tp2.receiver.example;\n dkim=fail (bodyhash) header.d=sender.example;\n spf=pass smtp.mailfrom=anexample.reply@a.sender.example\nReceived: from smtp-out.sender.example\n by mta1011.mail.tp2.receiver.example\n with SMTP id oB85W8xV000169;\n Sat, 08 Oct 2011 13:15:58 -0700 (PDT)\nDKIM-Signature: v=1; c=relaxed/simple; a=rsa-sha256;\n s=testkey; d=sender.example; h=From:To:Subject:Date;\n bh=2jUSOH9NhtVGCQWNr9BrIAPreKQjO6Sn7XIkfJVOzv8=;\n b=AuUoFEfDxTDkHlLXSZEpZj79LICEps6eda7W3deTVFOk4yAUoqOB\n 4nujc7YopdG5dWLSdNg6xNAZpOPr+kHxt1IrE+NahM6L/LbvaHut\n KVdkLLkpVaVVQPzeRDI009SO2Il5Lu7rDNH6mZckBdrIx0orEtZV\n 4bmp/YzhwvcubU4=\nReceived: from mail.sender.example\n by smtp-out.sender.example\n with SMTP id o3F52gxO029144;\n Sat, 08 Oct 2011 13:15:31 -0700 (PDT)\n Received: from internal-client-001.sender.example\n by mail.sender.example\n with SMTP id o3F3BwdY028431;\n Sat, 08 Oct 2011 13:15:24 -0700 (PDT)\nDate: Sat, 8 Oct 2011 16:15:24 -0400 (EDT)\nReply-To: anexample.reply@a.sender.example\nFrom: anexample@a.sender.example\nTo: someuser@receiver.example\nSubject: You have a new bill from your bank\nMessage-ID: <87913910.1318094604546@out.sender.example>\n\n--------------Boundary-00=_3BCR4Y7kX93yP9uUPRhg--\n\n"
  },
  {
    "path": "tests/resources/smtp/reports/dmarc1.eml",
    "content": "Received: from mail.stalw.art ([mail.stalw.art])\n\tby 127.0.0.1 (Stalwart JMAP) with LMTP;\n\tMon, 28 Nov 2022 10:51:56 +0000\nReceived: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a])\n\t(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)\n\t key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)\n\t(No client certificate requested)\n\tby mail.stalw.art (Postfix) with ESMTPS id 1145E7CC0B\n\tfor <domains@stalw.art>; Mon, 28 Nov 2022 10:51:53 +0000 (UTC)\nReceived: by mail-qv1-xf4a.google.com with SMTP id 71-20020a0c804d000000b004b2fb260447so12985969qva.10\n        for <domains@stalw.art>; Mon, 28 Nov 2022 02:51:52 -0800 (PST)\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\n        d=google.com; s=20210112;\n        h=content-transfer-encoding:content-disposition:to:from:subject\n         :message-id:date:mime-version:from:to:cc:subject:date:message-id\n         :reply-to;\n        bh=sMF/38UFRhmUYFRJST4vLBu/U1BXgsdCUE02HF8nXx8=;\n        b=I7WONP7tMsULp4eKjJeeKtM+nDYqMSIYMxqNHqCP1bTsnUiW2xM278I2+F8EjtFNYf\n         XOgusNn8kqbSnA4w1+q4G87zTF4K3tGnxNpuUMQ7GzcofBKtr7VPv9XFqvTPJ+N8YSwe\n         926ec7xi71BpSHAgqp5Wqocj8ruIVjcCZ37hYrG0C4s+FVBtbaU3EeyPpkESaaY2vE5y\n         Qa2KsrMsyJXlbyW/sFJ7AGDDuXwyGkTa+btP/xIiQM2HlBKy7vNOFZKkxInOuQsXJgZy\n         3H7ivlpD3hMrszwU77o5jBArVwN0RIkUSosAPQf+pzgvRlkseRlDrmzKQutvYWIaTP3/\n         FHPA==\nX-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\n        d=1e100.net; s=20210112;\n        h=content-transfer-encoding:content-disposition:to:from:subject\n         :message-id:date:mime-version:x-gm-message-state:from:to:cc:subject\n         :date:message-id:reply-to;\n        bh=sMF/38UFRhmUYFRJST4vLBu/U1BXgsdCUE02HF8nXx8=;\n        b=D3oClvT5AKcTpEjjffHQqPPQ9j5mmtExiviSq7iBYkoq+322LtR2hGqGxtvlAwRDsQ\n         VIfuKVExygw3c9bckjzKtJYX128HGK35gHnmsrzqvCC93JlRaC/55kcM9Bhks0xJnl7i\n         yNFHPZ0DY/jasdUdQ1QqnI+8qiPy+/12JvD+/TGlaDuS+RWYFU4/ky46S3vMXwXmRt6D\n         IGggXoW7snSaM4s88DzMUl0U7DH823UPQrUxnA5Oxscwn9M1ENJUWD/3EJo5ZEUMw0ll\n         Y8AlyhjWFgVqs1Y4V/LVWeXdF10fpm78+jm8QyIZYZJjh4I33AekdsWVM71ZNNYGL0+8\n         +GDg==\nX-Gm-Message-State: ANoB5pn0zRZSWXdFXd9G0tawbSeUYuhxToVkIYoLf8OJzoBLcIc0wKcB\n\tAfU9Coz5vAuiM1mASWJhbg==\nX-Google-Smtp-Source: AA0mqf6WWsnqMD4cHE40jB89/zblmT7yNKeHKlsvvCmlYANKmpKLTQTaCm5qCA0mVmxR/PTQogsNndWH/qe0ug==\nMIME-Version: 1.0\nX-Received: by 2002:ac8:5182:0:b0:39c:cb6a:300b with SMTP id\n c2-20020ac85182000000b0039ccb6a300bmr48409299qtn.181.1669632711968; Mon, 28\n Nov 2022 02:51:51 -0800 (PST)\nDate: Sun, 27 Nov 2022 15:59:59 -0800\nMessage-ID: <5264580628977113351@google.com>\nSubject: Report domain: stalw.art Submitter: google.com Report-ID: 5264580628977113351\nFrom: noreply-dmarc-support@google.com\nTo: domains@stalw.art\nContent-Type: application/zip; \n\tname=\"google.com!stalw.art!1669507200!1669593599.zip\"\nContent-Disposition: attachment; \n\tfilename=\"google.com!stalw.art!1669507200!1669593599.zip\"\nContent-Transfer-Encoding: base64\n\nUEsDBAoAAAAIAHFUfFWAeOSU8QEAAKkEAAAuAAAAZ29vZ2xlLmNvbSFzdGFsdy5hcnQhMTY2OTUw\nNzIwMCExNjY5NTkzNTk5LnhtbKVUwZKjIBC9z1ekck9Qk5hoMcye9gt2zxbB1lBBoACTmb9fHNCw\nma257El83f2632sUv70PYnUDY7mSr+t8m61XIJlquexf179//dyc1qs38oI7gPZM2ZW8rFbYgFbG\nNQM42lJHJ8yjyvSNpAOQXqlewJapAaMFDDkwUC6IVJ5BfGzagRq2saOe6H6kZSEv1rw7QxumpKPM\nNVx2ilyc07ZGKJZuH6WIIirtHQwq9mV5OGWe62t9II4yeEsORbn3uWVxqo7HPN/tDjlGj3BI91Kh\nMVT2UYyHztBzSfKyrA7Zsch8s4DMcZBtiFa7Q1X5UeRMhv5mW7qlnmKtBGcfjR7PgtsLLIMo744k\n1lFx31LjPFlAQpi2Vz4Qg1E4RNDq7hObngHSfg8SMNLx3c6AnRHNHMknVdPhc8p/TeR9ZMrMwxl1\nX+RbNRoGDdekoFle77uqZlme1+f9jtW1t/iRMJcwNUrfFKNwmOHYF25UjN64dg5MbnCrleXOX+A4\nf4okeZMZmlrrExZfovAuBhZzEq1PPf2mZoWYtyAd77j/fJayC9AWTNMZNaQbSuHI86Ua09FdGgN2\nFO5B+DTs98uP93piiJLiS6IWBDCnDLmB4FdujaayKLz2GV8MSDvjxJr/niIx2t/IJ9FTcrhPGD3+\nOn8AUEsBAgoACgAAAAgAcVR8VYB45JTxAQAAqQQAAC4AAAAAAAAAAAAAAAAAAAAAAGdvb2dsZS5j\nb20hc3RhbHcuYXJ0ITE2Njk1MDcyMDAhMTY2OTU5MzU5OS54bWxQSwUGAAAAAAEAAQBcAAAAPQIA\nAAAA"
  },
  {
    "path": "tests/resources/smtp/reports/dmarc2.eml",
    "content": "Received: from mail.stalw.art ([mail.stalw.art])\n\tby 127.0.0.1 (Stalwart JMAP) with LMTP;\n\tThu, 10 Nov 2022 03:27:19 +0000\nReceived: from mx0.backschues.net (lnxs001.backschues.net [85.183.142.13])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)\n\t(No client certificate requested)\n\tby mail.stalw.art (Postfix) with ESMTPS id 6DD117CC0B\n\tfor <domains@stalw.art>; Thu, 10 Nov 2022 03:27:16 +0000 (UTC)\nReceived: from mx0.backschues.net (localhost [127.0.0.1])\n\tby mx0.backschues.net with SMTP id 4N76hg4lNgz9ryP\n\tfor <domains@stalw.art>; Thu, 10 Nov 2022 04:27:15 +0100 (CET)\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=backschues.net;\n\ts=mail-2014-01; t=1668050835;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\t to:to:cc:mime-version:mime-version:content-type:content-type;\n\tbh=LTj1tdFz9JQFL/mVJASN0b9hGcolcCtY5v0bhnChJYY=;\n\tb=AtRegYc51PTYqDOy/6fB4xETTWAbVc2ivf8AfF4ygu3+6+oqBPyloTuOnEt7xYmjLFnll/\n\tSMZFFpRETsMlkiVg/1O0VpPRpIpiTbh4dwtUrRyo1Uw/cDJv5auz4rBMxcRNnDKypHwUKs\n\tBUahHWsVKH/TL5SzV79kqyjlYAs1HdJvS+wRINYBaptkeT6UeHGZakL21NnQUdOGt0fj4y\n\teJvWVtCYHZ5DUJ8K8h2W1NlTAWP8nTBoQVVDQrI5Zi1AEvnUWw+H7E8d/q2cF756/IBYso\n\trT56D3PYo2iuSt3aIBth1wL7/GJwc6N4JHcNpJ9XPV6xQbt+lm2b3+W59osL0Q==\nDKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=backschues.net;\n\ts=ed25519-mail-2018-10; t=1668050835;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\t to:to:cc:mime-version:mime-version:content-type:content-type;\n\tbh=LTj1tdFz9JQFL/mVJASN0b9hGcolcCtY5v0bhnChJYY=;\n\tb=y7d79OWWCrDX40k91FoBdGnUcrjN7xvWYyqskPfQmMoaSqFNSlTHH8gMXC/vXwiYIP3Oxp\n\td/hVvEuIIQBlwMDQ==\nFrom: \"DMARC Aggregate Report\" <noreply-dmarc-support@backschues.net>\nTo: domains@stalw.art\nSubject: Report Domain: stalw.art\n\tSubmitter: backschues.net\n\tReport-ID: stalw.art.1667948400.1668034800\nDate: Thu, 10 Nov 2022 03:27:02 GMT\nMIME-Version: 1.0\nMessage-ID: <afe541eab3ec091f@backschues.net>\nContent-Type: multipart/mixed;\n\tboundary=\"----=_NextPart_84e1fdd0-b285-4922-9fc7-88b070204303\"\n\nThis is a multipart message in MIME format.\n\n------=_NextPart_84e1fdd0-b285-4922-9fc7-88b070204303\nContent-Type: text/plain; charset=\"us-ascii\"\nContent-Transfer-Encoding: 7bit\n\nThis is an aggregate report from backschues.net.\n\nReport domain: stalw.art\nSubmitter: backschues.net\nReport ID: stalw.art.1667948400.1668034800\n\n------=_NextPart_84e1fdd0-b285-4922-9fc7-88b070204303\nContent-Type: application/gzip\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment;\n\tfilename=\"backschues.net!stalw.art!1667948400!1668034800.xml.gz\"\n\nH4sIAAAAAAAAA5VUsXLbMAzd/RU6D9ksSo6b2heG6dKOndJZR5OQzYtEsiSVNH9fUqQoqXWHT\ngIfgAfgASf8/KvvijcwVij5tK3LaluAZIoLeXna/nj5tjtui2eywS0AP1P2SjZFgQ1oZVzTg6\nOcOhowjypzaSTtgdz9HJR7DNGWXQew5fevLxhld4yGnoqOSOW5uo8d76lhOzvoQPxlkSrBYRR\njY16qLTixjnbvJTWurB8ePp8Ox0NVBfNY3R+OVYXRHBpTfa/QGCovqQcPneEiJJnzMYrI5AfJ\nyZIyvCMZWrPlaktRsFadYB+NHs6dsFfIjSg/kJwH8GQRiW7KX0VPDEbRSKDV7YiFb4S0l08CR\njq97QTYCdHMkTr0HYyxy7878ooyZXhcrHqfuNRgGDRCk09Vud/fl/X+VNangyfPnhjJ1CB9FY\nyikQrHMvBGu8HrxLOgXFitrHD+3FKzSyRHhblbv3TvzhKME7YJnlVAt2r5dcRRsOAgnWiFP/G\nUcAXKwTStUf1yBUt4ZPgjE9PBXRsDdujcRLVqLu1QgGtLf+zrpY6XG1KJptaGaxkf0y0Fnpts\n/7iRmS7KcYNukxX7zwbjWtaMicaf30qEEBaPB6P8h/gNNHLX4VQEAAA=\n------=_NextPart_84e1fdd0-b285-4922-9fc7-88b070204303--\n"
  },
  {
    "path": "tests/resources/smtp/reports/dmarc3.eml",
    "content": "Received: from mail.stalw.art ([mail.stalw.art])\n\tby 127.0.0.1 (Stalwart JMAP) with LMTP;\n\tTue, 08 Nov 2022 23:26:41 +0000\nReceived: from relay7.m.smailru.net (relay7.m.smailru.net [94.100.178.51])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby mail.stalw.art (Postfix) with ESMTPS id DD4337CC09\n\tfor <domains@stalw.art>; Tue,  8 Nov 2022 23:26:38 +0000 (UTC)\nDKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=corp.mail.ru; s=mail4;\n\th=Date:Message-ID:To:From:Subject:MIME-Version:Content-Type:From:Subject:Content-Type:Content-Transfer-Encoding:To:Cc; bh=fooa0+RBCZvyV2mP8Nx/UsLQ5RhazFg+SPGNtxZrCX0=;\n\tt=1667950001;x=1668040001; \n\tb=J9aMEkY9eVdOxjkNxaPFJ2Yk+/NCux9uOZl3iJXI0hEFaeYj9g7l+WtmXczk+YvgH3yhVhtvONUEYFsValRWWCAfmePm429N3mSuclVktk7t6RPJ4O5EcMjwrD9882vmX1xpI7ecPOzd5AD67HPt5SIA1RIa5injaOI5CWUXBBa5c0zDfmciyANAiDw0gm1axEMK4AUc61txPsX7H1qRq/FxGNITnnpYdqkkT2lR8sTl5HPwTjEsw4sYGKr5SiMpROhhbLZTM8RpojkP73bmw3UBZ9FI8iKApJUFB8i9tu0hjzHkev4uoDXgOXFYs/RAI1JkCWEp2Rjb3LpTSHT6cA==;\nReceived: from [10.161.4.115] (port=60844 helo=60)\n\tby relay7.m.smailru.net with esmtp (envelope-from <dmarc_support@corp.mail.ru>)\n\tid 1osXzK-0007VC-BD\n\tfor domains@stalw.art; Wed, 09 Nov 2022 02:26:38 +0300\nContent-Type: multipart/mixed; boundary=\"===============5640625649776607409==\"\nMIME-Version: 1.0\nSubject: Report Domain: stalw.art; Submitter: Mail.Ru;\n Report-ID: 28551467700969547611667865600\nFrom: dmarc_support@corp.mail.ru\nTo: domains@stalw.art\nMessage-ID: <dmarc-1667949998@corp.mail.ru>\nDate: Wed, 09 Nov 2022 02:26:38 +0300\nAuto-Submitted: auto-generated\nAuthentication-Results: relay7.m.smailru.net; auth=pass smtp.auth=dmarc_support@corp.mail.ru smtp.mailfrom=dmarc_support@corp.mail.ru; iprev=pass policy.iprev=10.161.4.115\n\n--===============5640625649776607409==\nMIME-Version: 1.0\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: base64\n\nVGhpcyBpcyBhbiBhZ2dyZWdhdGUgcmVwb3J0IGZyb20gTWFpbC5SdS4=\n\n--===============5640625649776607409==\nContent-Type: application/gzip\nMIME-Version: 1.0\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment;\n filename=\"mail.ru!stalw.art!1667865600!1667952000.xml.gz\"\n\nH4sICK7lamMC/21haWwucnUhc3RhbHcuYXJ0ITE2Njc4NjU2MDAhMTY2Nzk1MjAwMC54bWwAdVNB\ncqMwELzvK3LzKQhYg01qouwHctkPULIYjMogqSThJL/fEQSC18kFzbRmWt0jAS/vQ/9wReeV0c+7\nLEl3D6ilaZQ+P+/G0D4edy/8F7SIzUnICweH1rhQDxhEI4LgYNy51mJA/ipUn/wdga0I4EAYbwbh\nZO1HGzv/SONsEvHEUe1cAfgenKil0UHIUCvdGt6FYJ8Y67Bfy1lcHyNCjfcdizbV8PxYFNm+PBzS\ntCqrYn8os6wsD8eyKNMU2FchkAmsndBnknvCs9J8WzgjgLqZ4KrI0wjHHNi2ld3NxZpeyY/ajqde\n+Q7jUYb0a+6D6N8S4QIxzAiI5qIG7oDNAQhv2ymNK1iujUZgloNfYgrAysCzKCcG9L070CENO67m\njVrN6CTWyvIiTfL8d5LlVZJVe+Jad0CaURMpsDlYTOBV9CO5jSaUt8arQO/lU8oWgUl/S9dE+GQl\nOpjzyQu7Z2STPNWgDqpV9BQ5dCgadHXrzLAd1xYGdtMhxtDVDv3YB/+pYpm3wtAm9Ca/xu2xRxmM\nm7bI7JrDzMCt8D5e6ZQsTm5Iv7nEleWKvboo76zQef4N+zyO/9in6fysWBqLfIjOiXBKftA6T/l2\nHGx5CGz9j/8BQWPZIPkDAAA=\n--===============5640625649776607409==--"
  },
  {
    "path": "tests/resources/smtp/reports/dmarc4.eml",
    "content": "Received: from mail.stalw.art ([mail.stalw.art])\tby 127.0.0.1 (Stalwart JMAP) with LMTP;\tTue, 25 Oct 2022 04:08:22 +0000\nReceived: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2073.outbound.protection.outlook.com [40.107.244.73])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby mail.stalw.art (Postfix) with ESMTPS id A24107CC0A\n\tfor <domains@stalw.art>; Tue, 25 Oct 2022 04:08:22 +0000 (UTC)\nARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;\n b=SvolQ1oIEgdfCI6dbwmJ1jS0ovWmprW6kT3q9NgrbX+CMhIsdrqyS3Q1sO16KT2wCQAyNofiEZ5tKY0e1PzzMqeR29jUWvEye9T43fCfUeLFx9b45YrfkGYwqLeDIq0Ywl+ggVmsm7X83XqI6+9EC6qMukCb0cbLazu3rW/Rbyc6d5+fq6QTFZovATGRvHz71H9t7e//hYI23XjU5Q3Enw0Qq3xPSyusWDi3t7CfGXn9i2120XlNLnPxef5PCmwy4E+OTJ5qC5WtMthOskKKuFvx8onOYmc/JjJ3VrtZwALx9C+ulzix5US6H7pFvZ2jtDbMnW4U7ir/hp5xn5adFw==\nARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector9901;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=65M39Uvc5w8zbmK6TxEdoblSMXlIyqTXJHNalJ80wk4=;\n b=PN7QPeXJLr6tmH2CxydbDQjHqBtFKNN9HjGimeUHaIeSr82WHf4R295QbVX7gxw6sFE7Z9lZMTrMSqbRVI7rhbx+SEkxCfAothf9207FDX6t37Zt0wd/5EwR6dzfbcNJBL+U0/iG4J03L5b1geWY+e68mHKYH4/ybGcr+SBKuv/LgfZNtOfbQ3ioiKvFcpSDqd/qGUs4U9l2tVlXgbcKkct04sCuPciqgLEuIGirPLLbDUaBRJc51ZZB6CeporySRdHp6uFXyy3VBvvLVuwDNnnPrW4BUL05AuutzK7rc8ZQEpWf7r0gUEg2ArSrvs6Znnfe97oRa01L2SeFwuZsMA==\nARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none\n action=none header.from=microsoft.com; dkim=none (message not signed);\n arc=none\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=notification.microsoft.com; s=selector1;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=65M39Uvc5w8zbmK6TxEdoblSMXlIyqTXJHNalJ80wk4=;\n b=NjqsA7D6sEq1WgCZ/E1f5/B+XUXe5F4uv6CF2KQYVuyRnxItdox09LCWqZQ+fNQ6BbJ4Ne05Cb1BPbPP9yvb8Y6B1s2QvuxkUb69UFbAhoFgsRT6A4K76ykKQQyiPoYpxlO6FEyy+gel4y7c9XRLiWW6OxMIBcjBGB5ziP7mGFaJx4qXJ2mROfO7uZfrCu5pzOimkjPw6extWv4i0Kl3XKvBtXZnsr9eoC10mJvEAp7E2cpnaZnP46RQc9cmXzlmvhKPvCQCUWipJN9f1BTTvFjJ9ff6ehmN9RSzCckj3SZGw9XAnd0WYqh4evt6Y1RxQ4iQDSaZHNRpyMOtmkWc/w==\nAuthentication-Results: dkim=none (message not signed)\n header.d=none;dmarc=none action=none header.from=microsoft.com;\nReceived: from BN9PR03CA0046.namprd03.prod.outlook.com (2603:10b6:408:fb::21)\n by SJ0PR18MB3916.namprd18.prod.outlook.com (2603:10b6:a03:2c9::21) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.21; Tue, 25 Oct\n 2022 04:08:19 +0000\nReceived: from BN7NAM10FT048.eop-nam10.prod.protection.outlook.com\n (2603:10b6:408:fb:cafe::d7) by BN9PR03CA0046.outlook.office365.com\n (2603:10b6:408:fb::21) with Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.27 via Frontend\n Transport; Tue, 25 Oct 2022 04:08:19 +0000\nReceived: from nam10.map.protection.outlook.com (2a01:111:f400:fe53::30) by\n BN7NAM10FT048.mail.protection.outlook.com (2a01:111:e400:7e8f::199) with\n Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.16 via Frontend\n Transport; Tue, 25 Oct 2022 04:08:19 +0000\nMessage-ID: <725cbfbe133940149987cfc528387235@microsoft.com>\nX-Sender: <dmarcreport@microsoft.com> XATTRDIRECT=Originating XATTRORGID=xorgid:96f9e21d-a1c4-44a3-99e4-37191ac61848\nMIME-Version: 1.0\nFrom: \"DMARC Aggregate Report\" <dmarcreport@microsoft.com>\nTo: <domains@stalw.art>\nSubject: =?utf-8?B?UmVwb3J0IERvbWFpbjogc3RhbHcuYXJ0IFN1Ym1pdHRlcjogcHJvdGVjdGlvbi5vdXRsb29rLmNvbSBSZXBvcnQtSUQ6IDcyNWNiZmJlMTMzOTQwMTQ5OTg3Y2ZjNTI4Mzg3MjM1?=\nContent-Type: multipart/mixed;\n\tboundary=\"_mpm_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\"\nDate: Tue, 25 Oct 2022 04:08:19 +0000\nX-EOPAttributedMessage: 0\nX-MS-PublicTrafficType: Email\nX-MS-TrafficTypeDiagnostic: BN7NAM10FT048:EE_|SJ0PR18MB3916:EE_\nX-MS-Office365-Filtering-Correlation-Id: 7f843e40-ccc6-4c17-1ce7-08dab63e8cd1\nX-MS-Exchange-SenderADCheck: 2\nX-MS-Exchange-AntiSpam-Relay: 0\nX-Microsoft-Antispam: BCL:0;\nX-Microsoft-Antispam-Message-Info:\n\tPSiI3L4DKj/cyRBl8/bmbyQMrr1DvsYEB1+aTn/3Y39oHnyJ5HXcxu6jNUl32WcPW6Gfqmhc6P1RFE5L/9ev0cWnqh4GgIs2qmHicLexPmMjP8viPdjb1N7TSSOv1hhXMT+gVLx889X5sltd4qpfIAWhoxNonjQpVIgt4VOVnbCWTu1hyOjOVplq0rKqIF04BQGHZnBRfkcD1No+mZrvx8RLWIwInU3fpPeGz77Wn3TIvHtzypR/d22WpZ8eHk3aIxxdjwp5WLg4unpiJaieyQN7BRhD/v6b3pLFVJP8Ii2+FGjTsKASczEL4dHnIoIrHYE0wwaFFPcSNzovLhzYguDV42EGS8Fm7soiew4ch+hICM0LPNTGTZIDe7wm2eSwhN2tkJK4QCfh1DON39jXninVr88ZlzMcDXnXpgvWHHiur8az7Gvs9zHH/1tFMsPVSh7BS+8fHEcBYpdtihrP22GcjbOd98IiTAs/dVzSy0TUg6WEgJO6oUklGjqVbi99CrNZI1BtLP4vH4aSlz9JYg4et6SxiJlKyoSzqUr2NN9/pyFdQ//5d/EEjKJz8CAcQmCjjPEObGFttT3maY2+zsa2THodZgpfMyDbA3WUKxE=\nX-Forefront-Antispam-Report:\n\tCIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nam10.map.protection.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(396003)(39860400002)(346002)(34036004)(366004)(376002)(136003)(47540400005)(451199015)(2616005)(52230400001)(121820200001)(83380400001)(166002)(86362001)(41300700001)(2906002)(4001150100001)(8936002)(316002)(5660300002)(235185007)(41320700001)(508600001)(6486002)(6512007)(6506007)(24736004)(108616005)(68406010)(85236043)(8676002)(10290500003)(6916009)(36736006)(36756003)(66899015);DIR:OUT;SFP:1101;\nX-OriginatorOrg: dmarcrep.onmicrosoft.com\nX-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2022 04:08:19.1682\n (UTC)\nX-MS-Exchange-CrossTenant-Network-Message-Id: 7f843e40-ccc6-4c17-1ce7-08dab63e8cd1\nX-MS-Exchange-CrossTenant-AuthSource: BN7NAM10FT048.eop-nam10.prod.protection.outlook.com\nX-MS-Exchange-CrossTenant-AuthAs: Internal\nX-MS-Exchange-CrossTenant-Id: 96f9e21d-a1c4-44a3-99e4-37191ac61848\nX-MS-Exchange-CrossTenant-FromEntityHeader: Internet\nX-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR18MB3916\n\nThis is a multi-part message in MIME format.\n\n--_mpm_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\nContent-Type: multipart/related;\n\tboundary=\"_rv_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\"\n\n--_rv_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\nContent-Type: multipart/alternative;\n\tboundary=\"_av_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\"\n\n--_av_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\n\n\n--_av_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\nContent-Type: text/html; charset=us-ascii\nContent-Transfer-Encoding: base64\n\nPGRpdiBzdHlsZSA9ImZvbnQtZmFtaWx5OlNlZ29lIFVJOyBmb250LXNpemU6MTRweDsiPlRoaXMgaX\nMgYSBETUFSQyBhZ2dyZWdhdGUgcmVwb3J0IGZyb20gTWljcm9zb2Z0IENvcnBvcmF0aW9uLiBGb3Ig\nRW1haWxzIHJlY2VpdmVkIGJldHdlZW4gMjAyMi0xMC0yMyAwMDowMDowMCBVVEMgdG8gMjAyMi0xMC\n0yNCAwMDowMDowMCBVVEMuPC8gZGl2PjxiciAvPjxiciAvPllvdSdyZSByZWNlaXZpbmcgdGhpcyBl\nbWFpbCBiZWNhdXNlIHlvdSBoYXZlIGluY2x1ZGVkIHlvdXIgZW1haWwgYWRkcmVzcyBpbiB0aGUgJ3\nJ1YScgdGFnIG9mIHlvdXIgRE1BUkMgcmVjb3JkIGluIEROUyBmb3Igc3RhbHcuYXJ0LiBQbGVhc2Ug\ncmVtb3ZlIHlvdXIgZW1haWwgYWRkcmVzcyBmcm9tIHRoZSAncnVhJyB0YWcgaWYgeW91IGRvbid0IH\ndhbnQgdG8gcmVjZWl2ZSB0aGlzIGVtYWlsLjxiciAvPjxiciAvPjxkaXYgc3R5bGUgPSJmb250LWZh\nbWlseTpTZWdvZSBVSTsgZm9udC1zaXplOjEycHg7IGNvbG9yOiM2NjY2NjY7Ij5QbGVhc2UgZG8gbm\n90IHJlc3BvbmQgdG8gdGhpcyBlLW1haWwuIFRoaXMgbWFpbGJveCBpcyBub3QgbW9uaXRvcmVkIGFu\nZCB5b3Ugd2lsbCBub3QgcmVjZWl2ZSBhIHJlc3BvbnNlLiBGb3IgYW55IGZlZWRiYWNrL3N1Z2dlc3\nRpb25zLCBraW5kbHkgbWFpbCB0byBkbWFyY3JlcG9ydGZlZWRiYWNrQG1pY3Jvc29mdC5jb20uPGJy\nIC8+PGJyIC8+TWljcm9zb2Z0IHJlc3BlY3RzIHlvdXIgcHJpdmFjeS4gUmV2aWV3IG91ciBPbmxpbm\nUgU2VydmljZXMgPGEgaHJlZiA9Imh0dHBzOi8vcHJpdmFjeS5taWNyb3NvZnQuY29tL2VuLXVzL3By\naXZhY3lzdGF0ZW1lbnQiPlByaXZhY3kgU3RhdGVtZW50PC9hPi48YnIgLz5PbmUgTWljcm9zb2Z0IF\ndheSwgUmVkbW9uZCwgV0EsIFVTQSA5ODA1Mi48LyBkaXYgPg==\n\n--_av_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_--\n\n--_rv_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_--\n\n--_mpm_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_\nContent-Type: application/gzip\nContent-Transfer-Encoding: base64\nContent-ID: <3ff45643-7977-4f3c-a97d-14b9e7faa5e7>\nContent-Description: protection.outlook.com!stalw.art!1666483200!1666569600.xml.gz\nContent-Disposition: attachment; filename=\"protection.outlook.com!stalw.art!1666483200!1666569600.xml.gz\";\n\nH4sIAAAAAAAEAM1VzY7bIBi8V+o7RLnXxHZ+VyzbB2jVQy+9WQTjBMUGBDjZvn0/G4JJsu3usZcE5h\nvzDcNg45fXrp2dubFCyed5ni3mL+TzJ9xwXu8pO82gLO3Tq62f50fn9BNCl8slu5SZMgdULBY5+vX9\n20925B2dR7J4n/xFSOuoZHwO7WYzHCQQUIDRdTJWDNfKuKrjjtbU0REEGJasJO04+dG7VqlTxlSHUU\nQDCzqJltQdNcyv87UTzCirGucf8ITADq1ETTbFiu2bPc/Lcrdc5MvdbrthDVsV23K7KcoVRhM3PAzi\neGWoPFybA7bnBwF7Wq/Xy20JBmDkkUjgsh7Lq/VuPZSHeVgP3S0YW944gbVqBftd6X7fCnvkkxwFO5\nMETG4vGTUO1vNIqNP6JDpiMPKDK2p1M4LDf8A0kUpyjPQVsFfERkgzR/JhA8MgYI0iAMCvV/+mULCc\nKRNFG3WZvLGqN4xXQpPVIiuKMsuLXZbvltA3ViKZqV6CBIz8IOKhKz/Ttgc/61gZLBJWKyvcEDW/oR\nRJiYNDDQQFGJNZwYsmVCbHkt3e94VDjFvEoubSiUZA2tNEnHmrNK+cIiqNdlp4ZDdGdURw1wx3LSGP\neKQfOa258WCSjBS+6nwUh2nvjpXhtm9dIvjekRCzSctN7rxpvOXMKTOS4MziPOH4PkRTa4fkj5PJ3p\num/7GEv12/Ww1wVuIkrNFUFsU/tfiovaMlTeIH3WCQFdINAYD24+TDPiRvCvSQkIEfLji8CsJHhfwB\nwJC79XYGAAA=\n\n--_mpm_a4bcd9a515b44b9d8eceb05d7333675fpiotk5m200exchangecorpm_--\n"
  },
  {
    "path": "tests/resources/smtp/reports/dmarc5.eml",
    "content": "Received: from mail.stalw.art ([mail.stalw.art])\tby 127.0.0.1 (Stalwart JMAP) with LMTP;\tTue, 20 Sep 2022 10:28:19 +0000\nReceived: from a14-92.smtp-out.amazonses.com (a14-92.smtp-out.amazonses.com [54.240.14.92])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby mail.stalw.art (Postfix) with ESMTPS id 1337D7E19D\n\tfor <domains@stalw.art>; Tue, 20 Sep 2022 10:28:18 +0000 (UTC)\nDKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;\n\ts=a66wkfbz3zwxdt2n5p6d7lj2ja7sdwuc; d=amazonses.com; t=1663669697;\n\th=From:To:Message-ID:Subject:MIME-Version:Content-Type:Date;\n\tbh=h9v7dueDYfUxVokuKSLTqLuOwisdgdDRQ6TLwJOzXes=;\n\tb=dHR5EJhoY9s8g2/Y4K4rHdz44k67r7fyC4wr2AWZmemrVBoxYHJPwa295S2VJQtY\n\tkxTxppN2GEcNxhUMw8TXBrRwNKdoOLU38ZtrAN1a4hWVxmlwky1dtjXETQ/qJ257Nzg\n\tbsXkAo4S1RABFmkQQJ0zSPZGkMW+lpZTBCDzlOHU=\nDKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;\n\ts=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1663669697;\n\th=From:To:Message-ID:Subject:MIME-Version:Content-Type:Date:Feedback-ID;\n\tbh=h9v7dueDYfUxVokuKSLTqLuOwisdgdDRQ6TLwJOzXes=;\n\tb=UDIvc6rvbihyGbzGRsmSSSzVNFgpfb3V3j0UivcNjlX2y63vjLinol463Z/+3Xh3\n\tBmxAOiLHF/DbVnqqNg5ygdxsa7MBHXEJ5we3W8vQr37xNk5DqhV7HPBSFttWP5sy0dg\n\trdjyfMIjqJ1J/2+aM4opFA/6EWif7TGmjo7N1KKM=\nFrom: postmaster@amazonses.com\nTo: domains@stalw.art\nMessage-ID: <010001835a70fc8d-a3d7eff5-7adb-41cc-87bd-a646d9776a69-000000@email.amazonses.com>\nSubject: Dmarc Aggregate Report Domain: {stalw.art}  Submitter: {Amazon SES}\n  Date: {2022-09-19}  Report-ID: {6b06c366-0631-4ca0-8337-f5aecf137918}\nMIME-Version: 1.0\nContent-Type: multipart/mixed; \n\tboundary=\"----=_Part_42492_694130218.1663669697673\"\nDate: Tue, 20 Sep 2022 10:28:17 +0000\nFeedback-ID: 1.us-east-1.CTa/CO4t1eWkL0VlHBu5/eINCZhxZraAIsQC/FZHIgk=:AmazonSES\nX-SES-Outgoing: 2022.09.20-54.240.14.92\n\n------=_Part_42492_694130218.1663669697673\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\nThis MIME email was sent through Amazon SES.\n------=_Part_42492_694130218.1663669697673\nContent-Type: application/octet-stream; \n\tname=amazonses.com!stalw.art!1663545600!1663632000.xml.gz\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment; \n\tfilename=amazonses.com!stalw.art!1663545600!1663632000.xml.gz\n\nH4sIAAAAAAAAAG1TwXLbIBA9O1/RyV1CWLHszlDSHHJMe8itFw1GK5uJBAwgp+3XlwXJVjK9SOzb\n1b59vBV7/D0OXy7gvDL62z0tq/tHfsd6gO4o5Bu/27A5yauSMrIEEXdgjQvtCEF0IogIbZhxp1aL\nEfjTy9Ovnz+K1+dXRq4gVsAo1MCt8WEUPoD7Lkbx12gPvpRmZCTnsXLurzreHKtG1k1TVE1Niwcp\nquJQ1/ui3wmQPa33X+mBkVs9fh1HgtYJfUq0G3aEk9KcNk29e9g1VcVIRlISdJdSTb2tMIUxNiEf\nulwpVpKZNYOSf1o7HQflzzCTm6hCcx/E8F4KF2KjjGBSdG9q5I6RfEiQt31C8I2A5dpoYMSmyC+h\nz7GVgVOcEw8I9IbHKD5xyP9MFO9SGpdnc+Y9i/ZmchJaZfm22pd0T0t6OJRb7HtLpUppJh0ZGcmH\nhM0scBHDFC8p9UblykdvVcAdyTOvkbkGZffR5picbyCJ7GdwvoSblA8k0YWsgKkOdFC9iiu52HiB\nwVhoe2dGnher1AP6uU6k2jOIDlwGVj6t4UT2iYSJKZxbB34awsy6jHu1fUV8sz0tNH7FrfAeVykF\nWediO/nUHcuycdHd5Zf8BxMenbqzAwAA\n------=_Part_42492_694130218.1663669697673--\n"
  },
  {
    "path": "tests/resources/smtp/reports/tls1.eml",
    "content": "From: tlsrpt@mail.sender.example.com\nDate: Fri, May 09 2017 16:54:30 -0800\nTo: mts-sts-tlsrpt@example.net\nSubject: Report Domain: example.net\n    Submitter: mail.sender.example.com\n    Report-ID: <735ff.e317+bf22029@example.net>\nTLS-Report-Domain: example.net\nTLS-Report-Submitter: mail.sender.example.com\nMIME-Version: 1.0\nContent-Type: multipart/report; report-type=\"tlsrpt\";\n    boundary=\"----=_NextPart_000_024E_01CC9B0A.AFE54C00\"\nContent-Language: en-us\n\nThis is a multipart message in MIME format.\n\n------=_NextPart_000_024E_01CC9B0A.AFE54C00\nContent-Type: text/plain; charset=\"us-ascii\"\nContent-Transfer-Encoding: 7bit\n\nThis is an aggregate TLS report from mail.sender.example.com\n\n------=_NextPart_000_024E_01CC9B0A.AFE54C00\nContent-Type: application/tlsrpt+gzip\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment;\n    filename=\"mail.sender.example!example.com!1013662812!1013749130.json.gz\"\n\nH4sICCpFtWMAA3JwdDAxLmpzb24uMQCtVVtr2zAYfe+vEN7bmFzZjt3EMLbRhu1hdCUJI+soRpGU\nVMy2jCSHZCX/fZIvzVLHpc0WDDHSOfqOznfxwxkwP0fIFc75b6y5yGGOM+bEwLkUWYHzLZw772oU\nxZpBifOV3X6o1qp1pbHU0O5qXlN95EUQDSDyZgjF1XPbnFIxWE778H4QhyPz3DoVfNfEJiLXmGjI\n86WwDKUVlKwQUvN89ZE0Ujcu2+CsSFkruYZATi0nRFE48C8I9AMawMEFwXARMQRHg4hhxIZksHgk\nFiLlhDNleD8fde/vvMdsD7x4sgf1tmCN3L/u/xSltDS3OAh1AFszqUxmYjCdTdfekYMqVCYoi4Fm\nylrSC9rE4K2bYZ66rWnbJ6Z1OXiT4JU5exgNEHI6oLv+m1FhQuXWgZdEM+rgvVC634le6V1RByu7\nw2COKrMMy57caaFxClVJCFNqWZpX8287g4gyt+LCwI+OqK95SyOwlKxDClDwrKSWR5k2b+qoB12x\nFVUyVab6sdgIM12x5MS2K9sUXDLal1plOtFUC8w0hryoWxF5MV0MY7wg1PSt58dxb8lJRhhfVwfU\nmWtnR7bxXllk9vqMdlzzEOrgd90jXmZMNah0qmAutMlvYWfDP3kTnOaN/0pv9ke1OgIXuZ4XuGH0\nSj/NFXoImFJu578pYTtkZVZ9DWy4e60LFZ+f18NUuZ1p2+wklgc+AE7Be9AMW0AABH4AaPAGTBv7\nr4WetuaDbueenN41Tjmtv2FNM708td5o6Iaea8rNjfwT8jA8oQzgApNfZfF/GiV4Bm7HimRYVXBa\nRZ+HaJR8T8aTSXIz+Tb/kdx8mn1Jvo6vP5u/8fxyPL4aXx3JzcHKfsbW63dnuz+8byfQUQgAAA==\n\n------=_NextPart_000_024E_01CC9B0A.AFE54C00--\n"
  },
  {
    "path": "tests/resources/smtp/reports/tls2.eml",
    "content": "From: tlsrpt@mail.sender.example.com\nDate: Fri, May 09 2017 16:54:30 -0800\nTo: mts-sts-tlsrpt@example.net\nSubject: Report Domain: example.net\n    Submitter: mail.sender.example.com\n    Report-ID: <735ff.e317+bf22029@example.net>\nTLS-Report-Domain: example.net\nTLS-Report-Submitter: mail.sender.example.com\nMIME-Version: 1.0\nContent-Type: multipart/report; report-type=\"tlsrpt\";\n    boundary=\"----=_NextPart_000_024E_01CC9B0A.AFE54C00\"\nContent-Language: en-us\n\nThis is a multipart message in MIME format.\n\n------=_NextPart_000_024E_01CC9B0A.AFE54C00\nContent-Type: text/plain; charset=\"us-ascii\"\nContent-Transfer-Encoding: 7bit\n\nThis is an aggregate TLS report from mail.sender.example.com\n\n------=_NextPart_000_024E_01CC9B0A.AFE54C00\nContent-Type: application/tlsrpt\nContent-Disposition: attachment;\n    filename=\"mail.sender.example!example.com!1013662812!1013749130.json\"\n\n{\n    \"report-id\": \"2020-01-01T00:00:00Z_example.com\",\n    \"date-range\": {\n        \"start-datetime\": \"2020-01-01T00:00:00Z\",\n        \"end-datetime\": \"2020-01-07T23:59:59Z\"\n    },\n    \"organization-name\": \"Google Inc.\",\n    \"contact-info\": \"smtp-tls-reporting@google.com\",\n    \"policies\": [\n        {\n            \"policy\": {\n                \"policy-type\": \"sts\",\n                \"policy-string\": [\n                    \"version: STSv1\",\n                    \"mode: enforce\",\n                    \"mx: demo.example.com\",\n                    \"max_age: 604800\"\n                ],\n                \"policy-domain\": \"example.com\"\n            },\n            \"summary\": {\n                \"total-successful-session-count\": 23,\n                \"total-failure-session-count\": 1\n            },\n            \"failure-details\": [\n                {\n                    \"result-type\": \"certificate-host-mismatch\",\n                    \"sending-mta-ip\": \"123.123.123.123\",\n                    \"receiving-ip\": \"234.234.234.234\",\n                    \"receiving-mx-hostname\": \"demo.example.com\",\n                    \"failed-session-count\": 1\n                }\n            ]\n        }\n    ]\n}\n\n------=_NextPart_000_024E_01CC9B0A.AFE54C00--\n"
  },
  {
    "path": "tests/resources/smtp/sieve/awl.sieve",
    "content": "require [\"variables\", \"include\", \"vnd.stalwart.expressions\", \"reject\"];\n\nglobal \"score\";\n\n# Create AWL table\nif eval \"!query('sql', 'CREATE TABLE awl (score FLOAT, count INT, sender TEXT NOT NULL, ip TEXT NOT NULL, PRIMARY KEY (sender, ip))' , [])\" {\n    reject \"create table query failed\";\n    stop;\n}\n\n\nset \"score\" \"1.1\";\ninclude \"awl_include\";\nif eval \"score != 1.1\" {\n    reject \"awl_include #1 set score to ${score}\";\n    stop;\n}\n\nset \"score\" \"2.2\";\ninclude \"awl_include\";\nif eval \"score != 1.6500000000000001\" {\n    reject \"awl_include #2 set score to ${score}\";\n    stop;\n}\n\nset \"score\" \"9.3\";\ninclude \"awl_include\";\nif eval \"score != 5.4750000000000005\" {\n    reject \"awl_include #3 set score to ${score}\";\n    stop;\n}\n\n"
  },
  {
    "path": "tests/resources/smtp/sieve/awl_include.sieve",
    "content": "require [\"variables\", \"include\", \"vnd.stalwart.expressions\", \"reject\"];\n\nglobal \"score\";\nset \"awl_factor\" \"0.5\";\n\nlet \"result\" \"query('sql', 'SELECT score, count FROM awl WHERE sender = ? AND ip = ?', [env.from, env.remote_ip])\";\n\nlet \"awl_score\" \"result[0]\";\nlet \"awl_count\" \"result[1]\";\n\nif eval \"awl_count > 0\" {\n\tif eval \"!query('sql', 'UPDATE awl SET score = score + ?, count = count + 1 WHERE sender = ? AND ip = ?', [score, env.from, env.remote_ip])\" {\n\t\treject \"update query failed\";\n\t\tstop;\n\t}\n\tlet \"score\" \"score + ((awl_score / awl_count) - score) * awl_factor\";\n} elsif eval \"!query('sql', 'INSERT INTO awl (score, count, sender, ip) VALUES (?, 1, ?, ?)', [score, env.from, env.remote_ip])\" {\n\treject \"insert query failed\";\n\tstop;\n}\n"
  },
  {
    "path": "tests/resources/smtp/sieve/stage_connect.sieve",
    "content": "require [\"variables\", \"reject\"];\n\nif string \"${env.remote_ip}\" \"10.0.0.88\" {\n    reject \"Your IP '${env.remote_ip}' is not welcomed here.\";\n}\n"
  },
  {
    "path": "tests/resources/smtp/sieve/stage_data.sieve",
    "content": "require [\"envelope\", \"reject\", \"variables\", \"replace\", \"mime\", \"foreverypart\", \"editheader\", \"extracttext\", \"enotify\"];\n\nif envelope :localpart :is \"to\" \"thomas\" {\n    deleteheader \"from\";\n    addheader \"From\" \"no-reply@my.domain\";\n    redirect \"redirect@here.email\";\n    discard;\n}\n\nif envelope :localpart :is \"to\" \"bob\" {\n    redirect \"redirect@somewhere.email\";\n    discard;\n}\n\nif envelope :localpart :is \"to\" \"bill\" {\n    reject \"Bill cannot receive messages.\";\n    stop;\n}\n\nif envelope :localpart :is \"to\" \"jane\" {\n    set \"counter\" \"a\";\n    foreverypart {\n        if header :mime :contenttype \"content-type\" \"text/html\" {\n            extracttext :upper \"text_content\";\n            replace \"${text_content}\";\n        }\n        set :length \"part_num\" \"${counter}\";\n        addheader :last \"X-Part-Number\" \"${part_num}\";\n        set \"counter\" \"${counter}a\";\n    }\n}\n\nif envelope :domain :is \"to\" \"foobar.net\" {\n    notify \"mailto:john@example.net?cc=jane@example.org&subject=You%20have%20got%20mail\";\n}\n"
  },
  {
    "path": "tests/resources/smtp/sieve/stage_ehlo.sieve",
    "content": "require [\"variables\", \"extlists\", \"reject\"];\n\nif eval \"contains(['spammer.org', 'spammer.net'], env.helo_domain)\" {\n    reject \"551 5.1.1 Your domain '${env.helo_domain}' has been blocklisted.\";\n}\n"
  },
  {
    "path": "tests/resources/smtp/sieve/stage_mail.sieve",
    "content": "require [\"variables\", \"envelope\", \"reject\", \"vnd.stalwart.expressions\"];\n\nif envelope :localpart :is \"from\" \"spammer\" {\n    reject \"450 4.1.1 Invalid address\";\n}\n\neval \"query('sql', 'CREATE TABLE IF NOT EXISTS blocked_senders (addr TEXT PRIMARY KEY)', [])\";\neval \"query('sql', 'INSERT OR IGNORE INTO blocked_senders (addr) VALUES (?)', 'marketing@spam-domain.com')\";\n\nif eval \"query('sql', 'SELECT 1 FROM blocked_senders WHERE addr=? LIMIT 1', [envelope.from])\" {\n    reject \"Your address has been blocked.\";\n}\n\nif eval \"!is_local_domain('', 'localdomain.org') || is_local_domain('', 'other.org')\" {\n    let \"reason\" \"'result: ' + is_local_domain('', 'localdomain.org') + ' ' + is_local_domain('', 'other.org')\";\n    reject \"is_local_domain function failed: ${reason}\";\n}\n"
  },
  {
    "path": "tests/resources/smtp/sieve/stage_rcpt.sieve",
    "content": "require [\"variables\", \"envelope\", \"reject\", \"vnd.stalwart.expressions\"];\n\nif envelope :domain :is \"to\" \"foobar.org\" {\n    eval \"query('sql', 'CREATE TABLE IF NOT EXISTS greylist (addr TEXT PRIMARY KEY)', [])\";\n\n    set \"triplet\" \"${env.remote_ip}.${envelope.from}.${envelope.to}\";\n\n    if eval \"!query('sql', 'SELECT 1 FROM greylist WHERE addr=? LIMIT 1', [triplet])\" {\n        eval \"query('sql', 'INSERT INTO greylist (addr) VALUES (?)', [triplet])\";\n        reject \"422 4.2.2 You have been greylisted '${triplet}'.\";\n    }\n}\n"
  },
  {
    "path": "tests/resources/tls_cert.pem",
    "content": "-----BEGIN CERTIFICATE-----\nMIIFCTCCAvGgAwIBAgIUCgHGQYUqtelbHGVSzCVwBL3fyEUwDQYJKoZIhvcNAQEL\nBQAwFDESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTIyMDUxNjExNDAzNFoXDTIzMDUx\nNjExNDAzNFowFDESMBAGA1UEAwwJbG9jYWxob3N0MIICIjANBgkqhkiG9w0BAQEF\nAAOCAg8AMIICCgKCAgEAtwS0Fzl3SjaCuKEXgZ/fdWbDoj/qDphyNCAKNevQ0+D0\nSTNkWCO04aFSH0zcL8zoD9gokNos0i7OU9//ZhZQmex4V6EFdZn8bFwUWN/scUvW\nHEFXVjtHldO2isZgIxH9LuwRv7KAgkISuWahqerOVDhe7SeQUV0AJGNEh3cT9PZr\ngSY931BxB7n+5k8eoSk8Z1gtBzQzL62kVGpHDKfw8yX8m65owF9eLUBrNzgxmXfC\nxpuHwj7hmVhS09PPKeN/RsFS8PsYO7bo0u8jEKalteumjRT7RyUEbioqfo6ZFOGj\nFHPIq/uKXS9zN1fpoyNh3ur5hMznQhrqlwBM9KlM7GdBJ0pZ3ad0YjT8IL/GnGKR\n85J2WZdLqaQdUZo7nV67FhqdDlNE4MdwiykTMjfmLRXGAVhAzJHKyRKNwmkI2aqe\nS7aqeNgvuDBwY80Q9a2rb5py1Aw+L8yCkUBuHboToDpxSVRDNN8DrWNmmsXnxsOG\nwRDODy4GICKyxlP+RFSM8xWSQ6y9ktS2OfDBm+Eqcw+3pZKhdz2wgxLkUBJ8X1eh\nkJrCA/6LTuhy6m6mMjAfoSOFU7fu88jxaWPgvP7GKyH+LM/t9eucobz2ks5rtSjz\nV4Dc5DCS94/OpVRHwHdaFSPbJKBN9Ev8gnNrAyx/aBPGoHBPG/QUiU7dcUNIPt0C\nAwEAAaNTMFEwHQYDVR0OBBYEFI167IxBmErB11EqiPPqFLa31ZaMMB8GA1UdIwQY\nMBaAFI167IxBmErB11EqiPPqFLa31ZaMMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZI\nhvcNAQELBQADggIBALU00IOiH5ubEauVCmakms5ermNTZfculnhnDfWTLMeh2+a7\nG4cqADErfMhm/mmLbrw33t9s6tCAhQltvewKR40ST9uMPSyiQbYaCXd5DXnuI6Ox\nJtNW+UOWIaMf8abnkdLvREOvb8dVQS1i3xq14tAjY5XgpGwCPP8m54b7N3Q7soLn\ne5PDhPNTnhRIn2RLuYoZmQmMA5fcqEUDYff4epUww7PhrM1QckZligI3566NlGOf\nj1G9JrivBtY0eaJtamIFnGMBT0ThDudxVja2Nv0C2Elry0p4T/o4nc4M67BJ/y1R\nvjNLAgFhbxssemU3lZqSd+pykpJBwDBjFSPrZZmQcbk7H6Uz8V1xr/xuzfw6fA13\nNWZ5vLgP/DQ13sM+XFlxThKfbPMPVe/UCTvfGtNW+3XyBgPntEkR+fNEawQmzbYl\nR+X1ymT9MZnEZqRMf7/UD/SYek1aUJefoew3upjMgxYVvh4F8dqJ+39F+xoFzIA2\n1dDAEMzXtjA3zKhZ2cycZbEzpJvYA3eGLuR16Suqfi4kPvfwK0mOhCxQmpayt7/X\nvuEzW6dPCH8Hgbb0WvsSppGOvhdbDaZFNfFc5eNSxhyKzu3H3ACNImZRtZE+yixx\n0fR8+xz9kDLf8xupV+X9heyFGHSyYU2Lveaevtr2Ij3weLRgJ6LbNALoeKXk\n-----END CERTIFICATE-----\n"
  },
  {
    "path": "tests/resources/tls_privatekey.pem",
    "content": "-----BEGIN PRIVATE KEY-----\nMIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC3BLQXOXdKNoK4\noReBn991ZsOiP+oOmHI0IAo169DT4PRJM2RYI7ThoVIfTNwvzOgP2CiQ2izSLs5T\n3/9mFlCZ7HhXoQV1mfxsXBRY3+xxS9YcQVdWO0eV07aKxmAjEf0u7BG/soCCQhK5\nZqGp6s5UOF7tJ5BRXQAkY0SHdxP09muBJj3fUHEHuf7mTx6hKTxnWC0HNDMvraRU\nakcMp/DzJfybrmjAX14tQGs3ODGZd8LGm4fCPuGZWFLT088p439GwVLw+xg7tujS\n7yMQpqW166aNFPtHJQRuKip+jpkU4aMUc8ir+4pdL3M3V+mjI2He6vmEzOdCGuqX\nAEz0qUzsZ0EnSlndp3RiNPwgv8acYpHzknZZl0uppB1RmjudXrsWGp0OU0Tgx3CL\nKRMyN+YtFcYBWEDMkcrJEo3CaQjZqp5Ltqp42C+4MHBjzRD1ratvmnLUDD4vzIKR\nQG4duhOgOnFJVEM03wOtY2aaxefGw4bBEM4PLgYgIrLGU/5EVIzzFZJDrL2S1LY5\n8MGb4SpzD7elkqF3PbCDEuRQEnxfV6GQmsID/otO6HLqbqYyMB+hI4VTt+7zyPFp\nY+C8/sYrIf4sz+3165yhvPaSzmu1KPNXgNzkMJL3j86lVEfAd1oVI9skoE30S/yC\nc2sDLH9oE8agcE8b9BSJTt1xQ0g+3QIDAQABAoICABq5oxqpF5RMtXYEgAw7rkPU\nh8jPkHwlIrgd3Z/WGZ53APUXfhWo0ScJiZZsgNKyF0kJBZNxaI4gq5xv3zmnFIoF\nj+Ur7EIqBERGheoceMhqjI9/syMycNeeHM/S/ALjA5ewfT8C7+UVhOpx5DWNxidi\nO+phlp9q9zRZEo69grqIqVYooWxUsMyyCljTQOPDw8BLjfe5VagmsRJqmolslLDM\n4UBSjZVZ18S/3Wgo2oVQia660244BHWCAkZQbbXuNI2+eUAbSoSdxw3WQcaSrywL\nhzyezbqr2yPDIIVuiUgVUt0Ps0P57VCCN07jlYhvCEGnClysFzD+ATefoZ0wg7za\ndQu2E+d166rAjnssyhzcHMn3pxgSdtXD+dQR/xfIGbPABucCupEFqKmhLdMm9+ud\nlHay87qzMpIa8cITJwEQROfXqWAhNUU98pKCOx1SVXBqQC7QVqGQ5solDf0eMSVh\nngQ6Dz2WUI2ty75LteiFwlyTgnU9nyPN0NXsrMEET2BHWre7ufTQqiULtQ7+9BwH\nAMxEKvrQHjMUjdfbXuzdyc5w5mPYJZfFVSQ1HMslx66h9yCpRIsBZvUGvoaP8Tpe\nnQ66FTYRbiOkkdJ7k8DtrnhsJI1oOGjnvj/rvZ8D2pvrlJcIH2AyN3MOL8Jp5Oj1\nnCFt77TwpF92pgl0g9gBAoIBAQDcarmP54QboaIQ9S2gE/4gSVC5i44iDJuSRdI8\nK081RQcWiNzqQXTRc5nqJ7KzLyPiGlg+6rWsBKLos5l4t+MdhhH+KUvk/OtT/g8V\n0NZBNXLIbSb8j8ix4v3/f2qKHN3Co6QOlxb3gFvobKDdoKqUNiSH1zTZ8/Y/BzkM\njqWKhTdaLz6eyzhKfOTA4LO8kJ3VF8HUM1N9/e8Gjorl+gZpJUXUQS0+AIi8W76C\nOwDrVb3BPGVnApQJfWF78h4g20RwXrx/GYUW2vOMcLjXXDV5U7+nobPUoJnLxoZC\n16o88y0Ivan8dBNXsc1epyPvvEqp6MJbAyyVuNeuRJcgYA0BAoIBAQDUkGRV7fLG\nwCr5rNysUO+FKzVtTJnf9KEsqAqUmmVnG4oubxAJJtiB5n2+DT+CtO8Nrtz05BbR\nuxfWm+lbEw6lVMj63bywtp0NdULg7/2t+oq2Svv16KrZIRJttXMkdEiFFmkVAEhX\nl8Fyl6PJPfSMwbPdXEUPUAaNrXweVFffXczHc4W2G212ZzDB0z7QQSgEntbTDFB/\n2Cg5dvuojlM9zw0fuEyLwItZs7n16j/ONZLgBHyroMU9ZPxbnLrVyoZlqtob+RWm\nJu2fSIL9QqG6O4td1TqcUBGvFQYjGvKA+q5fsG26NBJ0Ac48cNK6PS4lMkN3Av2J\nccloYaMEHAXdAoIBAE8WMCy1Ok6byUXiYxOL+OPmyoM40q/e7DcovE2AkLQhZ3Cr\nfPDEucCphPFiexkV8f8fysgQeU0WgMmUH54UBPbD81LJyISKR3nkr875Ftdg8SV/\nHL0EblN9ifuR4U1bHCrJgoUFq2T09oVH7NR44Ju7bZIcIseNZK6qzcp2qGkycXD3\ngLWDX1hCxeV6+qLPFQKvuomEPRH4+jnVDXuFIaW6jPqixDP6BxXmqU2bFDJcmnBq\nVkwGvc1F4qORdUP+yOi05VeJdZqEx1x92aTUXg+BgEQKnjbNxUE7o1L6hQfHjUIU\no5iEoagWkQTEXf2YBwY+EPaNBgNWxnSuAbfJHwECggEBALOF95ezTVWauzD/U6ic\n+o3n/kl/Zn4FJ5KFodn7xCSe18d7uXlhO34KYqx+l+MWWMefpbGWacdcUjfImf93\nSulLgCqP12sP7/iLzp4XUpL7hOeM0NvRU2nqSpwpoUNqik0Mrlc0U+TWoGTduVCf\naMjwV65e3VyfY8mIeclLxqM5n1fcM1OoOnzDjiRE+0n7nYa5eAnq3pn6v4449TZY\nbelH03e0ucFWLtrltesBmj3YdWGJqJlzQOInRhNBfXJOh8+ZynfRmP0o54udPDQV\ncG3PGFd5XPTjkuvhv7sqaSGRlm/um92lWOhtFfdp+i+cuDpmByCef+7zEP19aKZx\n3GkCggEAFTs7KNMfvIEaLH0yQUFeq2gLmtcMofmOmeoIECycN1rG7iJo07lJLIs0\nbVODH8Z0kX8llu3cjGMAH/6R2uugJSxkmFiZKrngTzKmxDPvTCKWR4RFwXH9j8IO\ncPq7FtKN4SgrPy9ciAPdkcGmu3zz/sBKOaoPwvU2PdBRT+v/aoz+GCLXAvzFlKVe\n9/7zdg87ilo8+AtV+71EJeR3kyBPKS9JrWYUKfiams12+uuH4/53rMFZfNCAaZ3Z\n1sdXEO4o3Loc5TX4DbO9FVdBSBe6klEXx4T0QJboO6uBvTBnnRL2SQriJQQFwYT6\nXzVV5pwOxkIDBWDIqMUfwJDChBKfpw==\n-----END PRIVATE KEY-----\n"
  },
  {
    "path": "tests/src/cluster/broadcast.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::ClusterTest;\nuse crate::imap::idle;\nuse directory::backend::internal::{\n    PrincipalAction, PrincipalField, PrincipalUpdate, PrincipalValue,\n    manage::{ManageDirectory, UpdatePrincipal},\n};\nuse groupware::cache::GroupwareCache;\nuse std::net::IpAddr;\nuse types::collection::SyncCollection;\n\npub async fn test(cluster: &ClusterTest) {\n    println!(\"Running cluster broadcast tests...\");\n\n    // Run IMAP idle tests across nodes\n    let server1 = cluster.server(1);\n    let server2 = cluster.server(2);\n    let mut node1_client = cluster.imap_client(\"john\", 1).await;\n    let mut node2_client = cluster.imap_client(\"john\", 2).await;\n    idle::test(&mut node1_client, &mut node2_client, true).await;\n\n    // Test event broadcast\n    let test_ip: IpAddr = \"8.8.8.8\".parse().unwrap();\n    assert!(!server1.is_ip_blocked(&test_ip));\n    assert!(!server2.is_ip_blocked(&test_ip));\n    server1.block_ip(test_ip).await.unwrap();\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    assert!(server1.is_ip_blocked(&test_ip));\n    assert!(server2.is_ip_blocked(&test_ip));\n\n    // Change John's password and expect it to propagate\n    let account_id = cluster.account_id(\"john\");\n    assert!(server1.inner.cache.access_tokens.get(&account_id).is_some());\n    assert!(server2.inner.cache.access_tokens.get(&account_id).is_some());\n    let changes = server1\n        .core\n        .storage\n        .data\n        .update_principal(\n            UpdatePrincipal::by_id(account_id).with_updates(vec![PrincipalUpdate {\n                action: PrincipalAction::AddItem,\n                field: PrincipalField::Secrets,\n                value: PrincipalValue::String(\"hello\".into()),\n            }]),\n        )\n        .await\n        .unwrap();\n    server1.invalidate_principal_caches(changes).await;\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    assert!(server1.inner.cache.access_tokens.get(&account_id).is_none());\n    assert!(server2.inner.cache.access_tokens.get(&account_id).is_none());\n\n    // Rename John to Juan and expect DAV caches to be invalidated\n    let access_token = server1.get_access_token(account_id).await.unwrap();\n    server1\n        .fetch_dav_resources(&access_token, account_id, SyncCollection::Calendar)\n        .await\n        .unwrap();\n    server2\n        .fetch_dav_resources(&access_token, account_id, SyncCollection::Calendar)\n        .await\n        .unwrap();\n    assert!(server1.inner.cache.events.get(&account_id).is_some());\n    assert!(server2.inner.cache.events.get(&account_id).is_some());\n    let changes = server1\n        .core\n        .storage\n        .data\n        .update_principal(\n            UpdatePrincipal::by_id(account_id).with_updates(vec![PrincipalUpdate {\n                action: PrincipalAction::Set,\n                field: PrincipalField::Name,\n                value: PrincipalValue::String(\"juan\".into()),\n            }]),\n        )\n        .await\n        .unwrap();\n    server1.invalidate_principal_caches(changes).await;\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    assert!(server1.inner.cache.events.get(&account_id).is_none());\n    assert!(server2.inner.cache.events.get(&account_id).is_none());\n}\n"
  },
  {
    "path": "tests/src/cluster/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    AssertConfig, TEST_USERS, add_test_certs,\n    directory::internal::TestInternalDirectory,\n    imap::{ImapConnection, Type},\n    jmap::server::enterprise::EnterpriseCore,\n    store::cleanup::store_destroy,\n};\nuse ahash::AHashMap;\nuse common::{\n    Caches, Core, Data, Inner, Server,\n    config::{\n        server::{Listeners, ServerProtocol},\n        telemetry::Telemetry,\n    },\n    core::BuildServer,\n    manager::{\n        boot::build_ipc,\n        config::{ConfigManager, Patterns},\n    },\n};\nuse http::HttpSessionManager;\nuse imap::core::ImapSessionManager;\nuse imap_proto::ResponseType;\nuse jmap_client::client::{Client, Credentials};\nuse managesieve::core::ManageSieveSessionManager;\nuse pop3::Pop3SessionManager;\nuse services::{SpawnServices, broadcast::subscriber::spawn_broadcast_subscriber};\nuse smtp::{SpawnQueueManager, core::SmtpSessionManager};\nuse std::{path::PathBuf, sync::Arc, time::Duration};\nuse store::Stores;\nuse tokio::sync::watch;\nuse utils::config::Config;\n\npub mod broadcast;\npub mod stress;\n\npub const NUM_NODES: usize = 3;\n\n#[tokio::test(flavor = \"multi_thread\")]\npub async fn cluster_tests() {\n    let params = init_cluster_tests(true).await;\n    //stress::test(params.server.clone(), params.client).await;\n    broadcast::test(&params).await;\n}\n\n#[allow(dead_code)]\npub struct ClusterTest {\n    servers: Vec<Server>,\n    account_ids: AHashMap<String, u32>,\n    shutdown_txs: Vec<watch::Sender<bool>>,\n}\n\nasync fn init_cluster_tests(delete_if_exists: bool) -> ClusterTest {\n    // Load and parse config\n    let store_id = std::env::var(\"STORE\").expect(\n        \"Missing store type. Try running `STORE=<store_type> PUBSUB=<pubsub_type> cargo test`\",\n    );\n    let pubsub_id = std::env::var(\"PUBSUB\").expect(\n        \"Missing store type. Try running `STORE=<store_type> PUBSUB=<pubsub_type> cargo test`\",\n    );\n    let mut pubsub_config = match pubsub_id.as_str() {\n        \"nats\" => Config::new(SERVER_NATS).unwrap(),\n        \"redis\" => Config::new(SERVER_REDIS).unwrap(),\n        _ => panic!(\"Unsupported pubsub type: {}\", pubsub_id),\n    };\n\n    // Build configs\n    let mut configs = Vec::with_capacity(NUM_NODES);\n    for node_id in 0..NUM_NODES {\n        let mut config = Config::new(\n            add_test_certs(SERVER)\n                .replace(\"{STORE}\", &store_id)\n                .replace(\"{PUBSUB}\", &pubsub_id)\n                .replace(\"{NODE_ID}\", &node_id.to_string())\n                .replace(\n                    \"{LEVEL}\",\n                    &std::env::var(\"LOG\").unwrap_or_else(|_| \"disable\".to_string()),\n                ),\n        )\n        .unwrap();\n        config.resolve_all_macros().await;\n        configs.push(config);\n    }\n\n    // Build stores\n    let stores = Stores::parse_all(configs.first_mut().unwrap(), false).await;\n\n    // Build servers\n    let mut servers = Vec::with_capacity(NUM_NODES);\n    let mut shutdown_txs = Vec::with_capacity(NUM_NODES);\n    for config in configs {\n        let mut stores = stores.clone();\n        stores.pubsub_stores = Stores::parse(&mut pubsub_config).await.pubsub_stores;\n        let (server, shutdown_tx) = build_server(config, stores).await;\n        servers.push(server);\n        shutdown_txs.push(shutdown_tx);\n    }\n\n    let store = servers.first().unwrap().store().clone();\n    if delete_if_exists {\n        store_destroy(&store).await;\n    }\n\n    // Create test users\n    let mut account_ids = AHashMap::new();\n    for (account, secret, name, email) in TEST_USERS {\n        let account_id = store\n            .create_test_user(account, secret, name, &[email])\n            .await;\n        account_ids.insert(account.to_string(), account_id);\n    }\n\n    ClusterTest {\n        servers,\n        shutdown_txs,\n        account_ids,\n    }\n}\n\nimpl ClusterTest {\n    pub async fn jmap_client(&self, login: &str, node_id: u32) -> Client {\n        Client::new()\n            .credentials(Credentials::basic(login, find_account_secret(login)))\n            .timeout(Duration::from_secs(3600))\n            .accept_invalid_certs(true)\n            .connect(&format!(\"https://127.0.0.1:1800{node_id}\"))\n            .await\n            .unwrap()\n    }\n\n    pub async fn imap_client(&self, login: &str, node_id: u32) -> ImapConnection {\n        let mut conn = ImapConnection::connect_to(b\"A1 \", format!(\"127.0.0.1:1900{node_id}\")).await;\n        conn.assert_read(Type::Untagged, ResponseType::Ok).await;\n        conn.authenticate(login, find_account_secret(login)).await;\n        conn\n    }\n\n    pub fn server(&self, node_id: usize) -> &Server {\n        self.servers\n            .get(node_id)\n            .unwrap_or_else(|| panic!(\"No server found for node ID: {}\", node_id))\n    }\n\n    pub fn account_id(&self, login: &str) -> u32 {\n        self.account_ids\n            .get(login)\n            .cloned()\n            .unwrap_or_else(|| panic!(\"No account ID found for login: {}\", login))\n    }\n}\n\nfn find_account_secret(login: &str) -> &str {\n    TEST_USERS\n        .iter()\n        .find(|(account, _, _, _)| account == &login)\n        .map(|(_, secret, _, _)| secret)\n        .unwrap_or_else(|| panic!(\"No account found for login: {}\", login))\n}\n\nasync fn build_server(mut config: Config, stores: Stores) -> (Server, watch::Sender<bool>) {\n    // Parse servers\n    let mut servers = Listeners::parse(&mut config);\n\n    // Bind ports and drop privileges\n    servers.bind_and_drop_priv(&mut config);\n\n    // Parse core\n    let config_manager = ConfigManager {\n        cfg_local: Default::default(),\n        cfg_local_path: PathBuf::new(),\n        cfg_local_patterns: Patterns::parse(&mut config).into(),\n        cfg_store: config\n            .value(\"storage.data\")\n            .and_then(|id| stores.stores.get(id))\n            .cloned()\n            .unwrap_or_default(),\n    };\n    let tracers = Telemetry::parse(&mut config, &stores);\n    let core = Core::parse(&mut config, stores, config_manager)\n        .await\n        .enable_enterprise();\n    let data = Data::parse(&mut config);\n    let cache = Caches::parse(&mut config);\n    let (ipc, mut ipc_rxs) = build_ipc(true);\n    let inner = Arc::new(Inner {\n        shared_core: core.into_shared(),\n        data,\n        ipc,\n        cache,\n    });\n\n    // Parse acceptors\n    servers.parse_tcp_acceptors(&mut config, inner.clone());\n\n    // Enable tracing\n    tracers.enable(true);\n\n    // Start services\n    config.assert_no_errors();\n    ipc_rxs.spawn_queue_manager(inner.clone());\n    ipc_rxs.spawn_services(inner.clone());\n\n    // Spawn servers\n    let (shutdown_tx, shutdown_rx) = servers.spawn(|server, acceptor, shutdown_rx| {\n        match &server.protocol {\n            ServerProtocol::Smtp | ServerProtocol::Lmtp => server.spawn(\n                SmtpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Http => server.spawn(\n                HttpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Imap => server.spawn(\n                ImapSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Pop3 => server.spawn(\n                Pop3SessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::ManageSieve => server.spawn(\n                ManageSieveSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n        };\n    });\n\n    // Start broadcast subscriber\n    spawn_broadcast_subscriber(inner.clone(), shutdown_rx);\n\n    (inner.build_server(), shutdown_tx)\n}\n\nconst SERVER: &str = r#\"\n[server]\nhostname = \"'server{NODE_ID}.example.org'\"\n\n[http]\nurl = \"'https://127.0.0.1:800{NODE_ID}'\"\n\n[cluster]\nnode-id = {NODE_ID}\ncoordinator = \"{PUBSUB}\"\n\n[server.listener.http]\nbind = [\"127.0.0.1:1800{NODE_ID}\"]\nprotocol = \"http\"\nmax-connections = 81920\ntls.implicit = true\n\n[server.listener.imap]\nbind = [\"127.0.0.1:1900{NODE_ID}\"]\nprotocol = \"imap\"\nmax-connections = 81920\n\n[server.listener.lmtp]\nbind = ['127.0.0.1:1700{NODE_ID}']\nprotocol = 'lmtp'\ntls.implicit = false\n\n[server.socket]\nreuse-addr = true\n\n[server.tls]\nenable = true\nimplicit = false\ncertificate = \"default\"\n\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = [ { if = \"!is_empty(authenticated_as)\", then = true }, \n          { else = false } ]\ndirectory = \"'{STORE}'\"\n\n[session.rcpt.errors]\ntotal = 5\nwait = \"1ms\"\n\n[session.auth]\nmechanisms = \"[plain, login, oauthbearer]\"\ndirectory = \"'{STORE}'\"\n\n[resolver]\ntype = \"system\"\n\n[queue.strategy]\nroute = [ { if = \"rcpt_domain == 'example.com'\", then = \"'local'\" }, \n             { else = \"'mx'\" } ]\n\n[store.\"foundationdb\"]\ntype = \"foundationdb\"\n\n[store.\"postgresql\"]\ntype = \"postgresql\"\nhost = \"localhost\"\nport = 5432\ndatabase = \"stalwart\"\nuser = \"postgres\"\npassword = \"mysecretpassword\"\n\n[store.\"mysql\"]\ntype = \"mysql\"\nhost = \"localhost\"\nport = 3307\ndatabase = \"stalwart\"\nuser = \"root\"\npassword = \"password\"\n\n[certificate.default]\ncert = \"%{file:{CERT}}%\"\nprivate-key = \"%{file:{PK}}%\"\n\n[storage]\ndata = \"{STORE}\"\nfts = \"{STORE}\"\nblob = \"{STORE}\"\nlookup = \"{STORE}\"\ndirectory = \"{STORE}\"\n\n[directory.\"{STORE}\"]\ntype = \"internal\"\nstore = \"{STORE}\"\n\n[imap.auth]\nallow-plain-text = true\n\n[oauth]\nkey = \"parerga_und_paralipomena\"\n\n[spam-filter]\nenable = false\n\n[tracer.console]\ntype = \"console\"\nlevel = \"{LEVEL}\"\nmultiline = false\nansi = true\ndisabled-events = [\"network.*\", \"telemetry.webhook-error\", \"http.request-body\", \n                   \"eval.result\", \"store.*\", \"dkim.*\", \"queue.*\", \"delivery.*\",\n                   \"*.raw-input\", \"*.raw-output\" ]\n\"#;\n\nconst SERVER_NATS: &str = r#\"\n[store.\"nats\"]\ntype = \"nats\"\nurls = \"127.0.0.1:4444\"\n\"#;\n\nconst SERVER_REDIS: &str = r#\"\n[store.\"redis\"]\ntype = \"redis\"\nurls = \"redis://127.0.0.1\"\nredis-type = \"single\"\n\n\"#;\n"
  },
  {
    "path": "tests/src/cluster/stress.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{assert_is_empty, mail::mailbox::destroy_all_mailboxes_no_wait, wait_for_index};\nuse common::Server;\nuse directory::backend::internal::manage::ManageDirectory;\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    message::metadata::MessageData,\n};\nuse futures::future::join_all;\nuse jmap_client::{\n    client::Client,\n    core::set::{SetErrorType, SetObject},\n    mailbox::{self, Mailbox, Role},\n};\nuse std::{str::FromStr, sync::Arc, time::Duration};\nuse store::{\n    ValueKey,\n    rand::{self, Rng},\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive},\n};\nuse types::{collection::Collection, id::Id};\n\nconst TEST_USER_ID: u32 = 1;\nconst NUM_PASSES: usize = 1;\n\npub async fn test(server: Server, mut client: Client) {\n    println!(\"Running cluster concurrency stress tests...\");\n    server\n        .core\n        .storage\n        .data\n        .get_or_create_principal_id(\"john\", directory::Type::Individual)\n        .await\n        .unwrap();\n    client.set_default_account_id(Id::from(TEST_USER_ID).to_string());\n    let client = Arc::new(client);\n    email_tests(server.clone(), client.clone()).await;\n    mailbox_tests(server.clone(), client.clone()).await;\n}\n\nasync fn email_tests(server: Server, client: Arc<Client>) {\n    for pass in 0..NUM_PASSES {\n        println!(\n            \"----------------- EMAIL STRESS TEST {} -----------------\",\n            pass\n        );\n        let mailboxes = Arc::new(vec![\n            client\n                .mailbox_create(\"Stress 1\", None::<String>, Role::None)\n                .await\n                .unwrap()\n                .take_id(),\n            client\n                .mailbox_create(\"Stress 2\", None::<String>, Role::None)\n                .await\n                .unwrap()\n                .take_id(),\n            client\n                .mailbox_create(\"Stress 3\", None::<String>, Role::None)\n                .await\n                .unwrap()\n                .take_id(),\n        ]);\n        let mut futures = Vec::new();\n\n        for num in 0..1000 {\n            match rand::rng().random_range(0..3) {\n                0 => {\n                    let client = client.clone();\n                    let mailboxes = mailboxes.clone();\n                    futures.push(tokio::spawn(async move {\n                        let mailbox_num = rand::rng().random_range::<usize, _>(0..mailboxes.len());\n                        let _message_id = client\n                            .email_import(\n                                format!(\n                                    concat!(\n                                        \"From: test@test.com\\n\",\n                                        \"To: test@test.com\\r\\n\",\n                                        \"Subject: test {}\\r\\n\\r\\ntest {}\\r\\n\"\n                                    ),\n                                    num, num\n                                )\n                                .into_bytes(),\n                                [&mailboxes[mailbox_num]],\n                                None::<Vec<String>>,\n                                None,\n                            )\n                            .await\n                            .unwrap()\n                            .take_id();\n                        /*println!(\n                            \"Inserted message {}.\",\n                            Id::from_bytes(_message_id.as_bytes())\n                                .unwrap()\n                                .document_id()\n                        );*/\n                    }));\n                }\n\n                1 => {\n                    let client = client.clone();\n                    futures.push(tokio::spawn(async move {\n                        loop {\n                            let mut req = client.build();\n                            req.query_email();\n                            let ids = req.send_query_email().await.unwrap().take_ids();\n                            if !ids.is_empty() {\n                                let message_id = &ids[rand::rng().random_range(0..ids.len())];\n                                /*println!(\n                                    \"Deleting message {}.\",\n                                    Id::from_bytes(message_id.as_bytes()).unwrap().document_id()\n                                );*/\n                                match client.email_destroy(message_id).await {\n                                    Ok(_) => {\n                                        break;\n                                    }\n                                    Err(jmap_client::Error::Set(err)) => match err.error() {\n                                        SetErrorType::NotFound => {\n                                            break;\n                                        }\n                                        SetErrorType::Forbidden => {\n                                            // Concurrency issue, try again.\n                                            //println!(\"Concurrent update, trying again.\");\n                                        }\n                                        _ => {\n                                            panic!(\"Unexpected error: {:?}\", err);\n                                        }\n                                    },\n                                    Err(err) => {\n                                        panic!(\"Unexpected error: {:?}\", err);\n                                    }\n                                }\n                            } else {\n                                break;\n                            }\n                        }\n                    }));\n                }\n                _ => {\n                    let client = client.clone();\n                    let mailboxes = mailboxes.clone();\n                    futures.push(tokio::spawn(async move {\n                        let mut req = client.build();\n                        let ref_id = req.query_email().result_reference();\n                        req.get_email()\n                            .ids_ref(ref_id)\n                            .properties([jmap_client::email::Property::MailboxIds]);\n                        let emails = req\n                            .send()\n                            .await\n                            .unwrap()\n                            .unwrap_method_responses()\n                            .pop()\n                            .unwrap()\n                            .unwrap_get_email()\n                            .unwrap()\n                            .take_list();\n\n                        if !emails.is_empty() {\n                            let message = &emails[rand::rng().random_range(0..emails.len())];\n                            let message_id = message.id().unwrap();\n                            let mailbox_ids = message.mailbox_ids();\n                            assert_eq!(mailbox_ids.len(), 1, \"{:#?}\", message);\n                            let mailbox_id = mailbox_ids.last().unwrap();\n                            loop {\n                                let new_mailbox_id =\n                                    &mailboxes[rand::rng().random_range(0..mailboxes.len())];\n                                if new_mailbox_id != mailbox_id {\n                                    /*println!(\n                                        \"Moving message {} from {} to {}.\",\n                                        Id::from_bytes(message_id.as_bytes())\n                                            .unwrap()\n                                            .document_id(),\n                                        Id::from_bytes(mailbox_id.as_bytes())\n                                            .unwrap()\n                                            .document_id(),\n                                        Id::from_bytes(new_mailbox_id.as_bytes())\n                                            .unwrap()\n                                            .document_id()\n                                    );*/\n                                    let mut req = client.build();\n                                    req.set_email()\n                                        .update(message_id)\n                                        .mailbox_ids([new_mailbox_id]);\n                                    req.send_set_email().await.unwrap();\n\n                                    break;\n                                }\n                            }\n                        }\n                    }));\n                }\n            }\n            tokio::time::sleep(Duration::from_millis(rand::rng().random_range(5..10))).await;\n        }\n\n        join_all(futures).await;\n\n        let cache = server.get_cached_messages(TEST_USER_ID).await.unwrap();\n        let email_ids = cache\n            .emails\n            .items\n            .iter()\n            .map(|e| e.document_id)\n            .collect::<RoaringBitmap>();\n        let mailbox_ids = cache\n            .mailboxes\n            .items\n            .iter()\n            .map(|m| m.document_id)\n            .collect::<RoaringBitmap>();\n        assert_eq!(mailbox_ids.len(), 8);\n\n        for mailbox in mailboxes.iter() {\n            let mailbox_id = Id::from_str(mailbox).unwrap().document_id();\n            let email_ids_in_mailbox =\n                RoaringBitmap::from_iter(cache.in_mailbox(mailbox_id).map(|m| m.document_id));\n            let mut email_ids_check = email_ids_in_mailbox.clone();\n            email_ids_check &= &email_ids;\n            assert_eq!(email_ids_in_mailbox, email_ids_check);\n\n            //println!(\"Emails {:?}\", email_ids_in_mailbox);\n\n            for email_id in &email_ids_in_mailbox {\n                if let Some(mailbox_tags) = server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        TEST_USER_ID,\n                        Collection::Email,\n                        email_id,\n                    ))\n                    .await\n                    .unwrap()\n                {\n                    let mailbox_tags = mailbox_tags.deserialize::<MessageData>().unwrap().mailboxes;\n                    if mailbox_tags.len() != 1 {\n                        panic!(\n                            \"Email ORM has more than one mailbox {:?}! Id {} in mailbox {} with messages {:?}\",\n                            mailbox_tags, email_id, mailbox_id, email_ids_in_mailbox\n                        );\n                    }\n                    let mailbox_tag = mailbox_tags[0];\n                    assert!(mailbox_tag.uid != 0);\n                    if mailbox_tag.mailbox_id != mailbox_id {\n                        panic!(\n                            concat!(\n                                \"Email ORM has an unexpected mailbox tag {:?}! Id {} in \",\n                                \"mailbox {} with messages {:?}\"\n                            ),\n                            mailbox_tag, email_id, mailbox_id, email_ids_in_mailbox,\n                        );\n                    }\n                } else {\n                    panic!(\n                        \"Email tags not found! Id {} in mailbox {} with messages {:?}\",\n                        email_id, mailbox_id, email_ids_in_mailbox\n                    );\n                }\n            }\n        }\n\n        wait_for_index(&server).await;\n        destroy_all_mailboxes_no_wait(&client).await;\n        assert_is_empty(&server).await;\n    }\n}\n\nasync fn mailbox_tests(server: Server, client: Arc<Client>) {\n    let mailboxes = Arc::new(vec![\n        \"test/test1/test2/test3\".to_string(),\n        \"test1/test2/test3\".to_string(),\n        \"test2/test3/test4\".to_string(),\n        \"test3/test4/test5\".to_string(),\n        \"test4\".to_string(),\n        \"test5\".to_string(),\n    ]);\n    let mut futures = Vec::new();\n\n    println!(\"----------------- MAILBOX STRESS TEST -----------------\");\n\n    for _ in 0..1000 {\n        match rand::rng().random_range(0..=3) {\n            0 => {\n                for pos in 0..mailboxes.len() {\n                    let client = client.clone();\n                    let mailboxes = mailboxes.clone();\n                    futures.push(tokio::spawn(async move {\n                        //println!(\"Creating mailbox {}.\", mailboxes[pos]);\n                        create_mailbox(&client, &mailboxes[pos]).await;\n                    }));\n                }\n            }\n\n            1 => {\n                let client = client.clone();\n                futures.push(tokio::spawn(async move {\n                    //print!(\"Querying mailboxes...\");\n                    query_mailboxes(&client).await;\n                }));\n            }\n\n            2 => {\n                let client = client.clone();\n                futures.push(tokio::spawn(async move {\n                    for mailbox_id in client\n                        .mailbox_query(None::<mailbox::query::Filter>, None::<Vec<_>>)\n                        .await\n                        .unwrap()\n                        .take_ids()\n                    {\n                        let client = client.clone();\n                        tokio::spawn(async move {\n                            //println!(\"Deleting mailbox {}.\", mailbox_id);\n                            delete_mailbox(&client, &mailbox_id).await;\n                        });\n                    }\n                }));\n            }\n\n            _ => {\n                let client = client.clone();\n                futures.push(tokio::spawn(async move {\n                    let mut ids = client\n                        .mailbox_query(None::<mailbox::query::Filter>, None::<Vec<_>>)\n                        .await\n                        .unwrap()\n                        .take_ids();\n                    if !ids.is_empty() {\n                        let id = ids.swap_remove(rand::rng().random_range(0..ids.len()));\n                        let sort_order = rand::rng().random_range(0..100);\n                        //println!(\"Updating mailbox {}.\", id);\n                        client.mailbox_update_sort_order(&id, sort_order).await.ok();\n                    }\n                }));\n            }\n        }\n        tokio::time::sleep(Duration::from_millis(rand::rng().random_range(5..10))).await;\n    }\n\n    join_all(futures).await;\n\n    wait_for_index(&server).await;\n    for mailbox_id in client\n        .mailbox_query(None::<mailbox::query::Filter>, None::<Vec<_>>)\n        .await\n        .unwrap()\n        .take_ids()\n    {\n        let _ = client.mailbox_move(&mailbox_id, None::<String>).await;\n    }\n    for mailbox_id in client\n        .mailbox_query(None::<mailbox::query::Filter>, None::<Vec<_>>)\n        .await\n        .unwrap()\n        .take_ids()\n    {\n        let _ = client.mailbox_destroy(&mailbox_id, true).await;\n    }\n    assert_is_empty(&server).await;\n}\n\nasync fn create_mailbox(client: &Client, mailbox: &str) -> Vec<String> {\n    let mut request = client.build();\n    let mut create_ids: Vec<String> = Vec::new();\n    let set_request = request.set_mailbox();\n    for path_item in mailbox.split('/') {\n        let create_item = set_request.create().name(path_item);\n        if let Some(create_id) = create_ids.last() {\n            create_item.parent_id_ref(create_id);\n        }\n        create_ids.push(create_item.create_id().unwrap());\n    }\n    let mut response = request.send_set_mailbox().await.unwrap();\n    let mut ids = Vec::with_capacity(create_ids.len());\n    for create_id in create_ids {\n        if let Ok(mut id) = response.created(&create_id) {\n            ids.push(id.take_id());\n        }\n    }\n    ids\n}\n\nasync fn query_mailboxes(client: &Client) -> Vec<Mailbox> {\n    let mut request = client.build();\n    let query_result = request\n        .query_mailbox()\n        .calculate_total(true)\n        .result_reference();\n    request.get_mailbox().ids_ref(query_result).properties([\n        jmap_client::mailbox::Property::Id,\n        jmap_client::mailbox::Property::Name,\n        jmap_client::mailbox::Property::IsSubscribed,\n        jmap_client::mailbox::Property::ParentId,\n        jmap_client::mailbox::Property::Role,\n        jmap_client::mailbox::Property::TotalEmails,\n        jmap_client::mailbox::Property::UnreadEmails,\n    ]);\n\n    request\n        .send()\n        .await\n        .unwrap()\n        .unwrap_method_responses()\n        .pop()\n        .unwrap()\n        .unwrap_get_mailbox()\n        .unwrap()\n        .take_list()\n}\n\nasync fn delete_mailbox(client: &Client, mailbox_id: &str) {\n    for _ in 0..3 {\n        match client.mailbox_destroy(mailbox_id, true).await {\n            Ok(_) => return,\n            Err(err) => match err {\n                jmap_client::Error::Set(_) => break,\n                jmap_client::Error::Transport(_) => {\n                    let backoff = rand::rng().random_range(50..=300);\n                    tokio::time::sleep(Duration::from_millis(backoff)).await;\n                }\n                _ => panic!(\"Failed: {:?}\", err),\n            },\n        }\n    }\n    /*println!(\n        \"Warning: Too many transport errors while deleting mailbox {}.\",\n        mailbox_id\n    );*/\n}\n"
  },
  {
    "path": "tests/src/directory/imap.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse common::listener::limiter::{ConcurrencyLimiter, InFlight};\nuse directory::QueryParams;\nuse mail_parser::decoders::base64::base64_decode;\nuse mail_send::Credentials;\nuse tokio::{\n    io::{AsyncReadExt, AsyncWriteExt},\n    net::{TcpListener, TcpStream},\n    sync::watch,\n};\nuse tokio_rustls::TlsAcceptor;\n\nuse crate::directory::{DirectoryTest, Item, LookupResult};\n\nuse super::dummy_tls_acceptor;\n\n#[tokio::test]\nasync fn imap_directory() {\n    // Enable logging\n    /*tracing::subscriber::set_global_default(\n        tracing_subscriber::FmtSubscriber::builder()\n            .with_max_level(tracing::Level::DEBUG)\n            .finish(),\n    )\n    .unwrap();*/\n\n    // Spawn mock LMTP server\n    let shutdown = spawn_mock_imap_server(5);\n    tokio::time::sleep(std::time::Duration::from_millis(100)).await;\n\n    // Obtain directory handle\n    let mut config = DirectoryTest::new(None).await;\n    let handle = config.directories.directories.remove(\"imap\").unwrap();\n\n    // Basic lookup\n    let tests = vec![\n        (\n            Item::Authenticate(Credentials::Plain {\n                username: \"john\".to_string(),\n                secret: \"ok\".to_string(),\n            }),\n            LookupResult::True,\n        ),\n        (\n            Item::Authenticate(Credentials::Plain {\n                username: \"john\".to_string(),\n                secret: \"bad\".to_string(),\n            }),\n            LookupResult::False,\n        ),\n    ];\n\n    for (item, expected) in &tests {\n        assert_eq!(\n            &LookupResult::from(\n                handle\n                    .query(\n                        QueryParams::credentials(item.as_credentials()).with_return_member_of(true)\n                    )\n                    .await\n                    .unwrap()\n                    .is_some()\n            ),\n            expected\n        );\n    }\n\n    // Concurrent requests\n    let mut requests = Vec::new();\n    for n in 0..10 {\n        let (item, expected) = &tests[n % tests.len()];\n        let item = item.append(n);\n        let item_clone = item.clone();\n        let handle = handle.clone();\n        requests.push((\n            tokio::spawn(async move {\n                LookupResult::from(\n                    handle\n                        .query(\n                            QueryParams::credentials(item.as_credentials())\n                                .with_return_member_of(true),\n                        )\n                        .await\n                        .unwrap()\n                        .is_some(),\n                )\n            }),\n            item_clone,\n            expected.append(n),\n        ));\n    }\n    for (result, item, expected_result) in requests {\n        assert_eq!(\n            result.await.unwrap(),\n            expected_result,\n            \"Failed for {item:?}\"\n        );\n    }\n\n    // Shutdown\n    shutdown.send(false).ok();\n}\n\npub fn spawn_mock_imap_server(max_concurrency: u64) -> watch::Sender<bool> {\n    let (tx, mut rx) = watch::channel(true);\n\n    tokio::spawn(async move {\n        let listener = TcpListener::bind(\"127.0.0.1:9198\")\n            .await\n            .unwrap_or_else(|e| {\n                panic!(\"Failed to bind mock IMAP server to 127.0.0.1:9198: {e}\");\n            });\n        let acceptor = dummy_tls_acceptor();\n        let limited = ConcurrencyLimiter::new(max_concurrency);\n        loop {\n            tokio::select! {\n                stream = listener.accept() => {\n                    match stream {\n                        Ok((stream, _)) => {\n                            //println!(\"--- Accepted connection --- \");\n                            let acceptor = acceptor.clone();\n                            let in_flight = limited.is_allowed();\n                            tokio::spawn(accept_imap(stream, acceptor, in_flight.into()));\n                        }\n                        Err(err) => {\n                            panic!(\"Something went wrong: {err}\" );\n                        }\n                    }\n                },\n                _ = rx.changed() => {\n                    break;\n                }\n            };\n        }\n    });\n\n    tx\n}\n\nasync fn accept_imap(stream: TcpStream, acceptor: Arc<TlsAcceptor>, in_flight: Option<InFlight>) {\n    let mut stream = acceptor.accept(stream).await.unwrap();\n    stream\n        .write_all(b\"* OK Clueless host service ready\\r\\n\")\n        .await\n        .unwrap();\n\n    if in_flight.is_none() {\n        eprintln!(\"WARNING: Concurrency exceeded!\");\n    }\n\n    let mut buf_u8 = vec![0u8; 1024];\n\n    while let Ok(br) = stream.read(&mut buf_u8).await {\n        let buf = std::str::from_utf8(&buf_u8[0..br]).unwrap();\n        let (op, buf) = buf.split_once(' ').unwrap();\n\n        //print!(\"-> {}\", buf);\n        let response = if buf.starts_with(\"CAPABILITY\") {\n            format!(\n                \"* CAPABILITY IMAP4rev2 IMAP4rev1 AUTH=PLAIN\\r\\n{op} OK CAPABILITY completed\\r\\n\",\n            )\n        } else if buf.starts_with(\"NOOP\") {\n            format!(\"{op} OK NOOP completed\\r\\n\")\n        } else if buf.starts_with(\"AUTHENTICATE PLAIN\") {\n            let buf = base64_decode(buf.rsplit_once(' ').unwrap().1.as_bytes()).unwrap();\n            if String::from_utf8_lossy(&buf).contains(\"ok\") {\n                format!(\"{op} OK Great success!\\r\\n\")\n            } else {\n                format!(\"{op} BAD No soup for you!\\r\\n\")\n            }\n        } else if buf.starts_with(\"LOGOUT\") {\n            format!(\"* BYE\\r\\n{op} OK LOGOUT completed\\r\\n\")\n        } else {\n            panic!(\"Unknown command: {}\", buf.trim());\n        };\n        //print!(\"<- {}\", response);\n        for line in response.split_inclusive('\\n') {\n            stream.write_all(line.as_bytes()).await.unwrap();\n            stream.flush().await.unwrap();\n            tokio::time::sleep(std::time::Duration::from_millis(100)).await;\n        }\n\n        if buf.contains(\"bye\") || buf.starts_with(\"LOGOUT\") {\n            return;\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/directory/internal.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse crate::{\n    directory::{DirectoryTest, IntoTestPrincipal, TestPrincipal},\n    store::cleanup::{store_assert_is_empty, store_destroy},\n};\nuse ahash::AHashSet;\nuse common::{Core, Inner, Server, config::storage::Storage};\nuse directory::{\n    Permission, QueryBy, QueryParams, Type,\n    backend::{\n        RcptType,\n        internal::{\n            PrincipalField, PrincipalSet, PrincipalUpdate, PrincipalValue,\n            lookup::DirectoryStore,\n            manage::{self, ChangedPrincipals, ManageDirectory, UpdatePrincipal},\n        },\n    },\n};\nuse http::management::stores::destroy_account_data;\nuse mail_send::Credentials;\nuse store::{\n    IterateParams, Store, ValueKey,\n    write::{BatchBuilder, ValueClass},\n};\nuse types::collection::Collection;\n\n#[tokio::test]\nasync fn internal_directory() {\n    let config = DirectoryTest::new(None).await;\n\n    for (store_id, store) in config.stores.stores {\n        println!(\"Testing internal directory with store {:?}\", store_id);\n        store_destroy(&store).await;\n\n        // A principal without name should fail\n        assert_eq!(\n            store\n                .create_principal(PrincipalSet::default(), None, None)\n                .await,\n            Err(manage::err_missing(PrincipalField::Name))\n        );\n\n        // Basic account creation\n        let john_id = store\n            .create_principal(\n                TestPrincipal {\n                    name: \"john\".into(),\n                    description: Some(\"John Doe\".into()),\n                    secrets: vec![\"secret\".into(), \"$app$secret2\".into()],\n                    ..Default::default()\n                }\n                .into(),\n                None,\n                None,\n            )\n            .await\n            .unwrap()\n            .id;\n\n        // Two accounts with the same name should fail\n        assert_eq!(\n            store\n                .create_principal(\n                    TestPrincipal {\n                        name: \"john\".into(),\n                        ..Default::default()\n                    }\n                    .into(),\n                    None,\n                    None\n                )\n                .await,\n            Err(manage::err_exists(PrincipalField::Name, \"john\"))\n        );\n\n        // An account using a non-existent domain should fail\n        assert_eq!(\n            store\n                .create_principal(\n                    TestPrincipal {\n                        name: \"jane\".into(),\n                        emails: vec![\"jane@example.org\".into()],\n                        ..Default::default()\n                    }\n                    .into(),\n                    None,\n                    None\n                )\n                .await,\n            Err(manage::not_found(\"example.org\"))\n        );\n\n        // Create a domain name\n        store\n            .create_principal(\n                TestPrincipal {\n                    name: \"example.org\".into(),\n                    typ: Type::Domain,\n                    ..Default::default()\n                }\n                .into(),\n                None,\n                None,\n            )\n            .await\n            .unwrap();\n        assert!(store.is_local_domain(\"example.org\").await.unwrap());\n        assert!(!store.is_local_domain(\"otherdomain.org\").await.unwrap());\n\n        // Add an email address\n        assert!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john\").with_updates(vec![\n                    PrincipalUpdate::add_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(\"john@example.org\".into()),\n                    )\n                ]))\n                .await\n                .is_ok()\n        );\n        assert_eq!(\n            store.rcpt(\"john@example.org\").await.unwrap(),\n            RcptType::Mailbox\n        );\n        assert_eq!(\n            store.email_to_id(\"john@example.org\").await.unwrap(),\n            Some(john_id)\n        );\n\n        // Using non-existent domain should fail\n        assert_eq!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john\").with_updates(vec![\n                    PrincipalUpdate::add_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(\"john@otherdomain.org\".into()),\n                    )\n                ]))\n                .await,\n            Err(manage::not_found(\"otherdomain.org\"))\n        );\n\n        // Create an account with an email address\n        let jane_id = store\n            .create_principal(\n                TestPrincipal {\n                    name: \"jane\".into(),\n                    description: Some(\"Jane Doe\".into()),\n                    secrets: vec![\"my_secret\".into(), \"$app$my_secret2\".into()],\n                    emails: vec![\"jane@example.org\".into()],\n                    quota: 123,\n                    ..Default::default()\n                }\n                .into(),\n                None,\n                None,\n            )\n            .await\n            .unwrap()\n            .id;\n\n        assert_eq!(\n            store.rcpt(\"jane@example.org\").await.unwrap(),\n            RcptType::Mailbox\n        );\n        assert_eq!(\n            store.rcpt(\"jane@otherdomain.org\").await.unwrap(),\n            RcptType::Invalid\n        );\n        assert_eq!(\n            store.email_to_id(\"jane@example.org\").await.unwrap(),\n            Some(jane_id)\n        );\n        assert_eq!(store.vrfy(\"jane\").await.unwrap(), vec![\"jane@example.org\"]);\n        assert_eq!(\n            store\n                .query(\n                    QueryParams::credentials(&Credentials::new(\"jane\".into(), \"my_secret\".into()))\n                        .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .map(|p| p.into_test()),\n            Some(TestPrincipal {\n                id: jane_id,\n                name: \"jane\".into(),\n                description: Some(\"Jane Doe\".into()),\n                emails: vec![\"jane@example.org\".into()],\n                secrets: vec![\"my_secret\".into(), \"$app$my_secret2\".into()],\n                quota: 123,\n                ..Default::default()\n            })\n        );\n        assert_eq!(\n            store\n                .query(\n                    QueryParams::credentials(&Credentials::new(\n                        \"jane\".into(),\n                        \"wrong_password\".into()\n                    ))\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap(),\n            None\n        );\n\n        // Duplicate email address should fail\n        assert_eq!(\n            store\n                .create_principal(\n                    TestPrincipal {\n                        name: \"janeth\".into(),\n                        description: Some(\"Janeth Doe\".into()),\n                        emails: vec![\"jane@example.org\".into()],\n                        ..Default::default()\n                    }\n                    .into(),\n                    None,\n                    None\n                )\n                .await,\n            Err(manage::err_exists(\n                PrincipalField::Emails,\n                \"jane@example.org\"\n            ))\n        );\n\n        // Create a mailing list\n        let list_id = store\n            .create_principal(\n                TestPrincipal {\n                    name: \"list\".into(),\n                    typ: Type::List,\n                    emails: vec![\"list@example.org\".into()],\n                    ..Default::default()\n                }\n                .into(),\n                None,\n                None,\n            )\n            .await\n            .unwrap()\n            .id;\n        assert!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"list\").with_updates(vec![\n                    PrincipalUpdate::set(\n                        PrincipalField::Members,\n                        PrincipalValue::StringList(vec![\"john\".into(), \"jane\".into()]),\n                    ),\n                    PrincipalUpdate::set(\n                        PrincipalField::ExternalMembers,\n                        PrincipalValue::StringList(vec![\n                            \"mike@other.org\".into(),\n                            \"lucy@foobar.net\".into()\n                        ]),\n                    )\n                ]))\n                .await\n                .is_ok()\n        );\n\n        assert_list_members(\n            &store,\n            \"list@example.org\",\n            [\n                \"john@example.org\",\n                \"mike@other.org\",\n                \"lucy@foobar.net\",\n                \"jane@example.org\",\n            ],\n        )\n        .await;\n\n        assert_eq!(\n            store\n                .query(QueryParams::name(\"list\").with_return_member_of(true))\n                .await\n                .unwrap()\n                .unwrap()\n                .into_test(),\n            TestPrincipal {\n                name: \"list\".into(),\n                id: list_id,\n                typ: Type::List,\n                emails: vec![\"list@example.org\".into()],\n                ..Default::default()\n            }\n        );\n        assert_eq!(\n            store\n                .expn(\"list@example.org\")\n                .await\n                .unwrap()\n                .into_iter()\n                .collect::<AHashSet<_>>(),\n            [\n                \"john@example.org\",\n                \"mike@other.org\",\n                \"lucy@foobar.net\",\n                \"jane@example.org\"\n            ]\n            .into_iter()\n            .map(|s| s.into())\n            .collect::<AHashSet<_>>()\n        );\n\n        // Create groups\n        store\n            .create_principal(\n                TestPrincipal {\n                    name: \"sales\".into(),\n                    description: Some(\"Sales Team\".into()),\n                    typ: Type::Group,\n                    ..Default::default()\n                }\n                .into(),\n                None,\n                None,\n            )\n            .await\n            .unwrap();\n        store\n            .create_principal(\n                TestPrincipal {\n                    name: \"support\".into(),\n                    description: Some(\"Support Team\".into()),\n                    typ: Type::Group,\n                    ..Default::default()\n                }\n                .into(),\n                None,\n                None,\n            )\n            .await\n            .unwrap();\n\n        // Add John to the Sales and Support groups\n        assert!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john\").with_updates(vec![\n                    PrincipalUpdate::add_item(\n                        PrincipalField::MemberOf,\n                        PrincipalValue::String(\"sales\".into()),\n                    ),\n                    PrincipalUpdate::add_item(\n                        PrincipalField::MemberOf,\n                        PrincipalValue::String(\"support\".into()),\n                    )\n                ]))\n                .await\n                .is_ok()\n        );\n        let principal = store\n            .query(QueryParams::name(\"john\").with_return_member_of(true))\n            .await\n            .unwrap()\n            .unwrap();\n        let principal = store.map_principal(principal, &[]).await.unwrap();\n        assert_eq!(\n            principal.into_test().into_sorted(),\n            TestPrincipal {\n                id: john_id,\n                name: \"john\".into(),\n                description: Some(\"John Doe\".into()),\n                secrets: vec![\"secret\".into(), \"$app$secret2\".into()],\n                emails: vec![\"john@example.org\".into()],\n                member_of: vec![\"sales\".into(), \"support\".into()],\n                lists: vec![\"list\".into()],\n                ..Default::default()\n            }\n        );\n\n        // Adding a non-existent user should fail\n        assert_eq!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john\").with_updates(vec![\n                    PrincipalUpdate::add_item(\n                        PrincipalField::MemberOf,\n                        PrincipalValue::String(\"accounting\".into()),\n                    )\n                ]))\n                .await,\n            Err(manage::not_found(\"accounting\"))\n        );\n\n        // Remove a member from a group\n        assert!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john\").with_updates(vec![\n                    PrincipalUpdate::remove_item(\n                        PrincipalField::MemberOf,\n                        PrincipalValue::String(\"support\".into()),\n                    )\n                ]))\n                .await\n                .is_ok()\n        );\n        let principal = store\n            .query(QueryParams::name(\"john\").with_return_member_of(true))\n            .await\n            .unwrap()\n            .unwrap();\n        let principal = store.map_principal(principal, &[]).await.unwrap();\n        assert_eq!(\n            principal.into_test().into_sorted(),\n            TestPrincipal {\n                id: john_id,\n                name: \"john\".into(),\n                description: Some(\"John Doe\".into()),\n                secrets: vec![\"secret\".into(), \"$app$secret2\".into()],\n                emails: vec![\"john@example.org\".into()],\n                member_of: vec![\"sales\".into()],\n                lists: vec![\"list\".into()],\n                ..Default::default()\n            }\n        );\n\n        // Update multiple fields\n        assert!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john\").with_updates(vec![\n                    PrincipalUpdate::set(\n                        PrincipalField::Name,\n                        PrincipalValue::String(\"john.doe\".into())\n                    ),\n                    PrincipalUpdate::set(\n                        PrincipalField::Description,\n                        PrincipalValue::String(\"Johnny Doe\".into())\n                    ),\n                    PrincipalUpdate::set(\n                        PrincipalField::Secrets,\n                        PrincipalValue::StringList(vec![\"12345\".into()])\n                    ),\n                    PrincipalUpdate::set(PrincipalField::Quota, PrincipalValue::Integer(1024)),\n                    PrincipalUpdate::remove_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(\"john@example.org\".into()),\n                    ),\n                    PrincipalUpdate::add_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(\"john.doe@example.org\".into()),\n                    )\n                ]))\n                .await\n                .is_ok()\n        );\n\n        let principal = store\n            .query(QueryParams::name(\"john.doe\").with_return_member_of(true))\n            .await\n            .unwrap()\n            .unwrap();\n        let principal = store.map_principal(principal, &[]).await.unwrap();\n        assert_eq!(\n            principal.into_test().into_sorted(),\n            TestPrincipal {\n                id: john_id,\n                name: \"john.doe\".into(),\n                description: Some(\"Johnny Doe\".into()),\n                secrets: vec![\"12345\".into()],\n                emails: vec![\"john.doe@example.org\".into()],\n                quota: 1024,\n                typ: Type::Individual,\n                member_of: vec![\"sales\".into()],\n                lists: vec![\"list\".into()],\n                ..Default::default()\n            }\n        );\n        assert_eq!(store.get_principal_id(\"john\").await.unwrap(), None);\n        assert_eq!(\n            store.rcpt(\"john@example.org\").await.unwrap(),\n            RcptType::Invalid\n        );\n        assert_eq!(\n            store.rcpt(\"john.doe@example.org\").await.unwrap(),\n            RcptType::Mailbox\n        );\n\n        // Remove a member from a mailing list and then add it back\n        assert!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"list\").with_updates(vec![\n                    PrincipalUpdate::remove_item(\n                        PrincipalField::Members,\n                        PrincipalValue::String(\"john.doe\".into()),\n                    )\n                ]))\n                .await\n                .is_ok()\n        );\n        assert_list_members(\n            &store,\n            \"list@example.org\",\n            [\"jane@example.org\", \"mike@other.org\", \"lucy@foobar.net\"],\n        )\n        .await;\n        assert!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"list\").with_updates(vec![\n                    PrincipalUpdate::add_item(\n                        PrincipalField::Members,\n                        PrincipalValue::String(\"john.doe\".into()),\n                    )\n                ]))\n                .await\n                .is_ok()\n        );\n        assert_list_members(\n            &store,\n            \"list@example.org\",\n            [\n                \"john.doe@example.org\",\n                \"jane@example.org\",\n                \"mike@other.org\",\n                \"lucy@foobar.net\",\n            ],\n        )\n        .await;\n\n        // Field validation\n        assert_eq!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john.doe\").with_updates(vec![\n                    PrincipalUpdate::set(\n                        PrincipalField::Name,\n                        PrincipalValue::String(\"jane\".into())\n                    ),\n                ]))\n                .await,\n            Err(manage::err_exists(PrincipalField::Name, \"jane\"))\n        );\n        assert_eq!(\n            store\n                .update_principal(UpdatePrincipal::by_name(\"john.doe\").with_updates(vec![\n                    PrincipalUpdate::add_item(\n                        PrincipalField::Emails,\n                        PrincipalValue::String(\"jane@example.org\".into())\n                    ),\n                ]))\n                .await,\n            Err(manage::err_exists(\n                PrincipalField::Emails,\n                \"jane@example.org\"\n            ))\n        );\n\n        // List accounts\n        assert_eq!(\n            store\n                .list_principals(\n                    None,\n                    None,\n                    &[Type::Individual, Type::Group, Type::List],\n                    true,\n                    0,\n                    0\n                )\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<AHashSet<_>>(),\n            [\"jane\", \"john.doe\", \"list\", \"sales\", \"support\"]\n                .into_iter()\n                .map(|s| s.into())\n                .collect::<AHashSet<_>>()\n        );\n        assert_eq!(\n            store\n                .list_principals(\"john\".into(), None, &[], true, 0, 0)\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<Vec<_>>(),\n            vec![\"john.doe\"]\n        );\n        assert_eq!(\n            store\n                .list_principals(None, None, &[Type::Individual], true, 0, 0)\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<AHashSet<_>>(),\n            [\"jane\", \"john.doe\"]\n                .into_iter()\n                .map(|s| s.into())\n                .collect::<AHashSet<_>>()\n        );\n        assert_eq!(\n            store\n                .list_principals(None, None, &[Type::Group], true, 0, 0)\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<AHashSet<_>>(),\n            [\"sales\", \"support\"]\n                .into_iter()\n                .map(|s| s.into())\n                .collect::<AHashSet<_>>()\n        );\n        assert_eq!(\n            store\n                .list_principals(None, None, &[Type::List], true, 0, 0)\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<Vec<_>>(),\n            vec![\"list\"]\n        );\n        assert_eq!(\n            store\n                .list_principals(\"example.org\".into(), None, &[], true, 0, 0)\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<Vec<_>>(),\n            vec![\"example.org\", \"jane\", \"john.doe\", \"list\"]\n        );\n        assert_eq!(\n            store\n                .list_principals(\"johnny doe\".into(), None, &[], true, 0, 0)\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<Vec<_>>(),\n            vec![\"john.doe\"]\n        );\n\n        // Write records on John's and Jane's accounts\n        let mut document_id = u32::MAX;\n        for account_id in [john_id, jane_id] {\n            document_id = store\n                .assign_document_ids(u32::MAX, Collection::Principal, 1)\n                .await\n                .unwrap();\n            store\n                .write(\n                    BatchBuilder::new()\n                        .with_account_id(account_id)\n                        .with_collection(Collection::Email)\n                        .with_document(document_id)\n                        .set(ValueClass::Property(0), \"hello\".as_bytes())\n                        .build_all(),\n                )\n                .await\n                .unwrap();\n            assert_eq!(\n                store\n                    .get_value::<String>(ValueKey {\n                        account_id,\n                        collection: Collection::Email.into(),\n                        document_id,\n                        class: ValueClass::Property(0)\n                    })\n                    .await\n                    .unwrap(),\n                Some(\"hello\".into())\n            );\n        }\n\n        // Delete John's account and make sure his records are gone\n        let server = Server {\n            inner: Arc::new(Inner::default()),\n            core: Arc::new(Core {\n                storage: Storage {\n                    data: store.clone(),\n                    blob: store.clone().into(),\n                    fts: store.clone().into(),\n                    ..Default::default()\n                },\n                ..Default::default()\n            }),\n        };\n        store.delete_principal(QueryBy::Id(john_id)).await.unwrap();\n        destroy_account_data(&server, john_id, true).await.unwrap();\n        assert_eq!(store.get_principal_id(\"john.doe\").await.unwrap(), None);\n        assert_eq!(\n            store.email_to_id(\"john.doe@example.org\").await.unwrap(),\n            None\n        );\n        assert_eq!(\n            store.rcpt(\"john.doe@example.org\").await.unwrap(),\n            RcptType::Invalid\n        );\n        assert_eq!(\n            store\n                .list_principals(\n                    None,\n                    None,\n                    &[Type::Individual, Type::Group, Type::List],\n                    true,\n                    0,\n                    0\n                )\n                .await\n                .unwrap()\n                .items\n                .into_iter()\n                .map(|p| p.name)\n                .collect::<AHashSet<_>>(),\n            [\"jane\", \"list\", \"sales\", \"support\"]\n                .into_iter()\n                .map(|s| s.into())\n                .collect::<AHashSet<_>>()\n        );\n        assert!(!account_has_emails(&store, john_id).await);\n        assert_eq!(\n            store\n                .get_value::<String>(ValueKey {\n                    account_id: john_id,\n                    collection: Collection::Email.into(),\n                    document_id: 0,\n                    class: ValueClass::Property(0)\n                })\n                .await\n                .unwrap(),\n            None\n        );\n\n        // Make sure Jane's records are still there\n        assert_eq!(store.get_principal_id(\"jane\").await.unwrap(), Some(jane_id));\n        assert_eq!(\n            store.email_to_id(\"jane@example.org\").await.unwrap(),\n            Some(jane_id)\n        );\n        assert_eq!(\n            store.rcpt(\"jane@example.org\").await.unwrap(),\n            RcptType::Mailbox\n        );\n        assert!(account_has_emails(&store, jane_id).await);\n        assert_eq!(\n            store\n                .get_value::<String>(ValueKey {\n                    account_id: jane_id,\n                    collection: Collection::Email.into(),\n                    document_id,\n                    class: ValueClass::Property(0)\n                })\n                .await\n                .unwrap(),\n            Some(\"hello\".into())\n        );\n\n        // Clean up\n        destroy_account_data(&server, jane_id, true).await.unwrap();\n        for principal_name in [\"jane\", \"list\", \"sales\", \"support\", \"example.org\"] {\n            store\n                .delete_principal(QueryBy::Name(principal_name))\n                .await\n                .unwrap();\n        }\n        store_assert_is_empty(&store, store.clone().into(), true).await;\n    }\n}\n\n#[allow(async_fn_in_trait)]\npub trait TestInternalDirectory {\n    async fn create_test_user(&self, login: &str, secret: &str, name: &str, emails: &[&str])\n    -> u32;\n    async fn create_test_group(&self, login: &str, name: &str, emails: &[&str]) -> u32;\n    async fn create_test_list(&self, login: &str, name: &str, emails: &[&str]) -> u32;\n    async fn set_test_quota(&self, login: &str, quota: u32);\n    async fn add_permissions(&self, login: &str, permissions: impl IntoIterator<Item = Permission>);\n    async fn remove_permissions(\n        &self,\n        login: &str,\n        permissions: impl IntoIterator<Item = Permission>,\n    );\n    async fn add_to_group(&self, login: &str, group: &str) -> ChangedPrincipals;\n    async fn remove_from_group(&self, login: &str, group: &str) -> ChangedPrincipals;\n    async fn remove_test_alias(&self, login: &str, alias: &str);\n    async fn create_test_domains(&self, domains: &[&str]);\n}\n\nimpl TestInternalDirectory for Store {\n    async fn create_test_user(\n        &self,\n        login: &str,\n        secret: &str,\n        name: &str,\n        emails: &[&str],\n    ) -> u32 {\n        let role = if login == \"admin\" { \"admin\" } else { \"user\" };\n        self.create_test_domains(emails).await;\n        if let Some(principal) = self\n            .query(QueryParams::name(login).with_return_member_of(false))\n            .await\n            .unwrap()\n        {\n            self.update_principal(UpdatePrincipal::by_id(principal.id()).with_updates(vec![\n                PrincipalUpdate::set(\n                    PrincipalField::Secrets,\n                    PrincipalValue::StringList(vec![secret.into()]),\n                ),\n                PrincipalUpdate::set(\n                    PrincipalField::Description,\n                    PrincipalValue::String(name.into()),\n                ),\n                PrincipalUpdate::set(\n                    PrincipalField::Emails,\n                    PrincipalValue::StringList(emails.iter().map(|s| (*s).into()).collect()),\n                ),\n                PrincipalUpdate::add_item(\n                    PrincipalField::Roles,\n                    PrincipalValue::String(role.into()),\n                ),\n                PrincipalUpdate::add_item(\n                    PrincipalField::EnabledPermissions,\n                    PrincipalValue::String(Permission::UnlimitedRequests.name().into()),\n                ),\n            ]))\n            .await\n            .unwrap();\n            principal.id()\n        } else {\n            self.create_principal(\n                PrincipalSet::new(0, Type::Individual)\n                    .with_field(PrincipalField::Name, login)\n                    .with_field(PrincipalField::Description, name)\n                    .with_field(\n                        PrincipalField::Secrets,\n                        PrincipalValue::StringList(vec![secret.into()]),\n                    )\n                    .with_field(\n                        PrincipalField::Emails,\n                        PrincipalValue::StringList(emails.iter().map(|s| (*s).into()).collect()),\n                    )\n                    .with_field(\n                        PrincipalField::Roles,\n                        PrincipalValue::StringList(vec![role.into()]),\n                    )\n                    .with_field(\n                        PrincipalField::EnabledPermissions,\n                        PrincipalValue::StringList(vec![\n                            Permission::UnlimitedRequests.name().into(),\n                        ]),\n                    ),\n                None,\n                None,\n            )\n            .await\n            .unwrap()\n            .id\n        }\n    }\n\n    async fn create_test_group(&self, login: &str, name: &str, emails: &[&str]) -> u32 {\n        self.create_test_domains(emails).await;\n        if let Some(principal) = self\n            .query(QueryParams::name(login).with_return_member_of(false))\n            .await\n            .unwrap()\n        {\n            principal.id()\n        } else {\n            self.create_principal(\n                PrincipalSet::new(0, Type::Group)\n                    .with_field(PrincipalField::Name, login)\n                    .with_field(PrincipalField::Description, name)\n                    .with_field(\n                        PrincipalField::Emails,\n                        PrincipalValue::StringList(emails.iter().map(|s| (*s).into()).collect()),\n                    )\n                    .with_field(\n                        PrincipalField::Roles,\n                        PrincipalValue::StringList(vec![\"user\".into()]),\n                    ),\n                None,\n                None,\n            )\n            .await\n            .unwrap()\n            .id\n        }\n    }\n\n    async fn create_test_list(&self, login: &str, name: &str, members: &[&str]) -> u32 {\n        if let Some(principal) = self\n            .query(QueryParams::name(login).with_return_member_of(false))\n            .await\n            .unwrap()\n        {\n            principal.id()\n        } else {\n            self.create_test_domains(&[login]).await;\n            self.create_principal(\n                PrincipalSet::new(0, Type::List)\n                    .with_field(PrincipalField::Name, login)\n                    .with_field(PrincipalField::Description, name)\n                    .with_field(\n                        PrincipalField::Members,\n                        PrincipalValue::StringList(members.iter().map(|s| (*s).into()).collect()),\n                    )\n                    .with_field(\n                        PrincipalField::Emails,\n                        PrincipalValue::StringList(vec![login.into()]),\n                    ),\n                None,\n                None,\n            )\n            .await\n            .unwrap()\n            .id\n        }\n    }\n\n    async fn set_test_quota(&self, login: &str, quota: u32) {\n        self.update_principal(UpdatePrincipal::by_name(login).with_updates(vec![\n            PrincipalUpdate::set(PrincipalField::Quota, PrincipalValue::Integer(quota as u64)),\n        ]))\n        .await\n        .unwrap();\n    }\n\n    async fn add_permissions(\n        &self,\n        login: &str,\n        permissions: impl IntoIterator<Item = Permission>,\n    ) {\n        self.update_principal(\n            UpdatePrincipal::by_name(login).with_updates(\n                permissions\n                    .into_iter()\n                    .map(|p| {\n                        PrincipalUpdate::add_item(\n                            PrincipalField::EnabledPermissions,\n                            PrincipalValue::String(p.name().to_string()),\n                        )\n                    })\n                    .collect(),\n            ),\n        )\n        .await\n        .unwrap();\n    }\n\n    async fn remove_permissions(\n        &self,\n        login: &str,\n        permissions: impl IntoIterator<Item = Permission>,\n    ) {\n        self.update_principal(\n            UpdatePrincipal::by_name(login).with_updates(\n                permissions\n                    .into_iter()\n                    .map(|p| {\n                        PrincipalUpdate::remove_item(\n                            PrincipalField::EnabledPermissions,\n                            PrincipalValue::String(p.name().to_string()),\n                        )\n                    })\n                    .collect(),\n            ),\n        )\n        .await\n        .unwrap();\n    }\n\n    async fn add_to_group(&self, login: &str, group: &str) -> ChangedPrincipals {\n        self.update_principal(UpdatePrincipal::by_name(login).with_updates(vec![\n            PrincipalUpdate::add_item(\n                PrincipalField::MemberOf,\n                PrincipalValue::String(group.into()),\n            ),\n        ]))\n        .await\n        .unwrap()\n    }\n\n    async fn remove_from_group(&self, login: &str, group: &str) -> ChangedPrincipals {\n        self.update_principal(UpdatePrincipal::by_name(login).with_updates(vec![\n            PrincipalUpdate::remove_item(\n                PrincipalField::MemberOf,\n                PrincipalValue::String(group.into()),\n            ),\n        ]))\n        .await\n        .unwrap()\n    }\n\n    async fn remove_test_alias(&self, login: &str, alias: &str) {\n        self.update_principal(UpdatePrincipal::by_name(login).with_updates(vec![\n            PrincipalUpdate::remove_item(\n                PrincipalField::Emails,\n                PrincipalValue::String(alias.into()),\n            ),\n        ]))\n        .await\n        .unwrap();\n    }\n\n    async fn create_test_domains(&self, domains: &[&str]) {\n        for domain in domains {\n            let domain = domain.rsplit_once('@').map_or(*domain, |(_, d)| d);\n            if self\n                .query(QueryParams::name(domain).with_return_member_of(false))\n                .await\n                .unwrap()\n                .is_none()\n            {\n                self.create_principal(\n                    PrincipalSet::new(0, Type::Domain).with_field(PrincipalField::Name, domain),\n                    None,\n                    None,\n                )\n                .await\n                .unwrap();\n            }\n        }\n    }\n}\n\nasync fn account_has_emails(store: &Store, account_id: u32) -> bool {\n    let mut has_emails = false;\n    store\n        .iterate(\n            IterateParams::new(\n                ValueKey {\n                    account_id,\n                    collection: Collection::Email.into(),\n                    document_id: 0,\n                    class: ValueClass::Property(0),\n                },\n                ValueKey {\n                    account_id,\n                    collection: Collection::Email.into(),\n                    document_id: u32::MAX,\n                    class: ValueClass::Property(u8::MAX),\n                },\n            )\n            .no_values(),\n            |_, _| {\n                has_emails = true;\n                Ok(false)\n            },\n        )\n        .await\n        .unwrap();\n    has_emails\n}\n\nasync fn assert_list_members(\n    store: &Store,\n    list_addr: &str,\n    members: impl IntoIterator<Item = &str>,\n) {\n    match store.rcpt(list_addr).await.unwrap() {\n        RcptType::List(items) => {\n            assert_eq!(\n                items.into_iter().collect::<AHashSet<_>>(),\n                members\n                    .into_iter()\n                    .map(|s| s.into())\n                    .collect::<AHashSet<_>>()\n            );\n        }\n        other => panic!(\"invalid {other:?}\"),\n    }\n}\n"
  },
  {
    "path": "tests/src/directory/ldap.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::fmt::Debug;\n\nuse directory::{\n    QueryParams, ROLE_USER, Type,\n    backend::{RcptType, internal::manage::ManageDirectory},\n};\nuse mail_send::Credentials;\n\nuse crate::directory::{\n    DirectoryTest, IntoTestPrincipal, TestPrincipal, map_account_id, map_account_ids,\n};\n\n#[tokio::test]\nasync fn ldap_directory() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Obtain directory handle\n    let mut config = DirectoryTest::new(\"sqlite\".into()).await;\n    let handle = config.directories.directories.remove(\"ldap\").unwrap();\n    let base_store = config.stores.stores.get(\"sqlite\").unwrap();\n    let core = config.server;\n\n    // Test authentication\n    for (auth_type, handle) in [\n        (\"Default\", handle.clone()),\n        (\n            \"Bind template\",\n            config\n                .directories\n                .directories\n                .remove(\"ldap-bind-template\")\n                .unwrap(),\n        ),\n        (\n            \"Bind lookup\",\n            config\n                .directories\n                .directories\n                .remove(\"ldap-bind-lookup\")\n                .unwrap(),\n        ),\n    ] {\n        println!(\"Testing {auth_type} LDAP authentication...\");\n        assert_eq!(\n            handle\n                .query(\n                    QueryParams::credentials(&Credentials::Plain {\n                        username: \"john\".into(),\n                        secret: \"12345\".into()\n                    })\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .unwrap()\n                .into_test()\n                .into_sorted(),\n            TestPrincipal {\n                id: base_store.get_principal_id(\"john\").await.unwrap().unwrap(),\n                name: \"john\".into(),\n                description: Some(\"John Doe\".into()),\n                secrets: vec![\"12345\".into()],\n                typ: Type::Individual,\n                member_of: map_account_ids(base_store, vec![\"sales\"])\n                    .await\n                    .into_iter()\n                    .map(|v| v.to_string())\n                    .collect(),\n                emails: vec![\"john@example.org\".into(), \"john.doe@example.org\".into()],\n                roles: vec![ROLE_USER.to_string()],\n                ..Default::default()\n            }\n            .into_sorted()\n        );\n        assert_eq!(\n            handle\n                .query(\n                    QueryParams::credentials(&Credentials::Plain {\n                        username: \"bill\".into(),\n                        secret: \"password\".into()\n                    })\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .unwrap()\n                .into_test()\n                .into_sorted(),\n            TestPrincipal {\n                id: base_store.get_principal_id(\"bill\").await.unwrap().unwrap(),\n                name: \"bill\".into(),\n                description: Some(\"Bill Foobar\".into()),\n                secrets: vec![\n                    \"$2y$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe\".into()\n                ],\n                typ: Type::Individual,\n                quota: 500000,\n                emails: vec![\"bill@example.org\".into(),],\n                roles: vec![ROLE_USER.to_string()],\n                ..Default::default()\n            }\n            .into_sorted()\n        );\n        assert!(\n            handle\n                .query(\n                    QueryParams::credentials(&Credentials::Plain {\n                        username: \"bill\".into(),\n                        secret: \"invalid\".into()\n                    })\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .is_none()\n        );\n    }\n\n    // Get user by name\n    assert_eq!(\n        handle\n            .query(QueryParams::name(\"jane\").with_return_member_of(true))\n            .await\n            .unwrap()\n            .unwrap()\n            .into_test()\n            .into_sorted(),\n        TestPrincipal {\n            id: base_store.get_principal_id(\"jane\").await.unwrap().unwrap(),\n            name: \"jane\".into(),\n            description: Some(\"Jane Doe\".into()),\n            typ: Type::Individual,\n            secrets: vec![\"abcde\".into()],\n            member_of: map_account_ids(base_store, vec![\"sales\", \"support\"])\n                .await\n                .into_iter()\n                .map(|v| v.to_string())\n                .collect(),\n            emails: vec![\"jane@example.org\".into(),],\n            roles: vec![ROLE_USER.to_string()],\n            ..Default::default()\n        }\n        .into_sorted()\n    );\n\n    // Get group by name\n    assert_eq!(\n        handle\n            .query(QueryParams::name(\"sales\").with_return_member_of(true))\n            .await\n            .unwrap()\n            .unwrap()\n            .into_test(),\n        TestPrincipal {\n            id: base_store.get_principal_id(\"sales\").await.unwrap().unwrap(),\n            name: \"sales\".into(),\n            description: Some(\"sales\".into()),\n            typ: Type::Group,\n            roles: vec![ROLE_USER.to_string()],\n            ..Default::default()\n        }\n    );\n\n    // Ids by email\n    assert_eq!(\n        core.email_to_id(&handle, \"jane@example.org\", 0)\n            .await\n            .unwrap(),\n        Some(map_account_id(base_store, \"jane\").await),\n    );\n    assert_eq!(\n        core.email_to_id(&handle, \"jane+alias@example.org\", 0)\n            .await\n            .unwrap(),\n        Some(map_account_id(base_store, \"jane\").await),\n    );\n    assert_eq!(\n        core.email_to_id(&handle, \"unknown@example.org\", 0)\n            .await\n            .unwrap(),\n        None,\n    );\n    assert_eq!(\n        core.email_to_id(&handle, \"anything@catchall.org\", 0)\n            .await\n            .unwrap(),\n        Some(map_account_id(base_store, \"robert\").await)\n    );\n\n    // Domain validation\n    assert!(handle.is_local_domain(\"example.org\").await.unwrap());\n    assert!(!handle.is_local_domain(\"other.org\").await.unwrap());\n\n    // RCPT TO\n    assert_eq!(\n        core.rcpt(&handle, \"jane@example.org\", 0).await.unwrap(),\n        RcptType::Mailbox\n    );\n    assert_eq!(\n        core.rcpt(&handle, \"info@example.org\", 0).await.unwrap(),\n        RcptType::Mailbox\n    );\n    assert_eq!(\n        core.rcpt(&handle, \"jane+alias@example.org\", 0)\n            .await\n            .unwrap(),\n        RcptType::Mailbox\n    );\n    assert_eq!(\n        core.rcpt(&handle, \"info+alias@example.org\", 0)\n            .await\n            .unwrap(),\n        RcptType::Mailbox\n    );\n    assert_eq!(\n        core.rcpt(&handle, \"random_user@catchall.org\", 0)\n            .await\n            .unwrap(),\n        RcptType::Mailbox\n    );\n    assert_eq!(\n        core.rcpt(&handle, \"invalid@example.org\", 0).await.unwrap(),\n        RcptType::Invalid\n    );\n\n    // VRFY\n    compare_sorted(\n        core.vrfy(&handle, \"jane\", 0).await.unwrap(),\n        vec![\"jane@example.org\".into()],\n    );\n    compare_sorted(\n        core.vrfy(&handle, \"john\", 0).await.unwrap(),\n        vec![\"john@example.org\".into(), \"john.doe@example.org\".into()],\n    );\n    compare_sorted(\n        core.vrfy(&handle, \"jane+alias@example\", 0).await.unwrap(),\n        vec![\"jane@example.org\".into()],\n    );\n    compare_sorted(\n        core.vrfy(&handle, \"info\", 0).await.unwrap(),\n        Vec::<String>::new(),\n    );\n    compare_sorted(\n        core.vrfy(&handle, \"invalid\", 0).await.unwrap(),\n        Vec::<String>::new(),\n    );\n\n    // EXPN\n    // Now handled by the internal directory\n    /*compare_sorted(\n        core.expn(&handle, \"info@example.org\", 0).await.unwrap(),\n        vec![\n            \"bill@example.org\".into(),\n            \"jane@example.org\".into(),\n            \"john@example.org\".into(),\n        ],\n    );\n    compare_sorted(\n        core.expn(&handle, \"john@example.org\", 0).await.unwrap(),\n        Vec::<String>::new(),\n    );*/\n}\n\nfn compare_sorted<T: Eq + Debug>(v1: Vec<T>, v2: Vec<T>) {\n    for val in v1.iter() {\n        assert!(v2.contains(val), \"{v1:?} != {v2:?}\");\n    }\n\n    for val in v2.iter() {\n        assert!(v1.contains(val), \"{v1:?} != {v2:?}\");\n    }\n}\n"
  },
  {
    "path": "tests/src/directory/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod imap;\npub mod internal;\npub mod ldap;\npub mod oidc;\npub mod smtp;\npub mod sql;\n\nuse common::{Core, Server, config::smtp::session::AddressMapping};\nuse directory::{\n    Directories, Principal, PrincipalData, Type,\n    backend::internal::{PrincipalField, PrincipalSet, manage::ManageDirectory},\n};\nuse mail_send::Credentials;\nuse rustls::ServerConfig;\nuse rustls_pemfile::{certs, pkcs8_private_keys};\nuse rustls_pki_types::PrivateKeyDer;\nuse std::{borrow::Cow, io::BufReader, sync::Arc};\nuse store::{Store, Stores};\nuse tokio_rustls::TlsAcceptor;\n\nuse crate::{AssertConfig, store::TempDir};\n\nconst CONFIG: &str = r#\"\n[directory.\"rocksdb\"]\ntype = \"internal\"\nstore = \"rocksdb\"\n\n[directory.\"foundationdb\"]\ntype = \"internal\"\nstore = \"foundationdb\"\n\n[directory.\"sqlite\"]\ntype = \"sql\"\nstore = \"sqlite\"\n\n[directory.\"sqlite\".columns]\nname = \"name\"\ndescription = \"description\"\nsecret = \"secret\"\nemail = \"address\"\nquota = \"quota\"\nclass = \"type\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/rocksdb\"\n\n[store.\"foundationdb\"]\ntype = \"foundationdb\"\n\n[store.\"sqlite\"]\ntype = \"sqlite\"\npath = \"{TMP}/auth.db\"\n\n[store.\"sqlite\".query]\nname = \"SELECT name, type, secret, description, quota FROM accounts WHERE name = ? AND active = true\"\nmembers = \"SELECT member_of FROM group_members WHERE name = ?\"\nrecipients = \"SELECT name FROM emails WHERE address = ? ORDER BY name ASC\"\nemails = \"SELECT address FROM emails WHERE name = ? AND type != 'list' ORDER BY type DESC, address ASC\"\nverify = \"SELECT address FROM emails WHERE address LIKE '%' || ? || '%' AND type = 'primary' ORDER BY address LIMIT 5\"\nexpand = \"SELECT p.address FROM emails AS p JOIN emails AS l ON p.name = l.name WHERE p.type = 'primary' AND l.address = ? AND l.type = 'list' ORDER BY p.address LIMIT 50\"\ndomains = \"SELECT 1 FROM emails WHERE address LIKE '%@' || ? LIMIT 1\"\n\n[storage]\nlookup = \"sqlite\"\n\n##############################################################################\n\n[directory.\"postgresql\"]\ntype = \"sql\"\nstore = \"postgresql\"\n\n[directory.\"postgresql\".columns]\nname = \"name\"\ndescription = \"description\"\nsecret = \"secret\"\nemail = \"address\"\nquota = \"quota\"\nclass = \"type\"\n\n[store.\"postgresql\"]\ntype = \"postgresql\"\nhost = \"localhost\"\nport = 5432\ndatabase = \"stalwart\"\nuser = \"postgres\"\npassword = \"mysecretpassword\"\n\n[store.\"postgresql\".query]\nname = \"SELECT name, type, secret, description, quota FROM accounts WHERE name = $1 AND active = true\"\nmembers = \"SELECT member_of FROM group_members WHERE name = $1\"\nrecipients = \"SELECT name FROM emails WHERE address = $1 ORDER BY name ASC\"\nemails = \"SELECT address FROM emails WHERE name = $1 AND type != 'list' ORDER BY type DESC, address ASC\"\nverify = \"SELECT address FROM emails WHERE address LIKE '%' || $1 || '%' AND type = 'primary' ORDER BY address LIMIT 5\"\nexpand = \"SELECT p.address FROM emails AS p JOIN emails AS l ON p.name = l.name WHERE p.type = 'primary' AND l.address = $1 AND l.type = 'list' ORDER BY p.address LIMIT 50\"\ndomains = \"SELECT 1 FROM emails WHERE address LIKE '%@' || $1 LIMIT 1\"\n\n##############################################################################\n\n[directory.\"mysql\"]\ntype = \"sql\"\nstore = \"mysql\"\n\n[directory.\"mysql\".columns]\nname = \"name\"\ndescription = \"description\"\nsecret = \"secret\"\nemail = \"address\"\nquota = \"quota\"\nclass = \"type\"\n\n[store.\"mysql\"]\ntype = \"mysql\"\nhost = \"localhost\"\nport = 3307\ndatabase = \"stalwart\"\nuser = \"root\"\npassword = \"password\"\n\n[store.\"mysql\".query]\nname = \"SELECT name, type, secret, description, quota FROM accounts WHERE name = ? AND active = true\"\nmembers = \"SELECT member_of FROM group_members WHERE name = ?\"\nrecipients = \"SELECT name FROM emails WHERE address = ? ORDER BY name ASC\"\nemails = \"SELECT address FROM emails WHERE name = ? AND type != 'list' ORDER BY type DESC, address ASC\"\nverify = \"SELECT address FROM emails WHERE address LIKE CONCAT('%', ?, '%') AND type = 'primary' ORDER BY address LIMIT 5\"\nexpand = \"SELECT p.address FROM emails AS p JOIN emails AS l ON p.name = l.name WHERE p.type = 'primary' AND l.address = ? AND l.type = 'list' ORDER BY p.address LIMIT 50\"\ndomains = \"SELECT 1 FROM emails WHERE address LIKE CONCAT('%@', ?) LIMIT 1\"\n\n##############################################################################\n\n[directory.\"ldap\"]\ntype = \"ldap\"\nurl = \"ldap://localhost:3893\"\nbase-dn = \"dc=example,dc=org\"\n\n[directory.\"ldap\".bind]\ndn = \"cn=serviceuser,ou=svcaccts,dc=example,dc=org\"\nsecret = \"mysecret\"\n\n[directory.\"ldap\".filter]\nname = \"(&(|(objectClass=posixAccount)(objectClass=posixGroup))(uid=?))\"\nemail = \"(&(|(objectClass=posixAccount)(objectClass=posixGroup))(|(mail=?)(givenName=?)(sn=?)))\"\n\n[directory.\"ldap\".attributes]\nname = \"uid\"\ndescription = [\"principalName\", \"description\"]\nsecret = \"userPassword\"\ngroups = [\"memberOf\", \"otherGroups\"]\nemail = \"mail\"\nemail-alias = \"givenName\"\nquota = \"diskQuota\"\nclass = \"objectClass\"\n\n[directory.\"ldap-bind-template\"]\ntype = \"ldap\"\nurl = \"ldap://localhost:3893\"\nbase-dn = \"dc=example,dc=org\"\n\n[directory.\"ldap-bind-template\".bind]\ndn = \"cn=serviceuser,ou=svcaccts,dc=example,dc=org\"\nsecret = \"mysecret\"\n\n[directory.\"ldap-bind-template\".bind.auth]\nmethod = \"template\"\ntemplate = \"cn={username},ou=,dc=example,dc=org\"\nsearch = false\n\n[directory.\"ldap-bind-template\".filter]\nname = \"(&(|(objectClass=posixAccount)(objectClass=posixGroup))(uid=?))\"\nemail = \"(&(|(objectClass=posixAccount)(objectClass=posixGroup))(|(mail=?)(givenName=?)(sn=?)))\"\n\n[directory.\"ldap-bind-template\".attributes]\nname = \"uid\"\ndescription = [\"principalName\", \"description\"]\nsecret = \"userPassword\"\ngroups = [\"memberOf\", \"otherGroups\"]\nemail = \"mail\"\nemail-alias = \"givenName\"\nquota = \"diskQuota\"\nclass = \"objectClass\"\n\n[directory.\"ldap-bind-lookup\"]\ntype = \"ldap\"\nurl = \"ldap://localhost:3893\"\nbase-dn = \"dc=example,dc=org\"\n\n[directory.\"ldap-bind-lookup\".bind]\ndn = \"cn=serviceuser,ou=svcaccts,dc=example,dc=org\"\nsecret = \"mysecret\"\n\n[directory.\"ldap-bind-lookup\".bind.auth]\nmethod = \"lookup\"\n\n[directory.\"ldap-bind-lookup\".filter]\nname = \"(&(|(objectClass=posixAccount)(objectClass=posixGroup))(uid=?))\"\nemail = \"(&(|(objectClass=posixAccount)(objectClass=posixGroup))(|(mail=?)(givenName=?)(sn=?)))\"\n\n[directory.\"ldap-bind-lookup\".attributes]\nname = \"uid\"\ndescription = [\"principalName\", \"description\"]\nsecret = \"userPassword\"\ngroups = [\"memberOf\", \"otherGroups\"]\nemail = \"mail\"\nemail-alias = \"givenName\"\nquota = \"diskQuota\"\nclass = \"objectClass\"\n\n##############################################################################\n\n[directory.\"imap\"]\ntype = \"imap\"\nhost = \"127.0.0.1\"\nport = 9198\n\n[directory.\"imap\".pool]\nmax-connections = 5\n\n[directory.\"imap\".tls]\nenable = true\nallow-invalid-certs = true\n\n##############################################################################\n\n[directory.\"smtp\"]\ntype = \"lmtp\"\nhost = \"127.0.0.1\"\nport = 9199\n\n[directory.\"smtp\".limits]\nauth-errors = 3\nrcpt = 5\n\n[directory.\"smtp\".pool]\nmax-connections = 5\n\n[directory.\"smtp\".tls]\nenable = true\nallow-invalid-certs = true\n\n[directory.\"smtp\".cache]\nentries = 500\nttl = {positive = '10s', negative = '5s'}\n\n##############################################################################\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\nclass = \"individual\"\ndescription = \"John Doe\"\nsecret = \"12345\"\nemail = [\"john@example.org\", \"jdoe@example.org\", \"john.doe@example.org\"]\nemail-list = [\"info@example.org\"]\nmember-of = [\"sales\"]\n\n[[directory.\"local\".principals]]\nname = \"jane\"\nclass = \"individual\"\ndescription = \"Jane Doe\"\nsecret = \"abcde\"\nemail = \"jane@example.org\"\nemail-list = [\"info@example.org\"]\nmember-of = [\"sales\", \"support\"]\n\n[[directory.\"local\".principals]]\nname = \"bill\"\nclass = \"individual\"\ndescription = \"Bill Foobar\"\nsecret = \"$2y$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe\"\nquota = 500000\nemail = \"bill@example.org\"\nemail-list = [\"info@example.org\"]\n\n[[directory.\"local\".principals]]\nname = \"sales\"\nclass = \"group\"\ndescription = \"Sales Team\"\n\n[[directory.\"local\".principals]]\nname = \"support\"\nclass = \"group\"\ndescription = \"Support Team\"\n\n##############################################################################\n\n[directory.\"oidc-userinfo\"]\ntype = \"oidc\"\nstore = \"rocksdb\"\ntimeout = \"1s\"\nendpoint.url = \"https://127.0.0.1:9090/userinfo\"\nendpoint.method = \"userinfo\"\nfields.email = \"email\"\nfields.username = \"preferred_username\"\nfields.full-name = \"name\"\n\n[directory.\"oidc-introspect-none\"]\ntype = \"oidc\"\nstore = \"rocksdb\"\ntimeout = \"1s\"\nendpoint.url = \"https://127.0.0.1:9090/introspect-none\"\nendpoint.method = \"introspect\"\nauth.method = \"none\"\nfields.email = \"email\"\nfields.username = \"preferred_username\"\nfields.full-name = \"name\"\n\n[directory.\"oidc-introspect-user-token\"]\ntype = \"oidc\"\nstore = \"rocksdb\"\ntimeout = \"1s\"\nendpoint.url = \"https://127.0.0.1:9090/introspect-user-token\"\nendpoint.method = \"introspect\"\nauth.method = \"user-token\"\nfields.email = \"email\"\nfields.username = \"preferred_username\"\nfields.full-name = \"name\"\n\n[directory.\"oidc-introspect-token\"]\ntype = \"oidc\"\nstore = \"rocksdb\"\ntimeout = \"1s\"\nendpoint.url = \"https://127.0.0.1:9090/introspect-token\"\nendpoint.method = \"introspect\"\nauth.method = \"token\"\nauth.token = \"token_of_gratitude\"\nfields.email = \"email\"\nfields.username = \"preferred_username\"\nfields.full-name = \"name\"\n\n[directory.\"oidc-introspect-basic\"]\ntype = \"oidc\"\nstore = \"rocksdb\"\ntimeout = \"1s\"\nendpoint.url = \"https://127.0.0.1:9090/introspect-basic\"\nendpoint.method = \"introspect\"\nauth.method = \"basic\"\nauth.username = \"myuser\"\nauth.secret = \"mypass\"\nfields.email = \"email\"\nfields.username = \"preferred_username\"\nfields.full-name = \"name\"\n\n\"#;\n\npub struct DirectoryStore {\n    pub store: Store,\n}\n\npub struct DirectoryTest {\n    pub directories: Directories,\n    pub stores: Stores,\n    pub temp_dir: TempDir,\n    pub server: Server,\n}\n\n#[derive(Debug, Default, Clone, PartialEq, Eq)]\npub struct TestPrincipal {\n    pub id: u32,\n    pub typ: Type,\n    pub quota: u64,\n    pub name: String,\n    pub secrets: Vec<String>,\n    pub emails: Vec<String>,\n    pub member_of: Vec<String>,\n    pub roles: Vec<String>,\n    pub lists: Vec<String>,\n    pub description: Option<String>,\n}\n\nimpl DirectoryTest {\n    pub async fn new(id_store: Option<&str>) -> DirectoryTest {\n        let temp_dir = TempDir::new(\"directory_tests\", true);\n        let mut config_file = CONFIG.replace(\"{TMP}\", &temp_dir.path.to_string_lossy());\n        if id_store.is_some() {\n            // Disable foundationdb store for SQL tests (the fdb select api version can only be run once per process)\n            config_file = config_file\n                .replace(\n                    \"type = \\\"foundationdb\\\"\",\n                    \"type = \\\"foundationdb\\\"\\ndisable = true\",\n                )\n                .replace(\n                    \"store = \\\"foundationdb\\\"\",\n                    \"store = \\\"foundationdb\\\"\\ndisable = true\",\n                )\n        } else {\n            // Disable internal store\n            config_file =\n                config_file.replace(\"type = \\\"memory\\\"\", \"type = \\\"memory\\\"\\ndisable = true\")\n        }\n        let mut config = utils::config::Config::new(&config_file).unwrap();\n        let stores = Stores::parse_all(&mut config, false).await;\n        let directories = Directories::parse(\n            &mut config,\n            &stores,\n            id_store\n                .map(|id| stores.stores.get(id).unwrap().clone())\n                .unwrap_or_default(),\n            true,\n        )\n        .await;\n\n        config.assert_no_errors();\n\n        // Enable catch-all and subaddressing\n        let mut core = Core::default();\n        core.smtp.session.rcpt.catch_all = AddressMapping::Enable;\n        core.smtp.session.rcpt.subaddressing = AddressMapping::Enable;\n\n        DirectoryTest {\n            directories,\n            stores,\n            temp_dir,\n            server: Server {\n                inner: Default::default(),\n                core: core.into(),\n            },\n        }\n    }\n}\n\nconst CERT: &str = \"-----BEGIN CERTIFICATE-----\nMIIFCTCCAvGgAwIBAgIUCgHGQYUqtelbHGVSzCVwBL3fyEUwDQYJKoZIhvcNAQEL\nBQAwFDESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTIyMDUxNjExNDAzNFoXDTIzMDUx\nNjExNDAzNFowFDESMBAGA1UEAwwJbG9jYWxob3N0MIICIjANBgkqhkiG9w0BAQEF\nAAOCAg8AMIICCgKCAgEAtwS0Fzl3SjaCuKEXgZ/fdWbDoj/qDphyNCAKNevQ0+D0\nSTNkWCO04aFSH0zcL8zoD9gokNos0i7OU9//ZhZQmex4V6EFdZn8bFwUWN/scUvW\nHEFXVjtHldO2isZgIxH9LuwRv7KAgkISuWahqerOVDhe7SeQUV0AJGNEh3cT9PZr\ngSY931BxB7n+5k8eoSk8Z1gtBzQzL62kVGpHDKfw8yX8m65owF9eLUBrNzgxmXfC\nxpuHwj7hmVhS09PPKeN/RsFS8PsYO7bo0u8jEKalteumjRT7RyUEbioqfo6ZFOGj\nFHPIq/uKXS9zN1fpoyNh3ur5hMznQhrqlwBM9KlM7GdBJ0pZ3ad0YjT8IL/GnGKR\n85J2WZdLqaQdUZo7nV67FhqdDlNE4MdwiykTMjfmLRXGAVhAzJHKyRKNwmkI2aqe\nS7aqeNgvuDBwY80Q9a2rb5py1Aw+L8yCkUBuHboToDpxSVRDNN8DrWNmmsXnxsOG\nwRDODy4GICKyxlP+RFSM8xWSQ6y9ktS2OfDBm+Eqcw+3pZKhdz2wgxLkUBJ8X1eh\nkJrCA/6LTuhy6m6mMjAfoSOFU7fu88jxaWPgvP7GKyH+LM/t9eucobz2ks5rtSjz\nV4Dc5DCS94/OpVRHwHdaFSPbJKBN9Ev8gnNrAyx/aBPGoHBPG/QUiU7dcUNIPt0C\nAwEAAaNTMFEwHQYDVR0OBBYEFI167IxBmErB11EqiPPqFLa31ZaMMB8GA1UdIwQY\nMBaAFI167IxBmErB11EqiPPqFLa31ZaMMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZI\nhvcNAQELBQADggIBALU00IOiH5ubEauVCmakms5ermNTZfculnhnDfWTLMeh2+a7\nG4cqADErfMhm/mmLbrw33t9s6tCAhQltvewKR40ST9uMPSyiQbYaCXd5DXnuI6Ox\nJtNW+UOWIaMf8abnkdLvREOvb8dVQS1i3xq14tAjY5XgpGwCPP8m54b7N3Q7soLn\ne5PDhPNTnhRIn2RLuYoZmQmMA5fcqEUDYff4epUww7PhrM1QckZligI3566NlGOf\nj1G9JrivBtY0eaJtamIFnGMBT0ThDudxVja2Nv0C2Elry0p4T/o4nc4M67BJ/y1R\nvjNLAgFhbxssemU3lZqSd+pykpJBwDBjFSPrZZmQcbk7H6Uz8V1xr/xuzfw6fA13\nNWZ5vLgP/DQ13sM+XFlxThKfbPMPVe/UCTvfGtNW+3XyBgPntEkR+fNEawQmzbYl\nR+X1ymT9MZnEZqRMf7/UD/SYek1aUJefoew3upjMgxYVvh4F8dqJ+39F+xoFzIA2\n1dDAEMzXtjA3zKhZ2cycZbEzpJvYA3eGLuR16Suqfi4kPvfwK0mOhCxQmpayt7/X\nvuEzW6dPCH8Hgbb0WvsSppGOvhdbDaZFNfFc5eNSxhyKzu3H3ACNImZRtZE+yixx\n0fR8+xz9kDLf8xupV+X9heyFGHSyYU2Lveaevtr2Ij3weLRgJ6LbNALoeKXk\n-----END CERTIFICATE-----\n\";\nconst PK: &str = \"-----BEGIN PRIVATE KEY-----\nMIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC3BLQXOXdKNoK4\noReBn991ZsOiP+oOmHI0IAo169DT4PRJM2RYI7ThoVIfTNwvzOgP2CiQ2izSLs5T\n3/9mFlCZ7HhXoQV1mfxsXBRY3+xxS9YcQVdWO0eV07aKxmAjEf0u7BG/soCCQhK5\nZqGp6s5UOF7tJ5BRXQAkY0SHdxP09muBJj3fUHEHuf7mTx6hKTxnWC0HNDMvraRU\nakcMp/DzJfybrmjAX14tQGs3ODGZd8LGm4fCPuGZWFLT088p439GwVLw+xg7tujS\n7yMQpqW166aNFPtHJQRuKip+jpkU4aMUc8ir+4pdL3M3V+mjI2He6vmEzOdCGuqX\nAEz0qUzsZ0EnSlndp3RiNPwgv8acYpHzknZZl0uppB1RmjudXrsWGp0OU0Tgx3CL\nKRMyN+YtFcYBWEDMkcrJEo3CaQjZqp5Ltqp42C+4MHBjzRD1ratvmnLUDD4vzIKR\nQG4duhOgOnFJVEM03wOtY2aaxefGw4bBEM4PLgYgIrLGU/5EVIzzFZJDrL2S1LY5\n8MGb4SpzD7elkqF3PbCDEuRQEnxfV6GQmsID/otO6HLqbqYyMB+hI4VTt+7zyPFp\nY+C8/sYrIf4sz+3165yhvPaSzmu1KPNXgNzkMJL3j86lVEfAd1oVI9skoE30S/yC\nc2sDLH9oE8agcE8b9BSJTt1xQ0g+3QIDAQABAoICABq5oxqpF5RMtXYEgAw7rkPU\nh8jPkHwlIrgd3Z/WGZ53APUXfhWo0ScJiZZsgNKyF0kJBZNxaI4gq5xv3zmnFIoF\nj+Ur7EIqBERGheoceMhqjI9/syMycNeeHM/S/ALjA5ewfT8C7+UVhOpx5DWNxidi\nO+phlp9q9zRZEo69grqIqVYooWxUsMyyCljTQOPDw8BLjfe5VagmsRJqmolslLDM\n4UBSjZVZ18S/3Wgo2oVQia660244BHWCAkZQbbXuNI2+eUAbSoSdxw3WQcaSrywL\nhzyezbqr2yPDIIVuiUgVUt0Ps0P57VCCN07jlYhvCEGnClysFzD+ATefoZ0wg7za\ndQu2E+d166rAjnssyhzcHMn3pxgSdtXD+dQR/xfIGbPABucCupEFqKmhLdMm9+ud\nlHay87qzMpIa8cITJwEQROfXqWAhNUU98pKCOx1SVXBqQC7QVqGQ5solDf0eMSVh\nngQ6Dz2WUI2ty75LteiFwlyTgnU9nyPN0NXsrMEET2BHWre7ufTQqiULtQ7+9BwH\nAMxEKvrQHjMUjdfbXuzdyc5w5mPYJZfFVSQ1HMslx66h9yCpRIsBZvUGvoaP8Tpe\nnQ66FTYRbiOkkdJ7k8DtrnhsJI1oOGjnvj/rvZ8D2pvrlJcIH2AyN3MOL8Jp5Oj1\nnCFt77TwpF92pgl0g9gBAoIBAQDcarmP54QboaIQ9S2gE/4gSVC5i44iDJuSRdI8\nK081RQcWiNzqQXTRc5nqJ7KzLyPiGlg+6rWsBKLos5l4t+MdhhH+KUvk/OtT/g8V\n0NZBNXLIbSb8j8ix4v3/f2qKHN3Co6QOlxb3gFvobKDdoKqUNiSH1zTZ8/Y/BzkM\njqWKhTdaLz6eyzhKfOTA4LO8kJ3VF8HUM1N9/e8Gjorl+gZpJUXUQS0+AIi8W76C\nOwDrVb3BPGVnApQJfWF78h4g20RwXrx/GYUW2vOMcLjXXDV5U7+nobPUoJnLxoZC\n16o88y0Ivan8dBNXsc1epyPvvEqp6MJbAyyVuNeuRJcgYA0BAoIBAQDUkGRV7fLG\nwCr5rNysUO+FKzVtTJnf9KEsqAqUmmVnG4oubxAJJtiB5n2+DT+CtO8Nrtz05BbR\nuxfWm+lbEw6lVMj63bywtp0NdULg7/2t+oq2Svv16KrZIRJttXMkdEiFFmkVAEhX\nl8Fyl6PJPfSMwbPdXEUPUAaNrXweVFffXczHc4W2G212ZzDB0z7QQSgEntbTDFB/\n2Cg5dvuojlM9zw0fuEyLwItZs7n16j/ONZLgBHyroMU9ZPxbnLrVyoZlqtob+RWm\nJu2fSIL9QqG6O4td1TqcUBGvFQYjGvKA+q5fsG26NBJ0Ac48cNK6PS4lMkN3Av2J\nccloYaMEHAXdAoIBAE8WMCy1Ok6byUXiYxOL+OPmyoM40q/e7DcovE2AkLQhZ3Cr\nfPDEucCphPFiexkV8f8fysgQeU0WgMmUH54UBPbD81LJyISKR3nkr875Ftdg8SV/\nHL0EblN9ifuR4U1bHCrJgoUFq2T09oVH7NR44Ju7bZIcIseNZK6qzcp2qGkycXD3\ngLWDX1hCxeV6+qLPFQKvuomEPRH4+jnVDXuFIaW6jPqixDP6BxXmqU2bFDJcmnBq\nVkwGvc1F4qORdUP+yOi05VeJdZqEx1x92aTUXg+BgEQKnjbNxUE7o1L6hQfHjUIU\no5iEoagWkQTEXf2YBwY+EPaNBgNWxnSuAbfJHwECggEBALOF95ezTVWauzD/U6ic\n+o3n/kl/Zn4FJ5KFodn7xCSe18d7uXlhO34KYqx+l+MWWMefpbGWacdcUjfImf93\nSulLgCqP12sP7/iLzp4XUpL7hOeM0NvRU2nqSpwpoUNqik0Mrlc0U+TWoGTduVCf\naMjwV65e3VyfY8mIeclLxqM5n1fcM1OoOnzDjiRE+0n7nYa5eAnq3pn6v4449TZY\nbelH03e0ucFWLtrltesBmj3YdWGJqJlzQOInRhNBfXJOh8+ZynfRmP0o54udPDQV\ncG3PGFd5XPTjkuvhv7sqaSGRlm/um92lWOhtFfdp+i+cuDpmByCef+7zEP19aKZx\n3GkCggEAFTs7KNMfvIEaLH0yQUFeq2gLmtcMofmOmeoIECycN1rG7iJo07lJLIs0\nbVODH8Z0kX8llu3cjGMAH/6R2uugJSxkmFiZKrngTzKmxDPvTCKWR4RFwXH9j8IO\ncPq7FtKN4SgrPy9ciAPdkcGmu3zz/sBKOaoPwvU2PdBRT+v/aoz+GCLXAvzFlKVe\n9/7zdg87ilo8+AtV+71EJeR3kyBPKS9JrWYUKfiams12+uuH4/53rMFZfNCAaZ3Z\n1sdXEO4o3Loc5TX4DbO9FVdBSBe6klEXx4T0QJboO6uBvTBnnRL2SQriJQQFwYT6\nXzVV5pwOxkIDBWDIqMUfwJDChBKfpw==\n-----END PRIVATE KEY-----\n\";\n\npub fn dummy_tls_acceptor() -> Arc<TlsAcceptor> {\n    // Init server config builder with safe defaults\n    let config = ServerConfig::builder().with_no_client_auth();\n\n    // load TLS key/cert files\n    let cert_file = &mut BufReader::new(CERT.as_bytes());\n    let key_file = &mut BufReader::new(PK.as_bytes());\n\n    // convert files to key/cert objects\n    let cert_chain = certs(cert_file).map(|r| r.unwrap()).collect();\n    let mut keys: Vec<PrivateKeyDer> = pkcs8_private_keys(key_file)\n        .map(|v| PrivateKeyDer::Pkcs8(v.unwrap()))\n        .collect();\n\n    // exit if no keys could be parsed\n    if keys.is_empty() {\n        panic!(\"Could not locate PKCS 8 private keys.\");\n    }\n\n    Arc::new(TlsAcceptor::from(Arc::new(\n        config.with_single_cert(cert_chain, keys.remove(0)).unwrap(),\n    )))\n}\n\ntrait IntoTestPrincipal {\n    fn into_test(self) -> TestPrincipal;\n}\n\nimpl IntoTestPrincipal for PrincipalSet {\n    fn into_test(self) -> TestPrincipal {\n        TestPrincipal::from(self)\n    }\n}\n\nimpl IntoTestPrincipal for Principal {\n    fn into_test(self) -> TestPrincipal {\n        TestPrincipal::from(self)\n    }\n}\n\nimpl TestPrincipal {\n    pub fn into_sorted(mut self) -> Self {\n        self.member_of.sort_unstable();\n        self.emails.sort_unstable();\n        self\n    }\n}\n\nimpl From<PrincipalSet> for TestPrincipal {\n    fn from(mut value: PrincipalSet) -> Self {\n        Self {\n            id: value.id(),\n            typ: value.typ(),\n            quota: value.quota(),\n            name: value.take_str(PrincipalField::Name).unwrap_or_default(),\n            secrets: value\n                .take_str_array(PrincipalField::Secrets)\n                .unwrap_or_default(),\n            emails: value\n                .take_str_array(PrincipalField::Emails)\n                .unwrap_or_default(),\n            member_of: value\n                .take_str_array(PrincipalField::MemberOf)\n                .unwrap_or_default(),\n            roles: value\n                .take_str_array(PrincipalField::Roles)\n                .unwrap_or_default(),\n            lists: value\n                .take_str_array(PrincipalField::Lists)\n                .unwrap_or_default(),\n            description: value.take_str(PrincipalField::Description),\n        }\n    }\n}\n\nimpl From<Principal> for TestPrincipal {\n    fn from(value: Principal) -> Self {\n        Self {\n            id: value.id(),\n            typ: value.typ(),\n            quota: value.quota().unwrap_or_default(),\n            member_of: value.member_of().map(|v| v.to_string()).collect(),\n            roles: value.roles().map(|v| v.to_string()).collect(),\n            lists: value.lists().map(|v| v.to_string()).collect(),\n            secrets: value\n                .data\n                .iter()\n                .filter_map(|v| match v {\n                    PrincipalData::Password(s)\n                    | PrincipalData::AppPassword(s)\n                    | PrincipalData::OtpAuth(s) => Some(s.to_string()),\n                    _ => None,\n                })\n                .collect(),\n            emails: value.email_addresses().map(|v| v.to_string()).collect(),\n            description: value.description().map(|v| v.to_string()),\n            name: value.name,\n        }\n    }\n}\n\nimpl From<TestPrincipal> for PrincipalSet {\n    fn from(value: TestPrincipal) -> Self {\n        PrincipalSet::new(value.id, value.typ)\n            .with_field(PrincipalField::Name, value.name)\n            .with_field(PrincipalField::Quota, value.quota)\n            .with_field(PrincipalField::Secrets, value.secrets)\n            .with_field(PrincipalField::Emails, value.emails)\n            .with_field(PrincipalField::MemberOf, value.member_of)\n            .with_field(PrincipalField::Lists, value.lists)\n            .with_opt_field(PrincipalField::Description, value.description)\n    }\n}\n\n#[derive(Clone, PartialEq, Eq, Hash)]\npub enum Item {\n    IsAccount(String),\n    Authenticate(Credentials<String>),\n    Verify(String),\n    Expand(String),\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum LookupResult {\n    True,\n    False,\n    Values(Vec<String>),\n}\n\nimpl Item {\n    pub fn append(&self, append: usize) -> Self {\n        match self {\n            Item::IsAccount(str) => Item::IsAccount(format!(\"{append}{str}\")),\n            Item::Authenticate(str) => Item::Authenticate(match str {\n                Credentials::Plain { username, secret } => Credentials::Plain {\n                    username: username.to_string(),\n                    secret: format!(\"{append}{secret}\"),\n                },\n                Credentials::OAuthBearer { token } => Credentials::OAuthBearer {\n                    token: format!(\"{append}{token}\"),\n                },\n                Credentials::XOauth2 { username, secret } => Credentials::XOauth2 {\n                    username: username.to_string(),\n                    secret: format!(\"{append}{secret}\"),\n                },\n            }),\n            Item::Verify(str) => Item::Verify(format!(\"{append}{str}\")),\n            Item::Expand(str) => Item::Expand(format!(\"{append}{str}\")),\n        }\n    }\n\n    pub fn as_credentials(&self) -> &Credentials<String> {\n        match self {\n            Item::Authenticate(c) => c,\n            _ => panic!(\"Item is not a Credentials\"),\n        }\n    }\n}\n\nimpl LookupResult {\n    fn append(&self, append: usize) -> Self {\n        match self {\n            LookupResult::True => LookupResult::True,\n            LookupResult::False => LookupResult::False,\n            LookupResult::Values(v) => {\n                let mut r = Vec::with_capacity(v.len());\n                for (pos, val) in v.iter().enumerate() {\n                    r.push(if pos == 0 {\n                        format!(\"{append}{val}\")\n                    } else {\n                        val.to_string()\n                    });\n                }\n                LookupResult::Values(r)\n            }\n        }\n    }\n}\n\nimpl From<bool> for LookupResult {\n    fn from(b: bool) -> Self {\n        if b {\n            LookupResult::True\n        } else {\n            LookupResult::False\n        }\n    }\n}\n\nimpl From<Vec<String>> for LookupResult {\n    fn from(v: Vec<String>) -> Self {\n        LookupResult::Values(v)\n    }\n}\n\nimpl core::fmt::Debug for Item {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        match self {\n            Self::IsAccount(arg0) => f.debug_tuple(\"Rcpt\").field(arg0).finish(),\n            Self::Authenticate(_) => f.debug_tuple(\"Auth\").finish(),\n            Self::Expand(arg0) => f.debug_tuple(\"Expn\").field(arg0).finish(),\n            Self::Verify(arg0) => f.debug_tuple(\"Vrfy\").field(arg0).finish(),\n        }\n    }\n}\n\n#[tokio::test]\nasync fn address_mappings() {\n    const MAPPINGS: &str = r#\"\n    [enable]\n    catch-all = true\n    subaddressing = true\n    expected-sub = \"john.doe@example.org\"\n    expected-sub-nomatch = \"jane@example.org\"\n    expected-catch = \"@example.org\"\n\n    [disable]\n    catch-all = false\n    subaddressing = false\n    expected-sub = \"john.doe+alias@example.org\"\n    expected-sub-nomatch = \"jane@example.org\"\n    expected-catch = false\n\n    [custom]\n    catch-all = [{if = \"matches('(.+)@(.+)$', address)\", then = \"'info@' + $2\"}, {else = false}]\n    subaddressing = [{ if = \"matches('^([^.]+)\\\\.([^.]+)@(.+)$', address)\", then = \"$2 + '@' + $3\" }, {else = false}]\n    expected-sub = \"doe+alias@example.org\"\n    expected-sub-nomatch = \"jane@example.org\"\n    expected-catch = \"info@example.org\"\n    \"#;\n\n    let mut config = utils::config::Config::new(MAPPINGS).unwrap();\n    const ADDR: &str = \"john.doe+alias@example.org\";\n    const ADDR_NO_MATCH: &str = \"jane@example.org\";\n    let core = Server::default();\n\n    for test in [\"enable\", \"disable\", \"custom\"] {\n        let catch_all = AddressMapping::parse(&mut config, (test, \"catch-all\"));\n        let subaddressing = AddressMapping::parse(&mut config, (test, \"subaddressing\"));\n\n        assert_eq!(\n            subaddressing.to_subaddress(&core, ADDR, 0).await,\n            config.value_require((test, \"expected-sub\")).unwrap(),\n            \"failed subaddress for {test:?}\"\n        );\n\n        assert_eq!(\n            subaddressing.to_subaddress(&core, ADDR_NO_MATCH, 0).await,\n            config\n                .value_require((test, \"expected-sub-nomatch\"))\n                .unwrap(),\n            \"failed subaddress no match for {test:?}\"\n        );\n\n        assert_eq!(\n            catch_all.to_catch_all(&core, ADDR, 0).await,\n            config\n                .property_require::<Option<String>>((test, \"expected-catch\"))\n                .unwrap()\n                .map(Cow::Owned),\n            \"failed catch-all for {test:?}\"\n        );\n    }\n}\n\nasync fn map_account_ids(store: &Store, names: Vec<impl AsRef<str>>) -> Vec<u32> {\n    let mut ids = Vec::with_capacity(names.len());\n    for name in names {\n        ids.push(map_account_id(store, name).await);\n    }\n    ids\n}\n\nasync fn map_account_id(store: &Store, name: impl AsRef<str>) -> u32 {\n    store\n        .get_principal_id(name.as_ref())\n        .await\n        .unwrap()\n        .unwrap()\n}\n"
  },
  {
    "path": "tests/src/directory/oidc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse crate::{\n    directory::DirectoryTest,\n    http_server::{HttpMessage, spawn_mock_http_server},\n};\nuse base64::{Engine, engine::general_purpose};\nuse directory::QueryParams;\nuse http_proto::{JsonProblemResponse, JsonResponse, ToHttpResponse};\nuse hyper::{Method, StatusCode};\nuse mail_send::Credentials;\nuse serde_json::json;\nuse std::sync::Arc;\nuse trc::{AuthEvent, EventType};\n\nstatic TEST_TOKEN: &str = \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ\";\n\n#[tokio::test]\nasync fn oidc_directory() {\n    // Obtain directory handle\n    let mut config = DirectoryTest::new(\"rocksdb\".into()).await;\n\n    // Spawn mock OIDC server\n    let _tx = spawn_mock_http_server(Arc::new(|req: HttpMessage| {\n        let success_response = JsonResponse::new(json!({\n            \"email\": \"john@example.org\",\n            \"preferred_username\": \"jdoe\",\n            \"name\": \"John Doe\",\n        }))\n        .into_http_response();\n\n        match (req.method.clone(), req.uri.path().split('/').nth(1)) {\n            (Method::GET, Some(\"userinfo\")) => match req.headers.get(\"authorization\") {\n                Some(auth) if auth == &format!(\"Bearer {TEST_TOKEN}\") => success_response,\n                Some(_) => JsonProblemResponse(StatusCode::UNAUTHORIZED).into_http_response(),\n                None => panic!(\"Missing Authorization header: {req:#?}\"),\n            },\n            (Method::POST, Some(\"introspect-none\")) => {\n                assert!(req.headers.get(\"authorization\").is_none());\n                if req.get_url_encoded(\"token\").as_deref() == Some(TEST_TOKEN) {\n                    success_response\n                } else {\n                    JsonProblemResponse(StatusCode::UNAUTHORIZED).into_http_response()\n                }\n            }\n            (Method::POST, Some(\"introspect-user-token\")) => match req.headers.get(\"authorization\")\n            {\n                Some(auth)\n                    if auth == &format!(\"Bearer {TEST_TOKEN}\")\n                        && req.get_url_encoded(\"token\").as_deref() == Some(TEST_TOKEN) =>\n                {\n                    success_response\n                }\n                Some(_) => JsonProblemResponse(StatusCode::UNAUTHORIZED).into_http_response(),\n                None => panic!(\"Missing Authorization header: {req:#?}\"),\n            },\n            (Method::POST, Some(\"introspect-token\")) => match req.headers.get(\"authorization\") {\n                Some(auth)\n                    if auth == \"Bearer token_of_gratitude\"\n                        && req.get_url_encoded(\"token\").as_deref() == Some(TEST_TOKEN) =>\n                {\n                    success_response\n                }\n                Some(_) => JsonProblemResponse(StatusCode::UNAUTHORIZED).into_http_response(),\n                None => panic!(\"Missing Authorization header: {req:#?}\"),\n            },\n            (Method::POST, Some(\"introspect-basic\")) => match req.headers.get(\"authorization\") {\n                Some(auth)\n                    if auth\n                        == &format!(\n                            \"Basic {}\",\n                            general_purpose::STANDARD.encode(\"myuser:mypass\".as_bytes())\n                        )\n                        && req.get_url_encoded(\"token\").as_deref() == Some(TEST_TOKEN) =>\n                {\n                    success_response\n                }\n                Some(_) => JsonProblemResponse(StatusCode::UNAUTHORIZED).into_http_response(),\n                None => panic!(\"Missing Authorization header: {req:#?}\"),\n            },\n            _ => panic!(\"Unexpected request: {:?}\", req),\n        }\n    }))\n    .await;\n\n    for test in [\n        \"oidc-userinfo\",\n        \"oidc-introspect-none\",\n        \"oidc-introspect-user-token\",\n        \"oidc-introspect-token\",\n        \"oidc-introspect-basic\",\n    ] {\n        println!(\"Running OIDC test {test:?}...\");\n        let directory = config.directories.directories.remove(test).unwrap();\n\n        // Test an invalid token\n        let err = directory\n            .query(\n                QueryParams::credentials(&Credentials::OAuthBearer {\n                    token: \"invalid_or_expired_token\".to_string(),\n                })\n                .with_return_member_of(false),\n            )\n            .await\n            .unwrap_err();\n        assert!(\n            err.matches(EventType::Auth(AuthEvent::Failed)),\n            \"Unexpected error: {:?}\",\n            err\n        );\n\n        // Test a valid token\n        let principal = directory\n            .query(\n                QueryParams::credentials(&Credentials::OAuthBearer {\n                    token: TEST_TOKEN.to_string(),\n                })\n                .with_return_member_of(false),\n            )\n            .await\n            .unwrap()\n            .unwrap();\n        assert_eq!(principal.name(), \"jdoe\");\n        assert_eq!(principal.email_addresses().next(), Some(\"john@example.org\"));\n        assert_eq!(principal.description(), Some(\"John Doe\"));\n    }\n}\n"
  },
  {
    "path": "tests/src/directory/smtp.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::dummy_tls_acceptor;\nuse crate::directory::{DirectoryTest, Item, LookupResult};\nuse common::listener::limiter::{ConcurrencyLimiter, InFlight};\nuse directory::{QueryParams, backend::RcptType};\nuse mail_parser::decoders::base64::base64_decode;\nuse mail_send::Credentials;\nuse std::sync::Arc;\nuse tokio::{\n    io::{AsyncReadExt, AsyncWriteExt},\n    net::{TcpListener, TcpStream},\n    sync::watch,\n};\nuse tokio_rustls::TlsAcceptor;\n\n#[tokio::test]\nasync fn lmtp_directory() {\n    // Spawn mock LMTP server\n    let shutdown = spawn_mock_lmtp_server(5);\n    tokio::time::sleep(std::time::Duration::from_millis(100)).await;\n\n    // Obtain directory handle\n    let mut config = DirectoryTest::new(None).await;\n    let handle = config.directories.directories.remove(\"smtp\").unwrap();\n    let core = config.server;\n\n    // Basic lookup\n    let tests = vec![\n        (Item::IsAccount(\"john-ok@domain\".into()), LookupResult::True),\n        (\n            Item::IsAccount(\"john-bad@domain\".into()),\n            LookupResult::False,\n        ),\n        (\n            Item::Verify(\"john-ok@domain\".into()),\n            LookupResult::Values(vec![\"john-ok@domain\".into()]),\n        ),\n        (\n            Item::Verify(\"doesnot@exist.org\".into()),\n            LookupResult::False,\n        ),\n        (\n            Item::Expand(\"sales-ok,item1,item2,item3\".into()),\n            LookupResult::Values(vec![\n                \"sales-ok\".into(),\n                \"item1\".into(),\n                \"item2\".into(),\n                \"item3\".into(),\n            ]),\n        ),\n        (Item::Expand(\"other\".into()), LookupResult::False),\n        (\n            Item::Authenticate(Credentials::Plain {\n                username: \"john\".into(),\n                secret: \"ok\".into(),\n            }),\n            LookupResult::True,\n        ),\n        (\n            Item::Authenticate(Credentials::Plain {\n                username: \"john\".into(),\n                secret: \"bad\".into(),\n            }),\n            LookupResult::False,\n        ),\n    ];\n\n    for (item, expected) in &tests {\n        let result: LookupResult = match item {\n            Item::IsAccount(v) => {\n                (core.rcpt(&handle, v, 0).await.unwrap() == RcptType::Mailbox).into()\n            }\n            Item::Authenticate(v) => handle\n                .query(QueryParams::credentials(v).with_return_member_of(true))\n                .await\n                .unwrap()\n                .is_some()\n                .into(),\n            Item::Verify(v) => match core.vrfy(&handle, v, 0).await {\n                Ok(v) => v.into(),\n                Err(e) => {\n                    if e.matches(trc::EventType::Store(trc::StoreEvent::NotSupported)) {\n                        LookupResult::False\n                    } else {\n                        panic!(\"Unexpected error: {e:?}\")\n                    }\n                }\n            },\n            Item::Expand(v) => match core.expn(&handle, v, 0).await {\n                Ok(v) => v.into(),\n                Err(e) => {\n                    if e.matches(trc::EventType::Store(trc::StoreEvent::NotSupported)) {\n                        LookupResult::False\n                    } else {\n                        panic!(\"Unexpected error: {e:?}\")\n                    }\n                }\n            },\n        };\n\n        assert_eq!(&result, expected);\n    }\n\n    // Concurrent requests\n    let mut requests = Vec::new();\n    let core = Arc::new(core);\n    for n in 0..100 {\n        let (item, expected) = &tests[n % tests.len()];\n        let item = item.append(n);\n        let item_clone = item.clone();\n        let handle = handle.clone();\n        let core = core.clone();\n        requests.push((\n            tokio::spawn(async move {\n                let result: LookupResult = match &item {\n                    Item::IsAccount(v) => {\n                        (core.rcpt(&handle, v, 0).await.unwrap() == RcptType::Mailbox).into()\n                    }\n                    Item::Authenticate(v) => handle\n                        .query(QueryParams::credentials(v).with_return_member_of(true))\n                        .await\n                        .unwrap()\n                        .is_some()\n                        .into(),\n                    Item::Verify(v) => match core.vrfy(&handle, v, 0).await {\n                        Ok(v) => v.into(),\n                        Err(e) => {\n                            if e.matches(trc::EventType::Store(trc::StoreEvent::NotSupported)) {\n                                LookupResult::False\n                            } else {\n                                panic!(\"Unexpected error: {e:?}\")\n                            }\n                        }\n                    },\n                    Item::Expand(v) => match core.expn(&handle, v, 0).await {\n                        Ok(v) => v.into(),\n                        Err(e) => {\n                            if e.matches(trc::EventType::Store(trc::StoreEvent::NotSupported)) {\n                                LookupResult::False\n                            } else {\n                                panic!(\"Unexpected error: {e:?}\")\n                            }\n                        }\n                    },\n                };\n\n                result\n            }),\n            item_clone,\n            expected.append(n),\n        ));\n    }\n    for (result, item, expected_result) in requests {\n        assert_eq!(\n            result.await.unwrap(),\n            expected_result,\n            \"Failed for {item:?}\"\n        );\n    }\n\n    // Shutdown\n    shutdown.send(false).ok();\n\n    // Verify that caching works\n    TcpStream::connect(\"127.0.0.1:9199\").await.unwrap_err();\n\n    let mut requests = Vec::new();\n    for n in 0..100 {\n        let (item, expected) = &tests[n % tests.len()];\n        if matches!(item, Item::IsAccount(_)) {\n            let item = item.append(n);\n            let item_clone = item.clone();\n            let handle = handle.clone();\n            let core = core.clone();\n            requests.push((\n                tokio::spawn(async move {\n                    let result: LookupResult = match &item {\n                        Item::IsAccount(v) => {\n                            (core.rcpt(&handle, v, 0).await.unwrap() == RcptType::Mailbox).into()\n                        }\n                        _ => unreachable!(),\n                    };\n\n                    result\n                }),\n                item_clone,\n                expected.append(n),\n            ));\n        }\n    }\n    assert!(!requests.is_empty());\n    for (result, item, expected_result) in requests {\n        assert_eq!(\n            result.await.unwrap(),\n            expected_result,\n            \"Failed for {item:?}\"\n        );\n    }\n}\n\npub fn spawn_mock_lmtp_server(max_concurrency: u64) -> watch::Sender<bool> {\n    let (tx, rx) = watch::channel(true);\n\n    tokio::spawn(async move {\n        let listener = TcpListener::bind(\"127.0.0.1:9199\")\n            .await\n            .unwrap_or_else(|e| {\n                panic!(\"Failed to bind mock SMTP server to 127.0.0.1:9199: {e}\");\n            });\n        let acceptor = dummy_tls_acceptor();\n        let limited = ConcurrencyLimiter::new(max_concurrency);\n        let mut rx_ = rx.clone();\n        loop {\n            tokio::select! {\n                stream = listener.accept() => {\n                    match stream {\n                        Ok((stream, _)) => {\n                            let acceptor = acceptor.clone();\n                            let in_flight = limited.is_allowed();\n                            tokio::spawn(accept_smtp(stream, rx.clone(), acceptor, in_flight.into()));\n                        }\n                        Err(err) => {\n                            panic!(\"Something went wrong: {err}\" );\n                        }\n                    }\n                },\n                _ = rx_.changed() => {\n                    break;\n                }\n            };\n        }\n    });\n\n    tx\n}\n\nasync fn accept_smtp(\n    stream: TcpStream,\n    mut rx: watch::Receiver<bool>,\n    acceptor: Arc<TlsAcceptor>,\n    in_flight: Option<InFlight>,\n) {\n    let mut stream = acceptor.accept(stream).await.unwrap();\n    stream\n        .write_all(b\"220 [127.0.0.1] Clueless host service ready\\r\\n\")\n        .await\n        .unwrap();\n\n    if in_flight.is_none() {\n        eprintln!(\"WARNING: Concurrency exceeded!\");\n    }\n\n    let mut buf_u8 = vec![0u8; 1024];\n\n    loop {\n        let br = tokio::select! {\n            br = stream.read(&mut buf_u8) => {\n                match br {\n                    Ok(br) => {\n                        br\n                    }\n                    Err(_) => {\n                        break;\n                    }\n                }\n            },\n            _ = rx.changed() => {\n                break;\n            }\n        };\n\n        let buf = std::str::from_utf8(&buf_u8[0..br]).unwrap();\n        let response = if buf.starts_with(\"LHLO\") {\n            \"250-mx.foobar.org\\r\\n250 AUTH PLAIN\\r\\n\".into()\n        } else if buf.starts_with(\"MAIL FROM\") {\n            if buf.contains(\"<>\") || buf.contains(\"ok@\") {\n                \"250 OK\\r\\n\".into()\n            } else {\n                \"552-I do not\\r\\n552 like that MAIL FROM.\\r\\n\".into()\n            }\n        } else if buf.starts_with(\"RCPT TO\") {\n            if buf.contains(\"ok\") {\n                \"250 OK\\r\\n\".into()\n            } else {\n                \"550-I refuse to\\r\\n550 accept that recipient.\\r\\n\".into()\n            }\n        } else if buf.starts_with(\"VRFY\") {\n            if buf.contains(\"ok\") {\n                format!(\"250 {}\\r\\n\", buf.split_once(' ').unwrap().1)\n            } else {\n                \"550-I refuse to\\r\\n550 verify that recipient.\\r\\n\".into()\n            }\n        } else if buf.starts_with(\"EXPN\") {\n            if buf.contains(\"ok\") {\n                let parts = buf\n                    .split_once(' ')\n                    .unwrap()\n                    .1\n                    .split(',')\n                    .filter_map(|s| {\n                        if !s.is_empty() {\n                            s.to_string().into()\n                        } else {\n                            None\n                        }\n                    })\n                    .collect::<Vec<_>>();\n                let mut buf = String::with_capacity(16);\n                for (pos, part) in parts.iter().enumerate() {\n                    buf.push_str(\"250\");\n                    buf.push(if pos == parts.len() - 1 { ' ' } else { '-' });\n                    buf.push_str(part);\n                    buf.push_str(\"\\r\\n\");\n                }\n\n                buf\n            } else {\n                \"550-I refuse to\\r\\n550 accept that recipient.\\r\\n\".into()\n            }\n        } else if buf.starts_with(\"AUTH PLAIN\") {\n            let buf = base64_decode(buf.rsplit_once(' ').unwrap().1.as_bytes()).unwrap();\n            if String::from_utf8_lossy(&buf).contains(\"ok\") {\n                \"235 Great success!\\r\\n\".into()\n            } else {\n                \"535 No soup for you\\r\\n\".into()\n            }\n        } else if buf.starts_with(\"NOOP\") {\n            \"250 Siesta time\\r\\n\".into()\n        } else if buf.starts_with(\"QUIT\") {\n            \"250 Arrivederci!\\r\\n\".into()\n        } else if buf.starts_with(\"RSET\") {\n            \"250 Your wish is my command.\\r\\n\".into()\n        } else {\n            panic!(\"Unknown command: {}\", buf.trim());\n        };\n        //print!(\"<- {}\", response);\n        stream.write_all(response.as_bytes()).await.unwrap();\n\n        if buf.contains(\"bye\") || buf.starts_with(\"QUIT\") {\n            return;\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/directory/sql.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse directory::{\n    QueryParams, ROLE_ADMIN, ROLE_USER, Type,\n    backend::{RcptType, internal::manage::ManageDirectory},\n};\nuse mail_send::Credentials;\n\n#[allow(unused_imports)]\nuse store::{InMemoryStore, Store};\n\nuse crate::{\n    directory::{DirectoryTest, IntoTestPrincipal, TestPrincipal, map_account_id, map_account_ids},\n    store::cleanup::store_destroy,\n};\n\nuse super::DirectoryStore;\n\n#[tokio::test]\nasync fn sql_directory() {\n    // Enable logging\n    /*tracing::subscriber::set_global_default(\n        tracing_subscriber::FmtSubscriber::builder()\n            .with_max_level(tracing::Level::TRACE)\n            .finish(),\n    )\n    .unwrap();*/\n\n    // Obtain directory handle\n    for directory_id in [\"sqlite\", \"postgresql\", \"mysql\"] {\n        // Parse config\n        let mut config = DirectoryTest::new(directory_id.into()).await;\n\n        println!(\"Testing SQL directory {:?}\", directory_id);\n        let handle = config.directories.directories.remove(directory_id).unwrap();\n        let store = DirectoryStore {\n            store: config.stores.stores.remove(directory_id).unwrap(),\n        };\n        let base_store = &store.store;\n        let core = config.server;\n\n        // Create tables\n        store_destroy(base_store).await;\n        store.create_test_directory().await;\n\n        // Create test users\n        store\n            .create_test_user(\"admin\", \"very_secret\", \"Administrator\")\n            .await;\n        store.create_test_user(\"john\", \"12345\", \"John Doe\").await;\n        store.create_test_user(\"jane\", \"abcde\", \"Jane Doe\").await;\n        store\n            .create_test_user(\n                \"bill\",\n                \"$2y$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe\",\n                \"Bill Foobar\",\n            )\n            .await;\n        store.set_test_quota(\"bill\", 500000).await;\n\n        // Create test groups\n        store.create_test_group(\"sales\", \"Sales Team\").await;\n        store.create_test_group(\"support\", \"Support Team\").await;\n\n        // Link users to groups\n        store.add_to_group(\"john\", \"sales\").await;\n        store.add_to_group(\"jane\", \"sales\").await;\n        store.add_to_group(\"jane\", \"support\").await;\n\n        // Add email addresses\n        store\n            .link_test_address(\"john\", \"john@example.org\", \"primary\")\n            .await;\n        store\n            .link_test_address(\"jane\", \"jane@example.org\", \"primary\")\n            .await;\n        store\n            .link_test_address(\"bill\", \"bill@example.org\", \"primary\")\n            .await;\n\n        // Add aliases and lists\n        store\n            .link_test_address(\"john\", \"john.doe@example.org\", \"alias\")\n            .await;\n        store\n            .link_test_address(\"john\", \"jdoe@example.org\", \"alias\")\n            .await;\n        store\n            .link_test_address(\"john\", \"info@example.org\", \"list\")\n            .await;\n        store\n            .link_test_address(\"jane\", \"info@example.org\", \"list\")\n            .await;\n        store\n            .link_test_address(\"bill\", \"info@example.org\", \"list\")\n            .await;\n\n        // Add catch-all user\n        store\n            .create_test_user(\"robert\", \"abcde\", \"Robert Foobar\")\n            .await;\n        store\n            .link_test_address(\"robert\", \"robert@catchall.org\", \"primary\")\n            .await;\n        store\n            .link_test_address(\"robert\", \"@catchall.org\", \"alias\")\n            .await;\n\n        // Test authentication\n        assert_eq!(\n            handle\n                .query(\n                    QueryParams::credentials(&Credentials::Plain {\n                        username: \"john\".into(),\n                        secret: \"12345\".into()\n                    })\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .unwrap()\n                .into_test(),\n            TestPrincipal {\n                id: base_store.get_principal_id(\"john\").await.unwrap().unwrap(),\n                name: \"john\".into(),\n                description: Some(\"John Doe\".into()),\n                secrets: vec![\"12345\".into()],\n                typ: Type::Individual,\n                member_of: map_account_ids(base_store, vec![\"sales\"])\n                    .await\n                    .into_iter()\n                    .map(|v| v.to_string())\n                    .collect(),\n                emails: vec![\n                    \"john@example.org\".into(),\n                    \"jdoe@example.org\".into(),\n                    \"john.doe@example.org\".into()\n                ],\n                roles: vec![ROLE_USER.to_string()],\n                ..Default::default()\n            }\n        );\n        assert_eq!(\n            handle\n                .query(\n                    QueryParams::credentials(&Credentials::Plain {\n                        username: \"bill\".into(),\n                        secret: \"password\".into()\n                    })\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .unwrap()\n                .into_test(),\n            TestPrincipal {\n                id: base_store.get_principal_id(\"bill\").await.unwrap().unwrap(),\n                name: \"bill\".into(),\n                description: Some(\"Bill Foobar\".into()),\n                secrets: vec![\n                    \"$2y$05$bvIG6Nmid91Mu9RcmmWZfO5HJIMCT8riNW0hEp8f6/FuA2/mHZFpe\".into()\n                ],\n                typ: Type::Individual,\n                quota: 500000,\n                emails: vec![\"bill@example.org\".into(),],\n                roles: vec![ROLE_USER.to_string()],\n                ..Default::default()\n            }\n        );\n        assert_eq!(\n            handle\n                .query(\n                    QueryParams::credentials(&Credentials::Plain {\n                        username: \"admin\".into(),\n                        secret: \"very_secret\".into()\n                    })\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .unwrap()\n                .into_test(),\n            TestPrincipal {\n                id: base_store.get_principal_id(\"admin\").await.unwrap().unwrap(),\n                name: \"admin\".into(),\n                description: Some(\"Administrator\".into()),\n                secrets: vec![\"very_secret\".into()],\n                typ: Type::Individual,\n                roles: vec![ROLE_ADMIN.to_string()],\n                ..Default::default()\n            }\n        );\n        assert!(\n            handle\n                .query(\n                    QueryParams::credentials(&Credentials::Plain {\n                        username: \"bill\".into(),\n                        secret: \"invalid\".into()\n                    })\n                    .with_return_member_of(true)\n                )\n                .await\n                .unwrap()\n                .is_none()\n        );\n\n        // Get user by name\n        let mut p = handle\n            .query(QueryParams::name(\"jane\").with_return_member_of(true))\n            .await\n            .unwrap()\n            .unwrap()\n            .into_test();\n        p.member_of.sort();\n        assert_eq!(\n            p,\n            TestPrincipal {\n                id: base_store.get_principal_id(\"jane\").await.unwrap().unwrap(),\n                name: \"jane\".into(),\n                description: Some(\"Jane Doe\".into()),\n                typ: Type::Individual,\n                secrets: vec![\"abcde\".into()],\n                member_of: map_account_ids(base_store, vec![\"sales\", \"support\"])\n                    .await\n                    .into_iter()\n                    .map(|v| v.to_string())\n                    .collect(),\n                emails: vec![\"jane@example.org\".into(),],\n                roles: vec![ROLE_USER.to_string()],\n                ..Default::default()\n            }\n        );\n\n        // Get group by name\n        assert_eq!(\n            handle\n                .query(QueryParams::name(\"sales\").with_return_member_of(true))\n                .await\n                .unwrap()\n                .unwrap()\n                .into_test(),\n            TestPrincipal {\n                id: base_store.get_principal_id(\"sales\").await.unwrap().unwrap(),\n                name: \"sales\".into(),\n                description: Some(\"Sales Team\".into()),\n                typ: Type::Group,\n                roles: vec![ROLE_USER.to_string()],\n                ..Default::default()\n            }\n        );\n\n        // Ids by email\n        assert_eq!(\n            core.email_to_id(&handle, \"jane@example.org\", 0)\n                .await\n                .unwrap(),\n            Some(map_account_id(base_store, \"jane\").await)\n        );\n        assert_eq!(\n            core.email_to_id(&handle, \"jane+alias@example.org\", 0)\n                .await\n                .unwrap(),\n            Some(map_account_id(base_store, \"jane\").await)\n        );\n        assert_eq!(\n            core.email_to_id(&handle, \"unknown@example.org\", 0)\n                .await\n                .unwrap(),\n            None\n        );\n        assert_eq!(\n            core.email_to_id(&handle, \"anything@catchall.org\", 0)\n                .await\n                .unwrap(),\n            Some(map_account_id(base_store, \"robert\").await)\n        );\n\n        // Domain validation\n        assert!(handle.is_local_domain(\"example.org\").await.unwrap());\n        assert!(!handle.is_local_domain(\"other.org\").await.unwrap());\n\n        // RCPT TO\n        assert_eq!(\n            core.rcpt(&handle, \"jane@example.org\", 0).await.unwrap(),\n            RcptType::Mailbox\n        );\n        assert_eq!(\n            core.rcpt(&handle, \"info@example.org\", 0).await.unwrap(),\n            RcptType::Mailbox\n        );\n        assert_eq!(\n            core.rcpt(&handle, \"jane+alias@example.org\", 0)\n                .await\n                .unwrap(),\n            RcptType::Mailbox\n        );\n        assert_eq!(\n            core.rcpt(&handle, \"info+alias@example.org\", 0)\n                .await\n                .unwrap(),\n            RcptType::Mailbox\n        );\n        assert_eq!(\n            core.rcpt(&handle, \"random_user@catchall.org\", 0)\n                .await\n                .unwrap(),\n            RcptType::Mailbox\n        );\n        assert_eq!(\n            core.rcpt(&handle, \"invalid@example.org\", 0).await.unwrap(),\n            RcptType::Invalid\n        );\n\n        // VRFY\n        assert_eq!(\n            core.vrfy(&handle, \"jane\", 0).await.unwrap(),\n            vec![\"jane@example.org\".to_string()]\n        );\n        assert_eq!(\n            core.vrfy(&handle, \"john\", 0).await.unwrap(),\n            vec![\n                \"john.doe@example.org\".to_string(),\n                \"john@example.org\".to_string(),\n            ]\n        );\n        assert_eq!(\n            core.vrfy(&handle, \"jane+alias@example\", 0).await.unwrap(),\n            vec![\"jane@example.org\".to_string()]\n        );\n        assert_eq!(\n            core.vrfy(&handle, \"info\", 0).await.unwrap(),\n            Vec::<String>::new()\n        );\n        assert_eq!(\n            core.vrfy(&handle, \"invalid\", 0).await.unwrap(),\n            Vec::<String>::new()\n        );\n\n        // EXPN (now handled by the internal store)\n        /*assert_eq!(\n            core.expn(&handle, \"info@example.org\", 0).await.unwrap(),\n            vec![\n                \"bill@example.org\".into(),\n                \"jane@example.org\".into(),\n                \"john@example.org\".into()\n            ]\n        );\n        assert_eq!(\n            core.expn(&handle, \"john@example.org\", 0).await.unwrap(),\n            Vec::<String>::new()\n        );*/\n    }\n}\n\nimpl DirectoryStore {\n    pub async fn create_test_directory(&self) {\n        // Create tables\n        for table in [\"accounts\", \"group_members\", \"emails\"] {\n            self.store\n                .sql_query::<usize>(&format!(\"DROP TABLE IF EXISTS {table}\"), vec![])\n                .await\n                .unwrap();\n        }\n        for query in [\n            concat!(\n                \"CREATE TABLE accounts (name TEXT PRIMARY KEY, secret TEXT, description TEXT,\",\n                \" type TEXT NOT NULL, quota INTEGER \",\n                \"DEFAULT 0, active BOOLEAN DEFAULT TRUE)\"\n            ),\n            concat!(\n                \"CREATE TABLE group_members (name TEXT NOT NULL, member_of \",\n                \"TEXT NOT NULL, PRIMARY KEY (name, member_of))\"\n            ),\n            concat!(\n                \"CREATE TABLE emails (name TEXT NOT NULL, address TEXT NOT\",\n                \" NULL, type TEXT, PRIMARY KEY (name, address))\"\n            ),\n            \"INSERT INTO accounts (name, secret, type) VALUES ('admin', 'secret', 'admin')\",\n        ] {\n            let query = if self.is_mysql() {\n                query.replace(\"TEXT\", \"VARCHAR(255)\")\n            } else {\n                query.into()\n            };\n\n            self.store\n                .sql_query::<usize>(&query, vec![])\n                .await\n                .unwrap_or_else(|_| panic!(\"failed for {query}\"));\n        }\n    }\n\n    pub async fn create_test_user(&self, login: &str, secret: &str, name: &str) {\n        let account_type = if login == \"admin\" {\n            \"admin\"\n        } else {\n            \"individual\"\n        };\n        self.store\n            .sql_query::<usize>(\n                if self.is_postgresql() {\n                    concat!(\n                        \"INSERT INTO accounts (name, secret, description, \",\n                        \"type, active) VALUES ($1, $2, $3, $4, true) \",\n                        \"ON CONFLICT (name) \",\n                        \"DO UPDATE SET secret = $2, description = $3, type = $4, active = true\"\n                    )\n                } else if self.is_mysql() {\n                    concat!(\n                        \"INSERT INTO accounts (name, secret, description, \",\n                        \"type, active) VALUES (?, ?, ?, ?, true) \",\n                        \"ON DUPLICATE KEY UPDATE \",\n                        \"secret = VALUES(secret), description = VALUES(description), \",\n                        \"type = VALUES(type), active = true\"\n                    )\n                } else {\n                    concat!(\n                        \"INSERT INTO accounts (name, secret, description, \",\n                        \"type, active) VALUES (?, ?, ?, ?, true) \",\n                        \"ON CONFLICT(name) DO UPDATE SET \",\n                        \"secret = excluded.secret, description = excluded.description, \",\n                        \"type = excluded.type, active = true\"\n                    )\n                },\n                vec![\n                    login.into(),\n                    secret.into(),\n                    name.into(),\n                    account_type.into(),\n                ],\n            )\n            .await\n            .unwrap();\n    }\n\n    pub async fn create_test_user_with_email(&self, login: &str, secret: &str, name: &str) {\n        self.create_test_user(login, secret, name).await;\n        self.link_test_address(login, login, \"primary\").await;\n    }\n\n    pub async fn create_test_group(&self, login: &str, name: &str) {\n        self.store\n            .sql_query::<usize>(\n                if self.is_postgresql() {\n                    concat!(\n                        \"INSERT INTO accounts (name, description, \",\n                        \"type, active) VALUES ($1, $2, $3, $4) ON CONFLICT (name) DO NOTHING\"\n                    )\n                } else if self.is_mysql() {\n                    concat!(\n                        \"INSERT IGNORE INTO accounts (name, description, \",\n                        \"type, active) VALUES (?, ?, ?, ?)\"\n                    )\n                } else {\n                    concat!(\n                        \"INSERT OR IGNORE INTO accounts (name, description, \",\n                        \"type, active) VALUES (?, ?, ?, ?)\"\n                    )\n                },\n                vec![login.into(), name.into(), \"group\".into(), true.into()],\n            )\n            .await\n            .unwrap();\n    }\n\n    pub async fn create_test_group_with_email(&self, login: &str, name: &str) {\n        self.create_test_group(login, name).await;\n        self.link_test_address(login, login, \"primary\").await;\n    }\n\n    pub async fn link_test_address(&self, login: &str, address: &str, typ: &str) {\n        self.store\n            .sql_query::<usize>(\n                if self.is_postgresql() {\n                    \"INSERT INTO emails (name, address, type) VALUES ($1, $2, $3) ON CONFLICT (name, address) DO NOTHING\"\n                } else if self.is_mysql() {\n                    \"INSERT IGNORE INTO emails (name, address, type) VALUES (?, ?, ?)\"\n                } else {\n                    \"INSERT OR IGNORE INTO emails (name, address, type) VALUES (?, ?, ?)\"\n                },\n                vec![login.into(), address.into(), typ.into()],\n            )\n            .await\n            .unwrap();\n    }\n\n    pub async fn set_test_quota(&self, login: &str, quota: u32) {\n        self.store\n            .sql_query::<usize>(\n                if self.is_postgresql() {\n                    \"UPDATE accounts SET quota = $1 where name = $2\"\n                } else {\n                    \"UPDATE accounts SET quota = ? where name = ?\"\n                },\n                vec![quota.into(), login.into()],\n            )\n            .await\n            .unwrap();\n    }\n\n    pub async fn add_to_group(&self, login: &str, group: &str) {\n        self.store\n            .sql_query::<usize>(\n                if self.is_postgresql() {\n                    \"INSERT INTO group_members (name, member_of) VALUES ($1, $2)\"\n                } else {\n                    \"INSERT INTO group_members (name, member_of) VALUES (?, ?)\"\n                },\n                vec![login.into(), group.into()],\n            )\n            .await\n            .unwrap();\n    }\n\n    pub async fn remove_from_group(&self, login: &str, group: &str) {\n        self.store\n            .sql_query::<usize>(\n                if self.is_postgresql() {\n                    \"DELETE FROM group_members WHERE name = $1 AND member_of = $2\"\n                } else {\n                    \"DELETE FROM group_members WHERE name = ? AND member_of = ?\"\n                },\n                vec![login.into(), group.into()],\n            )\n            .await\n            .unwrap();\n    }\n\n    pub async fn remove_test_alias(&self, login: &str, alias: &str) {\n        self.store\n            .sql_query::<usize>(\n                if self.is_postgresql() {\n                    \"DELETE FROM emails WHERE name = $1 AND address = $2\"\n                } else {\n                    \"DELETE FROM emails WHERE name = ? AND address = ?\"\n                },\n                vec![login.into(), alias.into()],\n            )\n            .await\n            .unwrap();\n    }\n\n    fn is_mysql(&self) -> bool {\n        #[cfg(feature = \"mysql\")]\n        {\n            matches!(self.store, Store::MySQL(_))\n        }\n        #[cfg(not(feature = \"mysql\"))]\n        {\n            false\n        }\n    }\n\n    fn is_postgresql(&self) -> bool {\n        #[cfg(feature = \"postgres\")]\n        {\n            matches!(self.store, Store::PostgreSQL(_))\n        }\n        #[cfg(not(feature = \"postgres\"))]\n        {\n            false\n        }\n    }\n\n    #[allow(dead_code)]\n    fn is_sqlite(&self) -> bool {\n        #[cfg(feature = \"sqlite\")]\n        {\n            matches!(self.store, Store::SQLite(_))\n        }\n        #[cfg(not(feature = \"sqlite\"))]\n        {\n            false\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/http_server.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse ahash::AHashMap;\nuse common::{Caches, Core, Data, Inner, config::server::Listeners, listener::SessionData};\nuse http_proto::{HttpResponse, request::fetch_body};\nuse hyper::{Method, Uri, body, server::conn::http1, service::service_fn};\nuse hyper_util::rt::TokioIo;\nuse tokio::sync::watch;\nuse utils::config::Config;\n\nuse crate::{AssertConfig, add_test_certs};\n\nconst MOCK_HTTP_SERVER: &str = r#\"\n[server]\nhostname = \"'oidc.example.org'\"\n\n[http]\nurl = \"'https://127.0.0.1:9090'\"\n\n[server.listener.jmap]\nbind = ['127.0.0.1:9090']\nprotocol = 'http'\ntls.implicit = true\n\n[server.socket]\nreuse-addr = true\n\n[certificate.default]\ncert = '%{file:{CERT}}%'\nprivate-key = '%{file:{PK}}%'\ndefault = true\n\"#;\n\n#[derive(Clone)]\npub struct HttpSessionManager {\n    inner: HttpRequestHandler,\n}\n\npub type HttpRequestHandler = Arc<dyn Fn(HttpMessage) -> HttpResponse + Sync + Send>;\n\n#[derive(Debug)]\npub struct HttpMessage {\n    pub method: Method,\n    pub headers: AHashMap<String, String>,\n    pub uri: Uri,\n    pub body: Option<Vec<u8>>,\n}\n\nimpl HttpMessage {\n    pub fn get_url_encoded(&self, key: &str) -> Option<String> {\n        form_urlencoded::parse(self.body.as_ref()?.as_slice())\n            .find(|(k, _)| k == key)\n            .map(|(_, v)| v.into_owned())\n    }\n}\n\npub async fn spawn_mock_http_server(\n    handler: HttpRequestHandler,\n) -> (watch::Sender<bool>, watch::Receiver<bool>) {\n    // Start mock push server\n    let mut settings = Config::new(add_test_certs(MOCK_HTTP_SERVER)).unwrap();\n    settings.resolve_all_macros().await;\n    let mock_inner = Arc::new(Inner {\n        shared_core: Core::parse(&mut settings, Default::default(), Default::default())\n            .await\n            .into_shared(),\n        data: Data::parse(&mut settings),\n        cache: Caches::parse(&mut settings),\n        ..Default::default()\n    });\n    settings.errors.clear();\n    settings.warnings.clear();\n    let mut servers = Listeners::parse(&mut settings);\n    servers.parse_tcp_acceptors(&mut settings, mock_inner.clone());\n\n    // Start JMAP server\n    servers.bind_and_drop_priv(&mut settings);\n    settings.assert_no_errors();\n    servers.spawn(|server, acceptor, shutdown_rx| {\n        server.spawn(\n            HttpSessionManager {\n                inner: handler.clone(),\n            },\n            mock_inner.clone(),\n            acceptor,\n            shutdown_rx,\n        );\n    })\n}\n\nimpl common::listener::SessionManager for HttpSessionManager {\n    #[allow(clippy::manual_async_fn)]\n    fn handle<T: common::listener::SessionStream>(\n        self,\n        session: SessionData<T>,\n    ) -> impl std::future::Future<Output = ()> + Send {\n        async move {\n            let sender = self.inner;\n            let _ = http1::Builder::new()\n                .keep_alive(false)\n                .serve_connection(\n                    TokioIo::new(session.stream),\n                    service_fn(|mut req: hyper::Request<body::Incoming>| {\n                        let sender = sender.clone();\n\n                        async move {\n                            let response = sender(HttpMessage {\n                                method: req.method().clone(),\n                                uri: req.uri().clone(),\n                                headers: req\n                                    .headers()\n                                    .iter()\n                                    .map(|(k, v)| {\n                                        (k.as_str().to_lowercase(), v.to_str().unwrap().to_string())\n                                    })\n                                    .collect(),\n                                body: fetch_body(&mut req, 1024 * 1024, 0).await,\n                            });\n\n                            Ok::<_, hyper::Error>(response.build())\n                        }\n                    }),\n                )\n                .await;\n        }\n    }\n\n    #[allow(clippy::manual_async_fn)]\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send {\n        async {}\n    }\n}\n"
  },
  {
    "path": "tests/src/imap/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::mail::delivery::SmtpConnection;\n\nuse super::{AssertResult, ImapConnection, Type, append::assert_append_message};\nuse imap_proto::ResponseType;\n\npub async fn test(mut imap_john: &mut ImapConnection, _imap_check: &mut ImapConnection) {\n    // Delivery to support account\n    println!(\"Running ACL tests...\");\n    let mut lmtp = SmtpConnection::connect_port(11201).await;\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"support@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: support@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n\n    // Connect to all test accounts\n    let mut imap_jane = ImapConnection::connect(b\"_w \").await;\n    let mut imap_bill = ImapConnection::connect(b\"_z \").await;\n    for (imap, secret) in [\n        (&mut imap_jane, \"AGphbmUuc21pdGhAZXhhbXBsZS5jb20Ac2VjcmV0\"),\n        (&mut imap_bill, \"AGZvb2JhckBleGFtcGxlLmNvbQBzZWNyZXQ=\"),\n    ] {\n        imap.assert_read(Type::Untagged, ResponseType::Ok).await;\n        imap.send(&format!(\n            \"AUTHENTICATE PLAIN {{{}+}}\\r\\n{}\",\n            secret.len(),\n            secret\n        ))\n        .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    }\n\n    // Jane should see the Support account\n    imap_jane.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"Shared Folders/support@example.com/INBOX\");\n    imap_jane\n        .send(\"SELECT \\\"Shared Folders/support@example.com/INBOX\\\"\")\n        .await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_jane.send(\"FETCH 1 (PREVIEW)\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"TPS reports ASAP\");\n    imap_jane.send(\"UNSELECT\").await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Jane should be able to create folders under the Support account\n    imap_jane\n        .send(\"CREATE \\\"Shared Folders/support@example.com/inbox/Jane's Folder\\\"\")\n        .await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_jane.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\n            \"* LIST () \\\"/\\\" \\\"Shared Folders/support@example.com/INBOX/Jane's Folder\\\"\",\n        );\n    imap_jane\n        .send(\"DELETE \\\"Shared Folders/support@example.com/INBOX/Jane's Folder\\\"\")\n        .await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // John should have no shared folders\n    imap_john.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"Shared Folders\", 0);\n    imap_john.send(\"NAMESPACE\").await;\n    imap_john.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // List rights\n    imap_jane.send(\"LISTRIGHTS INBOX jdoe@example.com\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* LISTRIGHTS \\\"INBOX\\\" \\\"jdoe@example.com\\\" r l ws i et k x p a\");\n\n    // Jane shares her Inbox to John, expect a Shared Folders item in John's list\n    imap_jane.send(\"SETACL INBOX jdoe@example.com lr\").await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_john.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* LIST (\\\\NoSelect) \\\"/\\\" \\\"Shared Folders\\\"\")\n        .assert_equals(\"* LIST (\\\\NoSelect) \\\"/\\\" \\\"Shared Folders/jane.smith@example.com\\\"\")\n        .assert_equals(\"* LIST () \\\"/\\\" \\\"Shared Folders/jane.smith@example.com/INBOX\\\"\");\n\n    // Grant access to Bill and check ACLs\n    imap_jane.send(\"GETACL INBOX\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"jdoe@example.com\\\" rl\");\n\n    imap_jane\n        .send(\"SETACL INBOX foobar@example.com lrxtws\")\n        .await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_jane.send(\"GETACL INBOX\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"jdoe@example.com\\\" rl\")\n        .assert_contains(\"\\\"foobar@example.com\\\" tewsrxl\");\n\n    imap_bill.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"Shared Folders/jane.smith@example.com/INBOX\");\n\n    // Namespace should now return the Shared Folders namespace\n    imap_john.send(\"NAMESPACE\").await;\n    imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* NAMESPACE ((\\\"\\\" \\\"/\\\")) ((\\\"Shared Folders\\\" \\\"/\\\")) NIL\");\n\n    // List John's right on Jane's Inbox\n    imap_john\n        .send(\"MYRIGHTS \\\"Shared Folders/jane.smith@example.com/INBOX\\\"\")\n        .await;\n    imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* MYRIGHTS \\\"Shared Folders/jane.smith@example.com/INBOX\\\" rl\");\n\n    // John should not be able to append messages\n    assert_append_message(\n        imap_john,\n        \"Shared Folders/jane.smith@example.com/INBOX\",\n        \"From: john\\n\\ncontents\",\n        ResponseType::No,\n    )\n    .await;\n\n    // Grant insert access to John on Jane's Inbox, and try inserting the\n    // message again.\n    imap_jane.send(\"SETACL INBOX jdoe@example.com +i\").await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_john\n        .send(\"MYRIGHTS \\\"Shared Folders/jane.smith@example.com/INBOX\\\"\")\n        .await;\n    imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* MYRIGHTS \\\"Shared Folders/jane.smith@example.com/INBOX\\\" rli\");\n    assert_append_message(\n        imap_john,\n        \"Shared Folders/jane.smith@example.com/INBOX\",\n        \"From: john\\n\\ncontents\",\n        ResponseType::Ok,\n    )\n    .await;\n\n    // Only Bill should be allowed to delete messages on Jane's Inbox\n    for imap in [&mut imap_john, &mut imap_bill] {\n        imap.send(\"SELECT \\\"Shared Folders/jane.smith@example.com/INBOX\\\"\")\n            .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    }\n    imap_john.send(\"UID STORE 1 +FLAGS (\\\\Deleted)\").await;\n    imap_john.assert_read(Type::Tagged, ResponseType::No).await;\n\n    imap_bill.send(\"UID STORE 1 +FLAGS (\\\\Deleted)\").await;\n    imap_bill.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_john.send(\"UID EXPUNGE\").await;\n    imap_john.assert_read(Type::Tagged, ResponseType::No).await;\n\n    imap_john.send(\"UID FETCH 1 (PREVIEW)\").await;\n    imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"contents\");\n\n    imap_bill.send(\"UID EXPUNGE\").await;\n    imap_bill.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_bill.send(\"UID FETCH 1 (PREVIEW)\").await;\n    imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"contents\", 0);\n\n    imap_bill\n        .send(\"STATUS \\\"Shared Folders/jane.smith@example.com/INBOX\\\" (MESSAGES)\")\n        .await;\n    imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"(MESSAGES 0)\");\n\n    // Test copying and moving between shared mailboxes\n    let uid = assert_append_message(\n        imap_john,\n        \"INBOX\",\n        \"From: john\\n\\ncopy test\",\n        ResponseType::Ok,\n    )\n    .await\n    .into_append_uid();\n\n    imap_john.send(\"SELECT INBOX\").await;\n    imap_john.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Copy from John's Inbox to Jane's Inbox\n    imap_john\n        .send(&format!(\n            \"UID COPY {} \\\"Shared Folders/jane.smith@example.com/INBOX\\\"\",\n            uid\n        ))\n        .await;\n    let uid = imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .into_copy_uid();\n\n    // Check that both Bill and Jane can see the message\n    imap_bill.send(\"NOOP\").await;\n    imap_bill.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_bill\n        .send(&format!(\"UID FETCH {} (PREVIEW)\", uid))\n        .await;\n    imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"copy test\");\n\n    imap_jane.send(\"SELECT INBOX\").await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_jane\n        .send(&format!(\"UID FETCH {} (PREVIEW)\", uid))\n        .await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"copy test\");\n\n    // Bill now moves the message to his own Inbox\n    imap_bill.send(&format!(\"UID MOVE {} INBOX\", uid)).await;\n    let uid_moved = imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .into_copy_uid();\n\n    // Both Jane and Bill should not see the message on Jane's Inbox anymore\n    imap_bill\n        .send(&format!(\"UID FETCH {} (PREVIEW)\", uid))\n        .await;\n    imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"copy test\", 0);\n\n    imap_jane\n        .send(&format!(\"UID FETCH {} (PREVIEW)\", uid))\n        .await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"copy test\", 0);\n\n    // Check that the message has been moved to Bill's Inbox\n    imap_bill.send(\"SELECT INBOX\").await;\n    imap_bill.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_bill\n        .send(&format!(\"UID FETCH {} (PREVIEW)\", uid_moved))\n        .await;\n    imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"copy test\");\n\n    // Jane stops sharing with Bill, and removes Insert access to John\n    imap_jane.send(\"DELETEACL INBOX foobar@example.com\").await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_jane.send(\"SETACL INBOX jdoe@example.com -i\").await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_jane.send(\"GETACL INBOX\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"jdoe@example.com\\\" rl\")\n        .assert_count(\"foobar@example.com\", 0);\n\n    // Bill should not have access to Jane's Inbox anymore\n    imap_bill.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_bill\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"Shared Folders\", 0);\n\n    // And John should still have access\n    imap_john.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_john\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"Shared Folders\", 3);\n}\n"
  },
  {
    "path": "tests/src/imap/antispam.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{IMAPTest, ImapConnection};\nuse crate::{imap::Type, jmap::mail::delivery::SmtpConnection, smtp::session::VerifyResponse};\nuse common::{Server, manager::SPAM_TRAINER_KEY};\nuse imap_proto::ResponseType;\nuse spam_filter::modules::classifier::{SpamClassifier, SpamTrainer};\nuse store::{\n    Deserialize, IterateParams, U32_LEN, U64_LEN, ValueKey,\n    write::{AlignedBytes, Archive, BlobOp, ValueClass, key::DeserializeBigEndian},\n};\nuse types::blob_hash::BlobHash;\n\npub async fn test(handle: &IMAPTest) {\n    println!(\"Running Spam classifier tests...\");\n    let mut imap = ImapConnection::connect(b\"_x \").await;\n    imap.assert_read(Type::Untagged, ResponseType::Ok).await;\n    imap.authenticate(\"sgd@example.com\", \"secret\").await;\n\n    let account_id = handle\n        .server\n        .directory()\n        .email_to_id(\"sgd@example.com\")\n        .await\n        .unwrap()\n        .unwrap();\n\n    // Make sure there are no training samples\n    spam_delete_samples(&handle.server).await;\n    assert_eq!(spam_training_samples(&handle.server).await.total_count, 0);\n\n    // Train the classifier via APPEND\n    imap.append(\"INBOX\", HAM[0]).await;\n    imap.append(\"Junk Mail\", SPAM[0]).await;\n    let samples = spam_training_samples(&handle.server).await;\n    assert_eq!(samples.ham_count, 1);\n    assert_eq!(samples.spam_count, 1);\n\n    // Append two spam samples to \"Drafts\", then train the classifier via STORE and MOVE\n    imap.append(\"Drafts\", SPAM[1]).await;\n    imap.append(\"Drafts\", SPAM[2]).await;\n    imap.send_ok(\"SELECT Drafts\").await;\n    imap.send_ok(\"STORE 1 +FLAGS ($Junk)\").await;\n    imap.send_ok(\"MOVE 2 \\\"Junk Mail\\\"\").await;\n    let samples = spam_training_samples(&handle.server).await;\n    assert_eq!(samples.ham_count, 1);\n    assert_eq!(samples.spam_count, 3);\n\n    // Add the remaining messages via APPEND\n    for message in HAM.iter().skip(1) {\n        imap.append(\"INBOX\", message).await;\n    }\n    for message in SPAM.iter().skip(3) {\n        imap.append(\"Junk Mail\", message).await;\n    }\n    let samples = spam_training_samples(&handle.server).await;\n    assert_eq!(samples.ham_count, 10);\n    assert_eq!(samples.spam_count, 10);\n    assert_eq!(samples.samples.len(), 20);\n    assert!(\n        samples\n            .samples\n            .iter()\n            .all(|s| s.account_id == account_id && s.remove.is_none())\n    );\n\n    // Train the classifier\n    handle.server.spam_train(false).await.unwrap();\n    let model = spam_classifier_model(&handle.server).await;\n    assert_eq!(model.reservoir.ham.total_seen, 10);\n    assert_eq!(model.reservoir.spam.total_seen, 10);\n    assert_eq!(\n        model.last_sample_expiry,\n        samples.samples.iter().map(|s| s.until).max().unwrap()\n    );\n    assert_eq!(spam_training_samples(&handle.server).await.total_count, 20);\n    assert!(handle.server.inner.data.spam_classifier.load().is_active());\n\n    // Send 3 test emails\n    for message in TEST {\n        let mut lmtp = SmtpConnection::connect_port(11201).await;\n        lmtp.ingest(\"bill@example.com\", &[\"sgd@example.com\"], message)\n            .await;\n    }\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    imap.send_ok(\"SELECT INBOX\").await;\n    imap.send(\"FETCH 11 (FLAGS RFC822.TEXT)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_not_contains(\"FLAGS ($Junk\")\n        .assert_contains(\"Subject: can someone explain\")\n        .assert_contains(\"X-Spam-Status: No\")\n        .assert_contains(\"PROB_HAM_HIGH\");\n    imap.send(\"FETCH 12 (FLAGS RFC822.TEXT)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_not_contains(\"FLAGS ($Junk\")\n        .assert_contains(\"Subject: classifier test\")\n        .assert_contains(\"X-Spam-Status: No\")\n        .assert_contains(\"PROB_SPAM_UNCERTAIN\");\n    imap.send_ok(\"SELECT \\\"Junk Mail\\\"\").await;\n    imap.send(\"FETCH 10 (FLAGS RFC822.TEXT)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"FLAGS ($Junk\")\n        .assert_contains(\"Subject: save up to\")\n        .assert_contains(\"X-Spam-Status: Yes\")\n        .assert_contains(\"PROB_SPAM_HIGH\");\n    imap.send_ok(\"MOVE 10 INBOX\").await;\n    let samples = spam_training_samples(&handle.server).await;\n    assert_eq!(samples.ham_count, 11);\n    assert_eq!(samples.spam_count, 10);\n\n    // Make sure spam traps trigger spam classification\n    let mut lmtp = SmtpConnection::connect_port(11201).await;\n    lmtp.ingest(\"bill@example.com\", &[\"spamtrap@example.com\"], SPAM[4])\n        .await;\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    let samples = spam_training_samples(&handle.server).await;\n    assert_eq!(samples.ham_count, 11);\n    assert_eq!(samples.spam_count, 11);\n}\n\n#[derive(Default, Debug)]\npub struct TrainingSamples {\n    pub samples: Vec<TrainingSample>,\n    pub spam_count: usize,\n    pub ham_count: usize,\n    pub total_count: usize,\n}\n\n#[derive(Debug)]\n#[allow(dead_code)]\npub struct TrainingSample {\n    pub hash: BlobHash,\n    pub account_id: u32,\n    pub is_spam: bool,\n    pub remove: Option<u64>,\n    pub until: u64,\n}\n\npub async fn spam_classifier_model(server: &Server) -> SpamTrainer {\n    server\n        .blob_store()\n        .get_blob(SPAM_TRAINER_KEY, 0..usize::MAX)\n        .await\n        .and_then(|archive| match archive {\n            Some(archive) => <Archive<AlignedBytes> as Deserialize>::deserialize(&archive)\n                .and_then(|archive| archive.deserialize_untrusted::<SpamTrainer>())\n                .map(Some),\n            None => Ok(None),\n        })\n        .unwrap()\n        .unwrap()\n}\n\npub async fn spam_delete_samples(server: &Server) {\n    let from_key = ValueKey {\n        account_id: 0,\n        collection: 0,\n        document_id: 0,\n        class: ValueClass::Blob(BlobOp::SpamSample {\n            hash: BlobHash::default(),\n            until: 0,\n        }),\n    };\n    let to_key = ValueKey {\n        account_id: u32::MAX,\n        collection: u8::MAX,\n        document_id: u32::MAX,\n        class: ValueClass::Blob(BlobOp::SpamSample {\n            hash: BlobHash::new_max(),\n            until: u64::MAX,\n        }),\n    };\n    server.store().delete_range(from_key, to_key).await.unwrap();\n}\n\npub async fn spam_training_samples(server: &Server) -> TrainingSamples {\n    let mut samples = TrainingSamples::default();\n    let from_key = ValueKey {\n        account_id: 0,\n        collection: 0,\n        document_id: 0,\n        class: ValueClass::Blob(BlobOp::SpamSample {\n            hash: BlobHash::default(),\n            until: 0,\n        }),\n    };\n    let to_key = ValueKey {\n        account_id: u32::MAX,\n        collection: u8::MAX,\n        document_id: u32::MAX,\n        class: ValueClass::Blob(BlobOp::SpamSample {\n            hash: BlobHash::new_max(),\n            until: u64::MAX,\n        }),\n    };\n    server\n        .store()\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                let until = key.deserialize_be_u64(1)?;\n                let account_id = key.deserialize_be_u32(U64_LEN + 1)?;\n                let hash =\n                    BlobHash::try_from_hash_slice(key.get(U64_LEN + U32_LEN + 1..).ok_or_else(\n                        || trc::Error::corrupted_key(key, value.into(), trc::location!()),\n                    )?)\n                    .unwrap();\n                let (Some(is_spam), Some(hold)) = (value.first(), value.get(1)) else {\n                    return Err(trc::Error::corrupted_key(\n                        key,\n                        value.into(),\n                        trc::location!(),\n                    ));\n                };\n\n                let do_remove = *hold == 0;\n                let is_spam = *is_spam == 1;\n                samples.samples.push(TrainingSample {\n                    hash,\n                    account_id,\n                    is_spam,\n                    remove: do_remove.then_some(until),\n                    until,\n                });\n                if is_spam {\n                    samples.spam_count += 1;\n                } else {\n                    samples.ham_count += 1;\n                }\n                samples.total_count += 1;\n\n                Ok(true)\n            },\n        )\n        .await\n        .unwrap();\n\n    samples\n}\n\nimpl ImapConnection {\n    async fn append(&mut self, mailbox: &str, message: &str) {\n        self.send_ok(&format!(\n            \"APPEND {:?} {{{}+}}\\r\\n{}\",\n            mailbox,\n            message.len(),\n            message\n        ))\n        .await;\n    }\n\n    async fn send_ok(&mut self, cmd: &str) {\n        self.send(cmd).await;\n        self.assert_read(Type::Tagged, ResponseType::Ok).await;\n    }\n}\n\npub const SPAM: [&str; 10] = [\n    concat!(\n        \"Subject: save up to = on life insurance\\r\\n\\r\\n wh\",\n        \"y spend more than you have to life quote savings e\",\n        \"nsuring your family s financial security is very i\",\n        \"mportant life quote savings makes buying life insu\",\n        \"rance simple and affordable we provide free access\",\n        \" to the very best companies and the lowest rates l\",\n        \"ife quote savings is fast easy and saves you money\",\n        \" let us help you get started with the best values \",\n        \"in the country on new coverage you can save hundre\",\n        \"ds or even thousands of dollars by requesting a fr\",\n        \"ee quote from lifequote savings our service will t\",\n        \"ake you less than = minutes to complete shop \",\n        \"and compare save up to = on all types of life\",\n        \" insurance hyperlink click here for your free quot\",\n        \"e protecting your family is the best investment yo\",\n        \"u ll ever make if you are in receipt of this email\",\n        \" in error and or wish to be removed from our list \",\n        \"hyperlink please click here and type remove if you\",\n        \" reside in any state which prohibits e mail solici\",\n        \"tations for insurance please disregard this email\\r\\n\",\n        \" \\r\\n\"\n    ),\n    concat!(\n        \"Subject: a powerhouse gifting program\\r\\n\\r\\nyou don t \",\n        \"want to miss get in with the founders the major pl\",\n        \"ayers are on this one for once be where the player\",\n        \"s are this is your private invitation experts are \",\n        \"calling this the fastest way to huge cash flow eve\",\n        \"r conceived leverage = = into = NUM\",\n        \"BER over and over again the question here is you e\",\n        \"ither want to be wealthy or you don t which one ar\",\n        \"e you i am tossing you a financial lifeline and fo\",\n        \"r your sake i hope you grab onto it and hold on ti\",\n        \"ght for the ride of your life testimonials hear wh\",\n        \"at average people are doing their first few days w\",\n        \"e ve received = = in = day and we a\",\n        \"re doing that over and over again q s in al i m a \",\n        \"single mother in fl and i ve received = NUMBE\",\n        \"R in the last = days d s in fl i was not sure\",\n        \" about this when i sent off my = = pledg\",\n        \"e but i got back = = the very next day l\",\n        \" l in ky i didn t have the money so i found myself\",\n        \" a partner to work this with we have received NUMB\",\n        \"ER = over the last = days i think i made\",\n        \" the right decision don t you k c in fl i pick up \",\n        \"= = my first day and i they gave me free\",\n        \" leads and all the training you can too j w in ca \",\n        \"announcing we will close your sales for you and he\",\n        \"lp you get a fax blast immediately upon your entry\",\n        \" you make the money free leads training don t wait\",\n        \" call now fax back to = = = = \",\n        \"or call = = = = name__________\",\n        \"________________________phone_____________________\",\n        \"______________________ fax________________________\",\n        \"_____________email________________________________\",\n        \"____________ best time to call____________________\",\n        \"_____time zone____________________________________\",\n        \"____ this message is sent in compliance of the new\",\n        \" e mail bill per section = paragraph a =\",\n        \" c of s = further transmissions by the sender\",\n        \" of this email may be stopped at no cost to you by\",\n        \" sending a reply to this email address with the wo\",\n        \"rd remove in the subject line errors omissions and\",\n        \" exceptions excluded this is not spam i have compi\",\n        \"led this list from our replicate database relative\",\n        \" to seattle marketing group the gigt or turbo team\",\n        \" for the sole purpose of these communications your\",\n        \" continued inclusion is only by your gracious perm\",\n        \"ission if you wish to not receive this mail from m\",\n        \"e please send an email to tesrewinter  with rem\",\n        \"ove in the subject and you will be deleted immedia\",\n        \"tely\\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Subject: help wanted \\r\\n\\r\\nwe are a = year old f\",\n        \"ortune = company that is growing at a tremend\",\n        \"ous rate we are looking for individuals who want t\",\n        \"o work from home this is an opportunity to make an\",\n        \" excellent income no experience is required we wil\",\n        \"l train you so if you are looking to be employed f\",\n        \"rom home with a career that has vast opportunities\",\n        \" then go  we are looking for energetic and self\",\n        \" motivated people if that is you than click on the\",\n        \" link and fill out the form and one of our employe\",\n        \"ment specialist will contact you to be removed fro\",\n        \"m our link simple go to  \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Subject: tired of the bull out there\\r\\n\\r\\n want to st\",\n        \"op losing money want a real money maker receive NU\",\n        \"MBER = = = today experts are callin\",\n        \"g this the fastest way to huge cash flow ever conc\",\n        \"eived a powerhouse gifting program you don t want \",\n        \"to miss we work as a team this is your private inv\",\n        \"itation get in with the founders this is where the\",\n        \" big boys play the major players are on this one f\",\n        \"or once be where the players are this is a system \",\n        \"that will drive = = s to your doorstep i\",\n        \"n a short period of time leverage = = in\",\n        \"to = = over and over again the question \",\n        \"here is you either want to be wealthy or you don t\",\n        \" which one are you i am tossing you a financial li\",\n        \"feline and for your sake i hope you grab onto it a\",\n        \"nd hold on tight for the ride of your life testimo\",\n        \"nials hear what average people are doing their fir\",\n        \"st few days we ve received = = in =\",\n        \" day and we are doing that over and over again q s\",\n        \" in al i m a single mother in fl and i ve received\",\n        \" = = in the last = days d s in fl i\",\n        \" was not sure about this when i sent off my =\",\n        \" = pledge but i got back = = the ve\",\n        \"ry next day l l in ky i didn t have the money so i\",\n        \" found myself a partner to work this with we have \",\n        \"received = = over the last = days i\",\n        \" think i made the right decision don t you k c in \",\n        \"fl i pick up = = my first day and i they\",\n        \" gave me free leads and all the training you can t\",\n        \"oo j w in ca this will be the most important call \",\n        \"you make this year free leads training announcing \",\n        \"we will close your sales for you and help you get \",\n        \"a fax blast immediately upon your entry you make t\",\n        \"he money free leads training don t wait call now N\",\n        \"UMBER = = = print and fax to =\",\n        \" = = = or send an email requesting \",\n        \"more information to successleads  please includ\",\n        \"e your name and telephone number receive = NU\",\n        \"MBER free leads just for responding a = NUMBE\",\n        \"R value name___________________________________ ph\",\n        \"one___________________________________ fax________\",\n        \"_____________________________ email_______________\",\n        \"____________________ this message is sent in compl\",\n        \"iance of the new e mail bill per section = pa\",\n        \"ragraph a = c of s = further transmissio\",\n        \"ns by the sender of this email may be stopped at n\",\n        \"o cost to you by sending a reply to this email add\",\n        \"ress with the word remove in the subject line erro\",\n        \"rs omissions and exceptions excluded this is not s\",\n        \"pam i have compiled this list from our replicate d\",\n        \"atabase relative to seattle marketing group the gi\",\n        \"gt or turbo team for the sole purpose of these com\",\n        \"munications your continued inclusion is only by yo\",\n        \"ur gracious permission if you wish to not receive \",\n        \"this mail from me please send an email to tesrewin\",\n        \"ter  with remove in the subject and you will be\",\n        \" deleted immediately\\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Subject: cellular phone accessories \\r\\n\\r\\n all at bel\",\n        \"ow wholesale prices http = = = NUMB\",\n        \"ER = sites merchant sales hands free ear buds\",\n        \" = = phone holsters = = booste\",\n        \"r antennas only = = phone cases = N\",\n        \"UMBER car chargers = = face plates as lo\",\n        \"w as = = lithium ion batteries as low as\",\n        \" = = http = = = = NU\",\n        \"MBER sites merchant sales click below for accessor\",\n        \"ies on all nokia motorola lg nextel samsung qualco\",\n        \"mm ericsson audiovox phones at below wholesale pri\",\n        \"ces http = = = = = sites \",\n        \"merchant sales if you need assistance please call \",\n        \"us = = = to be removed from future \",\n        \"mailings please send your remove request to remove\",\n        \" me now =  thank you and have a super day\\r\\n\",\n        \" \\r\\n\"\n    ),\n    concat!(\n        \"Subject: conferencing made easy\\r\\n\\r\\n only = cen\",\n        \"ts per minute including long distance no setup fee\",\n        \"s no contracts or monthly fees call anytime from a\",\n        \"nywhere to anywhere connects up to = particip\",\n        \"ants simplicity in set up and administration opera\",\n        \"tor help available = = the highest quali\",\n        \"ty service for the lowest rate in the industry fil\",\n        \"l out the form below to find out how you can lower\",\n        \" your phone bill every month required input field \",\n        \"name web address company name state business phone\",\n        \" home phone email address type of business to be r\",\n        \"emoved from our distribution lists please hyperlin\",\n        \"k click here\\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Subject: dear friend\\r\\n\\r\\n i am mrs sese seko widow o\",\n        \"f late president mobutu sese seko of zaire now kno\",\n        \"wn as democratic republic of congo drc i am moved \",\n        \"to write you this letter this was in confidence co\",\n        \"nsidering my presentcircumstance and situation i e\",\n        \"scaped along with my husband and two of our sons g\",\n        \"eorge kongolo and basher out of democratic republi\",\n        \"c of congo drc to abidjan cote d ivoire where my f\",\n        \"amily and i settled while we later moved to settle\",\n        \"d in morroco where my husband later died of cancer\",\n        \" disease however due to this situation we decided \",\n        \"to changed most of my husband s billions of dollar\",\n        \"s deposited in swiss bank and other countries into\",\n        \" other forms of money coded for safe purpose becau\",\n        \"se the new head of state of dr mr laurent kabila h\",\n        \"as made arrangement with the swiss government and \",\n        \"other european countries to freeze all my late hus\",\n        \"band s treasures deposited in some european countr\",\n        \"ies hence my children and i decided laying low in \",\n        \"africa to study the situation till when things get\",\n        \"s better like now that president kabila is dead an\",\n        \"d the son taking over joseph kabila one of my late\",\n        \" husband s chateaux in southern france was confisc\",\n        \"ated by the french government and as such i had to\",\n        \" change my identity so that my investment will not\",\n        \" be traced and confiscated i have deposited the su\",\n        \"m eighteen million united state dollars us = \",\n        \"= = = with a security company for s\",\n        \"afekeeping the funds are security coded to prevent\",\n        \" them from knowing the content what i want you to \",\n        \"do is to indicate your interest that you will assi\",\n        \"st us by receiving the money on our behalf acknowl\",\n        \"edge this message so that i can introduce you to m\",\n        \"y son kongolo who has the out modalities for the c\",\n        \"laim of the said funds i want you to assist in inv\",\n        \"esting this money but i will not want my identity \",\n        \"revealed i will also want to buy properties and st\",\n        \"ock in multi national companies and to engage in o\",\n        \"ther safe and non speculative investments may i at\",\n        \" this point emphasise the high level of confidenti\",\n        \"ality which this business demands and hope you wil\",\n        \"l not betray the trust and confidence which i repo\",\n        \"se in you in conclusion if you want to assist us m\",\n        \"y son shall put you in the picture of the business\",\n        \" tell you where the funds are currently being main\",\n        \"tained and also discuss other modalities including\",\n        \" remunerationfor your services for this reason kin\",\n        \"dly furnish us your contact information that is yo\",\n        \"ur personal telephone and fax number for confident\",\n        \"ial  regards mrs m sese seko\\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Subject: lowest rates available for term life insu\",\n        \"rance\\r\\n\\r\\n take a moment and fill out our online for\",\n        \"m to see the low rate you qualify for save up to N\",\n        \"UMBER from regular rates smokers accepted  repr\",\n        \"esenting quality nationwide carriers act now to ea\",\n        \"sily remove your address from the list go to  p\",\n        \"lease allow = = hours for removal\\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Subject: central bank of nigeria foreign remittanc\",\n        \"e \\r\\n\\r\\n dept tinubu square lagos nigeria email smith\",\n        \"_j  =th of august = attn president ce\",\n        \"o strictly private business proposal i am mr johns\",\n        \"on s abu the bills and exchange director at the fo\",\n        \"reignremittance department of the central bank of \",\n        \"nigeria i am writingyou this letter to ask for you\",\n        \"r support and cooperation to carrying thisbusiness\",\n        \" opportunity in my department we discovered abando\",\n        \"ned the sumof us = = = = thirt\",\n        \"y seven million four hundred thousand unitedstates\",\n        \" dollars in an account that belong to one of our f\",\n        \"oreign customers an american late engr john creek \",\n        \"junior an oil merchant with the federal government\",\n        \" of nigeria who died along with his entire family \",\n        \"of a wifeand two children in kenya airbus a= \",\n        \"= flight kq= in november= since we \",\n        \"heard of his death we have been expecting his next\",\n        \" of kin tocome over and put claims for his money a\",\n        \"s the heir because we cannotrelease the fund from \",\n        \"his account unless someone applies for claims asth\",\n        \"e next of kin to the deceased as indicated in our \",\n        \"banking guidelines unfortunately neither their fam\",\n        \"ily member nor distant relative hasappeared to cla\",\n        \"im the said fund upon this discovery i and other o\",\n        \"fficialsin my department have agreed to make busin\",\n        \"ess with you release the totalamount into your acc\",\n        \"ount as the heir of the fund since no one came for\",\n        \"it or discovered either maintained account with ou\",\n        \"r bank other wisethe fund will be returned to the \",\n        \"bank treasury as unclaimed fund we have agreed tha\",\n        \"t our ratio of sharing will be as stated thus NUMB\",\n        \"ER for you as foreign partner and = for us th\",\n        \"e officials in my department upon the successful c\",\n        \"ompletion of this transfer my colleague and i will\",\n        \"come to your country and mind our share it is from\",\n        \" our = we intendto import computer accessorie\",\n        \"s into my country as way of recycling thefund to c\",\n        \"ommence this transaction we require you to immedia\",\n        \"tely indicateyour interest by calling me or sendin\",\n        \"g me a fax immediately on the abovetelefax and enc\",\n        \"lose your private contact telephone fax full namea\",\n        \"nd address and your designated banking co ordinate\",\n        \"s to enable us fileletter of claim to the appropri\",\n        \"ate department for necessary approvalsbefore the t\",\n        \"ransfer can be made note also this transaction mus\",\n        \"t be kept strictly confidential becauseof its natu\",\n        \"re nb please remember to give me your phone and fa\",\n        \"x no mr johnson smith abu irish linux users group \",\n        \"ilug   for un subscription information list \",\n        \"maintainer listmaster \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Subject: dear stuart\\r\\n\\r\\n are you tired of searching\",\n        \" for love in all the wrong places find love now at\",\n        \"   browse through thousands of personals in \",\n        \"your area join for free  search e mail chat use\",\n        \"  to meet cool guys and hot girls go = on \",\n        \"= or use our private chat rooms click on the \",\n        \"link to get started  find love now you have rec\",\n        \"eived this email because you have registerd with e\",\n        \"mailrewardz or subscribed through one of our marke\",\n        \"ting partners if you have received this message in\",\n        \" error or wish to stop receiving these great offer\",\n        \"s please click the remove link above to unsubscrib\",\n        \"e from these mailings please click here \\r\\n\\r\\n\"\n    ),\n];\n\npub const HAM: [&str; 10] = [\n    concat!(\n        \"Message-ID: <mid1@foobar.org>\\r\\nSubject: i have been\",\n        \" trying to research via sa mirrors and search engi\",\n        \"nes\\r\\n\\r\\nif a canned script exists giving clients acce\",\n        \"ss to their user_prefs options via a web based cgi\",\n        \" interface numerous isps provide this feature to c\",\n        \"lients but so far i can find nothing our configura\",\n        \"tion uses amavis postfix and clamav for virus filt\",\n        \"ering and procmail with spamassassin for spam filt\",\n        \"ering i would prefer not to have to write a script\",\n        \" myself but will appreciate any suggestions this U\",\n        \"RL email is sponsored by osdn tired of that same o\",\n        \"ld cell phone get a new here for free  ________\",\n        \"_______________________________________ spamassass\",\n        \"in talk mailing list spamassassin talk  \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: mid2@foobar.org\\r\\nSubject: hello\\r\\n\\r\\nhave y\",\n        \"ou seen and discussed this article and his approac\",\n        \"h thank you  hell there are no rules here we re\",\n        \" trying to accomplish something thomas alva edison\",\n        \" this  email is sponsored by osdn tired of that\",\n        \" same old cell phone get a new here for free  _\",\n        \"______________________________________________ spa\",\n        \"massassin devel mailing list spamassassin devel UR\",\n        \"L  \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid3@foobar.org>\\r\\nSubject: hi all apol\",\n        \"ogies for the possible silly question\\r\\n\\r\\ni don t thi\",\n        \"nk it is but but is eircom s adsl service nat ed a\",\n        \"nd what implications would that have for voip i kn\",\n        \"ow there are difficulties with voip or connecting \",\n        \"to clients connected to a nat ed network from the \",\n        \"internet wild i e machines with static real ips an\",\n        \"y help pointers would be helpful cheers rgrds bern\",\n        \"ard bernard tyers national centre for sensor resea\",\n        \"rch p = = = = e bernard tyers \",\n        \" w  l n= ______________________________\",\n        \"_________________ iiu mailing list iiu   \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid4@foobar.org>\\r\\nSubject: can someone\",\n        \" explain\\r\\n\\r\\nwhat type of operating system solaris is\",\n        \" as ive never seen or used it i dont know wheather\",\n        \" to get a server from sun or from dell i would pre\",\n        \"fer a linux based server and sun seems to be the o\",\n        \"ne for that but im not sure if solaris is a distro\",\n        \" of linux or a completely different operating syst\",\n        \"em can someone explain kiall mac innes irish linux\",\n        \" users group ilug   for un subscription info\",\n        \"rmation list maintainer listmaster  \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid5@foobar.org>\\r\\nSubject: folks my fi\",\n        \"rst time posting\\r\\n\\r\\nhave a bit of unix experience bu\",\n        \"t am new to linux just got a new pc at home dell b\",\n        \"ox with windows xp added a second hard disk for li\",\n        \"nux partitioned the disk and have installed suse N\",\n        \"UMBER = from cd which went fine except it did\",\n        \"n t pick up my monitor i have a dell branded eNUMB\",\n        \"ERfpp = lcd flat panel monitor and a nvidia g\",\n        \"eforce= ti= video card both of which are\",\n        \" probably too new to feature in suse s default set\",\n        \" i downloaded a driver from the nvidia website and\",\n        \" installed it using rpm then i ran sax= as wa\",\n        \"s recommended in some postings i found on the net \",\n        \"but it still doesn t feature my video card in the \",\n        \"available list what next another problem i have a \",\n        \"dell branded keyboard and if i hit caps lock twice\",\n        \" the whole machine crashes in linux not windows ev\",\n        \"en the on off switch is inactive leaving me to rea\",\n        \"ch for the power cable instead if anyone can help \",\n        \"me in any way with these probs i d be really grate\",\n        \"ful i ve searched the net but have run out of idea\",\n        \"s or should i be going for a different version of \",\n        \"linux such as redhat opinions welcome thanks a lot\",\n        \" peter irish linux users group ilug   for un\",\n        \" subscription information list maintainer listmast\",\n        \"er \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid6@foobar.org>\\r\\nSubject: has anyone\\r\\n\",\n        \"\\r\\nseen heard of used some package that would let a \",\n        \"random person go to a webpage create a mailing lis\",\n        \"t then administer that list also of course let ppl\",\n        \" sign up for the lists and manage their subscripti\",\n        \"ons similar to the old  but i d like to have it\",\n        \" running on my server not someone elses chris  \",\n        \"\\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid7@foobar.org>\\r\\nSubject: hi thank yo\",\n        \"u for the useful replies\\r\\n\\r\\ni have found some intere\",\n        \"sting tutorials in the ibm developer connection UR\",\n        \"L and  registration is needed i will post the s\",\n        \"ame message on the web application security list a\",\n        \"s suggested by someone for now i thing i will use \",\n        \"md= for password checking i will use the appr\",\n        \"oach described in secure programmin fo linux and u\",\n        \"nix how to i will separate the authentication modu\",\n        \"le so i can change its implementation at anytime t\",\n        \"hank you again mario torre please avoid sending me\",\n        \" word or powerpoint attachments see  \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid8@foobar.org>\\r\\nSubject: hehe sorry\\r\\n\",\n        \"\\r\\nbut if you hit caps lock twice the computer crash\",\n        \"es theres one ive never heard before have you trye\",\n        \"d dell support yet i think dell computers prefer r\",\n        \"edhat dell provide some computers pre loaded with \",\n        \"red hat i dont know for sure tho so get someone el\",\n        \"ses opnion as well as mine original message from i\",\n        \"lug admin  mailto ilug admin  on behalf of p\",\n        \"eter staunton sent = august = = NUM\",\n        \"BER to ilug  subject ilug newbie seeks advice s\",\n        \"use = = folks my first time posting have\",\n        \" a bit of unix experience but am new to linux just\",\n        \" got a new pc at home dell box with windows xp add\",\n        \"ed a second hard disk for linux partitioned the di\",\n        \"sk and have installed suse = = from cd w\",\n        \"hich went fine except it didn t pick up my monitor\",\n        \" i have a dell branded e=fpp = lcd flat \",\n        \"panel monitor and a nvidia geforce= ti= \",\n        \"video card both of which are probably too new to f\",\n        \"eature in suse s default set i downloaded a driver\",\n        \" from the nvidia website and installed it using rp\",\n        \"m then i ran sax= as was recommended in some \",\n        \"postings i found on the net but it still doesn t f\",\n        \"eature my video card in the available list what ne\",\n        \"xt another problem i have a dell branded keyboard \",\n        \"and if i hit caps lock twice the whole machine cra\",\n        \"shes in linux not windows even the on off switch i\",\n        \"s inactive leaving me to reach for the power cable\",\n        \" instead if anyone can help me in any way with the\",\n        \"se probs i d be really grateful i ve searched the \",\n        \"net but have run out of ideas or should i be going\",\n        \" for a different version of linux such as redhat o\",\n        \"pinions welcome thanks a lot peter irish linux use\",\n        \"rs group ilug   for un subscription informat\",\n        \"ion list maintainer listmaster  irish linux use\",\n        \"rs group ilug   for un subscription informat\",\n        \"ion list maintainer listmaster \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid9@foobar.org>\\r\\nSubject: it will fun\",\n        \"ction as a router\\r\\n\\r\\nif that is what you wish it eve\",\n        \"n looks like the modem s embedded os is some kind \",\n        \"of linux being that it has interesting interfaces \",\n        \"like eth= i don t use it as a router though i\",\n        \" just have it do the absolute minimum dsl stuff an\",\n        \"d do all the really fun stuff like pppoe on my lin\",\n        \"ux box also the manual tells you what the default \",\n        \"password is don t forget to run pppoe over the alc\",\n        \"atel speedtouch =i as in my case you have to \",\n        \"have a bridge configured in the router modem s sof\",\n        \"tware this lists your vci values etc also does any\",\n        \"one know if the high end speedtouch with = et\",\n        \"hernet ports can act as a full router or do i stil\",\n        \"l need to run a pppoe stack on the linux box regar\",\n        \"ds vin irish linux users group ilug   for un\",\n        \" subscription information list maintainer listmast\",\n        \"er  irish linux users group ilug   for un\",\n        \" subscription information list maintainer listmast\",\n        \"er  \\r\\n\\r\\n\"\n    ),\n    concat!(\n        \"Message-ID: <mid10@foobar.org>\\r\\nSubject: all is it \",\n        \"just me\\r\\n\\r\\nor has there been a massive increase in t\",\n        \"he amount of email being falsely bounced around th\",\n        \"e place i ve already received email from a number \",\n        \"of people i don t know asking why i am sending the\",\n        \"m email these can be explained by servers from rus\",\n        \"sia and elsewhere coupled with the false emails i \",\n        \"received myself it s really starting to annoy me a\",\n        \"m i the only one seeing an increase in recent week\",\n        \"s martin martin whelan déise design  tel NUMBE\",\n        \"R = our core product déiseditor allows organ\",\n        \"isations to publish information to their web site \",\n        \"in a fast and cost effective manner there is no ne\",\n        \"ed for a full time web developer as the site can b\",\n        \"e easily updated by the organisations own staff in\",\n        \"stant updates to keep site information fresh sites\",\n        \" which are updated regularly bring users back visi\",\n        \"t  for a demonstration déiseditor managing you\",\n        \"r information ____________________________________\",\n        \"___________ iiu mailing list iiu   ,0\\r\\n\"\n    ),\n];\n\nconst TEST: [&str; 3] = [\n    concat!(\n        \"Subject: save up to = on life insurance\\r\\n\\r\\nwhy \",\n        \"spend more than you have to life quote savings ens\",\n        \"uring your family s financial security is very imp\",\n        \"ortant life quote savings makes buying life insura\",\n        \"nce simple and affordable we provide free access t\",\n        \"o the very best companies and the lowest rates lif\",\n        \"e quote savings is fast easy and saves you money l\",\n        \"et us help you get started with the best values in\",\n        \" the country on new coverage you can save hundreds\",\n        \" or even thousands of dollars by requesting a free\",\n        \" quote from lifequote savings our service will tak\",\n        \"e you less than = minutes to complete shop an\",\n        \"d compare save up to = on all types of life i\",\n        \"nsurance hyperlink click here for your free quote \",\n        \"protecting your family is the best investment you \",\n        \"ll ever make if you are in receipt of this email i\",\n        \"n error and or wish to be removed from our list hy\",\n        \"perlink please click here and type remove if you r\",\n        \"eside in any state which prohibits e mail solicita\",\n        \"tions for insurance please disregard this email\\r\\n\"\n    ),\n    concat!(\n        \"Subject: can someone explain\\r\\n\\r\\nwhat type of operati\",\n        \"ng system solaris is as ive never seen or used it \",\n        \"i dont know wheather to get a server from sun or f\",\n        \"rom dell i would prefer a linux based server and s\",\n        \"un seems to be the one for that but im not sure if\",\n        \" solaris is a distro of linux or a completely diff\",\n        \"erent operating system can someone explain kiall m\",\n        \"ac innes irish linux users group ilug   for \",\n        \"un subscription information list maintainer listma\",\n        \"ster  \\r\\n\"\n    ),\n    concat!(\n        \"Subject: classifier test\\r\\n\\r\\nthis is a novel text tha\",\n        \"t the sgd classifier has never seen before, it s\",\n        \"hould be classified as ham or non-ham\\r\\n\"\n    ),\n];\n"
  },
  {
    "path": "tests/src/imap/append.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fs, io};\n\nuse imap_proto::ResponseType;\n\nuse crate::jmap::wait_for_index;\n\nuse super::{AssertResult, IMAPTest, ImapConnection, Type, resources_dir};\n\npub async fn test(imap: &mut ImapConnection, _imap_check: &mut ImapConnection, handle: &IMAPTest) {\n    println!(\"Running APPEND tests...\");\n\n    // Invalid APPEND commands\n    imap.send(\"APPEND \\\"Does not exist\\\" {1+}\\r\\na\").await;\n    imap.assert_read(Type::Tagged, ResponseType::No)\n        .await\n        .assert_response_code(\"TRYCREATE\");\n\n    // Import test messages\n    let mut entries = fs::read_dir(resources_dir())\n        .unwrap()\n        .map(|res| res.map(|e| e.path()))\n        .collect::<Result<Vec<_>, io::Error>>()\n        .unwrap();\n\n    entries.sort();\n\n    let mut expected_uid = 1;\n    for file_name in entries.into_iter().take(20) {\n        if file_name.extension().is_none_or(|e| e != \"txt\") {\n            continue;\n        }\n        let raw_message = fs::read(&file_name).unwrap();\n\n        imap.send(&format!(\n            \"APPEND INBOX (Flag_{}) {{{}}}\",\n            file_name\n                .file_name()\n                .unwrap()\n                .to_str()\n                .unwrap()\n                .split_once('.')\n                .unwrap()\n                .0,\n            raw_message.len()\n        ))\n        .await;\n        imap.assert_read(Type::Continuation, ResponseType::Ok).await;\n        imap.send_untagged(std::str::from_utf8(&raw_message).unwrap())\n            .await;\n        let result = imap\n            .assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_response_code();\n        let mut code = result.split(' ');\n        assert_eq!(code.next(), Some(\"APPENDUID\"));\n        assert_ne!(code.next(), Some(\"0\"));\n        assert_eq!(code.next(), Some(expected_uid.to_string().as_str()));\n        expected_uid += 1;\n    }\n\n    wait_for_index(&handle.server).await;\n}\n\npub async fn assert_append_message(\n    imap: &mut ImapConnection,\n    folder: &str,\n    message: &str,\n    expected_response: ResponseType,\n) -> Vec<String> {\n    imap.send(&format!(\"APPEND \\\"{}\\\" {{{}}}\", folder, message.len()))\n        .await;\n    imap.assert_read(Type::Continuation, ResponseType::Ok).await;\n    imap.send_untagged(message).await;\n    imap.assert_read(Type::Tagged, expected_response).await\n}\n\nfn build_message(message: usize, in_reply_to: Option<usize>, thread_num: usize) -> String {\n    if let Some(in_reply_to) = in_reply_to {\n        format!(\n            \"Message-ID: <{}@domain>\\nReferences: <{}@domain>\\nSubject: re: T{}\\n\\nreply\\n\",\n            message, in_reply_to, thread_num\n        )\n    } else {\n        format!(\n            \"Message-ID: <{}@domain>\\nSubject: T{}\\n\\nmsg\\n\",\n            message, thread_num\n        )\n    }\n}\n\npub fn build_messages() -> Vec<String> {\n    let mut messages = Vec::new();\n    for parent in 0..3 {\n        messages.push(build_message(parent, None, parent));\n        for child in 0..3 {\n            messages.push(build_message(\n                ((parent + 1) * 10) + child,\n                parent.into(),\n                parent,\n            ));\n        }\n    }\n    messages\n}\n"
  },
  {
    "path": "tests/src/imap/basic.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::auth::sasl::sasl_decode_challenge_oauth;\nuse imap_proto::ResponseType;\nuse mail_parser::decoders::base64::base64_decode;\nuse mail_send::Credentials;\n\nuse super::{AssertResult, ImapConnection, Type};\n\npub async fn test(imap: &mut ImapConnection, _imap_check: &mut ImapConnection) {\n    println!(\"Running basic tests...\");\n\n    // Test OAuth Bearer decoding\n    assert!(\n        Credentials::OAuthBearer {\n            token: \"vF9dft4qmTc2Nvb3RlckBhbHRhdmlzdGEuY29tCg==\".to_string()\n        } == sasl_decode_challenge_oauth(\n            &base64_decode(\n                concat!(\n                    \"bixhPXVzZXJAZXhhbXBsZS5jb20sAWhv\",\n                    \"c3Q9c2VydmVyLmV4YW1wbGUuY29tAXBvcnQ9MTQzAWF1dGg9QmVhcmVyI\",\n                    \"HZGOWRmdDRxbVRjMk52YjNSbGNrQmhiSFJoZG1semRHRXVZMjl0Q2c9PQ\",\n                    \"EB\"\n                )\n                .as_bytes(),\n            )\n            .unwrap(),\n        )\n        .unwrap()\n    );\n\n    // Test CAPABILITY\n    imap.send(\"CAPABILITY\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Test NOOP\n    imap.send(\"NOOP\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Test ID\n    imap.send(\"ID\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* ID (\\\"name\\\" \\\"Stalwart\\\" \\\"version\\\" \");\n\n    // Login should be disabled\n    imap.send(\"LOGIN jdoe@example.com secret\").await;\n    imap.assert_read(Type::Tagged, ResponseType::No).await;\n\n    // Try logging in with wrong password\n    imap.send(\"AUTHENTICATE PLAIN {24}\").await;\n    imap.assert_read(Type::Continuation, ResponseType::Ok).await;\n    imap.send_untagged(\"AGJvYXR5AG1jYm9hdGZhY2U=\").await;\n    imap.assert_read(Type::Tagged, ResponseType::No).await;\n}\n"
  },
  {
    "path": "tests/src/imap/body_structure.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::resources_dir;\nuse email::message::metadata::{MessageMetadata, build_metadata_contents};\nuse imap::op::fetch::AsImapDataItem;\nuse imap_proto::{\n    ResponseCode, StatusResponse,\n    protocol::fetch::{BodyContents, DataItem, Section},\n};\nuse mail_parser::MessageParser;\nuse std::fs;\nuse store::{\n    Deserialize, Serialize,\n    write::{Archive, Archiver},\n};\nuse utils::chained_bytes::ChainedBytes;\n\npub fn test() {\n    println!(\"Running BODYSTRUCTURE...\");\n\n    for file_name in fs::read_dir(resources_dir()).unwrap() {\n        let mut file_name = file_name.as_ref().unwrap().path();\n        if file_name.extension().is_none_or(|e| e != \"txt\") {\n            continue;\n        }\n\n        let mut buf = Vec::new();\n        let raw_message = fs::read(&file_name).unwrap();\n        let message_ = MessageParser::new().parse(&raw_message).unwrap();\n        let metadata = MessageMetadata {\n            preview: Default::default(),\n            raw_headers: message_\n                .raw_message\n                .as_ref()\n                .get(\n                    message_.root_part().offset_header as usize\n                        ..message_.root_part().offset_body as usize,\n                )\n                .unwrap_or_default()\n                .into(),\n            blob_hash: Default::default(),\n            blob_body_offset: message_.root_part().offset_body as u32,\n            contents: build_metadata_contents(message_),\n            rcvd_attach: 0,\n        };\n        let metadata_ =\n            Archive::deserialize_owned(Archiver::new(metadata).serialize().unwrap()).unwrap();\n        let metadata = metadata_.unarchive::<MessageMetadata>().unwrap();\n        let raw_message = ChainedBytes::new(metadata.raw_headers.as_ref()).with_last(\n            raw_message\n                .get(metadata.blob_body_offset.to_native() as usize..)\n                .unwrap_or_default(),\n        );\n        let decoded = metadata.decode_contents(raw_message);\n\n        // Serialize body and bodystructure\n        for is_extended in [false, true] {\n            let mut buf_ = Vec::new();\n            metadata\n                .body_structure(&decoded, is_extended)\n                .serialize(&mut buf_, is_extended);\n            if is_extended {\n                buf.extend_from_slice(b\"BODYSTRUCTURE \");\n            } else {\n                buf.extend_from_slice(b\"BODY \");\n            }\n\n            // Poor man's indentation\n            let mut indent_count = 0;\n            let mut in_quote = false;\n            for ch in buf_ {\n                if ch == b'(' && !in_quote {\n                    buf.extend_from_slice(b\"(\\n\");\n                    indent_count += 1;\n                    for _ in 0..indent_count {\n                        buf.extend_from_slice(b\"   \");\n                    }\n                } else if ch == b')' && !in_quote {\n                    buf.push(b'\\n');\n                    indent_count -= 1;\n                    for _ in 0..indent_count {\n                        buf.extend_from_slice(b\"   \");\n                    }\n                    buf.push(b')');\n                } else {\n                    if ch == b'\"' {\n                        in_quote = !in_quote;\n                    }\n                    buf.push(ch);\n                }\n            }\n            buf.extend_from_slice(b\"\\n\\n\");\n        }\n\n        // Serialize body parts\n        let mut iter = 1..9;\n        let mut stack = Vec::new();\n        let mut sections = Vec::new();\n        loop {\n            'inner: while let Some(part_id) = iter.next() {\n                if part_id == 1 {\n                    for section in [\n                        None,\n                        Some(Section::Header),\n                        Some(Section::Text),\n                        Some(Section::Mime),\n                    ] {\n                        let mut body_sections = sections\n                            .iter()\n                            .map(|id| Section::Part { num: *id })\n                            .collect::<Vec<_>>();\n                        let is_first = if let Some(section) = section {\n                            body_sections.push(section);\n                            false\n                        } else {\n                            true\n                        };\n\n                        if let Some(contents) =\n                            metadata.body_section(&decoded, &body_sections, None)\n                        {\n                            DataItem::BodySection {\n                                sections: body_sections,\n                                origin_octet: None,\n                                contents,\n                            }\n                            .serialize(&mut buf);\n\n                            if is_first {\n                                match metadata.binary(&decoded, &sections, None) {\n                                    Ok(Some(contents)) => {\n                                        buf.push(b'\\n');\n                                        DataItem::Binary {\n                                            sections: sections.clone(),\n                                            offset: None,\n                                            contents: match contents {\n                                                BodyContents::Bytes(_) => {\n                                                    BodyContents::Text(\"[binary content]\".into())\n                                                }\n                                                text => text,\n                                            },\n                                        }\n                                        .serialize(&mut buf);\n                                    }\n                                    Ok(None) => (),\n                                    Err(_) => {\n                                        buf.push(b'\\n');\n                                        buf.extend_from_slice(\n                                            &StatusResponse::no(format!(\n                                                \"Failed to decode part {} of message {}.\",\n                                                sections\n                                                    .iter()\n                                                    .map(|s| s.to_string())\n                                                    .collect::<Vec<_>>()\n                                                    .join(\".\"),\n                                                0\n                                            ))\n                                            .with_code(ResponseCode::UnknownCte)\n                                            .serialize(Vec::new()),\n                                        );\n                                    }\n                                }\n\n                                if let Some(size) = metadata.binary_size(&decoded, &sections) {\n                                    buf.push(b'\\n');\n                                    DataItem::BinarySize {\n                                        sections: sections.clone(),\n                                        size,\n                                    }\n                                    .serialize(&mut buf);\n                                }\n                            }\n\n                            buf.extend_from_slice(b\"\\n----------------------------------\\n\");\n                        } else {\n                            break 'inner;\n                        }\n                    }\n                }\n                sections.push(part_id);\n                stack.push(iter);\n                iter = 1..9;\n            }\n            if let Some(prev_iter) = stack.pop() {\n                sections.pop();\n                iter = prev_iter;\n            } else {\n                break;\n            }\n        }\n\n        // Check header fields and partial sections\n        for sections in [\n            vec![Section::HeaderFields {\n                not: false,\n                fields: vec![\"From\".into(), \"To\".into()],\n            }],\n            vec![Section::HeaderFields {\n                not: true,\n                fields: vec![\"Subject\".into(), \"Cc\".into()],\n            }],\n        ] {\n            DataItem::BodySection {\n                contents: metadata.body_section(&decoded, &sections, None).unwrap(),\n                sections: sections.clone(),\n                origin_octet: None,\n            }\n            .serialize(&mut buf);\n            buf.extend_from_slice(b\"\\n----------------------------------\\n\");\n            DataItem::BodySection {\n                contents: metadata\n                    .body_section(&decoded, &sections, (10, 25).into())\n                    .unwrap(),\n                sections,\n                origin_octet: 10.into(),\n            }\n            .serialize(&mut buf);\n            buf.extend_from_slice(b\"\\n----------------------------------\\n\");\n        }\n\n        file_name.set_extension(\"imap\");\n\n        let expected_result = fs::read(&file_name).unwrap();\n\n        if buf != expected_result {\n            file_name.set_extension(\"imap_failed\");\n            fs::write(&file_name, buf).unwrap();\n            panic!(\"Failed test, written output to {}\", file_name.display());\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/imap/condstore.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap_proto::ResponseType;\n\nuse crate::imap::{\n    AssertResult,\n    append::{assert_append_message, build_messages},\n};\n\nuse super::{ImapConnection, Type};\n\npub async fn test(imap: &mut ImapConnection, imap_check: &mut ImapConnection) {\n    println!(\"Running CONDSTORE...\");\n\n    // Test CONDSTORE parameter\n    imap.send(\"SELECT INBOX (CONDSTORE)\").await;\n    let hms = imap\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .into_highest_modseq();\n\n    // Unselect\n    imap.send(\"UNSELECT\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Create test folders\n    imap.send(\"CREATE Pecorino\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Enable CONDSTORE and QRESYNC\n    imap.send(\"ENABLE CONDSTORE QRESYNC\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Make sure modseq did not change after creating a mailbox\n    imap.send(\"SELECT Pecorino\").await;\n    assert_eq!(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n        hms\n    );\n    imap_check.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_check.send(\"SELECT Pecorino (CONDSTORE)\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // SEQ 0: Init\n    let mut messages = build_messages();\n    let mut modseqs = vec![hms];\n\n    // SEQ 1: Append a message and make sure the modseq increased\n    assert_append_message(imap, \"Pecorino\", &messages.pop().unwrap(), ResponseType::Ok).await;\n    imap.send(\"STATUS Pecorino (HIGHESTMODSEQ)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n    );\n    assert_ne!(modseqs[modseqs.len() - 1], modseqs[modseqs.len() - 2]);\n\n    // SEQ 2: Move out the message and make sure the modseq increased\n    imap.send(\"UID MOVE 1 \\\"Deleted Items\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* VANISHED 1\");\n    imap.send(\"STATUS Pecorino (HIGHESTMODSEQ)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n    );\n    assert_ne!(modseqs[modseqs.len() - 1], modseqs[modseqs.len() - 2]);\n\n    // SEQ 3: Insert message\n    assert_append_message(imap, \"Pecorino\", &messages.pop().unwrap(), ResponseType::Ok).await;\n    imap.send(\"STATUS Pecorino (HIGHESTMODSEQ)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n    );\n\n    // SEQ 4: Insert message\n    assert_append_message(imap, \"Pecorino\", &messages.pop().unwrap(), ResponseType::Ok).await;\n    imap.send(\"STATUS Pecorino (HIGHESTMODSEQ)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n    );\n\n    // SEQ 5: Insert message\n    assert_append_message(imap, \"Pecorino\", &messages.pop().unwrap(), ResponseType::Ok).await;\n    imap.send(\"STATUS Pecorino (HIGHESTMODSEQ)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n    );\n\n    // SEQ 6: Change a message flag\n    imap.send(\"UID STORE 4 +FLAGS.SILENT (\\\\Answered)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_modseq(),\n    );\n\n    // SEQ 7: Insert message\n    assert_append_message(imap, \"Pecorino\", &messages.pop().unwrap(), ResponseType::Ok).await;\n    imap.send(\"STATUS Pecorino (HIGHESTMODSEQ)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n    );\n\n    // SEQ 8: Delete a message\n    imap.send(\"UID STORE 2 +FLAGS.SILENT (\\\\Deleted)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"EXPUNGE\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"VANISHED 2\")\n        .assert_contains(\"* 3 EXISTS\");\n    imap.send(\"STATUS Pecorino (HIGHESTMODSEQ)\").await;\n    modseqs.push(\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .into_highest_modseq(),\n    );\n\n    // Fetch changes since SEQ 0\n    imap.send(&format!(\n        \"UID FETCH 1:* (FLAGS) (CHANGEDSINCE {} VANISHED)\",\n        modseqs[0]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"FETCH (\", 3)\n        .assert_count(\"VANISHED\", 0);\n\n    // Fetch changes since SEQ 1, UID MOVE should count as a deletion\n    imap.send(&format!(\n        \"UID FETCH 1:* (FLAGS) (CHANGEDSINCE {} VANISHED)\",\n        modseqs[1]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"VANISHED\", 1)\n        .assert_contains(\"VANISHED (EARLIER) 1\")\n        .assert_count(\"FETCH (\", 3);\n\n    // Fetch changes since SEQ 3\n    imap.send(&format!(\n        \"UID FETCH 1:* (FLAGS) (CHANGEDSINCE {} VANISHED)\",\n        modseqs[3]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"VANISHED\", 1)\n        .assert_contains(\"VANISHED (EARLIER) 2\")\n        .assert_count(\"FETCH (\", 3);\n\n    // Fetch changes since SEQ 4\n    imap.send(&format!(\n        \"UID FETCH 1:* (FLAGS) (CHANGEDSINCE {} VANISHED)\",\n        modseqs[4]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"VANISHED\", 1)\n        .assert_contains(\"VANISHED (EARLIER) 2\")\n        .assert_count(\"FETCH (\", 2);\n\n    // Fetch changes since SEQ 6\n    imap.send(&format!(\n        \"UID FETCH 1:* (FLAGS) (CHANGEDSINCE {} VANISHED)\",\n        modseqs[6]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"VANISHED\", 1)\n        .assert_contains(\"VANISHED (EARLIER) 2\")\n        .assert_count(\"FETCH (\", 1);\n\n    // Fetch changes since SEQ 7\n    imap.send(&format!(\n        \"UID FETCH 1:* (FLAGS) (CHANGEDSINCE {} VANISHED)\",\n        modseqs[7]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"VANISHED\", 1)\n        .assert_contains(\"VANISHED (EARLIER) 2\")\n        .assert_count(\"FETCH (\", 0);\n\n    // Fetch changes since SEQ 8\n    imap.send(&format!(\n        \"UID FETCH 1:* (FLAGS) (CHANGEDSINCE {} VANISHED)\",\n        modseqs[8]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"VANISHED\", 0)\n        .assert_count(\"FETCH (\", 0);\n\n    // Search since MODSEQ\n    imap.send(&format!(\"SEARCH RETURN (ALL) MODSEQ {}\", modseqs[3]))\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"ALL 1:3 MODSEQ\");\n\n    imap_check.send(\"NOOP\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"3 EXISTS\");\n    imap_check\n        .send(&format!(\"SEARCH MODSEQ {}\", modseqs[4]))\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"SEARCH 2 3 (MODSEQ\");\n\n    // Store unchanged since\n    imap.send(&format!(\n        \"UID STORE 2:5 (UNCHANGEDSINCE {}) +FLAGS.SILENT (\\\\Junk)\",\n        modseqs[5]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 FETCH\")\n        .assert_contains(\"(UID 3 MODSEQ\")\n        .assert_count(\"FETCH (\", 1)\n        .assert_contains(\"[MODIFIED 2,4:5]\");\n\n    imap.send(&format!(\n        \"UID STORE 4,5 (UNCHANGEDSINCE {}) -FLAGS.SILENT (\\\\Answered)\",\n        modseqs[6]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 2 FETCH\")\n        .assert_contains(\"(UID 4 MODSEQ\")\n        .assert_count(\"FETCH (\", 1)\n        .assert_contains(\"[MODIFIED 5]\");\n\n    // QResync\n    imap.send(\"STATUS Pecorino (UIDVALIDITY)\").await;\n    let uid_validity = imap\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .into_uid_validity();\n\n    imap.send(&format!(\n        \"SELECT Pecorino (QRESYNC ({} {} 1:5)) \",\n        uid_validity, modseqs[6]\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"FETCH (\", 3)\n        .assert_contains(\"VANISHED (EARLIER) 2\");\n}\n"
  },
  {
    "path": "tests/src/imap/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap_proto::ResponseType;\n\nuse super::{AssertResult, ImapConnection, Type};\n\npub async fn test(_imap: &mut ImapConnection, imap_check: &mut ImapConnection) {\n    println!(\"Running COPY/MOVE tests...\");\n\n    // Check status\n    imap_check\n        .send(\"LIST \\\"\\\" % RETURN (STATUS (UIDNEXT MESSAGES UNSEEN SIZE RECENT))\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"INBOX\\\" (UIDNEXT 11 MESSAGES 10 UNSEEN 10 RECENT 0 SIZE 12193)\");\n\n    // Select INBOX\n    imap_check.send(\"SELECT INBOX\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Copying to the same mailbox should fail\n    imap_check.send(\"COPY 1:* INBOX\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::No)\n        .await\n        .assert_response_code(\"CANNOT\");\n\n    // Copying to a non-existent mailbox should fail\n    imap_check.send(\"COPY 1:* \\\"/dev/null\\\"\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::No)\n        .await\n        .assert_response_code(\"TRYCREATE\");\n\n    // Create test folders\n    imap_check.send(\"CREATE \\\"Scamorza Affumicata\\\"\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_check.send(\"CREATE \\\"Burrata al Tartufo\\\"\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Copy messages\n    imap_check\n        .send(\"COPY 1,3,5,7 \\\"Scamorza Affumicata\\\"\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"COPYUID\")\n        .assert_contains(\"1:4\");\n\n    // Check status\n    imap_check\n        .send(\"STATUS \\\"Scamorza Affumicata\\\" (UIDNEXT MESSAGES UNSEEN SIZE RECENT)\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 4\")\n        //.assert_contains(\"RECENT 4\")\n        .assert_contains(\"UNSEEN 4\")\n        .assert_contains(\"UIDNEXT 5\")\n        .assert_contains(\"SIZE 5851\");\n\n    // Check \\Recent flag\n    /*imap_check.send(\"SELECT \\\"Scamorza Affumicata\\\"\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 4 RECENT\");\n    imap_check.send(\"FETCH 1:* (UID FLAGS)\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"\\\\Recent\", 4);\n    imap_check.send(\"UNSELECT\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_check\n        .send(\"STATUS \\\"Scamorza Affumicata\\\" (UIDNEXT MESSAGES UNSEEN SIZE RECENT)\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 4\")\n        .assert_contains(\"RECENT 0\")\n        .assert_contains(\"UNSEEN 4\")\n        .assert_contains(\"UIDNEXT 5\")\n        .assert_contains(\"SIZE 5851\");\n    imap_check.send(\"SELECT \\\"Scamorza Affumicata\\\"\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 0 RECENT\");\n    imap_check.send(\"FETCH 1:* (UID FLAGS)\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"\\\\Recent\", 0);*/\n\n    // Move all messages to Burrata\n    imap_check.send(\"SELECT \\\"Scamorza Affumicata\\\"\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_check.send(\"MOVE 1:* \\\"Burrata al Tartufo\\\"\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* OK [COPYUID\")\n        .assert_contains(\"1:4\")\n        .assert_contains(\"* 1 EXPUNGE\")\n        .assert_contains(\"* 1 EXPUNGE\")\n        .assert_contains(\"* 1 EXPUNGE\")\n        .assert_contains(\"* 1 EXPUNGE\");\n\n    // Check status\n    imap_check\n        .send(\"LIST \\\"\\\" % RETURN (STATUS (UIDNEXT MESSAGES UNSEEN SIZE))\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"Burrata al Tartufo\\\" (UIDNEXT 5 MESSAGES 4 UNSEEN 4 SIZE 5851)\")\n        .assert_contains(\"\\\"Scamorza Affumicata\\\" (UIDNEXT 5 MESSAGES 0 UNSEEN 0 SIZE 0)\")\n        .assert_contains(\"\\\"INBOX\\\" (UIDNEXT 11 MESSAGES 10 UNSEEN 10 SIZE 12193)\");\n\n    // Move the messages back to Scamorza, UIDNEXT should increase.\n    imap_check.send(\"SELECT \\\"Burrata al Tartufo\\\"\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_check.send(\"MOVE 1:* \\\"Scamorza Affumicata\\\"\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* OK [COPYUID\")\n        .assert_contains(\"5:8\")\n        .assert_contains(\"* 1 EXPUNGE\")\n        .assert_contains(\"* 1 EXPUNGE\")\n        .assert_contains(\"* 1 EXPUNGE\")\n        .assert_contains(\"* 1 EXPUNGE\");\n\n    // Check status\n    imap_check\n        .send(\"LIST \\\"\\\" % RETURN (STATUS (UIDNEXT MESSAGES UNSEEN SIZE))\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"Burrata al Tartufo\\\" (UIDNEXT 5 MESSAGES 0 UNSEEN 0 SIZE 0)\")\n        .assert_contains(\"\\\"Scamorza Affumicata\\\" (UIDNEXT 9 MESSAGES 4 UNSEEN 4 SIZE 5851)\")\n        .assert_contains(\"\\\"INBOX\\\" (UIDNEXT 11 MESSAGES 10 UNSEEN 10 SIZE 12193)\");\n}\n"
  },
  {
    "path": "tests/src/imap/fetch.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap_proto::ResponseType;\n\nuse super::{AssertResult, ImapConnection, Type};\n\npub async fn test(imap: &mut ImapConnection, _imap_check: &mut ImapConnection) {\n    println!(\"Running FETCH tests...\");\n\n    // Examine INBOX\n    imap.send(\"EXAMINE INBOX\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"10 EXISTS\")\n        .assert_contains(\"[UIDNEXT 11]\");\n\n    // Fetch all properties available from JMAP\n    imap.send(concat!(\n        \"FETCH 10 (FLAGS INTERNALDATE PREVIEW EMAILID THREADID \",\n        \"RFC822.SIZE UID ENVELOPE BODYSTRUCTURE)\"\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"FLAGS (Flag_009)\")\n        .assert_contains(\"RFC822.SIZE 1457\")\n        .assert_contains(\"UID 10\")\n        .assert_contains(\"INTERNALDATE\")\n        .assert_contains(\"THREADID (\")\n        .assert_contains(\"EMAILID (\")\n        .assert_contains(\"but then I thought, why not do both?\")\n        .assert_contains(concat!(\n            \"ENVELOPE (\\\"Sat, 20 Nov 2021 14:22:01 -0800\\\" \",\n            \"\\\"Why not both importing AND exporting? ☺\\\" \",\n            \"((\\\"Art Vandelay (Vandelay Industries)\\\" NIL \\\"art\\\" \\\"vandelay.com\\\")) \",\n            \"((\\\"Art Vandelay (Vandelay Industries)\\\" NIL \\\"art\\\" \\\"vandelay.com\\\")) \",\n            \"((\\\"Art Vandelay (Vandelay Industries)\\\" NIL \\\"art\\\" \\\"vandelay.com\\\")) \",\n            \"((NIL NIL \\\"Colleagues\\\" NIL)\",\n            \"(\\\"James Smythe\\\" NIL \\\"james\\\" \\\"vandelay.com\\\")\",\n            \"(NIL NIL NIL NIL)(NIL NIL \\\"Friends\\\" NIL)\",\n            \"(NIL NIL \\\"jane\\\" \\\"example.com\\\")\",\n            \"(\\\"John Smîth\\\" NIL \\\"john\\\" \\\"example.com\\\")\",\n            \"(NIL NIL NIL NIL)) NIL NIL NIL NIL)\"\n        ))\n        .assert_contains(concat!(\n            \"BODYSTRUCTURE ((\\\"text\\\" \\\"html\\\" (\\\"charset\\\" \\\"us-ascii\\\") NIL NIL \",\n            \"\\\"base64\\\" 239 3 \\\"07aab44e51c5f1833a5d19f2e1804c4b\\\" NIL NIL NIL)\",\n            \"(\\\"message\\\" \\\"rfc822\\\" NIL NIL NIL NIL 723 \",\n            \"(NIL \\\"Exporting my book about coffee tables\\\" \",\n            \"((\\\"Cosmo Kramer\\\" NIL \\\"kramer\\\" \\\"kramerica.com\\\")) \",\n            \"((\\\"Cosmo Kramer\\\" NIL \\\"kramer\\\" \\\"kramerica.com\\\")) \",\n            \"((\\\"Cosmo Kramer\\\" NIL \\\"kramer\\\" \\\"kramerica.com\\\")) \",\n            \"NIL NIL NIL NIL NIL) \",\n            \"((\\\"text\\\" \\\"plain\\\" (\\\"charset\\\" \\\"utf-16\\\") NIL NIL \",\n            \"\\\"quoted-printable\\\" 228 3 \\\"3a942a99cdd8a099ae107d3867ec20fb\\\" NIL NIL NIL)\",\n            \"(\\\"image\\\" \\\"gif\\\" (\\\"name\\\" \\\"Book about ☕ tables.gif\\\") \",\n            \"NIL NIL \\\"Base64\\\" 56 \\\"d40fa7f401e9dc2df56cbb740d65ff52\\\" \",\n            \"(\\\"attachment\\\" NIL) NIL NIL) \\\"mixed\\\" (\\\"boundary\\\" \\\"giddyup\\\") NIL NIL NIL)\",\n            \" 0 \\\"cdb0382a03a15601fb1b3c7422521620\\\" NIL NIL NIL) \",\n            \"\\\"mixed\\\" (\\\"boundary\\\" \\\"festivus\\\") NIL NIL NIL)\"\n        ));\n\n    // Fetch bodyparts\n    imap.send(concat!(\n        \"UID FETCH 10 (BINARY[1] BINARY.SIZE[1] BODY[1.TEXT] BODY[2.1.HEADER] \",\n        \"BINARY[2.1] BODY[MIME] BODY[HEADER.FIELDS (From)]<10.8>)\"\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"BINARY[1] {175}\")\n        .assert_contains(\"BINARY.SIZE[1] 175\")\n        .assert_contains(\"BODY[1.TEXT] {239}\")\n        .assert_contains(\"BODY[2.1.HEADER] {88}\")\n        .assert_contains(\"BINARY[2.1] {101}\")\n        .assert_contains(\"BODY[MIME] {54}\")\n        .assert_contains(\"BODY[HEADER.FIELDS (FROM)]<10> {8}\")\n        .assert_contains(\"&ldquo;exporting&rdquo;\")\n        .assert_contains(\"PGh0bWw+PHA+\")\n        .assert_contains(\"Content-Transfer-Encoding: quoted-printable\")\n        .assert_contains(\"ℌ𝔢𝔩𝔭 𝔪𝔢 𝔢𝔵𝔭𝔬𝔯𝔱 𝔪𝔶 𝔟𝔬𝔬𝔨\")\n        .assert_contains(\"Vandelay\");\n\n    // We are in EXAMINE mode, fetching body should not set \\Seen\n    imap.send(\"UID FETCH 10 (FLAGS)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"FLAGS (Flag_009)\");\n\n    // Switch to SELECT mode\n    imap.send(\"SELECT INBOX\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Peek bodyparts\n    imap.send(\"UID FETCH 10 (BINARY.PEEK[1] BINARY.SIZE[1] BODY.PEEK[1.TEXT])\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"BINARY[1] {175}\")\n        .assert_contains(\"BINARY.SIZE[1] 175\")\n        .assert_contains(\"BODY[1.TEXT] {239}\");\n\n    // PEEK was used, \\Seen should not be set\n    imap.send(\"UID FETCH 10 (FLAGS)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"FLAGS (Flag_009)\");\n\n    // Fetching a body section should set the \\Seen flag\n    imap.send(\"UID FETCH 10 (BODY[1.TEXT])\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"FLAGS\")\n        .assert_contains(\"\\\\Seen\");\n\n    // Fetch a sequence\n    imap.send(\"FETCH 1:5,7:10 (UID FLAGS)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 FETCH (UID 1 \")\n        .assert_contains(\"* 2 FETCH (UID 2 \")\n        .assert_contains(\"* 3 FETCH (UID 3 \")\n        .assert_contains(\"* 4 FETCH (UID 4 \")\n        .assert_contains(\"* 5 FETCH (UID 5 \")\n        .assert_contains(\"* 7 FETCH (UID 7 \")\n        .assert_contains(\"* 8 FETCH (UID 8 \")\n        .assert_contains(\"* 9 FETCH (UID 9 \")\n        .assert_contains(\"* 10 FETCH (UID 10 \")\n        .assert_count(\"\\\\Recent\", 0);\n\n    imap.send(\"FETCH 7:* (UID FLAGS)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 7 FETCH (UID 7 \")\n        .assert_contains(\"* 8 FETCH (UID 8 \")\n        .assert_contains(\"* 9 FETCH (UID 9 \")\n        .assert_contains(\"* 10 FETCH (UID 10 \");\n\n    // Fetch using a saved search\n    imap.send(\"UID SEARCH RETURN (SAVE) FROM \\\"nathaniel\\\"\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"FETCH $ (UID PREVIEW)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 FETCH (UID 1 \")\n        .assert_contains(\"* 4 FETCH (UID 4 \")\n        .assert_contains(\"* 6 FETCH (UID 6 \")\n        .assert_contains(\"Some text appears here\")\n        .assert_contains(\"plain text version of message goes here\")\n        .assert_contains(\"This is implicitly typed plain US-ASCII text.\");\n}\n"
  },
  {
    "path": "tests/src/imap/idle.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::mail::delivery::SmtpConnection;\n\nuse super::{AssertResult, ImapConnection, Type};\nuse imap_proto::ResponseType;\nuse std::time::Duration;\n\nconst SLEEP: Duration = Duration::from_millis(200);\n\npub async fn test(\n    imap: &mut ImapConnection,\n    imap_check: &mut ImapConnection,\n    is_cluster_test: bool,\n) {\n    println!(\"Running IDLE tests...\");\n\n    // Switch connection to IDLE mode\n    imap_check.send(\"CREATE Parmeggiano\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_check.send(\"SELECT Parmeggiano\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_check.send(\"NOOP\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_check.send(\"IDLE\").await;\n    imap_check\n        .assert_read(Type::Continuation, ResponseType::Ok)\n        .await;\n\n    // Expect a new mailbox update\n    imap.send(\"CREATE Provolone\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"LIST () \\\"/\\\" \\\"Provolone\\\"\");\n\n    // Insert a message in the new folder and expect an update\n    let message = \"From: test@domain.com\\nSubject: Test\\n\\nTest message\\n\";\n    imap.send(&format!(\"APPEND Provolone {{{}}}\", message.len()))\n        .await;\n    imap.assert_read(Type::Continuation, ResponseType::Ok).await;\n    imap.send_untagged(message).await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"STATUS \\\"Provolone\\\"\")\n        .assert_contains(\"MESSAGES 1\")\n        .assert_contains(\"UNSEEN 1\")\n        .assert_contains(\"UIDNEXT 2\");\n\n    // Change message to Seen and expect an update\n    imap.send(\"SELECT Provolone\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"STORE 1:* +FLAGS (\\\\Seen)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"STATUS \\\"Provolone\\\"\")\n        .assert_contains(\"MESSAGES 1\")\n        .assert_contains(\"UNSEEN 0\")\n        .assert_contains(\"UIDNEXT 2\");\n\n    // Delete message and expect an update\n    imap.send(\"STORE 1:* +FLAGS (\\\\Deleted)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"CLOSE\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"STATUS \\\"Provolone\\\"\")\n        .assert_contains(\"MESSAGES 0\")\n        .assert_contains(\"UNSEEN 0\")\n        .assert_contains(\"UIDNEXT 2\");\n\n    // Delete folder and expect an update\n    imap.send(\"DELETE Provolone\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"LIST (\\\\NonExistent) \\\"/\\\" \\\"Provolone\\\"\");\n\n    // Add a message to Inbox and expect an update\n    imap.send(&format!(\"APPEND Parmeggiano {{{}}}\", message.len()))\n        .await;\n    imap.assert_read(Type::Continuation, ResponseType::Ok).await;\n    imap.send_untagged(message).await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 1\")\n        .assert_contains(\"UNSEEN 1\");\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 EXISTS\");\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 FETCH (FLAGS () UID 1)\");\n\n    // Delete message and expect an update\n    imap.send(\"SELECT Parmeggiano\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap.send(\"STORE 1 +FLAGS (\\\\Deleted)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 FETCH (FLAGS (\\\\Deleted) UID 1)\");\n\n    imap.send(\"UID EXPUNGE\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 EXPUNGE\")\n        .assert_contains(\"* 0 EXISTS\");\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 0\")\n        .assert_contains(\"UNSEEN 0\");\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 1 EXPUNGE\");\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"* 0 EXISTS\");\n\n    // Test SMTP delivery notifications\n    let mut lmtp = SmtpConnection::connect_port(if is_cluster_test { 17000 } else { 11201 }).await;\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"X-Spam-Status: No\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n    if is_cluster_test {\n        tokio::time::sleep(SLEEP).await;\n    }\n    imap_check\n        .assert_read(Type::Status, ResponseType::Ok)\n        .await\n        .assert_contains(\"STATUS \\\"INBOX\\\"\")\n        .assert_contains(if is_cluster_test {\n            \"MESSAGES 1\"\n        } else {\n            \"MESSAGES 11\"\n        });\n\n    // Stop IDLE mode\n    imap_check.send_raw(\"DONE\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_check.send(\"NOOP\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n}\n"
  },
  {
    "path": "tests/src/imap/mailbox.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap::op::list::matches_pattern;\nuse imap_proto::ResponseType;\n\nuse super::{AssertResult, ImapConnection, Type};\n\npub async fn test(mut imap: &mut ImapConnection, mut imap_check: &mut ImapConnection) {\n    println!(\"Running mailbox tests...\");\n\n    // Pattern matching tests\n    mailbox_matches_pattern();\n\n    // Create third connection for testing\n    let mut other_conn = ImapConnection::connect(b\"_z \").await;\n    other_conn\n        .send(\"AUTHENTICATE PLAIN {32+}\\r\\nAGpkb2VAZXhhbXBsZS5jb20Ac2VjcmV0\")\n        .await;\n    other_conn.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // List folders\n    imap.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders([(\"INBOX\", [\"\"]), (\"Deleted Items\", [\"\"])], true);\n\n    // Create folders\n    imap.send(\"CREATE \\\"Tofu\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"CREATE \\\"Fruit\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"CREATE \\\"Fruit/Apple\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"CREATE \\\"Fruit/Apple/Green\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"CREATE \\\"L&APg-bende opgaver\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Select folder from another connection\n    other_conn.send(\"SELECT \\\"Tofu\\\"\").await;\n    other_conn.assert_read(Type::Tagged, ResponseType::Ok).await;\n    other_conn.send(\"SELECT \\\"L&APg-bende opgaver\\\"\").await;\n    other_conn.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Make sure folders are visible\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .assert_folders(\n                [\n                    (\"INBOX\", [\"\"]),\n                    (\"Deleted Items\", [\"\"]),\n                    (\"Fruit\", [\"\"]),\n                    (\"Fruit/Apple\", [\"\"]),\n                    (\"Fruit/Apple/Green\", [\"\"]),\n                    (\"Tofu\", [\"\"]),\n                    (\"L&APg-bende opgaver\", [\"\"]),\n                ],\n                true,\n            );\n    }\n    imap.send(\"DELETE \\\"L&APg-bende opgaver\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Special use folders that already exist should not be allowed\n    imap.send(\"CREATE \\\"Second trash\\\" (USE (\\\\Trash))\").await;\n    imap.assert_read(Type::Tagged, ResponseType::No).await;\n\n    // Enable IMAP4rev2\n    imap.send(\"ENABLE IMAP4rev2\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Create and delete using IMAP4rev2\n    imap.send(\"CREATE \\\"L&APg-bende opgaver\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"SELECT \\\"L&APg-bende opgaver\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"UNSELECT \\\"L&APg-bende opgaver\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"DELETE \\\"L&APg-bende opgaver\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Create missing parent folders\n    imap.send(\"CREATE \\\"/Vegetable/Broccoli\\\" (USE (\\\\Important))\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"[MAILBOXID (\");\n\n    imap.send(\"CREATE \\\" Cars/Electric /4 doors/ Red/\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"LIST \\\"\\\" \\\"*\\\" RETURN (CHILDREN SPECIAL-USE)\")\n            .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .assert_folders(\n                [\n                    (\"INBOX\", [\"HasNoChildren\", \"\"]),\n                    (\"Deleted Items\", [\"HasNoChildren\", \"Trash\"]),\n                    (\"Cars/Electric/4 doors/Red\", [\"HasNoChildren\", \"\"]),\n                    (\"Cars/Electric/4 doors\", [\"HasChildren\", \"\"]),\n                    (\"Cars/Electric\", [\"HasChildren\", \"\"]),\n                    (\"Cars\", [\"HasChildren\", \"\"]),\n                    (\"Fruit\", [\"HasChildren\", \"\"]),\n                    (\"Fruit/Apple\", [\"HasChildren\", \"\"]),\n                    (\"Fruit/Apple/Green\", [\"HasNoChildren\", \"\"]),\n                    (\"Vegetable\", [\"HasChildren\", \"\"]),\n                    (\"Vegetable/Broccoli\", [\"HasNoChildren\", \"\\\\Important\"]),\n                    (\"Tofu\", [\"HasNoChildren\", \"\"]),\n                ],\n                true,\n            );\n    }\n\n    // Rename folders\n    imap.send(\"RENAME \\\"Fruit/Apple/Green\\\" \\\"Fruit/Apple/Red\\\"\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"RENAME \\\"Cars\\\" \\\"Vehicles\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"RENAME \\\"Vegetable/Broccoli\\\" \\\"Veggies/Green/Broccoli\\\"\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"RENAME \\\"Tofu\\\" \\\"INBOX\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::No).await;\n    imap.send(\"RENAME \\\"Tofu\\\" \\\"Inbox/Tofu\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"RENAME \\\"Deleted Items\\\" \\\"Recycle Bin\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"LIST \\\"\\\" \\\"*\\\" RETURN (CHILDREN SPECIAL-USE)\")\n            .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .assert_folders(\n                [\n                    (\"INBOX\", [\"HasChildren\", \"\"]),\n                    (\"INBOX/Tofu\", [\"HasNoChildren\", \"\"]),\n                    (\"Recycle Bin\", [\"HasNoChildren\", \"Trash\"]),\n                    (\"Vehicles/Electric/4 doors/Red\", [\"HasNoChildren\", \"\"]),\n                    (\"Vehicles/Electric/4 doors\", [\"HasChildren\", \"\"]),\n                    (\"Vehicles/Electric\", [\"HasChildren\", \"\"]),\n                    (\"Vehicles\", [\"HasChildren\", \"\"]),\n                    (\"Fruit\", [\"HasChildren\", \"\"]),\n                    (\"Fruit/Apple\", [\"HasChildren\", \"\"]),\n                    (\"Fruit/Apple/Red\", [\"HasNoChildren\", \"\"]),\n                    (\"Vegetable\", [\"HasNoChildren\", \"\"]),\n                    (\"Veggies\", [\"HasChildren\", \"\"]),\n                    (\"Veggies/Green\", [\"HasChildren\", \"\"]),\n                    (\"Veggies/Green/Broccoli\", [\"HasNoChildren\", \"\"]),\n                ],\n                true,\n            );\n    }\n\n    // Delete folders\n    imap.send(\"DELETE \\\"INBOX/Tofu\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"DELETE \\\"Vegetable\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"DELETE \\\"Vehicles\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::No).await;\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"LIST \\\"\\\" \\\"*\\\" RETURN (CHILDREN SPECIAL-USE)\")\n            .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .assert_folders(\n                [\n                    (\"INBOX\", [\"HasNoChildren\", \"\"]),\n                    (\"Recycle Bin\", [\"HasNoChildren\", \"Trash\"]),\n                    (\"Vehicles/Electric/4 doors/Red\", [\"HasNoChildren\", \"\"]),\n                    (\"Vehicles/Electric/4 doors\", [\"HasChildren\", \"\"]),\n                    (\"Vehicles/Electric\", [\"HasChildren\", \"\"]),\n                    (\"Vehicles\", [\"HasChildren\", \"\"]),\n                    (\"Fruit\", [\"HasChildren\", \"\"]),\n                    (\"Fruit/Apple\", [\"HasChildren\", \"\"]),\n                    (\"Fruit/Apple/Red\", [\"HasNoChildren\", \"\"]),\n                    (\"Veggies\", [\"HasChildren\", \"\"]),\n                    (\"Veggies/Green\", [\"HasChildren\", \"\"]),\n                    (\"Veggies/Green/Broccoli\", [\"HasNoChildren\", \"\"]),\n                ],\n                true,\n            );\n    }\n\n    // Subscribe\n    imap.send(\"SUBSCRIBE \\\"INBOX\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"SUBSCRIBE \\\"Vehicles/Electric/4 doors/Red\\\"\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"LIST \\\"\\\" \\\"*\\\" RETURN (SUBSCRIBED SPECIAL-USE)\")\n            .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .assert_folders(\n                [\n                    (\"INBOX\", [\"Subscribed\", \"\"]),\n                    (\"Recycle Bin\", [\"\", \"Trash\"]),\n                    (\"Vehicles/Electric/4 doors/Red\", [\"Subscribed\", \"\"]),\n                    (\"Vehicles/Electric/4 doors\", [\"\", \"\"]),\n                    (\"Vehicles/Electric\", [\"\", \"\"]),\n                    (\"Vehicles\", [\"\", \"\"]),\n                    (\"Fruit\", [\"\", \"\"]),\n                    (\"Fruit/Apple\", [\"\", \"\"]),\n                    (\"Fruit/Apple/Red\", [\"\", \"\"]),\n                    (\"Veggies\", [\"\", \"\"]),\n                    (\"Veggies/Green\", [\"\", \"\"]),\n                    (\"Veggies/Green/Broccoli\", [\"\", \"\"]),\n                ],\n                true,\n            );\n    }\n\n    // Filter by subscribed including children\n    imap.send(\"LIST (SUBSCRIBED) \\\"\\\" \\\"*\\\" RETURN (CHILDREN)\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders(\n            [\n                (\"INBOX\", [\"Subscribed\", \"HasNoChildren\"]),\n                (\n                    \"Vehicles/Electric/4 doors/Red\",\n                    [\"Subscribed\", \"HasNoChildren\"],\n                ),\n            ],\n            true,\n        );\n\n    // Recursive match including children\n    imap.send(\"LIST (SUBSCRIBED RECURSIVEMATCH) \\\"\\\" \\\"*\\\" RETURN (CHILDREN)\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders(\n            [\n                (\"INBOX\", [\"Subscribed\", \"HasNoChildren\"]),\n                (\n                    \"Vehicles/Electric/4 doors/Red\",\n                    [\"Subscribed\", \"HasNoChildren\"],\n                ),\n                (\n                    \"Vehicles/Electric/4 doors\",\n                    [\"\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\")\", \"HasChildren\"],\n                ),\n                (\n                    \"Vehicles/Electric\",\n                    [\"\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\")\", \"HasChildren\"],\n                ),\n                (\n                    \"Vehicles\",\n                    [\"\\\"CHILDINFO\\\" (\\\"SUBSCRIBED\\\")\", \"HasChildren\"],\n                ),\n            ],\n            true,\n        );\n\n    // Imap4rev1 LSUB\n    imap.send(\"LSUB \\\"\\\" \\\"*\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders(\n            [(\"INBOX\", [\"\"]), (\"Vehicles/Electric/4 doors/Red\", [\"\"])],\n            true,\n        );\n\n    // Unsubscribe\n    imap.send(\"UNSUBSCRIBE \\\"Vehicles/Electric/4 doors/Red\\\"\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"LIST (SUBSCRIBED RECURSIVEMATCH) \\\"\\\" \\\"*\\\" RETURN (CHILDREN)\")\n            .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .assert_folders([(\"INBOX\", [\"Subscribed\", \"HasNoChildren\"])], true);\n    }\n\n    // LIST Filters\n    imap.send(\"LIST \\\"\\\" \\\"%\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders(\n            [\n                (\"INBOX\", [\"\"]),\n                (\"Recycle Bin\", [\"\"]),\n                (\"Vehicles\", [\"\"]),\n                (\"Fruit\", [\"\"]),\n                (\"Veggies\", [\"\"]),\n            ],\n            true,\n        );\n\n    imap.send(\"LIST \\\"\\\" \\\"*/Red\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders(\n            [\n                (\"Vehicles/Electric/4 doors/Red\", [\"\"]),\n                (\"Fruit/Apple/Red\", [\"\"]),\n            ],\n            true,\n        );\n\n    imap.send(\"LIST \\\"\\\" \\\"Fruit/*\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders([(\"Fruit/Apple/Red\", [\"\"]), (\"Fruit/Apple\", [\"\"])], true);\n\n    imap.send(\"LIST \\\"\\\" \\\"Fruit/%\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders([(\"Fruit/Apple\", [\"\"])], true);\n\n    // Restore Trash folder's original name\n    imap.send(\"RENAME \\\"Recycle Bin\\\" \\\"Deleted Items\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Shared folder creation tests\n    let mut imap_jane = ImapConnection::connect(b\"_z \").await;\n    imap_jane\n        .authenticate(\"jane.smith@example.com\", \"secret\")\n        .await;\n    imap_jane\n        .send(\"CREATE \\\"Shared Folders/support@example.com/INBOX/Test\\\"\")\n        .await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_jane\n        .send(\"CREATE \\\"Shared Folders/support@example.com/Test\\\"\")\n        .await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    imap_jane\n        .send(\"CREATE \\\"Shared Folders/support@example.com/Test/TestSubfolder\\\"\")\n        .await;\n    imap_jane.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap_jane.send(\"LIST \\\"\\\" \\\"*\\\"\").await;\n    imap_jane\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_folders(\n            [\n                (\"INBOX\", [\"\"]),\n                (\"Deleted Items\", [\"\"]),\n                (\"Drafts\", [\"\"]),\n                (\"Junk Mail\", [\"\"]),\n                (\"Sent Items\", [\"\"]),\n                (\"Shared Folders\", [\"\"]),\n                (\"Shared Folders/support@example.com\", [\"\"]),\n                (\"Shared Folders/support@example.com/Deleted Items\", [\"\"]),\n                (\"Shared Folders/support@example.com/Drafts\", [\"\"]),\n                (\"Shared Folders/support@example.com/INBOX\", [\"\"]),\n                (\"Shared Folders/support@example.com/INBOX/Test\", [\"\"]),\n                (\"Shared Folders/support@example.com/Junk Mail\", [\"\"]),\n                (\"Shared Folders/support@example.com/Sent Items\", [\"\"]),\n                (\"Shared Folders/support@example.com/Test\", [\"\"]),\n                (\n                    \"Shared Folders/support@example.com/Test/TestSubfolder\",\n                    [\"\"],\n                ),\n            ],\n            true,\n        );\n}\n\nfn mailbox_matches_pattern() {\n    let mailboxes = [\n        \"imaptest\",\n        \"imaptest/test\",\n        \"imaptest/test2\",\n        \"imaptest/test3\",\n        \"imaptest/test3/test4\",\n        \"imaptest/test3/test4/test5\",\n        \"foobar/test\",\n        \"foobar/test/test\",\n        \"foobar/test1/test1\",\n    ];\n\n    for (pattern, expected_match) in [\n        (\n            \"imaptest/%\",\n            vec![\"imaptest/test\", \"imaptest/test2\", \"imaptest/test3\"],\n        ),\n        (\"imaptest/%/%\", vec![\"imaptest/test3/test4\"]),\n        (\n            \"imaptest/*\",\n            vec![\n                \"imaptest/test\",\n                \"imaptest/test2\",\n                \"imaptest/test3\",\n                \"imaptest/test3/test4\",\n                \"imaptest/test3/test4/test5\",\n            ],\n        ),\n        (\"imaptest/*test4\", vec![\"imaptest/test3/test4\"]),\n        (\n            \"imaptest/*test*\",\n            vec![\n                \"imaptest/test\",\n                \"imaptest/test2\",\n                \"imaptest/test3\",\n                \"imaptest/test3/test4\",\n                \"imaptest/test3/test4/test5\",\n            ],\n        ),\n        (\"imaptest/%3/%\", vec![\"imaptest/test3/test4\"]),\n        (\"imaptest/%3/%4\", vec![\"imaptest/test3/test4\"]),\n        (\"imaptest/%t*4\", vec![\"imaptest/test3/test4\"]),\n        (\"*st/%3/%4/%5\", vec![\"imaptest/test3/test4/test5\"]),\n        (\n            \"*%*%*%\",\n            vec![\n                \"imaptest\",\n                \"imaptest/test\",\n                \"imaptest/test2\",\n                \"imaptest/test3\",\n                \"imaptest/test3/test4\",\n                \"imaptest/test3/test4/test5\",\n                \"foobar/test\",\n                \"foobar/test/test\",\n                \"foobar/test1/test1\",\n            ],\n        ),\n        (\"foobar*test\", vec![\"foobar/test\", \"foobar/test/test\"]),\n    ] {\n        let patterns = vec![pattern.into()];\n        let mut matched_mailboxes = Vec::new();\n        for mailbox in mailboxes {\n            if matches_pattern(&patterns, mailbox) {\n                matched_mailboxes.push(mailbox);\n            }\n        }\n        assert_eq!(matched_mailboxes, expected_match, \"for pattern {}\", pattern);\n    }\n}\n"
  },
  {
    "path": "tests/src/imap/managesieve.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse imap_proto::ResponseType;\nuse mail_send::smtp::tls::build_tls_connector;\nuse rustls_pki_types::ServerName;\nuse tokio::{\n    io::{AsyncBufReadExt, AsyncWriteExt, BufReader, Lines, ReadHalf, WriteHalf},\n    net::TcpStream,\n};\nuse tokio_rustls::client::TlsStream;\n\nuse super::AssertResult;\n\npub async fn test() {\n    println!(\"Running ManageSieve tests...\");\n\n    // Connect to ManageSieve\n    let mut sieve = SieveConnection::connect().await;\n    sieve\n        .assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"IMPLEMENTATION\");\n\n    // Authenticate\n    sieve\n        .send(\"AUTHENTICATE \\\"PLAIN\\\" \\\"AGpkb2VAZXhhbXBsZS5jb20Ac2VjcmV0\\\"\")\n        .await;\n    sieve.assert_read(ResponseType::Ok).await;\n    /*sieve\n    .assert_read(ResponseType::Ok)\n    .await\n    .assert_contains(\"MAXREDIRECTS\");*/\n\n    // CheckScript\n    sieve.send(\"CHECKSCRIPT \\\"if true { keep; }\\\"\").await;\n    sieve.assert_read(ResponseType::Ok).await;\n    sieve.send(\"CHECKSCRIPT \\\"keep :invalidtag;\\\"\").await;\n    sieve.assert_read(ResponseType::No).await;\n\n    // PutScript\n    sieve\n        .send_literal(\"PUTSCRIPT \\\"simple script\\\" \", \"if true { keep; }\\r\\n\")\n        .await;\n    sieve.assert_read(ResponseType::Ok).await;\n\n    // PutScript should overwrite existing scripts\n    sieve.send(\"PUTSCRIPT \\\"holidays\\\" \\\"discard;\\\"\").await;\n    sieve.assert_read(ResponseType::Ok).await;\n    sieve\n        .send_literal(\n            \"PUTSCRIPT \\\"holidays\\\" \",\n            \"require \\\"vacation\\\"; vacation \\\"Gone fishin'\\\";\\r\\n\",\n        )\n        .await;\n    sieve.assert_read(ResponseType::Ok).await;\n\n    // GetScript\n    sieve.send(\"GETSCRIPT \\\"simple script\\\"\").await;\n    sieve\n        .assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"if true\");\n    sieve.send(\"GETSCRIPT \\\"holidays\\\"\").await;\n    sieve\n        .assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"Gone fishin'\");\n    sieve.send(\"GETSCRIPT \\\"dummy\\\"\").await;\n    sieve.assert_read(ResponseType::No).await;\n\n    // ListScripts\n    sieve.send(\"LISTSCRIPTS\").await;\n    sieve\n        .assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"simple script\")\n        .assert_contains(\"holidays\")\n        .assert_count(\"ACTIVE\", 0);\n\n    // RenameScript\n    sieve\n        .send(\"RENAMESCRIPT \\\"simple script\\\" \\\"minimalist script\\\"\")\n        .await;\n    sieve.assert_read(ResponseType::Ok).await;\n    sieve\n        .send(\"RENAMESCRIPT \\\"holidays\\\" \\\"minimalist script\\\"\")\n        .await;\n    sieve\n        .assert_read(ResponseType::No)\n        .await\n        .assert_contains(\"ALREADYEXISTS\");\n\n    // SetActive\n    sieve.send(\"SETACTIVE \\\"holidays\\\"\").await;\n    sieve.assert_read(ResponseType::Ok).await;\n\n    sieve.send(\"LISTSCRIPTS\").await;\n    sieve\n        .assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"minimalist script\")\n        .assert_contains(\"holidays\\\" ACTIVE\");\n\n    // Deleting an active script should not be allowed\n    sieve.send(\"DELETESCRIPT \\\"holidays\\\"\").await;\n    sieve\n        .assert_read(ResponseType::No)\n        .await\n        .assert_contains(\"ACTIVE\");\n\n    // Deactivate all\n    sieve.send(\"SETACTIVE \\\"\\\"\").await;\n    sieve.assert_read(ResponseType::Ok).await;\n\n    sieve.send(\"LISTSCRIPTS\").await;\n    sieve\n        .assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"minimalist script\")\n        .assert_contains(\"holidays\")\n        .assert_count(\"ACTIVE\", 0);\n\n    // DeleteScript\n    sieve.send(\"DELETESCRIPT \\\"holidays\\\"\").await;\n    sieve.assert_read(ResponseType::Ok).await;\n    sieve.send(\"DELETESCRIPT \\\"minimalist script\\\"\").await;\n    sieve.assert_read(ResponseType::Ok).await;\n\n    sieve.send(\"LISTSCRIPTS\").await;\n    sieve\n        .assert_read(ResponseType::Ok)\n        .await\n        .assert_count(\"minimalist script\", 0)\n        .assert_count(\"holidays\", 0);\n}\n\npub struct SieveConnection {\n    reader: Lines<BufReader<ReadHalf<TlsStream<TcpStream>>>>,\n    writer: WriteHalf<TlsStream<TcpStream>>,\n}\n\nimpl SieveConnection {\n    pub async fn connect() -> Self {\n        let (reader, writer) = tokio::io::split(\n            build_tls_connector(true)\n                .connect(\n                    ServerName::try_from(\"imap.example.org\").unwrap().to_owned(),\n                    TcpStream::connect(\"127.0.0.1:4190\").await.unwrap(),\n                )\n                .await\n                .unwrap(),\n        );\n        SieveConnection {\n            reader: BufReader::new(reader).lines(),\n            writer,\n        }\n    }\n\n    pub async fn assert_read(&mut self, rt: ResponseType) -> Vec<String> {\n        let lines = self.read().await;\n        let mut buf = Vec::with_capacity(10);\n        rt.serialize(&mut buf);\n        if lines\n            .last()\n            .unwrap()\n            .starts_with(&String::from_utf8(buf).unwrap())\n        {\n            lines\n        } else {\n            panic!(\"Expected {:?} from server but got: {:?}\", rt, lines);\n        }\n    }\n\n    pub async fn read(&mut self) -> Vec<String> {\n        let mut lines = Vec::new();\n        loop {\n            match tokio::time::timeout(Duration::from_millis(1500), self.reader.next_line()).await {\n                Ok(Ok(Some(line))) => {\n                    let is_done =\n                        line.starts_with(\"OK\") || line.starts_with(\"NO\") || line.starts_with(\"BYE\");\n                    //println!(\"<- {:?}\", line);\n                    lines.push(line);\n                    if is_done {\n                        return lines;\n                    }\n                }\n                Ok(Ok(None)) => {\n                    panic!(\"Invalid response: {:?}.\", lines);\n                }\n                Ok(Err(err)) => {\n                    panic!(\"Connection broken: {} ({:?})\", err, lines);\n                }\n                Err(_) => panic!(\"Timeout while waiting for server response: {:?}\", lines),\n            }\n        }\n    }\n\n    pub async fn send(&mut self, text: &str) {\n        //println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n        self.writer.write_all(b\"\\r\\n\").await.unwrap();\n    }\n\n    pub async fn send_raw(&mut self, text: &str) {\n        //println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n    }\n\n    pub async fn send_literal(&mut self, text: &str, literal: &str) {\n        self.send(&format!(\"{}{{{}+}}\\r\\n{}\", text, literal.len(), literal))\n            .await;\n    }\n}\n"
  },
  {
    "path": "tests/src/imap/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod antispam;\npub mod append;\npub mod basic;\npub mod body_structure;\npub mod condstore;\npub mod copy_move;\npub mod fetch;\npub mod idle;\npub mod mailbox;\npub mod managesieve;\npub mod pop;\npub mod search;\npub mod store;\npub mod thread;\n\nuse crate::{\n    AssertConfig, add_test_certs,\n    directory::internal::TestInternalDirectory,\n    store::{\n        TempDir, build_store_config,\n        cleanup::{search_store_destroy, store_destroy},\n    },\n};\nuse ::managesieve::core::ManageSieveSessionManager;\nuse ::store::Stores;\nuse ahash::AHashSet;\nuse base64::{Engine, engine::general_purpose};\nuse common::{\n    Caches, Core, Data, Inner, Server,\n    config::{\n        server::{Listeners, ServerProtocol},\n        telemetry::Telemetry,\n    },\n    core::BuildServer,\n    manager::boot::build_ipc,\n};\nuse http::HttpSessionManager;\nuse imap::core::ImapSessionManager;\nuse imap_proto::ResponseType;\nuse pop3::Pop3SessionManager;\nuse services::SpawnServices;\nuse smtp::{SpawnQueueManager, core::SmtpSessionManager};\nuse std::{\n    path::PathBuf,\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse tokio::{\n    io::{AsyncBufReadExt, AsyncWriteExt, BufReader, Lines, ReadHalf, WriteHalf},\n    net::TcpStream,\n    sync::watch,\n};\nuse utils::config::Config;\n\n#[tokio::test]\npub async fn imap_tests() {\n    // Prepare settings\n    let start_time = Instant::now();\n    let delete = true;\n    let handle = init_imap_tests(delete).await;\n\n    // Body structure tests\n    body_structure::test();\n\n    // Connect to IMAP server\n    let mut imap_check = ImapConnection::connect(b\"_y \").await;\n    let mut imap = ImapConnection::connect(b\"_x \").await;\n    for imap in [&mut imap, &mut imap_check] {\n        imap.assert_read(Type::Untagged, ResponseType::Ok).await;\n    }\n\n    // Unauthenticated tests\n    basic::test(&mut imap, &mut imap_check).await;\n\n    // Login\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"AUTHENTICATE PLAIN {32+}\\r\\nAGpkb2VAZXhhbXBsZS5jb20Ac2VjcmV0\")\n            .await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    }\n\n    // Delete folders\n    for mailbox in [\"Drafts\", \"Junk Mail\", \"Sent Items\"] {\n        imap.send(&format!(\"DELETE \\\"{}\\\"\", mailbox)).await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    }\n\n    mailbox::test(&mut imap, &mut imap_check).await;\n    append::test(&mut imap, &mut imap_check, &handle).await;\n    search::test(&mut imap, &mut imap_check, &handle).await;\n    fetch::test(&mut imap, &mut imap_check).await;\n    store::test(&mut imap, &mut imap_check, &handle).await;\n    copy_move::test(&mut imap, &mut imap_check).await;\n    thread::test(&mut imap, &mut imap_check, &handle).await;\n    idle::test(&mut imap, &mut imap_check, false).await;\n    condstore::test(&mut imap, &mut imap_check).await;\n    acl::test(&mut imap, &mut imap_check).await;\n\n    // Logout\n    for imap in [&mut imap, &mut imap_check] {\n        imap.send(\"UNAUTHENTICATE\").await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n        imap.send(\"LOGOUT\").await;\n        imap.assert_read(Type::Untagged, ResponseType::Bye).await;\n    }\n\n    // Antispam training\n    antispam::test(&handle).await;\n\n    // Run ManageSieve tests\n    managesieve::test().await;\n\n    // Run POP3 tests\n    pop::test().await;\n\n    // Print elapsed time\n    let elapsed = start_time.elapsed();\n    println!(\n        \"Elapsed: {}.{:03}s\",\n        elapsed.as_secs(),\n        elapsed.subsec_millis()\n    );\n\n    // Remove test data\n    if delete {\n        handle.temp_dir.delete();\n    }\n}\n\n#[allow(dead_code)]\npub struct IMAPTest {\n    server: Server,\n    temp_dir: TempDir,\n    shutdown_tx: watch::Sender<bool>,\n}\n\nasync fn init_imap_tests(delete_if_exists: bool) -> IMAPTest {\n    // Load and parse config\n    let temp_dir = TempDir::new(\"imap_tests\", delete_if_exists);\n    let mut config = Config::new(\n        add_test_certs(&(build_store_config(&temp_dir.path.to_string_lossy()) + SERVER))\n            .replace(\"{TMP}\", &temp_dir.path.display().to_string())\n            .replace(\n                \"{LEVEL}\",\n                &std::env::var(\"LOG\").unwrap_or_else(|_| \"disable\".to_string()),\n            ),\n    )\n    .unwrap();\n    config.resolve_all_macros().await;\n\n    // Parse servers\n    let mut servers = Listeners::parse(&mut config);\n\n    // Bind ports and drop privileges\n    servers.bind_and_drop_priv(&mut config);\n\n    // Build stores\n    let stores = Stores::parse_all(&mut config, false).await;\n\n    // Parse core\n    let tracers = Telemetry::parse(&mut config, &stores);\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    let data = Data::parse(&mut config);\n    let cache = Caches::parse(&mut config);\n\n    let store = core.storage.data.clone();\n    let search_store = core.storage.fts.clone();\n    let (ipc, mut ipc_rxs) = build_ipc(false);\n    let inner = Arc::new(Inner {\n        shared_core: core.into_shared(),\n        data,\n        ipc,\n        cache,\n    });\n\n    // Parse acceptors\n    servers.parse_tcp_acceptors(&mut config, inner.clone());\n\n    // Enable tracing\n    tracers.enable(true);\n\n    // Start services\n    config.assert_no_errors();\n    ipc_rxs.spawn_queue_manager(inner.clone());\n    ipc_rxs.spawn_services(inner.clone());\n\n    // Spawn servers\n    let (shutdown_tx, _) = servers.spawn(|server, acceptor, shutdown_rx| {\n        match &server.protocol {\n            ServerProtocol::Smtp | ServerProtocol::Lmtp => server.spawn(\n                SmtpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Http => server.spawn(\n                HttpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Imap => server.spawn(\n                ImapSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Pop3 => server.spawn(\n                Pop3SessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::ManageSieve => server.spawn(\n                ManageSieveSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n        };\n    });\n\n    if delete_if_exists {\n        store_destroy(&store).await;\n        search_store_destroy(&search_store).await;\n    }\n\n    // Create tables and test accounts\n    store\n        .create_test_user(\"admin\", \"secret\", \"Superuser\", &[])\n        .await;\n    store\n        .create_test_user(\n            \"jdoe@example.com\",\n            \"secret\",\n            \"John Doe\",\n            &[\"jdoe@example.com\"],\n        )\n        .await;\n    store\n        .create_test_user(\n            \"jane.smith@example.com\",\n            \"secret\",\n            \"Jane Smith\",\n            &[\"jane.smith@example.com\"],\n        )\n        .await;\n    store\n        .create_test_user(\n            \"foobar@example.com\",\n            \"secret\",\n            \"Bill Foobar\",\n            &[\"foobar@example.com\"],\n        )\n        .await;\n    store\n        .create_test_user(\n            \"popper@example.com\",\n            \"secret\",\n            \"Karl Popper\",\n            &[\"popper@example.com\"],\n        )\n        .await;\n    store\n        .create_test_user(\n            \"sgd@example.com\",\n            \"secret\",\n            \"Sigmund Gudmund Dudmundsson\",\n            &[\"sgd@example.com\"],\n        )\n        .await;\n    store\n        .create_test_user(\n            \"spamtrap@example.com\",\n            \"secret\",\n            \"Spam Trap\",\n            &[\"spamtrap@example.com\"],\n        )\n        .await;\n    store\n        .create_test_group(\n            \"support@example.com\",\n            \"Support Group\",\n            &[\"support@example.com\"],\n        )\n        .await;\n    store\n        .add_to_group(\"jane.smith@example.com\", \"support@example.com\")\n        .await;\n\n    IMAPTest {\n        server: inner.build_server(),\n        temp_dir,\n        shutdown_tx,\n    }\n}\n\npub struct ImapConnection {\n    tag: &'static [u8],\n    reader: Lines<BufReader<ReadHalf<TcpStream>>>,\n    writer: WriteHalf<TcpStream>,\n}\n\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum Type {\n    Tagged,\n    Untagged,\n    Continuation,\n    Status,\n}\n\nimpl ImapConnection {\n    pub async fn connect(tag: &'static [u8]) -> Self {\n        Self::connect_to(tag, \"127.0.0.1:9991\").await\n    }\n\n    pub async fn connect_to(tag: &'static [u8], addr: impl AsRef<str>) -> Self {\n        let (reader, writer) = tokio::io::split(TcpStream::connect(addr.as_ref()).await.unwrap());\n        ImapConnection {\n            tag,\n            reader: BufReader::new(reader).lines(),\n            writer,\n        }\n    }\n\n    pub async fn assert_read(&mut self, t: Type, rt: ResponseType) -> Vec<String> {\n        let lines = self.read(t).await;\n        let mut buf = Vec::with_capacity(10);\n        buf.extend_from_slice(match t {\n            Type::Tagged => self.tag,\n            Type::Untagged | Type::Status => b\"* \",\n            Type::Continuation => b\"+ \",\n        });\n        if !matches!(t, Type::Continuation | Type::Status) {\n            rt.serialize(&mut buf);\n        }\n        if lines\n            .last()\n            .unwrap()\n            .starts_with(&String::from_utf8(buf).unwrap())\n        {\n            lines\n        } else {\n            panic!(\"Expected {:?}/{:?} from server but got: {:?}\", t, rt, lines);\n        }\n    }\n\n    pub async fn assert_disconnect(&mut self) {\n        match tokio::time::timeout(Duration::from_millis(1500), self.reader.next_line()).await {\n            Ok(Ok(None)) => {}\n            Ok(Ok(Some(line))) => {\n                panic!(\"Expected connection to be closed, but got {:?}\", line);\n            }\n            Ok(Err(err)) => {\n                panic!(\"Connection broken: {:?}\", err);\n            }\n            Err(_) => panic!(\"Timeout while waiting for server response.\"),\n        }\n    }\n\n    pub async fn read(&mut self, t: Type) -> Vec<String> {\n        let mut lines = Vec::new();\n        loop {\n            match tokio::time::timeout(Duration::from_millis(1500), self.reader.next_line()).await {\n                Ok(Ok(Some(line))) => {\n                    let is_done = line.starts_with(match t {\n                        Type::Tagged => std::str::from_utf8(self.tag).unwrap(),\n                        Type::Untagged | Type::Status => \"* \",\n                        Type::Continuation => \"+ \",\n                    });\n                    //let c = println!(\"<- {:?}\", line);\n                    lines.push(line);\n                    if is_done {\n                        return lines;\n                    }\n                }\n                Ok(Ok(None)) => {\n                    panic!(\"Invalid response: {:?}.\", lines);\n                }\n                Ok(Err(err)) => {\n                    panic!(\"Connection broken: {} ({:?})\", err, lines);\n                }\n                Err(_) => panic!(\"Timeout while waiting for server response: {:?}\", lines),\n            }\n        }\n    }\n\n    pub async fn authenticate(&mut self, user: &str, pass: &str) {\n        let creds = general_purpose::STANDARD.encode(format!(\"\\0{user}\\0{pass}\"));\n        self.send(&format!(\n            \"AUTHENTICATE PLAIN {{{}+}}\\r\\n{creds}\",\n            creds.len()\n        ))\n        .await;\n        self.assert_read(Type::Tagged, ResponseType::Ok).await;\n    }\n\n    pub async fn send(&mut self, text: &str) {\n        //let c = println!(\"-> {}{:?}\", std::str::from_utf8(self.tag).unwrap(), text);\n        self.writer.write_all(self.tag).await.unwrap();\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n        self.writer.write_all(b\"\\r\\n\").await.unwrap();\n    }\n\n    pub async fn send_untagged(&mut self, text: &str) {\n        //let c = println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n        self.writer.write_all(b\"\\r\\n\").await.unwrap();\n    }\n\n    pub async fn send_raw(&mut self, text: &str) {\n        //let c = println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n    }\n}\n\npub trait AssertResult: Sized {\n    fn assert_folders<'x>(\n        self,\n        expected: impl IntoIterator<Item = (&'x str, impl IntoIterator<Item = &'x str>)>,\n        match_all: bool,\n    ) -> Self;\n\n    fn assert_response_code(self, code: &str) -> Self;\n    fn assert_contains(self, text: &str) -> Self;\n    fn assert_count(self, text: &str, occurrences: usize) -> Self;\n    fn assert_equals(self, text: &str) -> Self;\n    fn into_response_code(self) -> String;\n    fn into_highest_modseq(self) -> String;\n    fn into_uid_validity(self) -> String;\n    fn into_append_uid(self) -> String;\n    fn into_copy_uid(self) -> String;\n    fn into_modseq(self) -> String;\n}\n\nimpl AssertResult for Vec<String> {\n    fn assert_folders<'x>(\n        self,\n        expected: impl IntoIterator<Item = (&'x str, impl IntoIterator<Item = &'x str>)>,\n        match_all: bool,\n    ) -> Self {\n        let mut match_count = 0;\n        'outer: for (mailbox_name, flags) in expected.into_iter() {\n            for result in self.iter() {\n                if result.contains(&format!(\"\\\"{}\\\"\", mailbox_name)) {\n                    for flag in flags {\n                        if !flag.is_empty() && !result.contains(flag) {\n                            panic!(\"Expected mailbox {} to have flag {}\", mailbox_name, flag);\n                        }\n                    }\n                    match_count += 1;\n                    continue 'outer;\n                }\n            }\n            panic!(\"Mailbox {} is not present.\", mailbox_name);\n        }\n        if match_all && match_count != self.len() - 1 {\n            panic!(\n                \"Expected {} mailboxes, but got {}: {:?}\",\n                match_count,\n                self.len() - 1,\n                self.iter().collect::<Vec<_>>()\n            );\n        }\n        self\n    }\n\n    fn assert_response_code(self, code: &str) -> Self {\n        if !self.last().unwrap().contains(&format!(\"[{}]\", code)) {\n            panic!(\n                \"Response code {:?} not found, got {:?}\",\n                code,\n                self.last().unwrap()\n            );\n        }\n        self\n    }\n\n    fn assert_contains(self, text: &str) -> Self {\n        for line in &self {\n            if line.contains(text) {\n                return self;\n            }\n        }\n        panic!(\"Expected response to contain {:?}, got {:?}\", text, self);\n    }\n\n    fn assert_count(self, text: &str, occurrences: usize) -> Self {\n        assert_eq!(\n            self.iter().filter(|l| l.contains(text)).count(),\n            occurrences,\n            \"Expected {} occurrences of {:?}, found {} in {:?}.\",\n            occurrences,\n            text,\n            self.iter().filter(|l| l.contains(text)).count(),\n            self\n        );\n        self\n    }\n\n    fn assert_equals(self, text: &str) -> Self {\n        for line in &self {\n            if line == text {\n                return self;\n            }\n        }\n        panic!(\"Expected response to be {:?}, got {:?}\", text, self);\n    }\n\n    fn into_response_code(self) -> String {\n        if let Some((_, code)) = self.last().unwrap().split_once('[')\n            && let Some((code, _)) = code.split_once(']')\n        {\n            return code.to_string();\n        }\n        panic!(\"No response code found in {:?}\", self.last().unwrap());\n    }\n\n    fn into_append_uid(self) -> String {\n        if let Some((_, code)) = self.last().unwrap().split_once(\"[APPENDUID \")\n            && let Some((code, _)) = code.split_once(']')\n            && let Some((_, uid)) = code.split_once(' ')\n        {\n            return uid.to_string();\n        }\n        panic!(\"No APPENDUID found in {:?}\", self.last().unwrap());\n    }\n\n    fn into_copy_uid(self) -> String {\n        for line in &self {\n            if let Some((_, code)) = line.split_once(\"[COPYUID \")\n                && let Some((code, _)) = code.split_once(']')\n                && let Some((_, uid)) = code.rsplit_once(' ')\n            {\n                return uid.to_string();\n            }\n        }\n        panic!(\"No COPYUID found in {:?}\", self);\n    }\n\n    fn into_highest_modseq(self) -> String {\n        for line in &self {\n            if let Some((_, value)) = line.split_once(\"HIGHESTMODSEQ \") {\n                if let Some((value, _)) = value.split_once(']') {\n                    return value.to_string();\n                } else if let Some((value, _)) = value.split_once(')') {\n                    return value.to_string();\n                } else {\n                    panic!(\"No HIGHESTMODSEQ delimiter found in {:?}\", line);\n                }\n            }\n        }\n        panic!(\"No HIGHESTMODSEQ entries found in {:?}\", self);\n    }\n\n    fn into_modseq(self) -> String {\n        for line in &self {\n            if let Some((_, value)) = line.split_once(\"MODSEQ (\") {\n                if let Some((value, _)) = value.split_once(')') {\n                    return value.to_string();\n                } else {\n                    panic!(\"No MODSEQ delimiter found in {:?}\", line);\n                }\n            }\n        }\n        panic!(\"No MODSEQ entries found in {:?}\", self);\n    }\n\n    fn into_uid_validity(self) -> String {\n        for line in &self {\n            if let Some((_, value)) = line.split_once(\"UIDVALIDITY \") {\n                if let Some((value, _)) = value.split_once(']') {\n                    return value.to_string();\n                } else if let Some((value, _)) = value.split_once(')') {\n                    return value.to_string();\n                } else {\n                    panic!(\"No UIDVALIDITY delimiter found in {:?}\", line);\n                }\n            }\n        }\n        panic!(\"No UIDVALIDITY entries found in {:?}\", self);\n    }\n}\n\npub fn expand_uid_list(list: &str) -> AHashSet<u32> {\n    let mut items = AHashSet::new();\n    for uid in list.split(',') {\n        if let Some((start, end)) = uid.split_once(':') {\n            let start = start.parse::<u32>().unwrap();\n            let end = end.parse::<u32>().unwrap();\n            for uid in start..=end {\n                items.insert(uid);\n            }\n        } else {\n            items.insert(uid.parse::<u32>().unwrap());\n        }\n    }\n\n    items\n}\n\nfn resources_dir() -> PathBuf {\n    let mut resources = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    resources.push(\"resources\");\n    resources.push(\"imap\");\n    resources\n}\n\nconst SERVER: &str = r#\"\n[server]\nhostname = \"imap.example.org\"\n\n[server.listener.imap]\nbind = [\"127.0.0.1:9991\"]\nprotocol = \"imap\"\nmax-connections = 81920\n\n[server.listener.imaptls]\nbind = [\"127.0.0.1:9992\"]\nprotocol = \"imap\"\nmax-connections = 81920\ntls.implicit = true\n\n[server.listener.sieve]\nbind = [\"127.0.0.1:4190\"]\nprotocol = \"managesieve\"\nmax-connections = 81920\ntls.implicit = true\n\n[server.listener.pop3]\nbind = [\"127.0.0.1:4110\"]\nprotocol = \"pop3\"\nmax-connections = 81920\ntls.implicit = true\n\n[server.listener.lmtp-debug]\nbind = ['127.0.0.1:11201']\ngreeting = 'Test LMTP instance'\nprotocol = 'lmtp'\ntls.implicit = false\n\n[server.socket]\nreuse-addr = true\n\n[server.tls]\nenable = true\nimplicit = false\ncertificate = \"default\"\n\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = [ { if = \"!is_empty(authenticated_as)\", then = true }, \n          { else = false } ]\n\n[session.rcpt.errors]\ntotal = 5\nwait = \"1ms\"\n\n[spam-filter]\nenable = true\n\n[spam-filter.list]\nscores = {\"PROB_SPAM_LOW\" = \"10.0\", \"PROB_SPAM_HIGH\" = \"10.0\", \"SPAM_TRAP\" = \"100.0\"}\n\n[spam-filter.classifier.samples]\nmin-ham = 10\nmin-spam = 10\n\n[lookup]\n\"spam-traps\" = {\"spamtrap@*\"}\n\n[resolver]\ntype = \"system\"\n\n[queue.strategy]\nroute = [ { if = \"rcpt_domain == 'example.com'\", then = \"'local'\" }, \n             { if = \"contains(['remote.org', 'foobar.com', 'test.com', 'other_domain.com'], rcpt_domain)\", then = \"'mock-smtp'\" },\n             { else = \"'mx'\" } ]\n\n[queue.route.\"mock-smtp\"]\ntype = \"relay\"\naddress = \"localhost\"\nport = 9999\nprotocol = \"smtp\"\n\n[queue.route.\"mock-smtp\".tls]\nenable = false\nallow-invalid-certs = true\n\n[session.data]\nspam-filter = \"recipients[0] != 'popper@example.com'\"\n\n[session.data.add-headers]\ndelivered-to = false\n\n[session.extensions]\nfuture-release = [ { if = \"!is_empty(authenticated_as)\", then = \"99999999d\"},\n                   { else = false } ]\n\n[certificate.default]\ncert = \"%{file:{CERT}}%\"\nprivate-key = \"%{file:{PK}}%\"\n\n[imap.protocol]\nuidplus = true\n\n[jmap.protocol]\nset.max-objects = 100000\n\n[jmap.protocol.request]\nmax-concurrent = 8\n\n[jmap.protocol.upload]\nmax-size = 5000000\nmax-concurrent = 4\nttl = \"1m\"\n\n[jmap.protocol.upload.quota]\nfiles = 3\nsize = 50000\n\n[jmap.rate-limit]\naccount = \"1000/1m\"\nauthentication = \"100/2s\"\nanonymous = \"100/1m\"\n\n[jmap.event-source]\nthrottle = \"500ms\"\n\n[jmap.web-sockets]\nthrottle = \"500ms\"\n\n[jmap.push]\nthrottle = \"500ms\"\nattempts.interval = \"500ms\"\n\n[email.folders.inbox]\nname = \"Inbox\"\nsubscribe = false\n\n[email.folders.sent]\nname = \"Sent Items\"\nsubscribe = false\n\n[email.folders.trash]\nname = \"Deleted Items\"\nsubscribe = false\n\n[email.folders.junk]\nname = \"Junk Mail\"\nsubscribe = false\n\n[email.folders.drafts]\nname = \"Drafts\"\nsubscribe = false\n\n[store.\"auth\"]\ntype = \"sqlite\"\npath = \"{TMP}/auth.db\"\n\n[store.\"auth\".query]\nname = \"SELECT name, type, secret, description, quota FROM accounts WHERE name = ? AND active = true\"\nmembers = \"SELECT member_of FROM group_members WHERE name = ?\"\nrecipients = \"SELECT name FROM emails WHERE address = ?\"\nemails = \"SELECT address FROM emails WHERE name = ? AND type != 'list' ORDER BY type DESC, address ASC\"\nverify = \"SELECT address FROM emails WHERE address LIKE '%' || ? || '%' AND type = 'primary' ORDER BY address LIMIT 5\"\nexpand = \"SELECT p.address FROM emails AS p JOIN emails AS l ON p.name = l.name WHERE p.type = 'primary' AND l.address = ? AND l.type = 'list' ORDER BY p.address LIMIT 50\"\ndomains = \"SELECT 1 FROM emails WHERE address LIKE '%@' || ? LIMIT 1\"\n\n[oauth]\nkey = \"parerga_und_paralipomena\"\n[oauth.auth]\nmax-attempts = 1\n\n[oauth.expiry]\nuser-code = \"1s\"\ntoken = \"1s\"\nrefresh-token = \"3s\"\nrefresh-token-renew = \"2s\"\n\n[tracer.console]\ntype = \"console\"\nlevel = \"{LEVEL}\"\nmultiline = false\nansi = true\ndisabled-events = [\"network.*\"]\n\n\"#;\n"
  },
  {
    "path": "tests/src/imap/pop.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{jmap::mail::delivery::SmtpConnection, smtp::session::VerifyResponse};\nuse mail_send::smtp::tls::build_tls_connector;\nuse rustls_pki_types::ServerName;\nuse std::time::Duration;\nuse tokio::{\n    io::{AsyncBufReadExt, AsyncWriteExt, BufReader, Lines, ReadHalf, WriteHalf},\n    net::TcpStream,\n};\nuse tokio_rustls::client::TlsStream;\n\npub async fn test() {\n    println!(\"Running POP3 tests...\");\n\n    // Send 3 test emails\n    for i in 0..3 {\n        let mut lmtp = SmtpConnection::connect_port(11201).await;\n        lmtp.ingest(\n            \"bill@example.com\",\n            &[\"popper@example.com\"],\n            &format!(\n                concat!(\n                    \"From: bill@example.com\\r\\n\",\n                    \"To: popper@example.com\\r\\n\",\n                    \"Subject: TPS Report {}\\r\\n\",\n                    \"X-Spam-Status: No\\r\\n\",\n                    \"\\r\\n\",\n                    \"I'm going to need those TPS {} reports ASAP.\\r\\n\",\n                    \"..\\r\\n\",\n                    \"So, if you could do that, that'd be great.\"\n                ),\n                i, i\n            ),\n        )\n        .await;\n    }\n\n    // Connect to POP3\n    let mut pop3 = Pop3Connection::connect().await;\n    pop3.assert_read(ResponseType::Ok).await;\n\n    // Capabilities\n    pop3.send(\"CAPA\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"SASL PLAIN\")\n        .assert_contains(\"IMPLEMENTATION\");\n\n    // Noop\n    pop3.send(\"NOOP\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n\n    // Authenticate user/pass\n    pop3.send(\"PASS secret\").await;\n    pop3.assert_read(ResponseType::Err).await;\n    pop3.send(\"USER popper@example.com\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"PASS wrong_secret\").await;\n    pop3.assert_read(ResponseType::Err).await;\n    pop3.send(\"USER popper@example.com\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"PASS secret\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"QUIT\").await;\n\n    // Authenticate using AUTH PLAIN\n    let mut pop3 = Pop3Connection::connect().await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"AUTH PLAIN AHBvcHBlckBleGFtcGxlLmNvbQBzZWNyZXQ=\")\n        .await;\n    pop3.assert_read(ResponseType::Ok).await;\n\n    // STAT\n    pop3.send(\"STAT\").await;\n    pop3.assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"+OK 3 603\");\n\n    // UTF8\n    pop3.send(\"UTF8\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n\n    // LIST\n    pop3.send(\"LIST\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"+OK 3 messages\")\n        .assert_contains(\"1 201\")\n        .assert_contains(\"2 201\")\n        .assert_contains(\"3 201\");\n    pop3.send(\"LIST 2\").await;\n    pop3.assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"+OK 2 201\");\n\n    // UIDL\n    pop3.send(\"UIDL\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"+OK 3 messages\")\n        .assert_contains(\"1 \")\n        .assert_contains(\"2 \")\n        .assert_contains(\"3 \");\n    pop3.send(\"UIDL 2\").await;\n    pop3.assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"+OK 2 \");\n\n    // RETR\n    pop3.send(\"RETR 1\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"+OK 201 octets\")\n        .assert_contains(\"I'm going to need those TPS 0 reports ASAP.\")\n        .assert_contains(\"So, if you could do that, that'd be great.\");\n    pop3.send(\"RETR 3\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"+OK 201 octets\")\n        .assert_contains(\"I'm going to need those TPS 2 reports ASAP.\")\n        .assert_contains(\"So, if you could do that, that'd be great.\");\n    pop3.send(\"RETR 4\").await;\n    pop3.assert_read(ResponseType::Err).await;\n\n    // TOP\n    pop3.send(\"TOP 1 4\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"+OK 201 octets\")\n        .assert_contains(\"Subject: TPS Report 0\")\n        .assert_not_contains(\"I'm going to need those TPS 0 reports ASAP.\");\n    pop3.send(\"TOP 3 4\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"+OK 201 octets\")\n        .assert_contains(\"Subject: TPS Report 2\")\n        .assert_not_contains(\"I'm going to need those TPS 2 reports ASAP.\");\n\n    // DELE + RSET + QUIT (should not delete messages)\n    pop3.send(\"DELE 1\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"DELE 4\").await;\n    pop3.assert_read(ResponseType::Err).await;\n    pop3.send(\"RSET\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"QUIT\").await;\n    let mut pop3 = Pop3Connection::connect_and_login().await;\n    pop3.send(\"STAT\").await;\n    pop3.assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"+OK 3 603\");\n\n    // DELE + QUIT (should delete messages)\n    pop3.send(\"DELE 2\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"QUIT\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    let mut pop3 = Pop3Connection::connect_and_login().await;\n    pop3.send(\"STAT\").await;\n    pop3.assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"+OK 2 402\");\n    pop3.send(\"TOP 1 4\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"TPS Report 0\");\n    pop3.send(\"TOP 2 4\").await;\n    pop3.assert_read(ResponseType::Multiline)\n        .await\n        .assert_contains(\"TPS Report 2\");\n\n    // DELE using pipelining\n    pop3.send(\"DELE 1\\r\\nDELE 2\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.assert_read(ResponseType::Ok).await;\n    pop3.send(\"QUIT\").await;\n    pop3.assert_read(ResponseType::Ok).await;\n    let mut pop3 = Pop3Connection::connect_and_login().await;\n    pop3.send(\"STAT\").await;\n    pop3.assert_read(ResponseType::Ok)\n        .await\n        .assert_contains(\"+OK 0 0\");\n    pop3.send(\"QUIT\").await;\n}\n\n#[derive(Debug, Clone, PartialEq, Eq)]\npub enum ResponseType {\n    Ok,\n    Multiline,\n    Err,\n}\n\npub struct Pop3Connection {\n    reader: Lines<BufReader<ReadHalf<TlsStream<TcpStream>>>>,\n    writer: WriteHalf<TlsStream<TcpStream>>,\n}\n\nimpl Pop3Connection {\n    pub async fn connect() -> Self {\n        let (reader, writer) = tokio::io::split(\n            build_tls_connector(true)\n                .connect(\n                    ServerName::try_from(\"pop3.example.org\").unwrap().to_owned(),\n                    TcpStream::connect(\"127.0.0.1:4110\").await.unwrap(),\n                )\n                .await\n                .unwrap(),\n        );\n        Pop3Connection {\n            reader: BufReader::new(reader).lines(),\n            writer,\n        }\n    }\n\n    pub async fn connect_and_login() -> Self {\n        let mut pop3 = Self::connect().await;\n        pop3.assert_read(ResponseType::Ok).await;\n        pop3.send(\"AUTH PLAIN AHBvcHBlckBleGFtcGxlLmNvbQBzZWNyZXQ=\")\n            .await;\n        pop3.assert_read(ResponseType::Ok).await;\n        pop3\n    }\n\n    pub async fn assert_read(&mut self, rt: ResponseType) -> Vec<String> {\n        let lines = self.read(matches!(rt, ResponseType::Multiline)).await;\n        if lines.last().unwrap().starts_with(match rt {\n            ResponseType::Ok => \"+OK\",\n            ResponseType::Multiline => \".\",\n            ResponseType::Err => \"-ERR\",\n        }) {\n            lines\n        } else {\n            panic!(\"Expected {:?} from server but got: {:?}\", rt, lines);\n        }\n    }\n\n    pub async fn read(&mut self, is_multiline: bool) -> Vec<String> {\n        let mut lines = Vec::new();\n        loop {\n            match tokio::time::timeout(Duration::from_millis(1500), self.reader.next_line()).await {\n                Ok(Ok(Some(line))) => {\n                    let is_done = (!is_multiline && line.starts_with(\"+OK\"))\n                        || (is_multiline && line == \".\")\n                        || line.starts_with(\"-ERR\");\n                    //let c = println!(\"<- {:?}\", line);\n                    lines.push(line);\n                    if is_done {\n                        return lines;\n                    }\n                }\n                Ok(Ok(None)) => {\n                    panic!(\"Invalid response: {:?}.\", lines);\n                }\n                Ok(Err(err)) => {\n                    panic!(\"Connection broken: {} ({:?})\", err, lines);\n                }\n                Err(_) => panic!(\"Timeout while waiting for server response: {:?}\", lines),\n            }\n        }\n    }\n\n    pub async fn send(&mut self, text: &str) {\n        //let c = println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n        self.writer.write_all(b\"\\r\\n\").await.unwrap();\n    }\n\n    pub async fn send_raw(&mut self, text: &str) {\n        //let c = println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n    }\n}\n"
  },
  {
    "path": "tests/src/imap/search.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{AssertResult, ImapConnection, Type};\nuse crate::imap::IMAPTest;\nuse imap_proto::ResponseType;\n\npub async fn test(imap: &mut ImapConnection, imap_check: &mut ImapConnection, handle: &IMAPTest) {\n    println!(\"Running SEARCH tests...\");\n\n    // Searches without selecting a mailbox should fail.\n    imap.send(\"SEARCH RETURN (MIN MAX COUNT ALL) ALL\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Bad).await;\n\n    // Select INBOX\n    imap.send(\"SELECT INBOX\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"10 EXISTS\")\n        .assert_contains(\"[UIDNEXT 11]\");\n    imap_check.send(\"SELECT INBOX\").await;\n    imap_check.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Min, Max and Count\n    imap.send(\"SEARCH RETURN (MIN MAX COUNT ALL) ALL\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"COUNT 10 MIN 1 MAX 10 ALL 1,10\");\n    imap_check.send(\"UID SEARCH ALL\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 1 2 3 4 5 6 7 8 9 10\");\n\n    // Filters\n    imap_check\n        .send(\"UID SEARCH OR FROM nathaniel SUBJECT argentina\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 1 3 4 6\");\n\n    imap_check\n        .send(\"UID SEARCH UNSEEN OR KEYWORD Flag_007 KEYWORD Flag_004\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 5 8\");\n\n    imap_check\n        .send(\"UID SEARCH TEXT coffee FROM vandelay SUBJECT exporting SENTON 20-Nov-2021\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 10\");\n\n    imap_check\n        .send(\"UID SEARCH NOT (FROM nathaniel ANSWERED)\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 2 3 5 7 8 9 10\");\n\n    imap_check\n        .send(\"UID SEARCH UID 0:6 LARGER 1000 SMALLER 2000\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 1 2\");\n\n    // Saved search\n    imap_check.send(\n        \"UID SEARCH RETURN (SAVE ALL) OR OR FROM nathaniel FROM vandelay OR SUBJECT rfc FROM gore\",\n    )\n    .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"1,3:4,6,8,10\");\n\n    imap_check.send(\"UID SEARCH NOT $\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 2 5 7 9\");\n\n    imap_check\n        .send(\"UID SEARCH $ SMALLER 1000 SUBJECT section\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SEARCH 8\");\n\n    imap_check.send(\"UID SEARCH RETURN (MIN MAX) NOT $\").await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MIN 2 MAX 9\");\n\n    // Sort\n    imap_check\n        .send(\"UID SORT (REVERSE SUBJECT REVERSE DATE) UTF-8 FROM Nathaniel\")\n        .await;\n    imap_check\n        .assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_equals(\"* SORT 6 4 1\");\n\n    imap.send(\"UID SORT RETURN (COUNT ALL) (DATE SUBJECT) UTF-8 ALL\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(if !handle.server.search_store().is_mysql() {\n            \"COUNT 10 ALL 6,4:5,1,10,3,7:8,2,9\"\n        } else {\n            \"COUNT 10 ALL 9,3,7:8,2,6,4:5,1,10\"\n        }); //6,4:5,1,10,9,3,7:8,2\");\n}\n"
  },
  {
    "path": "tests/src/imap/store.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap_proto::ResponseType;\n\nuse crate::jmap::wait_for_index;\n\nuse super::{AssertResult, IMAPTest, ImapConnection, Type};\n\npub async fn test(imap: &mut ImapConnection, _imap_check: &mut ImapConnection, handle: &IMAPTest) {\n    println!(\"Running STORE tests...\");\n\n    // Select INBOX\n    imap.send(\"SELECT INBOX\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"10 EXISTS\")\n        .assert_contains(\"[UIDNEXT 11]\");\n\n    // Set all messages to flag \"Seen\"\n    imap.send(\"UID STORE 1:10 +FLAGS.SILENT (\\\\Seen)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"FLAGS\", 0);\n\n    // Check that the flags were set\n    imap.send(\"UID FETCH 1:* (Flags)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"\\\\Seen\", 10);\n\n    // Check status\n    imap.send(\"STATUS INBOX (UIDNEXT MESSAGES UNSEEN)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 10\")\n        .assert_contains(\"UNSEEN 0\")\n        .assert_contains(\"UIDNEXT 11\");\n\n    // Remove Seen flag from all messages\n    imap.send(\"UID STORE 1:10 -FLAGS (\\\\Seen)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"FLAGS\", 10)\n        .assert_count(\"Seen\", 0);\n\n    // Check that the flags were removed\n    imap.send(\"UID FETCH 1:* (Flags)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"\\\\Seen\", 0);\n    imap.send(\"STATUS INBOX (UIDNEXT MESSAGES UNSEEN)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 10\")\n        .assert_contains(\"UNSEEN 10\")\n        .assert_contains(\"UIDNEXT 11\");\n\n    // Store using saved searches\n    wait_for_index(&handle.server).await;\n    imap.send(\"SEARCH RETURN (SAVE) FROM nathaniel\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"UID STORE $ +FLAGS (\\\\Answered)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"FLAGS\", 3);\n\n    // Remove Answered flag\n    imap.send(\"UID STORE 1:* -FLAGS (\\\\Answered)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"FLAGS\", 3)\n        .assert_count(\"Answered\", 0);\n}\n"
  },
  {
    "path": "tests/src/imap/thread.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse imap_proto::ResponseType;\n\nuse crate::imap::{AssertResult, IMAPTest, expand_uid_list};\n\nuse super::{ImapConnection, Type, append::build_messages};\n\npub async fn test(imap: &mut ImapConnection, _imap_check: &mut ImapConnection, handle: &IMAPTest) {\n    println!(\"Running THREAD tests...\");\n\n    // Create test messages\n    let messages = build_messages();\n\n    // Insert messages using Multiappend\n    imap.send(\"CREATE Manchego\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    for (pos, message) in messages.iter().enumerate() {\n        if pos == 0 {\n            imap.send(&format!(\"APPEND Manchego {{{}}}\", message.len()))\n                .await;\n        } else {\n            imap.send_untagged(&format!(\" {{{}}}\", message.len())).await;\n        }\n        imap.assert_read(Type::Continuation, ResponseType::Ok).await;\n        if pos < messages.len() - 1 {\n            imap.send_raw(message).await;\n        } else {\n            imap.send_untagged(message).await;\n            assert_eq!(\n                expand_uid_list(\n                    &imap\n                        .assert_read(Type::Tagged, ResponseType::Ok)\n                        .await\n                        .into_append_uid()\n                )\n                .len(),\n                messages.len(),\n            );\n        }\n    }\n\n    // Obtain ThreadId and MessageId of the first message\n    imap.send(\"SELECT Manchego\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    let mut email_id = None;\n    let mut thread_id = None;\n    imap.send(\"UID FETCH 1 (EMAILID THREADID)\").await;\n    for line in imap.assert_read(Type::Tagged, ResponseType::Ok).await {\n        if let Some((_, value)) = line.split_once(\"EMAILID (\") {\n            email_id = value\n                .split_once(')')\n                .expect(\"Missing delimiter\")\n                .0\n                .to_string()\n                .into();\n        }\n        if let Some((_, value)) = line.split_once(\"THREADID (\") {\n            thread_id = value\n                .split_once(')')\n                .expect(\"Missing delimiter\")\n                .0\n                .to_string()\n                .into();\n        }\n    }\n    let email_id = email_id.expect(\"Missing EMAILID\");\n    let thread_id = thread_id.expect(\"Missing THREADID\");\n\n    // 4 different threads are expected\n    imap.send(\"THREAD REFERENCES UTF-8 1:*\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"(1 2 3 4)\")\n        .assert_contains(\"(5 6 7 8)\")\n        .assert_contains(\"(9 10 11 12)\");\n\n    // Filter by subject (mySQL does not support searching for short keywords)\n    if !handle.server.search_store().is_mysql() {\n        imap.send(\"THREAD REFERENCES UTF-8 SUBJECT T1\").await;\n        imap.assert_read(Type::Tagged, ResponseType::Ok)\n            .await\n            .assert_contains(\"(5 6 7 8)\")\n            .assert_count(\"(1 2 3 4)\", 0)\n            .assert_count(\"(9 10 11 12)\", 0);\n    }\n\n    // Filter by threadId and messageId\n    imap.send(&format!(\n        \"UID THREAD REFERENCES UTF-8 THREADID {}\",\n        thread_id\n    ))\n    .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"(1 2 3 4)\")\n        .assert_count(\"(\", 1);\n\n    imap.send(&format!(\"UID THREAD REFERENCES UTF-8 EMAILID {}\", email_id))\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"(1)\")\n        .assert_count(\"(\", 1);\n\n    // Delete all messages\n    imap.send(\"STORE 1:* +FLAGS.SILENT (\\\\Deleted)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"EXPUNGE\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_count(\"EXPUNGE\", 13);\n}\n"
  },
  {
    "path": "tests/src/jmap/auth/limits.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    directory::internal::TestInternalDirectory,\n    imap::{ImapConnection, Type},\n    jmap::{JMAPTest},\n};\nuse common::listener::blocked::BLOCKED_IP_KEY;\nuse directory::Permission;\nuse imap_proto::ResponseType;\nuse jmap_client::{\n    client::{Client, Credentials},\n    core::set::{SetError, SetErrorType},\n    mailbox::{self},\n};\nuse std::{\n    net::{IpAddr, Ipv4Addr},\n    sync::Arc,\n    time::Duration,\n};\nuse store::write::now;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Authorization tests...\");\n\n    // Create test account\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n\n    // Remove unlimited requests permission\n    params\n        .server\n        .store()\n        .remove_permissions(account.name(), [Permission::UnlimitedRequests])\n        .await;\n    params.server.inner.cache.access_tokens.clear();\n\n    // Reset rate limiters\n    params.webhook.clear();\n\n    // Incorrect passwords should be rejected with a 401 error\n    assert!(matches!(\n        Client::new()\n            .credentials(Credentials::basic(\"jdoe@example.com\", \"abcde\"))\n            .accept_invalid_certs(true) .follow_redirects([\"127.0.0.1\"])\n            .connect(\"https://127.0.0.1:8899\")\n            .await,\n        Err(jmap_client::Error::Problem(err)) if err.status() == Some(401)));\n\n    // Wait until the beginning of the 5 seconds bucket\n    const LIMIT: u64 = 5;\n    let now = now();\n    let range_start = now / LIMIT;\n    let range_end = (range_start * LIMIT) + LIMIT;\n    tokio::time::sleep(Duration::from_secs(range_end - now)).await;\n\n    // Test fail2ban\n    assert_eq!(\n        server\n            .core\n            .storage\n            .config\n            .get(format!(\"{BLOCKED_IP_KEY}.127.0.0.1\"))\n            .await\n            .unwrap(),\n        None\n    );\n    for n in 0..98 {\n        match Client::new()\n            .credentials(Credentials::basic(\n                \"not_an_account@example.com\",\n                &format!(\"brute_force{}\", n),\n            ))\n            .accept_invalid_certs(true)\n            .follow_redirects([\"127.0.0.1\"])\n            .connect(\"https://127.0.0.1:8899\")\n            .await\n        {\n            Err(jmap_client::Error::Problem(_)) => {}\n            Err(err) => {\n                panic!(\"Unexpected response: {:?}\", err);\n            }\n            Ok(_) => {\n                panic!(\"Unexpected success\");\n            }\n        }\n    }\n\n    let mut imap = ImapConnection::connect(b\"_x \").await;\n    imap.send(\"AUTHENTICATE PLAIN AGpvaG4AY2hpbWljaGFuZ2Fz\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::No).await;\n\n    // There are already 100 failed login attempts for this IP address\n    // so the next one should be rejected, even if done over IMAP\n    imap.send(\"AUTHENTICATE PLAIN AGpvaG4AY2hpbWljaGFuZ2Fz\")\n        .await;\n    imap.assert_disconnect().await;\n\n    // Make sure the IP address is blocked\n    assert_eq!(\n        server\n            .core\n            .storage\n            .config\n            .get(format!(\"{BLOCKED_IP_KEY}.127.0.0.1\"))\n            .await\n            .unwrap(),\n        Some(String::new())\n    );\n    ImapConnection::connect(b\"_y \")\n        .await\n        .assert_disconnect()\n        .await;\n\n    // Lift ban\n    server\n        .core\n        .storage\n        .config\n        .clear(format!(\"{BLOCKED_IP_KEY}.127.0.0.1\"))\n        .await\n        .unwrap();\n    server\n        .inner\n        .data\n        .blocked_ips\n        .write()\n        .remove(&IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)));\n\n    // Valid authentication requests should not be rate limited\n    for _ in 0..110 {\n        Client::new()\n            .credentials(Credentials::basic(account.name(), account.secret()))\n            .accept_invalid_certs(true)\n            .follow_redirects([\"127.0.0.1\"])\n            .connect(\"https://127.0.0.1:8899\")\n            .await\n            .unwrap();\n    }\n\n    // Login with the correct credentials\n    let client = Client::new()\n        .credentials(Credentials::basic(account.name(), account.secret()))\n        .accept_invalid_certs(true)\n        .follow_redirects([\"127.0.0.1\"])\n        .connect(\"https://127.0.0.1:8899\")\n        .await\n        .unwrap();\n    assert_eq!(client.session().username(), account.name());\n    assert_eq!(\n        client\n            .session()\n            .account(account.id_string())\n            .unwrap()\n            .name(),\n        account.name()\n    );\n    assert!(\n        client\n            .session()\n            .account(account.id_string())\n            .unwrap()\n            .is_personal()\n    );\n\n    // Uploads up to 5000000 bytes should be allowed\n    assert_eq!(\n        client\n            .upload(None, vec![b'A'; 5000000], None)\n            .await\n            .unwrap()\n            .size(),\n        5000000\n    );\n    assert!(\n        client\n            .upload(None, vec![b'A'; 5000001], None)\n            .await\n            .is_err()\n    );\n\n    // Users should be allowed to create identities only\n    // using email addresses associated to their principal\n    let iid1 = client\n        .identity_create(\"John Doe\", \"jdoe@example.com\")\n        .await\n        .unwrap()\n        .take_id();\n    let iid2 = client\n        .identity_create(\"John Doe (secondary)\", \"john.doe@example.com\")\n        .await\n        .unwrap()\n        .take_id();\n    assert!(matches!(\n        client\n            .identity_create(\"John the Spammer\", \"spammy@mcspamface.com\")\n            .await,\n        Err(jmap_client::Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n    client.identity_destroy(&iid1).await.unwrap();\n    client.identity_destroy(&iid2).await.unwrap();\n\n    // Concurrent requests check\n    let client = Arc::new(client);\n    for _ in 0..8 {\n        let client_ = client.clone();\n        tokio::spawn(async move {\n            let _ = client_\n                .mailbox_query(\n                    mailbox::query::Filter::name(\"__sleep\").into(),\n                    [mailbox::query::Comparator::name()].into(),\n                )\n                .await;\n        });\n    }\n    tokio::time::sleep(Duration::from_millis(500)).await;\n    assert!(matches!(\n        client\n            .mailbox_query(\n                mailbox::query::Filter::name(\"__sleep\").into(),\n                [mailbox::query::Comparator::name()].into(),\n            )\n            .await,\n            Err(jmap_client::Error::Problem(err)) if err.status() == Some(400)));\n\n    // Wait for sleep to be done\n    tokio::time::sleep(Duration::from_millis(1000)).await;\n\n    // Concurrent upload test\n    for _ in 0..4 {\n        let client_ = client.clone();\n        tokio::spawn(async move {\n            client_.upload(None, b\"sleep\".to_vec(), None).await.unwrap();\n        });\n    }\n    tokio::time::sleep(Duration::from_millis(500)).await;\n    assert!(matches!(\n        client.upload(None, b\"sleep\".to_vec(), None).await,\n        Err(jmap_client::Error::Problem(err)) if err.status() == Some(400)));\n\n    // Add unlimited requests permission\n    params\n        .server\n        .store()\n        .add_permissions(account.name(), [Permission::UnlimitedRequests])\n        .await;\n    params.server.inner.cache.access_tokens.clear();\n\n    // Destroy test accounts\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n\n    // Check webhook events\n    params\n        .webhook\n        .assert_contains(&[\"auth.failed\", \"auth.success\", \"security.authentication-ban\"]);\n}\n"
  },
  {
    "path": "tests/src/jmap/auth/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod limits;\npub mod oauth;\npub mod permissions;\npub mod quota;\n"
  },
  {
    "path": "tests/src/jmap/auth/oauth.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    imap::{\n        ImapConnection, Type,\n        pop::{self, Pop3Connection},\n    },\n    jmap::{JMAPTest, ManagementApi, mail::delivery::SmtpConnection},\n};\nuse base64::{Engine, engine::general_purpose};\nuse biscuit::{JWT, SingleOrMultiple, jwk::JWKSet};\nuse bytes::Bytes;\nuse common::auth::oauth::{\n    introspect::OAuthIntrospect,\n    oidc::StandardClaims,\n    registration::{ClientRegistrationRequest, ClientRegistrationResponse},\n};\nuse http::auth::oauth::{\n    DeviceAuthResponse, ErrorType, OAuthCodeRequest, TokenResponse, auth::OAuthMetadata,\n    openid::OpenIdMetadata,\n};\nuse imap_proto::ResponseType;\nuse jmap_client::{\n    client::{Client, Credentials},\n    mailbox::query::Filter,\n};\nuse serde::{Serialize, de::DeserializeOwned};\nuse std::time::{Duration, Instant};\nuse store::ahash::AHashMap;\n\n#[derive(serde::Deserialize, Debug)]\n#[allow(dead_code)]\nstruct OAuthCodeResponse {\n    pub code: String,\n    #[serde(rename = \"isEnterprise\")]\n    pub is_enterprise: bool,\n}\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running OAuth tests...\");\n\n    // Create test account\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n\n    // Build API\n    let api = ManagementApi::new(8899, \"jdoe@example.com\", \"12345\");\n\n    // Obtain OAuth metadata\n    let metadata: OAuthMetadata =\n        get(\"https://127.0.0.1:8899/.well-known/oauth-authorization-server\").await;\n    let oidc_metadata: OpenIdMetadata =\n        get(\"https://127.0.0.1:8899/.well-known/openid-configuration\").await;\n    let jwk_set: JWKSet<()> = get(&oidc_metadata.jwks_uri).await;\n\n    // Register client\n    let registration: ClientRegistrationResponse = post_json(\n        &metadata.registration_endpoint,\n        None,\n        &ClientRegistrationRequest {\n            redirect_uris: vec![\"https://localhost\".to_string()],\n            ..Default::default()\n        },\n    )\n    .await;\n    let client_id = registration.client_id;\n\n    /*println!(\"OAuth metadata: {:#?}\", metadata);\n    println!(\"OpenID metadata: {:#?}\", oidc_metadata);\n    println!(\"JWKSet: {:#?}\", jwk_set);*/\n\n    // ------------------------\n    // Authorization code flow\n    // ------------------------\n\n    // Authenticate with the correct password\n    let response = api\n        .post::<OAuthCodeResponse>(\n            \"/api/oauth\",\n            &OAuthCodeRequest::Code {\n                client_id: client_id.to_string(),\n                redirect_uri: \"https://localhost\".to_string().into(),\n                nonce: \"abc1234\".to_string().into(),\n            },\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Both client_id and redirect_uri have to match\n    let mut token_params = AHashMap::from_iter([\n        (\"client_id\".to_string(), \"invalid_client\".to_string()),\n        (\"redirect_uri\".to_string(), \"https://localhost\".to_string()),\n        (\"grant_type\".to_string(), \"authorization_code\".to_string()),\n        (\"code\".to_string(), response.code),\n    ]);\n    assert_eq!(\n        post::<TokenResponse>(&metadata.token_endpoint, &token_params).await,\n        TokenResponse::Error {\n            error: ErrorType::InvalidClient\n        }\n    );\n    token_params.insert(\"client_id\".to_string(), client_id.to_string());\n    token_params.insert(\n        \"redirect_uri\".to_string(),\n        \"https://some-other.url\".to_string(),\n    );\n    assert_eq!(\n        post::<TokenResponse>(&metadata.token_endpoint, &token_params).await,\n        TokenResponse::Error {\n            error: ErrorType::InvalidClient\n        }\n    );\n\n    // Obtain token\n    token_params.insert(\"redirect_uri\".to_string(), \"https://localhost\".to_string());\n    let (token, refresh_token, id_token) =\n        unwrap_oidc_token_response(post(&metadata.token_endpoint, &token_params).await);\n\n    // Connect to account using token and attempt to search\n    let john_client = Client::new()\n        .credentials(Credentials::bearer(&token))\n        .accept_invalid_certs(true)\n        .follow_redirects([\"127.0.0.1\"])\n        .connect(\"https://127.0.0.1:8899\")\n        .await\n        .unwrap();\n    assert_eq!(john_client.default_account_id(), account.id_string());\n    assert!(\n        !john_client\n            .mailbox_query(None::<Filter>, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .ids()\n            .is_empty()\n    );\n\n    // Verify ID token using the JWK set\n    let id_token = JWT::<StandardClaims, biscuit::Empty>::new_encoded(&id_token)\n        .decode_with_jwks(&jwk_set, None)\n        .unwrap();\n    let claims = id_token.payload().unwrap();\n    let registered_claims = &claims.registered;\n    let private_claims = &claims.private;\n    assert_eq!(registered_claims.issuer, Some(oidc_metadata.issuer));\n    assert_eq!(\n        registered_claims.subject,\n        Some(account.id().document_id().to_string())\n    );\n    assert_eq!(\n        registered_claims.audience,\n        Some(SingleOrMultiple::Single(client_id.to_string()))\n    );\n    assert_eq!(private_claims.nonce, Some(\"abc1234\".into()));\n    assert_eq!(\n        private_claims.preferred_username,\n        Some(\"jdoe@example.com\".into())\n    );\n    assert_eq!(private_claims.email, Some(\"jdoe@example.com\".into()));\n\n    // Introspect token\n    let access_introspect: OAuthIntrospect = post_with_auth::<OAuthIntrospect>(\n        &metadata.introspection_endpoint,\n        token.as_str().into(),\n        &AHashMap::from_iter([(\"token\".to_string(), token.to_string())]),\n    )\n    .await;\n    assert_eq!(access_introspect.username.unwrap(), \"jdoe@example.com\");\n    assert_eq!(access_introspect.token_type.unwrap(), \"bearer\");\n    assert_eq!(access_introspect.client_id.unwrap(), client_id);\n    assert!(access_introspect.active);\n    let refresh_introspect = post_with_auth::<OAuthIntrospect>(\n        &metadata.introspection_endpoint,\n        token.as_str().into(),\n        &AHashMap::from_iter([(\"token\".to_string(), refresh_token.unwrap())]),\n    )\n    .await;\n    assert_eq!(refresh_introspect.username.unwrap(), \"jdoe@example.com\");\n    assert_eq!(refresh_introspect.client_id.unwrap(), client_id);\n    assert!(refresh_introspect.active);\n    assert_eq!(\n        refresh_introspect.iat.unwrap(),\n        access_introspect.iat.unwrap()\n    );\n\n    // Try SMTP OAUTHBEARER auth\n    let oauth_bearer_invalid_sasl = general_purpose::STANDARD.encode(format!(\n        \"n,a={},\\u{1}auth=Bearer {}\\u{1}\\u{1}\",\n        \"user@domain\", \"invalid_token\"\n    ));\n    let oauth_bearer_sasl = general_purpose::STANDARD.encode(format!(\n        \"n,a={},\\u{1}auth=Bearer {}\\u{1}\\u{1}\",\n        \"user@domain\", token\n    ));\n    let mut smtp = SmtpConnection::connect().await;\n    smtp.send(&format!(\"AUTH OAUTHBEARER {oauth_bearer_invalid_sasl}\",))\n        .await;\n    smtp.read(1, 4).await;\n    smtp.send(&format!(\"AUTH OAUTHBEARER {oauth_bearer_sasl}\",))\n        .await;\n    smtp.read(1, 2).await;\n\n    // Try IMAP OAUTHBEARER auth\n    let mut imap = ImapConnection::connect(b\"_x \").await;\n    imap.assert_read(Type::Untagged, ResponseType::Ok).await;\n    imap.send(&format!(\"AUTHENTICATE OAUTHBEARER {oauth_bearer_sasl}\"))\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Try POP3 OAUTHBEARER auth\n    let mut pop3 = Pop3Connection::connect().await;\n    pop3.assert_read(pop::ResponseType::Ok).await;\n    pop3.send(&format!(\"AUTH OAUTHBEARER {oauth_bearer_sasl}\"))\n        .await;\n    pop3.assert_read(pop::ResponseType::Ok).await;\n\n    // ------------------------\n    // Device code flow\n    // ------------------------\n\n    // Request a device code\n    let device_code_params =\n        AHashMap::from_iter([(\"client_id\".to_string(), client_id.to_string())]);\n    let device_response: DeviceAuthResponse =\n        post(&metadata.device_authorization_endpoint, &device_code_params).await;\n    //println!(\"Device response: {:#?}\", device_response);\n\n    // Status should be pending\n    let mut token_params = AHashMap::from_iter([\n        (\"client_id\".to_string(), client_id.to_string()),\n        (\n            \"grant_type\".to_string(),\n            \"urn:ietf:params:oauth:grant-type:device_code\".to_string(),\n        ),\n        (\n            \"device_code\".to_string(),\n            device_response.device_code.to_string(),\n        ),\n    ]);\n    assert_eq!(\n        post::<TokenResponse>(&metadata.token_endpoint, &token_params).await,\n        TokenResponse::Error {\n            error: ErrorType::AuthorizationPending\n        }\n    );\n\n    // Let the code expire and make sure it's invalidated\n    tokio::time::sleep(Duration::from_secs(1)).await;\n    assert!(\n        !api.post::<bool>(\n            \"/api/oauth\",\n            &OAuthCodeRequest::Device {\n                code: device_response.user_code.clone(),\n            },\n        )\n        .await\n        .unwrap()\n        .unwrap_data(),\n        \"Code should be expired\"\n    );\n    assert_eq!(\n        post::<TokenResponse>(&metadata.token_endpoint, &token_params).await,\n        TokenResponse::Error {\n            error: ErrorType::ExpiredToken\n        }\n    );\n\n    // Authenticate account using a valid code\n    let device_response: DeviceAuthResponse =\n        post(&metadata.device_authorization_endpoint, &device_code_params).await;\n    token_params.insert(\n        \"device_code\".to_string(),\n        device_response.device_code.to_string(),\n    );\n    assert!(\n        api.post::<bool>(\n            \"/api/oauth\",\n            &OAuthCodeRequest::Device {\n                code: device_response.user_code.clone(),\n            },\n        )\n        .await\n        .unwrap()\n        .unwrap_data(),\n        \"Code is invalid\"\n    );\n\n    // Obtain token\n    let time_first_token = Instant::now();\n    let (token, refresh_token, _) =\n        unwrap_token_response(post(&metadata.token_endpoint, &token_params).await);\n    let refresh_token = refresh_token.unwrap();\n\n    // Authorization codes can only be used once\n    assert_eq!(\n        post::<TokenResponse>(&metadata.token_endpoint, &token_params).await,\n        TokenResponse::Error {\n            error: ErrorType::ExpiredToken\n        }\n    );\n\n    // Connect to account using token and attempt to search\n    let john_client = Client::new()\n        .credentials(Credentials::bearer(&token))\n        .accept_invalid_certs(true)\n        .follow_redirects([\"127.0.0.1\"])\n        .connect(\"https://127.0.0.1:8899\")\n        .await\n        .unwrap();\n    assert_eq!(john_client.default_account_id(), account.id_string());\n    assert!(\n        !john_client\n            .mailbox_query(None::<Filter>, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .ids()\n            .is_empty()\n    );\n\n    // Connecting using the refresh token should not work\n    assert_unauthorized(\"https://127.0.0.1:8899\", &refresh_token).await;\n\n    // Refreshing a token using the access token should not work\n    assert_eq!(\n        post::<TokenResponse>(\n            &metadata.token_endpoint,\n            &AHashMap::from_iter([\n                (\"client_id\".to_string(), client_id.to_string()),\n                (\"grant_type\".to_string(), \"refresh_token\".to_string()),\n                (\"refresh_token\".to_string(), token),\n            ]),\n        )\n        .await,\n        TokenResponse::Error {\n            error: ErrorType::InvalidGrant\n        }\n    );\n\n    // Refreshing the access token before expiration should not include a new refresh token\n    let refresh_params = AHashMap::from_iter([\n        (\"client_id\".to_string(), client_id.to_string()),\n        (\"grant_type\".to_string(), \"refresh_token\".to_string()),\n        (\"refresh_token\".to_string(), refresh_token),\n    ]);\n    let time_before_post: Instant = Instant::now();\n    let (token, new_refresh_token, _) =\n        unwrap_token_response(post(&metadata.token_endpoint, &refresh_params).await);\n    assert_eq!(\n        new_refresh_token,\n        None,\n        \"Refreshed token in {:?}, since start {:?}\",\n        time_before_post.elapsed(),\n        time_first_token.elapsed()\n    );\n\n    // Wait 1 second and make sure the access token expired\n    tokio::time::sleep(Duration::from_secs(1)).await;\n    assert_unauthorized(\"https://127.0.0.1:8899\", &token).await;\n\n    // Wait another second for the refresh token to be about to expire\n    // and expect a new refresh token\n    tokio::time::sleep(Duration::from_secs(1)).await;\n    let (_, new_refresh_token, _) =\n        unwrap_token_response(post(&metadata.token_endpoint, &refresh_params).await);\n    //println!(\"New refresh token: {:?}\", new_refresh_token);\n    assert_ne!(new_refresh_token, None);\n\n    // Wait another second and make sure the refresh token expired\n    tokio::time::sleep(Duration::from_secs(1)).await;\n    assert_eq!(\n        post::<TokenResponse>(&metadata.token_endpoint, &refresh_params).await,\n        TokenResponse::Error {\n            error: ErrorType::InvalidGrant\n        }\n    );\n\n    // Destroy test accounts\n    server\n        .core\n        .storage\n        .lookup\n        .purge_in_memory_store()\n        .await\n        .unwrap();\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\nasync fn post_bytes(\n    url: &str,\n    auth_token: Option<&str>,\n    params: &AHashMap<String, String>,\n) -> Bytes {\n    let mut client = reqwest::Client::builder()\n        .timeout(Duration::from_millis(500))\n        .danger_accept_invalid_certs(true)\n        .build()\n        .unwrap_or_default()\n        .post(url);\n\n    if let Some(auth_token) = auth_token {\n        client = client.bearer_auth(auth_token);\n    }\n\n    client\n        .form(params)\n        .send()\n        .await\n        .unwrap()\n        .bytes()\n        .await\n        .unwrap()\n}\n\nasync fn post_json<D: DeserializeOwned>(\n    url: &str,\n    auth_token: Option<&str>,\n    body: &impl Serialize,\n) -> D {\n    let mut client = reqwest::Client::builder()\n        .timeout(Duration::from_millis(500))\n        .danger_accept_invalid_certs(true)\n        .build()\n        .unwrap_or_default()\n        .post(url);\n\n    if let Some(auth_token) = auth_token {\n        client = client.bearer_auth(auth_token);\n    }\n\n    serde_json::from_slice(\n        &client\n            .body(serde_json::to_string(body).unwrap().into_bytes())\n            .send()\n            .await\n            .unwrap()\n            .bytes()\n            .await\n            .unwrap(),\n    )\n    .unwrap()\n}\n\nasync fn post<T: DeserializeOwned>(url: &str, params: &AHashMap<String, String>) -> T {\n    post_with_auth(url, None, params).await\n}\nasync fn post_with_auth<T: DeserializeOwned>(\n    url: &str,\n    auth_token: Option<&str>,\n    params: &AHashMap<String, String>,\n) -> T {\n    serde_json::from_slice(&post_bytes(url, auth_token, params).await).unwrap()\n}\n\nasync fn get_bytes(url: &str) -> Bytes {\n    reqwest::Client::builder()\n        .timeout(Duration::from_millis(500))\n        .danger_accept_invalid_certs(true)\n        .build()\n        .unwrap_or_default()\n        .get(url)\n        .send()\n        .await\n        .unwrap()\n        .bytes()\n        .await\n        .unwrap()\n}\n\nasync fn get<T: DeserializeOwned>(url: &str) -> T {\n    serde_json::from_slice(&get_bytes(url).await).unwrap()\n}\n\nasync fn assert_unauthorized(base_url: &str, token: &str) {\n    match Client::new()\n        .credentials(Credentials::bearer(token))\n        .accept_invalid_certs(true)\n        .follow_redirects([\"127.0.0.1\"])\n        .connect(base_url)\n        .await\n    {\n        Ok(_) => panic!(\"Expected unauthorized access.\"),\n        Err(err) => {\n            let err = err.to_string();\n            assert!(err.contains(\"Unauthorized\"), \"{}\", err);\n        }\n    }\n}\n\nfn unwrap_token_response(response: TokenResponse) -> (String, Option<String>, u64) {\n    match response {\n        TokenResponse::Granted(granted) => {\n            assert_eq!(granted.token_type, \"bearer\");\n            (\n                granted.access_token,\n                granted.refresh_token,\n                granted.expires_in,\n            )\n        }\n        TokenResponse::Error { error } => panic!(\"Expected granted, got {:?}\", error),\n    }\n}\n\nfn unwrap_oidc_token_response(response: TokenResponse) -> (String, Option<String>, String) {\n    match response {\n        TokenResponse::Granted(granted) => {\n            assert_eq!(granted.token_type, \"bearer\");\n            (\n                granted.access_token,\n                granted.refresh_token,\n                granted.id_token.unwrap(),\n            )\n        }\n        TokenResponse::Error { error } => panic!(\"Expected granted, got {:?}\", error),\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/auth/permissions.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    directory::internal::TestInternalDirectory,\n    jmap::{JMAPTest, ManagementApi, server::List},\n};\nuse ahash::AHashSet;\nuse common::auth::{AccessToken, TenantInfo};\nuse directory::{\n    Permission, Type,\n    backend::internal::{PrincipalField, PrincipalSet, PrincipalUpdate, PrincipalValue},\n};\nuse email::message::delivery::{IngestMessage, IngestRecipient, LocalDeliveryStatus, MailDelivery};\nuse std::sync::Arc;\n\npub async fn test(params: &JMAPTest) {\n    println!(\"Running permissions tests...\");\n    let server = params.server.clone();\n\n    // Disable spam filtering to avoid adding extra headers\n    let old_core = params.server.core.clone();\n    let mut new_core = old_core.as_ref().clone();\n    new_core.spam.enabled = false;\n    new_core.smtp.session.data.add_delivered_to = false;\n    params.server.inner.shared_core.store(Arc::new(new_core));\n\n    // Remove unlimited requests permission\n    for &account in params.accounts.keys() {\n        params\n            .server\n            .store()\n            .remove_permissions(account, [Permission::UnlimitedRequests])\n            .await;\n    }\n\n    // Prepare management API\n    let api = ManagementApi::new(8899, \"admin\", \"secret\");\n\n    // Create a user with the default 'user' role\n    let account_id = api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Individual)\n                .with_field(PrincipalField::Name, \"role_player\")\n                .with_field(PrincipalField::Roles, vec![\"user\".to_string()])\n                .with_field(\n                    PrincipalField::DisabledPermissions,\n                    vec![Permission::Pop3Dele.name().to_string()],\n                ),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n    let revision = server\n        .get_access_token(account_id)\n        .await\n        .unwrap()\n        .validate_permissions(\n            Permission::all().filter(|p| p.is_user_permission() && *p != Permission::Pop3Dele),\n        )\n        .revision;\n\n    // Create multiple roles\n    for (role, permissions, parent_role) in &[\n        (\n            \"pop3_user\",\n            vec![Permission::Pop3Authenticate, Permission::Pop3List],\n            vec![],\n        ),\n        (\n            \"imap_user\",\n            vec![Permission::ImapAuthenticate, Permission::ImapList],\n            vec![],\n        ),\n        (\n            \"jmap_user\",\n            vec![\n                Permission::JmapEmailQuery,\n                Permission::AuthenticateOauth,\n                Permission::ManageEncryption,\n            ],\n            vec![],\n        ),\n        (\n            \"email_user\",\n            vec![Permission::EmailSend, Permission::EmailReceive],\n            vec![\"pop3_user\", \"imap_user\", \"jmap_user\"],\n        ),\n    ] {\n        api.post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Role)\n                .with_field(PrincipalField::Name, role.to_string())\n                .with_field(\n                    PrincipalField::EnabledPermissions,\n                    permissions\n                        .iter()\n                        .map(|p| p.name().to_string())\n                        .collect::<Vec<_>>(),\n                )\n                .with_field(\n                    PrincipalField::Roles,\n                    parent_role\n                        .iter()\n                        .map(|r| r.to_string())\n                        .collect::<Vec<_>>(),\n                ),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n    }\n\n    // Update email_user role\n    api.patch::<()>(\n        \"/api/principal/email_user\",\n        &vec![PrincipalUpdate::add_item(\n            PrincipalField::DisabledPermissions,\n            PrincipalValue::String(Permission::ManageEncryption.name().to_string()),\n        )],\n    )\n    .await\n    .unwrap()\n    .unwrap_data();\n\n    // Update the user role to the nested 'email_user' role\n    api.patch::<()>(\n        \"/api/principal/role_player\",\n        &vec![PrincipalUpdate::set(\n            PrincipalField::Roles,\n            PrincipalValue::StringList(vec![\"email_user\".to_string()]),\n        )],\n    )\n    .await\n    .unwrap()\n    .unwrap_data();\n    assert_ne!(\n        server\n            .get_access_token(account_id)\n            .await\n            .unwrap()\n            .validate_permissions([\n                Permission::EmailSend,\n                Permission::EmailReceive,\n                Permission::JmapEmailQuery,\n                Permission::AuthenticateOauth,\n                Permission::ImapAuthenticate,\n                Permission::ImapList,\n                Permission::Pop3Authenticate,\n                Permission::Pop3List,\n            ])\n            .revision,\n        revision\n    );\n\n    // Query all principals\n    api.get::<List<PrincipalSet>>(\"/api/principal\")\n        .await\n        .unwrap()\n        .unwrap_data()\n        .assert_count(12)\n        .assert_exists(\n            \"admin\",\n            Type::Individual,\n            [\n                (PrincipalField::Roles, &[\"admin\"][..]),\n                (PrincipalField::Members, &[][..]),\n                (PrincipalField::EnabledPermissions, &[][..]),\n                (PrincipalField::DisabledPermissions, &[][..]),\n            ],\n        )\n        .assert_exists(\n            \"role_player\",\n            Type::Individual,\n            [\n                (PrincipalField::Roles, &[\"email_user\"][..]),\n                (PrincipalField::Members, &[][..]),\n                (PrincipalField::EnabledPermissions, &[][..]),\n                (\n                    PrincipalField::DisabledPermissions,\n                    &[Permission::Pop3Dele.name()][..],\n                ),\n            ],\n        )\n        .assert_exists(\n            \"email_user\",\n            Type::Role,\n            [\n                (\n                    PrincipalField::Roles,\n                    &[\"pop3_user\", \"imap_user\", \"jmap_user\"][..],\n                ),\n                (PrincipalField::Members, &[\"role_player\"][..]),\n                (\n                    PrincipalField::EnabledPermissions,\n                    &[\n                        Permission::EmailReceive.name(),\n                        Permission::EmailSend.name(),\n                    ][..],\n                ),\n                (\n                    PrincipalField::DisabledPermissions,\n                    &[Permission::ManageEncryption.name()][..],\n                ),\n            ],\n        )\n        .assert_exists(\n            \"pop3_user\",\n            Type::Role,\n            [\n                (PrincipalField::Roles, &[][..]),\n                (PrincipalField::Members, &[\"email_user\"][..]),\n                (\n                    PrincipalField::EnabledPermissions,\n                    &[\n                        Permission::Pop3Authenticate.name(),\n                        Permission::Pop3List.name(),\n                    ][..],\n                ),\n                (PrincipalField::DisabledPermissions, &[][..]),\n            ],\n        )\n        .assert_exists(\n            \"imap_user\",\n            Type::Role,\n            [\n                (PrincipalField::Roles, &[][..]),\n                (PrincipalField::Members, &[\"email_user\"][..]),\n                (\n                    PrincipalField::EnabledPermissions,\n                    &[\n                        Permission::ImapAuthenticate.name(),\n                        Permission::ImapList.name(),\n                    ][..],\n                ),\n                (PrincipalField::DisabledPermissions, &[][..]),\n            ],\n        )\n        .assert_exists(\n            \"jmap_user\",\n            Type::Role,\n            [\n                (PrincipalField::Roles, &[][..]),\n                (PrincipalField::Members, &[\"email_user\"][..]),\n                (\n                    PrincipalField::EnabledPermissions,\n                    &[\n                        Permission::JmapEmailQuery.name(),\n                        Permission::AuthenticateOauth.name(),\n                        Permission::ManageEncryption.name(),\n                    ][..],\n                ),\n                (PrincipalField::DisabledPermissions, &[][..]),\n            ],\n        );\n\n    // Create new tenants\n    let tenant_id = api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Tenant)\n                .with_field(PrincipalField::Name, \"foobar\")\n                .with_field(\n                    PrincipalField::Roles,\n                    vec![\"tenant-admin\".to_string(), \"user\".to_string()],\n                )\n                .with_field(\n                    PrincipalField::Quota,\n                    PrincipalValue::IntegerList(vec![TENANT_QUOTA, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]),\n                ),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n    let other_tenant_id = api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Tenant)\n                .with_field(PrincipalField::Name, \"xanadu\")\n                .with_field(\n                    PrincipalField::Roles,\n                    vec![\"tenant-admin\".to_string(), \"user\".to_string()],\n                ),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Creating a tenant without a valid domain should fail\n    api.post::<u32>(\n        \"/api/principal\",\n        &PrincipalSet::new(u32::MAX, Type::Individual)\n            .with_field(PrincipalField::Name, \"admin-foobar\")\n            .with_field(PrincipalField::Roles, vec![\"tenant-admin\".to_string()])\n            .with_field(\n                PrincipalField::Secrets,\n                PrincipalValue::String(\"mytenantpass\".to_string()),\n            )\n            .with_field(\n                PrincipalField::Tenant,\n                PrincipalValue::String(\"foobar\".to_string()),\n            ),\n    )\n    .await\n    .unwrap()\n    .expect_error(\"Principal name must include a valid domain assigned to the tenant\");\n\n    // Create domain for the tenant and one outside the tenant\n    api.post::<u32>(\n        \"/api/principal\",\n        &PrincipalSet::new(u32::MAX, Type::Domain)\n            .with_field(PrincipalField::Name, \"foobar.org\")\n            .with_field(\n                PrincipalField::Tenant,\n                PrincipalValue::String(\"foobar\".to_string()),\n            ),\n    )\n    .await\n    .unwrap()\n    .unwrap_data();\n    api.post::<u32>(\n        \"/api/principal\",\n        &PrincipalSet::new(u32::MAX, Type::Domain).with_field(PrincipalField::Name, \"example.org\"),\n    )\n    .await\n    .unwrap()\n    .unwrap_data();\n\n    // Create tenant admin\n    let tenant_admin_id = api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Individual)\n                .with_field(PrincipalField::Name, \"admin@foobar.org\")\n                .with_field(PrincipalField::Roles, vec![\"tenant-admin\".to_string()])\n                .with_field(\n                    PrincipalField::Secrets,\n                    PrincipalValue::String(\"mytenantpass\".to_string()),\n                )\n                .with_field(\n                    PrincipalField::Tenant,\n                    PrincipalValue::String(\"foobar\".to_string()),\n                ),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Verify permissions\n    server\n        .get_access_token(tenant_admin_id)\n        .await\n        .unwrap()\n        .validate_permissions(Permission::all().filter(|p| p.is_tenant_admin_permission()))\n        .validate_tenant(tenant_id, TENANT_QUOTA);\n\n    // Prepare tenant admin API\n    let tenant_api = ManagementApi::new(8899, \"admin@foobar.org\", \"mytenantpass\");\n\n    // Tenant should not be able to create other tenants or modify its tenant id\n    tenant_api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Tenant)\n                .with_field(PrincipalField::Name, \"subfoobar\"),\n        )\n        .await\n        .unwrap()\n        .expect_request_error(\"Forbidden\");\n    tenant_api\n        .patch::<()>(\n            \"/api/principal/foobar\",\n            &vec![PrincipalUpdate::set(\n                PrincipalField::Tenant,\n                PrincipalValue::String(\"subfoobar\".to_string()),\n            )],\n        )\n        .await\n        .unwrap()\n        .expect_request_error(\"Forbidden\");\n    tenant_api\n        .get::<()>(\"/api/principal/foobar\")\n        .await\n        .unwrap()\n        .expect_request_error(\"Forbidden\");\n    tenant_api\n        .get::<()>(\"/api/principal?type=tenant\")\n        .await\n        .unwrap()\n        .expect_request_error(\"Forbidden\");\n\n    // Create a second domain for the tenant\n    tenant_api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Domain)\n                .with_field(PrincipalField::Name, \"foobar.com\"),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Creating a third domain should be limited by quota\n    tenant_api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Domain)\n                .with_field(PrincipalField::Name, \"foobar.net\"),\n        )\n        .await\n        .unwrap()\n        .expect_request_error(\"Tenant quota exceeded\");\n\n    // Creating a tenant user without a valid domain or with a domain outside the tenant should fail\n    for user in [\"mytenantuser\", \"john@example.org\"] {\n        tenant_api\n            .post::<u32>(\n                \"/api/principal\",\n                &PrincipalSet::new(u32::MAX, Type::Individual)\n                    .with_field(PrincipalField::Name, user.to_string())\n                    .with_field(PrincipalField::Roles, vec![\"tenant-admin\".to_string()]),\n            )\n            .await\n            .unwrap()\n            .expect_error(\"Principal name must include a valid domain assigned to the tenant\");\n    }\n\n    // Create an account\n    let tenant_user_id = tenant_api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Individual)\n                .with_field(PrincipalField::Name, \"john@foobar.org\")\n                .with_field(\n                    PrincipalField::Roles,\n                    vec![\"tenant-admin\".to_string(), \"user\".to_string()],\n                )\n                .with_field(\n                    PrincipalField::Secrets,\n                    PrincipalValue::String(\"tenantpass\".to_string()),\n                )\n                .with_field(\n                    PrincipalField::Tenant,\n                    PrincipalValue::String(\"xanadu\".to_string()),\n                ),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Although super user privileges were used and a different tenant name was provided, this should be ignored\n    server\n        .get_access_token(tenant_user_id)\n        .await\n        .unwrap()\n        .validate_permissions(\n            Permission::all().filter(|p| p.is_tenant_admin_permission() || p.is_user_permission()),\n        )\n        .validate_tenant(tenant_id, TENANT_QUOTA);\n\n    // Create a second account should be limited by quota\n    tenant_api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Individual)\n                .with_field(PrincipalField::Name, \"jane@foobar.org\")\n                .with_field(PrincipalField::Roles, vec![\"tenant-admin\".to_string()]),\n        )\n        .await\n        .unwrap()\n        .expect_request_error(\"Tenant quota exceeded\");\n\n    // Create an tenant role\n    tenant_api\n        .post::<u32>(\n            \"/api/principal\",\n            &PrincipalSet::new(u32::MAX, Type::Role)\n                .with_field(PrincipalField::Name, \"no-mail-for-you@foobar.com\")\n                .with_field(\n                    PrincipalField::DisabledPermissions,\n                    vec![Permission::EmailReceive.name().to_string()],\n                ),\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Assigning a role that does not belong to the tenant should fail\n    tenant_api\n        .patch::<()>(\n            \"/api/principal/john@foobar.org\",\n            &vec![PrincipalUpdate::add_item(\n                PrincipalField::Roles,\n                PrincipalValue::String(\"imap_user\".to_string()),\n            )],\n        )\n        .await\n        .unwrap()\n        .expect_error(\"notFound\");\n\n    // Add tenant defined role\n    tenant_api\n        .patch::<()>(\n            \"/api/principal/john@foobar.org\",\n            &vec![PrincipalUpdate::add_item(\n                PrincipalField::Roles,\n                PrincipalValue::String(\"no-mail-for-you@foobar.com\".to_string()),\n            )],\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Check updated permissions\n    server\n        .get_access_token(tenant_user_id)\n        .await\n        .unwrap()\n        .validate_permissions(Permission::all().filter(|p| {\n            (p.is_tenant_admin_permission() || p.is_user_permission())\n                && *p != Permission::EmailReceive\n        }));\n\n    // Changing the tenant of a user should fail\n    tenant_api\n        .patch::<()>(\n            \"/api/principal/john@foobar.org\",\n            &vec![PrincipalUpdate::set(\n                PrincipalField::Tenant,\n                PrincipalValue::String(\"xanadu\".to_string()),\n            )],\n        )\n        .await\n        .unwrap()\n        .expect_request_error(\"Forbidden\");\n\n    // Renaming a tenant account without a valid domain should fail\n    for user in [\"john\", \"john@example.org\"] {\n        tenant_api\n            .patch::<()>(\n                \"/api/principal/john@foobar.org\",\n                &vec![PrincipalUpdate::set(\n                    PrincipalField::Name,\n                    PrincipalValue::String(user.to_string()),\n                )],\n            )\n            .await\n            .unwrap()\n            .expect_error(\"Principal name must include a valid domain assigned to the tenant\");\n    }\n\n    // Rename the tenant account and add an email address\n    tenant_api\n        .patch::<()>(\n            \"/api/principal/john@foobar.org\",\n            &vec![\n                PrincipalUpdate::set(\n                    PrincipalField::Name,\n                    PrincipalValue::String(\"john.doe@foobar.org\".to_string()),\n                ),\n                PrincipalUpdate::add_item(\n                    PrincipalField::Emails,\n                    PrincipalValue::String(\"john@foobar.org\".to_string()),\n                ),\n            ],\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Tenants should only see their own principals\n    tenant_api\n        .get::<List<PrincipalSet>>(\"/api/principal?types=individual,group,role,list\")\n        .await\n        .unwrap()\n        .unwrap_data()\n        .assert_count(3)\n        .assert_exists(\n            \"admin@foobar.org\",\n            Type::Individual,\n            [\n                (PrincipalField::Roles, &[\"tenant-admin\"][..]),\n                (PrincipalField::Members, &[][..]),\n                (PrincipalField::EnabledPermissions, &[][..]),\n                (PrincipalField::DisabledPermissions, &[][..]),\n            ],\n        )\n        .assert_exists(\n            \"john.doe@foobar.org\",\n            Type::Individual,\n            [\n                (\n                    PrincipalField::Roles,\n                    &[\"tenant-admin\", \"no-mail-for-you@foobar.com\", \"user\"][..],\n                ),\n                (PrincipalField::Members, &[][..]),\n                (PrincipalField::EnabledPermissions, &[][..]),\n                (PrincipalField::DisabledPermissions, &[][..]),\n            ],\n        )\n        .assert_exists(\n            \"no-mail-for-you@foobar.com\",\n            Type::Role,\n            [\n                (PrincipalField::Roles, &[][..]),\n                (PrincipalField::Members, &[\"john.doe@foobar.org\"][..]),\n                (PrincipalField::EnabledPermissions, &[][..]),\n                (\n                    PrincipalField::DisabledPermissions,\n                    &[Permission::EmailReceive.name()][..],\n                ),\n            ],\n        );\n\n    // John should not be allowed to receive email\n    let (message_blob, _) = server\n        .put_temporary_blob(tenant_user_id, TEST_MESSAGE.as_bytes(), 60)\n        .await\n        .unwrap();\n    assert_eq!(\n        server\n            .deliver_message(IngestMessage {\n                sender_address: \"bill@foobar.org\".to_string(),\n                sender_authenticated: true,\n                recipients: vec![IngestRecipient {\n                    address: \"john@foobar.org\".to_string(),\n                    is_spam: false\n                }],\n                message_blob: message_blob.clone(),\n                message_size: TEST_MESSAGE.len() as u64,\n                session_id: 0,\n            })\n            .await\n            .status,\n        vec![LocalDeliveryStatus::PermanentFailure {\n            code: [5, 5, 0],\n            reason: \"This account is not authorized to receive email.\".into()\n        }]\n    );\n\n    // Remove the restriction\n    tenant_api\n        .patch::<()>(\n            \"/api/principal/john.doe@foobar.org\",\n            &vec![PrincipalUpdate::remove_item(\n                PrincipalField::Roles,\n                PrincipalValue::String(\"no-mail-for-you@foobar.com\".to_string()),\n            )],\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n    server\n        .get_access_token(tenant_user_id)\n        .await\n        .unwrap()\n        .validate_permissions(\n            Permission::all().filter(|p| p.is_tenant_admin_permission() || p.is_user_permission()),\n        );\n\n    // Delivery should now succeed\n    assert_eq!(\n        server\n            .deliver_message(IngestMessage {\n                sender_address: \"bill@foobar.org\".to_string(),\n                sender_authenticated: true,\n                recipients: vec![IngestRecipient {\n                    address: \"john@foobar.org\".to_string(),\n                    is_spam: false\n                }],\n                message_blob: message_blob.clone(),\n                message_size: TEST_MESSAGE.len() as u64,\n                session_id: 0,\n            })\n            .await\n            .status,\n        vec![LocalDeliveryStatus::Success]\n    );\n\n    // Quota for the tenant and user should be updated\n    const EXTRA_BYTES: i64 = 19; // Storage overhead\n    assert_eq!(\n        server.get_used_quota(tenant_id).await.unwrap(),\n        TEST_MESSAGE.len() as i64 + EXTRA_BYTES\n    );\n    assert_eq!(\n        server.get_used_quota(tenant_user_id).await.unwrap(),\n        TEST_MESSAGE.len() as i64 + EXTRA_BYTES\n    );\n\n    // Next delivery should fail due to tenant quota\n    assert_eq!(\n        server\n            .deliver_message(IngestMessage {\n                sender_address: \"bill@foobar.org\".to_string(),\n                sender_authenticated: true,\n                recipients: vec![IngestRecipient {\n                    address: \"john@foobar.org\".to_string(),\n                    is_spam: false\n                }],\n                message_blob,\n                message_size: TEST_MESSAGE.len() as u64,\n                session_id: 0,\n            })\n            .await\n            .status,\n        vec![LocalDeliveryStatus::TemporaryFailure {\n            reason: \"Organization over quota.\".into()\n        }]\n    );\n\n    // Moving a user to another tenant should move its quota too\n    api.patch::<()>(\n        \"/api/principal/john.doe@foobar.org\",\n        &vec![PrincipalUpdate::set(\n            PrincipalField::Tenant,\n            PrincipalValue::String(\"xanadu\".to_string()),\n        )],\n    )\n    .await\n    .unwrap()\n    .unwrap_data();\n\n    assert_eq!(server.get_used_quota(tenant_id).await.unwrap(), 0);\n    assert_eq!(\n        server.get_used_quota(other_tenant_id).await.unwrap(),\n        TEST_MESSAGE.len() as i64 + EXTRA_BYTES\n    );\n\n    // Deleting tenants with data should fail\n    api.delete::<()>(\"/api/principal/xanadu\")\n        .await\n        .unwrap()\n        .expect_error(\"Tenant has members\");\n\n    // Delete user\n    api.delete::<()>(\"/api/principal/john.doe@foobar.org\")\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Quota usage for tenant should be updated\n    assert_eq!(server.get_used_quota(other_tenant_id).await.unwrap(), 0);\n\n    // Delete tenant\n    api.delete::<()>(\"/api/principal/xanadu\")\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    // Delete tenant information\n    for query in [\n        \"/api/principal/no-mail-for-you@foobar.com\",\n        \"/api/principal/admin@foobar.org\",\n        \"/api/principal/foobar.org\",\n        \"/api/principal/foobar.com\",\n    ] {\n        api.delete::<()>(query).await.unwrap().unwrap_data();\n    }\n\n    // Delete tenant\n    api.delete::<()>(\"/api/principal/foobar\")\n        .await\n        .unwrap()\n        .unwrap_data();\n\n    server\n        .core\n        .storage\n        .config\n        .clear(\"report.domain\")\n        .await\n        .unwrap();\n\n    params.assert_is_empty().await;\n}\n\nconst TENANT_QUOTA: u64 = TEST_MESSAGE.len() as u64;\nconst TEST_MESSAGE: &str = concat!(\n    \"From: bill@foobar.org\\r\\n\",\n    \"To: jdoe@foobar.com\\r\\n\",\n    \"Subject: TPS Report\\r\\n\",\n    \"\\r\\n\",\n    \"I'm going to need those TPS reports ASAP. \",\n    \"So, if you could do that, that'd be great.\"\n);\n\ntrait ValidatePrincipalList {\n    fn assert_exists<'x>(\n        self,\n        name: &str,\n        typ: Type,\n        items: impl IntoIterator<Item = (PrincipalField, &'x [&'x str])>,\n    ) -> Self;\n    fn assert_count(self, count: usize) -> Self;\n}\n\nimpl ValidatePrincipalList for List<PrincipalSet> {\n    fn assert_exists<'x>(\n        self,\n        name: &str,\n        typ: Type,\n        items: impl IntoIterator<Item = (PrincipalField, &'x [&'x str])>,\n    ) -> Self {\n        for item in &self.items {\n            if item.name() == name {\n                item.validate(typ, items);\n                return self;\n            }\n        }\n\n        panic!(\"Principal not found: {}\", name);\n    }\n\n    fn assert_count(self, count: usize) -> Self {\n        assert_eq!(self.items.len(), count, \"Principal count failed validation\");\n        assert_eq!(self.total, count, \"Principal total failed validation\");\n        self\n    }\n}\n\ntrait ValidatePrincipal {\n    fn validate<'x>(\n        &self,\n        typ: Type,\n        items: impl IntoIterator<Item = (PrincipalField, &'x [&'x str])>,\n    );\n}\n\nimpl ValidatePrincipal for PrincipalSet {\n    fn validate<'x>(\n        &self,\n        typ: Type,\n        items: impl IntoIterator<Item = (PrincipalField, &'x [&'x str])>,\n    ) {\n        assert_eq!(self.typ(), typ, \"Type failed validation\");\n\n        for (field, values) in items {\n            match (\n                self.get_str_array(field).filter(|v| !v.is_empty()),\n                (!values.is_empty()).then_some(values),\n            ) {\n                (Some(values), Some(expected)) => {\n                    assert_eq!(\n                        values.iter().map(|s| s.as_str()).collect::<AHashSet<_>>(),\n                        expected.iter().copied().collect::<AHashSet<_>>(),\n                        \"Field {field:?} failed validation: {values:?} != {expected:?}\"\n                    );\n                }\n                (None, None) => {}\n                (values, expected) => {\n                    panic!(\"Field {field:?} failed validation: {values:?} != {expected:?}\");\n                }\n            }\n        }\n    }\n}\n\ntrait ValidatePermissions {\n    fn validate_permissions(\n        self,\n        expected_permissions: impl IntoIterator<Item = Permission>,\n    ) -> Self;\n    fn validate_tenant(self, tenant_id: u32, tenant_quota: u64) -> Self;\n}\n\nimpl ValidatePermissions for Arc<AccessToken> {\n    fn validate_permissions(\n        self,\n        expected_permissions: impl IntoIterator<Item = Permission>,\n    ) -> Self {\n        let expected_permissions: AHashSet<_> = expected_permissions.into_iter().collect();\n\n        let permissions = self.permissions();\n        for permission in &permissions {\n            assert!(\n                expected_permissions.contains(permission),\n                \"Permission {:?} failed validation\",\n                permission\n            );\n        }\n        assert_eq!(\n            permissions.into_iter().collect::<AHashSet<_>>(),\n            expected_permissions\n        );\n\n        for permission in Permission::all() {\n            if self.has_permission(permission) {\n                assert!(\n                    expected_permissions.contains(&permission),\n                    \"Permission {:?} failed validation\",\n                    permission\n                );\n            }\n        }\n        self\n    }\n\n    fn validate_tenant(self, tenant_id: u32, tenant_quota: u64) -> Self {\n        assert_eq!(\n            self.tenant,\n            Some(TenantInfo {\n                id: tenant_id,\n                quota: tenant_quota\n            })\n        );\n        self\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/auth/quota.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    directory::internal::TestInternalDirectory,\n    jmap::{JMAPTest, mail::delivery::SmtpConnection, wait_for_index},\n    smtp::queue::QueuedEvents,\n    store::cleanup::store_blob_expire_all,\n};\nuse common::config::smtp::queue::QueueName;\nuse email::{cache::MessageCacheFetch, mailbox::INBOX_ID};\nuse http::management::stores::recalculate_quota;\nuse jmap::blob::upload::DISABLE_UPLOAD_QUOTA;\nuse jmap_client::{\n    core::set::{SetErrorType, SetObject},\n    email::EmailBodyPart,\n};\nuse serde_json::json;\nuse smtp::queue::spool::SmtpSpool;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running quota tests...\");\n    let server = params.server.clone();\n\n    let account = params.account(\"robert@example.com\");\n    let other_account = params.account(\"jdoe@example.com\");\n\n    server\n        .core\n        .storage\n        .data\n        .set_test_quota(\"robert@example.com\", 1024)\n        .await;\n    server\n        .core\n        .storage\n        .data\n        .add_to_group(\"robert@example.com\", \"jdoe@example.com\")\n        .await;\n    server.inner.cache.access_tokens.clear();\n\n    // Delete temporary blobs from previous tests\n    store_blob_expire_all(&server.core.storage.data).await;\n\n    // Test temporary blob quota (3 files)\n    DISABLE_UPLOAD_QUOTA.store(false, std::sync::atomic::Ordering::Relaxed);\n    let client = account.client();\n    for i in 0..3 {\n        assert_eq!(\n            client\n                .upload(None, vec![b'A' + i; 1024], None)\n                .await\n                .unwrap()\n                .size(),\n            1024\n        );\n    }\n    match client\n        .upload(None, vec![b'Z'; 1024], None)\n        .await\n        .unwrap_err()\n    {\n        jmap_client::Error::Problem(err) if err.detail().unwrap().contains(\"quota\") => (),\n        other => panic!(\"Unexpected error: {:?}\", other),\n    }\n    store_blob_expire_all(&server.core.storage.data).await;\n\n    // Test temporary blob quota (50000 bytes)\n    for i in 0..2 {\n        assert_eq!(\n            client\n                .upload(None, vec![b'a' + i; 25000], None)\n                .await\n                .unwrap()\n                .size(),\n            25000\n        );\n    }\n    match client\n        .upload(None, vec![b'z'; 1024], None)\n        .await\n        .unwrap_err()\n    {\n        jmap_client::Error::Problem(err) if err.detail().unwrap().contains(\"quota\") => (),\n        other => panic!(\"Unexpected error: {:?}\", other),\n    }\n    store_blob_expire_all(&server.core.storage.data).await;\n\n    // Test JMAP Quotas extension\n    let response = account\n        .jmap_method_call(\n            \"Quota/get\",\n            json!({\n                \"ids\": null\n            }),\n        )\n        .await\n        .to_string();\n    assert!(response.contains(\"\\\"used\\\":0\"), \"{}\", response);\n    assert!(response.contains(\"\\\"hardLimit\\\":1024\"), \"{}\", response);\n    assert!(response.contains(\"\\\"scope\\\":\\\"account\\\"\"), \"{}\", response);\n    assert!(\n        response.contains(\"\\\"name\\\":\\\"robert@example.com\\\"\"),\n        \"{}\",\n        response\n    );\n\n    // Test Email/import quota\n    let inbox_id = Id::new(INBOX_ID as u64).to_string();\n    let mut message_ids = Vec::new();\n    for i in 0..2 {\n        message_ids.push(\n            client\n                .email_import(\n                    create_message_with_size(\n                        \"jdoe@example.com\",\n                        \"robert@example.com\",\n                        &format!(\"Test {i}\"),\n                        512,\n                    ),\n                    vec![&inbox_id],\n                    None::<Vec<String>>,\n                    None,\n                )\n                .await\n                .unwrap()\n                .take_id(),\n        );\n    }\n\n    assert_over_quota(\n        client\n            .email_import(\n                create_message_with_size(\"test@example.com\", \"jdoe@example.com\", \"Test 3\", 100),\n                vec![&inbox_id],\n                None::<Vec<String>>,\n                None,\n            )\n            .await,\n    );\n\n    // Test JMAP Quotas extension\n    let response = account\n        .jmap_method_call(\n            \"Quota/get\",\n            json!({\n                \"ids\": null\n            }),\n        )\n        .await\n        .to_string();\n    assert!(response.contains(\"\\\"used\\\":1024\"), \"{}\", response);\n    assert!(response.contains(\"\\\"hardLimit\\\":1024\"), \"{}\", response);\n\n    // Delete messages and check available quota\n    for message_id in message_ids {\n        client.email_destroy(&message_id).await.unwrap();\n    }\n\n    // Wait for pending index tasks\n    wait_for_index(&server).await;\n    assert_eq!(\n        server\n            .get_used_quota(account.id().document_id())\n            .await\n            .unwrap(),\n        0\n    );\n\n    // Test Email/set quota\n    let mut message_ids = Vec::new();\n    for i in 0..2 {\n        let mut request = client.build();\n        let create_item = request.set_email().create();\n        create_item\n            .mailbox_ids([&inbox_id])\n            .subject(format!(\"Test {i}\"))\n            .from([\"jdoe@example.com\"])\n            .to([\"robert@example.com\"])\n            .body_value(\"a\".to_string(), String::from_utf8(vec![b'A'; 200]).unwrap())\n            .text_body(EmailBodyPart::new().part_id(\"a\"));\n        let create_id = create_item.create_id().unwrap();\n        message_ids.push(\n            request\n                .send_set_email()\n                .await\n                .unwrap()\n                .created(&create_id)\n                .unwrap()\n                .take_id(),\n        );\n    }\n    let mut request = client.build();\n    let create_item = request.set_email().create();\n    create_item\n        .mailbox_ids([&inbox_id])\n        .subject(\"Test 3\")\n        .from([\"jdoe@example.com\"])\n        .to([\"robert@example.com\"])\n        .body_value(\"a\".to_string(), String::from_utf8(vec![b'A'; 400]).unwrap())\n        .text_body(EmailBodyPart::new().part_id(\"a\"));\n    let create_id = create_item.create_id().unwrap();\n    assert_over_quota(request.send_set_email().await.unwrap().created(&create_id));\n\n    // Recalculate quota\n    let prev_quota = server\n        .get_used_quota(account.id().document_id())\n        .await\n        .unwrap();\n    recalculate_quota(&server, account.id().document_id())\n        .await\n        .unwrap();\n    assert_eq!(\n        server\n            .get_used_quota(account.id().document_id())\n            .await\n            .unwrap(),\n        prev_quota\n    );\n\n    // Delete messages and check available quota\n    for message_id in message_ids {\n        client.email_destroy(&message_id).await.unwrap();\n    }\n    // Wait for pending index tasks\n    wait_for_index(&server).await;\n    assert_eq!(\n        server\n            .get_used_quota(account.id().document_id())\n            .await\n            .unwrap(),\n        0\n    );\n\n    // Test Email/copy quota\n    let other_client = other_account.client();\n    let mut other_message_ids = Vec::new();\n    let mut message_ids = Vec::new();\n    for i in 0..3 {\n        other_message_ids.push(\n            other_client\n                .email_import(\n                    create_message_with_size(\n                        \"jane@example.com\",\n                        \"jdoe@example.com\",\n                        &format!(\"Other Test {i}\"),\n                        512,\n                    ),\n                    vec![&inbox_id],\n                    None::<Vec<String>>,\n                    None,\n                )\n                .await\n                .unwrap()\n                .take_id(),\n        );\n    }\n    for id in other_message_ids.iter().take(2) {\n        message_ids.push(\n            client\n                .email_copy(\n                    other_account.id_string(),\n                    id,\n                    vec![&inbox_id],\n                    None::<Vec<String>>,\n                    None,\n                )\n                .await\n                .unwrap()\n                .take_id(),\n        );\n    }\n    assert_over_quota(\n        client\n            .email_copy(\n                other_account.id_string(),\n                &other_message_ids[2],\n                vec![&inbox_id],\n                None::<Vec<String>>,\n                None,\n            )\n            .await,\n    );\n\n    // Delete messages and check available quota\n    for message_id in message_ids {\n        client.email_destroy(&message_id).await.unwrap();\n    }\n    // Wait for pending index tasks\n    wait_for_index(&server).await;\n    assert_eq!(\n        server\n            .get_used_quota(account.id().document_id())\n            .await\n            .unwrap(),\n        0\n    );\n\n    // Test delivery quota\n    let mut lmtp = SmtpConnection::connect().await;\n    for i in 0..2 {\n        lmtp.ingest(\n            \"jane@example.com\",\n            &[\"robert@example.com\"],\n            &String::from_utf8(create_message_with_size(\n                \"jane@example.com\",\n                \"robert@example.com\",\n                &format!(\"Ingest test {i}\"),\n                513,\n            ))\n            .unwrap(),\n        )\n        .await;\n    }\n    let quota = server\n        .get_used_quota(account.id().document_id())\n        .await\n        .unwrap();\n    assert!(quota > 0 && quota <= 1024, \"Quota is {}\", quota);\n    assert_eq!(\n        server\n            .get_cached_messages(account.id().document_id())\n            .await\n            .unwrap()\n            .emails\n            .items\n            .len(),\n        1,\n    );\n\n    DISABLE_UPLOAD_QUOTA.store(true, std::sync::atomic::Ordering::Relaxed);\n\n    // Remove test data\n    params.destroy_all_mailboxes(account).await;\n    params.destroy_all_mailboxes(other_account).await;\n\n    for event in server.all_queued_messages().await.messages {\n        server\n            .read_message(event.queue_id, QueueName::default())\n            .await\n            .unwrap()\n            .remove(&server, event.due.into())\n            .await;\n    }\n    params.assert_is_empty().await;\n}\n\nfn assert_over_quota<T: std::fmt::Debug>(result: Result<T, jmap_client::Error>) {\n    match result {\n        Ok(result) => panic!(\"Expected error, got {:?}\", result),\n        Err(jmap_client::Error::Set(err)) if err.error() == &SetErrorType::OverQuota => (),\n        Err(err) => panic!(\"Expected OverQuota SetError, got {:?}\", err),\n    }\n}\n\nfn create_message_with_size(from: &str, to: &str, subject: &str, size: usize) -> Vec<u8> {\n    let mut message = format!(\n        \"From: {}\\r\\nTo: {}\\r\\nSubject: {}\\r\\n\\r\\n\",\n        from, to, subject\n    );\n    for _ in 0..size - message.len() {\n        message.push('A');\n    }\n\n    message.into_bytes()\n}\n"
  },
  {
    "path": "tests/src/jmap/calendar/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, JmapUtils};\nuse calcard::jscalendar::JSCalendarProperty;\nuse jmap_proto::{\n    object::{calendar::CalendarProperty, share_notification::ShareNotificationProperty},\n    request::method::MethodObject,\n};\nuse serde_json::json;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Calendar ACL tests...\");\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let john_id = john.id_string().to_string();\n    let jane_id = jane.id_string().to_string();\n\n    // Create test calendars\n    let response = john\n        .jmap_create(\n            MethodObject::Calendar,\n            [json!({\n                \"name\": \"Test #1\",\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let john_calendar_id = response.created(0).id().to_string();\n    let john_event_id = john\n        .jmap_create(\n            MethodObject::CalendarEvent,\n            [json!({\n                \"@type\": \"Event\",\n                \"uid\": \"a8df6573-0474-496d-8496-033ad45d7fea\",\n                \"updated\": \"2020-01-02T18:23:04Z\",\n                \"title\": \"John's Simple Event\",\n                \"start\": \"2020-01-15T13:00:00\",\n                \"timeZone\": \"America/New_York\",\n                \"duration\": \"PT1H\",\n                \"calendarIds\": {\n                    &john_calendar_id: true\n                },\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id()\n        .to_string();\n    let response = jane\n        .jmap_create(\n            MethodObject::Calendar,\n            [json!({\n                \"name\": \"Test #1\",\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let jane_calendar_id = response.created(0).id().to_string();\n    let jane_event_id = jane\n        .jmap_create(\n            MethodObject::CalendarEvent,\n            [json!({\n                \"uid\": \"a8df6575-0474-496d-8496-033ad45d7fea\",\n                \"updated\": \"2020-01-02T18:23:04Z\",\n                \"title\": \"Jane's Simple Event\",\n                \"start\": \"2020-01-15T13:00:00\",\n                \"timeZone\": \"America/New_York\",\n                \"duration\": \"PT1H\",\n                \"calendarIds\": {\n                    &jane_calendar_id: true\n                },\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id()\n        .to_string();\n\n    // Verify myRights\n    john.jmap_get(\n        MethodObject::Calendar,\n        [\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::MyRights,\n            CalendarProperty::ShareWith,\n        ],\n        [john_calendar_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_calendar_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n            \"mayReadItems\": true,\n            \"mayWriteAll\": true,\n            \"mayDelete\": true,\n            \"mayShare\": true,\n            \"mayWriteOwn\": true,\n            \"mayReadFreeBusy\": true,\n            \"mayUpdatePrivate\": true,\n            \"mayRSVP\": true\n        },\n        \"shareWith\": {}\n        }));\n\n    // Obtain share notifications\n    let mut jane_share_change_id = jane\n        .jmap_get(\n            MethodObject::ShareNotification,\n            Vec::<&str>::new(),\n            Vec::<&str>::new(),\n        )\n        .await\n        .state()\n        .to_string();\n\n    // Make sure Jane has no access\n    assert_eq!(\n        jane.jmap_get_account(\n            john,\n            MethodObject::Calendar,\n            Vec::<&str>::new(),\n            [john_calendar_id.as_str()],\n        )\n        .await\n        .method_response()\n        .typ(),\n        \"forbidden\"\n    );\n\n    // Share calendar with Jane\n    john.jmap_update(\n        MethodObject::Calendar,\n        [(\n            &john_calendar_id,\n            json!({\n                \"shareWith\": {\n                   &jane_id : {\n                     \"mayReadItems\": true,\n                   }\n                }\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_calendar_id);\n    john.jmap_get(\n        MethodObject::Calendar,\n        [\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::ShareWith,\n        ],\n        [john_calendar_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_calendar_id,\n        \"name\": \"Test #1\",\n        \"shareWith\": {\n            &jane_id : {\n                \"mayReadItems\": true,\n                \"mayWriteAll\": false,\n                \"mayDelete\": false,\n                \"mayShare\": false,\n                \"mayWriteOwn\": false,\n                \"mayReadFreeBusy\": false,\n                \"mayUpdatePrivate\": false,\n                \"mayRSVP\": false\n            }\n        }\n        }));\n\n    // Verify Jane can access the event\n    jane.jmap_get_account(\n        john,\n        MethodObject::Calendar,\n        [\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::MyRights,\n        ],\n        [john_calendar_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_calendar_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n            \"mayReadItems\": true,\n            \"mayWriteAll\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n        }\n        }));\n    jane.jmap_get_account(\n        john,\n        MethodObject::CalendarEvent,\n        [JSCalendarProperty::<Id>::Id, JSCalendarProperty::Title],\n        [john_event_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_event_id,\n        \"title\": \"John's Simple Event\",\n        }));\n\n    // Verify Jane received a share notification\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    jane_share_change_id = response.new_state().to_string();\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"Calendar\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_calendar_id,\n          \"oldRights\": {\n            \"mayReadItems\": false,\n            \"mayWriteAll\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n          },\n          \"newRights\": {\n            \"mayReadItems\": true,\n            \"mayWriteAll\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n          },\n          \"name\": null\n        }));\n\n    // Updating and deleting should fail\n    assert_eq!(\n        jane.jmap_update_account(\n            john,\n            MethodObject::Calendar,\n            [(&john_calendar_id, json!({}))],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_updated(&john_calendar_id)\n        .description(),\n        \"You are not allowed to modify this calendar.\"\n    );\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::Calendar,\n            [&john_calendar_id],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_destroyed(&john_calendar_id)\n        .description(),\n        \"You are not allowed to delete this calendar.\"\n    );\n    assert!(\n        jane.jmap_update_account(\n            john,\n            MethodObject::CalendarEvent,\n            [(&john_event_id, json!({}))],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_updated(&john_event_id)\n        .description()\n        .contains(\"You are not allowed to modify calendar\"),\n    );\n    assert!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::CalendarEvent,\n            [&john_event_id],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_destroyed(&john_event_id)\n        .description()\n        .contains(\"You are not allowed to remove events from calendar\"),\n    );\n\n    // Grant Jane write access\n    john.jmap_update(\n        MethodObject::Calendar,\n        [(\n            &john_calendar_id,\n            json!({\n                format!(\"shareWith/{jane_id}/mayWriteAll\"): true,\n                format!(\"shareWith/{jane_id}/mayDelete\"): true,\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_calendar_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::Calendar,\n        [\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::MyRights,\n        ],\n        [john_calendar_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_calendar_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n            \"mayReadItems\": true,\n            \"mayWriteAll\": true,\n            \"mayDelete\": true,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n        }\n        }));\n\n    // Verify Jane received a share notification with the updated rights\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    jane_share_change_id = response.new_state().to_string();\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"Calendar\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_calendar_id,\n          \"oldRights\": {\n            \"mayReadItems\": true,\n            \"mayWriteAll\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n          },\n          \"newRights\": {\n            \"mayReadItems\": true,\n            \"mayWriteAll\": true,\n            \"mayDelete\": true,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n          },\n          \"name\": null\n        }));\n\n    // Creating a root folder should fail\n    assert_eq!(\n        jane.jmap_create_account(\n            john,\n            MethodObject::Calendar,\n            [json!({\n                \"name\": \"A new shared calendar\",\n            })],\n            Vec::<(&str, &str)>::new()\n        )\n        .await\n        .not_created(0)\n        .description(),\n        \"Cannot create calendars in a shared account.\"\n    );\n\n    // Copy Jane's event into John's calendar\n    let john_copied_event_id = jane\n        .jmap_copy(\n            jane,\n            john,\n            MethodObject::CalendarEvent,\n            [(\n                &jane_event_id,\n                json!({\n                    \"calendarIds\": {\n                        &john_calendar_id: true\n                    }\n                }),\n            )],\n            false,\n        )\n        .await\n        .copied(&jane_event_id)\n        .id()\n        .to_string();\n    jane.jmap_get_account(\n        john,\n        MethodObject::CalendarEvent,\n        [\n            JSCalendarProperty::<Id>::Id,\n            JSCalendarProperty::CalendarIds,\n            JSCalendarProperty::Title,\n        ],\n        [john_copied_event_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_copied_event_id,\n        \"title\": \"Jane's Simple Event\",\n        \"calendarIds\": {\n            &john_calendar_id: true\n        }\n        }));\n\n    // Destroy the copied event\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::CalendarEvent,\n            [john_copied_event_id.as_str()],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .destroyed()\n        .collect::<Vec<_>>(),\n        [&john_copied_event_id]\n    );\n\n    // Update John's event\n    jane.jmap_update_account(\n        john,\n        MethodObject::CalendarEvent,\n        [(\n            &john_event_id,\n            json!({\n                \"title\": \"John's Updated Event\",\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_event_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::CalendarEvent,\n        [JSCalendarProperty::<Id>::Id, JSCalendarProperty::Title],\n        [john_event_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_event_id,\n        \"title\": \"John's Updated Event\",\n        }));\n\n    // Update John's calendar name\n    jane.jmap_update_account(\n        john,\n        MethodObject::Calendar,\n        [(\n            &john_calendar_id,\n            json!({\n                \"name\": \"Jane's version of John's Calendar\",\n                \"description\": \"This is John's calendar, but Jane can edit it now\"\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_calendar_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::Calendar,\n        [\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::Description,\n        ],\n        [john_calendar_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_calendar_id,\n        \"name\": \"Jane's version of John's Calendar\",\n        \"description\": \"This is John's calendar, but Jane can edit it now\"\n        }));\n\n    // John should still see the old name\n    john.jmap_get(\n        MethodObject::Calendar,\n        [\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::Description,\n        ],\n        [john_calendar_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_calendar_id,\n        \"name\": \"Test #1\",\n        \"description\": null\n        }));\n\n    // Revoke Jane's access\n    john.jmap_update(\n        MethodObject::Calendar,\n        [(\n            &john_calendar_id,\n            json!({\n                format!(\"shareWith/{jane_id}\"): ()\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_calendar_id);\n    john.jmap_get(\n        MethodObject::Calendar,\n        [\n            CalendarProperty::Id,\n            CalendarProperty::Name,\n            CalendarProperty::ShareWith,\n        ],\n        [john_calendar_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_calendar_id,\n        \"name\": \"Test #1\",\n        \"shareWith\": {}\n        }));\n\n    // Verify Jane can no longer access the calendar or its events\n    assert_eq!(\n        jane.jmap_get_account(\n            john,\n            MethodObject::Calendar,\n            Vec::<&str>::new(),\n            [john_calendar_id.as_str()],\n        )\n        .await\n        .method_response()\n        .typ(),\n        \"forbidden\"\n    );\n\n    // Verify Jane received a share notification with the updated rights\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"Calendar\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_calendar_id,\n          \"oldRights\": {\n            \"mayReadItems\": true,\n            \"mayWriteAll\": true,\n            \"mayDelete\": true,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n          },\n          \"newRights\": {\n            \"mayReadItems\": false,\n            \"mayWriteAll\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false,\n            \"mayWriteOwn\": false,\n            \"mayReadFreeBusy\": false,\n            \"mayUpdatePrivate\": false,\n            \"mayRSVP\": false\n          },\n          \"name\": null\n        }));\n\n    // Grant Jane delete access once again\n    john.jmap_update(\n        MethodObject::Calendar,\n        [(\n            &john_calendar_id,\n            json!({\n                format!(\"shareWith/{jane_id}/mayReadItems\"): true,\n                format!(\"shareWith/{jane_id}/mayDelete\"): true,\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_calendar_id);\n\n    // Verify Jane can delete the calendar\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::Calendar,\n            [john_calendar_id.as_str()],\n            [(\"onDestroyRemoveEvents\", true)],\n        )\n        .await\n        .destroyed()\n        .collect::<Vec<_>>(),\n        [john_calendar_id.as_str()]\n    );\n\n    // Destroy all mailboxes\n    john.destroy_all_calendars().await;\n    jane.destroy_all_calendars().await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/calendar/alarm.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse futures::StreamExt;\nuse jmap_client::{\n    CalendarAlert, PushObject, client_ws::WebSocketMessage, event_source::PushNotification,\n};\nuse jmap_proto::request::method::MethodObject;\nuse mail_parser::DateTime;\nuse serde_json::json;\nuse std::time::Instant;\nuse store::write::now;\nuse tokio::sync::mpsc;\n\nuse crate::jmap::{IntoJmapSet, JMAPTest, JmapUtils};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Calendar Alarm tests...\");\n    let account = params.account(\"jdoe@example.com\");\n    let account_id = account.id_string();\n    let client = account.client();\n    let client_ws = account.client_owned().await;\n\n    // Create test calendar\n    let response = account\n        .jmap_create(\n            MethodObject::Calendar,\n            [json!({\n                \"name\": \"Alarming Calendar\",\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let calendar_id = response.created(0).id().to_string();\n\n    // Connect to EventSource\n    let (event_tx, mut event_rx) = mpsc::channel::<PushNotification>(100);\n    let mut notifications = client\n        .event_source(None::<Vec<_>>, false, 1.into(), None)\n        .await\n        .unwrap();\n    tokio::spawn(async move {\n        while let Some(notification) = notifications.next().await {\n            if let Err(_err) = event_tx.send(notification.unwrap()).await {\n                break;\n            }\n        }\n    });\n\n    // Connect to WebSocket\n    let mut ws_stream = client_ws.connect_ws().await.unwrap();\n    let (stream_tx, mut stream_rx) = mpsc::channel::<WebSocketMessage>(100);\n    tokio::spawn(async move {\n        while let Some(change) = ws_stream.next().await {\n            if stream_tx.send(change.unwrap()).await.is_err() {\n                break;\n            }\n        }\n    });\n    client_ws\n        .enable_push_ws(None::<Vec<_>>, None::<&str>)\n        .await\n        .unwrap();\n\n    // Create test event\n    let response = account\n        .jmap_create(\n            MethodObject::CalendarEvent,\n            [json!({\n              \"@type\": \"Event\",\n              \"calendarIds\": ([calendar_id.as_str()].into_jmap_set()),\n              \"description\": \"What mirror where?!\",\n              \"timeZone\": \"Etc/UTC\",\n              \"start\": DateTime::from_timestamp(now() as i64 + 5)\n                        .to_rfc3339().trim_end_matches(\"Z\").to_string(),\n              \"title\": \"See the pretty girl in that mirror there\",\n              \"alerts\": {\n                \"k1\": {\n                  \"@type\": \"Alert\",\n                  \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"offset\": \"-PT2S\"\n                  },\n                  \"action\": \"display\"\n                },\n                \"k2\": {\n                  \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"offset\": \"-PT4S\"\n                  },\n                  \"action\": \"display\",\n                  \"@type\": \"Alert\"\n                }\n              },\n              \"locations\": {\n                \"0b7168ae-ed3e-5eae-9540-89ba3a469b16\": {\n                  \"name\": \"West Side\",\n                  \"@type\": \"Location\"\n                }\n              },\n              \"uid\": \"2371c2d9-a136-43b0-bba3-f6ab249ad46e\",\n              \"duration\": \"P1D\"\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let event_id = response.created(0).id().to_string();\n\n    // Wait for alarm notifications\n    let start = Instant::now();\n    let mut ws_events = Vec::new();\n    let mut es_events = Vec::new();\n\n    while start.elapsed().as_secs() < 7 && (ws_events.len() < 2 || es_events.len() < 2) {\n        tokio::select! {\n            Some(notification) = event_rx.recv() => {\n                if let PushNotification::CalendarAlert(alert) = notification {\n                    es_events.push(alert);\n                }\n            }\n            Some(message) = stream_rx.recv() => {\n                match message {\n                    WebSocketMessage::PushNotification(PushObject::CalendarAlert(alert)) => {\n                        ws_events.push(alert);\n                    }\n                    WebSocketMessage::PushNotification(PushObject::Group {entries} ) => {\n                        ws_events.extend(entries.into_iter().filter_map(|entry| {\n                            if let PushObject::CalendarAlert(alert) = entry {\n                                Some(alert)\n                            } else {\n                                None\n                            }\n                        }));\n                    }\n                    _ => {}\n                }\n            }\n            _ = tokio::time::sleep(std::time::Duration::from_secs(6)) => {\n                break;\n            }\n        }\n    }\n\n    let expected_alerts = vec![\n        CalendarAlert {\n            account_id: account_id.to_string(),\n            calendar_event_id: event_id.clone(),\n            uid: \"2371c2d9-a136-43b0-bba3-f6ab249ad46e\".to_string(),\n            recurrence_id: None,\n            alert_id: \"k2\".to_string(),\n        },\n        CalendarAlert {\n            account_id: account_id.to_string(),\n            calendar_event_id: event_id.clone(),\n            uid: \"2371c2d9-a136-43b0-bba3-f6ab249ad46e\".to_string(),\n            recurrence_id: None,\n            alert_id: \"k1\".to_string(),\n        },\n    ];\n\n    assert_eq!(\n        es_events, expected_alerts,\n        \"EventSource alarms do not match\"\n    );\n    assert_eq!(ws_events, expected_alerts, \"WebSocket alarms do not match\");\n\n    // Cleanup\n    account.destroy_all_calendars().await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/calendar/calendars.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{ChangeType, JMAPTest, JmapUtils};\nuse jmap_proto::{object::calendar::CalendarProperty, request::method::MethodObject};\nuse serde_json::json;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Calendar tests...\");\n    let account = params.account(\"jdoe@example.com\");\n\n    // Make sure the default calendar exists\n    let response = account\n        .jmap_get(\n            MethodObject::Calendar,\n            [\n                CalendarProperty::Id,\n                CalendarProperty::Name,\n                CalendarProperty::Description,\n                CalendarProperty::SortOrder,\n                CalendarProperty::Color,\n                CalendarProperty::TimeZone,\n                CalendarProperty::IsSubscribed,\n                CalendarProperty::IsDefault,\n                CalendarProperty::IsVisible,\n                CalendarProperty::IncludeInAvailability,\n                CalendarProperty::DefaultAlertsWithTime,\n                CalendarProperty::DefaultAlertsWithoutTime,\n            ],\n            Vec::<&str>::new(),\n        )\n        .await;\n    let list = response.list();\n    assert_eq!(list.len(), 1);\n    let default_calendar_id = list[0].id().to_string();\n    assert_eq!(\n        list[0],\n        json!({\n            \"id\": default_calendar_id,\n            \"name\": \"Stalwart Calendar (jdoe@example.com)\",\n            \"description\": null,\n            \"sortOrder\": 0,\n            \"isSubscribed\": false,\n            \"isDefault\": true,\n            \"color\": null,\n            \"timeZone\": null,\n            \"isVisible\": true,\n            \"includeInAvailability\": \"all\",\n            \"defaultAlertsWithTime\": {},\n            \"defaultAlertsWithoutTime\": {}\n        })\n    );\n    let change_id = response.state();\n\n    // Create Calendar\n    let calendar_id = account\n        .jmap_create(\n            MethodObject::Calendar,\n            [json!({\n                \"name\": \"Test calendar\",\n                \"description\": \"My personal calendar\",\n                \"sortOrder\": 1,\n                \"isSubscribed\": true,\n                \"color\": \"#ff0000\",\n                \"timeZone\": \"Indian/Christmas\",\n                \"isVisible\": false,\n                \"includeInAvailability\": \"attending\",\n                \"defaultAlertsWithTime\": {\n                    \"0\": {\n                        \"action\": \"display\",\n                        \"trigger\": {\n                            \"relativeTo\": \"start\",\n                            \"offset\": \"PT15M\"\n                        }\n                    },\n                    \"1\": {\n                        \"action\": \"email\",\n                        \"trigger\": {\n                            \"relativeTo\": \"end\",\n                            \"offset\": \"PT30M\"\n                        }\n                    }\n                },\n                \"defaultAlertsWithoutTime\": {\n                    \"0\": {\n                        \"action\": \"display\",\n                        \"trigger\": {\n                            \"relativeTo\": \"start\",\n                            \"offset\": \"P1D\"\n                        }\n                    },\n                    \"1\": {\n                        \"action\": \"email\",\n                        \"trigger\": {\n                            \"relativeTo\": \"end\",\n                            \"offset\": \"P2D\"\n                        }\n                    }\n                }\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id()\n        .to_string();\n\n    // Validate changes\n    assert_eq!(\n        account\n            .jmap_changes(MethodObject::Calendar, change_id)\n            .await\n            .changes()\n            .collect::<Vec<_>>(),\n        [ChangeType::Created(&calendar_id)]\n    );\n\n    // Get Calendar\n    let response = account\n        .jmap_get(\n            MethodObject::Calendar,\n            [\n                CalendarProperty::Id,\n                CalendarProperty::Name,\n                CalendarProperty::Description,\n                CalendarProperty::SortOrder,\n                CalendarProperty::Color,\n                CalendarProperty::TimeZone,\n                CalendarProperty::IsSubscribed,\n                CalendarProperty::IsDefault,\n                CalendarProperty::IsVisible,\n                CalendarProperty::IncludeInAvailability,\n                CalendarProperty::DefaultAlertsWithTime,\n                CalendarProperty::DefaultAlertsWithoutTime,\n            ],\n            [&calendar_id],\n        )\n        .await;\n\n    response.list()[0].assert_is_equal(json!({\n        \"name\": \"Test calendar\",\n        \"description\": \"My personal calendar\",\n        \"sortOrder\": 1,\n        \"isSubscribed\": true,\n        \"isVisible\": false,\n        \"isDefault\": false,\n        \"color\": \"#ff0000\",\n        \"timeZone\": \"Indian/Christmas\",\n        \"includeInAvailability\": \"attending\",\n        \"defaultAlertsWithTime\": {\n            \"0\": {\n                \"@type\": \"Alert\",\n                \"action\": \"display\",\n                \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"relativeTo\": \"start\",\n                    \"offset\": \"PT15M\"\n                }\n            },\n            \"1\": {\n                \"@type\": \"Alert\",\n                \"action\": \"email\",\n                \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"relativeTo\": \"end\",\n                    \"offset\": \"PT30M\"\n                }\n            }\n        },\n        \"defaultAlertsWithoutTime\": {\n            \"0\": {\n                \"@type\": \"Alert\",\n                \"action\": \"display\",\n                \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"relativeTo\": \"start\",\n                    \"offset\": \"P1D\"\n                }\n            },\n            \"1\": {\n                \"@type\": \"Alert\",\n                \"action\": \"email\",\n                \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"relativeTo\": \"end\",\n                    \"offset\": \"P2D\"\n                }\n            }\n        },\n        \"id\": calendar_id,\n    }));\n\n    // Update Calendar and set it as default\n    account\n        .jmap_update(\n            MethodObject::Calendar,\n            [(\n                calendar_id.as_str(),\n                json!({\n                    \"name\": \"Updated calendar\",\n                    \"description\": \"My updated personal calendar\",\n                    \"sortOrder\": 2,\n                    \"isSubscribed\": false,\n                    \"isVisible\": true,\n                    \"timeZone\": null,\n                    \"color\": null,\n                    \"includeInAvailability\": \"none\",\n                    \"defaultAlertsWithTime\": {\n                        \"0\": {\n                            \"action\": \"email\",\n                            \"trigger\": {\n                                \"relativeTo\": \"start\",\n                                \"offset\": \"PT10M\"\n                            }\n                        }\n                    },\n                    \"defaultAlertsWithoutTime/0\": {\n                        \"action\": \"email\",\n                        \"trigger\": {\n                            \"relativeTo\": \"start\",\n                            \"offset\": \"P3D\"\n                        }\n                    },\n                    \"defaultAlertsWithoutTime/1\": null,\n                    \"defaultAlertsWithoutTime/2\": {\n                        \"action\": \"display\",\n                        \"trigger\": {\n                            \"relativeTo\": \"end\",\n                            \"offset\": \"P1W\"\n                        }\n                    }\n                }),\n            )],\n            [(\"onSuccessSetIsDefault\", calendar_id.as_str())],\n        )\n        .await\n        .updated(&calendar_id);\n\n    // Validate changes\n    let response = account\n        .jmap_get(\n            MethodObject::Calendar,\n            [\n                CalendarProperty::Id,\n                CalendarProperty::Name,\n                CalendarProperty::Description,\n                CalendarProperty::SortOrder,\n                CalendarProperty::Color,\n                CalendarProperty::TimeZone,\n                CalendarProperty::IsSubscribed,\n                CalendarProperty::IsDefault,\n                CalendarProperty::IsVisible,\n                CalendarProperty::IncludeInAvailability,\n                CalendarProperty::DefaultAlertsWithTime,\n                CalendarProperty::DefaultAlertsWithoutTime,\n            ],\n            [&calendar_id, &default_calendar_id],\n        )\n        .await;\n    response.list()[0].assert_is_equal(json!({\n        \"id\": calendar_id,\n        \"name\": \"Updated calendar\",\n        \"description\": \"My updated personal calendar\",\n        \"sortOrder\": 2,\n        \"isSubscribed\": false,\n        \"isDefault\": true,\n        \"color\": null,\n        \"timeZone\": null,\n        \"isVisible\": true,\n        \"includeInAvailability\": \"none\",\n        \"defaultAlertsWithTime\": {\n            \"0\": {\n                \"@type\": \"Alert\",\n                \"action\": \"email\",\n                \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"relativeTo\": \"start\",\n                    \"offset\": \"PT10M\"\n                }\n            }\n        },\n        \"defaultAlertsWithoutTime\": {\n            \"0\": {\n                \"@type\": \"Alert\",\n                \"action\": \"email\",\n                \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"relativeTo\": \"start\",\n                    \"offset\": \"P3D\"\n                }\n            },\n            \"2\": {\n                \"@type\": \"Alert\",\n                \"action\": \"display\",\n                \"trigger\": {\n                    \"@type\": \"OffsetTrigger\",\n                    \"relativeTo\": \"end\",\n                    \"offset\": \"P1W\"\n                }\n            }\n        }\n    }));\n    response.list()[1].assert_is_equal(json!({\n        \"id\": default_calendar_id,\n        \"name\": \"Stalwart Calendar (jdoe@example.com)\",\n        \"description\": (),\n        \"sortOrder\": 0,\n        \"isSubscribed\": false,\n        \"isDefault\": false,\n        \"color\": null,\n        \"timeZone\": null,\n        \"isVisible\": true,\n        \"includeInAvailability\": \"all\",\n        \"defaultAlertsWithTime\": {},\n        \"defaultAlertsWithoutTime\": {}\n    }));\n\n    // Create an event\n    let _ = account\n        .jmap_create(\n            MethodObject::CalendarEvent,\n            [json!({\n                \"calendarIds\": {\n                    &calendar_id: true\n                },\n                \"@type\": \"Event\",\n                \"uid\": \"a8df6573-0474-496d-8496-033ad45d7fea\",\n                \"updated\": \"2020-01-02T18:23:04Z\",\n                \"title\": \"Some event\",\n                \"start\": \"2020-01-15T13:00:00\",\n                \"timeZone\": \"America/New_York\",\n                \"duration\": \"PT1H\"\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id();\n\n    // Try destroying the calendar (should fail)\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::Calendar,\n                [&calendar_id],\n                Vec::<(&str, &str)>::new(),\n            )\n            .await\n            .not_destroyed(&calendar_id)\n            .typ(),\n        \"calendarHasEvent\"\n    );\n\n    // Destroy using force\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::Calendar,\n                [&calendar_id],\n                [(\"onDestroyRemoveEvents\", true)],\n            )\n            .await\n            .destroyed()\n            .collect::<Vec<_>>(),\n        vec![&calendar_id]\n    );\n\n    // Destroy all mailboxes\n    account.destroy_all_calendars().await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/calendar/event.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    jmap::{ChangeType, IntoJmapSet, JMAPTest, JmapUtils, wait_for_index},\n    webdav::DummyWebDavClient,\n};\nuse ahash::AHashSet;\nuse calcard::jscalendar::JSCalendarProperty;\nuse groupware::cache::GroupwareCache;\nuse hyper::StatusCode;\nuse jmap_proto::request::method::MethodObject;\nuse serde_json::{Value, json};\nuse types::{collection::SyncCollection, id::Id};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Calendar Event tests...\");\n    let account = params.account(\"jdoe@example.com\");\n\n    // Create test calendars\n    let response = account\n        .jmap_create(\n            MethodObject::Calendar,\n            [\n                json!({\n                    \"name\": \"Holy Calendar, Batman!\",\n                    \"timeZone\": \"Europe/Vatican\",\n                }),\n                json!({\n                    \"name\": \"Calendar with Alerts\",\n                    \"defaultAlertsWithTime\": {\n                        \"abc\": {\n                            \"action\": \"display\",\n                            \"trigger\": {\n                                \"relativeTo\": \"start\",\n                                \"offset\": \"PT15M\"\n                            }\n                        }\n                    },\n                }),\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let calendar1_id = response.created(0).id().to_string();\n    let calendar2_id = response.created(1).id().to_string();\n\n    // Obtain state\n    let change_id = account\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            Vec::<&str>::new(),\n            Vec::<&str>::new(),\n        )\n        .await\n        .state()\n        .to_string();\n\n    // Create test events\n    let event_1 = test_jscalendar_1().with_property(\n        JSCalendarProperty::<Id>::CalendarIds,\n        [calendar1_id.as_str()].into_jmap_set(),\n    );\n    let event_2 = test_jscalendar_2().with_property(\n        JSCalendarProperty::<Id>::CalendarIds,\n        [calendar2_id.as_str()].into_jmap_set(),\n    );\n    let event_3 = test_jscalendar_3().with_property(\n        JSCalendarProperty::<Id>::CalendarIds,\n        [calendar1_id.as_str(), calendar2_id.as_str()].into_jmap_set(),\n    );\n    let event_4 = test_jscalendar_4().with_property(\n        JSCalendarProperty::<Id>::CalendarIds,\n        [calendar1_id.as_str()].into_jmap_set(),\n    );\n    let response = account\n        .jmap_create(\n            MethodObject::CalendarEvent,\n            [\n                event_1\n                    .clone()\n                    .with_property(JSCalendarProperty::<Id>::IsDraft, true)\n                    .with_property(JSCalendarProperty::<Id>::MayInviteSelf, true)\n                    .with_property(JSCalendarProperty::<Id>::MayInviteOthers, true)\n                    .with_property(JSCalendarProperty::<Id>::HideAttendees, true),\n                event_2\n                    .clone()\n                    .with_property(JSCalendarProperty::<Id>::UseDefaultAlerts, true),\n                event_3.clone(),\n                event_4,\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let event_1_id = response.created(0).id().to_string();\n    let event_2_id = response.created(1).id().to_string();\n    let event_3_id = response.created(2).id().to_string();\n    let event_4_id = response.created(3).id().to_string();\n\n    // Destroy tmp event\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::CalendarEvent,\n                [event_4_id.as_str()],\n                Vec::<(&str, &str)>::new(),\n            )\n            .await\n            .destroyed()\n            .next(),\n        Some(event_4_id.as_str())\n    );\n\n    // Validate changes\n    assert_eq!(\n        account\n            .jmap_changes(MethodObject::CalendarEvent, &change_id)\n            .await\n            .changes()\n            .collect::<AHashSet<_>>(),\n        [\n            ChangeType::Created(&event_1_id),\n            ChangeType::Created(&event_2_id),\n            ChangeType::Created(&event_3_id)\n        ]\n        .into_iter()\n        .collect::<AHashSet<_>>(),\n    );\n\n    // Verify event contents\n    let response = account\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            Vec::<&str>::new(),\n            [&event_1_id, &event_2_id, &event_3_id],\n        )\n        .await;\n\n    response.list()[0].assert_is_equal(\n        event_1\n            .with_property(JSCalendarProperty::<Id>::Id, event_1_id.as_str())\n            .with_property(JSCalendarProperty::<Id>::IsDraft, true)\n            .with_property(JSCalendarProperty::<Id>::IsOrigin, true),\n    );\n    response.list()[1].assert_is_equal(\n        event_2\n            .with_property(JSCalendarProperty::<Id>::Id, event_2_id.as_str())\n            .with_property(JSCalendarProperty::<Id>::IsDraft, false)\n            .with_property(JSCalendarProperty::<Id>::IsOrigin, true)\n            .with_property(\n                JSCalendarProperty::<Id>::Alerts,\n                json!({\n                  \"k1\": {\n                    \"action\": \"display\",\n                    \"trigger\": {\n                      \"@type\": \"OffsetTrigger\",\n                      \"offset\": \"PT15M\"\n                    },\n                    \"@type\": \"Alert\"\n                  }\n                }),\n            ),\n    );\n    response.list()[2].assert_is_equal(\n        event_3\n            .with_property(JSCalendarProperty::<Id>::Id, event_3_id.as_str())\n            .with_property(JSCalendarProperty::<Id>::IsDraft, false)\n            .with_property(JSCalendarProperty::<Id>::IsOrigin, false),\n    );\n\n    // Verify JMAP for Calendars properties\n    let response = account\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            [\n                JSCalendarProperty::<Id>::Id,\n                JSCalendarProperty::MayInviteSelf,\n                JSCalendarProperty::MayInviteOthers,\n                JSCalendarProperty::HideAttendees,\n                JSCalendarProperty::UtcStart,\n                JSCalendarProperty::UtcEnd,\n            ],\n            [&event_1_id, &event_2_id, &event_3_id],\n        )\n        .await;\n    response.list()[0].assert_is_equal(json!({\n      \"id\": &event_1_id,\n      \"mayInviteSelf\": true,\n      \"mayInviteOthers\": true,\n      \"hideAttendees\": true,\n      \"utcStart\": \"2006-01-02T15:00:00Z\",\n      \"utcEnd\": \"2006-01-02T16:00:00Z\"\n    }));\n    response.list()[1].assert_is_equal(json!({\n      \"id\": &event_2_id,\n      \"mayInviteSelf\": false,\n      \"mayInviteOthers\": false,\n      \"hideAttendees\": false,\n      \"utcStart\": \"2006-01-02T17:00:00Z\",\n      \"utcEnd\": \"2006-01-02T18:00:00Z\"\n    }));\n    response.list()[2].assert_is_equal(json!({\n        \"id\": &event_3_id,\n        \"mayInviteSelf\": false,\n        \"mayInviteOthers\": false,\n        \"hideAttendees\": false,\n        \"utcStart\": \"2006-01-04T15:00:00Z\",\n        \"utcEnd\": \"2006-01-04T16:00:00Z\"\n    }));\n\n    // Test /get parameters\n    let response = account\n        .jmap_method_calls(json!([[\n            \"CalendarEvent/get\",\n            {\n                \"properties\": [\"id\", \"title\", \"recurrenceOverrides\", \"participants\"],\n                \"ids\": [&event_2_id, &event_3_id],\n                \"recurrenceOverridesBefore\": \"2006-01-07T00:00:00Z\",\n                \"recurrenceOverridesAfter\": \"2006-01-06T00:00:00Z\",\n                \"reduceParticipants\": true,\n            },\n            \"0\"\n        ]]))\n        .await;\n    response.list_array().assert_is_equal(json!([\n      {\n        \"title\": \"Event #2\",\n        \"recurrenceOverrides\": {\n          \"2006-01-06T12:00:00\": {\n            \"updated\": \"2006-02-06T00:11:21Z\",\n            \"start\": \"2006-01-06T14:00:00\",\n            \"title\": \"Event #2 bis bis\",\n            \"duration\": \"PT1H\"\n          }\n        },\n        \"id\": \"c\"\n      },\n      {\n        \"title\": \"Event #3\",\n        \"participants\": {\n          \"3f5bc8c0-c722-5345-b7d9-5a899db08a30\": {\n            \"calendarAddress\": \"mailto:cyrus@example.com\",\n            \"@type\": \"Participant\"\n          }\n        },\n        \"id\": \"d\"\n      }\n    ]));\n\n    // Creating an event without calendar should fail\n    assert_eq!(\n        account\n            .jmap_create(\n                MethodObject::CalendarEvent,\n                [json!({\n                    \"title\": \"Event #5\",\n                    \"start\": \"2006-01-22T10:00:00\",\n                    \"duration\": \"PT1H\",\n                    \"timeZone\": \"US/Eastern\",\n                    \"calendarIds\": {},\n                }),],\n                Vec::<(&str, &str)>::new()\n            )\n            .await\n            .not_created(0)\n            .description(),\n        \"Event has to belong to at least one calendar.\"\n    );\n\n    // Creating an event with a duplicate UID should fail\n    assert_eq!(\n        account\n            .jmap_create(\n                MethodObject::CalendarEvent,\n                [json!({\n                    \"title\": \"Event #5\",\n                    \"start\": \"2006-01-22T10:00:00\",\n                    \"duration\": \"PT1H\",\n                    \"timeZone\": \"US/Eastern\",\n                    \"uid\": \"00959BC664CA650E933C892C@example.com\",\n                    \"calendarIds\": {\n                        &calendar1_id: true\n                    },\n                })],\n                Vec::<(&str, &str)>::new()\n            )\n            .await\n            .not_created(0)\n            .description(),\n        \"An event with UID 00959BC664CA650E933C892C@example.com already exists.\"\n    );\n\n    // Patching tests\n    let response = account\n        .jmap_update(\n            MethodObject::CalendarEvent,\n            [\n                (\n                    &event_1_id,\n                    json!({\n                        \"isDraft\": false,\n                        \"mayInviteSelf\": false,\n                        \"mayInviteOthers\": false,\n                        \"hideAttendees\": false,\n                        \"description\": null,\n                        \"title\": \"Event one\",\n                        \"keywords\": {\"work\": true},\n                        format!(\"calendarIds/{calendar2_id}\"): true\n                    }),\n                ),\n                (\n                    &event_2_id,\n                    json!({\n                        \"calendarIds\": {\n                            &calendar1_id: true,\n                            &calendar2_id: true\n                        },\n                        \"title\": \"Event two\",\n                        \"description\": \"Updated description\",\n                        \"recurrenceOverrides/2006-01-04T12:00:00/title\":\n                        \"Event two overridden\",\n                        \"recurrenceOverrides/2006-01-06T12:00:00/title\":\n                        \"Event two overridden twice\",\n\n                    }),\n                ),\n                (\n                    &event_3_id,\n                    json!({\n                        format!(\"calendarIds/{calendar2_id}\"): false,\n                        \"title\": \"Event three\",\n                        \"utcStart\": \"2006-01-04T14:00:00Z\",\n                        \"utcEnd\": \"2006-01-04T16:00:00Z\",\n                        \"participants/3f5bc8c0-c722-5345-b7d9-5a899db08a30/roles/chair\": false,\n                        \"participants/3f5bc8c0-c722-5345-b7d9-5a899db08a30/roles/owner\": true,\n                        \"participants/ec5e7db5-22a3-5ed5-89bf-c8894ab86805\" : null,\n                        \"participants/7f2bd210-6c66-5b64-8562-0176b74462b1\": {\n                            \"calendarAddress\": \"mailto:rupert@example.com\",\n                            \"@type\": \"Participant\",\n                            \"participationStatus\": \"needs-action\"\n                        }\n                    }),\n                ),\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n\n    response.updated(&event_1_id);\n    response.updated(&event_2_id);\n    response.updated(&event_3_id);\n\n    // Verify patches\n    let response = account\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            [\n                JSCalendarProperty::<Id>::Id,\n                JSCalendarProperty::CalendarIds,\n                JSCalendarProperty::Title,\n                JSCalendarProperty::Start,\n                JSCalendarProperty::Description,\n                JSCalendarProperty::Keywords,\n                JSCalendarProperty::RecurrenceOverrides,\n                JSCalendarProperty::Participants,\n                JSCalendarProperty::MayInviteOthers,\n                JSCalendarProperty::MayInviteSelf,\n                JSCalendarProperty::HideAttendees,\n                JSCalendarProperty::IsDraft,\n            ],\n            [&event_1_id, &event_2_id, &event_3_id],\n        )\n        .await;\n\n    response.list()[0].assert_is_equal(json!({\n      \"id\": &event_1_id,\n      \"calendarIds\":  {\n        &calendar1_id: true,\n        &calendar2_id: true\n      },\n      \"isDraft\": false,\n      \"mayInviteSelf\": false,\n      \"mayInviteOthers\": false,\n      \"hideAttendees\": false,\n      \"title\": \"Event one\",\n      \"start\": \"2006-01-02T10:00:00\",\n      \"keywords\": {\n        \"work\": true\n      }\n    }));\n\n    response.list()[1].assert_is_equal(json!({\n        \"id\": &event_2_id,\n        \"calendarIds\": {\n          &calendar1_id: true,\n          &calendar2_id: true\n        },\n        \"title\": \"Event two\",\n        \"start\": \"2006-01-02T12:00:00\",\n        \"description\": \"Updated description\",\n        \"recurrenceOverrides\": {\n            \"2006-01-04T12:00:00\": {\n                \"title\": \"Event two overridden\",\n                \"start\": \"2006-01-04T14:00:00\",\n                \"duration\": \"PT1H\",\n                \"updated\": \"2006-02-06T00:11:21Z\"\n            },\n            \"2006-01-06T12:00:00\": {\n                \"title\": \"Event two overridden twice\",\n                \"start\": \"2006-01-06T14:00:00\",\n                \"duration\": \"PT1H\",\n                \"updated\": \"2006-02-06T00:11:21Z\"\n            }\n        },\n        \"title\": \"Event two\",\n        \"start\": \"2006-01-02T12:00:00\",\n        \"mayInviteOthers\": false,\n        \"mayInviteSelf\": false,\n        \"hideAttendees\": false,\n        \"isDraft\": false\n    }));\n\n    response.list()[2].assert_is_equal(json!({\n        \"id\": event_3_id,\n        \"calendarIds\": {\n            &calendar1_id: true,\n        },\n        \"title\": \"Event three\",\n        \"start\": \"2006-01-04T09:00:00\",\n        \"participants\": {\n            \"3f5bc8c0-c722-5345-b7d9-5a899db08a30\": {\n                \"calendarAddress\": \"mailto:cyrus@example.com\",\n                \"@type\": \"Participant\",\n                \"roles\": {\n                    \"owner\": true\n                },\n                \"participationStatus\": \"accepted\"\n            },\n            \"7f2bd210-6c66-5b64-8562-0176b74462b1\": {\n                \"calendarAddress\": \"mailto:rupert@example.com\",\n                \"@type\": \"Participant\",\n                \"participationStatus\": \"needs-action\"\n            }\n        },\n        \"mayInviteOthers\": false,\n        \"mayInviteSelf\": false,\n        \"hideAttendees\": false,\n        \"isDraft\": false\n    }));\n\n    // Query tests\n    wait_for_index(&params.server).await;\n    assert_eq!(\n        account\n            .jmap_query(\n                MethodObject::CalendarEvent,\n                [\n                    (\"text\", \"Event one\"),\n                    (\"inCalendar\", calendar1_id.as_str()),\n                    (\"uid\", \"74855313FA803DA593CD579A@example.com\"),\n                    (\"after\", \"2006-01-02T10:59:59\"),\n                    (\"before\", \"2006-01-02T10:00:01\"),\n                ],\n                [\"start\"],\n                [(\"timeZone\", \"US/Eastern\")],\n            )\n            .await\n            .ids()\n            .collect::<AHashSet<_>>(),\n        [event_1_id.as_str()].into_iter().collect::<AHashSet<_>>()\n    );\n\n    // Recurrence expansion tests\n    let response = account\n        .jmap_query(\n            MethodObject::CalendarEvent,\n            [\n                (\"after\", \"2006-01-01T00:00:00\"),\n                (\"before\", \"2006-01-08T00:00:00\"),\n            ],\n            [\"start\"],\n            [\n                (\"timeZone\", Value::String(\"US/Eastern\".into())),\n                (\"expandRecurrences\", Value::Bool(true)),\n            ],\n        )\n        .await;\n    let ids = response.ids().collect::<Vec<_>>();\n    assert_eq!(ids.len(), 7);\n    account\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            [\n                JSCalendarProperty::<Id>::Id,\n                JSCalendarProperty::BaseEventId,\n                JSCalendarProperty::Start,\n                JSCalendarProperty::Duration,\n                JSCalendarProperty::TimeZone,\n                JSCalendarProperty::Title,\n                JSCalendarProperty::RecurrenceId,\n            ],\n            ids.clone(),\n        )\n        .await\n        .list_array()\n        .assert_is_equal(json!([\n          {\n            \"duration\": \"PT1H\",\n            \"title\": \"Event one\",\n            \"start\": \"2006-01-02T10:00:00\",\n            \"timeZone\": \"US/Eastern\",\n            \"id\": &ids[0],\n            \"baseEventId\": &event_1_id\n          },\n          {\n            \"recurrenceId\": \"2006-01-02T12:00:00\",\n            \"title\": \"Event two\",\n            \"duration\": \"PT1H\",\n            \"start\": \"2006-01-02T12:00:00\",\n            \"timeZone\": \"US/Eastern\",\n            \"id\": &ids[1],\n            \"baseEventId\": &event_2_id\n          },\n          {\n            \"duration\": \"PT1H\",\n            \"start\": \"2006-01-03T12:00:00\",\n            \"timeZone\": \"US/Eastern\",\n            \"title\": \"Event two\",\n            \"recurrenceId\": \"2006-01-03T12:00:00\",\n            \"id\": &ids[2],\n            \"baseEventId\": &event_2_id\n          },\n          {\n            \"start\": \"2006-01-04T09:00:00\",\n            \"timeZone\": \"US/Eastern\",\n            \"duration\": \"PT2H\",\n            \"title\": \"Event three\",\n            \"id\": &ids[3],\n            \"baseEventId\": &event_3_id\n          },\n          {\n            \"recurrenceId\": \"2006-01-04T14:00:00\",\n            \"title\": \"Event two overridden\",\n            \"start\": \"2006-01-04T14:00:00\",\n            \"timeZone\": \"US/Eastern\",\n            \"duration\": \"PT1H\",\n            \"id\": &ids[4],\n            \"baseEventId\": &event_2_id\n          },\n          {\n            \"recurrenceId\": \"2006-01-05T12:00:00\",\n            \"duration\": \"PT1H\",\n            \"timeZone\": \"US/Eastern\",\n            \"start\": \"2006-01-05T12:00:00\",\n            \"title\": \"Event two\",\n            \"id\": &ids[5],\n            \"baseEventId\": &event_2_id\n          },\n          {\n            \"recurrenceId\": \"2006-01-06T14:00:00\",\n            \"duration\": \"PT1H\",\n            \"title\": \"Event two overridden twice\",\n            \"timeZone\": \"US/Eastern\",\n            \"start\": \"2006-01-06T14:00:00\",\n            \"id\": &ids[6],\n            \"baseEventId\": &event_2_id\n          }\n        ]));\n\n    // Parse tests\n    account\n        .jmap_method_calls(json!([\n         [\n          \"Blob/upload\",\n          {\n           \"create\": {\n            \"ical\": {\n             \"data\": [\n              {\n               \"data:asText\": r#\"BEGIN:VCALENDAR\nPRODID:-//xyz Corp//NONSGML PDA Calendar Version 1.0//EN\nVERSION:2.0\nBEGIN:VEVENT\nDTSTAMP:19960704T120000Z\nUID:uid1@example.com\nORGANIZER:mailto:jsmith@example.com\nDTSTART:19960918T143000Z\nDTEND:19960920T220000Z\nSTATUS:CONFIRMED\nCATEGORIES:CONFERENCE\nSUMMARY:Networld+Interop Conference\nDESCRIPTION:Networld+Interop Conference\n and Exhibit\\nAtlanta World Congress Center\\n\nAtlanta\\, Georgia\nEND:VEVENT\nEND:VCALENDAR\n\"#\n              }\n            ]\n           }\n          }\n         },\n         \"S4\"\n        ],\n        [\n          \"CalendarEvent/parse\",\n          {\n           \"blobIds\": [\n             \"#ical\"\n           ]\n          },\n          \"G4\"\n         ]\n        ]))\n        .await\n        .pointer(\"/methodResponses/1/1/parsed\")\n        .unwrap()\n        .as_object()\n        .unwrap()\n        .iter()\n        .next()\n        .unwrap()\n        .1\n        .assert_is_equal(json!([\n  {\n    \"updated\": \"1996-07-04T12:00:00Z\",\n    \"title\": \"Networld+Interop Conference\",\n    \"description\": \"Networld+Interop Conferenceand Exhibit\\nAtlanta World Congress Center\\n\",\n    \"timeZone\": \"Etc/UTC\",\n    \"start\": \"1996-09-18T14:30:00\",\n    \"status\": \"confirmed\",\n    \"iCalendar\": {\n      \"convertedProperties\": {\n        \"duration\": {\n          \"name\": \"dtend\"\n        }\n      },\n      \"name\": \"vevent\"\n    },\n    \"@type\": \"Event\",\n    \"uid\": \"uid1@example.com\",\n    \"participants\": {\n      \"25d7647e-52fc-559b-88df-d66f08da079c\": {\n        \"calendarAddress\": \"mailto:jsmith@example.com\",\n        \"@type\": \"Participant\"\n      }\n    },\n    \"keywords\": {\n      \"CONFERENCE\": true\n    },\n    \"organizerCalendarAddress\": \"mailto:jsmith@example.com\",\n    \"duration\": \"P2DT7H30M\"\n  }\n]));\n\n    // Deletion tests\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::CalendarEvent,\n                [event_2_id.as_str(), event_3_id.as_str()],\n                Vec::<(&str, &str)>::new()\n            )\n            .await\n            .destroyed()\n            .collect::<AHashSet<_>>(),\n        [event_2_id.as_str(), event_3_id.as_str()]\n            .into_iter()\n            .collect::<AHashSet<_>>()\n    );\n\n    // CardDAV compatibility tests\n    let account_id = account.id().document_id();\n    let dav_client = DummyWebDavClient::new(\n        u32::MAX,\n        account.name(),\n        account.secret(),\n        account.emails()[0],\n    );\n    let resources = params\n        .server\n        .fetch_dav_resources(\n            &params.server.get_access_token(account_id).await.unwrap(),\n            account_id,\n            SyncCollection::Calendar,\n        )\n        .await\n        .unwrap();\n    let path = format!(\n        \"{}{}\",\n        resources.base_path,\n        resources\n            .paths\n            .iter()\n            .find(|v| v.parent_id.is_some())\n            .unwrap()\n            .path\n    );\n\n    let ical = dav_client\n        .request(\"GET\", &path, \"\")\n        .await\n        .with_status(StatusCode::OK)\n        .expect_body()\n        .lines()\n        .map(String::from)\n        .collect::<AHashSet<_>>();\n    let expected_ical = TEST_ICAL_1\n        .lines()\n        .map(String::from)\n        .collect::<AHashSet<_>>();\n    assert_eq!(ical, expected_ical);\n\n    // Clean up\n    account.destroy_all_calendars().await;\n    params.assert_is_empty().await;\n}\n\npub fn test_jscalendar_1() -> Value {\n    json!({\n      \"duration\": \"PT1H\",\n      \"@type\": \"Event\",\n      \"description\": \"Go Steelers!\",\n      \"updated\": \"2006-02-06T00:11:02Z\",\n      \"timeZone\": \"US/Eastern\",\n      \"start\": \"2006-01-02T10:00:00\",\n      \"title\": \"Event #1\",\n      \"uid\": \"74855313FA803DA593CD579A@example.com\"\n    })\n}\n\npub fn test_jscalendar_2() -> Value {\n    json!({\n      \"title\": \"Event #2\",\n      \"duration\": \"PT1H\",\n      \"updated\": \"2006-02-06T00:11:21Z\",\n      \"recurrenceRule\": {\n        \"frequency\": \"daily\",\n        \"count\": 5\n      },\n      \"start\": \"2006-01-02T12:00:00\",\n      \"uid\": \"00959BC664CA650E933C892C@example.com\",\n      \"@type\": \"Event\",\n      \"timeZone\": \"US/Eastern\",\n      \"recurrenceOverrides\": {\n        \"2006-01-04T12:00:00\": {\n          \"title\": \"Event #2 bis\",\n          \"start\": \"2006-01-04T14:00:00\",\n          \"updated\": \"2006-02-06T00:11:21Z\",\n          \"duration\": \"PT1H\"\n        },\n        \"2006-01-06T12:00:00\": {\n          \"title\": \"Event #2 bis bis\",\n          \"start\": \"2006-01-06T14:00:00\",\n          \"updated\": \"2006-02-06T00:11:21Z\",\n          \"duration\": \"PT1H\"\n        }\n      }\n    })\n}\n\npub fn test_jscalendar_3() -> Value {\n    json!({\n      \"duration\": \"PT1H\",\n      \"organizerCalendarAddress\": \"mailto:cyrus@example.com\",\n      \"@type\": \"Event\",\n      \"start\": \"2006-01-04T10:00:00\",\n      \"status\": \"tentative\",\n      \"uid\": \"DC6C50A017428C5216A2F1CD@example.com\",\n      \"sequence\": 1,\n      \"participants\": {\n        \"3f5bc8c0-c722-5345-b7d9-5a899db08a30\": {\n          \"calendarAddress\": \"mailto:cyrus@example.com\",\n          \"@type\": \"Participant\",\n          \"roles\": {\n            \"chair\": true\n          },\n          \"participationStatus\": \"accepted\"\n        },\n        \"ec5e7db5-22a3-5ed5-89bf-c8894ab86805\": {\n          \"calendarAddress\": \"mailto:lisa@example.com\",\n          \"@type\": \"Participant\",\n          \"participationStatus\": \"needs-action\"\n        }\n      },\n      \"title\": \"Event #3\",\n      \"updated\": \"2006-02-06T00:12:20Z\",\n      \"timeZone\": \"US/Eastern\"\n    })\n}\n\npub fn test_jscalendar_4() -> Value {\n    json!({\n      \"duration\": \"PT1H\",\n      \"@type\": \"Event\",\n      \"description\": \"Tmp Event\",\n      \"updated\": \"2006-02-06T00:11:02Z\",\n      \"timeZone\": \"US/Eastern\",\n      \"start\": \"2006-01-02T10:00:00\",\n      \"title\": \"Tmp Event\",\n      \"uid\": \"tmp-event@example.com\"\n    })\n}\n\nconst TEST_ICAL_1: &str = r#\"BEGIN:VCALENDAR\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060102T100000\nUID:74855313FA803DA593CD579A@example.com\nDURATION:PT1H\nSUMMARY:Event one\nDTSTAMP:20060206T001102Z\nCATEGORIES:work\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n"
  },
  {
    "path": "tests/src/jmap/calendar/identity.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, JmapUtils};\nuse jmap_proto::{\n    object::participant_identity::ParticipantIdentityProperty, request::method::MethodObject,\n};\nuse serde_json::json;\nuse store::write::BatchBuilder;\nuse types::{collection::Collection, field::PrincipalField};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Participant Identity tests...\");\n    let account = params.account(\"jdoe@example.com\");\n\n    // Obtain all identities\n    let response = account\n        .jmap_get(\n            MethodObject::ParticipantIdentity,\n            [\n                ParticipantIdentityProperty::Id,\n                ParticipantIdentityProperty::Name,\n                ParticipantIdentityProperty::CalendarAddress,\n                ParticipantIdentityProperty::IsDefault,\n            ],\n            Vec::<&str>::new(),\n        )\n        .await;\n    response.list_array().assert_is_equal(json!([\n      {\n        \"id\": \"a\",\n        \"name\": \"John Doe\",\n        \"calendarAddress\": \"mailto:jdoe@example.com\",\n        \"isDefault\": true\n      },\n      {\n        \"id\": \"b\",\n        \"name\": \"John Doe\",\n        \"calendarAddress\": \"mailto:john.doe@example.com\",\n        \"isDefault\": false\n      }\n    ]));\n\n    // Destroy identity b\n    let response = account\n        .jmap_destroy(\n            MethodObject::ParticipantIdentity,\n            [\"b\"],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    assert_eq!(response.destroyed().next(), Some(\"b\"));\n    let response = account\n        .jmap_get(\n            MethodObject::ParticipantIdentity,\n            [\n                ParticipantIdentityProperty::Id,\n                ParticipantIdentityProperty::Name,\n                ParticipantIdentityProperty::CalendarAddress,\n                ParticipantIdentityProperty::IsDefault,\n            ],\n            Vec::<&str>::new(),\n        )\n        .await;\n    response.list_array().assert_is_equal(json!([\n      {\n        \"id\": \"a\",\n        \"name\": \"John Doe\",\n        \"calendarAddress\": \"mailto:jdoe@example.com\",\n        \"isDefault\": true\n      }\n    ]));\n\n    // Creating a new identity with an unauthorized calendar address should fail\n    let response = account\n        .jmap_create(\n            MethodObject::ParticipantIdentity,\n            [\n                json!({\n                    \"name\": \"Work\",\n                    \"calendarAddress\": \"mailto:work@example.com\"\n                }),\n                json!({\n                    \"name\": \"Work\",\n                    \"calendarAddress\": \"work@example.com\"\n                }),\n            ],\n            [(\"onSuccessSetIsDefault\", \"#i0\")],\n        )\n        .await;\n    assert_eq!(\n        response.not_created(0).description(),\n        \"Calendar address not configured for this account.\"\n    );\n    assert_eq!(\n        response.not_created(1).description(),\n        \"Calendar address not configured for this account.\"\n    );\n\n    // Create a new identity and set it as default\n    let response = account\n        .jmap_create(\n            MethodObject::ParticipantIdentity,\n            [json!({\n                \"name\": \"Johnny B Goode\",\n                \"calendarAddress\": \"mailto:john.doe@example.com\"\n            })],\n            [(\"onSuccessSetIsDefault\", \"#i0\")],\n        )\n        .await;\n    response.created(0);\n    let response = account\n        .jmap_get(\n            MethodObject::ParticipantIdentity,\n            [\n                ParticipantIdentityProperty::Id,\n                ParticipantIdentityProperty::Name,\n                ParticipantIdentityProperty::CalendarAddress,\n                ParticipantIdentityProperty::IsDefault,\n            ],\n            Vec::<&str>::new(),\n        )\n        .await;\n    response.list_array().assert_is_equal(json!([\n      {\n        \"id\": \"a\",\n        \"name\": \"John Doe\",\n        \"calendarAddress\": \"mailto:jdoe@example.com\",\n        \"isDefault\": false\n      },\n      {\n        \"id\": \"b\",\n        \"name\": \"Johnny B Goode\",\n        \"calendarAddress\": \"mailto:john.doe@example.com\",\n        \"isDefault\": true\n      }\n    ]));\n\n    // Cleanup\n    let mut batch = BatchBuilder::new();\n    batch\n        .with_account_id(account.id().document_id())\n        .with_collection(Collection::Principal)\n        .with_document(0)\n        .clear(PrincipalField::ParticipantIdentities);\n    params.server.commit_batch(batch).await.unwrap();\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/calendar/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod alarm;\npub mod calendars;\npub mod event;\npub mod identity;\npub mod notification;\n"
  },
  {
    "path": "tests/src/jmap/calendar/notification.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{IntoJmapSet, JMAPTest, JmapUtils, wait_for_index};\nuse calcard::jscalendar::JSCalendarProperty;\nuse jmap_proto::{\n    object::calendar_event_notification::CalendarEventNotificationProperty,\n    request::method::MethodObject,\n};\nuse mail_parser::DateTime;\nuse serde_json::{Value, json};\nuse store::write::now;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Calendar Event Notification tests...\");\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let bill = params.account(\"bill@example.com\");\n\n    let john_id = john.id_string().to_string();\n    let jane_id = jane.id_string().to_string();\n    let bill_id = bill.id_string().to_string();\n\n    let mut john_change_id = String::new();\n    let mut jane_change_id = String::new();\n    let mut bill_change_id = String::new();\n\n    // Obtain share notification change ids for all accounts\n    for (change_id, client) in [\n        (&mut john_change_id, john),\n        (&mut jane_change_id, jane),\n        (&mut bill_change_id, bill),\n    ] {\n        let response = client\n            .jmap_get(\n                MethodObject::CalendarEventNotification,\n                [CalendarEventNotificationProperty::Id],\n                Vec::<&str>::new(),\n            )\n            .await;\n        response.list_array().assert_is_equal(json!([]));\n        *change_id = response.state().to_string();\n    }\n\n    // Create test calendars\n    let response = john\n        .jmap_create(\n            MethodObject::Calendar,\n            [json!({\n                \"name\": \"Test Calendar\",\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let john_calendar_id = response.created(0).id().to_string();\n\n    // Sent invitation to Jane and Bill\n    let john_event = test_event();\n    let response = john\n        .jmap_create(\n            MethodObject::CalendarEvent,\n            [john_event.clone().with_property(\n                JSCalendarProperty::<Id>::CalendarIds,\n                [john_calendar_id.as_str()].into_jmap_set(),\n            )],\n            [(\"sendSchedulingMessages\", true)],\n        )\n        .await;\n    let john_event_id = response.created(0).id().to_string();\n\n    tokio::time::sleep(std::time::Duration::from_millis(600)).await;\n    wait_for_index(&params.server).await;\n\n    // Verify Jane and Bill received the share notification\n    let mut jane_event_id = String::new();\n    let mut bill_event_id = String::new();\n    for (change_id, event_id, client) in [\n        (&mut jane_change_id, &mut jane_event_id, jane),\n        (&mut bill_change_id, &mut bill_event_id, bill),\n    ] {\n        // Obtain changes\n        let response = client\n            .jmap_changes(MethodObject::CalendarEventNotification, &change_id)\n            .await;\n        let changes = response.changes().collect::<Vec<_>>();\n        assert_eq!(changes.len(), 1);\n        *change_id = response.new_state().to_string();\n        let notification_id = changes[0].as_created();\n\n        // Obtain and verify notification\n        let response = client\n            .jmap_get(\n                MethodObject::CalendarEventNotification,\n                [\n                    CalendarEventNotificationProperty::Id,\n                    CalendarEventNotificationProperty::Created,\n                    CalendarEventNotificationProperty::ChangedBy,\n                    CalendarEventNotificationProperty::Comment,\n                    CalendarEventNotificationProperty::Type,\n                    CalendarEventNotificationProperty::CalendarEventId,\n                    CalendarEventNotificationProperty::IsDraft,\n                    CalendarEventNotificationProperty::Event,\n                    CalendarEventNotificationProperty::EventPatch,\n                ],\n                [notification_id],\n            )\n            .await;\n        let notification = &response.list()[0];\n        *event_id = notification.text_field(\"calendarEventId\").to_string();\n        notification.assert_is_equal(json!({\n          \"id\": &notification_id,\n          \"created\": &notification.text_field(\"created\"),\n          \"changedBy\": {\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\",\n            \"principalId\": &john_id\n          },\n          \"type\": \"created\",\n          \"calendarEventId\": event_id,\n          \"isDraft\": false,\n          \"event\": john_event\n            .clone()\n            .with_property(\"sequence\", 1)\n            .with_property(\n                \"updated\",\n                notification\n                    .text_field(\"event/updated\")\n            )\n        }));\n\n        // Verify the event exists\n        let response = client\n            .jmap_get(\n                MethodObject::CalendarEvent,\n                [JSCalendarProperty::<Id>::Id, JSCalendarProperty::Title],\n                [&event_id],\n            )\n            .await;\n        response.list()[0].assert_is_equal(json!({\n          \"id\": &event_id,\n          \"title\": \"Lunch\"\n        }));\n    }\n\n    // Jane and Bill accept the invitation\n    let response = jane\n        .jmap_update(\n            MethodObject::CalendarEvent,\n            [(\n                &jane_event_id,\n                json!({\n             \"participants/a0171748-fe8d-57d8-879e-56036a5251d1/participationStatus\":\n             \"accepted\"}),\n            )],\n            [(\"sendSchedulingMessages\", true)],\n        )\n        .await;\n    response.updated(&jane_event_id);\n    let response = bill\n        .jmap_update(\n            MethodObject::CalendarEvent,\n            [(\n                &bill_event_id,\n                json!({\n             \"participants/86720268-d67c-58c3-9217-03df7d7ee4d8/participationStatus\":\n             \"accepted\"}),\n            )],\n            [(\"sendSchedulingMessages\", true)],\n        )\n        .await;\n    response.updated(&bill_event_id);\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n\n    // Verify John received two share notifications\n    let response = john\n        .jmap_changes(MethodObject::CalendarEventNotification, &john_change_id)\n        .await;\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 2);\n    for (i, change) in changes.into_iter().enumerate() {\n        let notification_id = change.as_created();\n\n        // Obtain and verify notification\n        let response = john\n            .jmap_get(\n                MethodObject::CalendarEventNotification,\n                [\n                    CalendarEventNotificationProperty::Id,\n                    CalendarEventNotificationProperty::ChangedBy,\n                    CalendarEventNotificationProperty::Comment,\n                    CalendarEventNotificationProperty::Type,\n                    CalendarEventNotificationProperty::CalendarEventId,\n                    CalendarEventNotificationProperty::IsDraft,\n                ],\n                [notification_id],\n            )\n            .await;\n        let changed_by = if i == 0 {\n            json!({\n                \"name\": \"Jane Smith\",\n                \"email\": \"jane.smith@example.com\",\n                \"principalId\": &jane_id,\n            })\n        } else {\n            json!({\n                \"name\": \"Bill Foobar\",\n                \"email\": \"bill@example.com\",\n                \"principalId\": &bill_id,\n            })\n        };\n\n        response.list()[0].assert_is_equal(json!({\n            \"id\": &notification_id,\n            \"changedBy\": changed_by,\n            \"type\": \"updated\",\n            \"calendarEventId\": &john_event_id,\n            \"isDraft\": false\n        }));\n    }\n\n    // Verify the event was updated\n    let response = john\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            [\n                JSCalendarProperty::<Id>::Id,\n                JSCalendarProperty::Title,\n                JSCalendarProperty::Participants,\n            ],\n            [&john_event_id],\n        )\n        .await;\n    response.list()[0].assert_is_equal(json!({\n        \"participants\": {\n        \"8584f8f9-5414-55e3-8a1c-ad6fc2f3ffb6\": {\n            \"calendarAddress\": \"mailto:jdoe@example.com\",\n            \"@type\": \"Participant\",\n            \"roles\": {\n                \"chair\": true\n            },\n            \"participationStatus\": \"accepted\"\n        },\n        \"a0171748-fe8d-57d8-879e-56036a5251d1\": {\n            \"calendarAddress\": \"mailto:jane.smith@example.com\",\n            \"@type\": \"Participant\",\n            \"participationStatus\": \"accepted\",\n            \"kind\": \"individual\"\n        },\n        \"86720268-d67c-58c3-9217-03df7d7ee4d8\": {\n            \"calendarAddress\": \"mailto:bill@example.com\",\n            \"@type\": \"Participant\",\n            \"kind\": \"individual\",\n            \"participationStatus\": \"accepted\"\n        }\n        },\n        \"title\": \"Lunch\",\n        \"id\": &john_event_id\n    }));\n\n    // Jane later declines the invitation\n    let response = jane\n        .jmap_update(\n            MethodObject::CalendarEvent,\n            [(\n                &jane_event_id,\n                json!({\n             \"participants/a0171748-fe8d-57d8-879e-56036a5251d1/participationStatus\":\n             \"declined\"}),\n            )],\n            [(\"sendSchedulingMessages\", true)],\n        )\n        .await;\n    response.updated(&jane_event_id);\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n\n    // Make sure John received the update\n    let response = john\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            [\n                JSCalendarProperty::<Id>::Id,\n                JSCalendarProperty::Title,\n                JSCalendarProperty::Participants,\n            ],\n            [&john_event_id],\n        )\n        .await;\n    response.list()[0].assert_is_equal(json!({\n        \"participants\": {\n        \"8584f8f9-5414-55e3-8a1c-ad6fc2f3ffb6\": {\n            \"calendarAddress\": \"mailto:jdoe@example.com\",\n            \"@type\": \"Participant\",\n            \"roles\": {\n                \"chair\": true\n            },\n            \"participationStatus\": \"accepted\"\n        },\n        \"a0171748-fe8d-57d8-879e-56036a5251d1\": {\n            \"calendarAddress\": \"mailto:jane.smith@example.com\",\n            \"@type\": \"Participant\",\n            \"participationStatus\": \"declined\",\n            \"kind\": \"individual\"\n        },\n        \"86720268-d67c-58c3-9217-03df7d7ee4d8\": {\n            \"calendarAddress\": \"mailto:bill@example.com\",\n            \"@type\": \"Participant\",\n            \"kind\": \"individual\",\n            \"participationStatus\": \"accepted\"\n        }\n        },\n        \"title\": \"Lunch\",\n        \"id\": &john_event_id\n    }));\n\n    // John deletes the event\n    let response = john\n        .jmap_destroy(\n            MethodObject::CalendarEvent,\n            [&john_event_id],\n            [(\"sendSchedulingMessages\", true)],\n        )\n        .await;\n    assert_eq!(response.destroyed().collect::<Vec<_>>(), [&john_event_id]);\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n\n    // Verify that only Bill received the cancellation\n    let response = jane\n        .jmap_changes(MethodObject::CalendarEventNotification, &jane_change_id)\n        .await;\n    assert_eq!(response.changes().next(), None);\n    let response = bill\n        .jmap_changes(MethodObject::CalendarEventNotification, &bill_change_id)\n        .await;\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let notification_id = changes[0].as_created();\n    let response = bill\n        .jmap_get(\n            MethodObject::CalendarEventNotification,\n            [\n                CalendarEventNotificationProperty::Id,\n                CalendarEventNotificationProperty::ChangedBy,\n                CalendarEventNotificationProperty::Comment,\n                CalendarEventNotificationProperty::Type,\n                CalendarEventNotificationProperty::CalendarEventId,\n                CalendarEventNotificationProperty::IsDraft,\n            ],\n            [notification_id],\n        )\n        .await;\n    response.list()[0].assert_is_equal(json!({\n        \"id\": &notification_id,\n        \"changedBy\": {\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\",\n            \"principalId\": &john_id\n        },\n        \"type\": \"updated\",\n        \"calendarEventId\": &bill_event_id,\n        \"isDraft\": false\n    }));\n\n    // Verify Bill's event was updated\n    let response = bill\n        .jmap_get(\n            MethodObject::CalendarEvent,\n            [\n                JSCalendarProperty::<Id>::Id,\n                JSCalendarProperty::Title,\n                JSCalendarProperty::Status,\n            ],\n            [&bill_event_id],\n        )\n        .await;\n    response.list()[0].assert_is_equal(json!({\n        \"id\": &bill_event_id,\n        \"title\": \"Lunch\",\n        \"status\": \"cancelled\"\n    }));\n\n    // Cleanup\n    for client in [john, jane, bill] {\n        client.destroy_all_calendars().await;\n        client.destroy_all_event_notifications().await;\n        params.destroy_all_mailboxes(client).await;\n    }\n    params.assert_is_empty().await;\n}\n\nfn test_event() -> Value {\n    json!({\n      \"uid\": \"9263504FD3AD\",\n      \"title\": \"Lunch\",\n      \"timeZone\": \"Europe/London\",\n      \"start\": DateTime::from_timestamp(now() as i64 + 60 * 60)\n        .to_rfc3339().trim_end_matches(\"Z\").to_string(),\n      \"duration\": \"PT1H\",\n      \"freeBusyStatus\": \"busy\",\n      \"updated\": \"2009-06-02T17:00:00Z\",\n      \"sequence\": 0,\n      \"@type\": \"Event\",\n      \"participants\": {\n        \"8584f8f9-5414-55e3-8a1c-ad6fc2f3ffb6\": {\n          \"calendarAddress\": \"mailto:jdoe@example.com\",\n          \"participationStatus\": \"accepted\",\n          \"roles\": {\n            \"chair\": true\n          },\n          \"@type\": \"Participant\"\n        },\n        \"a0171748-fe8d-57d8-879e-56036a5251d1\": {\n          \"calendarAddress\": \"mailto:jane.smith@example.com\",\n          \"@type\": \"Participant\",\n          \"participationStatus\": \"needs-action\",\n          \"kind\": \"individual\"\n        },\n        \"86720268-d67c-58c3-9217-03df7d7ee4d8\": {\n          \"calendarAddress\": \"mailto:bill@example.com\",\n          \"participationStatus\": \"needs-action\",\n          \"@type\": \"Participant\",\n          \"kind\": \"individual\"\n        }\n      },\n      \"organizerCalendarAddress\": \"mailto:jdoe@example.com\"\n    })\n}\n"
  },
  {
    "path": "tests/src/jmap/contacts/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, JmapUtils};\nuse calcard::jscontact::JSContactProperty;\nuse jmap_proto::{\n    object::{addressbook::AddressBookProperty, share_notification::ShareNotificationProperty},\n    request::method::MethodObject,\n};\nuse serde_json::json;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Contacts ACL tests...\");\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let john_id = john.id_string().to_string();\n    let jane_id = jane.id_string().to_string();\n\n    // Create test address books\n    let response = john\n        .jmap_create(\n            MethodObject::AddressBook,\n            [json!({\n                \"name\": \"Test #1\",\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let john_book_id = response.created(0).id().to_string();\n    let john_contact_id = john\n        .jmap_create(\n            MethodObject::ContactCard,\n            [json!({\n                \"uid\": \"abc123\",\n                \"name\": {\n                    \"full\": \"John's Simple Contact\",\n                },\n                \"addressBookIds\": {\n                    &john_book_id: true\n                },\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id()\n        .to_string();\n    let response = jane\n        .jmap_create(\n            MethodObject::AddressBook,\n            [json!({\n                \"name\": \"Test #1\",\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let jane_book_id = response.created(0).id().to_string();\n    let jane_contact_id = jane\n        .jmap_create(\n            MethodObject::ContactCard,\n            [json!({\n                \"uid\": \"abc456\",\n                \"name\": {\n                    \"full\": \"Jane's Simple Contact\",\n                },\n                \"addressBookIds\": {\n                    &jane_book_id: true\n                },\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id()\n        .to_string();\n\n    // Verify myRights\n    john.jmap_get(\n        MethodObject::AddressBook,\n        [\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::MyRights,\n            AddressBookProperty::ShareWith,\n        ],\n        [john_book_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_book_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n          \"mayRead\": true,\n          \"mayWrite\": true,\n          \"mayDelete\": true,\n          \"mayShare\": true\n        },\n        \"shareWith\": {}\n        }));\n\n    // Obtain share notifications\n    let mut jane_share_change_id = jane\n        .jmap_get(\n            MethodObject::ShareNotification,\n            Vec::<&str>::new(),\n            Vec::<&str>::new(),\n        )\n        .await\n        .state()\n        .to_string();\n\n    // Make sure Jane has no access\n    assert_eq!(\n        jane.jmap_get_account(\n            john,\n            MethodObject::AddressBook,\n            Vec::<&str>::new(),\n            [john_book_id.as_str()],\n        )\n        .await\n        .method_response()\n        .typ(),\n        \"forbidden\"\n    );\n\n    // Share address book with Jane\n    john.jmap_update(\n        MethodObject::AddressBook,\n        [(\n            &john_book_id,\n            json!({\n                \"shareWith\": {\n                   &jane_id : {\n                     \"mayRead\": true,\n                   }\n                }\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_book_id);\n    john.jmap_get(\n        MethodObject::AddressBook,\n        [\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::ShareWith,\n        ],\n        [john_book_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_book_id,\n        \"name\": \"Test #1\",\n        \"shareWith\": {\n            &jane_id : {\n                \"mayRead\": true,\n                \"mayWrite\": false,\n                \"mayDelete\": false,\n                \"mayShare\": false\n            }\n        }\n        }));\n\n    // Verify Jane can access the contact\n    jane.jmap_get_account(\n        john,\n        MethodObject::AddressBook,\n        [\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::MyRights,\n        ],\n        [john_book_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_book_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false\n        }\n        }));\n    jane.jmap_get_account(\n        john,\n        MethodObject::ContactCard,\n        [AddressBookProperty::Id, AddressBookProperty::Name],\n        [john_contact_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_contact_id,\n        \"name\": {\n                \"full\": \"John's Simple Contact\"\n            },\n        }));\n\n    // Verify Jane received a share notification\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    jane_share_change_id = response.new_state().to_string();\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"AddressBook\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_book_id,\n          \"oldRights\": {\n            \"mayRead\": false,\n            \"mayWrite\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false\n          },\n          \"newRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false\n          },\n          \"name\": null\n        }));\n\n    // Updating and deleting should fail\n    assert_eq!(\n        jane.jmap_update_account(\n            john,\n            MethodObject::AddressBook,\n            [(&john_book_id, json!({}))],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_updated(&john_book_id)\n        .description(),\n        \"You are not allowed to modify this address book.\"\n    );\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::AddressBook,\n            [&john_book_id],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_destroyed(&john_book_id)\n        .description(),\n        \"You are not allowed to delete this address book.\"\n    );\n    assert!(\n        jane.jmap_update_account(\n            john,\n            MethodObject::ContactCard,\n            [(&john_contact_id, json!({}))],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_updated(&john_contact_id)\n        .description()\n        .contains(\"You are not allowed to modify address book\"),\n    );\n    assert!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::ContactCard,\n            [&john_contact_id],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_destroyed(&john_contact_id)\n        .description()\n        .contains(\"You are not allowed to remove contacts from address book\"),\n    );\n\n    // Grant Jane write access\n    john.jmap_update(\n        MethodObject::AddressBook,\n        [(\n            &john_book_id,\n            json!({\n                format!(\"shareWith/{jane_id}/mayWrite\"): true,\n                format!(\"shareWith/{jane_id}/mayDelete\"): true,\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_book_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::AddressBook,\n        [\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::MyRights,\n        ],\n        [john_book_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_book_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": true,\n            \"mayDelete\": true,\n            \"mayShare\": false\n        }\n        }));\n\n    // Verify Jane received a share notification with the updated rights\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    jane_share_change_id = response.new_state().to_string();\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"AddressBook\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_book_id,\n          \"oldRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false\n          },\n          \"newRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": true,\n            \"mayDelete\": true,\n            \"mayShare\": false\n          },\n          \"name\": null\n        }));\n\n    // Creating a root folder should fail\n    assert_eq!(\n        jane.jmap_create_account(\n            john,\n            MethodObject::AddressBook,\n            [json!({\n                \"name\": \"A new shared address book\",\n            })],\n            Vec::<(&str, &str)>::new()\n        )\n        .await\n        .not_created(0)\n        .description(),\n        \"Cannot create address books in a shared account.\"\n    );\n\n    // Copy Jane's contact into John's address book\n    let john_copied_contact_id = jane\n        .jmap_copy(\n            jane,\n            john,\n            MethodObject::ContactCard,\n            [(\n                &jane_contact_id,\n                json!({\n                    \"addressBookIds\": {\n                        &john_book_id: true\n                    }\n                }),\n            )],\n            false,\n        )\n        .await\n        .copied(&jane_contact_id)\n        .id()\n        .to_string();\n    jane.jmap_get_account(\n        john,\n        MethodObject::ContactCard,\n        [\n            JSContactProperty::<Id>::Id,\n            JSContactProperty::AddressBookIds,\n            JSContactProperty::Name,\n        ],\n        [john_copied_contact_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_copied_contact_id,\n        \"name\": {\n                \"full\": \"Jane's Simple Contact\"\n            },\n        \"addressBookIds\": {\n            &john_book_id: true\n        }\n        }));\n\n    // Destroy the copied contact\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::ContactCard,\n            [john_copied_contact_id.as_str()],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .destroyed()\n        .collect::<Vec<_>>(),\n        [&john_copied_contact_id]\n    );\n\n    // Update John's contact\n    jane.jmap_update_account(\n        john,\n        MethodObject::ContactCard,\n        [(\n            &john_contact_id,\n            json!({\n                \"name\": {\n                    \"full\": \"John's Updated Contact\",\n                }\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_contact_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::ContactCard,\n        [JSContactProperty::<Id>::Id, JSContactProperty::Name],\n        [john_contact_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_contact_id,\n        \"name\": {\n                \"full\": \"John's Updated Contact\"\n            },\n        }));\n\n    // Update John's address book name\n    jane.jmap_update_account(\n        john,\n        MethodObject::AddressBook,\n        [(\n            &john_book_id,\n            json!({\n                \"name\": \"Jane's version of John's Address Book\",\n                \"description\": \"This is John's address book, but Jane can edit it now\"\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_book_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::AddressBook,\n        [\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::Description,\n        ],\n        [john_book_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_book_id,\n        \"name\": \"Jane's version of John's Address Book\",\n        \"description\": \"This is John's address book, but Jane can edit it now\"\n        }));\n\n    // John should still see the old name\n    john.jmap_get(\n        MethodObject::AddressBook,\n        [\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::Description,\n        ],\n        [john_book_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_book_id,\n        \"name\": \"Test #1\",\n        \"description\": null\n        }));\n\n    // Revoke Jane's access\n    john.jmap_update(\n        MethodObject::AddressBook,\n        [(\n            &john_book_id,\n            json!({\n                format!(\"shareWith/{jane_id}\"): ()\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_book_id);\n    john.jmap_get(\n        MethodObject::AddressBook,\n        [\n            AddressBookProperty::Id,\n            AddressBookProperty::Name,\n            AddressBookProperty::ShareWith,\n        ],\n        [john_book_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_book_id,\n        \"name\": \"Test #1\",\n        \"shareWith\": {}\n        }));\n\n    // Verify Jane can no longer access the address book or its contacts\n    assert_eq!(\n        jane.jmap_get_account(\n            john,\n            MethodObject::AddressBook,\n            Vec::<&str>::new(),\n            [john_book_id.as_str()],\n        )\n        .await\n        .method_response()\n        .typ(),\n        \"forbidden\"\n    );\n\n    // Verify Jane received a share notification with the updated rights\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"AddressBook\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_book_id,\n          \"oldRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": true,\n            \"mayDelete\": true,\n            \"mayShare\": false\n          },\n          \"newRights\": {\n            \"mayRead\": false,\n            \"mayWrite\": false,\n            \"mayDelete\": false,\n            \"mayShare\": false\n          },\n          \"name\": null\n        }));\n\n    // Grant Jane delete access once again\n    john.jmap_update(\n        MethodObject::AddressBook,\n        [(\n            &john_book_id,\n            json!({\n                format!(\"shareWith/{jane_id}/mayRead\"): true,\n                format!(\"shareWith/{jane_id}/mayDelete\"): true,\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_book_id);\n\n    // Verify Jane can delete the address book\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::AddressBook,\n            [john_book_id.as_str()],\n            [(\"onDestroyRemoveContents\", true)],\n        )\n        .await\n        .destroyed()\n        .collect::<Vec<_>>(),\n        [john_book_id.as_str()]\n    );\n\n    // Destroy all mailboxes\n    john.destroy_all_addressbooks().await;\n    jane.destroy_all_addressbooks().await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/contacts/addressbook.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse jmap_proto::{object::addressbook::AddressBookProperty, request::method::MethodObject};\nuse serde_json::json;\n\nuse crate::jmap::{ChangeType, JMAPTest, JmapUtils};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running AddressBook tests...\");\n    let account = params.account(\"jdoe@example.com\");\n\n    // Make sure the default address book exists\n    let response = account\n        .jmap_get(\n            MethodObject::AddressBook,\n            [\n                AddressBookProperty::Id,\n                AddressBookProperty::Name,\n                AddressBookProperty::Description,\n                AddressBookProperty::SortOrder,\n                AddressBookProperty::IsSubscribed,\n                AddressBookProperty::IsDefault,\n            ],\n            Vec::<&str>::new(),\n        )\n        .await;\n    let list = response.list();\n    assert_eq!(list.len(), 1);\n    let default_addressbook_id = list[0].id().to_string();\n    assert_eq!(\n        list[0],\n        json!({\n            \"name\": \"Stalwart Address Book (jdoe@example.com)\",\n            \"description\": (),\n            \"sortOrder\": 0,\n            \"isSubscribed\": false,\n            \"isDefault\": true,\n            \"id\": default_addressbook_id,\n        })\n    );\n    let change_id = response.state();\n\n    // Create Address Book\n    let addressbook_id = account\n        .jmap_create(\n            MethodObject::AddressBook,\n            [json!({\n                \"name\": \"Test address book\",\n                \"description\": \"My personal address book\",\n                \"sortOrder\": 1,\n                \"isSubscribed\": true\n\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id()\n        .to_string();\n\n    // Validate changes\n    assert_eq!(\n        account\n            .jmap_changes(MethodObject::AddressBook, change_id)\n            .await\n            .changes()\n            .collect::<Vec<_>>(),\n        [ChangeType::Created(&addressbook_id)]\n    );\n\n    // Get Address Book\n    let response = account\n        .jmap_get(\n            MethodObject::AddressBook,\n            [\n                AddressBookProperty::Id,\n                AddressBookProperty::Name,\n                AddressBookProperty::Description,\n                AddressBookProperty::SortOrder,\n                AddressBookProperty::IsSubscribed,\n                AddressBookProperty::IsDefault,\n            ],\n            [&addressbook_id],\n        )\n        .await;\n    assert_eq!(\n        response.list()[0],\n        json!({\n            \"name\": \"Test address book\",\n            \"description\": \"My personal address book\",\n            \"sortOrder\": 1,\n            \"isSubscribed\": true,\n            \"isDefault\": false,\n            \"id\": addressbook_id,\n        })\n    );\n\n    // Update Address Book and set it as default\n    account\n        .jmap_update(\n            MethodObject::AddressBook,\n            [(\n                addressbook_id.as_str(),\n                json!({\n                    \"name\": \"Updated address book\",\n                    \"description\": \"My updated personal address book\",\n                    \"sortOrder\": 2,\n                    \"isSubscribed\": false\n                }),\n            )],\n            [(\"onSuccessSetIsDefault\", addressbook_id.as_str())],\n        )\n        .await\n        .updated(&addressbook_id);\n\n    // Validate changes\n    assert_eq!(\n        account\n            .jmap_get(\n                MethodObject::AddressBook,\n                [\n                    AddressBookProperty::Id,\n                    AddressBookProperty::Name,\n                    AddressBookProperty::Description,\n                    AddressBookProperty::SortOrder,\n                    AddressBookProperty::IsSubscribed,\n                    AddressBookProperty::IsDefault,\n                ],\n                [&addressbook_id, &default_addressbook_id],\n            )\n            .await\n            .list(),\n        vec![\n            json!({\n                \"name\": \"Updated address book\",\n                \"description\": \"My updated personal address book\",\n                \"sortOrder\": 2,\n                \"isSubscribed\": false,\n                \"isDefault\": true,\n                \"id\": addressbook_id,\n            }),\n            json!({\n                \"name\": \"Stalwart Address Book (jdoe@example.com)\",\n                \"description\": (),\n                \"sortOrder\": 0,\n                \"isSubscribed\": false,\n                \"isDefault\": false,\n                \"id\": default_addressbook_id,\n            })\n        ]\n    );\n\n    // Create a contact\n    let _ = account\n        .jmap_create(\n            MethodObject::ContactCard,\n            [json!({\n              \"addressBookIds\": {\n                &addressbook_id: true\n              },\n              \"name\": {\n                \"components\": [\n                  { \"kind\": \"given\", \"value\": \"Joe\" },\n                  { \"kind\": \"surname\", \"value\": \"Bloggs\" }\n                ]\n              },\n              \"emails\": {\n                \"0\": {\n                  \"address\": \"joe.bloggs@example.com\"\n                }\n              }\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .created(0)\n        .id();\n\n    // Try destroying the address book (should fail)\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::AddressBook,\n                [&addressbook_id],\n                Vec::<(&str, &str)>::new(),\n            )\n            .await\n            .not_destroyed(&addressbook_id)\n            .typ(),\n        \"addressBookHasContents\"\n    );\n\n    // Destroy using force\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::AddressBook,\n                [&addressbook_id],\n                [(\"onDestroyRemoveContents\", true)],\n            )\n            .await\n            .destroyed()\n            .collect::<Vec<_>>(),\n        vec![&addressbook_id]\n    );\n\n    // Destroy all mailboxes\n    account.destroy_all_addressbooks().await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/contacts/contact.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    jmap::{ChangeType, IntoJmapSet, JMAPTest, JmapUtils, wait_for_index},\n    webdav::DummyWebDavClient,\n};\nuse ahash::AHashSet;\nuse calcard::jscontact::JSContactProperty;\nuse groupware::cache::GroupwareCache;\nuse hyper::StatusCode;\nuse jmap_proto::request::method::MethodObject;\nuse serde_json::{Value, json};\nuse types::{collection::SyncCollection, id::Id};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Contact Card tests...\");\n    let account = params.account(\"jdoe@example.com\");\n\n    // Create test address books\n    let response = account\n        .jmap_create(\n            MethodObject::AddressBook,\n            [\n                json!({\n                    \"name\": \"Test #1\",\n                }),\n                json!({\n                    \"name\": \"Test #2\",\n                }),\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let book1_id = response.created(0).id().to_string();\n    let book2_id = response.created(1).id().to_string();\n\n    // Obtain state\n    let change_id = account\n        .jmap_get(\n            MethodObject::ContactCard,\n            Vec::<&str>::new(),\n            Vec::<&str>::new(),\n        )\n        .await\n        .state()\n        .to_string();\n\n    // Create test contacts\n    let sarah_contact = test_jscontact_1().with_property(\n        JSContactProperty::<Id>::AddressBookIds,\n        [book1_id.as_str()].into_jmap_set(),\n    );\n    let carlos_contact = test_jscontact_2().with_property(\n        JSContactProperty::<Id>::AddressBookIds,\n        [book2_id.as_str()].into_jmap_set(),\n    );\n    let acme_contact = test_jscontact_3().with_property(\n        JSContactProperty::<Id>::AddressBookIds,\n        [book1_id.as_str(), book2_id.as_str()].into_jmap_set(),\n    );\n    let tmp_contact = test_jscontact_4().with_property(\n        JSContactProperty::<Id>::AddressBookIds,\n        [book2_id.as_str()].into_jmap_set(),\n    );\n    let response = account\n        .jmap_create(\n            MethodObject::ContactCard,\n            [\n                sarah_contact.clone(),\n                carlos_contact.clone(),\n                acme_contact.clone(),\n                tmp_contact,\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let sarah_contact_id = response.created(0).id().to_string();\n    let carlos_contact_id = response.created(1).id().to_string();\n    let acme_contact_id = response.created(2).id().to_string();\n    let tmp_contact_id = response.created(3).id().to_string();\n\n    // Destroy tmp contact\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::ContactCard,\n                [tmp_contact_id.as_str()],\n                Vec::<(&str, &str)>::new(),\n            )\n            .await\n            .destroyed()\n            .next(),\n        Some(tmp_contact_id.as_str())\n    );\n\n    // Validate changes\n    assert_eq!(\n        account\n            .jmap_changes(MethodObject::ContactCard, &change_id)\n            .await\n            .changes()\n            .collect::<AHashSet<_>>(),\n        [\n            ChangeType::Created(&sarah_contact_id),\n            ChangeType::Created(&carlos_contact_id),\n            ChangeType::Created(&acme_contact_id)\n        ]\n        .into_iter()\n        .collect::<AHashSet<_>>(),\n    );\n\n    // Fetch contacts and verify\n    let response = account\n        .jmap_get(\n            MethodObject::ContactCard,\n            Vec::<&str>::new(),\n            [&sarah_contact_id, &carlos_contact_id, &acme_contact_id],\n        )\n        .await;\n    response.list()[0].assert_is_equal(\n        sarah_contact.with_property(JSContactProperty::<Id>::Id, sarah_contact_id.as_str()),\n    );\n    response.list()[1].assert_is_equal(\n        carlos_contact.with_property(JSContactProperty::<Id>::Id, carlos_contact_id.as_str()),\n    );\n    response.list()[2].assert_is_equal(\n        acme_contact.with_property(JSContactProperty::<Id>::Id, acme_contact_id.as_str()),\n    );\n\n    // Creating a contact without address book should fail\n    assert_eq!(\n        account\n            .jmap_create(\n                MethodObject::ContactCard,\n                [json!({\n                    \"name\": {\n                        \"full\": \"Simple Contact\",\n                    },\n                    \"addressBookIds\": {},\n                }),],\n                Vec::<(&str, &str)>::new()\n            )\n            .await\n            .not_created(0)\n            .description(),\n        \"Contact has to belong to at least one address book.\"\n    );\n\n    // Creating a contact with a duplicate UID should fail\n    assert!(\n        account\n            .jmap_create(\n                MethodObject::ContactCard,\n                [json!({\n                    \"uid\": \"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6\",\n                    \"name\": {\n                        \"full\": \"Simple Contact\",\n                    },\n                    \"addressBookIds\": {\n                        &book1_id: true\n                    },\n                }),],\n                Vec::<(&str, &str)>::new()\n            )\n            .await\n            .not_created(0)\n            .description()\n            .contains(\n                \"Contact with UID urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 already exists\"\n            ),\n    );\n\n    // Patching tests\n    let response = account\n        .jmap_update(\n            MethodObject::ContactCard,\n            [\n                (\n                    &sarah_contact_id,\n                    json!({\n                        \"name/full\": \"Sarah O'Connor\",\n                        \"name/components/0/value\": \"O'Connor\",\n                        format!(\"addressBookIds/{book2_id}\"): true\n                    }),\n                ),\n                (\n                    &carlos_contact_id,\n                    json!({\n                        \"addressBookIds\": {\n                            &book1_id: true,\n                            &book2_id: true\n                        },\n                        \"nicknames/k1\": (),\n                        \"nicknames/k2\": {\n                            \"name\": \"Carlitos\"\n                        },\n                    }),\n                ),\n                (\n                    &acme_contact_id,\n                    json!({\n                        format!(\"addressBookIds/{book2_id}\"): false,\n                        \"keywords/B2B\": false,\n                        \"keywords/B2C\": true,\n                    }),\n                ),\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    response.updated(&sarah_contact_id);\n    response.updated(&carlos_contact_id);\n    response.updated(&acme_contact_id);\n\n    // Verify patches\n    let response = account\n        .jmap_get(\n            MethodObject::ContactCard,\n            [\n                JSContactProperty::<Id>::Id,\n                JSContactProperty::AddressBookIds,\n                JSContactProperty::Name,\n                JSContactProperty::Keywords,\n                JSContactProperty::Nicknames,\n            ],\n            [&sarah_contact_id, &carlos_contact_id, &acme_contact_id],\n        )\n        .await;\n\n    response.list()[0].assert_is_equal(json!({\n      \"id\": &sarah_contact_id,\n      \"name\": {\n        \"full\": \"Sarah O'Connor\",\n        \"components\": [\n          {\n            \"kind\": \"surname\",\n            \"value\": \"O'Connor\"\n          },\n          {\n            \"kind\": \"given\",\n            \"value\": \"Sarah\"\n          },\n          {\n            \"kind\": \"given2\",\n            \"value\": \"Marie\"\n          },\n          {\n            \"kind\": \"title\",\n            \"value\": \"Dr.\"\n          },\n          {\n            \"kind\": \"credential\",\n            \"value\": \"Ph.D.\"\n          }\n        ],\n        \"isOrdered\": true\n      },\n      \"nicknames\": {\n        \"k1\": {\n            \"name\": \"Sadie\"\n         }\n      },\n      \"keywords\": {\n        \"Work\": true,\n        \"Research\": true,\n        \"VIP\": true\n      },\n      \"addressBookIds\":  {\n        &book1_id: true,\n        &book2_id: true\n      },\n    }));\n\n    response.list()[1].assert_is_equal(json!({\n      \"id\": &carlos_contact_id,\n      \"name\": {\n        \"components\": [\n          {\n            \"kind\": \"surname\",\n            \"value\": \"Rodriguez-Martinez\"\n          },\n          {\n            \"kind\": \"given\",\n            \"value\": \"Carlos\"\n          },\n          {\n            \"kind\": \"given2\",\n            \"value\": \"Alberto\"\n          },\n          {\n            \"kind\": \"title\",\n            \"value\": \"Mr.\"\n          },\n          {\n            \"kind\": \"credential\",\n            \"value\": \"Jr.\"\n          }\n        ],\n        \"isOrdered\": true,\n        \"full\": \"Carlos Rodriguez-Martinez\"\n      },\n      \"keywords\": {\n        \"Marketing\": true,\n        \"Management\": true,\n        \"International\": true\n      },\n      \"nicknames\": {\n        \"k2\": {\n        \"name\": \"Carlitos\"\n        }\n      },\n      \"addressBookIds\": {\n        &book1_id: true,\n        &book2_id: true\n      },\n    }));\n\n    response.list()[2].assert_is_equal(json!({\n        \"id\": acme_contact_id,\n        \"addressBookIds\": {\n            &book1_id: true,\n        },\n        \"name\": {\n            \"full\": \"Acme Business Solutions Ltd.\"\n        },\n        \"keywords\": {\n            \"Technology\": true,\n            \"B2C\": true,\n            \"Solutions\": true,\n            \"Services\": true\n        }\n    }));\n\n    // Query tests\n    wait_for_index(&params.server).await;\n    let email = if !params.server.search_store().is_mysql() {\n        \"sarah.johnson@example.com\"\n    } else {\n        \"sarah.johnson@example\"\n    };\n    assert_eq!(\n        account\n            .jmap_query(\n                MethodObject::ContactCard,\n                [\n                    (\"text\", \"Sarah\"),\n                    (\"inAddressBook\", book1_id.as_str()),\n                    (\"uid\", \"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6\"),\n                    (\"email\", email),\n                ],\n                [\"created\"],\n                Vec::<(&str, &str)>::new(),\n            )\n            .await\n            .ids()\n            .collect::<AHashSet<_>>(),\n        [sarah_contact_id.as_str()]\n            .into_iter()\n            .collect::<AHashSet<_>>()\n    );\n\n    // Parse tests\n    account\n        .jmap_method_calls(json!([\n         [\n          \"Blob/upload\",\n          {\n           \"create\": {\n            \"vcard\": {\n             \"data\": [\n              {\n               \"data:asText\": r#\"BEGIN:VCARD\nVERSION:4.0\nKIND:individual\nFN:Jane Doe\nORG:ABC\\, Inc.;North American Division;Marketing\nEND:VCARD\"#\n              }\n            ]\n           }\n          }\n         },\n         \"S4\"\n        ],\n        [\n          \"ContactCard/parse\",\n          {\n           \"blobIds\": [\n             \"#vcard\"\n           ]\n          },\n          \"G4\"\n         ]\n        ]))\n        .await\n        .pointer(\"/methodResponses/1/1/parsed\")\n        .unwrap()\n        .as_object()\n        .unwrap()\n        .iter()\n        .next()\n        .unwrap()\n        .1\n        .assert_is_equal(json!({\n          \"name\": {\n            \"full\": \"Jane Doe\"\n          },\n          \"version\": \"1.0\",\n          \"vCard\": {\n            \"properties\": [\n              [\n                \"version\",\n                {},\n                \"unknown\",\n                \"4.0\"\n              ]\n            ]\n          },\n          \"organizations\": {\n            \"k1\": {\n              \"name\": \"ABC, Inc.\",\n              \"units\": [\n                {\n                  \"name\": \"North American Division\"\n                },\n                {\n                  \"name\": \"Marketing\"\n                }\n              ]\n            }\n          },\n          \"@type\": \"Card\",\n          \"kind\": \"individual\"\n        }));\n\n    // Deletion tests\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::ContactCard,\n                [carlos_contact_id.as_str(), acme_contact_id.as_str()],\n                Vec::<(&str, &str)>::new()\n            )\n            .await\n            .destroyed()\n            .collect::<AHashSet<_>>(),\n        [carlos_contact_id.as_str(), acme_contact_id.as_str()]\n            .into_iter()\n            .collect::<AHashSet<_>>()\n    );\n\n    // CardDAV compatibility tests\n    let account_id = account.id().document_id();\n    let dav_client = DummyWebDavClient::new(\n        u32::MAX,\n        account.name(),\n        account.secret(),\n        account.emails()[0],\n    );\n    let resources = params\n        .server\n        .fetch_dav_resources(\n            &params.server.get_access_token(account_id).await.unwrap(),\n            account_id,\n            SyncCollection::AddressBook,\n        )\n        .await\n        .unwrap();\n    let path = format!(\n        \"{}{}\",\n        resources.base_path,\n        resources\n            .paths\n            .iter()\n            .find(|v| v.parent_id.is_some())\n            .unwrap()\n            .path\n    );\n    let vcard = dav_client\n        .request(\"GET\", &path, \"\")\n        .await\n        .with_status(StatusCode::OK)\n        .expect_body()\n        .lines()\n        .map(String::from)\n        .collect::<AHashSet<_>>();\n    let expected_vcard = TEST_VCARD_1\n        .lines()\n        .map(String::from)\n        .collect::<AHashSet<_>>();\n    assert_eq!(vcard, expected_vcard);\n\n    // Clean up\n    account.destroy_all_addressbooks().await;\n    params.assert_is_empty().await;\n}\n\nfn test_jscontact_1() -> Value {\n    json!({\n      \"uid\": \"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6\",\n      \"@type\": \"Card\",\n      \"preferredLanguages\": {\n        \"k1\": {\n          \"language\": \"en\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 1\n        },\n        \"k2\": {\n          \"language\": \"fr\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 2\n        }\n      },\n      \"name\": {\n        \"full\": \"Sarah Johnson\",\n        \"components\": [\n          {\n            \"kind\": \"surname\",\n            \"value\": \"Johnson\"\n          },\n          {\n            \"kind\": \"given\",\n            \"value\": \"Sarah\"\n          },\n          {\n            \"kind\": \"given2\",\n            \"value\": \"Marie\"\n          },\n          {\n            \"kind\": \"title\",\n            \"value\": \"Dr.\"\n          },\n          {\n            \"kind\": \"credential\",\n            \"value\": \"Ph.D.\"\n          }\n        ],\n        \"isOrdered\": true\n      },\n      \"cryptoKeys\": {\n        \"k1\": {\n          \"uri\": \"https://pgp.example.com/pks/lookup?op=get&search=sarah.johnson@example.com\",\n          \"contexts\": {\n            \"pgp\": true\n          }\n        }\n      },\n      \"keywords\": {\n        \"Work\": true,\n        \"Research\": true,\n        \"VIP\": true\n      },\n      \"anniversaries\": {\n        \"k1\": {\n          \"date\": {\n            \"@type\": \"PartialDate\",\n            \"year\": 1985,\n            \"month\": 4,\n            \"day\": 15\n          },\n          \"kind\": \"birth\"\n        },\n        \"k2\": {\n          \"date\": {\n            \"@type\": \"PartialDate\",\n            \"year\": 2010,\n            \"month\": 6,\n            \"day\": 10\n          },\n          \"kind\": \"wedding\"\n        }\n      },\n      \"links\": {\n        \"k1\": {\n          \"uri\": \"https://www.example.com/staff/sjohnson\",\n          \"contexts\": {\n            \"work\": true\n          }\n        },\n        \"k2\": {\n          \"uri\": \"https://www.sarahjohnson.example.com\",\n          \"contexts\": {\n            \"private\": true\n          }\n        }\n      },\n      \"organizations\": {\n        \"k1\": {\n          \"name\": \"Acme Technologies Inc.\",\n          \"units\": [\n            {\n              \"name\": \"Research Department\"\n            }\n          ]\n        }\n      },\n      \"emails\": {\n        \"k1\": {\n          \"address\": \"sarah.johnson@example.com\",\n          \"contexts\": {\n            \"work\": true\n          }\n        },\n        \"k2\": {\n          \"address\": \"sarahjpersonal@example.com\",\n          \"contexts\": {\n            \"private\": true,\n            \"pref\": true\n          }\n        }\n      },\n      \"phones\": {\n        \"k1\": {\n          \"number\": \"+1-555-123-4567\",\n          \"contexts\": {\n            \"pref\": true\n          },\n          \"features\": {\n            \"mobile\": true,\n            \"voice\": true\n          }\n        },\n        \"k2\": {\n          \"number\": \"+1-555-987-6543\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"features\": {\n            \"voice\": true\n          }\n        },\n        \"k3\": {\n          \"number\": \"+1-555-456-7890\",\n          \"contexts\": {\n            \"private\": true\n          },\n          \"features\": {\n            \"voice\": true\n          }\n        }\n      },\n      \"version\": \"1.0\",\n      \"addresses\": {\n        \"k1\": {\n          \"contexts\": {\n            \"work\": true\n          },\n          \"full\": \"123 Business Ave\\nSuite 400\\nNew York, NY 10001\\nUSA\",\n          \"components\": [\n            {\n              \"kind\": \"name\",\n              \"value\": \"123 Business Ave\"\n            },\n            {\n              \"kind\": \"locality\",\n              \"value\": \"New York\"\n            },\n            {\n              \"kind\": \"region\",\n              \"value\": \"NY\"\n            },\n            {\n              \"kind\": \"postcode\",\n              \"value\": \"10001\"\n            },\n            {\n              \"kind\": \"country\",\n              \"value\": \"USA\"\n            }\n          ],\n          \"timeZone\": \"Etc/GMT+5\",\n          \"coordinates\": \"40.7128;-74.0060\",\n          \"isOrdered\": true\n        },\n        \"k2\": {\n          \"contexts\": {\n            \"private\": true,\n            \"pref\": true\n          },\n          \"full\": \"456 Residential St\\nApt 7B\\nBrooklyn, NY 11201\\nUSA\",\n          \"components\": [\n            {\n              \"kind\": \"name\",\n              \"value\": \"456 Residential St\"\n            },\n            {\n              \"kind\": \"locality\",\n              \"value\": \"Brooklyn\"\n            },\n            {\n              \"kind\": \"region\",\n              \"value\": \"NY\"\n            },\n            {\n              \"kind\": \"postcode\",\n              \"value\": \"11201\"\n            },\n            {\n              \"kind\": \"country\",\n              \"value\": \"USA\"\n            }\n          ],\n          \"isOrdered\": true\n        }\n      },\n      \"titles\": {\n        \"k1\": {\n          \"name\": \"Senior Research Scientist\",\n          \"kind\": \"title\"\n        },\n        \"k2\": {\n          \"name\": \"Team Lead\",\n          \"kind\": \"role\",\n          \"organizationId\": \"k1\"\n        }\n      },\n      \"nicknames\": {\n        \"k1\": {\n          \"name\": \"Sadie\"\n        }\n      },\n      \"notes\": {\n        \"k1\": {\n          \"note\": \"Sarah prefers video calls over phone calls. Available Mon-Thu 9-5 EST.\"\n        }\n      },\n      \"updated\": \"2022-03-15T13:30:00Z\"\n    })\n}\n\nfn test_jscontact_2() -> Value {\n    json!({\n      \"phones\": {\n        \"k1\": {\n          \"number\": \"+34-611-234-567\",\n          \"contexts\": {\n            \"pref\": true\n          },\n          \"features\": {\n            \"mobile\": true,\n            \"voice\": true\n          }\n        },\n        \"k2\": {\n          \"number\": \"+34-911-876-543\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"features\": {\n            \"voice\": true\n          }\n        },\n        \"k3\": {\n          \"number\": \"+34-644-321-987\",\n          \"contexts\": {\n            \"private\": true\n          },\n          \"features\": {\n            \"voice\": true\n          }\n        },\n        \"k4\": {\n          \"number\": \"+34-911-876-544\",\n          \"features\": {\n            \"fax\": true\n          }\n        }\n      },\n      \"keywords\": {\n        \"Marketing\": true,\n        \"Management\": true,\n        \"International\": true\n      },\n      \"kind\": \"individual\",\n      \"anniversaries\": {\n        \"k1\": {\n          \"date\": {\n            \"@type\": \"PartialDate\",\n            \"month\": 6,\n            \"day\": 23\n          },\n          \"kind\": \"birth\"\n        },\n        \"k2\": {\n          \"date\": {\n            \"@type\": \"PartialDate\",\n            \"year\": 2015,\n            \"month\": 8,\n            \"day\": 9\n          },\n          \"kind\": \"wedding\"\n        }\n      },\n      \"members\": {\n        \"urn:uuid:03a0e51f-d1aa-4385-8a53-e29025acd8af\": true\n      },\n      \"uid\": \"urn:uuid:e1ee798b-3d4c-41b0-b217-b9c918e4686a\",\n      \"name\": {\n        \"components\": [\n          {\n            \"kind\": \"surname\",\n            \"value\": \"Rodriguez-Martinez\"\n          },\n          {\n            \"kind\": \"given\",\n            \"value\": \"Carlos\"\n          },\n          {\n            \"kind\": \"given2\",\n            \"value\": \"Alberto\"\n          },\n          {\n            \"kind\": \"title\",\n            \"value\": \"Mr.\"\n          },\n          {\n            \"kind\": \"credential\",\n            \"value\": \"Jr.\"\n          }\n        ],\n        \"full\": \"Carlos Rodriguez-Martinez\",\n        \"isOrdered\": true\n      },\n      \"nicknames\": {\n        \"k1\": {\n          \"name\": \"Charlie\"\n        }\n      },\n      \"relatedTo\": {\n        \"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6\": {\n          \"relation\": {\n            \"friend\": true\n          }\n        }\n      },\n      \"emails\": {\n        \"k1\": {\n          \"address\": \"carlos.rodriguez@example-corp.com\",\n          \"contexts\": {\n            \"work\": true,\n            \"pref\": true\n          }\n        },\n        \"k2\": {\n          \"address\": \"carlosrm@personalmail.example\",\n          \"contexts\": {\n            \"private\": true\n          }\n        }\n      },\n      \"directories\": {\n        \"k1\": {\n          \"uri\": \"https://contacts.example.com/carlosrodriguez.vcf\",\n          \"kind\": \"entry\"\n        }\n      },\n      \"cryptoKeys\": {\n        \"k1\": {\n          \"uri\": \"https://pgp.example.com/pks/lookup?op=get&search=carlos.rodriguez@example-corp.com\",\n          \"contexts\": {\n            \"pgp\": true\n          }\n        }\n      },\n      \"version\": \"1.0\",\n      \"notes\": {\n        \"k1\": {\n          \"note\": \"Carlos speaks English, Spanish, and Portuguese fluently. Prefers communication via email. Do not contact after 7PM CET.\"\n        }\n      },\n      \"updated\": \"2023-07-12T09:21:35Z\",\n      \"links\": {\n        \"k1\": {\n          \"uri\": \"https://www.example-corp.com/team/carlos\",\n          \"contexts\": {\n            \"work\": true\n          }\n        },\n        \"k2\": {\n          \"uri\": \"https://www.carlosrodriguez.example\",\n          \"contexts\": {\n            \"private\": true\n          }\n        },\n        \"k3\": {\n          \"uri\": \"https://linkedin.com/in/carlosrodriguezm\",\n          \"contexts\": {\n            \"social\": true\n          }\n        }\n      },\n      \"@type\": \"Card\",\n      \"titles\": {\n        \"k1\": {\n          \"name\": \"Digital Marketing Director\",\n          \"kind\": \"title\"\n        },\n        \"k2\": {\n          \"name\": \"Department Head\",\n          \"kind\": \"role\",\n          \"organizationId\": \"k1\"\n        }\n      },\n      \"preferredLanguages\": {\n        \"k1\": {\n          \"language\": \"es\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 1\n        },\n        \"k2\": {\n          \"language\": \"en\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 2\n        },\n        \"k3\": {\n          \"language\": \"pt\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 3\n        }\n      },\n      \"addresses\": {\n        \"k1\": {\n          \"contexts\": {\n            \"work\": true\n          },\n          \"full\": \"Calle Empresarial 42\\nPlanta 3\\nMadrid, 28001\\nSpain\",\n          \"components\": [\n            {\n              \"kind\": \"name\",\n              \"value\": \"Calle Empresarial 42\"\n            },\n            {\n              \"kind\": \"locality\",\n              \"value\": \"Madrid\"\n            },\n            {\n              \"kind\": \"postcode\",\n              \"value\": \"28001\"\n            },\n            {\n              \"kind\": \"country\",\n              \"value\": \"Spain\"\n            }\n          ],\n          \"timeZone\": \"Etc/GMT-1\",\n          \"coordinates\": \"40.4168;-3.7038\",\n          \"isOrdered\": true\n        },\n        \"k2\": {\n          \"contexts\": {\n            \"private\": true,\n            \"pref\": true\n          },\n          \"full\": \"Avenida Residencial 15\\nPiso 7, Puerta C\\nMadrid, 28045\\nSpain\",\n          \"components\": [\n            {\n              \"kind\": \"name\",\n              \"value\": \"Avenida Residencial 15\"\n            },\n            {\n              \"kind\": \"locality\",\n              \"value\": \"Madrid\"\n            },\n            {\n              \"kind\": \"postcode\",\n              \"value\": \"28045\"\n            },\n            {\n              \"kind\": \"country\",\n              \"value\": \"Spain\"\n            }\n          ],\n          \"isOrdered\": true\n        }\n      },\n      \"organizations\": {\n        \"k1\": {\n          \"name\": \"Global Solutions S.L.\",\n          \"units\": [\n            {\n              \"name\": \"Marketing Division\"\n            }\n          ]\n        }\n      }\n    })\n}\n\nfn test_jscontact_3() -> Value {\n    json!({\n      \"kind\": \"org\",\n      \"organizations\": {\n        \"k1\": {\n          \"name\": \"Acme Business Solutions Ltd.\",\n          \"units\": [\n            {\n              \"name\": \"Technology Division\"\n            }\n          ]\n        }\n      },\n      \"preferredLanguages\": {\n        \"k1\": {\n          \"language\": \"en\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 1\n        },\n        \"k2\": {\n          \"language\": \"de\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 2\n        },\n        \"k3\": {\n          \"language\": \"fr\",\n          \"contexts\": {\n            \"work\": true\n          },\n          \"pref\": 3\n        }\n      },\n      \"directories\": {\n        \"k1\": {\n          \"uri\": \"https://directory.example.com/acme.vcf\",\n          \"kind\": \"entry\"\n        }\n      },\n      \"cryptoKeys\": {\n        \"k1\": {\n          \"uri\": \"https://pgp.example.com/pks/lookup?op=get&search=info@acme-solutions.example\",\n          \"contexts\": {\n            \"pgp\": true\n          }\n        }\n      },\n      \"links\": {\n        \"k1\": {\n          \"uri\": \"https://www.acme-solutions.example\",\n          \"contexts\": {\n            \"work\": true\n          }\n        },\n        \"k2\": {\n          \"uri\": \"https://support.acme-solutions.example\",\n          \"contexts\": {\n            \"support\": true\n          }\n        }\n      },\n      \"name\": {\n        \"full\": \"Acme Business Solutions Ltd.\"\n      },\n      \"notes\": {\n        \"k1\": {\n          \"note\": \"Business hours: Mon-Fri 9:00-17:30 GMT. Closed on UK bank holidays. VAT Reg: GB123456789\"\n        }\n      },\n      \"uid\": \"urn:uuid:a9e95948-7b1c-46e8-bd85-c729a9e910f2\",\n      \"@type\": \"Card\",\n      \"prodId\": \"-//Example Corp.//Contact Manager 3.0//EN\",\n      \"version\": \"1.0\",\n      \"emails\": {\n        \"k1\": {\n          \"address\": \"info@acme-solutions.example\",\n          \"contexts\": {\n            \"work\": true,\n            \"pref\": true\n          }\n        },\n        \"k2\": {\n          \"address\": \"support@acme-solutions.example\",\n          \"contexts\": {\n            \"support\": true\n          }\n        },\n        \"k3\": {\n          \"address\": \"sales@acme-solutions.example\",\n          \"contexts\": {\n            \"sales\": true\n          }\n        }\n      },\n      \"phones\": {\n        \"k1\": {\n          \"number\": \"+44-20-1234-5678\",\n          \"contexts\": {\n            \"work\": true,\n            \"pref\": true\n          },\n          \"features\": {\n            \"voice\": true\n          }\n        },\n        \"k2\": {\n          \"number\": \"+44-20-1234-5679\",\n          \"features\": {\n            \"fax\": true\n          }\n        },\n        \"k3\": {\n          \"number\": \"+44-800-987-6543\",\n          \"contexts\": {\n            \"support\": true\n          }\n        }\n      },\n      \"addresses\": {\n        \"k1\": {\n          \"contexts\": {\n            \"work\": true\n          },\n          \"full\": \"10 Enterprise Way\\nTech Park\\nLondon, EC1A 1BB\\nUnited Kingdom\",\n          \"components\": [\n            {\n              \"kind\": \"name\",\n              \"value\": \"10 Enterprise Way, Tech Park\"\n            },\n            {\n              \"kind\": \"locality\",\n              \"value\": \"London\"\n            },\n            {\n              \"kind\": \"postcode\",\n              \"value\": \"EC1A 1BB\"\n            },\n            {\n              \"kind\": \"country\",\n              \"value\": \"United Kingdom\"\n            }\n          ],\n          \"timeZone\": \"Etc/UTC\",\n          \"coordinates\": \"51.5074;-0.1278\",\n          \"isOrdered\": true\n        },\n        \"k2\": {\n          \"contexts\": {\n            \"branch\": true\n          },\n          \"full\": \"25 Innovation Street\\nManchester, M1 5QF\\nUnited Kingdom\",\n          \"components\": [\n            {\n              \"kind\": \"name\",\n              \"value\": \"25 Innovation Street\"\n            },\n            {\n              \"kind\": \"locality\",\n              \"value\": \"Manchester\"\n            },\n            {\n              \"kind\": \"postcode\",\n              \"value\": \"M1 5QF\"\n            },\n            {\n              \"kind\": \"country\",\n              \"value\": \"United Kingdom\"\n            }\n          ],\n          \"isOrdered\": true\n        }\n      },\n      \"updated\": \"2023-04-15T15:30:00Z\",\n      \"keywords\": {\n        \"Technology\": true,\n        \"B2B\": true,\n        \"Solutions\": true,\n        \"Services\": true\n      },\n      \"relatedTo\": {\n        \"urn:uuid:b9e93fdb-4d34-45fa-a1e2-47da0428c4a1\": {\n          \"relation\": {\n            \"contact\": true\n          }\n        },\n        \"urn:uuid:c8e74dfe-6b34-45fa-b1e2-47ea0428c4b2\": {\n          \"relation\": {\n            \"contact\": true\n          }\n        }\n      }\n    })\n}\n\nfn test_jscontact_4() -> Value {\n    json!({\n    \"@type\": \"Card\",\n    \"version\": \"1.0\",\n    \"kind\": \"individual\",\n    \"name\": {\n      \"@type\": \"Name\",\n      \"full\": \"Temporary Contact\"\n    }})\n}\n\nconst TEST_VCARD_1: &str = r#\"BEGIN:VCARD\nVERSION:4.0\nUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6\nLANG;TYPE=WORK;PREF=1;PROP-ID=k1:en\nLANG;TYPE=WORK;PREF=2;PROP-ID=k2:fr\nFN:Sarah O'Connor\nN;JSCOMPS=\";0;1;2;3;4\":O'Connor;Sarah;Marie;Dr.;Ph.D.;;\nKEY;TYPE=PGP;PROP-ID=k1:https://pgp.example.com/pks/lookup?op=get&search=sar\n ah.johnson@example.com\nCATEGORIES:Work,Research,VIP\nBDAY;PROP-ID=k1:19850415\nANNIVERSARY;PROP-ID=k2:20100610\nURL;TYPE=WORK;PROP-ID=k1:https://www.example.com/staff/sjohnson\nURL;TYPE=HOME;PROP-ID=k2:https://www.sarahjohnson.example.com\nORG;PROP-ID=k1:Acme Technologies Inc.;Research Department\nEMAIL;TYPE=WORK;PROP-ID=k1:sarah.johnson@example.com\nEMAIL;TYPE=HOME,PREF;PROP-ID=k2:sarahjpersonal@example.com\nTEL;TYPE=PREF,CELL,VOICE;PROP-ID=k1:+1-555-123-4567\nTEL;TYPE=WORK,VOICE;PROP-ID=k2:+1-555-987-6543\nTEL;TYPE=HOME,VOICE;PROP-ID=k3:+1-555-456-7890\nADR;TYPE=WORK;LABEL=\"123 Business Ave\\nSuite 400\\nNew York, NY 10001\\nUSA\";\n TZ=Etc/GMT+5;GEO=\"40.7128;-74.0060\";PROP-ID=k1;JSCOMPS=\";11;3;4;5;6\":;;123 B\n usiness Ave;New York;NY;10001;USA;;;;;123 Business Ave;;;;;;\nADR;TYPE=HOME,PREF;LABEL=\"456 Residential St\\nApt 7B\\nBrooklyn, NY 11201\\nU\n SA\";PROP-ID=k2;JSCOMPS=\";11;3;4;5;6\":;;456 Residential St;Brooklyn;NY;11201;\n USA;;;;;456 Residential St;;;;;;\nTITLE;PROP-ID=k1:Senior Research Scientist\nJSPROP;JSPTR=titles/k2/organizationId:\"k1\"\nROLE;PROP-ID=k2:Team Lead\nNICKNAME;PROP-ID=k1:Sadie\nNOTE;PROP-ID=k1:Sarah prefers video calls over phone calls. Available Mon-Th\n u 9-5 EST.\nREV:20220315T133000Z\nEND:VCARD\n\"#;\n"
  },
  {
    "path": "tests/src/jmap/contacts/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod addressbook;\npub mod contact;\n"
  },
  {
    "path": "tests/src/jmap/core/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{jmap::JMAPTest, store::cleanup::store_blob_expire_all};\nuse email::mailbox::INBOX_ID;\nuse serde_json::{Value, json};\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running blob tests...\");\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    store_blob_expire_all(&server.core.storage.data).await;\n\n    // Blob/set simple test\n    let response = account.jmap_method_call(\"Blob/upload\", json!({\n             \"create\": {\n              \"abc\": {\n               \"data\" : [\n               {\n                \"data:asBase64\": \"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKAAAAA1BMVEX/AAAZ4gk3AAAAAXRSTlN/gFy0ywAAAApJREFUeJxjYgAAAAYAAzY3fKgAAAAASUVORK5CYII=\"\n               }\n              ],\n              \"type\": \"image/png\"\n              }\n             }\n            })).await;\n\n    assert_eq!(\n        response\n            .pointer(\"/methodResponses/0/1/created/abc/type\")\n            .and_then(|v| v.as_str())\n            .unwrap_or_default(),\n        \"image/png\",\n        \"Response: {:?}\",\n        response\n    );\n    assert_eq!(\n        response\n            .pointer(\"/methodResponses/0/1/created/abc/size\")\n            .and_then(|v| v.as_i64())\n            .unwrap_or_default(),\n        95,\n        \"Response: {:?}\",\n        response\n    );\n\n    // Blob/get simple test\n    let blob_id = account\n        .jmap_method_call(\n            \"Blob/upload\",\n            json!({\n             \"create\": {\n              \"abc\": {\n               \"data\" : [\n               {\n                \"data:asText\": \"The quick brown fox jumped over the lazy dog.\"\n               }\n              ]\n              }\n             }\n            }),\n        )\n        .await\n        .pointer(\"/methodResponses/0/1/created/abc/id\")\n        .and_then(|v| v.as_str())\n        .unwrap()\n        .to_string();\n\n    let response = account\n        .jmap_method_calls(json!([[\n            \"Blob/get\",\n            {\n              \"ids\" : [\n                blob_id\n              ],\n              \"properties\" : [\n                \"data:asText\",\n                \"digest:sha\",\n                \"size\"\n              ]\n            },\n            \"R1\"\n          ],\n          [\n            \"Blob/get\",\n            {\n              \"ids\" : [\n                blob_id\n              ],\n              \"properties\" : [\n                \"data:asText\",\n                \"digest:sha\",\n                \"digest:sha-256\",\n                \"size\"\n              ],\n              \"offset\" : 4,\n              \"length\" : 9\n            },\n            \"R2\"\n          ]\n        ]))\n        .await;\n\n    for (pointer, expected) in [\n        (\n            \"/methodResponses/0/1/list/0/data:asText\",\n            \"The quick brown fox jumped over the lazy dog.\",\n        ),\n        (\n            \"/methodResponses/0/1/list/0/digest:sha\",\n            \"wIVPufsDxBzOOALLDSIFKebu+U4=\",\n        ),\n        (\"/methodResponses/0/1/list/0/size\", \"45\"),\n        (\"/methodResponses/1/1/list/0/data:asText\", \"quick bro\"),\n        (\n            \"/methodResponses/1/1/list/0/digest:sha\",\n            \"QiRAPtfyX8K6tm1iOAtZ87Xj3Ww=\",\n        ),\n        (\n            \"/methodResponses/1/1/list/0/digest:sha-256\",\n            \"gdg9INW7lwHK6OQ9u0dwDz2ZY/gubi0En0xlFpKt0OA=\",\n        ),\n    ] {\n        assert_eq!(\n            response\n                .pointer(pointer)\n                .and_then(|v| match v {\n                    Value::String(s) => Some(s.to_string()),\n                    Value::Number(n) => Some(n.to_string()),\n                    _ => None,\n                })\n                .unwrap_or_default(),\n            expected,\n            \"Pointer {pointer:?} Response: {response:?}\",\n        );\n    }\n\n    store_blob_expire_all(&server.core.storage.data).await;\n\n    // Blob/upload Complex Example\n    let response = account\n        .jmap_method_calls(json!([\n         [\n          \"Blob/upload\",\n          {\n           \"create\": {\n            \"b4\": {\n             \"data\": [\n              {\n               \"data:asText\": \"The quick brown fox jumped over the lazy dog.\"\n              }\n            ]\n           }\n          }\n         },\n         \"S4\"\n        ],\n        [\n          \"Blob/upload\",\n          {\n           \"create\": {\n             \"cat\": {\n               \"data\": [\n                 {\n                   \"data:asText\": \"How\"\n                 },\n                 {\n                   \"blobId\": \"#b4\",\n                   \"length\": 7,\n                   \"offset\": 3\n                 },\n                 {\n                   \"data:asText\": \"was t\"\n                 },\n                 {\n                   \"blobId\": \"#b4\",\n                   \"length\": 1,\n                   \"offset\": 1\n                 },\n                 {\n                   \"data:asBase64\": \"YXQ/\"\n                 }\n               ]\n             }\n           }\n          },\n          \"CAT\"\n        ],\n        [\n          \"Blob/get\",\n          {\n           \"properties\": [\n             \"data:asText\",\n             \"size\"\n           ],\n           \"ids\": [\n             \"#cat\"\n           ]\n          },\n          \"G4\"\n         ]\n        ]))\n        .await;\n\n    for (pointer, expected) in [\n        (\n            \"/methodResponses/2/1/list/0/data:asText\",\n            \"How quick was that?\",\n        ),\n        (\"/methodResponses/2/1/list/0/size\", \"19\"),\n    ] {\n        assert_eq!(\n            response\n                .pointer(pointer)\n                .and_then(|v| match v {\n                    Value::String(s) => Some(s.to_string()),\n                    Value::Number(n) => Some(n.to_string()),\n                    _ => None,\n                })\n                .unwrap_or_default(),\n            expected,\n            \"Pointer {pointer:?} Response: {response:?}\",\n        );\n    }\n    store_blob_expire_all(&server.core.storage.data).await;\n\n    // Blob/get Example with Range and Encoding Errors\n    let response = account.jmap_method_calls(json!([\n            [\n              \"Blob/upload\",\n              {\n                \"create\": {\n                  \"b1\": {\n                    \"data\": [\n                      {\n                        \"data:asBase64\": \"VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wZWQgb3ZlciB0aGUggYEgZG9nLg==\"\n                      }\n                    ]\n                  },\n                  \"b2\": {\n                    \"data\": [\n                      {\n                        \"data:asText\": \"hello world\"\n                      }\n                    ],\n                    \"type\" : \"text/plain\"\n                  }\n                }\n              },\n              \"S1\"\n            ],\n            [\n              \"Blob/get\",\n              {\n                \"ids\": [\n                  \"#b1\",\n                  \"#b2\"\n                ]\n              },\n              \"G1\"\n            ],\n            [\n              \"Blob/get\",\n              {\n                \"ids\": [\n                  \"#b1\",\n                  \"#b2\"\n                ],\n                \"properties\": [\n                  \"data:asText\",\n                  \"size\"\n                ]\n              },\n              \"G2\"\n            ],\n            [\n              \"Blob/get\",\n              {\n                \"ids\": [\n                  \"#b1\",\n                  \"#b2\"\n                ],\n                \"properties\": [\n                  \"data:asBase64\",\n                  \"size\"\n                ]\n              },\n              \"G3\"\n            ],\n            [\n              \"Blob/get\",\n              {\n                \"offset\": 0,\n                \"length\": 5,\n                \"ids\": [\n                  \"#b1\",\n                  \"#b2\"\n                ]\n              },\n              \"G4\"\n            ],\n            [\n              \"Blob/get\",\n              {\n                \"offset\": 20,\n                \"length\": 100,\n                \"ids\": [\n                  \"#b1\",\n                  \"#b2\"\n                ]\n              },\n              \"G5\"\n            ]\n          ])).await;\n\n    for (pointer, expected) in [\n        (\n            \"/methodResponses/1/1/list/0/data:asBase64\",\n            \"VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wZWQgb3ZlciB0aGUggYEgZG9nLg==\",\n        ),\n        (\"/methodResponses/1/1/list/1/data:asText\", \"hello world\"),\n        (\"/methodResponses/2/1/list/0/isEncodingProblem\", \"true\"),\n        (\"/methodResponses/2/1/list/1/data:asText\", \"hello world\"),\n        (\n            \"/methodResponses/3/1/list/0/data:asBase64\",\n            \"VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wZWQgb3ZlciB0aGUggYEgZG9nLg==\",\n        ),\n        (\n            \"/methodResponses/3/1/list/1/data:asBase64\",\n            \"aGVsbG8gd29ybGQ=\",\n        ),\n        (\"/methodResponses/4/1/list/0/data:asText\", \"The q\"),\n        (\"/methodResponses/4/1/list/1/data:asText\", \"hello\"),\n        (\"/methodResponses/5/1/list/0/isEncodingProblem\", \"true\"),\n        (\"/methodResponses/5/1/list/0/isTruncated\", \"true\"),\n        (\"/methodResponses/5/1/list/1/isTruncated\", \"true\"),\n    ] {\n        assert_eq!(\n            response\n                .pointer(pointer)\n                .and_then(|v| match v {\n                    Value::String(s) => Some(s.to_string()),\n                    Value::Number(n) => Some(n.to_string()),\n                    Value::Bool(b) => Some(b.to_string()),\n                    _ => None,\n                })\n                .unwrap_or_default(),\n            expected,\n            \"Pointer {pointer:?} Response: {response:?}\",\n        );\n    }\n    store_blob_expire_all(&server.core.storage.data).await;\n\n    // Blob/lookup\n    let client = account.client();\n    let blob_id = client\n        .email_import(\n            concat!(\n                \"From: bill@example.com\\r\\n\",\n                \"To: jdoe@example.com\\r\\n\",\n                \"Subject: TPS Report\\r\\n\",\n                \"\\r\\n\",\n                \"I'm going to need those TPS reports ASAP. \",\n                \"So, if you could do that, that'd be great.\"\n            )\n            .as_bytes()\n            .to_vec(),\n            [&Id::from(INBOX_ID).to_string()],\n            None::<Vec<&str>>,\n            None,\n        )\n        .await\n        .unwrap()\n        .take_blob_id();\n\n    let response = account\n        .jmap_method_call(\n            \"Blob/lookup\",\n            json!({\n              \"typeNames\": [\n                \"Mailbox\",\n                \"Thread\",\n                \"Email\"\n              ],\n              \"ids\": [\n                blob_id,\n                \"not-a-blob\"\n              ]\n            }),\n        )\n        .await;\n\n    for pointer in [\n        \"/methodResponses/0/1/list/0/matchedIds/Email\",\n        \"/methodResponses/0/1/list/0/matchedIds/Mailbox\",\n        \"/methodResponses/0/1/list/0/matchedIds/Thread\",\n    ] {\n        assert_eq!(\n            response\n                .pointer(pointer)\n                .and_then(|v| v.as_array())\n                .map(|arr| arr.len())\n                .unwrap_or_default(),\n            1,\n            \"Pointer {pointer:?} Response: {response:#?}\",\n        );\n    }\n\n    // Remove test data\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/core/event_source.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, mail::delivery::SmtpConnection};\nuse email::mailbox::INBOX_ID;\nuse futures::StreamExt;\nuse jmap_client::{\n    DataType,\n    event_source::{Changes, PushNotification},\n    mailbox::Role,\n};\nuse std::time::Duration;\nuse store::ahash::AHashSet;\nuse tokio::sync::mpsc;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running EventSource tests...\");\n\n    // Create test account\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    let mut changes = client\n        .event_source(None::<Vec<_>>, false, 1.into(), None)\n        .await\n        .unwrap();\n\n    let (event_tx, mut event_rx) = mpsc::channel::<Changes>(100);\n\n    tokio::spawn(async move {\n        while let Some(change) = changes.next().await {\n            if let Err(_err) = event_tx\n                .send(match change.unwrap() {\n                    PushNotification::StateChange(changes) => changes,\n                    PushNotification::CalendarAlert(_) => unreachable!(),\n                })\n                .await\n            {\n                //println!(\"Error sending event: {}\", _err);\n                break;\n            }\n        }\n    });\n\n    assert_ping(&mut event_rx).await;\n\n    // Create mailbox and expect state change\n    let mailbox_id = client\n        .mailbox_create(\"EventSource Test\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n    assert_state(&mut event_rx, account.id_string(), &[DataType::Mailbox]).await;\n\n    // Multiple changes should be grouped and delivered in intervals\n    for num in 0..5 {\n        client\n            .mailbox_update_sort_order(&mailbox_id, num)\n            .await\n            .unwrap();\n    }\n    assert_state(&mut event_rx, account.id_string(), &[DataType::Mailbox]).await;\n    assert_ping(&mut event_rx).await; // Pings are only received in cfg(test)\n\n    // Ingest email and expect state change\n    let mut lmtp = SmtpConnection::connect().await;\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n    lmtp.quit().await;\n\n    assert_state(\n        &mut event_rx,\n        account.id_string(),\n        &[\n            DataType::EmailDelivery,\n            DataType::Email,\n            DataType::Thread,\n            DataType::Mailbox,\n        ],\n    )\n    .await;\n    assert_ping(&mut event_rx).await;\n\n    // Destroy mailbox\n    client.mailbox_destroy(&mailbox_id, true).await.unwrap();\n    assert_state(&mut event_rx, account.id_string(), &[DataType::Mailbox]).await;\n\n    // Destroy Inbox\n    client\n        .mailbox_destroy(&Id::from(INBOX_ID).to_string(), true)\n        .await\n        .unwrap();\n    assert_state(\n        &mut event_rx,\n        account.id_string(),\n        &[DataType::Email, DataType::Thread, DataType::Mailbox],\n    )\n    .await;\n    assert_ping(&mut event_rx).await;\n    assert_ping(&mut event_rx).await;\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\nasync fn assert_state(\n    event_rx: &mut mpsc::Receiver<Changes>,\n    account_id: &str,\n    state: &[DataType],\n) {\n    match tokio::time::timeout(Duration::from_millis(700), event_rx.recv()).await {\n        Ok(Some(changes)) => {\n            assert_eq!(\n                changes\n                    .changes(account_id)\n                    .unwrap()\n                    .map(|x| x.0)\n                    .collect::<AHashSet<&DataType>>(),\n                state.iter().collect::<AHashSet<&DataType>>()\n            );\n        }\n        result => {\n            panic!(\"Timeout waiting for event {:?}: {:?}\", state, result);\n        }\n    }\n}\n\nasync fn assert_ping(event_rx: &mut mpsc::Receiver<Changes>) {\n    match tokio::time::timeout(Duration::from_millis(1100), event_rx.recv()).await {\n        Ok(Some(changes)) => {\n            assert!(changes.changes(\"ping\").is_some(),);\n        }\n        _ => {\n            panic!(\"Did not receive ping.\");\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/core/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod blob;\npub mod event_source;\npub mod push_subscription;\npub mod websocket;\n"
  },
  {
    "path": "tests/src/jmap/core/push_subscription.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{AssertConfig, add_test_certs, jmap::JMAPTest};\nuse base64::{Engine, engine::general_purpose};\nuse common::{Caches, Core, Data, Inner, config::server::Listeners, listener::SessionData};\nuse ece::EcKeyComponents;\nuse http_proto::{HtmlResponse, ToHttpResponse, request::fetch_body};\nuse hyper::{StatusCode, body, header::CONTENT_ENCODING, server::conn::http1, service::service_fn};\nuse hyper_util::rt::TokioIo;\nuse jmap_client::{mailbox::Role, push_subscription::Keys};\nuse jmap_proto::{response::status::PushObject, types::state::State};\nuse services::state_manager::ece::ece_encrypt;\nuse std::{\n    sync::{\n        Arc,\n        atomic::{AtomicBool, Ordering},\n    },\n    time::Duration,\n};\nuse store::ahash::AHashSet;\nuse tokio::sync::mpsc;\nuse types::{id::Id, type_state::DataType};\nuse utils::{config::Config, map::vec_map::VecMap};\n\nconst SERVER: &str = r#\"\n[server]\nhostname = \"'jmap-push.example.org'\"\n\n[http]\nurl = \"'https://127.0.0.1:9000'\"\n\n[server.listener.jmap]\nbind = ['127.0.0.1:9000']\nprotocol = 'http'\ntls.implicit = true\n\n[server.socket]\nreuse-addr = true\n\n[certificate.default]\ncert = '%{file:{CERT}}%'\nprivate-key = '%{file:{PK}}%'\ndefault = true\n\"#;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Push Subscription tests...\");\n\n    // ECE roundtrip test\n    ece_roundtrip();\n\n    // Create test account\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    // Create channels\n    let (event_tx, mut event_rx) = mpsc::channel::<PushMessage>(100);\n\n    // Create subscription keys\n    let (keypair, auth_secret) = ece::generate_keypair_and_auth_secret().unwrap();\n    let pubkey = keypair.pub_as_raw().unwrap();\n    let keys = Keys::new(&pubkey, &auth_secret);\n\n    let push_server = Arc::new(PushServer {\n        keypair: keypair.raw_components().unwrap(),\n        auth_secret: auth_secret.to_vec(),\n        tx: event_tx,\n        fail_requests: false.into(),\n    });\n\n    // Start mock push server\n    let mut settings = Config::new(add_test_certs(SERVER)).unwrap();\n    settings.resolve_all_macros().await;\n    let mock_inner = Arc::new(Inner {\n        shared_core: Core::parse(&mut settings, Default::default(), Default::default())\n            .await\n            .into_shared(),\n        data: Data::parse(&mut settings),\n        cache: Caches::parse(&mut settings),\n        ..Default::default()\n    });\n    settings.errors.clear();\n    settings.warnings.clear();\n    let mut servers = Listeners::parse(&mut settings);\n    servers.parse_tcp_acceptors(&mut settings, mock_inner.clone());\n\n    // Start JMAP server\n    servers.bind_and_drop_priv(&mut settings);\n    settings.assert_no_errors();\n    let _shutdown_tx = servers.spawn(|server, acceptor, shutdown_rx| {\n        server.spawn(\n            SessionManager::from(push_server.clone()),\n            mock_inner.clone(),\n            acceptor,\n            shutdown_rx,\n        );\n    });\n\n    // Register push notification (no encryption)\n    let push_id = client\n        .push_subscription_create(\"123\", \"https://127.0.0.1:9000/push\", None)\n        .await\n        .unwrap()\n        .take_id();\n\n    // Expect push verification\n    let verification = expect_push(&mut event_rx).await.unwrap_verification();\n    assert_eq!(verification.push_subscription_id, push_id);\n\n    // Update verification code\n    client\n        .push_subscription_verify(&push_id, verification.verification_code)\n        .await\n        .unwrap();\n\n    // Create a mailbox and expect a state change\n    let mailbox_id = client\n        .mailbox_create(\"PushSubscription Test\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    assert_state(&mut event_rx, account.id(), &[DataType::Mailbox]).await;\n\n    // Receive states just for the requested types\n    client\n        .push_subscription_update_types(&push_id, [jmap_client::DataType::Email].into())\n        .await\n        .unwrap();\n    client\n        .mailbox_update_sort_order(&mailbox_id, 123)\n        .await\n        .unwrap();\n    expect_nothing(&mut event_rx).await;\n\n    // Destroy subscription\n    client.push_subscription_destroy(&push_id).await.unwrap();\n\n    // Only one verification per minute is allowed\n    let push_id = client\n        .push_subscription_create(\"invalid\", \"https://127.0.0.1:9000/push\", None)\n        .await\n        .unwrap()\n        .take_id();\n    expect_nothing(&mut event_rx).await;\n    client.push_subscription_destroy(&push_id).await.unwrap();\n\n    // Register push notification (with encryption)\n    let push_id = client\n        .push_subscription_create(\n            \"123\",\n            \"https://127.0.0.1:9000/push?skip_checks=true\", // skip_checks only works in cfg(test)\n            keys.into(),\n        )\n        .await\n        .unwrap()\n        .take_id();\n\n    // Expect push verification\n    let verification = expect_push(&mut event_rx).await.unwrap_verification();\n    assert_eq!(verification.push_subscription_id, push_id);\n\n    // Update verification code\n    client\n        .push_subscription_verify(&push_id, verification.verification_code)\n        .await\n        .unwrap();\n\n    // Failed deliveries should be re-attempted\n    push_server.fail_requests.store(true, Ordering::Relaxed);\n    client\n        .mailbox_update_sort_order(&mailbox_id, 101)\n        .await\n        .unwrap();\n    tokio::time::sleep(Duration::from_millis(200)).await;\n    push_server.fail_requests.store(false, Ordering::Relaxed);\n    assert_state(&mut event_rx, account.id(), &[DataType::Mailbox]).await;\n\n    // Make a mailbox change and expect state change\n    client\n        .mailbox_rename(&mailbox_id, \"My Mailbox\")\n        .await\n        .unwrap();\n    assert_state(&mut event_rx, account.id(), &[DataType::Mailbox]).await;\n    //expect_nothing(&mut event_rx).await;\n\n    // Multiple change updates should be grouped and pushed in intervals\n    for num in 0..5 {\n        client\n            .mailbox_update_sort_order(&mailbox_id, num)\n            .await\n            .unwrap();\n    }\n    assert_state(&mut event_rx, account.id(), &[DataType::Mailbox]).await;\n    expect_nothing(&mut event_rx).await;\n\n    // Destroy mailbox\n    client.push_subscription_destroy(&push_id).await.unwrap();\n    client.mailbox_destroy(&mailbox_id, true).await.unwrap();\n    expect_nothing(&mut event_rx).await;\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\n#[derive(Clone)]\npub struct SessionManager {\n    pub inner: Arc<PushServer>,\n}\n\nimpl From<Arc<PushServer>> for SessionManager {\n    fn from(inner: Arc<PushServer>) -> Self {\n        SessionManager { inner }\n    }\n}\npub struct PushServer {\n    keypair: EcKeyComponents,\n    auth_secret: Vec<u8>,\n    tx: mpsc::Sender<PushMessage>,\n    fail_requests: AtomicBool,\n}\n\n#[derive(serde::Deserialize, Debug)]\n#[serde(untagged)]\nenum PushMessage {\n    PushObject(PushObject),\n    Verification(PushVerification),\n}\n\nimpl PushMessage {\n    pub fn unwrap_state_change(self) -> VecMap<Id, VecMap<DataType, State>> {\n        match self {\n            PushMessage::PushObject(PushObject::StateChange { changed }) => changed,\n            _ => panic!(\"Expected PushObject\"),\n        }\n    }\n\n    pub fn unwrap_verification(self) -> PushVerification {\n        match self {\n            PushMessage::Verification(verification) => verification,\n            _ => panic!(\"Expected Verification\"),\n        }\n    }\n}\n\n#[derive(serde::Deserialize, Debug)]\nenum PushVerificationType {\n    PushVerification,\n}\n\n#[derive(serde::Deserialize, Debug)]\nstruct PushVerification {\n    #[serde(rename = \"@type\")]\n    _type: PushVerificationType,\n    #[serde(rename = \"pushSubscriptionId\")]\n    pub push_subscription_id: String,\n    #[serde(rename = \"verificationCode\")]\n    pub verification_code: String,\n}\n\nimpl common::listener::SessionManager for SessionManager {\n    #[allow(clippy::manual_async_fn)]\n    fn handle<T: common::listener::SessionStream>(\n        self,\n        session: SessionData<T>,\n    ) -> impl std::future::Future<Output = ()> + Send {\n        async move {\n            let push = self.inner;\n            let _ = http1::Builder::new()\n                .keep_alive(false)\n                .serve_connection(\n                    TokioIo::new(session.stream),\n                    service_fn(|mut req: hyper::Request<body::Incoming>| {\n                        let push = push.clone();\n\n                        async move {\n                            if push.fail_requests.load(Ordering::Relaxed) {\n                                return Ok(HtmlResponse::with_status(\n                                    StatusCode::TOO_MANY_REQUESTS,\n                                    \"too many requests\".to_string(),\n                                )\n                                .into_http_response()\n                                .build());\n                            }\n                            let is_encrypted = req\n                                .headers()\n                                .get(CONTENT_ENCODING)\n                                .is_some_and(|encoding| encoding.to_str().unwrap() == \"aes128gcm\");\n                            let body = fetch_body(&mut req, 1024 * 1024, 0).await.unwrap();\n                            let message = serde_json::from_slice::<PushMessage>(&if is_encrypted {\n                                ece::decrypt(\n                                    &push.keypair,\n                                    &push.auth_secret,\n                                    &general_purpose::URL_SAFE.decode(body).unwrap(),\n                                )\n                                .unwrap()\n                            } else {\n                                body\n                            })\n                            .unwrap();\n\n                            //println!(\"Push received ({}): {:?}\", is_encrypted, message);\n\n                            push.tx.send(message).await.unwrap();\n\n                            Ok::<_, hyper::Error>(\n                                HtmlResponse::new(\"ok\".to_string())\n                                    .into_http_response()\n                                    .build(),\n                            )\n                        }\n                    }),\n                )\n                .await;\n        }\n    }\n\n    #[allow(clippy::manual_async_fn)]\n    fn shutdown(&self) -> impl std::future::Future<Output = ()> + Send {\n        async {}\n    }\n}\n\nasync fn expect_push(event_rx: &mut mpsc::Receiver<PushMessage>) -> PushMessage {\n    match tokio::time::timeout(Duration::from_millis(1500), event_rx.recv()).await {\n        Ok(Some(push)) => {\n            //println!(\"Push received: {:?}\", push);\n            push\n        }\n        result => {\n            panic!(\"Timeout waiting for push: {:?}\", result);\n        }\n    }\n}\n\nasync fn expect_nothing(event_rx: &mut mpsc::Receiver<PushMessage>) {\n    match tokio::time::timeout(Duration::from_millis(1000), event_rx.recv()).await {\n        Err(_) => {}\n        message => {\n            panic!(\"Received a message when expecting nothing: {:?}\", message);\n        }\n    }\n}\n\nasync fn assert_state(event_rx: &mut mpsc::Receiver<PushMessage>, id: &Id, state: &[DataType]) {\n    assert_eq!(\n        expect_push(event_rx)\n            .await\n            .unwrap_state_change()\n            .get(id)\n            .unwrap()\n            .iter()\n            .map(|x| x.0)\n            .collect::<AHashSet<&DataType>>(),\n        state.iter().collect::<AHashSet<&DataType>>()\n    );\n}\n\nfn ece_roundtrip() {\n    for len in [1, 2, 5, 16, 256, 1024, 2048, 4096, 1024 * 1024] {\n        let (keypair, auth_secret) = ece::generate_keypair_and_auth_secret().unwrap();\n\n        let bytes: Vec<u8> = (0..len).map(|_| store::rand::random::<u8>()).collect();\n\n        let encrypted_bytes =\n            ece_encrypt(&keypair.pub_as_raw().unwrap(), &auth_secret, &bytes).unwrap();\n\n        let decrypted_bytes = ece::decrypt(\n            &keypair.raw_components().unwrap(),\n            &auth_secret,\n            &encrypted_bytes,\n        )\n        .unwrap();\n\n        assert_eq!(bytes, decrypted_bytes, \"len: {}\", len);\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/core/websocket.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::JMAPTest;\nuse ahash::AHashSet;\nuse futures::StreamExt;\nuse jmap_client::{\n    DataType, PushObject,\n    client_ws::WebSocketMessage,\n    core::{\n        response::{Response, TaggedMethodResponse},\n        set::SetObject,\n    },\n};\nuse std::time::Duration;\nuse tokio::sync::mpsc;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running WebSockets tests...\");\n\n    // Authenticate all accounts\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    let mut ws_stream = client.connect_ws().await.unwrap();\n\n    let (stream_tx, mut stream_rx) = mpsc::channel::<WebSocketMessage>(100);\n\n    tokio::spawn(async move {\n        while let Some(change) = ws_stream.next().await {\n            stream_tx.send(change.unwrap()).await.unwrap();\n        }\n    });\n\n    // Create mailbox\n    let mut request = client.build();\n    let create_id = request\n        .set_mailbox()\n        .create()\n        .name(\"WebSocket Test\")\n        .create_id()\n        .unwrap();\n    let request_id = request.send_ws().await.unwrap();\n    let mut response = expect_response(&mut stream_rx).await;\n    assert_eq!(request_id, response.request_id().unwrap());\n    let mailbox_id = response\n        .pop_method_response()\n        .unwrap()\n        .unwrap_set_mailbox()\n        .unwrap()\n        .created(&create_id)\n        .unwrap()\n        .take_id();\n\n    // Enable push notifications\n    client\n        .enable_push_ws(None::<Vec<_>>, None::<&str>)\n        .await\n        .unwrap();\n\n    // Make changes over standard HTTP and expect a push notification via WebSockets\n    client\n        .mailbox_update_sort_order(&mailbox_id, 1)\n        .await\n        .unwrap();\n    assert_state(&mut stream_rx, account.id_string(), &[DataType::Mailbox]).await;\n\n    // Multiple changes should be grouped and delivered in intervals\n    for num in 0..5 {\n        client\n            .mailbox_update_sort_order(&mailbox_id, num)\n            .await\n            .unwrap();\n    }\n    tokio::time::sleep(Duration::from_millis(500)).await;\n    assert_state(&mut stream_rx, account.id_string(), &[DataType::Mailbox]).await;\n    expect_nothing(&mut stream_rx).await;\n\n    // Disable push notifications\n    client.disable_push_ws().await.unwrap();\n\n    // No more changes should be received\n    let mut request = client.build();\n    request.set_mailbox().destroy([&mailbox_id]);\n    request.send_ws().await.unwrap();\n    expect_response(&mut stream_rx)\n        .await\n        .pop_method_response()\n        .unwrap()\n        .unwrap_set_mailbox()\n        .unwrap()\n        .destroyed(&mailbox_id)\n        .unwrap();\n    expect_nothing(&mut stream_rx).await;\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\nasync fn expect_response(\n    stream_rx: &mut mpsc::Receiver<WebSocketMessage>,\n) -> Response<TaggedMethodResponse> {\n    match tokio::time::timeout(Duration::from_millis(100), stream_rx.recv()).await {\n        Ok(Some(message)) => match message {\n            WebSocketMessage::Response(response) => response,\n            _ => panic!(\"Expected response, got: {:?}\", message),\n        },\n        result => {\n            panic!(\"Timeout waiting for websocket: {:?}\", result);\n        }\n    }\n}\n\nasync fn assert_state(\n    stream_rx: &mut mpsc::Receiver<WebSocketMessage>,\n    id: &str,\n    state: &[DataType],\n) {\n    match tokio::time::timeout(Duration::from_millis(700), stream_rx.recv()).await {\n        Ok(Some(message)) => match message {\n            WebSocketMessage::PushNotification(PushObject::StateChange { changed }) => {\n                assert_eq!(\n                    changed\n                        .get(id)\n                        .unwrap()\n                        .keys()\n                        .collect::<AHashSet<&DataType>>(),\n                    state.iter().collect::<AHashSet<&DataType>>()\n                );\n            }\n            _ => panic!(\"Expected state change, got: {:?}\", message),\n        },\n        result => {\n            panic!(\"Timeout waiting for websocket: {:?}\", result);\n        }\n    }\n}\n\nasync fn expect_nothing(stream_rx: &mut mpsc::Receiver<WebSocketMessage>) {\n    match tokio::time::timeout(Duration::from_millis(1000), stream_rx.recv()).await {\n        Err(_) => {}\n        message => {\n            panic!(\"Received a message when expecting nothing: {:?}\", message);\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/files/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, JmapUtils};\nuse jmap_proto::{\n    object::{file_node::FileNodeProperty, share_notification::ShareNotificationProperty},\n    request::method::MethodObject,\n};\nuse serde_json::json;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running File Storage ACL tests...\");\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let john_id = john.id_string().to_string();\n    let jane_id = jane.id_string().to_string();\n\n    // Create test folders\n    let response = john\n        .jmap_create(\n            MethodObject::FileNode,\n            [json!({\n                \"name\": \"Test #1\",\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let john_folder_id = response.created(0).id().to_string();\n\n    // Verify myRights\n    john.jmap_get(\n        MethodObject::FileNode,\n        [\n            FileNodeProperty::Id,\n            FileNodeProperty::Name,\n            FileNodeProperty::MyRights,\n            FileNodeProperty::ShareWith,\n        ],\n        [john_folder_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_folder_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n          \"mayRead\": true,\n          \"mayWrite\": true,\n          \"mayShare\": true\n        },\n        \"shareWith\": {}\n        }));\n\n    // Obtain share notifications\n    let mut jane_share_change_id = jane\n        .jmap_get(\n            MethodObject::ShareNotification,\n            Vec::<&str>::new(),\n            Vec::<&str>::new(),\n        )\n        .await\n        .state()\n        .to_string();\n\n    // Make sure Jane has no access\n    assert_eq!(\n        jane.jmap_get_account(\n            john,\n            MethodObject::FileNode,\n            Vec::<&str>::new(),\n            [john_folder_id.as_str()],\n        )\n        .await\n        .method_response()\n        .typ(),\n        \"forbidden\"\n    );\n\n    // Share folder with Jane\n    john.jmap_update(\n        MethodObject::FileNode,\n        [(\n            &john_folder_id,\n            json!({\n                \"shareWith\": {\n                   &jane_id : {\n                     \"mayRead\": true,\n                   }\n                }\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_folder_id);\n    john.jmap_get(\n        MethodObject::FileNode,\n        [\n            FileNodeProperty::Id,\n            FileNodeProperty::Name,\n            FileNodeProperty::ShareWith,\n        ],\n        [john_folder_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_folder_id,\n        \"name\": \"Test #1\",\n        \"shareWith\": {\n            &jane_id : {\n                \"mayRead\": true,\n                \"mayWrite\": false,\n                \"mayShare\": false\n            }\n        }\n        }));\n\n    // Verify Jane can access the contact\n    jane.jmap_get_account(\n        john,\n        MethodObject::FileNode,\n        [\n            FileNodeProperty::Id,\n            FileNodeProperty::Name,\n            FileNodeProperty::MyRights,\n        ],\n        [john_folder_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_folder_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": false,\n            \"mayShare\": false\n        }\n        }));\n\n    // Verify Jane received a share notification\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    jane_share_change_id = response.new_state().to_string();\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"FileNode\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_folder_id,\n          \"oldRights\": {\n            \"mayRead\": false,\n            \"mayWrite\": false,\n            \"mayShare\": false\n          },\n          \"newRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": false,\n            \"mayShare\": false\n          },\n          \"name\": null\n        }));\n\n    // Updating and deleting should fail\n    assert_eq!(\n        jane.jmap_update_account(\n            john,\n            MethodObject::FileNode,\n            [(&john_folder_id, json!({}))],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_updated(&john_folder_id)\n        .description(),\n        \"You are not allowed to modify this file node.\"\n    );\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::FileNode,\n            [&john_folder_id],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await\n        .not_destroyed(&john_folder_id)\n        .description(),\n        \"You are not allowed to delete this file node.\"\n    );\n\n    // Grant Jane write access\n    john.jmap_update(\n        MethodObject::FileNode,\n        [(\n            &john_folder_id,\n            json!({\n                format!(\"shareWith/{jane_id}/mayWrite\"): true,\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_folder_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::FileNode,\n        [\n            FileNodeProperty::Id,\n            FileNodeProperty::Name,\n            FileNodeProperty::MyRights,\n        ],\n        [john_folder_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_folder_id,\n        \"name\": \"Test #1\",\n        \"myRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": true,\n            \"mayShare\": false\n        }\n        }));\n\n    // Verify Jane received a share notification with the updated rights\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    jane_share_change_id = response.new_state().to_string();\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"FileNode\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_folder_id,\n          \"oldRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": false,\n            \"mayShare\": false\n          },\n          \"newRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": true,\n            \"mayShare\": false\n          },\n          \"name\": null\n        }));\n\n    // Creating a root folder should fail\n    assert_eq!(\n        jane.jmap_create_account(\n            john,\n            MethodObject::FileNode,\n            [json!({\n                \"name\": \"A new shared folder\",\n            })],\n            Vec::<(&str, &str)>::new()\n        )\n        .await\n        .not_created(0)\n        .description(),\n        \"Cannot create top-level folder in a shared account.\"\n    );\n\n    // Update John's folder name\n    jane.jmap_update_account(\n        john,\n        MethodObject::FileNode,\n        [(\n            &john_folder_id,\n            json!({\n                \"name\": \"Jane's updated name\",\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_folder_id);\n    jane.jmap_get_account(\n        john,\n        MethodObject::FileNode,\n        [FileNodeProperty::Id, FileNodeProperty::Name],\n        [john_folder_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_folder_id,\n        \"name\": \"Jane's updated name\",\n        }));\n\n    // Revoke Jane's access\n    john.jmap_update(\n        MethodObject::FileNode,\n        [(\n            &john_folder_id,\n            json!({\n                format!(\"shareWith/{jane_id}\"): ()\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_folder_id);\n    john.jmap_get(\n        MethodObject::FileNode,\n        [\n            FileNodeProperty::Id,\n            FileNodeProperty::Name,\n            FileNodeProperty::ShareWith,\n        ],\n        [john_folder_id.as_str()],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n        \"id\": john_folder_id,\n        \"name\": \"Jane's updated name\",\n        \"shareWith\": {}\n        }));\n\n    // Verify Jane can no longer access the folder or its contacts\n    assert_eq!(\n        jane.jmap_get_account(\n            john,\n            MethodObject::FileNode,\n            Vec::<&str>::new(),\n            [john_folder_id.as_str()],\n        )\n        .await\n        .method_response()\n        .typ(),\n        \"forbidden\"\n    );\n\n    // Verify Jane received a share notification with the updated rights\n    let response = jane\n        .jmap_changes(MethodObject::ShareNotification, &jane_share_change_id)\n        .await;\n    let changes = response.changes().collect::<Vec<_>>();\n    assert_eq!(changes.len(), 1);\n    let share_id = changes[0].as_created();\n    jane.jmap_get(\n        MethodObject::ShareNotification,\n        [\n            ShareNotificationProperty::Id,\n            ShareNotificationProperty::ChangedBy,\n            ShareNotificationProperty::ObjectType,\n            ShareNotificationProperty::ObjectAccountId,\n            ShareNotificationProperty::ObjectId,\n            ShareNotificationProperty::OldRights,\n            ShareNotificationProperty::NewRights,\n            ShareNotificationProperty::Name,\n        ],\n        [share_id],\n    )\n    .await\n    .list()[0]\n        .assert_is_equal(json!({\n          \"id\": &share_id,\n          \"changedBy\": {\n            \"principalId\": &john_id,\n            \"name\": \"John Doe\",\n            \"email\": \"jdoe@example.com\"\n          },\n          \"objectType\": \"FileNode\",\n          \"objectAccountId\": &john_id,\n          \"objectId\": &john_folder_id,\n          \"oldRights\": {\n            \"mayRead\": true,\n            \"mayWrite\": true,\n            \"mayShare\": false\n          },\n          \"newRights\": {\n            \"mayRead\": false,\n            \"mayWrite\": false,\n            \"mayShare\": false\n          },\n          \"name\": null\n        }));\n\n    // Grant Jane delete access once again\n    john.jmap_update(\n        MethodObject::FileNode,\n        [(\n            &john_folder_id,\n            json!({\n                format!(\"shareWith/{jane_id}/mayRead\"): true,\n                format!(\"shareWith/{jane_id}/mayWrite\"): true,\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&john_folder_id);\n\n    // Verify Jane can delete the folder\n    assert_eq!(\n        jane.jmap_destroy_account(\n            john,\n            MethodObject::FileNode,\n            [john_folder_id.as_str()],\n            [(\"onDestroyRemoveChildren\", true)],\n        )\n        .await\n        .destroyed()\n        .collect::<Vec<_>>(),\n        [john_folder_id.as_str()]\n    );\n\n    // Destroy all mailboxes\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/files/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod node;\n"
  },
  {
    "path": "tests/src/jmap/files/node.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{ChangeType, JMAPTest, JmapUtils};\nuse ahash::AHashSet;\nuse jmap_proto::{object::file_node::FileNodeProperty, request::method::MethodObject};\nuse serde_json::json;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running File Storage tests...\");\n    let account = params.account(\"jdoe@example.com\");\n\n    // Obtain change id\n    let change_id = account\n        .jmap_get(\n            MethodObject::FileNode,\n            [FileNodeProperty::Id],\n            Vec::<&str>::new(),\n        )\n        .await\n        .state()\n        .to_string();\n\n    // Create test folders\n    let response = account\n        .jmap_create(\n            MethodObject::FileNode,\n            [\n                json!({\n                    \"name\": \"Root Folder\",\n                    \"parentId\": null,\n                }),\n                json!({\n                    \"name\": \"Sub Folder\",\n                    \"parentId\": \"#i0\",\n                }),\n                json!({\n                    \"name\": \"Sub-sub Folder\",\n                    \"parentId\": \"#i1\",\n                }),\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let root_folder_id = response.created(0).id().to_string();\n    let sub_folder_id = response.created(1).id().to_string();\n    let sub_sub_folder_id = response.created(2).id().to_string();\n\n    // Validate changes\n    assert_eq!(\n        account\n            .jmap_changes(MethodObject::FileNode, change_id)\n            .await\n            .changes()\n            .collect::<AHashSet<_>>(),\n        [\n            ChangeType::Created(&root_folder_id),\n            ChangeType::Created(&sub_folder_id),\n            ChangeType::Created(&sub_sub_folder_id)\n        ]\n        .into_iter()\n        .collect::<AHashSet<_>>()\n    );\n\n    // Verify folder structure\n    let response = account\n        .jmap_get(\n            MethodObject::FileNode,\n            [\n                FileNodeProperty::Id,\n                FileNodeProperty::Name,\n                FileNodeProperty::ParentId,\n            ],\n            [&root_folder_id, &sub_folder_id, &sub_sub_folder_id],\n        )\n        .await;\n    let list = response.list();\n    assert_eq!(list.len(), 3);\n    list[0].assert_is_equal(json!({\n        \"id\": &root_folder_id,\n        \"name\": \"Root Folder\",\n        \"parentId\": null,\n    }));\n    list[1].assert_is_equal(json!({\n        \"id\": &sub_folder_id,\n        \"name\": \"Sub Folder\",\n        \"parentId\": &root_folder_id,\n    }));\n    list[2].assert_is_equal(json!({\n        \"id\": &sub_sub_folder_id,\n        \"name\": \"Sub-sub Folder\",\n        \"parentId\": &sub_folder_id,\n    }));\n\n    // Create file in root folder\n    let response = account\n        .jmap_method_calls(json!([\n         [\n          \"Blob/upload\",\n          {\n           \"create\": {\n            \"hello\": {\n             \"data\": [\n              {\n               \"data:asText\": r#\"hello world\"#\n              }\n            ]\n           }\n          }\n         },\n         \"S4\"\n        ],\n        [\n          \"FileNode/set\",\n          {\n            \"create\": {\n              \"i0\": {\n                \"name\": \"hello.txt\",\n                \"parentId\": &root_folder_id,\n                \"blobId\": \"#hello\",\n                \"type\": \"text/plain\",\n              }\n            }\n          },\n          \"G4\"\n         ]\n        ]))\n        .await;\n    let file_id = response\n        .pointer(\"/methodResponses/1/1/created/i0\")\n        .unwrap()\n        .id()\n        .to_string();\n\n    // Verify file creation\n    let response = account\n        .jmap_get(\n            MethodObject::FileNode,\n            [\n                FileNodeProperty::Id,\n                FileNodeProperty::BlobId,\n                FileNodeProperty::Name,\n                FileNodeProperty::ParentId,\n                FileNodeProperty::Type,\n                FileNodeProperty::Size,\n            ],\n            [&file_id],\n        )\n        .await;\n    let blob_id = response.list()[0].blob_id().to_string();\n    response.list()[0].assert_is_equal(json!({\n        \"id\": &file_id,\n        \"name\": \"hello.txt\",\n        \"parentId\": &root_folder_id,\n        \"type\": \"text/plain\",\n        \"size\": 11,\n        \"blobId\": &blob_id,\n    }));\n    assert_eq!(\n        account\n            .jmap_get(MethodObject::Blob, [\"data:asText\"], [&blob_id])\n            .await\n            .list()[0]\n            .text_field(\"data:asText\"),\n        \"hello world\"\n    );\n\n    // Creating folders with invalid names or parent ids should fail\n    let response = account\n        .jmap_create(\n            MethodObject::FileNode,\n            [\n                json!({\n                    \"name\": \"Sub Folder\",\n                    \"parentId\": &root_folder_id,\n                }),\n                json!({\n                    \"name\": \"Folder under file\",\n                    \"parentId\": &file_id,\n                }),\n                json!({\n                    \"name\": \"My/Sub/Folder\",\n                }),\n                json!({\n                    \"name\": \".\",\n                }),\n                json!({\n                    \"name\": \"..\",\n                }),\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    assert_eq!(\n        response.not_created(0).description(),\n        \"A node with the same name already exists in this folder.\"\n    );\n    assert_eq!(\n        response.not_created(1).description(),\n        \"Parent ID does not exist or is not a folder.\"\n    );\n    assert_eq!(\n        response.not_created(2).description(),\n        \"Field could not be set.\"\n    );\n    assert_eq!(\n        response.not_created(3).description(),\n        \"Field could not be set.\"\n    );\n    assert_eq!(\n        response.not_created(4).description(),\n        \"Field could not be set.\"\n    );\n\n    // Circular folder references should fail\n    let response = account\n        .jmap_update(\n            MethodObject::FileNode,\n            [(\n                &root_folder_id,\n                json!({\n                    \"parentId\": &sub_sub_folder_id,\n                }),\n            )],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    assert_eq!(\n        response.not_updated(&root_folder_id).description(),\n        \"Circular reference in parent ids.\"\n    );\n\n    // Rename folder and file\n    let response = account\n        .jmap_update(\n            MethodObject::FileNode,\n            [\n                (\n                    &sub_folder_id,\n                    json!({\n                        \"name\": \"Renamed Sub Folder\",\n                    }),\n                ),\n                (\n                    &file_id,\n                    json!({\n                        \"name\": \"renamed-hello.txt\",\n                    }),\n                ),\n            ],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    response.updated(&sub_folder_id);\n    response.updated(&file_id);\n\n    // Verify rename\n    let response = account\n        .jmap_get(\n            MethodObject::FileNode,\n            [\n                FileNodeProperty::Id,\n                FileNodeProperty::Name,\n                FileNodeProperty::ParentId,\n            ],\n            [&sub_folder_id, &file_id],\n        )\n        .await;\n    let list = response.list();\n    assert_eq!(list.len(), 2);\n    list[0].assert_is_equal(json!({\n        \"id\": &sub_folder_id,\n        \"name\": \"Renamed Sub Folder\",\n        \"parentId\": &root_folder_id,\n    }));\n    list[1].assert_is_equal(json!({\n        \"id\": &file_id,\n        \"name\": \"renamed-hello.txt\",\n        \"parentId\": &root_folder_id,\n    }));\n\n    // Destroying a folder with children should fail\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::FileNode,\n                [&root_folder_id],\n                Vec::<(&str, &str)>::new(),\n            )\n            .await\n            .not_destroyed(&root_folder_id)\n            .description(),\n        \"Cannot delete non-empty folder.\"\n    );\n\n    // Delete file and sub folders\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::FileNode,\n                [&file_id],\n                [(\"onDestroyRemoveChildren\", true)],\n            )\n            .await\n            .destroyed()\n            .collect::<AHashSet<_>>(),\n        [file_id.as_str(),].into_iter().collect::<AHashSet<_>>()\n    );\n    assert_eq!(\n        account\n            .jmap_destroy(\n                MethodObject::FileNode,\n                [&root_folder_id],\n                [(\"onDestroyRemoveChildren\", true)],\n            )\n            .await\n            .destroyed()\n            .collect::<AHashSet<_>>(),\n        [\n            sub_sub_folder_id.as_str(),\n            sub_folder_id.as_str(),\n            root_folder_id.as_str()\n        ]\n        .into_iter()\n        .collect::<AHashSet<_>>()\n    );\n\n    // Make sure everything is gone\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{directory::internal::TestInternalDirectory, jmap::JMAPTest};\nuse ::email::mailbox::{INBOX_ID, TRASH_ID};\nuse jmap_client::{\n    core::{\n        error::{MethodError, MethodErrorType},\n        set::{SetError, SetErrorType},\n    },\n    email::{self, Property, import::EmailImportResponse, query::Filter},\n    mailbox::{self, Role},\n    principal::ACL,\n};\nuse std::fmt::Debug;\nuse store::ahash::AHashMap;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running ACL tests...\");\n    let server = params.server.clone();\n\n    // Create a group and three test accounts\n    let inbox_id = Id::new(INBOX_ID as u64).to_string();\n    let trash_id = Id::new(TRASH_ID as u64).to_string();\n\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let bill = params.account(\"bill@example.com\");\n    let sales = params.account(\"sales@example.com\");\n\n    // Authenticate all accounts\n    let mut john_client = john.client_owned().await;\n    let mut jane_client = jane.client_owned().await;\n    let mut bill_client = bill.client_owned().await;\n\n    // Insert two emails in each account\n    let mut email_ids = AHashMap::default();\n    for (client, account_id, name) in [\n        (&mut john_client, john.id(), \"john\"),\n        (&mut jane_client, jane.id(), \"jane\"),\n        (&mut bill_client, bill.id(), \"bill\"),\n        (\n            &mut params.account(\"admin\").client_owned().await,\n            sales.id(),\n            \"sales\",\n        ),\n    ] {\n        let user_name = client.session().username().to_string();\n        let mut ids = Vec::with_capacity(2);\n        for (mailbox_id, mailbox_name) in [(&inbox_id, \"inbox\"), (&trash_id, \"trash\")] {\n            ids.push(\n                client\n                    .set_default_account_id(account_id.to_string())\n                    .email_import(\n                        format!(\n                            concat!(\n                                \"From: acl_test@example.com\\r\\n\",\n                                \"To: {}\\r\\n\",\n                                \"Subject: Owned by {} in {}\\r\\n\",\n                                \"\\r\\n\",\n                                \"This message is owned by {}.\",\n                            ),\n                            user_name, name, mailbox_name, name\n                        )\n                        .into_bytes(),\n                        [mailbox_id],\n                        None::<Vec<&str>>,\n                        None,\n                    )\n                    .await\n                    .unwrap()\n                    .take_id(),\n            );\n        }\n        email_ids.insert(name, ids);\n    }\n\n    // John should have access to his emails only\n    assert_eq!(\n        john_client\n            .email_get(\n                email_ids.get(\"john\").unwrap().first().unwrap(),\n                [Property::Subject].into(),\n            )\n            .await\n            .unwrap()\n            .unwrap()\n            .subject()\n            .unwrap(),\n        \"Owned by john in inbox\"\n    );\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_get(\n                email_ids.get(\"jane\").unwrap().first().unwrap(),\n                [Property::Subject].into(),\n            )\n            .await,\n    );\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_get(&inbox_id, None::<Vec<_>>)\n            .await,\n    );\n    assert_forbidden(\n        john_client\n            .set_default_account_id(sales.id_string())\n            .email_get(\n                email_ids.get(\"sales\").unwrap().first().unwrap(),\n                [Property::Subject].into(),\n            )\n            .await,\n    );\n    assert_forbidden(\n        john_client\n            .set_default_account_id(sales.id_string())\n            .mailbox_get(&inbox_id, None::<Vec<_>>)\n            .await,\n    );\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_query(None::<Filter>, None::<Vec<_>>)\n            .await,\n    );\n\n    // Jane grants Inbox ReadItems access to John\n    jane_client\n        .mailbox_update_acl(&inbox_id, john.id_string(), [ACL::ReadItems])\n        .await\n        .unwrap();\n\n    // John should have ReadItems access to Inbox\n    assert_eq!(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_get(\n                email_ids.get(\"jane\").unwrap().first().unwrap(),\n                [Property::Subject].into(),\n            )\n            .await\n            .unwrap()\n            .unwrap()\n            .subject()\n            .unwrap(),\n        \"Owned by jane in inbox\"\n    );\n    assert_eq!(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_query(None::<Filter>, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .ids(),\n        [email_ids.get(\"jane\").unwrap().first().unwrap().as_str()]\n    );\n\n    // John's session resource should contain Jane's account details\n    john_client.refresh_session().await.unwrap();\n    assert_eq!(\n        john_client\n            .session()\n            .account(jane.id_string())\n            .unwrap()\n            .name(),\n        \"jane.smith@example.com\"\n    );\n\n    // John should not have access to emails in Jane's Trash folder\n    assert!(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_get(\n                email_ids.get(\"jane\").unwrap().last().unwrap(),\n                [Property::Subject].into(),\n            )\n            .await\n            .unwrap()\n            .is_none()\n    );\n\n    // John should only be able to copy blobs he has access to\n    let blob_id = jane_client\n        .email_get(\n            email_ids.get(\"jane\").unwrap().first().unwrap(),\n            [Property::BlobId].into(),\n        )\n        .await\n        .unwrap()\n        .unwrap()\n        .take_blob_id();\n    john_client\n        .set_default_account_id(john.id_string())\n        .blob_copy(jane.id_string(), &blob_id)\n        .await\n        .unwrap();\n    let blob_id = jane_client\n        .email_get(\n            email_ids.get(\"jane\").unwrap().last().unwrap(),\n            [Property::BlobId].into(),\n        )\n        .await\n        .unwrap()\n        .unwrap()\n        .take_blob_id();\n    assert_forbidden(\n        john_client\n            .set_default_account_id(john.id_string())\n            .blob_copy(jane.id_string(), &blob_id)\n            .await,\n    );\n\n    // John only has ReadItems access to Inbox\n    jane_client\n        .mailbox_update_acl(&inbox_id, john.id_string(), [ACL::ReadItems])\n        .await\n        .unwrap();\n    assert_eq!(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_get(&inbox_id, [mailbox::Property::MyRights].into())\n            .await\n            .unwrap()\n            .unwrap()\n            .my_rights()\n            .unwrap()\n            .acl_list(),\n        vec![ACL::ReadItems]\n    );\n\n    // Try to add items using import and copy\n    let blob_id = john_client\n        .set_default_account_id(john.id_string())\n        .upload(\n            Some(john.id_string()),\n            concat!(\n                \"From: acl_test@example.com\\r\\n\",\n                \"To: jane.smith@example.com\\r\\n\",\n                \"Subject: Created by john in jane's inbox\\r\\n\",\n                \"\\r\\n\",\n                \"This message is owned by jane.\",\n            )\n            .as_bytes()\n            .to_vec(),\n            None,\n        )\n        .await\n        .unwrap()\n        .take_blob_id();\n    let mut request = john_client.set_default_account_id(jane.id_string()).build();\n    let email_id = request\n        .import_email()\n        .email(&blob_id)\n        .mailbox_ids([&inbox_id])\n        .create_id();\n    assert_forbidden(\n        request\n            .send_single::<EmailImportResponse>()\n            .await\n            .unwrap()\n            .created(&email_id),\n    );\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_copy(\n                john.id_string(),\n                email_ids.get(\"john\").unwrap().last().unwrap(),\n                [&inbox_id],\n                None::<Vec<&str>>,\n                None,\n            )\n            .await,\n    );\n\n    // Grant access and try again\n    jane_client\n        .mailbox_update_acl(&inbox_id, john.id_string(), [ACL::ReadItems, ACL::AddItems])\n        .await\n        .unwrap();\n\n    let mut request = john_client.set_default_account_id(jane.id_string()).build();\n    let email_id = request\n        .import_email()\n        .email(&blob_id)\n        .mailbox_ids([&inbox_id])\n        .create_id();\n    let email_id = request\n        .send_single::<EmailImportResponse>()\n        .await\n        .unwrap()\n        .created(&email_id)\n        .unwrap()\n        .take_id();\n    let email_id_2 = john_client\n        .set_default_account_id(jane.id_string())\n        .email_copy(\n            john.id_string(),\n            email_ids.get(\"john\").unwrap().last().unwrap(),\n            [&inbox_id],\n            None::<Vec<&str>>,\n            None,\n        )\n        .await\n        .unwrap()\n        .take_id();\n\n    assert_eq!(\n        jane_client\n            .email_get(&email_id, [Property::Subject].into(),)\n            .await\n            .unwrap()\n            .unwrap()\n            .subject()\n            .unwrap(),\n        \"Created by john in jane's inbox\"\n    );\n    assert_eq!(\n        jane_client\n            .email_get(&email_id_2, [Property::Subject].into(),)\n            .await\n            .unwrap()\n            .unwrap()\n            .subject()\n            .unwrap(),\n        \"Owned by john in trash\"\n    );\n\n    // Try removing items\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_destroy(&email_id)\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(\n            &inbox_id,\n            john.id_string(),\n            [ACL::ReadItems, ACL::AddItems, ACL::RemoveItems],\n        )\n        .await\n        .unwrap();\n    john_client\n        .set_default_account_id(jane.id_string())\n        .email_destroy(&email_id)\n        .await\n        .unwrap();\n\n    // Try to set keywords\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_set_keyword(&email_id_2, \"$seen\", true)\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(\n            &inbox_id,\n            john.id_string(),\n            [\n                ACL::ReadItems,\n                ACL::AddItems,\n                ACL::RemoveItems,\n                ACL::SetKeywords,\n            ],\n        )\n        .await\n        .unwrap();\n    john_client\n        .set_default_account_id(jane.id_string())\n        .email_set_keyword(&email_id_2, \"$seen\", true)\n        .await\n        .unwrap();\n    john_client\n        .set_default_account_id(jane.id_string())\n        .email_set_keyword(&email_id_2, \"my-keyword\", true)\n        .await\n        .unwrap();\n\n    // Try to create a child\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_create(\"John's mailbox\", None::<&str>, Role::None)\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(\n            &inbox_id,\n            john.id_string(),\n            [\n                ACL::ReadItems,\n                ACL::AddItems,\n                ACL::RemoveItems,\n                ACL::SetKeywords,\n                ACL::CreateChild,\n            ],\n        )\n        .await\n        .unwrap();\n    let mailbox_id = john_client\n        .set_default_account_id(jane.id_string())\n        .mailbox_create(\"John's mailbox\", Some(&inbox_id), Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    // Try renaming a mailbox\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_rename(&mailbox_id, \"John's private mailbox\")\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(&mailbox_id, john.id_string(), [ACL::ReadItems, ACL::Rename])\n        .await\n        .unwrap();\n    john_client\n        .set_default_account_id(jane.id_string())\n        .mailbox_rename(&mailbox_id, \"John's private mailbox\")\n        .await\n        .unwrap();\n\n    // Try moving a message\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_set_mailbox(&email_id_2, &mailbox_id, true)\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(\n            &mailbox_id,\n            john.id_string(),\n            [ACL::ReadItems, ACL::Rename, ACL::AddItems],\n        )\n        .await\n        .unwrap();\n    john_client\n        .set_default_account_id(jane.id_string())\n        .email_set_mailbox(&email_id_2, &mailbox_id, true)\n        .await\n        .unwrap();\n\n    // Try deleting a mailbox\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_destroy(&mailbox_id, true)\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(\n            &mailbox_id,\n            john.id_string(),\n            [ACL::ReadItems, ACL::Rename, ACL::AddItems, ACL::Delete],\n        )\n        .await\n        .unwrap();\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_destroy(&mailbox_id, true)\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(\n            &mailbox_id,\n            john.id_string(),\n            [\n                ACL::ReadItems,\n                ACL::Rename,\n                ACL::AddItems,\n                ACL::Delete,\n                ACL::RemoveItems,\n            ],\n        )\n        .await\n        .unwrap();\n    john_client\n        .set_default_account_id(jane.id_string())\n        .mailbox_destroy(&mailbox_id, true)\n        .await\n        .unwrap();\n\n    // Try changing ACL\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_update_acl(&inbox_id, bill.id_string(), [ACL::ReadItems])\n            .await,\n    );\n    assert_forbidden(\n        bill_client\n            .set_default_account_id(jane.id_string())\n            .email_query(None::<Filter>, None::<Vec<_>>)\n            .await,\n    );\n    jane_client\n        .mailbox_update_acl(\n            &inbox_id,\n            john.id_string(),\n            [\n                ACL::ReadItems,\n                ACL::AddItems,\n                ACL::RemoveItems,\n                ACL::SetKeywords,\n                ACL::CreateChild,\n                ACL::Rename,\n                ACL::Administer,\n            ],\n        )\n        .await\n        .unwrap();\n    assert_eq!(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .mailbox_get(&inbox_id, [mailbox::Property::MyRights].into())\n            .await\n            .unwrap()\n            .unwrap()\n            .my_rights()\n            .unwrap()\n            .acl_list(),\n        vec![\n            ACL::ReadItems,\n            ACL::AddItems,\n            ACL::RemoveItems,\n            ACL::SetSeen,\n            ACL::SetKeywords,\n            ACL::CreateChild,\n            ACL::Rename\n        ]\n    );\n    john_client\n        .set_default_account_id(jane.id_string())\n        .mailbox_update_acl(&inbox_id, bill.id_string(), [ACL::ReadItems])\n        .await\n        .unwrap();\n    assert_eq!(\n        bill_client\n            .set_default_account_id(jane.id_string())\n            .email_query(\n                None::<Filter>,\n                vec![email::query::Comparator::subject()].into()\n            )\n            .await\n            .unwrap()\n            .ids(),\n        [\n            email_ids.get(\"jane\").unwrap().first().unwrap().as_str(),\n            &email_id_2\n        ]\n    );\n\n    // Revoke all access to John\n    jane_client\n        .mailbox_update_acl(&inbox_id, john.id_string(), [])\n        .await\n        .unwrap();\n    assert_forbidden(\n        john_client\n            .set_default_account_id(jane.id_string())\n            .email_get(\n                email_ids.get(\"jane\").unwrap().first().unwrap(),\n                [Property::Subject].into(),\n            )\n            .await,\n    );\n    john_client.refresh_session().await.unwrap();\n    assert!(john_client.session().account(jane.id_string()).is_none());\n    assert_eq!(\n        bill_client\n            .set_default_account_id(jane.id_string())\n            .email_get(\n                email_ids.get(\"jane\").unwrap().first().unwrap(),\n                [Property::Subject].into(),\n            )\n            .await\n            .unwrap()\n            .unwrap()\n            .subject()\n            .unwrap(),\n        \"Owned by jane in inbox\"\n    );\n\n    // Add John and Jane to the Sales group\n    for name in [\"jdoe@example.com\", \"jane.smith@example.com\"] {\n        server\n            .invalidate_principal_caches(\n                server\n                    .core\n                    .storage\n                    .data\n                    .add_to_group(name, \"sales@example.com\")\n                    .await,\n            )\n            .await;\n    }\n    john_client.refresh_session().await.unwrap();\n    jane_client.refresh_session().await.unwrap();\n    bill_client.refresh_session().await.unwrap();\n    assert_eq!(\n        john_client\n            .session()\n            .account(sales.id_string())\n            .unwrap()\n            .name(),\n        \"sales@example.com\"\n    );\n    assert!(\n        !john_client\n            .session()\n            .account(sales.id_string())\n            .unwrap()\n            .is_personal()\n    );\n    assert_eq!(\n        jane_client\n            .session()\n            .account(sales.id_string())\n            .unwrap()\n            .name(),\n        \"sales@example.com\"\n    );\n    assert!(bill_client.session().account(sales.id_string()).is_none());\n\n    // Insert a message in Sales's inbox\n    let blob_id = john_client\n        .set_default_account_id(sales.id_string())\n        .upload(\n            Some(sales.id_string()),\n            concat!(\n                \"From: acl_test@example.com\\r\\n\",\n                \"To: sales@example.com\\r\\n\",\n                \"Subject: Created by john in sales\\r\\n\",\n                \"\\r\\n\",\n                \"This message is owned by sales.\",\n            )\n            .as_bytes()\n            .to_vec(),\n            None,\n        )\n        .await\n        .unwrap()\n        .take_blob_id();\n    let mut request = john_client.build();\n    let email_id = request\n        .import_email()\n        .email(&blob_id)\n        .mailbox_ids([&inbox_id])\n        .create_id();\n    let email_id = request\n        .send_single::<EmailImportResponse>()\n        .await\n        .unwrap()\n        .created(&email_id)\n        .unwrap()\n        .take_id();\n\n    // Both Jane and John should be able to see this message, but not Bill\n    assert_eq!(\n        john_client\n            .set_default_account_id(sales.id_string())\n            .email_get(&email_id, [Property::Subject].into(),)\n            .await\n            .unwrap()\n            .unwrap()\n            .subject()\n            .unwrap(),\n        \"Created by john in sales\"\n    );\n    assert_eq!(\n        jane_client\n            .set_default_account_id(sales.id_string())\n            .email_get(&email_id, [Property::Subject].into(),)\n            .await\n            .unwrap()\n            .unwrap()\n            .subject()\n            .unwrap(),\n        \"Created by john in sales\"\n    );\n    assert_forbidden(\n        bill_client\n            .set_default_account_id(sales.id_string())\n            .email_get(&email_id, [Property::Subject].into())\n            .await,\n    );\n\n    // Remove John from the sales group\n    server\n        .invalidate_principal_caches(\n            server\n                .core\n                .storage\n                .data\n                .remove_from_group(\"jdoe@example.com\", \"sales@example.com\")\n                .await,\n        )\n        .await;\n    assert_forbidden(\n        john_client\n            .set_default_account_id(sales.id_string())\n            .email_get(&email_id, [Property::Subject].into())\n            .await,\n    );\n\n    // Destroy test account data\n    for id in [john, bill, jane, sales] {\n        params.destroy_all_mailboxes(id).await;\n    }\n    params.assert_is_empty().await;\n}\n\npub fn assert_forbidden<T: Debug>(result: Result<T, jmap_client::Error>) {\n    if !matches!(\n        result,\n        Err(jmap_client::Error::Method(MethodError {\n            p_type: MethodErrorType::Forbidden\n        })) | Err(jmap_client::Error::Set(SetError {\n            type_: SetErrorType::BlobNotFound | SetErrorType::Forbidden,\n            ..\n        }))\n    ) {\n        panic!(\"Expected forbidden, got {:?}\", result);\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/antispam.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse email::mailbox::{DRAFTS_ID, INBOX_ID, JUNK_ID};\nuse store::write::now;\nuse types::{id::Id, keyword::Keyword};\n\nuse crate::{imap::antispam::*, jmap::JMAPTest};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email Spam classifier tests...\");\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n    let account_id = account.id().document_id();\n\n    // Make sure there are no training samples\n    spam_delete_samples(&params.server).await;\n    assert_eq!(spam_training_samples(&params.server).await.total_count, 0);\n\n    // Import samples\n    let mut spam_ids = vec![];\n    let mut ham_ids = vec![];\n    for (idx, samples) in [&SPAM, &HAM].into_iter().enumerate() {\n        let is_spam = idx == 0;\n        for (num, sample) in samples.iter().enumerate() {\n            let mut mailbox_ids = vec![];\n            let mut keywords = vec![];\n\n            if num == 0 {\n                if is_spam {\n                    mailbox_ids.push(Id::from(JUNK_ID).to_string());\n                    keywords.push(Keyword::Junk.to_string());\n                } else {\n                    mailbox_ids.push(Id::from(INBOX_ID).to_string());\n                    keywords.push(Keyword::NotJunk.to_string());\n                }\n            } else {\n                mailbox_ids.push(Id::from(DRAFTS_ID).to_string());\n            }\n\n            let mail_id = client\n                .email_import(\n                    sample.as_bytes().to_vec(),\n                    &mailbox_ids,\n                    Some(&keywords),\n                    None,\n                )\n                .await\n                .unwrap()\n                .take_id();\n            if is_spam {\n                spam_ids.push(mail_id);\n            } else {\n                ham_ids.push(mail_id);\n            }\n        }\n    }\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 1);\n    assert_eq!(samples.spam_count, 1);\n\n    // Train the classifier via JMAP\n    for (ids, is_spam) in [(&spam_ids, true), (&ham_ids, false)] {\n        for (idx, id) in ids.iter().skip(1).enumerate() {\n            // Set keywords and mailboxes\n            let mut request = client.build();\n            let req = request.set_email().update(id);\n            if idx < 5 || !is_spam {\n                // Update via keywords\n                let keyword = if is_spam {\n                    Keyword::Junk\n                } else {\n                    Keyword::NotJunk\n                }\n                .to_string();\n                req.keywords([&keyword]);\n            } else {\n                // Update via mailbox\n                let mailbox_id = if is_spam { JUNK_ID } else { INBOX_ID };\n                req.mailbox_ids([&Id::from(mailbox_id).to_string()]);\n            }\n\n            request.send_set_email().await.unwrap().updated(id).unwrap();\n        }\n    }\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 10);\n    assert_eq!(samples.spam_count, 10);\n\n    // Reclassifying an email should not add a new sample\n    let mut request = client.build();\n    request\n        .set_email()\n        .update(&ham_ids[0])\n        .keywords([Keyword::Junk.to_string()]);\n    request\n        .send_set_email()\n        .await\n        .unwrap()\n        .updated(&ham_ids[0])\n        .unwrap();\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 9);\n    assert_eq!(samples.spam_count, 11);\n    assert_eq!(samples.samples.len(), 20);\n    let hold_for = params\n        .server\n        .core\n        .spam\n        .classifier\n        .as_ref()\n        .unwrap()\n        .hold_samples_for;\n    assert!(hold_for > 2 * 86400);\n    let hold_until = now() + hold_for;\n    let hold_range = (hold_until - 86400)..=hold_until;\n    assert!(samples.samples.iter().all(|s| s.account_id == account_id\n        && s.remove.is_none()\n        && hold_range.contains(&s.until)));\n\n    // Purging blobs should not remove training samples\n    params\n        .server\n        .store()\n        .purge_blobs(params.server.blob_store().clone())\n        .await\n        .unwrap();\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 9);\n    assert_eq!(samples.spam_count, 11);\n    assert_eq!(samples.samples.len(), 20);\n\n    // Extend hold period so a new training sample is generated\n    let old_core = params.server.core.clone();\n    let mut new_core = old_core.as_ref().clone();\n    new_core.spam.classifier.as_mut().unwrap().hold_samples_for += 2 * 86400;\n    params.server.inner.shared_core.store(Arc::new(new_core));\n\n    // Reclassifying an email will now add a new sample\n    let mut request = client.build();\n    request\n        .set_email()\n        .update(&ham_ids[0])\n        .keywords([Keyword::NotJunk.to_string()]);\n    request\n        .send_set_email()\n        .await\n        .unwrap()\n        .updated(&ham_ids[0])\n        .unwrap();\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 10);\n    assert_eq!(samples.spam_count, 11);\n    assert_eq!(samples.samples.len(), 21);\n\n    // Blob purge should remove the duplicated sample\n    params\n        .server\n        .store()\n        .purge_blobs(params.server.blob_store().clone())\n        .await\n        .unwrap();\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 10);\n    assert_eq!(samples.spam_count, 10);\n    assert_eq!(samples.samples.len(), 20);\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/changes.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::JMAPTest;\nuse jmap_proto::types::state::State;\nuse std::str::FromStr;\nuse store::{ahash::AHashSet, write::BatchBuilder};\nuse types::{\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email Changes tests...\");\n\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n    let mut states = vec![State::Initial];\n\n    for (changes, expected_changelog) in [\n        (\n            vec![\n                LogAction::Insert(0),\n                LogAction::Insert(1),\n                LogAction::Insert(2),\n            ],\n            vec![vec![vec![0, 1, 2], vec![], vec![]]],\n        ),\n        (\n            vec![\n                LogAction::Move(0, 3),\n                LogAction::Insert(4),\n                LogAction::Insert(5),\n                LogAction::Update(1),\n                LogAction::Update(2),\n            ],\n            vec![\n                vec![vec![1, 2, 3, 4, 5], vec![], vec![]],\n                vec![vec![3, 4, 5], vec![1, 2], vec![0]],\n            ],\n        ),\n        (\n            vec![\n                LogAction::Delete(1),\n                LogAction::Insert(6),\n                LogAction::Insert(7),\n                LogAction::Update(2),\n            ],\n            vec![\n                vec![vec![2, 3, 4, 5, 6, 7], vec![], vec![]],\n                vec![vec![3, 4, 5, 6, 7], vec![2], vec![0, 1]],\n                vec![vec![6, 7], vec![2], vec![1]],\n            ],\n        ),\n        (\n            vec![\n                LogAction::Update(4),\n                LogAction::Update(5),\n                LogAction::Update(6),\n                LogAction::Update(7),\n            ],\n            vec![\n                vec![vec![2, 3, 4, 5, 6, 7], vec![], vec![]],\n                vec![vec![3, 4, 5, 6, 7], vec![2], vec![0, 1]],\n                vec![vec![6, 7], vec![2, 4, 5], vec![1]],\n                vec![vec![], vec![4, 5, 6, 7], vec![]],\n            ],\n        ),\n        (\n            vec![\n                LogAction::Delete(4),\n                LogAction::Delete(5),\n                LogAction::Delete(6),\n                LogAction::Delete(7),\n            ],\n            vec![\n                vec![vec![2, 3], vec![], vec![]],\n                vec![vec![3], vec![2], vec![0, 1]],\n                vec![vec![], vec![2], vec![1, 4, 5]],\n                vec![vec![], vec![], vec![4, 5, 6, 7]],\n                vec![vec![], vec![], vec![4, 5, 6, 7]],\n            ],\n        ),\n        (\n            vec![\n                LogAction::Insert(8),\n                LogAction::Insert(9),\n                LogAction::Insert(10),\n                LogAction::Update(3),\n            ],\n            vec![\n                vec![vec![2, 3, 8, 9, 10], vec![], vec![]],\n                vec![vec![3, 8, 9, 10], vec![2], vec![0, 1]],\n                vec![vec![8, 9, 10], vec![2, 3], vec![1, 4, 5]],\n                vec![vec![8, 9, 10], vec![3], vec![4, 5, 6, 7]],\n                vec![vec![8, 9, 10], vec![3], vec![4, 5, 6, 7]],\n                vec![vec![8, 9, 10], vec![3], vec![]],\n            ],\n        ),\n        (\n            vec![LogAction::Update(2), LogAction::Update(8)],\n            vec![\n                vec![vec![2, 3, 8, 9, 10], vec![], vec![]],\n                vec![vec![3, 8, 9, 10], vec![2], vec![0, 1]],\n                vec![vec![8, 9, 10], vec![2, 3], vec![1, 4, 5]],\n                vec![vec![8, 9, 10], vec![2, 3], vec![4, 5, 6, 7]],\n                vec![vec![8, 9, 10], vec![2, 3], vec![4, 5, 6, 7]],\n                vec![vec![8, 9, 10], vec![2, 3], vec![]],\n                vec![vec![], vec![2, 8], vec![]],\n            ],\n        ),\n        (\n            vec![\n                LogAction::Move(9, 11),\n                LogAction::Move(10, 12),\n                LogAction::Delete(8),\n            ],\n            vec![\n                vec![vec![2, 3, 11, 12], vec![], vec![]],\n                vec![vec![3, 11, 12], vec![2], vec![0, 1]],\n                vec![vec![11, 12], vec![2, 3], vec![1, 4, 5]],\n                vec![vec![11, 12], vec![2, 3], vec![4, 5, 6, 7]],\n                vec![vec![11, 12], vec![2, 3], vec![4, 5, 6, 7]],\n                vec![vec![11, 12], vec![2, 3], vec![]],\n                vec![vec![11, 12], vec![2], vec![8, 9, 10]],\n                vec![vec![11, 12], vec![], vec![8, 9, 10]],\n            ],\n        ),\n    ]\n    .into_iter()\n    {\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(account.id().document_id())\n            .with_collection(Collection::Email);\n\n        for change in changes {\n            match change {\n                LogAction::Insert(id) => {\n                    batch\n                        .with_document(id as u32)\n                        .log_item_insert(SyncCollection::Email, None);\n                }\n                LogAction::Update(id) => {\n                    batch\n                        .with_document(id as u32)\n                        .log_item_update(SyncCollection::Email, None);\n                }\n                LogAction::Delete(id) => {\n                    batch\n                        .with_document(id as u32)\n                        .log_item_delete(SyncCollection::Email, None);\n                }\n                LogAction::UpdateChild(id) => {\n                    batch.log_container_property_change(SyncCollection::Email, id as u32);\n                }\n                LogAction::Move(old_id, new_id) => {\n                    batch\n                        .with_document(old_id as u32)\n                        .log_item_delete(SyncCollection::Email, None)\n                        .with_document(new_id as u32)\n                        .log_item_insert(SyncCollection::Email, None);\n                }\n            }\n        }\n\n        server\n            .core\n            .storage\n            .data\n            .write(batch.build_all())\n            .await\n            .unwrap();\n\n        let mut new_state = State::Initial;\n        for (test_num, state) in (states).iter().enumerate() {\n            let changes = client.email_changes(state.to_string(), None).await.unwrap();\n\n            assert_eq!(\n                expected_changelog[test_num],\n                [changes.created(), changes.updated(), changes.destroyed()]\n                    .into_iter()\n                    .map(|list| {\n                        let mut list = list\n                            .iter()\n                            .map(|i| Id::from_str(i).unwrap().into())\n                            .collect::<Vec<u64>>();\n                        list.sort_unstable();\n                        list\n                    })\n                    .collect::<Vec<Vec<_>>>(),\n                \"test_num: {}, state: {:?}\",\n                test_num,\n                state\n            );\n\n            if &State::Initial == state {\n                new_state = State::parse_str(changes.new_state()).unwrap();\n            }\n\n            for max_changes in 1..=8 {\n                let mut insertions = expected_changelog[test_num][0]\n                    .iter()\n                    .copied()\n                    .collect::<AHashSet<_>>();\n                let mut updates = expected_changelog[test_num][1]\n                    .iter()\n                    .copied()\n                    .collect::<AHashSet<_>>();\n                let mut deletions = expected_changelog[test_num][2]\n                    .iter()\n                    .copied()\n                    .collect::<AHashSet<_>>();\n\n                let mut int_state = state.clone();\n\n                for _ in 0..100 {\n                    let changes = client\n                        .email_changes(int_state.to_string(), max_changes.into())\n                        .await\n                        .unwrap();\n\n                    assert!(\n                        changes.created().len()\n                            + changes.updated().len()\n                            + changes.destroyed().len()\n                            <= max_changes,\n                        \"{} > {}\",\n                        changes.created().len()\n                            + changes.updated().len()\n                            + changes.destroyed().len(),\n                        max_changes\n                    );\n\n                    changes.created().iter().for_each(|id| {\n                        assert!(\n                            insertions.remove(&Id::from_str(id).unwrap()),\n                            \"{:?} != {}\",\n                            insertions,\n                            Id::from_str(id).unwrap()\n                        );\n                    });\n                    changes.updated().iter().for_each(|id| {\n                        assert!(\n                            updates.remove(&Id::from_str(id).unwrap()),\n                            \"{:?} != {}\",\n                            updates,\n                            Id::from_str(id).unwrap()\n                        );\n                    });\n                    changes.destroyed().iter().for_each(|id| {\n                        assert!(\n                            deletions.remove(&Id::from_str(id).unwrap()),\n                            \"{:?} != {}\",\n                            deletions,\n                            Id::from_str(id).unwrap()\n                        );\n                    });\n\n                    int_state = State::parse_str(changes.new_state()).unwrap();\n\n                    if !changes.has_more_changes() {\n                        break;\n                    }\n                }\n\n                assert_eq!(\n                    insertions.len(),\n                    0,\n                    \"test_num: {}, state: {:?}, pending: {:?}\",\n                    test_num,\n                    state,\n                    insertions\n                );\n                assert_eq!(\n                    updates.len(),\n                    0,\n                    \"test_num: {}, state: {:?}, pending: {:?}\",\n                    test_num,\n                    state,\n                    updates\n                );\n                assert_eq!(\n                    deletions.len(),\n                    0,\n                    \"test_num: {}, state: {:?}, pending: {:?}\",\n                    test_num,\n                    state,\n                    deletions\n                );\n            }\n        }\n\n        states.push(new_state);\n    }\n\n    let changes = client\n        .email_changes(State::Initial.to_string(), None)\n        .await\n        .unwrap();\n    let mut created = changes\n        .created()\n        .iter()\n        .map(|i| Id::from_str(i).unwrap().into())\n        .collect::<Vec<u64>>();\n    created.sort_unstable();\n\n    assert_eq!(created, vec![2, 3, 11, 12]);\n    assert_eq!(changes.updated(), Vec::<String>::new());\n    assert_eq!(changes.destroyed(), Vec::<String>::new());\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\n#[derive(Debug, Clone, Copy)]\npub enum LogAction {\n    Insert(u64),\n    Update(u64),\n    Delete(u64),\n    UpdateChild(u64),\n    Move(u64, u64),\n}\n\npub trait ParseState: Sized {\n    fn parse_str(state: &str) -> Option<Self>;\n}\n\nimpl ParseState for State {\n    fn parse_str(state: &str) -> Option<Self> {\n        State::parse(state)\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/copy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, mail::mailbox::destroy_all_mailboxes_for_account};\nuse jmap_client::mailbox::Role;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email Copy tests...\");\n    let account = params.account(\"admin\");\n    let mut client = account.client_owned().await;\n\n    // Create a mailbox on account 1\n    let ac1_mailbox_id = client\n        .set_default_account_id(Id::new(1).to_string())\n        .mailbox_create(\"Copy Test Ac# 1\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    // Insert a message on account 1\n    let ac1_email_id = client\n        .email_import(\n            concat!(\n                \"From: bill@example.com\\r\\n\",\n                \"To: jdoe@example.com\\r\\n\",\n                \"Subject: TPS Report\\r\\n\",\n                \"\\r\\n\",\n                \"I'm going to need those TPS reports ASAP. \",\n                \"So, if you could do that, that'd be great.\"\n            )\n            .as_bytes()\n            .to_vec(),\n            [&ac1_mailbox_id],\n            None::<Vec<&str>>,\n            None,\n        )\n        .await\n        .unwrap()\n        .take_id();\n\n    // Create a mailbox on account 2\n    let ac2_mailbox_id = client\n        .set_default_account_id(Id::new(2).to_string())\n        .mailbox_create(\"Copy Test Ac# 2\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    // Copy the email and delete it from the first account\n    let mut request = client.build();\n    request\n        .copy_email(Id::new(1).to_string())\n        .on_success_destroy_original(true)\n        .create(&ac1_email_id)\n        .mailbox_id(&ac2_mailbox_id, true)\n        .keyword(\"$draft\", true)\n        .received_at(311923920);\n    let ac2_email_id = request\n        .send()\n        .await\n        .unwrap()\n        .method_response_by_pos(0)\n        .unwrap_copy_email()\n        .unwrap()\n        .created(&ac1_email_id)\n        .unwrap()\n        .take_id();\n\n    // Check that the email was copied\n    let email = client\n        .email_get(&ac2_email_id, None::<Vec<_>>)\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(\n        email.preview().unwrap(),\n        \"I'm going to need those TPS reports ASAP. So, if you could do that, that'd be great.\"\n    );\n    assert_eq!(email.subject().unwrap(), \"TPS Report\");\n    assert_eq!(email.mailbox_ids(), &[&ac2_mailbox_id]);\n    assert_eq!(email.keywords(), &[\"$draft\"]);\n    assert_eq!(email.received_at().unwrap(), 311923920);\n\n    // Check that the email was deleted\n    assert!(\n        client\n            .set_default_account_id(Id::new(1).to_string())\n            .email_get(&ac1_email_id, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .is_none()\n    );\n\n    // Empty store\n    destroy_all_mailboxes_for_account(1).await;\n    destroy_all_mailboxes_for_account(2).await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/crypto.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, ManagementApi, mail::delivery::SmtpConnection};\nuse email::message::crypto::{\n    Algorithm, EncryptMessage, EncryptionMethod, EncryptionParams, EncryptionType, try_parse_certs,\n};\nuse mail_parser::{MessageParser, MimeHeaders};\nuse std::path::PathBuf;\nuse store::{\n    Deserialize, Serialize,\n    write::{Archive, Archiver},\n};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Encryption-at-rest tests...\");\n\n    // Check encryption\n    check_is_encrypted();\n    import_certs_and_encrypt().await;\n\n    // Create test account\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    // Build API\n    let api = ManagementApi::new(8899, \"jdoe@example.com\", \"12345\");\n\n    // Try importing using multiple methods and symmetric algos\n    for (file_name, method, num_certs) in [\n        (\"cert_smime.pem\", EncryptionMethod::SMIME, 3),\n        (\"cert_pgp.pem\", EncryptionMethod::PGP, 1),\n    ] {\n        let certs = std::fs::read_to_string(\n            PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n                .join(\"resources\")\n                .join(\"crypto\")\n                .join(file_name),\n        )\n        .unwrap();\n\n        for algo in [Algorithm::Aes128, Algorithm::Aes256] {\n            let request = match method {\n                EncryptionMethod::PGP => EncryptionType::PGP {\n                    algo,\n                    certs: certs.clone(),\n                    allow_spam_training: true,\n                },\n                EncryptionMethod::SMIME => EncryptionType::SMIME {\n                    algo,\n                    certs: certs.clone(),\n                    allow_spam_training: true,\n                },\n            };\n\n            assert_eq!(\n                api.post::<u32>(\"/api/account/crypto\", &request)\n                    .await\n                    .unwrap()\n                    .unwrap_data(),\n                num_certs\n            );\n        }\n    }\n\n    // Send a new message, which should be encrypted\n    let mut lmtp = SmtpConnection::connect().await;\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report (should be encrypted)\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n\n    // Send an encrypted message\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report (already encrypted)\\r\\n\",\n            \"Content-Type: application/pkcs7-mime; name=\\\"smime.p7m\\\"; smime-type=enveloped-data\\r\\n\",\n            \"\\r\\n\",\n            \"xjMEZMYfNhYJKwYBBAHaRw8BAQdAYyTN1HzqapLw8xwkCGwa0OjsgT/JqhcB/+Dy\",\n            \"Ga1fsBrNG0pvaG4gRG9lIDxqb2huQGV4YW1wbGUub3JnPsKJBBMWCAAxFiEEg836\",\n            \"pwbXpuQ/THMtpJwd4oBfIrUFAmTGHzYCGwMECwkIBwUVCAkKCwUWAgMBAAAKCRCk\",\n            \"nB3igF8itYhyAQD2jEdeYa3gyQ47X9YWZTK1wEJkN8W9//V1fYl2XQwqlQEA0qBv\",\n            \"Ai6nUh99oDw+/zQ8DFIKdeb5Ti4tu/X58PdpiQ7OOARkxh82EgorBgEEAZdVAQUB\",\n            \"AQdAvXz2FbFN0DovQF/ACnZyczTsSIQp0mvmF1PE+aijbC8DAQgHwngEGBYIACAW\",\n            \"IQSDzfqnBtem5D9Mcy2knB3igF8itQUCZMYfNgIbDAAKCRCknB3igF8itRnoAQC3\",\n            \"GzPmgx7TnB+SexPuJV/DoKSMJ0/X+hbEFcZkulxaDQEAh+xiJCvf+ZNAKw6kFhsL\",\n            \"UuZhEDktxnY6Ehz3aB7FawA=\",\n            \"=KGrr\",\n        ),\n    )\n    .await;\n\n    // Disable encryption\n    assert_eq!(\n        api.post::<Option<String>>(\"/api/account/crypto\", &EncryptionType::Disabled)\n            .await\n            .unwrap()\n            .unwrap_data(),\n        None\n    );\n\n    // Send a new message, which should NOT be encrypted\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report (plain text)\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n\n    // Check messages\n    let mut request = client.build();\n    request.get_email();\n    let emails = request.send_get_email().await.unwrap().take_list();\n    assert_eq!(emails.len(), 3, \"3 messages were expected: {:#?}.\", emails);\n\n    for email in emails {\n        let message =\n            String::from_utf8(client.download(email.blob_id().unwrap()).await.unwrap()).unwrap();\n        if message.contains(\"should be encrypted\") {\n            assert!(\n                message.contains(\"Content-Type: multipart/encrypted\"),\n                \"got message {message}, expected encrypted message\"\n            );\n        } else if message.contains(\"already encrypted\") {\n            assert!(\n                message.contains(\"Content-Type: application/pkcs7-mime\")\n                    && message.contains(\"xjMEZMYfNhYJKwYBBAHaRw8BAQdAYy\"),\n                \"got message {message}, expected message to be left intact\"\n            );\n        } else if message.contains(\"plain text\") {\n            assert!(\n                message.contains(\"I'm going to need those TPS reports ASAP.\"),\n                \"got message {message}, expected plain text message\"\n            );\n        } else {\n            panic!(\"Unexpected message: {:#?}\", message)\n        }\n    }\n}\n\npub async fn import_certs_and_encrypt() {\n    for (name, method, expected_certs) in [\n        (\"cert_pgp.pem\", EncryptionMethod::PGP, 1),\n        //(\"cert_pgp.der\", EncryptionMethod::PGP, 1),\n        (\"cert_smime.pem\", EncryptionMethod::SMIME, 3),\n        (\"cert_smime.der\", EncryptionMethod::SMIME, 1),\n    ] {\n        let mut certs = try_parse_certs(\n            method,\n            std::fs::read(\n                PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n                    .join(\"resources\")\n                    .join(\"crypto\")\n                    .join(name),\n            )\n            .unwrap(),\n        )\n        .expect(name);\n\n        assert_eq!(certs.len(), expected_certs);\n\n        if method == EncryptionMethod::PGP && certs.len() == 2 {\n            // PGP library won't encrypt using EC\n            let mut certs_ = certs.to_vec();\n            certs_.pop();\n            certs = certs_.into();\n        }\n\n        let mut params = EncryptionParams {\n            certs,\n            flags: method.flags(),\n        };\n\n        for algo in [Algorithm::Aes128, Algorithm::Aes256] {\n            let message = MessageParser::new()\n                .parse(b\"Subject: test\\r\\ntest\\r\\n\")\n                .unwrap();\n            assert!(!message.is_encrypted());\n            params.flags = algo.flags() | method.flags();\n            let arch =\n                Archive::deserialize_owned(Archiver::new(params.clone()).serialize().unwrap())\n                    .unwrap();\n            message\n                .encrypt(arch.unarchive::<EncryptionParams>().unwrap())\n                .await\n                .unwrap();\n        }\n    }\n\n    // S/MIME and PGP should not be allowed mixed\n    assert!(\n        try_parse_certs(\n            EncryptionMethod::PGP,\n            std::fs::read(\n                PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n                    .join(\"resources\")\n                    .join(\"crypto\")\n                    .join(\"cert_mixed.pem\"),\n            )\n            .unwrap(),\n        )\n        .is_err()\n    );\n}\n\npub fn check_is_encrypted() {\n    let messages = std::fs::read_to_string(\n        PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n            .join(\"resources\")\n            .join(\"crypto\")\n            .join(\"is_encrypted.txt\"),\n    )\n    .unwrap();\n\n    for raw_message in messages.split(\"!!!\") {\n        let is_encrypted = raw_message.contains(\"TRUE\");\n        let message = MessageParser::new()\n            .parse(raw_message.trim().as_bytes())\n            .unwrap();\n        assert!(message.content_type().is_some());\n        assert_eq!(\n            message.is_encrypted(),\n            is_encrypted,\n            \"failed for {raw_message}\"\n        );\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/delivery.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    directory::internal::TestInternalDirectory,\n    imap::antispam::{spam_delete_samples, spam_training_samples},\n    jmap::JMAPTest,\n    store::cleanup::store_blob_expire_all,\n    webdav::DummyWebDavClient,\n};\nuse common::Server;\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    mailbox::{INBOX_ID, JUNK_ID, SENT_ID},\n    message::metadata::MessageMetadata,\n};\nuse groupware::DavResourceName;\nuse jmap::blob::download::BlobDownload;\nuse std::{sync::Arc, time::Duration};\nuse store::{\n    ValueKey,\n    roaring::RoaringBitmap,\n    write::{AlignedBytes, Archive},\n};\nuse tokio::{\n    io::{AsyncBufReadExt, AsyncWriteExt, BufReader, Lines, ReadHalf, WriteHalf},\n    net::TcpStream,\n};\nuse types::{\n    blob::{BlobClass, BlobId},\n    collection::Collection,\n    field::EmailField,\n    id::Id,\n};\nuse utils::chained_bytes::ChainedBytes;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running message delivery tests...\");\n\n    // Enable delivered to\n    let old_core = params.server.core.clone();\n    let mut new_core = old_core.as_ref().clone();\n    new_core.smtp.session.data.add_delivered_to = true;\n    params.server.inner.shared_core.store(Arc::new(new_core));\n\n    // Create a domain name and a test account\n    let server = params.server.clone();\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let bill = params.account(\"bill@example.com\");\n\n    // Create a mailing list\n    server\n        .store()\n        .create_test_list(\n            \"members@example.com\",\n            \"Mailing List\",\n            &[\n                \"jdoe@example.com\",\n                \"jane.smith@example.com\",\n                \"bill@example.com\",\n            ],\n        )\n        .await;\n\n    // Delivering to individuals\n    let mut lmtp = SmtpConnection::connect().await;\n    params.webhook.clear();\n\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n\n    let john_cache = server\n        .get_cached_messages(john.id().document_id())\n        .await\n        .unwrap();\n\n    assert_eq!(john_cache.emails.items.len(), 1);\n    assert_eq!(john_cache.in_mailbox(INBOX_ID).count(), 1);\n    assert_eq!(john_cache.in_mailbox(JUNK_ID).count(), 0);\n\n    // Make sure there are no spam training samples\n    spam_delete_samples(&params.server).await;\n    assert_eq!(spam_training_samples(&params.server).await.total_count, 0);\n\n    // Test spam filtering\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"john.doe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: john.doe@example.com\\r\\n\",\n            \"Subject: XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X\\r\\n\",\n            \"\\r\\n\",\n            \"--- Forwarded Message ---\\r\\n\\r\\n \",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n    let john_cache = server\n        .get_cached_messages(john.id().document_id())\n        .await\n        .unwrap();\n    let inbox_ids = john_cache\n        .in_mailbox(INBOX_ID)\n        .map(|e| e.document_id)\n        .collect::<RoaringBitmap>();\n    let junk_ids = john_cache\n        .in_mailbox(JUNK_ID)\n        .map(|e| e.document_id)\n        .collect::<RoaringBitmap>();\n    assert_eq!(john_cache.emails.items.len(), 2);\n    assert_eq!(inbox_ids.len(), 1);\n    assert_eq!(junk_ids.len(), 1);\n    assert_message_headers_contains(\n        &server,\n        john.id().document_id(),\n        junk_ids.min().unwrap(),\n        \"X-Spam-Status: Yes\",\n    )\n    .await;\n    assert_eq!(spam_training_samples(&params.server).await.total_count, 0);\n\n    // CardDAV spam override\n    let dav_client = DummyWebDavClient::new(u32::MAX, john.name(), john.secret(), john.emails()[0]);\n    dav_client\n        .request(\n            \"PUT\",\n            &format!(\n                \"{}/jdoe%40example.com/default/bill.vcf\",\n                DavResourceName::Card.base_path()\n            ),\n            r#\"BEGIN:VCARD\nVERSION:4.0\nFN:Bill Foobar\nEMAIL;TYPE=WORK:dmarc-bill@example.com\nUID:urn:uuid:e1ee798b-3d4c-41b0-b217-b9c918e4686f\nEND:VCARD\n\"#,\n        )\n        .await\n        .with_status(hyper::StatusCode::CREATED);\n    lmtp.ingest(\n        \"dmarc-bill@example.com\",\n        &[\"john.doe@example.com\"],\n        concat!(\n            \"From: dmarc-bill@example.com\\r\\n\",\n            \"To: john.doe@example.com\\r\\n\",\n            \"Subject: XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X\\r\\n\",\n            \"\\r\\n\",\n            \"--- Forwarded Message ---\\r\\n\\r\\n \",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n    let john_cache = server\n        .get_cached_messages(john.id().document_id())\n        .await\n        .unwrap();\n    let inbox_ids = john_cache\n        .in_mailbox(INBOX_ID)\n        .map(|e| e.document_id)\n        .collect::<RoaringBitmap>();\n    let junk_ids = john_cache\n        .in_mailbox(JUNK_ID)\n        .map(|e| e.document_id)\n        .collect::<RoaringBitmap>();\n    assert_eq!(john_cache.emails.items.len(), 3);\n    assert_eq!(inbox_ids.len(), 2);\n    assert_eq!(junk_ids.len(), 1);\n    dav_client.delete_default_containers().await;\n    assert_message_headers_contains(\n        &server,\n        john.id().document_id(),\n        inbox_ids.max().unwrap(),\n        \"X-Spam-Status: No, reason=card-exists\",\n    )\n    .await;\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 1);\n    assert_eq!(samples.spam_count, 0);\n\n    // Test trusted reply override\n    john.client()\n        .email_import(\n            concat!(\n                \"From: john.doe@example.com\\r\\n\",\n                \"To: dmarc-bill@example.com\\r\\n\",\n                \"Message-ID: <trusted@message-id.example.com>\\r\\n\",\n                \"Subject: XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X\\r\\n\",\n                \"\\r\\n\",\n                \"This is a trusted reply.\"\n            )\n            .as_bytes()\n            .to_vec(),\n            vec![Id::from(SENT_ID).to_string()],\n            None::<Vec<String>>,\n            None,\n        )\n        .await\n        .unwrap()\n        .take_id();\n    assert_eq!(\n        server\n            .get_cached_messages(john.id().document_id())\n            .await\n            .unwrap()\n            .emails\n            .items\n            .len(),\n        4\n    );\n    lmtp.ingest(\n        \"dmarc-bill@example.com\",\n        &[\"john.doe@example.com\"],\n        concat!(\n            \"From: dmarc-bill@example.com\\r\\n\",\n            \"To: john.doe@example.com\\r\\n\",\n            \"Message-ID: <other@message-id.example.com>\\r\\n\",\n            \"References: <trusted@message-id.example.com>\\r\\n\",\n            \"Subject: XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X\\r\\n\",\n            \"\\r\\n\",\n            \"--- Forwarded Message ---\\r\\n\\r\\n \",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n    let john_cache = server\n        .get_cached_messages(john.id().document_id())\n        .await\n        .unwrap();\n    let inbox_ids = john_cache\n        .in_mailbox(INBOX_ID)\n        .map(|e| e.document_id)\n        .collect::<RoaringBitmap>();\n    let junk_ids = john_cache\n        .in_mailbox(JUNK_ID)\n        .map(|e| e.document_id)\n        .collect::<RoaringBitmap>();\n    assert_eq!(john_cache.emails.items.len(), 5);\n    assert_eq!(inbox_ids.len(), 3);\n    assert_eq!(junk_ids.len(), 1);\n    assert_message_headers_contains(\n        &server,\n        john.id().document_id(),\n        inbox_ids.max().unwrap(),\n        \"X-Spam-Status: No, reason=trusted-reply\",\n    )\n    .await;\n    let samples = spam_training_samples(&params.server).await;\n    assert_eq!(samples.ham_count, 2);\n    assert_eq!(samples.spam_count, 0);\n\n    // EXPN and VRFY\n    lmtp.expn(\"members@example.com\", 2)\n        .await\n        .assert_contains(\"jdoe@example.com\")\n        .assert_contains(\"jane.smith@example.com\")\n        .assert_contains(\"bill@example.com\");\n    lmtp.expn(\"non_existant@example.com\", 5).await;\n    lmtp.expn(\"jdoe@example.com\", 5).await;\n    lmtp.vrfy(\"jdoe@example.com\", 2).await;\n    lmtp.vrfy(\"members@example.com\", 5).await;\n    lmtp.vrfy(\"non_existant@example.com\", 5).await;\n\n    // Delivering to a mailing list\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"members@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: members@example.com\\r\\n\",\n            \"Subject: WFH policy\\r\\n\",\n            \"\\r\\n\",\n            \"We need the entire staff back in the office, \",\n            \"TPS reports cannot be filed properly from home.\"\n        ),\n    )\n    .await;\n\n    tokio::time::sleep(Duration::from_millis(200)).await;\n\n    for (account, num_messages) in [(john, 6), (jane, 1), (bill, 1)] {\n        assert_eq!(\n            server\n                .get_cached_messages(account.id().document_id())\n                .await\n                .unwrap()\n                .emails\n                .items\n                .len(),\n            num_messages,\n            \"for {}\",\n            account.id_string()\n        );\n    }\n\n    // Removing members from the mailing list and chunked ingest\n    params\n        .server\n        .core\n        .storage\n        .data\n        .remove_from_group(\"jdoe@example.com\", \"members@example.com\")\n        .await;\n    lmtp.ingest_chunked(\n        \"bill@example.com\",\n        &[\"members@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: members@example.com\\r\\n\",\n            \"Subject: WFH policy (reminder)\\r\\n\",\n            \"\\r\\n\",\n            \"This is a reminder that we need the entire staff back in the office, \",\n            \"TPS reports cannot be filed properly from home.\"\n        ),\n        10,\n    )\n    .await;\n\n    for (account, num_messages) in [(john, 6), (jane, 2), (bill, 2)] {\n        assert_eq!(\n            server\n                .get_cached_messages(account.id().document_id())\n                .await\n                .unwrap()\n                .emails\n                .items\n                .len(),\n            num_messages,\n            \"for {}\",\n            account.id_string()\n        );\n    }\n\n    // Deduplication of recipients\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\n            \"members@example.com\",\n            \"jdoe@example.com\",\n            \"john.doe@example.com\",\n            \"jane.smith@example.com\",\n            \"bill@example.com\",\n        ],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"Bcc: Undisclosed recipients;\\r\\n\",\n            \"Subject: Holidays\\r\\n\",\n            \"\\r\\n\",\n            \"Remember to file your TPS reports before \",\n            \"going on holidays.\"\n        ),\n    )\n    .await;\n\n    // Make sure blobs are properly linked\n    store_blob_expire_all(params.server.store()).await;\n\n    for (account, num_messages) in [(john, 7), (jane, 3), (bill, 3)] {\n        let account_id = account.id().document_id();\n        let cache = server.get_cached_messages(account_id).await.unwrap();\n        assert_eq!(\n            cache.emails.items.len(),\n            num_messages,\n            \"for {}\",\n            account.id_string()\n        );\n        let access_token = server.get_access_token(account_id).await.unwrap();\n\n        for document_id in cache.in_mailbox(INBOX_ID).map(|e| e.document_id) {\n            let metadata = message_metadata(&server, account_id, document_id).await;\n            let partial_message = server\n                .store()\n                .get_blob(metadata.blob_hash.0.as_ref(), 0..usize::MAX)\n                .await\n                .unwrap()\n                .unwrap();\n            assert_ne!(metadata.blob_body_offset, 0);\n            let expected_full_message = String::from_utf8(\n                ChainedBytes::new(metadata.raw_headers.as_ref())\n                    .with_last(\n                        partial_message\n                            .get(metadata.blob_body_offset as usize..)\n                            .unwrap_or_default(),\n                    )\n                    .to_bytes(),\n            )\n            .unwrap();\n            assert!(\n                expected_full_message.contains(\"Delivered-To:\")\n                    && expected_full_message.contains(\"Subject:\"),\n                \"for {account_id}: {expected_full_message}\"\n            );\n            let full_message = String::from_utf8(\n                server\n                    .blob_download(\n                        &BlobId {\n                            hash: metadata.blob_hash,\n                            class: BlobClass::Linked {\n                                account_id,\n                                collection: Collection::Email.into(),\n                                document_id,\n                            },\n                            section: None,\n                        },\n                        &access_token,\n                    )\n                    .await\n                    .unwrap()\n                    .unwrap(),\n            )\n            .unwrap();\n            assert_eq!(full_message, expected_full_message, \"for {account_id}\");\n        }\n    }\n\n    // Remove test data\n    for account in [john, jane, bill] {\n        params.destroy_all_mailboxes(account).await;\n    }\n    params.assert_is_empty().await;\n\n    // Restore core\n    params.server.inner.shared_core.store(old_core);\n\n    // Check webhook events\n    params.webhook.assert_contains(&[\n        \"message-ingest.\",\n        \"delivery.dsn\",\n        \"\\\"from\\\": \\\"bill@example.com\\\"\",\n        \"\\\"john.doe@example.com\\\"\",\n    ]);\n}\n\nasync fn assert_message_headers_contains(\n    server: &Server,\n    account_id: u32,\n    document_id: u32,\n    value: &str,\n) {\n    let headers = message_headers(server, account_id, document_id).await;\n    assert!(\n        headers.contains(value),\n        \"Expected message headers to contain {:?}, got {:?}\",\n        value,\n        headers\n    );\n}\n\nasync fn message_headers(server: &Server, account_id: u32, document_id: u32) -> String {\n    std::str::from_utf8(\n        message_metadata(server, account_id, document_id)\n            .await\n            .raw_headers\n            .as_ref(),\n    )\n    .unwrap()\n    .to_string()\n}\n\nasync fn message_metadata(server: &Server, account_id: u32, document_id: u32) -> MessageMetadata {\n    server\n        .store()\n        .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n            account_id,\n            Collection::Email,\n            document_id,\n            EmailField::Metadata,\n        ))\n        .await\n        .unwrap()\n        .unwrap()\n        .deserialize::<MessageMetadata>()\n        .unwrap()\n}\n\npub struct SmtpConnection {\n    reader: Lines<BufReader<ReadHalf<TcpStream>>>,\n    writer: WriteHalf<TcpStream>,\n}\n\nimpl SmtpConnection {\n    pub async fn ingest_with_code(\n        &mut self,\n        from: &str,\n        recipients: &[&str],\n        message: &str,\n        code: u8,\n    ) -> Vec<String> {\n        self.mail_from(from, 2).await;\n        for recipient in recipients {\n            self.rcpt_to(recipient, 2).await;\n        }\n        self.data(3).await;\n        let result = self.data_bytes(message, recipients.len(), code).await;\n        tokio::time::sleep(Duration::from_millis(500)).await;\n        result\n    }\n\n    pub async fn ingest(&mut self, from: &str, recipients: &[&str], message: &str) {\n        self.ingest_with_code(from, recipients, message, 2).await;\n    }\n\n    async fn ingest_chunked(\n        &mut self,\n        from: &str,\n        recipients: &[&str],\n        message: &str,\n        chunk_size: usize,\n    ) {\n        self.mail_from(from, 2).await;\n        for recipient in recipients {\n            self.rcpt_to(recipient, 2).await;\n        }\n        for chunk in message.as_bytes().chunks(chunk_size) {\n            self.bdat(std::str::from_utf8(chunk).unwrap(), 2).await;\n        }\n        self.bdat_last(\"\", recipients.len(), 2).await;\n        tokio::time::sleep(Duration::from_millis(500)).await;\n    }\n\n    pub async fn connect() -> Self {\n        SmtpConnection::connect_port(11200).await\n    }\n\n    pub async fn connect_port(port: u16) -> Self {\n        let (reader, writer) = tokio::io::split(\n            TcpStream::connect(&format!(\"127.0.0.1:{port}\"))\n                .await\n                .unwrap(),\n        );\n        let mut conn = SmtpConnection {\n            reader: BufReader::new(reader).lines(),\n            writer,\n        };\n        conn.read(1, 2).await;\n        conn.lhlo().await;\n        conn\n    }\n\n    pub async fn lhlo(&mut self) -> Vec<String> {\n        self.send(\"LHLO localhost\").await;\n        self.read(1, 2).await\n    }\n\n    pub async fn mail_from(&mut self, sender: &str, code: u8) -> Vec<String> {\n        self.send(&format!(\"MAIL FROM:<{}>\", sender)).await;\n        self.read(1, code).await\n    }\n\n    pub async fn rcpt_to(&mut self, rcpt: &str, code: u8) -> Vec<String> {\n        self.send(&format!(\"RCPT TO:<{}>\", rcpt)).await;\n        self.read(1, code).await\n    }\n\n    pub async fn vrfy(&mut self, rcpt: &str, code: u8) -> Vec<String> {\n        self.send(&format!(\"VRFY {}\", rcpt)).await;\n        self.read(1, code).await\n    }\n\n    pub async fn expn(&mut self, rcpt: &str, code: u8) -> Vec<String> {\n        self.send(&format!(\"EXPN {}\", rcpt)).await;\n        self.read(1, code).await\n    }\n\n    pub async fn data(&mut self, code: u8) -> Vec<String> {\n        self.send(\"DATA\").await;\n        self.read(1, code).await\n    }\n\n    pub async fn data_bytes(\n        &mut self,\n        message: &str,\n        num_responses: usize,\n        code: u8,\n    ) -> Vec<String> {\n        self.send_raw(message).await;\n        self.send_raw(\"\\r\\n.\\r\\n\").await;\n        self.read(num_responses, code).await\n    }\n\n    pub async fn bdat(&mut self, chunk: &str, code: u8) -> Vec<String> {\n        self.send_raw(&format!(\"BDAT {}\\r\\n{}\", chunk.len(), chunk))\n            .await;\n        self.read(1, code).await\n    }\n\n    pub async fn bdat_last(&mut self, chunk: &str, num_responses: usize, code: u8) -> Vec<String> {\n        self.send_raw(&format!(\"BDAT {} LAST\\r\\n{}\", chunk.len(), chunk))\n            .await;\n        self.read(num_responses, code).await\n    }\n\n    pub async fn rset(&mut self) -> Vec<String> {\n        self.send(\"RSET\").await;\n        self.read(1, 2).await\n    }\n\n    pub async fn noop(&mut self) -> Vec<String> {\n        self.send(\"NOOP\").await;\n        self.read(1, 2).await\n    }\n\n    pub async fn quit(&mut self) -> Vec<String> {\n        self.send(\"QUIT\").await;\n        self.read(1, 2).await\n    }\n\n    pub async fn read(&mut self, mut num_responses: usize, code: u8) -> Vec<String> {\n        let mut lines = Vec::new();\n        loop {\n            match tokio::time::timeout(Duration::from_millis(1500), self.reader.next_line()).await {\n                Ok(Ok(Some(line))) => {\n                    let is_done = line.as_bytes()[3] == b' ';\n                    //let c = println!(\"<- {:?}\", line);\n                    lines.push(line);\n                    if is_done {\n                        num_responses -= 1;\n                        if num_responses != 0 {\n                            continue;\n                        }\n\n                        if code != u8::MAX {\n                            for line in &lines {\n                                if line.as_bytes()[0] - b'0' != code {\n                                    panic!(\"Expected completion code {}, got {:?}.\", code, lines);\n                                }\n                            }\n                        }\n                        return lines;\n                    }\n                }\n                Ok(Ok(None)) => {\n                    panic!(\"Invalid response: {:?}.\", lines);\n                }\n                Ok(Err(err)) => {\n                    panic!(\"Connection broken: {} ({:?})\", err, lines);\n                }\n                Err(_) => panic!(\"Timeout while waiting for server response: {:?}\", lines),\n            }\n        }\n    }\n\n    pub async fn send(&mut self, text: &str) {\n        //let c = println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n        self.writer.write_all(b\"\\r\\n\").await.unwrap();\n        self.writer.flush().await.unwrap();\n    }\n\n    pub async fn send_raw(&mut self, text: &str) {\n        //let c = println!(\"-> {:?}\", text);\n        self.writer.write_all(text.as_bytes()).await.unwrap();\n    }\n}\n\npub trait AssertResult: Sized {\n    fn assert_contains(self, text: &str) -> Self;\n    fn assert_count(self, text: &str, occurrences: usize) -> Self;\n    fn assert_equals(self, text: &str) -> Self;\n}\n\nimpl AssertResult for Vec<String> {\n    fn assert_contains(self, text: &str) -> Self {\n        for line in &self {\n            if line.contains(text) {\n                return self;\n            }\n        }\n        panic!(\"Expected response to contain {:?}, got {:?}\", text, self);\n    }\n\n    fn assert_count(self, text: &str, occurrences: usize) -> Self {\n        assert_eq!(\n            self.iter().filter(|l| l.contains(text)).count(),\n            occurrences,\n            \"Expected {} occurrences of {:?}, found {}.\",\n            occurrences,\n            text,\n            self.iter().filter(|l| l.contains(text)).count()\n        );\n        self\n    }\n\n    fn assert_equals(self, text: &str) -> Self {\n        for line in &self {\n            if line == text {\n                return self;\n            }\n        }\n        panic!(\"Expected response to be {:?}, got {:?}\", text, self);\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, replace_blob_ids};\nuse ::email::mailbox::INBOX_ID;\nuse jmap_client::email::{self, Header, HeaderForm, import::EmailImportResponse};\nuse mail_parser::HeaderName;\nuse std::{fs, path::PathBuf};\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email Get tests...\");\n\n    let mut test_dir = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    test_dir.push(\"resources\");\n    test_dir.push(\"jmap\");\n    test_dir.push(\"email_get\");\n\n    let mailbox_id = Id::from(INBOX_ID).to_string();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    for file_name in fs::read_dir(&test_dir).unwrap() {\n        let mut file_name = file_name.as_ref().unwrap().path();\n        if file_name.extension().is_none_or(|e| e != \"eml\") {\n            continue;\n        }\n        let is_headers_test = file_name.file_name().unwrap() == \"headers.eml\";\n\n        let blob = fs::read(&file_name).unwrap();\n        let blob_len = blob.len();\n\n        // Import email\n        let mut request = client.build();\n        let import_request = request\n            .import_email()\n            .email(\n                client\n                    .upload(None, blob, None)\n                    .await\n                    .unwrap()\n                    .take_blob_id(),\n            )\n            .mailbox_ids([mailbox_id.clone()])\n            .keywords([\"tag\".to_string()])\n            .received_at((blob_len * 1000000) as i64);\n        let id = import_request.create_id();\n        let mut response = request.send_single::<EmailImportResponse>().await.unwrap();\n        assert_ne!(response.old_state(), Some(response.new_state()));\n        let email = response.created(&id).unwrap();\n\n        let mut request = client.build();\n        request\n            .get_email()\n            .ids([email.id().unwrap()])\n            .properties([\n                email::Property::Id,\n                email::Property::BlobId,\n                email::Property::ThreadId,\n                email::Property::MailboxIds,\n                email::Property::Keywords,\n                email::Property::Size,\n                email::Property::ReceivedAt,\n                email::Property::MessageId,\n                email::Property::InReplyTo,\n                email::Property::References,\n                email::Property::Sender,\n                email::Property::From,\n                email::Property::To,\n                email::Property::Cc,\n                email::Property::Bcc,\n                email::Property::ReplyTo,\n                email::Property::Subject,\n                email::Property::SentAt,\n                email::Property::HasAttachment,\n                email::Property::Preview,\n                email::Property::BodyValues,\n                email::Property::TextBody,\n                email::Property::HtmlBody,\n                email::Property::Attachments,\n                email::Property::BodyStructure,\n            ])\n            .arguments()\n            .body_properties(if !is_headers_test {\n                [\n                    email::BodyProperty::PartId,\n                    email::BodyProperty::BlobId,\n                    email::BodyProperty::Size,\n                    email::BodyProperty::Name,\n                    email::BodyProperty::Type,\n                    email::BodyProperty::Charset,\n                    email::BodyProperty::Headers,\n                    email::BodyProperty::Disposition,\n                    email::BodyProperty::Cid,\n                    email::BodyProperty::Language,\n                    email::BodyProperty::Location,\n                ]\n            } else {\n                [\n                    email::BodyProperty::PartId,\n                    email::BodyProperty::Size,\n                    email::BodyProperty::Name,\n                    email::BodyProperty::Type,\n                    email::BodyProperty::Charset,\n                    email::BodyProperty::Disposition,\n                    email::BodyProperty::Cid,\n                    email::BodyProperty::Language,\n                    email::BodyProperty::Location,\n                    email::BodyProperty::Header(Header {\n                        name: \"X-Custom-Header\".into(),\n                        form: HeaderForm::Raw,\n                        all: false,\n                    }),\n                    email::BodyProperty::Header(Header {\n                        name: \"X-Custom-Header-2\".into(),\n                        form: HeaderForm::Raw,\n                        all: false,\n                    }),\n                ]\n            })\n            .fetch_all_body_values(true)\n            .max_body_value_bytes(100);\n\n        let mut result = request\n            .send_get_email()\n            .await\n            .unwrap()\n            .take_list()\n            .pop()\n            .unwrap()\n            .into_test();\n\n        if is_headers_test {\n            for property in all_headers() {\n                let mut request = client.build();\n                request\n                    .get_email()\n                    .ids([email.id().unwrap()])\n                    .properties([property]);\n                result.headers.extend(\n                    request\n                        .send_get_email()\n                        .await\n                        .unwrap()\n                        .take_list()\n                        .pop()\n                        .unwrap()\n                        .into_test()\n                        .headers,\n                );\n            }\n        }\n\n        let result = replace_blob_ids(serde_json::to_string_pretty(&result).unwrap());\n\n        file_name.set_extension(\"json\");\n\n        if fs::read(&file_name).unwrap() != result.as_bytes() {\n            file_name.set_extension(\"failed\");\n            fs::write(&file_name, result.as_bytes()).unwrap();\n            panic!(\"Test failed, output saved to {}\", file_name.display());\n        }\n    }\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\npub fn all_headers() -> Vec<email::Property> {\n    let mut properties = Vec::new();\n\n    for header in [\n        HeaderName::From,\n        HeaderName::To,\n        HeaderName::Cc,\n        HeaderName::Bcc,\n        HeaderName::Other(\"X-Address-Single\".into()),\n        HeaderName::Other(\"X-Address\".into()),\n        HeaderName::Other(\"X-AddressList-Single\".into()),\n        HeaderName::Other(\"X-AddressList\".into()),\n        HeaderName::Other(\"X-AddressesGroup-Single\".into()),\n        HeaderName::Other(\"X-AddressesGroup\".into()),\n    ] {\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Addresses,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Addresses,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::GroupedAddresses,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::GroupedAddresses,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n    }\n\n    for header in [\n        HeaderName::ListPost,\n        HeaderName::ListSubscribe,\n        HeaderName::ListUnsubscribe,\n        HeaderName::ListOwner,\n        HeaderName::Other(\"X-List-Single\".into()),\n        HeaderName::Other(\"X-List\".into()),\n    ] {\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::URLs,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::URLs,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n    }\n\n    for header in [\n        HeaderName::Date,\n        HeaderName::ResentDate,\n        HeaderName::Other(\"X-Date-Single\".into()),\n        HeaderName::Other(\"X-Date\".into()),\n    ] {\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Date,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Date,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n    }\n\n    for header in [\n        HeaderName::MessageId,\n        HeaderName::References,\n        HeaderName::Other(\"X-Id-Single\".into()),\n        HeaderName::Other(\"X-Id\".into()),\n    ] {\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::MessageIds,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::MessageIds,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n    }\n\n    for header in [\n        HeaderName::Subject,\n        HeaderName::Keywords,\n        HeaderName::Other(\"X-Text-Single\".into()),\n        HeaderName::Other(\"X-Text\".into()),\n    ] {\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Raw,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Text,\n            name: header.as_str().to_string(),\n            all: true,\n        }));\n        properties.push(email::Property::Header(Header {\n            form: HeaderForm::Text,\n            name: header.as_str().to_string(),\n            all: false,\n        }));\n    }\n\n    properties\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/mailbox.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{Account, JMAPTest, wait_for_index};\nuse jmap_client::{\n    Error, Set,\n    client::{Client, Credentials},\n    core::{\n        query::Filter,\n        set::{SetError, SetErrorType, SetObject, SetRequest},\n    },\n    mailbox::{self, Mailbox, Role},\n};\nuse jmap_proto::types::state::State;\nuse serde::{Deserialize, Serialize};\nuse std::time::Duration;\nuse store::ahash::AHashMap;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Mailbox tests...\");\n    let account = params.account(\"admin\");\n    let mut client = account.client_owned().await;\n\n    // Create test mailboxes\n    client.set_default_account_id(Id::from(0u64));\n    let id_map = create_test_mailboxes(&client).await;\n\n    // Sort by name\n    assert_eq!(\n        client\n            .mailbox_query(\n                None::<mailbox::query::Filter>,\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\n            \"drafts\",\n            \"spam2\",\n            \"inbox\",\n            \"l.1\",\n            \"l.2\",\n            \"l.3\",\n            \"sent\",\n            \"spam\",\n            \"1.1\",\n            \"1.2\",\n            \"trash\",\n            \"spam1\",\n            \"1.1.1.1\",\n            \"1.1.1.1.1\",\n            \"1.1.1\",\n            \"1.2.1\"\n        ]\n    );\n\n    // Sort by name as tree\n    let mut request = client.build();\n    request\n        .query_mailbox()\n        .sort([mailbox::query::Comparator::name()])\n        .arguments()\n        .sort_as_tree(true);\n    assert_eq!(\n        request\n            .send_query_mailbox()\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\n            \"drafts\",\n            \"inbox\",\n            \"l.1\",\n            \"1.1\",\n            \"1.1.1\",\n            \"1.1.1.1\",\n            \"1.1.1.1.1\",\n            \"1.2\",\n            \"1.2.1\",\n            \"l.2\",\n            \"l.3\",\n            \"sent\",\n            \"spam\",\n            \"spam1\",\n            \"spam2\",\n            \"trash\"\n        ]\n    );\n\n    // Sort as tree with filters\n    let mut request = client.build();\n    request\n        .query_mailbox()\n        .filter(mailbox::query::Filter::name(\"level\"))\n        .sort([mailbox::query::Comparator::name()])\n        .arguments()\n        .sort_as_tree(true);\n    assert_eq!(\n        request\n            .send_query_mailbox()\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\n            \"l.1\",\n            \"1.1\",\n            \"1.1.1\",\n            \"1.1.1.1\",\n            \"1.1.1.1.1\",\n            \"1.2\",\n            \"1.2.1\",\n            \"l.2\",\n            \"l.3\"\n        ]\n    );\n\n    // Filter as tree\n    let mut request = client.build();\n    request\n        .query_mailbox()\n        .filter(mailbox::query::Filter::name(\"spam\"))\n        .sort([mailbox::query::Comparator::name()])\n        .arguments()\n        .filter_as_tree(true)\n        .sort_as_tree(true);\n    assert_eq!(\n        request\n            .send_query_mailbox()\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\"spam\", \"spam1\", \"spam2\"]\n    );\n\n    let mut request = client.build();\n    request\n        .query_mailbox()\n        .filter(mailbox::query::Filter::name(\"level\"))\n        .sort([mailbox::query::Comparator::name()])\n        .arguments()\n        .filter_as_tree(true)\n        .sort_as_tree(true);\n    assert_eq!(\n        request.send_query_mailbox().await.unwrap().ids(),\n        Vec::<&str>::new()\n    );\n\n    // Filter by role\n    assert_eq!(\n        client\n            .mailbox_query(\n                mailbox::query::Filter::role(Role::Inbox).into(),\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\"inbox\"]\n    );\n\n    assert_eq!(\n        client\n            .mailbox_query(\n                mailbox::query::Filter::has_any_role(true).into(),\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\"drafts\", \"inbox\", \"sent\", \"spam\", \"trash\"]\n    );\n\n    // Duplicate role\n    let mut request = client.build();\n    request\n        .set_mailbox()\n        .update(&id_map[\"sent\"])\n        .role(Role::Inbox);\n    assert!(matches!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .updated(&id_map[\"sent\"]),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n\n    // Duplicate name\n    let mut request = client.build();\n    request.set_mailbox().update(&id_map[\"l.2\"]).name(\"Level 3\");\n    let result = request\n        .send_set_mailbox()\n        .await\n        .unwrap()\n        .updated(&id_map[\"l.2\"]);\n    assert!(\n        matches!(\n            result,\n            Err(Error::Set(SetError {\n                type_: SetErrorType::InvalidProperties,\n                ..\n            }))\n        ),\n        \"{result:?}\",\n    );\n\n    // Circular relationship\n    let mut request = client.build();\n    request\n        .set_mailbox()\n        .update(&id_map[\"l.1\"])\n        .parent_id((&id_map[\"1.1.1.1.1\"]).into());\n    assert!(matches!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .updated(&id_map[\"l.1\"]),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n\n    let mut request = client.build();\n    request\n        .set_mailbox()\n        .update(&id_map[\"l.1\"])\n        .parent_id((&id_map[\"l.1\"]).into());\n    assert!(matches!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .updated(&id_map[\"l.1\"]),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n\n    // Invalid parentId\n    let mut request = client.build();\n    request\n        .set_mailbox()\n        .update(&id_map[\"l.1\"])\n        .parent_id(Id::new(u64::MAX).to_string().into());\n    assert!(matches!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .updated(&id_map[\"l.1\"]),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n\n    // Obtain state\n    let state = client\n        .mailbox_changes(State::Initial.to_string(), 0)\n        .await\n        .unwrap()\n        .new_state()\n        .to_string();\n\n    // Rename and move mailbox\n    let mut request = client.build();\n    request\n        .set_mailbox()\n        .update(&id_map[\"1.1.1.1.1\"])\n        .name(\"Renamed and moved\")\n        .parent_id((&id_map[\"l.2\"]).into());\n    assert!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .updated(&id_map[\"1.1.1.1.1\"])\n            .is_ok()\n    );\n\n    // Verify changes\n    let state = client.mailbox_changes(state, 0).await.unwrap();\n    assert_eq!(state.created().len(), 0);\n    assert_eq!(state.updated().len(), 1);\n    assert_eq!(state.destroyed().len(), 0);\n    assert_eq!(state.arguments().updated_properties(), None);\n    let state = state.new_state().to_string();\n\n    // Insert email into Inbox\n    let mail_id = client\n        .email_import(\n            b\"From: test@test.com\\nSubject: hey\\n\\ntest\".to_vec(),\n            [&id_map[\"inbox\"]],\n            None::<Vec<&str>>,\n            None,\n        )\n        .await\n        .unwrap()\n        .take_id();\n\n    // Inbox's total and unread count should have increased\n    let inbox = client\n        .mailbox_get(\n            &id_map[\"inbox\"],\n            [\n                mailbox::Property::TotalEmails,\n                mailbox::Property::UnreadEmails,\n                mailbox::Property::TotalThreads,\n                mailbox::Property::UnreadThreads,\n            ]\n            .into(),\n        )\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(inbox.total_emails(), 1);\n    assert_eq!(inbox.unread_emails(), 1);\n    assert_eq!(inbox.total_threads(), 1);\n    assert_eq!(inbox.unread_threads(), 1);\n\n    // Set email to read and fetch properties again\n    client\n        .email_set_keyword(&mail_id, \"$seen\", true)\n        .await\n        .unwrap();\n    let inbox = client\n        .mailbox_get(\n            &id_map[\"inbox\"],\n            [\n                mailbox::Property::TotalEmails,\n                mailbox::Property::UnreadEmails,\n                mailbox::Property::TotalThreads,\n                mailbox::Property::UnreadThreads,\n            ]\n            .into(),\n        )\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(inbox.total_emails(), 1);\n    assert_eq!(inbox.unread_emails(), 0);\n    assert_eq!(inbox.total_threads(), 1);\n    assert_eq!(inbox.unread_threads(), 0);\n\n    // Only email properties must have changed\n    let prev_state = state.clone();\n    let state = client.mailbox_changes(state, 0).await.unwrap();\n    assert_eq!(state.created().len(), 0);\n    assert_eq!(\n        state\n            .updated()\n            .iter()\n            .map(|s| s.as_str())\n            .collect::<Vec<_>>(),\n        &[&id_map[\"inbox\"]]\n    );\n    assert_eq!(state.destroyed().len(), 0);\n    assert_eq!(\n        state.arguments().updated_properties(),\n        Some(\n            &[\n                mailbox::Property::TotalEmails,\n                mailbox::Property::UnreadEmails,\n                mailbox::Property::TotalThreads,\n                mailbox::Property::UnreadThreads,\n            ][..]\n        )\n    );\n    let state = state.new_state().to_string();\n\n    // Use updatedProperties in a query\n    let mut request = client.build();\n    let changes_request = request.changes_mailbox(prev_state).max_changes(0);\n    let properties_ref = changes_request.updated_properties_reference();\n    let updated_ref = changes_request.updated_reference();\n    request\n        .get_mailbox()\n        .ids_ref(updated_ref)\n        .properties_ref(properties_ref);\n    let mut changed_mailboxes = request\n        .send()\n        .await\n        .unwrap()\n        .unwrap_method_responses()\n        .pop()\n        .unwrap()\n        .unwrap_get_mailbox()\n        .unwrap()\n        .take_list();\n    assert_eq!(changed_mailboxes.len(), 1);\n    let inbox = changed_mailboxes.pop().unwrap();\n    assert_eq!(inbox.id().unwrap(), &id_map[\"inbox\"]);\n    assert_eq!(inbox.total_emails(), 1);\n    assert_eq!(inbox.unread_emails(), 0);\n    assert_eq!(inbox.total_threads(), 1);\n    assert_eq!(inbox.unread_threads(), 0);\n    assert_eq!(inbox.name(), None);\n    assert_eq!(inbox.my_rights(), None);\n\n    // Move email from Inbox to Trash\n    client\n        .email_set_mailboxes(&mail_id, [&id_map[\"trash\"]])\n        .await\n        .unwrap();\n\n    // E-mail properties of both Inbox and Trash must have changed\n    let state = client.mailbox_changes(state, 0).await.unwrap();\n    assert_eq!(state.created().len(), 0);\n    assert_eq!(state.updated().len(), 2);\n    assert_eq!(state.destroyed().len(), 0);\n    let mut folder_ids = vec![&id_map[\"trash\"], &id_map[\"inbox\"]];\n    let mut updated_ids = state\n        .updated()\n        .iter()\n        .map(|s| s.as_str())\n        .collect::<Vec<_>>();\n    updated_ids.sort_unstable();\n    folder_ids.sort_unstable();\n    assert_eq!(updated_ids, folder_ids);\n    assert_eq!(\n        state.arguments().updated_properties(),\n        Some(\n            &[\n                mailbox::Property::TotalEmails,\n                mailbox::Property::UnreadEmails,\n                mailbox::Property::TotalThreads,\n                mailbox::Property::UnreadThreads,\n            ][..]\n        )\n    );\n\n    // Deleting folders with children is not allowed\n    let mut request = client.build();\n    request.set_mailbox().destroy([&id_map[\"l.1\"]]);\n    assert!(matches!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .destroyed(&id_map[\"l.1\"]),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::MailboxHasChild,\n            ..\n        }))\n    ));\n\n    // Deleting folders with contents is not allowed (unless remove_emails is true)\n    let mut request = client.build();\n    request.set_mailbox().destroy([&id_map[\"trash\"]]);\n    assert!(matches!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .destroyed(&id_map[\"trash\"]),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::MailboxHasEmail,\n            ..\n        }))\n    ));\n\n    // Delete Trash folder and its contents\n    let mut request = client.build();\n    request\n        .set_mailbox()\n        .destroy([&id_map[\"trash\"]])\n        .arguments()\n        .on_destroy_remove_emails(true);\n    assert!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .destroyed(&id_map[\"trash\"])\n            .is_ok()\n    );\n\n    // Verify that Trash folder and its contents are gone\n    assert!(\n        client\n            .mailbox_get(&id_map[\"trash\"], None::<Vec<_>>)\n            .await\n            .unwrap()\n            .is_none()\n    );\n    assert!(\n        client\n            .email_get(&mail_id, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .is_none()\n    );\n\n    // Check search results after changing folder properties\n    let mut request = client.build();\n    request\n        .set_mailbox()\n        .update(&id_map[\"drafts\"])\n        .name(\"Borradores\")\n        .sort_order(100)\n        .parent_id((&id_map[\"l.2\"]).into())\n        .role(Role::None);\n    assert!(\n        request\n            .send_set_mailbox()\n            .await\n            .unwrap()\n            .updated(&id_map[\"drafts\"])\n            .is_ok()\n    );\n    assert_eq!(\n        client\n            .mailbox_query(\n                Filter::and([\n                    mailbox::query::Filter::name(\"Borradores\").into(),\n                    mailbox::query::Filter::parent_id((&id_map[\"l.2\"]).into()).into(),\n                    Filter::not([mailbox::query::Filter::has_any_role(true)])\n                ])\n                .into(),\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\"drafts\"]\n    );\n    assert!(\n        client\n            .mailbox_query(\n                mailbox::query::Filter::name(\"Drafts\").into(),\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .is_empty()\n    );\n    assert!(\n        client\n            .mailbox_query(\n                mailbox::query::Filter::role(Role::Drafts).into(),\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .is_empty()\n    );\n    assert_eq!(\n        client\n            .mailbox_query(\n                mailbox::query::Filter::parent_id(None::<&str>).into(),\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\"inbox\", \"sent\", \"spam\"]\n    );\n    assert_eq!(\n        client\n            .mailbox_query(\n                mailbox::query::Filter::has_any_role(true).into(),\n                [mailbox::query::Comparator::name()].into()\n            )\n            .await\n            .unwrap()\n            .ids()\n            .iter()\n            .map(|id| id_map.get(id).unwrap())\n            .collect::<Vec<_>>(),\n        [\"inbox\", \"sent\", \"spam\"]\n    );\n\n    destroy_all_mailboxes_no_wait(&client).await;\n    params.assert_is_empty().await;\n}\n\nasync fn create_test_mailboxes(client: &Client) -> AHashMap<String, String> {\n    let mut mailbox_map = AHashMap::default();\n    let mut request = client.build();\n    build_create_query(\n        request.set_mailbox(),\n        &mut mailbox_map,\n        serde_json::from_slice(TEST_MAILBOXES).unwrap(),\n        None,\n    );\n    let mut result = request.send_set_mailbox().await.unwrap();\n    let mut id_map = AHashMap::with_capacity(mailbox_map.len());\n    for (create_id, local_id) in mailbox_map {\n        let server_id = result.created(&create_id).unwrap().take_id();\n        id_map.insert(local_id.clone(), server_id.clone());\n        id_map.insert(server_id, local_id);\n    }\n    id_map\n}\n\nfn build_create_query(\n    request: &mut SetRequest<Mailbox<Set>>,\n    mailbox_map: &mut AHashMap<String, String>,\n    mailboxes: Vec<TestMailbox>,\n    parent_id: Option<String>,\n) {\n    for mailbox in mailboxes {\n        let create_mailbox = request\n            .create()\n            .name(mailbox.name)\n            .sort_order(mailbox.order);\n        if let Some(role) = mailbox.role {\n            create_mailbox.role(role);\n        }\n        if let Some(parent_id) = &parent_id {\n            create_mailbox.parent_id_ref(parent_id);\n        }\n        let create_mailbox_id = create_mailbox.create_id().unwrap();\n        mailbox_map.insert(create_mailbox_id.clone(), mailbox.id);\n\n        if let Some(children) = mailbox.children {\n            build_create_query(request, mailbox_map, children, create_mailbox_id.into());\n        }\n    }\n}\n\nimpl JMAPTest {\n    pub async fn destroy_all_mailboxes(&self, account: &Account) {\n        wait_for_index(&self.server).await;\n        destroy_all_mailboxes_no_wait(account.client()).await;\n    }\n}\n\npub async fn destroy_all_mailboxes_for_account(account_id: u32) {\n    let mut client = Client::new()\n        .credentials(Credentials::basic(\"admin\", \"secret\"))\n        .follow_redirects([\"127.0.0.1\"])\n        .timeout(Duration::from_secs(3600))\n        .accept_invalid_certs(true)\n        .connect(\"https://127.0.0.1:8899\")\n        .await\n        .unwrap();\n    client.set_default_account_id(Id::from(account_id));\n    destroy_all_mailboxes_no_wait(&client).await;\n}\n\npub async fn destroy_all_mailboxes_no_wait(client: &Client) {\n    let mut request = client.build();\n    request.query_mailbox().arguments().sort_as_tree(true);\n    let mut ids = request.send_query_mailbox().await.unwrap().take_ids();\n    ids.reverse();\n    for id in ids {\n        client.mailbox_destroy(&id, true).await.unwrap();\n    }\n}\n\n#[derive(Serialize, Deserialize)]\nstruct TestMailbox {\n    id: String,\n    name: String,\n    role: Option<Role>,\n    order: u32,\n    children: Option<Vec<TestMailbox>>,\n}\n\nconst TEST_MAILBOXES: &[u8] = br#\"\n[\n    {\n        \"id\": \"inbox\",\n        \"name\": \"Inbox\",\n        \"role\": \"INBOX\",\n        \"order\": 5,\n        \"children\": [\n            {\n                \"name\": \"Level 1\",\n                \"id\": \"l.1\",\n                \"order\": 4,\n                \"children\": [\n                    {\n                        \"name\": \"Sub-Level 1.1\",\n                        \"id\": \"1.1\",\n\n                        \"order\": 3,\n                        \"children\": [\n                            {\n                                \"name\": \"Z-Sub-Level 1.1.1\",\n                                \"id\": \"1.1.1\",\n                                \"order\": 2,\n                                \"children\": [\n                                    {\n                                        \"name\": \"X-Sub-Level 1.1.1.1\",\n                                        \"id\": \"1.1.1.1\",\n                                        \"order\": 1,\n                                        \"children\": [\n                                            {\n                                                \"name\": \"Y-Sub-Level 1.1.1.1.1\",\n                                                \"id\": \"1.1.1.1.1\",\n                                                \"order\": 0\n                                            }\n                                        ]\n                                    }\n                                ]\n                            }\n                        ]\n                    },\n                    {\n                        \"name\": \"Sub-Level 1.2\",\n                        \"id\": \"1.2\",\n                        \"order\": 7,\n                        \"children\": [\n                            {\n                                \"name\": \"Z-Sub-Level 1.2.1\",\n                                \"id\": \"1.2.1\",\n                                \"order\": 6\n                            }\n                        ]\n                    }\n                ]\n            },\n            {\n                \"name\": \"Level 2\",\n                \"id\": \"l.2\",\n                \"order\": 8\n            },\n            {\n                \"name\": \"Level 3\",\n                \"id\": \"l.3\",\n                \"order\": 9\n            }\n        ]\n    },\n    {\n        \"id\": \"sent\",\n        \"name\": \"Sent\",\n        \"role\": \"SENT\",\n        \"order\": 15\n    },\n    {\n        \"id\": \"drafts\",\n        \"name\": \"Drafts\",\n        \"role\": \"DRAFTS\",\n        \"order\": 14\n    },\n    {\n        \"id\": \"trash\",\n        \"name\": \"Trash\",\n        \"role\": \"TRASH\",\n        \"order\": 13\n    },\n    {\n        \"id\": \"spam\",\n        \"name\": \"Spam\",\n        \"role\": \"JUNK\",\n        \"order\": 12,\n        \"children\": [{\n            \"id\": \"spam1\",\n            \"name\": \"Work Spam\",\n            \"order\": 11,\n            \"children\": [{\n                \"id\": \"spam2\",\n                \"name\": \"Friendly Spam\",\n                \"order\": 10\n            }]\n        }]\n    }\n]\n\"#;\n"
  },
  {
    "path": "tests/src/jmap/mail/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod acl;\npub mod antispam;\npub mod changes;\npub mod copy;\npub mod crypto;\npub mod delivery;\npub mod get;\npub mod mailbox;\npub mod parse;\npub mod query;\npub mod query_changes;\npub mod search_snippet;\npub mod set;\npub mod sieve_script;\npub mod submission;\npub mod thread_get;\npub mod thread_merge;\npub mod vacation_response;\n"
  },
  {
    "path": "tests/src/jmap/mail/parse.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, mail::get::all_headers, replace_blob_ids};\nuse jmap_client::{\n    email::{self, Header, HeaderForm},\n    mailbox::Role,\n};\nuse std::{fs, path::PathBuf};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email Parse tests...\");\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    let mut test_dir = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    test_dir.push(\"resources\");\n    test_dir.push(\"jmap\");\n    test_dir.push(\"email_parse\");\n\n    let mailbox_id = client\n        .mailbox_create(\"JMAP Parse\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    // Test parsing an email attachment\n    for test_name in [\"attachment.eml\", \"attachment_b64.eml\"] {\n        let mut test_file = test_dir.clone();\n        test_file.push(test_name);\n\n        let email = client\n            .email_import(\n                fs::read(&test_file).unwrap(),\n                [mailbox_id.clone()],\n                None::<Vec<String>>,\n                None,\n            )\n            .await\n            .unwrap();\n\n        let blob_id = client\n            .email_get(email.id().unwrap(), Some([email::Property::Attachments]))\n            .await\n            .unwrap()\n            .unwrap()\n            .attachments()\n            .unwrap()\n            .first()\n            .unwrap()\n            .blob_id()\n            .unwrap()\n            .to_string();\n\n        let email = client\n            .email_parse(\n                &blob_id,\n                [\n                    email::Property::Id,\n                    email::Property::BlobId,\n                    email::Property::ThreadId,\n                    email::Property::MailboxIds,\n                    email::Property::Keywords,\n                    email::Property::Size,\n                    email::Property::ReceivedAt,\n                    email::Property::MessageId,\n                    email::Property::InReplyTo,\n                    email::Property::References,\n                    email::Property::Sender,\n                    email::Property::From,\n                    email::Property::To,\n                    email::Property::Cc,\n                    email::Property::Bcc,\n                    email::Property::ReplyTo,\n                    email::Property::Subject,\n                    email::Property::SentAt,\n                    email::Property::HasAttachment,\n                    email::Property::Preview,\n                    email::Property::BodyValues,\n                    email::Property::TextBody,\n                    email::Property::HtmlBody,\n                    email::Property::Attachments,\n                    email::Property::BodyStructure,\n                ]\n                .into(),\n                [\n                    email::BodyProperty::PartId,\n                    email::BodyProperty::BlobId,\n                    email::BodyProperty::Size,\n                    email::BodyProperty::Name,\n                    email::BodyProperty::Type,\n                    email::BodyProperty::Charset,\n                    email::BodyProperty::Headers,\n                    email::BodyProperty::Disposition,\n                    email::BodyProperty::Cid,\n                    email::BodyProperty::Language,\n                    email::BodyProperty::Location,\n                ]\n                .into(),\n                100.into(),\n            )\n            .await\n            .unwrap();\n\n        if !test_name.contains(\"_b64\") {\n            for parts in [\n                email.text_body().unwrap(),\n                email.html_body().unwrap(),\n                email.attachments().unwrap(),\n            ] {\n                for part in parts {\n                    let blob_id = part.blob_id().unwrap();\n\n                    let inner_blob = client.download(blob_id).await.unwrap();\n\n                    test_file.set_extension(format!(\"part{}\", part.part_id().unwrap()));\n\n                    //fs::write(&test_file, inner_blob).unwrap();\n                    let expected_inner_blob = fs::read(&test_file).unwrap();\n\n                    assert_eq!(\n                        inner_blob,\n                        expected_inner_blob,\n                        \"file: {}\",\n                        test_file.display()\n                    );\n                }\n            }\n        }\n\n        test_file.set_extension(\"json\");\n\n        let result = replace_blob_ids(serde_json::to_string_pretty(&email.into_test()).unwrap());\n\n        if fs::read(&test_file).unwrap() != result.as_bytes() {\n            test_file.set_extension(\"failed\");\n            fs::write(&test_file, result.as_bytes()).unwrap();\n            panic!(\"Test failed, output saved to {}\", test_file.display());\n        }\n    }\n\n    // Test header parsing on a temporary blob\n    let mut test_file = test_dir;\n    test_file.push(\"headers.eml\");\n    let blob_id = client\n        .upload(None, fs::read(&test_file).unwrap(), None)\n        .await\n        .unwrap()\n        .take_blob_id();\n\n    let mut email = client\n        .email_parse(\n            &blob_id,\n            [\n                email::Property::Id,\n                email::Property::MessageId,\n                email::Property::InReplyTo,\n                email::Property::References,\n                email::Property::Sender,\n                email::Property::From,\n                email::Property::To,\n                email::Property::Cc,\n                email::Property::Bcc,\n                email::Property::ReplyTo,\n                email::Property::Subject,\n                email::Property::SentAt,\n                email::Property::Preview,\n                email::Property::TextBody,\n                email::Property::HtmlBody,\n                email::Property::Attachments,\n            ]\n            .into(),\n            [\n                email::BodyProperty::Size,\n                email::BodyProperty::Name,\n                email::BodyProperty::Type,\n                email::BodyProperty::Charset,\n                email::BodyProperty::Disposition,\n                email::BodyProperty::Cid,\n                email::BodyProperty::Language,\n                email::BodyProperty::Location,\n                email::BodyProperty::Header(Header {\n                    name: \"X-Custom-Header\".into(),\n                    form: HeaderForm::Raw,\n                    all: false,\n                }),\n                email::BodyProperty::Header(Header {\n                    name: \"X-Custom-Header-2\".into(),\n                    form: HeaderForm::Raw,\n                    all: false,\n                }),\n            ]\n            .into(),\n            100.into(),\n        )\n        .await\n        .unwrap()\n        .into_test();\n\n    for property in all_headers() {\n        email.headers.extend(\n            client\n                .email_parse(&blob_id, [property].into(), [].into(), None)\n                .await\n                .unwrap()\n                .into_test()\n                .headers,\n        );\n    }\n\n    test_file.set_extension(\"json\");\n\n    let result = replace_blob_ids(serde_json::to_string_pretty(&email).unwrap());\n\n    if fs::read(&test_file).unwrap() != result.as_bytes() {\n        test_file.set_extension(\"failed\");\n        fs::write(&test_file, result.as_bytes()).unwrap();\n        panic!(\"Test failed, output saved to {}\", test_file.display());\n    }\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    jmap::{Account, JMAPTest, wait_for_index},\n    store::{deflate_test_resource, query::FIELDS},\n};\nuse ::email::{cache::MessageCacheFetch, mailbox::Mailbox};\nuse ahash::AHashSet;\nuse common::{Server, storage::index::ObjectIndexBuilder};\nuse jmap_client::{\n    client::Client,\n    core::query::{Comparator, Filter},\n    email,\n};\nuse mail_builder::{\n    MessageBuilder,\n    headers::{date::Date, message_id::MessageId, text::Text},\n};\nuse mail_parser::HeaderName;\nuse std::{collections::hash_map::Entry, str::FromStr, time::Instant};\nuse store::{\n    ahash::AHashMap,\n    write::{BatchBuilder, now},\n};\nuse types::{collection::Collection, id::Id, special_use::SpecialUse};\n\nconst MAX_THREADS: usize = 100;\nconst MAX_MESSAGES: usize = 1000;\nconst MAX_MESSAGES_PER_THREAD: usize = 100;\n\npub async fn test(params: &mut JMAPTest, insert: bool) {\n    println!(\"Running Email Query tests...\");\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    if insert {\n        // Add some \"virtual\" mailbox ids so create doesn't fail\n        let mut batch = BatchBuilder::new();\n        let account_id = Id::from_str(client.default_account_id())\n            .unwrap()\n            .document_id();\n        batch\n            .with_account_id(account_id)\n            .with_collection(Collection::Mailbox);\n        for mailbox_id in 1545..3010 {\n            batch\n                .with_document(mailbox_id)\n                .custom(ObjectIndexBuilder::<(), _>::new().with_changes(Mailbox {\n                    name: format!(\"Mailbox {mailbox_id}\"),\n                    role: SpecialUse::None,\n                    parent_id: 0,\n                    sort_order: None,\n                    uid_validity: 0,\n                    subscribers: vec![],\n                    acls: vec![],\n                }))\n                .unwrap();\n        }\n        server\n            .core\n            .storage\n            .data\n            .write(batch.build_all())\n            .await\n            .unwrap();\n\n        // Create test messages\n        println!(\"Inserting JMAP Mail query test messages...\");\n        create(&server, account).await;\n\n        assert_eq!(\n            params\n                .server\n                .get_cached_messages(account_id)\n                .await\n                .unwrap()\n                .emails\n                .items\n                .iter()\n                .map(|m| m.thread_id)\n                .collect::<AHashSet<_>>()\n                .len(),\n            MAX_THREADS\n        );\n\n        // Wait for indexing to complete\n        wait_for_index(&server).await;\n    }\n\n    let can_stem = !params.server.search_store().is_mysql();\n\n    println!(\"Running JMAP Mail query tests...\");\n    query(client, can_stem).await;\n\n    println!(\"Running JMAP Mail query options tests...\");\n    query_options(client).await;\n\n    println!(\"Deleting all messages...\");\n    let mut request = client.build();\n    let result_ref = request.query_email().result_reference();\n    request.set_email().destroy_ref(result_ref);\n    let response = request.send().await.unwrap();\n    response\n        .unwrap_method_responses()\n        .pop()\n        .unwrap()\n        .unwrap_set_email()\n        .unwrap();\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\npub async fn query(client: &Client, can_stem: bool) {\n    for (filter, sort, expected_results) in [\n        (\n            Filter::and(vec![\n                (email::query::Filter::after(1850)),\n                (email::query::Filter::from(\"george\")),\n            ]),\n            vec![\n                email::query::Comparator::subject(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"N01389\", \"T10115\", \"N00618\", \"N03500\", \"T01587\", \"T00397\", \"N01561\", \"N05250\",\n                \"N03973\", \"N04973\", \"N04057\", \"N01940\", \"N01539\", \"N01612\", \"N04484\", \"N01954\",\n                \"N05998\", \"T02053\", \"AR00171\", \"AR00172\", \"AR00176\",\n            ],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::in_mailbox(Id::new(1768u64).to_string())),\n                (email::query::Filter::cc(\"canvas\")),\n            ]),\n            vec![\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\"T01882\", \"N04689\", \"T00925\", \"N00121\"],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::text(if can_stem { \"study\" } else { \"studies\" })),\n                (email::query::Filter::in_mailbox_other_than(vec![\n                    Id::new(1991).to_string(),\n                    Id::new(1870).to_string(),\n                    Id::new(2011).to_string(),\n                    Id::new(1951).to_string(),\n                    Id::new(1902).to_string(),\n                    Id::new(1808).to_string(),\n                    Id::new(1963).to_string(),\n                ])),\n            ]),\n            vec![\n                email::query::Comparator::subject(),\n                email::query::Comparator::sent_at(),\n            ],\n            if can_stem {\n                vec![\n                    \"T10330\", \"N01744\", \"N01743\", \"N04885\", \"N02688\", \"N02122\", \"A00059\", \"A00058\",\n                    \"N02123\", \"T00651\", \"T09439\", \"N05001\", \"T05848\", \"T05508\",\n                ]\n            } else {\n                vec![\"T10330\", \"N02122\", \"N02123\", \"T09439\"]\n            },\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::has_keyword(\"N0\")).into(),\n                Filter::not(vec![(email::query::Filter::from(\"collins\"))]),\n                (email::query::Filter::body(\"bequeathed\")).into(),\n            ]),\n            vec![\n                email::query::Comparator::subject(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"N02640\", \"A01020\", \"N01250\", \"T03430\", \"N01800\", \"N00620\", \"N05250\", \"N04630\",\n                \"A01040\",\n            ],\n        ),\n        (\n            email::query::Filter::not_keyword(\"artist\").into(),\n            vec![\n                email::query::Comparator::subject(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\"T08626\", \"T09334\", \"T09455\", \"N01737\", \"T10965\"],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::after(1970)),\n                (email::query::Filter::before(1972)),\n                (email::query::Filter::text(\"colour\")),\n            ]),\n            vec![\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\"T01745\", \"P01436\", \"P01437\"],\n        ),\n        (\n            Filter::and(vec![(email::query::Filter::text(\"'cats and dogs'\"))]),\n            vec![email::query::Comparator::from()],\n            vec![\"P77623\"],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::header(\n                    HeaderName::Comments.to_string(),\n                    Some(\"attributed\"),\n                )),\n                (email::query::Filter::from(\"john\")),\n                (email::query::Filter::cc(\"oil\")),\n            ]),\n            vec![email::query::Comparator::from()],\n            vec![\"T10965\"],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::all_in_thread_have_keyword(\"N\")),\n                (email::query::Filter::before(1800)),\n            ]),\n            vec![\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"N01496\", \"N05916\", \"N01046\", \"N00675\", \"N01320\", \"N01321\", \"N00273\", \"N01453\",\n                \"N02984\",\n            ],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::none_in_thread_have_keyword(\"N\")),\n                (email::query::Filter::after(1995)),\n            ]),\n            vec![\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"AR00163\", \"AR00164\", \"AR00472\", \"P11481\", \"AR00066\", \"AR00178\", \"P77895\",\n                \"P77896\", \"P77897\",\n            ],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::some_in_thread_have_keyword(\"Bronze\")),\n                (email::query::Filter::before(1878)),\n            ]),\n            vec![\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"N04326\", \"N01610\", \"N02920\", \"N01587\", \"T00167\", \"T00168\", \"N01554\", \"N01535\",\n                \"N01536\", \"N01622\", \"N01754\", \"N01594\",\n            ],\n        ),\n        // Sorting tests\n        (\n            email::query::Filter::before(1800).into(),\n            vec![\n                email::query::Comparator::all_in_thread_have_keyword(\"N\"),\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"N01496\", \"N05916\", \"N01046\", \"N00675\", \"N01320\", \"N01321\", \"N00273\", \"N01453\",\n                \"N02984\", \"T09417\", \"T01882\", \"T08820\", \"N04689\", \"T08891\", \"T00986\", \"N00316\",\n                \"N03544\", \"N04296\", \"N04297\", \"T08234\", \"N00112\", \"T00211\", \"N01497\", \"N02639\",\n                \"N02640\", \"T00925\", \"T11683\", \"T08269\", \"D00001\", \"D00002\", \"D00046\", \"N00121\",\n                \"N00126\", \"T08626\",\n            ],\n        ),\n        (\n            email::query::Filter::before(1800).into(),\n            vec![\n                email::query::Comparator::all_in_thread_have_keyword(\"N\").descending(),\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"T09417\", \"T01882\", \"T08820\", \"N04689\", \"T08891\", \"T00986\", \"N00316\", \"N03544\",\n                \"N04296\", \"N04297\", \"T08234\", \"N00112\", \"T00211\", \"N01497\", \"N02639\", \"N02640\",\n                \"T00925\", \"T11683\", \"T08269\", \"D00001\", \"D00002\", \"D00046\", \"N00121\", \"N00126\",\n                \"T08626\", \"N01496\", \"N05916\", \"N01046\", \"N00675\", \"N01320\", \"N01321\", \"N00273\",\n                \"N01453\", \"N02984\",\n            ],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::after(1875)),\n                (email::query::Filter::before(1878)),\n            ]),\n            vec![\n                email::query::Comparator::some_in_thread_have_keyword(\"Bronze\"),\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"N04326\", \"N01610\", \"N02920\", \"N01587\", \"T00167\", \"T00168\", \"N01554\", \"N01535\",\n                \"N01536\", \"N01622\", \"N01754\", \"N01594\", \"N01559\", \"N02123\", \"N01940\", \"N03594\",\n                \"N01494\", \"N04271\",\n            ],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::after(1875)),\n                (email::query::Filter::before(1878)),\n            ]),\n            vec![\n                email::query::Comparator::some_in_thread_have_keyword(\"Bronze\").descending(),\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"N01559\", \"N02123\", \"N01940\", \"N03594\", \"N01494\", \"N04271\", \"N04326\", \"N01610\",\n                \"N02920\", \"N01587\", \"T00167\", \"T00168\", \"N01554\", \"N01535\", \"N01536\", \"N01622\",\n                \"N01754\", \"N01594\",\n            ],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::after(1786)),\n                (email::query::Filter::before(1840)),\n                (email::query::Filter::has_keyword(\"T\")),\n            ]),\n            vec![\n                email::query::Comparator::has_keyword(\"attributed to\"),\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"T09455\", \"T09334\", \"T10965\", \"T08626\", \"T09417\", \"T08951\", \"T01851\", \"T01852\",\n                \"T08761\", \"T08123\", \"T08756\", \"T10561\", \"T10562\", \"T10563\", \"T00986\", \"T03424\",\n                \"T03427\", \"T08234\", \"T08133\", \"T06866\", \"T08897\", \"T00996\", \"T00997\", \"T01095\",\n                \"T03393\", \"T09456\", \"T00188\", \"T02362\", \"T09065\", \"T09547\", \"T10330\", \"T09187\",\n                \"T03433\", \"T08635\", \"T02366\", \"T03436\", \"T09150\", \"T01861\", \"T09759\", \"T11683\",\n                \"T02368\", \"T02369\", \"T08269\", \"T01018\", \"T10066\", \"T01710\", \"T01711\", \"T05764\",\n            ],\n        ),\n        (\n            Filter::and(vec![\n                (email::query::Filter::after(1786)),\n                (email::query::Filter::before(1840)),\n                (email::query::Filter::has_keyword(\"T\")),\n            ]),\n            vec![\n                email::query::Comparator::has_keyword(\"attributed to\").descending(),\n                email::query::Comparator::from(),\n                email::query::Comparator::sent_at(),\n            ],\n            vec![\n                \"T09417\", \"T08951\", \"T01851\", \"T01852\", \"T08761\", \"T08123\", \"T08756\", \"T10561\",\n                \"T10562\", \"T10563\", \"T00986\", \"T03424\", \"T03427\", \"T08234\", \"T08133\", \"T06866\",\n                \"T08897\", \"T00996\", \"T00997\", \"T01095\", \"T03393\", \"T09456\", \"T00188\", \"T02362\",\n                \"T09065\", \"T09547\", \"T10330\", \"T09187\", \"T03433\", \"T08635\", \"T02366\", \"T03436\",\n                \"T09150\", \"T01861\", \"T09759\", \"T11683\", \"T02368\", \"T02369\", \"T08269\", \"T01018\",\n                \"T10066\", \"T01710\", \"T01711\", \"T05764\", \"T09455\", \"T09334\", \"T10965\", \"T08626\",\n            ],\n        ),\n    ] {\n        let mut request = client.build();\n        let query_request = request\n            .query_email()\n            .filter(filter.clone())\n            .sort(sort.clone())\n            .calculate_total(true);\n        query_request.arguments().collapse_threads(false);\n        let query_result_ref = query_request.result_reference();\n        request\n            .get_email()\n            .ids_ref(query_result_ref)\n            .properties([email::Property::MessageId]);\n        let results = request\n            .send()\n            .await\n            .unwrap_or_else(|_| panic!(\"invalid response for {filter:?}\"))\n            .unwrap_method_responses()\n            .pop()\n            .unwrap_or_else(|| panic!(\"invalid response for {filter:?}\"))\n            .unwrap_get_email()\n            .unwrap_or_else(|_| panic!(\"invalid response for {filter:?}\"))\n            .take_list()\n            .into_iter()\n            .map(|e| e.message_id().unwrap().first().unwrap().to_string())\n            .collect::<Vec<_>>();\n\n        let mut missing = Vec::new();\n        let mut extra = Vec::new();\n        for &expected in &expected_results {\n            if !results.iter().any(|r| r.as_str() == expected) {\n                missing.push(expected);\n            }\n        }\n        for result in &results {\n            if !expected_results.contains(&result.as_str()) {\n                extra.push(result.as_str());\n            }\n        }\n\n        assert_eq!(\n            results, expected_results,\n            \"failed test!\\nfilter: {filter:?}\\nsort: {sort:?}\\nmissing: {missing:?}\\nextra: {extra:?}\"\n        );\n    }\n}\n\npub async fn query_options(client: &Client) {\n    for (query, expected_results, expected_results_collapsed) in [\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 0,\n                anchor: None,\n                anchor_offset: 0,\n                limit: 10,\n            },\n            vec![\n                \"N01496\", \"N01320\", \"N01321\", \"N05916\", \"N00273\", \"N01453\", \"N02984\", \"T08820\",\n                \"N00112\", \"T00211\",\n            ],\n            vec![\n                \"N01496\", \"N01320\", \"N05916\", \"N01453\", \"T08820\", \"N01046\", \"N00675\", \"T08891\",\n                \"T01882\", \"N04296\",\n            ],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 10,\n                anchor: None,\n                anchor_offset: 0,\n                limit: 10,\n            },\n            vec![\n                \"N01046\", \"N00675\", \"T08891\", \"N00126\", \"T01882\", \"N04689\", \"T00925\", \"N00121\",\n                \"N04296\", \"N04297\",\n            ],\n            vec![\n                \"T08234\", \"T09417\", \"N01110\", \"T08123\", \"N01039\", \"T09456\", \"T08951\", \"N01273\",\n                \"N00373\", \"T09547\",\n            ],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: -10,\n                anchor: None,\n                anchor_offset: 0,\n                limit: 0,\n            },\n            vec![\n                \"T07236\", \"P11481\", \"AR00066\", \"P77895\", \"P77896\", \"P77897\", \"AR00163\", \"AR00164\",\n                \"AR00472\", \"AR00178\",\n            ],\n            vec![\n                \"P07639\", \"P07522\", \"AR00089\", \"P02949\", \"T05820\", \"P11441\", \"T06971\", \"P11481\",\n                \"AR00163\", \"AR00164\",\n            ],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: -20,\n                anchor: None,\n                anchor_offset: 0,\n                limit: 10,\n            },\n            vec![\n                \"P20079\", \"AR00024\", \"AR00182\", \"P20048\", \"P20044\", \"P20045\", \"P20046\", \"T06971\",\n                \"AR00177\", \"P77935\",\n            ],\n            vec![\n                \"T00300\", \"P06033\", \"T02310\", \"T02135\", \"P04006\", \"P03166\", \"P01358\", \"P07133\",\n                \"P03138\", \"T03562\",\n            ],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: -100000,\n                anchor: None,\n                anchor_offset: 0,\n                limit: 1,\n            },\n            vec![\"N01496\"],\n            vec![\"N01496\"],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: -1,\n                anchor: None,\n                anchor_offset: 0,\n                limit: 100000,\n            },\n            vec![\"AR00178\"],\n            vec![\"AR00164\"],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 0,\n                anchor: get_anchor(client, \"N01205\").await,\n                anchor_offset: 0,\n                limit: 10,\n            },\n            vec![\n                \"N01205\", \"N01976\", \"T01139\", \"N01525\", \"T00176\", \"N01405\", \"N02396\", \"N04885\",\n                \"N01526\", \"N02134\",\n            ],\n            vec![\n                \"N01205\", \"N01526\", \"T01455\", \"N01969\", \"N05250\", \"N01781\", \"N00759\", \"A00057\",\n                \"N03527\", \"N01558\",\n            ],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 0,\n                anchor: get_anchor(client, \"N01205\").await,\n                anchor_offset: 10,\n                limit: 10,\n            },\n            vec![\n                \"N01933\", \"N03618\", \"T03904\", \"N02398\", \"N02399\", \"N02688\", \"T01455\", \"N03051\",\n                \"N01500\", \"N03411\",\n            ],\n            vec![\n                \"N01559\", \"N04326\", \"N06017\", \"N01553\", \"N01617\", \"N01528\", \"N01539\", \"T09439\",\n                \"N01593\", \"N03988\",\n            ],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 0,\n                anchor: get_anchor(client, \"N01205\").await,\n                anchor_offset: -10,\n                limit: 10,\n            },\n            vec![\n                \"N05779\", \"N04652\", \"N01534\", \"A00845\", \"N03409\", \"N03410\", \"N02061\", \"N02426\",\n                \"N00662\", \"N01205\",\n            ],\n            vec![\n                \"N00443\", \"N02237\", \"T03025\", \"N01722\", \"N01356\", \"N01800\", \"T05475\", \"T01587\",\n                \"N05779\", \"N01205\",\n            ],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 0,\n                anchor: get_anchor(client, \"N01496\").await,\n                anchor_offset: -10,\n                limit: 10,\n            },\n            vec![\"N01496\"],\n            vec![\"N01496\"],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 0,\n                anchor: get_anchor(client, \"AR00164\").await,\n                anchor_offset: 10,\n                limit: 10,\n            },\n            vec![],\n            vec![],\n        ),\n        (\n            EmailQuery {\n                filter: None,\n                sort: vec![\n                    email::query::Comparator::subject(),\n                    email::query::Comparator::from(),\n                    email::query::Comparator::sent_at(),\n                ],\n                position: 0,\n                anchor: get_anchor(client, \"AR00164\").await,\n                anchor_offset: 0,\n                limit: 0,\n            },\n            vec![\"AR00164\", \"AR00472\", \"AR00178\"],\n            vec![\"AR00164\"],\n        ),\n    ] {\n        for (test_num, expected_results) in [expected_results, expected_results_collapsed]\n            .into_iter()\n            .enumerate()\n        {\n            let mut request = client.build();\n            let query_request = request\n                .query_email()\n                .sort(query.sort.clone())\n                .position(query.position)\n                .calculate_total(true);\n            if query.limit > 0 {\n                query_request.limit(query.limit);\n            }\n            if let Some(filter) = query.filter.as_ref() {\n                query_request.filter(filter.clone());\n            }\n            if let Some(anchor) = query.anchor.as_ref() {\n                query_request.anchor(anchor);\n                query_request.anchor_offset(query.anchor_offset);\n            }\n            query_request.arguments().collapse_threads(test_num == 1);\n\n            if !expected_results.is_empty() {\n                let query_result_ref = query_request.result_reference();\n                request\n                    .get_email()\n                    .ids_ref(query_result_ref)\n                    .properties([email::Property::MessageId]);\n\n                assert_eq!(\n                    request\n                        .send()\n                        .await\n                        .unwrap()\n                        .unwrap_method_responses()\n                        .pop()\n                        .unwrap()\n                        .unwrap_get_email()\n                        .unwrap()\n                        .take_list()\n                        .into_iter()\n                        .map(|e| e.message_id().unwrap().first().unwrap().to_string())\n                        .collect::<Vec<_>>(),\n                    expected_results,\n                    \"{:#?} ({})\",\n                    query,\n                    test_num == 1\n                );\n            } else {\n                assert_eq!(\n                    request.send_query_email().await.unwrap().ids(),\n                    Vec::<&str>::new()\n                );\n            }\n        }\n    }\n}\n\npub async fn create(server: &Server, account: &Account) {\n    let sent_at = now();\n    let now = Instant::now();\n    let mut fields = AHashMap::default();\n    for (field_num, field) in FIELDS.iter().enumerate() {\n        fields.insert(field.to_string(), field_num);\n    }\n\n    let mut total_messages = 0;\n    let mut total_threads = 0;\n    let mut thread_count = AHashMap::default();\n    let mut artist_count = AHashMap::default();\n\n    let mut messages = Vec::new();\n    let mut chunks = Vec::new();\n\n    'outer: for (idx, record) in csv::ReaderBuilder::new()\n        .has_headers(true)\n        .from_reader(&deflate_test_resource(\"artwork_data.csv.gz\")[..])\n        .records()\n        .enumerate()\n    {\n        let record = record.unwrap();\n        let mut values_str = AHashMap::default();\n        let mut values_int = AHashMap::default();\n\n        for field_name in [\n            \"year\",\n            \"acquisitionYear\",\n            \"accession_number\",\n            \"artist\",\n            \"artistRole\",\n            \"medium\",\n            \"title\",\n            \"creditLine\",\n            \"inscription\",\n        ] {\n            let field = record.get(fields[field_name]).unwrap();\n            if field.is_empty()\n                || (field_name == \"title\" && (field.contains('[') || field.contains(']')))\n            {\n                continue 'outer;\n            } else if field_name == \"year\" || field_name == \"acquisitionYear\" {\n                let field = field.parse::<i32>().unwrap_or(0);\n                if field < 1000 {\n                    continue 'outer;\n                }\n                values_int.insert(field_name.to_string(), field);\n            } else {\n                values_str.insert(field_name.to_string(), field.to_string());\n            }\n        }\n\n        let val = artist_count\n            .entry(values_str[\"artist\"].clone())\n            .or_insert(0);\n        if *val == 3 {\n            continue;\n        }\n        *val += 1;\n\n        match thread_count.entry(values_int[\"year\"]) {\n            Entry::Occupied(mut e) => {\n                let messages_per_thread = e.get_mut();\n                if *messages_per_thread == MAX_MESSAGES_PER_THREAD {\n                    continue;\n                }\n                *messages_per_thread += 1;\n            }\n            Entry::Vacant(e) => {\n                if total_threads == MAX_THREADS {\n                    continue;\n                }\n                total_threads += 1;\n                e.insert(1);\n            }\n        }\n\n        total_messages += 1;\n\n        let mut keywords = Vec::new();\n        for keyword in [\n            values_str[\"medium\"].to_string(),\n            values_str[\"artistRole\"].to_string(),\n            values_str[\"accession_number\"][0..1].to_string(),\n            format!(\n                \"N{}\",\n                &values_str[\"accession_number\"][values_str[\"accession_number\"].len() - 1..]\n            ),\n        ] {\n            if keyword == \"attributed to\"\n                || keyword == \"T\"\n                || keyword == \"N0\"\n                || keyword == \"N\"\n                || keyword == \"artist\"\n                || keyword == \"Bronze\"\n            {\n                keywords.push(keyword);\n            }\n        }\n\n        let message = MessageBuilder::new()\n            .from((values_str[\"artist\"].as_str(), \"artist@domain.com\"))\n            .cc((values_str[\"medium\"].as_str(), \"cc@domain.com\"))\n            .subject(format!(\"Year {}\", values_int[\"year\"]))\n            .date(Date::new(sent_at as i64 + idx as i64))\n            .message_id(values_str[\"accession_number\"].as_str())\n            .header(\"References\", MessageId::new(values_int[\"year\"].to_string()))\n            .header(\"Comments\", Text::new(values_str[\"artistRole\"].as_str()))\n            .text_body(format!(\n                \"{}\\n{}\\n\",\n                values_str[\"creditLine\"], values_str[\"inscription\"]\n            ))\n            .attachment(\"text/plain\", \"details.txt\", values_str[\"title\"].as_bytes())\n            .write_to_vec()\n            .unwrap();\n\n        messages.push((\n            message,\n            [\n                Id::new(values_int[\"year\"] as u64).to_string(),\n                Id::new((values_int[\"acquisitionYear\"] + 1000) as u64).to_string(),\n            ],\n            keywords,\n            values_int[\"year\"] as i64,\n        ));\n\n        if messages.len() == 100 {\n            chunks.push(messages);\n            messages = Vec::new();\n        }\n\n        if total_messages == MAX_MESSAGES {\n            break;\n        }\n    }\n\n    if !messages.is_empty() {\n        chunks.push(messages);\n    }\n\n    let mut tasks = Vec::new();\n    for chunk in chunks {\n        let client = account.client_owned().await;\n        tasks.push(tokio::spawn(async move {\n            for (raw_message, mailbox_ids, keywords, sent_at) in chunk {\n                client\n                    .email_import(raw_message, mailbox_ids, keywords.into(), Some(sent_at))\n                    .await\n                    .unwrap();\n            }\n        }));\n    }\n\n    for task in tasks {\n        task.await.unwrap();\n    }\n\n    wait_for_index(server).await;\n\n    println!(\n        \"Imported {} messages in {} ms (single thread).\",\n        total_messages,\n        now.elapsed().as_millis()\n    );\n}\n\nasync fn get_anchor(client: &Client, anchor: &str) -> Option<String> {\n    client\n        .email_query(\n            email::query::Filter::header(\"Message-Id\", anchor.into()).into(),\n            None::<Vec<_>>,\n        )\n        .await\n        .unwrap()\n        .take_ids()\n        .pop()\n        .unwrap()\n        .into()\n}\n\n#[derive(Debug, Clone)]\npub struct EmailQuery {\n    pub filter: Option<Filter<email::query::Filter>>,\n    pub sort: Vec<Comparator<email::query::Comparator>>,\n    pub position: i32,\n    pub anchor: Option<String>,\n    pub anchor_offset: i32,\n    pub limit: usize,\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/query_changes.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{\n    JMAPTest,\n    mail::changes::{LogAction, ParseState},\n};\nuse ::email::message::metadata::MessageData;\nuse common::storage::index::ObjectIndexBuilder;\nuse jmap_client::{\n    core::query::{Comparator, Filter},\n    email,\n    mailbox::Role,\n};\nuse jmap_proto::types::state::State;\nuse std::str::FromStr;\nuse store::{\n    ValueKey,\n    ahash::{AHashMap, AHashSet},\n    write::{AlignedBytes, Archive, BatchBuilder},\n};\nuse types::{\n    collection::{Collection, SyncCollection},\n    id::Id,\n};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email QueryChanges tests...\");\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    let mailbox1_id = client\n        .mailbox_create(\"JMAP Changes 1\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n    let mailbox2_id = client\n        .mailbox_create(\"JMAP Changes 2\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    let mut states = vec![State::Initial];\n    let mut id_map = AHashMap::default();\n\n    let mut updated_ids = AHashSet::default();\n    let mut removed_ids = AHashSet::default();\n    let mut type1_ids = AHashSet::default();\n    let mut thread_id_map: AHashMap<u32, Id> = AHashMap::default();\n\n    let mut thread_id = 100;\n\n    for (change_num, change) in [\n        LogAction::Insert(0),\n        LogAction::Insert(1),\n        LogAction::Insert(2),\n        LogAction::Move(0, 3),\n        LogAction::Insert(4),\n        LogAction::Insert(5),\n        LogAction::Update(1),\n        LogAction::Update(2),\n        LogAction::Delete(1),\n        LogAction::Insert(6),\n        LogAction::Insert(7),\n        LogAction::Update(2),\n        LogAction::Update(4),\n        LogAction::Update(5),\n        LogAction::Update(6),\n        LogAction::Update(7),\n        LogAction::Delete(4),\n        LogAction::Delete(5),\n        LogAction::Delete(6),\n        LogAction::Insert(8),\n        LogAction::Insert(9),\n        LogAction::Insert(10),\n        LogAction::Update(3),\n        LogAction::Update(2),\n        LogAction::Update(8),\n        LogAction::Move(9, 11),\n        LogAction::Move(10, 12),\n        LogAction::Delete(8),\n    ]\n    .iter()\n    .enumerate()\n    {\n        match &change {\n            LogAction::Insert(id) => {\n                let jmap_id = Id::from_str(\n                    client\n                        .email_import(\n                            format!(\n                                \"From: test_{}\\nSubject: test_{}\\n\\ntest\",\n                                if change_num % 2 == 0 { 1 } else { 2 },\n                                *id\n                            )\n                            .into_bytes(),\n                            [if change_num % 2 == 0 {\n                                &mailbox1_id\n                            } else {\n                                &mailbox2_id\n                            }],\n                            [if change_num % 2 == 0 { \"1\" } else { \"2\" }].into(),\n                            Some(*id as i64),\n                        )\n                        .await\n                        .unwrap()\n                        .id()\n                        .unwrap(),\n                )\n                .unwrap();\n\n                id_map.insert(*id, jmap_id);\n                if change_num % 2 == 0 {\n                    type1_ids.insert(jmap_id);\n                }\n                thread_id_map.entry(jmap_id.prefix_id()).or_insert(jmap_id);\n            }\n            LogAction::Update(id) => {\n                let id = *id_map.get(id).unwrap();\n                let mut batch = BatchBuilder::new();\n                batch\n                    .with_document(id.document_id())\n                    .log_item_update(SyncCollection::Email, id.prefix_id().into());\n                server.store().write(batch.build_all()).await.unwrap();\n                updated_ids.insert(id);\n            }\n            LogAction::Delete(id) => {\n                let id = *id_map.get(id).unwrap();\n                client.email_destroy(&id.to_string()).await.unwrap();\n                removed_ids.insert(id);\n            }\n            LogAction::Move(from, to) => {\n                let id = *id_map.get(from).unwrap();\n                let new_id = Id::from_parts(thread_id, id.document_id());\n\n                //let new_thread_id = store::rand::random::<u32>();\n\n                let old_message_ = server\n                    .store()\n                    .get_value::<Archive<AlignedBytes>>(ValueKey::archive(\n                        account.id().document_id(),\n                        Collection::Email,\n                        id.document_id(),\n                    ))\n                    .await\n                    .unwrap()\n                    .unwrap();\n                let old_message = old_message_.to_unarchived::<MessageData>().unwrap();\n                let mut new_message = old_message.deserialize::<MessageData>().unwrap();\n                new_message.thread_id = thread_id;\n\n                server\n                    .core\n                    .storage\n                    .data\n                    .write(\n                        BatchBuilder::new()\n                            .with_account_id(account.id().document_id())\n                            .with_collection(Collection::Email)\n                            .with_document(id.document_id())\n                            .custom(\n                                ObjectIndexBuilder::new()\n                                    .with_current(old_message)\n                                    .with_changes(new_message),\n                            )\n                            .unwrap()\n                            .build_all(),\n                    )\n                    .await\n                    .unwrap();\n\n                id_map.insert(*to, new_id);\n                if type1_ids.contains(&id) {\n                    type1_ids.insert(new_id);\n                }\n                removed_ids.insert(id);\n                thread_id_map.insert(new_id.prefix_id(), new_id);\n                thread_id += 1;\n            }\n            LogAction::UpdateChild(_) => unreachable!(),\n        }\n\n        let mut new_state = State::Initial;\n        for state in &states {\n            for (test_num, query) in vec![\n                QueryChanges {\n                    filter: None,\n                    sort: vec![email::query::Comparator::received_at()],\n                    since_query_state: state.clone(),\n                    max_changes: 0,\n                    up_to_id: None,\n                    collapse_threads: false,\n                },\n                QueryChanges {\n                    filter: Some(email::query::Filter::from(\"test_1\").into()),\n                    sort: vec![email::query::Comparator::received_at()],\n                    since_query_state: state.clone(),\n                    max_changes: 0,\n                    up_to_id: None,\n                    collapse_threads: false,\n                },\n                QueryChanges {\n                    filter: Some(email::query::Filter::in_mailbox(&mailbox1_id).into()),\n                    sort: vec![email::query::Comparator::received_at()],\n                    since_query_state: state.clone(),\n                    max_changes: 0,\n                    up_to_id: None,\n                    collapse_threads: false,\n                },\n                QueryChanges {\n                    filter: None,\n                    sort: vec![email::query::Comparator::received_at()],\n                    since_query_state: state.clone(),\n                    max_changes: 0,\n                    up_to_id: id_map\n                        .get(&7)\n                        .map(|id| id.to_string().into())\n                        .unwrap_or(None),\n                    collapse_threads: false,\n                },\n                QueryChanges {\n                    filter: None,\n                    sort: vec![email::query::Comparator::received_at()],\n                    since_query_state: state.clone(),\n                    max_changes: 0,\n                    up_to_id: None,\n                    collapse_threads: true,\n                },\n            ]\n            .into_iter()\n            .enumerate()\n            {\n                if (test_num == 3 || test_num == 4) && query.up_to_id.is_none() {\n                    continue;\n                }\n                if test_num == 4 && !query.collapse_threads {\n                    continue;\n                }\n                let mut request = client.build();\n                let query_request = request\n                    .query_email_changes(query.since_query_state.to_string())\n                    .sort(query.sort);\n\n                if let Some(filter) = query.filter {\n                    query_request.filter(filter);\n                }\n\n                if let Some(up_to_id) = query.up_to_id {\n                    query_request.up_to_id(up_to_id);\n                }\n\n                if query.collapse_threads {\n                    query_request.arguments().collapse_threads(true);\n                }\n\n                let changes = request.send_query_email_changes().await.unwrap();\n\n                if test_num == 0 || test_num == 1 {\n                    // Immutable filters should not return modified ids, only deletions.\n                    for id in changes.removed() {\n                        let id = Id::from_str(id).unwrap();\n                        assert!(\n                            removed_ids.contains(&id),\n                            \"{:?} (id: {:?})\",\n                            changes,\n                            id_map.iter().find(|(_, v)| **v == id).map(|(k, _)| k)\n                        );\n                    }\n                }\n                if test_num == 1 || test_num == 2 {\n                    // Only type 1 results should be added to the list.\n                    for item in changes.added() {\n                        let id = Id::from_str(item.id()).unwrap();\n                        assert!(\n                            type1_ids.contains(&id),\n                            \"{:?} (id: {:?})\",\n                            changes,\n                            id_map.iter().find(|(_, v)| **v == id).map(|(k, _)| k)\n                        );\n                    }\n                }\n                if test_num == 3 {\n                    // Only ids up to 7 should be added to the list.\n                    for item in changes.added() {\n                        let item_id = Id::from_str(item.id()).unwrap();\n                        let id = id_map.iter().find(|(_, v)| **v == item_id).unwrap().0;\n                        assert!(id <= &7, \"{:?} (id: {})\", changes, id);\n                    }\n                }\n                if test_num == 4 {\n                    // With collapse_threads, only first email per thread should be added.\n                    let mut seen_threads = AHashSet::new();\n                    for item in changes.added() {\n                        let item_id = Id::from_str(item.id()).unwrap();\n                        let thread_id = item_id.prefix_id();\n                        assert!(\n                            seen_threads.insert(thread_id),\n                            \"Thread {} appears multiple times with collapse_threads: {:?}\",\n                            thread_id,\n                            changes\n                        );\n                        // Verify this is the first email in this thread\n                        assert_eq!(\n                            thread_id_map.get(&thread_id),\n                            Some(&item_id),\n                            \"Expected first email in thread {}, got {:?}\",\n                            thread_id,\n                            item_id\n                        );\n                    }\n                }\n\n                if let State::Initial = state {\n                    new_state = State::parse_str(changes.new_query_state()).unwrap();\n                }\n            }\n        }\n        states.push(new_state);\n    }\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\n#[derive(Debug, Clone)]\npub struct QueryChanges {\n    pub filter: Option<Filter<email::query::Filter>>,\n    pub sort: Vec<Comparator<email::query::Comparator>>,\n    pub since_query_state: State,\n    pub max_changes: usize,\n    pub up_to_id: Option<String>,\n    pub collapse_threads: bool,\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/search_snippet.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, wait_for_index};\nuse email::mailbox::INBOX_ID;\nuse jmap_client::{core::query, email::query::Filter};\nuse std::{fs, path::PathBuf};\nuse store::ahash::AHashMap;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running SearchSnippet tests...\");\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n    let mailbox_id = Id::from(INBOX_ID).to_string();\n\n    let mut email_ids = AHashMap::default();\n\n    let mut test_dir = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    test_dir.push(\"resources\");\n    test_dir.push(\"jmap\");\n    test_dir.push(\"email_snippet\");\n\n    // Import test messages\n    for email_name in [\n        \"html\",\n        \"subpart\",\n        \"mixed\",\n        \"text_plain\",\n        \"text_plain_chinese\",\n    ] {\n        let mut file_name = test_dir.clone();\n        file_name.push(format!(\"{}.eml\", email_name));\n        let email_id = client\n            .email_import(\n                fs::read(&file_name).unwrap(),\n                [&mailbox_id],\n                None::<Vec<&str>>,\n                None,\n            )\n            .await\n            .unwrap()\n            .take_id();\n        email_ids.insert(email_name, email_id);\n    }\n    wait_for_index(&server).await;\n\n    let can_stem = params.server.search_store().internal_fts().is_some();\n\n    // Run tests\n    for (filter, email_name, snippet_subject, snippet_preview) in [\n        (\n            query::Filter::or(vec![\n                query::Filter::or(vec![Filter::subject(\"friend\"), Filter::subject(\"help\")]),\n                query::Filter::or(vec![Filter::body(\"secret\"), Filter::body(\"call\")]),\n            ]),\n            \"text_plain\",\n            Some(\"<mark>Help</mark> a <mark>friend</mark> from Abidjan Côte d'Ivoire\"),\n            Some(concat!(\n                \"d'Ivoire. He <mark>secretly</mark> <mark>called</mark> me on his bedside \",\n                \"and told me that he has a sum of $7.5M (Seven Million five Hundred Thousand\",\n                \" Dollars) left in a suspense account in a local bank here in Abidjan Côte \",\n                \"d'Ivoire, that he used my name a\"\n            )),\n        ),\n        (\n            Filter::text(\"côte\").into(),\n            \"text_plain\",\n            Some(\"Help a friend from Abidjan <mark>Côte</mark> d'Ivoire\"),\n            Some(concat!(\n                \"in Abidjan <mark>Côte</mark> d'Ivoire. He secretly called me on \",\n                \"his bedside and told me that he has a sum of $7.5M (Seven \",\n                \"Million five Hundred Thousand Dollars) left in a suspense \",\n                \"account in a local bank here in Abidjan <mark>Côte</mark> d'Ivoire, that \"\n            )),\n        ),\n        (\n            Filter::text(\"\\\"your country\\\"\").into(),\n            \"text_plain\",\n            None,\n            Some(concat!(\n                \"over to <mark>your</mark> <mark>country</mark> to further my education and \",\n                \"to secure a residential permit for me in <mark>your</mark> <mark>country\",\n                \"</mark>. Moreover, I am willing to offer you 30 percent of the total sum as \",\n                \"compensation for your effort inp\",\n            )),\n        ),\n        (\n            Filter::text(\"overseas\").into(),\n            \"text_plain\",\n            None,\n            Some(\"nominated account <mark>overseas</mark>. \"),\n        ),\n        (\n            Filter::text(\"孫子兵法\").into(),\n            \"text_plain_chinese\",\n            Some(\"<mark>孫</mark><mark>子</mark><mark>兵法</mark>\"),\n            Some(concat!(\n                \"&lt;&quot;<mark>孫</mark><mark>子</mark><mark>兵法</mark>：&quot;&gt; \",\n                \"<mark>孫</mark><mark>子</mark>曰：兵者，國之大事，死生之地，存亡之道，\",\n                \"不可不察也。 <mark>孫</mark><mark>子</mark>曰：凡用兵之法，馳車千駟\"\n            )),\n        ),\n        (\n            Filter::text(\"cia\").into(),\n            \"subpart\",\n            None,\n            Some(\"shouldn't the <mark>CIA</mark> have something like that? Bill\"),\n        ),\n        (\n            Filter::text(\"frösche\").into(),\n            \"html\",\n            Some(\"Die Hasen und die <mark>Frösche</mark>\"),\n            Some(concat!(\n                \"und die <mark>Frösche</mark> Die Hasen klagten einst über ihre mißliche Lage; \",\n                \"&quot;wir leben&quot;, sprach ein Redner, &quot;in steter Furcht vor Menschen und \",\n                \"Tieren, eine Beute der Hunde, der Adler, ja fast aller Raubtiere! \",\n                \"Unsere stete Angst ist är\"\n            )),\n        ),\n        (\n            Filter::text(if can_stem {\n                \"es:galería vasto biblioteca\"\n            } else {\n                \"es:galería vastos biblioteca\"\n            })\n            .into(),\n            \"mixed\",\n            Some(\"<mark>Biblioteca</mark> de Babel\"),\n            Some(concat!(\n                \"llaman la *<mark>Biblioteca</mark>*) se compone de un número indefinido, y tal \",\n                \"vez infinito, de <mark>galerías</mark> hexagonales, con <mark>vastos</mark> \",\n                \"pozos de ventilación en el medio, cercados por barandas bajísimas. Desde \",\n                \"cualquier hexágono se \"\n            )),\n        ),\n    ] {\n        let mut request = client.build();\n        let result_ref = request\n            .query_email()\n            .filter(filter.clone())\n            .result_reference();\n        request\n            .get_search_snippet()\n            .filter(filter)\n            .email_ids_ref(result_ref);\n        let response = request\n            .send()\n            .await\n            .unwrap()\n            .unwrap_method_responses()\n            .pop()\n            .unwrap()\n            .unwrap_get_search_snippet()\n            .unwrap();\n        let snippet = response\n            .snippet(email_ids.get(email_name).unwrap())\n            .unwrap_or_else(|| panic!(\"No snippet for {}\", email_name));\n        assert_eq!(snippet_subject, snippet.subject());\n        assert_eq!(snippet_preview, snippet.preview());\n        assert!(\n            snippet.preview().map_or(0, |p| p.len()) <= 255,\n            \"len: {}\",\n            snippet.preview().map_or(0, |p| p.len())\n        );\n    }\n\n    // Destroy test data\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/set.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, find_values, replace_blob_ids, replace_boundaries, replace_values};\nuse ::email::mailbox::INBOX_ID;\nuse ahash::AHashSet;\nuse jmap_client::{\n    Error, Set,\n    client::Client,\n    core::set::{SetError, SetErrorType},\n    email::{self, Email},\n    mailbox::Role,\n};\nuse std::{fs, path::PathBuf};\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email Set tests...\");\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n    let mailbox_id = Id::from(INBOX_ID).to_string();\n\n    create(client, &mailbox_id).await;\n    update(client, &mailbox_id).await;\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\nasync fn create(client: &Client, mailbox_id: &str) {\n    let mut test_dir = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    test_dir.push(\"resources\");\n    test_dir.push(\"jmap\");\n    test_dir.push(\"email_set\");\n\n    for file_name in fs::read_dir(&test_dir).unwrap() {\n        let mut file_name = file_name.as_ref().unwrap().path();\n        if file_name.extension().is_none_or(|e| e != \"json\") {\n            continue;\n        }\n        println!(\"Creating email from {:?}\", file_name);\n\n        // Upload blobs\n        let mut json_request = String::from_utf8(fs::read(&file_name).unwrap()).unwrap();\n        let blob_values = find_values(&json_request, \"\\\"blobId\\\"\");\n        if !blob_values.is_empty() {\n            let mut blob_ids = Vec::with_capacity(blob_values.len());\n            for blob_value in &blob_values {\n                let blob_value = blob_value.replace(\"\\\\r\", \"\\r\").replace(\"\\\\n\", \"\\n\");\n                blob_ids.push(\n                    client\n                        .upload(None, blob_value.into_bytes(), None)\n                        .await\n                        .unwrap()\n                        .take_blob_id(),\n                );\n            }\n            json_request = replace_values(json_request, &blob_values, &blob_ids);\n        }\n\n        // Create message and obtain its blobId\n        let mut request = client.build();\n        let mut create_item =\n            serde_json::from_slice::<Email<Set>>(json_request.as_bytes()).unwrap();\n        create_item.mailbox_ids([mailbox_id]);\n        let create_id = request.set_email().create_item(create_item);\n        let created_email = request\n            .send_set_email()\n            .await\n            .unwrap()\n            .created(&create_id)\n            .unwrap();\n\n        // Download raw message\n        let raw_message = client\n            .download(created_email.blob_id().unwrap())\n            .await\n            .unwrap();\n\n        // Fetch message\n        let mut request = client.build();\n        request\n            .get_email()\n            .ids([created_email.id().unwrap()])\n            .properties([\n                email::Property::Id,\n                email::Property::BlobId,\n                email::Property::ThreadId,\n                email::Property::MailboxIds,\n                email::Property::Keywords,\n                email::Property::ReceivedAt,\n                email::Property::MessageId,\n                email::Property::InReplyTo,\n                email::Property::References,\n                email::Property::Sender,\n                email::Property::From,\n                email::Property::To,\n                email::Property::Cc,\n                email::Property::Bcc,\n                email::Property::ReplyTo,\n                email::Property::Subject,\n                email::Property::SentAt,\n                email::Property::HasAttachment,\n                email::Property::Preview,\n                email::Property::BodyValues,\n                email::Property::TextBody,\n                email::Property::HtmlBody,\n                email::Property::Attachments,\n                email::Property::BodyStructure,\n            ])\n            .arguments()\n            .body_properties([\n                email::BodyProperty::PartId,\n                email::BodyProperty::BlobId,\n                email::BodyProperty::Size,\n                email::BodyProperty::Name,\n                email::BodyProperty::Type,\n                email::BodyProperty::Charset,\n                email::BodyProperty::Headers,\n                email::BodyProperty::Disposition,\n                email::BodyProperty::Cid,\n                email::BodyProperty::Language,\n                email::BodyProperty::Location,\n            ])\n            .fetch_all_body_values(true)\n            .max_body_value_bytes(100);\n        let email = request\n            .send_get_email()\n            .await\n            .unwrap()\n            .pop()\n            .unwrap()\n            .into_test();\n\n        // Compare raw message\n        file_name.set_extension(\"eml\");\n        let result = replace_boundaries(String::from_utf8(raw_message).unwrap());\n\n        if fs::read(&file_name).unwrap() != result.as_bytes() {\n            file_name.set_extension(\"eml_failed\");\n            fs::write(&file_name, result.as_bytes()).unwrap();\n            panic!(\"Test failed, output saved to {}\", file_name.display());\n        }\n\n        // Compare response\n        file_name.set_extension(\"jmap\");\n        let result = replace_blob_ids(replace_boundaries(\n            serde_json::to_string_pretty(&email).unwrap(),\n        ));\n        if fs::read(&file_name).unwrap() != result.as_bytes() {\n            file_name.set_extension(\"jmap_failed\");\n            fs::write(&file_name, result.as_bytes()).unwrap();\n            panic!(\"Test failed, output saved to {}\", file_name.display());\n        }\n    }\n}\n\nasync fn update(client: &Client, root_mailbox_id: &str) {\n    // Obtain all messageIds previously created\n    let mailbox = client\n        .email_query(\n            email::query::Filter::in_mailbox(root_mailbox_id).into(),\n            None::<Vec<_>>,\n        )\n        .await\n        .unwrap();\n\n    // Create two test mailboxes\n    let test_mailbox1_id = client\n        .mailbox_create(\"Test 1\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n    let test_mailbox2_id = client\n        .mailbox_create(\"Test 2\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    // Set keywords and mailboxes\n    let mut request = client.build();\n    request\n        .set_email()\n        .update(mailbox.id(0))\n        .mailbox_ids([&test_mailbox1_id, &test_mailbox2_id])\n        .keywords([\"test1\", \"test2\"]);\n    request\n        .send_set_email()\n        .await\n        .unwrap()\n        .updated(mailbox.id(0))\n        .unwrap();\n    assert_email_properties(\n        client,\n        mailbox.id(0),\n        &[&test_mailbox1_id, &test_mailbox2_id],\n        &[\"test1\", \"test2\"],\n    )\n    .await;\n\n    // Patch keywords and mailboxes\n    let mut request = client.build();\n    request\n        .set_email()\n        .update(mailbox.id(0))\n        .mailbox_id(&test_mailbox1_id, false)\n        .keyword(\"test1\", true)\n        .keyword(\"test2\", false)\n        .keyword(\"test3\", true);\n    request\n        .send_set_email()\n        .await\n        .unwrap()\n        .updated(mailbox.id(0))\n        .unwrap();\n    assert_email_properties(\n        client,\n        mailbox.id(0),\n        &[&test_mailbox2_id],\n        &[\"test1\", \"test3\"],\n    )\n    .await;\n\n    // Orphan messages should not be permitted\n    let mut request = client.build();\n    request\n        .set_email()\n        .update(mailbox.id(0))\n        .mailbox_id(&test_mailbox2_id, false);\n    assert!(matches!(\n        request\n            .send_set_email()\n            .await\n            .unwrap()\n            .updated(mailbox.id(0)),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n\n    // Updating and destroying the same item should not be allowed\n    let mut request = client.build();\n    let set_email_request = request.set_email();\n    set_email_request\n        .update(mailbox.id(0))\n        .mailbox_id(&test_mailbox2_id, false);\n    set_email_request.destroy([mailbox.id(0)]);\n    assert!(matches!(\n        request\n            .send_set_email()\n            .await\n            .unwrap()\n            .updated(mailbox.id(0)),\n        Err(Error::Set(SetError {\n            type_: SetErrorType::WillDestroy,\n            ..\n        }))\n    ));\n\n    // Delete some messages\n    let mut request = client.build();\n    request.set_email().destroy([mailbox.id(1), mailbox.id(2)]);\n    assert_eq!(\n        request\n            .send_set_email()\n            .await\n            .unwrap()\n            .destroyed_ids()\n            .unwrap()\n            .count(),\n        2\n    );\n    let mut request = client.build();\n    request.get_email().ids([mailbox.id(1), mailbox.id(2)]);\n    assert_eq!(request.send_get_email().await.unwrap().not_found().len(), 2);\n\n    // Destroy test mailboxes\n    client\n        .mailbox_destroy(&test_mailbox1_id, true)\n        .await\n        .unwrap();\n    client\n        .mailbox_destroy(&test_mailbox2_id, true)\n        .await\n        .unwrap();\n}\n\npub async fn assert_email_properties(\n    client: &Client,\n    message_id: &str,\n    mailbox_ids: &[&str],\n    keywords: &[&str],\n) {\n    let result = client\n        .email_get(\n            message_id,\n            [email::Property::MailboxIds, email::Property::Keywords].into(),\n        )\n        .await\n        .unwrap()\n        .unwrap();\n\n    assert_eq!(\n        mailbox_ids.iter().copied().collect::<AHashSet<_>>(),\n        result\n            .mailbox_ids()\n            .iter()\n            .copied()\n            .collect::<AHashSet<_>>()\n    );\n\n    assert_eq!(\n        keywords.iter().copied().collect::<AHashSet<_>>(),\n        result.keywords().iter().copied().collect::<AHashSet<_>>()\n    );\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/sieve_script.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    jmap::{\n        JMAPTest,\n        mail::{\n            delivery::SmtpConnection,\n            submission::{MockMessage, assert_message_delivery, spawn_mock_smtp_server},\n        },\n    },\n    smtp::DnsCache,\n};\nuse jmap_client::{\n    Error,\n    core::set::{SetError, SetErrorType},\n    email, mailbox,\n    sieve::query::{Comparator, Filter},\n};\nuse std::{\n    fs,\n    path::PathBuf,\n    time::{Duration, Instant},\n};\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Sieve tests...\");\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    // Validate scripts\n    client\n        .sieve_script_validate(get_script(\"validate_ok\"))\n        .await\n        .unwrap();\n    assert!(matches!(\n        client\n            .sieve_script_validate(get_script(\"validate_error\"))\n            .await,\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidScript,\n            ..\n        }))\n    ));\n\n    // Create 5 Sieve scripts, all deactivated.\n    let mut script_ids = Vec::new();\n    for i in 0..5 {\n        script_ids.push(\n            client\n                .sieve_script_create(\n                    format!(\"script_{}\", i + 1),\n                    format!(\"require \\\"fileinto\\\"; fileinto \\\"{}\\\";\", i + 1).into_bytes(),\n                    false,\n                )\n                .await\n                .unwrap()\n                .take_id(),\n        );\n    }\n\n    let response = client\n        .sieve_script_query(Filter::is_active(false).into(), [Comparator::name()].into())\n        .await\n        .unwrap();\n    assert_eq!(response.ids().len(), 5);\n    for (pos, id) in response.ids().iter().enumerate() {\n        let script = client\n            .sieve_script_get(id, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .unwrap();\n        assert_eq!(script.name().unwrap(), format!(\"script_{}\", pos + 1));\n        assert_eq!(\n            String::from_utf8(client.download(script.blob_id().unwrap()).await.unwrap()).unwrap(),\n            format!(\"require \\\"fileinto\\\"; fileinto \\\"{}\\\";\", pos + 1)\n        );\n    }\n\n    // Activate last script twice and then the first script\n    for _ in 0..2 {\n        client\n            .sieve_script_activate(script_ids.last().unwrap())\n            .await\n            .unwrap();\n        assert_eq!(\n            client\n                .sieve_script_query(Filter::is_active(true).into(), [Comparator::name()].into())\n                .await\n                .unwrap()\n                .ids(),\n            vec![script_ids.last().unwrap().to_string()]\n        );\n    }\n    client\n        .sieve_script_activate(script_ids.first().unwrap())\n        .await\n        .unwrap();\n    assert_eq!(\n        client\n            .sieve_script_query(Filter::is_active(true).into(), [Comparator::name()].into())\n            .await\n            .unwrap()\n            .ids(),\n        vec![script_ids.first().unwrap().to_string()]\n    );\n\n    // Destroying an active script should not work\n    assert!(matches!(\n        client\n            .sieve_script_destroy(script_ids.first().unwrap())\n            .await,\n        Err(Error::Set(SetError {\n            type_: SetErrorType::ScriptIsActive,\n            ..\n        }))\n    ));\n\n    // Deactivate all scripts\n    client.sieve_script_deactivate().await.unwrap();\n    assert_eq!(\n        client\n            .sieve_script_query(Filter::is_active(true).into(), [Comparator::name()].into())\n            .await\n            .unwrap()\n            .ids(),\n        Vec::<String>::new()\n    );\n\n    // Connect to LMTP service\n    let mut lmtp = SmtpConnection::connect().await;\n\n    // Run mailbox, fileinto, flags tests\n    client\n        .sieve_script_create(\"test_mailbox\", get_script(\"test_mailbox\"), true)\n        .await\n        .unwrap();\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n\n    // Make sure all folders were created\n    let mailbox_names = \"My/Nested/Mailbox/with/multiple/levels/Folder\"\n        .split('/')\n        .collect::<Vec<_>>();\n    let mut mailbox_ids = Vec::new();\n    for &mailbox in &mailbox_names {\n        let mut response = client\n            .mailbox_query(mailbox::query::Filter::name(mailbox).into(), None::<Vec<_>>)\n            .await\n            .unwrap();\n        assert!(\n            !response.ids().is_empty(),\n            \"Mailbox {} was not created.\",\n            mailbox\n        );\n        mailbox_ids.extend(response.take_ids());\n    }\n    assert_eq!(mailbox_ids.len(), mailbox_names.len());\n\n    // Make sure the message was delivered to the right folders\n    let message_ids = client\n        .email_query(None::<email::query::Filter>, None::<Vec<_>>)\n        .await\n        .unwrap()\n        .take_ids();\n    assert_eq!(message_ids.len(), 1, \"too many messages {:?}\", message_ids);\n    let email = client\n        .email_get(\n            message_ids.last().unwrap(),\n            [email::Property::MailboxIds, email::Property::Keywords].into(),\n        )\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(\n        email.keywords().len(),\n        2,\n        \"Expected 2 keywords, found {:?}.\",\n        email.keywords()\n    );\n    for keyword in [\"$important\", \"$seen\"] {\n        if !email.keywords().contains(&keyword) {\n            panic!(\"Keyword {} not found in {:?}.\", keyword, email.keywords());\n        }\n    }\n    assert_eq!(\n        email.mailbox_ids().len(),\n        2,\n        \"Expected 2 mailbox ids, found {:?}.\",\n        email.mailbox_ids()\n    );\n    for mailbox_pos in [mailbox_ids.len() - 1, mailbox_ids.len() - 2] {\n        if !email\n            .mailbox_ids()\n            .contains(&mailbox_ids[mailbox_pos].as_str())\n        {\n            panic!(\n                \"Mailbox {} ({}) not found in {:?}.\",\n                mailbox_names[mailbox_pos],\n                mailbox_ids[mailbox_pos],\n                email.keywords()\n            );\n        }\n    }\n\n    // Run discard and duplicate tests\n    client\n        .sieve_script_create(\n            \"test_discard_reject\",\n            get_script(\"test_discard_reject\"),\n            true,\n        )\n        .await\n        .unwrap();\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"Bcc: Undisclosed recipients;\\r\\n\",\n            \"Message-ID: <1234@example.com>\\r\\n\",\n            \"Subject: Holidays\\r\\n\",\n            \"\\r\\n\",\n            \"Remember to file your TPS reports before \",\n            \"going on holidays.\"\n        ),\n    )\n    .await;\n    assert_eq!(\n        client\n            .email_query(None::<email::query::Filter>, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .ids()\n            .len(),\n        1,\n        \"Discard failed.\"\n    );\n\n    // Let one sec duplicate ids expire\n    tokio::time::sleep(Duration::from_millis(1100)).await;\n\n    // Start mock SMTP server\n    let (mut smtp_rx, smtp_settings) = spawn_mock_smtp_server();\n    server.ipv4_add(\n        \"localhost\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    // Run reject and duplicate check tests\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"Bcc: Undisclosed recipients;\\r\\n\",\n            \"Message-ID: <1234@example.com>\\r\\n\",\n            \"Subject: Holidays\\r\\n\",\n            \"\\r\\n\",\n            \"Remember to file your T.P.S. reports before \",\n            \"going on holidays.\"\n        ),\n    )\n    .await;\n\n    assert_eq!(\n        client\n            .email_query(None::<email::query::Filter>, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .ids()\n            .len(),\n        1,\n        \"Reject failed.\"\n    );\n\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\"<>\", [\"<bill@remote.org>\"], \"@No soup for you\"),\n    )\n    .await;\n\n    // Run include tests\n    client\n        .sieve_script_create(\"test_include_this\", get_script(\"test_include_this\"), false)\n        .await\n        .unwrap();\n    client\n        .sieve_script_create(\"test_include\", get_script(\"test_include\"), true)\n        .await\n        .unwrap();\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"Bcc: Undisclosed recipients;\\r\\n\",\n            \"Message-ID: <1234@example.com>\\r\\n\",\n            \"Subject: Holidays\\r\\n\",\n            \"\\r\\n\",\n            \"Remember to file your T.P.S. reports before \",\n            \"going on holidays.\"\n        ),\n    )\n    .await;\n\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\n            \"<>\",\n            [\"<bill@remote.org>\"],\n            \"@Rejected from an included script\",\n        ),\n    )\n    .await;\n\n    // Run include global tests\n    client\n        .sieve_script_create(\n            \"test_include_global\",\n            get_script(\"test_include_global\"),\n            true,\n        )\n        .await\n        .unwrap();\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"Bcc: Undisclosed recipients;\\r\\n\",\n            \"Message-ID: <1234@example.com>\\r\\n\",\n            \"Subject: Holidays\\r\\n\",\n            \"\\r\\n\",\n            \"Remember to file your T.P.S. reports before \",\n            \"going on holidays.\"\n        ),\n    )\n    .await;\n\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\n            \"<>\",\n            [\"<bill@remote.org>\"],\n            \"@Rejected from a global script\",\n        ),\n    )\n    .await;\n\n    // Run enclose + redirect tests\n    client\n        .sieve_script_create(\n            \"test_redirect_enclose\",\n            get_script(\"test_redirect_enclose\"),\n            true,\n        )\n        .await\n        .unwrap();\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\n            \"<jdoe@example.com>\",\n            [\"<jane@remote.org>\"],\n            \"@Attached you'll find\",\n        ),\n    )\n    .await;\n    assert_eq!(\n        client\n            .email_query(None::<email::query::Filter>, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .ids()\n            .len(),\n        1,\n        \"Redirected message was stored.\"\n    );\n\n    // Run notify + editheader + notify + fcc tests\n    client\n        .sieve_script_create(\"test_notify_fcc\", get_script(\"test_notify_fcc\"), true)\n        .await\n        .unwrap();\n    smtp_settings.lock().do_stop = true;\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: Urgently I need those TPS Reports\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\n            \"<jdoe@example.com>\",\n            [\"<sms_gateway@remote.org>\"],\n            \"@It's TPS-o-clock\",\n        ),\n    )\n    .await;\n\n    let mut request = client.build();\n    request.get_email().properties([\n        email::Property::MailboxIds,\n        email::Property::Keywords,\n        email::Property::Subject,\n    ]);\n    let emails = request.send_get_email().await.unwrap().take_list();\n\n    assert_eq!(\n        emails.len(),\n        3,\n        \"Two new messages were expected: {:#?}.\",\n        emails\n    );\n\n    'outer: for (subject, folder, keywords) in [\n        (\"It's TPS-o-clock\", \"Notifications\", \"\"),\n        (\n            \"Urgently I need those **censored** Reports\",\n            \"Inbox\",\n            \"$seen\",\n        ),\n    ] {\n        for email in &emails {\n            if email.subject().unwrap().eq(subject) {\n                if !keywords.is_empty() && !email.keywords().contains(&keywords) {\n                    panic!(\"Keyword {:?} not found in: {:#?}\", keywords, email);\n                }\n\n                let mailbox_id = client\n                    .mailbox_query(\n                        mailbox::query::Filter::name(folder.to_string()).into(),\n                        None::<Vec<_>>,\n                    )\n                    .await\n                    .unwrap()\n                    .take_ids()\n                    .pop()\n                    .unwrap_or_else(|| panic!(\"Mailbox {:?} not found\", folder));\n\n                if !email.mailbox_ids().contains(&mailbox_id.as_str()) {\n                    panic!(\n                        \"Mailbox {:?} ({}) not found in: {:#?}\",\n                        folder, mailbox_id, email\n                    );\n                }\n\n                continue 'outer;\n            }\n        }\n        panic!(\"Email {:?} not found in: {:#?}\", subject, emails);\n    }\n\n    // Remove test data\n    client.sieve_script_deactivate().await.unwrap();\n    let mut request = client.build();\n    request.query_sieve_script();\n    for id in request.send_query_sieve_script().await.unwrap().take_ids() {\n        client.sieve_script_destroy(&id).await.unwrap();\n    }\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\nfn get_script(name: &str) -> Vec<u8> {\n    let mut script_path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    script_path.push(\"resources\");\n    script_path.push(\"jmap\");\n    script_path.push(\"sieve\");\n    script_path.push(format!(\"{}.sieve\", name));\n    fs::read(script_path).unwrap()\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/submission.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    jmap::{JMAPTest, mail::set::assert_email_properties},\n    smtp::DnsCache,\n};\nuse ahash::AHashMap;\nuse jmap_client::{\n    Error,\n    core::set::{SetError, SetErrorType, SetObject},\n    email_submission::{Address, Delivered, DeliveryStatus, Displayed, UndoStatus, query::Filter},\n    mailbox::Role,\n};\nuse mail_parser::DateTime;\nuse std::{\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse store::parking_lot::Mutex;\nuse tokio::{\n    io::{AsyncBufReadExt, AsyncWriteExt, BufReader},\n    net::TcpListener,\n    sync::mpsc,\n};\nuse types::id::Id;\n\n#[derive(Default, Debug, PartialEq, Eq)]\npub struct MockMessage {\n    pub mail_from: String,\n    pub rcpt_to: Vec<String>,\n    pub message: String,\n}\n\nimpl MockMessage {\n    pub fn new<T, U>(mail_from: T, rcpt_to: U, message: T) -> Self\n    where\n        T: Into<String>,\n        U: IntoIterator<Item = T>,\n    {\n        Self {\n            mail_from: mail_from.into(),\n            rcpt_to: rcpt_to.into_iter().map(|s| s.into()).collect(),\n            message: message.into(),\n        }\n    }\n}\n\n#[derive(Default)]\npub struct MockSMTPSettings {\n    pub fail_mail_from: bool,\n    pub fail_rcpt_to: bool,\n    pub fail_message: bool,\n    pub do_stop: bool,\n}\n\n#[allow(clippy::disallowed_types)]\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running E-mail submissions tests...\");\n    // Start mock SMTP server\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n    let (mut smtp_rx, smtp_settings) = spawn_mock_smtp_server();\n    server.ipv4_add(\n        \"localhost\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + std::time::Duration::from_secs(10),\n    );\n\n    // Test automatic identity creation\n    for (identity_id, email) in [(2u64, \"jdoe@example.com\"), (1u64, \"john.doe@example.com\")] {\n        let identity = client\n            .identity_get(&Id::from(identity_id).to_string(), None)\n            .await\n            .unwrap()\n            .unwrap();\n        assert_eq!(identity.email().unwrap(), email);\n        assert_eq!(identity.name().unwrap(), \"John Doe\");\n    }\n\n    // Create an identity without using a valid address should fail\n    match client\n        .identity_create(\"John Doe\", \"someaddress@domain.com\")\n        .await\n        .unwrap_err()\n    {\n        Error::Set(err) => assert_eq!(err.error(), &SetErrorType::InvalidProperties),\n        err => panic!(\"Unexpected error: {:?}\", err),\n    }\n\n    // Create an identity\n    let identity_id = client\n        .identity_create(\"John Doe (manually created)\", \"jdoe@example.com\")\n        .await\n        .unwrap()\n        .take_id();\n\n    // Create test mailboxes\n    let mailbox_id = client\n        .mailbox_create(\"JMAP EmailSubmission\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n    let mailbox_id_2 = client\n        .mailbox_create(\"JMAP EmailSubmission 2\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    // Import an email without any recipients\n    let email_id = client\n        .email_import(\n            b\"From: jdoe@example.com\\nSubject: hey\\n\\ntest\".to_vec(),\n            [&mailbox_id],\n            None::<Vec<&str>>,\n            None,\n        )\n        .await\n        .unwrap()\n        .take_id();\n\n    // Submission without a valid emailId or identityId should fail\n    assert!(matches!(\n        client\n            .email_submission_create(Id::new(123456).to_string(), &identity_id)\n            .await,\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n    assert!(matches!(\n        client\n            .email_submission_create(&email_id, Id::new(123456).to_string())\n            .await,\n        Err(Error::Set(SetError {\n            type_: SetErrorType::InvalidProperties,\n            ..\n        }))\n    ));\n\n    // Submissions of e-mails without any recipients should fail\n    assert!(matches!(\n        client\n            .email_submission_create(&email_id, &identity_id)\n            .await,\n        Err(Error::Set(SetError {\n            type_: SetErrorType::NoRecipients,\n            ..\n        }))\n    ));\n\n    // Submissions with an envelope that does not match\n    // the identity from address should fail\n    assert!(matches!(\n        client\n            .email_submission_create_envelope(\n                &email_id,\n                &identity_id,\n                \"other_address@example.com\",\n                Vec::<Address>::new(),\n            )\n            .await,\n        Err(Error::Set(SetError {\n            type_: SetErrorType::ForbiddenFrom,\n            ..\n        }))\n    ));\n\n    // Submit a valid message submission\n    let email_body = concat!(\n        \"From: jdoe@example.com\\r\\n\",\n        \"To: jane_smith@remote.org\\r\\n\",\n        \"Bcc: bill@remote.org\\r\\n\",\n        \"Subject: hey\\r\\n\\r\\n\",\n        \"test\"\n    );\n    let email_id = client\n        .email_import(\n            email_body.as_bytes().to_vec(),\n            [&mailbox_id],\n            None::<Vec<&str>>,\n            None,\n        )\n        .await\n        .unwrap()\n        .take_id();\n    client\n        .email_submission_create(&email_id, &identity_id)\n        .await\n        .unwrap();\n\n    // Confirm that the message has been delivered\n    let email_body = email_body.replace(\"Bcc: bill@remote.org\\r\\n\", \"\");\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\n            \"<jdoe@example.com>\",\n            [\"<bill@remote.org>\", \"<jane_smith@remote.org>\"],\n            &email_body,\n        ),\n    )\n    .await;\n\n    // Manually add recipients to the envelope and confirm submission\n    let email_submission_id = client\n        .email_submission_create_envelope(\n            &email_id,\n            &identity_id,\n            \"jdoe@example.com\",\n            [\n                \"tim@foobar.com\", // Should be de-duplicated\n                \"tim@foobar.com\",\n                \"tim@foobar.com  \",\n                \" james@other_domain.com \", // Should be sanitized\n                \"  secret_rcpt@test.com  \",\n            ],\n        )\n        .await\n        .unwrap()\n        .take_id();\n\n    for _ in 0..3 {\n        let mut message = expect_message_delivery(&mut smtp_rx).await;\n\n        assert_eq!(message.mail_from, \"<jdoe@example.com>\");\n        let rcpt_to = message.rcpt_to.pop().unwrap();\n        assert!(\n            [\n                \"<james@other_domain.com>\",\n                \"<secret_rcpt@test.com>\",\n                \"<tim@foobar.com>\",\n            ]\n            .contains(&rcpt_to.as_str())\n        );\n\n        assert!(\n            message.message.contains(&email_body),\n            \"Got [{}], Expected[{}]\",\n            message.message,\n            email_body\n        );\n    }\n\n    // Confirm that the email submission status was updated\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    let email_submission = client\n        .email_submission_get(&email_submission_id, None)\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(email_submission.undo_status().unwrap(), &UndoStatus::Final);\n    assert_eq!(\n        email_submission.delivery_status().unwrap(),\n        &AHashMap::from_iter([\n            (\n                \"tim@foobar.com\".to_string(),\n                DeliveryStatus::new(\"250 2.1.5 Queued\", Delivered::Unknown, Displayed::Unknown)\n            ),\n            (\n                \"secret_rcpt@test.com\".to_string(),\n                DeliveryStatus::new(\"250 2.1.5 Queued\", Delivered::Unknown, Displayed::Unknown)\n            ),\n            (\n                \"james@other_domain.com\".to_string(),\n                DeliveryStatus::new(\"250 2.1.5 Queued\", Delivered::Unknown, Displayed::Unknown)\n            ),\n        ])\n    );\n\n    // SMTP rejects some of the recipients\n    let email_submission_id = client\n        .email_submission_create_envelope(\n            &email_id,\n            &identity_id,\n            \"jdoe@example.com\",\n            [\n                \"nonexistant@example.com\",\n                \"delay@other_domain.com\",\n                \"fail@test.com\",\n                \"tim@foobar.com\",\n            ],\n        )\n        .await\n        .unwrap()\n        .take_id();\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\"<jdoe@example.com>\", [\"<tim@foobar.com>\"], &email_body),\n    )\n    .await;\n    expect_nothing(&mut smtp_rx).await;\n\n    // Verify SMTP replies\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    let email_submission = client\n        .email_submission_get(&email_submission_id, None)\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(\n        email_submission.undo_status().unwrap(),\n        &UndoStatus::Pending\n    );\n    assert_eq!(\n        email_submission.delivery_status().unwrap(),\n        &AHashMap::from_iter([\n            (\n                \"nonexistant@example.com\".to_string(),\n                DeliveryStatus::new(\n                    \"550 5.1.2 Mailbox does not exist.\",\n                    Delivered::No,\n                    Displayed::Unknown\n                )\n            ),\n            (\n                \"delay@other_domain.com\".to_string(),\n                DeliveryStatus::new(\n                    \"Code: 451, Enhanced code: 4.5.3, Message: Try again later.\",\n                    Delivered::Queued,\n                    Displayed::Unknown\n                )\n            ),\n            (\n                \"fail@test.com\".to_string(),\n                DeliveryStatus::new(\n                    \"Code: 550, Enhanced code: 0.0.0, Message: I refuse to accept that recipient.\",\n                    Delivered::No,\n                    Displayed::Unknown\n                )\n            ),\n            (\n                \"tim@foobar.com\".to_string(),\n                DeliveryStatus::new(\n                    \"Code: 250, Enhanced code: 0.0.0, Message: OK\",\n                    Delivered::Yes,\n                    Displayed::Unknown\n                )\n            ),\n        ])\n    );\n\n    // Cancel submission\n    client\n        .email_submission_change_status(&email_submission_id, UndoStatus::Canceled)\n        .await\n        .unwrap();\n    let email_submission = client\n        .email_submission_get(&email_submission_id, None)\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(\n        email_submission.undo_status().unwrap(),\n        &UndoStatus::Canceled\n    );\n    assert_eq!(\n        email_submission.delivery_status().unwrap(),\n        &AHashMap::from_iter([\n            (\n                \"nonexistant@example.com\".to_string(),\n                DeliveryStatus::new(\n                    \"550 5.1.2 Mailbox does not exist.\",\n                    Delivered::No,\n                    Displayed::Unknown\n                )\n            ),\n            (\n                \"delay@other_domain.com\".to_string(),\n                DeliveryStatus::new(\"250 2.1.5 Queued\", Delivered::Unknown, Displayed::Unknown)\n            ),\n            (\n                \"fail@test.com\".to_string(),\n                DeliveryStatus::new(\"250 2.1.5 Queued\", Delivered::Unknown, Displayed::Unknown)\n            ),\n            (\n                \"tim@foobar.com\".to_string(),\n                DeliveryStatus::new(\"250 2.1.5 Queued\", Delivered::Unknown, Displayed::Unknown)\n            ),\n        ])\n    );\n\n    // Confirm that the sendAt property is updated when using FUTURERELEASE\n    let hold_until = DateTime::parse_rfc3339(\"2079-11-20T05:00:00Z\")\n        .unwrap()\n        .to_timestamp();\n    let email_submission_id = client\n        .email_submission_create_envelope(\n            &email_id,\n            &identity_id,\n            Address::new(\"jdoe@example.com\").parameter(\"HOLDUNTIL\", Some(hold_until.to_string())),\n            [\"jane_smith@remote.org\"],\n        )\n        .await\n        .unwrap()\n        .take_id();\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    let email_submission = client\n        .email_submission_get(&email_submission_id, None)\n        .await\n        .unwrap()\n        .unwrap();\n    assert_eq!(email_submission.send_at().unwrap(), hold_until);\n    assert_eq!(\n        email_submission.undo_status().unwrap(),\n        &UndoStatus::Pending\n    );\n    assert_eq!(\n        email_submission.delivery_status().unwrap(),\n        &AHashMap::from_iter([(\n            \"jane_smith@remote.org\".to_string(),\n            DeliveryStatus::new(\"250 2.1.5 Queued\", Delivered::Queued, Displayed::Unknown)\n        ),])\n    );\n\n    // Verify onSuccessUpdateEmail action\n    let mut request = client.build();\n    let set_request = request.set_email_submission();\n    let create_id = set_request\n        .create()\n        .email_id(&email_id)\n        .identity_id(&identity_id)\n        .create_id()\n        .unwrap();\n    set_request\n        .arguments()\n        .on_success_update_email(&create_id)\n        .keyword(\"$draft\", true)\n        .mailbox_id(&mailbox_id, false)\n        .mailbox_id(&mailbox_id_2, true);\n    request.send().await.unwrap().unwrap_method_responses();\n\n    assert_email_properties(client, &email_id, &[&mailbox_id_2], &[\"$draft\"]).await;\n\n    // Verify onSuccessDestroyEmail action\n    let mut request = client.build();\n    let set_request = request.set_email_submission();\n    let create_id = set_request\n        .create()\n        .email_id(&email_id)\n        .identity_id(&identity_id)\n        .create_id()\n        .unwrap();\n    set_request.arguments().on_success_destroy_email(&create_id);\n    request.send().await.unwrap().unwrap_method_responses();\n\n    assert!(\n        client\n            .email_get(&email_id, None::<Vec<_>>)\n            .await\n            .unwrap()\n            .is_none()\n    );\n    smtp_settings.lock().do_stop = true;\n\n    // Destroy the created mailbox, identity and all submissions\n    for identity_id in [\n        identity_id,\n        Id::from(1u64).to_string(),\n        Id::from(2u64).to_string(),\n    ] {\n        client.identity_destroy(&identity_id).await.unwrap();\n    }\n    for id in client\n        .email_submission_query(None::<Filter>, None::<Vec<_>>)\n        .await\n        .unwrap()\n        .take_ids()\n    {\n        let _ = client\n            .email_submission_change_status(&id, UndoStatus::Canceled)\n            .await;\n        client.email_submission_destroy(&id).await.unwrap();\n    }\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\npub fn spawn_mock_smtp_server() -> (mpsc::Receiver<MockMessage>, Arc<Mutex<MockSMTPSettings>>) {\n    // Create channels\n    let (event_tx, event_rx) = mpsc::channel::<MockMessage>(100);\n    let _settings = Arc::new(Mutex::new(MockSMTPSettings::default()));\n    let settings = _settings.clone();\n\n    // Start mock SMTP server\n    tokio::spawn(async move {\n        let listener = TcpListener::bind(\"127.0.0.1:9999\")\n            .await\n            .unwrap_or_else(|e| {\n                panic!(\"Failed to bind mock SMTP server to 127.0.0.1:9999: {}\", e);\n            });\n\n        while let Ok((mut stream, _)) = listener.accept().await {\n            let (rx, mut tx) = stream.split();\n            let mut rx = BufReader::new(rx);\n            let mut buf = String::with_capacity(128);\n            let mut message = MockMessage::default();\n\n            tx.write_all(b\"220 [127.0.0.1] Clueless host service ready\\r\\n\")\n                .await\n                .unwrap();\n\n            while rx.read_line(&mut buf).await.is_ok() {\n                //print!(\"-> {}\", buf);\n                if buf.starts_with(\"EHLO\") {\n                    tx.write_all(b\"250 Hi there, but I have no extensions to offer :-(\\r\\n\")\n                        .await\n                        .unwrap();\n                } else if buf.starts_with(\"MAIL FROM\") {\n                    if settings.lock().fail_mail_from {\n                        tx.write_all(\"552-I do not\\r\\n552 like that MAIL FROM.\\r\\n\".as_bytes())\n                            .await\n                            .unwrap();\n                    } else {\n                        message.mail_from = buf.split_once(':').unwrap().1.trim().to_string();\n                        tx.write_all(b\"250 OK\\r\\n\").await.unwrap();\n                    }\n                } else if buf.starts_with(\"RCPT TO\") {\n                    if buf.contains(\"fail@\") {\n                        tx.write_all(\n                            \"550-I refuse to\\r\\n550 accept that recipient.\\r\\n\".as_bytes(),\n                        )\n                        .await\n                        .unwrap();\n                    } else if buf.contains(\"delay@\") {\n                        tx.write_all(\"451 4.5.3 Try again later.\\r\\n\".as_bytes())\n                            .await\n                            .unwrap();\n                    } else {\n                        message\n                            .rcpt_to\n                            .push(buf.split(':').nth(1).unwrap().trim().to_string());\n                        tx.write_all(b\"250 OK\\r\\n\").await.unwrap();\n                    }\n                } else if buf.starts_with(\"DATA\") {\n                    if settings.lock().fail_message {\n                        tx.write_all(\n                            \"503-Thank you but I am\\r\\n503 saving myself for dessert.\\r\\n\"\n                                .as_bytes(),\n                        )\n                        .await\n                        .unwrap();\n                    } else if !message.mail_from.is_empty() && !message.rcpt_to.is_empty() {\n                        tx.write_all(b\"354 Start feeding me now some quality content please\\r\\n\")\n                            .await\n                            .unwrap();\n                        buf.clear();\n                        while rx.read_line(&mut buf).await.is_ok() {\n                            if buf.starts_with('.') && buf.len() < 4 {\n                                message.message = message.message.trim().to_string();\n                                break;\n                            } else {\n                                message.message += buf.as_str();\n                                buf.clear();\n                            }\n                        }\n                        tx.write_all(b\"250 Great success!\\r\\n\").await.unwrap();\n                        message.rcpt_to.sort_unstable();\n                        event_tx.send(message).await.unwrap();\n                        message = MockMessage::default();\n                    } else {\n                        tx.write_all(\"554 You forgot to tell me a few things.\\r\\n\".as_bytes())\n                            .await\n                            .unwrap();\n                    }\n                } else if buf.starts_with(\"QUIT\") {\n                    tx.write_all(\"250 Arrivederci!\\r\\n\".as_bytes())\n                        .await\n                        .unwrap();\n                    break;\n                } else if buf.starts_with(\"RSET\") {\n                    tx.write_all(\"250 Your wish is my command.\\r\\n\".as_bytes())\n                        .await\n                        .unwrap();\n                    message = MockMessage::default();\n                } else {\n                    println!(\"Unknown command: {}\", buf.trim());\n                }\n                buf.clear();\n            }\n\n            if settings.lock().do_stop {\n                //println!(\"Mock SMTP server stopped.\");\n                break;\n            }\n        }\n    });\n\n    (event_rx, _settings)\n}\n\npub async fn expect_message_delivery(event_rx: &mut mpsc::Receiver<MockMessage>) -> MockMessage {\n    match tokio::time::timeout(Duration::from_millis(3000), event_rx.recv()).await {\n        Ok(Some(message)) => {\n            //println!(\"Got message [{}]\", message.message);\n\n            message\n        }\n        result => {\n            panic!(\"Timeout waiting for message, got: {:?}\", result);\n        }\n    }\n}\n\npub async fn assert_message_delivery(\n    event_rx: &mut mpsc::Receiver<MockMessage>,\n    expected_message: MockMessage,\n) {\n    let message = expect_message_delivery(event_rx).await;\n\n    assert_eq!(message.mail_from, expected_message.mail_from);\n    assert_eq!(message.rcpt_to, expected_message.rcpt_to);\n\n    if let Some(needle) = expected_message.message.strip_prefix('@') {\n        assert!(\n            message.message.contains(needle),\n            \"[{}] needle = {:?}\",\n            message.message,\n            needle\n        );\n    } else {\n        assert!(\n            message.message.contains(&expected_message.message),\n            \"Got [{}], Expected[{}]\",\n            message.message,\n            expected_message.message\n        );\n    }\n}\n\npub async fn expect_nothing(event_rx: &mut mpsc::Receiver<MockMessage>) {\n    match tokio::time::timeout(Duration::from_millis(500), event_rx.recv()).await {\n        Err(_) => {}\n        message => {\n            panic!(\"Received a message when expecting nothing: {:?}\", message);\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/thread_get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, wait_for_index};\nuse jmap_client::mailbox::Role;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Email Thread tests...\");\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    let mailbox_id = client\n        .mailbox_create(\"JMAP Get\", None::<String>, Role::None)\n        .await\n        .unwrap()\n        .take_id();\n\n    let mut expected_result = vec![\"\".to_string(); 5];\n    let mut thread_id = \"\".to_string();\n\n    for num in [5, 3, 1, 2, 4] {\n        let mut email = client\n            .email_import(\n                format!(\"Subject: test\\nReferences: <1234>\\n\\n{}\", num).into_bytes(),\n                [&mailbox_id],\n                None::<Vec<String>>,\n                Some(10000i64 + num as i64),\n            )\n            .await\n            .unwrap();\n        thread_id = email.thread_id().unwrap().to_string();\n        expected_result[num - 1] = email.take_id();\n    }\n\n    wait_for_index(&params.server).await;\n\n    assert_eq!(\n        client\n            .thread_get(&thread_id)\n            .await\n            .unwrap()\n            .unwrap()\n            .email_ids(),\n        expected_result\n    );\n\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/thread_merge.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    jmap::{JMAPTest, mail::mailbox::destroy_all_mailboxes_no_wait, wait_for_index},\n    store::deflate_test_resource,\n};\nuse ::email::{\n    cache::MessageCacheFetch,\n    mailbox::INBOX_ID,\n    message::ingest::{EmailIngest, IngestEmail, IngestSource},\n};\nuse common::auth::AccessToken;\nuse jmap_client::{email, mailbox::Role};\nuse mail_parser::{MessageParser, mailbox::mbox::MessageIterator};\nuse std::{io::Cursor, str::FromStr, time::Duration};\nuse store::{\n    ahash::{AHashMap, AHashSet},\n    rand::{self, Rng},\n};\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    test_single_thread(params).await;\n    test_multi_thread(params).await;\n}\n\nasync fn test_single_thread(params: &mut JMAPTest) {\n    println!(\"Running Email Merge Threads tests...\");\n    let account = params.account(\"admin\");\n    let mut client = account.client_owned().await;\n    let mut all_mailboxes = AHashMap::default();\n\n    for (base_test_num, test) in [test_1(), test_2(), test_3()].iter().enumerate() {\n        let base_test_num = ((base_test_num * 6) as u32) + 1;\n        let mut messages = Vec::new();\n        let mut total_messages = 0;\n        let mut messages_per_thread =\n            build_messages(test, &mut messages, &mut total_messages, None, 0);\n        messages_per_thread.sort_unstable();\n\n        let mut mailbox_ids = Vec::with_capacity(6);\n\n        for test_num in 0..=5 {\n            mailbox_ids.push(\n                client\n                    .set_default_account_id(Id::new((base_test_num + test_num) as u64).to_string())\n                    .mailbox_create(\"Thread nightmare\", None::<String>, Role::None)\n                    .await\n                    .unwrap()\n                    .take_id(),\n            );\n        }\n\n        for message in &messages {\n            client\n                .set_default_account_id(Id::new(base_test_num as u64).to_string())\n                .email_import(\n                    message.to_string().into_bytes(),\n                    [mailbox_ids[0].clone()],\n                    None::<Vec<String>>,\n                    None,\n                )\n                .await\n                .unwrap();\n        }\n\n        for message in messages.iter().rev() {\n            client\n                .set_default_account_id(Id::new((base_test_num + 1) as u64).to_string())\n                .email_import(\n                    message.to_string().into_bytes(),\n                    [mailbox_ids[1].clone()],\n                    None::<Vec<String>>,\n                    None,\n                )\n                .await\n                .unwrap();\n        }\n\n        for chunk in messages.chunks(5) {\n            client.set_default_account_id(Id::new((base_test_num + 2) as u64).to_string());\n\n            for message in chunk {\n                client\n                    .email_import(\n                        message.to_string().into_bytes(),\n                        [mailbox_ids[2].clone()],\n                        None::<Vec<String>>,\n                        None,\n                    )\n                    .await\n                    .unwrap();\n            }\n\n            client.set_default_account_id(Id::new((base_test_num + 3) as u64).to_string());\n\n            for message in chunk.iter().rev() {\n                client\n                    .email_import(\n                        message.to_string().into_bytes(),\n                        [mailbox_ids[3].clone()],\n                        None::<Vec<String>>,\n                        None,\n                    )\n                    .await\n                    .unwrap();\n            }\n        }\n\n        for chunk in messages.chunks(5).rev() {\n            client.set_default_account_id(Id::new((base_test_num + 4) as u64).to_string());\n\n            for message in chunk {\n                client\n                    .email_import(\n                        message.to_string().into_bytes(),\n                        [mailbox_ids[4].clone()],\n                        None::<Vec<String>>,\n                        None,\n                    )\n                    .await\n                    .unwrap();\n            }\n\n            client.set_default_account_id(Id::new((base_test_num + 5) as u64).to_string());\n\n            for message in chunk.iter().rev() {\n                client\n                    .email_import(\n                        message.to_string().into_bytes(),\n                        [mailbox_ids[5].clone()],\n                        None::<Vec<String>>,\n                        None,\n                    )\n                    .await\n                    .unwrap();\n            }\n        }\n\n        wait_for_index(&params.server).await;\n\n        for test_num in 0..=5 {\n            let result = client\n                .set_default_account_id(Id::new((base_test_num + test_num) as u64).to_string())\n                .email_query(\n                    email::query::Filter::in_mailbox(mailbox_ids[test_num as usize].clone()).into(),\n                    None::<Vec<_>>,\n                )\n                .await\n                .unwrap();\n\n            assert_eq!(\n                result.ids().len(),\n                total_messages,\n                \"test# {}/{}\",\n                base_test_num,\n                test_num\n            );\n\n            let thread_ids: AHashSet<u32> = result\n                .ids()\n                .iter()\n                .map(|id| Id::from_str(id).unwrap().prefix_id())\n                .collect();\n\n            assert_eq!(\n                thread_ids.len(),\n                messages_per_thread.len(),\n                \"{:?}: test# {}/{}\",\n                thread_ids,\n                base_test_num,\n                test_num\n            );\n\n            let mut messages_per_thread_db = Vec::new();\n\n            for thread_id in thread_ids {\n                messages_per_thread_db.push(\n                    client\n                        .thread_get(&Id::new(thread_id as u64).to_string())\n                        .await\n                        .unwrap()\n                        .unwrap()\n                        .email_ids()\n                        .len(),\n                );\n            }\n            messages_per_thread_db.sort_unstable();\n\n            assert_eq!(messages_per_thread_db, messages_per_thread);\n            println!(\"passed test# {}/{}\", base_test_num, test_num);\n        }\n\n        all_mailboxes.insert(base_test_num as usize, mailbox_ids);\n    }\n\n    // Delete all messages and make sure no keys are left in the store.\n    for (base_test_num, mailbox_ids) in all_mailboxes {\n        for (test_num, _) in mailbox_ids.into_iter().enumerate() {\n            client.set_default_account_id(Id::new((base_test_num + test_num) as u64).to_string());\n            destroy_all_mailboxes_no_wait(&client).await;\n        }\n    }\n\n    params.assert_is_empty().await;\n}\n\n#[allow(dead_code)]\nasync fn test_multi_thread(params: &mut JMAPTest) {\n    println!(\"Running Email Merge Threads tests (multi-threaded)...\");\n    let mut handles = vec![];\n    let account = params.account(\"jdoe@example.com\");\n    let account_id = account.id().document_id();\n    let mailbox_id = INBOX_ID;\n\n    for message in MessageIterator::new(Cursor::new(deflate_test_resource(\"mailbox.gz\")))\n        .collect::<Vec<_>>()\n        .into_iter()\n    {\n        let message = message.unwrap();\n        let server = params.server.clone();\n        handles.push(tokio::task::spawn(async move {\n            let mut retry_count = 0;\n            loop {\n                match server\n                    .email_ingest(IngestEmail {\n                        raw_message: message.contents(),\n                        message: MessageParser::new().parse(message.contents()),\n                        blob_hash: None,\n                        access_token: &AccessToken::from_id(account_id),\n                        mailbox_ids: vec![mailbox_id],\n                        keywords: vec![],\n                        received_at: None,\n                        source: IngestSource::Smtp {\n                            deliver_to: \"test@domain.org\",\n                            is_sender_authenticated: true,\n                            is_spam: false,\n                        },\n                        session_id: 0,\n                    })\n                    .await\n                {\n                    Ok(_) => break,\n                    Err(err) => {\n                        if err.is_assertion_failure() && retry_count < 10 {\n                            //println!(\"Retrying ingest for {}...\", message.from());\n                            let backoff = rand::rng().random_range(50..=300);\n                            tokio::time::sleep(Duration::from_millis(backoff)).await;\n                            retry_count += 1;\n                            continue;\n                        }\n                        panic!(\"Failed to ingest message: {:?}\", err);\n                    }\n                }\n            }\n        }));\n    }\n    // Wait for all tasks to complete\n    let messages = handles.len();\n    println!(\"Waiting for {} tasks to complete...\", messages);\n    for handle in handles {\n        handle.await.expect(\"Task panicked\");\n    }\n    assert_eq!(\n        messages,\n        params\n            .server\n            .get_cached_messages(account_id)\n            .await\n            .unwrap()\n            .emails\n            .items\n            .len(),\n    );\n    println!(\"Deleting all messages...\");\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n\nfn build_message(message: usize, in_reply_to: Option<usize>, thread_num: usize) -> String {\n    if let Some(in_reply_to) = in_reply_to {\n        format!(\n            \"Message-ID: <{}>\\nReferences: <{}>\\nSubject: re: T{}\\n\\nreply\\n\",\n            message, in_reply_to, thread_num\n        )\n    } else {\n        format!(\n            \"Message-ID: <{}>\\nSubject: T{}\\n\\nmsg\\n\",\n            message, thread_num\n        )\n    }\n}\n\nfn build_messages(\n    three: &ThreadTest,\n    messages: &mut Vec<String>,\n    total_messages: &mut usize,\n    in_reply_to: Option<usize>,\n    thread_num: usize,\n) -> Vec<usize> {\n    let mut messages_per_thread = Vec::new();\n    match three {\n        ThreadTest::Message => {\n            *total_messages += 1;\n            messages.push(build_message(*total_messages, in_reply_to, thread_num));\n        }\n        ThreadTest::MessageWithReplies(replies) => {\n            *total_messages += 1;\n            messages.push(build_message(*total_messages, in_reply_to, thread_num));\n            let in_reply_to = Some(*total_messages);\n            for reply in replies {\n                build_messages(reply, messages, total_messages, in_reply_to, thread_num);\n            }\n        }\n        ThreadTest::Root(items) => {\n            for (thread_num, item) in items.iter().enumerate() {\n                let count_start = *total_messages;\n                build_messages(item, messages, total_messages, None, thread_num);\n                messages_per_thread.push(*total_messages - count_start);\n            }\n        }\n    }\n    messages_per_thread\n}\n\npub fn build_thread_test_messages() -> Vec<String> {\n    let mut messages = Vec::new();\n    let mut total_messages = 0;\n    build_messages(&test_3(), &mut messages, &mut total_messages, None, 0);\n    messages\n}\n\npub enum ThreadTest {\n    Message,\n    MessageWithReplies(Vec<ThreadTest>),\n    Root(Vec<ThreadTest>),\n}\n\nfn test_1() -> ThreadTest {\n    ThreadTest::Root(vec![\n        ThreadTest::Message,\n        ThreadTest::MessageWithReplies(vec![\n            ThreadTest::Message,\n            ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n            ThreadTest::MessageWithReplies(vec![\n                ThreadTest::Message,\n                ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::Message,\n                    ThreadTest::Message,\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::Message,\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                        ]),\n                    ]),\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::Message,\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::Message,\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                    ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                ]),\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                ]),\n                            ]),\n                        ]),\n                    ]),\n                ]),\n            ]),\n        ]),\n    ])\n}\n\nfn test_2() -> ThreadTest {\n    ThreadTest::Root(vec![\n        ThreadTest::MessageWithReplies(vec![\n            ThreadTest::Message,\n            ThreadTest::Message,\n            ThreadTest::Message,\n            ThreadTest::MessageWithReplies(vec![\n                ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::Message,\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::Message,\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                    ]),\n                                    ThreadTest::Message,\n                                ]),\n                                ThreadTest::Message,\n                            ]),\n                            ThreadTest::Message,\n                        ]),\n                        ThreadTest::Message,\n                    ]),\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(\n                                vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::Message,\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                        ]),\n                                    ]),\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::Message,\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                    ]),\n                                ],\n                            )]),\n                            ThreadTest::Message,\n                        ]),\n                        ThreadTest::Message,\n                        ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                    ]),\n                                ]),\n                                ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::Message,\n                                        ThreadTest::Message,\n                                        ThreadTest::Message,\n                                    ]),\n                                    ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                    ThreadTest::Message,\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                    ]),\n                                ]),\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                ]),\n                            ]),\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                        ])]),\n                    ]),\n                ]),\n                ThreadTest::Message,\n                ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                        ]),\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                ]),\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                    ]),\n                                    ThreadTest::Message,\n                                ]),\n                            ]),\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::Message,\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                        ThreadTest::Message,\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                    ]),\n                                ]),\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                    ]),\n                                ]),\n                                ThreadTest::Message,\n                                ThreadTest::Message,\n                            ]),\n                        ]),\n                        ThreadTest::Message,\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                ]),\n                                ThreadTest::Message,\n                                ThreadTest::Message,\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                            ]),\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::Message,\n                                        ThreadTest::Message,\n                                    ]),\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                ]),\n                            ]),\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                        ]),\n                    ]),\n                    ThreadTest::Message,\n                    ThreadTest::Message,\n                ]),\n            ]),\n        ]),\n        ThreadTest::Message,\n        ThreadTest::MessageWithReplies(vec![ThreadTest::Message, ThreadTest::Message]),\n    ])\n}\n\nfn test_3() -> ThreadTest {\n    ThreadTest::Root(vec![\n        ThreadTest::MessageWithReplies(vec![ThreadTest::Message, ThreadTest::Message]),\n        ThreadTest::Message,\n        ThreadTest::MessageWithReplies(vec![\n            ThreadTest::MessageWithReplies(vec![\n                ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::Message,\n                    ThreadTest::Message,\n                    ThreadTest::Message,\n                ]),\n                ThreadTest::Message,\n                ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                ThreadTest::Message,\n            ]),\n            ThreadTest::Message,\n            ThreadTest::Message,\n        ]),\n        ThreadTest::Message,\n        ThreadTest::MessageWithReplies(vec![\n            ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n            ThreadTest::MessageWithReplies(vec![\n                ThreadTest::Message,\n                ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::Message,\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(\n                                vec![ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                ])],\n                            )]),\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                        ])]),\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::Message,\n                                ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::Message,\n                                    ]),\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                        ThreadTest::Message,\n                                    ]),\n                                ]),\n                            ]),\n                        ]),\n                    ]),\n                    ThreadTest::Message,\n                    ThreadTest::Message,\n                ])]),\n                ThreadTest::Message,\n            ]),\n            ThreadTest::MessageWithReplies(vec![\n                ThreadTest::Message,\n                ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::Message,\n                ])]),\n                ThreadTest::Message,\n            ]),\n        ]),\n        ThreadTest::MessageWithReplies(vec![\n            ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(vec![\n                ThreadTest::Message,\n                ThreadTest::MessageWithReplies(vec![ThreadTest::Message, ThreadTest::Message]),\n                ThreadTest::Message,\n                ThreadTest::Message,\n            ])]),\n            ThreadTest::MessageWithReplies(vec![\n                ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::Message,\n                        ThreadTest::Message,\n                        ThreadTest::Message,\n                        ThreadTest::Message,\n                    ]),\n                    ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(\n                                vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::Message,\n                                        ThreadTest::Message,\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                        ]),\n                                    ]),\n                                    ThreadTest::Message,\n                                ],\n                            )]),\n                            ThreadTest::Message,\n                            ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(\n                                vec![\n                                    ThreadTest::Message,\n                                    ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                                ],\n                            )]),\n                        ]),\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::Message,\n                        ]),\n                    ])]),\n                    ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                    ThreadTest::Message,\n                ]),\n                ThreadTest::MessageWithReplies(vec![ThreadTest::Message]),\n                ThreadTest::Message,\n            ]),\n        ]),\n        ThreadTest::MessageWithReplies(vec![\n            ThreadTest::Message,\n            ThreadTest::MessageWithReplies(vec![\n                ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::MessageWithReplies(vec![\n                                ThreadTest::MessageWithReplies(vec![\n                                    ThreadTest::MessageWithReplies(vec![\n                                        ThreadTest::MessageWithReplies(vec![\n                                            ThreadTest::Message,\n                                            ThreadTest::Message,\n                                            ThreadTest::MessageWithReplies(vec![\n                                                ThreadTest::Message,\n                                                ThreadTest::Message,\n                                            ]),\n                                        ]),\n                                        ThreadTest::Message,\n                                    ]),\n                                    ThreadTest::Message,\n                                    ThreadTest::Message,\n                                ]),\n                                ThreadTest::Message,\n                                ThreadTest::Message,\n                                ThreadTest::Message,\n                            ]),\n                            ThreadTest::Message,\n                        ]),\n                        ThreadTest::Message,\n                    ]),\n                    ThreadTest::Message,\n                ]),\n                ThreadTest::MessageWithReplies(vec![\n                    ThreadTest::Message,\n                    ThreadTest::MessageWithReplies(vec![\n                        ThreadTest::Message,\n                        ThreadTest::Message,\n                        ThreadTest::MessageWithReplies(vec![\n                            ThreadTest::Message,\n                            ThreadTest::MessageWithReplies(vec![ThreadTest::MessageWithReplies(\n                                vec![ThreadTest::Message, ThreadTest::Message],\n                            )]),\n                            ThreadTest::Message,\n                        ]),\n                        ThreadTest::Message,\n                    ]),\n                ]),\n            ]),\n        ]),\n    ])\n}\n"
  },
  {
    "path": "tests/src/jmap/mail/vacation_response.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    jmap::{\n        JMAPTest,\n        mail::{\n            delivery::SmtpConnection,\n            submission::{\n                MockMessage, assert_message_delivery, expect_nothing, spawn_mock_smtp_server,\n            },\n        },\n    },\n    smtp::DnsCache,\n};\nuse chrono::{TimeDelta, Utc};\nuse std::time::Instant;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Vacation Response tests...\");\n\n    // Create test account\n    let server = params.server.clone();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    // Start mock SMTP server\n    let (mut smtp_rx, smtp_settings) = spawn_mock_smtp_server();\n    server.ipv4_add(\n        \"localhost\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + std::time::Duration::from_secs(10),\n    );\n\n    // Let people know that we'll be down in Kokomo\n    client\n        .vacation_response_create(\n            \"Off the Florida Keys there's a place called Kokomo\",\n            \"That's where you wanna go to get away from it all\".into(),\n            \"That's where <b>you wanna go</b> to get away from it all\".into(),\n        )\n        .await\n        .unwrap();\n\n    // Connect to LMTP service\n    let mut lmtp = SmtpConnection::connect().await;\n\n    // Send a message\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n\n    // Await vacation response\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\"<jdoe@example.com>\", [\"<bill@remote.org>\"], \"@Kokomo\"),\n    )\n    .await;\n\n    // Further messages from the same recipient should not\n    // trigger a vacation response\n    lmtp.ingest(\n        \"bill@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@remote.org\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report -- friendly reminder\\r\\n\",\n            \"\\r\\n\",\n            \"Listen, are you gonna have those TPS reports for us this afternoon?\",\n        ),\n    )\n    .await;\n\n    expect_nothing(&mut smtp_rx).await;\n\n    // Messages from MAILER-DAEMON should not\n    // trigger a vacation response\n    lmtp.ingest(\n        \"MAILER-DAEMON@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: MAILER-DAEMON@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: Delivery Failure\\r\\n\",\n            \"\\r\\n\",\n            \"I tried so hard and got so far but in the end it wasn't delivered.\",\n        ),\n    )\n    .await;\n\n    expect_nothing(&mut smtp_rx).await;\n\n    // Vacation responses should honor the configured date ranges\n    client\n        .vacation_response_set_dates(\n            (Utc::now() + TimeDelta::try_days(1).unwrap_or_default())\n                .timestamp()\n                .into(),\n            None,\n        )\n        .await\n        .unwrap();\n    lmtp.ingest(\n        \"jane_smith@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: jane_smith@remote.org\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: When were you going on holidays?\\r\\n\",\n            \"\\r\\n\",\n            \"I'm asking because Bill really wants those TPS reports.\",\n        ),\n    )\n    .await;\n\n    expect_nothing(&mut smtp_rx).await;\n\n    client\n        .vacation_response_set_dates(\n            (Utc::now() - TimeDelta::try_days(1).unwrap_or_default())\n                .timestamp()\n                .into(),\n            None,\n        )\n        .await\n        .unwrap();\n    smtp_settings.lock().do_stop = true;\n    lmtp.ingest(\n        \"jane_smith@remote.org\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: jane_smith@remote.org\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: When were you going on holidays?\\r\\n\",\n            \"\\r\\n\",\n            \"I'm asking because Bill really wants those TPS reports.\",\n        ),\n    )\n    .await;\n    lmtp.quit().await;\n\n    assert_message_delivery(\n        &mut smtp_rx,\n        MockMessage::new(\"<jdoe@example.com>\", [\"<jane_smith@remote.org>\"], \"@Kokomo\"),\n    )\n    .await;\n\n    // Remove test data\n    client.vacation_response_destroy().await.unwrap();\n    params.destroy_all_mailboxes(account).await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    AssertConfig, add_test_certs,\n    directory::internal::TestInternalDirectory,\n    jmap::server::{\n        enterprise::{EnterpriseCore, insert_test_metrics},\n        webhooks::{MockWebhookEndpoint, spawn_mock_webhook_endpoint},\n    },\n    store::{\n        TempDir, build_store_config,\n        cleanup::{search_store_destroy, store_assert_is_empty, store_destroy},\n    },\n};\nuse ahash::AHashMap;\nuse base64::{\n    Engine,\n    engine::general_purpose::{self, STANDARD},\n};\nuse common::{\n    Caches, Core, Data, Inner, Server,\n    config::{\n        server::{Listeners, ServerProtocol},\n        telemetry::Telemetry,\n    },\n    core::BuildServer,\n    manager::{\n        boot::build_ipc,\n        config::{ConfigManager, Patterns},\n    },\n};\nuse http::HttpSessionManager;\nuse hyper::{Method, header::AUTHORIZATION};\nuse imap::core::ImapSessionManager;\nuse jmap_client::client::{Client, Credentials};\nuse jmap_proto::error::request::RequestError;\nuse managesieve::core::ManageSieveSessionManager;\nuse pop3::Pop3SessionManager;\nuse reqwest::header;\nuse serde::{Deserialize, Serialize, de::DeserializeOwned};\nuse serde_json::{Value, json};\nuse services::{\n    SpawnServices,\n    task_manager::{Task, TaskAction},\n};\nuse smtp::{SpawnQueueManager, core::SmtpSessionManager};\nuse std::{\n    fmt::{Debug, Display},\n    path::PathBuf,\n    sync::Arc,\n    time::Duration,\n};\nuse store::{\n    IterateParams, SUBSPACE_TASK_QUEUE, Stores, U32_LEN, U64_LEN,\n    write::{AnyKey, TaskEpoch, key::DeserializeBigEndian},\n};\nuse tokio::sync::watch;\nuse types::id::Id;\nuse utils::config::Config;\n\npub mod auth;\npub mod calendar;\npub mod contacts;\npub mod core;\npub mod files;\npub mod mail;\npub mod principal;\npub mod server;\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn jmap_tests() {\n    let delete = std::env::var(\"NO_DELETE\").is_err();\n    let mut params = init_jmap_tests(delete).await;\n\n    server::webhooks::test(&mut params).await;\n\n    mail::get::test(&mut params).await;\n    mail::set::test(&mut params).await;\n    mail::parse::test(&mut params).await;\n    mail::query::test(&mut params, delete).await;\n    mail::search_snippet::test(&mut params).await;\n    mail::changes::test(&mut params).await;\n    mail::query_changes::test(&mut params).await;\n    mail::copy::test(&mut params).await;\n    mail::thread_get::test(&mut params).await;\n    mail::thread_merge::test(&mut params).await;\n    mail::mailbox::test(&mut params).await;\n    mail::delivery::test(&mut params).await;\n    mail::acl::test(&mut params).await;\n    mail::sieve_script::test(&mut params).await;\n    mail::vacation_response::test(&mut params).await;\n    mail::submission::test(&mut params).await;\n    mail::crypto::test(&mut params).await;\n    mail::antispam::test(&mut params).await;\n\n    core::event_source::test(&mut params).await;\n    core::websocket::test(&mut params).await;\n    core::push_subscription::test(&mut params).await;\n    core::blob::test(&mut params).await;\n\n    auth::limits::test(&mut params).await;\n    auth::oauth::test(&mut params).await;\n    auth::quota::test(&mut params).await;\n    auth::permissions::test(&params).await;\n\n    contacts::addressbook::test(&mut params).await;\n    contacts::contact::test(&mut params).await;\n    contacts::acl::test(&mut params).await;\n\n    files::node::test(&mut params).await;\n    files::acl::test(&mut params).await;\n\n    calendar::calendars::test(&mut params).await;\n    calendar::event::test(&mut params).await;\n    calendar::notification::test(&mut params).await;\n    calendar::alarm::test(&mut params).await;\n\n    calendar::identity::test(&mut params).await;\n    calendar::acl::test(&mut params).await;\n\n    principal::get::test(&mut params).await;\n    principal::availability::test(&mut params).await;\n\n    server::purge::test(&mut params).await;\n    server::enterprise::test(&mut params).await;\n\n    assert_is_empty(&params.server).await;\n\n    if delete {\n        params.temp_dir.delete();\n    }\n}\n\n#[ignore]\n#[tokio::test(flavor = \"multi_thread\")]\npub async fn jmap_metric_tests() {\n    let params = init_jmap_tests(false).await;\n\n    insert_test_metrics(params.server.core.clone()).await;\n}\n\n#[allow(dead_code)]\npub struct JMAPTest {\n    server: Server,\n    accounts: AHashMap<&'static str, Account>,\n    temp_dir: TempDir,\n    webhook: Arc<MockWebhookEndpoint>,\n    shutdown_tx: watch::Sender<bool>,\n}\n\npub struct Account {\n    name: &'static str,\n    secret: &'static str,\n    emails: &'static [&'static str],\n    id: Id,\n    id_string: String,\n    client: Client,\n}\n\nimpl JMAPTest {\n    pub fn account(&self, name: &str) -> &Account {\n        self.accounts.get(name).unwrap()\n    }\n\n    pub async fn assert_is_empty(&self) {\n        assert_is_empty(&self.server).await;\n    }\n}\n\nimpl Account {\n    pub fn id(&self) -> &Id {\n        &self.id\n    }\n\n    pub fn id_string(&self) -> &str {\n        &self.id_string\n    }\n\n    pub fn client(&self) -> &Client {\n        &self.client\n    }\n\n    pub fn name(&self) -> &'static str {\n        self.name\n    }\n    pub fn secret(&self) -> &'static str {\n        self.secret\n    }\n\n    pub fn emails(&self) -> &'static [&'static str] {\n        self.emails\n    }\n\n    pub async fn client_owned(&self) -> Client {\n        Client::new()\n            .credentials(Credentials::basic(self.name(), self.secret()))\n            .timeout(Duration::from_secs(3600))\n            .accept_invalid_certs(true)\n            .follow_redirects([\"127.0.0.1\"])\n            .connect(\"https://127.0.0.1:8899\")\n            .await\n            .unwrap()\n    }\n}\n\npub async fn wait_for_index(server: &Server) {\n    let mut count = 0;\n    loop {\n        let mut has_index_tasks = None;\n        server\n            .core\n            .storage\n            .data\n            .iterate(\n                IterateParams::new(\n                    AnyKey {\n                        subspace: SUBSPACE_TASK_QUEUE,\n                        key: vec![0u8],\n                    },\n                    AnyKey {\n                        subspace: SUBSPACE_TASK_QUEUE,\n                        key: vec![u8::MAX; 16],\n                    },\n                )\n                .ascending(),\n                |key, value| {\n                    has_index_tasks = Some(\n                        Task::<TaskAction>::deserialize(key, value).unwrap_or_else(|_| Task {\n                            due: TaskEpoch::from_inner(\n                                key.deserialize_be_u64(key.len() - U64_LEN).unwrap(),\n                            ),\n                            account_id: key.deserialize_be_u32(U64_LEN).unwrap(),\n                            document_id: key.deserialize_be_u32(U64_LEN + U32_LEN + 1).unwrap(),\n                            action: TaskAction::SendImip,\n                        }),\n                    );\n\n                    Ok(false)\n                },\n            )\n            .await\n            .unwrap();\n\n        if let Some(task) = has_index_tasks {\n            count += 1;\n            if count % 10 == 0 {\n                println!(\"Waiting for pending task {:?}...\", task);\n            }\n            tokio::time::sleep(Duration::from_millis(300)).await;\n        } else {\n            break;\n        }\n    }\n}\n\npub async fn assert_is_empty(server: &Server) {\n    // Wait for pending index tasks\n    wait_for_index(server).await;\n\n    // Assert is empty\n    store_assert_is_empty(server.store(), server.core.storage.blob.clone(), false).await;\n    search_store_destroy(server.search_store()).await;\n\n    // Clean caches\n    for cache in [\n        &server.inner.cache.events,\n        &server.inner.cache.contacts,\n        &server.inner.cache.files,\n        &server.inner.cache.scheduling,\n    ] {\n        cache.clear();\n    }\n    server.inner.cache.messages.clear();\n}\n\nasync fn init_jmap_tests(delete_if_exists: bool) -> JMAPTest {\n    // Load and parse config\n    let temp_dir = TempDir::new(\"jmap_tests\", delete_if_exists);\n    let mut config = Config::new(\n        add_test_certs(&(build_store_config(&temp_dir.path.to_string_lossy()) + SERVER))\n            .replace(\"{TMP}\", &temp_dir.path.display().to_string())\n            .replace(\n                \"{LEVEL}\",\n                &std::env::var(\"LOG\").unwrap_or_else(|_| \"disable\".to_string()),\n            ),\n    )\n    .unwrap();\n    config.resolve_all_macros().await;\n\n    // Parse servers\n    let mut servers = Listeners::parse(&mut config);\n\n    // Bind ports and drop privileges\n    servers.bind_and_drop_priv(&mut config);\n\n    // Build stores\n    let stores = Stores::parse_all(&mut config, false).await;\n\n    // Parse core\n    let config_manager = ConfigManager {\n        cfg_local: Default::default(),\n        cfg_local_path: PathBuf::new(),\n        cfg_local_patterns: Patterns::parse(&mut config).into(),\n        cfg_store: config\n            .value(\"storage.data\")\n            .and_then(|id| stores.stores.get(id))\n            .cloned()\n            .unwrap_or_default(),\n    };\n    let tracers = Telemetry::parse(&mut config, &stores);\n    let core = Core::parse(&mut config, stores, config_manager)\n        .await\n        .enable_enterprise();\n    let data = Data::parse(&mut config);\n    let cache = Caches::parse(&mut config);\n    let store = core.storage.data.clone();\n    let search_store = core.storage.fts.clone();\n    let (ipc, mut ipc_rxs) = build_ipc(false);\n    let inner = Arc::new(Inner {\n        shared_core: core.into_shared(),\n        data,\n        ipc,\n        cache,\n    });\n\n    if delete_if_exists {\n        store_destroy(&store).await;\n        search_store_destroy(&search_store).await;\n    }\n\n    // Parse acceptors\n    servers.parse_tcp_acceptors(&mut config, inner.clone());\n\n    // Enable tracing\n    tracers.enable(true);\n\n    // Start services\n    config.assert_no_errors();\n    ipc_rxs.spawn_queue_manager(inner.clone());\n    ipc_rxs.spawn_services(inner.clone());\n\n    // Spawn servers\n    let (shutdown_tx, _) = servers.spawn(|server, acceptor, shutdown_rx| {\n        match &server.protocol {\n            ServerProtocol::Smtp | ServerProtocol::Lmtp => server.spawn(\n                SmtpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Http => server.spawn(\n                HttpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Imap => server.spawn(\n                ImapSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Pop3 => server.spawn(\n                Pop3SessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::ManageSieve => server.spawn(\n                ManageSieveSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n        };\n    });\n\n    // Create tables\n    let server = inner.build_server();\n    let mut accounts = AHashMap::new();\n\n    for (name, secret, description, emails) in [\n        (\"admin\", \"secret\", \"Superuser\", &[][..]),\n        (\n            \"jdoe@example.com\",\n            \"12345\",\n            \"John Doe\",\n            &[\"jdoe@example.com\", \"john.doe@example.com\"][..],\n        ),\n        (\n            \"jane.smith@example.com\",\n            \"abcde\",\n            \"Jane Smith\",\n            &[\"jane.smith@example.com\"],\n        ),\n        (\n            \"bill@example.com\",\n            \"098765\",\n            \"Bill Foobar\",\n            &[\"bill@example.com\"],\n        ),\n        (\n            \"robert@example.com\",\n            \"aabbcc\",\n            \"Robert Foobar\",\n            &[\"robert@example.com\"][..],\n        ),\n    ] {\n        let id: Id = server\n            .store()\n            .create_test_user(name, secret, description, emails)\n            .await\n            .into();\n        let id_string = id.to_string();\n\n        let mut client = Client::new()\n            .credentials(Credentials::basic(name, secret))\n            .timeout(Duration::from_secs(3600))\n            .accept_invalid_certs(true)\n            .follow_redirects([\"127.0.0.1\"])\n            .connect(\"https://127.0.0.1:8899\")\n            .await\n            .unwrap();\n        client.set_default_account_id(id_string.clone());\n\n        accounts.insert(\n            name,\n            Account {\n                name,\n                secret,\n                emails,\n                id,\n                id_string,\n                client,\n            },\n        );\n    }\n\n    for (name, description, emails) in\n        [(\"sales@example.com\", \"Sales Group\", &[\"sales@example.com\"])]\n    {\n        let id: Id = server\n            .store()\n            .create_test_group(name, description, emails)\n            .await\n            .into();\n        let id_string = id.to_string();\n\n        let mut client = Client::new()\n            .credentials(Credentials::basic(\"admin\", \"secret\"))\n            .timeout(Duration::from_secs(3600))\n            .accept_invalid_certs(true)\n            .follow_redirects([\"127.0.0.1\"])\n            .connect(\"https://127.0.0.1:8899\")\n            .await\n            .unwrap();\n        client.set_default_account_id(id_string.clone());\n\n        accounts.insert(\n            name,\n            Account {\n                name,\n                secret: \"\",\n                emails,\n                id,\n                id_string,\n                client,\n            },\n        );\n    }\n\n    JMAPTest {\n        server,\n        temp_dir,\n        accounts,\n        shutdown_tx,\n        webhook: spawn_mock_webhook_endpoint(),\n    }\n}\n\npub struct JmapResponse(pub Value);\n\nimpl Account {\n    pub async fn jmap_get(\n        &self,\n        object: impl Display,\n        properties: impl IntoIterator<Item = impl Display>,\n        ids: impl IntoIterator<Item = impl Display>,\n    ) -> JmapResponse {\n        self.jmap_get_account(self, object, properties, ids).await\n    }\n\n    pub async fn jmap_get_account(\n        &self,\n        account: &Account,\n        object: impl Display,\n        properties: impl IntoIterator<Item = impl Display>,\n        ids: impl IntoIterator<Item = impl Display>,\n    ) -> JmapResponse {\n        let ids = ids\n            .into_iter()\n            .map(|id| Value::String(id.to_string()))\n            .collect::<Vec<Value>>();\n        self.jmap_method_calls(json!([[\n            format!(\"{object}/get\"),\n            {\n                \"accountId\": account.id_string(),\n                \"properties\": properties\n                .into_iter()\n                .map(|p| Value::String(p.to_string()))\n                .collect::<Vec<_>>(),\n                \"ids\": if !ids.is_empty() { Some(ids) } else { None }\n            },\n            \"0\"\n        ]]))\n        .await\n    }\n\n    pub async fn jmap_query(\n        &self,\n        object: impl Display,\n        filter: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n        sort_by: impl IntoIterator<Item = impl Display>,\n        arguments: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n    ) -> JmapResponse {\n        let filter = filter\n            .into_iter()\n            .map(|(k, v)| (k.to_string(), v.into()))\n            .collect::<serde_json::Map<_, _>>();\n        let sort_by = sort_by\n            .into_iter()\n            .map(|id| {\n                json! ({\n                    \"property\": id.to_string()\n                })\n            })\n            .collect::<Vec<Value>>();\n        let arguments = [\n            (\"filter\".to_string(), Value::Object(filter)),\n            (\"sort\".to_string(), Value::Array(sort_by)),\n        ]\n        .into_iter()\n        .chain(\n            arguments\n                .into_iter()\n                .map(|(k, v)| (k.to_string(), v.into())),\n        )\n        .collect::<serde_json::Map<_, _>>();\n\n        self.jmap_method_calls(json!([[format!(\"{object}/query\"), arguments, \"0\"]]))\n            .await\n    }\n\n    pub async fn jmap_create(\n        &self,\n        object: impl Display,\n        items: impl IntoIterator<Item = Value>,\n        arguments: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n    ) -> JmapResponse {\n        self.jmap_create_account(self, object, items, arguments)\n            .await\n    }\n\n    pub async fn jmap_create_account(\n        &self,\n        account: &Account,\n        object: impl Display,\n        items: impl IntoIterator<Item = Value>,\n        arguments: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n    ) -> JmapResponse {\n        let create = items\n            .into_iter()\n            .enumerate()\n            .map(|(i, item)| (format!(\"i{i}\"), item))\n            .collect::<serde_json::Map<_, _>>();\n        let arguments = [\n            (\n                \"accountId\".to_string(),\n                Value::String(account.id_string().to_string()),\n            ),\n            (\"create\".to_string(), Value::Object(create)),\n        ]\n        .into_iter()\n        .chain(\n            arguments\n                .into_iter()\n                .map(|(k, v)| (k.to_string(), v.into())),\n        )\n        .collect::<serde_json::Map<_, _>>();\n\n        self.jmap_method_calls(json!([[format!(\"{object}/set\"), arguments, \"0\"]]))\n            .await\n    }\n\n    pub async fn jmap_update(\n        &self,\n        object: impl Display,\n        items: impl IntoIterator<Item = (impl Display, Value)>,\n        arguments: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n    ) -> JmapResponse {\n        self.jmap_update_account(self, object, items, arguments)\n            .await\n    }\n\n    pub async fn jmap_update_account(\n        &self,\n        account: &Account,\n        object: impl Display,\n        items: impl IntoIterator<Item = (impl Display, Value)>,\n        arguments: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n    ) -> JmapResponse {\n        let update = items\n            .into_iter()\n            .map(|(i, item)| (i.to_string(), item))\n            .collect::<serde_json::Map<_, _>>();\n        let arguments = [\n            (\n                \"accountId\".to_string(),\n                Value::String(account.id_string().to_string()),\n            ),\n            (\"update\".to_string(), Value::Object(update)),\n        ]\n        .into_iter()\n        .chain(\n            arguments\n                .into_iter()\n                .map(|(k, v)| (k.to_string(), v.into())),\n        )\n        .collect::<serde_json::Map<_, _>>();\n\n        self.jmap_method_calls(json!([[format!(\"{object}/set\"), arguments, \"0\"]]))\n            .await\n    }\n\n    pub async fn jmap_destroy(\n        &self,\n        object: impl Display,\n        items: impl IntoIterator<Item = impl Display>,\n        arguments: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n    ) -> JmapResponse {\n        self.jmap_destroy_account(self, object, items, arguments)\n            .await\n    }\n\n    pub async fn jmap_destroy_account(\n        &self,\n        account: &Account,\n        object: impl Display,\n        items: impl IntoIterator<Item = impl Display>,\n        arguments: impl IntoIterator<Item = (impl Display, impl Into<Value>)>,\n    ) -> JmapResponse {\n        let destroy = items\n            .into_iter()\n            .map(|id| Value::String(id.to_string()))\n            .collect::<Vec<_>>();\n        let arguments = [\n            (\n                \"accountId\".to_string(),\n                Value::String(account.id_string().to_string()),\n            ),\n            (\"destroy\".to_string(), Value::Array(destroy)),\n        ]\n        .into_iter()\n        .chain(\n            arguments\n                .into_iter()\n                .map(|(k, v)| (k.to_string(), v.into())),\n        )\n        .collect::<serde_json::Map<_, _>>();\n\n        self.jmap_method_calls(json!([[format!(\"{object}/set\"), arguments, \"0\"]]))\n            .await\n    }\n\n    pub async fn jmap_copy(\n        &self,\n        from_account: &Account,\n        to_account: &Account,\n        object: impl Display,\n        items: impl IntoIterator<Item = (impl Display, Value)>,\n        on_success_destroy: bool,\n    ) -> JmapResponse {\n        self.jmap_method_calls(json!([[\n            format!(\"{object}/copy\"),\n            {\n                \"fromAccountId\": from_account.id_string(),\n                \"accountId\": to_account.id_string(),\n                \"onSuccessDestroyOriginal\": on_success_destroy,\n                \"create\": items\n                        .into_iter()\n                        .map(|(i, item)| (i.to_string(), item)).collect::<serde_json::Map<_, _>>()\n            },\n            \"0\"\n        ]]))\n        .await\n    }\n\n    pub async fn jmap_changes(&self, object: impl Display, state: impl Display) -> JmapResponse {\n        self.jmap_method_calls(json!([[\n            format!(\"{object}/changes\"),\n            {\n                \"sinceState\": state.to_string()\n            },\n            \"0\"\n        ]]))\n        .await\n    }\n\n    pub async fn jmap_method_call(&self, method_name: &str, body: Value) -> JmapResponse {\n        self.jmap_method_calls(json!([[method_name, body, \"0\"]]))\n            .await\n    }\n\n    pub async fn jmap_method_calls(&self, calls: Value) -> JmapResponse {\n        let mut headers = header::HeaderMap::new();\n\n        headers.insert(\n            header::AUTHORIZATION,\n            header::HeaderValue::from_str(&format!(\n                \"Basic {}\",\n                general_purpose::STANDARD.encode(format!(\"{}:{}\", self.name(), self.secret()))\n            ))\n            .unwrap(),\n        );\n\n        let body = json!({\n          \"using\": [ \"urn:ietf:params:jmap:core\", \"urn:ietf:params:jmap:mail\", \"urn:ietf:params:jmap:quota\" ],\n          \"methodCalls\": calls\n        });\n\n        JmapResponse(\n            serde_json::from_slice(\n                &reqwest::Client::builder()\n                    .danger_accept_invalid_certs(true)\n                    .timeout(Duration::from_millis(1000))\n                    .default_headers(headers)\n                    .build()\n                    .unwrap()\n                    .post(\"https://127.0.0.1:8899/jmap\")\n                    .body(body.to_string())\n                    .send()\n                    .await\n                    .unwrap()\n                    .bytes()\n                    .await\n                    .unwrap(),\n            )\n            .unwrap(),\n        )\n    }\n\n    pub async fn jmap_session_object(&self) -> JmapResponse {\n        let mut headers = header::HeaderMap::new();\n\n        headers.insert(\n            header::AUTHORIZATION,\n            header::HeaderValue::from_str(&format!(\n                \"Basic {}\",\n                general_purpose::STANDARD.encode(format!(\"{}:{}\", self.name(), self.secret()))\n            ))\n            .unwrap(),\n        );\n\n        JmapResponse(\n            serde_json::from_slice(\n                &reqwest::Client::builder()\n                    .danger_accept_invalid_certs(true)\n                    .timeout(Duration::from_millis(1000))\n                    .default_headers(headers)\n                    .build()\n                    .unwrap()\n                    .get(\"https://127.0.0.1:8899/jmap/session\")\n                    .send()\n                    .await\n                    .unwrap()\n                    .bytes()\n                    .await\n                    .unwrap(),\n            )\n            .unwrap(),\n        )\n    }\n\n    pub async fn destroy_all_addressbooks(&self) {\n        self.jmap_method_calls(json!([[\n            \"AddressBook/get\",\n            {\n              \"ids\" : (),\n              \"properties\" : [\n                \"id\"\n              ]\n            },\n            \"R1\"\n          ],\n          [\n            \"AddressBook/set\",\n            {\n              \"#destroy\" : {\n                    \"resultOf\": \"R1\",\n                    \"name\": \"AddressBook/get\",\n                    \"path\": \"/list/*/id\"\n                },\n              \"onDestroyRemoveContents\" : true\n            },\n            \"R2\"\n          ]\n        ]))\n        .await;\n    }\n\n    pub async fn destroy_all_calendars(&self) {\n        self.jmap_method_calls(json!([[\n            \"Calendar/get\",\n            {\n              \"ids\" : (),\n              \"properties\" : [\n                \"id\"\n              ]\n            },\n            \"R1\"\n          ],\n          [\n            \"Calendar/set\",\n            {\n              \"#destroy\" : {\n                    \"resultOf\": \"R1\",\n                    \"name\": \"Calendar/get\",\n                    \"path\": \"/list/*/id\"\n                },\n              \"onDestroyRemoveEvents\" : true\n            },\n            \"R2\"\n          ]\n        ]))\n        .await;\n    }\n\n    pub async fn destroy_all_event_notifications(&self) {\n        self.jmap_method_calls(json!([[\n            \"CalendarEventNotification/get\",\n            {\n              \"ids\" : (),\n              \"properties\" : [\n                \"id\"\n              ]\n            },\n            \"R1\"\n          ],\n          [\n            \"CalendarEventNotification/set\",\n            {\n              \"#destroy\" : {\n                    \"resultOf\": \"R1\",\n                    \"name\": \"CalendarEventNotification/get\",\n                    \"path\": \"/list/*/id\"\n                }\n            },\n            \"R2\"\n          ]\n        ]))\n        .await;\n    }\n}\n\nimpl JmapResponse {\n    pub fn created(&self, item_idx: u32) -> &Value {\n        self.0\n            .pointer(&format!(\"/methodResponses/0/1/created/i{item_idx}\"))\n            .unwrap_or_else(|| panic!(\"Missing created item {item_idx}: {self:?}\"))\n    }\n\n    pub fn not_created(&self, item_idx: u32) -> &Value {\n        self.0\n            .pointer(&format!(\"/methodResponses/0/1/notCreated/i{item_idx}\"))\n            .unwrap_or_else(|| panic!(\"Missing not created item {item_idx}: {self:?}\"))\n    }\n\n    pub fn updated(&self, id: &str) -> &Value {\n        self.0\n            .pointer(&format!(\"/methodResponses/0/1/updated/{id}\"))\n            .unwrap_or_else(|| panic!(\"Missing updated item {id}: {self:?}\"))\n    }\n\n    pub fn not_updated(&self, id: &str) -> &Value {\n        self.0\n            .pointer(&format!(\"/methodResponses/0/1/notUpdated/{id}\"))\n            .unwrap_or_else(|| panic!(\"Missing not updated item {id}: {self:?}\"))\n    }\n\n    pub fn copied(&self, id: &str) -> &Value {\n        self.0\n            .pointer(&format!(\"/methodResponses/0/1/created/{id}\"))\n            .unwrap_or_else(|| panic!(\"Missing updated item {id}: {self:?}\"))\n    }\n\n    pub fn method_response(&self) -> &Value {\n        self.0\n            .pointer(\"/methodResponses/0/1\")\n            .unwrap_or_else(|| panic!(\"Missing method response in response: {self:?}\"))\n    }\n\n    pub fn list_array(&self) -> &Value {\n        self.0\n            .pointer(\"/methodResponses/0/1/list\")\n            .unwrap_or_else(|| panic!(\"Missing list in response: {self:?}\"))\n    }\n\n    pub fn list(&self) -> &[Value] {\n        self.0\n            .pointer(\"/methodResponses/0/1/list\")\n            .and_then(|v| v.as_array())\n            .unwrap_or_else(|| panic!(\"Missing list in response: {self:?}\"))\n    }\n\n    pub fn not_found(&self) -> impl Iterator<Item = &str> {\n        self.0\n            .pointer(\"/methodResponses/0/1/notFound\")\n            .and_then(|v| v.as_array())\n            .unwrap_or_else(|| panic!(\"Missing notFound in response: {self:?}\"))\n            .iter()\n            .map(|v| v.as_str().unwrap())\n    }\n\n    pub fn ids(&self) -> impl Iterator<Item = &str> {\n        self.0\n            .pointer(\"/methodResponses/0/1/ids\")\n            .and_then(|v| v.as_array())\n            .unwrap_or_else(|| panic!(\"Missing ids in response: {self:?}\"))\n            .iter()\n            .map(|v| v.as_str().unwrap())\n    }\n\n    pub fn destroyed(&self) -> impl Iterator<Item = &str> {\n        self.0\n            .pointer(\"/methodResponses/0/1/destroyed\")\n            .and_then(|v| v.as_array())\n            .unwrap_or_else(|| panic!(\"Missing destroyed in response: {self:?}\"))\n            .iter()\n            .map(|v| v.as_str().unwrap())\n    }\n\n    pub fn not_destroyed(&self, id: &str) -> &Value {\n        self.0\n            .pointer(&format!(\"/methodResponses/0/1/notDestroyed/{id}\"))\n            .unwrap_or_else(|| panic!(\"Missing not destroyed item {id}: {self:?}\"))\n    }\n\n    pub fn state(&self) -> &str {\n        self.0\n            .pointer(\"/methodResponses/0/1/state\")\n            .and_then(|v| v.as_str())\n            .unwrap_or_else(|| panic!(\"Missing state in response: {self:?}\"))\n    }\n\n    pub fn new_state(&self) -> &str {\n        self.0\n            .pointer(\"/methodResponses/0/1/newState\")\n            .and_then(|v| v.as_str())\n            .unwrap_or_else(|| panic!(\"Missing new state in response: {self:?}\"))\n    }\n\n    pub fn changes(&self) -> impl Iterator<Item = ChangeType<'_>> {\n        self.changes_by_type(\"created\")\n            .map(ChangeType::Created)\n            .chain(self.changes_by_type(\"updated\").map(ChangeType::Updated))\n            .chain(self.changes_by_type(\"destroyed\").map(ChangeType::Destroyed))\n    }\n\n    fn changes_by_type(&self, typ: &str) -> impl Iterator<Item = &str> {\n        self.0\n            .pointer(&format!(\"/methodResponses/0/1/{typ}\"))\n            .and_then(|v| v.as_array())\n            .unwrap_or_else(|| panic!(\"Missing {typ} changes in response: {self:?}\"))\n            .iter()\n            .map(|v| v.as_str().unwrap())\n    }\n\n    pub fn pointer(&self, pointer: &str) -> Option<&Value> {\n        self.0.pointer(pointer)\n    }\n\n    pub fn into_inner(self) -> Value {\n        self.0\n    }\n}\n\npub trait JmapUtils {\n    fn id(&self) -> &str {\n        self.text_field(\"id\")\n    }\n\n    fn blob_id(&self) -> &str {\n        self.text_field(\"blobId\")\n    }\n\n    fn typ(&self) -> &str {\n        self.text_field(\"type\")\n    }\n\n    fn description(&self) -> &str {\n        self.text_field(\"description\")\n    }\n\n    fn with_property(self, field: impl Display, value: impl Into<Value>) -> Self;\n\n    fn text_field(&self, field: &str) -> &str;\n\n    fn assert_is_equal(&self, other: Value);\n}\n\nimpl JmapUtils for Value {\n    fn text_field(&self, field: &str) -> &str {\n        self.pointer(&format!(\"/{field}\"))\n            .and_then(|v| v.as_str())\n            .unwrap_or_else(|| panic!(\"Missing {field} in object: {self:?}\"))\n    }\n    fn assert_is_equal(&self, expected: Value) {\n        if self != &expected {\n            panic!(\n                \"Values are not equal:\\ngot: {}\\nexpected: {}\",\n                serde_json::to_string_pretty(self).unwrap(),\n                serde_json::to_string_pretty(&expected).unwrap()\n            );\n        }\n    }\n    fn with_property(mut self, field: impl Display, value: impl Into<Value>) -> Self {\n        if let Value::Object(map) = &mut self {\n            map.insert(field.to_string(), value.into());\n        } else {\n            panic!(\"Not an object: {self:?}\");\n        }\n        self\n    }\n}\n\n#[derive(Debug, Clone, PartialEq, Eq, Hash)]\npub enum ChangeType<'x> {\n    Created(&'x str),\n    Updated(&'x str),\n    Destroyed(&'x str),\n}\n\nimpl<'x> ChangeType<'x> {\n    pub fn as_created(&self) -> &str {\n        match self {\n            ChangeType::Created(id) => id,\n            _ => panic!(\"Not a created change: {self:?}\"),\n        }\n    }\n\n    pub fn as_updated(&self) -> &str {\n        match self {\n            ChangeType::Updated(id) => id,\n            _ => panic!(\"Not an updated change: {self:?}\"),\n        }\n    }\n\n    pub fn as_destroyed(&self) -> &str {\n        match self {\n            ChangeType::Destroyed(id) => id,\n            _ => panic!(\"Not a destroyed change: {self:?}\"),\n        }\n    }\n}\n\nimpl Display for JmapResponse {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        std::fmt::Display::fmt(&self.0, f)\n    }\n}\n\nimpl Debug for JmapResponse {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        serde_json::to_string_pretty(&self.0)\n            .map_err(|_| std::fmt::Error)\n            .and_then(|s| std::fmt::Display::fmt(&s, f))\n    }\n}\n\npub trait IntoJmapSet {\n    fn into_jmap_set(self) -> Value;\n}\n\nimpl<T: IntoIterator<Item = impl Display>> IntoJmapSet for T {\n    fn into_jmap_set(self) -> Value {\n        Value::Object(\n            self.into_iter()\n                .map(|id| (id.to_string(), Value::Bool(true)))\n                .collect::<serde_json::Map<String, Value>>(),\n        )\n    }\n}\n\npub fn find_values(string: &str, name: &str) -> Vec<String> {\n    let mut last_pos = 0;\n    let mut values = Vec::new();\n\n    while let Some(pos) = string[last_pos..].find(name) {\n        let mut value = string[last_pos + pos + name.len()..]\n            .split('\"')\n            .nth(1)\n            .unwrap();\n        if value.ends_with('\\\\') {\n            value = &value[..value.len() - 1];\n        }\n        values.push(value.to_string());\n        last_pos += pos + name.len();\n    }\n\n    values\n}\n\npub fn replace_values(mut string: String, find: &[String], replace: &[String]) -> String {\n    for (find, replace) in find.iter().zip(replace.iter()) {\n        string = string.replace(find, replace);\n    }\n    string\n}\n\npub fn replace_boundaries(string: String) -> String {\n    let values = find_values(&string, \"boundary=\");\n    if !values.is_empty() {\n        replace_values(\n            string,\n            &values,\n            &(0..values.len())\n                .map(|i| format!(\"boundary_{}\", i))\n                .collect::<Vec<_>>(),\n        )\n    } else {\n        string\n    }\n}\n\npub fn replace_blob_ids(string: String) -> String {\n    let values = find_values(&string, \"blobId\\\":\");\n    if !values.is_empty() {\n        replace_values(\n            string,\n            &values,\n            &(0..values.len())\n                .map(|i| format!(\"blob_{}\", i))\n                .collect::<Vec<_>>(),\n        )\n    } else {\n        string\n    }\n}\n\n#[derive(Deserialize)]\n#[serde(untagged)]\npub enum Response<T> {\n    RequestError(RequestError<'static>),\n    Error {\n        error: String,\n        details: Option<String>,\n        item: Option<String>,\n        reason: Option<String>,\n    },\n    Data {\n        data: T,\n    },\n}\n\npub struct ManagementApi {\n    pub port: u16,\n    pub username: String,\n    pub password: String,\n}\n\nimpl Default for ManagementApi {\n    fn default() -> Self {\n        Self {\n            port: 9980,\n            username: \"admin\".to_string(),\n            password: \"secret\".to_string(),\n        }\n    }\n}\n\nimpl ManagementApi {\n    pub fn new(port: u16, username: &str, password: &str) -> Self {\n        Self {\n            port,\n            username: username.to_string(),\n            password: password.to_string(),\n        }\n    }\n\n    pub async fn post<T: DeserializeOwned>(\n        &self,\n        query: &str,\n        body: &impl Serialize,\n    ) -> Result<Response<T>, String> {\n        self.request_raw(\n            Method::POST,\n            query,\n            Some(serde_json::to_string(body).unwrap()),\n        )\n        .await\n        .map(|result| {\n            serde_json::from_str::<Response<T>>(&result)\n                .unwrap_or_else(|err| panic!(\"{err}: {result}\"))\n        })\n    }\n\n    pub async fn patch<T: DeserializeOwned>(\n        &self,\n        query: &str,\n        body: &impl Serialize,\n    ) -> Result<Response<T>, String> {\n        self.request_raw(\n            Method::PATCH,\n            query,\n            Some(serde_json::to_string(body).unwrap()),\n        )\n        .await\n        .map(|result| {\n            serde_json::from_str::<Response<T>>(&result)\n                .unwrap_or_else(|err| panic!(\"{err}: {result}\"))\n        })\n    }\n\n    pub async fn delete<T: DeserializeOwned>(&self, query: &str) -> Result<Response<T>, String> {\n        self.request_raw(Method::DELETE, query, None)\n            .await\n            .map(|result| {\n                serde_json::from_str::<Response<T>>(&result)\n                    .unwrap_or_else(|err| panic!(\"{err}: {result}\"))\n            })\n    }\n\n    pub async fn get<T: DeserializeOwned>(&self, query: &str) -> Result<Response<T>, String> {\n        self.request_raw(Method::GET, query, None)\n            .await\n            .map(|result| {\n                serde_json::from_str::<Response<T>>(&result)\n                    .unwrap_or_else(|err| panic!(\"{err}: {result}\"))\n            })\n    }\n    pub async fn request<T: DeserializeOwned>(\n        &self,\n        method: Method,\n        query: &str,\n    ) -> Result<Response<T>, String> {\n        self.request_raw(method, query, None).await.map(|result| {\n            serde_json::from_str::<Response<T>>(&result)\n                .unwrap_or_else(|err| panic!(\"{err}: {result}\"))\n        })\n    }\n\n    async fn request_raw(\n        &self,\n        method: Method,\n        query: &str,\n        body: Option<String>,\n    ) -> Result<String, String> {\n        let mut request = reqwest::Client::builder()\n            .timeout(Duration::from_millis(500))\n            .danger_accept_invalid_certs(true)\n            .build()\n            .unwrap()\n            .request(method, format!(\"https://127.0.0.1:{}{query}\", self.port));\n\n        if let Some(body) = body {\n            request = request.body(body);\n        }\n\n        request\n            .header(\n                AUTHORIZATION,\n                format!(\n                    \"Basic {}\",\n                    STANDARD.encode(format!(\"{}:{}\", self.username, self.password).as_bytes())\n                ),\n            )\n            .send()\n            .await\n            .map_err(|err| err.to_string())?\n            .bytes()\n            .await\n            .map(|bytes| String::from_utf8(bytes.to_vec()).unwrap())\n            .map_err(|err| err.to_string())\n    }\n}\n\nimpl<T: Debug> Response<T> {\n    pub fn unwrap_data(self) -> T {\n        match self {\n            Response::Data { data } => data,\n            Response::Error {\n                error,\n                details,\n                reason,\n                ..\n            } => {\n                panic!(\"Expected data, found error {error:?}: {details:?} {reason:?}\")\n            }\n            Response::RequestError(err) => {\n                panic!(\"Expected data, found error {err:?}\")\n            }\n        }\n    }\n\n    pub fn try_unwrap_data(self) -> Option<T> {\n        match self {\n            Response::Data { data } => Some(data),\n            Response::RequestError(error) if error.status == 404 => None,\n            Response::Error {\n                error,\n                details,\n                reason,\n                ..\n            } => {\n                panic!(\"Expected data, found error {error:?}: {details:?} {reason:?}\")\n            }\n            Response::RequestError(err) => {\n                panic!(\"Expected data, found error {err:?}\")\n            }\n        }\n    }\n\n    pub fn unwrap_error(self) -> (String, Option<String>, Option<String>) {\n        match self {\n            Response::Error {\n                error,\n                details,\n                reason,\n                ..\n            } => (error, details, reason),\n            Response::Data { data } => panic!(\"Expected error, found data: {data:?}\"),\n            Response::RequestError(err) => {\n                panic!(\"Expected error, found request error {err:?}\")\n            }\n        }\n    }\n\n    pub fn unwrap_request_error(self) -> RequestError<'static> {\n        match self {\n            Response::Error {\n                error,\n                details,\n                reason,\n                ..\n            } => {\n                panic!(\"Expected request error, found error {error:?}: {details:?} {reason:?}\")\n            }\n            Response::Data { data } => panic!(\"Expected request error, found data: {data:?}\"),\n            Response::RequestError(err) => err,\n        }\n    }\n\n    pub fn expect_request_error(self, value: &str) {\n        let err = self.unwrap_request_error();\n        if !err.detail.contains(value) && !err.title.as_ref().is_some_and(|t| t.contains(value)) {\n            panic!(\"Expected request error containing {value:?}, found {err:?}\")\n        }\n    }\n\n    pub fn expect_error(self, value: &str) {\n        let (error, details, reason) = self.unwrap_error();\n        if !error.contains(value)\n            && !details.as_ref().is_some_and(|d| d.contains(value))\n            && !reason.as_ref().is_some_and(|r| r.contains(value))\n        {\n            panic!(\"Expected error containing {value:?}, found {error:?}: {details:?} {reason:?}\")\n        }\n    }\n}\n\nconst SERVER: &str = r#\"\n[server]\nhostname = \"'jmap.example.org'\"\n\n[http]\nurl = \"'https://127.0.0.1:8899'\"\n\n[server.listener.jmap]\nbind = [\"127.0.0.1:8899\"]\nprotocol = \"http\"\nmax-connections = 81920\ntls.implicit = true\n\n[server.listener.imap]\nbind = [\"127.0.0.1:9991\"]\nprotocol = \"imap\"\nmax-connections = 81920\n\n[server.listener.lmtp-debug]\nbind = ['127.0.0.1:11200']\ngreeting = 'Test LMTP instance'\nprotocol = 'lmtp'\ntls.implicit = false\n\n[server.listener.pop3]\nbind = [\"127.0.0.1:4110\"]\nprotocol = \"pop3\"\nmax-connections = 81920\ntls.implicit = true\n\n[server.socket]\nreuse-addr = true\n\n[server.tls]\nenable = true\nimplicit = false\ncertificate = \"default\"\n\n[server.fail2ban]\nauthentication = \"100/5s\"\n\n[authentication]\nrate-limit = \"100/2s\"\n\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = [ { if = \"!is_empty(authenticated_as)\", then = true }, \n          { else = false } ]\n\n[session.rcpt.errors]\ntotal = 5\nwait = \"1ms\"\n\n[session.auth]\nmechanisms = \"[plain, login, oauthbearer]\"\n\n[session.data]\nspam-filter = \"recipients[0] != 'robert@example.com'\"\n\n[session.data.add-headers]\ndelivered-to = false\n\n[queue]\npath = \"{TMP}\"\nhash = 64\n\n[report]\npath = \"{TMP}\"\nhash = 64\n\n[resolver]\ntype = \"system\"\n\n[queue.strategy]\nroute = [ { if = \"rcpt_domain == 'example.com'\", then = \"'local'\" }, \n             { if = \"contains(['remote.org', 'foobar.com', 'test.com', 'other_domain.com'], rcpt_domain)\", then = \"'mock-smtp'\" },\n             { else = \"'mx'\" } ]\n\n[queue.route.\"mock-smtp\"]\ntype = \"relay\"\naddress = \"localhost\"\nport = 9999\nprotocol = \"smtp\"\n\n[queue.route.\"mock-smtp\".tls]\nimplicit = false\nallow-invalid-certs = true\n\n[session.extensions]\nfuture-release = [ { if = \"!is_empty(authenticated_as)\", then = \"99999999d\"},\n                   { else = false } ]\n\n[certificate.default]\ncert = \"%{file:{CERT}}%\"\nprivate-key = \"%{file:{PK}}%\"\n\n[jmap.protocol.get]\nmax-objects = 100000\n\n[jmap.protocol.set]\nmax-objects = 100000\n\n[jmap.protocol.request]\nmax-concurrent = 8\n\n[jmap.protocol.upload]\nmax-size = 5000000\nmax-concurrent = 4\nttl = \"1m\"\n\n[jmap.protocol.upload.quota]\nfiles = 3\nsize = 50000\n\n[jmap.rate-limit]\naccount = \"1000/1m\"\nanonymous = \"100/1m\"\n\n[jmap.event-source]\nthrottle = \"500ms\"\n\n[jmap.web-sockets]\nthrottle = \"500ms\"\n\n[jmap.push]\nthrottle = \"500ms\"\nattempts.interval = \"500ms\"\n\n[email]\nauto-expunge = \"1s\"\n\n[changes]\nmax-history = \"1\"\n\n[store.\"auth\"]\ntype = \"sqlite\"\npath = \"{TMP}/auth.db\"\n\n[store.\"auth\".query]\nname = \"SELECT name, type, secret, description, quota FROM accounts WHERE name = ? AND active = true\"\nmembers = \"SELECT member_of FROM group_members WHERE name = ?\"\nrecipients = \"SELECT name FROM emails WHERE address = ?\"\nemails = \"SELECT address FROM emails WHERE name = ? AND type != 'list' ORDER BY type DESC, address ASC\"\nverify = \"SELECT address FROM emails WHERE address LIKE '%' || ? || '%' AND type = 'primary' ORDER BY address LIMIT 5\"\nexpand = \"SELECT p.address FROM emails AS p JOIN emails AS l ON p.name = l.name WHERE p.type = 'primary' AND l.address = ? AND l.type = 'list' ORDER BY p.address LIMIT 50\"\ndomains = \"SELECT 1 FROM emails WHERE address LIKE '%@' || ? LIMIT 1\"\n\n[imap.auth]\nallow-plain-text = true\n\n[oauth]\nkey = \"parerga_und_paralipomena\"\n\n[oauth.auth]\nmax-attempts = 1\n\n[oauth.expiry]\nuser-code = \"1s\"\ntoken = \"1s\"\nrefresh-token = \"3s\"\nrefresh-token-renew = \"2s\"\n\n[oauth.client-registration]\nanonymous = true\nrequire = true\n\n[oauth.oidc]\nsignature-key = '''-----BEGIN PRIVATE KEY-----\nMIIEuwIBADANBgkqhkiG9w0BAQEFAASCBKUwggShAgEAAoIBAQDMXJI1bL3z8gaF\nZe/6493VjL+jHkFMP2Pc7fLwRF1fhkuIdYTp69LabzrSEJCRCz0UI2NHqPOgtOta\n+zRHKAMr7c7Z6uKO0K+aXiQYHw4Y70uSG8CnmNl7kb4OM/CAcoO6fePmvBsyESfn\nTmkJ5bfHEZQFDQEAoDlDjtjxuwYsAQQVQXuAydi8j8pyTWKAJ1RDgnUT+HbOub7j\nJrQ7sPe6MPCjXv5N76v9RMHKktfYwRNMlkLkxImQU55+vlvghNztgFlIlJDFfNiy\nUQPV5FTEZJli9BzMoj1JQK3sZyV8WV0W1zN41QQ+glAAC6+K7iTDPRMINBSwbHyn\n6Lb9Q6U7AgMBAAECggEAB93qZ5xrhYgEFeoyKO4mUdGsu4qZyJB0zNeWGgdaXCfZ\nzC4l8zFM+R6osix0EY6lXRtC95+6h9hfFQNa5FWseupDzmIQiEnim1EowjWef87l\nEayi0nDRB8TjqZKjR/aLOUhzrPlXHKrKEUk/RDkacCiDklwz9S0LIfLOSXlByBDM\n/n/eczfX2gUATexMHSeIXs8vN2jpuiVv0r+FPXcRvqdzDZnYSzS8BJ9k6RYXVQ4o\nNzCbfqgFIpVryB7nHgSTrNX9G7299If8/dXmesXWSFEJvvDSSpcBoINKbfgSlrxd\n6ubjiotcEIBUSlbaanRrydwShhLHnXyupNAb7tlvyQKBgQDsIipSK4+H9FGl1rAk\nGg9DLJ7P/94sidhoq1KYnj/CxwGLoRq22khZEUYZkSvYXDu1Qkj9Avi3TRhw8uol\nl2SK1VylL5FQvTLKhWB7b2hjrUd5llMRgS3/NIdLhOgDMB7w3UxJnCA/df/Rj+dM\nWhkyS1f0x3t7XPLwWGurW0nJcwKBgQDdjhrNfabrK7OQvDpAvNJizuwZK9WUL7CD\nrR0V0MpDGYW12BTEOY6tUK6XZgiRitAXf4EkEI6R0Q0bFzwDDLrg7TvGdTuzNeg/\n8vm8IlRlOkrdihtHZI4uRB7Ytmz24vzywEBE0p6enA7v4oniscUks/KKmDGr0V90\nyT9gIVrjGQKBgQCjnWC5otlHGLDiOgm+WhgtMWOxN9dYAQNkMyF+Alinu4CEoVKD\nVGhA3sk1ufMpbW8pvw4X0dFIITFIQeift3DBCemxw23rBc2FqjkaDi3EszINO22/\neUTHyjvcxfCFFPi7aHsNnhJyJm7lY9Kegudmg/Ij93zGE7d5darVBuHvpQKBgBBY\nYovUgFMLR1UfPeD2zUKy52I4BKrJFemxBNtOKw3mPSIcTfPoFymcMTVENs+eARoq\nsvlZK1uAo8ni3e+Pqd3cQrOyhHQFPxwwrdH+amGJemp7vOV4erDZH7l3Q/S27Fhw\nbI1nSIKFGukBupB58wRxLiyha9C0QqmYC0/pRg5JAn8Rbj5tP26oVCXjZEfWJL8J\naxxSxsGA4Vol6i6LYnVgZG+1ez2rP8vUORo1lRzmdeP4o1BSJf9TPwXkuppE5J+t\nUZVKtYGlEn1RqwGNd8I9TiWvU84rcY9nsxlDR86xwKRWFvYqVOiGYtzRyewYRdjU\nrTs9aqB3v1+OVxGxR6Na\n-----END PRIVATE KEY-----\n'''\nsignature-algorithm = \"RS256\"\n\n[oauth.oidc-ignore]\nsignature-key = '''-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQggybcqc86ulFFiOon\nWiYrLO4z8/kmkqvA7wGElBok9IqhRANCAAQxZK68FnQtHC0eyh8CA05xRIvxhVHn\n0ymka6XBh9aFtW4wfeoKhTkSKjHc/zjh9Rr2dr3kvmYe80fMGhW4ycGA\n-----END PRIVATE KEY-----\n'''\nsignature-algorithm = \"ES256\"\n\n[session.extensions]\nexpn = true\nvrfy = true\n\n[spam-filter]\nenable = true\n\n[spam-filter.list]\nscores = {\"GTUBE_TEST\" = \"1000.0\"}\n\n[sharing]\nallow-directory-query = true\n\n[calendar.alarms]\nminimum-interval = \"1s\"\n\n[tracer.console]\ntype = \"console\"\nlevel = \"{LEVEL}\"\nmultiline = false\nansi = true\n#disabled-events = [\"network.*\", \"telemetry.webhook-error\"]\ndisabled-events = [\"network.*\", \"telemetry.webhook-error\", \"http.request-body\"]\n\n[webhook.\"test\"]\nurl = \"http://127.0.0.1:8821/hook\"\nevents = [\"auth.*\", \"delivery.dsn*\", \"message-ingest.*\", \"security.authentication-ban\"]\nsignature-key = \"ovos-moles\"\nthrottle = \"100ms\"\n\n[sieve.untrusted.scripts.\"common\"]\ncontents = '''\nrequire \"reject\";\n\nreject \"Rejected from a global script.\";\nstop;\n'''\n\"#;\n"
  },
  {
    "path": "tests/src/jmap/principal/availability.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{IntoJmapSet, JMAPTest, JmapUtils, calendar::event::*};\nuse calcard::jscalendar::JSCalendarProperty;\nuse jmap_proto::request::method::MethodObject;\nuse serde_json::json;\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Principal Availability tests...\");\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let john_id = john.id_string().to_string();\n    let jane_id = jane.id_string().to_string();\n\n    // Create test calendars\n    let response = john\n        .jmap_create(\n            MethodObject::Calendar,\n            [json!({\n                \"name\": \"Test Calendar\",\n                \"includeInAvailability\": \"all\"\n            })],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let calendar1_id = response.created(0).id().to_string();\n\n    // Create test events\n    let event_1 = test_jscalendar_1().with_property(\n        JSCalendarProperty::<Id>::CalendarIds,\n        [calendar1_id.as_str()].into_jmap_set(),\n    );\n    let event_2 = test_jscalendar_2().with_property(\n        JSCalendarProperty::<Id>::CalendarIds,\n        [calendar1_id.as_str()].into_jmap_set(),\n    );\n    let event_3 = test_jscalendar_3()\n        .with_property(\n            JSCalendarProperty::<Id>::CalendarIds,\n            [calendar1_id.as_str()].into_jmap_set(),\n        )\n        .with_property(\n            JSCalendarProperty::<Id>::Participants,\n            json!({\n              \"3f5bc8c0-c722-5345-b7d9-5a899db08a30\": {\n                \"calendarAddress\": \"mailto:jdoe@example.com\",\n                \"@type\": \"Participant\",\n                \"roles\": {\n                  \"attendee\": true,\n                  \"chair\": true\n                },\n                \"participationStatus\": \"accepted\"\n              }\n            }),\n        );\n    let response = john\n        .jmap_create(\n            MethodObject::CalendarEvent,\n            [event_1, event_2, event_3],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    let _event_1_id = response.created(0).id().to_string();\n    let _event_2_id = response.created(1).id().to_string();\n    let event_3_id = response.created(2).id().to_string();\n\n    // Jane should not have access to John's availability\n    let response = jane\n        .jmap_method_calls(json!([[\n            \"Principal/getAvailability\",\n            {\n                \"id\": &john_id,\n                \"utcStart\": \"2006-01-01T00:00:00Z\",\n                \"utcEnd\": \"2006-01-08T00:00:00Z\",\n            },\n            \"0\"\n        ]]))\n        .await;\n    response.list_array().assert_is_equal(json!([]));\n\n    // Grant Jane free/busy access\n    john.jmap_update(\n        MethodObject::Calendar,\n        [(\n            &calendar1_id,\n            json!({\n                \"shareWith\": {\n                   &jane_id : {\n                     \"mayReadFreeBusy\": true,\n                   }\n                }\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&calendar1_id);\n\n    // Jane should see John's availability now\n    let response = jane\n        .jmap_method_calls(json!([[\n            \"Principal/getAvailability\",\n            {\n                \"id\": &john_id,\n                \"utcStart\": \"2006-01-01T00:00:00Z\",\n                \"utcEnd\": \"2006-01-08T00:00:00Z\",\n            },\n            \"0\"\n        ]]))\n        .await;\n    response.list_array().assert_is_equal(json!([\n      {\n        \"utcStart\": \"2006-01-02T15:00:00Z\",\n        \"utcEnd\": \"2006-01-02T16:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      },\n      {\n        \"utcStart\": \"2006-01-02T17:00:00Z\",\n        \"utcEnd\": \"2006-01-02T18:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      },\n      {\n        \"utcStart\": \"2006-01-03T17:00:00Z\",\n        \"utcEnd\": \"2006-01-03T18:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      },\n      {\n        \"utcStart\": \"2006-01-04T15:00:00Z\",\n        \"utcEnd\": \"2006-01-04T16:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      },\n      {\n        \"utcStart\": \"2006-01-04T19:00:00Z\",\n        \"utcEnd\": \"2006-01-04T20:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      },\n      {\n        \"utcStart\": \"2006-01-05T17:00:00Z\",\n        \"utcEnd\": \"2006-01-05T18:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      },\n      {\n        \"utcStart\": \"2006-01-06T19:00:00Z\",\n        \"utcEnd\": \"2006-01-06T20:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      }\n    ]));\n\n    // Update availability to none\n    john.jmap_update(\n        MethodObject::Calendar,\n        [(\n            &calendar1_id,\n            json!({\n                \"includeInAvailability\": \"none\"\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&calendar1_id);\n\n    // Jane should not see any events now\n    let response = jane\n        .jmap_method_calls(json!([[\n            \"Principal/getAvailability\",\n            {\n                \"id\": &john_id,\n                \"utcStart\": \"2006-01-01T00:00:00Z\",\n                \"utcEnd\": \"2006-01-08T00:00:00Z\",\n            },\n            \"0\"\n        ]]))\n        .await;\n    response.list_array().assert_is_equal(json!([]));\n\n    // Update availability to attending\n    john.jmap_update(\n        MethodObject::Calendar,\n        [(\n            &calendar1_id,\n            json!({\n                \"includeInAvailability\": \"attending\"\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&calendar1_id);\n\n    // Jane should only see events where John is attending\n    let response = jane\n        .jmap_method_calls(json!([[\n            \"Principal/getAvailability\",\n            {\n                \"id\": &john_id,\n                \"utcStart\": \"2006-01-01T00:00:00Z\",\n                \"utcEnd\": \"2006-01-08T00:00:00Z\",\n            },\n            \"0\"\n        ]]))\n        .await;\n    response.list_array().assert_is_equal(json!([\n      {\n        \"utcStart\": \"2006-01-04T15:00:00Z\",\n        \"utcEnd\": \"2006-01-04T16:00:00Z\",\n        \"busyStatus\": \"confirmed\",\n        \"event\": null\n      }\n    ]));\n\n    // Update attending event to not attending\n    john.jmap_update(\n        MethodObject::CalendarEvent,\n        [(\n            &event_3_id,\n            json!({\n                \"participants/3f5bc8c0-c722-5345-b7d9-5a899db08a30/participationStatus\": \"declined\"\n            }),\n        )],\n        Vec::<(&str, &str)>::new(),\n    )\n    .await\n    .updated(&event_3_id);\n\n    // Jane should not see any events now\n    let response = jane\n        .jmap_method_calls(json!([[\n            \"Principal/getAvailability\",\n            {\n                \"id\": &john_id,\n                \"utcStart\": \"2006-01-01T00:00:00Z\",\n                \"utcEnd\": \"2006-01-08T00:00:00Z\",\n            },\n            \"0\"\n        ]]))\n        .await;\n    response.list_array().assert_is_equal(json!([]));\n\n    // Cleanup\n    john.destroy_all_calendars().await;\n    params.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/jmap/principal/get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::jmap::{JMAPTest, JmapUtils};\nuse jmap_proto::{object::principal::PrincipalProperty, request::method::MethodObject};\nuse serde_json::json;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Principal get/query tests...\");\n    let john = params.account(\"jdoe@example.com\");\n    let jane = params.account(\"jane.smith@example.com\");\n    let bill = params.account(\"bill@example.com\");\n    let sales = params.account(\"sales@example.com\");\n\n    let john_id = john.id_string();\n    let jane_id = jane.id_string();\n    let bill_id = bill.id_string();\n    let sales_id = sales.id_string();\n\n    // Validate session object capabilities\n    let response = john.jmap_session_object().await.into_inner();\n    response.assert_is_equal(json!({\n      \"capabilities\": {\n        \"urn:ietf:params:jmap:core\": {\n          \"maxSizeUpload\": 5000000,\n          \"maxConcurrentUpload\": 4,\n          \"maxSizeRequest\": 10000000,\n          \"maxConcurrentRequests\": 8,\n          \"maxCallsInRequest\": 16,\n          \"maxObjectsInGet\": 100000,\n          \"maxObjectsInSet\": 100000,\n          \"collationAlgorithms\": [\n            \"i;ascii-numeric\",\n            \"i;ascii-casemap\",\n            \"i;unicode-casemap\"\n          ]\n        },\n        \"urn:ietf:params:jmap:mail\": {},\n        \"urn:ietf:params:jmap:calendars\": {},\n        \"urn:ietf:params:jmap:calendars:parse\": {},\n        \"urn:ietf:params:jmap:contacts\": {},\n        \"urn:ietf:params:jmap:contacts:parse\": {},\n        \"urn:ietf:params:jmap:filenode\": {},\n        \"urn:ietf:params:jmap:principals\": {},\n        \"urn:ietf:params:jmap:principals:availability\": {},\n        \"urn:ietf:params:jmap:submission\": {},\n        \"urn:ietf:params:jmap:vacationresponse\": {},\n        \"urn:ietf:params:jmap:sieve\": {\n          \"implementation\": \"Stalwart v1.0.0\"\n        },\n        \"urn:ietf:params:jmap:blob\": {},\n        \"urn:ietf:params:jmap:quota\": {},\n        \"urn:ietf:params:jmap:websocket\": {\n          \"url\": \"wss://127.0.0.1:8899/jmap/ws\",\n          \"supportsPush\": true\n        }\n      },\n      \"accounts\": {\n        john_id: {\n          \"name\": \"jdoe@example.com\",\n          \"isPersonal\": true,\n          \"isReadOnly\": false,\n          \"accountCapabilities\": {\n            \"urn:ietf:params:jmap:mail\": {\n              \"maxMailboxesPerEmail\": null,\n              \"maxMailboxDepth\": 10,\n              \"maxSizeMailboxName\": 255,\n              \"maxSizeAttachmentsPerEmail\": 50000000,\n              \"emailQuerySortOptions\": [\n                \"receivedAt\",\n                \"size\",\n                \"from\",\n                \"to\",\n                \"subject\",\n                \"sentAt\",\n                \"hasKeyword\",\n                \"allInThreadHaveKeyword\",\n                \"someInThreadHaveKeyword\"\n              ],\n              \"mayCreateTopLevelMailbox\": true\n            },\n            \"urn:ietf:params:jmap:submission\": {\n              \"maxDelayedSend\": 2592000,\n              \"submissionExtensions\": {\n                \"FUTURERELEASE\": [],\n                \"SIZE\": [],\n                \"DSN\": [],\n                \"DELIVERYBY\": [],\n                \"MT-PRIORITY\": [\n                  \"MIXER\"\n                ],\n                \"REQUIRETLS\": []\n              }\n            },\n            \"urn:ietf:params:jmap:vacationresponse\": {},\n            \"urn:ietf:params:jmap:contacts\": {\n              \"maxAddressBooksPerCard\": null,\n              \"mayCreateAddressBook\": true\n            },\n            \"urn:ietf:params:jmap:contacts:parse\": {},\n            \"urn:ietf:params:jmap:calendars\": {\n              \"maxCalendarsPerEvent\": null,\n              \"minDateTime\": \"0001-01-01T00:00:00Z\",\n              \"maxDateTime\": \"65534-12-31T23:59:59Z\",\n              \"maxExpandedQueryDuration\": \"P52W1D\",\n              \"maxParticipantsPerEvent\": 20,\n              \"mayCreateCalendar\": true\n            },\n            \"urn:ietf:params:jmap:calendars:parse\": {},\n            \"urn:ietf:params:jmap:websocket\": {},\n            \"urn:ietf:params:jmap:sieve\": {\n              \"maxSizeScriptName\": 512,\n              \"maxSizeScript\": 1048576,\n              \"maxNumberScripts\": 100,\n              \"maxNumberRedirects\": 1,\n              \"sieveExtensions\": [\n                \"body\",\n                \"comparator-elbonia\",\n                \"comparator-i;ascii-casemap\",\n                \"comparator-i;ascii-numeric\",\n                \"comparator-i;octet\",\n                \"convert\",\n                \"copy\",\n                \"date\",\n                \"duplicate\",\n                \"editheader\",\n                \"enclose\",\n                \"encoded-character\",\n                \"enotify\",\n                \"envelope\",\n                \"envelope-deliverby\",\n                \"envelope-dsn\",\n                \"environment\",\n                \"ereject\",\n                \"extlists\",\n                \"extracttext\",\n                \"fcc\",\n                \"fileinto\",\n                \"foreverypart\",\n                \"ihave\",\n                \"imap4flags\",\n                \"imapsieve\",\n                \"include\",\n                \"index\",\n                \"mailbox\",\n                \"mailboxid\",\n                \"mboxmetadata\",\n                \"mime\",\n                \"redirect-deliverby\",\n                \"redirect-dsn\",\n                \"regex\",\n                \"reject\",\n                \"relational\",\n                \"replace\",\n                \"servermetadata\",\n                \"spamtest\",\n                \"spamtestplus\",\n                \"special-use\",\n                \"subaddress\",\n                \"vacation\",\n                \"vacation-seconds\",\n                \"variables\",\n                \"virustest\"\n              ],\n              \"notificationMethods\": [\n                \"mailto\"\n              ],\n              \"externalLists\": null\n            },\n            \"urn:ietf:params:jmap:blob\": {\n              \"maxSizeBlobSet\": 7499488,\n              \"maxDataSources\": 16,\n              \"supportedTypeNames\": [\n                \"Email\",\n                \"Thread\",\n                \"SieveScript\"\n              ],\n              \"supportedDigestAlgorithms\": [\n                \"sha\",\n                \"sha-256\",\n                \"sha-512\"\n              ]\n            },\n            \"urn:ietf:params:jmap:quota\": {},\n            \"urn:ietf:params:jmap:principals\": {\n              \"currentUserPrincipalId\": john_id\n            },\n            \"urn:ietf:params:jmap:principals:availability\": {\n              \"maxAvailabilityDuration\": \"P52W1D\",\n            },\n            \"urn:ietf:params:jmap:filenode\": {\n              \"maxFileNodeDepth\": null,\n              \"maxSizeFileNodeName\": 255,\n              \"fileNodeQuerySortOptions\": [],\n              \"mayCreateTopLevelFileNode\": true\n            }\n          }\n        }\n      },\n      \"primaryAccounts\": {\n        \"urn:ietf:params:jmap:mail\": john_id,\n        \"urn:ietf:params:jmap:submission\": john_id,\n        \"urn:ietf:params:jmap:vacationresponse\": john_id,\n        \"urn:ietf:params:jmap:contacts\": john_id,\n        \"urn:ietf:params:jmap:contacts:parse\": john_id,\n        \"urn:ietf:params:jmap:calendars\": john_id,\n        \"urn:ietf:params:jmap:calendars:parse\": john_id,\n        \"urn:ietf:params:jmap:websocket\": john_id,\n        \"urn:ietf:params:jmap:sieve\": john_id,\n        \"urn:ietf:params:jmap:blob\": john_id,\n        \"urn:ietf:params:jmap:quota\": john_id,\n        \"urn:ietf:params:jmap:principals\": john_id,\n        \"urn:ietf:params:jmap:principals:availability\": john_id,\n        \"urn:ietf:params:jmap:filenode\": john_id\n      },\n      \"username\": \"jdoe@example.com\",\n      \"apiUrl\": \"https://127.0.0.1:8899/jmap/\",\n      \"downloadUrl\":\n      \"https://127.0.0.1:8899/jmap/download/{accountId}/{blobId}/{name}?accept={type}\",\n      \"uploadUrl\":\n      \"https://127.0.0.1:8899/jmap/upload/{accountId}/\",\n      \"eventSourceUrl\":\n      \"https://127.0.0.1:8899/jmap/eventsource/?types={types}&closeafter={closeafter}&ping={ping}\",\n      \"state\": response.text_field(\"state\")\n    }));\n\n    // Obtain principal ids for Jane, Bill and the sales group\n    let response = john\n        .jmap_query(\n            MethodObject::Principal,\n            [(\"email\", \"john.doe@example.com\")],\n            [\"name\"],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    assert_eq!(response.ids().collect::<Vec<_>>(), [john_id]);\n    let response = john\n        .jmap_query(\n            MethodObject::Principal,\n            [(\"name\", \"bill@example.com\")],\n            [\"name\"],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    assert_eq!(response.ids().collect::<Vec<_>>(), [bill_id]);\n    let response = john\n        .jmap_query(\n            MethodObject::Principal,\n            [(\"accountIds\", [jane_id])],\n            [\"name\"],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    assert_eq!(response.ids().collect::<Vec<_>>(), [jane_id]);\n    let response = john\n        .jmap_query(\n            MethodObject::Principal,\n            [(\"text\", \"sales group\")],\n            [\"name\"],\n            Vec::<(&str, &str)>::new(),\n        )\n        .await;\n    assert_eq!(response.ids().collect::<Vec<_>>(), [sales_id]);\n\n    // Validate principal contents\n    let response = john\n        .jmap_get(\n            MethodObject::Principal,\n            [\n                PrincipalProperty::Id,\n                PrincipalProperty::Type,\n                PrincipalProperty::Email,\n                PrincipalProperty::Description,\n                PrincipalProperty::Name,\n                PrincipalProperty::Timezone,\n                PrincipalProperty::Capabilities,\n                PrincipalProperty::Accounts,\n            ],\n            [john_id, jane_id, bill_id, sales_id],\n        )\n        .await;\n    let list = response.list();\n    assert_eq!(list.len(), 4);\n\n    list[0].assert_is_equal(json!({\n      \"id\": john_id,\n      \"type\": \"individual\",\n      \"email\": \"jdoe@example.com\",\n      \"description\": \"John Doe\",\n      \"name\": \"jdoe@example.com\",\n      \"timezone\": null,\n      \"capabilities\": {\n        \"urn:ietf:params:jmap:mail\": {},\n        \"urn:ietf:params:jmap:contacts\": {},\n        \"urn:ietf:params:jmap:calendars\": {},\n        \"urn:ietf:params:jmap:filenode\": {},\n        \"urn:ietf:params:jmap:principals\": {}\n      },\n      \"accounts\": {\n        john_id: {\n          \"urn:ietf:params:jmap:mail\": {},\n          \"urn:ietf:params:jmap:contacts\": {},\n          \"urn:ietf:params:jmap:calendars\": {\n            \"accountId\": john_id,\n            \"mayGetAvailability\": true,\n            \"mayShareWith\": true,\n            \"calendarAddress\": \"mailto:jdoe@example.com\"\n          },\n          \"urn:ietf:params:jmap:filenode\": {},\n          \"urn:ietf:params:jmap:principals\": {},\n          \"urn:ietf:params:jmap:principals:owner\": {\n            \"accountIdForPrincipal\": john_id,\n            \"principalId\": john_id\n          }\n        }\n      }\n    }));\n    list[1].assert_is_equal(json!({\n      \"id\": jane_id,\n      \"type\": \"individual\",\n      \"email\": \"jane.smith@example.com\",\n      \"description\": \"Jane Smith\",\n      \"name\": \"jane.smith@example.com\",\n      \"timezone\": null,\n      \"capabilities\": {\n        \"urn:ietf:params:jmap:mail\": {},\n        \"urn:ietf:params:jmap:contacts\": {},\n        \"urn:ietf:params:jmap:calendars\": {},\n        \"urn:ietf:params:jmap:filenode\": {},\n        \"urn:ietf:params:jmap:principals\": {}\n      },\n      \"accounts\": {\n        jane_id: {\n          \"urn:ietf:params:jmap:mail\": {},\n          \"urn:ietf:params:jmap:contacts\": {},\n          \"urn:ietf:params:jmap:calendars\": {\n            \"accountId\": jane_id,\n            \"mayGetAvailability\": true,\n            \"mayShareWith\": true,\n            \"calendarAddress\": \"mailto:jane.smith@example.com\"\n          },\n          \"urn:ietf:params:jmap:filenode\": {},\n          \"urn:ietf:params:jmap:principals\": {},\n          \"urn:ietf:params:jmap:principals:owner\": {\n            \"accountIdForPrincipal\": jane_id,\n            \"principalId\": jane_id\n          }\n        }\n      }\n    }));\n    list[2].assert_is_equal(json!({\n      \"id\": bill_id,\n      \"type\": \"individual\",\n      \"email\": \"bill@example.com\",\n      \"description\": \"Bill Foobar\",\n      \"name\": \"bill@example.com\",\n      \"timezone\": null,\n      \"capabilities\": {\n        \"urn:ietf:params:jmap:mail\": {},\n        \"urn:ietf:params:jmap:contacts\": {},\n        \"urn:ietf:params:jmap:calendars\": {},\n        \"urn:ietf:params:jmap:filenode\": {},\n        \"urn:ietf:params:jmap:principals\": {}\n      },\n      \"accounts\": {\n        bill_id: {\n          \"urn:ietf:params:jmap:mail\": {},\n          \"urn:ietf:params:jmap:contacts\": {},\n          \"urn:ietf:params:jmap:calendars\": {\n            \"accountId\": bill_id,\n            \"mayGetAvailability\": true,\n            \"mayShareWith\": true,\n            \"calendarAddress\": \"mailto:bill@example.com\"\n          },\n          \"urn:ietf:params:jmap:filenode\": {},\n          \"urn:ietf:params:jmap:principals\": {},\n          \"urn:ietf:params:jmap:principals:owner\": {\n            \"accountIdForPrincipal\": bill_id,\n            \"principalId\": bill_id\n          }\n        }\n      }\n    }));\n    list[3].assert_is_equal(json!({\n      \"id\": sales_id,\n      \"type\": \"group\",\n      \"email\": \"sales@example.com\",\n      \"description\": \"Sales Group\",\n      \"name\": \"sales@example.com\",\n      \"timezone\": null,\n      \"capabilities\": {\n        \"urn:ietf:params:jmap:mail\": {},\n        \"urn:ietf:params:jmap:contacts\": {},\n        \"urn:ietf:params:jmap:calendars\": {},\n        \"urn:ietf:params:jmap:filenode\": {},\n        \"urn:ietf:params:jmap:principals\": {}\n      },\n      \"accounts\": {\n        sales_id: {\n          \"urn:ietf:params:jmap:mail\": {},\n          \"urn:ietf:params:jmap:contacts\": {},\n          \"urn:ietf:params:jmap:calendars\": {\n            \"accountId\": sales_id,\n            \"mayGetAvailability\": true,\n            \"mayShareWith\": true,\n            \"calendarAddress\": \"mailto:sales@example.com\"\n          },\n          \"urn:ietf:params:jmap:filenode\": {},\n          \"urn:ietf:params:jmap:principals\": {},\n          \"urn:ietf:params:jmap:principals:owner\": {\n            \"accountIdForPrincipal\": sales_id,\n            \"principalId\": sales_id\n          }\n        }\n      }\n    }));\n}\n"
  },
  {
    "path": "tests/src/jmap/principal/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod availability;\npub mod get;\n"
  },
  {
    "path": "tests/src/jmap/server/enterprise.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: LicenseRef-SEL\n *\n * This file is subject to the Stalwart Enterprise License Agreement (SEL) and\n * is NOT open source software.\n *\n */\n\nuse crate::{\n    AssertConfig,\n    directory::internal::TestInternalDirectory,\n    imap::{ImapConnection, Type},\n    jmap::{\n        JMAPTest, ManagementApi,\n        mail::delivery::{AssertResult, SmtpConnection},\n        server::List,\n        wait_for_index,\n    },\n};\nuse common::{\n    Core, Server,\n    config::telemetry::{StoreTracer, TelemetrySubscriberType},\n    core::BuildServer,\n    enterprise::{\n        Enterprise, MetricStore, TraceStore, Undelete, config::parse_metric_alerts,\n        license::LicenseKey,\n    },\n    telemetry::{\n        metrics::store::{Metric, MetricsStore, SharedMetricHistory},\n        tracers::store::TracingStore,\n    },\n};\nuse directory::{QueryBy, backend::internal::manage::ManageDirectory};\nuse http::management::{\n    enterprise::undelete::{\n        DeletedBlobResponse, DeletedItemResponse, UndeleteRequest, UndeleteResponse,\n    },\n    stores::destroy_account_data,\n};\nuse imap_proto::ResponseType;\nuse nlp::language::Language;\nuse std::{sync::Arc, time::Duration};\nuse store::{\n    rand::{self, Rng},\n    search::{\n        SearchField, SearchFilter, SearchOperator, SearchQuery, SearchValue, TracingSearchField,\n    },\n    write::{SearchIndex, now},\n};\nuse trc::{\n    ipc::{bitset::Bitset, subscriber::SubscriberBuilder},\n    *,\n};\nuse utils::config::{Config, cron::SimpleCron};\n\nconst METRICS_CONFIG: &str = r#\"\n[metrics.alerts.expected]\nenable = true\ncondition = \"domain_count > 1 && cluster_publisher_error > 3\"\n\n[metrics.alerts.expected.notify.event]\nenable = true\nmessage = \"Yikes! Found %{cluster.publisher-error}% cluster errors!\"\n\n[metrics.alerts.expected.notify.email]\nenable = true\nfrom-name = \"Alert Subsystem\"\nfrom-addr = \"alert@example.com\"\nto = [\"jdoe@example.com\"]\nsubject = \"Found %{cluster.publisher-error}% cluster errors\"\nbody = \"Sorry for the bad news, but we found %{domain.count}% domains and %{cluster.publisher-error}% cluster errors.\"\n\n[metrics.alerts.unexpected]\nenable = true\ncondition = \"domain_count < 1 || cluster_publisher_error < 3\"\n\n[metrics.alerts.unexpected.notify.event]\nenable = true\nmessage = \"this should not have happened\"\n\n\"#;\n\nconst RAW_MESSAGE: &str = \"From: john@example.com\nTo: john@example.com\nSubject: undelete test\n\ntest\n\";\n\npub async fn test(params: &mut JMAPTest) {\n    // Enable Enterprise\n    println!(\"Running Enterprise tests...\");\n    let mut core = params.server.inner.shared_core.load_full().as_ref().clone();\n    let mut config = Config::new(METRICS_CONFIG).unwrap();\n    core.enterprise = Enterprise {\n        license: LicenseKey {\n            valid_to: now() + 3600,\n            valid_from: now() - 3600,\n            domain: String::new(),\n            accounts: 100,\n        },\n        undelete: Undelete {\n            retention: Duration::from_secs(2),\n        }\n        .into(),\n        trace_store: TraceStore {\n            retention: Some(Duration::from_secs(1)),\n            store: core.storage.data.clone(),\n        }\n        .into(),\n        metrics_store: MetricStore {\n            retention: Some(Duration::from_secs(1)),\n            store: core.storage.data.clone(),\n            interval: SimpleCron::Day { hour: 0, minute: 0 },\n        }\n        .into(),\n        metrics_alerts: parse_metric_alerts(&mut config),\n        logo_url: None,\n        ai_apis: Default::default(),\n        spam_filter_llm: None,\n        template_calendar_alarm: None,\n        template_scheduling_email: None,\n        template_scheduling_web: None,\n    }\n    .into();\n    config.assert_no_errors();\n    assert_ne!(core.enterprise.as_ref().unwrap().metrics_alerts.len(), 0);\n    params.server.inner.shared_core.store(core.into());\n    assert!(\n        params\n            .server\n            .inner\n            .shared_core\n            .load()\n            .is_enterprise_edition()\n    );\n\n    // Create test account\n    let server = params.server.inner.build_server();\n    let account_id = server\n        .store()\n        .create_test_user(\n            \"jdoe@example.com\",\n            \"12345\",\n            \"John Doe\",\n            &[\"jdoe@example.com\"],\n        )\n        .await;\n\n    alerts(&server).await;\n    undelete(params).await;\n    tracing(params).await;\n    metrics(params).await;\n\n    // Delete test account\n    server\n        .store()\n        .delete_principal(QueryBy::Id(account_id))\n        .await\n        .unwrap();\n    destroy_account_data(&server, account_id, true)\n        .await\n        .unwrap();\n    params.assert_is_empty().await;\n\n    params.server.inner.shared_core.store(\n        params\n            .server\n            .inner\n            .shared_core\n            .load_full()\n            .as_ref()\n            .clone()\n            .enable_enterprise()\n            .into(),\n    );\n}\n\npub trait EnterpriseCore {\n    fn enable_enterprise(self) -> Self;\n}\n\nimpl EnterpriseCore for Core {\n    fn enable_enterprise(mut self) -> Self {\n        self.enterprise = Enterprise {\n            license: LicenseKey {\n                valid_to: now() + 3600,\n                valid_from: now() - 3600,\n                domain: String::new(),\n                accounts: 100,\n            },\n            undelete: None,\n            trace_store: None,\n            metrics_store: None,\n            metrics_alerts: vec![],\n            logo_url: None,\n            ai_apis: Default::default(),\n            spam_filter_llm: None,\n            template_calendar_alarm: None,\n            template_scheduling_email: None,\n            template_scheduling_web: None,\n        }\n        .into();\n        self\n    }\n}\n\nasync fn alerts(server: &Server) {\n    // Make sure the required metrics are set to 0\n    assert_eq!(\n        Collector::read_event_metric(EventType::Cluster(ClusterEvent::PublisherError).id()),\n        0\n    );\n    assert_eq!(Collector::read_metric(MetricType::DomainCount), 0.0);\n    assert_eq!(\n        Collector::read_event_metric(EventType::Telemetry(TelemetryEvent::Alert).id()),\n        0\n    );\n\n    // Increment metrics to trigger alerts\n    Collector::update_event_counter(EventType::Cluster(ClusterEvent::PublisherError), 5);\n    Collector::update_gauge(MetricType::DomainCount, 3);\n\n    // Make sure the values were set\n    assert_eq!(\n        Collector::read_event_metric(EventType::Cluster(ClusterEvent::PublisherError).id()),\n        5\n    );\n    assert_eq!(Collector::read_metric(MetricType::DomainCount), 3.0);\n\n    // Process alerts\n    let message = server.process_alerts().await.unwrap().pop().unwrap();\n    assert_eq!(message.from, \"alert@example.com\");\n    assert_eq!(message.to, vec![\"jdoe@example.com\".to_string()]);\n    let body = String::from_utf8(message.body).unwrap();\n    assert!(\n        body.contains(\"Sorry for the bad news, but we found 3 domains and 5 cluster errors.\"),\n        \"{body:?}\"\n    );\n    assert!(body.contains(\"Subject: Found 5 cluster errors\"), \"{body:?}\");\n    assert!(\n        body.contains(\"From: \\\"Alert Subsystem\\\" <alert@example.com>\"),\n        \"{body:?}\"\n    );\n    assert!(body.contains(\"To: <jdoe@example.com>\"), \"{body:?}\");\n\n    // Make sure the event was triggered\n    assert_eq!(\n        Collector::read_event_metric(EventType::Telemetry(TelemetryEvent::Alert).id()),\n        1\n    );\n}\n\nasync fn tracing(params: &mut JMAPTest) {\n    // Enable tracing\n    let store = params.server.core.storage.data.clone();\n    let query = params.server.core.storage.fts.clone();\n    TelemetrySubscriberType::StoreTracer(StoreTracer {\n        store: store.clone(),\n    })\n    .spawn(\n        SubscriberBuilder::new(\"store-tracer\".to_string()).with_interests(Box::new(Bitset::all())),\n        true,\n    );\n\n    // Make sure there are no span entries in the db\n    store\n        .purge_spans(Duration::from_secs(0), Some(&query))\n        .await\n        .unwrap();\n    assert_eq!(\n        query\n            .query_global(SearchQuery::new(SearchIndex::Tracing).with_filters(vec![\n                SearchFilter::Operator {\n                    field: SearchField::Tracing(TracingSearchField::EventType),\n                    op: SearchOperator::Equal,\n                    value: SearchValue::Uint(EventType::Smtp(SmtpEvent::ConnectionStart).code())\n                }\n            ]))\n            .await\n            .unwrap(),\n        Vec::<u64>::new()\n    );\n\n    // Send an email\n    let mut lmtp = SmtpConnection::connect().await;\n    lmtp.ingest(\n        \"bill@example.com\",\n        &[\"jdoe@example.com\"],\n        concat!(\n            \"From: bill@example.com\\r\\n\",\n            \"To: jdoe@example.com\\r\\n\",\n            \"Subject: TPS Report\\r\\n\",\n            \"X-Spam-Status: No\\r\\n\",\n            \"\\r\\n\",\n            \"I'm going to need those TPS reports ASAP. \",\n            \"So, if you could do that, that'd be great.\"\n        ),\n    )\n    .await;\n    lmtp.quit().await;\n    tokio::time::sleep(Duration::from_millis(300)).await;\n\n    params.server.notify_task_queue();\n    wait_for_index(&params.server).await;\n\n    // Purge should not delete anything at this point\n    store\n        .purge_spans(Duration::from_secs(2), Some(&query))\n        .await\n        .unwrap();\n\n    // There should be a span entry in the db\n    for span_type in [\n        EventType::Delivery(DeliveryEvent::AttemptStart),\n        EventType::Smtp(SmtpEvent::ConnectionStart),\n    ] {\n        let spans = query\n            .query_global(SearchQuery::new(SearchIndex::Tracing).with_filters(vec![\n                SearchFilter::Operator {\n                    field: SearchField::Tracing(TracingSearchField::EventType),\n                    op: SearchOperator::Equal,\n                    value: SearchValue::Uint(span_type.code()),\n                },\n            ]))\n            .await\n            .unwrap();\n        assert_eq!(spans.len(), 1, \"{span_type:?}\");\n        assert_eq!(\n            store.get_span(spans[0]).await.unwrap()[0].inner.typ,\n            span_type\n        );\n    }\n\n    // Try searching\n    for keyword in [\"bill@example.com\", \"jdoe@example.com\", \"example.com\"] {\n        let spans = query\n            .query_global(SearchQuery::new(SearchIndex::Tracing).with_filters(vec![\n                SearchFilter::Operator {\n                    field: SearchField::Tracing(TracingSearchField::Keywords),\n                    op: SearchOperator::Equal,\n                    value: SearchValue::Text {\n                        value: keyword.to_string(),\n                        language: Language::None,\n                    },\n                },\n            ]))\n            .await\n            .unwrap();\n\n        assert_eq!(spans.len(), 2, \"keyword: {keyword}\");\n        assert!(spans[0] != spans[1], \"keyword: {keyword}\");\n    }\n\n    // Purge should delete the span entries\n    tokio::time::sleep(Duration::from_millis(800)).await;\n    store\n        .purge_spans(Duration::from_secs(1), Some(&query))\n        .await\n        .unwrap();\n\n    assert_eq!(\n        query\n            .query_global(SearchQuery::new(SearchIndex::Tracing).with_filters(vec![\n                SearchFilter::Operator {\n                    field: SearchField::Id,\n                    op: SearchOperator::GreaterThan,\n                    value: SearchValue::Uint(0),\n                },\n            ]))\n            .await\n            .unwrap(),\n        Vec::<u64>::new()\n    );\n}\n\nasync fn metrics(params: &mut JMAPTest) {\n    // Make sure there are no span entries in the db\n    let store = params.server.core.storage.data.clone();\n    assert_eq!(\n        store.query_metrics(0, u64::MAX).await.unwrap(),\n        Vec::<Metric<EventType, MetricType, u64>>::new()\n    );\n\n    insert_test_metrics(params.server.core.clone()).await;\n\n    let total = store.query_metrics(0, u64::MAX).await.unwrap();\n    assert!(!total.is_empty(), \"{total:?}\");\n\n    store.purge_metrics(Duration::from_secs(0)).await.unwrap();\n    assert_eq!(\n        store.query_metrics(0, u64::MAX).await.unwrap(),\n        Vec::<Metric<EventType, MetricType, u64>>::new()\n    );\n}\n\nasync fn undelete(params: &mut JMAPTest) {\n    // Authenticate\n    let mut imap = ImapConnection::connect(b\"_x \").await;\n    imap.authenticate(\"jdoe@example.com\", \"12345\").await;\n\n    // Insert test message\n    imap.send(\"STATUS INBOX (MESSAGES)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 0\");\n    imap.send(&format!(\"APPEND INBOX {{{}}}\", RAW_MESSAGE.len()))\n        .await;\n    imap.assert_read(Type::Continuation, ResponseType::Ok).await;\n    imap.send_untagged(RAW_MESSAGE).await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Make sure the message is there\n    imap.send(\"STATUS INBOX (MESSAGES)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 1\");\n    imap.send(\"SELECT INBOX\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Fetch message body\n    imap.send(\"FETCH 1 BODY[]\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"Subject: undelete test\");\n\n    // Delete and expunge message\n    imap.send(\"STORE 1 +FLAGS (\\\\Deleted)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"EXPUNGE\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Logout and reconnect\n    imap.send(\"LOGOUT\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    let mut imap = ImapConnection::connect(b\"_x \").await;\n    imap.authenticate(\"jdoe@example.com\", \"12345\").await;\n\n    // Make sure the message is gone\n    imap.send(\"STATUS INBOX (MESSAGES)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 0\");\n\n    // Query undelete API\n    let api = ManagementApi::new(8899, \"admin\", \"secret\");\n    api.get::<serde_json::Value>(\"/api/store/purge/account/jdoe@example.com\")\n        .await\n        .unwrap();\n    wait_for_index(&params.server).await;\n    tokio::time::sleep(Duration::from_millis(200)).await;\n    let deleted = api\n        .get::<List<DeletedBlobResponse>>(\"/api/store/undelete/jdoe@example.com\")\n        .await\n        .unwrap()\n        .unwrap_data()\n        .items;\n    assert_eq!(deleted.len(), 1);\n    let deleted = deleted.into_iter().next().unwrap();\n    match deleted.item {\n        DeletedItemResponse::Email { from, subject, .. } => {\n            assert_eq!(subject.as_ref(), \"undelete test\");\n            assert_eq!(from.as_ref(), \"john@example.com\");\n        }\n        other => {\n            panic!(\"Unexpected deleted item response: {:?}\", other);\n        }\n    }\n\n    // Undelete\n    let result = api\n        .post::<Vec<UndeleteResponse>>(\n            \"/api/store/undelete/jdoe@example.com\",\n            &vec![UndeleteRequest {\n                hash: deleted.hash,\n                collection: \"email\".to_string(),\n                time: deleted.deleted_at,\n                cancel_deletion: deleted.expires_at.into(),\n            }],\n        )\n        .await\n        .unwrap()\n        .unwrap_data();\n    assert_eq!(result, vec![UndeleteResponse::Success]);\n\n    // Make sure the message is back\n    imap.send(\"STATUS INBOX (MESSAGES)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 1\");\n\n    imap.send(\"SELECT INBOX\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n\n    // Fetch message body\n    imap.send(\"FETCH 1 BODY[]\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"Subject: undelete test\");\n}\n\npub async fn insert_test_metrics(core: Arc<Core>) {\n    let store = core.storage.data.clone();\n    store.purge_metrics(Duration::from_secs(0)).await.unwrap();\n    let mut start_time = now() - (90 * 24 * 60 * 60);\n    let timestamp = now();\n    let history = SharedMetricHistory::default();\n\n    while start_time < timestamp {\n        for event_type in [\n            EventType::Smtp(SmtpEvent::ConnectionStart),\n            EventType::Imap(ImapEvent::ConnectionStart),\n            EventType::Pop3(Pop3Event::ConnectionStart),\n            EventType::ManageSieve(ManageSieveEvent::ConnectionStart),\n            EventType::Http(HttpEvent::ConnectionStart),\n            EventType::Delivery(DeliveryEvent::AttemptStart),\n            EventType::Queue(QueueEvent::QueueMessage),\n            EventType::Queue(QueueEvent::QueueMessageAuthenticated),\n            EventType::Queue(QueueEvent::QueueDsn),\n            EventType::Queue(QueueEvent::QueueReport),\n            EventType::MessageIngest(MessageIngestEvent::Ham),\n            EventType::MessageIngest(MessageIngestEvent::Spam),\n            EventType::Auth(AuthEvent::Failed),\n            EventType::Security(SecurityEvent::AuthenticationBan),\n            EventType::Security(SecurityEvent::ScanBan),\n            EventType::Security(SecurityEvent::AbuseBan),\n            EventType::Security(SecurityEvent::LoiterBan),\n            EventType::Security(SecurityEvent::IpBlocked),\n            EventType::IncomingReport(IncomingReportEvent::DmarcReport),\n            EventType::IncomingReport(IncomingReportEvent::DmarcReportWithWarnings),\n            EventType::IncomingReport(IncomingReportEvent::TlsReport),\n            EventType::IncomingReport(IncomingReportEvent::TlsReportWithWarnings),\n        ] {\n            // Generate a random value between 0 and 100\n            Collector::update_event_counter(event_type, rand::rng().random_range(0..=100))\n        }\n\n        Collector::update_gauge(MetricType::QueueCount, rand::rng().random_range(0..=1000));\n        Collector::update_gauge(\n            MetricType::ServerMemory,\n            rand::rng().random_range(100 * 1024 * 1024..=300 * 1024 * 1024),\n        );\n\n        for metric_type in [\n            MetricType::MessageIngestionTime,\n            MetricType::MessageFtsIndexTime,\n            MetricType::DeliveryTime,\n            MetricType::DnsLookupTime,\n        ] {\n            Collector::update_histogram(metric_type, rand::rng().random_range(2..=1000))\n        }\n        Collector::update_histogram(\n            MetricType::DeliveryTotalTime,\n            rand::rng().random_range(1000..=5000),\n        );\n\n        store\n            .write_metrics(core.clone(), start_time, history.clone())\n            .await\n            .unwrap();\n        start_time += 60 * 60;\n    }\n}\n"
  },
  {
    "path": "tests/src/jmap/server/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod enterprise;\npub mod purge;\npub mod webhooks;\n\n#[derive(serde::Deserialize, Debug)]\n#[allow(dead_code)]\npub(crate) struct List<T> {\n    pub items: Vec<T>,\n    pub total: usize,\n}\n"
  },
  {
    "path": "tests/src/jmap/server/purge.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    imap::{AssertResult, ImapConnection, Type},\n    jmap::{JMAPTest, wait_for_index},\n};\nuse ahash::AHashSet;\nuse common::Server;\nuse directory::{QueryBy, backend::internal::manage::ManageDirectory};\nuse email::{\n    cache::{MessageCacheFetch, email::MessageCacheAccess},\n    mailbox::{INBOX_ID, JUNK_ID, TRASH_ID},\n    message::delete::EmailDeletion,\n};\nuse http::management::stores::destroy_account_data;\nuse imap_proto::ResponseType;\nuse store::{IterateParams, LogKey, U32_LEN, U64_LEN, write::key::DeserializeBigEndian};\nuse types::id::Id;\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running purge tests...\");\n    let server = params.server.clone();\n    let inbox_id = Id::from(INBOX_ID).to_string();\n    let trash_id = Id::from(TRASH_ID).to_string();\n    let junk_id = Id::from(JUNK_ID).to_string();\n    let account = params.account(\"jdoe@example.com\");\n    let client = account.client();\n\n    let mut imap = ImapConnection::connect(b\"_x \").await;\n    imap.assert_read(Type::Untagged, ResponseType::Ok).await;\n    imap.send(\"LOGIN \\\"jdoe@example.com\\\" \\\"12345\\\"\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok).await;\n    imap.send(\"STATUS INBOX (UIDNEXT MESSAGES UNSEEN)\").await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"MESSAGES 0\");\n\n    // Create test messages\n    let mut message_ids = Vec::new();\n    let mut pass = 0;\n    let mut changes = AHashSet::new();\n\n    loop {\n        pass += 1;\n        for folder_id in [&inbox_id, &trash_id, &junk_id] {\n            message_ids.push(\n                client\n                    .email_import(\n                        format!(\n                            concat!(\n                                \"From: bill@example.com\\r\\n\",\n                                \"To: jdoe@example.com\\r\\n\",\n                                \"Subject: TPS Report #{} {}\\r\\n\",\n                                \"\\r\\n\",\n                                \"I'm going to need those TPS reports ASAP. \",\n                                \"So, if you could do that, that'd be great.\"\n                            ),\n                            pass, folder_id\n                        )\n                        .into_bytes(),\n                        [folder_id],\n                        None::<Vec<&str>>,\n                        None,\n                    )\n                    .await\n                    .unwrap()\n                    .take_id(),\n            );\n        }\n\n        if pass == 1 {\n            let (changes_, is_truncated) = get_changes(&server).await;\n            assert!(!is_truncated);\n            changes = changes_;\n            tokio::time::sleep(std::time::Duration::from_secs(1)).await;\n        } else {\n            break;\n        }\n    }\n\n    // Check IMAP status\n    imap.send(\"LIST \\\"\\\" \\\"*\\\" RETURN (STATUS (MESSAGES))\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"INBOX\\\" (MESSAGES 2)\")\n        .assert_contains(\"\\\"Deleted Items\\\" (MESSAGES 2)\")\n        .assert_contains(\"\\\"Junk Mail\\\" (MESSAGES 2)\");\n\n    // Make sure both messages and changes are present\n    assert_eq!(\n        server\n            .get_cached_messages(account.id().document_id())\n            .await\n            .unwrap()\n            .emails\n            .items\n            .len(),\n        6\n    );\n\n    // Purge junk/trash messages and old changes\n    server.purge_account(account.id().document_id()).await;\n    let cache = server\n        .get_cached_messages(account.id().document_id())\n        .await\n        .unwrap();\n\n    // Only 4 messages should remain\n    assert_eq!(\n        server\n            .get_cached_messages(account.id().document_id())\n            .await\n            .unwrap()\n            .emails\n            .items\n            .len(),\n        4\n    );\n    assert_eq!(cache.in_mailbox(INBOX_ID).count(), 2);\n    assert_eq!(cache.in_mailbox(TRASH_ID).count(), 1);\n    assert_eq!(cache.in_mailbox(JUNK_ID).count(), 1);\n\n    // Check IMAP status\n    imap.send(\"LIST \\\"\\\" \\\"*\\\" RETURN (STATUS (MESSAGES))\")\n        .await;\n    imap.assert_read(Type::Tagged, ResponseType::Ok)\n        .await\n        .assert_contains(\"\\\"INBOX\\\" (MESSAGES 2)\")\n        .assert_contains(\"\\\"Deleted Items\\\" (MESSAGES 1)\")\n        .assert_contains(\"\\\"Junk Mail\\\" (MESSAGES 1)\");\n\n    // Compare changes\n    let (new_changes, is_truncated) = get_changes(&server).await;\n    assert!(!changes.is_empty());\n    assert!(!new_changes.is_empty());\n    assert!(is_truncated);\n    for change in &changes {\n        assert!(\n            !new_changes.contains(change),\n            \"Change {change:?} was not purged, expected {} changes, got {}\",\n            changes.len(),\n            new_changes.len()\n        );\n    }\n\n    // Delete account\n    wait_for_index(&server).await;\n    server\n        .store()\n        .delete_principal(QueryBy::Id(account.id().document_id()))\n        .await\n        .unwrap();\n    destroy_account_data(&server, account.id().document_id(), true)\n        .await\n        .unwrap();\n    params.assert_is_empty().await;\n}\n\nasync fn get_changes(server: &Server) -> (AHashSet<(u64, u8)>, bool) {\n    let mut changes = AHashSet::new();\n    let mut is_truncated = false;\n    server\n        .core\n        .storage\n        .data\n        .iterate(\n            IterateParams::new(\n                LogKey {\n                    account_id: 0,\n                    collection: 0,\n                    change_id: 0,\n                },\n                LogKey {\n                    account_id: u32::MAX,\n                    collection: u8::MAX,\n                    change_id: u64::MAX,\n                },\n            )\n            .ascending(),\n            |key, value| {\n                if !value.is_empty() {\n                    changes.insert((\n                        key.deserialize_be_u64(key.len() - U64_LEN).unwrap(),\n                        key[U32_LEN],\n                    ));\n                } else {\n                    is_truncated = true;\n                }\n                Ok(true)\n            },\n        )\n        .await\n        .unwrap();\n    (changes, is_truncated)\n}\n"
  },
  {
    "path": "tests/src/jmap/server/webhooks.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    sync::{\n        Arc,\n        atomic::{AtomicBool, Ordering},\n    },\n    time::Duration,\n};\n\nuse crate::jmap::JMAPTest;\nuse base64::{Engine, engine::general_purpose::STANDARD};\nuse common::manager::webadmin::Resource;\nuse http_proto::{ToHttpResponse, request::fetch_body};\nuse hyper::{body, server::conn::http1, service::service_fn};\nuse hyper_util::rt::TokioIo;\nuse jmap::api::ToJmapHttpResponse;\nuse jmap_proto::error::request::RequestError;\nuse ring::hmac;\nuse store::parking_lot::Mutex;\nuse tokio::{net::TcpListener, sync::watch};\n\npub struct MockWebhookEndpoint {\n    pub tx: watch::Sender<bool>,\n    pub events: Mutex<Vec<serde_json::Value>>,\n    pub reject: AtomicBool,\n}\n\npub async fn test(params: &mut JMAPTest) {\n    println!(\"Running Webhook tests...\");\n\n    // Webhooks endpoint starts disabled by default, make sure there are no events.\n    tokio::time::sleep(Duration::from_millis(200)).await;\n    params.webhook.assert_is_empty();\n\n    // Enable the endpoint\n    params.webhook.accept();\n    tokio::time::sleep(Duration::from_millis(1000)).await;\n\n    // Check for events\n    params.webhook.assert_contains(&[\"auth.success\"]);\n}\n\nimpl MockWebhookEndpoint {\n    pub fn assert_contains(&self, expected: &[&str]) {\n        let events =\n            serde_json::to_string_pretty(&self.events.lock().drain(..).collect::<Vec<_>>())\n                .unwrap();\n\n        for string in expected {\n            if !events.contains(string) {\n                panic!(\n                    \"Expected events to contain '{}', but it did not. Events: {}\",\n                    string, events\n                );\n            }\n        }\n    }\n\n    pub fn accept(&self) {\n        self.reject.store(false, Ordering::Relaxed);\n    }\n\n    pub fn reject(&self) {\n        self.reject.store(true, Ordering::Relaxed);\n    }\n\n    pub fn clear(&self) {\n        self.events.lock().clear();\n    }\n\n    pub fn assert_is_empty(&self) {\n        assert!(self.events.lock().is_empty());\n    }\n}\n\npub fn spawn_mock_webhook_endpoint() -> Arc<MockWebhookEndpoint> {\n    let (tx, rx) = watch::channel(true);\n    let endpoint_ = Arc::new(MockWebhookEndpoint {\n        tx,\n        events: Mutex::new(vec![]),\n        reject: true.into(),\n    });\n\n    let endpoint = endpoint_.clone();\n\n    tokio::spawn(async move {\n        let listener = TcpListener::bind(\"127.0.0.1:8821\")\n            .await\n            .unwrap_or_else(|e| {\n                panic!(\"Failed to bind mock Webhooks server to 127.0.0.1:8821: {e}\");\n            });\n        let mut rx_ = rx.clone();\n\n        loop {\n            tokio::select! {\n                stream = listener.accept() => {\n                    match stream {\n                        Ok((stream, _)) => {\n\n                            let _ = http1::Builder::new()\n                            .keep_alive(false)\n                            .serve_connection(\n                                TokioIo::new(stream),\n                                service_fn(|mut req: hyper::Request<body::Incoming>| {\n                                    let endpoint = endpoint.clone();\n\n                                    async move {\n                                        // Verify HMAC signature\n                                        let key = hmac::Key::new(hmac::HMAC_SHA256, \"ovos-moles\".as_bytes());\n                                        let body = fetch_body(&mut req, usize::MAX, 0).await.unwrap();\n                                        let tag = STANDARD.decode(req.headers().get(\"X-Signature\").unwrap().to_str().unwrap()).unwrap();\n                                        hmac::verify(&key, &body, &tag).expect(\"Invalid signature\");\n\n                                        // Deserialize JSON\n                                        #[derive(serde::Deserialize)]\n                                        struct WebhookRequest {\n                                            events: Vec<serde_json::Value>,\n                                        }\n                                        let request = serde_json::from_slice::<WebhookRequest>(&body)\n                                        .expect(\"Failed to parse JSON\");\n\n                                        if !endpoint.reject.load(Ordering::Relaxed) {\n                                            //let c = print!(\"received webhook: {}\", serde_json::to_string_pretty(&request).unwrap());\n\n                                            // Add events\n                                            endpoint.events.lock().extend(request.events);\n\n                                            Ok::<_, hyper::Error>(\n                                                Resource::new(\"application/json\", \"[]\".to_string().into_bytes())\n                                                .into_http_response().build(),\n                                            )\n                                        } else {\n                                            //let c = print!(\"rejected webhook: {}\", serde_json::to_string_pretty(&request).unwrap());\n\n                                            Ok::<_, hyper::Error>(\n                                                RequestError::not_found().into_http_response().build()\n                                            )\n                                        }\n\n                                    }\n                                }),\n                            )\n                            .await;\n                        }\n                        Err(err) => {\n                            panic!(\"Something went wrong: {err}\" );\n                        }\n                    }\n                },\n                _ = rx_.changed() => {\n                    //println!(\"Mock jMilter server stopping\");\n                    break;\n                }\n            };\n        }\n    });\n\n    endpoint_\n}\n"
  },
  {
    "path": "tests/src/lib.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::path::PathBuf;\n\n#[cfg(not(target_env = \"msvc\"))]\nuse jemallocator::Jemalloc;\n#[cfg(test)]\nuse trc::Collector;\n\n#[cfg(not(target_env = \"msvc\"))]\n#[global_allocator]\nstatic GLOBAL: Jemalloc = Jemalloc;\n\n#[cfg(test)]\npub mod cluster;\n#[cfg(test)]\npub mod directory;\n#[cfg(test)]\npub mod http_server;\n#[cfg(test)]\npub mod imap;\n#[cfg(test)]\npub mod jmap;\n#[cfg(test)]\npub mod smtp;\n#[cfg(test)]\npub mod store;\n#[cfg(test)]\npub mod webdav;\n\npub fn add_test_certs(config: &str) -> String {\n    let mut cert_path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    cert_path.push(\"resources\");\n    let mut cert = cert_path.clone();\n    cert.push(\"tls_cert.pem\");\n    let mut pk = cert_path.clone();\n    pk.push(\"tls_privatekey.pem\");\n\n    config\n        .replace(\"{CERT}\", cert.as_path().to_str().unwrap())\n        .replace(\"{PK}\", pk.as_path().to_str().unwrap())\n}\n\n#[cfg(test)]\npub trait AssertConfig {\n    fn assert_no_errors(self) -> Self;\n    fn assert_no_warnings(self) -> Self;\n}\n\n#[cfg(test)]\nimpl AssertConfig for utils::config::Config {\n    fn assert_no_errors(self) -> Self {\n        if !self.errors.is_empty() {\n            panic!(\"Errors: {:#?}\", self.errors);\n        }\n        self\n    }\n\n    fn assert_no_warnings(self) -> Self {\n        if !self.warnings.is_empty() {\n            panic!(\"Warnings: {:#?}\", self.warnings);\n        }\n        self\n    }\n}\n\n#[cfg(test)]\npub fn enable_logging() {\n    use common::config::telemetry::Telemetry;\n\n    if let Ok(level) = std::env::var(\"LOG\")\n        && !Collector::is_enabled()\n    {\n        Telemetry::test_tracer(level.parse().expect(\"Invalid log level\"));\n    }\n}\n\npub const TEST_USERS: &[(&str, &str, &str, &str)] = &[\n    (\"admin\", \"secret\", \"Superuser\", \"admin@example.com\"),\n    (\"john\", \"secret2\", \"John Doe\", \"jdoe@example.com\"),\n    (\n        \"jane\",\n        \"secret3\",\n        \"Jane Doe-Smith\",\n        \"jane.smith@example.com\",\n    ),\n    (\"bill\", \"secret4\", \"Bill Foobar\", \"bill@example.com\"),\n    (\"mike\", \"secret5\", \"Mike Noquota\", \"mike@example.com\"),\n];\n"
  },
  {
    "path": "tests/src/smtp/config.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fs, net::IpAddr, path::PathBuf, sync::Arc, time::Duration};\n\nuse common::{\n    Server,\n    config::{\n        server::{Listener, Listeners, ServerProtocol, TcpListener},\n        smtp::*,\n    },\n    expr::{functions::ResolveVariable, if_block::*, tokenizer::TokenMap, *},\n};\n\nuse compact_str::ToCompactString;\nuse throttle::parse_queue_rate_limiter;\nuse tokio::net::TcpSocket;\n\nuse utils::config::{Config, Rate};\n\nuse super::add_test_certs;\n\nstruct TestEnvelope {\n    pub local_ip: IpAddr,\n    pub remote_ip: IpAddr,\n    pub sender_domain: String,\n    pub sender: String,\n    pub rcpt_domain: String,\n    pub rcpt: String,\n    pub helo_domain: String,\n    pub authenticated_as: String,\n    pub mx: String,\n    pub listener_id: String,\n    pub priority: i16,\n}\n\n#[test]\nfn parse_if_blocks() {\n    let mut file = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    file.push(\"resources\");\n    file.push(\"smtp\");\n    file.push(\"config\");\n    file.push(\"if-blocks.toml\");\n\n    let mut config = Config::new(fs::read_to_string(file).unwrap()).unwrap();\n\n    // Create context and add some conditions\n\n    let token_map = TokenMap::default().with_variables(&[\n        V_RECIPIENT,\n        V_RECIPIENT_DOMAIN,\n        V_SENDER,\n        V_SENDER_DOMAIN,\n        V_AUTHENTICATED_AS,\n        V_LISTENER,\n        V_REMOTE_IP,\n        V_LOCAL_IP,\n        V_PRIORITY,\n    ]);\n\n    assert_eq!(\n        IfBlock::try_parse(&mut config, \"durations\", &token_map).unwrap(),\n        IfBlock {\n            key: \"durations\".into(),\n            if_then: vec![\n                IfThen {\n                    expr: Expression {\n                        items: vec![\n                            ExpressionItem::Variable(V_SENDER),\n                            ExpressionItem::Constant(Constant::String(\"jdoe\".into())),\n                            ExpressionItem::BinaryOperator(BinaryOperator::Eq)\n                        ]\n                    },\n                    then: Expression {\n                        items: vec![ExpressionItem::Constant(Constant::Integer(432000000))]\n                    }\n                },\n                IfThen {\n                    expr: Expression {\n                        items: vec![\n                            ExpressionItem::Variable(V_PRIORITY),\n                            ExpressionItem::Constant(Constant::Integer(1)),\n                            ExpressionItem::UnaryOperator(UnaryOperator::Minus),\n                            ExpressionItem::BinaryOperator(BinaryOperator::Eq),\n                            ExpressionItem::JmpIf { val: true, pos: 4 },\n                            ExpressionItem::Variable(V_RECIPIENT),\n                            ExpressionItem::Constant(Constant::String(\"jane\".into())),\n                            ExpressionItem::Function {\n                                id: 29,\n                                num_args: 2\n                            },\n                            ExpressionItem::BinaryOperator(BinaryOperator::Or)\n                        ]\n                    },\n                    then: Expression {\n                        items: vec![ExpressionItem::Constant(Constant::Integer(3600000))]\n                    }\n                }\n            ],\n            default: Expression {\n                items: vec![ExpressionItem::Constant(Constant::Integer(0))]\n            }\n        }\n    );\n\n    assert_eq!(\n        IfBlock::try_parse(&mut config, \"string-list\", &token_map).unwrap(),\n        IfBlock {\n            key: \"string-list\".into(),\n            if_then: vec![\n                IfThen {\n                    expr: Expression {\n                        items: vec![\n                            ExpressionItem::Variable(V_SENDER),\n                            ExpressionItem::Constant(Constant::String(\"jdoe\".into())),\n                            ExpressionItem::BinaryOperator(BinaryOperator::Eq)\n                        ]\n                    },\n                    then: Expression {\n                        items: vec![\n                            ExpressionItem::Constant(Constant::String(\"From\".into())),\n                            ExpressionItem::Constant(Constant::String(\"To\".into())),\n                            ExpressionItem::Constant(Constant::String(\"Date\".into())),\n                            ExpressionItem::ArrayBuild(3)\n                        ]\n                    }\n                },\n                IfThen {\n                    expr: Expression {\n                        items: vec![\n                            ExpressionItem::Variable(V_PRIORITY),\n                            ExpressionItem::Constant(Constant::Integer(1)),\n                            ExpressionItem::UnaryOperator(UnaryOperator::Minus),\n                            ExpressionItem::BinaryOperator(BinaryOperator::Eq),\n                            ExpressionItem::JmpIf { val: true, pos: 4 },\n                            ExpressionItem::Variable(V_RECIPIENT),\n                            ExpressionItem::Constant(Constant::String(\"jane\".into())),\n                            ExpressionItem::Function {\n                                id: 29,\n                                num_args: 2\n                            },\n                            ExpressionItem::BinaryOperator(BinaryOperator::Or)\n                        ]\n                    },\n                    then: Expression {\n                        items: vec![ExpressionItem::Constant(Constant::String(\n                            \"Other-ID\".into()\n                        ))]\n                    }\n                }\n            ],\n            default: Expression {\n                items: vec![ExpressionItem::ArrayBuild(0)]\n            }\n        }\n    );\n\n    assert_eq!(\n        IfBlock::try_parse(&mut config, \"string-list-bis\", &token_map).unwrap(),\n        IfBlock {\n            key: \"string-list-bis\".into(),\n            if_then: vec![\n                IfThen {\n                    expr: Expression {\n                        items: vec![\n                            ExpressionItem::Variable(V_SENDER),\n                            ExpressionItem::Constant(Constant::String(\"jdoe\".into())),\n                            ExpressionItem::BinaryOperator(BinaryOperator::Eq)\n                        ]\n                    },\n                    then: Expression {\n                        items: vec![\n                            ExpressionItem::Constant(Constant::String(\"From\".into())),\n                            ExpressionItem::Constant(Constant::String(\"To\".into())),\n                            ExpressionItem::Constant(Constant::String(\"Date\".into())),\n                            ExpressionItem::ArrayBuild(3)\n                        ]\n                    }\n                },\n                IfThen {\n                    expr: Expression {\n                        items: vec![\n                            ExpressionItem::Variable(V_PRIORITY),\n                            ExpressionItem::Constant(Constant::Integer(1)),\n                            ExpressionItem::UnaryOperator(UnaryOperator::Minus),\n                            ExpressionItem::BinaryOperator(BinaryOperator::Eq),\n                            ExpressionItem::JmpIf { val: true, pos: 4 },\n                            ExpressionItem::Variable(V_RECIPIENT),\n                            ExpressionItem::Constant(Constant::String(\"jane\".into())),\n                            ExpressionItem::Function {\n                                id: 29,\n                                num_args: 2\n                            },\n                            ExpressionItem::BinaryOperator(BinaryOperator::Or)\n                        ]\n                    },\n                    then: Expression {\n                        items: vec![ExpressionItem::ArrayBuild(0)]\n                    }\n                }\n            ],\n            default: Expression {\n                items: vec![\n                    ExpressionItem::Constant(Constant::String(\"ID-Bis\".into())),\n                    ExpressionItem::ArrayBuild(1)\n                ]\n            }\n        }\n    );\n\n    assert_eq!(\n        IfBlock::try_parse(&mut config, \"single-value\", &token_map).unwrap(),\n        IfBlock {\n            key: \"single-value\".into(),\n            if_then: vec![],\n            default: Expression {\n                items: vec![ExpressionItem::Constant(Constant::String(\n                    \"hello world\".into()\n                ))]\n            }\n        }\n    );\n\n    for bad_rule in [\n        \"bad-if-without-then\",\n        \"bad-if-without-else\",\n        \"bad-multiple-else\",\n    ] {\n        if let Some(value) = IfBlock::try_parse(&mut config, bad_rule, &token_map) {\n            panic!(\"Condition {bad_rule:?} had unexpected result {value:?}\");\n        }\n    }\n}\n\n#[test]\nfn parse_throttles() {\n    let mut file = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    file.push(\"resources\");\n    file.push(\"smtp\");\n    file.push(\"config\");\n    file.push(\"throttle.toml\");\n\n    let mut config = Config::new(fs::read_to_string(file).unwrap()).unwrap();\n    let throttle = parse_queue_rate_limiter(\n        &mut config,\n        \"throttle\",\n        &TokenMap::default().with_variables(&[\n            V_RECIPIENT,\n            V_RECIPIENT_DOMAIN,\n            V_SENDER,\n            V_SENDER_DOMAIN,\n            V_AUTHENTICATED_AS,\n            V_LISTENER,\n            V_REMOTE_IP,\n            V_LOCAL_IP,\n            V_PRIORITY,\n        ]),\n        u16::MAX,\n    );\n\n    assert_eq!(\n        throttle,\n        vec![\n            QueueRateLimiter {\n                id: \"0000\".into(),\n                expr: Expression {\n                    items: vec![\n                        ExpressionItem::Variable(8),\n                        ExpressionItem::Constant(Constant::String(\"127.0.0.1\".into())),\n                        ExpressionItem::BinaryOperator(BinaryOperator::Eq)\n                    ]\n                },\n                keys: THROTTLE_REMOTE_IP | THROTTLE_AUTH_AS,\n                rate: Rate {\n                    requests: 50,\n                    period: Duration::from_secs(30)\n                }\n            },\n            QueueRateLimiter {\n                id: \"0001\".into(),\n                expr: Expression::default(),\n                keys: THROTTLE_SENDER_DOMAIN,\n                rate: Rate {\n                    requests: 50,\n                    period: Duration::from_secs(30)\n                }\n            }\n        ]\n    );\n}\n\n#[test]\nfn parse_servers() {\n    let mut file = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    file.push(\"resources\");\n    file.push(\"smtp\");\n    file.push(\"config\");\n    file.push(\"servers.toml\");\n\n    let toml = add_test_certs(&fs::read_to_string(file).unwrap());\n\n    // Parse servers\n    let mut config = Config::new(toml).unwrap();\n    let servers = Listeners::parse(&mut config).servers;\n    let id_generator = Arc::new(utils::snowflake::SnowflakeIdGenerator::new());\n    let expected_servers = vec![\n        Listener {\n            id: \"smtp\".into(),\n            protocol: ServerProtocol::Smtp,\n            listeners: vec![TcpListener {\n                socket: TcpSocket::new_v4().unwrap(),\n                addr: \"127.0.0.1:9925\".parse().unwrap(),\n                ttl: 3600.into(),\n                backlog: 1024.into(),\n                linger: None,\n                nodelay: true,\n            }],\n            max_connections: 8192,\n            proxy_networks: vec![],\n            span_id_gen: id_generator.clone(),\n        },\n        Listener {\n            id: \"smtps\".into(),\n            protocol: ServerProtocol::Smtp,\n            listeners: vec![\n                TcpListener {\n                    socket: TcpSocket::new_v4().unwrap(),\n                    addr: \"127.0.0.1:9465\".parse().unwrap(),\n                    ttl: 4096.into(),\n                    backlog: 1024.into(),\n                    linger: None,\n                    nodelay: true,\n                },\n                TcpListener {\n                    socket: TcpSocket::new_v4().unwrap(),\n                    addr: \"127.0.0.1:9466\".parse().unwrap(),\n                    ttl: 4096.into(),\n                    backlog: 1024.into(),\n                    linger: None,\n                    nodelay: true,\n                },\n            ],\n            max_connections: 1024,\n            proxy_networks: vec![],\n            span_id_gen: id_generator.clone(),\n        },\n        Listener {\n            id: \"submission\".into(),\n            protocol: ServerProtocol::Smtp,\n            listeners: vec![TcpListener {\n                socket: TcpSocket::new_v4().unwrap(),\n                addr: \"127.0.0.1:9991\".parse().unwrap(),\n                ttl: 3600.into(),\n                backlog: 2048.into(),\n                linger: None,\n                nodelay: true,\n            }],\n            max_connections: 8192,\n            proxy_networks: vec![],\n            span_id_gen: id_generator.clone(),\n        },\n    ];\n\n    for (server, expected_server) in servers.into_iter().zip(expected_servers) {\n        assert_eq!(\n            server.id, expected_server.id,\n            \"failed for {}\",\n            expected_server.id\n        );\n        assert_eq!(\n            server.protocol, expected_server.protocol,\n            \"failed for {}\",\n            expected_server.id\n        );\n        for (listener, expected_listener) in\n            server.listeners.into_iter().zip(expected_server.listeners)\n        {\n            assert_eq!(\n                listener.addr, expected_listener.addr,\n                \"failed for {}\",\n                expected_server.id\n            );\n            assert_eq!(\n                listener.ttl, expected_listener.ttl,\n                \"failed for {}\",\n                expected_server.id\n            );\n            assert_eq!(\n                listener.backlog, expected_listener.backlog,\n                \"failed for {}\",\n                expected_server.id\n            );\n        }\n    }\n}\n\n#[tokio::test]\nasync fn eval_if() {\n    let mut file = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    file.push(\"resources\");\n    file.push(\"smtp\");\n    file.push(\"config\");\n    file.push(\"rules-eval.toml\");\n\n    let mut config = Config::new(fs::read_to_string(file).unwrap()).unwrap();\n    let envelope = TestEnvelope::from_config(&mut config);\n    let token_map = TokenMap::default().with_variables(&[\n        V_RECIPIENT,\n        V_RECIPIENT_DOMAIN,\n        V_SENDER,\n        V_SENDER_DOMAIN,\n        V_AUTHENTICATED_AS,\n        V_LISTENER,\n        V_REMOTE_IP,\n        V_LOCAL_IP,\n        V_PRIORITY,\n        V_MX,\n    ]);\n    let core = Server::default();\n\n    for (key, _) in config.keys.clone() {\n        if !key.starts_with(\"rule.\") {\n            continue;\n        }\n\n        //println!(\"============= Testing {:?} ==================\", key);\n        let (_, expected_result) = key.rsplit_once('-').unwrap();\n        assert_eq!(\n            core.eval_if::<Variable, _>(\n                &IfBlock {\n                    key: key.to_string(),\n                    if_then: vec![IfThen {\n                        expr: Expression::try_parse(&mut config, key.as_str(), &token_map).unwrap(),\n                        then: Expression::from(true),\n                    }],\n                    default: Expression::from(false),\n                },\n                &envelope,\n                0\n            )\n            .await\n            .unwrap()\n            .to_bool(),\n            expected_result.parse::<bool>().unwrap(),\n            \"failed for {key:?}\"\n        );\n    }\n}\n\n#[tokio::test]\nasync fn eval_dynvalue() {\n    let mut file = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    file.push(\"resources\");\n    file.push(\"smtp\");\n    file.push(\"config\");\n    file.push(\"rules-dynvalue.toml\");\n\n    let mut config = Config::new(fs::read_to_string(file).unwrap()).unwrap();\n    let envelope = TestEnvelope::from_config(&mut config);\n    let token_map = TokenMap::default().with_variables(&[\n        V_RECIPIENT,\n        V_RECIPIENT_DOMAIN,\n        V_SENDER,\n        V_SENDER_DOMAIN,\n        V_AUTHENTICATED_AS,\n        V_LISTENER,\n        V_REMOTE_IP,\n        V_LOCAL_IP,\n        V_PRIORITY,\n        V_MX,\n    ]);\n    let core = Server::default();\n\n    for test_name in config.sub_keys(\"eval\", \"\") {\n        //println!(\"============= Testing {:?} ==================\", key);\n        let if_block = IfBlock::try_parse(\n            &mut config,\n            (\"eval\", test_name.as_str(), \"test\"),\n            &token_map,\n        )\n        .unwrap();\n        let expected = config\n            .property_require::<Option<String>>((\"eval\", test_name.as_str(), \"expect\"))\n            .unwrap_or_else(|| panic!(\"Missing expect for test {test_name:?}\"));\n\n        assert_eq!(\n            core.eval_if::<String, _>(&if_block, &envelope, 0).await,\n            expected,\n            \"failed for test {test_name:?}\"\n        );\n    }\n}\n\nimpl ResolveVariable for TestEnvelope {\n    fn resolve_variable(&self, variable: u32) -> Variable<'_> {\n        match variable {\n            V_RECIPIENT => self.rcpt.as_str().into(),\n            V_RECIPIENT_DOMAIN => self.rcpt_domain.as_str().into(),\n            V_SENDER => self.sender.as_str().into(),\n            V_SENDER_DOMAIN => self.sender_domain.as_str().into(),\n            V_AUTHENTICATED_AS => self.authenticated_as.as_str().into(),\n            V_LISTENER => self.listener_id.to_compact_string().into(),\n            V_REMOTE_IP => self.remote_ip.to_compact_string().into(),\n            V_LOCAL_IP => self.local_ip.to_compact_string().into(),\n            V_PRIORITY => self.priority.to_compact_string().into(),\n            V_MX => self.mx.as_str().into(),\n            V_HELO_DOMAIN => self.helo_domain.as_str().into(),\n            _ => Default::default(),\n        }\n    }\n\n    fn resolve_global(&self, _: &str) -> Variable<'_> {\n        Variable::Integer(0)\n    }\n}\n\nimpl TestEnvelope {\n    pub fn from_config(config: &mut Config) -> Self {\n        Self {\n            local_ip: config.property_require(\"envelope.local-ip\").unwrap(),\n            remote_ip: config.property_require(\"envelope.remote-ip\").unwrap(),\n            sender_domain: config.property_require(\"envelope.sender-domain\").unwrap(),\n            sender: config.property_require(\"envelope.sender\").unwrap(),\n            rcpt_domain: config.property_require(\"envelope.rcpt-domain\").unwrap(),\n            rcpt: config.property_require(\"envelope.rcpt\").unwrap(),\n            authenticated_as: config\n                .property_require(\"envelope.authenticated-as\")\n                .unwrap(),\n            mx: config.property_require(\"envelope.mx\").unwrap(),\n            listener_id: config.property_require(\"envelope.listener\").unwrap(),\n            priority: config.property_require(\"envelope.priority\").unwrap(),\n            helo_domain: config.property_require(\"envelope.helo-domain\").unwrap(),\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/antispam.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    http_server::{HttpMessage, spawn_mock_http_server},\n    jmap::server::enterprise::EnterpriseCore,\n    smtp::{DnsCache, TempDir, TestSMTP, session::TestSession},\n};\nuse ahash::{AHashMap, AHashSet};\nuse common::{\n    Core, Server,\n    auth::AccessToken,\n    config::spamfilter::SpamFilterAction,\n    enterprise::{\n        SpamFilterLlmConfig,\n        llm::{\n            AiApiConfig, ChatCompletionChoice, ChatCompletionRequest, ChatCompletionResponse,\n            Message,\n        },\n    },\n};\nuse email::message::ingest::EmailIngest;\nuse http_proto::{JsonResponse, ToHttpResponse};\nuse hyper::Method;\nuse mail_auth::{\n    ArcOutput, DkimOutput, DkimResult, DmarcResult, IprevOutput, IprevResult, MX, SpfOutput,\n    SpfResult, dkim::Signature, dmarc::Policy,\n};\nuse mail_parser::MessageParser;\nuse smtp::core::{Session, SessionAddress};\nuse smtp_proto::{MAIL_BODY_8BITMIME, MAIL_SMTPUTF8};\nuse spam_filter::{\n    SpamFilterInput,\n    analysis::{\n        classifier::SpamFilterAnalyzeClassify, date::SpamFilterAnalyzeDate,\n        dmarc::SpamFilterAnalyzeDmarc, domain::SpamFilterAnalyzeDomain,\n        ehlo::SpamFilterAnalyzeEhlo, from::SpamFilterAnalyzeFrom,\n        headers::SpamFilterAnalyzeHeaders, html::SpamFilterAnalyzeHtml, init::SpamFilterInit,\n        ip::SpamFilterAnalyzeIp, llm::SpamFilterAnalyzeLlm, messageid::SpamFilterAnalyzeMid,\n        mime::SpamFilterAnalyzeMime, pyzor::SpamFilterAnalyzePyzor,\n        received::SpamFilterAnalyzeReceived, recipient::SpamFilterAnalyzeRecipient,\n        replyto::SpamFilterAnalyzeReplyTo, rules::SpamFilterAnalyzeRules,\n        score::SpamFilterAnalyzeScore, subject::SpamFilterAnalyzeSubject,\n        url::SpamFilterAnalyzeUrl,\n    },\n    modules::{\n        classifier::{SpamClassifier, Token},\n        html::{HtmlToken, html_to_tokens},\n    },\n};\nuse std::{\n    fs,\n    path::PathBuf,\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse store::{Stores, write::BatchBuilder};\nuse utils::config::Config;\n\nconst CONFIG: &str = r#\"\n[spam-filter.score]\nspam = \"5.0\"\n\n[spam-filter.llm]\nenable = true\nmodel = \"dummy\"\nprompt = \"You are an AI assistant specialized in analyzing email content to detect unsolicited, commercial, or harmful messages. Format your response as follows, separated by commas: Category,Confidence,Explanation\nHere's the email to analyze, please provide your analysis based on the above instructions, ensuring your response is in the specified comma-separated format.\"\nseparator = \",\"\ncategories = [\"Unsolicited\", \"Commercial\", \"Harmful\", \"Legitimate\"]\nconfidence = [\"High\", \"Medium\", \"Low\"]\n\n[spam-filter.llm.index]\ncategory = 0\nconfidence = 1\nexplanation = 2\n\n[spam-filter.classifier.samples]\nmin-ham = 10\nmin-spam = 10\n\n[session.rcpt]\nrelay = true\n\n[storage]\ndata = \"spamdb\"\nlookup = \"spamdb\"\nblob = \"spamdb\"\nfts = \"spamdb\"\ndirectory = \"spamdb\"\n\n[directory.\"spamdb\"]\ntype = \"internal\"\nstore = \"spamdb\"\n\n[store.\"spamdb\"]\ntype = \"rocksdb\"\npath = \"{PATH}/test_antispam.db\"\n\n#[store.\"redis\"]\n#type = \"redis\"\n#url = \"redis://127.0.0.1\"\n\n[http-lookup.STWT_OPENPHISH]\nenable = true\nurl = \"https://openphish.com/feed.txt\"\nformat = \"list\"\nretry = \"1h\"\nrefresh = \"12h\"\ntimeout = \"30s\"\nlimits.size = 104857600\nlimits.entries = 900000\nlimits.entry-size = 512\n\n[http-lookup.STWT_PHISHTANK]\nenable = true\nurl = \"http://data.phishtank.com/data/online-valid.csv.gz\"\nformat = \"csv\"\nseparator = \",\"\nindex.key = 1\nskip-first = true\ngzipped = true\nretry = \"1h\"\nrefresh = \"6h\"\ntimeout = \"30s\"\nlimits.size = 104857600\nlimits.entries = 900000\nlimits.entry-size = 512\n\n[http-lookup.STWT_DISPOSABLE_DOMAINS]\nenable = true\nurl = \"https://disposable.github.io/disposable-email-domains/domains_mx.txt\"\nformat = \"list\"\nretry = \"1h\"\nrefresh = \"24h\"\ntimeout = \"30s\"\nlimits.size = 104857600\nlimits.entries = 900000\nlimits.entry-size = 512\n\n[http-lookup.STWT_FREE_DOMAINS]\nenable = true\nurl = \"https://gist.githubusercontent.com/okutbay/5b4974b70673dfdcc21c517632c1f984/raw/993a35930a8d24a1faab1b988d19d38d92afbba4/free_email_provider_domains.txt\"\nformat = \"list\"\nretry = \"1h\"\nrefresh = \"720h\"\ntimeout = \"30s\"\nlimits.size = 104857600\nlimits.entries = 900000\nlimits.entry-size = 512\n\n[enterprise.ai.dummy]\nurl = \"https://127.0.0.1:9090/v1/chat/completions\"\ntype = \"chat\"\nmodel = \"gpt-dummy\"\nallow-invalid-certs = true\n\n[spam-filter.list]\n\"file-extensions\" = { \"html\" = \"text/html|BAD\", \n                \"pdf\" = \"application/pdf|NZ\", \n                \"txt\" = \"text/plain|message/disposition-notification|text/rfc822-headers\", \n                \"zip\" = \"AR\", \n                \"js\" = \"BAD|NZ\", \n                \"hta\" = \"BAD|NZ\" }\n[lookup]\n\"url-redirectors\" = {\"bit.ly\", \"redirect.io\", \"redirect.me\", \"redirect.org\", \"redirect.com\", \"redirect.net\", \"t.ly\", \"tinyurl.com\"}\n\"spam-traps\" = {\"spamtrap@*\"}\n\"trusted-domains\" = {\"stalw.art\"}\n\"surbl-hashbl\" = {\"bit.ly\", \"drive.google.com\", \"lnkiy.in\"}\n\"#;\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn antispam() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Prepare config\n    let tmp_dir = TempDir::new(\"smtp_antispam_test\", true);\n    let mut config = CONFIG.replace(\"{PATH}\", tmp_dir.temp_dir.as_path().to_str().unwrap());\n    let base_path = PathBuf::from(\n        std::env::var(\"SPAM_RULES_DIR\")\n            .unwrap_or_else(|_| \"/Users/me/code/spam-filter\".to_string()),\n    );\n    for section in [\"rules\", \"lists\"] {\n        for entry in fs::read_dir(base_path.join(section)).unwrap() {\n            let entry = entry.unwrap();\n            let path = entry.path();\n            if path.is_file() {\n                let file_name = path.file_name().unwrap().to_str().unwrap();\n                if file_name.ends_with(\".toml\")\n                    && ((section == \"rules\" && file_name != \"llm.toml\")\n                        || (section == \"lists\" && file_name == \"scores.toml\"))\n                {\n                    let contents = fs::read_to_string(&path).unwrap();\n                    config.push_str(\"\\n\\n\");\n                    config.push_str(&contents);\n                }\n            }\n        }\n    }\n\n    // Parse config\n    let mut config = Config::new(&config).unwrap();\n    config.resolve_all_macros().await;\n    let stores = Stores::parse_all(&mut config, false).await;\n    let mut core = Core::parse(&mut config, stores, Default::default())\n        .await\n        .enable_enterprise();\n    let ai_apis = AHashMap::from_iter([(\n        \"dummy\".to_string(),\n        AiApiConfig::parse(&mut config, \"dummy\").unwrap().into(),\n    )]);\n    core.enterprise.as_mut().unwrap().spam_filter_llm =\n        SpamFilterLlmConfig::parse(&mut config, &ai_apis);\n    crate::AssertConfig::assert_no_errors(config);\n    let server = TestSMTP::from_core(core).server;\n\n    // Add mock DNS entries\n    for (domain, ip) in [\n        (\"bank.com\", \"127.0.0.1\"),\n        (\"apple.com\", \"127.0.0.1\"),\n        (\"youtube.com\", \"127.0.0.1\"),\n        (\"twitter.com\", \"127.0.0.3\"),\n        (\"dkimtrusted.org.dwl.dnswl.org\", \"127.0.0.3\"),\n        (\"sh-malware.com.dbl.spamhaus.org\", \"127.0.1.5\"),\n        (\"surbl-abuse.com.multi.surbl.org\", \"127.0.0.64\"),\n        (\"uribl-grey.com.multi.uribl.com\", \"127.0.0.4\"),\n        (\"sem-uribl.com.uribl.spameatingmonkey.net\", \"127.0.0.2\"),\n        (\"sem-fresh15.com.fresh15.spameatingmonkey.net\", \"127.0.0.2\"),\n        (\n            \"b4a64d60f67529b0b18df66ea2f292e09e43c975.ebl.msbl.org\",\n            \"127.0.0.2\",\n        ),\n        (\n            \"a95bd658068a8315dc1864d6bb79632f47692621.ebl.msbl.org\",\n            \"127.0.1.3\",\n        ),\n        (\n            \"ba76e47680ba70a0cbff8d6c92139683.hashbl.surbl.org\",\n            \"127.0.0.16\",\n        ),\n        (\n            \"0ac5b387a1c6d8461a78bbf7b172a2a1.hashbl.surbl.org\",\n            \"127.0.0.64\",\n        ),\n        (\n            \"637d6717761b5de0c84108c894bb68f2.hashbl.surbl.org\",\n            \"127.0.0.8\",\n        ),\n    ] {\n        server.ipv4_add(\n            domain,\n            vec![ip.parse().unwrap()],\n            Instant::now() + Duration::from_secs(100),\n        );\n        server.dnsbl_add(\n            domain,\n            vec![ip.parse().unwrap()],\n            Instant::now() + Duration::from_secs(100),\n        );\n    }\n    for mx in [\n        \"domain.org\",\n        \"domain.co.uk\",\n        \"gmail.com\",\n        \"custom.disposable.org\",\n    ] {\n        server.mx_add(\n            mx,\n            vec![MX {\n                exchanges: vec![\"127.0.0.1\".parse().unwrap()],\n                preference: 10,\n            }],\n            Instant::now() + Duration::from_secs(100),\n        );\n    }\n\n    // Spawn mock OpenAI server\n    let _tx = spawn_mock_http_server(Arc::new(|req: HttpMessage| {\n        assert_eq!(req.uri.path(), \"/v1/chat/completions\");\n        assert_eq!(req.method, Method::POST);\n        let req =\n            serde_json::from_slice::<ChatCompletionRequest>(req.body.as_ref().unwrap()).unwrap();\n        assert_eq!(req.model, \"gpt-dummy\");\n        let message = &req.messages[0].content;\n        assert!(message.contains(\"You are an AI assistant specialized in analyzing email\"));\n\n        JsonResponse::new(&ChatCompletionResponse {\n            created: 0,\n            object: String::new(),\n            id: String::new(),\n            model: req.model,\n            choices: vec![ChatCompletionChoice {\n                index: 0,\n                finish_reason: \"stop\".to_string(),\n                message: Message {\n                    role: \"assistant\".to_string(),\n                    content: message.split_once(\"Subject: \").unwrap().1.to_string(),\n                },\n            }],\n        })\n        .into_http_response()\n    }))\n    .await;\n\n    // Run tests\n    let base_path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n        .join(\"resources\")\n        .join(\"smtp\")\n        .join(\"antispam\");\n    let filter_test = std::env::var(\"TEST_NAME\").ok();\n\n    for test_name in [\n        \"combined\",\n        \"ip\",\n        \"helo\",\n        \"received\",\n        \"messageid\",\n        \"date\",\n        \"from\",\n        \"subject\",\n        \"replyto\",\n        \"recipient\",\n        \"headers\",\n        \"url\",\n        \"html\",\n        \"mime\",\n        \"bounce\",\n        \"dmarc\",\n        \"rbl\",\n        \"spamtrap\",\n        \"classifier_html\",\n        \"classifier_features\",\n        \"classifier\",\n        \"pyzor\",\n        \"llm\",\n    ] {\n        if filter_test\n            .as_ref()\n            .is_some_and(|s| !s.eq_ignore_ascii_case(test_name))\n        {\n            continue;\n        }\n\n        println!(\"===== {test_name} =====\");\n        let contents = fs::read_to_string(base_path.join(format!(\"{test_name}.test\"))).unwrap();\n\n        match test_name {\n            \"classifier_html\" => {\n                html_tokens(contents);\n                continue;\n            }\n            \"classifier_features\" => {\n                classifier_features(&server, contents).await;\n                continue;\n            }\n            \"classifier\" => {\n                let mut batch = BatchBuilder::new();\n                batch.with_account_id(u32::MAX);\n                for class in [\"spam\", \"ham\"] {\n                    let contents =\n                        fs::read_to_string(base_path.join(format!(\"classifier.{class}\"))).unwrap();\n                    for sample in contents.split(\"<!-- NEXT TEST -->\") {\n                        let sample = sample.trim_start();\n                        if sample.is_empty() {\n                            continue;\n                        }\n\n                        let (hash, blob_hold) = server\n                            .put_temporary_blob(u32::MAX, sample.as_bytes(), 60)\n                            .await\n                            .unwrap();\n                        server.add_spam_sample(&mut batch, hash, class == \"spam\", true, 0);\n                        batch.clear(blob_hold);\n                    }\n                }\n                assert!(!batch.is_empty());\n                server.store().write(batch.build_all()).await.unwrap();\n                server.spam_train(false).await.unwrap();\n            }\n            _ => {}\n        }\n\n        let mut lines = contents.lines();\n        let mut has_more = true;\n\n        while has_more {\n            let mut message = String::new();\n            let mut in_params = true;\n\n            // Build session\n            let mut session = Session::test(server.clone());\n            let mut arc_result = None;\n            let mut dkim_result = None;\n            let mut dkim_signatures = vec![];\n            let mut dmarc_result = None;\n            let mut dmarc_policy = None;\n            let mut expected_tags: AHashSet<String> = AHashSet::new();\n            let mut expect_headers = String::new();\n            let mut body_params = 0;\n            let mut is_tls = false;\n\n            for line in lines.by_ref() {\n                if in_params {\n                    if line.is_empty() {\n                        in_params = false;\n                        continue;\n                    }\n                    let (param, value) = line.split_once(' ').unwrap();\n                    let value = value.trim();\n                    match param {\n                        \"remote_ip\" => {\n                            session.data.remote_ip_str = value.to_string();\n                            session.data.remote_ip = value.parse().unwrap();\n                        }\n                        \"helo_domain\" => {\n                            session.data.helo_domain = value.to_string();\n                        }\n                        \"authenticated_as\" => {\n                            session.data.authenticated_as = Some(Arc::new(AccessToken {\n                                name: value.to_string(),\n                                ..Default::default()\n                            }));\n                        }\n                        \"spf.result\" | \"spf_ehlo.result\" => {\n                            session.data.spf_mail_from =\n                                Some(SpfOutput::default().with_result(SpfResult::from_str(value)));\n                        }\n                        \"iprev.result\" => {\n                            session\n                                .data\n                                .iprev\n                                .get_or_insert(IprevOutput {\n                                    result: IprevResult::None,\n                                    ptr: None,\n                                })\n                                .result = IprevResult::from_str(value);\n                        }\n                        \"dkim.result\" => {\n                            dkim_result = match DkimResult::from_str(value) {\n                                DkimResult::Pass => DkimOutput::pass(),\n                                DkimResult::Neutral(error) => DkimOutput::neutral(error),\n                                DkimResult::Fail(error) => DkimOutput::fail(error),\n                                DkimResult::PermError(error) => DkimOutput::perm_err(error),\n                                DkimResult::TempError(error) => DkimOutput::temp_err(error),\n                                DkimResult::None => unreachable!(),\n                            }\n                            .into();\n                        }\n                        \"arc.result\" => {\n                            arc_result = ArcOutput::default()\n                                .with_result(DkimResult::from_str(value))\n                                .into();\n                        }\n                        \"dkim.domains\" => {\n                            dkim_signatures = value\n                                .split_ascii_whitespace()\n                                .map(|s| Signature {\n                                    d: s.to_lowercase(),\n                                    ..Default::default()\n                                })\n                                .collect();\n                        }\n                        \"envelope_from\" => {\n                            session.data.mail_from = Some(SessionAddress::new(value.to_string()));\n                        }\n                        \"envelope_to\" => {\n                            session\n                                .data\n                                .rcpt_to\n                                .push(SessionAddress::new(value.to_string()));\n                        }\n                        \"iprev.ptr\" => {\n                            session\n                                .data\n                                .iprev\n                                .get_or_insert(IprevOutput {\n                                    result: IprevResult::None,\n                                    ptr: None,\n                                })\n                                .ptr = Some(Arc::new(vec![value.to_string()]));\n                        }\n                        \"dmarc.result\" => {\n                            dmarc_result = DmarcResult::from_str(value).into();\n                        }\n                        \"dmarc.policy\" => {\n                            dmarc_policy = Policy::from_str(value).into();\n                        }\n                        \"expect\" => {\n                            expected_tags\n                                .extend(value.split_ascii_whitespace().map(|v| v.to_uppercase()));\n                        }\n                        \"expect_header\" => {\n                            let value = value.trim();\n                            if !value.is_empty() {\n                                if !expect_headers.is_empty() {\n                                    expect_headers.push(' ');\n                                }\n                                expect_headers.push_str(value);\n                            }\n                        }\n                        \"param.smtputf8\" => {\n                            body_params |= MAIL_SMTPUTF8;\n                        }\n                        \"param.8bitmime\" => {\n                            body_params |= MAIL_BODY_8BITMIME;\n                        }\n                        \"tls.version\" => {\n                            is_tls = true;\n                        }\n                        _ => panic!(\"Invalid parameter {param:?}\"),\n                    }\n                } else {\n                    has_more = line.trim().eq_ignore_ascii_case(\"<!-- NEXT TEST -->\");\n                    if !has_more {\n                        message.push_str(line);\n                        message.push_str(\"\\r\\n\");\n                    } else {\n                        break;\n                    }\n                }\n            }\n\n            if message.is_empty() {\n                panic!(\"No message found\");\n            }\n\n            if body_params != 0 {\n                session\n                    .data\n                    .mail_from\n                    .get_or_insert_with(|| SessionAddress::new(\"\".to_string()))\n                    .flags = body_params;\n            }\n\n            // Build input\n            let mut dkim_domains = vec![];\n            if let Some(dkim_result) = dkim_result {\n                if dkim_signatures.is_empty() {\n                    dkim_signatures.push(Signature {\n                        d: \"unknown.org\".to_string(),\n                        ..Default::default()\n                    });\n                }\n\n                for signature in &dkim_signatures {\n                    dkim_domains.push(dkim_result.clone().with_signature(signature));\n                }\n            }\n            let parsed_message = MessageParser::new().parse(&message).unwrap();\n\n            // Combined tests\n            if test_name == \"combined\" {\n                match session\n                    .spam_classify(\n                        &parsed_message,\n                        &dkim_domains,\n                        arc_result.as_ref(),\n                        dmarc_result.as_ref(),\n                        dmarc_policy.as_ref(),\n                    )\n                    .await\n                {\n                    SpamFilterAction::Allow(score) => {\n                        let mut last_ch = 'x';\n                        let mut result = String::with_capacity(score.headers.len());\n                        for ch in score.headers.chars() {\n                            if !ch.is_whitespace() {\n                                if last_ch.is_whitespace() {\n                                    result.push(' ');\n                                }\n                                result.push(ch);\n                            }\n                            last_ch = ch;\n                        }\n                        assert_eq!(result, expect_headers);\n                    }\n                    other => panic!(\"Unexpected action {other:?}\"),\n                }\n                continue;\n            }\n\n            // Initialize filter\n            let mut spam_input = session.build_spam_input(\n                &parsed_message,\n                &dkim_domains,\n                arc_result.as_ref(),\n                dmarc_result.as_ref(),\n                dmarc_policy.as_ref(),\n            );\n            spam_input.is_tls = is_tls;\n            let mut spam_ctx = server.spam_filter_init(spam_input);\n            match test_name {\n                \"html\" => {\n                    server.spam_filter_analyze_html(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                }\n                \"subject\" => {\n                    server.spam_filter_analyze_headers(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| t.starts_with(\"X_HDR_\"));\n                    server.spam_filter_analyze_subject(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| !t.starts_with(\"X_HDR_\"));\n                }\n                \"received\" => {\n                    server.spam_filter_analyze_headers(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| t.starts_with(\"X_HDR_\"));\n                    server.spam_filter_analyze_received(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| !t.starts_with(\"X_HDR_\"));\n                }\n                \"messageid\" => {\n                    server.spam_filter_analyze_message_id(&mut spam_ctx).await;\n                }\n                \"date\" => {\n                    server.spam_filter_analyze_date(&mut spam_ctx).await;\n                }\n                \"from\" => {\n                    server.spam_filter_analyze_from(&mut spam_ctx).await;\n                    server.spam_filter_analyze_domain(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                }\n                \"replyto\" => {\n                    server.spam_filter_analyze_reply_to(&mut spam_ctx).await;\n                    server.spam_filter_analyze_domain(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                }\n                \"recipient\" => {\n                    server.spam_filter_analyze_headers(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| t.starts_with(\"X_HDR_\"));\n                    server.spam_filter_analyze_recipient(&mut spam_ctx).await;\n                    server.spam_filter_analyze_domain(&mut spam_ctx).await;\n                    server.spam_filter_analyze_subject(&mut spam_ctx).await;\n                    server.spam_filter_analyze_url(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| !t.starts_with(\"X_HDR_\"));\n                }\n                \"mime\" => {\n                    server.spam_filter_analyze_mime(&mut spam_ctx).await;\n                }\n                \"headers\" => {\n                    server.spam_filter_analyze_headers(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| !t.starts_with(\"X_HDR_\"));\n                }\n                \"url\" => {\n                    server.spam_filter_analyze_url(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                }\n                \"dmarc\" => {\n                    server.spam_filter_analyze_dmarc(&mut spam_ctx).await;\n                    server.spam_filter_analyze_headers(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| !t.starts_with(\"X_HDR_\"));\n                }\n                \"ip\" => {\n                    server.spam_filter_analyze_ip(&mut spam_ctx).await;\n                }\n                \"helo\" => {\n                    server.spam_filter_analyze_ehlo(&mut spam_ctx).await;\n                }\n                \"bounce\" => {\n                    server.spam_filter_analyze_mime(&mut spam_ctx).await;\n                    server.spam_filter_analyze_headers(&mut spam_ctx).await;\n                    server.spam_filter_analyze_rules(&mut spam_ctx).await;\n                    spam_ctx.result.tags.retain(|t| !t.starts_with(\"X_HDR_\"));\n                }\n                \"rbl\" => {\n                    server.spam_filter_analyze_url(&mut spam_ctx).await;\n                    server.spam_filter_analyze_ip(&mut spam_ctx).await;\n                    server.spam_filter_analyze_domain(&mut spam_ctx).await;\n                }\n                \"spamtrap\" => {\n                    server.spam_filter_analyze_spam_trap(&mut spam_ctx).await;\n                    server.spam_filter_finalize(&mut spam_ctx).await;\n                }\n                \"classifier\" => {\n                    server.spam_filter_analyze_classify(&mut spam_ctx).await;\n                    match server.spam_filter_finalize(&mut spam_ctx).await {\n                        SpamFilterAction::Allow(r) => spam_ctx.result.tags.extend(\n                            r.headers\n                                .split_ascii_whitespace()\n                                .filter(|t| t.starts_with(\"PROB_\"))\n                                .map(|t| t.to_string()),\n                        ),\n                        _ => unreachable!(),\n                    }\n                }\n                \"pyzor\" => {\n                    server.spam_filter_analyze_pyzor(&mut spam_ctx).await;\n                }\n                \"llm\" => {\n                    server.spam_filter_analyze_llm(&mut spam_ctx).await;\n                }\n                _ => panic!(\"Invalid test {test_name:?}\"),\n            }\n\n            // Compare tags\n            if spam_ctx.result.tags != expected_tags {\n                for tag in &spam_ctx.result.tags {\n                    if !expected_tags.contains(tag) {\n                        println!(\"Unexpected tag: {tag:?}\");\n                    }\n                }\n\n                for tag in &expected_tags {\n                    if !spam_ctx.result.tags.contains(tag) {\n                        println!(\"Missing tag: {tag:?}\");\n                    }\n                }\n\n                panic!(\"Tags mismatch, expected {expected_tags:?}\");\n            } else {\n                println!(\"Tags matched: {expected_tags:?}\");\n            }\n        }\n    }\n}\n\nasync fn classifier_features(server: &Server, contents: String) {\n    let mut num_tests = 0;\n\n    for test in contents.split(\"<!-- NEXT TEST -->\") {\n        let test = test.trim();\n        if test.is_empty() {\n            continue;\n        }\n\n        let (input, expected) = test.split_once(\"<!-- EXPECT -->\").unwrap();\n        let input = input.trim();\n        let expected = expected.trim();\n\n        // Build features\n        let message = MessageParser::new().parse(input).unwrap_or_default();\n        let mut ctx =\n            server.spam_filter_init(SpamFilterInput::from_message(&message, 0).train_mode());\n        server.spam_filter_analyze_domain(&mut ctx).await;\n        server.spam_filter_analyze_url(&mut ctx).await;\n        let mut tokens = server\n            .spam_build_tokens(&ctx)\n            .await\n            .0\n            .into_keys()\n            .collect::<Vec<_>>();\n        tokens.sort();\n\n        assert!(!tokens.is_empty(), \"No tokens parsed for input: {}\", input);\n        let expected_tokens: Vec<Token<'_>> = serde_json::from_str(expected).unwrap();\n\n        if tokens != expected_tokens {\n            eprintln!(\"Input: {}\", input);\n            eprintln!(\"Expected Tokens: {}\", expected);\n            eprintln!(\n                \"Parsed Tokens:   {}\",\n                serde_json::to_string_pretty(&tokens).unwrap()\n            );\n            panic!(\"Tokens do not match\");\n        }\n        num_tests += 1;\n    }\n\n    assert_eq!(num_tests, 11, \"Expected number of tests to run\");\n}\n\nfn html_tokens(contents: String) {\n    let mut num_tests = 0;\n\n    for test in contents.split(\"<!-- NEXT TEST -->\") {\n        let test = test.trim();\n        if test.is_empty() {\n            continue;\n        }\n\n        let (input, expected) = test.split_once(\"<!-- EXPECT -->\").unwrap();\n        let input = input.trim();\n        let expected = expected.trim();\n\n        let tokens = html_to_tokens(input);\n        assert!(!tokens.is_empty(), \"No tokens parsed for input: {}\", input);\n        let expected_tokens: Vec<HtmlToken> = serde_json::from_str(expected).unwrap();\n\n        assert_eq!(tokens, expected_tokens, \"Input: {}\", input);\n        num_tests += 1;\n    }\n\n    assert_eq!(num_tests, 12, \"Expected number of tests to run\");\n}\n\ntrait ParseConfigValue: Sized {\n    fn from_str(value: &str) -> Self;\n}\n\nimpl ParseConfigValue for SpfResult {\n    fn from_str(value: &str) -> Self {\n        match value {\n            \"pass\" => SpfResult::Pass,\n            \"fail\" => SpfResult::Fail,\n            \"softfail\" => SpfResult::SoftFail,\n            \"neutral\" => SpfResult::Neutral,\n            \"none\" => SpfResult::None,\n            \"temperror\" => SpfResult::TempError,\n            \"permerror\" => SpfResult::PermError,\n            _ => panic!(\"Invalid SPF result\"),\n        }\n    }\n}\n\nimpl ParseConfigValue for IprevResult {\n    fn from_str(value: &str) -> Self {\n        match value {\n            \"pass\" => IprevResult::Pass,\n            \"fail\" => IprevResult::Fail(mail_auth::Error::NotAligned),\n            \"temperror\" => IprevResult::TempError(mail_auth::Error::NotAligned),\n            \"permerror\" => IprevResult::PermError(mail_auth::Error::NotAligned),\n            \"none\" => IprevResult::None,\n            _ => panic!(\"Invalid IPREV result\"),\n        }\n    }\n}\n\nimpl ParseConfigValue for DkimResult {\n    fn from_str(value: &str) -> Self {\n        match value {\n            \"pass\" => DkimResult::Pass,\n            \"none\" => DkimResult::None,\n            \"neutral\" => DkimResult::Neutral(mail_auth::Error::NotAligned),\n            \"fail\" => DkimResult::Fail(mail_auth::Error::NotAligned),\n            \"permerror\" => DkimResult::PermError(mail_auth::Error::NotAligned),\n            \"temperror\" => DkimResult::TempError(mail_auth::Error::NotAligned),\n            _ => panic!(\"Invalid DKIM result\"),\n        }\n    }\n}\n\nimpl ParseConfigValue for DmarcResult {\n    fn from_str(value: &str) -> Self {\n        match value {\n            \"pass\" => DmarcResult::Pass,\n            \"fail\" => DmarcResult::Fail(mail_auth::Error::NotAligned),\n            \"temperror\" => DmarcResult::TempError(mail_auth::Error::NotAligned),\n            \"permerror\" => DmarcResult::PermError(mail_auth::Error::NotAligned),\n            \"none\" => DmarcResult::None,\n            _ => panic!(\"Invalid DMARC result\"),\n        }\n    }\n}\n\nimpl ParseConfigValue for Policy {\n    fn from_str(value: &str) -> Self {\n        match value {\n            \"reject\" => Policy::Reject,\n            \"quarantine\" => Policy::Quarantine,\n            \"none\" => Policy::None,\n            _ => panic!(\"Invalid DMARC policy\"),\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/asn.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\n#[cfg(test)]\nmod tests {\n    use std::time::{Duration, Instant};\n\n    use common::{Core, Server, config::network::AsnGeoLookupConfig};\n\n    #[tokio::test]\n    #[ignore]\n    async fn lookup_asn_country_dns() {\n        let mut core = Core::default();\n        core.network.asn_geo_lookup = AsnGeoLookupConfig::Dns {\n            zone_ipv4: \"origin.asn.cymru.com\".to_string(),\n            zone_ipv6: \"origin6.asn.cymru.com\".to_string(),\n            separator: '|'.to_string(),\n            index_asn: 0,\n            index_asn_name: 3.into(),\n            index_country: 2.into(),\n        };\n        let server = Server {\n            core: core.into(),\n            inner: Default::default(),\n        };\n\n        for (ip, asn, asn_name, country) in [\n            (\"8.8.8.8\", 15169, \"arin\", \"US\"),\n            (\"1.1.1.1\", 13335, \"apnic\", \"AU\"),\n            (\"2a01:4f9:c011:b43c::1\", 24940, \"ripencc\", \"DE\"),\n            (\"1.33.1.1\", 2514, \"apnic\", \"JP\"),\n        ] {\n            let result = server.lookup_asn_country(ip.parse().unwrap()).await;\n            println!(\"{ip}: {result:?}\");\n            assert_eq!(result.asn.as_ref().map(|r| r.id), Some(asn));\n            assert_eq!(\n                result.asn.as_ref().and_then(|r| r.name.as_deref()),\n                Some(asn_name)\n            );\n            assert_eq!(result.country.as_ref().map(|s| s.as_str()), Some(country));\n        }\n    }\n\n    #[tokio::test]\n    #[ignore]\n    async fn lookup_asn_country_http() {\n        let mut core = Core::default();\n        core.network.asn_geo_lookup = AsnGeoLookupConfig::Resource {\n            expires: Duration::from_secs(86400),\n            timeout: Duration::from_secs(100),\n            max_size: 100 * 1024 * 1024,\n            headers: Default::default(),\n            asn_resources: vec![\n                //url: \"file:///Users/me/code/playground/asn-ipv4.csv\".to_string(),\n                //url: \"file:///Users/me/code/playground/asn-ipv6.csv\".to_string(),\n                \"https://cdn.jsdelivr.net/npm/@ip-location-db/asn/asn-ipv4.csv\".to_string(),\n                \"https://cdn.jsdelivr.net/npm/@ip-location-db/asn/asn-ipv6.csv\".to_string(),\n            ],\n            geo_resources: vec![\n                //url: \"file:///Users/me/code/playground/geolite2-geo-whois-asn-country-ipv4.csv\"\n                //    .to_string(),\n                //url: \"file:///Users/me/code/playground/geolite2-geo-whois-asn-country-ipv6.csv\"\n                //    .to_string(),\n                concat!(\n                    \"https://cdn.jsdelivr.net/npm/@ip-location-db/geolite2-geo-whois-\",\n                    \"asn-country/geolite2-geo-whois-asn-country-ipv4.csv\"\n                )\n                .to_string(),\n                concat!(\n                    \"https://cdn.jsdelivr.net/npm/@ip-location-db/geolite2-geo-whois-\",\n                    \"asn-country/geolite2-geo-whois-asn-country-ipv6.csv\"\n                )\n                .to_string(),\n            ],\n        };\n        let server = Server {\n            core: core.into(),\n            inner: Default::default(),\n        };\n\n        server.lookup_asn_country(\"8.8.8.8\".parse().unwrap()).await;\n        let time = Instant::now();\n        loop {\n            tokio::time::sleep(Duration::from_millis(500)).await;\n            if server.inner.data.asn_geo_data.lock.available_permits() > 0 {\n                break;\n            }\n        }\n        println!(\"Fetch took {:?}\", time.elapsed());\n\n        for (ip, asn, asn_name, country) in [\n            (\"8.8.8.8\", 15169, \"Google LLC\", \"US\"),\n            (\"1.1.1.1\", 13335, \"Cloudflare, Inc.\", \"AU\"),\n            (\"2a01:4f9:c011:b43c::1\", 24940, \"Hetzner Online GmbH\", \"FI\"),\n            (\"1.33.1.1\", 2514, \"NTT PC Communications, Inc.\", \"JP\"),\n        ] {\n            let result = server.lookup_asn_country(ip.parse().unwrap()).await;\n            println!(\"{ip}: {result:?}\");\n            assert_eq!(result.asn.as_ref().map(|r| r.id), Some(asn));\n            assert_eq!(\n                result.asn.as_ref().and_then(|r| r.name.as_deref()),\n                Some(asn_name)\n            );\n            assert_eq!(result.country.as_ref().map(|s| s.as_str()), Some(country));\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/auth.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Core;\n\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::{\n    AssertConfig,\n    smtp::{\n        TempDir, TestSMTP,\n        session::{TestSession, VerifyResponse},\n    },\n};\nuse smtp::core::{Session, State};\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\ndirectory = \"local\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\ndescription = \"John Doe\"\nsecret = \"secret\"\nemail = [\"john@example.org\", \"jdoe@example.org\", \"john.doe@example.org\"]\nemail-list = [\"info@example.org\"]\nmember-of = [\"sales\"]\n\n[[directory.\"local\".principals]]\nname = \"jane\"\ndescription = \"Jane Doe\"\nsecret = \"p4ssw0rd\"\nemail = \"jane@example.org\"\nemail-list = [\"info@example.org\"]\nmember-of = [\"sales\", \"support\"]\n\n[session.auth]\nrequire = [{if = \"remote_ip = '10.0.0.1'\", then = true},\n           {else = false}]\nmechanisms = [{if = \"remote_ip = '10.0.0.1' && is_tls\", then = \"[plain, login]\"},\n              {else = 0}]\ndirectory = [{if = \"remote_ip = '10.0.0.1'\", then = \"'local'\"},\n             {else = false}]\nmust-match-sender = true\n\n[session.auth.errors]\ntotal = [{if = \"remote_ip = '10.0.0.1'\", then = 2},\n              {else = 3}]\nwait = \"100ms\"\n\n[session.extensions]\nfuture-release = [{if = '!is_empty(authenticated_as)', then = '1d'},\n                  {else = false}]\n\"#;\n\n#[tokio::test]\nasync fn auth() {\n    // Enable logging\n    crate::enable_logging();\n\n    let tmp_dir = TempDir::new(\"smtp_auth_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    config.assert_no_errors();\n\n    // EHLO should not advertise plain text auth without TLS\n    let mut session = Session::test(TestSMTP::from_core(core).server);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.stream.tls = false;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_not_contains(\" PLAIN\")\n        .assert_not_contains(\" LOGIN\");\n\n    // EHLO should advertise AUTH for 10.0.0.1\n    session.stream.tls = true;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_contains(\"AUTH \")\n        .assert_contains(\" PLAIN\")\n        .assert_contains(\" LOGIN\")\n        .assert_not_contains(\"FUTURERELEASE\");\n\n    // Invalid password should be rejected\n    session\n        .cmd(\"AUTH PLAIN AGpvaG4AY2hpbWljaGFuZ2Fz\", \"535 5.7.8\")\n        .await;\n\n    // Session should be disconnected after second invalid auth attempt\n    session\n        .ingest(b\"AUTH PLAIN AGpvaG4AY2hpbWljaGFuZ2Fz\\r\\n\")\n        .await\n        .unwrap_err();\n    session.response().assert_code(\"455 4.3.0\");\n\n    // Should not be able to send without authenticating\n    session.state = State::default();\n    session.mail_from(\"bill@foobar.org\", \"503 5.5.1\").await;\n\n    // Successful PLAIN authentication\n    session.data.auth_errors = 0;\n    session\n        .cmd(\"AUTH PLAIN AGpvaG4Ac2VjcmV0\", \"235 2.7.0\")\n        .await;\n\n    // Users should be able to send emails only from their own email addresses\n    session.mail_from(\"bill@foobar.org\", \"501 5.5.4\").await;\n    session.mail_from(\"john@example.org\", \"250\").await;\n    session.data.mail_from.take();\n\n    // Should not be able to authenticate twice\n    session\n        .cmd(\"AUTH PLAIN AGpvaG4Ac2VjcmV0\", \"503 5.5.1\")\n        .await;\n\n    // FUTURERELEASE extension should be available after authenticating\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_not_contains(\"AUTH \")\n        .assert_not_contains(\" PLAIN\")\n        .assert_not_contains(\" LOGIN\")\n        .assert_contains(\"FUTURERELEASE 86400\");\n\n    // Successful LOGIN authentication\n    session.data.authenticated_as.take();\n    session.cmd(\"AUTH LOGIN\", \"334\").await;\n    session.cmd(\"amFuZQ==\", \"334\").await;\n    session.cmd(\"cDRzc3cwcmQ=\", \"235 2.7.0\").await;\n\n    // Login should not be advertised to 10.0.0.2\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.eval_session_params().await;\n    session.stream.tls = true;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_not_contains(\"AUTH \")\n        .assert_not_contains(\" PLAIN\")\n        .assert_not_contains(\" LOGIN\");\n    session\n        .cmd(\"AUTH PLAIN AGpvaG4Ac2VjcmV0\", \"503 5.5.1\")\n        .await;\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/basic.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Core;\nuse smtp::core::Session;\n\nuse crate::smtp::{\n    TestSMTP,\n    session::{TestSession, VerifyResponse},\n};\n\n#[tokio::test]\nasync fn basic_commands() {\n    // Enable logging\n    crate::enable_logging();\n\n    let mut session = Session::test(TestSMTP::from_core(Core::default()).server);\n\n    // STARTTLS should be available on clear text connections\n    session.stream.tls = false;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_contains(\"STARTTLS\");\n    assert!(!session.ingest(b\"STARTTLS\\r\\n\").await.unwrap());\n    session.response().assert_contains(\"220 2.0.0\");\n\n    // STARTTLS should not be offered on TLS connections\n    session.stream.tls = true;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_not_contains(\"STARTTLS\");\n    session.cmd(\"STARTTLS\", \"504 5.7.4\").await;\n\n    // Test NOOP\n    session.cmd(\"NOOP\", \"250\").await;\n\n    // Test RSET\n    session.cmd(\"RSET\", \"250\").await;\n\n    // Test HELP\n    session.cmd(\"HELP QUIT\", \"250\").await;\n\n    // Test LHLO on SMTP channel\n    session.cmd(\"LHLO domain.org\", \"502\").await;\n\n    // Test QUIT\n    session.ingest(b\"QUIT\\r\\n\").await.unwrap_err();\n    session.response().assert_code(\"221\");\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/data.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Core;\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::{\n    AssertConfig,\n    smtp::{\n        TempDir, TestSMTP,\n        inbound::TestMessage,\n        session::{TestSession, VerifyResponse, load_test_message},\n    },\n    store::cleanup::store_assert_is_empty,\n};\nuse smtp::core::Session;\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\ndirectory = \"local\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n[spam-filter]\nenable = false\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\ndescription = \"John Doe\"\nsecret = \"secret\"\nemail = [\"john@foobar.org\", \"jdoe@example.org\", \"john.doe@example.org\"]\n\n[[directory.\"local\".principals]]\nname = \"jane\"\ndescription = \"Jane Doe\"\nsecret = \"p4ssw0rd\"\nemail = \"jane@domain.net\"\n\n[[directory.\"local\".principals]]\nname = \"bill\"\ndescription = \"Bill Foobar\"\nsecret = \"p4ssw0rd\"\nemail = \"bill@foobar.org\"\n\n[[directory.\"local\".principals]]\nname = \"mike\"\ndescription = \"Mike Foobar\"\nsecret = \"p4ssw0rd\"\nemail = \"mike@test.com\"\n\n[session.rcpt]\ndirectory = \"'local'\"\n\n[session.data.limits]\nmessages = [{if = \"remote_ip = '10.0.0.1'\", then = 1},\n            {else = 100}]\nreceived-headers = 3\n\n[session.data.add-headers]\nreceived = [{if = \"remote_ip = '10.0.0.3'\", then = true},\n            {else = false}]\nreceived-spf =  [{if = \"remote_ip = '10.0.0.3'\", then = true},\n            {else = false}]\nauth-results =  [{if = \"remote_ip = '10.0.0.3'\", then = true},\n            {else = false}]\nmessage-id =  [{if = \"remote_ip = '10.0.0.3'\", then = true},\n               {else = false}]\ndate = [{if = \"remote_ip = '10.0.0.3'\", then = true},\n        {else = false}]\nreturn-path =  [{if = \"remote_ip = '10.0.0.3'\", then = true},\n            {else = false}]\n\n[[queue.quota]]\nmatch = \"sender = 'john@doe.org'\"\nkey = ['sender']\nmessages = 1\n\n[[queue.quota]]\nmatch = \"rcpt_domain = 'foobar.org'\"\nkey = ['rcpt_domain']\nsize = 450\nenable = true\n\n[[queue.quota]]\nmatch = \"rcpt = 'jane@domain.net'\"\nkey = ['rcpt']\nsize = 450\nenable = true\n\n\"#;\n\n#[tokio::test]\nasync fn data() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Create temp dir for queue\n    let tmp_dir = TempDir::new(\"smtp_data_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    config.assert_no_errors();\n\n    // Test queue message builder\n    let test = TestSMTP::from_core(core);\n    let mut qr = test.queue_receiver;\n    let mut session = Session::test(test.server.clone());\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.test_builder().await;\n\n    // Send DATA without RCPT\n    session.ehlo(\"mx.doe.org\").await;\n    session.ingest(b\"DATA\\r\\n\").await.unwrap();\n    session.response().assert_code(\"503 5.5.1\");\n\n    // Send broken message\n    session\n        .send_message(\"john@doe.org\", &[\"bill@foobar.org\"], \"invalid\", \"550 5.7.7\")\n        .await;\n\n    // Naive Loop detection\n    session\n        .send_message(\n            \"john@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:loop\",\n            \"450 4.4.6\",\n        )\n        .await;\n\n    // No headers should be added to messages from 10.0.0.1\n    session\n        .send_message(\"john@test.org\", &[\"mike@test.com\"], \"test:no_msgid\", \"250\")\n        .await;\n    assert_eq!(\n        qr.expect_message().await.read_message(&qr).await,\n        load_test_message(\"no_msgid\", \"messages\")\n    );\n\n    // Maximum one message per session is allowed for 10.0.0.1\n    session.mail_from(\"john@doe.org\", \"250\").await;\n    session.rcpt_to(\"bill@foobar.org\", \"250\").await;\n    session.ingest(b\"DATA\\r\\n\").await.unwrap();\n    session.response().assert_code(\"452 4.4.5\");\n    session.rset().await;\n\n    // Headers should be added to messages from 10.0.0.3\n    session.data.remote_ip_str = \"10.0.0.3\".into();\n    session.eval_session_params().await;\n    session\n        .send_message(\"bill@doe.org\", &[\"mike@test.com\"], \"test:no_msgid\", \"250\")\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"From: \")\n        .assert_contains(\"To: \")\n        .assert_contains(\"Subject: \")\n        .assert_contains(\"Date: \")\n        .assert_contains(\"Message-ID: \")\n        .assert_contains(\"Return-Path: \")\n        .assert_contains(\"Received: \")\n        .assert_contains(\"Authentication-Results: \")\n        .assert_contains(\"Received-SPF: \");\n\n    // Only one message is allowed in the queue from john@doe.org\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.eval_session_params().await;\n    session\n        .send_message(\"john@doe.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    session\n        .send_message(\n            \"john@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"452 4.3.1\",\n        )\n        .await;\n\n    // Release quota\n    qr.clear_queue(&test.server).await;\n\n    // Only 1500 bytes are allowed in the queue to domain foobar.org\n    session\n        .send_message(\n            \"jane@foobar.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    session\n        .send_message(\n            \"jane@foobar.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"452 4.3.1\",\n        )\n        .await;\n\n    // Only 1500 bytes are allowed in the queue to recipient jane@domain.net\n    session\n        .send_message(\n            \"jane@foobar.org\",\n            &[\"jane@domain.net\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    session\n        .send_message(\n            \"jane@foobar.org\",\n            &[\"jane@domain.net\"],\n            \"test:no_dkim\",\n            \"452 4.3.1\",\n        )\n        .await;\n\n    // Make sure store is empty\n    qr.clear_queue(&test.server).await;\n    store_assert_is_empty(test.server.store(), test.server.blob_store().clone(), false).await;\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/dmarc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::{Core, config::smtp::report::AggregateFrequency};\n\nuse mail_auth::{\n    common::{parse::TxtRecordParser, verify::DomainKey},\n    dkim::DomainKeyReport,\n    dmarc::Dmarc,\n    report::DmarcResult,\n    spf::Spf,\n};\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::smtp::{\n    DnsCache, TempDir, TestSMTP,\n    inbound::{TestMessage, TestReportingEvent, sign::SIGNATURES},\n    session::{TestSession, VerifyResponse},\n};\nuse smtp::core::Session;\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\ndescription = \"John Doe\"\nsecret = \"secret\"\nemail = [\"jdoe@example.com\"]\n\n[session.rcpt]\ndirectory = \"'local'\"\n\n[session.data.add-headers]\nreceived = true\nreceived-spf = true\nauth-results = true\nmessage-id = true\ndate = true\nreturn-path = false\n\n[report.dkim]\nsend = \"[1, 1s]\"\nsign = \"['rsa']\"\n\n[report.spf]\nsend = \"[1, 1s]\"\nsign = \"['rsa']\"\n\n[report.dmarc]\nsend = \"[1, 1s]\"\nsign = \"['rsa']\"\n\n[report.dmarc.aggregate]\nsend = \"daily\"\n\n[auth.spf.verify]\nehlo = [{if = \"remote_ip = '10.0.0.2'\", then = 'strict'},\n        { else = 'relaxed' }]\nmail-from = [{if = \"remote_ip = '10.0.0.2'\", then = 'strict'},\n             { else = 'relaxed' }]\n\n[auth.dmarc]\nverify = \"strict\"\n\n[auth.arc]\nverify = \"strict\"\n\n[auth.dkim]\nverify = [{if = \"sender_domain = 'test.net'\", then = 'relaxed'},\n         { else = 'strict' }]\n\n\"#;\n\n#[tokio::test]\nasync fn dmarc() {\n    // Enable logging\n    crate::enable_logging();\n\n    let tmp_dir = TempDir::new(\"smtp_dmarc_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG.to_string() + SIGNATURES)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    let test = TestSMTP::from_core(core);\n\n    // Add SPF, DKIM and DMARC records\n    test.server.txt_add(\n        \"mx.example.com\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 ip4:10.0.0.2 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"example.com\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 -all ra=spf-failures rr=e:f:s:n\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"foobar.com\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"ed._domainkey.example.com\",\n        DomainKey::parse(\n            concat!(\n                \"v=DKIM1; k=ed25519; \",\n                \"p=11qYAYKxCrfVS/7TyWQHOg7hcvPapiMlrwIaaPcHURo=\"\n            )\n            .as_bytes(),\n        )\n        .unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"default._domainkey.example.com\",\n        DomainKey::parse(\n            concat!(\n                \"v=DKIM1; t=s; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQ\",\n                \"KBgQDwIRP/UC3SBsEmGqZ9ZJW3/DkMoGeLnQg1fWn7/zYt\",\n                \"IxN2SnFCjxOCKG9v3b4jYfcTNh5ijSsq631uBItLa7od+v\",\n                \"/RtdC2UzJ1lWT947qR+Rcac2gbto/NMqJ0fzfVjH4OuKhi\",\n                \"tdY9tf6mcwGjaNBcWToIMmPSPDdQPNUYckcQ2QIDAQAB\",\n            )\n            .as_bytes(),\n        )\n        .unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"_report._domainkey.example.com\",\n        DomainKeyReport::parse(b\"ra=dkim-failures; rp=100; rr=d:o:p:s:u:v:x;\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"_dmarc.example.com\",\n        Dmarc::parse(\n            concat!(\n                \"v=DMARC1; p=reject; sp=quarantine; np=None; aspf=s; adkim=s; fo=1;\",\n                \"rua=mailto:dmarc-feedback@example.com;\",\n                \"ruf=mailto:dmarc-failures@example.com\"\n            )\n            .as_bytes(),\n        )\n        .unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n\n    // SPF must pass\n    let mut rr = test.report_receiver;\n    let mut qr = test.queue_receiver;\n    let mut session = Session::test(test.server.clone());\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.example.com\").await;\n    session.mail_from(\"bill@example.com\", \"550 5.7.23\").await;\n\n    // Expect SPF auth failure report\n    let message = qr.expect_message().await;\n    assert_eq!(\n        message.message.recipients.last().unwrap().address(),\n        \"spf-failures@example.com\"\n    );\n    message\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com;\")\n        .assert_contains(\"To: spf-failures@example.com\")\n        .assert_contains(\"Feedback-Type: auth-failure\")\n        .assert_contains(\"Auth-Failure: spf\");\n\n    // Second DKIM failure report should be rate limited\n    session.mail_from(\"bill@example.com\", \"550 5.7.23\").await;\n    qr.assert_no_events();\n\n    // Invalid DKIM signatures should be rejected\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    session.eval_session_params().await;\n    session\n        .send_message(\n            \"bill@example.com\",\n            &[\"jdoe@example.com\"],\n            \"test:invalid_dkim\",\n            \"550 5.7.20\",\n        )\n        .await;\n\n    // Expect DKIM auth failure report\n    let message = qr.expect_message().await;\n    assert_eq!(\n        message.message.recipients.last().unwrap().address(),\n        \"dkim-failures@example.com\"\n    );\n    message\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com;\")\n        .assert_contains(\"To: dkim-failures@example.com\")\n        .assert_contains(\"Feedback-Type: auth-failure\")\n        .assert_contains(\"Auth-Failure: bodyhash\");\n\n    // Second DKIM failure report should be rate limited\n    session\n        .send_message(\n            \"bill@example.com\",\n            &[\"jdoe@example.com\"],\n            \"test:invalid_dkim\",\n            \"550 5.7.20\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Invalid ARC should be rejected\n    session\n        .send_message(\n            \"bill@example.com\",\n            &[\"jdoe@example.com\"],\n            \"test:invalid_arc\",\n            \"550 5.7.29\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Unaligned DMARC should be rejected\n    test.server.txt_add(\n        \"test.net\",\n        Spf::parse(b\"v=spf1 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    session\n        .send_message(\n            \"joe@test.net\",\n            &[\"jdoe@example.com\"],\n            \"test:invalid_dkim\",\n            \"550 5.7.1\",\n        )\n        .await;\n\n    // Expect DMARC auth failure report\n    let message = qr.expect_message().await;\n    assert_eq!(\n        message.message.recipients.last().unwrap().address(),\n        \"dmarc-failures@example.com\"\n    );\n    message\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com;\")\n        .assert_contains(\"To: dmarc-failures@example.com\")\n        .assert_contains(\"Feedback-Type: auth-failure\")\n        .assert_contains(\"Auth-Failure: dmarc\")\n        .assert_contains(\"dmarc=3Dnone\");\n\n    // Expect DMARC aggregate report\n    let report = rr.read_report().await.unwrap_dmarc();\n    assert_eq!(report.domain, \"example.com\");\n    assert_eq!(report.interval, AggregateFrequency::Daily);\n    assert_eq!(report.dmarc_record.rua().len(), 1);\n    assert_eq!(report.report_record.dmarc_spf_result(), DmarcResult::Fail);\n\n    // Second DMARC failure report should be rate limited\n    session\n        .send_message(\n            \"joe@test.net\",\n            &[\"jdoe@example.com\"],\n            \"test:invalid_dkim\",\n            \"550 5.7.1\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Messages passing DMARC should be accepted\n    session\n        .send_message(\n            \"bill@example.com\",\n            &[\"jdoe@example.com\"],\n            \"test:dkim\",\n            \"250\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"dkim=pass\")\n        .assert_contains(\"spf=pass\")\n        .assert_contains(\"dmarc=pass\")\n        .assert_contains(\"Received-SPF: pass\");\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/ehlo.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::Core;\nuse mail_auth::{SpfResult, common::parse::TxtRecordParser, spf::Spf};\n\nuse smtp::core::Session;\nuse utils::config::Config;\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    session::{TestSession, VerifyResponse},\n};\n\nconst CONFIG: &str = r#\"\n[session.data.limits]\nsize = [{if = \"remote_ip = '10.0.0.1'\", then = 1024},\n        {else = 2048}]\n\n[session.extensions]\nfuture-release = [{if = \"remote_ip = '10.0.0.1'\", then = '1h'},\n                  {else = false}]\nmt-priority = [{if = \"remote_ip = '10.0.0.1'\", then = 'nsep'},\n               {else = false}]\n\n[session.ehlo]\nreject-non-fqdn = \"starts_with(remote_ip, '10.0.0.')\"\n\n[auth.spf.verify]\nehlo = [{if = \"remote_ip = '10.0.0.2'\", then = 'strict'},\n        {else = 'relaxed'}]\n\"#;\n\n#[tokio::test]\nasync fn ehlo() {\n    // Enable logging\n    crate::enable_logging();\n\n    let mut config = Config::new(CONFIG).unwrap();\n    let core = Core::parse(&mut config, Default::default(), Default::default()).await;\n    let server = TestSMTP::from_core(core).server;\n    server.txt_add(\n        \"mx1.foobar.org\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    server.txt_add(\n        \"mx2.foobar.org\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.2 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n\n    // Reject non-FQDN domains\n    let mut session = Session::test(server);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    session.stream.tls = false;\n    session.eval_session_params().await;\n    session.cmd(\"EHLO domain\", \"550 5.5.0\").await;\n\n    // EHLO capabilities evaluation\n    session\n        .cmd(\"EHLO mx1.foobar.org\", \"250\")\n        .await\n        .assert_contains(\"SIZE 1024\")\n        .assert_contains(\"MT-PRIORITY NSEP\")\n        .assert_contains(\"FUTURERELEASE 3600\")\n        .assert_contains(\"STARTTLS\");\n\n    // SPF should be a Pass for 10.0.0.1\n    assert_eq!(\n        session.data.spf_ehlo.as_ref().unwrap().result(),\n        SpfResult::Pass\n    );\n\n    // Test SPF strict mode\n    session.data.helo_domain = \"\".into();\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    session.stream.tls = true;\n    session.eval_session_params().await;\n    session.ingest(b\"EHLO mx1.foobar.org\\r\\n\").await.unwrap();\n    session.response().assert_code(\"550 5.7.23\");\n\n    // EHLO capabilities evaluation\n    session.ingest(b\"EHLO mx2.foobar.org\\r\\n\").await.unwrap();\n    assert_eq!(\n        session.data.spf_ehlo.as_ref().unwrap().result(),\n        SpfResult::Pass\n    );\n    session\n        .response()\n        .assert_code(\"250\")\n        .assert_contains(\"SIZE 2048\")\n        .assert_not_contains(\"MT-PRIORITY\")\n        .assert_not_contains(\"FUTURERELEASE\")\n        .assert_not_contains(\"STARTTLS\");\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/limits.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::Core;\nuse tokio::sync::watch;\n\nuse smtp::core::Session;\nuse utils::config::Config;\n\nuse crate::smtp::{\n    TestSMTP,\n    session::{TestSession, VerifyResponse},\n};\n\nconst CONFIG: &str = r#\"\n[session]\ntransfer-limit = [{if = \"remote_ip = '10.0.0.1'\", then = 10},\n                 {else = 1024}]\ntimeout = [{if = \"remote_ip = '10.0.0.2'\", then = '500ms'},\n           {else = '30m'}]\nduration = [{if = \"remote_ip = '10.0.0.3'\", then = '500ms'},\n            {else = '60m'}]\n\"#;\n\n#[tokio::test]\nasync fn limits() {\n    // Enable logging\n    crate::enable_logging();\n\n    let mut config = Config::new(CONFIG).unwrap();\n    let core = Core::parse(&mut config, Default::default(), Default::default()).await;\n\n    let (_tx, rx) = watch::channel(true);\n\n    // Exceed max line length\n    let mut session = Session::test_with_shutdown(TestSMTP::from_core(core).server, rx);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    let mut buf = vec![b'A'; 4097];\n    session.ingest(&buf).await.unwrap();\n    session.ingest(b\"\\r\\n\").await.unwrap();\n    session.response().assert_code(\"554 5.3.4\");\n\n    // Invalid command\n    buf.extend_from_slice(b\"\\r\\n\");\n    session.ingest(&buf).await.unwrap();\n    session.response().assert_code(\"500 5.5.1\");\n\n    // Exceed transfer quota\n    session.eval_session_params().await;\n    session.write_rx(\"MAIL FROM:<this_is_a_long@command_over_10_chars.com>\\r\\n\");\n    session.handle_conn().await;\n    session.response().assert_code(\"452 4.7.28\");\n\n    // Loitering\n    session.data.remote_ip_str = \"10.0.0.3\".into();\n    session.data.valid_until = Instant::now();\n    session.eval_session_params().await;\n    tokio::time::sleep(Duration::from_millis(600)).await;\n    session.write_rx(\"MAIL FROM:<this_is_a_long@command_over_10_chars.com>\\r\\n\");\n    session.handle_conn().await;\n    session.response().assert_code(\"421 4.3.2\");\n\n    // Timeout\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.data.valid_until = Instant::now();\n    session.eval_session_params().await;\n    session.write_rx(\"MAIL FROM:<this_is_a_long@command_over_10_chars.com>\\r\\n\");\n    session.handle_conn().await;\n    session.response().assert_code(\"221 2.0.0\");\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/mail.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant, SystemTime};\n\nuse common::Core;\nuse mail_auth::{IprevResult, SpfResult, common::parse::TxtRecordParser, spf::Spf};\nuse smtp_proto::{MAIL_BY_NOTIFY, MAIL_BY_RETURN, MAIL_REQUIRETLS};\n\nuse smtp::core::Session;\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::smtp::{\n    DnsCache, TempDir, TestSMTP,\n    session::{TestSession, VerifyResponse},\n};\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/data.db\"\n\n[session.ehlo]\nrequire = true\n\n[auth.spf.verify]\nehlo = 'relaxed'\nmail-from = [{if = \"remote_ip = '10.0.0.2'\", then = 'strict'},\n             {else = 'relaxed'}]\n\n[auth.iprev]\nverify = [{if = \"remote_ip = '10.0.0.2'\", then = 'strict'},\n          {else = 'relaxed'}]\n\n[session.extensions]\nfuture-release = [{if = \"remote_ip = '10.0.0.2'\", then = '1d'},\n                  {else = false}]\ndeliver-by = [{if = \"remote_ip = '10.0.0.2'\", then = '1d'},\n             {else = false}]\nrequiretls = [{if = \"remote_ip = '10.0.0.2'\", then = true},\n            {else = false}]\nmt-priority = [{if = \"remote_ip = '10.0.0.2'\", then = 'nsep'},\n               {else = false}]\n\n[session.mail]\nis-allowed = \"sender_domain != 'blocked.com'\"\n\n[session.data.limits]\nsize = [{if = \"remote_ip = '10.0.0.2'\", then = 2048},\n        {else = 1024}]\n\n[[queue.limiter.inbound]]\nmatch = \"remote_ip = '10.0.0.1'\"\nkey = 'sender'\nrate = '2/1s'\nenable = true\n\n\"#;\n\n#[tokio::test]\nasync fn mail() {\n    // Enable logging\n    crate::enable_logging();\n\n    let tmp_dir = TempDir::new(\"smtp_mail_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    let server = TestSMTP::from_core(core).server;\n\n    server.txt_add(\n        \"foobar.org\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    server.txt_add(\n        \"mx1.foobar.org\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    server.ptr_add(\n        \"10.0.0.1\".parse().unwrap(),\n        vec![\"mx1.foobar.org.\".to_string()],\n        Instant::now() + Duration::from_secs(5),\n    );\n    server.ipv4_add(\n        \"mx1.foobar.org.\",\n        vec![\"10.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(5),\n    );\n    server.ptr_add(\n        \"10.0.0.2\".parse().unwrap(),\n        vec![\"mx2.foobar.org.\".to_string()],\n        Instant::now() + Duration::from_secs(5),\n    );\n\n    // Be rude and do not say EHLO\n    let mut session = Session::test(server.clone());\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    session.eval_session_params().await;\n    session\n        .ingest(b\"MAIL FROM:<bill@foobar.org>\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"503 5.5.1\");\n\n    // Test sender not allowed\n    session.ingest(b\"EHLO mx1.foobar.org\\r\\n\").await.unwrap();\n    session.response().assert_code(\"250\");\n    session\n        .ingest(b\"MAIL FROM:<bill@blocked.com>\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"550 5.7.1\");\n\n    // Both IPREV and SPF should pass\n    session\n        .ingest(b\"MAIL FROM:<bill@foobar.org>\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    assert_eq!(\n        session.data.spf_ehlo.as_ref().unwrap().result(),\n        SpfResult::Pass\n    );\n    assert_eq!(\n        session.data.spf_mail_from.as_ref().unwrap().result(),\n        SpfResult::Pass\n    );\n    assert_eq!(\n        session.data.iprev.as_ref().unwrap().result(),\n        &IprevResult::Pass\n    );\n\n    // Multiple MAIL FROMs should not be allowed\n    session\n        .ingest(b\"MAIL FROM:<bill@foobar.org>\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"503 5.5.1\");\n\n    // Test rate limit\n    for n in 0..2 {\n        session.rset().await;\n        session\n            .ingest(b\"MAIL FROM:<bill@foobar.org>\\r\\n\")\n            .await\n            .unwrap();\n        session\n            .response()\n            .assert_code(if n == 0 { \"250\" } else { \"452 4.4.5\" });\n    }\n\n    // Test disabled extensions\n    for param in [\n        \"HOLDFOR=123\",\n        \"HOLDUNTIL=49374347\",\n        \"MT-PRIORITY=3\",\n        \"BY=120;R\",\n        \"REQUIRETLS\",\n    ] {\n        session\n            .ingest(format!(\"MAIL FROM:<params@foobar.org> {param}\\r\\n\").as_bytes())\n            .await\n            .unwrap();\n        session.response().assert_code(\"501 5.5.4\");\n    }\n\n    // Test size with a large value\n    session\n        .ingest(b\"MAIL FROM:<bill@foobar.org> SIZE=1512\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"552 5.3.4\");\n\n    // Test strict IPREV\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    session.data.iprev = None;\n    session.eval_session_params().await;\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org>\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"550 5.7.25\");\n    session.data.iprev = None;\n    server.ipv4_add(\n        \"mx2.foobar.org.\",\n        vec![\"10.0.0.2\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(5),\n    );\n\n    // Test strict SPF\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org>\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"550 5.7.23\");\n    server.txt_add(\n        \"foobar.org\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 ip4:10.0.0.2 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    session\n        .ingest(b\"MAIL FROM:<Jane@FooBar.org>\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    let mail_from = session.data.mail_from.as_ref().unwrap();\n    assert_eq!(mail_from.domain, \"foobar.org\");\n    assert_eq!(mail_from.address, \"Jane@FooBar.org\");\n    assert_eq!(mail_from.address_lcase, \"jane@foobar.org\");\n    session.rset().await;\n\n    // Test SIZE extension\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> SIZE=1023\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    session.rset().await;\n\n    // Test MT-PRIORITY extension\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> MT-PRIORITY=-3\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    assert_eq!(session.data.priority, -3);\n    session.rset().await;\n\n    // Test REQUIRETLS extension\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> REQUIRETLS\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    assert!((session.data.mail_from.as_ref().unwrap().flags & MAIL_REQUIRETLS) != 0);\n    session.rset().await;\n\n    // Test DELIVERBY extension with by-mode=R\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> BY=120;R\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    assert!((session.data.mail_from.as_ref().unwrap().flags & MAIL_BY_RETURN) != 0);\n    assert_eq!(session.data.delivery_by, 120);\n    session.rset().await;\n\n    // Test DELIVERBY extension with by-mode=N\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> BY=-456;N\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    assert!((session.data.mail_from.as_ref().unwrap().flags & MAIL_BY_NOTIFY) != 0);\n    assert_eq!(session.data.delivery_by, -456);\n    session.rset().await;\n\n    // Test DELIVERBY extension with invalid by-mode=R\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> BY=-1;R\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"501 5.5.4\");\n    session.rset().await;\n\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> BY=99999;R\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"501 5.5.4\");\n    session.rset().await;\n\n    // Test FUTURERELEASE extension with HOLDFOR\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> HOLDFOR=1234\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    assert_eq!(session.data.future_release, 1234);\n    session.rset().await;\n\n    // Test FUTURERELEASE extension with invalid HOLDFOR falue\n    session\n        .ingest(b\"MAIL FROM:<jane@foobar.org> HOLDFOR=99999\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"501 5.5.4\");\n    session.rset().await;\n\n    // Test FUTURERELEASE extension with HOLDUNTIL\n    let now = SystemTime::now()\n        .duration_since(SystemTime::UNIX_EPOCH)\n        .map_or(0, |d| d.as_secs());\n    session\n        .ingest(format!(\"MAIL FROM:<jane@foobar.org> HOLDUNTIL={}\\r\\n\", now + 10).as_bytes())\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    assert_eq!(session.data.future_release, 10);\n    session.rset().await;\n\n    // Test FUTURERELEASE extension with invalid HOLDUNTIL value\n    session\n        .ingest(format!(\"MAIL FROM:<jane@foobar.org> HOLDUNTIL={}\\r\\n\", now + 99999).as_bytes())\n        .await\n        .unwrap();\n    session.response().assert_code(\"501 5.5.4\");\n    session.rset().await;\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/milter.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{fs, net::SocketAddr, path::PathBuf, sync::Arc, time::Duration};\n\nuse ahash::AHashSet;\nuse common::{\n    Core,\n    config::smtp::session::{Milter, MilterVersion, Stage},\n    expr::if_block::IfBlock,\n    manager::webadmin::Resource,\n};\n\nuse http_proto::{ToHttpResponse, request::fetch_body};\nuse hyper::{body, server::conn::http1, service::service_fn};\nuse hyper_util::rt::TokioIo;\nuse mail_auth::AuthenticatedMessage;\nuse mail_parser::MessageParser;\nuse serde::Deserialize;\nuse smtp::{\n    core::{Session, SessionData},\n    inbound::{\n        hooks::{self, Request, SmtpResponse},\n        milter::{\n            Action, Command, Macros, MilterClient, Modification, Options, Response,\n            receiver::{FrameResult, Receiver},\n        },\n    },\n};\nuse store::Stores;\nuse tokio::{\n    io::{AsyncReadExt, AsyncWriteExt},\n    net::{TcpListener, TcpStream},\n    sync::watch,\n};\nuse utils::config::Config;\n\nuse crate::smtp::{\n    TempDir, TestSMTP,\n    inbound::TestMessage,\n    session::{TestSession, VerifyResponse, load_test_message},\n};\n\n#[derive(Debug, Deserialize)]\nstruct HeaderTest {\n    modifications: Vec<Modification>,\n    result: String,\n}\n\nconst CONFIG_MILTER: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n[session.rcpt]\nrelay = true\n\n[[session.milter]]\nhostname = \"127.0.0.1\"\nport = 9332\n#port = 11332\n#port = 7357\nenable = true\noptions.version = 6\ntls = false\nstages = [\"data\"]\n\n\"#;\n\nconst CONFIG_JMILTER: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n[session.rcpt]\nrelay = true\n\n[[session.hook]]\nurl = \"http://127.0.0.1:9333\"\nenable = true\nstages = [\"data\"]\n\"#;\n\n#[tokio::test]\nasync fn milter_session() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Configure tests\n    let tmp_dir = TempDir::new(\"smtp_milter_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG_MILTER)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    let _rx = spawn_mock_milter_server();\n    tokio::time::sleep(Duration::from_millis(100)).await;\n\n    // Build session\n    let test = TestSMTP::from_core(core);\n    let mut qr = test.queue_receiver;\n    let mut session = Session::test(test.server);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.doe.org\").await;\n\n    // Test reject\n    session\n        .send_message(\n            \"reject@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"503 5.5.3\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test discard\n    session\n        .send_message(\n            \"discard@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test temp fail\n    session\n        .send_message(\n            \"temp_fail@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"451 4.3.5\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test shutdown\n    session\n        .send_message(\n            \"shutdown@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"421 4.3.0\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test reply code\n    session\n        .send_message(\n            \"reply_code@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"321\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test accept with header addition\n    session\n        .send_message(\n            \"0@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"X-Hello: World\")\n        .assert_contains(\"Subject: Is dinner ready?\")\n        .assert_contains(\"Are you hungry yet?\");\n\n    // Test accept with header replacement\n    session\n        .send_message(\n            \"3@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"Subject: [SPAM] Saying Hello\")\n        .assert_count(\"References: \", 1)\n        .assert_contains(\"Are you hungry yet?\");\n\n    // Test accept with body replacement\n    session\n        .send_message(\n            \"2@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"X-Spam: Yes\")\n        .assert_contains(\"123456\");\n}\n\n#[tokio::test]\nasync fn mta_hook_session() {\n    // Enable logging\n    /*let disable = \"true\";\n    tracing::subscriber::set_global_default(\n        tracing_subscriber::FmtSubscriber::builder()\n            .with_max_level(tracing::Level::TRACE)\n            .finish(),\n    )\n    .unwrap();*/\n\n    // Configure tests\n    let tmp_dir = TempDir::new(\"smtp_mta_hook_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG_JMILTER)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    let _rx = spawn_mock_mta_hook_server();\n    tokio::time::sleep(Duration::from_millis(100)).await;\n\n    // Build session\n    let test = TestSMTP::from_core(core);\n    let mut qr = test.queue_receiver;\n    let mut session = Session::test(test.server);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.doe.org\").await;\n\n    // Test reject\n    session\n        .send_message(\n            \"reject@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"503 5.5.3\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test discard\n    session\n        .send_message(\n            \"discard@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test temp fail\n    session\n        .send_message(\n            \"temp_fail@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"451 4.3.5\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test shutdown\n    session\n        .send_message(\n            \"shutdown@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"421 4.3.0\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test reply code\n    session\n        .send_message(\n            \"reply_code@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"321\",\n        )\n        .await;\n    qr.assert_no_events();\n\n    // Test accept with header addition\n    session\n        .send_message(\n            \"0@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"X-Hello: World\")\n        .assert_contains(\"Subject: Is dinner ready?\")\n        .assert_contains(\"Are you hungry yet?\");\n\n    // Test accept with header replacement\n    session\n        .send_message(\n            \"3@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"Subject: [SPAM] Saying Hello\")\n        .assert_count(\"References: \", 1)\n        .assert_contains(\"Are you hungry yet?\");\n\n    // Test accept with body replacement\n    session\n        .send_message(\n            \"2@doe.org\",\n            &[\"bill@foobar.org\"],\n            \"test:no_dkim\",\n            \"250 2.0.0\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"X-Spam: Yes\")\n        .assert_contains(\"123456\");\n}\n\n#[test]\nfn milter_address_modifications() {\n    let test_message = fs::read_to_string(\n        PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n            .join(\"resources\")\n            .join(\"smtp\")\n            .join(\"milter\")\n            .join(\"message.eml\"),\n    )\n    .unwrap();\n    let parsed_test_message = AuthenticatedMessage::parse(test_message.as_bytes()).unwrap();\n\n    let mut data = SessionData::new(\n        \"127.0.0.1\".parse().unwrap(),\n        0,\n        \"127.0.0.1\".parse().unwrap(),\n        0,\n        Default::default(),\n        0,\n    );\n\n    // ChangeFrom\n    assert!(\n        data.apply_milter_modifications(\n            vec![Modification::ChangeFrom {\n                sender: \"<>\".into(),\n                args: \"\".into(),\n            }],\n            &parsed_test_message\n        )\n        .is_none()\n    );\n    let addr = data.mail_from.as_ref().unwrap();\n    assert_eq!(addr.address_lcase, \"\");\n    assert_eq!(addr.dsn_info, None);\n    assert_eq!(addr.flags, 0);\n\n    // ChangeFrom with parameters\n    assert!(\n        data.apply_milter_modifications(\n            vec![Modification::ChangeFrom {\n                sender: \"john@example.org\".into(),\n                args: \"REQUIRETLS ENVID=abc123\".into(), //\"NOTIFY=SUCCESS,FAILURE ENVID=abc123\\n\".into()\n            }],\n            &parsed_test_message\n        )\n        .is_none()\n    );\n    let addr = data.mail_from.as_ref().unwrap();\n    assert_eq!(addr.address_lcase, \"john@example.org\");\n    assert_ne!(addr.flags, 0);\n    assert_eq!(addr.dsn_info, Some(\"abc123\".into()));\n\n    // Add recipients\n    assert!(\n        data.apply_milter_modifications(\n            vec![\n                Modification::AddRcpt {\n                    recipient: \"bill@example.org\".into(),\n                    args: \"\".into(),\n                },\n                Modification::AddRcpt {\n                    recipient: \"jane@foobar.org\".into(),\n                    args: \"NOTIFY=SUCCESS,FAILURE ORCPT=rfc822;Jane.Doe@Foobar.org\".into(),\n                },\n                Modification::AddRcpt {\n                    recipient: \"<bill@example.org>\".into(),\n                    args: \"\".into(),\n                },\n                Modification::AddRcpt {\n                    recipient: \"<>\".into(),\n                    args: \"\".into(),\n                },\n            ],\n            &parsed_test_message\n        )\n        .is_none()\n    );\n    assert_eq!(data.rcpt_to.len(), 2);\n    let addr = data.rcpt_to.first().unwrap();\n    assert_eq!(addr.address_lcase, \"bill@example.org\");\n    assert_eq!(addr.dsn_info, None);\n    assert_eq!(addr.flags, 0);\n    let addr = data.rcpt_to.last().unwrap();\n    assert_eq!(addr.address_lcase, \"jane@foobar.org\");\n    assert_ne!(addr.flags, 0);\n    assert_eq!(addr.dsn_info, Some(\"Jane.Doe@Foobar.org\".into()));\n\n    // Remove recipients\n    assert!(\n        data.apply_milter_modifications(\n            vec![\n                Modification::DeleteRcpt {\n                    recipient: \"bill@example.org\".into(),\n                },\n                Modification::DeleteRcpt {\n                    recipient: \"<>\".into(),\n                },\n            ],\n            &parsed_test_message\n        )\n        .is_none()\n    );\n    assert_eq!(data.rcpt_to.len(), 1);\n    let addr = data.rcpt_to.last().unwrap();\n    assert_eq!(addr.address_lcase, \"jane@foobar.org\");\n    assert_ne!(addr.flags, 0);\n    assert_eq!(addr.dsn_info, Some(\"Jane.Doe@Foobar.org\".into()));\n}\n\n#[test]\nfn milter_message_modifications() {\n    // Read test message\n    let milter_path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n        .join(\"resources\")\n        .join(\"smtp\")\n        .join(\"milter\");\n    let test_message = fs::read_to_string(milter_path.join(\"message.eml\")).unwrap();\n    let tests = serde_json::from_str::<Vec<HeaderTest>>(\n        &fs::read_to_string(milter_path.join(\"message.json\")).unwrap(),\n    )\n    .unwrap();\n    let parsed_test_message = AuthenticatedMessage::parse(test_message.as_bytes()).unwrap();\n    let mut session_data = SessionData::new(\n        \"127.0.0.1\".parse().unwrap(),\n        0,\n        \"127.0.0.1\".parse().unwrap(),\n        0,\n        Default::default(),\n        0,\n    );\n\n    for test in tests {\n        assert_eq!(\n            test.result,\n            String::from_utf8(\n                session_data\n                    .apply_milter_modifications(test.modifications, &parsed_test_message)\n                    .unwrap()\n            )\n            .unwrap()\n        )\n    }\n}\n\n#[test]\nfn milter_frame_receiver() {\n    let mut stream = Vec::new();\n\n    for i in 0u32..100u32 {\n        stream.extend_from_slice((i + 1).to_be_bytes().as_ref());\n        stream.push(i as u8);\n        for v in 0..i {\n            stream.push(v as u8);\n        }\n    }\n\n    for chunk_size in [stream.len(), 1, 2, 3, 4, 10, 20, 30, 40, 100, 200, 300, 400] {\n        let mut receiver = Receiver::with_max_frame_len(100);\n        let mut frame_num = 0;\n\n        'outer: for chunk in stream.chunks(chunk_size) {\n            loop {\n                match receiver.read_frame(chunk) {\n                    FrameResult::Frame(bytes) => {\n                        /*println!(\n                            \"frame {frame_num}, chunk: {chunk_size}, {}\",\n                            if matches!(bytes, std::borrow::Cow::Borrowed(_)) {\n                                \"borrowed\"\n                            } else {\n                                \"owned\"\n                            }\n                        );*/\n                        assert_eq!(*bytes.first().unwrap(), frame_num);\n                        assert_eq!(bytes.len(), frame_num as usize + 1);\n                        frame_num += 1;\n                    }\n                    FrameResult::Incomplete => continue 'outer,\n                    FrameResult::TooLarge(size) => {\n                        panic!(\"Frame too large: {size}\")\n                    }\n                }\n            }\n        }\n\n        assert_eq!(frame_num, 100, \"chunk_size: {}\", chunk_size);\n    }\n}\n\n#[tokio::test]\n#[ignore]\nasync fn milter_client_test() {\n    //const PORT : u16 = 11332;\n    const PORT: u16 = 7357;\n    let mut client = MilterClient::connect(\n        &Milter {\n            enable: IfBlock::empty(\"\"),\n            id: Arc::new(\"test\".into()),\n            addrs: vec![SocketAddr::from(([127, 0, 0, 1], PORT))],\n            hostname: \"localhost\".into(),\n            port: PORT,\n            timeout_connect: Duration::from_secs(10),\n            timeout_command: Duration::from_secs(30),\n            timeout_data: Duration::from_secs(30),\n            tls: false,\n            tls_allow_invalid_certs: false,\n            tempfail_on_error: false,\n            max_frame_len: 5000000,\n            protocol_version: MilterVersion::V6,\n            flags_actions: None,\n            flags_protocol: None,\n            run_on_stage: AHashSet::from([Stage::Data]),\n        },\n        0,\n    )\n    .await\n    .unwrap();\n    client.init().await.unwrap();\n\n    let raw_message = load_test_message(\"arc\", \"messages\");\n    let message = MessageParser::new().parse(raw_message.as_bytes()).unwrap();\n\n    let r = client\n        .connection(\n            \"gmail.com\",\n            \"127.0.0.1\".parse().unwrap(),\n            1235,\n            Macros::new(),\n        )\n        .await\n        .unwrap();\n    println!(\"CONNECT: {:?}\", r);\n    let r = client\n        .mail_from(\"john@gmail.com\", None::<&[&str]>, Macros::new())\n        .await\n        .unwrap();\n    println!(\"MAIL FROM: {:?}\", r);\n    let r = client\n        .rcpt_to(\"user@gmail.com\", None::<&[&str]>, Macros::new())\n        .await\n        .unwrap();\n    println!(\"RCPT TO: {:?}\", r);\n\n    let r = client.data().await.unwrap();\n    println!(\"DATA: {:?}\", r);\n    let r = client.headers(message.headers_raw()).await.unwrap();\n    println!(\"HEADERS: {:?}\", r);\n    let r = client\n        .body(&message.raw_message()[message.root_part().raw_body_offset() as usize..])\n        .await\n        .unwrap();\n    println!(\"BODY: {:?}\", r);\n\n    client.quit().await.unwrap();\n}\n\npub fn spawn_mock_milter_server() -> watch::Sender<bool> {\n    let (tx, rx) = watch::channel(true);\n    let tests = Arc::new(\n        serde_json::from_str::<Vec<HeaderTest>>(\n            &fs::read_to_string(\n                PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n                    .join(\"resources\")\n                    .join(\"smtp\")\n                    .join(\"milter\")\n                    .join(\"message.json\"),\n            )\n            .unwrap(),\n        )\n        .unwrap(),\n    );\n\n    tokio::spawn(async move {\n        let listener = TcpListener::bind(\"127.0.0.1:9332\")\n            .await\n            .unwrap_or_else(|e| {\n                panic!(\"Failed to bind mock Milter server to 127.0.0.1:9332: {e}\");\n            });\n        let mut rx_ = rx.clone();\n        //println!(\"Mock Milter server listening on port 9332\");\n        loop {\n            tokio::select! {\n                stream = listener.accept() => {\n                    match stream {\n                        Ok((stream, _)) => {\n                            tokio::spawn(accept_milter(stream, rx.clone(), tests.clone()));\n                        }\n                        Err(err) => {\n                            panic!(\"Something went wrong: {err}\" );\n                        }\n                    }\n                },\n                _ = rx_.changed() => {\n                    //println!(\"Mock Milter server stopping\");\n                    break;\n                }\n            };\n        }\n    });\n\n    tx\n}\n\nasync fn accept_milter(\n    mut stream: TcpStream,\n    mut rx: watch::Receiver<bool>,\n    tests: Arc<Vec<HeaderTest>>,\n) {\n    let mut buf = vec![0u8; 1024];\n    let mut receiver = Receiver::with_max_frame_len(5000000);\n    let mut action = None;\n    let mut modifications = None;\n\n    'outer: loop {\n        let br = tokio::select! {\n            br = stream.read(&mut buf) => {\n                match br {\n                    Ok(br) => {\n                        br\n                    }\n                    Err(_) => {\n                        break;\n                    }\n                }\n            },\n            _ = rx.changed() => {\n                break;\n            }\n        };\n\n        if br == 0 {\n            break;\n        }\n\n        loop {\n            match receiver.read_frame(&buf[..br]) {\n                FrameResult::Frame(bytes) => {\n                    let cmd = Command::deserialize(bytes.as_ref());\n                    println!(\"CMD: {cmd}\");\n\n                    let response = match cmd {\n                        Command::Abort | Command::Macro { .. } => continue,\n                        Command::Body { .. }\n                        | Command::Data\n                        | Command::Connect { .. }\n                        | Command::Header { .. }\n                        | Command::Helo { .. }\n                        | Command::Rcpt { .. }\n                        | Command::QuitNewConnection\n                        | Command::EndOfHeader => Response::Action(Action::Accept),\n                        Command::OptionNegotiation(_) => Response::OptionNegotiation(Options {\n                            version: 6,\n                            actions: 0,\n                            protocol: 0,\n                        }),\n                        Command::MailFrom { sender, .. } => {\n                            let sender = std::str::from_utf8(sender).unwrap();\n                            action = match sender\n                                .strip_prefix('<')\n                                .unwrap()\n                                .split_once('@')\n                                .unwrap()\n                                .0\n                            {\n                                \"accept\" => Action::Accept,\n                                \"reject\" => Action::Reject,\n                                \"discard\" => Action::Discard,\n                                \"temp_fail\" => Action::TempFail,\n                                \"shutdown\" => Action::Shutdown,\n                                \"conn_fail\" => Action::ConnectionFailure,\n                                \"reply_code\" => Action::ReplyCode {\n                                    code: [b'3', b'2', b'1'],\n                                    text: \"test\".into(),\n                                },\n                                test_num => {\n                                    modifications = tests[test_num.parse::<usize>().unwrap()]\n                                        .modifications\n                                        .clone()\n                                        .into();\n                                    Action::Accept\n                                }\n                            }\n                            .into();\n                            Response::Action(Action::Accept)\n                        }\n                        Command::Quit => break 'outer,\n                        Command::EndOfBody => {\n                            if let Some(modifications) = modifications.take() {\n                                for modification in modifications {\n                                    // Write modifications\n                                    stream\n                                        .write_all(\n                                            &Response::Modification(modification).serialize(),\n                                        )\n                                        .await\n                                        .unwrap();\n                                }\n                            }\n\n                            Response::Action(action.take().unwrap())\n                        }\n                    };\n\n                    // Write response\n                    stream.write_all(&response.serialize()).await.unwrap();\n                }\n                FrameResult::Incomplete => continue 'outer,\n                FrameResult::TooLarge(size) => {\n                    panic!(\"Frame too large: {size}\")\n                }\n            }\n        }\n    }\n}\n\npub fn spawn_mock_mta_hook_server() -> watch::Sender<bool> {\n    let (tx, rx) = watch::channel(true);\n    let tests = Arc::new(\n        serde_json::from_str::<Vec<HeaderTest>>(\n            &fs::read_to_string(\n                PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n                    .join(\"resources\")\n                    .join(\"smtp\")\n                    .join(\"milter\")\n                    .join(\"message.json\"),\n            )\n            .unwrap(),\n        )\n        .unwrap(),\n    );\n\n    tokio::spawn(async move {\n        let listener = TcpListener::bind(\"127.0.0.1:9333\")\n            .await\n            .unwrap_or_else(|e| {\n                panic!(\"Failed to bind mock Milter server to 127.0.0.1:9333: {e}\");\n            });\n        let mut rx_ = rx.clone();\n        //println!(\"Mock jMilter server listening on port 9333\");\n        loop {\n            tokio::select! {\n                stream = listener.accept() => {\n                    match stream {\n                        Ok((stream, _)) => {\n\n                            let _ = http1::Builder::new()\n                            .keep_alive(false)\n                            .serve_connection(\n                                TokioIo::new(stream),\n                                service_fn(|mut req: hyper::Request<body::Incoming>| {\n                                    let tests = tests.clone();\n\n                                    async move {\n\n                                        let request = serde_json::from_slice::<Request>(&fetch_body(&mut req, 1024 * 1024,0).await.unwrap())\n                                        .unwrap();\n                                        let response = handle_mta_hook(request, tests);\n\n                                        Ok::<_, hyper::Error>(\n                                            Resource::new(\"application/json\", serde_json::to_string(&response).unwrap().into_bytes())\n                                            .into_http_response().build(),\n                                        )\n                                    }\n                                }),\n                            )\n                            .await;\n                        }\n                        Err(err) => {\n                            panic!(\"Something went wrong: {err}\" );\n                        }\n                    }\n                },\n                _ = rx_.changed() => {\n                    //println!(\"Mock jMilter server stopping\");\n                    break;\n                }\n            };\n        }\n    });\n\n    tx\n}\n\nfn handle_mta_hook(request: Request, tests: Arc<Vec<HeaderTest>>) -> hooks::Response {\n    match request\n        .envelope\n        .unwrap()\n        .from\n        .address\n        .split_once('@')\n        .unwrap()\n        .0\n    {\n        \"accept\" => hooks::Response {\n            action: hooks::Action::Accept,\n            response: None,\n            modifications: vec![],\n        },\n        \"reject\" => hooks::Response {\n            action: hooks::Action::Reject,\n            response: None,\n            modifications: vec![],\n        },\n        \"discard\" => hooks::Response {\n            action: hooks::Action::Discard,\n            response: None,\n            modifications: vec![],\n        },\n        \"temp_fail\" => hooks::Response {\n            action: hooks::Action::Reject,\n            response: SmtpResponse {\n                status: 451.into(),\n                enhanced_status: Some(\"4.3.5\".into()),\n                message: Some(\"Unable to accept message at this time.\".into()),\n                disconnect: false,\n            }\n            .into(),\n            modifications: vec![],\n        },\n        \"shutdown\" => hooks::Response {\n            action: hooks::Action::Reject,\n            response: SmtpResponse {\n                status: 421.into(),\n                enhanced_status: Some(\"4.3.0\".into()),\n                message: Some(\"Server shutting down\".into()),\n                disconnect: false,\n            }\n            .into(),\n            modifications: vec![],\n        },\n        \"conn_fail\" => hooks::Response {\n            action: hooks::Action::Accept,\n            response: SmtpResponse {\n                disconnect: true,\n                ..Default::default()\n            }\n            .into(),\n            modifications: vec![],\n        },\n        \"reply_code\" => hooks::Response {\n            action: hooks::Action::Reject,\n            response: SmtpResponse {\n                status: 321.into(),\n                enhanced_status: Some(\"3.1.1\".into()),\n                message: Some(\"Test\".into()),\n                disconnect: false,\n            }\n            .into(),\n            modifications: vec![],\n        },\n        test_num => hooks::Response {\n            action: hooks::Action::Accept,\n            response: None,\n            modifications: tests[test_num.parse::<usize>().unwrap()]\n                .modifications\n                .iter()\n                .map(|m| match m {\n                    Modification::ChangeFrom { sender, args } => hooks::Modification::ChangeFrom {\n                        value: sender.clone(),\n                        parameters: args\n                            .split_whitespace()\n                            .map(|arg| {\n                                let (key, value) = arg.split_once('=').unwrap();\n                                (key.into(), Some(value.into()))\n                            })\n                            .collect(),\n                    },\n                    Modification::AddRcpt { recipient, args } => {\n                        hooks::Modification::AddRecipient {\n                            value: recipient.clone(),\n                            parameters: args\n                                .split_whitespace()\n                                .map(|arg| {\n                                    let (key, value) = arg.split_once('=').unwrap();\n                                    (key.into(), Some(value.into()))\n                                })\n                                .collect(),\n                        }\n                    }\n                    Modification::DeleteRcpt { recipient } => {\n                        hooks::Modification::DeleteRecipient {\n                            value: recipient.clone(),\n                        }\n                    }\n                    Modification::ReplaceBody { value } => hooks::Modification::ReplaceContents {\n                        value: String::from_utf8(value.clone()).unwrap(),\n                    },\n                    Modification::AddHeader { name, value } => hooks::Modification::AddHeader {\n                        name: name.clone(),\n                        value: value.clone(),\n                    },\n                    Modification::InsertHeader { index, name, value } => {\n                        hooks::Modification::InsertHeader {\n                            index: *index,\n                            name: name.clone(),\n                            value: value.clone(),\n                        }\n                    }\n                    Modification::ChangeHeader { index, name, value } => {\n                        hooks::Modification::ChangeHeader {\n                            index: *index,\n                            name: name.clone(),\n                            value: value.clone(),\n                        }\n                    }\n                    Modification::Quarantine { reason } => hooks::Modification::AddHeader {\n                        name: \"X-Quarantine\".into(),\n                        value: reason.clone(),\n                    },\n                })\n                .collect(),\n        },\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse common::{\n    Server,\n    ipc::{DmarcEvent, QueueEvent, QueueEventStatus, ReportingEvent, TlsEvent},\n};\nuse store::{\n    Deserialize, IterateParams, U64_LEN, ValueKey,\n    write::{\n        AlignedBytes, Archive, QueueClass, ReportEvent, ValueClass, key::DeserializeBigEndian,\n    },\n};\nuse tokio::sync::mpsc::error::TryRecvError;\n\nuse smtp::queue::{Message, MessageWrapper, QueueId, QueuedMessage};\n\nuse super::{QueueReceiver, ReportReceiver};\n\npub mod antispam;\npub mod asn;\npub mod auth;\npub mod basic;\npub mod data;\npub mod dmarc;\npub mod ehlo;\npub mod limits;\npub mod mail;\npub mod milter;\npub mod rcpt;\npub mod rewrite;\npub mod scripts;\npub mod sign;\npub mod throttle;\npub mod vrfy;\n\nimpl QueueReceiver {\n    pub async fn read_event(&mut self) -> QueueEvent {\n        match tokio::time::timeout(Duration::from_millis(100), self.queue_rx.recv()).await {\n            Ok(Some(event)) => event,\n            Ok(None) => panic!(\"Channel closed.\"),\n            Err(_) => panic!(\"No queue event received.\"),\n        }\n    }\n\n    pub async fn try_read_event(&mut self) -> Option<QueueEvent> {\n        match tokio::time::timeout(Duration::from_millis(100), self.queue_rx.recv()).await {\n            Ok(Some(event)) => Some(event),\n            Ok(None) => panic!(\"Channel closed.\"),\n            Err(_) => None,\n        }\n    }\n\n    pub fn assert_no_events(&mut self) {\n        match self.queue_rx.try_recv() {\n            Err(TryRecvError::Empty) => (),\n            Ok(event) => panic!(\"Expected empty queue but got {event:?}\"),\n            Err(err) => panic!(\"Queue error: {err:?}\"),\n        }\n    }\n\n    pub async fn assert_queue_is_empty(&self) {\n        assert_eq!(self.read_queued_messages().await, vec![]);\n        assert_eq!(self.read_queued_events().await, vec![]);\n    }\n\n    pub async fn assert_report_is_empty(&self) {\n        assert_eq!(self.read_report_events().await, vec![]);\n\n        for (from_key, to_key) in [\n            (\n                ValueKey::from(ValueClass::Queue(QueueClass::TlsReportEvent(ReportEvent {\n                    due: 0,\n                    policy_hash: 0,\n                    seq_id: 0,\n                    domain: String::new(),\n                }))),\n                ValueKey::from(ValueClass::Queue(QueueClass::TlsReportEvent(ReportEvent {\n                    due: u64::MAX,\n                    policy_hash: 0,\n                    seq_id: 0,\n                    domain: String::new(),\n                }))),\n            ),\n            (\n                ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportEvent(\n                    ReportEvent {\n                        due: 0,\n                        policy_hash: 0,\n                        seq_id: 0,\n                        domain: String::new(),\n                    },\n                ))),\n                ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportEvent(\n                    ReportEvent {\n                        due: u64::MAX,\n                        policy_hash: 0,\n                        seq_id: 0,\n                        domain: String::new(),\n                    },\n                ))),\n            ),\n        ] {\n            self.store\n                .iterate(\n                    IterateParams::new(from_key, to_key).ascending().no_values(),\n                    |key, _| {\n                        panic!(\"Unexpected report event: {key:?}\");\n                    },\n                )\n                .await\n                .unwrap();\n        }\n    }\n\n    pub async fn expect_message(&mut self) -> MessageWrapper {\n        self.read_event().await.assert_refresh();\n        self.last_queued_message().await\n    }\n\n    pub async fn consume_message(&mut self, server: &Server) -> MessageWrapper {\n        self.read_event().await.assert_refresh();\n        let message = self.last_queued_message().await;\n        message\n            .clone()\n            .remove(server, self.last_queued_due().await.into())\n            .await;\n        message\n    }\n\n    pub async fn expect_message_then_deliver(&mut self) -> QueuedMessage {\n        let message = self.expect_message().await;\n\n        self.delivery_attempt(message.queue_id).await\n    }\n\n    pub async fn delivery_attempt(&mut self, queue_id: u64) -> QueuedMessage {\n        QueuedMessage {\n            due: self.message_due(queue_id).await,\n            queue_id,\n            queue_name: Default::default(),\n        }\n    }\n\n    pub async fn read_queued_events(&self) -> Vec<store::write::QueueEvent> {\n        let mut events = Vec::new();\n\n        let from_key = ValueKey::from(ValueClass::Queue(QueueClass::MessageEvent(\n            store::write::QueueEvent {\n                due: 0,\n                queue_id: 0,\n                queue_name: [0; 8],\n            },\n        )));\n        let to_key = ValueKey::from(ValueClass::Queue(QueueClass::MessageEvent(\n            store::write::QueueEvent {\n                due: u64::MAX,\n                queue_id: u64::MAX,\n                queue_name: [u8::MAX; 8],\n            },\n        )));\n\n        self.store\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending().no_values(),\n                |key, _| {\n                    events.push(store::write::QueueEvent {\n                        due: key.deserialize_be_u64(0)?,\n                        queue_id: key.deserialize_be_u64(U64_LEN)?,\n                        queue_name: key[U64_LEN + 1..U64_LEN + 9]\n                            .try_into()\n                            .expect(\"Queue name must be 8 bytes\"),\n                    });\n                    Ok(true)\n                },\n            )\n            .await\n            .unwrap();\n\n        events\n    }\n\n    pub async fn read_queued_messages(&self) -> Vec<MessageWrapper> {\n        let from_key = ValueKey::from(ValueClass::Queue(QueueClass::Message(0)));\n        let to_key = ValueKey::from(ValueClass::Queue(QueueClass::Message(u64::MAX)));\n        let mut messages = Vec::new();\n\n        self.store\n            .iterate(\n                IterateParams::new(from_key, to_key).descending(),\n                |key, value| {\n                    messages.push(MessageWrapper {\n                        queue_id: key.deserialize_be_u64(0)?,\n                        queue_name: Default::default(),\n                        is_multi_queue: false,\n                        span_id: 0,\n                        message: <Archive<AlignedBytes> as Deserialize>::deserialize(value)?\n                            .deserialize::<Message>()?,\n                    });\n                    Ok(true)\n                },\n            )\n            .await\n            .unwrap();\n\n        messages\n    }\n\n    pub async fn read_report_events(&self) -> Vec<QueueClass> {\n        let from_key = ValueKey::from(ValueClass::Queue(QueueClass::DmarcReportHeader(\n            ReportEvent {\n                due: 0,\n                policy_hash: 0,\n                seq_id: 0,\n                domain: String::new(),\n            },\n        )));\n        let to_key = ValueKey::from(ValueClass::Queue(QueueClass::TlsReportHeader(\n            ReportEvent {\n                due: u64::MAX,\n                policy_hash: 0,\n                seq_id: 0,\n                domain: String::new(),\n            },\n        )));\n\n        let mut events = Vec::new();\n        self.store\n            .iterate(\n                IterateParams::new(from_key, to_key).ascending().no_values(),\n                |key, _| {\n                    let event = ReportEvent::deserialize(key)?;\n                    // Skip lock\n                    if event.seq_id != 0 {\n                        events.push(if *key.last().unwrap() == 0 {\n                            QueueClass::DmarcReportHeader(event)\n                        } else {\n                            QueueClass::TlsReportHeader(event)\n                        });\n                    }\n                    Ok(true)\n                },\n            )\n            .await\n            .unwrap();\n        events\n    }\n\n    pub async fn last_queued_message(&self) -> MessageWrapper {\n        self.read_queued_messages()\n            .await\n            .into_iter()\n            .next()\n            .expect(\"No messages found in queue\")\n    }\n\n    pub async fn last_queued_due(&self) -> u64 {\n        self.message_due(self.last_queued_message().await.queue_id)\n            .await\n    }\n\n    pub async fn message_due(&self, queue_id: QueueId) -> u64 {\n        self.read_queued_events()\n            .await\n            .iter()\n            .find_map(|event| {\n                if event.queue_id == queue_id {\n                    Some(event.due)\n                } else {\n                    None\n                }\n            })\n            .expect(\"No event found in queue for message\")\n    }\n\n    pub async fn clear_queue(&self, server: &Server) {\n        for message in self.read_queued_messages().await {\n            let due = self.message_due(message.queue_id).await;\n            message.remove(server, due.into()).await;\n        }\n    }\n}\n\nimpl ReportReceiver {\n    pub async fn read_report(&mut self) -> ReportingEvent {\n        match tokio::time::timeout(Duration::from_millis(100), self.report_rx.recv()).await {\n            Ok(Some(event)) => event,\n            Ok(None) => panic!(\"Channel closed.\"),\n            Err(_) => panic!(\"No report event received.\"),\n        }\n    }\n\n    pub async fn try_read_report(&mut self) -> Option<ReportingEvent> {\n        match tokio::time::timeout(Duration::from_millis(100), self.report_rx.recv()).await {\n            Ok(Some(event)) => Some(event),\n            Ok(None) => panic!(\"Channel closed.\"),\n            Err(_) => None,\n        }\n    }\n    pub fn assert_no_reports(&mut self) {\n        match self.report_rx.try_recv() {\n            Err(TryRecvError::Empty) => (),\n            Ok(event) => panic!(\"Expected no reports but got {event:?}\"),\n            Err(err) => panic!(\"Report error: {err:?}\"),\n        }\n    }\n}\n\npub trait TestQueueEvent {\n    fn assert_refresh(self);\n    fn assert_done(self);\n    fn assert_refresh_or_done(self);\n}\n\nimpl TestQueueEvent for QueueEvent {\n    fn assert_refresh(self) {\n        match self {\n            QueueEvent::Refresh\n            | QueueEvent::WorkerDone {\n                status: QueueEventStatus::Deferred,\n                ..\n            } => (),\n            e => panic!(\"Unexpected event: {e:?}\"),\n        }\n    }\n\n    fn assert_done(self) {\n        match self {\n            QueueEvent::WorkerDone {\n                status: QueueEventStatus::Completed,\n                ..\n            } => (),\n            e => panic!(\"Unexpected event: {e:?}\"),\n        }\n    }\n\n    fn assert_refresh_or_done(self) {\n        match self {\n            QueueEvent::WorkerDone {\n                status: QueueEventStatus::Completed | QueueEventStatus::Deferred,\n                ..\n            } => (),\n            e => panic!(\"Unexpected event: {e:?}\"),\n        }\n    }\n}\n\npub trait TestReportingEvent {\n    fn unwrap_dmarc(self) -> Box<DmarcEvent>;\n    fn unwrap_tls(self) -> Box<TlsEvent>;\n}\n\nimpl TestReportingEvent for ReportingEvent {\n    fn unwrap_dmarc(self) -> Box<DmarcEvent> {\n        match self {\n            ReportingEvent::Dmarc(event) => event,\n            e => panic!(\"Unexpected event: {e:?}\"),\n        }\n    }\n\n    fn unwrap_tls(self) -> Box<TlsEvent> {\n        match self {\n            ReportingEvent::Tls(event) => event,\n            e => panic!(\"Unexpected event: {e:?}\"),\n        }\n    }\n}\n\n#[allow(async_fn_in_trait)]\npub trait TestMessage {\n    async fn read_message(&self, core: &QueueReceiver) -> String;\n    async fn read_lines(&self, core: &QueueReceiver) -> Vec<String>;\n}\n\nimpl TestMessage for MessageWrapper {\n    async fn read_message(&self, core: &QueueReceiver) -> String {\n        String::from_utf8(\n            core.blob_store\n                .get_blob(self.message.blob_hash.as_slice(), 0..usize::MAX)\n                .await\n                .unwrap()\n                .expect(\"Message blob not found\"),\n        )\n        .unwrap()\n    }\n\n    async fn read_lines(&self, core: &QueueReceiver) -> Vec<String> {\n        self.read_message(core)\n            .await\n            .split('\\n')\n            .map(|l| l.to_string())\n            .collect()\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/rcpt.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse common::Core;\n\nuse smtp_proto::{RCPT_NOTIFY_DELAY, RCPT_NOTIFY_FAILURE, RCPT_NOTIFY_SUCCESS};\nuse store::Stores;\nuse utils::config::Config;\n\nuse smtp::core::{Session, State};\n\nuse crate::smtp::{\n    TempDir, TestSMTP,\n    session::{TestSession, VerifyResponse},\n};\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\ndescription = \"John Doe\"\nsecret = \"secret\"\nemail = \"john@foobar.org\"\n\n[[directory.\"local\".principals]]\nname = \"jane\"\ndescription = \"Jane Doe\"\nsecret = \"p4ssw0rd\"\nemail = \"jane@foobar.org\"\n\n[[directory.\"local\".principals]]\nname = \"bill\"\ndescription = \"Bill Foobar\"\nsecret = \"p4ssw0rd\"\nemail = \"bill@foobar.org\"\n\n[[directory.\"local\".principals]]\nname = \"mike\"\ndescription = \"Mike Foobar\"\nsecret = \"p4ssw0rd\"\nemail = \"mike@foobar.org\"\n\n[session.rcpt]\ndirectory = \"'local'\"\nmax-recipients = [{if = \"remote_ip = '10.0.0.1'\", then = 3},\n                {else = 5}]\nrelay = [{if = \"remote_ip = '10.0.0.1'\", then = false},\n         {else = true}]\n\n[session.rcpt.errors]\ntotal = [{if = \"remote_ip = '10.0.0.1'\", then = 3},\n         {else = 100}]\nwait = [{if = \"remote_ip = '10.0.0.1'\", then = '5ms'},\n        {else = '1s'}]\n\n[session.extensions]\ndsn = [{if = \"remote_ip = '10.0.0.1'\", then = false},\n       {else = true}]\n\n[[queue.limiter.inbound]]\nmatch = \"remote_ip = '10.0.0.1' && !is_empty(rcpt)\"\nkey = 'sender'\nrate = '2/1s'\nenable = true\n\n\"#;\n\n#[tokio::test]\nasync fn rcpt() {\n    // Enable logging\n    crate::enable_logging();\n\n    let tmp_dir = TempDir::new(\"smtp_rcpt_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n\n    // RCPT without MAIL FROM\n    let mut session = Session::test(TestSMTP::from_core(core).server);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx1.foobar.org\").await;\n    session.rcpt_to(\"jane@foobar.org\", \"503 5.5.1\").await;\n\n    // Relaying is disabled for 10.0.0.1\n    session.mail_from(\"john@example.net\", \"250\").await;\n    session.rcpt_to(\"external@domain.com\", \"550 5.1.2\").await;\n\n    // DSN is disabled for 10.0.0.1\n    session\n        .ingest(b\"RCPT TO:<jane@foobar.org> NOTIFY=SUCCESS,FAILURE,DELAY\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"501 5.5.4\");\n\n    // Send to non-existing user\n    session.rcpt_to(\"tom@foobar.org\", \"550 5.1.2\").await;\n\n    // Exceeding max number of errors\n    session\n        .ingest(b\"RCPT TO:<sam@foobar.org>\\r\\n\")\n        .await\n        .unwrap_err();\n    session.response().assert_code(\"451 4.3.0\");\n\n    // Rate limit\n    session.data.rcpt_errors = 0;\n    session.state = State::default();\n    session.rcpt_to(\"Jane@FooBar.org\", \"250\").await;\n    session.rcpt_to(\"Bill@FooBar.org\", \"250\").await;\n    session.rcpt_to(\"Mike@FooBar.org\", \"452 4.4.5\").await;\n\n    // Restore rate limit\n    tokio::time::sleep(Duration::from_millis(1100)).await;\n    session.rcpt_to(\"Mike@FooBar.org\", \"250\").await;\n    session.rcpt_to(\"john@foobar.org\", \"455 4.5.3\").await;\n\n    // Check recipients\n    assert_eq!(session.data.rcpt_to.len(), 3);\n    for (rcpt, expected) in\n        session\n            .data\n            .rcpt_to\n            .iter()\n            .zip([\"Jane@FooBar.org\", \"Bill@FooBar.org\", \"Mike@FooBar.org\"])\n    {\n        assert_eq!(rcpt.address, expected);\n        assert_eq!(rcpt.domain, \"foobar.org\");\n        assert_eq!(rcpt.address_lcase, expected.to_lowercase());\n    }\n\n    // Relaying should be allowed for 10.0.0.2\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.eval_session_params().await;\n    session.rset().await;\n    session.mail_from(\"john@example.net\", \"250\").await;\n    session.rcpt_to(\"external@domain.com\", \"250\").await;\n\n    // DSN is enabled for 10.0.0.2\n    session\n        .ingest(b\"RCPT TO:<jane@foobar.org> NOTIFY=SUCCESS,FAILURE,DELAY ORCPT=rfc822;Jane.Doe@Foobar.org\\r\\n\")\n        .await\n        .unwrap();\n    session.response().assert_code(\"250\");\n    let rcpt = session.data.rcpt_to.last().unwrap();\n    assert!((rcpt.flags & (RCPT_NOTIFY_DELAY | RCPT_NOTIFY_SUCCESS | RCPT_NOTIFY_FAILURE)) != 0);\n    assert_eq!(rcpt.dsn_info.as_ref().unwrap(), \"Jane.Doe@Foobar.org\");\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/rewrite.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Core;\n\nuse smtp::core::Session;\nuse utils::config::Config;\n\nuse crate::smtp::{TestSMTP, session::TestSession};\n\nconst CONFIG: &str = r#\"\n[session.mail]\nrewrite = [ { if = \"ends_with(sender_domain, '.foobar.net') & matches('^([^.]+)@([^.]+)\\.(.+)$', sender)\", then = \"$1 + '+' + $2 + '@' + $3\"},\n            { else = false } ]\nscript = [ { if = \"sender_domain = 'foobar.org'\", then = \"'mail'\" }, \n            { else = false } ]\n\n[session.rcpt]\nrewrite = [ { if = \"rcpt_domain = 'foobar.net' & matches('^([^.]+)\\\\.([^.]+)@(.+)$', rcpt)\", then = \"$1 + '+' + $2 + '@' + $3\"},\n            { else = false } ]\nscript = [ { if = \"rcpt_domain = 'foobar.org'\", then = \"'rcpt'\" }, \n            { else = false } ]\nrelay = true\n\n[sieve.trusted]\nfrom-name = \"Sieve Daemon\"\nfrom-addr = \"sieve@foobar.org\"\nreturn-path = \"\"\nhostname = \"mx.foobar.org\"\n\n[sieve.trusted.limits]\nredirects = 3\nout-messages = 5\nreceived-headers = 50\ncpu = 10000\nnested-includes = 5\nduplicate-expiry = \"7d\"\n\n[sieve.trusted.scripts.\"mail\"]\ncontents = '''\nrequire [\"variables\", \"envelope\"];\n\nif allof( envelope :domain :is \"from\" \"foobar.org\", \n          envelope :localpart :contains \"from\" \"admin\" ) {\n     set \"envelope.from\" \"MAILER-DAEMON@foobar.org\";\n}\n\n'''\n\n[sieve.trusted.scripts.\"rcpt\"]\ncontents = '''\nrequire [\"variables\", \"envelope\", \"regex\"];\n\nif allof( envelope :localpart :contains \"to\" \".\",\n          envelope :regex \"to\" \"(.+)@(.+)$\") {\n    set :replace \".\" \"\" \"to\" \"${1}\";\n    set \"envelope.to\" \"${to}@${2}\";\n}\n\n'''\n\n\"#;\n\n#[tokio::test]\nasync fn address_rewrite() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Prepare config\n    let mut config = Config::new(CONFIG).unwrap();\n    let core = Core::parse(&mut config, Default::default(), Default::default()).await;\n\n    // Init session\n    let mut session = Session::test(TestSMTP::from_core(core).server);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.doe.org\").await;\n\n    // Sender rewrite using regex\n    session.mail_from(\"bill@doe.foobar.net\", \"250\").await;\n    assert_eq!(\n        session.data.mail_from.as_ref().unwrap().address,\n        \"bill+doe@foobar.net\"\n    );\n    session.reset();\n\n    // Sender rewrite using sieve\n    session.mail_from(\"this_is_admin@foobar.org\", \"250\").await;\n    assert_eq!(\n        session.data.mail_from.as_ref().unwrap().address_lcase,\n        \"mailer-daemon@foobar.org\"\n    );\n\n    // Recipient rewrite using regex\n    session.rcpt_to(\"mary.smith@foobar.net\", \"250\").await;\n    assert_eq!(\n        session.data.rcpt_to.last().unwrap().address,\n        \"mary+smith@foobar.net\"\n    );\n\n    // Remove duplicates\n    session.rcpt_to(\"mary.smith@foobar.net\", \"250\").await;\n    assert_eq!(session.data.rcpt_to.len(), 1);\n\n    // Recipient rewrite using sieve\n    session.rcpt_to(\"m.a.r.y.s.m.i.t.h@foobar.org\", \"250\").await;\n    assert_eq!(\n        session.data.rcpt_to.last().unwrap().address,\n        \"marysmith@foobar.org\"\n    );\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/scripts.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse core::panic;\nuse std::{fmt::Write, fs, path::PathBuf};\n\nuse crate::{\n    AssertConfig, enable_logging,\n    smtp::{\n        TempDir, TestSMTP,\n        inbound::{TestMessage, TestQueueEvent, sign::SIGNATURES},\n        session::{TestSession, VerifyResponse},\n    },\n};\nuse common::Core;\n\nuse smtp::{\n    core::Session,\n    scripts::{ScriptResult, event_loop::RunScript},\n};\nuse store::Stores;\nuse utils::config::Config;\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"sql\"\nlookup = \"sql\"\nblob = \"sql\"\nfts = \"sql\"\ndirectory = \"local\"\n\n[store.\"sql\"]\ntype = \"sqlite\"\npath = \"{TMP}/smtp_sieve.db\"\n\n[store.\"sql\".pool]\nmax-connections = 10\nmin-connections = 0\nidle-timeout = \"5m\"\n\n[spam-filter]\nenable = false\n\n[sieve.trusted]\nfrom-name = \"'Sieve Daemon'\"\nfrom-addr = \"'sieve@foobar.org'\"\nreturn-path = \"''\"\nhostname = \"mx.foobar.org\"\nsign = \"['rsa']\"\n\n[sieve.trusted.limits]\nredirects = 3\nout-messages = 5\nreceived-headers = 50\ncpu = 10000\nnested-includes = 5\nduplicate-expiry = \"7d\"\n\n[session.connect]\nscript = \"'stage_connect'\"\ngreeting = \"'mx.example.org at your service'\"\n\n[session.ehlo]\nscript = \"'stage_ehlo'\"\n\n[session.mail]\nscript = \"'stage_mail'\"\n\n[session.rcpt]\nscript = \"'stage_rcpt'\"\nrelay = true\n\n[session.data]\nscript = \"'stage_data'\"\n\n[session.data.add-headers]\nreceived = true\nreceived-spf = true\nauth-results = true\nmessage-id = true\ndate = true\nreturn-path = false\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\ndescription = \"John Doe\"\nsecret = \"secret\"\nemail = [\"john@localdomain.org\", \"jdoe@localdomain.org\", \"john.doe@localdomain.org\"]\nemail-list = [\"info@localdomain.org\"]\nmember-of = [\"sales\"]\n\n\"#;\n\n#[tokio::test]\nasync fn sieve_scripts() {\n    // Enable logging\n    enable_logging();\n\n    // Add test scripts\n    let mut config = CONFIG.to_string() + SIGNATURES;\n    for entry in fs::read_dir(\n        PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n            .join(\"resources\")\n            .join(\"smtp\")\n            .join(\"sieve\"),\n    )\n    .unwrap()\n    {\n        let entry = entry.unwrap();\n        writeln!(\n            &mut config,\n            \"[sieve.trusted.scripts.{}]\\ncontents = \\\"%{{file:{}}}%\\\"\",\n            entry\n                .file_name()\n                .to_str()\n                .unwrap()\n                .split_once('.')\n                .unwrap()\n                .0,\n            entry.path().to_str().unwrap()\n        )\n        .unwrap();\n    }\n\n    // Prepare config\n    let tmp_dir = TempDir::new(\"smtp_sieve_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(config)).unwrap();\n    config.resolve_all_macros().await;\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    config.assert_no_errors();\n\n    // Build session\n    let test = TestSMTP::from_core(core);\n    let mut qr = test.queue_receiver;\n    let mut session = Session::test(test.server.clone());\n    session.data.remote_ip_str = \"10.0.0.88\".parse().unwrap();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    assert!(!session.init_conn().await);\n\n    // Run tests\n    for (name, script) in &test.server.core.sieve.trusted_scripts {\n        if name.starts_with(\"stage_\") || name.ends_with(\"_include\") {\n            continue;\n        }\n        let script = script.clone();\n        let params = session\n            .build_script_parameters(\"data\")\n            .set_variable(\"from\", \"john.doe@example.org\")\n            .with_envelope(&test.server, &session, 0)\n            .await;\n        match test.server.run_script(name.into(), script, params).await {\n            ScriptResult::Accept { .. } => (),\n            ScriptResult::Reject(message) => panic!(\"{}\", message),\n            err => {\n                panic!(\"Unexpected script result {err:?}\");\n            }\n        }\n    }\n\n    // Test connect script\n    session\n        .response()\n        .assert_contains(\"503 5.5.3 Your IP '10.0.0.88' is not welcomed here\");\n    session.data.remote_ip_str = \"10.0.0.5\".parse().unwrap();\n    session.data.remote_ip = session.data.remote_ip_str.parse().unwrap();\n    assert!(session.init_conn().await);\n    session\n        .response()\n        .assert_contains(\"220 mx.example.org at your service\");\n\n    // Test EHLO script\n    session\n        .cmd(\n            \"EHLO spammer.org\",\n            \"551 5.1.1 Your domain 'spammer.org' has been blocklisted\",\n        )\n        .await;\n    session.cmd(\"EHLO foobar.net\", \"250\").await;\n\n    // Test MAIL-FROM script\n    session\n        .mail_from(\"spammer@domain.com\", \"450 4.1.1 Invalid address\")\n        .await;\n    session\n        .mail_from(\n            \"marketing@spam-domain.com\",\n            \"503 5.5.3 Your address has been blocked\",\n        )\n        .await;\n    session.mail_from(\"bill@foobar.org\", \"250\").await;\n\n    // Test RCPT-TO script\n    session\n        .rcpt_to(\n            \"jane@foobar.org\",\n            \"422 4.2.2 You have been greylisted '10.0.0.5.bill@foobar.org.jane@foobar.org'.\",\n        )\n        .await;\n    session.rcpt_to(\"jane@foobar.org\", \"250\").await;\n\n    // Expect a modified message\n    session.data(\"test:multipart\", \"250\").await;\n\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"X-Part-Number: 5\")\n        .assert_contains(\"THIS IS A PIECE OF HTML TEXT\");\n    qr.assert_no_events();\n\n    // Expect rejection for bill@foobar.net\n    session\n        .send_message(\n            \"test@example.net\",\n            &[\"bill@foobar.net\"],\n            \"test:multipart\",\n            \"503 5.5.3 Bill cannot receive messages\",\n        )\n        .await;\n    qr.assert_no_events();\n    qr.clear_queue(&test.server).await;\n\n    // Expect message delivery plus a notification\n    session\n        .send_message(\n            \"test@example.net\",\n            &[\"john@foobar.net\"],\n            \"test:multipart\",\n            \"250\",\n        )\n        .await;\n    qr.read_event().await.assert_refresh();\n    qr.read_event().await.assert_refresh();\n    let messages = qr.read_queued_messages().await;\n    assert_eq!(messages.len(), 2);\n    let mut messages = messages.into_iter();\n    let notification = messages.next().unwrap();\n    assert_eq!(notification.message.return_path.as_ref(), \"\");\n    assert_eq!(notification.message.recipients.len(), 2);\n    assert_eq!(\n        notification.message.recipients.first().unwrap().address(),\n        \"john@example.net\"\n    );\n    assert_eq!(\n        notification.message.recipients.last().unwrap().address(),\n        \"jane@example.org\"\n    );\n    notification\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com;\")\n        .assert_contains(\"From: \\\"Sieve Daemon\\\" <sieve@foobar.org>\")\n        .assert_contains(\"To: <john@example.net>\")\n        .assert_contains(\"Cc: <jane@example.org>\")\n        .assert_contains(\"Subject: You have got mail\")\n        .assert_contains(\"One Two Three Four\");\n\n    messages\n        .next()\n        .unwrap()\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"One Two Three Four\")\n        .assert_contains(\"multi-part message in MIME format\")\n        .assert_not_contains(\"X-Part-Number: 5\")\n        .assert_not_contains(\"THIS IS A PIECE OF HTML TEXT\");\n    qr.assert_no_events();\n    qr.clear_queue(&test.server).await;\n\n    // Expect a modified message delivery plus a notification\n    session\n        .send_message(\n            \"test@example.net\",\n            &[\"jane@foobar.net\"],\n            \"test:multipart\",\n            \"250\",\n        )\n        .await;\n    qr.read_event().await.assert_refresh();\n    qr.read_event().await.assert_refresh();\n    let messages = qr.read_queued_messages().await;\n    assert_eq!(messages.len(), 2);\n    let mut messages = messages.into_iter();\n\n    messages\n        .next()\n        .unwrap()\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com;\")\n        .assert_contains(\"From: \\\"Sieve Daemon\\\" <sieve@foobar.org>\")\n        .assert_contains(\"To: <john@example.net>\")\n        .assert_contains(\"Cc: <jane@example.org>\")\n        .assert_contains(\"Subject: You have got mail\")\n        .assert_contains(\"One Two Three Four\");\n\n    messages\n        .next()\n        .unwrap()\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"X-Part-Number: 5\")\n        .assert_contains(\"THIS IS A PIECE OF HTML TEXT\")\n        .assert_not_contains(\"X-My-Header: true\");\n    qr.clear_queue(&test.server).await;\n\n    // Expect a modified redirected message\n    session\n        .send_message(\n            \"test@example.net\",\n            &[\"thomas@foobar.gov\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n\n    let redirect = qr.expect_message().await;\n    assert_eq!(redirect.message.return_path.as_ref(), \"\");\n    assert_eq!(redirect.message.recipients.len(), 1);\n    assert_eq!(\n        redirect.message.recipients.first().unwrap().address(),\n        \"redirect@here.email\"\n    );\n    redirect\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"From: no-reply@my.domain\")\n        .assert_contains(\"To: Suzie Q <suzie@shopping.example.net>\")\n        .assert_contains(\"Subject: Is dinner ready?\")\n        .assert_contains(\"Message-ID: <20030712040037.46341.5F8J@football.example.com>\")\n        .assert_contains(\"Received: \")\n        .assert_not_contains(\"From: Joe SixPack <joe@football.example.com>\");\n    qr.assert_no_events();\n\n    // Expect an intact redirected message\n    session\n        .send_message(\n            \"test@example.net\",\n            &[\"bob@foobar.gov\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n\n    let redirect = qr.expect_message().await;\n    assert_eq!(redirect.message.return_path.as_ref(), \"\");\n    assert_eq!(redirect.message.recipients.len(), 1);\n    assert_eq!(\n        redirect.message.recipients.first().unwrap().address(),\n        \"redirect@somewhere.email\"\n    );\n    redirect\n        .read_lines(&qr)\n        .await\n        .assert_not_contains(\"From: no-reply@my.domain\")\n        .assert_contains(\"To: Suzie Q <suzie@shopping.example.net>\")\n        .assert_contains(\"Subject: Is dinner ready?\")\n        .assert_contains(\"Message-ID: <20030712040037.46341.5F8J@football.example.com>\")\n        .assert_contains(\"From: Joe SixPack <joe@football.example.com>\")\n        .assert_contains(\"Received: \")\n        .assert_contains(\"Authentication-Results: \");\n    qr.assert_no_events();\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/sign.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::Core;\n\nuse mail_auth::{\n    common::{parse::TxtRecordParser, verify::DomainKey},\n    spf::Spf,\n};\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::smtp::{\n    DnsCache, TempDir, TestSMTP,\n    inbound::TestMessage,\n    session::{TestSession, VerifyResponse},\n};\nuse smtp::core::Session;\n\npub const SIGNATURES: &str = \"\n[signature.rsa]\nprivate-key = '''\n-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAv9XYXG3uK95115mB4nJ37nGeNe2CrARm1agrbcnSk5oIaEfM\nZLUR/X8gPzoiNHZcfMZEVR6bAytxUhc5EvZIZrjSuEEeny+fFd/cTvcm3cOUUbIa\nUmSACj0dL2/KwW0LyUaza9z9zor7I5XdIl1M53qVd5GI62XBB76FH+Q0bWPZNkT4\nNclzTLspD/MTpNCCPhySM4Kdg5CuDczTH4aNzyS0TqgXdtw6A4Sdsp97VXT9fkPW\n9rso3lrkpsl/9EQ1mR/DWK6PBmRfIuSFuqnLKY6v/z2hXHxF7IoojfZLa2kZr9Ae\nd4l9WheQOTA19k5r2BmlRw/W9CrgCBo0Sdj+KQIDAQABAoIBAFPChEi/OvnulReB\nECQWhOUYuNKlFKQU++2YEvZJ4+bMn5UgnE7wfJ1pj2Pr9xlfALz+OMHNrjMxGbaV\nKzdrT2uCkYcf78XjnhuH9gKIiXDUv4L4N+P3u6w8yOx4bFgOS9IjS53yDOPM7SC5\ng6dIg5aigHaHlffqIuFFv4yQMI/+Ai+zBKxS7wRhxK/7nnAuo28fe5MEdp57ho9/\nAGlDNsdg9zCgjwhokwFE3+AaD+bkUFm4gQ1XjkUFrlmnQn8vDQ0i9toEWhCj+UPY\niOKL63MJnr90MXTXWLHoFj99wBp//mYygbF9Lj8fa28/oa8LWp3Jhb7QeMgH46iv\n3aLHbTECgYEA5M2dAw+nyMw9vYlkMejhwObKYP8Mr/6zcGMLCalYvRJM5iUAM0JI\nH6sM6pV9/nv167cbKocj3xYPdtE7FPOn4132MLM8Ne1f8nPE64Qrcbj5WBXvLnU8\nhpWbwe2Z8h7UUMKx6q4F1/TXYkc3ScxYwfjM4mP/pLsAOgVzRSEEgrUCgYEA1qNQ\nxaQHNWZ1O8WuTnqWd5JSsic6iURAmUcLeFDZY2PWhVoaQ8L/xMQhDYs1FIbLWArW\n4Qq3Ibu8AbSejAKuaJz7Uf26PX+PYVUwAOO0qamCJ8d/qd6So7qWMDyAY2yXI39Y\n1nMqRjr7bkEsggAZao7BKqA7ZtmogjOusBT38iUCgYEA06agJ8TDoKvOMRZ26PRU\nYO0dKLzGL8eclcoI29cbj0rud7aiiMg3j5PbTuUat95TjsjDCIQaWrM9etvxm2AJ\nXfn9Uu96MyhyKQWOk46f4YMKpMElkARDCPw8KRhx39dE77AqhLyWCz8iPndCXbH6\nKPTOEl4OjYOuof2Is9nnIkECgYBh948RdsnXhNlzm8nwhiGRmBbou+EK8D0v+O5y\nTyy6IcKzgSnFzgZh8EdJ4EUtBk1f9SqY8wQdgIvSl3daXorusuA/TzkngsaV3YUY\nktZOLlF7CKLrjOyPkMWmZKcROmpNyH1q/IvKHHfQnizLdXIkYd4nL5WNX0F7lE1i\nj1+QhQKBgB2lviBK7rJFwlFYdQUP1NAN2dKxMZk8uJS8JglHrM0+8nRI83HbTdEQ\nvB0ManEKBkbS4T5n+gRtdEqKSDmWDTXDlrBfcdCHNQLwYtBpOotCqQn/AmfjcPBl\nbyAbwh4+HiZ5JISoRZpiZqy67aJNVoXmdtb/E9mi7ozzytpxMNql\n-----END RSA PRIVATE KEY-----'''\ndomain = 'example.com'\nselector = 'rsa'\nheaders = ['From', 'To', 'Date', 'Subject', 'Message-ID']\nalgorithm = 'rsa-sha256'\ncanonicalization = 'simple/relaxed'\nexpire = '10d'\nset-body-length = true\nreport = true\n\n[signature.ed]\nprivate-key = '-----BEGIN PRIVATE KEY-----\nMC4CAQAwBQYDK2VwBCIEIAO3hAf144lTAVjTkht3ZwBTK0CMCCd1bI0alggneN3B\n-----END PRIVATE KEY-----'\ndomain = 'example.com'\nselector = 'ed'\nheaders = ['From', 'To', 'Date', 'Subject', 'Message-ID']\nalgorithm = 'ed25519-sha256'\ncanonicalization = 'relaxed/simple'\nset-body-length = false\n\";\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\ndescription = \"John Doe\"\nsecret = \"secret\"\nemail = [\"jdoe@example.com\"]\n\n[session.rcpt]\ndirectory = \"'local'\"\n\n[session.data.add-headers]\nreceived = true\nreceived-spf = true\nauth-results = true\nmessage-id = true\ndate = true\nreturn-path = false\n\n[auth.spf.verify]\nehlo = \"relaxed\"\nmail-from = \"relaxed\"\n\n[auth.dkim]\nverify = \"relaxed\"\nsign = \"['rsa']\"\n\n[auth.arc]\nverify = \"relaxed\"\nseal = \"'ed'\"\n\n[auth.dmarc]\nverify = \"relaxed\"\n\n\"#;\n\n#[tokio::test]\nasync fn sign_and_seal() {\n    // Enable logging\n    crate::enable_logging();\n\n    let tmp_dir = TempDir::new(\"smtp_sign_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG.to_string() + SIGNATURES)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    let test = TestSMTP::from_core(core);\n\n    // Add SPF, DKIM and DMARC records\n    test.server.txt_add(\n        \"mx.example.com\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 ip4:10.0.0.2 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"example.com\",\n        Spf::parse(b\"v=spf1 ip4:10.0.0.1 -all\").unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"ed._domainkey.scamorza.org\",\n        DomainKey::parse(\n            concat!(\n                \"v=DKIM1; k=ed25519; \",\n                \"p=11qYAYKxCrfVS/7TyWQHOg7hcvPapiMlrwIaaPcHURo=\"\n            )\n            .as_bytes(),\n        )\n        .unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n    test.server.txt_add(\n        \"rsa._domainkey.manchego.org\",\n        DomainKey::parse(\n            concat!(\n                \"v=DKIM1; t=s; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQ\",\n                \"KBgQDwIRP/UC3SBsEmGqZ9ZJW3/DkMoGeLnQg1fWn7/zYt\",\n                \"IxN2SnFCjxOCKG9v3b4jYfcTNh5ijSsq631uBItLa7od+v\",\n                \"/RtdC2UzJ1lWT947qR+Rcac2gbto/NMqJ0fzfVjH4OuKhi\",\n                \"tdY9tf6mcwGjaNBcWToIMmPSPDdQPNUYckcQ2QIDAQAB\",\n            )\n            .as_bytes(),\n        )\n        .unwrap(),\n        Instant::now() + Duration::from_secs(5),\n    );\n\n    // Test DKIM signing\n    let mut qr = test.queue_receiver;\n    let mut session = Session::test(test.server);\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.example.com\").await;\n    session\n        .send_message(\n            \"bill@foobar.org\",\n            &[\"jdoe@example.com\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\n            \"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com; c=simple/relaxed;\",\n        );\n\n    // Test ARC verify and seal\n    session\n        .send_message(\"bill@foobar.org\", &[\"jdoe@example.com\"], \"test:arc\", \"250\")\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"ARC-Seal: i=3; a=ed25519-sha256; s=ed; d=example.com; cv=pass;\")\n        .assert_contains(\n            \"ARC-Message-Signature: i=3; a=ed25519-sha256; s=ed; d=example.com; c=relaxed/simple;\",\n        );\n\n    // Test ARC sealing of a DKIM signed message\n    session\n        .send_message(\"bill@foobar.org\", &[\"jdoe@example.com\"], \"test:dkim\", \"250\")\n        .await;\n    qr.expect_message()\n        .await\n        .read_lines(&qr)\n        .await\n        .assert_contains(\"ARC-Seal: i=1; a=ed25519-sha256; s=ed; d=example.com; cv=none;\")\n        .assert_contains(\n            \"ARC-Message-Signature: i=1; a=ed25519-sha256; s=ed; d=example.com; c=relaxed/simple;\",\n        );\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/throttle.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse crate::smtp::{TempDir, TestSMTP, session::TestSession};\nuse common::Core;\nuse smtp::core::{Session, SessionAddress};\nuse store::Stores;\nuse utils::config::Config;\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/data.db\"\n\n[[queue.limiter.inbound]]\nmatch = \"remote_ip = '10.0.0.1'\"\nkey = 'remote_ip'\nrate = '2/1s'\nenable = true\n\n[[queue.limiter.inbound]]\nkey = 'sender'\nrate = '2/1s'\nenable = true\n\n[[queue.limiter.inbound]]\nkey = ['remote_ip', 'rcpt']\nrate = '2/1s'\nenable = true\n\n\"#;\n\n#[tokio::test]\nasync fn throttle_inbound() {\n    // Enable logging\n    crate::enable_logging();\n\n    let tmp_dir = TempDir::new(\"smtp_inbound_throttle\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n\n    // Test connection rate limit\n    let mut session = Session::test(TestSMTP::from_core(core).server);\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    assert!(session.is_allowed().await, \"Rate limiter too strict.\");\n    assert!(session.is_allowed().await, \"Rate limiter too strict.\");\n    assert!(!session.is_allowed().await, \"Rate limiter failed.\");\n    tokio::time::sleep(Duration::from_millis(1100)).await;\n    assert!(\n        session.is_allowed().await,\n        \"Rate limiter did not restore quota.\"\n    );\n\n    // Test mail from rate limit\n    session.data.mail_from = SessionAddress {\n        address: \"sender@test.org\".into(),\n        address_lcase: \"sender@test.org\".into(),\n        domain: \"test.org\".into(),\n        flags: 0,\n        dsn_info: None,\n    }\n    .into();\n    assert!(session.is_allowed().await, \"Rate limiter too strict.\");\n    assert!(session.is_allowed().await, \"Rate limiter too strict.\");\n    assert!(!session.is_allowed().await, \"Rate limiter failed.\");\n    session.data.mail_from = SessionAddress {\n        address: \"other-sender@test.org\".into(),\n        address_lcase: \"other-sender@test.org\".into(),\n        domain: \"test.org\".into(),\n        flags: 0,\n        dsn_info: None,\n    }\n    .into();\n    assert!(session.is_allowed().await, \"Rate limiter failed.\");\n\n    // Test recipient rate limit\n    session.data.rcpt_to.push(SessionAddress {\n        address: \"recipient@example.org\".into(),\n        address_lcase: \"recipient@example.org\".into(),\n        domain: \"example.org\".into(),\n        flags: 0,\n        dsn_info: None,\n    });\n    assert!(session.is_allowed().await, \"Rate limiter too strict.\");\n    assert!(session.is_allowed().await, \"Rate limiter too strict.\");\n    assert!(!session.is_allowed().await, \"Rate limiter failed.\");\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    assert!(session.is_allowed().await, \"Rate limiter too strict.\");\n}\n"
  },
  {
    "path": "tests/src/smtp/inbound/vrfy.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::Core;\n\nuse store::Stores;\nuse utils::config::Config;\n\nuse smtp::core::Session;\n\nuse crate::{\n    AssertConfig,\n    smtp::{\n        TempDir, TestSMTP,\n        session::{TestSession, VerifyResponse},\n    },\n};\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"rocksdb\"\nlookup = \"rocksdb\"\nblob = \"rocksdb\"\nfts = \"rocksdb\"\ndirectory = \"local\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/data.db\"\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"john\"\ndescription = \"John Doe\"\nsecret = \"secret\"\nemail = [\"john@foobar.org\"]\nemail-list = [\"sales@foobar.org\"]\n\n[[directory.\"local\".principals]]\nname = \"jane\"\ndescription = \"Jane Doe\"\nsecret = \"p4ssw0rd\"\nemail = \"jane@foobar.org\"\nemail-list = [\"sales@foobar.org\"]\n\n[[directory.\"local\".principals]]\nname = \"bill\"\ndescription = \"Bill Foobar\"\nsecret = \"p4ssw0rd\"\nemail = \"bill@foobar.org\"\nemail-list = [\"sales@foobar.org\"]\n\n[session.rcpt]\ndirectory = \"'local'\"\n\n[session.extensions]\nvrfy = [{if = \"remote_ip = '10.0.0.1'\", then = true},\n        {else = false}]\nexpn = [{if = \"remote_ip = '10.0.0.1'\", then = true},\n        {else = false}]\n\n\"#;\n\n#[tokio::test]\nasync fn vrfy_expn() {\n    // Enable logging\n    crate::enable_logging();\n\n    let tmp_dir = TempDir::new(\"smtp_vrfy_test\", true);\n    let mut config = Config::new(tmp_dir.update_config(CONFIG)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    config.assert_no_errors();\n\n    // EHLO should not advertise VRFY/EXPN to 10.0.0.2\n    let mut session = Session::test(TestSMTP::from_core(core).server);\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.eval_session_params().await;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_not_contains(\"EXPN\")\n        .assert_not_contains(\"VRFY\");\n    session.cmd(\"VRFY john\", \"252 2.5.1\").await;\n    session.cmd(\"EXPN sales@foobar.org\", \"252 2.5.1\").await;\n\n    // EHLO should advertise VRFY/EXPN for 10.0.0.1\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_contains(\"EXPN\")\n        .assert_contains(\"VRFY\");\n\n    // Successful VRFY\n    session.cmd(\"VRFY john\", \"250 john@foobar.org\").await;\n\n    // Successful EXPN\n    session\n        .cmd(\"EXPN sales@foobar.org\", \"250\")\n        .await\n        .assert_contains(\"250-john@foobar.org\")\n        .assert_contains(\"250-jane@foobar.org\")\n        .assert_contains(\"250 bill@foobar.org\");\n\n    // Non-existent VRFY\n    session.cmd(\"VRFY robert\", \"550 5.1.2\").await;\n\n    // Non-existent EXPN\n    session.cmd(\"EXPN procurement\", \"550 5.1.2\").await;\n}\n"
  },
  {
    "path": "tests/src/smtp/lookup/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod sql;\npub mod utils;\n"
  },
  {
    "path": "tests/src/smtp/lookup/sql.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::{\n    Core,\n    expr::{tokenizer::TokenMap, *},\n};\n\nuse directory::{\n    QueryParams, Type,\n    backend::internal::{PrincipalField, PrincipalSet, PrincipalValue, manage::ManageDirectory},\n};\nuse mail_auth::MX;\nuse store::Stores;\nuse utils::config::Config;\n\nuse crate::{\n    directory::DirectoryStore,\n    smtp::{\n        DnsCache, TempDir, TestSMTP,\n        session::{TestSession, VerifyResponse},\n    },\n};\nuse smtp::{core::Session, queue::RecipientDomain};\n\nconst CONFIG: &str = r#\"\n[storage]\ndata = \"sql\"\nblob = \"sql\"\nfts = \"sql\"\nlookup = \"sql\"\ndirectory = \"sql\"\n\n[store.\"sql\"]\ntype = \"sqlite\"\npath = \"{TMP}/smtp_sql.db\"\n\n[store.\"sql\".query]\nname = \"SELECT name, type, secret, description, quota FROM accounts WHERE name = ? AND active = true\"\nmembers = \"SELECT member_of FROM group_members WHERE name = ?\"\nrecipients = \"SELECT name FROM emails WHERE address = ?\"\nemails = \"SELECT address FROM emails WHERE name = ? AND type != 'list' ORDER BY type DESC, address ASC\"\nverify = \"SELECT address FROM emails WHERE address LIKE '%' || ? || '%' AND type = 'primary' ORDER BY address LIMIT 5\"\nexpand = \"SELECT p.address FROM emails AS p JOIN emails AS l ON p.name = l.name WHERE p.type = 'primary' AND l.address = ? AND l.type = 'list' ORDER BY p.address LIMIT 50\"\ndomains = \"SELECT 1 FROM emails WHERE address LIKE '%@' || ? LIMIT 1\"\n\n[directory.\"sql\"]\ntype = \"sql\"\nstore = \"sql\"\n\n[directory.\"sql\".columns]\nname = \"name\"\ndescription = \"description\"\nsecret = \"secret\"\nemail = \"address\"\nquota = \"quota\"\nclass = \"type\"\n\n[session.auth]\ndirectory = \"'sql'\"\nmechanisms = \"[plain, login]\"\nerrors.wait = \"5ms\"\n\n[session.rcpt]\ndirectory = \"'sql'\"\nrelay = false\nerrors.wait = \"5ms\"\n\n[session.extensions]\nrequiretls = [{if = \"sql_query('sql', 'SELECT addr FROM allowed_ips WHERE addr = ? LIMIT 1', remote_ip)\", then = true},\n              {else = false}]\nexpn = true\nvrfy = true\n\n[test.\"sql\"]\nexpr = \"sql_query('sql', 'SELECT description FROM domains WHERE name = ?', 'foobar.org')\"\nexpect = \"Main domain\"\n\n[test.\"dns\"]\nexpr = \"dns_query(rcpt_domain, 'mx')[0]\"\nexpect = \"mx.foobar.org\"\n\n[test.\"key_get\"]\nexpr = \"key_get('sql', 'hello') + '-' + key_exists('sql', 'hello') + '-' + key_set('sql', 'hello', 'world') + '-' + key_get('sql', 'hello') + '-' + key_exists('sql', 'hello')\"\nexpect = \"-0-1-world-1\"\n\n[test.\"counter_get\"]\nexpr = \"counter_get('sql', 'county') + '-' + counter_incr('sql', 'county', 1) + '-' + counter_incr('sql', 'county', 1) + '-' + counter_get('sql', 'county')\"\nexpect = \"0-1-2-2\"\n\n\"#;\n\n#[tokio::test]\nasync fn lookup_sql() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Parse settings\n    let temp_dir = TempDir::new(\"smtp_lookup_tests\", true);\n    let mut config = Config::new(temp_dir.update_config(CONFIG)).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n\n    // Obtain directory handle\n    let handle = DirectoryStore {\n        store: core.storage.stores.get(\"sql\").unwrap().clone(),\n    };\n    let test = TestSMTP::from_core(core);\n\n    test.server.mx_add(\n        \"test.org\",\n        vec![MX {\n            exchanges: vec![\"mx.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    // Create tables\n    handle.create_test_directory().await;\n\n    // Create test records\n    handle\n        .create_test_user_with_email(\"jane@foobar.org\", \"s3cr3tp4ss\", \"Jane\")\n        .await;\n    handle\n        .create_test_user_with_email(\"john@foobar.org\", \"mypassword\", \"John\")\n        .await;\n    handle\n        .create_test_user_with_email(\"bill@foobar.org\", \"123456\", \"Bill\")\n        .await;\n    handle\n        .create_test_user_with_email(\"mike@foobar.net\", \"098765\", \"Mike\")\n        .await;\n\n    for query in [\n        \"CREATE TABLE domains (name TEXT PRIMARY KEY, description TEXT);\",\n        \"INSERT INTO domains (name, description) VALUES ('foobar.org', 'Main domain');\",\n        \"INSERT INTO domains (name, description) VALUES ('foobar.net', 'Secondary domain');\",\n        \"CREATE TABLE allowed_ips (addr TEXT PRIMARY KEY);\",\n        \"INSERT INTO allowed_ips (addr) VALUES ('10.0.0.50');\",\n    ] {\n        handle\n            .store\n            .sql_query::<usize>(query, Vec::new())\n            .await\n            .unwrap();\n    }\n\n    // Create local domains\n    let internal_store = &test.server.core.storage.data;\n    for name in [\"foobar.org\", \"foobar.net\"] {\n        internal_store\n            .create_principal(\n                PrincipalSet::new(0, Type::Domain).with_field(PrincipalField::Name, name),\n                None,\n                None,\n            )\n            .await\n            .unwrap();\n    }\n\n    // Create lists\n    internal_store\n        .create_principal(\n            PrincipalSet::new(0, Type::List)\n                .with_field(PrincipalField::Name, \"support@foobar.org\")\n                .with_field(PrincipalField::Emails, \"support@foobar.org\")\n                .with_field(\n                    PrincipalField::ExternalMembers,\n                    PrincipalValue::StringList(vec![\"mike@foobar.net\".to_string()]),\n                ),\n            None,\n            None,\n        )\n        .await\n        .unwrap();\n    internal_store\n        .create_principal(\n            PrincipalSet::new(0, Type::List)\n                .with_field(PrincipalField::Name, \"sales@foobar.org\")\n                .with_field(PrincipalField::Emails, \"sales@foobar.org\")\n                .with_field(\n                    PrincipalField::ExternalMembers,\n                    PrincipalValue::StringList(vec![\n                        \"jane@foobar.org\".to_string(),\n                        \"john@foobar.org\".to_string(),\n                        \"bill@foobar.org\".to_string(),\n                    ]),\n                ),\n            None,\n            None,\n        )\n        .await\n        .unwrap();\n\n    // Test expression functions\n    let token_map = TokenMap::default().with_variables(&[\n        V_RECIPIENT,\n        V_RECIPIENT_DOMAIN,\n        V_SENDER,\n        V_SENDER_DOMAIN,\n        V_MX,\n        V_HELO_DOMAIN,\n        V_AUTHENTICATED_AS,\n        V_LISTENER,\n        V_REMOTE_IP,\n        V_LOCAL_IP,\n        V_PRIORITY,\n    ]);\n    for test_name in [\"sql\", \"dns\", \"key_get\", \"counter_get\"] {\n        let e =\n            Expression::try_parse(&mut config, (\"test\", test_name, \"expr\"), &token_map).unwrap();\n        assert_eq!(\n            test.server\n                .eval_expr::<String, _>(&e, &RecipientDomain::new(\"test.org\"), \"text\", 0)\n                .await\n                .unwrap(),\n            config.value((\"test\", test_name, \"expect\")).unwrap(),\n            \"failed for '{}'\",\n            test_name\n        );\n    }\n\n    let mut session = Session::test(test.server);\n    session.data.remote_ip_str = \"10.0.0.50\".parse().unwrap();\n    session.eval_session_params().await;\n    session.stream.tls = true;\n    session\n        .ehlo(\"mx.foobar.org\")\n        .await\n        .assert_contains(\"REQUIRETLS\");\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session\n        .ehlo(\"mx1.foobar.org\")\n        .await\n        .assert_not_contains(\"REQUIRETLS\");\n\n    // Test RCPT\n    session.mail_from(\"john@example.net\", \"250\").await;\n\n    // External domain\n    session.rcpt_to(\"user@otherdomain.org\", \"550 5.1.2\").await;\n\n    // Non-existent user\n    session.rcpt_to(\"jack@foobar.org\", \"550 5.1.2\").await;\n\n    // Valid users\n    session.rcpt_to(\"jane@foobar.org\", \"250\").await;\n    session.rcpt_to(\"john@foobar.org\", \"250\").await;\n    session.rcpt_to(\"bill@foobar.org\", \"250\").await;\n\n    // Lists\n    session.rcpt_to(\"sales@foobar.org\", \"250\").await;\n\n    // Test EXPN\n    session\n        .cmd(\"EXPN sales@foobar.org\", \"250\")\n        .await\n        .assert_contains(\"jane@foobar.org\")\n        .assert_contains(\"john@foobar.org\")\n        .assert_contains(\"bill@foobar.org\");\n    session\n        .cmd(\"EXPN support@foobar.org\", \"250\")\n        .await\n        .assert_contains(\"mike@foobar.net\");\n    session.cmd(\"EXPN marketing@foobar.org\", \"550 5.1.2\").await;\n\n    // Test VRFY\n    session\n        .server\n        .core\n        .storage\n        .directory\n        .query(QueryParams::name(\"john@foobar.org\").with_return_member_of(true))\n        .await\n        .unwrap()\n        .unwrap();\n    session\n        .server\n        .core\n        .storage\n        .directory\n        .query(QueryParams::name(\"jane@foobar.org\").with_return_member_of(true))\n        .await\n        .unwrap()\n        .unwrap();\n    session\n        .cmd(\"VRFY john\", \"250\")\n        .await\n        .assert_contains(\"john@foobar.org\");\n    session\n        .cmd(\"VRFY jane\", \"250\")\n        .await\n        .assert_contains(\"jane@foobar.org\");\n    session.cmd(\"VRFY tim\", \"550 5.1.2\").await;\n\n    // Test AUTH\n    session\n        .cmd(\n            \"AUTH PLAIN AGphbmVAZm9vYmFyLm9yZwB3cm9uZ3Bhc3M=\",\n            \"535 5.7.8\",\n        )\n        .await;\n    session\n        .cmd(\n            \"AUTH PLAIN AGphbmVAZm9vYmFyLm9yZwBzM2NyM3RwNHNz\",\n            \"235 2.7.0\",\n        )\n        .await;\n}\n"
  },
  {
    "path": "tests/src/smtp/lookup/utils.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::net::IpAddr;\n\nuse crate::smtp::TestSMTP;\nuse ::smtp::outbound::NextHop;\nuse common::{\n    Core,\n    config::smtp::{\n        queue::{MxConfig, QueueExpiry, QueueName},\n        report::AggregateFrequency,\n        resolver::{Mode, MxPattern, Policy},\n    },\n};\nuse mail_auth::{IpLookupStrategy, MX};\nuse mail_parser::DateTime;\nuse smtp::{\n    outbound::{\n        lookup::{SourceIp, ToNextHop},\n        mta_sts::parse::ParsePolicy,\n    },\n    queue::{\n        Error, ErrorDetails, FROM_AUTHENTICATED, Message, QueueEnvelope, Recipient, Schedule,\n        Status,\n    },\n    reporting::AggregateTimestamp,\n};\nuse store::write::now;\nuse utils::config::Config;\n\nconst CONFIG: &str = r#\"\n[queue.connection.test.timeout]\nconnect = \"10s\"\n\n[queue.connection.test]\nehlo-hostname = \"test.example.com\"\nsource-ips = [\"10.0.0.1\", \"10.0.0.2\", \"10.0.0.3\", \"10.0.0.4\", \n              \"a:b::1\", \"a:b::2\", \"a:b::3\", \"a:b::4\"]\n\n[queue.source-ip.\"10.0.0.1\"]\nehlo-hostname = \"test1.example.com\"\n\n[queue.source-ip.\"10.0.0.2\"]\nehlo-hostname = \"test2.example.com\"\n\n[queue.source-ip.\"10.0.0.3\"]\nehlo-hostname = \"test3.example.com\"\n\n[queue.source-ip.\"10.0.0.4\"]\nehlo-hostname = \"test4.example.com\"\n\n[queue.source-ip.\"a:b::1\"]\nehlo-hostname = \"test5.example.com\"\n\n[queue.source-ip.\"a:b::2\"]\nehlo-hostname = \"test6.example.com\"\n\n[queue.source-ip.\"a:b::3\"]\nehlo-hostname = \"test7.example.com\"\n\n[queue.source-ip.\"a:b::4\"]\nehlo-hostname = \"test8.example.com\"\n\n[queue.test-v4.type]\ntype = \"mx\"\nip-lookup-strategy = \"ipv4_then_ipv6\"\n\n[queue.test-v6.type]\ntype = \"mx\"\nip-lookup-strategy = \"ipv6_then_ipv4\"\n\n[queue.strategy]\nschedule = \"source + ' ' + received_from_ip + ' ' + received_via_port + ' ' + queue_name + ' ' + last_error + ' ' + rcpt_domain + ' ' + size + ' ' + queue_age\"\n\n\"#;\n\n#[tokio::test]\nasync fn strategies() {\n    // Enable logging\n    crate::enable_logging();\n\n    let ipv6: [IpAddr; 4] = [\n        \"a:b::1\".parse().unwrap(),\n        \"a:b::2\".parse().unwrap(),\n        \"a:b::3\".parse().unwrap(),\n        \"a:b::4\".parse().unwrap(),\n    ];\n    let ipv4: [IpAddr; 4] = [\n        \"10.0.0.1\".parse().unwrap(),\n        \"10.0.0.2\".parse().unwrap(),\n        \"10.0.0.3\".parse().unwrap(),\n        \"10.0.0.4\".parse().unwrap(),\n    ];\n    let ipv4_hosts = [\n        \"test1.example.com\".to_string(),\n        \"test2.example.com\".to_string(),\n        \"test3.example.com\".to_string(),\n        \"test4.example.com\".to_string(),\n    ];\n    let ipv6_hosts = [\n        \"test5.example.com\".to_string(),\n        \"test6.example.com\".to_string(),\n        \"test7.example.com\".to_string(),\n        \"test8.example.com\".to_string(),\n    ];\n\n    let mut config = Config::new(CONFIG).unwrap();\n    let test =\n        TestSMTP::from_core(Core::parse(&mut config, Default::default(), Default::default()).await);\n\n    let conn = test\n        .server\n        .core\n        .smtp\n        .queue\n        .connection_strategy\n        .get(\"test\")\n        .unwrap();\n\n    assert_eq!(conn.ehlo_hostname.as_ref().unwrap(), \"test.example.com\");\n\n    for is_ipv4 in [true, false] {\n        for _ in 0..10 {\n            let ip_host = conn.source_ip(is_ipv4).unwrap();\n            if is_ipv4 {\n                assert_eq!(\n                    &ipv4_hosts[ipv4.iter().position(|&ip| ip == ip_host.ip).unwrap()],\n                    ip_host.host.as_ref().unwrap()\n                );\n            } else {\n                assert_eq!(\n                    &ipv6_hosts[ipv6.iter().position(|&ip| ip == ip_host.ip).unwrap()],\n                    ip_host.host.as_ref().unwrap()\n                );\n            }\n        }\n    }\n\n    // Test strategy resolution\n    let message = Message {\n        created: now() - 123,\n        blob_hash: Default::default(),\n        received_from_ip: \"1.2.3.4\".parse().unwrap(),\n        received_via_port: 7911,\n        return_path: \"test@example.com\".into(),\n        recipients: vec![Recipient {\n            address: \"recipient@foobar.com\".into(),\n            retry: Schedule::now(),\n            notify: Schedule::now(),\n            expires: QueueExpiry::Ttl(3600),\n            queue: QueueName::new(\"test\").unwrap(),\n            status: Status::TemporaryFailure(ErrorDetails {\n                entity: \"test.example.com\".into(),\n                details: Error::TlsError(\"TLS handshake failed\".into()),\n            }),\n            flags: 0,\n            orcpt: None,\n        }],\n        flags: FROM_AUTHENTICATED,\n        env_id: None,\n        priority: 0,\n        size: 978,\n        quota_keys: Default::default(),\n    };\n\n    assert_eq!(\n        test.server\n            .eval_if::<String, _>(\n                &test.server.core.smtp.queue.queue,\n                &QueueEnvelope::new(&message, &message.recipients[0]),\n                0,\n            )\n            .await\n            .unwrap_or_else(|| \"default\".to_string()),\n        \"authenticated 1.2.3.4 7911 test tls foobar.com 978 123\"\n    );\n}\n\n#[test]\nfn to_remote_hosts() {\n    let mx = vec![\n        MX {\n            exchanges: vec![\"mx1\".to_string(), \"mx2\".to_string()],\n            preference: 10,\n        },\n        MX {\n            exchanges: vec![\n                \"mx3\".to_string(),\n                \"mx4\".to_string(),\n                \"mx5\".to_string(),\n                \"mx6\".to_string(),\n            ],\n            preference: 20,\n        },\n        MX {\n            exchanges: vec![\"mx7\".to_string(), \"mx8\".to_string()],\n            preference: 10,\n        },\n        MX {\n            exchanges: vec![\"mx9\".to_string(), \"mxA\".to_string()],\n            preference: 10,\n        },\n    ];\n    let mx_config = MxConfig {\n        max_mx: 7,\n        max_multi_homed: 2,\n        ip_lookup_strategy: IpLookupStrategy::Ipv4thenIpv6,\n    };\n    let hosts = mx.to_remote_hosts(\"domain\", &mx_config).unwrap();\n    assert_eq!(hosts.len(), 7);\n    for host in hosts {\n        if let NextHop::MX { host, .. } = host {\n            assert!((*host.as_bytes().last().unwrap() - b'0') <= 8);\n        }\n    }\n    let mx = vec![MX {\n        exchanges: vec![\".\".to_string()],\n        preference: 0,\n    }];\n    assert!(mx.to_remote_hosts(\"domain\", &mx_config).is_none());\n}\n\n#[test]\nfn parse_policy() {\n    for (policy, expected_policy) in [\n        (\n            r\"version: STSv1\nmode: enforce\nmx: mail.example.com\nmx: *.example.net\nmx: backupmx.example.com\nmax_age: 604800\",\n            Policy {\n                id: \"abc\".to_string(),\n                mode: Mode::Enforce,\n                mx: vec![\n                    MxPattern::Equals(\"mail.example.com\".to_string()),\n                    MxPattern::StartsWith(\"example.net\".to_string()),\n                    MxPattern::Equals(\"backupmx.example.com\".to_string()),\n                ],\n                max_age: 604800,\n            },\n        ),\n        (\n            r\"version: STSv1\nmode: testing\nmx: gmail-smtp-in.l.google.com\nmx: *.gmail-smtp-in.l.google.com\nmax_age: 86400\n\",\n            Policy {\n                id: \"abc\".to_string(),\n                mode: Mode::Testing,\n                mx: vec![\n                    MxPattern::Equals(\"gmail-smtp-in.l.google.com\".to_string()),\n                    MxPattern::StartsWith(\"gmail-smtp-in.l.google.com\".to_string()),\n                ],\n                max_age: 86400,\n            },\n        ),\n    ] {\n        assert_eq!(\n            Policy::parse(policy, expected_policy.id.to_string()).unwrap(),\n            expected_policy\n        );\n    }\n}\n\n#[test]\nfn aggregate_to_timestamp() {\n    for (freq, date, expected) in [\n        (\n            AggregateFrequency::Hourly,\n            \"2023-01-24T09:10:40Z\",\n            \"2023-01-24T09:00:00Z\",\n        ),\n        (\n            AggregateFrequency::Daily,\n            \"2023-01-24T09:10:40Z\",\n            \"2023-01-24T00:00:00Z\",\n        ),\n        (\n            AggregateFrequency::Weekly,\n            \"2023-01-24T09:10:40Z\",\n            \"2023-01-22T00:00:00Z\",\n        ),\n        (\n            AggregateFrequency::Weekly,\n            \"2023-01-28T23:59:59Z\",\n            \"2023-01-22T00:00:00Z\",\n        ),\n        (\n            AggregateFrequency::Weekly,\n            \"2023-01-22T23:59:59Z\",\n            \"2023-01-22T00:00:00Z\",\n        ),\n    ] {\n        assert_eq!(\n            DateTime::from_timestamp(\n                freq.to_timestamp_(DateTime::parse_rfc3339(date).unwrap()) as i64\n            )\n            .to_rfc3339(),\n            expected,\n            \"failed for {freq:?} {date} {expected}\"\n        );\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/management/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod queue;\npub mod report;\n"
  },
  {
    "path": "tests/src/smtp/management/queue.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse ahash::{AHashMap, HashMap, HashSet};\nuse common::config::server::ServerProtocol;\n\nuse http::management::queue::Message;\nuse mail_auth::MX;\nuse mail_parser::DateTime;\nuse reqwest::{Method, StatusCode, header::AUTHORIZATION};\n\nuse crate::{\n    jmap::ManagementApi,\n    smtp::{DnsCache, TestSMTP, session::TestSession},\n};\nuse smtp::queue::{QueueId, Status, manager::SpawnQueue};\n\nconst LOCAL: &str = r#\"\n[storage]\ndirectory = \"local\"\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"admin\"\ntype = \"admin\"\ndescription = \"Superuser\"\nsecret = \"secret\"\nclass = \"admin\"\n\n[queue.schedule.default]\nretry = \"1000s\"\nnotify = \"2000s\"\nexpire = \"3000s\"\nqueue-name = \"default\"\n\n[session.rcpt]\nrelay = true\nmax-recipients = 100\n\n[session.extensions]\ndsn = true\nfuture-release = \"1h\"\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\"#;\n\n#[derive(serde::Deserialize, Debug)]\n#[allow(dead_code)]\npub(super) struct List<T> {\n    pub items: Vec<T>,\n    pub total: usize,\n}\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn manage_queue() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start remote test server\n    let mut remote = TestSMTP::new(\"smtp_manage_queue_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n    let remote_core = remote.build_smtp();\n\n    // Start local management interface\n    let local = TestSMTP::new(\"smtp_manage_queue_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"mx1.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    core.ipv4_add(\n        \"mx1.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    let _rx_manage = local.start(&[ServerProtocol::Http]).await;\n\n    // Send test messages\n    let envelopes = HashMap::from_iter([\n        (\n            \"a\",\n            (\n                \"bill1@foobar.net\",\n                vec![\n                    \"rcpt1@example1.org\",\n                    \"rcpt1@example2.org\",\n                    \"rcpt1@example2.org\",\n                ],\n            ),\n        ),\n        (\n            \"b\",\n            (\n                \"bill2@foobar.net\",\n                vec![\"rcpt3@example1.net\", \"rcpt4@example1.net\"],\n            ),\n        ),\n        (\n            \"c\",\n            (\n                \"bill3@foobar.net\",\n                vec![\n                    \"rcpt5@example1.com\",\n                    \"rcpt6@example2.com\",\n                    \"rcpt7@example2.com\",\n                    \"rcpt8@example3.com\",\n                    \"rcpt9@example4.com\",\n                ],\n            ),\n        ),\n        (\"d\", (\"bill4@foobar.net\", vec![\"delay@foobar.org\"])),\n        (\"e\", (\"bill5@foobar.net\", vec![\"john@foobar.org\"])),\n        (\"f\", (\"\", vec![\"success@foobar.org\", \"delay@foobar.org\"])),\n    ]);\n    let mut session = local.new_session();\n    local\n        .queue_receiver\n        .queue_rx\n        .spawn(local.server.inner.clone());\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"foobar.net\").await;\n    for test_num in 0..6 {\n        let env_id = char::from(b'a' + test_num).to_string();\n        let hold_for = ((test_num + 1) as u32) * 100;\n        let (sender, recipients) = envelopes.get(env_id.as_str()).unwrap();\n        session\n            .send_message(\n                &if env_id != \"f\" {\n                    format!(\"<{sender}> ENVID={env_id} HOLDFOR={hold_for}\")\n                } else {\n                    format!(\"<{sender}> ENVID={env_id}\")\n                },\n                recipients,\n                \"test:no_dkim\",\n                \"250\",\n            )\n            .await;\n    }\n\n    // Expect delivery to success@foobar.org\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    assert_eq!(\n        remote\n            .queue_receiver\n            .consume_message(&remote_core)\n            .await\n            .message\n            .recipients\n            .into_iter()\n            .map(|r| r.address().to_string())\n            .collect::<Vec<_>>(),\n        vec![\"success@foobar.org\"]\n    );\n\n    // Fetch and validate messages\n    let api = ManagementApi::default();\n    let ids = api\n        .request::<List<QueueId>>(Method::GET, \"/api/queue/messages\")\n        .await\n        .unwrap()\n        .unwrap_data()\n        .items;\n    assert_eq!(ids.len(), 6);\n    let mut id_map = AHashMap::new();\n    let mut id_map_rev = AHashMap::new();\n    let mut test_search = String::new();\n    for (message, id) in api.get_messages(&ids).await.into_iter().zip(ids) {\n        let message = message.unwrap();\n        let env_id = message.env_id.as_ref().unwrap().clone();\n\n        // Validate return path and recipients\n        let (sender, recipients) = envelopes.get(env_id.as_str()).unwrap();\n        assert_eq!(&message.return_path, sender);\n        'outer: for recipient in recipients {\n            for rcpt in &message.recipients {\n                if &rcpt.address == recipient {\n                    continue 'outer;\n                }\n            }\n            panic!(\"Recipient {recipient} not found in message.\");\n        }\n\n        // Validate status and datetimes\n        let created = message.created.to_timestamp();\n        let hold_for = (env_id.as_bytes().first().unwrap() - b'a' + 1) as i64 * 100;\n        let next_retry = created + hold_for;\n        let next_notify = created + 2000 + hold_for;\n        let expires = created + 3000 + hold_for;\n        for rcpt in &message.recipients {\n            if env_id == \"c\" {\n                let mut dt = *rcpt.next_retry.as_ref().unwrap();\n                dt.second -= 1;\n                test_search = dt.to_rfc3339();\n            }\n            if env_id != \"f\" {\n                // HOLDFOR messages\n                assert_eq!(rcpt.retry_num, 0);\n                assert_timestamp(\n                    rcpt.next_retry.as_ref().unwrap(),\n                    next_retry,\n                    \"retry\",\n                    &message,\n                );\n                assert_timestamp(\n                    rcpt.next_notify.as_ref().unwrap(),\n                    next_notify,\n                    \"notify\",\n                    &message,\n                );\n                assert_timestamp(&rcpt.expires.unwrap(), expires, \"expires\", &message);\n                assert_eq!(&rcpt.status, &Status::Scheduled, \"{message:#?}\");\n            } else if rcpt.address == \"success@foobar.org\" {\n                assert_eq!(rcpt.retry_num, 0);\n                assert!(\n                    matches!(&rcpt.status, Status::Completed(_)),\n                    \"{:?}\",\n                    rcpt.status\n                );\n            } else {\n                assert_eq!(rcpt.retry_num, 1);\n                assert!(\n                    matches!(&rcpt.status, Status::TemporaryFailure(_)),\n                    \"{:?}\",\n                    rcpt.status\n                );\n            }\n        }\n\n        id_map.insert(env_id.clone(), id);\n        id_map_rev.insert(id, env_id);\n    }\n    assert_eq!(id_map.len(), 6);\n\n    // Test list search\n    for (query, expected_ids) in [\n        (\n            \"/api/queue/messages?from=bill1@foobar.net\".to_string(),\n            vec![\"a\"],\n        ),\n        (\n            \"/api/queue/messages?to=foobar.org\".to_string(),\n            vec![\"d\", \"e\", \"f\"],\n        ),\n        (\n            \"/api/queue/messages?from=bill3@foobar.net&to=rcpt5@example1.com\".to_string(),\n            vec![\"c\"],\n        ),\n        (\n            format!(\"/api/queue/messages?before={test_search}\"),\n            vec![\"a\", \"b\"],\n        ),\n        (\n            format!(\"/api/queue/messages?after={test_search}\"),\n            vec![\"d\", \"e\", \"f\", \"c\"],\n        ),\n    ] {\n        let expected_ids = HashSet::from_iter(expected_ids.into_iter().map(|s| s.to_string()));\n        let ids = api\n            .request::<List<QueueId>>(Method::GET, &query)\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .into_iter()\n            .map(|id| id_map_rev.get(&id).unwrap().clone())\n            .collect::<HashSet<_>>();\n        assert_eq!(ids, expected_ids, \"failed for {query}\");\n    }\n\n    // Retry delivery\n    for id in [id_map.get(\"e\").unwrap(), id_map.get(\"f\").unwrap()] {\n        assert!(\n            api.request::<bool>(Method::PATCH, &format!(\"/api/queue/messages/{id}\",))\n                .await\n                .unwrap()\n                .unwrap_data(),\n        );\n    }\n    assert!(\n        api.request::<bool>(\n            Method::PATCH,\n            &format!(\n                \"/api/queue/messages/{}?filter=example1.org&at=2200-01-01T00:00:00Z\",\n                id_map.get(\"a\").unwrap(),\n            )\n        )\n        .await\n        .unwrap()\n        .unwrap_data()\n    );\n\n    // Expect delivery to john@foobar.org\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    assert_eq!(\n        remote\n            .queue_receiver\n            .consume_message(&remote_core)\n            .await\n            .message\n            .recipients\n            .into_iter()\n            .map(|r| r.address().to_string())\n            .collect::<Vec<_>>(),\n        vec![\"john@foobar.org\".to_string()]\n    );\n\n    // Message 'e' should be gone, 'f' should have retry_num == 2\n    // while 'a' should have a retry time of 2200-01-01T00:00:00Z for example1.org\n    let mut messages = api\n        .get_messages(&[\n            *id_map.get(\"e\").unwrap(),\n            *id_map.get(\"f\").unwrap(),\n            *id_map.get(\"a\").unwrap(),\n        ])\n        .await\n        .into_iter();\n    assert_eq!(messages.next().unwrap(), None);\n    assert_eq!(\n        messages\n            .next()\n            .unwrap()\n            .unwrap()\n            .recipients\n            .first()\n            .unwrap()\n            .retry_num,\n        2\n    );\n    for rcpt in messages.next().unwrap().unwrap().recipients {\n        let next_retry = rcpt.next_retry.as_ref().unwrap().to_rfc3339();\n        let matched =\n            [\"2200-01-01T00:00:00Z\", \"2199-12-31T23:59:59Z\"].contains(&next_retry.as_str());\n        if rcpt.address.ends_with(\"example1.org\") {\n            assert!(matched, \"{next_retry}\");\n        } else {\n            assert!(!matched, \"{next_retry}\");\n        }\n    }\n\n    // Cancel deliveries\n    for (id, filter) in [\n        (\"a\", \"example2.org\"),\n        (\"b\", \"example1.net\"),\n        (\"c\", \"rcpt6@example2.com\"),\n        (\"d\", \"\"),\n    ] {\n        assert!(\n            api.request::<bool>(\n                Method::DELETE,\n                &format!(\n                    \"/api/queue/messages/{}{}{}\",\n                    id_map.get(id).unwrap(),\n                    if !filter.is_empty() { \"?filter=\" } else { \"\" },\n                    filter\n                )\n            )\n            .await\n            .unwrap()\n            .unwrap_data(),\n            \"failed for {id}: {filter}\"\n        );\n    }\n    assert_eq!(\n        api.request::<List<QueueId>>(Method::GET, \"/api/queue/messages\")\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .len(),\n        3\n    );\n    for (message, id) in api\n        .get_messages(&[\n            *id_map.get(\"a\").unwrap(),\n            *id_map.get(\"b\").unwrap(),\n            *id_map.get(\"c\").unwrap(),\n            *id_map.get(\"d\").unwrap(),\n        ])\n        .await\n        .into_iter()\n        .zip([\"a\", \"b\", \"c\", \"d\"])\n    {\n        if [\"b\", \"d\"].contains(&id) {\n            assert_eq!(message, None);\n        } else {\n            let message = message.unwrap();\n            assert!(!message.recipients.is_empty());\n            for rcpt in message.recipients {\n                match id {\n                    \"a\" => {\n                        if rcpt.address.ends_with(\"example2.org\") {\n                            assert!(matches!(&rcpt.status, Status::PermanentFailure(_)));\n                        } else {\n                            assert!(matches!(&rcpt.status, Status::Scheduled));\n                        }\n                    }\n                    \"c\" => {\n                        if rcpt.address.ends_with(\"example2.com\") {\n                            if rcpt.address == \"rcpt6@example2.com\" {\n                                assert!(matches!(&rcpt.status, Status::PermanentFailure(_)));\n                            } else {\n                                assert!(matches!(&rcpt.status, Status::Scheduled));\n                            }\n                        } else {\n                            assert!(matches!(&rcpt.status, Status::Scheduled));\n                        }\n                    }\n                    _ => unreachable!(),\n                }\n            }\n        }\n    }\n\n    // Bulk cancel\n    assert_eq!(\n        api.request::<List<Message>>(Method::GET, \"/api/queue/messages?values=1\")\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .len(),\n        3\n    );\n    assert!(\n        api.request::<bool>(Method::DELETE, \"/api/queue/messages?text=example2.com\")\n            .await\n            .unwrap()\n            .unwrap_data()\n    );\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    assert_eq!(\n        api.request::<List<QueueId>>(Method::GET, \"/api/queue/messages\")\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .len(),\n        2\n    );\n    assert!(\n        api.request::<bool>(Method::DELETE, \"/api/queue/messages\")\n            .await\n            .unwrap()\n            .unwrap_data()\n    );\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    assert_eq!(\n        api.request::<List<QueueId>>(Method::GET, \"/api/queue/messages\")\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .len(),\n        0\n    );\n\n    // Test authentication error\n    assert_eq!(\n        reqwest::Client::builder()\n            .timeout(Duration::from_millis(500))\n            .danger_accept_invalid_certs(true)\n            .build()\n            .unwrap()\n            .get(\"https://127.0.0.1:9980/api/queue/messages\")\n            .header(AUTHORIZATION, \"Basic YWRtaW46aGVsbG93b3JsZA==\")\n            .send()\n            .await\n            .unwrap()\n            .status(),\n        StatusCode::UNAUTHORIZED\n    );\n}\n\nfn assert_timestamp(timestamp: &DateTime, expected: i64, ctx: &str, message: &Message) {\n    let timestamp = timestamp.to_timestamp();\n    let diff = timestamp - expected;\n    if ![-2, -1, 0, 1, 2].contains(&diff) {\n        panic!(\n            \"Got timestamp {timestamp}, expected {expected} (diff {diff} for {ctx}) for {message:?}\"\n        );\n    }\n}\n\nimpl ManagementApi {\n    async fn get_messages(&self, ids: &[QueueId]) -> Vec<Option<Message>> {\n        let mut results = Vec::with_capacity(ids.len());\n\n        for id in ids {\n            let message = self\n                .request::<Message>(Method::GET, &format!(\"/api/queue/messages/{id}\",))\n                .await\n                .unwrap()\n                .try_unwrap_data();\n            results.push(message);\n        }\n\n        results\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/management/report.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse ahash::{AHashMap, HashSet};\nuse common::{\n    config::{server::ServerProtocol, smtp::report::AggregateFrequency},\n    ipc::{DmarcEvent, PolicyType, TlsEvent},\n};\n\nuse http::management::queue::Report;\nuse mail_auth::{\n    common::parse::TxtRecordParser,\n    dmarc::Dmarc,\n    mta_sts::TlsRpt,\n    report::{\n        ActionDisposition, DmarcResult, Record,\n        tlsrpt::{FailureDetails, ResultType},\n    },\n};\nuse reqwest::Method;\n\nuse crate::{\n    jmap::ManagementApi,\n    smtp::{TestSMTP, management::queue::List},\n};\nuse smtp::reporting::{SmtpReporting, scheduler::SpawnReport};\n\nconst CONFIG: &str = r#\"\n[storage]\ndirectory = \"local\"\n\n[directory.\"local\"]\ntype = \"memory\"\n\n[[directory.\"local\".principals]]\nname = \"admin\"\ntype = \"admin\"\ndescription = \"Superuser\"\nsecret = \"secret\"\nclass = \"admin\"\n\n[session.rcpt]\nrelay = true\n\n[report.dmarc.aggregate]\nmax-size = 1024\n\n[report.tls.aggregate]\nmax-size = 1024\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn manage_reports() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start reporting service\n    let local = TestSMTP::new(\"smtp_manage_reports\", CONFIG).await;\n    let _rx = local.start(&[ServerProtocol::Http]).await;\n    let core = local.build_smtp();\n    local\n        .report_receiver\n        .report_rx\n        .spawn(local.server.inner.clone());\n\n    // Send test reporting events\n    core.schedule_report(DmarcEvent {\n        domain: \"foobar.org\".to_string(),\n        report_record: Record::new()\n            .with_source_ip(\"192.168.1.2\".parse().unwrap())\n            .with_action_disposition(ActionDisposition::Pass)\n            .with_dmarc_dkim_result(DmarcResult::Pass)\n            .with_dmarc_spf_result(DmarcResult::Fail)\n            .with_envelope_from(\"hello@example.org\")\n            .with_envelope_to(\"other@example.org\")\n            .with_header_from(\"bye@example.org\"),\n        dmarc_record: Arc::new(\n            Dmarc::parse(b\"v=DMARC1; p=reject; rua=mailto:reports@foobar.org\").unwrap(),\n        ),\n        interval: AggregateFrequency::Daily,\n    })\n    .await;\n    core.schedule_report(DmarcEvent {\n        domain: \"foobar.net\".to_string(),\n        report_record: Record::new()\n            .with_source_ip(\"a:b:c::e:f\".parse().unwrap())\n            .with_action_disposition(ActionDisposition::Reject)\n            .with_dmarc_dkim_result(DmarcResult::Fail)\n            .with_dmarc_spf_result(DmarcResult::Pass),\n        dmarc_record: Arc::new(\n            Dmarc::parse(\n                b\"v=DMARC1; p=quarantine; rua=mailto:reports@foobar.net,mailto:reports@example.net\",\n            )\n            .unwrap(),\n        ),\n        interval: AggregateFrequency::Weekly,\n    })\n    .await;\n    core.schedule_report(TlsEvent {\n        domain: \"foobar.org\".to_string(),\n        policy: PolicyType::None,\n        failure: None,\n        tls_record: Arc::new(TlsRpt::parse(b\"v=TLSRPTv1;rua=mailto:reports@foobar.org\").unwrap()),\n        interval: AggregateFrequency::Daily,\n    })\n    .await;\n    core.schedule_report(TlsEvent {\n        domain: \"foobar.net\".to_string(),\n        policy: PolicyType::Sts(None),\n        failure: FailureDetails::new(ResultType::StsPolicyInvalid).into(),\n        tls_record: Arc::new(TlsRpt::parse(b\"v=TLSRPTv1;rua=mailto:reports@foobar.net\").unwrap()),\n        interval: AggregateFrequency::Weekly,\n    })\n    .await;\n\n    // List reports\n    let api = ManagementApi::default();\n    let ids = api\n        .request::<List<String>>(Method::GET, \"/api/queue/reports\")\n        .await\n        .unwrap()\n        .unwrap_data()\n        .items;\n    assert_eq!(ids.len(), 4);\n    let mut id_map = AHashMap::new();\n    let mut id_map_rev = AHashMap::new();\n    for (report, id) in api.get_reports(&ids).await.into_iter().zip(ids) {\n        let mut parts = id.split('!');\n        let report = report.unwrap();\n        let mut id_num = if parts.next().unwrap() == \"t\" {\n            assert!(matches!(report, Report::Tls { .. }));\n            2\n        } else {\n            assert!(matches!(report, Report::Dmarc { .. }));\n            0\n        };\n        let (domain, range_to, range_from) = match report {\n            Report::Dmarc {\n                domain,\n                range_to,\n                range_from,\n                ..\n            } => (domain, range_to, range_from),\n            Report::Tls {\n                domain,\n                range_to,\n                range_from,\n                ..\n            } => (domain, range_to, range_from),\n        };\n        assert_eq!(parts.next().unwrap(), domain);\n        let diff = range_to.to_timestamp() - range_from.to_timestamp();\n        if domain == \"foobar.org\" {\n            assert_eq!(diff, 86400);\n        } else {\n            assert_eq!(diff, 7 * 86400);\n            id_num += 1;\n        }\n        id_map.insert(char::from(b'a' + id_num).to_string(), id.clone());\n        id_map_rev.insert(id, char::from(b'a' + id_num).to_string());\n    }\n\n    // Test list search\n    for (query, expected_ids) in [\n        (\"/api/queue/reports?type=dmarc\", vec![\"a\", \"b\"]),\n        (\"/api/queue/reports?type=tls\", vec![\"c\", \"d\"]),\n        (\"/api/queue/reports?domain=foobar.org\", vec![\"a\", \"c\"]),\n        (\"/api/queue/reports?domain=foobar.net\", vec![\"b\", \"d\"]),\n        (\"/api/queue/reports?domain=foobar.org&type=dmarc\", vec![\"a\"]),\n        (\"/api/queue/reports?domain=foobar.net&type=tls\", vec![\"d\"]),\n    ] {\n        let expected_ids = HashSet::from_iter(expected_ids.into_iter().map(|s| s.to_string()));\n        let ids = api\n            .request::<List<String>>(Method::GET, query)\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .into_iter()\n            .map(|id| id_map_rev.get(&id).unwrap().clone())\n            .collect::<HashSet<_>>();\n        assert_eq!(ids, expected_ids, \"failed for {query}\");\n    }\n\n    // Cancel reports\n    for id in [\"a\", \"b\"] {\n        assert!(\n            api.request::<bool>(\n                Method::DELETE,\n                &format!(\"/api/queue/reports/{}\", id_map.get(id).unwrap(),)\n            )\n            .await\n            .unwrap()\n            .unwrap_data(),\n            \"failed for {id}\"\n        );\n    }\n    assert_eq!(\n        api.request::<List<String>>(Method::GET, \"/api/queue/reports\")\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .len(),\n        2\n    );\n    let mut ids = api\n        .get_reports(&[\n            id_map.get(\"a\").unwrap().clone(),\n            id_map.get(\"b\").unwrap().clone(),\n            id_map.get(\"c\").unwrap().clone(),\n            id_map.get(\"d\").unwrap().clone(),\n        ])\n        .await\n        .into_iter();\n    assert!(ids.next().unwrap().is_none());\n    assert!(ids.next().unwrap().is_none());\n    assert!(ids.next().unwrap().is_some());\n    assert!(ids.next().unwrap().is_some());\n\n    // Cancel all reports\n    assert!(\n        api.request::<bool>(Method::DELETE, \"/api/queue/reports\")\n            .await\n            .unwrap()\n            .unwrap_data()\n    );\n    assert_eq!(\n        api.request::<List<String>>(Method::GET, \"/api/queue/reports\")\n            .await\n            .unwrap()\n            .unwrap_data()\n            .items\n            .len(),\n        0\n    );\n}\n\nimpl ManagementApi {\n    async fn get_reports(&self, ids: &[String]) -> Vec<Option<Report>> {\n        let mut results = Vec::with_capacity(ids.len());\n\n        for id in ids {\n            let report = self\n                .request::<Report>(Method::GET, &format!(\"/api/queue/reports/{id}\",))\n                .await\n                .unwrap()\n                .try_unwrap_data();\n            results.push(report);\n        }\n\n        results\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    net::{IpAddr, Ipv4Addr, Ipv6Addr},\n    path::PathBuf,\n    sync::Arc,\n};\n\nuse common::{\n    Core, Data, Inner, Server,\n    config::{\n        server::{Listeners, ServerProtocol},\n        smtp::resolver::Tlsa,\n        spamfilter::IpResolver,\n    },\n    ipc::{QueueEvent, ReportingEvent},\n    manager::boot::{IpcReceivers, build_ipc},\n};\n\nuse http::HttpSessionManager;\nuse mail_auth::{MX, Txt, common::resolver::IntoFqdn};\nuse session::{DummyIo, TestSession};\nuse smtp::core::{Session, SmtpSessionManager};\nuse store::{BlobStore, Store, Stores};\nuse tokio::sync::{mpsc, watch};\nuse utils::config::Config;\n\nuse crate::{AssertConfig, store::cleanup::store_destroy};\n\npub mod config;\npub mod inbound;\npub mod lookup;\npub mod management;\npub mod outbound;\npub mod queue;\npub mod reporting;\npub mod session;\n\npub struct TempDir {\n    pub temp_dir: PathBuf,\n    pub delete: bool,\n}\n\nimpl TempDir {\n    pub fn new(name: &str, delete: bool) -> TempDir {\n        let mut temp_dir = std::env::temp_dir();\n        temp_dir.push(name);\n        if !temp_dir.exists() {\n            let _ = std::fs::create_dir(&temp_dir);\n        } else if delete {\n            let _ = std::fs::remove_dir_all(&temp_dir);\n            let _ = std::fs::create_dir(&temp_dir);\n        }\n        TempDir { temp_dir, delete }\n    }\n\n    pub fn update_config(&self, config: impl AsRef<str>) -> String {\n        config\n            .as_ref()\n            .replace(\"{TMP}\", self.temp_dir.to_str().unwrap())\n    }\n}\n\nimpl Drop for TempDir {\n    fn drop(&mut self) {\n        if self.delete {\n            let _ = std::fs::remove_dir_all(&self.temp_dir);\n        }\n    }\n}\n\npub fn add_test_certs(config: &str) -> String {\n    let mut cert_path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    cert_path.push(\"resources\");\n    cert_path.push(\"smtp\");\n    cert_path.push(\"certs\");\n    let mut cert = cert_path.clone();\n    cert.push(\"tls_cert.pem\");\n    let mut pk = cert_path.clone();\n    pk.push(\"tls_privatekey.pem\");\n\n    config\n        .replace(\"{CERT}\", cert.as_path().to_str().unwrap())\n        .replace(\"{PK}\", pk.as_path().to_str().unwrap())\n}\n\npub struct QueueReceiver {\n    store: Store,\n    blob_store: BlobStore,\n    pub queue_rx: mpsc::Receiver<QueueEvent>,\n}\n\npub struct ReportReceiver {\n    pub report_rx: mpsc::Receiver<ReportingEvent>,\n}\n\npub struct TestSMTP {\n    pub server: Server,\n    pub temp_dir: Option<TempDir>,\n    pub queue_receiver: QueueReceiver,\n    pub report_receiver: ReportReceiver,\n}\n\nconst CONFIG: &str = r#\"\n[session.connect]\nhostname = \"'mx.example.org'\"\ngreeting = \"'Test SMTP instance'\"\n\n[server.listener.smtp-debug]\nbind = ['127.0.0.1:9925']\nprotocol = 'smtp'\n\n[server.listener.lmtp-debug]\nbind = ['127.0.0.1:9924']\nprotocol = 'lmtp'\ntls.implicit = true\n\n[server.listener.management-debug]\nbind = ['127.0.0.1:9980']\nprotocol = 'http'\ntls.implicit = true\n\n[server.socket]\nreuse-addr = true\n\n[server.tls]\nenable = true\nimplicit = false\ncertificate = 'default'\n\n[certificate.default]\ncert = '%{file:{CERT}}%'\nprivate-key = '%{file:{PK}}%'\n\n[storage]\ndata = \"{STORE}\"\nfts = \"{STORE}\"\nblob = \"{STORE}\"\nlookup = \"{STORE}\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/queue.db\"\n\n#[store.\"foundationdb\"]\n#type = \"foundationdb\"\n\n[store.\"postgresql\"]\ntype = \"postgresql\"\nhost = \"localhost\"\nport = 5432\ndatabase = \"stalwart\"\nuser = \"postgres\"\npassword = \"mysecretpassword\"\n\n[store.\"mysql\"]\ntype = \"mysql\"\nhost = \"localhost\"\nport = 3307\ndatabase = \"stalwart\"\nuser = \"root\"\npassword = \"password\"\n\n\"#;\n\nimpl TestSMTP {\n    pub fn from_core(core: Core) -> Self {\n        Self::from_core_and_tempdir(core, Default::default(), None)\n    }\n\n    pub fn inner_with_rxs(&self) -> (Arc<Inner>, IpcReceivers) {\n        let (ipc, ipc_rxs) = build_ipc(false);\n\n        (\n            Inner {\n                shared_core: self.server.core.as_ref().clone().into_shared(),\n                data: Default::default(),\n                ipc,\n                cache: Default::default(),\n            }\n            .into(),\n            ipc_rxs,\n        )\n    }\n\n    fn from_core_and_tempdir(core: Core, data: Data, temp_dir: Option<TempDir>) -> Self {\n        let store = core.storage.data.clone();\n        let blob_store = core.storage.blob.clone();\n        let shared_core = core.into_shared();\n        let (ipc, mut ipc_rxs) = build_ipc(false);\n\n        TestSMTP {\n            queue_receiver: QueueReceiver {\n                store,\n                blob_store,\n                queue_rx: ipc_rxs.queue_rx.take().unwrap(),\n            },\n            report_receiver: ReportReceiver {\n                report_rx: ipc_rxs.report_rx.take().unwrap(),\n            },\n            server: Server {\n                core: shared_core.load_full(),\n                inner: Inner {\n                    shared_core,\n                    data,\n                    ipc,\n                    cache: Default::default(),\n                }\n                .into(),\n            },\n            temp_dir,\n        }\n    }\n\n    pub async fn new(name: &str, config: impl AsRef<str>) -> TestSMTP {\n        Self::with_database(name, config, \"rocksdb\").await\n    }\n\n    pub async fn with_database(\n        name: &str,\n        config: impl AsRef<str>,\n        store_id: impl AsRef<str>,\n    ) -> TestSMTP {\n        let temp_dir = TempDir::new(name, true);\n        let mut config = Config::new(\n            temp_dir\n                .update_config(add_test_certs(CONFIG) + config.as_ref())\n                .replace(\"{STORE}\", store_id.as_ref()),\n        )\n        .unwrap();\n        config.resolve_all_macros().await;\n        let stores = Stores::parse_all(&mut config, false).await;\n        let core = Core::parse(&mut config, stores, Default::default()).await;\n        let data = Data::parse(&mut config);\n        store_destroy(&core.storage.data).await;\n\n        Self::from_core_and_tempdir(core, data, Some(temp_dir))\n    }\n\n    pub async fn start(&self, protocols: &[ServerProtocol]) -> watch::Sender<bool> {\n        // Spawn listeners\n        let mut config = Config::new(CONFIG).unwrap();\n        let mut servers = Listeners::parse(&mut config);\n        servers.parse_tcp_acceptors(&mut config, self.server.inner.clone());\n\n        // Filter out protocols\n        servers\n            .servers\n            .retain(|server| protocols.contains(&server.protocol));\n\n        // Start servers\n        servers.bind_and_drop_priv(&mut config);\n        config.assert_no_errors();\n\n        servers\n            .spawn(|server, acceptor, shutdown_rx| {\n                match &server.protocol {\n                    ServerProtocol::Smtp | ServerProtocol::Lmtp => server.spawn(\n                        SmtpSessionManager::new(self.server.inner.clone()),\n                        self.server.inner.clone(),\n                        acceptor,\n                        shutdown_rx,\n                    ),\n                    ServerProtocol::Http => server.spawn(\n                        HttpSessionManager::new(self.server.inner.clone()),\n                        self.server.inner.clone(),\n                        acceptor,\n                        shutdown_rx,\n                    ),\n                    ServerProtocol::Imap | ServerProtocol::Pop3 | ServerProtocol::ManageSieve => {\n                        unreachable!()\n                    }\n                };\n            })\n            .0\n    }\n\n    pub fn new_session(&self) -> Session<DummyIo> {\n        Session::test(self.server.clone())\n    }\n\n    pub fn build_smtp(&self) -> Server {\n        self.server.clone()\n    }\n}\n\npub trait DnsCache {\n    fn txt_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: impl Into<Txt>,\n        valid_until: std::time::Instant,\n    );\n    fn ipv4_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: Vec<Ipv4Addr>,\n        valid_until: std::time::Instant,\n    );\n    fn ipv6_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: Vec<Ipv6Addr>,\n        valid_until: std::time::Instant,\n    );\n    fn dnsbl_add(&self, name: &str, value: Vec<Ipv4Addr>, valid_until: std::time::Instant);\n    fn ptr_add(&self, name: IpAddr, value: Vec<String>, valid_until: std::time::Instant);\n    fn mx_add<'x>(&self, name: impl IntoFqdn<'x>, value: Vec<MX>, valid_until: std::time::Instant);\n    fn tlsa_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: Arc<Tlsa>,\n        valid_until: std::time::Instant,\n    );\n}\n\nimpl DnsCache for Server {\n    fn txt_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: impl Into<Txt>,\n        valid_until: std::time::Instant,\n    ) {\n        self.inner.cache.dns_txt.insert_with_expiry(\n            name.into_fqdn().into_owned(),\n            value.into(),\n            valid_until,\n        );\n    }\n\n    fn ipv4_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: Vec<Ipv4Addr>,\n        valid_until: std::time::Instant,\n    ) {\n        self.inner.cache.dns_ipv4.insert_with_expiry(\n            name.into_fqdn().into_owned(),\n            Arc::new(value),\n            valid_until,\n        );\n    }\n\n    fn dnsbl_add(&self, name: &str, value: Vec<Ipv4Addr>, valid_until: std::time::Instant) {\n        self.inner.cache.dns_rbl.insert_with_expiry(\n            name.to_string(),\n            Some(Arc::new(IpResolver::new(\n                value\n                    .iter()\n                    .copied()\n                    .next()\n                    .unwrap_or(Ipv4Addr::BROADCAST)\n                    .into(),\n            ))),\n            valid_until,\n        );\n    }\n\n    fn ipv6_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: Vec<Ipv6Addr>,\n        valid_until: std::time::Instant,\n    ) {\n        self.inner.cache.dns_ipv6.insert_with_expiry(\n            name.into_fqdn().into_owned(),\n            Arc::new(value),\n            valid_until,\n        );\n    }\n\n    fn ptr_add(&self, name: IpAddr, value: Vec<String>, valid_until: std::time::Instant) {\n        self.inner\n            .cache\n            .dns_ptr\n            .insert_with_expiry(name, Arc::new(value), valid_until);\n    }\n\n    fn mx_add<'x>(&self, name: impl IntoFqdn<'x>, value: Vec<MX>, valid_until: std::time::Instant) {\n        self.inner.cache.dns_mx.insert_with_expiry(\n            name.into_fqdn().into_owned(),\n            Arc::new(value),\n            valid_until,\n        );\n    }\n\n    fn tlsa_add<'x>(\n        &self,\n        name: impl IntoFqdn<'x>,\n        value: Arc<Tlsa>,\n        valid_until: std::time::Instant,\n    ) {\n        self.inner.cache.dns_tlsa.insert_with_expiry(\n            name.into_fqdn().into_owned(),\n            value,\n            valid_until,\n        );\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/dane.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::{TestMessage, TestQueueEvent, TestReportingEvent},\n    session::{TestSession, VerifyResponse},\n};\nuse common::{\n    Core,\n    config::{\n        server::ServerProtocol,\n        smtp::resolver::{DnssecResolver, Resolvers, Tlsa, TlsaEntry},\n    },\n    ipc::PolicyType,\n};\nuse mail_auth::{\n    MX, MessageAuthenticator,\n    common::parse::TxtRecordParser,\n    hickory_resolver::{\n        TokioResolver,\n        config::{ResolverConfig, ResolverOpts},\n        name_server::TokioConnectionProvider,\n    },\n    mta_sts::{ReportUri, TlsRpt},\n    report::tlsrpt::ResultType,\n};\nuse rustls_pki_types::CertificateDer;\nuse smtp::outbound::dane::{dnssec::TlsaLookup, verify::TlsaVerify};\nuse smtp::queue::{Error, ErrorDetails, Status};\nuse std::{\n    collections::BTreeSet,\n    fs::{self, File},\n    io::{BufRead, BufReader},\n    num::ParseIntError,\n    path::PathBuf,\n    sync::Arc,\n    time::{Duration, Instant},\n};\n\nconst LOCAL: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[report.tls.aggregate]\nsend = \"weekly\"\n\n[queue.tls.default]\ndane = \"require\"\nstarttls = \"require\"\n\n\"#;\n\nconst REMOTE: &str = \"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[session.data.add-headers]\nreceived = true\nreceived-spf = true\nauth-results = true\nmessage-id = true\ndate = true\nreturn-path = false\n\n\";\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn dane_verify() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"smtp_dane_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n\n    // Fail on missing TLSA record\n    let mut local = TestSMTP::new(\"smtp_dane_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"mx.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.ipv4_add(\n        \"mx.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.txt_add(\n        \"_smtp._tls.foobar.org\",\n        TlsRpt::parse(b\"v=TLSRPTv1; rua=mailto:reports@foobar.org\").unwrap(),\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (DANE failed to authenticate\")\n        .assert_contains(\"No TLSA reco=\")\n        .assert_contains(\"rds found\");\n    local.queue_receiver.read_event().await.assert_done();\n    local.queue_receiver.assert_no_events();\n\n    // Expect TLS failure report\n    let report = local.report_receiver.read_report().await.unwrap_tls();\n    assert_eq!(report.domain, \"foobar.org\");\n    assert_eq!(report.policy, PolicyType::Tlsa(None));\n    assert_eq!(\n        report.failure.as_ref().unwrap().result_type,\n        ResultType::DaneRequired\n    );\n    assert_eq!(\n        report.failure.as_ref().unwrap().receiving_mx_hostname,\n        Some(\"mx.foobar.org\".to_string())\n    );\n    assert_eq!(\n        report.tls_record.rua,\n        vec![ReportUri::Mail(\"reports@foobar.org\".to_string())]\n    );\n\n    // DANE failure with no matching certificates\n    let tlsa = Arc::new(Tlsa {\n        entries: vec![TlsaEntry {\n            is_end_entity: true,\n            is_sha256: true,\n            is_spki: true,\n            data: vec![1, 2, 3],\n        }],\n        has_end_entities: true,\n        has_intermediates: false,\n    });\n    core.tlsa_add(\n        \"_25._tcp.mx.foobar.org\",\n        tlsa.clone(),\n        Instant::now() + Duration::from_secs(10),\n    );\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (DANE failed to authenticate\")\n        .assert_contains(\"No matching \")\n        .assert_contains(\"certificates found\");\n    local.queue_receiver.read_event().await.assert_done();\n    local.queue_receiver.assert_no_events();\n\n    // Expect TLS failure report\n    let report = local.report_receiver.read_report().await.unwrap_tls();\n    assert_eq!(report.policy, PolicyType::Tlsa(tlsa.into()));\n    assert_eq!(\n        report.failure.as_ref().unwrap().result_type,\n        ResultType::ValidationFailure\n    );\n    remote.queue_receiver.assert_no_events();\n\n    // DANE successful delivery\n    let tlsa = Arc::new(Tlsa {\n        entries: vec![TlsaEntry {\n            is_end_entity: true,\n            is_sha256: true,\n            is_spki: true,\n            data: vec![\n                73, 186, 44, 106, 13, 198, 100, 180, 0, 44, 158, 188, 15, 195, 39, 198, 61, 254,\n                215, 237, 100, 26, 15, 155, 219, 235, 120, 64, 128, 172, 17, 0,\n            ],\n        }],\n        has_end_entities: true,\n        has_intermediates: false,\n    });\n    core.tlsa_add(\n        \"_25._tcp.mx.foobar.org\",\n        tlsa.clone(),\n        Instant::now() + Duration::from_secs(10),\n    );\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local.queue_receiver.read_event().await.assert_done();\n    local.queue_receiver.assert_no_events();\n    remote\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&remote.queue_receiver)\n        .await\n        .assert_contains(\"using TLSv1.3 with cipher\");\n\n    // Expect TLS success report\n    let report = local.report_receiver.read_report().await.unwrap_tls();\n    assert_eq!(report.policy, PolicyType::Tlsa(tlsa.into()));\n    assert!(report.failure.is_none());\n}\n\n#[tokio::test]\nasync fn dane_test() {\n    let conf = ResolverConfig::cloudflare_tls();\n    let mut opts = ResolverOpts::default();\n    opts.validate = true;\n    opts.try_tcp_on_error = true;\n\n    let mut core = Core::default();\n    core.smtp.resolvers = Resolvers {\n        dns: MessageAuthenticator::new_cloudflare().unwrap(),\n        dnssec: DnssecResolver {\n            resolver: TokioResolver::builder_with_config(conf, TokioConnectionProvider::default())\n                .with_options(opts)\n                .build(),\n        },\n    };\n    let r = TestSMTP::from_core(core).build_smtp();\n\n    // Add dns entries\n    let mut path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    path.push(\"resources\");\n    path.push(\"smtp\");\n    path.push(\"dane\");\n    let mut file = path.clone();\n    file.push(\"dns.txt\");\n\n    let mut hosts = BTreeSet::new();\n    let mut tlsa = Tlsa {\n        entries: Vec::new(),\n        has_end_entities: false,\n        has_intermediates: false,\n    };\n    let mut hostname = String::new();\n\n    for line in BufReader::new(File::open(file).unwrap()).lines() {\n        let line = line.unwrap();\n        let mut is_end_entity = false;\n        for (pos, item) in line.split_whitespace().enumerate() {\n            match pos {\n                0 => {\n                    if hostname != item && !hostname.is_empty() {\n                        r.tlsa_add(\n                            hostname,\n                            tlsa.into(),\n                            Instant::now() + Duration::from_secs(30),\n                        );\n                        tlsa = Tlsa {\n                            entries: Vec::new(),\n                            has_end_entities: false,\n                            has_intermediates: false,\n                        };\n                    }\n                    hosts.insert(item.strip_prefix(\"_25._tcp.\").unwrap().to_string());\n                    hostname = item.to_string();\n                }\n                1 => {\n                    is_end_entity = item == \"3\";\n                }\n                4 => {\n                    if is_end_entity {\n                        tlsa.has_end_entities = true;\n                    } else {\n                        tlsa.has_intermediates = true;\n                    }\n                    tlsa.entries.push(TlsaEntry {\n                        is_end_entity,\n                        is_sha256: true,\n                        is_spki: true,\n                        data: decode_hex(item).unwrap(),\n                    });\n                }\n                _ => (),\n            }\n        }\n    }\n    r.tlsa_add(\n        hostname,\n        tlsa.into(),\n        Instant::now() + Duration::from_secs(30),\n    );\n\n    // Add certificates\n    assert!(!hosts.is_empty());\n    for host in hosts {\n        // Add certificates\n        let mut certs = Vec::new();\n        for num in 0..6 {\n            let mut file = path.clone();\n            file.push(format!(\"{host}.{num}.cert\"));\n            if file.exists() {\n                certs.push(CertificateDer::from(fs::read(file).unwrap()));\n            } else {\n                break;\n            }\n        }\n\n        // Successful DANE verification\n        let tlsa = r\n            .tlsa_lookup(format!(\"_25._tcp.{host}.\"))\n            .await\n            .unwrap()\n            .unwrap();\n\n        assert_eq!(tlsa.verify(0, &host, Some(&certs)), Ok(()));\n\n        // Failed DANE verification\n        certs.remove(0);\n        assert_eq!(\n            tlsa.verify(0, &host, Some(&certs)),\n            Err(Status::PermanentFailure(ErrorDetails {\n                entity: host.into(),\n                details: Error::DaneError(\"No matching certificates found in TLSA records\".into())\n            }))\n        );\n    }\n}\n\npub fn decode_hex(s: &str) -> Result<Vec<u8>, ParseIntError> {\n    (0..s.len())\n        .step_by(2)\n        .map(|i| u8::from_str_radix(&s[i..i + 2], 16))\n        .collect()\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/extensions.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::config::server::ServerProtocol;\nuse mail_auth::MX;\nuse smtp_proto::{MAIL_REQUIRETLS, MAIL_RET_HDRS, MAIL_SMTPUTF8, RCPT_NOTIFY_NEVER};\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::{TestMessage, TestQueueEvent},\n    session::{TestSession, VerifyResponse},\n};\n\nconst LOCAL: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[session.extensions]\ndsn = true\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[session.data.limits]\nsize = 1500\n\n[session.extensions]\ndsn = true\nrequiretls = true\n\n[session.data.add-headers]\nreceived = true\nreceived-spf = true\nauth-results = true\nmessage-id = true\ndate = true\nreturn-path = false\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn extensions() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"smtp_ext_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n\n    // Successful delivery with DSN\n    let mut local = TestSMTP::new(\"smtp_ext_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"mx.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.ipv4_add(\n        \"mx.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\n            \"john@test.org\",\n            &[\"<bill@foobar.org> NOTIFY=SUCCESS,FAILURE\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n\n    local\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (delivered to\")\n        .assert_contains(\"Final-Recipient: rfc822;bill@foobar.org\")\n        .assert_contains(\"Action: delivered\");\n    local.queue_receiver.read_event().await.assert_done();\n    remote\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&remote.queue_receiver)\n        .await\n        .assert_contains(\"using TLSv1.3 with cipher\");\n\n    // Test SIZE extension\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:arc\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (host 'mx.foobar.org' rejected command 'MAIL FROM:\")\n        .assert_contains(\"Action: failed\")\n        .assert_contains(\"Diagnostic-Code: smtp;552\")\n        .assert_contains(\"Status: 5.3.4\");\n    local.queue_receiver.read_event().await.assert_done();\n    remote.queue_receiver.assert_no_events();\n\n    // Test DSN, SMTPUTF8 and REQUIRETLS extensions\n    session\n        .send_message(\n            \"<john@test.org> ENVID=abc123 RET=HDRS REQUIRETLS SMTPUTF8\",\n            &[\"<bill@foobar.org> NOTIFY=NEVER\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local.queue_receiver.read_event().await.assert_done();\n    let message = remote.queue_receiver.expect_message().await;\n    assert_eq!(message.message.env_id, Some(\"abc123\".into()));\n    assert!((message.message.flags & MAIL_RET_HDRS) != 0);\n    assert!((message.message.flags & MAIL_REQUIRETLS) != 0);\n    assert!((message.message.flags & MAIL_SMTPUTF8) != 0);\n    assert!((message.message.recipients.last().unwrap().flags & RCPT_NOTIFY_NEVER) != 0);\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/fallback_relay.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::config::server::ServerProtocol;\nuse mail_auth::MX;\nuse store::write::now;\n\nuse crate::smtp::{DnsCache, TestSMTP, session::TestSession};\n\nconst LOCAL: &str = r#\"\n[queue.strategy]\nroute = [{if = \"retry_num > 0\", then = \"'fallback'\"},\n            {else = \"'mx'\"}]\n\n[session.rcpt]\nrelay = true\nmax-recipients = 100\n\n[session.extensions]\ndsn = true\n\n[queue.route.fallback]\ntype = \"relay\"\naddress = fallback.foobar.org\nport = 9925\nprotocol = 'smtp'\nconcurrency = 5\n\n[queue.route.fallback.tls]\nimplicit = false\nallow-invalid-certs = true\n\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.extensions]\ndsn = true\nchunking = false\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn fallback_relay() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"smtp_fallback_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n    let mut local = TestSMTP::new(\"smtp_fallback_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"_dns_error.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n    /*core.ipv4_add(\n        \"unreachable.foobar.org\",\n        vec![\"127.0.0.2\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );*/\n    core.ipv4_add(\n        \"fallback.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    let mut retry = local.queue_receiver.expect_message().await;\n    let prev_due = retry.message.recipients[0].retry.due;\n    let next_due = now();\n    let queue_id = retry.queue_id;\n    retry.message.recipients[0].retry.due = next_due;\n    retry.save_changes(&core, prev_due.into()).await;\n    local\n        .queue_receiver\n        .delivery_attempt(queue_id)\n        .await\n        .try_deliver(core.clone());\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    remote.queue_receiver.expect_message().await;\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/ip_lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::config::server::ServerProtocol;\nuse mail_auth::{IpLookupStrategy, MX};\n\nuse crate::smtp::{DnsCache, TestSMTP, session::TestSession};\n\nconst LOCAL: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[queue.route.mx]\nip-lookup = \"ipv6_then_ipv4\"\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn ip_lookup_strategy() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"smtp_iplookup_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n\n    for strategy in [IpLookupStrategy::Ipv6Only, IpLookupStrategy::Ipv6thenIpv4] {\n        //println!(\"-> Strategy: {:?}\", strategy);\n        // Add mock DNS entries\n        let mut local = TestSMTP::new(\"smtp_iplookup_local\", LOCAL).await;\n        let core = local.build_smtp();\n        core.mx_add(\n            \"foobar.org\",\n            vec![MX {\n                exchanges: vec![\"mx.foobar.org\".to_string()],\n                preference: 10,\n            }],\n            Instant::now() + Duration::from_secs(10),\n        );\n        if matches!(strategy, IpLookupStrategy::Ipv6thenIpv4) {\n            core.ipv4_add(\n                \"mx.foobar.org\",\n                vec![\"127.0.0.1\".parse().unwrap()],\n                Instant::now() + Duration::from_secs(10),\n            );\n        }\n        core.ipv6_add(\n            \"mx.foobar.org\",\n            vec![\"::1\".parse().unwrap()],\n            Instant::now() + Duration::from_secs(10),\n        );\n\n        // Retry on failed STARTTLS\n        let mut session = local.new_session();\n        session.data.remote_ip_str = \"10.0.0.1\".into();\n        session.eval_session_params().await;\n        session.ehlo(\"mx.test.org\").await;\n        session\n            .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n            .await;\n        local\n            .queue_receiver\n            .expect_message_then_deliver()\n            .await\n            .try_deliver(core.clone());\n        tokio::time::sleep(Duration::from_millis(100)).await;\n        if matches!(strategy, IpLookupStrategy::Ipv6thenIpv4) {\n            remote.queue_receiver.expect_message().await;\n        } else {\n            let message = local.queue_receiver.last_queued_message().await;\n            let status = message.message.recipients[0].status.to_string();\n            assert!(\n                status.contains(\"Connection refused\"),\n                \"Message: {:?}\",\n                message\n            );\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/lmtp.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::TestMessage,\n    queue::QueuedEvents,\n    session::{TestSession, VerifyResponse},\n};\nuse common::{\n    config::{server::ServerProtocol, smtp::queue::QueueName},\n    ipc::QueueEvent,\n};\nuse smtp::queue::spool::{QUEUE_REFRESH, SmtpSpool};\nuse store::write::now;\n\nconst REMOTE: &str = \"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[session.extensions]\ndsn = true\n\";\n\nconst LOCAL: &str = r#\"\n[queue.strategy]\nroute = [{if = \"rcpt_domain = 'foobar.org'\", then = \"'lmtp'\"},\n            {else = \"'mx'\"}]\nschedule = [{if = \"rcpt_domain = 'foobar.org'\", then = \"'foobar'\"},\n            {else = \"'default'\"}]\n\n[session.rcpt]\nrelay = true\nmax-recipients = 100\n\n[session.extensions]\ndsn = true\n\n[queue.schedule.default]\nretry = \"1s\"\nnotify = \"1s\"\nexpire = \"5s\"\nqueue-name = \"default\"\n\n[queue.schedule.foobar]\nretry = \"1s\"\nnotify = [\"1s\", \"2s\"]\nexpire = \"4s\"\nqueue-name = \"default\"\n\n[queue.connection.default.timeout]\nconnect = \"1s\"\ndata = \"50ms\"\n\n[queue.route.lmtp]\ntype = \"relay\"\naddress = lmtp.foobar.org\nport = 9924\nprotocol = 'lmtp'\nconcurrency = 5\n\n[queue.route.lmtp.tls]\nimplicit = true\nallow-invalid-certs = true\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn lmtp_delivery() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"lmtp_delivery_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Lmtp]).await;\n\n    // Multiple delivery attempts\n    let mut local = TestSMTP::new(\"lmtp_delivery_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.ipv4_add(\n        \"lmtp.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\n            \"john@test.org\",\n            &[\n                \"<bill@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<jane@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<john@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<delay@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<fail@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<invalid@domain.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n            ],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    let mut dsn = Vec::new();\n    loop {\n        match local.queue_receiver.try_read_event().await {\n            Some(QueueEvent::Refresh | QueueEvent::WorkerDone { .. }) => {}\n            Some(QueueEvent::Paused(_)) | Some(QueueEvent::ReloadSettings) => unreachable!(),\n            None | Some(QueueEvent::Stop) => break,\n        }\n\n        let mut events = core.all_queued_messages().await;\n        if events.messages.is_empty() {\n            let now = now();\n            if events.next_refresh < now + QUEUE_REFRESH {\n                tokio::time::sleep(Duration::from_secs(events.next_refresh - now)).await;\n                events = core.all_queued_messages().await;\n            } else {\n                break;\n            }\n        }\n        for event in events.messages {\n            let message = core\n                .read_message(event.queue_id, QueueName::default())\n                .await\n                .unwrap();\n            if message.message.return_path.is_empty() {\n                message.clone().remove(&core, event.due.into()).await;\n                dsn.push(message);\n            } else {\n                event.try_deliver(core.clone());\n                tokio::time::sleep(Duration::from_millis(100)).await;\n            }\n        }\n    }\n    local.queue_receiver.assert_queue_is_empty().await;\n    assert_eq!(dsn.len(), 4);\n\n    let mut dsn = dsn.into_iter();\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (delivered to\")\n        .assert_contains(\"<jane@foobar.org> (delivered to\")\n        .assert_contains(\"<john@foobar.org> (delivered to\")\n        .assert_contains(\"<invalid@domain.org> (failed to lookup\")\n        .assert_contains(\"<fail@foobar.org> (host 'lmtp.foobar.org' rejected command\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<delay@foobar.org> (host 'lmtp.foobar.org' rejected\")\n        .assert_contains(\"Action: delayed\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<delay@foobar.org> (host 'lmtp.foobar.org' rejected\")\n        .assert_contains(\"Action: delayed\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<delay@foobar.org> (host 'lmtp.foobar.org' rejected\")\n        .assert_contains(\"Action: failed\");\n\n    assert_eq!(\n        remote\n            .queue_receiver\n            .expect_message()\n            .await\n            .message\n            .recipients\n            .into_iter()\n            .map(|r| r.address().to_string())\n            .collect::<Vec<_>>(),\n        vec![\n            \"bill@foobar.org\".to_string(),\n            \"jane@foobar.org\".to_string(),\n            \"john@foobar.org\".to_string()\n        ]\n    );\n    remote.queue_receiver.assert_no_events();\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod dane;\npub mod extensions;\npub mod fallback_relay;\npub mod ip_lookup;\npub mod lmtp;\npub mod mta_sts;\npub mod smtp;\npub mod throttle;\npub mod tls;\n"
  },
  {
    "path": "tests/src/smtp/outbound/mta_sts.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    sync::Arc,\n    time::{Duration, Instant},\n};\n\nuse common::{\n    config::{server::ServerProtocol, smtp::resolver::Policy},\n    ipc::PolicyType,\n};\nuse mail_auth::{\n    MX,\n    common::parse::TxtRecordParser,\n    mta_sts::{MtaSts, ReportUri, TlsRpt},\n    report::tlsrpt::ResultType,\n};\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::{TestMessage, TestQueueEvent, TestReportingEvent},\n    session::{TestSession, VerifyResponse},\n};\nuse smtp::outbound::mta_sts::{lookup::STS_TEST_POLICY, parse::ParsePolicy};\n\nconst LOCAL: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[queue.tls.default]\nmta-sts = \"require\"\nallow-invalid-certs = false\n\n[report.tls.aggregate]\nsend = \"weekly\"\n\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[session.data.add-headers]\nreceived = true\nreceived-spf = true\nauth-results = true\nmessage-id = true\ndate = true\nreturn-path = false\n\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn mta_sts_verify() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"smtp_mta_sts_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n\n    // Fail on missing MTA-STS record\n    let mut local = TestSMTP::new(\"smtp_mta_sts_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"mx.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.ipv4_add(\n        \"mx.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.txt_add(\n        \"_smtp._tls.foobar.org\",\n        TlsRpt::parse(b\"v=TLSRPTv1; rua=mailto:reports@foobar.org\").unwrap(),\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (MTA-STS failed to authenticate\")\n        .assert_contains(\"Record not f=\")\n        .assert_contains(\"ound\");\n    local.queue_receiver.read_event().await.assert_done();\n\n    // Expect TLS failure report\n    let report = local.report_receiver.read_report().await.unwrap_tls();\n    assert_eq!(report.domain, \"foobar.org\");\n    assert_eq!(report.policy, PolicyType::Sts(None));\n    assert_eq!(\n        report.failure.as_ref().unwrap().result_type,\n        ResultType::Other\n    );\n    assert_eq!(\n        report.tls_record.rua,\n        vec![ReportUri::Mail(\"reports@foobar.org\".to_string())]\n    );\n\n    // MTA-STS policy fetch failure\n    core.txt_add(\n        \"_mta-sts.foobar.org\",\n        MtaSts::parse(b\"v=STSv1; id=policy_will_fail;\").unwrap(),\n        Instant::now() + Duration::from_secs(10),\n    );\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (MTA-STS failed to authenticate\")\n        .assert_contains(\"No 'mx' entries found\");\n    local.queue_receiver.read_event().await.assert_done();\n\n    // Expect TLS failure report\n    let report = local.report_receiver.read_report().await.unwrap_tls();\n    assert_eq!(report.policy, PolicyType::Sts(None));\n    assert_eq!(\n        report.failure.as_ref().unwrap().result_type,\n        ResultType::StsPolicyInvalid\n    );\n\n    // MTA-STS policy does not authorize mx.foobar.org\n    let policy = concat!(\n        \"version: STSv1\\n\",\n        \"mode: enforce\\n\",\n        \"mx: mail.foobar.net\\n\",\n        \"max_age: 604800\\n\"\n    );\n    STS_TEST_POLICY.lock().extend_from_slice(policy.as_bytes());\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<bill@foobar.org> (MTA-STS failed to authenticate\")\n        .assert_contains(\"not authorized by policy\");\n    local.queue_receiver.read_event().await.assert_done();\n\n    // Expect TLS failure report\n    let report = local.report_receiver.read_report().await.unwrap_tls();\n    assert_eq!(\n        report.policy,\n        PolicyType::Sts(\n            Arc::new(Policy::parse(policy, \"policy_will_fail\".to_string()).unwrap()).into()\n        )\n    );\n    assert_eq!(\n        report.failure.as_ref().unwrap().receiving_mx_hostname,\n        Some(\"mx.foobar.org\".to_string())\n    );\n    assert_eq!(\n        report.failure.as_ref().unwrap().result_type,\n        ResultType::ValidationFailure\n    );\n    remote.queue_receiver.assert_no_events();\n\n    // MTA-STS successful validation\n    core.txt_add(\n        \"_mta-sts.foobar.org\",\n        MtaSts::parse(b\"v=STSv1; id=policy_will_work;\").unwrap(),\n        Instant::now() + Duration::from_secs(10),\n    );\n    let policy = concat!(\n        \"version: STSv1\\n\",\n        \"mode: enforce\\n\",\n        \"mx: *.foobar.org\\n\",\n        \"max_age: 604800\\n\"\n    );\n    STS_TEST_POLICY.lock().clear();\n    STS_TEST_POLICY.lock().extend_from_slice(policy.as_bytes());\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local.queue_receiver.read_event().await.assert_done();\n    remote\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&remote.queue_receiver)\n        .await\n        .assert_contains(\"using TLSv1.3 with cipher\");\n\n    // Expect TLS success report\n    let report = local.report_receiver.read_report().await.unwrap_tls();\n    assert_eq!(\n        report.policy,\n        PolicyType::Sts(\n            Arc::new(Policy::parse(policy, \"policy_will_work\".to_string()).unwrap()).into()\n        )\n    );\n    assert!(report.failure.is_none());\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/smtp.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::{\n    config::{server::ServerProtocol, smtp::queue::QueueName},\n    ipc::QueueEvent,\n};\nuse mail_auth::MX;\nuse store::write::now;\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::{TestMessage, TestQueueEvent},\n    queue::QueuedEvents,\n    session::{TestSession, VerifyResponse},\n};\nuse smtp::queue::spool::{QUEUE_REFRESH, SmtpSpool};\n\nconst LOCAL: &str = r#\"\n[session.rcpt]\nrelay = true\nmax-recipients = 100\n\n[session.extensions]\ndsn = true\n\n[queue.schedule.default]\nretry = \"1s\"\nnotify = \"1s\"\nexpire = \"7s\"\nqueue-name = \"default\"\n\n[queue.schedule.foobar-org]\nretry = \"1s\"\nnotify = [\"1s\", \"2s\"]\nexpire = \"6s\"\nqueue-name = \"default\"\n\n[queue.schedule.foobar-com]\nretry = \"1s\"\nnotify = [\"5s\", \"6s\"]\nexpire = \"7s\"\nqueue-name = \"default\"\n\n\n[queue.strategy]\nschedule = [{if = \"rcpt_domain == 'foobar.org'\", then = \"'foobar-org'\"},\n            {if = \"rcpt_domain == 'foobar.com'\", then = \"'foobar-com'\"},\n            {else = \"'default'\"}]\n\n[spam-filter]\nenable = false\n\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[session.extensions]\ndsn = true\nchunking = false\n\n[spam-filter]\nenable = false\n\n\"#;\n\nconst SMUGGLER: &str = r#\"From: Joe SixPack <john@foobar.net>\nTo: Suzie Q <suzie@foobar.org>\nSubject: Is dinner ready?\n\nHi.\n\nWe lost the game. Are you hungry yet?\n.hey\nJoe.\n\n<SEP>.\nMAIL FROM:<admin@foobar.net>\nRCPT TO:<ok@foobar.org>\nDATA\nFrom: Joe SixPack <admin@foobar.net>\nTo: Suzie Q <suzie@foobar.org>\nSubject: smuggled message\n\nThis is a smuggled message\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn smtp_delivery() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"smtp_delivery_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n    let remote_core = remote.build_smtp();\n\n    // Multiple delivery attempts\n    let mut local = TestSMTP::new(\"smtp_delivery_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    for domain in [\"foobar.org\", \"foobar.net\", \"foobar.com\"] {\n        core.mx_add(\n            domain,\n            vec![MX {\n                exchanges: vec![format!(\"mx1.{domain}\"), format!(\"mx2.{domain}\")],\n                preference: 10,\n            }],\n            Instant::now() + Duration::from_secs(10),\n        );\n        core.ipv4_add(\n            format!(\"mx1.{domain}\"),\n            vec![\"127.0.0.1\".parse().unwrap()],\n            Instant::now() + Duration::from_secs(30),\n        );\n        core.ipv4_add(\n            format!(\"mx2.{domain}\"),\n            vec![\"127.0.0.1\".parse().unwrap()],\n            Instant::now() + Duration::from_secs(30),\n        );\n    }\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\n            \"john@test.org\",\n            &[\n                \"<ok@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<delay@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<fail@foobar.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<ok@foobar.net> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<delay@foobar.net> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<fail@foobar.net> NOTIFY=SUCCESS,DELAY,FAILURE\",\n                \"<invalid@domain.org> NOTIFY=SUCCESS,DELAY,FAILURE\",\n            ],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    let message = local.queue_receiver.expect_message().await;\n    let num_recipients = message.message.recipients.len();\n    assert_eq!(num_recipients, 7);\n    local\n        .queue_receiver\n        .delivery_attempt(message.queue_id)\n        .await\n        .try_deliver(core.clone());\n    let mut dsn = Vec::new();\n    let mut rcpt_retries = vec![0; num_recipients];\n    loop {\n        match local.queue_receiver.try_read_event().await {\n            Some(QueueEvent::Refresh | QueueEvent::WorkerDone { .. }) => {}\n            Some(QueueEvent::Paused(_)) | Some(QueueEvent::ReloadSettings) => unreachable!(),\n            None | Some(QueueEvent::Stop) => {\n                break;\n            }\n        }\n\n        let mut events = core.all_queued_messages().await;\n        if events.messages.is_empty() {\n            let now = now();\n            if events.next_refresh < now + QUEUE_REFRESH {\n                tokio::time::sleep(Duration::from_secs(events.next_refresh - now)).await;\n                events = core.all_queued_messages().await;\n            } else {\n                break;\n            }\n        }\n        for event in events.messages {\n            let message = core\n                .read_message(event.queue_id, QueueName::default())\n                .await\n                .unwrap();\n            if message.message.return_path.is_empty() {\n                message.clone().remove(&core, event.due.into()).await;\n                dsn.push(message);\n            } else {\n                for (idx, rcpt) in message.message.recipients.iter().enumerate() {\n                    rcpt_retries[idx] = rcpt.retry.inner;\n                }\n                event.try_deliver(core.clone());\n                tokio::time::sleep(Duration::from_millis(100)).await;\n            }\n        }\n    }\n    assert_eq!(rcpt_retries[0], 0, \"retries {rcpt_retries:?}\");\n    assert!(rcpt_retries[1] >= 5, \"retries {rcpt_retries:?}\");\n    assert_eq!(rcpt_retries[2], 0, \"retries {rcpt_retries:?}\");\n    assert_eq!(rcpt_retries[3], 0, \"retries {rcpt_retries:?}\");\n    assert!(rcpt_retries[4] >= 5, \"retries {rcpt_retries:?}\");\n    assert_eq!(rcpt_retries[5], 0, \"retries {rcpt_retries:?}\");\n    assert_eq!(rcpt_retries[6], 0, \"retries {rcpt_retries:?}\");\n    assert!(\n        rcpt_retries[1] >= rcpt_retries[4],\n        \"retries {rcpt_retries:?}\"\n    );\n\n    local.queue_receiver.assert_queue_is_empty().await;\n    assert_eq!(dsn.len(), 5);\n\n    let mut dsn = dsn.into_iter();\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<ok@foobar.net> (delivered to\")\n        .assert_contains(\"<ok@foobar.org> (delivered to\")\n        .assert_contains(\"<invalid@domain.org> (failed to lookup\")\n        .assert_contains(\"<fail@foobar.net> (host \")\n        .assert_contains(\"<fail@foobar.org> (host \");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<delay@foobar.net> (host \")\n        .assert_contains(\"<delay@foobar.org> (host \")\n        .assert_contains(\"Action: delayed\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<delay@foobar.org> (host \")\n        .assert_contains(\"Action: delayed\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<delay@foobar.org> (host \");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(&local.queue_receiver)\n        .await\n        .assert_contains(\"<delay@foobar.net> (host \")\n        .assert_contains(\"Action: failed\");\n\n    let mut recipients = remote\n        .queue_receiver\n        .consume_message(&remote_core)\n        .await\n        .message\n        .recipients\n        .into_iter()\n        .map(|r| r.address().to_string())\n        .collect::<Vec<_>>();\n    recipients.extend(\n        remote\n            .queue_receiver\n            .consume_message(&remote_core)\n            .await\n            .message\n            .recipients\n            .into_iter()\n            .map(|r| r.address().to_string()),\n    );\n    recipients.sort();\n    assert_eq!(\n        recipients,\n        vec![\"ok@foobar.net\".to_string(), \"ok@foobar.org\".to_string()]\n    );\n\n    remote.queue_receiver.assert_no_events();\n\n    // SMTP smuggling\n    for separator in [\"\\n\", \"\\r\"].iter() {\n        session.data.remote_ip_str = \"10.0.0.2\".into();\n        session.eval_session_params().await;\n        session.ehlo(\"mx.test.org\").await;\n\n        let message = SMUGGLER\n            .replace('\\r', \"\")\n            .replace('\\n', \"\\r\\n\")\n            .replace(\"<SEP>\", separator);\n\n        session\n            .send_message(\"john@doe.org\", &[\"bill@foobar.com\"], &message, \"250\")\n            .await;\n        local\n            .queue_receiver\n            .expect_message_then_deliver()\n            .await\n            .try_deliver(core.clone());\n        local\n            .queue_receiver\n            .read_event()\n            .await\n            .assert_refresh_or_done();\n\n        let message = remote\n            .queue_receiver\n            .consume_message(&remote_core)\n            .await\n            .read_message(&remote.queue_receiver)\n            .await;\n\n        assert!(\n            message.contains(\"This is a smuggled message\"),\n            \"message: {:?}\",\n            message\n        );\n        assert!(\n            message.contains(\"We lost the game.\"),\n            \"message: {:?}\",\n            message\n        );\n        assert!(\n            message.contains(&format!(\"{separator}..\\r\\nMAIL FROM:<\",)),\n            \"message: {:?}\",\n            message\n        );\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/throttle.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::TestQueueEvent,\n    queue::{build_rcpt, manager::new_message},\n    session::TestSession,\n};\nuse mail_auth::MX;\nuse smtp::queue::{Message, QueueEnvelope, Recipient, throttle::IsAllowed};\nuse std::{\n    net::{IpAddr, Ipv4Addr},\n    time::{Duration, Instant},\n};\nuse store::write::now;\n\nconst CONFIG: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[queue.schedule.default]\nretry = \"1h\"\nnotify = \"1h\"\nexpire = \"1h\"\n\n[[queue.limiter.outbound]]\nmatch = \"sender_domain = 'foobar.org'\"\nkey = 'sender_domain'\nenable = true\n\n[[queue.limiter.outbound]]\nmatch = \"sender_domain = 'foobar.net'\"\nkey = 'sender_domain'\nrate = '1/30m'\nenable = true\n\n[[queue.limiter.outbound]]\nmatch = \"rcpt_domain = 'example.org'\"\nkey = 'rcpt_domain'\nenable = true\n\n[[queue.limiter.outbound]]\nmatch = \"rcpt_domain = 'example.net'\"\nkey = 'rcpt_domain'\nrate = '1/40m'\nenable = true\n\n[[queue.limiter.outbound]]\nmatch = \"mx = 'mx.test.org'\"\nkey = 'mx'\nenable = true\n\n[[queue.limiter.outbound]]\nmatch = \"mx = 'mx.test.net'\"\nkey = 'mx'\nrate = '1/50m'\nenable = true\n\"#;\n\n#[tokio::test]\nasync fn throttle_outbound() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Build test message\n    let mut test_message = new_message(0).message;\n    test_message.return_path = \"test@foobar.org\".into();\n    test_message\n        .recipients\n        .push(build_rcpt(\"bill@test.org\", 0, 0, 0));\n\n    let mut local = TestSMTP::new(\"smtp_throttle_outbound\", CONFIG).await;\n\n    let core = local.build_smtp();\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\"john@foobar.org\", &[\"bill@test.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    assert_eq!(\n        local.queue_receiver.last_queued_due().await as i64 - now() as i64,\n        0\n    );\n\n    // Throttle sender\n    let throttle = &core.core.smtp.queue.outbound_limiters;\n    for t in &throttle.sender {\n        core.is_allowed(\n            t,\n            &QueueEnvelope::test(&test_message, &test_message.recipients[0], \"\"),\n            0,\n        )\n        .await\n        .unwrap();\n    }\n\n    // Expect concurrency throttle for sender domain 'foobar.org'\n    /*local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    local.queue_receiver.read_event().await.assert_on_hold();*/\n\n    // Expect rate limit throttle for sender domain 'foobar.net'\n    test_message.return_path = \"test@foobar.net\".into();\n    for t in &throttle.sender {\n        core.is_allowed(\n            t,\n            &QueueEnvelope::test(&test_message, &test_message.recipients[0], \"\"),\n            0,\n        )\n        .await\n        .unwrap();\n    }\n    test_message.recipients.clear();\n\n    session\n        .send_message(\"john@foobar.net\", &[\"bill@test.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    local.queue_receiver.read_event().await.assert_refresh();\n    let due = local.queue_receiver.last_queued_due().await - now();\n    assert!(due > 0, \"Due: {}\", due);\n\n    // Expect concurrency throttle for recipient domain 'example.org'\n    test_message.return_path = \"test@test.net\".into();\n    test_message\n        .recipients\n        .push(build_rcpt(\"test@example.org\", 0, 0, 0));\n    for t in &throttle.rcpt {\n        core.is_allowed(\n            t,\n            &QueueEnvelope::test(&test_message, &test_message.recipients[0], \"\"),\n            0,\n        )\n        .await\n        .unwrap();\n    }\n\n    /*session\n        .send_message(\n            \"john@test.net\",\n            &[\"jane@example.org\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    local.queue_receiver.read_event().await.assert_on_hold();*/\n\n    // Expect rate limit throttle for recipient domain 'example.net'\n    test_message\n        .recipients\n        .push(build_rcpt(\"test@example.net\", 0, 0, 0));\n    for t in &throttle.rcpt {\n        core.is_allowed(\n            t,\n            &QueueEnvelope::test(&test_message, &test_message.recipients[1], \"\"),\n            0,\n        )\n        .await\n        .unwrap();\n    }\n\n    session\n        .send_message(\n            \"john@test.net\",\n            &[\"jane@example.net\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    local.queue_receiver.read_event().await.assert_refresh();\n    let due = local.queue_receiver.last_queued_due().await - now();\n    assert!(due > 0, \"Due: {}\", due);\n\n    // Expect concurrency throttle for mx 'mx.test.org'\n    core.mx_add(\n        \"test.org\",\n        vec![MX {\n            exchanges: vec![\"mx.test.org\".into()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.ipv4_add(\n        \"mx.test.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n    test_message\n        .recipients\n        .push(build_rcpt(\"test@test.org\", 0, 0, 0));\n\n    for t in &throttle.remote {\n        core.is_allowed(\n            t,\n            &QueueEnvelope::test(&test_message, &test_message.recipients[2], \"mx.test.org\"),\n            0,\n        )\n        .await\n        .unwrap();\n    }\n\n    /*session\n        .send_message(\"john@test.net\", &[\"jane@test.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    local.queue_receiver.read_event().await.assert_on_hold();*/\n\n    // Expect rate limit throttle for mx 'mx.test.net'\n    core.mx_add(\n        \"test.net\",\n        vec![MX {\n            exchanges: vec![\"mx.test.net\".into()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.ipv4_add(\n        \"mx.test.net\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n    for t in &throttle.remote {\n        core.is_allowed(\n            t,\n            &QueueEnvelope::test(&test_message, &test_message.recipients[1], \"mx.test.net\"),\n            0,\n        )\n        .await\n        .unwrap();\n    }\n\n    session\n        .send_message(\"john@test.net\", &[\"jane@test.net\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    local.queue_receiver.read_event().await.assert_refresh();\n    let due = local.queue_receiver.last_queued_due().await - now();\n    assert!(due > 0, \"Due: {}\", due);\n}\n\npub trait TestQueueEnvelope<'x> {\n    fn test(message: &'x Message, rcpt: &'x Recipient, mx: &'x str) -> Self;\n}\n\nimpl<'x> TestQueueEnvelope<'x> for QueueEnvelope<'x> {\n    fn test(message: &'x Message, rcpt: &'x Recipient, mx: &'x str) -> Self {\n        QueueEnvelope {\n            message,\n            mx,\n            remote_ip: IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),\n            local_ip: IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),\n            domain: rcpt.domain_part(),\n            rcpt,\n        }\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/outbound/tls.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::config::server::ServerProtocol;\nuse mail_auth::MX;\nuse store::write::now;\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::TestMessage,\n    session::{TestSession, VerifyResponse},\n};\n\nconst LOCAL: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[queue.connection.default]\nehlo-hostname = \"badtls.foobar.org\"\n\n[queue.strategy]\ntls = [ { if = \"retry_num > 0 && last_error == 'tls'\", then = \"'no-tls'\"},\n        { else = \"'default'\" }]\n\n[queue.tls.no-tls]\nstarttls = false\nallow-invalid-certs = true\n\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.extensions]\ndsn = true\nchunking = false\n\"#;\n\n#[tokio::test]\n#[serial_test::serial]\nasync fn starttls_optional() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let mut remote = TestSMTP::new(\"smtp_starttls_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n\n    // Retry on failed STARTTLS\n    let mut local = TestSMTP::new(\"smtp_starttls_local\", LOCAL).await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"mx.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(10),\n    );\n    core.ipv4_add(\n        \"mx.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(10),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    local\n        .queue_receiver\n        .expect_message_then_deliver()\n        .await\n        .try_deliver(core.clone());\n    let mut retry = local.queue_receiver.expect_message().await;\n    let prev_due = retry.message.recipients[0].retry.due;\n    let next_due = now();\n    let queue_id = retry.queue_id;\n    retry.message.recipients[0].retry.due = next_due;\n    retry.save_changes(&core, prev_due.into()).await;\n    local\n        .queue_receiver\n        .delivery_attempt(queue_id)\n        .await\n        .try_deliver(core.clone());\n    tokio::time::sleep(Duration::from_millis(100)).await;\n    remote\n        .queue_receiver\n        .expect_message()\n        .await\n        .read_lines(&remote.queue_receiver)\n        .await\n        .assert_not_contains(\"using TLSv1.3 with cipher\");\n}\n"
  },
  {
    "path": "tests/src/smtp/queue/concurrent.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::{config::server::ServerProtocol, core::BuildServer, ipc::QueueEvent};\nuse mail_auth::MX;\n\nuse crate::{\n    smtp::{DnsCache, TestSMTP, session::TestSession},\n    store::cleanup::store_assert_is_empty,\n};\nuse smtp::queue::manager::Queue;\n\nconst LOCAL: &str = r#\"\n[spam-filter]\nenable = false\n\n[session.rcpt]\nrelay = true\n\n[session.data.limits]\nmessages = 2000\n\n[queue.virtual.default]\nthreads-per-node = 4\n\n[queue.schedule.default]\nretry = \"1s\"\nnotify = \"1d\"\nexpire = \"1d\"\nqueue-name = \"default\"\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[spam-filter]\nenable = false\n\n\"#;\n\nconst NUM_MESSAGES: usize = 100;\nconst NUM_QUEUES: usize = 10;\n\n#[tokio::test(flavor = \"multi_thread\", worker_threads = 18)]\n#[serial_test::serial]\nasync fn concurrent_queue() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let remote = TestSMTP::new(\"smtp_concurrent_queue_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n\n    let local = TestSMTP::with_database(\"smtp_concurrent_queue_local\", LOCAL, \"mysql\").await;\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"mx.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(100),\n    );\n    core.ipv4_add(\n        \"mx.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(100),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n\n    // Spawn concurrent queues\n    let mut inners = vec![];\n    for _ in 0..NUM_QUEUES {\n        let (inner, rxs) = local.inner_with_rxs();\n        let server = inner.build_server();\n        server.mx_add(\n            \"foobar.org\",\n            vec![MX {\n                exchanges: vec![\"mx.foobar.org\".to_string()],\n                preference: 10,\n            }],\n            Instant::now() + Duration::from_secs(100),\n        );\n        server.ipv4_add(\n            \"mx.foobar.org\",\n            vec![\"127.0.0.1\".parse().unwrap()],\n            Instant::now() + Duration::from_secs(100),\n        );\n        inners.push(inner.clone());\n        tokio::spawn(async move {\n            Queue::new(inner, rxs.queue_rx.unwrap()).start().await;\n        });\n    }\n\n    tokio::time::sleep(Duration::from_millis(200)).await;\n\n    // Send 1000 test messages\n    for _ in 0..(NUM_MESSAGES / 2) {\n        session\n            .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n            .await;\n    }\n\n    // Wake up all queues\n    for inner in &inners {\n        inner.ipc.queue_tx.send(QueueEvent::Refresh).await.unwrap();\n    }\n    for _ in 0..(NUM_MESSAGES / 2) {\n        session\n            .send_message(\n                \"john@test.org\",\n                &[\"delay-random@foobar.org\"],\n                \"test:no_dkim\",\n                \"250\",\n            )\n            .await;\n    }\n\n    loop {\n        tokio::time::sleep(Duration::from_millis(1500)).await;\n\n        let m = local.queue_receiver.read_queued_messages().await.len();\n        let e = local.queue_receiver.read_queued_events().await.len();\n\n        if m + e != 0 {\n            println!(\"Queue still has {} messages and {} events\", m, e);\n            /*for inner in &inners {\n                inner.ipc.queue_tx.send(QueueEvent::Refresh).await.unwrap();\n            }*/\n        } else {\n            break;\n        }\n    }\n\n    local.queue_receiver.assert_queue_is_empty().await;\n    let remote_messages = remote.queue_receiver.read_queued_messages().await;\n    assert_eq!(remote_messages.len(), NUM_MESSAGES);\n\n    // Make sure local store is queue\n    store_assert_is_empty(\n        &core.core.storage.data,\n        core.core.storage.blob.clone(),\n        false,\n    )\n    .await;\n}\n"
  },
  {
    "path": "tests/src/smtp/queue/dsn.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::smtp::{QueueReceiver, TestSMTP, inbound::sign::SIGNATURES};\nuse common::config::smtp::queue::{QueueExpiry, QueueName};\nuse smtp::queue::{\n    Error, ErrorDetails, HostResponse, Message, MessageWrapper, Recipient, Schedule, Status,\n    UnexpectedResponse, dsn::SendDsn,\n};\nuse smtp_proto::{RCPT_NOTIFY_DELAY, RCPT_NOTIFY_FAILURE, RCPT_NOTIFY_SUCCESS, Response};\nuse std::{\n    fs,\n    net::{IpAddr, Ipv4Addr},\n    path::PathBuf,\n    time::SystemTime,\n};\nuse store::write::now;\nuse types::blob_hash::BlobHash;\n\nconst CONFIG: &str = r#\"\n[report]\nsubmitter = \"'mx.example.org'\"\n\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[report.dsn]\nfrom-name = \"'Mail Delivery Subsystem'\"\nfrom-address = \"'MAILER-DAEMON@example.org'\"\nsign = \"['rsa']\"\n\n\"#;\n\n#[tokio::test]\nasync fn generate_dsn() {\n    // Enable logging\n    crate::enable_logging();\n\n    let mut path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    path.push(\"resources\");\n    path.push(\"smtp\");\n    path.push(\"dsn\");\n    path.push(\"original.txt\");\n    let size = fs::metadata(&path).unwrap().len() as u64;\n    let dsn_original = fs::read_to_string(&path).unwrap();\n\n    let flags = RCPT_NOTIFY_FAILURE | RCPT_NOTIFY_DELAY | RCPT_NOTIFY_SUCCESS;\n    let mut message = MessageWrapper {\n        queue_id: 0,\n        span_id: 0,\n        is_multi_queue: false,\n        queue_name: QueueName::default(),\n        message: Message {\n            size,\n            created: SystemTime::now()\n                .duration_since(SystemTime::UNIX_EPOCH)\n                .map_or(0, |d| d.as_secs()),\n            return_path: \"sender@foobar.org\".into(),\n            recipients: vec![Recipient {\n                address: \"foobar@example.org\".into(),\n                status: Status::PermanentFailure(ErrorDetails {\n                    entity: \"mx.example.org\".into(),\n                    details: Error::UnexpectedResponse(UnexpectedResponse {\n                        command: \"RCPT TO:<foobar@example.org>\".into(),\n                        response: Response {\n                            code: 550,\n                            esc: [5, 1, 2],\n                            message: \"User does not exist\".into(),\n                        },\n                    }),\n                }),\n                flags: 0,\n                orcpt: None,\n                retry: Schedule::now(),\n                notify: Schedule::now(),\n                expires: QueueExpiry::Ttl(10),\n                queue: QueueName::default(),\n            }],\n            flags: 0,\n            env_id: None,\n            priority: 0,\n            blob_hash: BlobHash::generate(dsn_original.as_bytes()),\n            quota_keys: Default::default(),\n            received_from_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),\n            received_via_port: 0,\n        },\n    };\n\n    // Load config\n    let mut local = TestSMTP::new(\"smtp_dsn_test\", CONFIG.to_string() + SIGNATURES).await;\n    let core = local.build_smtp();\n    let qr = &mut local.queue_receiver;\n\n    // Create temp dir for queue\n    qr.blob_store\n        .put_blob(\n            message.message.blob_hash.as_slice(),\n            dsn_original.as_bytes(),\n        )\n        .await\n        .unwrap();\n\n    // Disabled DSN\n    core.send_dsn(&mut message).await;\n    qr.assert_no_events();\n    qr.assert_queue_is_empty().await;\n\n    // Failure DSN\n    message.message.recipients[0].flags = flags;\n    core.send_dsn(&mut message).await;\n    let dsn_message = qr.expect_message().await;\n    qr.compare_dsn(dsn_message.message, \"failure.eml\").await;\n\n    // Success DSN\n    message.message.recipients.push(Recipient {\n        address: \"jane@example.org\".into(),\n        status: Status::Completed(HostResponse {\n            hostname: \"mx2.example.org\".into(),\n            response: Response {\n                code: 250,\n                esc: [2, 1, 5],\n                message: \"Message accepted for delivery\".into(),\n            },\n        }),\n        flags,\n        orcpt: None,\n        retry: Schedule::now(),\n        notify: Schedule::now(),\n        expires: QueueExpiry::Ttl(10),\n        queue: QueueName::default(),\n    });\n    core.send_dsn(&mut message).await;\n    let dsn_message = qr.expect_message().await;\n    qr.compare_dsn(dsn_message.message, \"success.eml\").await;\n\n    // Delay DSN\n    message.message.recipients.push(Recipient {\n        address: \"john.doe@example.org\".into(),\n        status: Status::TemporaryFailure(ErrorDetails {\n            entity: \"mx.domain.org\".into(),\n            details: Error::ConnectionError(\"Connection timeout\".into()),\n        }),\n        flags,\n        orcpt: Some(\"jdoe@example.org\".into()),\n        retry: Schedule::now(),\n        notify: Schedule::now(),\n        expires: QueueExpiry::Ttl(10),\n        queue: QueueName::default(),\n    });\n    core.send_dsn(&mut message).await;\n    let dsn_message = qr.expect_message().await;\n    qr.compare_dsn(dsn_message.message, \"delay.eml\").await;\n\n    // Mixed DSN\n    for rcpt in &mut message.message.recipients {\n        rcpt.flags = flags;\n    }\n    message.message.recipients.last_mut().unwrap().notify.due = now();\n    core.send_dsn(&mut message).await;\n    let dsn_message = qr.expect_message().await;\n    qr.compare_dsn(dsn_message.message, \"mixed.eml\").await;\n\n    // Load queue\n    let queue = qr.read_queued_messages().await;\n    assert_eq!(queue.len(), 4);\n}\n\nimpl QueueReceiver {\n    async fn compare_dsn(&self, message: Message, test: &str) {\n        let mut path = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n        path.push(\"resources\");\n        path.push(\"smtp\");\n        path.push(\"dsn\");\n        path.push(test);\n\n        let bytes = self\n            .blob_store\n            .get_blob(message.blob_hash.as_slice(), 0..usize::MAX)\n            .await\n            .unwrap()\n            .unwrap();\n\n        let dsn = remove_ids(bytes);\n        let dsn_expected = fs::read_to_string(&path).unwrap();\n\n        if dsn != dsn_expected {\n            let mut failed = PathBuf::from(&path);\n            failed.set_extension(\"failed\");\n            fs::write(&failed, dsn.as_bytes()).unwrap();\n            panic!(\n                \"Failed for {}, output saved to {}\",\n                path.display(),\n                failed.display()\n            );\n        }\n    }\n}\n\nfn remove_ids(message: Vec<u8>) -> String {\n    let old_message = String::from_utf8(message).unwrap();\n    let mut message = String::with_capacity(old_message.len());\n    let mut found_dkim = false;\n    let mut skip = false;\n\n    let mut boundary = \"\";\n    for line in old_message.split(\"\\r\\n\") {\n        if skip {\n            if line.chars().next().unwrap().is_ascii_whitespace() {\n                continue;\n            } else {\n                skip = false;\n            }\n        }\n        if line.starts_with(\"Date:\") || line.starts_with(\"Message-ID:\") {\n            continue;\n        } else if !found_dkim && line.starts_with(\"DKIM-Signature:\") {\n            found_dkim = true;\n            skip = true;\n            continue;\n        } else if line.starts_with(\"--\") {\n            message.push_str(&line.replace(boundary, \"mime_boundary\"));\n        } else if let Some((_, boundary_)) = line.split_once(\"boundary=\\\"\") {\n            boundary = boundary_.split_once('\"').unwrap().0;\n            message.push_str(&line.replace(boundary, \"mime_boundary\"));\n        } else if line.starts_with(\"Arrival-Date:\") {\n            message.push_str(\"Arrival-Date: <date goes here>\");\n        } else if line.starts_with(\"Will-Retry-Until:\") {\n            message.push_str(\"Will-Retry-Until: <date goes here>\");\n        } else {\n            message.push_str(line);\n        }\n        message.push_str(\"\\r\\n\");\n    }\n\n    if !found_dkim {\n        panic!(\"No DKIM signature found in: {old_message}\");\n    }\n\n    message\n}\n"
  },
  {
    "path": "tests/src/smtp/queue/manager.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::smtp::{\n    TestSMTP,\n    queue::{QueuedEvents, build_rcpt},\n};\nuse common::config::smtp::queue::QueueName;\nuse smtp::queue::{\n    Error, ErrorDetails, Message, MessageWrapper, Recipient, Status, spool::SmtpSpool,\n};\nuse std::{\n    net::{IpAddr, Ipv4Addr},\n    time::Duration,\n};\nuse store::write::now;\n\nconst CONFIG: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\"#;\n\n#[tokio::test]\nasync fn queue_due() {\n    // Enable logging\n    crate::enable_logging();\n\n    let local = TestSMTP::new(\"smtp_queue_due_test\", CONFIG).await;\n    let core = local.build_smtp();\n    let qr = &local.queue_receiver;\n\n    let mut message = new_message(0);\n    message.message.recipients.push(build_rcpt(\"c\", 3, 8, 9));\n    message.save_changes(&core, 0.into()).await;\n\n    let mut message = new_message(1);\n    message.message.recipients.push(build_rcpt(\"b\", 2, 6, 7));\n    message.save_changes(&core, 0.into()).await;\n\n    let mut message = new_message(2);\n    message.message.recipients.push(build_rcpt(\"a\", 1, 4, 5));\n    message.save_changes(&core, 0.into()).await;\n\n    for domain in vec![\"a\", \"b\", \"c\"].into_iter() {\n        let now = now();\n        let queued = core.all_queued_messages().await;\n        if queued.messages.is_empty() {\n            let wake_up = queued.next_refresh - now;\n            assert_eq!(wake_up, 1);\n            std::thread::sleep(Duration::from_secs(wake_up));\n        }\n\n        for queue_event in core.all_queued_messages().await.messages {\n            if let Some(message) = core\n                .read_message(queue_event.queue_id, QueueName::default())\n                .await\n            {\n                message.message.rcpt(domain);\n                message.remove(&core, queue_event.due.into()).await;\n            } else {\n                panic!(\"Message not found\");\n            }\n        }\n    }\n\n    qr.assert_queue_is_empty().await;\n}\n\n#[test]\nfn delivery_events() {\n    let mut message = new_message(0).message;\n    message.created = now();\n\n    message.recipients.push(build_rcpt(\"a\", 1, 2, 3));\n    message.recipients.push(build_rcpt(\"b\", 4, 5, 6));\n    message.recipients.push(build_rcpt(\"c\", 7, 8, 9));\n\n    for t in 0..2 {\n        assert_eq!(\n            message.next_event(None).unwrap(),\n            message.rcpt(\"a\").retry.due\n        );\n        assert_eq!(\n            message.next_delivery_event(None).unwrap(),\n            message.rcpt(\"a\").retry.due\n        );\n        assert_eq!(\n            next_event_after(\n                &message,\n                None,\n                message.rcpt(\"a\").expiration_time(message.created).unwrap()\n            )\n            .unwrap(),\n            message.rcpt(\"b\").retry.due\n        );\n        assert_eq!(\n            next_event_after(\n                &message,\n                None,\n                message.rcpt(\"b\").expiration_time(message.created).unwrap()\n            )\n            .unwrap(),\n            message.rcpt(\"c\").retry.due\n        );\n        assert_eq!(\n            next_event_after(&message, None, message.rcpt(\"c\").notify.due).unwrap(),\n            message.rcpt(\"c\").expiration_time(message.created).unwrap()\n        );\n        assert!(\n            next_event_after(\n                &message,\n                None,\n                message.rcpt(\"c\").expiration_time(message.created).unwrap()\n            )\n            .is_none()\n        );\n\n        if t == 0 {\n            message.recipients.reverse();\n        } else {\n            message.recipients.swap(0, 1);\n        }\n    }\n\n    message.rcpt_mut(\"a\").status = Status::PermanentFailure(ErrorDetails {\n        entity: \"localhost\".into(),\n        details: Error::ConcurrencyLimited,\n    });\n    assert_eq!(\n        message.next_event(None).unwrap(),\n        message.rcpt(\"b\").retry.due\n    );\n    assert_eq!(\n        message.next_delivery_event(None).unwrap(),\n        message.rcpt(\"b\").retry.due\n    );\n\n    message.rcpt_mut(\"b\").status = Status::PermanentFailure(ErrorDetails {\n        entity: \"localhost\".into(),\n        details: Error::ConcurrencyLimited,\n    });\n    assert_eq!(\n        message.next_event(None).unwrap(),\n        message.rcpt(\"c\").retry.due\n    );\n    assert_eq!(\n        message.next_delivery_event(None).unwrap(),\n        message.rcpt(\"c\").retry.due\n    );\n\n    message.rcpt_mut(\"c\").status = Status::PermanentFailure(ErrorDetails {\n        entity: \"localhost\".into(),\n        details: Error::ConcurrencyLimited,\n    });\n    assert!(message.next_event(None).is_none());\n}\n\npub fn new_message(queue_id: u64) -> MessageWrapper {\n    MessageWrapper {\n        queue_id,\n        span_id: 0,\n        queue_name: QueueName::default(),\n        is_multi_queue: false,\n        message: Message {\n            size: 0,\n            created: now(),\n            return_path: \"sender@foobar.org\".into(),\n            recipients: vec![],\n            flags: 0,\n            env_id: None,\n            priority: 0,\n            quota_keys: Default::default(),\n            blob_hash: Default::default(),\n            received_from_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),\n            received_via_port: 0,\n        },\n    }\n}\n\nfn next_event_after(message: &Message, queue: Option<QueueName>, instant: u64) -> Option<u64> {\n    let mut next_event = None;\n\n    for rcpt in &message.recipients {\n        if matches!(rcpt.status, Status::Scheduled | Status::TemporaryFailure(_))\n            && queue.is_none_or(|q| rcpt.queue == q)\n        {\n            if rcpt.retry.due > instant\n                && next_event.as_ref().is_none_or(|ne| rcpt.retry.due.lt(ne))\n            {\n                next_event = rcpt.retry.due.into();\n            }\n            if rcpt.notify.due > instant\n                && next_event.as_ref().is_none_or(|ne| rcpt.notify.due.lt(ne))\n            {\n                next_event = rcpt.notify.due.into();\n            }\n            if let Some(expires) = rcpt.expiration_time(message.created)\n                && expires > instant\n                && next_event.as_ref().is_none_or(|ne| expires.lt(ne))\n            {\n                next_event = expires.into();\n            }\n        }\n    }\n\n    next_event\n}\n\npub trait TestMessage {\n    fn rcpt(&self, name: &str) -> &Recipient;\n    fn rcpt_mut(&mut self, name: &str) -> &mut Recipient;\n}\n\nimpl TestMessage for Message {\n    fn rcpt(&self, name: &str) -> &Recipient {\n        self.recipients\n            .iter()\n            .find(|d| d.address() == name)\n            .unwrap_or_else(|| panic!(\"Expected rcpt {name} not found in {:?}\", self.recipients))\n    }\n\n    fn rcpt_mut(&mut self, name: &str) -> &mut Recipient {\n        self.recipients\n            .iter_mut()\n            .find(|d| d.address() == name)\n            .unwrap()\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/queue/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse common::{\n    Server,\n    config::smtp::queue::{QueueExpiry, QueueName},\n};\nuse smtp::queue::{\n    Recipient, Schedule, Status,\n    manager::Queue,\n    spool::{QueuedMessages, SmtpSpool},\n};\nuse tokio::sync::mpsc;\n\npub mod concurrent;\npub mod dsn;\npub mod manager;\npub mod retry;\npub mod virtualq;\n\npub fn build_rcpt(address: &str, retry: u64, notify: u64, expires: u64) -> Recipient {\n    Recipient {\n        address: address.into(),\n        retry: Schedule::later(retry),\n        notify: Schedule::later(notify),\n        expires: QueueExpiry::Ttl(expires),\n        status: Status::Scheduled,\n        flags: 0,\n        orcpt: None,\n        queue: QueueName::default(),\n    }\n}\n\npub trait QueuedEvents: Sync + Send {\n    fn all_queued_messages(&self) -> impl Future<Output = QueuedMessages> + Send;\n}\n\nimpl QueuedEvents for Server {\n    async fn all_queued_messages(&self) -> QueuedMessages {\n        self.next_event(&mut Queue::new(self.inner.clone(), mpsc::channel(100).1))\n            .await\n    }\n}\n"
  },
  {
    "path": "tests/src/smtp/queue/retry.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse crate::smtp::{\n    TestSMTP,\n    inbound::{TestMessage, TestQueueEvent},\n    queue::QueuedEvents,\n    session::{TestSession, VerifyResponse},\n};\nuse ahash::AHashSet;\nuse common::{\n    config::smtp::queue::QueueName,\n    ipc::{QueueEvent, QueueEventStatus},\n};\nuse smtp::queue::spool::{QUEUE_REFRESH, SmtpSpool};\nuse store::write::now;\n\nconst CONFIG: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[session.extensions]\ndeliver-by = \"1h\"\nfuture-release = \"1h\"\n\n[queue.schedule.sender-default]\nretry = [\"1s\", \"2s\", \"3s\"]\nnotify = [\"15h\", \"22h\"]\nexpire = \"1d\"\nqueue-name = \"default\"\n\n[queue.schedule.sender-test]\nretry = [\"1s\", \"2s\", \"3s\"]\nnotify = [\"1s\", \"2s\"]\nexpire = \"6s\"\n#max-attempts = 3\nqueue-name = \"default\"\n\n[queue.strategy]\nschedule = [{if = \"sender_domain == 'test.org'\", then = \"'sender-test'\"},\n           {else = \"'sender-default'\"}]\n\"#;\n\n#[tokio::test]\nasync fn queue_retry() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Create temp dir for queue\n    let mut local = TestSMTP::new(\"smtp_queue_retry_test\", CONFIG).await;\n\n    // Create test message\n    let core = local.build_smtp();\n    let mut session = local.new_session();\n    let qr = &mut local.queue_receiver;\n\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    let attempt = qr.expect_message_then_deliver().await;\n\n    // Expect a failed DSN\n    attempt.try_deliver(core.clone());\n    let message = qr.expect_message().await;\n    assert_eq!(message.message.return_path.as_ref(), \"\");\n    assert_eq!(\n        message.message.recipients.first().unwrap().address(),\n        \"john@test.org\"\n    );\n    message\n        .read_lines(qr)\n        .await\n        .assert_contains(\"Content-Type: multipart/report\")\n        .assert_contains(\"Final-Recipient: rfc822;bill@foobar.org\")\n        .assert_contains(\"Action: failed\");\n    qr.read_event().await.assert_done();\n    qr.clear_queue(&core).await;\n\n    // Expect a failed DSN for foobar.org, followed by two delayed DSN and\n    // a final failed DSN for _dns_error.org.\n    session\n        .send_message(\n            \"john@test.org\",\n            &[\"bill@foobar.org\", \"jane@_dns_error.org\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    let mut in_fight = AHashSet::new();\n    let attempt = qr.expect_message_then_deliver().await;\n    let mut dsn = Vec::new();\n    let mut retries = Vec::new();\n    in_fight.insert(attempt.queue_id);\n    attempt.try_deliver(core.clone());\n\n    loop {\n        match qr.try_read_event().await {\n            Some(QueueEvent::WorkerDone {\n                queue_id, status, ..\n            }) => {\n                in_fight.remove(&queue_id);\n                match &status {\n                    QueueEventStatus::Completed | QueueEventStatus::Deferred => (),\n                    _ => panic!(\"unexpected status {queue_id}: {status:?}\"),\n                }\n            }\n            Some(QueueEvent::Refresh) | Some(QueueEvent::ReloadSettings) => (),\n            None | Some(QueueEvent::Stop) | Some(QueueEvent::Paused(_)) => break,\n        }\n\n        let now = now();\n        let mut events = core.all_queued_messages().await;\n        if events.messages.is_empty() {\n            if events.next_refresh < now + QUEUE_REFRESH {\n                tokio::time::sleep(Duration::from_secs(events.next_refresh - now)).await;\n                events = core.all_queued_messages().await;\n            } else if in_fight.is_empty() {\n                break;\n            }\n        }\n\n        for event in events.messages {\n            if in_fight.contains(&event.queue_id) {\n                continue;\n            }\n\n            let message = core\n                .read_message(event.queue_id, QueueName::default())\n                .await\n                .unwrap();\n            if message.message.return_path.is_empty() {\n                message.clone().remove(&core, event.due.into()).await;\n                dsn.push(message);\n            } else {\n                retries.push(event.due.saturating_sub(now));\n                in_fight.insert(event.queue_id);\n                event.try_deliver(core.clone());\n                tokio::time::sleep(Duration::from_millis(100)).await;\n            }\n        }\n    }\n    qr.assert_queue_is_empty().await;\n    assert_eq!(retries, vec![1, 2, 3]);\n    assert_eq!(dsn.len(), 4);\n    let mut dsn = dsn.into_iter();\n\n    dsn.next()\n        .unwrap()\n        .read_lines(qr)\n        .await\n        .assert_contains(\"<bill@foobar.org> (failed to lookup 'foobar.org'\")\n        .assert_contains(\"Final-Recipient: rfc822;bill@foobar.org\")\n        .assert_contains(\"Action: failed\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(qr)\n        .await\n        .assert_contains(\"<jane@_dns_error.org> (failed to lookup '_dns_error.org'\")\n        .assert_contains(\"Final-Recipient: rfc822;jane@_dns_error.org\")\n        .assert_contains(\"Action: delayed\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(qr)\n        .await\n        .assert_contains(\"<jane@_dns_error.org> (failed to lookup '_dns_error.org'\")\n        .assert_contains(\"Final-Recipient: rfc822;jane@_dns_error.org\")\n        .assert_contains(\"Action: delayed\");\n\n    dsn.next()\n        .unwrap()\n        .read_lines(qr)\n        .await\n        .assert_contains(\"<jane@_dns_error.org> (failed to lookup '_dns_error.org'\")\n        .assert_contains(\"Final-Recipient: rfc822;jane@_dns_error.org\")\n        .assert_contains(\"Action: failed\");\n\n    // Test FUTURERELEASE + DELIVERBY (RETURN)\n    session.data.remote_ip_str = \"10.0.0.2\".into();\n    session.eval_session_params().await;\n    session\n        .send_message(\n            \"<bill@foobar.org> HOLDFOR=60 BY=3600;R\",\n            &[\"john@test.net\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    let now_ = now();\n    let message = qr.expect_message().await;\n    assert!([59, 60].contains(&(qr.message_due(message.queue_id).await - now_)));\n    assert!([59, 60].contains(&(message.message.next_delivery_event(None).unwrap() - now_)));\n    assert!(\n        [3599, 3600].contains(\n            &(message\n                .message\n                .recipients\n                .first()\n                .unwrap()\n                .expiration_time(message.message.created)\n                .unwrap()\n                - now_)\n        )\n    );\n    assert!(\n        [54059, 54060].contains(&(message.message.recipients.first().unwrap().notify.due - now_))\n    );\n\n    // Test DELIVERBY (NOTIFY)\n    session\n        .send_message(\n            \"<bill@foobar.org> BY=3600;N\",\n            &[\"john@test.net\"],\n            \"test:no_dkim\",\n            \"250\",\n        )\n        .await;\n    let schedule = qr.expect_message().await;\n    assert!(\n        [3599, 3600].contains(&(schedule.message.recipients.first().unwrap().notify.due - now()))\n    );\n}\n"
  },
  {
    "path": "tests/src/smtp/queue/virtualq.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::{Duration, Instant};\n\nuse common::{\n    config::{server::ServerProtocol, smtp::queue::QueueName},\n    core::BuildServer,\n    ipc::QueueEvent,\n};\nuse mail_auth::MX;\n\nuse crate::{\n    smtp::{DnsCache, TestSMTP, session::TestSession},\n    store::cleanup::store_assert_is_empty,\n};\nuse smtp::queue::manager::Queue;\n\nconst LOCAL: &str = r#\"\n[spam-filter]\nenable = false\n\n[session.rcpt]\nrelay = true\n\n[session.data.limits]\nmessages = 2000\n\n[queue.strategy]\nschedule = [ { if = \"rcpt == 'delay-random@foobar.org'\", then = \"'q2'\" },\n             { else = \"'q1'\"} ]\n\n[queue.virtual.q1]\nthreads-per-node = 5\n\n[queue.virtual.q2]\nthreads-per-node = 4\n\n[queue.schedule.q1]\nretry = \"1s\"\nnotify = \"1d\"\nexpire = \"1d\"\nqueue-name = \"q1\"\n\n[queue.schedule.q2]\nretry = \"1s\"\nnotify = \"1d\"\nexpire = \"1d\"\nqueue-name = \"q2\"\n\n\"#;\n\nconst REMOTE: &str = r#\"\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = true\n\n[spam-filter]\nenable = false\n\n\"#;\n\nconst NUM_MESSAGES: usize = 100;\nconst NUM_QUEUES: usize = 10;\n\n#[tokio::test(flavor = \"multi_thread\", worker_threads = 18)]\n#[serial_test::serial]\nasync fn virtual_queue() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Start test server\n    let remote = TestSMTP::new(\"smtp_virtual_queue_remote\", REMOTE).await;\n    let _rx = remote.start(&[ServerProtocol::Smtp]).await;\n    let local = TestSMTP::with_database(\"smtp_virtual_queue_local\", LOCAL, \"mysql\").await;\n\n    // Validate parsing\n    for value in [\"a\", \"ab\", \"abcdefgh\"] {\n        let queue_name = QueueName::new(value).unwrap();\n        assert_eq!(queue_name.to_string(), value);\n    }\n    assert_eq!(\n        local\n            .server\n            .core\n            .smtp\n            .queue\n            .virtual_queues\n            .get(&QueueName::new(\"q1\").unwrap())\n            .unwrap()\n            .threads,\n        5\n    );\n    assert_eq!(\n        local\n            .server\n            .core\n            .smtp\n            .queue\n            .virtual_queues\n            .get(&QueueName::new(\"q2\").unwrap())\n            .unwrap()\n            .threads,\n        4\n    );\n\n    // Add mock DNS entries\n    let core = local.build_smtp();\n    core.mx_add(\n        \"foobar.org\",\n        vec![MX {\n            exchanges: vec![\"mx.foobar.org\".to_string()],\n            preference: 10,\n        }],\n        Instant::now() + Duration::from_secs(100),\n    );\n    core.ipv4_add(\n        \"mx.foobar.org\",\n        vec![\"127.0.0.1\".parse().unwrap()],\n        Instant::now() + Duration::from_secs(100),\n    );\n\n    let mut session = local.new_session();\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n\n    // Spawn concurrent queues\n    let mut inners = vec![];\n    for _ in 0..NUM_QUEUES {\n        let (inner, rxs) = local.inner_with_rxs();\n        let server = inner.build_server();\n        server.mx_add(\n            \"foobar.org\",\n            vec![MX {\n                exchanges: vec![\"mx.foobar.org\".to_string()],\n                preference: 10,\n            }],\n            Instant::now() + Duration::from_secs(100),\n        );\n        server.ipv4_add(\n            \"mx.foobar.org\",\n            vec![\"127.0.0.1\".parse().unwrap()],\n            Instant::now() + Duration::from_secs(100),\n        );\n        inners.push(inner.clone());\n        tokio::spawn(async move {\n            Queue::new(inner, rxs.queue_rx.unwrap()).start().await;\n        });\n    }\n\n    tokio::time::sleep(Duration::from_millis(200)).await;\n\n    // Send 1000 test messages\n    for _ in 0..(NUM_MESSAGES / 2) {\n        session\n            .send_message(\n                \"john@test.org\",\n                &[\"bill@foobar.org\", \"delay-random@foobar.org\"],\n                \"test:no_dkim\",\n                \"250\",\n            )\n            .await;\n    }\n\n    // Wake up all queues\n    for inner in &inners {\n        inner.ipc.queue_tx.send(QueueEvent::Refresh).await.unwrap();\n    }\n    for _ in 0..(NUM_MESSAGES / 2) {\n        session\n            .send_message(\n                \"john@test.org\",\n                &[\"bill@foobar.org\", \"delay-random@foobar.org\"],\n                \"test:no_dkim\",\n                \"250\",\n            )\n            .await;\n    }\n\n    loop {\n        tokio::time::sleep(Duration::from_millis(1500)).await;\n\n        let m = local.queue_receiver.read_queued_messages().await;\n        let e = local.queue_receiver.read_queued_events().await;\n\n        if m.len() + e.len() != 0 {\n            println!(\n                \"Queue still has {} messages and {} events\",\n                m.len(),\n                e.len()\n            );\n            /*for inner in &inners {\n                inner.ipc.queue_tx.send(QueueEvent::Refresh).await.unwrap();\n            }*/\n        } else {\n            break;\n        }\n    }\n\n    local.queue_receiver.assert_queue_is_empty().await;\n    let remote_messages = remote.queue_receiver.read_queued_messages().await;\n    assert_eq!(remote_messages.len(), NUM_MESSAGES * 2);\n\n    // Make sure local store is queue\n    store_assert_is_empty(\n        &core.core.storage.data,\n        core.core.storage.blob.clone(),\n        false,\n    )\n    .await;\n}\n"
  },
  {
    "path": "tests/src/smtp/reporting/analyze.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse crate::smtp::{TestSMTP, inbound::TestQueueEvent, session::TestSession};\n\nuse store::{\n    IterateParams, ValueKey,\n    write::{ReportClass, ValueClass},\n};\n\nconst CONFIG: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[session.data.limits]\nmessages = 100\n\n[report.analysis]\naddresses = [\"reports@*\", \"*@dmarc.foobar.org\", \"feedback@foobar.org\"]\nforward = false\nstore = \"1s\"\n\"#;\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn report_analyze() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Create temp dir for queue\n    let mut local = TestSMTP::new(\"smtp_analyze_report_test\", CONFIG).await;\n\n    // Create test message\n    let mut session = local.new_session();\n    let qr = &mut local.queue_receiver;\n    session.data.remote_ip_str = \"10.0.0.1\".into();\n    session.eval_session_params().await;\n    session.ehlo(\"mx.test.org\").await;\n\n    let addresses = [\n        \"reports@foobar.org\",\n        \"rep@dmarc.foobar.org\",\n        \"feedback@foobar.org\",\n    ];\n    let mut ac = 0;\n    let mut total_reports_received = 0;\n    for (test, num_tests) in [(\"arf\", 5), (\"dmarc\", 5), (\"tls\", 2)] {\n        for num_test in 1..=num_tests {\n            total_reports_received += 1;\n            session\n                .send_message(\n                    \"john@test.org\",\n                    &[addresses[ac % addresses.len()]],\n                    &format!(\"report:{test}{num_test}\"),\n                    \"250\",\n                )\n                .await;\n            qr.assert_no_events();\n            ac += 1;\n        }\n    }\n    tokio::time::sleep(Duration::from_millis(200)).await;\n\n    //let c = tokio::time::sleep(Duration::from_secs(86400)).await;\n\n    // Purging the database shouldn't remove the reports\n    qr.store.purge_store().await.unwrap();\n\n    // Make sure the reports are in the store\n    let mut total_reports = 0;\n    qr.store\n        .iterate(\n            IterateParams::new(\n                ValueKey::from(ValueClass::Report(ReportClass::Tls { id: 0, expires: 0 })),\n                ValueKey::from(ValueClass::Report(ReportClass::Arf {\n                    id: u64::MAX,\n                    expires: u64::MAX,\n                })),\n            ),\n            |_, _| {\n                total_reports += 1;\n                Ok(true)\n            },\n        )\n        .await\n        .unwrap();\n    assert_eq!(total_reports, total_reports_received);\n\n    // Wait one second, purge, and make sure they are gone\n    tokio::time::sleep(Duration::from_secs(1)).await;\n    qr.store.purge_store().await.unwrap();\n    let mut total_reports = 0;\n    qr.store\n        .iterate(\n            IterateParams::new(\n                ValueKey::from(ValueClass::Report(ReportClass::Tls { id: 0, expires: 0 })),\n                ValueKey::from(ValueClass::Report(ReportClass::Arf {\n                    id: u64::MAX,\n                    expires: u64::MAX,\n                })),\n            ),\n            |_, _| {\n                total_reports += 1;\n                Ok(true)\n            },\n        )\n        .await\n        .unwrap();\n    assert_eq!(total_reports, 0);\n\n    // Test delivery to non-report addresses\n    session\n        .send_message(\"john@test.org\", &[\"bill@foobar.org\"], \"test:no_dkim\", \"250\")\n        .await;\n    qr.read_event().await.assert_refresh();\n    qr.last_queued_message().await;\n}\n"
  },
  {
    "path": "tests/src/smtp/reporting/dmarc.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{\n    net::IpAddr,\n    sync::Arc,\n    time::{Duration, Instant},\n};\n\nuse common::{config::smtp::report::AggregateFrequency, ipc::DmarcEvent};\nuse mail_auth::{\n    common::parse::TxtRecordParser,\n    dmarc::Dmarc,\n    report::{ActionDisposition, Disposition, DmarcResult, Record, Report},\n};\nuse smtp::reporting::dmarc::DmarcReporting;\nuse store::write::QueueClass;\n\nuse crate::smtp::{\n    DnsCache, TestSMTP,\n    inbound::{TestMessage, sign::SIGNATURES},\n    session::VerifyResponse,\n};\n\nconst CONFIG: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[server]\nhostname = \"mx.example.org\"\n\n[report]\nsubmitter = \"'mx.example.org'\"\n\n[report.dmarc.aggregate]\nfrom-name = \"'DMARC Report'\"\nfrom-address = \"'reports@' + config_get('report.domain')\"\norg-name = \"'Foobar, Inc.'\"\ncontact-info = \"'https://foobar.org/contact'\"\nsend = \"daily\"\nmax-size = 4096\nsign = \"['rsa']\"\n\n\"#;\n\n#[tokio::test]\nasync fn report_dmarc() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Create scheduler\n    let mut local = TestSMTP::new(\"smtp_report_dmarc_test\", CONFIG.to_string() + SIGNATURES).await;\n\n    // Authorize external report for foobar.org\n    let core = local.build_smtp();\n    core.txt_add(\n        \"foobar.org._report._dmarc.foobar.net\",\n        Dmarc::parse(b\"v=DMARC1;\").unwrap(),\n        Instant::now() + Duration::from_secs(10),\n    );\n    let qr = &mut local.queue_receiver;\n\n    // Schedule two events with a same policy and another one with a different policy\n    let dmarc_record = Arc::new(\n        Dmarc::parse(\n            b\"v=DMARC1; p=quarantine; rua=mailto:reports@foobar.net,mailto:reports@example.net\",\n        )\n        .unwrap(),\n    );\n    assert_eq!(dmarc_record.rua().len(), 2);\n    for _ in 0..2 {\n        core.schedule_dmarc(Box::new(DmarcEvent {\n            domain: \"foobar.org\".to_string(),\n            report_record: Record::new()\n                .with_source_ip(\"192.168.1.2\".parse().unwrap())\n                .with_action_disposition(ActionDisposition::Pass)\n                .with_dmarc_dkim_result(DmarcResult::Pass)\n                .with_dmarc_spf_result(DmarcResult::Fail)\n                .with_envelope_from(\"hello@example.org\")\n                .with_envelope_to(\"other@example.org\")\n                .with_header_from(\"bye@example.org\"),\n            dmarc_record: dmarc_record.clone(),\n            interval: AggregateFrequency::Weekly,\n        }))\n        .await;\n    }\n    core.schedule_dmarc(Box::new(DmarcEvent {\n        domain: \"foobar.org\".to_string(),\n        report_record: Record::new()\n            .with_source_ip(\"a:b:c::e:f\".parse().unwrap())\n            .with_action_disposition(ActionDisposition::Reject)\n            .with_dmarc_dkim_result(DmarcResult::Fail)\n            .with_dmarc_spf_result(DmarcResult::Pass),\n        dmarc_record: dmarc_record.clone(),\n        interval: AggregateFrequency::Weekly,\n    }))\n    .await;\n    tokio::time::sleep(Duration::from_millis(200)).await;\n    let reports = qr.read_report_events().await;\n    assert_eq!(reports.len(), 1);\n    match reports.into_iter().next().unwrap() {\n        QueueClass::DmarcReportHeader(event) => {\n            core.send_dmarc_aggregate_report(event).await;\n        }\n        _ => unreachable!(),\n    }\n\n    // Expect report\n    let message = qr.expect_message().await;\n    qr.assert_no_events();\n    assert_eq!(message.message.recipients.len(), 1);\n    assert_eq!(\n        message.message.recipients.last().unwrap().address(),\n        \"reports@foobar.net\"\n    );\n    assert_eq!(message.message.return_path.as_ref(), \"reports@example.org\");\n    message\n        .read_lines(qr)\n        .await\n        .assert_contains(\"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com;\")\n        .assert_contains(\"To: <reports@foobar.net>\")\n        .assert_contains(\"Report Domain: foobar.org\")\n        .assert_contains(\"Submitter: mx.example.org\");\n\n    // Verify generated report\n    let report = Report::parse_rfc5322(message.read_message(qr).await.as_bytes()).unwrap();\n    assert_eq!(report.domain(), \"foobar.org\");\n    assert_eq!(report.email(), \"reports@example.org\");\n    assert_eq!(report.org_name(), \"Foobar, Inc.\");\n    assert_eq!(\n        report.extra_contact_info().unwrap(),\n        \"https://foobar.org/contact\"\n    );\n    assert_eq!(report.p(), Disposition::Quarantine);\n    assert_eq!(report.records().len(), 2);\n    for record in report.records() {\n        let source_ip = record.source_ip().unwrap();\n        if source_ip == \"192.168.1.2\".parse::<IpAddr>().unwrap() {\n            assert_eq!(record.count(), 2);\n            assert_eq!(record.action_disposition(), ActionDisposition::Pass);\n            assert_eq!(record.envelope_from(), \"hello@example.org\");\n            assert_eq!(record.header_from(), \"bye@example.org\");\n            assert_eq!(record.envelope_to().unwrap(), \"other@example.org\");\n        } else if source_ip == \"a:b:c::e:f\".parse::<IpAddr>().unwrap() {\n            assert_eq!(record.count(), 1);\n            assert_eq!(record.action_disposition(), ActionDisposition::Reject);\n        } else {\n            panic!(\"unexpected ip {source_ip}\");\n        }\n    }\n    qr.assert_report_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/smtp/reporting/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod analyze;\npub mod dmarc;\npub mod scheduler;\npub mod tls;\n"
  },
  {
    "path": "tests/src/smtp/reporting/scheduler.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::sync::Arc;\n\nuse common::{\n    config::smtp::report::AggregateFrequency,\n    ipc::{DmarcEvent, PolicyType, TlsEvent},\n};\nuse mail_auth::{\n    common::parse::TxtRecordParser,\n    dmarc::{Dmarc, URI},\n    mta_sts::TlsRpt,\n    report::{ActionDisposition, Alignment, Disposition, DmarcResult, PolicyPublished, Record},\n};\nuse store::write::QueueClass;\n\nuse smtp::reporting::{\n    dmarc::{DmarcFormat, DmarcReporting},\n    tls::TlsReporting,\n};\n\nuse crate::smtp::TestSMTP;\n\nconst CONFIG: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[report.dmarc.aggregate]\nmax-size = 500\nsend = \"daily\"\n\n[report.tls.aggregate]\nmax-size = 550\nsend = \"daily\"\n\"#;\n\n#[tokio::test]\nasync fn report_scheduler() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Create scheduler\n    let local = TestSMTP::new(\"smtp_report_queue_test\", CONFIG).await;\n    let core = local.build_smtp();\n    let qr = &local.queue_receiver;\n\n    // Schedule two events with a same policy and another one with a different policy\n    let dmarc_record =\n        Arc::new(Dmarc::parse(b\"v=DMARC1; p=quarantine; rua=mailto:dmarc@foobar.org\").unwrap());\n    core.schedule_dmarc(Box::new(DmarcEvent {\n        domain: \"foobar.org\".to_string(),\n        report_record: Record::new()\n            .with_source_ip(\"192.168.1.2\".parse().unwrap())\n            .with_action_disposition(ActionDisposition::Pass)\n            .with_dmarc_dkim_result(DmarcResult::Pass)\n            .with_dmarc_spf_result(DmarcResult::Fail)\n            .with_envelope_from(\"hello@example.org\")\n            .with_envelope_to(\"other@example.org\")\n            .with_header_from(\"bye@example.org\"),\n        dmarc_record: dmarc_record.clone(),\n        interval: AggregateFrequency::Weekly,\n    }))\n    .await;\n\n    // No records should be added once the 550 bytes max size is reached\n    for _ in 0..10 {\n        core.schedule_dmarc(Box::new(DmarcEvent {\n            domain: \"foobar.org\".to_string(),\n            report_record: Record::new()\n                .with_source_ip(\"192.168.1.2\".parse().unwrap())\n                .with_action_disposition(ActionDisposition::Pass)\n                .with_dmarc_dkim_result(DmarcResult::Pass)\n                .with_dmarc_spf_result(DmarcResult::Fail)\n                .with_envelope_from(\"hello@example.org\")\n                .with_envelope_to(\"other@example.org\")\n                .with_header_from(\"bye@example.org\"),\n            dmarc_record: dmarc_record.clone(),\n            interval: AggregateFrequency::Weekly,\n        }))\n        .await;\n    }\n    let dmarc_record =\n        Arc::new(Dmarc::parse(b\"v=DMARC1; p=reject; rua=mailto:dmarc@foobar.org\").unwrap());\n    core.schedule_dmarc(Box::new(DmarcEvent {\n        domain: \"foobar.org\".to_string(),\n        report_record: Record::new()\n            .with_source_ip(\"a:b:c::e:f\".parse().unwrap())\n            .with_action_disposition(ActionDisposition::Reject)\n            .with_dmarc_dkim_result(DmarcResult::Fail)\n            .with_dmarc_spf_result(DmarcResult::Pass),\n        dmarc_record: dmarc_record.clone(),\n        interval: AggregateFrequency::Weekly,\n    }))\n    .await;\n\n    // Schedule TLS event\n    let tls_record = Arc::new(TlsRpt::parse(b\"v=TLSRPTv1;rua=mailto:reports@foobar.org\").unwrap());\n    core.schedule_tls(Box::new(TlsEvent {\n        domain: \"foobar.org\".to_string(),\n        policy: PolicyType::Tlsa(None),\n        failure: None,\n        tls_record: tls_record.clone(),\n        interval: AggregateFrequency::Daily,\n    }))\n    .await;\n    core.schedule_tls(Box::new(TlsEvent {\n        domain: \"foobar.org\".to_string(),\n        policy: PolicyType::Tlsa(None),\n        failure: None,\n        tls_record: tls_record.clone(),\n        interval: AggregateFrequency::Daily,\n    }))\n    .await;\n    core.schedule_tls(Box::new(TlsEvent {\n        domain: \"foobar.org\".to_string(),\n        policy: PolicyType::Sts(None),\n        failure: None,\n        tls_record: tls_record.clone(),\n        interval: AggregateFrequency::Daily,\n    }))\n    .await;\n    core.schedule_tls(Box::new(TlsEvent {\n        domain: \"foobar.org\".to_string(),\n        policy: PolicyType::None,\n        failure: None,\n        tls_record: tls_record.clone(),\n        interval: AggregateFrequency::Daily,\n    }))\n    .await;\n\n    // Verify sizes and counts\n    let mut total_tls = 0;\n    let mut total_tls_policies = 0;\n    let mut total_dmarc_policies = 0;\n    let mut last_domain = String::new();\n    for report in qr.read_report_events().await {\n        match report {\n            QueueClass::DmarcReportHeader(event) => {\n                total_dmarc_policies += 1;\n                assert_eq!(event.due - event.seq_id, 7 * 86400);\n            }\n            QueueClass::TlsReportHeader(event) => {\n                if event.domain != last_domain {\n                    last_domain.clone_from(&event.domain);\n                    total_tls += 1;\n                }\n                total_tls_policies += 1;\n                assert_eq!(event.due - event.seq_id, 86400);\n            }\n            _ => unreachable!(),\n        }\n    }\n    assert_eq!(total_tls, 1);\n    assert_eq!(total_tls_policies, 3);\n    assert_eq!(total_dmarc_policies, 2);\n}\n\n#[test]\nfn report_strip_json() {\n    let mut d = DmarcFormat {\n        rua: vec![\n            URI {\n                uri: \"hello\".to_string(),\n                max_size: 0,\n            },\n            URI {\n                uri: \"world\".to_string(),\n                max_size: 0,\n            },\n        ],\n        policy: PolicyPublished {\n            domain: \"example.org\".to_string(),\n            version_published: None,\n            adkim: Alignment::Relaxed,\n            aspf: Alignment::Strict,\n            p: Disposition::Quarantine,\n            sp: Disposition::Reject,\n            testing: false,\n            fo: None,\n        },\n        records: vec![\n            Record::default()\n                .with_count(1)\n                .with_envelope_from(\"domain.net\")\n                .with_envelope_to(\"other.org\"),\n        ],\n    };\n    let mut s = serde_json::to_string(&d).unwrap();\n    s.truncate(s.len() - 2);\n\n    let r = Record::default()\n        .with_count(2)\n        .with_envelope_from(\"otherdomain.net\")\n        .with_envelope_to(\"otherother.org\");\n    let rs = serde_json::to_string(&r).unwrap();\n\n    d.records.push(r);\n\n    assert_eq!(\n        serde_json::from_str::<DmarcFormat>(&format!(\"{s},{rs}]}}\")).unwrap(),\n        d\n    );\n}\n"
  },
  {
    "path": "tests/src/smtp/reporting/tls.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{io::Read, sync::Arc, time::Duration};\n\nuse common::{config::smtp::report::AggregateFrequency, ipc::TlsEvent};\nuse mail_auth::{\n    common::parse::TxtRecordParser,\n    flate2::read::GzDecoder,\n    mta_sts::TlsRpt,\n    report::tlsrpt::{FailureDetails, PolicyType, ResultType, TlsReport},\n};\nuse store::write::QueueClass;\n\nuse smtp::reporting::tls::{TLS_HTTP_REPORT, TlsReporting};\n\nuse crate::smtp::{\n    TestSMTP,\n    inbound::{TestMessage, sign::SIGNATURES},\n    session::VerifyResponse,\n};\n\nconst CONFIG: &str = r#\"\n[session.rcpt]\nrelay = true\n\n[report]\nsubmitter = \"'mx.example.org'\"\n\n[report.tls.aggregate]\nfrom-name = \"'Report Subsystem'\"\nfrom-address = \"'reports@example.org'\"\norg-name = \"'Foobar, Inc.'\"\ncontact-info = \"'https://foobar.org/contact'\"\nsend = \"daily\"\nmax-size = 1532\nsign = \"['rsa']\"\n\"#;\n\n#[tokio::test]\nasync fn report_tls() {\n    // Enable logging\n    crate::enable_logging();\n\n    // Create scheduler\n    let mut local = TestSMTP::new(\"smtp_report_tls_test\", CONFIG.to_string() + SIGNATURES).await;\n    let core = local.build_smtp();\n    let qr = &mut local.queue_receiver;\n\n    // Schedule TLS reports to be delivered via email\n    let tls_record = Arc::new(TlsRpt::parse(b\"v=TLSRPTv1;rua=mailto:reports@foobar.org\").unwrap());\n\n    for _ in 0..2 {\n        // Add two successful records\n        core.schedule_tls(Box::new(TlsEvent {\n            domain: \"foobar.org\".to_string(),\n            policy: common::ipc::PolicyType::None,\n            failure: None,\n            tls_record: tls_record.clone(),\n            interval: AggregateFrequency::Daily,\n        }))\n        .await;\n    }\n\n    for (policy, rt) in [\n        (\n            common::ipc::PolicyType::None, // Quota limited at 1532 bytes, this should not be included in the report.\n            ResultType::CertificateExpired,\n        ),\n        (common::ipc::PolicyType::Tlsa(None), ResultType::TlsaInvalid),\n        (\n            common::ipc::PolicyType::Sts(None),\n            ResultType::StsPolicyFetchError,\n        ),\n        (\n            common::ipc::PolicyType::Sts(None),\n            ResultType::StsPolicyInvalid,\n        ),\n        (\n            common::ipc::PolicyType::Sts(None),\n            ResultType::StsWebpkiInvalid,\n        ),\n    ] {\n        core.schedule_tls(Box::new(TlsEvent {\n            domain: \"foobar.org\".to_string(),\n            policy,\n            failure: FailureDetails::new(rt).into(),\n            tls_record: tls_record.clone(),\n            interval: AggregateFrequency::Daily,\n        }))\n        .await;\n    }\n\n    // Wait for flush\n    tokio::time::sleep(Duration::from_millis(200)).await;\n    let reports = qr.read_report_events().await;\n    assert_eq!(reports.len(), 3);\n    let mut tls_reports = Vec::with_capacity(3);\n    for report in reports {\n        match report {\n            QueueClass::TlsReportHeader(event) => {\n                tls_reports.push(event);\n            }\n            _ => unreachable!(),\n        }\n    }\n    core.send_tls_aggregate_report(tls_reports).await;\n\n    // Expect report\n    let message = qr.expect_message().await;\n    assert_eq!(\n        message.message.recipients.last().unwrap().address(),\n        \"reports@foobar.org\"\n    );\n    assert_eq!(message.message.return_path.as_ref(), \"reports@example.org\");\n    message\n        .read_lines(qr)\n        .await\n        .assert_contains(\"DKIM-Signature: v=1; a=rsa-sha256; s=rsa; d=example.com;\")\n        .assert_contains(\"To: <reports@foobar.org>\")\n        .assert_contains(\"Report Domain: foobar.org\")\n        .assert_contains(\"Submitter: mx.example.org\");\n\n    // Verify generated report\n    let report = TlsReport::parse_rfc5322(message.read_message(qr).await.as_bytes()).unwrap();\n    assert_eq!(report.organization_name.unwrap(), \"Foobar, Inc.\");\n    assert_eq!(report.contact_info.unwrap(), \"https://foobar.org/contact\");\n    assert_eq!(report.policies.len(), 3);\n    let mut seen = [false; 3];\n    for policy in report.policies {\n        match policy.policy.policy_type {\n            PolicyType::Tlsa => {\n                seen[0] = true;\n                assert_eq!(policy.summary.total_failure, 1);\n                assert_eq!(policy.summary.total_success, 0);\n                assert_eq!(policy.policy.policy_domain, \"foobar.org\");\n                assert_eq!(policy.failure_details.len(), 1);\n                assert_eq!(\n                    policy.failure_details.first().unwrap().result_type,\n                    ResultType::TlsaInvalid\n                );\n            }\n            PolicyType::Sts => {\n                seen[1] = true;\n                assert_eq!(policy.summary.total_failure, 2);\n                assert_eq!(policy.summary.total_success, 0);\n                assert_eq!(policy.policy.policy_domain, \"foobar.org\");\n                assert_eq!(policy.failure_details.len(), 2);\n                assert!(\n                    policy\n                        .failure_details\n                        .iter()\n                        .any(|d| d.result_type == ResultType::StsPolicyFetchError)\n                );\n                assert!(\n                    policy\n                        .failure_details\n                        .iter()\n                        .any(|d| d.result_type == ResultType::StsPolicyInvalid)\n                );\n            }\n            PolicyType::NoPolicyFound => {\n                seen[2] = true;\n                assert_eq!(policy.summary.total_failure, 1);\n                assert_eq!(policy.summary.total_success, 2);\n                assert_eq!(policy.policy.policy_domain, \"foobar.org\");\n                assert_eq!(policy.failure_details.len(), 1);\n                /*assert_eq!(\n                    policy.failure_details.first().unwrap().result_type,\n                    ResultType::CertificateExpired\n                );*/\n            }\n            PolicyType::Other => unreachable!(),\n        }\n    }\n\n    assert!(seen[0]);\n    assert!(seen[1]);\n    assert!(seen[2]);\n\n    // Schedule TLS reports to be delivered via https\n    let tls_record = Arc::new(TlsRpt::parse(b\"v=TLSRPTv1;rua=https://127.0.0.1/tls\").unwrap());\n\n    for _ in 0..2 {\n        // Add two successful records\n        core.schedule_tls(Box::new(TlsEvent {\n            domain: \"foobar.org\".to_string(),\n            policy: common::ipc::PolicyType::None,\n            failure: None,\n            tls_record: tls_record.clone(),\n            interval: AggregateFrequency::Daily,\n        }))\n        .await;\n    }\n\n    let reports = qr.read_report_events().await;\n    assert_eq!(reports.len(), 1);\n    match reports.into_iter().next().unwrap() {\n        QueueClass::TlsReportHeader(event) => {\n            core.send_tls_aggregate_report(vec![event]).await;\n        }\n        _ => unreachable!(),\n    }\n    tokio::time::sleep(Duration::from_millis(200)).await;\n\n    // Uncompress report\n    {\n        let gz_report = TLS_HTTP_REPORT.lock();\n        let mut file = GzDecoder::new(&gz_report[..]);\n        let mut buf = Vec::new();\n        file.read_to_end(&mut buf).unwrap();\n        let report = TlsReport::parse_json(&buf).unwrap();\n        assert_eq!(report.organization_name.unwrap(), \"Foobar, Inc.\");\n        assert_eq!(report.contact_info.unwrap(), \"https://foobar.org/contact\");\n        assert_eq!(report.policies.len(), 1);\n    }\n    qr.assert_report_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/smtp/session.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::{borrow::Cow, path::PathBuf, sync::Arc};\n\nuse common::{\n    Server,\n    config::server::ServerProtocol,\n    listener::{ServerInstance, SessionStream, TcpAcceptor, limiter::ConcurrencyLimiter},\n};\nuse rustls::{ServerConfig, server::ResolvesServerCert};\nuse tokio::{\n    io::{AsyncRead, AsyncWrite},\n    sync::watch,\n};\n\nuse smtp::core::{Session, SessionAddress, SessionData, SessionParameters, State};\nuse tokio_rustls::TlsAcceptor;\nuse utils::snowflake::SnowflakeIdGenerator;\n\npub struct DummyIo {\n    pub tx_buf: Vec<u8>,\n    pub rx_buf: Vec<u8>,\n    pub tls: bool,\n}\n\nimpl AsyncRead for DummyIo {\n    fn poll_read(\n        mut self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n        buf: &mut tokio::io::ReadBuf<'_>,\n    ) -> std::task::Poll<std::io::Result<()>> {\n        if !self.rx_buf.is_empty() {\n            buf.put_slice(&self.rx_buf);\n            self.rx_buf.clear();\n            std::task::Poll::Ready(Ok(()))\n        } else {\n            std::task::Poll::Pending\n        }\n    }\n}\n\nimpl AsyncWrite for DummyIo {\n    fn poll_write(\n        mut self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n        buf: &[u8],\n    ) -> std::task::Poll<Result<usize, std::io::Error>> {\n        self.tx_buf.extend_from_slice(buf);\n        std::task::Poll::Ready(Ok(buf.len()))\n    }\n\n    fn poll_flush(\n        self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n    ) -> std::task::Poll<Result<(), std::io::Error>> {\n        std::task::Poll::Ready(Ok(()))\n    }\n\n    fn poll_shutdown(\n        self: std::pin::Pin<&mut Self>,\n        _cx: &mut std::task::Context<'_>,\n    ) -> std::task::Poll<Result<(), std::io::Error>> {\n        std::task::Poll::Ready(Ok(()))\n    }\n}\n\nimpl SessionStream for DummyIo {\n    fn is_tls(&self) -> bool {\n        self.tls\n    }\n\n    fn tls_version_and_cipher(&self) -> (Cow<'static, str>, Cow<'static, str>) {\n        (\"\".into(), \"\".into())\n    }\n}\n\nimpl Unpin for DummyIo {}\n\n#[allow(async_fn_in_trait)]\npub trait TestSession {\n    fn test(server: Server) -> Self;\n    fn test_with_shutdown(server: Server, shutdown_rx: watch::Receiver<bool>) -> Self;\n    fn response(&mut self) -> Vec<String>;\n    fn write_rx(&mut self, data: &str);\n    async fn rset(&mut self);\n    async fn cmd(&mut self, cmd: &str, expected_code: &str) -> Vec<String>;\n    async fn ehlo(&mut self, host: &str) -> Vec<String>;\n    async fn mail_from(&mut self, from: &str, expected_code: &str);\n    async fn rcpt_to(&mut self, to: &str, expected_code: &str);\n    async fn data(&mut self, data: &str, expected_code: &str);\n    async fn send_message(&mut self, from: &str, to: &[&str], data: &str, expected_code: &str);\n    async fn test_builder(&self);\n}\n\nimpl TestSession for Session<DummyIo> {\n    fn test_with_shutdown(server: Server, shutdown_rx: watch::Receiver<bool>) -> Self {\n        Self {\n            state: State::default(),\n            instance: Arc::new(ServerInstance::test_with_shutdown(shutdown_rx)),\n            server,\n            stream: DummyIo {\n                rx_buf: vec![],\n                tx_buf: vec![],\n                tls: false,\n            },\n            data: SessionData::new(\n                \"127.0.0.1\".parse().unwrap(),\n                0,\n                \"127.0.0.1\".parse().unwrap(),\n                0,\n                Default::default(),\n                0,\n            ),\n            params: SessionParameters::default(),\n            hostname: \"localhost\".into(),\n        }\n    }\n\n    fn test(server: Server) -> Self {\n        Self::test_with_shutdown(server, watch::channel(false).1)\n    }\n\n    fn response(&mut self) -> Vec<String> {\n        if !self.stream.tx_buf.is_empty() {\n            let response = std::str::from_utf8(&self.stream.tx_buf)\n                .unwrap()\n                .split(\"\\r\\n\")\n                .filter_map(|r| {\n                    if !r.is_empty() {\n                        r.to_string().into()\n                    } else {\n                        None\n                    }\n                })\n                .collect::<Vec<_>>();\n            self.stream.tx_buf.clear();\n            response\n        } else {\n            panic!(\"There was no response.\");\n        }\n    }\n\n    fn write_rx(&mut self, data: &str) {\n        self.stream.rx_buf.extend_from_slice(data.as_bytes());\n    }\n\n    async fn rset(&mut self) {\n        self.ingest(b\"RSET\\r\\n\").await.unwrap();\n        self.response().assert_code(\"250\");\n    }\n\n    async fn cmd(&mut self, cmd: &str, expected_code: &str) -> Vec<String> {\n        self.ingest(format!(\"{cmd}\\r\\n\").as_bytes()).await.unwrap();\n        self.response().assert_code(expected_code)\n    }\n\n    async fn ehlo(&mut self, host: &str) -> Vec<String> {\n        self.ingest(format!(\"EHLO {host}\\r\\n\").as_bytes())\n            .await\n            .unwrap();\n        self.response().assert_code(\"250\")\n    }\n\n    async fn mail_from(&mut self, from: &str, expected_code: &str) {\n        self.ingest(\n            if !from.starts_with('<') {\n                format!(\"MAIL FROM:<{from}>\\r\\n\")\n            } else {\n                format!(\"MAIL FROM:{from}\\r\\n\")\n            }\n            .as_bytes(),\n        )\n        .await\n        .unwrap();\n        self.response().assert_code(expected_code);\n    }\n\n    async fn rcpt_to(&mut self, to: &str, expected_code: &str) {\n        self.ingest(\n            if !to.starts_with('<') {\n                format!(\"RCPT TO:<{to}>\\r\\n\")\n            } else {\n                format!(\"RCPT TO:{to}\\r\\n\")\n            }\n            .as_bytes(),\n        )\n        .await\n        .unwrap();\n        self.response().assert_code(expected_code);\n    }\n\n    async fn data(&mut self, data: &str, expected_code: &str) {\n        self.ingest(b\"DATA\\r\\n\").await.unwrap();\n        self.response().assert_code(\"354\");\n        if let Some(file) = data.strip_prefix(\"test:\") {\n            self.ingest(load_test_message(file, \"messages\").as_bytes())\n                .await\n                .unwrap();\n        } else if let Some(file) = data.strip_prefix(\"report:\") {\n            self.ingest(load_test_message(file, \"reports\").as_bytes())\n                .await\n                .unwrap();\n        } else {\n            self.ingest(data.as_bytes()).await.unwrap();\n        }\n        self.ingest(b\"\\r\\n.\\r\\n\").await.unwrap();\n        self.response().assert_code(expected_code);\n    }\n\n    async fn send_message(&mut self, from: &str, to: &[&str], data: &str, expected_code: &str) {\n        self.mail_from(from, \"250\").await;\n        for to in to {\n            self.rcpt_to(to, \"250\").await;\n        }\n        self.data(data, expected_code).await;\n    }\n\n    async fn test_builder(&self) {\n        let message = self\n            .build_message(\n                SessionAddress {\n                    address: \"bill@foobar.org\".into(),\n                    address_lcase: \"bill@foobar.org\".into(),\n                    domain: \"foobar.org\".into(),\n                    flags: 123,\n                    dsn_info: Some(\"envelope1\".into()),\n                },\n                vec![\n                    SessionAddress {\n                        address: \"a@foobar.org\".into(),\n                        address_lcase: \"a@foobar.org\".into(),\n                        domain: \"foobar.org\".into(),\n                        flags: 1,\n                        dsn_info: None,\n                    },\n                    SessionAddress {\n                        address: \"b@test.net\".into(),\n                        address_lcase: \"b@test.net\".into(),\n                        domain: \"test.net\".into(),\n                        flags: 2,\n                        dsn_info: None,\n                    },\n                    SessionAddress {\n                        address: \"c@foobar.org\".into(),\n                        address_lcase: \"c@foobar.org\".into(),\n                        domain: \"foobar.org\".into(),\n                        flags: 3,\n                        dsn_info: None,\n                    },\n                    SessionAddress {\n                        address: \"d@test.net\".into(),\n                        address_lcase: \"d@test.net\".into(),\n                        domain: \"test.net\".into(),\n                        flags: 4,\n                        dsn_info: None,\n                    },\n                ],\n                self.server.inner.data.queue_id_gen.generate(),\n                0,\n            )\n            .await;\n\n        let rcpts = [\"a@foobar.org\", \"b@test.net\", \"c@foobar.org\", \"d@test.net\"];\n        for rcpt in &message.message.recipients {\n            let idx = (rcpt.flags - 1) as usize;\n            assert_eq!(rcpts[idx], rcpt.address());\n        }\n    }\n}\n\npub fn load_test_message(file: &str, test: &str) -> String {\n    let mut test_file = PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    test_file.push(\"resources\");\n    test_file.push(\"smtp\");\n    test_file.push(test);\n    test_file.push(format!(\"{file}.eml\"));\n    std::fs::read_to_string(test_file).unwrap()\n}\n\npub trait VerifyResponse {\n    fn assert_code(self, expected_code: &str) -> Self;\n    fn assert_contains(self, expected_text: &str) -> Self;\n    fn assert_not_contains(self, expected_text: &str) -> Self;\n    fn assert_count(self, text: &str, occurrences: usize) -> Self;\n}\n\nimpl VerifyResponse for Vec<String> {\n    fn assert_code(self, expected_code: &str) -> Self {\n        if self.last().expect(\"response\").starts_with(expected_code) {\n            self\n        } else {\n            panic!(\"Expected {:?} but got {}.\", expected_code, self.join(\"\\n\"));\n        }\n    }\n\n    fn assert_contains(self, expected_text: &str) -> Self {\n        if self.iter().any(|line| line.contains(expected_text)) {\n            self\n        } else {\n            panic!(\"Expected {:?} but got {}.\", expected_text, self.join(\"\\n\"));\n        }\n    }\n\n    fn assert_not_contains(self, expected_text: &str) -> Self {\n        if !self.iter().any(|line| line.contains(expected_text)) {\n            self\n        } else {\n            panic!(\n                \"Not expecting {:?} but got it {}.\",\n                expected_text,\n                self.join(\"\\n\")\n            );\n        }\n    }\n\n    fn assert_count(self, text: &str, occurrences: usize) -> Self {\n        assert_eq!(\n            self.iter().filter(|l| l.contains(text)).count(),\n            occurrences,\n            \"Expected {} occurrences of {:?}, found {}.\",\n            occurrences,\n            text,\n            self.iter().filter(|l| l.contains(text)).count()\n        );\n        self\n    }\n}\n\npub trait TestServerInstance {\n    fn test_with_shutdown(shutdown_rx: watch::Receiver<bool>) -> Self;\n}\n\nimpl TestServerInstance for ServerInstance {\n    fn test_with_shutdown(shutdown_rx: watch::Receiver<bool>) -> Self {\n        let tls_config = Arc::new(\n            ServerConfig::builder()\n                .with_no_client_auth()\n                .with_cert_resolver(Arc::new(DummyCertResolver)),\n        );\n\n        Self {\n            id: \"smtp\".to_string(),\n            protocol: ServerProtocol::Smtp,\n            acceptor: TcpAcceptor::Tls {\n                config: tls_config.clone(),\n                acceptor: TlsAcceptor::from(tls_config),\n                implicit: false,\n            },\n            limiter: ConcurrencyLimiter::new(100),\n            shutdown_rx,\n            proxy_networks: vec![],\n            span_id_gen: Arc::new(SnowflakeIdGenerator::new()),\n        }\n    }\n}\n\n#[derive(Debug)]\npub struct DummyCertResolver;\n\nimpl ResolvesServerCert for DummyCertResolver {\n    fn resolve(&self, _: rustls::server::ClientHello) -> Option<Arc<rustls::sign::CertifiedKey>> {\n        None\n    }\n}\n\npub fn test_server_instance() -> ServerInstance {\n    ServerInstance::test_with_shutdown(watch::channel(false).1)\n}\n"
  },
  {
    "path": "tests/src/store/blob.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::store::{CONFIG, TempDir, cleanup::store_destroy};\nuse ahash::AHashMap;\nuse common::{Core, Inner, Server, config::storage::Storage};\nuse email::message::metadata::MessageMetadata;\nuse http::management::stores::destroy_account_blobs;\nuse std::sync::Arc;\nuse store::{\n    BlobStore, Serialize, SerializeInfallible, Stores,\n    write::{Archiver, BatchBuilder, BlobLink, BlobOp, ValueClass, blob::BlobQuota, now},\n};\nuse types::{blob::BlobClass, blob_hash::BlobHash, collection::Collection, field::EmailField};\nuse utils::config::Config;\n\n#[tokio::test]\npub async fn blob_tests() {\n    let temp_dir = TempDir::new(\"blob_tests\", true);\n    let mut config =\n        Config::new(CONFIG.replace(\"{TMP}\", temp_dir.path.as_path().to_str().unwrap())).unwrap();\n    let stores = Stores::parse_all(&mut config, false).await;\n\n    for (store_id, blob_store) in &stores.blob_stores {\n        println!(\"Testing blob store {}...\", store_id);\n        test_store(blob_store.clone()).await;\n    }\n\n    for (store_id, store) in stores.stores {\n        println!(\"Testing blob management on store {}...\", store_id);\n\n        // Init store\n        store_destroy(&store).await;\n\n        // Test internal blob store\n        let blob_store: BlobStore = store.clone().into();\n        let server = Server {\n            inner: Arc::new(Inner::default()),\n            core: Arc::new(Core {\n                storage: Storage {\n                    data: store.clone(),\n                    blob: blob_store.clone(),\n                    ..Default::default()\n                },\n                ..Default::default()\n            }),\n        };\n\n        // Blob hash exists\n        let hash = BlobHash::generate(b\"abc\".as_slice());\n        assert!(!store.blob_exists(&hash).await.unwrap());\n\n        // Reserve blob\n        let until = now() + 1;\n        store\n            .write(\n                BatchBuilder::new()\n                    .with_account_id(0)\n                    .set(\n                        BlobOp::Link {\n                            to: BlobLink::Temporary { until },\n                            hash: hash.clone(),\n                        },\n                        1024u32.serialize(),\n                    )\n                    .build_all(),\n            )\n            .await\n            .unwrap();\n\n        // Uncommitted blob, should not exist\n        assert!(!store.blob_exists(&hash).await.unwrap());\n\n        // Write blob to store\n        blob_store.put_blob(hash.as_ref(), b\"abc\").await.unwrap();\n\n        // Commit blob\n        store\n            .write(\n                BatchBuilder::new()\n                    .set(BlobOp::Commit { hash: hash.clone() }, Vec::new())\n                    .build_all(),\n            )\n            .await\n            .unwrap();\n\n        // Blob hash should now exist\n        assert!(store.blob_exists(&hash).await.unwrap());\n        assert!(\n            blob_store\n                .get_blob(hash.as_ref(), 0..usize::MAX)\n                .await\n                .unwrap()\n                .is_some()\n        );\n\n        // AccountId 0 should be able to read blob\n        assert!(\n            store\n                .blob_has_access(\n                    &hash,\n                    BlobClass::Reserved {\n                        account_id: 0,\n                        expires: until\n                    }\n                )\n                .await\n                .unwrap()\n        );\n\n        // AccountId 1 should not be able to read blob\n        assert!(\n            !store\n                .blob_has_access(\n                    &hash,\n                    BlobClass::Reserved {\n                        account_id: 1,\n                        expires: until\n                    }\n                )\n                .await\n                .unwrap()\n        );\n\n        // Blob already expired, quota should be 0\n        tokio::time::sleep(std::time::Duration::from_secs(1)).await;\n        assert_eq!(\n            store.blob_quota(0).await.unwrap(),\n            BlobQuota { bytes: 0, count: 0 }\n        );\n\n        // Purge expired blobs\n        store.purge_blobs(blob_store.clone()).await.unwrap();\n\n        // Blob hash should no longer exist\n        assert!(!store.blob_exists(&hash).await.unwrap());\n\n        // AccountId 0 should not be able to read blob\n        assert!(\n            !store\n                .blob_has_access(\n                    &hash,\n                    BlobClass::Reserved {\n                        account_id: 0,\n                        expires: until\n                    }\n                )\n                .await\n                .unwrap()\n        );\n\n        // Blob should no longer be in store\n        assert!(\n            blob_store\n                .get_blob(hash.as_ref(), 0..usize::MAX)\n                .await\n                .unwrap()\n                .is_none()\n        );\n\n        // Upload one linked blob to accountId 1, two linked blobs to accountId 0, and three unlinked (reserved) blobs to accountId 2\n        let expiry_times = AHashMap::from_iter([\n            (b\"abc\", now() - 10),\n            (b\"efg\", now() + 10),\n            (b\"hij\", now() + 10),\n        ]);\n        for (document_id, (blob, blob_value)) in [\n            (b\"123\", vec![]),\n            (b\"456\", vec![]),\n            (b\"789\", vec![]),\n            (b\"abc\", 5000u32.serialize()),\n            (b\"efg\", 1000u32.serialize()),\n            (b\"hij\", 2000u32.serialize()),\n        ]\n        .into_iter()\n        .enumerate()\n        {\n            let hash = BlobHash::generate(blob.as_slice());\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(if document_id > 0 { 0 } else { 1 })\n                .with_collection(Collection::Email)\n                .with_document(document_id as u32);\n            if let Some(until) = expiry_times.get(blob) {\n                if !blob_value.is_empty() {\n                    batch.set(\n                        BlobOp::Quota {\n                            hash: hash.clone(),\n                            until: *until,\n                        },\n                        blob_value,\n                    );\n                }\n                batch.set(\n                    BlobOp::Link {\n                        hash: hash.clone(),\n                        to: BlobLink::Temporary { until: *until },\n                    },\n                    vec![],\n                );\n            } else {\n                batch\n                    .set(\n                        BlobOp::Link {\n                            hash: hash.clone(),\n                            to: BlobLink::Document,\n                        },\n                        vec![],\n                    )\n                    .set(\n                        ValueClass::Property(EmailField::Metadata.into()),\n                        Archiver::new(MessageMetadata {\n                            contents: Default::default(),\n                            rcvd_attach: Default::default(),\n                            blob_hash: hash.clone(),\n                            blob_body_offset: Default::default(),\n                            preview: Default::default(),\n                            raw_headers: Default::default(),\n                        })\n                        .serialize()\n                        .unwrap(),\n                    );\n            };\n            batch.set(BlobOp::Commit { hash: hash.clone() }, vec![]);\n\n            store.write(batch.build_all()).await.unwrap();\n            blob_store\n                .put_blob(hash.as_ref(), blob.as_slice())\n                .await\n                .unwrap();\n        }\n\n        // One of the reserved blobs expired and should not count towards quota\n        assert_eq!(\n            store.blob_quota(0).await.unwrap(),\n            BlobQuota {\n                bytes: 3000,\n                count: 2\n            }\n        );\n        assert_eq!(\n            store.blob_quota(1).await.unwrap(),\n            BlobQuota { bytes: 0, count: 0 }\n        );\n\n        // Purge expired blobs and make sure nothing else is deleted\n        store.purge_blobs(blob_store.clone()).await.unwrap();\n        for (pos, (blob, blob_class)) in [\n            (\n                b\"abc\",\n                BlobClass::Reserved {\n                    account_id: 0,\n                    expires: expiry_times[&b\"abc\"],\n                },\n            ),\n            (\n                b\"123\",\n                BlobClass::Linked {\n                    account_id: 1,\n                    collection: 0,\n                    document_id: 0,\n                },\n            ),\n            (\n                b\"456\",\n                BlobClass::Linked {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 1,\n                },\n            ),\n            (\n                b\"789\",\n                BlobClass::Linked {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 2,\n                },\n            ),\n            (\n                b\"efg\",\n                BlobClass::Reserved {\n                    account_id: 0,\n                    expires: expiry_times[&b\"efg\"],\n                },\n            ),\n            (\n                b\"hij\",\n                BlobClass::Reserved {\n                    account_id: 0,\n                    expires: expiry_times[&b\"hij\"],\n                },\n            ),\n        ]\n        .into_iter()\n        .enumerate()\n        {\n            let ct = pos == 0;\n            let hash = BlobHash::generate(blob.as_slice());\n            assert!(store.blob_has_access(&hash, blob_class).await.unwrap() ^ ct);\n            assert!(store.blob_exists(&hash).await.unwrap() ^ ct);\n            assert!(\n                blob_store\n                    .get_blob(hash.as_ref(), 0..usize::MAX)\n                    .await\n                    .unwrap()\n                    .is_some()\n                    ^ ct\n            );\n        }\n\n        // AccountId 0 should not have access to accountId 1's blobs\n        assert!(\n            !store\n                .blob_has_access(\n                    BlobHash::generate(b\"123\".as_slice()),\n                    BlobClass::Linked {\n                        account_id: 0,\n                        collection: 0,\n                        document_id: 0,\n                    }\n                )\n                .await\n                .unwrap()\n        );\n\n        // Unlink blob\n        store\n            .write(\n                BatchBuilder::new()\n                    .with_account_id(0)\n                    .with_collection(Collection::Email)\n                    .with_document(2)\n                    .clear(BlobOp::Link {\n                        hash: BlobHash::generate(b\"789\".as_slice()),\n                        to: BlobLink::Document,\n                    })\n                    .build_all(),\n            )\n            .await\n            .unwrap();\n\n        // Purge and make sure blob is deleted\n        store.purge_blobs(blob_store.clone()).await.unwrap();\n        for (pos, (blob, blob_class)) in [\n            (\n                b\"789\",\n                BlobClass::Linked {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 2,\n                },\n            ),\n            (\n                b\"123\",\n                BlobClass::Linked {\n                    account_id: 1,\n                    collection: 0,\n                    document_id: 0,\n                },\n            ),\n            (\n                b\"456\",\n                BlobClass::Linked {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 1,\n                },\n            ),\n            (\n                b\"efg\",\n                BlobClass::Reserved {\n                    account_id: 0,\n                    expires: expiry_times[&b\"efg\"],\n                },\n            ),\n            (\n                b\"hij\",\n                BlobClass::Reserved {\n                    account_id: 0,\n                    expires: expiry_times[&b\"hij\"],\n                },\n            ),\n        ]\n        .into_iter()\n        .enumerate()\n        {\n            let ct = pos == 0;\n            let hash = BlobHash::generate(blob.as_slice());\n            assert!(store.blob_has_access(&hash, blob_class).await.unwrap() ^ ct);\n            assert!(store.blob_exists(&hash).await.unwrap() ^ ct);\n            assert!(\n                blob_store\n                    .get_blob(hash.as_ref(), 0..usize::MAX)\n                    .await\n                    .unwrap()\n                    .is_some()\n                    ^ ct\n            );\n        }\n\n        // Unlink all blobs from accountId 1 and purge\n        destroy_account_blobs(&server, 1).await.unwrap();\n        store.purge_blobs(blob_store.clone()).await.unwrap();\n\n        // Make sure only accountId 0's blobs are left\n        for (pos, (blob, blob_class)) in [\n            (\n                b\"123\",\n                BlobClass::Linked {\n                    account_id: 1,\n                    collection: 0,\n                    document_id: 0,\n                },\n            ),\n            (\n                b\"456\",\n                BlobClass::Linked {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 1,\n                },\n            ),\n            (\n                b\"efg\",\n                BlobClass::Reserved {\n                    account_id: 0,\n                    expires: expiry_times[&b\"efg\"],\n                },\n            ),\n            (\n                b\"hij\",\n                BlobClass::Reserved {\n                    account_id: 0,\n                    expires: expiry_times[&b\"hij\"],\n                },\n            ),\n        ]\n        .into_iter()\n        .enumerate()\n        {\n            let ct = pos == 0;\n            let hash = BlobHash::generate(blob.as_slice());\n            assert!(store.blob_has_access(&hash, blob_class).await.unwrap() ^ ct);\n            assert!(store.blob_exists(&hash).await.unwrap() ^ ct);\n            assert!(\n                blob_store\n                    .get_blob(hash.as_ref(), 0..usize::MAX)\n                    .await\n                    .unwrap()\n                    .is_some()\n                    ^ ct\n            );\n        }\n    }\n    temp_dir.delete();\n}\n\nasync fn test_store(store: BlobStore) {\n    // Test small blob\n    const DATA: &[u8] = b\"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce erat nisl, dignissim a porttitor id, varius nec arcu. Sed mauris.\";\n    let hash = BlobHash::generate(DATA);\n\n    store.put_blob(hash.as_slice(), DATA).await.unwrap();\n    assert_eq!(\n        String::from_utf8(\n            store\n                .get_blob(hash.as_slice(), 0..usize::MAX)\n                .await\n                .unwrap()\n                .unwrap()\n        )\n        .unwrap(),\n        std::str::from_utf8(DATA).unwrap()\n    );\n    assert_eq!(\n        String::from_utf8(\n            store\n                .get_blob(hash.as_slice(), 11..57)\n                .await\n                .unwrap()\n                .unwrap()\n        )\n        .unwrap(),\n        std::str::from_utf8(&DATA[11..57]).unwrap()\n    );\n    assert!(store.delete_blob(hash.as_slice()).await.unwrap());\n    assert!(\n        store\n            .get_blob(hash.as_slice(), 0..usize::MAX)\n            .await\n            .unwrap()\n            .is_none()\n    );\n\n    // Test large blob\n    let mut data = Vec::with_capacity(50 * 1024 * 1024);\n    while data.len() < 50 * 1024 * 1024 {\n        data.extend_from_slice(DATA);\n        let marker = format!(\" [{}] \", data.len());\n        data.extend_from_slice(marker.as_bytes());\n    }\n    let hash = BlobHash::generate(&data);\n    store.put_blob(hash.as_slice(), &data).await.unwrap();\n    assert_eq!(\n        String::from_utf8(\n            store\n                .get_blob(hash.as_slice(), 0..usize::MAX)\n                .await\n                .unwrap()\n                .unwrap()\n        )\n        .unwrap(),\n        std::str::from_utf8(&data).unwrap()\n    );\n\n    assert_eq!(\n        String::from_utf8(\n            store\n                .get_blob(hash.as_slice(), 3000111..4000999)\n                .await\n                .unwrap()\n                .unwrap()\n        )\n        .unwrap(),\n        std::str::from_utf8(&data[3000111..4000999]).unwrap()\n    );\n    assert!(store.delete_blob(hash.as_slice()).await.unwrap());\n    assert!(\n        store\n            .get_blob(hash.as_slice(), 0..usize::MAX)\n            .await\n            .unwrap()\n            .is_none()\n    );\n}\n"
  },
  {
    "path": "tests/src/store/cleanup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse store::{\n    ValueKey,\n    write::{key::DeserializeBigEndian, *},\n    *,\n};\nuse trc::AddContext;\nuse types::blob_hash::{BLOB_HASH_LEN, BlobHash};\n\npub async fn store_destroy(store: &Store) {\n    store_destroy_sql_indexes(store).await;\n\n    for subspace in [\n        SUBSPACE_ACL,\n        SUBSPACE_DIRECTORY,\n        SUBSPACE_TASK_QUEUE,\n        SUBSPACE_INDEXES,\n        SUBSPACE_BLOB_EXTRA,\n        SUBSPACE_BLOB_LINK,\n        SUBSPACE_LOGS,\n        SUBSPACE_IN_MEMORY_COUNTER,\n        SUBSPACE_IN_MEMORY_VALUE,\n        SUBSPACE_COUNTER,\n        SUBSPACE_PROPERTY,\n        SUBSPACE_SETTINGS,\n        SUBSPACE_BLOBS,\n        SUBSPACE_QUEUE_MESSAGE,\n        SUBSPACE_QUEUE_EVENT,\n        SUBSPACE_QUOTA,\n        SUBSPACE_REPORT_OUT,\n        SUBSPACE_REPORT_IN,\n        SUBSPACE_TELEMETRY_SPAN,\n        SUBSPACE_TELEMETRY_METRIC,\n        SUBSPACE_SEARCH_INDEX,\n    ] {\n        if subspace == SUBSPACE_SEARCH_INDEX && store.is_pg_or_mysql() {\n            continue;\n        }\n\n        store\n            .delete_range(\n                AnyKey {\n                    subspace,\n                    key: vec![0u8],\n                },\n                AnyKey {\n                    subspace,\n                    key: vec![u8::MAX; 16],\n                },\n            )\n            .await\n            .unwrap();\n    }\n}\n\npub async fn search_store_destroy(store: &SearchStore) {\n    match &store {\n        SearchStore::Store(store) => {\n            store_destroy_sql_indexes(store).await;\n        }\n        SearchStore::ElasticSearch(store) => {\n            if let Err(err) = store.drop_indexes().await {\n                eprintln!(\"Failed to drop elasticsearch indexes: {}\", err);\n            }\n            store.create_indexes(3, 0, false).await.unwrap();\n        }\n        SearchStore::MeiliSearch(store) => {\n            if let Err(err) = store.drop_indexes().await {\n                eprintln!(\"Failed to drop meilisearch indexes: {}\", err);\n            }\n            store.create_indexes().await.unwrap();\n        }\n    }\n}\n\n#[allow(unused_variables)]\nasync fn store_destroy_sql_indexes(store: &Store) {\n    #[cfg(any(feature = \"postgres\", feature = \"mysql\"))]\n    {\n        if store.is_pg_or_mysql() {\n            for index in [\n                SearchIndex::Email,\n                SearchIndex::Calendar,\n                SearchIndex::Contacts,\n                SearchIndex::Tracing,\n            ] {\n                #[cfg(feature = \"postgres\")]\n                let table = index.psql_table();\n                #[cfg(feature = \"mysql\")]\n                let table = index.mysql_table();\n\n                store\n                    .sql_query::<usize>(&format!(\"TRUNCATE TABLE {table}\"), vec![])\n                    .await\n                    .unwrap();\n            }\n        }\n    }\n}\n\npub async fn store_blob_expire_all(store: &Store) {\n    // Delete all temporary hashes\n    let from_key = ValueKey {\n        account_id: 0,\n        collection: 0,\n        document_id: 0,\n        class: ValueClass::Blob(BlobOp::Commit {\n            hash: BlobHash::default(),\n        }),\n    };\n    let to_key = ValueKey {\n        account_id: u32::MAX,\n        collection: u8::MAX,\n        document_id: u32::MAX,\n        class: ValueClass::Blob(BlobOp::Link {\n            hash: BlobHash::new_max(),\n            to: BlobLink::Document,\n        }),\n    };\n    let mut batch = BatchBuilder::new();\n    let mut last_account_id = u32::MAX;\n    store\n        .iterate(\n            IterateParams::new(from_key, to_key).ascending(),\n            |key, value| {\n                if key.len() == BLOB_HASH_LEN + U32_LEN + U64_LEN {\n                    let account_id = key\n                        .deserialize_be_u32(BLOB_HASH_LEN)\n                        .caused_by(trc::location!())?;\n                    if account_id != last_account_id {\n                        last_account_id = account_id;\n                        batch.with_account_id(account_id);\n                    }\n                    let hash =\n                        BlobHash::try_from_hash_slice(key.get(..BLOB_HASH_LEN).unwrap()).unwrap();\n                    let until = key\n                        .deserialize_be_u64(BLOB_HASH_LEN + U32_LEN)\n                        .caused_by(trc::location!())?;\n\n                    match value.first().copied() {\n                        Some(BlobLink::QUOTA_LINK) => {\n                            batch.clear(ValueClass::Blob(BlobOp::Quota {\n                                hash: hash.clone(),\n                                until,\n                            }));\n                        }\n                        Some(BlobLink::UNDELETE_LINK) => {\n                            batch.clear(ValueClass::Blob(BlobOp::Undelete {\n                                hash: hash.clone(),\n                                until,\n                            }));\n                        }\n                        Some(BlobLink::SPAM_SAMPLE_LINK) => {\n                            batch.clear(ValueClass::Blob(BlobOp::SpamSample {\n                                hash: hash.clone(),\n                                until,\n                            }));\n                        }\n                        _ => {}\n                    }\n\n                    batch.clear(ValueClass::Blob(BlobOp::Link {\n                        hash,\n                        to: BlobLink::Temporary { until },\n                    }));\n                }\n\n                Ok(true)\n            },\n        )\n        .await\n        .unwrap();\n    store.write(batch.build_all()).await.unwrap();\n}\n\npub async fn store_lookup_expire_all(store: &Store) {\n    // Delete all temporary counters\n    let from_key = ValueKey::from(ValueClass::InMemory(InMemoryClass::Key(vec![0u8])));\n    let to_key = ValueKey::from(ValueClass::InMemory(InMemoryClass::Key(vec![u8::MAX; 10])));\n\n    let mut expired_keys = Vec::new();\n    let mut expired_counters = Vec::new();\n\n    store\n        .iterate(IterateParams::new(from_key, to_key), |key, value| {\n            let expiry = value.deserialize_be_u64(0).caused_by(trc::location!())?;\n            if expiry == 0 {\n                expired_counters.push(key.to_vec());\n            } else if expiry != u64::MAX {\n                expired_keys.push(key.to_vec());\n            }\n            Ok(true)\n        })\n        .await\n        .unwrap();\n\n    if !expired_keys.is_empty() {\n        let mut batch = BatchBuilder::new();\n        for key in expired_keys {\n            batch.any_op(Operation::Value {\n                class: ValueClass::InMemory(InMemoryClass::Key(key)),\n                op: ValueOp::Clear,\n            });\n            if batch.is_large_batch() {\n                store.write(batch.build_all()).await.unwrap();\n                batch = BatchBuilder::new();\n            }\n        }\n        if !batch.is_empty() {\n            store.write(batch.build_all()).await.unwrap();\n        }\n    }\n\n    if !expired_counters.is_empty() {\n        let mut batch = BatchBuilder::new();\n        for key in expired_counters {\n            batch.any_op(Operation::Value {\n                class: ValueClass::InMemory(InMemoryClass::Counter(key.clone())),\n                op: ValueOp::Clear,\n            });\n            batch.any_op(Operation::Value {\n                class: ValueClass::InMemory(InMemoryClass::Key(key)),\n                op: ValueOp::Clear,\n            });\n            if batch.is_large_batch() {\n                store.write(batch.build_all()).await.unwrap();\n                batch = BatchBuilder::new();\n            }\n        }\n        if !batch.is_empty() {\n            store.write(batch.build_all()).await.unwrap();\n        }\n    }\n}\n\n#[allow(unused_variables)]\npub async fn store_assert_is_empty(store: &Store, blob_store: BlobStore, include_directory: bool) {\n    store_blob_expire_all(store).await;\n    store_lookup_expire_all(store).await;\n    store.purge_blobs(blob_store).await.unwrap();\n    store.purge_store().await.unwrap();\n\n    let store = store.clone();\n    let mut failed = false;\n\n    for (subspace, with_values) in [\n        (SUBSPACE_ACL, true),\n        (SUBSPACE_DIRECTORY, true),\n        (SUBSPACE_TASK_QUEUE, true),\n        (SUBSPACE_IN_MEMORY_VALUE, true),\n        (SUBSPACE_IN_MEMORY_COUNTER, false),\n        (SUBSPACE_PROPERTY, true),\n        (SUBSPACE_SETTINGS, true),\n        (SUBSPACE_QUEUE_MESSAGE, true),\n        (SUBSPACE_QUEUE_EVENT, true),\n        (SUBSPACE_REPORT_OUT, true),\n        (SUBSPACE_REPORT_IN, true),\n        (SUBSPACE_BLOB_EXTRA, true),\n        (SUBSPACE_BLOB_LINK, true),\n        (SUBSPACE_BLOBS, true),\n        (SUBSPACE_COUNTER, false),\n        (SUBSPACE_QUOTA, false),\n        (SUBSPACE_INDEXES, false),\n        (SUBSPACE_TELEMETRY_SPAN, true),\n        (SUBSPACE_TELEMETRY_METRIC, true),\n        (SUBSPACE_SEARCH_INDEX, true),\n    ] {\n        if (subspace == SUBSPACE_SEARCH_INDEX && store.is_pg_or_mysql())\n            || (subspace == SUBSPACE_DIRECTORY && !include_directory)\n        {\n            continue;\n        }\n\n        let from_key = AnyKey {\n            subspace,\n            key: vec![0u8],\n        };\n        let to_key = AnyKey {\n            subspace,\n            key: vec![u8::MAX; 10],\n        };\n\n        store\n            .iterate(\n                IterateParams::new(from_key, to_key).set_values(with_values),\n                |key, value| {\n                    match subspace {\n                        SUBSPACE_COUNTER if key.len() == U32_LEN + 1 || key.len() == U32_LEN => {\n                            // Message ID and change ID counters\n                            return Ok(true);\n                        }\n                        SUBSPACE_INDEXES => {\n                            println!(\n                                concat!(\n                                    \"Found index key, account {}, collection {}, \",\n                                    \"document {}, property {}, value {:?}: {:?}\"\n                                ),\n                                u32::from_be_bytes(key[0..4].try_into().unwrap()),\n                                key[4],\n                                u32::from_be_bytes(key[key.len() - 4..].try_into().unwrap()),\n                                key[5],\n                                String::from_utf8_lossy(&key[6..key.len() - 4]),\n                                key\n                            );\n                        }\n                        _ => {\n                            println!(\n                                \"Found key in {:?}: {:?} ({:?}) = {:?} ({:?})\",\n                                char::from(subspace),\n                                key,\n                                String::from_utf8_lossy(key),\n                                value,\n                                String::from_utf8_lossy(value)\n                            );\n                        }\n                    }\n                    failed = true;\n\n                    Ok(true)\n                },\n            )\n            .await\n            .unwrap();\n    }\n\n    // Delete logs and counters\n    store\n        .delete_range(\n            AnyKey {\n                subspace: SUBSPACE_LOGS,\n                key: &[0u8],\n            },\n            AnyKey {\n                subspace: SUBSPACE_LOGS,\n                key: &[\n                    u8::MAX,\n                    u8::MAX,\n                    u8::MAX,\n                    u8::MAX,\n                    u8::MAX,\n                    u8::MAX,\n                    u8::MAX,\n                ],\n            },\n        )\n        .await\n        .unwrap();\n\n    store\n        .delete_range(\n            AnyKey {\n                subspace: SUBSPACE_COUNTER,\n                key: &[0u8],\n            },\n            AnyKey {\n                subspace: SUBSPACE_COUNTER,\n                key: (u32::MAX / 2).to_be_bytes().as_slice(),\n            },\n        )\n        .await\n        .unwrap();\n\n    if failed {\n        panic!(\"Store is not empty.\");\n    }\n}\n"
  },
  {
    "path": "tests/src/store/import_export.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::store::{\n    TempDir,\n    cleanup::{store_assert_is_empty, store_destroy},\n};\nuse ahash::AHashSet;\nuse common::{Core, DATABASE_SCHEMA_VERSION, manager::backup::BackupParams};\nuse store::{\n    rand,\n    write::{\n        AnyClass, AnyKey, BatchBuilder, BlobLink, BlobOp, DirectoryClass, Operation, QueueClass,\n        QueueEvent, ValueClass,\n    },\n    *,\n};\nuse types::{\n    blob_hash::BlobHash,\n    collection::{Collection, SyncCollection},\n    field::{Field, MailboxField},\n};\n\npub async fn test(db: Store) {\n    let mut core = Core::default();\n    core.storage.data = db.clone();\n    core.storage.blob = db.clone().into();\n    core.storage.fts = db.clone().into();\n    core.storage.lookup = db.clone().into();\n\n    // Make sure the store is empty\n    store_assert_is_empty(&db, db.clone().into(), true).await;\n\n    // Create blobs\n    println!(\"Creating blobs...\");\n    let mut batch = BatchBuilder::new();\n    batch.set(\n        ValueClass::Any(AnyClass {\n            subspace: SUBSPACE_PROPERTY,\n            key: vec![0u8],\n        }),\n        DATABASE_SCHEMA_VERSION.serialize(),\n    );\n    let mut blob_hashes = Vec::new();\n    for blob_size in [16, 128, 1024, 2056, 102400] {\n        let data = random_bytes(blob_size);\n        let hash = BlobHash::generate(data.as_slice());\n        blob_hashes.push(hash.clone());\n        core.storage\n            .blob\n            .put_blob(hash.as_ref(), &data)\n            .await\n            .unwrap();\n        batch.set(ValueClass::Blob(BlobOp::Commit { hash }), vec![]);\n    }\n    db.write(batch.build_all()).await.unwrap();\n\n    // Create account data\n    println!(\"Creating account data...\");\n    for account_id in 0u32..10u32 {\n        let mut batch = BatchBuilder::new();\n        batch.with_account_id(account_id);\n\n        // Create properties of different sizes\n        for collection in [\n            Collection::Email,\n            Collection::Mailbox,\n            Collection::Thread,\n            Collection::Identity,\n        ] {\n            batch.with_collection(collection);\n\n            for document_id in [0, 10, 20, 30, 40] {\n                batch.with_document(document_id);\n\n                if collection == Collection::Mailbox {\n                    batch\n                        .set(\n                            ValueClass::Property(Field::ARCHIVE.into()),\n                            random_bytes(10),\n                        )\n                        .add(\n                            ValueClass::Property(MailboxField::UidCounter.into()),\n                            rand::random(),\n                        );\n                }\n\n                for (idx, value_size) in [16, 128, 1024, 2056, 102400].into_iter().enumerate() {\n                    batch.set(ValueClass::Property(idx as u8), random_bytes(value_size));\n                }\n\n                for grant_account_id in 0u32..10u32 {\n                    if account_id != grant_account_id {\n                        batch.set(\n                            ValueClass::Acl(grant_account_id),\n                            vec![account_id as u8, grant_account_id as u8, document_id as u8],\n                        );\n                    }\n                }\n\n                for hash in &blob_hashes {\n                    batch.set(\n                        ValueClass::Blob(BlobOp::Link {\n                            hash: hash.clone(),\n                            to: BlobLink::Document,\n                        }),\n                        vec![],\n                    );\n                }\n\n                batch.log_item_insert(SyncCollection::from(collection), None);\n\n                for field in 0..5 {\n                    batch.any_op(Operation::Index {\n                        field,\n                        key: random_bytes(field as usize + 2),\n                        set: true,\n                    });\n                }\n            }\n        }\n\n        db.write(batch.build_all()).await.unwrap();\n    }\n\n    // Create queue, config and lookup data\n    println!(\"Creating queue, config and lookup data...\");\n    let mut batch = BatchBuilder::new();\n    for idx in [1, 2, 3, 4, 5] {\n        batch.set(\n            ValueClass::Queue(QueueClass::Message(rand::random())),\n            random_bytes(idx),\n        );\n        batch.set(\n            ValueClass::Queue(QueueClass::MessageEvent(QueueEvent {\n                due: rand::random(),\n                queue_id: rand::random(),\n                queue_name: rand::random(),\n            })),\n            random_bytes(idx),\n        );\n        /*batch.set(\n            ValueClass::InMemory(InMemoryClass::Key(random_bytes(idx))),\n            random_bytes(idx),\n        );\n        batch.add(\n            ValueClass::InMemory(InMemoryClass::Counter(random_bytes(idx))),\n            rand::random(),\n        );*/\n        batch.set(\n            ValueClass::Config(random_bytes(idx + 10)),\n            random_bytes(idx + 10),\n        );\n    }\n    db.write(batch.build_all()).await.unwrap();\n\n    // Create directory data\n    println!(\"Creating directory data...\");\n    let mut batch = BatchBuilder::new();\n    batch\n        .with_account_id(u32::MAX)\n        .with_collection(Collection::Principal);\n\n    for account_id in [1, 2, 3, 4, 5] {\n        batch\n            .with_document(account_id)\n            .add(\n                ValueClass::Directory(DirectoryClass::UsedQuota(account_id)),\n                rand::random(),\n            )\n            .set(\n                ValueClass::Directory(DirectoryClass::NameToId(random_bytes(\n                    2 + account_id as usize,\n                ))),\n                random_bytes(4),\n            )\n            .set(\n                ValueClass::Directory(DirectoryClass::EmailToId(random_bytes(\n                    4 + account_id as usize,\n                ))),\n                random_bytes(4),\n            )\n            .set(\n                ValueClass::Directory(DirectoryClass::Principal(account_id)),\n                random_bytes(30),\n            )\n            .set(\n                ValueClass::Directory(DirectoryClass::MemberOf {\n                    principal_id: account_id,\n                    member_of: rand::random(),\n                }),\n                random_bytes(15),\n            )\n            .set(\n                ValueClass::Directory(DirectoryClass::Members {\n                    principal_id: account_id,\n                    has_member: rand::random(),\n                }),\n                random_bytes(15),\n            );\n    }\n    db.write(batch.build_all()).await.unwrap();\n\n    // Obtain store hash\n    println!(\"Calculating store hash...\");\n    let snapshot = Snapshot::new(&db).await;\n    assert!(!snapshot.keys.is_empty(), \"Store hash counts are empty\",);\n\n    // Export store\n    println!(\"Exporting store...\");\n    let temp_dir = TempDir::new(\"art_vandelay_tests\", true);\n    core.backup(BackupParams::new(temp_dir.path.clone())).await;\n\n    // Destroy store\n    println!(\"Destroying store...\");\n    store_destroy(&db).await;\n    store_assert_is_empty(&db, db.clone().into(), true).await;\n\n    // Import store\n    println!(\"Importing store...\");\n    core.restore(temp_dir.path.clone()).await;\n\n    // Verify hash\n    print!(\"Verifying store hash...\");\n    snapshot.assert_is_eq(&Snapshot::new(&db).await);\n    println!(\" GREAT SUCCESS!\");\n\n    // Destroy store\n    store_destroy(&db).await;\n    store_assert_is_empty(&db, db.clone().into(), true).await;\n    temp_dir.delete();\n}\n\n#[derive(Debug, PartialEq, Eq)]\nstruct Snapshot {\n    keys: AHashSet<KeyValue>,\n}\n\n#[derive(Debug, PartialEq, Eq, Hash)]\nstruct KeyValue {\n    subspace: u8,\n    key: Vec<u8>,\n    value: Vec<u8>,\n}\n\nimpl Snapshot {\n    async fn new(db: &Store) -> Self {\n        let is_sql = db.is_sql();\n\n        let mut keys = AHashSet::new();\n\n        for (subspace, with_values) in [\n            (SUBSPACE_ACL, true),\n            (SUBSPACE_DIRECTORY, true),\n            (SUBSPACE_TASK_QUEUE, true),\n            (SUBSPACE_INDEXES, false),\n            (SUBSPACE_BLOB_EXTRA, true),\n            (SUBSPACE_BLOB_LINK, true),\n            (SUBSPACE_BLOBS, true),\n            (SUBSPACE_LOGS, true),\n            (SUBSPACE_COUNTER, !is_sql),\n            (SUBSPACE_IN_MEMORY_COUNTER, !is_sql),\n            (SUBSPACE_IN_MEMORY_VALUE, true),\n            (SUBSPACE_PROPERTY, true),\n            (SUBSPACE_SETTINGS, true),\n            (SUBSPACE_QUEUE_MESSAGE, true),\n            (SUBSPACE_QUEUE_EVENT, true),\n            (SUBSPACE_QUOTA, !is_sql),\n            (SUBSPACE_REPORT_OUT, true),\n            (SUBSPACE_REPORT_IN, true),\n        ] {\n            let from_key = AnyKey {\n                subspace,\n                key: vec![0u8],\n            };\n            let to_key = AnyKey {\n                subspace,\n                key: vec![u8::MAX; 10],\n            };\n\n            db.iterate(\n                IterateParams::new(from_key, to_key).set_values(with_values),\n                |key, value| {\n                    keys.insert(KeyValue {\n                        subspace,\n                        key: key.to_vec(),\n                        value: value.to_vec(),\n                    });\n\n                    Ok(true)\n                },\n            )\n            .await\n            .unwrap();\n        }\n\n        Snapshot { keys }\n    }\n\n    fn assert_is_eq(&self, other: &Self) {\n        let mut is_err = false;\n        for key in &self.keys {\n            if !other.keys.contains(key) {\n                println!(\n                    \"Subspace {}, Key {:?} not found in restored snapshot\",\n                    char::from(key.subspace),\n                    key.key,\n                );\n                is_err = true;\n            }\n        }\n        for key in &other.keys {\n            if !self.keys.contains(key) {\n                println!(\n                    \"Subspace {}, Key {:?} not found in original snapshot\",\n                    char::from(key.subspace),\n                    key.key,\n                );\n                is_err = true;\n            }\n        }\n\n        if is_err {\n            panic!(\"Snapshot mismatch\");\n        }\n    }\n}\n\nfn random_bytes(len: usize) -> Vec<u8> {\n    (0..len).map(|_| rand::random::<u8>()).collect()\n}\n"
  },
  {
    "path": "tests/src/store/lookup.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse std::time::Duration;\n\nuse store::{InMemoryStore, Stores, dispatch::lookup::KeyValue};\nuse utils::config::{Config, Rate};\n\nuse crate::{\n    AssertConfig,\n    store::{\n        CONFIG, TempDir,\n        cleanup::{store_assert_is_empty, store_destroy},\n    },\n};\n\n#[tokio::test]\npub async fn lookup_tests() {\n    let temp_dir = TempDir::new(\"lookup_tests\", true);\n    let mut config =\n        Config::new(CONFIG.replace(\"{TMP}\", temp_dir.path.as_path().to_str().unwrap()))\n            .unwrap()\n            .assert_no_errors();\n    let stores = Stores::parse_all(&mut config, false).await;\n    let rate = Rate {\n        requests: 1,\n        period: Duration::from_secs(1),\n    };\n\n    for (store_id, store) in stores.in_memory_stores {\n        println!(\"Testing in-memory store {}...\", store_id);\n        if let InMemoryStore::Store(store) = &store {\n            store_destroy(store).await;\n        } else {\n            // Reset redis counter\n            store\n                .key_set(KeyValue::new(\"abc\", \"0\".as_bytes().to_vec()))\n                .await\n                .unwrap();\n        }\n\n        // Test key\n        let key = \"xyz\".as_bytes().to_vec();\n        store\n            .key_set(KeyValue::new(key.clone(), \"world\".to_string().into_bytes()))\n            .await\n            .unwrap();\n        store.purge_in_memory_store().await.unwrap();\n        assert_eq!(\n            store.key_get::<String>(key.clone()).await.unwrap(),\n            Some(\"world\".to_string())\n        );\n\n        // Test value expiry\n        store\n            .key_set(KeyValue::new(key.clone(), \"hello\".to_string().into_bytes()).expires(1))\n            .await\n            .unwrap();\n        assert_eq!(\n            store.key_get::<String>(key.clone()).await.unwrap(),\n            Some(\"hello\".to_string())\n        );\n        tokio::time::sleep(tokio::time::Duration::from_secs(2)).await;\n        assert_eq!(None, store.key_get::<String>(key.clone()).await.unwrap());\n\n        store.purge_in_memory_store().await.unwrap();\n        if let InMemoryStore::Store(store) = &store {\n            store_assert_is_empty(store, store.clone().into(), false).await;\n        }\n\n        // Test counter\n        let key = \"abc\".as_bytes().to_vec();\n        store\n            .counter_incr(KeyValue::new(key.clone(), 1), true)\n            .await\n            .unwrap();\n        assert_eq!(1, store.counter_get(key.clone()).await.unwrap());\n        store\n            .counter_incr(KeyValue::new(key.clone(), 2), true)\n            .await\n            .unwrap();\n        assert_eq!(3, store.counter_get(key.clone()).await.unwrap());\n        store\n            .counter_incr(KeyValue::new(key.clone(), -3), false)\n            .await\n            .unwrap();\n        assert_eq!(0, store.counter_get(key.clone()).await.unwrap());\n\n        // Test counter expiry\n        let key = \"fgh\".as_bytes().to_vec();\n        store\n            .counter_incr(KeyValue::new(key.clone(), 1).expires(1), false)\n            .await\n            .unwrap();\n        assert_eq!(1, store.counter_get(key.clone()).await.unwrap());\n        tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;\n        store.purge_in_memory_store().await.unwrap();\n        assert_eq!(0, store.counter_get(key.clone()).await.unwrap());\n\n        // Test rate limiter\n        assert!(\n            store\n                .is_rate_allowed(0, \"rate\".as_bytes(), &rate, false)\n                .await\n                .unwrap()\n                .is_none()\n        );\n        assert!(\n            store\n                .is_rate_allowed(0, \"rate\".as_bytes(), &rate, false)\n                .await\n                .unwrap()\n                .is_some()\n        );\n        tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;\n        assert!(\n            store\n                .is_rate_allowed(0, \"rate\".as_bytes(), &rate, false)\n                .await\n                .unwrap()\n                .is_none()\n        );\n        tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;\n        store.purge_in_memory_store().await.unwrap();\n        if let InMemoryStore::Store(store) = &store {\n            store_assert_is_empty(store, store.clone().into(), false).await;\n        }\n\n        // Test locking\n        for iteration in [1, 2] {\n            let mut tasks = Vec::new();\n            for _ in 0..100 {\n                let store = store.clone();\n                tasks.push(tokio::spawn(async move {\n                    store.try_lock(0, \"lock\".as_bytes(), 1).await.unwrap()\n                }));\n            }\n            // Only one should return true\n            let mut count = 0;\n            for task in tasks {\n                if task.await.unwrap() {\n                    count += 1;\n                }\n            }\n            assert_eq!(1, count, \"Iteration {}\", iteration);\n\n            // Wait 2 seconds for the lock to expire\n            tokio::time::sleep(tokio::time::Duration::from_secs(2)).await;\n        }\n        store.purge_in_memory_store().await.unwrap();\n        if let InMemoryStore::Store(store) = &store {\n            store_assert_is_empty(store, store.clone().into(), false).await;\n        }\n\n        // Test prefix delete\n        store\n            .key_set(KeyValue::with_prefix(\n                1,\n                [0],\n                \"hello\".to_string().into_bytes(),\n            ))\n            .await\n            .unwrap();\n        for v in 0u32..2020u32 {\n            store\n                .key_set(KeyValue::with_prefix(\n                    0,\n                    pack_u32(0, v),\n                    \"world\".to_string().into_bytes(),\n                ))\n                .await\n                .unwrap();\n            store\n                .counter_incr(\n                    KeyValue::with_prefix(0, pack_u32(1, v), 123).expires(3600),\n                    false,\n                )\n                .await\n                .unwrap();\n        }\n\n        // Make sure the keys are there\n        assert_eq!(\n            Some(\"hello\"),\n            store\n                .key_get::<String>(KeyValue::<()>::build_key(1, [0]))\n                .await\n                .unwrap()\n                .as_deref()\n        );\n        for v in [0, 1000, 1001, 2000, 2001] {\n            assert_eq!(\n                Some(\"world\"),\n                store\n                    .key_get::<String>(KeyValue::<()>::build_key(0, pack_u32(0, v)))\n                    .await\n                    .unwrap()\n                    .as_deref()\n            );\n        }\n        for v in [0, 1000, 1001, 2000, 2001] {\n            assert_ne!(\n                0,\n                store\n                    .counter_get(KeyValue::<()>::build_key(0, pack_u32(1, v)))\n                    .await\n                    .unwrap()\n            );\n        }\n\n        // Delete [0, 0, 0, 0, 1] prefix and make sure only the keys with that prefix are gone\n        store\n            .key_delete_prefix(&KeyValue::<()>::build_key(0, 1u32.to_be_bytes()))\n            .await\n            .unwrap();\n\n        assert_eq!(\n            Some(\"hello\"),\n            store\n                .key_get::<String>(KeyValue::<()>::build_key(1, [0]))\n                .await\n                .unwrap()\n                .as_deref()\n        );\n        for v in [0, 1000, 1001, 2000, 2001] {\n            assert_eq!(\n                Some(\"world\"),\n                store\n                    .key_get::<String>(KeyValue::<()>::build_key(0, pack_u32(0, v)))\n                    .await\n                    .unwrap()\n                    .as_deref()\n            );\n        }\n\n        for v in [0, 1000, 1001, 2000, 2001] {\n            assert_eq!(\n                0,\n                store\n                    .counter_get(KeyValue::<()>::build_key(0, pack_u32(1, v)))\n                    .await\n                    .unwrap()\n            );\n        }\n\n        // Delete [0, 0, 0, 0, 0] prefix and make sure only the keys with that prefix are gone\n        store\n            .key_delete_prefix(&KeyValue::<()>::build_key(0, 0u32.to_be_bytes()))\n            .await\n            .unwrap();\n\n        assert_eq!(\n            Some(\"hello\"),\n            store\n                .key_get::<String>(KeyValue::<()>::build_key(1, [0]))\n                .await\n                .unwrap()\n                .as_deref()\n        );\n        for v in [0, 1000, 1001, 2000, 2001] {\n            assert_eq!(\n                None,\n                store\n                    .key_get::<String>(KeyValue::<()>::build_key(0, pack_u32(0, v)))\n                    .await\n                    .unwrap()\n                    .as_deref()\n            );\n        }\n\n        // Delete [1, ...] prefix and make sure it's all gone\n        store.key_delete_prefix(&[1u8]).await.unwrap();\n\n        assert_eq!(\n            None,\n            store\n                .key_get::<String>(KeyValue::<()>::build_key(1, [0]))\n                .await\n                .unwrap()\n                .as_deref()\n        );\n\n        if let InMemoryStore::Store(store) = &store {\n            store_assert_is_empty(store, store.clone().into(), false).await;\n        }\n    }\n}\n\nfn pack_u32(a: u32, b: u32) -> Vec<u8> {\n    (((a as u64) << 32) | b as u64).to_be_bytes().to_vec()\n}\n"
  },
  {
    "path": "tests/src/store/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\npub mod blob;\npub mod cleanup;\npub mod import_export;\npub mod lookup;\npub mod ops;\npub mod query;\n\nuse crate::{\n    AssertConfig,\n    store::cleanup::{search_store_destroy, store_destroy},\n};\nuse std::io::Read;\nuse store::Stores;\nuse utils::config::Config;\n\npub struct TempDir {\n    pub path: std::path::PathBuf,\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\npub async fn store_tests() {\n    let insert = true;\n    let temp_dir = TempDir::new(\"store_tests\", insert);\n    let mut config = Config::new(build_store_config(&temp_dir.path.to_string_lossy()))\n        .unwrap()\n        .assert_no_errors();\n    let stores = Stores::parse_all(&mut config, false).await;\n\n    let store_id = std::env::var(\"STORE\")\n        .expect(\"Missing store type. Try running `STORE=<store_type> cargo test`\");\n    let store = stores\n        .stores\n        .get(&store_id)\n        .expect(\"Store not found\")\n        .clone();\n\n    println!(\"Testing store {}...\", store_id);\n    if insert {\n        store_destroy(&store).await;\n    }\n\n    import_export::test(store.clone()).await;\n    ops::test(store.clone()).await;\n\n    if insert {\n        temp_dir.delete();\n    }\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\npub async fn search_tests() {\n    let insert = std::env::var(\"NO_INSERT\").is_err();\n    let temp_dir = TempDir::new(\"search_store_tests\", insert);\n    let mut config = Config::new(build_store_config(&temp_dir.path.to_string_lossy()))\n        .unwrap()\n        .assert_no_errors();\n    let stores = Stores::parse_all(&mut config, false).await;\n\n    let store_id = std::env::var(\"SEARCH_STORE\")\n        .expect(\"Missing store type. Try running `SEARCH_STORE=<store_type> cargo test`\");\n    let store = stores\n        .search_stores\n        .get(&store_id)\n        .expect(\"Store not found\")\n        .clone();\n\n    println!(\"Testing store {}...\", store_id);\n    if insert {\n        search_store_destroy(&store).await;\n    }\n\n    query::test(store, insert).await;\n\n    if insert {\n        temp_dir.delete();\n    }\n}\n\npub fn deflate_test_resource(name: &str) -> Vec<u8> {\n    let mut csv_path = std::path::PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"));\n    csv_path.push(\"resources\");\n    csv_path.push(name);\n\n    let mut decoder = flate2::bufread::GzDecoder::new(std::io::BufReader::new(\n        std::fs::File::open(csv_path).unwrap(),\n    ));\n    let mut result = Vec::new();\n    decoder.read_to_end(&mut result).unwrap();\n    result\n}\n\nimpl TempDir {\n    pub fn new(name: &str, delete_if_exists: bool) -> Self {\n        let mut path = std::env::temp_dir();\n        path.push(name);\n        if delete_if_exists && path.exists() {\n            std::fs::remove_dir_all(&path).unwrap();\n        }\n        std::fs::create_dir_all(&path).unwrap();\n        Self { path }\n    }\n\n    pub fn delete(&self) {\n        std::fs::remove_dir_all(&self.path).unwrap();\n    }\n}\n\npub fn build_store_config(temp_dir: &str) -> String {\n    let store = std::env::var(\"STORE\")\n        .expect(\"Missing store type. Try running `STORE=<store_type> cargo test`\");\n    let fts_store = std::env::var(\"SEARCH_STORE\").unwrap_or_else(|_| store.clone());\n    let blob_store = std::env::var(\"BLOB_STORE\").unwrap_or_else(|_| store.clone());\n    let lookup_store = std::env::var(\"LOOKUP_STORE\").unwrap_or_else(|_| store.clone());\n\n    CONFIG\n        .replace(\"{STORE}\", &store)\n        .replace(\"{SEARCH_STORE}\", &fts_store)\n        .replace(\"{BLOB_STORE}\", &blob_store)\n        .replace(\"{LOOKUP_STORE}\", &lookup_store)\n        .replace(\"{TMP}\", temp_dir)\n        .replace(\n            \"{ELASTIC_ENABLED}\",\n            if fts_store != \"elastic\" {\n                \"true\"\n            } else {\n                \"false\"\n            },\n        )\n        .replace(\n            \"{MEILI_ENABLED}\",\n            if fts_store != \"meili\" {\n                \"true\"\n            } else {\n                \"false\"\n            },\n        )\n}\n\nconst CONFIG: &str = r#\"\n[store.\"sqlite\"]\ntype = \"sqlite\"\npath = \"{TMP}/sqlite.db\"\n\n[store.\"rocksdb\"]\ntype = \"rocksdb\"\npath = \"{TMP}/rocks.db\"\n\n[store.\"foundationdb\"]\ntype = \"foundationdb\"\n\n[store.\"postgresql\"]\ntype = \"postgresql\"\nhost = \"localhost\"\nport = 5432\ndatabase = \"stalwart\"\nuser = \"postgres\"\npassword = \"mysecretpassword\"\n\n[store.\"mysql\"]\ntype = \"mysql\"\nhost = \"localhost\"\nport = 3307\ndatabase = \"stalwart\"\nuser = \"root\"\npassword = \"password\"\n\n[store.\"elastic\"]\ntype = \"elasticsearch\"\nurl = \"https://localhost:9200\"\ntls.allow-invalid-certs = true\ndisable = {ELASTIC_ENABLED}\n[store.\"elastic\".auth]\nusername = \"elastic\"\nsecret = \"changeme\"\n\n[store.\"meili\"]\ntype = \"meilisearch\"\nurl = \"http://localhost:7700\"\ntls.allow-invalid-certs = true\ndisable = {MEILI_ENABLED}\n[store.\"meili\".task]\npoll-interval = \"100ms\"\n#[store.\"meili\".auth]\n#username = \"meili\"\n#secret = \"changeme\"\n\n#[store.\"s3\"]\n#type = \"s3\"\n#access-key = \"minioadmin\"\n#secret-key = \"minioadmin\"\n#region = \"eu-central-1\"\n#endpoint = \"http://localhost:9000\"\n#bucket = \"tmp\"\n\n[store.\"fs\"]\ntype = \"fs\"\npath = \"{TMP}\"\n\n[store.\"redis\"]\ntype = \"redis\"\nurls = \"redis://127.0.0.1\"\nredis-type = \"single\"\n\n#[store.\"psql-replica\"]\n#type = \"sql-read-replica\"\n#primary = \"postgresql\"\n#replicas = \"postgresql\"\n\n[storage]\ndata = \"{STORE}\"\nfts = \"{SEARCH_STORE}\"\nblob = \"{BLOB_STORE}\"\nlookup = \"{LOOKUP_STORE}\"\ndirectory = \"{STORE}\"\n\n[directory.\"{STORE}\"]\ntype = \"internal\"\nstore = \"{STORE}\"\n\n[session.rcpt]\ndirectory = \"'{STORE}'\"\n\n[session.auth]\ndirectory = \"'{STORE}'\"\n\n\"#;\n"
  },
  {
    "path": "tests/src/store/ops.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashSet;\nuse std::collections::HashSet;\nuse store::{\n    Store, ValueKey,\n    rand::{self, Rng},\n    write::{\n        AlignedBytes, Archive, Archiver, BatchBuilder, DirectoryClass, MergeResult, Params,\n        ValueClass,\n    },\n};\nuse types::collection::{Collection, SyncCollection};\n\nuse crate::store::cleanup::store_assert_is_empty;\n\n// FDB max value\nconst MAX_VALUE_SIZE: usize = 100000;\n\n#[cfg(feature = \"foundationdb\")]\nfn value_gen(chunks: impl IntoIterator<Item = (u8, usize)>) -> Vec<u8> {\n    let mut value = Vec::new();\n    for (byte, size) in chunks {\n        value.extend(std::iter::repeat_n(byte, size));\n    }\n    value\n}\n\npub async fn test(db: Store) {\n    #[cfg(feature = \"foundationdb\")]\n    if matches!(db, Store::FoundationDb(_)) {\n        use types::collection::Collection;\n\n        println!(\"Running FoundationDB chunked iterator test...\");\n        let kvs = [\n            (\"a\", value_gen([(b'a', 1)])),\n            (\"b\", value_gen([(b'b', MAX_VALUE_SIZE), (b'0', 1)])),\n            (\n                \"c\",\n                value_gen([\n                    (b'c', MAX_VALUE_SIZE),\n                    (b'1', MAX_VALUE_SIZE),\n                    (b'2', MAX_VALUE_SIZE),\n                ]),\n            ),\n            (\n                \"d\",\n                value_gen([(b'd', MAX_VALUE_SIZE), (b'3', MAX_VALUE_SIZE)]),\n            ),\n            (\"e\", value_gen([(b'e', 1)])),\n        ];\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(0)\n            .with_collection(Collection::Email)\n            .with_document(0);\n\n        for (key, value) in &kvs {\n            batch.set(ValueClass::Config(key.as_bytes().to_vec()), value.clone());\n        }\n        db.write(batch.build_all()).await.unwrap();\n\n        // Iterate over all keys\n        let mut results = Vec::new();\n        db.iterate(\n            store::IterateParams::new(\n                ValueKey {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 0,\n                    class: ValueClass::Config(b\"\".to_vec()),\n                },\n                ValueKey {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 0,\n                    class: ValueClass::Config(b\"\\xFF\".to_vec()),\n                },\n            ),\n            |key, value| {\n                results.push((String::from_utf8(key.to_vec()).unwrap(), value.to_vec()));\n                Ok(true)\n            },\n        )\n        .await\n        .unwrap();\n\n        assert_eq!(results.len(), kvs.len());\n\n        db.delete_range(\n            ValueKey {\n                account_id: 0,\n                collection: 0,\n                document_id: 0,\n                class: ValueClass::Config(b\"\".to_vec()),\n            },\n            ValueKey {\n                account_id: 0,\n                collection: 0,\n                document_id: 0,\n                class: ValueClass::Config(b\"\\xFF\".to_vec()),\n            },\n        )\n        .await\n        .unwrap();\n\n        if std::env::var(\"SLOW_FDB_TRX\").is_ok() {\n            println!(\"Running FoundationDB slow transaction tests...\");\n            // Create 900000 keys\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(0)\n                .with_collection(Collection::Email)\n                .with_document(0);\n            for n in 0..900000 {\n                batch.set(\n                    ValueClass::Config(format!(\"key{n:10}\").into_bytes()),\n                    format!(\"value{n:10}\").into_bytes(),\n                );\n\n                if n % 10000 == 0 {\n                    db.write(batch.build_all()).await.unwrap();\n                    batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(0)\n                        .with_collection(Collection::Email)\n                        .with_document(0);\n                }\n            }\n            db.write(batch.build_all()).await.unwrap();\n\n            println!(\"Created 900.000 keys...\");\n\n            // Iterate over all keys\n            let mut n = 0;\n            db.iterate(\n                store::IterateParams::new(\n                    ValueKey {\n                        account_id: 0,\n                        collection: 0,\n                        document_id: 0,\n                        class: ValueClass::Config(b\"\".to_vec()),\n                    },\n                    ValueKey {\n                        account_id: 0,\n                        collection: 0,\n                        document_id: 0,\n                        class: ValueClass::Config(b\"\\xFF\".to_vec()),\n                    },\n                ),\n                |key, value| {\n                    assert_eq!(std::str::from_utf8(key).unwrap(), format!(\"key{n:10}\"));\n                    assert_eq!(std::str::from_utf8(value).unwrap(), format!(\"value{n:10}\"));\n                    n += 1;\n                    if n % 10000 == 0 {\n                        println!(\"Iterated over {n} keys\");\n                        std::thread::sleep(std::time::Duration::from_millis(1000));\n                    }\n                    Ok(true)\n                },\n            )\n            .await\n            .unwrap();\n\n            // Delete 100 keys\n            let mut batch = BatchBuilder::new();\n            batch\n                .with_account_id(0)\n                .with_collection(Collection::Email)\n                .with_document(0);\n            for n in 0..900000 {\n                batch.clear(ValueClass::Config(format!(\"key{n:10}\").into_bytes()));\n\n                if n % 10000 == 0 {\n                    db.write(batch.build_all()).await.unwrap();\n                    batch = BatchBuilder::new();\n                    batch\n                        .with_account_id(0)\n                        .with_collection(Collection::Email)\n                        .with_document(0);\n                }\n            }\n            db.write(batch.build_all()).await.unwrap();\n        }\n    }\n\n    // Merge values 1000 times concurrently\n    let mut handles = Vec::new();\n    println!(\"Merge values 1000 times concurrently...\");\n    for _ in 0..1000 {\n        handles.push({\n            let db = db.clone();\n            tokio::spawn(async move {\n                for _ in 0..5 {\n                    let mut builder = BatchBuilder::new();\n                    builder\n                        .with_account_id(0)\n                        .with_collection(Collection::Email)\n                        .with_document(0)\n                        .merge_fnc(\n                            ValueClass::Property(3),\n                            Params::with_capacity(0),\n                            |_, _, bytes| {\n                                if let Some(bytes) = bytes {\n                                    Ok(MergeResult::Update(\n                                        (u64::from_be_bytes(bytes.try_into().unwrap()) + 1)\n                                            .to_be_bytes()\n                                            .to_vec(),\n                                    ))\n                                } else {\n                                    Ok(MergeResult::Update(0u64.to_be_bytes().to_vec()))\n                                }\n                            },\n                        );\n\n                    match db.write(builder.build_all()).await {\n                        Ok(_) => {\n                            break;\n                        }\n                        Err(e) if e.is_assertion_failure() => {\n                            // Retry on assertion failures\n                            continue;\n                        }\n                        Err(e) => {\n                            panic!(\"Merge failed: {:?}\", e);\n                        }\n                    }\n                }\n            })\n        });\n    }\n\n    for handle in handles {\n        handle.await.unwrap();\n    }\n\n    assert_eq!(\n        999,\n        db.get_value::<u64>(ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Property(3),\n        })\n        .await\n        .unwrap()\n        .unwrap()\n    );\n\n    // Increment a counter 1000 times concurrently\n    let mut handles = Vec::new();\n    let mut assigned_ids = HashSet::new();\n    println!(\"Incrementing counter 1000 times concurrently...\");\n    for _ in 0..1000 {\n        handles.push({\n            let db = db.clone();\n            tokio::spawn(async move {\n                let mut builder = BatchBuilder::new();\n                builder\n                    .with_account_id(0)\n                    .with_collection(Collection::Email)\n                    .with_document(0)\n                    .add_and_get(ValueClass::Directory(DirectoryClass::UsedQuota(0)), 1);\n                db.write(builder.build_all())\n                    .await\n                    .unwrap()\n                    .last_counter_id()\n                    .unwrap()\n            })\n        });\n    }\n\n    for handle in handles {\n        let assigned_id = handle.await.unwrap();\n        assert!(\n            assigned_ids.insert(assigned_id),\n            \"counter assigned {assigned_id} twice or more times.\"\n        );\n    }\n    assert_eq!(assigned_ids.len(), 1000);\n    assert_eq!(\n        db.get_counter(ValueKey {\n            account_id: 0,\n            collection: 0,\n            document_id: 0,\n            class: ValueClass::Directory(DirectoryClass::UsedQuota(0)),\n        })\n        .await\n        .unwrap(),\n        1000\n    );\n\n    // Concurrent changelog\n    let mut handles = Vec::new();\n    let mut assigned_ids = AHashSet::new();\n    print!(\"Incrementing changeId 1000 times concurrently...\");\n    let time = std::time::Instant::now();\n    for document_id in 0..1000 {\n        handles.push({\n            let db = db.clone();\n            tokio::spawn(async move {\n                let mut builder = BatchBuilder::new();\n                let value = if document_id != 0 {\n                    (0..rand::rng().random_range(1..=100))\n                        .map(|_| rand::rng().random_range(0..=255))\n                        .collect::<Vec<u8>>()\n                } else {\n                    vec![0u8; 100000]\n                };\n\n                let (offset, archived_value) = Archiver::new(value).serialize_versioned().unwrap();\n\n                builder\n                    .with_account_id(0)\n                    .with_collection(Collection::Email)\n                    .with_document(document_id)\n                    .set_fnc(\n                        ValueClass::Property(5),\n                        Params::with_capacity(2)\n                            .with_bytes(archived_value)\n                            .with_u64(offset),\n                        |params, ids| {\n                            let change_id = ids.current_change_id()?;\n                            let archive = params.bytes(0);\n                            let offset = params.u64(1);\n\n                            let mut bytes = Vec::with_capacity(archive.len());\n                            bytes.extend_from_slice(&archive[..offset as usize]);\n                            bytes.extend_from_slice(&change_id.to_be_bytes()[..]);\n                            bytes.push(archive.last().copied().unwrap()); // Marker\n                            Ok(bytes)\n                        },\n                    )\n                    .log_container_insert(SyncCollection::Email);\n                db.write(builder.build_all())\n                    .await\n                    .unwrap()\n                    .last_change_id(0)\n                    .unwrap()\n            })\n        });\n    }\n    for handle in handles {\n        let assigned_id = handle.await.unwrap();\n        assert!(\n            assigned_ids.insert(assigned_id),\n            \"counter assigned {assigned_id} twice or more times: {:?}.\",\n            assigned_ids\n        );\n    }\n    assert_eq!(assigned_ids.len(), 1000);\n    println!(\" done in {:?}ms\", time.elapsed().as_millis());\n    let mut change_ids = AHashSet::new();\n    for document_id in 0..1000 {\n        let archive = db\n            .get_value::<Archive<AlignedBytes>>(ValueKey {\n                account_id: 0,\n                collection: 0,\n                document_id,\n                class: ValueClass::Property(5),\n            })\n            .await\n            .unwrap()\n            .unwrap();\n        change_ids.insert(archive.version.change_id().unwrap());\n        archive.unarchive_untrusted::<Vec<u8>>().unwrap();\n    }\n    assert_eq!(change_ids, assigned_ids);\n\n    println!(\"Running chunking tests...\");\n    for (test_num, value) in [\n        vec![b'A'; 0],\n        vec![b'A'; 1],\n        vec![b'A'; 100],\n        vec![b'A'; MAX_VALUE_SIZE],\n        vec![b'B'; MAX_VALUE_SIZE + 1],\n        vec![b'C'; MAX_VALUE_SIZE]\n            .into_iter()\n            .chain(vec![b'D'; MAX_VALUE_SIZE])\n            .chain(vec![b'E'; MAX_VALUE_SIZE])\n            .collect::<Vec<_>>(),\n        vec![b'F'; MAX_VALUE_SIZE]\n            .into_iter()\n            .chain(vec![b'G'; MAX_VALUE_SIZE])\n            .chain(vec![b'H'; MAX_VALUE_SIZE + 1])\n            .collect::<Vec<_>>(),\n    ]\n    .into_iter()\n    .enumerate()\n    {\n        // Write value\n        let test_len = value.len();\n        db.write(\n            BatchBuilder::new()\n                .with_account_id(0)\n                .with_collection(Collection::Email)\n                .with_document(0)\n                .set(ValueClass::Property(1), value.as_slice())\n                .set(ValueClass::Property(0), \"check1\".as_bytes())\n                .set(ValueClass::Property(2), \"check2\".as_bytes())\n                .build_all(),\n        )\n        .await\n        .unwrap();\n\n        // Fetch value\n        assert_eq!(\n            String::from_utf8(value).unwrap(),\n            db.get_value::<String>(ValueKey {\n                account_id: 0,\n                collection: 0,\n                document_id: 0,\n                class: ValueClass::Property(1),\n            })\n            .await\n            .unwrap()\n            .unwrap_or_else(|| panic!(\"no value for test {test_num} with value length {test_len}\")),\n            \"failed for test {test_num} with value length {test_len}\"\n        );\n\n        // Delete value\n        db.write(\n            BatchBuilder::new()\n                .with_account_id(0)\n                .with_collection(Collection::Email)\n                .with_document(0)\n                .clear(ValueClass::Property(1))\n                .build_all(),\n        )\n        .await\n        .unwrap();\n\n        // Make sure value is deleted\n        assert_eq!(\n            None,\n            db.get_value::<String>(ValueKey {\n                account_id: 0,\n                collection: 0,\n                document_id: 0,\n                class: ValueClass::Property(1),\n            })\n            .await\n            .unwrap()\n        );\n\n        // Make sure other values are still there\n        for (class, value) in [\n            (ValueClass::Property(0), \"check1\"),\n            (ValueClass::Property(2), \"check2\"),\n        ] {\n            assert_eq!(\n                Some(value.to_string()),\n                db.get_value::<String>(ValueKey {\n                    account_id: 0,\n                    collection: 0,\n                    document_id: 0,\n                    class,\n                })\n                .await\n                .unwrap()\n            );\n        }\n\n        // Delete everything\n        let mut batch = BatchBuilder::new();\n        batch\n            .with_account_id(0)\n            .with_collection(Collection::Email)\n            .with_account_id(0)\n            .with_document(0)\n            .clear(ValueClass::Property(0))\n            .clear(ValueClass::Property(2))\n            .clear(ValueClass::Property(3))\n            .clear(ValueClass::Directory(DirectoryClass::UsedQuota(0)))\n            .clear(ValueClass::ChangeId);\n\n        for document_id in 0..1000 {\n            batch\n                .with_document(document_id)\n                .clear(ValueClass::Property(5));\n        }\n\n        db.write(batch.build_all()).await.unwrap();\n\n        // Make sure everything is deleted\n        store_assert_is_empty(&db, db.clone().into(), false).await;\n    }\n}\n"
  },
  {
    "path": "tests/src/store/query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::store::deflate_test_resource;\nuse ahash::AHashSet;\nuse nlp::language::Language;\nuse std::{\n    io::Write,\n    sync::{Arc, Mutex},\n    time::Instant,\n};\nuse store::{\n    SearchStore,\n    ahash::AHashMap,\n    rand::{self, Rng, distr::Alphanumeric},\n    roaring::RoaringBitmap,\n    search::{\n        EmailSearchField, IndexDocument, SearchComparator, SearchField, SearchFilter,\n        SearchOperator, SearchQuery, SearchValue, TracingSearchField,\n    },\n    write::SearchIndex,\n};\nuse utils::map::vec_map::VecMap;\n\npub const FIELDS: [&str; 20] = [\n    \"id\",\n    \"accession_number\",\n    \"artist\",\n    \"artistRole\",\n    \"artistId\",\n    \"title\",\n    \"dateText\",\n    \"medium\",\n    \"creditLine\",\n    \"year\",\n    \"acquisitionYear\",\n    \"dimensions\",\n    \"width\",\n    \"height\",\n    \"depth\",\n    \"units\",\n    \"inscription\",\n    \"thumbnailCopyright\",\n    \"thumbnailUrl\",\n    \"url\",\n];\n\n/*\n \"title\", // Subject\n \"year\".   // ReceivedAt\n \"width\",  // Size\n \"height\", // SentAt\n \"artist\" // Headers\n \"artistRole\" // Cc\n \"medium\",  // From\n \"creditLine\" // Body\n \"acquisitionYear\" // Bcc\n \"accession_number\" // To\n*/\n\nconst FIELD_MAPPINGS: [EmailSearchField; 20] = [\n    EmailSearchField::HasAttachment, // \"id\",\n    EmailSearchField::To,            // \"accession_number\",\n    EmailSearchField::Headers,       // \"artist\",\n    EmailSearchField::Cc,            // \"artistRole\",\n    EmailSearchField::HasAttachment, // \"artistId\",\n    EmailSearchField::Subject,       // \"title\",\n    EmailSearchField::HasAttachment, // \"dateText\",\n    EmailSearchField::From,          // \"medium\",\n    EmailSearchField::Body,          // \"creditLine\",\n    EmailSearchField::ReceivedAt,    // \"year\",\n    EmailSearchField::Bcc,           // \"acquisitionYear\",\n    EmailSearchField::HasAttachment, // \"dimensions\",\n    EmailSearchField::Size,          // \"width\",\n    EmailSearchField::SentAt,        // \"height\",\n    EmailSearchField::HasAttachment, // \"depth\",\n    EmailSearchField::HasAttachment, // \"units\",\n    EmailSearchField::HasAttachment, // \"inscription\",\n    EmailSearchField::HasAttachment, // \"thumbnailCopyright\",\n    EmailSearchField::HasAttachment, // \"thumbnailUrl\",\n    EmailSearchField::HasAttachment, // \"url\",\n];\n\nconst ALL_IDS: &[&str] = &[\n    \"p11293\", \"p79426\", \"p79427\", \"p79428\", \"p79429\", \"p79430\", \"d05503\", \"d00399\", \"d05352\",\n    \"p01764\", \"t05843\", \"n02478\", \"n02479\", \"n03568\", \"n03658\", \"n04327\", \"n04328\", \"n04721\",\n    \"n04739\", \"n05095\", \"n05096\", \"n05145\", \"n05157\", \"n05158\", \"n05159\", \"n05298\", \"n05303\",\n    \"n06070\", \"t01181\", \"t03571\", \"t05805\", \"t05806\", \"t12147\", \"t12154\", \"t12155\", \"ar00039\",\n    \"t12600\", \"p80203\", \"t13209\", \"t13560\", \"t13561\", \"t13655\", \"t13811\", \"p13352\", \"p13351\",\n    \"p13350\", \"p13349\", \"p13348\", \"p13347\", \"p13346\", \"p13345\", \"p13344\", \"p13342\", \"p13341\",\n    \"p13340\", \"p13339\", \"p13338\", \"p13337\", \"p13336\", \"p13335\", \"p13334\", \"p13333\", \"p13332\",\n    \"p13331\", \"p13330\", \"p13329\", \"p13328\", \"p13327\", \"p13326\", \"p13325\", \"p13324\", \"p13323\",\n    \"t13786\", \"p13322\", \"p13321\", \"p13320\", \"p13319\", \"p13318\", \"p13317\", \"p13316\", \"p13315\",\n    \"p13314\", \"t13588\", \"t13587\", \"t13586\", \"t13585\", \"t13584\", \"t13540\", \"t13444\", \"ar01154\",\n    \"ar01153\", \"t03681\", \"t12601\", \"ar00166\", \"t12625\", \"t12915\", \"p04182\", \"t06483\", \"ar00703\",\n    \"t07671\", \"ar00021\", \"t05557\", \"t07918\", \"p06298\", \"p05465\", \"p06640\", \"t12855\", \"t01355\",\n    \"t12800\", \"t12557\", \"t02078\", \"ar00052\", \"ar00627\", \"t00352\", \"t07275\", \"t12318\", \"t04931\",\n    \"t13683\", \"t13686\", \"t13687\", \"t13688\", \"t13689\", \"t13690\", \"t13691\", \"t13769\", \"t13773\",\n    \"t07151\", \"t13684\", \"t07523\", \"t12369\", \"t12567\", \"ar00627\", \"ar00052\", \"t00352\", \"t07275\",\n    \"t12318\", \"t04931\", \"t13683\", \"t13686\", \"t13687\", \"t13688\", \"t13689\", \"t13690\", \"t13691\",\n    \"t07766\", \"t07918\", \"t12993\", \"ar00044\", \"t13326\", \"t07614\", \"t12414\",\n];\n\n#[allow(clippy::mutex_atomic)]\npub async fn test(store: SearchStore, do_insert: bool) {\n    println!(\"Running Store query tests...\");\n\n    let pool = rayon::ThreadPoolBuilder::new()\n        .num_threads(8)\n        .build()\n        .unwrap();\n    let now = Instant::now();\n    let documents = Arc::new(Mutex::new(Vec::new()));\n    let mut mask = RoaringBitmap::new();\n    let mut fields = AHashMap::new();\n\n    // Global ids test\n    println!(\"Running global id filtering tests...\");\n    test_global(store.clone()).await;\n\n    // Large document insert test\n    println!(\"Running large document insert tests...\");\n    let mut large_text = String::with_capacity(20 * 1024 * 1024);\n    while large_text.len() < 20 * 1024 * 1024 {\n        let word = rand::rng()\n            .sample_iter(&Alphanumeric)\n            .take(rand::rng().random_range(3..10))\n            .map(char::from)\n            .collect::<String>();\n        large_text.push_str(&word);\n        large_text.push(' ');\n    }\n    let mut document = IndexDocument::new(SearchIndex::Email)\n        .with_account_id(1)\n        .with_document_id(1);\n    for field in [\n        EmailSearchField::From,\n        EmailSearchField::To,\n        EmailSearchField::Cc,\n        EmailSearchField::Bcc,\n        EmailSearchField::Subject,\n    ] {\n        document.index_text(field, &large_text[..10 * 1024], Language::English);\n    }\n    for field in [EmailSearchField::Body, EmailSearchField::Attachment] {\n        document.index_text(field, &large_text, Language::English);\n    }\n    for field in [\n        EmailSearchField::ReceivedAt,\n        EmailSearchField::SentAt,\n        EmailSearchField::Size,\n    ] {\n        document.index_unsigned(field, rand::rng().random_range(100u64..1_000_000u64));\n    }\n    store.index(vec![document]).await.unwrap();\n    // Refresh\n    if let SearchStore::ElasticSearch(store) = &store {\n        store.refresh_index(SearchIndex::Email).await.unwrap();\n    }\n\n    println!(\"Running account filtering tests...\");\n    let filter_ids = std::env::var(\"QUICK_TEST\").is_ok().then(|| {\n        let mut ids = AHashSet::new();\n        for &id in ALL_IDS {\n            ids.insert(id.to_string());\n            let id = id.as_bytes();\n            if id.last().unwrap() > &b'0' {\n                let mut alt_id = id.to_vec();\n                *alt_id.last_mut().unwrap() -= 1;\n                ids.insert(String::from_utf8(alt_id).unwrap());\n            }\n            if id.last().unwrap() < &b'9' {\n                let mut alt_id = id.to_vec();\n                *alt_id.last_mut().unwrap() += 1;\n                ids.insert(String::from_utf8(alt_id).unwrap());\n            }\n        }\n\n        ids\n    });\n\n    pool.scope_fifo(|s| {\n        for (document_id, record) in csv::ReaderBuilder::new()\n            .has_headers(true)\n            .from_reader(&deflate_test_resource(\"artwork_data.csv.gz\")[..])\n            .records()\n            .enumerate()\n        {\n            let record = record.unwrap();\n            let documents = documents.clone();\n\n            if let Some(filter_ids) = &filter_ids {\n                let id = record.get(1).unwrap().to_lowercase();\n                if !filter_ids.contains(&id) {\n                    continue;\n                }\n            }\n\n            s.spawn_fifo(move |_| {\n                let mut document = IndexDocument::new(SearchIndex::Email)\n                    .with_account_id(0)\n                    .with_document_id(document_id as u32);\n                for (pos, field) in record.iter().enumerate() {\n                    match FIELD_MAPPINGS[pos] {\n                        EmailSearchField::From\n                        | EmailSearchField::To\n                        | EmailSearchField::Cc\n                        | EmailSearchField::Bcc => {\n                            document.index_text(\n                                FIELD_MAPPINGS[pos].clone(),\n                                &field.to_lowercase(),\n                                Language::None,\n                            );\n                        }\n                        EmailSearchField::Subject\n                        | EmailSearchField::Body\n                        | EmailSearchField::Attachment => {\n                            document.index_text(\n                                FIELD_MAPPINGS[pos].clone(),\n                                &field\n                                    .replace(|ch: char| !ch.is_alphanumeric(), \" \")\n                                    .to_lowercase(),\n                                Language::English,\n                            );\n                        }\n                        EmailSearchField::Headers => {\n                            document.insert_key_value(\n                                EmailSearchField::Headers,\n                                \"artist\",\n                                field.to_lowercase(),\n                            );\n                        }\n                        EmailSearchField::ReceivedAt\n                        | EmailSearchField::SentAt\n                        | EmailSearchField::Size => {\n                            document.index_unsigned(\n                                FIELD_MAPPINGS[pos].clone(),\n                                field.parse::<u64>().unwrap_or(0),\n                            );\n                        }\n                        _ => {\n                            continue;\n                        }\n                    };\n                }\n\n                documents.lock().unwrap().push(document);\n            });\n        }\n    });\n\n    println!(\n        \"Parsed {} entries in {} ms.\",\n        documents.lock().unwrap().len(),\n        now.elapsed().as_millis()\n    );\n\n    let now = Instant::now();\n    let batches = documents.lock().unwrap().drain(..).collect::<Vec<_>>();\n\n    print!(\"Inserting... \",);\n    let mut chunks = Vec::new();\n    let mut chunk = Vec::new();\n    for document in batches {\n        let mut document_id = None;\n        let mut to_field = None;\n\n        for (key, value) in document.fields() {\n            if key == &SearchField::DocumentId {\n                if let SearchValue::Uint(id) = value {\n                    document_id = Some(*id as u32);\n                }\n            } else if key == &SearchField::Email(EmailSearchField::To)\n                && let SearchValue::Text { value, .. } = value\n            {\n                to_field = Some(value.to_string());\n            }\n        }\n        let document_id = document_id.unwrap();\n        let to_field = to_field.unwrap();\n        mask.insert(document_id);\n        fields.insert(document_id, to_field);\n\n        chunk.push(document);\n        if chunk.len() == 10 {\n            chunks.push(chunk);\n            chunk = Vec::new();\n        }\n    }\n    if !chunk.is_empty() {\n        chunks.push(chunk);\n    }\n\n    if do_insert {\n        let mut tasks = Vec::new();\n        for chunk in chunks {\n            let chunk_instance = Instant::now();\n            tasks.push({\n                let db = store.clone();\n                tokio::spawn(async move { db.index(chunk).await })\n            });\n\n            if tasks.len() == 100 {\n                for handle in tasks {\n                    handle.await.unwrap().unwrap();\n                }\n                print!(\" [{} ms]\", chunk_instance.elapsed().as_millis());\n                std::io::stdout().flush().unwrap();\n                tasks = Vec::new();\n            }\n        }\n\n        if !tasks.is_empty() {\n            for handle in tasks {\n                handle.await.unwrap().unwrap();\n            }\n        }\n\n        // Refresh\n        if let SearchStore::ElasticSearch(store) = &store {\n            store.refresh_index(SearchIndex::Email).await.unwrap();\n        }\n\n        println!(\"\\nInsert took {} ms.\", now.elapsed().as_millis());\n    }\n\n    if store.internal_fts().is_none() {\n        let ids = store\n            .query_account(\n                SearchQuery::new(SearchIndex::Email)\n                    .with_filters(vec![SearchFilter::eq(SearchField::AccountId, 0u32)])\n                    .with_comparator(SearchComparator::ascending(EmailSearchField::ReceivedAt))\n                    .with_mask(mask.clone()),\n            )\n            .await\n            .unwrap()\n            .into_iter()\n            .collect::<RoaringBitmap>();\n        assert_eq!(ids, mask);\n        let ids = store\n            .query_account(\n                SearchQuery::new(SearchIndex::Email)\n                    .with_filters(vec![\n                        SearchFilter::eq(SearchField::AccountId, 0u32),\n                        SearchFilter::ge(SearchField::DocumentId, 0u32),\n                    ])\n                    .with_mask(mask.clone()),\n            )\n            .await\n            .unwrap()\n            .into_iter()\n            .collect::<RoaringBitmap>();\n        assert_eq!(ids, mask);\n    }\n\n    println!(\"Running account filter tests...\");\n    let now = Instant::now();\n    test_filter(store.clone(), &fields, &mask).await;\n    println!(\"Filtering took {} ms.\", now.elapsed().as_millis());\n\n    println!(\"Running account sort tests...\");\n    let now = Instant::now();\n    test_sort(store.clone(), &fields, &mask).await;\n    println!(\"Sorting took {} ms.\", now.elapsed().as_millis());\n\n    println!(\"Running unindex tests...\");\n    let now = Instant::now();\n    test_unindex(store.clone(), &fields).await;\n    println!(\"Unindexing took {} ms.\", now.elapsed().as_millis());\n}\n\nasync fn test_filter(store: SearchStore, fields: &AHashMap<u32, String>, mask: &RoaringBitmap) {\n    let can_stem = !store.is_mysql();\n\n    let tests = [\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::has_english_text(EmailSearchField::Subject, \"water\"),\n                SearchFilter::eq(EmailSearchField::ReceivedAt, 1979u32),\n            ],\n            vec![\"p11293\"],\n        ),\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::has_keyword(EmailSearchField::From, \"gelatin\"),\n                SearchFilter::gt(EmailSearchField::ReceivedAt, 2000u32),\n                SearchFilter::lt(EmailSearchField::Size, 180u32),\n                SearchFilter::gt(EmailSearchField::Size, 0u32),\n            ],\n            vec![\"p79426\", \"p79427\", \"p79428\", \"p79429\", \"p79430\"],\n        ),\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::has_english_text(EmailSearchField::Subject, \"'rustic bridge'\"),\n            ],\n            vec![\"d05503\"],\n        ),\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::has_english_text(EmailSearchField::Subject, \"'rustic'\"),\n                SearchFilter::has_english_text(\n                    EmailSearchField::Subject,\n                    if can_stem { \"study\" } else { \"studies\" },\n                ),\n            ],\n            vec![\"d00399\", \"d05352\"],\n        ),\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::cond(\n                    EmailSearchField::Headers,\n                    SearchOperator::Contains,\n                    SearchValue::KeyValues(VecMap::from_iter([(\n                        \"artist\".to_string(),\n                        \"kunst, mauro\".to_string(),\n                    )])),\n                ),\n                SearchFilter::has_keyword(EmailSearchField::Cc, \"artist\"),\n                SearchFilter::Or,\n                SearchFilter::eq(EmailSearchField::ReceivedAt, 1969u32),\n                SearchFilter::eq(EmailSearchField::ReceivedAt, 1971u32),\n                SearchFilter::End,\n            ],\n            vec![\"p01764\", \"t05843\"],\n        ),\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::Not,\n                SearchFilter::has_keyword(EmailSearchField::From, \"oil\"),\n                SearchFilter::End,\n                SearchFilter::has_english_text(\n                    EmailSearchField::Body,\n                    if can_stem { \"bequeath\" } else { \"bequeathed\" },\n                ),\n                SearchFilter::Or,\n                SearchFilter::And,\n                SearchFilter::ge(EmailSearchField::ReceivedAt, 1900u32),\n                SearchFilter::lt(EmailSearchField::ReceivedAt, 1910u32),\n                SearchFilter::End,\n                SearchFilter::And,\n                SearchFilter::ge(EmailSearchField::ReceivedAt, 2000u32),\n                SearchFilter::lt(EmailSearchField::ReceivedAt, 2010u32),\n                SearchFilter::End,\n                SearchFilter::End,\n            ],\n            vec![\n                \"n02478\", \"n02479\", \"n03568\", \"n03658\", \"n04327\", \"n04328\", \"n04721\", \"n04739\",\n                \"n05095\", \"n05096\", \"n05145\", \"n05157\", \"n05158\", \"n05159\", \"n05298\", \"n05303\",\n                \"n06070\", \"t01181\", \"t03571\", \"t05805\", \"t05806\", \"t12147\", \"t12154\", \"t12155\",\n            ],\n        ),\n        (\n            vec![\n                SearchFilter::And,\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::cond(\n                    EmailSearchField::Headers,\n                    SearchOperator::Contains,\n                    SearchValue::KeyValues(VecMap::from_iter([(\n                        \"artist\".to_string(),\n                        \"warhol\".to_string(),\n                    )])),\n                ),\n                SearchFilter::Not,\n                SearchFilter::has_english_text(EmailSearchField::Subject, \"'campbell'\"),\n                SearchFilter::End,\n                SearchFilter::Not,\n                SearchFilter::Or,\n                SearchFilter::gt(EmailSearchField::ReceivedAt, 1980u32),\n                SearchFilter::And,\n                SearchFilter::gt(EmailSearchField::Size, 500u32),\n                SearchFilter::gt(EmailSearchField::SentAt, 500u32),\n                SearchFilter::End,\n                SearchFilter::End,\n                SearchFilter::End,\n                SearchFilter::eq(EmailSearchField::Bcc, \"2008\".to_string()),\n                SearchFilter::End,\n            ],\n            vec![\"ar00039\", \"t12600\"],\n        ),\n        (\n            if can_stem {\n                vec![\n                    SearchFilter::eq(SearchField::AccountId, 0u32),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"study\"),\n                    SearchFilter::has_keyword(EmailSearchField::From, \"paper\"),\n                    SearchFilter::has_english_text(EmailSearchField::Body, \"'purchased'\"),\n                    SearchFilter::Not,\n                    SearchFilter::Or,\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'anatomical'\"),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'discarded'\"),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'untitled'\"),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'girl'\"),\n                    SearchFilter::End,\n                    SearchFilter::End,\n                    SearchFilter::gt(EmailSearchField::ReceivedAt, 1900u32),\n                    SearchFilter::gt(EmailSearchField::Bcc, \"2008\".to_string()),\n                ]\n            } else {\n                vec![\n                    SearchFilter::eq(SearchField::AccountId, 0u32),\n                    SearchFilter::Or,\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"study\"),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"studies\"),\n                    SearchFilter::End,\n                    SearchFilter::has_keyword(EmailSearchField::From, \"paper\"),\n                    SearchFilter::has_english_text(EmailSearchField::Body, \"'purchased'\"),\n                    SearchFilter::Not,\n                    SearchFilter::Or,\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'anatomical'\"),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'discarded'\"),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'untitled'\"),\n                    SearchFilter::has_english_text(EmailSearchField::Subject, \"'girl'\"),\n                    SearchFilter::End,\n                    SearchFilter::End,\n                    SearchFilter::gt(EmailSearchField::ReceivedAt, 1900u32),\n                    SearchFilter::gt(EmailSearchField::Bcc, \"2008\".to_string()),\n                ]\n            },\n            vec![\"p80203\", \"t13209\", \"t13560\", \"t13561\"],\n        ),\n    ];\n\n    for (filters, expected_results) in tests {\n        //println!(\"Running test: {:?}\", filter);\n        let ids = store\n            .query_account(\n                SearchQuery::new(SearchIndex::Email)\n                    .with_filters(filters)\n                    .with_comparator(SearchComparator::ascending(EmailSearchField::To))\n                    .with_mask(mask.clone()),\n            )\n            .await\n            .unwrap();\n\n        let mut results = Vec::new();\n        for document_id in ids {\n            results.push(fields.get(&document_id).unwrap());\n        }\n        assert_eq!(results, expected_results);\n    }\n}\n\nasync fn test_sort(store: SearchStore, fields: &AHashMap<u32, String>, mask: &RoaringBitmap) {\n    let is_reversed = store.is_postgres();\n\n    let tests = [\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::gt(EmailSearchField::ReceivedAt, 0u32),\n                SearchFilter::gt(EmailSearchField::Bcc, \"0000\".to_string()),\n                SearchFilter::gt(EmailSearchField::Size, 0u32),\n            ],\n            vec![\n                SearchComparator::descending(EmailSearchField::ReceivedAt),\n                SearchComparator::ascending(EmailSearchField::Bcc),\n                SearchComparator::ascending(EmailSearchField::Size),\n                SearchComparator::descending(EmailSearchField::To),\n            ],\n            vec![\n                \"t13655\", \"t13811\", \"p13352\", \"p13351\", \"p13350\", \"p13349\", \"p13348\", \"p13347\",\n                \"p13346\", \"p13345\", \"p13344\", \"p13342\", \"p13341\", \"p13340\", \"p13339\", \"p13338\",\n                \"p13337\", \"p13336\", \"p13335\", \"p13334\", \"p13333\", \"p13332\", \"p13331\", \"p13330\",\n                \"p13329\", \"p13328\", \"p13327\", \"p13326\", \"p13325\", \"p13324\", \"p13323\", \"t13786\",\n                \"p13322\", \"p13321\", \"p13320\", \"p13319\", \"p13318\", \"p13317\", \"p13316\", \"p13315\",\n                \"p13314\", \"t13588\", \"t13587\", \"t13586\", \"t13585\", \"t13584\", \"t13540\", \"t13444\",\n                \"ar01154\", \"ar01153\",\n            ],\n        ),\n        (\n            vec![\n                SearchFilter::eq(SearchField::AccountId, 0u32),\n                SearchFilter::gt(EmailSearchField::Size, 0u32),\n                SearchFilter::gt(EmailSearchField::SentAt, 0u32),\n            ],\n            vec![\n                SearchComparator::descending(EmailSearchField::Size),\n                SearchComparator::ascending(EmailSearchField::SentAt),\n            ],\n            vec![\n                \"t03681\", \"t12601\", \"ar00166\", \"t12625\", \"t12915\", \"p04182\", \"t06483\", \"ar00703\",\n                \"t07671\", \"ar00021\", \"t05557\", \"t07918\", \"p06298\", \"p05465\", \"p06640\", \"t12855\",\n                \"t01355\", \"t12800\", \"t12557\", \"t02078\",\n            ],\n        ),\n        (\n            vec![SearchFilter::eq(SearchField::AccountId, 0u32)],\n            vec![\n                SearchComparator::descending(EmailSearchField::From),\n                SearchComparator::descending(EmailSearchField::Cc),\n                SearchComparator::ascending(EmailSearchField::To),\n            ],\n            if is_reversed {\n                vec![\n                    \"ar00052\", \"ar00627\", \"t00352\", \"t07275\", \"t12318\", \"t04931\", \"t13683\",\n                    \"t13686\", \"t13687\", \"t13688\", \"t13689\", \"t13690\", \"t13691\", \"t13769\", \"t13773\",\n                    \"t07151\", \"t13684\", \"t07523\", \"t12369\", \"t12567\",\n                ]\n            } else {\n                vec![\n                    \"ar00627\", \"ar00052\", \"t00352\", \"t07275\", \"t12318\", \"t04931\", \"t13683\",\n                    \"t13686\", \"t13687\", \"t13688\", \"t13689\", \"t13690\", \"t13691\", \"t07766\", \"t07918\",\n                    \"t12993\", \"ar00044\", \"t13326\", \"t07614\", \"t12414\",\n                ]\n            },\n        ),\n    ];\n\n    for (filters, comparators, expected_results) in tests {\n        //println!(\"Running test: {:?}\", sort);\n        let ids = store\n            .query_account(\n                SearchQuery::new(SearchIndex::Email)\n                    .with_filters(filters)\n                    .with_comparators(comparators)\n                    .with_mask(mask.clone()),\n            )\n            .await\n            .unwrap();\n\n        let mut results = Vec::new();\n        for document_id in ids.into_iter().take(expected_results.len()) {\n            results.push(fields.get(&document_id).unwrap());\n        }\n        assert_eq!(results, expected_results);\n    }\n}\n\nasync fn test_unindex(store: SearchStore, fields: &AHashMap<u32, String>) {\n    let ids = store\n        .query_account(\n            SearchQuery::new(SearchIndex::Email)\n                .with_mask(RoaringBitmap::from_iter(fields.keys().copied()))\n                .with_filters(vec![\n                    SearchFilter::has_keyword(EmailSearchField::From, \"gelatin\"),\n                    SearchFilter::gt(EmailSearchField::ReceivedAt, 2000u32),\n                    SearchFilter::lt(EmailSearchField::Size, 180u32),\n                    SearchFilter::gt(EmailSearchField::Size, 0u32),\n                ])\n                .with_account_id(0),\n        )\n        .await\n        .unwrap();\n    assert!(!ids.is_empty());\n    let expected_count = ids.len().saturating_sub(10);\n\n    let mut query = SearchQuery::new(SearchIndex::Email)\n        .with_account_id(0)\n        .with_filter(SearchFilter::Or);\n    for id in ids.into_iter().take(10) {\n        query = query.with_filter(SearchFilter::eq(SearchField::DocumentId, id));\n    }\n    query = query.with_filter(SearchFilter::End);\n\n    store.unindex(query).await.unwrap();\n\n    // Refresh\n    if let SearchStore::ElasticSearch(store) = &store {\n        store.refresh_index(SearchIndex::Email).await.unwrap();\n    }\n\n    assert_eq!(\n        store\n            .query_account(\n                SearchQuery::new(SearchIndex::Email)\n                    .with_filters(vec![\n                        SearchFilter::has_keyword(EmailSearchField::From, \"gelatin\"),\n                        SearchFilter::gt(EmailSearchField::ReceivedAt, 2000u32),\n                        SearchFilter::lt(EmailSearchField::Size, 180u32),\n                        SearchFilter::gt(EmailSearchField::Size, 0u32),\n                    ])\n                    .with_account_id(0)\n                    .with_mask(RoaringBitmap::from_iter(fields.keys().copied())),\n            )\n            .await\n            .unwrap()\n            .len(),\n        expected_count\n    );\n}\n\nasync fn test_global(store: SearchStore) {\n    // Insert global ids\n    for (id, queue_id, etyp, keywords) in [\n        (0, 1000u64, 1u64, \"init start\"),\n        (1, 1000u64, 2u64, \"init complete\"),\n        (2, 1001u64, 1u64, \"process start\"),\n        (3, 1001u64, 2u64, \"process complete\"),\n        (4, 1002u64, 1u64, \"cleanup start\"),\n        (5, 1002u64, 2u64, \"cleanup complete\"),\n    ] {\n        let mut document = IndexDocument::new(SearchIndex::Tracing).with_id(id);\n        document.index_unsigned(TracingSearchField::QueueId, queue_id);\n        document.index_unsigned(TracingSearchField::EventType, etyp);\n        document.index_text(TracingSearchField::Keywords, keywords, Language::None);\n        store.index(vec![document]).await.unwrap();\n    }\n\n    // Refresh\n    if let SearchStore::ElasticSearch(store) = &store {\n        store.refresh_index(SearchIndex::Tracing).await.unwrap();\n    }\n\n    // Query all\n    assert_eq!(\n        store\n            .query_global(\n                SearchQuery::new(SearchIndex::Tracing)\n                    .with_filter(SearchFilter::ge(SearchField::Id, 0u64))\n            )\n            .await\n            .unwrap()\n            .into_iter()\n            .collect::<AHashSet<_>>(),\n        AHashSet::from_iter([0, 1, 2, 3, 4, 5])\n    );\n\n    // Query with filter\n    assert_eq!(\n        store\n            .query_global(\n                SearchQuery::new(SearchIndex::Tracing)\n                    .with_filter(SearchFilter::gt(SearchField::Id, 1u64))\n                    .with_filter(SearchFilter::lt(SearchField::Id, 5u64))\n                    .with_filter(SearchFilter::has_keyword(\n                        TracingSearchField::Keywords,\n                        \"start\",\n                    )),\n            )\n            .await\n            .unwrap()\n            .into_iter()\n            .collect::<AHashSet<_>>(),\n        AHashSet::from_iter([2, 4])\n    );\n\n    // Delete by filter\n    store\n        .unindex(\n            SearchQuery::new(SearchIndex::Tracing)\n                .with_filter(SearchFilter::lt(SearchField::Id, 3u64)),\n        )\n        .await\n        .unwrap();\n\n    // Refresh\n    if let SearchStore::ElasticSearch(store) = &store {\n        store.refresh_index(SearchIndex::Tracing).await.unwrap();\n    }\n\n    assert_eq!(\n        store\n            .query_global(\n                SearchQuery::new(SearchIndex::Tracing)\n                    .with_filter(SearchFilter::ge(SearchField::Id, 0u64))\n            )\n            .await\n            .unwrap()\n            .into_iter()\n            .collect::<AHashSet<_>>(),\n        AHashSet::from_iter([3, 4, 5])\n    );\n}\n"
  },
  {
    "path": "tests/src/webdav/acl.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse dav_proto::schema::property::{DavProperty, WebDavProperty};\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\n\nuse crate::webdav::GenerateTestDavResource;\n\nuse super::{DavResponse, DummyWebDavClient, WebDavTest};\n\npub async fn test(test: &WebDavTest) {\n    let owner_client = test.client(\"bill\");\n    let sharee_client = test.client(\"john\");\n\n    for resource_type in [\n        DavResourceName::File,\n        DavResourceName::Cal,\n        DavResourceName::Card,\n    ] {\n        println!(\"Running ACL tests ({})...\", resource_type.base_path());\n        let is_file = resource_type == DavResourceName::File;\n        let sharee_principal = format!(\"{}/john/\", DavResourceName::Principal.base_path());\n        let sharee_base_path = format!(\"{}/john/\", resource_type.base_path());\n        let owner_principal = format!(\"{}/bill/\", DavResourceName::Principal.base_path());\n        let owner_base_path = format!(\"{}/bill/\", resource_type.base_path());\n\n        // Create a resource for the owner\n        let owner_folder = format!(\"{owner_base_path}test-shared/\");\n        let owner_folder_private = format!(\"{owner_base_path}test-private/\");\n        let owner_file = format!(\"{owner_folder}test-file\");\n        let owner_file_content = resource_type.generate();\n        let owner_file_private = format!(\"{owner_folder_private}test-file-private\");\n        let owner_file_content_private = resource_type.generate();\n        let sharee_created_file = format!(\"{owner_folder}test-file-sharee\");\n        for (folder, file, content) in [\n            (&owner_folder, &owner_file, &owner_file_content),\n            (\n                &owner_folder_private,\n                &owner_file_private,\n                &owner_file_content_private,\n            ),\n        ] {\n            owner_client\n                .request(\"MKCOL\", folder, \"\")\n                .await\n                .with_status(StatusCode::CREATED);\n            owner_client\n                .request(\"PUT\", file, content)\n                .await\n                .with_status(StatusCode::CREATED);\n        }\n\n        // Create a resource for the sharee\n        let sharee_folder = format!(\"{sharee_base_path}test-folder/\");\n        let sharee_file = format!(\"{sharee_folder}test-file\");\n        let sharee_file_content = resource_type.generate();\n        sharee_client\n            .request(\"MKCOL\", &sharee_folder, \"\")\n            .await\n            .with_status(StatusCode::CREATED);\n        sharee_client\n            .request(\"PUT\", &sharee_file, &sharee_file_content)\n            .await\n            .with_status(StatusCode::CREATED);\n\n        // Test 1: Sharee should only see their own resources\n        sharee_client\n            .propfind_with_headers(\n                resource_type.collection_path(),\n                [DavProperty::WebDav(WebDavProperty::GetETag)],\n                [(\"prefer\", \"depth-noroot\")],\n            )\n            .await\n            .with_hrefs([sharee_base_path.as_str()]);\n\n        // Test 2: Share a resource and make sure the root folder is visible\n        owner_client\n            .acl(&owner_folder, sharee_principal.as_str(), [\"read\"])\n            .await\n            .with_status(StatusCode::OK);\n        if is_file {\n            owner_client\n                .acl(&owner_file, sharee_principal.as_str(), [\"read\"])\n                .await\n                .with_status(StatusCode::OK);\n        }\n        sharee_client\n            .propfind_with_headers(\n                resource_type.collection_path(),\n                [DavProperty::WebDav(WebDavProperty::GetETag)],\n                [(\"prefer\", \"depth-noroot\")],\n            )\n            .await\n            .with_hrefs([sharee_base_path.as_str(), owner_base_path.as_str()]);\n\n        // Test 3: Verify that only the shared resource is visible\n        sharee_client\n            .propfind_with_headers(\n                &owner_base_path,\n                [DavProperty::WebDav(WebDavProperty::GetETag)],\n                [(\"prefer\", \"depth-noroot\")],\n            )\n            .await\n            .with_hrefs([owner_folder.as_str()]);\n\n        // Test 4: Verify that the sharee can access the shared resource\n        sharee_client\n            .propfind(\n                &owner_folder,\n                [DavProperty::WebDav(WebDavProperty::GetETag)],\n            )\n            .await\n            .with_hrefs([owner_folder.as_str(), owner_file.as_str()]);\n        sharee_client\n            .request(\"GET\", &owner_file, \"\")\n            .await\n            .with_status(StatusCode::OK)\n            .with_body(&owner_file_content);\n        match resource_type {\n            DavResourceName::Cal => {\n                sharee_client\n                    .multiget_calendar(&owner_folder, &[&owner_file])\n                    .await\n                    .properties(&owner_file)\n                    .with_status(StatusCode::OK)\n                    .is_defined(DavProperty::WebDav(WebDavProperty::GetETag));\n            }\n            DavResourceName::Card => {\n                sharee_client\n                    .multiget_addressbook(&owner_folder, &[&owner_file])\n                    .await\n                    .properties(&owner_file)\n                    .with_status(StatusCode::OK)\n                    .is_defined(DavProperty::WebDav(WebDavProperty::GetETag));\n            }\n            _ => {}\n        }\n\n        // Test 5: Read ACL as owner\n        let response = owner_client\n            .propfind(&owner_folder, [DavProperty::WebDav(WebDavProperty::Acl)])\n            .await;\n        response\n            .properties(&owner_folder)\n            .get(DavProperty::WebDav(WebDavProperty::Acl))\n            .with_values([\n                format!(\"D:ace.D:principal.D:href:{sharee_principal}\").as_str(),\n                \"D:ace.D:grant.D:privilege.D:read\",\n                \"D:ace.D:grant.D:privilege.D:read-current-user-privilege-set\",\n            ]);\n\n        // Test 6: acl-principal-prop-set REPORT\n        let response = owner_client\n            .request(\"REPORT\", &owner_folder, ACL_PRINCIPAL_QUERY)\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .into_propfind_response(None);\n        response\n            .properties(&sharee_principal)\n            .get(DavProperty::WebDav(WebDavProperty::DisplayName))\n            .with_values([\"John Doe\"]);\n\n        // Test 7: Verify current-user-privilege-set and owner\n        let response = sharee_client\n            .propfind(\n                &owner_folder,\n                [\n                    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n                    DavProperty::WebDav(WebDavProperty::Owner),\n                ],\n            )\n            .await;\n        for href in [owner_folder.as_str(), owner_file.as_str()] {\n            let props = response.properties(href);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n                .with_values([\n                    \"D:privilege.D:read\",\n                    \"D:privilege.D:read-current-user-privilege-set\",\n                ]);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::Owner))\n                .with_values([format!(\"D:href:{owner_principal}\").as_str()]);\n        }\n\n        // Test 8: Write operations should fail\n        for (path, dest, dest_copy) in [\n            (\n                &owner_folder,\n                &sharee_folder,\n                Some(format!(\"{sharee_base_path}copied/\")),\n            ),\n            (&owner_file, &sharee_file, None),\n        ] {\n            sharee_client\n                .proppatch(\n                    path,\n                    [(DavProperty::WebDav(WebDavProperty::DisplayName), \"test\")],\n                    [],\n                    [],\n                )\n                .await\n                .with_status(StatusCode::FORBIDDEN);\n            sharee_client\n                .request(\"DELETE\", path, \"\")\n                .await\n                .with_status(StatusCode::FORBIDDEN);\n            sharee_client\n                .request_with_headers(\"MOVE\", path, [(\"destination\", dest.as_str())], \"\")\n                .await\n                .with_status(StatusCode::FORBIDDEN);\n            if let Some(dest_copy) = dest_copy {\n                sharee_client\n                    .request_with_headers(\"COPY\", path, [(\"destination\", dest_copy.as_str())], \"\")\n                    .await\n                    .with_status(StatusCode::CREATED);\n            }\n        }\n        sharee_client\n            .request(\"PUT\", &owner_file, resource_type.generate())\n            .await\n            .with_status(StatusCode::FORBIDDEN);\n        sharee_client\n            .request(\"PUT\", &sharee_created_file, resource_type.generate())\n            .await\n            .with_status(StatusCode::FORBIDDEN);\n\n        // Test 9: Grant write access to the sharee\n        owner_client\n            .acl(\n                &owner_folder,\n                sharee_principal.as_str(),\n                [\"read\", \"write-content\", \"write-properties\"],\n            )\n            .await\n            .with_status(StatusCode::OK);\n        if is_file {\n            owner_client\n                .acl(\n                    &owner_file,\n                    sharee_principal.as_str(),\n                    [\"read\", \"write-content\", \"write-properties\"],\n                )\n                .await\n                .with_status(StatusCode::OK);\n        }\n        let response = owner_client\n            .propfind(&owner_folder, [DavProperty::WebDav(WebDavProperty::Acl)])\n            .await;\n        response\n            .properties(&owner_folder)\n            .get(DavProperty::WebDav(WebDavProperty::Acl))\n            .with_values([\n                format!(\"D:ace.D:principal.D:href:{sharee_principal}\").as_str(),\n                \"D:ace.D:grant.D:privilege.D:read\",\n                \"D:ace.D:grant.D:privilege.D:read-current-user-privilege-set\",\n                \"D:ace.D:grant.D:privilege.D:write-content\",\n                \"D:ace.D:grant.D:privilege.D:write-properties\",\n            ]);\n        let response = sharee_client\n            .propfind(\n                &owner_folder,\n                [DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet)],\n            )\n            .await;\n        for href in [owner_folder.as_str(), owner_file.as_str()] {\n            response\n                .properties(href)\n                .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n                .with_values([\n                    \"D:privilege.D:read\",\n                    \"D:privilege.D:read-current-user-privilege-set\",\n                    \"D:privilege.D:write-content\",\n                    \"D:privilege.D:write-properties\",\n                ]);\n        }\n\n        // Test 10: Delete operations should fail\n        for (path, dest) in [(&owner_folder, &sharee_folder), (&owner_file, &sharee_file)] {\n            sharee_client\n                .proppatch(\n                    path,\n                    [(DavProperty::WebDav(WebDavProperty::DisplayName), \"test\")],\n                    [],\n                    [],\n                )\n                .await\n                .with_status(StatusCode::MULTI_STATUS);\n            sharee_client\n                .request(\"DELETE\", path, \"\")\n                .await\n                .with_status(StatusCode::FORBIDDEN);\n            sharee_client\n                .request_with_headers(\"MOVE\", path, [(\"destination\", dest.as_str())], \"\")\n                .await\n                .with_status(StatusCode::FORBIDDEN);\n        }\n        sharee_client\n            .request(\"PUT\", &owner_file, &owner_file_content)\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        sharee_client\n            .request(\"PUT\", &sharee_created_file, resource_type.generate())\n            .await\n            .with_status(StatusCode::CREATED);\n\n        // Test 11: Grant delete access to the sharee and verify\n        owner_client\n            .acl(&owner_folder, sharee_principal.as_str(), [\"read\", \"write\"])\n            .await\n            .with_status(StatusCode::OK);\n        if is_file {\n            owner_client\n                .acl(&owner_file, sharee_principal.as_str(), [\"read\", \"write\"])\n                .await\n                .with_status(StatusCode::OK);\n            owner_client\n                .acl(\n                    &sharee_created_file,\n                    sharee_principal.as_str(),\n                    [\"read\", \"write\"],\n                )\n                .await\n                .with_status(StatusCode::OK);\n        }\n        sharee_client\n            .request_with_headers(\n                \"MOVE\",\n                &owner_file,\n                [(\"destination\", sharee_file.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        sharee_client\n            .request(\"DELETE\", &sharee_created_file, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        sharee_client\n            .request(\"DELETE\", &owner_folder, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n\n        // Test 12: Share and unshare a resource\n        owner_client\n            .acl(&owner_folder_private, sharee_principal.as_str(), [\"read\"])\n            .await\n            .with_status(StatusCode::OK);\n        sharee_client\n            .propfind_with_headers(\n                resource_type.collection_path(),\n                [DavProperty::WebDav(WebDavProperty::GetETag)],\n                [(\"prefer\", \"depth-noroot\")],\n            )\n            .await\n            .with_hrefs([sharee_base_path.as_str(), owner_base_path.as_str()]);\n        sharee_client\n            .propfind_with_headers(\n                &owner_base_path,\n                [DavProperty::WebDav(WebDavProperty::GetETag)],\n                [(\"prefer\", \"depth-noroot\")],\n            )\n            .await\n            .with_hrefs([owner_folder_private.as_str()]);\n        owner_client\n            .acl(&owner_folder_private, sharee_principal.as_str(), [])\n            .await\n            .with_status(StatusCode::OK);\n        sharee_client\n            .propfind_with_headers(\n                resource_type.collection_path(),\n                [DavProperty::WebDav(WebDavProperty::GetETag)],\n                [(\"prefer\", \"depth-noroot\")],\n            )\n            .await\n            .with_hrefs([sharee_base_path.as_str()]);\n\n        // Delete resources\n        owner_client\n            .request(\"DELETE\", &owner_folder_private, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        sharee_client\n            .request(\"DELETE\", &sharee_folder, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        sharee_client\n            .request(\"DELETE\", &format!(\"{sharee_base_path}copied/\"), \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n    }\n\n    sharee_client.delete_default_containers().await;\n    owner_client.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\nimpl DummyWebDavClient {\n    pub async fn acl<'x>(\n        &self,\n        query: &str,\n        principal_href: &str,\n        grant: impl IntoIterator<Item = &'x str>,\n    ) -> DavResponse {\n        let body = ACL_QUERY.replace(\"$HREF\", principal_href).replace(\n            \"$GRANT\",\n            &grant.into_iter().fold(String::new(), |mut output, g| {\n                use std::fmt::Write;\n                let _ = write!(output, \"<D:privilege><D:{g}/></D:privilege>\");\n                output\n            }),\n        );\n        self.request(\"ACL\", query, &body).await\n    }\n}\n\nconst ACL_QUERY: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:acl xmlns:D=\"DAV:\">\n     <D:ace>\n       <D:principal>\n         <D:href>$HREF</D:href>\n       </D:principal>\n       <D:grant>\n         $GRANT\n       </D:grant>\n     </D:ace>\n   </D:acl>\"#;\n\nconst ACL_PRINCIPAL_QUERY: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:acl-principal-prop-set xmlns:D=\"DAV:\">\n     <D:prop>\n       <D:displayname/>\n     </D:prop>\n   </D:acl-principal-prop-set>\"#;\n"
  },
  {
    "path": "tests/src/webdav/basic.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse dav_proto::Depth;\nuse hyper::StatusCode;\n\nuse super::WebDavTest;\n\npub async fn test(test: &WebDavTest) {\n    println!(\"Running basic tests...\");\n    let john = test.client(\"john\");\n    let jane = test.client(\"jane\");\n\n    // Test OPTIONS request\n    john.request(\"OPTIONS\", \"/dav/file\", \"\")\n        .await\n        .with_header(\n            \"dav\",\n            concat!(\n                \"1, 2, 3, access-control, extended-mkcol, calendar-access, \",\n                \"calendar-auto-schedule, calendar-no-timezone, addressbook\"\n            ),\n        )\n        .with_header(\n            \"allow\",\n            concat!(\n                \"OPTIONS, GET, HEAD, POST, PUT, DELETE, COPY, MOVE, \",\n                \"MKCALENDAR, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, REPORT, ACL\"\n            ),\n        );\n\n    // Test Discovery\n    john.request(\"PROPFIND\", \"/.well-known/carddav\", \"\")\n        .await\n        .with_values(\n            \"D:multistatus.D:response.D:href\",\n            [\"/dav/card/\", \"/dav/card/john/\"],\n        );\n    jane.request(\"PROPFIND\", \"/.well-known/caldav\", \"\")\n        .await\n        .with_values(\n            \"D:multistatus.D:response.D:href\",\n            [\"/dav/cal/\", \"/dav/cal/jane/\", \"/dav/cal/support/\"],\n        );\n\n    // Test 404 responses\n    jane.sync_collection(\n        \"/dav/cal/jane/default/\",\n        \"\",\n        Depth::Infinity,\n        None,\n        [\"D:getetag\"],\n    )\n    .await;\n    jane.sync_collection(\n        \"/dav/cal/jane/test-404/\",\n        \"\",\n        Depth::Infinity,\n        None,\n        [\"D:getetag\"],\n    )\n    .await;\n    jane.request(\"PROPFIND\", \"/dav/cal/jane/default/\", \"\")\n        .await\n        .with_status(StatusCode::MULTI_STATUS);\n    jane.request(\n        \"REPORT\",\n        \"/dav/cal/jane/default/\",\n        concat!(\n            r#\"<CAL:calendar-query xmlns=\"DAV:\" \"#,\n            r#\"xmlns:CAL=\"urn:ietf:params:xml:ns:caldav\"><prop><getetag />\"#,\n            r#\"</prop><CAL:filter><CAL:comp-filter name=\"VCALENDAR\">\"#,\n            r#\"<CAL:comp-filter name=\"VTODO\" /></CAL:comp-filter></CAL:filter>\"#,\n            r#\"</CAL:calendar-query>\"#\n        ),\n    )\n    .await\n    .with_status(StatusCode::MULTI_STATUS);\n    jane.request(\n        \"REPORT\",\n        \"/dav/cal/jane/test-404/\",\n        concat!(\n            r#\"<CAL:calendar-query xmlns=\"DAV:\" \"#,\n            r#\"xmlns:CAL=\"urn:ietf:params:xml:ns:caldav\"><prop><getetag />\"#,\n            r#\"</prop><CAL:filter><CAL:comp-filter name=\"VCALENDAR\">\"#,\n            r#\"<CAL:comp-filter name=\"VTODO\" /></CAL:comp-filter></CAL:filter>\"#,\n            r#\"</CAL:calendar-query>\"#\n        ),\n    )\n    .await\n    .with_status(StatusCode::MULTI_STATUS);\n    jane.request(\"PROPFIND\", \"/dav/cal/jane/test-404/\", \"\")\n        .await\n        .with_status(StatusCode::NOT_FOUND);\n\n    john.delete_default_containers().await;\n    jane.delete_default_containers().await;\n    jane.delete_default_containers_by_account(\"support\").await;\n    test.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/webdav/cal_alarm.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::WebDavTest;\nuse crate::jmap::mail::mailbox::destroy_all_mailboxes_for_account;\nuse email::cache::MessageCacheFetch;\nuse hyper::StatusCode;\nuse mail_parser::{DateTime, MessageParser};\nuse store::write::now;\n\npub async fn test(test: &WebDavTest) {\n    println!(\"Running calendar e-mail alarms tests...\");\n    let client = test.client(\"john\");\n    client\n        .request_with_headers(\n            \"PUT\",\n            \"/dav/cal/john/default/its-alarming-how-charming-i-feel.ics\",\n            [(\"content-type\", \"text/calendar; charset=utf-8\")],\n            TEST_ALARM_1.replace(\n                \"$START\",\n                &DateTime::from_timestamp(now() as i64 + 5)\n                    .to_rfc3339()\n                    .replace(['-', ':'], \"\"),\n            ),\n        )\n        .await\n        .with_status(StatusCode::CREATED);\n\n    tokio::time::sleep(std::time::Duration::from_secs(6)).await;\n\n    // Check that the alarm was sent\n    let messages = test\n        .server\n        .get_cached_messages(client.account_id)\n        .await\n        .unwrap();\n    assert_eq!(messages.emails.items.len(), 2);\n\n    for (idx, message) in messages.emails.items.iter().enumerate() {\n        let contents = test\n            .fetch_email(client.account_id, message.document_id)\n            .await;\n\n        let message = MessageParser::new().parse(&contents).unwrap();\n        let contents = message\n            .html_bodies()\n            .next()\n            .unwrap()\n            .text_contents()\n            .unwrap();\n\n        if idx == 0 {\n            // First alarm does not have a summary or description\n            assert!(\n                contents.contains(\"See the pretty girl in that mirror there\"),\n                \"failed for {contents}\"\n            );\n            assert!(\n                contents.contains(\"What mirror where?!\"),\n                \"failed for {contents}\"\n            );\n        } else {\n            assert!(\n                contents.contains(\"I feel pretty and witty and gay\"),\n                \"failed for {contents}\"\n            );\n            assert!(\n                contents.contains(\"It&#39;s alarming how charming I feel.\"),\n                \"failed for {contents}\"\n            );\n        }\n        assert!(\n            contents.contains(concat!(\n                \"/dav/cal/john/default/\",\n                \"its-alarming-how-charming-i-feel.ics\"\n            )),\n            \"failed for {contents}\"\n        );\n    }\n\n    client.delete_default_containers().await;\n    destroy_all_mailboxes_for_account(client.account_id).await;\n    test.assert_is_empty().await\n}\n\nconst TEST_ALARM_1: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nBEGIN:VEVENT\nUID: 2371c2d9-a136-43b0-bba3-f6ab249ad46e\nSUMMARY:See the pretty girl in that mirror there\nDESCRIPTION:What mirror where?!\nDTSTART:$START\nDTEND;TZID=America/New_York:21250221T180000\nLOCATION:West Side\nBEGIN:VALARM\nTRIGGER:-P2S\nACTION:EMAIL\nATTENDEE:mailto:john_doe@unknown.com\nSUMMARY:I feel pretty and witty and gay\nDESCRIPTION:I feel charming, Oh, so charming, It's alarming how charming I feel.\nEND:VALARM\nBEGIN:VALARM\nTRIGGER:-P4S\nACTION:EMAIL\nEND:VALARM\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n"
  },
  {
    "path": "tests/src/webdav/cal_itip.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse ahash::AHashMap;\nuse calcard::{\n    common::{IanaString, PartialDateTime},\n    icalendar::{ICalendar, ICalendarProperty, ICalendarValue},\n};\nuse groupware::scheduling::{\n    ItipMessage, ItipSummary,\n    event_cancel::itip_cancel,\n    event_create::itip_create,\n    event_update::itip_update,\n    inbound::{MergeResult, itip_import_message, itip_merge_changes, itip_process_message},\n    snapshot::itip_snapshot,\n};\nuse std::{collections::hash_map::Entry, path::PathBuf};\n\nstruct Test {\n    test_name: String,\n    command: Command,\n    line_num: usize,\n    parameters: Vec<String>,\n    payload: String,\n}\n\n#[derive(Debug, PartialEq, Eq)]\nenum Command {\n    Put,\n    Get,\n    Delete(bool),\n    Expect,\n    Send,\n    Reset,\n    Itip,\n}\n\npub fn test() {\n    for entry in std::fs::read_dir(\n        PathBuf::from(env!(\"CARGO_MANIFEST_DIR\"))\n            .join(\"resources\")\n            .join(\"itip\"),\n    )\n    .unwrap()\n    {\n        let entry = entry.unwrap();\n        let path = entry.path();\n        if path.extension().is_none_or(|ext| ext != \"txt\") {\n            continue;\n        }\n        let file_name = path.file_name().unwrap().to_str().unwrap();\n        let rules = std::fs::read_to_string(&path).unwrap();\n        let mut last_comment = \"\";\n        let mut last_command = \"\";\n        let mut last_line_num = 0;\n        let mut payload = String::new();\n        let mut commands = Vec::new();\n\n        for (line_num, line) in rules.lines().enumerate() {\n            if line.starts_with('#') {\n                last_comment = line.trim_start_matches('#').trim();\n            } else if let Some(command) = line.strip_prefix(\"> \") {\n                last_command = command.trim();\n                last_line_num = line_num;\n            } else if !line.is_empty() {\n                payload.push_str(line);\n                payload.push('\\n');\n            } else {\n                if last_command.is_empty() && payload.is_empty() {\n                    continue;\n                }\n                let mut command_and_args = last_command.split_whitespace();\n                let command = match command_and_args\n                    .next()\n                    .expect(\"Command should not be empty\")\n                {\n                    \"put\" => Command::Put,\n                    \"get\" => Command::Get,\n                    \"expect\" => Command::Expect,\n                    \"send\" => Command::Send,\n                    \"delete\" => Command::Delete(false),\n                    \"delete-force-send\" => Command::Delete(true),\n                    \"reset\" => Command::Reset,\n                    \"itip\" => Command::Itip,\n                    _ => panic!(\"Unknown command: {}\", last_command),\n                };\n\n                commands.push(Test {\n                    command,\n                    test_name: last_comment.to_string(),\n                    line_num: last_line_num,\n                    parameters: command_and_args.map(String::from).collect(),\n                    payload: payload.trim().to_string(),\n                });\n\n                last_command = \"\";\n                last_line_num = 0;\n                payload.clear();\n            }\n        }\n\n        if commands.is_empty() {\n            panic!(\"No commands found in file: {}\", file_name);\n        } else if !last_command.is_empty() {\n            panic!(\n                \"File ended with command '{}' at line {} without payload\",\n                last_command, last_line_num\n            );\n        }\n\n        println!(\"====== Running test: {} ======\", file_name);\n\n        let mut store: AHashMap<String, AHashMap<String, ICalendar>> = AHashMap::new();\n        let mut dtstamp_map: AHashMap<PartialDateTime, usize> = AHashMap::new();\n        let mut last_itip = None;\n\n        for command in &commands {\n            if command.command != Command::Put {\n                println!(\"{} (line {})\", command.test_name, command.line_num);\n            }\n            match command.command {\n                Command::Put => {\n                    let account = command\n                        .parameters\n                        .first()\n                        .expect(\"Account parameter is required\");\n                    let name = command\n                        .parameters\n                        .get(1)\n                        .expect(\"Name parameter is required\");\n                    let mut ical = ICalendar::parse(&command.payload)\n                        .expect(\"Failed to parse iCalendar payload\");\n                    match store\n                        .entry(account.to_string())\n                        .or_default()\n                        .entry(name.to_string())\n                    {\n                        Entry::Occupied(mut entry) => {\n                            last_itip = Some(itip_update(\n                                &mut ical,\n                                entry.get_mut(),\n                                &[account.to_string()],\n                            ));\n                            entry.insert(ical);\n                        }\n                        Entry::Vacant(entry) => {\n                            last_itip = Some(itip_create(&mut ical, &[account.to_string()]));\n                            entry.insert(ical);\n                        }\n                    }\n                }\n                Command::Get => {\n                    let account = command\n                        .parameters\n                        .first()\n                        .expect(\"Account parameter is required\")\n                        .as_str();\n                    let name = command\n                        .parameters\n                        .get(1)\n                        .expect(\"Name parameter is required\")\n                        .as_str();\n                    let ical = ICalendar::parse(&command.payload)\n                        .expect(\"Failed to parse iCalendar payload\")\n                        .to_string()\n                        .replace(\"\\r\\n\", \"\\n\");\n                    store\n                        .get(account)\n                        .and_then(|account_store| account_store.get(name))\n                        .map(|stored_ical| {\n                            let stored_ical = normalize_ical(stored_ical.clone(), &mut dtstamp_map);\n                            if stored_ical != ical {\n                                panic!(\n                                    \"ICalendar mismatch for {}: expected {}, got {}\",\n                                    command.test_name, ical, stored_ical\n                                );\n                            }\n                        })\n                        .unwrap_or_else(|| {\n                            panic!(\n                                \"ICalendar not found for account: {}, name: {}\",\n                                account, name\n                            );\n                        });\n                }\n                Command::Delete(force_send) => {\n                    let account = command\n                        .parameters\n                        .first()\n                        .expect(\"Account parameter is required\")\n                        .as_str();\n                    let name = command\n                        .parameters\n                        .get(1)\n                        .expect(\"Name parameter is required\")\n                        .as_str();\n                    let store = store.get_mut(account).expect(\"Account not found in store\");\n\n                    if let Some(ical) = store.remove(name) {\n                        last_itip = Some(\n                            itip_cancel(&ical, &[account.to_string()], force_send)\n                                .map(|message| vec![message]),\n                        );\n                    } else {\n                        panic!(\n                            \"ICalendar not found for account: {}, name: {}\",\n                            account, name\n                        );\n                    }\n                }\n                Command::Expect => {\n                    let last_itip_str = match last_itip\n                        .as_ref()\n                        .expect(\"No last iTIP message to compare against\")\n                    {\n                        Ok(m) => {\n                            let mut result = String::new();\n                            for (i, m) in m.iter().enumerate() {\n                                if i > 0 {\n                                    result.push_str(\"================================\\n\");\n                                }\n                                result.push_str(&m.to_string(&mut dtstamp_map));\n                            }\n                            result\n                        }\n                        Err(e) => format!(\"{e:?}\"),\n                    };\n\n                    assert_eq!(\n                        command.payload.trim(),\n                        last_itip_str.trim(),\n                        \"iTIP message mismatch for {} at line {}\\nEXPECTED {}\\n\\nRECEIVED {}\",\n                        command.test_name,\n                        command.line_num,\n                        command.payload,\n                        last_itip_str\n                    );\n                }\n                Command::Send => {\n                    let mut results = String::new();\n                    match last_itip {\n                        Some(Ok(messages)) => {\n                            for message in messages {\n                                for rcpt in &message.to {\n                                    let result = match itip_snapshot(\n                                        &message.message,\n                                        &[rcpt.to_string()],\n                                        false,\n                                    ) {\n                                        Ok(itip_snapshots) => {\n                                            match store\n                                                .entry(rcpt.to_string())\n                                                .or_default()\n                                                .entry(itip_snapshots.uid.to_string())\n                                            {\n                                                Entry::Occupied(mut entry) => {\n                                                    let ical = entry.get_mut();\n                                                    let snapshots = itip_snapshot(\n                                                        ical,\n                                                        &[rcpt.to_string()],\n                                                        false,\n                                                    )\n                                                    .expect(\"Failed to create iTIP snapshot\");\n\n                                                    match itip_process_message(\n                                                        ical,\n                                                        snapshots,\n                                                        &message.message,\n                                                        itip_snapshots,\n                                                        message.from.clone(),\n                                                    ) {\n                                                        Ok(result) => match result {\n                                                            MergeResult::Actions(changes) => {\n                                                                itip_merge_changes(ical, changes);\n                                                                Ok(None)\n                                                            }\n                                                            MergeResult::Message(message) => {\n                                                                Ok(Some(message))\n                                                            }\n                                                            MergeResult::None => Ok(None),\n                                                        },\n                                                        Err(err) => Err(err),\n                                                    }\n                                                }\n                                                Entry::Vacant(entry) => {\n                                                    let mut message = message.message.clone();\n                                                    itip_import_message(&mut message)\n                                                        .expect(\"Failed to import iTIP message\");\n                                                    entry.insert(message);\n                                                    Ok(None)\n                                                }\n                                            }\n                                        }\n                                        Err(err) => Err(err),\n                                    };\n\n                                    match result {\n                                        Ok(Some(itip_message)) => {\n                                            results.push_str(\n                                                &itip_message.to_string(&mut dtstamp_map),\n                                            );\n                                        }\n                                        Ok(None) => {}\n                                        Err(e) => {\n                                            results.push_str(&format!(\"{e:?}\"));\n                                        }\n                                    }\n                                }\n                            }\n\n                            assert_eq!(\n                                results.trim(),\n                                command.payload.trim(),\n                                \"iTIP send result mismatch for {} at line {}: expected {}, got {}\",\n                                command.test_name,\n                                command.line_num,\n                                command.payload,\n                                results\n                            );\n                        }\n                        Some(Err(e)) => {\n                            panic!(\n                                \"Failed to create iTIP message for {} at line {}: {:?}\",\n                                command.test_name, command.line_num, e\n                            );\n                        }\n                        None => {\n                            panic!(\n                                \"No iTIP message to send for {} at line {}\",\n                                command.test_name, command.line_num\n                            );\n                        }\n                    }\n                    last_itip = None;\n                }\n                Command::Itip => {\n                    let mut commands = command.parameters.iter();\n                    last_itip = Some(Ok(vec![ItipMessage {\n                        from_organizer: false,\n                        from: commands\n                            .next()\n                            .expect(\"From parameter is required\")\n                            .to_string(),\n                        to: commands.map(|s| s.to_string()).collect::<Vec<_>>(),\n                        summary: ItipSummary::Invite(vec![]),\n                        message: ICalendar::parse(&command.payload)\n                            .expect(\"Failed to parse iCalendar payload\"),\n                    }]))\n                }\n                Command::Reset => {\n                    store.clear();\n                    dtstamp_map.clear();\n                    last_itip = None;\n                }\n            }\n        }\n    }\n}\n\ntrait ItipMessageExt {\n    fn to_string(&self, map: &mut AHashMap<PartialDateTime, usize>) -> String;\n}\n\nimpl ItipMessageExt for ItipMessage<ICalendar> {\n    fn to_string(&self, map: &mut AHashMap<PartialDateTime, usize>) -> String {\n        use std::fmt::Write;\n        let mut f = String::new();\n        let mut to = self.to.iter().map(|t| t.as_str()).collect::<Vec<_>>();\n        to.sort_unstable();\n        writeln!(&mut f, \"from: {}\", self.from).unwrap();\n        writeln!(&mut f, \"to: {}\", to.join(\", \")).unwrap();\n        write!(&mut f, \"summary: \").unwrap();\n        let mut fields = Vec::new();\n        match &self.summary {\n            ItipSummary::Invite(itip_fields) => {\n                writeln!(&mut f, \"invite\").unwrap();\n                fields.push(itip_fields);\n            }\n            ItipSummary::Update {\n                method,\n                current,\n                previous,\n            } => {\n                writeln!(&mut f, \"update {}\", method.as_str()).unwrap();\n                fields.push(current);\n                fields.push(previous);\n            }\n            ItipSummary::Cancel(itip_fields) => {\n                writeln!(&mut f, \"cancel\").unwrap();\n                fields.push(itip_fields);\n            }\n            ItipSummary::Rsvp { part_stat, current } => {\n                writeln!(&mut f, \"rsvp {}\", part_stat.as_str()).unwrap();\n                fields.push(current);\n            }\n        }\n        for (pos, fields) in fields.into_iter().enumerate() {\n            let prefix = if pos > 0 { \"~summary.\" } else { \"summary.\" };\n            let mut fields = fields\n                .iter()\n                .map(|f| format!(\"{}: {:?}\", f.name.as_str().to_lowercase(), f.value))\n                .collect::<Vec<_>>();\n            fields.sort_unstable();\n            for field in fields {\n                writeln!(&mut f, \"{prefix}{}\", field).unwrap();\n            }\n        }\n\n        write!(&mut f, \"{}\", normalize_ical(self.message.clone(), map)).unwrap();\n        f\n    }\n}\n\nfn normalize_ical(mut ical: ICalendar, map: &mut AHashMap<PartialDateTime, usize>) -> String {\n    let mut comps = ical\n        .components\n        .iter()\n        .enumerate()\n        .filter(|(comp_id, _)| {\n            ical.components[0]\n                .component_ids\n                .contains(&(*comp_id as u32))\n        })\n        .collect::<Vec<_>>();\n    comps.sort_unstable_by_key(|(_, comp)| *comp);\n    ical.components[0].component_ids = comps.iter().map(|(comp_id, _)| *comp_id as u32).collect();\n\n    for comp in &mut ical.components {\n        for entry in &mut comp.entries {\n            if let (ICalendarProperty::Dtstamp, Some(ICalendarValue::PartialDateTime(dt))) =\n                (&entry.name, entry.values.first())\n            {\n                if let Some(index) = map.get(dt) {\n                    entry.values = vec![ICalendarValue::Integer(*index as i64)];\n                } else {\n                    let index = map.len();\n                    map.insert(dt.as_ref().clone(), index);\n                    entry.values = vec![ICalendarValue::Integer(index as i64)];\n                }\n            }\n        }\n        comp.entries.sort_unstable();\n    }\n    ical.to_string().replace(\"\\r\\n\", \"\\n\")\n}\n"
  },
  {
    "path": "tests/src/webdav/cal_query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::WebDavTest;\nuse ahash::AHashSet;\nuse calcard::{common::timezone::Tz, icalendar::ICalendar};\nuse groupware::{\n    DavResourceName,\n    calendar::{CalendarEventData, alarm::ExpandAlarm, expand::CalendarEventExpansion},\n};\nuse hyper::StatusCode;\nuse store::write::serialize::rkyv_unarchive;\nuse types::TimeRange;\n\npub async fn test(test: &WebDavTest) {\n    println!(\"Running REPORT calendar-query & free-busy-query tests...\");\n    let client = test.client(\"john\");\n    let cal_path = format!(\"{}/john/default/\", DavResourceName::Cal.base_path());\n\n    #[allow(clippy::never_loop)]\n    for (num, ics) in [\n        (1, ICAL_RFC_ABCD1_ICS),\n        (2, ICAL_RFC_ABCD2_ICS),\n        (3, ICAL_RFC_ABCD3_ICS),\n        (4, ICAL_RFC_ABCD4_ICS),\n        (5, ICAL_RFC_ABCD5_ICS),\n        (6, ICAL_RFC_ABCD6_ICS),\n        (7, ICAL_RFC_ABCD7_ICS),\n        (8, ICAL_RFC_ABCD8_ICS),\n    ] {\n        roundtrip_expansion(ics, false);\n        client\n            .request(\"PUT\", &rfc_file_name(num), ics)\n            .await\n            .with_status(StatusCode::CREATED);\n    }\n\n    // Test 1: Partial Retrieval of Events by Time Range\n    let response = client\n        .request(\"REPORT\", &cal_path, REPORT_1)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(2).as_str(), rfc_file_name(3).as_str()])\n        .into_propfind_response(None);\n    response\n        .properties(&rfc_file_name(2))\n        .calendar_data()\n        .with_values([REPORT_1_EXPECTED_ABCD2.replace('\\n', \"\\r\\n\").as_str()]);\n    response\n        .properties(&rfc_file_name(3))\n        .calendar_data()\n        .with_values([REPORT_1_EXPECTED_ABCD3.replace('\\n', \"\\r\\n\").as_str()]);\n\n    // Test 2: Partial Retrieval of Recurring Events\n    let response = client\n        .request(\"REPORT\", &cal_path, REPORT_2)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(2).as_str(), rfc_file_name(3).as_str()])\n        .into_propfind_response(None);\n    response\n        .properties(&rfc_file_name(2))\n        .calendar_data()\n        .with_values([REPORT_2_EXPECTED_ABCD2.replace('\\n', \"\\r\\n\").as_str()]);\n    response\n        .properties(&rfc_file_name(3))\n        .calendar_data()\n        .with_values([REPORT_2_EXPECTED_ABCD3.replace('\\n', \"\\r\\n\").as_str()]);\n\n    // Test 3: Expanded Retrieval of Recurring Events\n    let response = client\n        .request(\"REPORT\", &cal_path, REPORT_3)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(2).as_str(), rfc_file_name(3).as_str()])\n        .into_propfind_response(None);\n    response\n        .properties(&rfc_file_name(2))\n        .calendar_data()\n        .with_values([REPORT_3_EXPECTED_ABCD2.replace('\\n', \"\\r\\n\").as_str()]);\n    response\n        .properties(&rfc_file_name(3))\n        .calendar_data()\n        .with_values([REPORT_3_EXPECTED_ABCD3.replace('\\n', \"\\r\\n\").as_str()]);\n\n    // Test 4: Partial Retrieval of Stored Free Busy Components\n    let response = client\n        .request(\"REPORT\", &cal_path, REPORT_4)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(8).as_str()])\n        .into_propfind_response(None);\n    response\n        .properties(&rfc_file_name(8))\n        .calendar_data()\n        .with_values([REPORT_4_EXPECTED_ABCD8.replace('\\n', \"\\r\\n\").as_str()]);\n\n    // Test 5: Retrieval of To-Dos by Alarm Time Range\n    let response = client\n        .request(\"REPORT\", &cal_path, REPORT_5)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(5).as_str()])\n        .into_propfind_response(None);\n    response\n        .properties(&rfc_file_name(5))\n        .calendar_data()\n        .with_values([ICAL_RFC_ABCD5_ICS.replace('\\n', \"\\r\\n\").as_str()]);\n\n    // Test 6: Retrieval of Event by UID\n    client\n        .request(\"REPORT\", &cal_path, REPORT_6)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(3).as_str()])\n        .into_propfind_response(None);\n\n    // Test 7: Retrieval of Events by PARTSTAT\n    client\n        .request(\"REPORT\", &cal_path, REPORT_7)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(3).as_str()])\n        .into_propfind_response(None);\n\n    // Test 8: Retrieval of Events Only\n    client\n        .request(\"REPORT\", &cal_path, REPORT_8)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([\n            rfc_file_name(1).as_str(),\n            rfc_file_name(2).as_str(),\n            rfc_file_name(3).as_str(),\n        ])\n        .into_propfind_response(None);\n\n    // Test 9: Retrieval of All Pending To-Dos\n    client\n        .request(\"REPORT\", &cal_path, REPORT_9)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([rfc_file_name(4).as_str(), rfc_file_name(5).as_str()])\n        .into_propfind_response(None);\n\n    // Test 10: Successful CALDAV:free-busy-query REPORT\n    assert_eq!(\n        remove_dtstamp(\n            client\n                .request(\"REPORT\", &cal_path, REPORT_10)\n                .await\n                .with_status(StatusCode::OK)\n                .body\n                .as_ref()\n                .unwrap()\n        ),\n        remove_dtstamp(REPORT_10_RESPONSE)\n    );\n    assert_eq!(\n        remove_dtstamp(\n            client\n                .request(\"REPORT\", &cal_path, REPORT_11)\n                .await\n                .with_status(StatusCode::OK)\n                .body\n                .as_ref()\n                .unwrap()\n        ),\n        remove_dtstamp(REPORT_11_RESPONSE)\n    );\n\n    client.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\n#[test]\n#[ignore]\nfn ical_roundtrip_expansion() {\n    for entry in std::fs::read_dir(\"/Users/me/code/calcard/resources/ical\").unwrap() {\n        let entry = entry.unwrap();\n        let path = entry.path();\n        if path.extension().is_some_and(|ext| ext == \"ics\") {\n            println!(\"Testing: {:?}\", path);\n            let input = match String::from_utf8(std::fs::read(&path).unwrap()) {\n                Ok(input) => input,\n                Err(err) => {\n                    // ISO-8859-1\n                    err.as_bytes()\n                        .iter()\n                        .map(|&b| b as char)\n                        .collect::<String>()\n                }\n            };\n            roundtrip_expansion(&input, true);\n        }\n    }\n}\n\nfn roundtrip_expansion(ics: &str, ignore_errors: bool) {\n    let ical = if let Ok(ical) = ICalendar::parse(ics) {\n        ical\n    } else if ignore_errors {\n        return;\n    } else {\n        panic!(\"Failed to parse ICalendar {}\", ics);\n    };\n    let expanded = ical.expand_dates(Tz::UTC, 100);\n    if !ignore_errors {\n        assert!(expanded.errors.is_empty());\n    }\n    let mut min_utc = i64::MAX;\n    let mut max_utc = i64::MIN;\n    let mut events = expanded\n        .events\n        .into_iter()\n        .enumerate()\n        .map(|(i, e)| {\n            let e = e.try_into_date_time().unwrap();\n            let start = e.start.timestamp();\n            let end = e.end.timestamp();\n            let mut min = std::cmp::min(start, end);\n            let mut max = std::cmp::max(start, end);\n\n            for alarm in ical.alarms_for_id(e.comp_id) {\n                if let Some(alarm_time) = alarm\n                    .expand_alarm(0, 0)\n                    .and_then(|alarm| alarm.delta.to_timestamp(start, end, Tz::UTC))\n                {\n                    if alarm_time < min {\n                        min = alarm_time;\n                    }\n\n                    if alarm_time > max {\n                        max = alarm_time;\n                    }\n                }\n            }\n\n            if min < min_utc {\n                min_utc = min;\n            }\n            if max > max_utc {\n                max_utc = max;\n            }\n            CalendarEventExpansion {\n                comp_id: e.comp_id,\n                expansion_id: i as u32,\n                start,\n                end,\n            }\n        })\n        .collect::<Vec<_>>();\n\n    // Verify min/max UTC timestamps\n    let event_data = CalendarEventData::new(ical, Tz::UTC, 100, &mut None);\n    let from_time = event_data.base_time_utc as i64 + event_data.base_offset;\n    let to_time = from_time + event_data.duration as i64;\n\n    if min_utc != i64::MAX {\n        assert_eq!(\n            from_time,\n            min_utc,\n            \"diff: {}, failed for {}\",\n            from_time - min_utc,\n            ics\n        );\n        assert_eq!(\n            to_time,\n            max_utc,\n            \"diff: {}, failed for {}\",\n            to_time - max_utc,\n            ics\n        );\n    }\n\n    // Validate archive expansion\n    let expanded_bytes = rkyv::to_bytes::<rkyv::rancor::Error>(&event_data).unwrap();\n    let expanded_archive = rkyv_unarchive::<CalendarEventData>(&expanded_bytes).unwrap();\n    let mut events_archive = expanded_archive\n        .expand(\n            Tz::UTC,\n            TimeRange {\n                start: i64::MIN,\n                end: i64::MAX,\n            },\n        )\n        .unwrap();\n\n    assert_eq!(\n        events_archive,\n        event_data\n            .expand_from_ids(\n                &mut events\n                    .iter()\n                    .map(|e| e.expansion_id)\n                    .collect::<AHashSet<_>>(),\n                Tz::UTC\n            )\n            .unwrap()\n    );\n    events.sort_by(|a, b| {\n        if a.comp_id == b.comp_id {\n            a.start.cmp(&b.start)\n        } else {\n            a.comp_id.cmp(&b.comp_id)\n        }\n    });\n    events_archive.sort_by(|a, b| {\n        if a.comp_id == b.comp_id {\n            a.start.cmp(&b.start)\n        } else {\n            a.comp_id.cmp(&b.comp_id)\n        }\n    });\n    for event in events.iter_mut().chain(events_archive.iter_mut()) {\n        event.expansion_id = 0;\n    }\n\n    assert_eq!(events, events_archive);\n}\n\nfn rfc_file_name(num: usize) -> String {\n    format!(\n        \"{}/john/default/abcd{num}.ics\",\n        DavResourceName::Cal.base_path()\n    )\n}\n\nconst REPORT_1: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                 xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <D:getetag/>\n       <C:calendar-data>\n         <C:comp name=\"VCALENDAR\">\n           <C:prop name=\"VERSION\"/>\n           <C:comp name=\"VEVENT\">\n             <C:prop name=\"SUMMARY\"/>\n             <C:prop name=\"UID\"/>\n             <C:prop name=\"DTSTART\"/>\n             <C:prop name=\"DTEND\"/>\n             <C:prop name=\"DURATION\"/>\n             <C:prop name=\"RRULE\"/>\n             <C:prop name=\"RDATE\"/>\n             <C:prop name=\"EXRULE\"/>\n             <C:prop name=\"EXDATE\"/>\n             <C:prop name=\"RECURRENCE-ID\"/>\n           </C:comp>\n           <C:comp name=\"VTIMEZONE\"/>\n         </C:comp>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:time-range start=\"20060104T000000Z\"\n                         end=\"20060105T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_1_EXPECTED_ABCD2: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nLAST-MODIFIED:20040110T032845Z\nTZID:US/Eastern\nBEGIN:DAYLIGHT\nDTSTART:20000404T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:20001026T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060102T120000\nDURATION:PT1H\nRRULE:FREQ=DAILY;COUNT=5\nSUMMARY:Event #2\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060104T140000\nDURATION:PT1H\nRECURRENCE-ID;TZID=US/Eastern:20060104T120000\nSUMMARY:Event #2 bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060106T140000\nDURATION:PT1H\nRECURRENCE-ID;TZID=US/Eastern:20060106T120000\nSUMMARY:Event #2 bis bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst REPORT_1_EXPECTED_ABCD3: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nLAST-MODIFIED:20040110T032845Z\nTZID:US/Eastern\nBEGIN:DAYLIGHT\nDTSTART:20000404T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:20001026T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nDTSTART;TZID=US/Eastern:20060104T100000\nDURATION:PT1H\nSUMMARY:Event #3\nUID:DC6C50A017428C5216A2F1CD@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst REPORT_2: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <C:calendar-data>\n         <C:limit-recurrence-set start=\"20060103T000000Z\"\n                                 end=\"20060105T000000Z\"/>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:time-range start=\"20060103T000000Z\"\n                         end=\"20060105T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_2_EXPECTED_ABCD2: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nLAST-MODIFIED:20040110T032845Z\nTZID:US/Eastern\nBEGIN:DAYLIGHT\nDTSTART:20000404T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:20001026T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nDTSTAMP:20060206T001121Z\nDTSTART;TZID=US/Eastern:20060102T120000\nDURATION:PT1H\nRRULE:FREQ=DAILY;COUNT=5\nSUMMARY:Event #2\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nBEGIN:VEVENT\nDTSTAMP:20060206T001121Z\nDTSTART;TZID=US/Eastern:20060104T140000\nDURATION:PT1H\nRECURRENCE-ID;TZID=US/Eastern:20060104T120000\nSUMMARY:Event #2 bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst REPORT_2_EXPECTED_ABCD3: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nLAST-MODIFIED:20040110T032845Z\nTZID:US/Eastern\nBEGIN:DAYLIGHT\nDTSTART:20000404T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:20001026T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nATTENDEE;PARTSTAT=ACCEPTED;ROLE=CHAIR:mailto:cyrus@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:lisa@example.com\nDTSTAMP:20060206T001220Z\nDTSTART;TZID=US/Eastern:20060104T100000\nDURATION:PT1H\nLAST-MODIFIED:20060206T001330Z\nORGANIZER:mailto:cyrus@example.com\nSEQUENCE:1\nSTATUS:TENTATIVE\nSUMMARY:Event #3\nUID:DC6C50A017428C5216A2F1CD@example.com\nX-ABC-GUID:E1CX5Dr-0007ym-Hz@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst REPORT_3: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <C:calendar-data>\n         <C:comp name=\"VCALENDAR\">\n           <C:comp name=\"VEVENT\"/>\n         </C:comp>\n         <C:expand start=\"20060103T000000Z\"\n                   end=\"20060105T000000Z\"/>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:time-range start=\"20060103T000000Z\"\n                         end=\"20060105T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_3_EXPECTED_ABCD2: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VEVENT\nDTSTART:20060103T170000Z\nRECURRENCE-ID:20060103T170000Z\nDTSTAMP:20060206T001121Z\nDURATION:PT1H\nSUMMARY:Event #2\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nBEGIN:VEVENT\nDTSTART:20060104T190000Z\nRECURRENCE-ID:20060104T190000Z\nDTSTAMP:20060206T001121Z\nDURATION:PT1H\nSUMMARY:Event #2 bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst REPORT_3_EXPECTED_ABCD3: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VEVENT\nDTSTART:20060104T150000Z\nATTENDEE;PARTSTAT=ACCEPTED;ROLE=CHAIR:mailto:cyrus@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:lisa@example.com\nDTSTAMP:20060206T001220Z\nDURATION:PT1H\nLAST-MODIFIED:20060206T001330Z\nORGANIZER:mailto:cyrus@example.com\nSEQUENCE:1\nSTATUS:TENTATIVE\nSUMMARY:Event #3\nUID:DC6C50A017428C5216A2F1CD@example.com\nX-ABC-GUID:E1CX5Dr-0007ym-Hz@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst REPORT_4: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:D=\"DAV:\"\n                 xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <C:calendar-data>\n         <C:limit-freebusy-set start=\"20060102T000000Z\"\n                                 end=\"20060103T000000Z\"/>\n       </C:calendar-data>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VFREEBUSY\">\n           <C:time-range start=\"20060102T000000Z\"\n                           end=\"20060103T000000Z\"/>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_4_EXPECTED_ABCD8: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VFREEBUSY\nORGANIZER;CN=\"Bernard Desruisseaux\":mailto:bernard@example.com\nUID:76ef34-54a3d2@example.com\nDTSTAMP:20050530T123421Z\nDTSTART:20060101T000000Z\nDTEND:20060108T000000Z\nFREEBUSY;FBTYPE=BUSY-TENTATIVE:20060102T100000Z/20060102T120000Z\nEND:VFREEBUSY\nEND:VCALENDAR\n\"#;\n\nconst REPORT_5: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VTODO\">\n           <C:comp-filter name=\"VALARM\">\n             <C:time-range start=\"20060106T100000Z\"\n                             end=\"20060107T100000Z\"/>\n           </C:comp-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_6: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:prop-filter name=\"UID\">\n             <C:text-match collation=\"i;octet\"\n             >DC6C50A017428C5216A2F1CD@example.com</C:text-match>\n           </C:prop-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_7: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\">\n           <C:prop-filter name=\"ATTENDEE\">\n             <C:text-match collation=\"i;ascii-casemap\"\n              >mailto:lisa@example.com</C:text-match>\n             <C:param-filter name=\"PARTSTAT\">\n               <C:text-match collation=\"i;ascii-casemap\"\n                >NEEDS-ACTION</C:text-match>\n             </C:param-filter>\n           </C:prop-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_8: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VEVENT\"/>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_9: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop xmlns:D=\"DAV:\">\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     <C:filter>\n       <C:comp-filter name=\"VCALENDAR\">\n         <C:comp-filter name=\"VTODO\">\n           <C:prop-filter name=\"COMPLETED\">\n             <C:is-not-defined/>\n           </C:prop-filter>\n           <C:prop-filter name=\"STATUS\">\n             <C:text-match\n                negate-condition=\"yes\">CANCELLED</C:text-match>\n           </C:prop-filter>\n         </C:comp-filter>\n       </C:comp-filter>\n     </C:filter>\n   </C:calendar-query>\n\"#;\n\nconst REPORT_10: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:free-busy-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <C:time-range start=\"20060104T140000Z\"\n                     end=\"20060105T220000Z\"/>\n   </C:free-busy-query>\n\"#;\n\nconst REPORT_10_RESPONSE: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nBEGIN:VFREEBUSY\nDTSTART:20060104T140000Z\nDTEND:20060105T220000Z\nFREEBUSY;FBTYPE=BUSY-TENTATIVE:20060104T150000Z/20060104T160000Z\nFREEBUSY;FBTYPE=BUSY:20060104T190000Z/20060104T200000Z,20060105T170000Z/20060105T180000Z\nFREEBUSY;FBTYPE=BUSY-UNAVAILABLE:20060105T100000Z/20060105T120000Z\nEND:VFREEBUSY\nEND:VCALENDAR\n\"#;\n\nconst REPORT_11: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:free-busy-query xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <C:time-range start=\"20060101T000000Z\"\n                     end=\"20060104T140000Z\"/>\n   </C:free-busy-query>\n\"#;\n\nconst REPORT_11_RESPONSE: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Stalwart Labs LLC//Stalwart Server//EN\nBEGIN:VFREEBUSY\nDTSTART:20060101T000000Z\nDTEND:20060104T140000Z\nDTSTAMP:20250505T105255Z\nFREEBUSY;FBTYPE=BUSY-TENTATIVE:20060102T100000Z/20060102T120000Z\nFREEBUSY;FBTYPE=BUSY:20060102T150000Z/20060102T160000Z,20060102T170000Z/20060102T180000Z,\n 20060103T100000Z/20060103T120000Z,20060103T170000Z/20060103T180000Z,20060104T100000Z/20060104T120000Z\nEND:VFREEBUSY\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD1_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nLAST-MODIFIED:20040110T032845Z\nTZID:US/Eastern\nBEGIN:DAYLIGHT\nDTSTART:20000404T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:20001026T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nDTSTAMP:20060206T001102Z\nDTSTART;TZID=US/Eastern:20060102T100000\nDURATION:PT1H\nSUMMARY:Event #1\nDescription:Go Steelers!\nUID:74855313FA803DA593CD579A@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD2_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nLAST-MODIFIED:20040110T032845Z\nTZID:US/Eastern\nBEGIN:DAYLIGHT\nDTSTART:20000404T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:20001026T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nDTSTAMP:20060206T001121Z\nDTSTART;TZID=US/Eastern:20060102T120000\nDURATION:PT1H\nRRULE:FREQ=DAILY;COUNT=5\nSUMMARY:Event #2\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nBEGIN:VEVENT\nDTSTAMP:20060206T001121Z\nDTSTART;TZID=US/Eastern:20060104T140000\nDURATION:PT1H\nRECURRENCE-ID;TZID=US/Eastern:20060104T120000\nSUMMARY:Event #2 bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nBEGIN:VEVENT\nDTSTAMP:20060206T001121Z\nDTSTART;TZID=US/Eastern:20060106T140000\nDURATION:PT1H\nRECURRENCE-ID;TZID=US/Eastern:20060106T120000\nSUMMARY:Event #2 bis bis\nUID:00959BC664CA650E933C892C@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD3_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTIMEZONE\nLAST-MODIFIED:20040110T032845Z\nTZID:US/Eastern\nBEGIN:DAYLIGHT\nDTSTART:20000404T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZNAME:EDT\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nEND:DAYLIGHT\nBEGIN:STANDARD\nDTSTART:20001026T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZNAME:EST\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nEND:STANDARD\nEND:VTIMEZONE\nBEGIN:VEVENT\nATTENDEE;PARTSTAT=ACCEPTED;ROLE=CHAIR:mailto:cyrus@example.com\nATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:lisa@example.com\nDTSTAMP:20060206T001220Z\nDTSTART;TZID=US/Eastern:20060104T100000\nDURATION:PT1H\nLAST-MODIFIED:20060206T001330Z\nORGANIZER:mailto:cyrus@example.com\nSEQUENCE:1\nSTATUS:TENTATIVE\nSUMMARY:Event #3\nUID:DC6C50A017428C5216A2F1CD@example.com\nX-ABC-GUID:E1CX5Dr-0007ym-Hz@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD4_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTODO\nDTSTAMP:20060205T235335Z\nDUE;VALUE=DATE:20060104\nSTATUS:NEEDS-ACTION\nSUMMARY:Task #1\nUID:DDDEEB7915FA61233B861457@example.com\nBEGIN:VALARM\nACTION:AUDIO\nTRIGGER;RELATED=START:-PT10M\nEND:VALARM\nEND:VTODO\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD5_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTODO\nDTSTAMP:20060205T235300Z\nDUE;TZID=US/Eastern:20060106T120000\nLAST-MODIFIED:20060205T235308Z\nSEQUENCE:1\nSTATUS:NEEDS-ACTION\nSUMMARY:Task #2\nUID:E10BA47467C5C69BB74E8720@example.com\nBEGIN:VALARM\nACTION:AUDIO\nTRIGGER;RELATED=START:-PT10M\nEND:VALARM\nEND:VTODO\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD6_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTODO\nCOMPLETED:20051223T122322Z\nDTSTAMP:20060205T235400Z\nDUE;VALUE=DATE:20051225\nLAST-MODIFIED:20060205T235308Z\nSEQUENCE:1\nSTATUS:COMPLETED\nSUMMARY:Task #3\nUID:E10BA47467C5C69BB74E8722@example.com\nEND:VTODO\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD7_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VTODO\nDTSTAMP:20060205T235600Z\nDUE;VALUE=DATE:20060101\nLAST-MODIFIED:20060205T235308Z\nSEQUENCE:1\nSTATUS:CANCELLED\nSUMMARY:Task #4\nUID:E10BA47467C5C69BB74E8725@example.com\nEND:VTODO\nEND:VCALENDAR\n\"#;\n\nconst ICAL_RFC_ABCD8_ICS: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VFREEBUSY\nORGANIZER;CN=\"Bernard Desruisseaux\":mailto:bernard@example.com\nUID:76ef34-54a3d2@example.com\nDTSTAMP:20050530T123421Z\nDTSTART:20060101T000000Z\nDTEND:20060108T000000Z\nFREEBUSY:20050531T230000Z/20050601T010000Z\nFREEBUSY;FBTYPE=BUSY-TENTATIVE:20060102T100000Z/20060102T120000Z\nFREEBUSY:20060103T100000Z/20060103T120000Z\nFREEBUSY:20060104T100000Z/20060104T120000Z\nFREEBUSY;FBTYPE=BUSY-UNAVAILABLE:20060105T100000Z/20060105T120000Z\nFREEBUSY:20060106T100000Z/20060106T120000Z\nEND:VFREEBUSY\nEND:VCALENDAR\n\"#;\n\nfn remove_dtstamp(ics: &str) -> AHashSet<String> {\n    let mut result = AHashSet::new();\n    for line in ics.lines() {\n        if !line.starts_with(\"DTSTAMP:\") {\n            result.insert(line.to_string());\n        }\n    }\n    result\n}\n"
  },
  {
    "path": "tests/src/webdav/cal_scheduling.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::WebDavTest;\nuse crate::{\n    jmap::mail::mailbox::destroy_all_mailboxes_for_account,\n    webdav::{DummyWebDavClient, prop::ALL_DAV_PROPERTIES},\n};\nuse calcard::{\n    common::timezone::Tz,\n    icalendar::{\n        ICalendarDay, ICalendarFrequency, ICalendarMethod, ICalendarParticipationStatus,\n        ICalendarProperty, ICalendarRecurrenceRule, ICalendarWeekday,\n    },\n};\nuse common::{Server, auth::AccessToken};\nuse dav_proto::schema::property::{CalDavProperty, DavProperty, WebDavProperty};\nuse email::cache::MessageCacheFetch;\nuse groupware::{\n    cache::GroupwareCache,\n    scheduling::{\n        ArchivedItipSummary, ItipField, ItipParticipant, ItipSummary, ItipTime, ItipValue,\n    },\n};\nuse hyper::StatusCode;\nuse mail_parser::{DateTime, MessageParser};\nuse services::task_manager::imip::build_itip_template;\nuse std::str::FromStr;\nuse store::write::now;\nuse types::collection::SyncCollection;\n\npub async fn test(test: &WebDavTest) {\n    println!(\"Running calendar scheduling tests...\");\n    let bill_client = test.client(\"bill\");\n    let jane_client = test.client(\"jane\");\n    let john_client = test.client(\"john\");\n\n    // Validate hierarchy of scheduling resources\n    let response = jane_client\n        .propfind_with_headers(\"/dav/itip/jane/\", ALL_DAV_PROPERTIES, [(\"depth\", \"1\")])\n        .await;\n    let properties = response\n        .with_hrefs([\n            \"/dav/itip/jane/\",\n            \"/dav/itip/jane/inbox/\",\n            \"/dav/itip/jane/outbox/\",\n        ])\n        .properties(\"/dav/itip/jane/inbox/\");\n\n    // Validate schedule inbox properties\n    properties\n        .get(DavProperty::WebDav(WebDavProperty::ResourceType))\n        .with_values([\"D:collection\", \"A:schedule-inbox\"]);\n    properties\n        .get(DavProperty::CalDav(\n            CalDavProperty::ScheduleDefaultCalendarURL,\n        ))\n        .with_values([\"D:href:/dav/cal/jane/default/\"])\n        .with_status(StatusCode::OK);\n    properties\n        .get(DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet))\n        .with_some_values([\n            \"D:supported-privilege.D:privilege.D:all\",\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:privilege.D:read\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:privilege.A:schedule-deliver\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:supported-privilege.D:privilege.A:schedule-deliver-invite\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:supported-privilege.D:privilege.A:schedule-deliver-reply\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:supported-privilege.D:privilege.A:schedule-query-freebusy\"\n            ),\n        ]);\n    properties\n        .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n        .with_values([\n            \"D:privilege.D:write-properties\",\n            \"D:privilege.A:schedule-deliver-invite\",\n            \"D:privilege.D:write-content\",\n            \"D:privilege.A:schedule-deliver\",\n            \"D:privilege.D:read\",\n            \"D:privilege.D:all\",\n            \"D:privilege.A:schedule-query-freebusy\",\n            \"D:privilege.D:read-acl\",\n            \"D:privilege.D:write-acl\",\n            \"D:privilege.A:schedule-deliver-reply\",\n            \"D:privilege.D:write\",\n            \"D:privilege.D:read-current-user-privilege-set\",\n        ]);\n\n    // Validate schedule outbox properties\n    let properties = response.properties(\"/dav/itip/jane/outbox/\");\n    properties\n        .get(DavProperty::WebDav(WebDavProperty::ResourceType))\n        .with_values([\"D:collection\", \"A:schedule-outbox\"]);\n    properties\n        .get(DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet))\n        .with_some_values([\n            \"D:supported-privilege.D:privilege.D:all\",\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:privilege.D:read\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:privilege.A:schedule-send\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:supported-privilege.D:privilege.A:schedule-send-invite\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:supported-privilege.D:privilege.A:schedule-send-reply\"\n            ),\n            concat!(\n                \"D:supported-privilege.D:supported-privilege.\",\n                \"D:supported-privilege.D:privilege.A:schedule-send-freebusy\"\n            ),\n        ]);\n    properties\n        .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n        .with_values([\n            \"D:privilege.D:write-properties\",\n            \"D:privilege.A:schedule-send-invite\",\n            \"D:privilege.D:write-content\",\n            \"D:privilege.A:schedule-send\",\n            \"D:privilege.D:read\",\n            \"D:privilege.D:all\",\n            \"D:privilege.A:schedule-send-freebusy\",\n            \"D:privilege.D:read-acl\",\n            \"D:privilege.D:write-acl\",\n            \"D:privilege.A:schedule-send-reply\",\n            \"D:privilege.D:write\",\n            \"D:privilege.D:read-current-user-privilege-set\",\n        ]);\n\n    // Send invitation to Bill and Mike\n    let test_itip = TEST_ITIP\n        .replace(\n            \"$START\",\n            &DateTime::from_timestamp(now() as i64 + 60 * 60)\n                .to_rfc3339()\n                .replace(['-', ':'], \"\"),\n        )\n        .replace(\n            \"$END\",\n            &DateTime::from_timestamp(now() as i64 + 5 * 60 * 60)\n                .to_rfc3339()\n                .replace(['-', ':'], \"\"),\n        );\n    john_client\n        .request_with_headers(\n            \"PUT\",\n            \"/dav/cal/john/default/itip.ics\",\n            [(\"content-type\", \"text/calendar; charset=utf-8\")],\n            &test_itip,\n        )\n        .await\n        .with_status(StatusCode::CREATED);\n\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n\n    // Check that the invitation was received by Bill and Mike\n    for client in [bill_client, jane_client] {\n        let messages = test\n            .server\n            .get_cached_messages(client.account_id)\n            .await\n            .unwrap();\n        assert_eq!(messages.emails.items.len(), 1);\n        let access_token = test\n            .server\n            .get_access_token(client.account_id)\n            .await\n            .unwrap();\n        let events = test\n            .server\n            .fetch_dav_resources(&access_token, client.account_id, SyncCollection::Calendar)\n            .await\n            .unwrap();\n        assert_eq!(events.resources.len(), 2);\n        let events = test\n            .server\n            .fetch_dav_resources(\n                &access_token,\n                client.account_id,\n                SyncCollection::CalendarEventNotification,\n            )\n            .await\n            .unwrap();\n        assert_eq!(events.resources.len(), 3);\n    }\n\n    // Validate iTIP\n    let itips = fetch_and_remove_itips(jane_client).await;\n    assert_eq!(itips.len(), 1);\n    let itip = itips.first().unwrap();\n    assert!(\n        itip.contains(\"SUMMARY:Lunch\") && itip.contains(\"METHOD:REQUEST\"),\n        \"failed for itip: {itip}\"\n    );\n\n    // Fetch added calendar entry\n    let cals = fetch_icals(jane_client).await;\n    assert_eq!(cals.len(), 1);\n    let cal = cals.into_iter().next().unwrap();\n\n    // Using an invalid schedule tag should fail\n    let rsvp_ical = cal.ical.replace(\n        \"PARTSTAT=NEEDS-ACTION:mailto:jane.smith\",\n        \"PARTSTAT=ACCEPTED:mailto:jane.smith\",\n    );\n    jane_client\n        .request_with_headers(\n            \"PUT\",\n            &cal.href,\n            [\n                (\"content-type\", \"text/calendar; charset=utf-8\"),\n                (\"if-schedule-tag-match\", \"\\\"9999999\\\"\"),\n            ],\n            &rsvp_ical,\n        )\n        .await\n        .with_status(StatusCode::PRECONDITION_FAILED);\n\n    // RSVP the invitation\n    jane_client\n        .request_with_headers(\n            \"PUT\",\n            &cal.href,\n            [\n                (\"content-type\", \"text/calendar; charset=utf-8\"),\n                (\"if-schedule-tag-match\", cal.schedule_tag.as_str()),\n            ],\n            &rsvp_ical,\n        )\n        .await\n        .with_status(StatusCode::NO_CONTENT);\n\n    // Make sure that the schedule has not changed\n    assert_eq!(\n        fetch_icals(jane_client).await[0].schedule_tag,\n        cal.schedule_tag\n    );\n\n    // Check that John received the RSVP\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    test.wait_for_index().await;\n    let itips = fetch_and_remove_itips(john_client).await;\n    assert_eq!(itips.len(), 1);\n    assert!(\n        itips[0].contains(\"METHOD:REPLY\")\n            && itips[0].contains(\"PARTSTAT=ACCEPTED:mailto:jane.smith\"),\n        \"failed for itip: {}\",\n        itips[0]\n    );\n    let cals = fetch_icals(john_client).await;\n    assert_eq!(cals.len(), 1);\n    assert!(\n        cals[0]\n            .ical\n            .contains(\"PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:jane\"),\n        \"failed for cal: {}\",\n        cals[0].ical\n    );\n\n    // Changing the event name should not trigger a new iTIP\n    let updated_ical = rsvp_ical.replace(\"Lunch\", \"Dinner\");\n    jane_client\n        .request_with_headers(\n            \"PUT\",\n            &cal.href,\n            [(\"content-type\", \"text/calendar; charset=utf-8\")],\n            &updated_ical,\n        )\n        .await\n        .with_status(StatusCode::NO_CONTENT);\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    assert_eq!(\n        fetch_and_remove_itips(john_client).await,\n        Vec::<String>::new()\n    );\n\n    // Deleting the event should send a cancellation\n    jane_client\n        .request(\"DELETE\", &cal.href, \"\")\n        .await\n        .with_status(StatusCode::NO_CONTENT);\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    let itips = fetch_and_remove_itips(john_client).await;\n    assert_eq!(itips.len(), 1);\n    assert!(\n        itips[0].contains(\"METHOD:REPLY\")\n            && itips[0].contains(\"PARTSTAT=DECLINED:mailto:jane.smith\"),\n        \"failed for itip: {}\",\n        itips[0]\n    );\n    let cals = fetch_icals(john_client).await;\n    assert_eq!(cals.len(), 1);\n    let cal = cals.into_iter().next().unwrap();\n    assert!(\n        cal.ical.contains(\"PARTSTAT=DECLINED:mailto:jane\"),\n        \"failed for cal: {}\",\n        cal.ical\n    );\n\n    // Fetch Bill's email invitation and RSVP via HTTP\n    let document_id = test\n        .server\n        .get_cached_messages(bill_client.account_id)\n        .await\n        .unwrap()\n        .emails\n        .items[0]\n        .document_id;\n    let contents = test.fetch_email(bill_client.account_id, document_id).await;\n    let message = MessageParser::new().parse(&contents).unwrap();\n    let contents = message\n        .html_bodies()\n        .next()\n        .unwrap()\n        .text_contents()\n        .unwrap();\n    let url = contents\n        .split(\"href=\\\"\")\n        .filter_map(|s| {\n            let url = s.split_once('\\\"').map(|(url, _)| url)?;\n            if url.contains(\"m=ACCEPTED\") {\n                Some(url.strip_prefix(\"https://webdav.example.org\").unwrap())\n            } else {\n                None\n            }\n        })\n        .next()\n        .unwrap_or_else(|| {\n            panic!(\"Failed to find RSVP link in email contents: {contents}\");\n        });\n    let response = jane_client\n        .request(\"GET\", url, \"\")\n        .await\n        .with_status(StatusCode::OK)\n        .body\n        .unwrap();\n    assert!(\n        response.contains(\"Lunch\") && response.contains(\"RSVP has been recorded\"),\n        \"failed for response: {response}\"\n    );\n    let cals = fetch_icals(john_client).await;\n    assert_eq!(cals.len(), 1);\n    let cal = cals.into_iter().next().unwrap();\n    assert!(\n        cal.ical.contains(\"PARTSTAT=ACCEPTED:mailto:bill\"),\n        \"failed for cal: {}\",\n        cal.ical\n    );\n\n    // Test the schedule outbox\n    let test_outbox = TEST_FREEBUSY\n        .replace(\n            \"$START\",\n            &DateTime::from_timestamp(now() as i64)\n                .to_rfc3339()\n                .replace(['-', ':'], \"\"),\n        )\n        .replace(\n            \"$END\",\n            &DateTime::from_timestamp(now() as i64 + 100 * 60 * 60)\n                .to_rfc3339()\n                .replace(['-', ':'], \"\"),\n        );\n    let response = john_client\n        .request_with_headers(\n            \"POST\",\n            \"/dav/itip/john/outbox/\",\n            [(\"content-type\", \"text/calendar; charset=utf-8\")],\n            &test_outbox,\n        )\n        .await\n        .with_status(StatusCode::OK);\n    let mut account = \"\";\n    let mut found_data = false;\n    for (key, value) in &response.xml {\n        match key.as_str() {\n            \"A:schedule-response.A:response.A:recipient.D:href\" => {\n                account = value.strip_prefix(\"mailto:\").unwrap();\n            }\n            \"A:schedule-response.A:response.A:request-status\" => {\n                if account == \"unknown@example.com\" {\n                    assert_eq!(\n                        value,\n                        \"3.7;Invalid calendar user or insufficient permissions\"\n                    );\n                } else {\n                    assert_eq!(value, \"2.0;Success\");\n                }\n            }\n            \"A:schedule-response.A:response.A:calendar-data\" => {\n                assert!(\n                    value.contains(\"BEGIN:VFREEBUSY\"),\n                    \"missing freebusy data in response: {response:?}\"\n                );\n                if account == \"jdoe@example.com\" {\n                    assert!(\n                        value.contains(\"FREEBUSY;FBTYPE=BUSY:\"),\n                        \"missing freebusy data in response: {response:?}\"\n                    );\n                    found_data = true;\n                }\n            }\n            _ => {}\n        }\n    }\n    assert!(\n        found_data,\n        \"Missing calendar data in response: {response:?}\"\n    );\n\n    // Modifying john's event should only send updates to bill\n    let updated_ical = cal.ical.replace(\"Lunch\", \"Breakfast at Tiffany's\");\n    john_client\n        .request_with_headers(\n            \"PUT\",\n            &cal.href,\n            [(\"content-type\", \"text/calendar; charset=utf-8\")],\n            &updated_ical,\n        )\n        .await\n        .with_status(StatusCode::NO_CONTENT);\n\n    // Make sure that the schedule has changed\n    assert_ne!(\n        fetch_icals(john_client).await[0].schedule_tag,\n        cal.schedule_tag\n    );\n    let main_event_href = cal.href;\n\n    // Check that Bill received the update\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    test.wait_for_index().await;\n    let mut itips = fetch_and_remove_itips(bill_client).await;\n    itips.sort_unstable_by(|a, _| {\n        if a.contains(\"Lunch\") {\n            std::cmp::Ordering::Less\n        } else {\n            std::cmp::Ordering::Greater\n        }\n    });\n    assert_eq!(itips.len(), 2);\n    assert!(\n        itips[0].contains(\"METHOD:REQUEST\") && itips[0].contains(\"Lunch\"),\n        \"failed for itip: {}\",\n        itips[0]\n    );\n    assert!(\n        itips[1].contains(\"METHOD:REQUEST\") && itips[1].contains(\"Breakfast at Tiffany's\"),\n        \"failed for itip: {}\",\n        itips[1]\n    );\n    let cals = fetch_icals(bill_client).await;\n    assert_eq!(cals.len(), 1);\n    let cal = cals.into_iter().next().unwrap();\n    assert!(\n        cal.ical.contains(\"SUMMARY:Breakfast at Tiffany's\")\n            && cal.ical.contains(\"PARTSTAT=ACCEPTED:mailto:bill\"),\n        \"failed for cal: {}\",\n        cal.ical\n    );\n    let attendee_href = cal.href;\n    assert_eq!(\n        fetch_and_remove_itips(jane_client).await,\n        Vec::<String>::new()\n    );\n\n    // Removing the event should from John's calendar send a cancellation to Bill\n    john_client\n        .request(\"DELETE\", &main_event_href, \"\")\n        .await\n        .with_status(StatusCode::NO_CONTENT);\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    let itips = fetch_and_remove_itips(bill_client).await;\n    assert_eq!(itips.len(), 1);\n    assert!(\n        itips[0].contains(\"METHOD:CANCEL\") && itips[0].contains(\"STATUS:CANCELLED\"),\n        \"failed for itip: {}\",\n        itips[0]\n    );\n    let cals = fetch_icals(bill_client).await;\n    assert_eq!(cals.len(), 1);\n    let cal = cals.into_iter().next().unwrap();\n    assert!(\n        cal.ical.contains(\"STATUS:CANCELLED\"),\n        \"failed for cal: {}\",\n        cal.ical\n    );\n    assert_eq!(\n        fetch_and_remove_itips(jane_client).await,\n        Vec::<String>::new()\n    );\n\n    // Delete the event from Bill's calendar disabling schedule replies\n    bill_client\n        .request_with_headers(\"DELETE\", &attendee_href, [(\"Schedule-Reply\", \"F\")], \"\")\n        .await\n        .with_status(StatusCode::NO_CONTENT);\n    tokio::time::sleep(std::time::Duration::from_millis(200)).await;\n    assert_eq!(\n        fetch_and_remove_itips(john_client).await,\n        Vec::<String>::new()\n    );\n\n    for client in [bill_client, jane_client, john_client] {\n        client.delete_default_containers().await;\n        destroy_all_mailboxes_for_account(client.account_id).await;\n    }\n\n    test.assert_is_empty().await;\n}\n\nasync fn fetch_and_remove_itips(client: &DummyWebDavClient) -> Vec<String> {\n    let inbox_href = format!(\"/dav/itip/{}/inbox/\", client.name);\n    let response = client\n        .propfind_with_headers(&inbox_href, ALL_DAV_PROPERTIES, [(\"depth\", \"1\")])\n        .await;\n    let mut itips = vec![];\n\n    for href in response.hrefs.keys().filter(|&href| href != &inbox_href) {\n        let itip = client\n            .request(\"GET\", href, \"\")\n            .await\n            .with_status(StatusCode::OK)\n            .body\n            .expect(\"Missing body\");\n        client\n            .request(\"DELETE\", href, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        itips.push(itip);\n    }\n\n    itips\n}\n\n#[derive(Debug)]\nstruct CalEntry {\n    href: String,\n    ical: String,\n    schedule_tag: String,\n}\n\nasync fn fetch_icals(client: &DummyWebDavClient) -> Vec<CalEntry> {\n    let cal_inbox = format!(\"/dav/cal/{}/default/\", client.name);\n    let response = client\n        .propfind_with_headers(&cal_inbox, ALL_DAV_PROPERTIES, [(\"depth\", \"1\")])\n        .await;\n    let mut cals = vec![];\n\n    for href in response.hrefs.keys().filter(|&href| href != &cal_inbox) {\n        let ical = client\n            .request(\"GET\", href, \"\")\n            .await\n            .with_status(StatusCode::OK)\n            .body\n            .expect(\"Missing body\");\n        let properties = response.properties(href);\n\n        assert!(\n            !ical.contains(\"METHOD:\"),\n            \"iTIP method found in calendar entry: {ical}\"\n        );\n\n        cals.push(CalEntry {\n            href: href.to_string(),\n            ical,\n            schedule_tag: properties\n                .get(DavProperty::CalDav(CalDavProperty::ScheduleTag))\n                .value()\n                .to_string(),\n        });\n    }\n\n    cals\n}\n\npub async fn test_build_itip_templates(server: &Server) {\n    let dummy_access_token = AccessToken::from_id(0);\n\n    for (idx, summary) in [\n        ItipSummary::Invite(vec![\n            ItipField {\n                name: ICalendarProperty::Summary,\n                value: ItipValue::Text(\"Lunch\".to_string()),\n            },\n            ItipField {\n                name: ICalendarProperty::Description,\n                value: ItipValue::Text(\"Lunch at the cafe\".to_string()),\n            },\n            ItipField {\n                name: ICalendarProperty::Location,\n                value: ItipValue::Text(\"Cafe Corner\".to_string()),\n            },\n            ItipField {\n                name: ICalendarProperty::Dtstart,\n                value: ItipValue::Time(ItipTime {\n                    start: 1750616068,\n                    tz_id: Tz::from_str(\"New Zealand\").unwrap().as_id(),\n                }),\n            },\n            ItipField {\n                name: ICalendarProperty::Attendee,\n                value: ItipValue::Participants(vec![\n                    ItipParticipant {\n                        email: \"jdoe@domain.com\".to_string(),\n                        name: Some(\"John Doe\".to_string()),\n                        is_organizer: true,\n                    },\n                    ItipParticipant {\n                        email: \"jane@domain.com\".to_string(),\n                        name: Some(\"Jane Smith\".to_string()),\n                        is_organizer: false,\n                    },\n                ]),\n            },\n        ]),\n        ItipSummary::Cancel(vec![\n            ItipField {\n                name: ICalendarProperty::Summary,\n                value: ItipValue::Text(\"Lunch\".to_string()),\n            },\n            ItipField {\n                name: ICalendarProperty::Description,\n                value: ItipValue::Text(\"Lunch at the cafe\".to_string()),\n            },\n            ItipField {\n                name: ICalendarProperty::Location,\n                value: ItipValue::Text(\"Cafe Corner\".to_string()),\n            },\n            ItipField {\n                name: ICalendarProperty::Dtstart,\n                value: ItipValue::Time(ItipTime {\n                    start: 1750616068,\n                    tz_id: Tz::from_str(\"New Zealand\").unwrap().as_id(),\n                }),\n            },\n        ]),\n        ItipSummary::Rsvp {\n            part_stat: ICalendarParticipationStatus::Accepted,\n            current: vec![\n                ItipField {\n                    name: ICalendarProperty::Summary,\n                    value: ItipValue::Text(\"Lunch\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Description,\n                    value: ItipValue::Text(\"Lunch at the cafe\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Location,\n                    value: ItipValue::Text(\"Cafe Corner\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Dtstart,\n                    value: ItipValue::Time(ItipTime {\n                        start: 1750616068,\n                        tz_id: Tz::from_str(\"New Zealand\").unwrap().as_id(),\n                    }),\n                },\n                ItipField {\n                    name: ICalendarProperty::Rrule,\n                    value: ItipValue::Rrule(Box::new(ICalendarRecurrenceRule {\n                        freq: ICalendarFrequency::Weekly,\n                        until: None,\n                        count: Some(2),\n                        interval: Some(3),\n                        bysecond: Default::default(),\n                        byday: vec![\n                            ICalendarDay {\n                                ordwk: None,\n                                weekday: ICalendarWeekday::Monday,\n                            },\n                            ICalendarDay {\n                                ordwk: None,\n                                weekday: ICalendarWeekday::Wednesday,\n                            },\n                        ],\n                        ..Default::default()\n                    })),\n                },\n            ],\n        },\n        ItipSummary::Rsvp {\n            part_stat: ICalendarParticipationStatus::Declined,\n            current: vec![\n                ItipField {\n                    name: ICalendarProperty::Summary,\n                    value: ItipValue::Text(\"Lunch\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Description,\n                    value: ItipValue::Text(\"Lunch at the cafe\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Location,\n                    value: ItipValue::Text(\"Cafe Corner\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Dtstart,\n                    value: ItipValue::Time(ItipTime {\n                        start: 1750616068,\n                        tz_id: Tz::from_str(\"New Zealand\").unwrap().as_id(),\n                    }),\n                },\n            ],\n        },\n        ItipSummary::Update {\n            method: ICalendarMethod::Request,\n            current: vec![\n                ItipField {\n                    name: ICalendarProperty::Summary,\n                    value: ItipValue::Text(\"Lunch\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Description,\n                    value: ItipValue::Text(\"Lunch at the cafe\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Location,\n                    value: ItipValue::Text(\"Cafe Corner\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Dtstart,\n                    value: ItipValue::Time(ItipTime {\n                        start: 1750616068,\n                        tz_id: Tz::from_str(\"New Zealand\").unwrap().as_id(),\n                    }),\n                },\n                ItipField {\n                    name: ICalendarProperty::Attendee,\n                    value: ItipValue::Participants(vec![\n                        ItipParticipant {\n                            email: \"jdoe@domain.com\".to_string(),\n                            name: Some(\"John Doe\".to_string()),\n                            is_organizer: true,\n                        },\n                        ItipParticipant {\n                            email: \"jane@domain.com\".to_string(),\n                            name: Some(\"Jane Smith\".to_string()),\n                            is_organizer: false,\n                        },\n                    ]),\n                },\n            ],\n            previous: vec![\n                ItipField {\n                    name: ICalendarProperty::Summary,\n                    value: ItipValue::Text(\"Dinner\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Description,\n                    value: ItipValue::Text(\"Dinner at the cafe\".to_string()),\n                },\n                ItipField {\n                    name: ICalendarProperty::Dtstart,\n                    value: ItipValue::Time(ItipTime {\n                        start: 1750916068,\n                        tz_id: Tz::from_str(\"New Zealand\").unwrap().as_id(),\n                    }),\n                },\n            ],\n        },\n    ]\n    .into_iter()\n    .enumerate()\n    {\n        let bytes = rkyv::to_bytes::<rkyv::rancor::Error>(&summary)\n            .unwrap()\n            .to_vec();\n        let summary = rkyv::access::<ArchivedItipSummary, rkyv::rancor::Error>(&bytes).unwrap();\n\n        let html = build_itip_template(\n            server,\n            &dummy_access_token,\n            0,\n            1,\n            \"john.doe@example.org\",\n            \"jane.smith@example.net\",\n            summary,\n            \"124\",\n        )\n        .await;\n\n        println!(\"iTIP template {idx}: {}\", html.subject);\n        std::fs::write(format!(\"itip_template_{idx}.html\"), html.body)\n            .expect(\"Failed to write iTIP template to file\");\n    }\n}\n\nconst TEST_ITIP: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nBEGIN:VEVENT\nUID:9263504FD3AD\nSEQUENCE:0\nDTSTART:$START\nDTEND:$END\nDTSTAMP:20090602T170000Z\nTRANSP:OPAQUE\nSUMMARY:Lunch\nORGANIZER:mailto:jdoe@example.com\nATTENDEE;CUTYPE=INDIVIDUAL:mailto:jane.smith@example.com\nATTENDEE;CUTYPE=INDIVIDUAL:mailto:bill@example.com\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\nconst TEST_FREEBUSY: &str = r#\"BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nMETHOD:REQUEST\nBEGIN:VFREEBUSY\nUID:4FD3AD926350\nDTSTAMP:20090602T190420Z\nDTSTART:$START\nDTEND:$END\nORGANIZER:mailto:jdoe@example.com\nATTENDEE:mailto:jdoe@example.com\nATTENDEE:mailto:jane.smith@example.com\nATTENDEE:mailto:bill@example.com\nATTENDEE:mailto:unknown@example.com\nEND:VFREEBUSY\nEND:VCALENDAR\n\"#;\n"
  },
  {
    "path": "tests/src/webdav/card_query.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::WebDavTest;\nuse dav_proto::schema::property::{CardDavProperty, DavProperty, WebDavProperty};\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\n\npub async fn test(test: &WebDavTest) {\n    println!(\"Running REPORT addressbook-query tests...\");\n    let client = test.client(\"john\");\n\n    // Create test data\n    let default_path = format!(\"{}/john/default/\", DavResourceName::Card.base_path());\n    let mut hrefs = Vec::with_capacity(3);\n    for (i, vcard) in [VCARD1, VCARD2, VCARD3].iter().enumerate() {\n        let href = format!(\"{default_path}contact-{i}.vcf\",);\n        client\n            .request(\"PUT\", &href, *vcard)\n            .await\n            .with_status(hyper::StatusCode::CREATED);\n        hrefs.push(href);\n    }\n    let uri_sarah = hrefs[0].as_str();\n    let uri_carlos = hrefs[1].as_str();\n    let uri_acme = hrefs[2].as_str();\n\n    // Test 1: RFC6352 8.6.3 example 1\n    let response = client\n        .request(\"REPORT\", &default_path, QUERY1)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([uri_carlos])\n        .into_propfind_response(None);\n    let props = response.properties(uri_carlos);\n    props\n        .get(DavProperty::WebDav(WebDavProperty::GetETag))\n        .is_not_empty();\n    props\n        .get(DavProperty::CardDav(CardDavProperty::AddressData(\n            Default::default(),\n        )))\n        .with_values([r#\"BEGIN:VCARD\nVERSION:4.0\nFN:Carlos Rodriguez-Martinez\nNICKNAME:Charlie\nEMAIL;TYPE=WORK,pref:carlos.rodriguez@example-corp.com\nEMAIL;TYPE=HOME:carlosrm@personalmail.example\nUID:urn:uuid:e1ee798b-3d4c-41b0-b217-b9c918e4686a\nEND:VCARD\n\"#\n        .replace('\\n', \"\\r\\n\")\n        .as_str()]);\n\n    // Test 2: RFC6352 8.6.3 example 2\n    let response = client\n        .request(\"REPORT\", &default_path, QUERY2)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([uri_carlos, uri_sarah])\n        .into_propfind_response(None);\n    let props = response.properties(uri_carlos);\n    props\n        .get(DavProperty::WebDav(WebDavProperty::GetETag))\n        .is_not_empty();\n    props\n        .get(DavProperty::CardDav(CardDavProperty::AddressData(\n            Default::default(),\n        )))\n        .with_values([r#\"BEGIN:VCARD\nFN:Carlos Rodriguez-Martinez\nBDAY:--0623\nCATEGORIES:Marketing,Management,International\nLANG;TYPE=WORK;PREF=1:es\nLANG;TYPE=WORK;PREF=2:en\nLANG;TYPE=WORK;PREF=3:pt\nEND:VCARD\n\"#\n        .replace('\\n', \"\\r\\n\")\n        .as_str()]);\n    let props = response.properties(uri_sarah);\n    props\n        .get(DavProperty::WebDav(WebDavProperty::GetETag))\n        .is_not_empty();\n    props\n        .get(DavProperty::CardDav(CardDavProperty::AddressData(\n            Default::default(),\n        )))\n        .with_values([r#\"BEGIN:VCARD\nFN:Sarah Johnson\nBDAY:19850415\nCATEGORIES:Work,Research,VIP\nLANG;TYPE=WORK;PREF=1:en\nLANG;TYPE=WORK;PREF=2:fr\nEND:VCARD\n\"#\n        .replace('\\n', \"\\r\\n\")\n        .as_str()]);\n\n    // Test 3: Search within parameters\n    let response = client\n        .request(\"REPORT\", &default_path, QUERY3)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_hrefs([uri_acme])\n        .into_propfind_response(None);\n    let props = response.properties(uri_acme);\n    props\n        .get(DavProperty::CardDav(CardDavProperty::AddressData(\n            Default::default(),\n        )))\n        .with_values([VCARD3.replace('\\n', \"\\r\\n\").as_str()]);\n\n    // Test 4: Search using limit\n    client\n        .request(\"REPORT\", &default_path, QUERY4)\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .with_value(\n            \"D:multistatus.D:response.D:status\",\n            \"HTTP/1.1 507 Insufficient Storage\",\n        )\n        .with_value(\n            \"D:multistatus.D:response.D:error.D:number-of-matches-within-limits\",\n            \"\",\n        )\n        .with_value(\n            \"D:multistatus.D:response.D:responsedescription\",\n            \"The number of matches exceeds the limit of 2\",\n        )\n        .with_href_count(3);\n\n    client.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\nconst QUERY1: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data>\n         <C:prop name=\"VERSION\"/>\n         <C:prop name=\"UID\"/>\n         <C:prop name=\"NICKNAME\"/>\n         <C:prop name=\"EMAIL\"/>\n         <C:prop name=\"FN\"/>\n       </C:address-data>\n     </D:prop>\n     <C:filter>\n       <C:prop-filter name=\"NICKNAME\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"equals\"\n         >charlie</C:text-match>\n       </C:prop-filter>\n     </C:filter>\n   </C:addressbook-query>\"#;\n\nconst QUERY2: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data>\n         <C:prop name=\"FN\"/>\n         <C:prop name=\"BDAY\"/>\n         <C:prop name=\"CATEGORIES\"/>\n         <C:prop name=\"LANG\"/>\n       </C:address-data>\n     </D:prop>\n     <C:filter test=\"anyof\">\n       <C:prop-filter name=\"FN\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"contains\"\n         >john</C:text-match>\n       </C:prop-filter>\n       <C:prop-filter name=\"EMAIL\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"contains\"\n         >rodriguez</C:text-match>\n       </C:prop-filter>\n     </C:filter>\n   </C:addressbook-query>\"#;\n\nconst QUERY3: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<C:addressbook-query xmlns:D=\"DAV:\" xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n  <D:prop>\n    <D:getetag/>\n    <C:address-data/>\n  </D:prop>\n  <C:filter test=\"anyof\">\n    <C:prop-filter name=\"ADR\">\n      <C:param-filter name=\"LABEL\">\n        <C:text-match collation=\"i;unicode-casemap\" match-type=\"contains\">enterprise</C:text-match>\n      </C:param-filter>\n    </C:prop-filter>\n  </C:filter>\n</C:addressbook-query>\"#;\n\nconst QUERY4: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-query xmlns:D=\"DAV:\"\n                     xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n     </D:prop>\n     <C:filter test=\"anyof\">\n       <C:prop-filter name=\"ORG\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"contains\"\n         >acme</C:text-match>\n       </C:prop-filter>\n       <C:prop-filter name=\"ORG\">\n         <C:text-match collation=\"i;unicode-casemap\"\n                       match-type=\"contains\"\n         >global</C:text-match>\n       </C:prop-filter>\n     </C:filter>\n     <C:limit>\n       <C:nresults>2</C:nresults>\n     </C:limit>\n   </C:addressbook-query>\"#;\n\nconst VCARD1: &str = r#\"BEGIN:VCARD\nVERSION:4.0\nFN:Sarah Johnson\nN:Johnson;Sarah;Marie;Dr.;Ph.D.\nNICKNAME:Sadie\nGENDER:F\nBDAY:19850415\nANNIVERSARY:20100610\nEMAIL;TYPE=work:sarah.johnson@example.com\nEMAIL;TYPE=home,pref:sarahjpersonal@example.com\nTEL;TYPE=cell,voice,pref:+1-555-123-4567\nTEL;TYPE=work,voice:+1-555-987-6543\nTEL;TYPE=home,voice:+1-555-456-7890\nADR;TYPE=work;LABEL=\"123 Business Ave\\nSuite 400\\nNew York, NY 10001\\nUSA\":;;123 Business Ave;New York;NY;10001;USA\nADR;TYPE=home,pref;LABEL=\"456 Residential St\\nApt 7B\\nBrooklyn, NY 11201\\nUSA\":;;456 Residential St;Brooklyn;NY;11201;USA\nORG:Acme Technologies Inc.;Research Department\nTITLE:Senior Research Scientist\nROLE:Team Lead\nCATEGORIES:Work,Research,VIP\nURL;TYPE=work:https://www.example.com/staff/sjohnson\nURL;TYPE=home:https://www.sarahjohnson.example.com\nKEY;TYPE=PGP:https://pgp.example.com/pks/lookup?op=get&search=sarah.johnson@example.com\nNOTE:Sarah prefers video calls over phone calls. Available Mon-Thu 9-5 EST.\nLANG;TYPE=work;PREF=1:en\nLANG;TYPE=work;PREF=2:fr\nTZ:-0500\nGEO:40.7128;-74.0060\nUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6\nREV:20220315T133000Z\nEND:VCARD\n\"#;\n\nconst VCARD2: &str = r#\"BEGIN:VCARD\nVERSION:4.0\nFN:Carlos Rodriguez-Martinez\nN:Rodriguez-Martinez;Carlos;Alberto;Mr.;Jr.\nNICKNAME:Charlie\nGENDER:M\nBDAY:--0623\nANNIVERSARY:20150809\nEMAIL;TYPE=work,pref:carlos.rodriguez@example-corp.com\nEMAIL;TYPE=home:carlosrm@personalmail.example\nTEL;TYPE=cell,voice,pref:+34-611-234-567\nTEL;TYPE=work,voice:+34-911-876-543\nTEL;TYPE=home,voice:+34-644-321-987\nTEL;TYPE=fax:+34-911-876-544\nADR;TYPE=work;LABEL=\"Calle Empresarial 42\\nPlanta 3\\nMadrid, 28001\\nSpain\":;;Calle Empresarial 42;Madrid;;28001;Spain\nADR;TYPE=home,pref;LABEL=\"Avenida Residencial 15\\nPiso 7, Puerta C\\nMadrid, 28045\\nSpain\":;;Avenida Residencial 15;Madrid;;28045;Spain\nORG:Global Solutions S.L.;Marketing Division\nTITLE:Digital Marketing Director\nROLE:Department Head\nCATEGORIES:Marketing,Management,International\nURL;TYPE=work:https://www.example-corp.com/team/carlos\nURL;TYPE=home:https://www.carlosrodriguez.example\nURL;TYPE=social:https://linkedin.com/in/carlosrodriguezm\nKEY;TYPE=PGP:https://pgp.example.com/pks/lookup?op=get&search=carlos.rodriguez@example-corp.com\nNOTE:Carlos speaks English, Spanish, and Portuguese fluently. Prefers communication via email. Do not contact after 7PM CET.\nLANG;TYPE=work;PREF=1:es\nLANG;TYPE=work;PREF=2:en\nLANG;TYPE=work;PREF=3:pt\nTZ:+0100\nGEO:40.4168;-3.7038\nUID:urn:uuid:e1ee798b-3d4c-41b0-b217-b9c918e4686a\nREV:20230712T092135Z\nSOURCE:https://contacts.example.com/carlosrodriguez.vcf\nKIND:individual\nMEMBER:urn:uuid:03a0e51f-d1aa-4385-8a53-e29025acd8af\nRELATED;TYPE=friend:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6\nEND:VCARD\n\"#;\n\nconst VCARD3: &str = r#\"BEGIN:VCARD\nVERSION:4.0\nFN:Acme Business Solutions Ltd.\nN:;;;;\nKIND:ORG\nORG:Acme Business Solutions Ltd.;Technology Division\nEMAIL;TYPE=WORK,pref:info@acme-solutions.example\nEMAIL;TYPE=support:support@acme-solutions.example\nEMAIL;TYPE=sales:sales@acme-solutions.example\nTEL;TYPE=WORK,VOICE,pref:+44-20-1234-5678\nTEL;TYPE=FAX:+44-20-1234-5679\nTEL;TYPE=support:+44-800-987-6543\nADR;TYPE=WORK;LABEL=\"10 Enterprise Way\\nTech Park\\nLondon, EC1A 1BB\\nUnited\n  Kingdom\":;;10 Enterprise Way\\, Tech Park;London;;EC1A 1BB;United Kingdom\nADR;TYPE=branch;LABEL=\"25 Innovation Street\\nManchester, M1 5QF\\nUnited Kin\n gdom\":;;25 Innovation Street;Manchester;;M1 5QF;United Kingdom\nURL;TYPE=WORK:https://www.acme-solutions.example\nURL;TYPE=support:https://support.acme-solutions.example\nCATEGORIES:Technology,B2B,Solutions,Services\nNOTE:Business hours: Mon-Fri 9:00-17:30 GMT. Closed on UK bank holidays. VAT\n  Reg: GB123456789\nTZ:Z\nGEO:51.5074;-0.1278\nKEY;TYPE=PGP:https://pgp.example.com/pks/lookup?op=get&search=info@acme-solu\n tions.example\nUID:urn:uuid:a9e95948-7b1c-46e8-bd85-c729a9e910f2\nREV:20230415T153000Z\nLANG;TYPE=WORK;PREF=1:en\nLANG;TYPE=WORK;PREF=2:de\nLANG;TYPE=WORK;PREF=3:fr\nSOURCE:https://directory.example.com/acme.vcf\nRELATED;TYPE=CONTACT:urn:uuid:b9e93fdb-4d34-45fa-a1e2-47da0428c4a1\nRELATED;TYPE=CONTACT:urn:uuid:c8e74dfe-6b34-45fa-b1e2-47ea0428c4b2\nX-ABLabel:Company\nPRODID:-//Example Corp.//Contact Manager 3.0//EN\nEND:VCARD\n\"#;\n"
  },
  {
    "path": "tests/src/webdav/copy_move.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{DavResponse, WebDavTest};\nuse crate::webdav::GenerateTestDavResource;\nuse ahash::AHashSet;\nuse dav_proto::Depth;\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\n\npub async fn test(test: &WebDavTest, assisted_discovery: bool) {\n    let client = test.client(\"jane\");\n    let mike_noquota = test.client(\"mike\");\n\n    for resource_type in [\n        DavResourceName::File,\n        DavResourceName::Cal,\n        DavResourceName::Card,\n    ] {\n        println!(\"Running COPY/MOVE tests ({})...\", resource_type.base_path());\n        let user_base_path = format!(\"{}/jane\", resource_type.base_path());\n        let group_base_path = format!(\"{}/support\", resource_type.base_path());\n        let default_test_depth = if resource_type == DavResourceName::File {\n            2\n        } else {\n            0\n        };\n\n        // Obtain sync token\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n\n        // TODO: Fix tests for assisted discovery\n        assert_eq!(\n            response.hrefs().len(),\n            if resource_type == DavResourceName::File {\n                1\n            } else {\n                2 + usize::from(assisted_discovery)\n            },\n            \"{:?}\",\n            response.hrefs()\n        );\n\n        // Create nested files and folders\n        let (hierarchy_root, mut hierarchy) = client\n            .create_hierarchy(&user_base_path, default_test_depth, 2, 3)\n            .await;\n        let prev_sync_token = response.sync_token();\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                prev_sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        let sync_token = response.sync_token();\n        let changed_hrefs = response.hrefs();\n        assert_ne!(sync_token, prev_sync_token);\n        assert_eq!(\n            changed_hrefs,\n            hierarchy.iter().map(|x| x.0.as_str()).collect::<Vec<_>>(),\n            \"lengths {} & {}\",\n            changed_hrefs.len(),\n            hierarchy.len()\n        );\n        client.validate_values(&hierarchy).await;\n\n        // Delete cache an resync\n        test.clear_cache();\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                prev_sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        let sync_token = response.sync_token();\n        let changed_hrefs = response.hrefs();\n        assert_ne!(sync_token, prev_sync_token);\n        assert_eq!(\n            changed_hrefs,\n            hierarchy.iter().map(|x| x.0.as_str()).collect::<Vec<_>>(),\n            \"lengths {} & {}\",\n            changed_hrefs.len(),\n            hierarchy.len()\n        );\n\n        // Copying and moving to the same or root containers is invalid\n        for method in [\"COPY\", \"MOVE\"] {\n            for destination in [\n                \"/dav\",\n                \"/dav/cal\",\n                \"/dav/card\",\n                \"/dav/file\",\n                \"/dav/pal\",\n                hierarchy_root.as_str(),\n            ] {\n                client\n                    .request_with_headers(\n                        method,\n                        &hierarchy_root,\n                        [(\"destination\", destination)],\n                        \"\",\n                    )\n                    .await\n                    .with_status(StatusCode::BAD_GATEWAY);\n            }\n        }\n\n        // Test 1: Rename container\n        let new_hierarchy_root = format!(\"{user_base_path}/Test_Folder/\");\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &hierarchy_root,\n                [(\"destination\", new_hierarchy_root.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        replace_prefix(&mut hierarchy, &hierarchy_root, &new_hierarchy_root);\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n        // Validate changes\n        let changes = client\n            .sync_collection(\n                &user_base_path,\n                sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await\n            .with_href_count(2)\n            .into_propfind_response(None);\n        changes\n            .properties(&hierarchy_root)\n            .with_status(StatusCode::NOT_FOUND);\n        changes\n            .properties(&new_hierarchy_root)\n            .with_status(StatusCode::OK);\n        let hierarchy_root = new_hierarchy_root;\n\n        // Test 2: Copy container\n        let new_hierarchy_root = format!(\"{user_base_path}/Test_Folder_Copy/\");\n        client\n            .request_with_headers(\n                \"COPY\",\n                &hierarchy_root,\n                [(\"destination\", new_hierarchy_root.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        let mut copied_hierarchy = hierarchy.clone();\n        replace_prefix(&mut copied_hierarchy, &hierarchy_root, &new_hierarchy_root);\n        copied_hierarchy.extend_from_slice(&hierarchy);\n        assert_result(&response, &copied_hierarchy);\n        client.validate_values(&copied_hierarchy).await;\n\n        // Test 3: Delete original container\n        client\n            .request(\"DELETE\", &new_hierarchy_root, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n\n        // Test 4: Create a shallow container and overwrite the previous one using MOVE\n        let (new_hierarchy_root, mut hierarchy) =\n            client.create_hierarchy(&user_base_path, 0, 0, 3).await;\n        let sync_token = client\n            .sync_collection(\n                &user_base_path,\n                sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await\n            .sync_token()\n            .to_string();\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &new_hierarchy_root,\n                [(\"destination\", hierarchy_root.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        replace_prefix(&mut hierarchy, &new_hierarchy_root, &hierarchy_root);\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n        // Validate changes\n        let changes = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await\n            .into_propfind_response(None);\n        changes\n            .properties(&new_hierarchy_root)\n            .with_status(StatusCode::NOT_FOUND);\n        changes\n            .properties(&hierarchy_root)\n            .with_status(StatusCode::OK);\n\n        // Test 5: Create a deep container and overwrite the previous one using COPY\n        let (new_hierarchy_root, new_hierarchy) = client\n            .create_hierarchy(&user_base_path, default_test_depth, 1, 2)\n            .await;\n        client\n            .request_with_headers(\n                \"COPY\",\n                &new_hierarchy_root,\n                [(\"destination\", hierarchy_root.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        let mut orig_hierarchy = new_hierarchy.clone();\n        replace_prefix(&mut orig_hierarchy, &new_hierarchy_root, &hierarchy_root);\n        let mut full_hierarchy = new_hierarchy.clone();\n        full_hierarchy.extend_from_slice(&orig_hierarchy);\n        assert_result(&response, &full_hierarchy);\n        client.validate_values(&full_hierarchy).await;\n\n        // Test 6: Copy and move containers to a shared account\n        let shared_hierarchy_root_1 = format!(\"{group_base_path}/Test_Shared_Folder_1/\");\n        let shared_hierarchy_root_2 = format!(\"{group_base_path}/Test_Shared_Folder_2/\");\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &new_hierarchy_root,\n                [(\"destination\", shared_hierarchy_root_1.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        client\n            .request_with_headers(\n                \"COPY\",\n                &hierarchy_root,\n                [(\"destination\", shared_hierarchy_root_2.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &orig_hierarchy);\n        client.validate_values(&orig_hierarchy).await;\n        let response = client\n            .sync_collection(&group_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        replace_prefix(\n            &mut full_hierarchy,\n            &new_hierarchy_root,\n            &shared_hierarchy_root_1,\n        );\n        replace_prefix(\n            &mut full_hierarchy,\n            &hierarchy_root,\n            &shared_hierarchy_root_2,\n        );\n        assert_result(&response, &full_hierarchy);\n        client.validate_values(&full_hierarchy).await;\n\n        // Delete all containers\n        for shared_container in [\n            shared_hierarchy_root_1,\n            shared_hierarchy_root_2,\n            hierarchy_root,\n        ] {\n            client\n                .request(\"DELETE\", &shared_container, \"\")\n                .await\n                .with_status(StatusCode::NO_CONTENT);\n        }\n\n        // Create test containers\n        let mut hierarchy = vec![];\n        for folder_name in [\"folder1\", \"folder2\", \"folder3\"] {\n            let folder_path = format!(\"{user_base_path}/{folder_name}/\");\n\n            client\n                .mkcol(\"MKCOL\", &folder_path, [], [])\n                .await\n                .with_status(StatusCode::CREATED);\n\n            for file_name in [\"file1\", \"file2\", \"file3\"] {\n                let file_path = format!(\"{folder_path}{file_name}\");\n                let file_contents = resource_type.generate();\n                client\n                    .request(\"PUT\", &file_path, &file_contents)\n                    .await\n                    .with_status(StatusCode::CREATED);\n                hierarchy.push((file_path, file_contents));\n            }\n\n            hierarchy.push((folder_path, \"\".to_string()));\n        }\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n\n        // Test 7: Copying or moving files to the root container is not allowed\n        let folder1_file1 = format!(\"{user_base_path}/folder1/file1\");\n        if resource_type != DavResourceName::File {\n            for method in [\"COPY\", \"MOVE\"] {\n                client\n                    .request_with_headers(\n                        method,\n                        &folder1_file1,\n                        [(\"destination\", user_base_path.as_str())],\n                        \"\",\n                    )\n                    .await\n                    .with_status(StatusCode::BAD_GATEWAY);\n                client\n                    .request_with_headers(\n                        method,\n                        &folder1_file1,\n                        [(\"destination\", format!(\"{user_base_path}/folder2\").as_str())],\n                        \"\",\n                    )\n                    .await\n                    .with_status(StatusCode::BAD_GATEWAY);\n            }\n        }\n\n        // Test 8: Copying or moving to the same location is not allowed\n        for method in [\"COPY\", \"MOVE\"] {\n            client\n                .request_with_headers(\n                    method,\n                    &folder1_file1,\n                    [(\"destination\", folder1_file1.as_str())],\n                    \"\",\n                )\n                .await\n                .with_status(StatusCode::BAD_GATEWAY);\n        }\n\n        // Test 9: Rename file\n        let folder1_file1_new = format!(\"{user_base_path}/folder1/file1_new\");\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &folder1_file1,\n                [(\"destination\", folder1_file1_new.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        rename(&mut hierarchy, &folder1_file1, &folder1_file1_new);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n\n        // Test 10: Move a file under a different container\n        let folder2_file1_from_folder1 = format!(\"{user_base_path}/folder2/file1_from_folder1\");\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &folder1_file1_new,\n                [(\"destination\", folder2_file1_from_folder1.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        rename(\n            &mut hierarchy,\n            &folder1_file1_new,\n            &folder2_file1_from_folder1,\n        );\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n\n        // Test 11: Move and overwrite a file under a different container\n        let folder1_file2 = format!(\"{user_base_path}/folder1/file2\");\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &folder2_file1_from_folder1,\n                [(\"destination\", folder1_file2.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        delete(&mut hierarchy, &folder1_file2);\n        rename(&mut hierarchy, &folder2_file1_from_folder1, &folder1_file2);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n\n        // Test 12: Copy a file under a different container\n        let file3_path = format!(\"{user_base_path}/folder1/file3\");\n        let folder3_file3_from_folder1 = format!(\"{user_base_path}/folder3/file3_from_folder1\");\n        client\n            .request_with_headers(\n                \"COPY\",\n                &file3_path,\n                [(\"destination\", folder3_file3_from_folder1.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        copy(&mut hierarchy, &file3_path, &folder3_file3_from_folder1);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n\n        // Test 12: Copy and overwrite a file under a different container\n        let folder2_file2 = format!(\"{user_base_path}/folder2/file2\");\n        client\n            .request_with_headers(\n                \"COPY\",\n                &folder3_file3_from_folder1,\n                [(\"destination\", folder2_file2.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        delete(&mut hierarchy, &folder2_file2);\n        copy(&mut hierarchy, &folder3_file3_from_folder1, &folder2_file2);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n\n        // Test 13: Copy and move files to a shared container\n        let shared_hierarchy_root = format!(\"{group_base_path}/Test_Child_Folder/\");\n        let folder3_file1 = format!(\"{user_base_path}/folder3/file1\");\n        let shared_file_1 = format!(\"{shared_hierarchy_root}shared_file_1\");\n        let shared_file_2 = format!(\"{shared_hierarchy_root}shared_file_2\");\n        client\n            .mkcol(\"MKCOL\", &shared_hierarchy_root, [], [])\n            .await\n            .with_status(StatusCode::CREATED);\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &folder3_file1,\n                [(\"destination\", shared_file_1.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        client\n            .request_with_headers(\n                \"COPY\",\n                &folder1_file2,\n                [(\"destination\", shared_file_2.as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n        let shared_hierarchy = vec![\n            (shared_hierarchy_root.clone(), \"\".to_string()),\n            (\n                shared_file_1,\n                get_contents(&hierarchy, &folder3_file1).unwrap(),\n            ),\n            (\n                shared_file_2,\n                get_contents(&hierarchy, &folder1_file2).unwrap(),\n            ),\n        ];\n        delete(&mut hierarchy, &folder3_file1);\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &hierarchy);\n        client.validate_values(&hierarchy).await;\n        let response = client\n            .sync_collection(&group_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_result(&response, &shared_hierarchy);\n        client.validate_values(&shared_hierarchy).await;\n        client\n            .request(\"DELETE\", &shared_hierarchy_root, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n\n        if resource_type == DavResourceName::File {\n            // Test 14: Move a container under a different container\n            let folder2 = format!(\"{user_base_path}/folder2/\");\n            let folder3 = format!(\"{user_base_path}/folder3/\");\n            let folder2_folder3 = format!(\"{user_base_path}/folder2/folder3/\");\n            client\n                .request_with_headers(\n                    \"MOVE\",\n                    &folder3,\n                    [(\"destination\", folder2_folder3.as_str())],\n                    \"\",\n                )\n                .await\n                .with_status(StatusCode::CREATED);\n            replace_prefix(&mut hierarchy, &folder3, &folder2_folder3);\n            let response = client\n                .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n                .await;\n            assert_result(&response, &hierarchy);\n            client.validate_values(&hierarchy).await;\n\n            // Test 15: Moving or copying a parent under a child is not allowed\n            for method in [\"MOVE\", \"COPY\"] {\n                client\n                    .request_with_headers(\n                        method,\n                        &folder2_folder3,\n                        [(\"destination\", folder2.as_str())],\n                        \"\",\n                    )\n                    .await\n                    .with_status(StatusCode::BAD_GATEWAY);\n            }\n\n            // Test 16: Copy a container under a different container\n            let folder1 = format!(\"{user_base_path}/folder1/\");\n            let folder2_folder1 = format!(\"{user_base_path}/folder2/folder1/\");\n            client\n                .request_with_headers(\n                    \"COPY\",\n                    &folder1,\n                    [(\"destination\", folder2_folder1.as_str())],\n                    \"\",\n                )\n                .await\n                .with_status(StatusCode::CREATED);\n            let response = client\n                .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n                .await;\n            copy_prefix(&mut hierarchy, &folder1, &folder2_folder1);\n            assert_result(&response, &hierarchy);\n            client.validate_values(&hierarchy).await;\n        } else {\n            // Test 17: UID collision\n            let folder1 = format!(\"{user_base_path}/folder1/\");\n            let folder2 = format!(\"{user_base_path}/folder2/\");\n            let file_contents = resource_type.generate();\n            for folder_path in [&folder1, &folder2] {\n                let file_path = format!(\"{folder_path}uid_test\");\n                client\n                    .request(\"PUT\", &file_path, file_contents.as_str())\n                    .await\n                    .with_status(StatusCode::CREATED);\n            }\n            let uid_file_src = format!(\"{folder1}uid_test\");\n            let uid_file_dest = format!(\"{folder2}uid_test_dup\");\n            for method in [\"COPY\", \"MOVE\"] {\n                client\n                    .request_with_headers(\n                        method,\n                        &uid_file_src,\n                        [(\"destination\", uid_file_dest.as_str())],\n                        \"\",\n                    )\n                    .await\n                    .with_status(StatusCode::PRECONDITION_FAILED)\n                    .with_failed_precondition(\n                        if resource_type == DavResourceName::Cal {\n                            \"A:no-uid-conflict.D:href\"\n                        } else {\n                            \"B:no-uid-conflict.D:href\"\n                        },\n                        &format!(\"{folder2}uid_test\"),\n                    );\n            }\n        }\n\n        // Delete all containers and create a new one\n        client\n            .request(\"DELETE\", &format!(\"{user_base_path}/folder3/\"), \"\")\n            .await\n            .with_status(if resource_type == DavResourceName::File {\n                StatusCode::NOT_FOUND\n            } else {\n                StatusCode::NO_CONTENT\n            });\n        for folder in [\"folder1\", \"folder2\"] {\n            let folder_path = format!(\"{user_base_path}/{folder}/\");\n            client\n                .request(\"DELETE\", &folder_path, \"\")\n                .await\n                .with_status(StatusCode::NO_CONTENT);\n        }\n\n        // Create a new test container and file\n        let test_base_path = format!(\"{user_base_path}/My_Test_Folder/\");\n        client\n            .mkcol(\"MKCOL\", &test_base_path, [], [])\n            .await\n            .with_status(StatusCode::CREATED);\n        let test_contents_1 = resource_type.generate();\n        let test_contents_2 = resource_type.generate();\n        let test_file1_path = format!(\"{test_base_path}test_file_1\");\n        let test_file2_path = format!(\"{test_base_path}test_file_2\");\n        let test_etag_1 = client\n            .request(\"PUT\", &test_file1_path, test_contents_1.as_str())\n            .await\n            .with_status(StatusCode::CREATED)\n            .etag()\n            .to_string();\n        let test_etag_2 = client\n            .request(\"PUT\", &test_file2_path, test_contents_2.as_str())\n            .await\n            .with_status(StatusCode::CREATED)\n            .etag()\n            .to_string();\n\n        // Test 18: Failed DAV preconditions\n        for method in [\"COPY\", \"MOVE\"] {\n            client\n                .request_with_headers(\n                    method,\n                    &test_file1_path,\n                    [\n                        (\"destination\", test_file2_path.as_str()),\n                        (\"overwrite\", \"F\"),\n                    ],\n                    \"\",\n                )\n                .await\n                .with_status(StatusCode::PRECONDITION_FAILED)\n                .with_empty_body();\n\n            client\n                .request_with_headers(\n                    method,\n                    &test_file1_path,\n                    [\n                        (\"destination\", test_file2_path.as_str()),\n                        (\"if-none-match\", \"*\"),\n                    ],\n                    \"\",\n                )\n                .await\n                .with_status(StatusCode::PRECONDITION_FAILED)\n                .with_empty_body();\n\n            let iff = format!(\n                \"<{test_file1_path}> (Not [{test_etag_1}]) <{test_file2_path}> (Not [{test_etag_2}])\",\n            );\n            client\n                .request_with_headers(\n                    method,\n                    &test_file1_path,\n                    [\n                        (\"destination\", test_file2_path.as_str()),\n                        (\"if\", iff.as_str()),\n                    ],\n                    \"\",\n                )\n                .await\n                .with_status(StatusCode::PRECONDITION_FAILED)\n                .with_empty_body();\n        }\n\n        // Test 18: Successful DAV preconditions\n        let iff =\n            format!(\"<{test_file1_path}> ([{test_etag_1}]) <{test_file2_path}> ([{test_etag_2}])\",);\n        client\n            .request_with_headers(\n                \"MOVE\",\n                &test_file1_path,\n                [\n                    (\"destination\", test_file2_path.as_str()),\n                    (\"if\", iff.as_str()),\n                ],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n\n        // Delete the test container\n        client\n            .request(\"DELETE\", &test_base_path, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n\n        // Test 19: Quota enforcement (on CalDAV/CardDAV items are linked, not copied therefore there is no quota increase)\n        if resource_type == DavResourceName::File {\n            let path = format!(\"{}/mike/quota-test/\", resource_type.base_path());\n            let content = resource_type.generate();\n            mike_noquota\n                .mkcol(\"MKCOL\", &path, [], [])\n                .await\n                .with_status(StatusCode::CREATED);\n            mike_noquota\n                .request_with_headers(\"PUT\", &format!(\"{path}file\"), [], &content)\n                .await\n                .with_status(StatusCode::CREATED);\n            let mut num_success = 0;\n            let mut did_fail = false;\n\n            for i in 0..100 {\n                let response = mike_noquota\n                    .request_with_headers(\n                        \"COPY\",\n                        &path,\n                        [(\n                            \"destination\",\n                            format!(\"{}/mike/quota-test{i}\", resource_type.base_path()).as_str(),\n                        )],\n                        &content,\n                    )\n                    .await;\n                match response.status {\n                    StatusCode::CREATED => {\n                        num_success += 1;\n                    }\n                    StatusCode::PRECONDITION_FAILED => {\n                        did_fail = true;\n                        break;\n                    }\n                    _ => panic!(\"Unexpected status code: {:?}\", response.status),\n                }\n            }\n            if !did_fail {\n                panic!(\"Quota test failed: {} files created\", num_success);\n            }\n            if num_success == 0 {\n                panic!(\"Quota test failed: no files created\");\n            }\n\n            mike_noquota\n                .request(\"DELETE\", &path, \"\")\n                .await\n                .with_status(StatusCode::NO_CONTENT);\n            for i in 0..num_success {\n                mike_noquota\n                    .request(\n                        \"DELETE\",\n                        &format!(\"{}/mike/quota-test{i}\", resource_type.base_path()),\n                        \"\",\n                    )\n                    .await\n                    .with_status(StatusCode::NO_CONTENT);\n            }\n        }\n    }\n\n    client.delete_default_containers().await;\n    client.delete_default_containers_by_account(\"support\").await;\n    mike_noquota.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\nfn assert_result(response: &DavResponse, hierarchy: &[(String, String)]) {\n    assert!(!hierarchy.is_empty());\n    let response = response\n        .hrefs()\n        .into_iter()\n        .filter(|h| {\n            !h.ends_with(\"/jane/\") && !h.ends_with(\"/support/\") && !h.ends_with(\"/default/\")\n        })\n        .collect::<AHashSet<_>>();\n    let hierarchy = hierarchy\n        .iter()\n        .map(|x| x.0.as_str())\n        .collect::<AHashSet<_>>();\n\n    if hierarchy != response {\n        println!(\"\\nMissing: {:?}\", hierarchy.difference(&response));\n        println!(\"\\nExtra: {:?}\", response.difference(&hierarchy));\n\n        panic!(\n            \"Hierarchy mismatch: expected {} items, received {} items\",\n            hierarchy.len(),\n            response.len()\n        );\n    }\n}\n\nfn replace_prefix(items: &mut [(String, String)], old_prefix: &str, new_prefix: &str) {\n    let mut did_replace = false;\n    for (href, _) in items.iter_mut() {\n        if let Some(value) = href.strip_prefix(old_prefix) {\n            *href = format!(\"{new_prefix}{value}\");\n            did_replace = true;\n        }\n    }\n    if !did_replace {\n        panic!(\"Prefix not found: {}\", old_prefix);\n    }\n}\n\nfn rename(items: &mut [(String, String)], old_name: &str, new_name: &str) {\n    for (href, _) in items.iter_mut() {\n        if href == old_name {\n            *href = new_name.to_string();\n            return;\n        }\n    }\n    panic!(\"Item not found: {}\", old_name);\n}\n\nfn delete(items: &mut Vec<(String, String)>, name: &str) {\n    let mut did_delete = false;\n    items.retain(|(href, _)| {\n        did_delete = did_delete || href == name;\n        href != name\n    });\n\n    if !did_delete {\n        panic!(\"Item not found: {}\", name);\n    }\n}\n\nfn copy(items: &mut Vec<(String, String)>, old_name: &str, new_name: &str) {\n    for (href, contents) in items.iter_mut() {\n        if href == old_name {\n            let value = (new_name.to_string(), contents.to_string());\n            items.push(value);\n            return;\n        }\n    }\n    panic!(\"Item not found: {}\", old_name);\n}\n\nfn copy_prefix(items: &mut Vec<(String, String)>, old_prefix: &str, new_prefix: &str) {\n    let mut new_items = vec![];\n    for (href, contents) in items.iter() {\n        if let Some(value) = href.strip_prefix(old_prefix) {\n            new_items.push((format!(\"{new_prefix}{value}\"), contents.to_string()));\n        }\n    }\n    if !new_items.is_empty() {\n        items.extend(new_items);\n    } else {\n        panic!(\"Prefix not found: {}\", old_prefix);\n    }\n}\n\nfn get_contents(items: &[(String, String)], name: &str) -> Option<String> {\n    for (href, contents) in items.iter() {\n        if href == name {\n            return Some(contents.to_string());\n        }\n    }\n    None\n}\n"
  },
  {
    "path": "tests/src/webdav/lock.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{DavResponse, DummyWebDavClient, WebDavTest};\nuse crate::webdav::GenerateTestDavResource;\nuse dav_proto::schema::property::{DavProperty, WebDavProperty};\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\n\npub async fn test(test: &WebDavTest) {\n    let client = test.client(\"john\");\n\n    for resource_type in [\n        DavResourceName::File,\n        DavResourceName::Cal,\n        DavResourceName::Card,\n    ] {\n        println!(\n            \"Running LOCK/UNLOCK tests ({})...\",\n            resource_type.base_path()\n        );\n        let base_path = format!(\"{}/john\", resource_type.base_path());\n\n        // Test 1: Creating a collection under an unmapped resource without providing a lock token should fail\n        let path = format!(\"{base_path}/do-not-write\");\n        let response = client\n            .lock_create(&path, \"super-owner\", true, \"infinity\", \"Second-123\")\n            .await\n            .with_status(StatusCode::CREATED);\n        let lock_token = response\n            .with_value(\n                \"D:prop.D:lockdiscovery.D:activelock.D:owner.href\",\n                \"super-owner\",\n            )\n            .with_value(\"D:prop.D:lockdiscovery.D:activelock.D:depth\", \"infinity\")\n            .with_value(\n                \"D:prop.D:lockdiscovery.D:activelock.D:timeout\",\n                \"Second-123\",\n            )\n            .lock_token()\n            .to_string();\n\n        // Test 2: Refreshing a lock token with an invalid a lock token should fail\n        client\n            .lock_refresh(&path, \"urn:stalwart:davlock:1234\", \"infinity\", \"Second-456\")\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED);\n\n        // Test 3: Refreshing a lock token with valid a lock token should succeed\n        client\n            .lock_refresh(&path, &lock_token, \"infinity\", \"Second-456\")\n            .await\n            .with_status(StatusCode::OK)\n            .with_value(\n                \"D:prop.D:lockdiscovery.D:activelock.D:owner.href\",\n                \"super-owner\",\n            )\n            .with_any_value(\n                \"D:prop.D:lockdiscovery.D:activelock.D:timeout\",\n                [\"Second-456\", \"Second-455\"],\n            );\n\n        // Test 3: Creating a collection under an unmapped resource with a lock token should fail\n        client\n            .request_with_headers(\"MKCOL\", &path, [], \"\")\n            .await\n            .with_status(StatusCode::LOCKED)\n            .with_value(\"D:error.D:lock-token-submitted.D:href\", &path);\n\n        // Test 4: Creating a collection under a mapped resource with a lock token should succeed\n        client\n            .request_with_headers(\n                \"MKCOL\",\n                &path,\n                [(\"if\", format!(\"(<{lock_token}>)\").as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n\n        // Test 5: Creating a lock under an infinity locked resource should fail\n        let file_path = format!(\"{path}/file.txt\");\n        client\n            .lock_create(&file_path, \"super-owner\", true, \"0\", \"Second-123\")\n            .await\n            .with_status(StatusCode::LOCKED)\n            .with_value(\"D:error.D:lock-token-submitted.D:href\", &path);\n\n        // Test 6: Creating a file under a locked resource without a lock token should fail\n        let contents = resource_type.generate();\n        client\n            .request(\"PUT\", &file_path, &contents)\n            .await\n            .with_status(StatusCode::LOCKED)\n            .with_value(\"D:error.D:lock-token-submitted.D:href\", &path);\n\n        // Test 7: Creating a file under a locked resource with a lock token should succeed\n        client\n            .request_with_headers(\n                \"PUT\",\n                &file_path,\n                [(\"if\", format!(\"(<{lock_token}>)\").as_str())],\n                &contents,\n            )\n            .await\n            .with_status(StatusCode::CREATED);\n\n        // Test 8: Locks should be included in propfind responses\n        let response = client\n            .propfind(&path, [DavProperty::WebDav(WebDavProperty::LockDiscovery)])\n            .await;\n        for href in [path.clone() + \"/\", file_path] {\n            let props = response.properties(&href);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::LockDiscovery))\n                .with_some_values([\n                    \"D:activelock.D:owner.href:super-owner\",\n                    \"D:activelock.D:depth:infinity\",\n                    format!(\"D:activelock.D:locktoken.D:href:{lock_token}\").as_str(),\n                    format!(\"D:activelock.D:lockroot.D:href:{path}\").as_str(),\n                    \"D:activelock.D:locktype.D:write\",\n                    \"D:activelock.D:lockscope.D:exclusive\",\n                ])\n                .with_any_values([\n                    \"D:activelock.D:timeout:Second-456\",\n                    \"D:activelock.D:timeout:Second-455\",\n                ]);\n        }\n\n        // Test 9: Delete with and without a lock token\n        client\n            .request(\"DELETE\", &path, \"\")\n            .await\n            .with_status(StatusCode::LOCKED)\n            .with_value(\"D:error.D:lock-token-submitted.D:href\", &path);\n        client\n            .request_with_headers(\n                \"DELETE\",\n                &path,\n                [(\"if\", format!(\"(<{lock_token}>)\").as_str())],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n\n        // Test 10: Unlock with and without a lock token\n        client\n            .unlock(&path, \"urn:stalwart:davlock:1234\")\n            .await\n            .with_status(StatusCode::CONFLICT)\n            .with_value(\"D:error.D:lock-token-matches-request-uri\", \"\");\n        client\n            .unlock(&path, &lock_token)\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n\n        // Test 11: Locking with a large dead property should fail\n        let path = format!(\"{base_path}/invalid-lock\");\n        client\n            .lock_create(\n                &path,\n                (0..=test.server.core.groupware.dead_property_size.unwrap() + 1)\n                    .map(|_| \"a\")\n                    .collect::<String>()\n                    .as_str(),\n                true,\n                \"infinity\",\n                \"Second-123\",\n            )\n            .await\n            .with_status(StatusCode::PAYLOAD_TOO_LARGE);\n\n        // Test 12: Too many locks should fail\n        for i in 0..test.server.core.groupware.max_locks_per_user {\n            client\n                .lock_create(\n                    &format!(\"{base_path}/invalid-lock-{i}\"),\n                    \"super-owner\",\n                    true,\n                    \"infinity\",\n                    \"Second-123\",\n                )\n                .await\n                .with_status(StatusCode::CREATED);\n        }\n        client\n            .lock_create(\n                &format!(\"{base_path}/invalid-lock-greedy\"),\n                \"super-owner\",\n                true,\n                \"infinity\",\n                \"Second-123\",\n            )\n            .await\n            .with_status(StatusCode::TOO_MANY_REQUESTS);\n    }\n\n    client.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\nconst LOCK_REQUEST: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n     <D:lockinfo xmlns:D='DAV:'>\n       <D:lockscope><D:$TYPE/></D:lockscope>\n       <D:locktype><D:write/></D:locktype>\n       <D:owner>\n         <D:href>$OWNER</D:href>\n       </D:owner>\n     </D:lockinfo>\"#;\n\nimpl DummyWebDavClient {\n    pub async fn lock_create(\n        &self,\n        path: &str,\n        owner: &str,\n        is_exclusive: bool,\n        depth: &str,\n        timeout: &str,\n    ) -> DavResponse {\n        let lock_request = LOCK_REQUEST\n            .replace(\"$TYPE\", if is_exclusive { \"exclusive\" } else { \"shared\" })\n            .replace(\"$OWNER\", owner);\n        self.request_with_headers(\n            \"LOCK\",\n            path,\n            [(\"depth\", depth), (\"timeout\", timeout)],\n            &lock_request,\n        )\n        .await\n    }\n\n    pub async fn lock_refresh(\n        &self,\n        path: &str,\n        lock_token: &str,\n        depth: &str,\n        timeout: &str,\n    ) -> DavResponse {\n        let condition = format!(\"(<{lock_token}>)\");\n        self.request_with_headers(\n            \"LOCK\",\n            path,\n            [\n                (\"if\", condition.as_str()),\n                (\"depth\", depth),\n                (\"timeout\", timeout),\n            ],\n            \"\",\n        )\n        .await\n    }\n\n    pub async fn unlock(&self, path: &str, lock_token: &str) -> DavResponse {\n        let condition = format!(\"<{lock_token}>\");\n        self.request_with_headers(\"UNLOCK\", path, [(\"lock-token\", condition.as_str())], \"\")\n            .await\n    }\n}\n\nimpl DavResponse {\n    pub fn lock_token(&self) -> &str {\n        self.value(\"D:prop.D:lockdiscovery.D:activelock.D:locktoken.D:href\")\n    }\n}\n"
  },
  {
    "path": "tests/src/webdav/mkcol.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse hyper::StatusCode;\n\nuse crate::webdav::{TEST_FILE_1, TEST_ICAL_1, TEST_VCARD_1, TEST_VTIMEZONE_1};\n\nuse super::{DavResponse, DummyWebDavClient, WebDavTest};\n\npub async fn test(test: &WebDavTest) {\n    println!(\"Running MKCOL tests...\");\n    let client = test.client(\"john\");\n\n    // Creating collections in root elements is not allowed\n    for path in [\n        \"/dav/file/test\",\n        \"/dav/card/test\",\n        \"/dav/cal/test\",\n        \"/dav/test\",\n    ] {\n        client\n            .request(\"MKCOL\", path, \"\")\n            .await\n            .with_status(StatusCode::NOT_FOUND);\n    }\n\n    // Create collections using MKCOL (empty body)\n    for path in [\n        \"/dav/file/john/my-files\",\n        \"/dav/card/john/my-cards\",\n        \"/dav/cal/john/my-events\",\n    ] {\n        client\n            .request(\"MKCOL\", path, \"\")\n            .await\n            .with_status(StatusCode::CREATED);\n    }\n\n    // Create resources under the newly created collections\n    for (path, content) in [\n        (\"/dav/file/john/my-files/file1.txt\", TEST_FILE_1),\n        (\"/dav/card/john/my-cards/card1.vcf\", TEST_VCARD_1),\n        (\"/dav/cal/john/my-events/event1.ics\", TEST_ICAL_1),\n    ] {\n        client\n            .request(\"PUT\", path, content)\n            .await\n            .with_status(StatusCode::CREATED);\n    }\n\n    // Creating a collection on a mapped resource should fail\n    for path in [\n        \"/dav/file/john/my-files\",\n        \"/dav/card/john/my-cards\",\n        \"/dav/cal/john/my-events\",\n        \"/dav/file/john/my-files/file1.txt\",\n        \"/dav/card/john/my-cards/card1.vcf\",\n        \"/dav/cal/john/my-events/event1.ics\",\n    ] {\n        client\n            .request(\"MKCOL\", path, \"\")\n            .await\n            .with_status(StatusCode::METHOD_NOT_ALLOWED);\n    }\n\n    // Creating a sub-collections is allowed in FileDAV but in CalDAV and CardDAV\n    for (path, expected_status) in [\n        (\"/dav/file/john/my-files/my-sub-files\", StatusCode::CREATED),\n        (\n            \"/dav/card/john/my-cards/my-sub-cards\",\n            StatusCode::METHOD_NOT_ALLOWED,\n        ),\n        (\n            \"/dav/cal/john/my-events/my-sub-events\",\n            StatusCode::METHOD_NOT_ALLOWED,\n        ),\n    ] {\n        client\n            .request(\"MKCOL\", path, \"\")\n            .await\n            .with_status(expected_status);\n    }\n\n    // Extended MKCOL with an unsupported resource types should fail\n    for (path, resource_type) in [\n        (\"/dav/file/john/my-named-files\", \"B:addressbook\"),\n        (\"/dav/card/john/my-named-cards\", \"A:calendar\"),\n        (\"/dav/cal/john/my-named-events\", \"B:addressbook\"),\n    ] {\n        client\n            .mkcol(\"MKCOL\", path, [\"D:collection\", resource_type], [])\n            .await\n            .with_status(StatusCode::FORBIDDEN)\n            .with_value(\n                \"D:mkcol-response.D:propstat.D:error.D:valid-resourcetype\",\n                \"\",\n            )\n            .with_value(\"D:mkcol-response.D:propstat.D:prop.D:resourcetype\", \"\");\n    }\n\n    // Create using extended MKCOL\n    for (path, expected_properties, resource_types) in [\n        (\n            \"/dav/file/john/my-named-files/\",\n            [(\"D:displayname\", \"Named Files\")].as_slice(),\n            [\"D:collection\"].as_slice(),\n        ),\n        (\n            \"/dav/card/john/my-named-cards/\",\n            [\n                (\"D:displayname\", \"Named Cards\"),\n                (\"B:addressbook-description\", \"Some amazing contacts\"),\n            ]\n            .as_slice(),\n            [\"D:collection\", \"B:addressbook\"].as_slice(),\n        ),\n        (\n            \"/dav/cal/john/my-named-events/\",\n            [\n                (\"D:displayname\", \"Named Events\"),\n                (\"A:calendar-description\", \"Some amazing events\"),\n                (\n                    \"A:calendar-timezone\",\n                    &TEST_VTIMEZONE_1.replace(\"\\n\", \"\\r\\n\"),\n                ),\n            ]\n            .as_slice(),\n            [\"D:collection\", \"A:calendar\"].as_slice(),\n        ),\n    ] {\n        let response = client\n            .mkcol(\n                \"MKCOL\",\n                path,\n                resource_types.iter().copied(),\n                expected_properties.iter().copied(),\n            )\n            .await\n            .with_status(StatusCode::CREATED)\n            .into_propfind_response(\"D:mkcol-response\".into());\n        let properties = response.properties(\"\");\n        for (property, _) in expected_properties {\n            properties\n                .get(property)\n                .with_status(StatusCode::OK)\n                .with_values([\"\"]);\n        }\n\n        // Check the properties of the created collection\n        let response = client\n            .propfind(path, expected_properties.iter().map(|x| x.0))\n            .await;\n        let properties = response.properties(path);\n        for (property, value) in expected_properties {\n            properties\n                .get(property)\n                .with_status(StatusCode::OK)\n                .with_values([*value]);\n        }\n    }\n\n    // Test MKCALENDAR\n    client\n        .mkcol(\n            \"MKCALENDAR\",\n            \"/dav/cal/john/my-named-events2\",\n            [],\n            [\n                (\"D:displayname\", \"Named Events 2\"),\n                (\"A:calendar-description\", \"\"),\n            ],\n        )\n        .await\n        .with_status(StatusCode::CREATED)\n        .with_value(\"A:mkcalendar-response.D:propstat.D:prop.D:displayname\", \"\")\n        .with_values(\n            \"A:mkcalendar-response.D:propstat.D:status\",\n            [\"HTTP/1.1 200 OK\"],\n        );\n    client\n        .mkcol(\n            \"MKCALENDAR\",\n            \"/dav/cal/john/my-named-events3\",\n            [],\n            [\n                (\"D:displayname\", \"Named Events 3\"),\n                (\n                    \"A:supported-calendar-component-set\",\n                    \"<A:comp name=\\\"VEVENT\\\"/><A:comp name=\\\"VTODO\\\"/>\",\n                ),\n            ],\n        )\n        .await\n        .with_status(StatusCode::CREATED)\n        .with_value(\"A:mkcalendar-response.D:propstat.D:prop.D:displayname\", \"\")\n        .with_values(\n            \"A:mkcalendar-response.D:propstat.D:status\",\n            [\"HTTP/1.1 200 OK\"],\n        );\n    // Check the properties of the created calendars\n    client\n        .propfind(\n            \"/dav/cal/john/my-named-events2/\",\n            [\"A:supported-calendar-component-set\"],\n        )\n        .await\n        .properties(\"/dav/cal/john/my-named-events2/\")\n        .get(\"A:supported-calendar-component-set\")\n        .with_status(StatusCode::OK)\n        .with_values([\n            \"A:comp.[name]:VJOURNAL\",\n            \"A:comp.[name]:VTIMEZONE\",\n            \"A:comp.[name]:VAVAILABILITY\",\n            \"A:comp.[name]:VALARM\",\n            \"A:comp.[name]:VRESOURCE\",\n            \"A:comp.[name]:AVAILABLE\",\n            \"A:comp.[name]:VTODO\",\n            \"A:comp.[name]:VFREEBUSY\",\n            \"A:comp.[name]:VEVENT\",\n            \"A:comp.[name]:STANDARD\",\n            \"A:comp.[name]:DAYLIGHT\",\n            \"A:comp.[name]:VLOCATION\",\n            \"A:comp.[name]:PARTICIPANT\",\n        ]);\n    client\n        .propfind(\n            \"/dav/cal/john/my-named-events3/\",\n            [\"A:supported-calendar-component-set\"],\n        )\n        .await\n        .properties(\"/dav/cal/john/my-named-events3/\")\n        .get(\"A:supported-calendar-component-set\")\n        .with_status(StatusCode::OK)\n        .with_values([\"A:comp.[name]:VEVENT\", \"A:comp.[name]:VTODO\"]);\n\n    // Delete everything\n    for path in [\n        \"/dav/file/john/my-files\",\n        \"/dav/card/john/my-cards\",\n        \"/dav/cal/john/my-events\",\n        \"/dav/file/john/my-named-files\",\n        \"/dav/card/john/my-named-cards\",\n        \"/dav/cal/john/my-named-events\",\n        \"/dav/cal/john/my-named-events2\",\n        \"/dav/cal/john/my-named-events3\",\n    ] {\n        client\n            .request(\"DELETE\", path, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n    }\n    client.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\nimpl DummyWebDavClient {\n    pub async fn mkcol(\n        &self,\n        method: &str,\n        path: &str,\n        resource_types: impl IntoIterator<Item = &str>,\n        properties: impl IntoIterator<Item = (&str, &str)>,\n    ) -> DavResponse {\n        let mut request = concat!(\n            \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\",\n            \"<D:mkcol xmlns:D=\\\"DAV:\\\" xmlns:A=\\\"urn:ietf:params:xml:ns:caldav\\\" xmlns:B=\\\"urn:ietf:params:xml:ns:carddav\\\">\",\n            \"<D:set><D:prop>\"\n        )\n        .to_string();\n\n        let mut has_resource_type = false;\n        for (idx, resource_type) in resource_types.into_iter().enumerate() {\n            if idx == 0 {\n                request.push_str(\"<D:resourcetype>\");\n            }\n            request.push_str(&format!(\"<{resource_type}/>\"));\n            has_resource_type = true;\n        }\n\n        if has_resource_type {\n            request.push_str(\"</D:resourcetype>\");\n        }\n\n        for (key, value) in properties {\n            request.push_str(&format!(\"<{key}>{value}</{key}>\"));\n        }\n        request.push_str(\"</D:prop></D:set></D:mkcol>\");\n\n        if method == \"MKCALENDAR\" {\n            request = request.replace(\"D:mkcol\", \"A:mkcalendar\");\n        }\n\n        self.request(method, path, &request).await\n    }\n}\n"
  },
  {
    "path": "tests/src/webdav/mod.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse crate::{\n    AssertConfig, TEST_USERS, add_test_certs,\n    directory::internal::TestInternalDirectory,\n    jmap::{assert_is_empty, wait_for_index},\n    store::{\n        TempDir, build_store_config,\n        cleanup::{search_store_destroy, store_destroy},\n    },\n};\nuse ::managesieve::core::ManageSieveSessionManager;\nuse ::store::Stores;\nuse ahash::{AHashMap, AHashSet};\nuse base64::{Engine, engine::general_purpose::STANDARD};\nuse common::{\n    Caches, Core, Data, DavResource, DavResources, Inner, Server,\n    config::{\n        server::{Listeners, ServerProtocol},\n        telemetry::Telemetry,\n    },\n    core::BuildServer,\n    manager::boot::build_ipc,\n};\nuse dav_proto::{\n    schema::property::{DavProperty, WebDavProperty},\n    xml_pretty_print,\n};\nuse directory::Permission;\nuse email::message::metadata::MessageMetadata;\nuse groupware::{DavResourceName, cache::GroupwareCache};\nuse http::HttpSessionManager;\nuse hyper::{HeaderMap, Method, StatusCode, header::AUTHORIZATION};\nuse imap::core::ImapSessionManager;\nuse pop3::Pop3SessionManager;\nuse quick_xml::Reader;\nuse quick_xml::events::Event;\nuse services::SpawnServices;\nuse smtp::{SpawnQueueManager, core::SmtpSessionManager};\nuse std::{borrow::Cow, str};\nuse std::{\n    sync::Arc,\n    time::{Duration, Instant},\n};\nuse store::{\n    ValueKey,\n    rand::{Rng, distr::Alphanumeric, rng},\n    write::{AlignedBytes, Archive},\n};\nuse tokio::sync::watch;\nuse types::{collection::Collection, field::EmailField};\nuse utils::config::Config;\n\npub mod acl;\npub mod basic;\npub mod cal_alarm;\npub mod cal_itip;\npub mod cal_query;\npub mod cal_scheduling;\npub mod card_query;\npub mod copy_move;\npub mod lock;\npub mod mkcol;\npub mod multiget;\npub mod principals;\npub mod prop;\npub mod put_get;\npub mod sync;\n\n#[test]\nfn webdav_tests() {\n    //test_build_itip_templates(&handle.server).await;\n\n    tokio::runtime::Builder::new_multi_thread()\n        .thread_stack_size(8 * 1024 * 1024) // 8MB stack\n        .enable_all()\n        .build()\n        .unwrap()\n        .block_on(async {\n            // Prepare settings\n            let assisted_discovery = std::env::var(\"ASSISTED_DISCOVERY\").unwrap_or_default() == \"1\";\n            let start_time = Instant::now();\n            let delete = true;\n            let handle = init_webdav_tests(assisted_discovery, delete).await;\n\n            basic::test(&handle).await;\n            put_get::test(&handle).await;\n            mkcol::test(&handle).await;\n            copy_move::test(&handle, assisted_discovery).await;\n            prop::test(&handle, assisted_discovery).await;\n            multiget::test(&handle).await;\n            sync::test(&handle).await;\n            lock::test(&handle).await;\n            principals::test(&handle, assisted_discovery).await;\n            acl::test(&handle).await;\n            card_query::test(&handle).await;\n            cal_query::test(&handle).await;\n            cal_alarm::test(&handle).await;\n            cal_itip::test();\n            cal_scheduling::test(&handle).await;\n\n            // Print elapsed time\n            let elapsed = start_time.elapsed();\n            println!(\n                \"Elapsed: {}.{:03}s\",\n                elapsed.as_secs(),\n                elapsed.subsec_millis()\n            );\n\n            // Remove test data\n            if delete {\n                handle.temp_dir.delete();\n            }\n        });\n}\n\n#[allow(dead_code)]\npub struct WebDavTest {\n    server: Server,\n    clients: AHashMap<&'static str, DummyWebDavClient>,\n    temp_dir: TempDir,\n    shutdown_tx: watch::Sender<bool>,\n}\n\nasync fn init_webdav_tests(assisted_discovery: bool, delete_if_exists: bool) -> WebDavTest {\n    // Load and parse config\n    let temp_dir = TempDir::new(\"webdav_tests\", delete_if_exists);\n    let mut config = Config::new(\n        add_test_certs(&(build_store_config(&temp_dir.path.to_string_lossy()) + SERVER))\n            .replace(\"{TMP}\", &temp_dir.path.display().to_string())\n            .replace(\"{ASSISTED_DISCOVERY}\", &assisted_discovery.to_string())\n            .replace(\n                \"{LEVEL}\",\n                &std::env::var(\"LOG\").unwrap_or_else(|_| \"disable\".to_string()),\n            ),\n    )\n    .unwrap();\n    config.resolve_all_macros().await;\n\n    // Parse servers\n    let mut servers = Listeners::parse(&mut config);\n\n    // Bind ports and drop privileges\n    servers.bind_and_drop_priv(&mut config);\n\n    // Build stores\n    let stores = Stores::parse_all(&mut config, false).await;\n\n    // Parse core\n    let tracers = Telemetry::parse(&mut config, &stores);\n    let core = Core::parse(&mut config, stores, Default::default()).await;\n    let data = Data::parse(&mut config);\n    let cache = Caches::parse(&mut config);\n\n    let store = core.storage.data.clone();\n    let search_store = core.storage.fts.clone();\n    let (ipc, mut ipc_rxs) = build_ipc(false);\n    let inner = Arc::new(Inner {\n        shared_core: core.into_shared(),\n        data,\n        ipc,\n        cache,\n    });\n\n    // Parse acceptors\n    servers.parse_tcp_acceptors(&mut config, inner.clone());\n\n    // Enable tracing\n    tracers.enable(true);\n\n    // Start services\n    config.assert_no_errors();\n    ipc_rxs.spawn_queue_manager(inner.clone());\n    ipc_rxs.spawn_services(inner.clone());\n\n    // Spawn servers\n    let (shutdown_tx, _) = servers.spawn(|server, acceptor, shutdown_rx| {\n        match &server.protocol {\n            ServerProtocol::Smtp | ServerProtocol::Lmtp => server.spawn(\n                SmtpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Http => server.spawn(\n                HttpSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Imap => server.spawn(\n                ImapSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::Pop3 => server.spawn(\n                Pop3SessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n            ServerProtocol::ManageSieve => server.spawn(\n                ManageSieveSessionManager::new(inner.clone()),\n                inner.clone(),\n                acceptor,\n                shutdown_rx,\n            ),\n        };\n    });\n\n    if delete_if_exists {\n        store_destroy(&store).await;\n        search_store_destroy(&search_store).await;\n    }\n\n    // Create test accounts\n    let mut clients = AHashMap::new();\n    for (account, secret, name, email) in TEST_USERS {\n        let account_id = store\n            .create_test_user(account, secret, name, &[email])\n            .await;\n        clients.insert(\n            *account,\n            DummyWebDavClient::new(account_id, account, secret, email),\n        );\n        store\n            .add_permissions(\n                account,\n                [Permission::DavPrincipalList, Permission::DavPrincipalSearch],\n            )\n            .await;\n        if *account == \"mike\" {\n            store.set_test_quota(account, 1024).await;\n        }\n    }\n    store\n        .create_test_group(\"support\", \"Support Group\", &[\"support@example.com\"])\n        .await;\n    store.add_to_group(\"jane\", \"support\").await;\n\n    WebDavTest {\n        server: inner.build_server(),\n        clients,\n        temp_dir,\n        shutdown_tx,\n    }\n}\n\nimpl WebDavTest {\n    pub fn client(&self, name: &'static str) -> &DummyWebDavClient {\n        self.clients.get(name).unwrap()\n    }\n\n    pub async fn resources(&self, name: &'static str, collection: Collection) -> Arc<DavResources> {\n        let account_id = self.client(name).account_id;\n        let access_token = self.server.get_access_token(account_id).await.unwrap();\n        self.server\n            .fetch_dav_resources(&access_token, account_id, collection.into())\n            .await\n            .unwrap()\n    }\n\n    pub fn clear_cache(&self) {\n        for cache in [\n            &self.server.inner.cache.events,\n            &self.server.inner.cache.contacts,\n            &self.server.inner.cache.files,\n        ] {\n            cache.clear();\n        }\n    }\n\n    pub async fn assert_is_empty(&self) {\n        assert_is_empty(&self.server).await;\n        self.clear_cache();\n    }\n\n    pub async fn wait_for_index(&self) {\n        wait_for_index(&self.server).await;\n    }\n}\n\n#[allow(dead_code)]\n#[derive(Debug)]\npub struct DummyWebDavClient {\n    account_id: u32,\n    name: &'static str,\n    email: &'static str,\n    credentials: String,\n}\n\n#[derive(Debug)]\npub struct DavResponse {\n    headers: AHashMap<String, String>,\n    status: StatusCode,\n    body: Result<String, String>,\n    xml: Vec<(String, String)>,\n}\n\nimpl DummyWebDavClient {\n    pub fn new(\n        account_id: u32,\n        name: &'static str,\n        secret: &'static str,\n        email: &'static str,\n    ) -> Self {\n        Self {\n            account_id,\n            name,\n            email,\n            credentials: format!(\n                \"Basic {}\",\n                STANDARD.encode(format!(\"{name}:{secret}\").as_bytes())\n            ),\n        }\n    }\n\n    pub async fn request(&self, method: &str, query: &str, body: impl Into<String>) -> DavResponse {\n        self.request_with_headers(method, query, [].into_iter(), body)\n            .await\n    }\n\n    pub async fn request_with_headers(\n        &self,\n        method: &str,\n        query: &str,\n        headers: impl IntoIterator<Item = (&'static str, &str)>,\n        body: impl Into<String>,\n    ) -> DavResponse {\n        let mut request = reqwest::Client::builder()\n            .timeout(Duration::from_millis(500))\n            .danger_accept_invalid_certs(true)\n            .build()\n            .unwrap()\n            .request(\n                Method::from_bytes(method.as_bytes()).unwrap(),\n                format!(\"https://127.0.0.1:8899{query}\"),\n            );\n\n        let body = body.into();\n        if !body.is_empty() {\n            request = request.body(body);\n        }\n\n        let mut request_headers = HeaderMap::new();\n        for (key, value) in headers {\n            request_headers.insert(key, value.parse().unwrap());\n        }\n        request_headers.insert(AUTHORIZATION, self.credentials.parse().unwrap());\n\n        let response = request.headers(request_headers).send().await.unwrap();\n        let status = response.status();\n        let headers = response\n            .headers()\n            .iter()\n            .map(|(k, v)| {\n                (\n                    k.to_string().to_lowercase(),\n                    v.to_str().unwrap().to_string(),\n                )\n            })\n            .collect();\n        let body = response\n            .bytes()\n            .await\n            .map(|bytes| String::from_utf8(bytes.to_vec()).unwrap())\n            .map_err(|err| err.to_string());\n        let xml = match &body {\n            Ok(body) if body.starts_with(\"<?xml\") => flatten_xml(body),\n            _ => vec![],\n        };\n\n        DavResponse {\n            headers,\n            status,\n            body,\n            xml,\n        }\n    }\n\n    pub async fn available_quota(&self, path: &str) -> u64 {\n        self.propfind(\n            path,\n            [DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes)],\n        )\n        .await\n        .properties(path)\n        .get(DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes))\n        .value()\n        .parse()\n        .unwrap()\n    }\n\n    pub async fn create_hierarchy(\n        &self,\n        base_path: &str,\n        max_depth: usize,\n        containers_per_level: usize,\n        files_per_container: usize,\n    ) -> (String, Vec<(String, String)>) {\n        let resource_type = if base_path.starts_with(\"/dav/card/\") {\n            DavResourceName::Card\n        } else if base_path.starts_with(\"/dav/cal/\") {\n            DavResourceName::Cal\n        } else {\n            DavResourceName::File\n        };\n\n        let mut created_resources = Vec::new();\n\n        self.create_hierarchy_recursive(\n            resource_type,\n            base_path,\n            max_depth,\n            containers_per_level,\n            files_per_container,\n            0,\n            &mut created_resources,\n        )\n        .await;\n\n        let root_folder = created_resources.first().unwrap().0.clone();\n        created_resources.sort_unstable_by(|a, b| a.0.cmp(&b.0));\n        (root_folder, created_resources)\n    }\n\n    #[allow(clippy::too_many_arguments)]\n    async fn create_hierarchy_recursive(\n        &self,\n        resource_type: DavResourceName,\n        base_path: &str,\n        max_depth: usize,\n        containers_per_level: usize,\n        files_per_container: usize,\n        current_depth: usize,\n        created_resources: &mut Vec<(String, String)>,\n    ) {\n        let folder_name = generate_random_name(4);\n        let folder_path = format!(\"{base_path}/Folder_{folder_name}\");\n\n        self.mkcol(\"MKCOL\", &folder_path, [], [])\n            .await\n            .with_status(StatusCode::CREATED);\n\n        created_resources.push((format!(\"{folder_path}/\"), \"\".to_string()));\n\n        for _ in 0..files_per_container {\n            let file_name = generate_random_name(8);\n            let file_path = format!(\n                \"{folder_path}/{file_name}.{}\",\n                match resource_type {\n                    DavResourceName::Card => \"vcf\",\n                    DavResourceName::Cal => \"ics\",\n                    DavResourceName::File => \"txt\",\n                    _ => unreachable!(),\n                }\n            );\n            let content = match resource_type {\n                DavResourceName::Card => generate_random_vcard(),\n                DavResourceName::Cal => generate_random_ical(),\n                DavResourceName::File => generate_random_content(100, 500),\n                _ => unreachable!(),\n            };\n\n            self.request(\"PUT\", &file_path, &content)\n                .await\n                .with_status(StatusCode::CREATED);\n\n            created_resources.push((file_path, content));\n        }\n\n        if current_depth < max_depth {\n            for _ in 0..containers_per_level {\n                Box::pin(self.create_hierarchy_recursive(\n                    resource_type,\n                    &folder_path,\n                    max_depth,\n                    containers_per_level,\n                    files_per_container,\n                    current_depth + 1,\n                    created_resources,\n                ))\n                .await;\n            }\n        }\n    }\n\n    pub async fn validate_values(&self, items: &[(String, String)]) {\n        for (path, value) in items {\n            if !path.ends_with('/') {\n                self.request(\"GET\", path, \"\")\n                    .await\n                    .with_status(StatusCode::OK)\n                    .with_body(value);\n            }\n        }\n    }\n\n    pub async fn delete_default_containers(&self) {\n        self.delete_default_containers_by_account(self.name).await;\n    }\n\n    pub async fn delete_default_containers_by_account(&self, account: &str) {\n        for col in [\"card\", \"cal\"] {\n            self.request(\"DELETE\", &format!(\"/dav/{col}/{account}/default\"), \"\")\n                .await\n                .with_status(StatusCode::NO_CONTENT);\n        }\n    }\n}\n\nimpl DavResponse {\n    pub fn with_status(self, status: StatusCode) -> Self {\n        if self.status != status {\n            self.dump_response();\n            panic!(\"Expected {status} but got {}\", self.status)\n        }\n        self\n    }\n\n    pub fn with_redirect_to(self, url: &str) -> Self {\n        self.with_status(StatusCode::TEMPORARY_REDIRECT)\n            .with_header(\"location\", url)\n    }\n\n    pub fn with_header(self, header: &str, value: &str) -> Self {\n        if self.headers.get(header).is_some_and(|v| v == value) {\n            self\n        } else {\n            self.dump_response();\n            panic!(\"Header {header}:{value} not found.\")\n        }\n    }\n\n    pub fn with_body(self, expect_body: impl AsRef<str>) -> Self {\n        let expect_body = expect_body.as_ref();\n        if self.body.is_ok() {\n            let body = self.body.as_ref().unwrap();\n            if body != expect_body {\n                self.dump_response();\n                assert_eq!(body, &expect_body);\n            }\n            self\n        } else {\n            self.dump_response();\n            panic!(\"Expected body {expect_body:?} but no body was returned.\")\n        }\n    }\n\n    pub fn with_empty_body(self) -> Self {\n        if self.body.is_ok() {\n            let body = self.body.as_ref().unwrap();\n            if !body.is_empty() {\n                self.dump_response();\n                panic!(\"Expected empty body but got {body:?}\");\n            }\n            self\n        } else {\n            self.dump_response();\n            panic!(\"Expected empty body but no body was returned.\")\n        }\n    }\n\n    pub fn expect_body(&self) -> &str {\n        if self.body.is_ok() {\n            self.body.as_ref().unwrap()\n        } else {\n            self.dump_response();\n            panic!(\"Expected body but no body was returned.\")\n        }\n    }\n\n    pub fn header(&self, header: &str) -> &str {\n        if let Some(value) = self.headers.get(header) {\n            value\n        } else {\n            self.dump_response();\n            panic!(\"Header {header} not found.\")\n        }\n    }\n\n    pub fn etag(&self) -> &str {\n        self.header(\"etag\")\n    }\n\n    pub fn sync_token(&self) -> &str {\n        self.find_keys(\"D:multistatus.D:sync-token\")\n            .next()\n            .filter(|v| !v.is_empty())\n            .unwrap_or_else(|| {\n                self.dump_response();\n                panic!(\"Sync token not found.\")\n            })\n    }\n\n    pub fn hrefs(&self) -> Vec<&str> {\n        let mut hrefs = self\n            .find_keys(\"D:multistatus.D:response.D:href\")\n            .collect::<Vec<_>>();\n        hrefs.sort_unstable();\n        hrefs\n    }\n\n    pub fn with_href_count(self, count: usize) -> Self {\n        let href_count = self.find_keys(\"D:multistatus.D:response.D:href\").count();\n        if href_count != count {\n            self.dump_response();\n            panic!(\"Expected {} hrefs but got {}\", count, href_count);\n        }\n        self\n    }\n\n    pub fn with_hrefs<'x>(self, hrefs: impl IntoIterator<Item = &'x str>) -> Self {\n        let expected_hrefs = hrefs.into_iter().collect::<AHashSet<_>>();\n        let hrefs = self\n            .find_keys(\"D:multistatus.D:response.D:href\")\n            .collect::<AHashSet<_>>();\n        if expected_hrefs != hrefs {\n            self.dump_response();\n\n            println!(\"\\nMissing: {:?}\", expected_hrefs.difference(&hrefs));\n            println!(\"\\nExtra: {:?}\", hrefs.difference(&expected_hrefs));\n\n            panic!(\n                \"Hierarchy mismatch: expected {} items, received {} items\",\n                expected_hrefs.len(),\n                hrefs.len()\n            );\n        }\n        self\n    }\n\n    fn dump_response(&self) {\n        eprintln!(\"-------------------------------------\");\n        eprintln!(\"Status: {}\", self.status);\n        eprintln!(\"Headers:\");\n        for (key, value) in self.headers.iter() {\n            eprintln!(\"  {}: {:?}\", key, value);\n        }\n        if !self.xml.is_empty() {\n            eprintln!(\"XML: {}\", xml_pretty_print(self.body.as_ref().unwrap()));\n\n            for (key, value) in self.xml.iter() {\n                eprintln!(\"{} -> {:?}\", key, value);\n            }\n        } else {\n            eprintln!(\"Body: {:?}\", self.body);\n        }\n    }\n\n    fn find_keys(&self, name: &str) -> impl Iterator<Item = &str> {\n        self.xml\n            .iter()\n            .filter(move |(key, _)| name == key)\n            .map(|(_, value)| value.as_str())\n    }\n\n    pub fn value(&self, name: &str) -> &str {\n        self.find_keys(name).next().unwrap_or_else(|| {\n            self.dump_response();\n            panic!(\"Key {name} not found.\")\n        })\n    }\n\n    // Poor man's XPath\n    pub fn with_value(self, query: &str, expect: impl AsRef<str>) -> Self {\n        let expect = expect.as_ref();\n        if let Some(value) = self.find_keys(query).next() {\n            if value != expect {\n                self.dump_response();\n                panic!(\"Expected {query} = {expect:?} but got {value:?}\");\n            }\n        } else {\n            self.dump_response();\n            panic!(\"Key {query} not found.\");\n        }\n        self\n    }\n\n    pub fn with_any_value<'x>(\n        self,\n        query: &str,\n        expect: impl IntoIterator<Item = &'x str>,\n    ) -> Self {\n        let expect = expect.into_iter().collect::<AHashSet<_>>();\n        if let Some(value) = self.find_keys(query).next() {\n            if !expect.contains(value) {\n                self.dump_response();\n                panic!(\"Expected {query} = {expect:?} but got {value:?}\");\n            }\n        } else {\n            self.dump_response();\n            panic!(\"Key {query} not found.\");\n        }\n        self\n    }\n\n    pub fn with_values<I, T>(self, query: &str, expect: I) -> Self\n    where\n        I: IntoIterator<Item = T>,\n        T: AsRef<str>,\n    {\n        let expect_owned: Vec<T> = expect.into_iter().collect();\n        let expect = expect_owned.iter().map(|s| s.as_ref()).collect::<Vec<_>>();\n        let found = self.find_keys(query).collect::<Vec<_>>();\n        if expect != found {\n            self.dump_response();\n            panic!(\"Expected {query} = {expect:?} but got {found:?}\");\n        }\n        self\n    }\n\n    pub fn with_failed_precondition(self, precondition: &str, value: &str) -> Self {\n        let error = format!(\"D:error.{precondition}\");\n        if self.find_keys(&error).next().is_none_or(|v| v != value) {\n            self.dump_response();\n            panic!(\"Precondition {precondition} did not match.\");\n        }\n        self\n    }\n}\n\npub trait DavResourcesTest {\n    fn items(&self) -> Vec<DavResource>;\n}\n\nimpl DavResourcesTest for DavResources {\n    fn items(&self) -> Vec<DavResource> {\n        self.resources.clone()\n    }\n}\n\nfn flatten_xml(xml: &str) -> Vec<(String, String)> {\n    let mut reader = Reader::from_str(xml);\n\n    let mut path: Vec<String> = Vec::new();\n    let mut result: Vec<(String, String)> = Vec::new();\n    let mut buf = Vec::new();\n    let mut text_content: Option<String> = None;\n\n    loop {\n        match reader.read_event_into(&mut buf).unwrap() {\n            Event::Start(ref e) => {\n                let name = str::from_utf8(e.name().as_ref()).unwrap().to_string();\n                path.push(name);\n                let base_path = path.join(\".\");\n                for attr in e.attributes() {\n                    let attr = attr.unwrap();\n                    let key = str::from_utf8(attr.key.as_ref()).unwrap().to_string();\n                    let value = attr.unescape_value().unwrap();\n                    let value_str = value.trim().to_string();\n\n                    result.push((format!(\"{}.[{}]\", base_path, key), value_str));\n                }\n                text_content = None;\n            }\n            Event::Empty(ref e) => {\n                let name = str::from_utf8(e.name().as_ref()).unwrap().to_string();\n                let base_path = format!(\"{}.{}\", path.join(\".\"), name);\n                let mut has_attrs = false;\n\n                for attr in e.attributes() {\n                    let attr = attr.unwrap();\n                    let key = str::from_utf8(attr.key.as_ref()).unwrap().to_string();\n                    let value = attr.unescape_value().unwrap();\n                    let value_str = value.trim().to_string();\n                    has_attrs = true;\n                    result.push((format!(\"{}.[{}]\", base_path, key), value_str));\n                }\n\n                if !has_attrs {\n                    result.push((base_path, \"\".to_string()));\n                }\n            }\n            Event::Text(e) => {\n                let text = e.xml_content().unwrap();\n                let trimmed = text.trim();\n                if !trimmed.is_empty() {\n                    if let Some(text_content) = text_content.as_mut() {\n                        text_content.push_str(trimmed);\n                    } else {\n                        text_content = Some(trimmed.to_string());\n                    }\n                }\n            }\n            Event::GeneralRef(entity) => {\n                let value: Cow<str> = match entity.as_ref() {\n                    b\"lt\" => \"<\".into(),\n                    b\"gt\" => \">\".into(),\n                    b\"amp\" => \"&\".into(),\n                    b\"apos\" => \"'\".into(),\n                    b\"quot\" => \"\\\"\".into(),\n                    _ => {\n                        if let Ok(Some(gr)) = entity.resolve_char_ref() {\n                            gr.to_string().into()\n                        } else {\n                            std::str::from_utf8(entity.as_ref())\n                                .unwrap_or_default()\n                                .into()\n                        }\n                    }\n                };\n\n                if let Some(text_content) = text_content.as_mut() {\n                    text_content.push_str(value.as_ref());\n                } else {\n                    text_content = Some(value.into_owned());\n                }\n            }\n            Event::CData(e) => {\n                text_content = Some(std::str::from_utf8(e.as_ref()).unwrap().to_string());\n            }\n            Event::End(_) => {\n                if let Some(text) = text_content.take() {\n                    result.push((path.join(\".\"), text));\n                }\n\n                if !path.is_empty() {\n                    path.pop();\n                }\n            }\n            Event::Eof => break,\n            _ => {}\n        }\n        buf.clear();\n    }\n\n    result\n}\n\npub const TEST_VCARD_1: &str = r#\"BEGIN:VCARD\nVERSION:4.0\nUID:18F098B5-7383-4FD6-B482-48F2181D73AA\nX-TEST:SEQ1\nN:Coyote;Wile;E.;;\nFN:Wile E. Coyote\nORG:ACME Inc.;\nEND:VCARD\n\"#;\n\npub const TEST_VCARD_2: &str = r#\"BEGIN:VCARD\nVERSION:4.0\nUID:6exhjr32bt783wwlr9u0sr8lfqse5x7zqc8y\nX-TEST:SEQ1\nFN:Joe Citizen\nN:Citizen;Joe;;;\nNICKNAME:human_being\nEMAIL;TYPE=pref:jcitizen@foo.com\nREV:20200411T072429Z\nEND:VCARD\n\"#;\n\npub const TEST_ICAL_1: &str = r#\"BEGIN:VCALENDAR\nSOURCE;VALUE=URI:http://calendar.example.com/event_with_html.ics\nX-TEST:SEQ1\nBEGIN:VEVENT\nUID: 2371c2d9-a136-43b0-bba3-f6ab249ad46e\nSUMMARY:What a nice present: 🎁\nDTSTART;TZID=America/New_York:20190221T170000\nDTEND;TZID=America/New_York:20190221T180000\nLOCATION:Germany\nDESCRIPTION:<html><body><h1>Title</h1><p><ul><li><b>first</b> Row </li><li><\n i>second</i> Row</li></ul></p></body></html>\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\npub const TEST_ICAL_2: &str = r#\"BEGIN:VCALENDAR\nX-TEST:SEQ1\nBEGIN:VEVENT\nUID:0000001\nSUMMARY:Treasure Hunting\nDTSTART;TZID=America/Los_Angeles:20150706T120000\nDTEND;TZID=America/Los_Angeles:20150706T130000\nRRULE:FREQ=DAILY;COUNT=10\nEXDATE;TZID=America/Los_Angeles:20150708T120000\nEXDATE;TZID=America/Los_Angeles:20150710T120000\nEND:VEVENT\nBEGIN:VEVENT\nUID:0000001\nSUMMARY:More Treasure Hunting\nLOCATION:The other island\nDTSTART;TZID=America/Los_Angeles:20150709T150000\nDTEND;TZID=America/Los_Angeles:20150707T160000\nRECURRENCE-ID;TZID=America/Los_Angeles:20150707T120000\nEND:VEVENT\nEND:VCALENDAR\n\"#;\n\npub const TEST_FILE_1: &str = r#\"this is a test file\nwith some text\nand some more text\n\nX-TEST:SEQ1\n\"#;\n\npub const TEST_FILE_2: &str = r#\"another test file\nwith amazing content\nand some more text\n\nX-TEST:SEQ1\n\"#;\n\npub const TEST_VTIMEZONE_1: &str = r#\"BEGIN:VCALENDAR\nPRODID:-//Example Corp.//CalDAV Client//EN\nVERSION:2.0\nBEGIN:VTIMEZONE\nTZID:US-Eastern\nLAST-MODIFIED:19870101T000000Z\nBEGIN:STANDARD\nDTSTART:19671029T020000\nRRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10\nTZOFFSETFROM:-0400\nTZOFFSETTO:-0500\nTZNAME:Eastern Standard Time (US Canada)\nEND:STANDARD\nBEGIN:DAYLIGHT\nDTSTART:19870405T020000\nRRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4\nTZOFFSETFROM:-0500\nTZOFFSETTO:-0400\nTZNAME:Eastern Daylight Time (US Canada)\nEND:DAYLIGHT\nEND:VTIMEZONE\nEND:VCALENDAR\n\"#;\n\npub trait GenerateTestDavResource {\n    fn generate(&self) -> String;\n}\n\nimpl GenerateTestDavResource for DavResourceName {\n    fn generate(&self) -> String {\n        match self {\n            DavResourceName::Card => generate_random_vcard(),\n            DavResourceName::Cal => generate_random_ical(),\n            DavResourceName::File => generate_random_content(100, 200),\n            _ => unreachable!(),\n        }\n    }\n}\n\nfn generate_random_vcard() -> String {\n    r#\"BEGIN:VCARD\nVERSION:4.0\nUID:$UID\nFN:$NAME\nEND:VCARD\n\"#\n    .replace(\"$UID\", &generate_random_name(8))\n    .replace(\"$NAME\", &generate_random_name(10))\n    .replace('\\n', \"\\r\\n\")\n}\n\nfn generate_random_ical() -> String {\n    r#\"BEGIN:VCALENDAR\nVERSION:2.0\nBEGIN:VEVENT\nUID:$UID\nSUMMARY:$SUMMARY\nDESCRIPTION:$DESCRIPTION\nEND:VEVENT\nEND:VCALENDAR\n\"#\n    .replace(\"$UID\", &generate_random_name(8))\n    .replace(\"$SUMMARY\", &generate_random_name(10))\n    .replace(\"$DESCRIPTION\", &generate_random_name(20))\n    .replace('\\n', \"\\r\\n\")\n}\n\nfn generate_random_content(min_chars: usize, max_chars: usize) -> String {\n    let mut rng = rng();\n    let length = rng.random_range(min_chars..=max_chars);\n\n    let words = [\n        \"lorem\",\n        \"ipsum\",\n        \"dolor\",\n        \"sit\",\n        \"amet\",\n        \"consectetur\",\n        \"adipiscing\",\n        \"elit\",\n        \"sed\",\n        \"do\",\n        \"eiusmod\",\n        \"tempor\",\n        \"incididunt\",\n        \"ut\",\n        \"labore\",\n        \"et\",\n        \"dolore\",\n        \"magna\",\n        \"aliqua\",\n        \"ut\",\n        \"enim\",\n        \"ad\",\n        \"minim\",\n        \"veniam\",\n        \"quis\",\n        \"nostrud\",\n        \"exercitation\",\n        \"ullamco\",\n        \"laboris\",\n        \"nisi\",\n        \"ut\",\n        \"aliquip\",\n        \"ex\",\n        \"ea\",\n        \"commodo\",\n        \"consequat\",\n    ];\n\n    let mut content = String::with_capacity(length);\n\n    while content.len() < length {\n        let word_idx = rng.random_range(0..words.len());\n        if !content.is_empty() {\n            content.push(' ');\n        }\n        if rng.random_ratio(1, 10) {\n            content.push('.');\n            let word = words[word_idx];\n            let mut chars = word.chars();\n            if let Some(first_char) = chars.next() {\n                content.push_str(&first_char.to_uppercase().to_string());\n                content.push_str(chars.as_str());\n            }\n        } else {\n            content.push_str(words[word_idx]);\n        }\n    }\n\n    if !content.ends_with('.') {\n        content.push('.');\n    }\n\n    content\n}\n\nfn generate_random_name(length: usize) -> String {\n    let mut rng = rng();\n    (0..length)\n        .map(|_| rng.sample(Alphanumeric) as char)\n        .collect()\n}\n\nimpl WebDavTest {\n    pub async fn fetch_email(&self, account_id: u32, document_id: u32) -> Vec<u8> {\n        let metadata_ = self\n            .server\n            .store()\n            .get_value::<Archive<AlignedBytes>>(ValueKey::property(\n                account_id,\n                Collection::Email,\n                document_id,\n                EmailField::Metadata,\n            ))\n            .await\n            .unwrap()\n            .unwrap();\n        self.server\n            .blob_store()\n            .get_blob(\n                metadata_\n                    .unarchive::<MessageMetadata>()\n                    .unwrap()\n                    .blob_hash\n                    .0\n                    .as_slice(),\n                0..usize::MAX,\n            )\n            .await\n            .unwrap()\n            .unwrap()\n    }\n}\n\nconst SERVER: &str = r#\"\n[server]\nhostname = \"webdav.example.org\"\n\n[spam-filter]\nenable = false\n\n[http]\nurl = \"'https://127.0.0.1:8899'\"\n\n[server.listener.webdav]\nbind = [\"127.0.0.1:8899\"]\nprotocol = \"http\"\nmax-connections = 81920\ntls.implicit = true\n\n[server.socket]\nreuse-addr = true\n\n[server.tls]\nenable = true\nimplicit = false\ncertificate = \"default\"\n\n[session.ehlo]\nreject-non-fqdn = false\n\n[session.rcpt]\nrelay = [ { if = \"!is_empty(authenticated_as)\", then = true }, \n        { else = false } ]\n\n[session.rcpt.errors]\ntotal = 5\nwait = \"1ms\"\n\n[resolver]\ntype = \"system\"\n\n[queue.strategy]\nroute = [ { if = \"rcpt_domain == 'example.com'\", then = \"'local'\" }, \n            { else = \"'mx'\" } ]\n\n[session.data.add-headers]\ndelivered-to = false\n\n[session.extensions]\nfuture-release = [ { if = \"!is_empty(authenticated_as)\", then = \"99999999d\"},\n                { else = false } ]\n\n[certificate.default]\ncert = \"%{file:{CERT}}%\"\nprivate-key = \"%{file:{PK}}%\"\n\n[jmap.protocol]\nset.max-objects = 100000\n\n[jmap.protocol.request]\nmax-concurrent = 8\n\n[jmap.protocol.upload]\nmax-size = 5000000\nmax-concurrent = 4\nttl = \"1m\"\n\n[jmap.protocol.upload.quota]\nfiles = 3\nsize = 50000\n\n[jmap.rate-limit]\naccount = \"1000/1m\"\nauthentication = \"100/2s\"\nanonymous = \"100/1m\"\n\n[calendar.alarms]\nminimum-interval = \"1s\"\n\n[calendar.scheduling.inbound]\nauto-add = true\n\n[dav.collection]\nassisted-discovery = {ASSISTED_DISCOVERY}\n\n[sharing]\nallow-directory-query = true\n\n[store.\"auth\"]\ntype = \"sqlite\"\npath = \"{TMP}/auth.db\"\n\n[store.\"auth\".query]\nname = \"SELECT name, type, secret, description, quota FROM accounts WHERE name = ? AND active = true\"\nmembers = \"SELECT member_of FROM group_members WHERE name = ?\"\nrecipients = \"SELECT name FROM emails WHERE address = ?\"\nemails = \"SELECT address FROM emails WHERE name = ? AND type != 'list' ORDER BY type DESC, address ASC\"\nverify = \"SELECT address FROM emails WHERE address LIKE '%' || ? || '%' AND type = 'primary' ORDER BY address LIMIT 5\"\nexpand = \"SELECT p.address FROM emails AS p JOIN emails AS l ON p.name = l.name WHERE p.type = 'primary' AND l.address = ? AND l.type = 'list' ORDER BY p.address LIMIT 50\"\ndomains = \"SELECT 1 FROM emails WHERE address LIKE '%@' || ? LIMIT 1\"\n\n[oauth]\nkey = \"parerga_und_paralipomena\"\n\n[oauth.auth]\nmax-attempts = 1\n\n[oauth.expiry]\nuser-code = \"1s\"\ntoken = \"1s\"\nrefresh-token = \"3s\"\nrefresh-token-renew = \"2s\"\n\n[tracer.console]\ntype = \"console\"\nlevel = \"{LEVEL}\"\nmultiline = false\nansi = true\ndisabled-events = [\"network.*\"]\n \n\"#;\n"
  },
  {
    "path": "tests/src/webdav/multiget.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::WebDavTest;\nuse crate::webdav::{DummyWebDavClient, GenerateTestDavResource, prop::DavMultiStatus};\nuse dav_proto::schema::property::{CalDavProperty, CardDavProperty, DavProperty, WebDavProperty};\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\n\nconst MULTIGET_CALENDAR: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:calendar-multiget xmlns:D=\"DAV:\" xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\n     <D:prop>\n       <D:getetag/>\n       <C:calendar-data/>\n     </D:prop>\n     $PATH\n   </C:calendar-multiget>\n\"#;\nconst MULTIGET_ADDRESSBOOK: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <C:addressbook-multiget xmlns:D=\"DAV:\"\n                        xmlns:C=\"urn:ietf:params:xml:ns:carddav\">\n     <D:prop>\n       <D:getetag/>\n       <C:address-data/>\n     </D:prop>\n     $PATH\n   </C:addressbook-multiget>\n\"#;\n\npub async fn test(test: &WebDavTest) {\n    let client = test.client(\"john\");\n\n    for resource_type in [DavResourceName::Cal, DavResourceName::Card] {\n        println!(\n            \"Running REPORT multiget tests ({})...\",\n            resource_type.base_path()\n        );\n\n        let mut paths = Vec::new();\n        for name in [\"file1\", \"file2\"] {\n            let contents = resource_type.generate();\n            let path = format!(\"{}/john/default/{}\", resource_type.base_path(), name);\n            let etag = client\n                .request(\"PUT\", &path, contents.as_str())\n                .await\n                .with_status(StatusCode::CREATED)\n                .etag()\n                .to_string();\n            paths.push((path, etag, contents));\n        }\n\n        if resource_type == DavResourceName::Cal {\n            let path = format!(\"{}/john\", resource_type.base_path());\n            let response = client\n                .multiget_calendar(&path, &[&paths[0].0, &paths[1].0])\n                .await;\n            for (path, etag, contents) in paths {\n                let props = response.properties(&path);\n                props\n                    .get(DavProperty::WebDav(WebDavProperty::GetETag))\n                    .with_values([etag.as_str()]);\n                props\n                    .get(DavProperty::CalDav(CalDavProperty::CalendarData(\n                        Default::default(),\n                    )))\n                    .with_values([contents.as_str()]);\n            }\n        } else {\n            let path = format!(\"{}/john\", resource_type.base_path());\n            let response = client\n                .multiget_addressbook(&path, &[&paths[0].0, &paths[1].0])\n                .await;\n            for (path, etag, contents) in paths {\n                let props = response.properties(&path);\n                props\n                    .get(DavProperty::WebDav(WebDavProperty::GetETag))\n                    .with_values([etag.as_str()]);\n                props\n                    .get(DavProperty::CardDav(CardDavProperty::AddressData(\n                        Default::default(),\n                    )))\n                    .with_values([contents.as_str()]);\n            }\n        }\n    }\n\n    client.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\nimpl DummyWebDavClient {\n    pub async fn multiget_calendar(&self, path: &str, uris: &[&str]) -> DavMultiStatus {\n        let mut paths = String::new();\n        for uri in uris {\n            paths.push_str(&format!(\"<D:href>{}</D:href>\", uri));\n        }\n\n        self.request(\"REPORT\", path, &MULTIGET_CALENDAR.replace(\"$PATH\", &paths))\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .into_propfind_response(None)\n    }\n\n    pub async fn multiget_addressbook(&self, path: &str, uris: &[&str]) -> DavMultiStatus {\n        let mut paths = String::new();\n        for uri in uris {\n            paths.push_str(&format!(\"<D:href>{}</D:href>\", uri));\n        }\n\n        self.request(\n            \"REPORT\",\n            path,\n            &MULTIGET_ADDRESSBOOK.replace(\"$PATH\", &paths),\n        )\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .into_propfind_response(None)\n    }\n}\n"
  },
  {
    "path": "tests/src/webdav/principals.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::WebDavTest;\nuse crate::{TEST_USERS, webdav::prop::ALL_DAV_PROPERTIES};\nuse dav_proto::schema::property::{DavProperty, PrincipalProperty, WebDavProperty};\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\n\npub async fn test(test: &WebDavTest, assisted_discovery: bool) {\n    println!(\"Running principals tests...\");\n    let client = test.client(\"jane\");\n    let principal_path = format!(\"D:href:{}/\", DavResourceName::Principal.base_path());\n    let jane_principal_path = format!(\"D:href:{}/jane/\", DavResourceName::Principal.base_path());\n\n    let path_support_card = format!(\"D:href:{}/support/\", DavResourceName::Card.base_path());\n    let path_support_cal = format!(\"D:href:{}/support/\", DavResourceName::Cal.base_path());\n\n    // Test 1: PROPFIND on /dav/pal should return all principals\n    let response = client\n        .propfind(\n            DavResourceName::Principal.collection_path(),\n            ALL_DAV_PROPERTIES,\n        )\n        .await;\n    for (account, _, name, email) in TEST_USERS {\n        let props = response.properties(&format!(\n            \"{}/{}/\",\n            DavResourceName::Principal.base_path(),\n            account\n        ));\n        let path_pal = format!(\n            \"D:href:{}/{}/\",\n            DavResourceName::Principal.base_path(),\n            account\n        );\n        let path_card = format!(\"D:href:{}/{}/\", DavResourceName::Card.base_path(), account);\n        let path_cal = format!(\"D:href:{}/{}/\", DavResourceName::Cal.base_path(), account);\n        props\n            .get(DavProperty::WebDav(WebDavProperty::DisplayName))\n            .with_values([*name])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal))\n            .with_values([jane_principal_path.as_str()])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::Principal(PrincipalProperty::PrincipalURL))\n            .with_values([path_pal.as_str()])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::WebDav(WebDavProperty::Owner))\n            .with_values([path_pal.as_str()])\n            .with_status(StatusCode::OK);\n        if *account == \"jane\" && !assisted_discovery {\n            props\n                .get(DavProperty::Principal(PrincipalProperty::CalendarHomeSet))\n                .with_values([path_cal.as_str(), path_support_cal.as_str()])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::Principal(\n                    PrincipalProperty::AddressbookHomeSet,\n                ))\n                .with_values([path_card.as_str(), path_support_card.as_str()])\n                .with_status(StatusCode::OK);\n        } else {\n            props\n                .get(DavProperty::Principal(PrincipalProperty::CalendarHomeSet))\n                .with_values([path_cal.as_str()])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::Principal(\n                    PrincipalProperty::AddressbookHomeSet,\n                ))\n                .with_values([path_card.as_str()])\n                .with_status(StatusCode::OK);\n        }\n        props\n            .get(DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet))\n            .with_values([principal_path.as_str()])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::WebDav(WebDavProperty::SupportedReportSet))\n            .with_values([\n                \"D:supported-report.D:report.D:principal-property-search\",\n                \"D:supported-report.D:report.D:principal-search-property-set\",\n                \"D:supported-report.D:report.D:principal-match\",\n            ])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::WebDav(WebDavProperty::ResourceType))\n            .with_values([\"D:principal\", \"D:collection\"])\n            .with_status(StatusCode::OK);\n\n        // Scheduling properties\n        props\n            .get(DavProperty::Principal(\n                PrincipalProperty::CalendarUserAddressSet,\n            ))\n            .with_values([format!(\"D:href:mailto:{email}\",).as_str()])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::Principal(PrincipalProperty::CalendarUserType))\n            .with_values([\"INDIVIDUAL\"])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::Principal(PrincipalProperty::ScheduleInboxURL))\n            .with_values([format!(\n                \"D:href:{}/{account}/inbox/\",\n                DavResourceName::Scheduling.base_path()\n            )\n            .as_str()])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::Principal(PrincipalProperty::ScheduleOutboxURL))\n            .with_values([format!(\n                \"D:href:{}/{account}/outbox/\",\n                DavResourceName::Scheduling.base_path()\n            )\n            .as_str()])\n            .with_status(StatusCode::OK);\n    }\n\n    // Test 2: PROPFIND on /dav/[resource] should return user and shared resources\n    for resource_type in [\n        DavResourceName::File,\n        DavResourceName::Cal,\n        DavResourceName::Card,\n    ] {\n        let supported_reports = match resource_type {\n            DavResourceName::File => [\n                \"D:supported-report.D:report.D:sync-collection\",\n                \"D:supported-report.D:report.D:acl-principal-prop-set\",\n                \"D:supported-report.D:report.D:principal-match\",\n            ]\n            .as_slice(),\n            DavResourceName::Cal => [\n                \"D:supported-report.D:report.A:free-busy-query\",\n                \"D:supported-report.D:report.A:calendar-query\",\n                \"D:supported-report.D:report.D:expand-property\",\n                \"D:supported-report.D:report.D:sync-collection\",\n                \"D:supported-report.D:report.D:acl-principal-prop-set\",\n                \"D:supported-report.D:report.D:principal-match\",\n                \"D:supported-report.D:report.A:calendar-multiget\",\n            ]\n            .as_slice(),\n            DavResourceName::Card => [\n                \"D:supported-report.D:report.B:addressbook-query\",\n                \"D:supported-report.D:report.D:acl-principal-prop-set\",\n                \"D:supported-report.D:report.D:expand-property\",\n                \"D:supported-report.D:report.B:addressbook-multiget\",\n                \"D:supported-report.D:report.D:principal-match\",\n                \"D:supported-report.D:report.D:sync-collection\",\n            ]\n            .as_slice(),\n            _ => unreachable!(),\n        };\n        let privilege_set = if resource_type == DavResourceName::Cal {\n            [\n                \"D:privilege.D:read-current-user-privilege-set\",\n                \"D:privilege.D:write-acl\",\n                \"D:privilege.A:read-free-busy\",\n                \"D:privilege.D:read-acl\",\n                \"D:privilege.D:write-properties\",\n                \"D:privilege.D:write\",\n                \"D:privilege.D:write-content\",\n                \"D:privilege.D:unlock\",\n                \"D:privilege.D:all\",\n                \"D:privilege.D:read\",\n                \"D:privilege.D:bind\",\n                \"D:privilege.D:unbind\",\n            ]\n            .as_slice()\n        } else {\n            [\n                \"D:privilege.D:all\",\n                \"D:privilege.D:read\",\n                \"D:privilege.D:write\",\n                \"D:privilege.D:write-properties\",\n                \"D:privilege.D:write-content\",\n                \"D:privilege.D:unlock\",\n                \"D:privilege.D:read-acl\",\n                \"D:privilege.D:read-current-user-privilege-set\",\n                \"D:privilege.D:write-acl\",\n                \"D:privilege.D:bind\",\n                \"D:privilege.D:unbind\",\n            ]\n            .as_slice()\n        };\n\n        let response = client\n            .propfind(resource_type.collection_path(), ALL_DAV_PROPERTIES)\n            .await;\n        let props = response.properties(resource_type.collection_path());\n        props\n            .get(DavProperty::WebDav(WebDavProperty::SupportedReportSet))\n            .with_values(supported_reports.iter().copied())\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::WebDav(WebDavProperty::ResourceType))\n            .with_values([\"D:collection\"])\n            .with_status(StatusCode::OK);\n        props\n            .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal))\n            .with_values([jane_principal_path.as_str()])\n            .with_status(StatusCode::OK);\n        if assisted_discovery {\n            props\n                .get(DavProperty::Principal(PrincipalProperty::CalendarHomeSet))\n                .with_values(\n                    [format!(\"D:href:{}/jane/\", DavResourceName::Cal.base_path()).as_str()],\n                )\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::Principal(\n                    PrincipalProperty::AddressbookHomeSet,\n                ))\n                .with_values([\n                    format!(\"D:href:{}/jane/\", DavResourceName::Card.base_path()).as_str(),\n                ])\n                .with_status(StatusCode::OK);\n        } else {\n            props\n                .get(DavProperty::Principal(PrincipalProperty::CalendarHomeSet))\n                .with_values([\n                    format!(\"D:href:{}/jane/\", DavResourceName::Cal.base_path()).as_str(),\n                    format!(\"D:href:{}/support/\", DavResourceName::Cal.base_path()).as_str(),\n                ])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::Principal(\n                    PrincipalProperty::AddressbookHomeSet,\n                ))\n                .with_values([\n                    format!(\"D:href:{}/jane/\", DavResourceName::Card.base_path()).as_str(),\n                    format!(\"D:href:{}/support/\", DavResourceName::Card.base_path()).as_str(),\n                ])\n                .with_status(StatusCode::OK);\n        }\n\n        for (account, _, name, _) in TEST_USERS\n            .iter()\n            .filter(|(account, _, _, _)| [\"jane\", \"support\"].contains(account))\n        {\n            let path_card = format!(\"D:href:{}/{}/\", DavResourceName::Card.base_path(), account);\n            let path_cal = format!(\"D:href:{}/{}/\", DavResourceName::Cal.base_path(), account);\n            let path_pal = format!(\n                \"D:href:{}/{}/\",\n                DavResourceName::Principal.base_path(),\n                account\n            );\n            let props = response.properties(&format!(\"{}/{account}/\", resource_type.base_path()));\n\n            props\n                .get(DavProperty::WebDav(WebDavProperty::DisplayName))\n                .with_values([*name])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::ResourceType))\n                .with_values([\"D:collection\"])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal))\n                .with_values([jane_principal_path.as_str()])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n                .with_values(privilege_set.iter().copied())\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::SupportedReportSet))\n                .with_values(supported_reports.iter().copied())\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::Principal(PrincipalProperty::PrincipalURL))\n                .with_values([path_pal.as_str()])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet))\n                .with_values([principal_path.as_str()])\n                .with_status(StatusCode::OK);\n            props\n                .get(DavProperty::WebDav(WebDavProperty::Owner))\n                .with_values([path_pal.as_str()])\n                .with_status(StatusCode::OK);\n            if *account == \"jane\" && !assisted_discovery {\n                props\n                    .get(DavProperty::Principal(PrincipalProperty::CalendarHomeSet))\n                    .with_values([path_cal.as_str(), path_support_cal.as_str()])\n                    .with_status(StatusCode::OK);\n                props\n                    .get(DavProperty::Principal(\n                        PrincipalProperty::AddressbookHomeSet,\n                    ))\n                    .with_values([path_card.as_str(), path_support_card.as_str()])\n                    .with_status(StatusCode::OK);\n            } else {\n                props\n                    .get(DavProperty::Principal(PrincipalProperty::CalendarHomeSet))\n                    .with_values([path_cal.as_str()])\n                    .with_status(StatusCode::OK);\n                props\n                    .get(DavProperty::Principal(\n                        PrincipalProperty::AddressbookHomeSet,\n                    ))\n                    .with_values([path_card.as_str()])\n                    .with_status(StatusCode::OK);\n            }\n            props\n                .get(DavProperty::WebDav(WebDavProperty::SyncToken))\n                .with_status(StatusCode::OK)\n                .is_not_empty();\n            props\n                .get(DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes))\n                .with_status(StatusCode::OK)\n                .is_not_empty();\n            props\n                .get(DavProperty::WebDav(WebDavProperty::QuotaUsedBytes))\n                .with_status(StatusCode::OK)\n                .is_not_empty();\n        }\n\n        // Test 3: principal-match-query on resources\n        let response = client\n            .request(\n                \"REPORT\",\n                resource_type.collection_path(),\n                PRINCIPAL_MATCH_QUERY,\n            )\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .into_propfind_response(None);\n        response.with_hrefs([\n            format!(\"{}/jane/\", resource_type.base_path()).as_str(),\n            format!(\"{}/support/\", resource_type.base_path()).as_str(),\n        ]);\n    }\n\n    // Test 4: principal-match-query on principals\n    let response = client\n        .request(\n            \"REPORT\",\n            DavResourceName::Principal.collection_path(),\n            PRINCIPAL_MATCH_QUERY,\n        )\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .into_propfind_response(None);\n    response.with_hrefs([\n        format!(\"{}/jane/\", DavResourceName::Principal.base_path()).as_str(),\n        format!(\"{}/support/\", DavResourceName::Principal.base_path()).as_str(),\n    ]);\n\n    // Test 5: principal-search-property-set REPORT\n    let response = client\n        .request(\n            \"REPORT\",\n            DavResourceName::Principal.collection_path(),\n            PRINCIPAL_SEARCH_PROPERTY_SET_QUERY,\n        )\n        .await\n        .with_status(StatusCode::OK);\n    response\n        .with_value(\n            \"D:principal-search-property-set.D:principal-search-property.D:prop.D:displayname\",\n            \"\",\n        )\n        .with_value(\n            \"D:principal-search-property-set.D:principal-search-property.D:description\",\n            \"Account or Group name\",\n        );\n\n    // Test 6: principal-property-search REPORT\n    let response = client\n        .request(\n            \"REPORT\",\n            DavResourceName::Principal.collection_path(),\n            PRINCIPAL_PROPERTY_SEARCH_QUERY.replace(\"$NAME\", \"doe\"),\n        )\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .into_propfind_response(None);\n    response.with_hrefs([\n        format!(\"{}/jane/\", DavResourceName::Principal.base_path()).as_str(),\n        format!(\"{}/john/\", DavResourceName::Principal.base_path()).as_str(),\n    ]);\n    response\n        .properties(&format!(\"{}/jane/\", DavResourceName::Principal.base_path()))\n        .get(DavProperty::WebDav(WebDavProperty::DisplayName))\n        .with_values([TEST_USERS\n            .iter()\n            .find(|(account, _, _, _)| *account == \"jane\")\n            .unwrap()\n            .2])\n        .with_status(StatusCode::OK);\n    client\n        .request(\n            \"REPORT\",\n            DavResourceName::Principal.collection_path(),\n            PRINCIPAL_PROPERTY_SEARCH_QUERY.replace(\"$NAME\", \"support\"),\n        )\n        .await\n        .with_status(StatusCode::MULTI_STATUS)\n        .into_propfind_response(None)\n        .with_hrefs([format!(\"{}/support/\", DavResourceName::Principal.base_path()).as_str()]);\n\n    client.delete_default_containers().await;\n    client.delete_default_containers_by_account(\"support\").await;\n    test.assert_is_empty().await;\n}\n\nconst PRINCIPAL_MATCH_QUERY: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:principal-match xmlns:D=\"DAV:\">\n<D:principal-property>\n<D:owner/>\n<D:displayname/>\n</D:principal-property>\n</D:principal-match>\"#;\n\nconst PRINCIPAL_SEARCH_PROPERTY_SET_QUERY: &str =\n    r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?><D:principal-search-property-set xmlns:D=\"DAV:\"/>\"#;\n\nconst PRINCIPAL_PROPERTY_SEARCH_QUERY: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n   <D:principal-property-search xmlns:D=\"DAV:\">\n     <D:property-search>\n       <D:prop>\n         <D:displayname/>\n       </D:prop>\n       <D:match>$NAME</D:match>\n     </D:property-search>\n     <D:prop xmlns:B=\"http://www.example.com/ns/\">\n       <D:displayname/>\n     </D:prop>\n</D:principal-property-search>\"#;\n"
  },
  {
    "path": "tests/src/webdav/prop.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{DavResponse, DummyWebDavClient, WebDavTest};\nuse crate::webdav::{GenerateTestDavResource, TEST_ICAL_2, TEST_VTIMEZONE_1};\nuse ahash::{AHashMap, AHashSet};\nuse dav_proto::schema::property::{\n    CalDavProperty, CardDavProperty, DavProperty, PrincipalProperty, WebDavProperty,\n};\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\nuse types::dead_property::DeadElementTag;\n\npub async fn test(test: &WebDavTest, assisted_discovery: bool) {\n    let client = test.client(\"jane\");\n\n    for resource_type in [\n        DavResourceName::File,\n        DavResourceName::Cal,\n        DavResourceName::Card,\n    ] {\n        println!(\n            \"Running PROPFIND/PROPPATCH tests ({})...\",\n            resource_type.base_path()\n        );\n        let user_base_path = format!(\"{}/jane\", resource_type.base_path());\n        let group_base_path = format!(\"{}/support\", resource_type.base_path());\n\n        // Create a new test container and file\n        let test_base_path = format!(\"{user_base_path}/PropFind_Folder/\");\n        let etag_folder = client\n            .mkcol(\"MKCOL\", &test_base_path, [], [])\n            .await\n            .with_status(StatusCode::CREATED)\n            .etag()\n            .to_string();\n        let test_contents = resource_type.generate();\n        let test_path = format!(\"{test_base_path}test_file\");\n        let etag_file = client\n            .request_with_headers(\n                \"PUT\",\n                &test_path,\n                [(\"content-type\", \"text/x-other\")],\n                test_contents.as_str(),\n            )\n            .await\n            .with_status(StatusCode::CREATED)\n            .etag()\n            .to_string();\n\n        // Test 1: PROPFIND Depth 0 on root\n        client\n            .request_with_headers(\"PROPFIND\", resource_type.base_path(), [(\"depth\", \"0\")], \"\")\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .with_hrefs([resource_type.collection_path()]);\n\n        // Test 2: PROPFIND Depth 0 on user base path\n        client\n            .request_with_headers(\"PROPFIND\", &user_base_path, [(\"depth\", \"0\")], \"\")\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .with_hrefs([format!(\"{user_base_path}/\").as_str()]);\n\n        // Test 3: PROPFIND Depth 1 on root\n        client\n            .request_with_headers(\"PROPFIND\", resource_type.base_path(), [(\"depth\", \"1\")], \"\")\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .with_hrefs([\n                resource_type.collection_path(),\n                format!(\"{user_base_path}/\").as_str(),\n                format!(\"{group_base_path}/\").as_str(),\n            ]);\n\n        // Test 4: Infinity depth is not allowed\n        for path in [resource_type.base_path(), user_base_path.as_str()] {\n            client\n                .request_with_headers(\"PROPFIND\", path, [(\"depth\", \"infinity\")], \"\")\n                .await\n                .with_status(StatusCode::FORBIDDEN);\n        }\n\n        // Test 5: PROPFIND Depth 1 on user base path\n        client\n            .request_with_headers(\"PROPFIND\", &user_base_path, [(\"depth\", \"1\")], \"\")\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .with_hrefs(\n                [\n                    format!(\"{group_base_path}/default/\").as_str(),\n                    format!(\"{user_base_path}/default/\").as_str(),\n                    format!(\"{user_base_path}/\").as_str(),\n                    &test_base_path,\n                ]\n                .into_iter()\n                .skip(if resource_type == DavResourceName::File {\n                    2\n                } else if !assisted_discovery {\n                    1\n                } else {\n                    0\n                }),\n            );\n\n        // Test 6: PROPFIND Depth 1 on created collection\n        client\n            .request_with_headers(\"PROPFIND\", &test_base_path, [(\"depth\", \"1\")], \"\")\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .with_hrefs([test_base_path.as_str(), test_path.as_str()]);\n\n        // Test 7: Infinity depth is not allowed on file containers\n        client\n            .request_with_headers(\"PROPFIND\", &test_base_path, [(\"depth\", \"infinity\")], \"\")\n            .await\n            .with_status(if resource_type == DavResourceName::File {\n                StatusCode::FORBIDDEN\n            } else {\n                StatusCode::MULTI_STATUS\n            });\n\n        // Test 8 PROPFIND with depth-no-root\n        client\n            .request_with_headers(\n                \"PROPFIND\",\n                &user_base_path,\n                [(\"depth\", \"1\"), (\"prefer\", \"depth-noroot\")],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .with_hrefs(\n                [\n                    format!(\"{group_base_path}/default/\").as_str(),\n                    format!(\"{user_base_path}/default/\").as_str(),\n                    &test_base_path,\n                ]\n                .into_iter()\n                .skip(if resource_type == DavResourceName::File {\n                    2\n                } else if !assisted_discovery {\n                    1\n                } else {\n                    0\n                }),\n            );\n        client\n            .request_with_headers(\n                \"PROPFIND\",\n                &test_base_path,\n                [(\"depth\", \"1\"), (\"prefer\", \"depth-noroot\")],\n                \"\",\n            )\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .with_hrefs([test_path.as_str()]);\n\n        // Test 8 PROPFIND with prefer return=minimal\n        let response = client\n            .propfind_with_headers(&test_base_path, ALL_DAV_PROPERTIES, [])\n            .await;\n        response\n            .properties(&test_base_path)\n            .is_defined(DavProperty::WebDav(WebDavProperty::GetETag))\n            .is_defined(DavProperty::Principal(PrincipalProperty::GroupMembership));\n        let response = client\n            .propfind_with_headers(\n                &test_base_path,\n                ALL_DAV_PROPERTIES,\n                [(\"prefer\", \"return=minimal\")],\n            )\n            .await;\n        response\n            .properties(&test_base_path)\n            .is_defined(DavProperty::WebDav(WebDavProperty::GetETag))\n            .is_undefined(DavProperty::Principal(PrincipalProperty::GroupMembership));\n\n        // Test 9: Retrieve all static properties\n        for (path, etag, is_file) in [\n            (&test_base_path, &etag_folder, false),\n            (&test_path, &etag_file, true),\n        ] {\n            let response = client.propfind(path, ALL_DAV_PROPERTIES).await;\n            let properties = response.properties(path);\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::CreationDate))\n                .is_not_empty();\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::GetLastModified))\n                .is_not_empty();\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::SyncToken))\n                .is_not_empty();\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::GetETag))\n                .with_values([etag.as_str()]);\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::SupportedLock))\n                .with_values([\n                    \"D:lockentry.D:lockscope.D:exclusive\",\n                    \"D:lockentry.D:locktype.D:write\",\n                    \"D:lockentry.D:lockscope.D:shared\",\n                    \"D:lockentry.D:locktype.D:write\",\n                ]);\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal))\n                .with_values([\n                    format!(\"D:href:{}/jane/\", DavResourceName::Principal.base_path()).as_str(),\n                ]);\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::Owner))\n                .with_values([\n                    format!(\"D:href:{}/jane/\", DavResourceName::Principal.base_path()).as_str(),\n                ]);\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet))\n                .is_not_empty();\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::AclRestrictions))\n                .with_values([\"D:grant-only\", \"D:no-invert\"]);\n            properties\n                .get(DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet))\n                .with_values([\n                    format!(\"D:href:{}\", DavResourceName::Principal.collection_path()).as_str(),\n                ]);\n\n            if is_file {\n                // File specific properties\n                properties\n                    .get(DavProperty::WebDav(WebDavProperty::GetContentType))\n                    .with_values([match resource_type {\n                        DavResourceName::File => \"text/x-other\",\n                        DavResourceName::Cal => \"text/calendar\",\n                        DavResourceName::Card => \"text/vcard\",\n                        _ => unreachable!(),\n                    }]);\n                properties\n                    .get(DavProperty::WebDav(WebDavProperty::GetContentLength))\n                    .with_values([test_contents.len().to_string().as_str()]);\n            } else {\n                // Collection specific properties\n                properties\n                    .get(DavProperty::WebDav(WebDavProperty::GetCTag))\n                    .is_not_empty();\n                properties\n                    .get(DavProperty::WebDav(WebDavProperty::ResourceType))\n                    .with_values(match resource_type {\n                        DavResourceName::File => [\"D:collection\"].as_slice().iter().copied(),\n                        DavResourceName::Cal => {\n                            [\"D:collection\", \"A:calendar\"].as_slice().iter().copied()\n                        }\n                        DavResourceName::Card => {\n                            [\"D:collection\", \"B:addressbook\"].as_slice().iter().copied()\n                        }\n                        _ => unreachable!(),\n                    });\n                let used_bytes: u64 = properties\n                    .get(DavProperty::WebDav(WebDavProperty::QuotaUsedBytes))\n                    .value()\n                    .parse()\n                    .unwrap();\n                let available_bytes: u64 = properties\n                    .get(DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes))\n                    .value()\n                    .parse()\n                    .unwrap();\n                assert!(used_bytes > 0);\n                assert!(available_bytes > 0);\n                properties\n                    .get(DavProperty::WebDav(WebDavProperty::SupportedReportSet))\n                    .with_values(match resource_type {\n                        DavResourceName::File => [\n                            \"D:supported-report.D:report.D:sync-collection\",\n                            \"D:supported-report.D:report.D:acl-principal-prop-set\",\n                            \"D:supported-report.D:report.D:principal-match\",\n                        ]\n                        .as_slice()\n                        .iter()\n                        .copied(),\n                        DavResourceName::Cal => [\n                            \"D:supported-report.D:report.A:calendar-query\",\n                            \"D:supported-report.D:report.D:sync-collection\",\n                            \"D:supported-report.D:report.D:acl-principal-prop-set\",\n                            \"D:supported-report.D:report.D:expand-property\",\n                            \"D:supported-report.D:report.A:free-busy-query\",\n                            \"D:supported-report.D:report.A:calendar-multiget\",\n                            \"D:supported-report.D:report.D:principal-match\",\n                        ]\n                        .as_slice()\n                        .iter()\n                        .copied(),\n                        DavResourceName::Card => [\n                            \"D:supported-report.D:report.B:addressbook-multiget\",\n                            \"D:supported-report.D:report.D:sync-collection\",\n                            \"D:supported-report.D:report.D:acl-principal-prop-set\",\n                            \"D:supported-report.D:report.D:principal-match\",\n                            \"D:supported-report.D:report.B:addressbook-query\",\n                            \"D:supported-report.D:report.D:expand-property\",\n                        ]\n                        .as_slice()\n                        .iter()\n                        .copied(),\n                        _ => unreachable!(),\n                    });\n\n                if resource_type == DavResourceName::Cal {\n                    properties\n                        .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n                        .with_values([\n                            \"D:privilege.D:all\",\n                            \"D:privilege.D:read\",\n                            \"D:privilege.D:write\",\n                            \"D:privilege.D:write-properties\",\n                            \"D:privilege.D:write-content\",\n                            \"D:privilege.D:unlock\",\n                            \"D:privilege.D:read-acl\",\n                            \"D:privilege.D:read-current-user-privilege-set\",\n                            \"D:privilege.D:write-acl\",\n                            \"D:privilege.D:bind\",\n                            \"D:privilege.D:unbind\",\n                            \"D:privilege.A:read-free-busy\",\n                        ]);\n                    properties\n                        .get(DavProperty::CalDav(\n                            CalDavProperty::SupportedCalendarComponentSet,\n                        ))\n                        .with_values([\n                            \"A:comp.[name]:VAVAILABILITY\",\n                            \"A:comp.[name]:AVAILABLE\",\n                            \"A:comp.[name]:VRESOURCE\",\n                            \"A:comp.[name]:VTODO\",\n                            \"A:comp.[name]:DAYLIGHT\",\n                            \"A:comp.[name]:STANDARD\",\n                            \"A:comp.[name]:VLOCATION\",\n                            \"A:comp.[name]:VTIMEZONE\",\n                            \"A:comp.[name]:VFREEBUSY\",\n                            \"A:comp.[name]:VEVENT\",\n                            \"A:comp.[name]:VJOURNAL\",\n                            \"A:comp.[name]:PARTICIPANT\",\n                            \"A:comp.[name]:VALARM\",\n                        ]);\n                    properties\n                        .get(DavProperty::CalDav(CalDavProperty::SupportedCalendarData))\n                        .with_values([\n                            concat!(\"A:calendar-data-type.\", \"[content-type]:text/calendar\"),\n                            \"A:calendar-data-type.[version]:2.0\",\n                            \"A:calendar-data-type.[version]:1.0\",\n                        ]);\n                    properties\n                        .get(DavProperty::CalDav(CalDavProperty::SupportedCollationSet))\n                        .with_values([\n                            \"A:supported-collation:i;unicode-casemap\",\n                            \"A:supported-collation:i;ascii-casemap\",\n                        ]);\n                    properties\n                        .get(DavProperty::CalDav(CalDavProperty::MinDateTime))\n                        .with_values([\"0001-01-01T00:00:00Z\"]);\n                    properties\n                        .get(DavProperty::CalDav(CalDavProperty::MaxDateTime))\n                        .with_values([\"9999-12-31T23:59:59Z\"]);\n                    for (key, value) in [\n                        (\n                            DavProperty::CalDav(CalDavProperty::MaxResourceSize),\n                            test.server.core.groupware.max_ical_size,\n                        ),\n                        (\n                            DavProperty::CalDav(CalDavProperty::MaxInstances),\n                            test.server.core.groupware.max_ical_instances,\n                        ),\n                        (\n                            DavProperty::CalDav(CalDavProperty::MaxAttendeesPerInstance),\n                            test.server.core.groupware.max_ical_attendees_per_instance,\n                        ),\n                    ] {\n                        properties\n                            .get(key)\n                            .with_values([value.to_string().as_str()]);\n                    }\n                } else {\n                    if resource_type == DavResourceName::Card {\n                        properties\n                            .get(DavProperty::CardDav(CardDavProperty::SupportedAddressData))\n                            .with_values([\n                                concat!(\"B:address-data-type.\", \"[content-type]:text/vcard\"),\n                                \"B:address-data-type.[version]:3.0\",\n                                \"B:address-data-type.[version]:4.0\",\n                                \"B:address-data-type.[version]:2.1\",\n                            ]);\n                        properties\n                            .get(DavProperty::CardDav(CardDavProperty::SupportedCollationSet))\n                            .with_values([\n                                \"B:supported-collation:i;unicode-casemap\",\n                                \"B:supported-collation:i;ascii-casemap\",\n                            ]);\n                        properties\n                            .get(DavProperty::CardDav(CardDavProperty::MaxResourceSize))\n                            .with_values([test\n                                .server\n                                .core\n                                .groupware\n                                .max_vcard_size\n                                .to_string()\n                                .as_str()]);\n                    }\n\n                    properties\n                        .get(DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet))\n                        .with_values([\n                            \"D:privilege.D:all\",\n                            \"D:privilege.D:read\",\n                            \"D:privilege.D:write\",\n                            \"D:privilege.D:write-properties\",\n                            \"D:privilege.D:write-content\",\n                            \"D:privilege.D:unlock\",\n                            \"D:privilege.D:read-acl\",\n                            \"D:privilege.D:read-current-user-privilege-set\",\n                            \"D:privilege.D:write-acl\",\n                            \"D:privilege.D:bind\",\n                            \"D:privilege.D:unbind\",\n                        ]);\n                }\n            }\n        }\n\n        // Test 10: expand-property report\n        for path in [&test_base_path, &test_path] {\n            let response = client\n                .request(\"REPORT\", path, EXPAND_REPORT_QUERY)\n                .await\n                .with_status(StatusCode::MULTI_STATUS)\n                .into_propfind_response(None);\n            let properties = response.properties(path);\n            for prop in [\n                DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n                DavProperty::WebDav(WebDavProperty::Owner),\n            ] {\n                properties.get(prop).with_some_values([\n                    format!(\n                        \"D:response.D:href:{}/jane/\",\n                        DavResourceName::Principal.base_path(),\n                    )\n                    .as_str(),\n                    \"D:response.D:propstat.D:prop.D:displayname:Jane Doe-Smith\",\n                ]);\n            }\n        }\n\n        for (path, etag, is_file) in [\n            (&test_base_path, &etag_folder, false),\n            (&test_path, &etag_file, true),\n        ] {\n            // Test 11: PROPPATCH should fail when a precondition fails\n            client\n                .proppatch(\n                    path,\n                    [(\n                        DavProperty::WebDav(WebDavProperty::DisplayName),\n                        \"Magnific name\",\n                    )],\n                    [],\n                    [(\"if\", format!(\"(Not [{etag}])\").as_str())],\n                )\n                .await\n                .with_status(StatusCode::PRECONDITION_FAILED);\n            client\n                .proppatch(\n                    path,\n                    [(\n                        DavProperty::WebDav(WebDavProperty::DisplayName),\n                        \"Magnific name - second try\",\n                    )],\n                    [],\n                    [(\"if\", format!(\"([{etag}])\").as_str())],\n                )\n                .await\n                .with_status(StatusCode::MULTI_STATUS);\n            client\n                .propfind(path, [DavProperty::WebDav(WebDavProperty::GetETag)])\n                .await\n                .properties(path)\n                .get(DavProperty::WebDav(WebDavProperty::GetETag))\n                .with_status(StatusCode::OK)\n                .without_values([etag.as_str()]);\n\n            // Test 12: PROPPATCH set on DAV properties\n            client\n                .patch_and_check(\n                    path,\n                    [\n                        (\n                            DavProperty::WebDav(WebDavProperty::DisplayName),\n                            \"New display name\",\n                        ),\n                        (\n                            DavProperty::WebDav(WebDavProperty::CreationDate),\n                            \"2000-01-01T00:00:00Z\",\n                        ),\n                        (\n                            DavProperty::DeadProperty(DeadElementTag::new(\n                                \"my-dead-element\".to_string(),\n                                Some(\"xmlns=\\\"http://example.com/ns/\\\" prop=\\\"abc\\\"\".to_string()),\n                            )),\n                            \"this is a dead but exciting element\",\n                        ),\n                    ],\n                )\n                .await;\n            client\n                .patch_and_check(\n                    path,\n                    [(\n                        DavProperty::DeadProperty(DeadElementTag::new(\n                            \"my-dead-element\".to_string(),\n                            Some(\"xmlns=\\\"http://example.com/ns/\\\" prop=\\\"xyz\\\"\".to_string()),\n                        )),\n                        \"this is a modified dead but exciting element\",\n                    )],\n                )\n                .await;\n\n            // Test 13: PROPPATCH remove on DAV properties\n            let mut props = vec![\n                (\n                    DavProperty::DeadProperty(DeadElementTag::new(\n                        \"my-dead-element\".to_string(),\n                        Some(\"xmlns=\\\"http://example.com/ns/\\\"\".to_string()),\n                    )),\n                    \"\",\n                ),\n                (DavProperty::WebDav(WebDavProperty::DisplayName), \"\"),\n            ];\n            if !is_file {\n                // DisplayName can't be removed from calendar/contact collections\n                props.pop();\n            }\n            client.patch_and_check(path, props).await;\n\n            match resource_type {\n                DavResourceName::File if is_file => {\n                    // Test 14: Change a file's content-type\n                    client\n                        .patch_and_check(\n                            path,\n                            [(\n                                DavProperty::WebDav(WebDavProperty::GetContentType),\n                                \"text/x-yadda-yadda\",\n                            )],\n                        )\n                        .await;\n                }\n                DavResourceName::Cal if !is_file => {\n                    // Test 15: Change a calendar's properties\n                    client\n                        .patch_and_check(\n                            path,\n                            [\n                                (\n                                    DavProperty::CalDav(CalDavProperty::CalendarDescription),\n                                    \"New calendar description\",\n                                ),\n                                (\n                                    DavProperty::CalDav(CalDavProperty::TimezoneId),\n                                    \"Europe/Ljubljana\",\n                                ),\n                            ],\n                        )\n                        .await;\n                    client\n                        .patch_and_check(\n                            path,\n                            [\n                                (DavProperty::CalDav(CalDavProperty::CalendarDescription), \"\"),\n                                (DavProperty::CalDav(CalDavProperty::TimezoneId), \"\"),\n                            ],\n                        )\n                        .await;\n                    client\n                        .patch_and_check(\n                            path,\n                            [(\n                                DavProperty::CalDav(CalDavProperty::CalendarTimezone),\n                                TEST_VTIMEZONE_1.replace('\\n', \"\\r\\n\").as_str(),\n                            )],\n                        )\n                        .await;\n                }\n                DavResourceName::Card if !is_file => {\n                    // Test 16: Change an addressbook's properties\n                    client\n                        .patch_and_check(\n                            path,\n                            [(\n                                DavProperty::CardDav(CardDavProperty::AddressbookDescription),\n                                \"New calendar description\",\n                            )],\n                        )\n                        .await;\n                    client\n                        .patch_and_check(\n                            path,\n                            [(\n                                DavProperty::CardDav(CardDavProperty::AddressbookDescription),\n                                \"\",\n                            )],\n                        )\n                        .await;\n                }\n                _ => (),\n            }\n\n            // Test 17: PROPPATCH should fail on large properties\n            let mut chunky_props = vec![\n                DavProperty::WebDav(WebDavProperty::DisplayName),\n                DavProperty::DeadProperty(DeadElementTag::new(\n                    \"my-chunky-dead-element\".to_string(),\n                    Some(\"xmlns=\\\"http://example.com/ns/\\\"\".to_string()),\n                )),\n            ];\n            if !is_file {\n                if resource_type == DavResourceName::Cal {\n                    chunky_props.push(DavProperty::CalDav(CalDavProperty::CalendarDescription));\n                } else if resource_type == DavResourceName::Card {\n                    chunky_props.push(DavProperty::CardDav(\n                        CardDavProperty::AddressbookDescription,\n                    ));\n                }\n            }\n            let chunky_live_contents = (0..=(test.server.core.groupware.live_property_size + 1))\n                .map(|_| \"a\")\n                .collect::<String>();\n            let chunky_dead_contents =\n                (0..=(test.server.core.groupware.dead_property_size.unwrap() + 1))\n                    .map(|_| \"a\")\n                    .collect::<String>();\n            let response = client\n                .proppatch(\n                    path,\n                    chunky_props.iter().map(|prop| {\n                        (\n                            prop.clone(),\n                            if matches!(prop, DavProperty::DeadProperty(_)) {\n                                &chunky_dead_contents\n                            } else {\n                                &chunky_live_contents\n                            }\n                            .as_str(),\n                        )\n                    }),\n                    [],\n                    [],\n                )\n                .await\n                .into_propfind_response(None);\n            let props = response.properties(path);\n            for prop in chunky_props {\n                props\n                    .get(prop)\n                    .with_status(StatusCode::INSUFFICIENT_STORAGE)\n                    .with_description(\"Property value is too long\");\n            }\n\n            // Test 18: PROPPATCH should fail on invalid calendar property values\n            if !is_file && resource_type == DavResourceName::Cal {\n                let response = client\n                    .proppatch(\n                        path,\n                        [\n                            (\n                                DavProperty::CalDav(CalDavProperty::TimezoneId),\n                                \"unknown/zone\",\n                            ),\n                            (\n                                DavProperty::CalDav(CalDavProperty::CalendarTimezone),\n                                TEST_ICAL_2,\n                            ),\n                        ],\n                        [],\n                        [],\n                    )\n                    .await\n                    .into_propfind_response(None);\n                let props = response.properties(path);\n                props\n                    .get(DavProperty::CalDav(CalDavProperty::TimezoneId))\n                    .with_status(StatusCode::PRECONDITION_FAILED)\n                    .with_description(\"Invalid timezone ID\");\n                props\n                    .get(DavProperty::CalDav(CalDavProperty::CalendarTimezone))\n                    .with_status(StatusCode::PRECONDITION_FAILED)\n                    .with_description(\"Invalid calendar timezone\");\n            }\n        }\n\n        client\n            .request(\"DELETE\", &test_base_path, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n    }\n\n    client.delete_default_containers().await;\n    client.delete_default_containers_by_account(\"support\").await;\n    test.assert_is_empty().await;\n}\n\n#[derive(Debug)]\npub struct DavMultiStatus {\n    pub response: DavResponse,\n    pub hrefs: AHashMap<String, DavProperties>,\n}\n\n#[derive(Debug, serde::Serialize)]\npub struct DavItem {\n    #[serde(serialize_with = \"serialize_status_code\")]\n    pub status: StatusCode,\n    pub values: AHashMap<String, Vec<String>>,\n    pub error: Vec<String>,\n    pub description: Option<String>,\n}\n\n#[derive(Debug, serde::Serialize)]\npub struct DavProperties {\n    #[serde(skip)]\n    status: StatusCode,\n    props: Vec<DavItem>,\n}\n\nimpl DavMultiStatus {\n    pub fn properties(&self, href: &str) -> DavPropertyResult<'_> {\n        DavPropertyResult {\n            response: &self.response,\n            properties: self.hrefs.get(href).unwrap_or_else(|| {\n                self.response.dump_response();\n                panic!(\n                    \"No properties found for href: {href} in {}\",\n                    serde_json::to_string_pretty(&self.hrefs).unwrap()\n                )\n            }),\n        }\n    }\n\n    pub fn with_hrefs<'x>(&self, expect_hrefs: impl IntoIterator<Item = &'x str>) -> &Self {\n        let expect_hrefs: AHashSet<_> = expect_hrefs.into_iter().collect();\n        let hrefs: AHashSet<_> = self.hrefs.keys().map(|s| s.as_str()).collect();\n        if hrefs != expect_hrefs {\n            self.response.dump_response();\n            panic!(\"Expected hrefs {expect_hrefs:?}, but got {hrefs:?}\",);\n        }\n        self\n    }\n}\n\npub struct DavPropertyResult<'x> {\n    pub response: &'x DavResponse,\n    pub properties: &'x DavProperties,\n}\n\npub struct DavQueryResult<'x> {\n    pub response: &'x DavResponse,\n    pub prop: &'x DavItem,\n    pub values: &'x [String],\n}\n\nimpl DavPropertyResult<'_> {\n    pub fn get(&self, name: impl AsRef<str>) -> DavQueryResult<'_> {\n        let name = name.as_ref();\n        self.properties\n            .props\n            .iter()\n            .find_map(|prop| {\n                prop.values.get(name).map(|values| DavQueryResult {\n                    response: self.response,\n                    prop,\n                    values,\n                })\n            })\n            .unwrap_or_else(|| {\n                self.response.dump_response();\n                panic!(\n                    \"No property found for name: {name} in {}\",\n                    serde_json::to_string_pretty(&self.properties.props).unwrap()\n                )\n            })\n    }\n\n    pub fn with_status(&self, status: StatusCode) -> &Self {\n        if self.properties.status != status {\n            self.response.dump_response();\n            panic!(\n                \"Expected status {status}, but got {}\",\n                self.properties.status\n            );\n        }\n        self\n    }\n\n    pub fn is_defined(&self, name: impl AsRef<str>) -> &Self {\n        if self\n            .properties\n            .props\n            .iter()\n            .any(|prop| prop.values.contains_key(name.as_ref()))\n        {\n            self\n        } else {\n            self.response.dump_response();\n            panic!(\"Expected property {} to be defined\", name.as_ref());\n        }\n    }\n\n    pub fn is_undefined(&self, name: impl AsRef<str>) -> &Self {\n        if self\n            .properties\n            .props\n            .iter()\n            .any(|prop| prop.values.contains_key(name.as_ref()))\n        {\n            self.response.dump_response();\n            panic!(\"Expected property {} to be undefined\", name.as_ref());\n        }\n        self\n    }\n\n    pub fn calendar_data(&self) -> DavQueryResult<'_> {\n        self.get(DavProperty::CalDav(CalDavProperty::CalendarData(\n            Default::default(),\n        )))\n    }\n}\n\nimpl<'x> DavQueryResult<'x> {\n    pub fn with_values(&self, expected_values: impl IntoIterator<Item = &'x str>) -> &Self {\n        let expected_values = AHashSet::from_iter(expected_values);\n        let values = self\n            .values\n            .iter()\n            .map(|s| s.as_str())\n            .collect::<AHashSet<_>>();\n\n        if values != expected_values {\n            self.response.dump_response();\n            assert_eq!(values, expected_values,);\n        }\n        self\n    }\n\n    pub fn with_some_values(&self, expected_values: impl IntoIterator<Item = &'x str>) -> &Self {\n        let values = self\n            .values\n            .iter()\n            .map(|s| s.as_str())\n            .collect::<AHashSet<_>>();\n\n        for expected_value in expected_values {\n            if !values.contains(expected_value) {\n                self.response.dump_response();\n                panic!(\"Expected at least one of {expected_value:?} values, but got {values:?}\",);\n            }\n        }\n\n        self\n    }\n\n    pub fn with_any_values(&self, expected_values: impl IntoIterator<Item = &'x str>) -> &Self {\n        let values = self\n            .values\n            .iter()\n            .map(|s| s.as_str())\n            .collect::<AHashSet<_>>();\n        let expected_values = AHashSet::from_iter(expected_values);\n\n        if values.is_disjoint(&expected_values) {\n            self.response.dump_response();\n            panic!(\"Expected at least one of {expected_values:?} values, but got {values:?}\",);\n        }\n\n        self\n    }\n\n    pub fn without_values(&self, expected_values: impl IntoIterator<Item = &'x str>) -> &Self {\n        let expected_values = AHashSet::from_iter(expected_values);\n        let values = self\n            .values\n            .iter()\n            .map(|s| s.as_str())\n            .collect::<AHashSet<_>>();\n\n        if !expected_values.is_disjoint(&values) {\n            self.response.dump_response();\n            panic!(\"Expected no {expected_values:?} values, but got {values:?}\",);\n        }\n        self\n    }\n\n    pub fn is_not_empty(&self) -> &Self {\n        if self.values.is_empty() || self.values.iter().all(|s| s.is_empty()) {\n            self.response.dump_response();\n            panic!(\"Expected non-empty values, but got {:?}\", self.values);\n        }\n        self\n    }\n\n    pub fn value(&self) -> &str {\n        if let Some(value) = self.values.iter().find(|s| !s.is_empty()) {\n            value\n        } else {\n            self.response.dump_response();\n            panic!(\"Expected a value, but got {:?}\", self.values);\n        }\n    }\n\n    pub fn with_status(&self, status: StatusCode) -> &Self {\n        if self.prop.status != status {\n            self.response.dump_response();\n            panic!(\"Expected status {status}, but got {}\", self.prop.status);\n        }\n        self\n    }\n\n    pub fn with_description(&self, description: &str) -> &Self {\n        if self.prop.description.as_deref() != Some(description) {\n            self.response.dump_response();\n            panic!(\n                \"Expected description {description}, but got {:?}\",\n                self.prop.description\n            );\n        }\n        self\n    }\n    pub fn with_error(&self, error: &str) -> &Self {\n        if !self.prop.error.contains(&error.to_string()) {\n            self.response.dump_response();\n            panic!(\"Expected error {error}, but got {:?}\", self.prop.error);\n        }\n        self\n    }\n}\n\nimpl DavResponse {\n    pub fn into_propfind_response(mut self, prop_prefix: Option<&str>) -> DavMultiStatus {\n        if let Some(prop_prefix) = prop_prefix {\n            for (key, _) in self.xml.iter_mut() {\n                if let Some(suffix) = key.strip_prefix(prop_prefix) {\n                    *key = format!(\"D:multistatus.D:response{suffix}\");\n                }\n            }\n            self.xml.push((\n                \"D:multistatus.D:response.D:href\".to_string(),\n                \"\".to_string(),\n            ));\n        }\n\n        let mut result = DavMultiStatus {\n            response: self,\n            hrefs: AHashMap::new(),\n        };\n        let mut href = None;\n        let mut href_status = StatusCode::OK;\n        let mut props = Vec::new();\n        let mut prop = DavItem::default();\n\n        for (key, value) in &result.response.xml {\n            match key.as_str() {\n                \"D:multistatus.D:response.D:href\" => {\n                    if let Some(href) = href.take() {\n                        if !prop.is_empty() {\n                            props.push(std::mem::take(&mut prop));\n                        }\n                        result.hrefs.insert(\n                            href,\n                            DavProperties {\n                                status: href_status,\n                                props: std::mem::take(&mut props),\n                            },\n                        );\n                        href_status = StatusCode::OK;\n                    }\n                    href = Some(value.to_string());\n                }\n                \"D:multistatus.D:response.D:status\" => {\n                    href_status = value\n                        .split_ascii_whitespace()\n                        .nth(1)\n                        .unwrap_or_default()\n                        .parse()\n                        .unwrap();\n                }\n                \"D:multistatus.D:response.D:propstat.D:status\" => {\n                    prop.status = value\n                        .split_ascii_whitespace()\n                        .nth(1)\n                        .unwrap_or_default()\n                        .parse()\n                        .unwrap();\n                }\n                \"D:multistatus.D:response.D:propstat.D:responsedescription\" => {\n                    prop.description = Some(value.to_string());\n                }\n                _ => {\n                    if let Some(prop_name) =\n                        key.strip_prefix(\"D:multistatus.D:response.D:propstat.D:prop.\")\n                    {\n                        if prop.status != StatusCode::PROXY_AUTHENTICATION_REQUIRED {\n                            props.push(std::mem::take(&mut prop));\n                        }\n\n                        let (prop_name, prop_value) =\n                            if let Some((prop_name, prop_sub_name)) = prop_name.split_once('.') {\n                                if value.is_empty() {\n                                    (prop_name, prop_sub_name.to_string())\n                                } else {\n                                    (prop_name, format!(\"{}:{}\", prop_sub_name, value))\n                                }\n                            } else {\n                                (prop_name, value.to_string())\n                            };\n                        prop.values\n                            .entry(prop_name.to_string())\n                            .or_default()\n                            .push(prop_value);\n                    }\n                }\n            }\n        }\n\n        if let Some(href) = href.take() {\n            if !prop.is_empty() {\n                props.push(prop);\n            }\n            result.hrefs.insert(\n                href,\n                DavProperties {\n                    status: href_status,\n                    props,\n                },\n            );\n        }\n\n        result\n    }\n}\n\nimpl DummyWebDavClient {\n    pub async fn patch_and_check<T>(\n        &self,\n        path: &str,\n        properties: impl IntoIterator<Item = (T, &str)>,\n    ) where\n        T: AsRef<str> + Clone,\n    {\n        let mut expect_set = Vec::new();\n        let mut expect_remove = Vec::new();\n\n        for (key, value) in properties {\n            if !value.is_empty() {\n                expect_set.push((key, value));\n            } else {\n                expect_remove.push(key);\n            }\n        }\n\n        let response = self\n            .proppatch(\n                path,\n                expect_set.iter().cloned(),\n                expect_remove.iter().cloned(),\n                [],\n            )\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .into_propfind_response(None);\n        let patch_prop = response.properties(path);\n        for (key, _) in &expect_set {\n            patch_prop.get(key.as_ref()).with_status(StatusCode::OK);\n        }\n        for key in &expect_remove {\n            patch_prop\n                .get(key.as_ref())\n                .with_status(StatusCode::NO_CONTENT);\n        }\n\n        let response = self\n            .propfind(\n                path,\n                expect_set\n                    .iter()\n                    .map(|(k, _)| k)\n                    .chain(expect_remove.iter()),\n            )\n            .await;\n        let prop = response.properties(path);\n\n        for (key, value) in expect_set {\n            prop.get(key.as_ref())\n                .with_values([value])\n                .with_status(StatusCode::OK);\n        }\n\n        for key in expect_remove {\n            prop.get(key.as_ref()).with_status(StatusCode::NOT_FOUND);\n        }\n    }\n\n    pub async fn propfind<I, T>(&self, path: &str, properties: I) -> DavMultiStatus\n    where\n        I: IntoIterator<Item = T>,\n        T: AsRef<str>,\n    {\n        self.propfind_with_headers(path, properties, []).await\n    }\n\n    pub async fn propfind_with_headers<I, T>(\n        &self,\n        path: &str,\n        properties: I,\n        headers: impl IntoIterator<Item = (&'static str, &str)>,\n    ) -> DavMultiStatus\n    where\n        I: IntoIterator<Item = T>,\n        T: AsRef<str>,\n    {\n        let mut request = concat!(\n            \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\",\n            \"<D:propfind xmlns:D=\\\"DAV:\\\" xmlns:A=\\\"urn:ietf:params:xml:ns:caldav\\\" \",\n            \"xmlns:B=\\\"urn:ietf:params:xml:ns:carddav\\\" xmlns:C=\\\"http://calendarserver.org/ns/\\\">\",\n            \"<D:prop>\"\n        )\n        .to_string();\n\n        for property in properties {\n            request.push_str(&format!(\"<{}/>\", property.as_ref()));\n        }\n\n        request.push_str(\"</D:prop></D:propfind>\");\n\n        self.request_with_headers(\"PROPFIND\", path, headers, &request)\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n            .into_propfind_response(None)\n    }\n\n    pub async fn proppatch<T>(\n        &self,\n        path: &str,\n        set: impl IntoIterator<Item = (T, &str)>,\n        clear: impl IntoIterator<Item = T>,\n        headers: impl IntoIterator<Item = (&'static str, &str)>,\n    ) -> DavResponse\n    where\n        T: AsRef<str>,\n    {\n        let mut request = concat!(\n            \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\",\n            \"<D:propertyupdate xmlns:D=\\\"DAV:\\\" xmlns:A=\\\"urn:ietf:params:xml:ns:caldav\\\" \",\n            \"xmlns:B=\\\"urn:ietf:params:xml:ns:carddav\\\" xmlns:C=\\\"http://calendarserver.org/ns/\\\">\",\n            \"<D:remove><D:prop>\"\n        )\n        .to_string();\n\n        for property in clear {\n            request.push_str(&format!(\"<{}/>\", property.as_ref()));\n        }\n\n        request.push_str(\"</D:prop></D:remove><D:set><D:prop>\");\n\n        for (key, value) in set {\n            let key = key.as_ref();\n            request.push_str(&format!(\"<{key}>{value}</{key}>\"));\n        }\n\n        request.push_str(\"</D:prop></D:set></D:propertyupdate>\");\n\n        self.request_with_headers(\"PROPPATCH\", path, headers, &request)\n            .await\n    }\n}\n\nimpl DavItem {\n    pub fn is_empty(&self) -> bool {\n        self.values.is_empty()\n            && self.status == StatusCode::PROXY_AUTHENTICATION_REQUIRED\n            && self.error.is_empty()\n            && self.description.is_none()\n    }\n}\n\nimpl Default for DavItem {\n    fn default() -> Self {\n        DavItem {\n            status: StatusCode::PROXY_AUTHENTICATION_REQUIRED,\n            values: AHashMap::new(),\n            error: Vec::new(),\n            description: None,\n        }\n    }\n}\n\nconst EXPAND_REPORT_QUERY: &str = r#\"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<D:expand-property xmlns:D=\"DAV:\" \n xmlns:A=\"urn:ietf:params:xml:ns:caldav\" \n xmlns:B=\"urn:ietf:params:xml:ns:carddav\">\n   <A:property name=\"calendar-description\"/>\n   <B:property name=\"addressbook-description\"/>\n   <D:property name=\"current-user-principal\">\n      <D:property name=\"displayname\"/>\n   </D:property>\n   <D:property name=\"owner\">\n      <D:property name=\"displayname\"/>\n   </D:property>\n</D:expand-property>\"#;\n\npub const ALL_DAV_PROPERTIES: &[DavProperty] = &[\n    DavProperty::WebDav(WebDavProperty::CreationDate),\n    DavProperty::WebDav(WebDavProperty::DisplayName),\n    DavProperty::WebDav(WebDavProperty::GetContentLanguage),\n    DavProperty::WebDav(WebDavProperty::GetContentLength),\n    DavProperty::WebDav(WebDavProperty::GetContentType),\n    DavProperty::WebDav(WebDavProperty::GetETag),\n    DavProperty::WebDav(WebDavProperty::GetLastModified),\n    DavProperty::WebDav(WebDavProperty::ResourceType),\n    DavProperty::WebDav(WebDavProperty::LockDiscovery),\n    DavProperty::WebDav(WebDavProperty::SupportedLock),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrincipal),\n    DavProperty::WebDav(WebDavProperty::QuotaAvailableBytes),\n    DavProperty::WebDav(WebDavProperty::QuotaUsedBytes),\n    DavProperty::WebDav(WebDavProperty::SupportedReportSet),\n    DavProperty::WebDav(WebDavProperty::SyncToken),\n    DavProperty::WebDav(WebDavProperty::Owner),\n    DavProperty::WebDav(WebDavProperty::Group),\n    DavProperty::WebDav(WebDavProperty::SupportedPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::CurrentUserPrivilegeSet),\n    DavProperty::WebDav(WebDavProperty::Acl),\n    DavProperty::WebDav(WebDavProperty::AclRestrictions),\n    DavProperty::WebDav(WebDavProperty::InheritedAclSet),\n    DavProperty::WebDav(WebDavProperty::PrincipalCollectionSet),\n    DavProperty::WebDav(WebDavProperty::GetCTag),\n    DavProperty::CardDav(CardDavProperty::AddressbookDescription),\n    DavProperty::CardDav(CardDavProperty::SupportedAddressData),\n    DavProperty::CardDav(CardDavProperty::SupportedCollationSet),\n    DavProperty::CardDav(CardDavProperty::MaxResourceSize),\n    DavProperty::CalDav(CalDavProperty::CalendarDescription),\n    DavProperty::CalDav(CalDavProperty::CalendarTimezone),\n    DavProperty::CalDav(CalDavProperty::SupportedCalendarComponentSet),\n    DavProperty::CalDav(CalDavProperty::SupportedCalendarData),\n    DavProperty::CalDav(CalDavProperty::SupportedCollationSet),\n    DavProperty::CalDav(CalDavProperty::MaxResourceSize),\n    DavProperty::CalDav(CalDavProperty::MinDateTime),\n    DavProperty::CalDav(CalDavProperty::MaxDateTime),\n    DavProperty::CalDav(CalDavProperty::MaxInstances),\n    DavProperty::CalDav(CalDavProperty::MaxAttendeesPerInstance),\n    DavProperty::CalDav(CalDavProperty::TimezoneServiceSet),\n    DavProperty::CalDav(CalDavProperty::TimezoneId),\n    DavProperty::CalDav(CalDavProperty::ScheduleDefaultCalendarURL),\n    DavProperty::CalDav(CalDavProperty::ScheduleTag),\n    DavProperty::CalDav(CalDavProperty::ScheduleCalendarTransp),\n    DavProperty::Principal(PrincipalProperty::AlternateURISet),\n    DavProperty::Principal(PrincipalProperty::PrincipalURL),\n    DavProperty::Principal(PrincipalProperty::GroupMemberSet),\n    DavProperty::Principal(PrincipalProperty::GroupMembership),\n    DavProperty::Principal(PrincipalProperty::CalendarHomeSet),\n    DavProperty::Principal(PrincipalProperty::AddressbookHomeSet),\n    DavProperty::Principal(PrincipalProperty::PrincipalAddress),\n    DavProperty::Principal(PrincipalProperty::CalendarUserAddressSet),\n    DavProperty::Principal(PrincipalProperty::CalendarUserType),\n    DavProperty::Principal(PrincipalProperty::ScheduleInboxURL),\n    DavProperty::Principal(PrincipalProperty::ScheduleOutboxURL),\n];\n\nfn serialize_status_code<S>(status_code: &StatusCode, serializer: S) -> Result<S::Ok, S::Error>\nwhere\n    S: serde::Serializer,\n{\n    serializer.serialize_str(&status_code.to_string())\n}\n"
  },
  {
    "path": "tests/src/webdav/put_get.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse types::collection::Collection;\n\nuse super::WebDavTest;\nuse crate::webdav::*;\n\npub async fn test(test: &WebDavTest) {\n    println!(\"Running PUT/GET tests...\");\n    let client = test.client(\"john\");\n\n    // Simple PUT\n    let mut files = AHashMap::new();\n    for (path, ct, content) in [\n        (\"/dav/file/john/file1.txt\", \"text/plain\", TEST_FILE_1),\n        (\"/dav/file/john/file2.txt\", \"text/x-other\", TEST_FILE_2),\n        (\n            \"/dav/card/john/default/card1.vcf\",\n            \"text/vcard; charset=utf-8\",\n            TEST_VCARD_1,\n        ),\n        (\n            \"/dav/card/john/default/card2.vcf\",\n            \"text/vcard; charset=utf-8\",\n            TEST_VCARD_2,\n        ),\n        (\n            \"/dav/cal/john/default/event1.ics\",\n            \"text/calendar; charset=utf-8\",\n            TEST_ICAL_1,\n        ),\n        (\n            \"/dav/cal/john/default/event2.ics\",\n            \"text/calendar; charset=utf-8\",\n            TEST_ICAL_2,\n        ),\n    ] {\n        let content = content.replace(\"\\n\", \"\\r\\n\");\n        let etag = client\n            .request_with_headers(\"PUT\", path, [(\"content-type\", ct)], &content)\n            .await\n            .with_status(StatusCode::CREATED)\n            .etag()\n            .to_string();\n        files.insert(path, (content, ct, etag));\n    }\n\n    // Test GET\n    for (path, (content, ct, etag)) in &files {\n        client\n            .request(\"GET\", path, \"\")\n            .await\n            .with_status(StatusCode::OK)\n            .with_header(\"etag\", etag)\n            .with_header(\"content-type\", ct)\n            .with_body(content);\n    }\n\n    // PUT under a non-existing parent should fail\n    for (path, contents) in [\n        (\"/dav/file/john/foo/file1.txt\", TEST_FILE_1),\n        (\"/dav/card/john/foo/card1.vcf\", TEST_VCARD_1),\n        (\"/dav/cal/john/foo/event1.ics\", TEST_ICAL_1),\n    ] {\n        client\n            .request(\"PUT\", path, contents)\n            .await\n            .with_status(StatusCode::CONFLICT);\n    }\n\n    // PUT under resources should fail\n    for (path, contents) in [\n        (\"/dav/file/john/file1.txt/other-file.txt\", TEST_FILE_1),\n        (\n            \"/dav/card/john/default/card1.vcf/other-file.vcf\",\n            TEST_VCARD_1,\n        ),\n        (\n            \"/dav/cal/john/default/event1.ics/other-file.ical\",\n            TEST_ICAL_1,\n        ),\n    ] {\n        client\n            .request(\"PUT\", path, contents)\n            .await\n            .with_status(StatusCode::METHOD_NOT_ALLOWED);\n    }\n\n    // PUT a non-vCard/iCalendar file should fail\n    for (path, ct, content, precondition) in [\n        (\n            \"/dav/card/john/card3.vcf\",\n            \"text/vcard; charset=utf-8\",\n            TEST_FILE_1,\n            \"B:supported-address-data\",\n        ),\n        (\n            \"/dav/cal/john/event3.ics\",\n            \"text/calendar; charset=utf-8\",\n            TEST_FILE_2,\n            \"A:supported-calendar-data\",\n        ),\n    ] {\n        client\n            .request_with_headers(\"PUT\", path, [(\"content-type\", ct)], content)\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED)\n            .with_failed_precondition(precondition, \"\");\n    }\n\n    // Exceeding the configured file limits should fail\n    let conf = &test.server.core.groupware;\n    for (path, contents, max_size, expect) in [\n        (\n            \"/dav/file/john/chunky-file1.txt\",\n            TEST_FILE_1,\n            conf.max_file_size,\n            None,\n        ),\n        (\n            \"/dav/card/john/chunky-card1.vcf\",\n            TEST_VCARD_1,\n            conf.max_vcard_size,\n            Some(\"B:max-resource-size\"),\n        ),\n        (\n            \"/dav/cal/john/chunky-event1.ics\",\n            TEST_ICAL_1,\n            conf.max_ical_size,\n            Some(\"A:max-resource-size\"),\n        ),\n    ] {\n        let mut chunky_contents = String::with_capacity(max_size + contents.len());\n        while chunky_contents.len() < max_size {\n            chunky_contents.push_str(contents);\n        }\n        let response = client\n            .request(\"PUT\", path, chunky_contents)\n            .await\n            .with_status(\n                expect\n                    .map(|_| StatusCode::PRECONDITION_FAILED)\n                    .unwrap_or(StatusCode::PAYLOAD_TOO_LARGE),\n            );\n        if let Some(expect) = expect {\n            response.with_failed_precondition(expect, &max_size.to_string());\n        }\n    }\n\n    // PUT requests cannot exceed quota\n    let mike_noquota = test.client(\"mike\");\n    for resource_type in [\n        DavResourceName::File,\n        DavResourceName::Card,\n        DavResourceName::Cal,\n    ] {\n        let path = format!(\"{}/mike/quota-test/\", resource_type.base_path());\n        mike_noquota\n            .mkcol(\"MKCOL\", &path, [], [])\n            .await\n            .with_status(StatusCode::CREATED);\n        let mut num_success = 0;\n        let mut did_fail = false;\n\n        for i in 0..100 {\n            let content = resource_type.generate();\n            let available = mike_noquota.available_quota(&path).await;\n\n            let response = mike_noquota\n                .request_with_headers(\"PUT\", &format!(\"{path}file{i}\"), [], &content)\n                .await;\n            if available > content.len() as u64 {\n                num_success += 1;\n                response.with_status(StatusCode::CREATED);\n            } else {\n                response\n                    .with_status(StatusCode::PRECONDITION_FAILED)\n                    .with_failed_precondition(\"D:quota-not-exceeded\", \"\");\n                did_fail = true;\n                break;\n            }\n        }\n        if !did_fail {\n            panic!(\"Quota test failed: {} files created\", num_success);\n        }\n        if num_success == 0 {\n            panic!(\"Quota test failed: no files created\");\n        }\n\n        mike_noquota\n            .request(\"DELETE\", &path, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n    }\n\n    // PUT precondition enforcement\n    let modseq = [\n        test.resources(\"john\", Collection::FileNode)\n            .await\n            .highest_change_id,\n        test.resources(\"john\", Collection::Calendar)\n            .await\n            .highest_change_id,\n        test.resources(\"john\", Collection::AddressBook)\n            .await\n            .highest_change_id,\n    ];\n    for (path, ct, content) in [\n        (\"/dav/file/john/file1.txt\", \"text/plain\", TEST_FILE_1),\n        (\n            \"/dav/card/john/default/card1.vcf\",\n            \"text/vcard; charset=utf-8\",\n            TEST_VCARD_1,\n        ),\n        (\n            \"/dav/cal/john/default/event1.ics\",\n            \"text/calendar; charset=utf-8\",\n            TEST_ICAL_1,\n        ),\n    ] {\n        let content = content.replace(\"\\n\", \"\\r\\n\");\n        client\n            .request_with_headers(\n                \"PUT\",\n                path,\n                [(\"content-type\", ct), (\"if-none-match\", \"*\")],\n                &content,\n            )\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED);\n\n        client\n            .request_with_headers(\n                \"PUT\",\n                path,\n                [(\"content-type\", ct), (\"overwrite\", \"F\")],\n                &content,\n            )\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED);\n\n        client\n            .request_with_headers(\n                \"PUT\",\n                path,\n                [(\"content-type\", ct), (\"if\", \"([\\\"3827\\\"])\")],\n                &content,\n            )\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED);\n\n        client\n            .request_with_headers(\n                \"PUT\",\n                path,\n                [\n                    (\"content-type\", ct),\n                    (\"if\", \"([\\\"3827\\\"])\"),\n                    (\"prefer\", \"return=representation\"),\n                ],\n                &content,\n            )\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED)\n            .with_header(\"preference-applied\", \"return=representation\")\n            .with_body(&content);\n    }\n    assert_eq!(\n        [\n            test.resources(\"john\", Collection::FileNode)\n                .await\n                .highest_change_id,\n            test.resources(\"john\", Collection::Calendar)\n                .await\n                .highest_change_id,\n            test.resources(\"john\", Collection::AddressBook)\n                .await\n                .highest_change_id,\n        ],\n        modseq\n    );\n\n    // Update files using etags\n    for (path, (content, ct, etag)) in &mut files {\n        let condition = format!(\"([{}])\", etag);\n        *content = content.replace(\"X-TEST:SEQ1\", \"X-TEST:SEQ2\");\n        *etag = client\n            .request_with_headers(\n                \"PUT\",\n                path,\n                [(\"content-type\", &**ct), (\"if\", condition.as_str())],\n                content.as_str(),\n            )\n            .await\n            .with_status(StatusCode::NO_CONTENT)\n            .etag()\n            .to_string();\n    }\n\n    // Test GET\n    for (path, (content, ct, etag)) in &files {\n        client\n            .request(\"GET\", path, \"\")\n            .await\n            .with_status(StatusCode::OK)\n            .with_header(\"etag\", etag)\n            .with_header(\"content-type\", ct)\n            .with_body(content);\n    }\n\n    // PUT requests require unique UIDs\n    for (path, ct, content, precond_key, precond_value) in [\n        (\n            \"/dav/card/john/default/card5.vcf\",\n            \"text/vcard; charset=utf-8\",\n            TEST_VCARD_1,\n            \"B:no-uid-conflict.D:href\",\n            \"/dav/card/john/default/card1.vcf\",\n        ),\n        (\n            \"/dav/cal/john/default/event5.ics\",\n            \"text/calendar; charset=utf-8\",\n            TEST_ICAL_1,\n            \"A:no-uid-conflict.D:href\",\n            \"/dav/cal/john/default/event1.ics\",\n        ),\n    ] {\n        client\n            .request_with_headers(\n                \"PUT\",\n                path,\n                [(\"content-type\", ct), (\"if-none-match\", \"*\")],\n                content,\n            )\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED)\n            .with_failed_precondition(precond_key, precond_value);\n    }\n\n    // iCal containing different component types should fail\n    client\n        .request_with_headers(\n            \"PUT\",\n            \"/dav/cal/john/default/invalid.ics\",\n            [\n                (\"content-type\", \"text/calendar; charset=utf-8\"),\n                (\"if-none-match\", \"*\"),\n            ],\n            r#\"BEGIN:VCALENDAR\nVERSION:2.0\nBEGIN:VEVENT\nUID:1234567890\nSUMMARY:Test Event\nDTSTART;TZID=Europe/London:20231001T120000\nDTEND;TZID=Europe/London:20231001T130000\nEND:VEVENT\nBEGIN:VTODO\nUID:1234567890\nSUMMARY:Test Task\nDTSTART;TZID=Europe/London:20231001T120000\nDTEND;TZID=Europe/London:20231001T130000\nEND:VTODO\nEND:VCALENDAR\n\"#,\n        )\n        .await\n        .with_status(StatusCode::PRECONDITION_FAILED)\n        .with_failed_precondition(\"A:valid-calendar-object-resource\", \"\");\n\n    // iCal referencing more than one UID should fail\n    client\n        .request_with_headers(\n            \"PUT\",\n            \"/dav/cal/john/default/invalid.ics\",\n            [\n                (\"content-type\", \"text/calendar; charset=utf-8\"),\n                (\"if-none-match\", \"*\"),\n            ],\n            r#\"BEGIN:VCALENDAR\nVERSION:2.0\nBEGIN:VEVENT\nUID:1234567890\nSUMMARY:Test Event 1\nDTSTART;TZID=Europe/London:20231001T120000\nDTEND;TZID=Europe/London:20231001T130000\nEND:VEVENT\nBEGIN:VEVENT\nUID:1234567891\nSUMMARY:Test Event 2\nDTSTART;TZID=Europe/London:20231001T120000\nDTEND;TZID=Europe/London:20231001T130000\nEND:VEVENT\nEND:VCALENDAR\n\"#,\n        )\n        .await\n        .with_status(StatusCode::PRECONDITION_FAILED)\n        .with_failed_precondition(\"A:valid-calendar-object-resource\", \"\");\n\n    // Deleting unknown/invalid destinations should fail\n    for (path, expect) in [\n        (\"/dav/file/john/unknown.txt\", StatusCode::NOT_FOUND),\n        (\"/dav/card/john/default/unknown.txt\", StatusCode::NOT_FOUND),\n        (\"/dav/cal/john/default/unknown.txt\", StatusCode::NOT_FOUND),\n        (\"/dav/file/john\", StatusCode::FORBIDDEN),\n        (\"/dav/cal/john\", StatusCode::FORBIDDEN),\n        (\"/dav/card/john\", StatusCode::FORBIDDEN),\n        (\"/dav/pal/john\", StatusCode::METHOD_NOT_ALLOWED),\n        (\"/dav/file\", StatusCode::FORBIDDEN),\n        (\"/dav/cal\", StatusCode::FORBIDDEN),\n        (\"/dav/card\", StatusCode::FORBIDDEN),\n        (\"/dav/pal\", StatusCode::METHOD_NOT_ALLOWED),\n    ] {\n        client.request(\"DELETE\", path, \"\").await.with_status(expect);\n    }\n\n    // Delete files\n    for (path, (_, _, etag)) in &files {\n        client\n            .request_with_headers(\"DELETE\", path, [(\"if\", \"([\\\"3827\\\"])\")], \"\")\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED);\n\n        let condition = format!(\"([{}])\", etag);\n        client\n            .request_with_headers(\"DELETE\", path, [(\"if\", condition.as_str())], \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n\n        client\n            .request(\"DELETE\", path, \"\")\n            .await\n            .with_status(StatusCode::NOT_FOUND);\n    }\n\n    client.delete_default_containers().await;\n    mike_noquota.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n"
  },
  {
    "path": "tests/src/webdav/sync.rs",
    "content": "/*\n * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>\n *\n * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL\n */\n\nuse super::{DavResponse, DummyWebDavClient, WebDavTest};\nuse crate::webdav::GenerateTestDavResource;\nuse ahash::AHashSet;\nuse dav_proto::Depth;\nuse groupware::DavResourceName;\nuse hyper::StatusCode;\n\npub async fn test(test: &WebDavTest) {\n    let client = test.client(\"john\");\n\n    for resource_type in [\n        DavResourceName::File,\n        DavResourceName::Cal,\n        DavResourceName::Card,\n    ] {\n        println!(\n            \"Running REPORT sync-collection tests ({})...\",\n            resource_type.base_path()\n        );\n        let user_base_path = format!(\"{}/john/\", resource_type.base_path());\n\n        // Test 1: Initial sync\n        let response = client\n            .sync_collection(&user_base_path, \"\", Depth::Infinity, None, [\"D:getetag\"])\n            .await;\n        assert_eq!(\n            response.hrefs().len(),\n            if resource_type == DavResourceName::File {\n                1\n            } else {\n                2\n            },\n            \"{:?}\",\n            response.hrefs()\n        );\n        let sync_token_1 = response.sync_token().to_string();\n\n        // Test 2: No changes since last sync\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token_1,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        assert_eq!(response.hrefs(), Vec::<String>::new());\n\n        // Test 3: Create a collection and make sure it is synced\n        let new_collection = format!(\"{}new-collection/\", user_base_path);\n        client\n            .mkcol(\"MKCOL\", &new_collection, [], [])\n            .await\n            .with_status(StatusCode::CREATED);\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token_1,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        assert_eq!(response.hrefs(), vec![new_collection.clone()]);\n        let sync_token_2 = response.sync_token().to_string();\n\n        // Test 4: Create a file and make sure it is synced\n        let new_file = format!(\"{new_collection}new-file\");\n        let contents = resource_type.generate();\n        client\n            .request(\"PUT\", &new_file, &contents)\n            .await\n            .with_status(StatusCode::CREATED);\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token_1,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        assert_eq!(\n            response.hrefs(),\n            vec![new_collection.clone(), new_file.clone()]\n        );\n        let sync_token_3 = response.sync_token().to_string();\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token_2,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        assert_eq!(response.hrefs(), vec![new_file.clone()]);\n\n        // Test 5: sync-token with Depth 1\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token_1,\n                Depth::One,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        assert_eq!(response.hrefs(), vec![new_collection.clone()]);\n\n        // Test 6: sync-token with Depth 0\n        let response = client\n            .sync_collection(\n                &new_collection,\n                &sync_token_1,\n                Depth::Zero,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        assert_eq!(response.hrefs(), vec![new_collection.clone()]);\n\n        // Test 7: Outdated sync-token in If header should fail\n        let new_file2 = format!(\"{new_collection}new-file2\");\n        let contents = resource_type.generate();\n        let condition = format!(\"(<{sync_token_2}>)\");\n        client\n            .request_with_headers(\n                \"PUT\",\n                &new_file2,\n                [(\"if\", condition.as_str())],\n                contents.as_str(),\n            )\n            .await\n            .with_status(StatusCode::PRECONDITION_FAILED)\n            .with_empty_body();\n\n        // Test 8: Correct sync-token in If header should work\n        let condition = format!(\"(<{sync_token_3}>)\");\n        client\n            .request_with_headers(\n                \"PUT\",\n                &new_file2,\n                [(\"if\", condition.as_str())],\n                contents.as_str(),\n            )\n            .await\n            .with_status(StatusCode::CREATED)\n            .with_empty_body();\n\n        // Test 9: Limit\n        let mut sync_token = client\n            .sync_collection(\n                &new_collection,\n                &sync_token_3,\n                Depth::Zero,\n                None,\n                [\"D:getetag\"],\n            )\n            .await\n            .sync_token()\n            .to_string();\n        let (folder_name, files) = client\n            .create_hierarchy(user_base_path.trim_end_matches('/'), 1, 0, 10)\n            .await;\n        let mut expected_changes = files\n            .iter()\n            .map(|x| x.0.as_str())\n            .chain([folder_name.as_str()])\n            .collect::<AHashSet<_>>();\n        for _ in 0..10 {\n            let response = client\n                .sync_collection(\n                    &user_base_path,\n                    &sync_token,\n                    Depth::Infinity,\n                    2.into(),\n                    [\"D:getetag\"],\n                )\n                .await;\n            sync_token = response.sync_token().to_string();\n            let hrefs = response.hrefs();\n            if hrefs.is_empty() {\n                break;\n            }\n            let mut has_user_base_path = false;\n            let mut item_count = 0;\n            for href in hrefs {\n                if href == user_base_path {\n                    has_user_base_path = true;\n                } else if expected_changes.remove(href) {\n                    item_count += 1;\n                } else {\n                    panic!(\"Unexpected href: {href}\");\n                }\n            }\n            if has_user_base_path {\n                assert_eq!(item_count, 2);\n                response\n                    .with_value(\n                        \"D:multistatus.D:response.D:status\",\n                        \"HTTP/1.1 507 Insufficient Storage\",\n                    )\n                    .with_value(\n                        \"D:multistatus.D:response.D:error.D:number-of-matches-within-limits\",\n                        \"\",\n                    )\n                    .with_value(\n                        \"D:multistatus.D:response.D:responsedescription\",\n                        \"The number of matches exceeds the limit of 2\",\n                    );\n            } else {\n                assert!(item_count <= 2);\n                break;\n            }\n        }\n        assert!(expected_changes.is_empty(), \"{:?}\", expected_changes);\n\n        // Test 10: Expect changes after deletion\n        client\n            .request(\"DELETE\", &new_file, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        sync_token = response.sync_token().to_string();\n        response\n            .with_href_count(1)\n            .with_value(\"D:multistatus.D:response.D:href\", &new_file)\n            .with_value(\n                \"D:multistatus.D:response.D:status\",\n                \"HTTP/1.1 404 Not Found\",\n            );\n        client\n            .request(\"DELETE\", &new_collection, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        let response = client\n            .sync_collection(\n                &user_base_path,\n                &sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await;\n        sync_token = response.sync_token().to_string();\n        response\n            .with_href_count(1)\n            .with_value(\"D:multistatus.D:response.D:href\", &new_collection)\n            .with_value(\n                \"D:multistatus.D:response.D:status\",\n                \"HTTP/1.1 404 Not Found\",\n            );\n        client\n            .request(\"DELETE\", &folder_name, \"\")\n            .await\n            .with_status(StatusCode::NO_CONTENT);\n        client\n            .sync_collection(\n                &user_base_path,\n                &sync_token,\n                Depth::Infinity,\n                None,\n                [\"D:getetag\"],\n            )\n            .await\n            .with_href_count(1)\n            .with_value(\"D:multistatus.D:response.D:href\", &folder_name)\n            .with_value(\n                \"D:multistatus.D:response.D:status\",\n                \"HTTP/1.1 404 Not Found\",\n            );\n    }\n\n    client.delete_default_containers().await;\n    test.assert_is_empty().await;\n}\n\nimpl DummyWebDavClient {\n    pub async fn sync_collection(\n        &self,\n        path: &str,\n        sync_token: &str,\n        depth: Depth,\n        limit: Option<usize>,\n        properties: impl IntoIterator<Item = &str>,\n    ) -> DavResponse {\n        let mut request = concat!(\n            \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\",\n            \"<D:sync-collection xmlns:D=\\\"DAV:\\\" xmlns:A=\\\"urn:ietf:params:xml:ns:caldav\\\" xmlns:B=\\\"urn:ietf:params:xml:ns:carddav\\\">\",\n            \"<D:prop>\"\n        )\n        .to_string();\n\n        for property in properties {\n            request.push_str(&format!(\"<{property}/>\"));\n        }\n\n        request.push_str(\"</D:prop><D:sync-token>\");\n        request.push_str(sync_token);\n        request.push_str(\"</D:sync-token><D:sync-level>\");\n        request.push_str(match depth {\n            Depth::One => \"1\",\n            Depth::Infinity => \"infinite\",\n            _ => \"0\",\n        });\n        request.push_str(\"</D:sync-level>\");\n\n        if let Some(limit) = limit {\n            request.push_str(\"<D:limit><D:nresults>\");\n            request.push_str(&limit.to_string());\n            request.push_str(\"</D:nresults></D:limit>\");\n        }\n\n        request.push_str(\"</D:sync-collection>\");\n\n        self.request(\"REPORT\", path, &request)\n            .await\n            .with_status(StatusCode::MULTI_STATUS)\n    }\n}\n"
  }
]